2024-02-19T18:37:26.335 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-19T18:37:26.336 DEBUG:teuthology.run:Teuthology command: teuthology --name pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646 --owner scheduled_pdonnell@teuthology --verbose --description fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 0a 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} -- /home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646/orig.config.yaml 2024-02-19T18:37:26.389 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T18:37:26.482 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646 branch: wip-batrick-testing-20240215.160715 description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 0a 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: pdonnell@redhat.com first_in_suite: false job_id: '7566646' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.0 ' name: pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-batrick-testing-20240215.160715 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: f78a58c0ffd401d1493058a1022c35f011d65275 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f78a58c0ffd401d1493058a1022c35f011d65275 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: i64440 sha1: 1ce7ea8877c709fa862eac608550374bd96e166e owner: scheduled_pdonnell@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 2975 sha1: f78a58c0ffd401d1493058a1022c35f011d65275 sleep_before_teardown: 0 suite: fs:upgrade:mds_upgrade_sequence suite_branch: i64440 suite_path: /home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa suite_relpath: qa suite_repo: https://github.com/batrick/ceph.git suite_sha1: 1ce7ea8877c709fa862eac608550374bd96e166e targets: smithi088.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPtDDHr2o50eWVnV9CPqYYq7STB65lDrHSrVT5wdXvjchvcjwFFmGArhdR3fCPD8QlRNAzqhFiiVenj7DG938Fw= smithi132.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC/NtErFbvag6kQx7tNCcOaAIuzoCX3oIoOKi9mTgnQwZvPQiqBWoHr+67OR+URfwwUfJ8bM4hamLqcff/iNC9c= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.0 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.0 roleless: true - print: '**** done end installing v18.2.0 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - full_sequential_finally: - sleep: duration: 3600 - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-19_18:28:45 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-02-19T18:37:26.482 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa; will attempt to use it 2024-02-19T18:37:26.483 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa/tasks 2024-02-19T18:37:26.483 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-19T18:37:26.485 INFO:teuthology.task.internal:Checking packages... 2024-02-19T18:37:26.515 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f78a58c0ffd401d1493058a1022c35f011d65275' 2024-02-19T18:37:26.516 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-19T18:37:26.516 INFO:teuthology.packaging:ref: None 2024-02-19T18:37:26.516 INFO:teuthology.packaging:tag: None 2024-02-19T18:37:26.516 INFO:teuthology.packaging:branch: wip-batrick-testing-20240215.160715 2024-02-19T18:37:26.516 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:37:26.516 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-batrick-testing-20240215.160715 2024-02-19T18:37:26.658 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1450.gf78a58c0 2024-02-19T18:37:26.659 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-19T18:37:26.666 INFO:teuthology.task.internal:no buildpackages task found 2024-02-19T18:37:26.666 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-19T18:37:26.709 INFO:teuthology.task.internal:Saving configuration 2024-02-19T18:37:26.728 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-19T18:37:26.733 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-19T18:37:26.761 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-19 18:29:59.661105', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPtDDHr2o50eWVnV9CPqYYq7STB65lDrHSrVT5wdXvjchvcjwFFmGArhdR3fCPD8QlRNAzqhFiiVenj7DG938Fw='} 2024-02-19T18:37:26.782 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-19 18:29:59.662526', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC/NtErFbvag6kQx7tNCcOaAIuzoCX3oIoOKi9mTgnQwZvPQiqBWoHr+67OR+URfwwUfJ8bM4hamLqcff/iNC9c='} 2024-02-19T18:37:26.782 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-19T18:37:26.788 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-02-19T18:37:26.788 INFO:teuthology.task.internal:roles: ubuntu@smithi132.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-02-19T18:37:26.788 INFO:teuthology.run_tasks:Running task console_log... 2024-02-19T18:37:26.854 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff57b5f50d0>, signals=[15]) 2024-02-19T18:37:26.854 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-19T18:37:26.860 INFO:teuthology.task.internal:Opening connections... 2024-02-19T18:37:26.861 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T18:37:26.862 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:37:26.955 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi132.front.sepia.ceph.com 2024-02-19T18:37:26.955 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:37:27.043 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-19T18:37:27.049 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2024-02-19T18:37:27.067 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2024-02-19T18:37:27.067 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-02-19T18:37:27.124 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2024-02-19T18:37:27.124 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9" 2024-02-19T18:37:27.124 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2024-02-19T18:37:27.124 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2024-02-19T18:37:27.124 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9" 2024-02-19T18:37:27.124 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9" 2024-02-19T18:37:27.124 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-19T18:37:27.124 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2024-02-19T18:37:27.124 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon" 2024-02-19T18:37:27.125 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-19T18:37:27.125 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2024-02-19T18:37:27.125 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-19T18:37:27.125 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-19T18:37:27.125 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-19T18:37:27.125 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-02-19T18:37:27.151 DEBUG:teuthology.orchestra.run.smithi132:> uname -m 2024-02-19T18:37:27.168 INFO:teuthology.orchestra.run.smithi132.stdout:x86_64 2024-02-19T18:37:27.168 DEBUG:teuthology.orchestra.run.smithi132:> cat /etc/os-release 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:NAME="CentOS Stream" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION="9" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:ID="centos" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:ID_LIKE="rhel fedora" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:VERSION_ID="9" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:PLATFORM_ID="platform:el9" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:ANSI_COLOR="0;31" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:LOGO="fedora-logo-icon" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:HOME_URL="https://centos.org/" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-19T18:37:27.226 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-19T18:37:27.227 INFO:teuthology.orchestra.run.smithi132.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-19T18:37:27.227 INFO:teuthology.lock.ops:Updating smithi132.front.sepia.ceph.com on lock server 2024-02-19T18:37:27.250 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-19T18:37:27.259 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-19T18:37:27.264 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-19T18:37:27.264 DEBUG:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2024-02-19T18:37:27.266 DEBUG:teuthology.orchestra.run.smithi132:> test '!' -e /home/ubuntu/cephtest 2024-02-19T18:37:27.283 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-19T18:37:27.288 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-19T18:37:27.288 DEBUG:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2024-02-19T18:37:27.322 DEBUG:teuthology.orchestra.run.smithi132:> test -z $(ls -A /var/lib/ceph) 2024-02-19T18:37:27.614 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-19T18:37:27.667 INFO:teuthology.run_tasks:Running task kernel... 2024-02-19T18:37:27.682 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-19T18:37:27.682 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-19T18:37:27.682 DEBUG:teuthology.orchestra.run.smithi088:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-19T18:37:27.683 DEBUG:teuthology.orchestra.run.smithi132:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-19T18:37:27.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T18:37:27.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T18:37:27.699 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-02-19T18:37:27.700 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-02-19T18:37:27.756 INFO:teuthology.orchestra.run.smithi132.stdout:5.14.0-361.el9.x86_64 2024-02-19T18:37:27.756 INFO:teuthology.task.kernel:Running kernel on smithi132: 5.14.0-361.el9.x86_64 2024-02-19T18:37:27.756 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum install -y kernel 2024-02-19T18:37:27.757 INFO:teuthology.orchestra.run.smithi088.stdout:5.14.0-361.el9.x86_64 2024-02-19T18:37:27.757 INFO:teuthology.task.kernel:Running kernel on smithi088: 5.14.0-361.el9.x86_64 2024-02-19T18:37:27.758 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum install -y kernel 2024-02-19T18:37:29.776 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - AppStream 16 MB/s | 6.7 MB 00:00 2024-02-19T18:37:29.837 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-02-19T18:37:31.810 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - BaseOS 2.2 MB/s | 2.0 MB 00:00 2024-02-19T18:37:31.845 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-9 - BaseOS 2.2 MB/s | 2.0 MB 00:00 2024-02-19T18:37:39.559 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.0 MB 00:07 2024-02-19T18:37:40.715 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 954 kB/s | 8.0 MB 00:08 2024-02-19T18:37:44.567 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 6.9 MB/s | 19 MB 00:02 2024-02-19T18:37:44.593 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 4.9 MB/s | 19 MB 00:03 2024-02-19T18:37:50.943 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.1 MB 00:02 2024-02-19T18:37:52.443 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 15 kB 00:00 2024-02-19T18:37:57.125 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 695 kB/s | 6.1 MB 00:08 2024-02-19T18:37:58.517 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 61 kB/s | 15 kB 00:00 2024-02-19T18:38:03.411 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 4.3 MB/s | 20 MB 00:04 2024-02-19T18:38:03.495 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 1.9 MB/s | 20 MB 00:10 2024-02-19T18:38:08.351 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-02-19T18:38:08.359 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-02-19T18:38:10.566 INFO:teuthology.orchestra.run.smithi088.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-19T18:38:10.573 INFO:teuthology.orchestra.run.smithi088.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-19T18:38:10.619 INFO:teuthology.orchestra.run.smithi132.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-19T18:38:10.619 INFO:teuthology.orchestra.run.smithi132.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-19T18:38:10.646 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:38:10.656 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:38:10.657 INFO:teuthology.orchestra.run.smithi088.stdout:Install 4 Packages 2024-02-19T18:38:10.658 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:38:10.658 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 100 M 2024-02-19T18:38:10.658 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 123 M 2024-02-19T18:38:10.658 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:38:10.706 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:38:10.712 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout: kernel x86_64 5.14.0-421.el9 CentOS-BaseOS 6.1 M 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-core x86_64 5.14.0-421.el9 CentOS-BaseOS 21 M 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules x86_64 5.14.0-421.el9 CentOS-BaseOS 39 M 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules-core x86_64 5.14.0-421.el9 CentOS-BaseOS 33 M 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout:Install 4 Packages 2024-02-19T18:38:10.713 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:38:10.714 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 100 M 2024-02-19T18:38:10.714 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 123 M 2024-02-19T18:38:10.714 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:38:11.502 INFO:teuthology.orchestra.run.smithi088.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 7.2 MB/s | 6.1 MB 00:00 2024-02-19T18:38:11.752 INFO:teuthology.orchestra.run.smithi132.stdout:(1/4): kernel-5.14.0-421.el9.x86_64.rpm 5.8 MB/s | 6.1 MB 00:01 2024-02-19T18:38:12.627 INFO:teuthology.orchestra.run.smithi088.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 11 MB/s | 21 MB 00:01 2024-02-19T18:38:12.736 INFO:teuthology.orchestra.run.smithi132.stdout:(2/4): kernel-core-5.14.0-421.el9.x86_64.rpm 10 MB/s | 21 MB 00:02 2024-02-19T18:38:13.760 INFO:teuthology.orchestra.run.smithi088.stdout:(3/4): kernel-modules-core-5.14.0-421.el9.x86_6 15 MB/s | 33 MB 00:02 2024-02-19T18:38:13.811 INFO:teuthology.orchestra.run.smithi132.stdout:(3/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 13 MB/s | 39 MB 00:03 2024-02-19T18:38:14.135 INFO:teuthology.orchestra.run.smithi088.stdout:(4/4): kernel-modules-5.14.0-421.el9.x86_64.rpm 11 MB/s | 39 MB 00:03 2024-02-19T18:38:14.139 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:38:14.139 INFO:teuthology.orchestra.run.smithi088.stdout:Total 29 MB/s | 100 MB 00:03 2024-02-19T18:38:14.139 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:38:14.236 INFO:teuthology.orchestra.run.smithi132.stdout:(4/4): kernel-modules-core-5.14.0-421.el9.x86_6 13 MB/s | 33 MB 00:02 2024-02-19T18:38:14.240 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:38:14.240 INFO:teuthology.orchestra.run.smithi132.stdout:Total 28 MB/s | 100 MB 00:03 2024-02-19T18:38:14.240 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:38:14.511 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:38:14.511 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:38:14.624 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:38:14.624 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:38:15.550 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:38:15.551 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:38:15.703 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:38:15.704 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:38:17.959 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:38:18.109 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:38:18.631 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-19T18:38:18.760 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-19T18:38:18.920 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-modules-core-5.14.0-421.el9.x86_64 1/4 2024-02-19T18:38:19.054 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-19T18:38:21.535 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-19T18:38:21.721 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-19T18:38:21.912 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-19T18:38:22.110 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-19T18:38:26.403 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-19T18:38:26.591 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:38:26.832 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-19T18:38:27.022 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : kernel-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:38:31.304 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:38:31.736 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:39:00.545 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:39:00.545 INFO:teuthology.orchestra.run.smithi088.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-19T18:39:00.545 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:39:00.615 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:39:00.730 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-core-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:39:00.730 INFO:teuthology.orchestra.run.smithi132.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-19T18:39:00.730 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:39:00.801 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-modules-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:39:01.108 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:39:01.108 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-19T18:39:01.108 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-19T18:39:01.109 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-19T18:39:01.216 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: kernel-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:39:01.217 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-5.14.0-421.el9.x86_64 1/4 2024-02-19T18:39:01.217 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-core-5.14.0-421.el9.x86_64 2/4 2024-02-19T18:39:01.217 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-modules-5.14.0-421.el9.x86_64 3/4 2024-02-19T18:39:01.551 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:39:01.551 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:39:01.551 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:39:01.551 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-19T18:39:01.551 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-19T18:39:01.551 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-19T18:39:01.551 INFO:teuthology.orchestra.run.smithi088.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-19T18:39:01.551 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:39:01.552 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:39:01.619 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : kernel-modules-core-5.14.0-421.el9.x86_64 4/4 2024-02-19T18:39:01.620 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:39:01.620 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:39:01.620 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-5.14.0-421.el9.x86_64 2024-02-19T18:39:01.620 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-core-5.14.0-421.el9.x86_64 2024-02-19T18:39:01.620 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules-5.14.0-421.el9.x86_64 2024-02-19T18:39:01.620 INFO:teuthology.orchestra.run.smithi132.stdout: kernel-modules-core-5.14.0-421.el9.x86_64 2024-02-19T18:39:01.620 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:39:01.620 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:39:02.097 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q kernel | sort -rV | head -n 1 2024-02-19T18:39:02.169 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q kernel | sort -rV | head -n 1 2024-02-19T18:39:02.245 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-19T18:39:02.245 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-19T18:39:02.245 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-19T18:39:02.245 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-19T18:39:02.245 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-19T18:39:02.245 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-02-19T18:39:02.246 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi088.front.sepia.ceph.com, path=None, version=distro) 2024-02-19T18:39:02.246 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q kernel | sort -rV | head -n 1 2024-02-19T18:39:02.309 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-19T18:39:02.309 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-19T18:39:02.309 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-421.el9.x86_64 2024-02-19T18:39:02.309 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-19T18:39:02.309 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-19T18:39:02.310 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-02-19T18:39:02.310 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi132.front.sepia.ceph.com, path=None, version=distro) 2024-02-19T18:39:02.310 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q kernel | sort -rV | head -n 1 2024-02-19T18:39:02.353 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-19T18:39:02.353 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-19T18:39:02.354 DEBUG:teuthology.orchestra.run.smithi088:> sudo rpm -qi grub2-tools 2024-02-19T18:39:02.387 INFO:teuthology.orchestra.run.smithi088.stdout:Name : grub2-tools 2024-02-19T18:39:02.387 INFO:teuthology.orchestra.run.smithi088.stdout:Epoch : 1 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Version : 2.06 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Release : 68.el9 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Architecture: x86_64 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Group : Unspecified 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Size : 8264254 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:License : GPLv3+ 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Packager : builder@centos.org 2024-02-19T18:39:02.388 INFO:teuthology.orchestra.run.smithi088.stdout:Vendor : CentOS 2024-02-19T18:39:02.389 INFO:teuthology.orchestra.run.smithi088.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-19T18:39:02.389 INFO:teuthology.orchestra.run.smithi088.stdout:Summary : Support tools for GRUB. 2024-02-19T18:39:02.389 INFO:teuthology.orchestra.run.smithi088.stdout:Description : 2024-02-19T18:39:02.389 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:39:02.389 INFO:teuthology.orchestra.run.smithi088.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-19T18:39:02.389 INFO:teuthology.orchestra.run.smithi088.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-19T18:39:02.389 INFO:teuthology.orchestra.run.smithi088.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-19T18:39:02.389 INFO:teuthology.orchestra.run.smithi088.stdout:hardware devices. 2024-02-19T18:39:02.389 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:39:02.389 INFO:teuthology.orchestra.run.smithi088.stdout:This subpackage provides tools for support of all platforms. 2024-02-19T18:39:02.390 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-19T18:39:02.390 INFO:teuthology.task.kernel:Updating grub on smithi088 to boot 5.14.0-421.el9.x86_64 2024-02-19T18:39:02.390 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-19T18:39:02.419 INFO:teuthology.orchestra.run.smithi132.stdout:kernel-5.14.0-421.el9.x86_64 2024-02-19T18:39:02.420 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-421.el9.x86_64 2024-02-19T18:39:02.420 DEBUG:teuthology.orchestra.run.smithi132:> sudo rpm -qi grub2-tools 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Name : grub2-tools 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Epoch : 1 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Version : 2.06 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Release : 68.el9 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Architecture: x86_64 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Group : Unspecified 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Size : 8264254 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:License : GPLv3+ 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-19T18:39:02.454 INFO:teuthology.orchestra.run.smithi132.stdout:Packager : builder@centos.org 2024-02-19T18:39:02.455 INFO:teuthology.orchestra.run.smithi132.stdout:Vendor : CentOS 2024-02-19T18:39:02.455 INFO:teuthology.orchestra.run.smithi132.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-19T18:39:02.455 INFO:teuthology.orchestra.run.smithi132.stdout:Summary : Support tools for GRUB. 2024-02-19T18:39:02.455 INFO:teuthology.orchestra.run.smithi132.stdout:Description : 2024-02-19T18:39:02.455 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:39:02.455 INFO:teuthology.orchestra.run.smithi132.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-19T18:39:02.455 INFO:teuthology.orchestra.run.smithi132.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-19T18:39:02.455 INFO:teuthology.orchestra.run.smithi132.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-19T18:39:02.455 INFO:teuthology.orchestra.run.smithi132.stdout:hardware devices. 2024-02-19T18:39:02.455 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:39:02.455 INFO:teuthology.orchestra.run.smithi132.stdout:This subpackage provides tools for support of all platforms. 2024-02-19T18:39:02.456 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-19T18:39:02.456 INFO:teuthology.task.kernel:Updating grub on smithi132 to boot 5.14.0-421.el9.x86_64 2024-02-19T18:39:02.456 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-19T18:39:03.030 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2024-02-19T18:39:03.074 INFO:teuthology.orchestra.run.smithi132.stderr:Generating grub configuration file ... 2024-02-19T18:39:04.443 INFO:teuthology.orchestra.run.smithi088.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-19T18:39:04.488 INFO:teuthology.orchestra.run.smithi088.stderr:done 2024-02-19T18:39:04.491 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-02-19T18:39:04.506 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.EbwPQKkyWm 2024-02-19T18:39:04.507 DEBUG:teuthology.orchestra.run.smithi088:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EbwPQKkyWm 2024-02-19T18:39:04.532 INFO:teuthology.orchestra.run.smithi132.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-19T18:39:04.569 INFO:teuthology.orchestra.run.smithi132.stderr:done 2024-02-19T18:39:04.571 DEBUG:teuthology.orchestra.run.smithi132:> mktemp 2024-02-19T18:39:04.576 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.EbwPQKkyWm 2024-02-19T18:39:04.586 INFO:teuthology.orchestra.run.smithi132.stdout:/tmp/tmp.XmwBkDYZ21 2024-02-19T18:39:04.586 DEBUG:teuthology.orchestra.run.smithi132:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XmwBkDYZ21 2024-02-19T18:39:04.654 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0666 /tmp/tmp.XmwBkDYZ21 2024-02-19T18:39:04.722 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.EbwPQKkyWm is 6KB 2024-02-19T18:39:04.770 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.EbwPQKkyWm 2024-02-19T18:39:04.784 DEBUG:teuthology.orchestra.run.smithi088:> sudo /bin/ls /boot/loader/entries || true 2024-02-19T18:39:04.791 DEBUG:teuthology.orchestra.remote:smithi132:/tmp/tmp.XmwBkDYZ21 is 6KB 2024-02-19T18:39:04.840 DEBUG:teuthology.orchestra.run.smithi132:> rm -fr /tmp/tmp.XmwBkDYZ21 2024-02-19T18:39:04.853 INFO:teuthology.orchestra.run.smithi088.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-19T18:39:04.853 INFO:teuthology.orchestra.run.smithi088.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-19T18:39:04.853 INFO:teuthology.orchestra.run.smithi088.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-19T18:39:04.853 INFO:teuthology.orchestra.run.smithi088.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-19T18:39:04.854 DEBUG:teuthology.orchestra.run.smithi132:> sudo /bin/ls /boot/loader/entries || true 2024-02-19T18:39:04.855 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-19T18:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-19T18:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-19T18:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-19T18:39:04.919 INFO:teuthology.orchestra.run.smithi132.stdout:15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64.conf 2024-02-19T18:39:04.920 DEBUG:teuthology.orchestra.run.smithi132:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-421.el9.x86_64 2024-02-19T18:39:05.015 DEBUG:teuthology.orchestra.run.smithi088:> sudo shutdown -r now 2024-02-19T18:39:05.076 DEBUG:teuthology.orchestra.run.smithi132:> sudo shutdown -r now 2024-02-19T18:39:35.047 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-02-19T18:39:35.048 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-02-19T18:39:35.049 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:39:35.079 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-02-19T18:39:35.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-02-19T18:39:35.079 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:39:53.408 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-02-19T18:39:53.409 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.132 2024-02-19T18:40:02.416 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-02-19T18:40:02.417 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:40:02.419 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-02-19T18:40:02.420 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:40:05.468 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-02-19T18:40:17.480 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-02-19T18:40:17.481 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:40:20.540 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-02-19T18:40:35.552 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-02-19T18:40:35.553 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:40:54.077 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-02-19T18:41:02.428 DEBUG:teuthology.orchestra.remote:timed out 2024-02-19T18:41:12.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-02-19T18:41:12.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:41:12.400 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-02-19T18:41:12.786 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-02-19T18:41:12.786 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-421.el9.x86_64"... 2024-02-19T18:41:12.786 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-02-19T18:41:12.801 INFO:teuthology.orchestra.run.smithi088.stdout:5.14.0-421.el9.x86_64 2024-02-19T18:41:12.801 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-19T18:41:12.802 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-19T18:41:12.802 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-19T18:41:13.802 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-02-19T18:41:13.803 DEBUG:teuthology.orchestra.run.smithi088:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-19T18:41:13.865 INFO:teuthology.orchestra.run.smithi088.stdout:ttyS1 2024-02-19T18:41:13.896 DEBUG:teuthology.parallel:result is None 2024-02-19T18:41:14.429 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-02-19T18:41:14.430 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:41:14.927 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-02-19T18:41:15.338 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-02-19T18:41:15.338 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-421.el9.x86_64"... 2024-02-19T18:41:15.338 DEBUG:teuthology.orchestra.run.smithi132:> uname -r 2024-02-19T18:41:15.354 INFO:teuthology.orchestra.run.smithi132.stdout:5.14.0-421.el9.x86_64 2024-02-19T18:41:15.354 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-421.el9.x86_64 vs 5.14.0-421.el9.x86_64 2024-02-19T18:41:15.354 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-19T18:41:15.354 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-19T18:41:16.355 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-02-19T18:41:16.356 DEBUG:teuthology.orchestra.run.smithi132:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-19T18:41:16.421 INFO:teuthology.orchestra.run.smithi132.stdout:ttyS1 2024-02-19T18:41:16.472 DEBUG:teuthology.parallel:result is None 2024-02-19T18:41:16.473 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-19T18:41:16.481 INFO:teuthology.task.internal:Creating test directory... 2024-02-19T18:41:16.481 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-19T18:41:16.484 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-19T18:41:16.501 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-19T18:41:16.527 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-19T18:41:16.536 INFO:teuthology.task.internal:Creating archive directory... 2024-02-19T18:41:16.537 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-19T18:41:16.543 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-19T18:41:16.583 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-19T18:41:16.590 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-19T18:41:16.591 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-19T18:41:16.621 DEBUG:teuthology.orchestra.run.smithi132:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-19T18:41:16.653 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-19T18:41:16.655 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-19T18:41:16.667 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-19T18:41:16.669 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-19T18:41:16.672 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-19T18:41:16.679 INFO:teuthology.task.internal:Configuring sudo... 2024-02-19T18:41:16.679 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-19T18:41:16.711 DEBUG:teuthology.orchestra.run.smithi132:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-19T18:41:16.741 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-19T18:41:16.752 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-19T18:41:16.753 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-19T18:41:16.781 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-19T18:41:16.798 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-19T18:41:16.880 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-19T18:41:16.927 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-19T18:41:17.011 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-19T18:41:17.052 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:41:17.052 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-19T18:41:17.121 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-19T18:41:17.167 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-19T18:41:17.218 DEBUG:teuthology.orchestra.run.smithi132:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-19T18:41:17.302 DEBUG:teuthology.orchestra.run.smithi132:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-19T18:41:17.346 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:41:17.347 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-19T18:41:17.416 DEBUG:teuthology.orchestra.run.smithi088:> sudo service rsyslog restart 2024-02-19T18:41:17.418 DEBUG:teuthology.orchestra.run.smithi132:> sudo service rsyslog restart 2024-02-19T18:41:17.456 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-19T18:41:17.510 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-19T18:41:17.977 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-19T18:41:17.985 INFO:teuthology.task.internal:Starting timer... 2024-02-19T18:41:17.985 INFO:teuthology.run_tasks:Running task pcp... 2024-02-19T18:41:18.002 INFO:teuthology.run_tasks:Running task selinux... 2024-02-19T18:41:18.011 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-19T18:41:18.011 DEBUG:teuthology.orchestra.run.smithi088:> sudo service auditd rotate 2024-02-19T18:41:18.082 INFO:teuthology.orchestra.run.smithi088.stdout:Rotating logs: 2024-02-19T18:41:18.084 DEBUG:teuthology.orchestra.run.smithi132:> sudo service auditd rotate 2024-02-19T18:41:18.146 INFO:teuthology.orchestra.run.smithi132.stdout:Rotating logs: 2024-02-19T18:41:18.148 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-19T18:41:18.149 DEBUG:teuthology.orchestra.run.smithi088:> /usr/sbin/getenforce 2024-02-19T18:41:18.171 INFO:teuthology.orchestra.run.smithi088.stdout:Permissive 2024-02-19T18:41:18.171 DEBUG:teuthology.orchestra.run.smithi132:> /usr/sbin/getenforce 2024-02-19T18:41:18.191 INFO:teuthology.orchestra.run.smithi132.stdout:Permissive 2024-02-19T18:41:18.192 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi088.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi132.front.sepia.ceph.com': 'permissive'} 2024-02-19T18:41:18.192 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-19T18:41:18.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T18:41:18.239 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-19T18:41:18.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T18:41:18.268 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-19T18:41:18.268 DEBUG:teuthology.orchestra.run.smithi088:> sudo /usr/sbin/setenforce permissive 2024-02-19T18:41:18.323 DEBUG:teuthology.orchestra.run.smithi132:> sudo /usr/sbin/setenforce permissive 2024-02-19T18:41:18.360 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-19T18:41:18.370 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-19T18:41:18.388 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-19T18:41:18.389 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi088.front.sepia.ceph.com,smithi132.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-19T18:47:01.162 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi088.front.sepia.ceph.com'), Remote(name='ubuntu@smithi132.front.sepia.ceph.com')] 2024-02-19T18:47:01.163 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-02-19T18:47:01.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:47:01.281 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-02-19T18:47:01.368 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-02-19T18:47:01.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-02-19T18:47:01.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi132.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-19T18:47:01.469 DEBUG:teuthology.orchestra.run.smithi132:> true 2024-02-19T18:47:01.556 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi132.front.sepia.ceph.com' 2024-02-19T18:47:01.556 INFO:teuthology.run_tasks:Running task clock... 2024-02-19T18:47:01.568 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-19T18:47:01.568 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-19T18:47:01.569 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-19T18:47:01.571 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-19T18:47:01.572 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-19T18:47:01.600 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-19T18:47:01.616 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-19T18:47:01.628 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-19T18:47:01.645 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: ntpd: command not found 2024-02-19T18:47:01.646 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-19T18:47:01.659 INFO:teuthology.orchestra.run.smithi088.stdout:506 Cannot talk to daemon 2024-02-19T18:47:01.675 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-19T18:47:01.676 INFO:teuthology.orchestra.run.smithi132.stderr:sudo: ntpd: command not found 2024-02-19T18:47:01.689 INFO:teuthology.orchestra.run.smithi132.stdout:506 Cannot talk to daemon 2024-02-19T18:47:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-19T18:47:01.706 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-19T18:47:01.722 INFO:teuthology.orchestra.run.smithi132.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-19T18:47:01.752 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-02-19T18:47:01.755 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-19T18:47:01.755 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-02-19T18:47:01.755 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T18:47:01.755 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T18:47:01.756 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T18:47:01.756 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T18:47:01.779 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-02-19T18:47:01.783 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-19T18:47:01.783 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-02-19T18:47:01.783 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T18:47:01.783 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T18:47:01.783 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T18:47:01.783 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T18:47:01.783 INFO:teuthology.run_tasks:Running task install... 2024-02-19T18:47:01.792 DEBUG:teuthology.task.install:project ceph 2024-02-19T18:47:01.796 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'}} 2024-02-19T18:47:01.796 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.0', 'flavor': 'default', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'} 2024-02-19T18:47:01.796 INFO:teuthology.task.install:Using flavor: default 2024-02-19T18:47:01.802 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-19T18:47:01.802 INFO:teuthology.task.install:extra packages: [] 2024-02-19T18:47:01.803 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-02-19T18:47:01.803 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-19T18:47:01.803 INFO:teuthology.packaging:ref: None 2024-02-19T18:47:01.803 INFO:teuthology.packaging:tag: v18.2.0 2024-02-19T18:47:01.803 INFO:teuthology.packaging:branch: None 2024-02-19T18:47:01.803 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:47:02.083 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v18.2.0^{} -> None 2024-02-19T18:47:02.083 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-19T18:47:03.764 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.0^{} -> 5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-19T18:47:03.765 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-19T18:47:03.768 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-02-19T18:47:03.768 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-19T18:47:03.768 INFO:teuthology.packaging:ref: None 2024-02-19T18:47:03.768 INFO:teuthology.packaging:tag: v18.2.0 2024-02-19T18:47:03.768 INFO:teuthology.packaging:branch: None 2024-02-19T18:47:03.768 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:47:03.769 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-19T18:47:03.769 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-19T18:47:03.999 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-02-19T18:47:03.999 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-02-19T18:47:04.007 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-02-19T18:47:04.008 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-02-19T18:47:04.058 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-19T18:47:04.058 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:47:04.058 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-19T18:47:04.079 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-19T18:47:04.079 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:47:04.079 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-19T18:47:04.092 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-19T18:47:04.093 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-19T18:47:04.093 INFO:teuthology.packaging:ref: None 2024-02-19T18:47:04.093 INFO:teuthology.packaging:tag: v18.2.0 2024-02-19T18:47:04.093 INFO:teuthology.packaging:branch: None 2024-02-19T18:47:04.093 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:47:04.093 DEBUG:teuthology.orchestra.run.smithi132:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-19T18:47:04.112 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-02-19T18:47:04.112 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-19T18:47:04.112 INFO:teuthology.packaging:ref: None 2024-02-19T18:47:04.112 INFO:teuthology.packaging:tag: v18.2.0 2024-02-19T18:47:04.112 INFO:teuthology.packaging:branch: None 2024-02-19T18:47:04.112 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:47:04.112 DEBUG:teuthology.orchestra.run.smithi088:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-19T18:47:04.165 DEBUG:teuthology.orchestra.run.smithi132:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-19T18:47:04.183 DEBUG:teuthology.orchestra.run.smithi088:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-19T18:47:04.252 DEBUG:teuthology.orchestra.run.smithi132:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-19T18:47:04.265 DEBUG:teuthology.orchestra.run.smithi088:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-19T18:47:04.322 INFO:teuthology.orchestra.run.smithi132.stdout:check_obsoletes = 1 2024-02-19T18:47:04.323 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum clean all 2024-02-19T18:47:04.332 INFO:teuthology.orchestra.run.smithi088.stdout:check_obsoletes = 1 2024-02-19T18:47:04.334 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum clean all 2024-02-19T18:47:04.574 INFO:teuthology.orchestra.run.smithi132.stdout:74 files removed 2024-02-19T18:47:04.583 INFO:teuthology.orchestra.run.smithi088.stdout:74 files removed 2024-02-19T18:47:04.601 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-radosgw 2024-02-19T18:47:04.609 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-radosgw 2024-02-19T18:47:05.051 INFO:teuthology.orchestra.run.smithi132.stdout:ceph packages for x86_64 589 kB/s | 76 kB 00:00 2024-02-19T18:47:05.053 INFO:teuthology.orchestra.run.smithi088.stdout:ceph packages for x86_64 600 kB/s | 76 kB 00:00 2024-02-19T18:47:05.237 INFO:teuthology.orchestra.run.smithi088.stdout:ceph noarch packages 69 kB/s | 9.3 kB 00:00 2024-02-19T18:47:05.243 INFO:teuthology.orchestra.run.smithi132.stdout:ceph noarch packages 65 kB/s | 9.3 kB 00:00 2024-02-19T18:47:05.419 INFO:teuthology.orchestra.run.smithi132.stdout:ceph source packages 16 kB/s | 2.2 kB 00:00 2024-02-19T18:47:05.421 INFO:teuthology.orchestra.run.smithi088.stdout:ceph source packages 15 kB/s | 2.2 kB 00:00 2024-02-19T18:47:06.357 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-9 - AppStream 7.5 MB/s | 6.7 MB 00:00 2024-02-19T18:47:06.505 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - AppStream 6.4 MB/s | 6.7 MB 00:01 2024-02-19T18:47:08.157 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS-9 - BaseOS 3.0 MB/s | 2.0 MB 00:00 2024-02-19T18:47:08.230 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS-9 - BaseOS 3.3 MB/s | 2.0 MB 00:00 2024-02-19T18:47:09.172 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-02-19T18:47:10.672 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 3.7 MB/s | 8.0 MB 00:02 2024-02-19T18:47:14.990 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-02-19T18:47:19.749 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 5.0 MB/s | 6.1 MB 00:01 2024-02-19T18:47:21.901 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 15 kB 00:00 2024-02-19T18:47:50.709 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - AppStream 484 kB/s | 19 MB 00:40 2024-02-19T18:47:54.593 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.1 MB 00:00 2024-02-19T18:47:56.027 INFO:teuthology.orchestra.run.smithi132.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 15 kB 00:00 2024-02-19T18:47:57.373 INFO:teuthology.orchestra.run.smithi132.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2024-02-19T18:47:58.552 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 571 kB/s | 20 MB 00:36 2024-02-19T18:48:02.281 INFO:teuthology.orchestra.run.smithi132.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-02-19T18:48:03.594 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-02-19T18:48:04.866 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:48:04.882 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-02-19T18:48:04.882 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:04.882 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:48:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-02-19T18:48:04.882 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrading: 2024-02-19T18:48:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-02-19T18:48:04.882 INFO:teuthology.orchestra.run.smithi132.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-19T18:48:04.882 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-19T18:48:04.883 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-02-19T18:48:04.884 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-02-19T18:48:04.885 INFO:teuthology.orchestra.run.smithi132.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-19T18:48:04.885 INFO:teuthology.orchestra.run.smithi132.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-19T18:48:04.885 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:04.885 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:48:04.885 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:04.885 INFO:teuthology.orchestra.run.smithi132.stdout:Install 28 Packages 2024-02-19T18:48:04.885 INFO:teuthology.orchestra.run.smithi132.stdout:Upgrade 2 Packages 2024-02-19T18:48:04.885 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:04.886 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 53 M 2024-02-19T18:48:04.886 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:48:05.313 INFO:teuthology.orchestra.run.smithi132.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 24 MB/s | 5.2 MB 00:00 2024-02-19T18:48:05.330 INFO:teuthology.orchestra.run.smithi132.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 1.4 MB/s | 24 kB 00:00 2024-02-19T18:48:05.422 INFO:teuthology.orchestra.run.smithi132.stdout:(3/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 23 MB/s | 7.6 MB 00:00 2024-02-19T18:48:05.448 INFO:teuthology.orchestra.run.smithi132.stdout:(4/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 5.5 MB/s | 653 kB 00:00 2024-02-19T18:48:05.472 INFO:teuthology.orchestra.run.smithi132.stdout:(5/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 9.3 MB/s | 474 kB 00:00 2024-02-19T18:48:05.489 INFO:teuthology.orchestra.run.smithi132.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 2.6 MB/s | 45 kB 00:00 2024-02-19T18:48:05.515 INFO:teuthology.orchestra.run.smithi132.stdout:(7/30): python3-ceph-common-18.2.0-0.el9.x86_64 4.6 MB/s | 119 kB 00:00 2024-02-19T18:48:05.540 INFO:teuthology.orchestra.run.smithi132.stdout:(8/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 6.1 MB/s | 155 kB 00:00 2024-02-19T18:48:05.565 INFO:teuthology.orchestra.run.smithi132.stdout:(9/30): python3-rados-18.2.0-0.el9.x86_64.rpm 13 MB/s | 321 kB 00:00 2024-02-19T18:48:05.599 INFO:teuthology.orchestra.run.smithi132.stdout:(10/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 8.7 MB/s | 297 kB 00:00 2024-02-19T18:48:05.624 INFO:teuthology.orchestra.run.smithi132.stdout:(11/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 3.9 MB/s | 99 kB 00:00 2024-02-19T18:48:05.691 INFO:teuthology.orchestra.run.smithi132.stdout:(12/30): librgw2-18.2.0-0.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-02-19T18:48:05.966 INFO:teuthology.orchestra.run.smithi132.stdout:(13/30): ceph-common-18.2.0-0.el9.x86_64.rpm 21 MB/s | 18 MB 00:00 2024-02-19T18:48:05.991 INFO:teuthology.orchestra.run.smithi132.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 288 kB/s | 106 kB 00:00 2024-02-19T18:48:06.008 INFO:teuthology.orchestra.run.smithi132.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-02-19T18:48:06.042 INFO:teuthology.orchestra.run.smithi132.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 486 kB/s | 159 kB 00:00 2024-02-19T18:48:06.100 INFO:teuthology.orchestra.run.smithi132.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 661 kB 00:00 2024-02-19T18:48:06.126 INFO:teuthology.orchestra.run.smithi132.stdout:(18/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 282 kB 00:00 2024-02-19T18:48:06.143 INFO:teuthology.orchestra.run.smithi132.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 411 kB/s | 41 kB 00:00 2024-02-19T18:48:06.187 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:48:06.201 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:06.201 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrading: 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-02-19T18:48:06.202 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-02-19T18:48:06.203 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout:Install 28 Packages 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrade 2 Packages 2024-02-19T18:48:06.204 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:06.205 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 53 M 2024-02-19T18:48:06.205 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:48:06.453 INFO:teuthology.orchestra.run.smithi088.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 26 MB/s | 5.2 MB 00:00 2024-02-19T18:48:06.485 INFO:teuthology.orchestra.run.smithi132.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 289 kB/s | 111 kB 00:00 2024-02-19T18:48:06.554 INFO:teuthology.orchestra.run.smithi088.stdout:(2/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 25 MB/s | 7.6 MB 00:00 2024-02-19T18:48:06.568 INFO:teuthology.orchestra.run.smithi132.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-02-19T18:48:06.570 INFO:teuthology.orchestra.run.smithi088.stdout:(3/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 205 kB/s | 24 kB 00:00 2024-02-19T18:48:06.602 INFO:teuthology.orchestra.run.smithi132.stdout:(22/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 673 kB/s | 309 kB 00:00 2024-02-19T18:48:06.604 INFO:teuthology.orchestra.run.smithi088.stdout:(4/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 19 MB/s | 653 kB 00:00 2024-02-19T18:48:06.638 INFO:teuthology.orchestra.run.smithi088.stdout:(5/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 7.0 MB/s | 474 kB 00:00 2024-02-19T18:48:06.643 INFO:teuthology.orchestra.run.smithi132.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 645 kB/s | 49 kB 00:00 2024-02-19T18:48:06.654 INFO:teuthology.orchestra.run.smithi088.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 2.7 MB/s | 45 kB 00:00 2024-02-19T18:48:06.677 INFO:teuthology.orchestra.run.smithi132.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 895 kB/s | 67 kB 00:00 2024-02-19T18:48:06.688 INFO:teuthology.orchestra.run.smithi088.stdout:(7/30): python3-ceph-common-18.2.0-0.el9.x86_64 3.5 MB/s | 119 kB 00:00 2024-02-19T18:48:06.713 INFO:teuthology.orchestra.run.smithi088.stdout:(8/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 6.1 MB/s | 155 kB 00:00 2024-02-19T18:48:06.739 INFO:teuthology.orchestra.run.smithi088.stdout:(9/30): python3-rados-18.2.0-0.el9.x86_64.rpm 12 MB/s | 321 kB 00:00 2024-02-19T18:48:06.764 INFO:teuthology.orchestra.run.smithi088.stdout:(10/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 12 MB/s | 297 kB 00:00 2024-02-19T18:48:06.769 INFO:teuthology.orchestra.run.smithi132.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-02-19T18:48:06.789 INFO:teuthology.orchestra.run.smithi088.stdout:(11/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 3.9 MB/s | 99 kB 00:00 2024-02-19T18:48:06.853 INFO:teuthology.orchestra.run.smithi132.stdout:(26/30): libarrow-9.0.0-10.el9.x86_64.rpm 6.2 MB/s | 4.4 MB 00:00 2024-02-19T18:48:06.865 INFO:teuthology.orchestra.run.smithi088.stdout:(12/30): librgw2-18.2.0-0.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-02-19T18:48:06.887 INFO:teuthology.orchestra.run.smithi132.stdout:(27/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.4 MB/s | 838 kB 00:00 2024-02-19T18:48:06.954 INFO:teuthology.orchestra.run.smithi132.stdout:(28/30): librados2-18.2.0-0.el9.x86_64.rpm 33 MB/s | 3.3 MB 00:00 2024-02-19T18:48:07.004 INFO:teuthology.orchestra.run.smithi132.stdout:(29/30): librbd1-18.2.0-0.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-02-19T18:48:07.046 INFO:teuthology.orchestra.run.smithi132.stdout:(30/30): thrift-0.15.0-2.el9.x86_64.rpm 5.7 MB/s | 1.6 MB 00:00 2024-02-19T18:48:07.052 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:07.052 INFO:teuthology.orchestra.run.smithi132.stdout:Total 24 MB/s | 53 MB 00:02 2024-02-19T18:48:07.052 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:48:07.073 INFO:teuthology.orchestra.run.smithi088.stdout:(13/30): ceph-common-18.2.0-0.el9.x86_64.rpm 22 MB/s | 18 MB 00:00 2024-02-19T18:48:07.083 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:48:07.083 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:48:07.140 INFO:teuthology.orchestra.run.smithi088.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 578 kB/s | 159 kB 00:00 2024-02-19T18:48:07.166 INFO:teuthology.orchestra.run.smithi088.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 488 kB/s | 44 kB 00:00 2024-02-19T18:48:07.224 INFO:teuthology.orchestra.run.smithi088.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 661 kB 00:00 2024-02-19T18:48:07.258 INFO:teuthology.orchestra.run.smithi088.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2024-02-19T18:48:07.283 INFO:teuthology.orchestra.run.smithi088.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 709 kB/s | 41 kB 00:00 2024-02-19T18:48:07.525 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:48:07.525 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:48:07.675 INFO:teuthology.orchestra.run.smithi088.stdout:(19/30): fmt-8.1.1-5.el9.x86_64.rpm 266 kB/s | 111 kB 00:00 2024-02-19T18:48:07.734 INFO:teuthology.orchestra.run.smithi088.stdout:(20/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 686 kB/s | 309 kB 00:00 2024-02-19T18:48:07.842 INFO:teuthology.orchestra.run.smithi088.stdout:(21/30): libarrow-doc-9.0.0-10.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-02-19T18:48:07.943 INFO:teuthology.orchestra.run.smithi088.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 485 kB/s | 49 kB 00:00 2024-02-19T18:48:08.043 INFO:teuthology.orchestra.run.smithi088.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 674 kB/s | 67 kB 00:00 2024-02-19T18:48:08.260 INFO:teuthology.orchestra.run.smithi088.stdout:(24/30): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 838 kB 00:00 2024-02-19T18:48:08.360 INFO:teuthology.orchestra.run.smithi088.stdout:(25/30): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-02-19T18:48:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:(26/30): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-02-19T18:48:08.694 INFO:teuthology.orchestra.run.smithi088.stdout:(27/30): librados2-18.2.0-0.el9.x86_64.rpm 44 MB/s | 3.3 MB 00:00 2024-02-19T18:48:08.761 INFO:teuthology.orchestra.run.smithi088.stdout:(28/30): librbd1-18.2.0-0.el9.x86_64.rpm 45 MB/s | 3.0 MB 00:00 2024-02-19T18:48:08.890 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:48:08.953 INFO:teuthology.orchestra.run.smithi088.stdout:(29/30): boost-program-options-1.75.0-8.el9.x86 49 kB/s | 106 kB 00:02 2024-02-19T18:48:08.962 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-02-19T18:48:08.995 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-02-19T18:48:09.043 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-02-19T18:48:09.312 INFO:teuthology.orchestra.run.smithi088.stdout:(30/30): libarrow-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-02-19T18:48:09.317 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:09.318 INFO:teuthology.orchestra.run.smithi088.stdout:Total 17 MB/s | 53 MB 00:03 2024-02-19T18:48:09.318 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:48:09.352 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:48:09.352 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:48:09.360 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-02-19T18:48:09.408 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-19T18:48:09.482 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-19T18:48:09.563 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-02-19T18:48:09.593 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-19T18:48:09.657 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-19T18:48:09.684 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-02-19T18:48:09.737 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-02-19T18:48:09.791 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:48:09.791 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:48:09.809 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-19T18:48:09.837 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-19T18:48:09.894 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-19T18:48:09.999 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-02-19T18:48:10.045 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-02-19T18:48:10.085 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-02-19T18:48:10.117 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-02-19T18:48:10.144 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-02-19T18:48:10.180 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-02-19T18:48:10.212 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-02-19T18:48:10.510 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-02-19T18:48:10.534 INFO:teuthology.orchestra.run.smithi132.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-19T18:48:10.624 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-19T18:48:10.655 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-02-19T18:48:10.701 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-02-19T18:48:11.092 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-02-19T18:48:11.128 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-19T18:48:11.160 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:48:11.183 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-19T18:48:11.257 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-02-19T18:48:11.260 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-02-19T18:48:11.281 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-02-19T18:48:11.291 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-02-19T18:48:11.341 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-02-19T18:48:11.659 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-02-19T18:48:11.707 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-19T18:48:11.799 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-02-19T18:48:11.879 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-02-19T18:48:11.917 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-19T18:48:11.980 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-02-19T18:48:12.008 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-02-19T18:48:12.056 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-02-19T18:48:12.118 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-19T18:48:12.145 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-19T18:48:12.193 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-02-19T18:48:12.313 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-02-19T18:48:12.362 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 13/32 2024-02-19T18:48:12.401 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-02-19T18:48:12.433 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-02-19T18:48:12.461 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 16/32 2024-02-19T18:48:12.496 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-02-19T18:48:12.529 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-02-19T18:48:12.831 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-02-19T18:48:12.859 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-19T18:48:12.950 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-02-19T18:48:12.981 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-02-19T18:48:13.022 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-02-19T18:48:13.265 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-19T18:48:13.302 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-19T18:48:13.412 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 23/32 2024-02-19T18:48:13.446 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-19T18:48:13.501 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-02-19T18:48:13.576 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-02-19T18:48:13.606 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-02-19T18:48:13.845 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-19T18:48:13.889 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-19T18:48:13.980 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-19T18:48:13.980 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-19T18:48:13.980 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-19T18:48:13.981 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:14.006 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-19T18:48:15.638 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-19T18:48:15.678 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-19T18:48:16.237 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-02-19T18:48:16.290 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-19T18:48:16.373 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-02-19T18:48:16.373 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-02-19T18:48:16.373 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-02-19T18:48:16.373 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:16.416 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-19T18:48:23.745 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-19T18:48:23.745 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /sys 2024-02-19T18:48:23.745 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /proc 2024-02-19T18:48:23.745 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /mnt 2024-02-19T18:48:23.745 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /var/tmp 2024-02-19T18:48:23.745 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /home 2024-02-19T18:48:23.745 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /root 2024-02-19T18:48:23.745 INFO:teuthology.orchestra.run.smithi132.stdout:skipping the directory /tmp 2024-02-19T18:48:23.746 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:24.477 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-19T18:48:24.500 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-19T18:48:24.500 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:24.501 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-19T18:48:24.501 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-19T18:48:24.501 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-19T18:48:24.501 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:24.502 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-19T18:48:24.587 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-19T18:48:24.588 INFO:teuthology.orchestra.run.smithi132.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-19T18:48:25.674 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-19T18:48:25.674 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-02-19T18:48:25.674 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-02-19T18:48:25.674 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-02-19T18:48:25.674 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-02-19T18:48:25.674 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-02-19T18:48:25.674 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-02-19T18:48:25.675 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-02-19T18:48:25.675 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-02-19T18:48:25.675 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-02-19T18:48:25.675 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-19T18:48:25.675 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-02-19T18:48:25.675 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-02-19T18:48:25.675 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-02-19T18:48:25.675 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-02-19T18:48:25.675 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-02-19T18:48:25.675 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-02-19T18:48:25.675 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-02-19T18:48:25.676 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-02-19T18:48:25.677 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-02-19T18:48:25.677 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-02-19T18:48:25.889 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-02-19T18:48:25.889 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /sys 2024-02-19T18:48:25.890 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /proc 2024-02-19T18:48:25.890 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /mnt 2024-02-19T18:48:25.890 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /var/tmp 2024-02-19T18:48:25.890 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /home 2024-02-19T18:48:25.890 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /root 2024-02-19T18:48:25.890 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /tmp 2024-02-19T18:48:25.890 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:26.650 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-19T18:48:26.675 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-02-19T18:48:26.675 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:26.675 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-02-19T18:48:26.675 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-19T18:48:26.675 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-02-19T18:48:26.676 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:26.677 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-19T18:48:26.753 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-02-19T18:48:26.753 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-19T18:48:26.802 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-02-19T18:48:26.802 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:26.802 INFO:teuthology.orchestra.run.smithi132.stdout:Upgraded: 2024-02-19T18:48:26.802 INFO:teuthology.orchestra.run.smithi132.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.802 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:48:26.802 INFO:teuthology.orchestra.run.smithi132.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-19T18:48:26.802 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: fmt-8.1.1-5.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-19T18:48:26.803 INFO:teuthology.orchestra.run.smithi132.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: re2-1:20211101-3.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:26.804 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:48:27.059 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-test 2024-02-19T18:48:27.643 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:25 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:48:27.807 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:48:27.808 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:27.809 INFO:teuthology.orchestra.run.smithi132.stdout:Install 6 Packages 2024-02-19T18:48:27.809 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:27.810 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 40 M 2024-02-19T18:48:27.810 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 167 M 2024-02-19T18:48:27.811 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:48:27.956 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-02-19T18:48:27.956 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-02-19T18:48:27.956 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-02-19T18:48:27.956 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-02-19T18:48:27.956 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-02-19T18:48:27.956 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-02-19T18:48:27.956 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-02-19T18:48:27.956 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-02-19T18:48:27.957 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-02-19T18:48:27.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-02-19T18:48:27.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 22/32 2024-02-19T18:48:27.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 23/32 2024-02-19T18:48:27.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-02-19T18:48:27.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-02-19T18:48:27.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 26/32 2024-02-19T18:48:27.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-02-19T18:48:27.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-02-19T18:48:27.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-02-19T18:48:27.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-02-19T18:48:27.958 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-02-19T18:48:27.962 INFO:teuthology.orchestra.run.smithi132.stdout:(1/6): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.0 MB/s | 161 kB 00:00 2024-02-19T18:48:28.070 INFO:teuthology.orchestra.run.smithi132.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 1.1 MB/s | 300 kB 00:00 2024-02-19T18:48:28.095 INFO:teuthology.orchestra.run.smithi132.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 469 kB/s | 63 kB 00:00 2024-02-19T18:48:28.146 INFO:teuthology.orchestra.run.smithi132.stdout:(4/6): jq-1.6-16.el9.x86_64.rpm 2.4 MB/s | 186 kB 00:00 2024-02-19T18:48:28.188 INFO:teuthology.orchestra.run.smithi132.stdout:(5/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.3 MB/s | 217 kB 00:00 2024-02-19T18:48:28.580 INFO:teuthology.orchestra.run.smithi132.stdout:(6/6): ceph-test-18.2.0-0.el9.x86_64.rpm 51 MB/s | 40 MB 00:00 2024-02-19T18:48:28.583 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:28.584 INFO:teuthology.orchestra.run.smithi132.stdout:Total 52 MB/s | 40 MB 00:00 2024-02-19T18:48:28.584 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:48:28.604 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:48:28.604 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:48:28.848 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-02-19T18:48:28.848 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:28.848 INFO:teuthology.orchestra.run.smithi088.stdout:Upgraded: 2024-02-19T18:48:28.848 INFO:teuthology.orchestra.run.smithi088.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.848 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:48:28.848 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-02-19T18:48:28.848 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.848 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.848 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: fmt-8.1.1-5.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-9.0.0-10.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: liboath-2.6.7-2.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.849 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind-1.6.2-1.el9.x86_64 2024-02-19T18:48:28.850 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-02-19T18:48:28.850 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-02-19T18:48:28.850 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.850 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.850 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.850 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-02-19T18:48:28.850 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.850 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.850 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:28.850 INFO:teuthology.orchestra.run.smithi088.stdout: re2-1:20211101-3.el9.x86_64 2024-02-19T18:48:28.850 INFO:teuthology.orchestra.run.smithi088.stdout: thrift-0.15.0-2.el9.x86_64 2024-02-19T18:48:28.851 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:28.851 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:48:28.954 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:48:28.954 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:48:29.222 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-test 2024-02-19T18:48:29.506 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:48:29.586 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-19T18:48:29.612 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-19T18:48:29.652 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-19T18:48:29.699 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-19T18:48:29.718 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-19T18:48:29.802 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:26 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:48:29.967 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout: jq x86_64 1.6-16.el9 CentOS-BaseOS 186 k 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-BaseOS 217 k 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-02-19T18:48:29.968 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:29.969 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:48:29.969 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:29.969 INFO:teuthology.orchestra.run.smithi088.stdout:Install 6 Packages 2024-02-19T18:48:29.969 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:29.969 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 40 M 2024-02-19T18:48:29.969 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 167 M 2024-02-19T18:48:29.969 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:48:30.097 INFO:teuthology.orchestra.run.smithi088.stdout:(1/6): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.2 MB/s | 161 kB 00:00 2024-02-19T18:48:30.597 INFO:teuthology.orchestra.run.smithi088.stdout:(2/6): ceph-test-18.2.0-0.el9.x86_64.rpm 63 MB/s | 40 MB 00:00 2024-02-19T18:48:30.622 INFO:teuthology.orchestra.run.smithi088.stdout:(3/6): socat-1.7.4.1-5.el9.x86_64.rpm 460 kB/s | 300 kB 00:00 2024-02-19T18:48:30.647 INFO:teuthology.orchestra.run.smithi088.stdout:(4/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 114 kB/s | 63 kB 00:00 2024-02-19T18:48:30.756 INFO:teuthology.orchestra.run.smithi088.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 186 kB 00:00 2024-02-19T18:48:30.782 INFO:teuthology.orchestra.run.smithi088.stdout:(6/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.6 MB/s | 217 kB 00:00 2024-02-19T18:48:30.798 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:30.798 INFO:teuthology.orchestra.run.smithi088.stdout:Total 50 MB/s | 40 MB 00:00 2024-02-19T18:48:30.798 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:48:30.805 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:48:30.805 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:48:31.155 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:48:31.155 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:48:31.721 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:48:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 1/6 2024-02-19T18:48:31.848 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : jq-1.6-16.el9.x86_64 2/6 2024-02-19T18:48:31.892 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 3/6 2024-02-19T18:48:31.932 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 4/6 2024-02-19T18:48:31.960 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-19T18:48:33.435 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-19T18:48:33.509 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-19T18:48:35.599 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-19T18:48:35.599 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/6 2024-02-19T18:48:35.599 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/6 2024-02-19T18:48:35.599 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-19T18:48:35.599 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-19T18:48:35.600 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-19T18:48:35.703 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 5/6 2024-02-19T18:48:35.784 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-19T18:48:36.000 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-19T18:48:36.000 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:36.000 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:48:36.000 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-test-2:18.2.0-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-02-19T18:48:36.000 INFO:teuthology.orchestra.run.smithi132.stdout: libcephsqlite-2:18.2.0-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-02-19T18:48:36.000 INFO:teuthology.orchestra.run.smithi132.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-02-19T18:48:36.000 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:36.001 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:48:36.288 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph 2024-02-19T18:48:36.872 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:34 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:48:37.038 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-19T18:48:37.041 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-19T18:48:37.042 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:37.043 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:48:37.044 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:37.044 INFO:teuthology.orchestra.run.smithi132.stdout:Install 33 Packages 2024-02-19T18:48:37.044 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:37.044 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 30 M 2024-02-19T18:48:37.044 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 109 M 2024-02-19T18:48:37.044 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:48:37.343 INFO:teuthology.orchestra.run.smithi132.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 51 kB/s | 6.4 kB 00:00 2024-02-19T18:48:37.393 INFO:teuthology.orchestra.run.smithi132.stdout:(2/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 8.2 MB/s | 1.4 MB 00:00 2024-02-19T18:48:37.443 INFO:teuthology.orchestra.run.smithi132.stdout:(3/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 9.3 MB/s | 2.1 MB 00:00 2024-02-19T18:48:37.477 INFO:teuthology.orchestra.run.smithi132.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 7.0 MB/s | 240 kB 00:00 2024-02-19T18:48:37.552 INFO:teuthology.orchestra.run.smithi132.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-02-19T18:48:37.686 INFO:teuthology.orchestra.run.smithi132.stdout:(6/33): python3-markupsafe-1.1.1-12.el9.x86_64. 287 kB/s | 32 kB 00:00 2024-02-19T18:48:37.711 INFO:teuthology.orchestra.run.smithi132.stdout:(7/33): python3-mako-1.1.4-6.el9.noarch.rpm 616 kB/s | 144 kB 00:00 2024-02-19T18:48:37.736 INFO:teuthology.orchestra.run.smithi132.stdout:(8/33): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2024-02-19T18:48:37.761 INFO:teuthology.orchestra.run.smithi132.stdout:(9/33): python3-toml-0.10.2-6.el9.noarch.rpm 772 kB/s | 39 kB 00:00 2024-02-19T18:48:37.937 INFO:teuthology.orchestra.run.smithi132.stdout:(10/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 32 MB/s | 18 MB 00:00 2024-02-19T18:48:37.962 INFO:teuthology.orchestra.run.smithi132.stdout:(11/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 241 kB 00:00 2024-02-19T18:48:37.991 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 6/6 2024-02-19T18:48:37.991 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/6 2024-02-19T18:48:37.991 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/6 2024-02-19T18:48:37.991 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/6 2024-02-19T18:48:37.991 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/6 2024-02-19T18:48:37.991 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : jq-1.6-16.el9.x86_64 5/6 2024-02-19T18:48:38.037 INFO:teuthology.orchestra.run.smithi132.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 4.2 MB/s | 1.1 MB 00:00 2024-02-19T18:48:38.063 INFO:teuthology.orchestra.run.smithi132.stdout:(13/33): python3-ply-3.11-14.el9.noarch.rpm 825 kB/s | 103 kB 00:00 2024-02-19T18:48:38.113 INFO:teuthology.orchestra.run.smithi132.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 826 kB/s | 124 kB 00:00 2024-02-19T18:48:38.138 INFO:teuthology.orchestra.run.smithi132.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 1.1 MB/s | 113 kB 00:00 2024-02-19T18:48:38.172 INFO:teuthology.orchestra.run.smithi132.stdout:(16/33): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 187 kB 00:00 2024-02-19T18:48:38.330 INFO:teuthology.orchestra.run.smithi132.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 200 kB/s | 43 kB 00:00 2024-02-19T18:48:38.380 INFO:teuthology.orchestra.run.smithi132.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 214 kB/s | 11 kB 00:00 2024-02-19T18:48:38.405 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 6/6 2024-02-19T18:48:38.405 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:38.405 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:48:38.405 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test-2:18.2.0-0.el9.x86_64 jq-1.6-16.el9.x86_64 2024-02-19T18:48:38.405 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite-2:18.2.0-0.el9.x86_64 oniguruma-6.9.6-1.el9.5.x86_64 2024-02-19T18:48:38.405 INFO:teuthology.orchestra.run.smithi088.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-02-19T18:48:38.405 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:38.405 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:48:38.406 INFO:teuthology.orchestra.run.smithi132.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 644 kB/s | 172 kB 00:00 2024-02-19T18:48:38.423 INFO:teuthology.orchestra.run.smithi132.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 420 kB/s | 18 kB 00:00 2024-02-19T18:48:38.448 INFO:teuthology.orchestra.run.smithi132.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 552 kB/s | 23 kB 00:00 2024-02-19T18:48:38.473 INFO:teuthology.orchestra.run.smithi132.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-02-19T18:48:38.490 INFO:teuthology.orchestra.run.smithi132.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 289 kB/s | 19 kB 00:00 2024-02-19T18:48:38.507 INFO:teuthology.orchestra.run.smithi132.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 334 kB/s | 20 kB 00:00 2024-02-19T18:48:38.524 INFO:teuthology.orchestra.run.smithi132.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 917 kB/s | 46 kB 00:00 2024-02-19T18:48:38.558 INFO:teuthology.orchestra.run.smithi132.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 491 kB/s | 16 kB 00:00 2024-02-19T18:48:38.583 INFO:teuthology.orchestra.run.smithi132.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-02-19T18:48:38.617 INFO:teuthology.orchestra.run.smithi132.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.5 MB/s | 90 kB 00:00 2024-02-19T18:48:38.642 INFO:teuthology.orchestra.run.smithi132.stdout:(29/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-02-19T18:48:38.659 INFO:teuthology.orchestra.run.smithi132.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 473 kB/s | 36 kB 00:00 2024-02-19T18:48:38.690 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph 2024-02-19T18:48:38.709 INFO:teuthology.orchestra.run.smithi132.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 399 kB/s | 20 kB 00:00 2024-02-19T18:48:38.743 INFO:teuthology.orchestra.run.smithi132.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-02-19T18:48:38.885 INFO:teuthology.orchestra.run.smithi132.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.9 MB/s | 427 kB 00:00 2024-02-19T18:48:38.891 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:38.891 INFO:teuthology.orchestra.run.smithi132.stdout:Total 16 MB/s | 30 MB 00:01 2024-02-19T18:48:38.891 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:48:38.926 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:48:38.926 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:48:39.267 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:36 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:48:39.292 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:48:39.292 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:48:39.434 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:48:39.436 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:39.436 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-02-19T18:48:39.436 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:39.436 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:48:39.436 INFO:teuthology.orchestra.run.smithi088.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-02-19T18:48:39.437 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-02-19T18:48:39.438 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests noarch 2.25.1-8.el9 CentOS-BaseOS 113 k 2024-02-19T18:48:39.439 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-02-19T18:48:39.439 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-02-19T18:48:39.439 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-02-19T18:48:39.439 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-02-19T18:48:39.439 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-02-19T18:48:39.439 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-02-19T18:48:39.439 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:39.439 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:48:39.439 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:39.439 INFO:teuthology.orchestra.run.smithi088.stdout:Install 33 Packages 2024-02-19T18:48:39.439 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:39.440 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 30 M 2024-02-19T18:48:39.440 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 109 M 2024-02-19T18:48:39.440 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:48:39.933 INFO:teuthology.orchestra.run.smithi088.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 49 kB/s | 6.4 kB 00:00 2024-02-19T18:48:39.991 INFO:teuthology.orchestra.run.smithi088.stdout:(2/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 11 MB/s | 2.1 MB 00:00 2024-02-19T18:48:40.033 INFO:teuthology.orchestra.run.smithi088.stdout:(3/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 6.2 MB/s | 1.4 MB 00:00 2024-02-19T18:48:40.058 INFO:teuthology.orchestra.run.smithi088.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 9.4 MB/s | 240 kB 00:00 2024-02-19T18:48:40.142 INFO:teuthology.orchestra.run.smithi088.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-02-19T18:48:40.409 INFO:teuthology.orchestra.run.smithi088.stdout:(6/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 42 MB/s | 18 MB 00:00 2024-02-19T18:48:40.426 INFO:teuthology.orchestra.run.smithi088.stdout:(7/33): python3-markupsafe-1.1.1-12.el9.x86_64. 121 kB/s | 32 kB 00:00 2024-02-19T18:48:40.451 INFO:teuthology.orchestra.run.smithi088.stdout:(8/33): python3-mako-1.1.4-6.el9.noarch.rpm 367 kB/s | 144 kB 00:00 2024-02-19T18:48:40.477 INFO:teuthology.orchestra.run.smithi088.stdout:(9/33): python3-pytz-2021.1-5.el9.noarch.rpm 950 kB/s | 47 kB 00:00 2024-02-19T18:48:40.493 INFO:teuthology.orchestra.run.smithi088.stdout:(10/33): python3-toml-0.10.2-6.el9.noarch.rpm 580 kB/s | 39 kB 00:00 2024-02-19T18:48:40.527 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:48:40.544 INFO:teuthology.orchestra.run.smithi088.stdout:(11/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 241 kB 00:00 2024-02-19T18:48:40.595 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-19T18:48:40.628 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-19T18:48:40.661 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-19T18:48:40.752 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-19T18:48:40.760 INFO:teuthology.orchestra.run.smithi088.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 4.0 MB/s | 1.1 MB 00:00 2024-02-19T18:48:40.786 INFO:teuthology.orchestra.run.smithi088.stdout:(13/33): python3-ply-3.11-14.el9.noarch.rpm 352 kB/s | 103 kB 00:00 2024-02-19T18:48:40.811 INFO:teuthology.orchestra.run.smithi088.stdout:(14/33): python3-pycparser-2.20-6.el9.noarch.rp 465 kB/s | 124 kB 00:00 2024-02-19T18:48:40.830 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-19T18:48:40.836 INFO:teuthology.orchestra.run.smithi088.stdout:(15/33): python3-requests-2.25.1-8.el9.noarch.r 1.5 MB/s | 113 kB 00:00 2024-02-19T18:48:40.871 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-19T18:48:40.903 INFO:teuthology.orchestra.run.smithi088.stdout:(16/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.0 MB/s | 187 kB 00:00 2024-02-19T18:48:40.943 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-19T18:48:40.997 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-02-19T18:48:41.043 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-02-19T18:48:41.092 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-02-19T18:48:41.153 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-02-19T18:48:41.229 INFO:teuthology.orchestra.run.smithi088.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 104 kB/s | 43 kB 00:00 2024-02-19T18:48:41.337 INFO:teuthology.orchestra.run.smithi088.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 99 kB/s | 11 kB 00:00 2024-02-19T18:48:41.340 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-02-19T18:48:41.390 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-02-19T18:48:41.429 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-02-19T18:48:41.437 INFO:teuthology.orchestra.run.smithi088.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 286 kB/s | 172 kB 00:00 2024-02-19T18:48:41.463 INFO:teuthology.orchestra.run.smithi088.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 141 kB/s | 18 kB 00:00 2024-02-19T18:48:41.463 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-02-19T18:48:41.493 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-19T18:48:41.538 INFO:teuthology.orchestra.run.smithi088.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 231 kB/s | 23 kB 00:00 2024-02-19T18:48:41.570 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-02-19T18:48:41.571 INFO:teuthology.orchestra.run.smithi088.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 179 kB/s | 19 kB 00:00 2024-02-19T18:48:41.597 INFO:teuthology.orchestra.run.smithi088.stdout:(23/33): python3-cherrypy-18.6.1-2.el9.noarch.r 559 kB/s | 358 kB 00:00 2024-02-19T18:48:41.612 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-19T18:48:41.630 INFO:teuthology.orchestra.run.smithi088.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 213 kB/s | 20 kB 00:00 2024-02-19T18:48:41.646 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-02-19T18:48:41.686 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-02-19T18:48:41.706 INFO:teuthology.orchestra.run.smithi088.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 726 kB/s | 79 kB 00:00 2024-02-19T18:48:41.729 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-02-19T18:48:41.731 INFO:teuthology.orchestra.run.smithi088.stdout:(26/33): python3-logutils-0.3.5-21.el9.noarch.r 290 kB/s | 46 kB 00:00 2024-02-19T18:48:41.764 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-02-19T18:48:41.815 INFO:teuthology.orchestra.run.smithi088.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 151 kB/s | 16 kB 00:00 2024-02-19T18:48:41.840 INFO:teuthology.orchestra.run.smithi088.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-02-19T18:48:41.844 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-02-19T18:48:41.865 INFO:teuthology.orchestra.run.smithi088.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 672 kB/s | 90 kB 00:00 2024-02-19T18:48:41.886 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-02-19T18:48:41.959 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-02-19T18:48:41.959 INFO:teuthology.orchestra.run.smithi088.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 248 kB/s | 36 kB 00:00 2024-02-19T18:48:41.991 INFO:teuthology.orchestra.run.smithi088.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-02-19T18:48:42.033 INFO:teuthology.orchestra.run.smithi088.stdout:(32/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.5 MB/s | 427 kB 00:00 2024-02-19T18:48:42.066 INFO:teuthology.orchestra.run.smithi088.stdout:(33/33): python3-zc-lockfile-2.0-10.el9.noarch. 187 kB/s | 20 kB 00:00 2024-02-19T18:48:42.074 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:42.074 INFO:teuthology.orchestra.run.smithi088.stdout:Total 11 MB/s | 30 MB 00:02 2024-02-19T18:48:42.074 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:48:42.095 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-02-19T18:48:42.113 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:48:42.113 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:48:42.221 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-19T18:48:42.383 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-02-19T18:48:42.408 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-19T18:48:42.434 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-19T18:48:42.434 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:42.434 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-19T18:48:42.434 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-19T18:48:42.434 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-19T18:48:42.434 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:42.482 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:48:42.482 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:48:43.708 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:48:43.769 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-02-19T18:48:43.801 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-02-19T18:48:43.834 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-02-19T18:48:43.926 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-02-19T18:48:43.934 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-19T18:48:43.960 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-19T18:48:43.960 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:43.960 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-19T18:48:43.960 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-19T18:48:43.960 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-19T18:48:43.961 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:44.013 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-02-19T18:48:44.062 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-02-19T18:48:44.143 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-02-19T18:48:44.195 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 8/33 2024-02-19T18:48:44.234 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 9/33 2024-02-19T18:48:44.292 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ply-3.11-14.el9.noarch 10/33 2024-02-19T18:48:44.365 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 11/33 2024-02-19T18:48:44.410 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-19T18:48:44.433 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-19T18:48:44.433 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:44.433 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-19T18:48:44.433 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-19T18:48:44.433 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-19T18:48:44.433 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:44.551 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 12/33 2024-02-19T18:48:44.597 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 13/33 2024-02-19T18:48:44.635 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 14/33 2024-02-19T18:48:44.635 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-19T18:48:44.658 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-19T18:48:44.659 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:44.659 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-19T18:48:44.659 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-19T18:48:44.659 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-19T18:48:44.659 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:44.678 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 15/33 2024-02-19T18:48:44.716 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-19T18:48:44.716 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 16/33 2024-02-19T18:48:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 17/33 2024-02-19T18:48:44.827 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-19T18:48:44.860 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 19/33 2024-02-19T18:48:44.901 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 20/33 2024-02-19T18:48:44.936 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/33 2024-02-19T18:48:44.970 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 22/33 2024-02-19T18:48:45.061 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 23/33 2024-02-19T18:48:45.101 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 24/33 2024-02-19T18:48:45.183 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/33 2024-02-19T18:48:45.322 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 26/33 2024-02-19T18:48:45.452 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-19T18:48:45.616 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-02-19T18:48:45.648 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-19T18:48:45.673 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-02-19T18:48:45.673 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:45.673 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-02-19T18:48:45.673 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-19T18:48:45.673 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-02-19T18:48:45.674 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:46.563 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-19T18:48:46.563 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-02-19T18:48:46.563 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-02-19T18:48:46.563 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-02-19T18:48:46.563 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-02-19T18:48:46.563 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-02-19T18:48:46.563 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-02-19T18:48:46.563 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-02-19T18:48:46.563 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-19T18:48:46.564 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-19T18:48:46.565 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-19T18:48:46.565 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-19T18:48:46.565 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-19T18:48:46.565 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-19T18:48:46.565 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-19T18:48:46.565 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-19T18:48:46.565 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-19T18:48:46.565 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-19T18:48:46.565 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-19T18:48:46.565 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-19T18:48:46.565 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-19T18:48:47.190 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-19T18:48:47.216 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-02-19T18:48:47.216 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:47.216 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-02-19T18:48:47.216 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-19T18:48:47.216 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-02-19T18:48:47.216 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:47.420 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-19T18:48:47.420 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:47.420 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:48:47.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:47.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:47.420 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-19T18:48:47.421 INFO:teuthology.orchestra.run.smithi132.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-ply-3.11-14.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-19T18:48:47.422 INFO:teuthology.orchestra.run.smithi132.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-19T18:48:47.423 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:47.423 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:48:47.666 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-19T18:48:47.689 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-02-19T18:48:47.690 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:47.690 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-02-19T18:48:47.690 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-19T18:48:47.690 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-02-19T18:48:47.690 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:47.767 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-base 2024-02-19T18:48:47.891 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-19T18:48:47.914 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-02-19T18:48:47.914 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:47.914 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-02-19T18:48:47.914 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-19T18:48:47.915 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-02-19T18:48:47.915 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:47.980 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-19T18:48:48.362 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:46 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:48:48.475 INFO:teuthology.orchestra.run.smithi132.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:48:48.526 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:48:48.527 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-02-19T18:48:48.527 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:48:48.592 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install cephadm 2024-02-19T18:48:49.173 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:47 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:48:49.337 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:48:49.338 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:49.338 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-02-19T18:48:49.338 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:49.338 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:48:49.338 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-02-19T18:48:49.338 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:49.338 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:48:49.338 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:49.338 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-02-19T18:48:49.338 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:49.338 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 209 k 2024-02-19T18:48:49.339 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 200 k 2024-02-19T18:48:49.339 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:48:49.410 INFO:teuthology.orchestra.run.smithi132.stdout:cephadm-18.2.0-0.el9.noarch.rpm 2.9 MB/s | 209 kB 00:00 2024-02-19T18:48:49.411 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:49.411 INFO:teuthology.orchestra.run.smithi132.stdout:Total 2.8 MB/s | 209 kB 00:00 2024-02-19T18:48:49.411 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:48:49.416 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:48:49.416 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:48:49.418 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:48:49.419 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:48:49.491 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:48:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-02-19T18:48:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-02-19T18:48:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-02-19T18:48:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-02-19T18:48:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-02-19T18:48:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-02-19T18:48:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-02-19T18:48:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 7/33 2024-02-19T18:48:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 8/33 2024-02-19T18:48:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 9/33 2024-02-19T18:48:49.755 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 10/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 12/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 15/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-02-19T18:48:49.756 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-02-19T18:48:49.757 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-02-19T18:48:49.757 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-02-19T18:48:49.757 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-02-19T18:48:49.757 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-02-19T18:48:49.757 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-02-19T18:48:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-02-19T18:48:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-02-19T18:48:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-02-19T18:48:49.758 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-02-19T18:48:49.920 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-19T18:48:50.022 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-19T18:48:50.385 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-19T18:48:50.606 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-02-19T18:48:50.606 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:50.606 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:48:50.606 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:50.606 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:50.606 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:50.607 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-02-19T18:48:50.607 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:50.607 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:50.607 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-02-19T18:48:50.607 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-02-19T18:48:50.607 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-02-19T18:48:50.607 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-02-19T18:48:50.607 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-02-19T18:48:50.608 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-02-19T18:48:50.608 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-02-19T18:48:50.608 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-02-19T18:48:50.608 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-02-19T18:48:50.608 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-02-19T18:48:50.608 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-02-19T18:48:50.608 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako-1.1.4-6.el9.noarch 2024-02-19T18:48:50.609 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-02-19T18:48:50.609 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-02-19T18:48:50.609 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-02-19T18:48:50.609 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply-3.11-14.el9.noarch 2024-02-19T18:48:50.609 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend-3.1.0-2.el9.noarch 2024-02-19T18:48:50.609 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-02-19T18:48:50.609 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser-2.20-6.el9.noarch 2024-02-19T18:48:50.609 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz-2021.1-5.el9.noarch 2024-02-19T18:48:50.609 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-2.25.1-8.el9.noarch 2024-02-19T18:48:50.610 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-02-19T18:48:50.610 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml-0.10.2-6.el9.noarch 2024-02-19T18:48:50.610 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-02-19T18:48:50.610 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob-1.8.7-6.el9.noarch 2024-02-19T18:48:50.610 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-02-19T18:48:50.610 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-02-19T18:48:50.610 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:50.610 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:48:50.716 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-19T18:48:50.716 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:50.717 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:48:50.717 INFO:teuthology.orchestra.run.smithi132.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-02-19T18:48:50.717 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:50.717 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:48:50.893 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-base 2024-02-19T18:48:50.907 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-immutable-object-cache 2024-02-19T18:48:51.478 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:48 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:48:51.493 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:49 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:48:51.593 INFO:teuthology.orchestra.run.smithi088.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:48:51.644 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:48:51.645 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-02-19T18:48:51.646 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:48:51.664 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:48:51.665 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:51.665 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repo Size 2024-02-19T18:48:51.665 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:51.665 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:48:51.665 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-02-19T18:48:51.665 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:51.665 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:48:51.665 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:51.665 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-02-19T18:48:51.665 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:51.665 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 142 k 2024-02-19T18:48:51.666 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 399 k 2024-02-19T18:48:51.666 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:48:51.712 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install cephadm 2024-02-19T18:48:51.736 INFO:teuthology.orchestra.run.smithi132.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 2.0 MB/s | 142 kB 00:00 2024-02-19T18:48:51.736 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:51.737 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.9 MB/s | 142 kB 00:00 2024-02-19T18:48:51.737 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:48:51.744 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:48:51.744 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:48:51.796 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:48:51.796 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:48:51.933 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:48:52.004 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:48:52.027 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:48:52.028 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:52.028 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-19T18:48:52.028 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:52.286 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:49 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:48:52.449 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 209 k 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 200 k 2024-02-19T18:48:52.450 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:48:52.515 INFO:teuthology.orchestra.run.smithi088.stdout:cephadm-18.2.0-0.el9.noarch.rpm 3.2 MB/s | 209 kB 00:00 2024-02-19T18:48:52.515 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:52.515 INFO:teuthology.orchestra.run.smithi088.stdout:Total 3.1 MB/s | 209 kB 00:00 2024-02-19T18:48:52.515 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:48:52.520 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:48:52.520 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:48:52.523 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:48:52.523 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:48:52.595 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:48:53.007 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:48:53.008 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:53.008 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:48:53.008 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:53.008 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:53.008 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:48:53.032 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-19T18:48:53.126 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-19T18:48:53.307 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mgr 2024-02-19T18:48:53.529 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-19T18:48:53.833 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-02-19T18:48:53.833 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:53.833 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:48:53.833 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-02-19T18:48:53.834 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:53.834 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:48:53.898 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:51 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:48:54.011 INFO:teuthology.orchestra.run.smithi132.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:48:54.061 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-immutable-object-cache 2024-02-19T18:48:54.062 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:48:54.063 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-02-19T18:48:54.063 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:48:54.124 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mgr-dashboard 2024-02-19T18:48:54.651 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:51 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:48:54.704 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:52 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:48:54.817 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:48:54.818 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:54.818 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repo Size 2024-02-19T18:48:54.818 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:54.818 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:48:54.818 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-02-19T18:48:54.818 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:54.818 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:48:54.818 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:54.818 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-02-19T18:48:54.818 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:54.818 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 142 k 2024-02-19T18:48:54.819 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 399 k 2024-02-19T18:48:54.819 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:48:54.870 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:48:54.870 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:54.870 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-02-19T18:48:54.870 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:54.870 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout:Installing weak dependencies: 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout:Install 7 Packages 2024-02-19T18:48:54.871 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:54.872 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 2.0 M 2024-02-19T18:48:54.872 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 8.1 M 2024-02-19T18:48:54.872 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:48:54.886 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 2.1 MB/s | 142 kB 00:00 2024-02-19T18:48:54.887 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:54.887 INFO:teuthology.orchestra.run.smithi088.stdout:Total 2.0 MB/s | 142 kB 00:00 2024-02-19T18:48:54.887 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:48:54.894 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:48:54.894 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:48:54.945 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:48:54.946 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:48:55.057 INFO:teuthology.orchestra.run.smithi132.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 201 kB/s | 23 kB 00:00 2024-02-19T18:48:55.074 INFO:teuthology.orchestra.run.smithi132.stdout:(2/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 111 kB/s | 15 kB 00:00 2024-02-19T18:48:55.087 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:48:55.125 INFO:teuthology.orchestra.run.smithi132.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 9.3 MB/s | 1.7 MB 00:00 2024-02-19T18:48:55.137 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:48:55.160 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:48:55.160 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:48:55.160 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-02-19T18:48:55.160 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:55.308 INFO:teuthology.orchestra.run.smithi132.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 36 kB/s | 9.0 kB 00:00 2024-02-19T18:48:55.475 INFO:teuthology.orchestra.run.smithi132.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 88 kB/s | 31 kB 00:00 2024-02-19T18:48:55.525 INFO:teuthology.orchestra.run.smithi132.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 91 kB/s | 41 kB 00:00 2024-02-19T18:48:55.759 INFO:teuthology.orchestra.run.smithi132.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 418 kB/s | 188 kB 00:00 2024-02-19T18:48:55.760 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:55.761 INFO:teuthology.orchestra.run.smithi132.stdout:Total 2.2 MB/s | 2.0 MB 00:00 2024-02-19T18:48:55.761 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:48:55.769 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:48:55.769 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:48:55.842 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:48:55.843 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:48:56.153 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:48:56.163 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:48:56.163 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:56.163 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:48:56.163 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-02-19T18:48:56.164 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:56.164 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:48:56.210 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-19T18:48:56.245 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-19T18:48:56.299 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-19T18:48:56.331 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-19T18:48:56.359 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-02-19T18:48:56.415 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr 2024-02-19T18:48:56.587 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-02-19T18:48:56.621 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-19T18:48:57.011 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:53 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:48:57.124 INFO:teuthology.orchestra.run.smithi088.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:48:57.160 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-19T18:48:57.160 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-02-19T18:48:57.160 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-02-19T18:48:57.160 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-02-19T18:48:57.160 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-19T18:48:57.160 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-19T18:48:57.160 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-19T18:48:57.176 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:48:57.177 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-02-19T18:48:57.177 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:48:57.252 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-dashboard 2024-02-19T18:48:57.482 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-19T18:48:57.483 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:57.483 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:48:57.483 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-02-19T18:48:57.483 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-02-19T18:48:57.483 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-02-19T18:48:57.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-19T18:48:57.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-19T18:48:57.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-19T18:48:57.483 INFO:teuthology.orchestra.run.smithi132.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-19T18:48:57.483 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:57.483 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:48:57.693 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-19T18:48:57.827 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:54 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:48:57.994 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:48:57.995 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:57.995 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-02-19T18:48:57.995 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout:Installing weak dependencies: 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:48:57.996 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:48:57.997 INFO:teuthology.orchestra.run.smithi088.stdout:Install 7 Packages 2024-02-19T18:48:57.997 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:48:57.997 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 2.0 M 2024-02-19T18:48:57.997 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 8.1 M 2024-02-19T18:48:57.997 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:48:58.269 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:00:56 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:48:58.373 INFO:teuthology.orchestra.run.smithi088.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 195 kB/s | 23 kB 00:00 2024-02-19T18:48:58.398 INFO:teuthology.orchestra.run.smithi088.stdout:(2/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 102 kB/s | 15 kB 00:00 2024-02-19T18:48:58.435 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:48:58.436 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:58.436 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-02-19T18:48:58.436 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:58.436 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:48:58.436 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-02-19T18:48:58.436 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:58.437 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:48:58.438 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:48:58.438 INFO:teuthology.orchestra.run.smithi132.stdout:Install 12 Packages 2024-02-19T18:48:58.438 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:48:58.438 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 37 M 2024-02-19T18:48:58.438 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 209 M 2024-02-19T18:48:58.438 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:48:58.449 INFO:teuthology.orchestra.run.smithi088.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 8.8 MB/s | 1.7 MB 00:00 2024-02-19T18:48:58.699 INFO:teuthology.orchestra.run.smithi088.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 28 kB/s | 9.0 kB 00:00 2024-02-19T18:48:58.808 INFO:teuthology.orchestra.run.smithi088.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 100 kB/s | 41 kB 00:00 2024-02-19T18:48:58.833 INFO:teuthology.orchestra.run.smithi088.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 80 kB/s | 31 kB 00:00 2024-02-19T18:48:58.957 INFO:teuthology.orchestra.run.smithi132.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.0-0. 37 MB/s | 7.4 MB 00:00 2024-02-19T18:48:59.083 INFO:teuthology.orchestra.run.smithi088.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 492 kB/s | 188 kB 00:00 2024-02-19T18:48:59.086 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:48:59.087 INFO:teuthology.orchestra.run.smithi088.stdout:Total 1.8 MB/s | 2.0 MB 00:01 2024-02-19T18:48:59.087 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:48:59.091 INFO:teuthology.orchestra.run.smithi132.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 94 kB/s | 32 kB 00:00 2024-02-19T18:48:59.094 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:48:59.094 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:48:59.116 INFO:teuthology.orchestra.run.smithi132.stdout:(3/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 110 kB/s | 17 kB 00:00 2024-02-19T18:48:59.162 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:48:59.162 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:48:59.183 INFO:teuthology.orchestra.run.smithi132.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 373 kB/s | 34 kB 00:00 2024-02-19T18:48:59.450 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:48:59.510 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-02-19T18:48:59.543 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-02-19T18:48:59.589 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-02-19T18:48:59.608 INFO:teuthology.orchestra.run.smithi132.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.5 MB/s | 3.0 MB 00:00 2024-02-19T18:48:59.630 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-02-19T18:48:59.658 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-02-19T18:48:59.800 INFO:teuthology.orchestra.run.smithi132.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 6.9 MB/s | 4.7 MB 00:00 2024-02-19T18:48:59.886 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-02-19T18:48:59.919 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-19T18:48:59.975 INFO:teuthology.orchestra.run.smithi132.stdout:(7/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.0 MB/s | 4.8 MB 00:00 2024-02-19T18:49:00.001 INFO:teuthology.orchestra.run.smithi132.stdout:(8/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 557 kB/s | 217 kB 00:00 2024-02-19T18:49:00.474 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-02-19T18:49:00.474 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-02-19T18:49:00.474 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-02-19T18:49:00.474 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-02-19T18:49:00.474 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-02-19T18:49:00.474 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-02-19T18:49:00.474 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-02-19T18:49:00.797 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-02-19T18:49:00.797 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:00.797 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:49:00.797 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-02-19T18:49:00.797 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-02-19T18:49:00.797 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-02-19T18:49:00.797 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-02-19T18:49:00.797 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-02-19T18:49:00.797 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-02-19T18:49:00.797 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes-2.5.1-5.el9.noarch 2024-02-19T18:49:00.798 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:00.798 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:01.033 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-diskprediction-local 2024-02-19T18:49:01.301 INFO:teuthology.orchestra.run.smithi132.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-02-19T18:49:01.335 INFO:teuthology.orchestra.run.smithi132.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 591 kB/s | 802 kB 00:01 2024-02-19T18:49:01.360 INFO:teuthology.orchestra.run.smithi132.stdout:(11/12): libquadmath-11.4.1-3.el9.x86_64.rpm 143 kB/s | 194 kB 00:01 2024-02-19T18:49:01.610 INFO:teuthology.orchestra.run.smithi132.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 800 kB/s | 245 kB 00:00 2024-02-19T18:49:01.611 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:00:58 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:01.613 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:01.613 INFO:teuthology.orchestra.run.smithi132.stdout:Total 12 MB/s | 37 MB 00:03 2024-02-19T18:49:01.630 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:49:01.657 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:49:01.657 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:49:01.777 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:01.778 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:01.778 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-02-19T18:49:01.778 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:01.778 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran x86_64 11.4.1-3.el9 CentOS-BaseOS 802 k 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath x86_64 11.4.1-3.el9 CentOS-BaseOS 194 k 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-02-19T18:49:01.779 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:01.780 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:49:01.780 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:01.780 INFO:teuthology.orchestra.run.smithi088.stdout:Install 12 Packages 2024-02-19T18:49:01.780 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:01.780 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 37 M 2024-02-19T18:49:01.780 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 209 M 2024-02-19T18:49:01.780 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:49:02.077 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:49:02.078 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:49:02.156 INFO:teuthology.orchestra.run.smithi088.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.0-0. 32 MB/s | 7.4 MB 00:00 2024-02-19T18:49:02.181 INFO:teuthology.orchestra.run.smithi088.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 123 kB/s | 32 kB 00:00 2024-02-19T18:49:02.231 INFO:teuthology.orchestra.run.smithi088.stdout:(3/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 233 kB/s | 17 kB 00:00 2024-02-19T18:49:02.273 INFO:teuthology.orchestra.run.smithi088.stdout:(4/12): openblas-0.3.21-2.el9.x86_64.rpm 371 kB/s | 34 kB 00:00 2024-02-19T18:49:02.777 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:49:02.849 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-19T18:49:02.857 INFO:teuthology.orchestra.run.smithi088.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.2 MB/s | 3.0 MB 00:00 2024-02-19T18:49:02.881 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-19T18:49:02.972 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-19T18:49:03.017 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-19T18:49:03.193 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-19T18:49:03.225 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-19T18:49:03.257 INFO:teuthology.orchestra.run.smithi088.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 4.6 MB/s | 4.7 MB 00:01 2024-02-19T18:49:03.274 INFO:teuthology.orchestra.run.smithi088.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 523 kB/s | 217 kB 00:00 2024-02-19T18:49:03.300 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-19T18:49:03.332 INFO:teuthology.orchestra.run.smithi088.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.5 MB/s | 4.8 MB 00:01 2024-02-19T18:49:03.783 INFO:teuthology.orchestra.run.smithi088.stdout:(9/12): libquadmath-11.4.1-3.el9.x86_64.rpm 432 kB/s | 194 kB 00:00 2024-02-19T18:49:03.833 INFO:teuthology.orchestra.run.smithi088.stdout:(10/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.4 MB/s | 802 kB 00:00 2024-02-19T18:49:03.941 INFO:teuthology.orchestra.run.smithi088.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.5 MB/s | 245 kB 00:00 2024-02-19T18:49:03.977 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-19T18:49:04.079 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-19T18:49:04.392 INFO:teuthology.orchestra.run.smithi088.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-02-19T18:49:04.394 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:04.395 INFO:teuthology.orchestra.run.smithi088.stdout:Total 14 MB/s | 37 MB 00:02 2024-02-19T18:49:04.417 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:49:04.443 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:49:04.443 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:49:04.911 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:49:04.911 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:49:05.324 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-19T18:49:05.623 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:49:05.698 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-02-19T18:49:05.737 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-02-19T18:49:05.830 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-02-19T18:49:05.865 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-02-19T18:49:06.043 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-19T18:49:06.073 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-19T18:49:06.149 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-02-19T18:49:06.340 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-19T18:49:06.367 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-19T18:49:06.819 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-02-19T18:49:06.935 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-02-19T18:49:08.188 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-02-19T18:49:09.067 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-19T18:49:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-02-19T18:49:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-19T18:49:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-19T18:49:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-19T18:49:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-19T18:49:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-19T18:49:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-19T18:49:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-19T18:49:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-19T18:49:09.068 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-19T18:49:09.069 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-19T18:49:09.221 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-02-19T18:49:09.255 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-19T18:49:09.574 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-19T18:49:09.574 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:09.574 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:49:09.574 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:09.575 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:09.936 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mgr-rook 2024-02-19T18:49:10.530 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:08 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:10.697 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-19T18:49:10.699 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-19T18:49:10.700 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-19T18:49:10.700 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-19T18:49:10.700 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-19T18:49:10.700 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-19T18:49:10.700 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-19T18:49:10.700 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-19T18:49:10.700 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:10.700 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:49:10.700 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:10.700 INFO:teuthology.orchestra.run.smithi132.stdout:Install 13 Packages 2024-02-19T18:49:10.700 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:10.701 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 2.0 M 2024-02-19T18:49:10.701 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 26 M 2024-02-19T18:49:10.701 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:49:10.843 INFO:teuthology.orchestra.run.smithi132.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 644 kB/s | 47 kB 00:00 2024-02-19T18:49:10.968 INFO:teuthology.orchestra.run.smithi132.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 123 kB/s | 24 kB 00:00 2024-02-19T18:49:11.002 INFO:teuthology.orchestra.run.smithi132.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 78 kB/s | 18 kB 00:00 2024-02-19T18:49:11.094 INFO:teuthology.orchestra.run.smithi132.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 132 kB 00:00 2024-02-19T18:49:11.119 INFO:teuthology.orchestra.run.smithi132.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 763 kB/s | 174 kB 00:00 2024-02-19T18:49:11.144 INFO:teuthology.orchestra.run.smithi132.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.4 MB/s | 211 kB 00:00 2024-02-19T18:49:11.170 INFO:teuthology.orchestra.run.smithi132.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 567 kB/s | 43 kB 00:00 2024-02-19T18:49:11.687 INFO:teuthology.orchestra.run.smithi132.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 57 kB/s | 32 kB 00:00 2024-02-19T18:49:11.712 INFO:teuthology.orchestra.run.smithi132.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 26 kB/s | 15 kB 00:00 2024-02-19T18:49:11.937 INFO:teuthology.orchestra.run.smithi132.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 263 kB/s | 59 kB 00:00 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-02-19T18:49:11.984 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-02-19T18:49:12.146 INFO:teuthology.orchestra.run.smithi132.stdout:(11/13): python3-google-auth-2.27.0-1.el9.noarc 208 kB/s | 203 kB 00:00 2024-02-19T18:49:12.171 INFO:teuthology.orchestra.run.smithi132.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 384 kB/s | 90 kB 00:00 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-0.3.21-2.el9.x86_64 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-02-19T18:49:12.397 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-02-19T18:49:12.398 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-02-19T18:49:12.398 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-02-19T18:49:12.398 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-02-19T18:49:12.398 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:12.398 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:12.798 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-rook 2024-02-19T18:49:13.389 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:10 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:13.438 INFO:teuthology.orchestra.run.smithi132.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 597 kB/s | 1.0 MB 00:01 2024-02-19T18:49:13.444 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:13.444 INFO:teuthology.orchestra.run.smithi132.stdout:Total 763 kB/s | 2.0 MB 00:02 2024-02-19T18:49:13.444 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:49:13.459 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:49:13.459 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:49:13.555 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:13.556 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:13.556 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-02-19T18:49:13.556 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:13.556 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:49:13.556 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth noarch 1:2.27.0-1.el9 epel 203 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:13.557 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:49:13.558 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:13.558 INFO:teuthology.orchestra.run.smithi088.stdout:Install 13 Packages 2024-02-19T18:49:13.558 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:13.558 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 2.0 M 2024-02-19T18:49:13.558 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 26 M 2024-02-19T18:49:13.558 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:49:13.562 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:49:13.563 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:49:13.773 INFO:teuthology.orchestra.run.smithi088.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 842 kB/s | 47 kB 00:00 2024-02-19T18:49:13.898 INFO:teuthology.orchestra.run.smithi088.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 134 kB/s | 24 kB 00:00 2024-02-19T18:49:13.932 INFO:teuthology.orchestra.run.smithi088.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 84 kB/s | 18 kB 00:00 2024-02-19T18:49:14.040 INFO:teuthology.orchestra.run.smithi088.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 932 kB/s | 132 kB 00:00 2024-02-19T18:49:14.088 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:49:14.107 INFO:teuthology.orchestra.run.smithi088.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 607 kB/s | 174 kB 00:00 2024-02-19T18:49:14.132 INFO:teuthology.orchestra.run.smithi088.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.0 MB/s | 211 kB 00:00 2024-02-19T18:49:14.158 INFO:teuthology.orchestra.run.smithi088.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 365 kB/s | 43 kB 00:00 2024-02-19T18:49:14.268 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-19T18:49:14.322 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-19T18:49:14.375 INFO:teuthology.orchestra.run.smithi088.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 69 kB/s | 15 kB 00:00 2024-02-19T18:49:14.378 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-19T18:49:14.408 INFO:teuthology.orchestra.run.smithi088.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 107 kB/s | 32 kB 00:00 2024-02-19T18:49:14.415 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-19T18:49:14.456 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-19T18:49:14.558 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-19T18:49:14.583 INFO:teuthology.orchestra.run.smithi088.stdout:(10/13): python3-google-auth-2.27.0-1.el9.noarc 477 kB/s | 203 kB 00:00 2024-02-19T18:49:14.609 INFO:teuthology.orchestra.run.smithi088.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 295 kB/s | 59 kB 00:00 2024-02-19T18:49:14.666 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-19T18:49:14.724 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-19T18:49:14.725 INFO:teuthology.orchestra.run.smithi088.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 632 kB/s | 90 kB 00:00 2024-02-19T18:49:14.834 INFO:teuthology.orchestra.run.smithi088.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.2 MB/s | 1.0 MB 00:00 2024-02-19T18:49:14.837 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:14.837 INFO:teuthology.orchestra.run.smithi088.stdout:Total 1.6 MB/s | 2.0 MB 00:01 2024-02-19T18:49:14.837 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:49:14.850 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:49:14.850 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:49:14.947 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:49:14.948 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:49:15.445 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-19T18:49:15.461 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:49:15.497 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-19T18:49:15.530 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-19T18:49:15.576 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-19T18:49:15.601 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-19T18:49:15.629 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-02-19T18:49:15.693 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-02-19T18:49:15.748 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-02-19T18:49:15.786 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-02-19T18:49:15.827 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-02-19T18:49:15.945 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-02-19T18:49:16.055 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-google-auth-1:2.27.0-1.el9.noarch 7/13 2024-02-19T18:49:16.104 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-02-19T18:49:16.435 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-19T18:49:16.435 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-02-19T18:49:16.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-19T18:49:16.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-19T18:49:16.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-19T18:49:16.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-19T18:49:16.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-19T18:49:16.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-19T18:49:16.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-19T18:49:16.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-19T18:49:16.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-19T18:49:16.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-19T18:49:16.436 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-19T18:49:16.813 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-19T18:49:16.814 INFO:teuthology.orchestra.run.smithi132.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-19T18:49:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-19T18:49:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-19T18:49:16.815 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:16.815 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:16.823 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-02-19T18:49:16.875 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-02-19T18:49:16.909 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-02-19T18:49:16.955 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-02-19T18:49:16.988 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-19T18:49:17.068 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-mgr-cephadm 2024-02-19T18:49:17.681 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:15 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:17.776 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-02-19T18:49:17.776 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-02-19T18:49:17.776 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-02-19T18:49:17.777 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-02-19T18:49:17.777 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-02-19T18:49:17.777 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-02-19T18:49:17.777 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-02-19T18:49:17.777 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-02-19T18:49:17.777 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-02-19T18:49:17.777 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-02-19T18:49:17.777 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-google-auth-1:2.27.0-1.el9.noarch 10/13 2024-02-19T18:49:17.777 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-02-19T18:49:17.777 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-02-19T18:49:17.850 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:17.851 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:17.851 INFO:teuthology.orchestra.run.smithi132.stdout: Package Arch Version Repository Size 2024-02-19T18:49:17.851 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:17.851 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:49:17.851 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-02-19T18:49:17.851 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-02-19T18:49:17.851 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-19T18:49:17.851 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-19T18:49:17.851 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-19T18:49:17.851 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-19T18:49:17.851 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-19T18:49:17.852 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:17.852 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:49:17.852 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:17.852 INFO:teuthology.orchestra.run.smithi132.stdout:Install 6 Packages 2024-02-19T18:49:17.852 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:17.852 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 6.8 M 2024-02-19T18:49:17.853 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 33 M 2024-02-19T18:49:17.853 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:49:18.153 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth-1:2.27.0-1.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa-4.9-2.el9.noarch 2024-02-19T18:49:18.154 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-02-19T18:49:18.155 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:18.155 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:18.452 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-mgr-cephadm 2024-02-19T18:49:18.615 INFO:teuthology.orchestra.run.smithi132.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 1.0 MB/s | 127 kB 00:00 2024-02-19T18:49:19.033 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:16 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:19.190 INFO:teuthology.orchestra.run.smithi132.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 326 kB/s | 227 kB 00:00 2024-02-19T18:49:19.197 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:19.198 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:19.198 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-02-19T18:49:19.198 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:19.198 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:49:19.198 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-02-19T18:49:19.198 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-02-19T18:49:19.199 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh noarch 2.13.2-4.el9 epel 548 k 2024-02-19T18:49:19.199 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-02-19T18:49:19.199 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2 noarch 2.11.3-5.el9 CentOS-AppStream 227 k 2024-02-19T18:49:19.199 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-02-19T18:49:19.199 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-02-19T18:49:19.199 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:19.199 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:49:19.199 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:19.199 INFO:teuthology.orchestra.run.smithi088.stdout:Install 6 Packages 2024-02-19T18:49:19.199 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:19.200 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 6.8 M 2024-02-19T18:49:19.200 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 33 M 2024-02-19T18:49:19.200 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:49:19.332 INFO:teuthology.orchestra.run.smithi132.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 6.9 MB/s | 5.8 MB 00:00 2024-02-19T18:49:19.794 INFO:teuthology.orchestra.run.smithi088.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 1.2 MB/s | 127 kB 00:00 2024-02-19T18:49:19.824 INFO:teuthology.orchestra.run.smithi132.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 91 kB/s | 58 kB 00:00 2024-02-19T18:49:19.857 INFO:teuthology.orchestra.run.smithi132.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 97 kB/s | 51 kB 00:00 2024-02-19T18:49:20.061 INFO:teuthology.orchestra.run.smithi088.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 608 kB/s | 227 kB 00:00 2024-02-19T18:49:20.269 INFO:teuthology.orchestra.run.smithi088.stdout:(3/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 1.1 MB/s | 548 kB 00:00 2024-02-19T18:49:20.344 INFO:teuthology.orchestra.run.smithi088.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 685 kB/s | 51 kB 00:00 2024-02-19T18:49:20.369 INFO:teuthology.orchestra.run.smithi088.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 187 kB/s | 58 kB 00:00 2024-02-19T18:49:20.458 INFO:teuthology.orchestra.run.smithi132.stdout:(6/6): python3-asyncssh-2.13.2-4.el9.noarch.rpm 297 kB/s | 548 kB 00:01 2024-02-19T18:49:20.462 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:20.463 INFO:teuthology.orchestra.run.smithi132.stdout:Total 2.6 MB/s | 6.8 MB 00:02 2024-02-19T18:49:20.463 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:49:20.473 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:49:20.473 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:49:20.568 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:49:20.568 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:49:20.778 INFO:teuthology.orchestra.run.smithi088.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.3 MB/s | 5.8 MB 00:01 2024-02-19T18:49:20.783 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:20.783 INFO:teuthology.orchestra.run.smithi088.stdout:Total 4.3 MB/s | 6.8 MB 00:01 2024-02-19T18:49:20.783 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:49:20.792 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:49:20.792 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:49:20.881 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:49:20.881 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:49:20.895 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:49:21.072 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-19T18:49:21.126 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-02-19T18:49:21.186 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:49:21.361 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-02-19T18:49:21.413 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-asyncssh-2.13.2-4.el9.noarch 2/6 2024-02-19T18:49:21.637 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-19T18:49:21.720 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-19T18:49:21.779 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-19T18:49:21.807 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-19T18:49:21.931 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-02-19T18:49:22.015 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-02-19T18:49:22.082 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-02-19T18:49:22.109 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-19T18:49:22.625 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-19T18:49:22.625 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-02-19T18:49:22.625 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-19T18:49:22.625 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-19T18:49:22.625 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-02-19T18:49:22.625 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-19T18:49:22.937 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-02-19T18:49:22.937 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-02-19T18:49:22.937 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-02-19T18:49:22.937 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-02-19T18:49:22.937 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-asyncssh-2.13.2-4.el9.noarch 4/6 2024-02-19T18:49:22.938 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-02-19T18:49:22.946 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-19T18:49:22.946 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:22.946 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:49:22.946 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-02-19T18:49:22.947 INFO:teuthology.orchestra.run.smithi132.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-19T18:49:22.947 INFO:teuthology.orchestra.run.smithi132.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-19T18:49:22.947 INFO:teuthology.orchestra.run.smithi132.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-19T18:49:22.947 INFO:teuthology.orchestra.run.smithi132.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-19T18:49:22.947 INFO:teuthology.orchestra.run.smithi132.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-19T18:49:22.947 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:22.947 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:23.243 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install ceph-fuse 2024-02-19T18:49:23.268 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-02-19T18:49:23.268 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:23.268 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:49:23.269 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-02-19T18:49:23.269 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh-2.13.2-4.el9.noarch 2024-02-19T18:49:23.269 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel-2.9.1-2.el9.noarch 2024-02-19T18:49:23.269 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-02-19T18:49:23.269 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-02-19T18:49:23.269 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-02-19T18:49:23.269 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:23.269 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:23.572 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-fuse 2024-02-19T18:49:23.826 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:21 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:23.992 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:23.992 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:23.992 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:23.992 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:23.992 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:49:23.992 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-02-19T18:49:23.993 INFO:teuthology.orchestra.run.smithi132.stdout:Installing dependencies: 2024-02-19T18:49:23.993 INFO:teuthology.orchestra.run.smithi132.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-19T18:49:23.993 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:23.993 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:49:23.993 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:23.993 INFO:teuthology.orchestra.run.smithi132.stdout:Install 2 Packages 2024-02-19T18:49:23.993 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:23.993 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 913 k 2024-02-19T18:49:23.994 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 2.7 M 2024-02-19T18:49:23.994 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:49:24.102 INFO:teuthology.orchestra.run.smithi132.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 7.5 MB/s | 835 kB 00:00 2024-02-19T18:49:24.152 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:21 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:24.318 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:24.318 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:24.319 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:24.319 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:24.319 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:49:24.319 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-02-19T18:49:24.319 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-02-19T18:49:24.319 INFO:teuthology.orchestra.run.smithi088.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-02-19T18:49:24.319 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:24.319 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:49:24.319 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:24.319 INFO:teuthology.orchestra.run.smithi088.stdout:Install 2 Packages 2024-02-19T18:49:24.319 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:24.320 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 913 k 2024-02-19T18:49:24.320 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 2.7 M 2024-02-19T18:49:24.320 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:49:24.327 INFO:teuthology.orchestra.run.smithi132.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 236 kB/s | 79 kB 00:00 2024-02-19T18:49:24.328 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:24.328 INFO:teuthology.orchestra.run.smithi132.stdout:Total 2.7 MB/s | 913 kB 00:00 2024-02-19T18:49:24.329 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:49:24.349 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:49:24.349 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:49:24.396 INFO:teuthology.orchestra.run.smithi088.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 11 MB/s | 835 kB 00:00 2024-02-19T18:49:24.429 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:49:24.430 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:49:24.605 INFO:teuthology.orchestra.run.smithi088.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 276 kB/s | 79 kB 00:00 2024-02-19T18:49:24.606 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:24.606 INFO:teuthology.orchestra.run.smithi088.stdout:Total 3.1 MB/s | 913 kB 00:00 2024-02-19T18:49:24.606 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:49:24.621 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:49:24.627 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:49:24.627 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:49:24.707 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:49:24.707 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:49:24.748 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-19T18:49:24.834 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-19T18:49:24.890 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:49:25.019 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-02-19T18:49:25.088 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-19T18:49:25.555 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-19T18:49:25.555 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-02-19T18:49:25.793 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-02-19T18:49:25.794 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-02-19T18:49:25.909 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-19T18:49:25.910 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:25.910 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:49:25.910 INFO:teuthology.orchestra.run.smithi132.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-19T18:49:25.910 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:25.910 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:26.123 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-02-19T18:49:26.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:26.123 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:49:26.124 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-02-19T18:49:26.124 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:26.124 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:26.225 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install librados-devel 2024-02-19T18:49:26.374 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install librados-devel 2024-02-19T18:49:26.824 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:24 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:26.965 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:23 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:26.989 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:26.990 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:26.990 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:26.990 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:26.990 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:49:26.990 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-02-19T18:49:26.990 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:26.991 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:49:26.991 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:26.991 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-02-19T18:49:26.991 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:26.991 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 126 k 2024-02-19T18:49:26.991 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 456 k 2024-02-19T18:49:26.992 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:49:27.057 INFO:teuthology.orchestra.run.smithi132.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.9 MB/s | 126 kB 00:00 2024-02-19T18:49:27.058 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:27.058 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.8 MB/s | 126 kB 00:00 2024-02-19T18:49:27.058 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:49:27.064 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:49:27.064 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:49:27.118 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:49:27.118 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:49:27.136 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:27.137 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:27.137 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:27.137 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:27.137 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:49:27.137 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-02-19T18:49:27.137 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:27.137 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:49:27.137 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:27.137 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-02-19T18:49:27.137 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:27.138 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 126 k 2024-02-19T18:49:27.138 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 456 k 2024-02-19T18:49:27.138 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:49:27.202 INFO:teuthology.orchestra.run.smithi088.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.9 MB/s | 126 kB 00:00 2024-02-19T18:49:27.203 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:27.203 INFO:teuthology.orchestra.run.smithi088.stdout:Total 1.9 MB/s | 126 kB 00:00 2024-02-19T18:49:27.203 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:49:27.208 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:49:27.208 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:49:27.260 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:49:27.264 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:49:27.264 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:49:27.375 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:27.405 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:49:27.508 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:27.830 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:27.997 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:28.114 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:28.114 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:28.114 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:49:28.114 INFO:teuthology.orchestra.run.smithi132.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-02-19T18:49:28.114 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:28.114 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:28.307 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:28.307 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:49:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-02-19T18:49:28.308 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:28.308 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:28.441 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install libcephfs2 2024-02-19T18:49:28.584 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install libcephfs2 2024-02-19T18:49:29.032 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:27 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:29.151 INFO:teuthology.orchestra.run.smithi132.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:29.180 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:26 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:29.205 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:29.206 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-02-19T18:49:29.206 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:29.284 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install libcephfs-devel 2024-02-19T18:49:29.298 INFO:teuthology.orchestra.run.smithi088.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:29.351 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:29.352 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-02-19T18:49:29.352 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:29.439 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install libcephfs-devel 2024-02-19T18:49:29.871 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:27 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:30.026 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:27 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:30.036 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:30.037 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:30.037 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:30.038 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:30.038 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:49:30.038 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-02-19T18:49:30.038 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:30.038 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:49:30.038 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:30.038 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-02-19T18:49:30.038 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:30.038 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 30 k 2024-02-19T18:49:30.038 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 137 k 2024-02-19T18:49:30.038 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:49:30.113 INFO:teuthology.orchestra.run.smithi132.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 407 kB/s | 30 kB 00:00 2024-02-19T18:49:30.114 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:30.114 INFO:teuthology.orchestra.run.smithi132.stdout:Total 399 kB/s | 30 kB 00:00 2024-02-19T18:49:30.114 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:49:30.118 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:49:30.118 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:49:30.153 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:49:30.154 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:49:30.194 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 30 k 2024-02-19T18:49:30.195 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 137 k 2024-02-19T18:49:30.196 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:49:30.245 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:49:30.264 INFO:teuthology.orchestra.run.smithi088.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 445 kB/s | 30 kB 00:00 2024-02-19T18:49:30.265 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:30.265 INFO:teuthology.orchestra.run.smithi088.stdout:Total 436 kB/s | 30 kB 00:00 2024-02-19T18:49:30.265 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:49:30.268 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:49:30.268 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:49:30.296 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:49:30.296 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:49:30.361 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:30.378 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:49:30.504 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:30.772 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:30.914 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:31.042 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:31.042 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:31.042 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:49:31.043 INFO:teuthology.orchestra.run.smithi132.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-02-19T18:49:31.043 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:31.043 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:31.200 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:31.200 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:31.203 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:49:31.203 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-02-19T18:49:31.204 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:31.204 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:31.246 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install librados2 2024-02-19T18:49:31.432 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install librados2 2024-02-19T18:49:31.830 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:29 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:31.942 INFO:teuthology.orchestra.run.smithi132.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:31.995 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:31.996 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-02-19T18:49:31.996 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:32.008 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:28 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:32.076 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install librbd1 2024-02-19T18:49:32.120 INFO:teuthology.orchestra.run.smithi088.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:32.173 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:32.174 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-02-19T18:49:32.174 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:32.253 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install librbd1 2024-02-19T18:49:32.660 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:30 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:32.774 INFO:teuthology.orchestra.run.smithi132.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:32.827 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:32.828 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-02-19T18:49:32.828 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:32.833 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:29 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:32.909 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-rados 2024-02-19T18:49:32.945 INFO:teuthology.orchestra.run.smithi088.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:32.996 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:32.997 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-02-19T18:49:32.997 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:33.078 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-rados 2024-02-19T18:49:33.490 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:31 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:33.603 INFO:teuthology.orchestra.run.smithi132.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:33.647 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:30 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:33.655 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:33.656 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-02-19T18:49:33.656 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:33.734 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-rgw 2024-02-19T18:49:33.760 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:33.812 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:33.813 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-02-19T18:49:33.813 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:33.896 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-rgw 2024-02-19T18:49:34.309 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:32 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:34.422 INFO:teuthology.orchestra.run.smithi132.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:34.468 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:31 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:34.474 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:34.475 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-02-19T18:49:34.475 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:34.550 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-cephfs 2024-02-19T18:49:34.582 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:34.634 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:34.635 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-02-19T18:49:34.636 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:34.721 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-cephfs 2024-02-19T18:49:35.133 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:33 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:35.246 INFO:teuthology.orchestra.run.smithi132.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:35.298 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:35.299 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-02-19T18:49:35.299 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:35.308 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:32 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:35.372 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install python3-rbd 2024-02-19T18:49:35.422 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:35.473 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-02-19T18:49:35.474 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:35.566 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install python3-rbd 2024-02-19T18:49:35.952 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:33 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:36.067 INFO:teuthology.orchestra.run.smithi132.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:36.120 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:36.120 INFO:teuthology.orchestra.run.smithi132.stdout:Nothing to do. 2024-02-19T18:49:36.121 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:36.145 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:33 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:36.208 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install rbd-fuse 2024-02-19T18:49:36.260 INFO:teuthology.orchestra.run.smithi088.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-02-19T18:49:36.313 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:Nothing to do. 2024-02-19T18:49:36.314 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:36.406 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install rbd-fuse 2024-02-19T18:49:36.814 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:34 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:36.990 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:33 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:36.994 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:36.994 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:36.995 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:36.995 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:36.995 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:49:36.995 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-02-19T18:49:36.995 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:36.995 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:49:36.995 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:36.995 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-02-19T18:49:36.995 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:36.996 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 86 k 2024-02-19T18:49:36.996 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 230 k 2024-02-19T18:49:36.996 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:49:37.058 INFO:teuthology.orchestra.run.smithi132.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.3 MB/s | 86 kB 00:00 2024-02-19T18:49:37.059 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:37.060 INFO:teuthology.orchestra.run.smithi132.stdout:Total 1.3 MB/s | 86 kB 00:00 2024-02-19T18:49:37.060 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:49:37.067 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:49:37.067 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:49:37.123 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:49:37.124 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:49:37.156 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:37.157 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:37.157 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:37.157 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:37.157 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:49:37.157 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-02-19T18:49:37.157 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:37.157 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:49:37.157 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:37.157 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-02-19T18:49:37.157 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:37.158 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 86 k 2024-02-19T18:49:37.158 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 230 k 2024-02-19T18:49:37.158 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:49:37.225 INFO:teuthology.orchestra.run.smithi088.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.2 MB/s | 86 kB 00:00 2024-02-19T18:49:37.225 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:37.226 INFO:teuthology.orchestra.run.smithi088.stdout:Total 1.2 MB/s | 86 kB 00:00 2024-02-19T18:49:37.226 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:49:37.232 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:49:37.232 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:49:37.286 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:49:37.287 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:49:37.290 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:49:37.392 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:37.422 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:49:37.540 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:37.802 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:37.981 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:38.069 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:38.069 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:38.069 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:49:38.069 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-02-19T18:49:38.070 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:38.070 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:38.281 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:38.281 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:38.281 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:49:38.281 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-02-19T18:49:38.281 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:38.281 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:38.333 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install rbd-mirror 2024-02-19T18:49:38.577 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install rbd-mirror 2024-02-19T18:49:38.926 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:36 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:39.101 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:39.102 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:39.102 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:39.102 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:39.102 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:49:39.102 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-19T18:49:39.102 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:39.102 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:49:39.102 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:39.102 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-02-19T18:49:39.103 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:39.103 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 3.0 M 2024-02-19T18:49:39.103 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 12 M 2024-02-19T18:49:39.103 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:49:39.162 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:36 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:39.204 INFO:teuthology.orchestra.run.smithi132.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 29 MB/s | 3.0 MB 00:00 2024-02-19T18:49:39.205 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:39.205 INFO:teuthology.orchestra.run.smithi132.stdout:Total 29 MB/s | 3.0 MB 00:00 2024-02-19T18:49:39.205 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:49:39.214 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:49:39.215 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:49:39.286 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:49:39.286 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:49:39.337 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:39.338 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:39.338 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:39.338 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:39.338 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:49:39.338 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-02-19T18:49:39.338 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:39.338 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:49:39.338 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:39.339 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-02-19T18:49:39.339 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:39.339 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 3.0 M 2024-02-19T18:49:39.339 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 12 M 2024-02-19T18:49:39.339 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:49:39.453 INFO:teuthology.orchestra.run.smithi088.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 26 MB/s | 3.0 MB 00:00 2024-02-19T18:49:39.453 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:39.454 INFO:teuthology.orchestra.run.smithi088.stdout:Total 26 MB/s | 3.0 MB 00:00 2024-02-19T18:49:39.454 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:49:39.463 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:49:39.463 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:49:39.533 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:49:39.533 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:49:39.700 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:49:39.772 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:39.796 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:39.797 INFO:teuthology.orchestra.run.smithi132.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:49:39.797 INFO:teuthology.orchestra.run.smithi132.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-19T18:49:39.797 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-19T18:49:39.797 INFO:teuthology.orchestra.run.smithi132.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-19T18:49:39.797 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:39.953 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:49:40.029 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:40.052 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:40.052 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-02-19T18:49:40.052 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-02-19T18:49:40.052 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-19T18:49:40.052 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-02-19T18:49:40.052 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:40.875 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:40.875 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:40.875 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:49:40.876 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-02-19T18:49:40.876 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:40.876 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:41.180 DEBUG:teuthology.orchestra.run.smithi132:> sudo yum -y install rbd-nbd 2024-02-19T18:49:41.256 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:41.256 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:41.256 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:49:41.256 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-02-19T18:49:41.256 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:41.256 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:41.530 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install rbd-nbd 2024-02-19T18:49:41.775 INFO:teuthology.orchestra.run.smithi132.stdout:Last metadata expiration check: 0:01:39 ago on Mon 19 Feb 2024 06:48:02 PM UTC. 2024-02-19T18:49:41.940 INFO:teuthology.orchestra.run.smithi132.stdout:Dependencies resolved. 2024-02-19T18:49:41.941 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:41.941 INFO:teuthology.orchestra.run.smithi132.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:41.941 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:41.941 INFO:teuthology.orchestra.run.smithi132.stdout:Installing: 2024-02-19T18:49:41.941 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-02-19T18:49:41.941 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:41.941 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction Summary 2024-02-19T18:49:41.941 INFO:teuthology.orchestra.run.smithi132.stdout:================================================================================ 2024-02-19T18:49:41.941 INFO:teuthology.orchestra.run.smithi132.stdout:Install 1 Package 2024-02-19T18:49:41.941 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:41.942 INFO:teuthology.orchestra.run.smithi132.stdout:Total download size: 169 k 2024-02-19T18:49:41.942 INFO:teuthology.orchestra.run.smithi132.stdout:Installed size: 490 k 2024-02-19T18:49:41.942 INFO:teuthology.orchestra.run.smithi132.stdout:Downloading Packages: 2024-02-19T18:49:42.000 INFO:teuthology.orchestra.run.smithi132.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 2.9 MB/s | 169 kB 00:00 2024-02-19T18:49:42.000 INFO:teuthology.orchestra.run.smithi132.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:42.001 INFO:teuthology.orchestra.run.smithi132.stdout:Total 2.8 MB/s | 169 kB 00:00 2024-02-19T18:49:42.001 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction check 2024-02-19T18:49:42.009 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction check succeeded. 2024-02-19T18:49:42.009 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction test 2024-02-19T18:49:42.064 INFO:teuthology.orchestra.run.smithi132.stdout:Transaction test succeeded. 2024-02-19T18:49:42.064 INFO:teuthology.orchestra.run.smithi132.stdout:Running transaction 2024-02-19T18:49:42.127 INFO:teuthology.orchestra.run.smithi088.stdout:Last metadata expiration check: 0:01:39 ago on Mon 19 Feb 2024 06:48:03 PM UTC. 2024-02-19T18:49:42.221 INFO:teuthology.orchestra.run.smithi132.stdout: Preparing : 1/1 2024-02-19T18:49:42.294 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-02-19T18:49:42.295 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout:Install 1 Package 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 169 k 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout:Installed size: 490 k 2024-02-19T18:49:42.296 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-02-19T18:49:42.332 INFO:teuthology.orchestra.run.smithi132.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:42.356 INFO:teuthology.orchestra.run.smithi088.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 2.8 MB/s | 169 kB 00:00 2024-02-19T18:49:42.357 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:49:42.357 INFO:teuthology.orchestra.run.smithi088.stdout:Total 2.7 MB/s | 169 kB 00:00 2024-02-19T18:49:42.357 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-02-19T18:49:42.365 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-02-19T18:49:42.365 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-02-19T18:49:42.421 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-02-19T18:49:42.421 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-02-19T18:49:42.578 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-02-19T18:49:42.679 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:42.783 INFO:teuthology.orchestra.run.smithi132.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:43.082 INFO:teuthology.orchestra.run.smithi132.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:43.082 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:43.082 INFO:teuthology.orchestra.run.smithi132.stdout:Installed: 2024-02-19T18:49:43.082 INFO:teuthology.orchestra.run.smithi132.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-02-19T18:49:43.083 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:49:43.083 INFO:teuthology.orchestra.run.smithi132.stdout:Complete! 2024-02-19T18:49:43.131 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:43.339 DEBUG:teuthology.parallel:result is None 2024-02-19T18:49:43.422 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-02-19T18:49:43.422 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:43.422 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-02-19T18:49:43.422 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-02-19T18:49:43.422 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:49:43.423 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-02-19T18:49:43.734 DEBUG:teuthology.parallel:result is None 2024-02-19T18:49:43.735 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-19T18:49:43.735 INFO:teuthology.packaging:ref: None 2024-02-19T18:49:43.735 INFO:teuthology.packaging:tag: v18.2.0 2024-02-19T18:49:43.735 INFO:teuthology.packaging:branch: None 2024-02-19T18:49:43.735 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:49:43.735 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-19T18:49:43.735 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-19T18:49:44.037 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-19T18:49:44.057 INFO:teuthology.orchestra.run.smithi088.stdout:18.2.0-0.el9 2024-02-19T18:49:44.058 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-02-19T18:49:44.058 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-02-19T18:49:44.060 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-02-19T18:49:44.060 INFO:teuthology.packaging:ref: None 2024-02-19T18:49:44.060 INFO:teuthology.packaging:tag: v18.2.0 2024-02-19T18:49:44.060 INFO:teuthology.packaging:branch: None 2024-02-19T18:49:44.060 INFO:teuthology.packaging:sha1: f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:49:44.060 INFO:teuthology.packaging:Tag 'v18.2.0' not found in https://git.ceph.com/ceph-ci; will also look in https://git.ceph.com/ceph 2024-02-19T18:49:44.061 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-02-19T18:49:44.202 DEBUG:teuthology.orchestra.run.smithi132:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-19T18:49:44.223 INFO:teuthology.orchestra.run.smithi132.stdout:18.2.0-0.el9 2024-02-19T18:49:44.223 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-02-19T18:49:44.223 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-02-19T18:49:44.225 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-19T18:49:44.225 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:49:44.225 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-19T18:49:44.252 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:49:44.252 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-19T18:49:44.292 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-19T18:49:44.292 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:49:44.293 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/daemon-helper 2024-02-19T18:49:44.319 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-19T18:49:44.386 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:49:44.386 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/daemon-helper 2024-02-19T18:49:44.414 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-19T18:49:44.481 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-19T18:49:44.481 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:49:44.481 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-19T18:49:44.508 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-19T18:49:44.575 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:49:44.575 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-19T18:49:44.602 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-19T18:49:44.669 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-19T18:49:44.669 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:49:44.669 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/stdin-killer 2024-02-19T18:49:44.696 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-19T18:49:44.763 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:49:44.763 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/usr/bin/stdin-killer 2024-02-19T18:49:44.790 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-19T18:49:44.857 INFO:teuthology.run_tasks:Running task print... 2024-02-19T18:49:44.867 INFO:teuthology.task.print:**** done install task... 2024-02-19T18:49:44.867 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-19T18:49:44.992 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.0', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275'} 2024-02-19T18:49:44.992 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.0 2024-02-19T18:49:44.992 INFO:tasks.cephadm:Cluster fsid is a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:49:44.992 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-19T18:49:44.993 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-02-19T18:49:44.993 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi088': '172.21.15.88', 'mon.smithi132': '172.21.15.132'} 2024-02-19T18:49:44.993 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-19T18:49:44.993 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname $(hostname -s) 2024-02-19T18:49:45.020 DEBUG:teuthology.orchestra.run.smithi132:> sudo hostname $(hostname -s) 2024-02-19T18:49:45.047 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-02-19T18:49:45.047 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:49:45.196 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76790/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1450-gf78a58c0', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1450.gf78a58c0'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-15 19:25:41.455235', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240215.160715', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'f78a58c0ffd401d1493058a1022c35f011d65275', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76792/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1450-gf78a58c0', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1450.gf78a58c0'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-15 19:17:06.436952', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-batrick-testing-20240215.160715', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-batrick-testing-20240215.160715/f78a58c0ffd401d1493058a1022c35f011d65275/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-19T18:49:45.421 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef, sha1 bb2b5a32fc53d75aa463fb74c73b7d0b0ada6320 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-02-19T18:49:45.422 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef/bb2b5a32fc53d75aa463fb74c73b7d0b0ada6320/centos/9/x86_64/flavors/default/cephadm 2024-02-19T18:49:45.423 DEBUG:teuthology.orchestra.run.smithi088:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/bb2b5a32fc53d75aa463fb74c73b7d0b0ada6320/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-19T18:49:46.213 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r--. 1 ubuntu ubuntu 214738 Feb 19 18:49 /home/ubuntu/cephtest/cephadm 2024-02-19T18:49:46.213 DEBUG:teuthology.orchestra.run.smithi132:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/bb2b5a32fc53d75aa463fb74c73b7d0b0ada6320/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-19T18:49:46.580 INFO:teuthology.orchestra.run.smithi132.stdout:-rw-r--r--. 1 ubuntu ubuntu 214738 Feb 19 18:49 /home/ubuntu/cephtest/cephadm 2024-02-19T18:49:46.580 DEBUG:teuthology.orchestra.run.smithi088:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-19T18:49:46.599 DEBUG:teuthology.orchestra.run.smithi132:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-19T18:49:46.628 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.0 on all hosts... 2024-02-19T18:49:46.628 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-02-19T18:49:46.641 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-02-19T18:49:47.432 INFO:teuthology.orchestra.run.smithi088.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-02-19T18:49:47.527 INFO:teuthology.orchestra.run.smithi132.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-02-19T18:50:09.599 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T18:50:09.600 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-19T18:50:09.600 INFO:teuthology.orchestra.run.smithi088.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-02-19T18:50:09.600 INFO:teuthology.orchestra.run.smithi088.stdout: "repo_digests": [ 2024-02-19T18:50:09.600 INFO:teuthology.orchestra.run.smithi088.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-02-19T18:50:09.600 INFO:teuthology.orchestra.run.smithi088.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-02-19T18:50:09.600 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-02-19T18:50:09.600 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T18:50:18.085 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-02-19T18:50:18.085 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-19T18:50:18.085 INFO:teuthology.orchestra.run.smithi132.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-02-19T18:50:18.085 INFO:teuthology.orchestra.run.smithi132.stdout: "repo_digests": [ 2024-02-19T18:50:18.085 INFO:teuthology.orchestra.run.smithi132.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-02-19T18:50:18.085 INFO:teuthology.orchestra.run.smithi132.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-02-19T18:50:18.085 INFO:teuthology.orchestra.run.smithi132.stdout: ] 2024-02-19T18:50:18.085 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-02-19T18:50:18.106 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /etc/ceph 2024-02-19T18:50:18.142 DEBUG:teuthology.orchestra.run.smithi132:> sudo mkdir -p /etc/ceph 2024-02-19T18:50:18.173 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 777 /etc/ceph 2024-02-19T18:50:18.211 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 777 /etc/ceph 2024-02-19T18:50:18.241 INFO:tasks.cephadm:Writing seed config... 2024-02-19T18:50:18.242 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-02-19T18:50:18.242 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-02-19T18:50:18.242 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-02-19T18:50:18.242 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-02-19T18:50:18.242 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-02-19T18:50:18.242 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-19T18:50:18.242 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-19T18:50:18.242 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-02-19T18:50:18.242 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-02-19T18:50:18.242 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-19T18:50:18.242 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-19T18:50:18.243 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-19T18:50:18.244 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-19T18:50:18.245 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-19T18:50:18.245 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:50:18.246 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-19T18:50:18.267 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = a60b4d3a-cf57-11ee-95bb-87774f69a715 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-19T18:50:18.268 DEBUG:teuthology.orchestra.run.smithi088:mon.smithi088> sudo journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi088.service 2024-02-19T18:50:18.310 INFO:tasks.cephadm:Bootstrapping... 2024-02-19T18:50:18.310 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 -v bootstrap --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.88 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-19T18:50:18.469 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-02-19T18:50:18.470 INFO:teuthology.orchestra.run.smithi088.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.0', '-v', 'bootstrap', '--fsid', 'a60b4d3a-cf57-11ee-95bb-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.88', '--skip-admin-label'] 2024-02-19T18:50:18.493 INFO:teuthology.orchestra.run.smithi088.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-19T18:50:18.494 INFO:teuthology.orchestra.run.smithi088.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-19T18:50:18.494 INFO:teuthology.orchestra.run.smithi088.stdout:Verifying podman|docker is present... 2024-02-19T18:50:18.518 INFO:teuthology.orchestra.run.smithi088.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-19T18:50:18.518 INFO:teuthology.orchestra.run.smithi088.stdout:Verifying lvm2 is present... 2024-02-19T18:50:18.518 INFO:teuthology.orchestra.run.smithi088.stdout:Verifying time synchronization is in place... 2024-02-19T18:50:18.525 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-19T18:50:18.525 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-19T18:50:18.531 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-19T18:50:18.531 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout inactive 2024-02-19T18:50:18.537 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout enabled 2024-02-19T18:50:18.543 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout active 2024-02-19T18:50:18.543 INFO:teuthology.orchestra.run.smithi088.stdout:Unit chronyd.service is enabled and running 2024-02-19T18:50:18.543 INFO:teuthology.orchestra.run.smithi088.stdout:Repeating the final host check... 2024-02-19T18:50:18.566 INFO:teuthology.orchestra.run.smithi088.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-19T18:50:18.566 INFO:teuthology.orchestra.run.smithi088.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-19T18:50:18.566 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl is present 2024-02-19T18:50:18.566 INFO:teuthology.orchestra.run.smithi088.stdout:lvcreate is present 2024-02-19T18:50:18.572 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-19T18:50:18.572 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-19T18:50:18.578 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-19T18:50:18.578 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout inactive 2024-02-19T18:50:18.584 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout enabled 2024-02-19T18:50:18.590 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout active 2024-02-19T18:50:18.590 INFO:teuthology.orchestra.run.smithi088.stdout:Unit chronyd.service is enabled and running 2024-02-19T18:50:18.590 INFO:teuthology.orchestra.run.smithi088.stdout:Host looks OK 2024-02-19T18:50:18.590 INFO:teuthology.orchestra.run.smithi088.stdout:Cluster fsid: a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:50:18.590 INFO:teuthology.orchestra.run.smithi088.stdout:Acquiring lock 140319208745184 on /run/cephadm/a60b4d3a-cf57-11ee-95bb-87774f69a715.lock 2024-02-19T18:50:18.590 INFO:teuthology.orchestra.run.smithi088.stdout:Lock 140319208745184 acquired on /run/cephadm/a60b4d3a-cf57-11ee-95bb-87774f69a715.lock 2024-02-19T18:50:18.591 INFO:teuthology.orchestra.run.smithi088.stdout:Verifying IP 172.21.15.88 port 3300 ... 2024-02-19T18:50:18.591 INFO:teuthology.orchestra.run.smithi088.stdout:Verifying IP 172.21.15.88 port 6789 ... 2024-02-19T18:50:18.591 INFO:teuthology.orchestra.run.smithi088.stdout:Base mon IP(s) is [172.21.15.88:3300, 172.21.15.88:6789], mon addrv is [v2:172.21.15.88:3300,v1:172.21.15.88:6789] 2024-02-19T18:50:18.594 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.88 metric 100 2024-02-19T18:50:18.594 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.88 metric 100 2024-02-19T18:50:18.596 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-19T18:50:18.596 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-19T18:50:18.596 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1773sec hoplimit 64 pref medium 2024-02-19T18:50:18.598 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-19T18:50:18.598 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-19T18:50:18.599 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-19T18:50:18.599 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-19T18:50:18.599 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2024-02-19T18:50:18.599 INFO:teuthology.orchestra.run.smithi088.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-19T18:50:18.600 INFO:teuthology.orchestra.run.smithi088.stdout:Mon IP `172.21.15.88` is in CIDR network `172.21.0.0/20` 2024-02-19T18:50:18.600 INFO:teuthology.orchestra.run.smithi088.stdout:Mon IP `172.21.15.88` is in CIDR network `172.21.0.0/20` 2024-02-19T18:50:18.600 INFO:teuthology.orchestra.run.smithi088.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-19T18:50:18.600 INFO:teuthology.orchestra.run.smithi088.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-19T18:50:18.600 INFO:teuthology.orchestra.run.smithi088.stdout:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-02-19T18:50:19.534 INFO:teuthology.orchestra.run.smithi088.stdout:/bin/podman: stdout dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-02-19T18:50:19.534 INFO:teuthology.orchestra.run.smithi088.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.0... 2024-02-19T18:50:19.534 INFO:teuthology.orchestra.run.smithi088.stdout:/bin/podman: stderr Getting image source signatures 2024-02-19T18:50:19.535 INFO:teuthology.orchestra.run.smithi088.stdout:/bin/podman: stderr Copying blob sha256:3bd20aeff60302f668275dc2005d10679ae56492967a3a5a54fd3dde85333aec 2024-02-19T18:50:19.535 INFO:teuthology.orchestra.run.smithi088.stdout:/bin/podman: stderr Copying blob sha256:46af8f5390d4e94fc57efb422ccb97bb53dfe5b948546bfc191b46557eb2dbd9 2024-02-19T18:50:19.535 INFO:teuthology.orchestra.run.smithi088.stdout:/bin/podman: stderr Copying config sha256:dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-02-19T18:50:19.535 INFO:teuthology.orchestra.run.smithi088.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-19T18:50:20.447 INFO:teuthology.orchestra.run.smithi088.stdout:ceph: stdout ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-02-19T18:50:20.447 INFO:teuthology.orchestra.run.smithi088.stdout:Ceph version: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-02-19T18:50:20.447 INFO:teuthology.orchestra.run.smithi088.stdout:Extracting ceph user uid/gid from container image... 2024-02-19T18:50:21.318 INFO:teuthology.orchestra.run.smithi088.stdout:stat: stdout 167 167 2024-02-19T18:50:21.318 INFO:teuthology.orchestra.run.smithi088.stdout:Creating initial keys... 2024-02-19T18:50:22.141 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph-authtool: stdout AQDtotNl26cEJBAAormjS1NLTgK+Hcr9B0nPLg== 2024-02-19T18:50:23.014 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph-authtool: stdout AQDuotNlskPvGxAATL/nW/yHIMbab+tcoIf19g== 2024-02-19T18:50:23.919 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph-authtool: stdout AQDvotNlf8zGERAAjucaGgdrHNIXKtCvs9lHTQ== 2024-02-19T18:50:23.919 INFO:teuthology.orchestra.run.smithi088.stdout:Creating initial monmap... 2024-02-19T18:50:24.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-19T18:50:24.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-02-19T18:50:24.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:50:24.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-19T18:50:24.759 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool for smithi088 [v2:172.21.15.88:3300,v1:172.21.15.88:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-19T18:50:24.759 INFO:teuthology.orchestra.run.smithi088.stdout:setting min_mon_release = pacific 2024-02-19T18:50:24.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/monmaptool: set fsid to a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:50:24.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-19T18:50:24.759 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:50:24.759 INFO:teuthology.orchestra.run.smithi088.stdout:Creating mon... 2024-02-19T18:50:25.697 INFO:teuthology.orchestra.run.smithi088.stdout:create mon.smithi088 on 2024-02-19T18:50:25.894 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-02-19T18:50:26.037 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-19T18:50:26.212 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715.target → /etc/systemd/system/ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715.target. 2024-02-19T18:50:26.213 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715.target → /etc/systemd/system/ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715.target. 2024-02-19T18:50:26.416 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi088 2024-02-19T18:50:26.416 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi088.service: Unit ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi088.service not loaded. 2024-02-19T18:50:26.579 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715.target.wants/ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi088.service → /etc/systemd/system/ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@.service. 2024-02-19T18:50:26.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:26 smithi088 podman[28873]: 2024-02-19 18:50:26.70369619 +0000 UTC m=+0.018661232 image pull dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 quay.io/ceph/ceph:v18.2.0 2024-02-19T18:50:27.062 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-19T18:50:27.062 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout disabled 2024-02-19T18:50:27.068 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-19T18:50:27.068 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout inactive 2024-02-19T18:50:27.069 INFO:teuthology.orchestra.run.smithi088.stdout:firewalld.service is not enabled 2024-02-19T18:50:27.069 INFO:teuthology.orchestra.run.smithi088.stdout:Not possible to enable service . firewalld.service is not available 2024-02-19T18:50:27.069 INFO:teuthology.orchestra.run.smithi088.stdout:Waiting for mon to start... 2024-02-19T18:50:27.069 INFO:teuthology.orchestra.run.smithi088.stdout:Waiting for mon... 2024-02-19T18:50:27.192 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:26 smithi088 podman[28873]: 2024-02-19 18:50:26.959514982 +0000 UTC m=+0.274480015 container init 7d05c4bb3ea08880a9bf6acb2e1c5530cfb76e259ba93e0e4a533fada6258885 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-19T18:50:27.192 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:26 smithi088 podman[28873]: 2024-02-19 18:50:26.962374257 +0000 UTC m=+0.277339290 container start 7d05c4bb3ea08880a9bf6acb2e1c5530cfb76e259ba93e0e4a533fada6258885 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, RELEASE=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212) 2024-02-19T18:50:27.193 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:27 smithi088 ceph-mon[28891]: mkfs a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:50:27.193 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:27 smithi088 ceph-mon[28891]: mon.smithi088 is new leader, mons smithi088 in quorum (ranks 0) 2024-02-19T18:50:27.193 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:27 smithi088 bash[28873]: 7d05c4bb3ea08880a9bf6acb2e1c5530cfb76e259ba93e0e4a533fada6258885 2024-02-19T18:50:27.193 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:27 smithi088 systemd[1]: Started Ceph mon.smithi088 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T18:50:28.125 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:28 smithi088 ceph-mon[28891]: mon.smithi088 is new leader, mons smithi088 in quorum (ranks 0) 2024-02-19T18:50:28.125 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:28 smithi088 ceph-mon[28891]: monmap e1: 1 mons at {smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0]} removed_ranks: {} 2024-02-19T18:50:28.125 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:28 smithi088 ceph-mon[28891]: fsmap 2024-02-19T18:50:28.125 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:28 smithi088 ceph-mon[28891]: osdmap e1: 0 total, 0 up, 0 in 2024-02-19T18:50:28.125 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:28 smithi088 ceph-mon[28891]: mgrmap e1: no daemons active 2024-02-19T18:50:28.125 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:28 smithi088 ceph-mon[28891]: from='client.? 172.21.15.88:0/471844657' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-19T18:50:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout cluster: 2024-02-19T18:50:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout id: a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:50:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-19T18:50:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout services: 2024-02-19T18:50:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi088 (age 0.524328s) 2024-02-19T18:50:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout data: 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout pgs: 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.472+0000 7fb9da846700 1 Processor -- start 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.472+0000 7fb9da846700 1 -- start start 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.472+0000 7fb9da846700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d410a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:28.134 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.472+0000 7fb9da846700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d4106370 con 0x7fb9d4079cb0 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.472+0000 7fb9d3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d410a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.472+0000 7fb9d3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d410a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45414/0 (socket says 172.21.15.88:45414) 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.472+0000 7fb9d3fff700 1 -- 172.21.15.88:0/1877962519 learned_addr learned my addr 172.21.15.88:0/1877962519 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.473+0000 7fb9d3fff700 1 -- 172.21.15.88:0/1877962519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d410aec0 con 0x7fb9d4079cb0 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.473+0000 7fb9d3fff700 1 --2- 172.21.15.88:0/1877962519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d410a980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40098a0 tx=0x7fb9c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae52e34ece5cdbe server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.473+0000 7fb9d2ffd700 1 -- 172.21.15.88:0/1877962519 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9c4004180 con 0x7fb9d4079cb0 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.473+0000 7fb9d2ffd700 1 -- 172.21.15.88:0/1877962519 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c40042e0 con 0x7fb9d4079cb0 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.473+0000 7fb9d2ffd700 1 -- 172.21.15.88:0/1877962519 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9c40036a0 con 0x7fb9d4079cb0 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.474+0000 7fb9da846700 1 -- 172.21.15.88:0/1877962519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 msgr2=0x7fb9d410a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.474+0000 7fb9da846700 1 --2- 172.21.15.88:0/1877962519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d410a980 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb9c40098a0 tx=0x7fb9c4009bb0 comp rx=0 tx=0).stop 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.474+0000 7fb9da846700 1 -- 172.21.15.88:0/1877962519 shutdown_connections 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.474+0000 7fb9da846700 1 --2- 172.21.15.88:0/1877962519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d410a980 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:28.135 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.474+0000 7fb9da846700 1 -- 172.21.15.88:0/1877962519 >> 172.21.15.88:0/1877962519 conn(0x7fb9d40ffed0 msgr2=0x7fb9d4102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.474+0000 7fb9da846700 1 -- 172.21.15.88:0/1877962519 shutdown_connections 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.474+0000 7fb9da846700 1 -- 172.21.15.88:0/1877962519 wait complete. 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.474+0000 7fb9da846700 1 Processor -- start 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.474+0000 7fb9da846700 1 -- start start 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.474+0000 7fb9da846700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d419b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.474+0000 7fb9da846700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d419b5e0 con 0x7fb9d4079cb0 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9d3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d419b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9d3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d419b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45420/0 (socket says 172.21.15.88:45420) 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9d3fff700 1 -- 172.21.15.88:0/471844657 learned_addr learned my addr 172.21.15.88:0/471844657 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9d3fff700 1 -- 172.21.15.88:0/471844657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c4009550 con 0x7fb9d4079cb0 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9d3fff700 1 --2- 172.21.15.88:0/471844657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d419b0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4004000 tx=0x7fb9c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:28.136 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9d17fa700 1 -- 172.21.15.88:0/471844657 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9c4003ce0 con 0x7fb9d4079cb0 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9d17fa700 1 -- 172.21.15.88:0/471844657 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c4020070 con 0x7fb9d4079cb0 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9da846700 1 -- 172.21.15.88:0/471844657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d419b7e0 con 0x7fb9d4079cb0 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9d17fa700 1 -- 172.21.15.88:0/471844657 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb9c4011630 con 0x7fb9d4079cb0 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9da846700 1 -- 172.21.15.88:0/471844657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d419bc80 con 0x7fb9d4079cb0 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9d17fa700 1 -- 172.21.15.88:0/471844657 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb9c4022030 con 0x7fb9d4079cb0 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.475+0000 7fb9d17fa700 1 -- 172.21.15.88:0/471844657 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb9c4025d40 con 0x7fb9d4079cb0 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.476+0000 7fb9da846700 1 -- 172.21.15.88:0/471844657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d4194eb0 con 0x7fb9d4079cb0 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.479+0000 7fb9d17fa700 1 -- 172.21.15.88:0/471844657 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb9c4025480 con 0x7fb9d4079cb0 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.549+0000 7fb9da846700 1 -- 172.21.15.88:0/471844657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fb9d402cc30 con 0x7fb9d4079cb0 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.549+0000 7fb9d17fa700 1 -- 172.21.15.88:0/471844657 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fb9c4036070 con 0x7fb9d4079cb0 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.550+0000 7fb9da846700 1 -- 172.21.15.88:0/471844657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 msgr2=0x7fb9d419b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.550+0000 7fb9da846700 1 --2- 172.21.15.88:0/471844657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d419b0a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4004000 tx=0x7fb9c4004750 comp rx=0 tx=0).stop 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.550+0000 7fb9da846700 1 -- 172.21.15.88:0/471844657 shutdown_connections 2024-02-19T18:50:28.137 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.550+0000 7fb9da846700 1 --2- 172.21.15.88:0/471844657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9d4079cb0 0x7fb9d419b0a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:28.138 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.550+0000 7fb9da846700 1 -- 172.21.15.88:0/471844657 >> 172.21.15.88:0/471844657 conn(0x7fb9d40ffed0 msgr2=0x7fb9d41017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:28.138 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.550+0000 7fb9da846700 1 -- 172.21.15.88:0/471844657 shutdown_connections 2024-02-19T18:50:28.138 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:27.550+0000 7fb9da846700 1 -- 172.21.15.88:0/471844657 wait complete. 2024-02-19T18:50:28.138 INFO:teuthology.orchestra.run.smithi088.stdout:mon is available 2024-02-19T18:50:28.138 INFO:teuthology.orchestra.run.smithi088.stdout:Assimilating anything we can from ceph.conf... 2024-02-19T18:50:29.048 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:29.049 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout [global] 2024-02-19T18:50:29.049 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout fsid = a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:50:29.049 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.88:3300,v1:172.21.15.88:6789] 2024-02-19T18:50:29.049 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-19T18:50:29.049 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-19T18:50:29.049 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-19T18:50:29.049 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-19T18:50:29.049 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:29.049 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-19T18:50:29.049 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-19T18:50:29.049 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout [osd] 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.480+0000 7f40676ea700 1 Processor -- start 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.481+0000 7f40676ea700 1 -- start start 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.481+0000 7f40676ea700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f40601062b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.481+0000 7f40676ea700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40601067f0 con 0x7f4060105ea0 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.481+0000 7f4065486700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f40601062b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.481+0000 7f4065486700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f40601062b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45426/0 (socket says 172.21.15.88:45426) 2024-02-19T18:50:29.050 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.481+0000 7f4065486700 1 -- 172.21.15.88:0/1133778253 learned_addr learned my addr 172.21.15.88:0/1133778253 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.481+0000 7f4065486700 1 -- 172.21.15.88:0/1133778253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4060106930 con 0x7f4060105ea0 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.481+0000 7f4065486700 1 --2- 172.21.15.88:0/1133778253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f40601062b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f405c0098a0 tx=0x7f405c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9df31f151157e19 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.482+0000 7f4057fff700 1 -- 172.21.15.88:0/1133778253 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f405c004180 con 0x7f4060105ea0 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.482+0000 7f4057fff700 1 -- 172.21.15.88:0/1133778253 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c0042e0 con 0x7f4060105ea0 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.482+0000 7f4057fff700 1 -- 172.21.15.88:0/1133778253 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f405c0036a0 con 0x7f4060105ea0 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.482+0000 7f40676ea700 1 -- 172.21.15.88:0/1133778253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 msgr2=0x7f40601062b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.482+0000 7f40676ea700 1 --2- 172.21.15.88:0/1133778253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f40601062b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f405c0098a0 tx=0x7f405c009bb0 comp rx=0 tx=0).stop 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.482+0000 7f40676ea700 1 -- 172.21.15.88:0/1133778253 shutdown_connections 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.482+0000 7f40676ea700 1 --2- 172.21.15.88:0/1133778253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f40601062b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.482+0000 7f40676ea700 1 -- 172.21.15.88:0/1133778253 >> 172.21.15.88:0/1133778253 conn(0x7f40600ff670 msgr2=0x7f4060101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.482+0000 7f40676ea700 1 -- 172.21.15.88:0/1133778253 shutdown_connections 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.482+0000 7f40676ea700 1 -- 172.21.15.88:0/1133778253 wait complete. 2024-02-19T18:50:29.051 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.483+0000 7f40676ea700 1 Processor -- start 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.483+0000 7f40676ea700 1 -- start start 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.483+0000 7f40676ea700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f4060197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.483+0000 7f40676ea700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4060198280 con 0x7f4060105ea0 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.483+0000 7f4065486700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f4060197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.483+0000 7f4065486700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f4060197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45442/0 (socket says 172.21.15.88:45442) 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.483+0000 7f4065486700 1 -- 172.21.15.88:0/3387252564 learned_addr learned my addr 172.21.15.88:0/3387252564 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.483+0000 7f4065486700 1 -- 172.21.15.88:0/3387252564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f405c009550 con 0x7f4060105ea0 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.484+0000 7f4065486700 1 --2- 172.21.15.88:0/3387252564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f4060197d40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4060066ac0 tx=0x7f405c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.484+0000 7f40567fc700 1 -- 172.21.15.88:0/3387252564 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f405c003ce0 con 0x7f4060105ea0 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.484+0000 7f40567fc700 1 -- 172.21.15.88:0/3387252564 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c020070 con 0x7f4060105ea0 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.484+0000 7f40676ea700 1 -- 172.21.15.88:0/3387252564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4060198480 con 0x7f4060105ea0 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.484+0000 7f40567fc700 1 -- 172.21.15.88:0/3387252564 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f405c011630 con 0x7f4060105ea0 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.484+0000 7f40676ea700 1 -- 172.21.15.88:0/3387252564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4060198920 con 0x7f4060105ea0 2024-02-19T18:50:29.052 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.485+0000 7f40567fc700 1 -- 172.21.15.88:0/3387252564 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f405c022030 con 0x7f4060105ea0 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.485+0000 7f40676ea700 1 -- 172.21.15.88:0/3387252564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4060191590 con 0x7f4060105ea0 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.485+0000 7f40567fc700 1 -- 172.21.15.88:0/3387252564 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f405c025480 con 0x7f4060105ea0 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.487+0000 7f40567fc700 1 -- 172.21.15.88:0/3387252564 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f405c025980 con 0x7f4060105ea0 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.557+0000 7f40676ea700 1 -- 172.21.15.88:0/3387252564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f406002ce30 con 0x7f4060105ea0 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.561+0000 7f40567fc700 1 -- 172.21.15.88:0/3387252564 <== mon.0 v2:172.21.15.88:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f405c01b450 con 0x7f4060105ea0 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.561+0000 7f40567fc700 1 -- 172.21.15.88:0/3387252564 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f405c037540 con 0x7f4060105ea0 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.562+0000 7f40676ea700 1 -- 172.21.15.88:0/3387252564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 msgr2=0x7f4060197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.562+0000 7f40676ea700 1 --2- 172.21.15.88:0/3387252564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f4060197d40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4060066ac0 tx=0x7f405c004750 comp rx=0 tx=0).stop 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.562+0000 7f40676ea700 1 -- 172.21.15.88:0/3387252564 shutdown_connections 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.562+0000 7f40676ea700 1 --2- 172.21.15.88:0/3387252564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4060105ea0 0x7f4060197d40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.562+0000 7f40676ea700 1 -- 172.21.15.88:0/3387252564 >> 172.21.15.88:0/3387252564 conn(0x7f40600ff670 msgr2=0x7f406018c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.562+0000 7f40676ea700 1 -- 172.21.15.88:0/3387252564 shutdown_connections 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:28.562+0000 7f40676ea700 1 -- 172.21.15.88:0/3387252564 wait complete. 2024-02-19T18:50:29.053 INFO:teuthology.orchestra.run.smithi088.stdout:Generating new minimal ceph.conf... 2024-02-19T18:50:29.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:29 smithi088 ceph-mon[28891]: from='client.? 172.21.15.88:0/3387252564' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-19T18:50:29.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:29 smithi088 ceph-mon[28891]: from='client.? 172.21.15.88:0/3387252564' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-19T18:50:30.120 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.451+0000 7f619949e700 1 Processor -- start 2024-02-19T18:50:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.451+0000 7f619949e700 1 -- start start 2024-02-19T18:50:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.451+0000 7f619949e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f61941040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.451+0000 7f619949e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6194104600 con 0x7f6194103cb0 2024-02-19T18:50:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.451+0000 7f6192ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f61941040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.451+0000 7f6192ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f61941040c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45456/0 (socket says 172.21.15.88:45456) 2024-02-19T18:50:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.451+0000 7f6192ffd700 1 -- 172.21.15.88:0/3715499571 learned_addr learned my addr 172.21.15.88:0/3715499571 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.451+0000 7f6192ffd700 1 -- 172.21.15.88:0/3715499571 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6194104740 con 0x7f6194103cb0 2024-02-19T18:50:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.452+0000 7f6192ffd700 1 --2- 172.21.15.88:0/3715499571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f61941040c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61840098a0 tx=0x7f6184009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66583e4d3d651a7b server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.452+0000 7f6191ffb700 1 -- 172.21.15.88:0/3715499571 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6184004180 con 0x7f6194103cb0 2024-02-19T18:50:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.452+0000 7f6191ffb700 1 -- 172.21.15.88:0/3715499571 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61840042e0 con 0x7f6194103cb0 2024-02-19T18:50:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.452+0000 7f6191ffb700 1 -- 172.21.15.88:0/3715499571 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6184011a50 con 0x7f6194103cb0 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.452+0000 7f619949e700 1 -- 172.21.15.88:0/3715499571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 msgr2=0x7f61941040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.452+0000 7f619949e700 1 --2- 172.21.15.88:0/3715499571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f61941040c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f61840098a0 tx=0x7f6184009bb0 comp rx=0 tx=0).stop 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.452+0000 7f619949e700 1 -- 172.21.15.88:0/3715499571 shutdown_connections 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.452+0000 7f619949e700 1 --2- 172.21.15.88:0/3715499571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f61941040c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.452+0000 7f619949e700 1 -- 172.21.15.88:0/3715499571 >> 172.21.15.88:0/3715499571 conn(0x7f61940ff670 msgr2=0x7f6194101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.452+0000 7f619949e700 1 -- 172.21.15.88:0/3715499571 shutdown_connections 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.452+0000 7f619949e700 1 -- 172.21.15.88:0/3715499571 wait complete. 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.453+0000 7f619949e700 1 Processor -- start 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.453+0000 7f619949e700 1 -- start start 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.453+0000 7f619949e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f619419b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.453+0000 7f619949e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619419bee0 con 0x7f6194103cb0 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.453+0000 7f6192ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f619419b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.453+0000 7f6192ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f619419b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45462/0 (socket says 172.21.15.88:45462) 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.453+0000 7f6192ffd700 1 -- 172.21.15.88:0/2564410662 learned_addr learned my addr 172.21.15.88:0/2564410662 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.454+0000 7f6192ffd700 1 -- 172.21.15.88:0/2564410662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6184009550 con 0x7f6194103cb0 2024-02-19T18:50:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.454+0000 7f6192ffd700 1 --2- 172.21.15.88:0/2564410662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f619419b9a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6184003de0 tx=0x7f6184011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.454+0000 7f617bfff700 1 -- 172.21.15.88:0/2564410662 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6184003930 con 0x7f6194103cb0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.454+0000 7f619949e700 1 -- 172.21.15.88:0/2564410662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619419c0e0 con 0x7f6194103cb0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.454+0000 7f617bfff700 1 -- 172.21.15.88:0/2564410662 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6184003a90 con 0x7f6194103cb0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.454+0000 7f617bfff700 1 -- 172.21.15.88:0/2564410662 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61840196c0 con 0x7f6194103cb0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.454+0000 7f619949e700 1 -- 172.21.15.88:0/2564410662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619419c580 con 0x7f6194103cb0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.454+0000 7f617bfff700 1 -- 172.21.15.88:0/2564410662 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6184019c50 con 0x7f6194103cb0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.454+0000 7f617bfff700 1 -- 172.21.15.88:0/2564410662 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f618402c3e0 con 0x7f6194103cb0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.455+0000 7f619949e700 1 -- 172.21.15.88:0/2564410662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6194062380 con 0x7f6194103cb0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.458+0000 7f617bfff700 1 -- 172.21.15.88:0/2564410662 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f618401abc0 con 0x7f6194103cb0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.527+0000 7f619949e700 1 -- 172.21.15.88:0/2564410662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f619419f100 con 0x7f6194103cb0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.527+0000 7f617bfff700 1 -- 172.21.15.88:0/2564410662 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f618401a430 con 0x7f6194103cb0 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.528+0000 7f619949e700 1 -- 172.21.15.88:0/2564410662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 msgr2=0x7f619419b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.528+0000 7f619949e700 1 --2- 172.21.15.88:0/2564410662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f619419b9a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6184003de0 tx=0x7f6184011ab0 comp rx=0 tx=0).stop 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.528+0000 7f619949e700 1 -- 172.21.15.88:0/2564410662 shutdown_connections 2024-02-19T18:50:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.528+0000 7f619949e700 1 --2- 172.21.15.88:0/2564410662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6194103cb0 0x7f619419b9a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:30.124 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.528+0000 7f619949e700 1 -- 172.21.15.88:0/2564410662 >> 172.21.15.88:0/2564410662 conn(0x7f61940ff670 msgr2=0x7f6194100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:30.124 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.529+0000 7f619949e700 1 -- 172.21.15.88:0/2564410662 shutdown_connections 2024-02-19T18:50:30.124 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:29.529+0000 7f619949e700 1 -- 172.21.15.88:0/2564410662 wait complete. 2024-02-19T18:50:30.124 INFO:teuthology.orchestra.run.smithi088.stdout:Restarting the monitor... 2024-02-19T18:50:30.145 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:30 smithi088 ceph-mon[28891]: from='client.? 172.21.15.88:0/2564410662' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:50:30.517 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:30 smithi088 systemd[1]: Stopping Ceph mon.smithi088 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T18:50:30.517 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:30 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[28887]: 2024-02-19T18:50:30.367+0000 7f00121de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi088 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T18:50:30.517 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:30 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[28887]: 2024-02-19T18:50:30.367+0000 7f00121de700 -1 mon.smithi088@0(leader) e1 *** Got Signal Terminated *** 2024-02-19T18:50:30.791 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:30 smithi088 podman[29146]: 2024-02-19 18:50:30.501801524 +0000 UTC m=+0.231219003 container died 7d05c4bb3ea08880a9bf6acb2e1c5530cfb76e259ba93e0e4a533fada6258885 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-02-19T18:50:30.791 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:30 smithi088 podman[29146]: 2024-02-19 18:50:30.776385485 +0000 UTC m=+0.505802972 container cleanup 7d05c4bb3ea08880a9bf6acb2e1c5530cfb76e259ba93e0e4a533fada6258885 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, org.label-schema.build-date=20231212, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, ceph=True, GIT_BRANCH=HEAD) 2024-02-19T18:50:31.095 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:30 smithi088 bash[29146]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088 2024-02-19T18:50:31.095 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 podman[29160]: 2024-02-19 18:50:31.010777865 +0000 UTC m=+0.502917577 container remove 7d05c4bb3ea08880a9bf6acb2e1c5530cfb76e259ba93e0e4a533fada6258885 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, RELEASE=HEAD, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-19T18:50:31.463 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi088.service: Deactivated successfully. 2024-02-19T18:50:31.463 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 systemd[1]: Stopped Ceph mon.smithi088 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T18:50:31.463 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 systemd[1]: Starting Ceph mon.smithi088 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T18:50:31.463 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 podman[29254]: 2024-02-19 18:50:31.344885324 +0000 UTC m=+0.111005794 container create 3ea5108455bc6bd4da387b645ceea6ca82bc1ef71f39070e02e79bb88b5f3baf (image=quay.io/ceph/ceph:v18.2.0, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-19T18:50:31.464 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 podman[29254]: 2024-02-19 18:50:31.251697279 +0000 UTC m=+0.017817756 image pull dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 quay.io/ceph/ceph:v18.2.0 2024-02-19T18:50:32.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 podman[29254]: 2024-02-19 18:50:31.823422183 +0000 UTC m=+0.589542653 container init 3ea5108455bc6bd4da387b645ceea6ca82bc1ef71f39070e02e79bb88b5f3baf (image=quay.io/ceph/ceph:v18.2.0, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, org.label-schema.vendor=CentOS, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, io.buildah.version=1.29.1, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-19T18:50:32.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 podman[29254]: 2024-02-19 18:50:31.826713685 +0000 UTC m=+0.592834156 container start 3ea5108455bc6bd4da387b645ceea6ca82bc1ef71f39070e02e79bb88b5f3baf (image=quay.io/ceph/ceph:v18.2.0, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, RELEASE=HEAD, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-19T18:50:32.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: set uid:gid to 167:167 (ceph:ceph) 2024-02-19T18:50:32.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-02-19T18:50:32.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: pidfile_write: ignore empty --pid-file 2024-02-19T18:50:32.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: load: jerasure load: lrc 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: RocksDB version: 7.9.2 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Git sha 0 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Compile date 2023-08-03 19:21:13 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: DB SUMMARY 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: DB Session ID: ANV929TMG29BDS2YD2OE 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: CURRENT file: CURRENT 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: IDENTITY file: IDENTITY 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi088/store.db dir, Total Num: 1, files: 000008.sst 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi088/store.db: 000009.log size: 97392 ; 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.error_if_exists: 0 2024-02-19T18:50:32.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.create_if_missing: 0 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.paranoid_checks: 1 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.env: 0x56517e457720 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.fs: PosixFileSystem 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.info_log: 0x56517fc492c0 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_file_opening_threads: 16 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.statistics: (nil) 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.use_fsync: 0 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_log_file_size: 0 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-19T18:50:32.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.keep_log_file_num: 1000 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.recycle_log_file_num: 0 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.allow_fallocate: 1 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.allow_mmap_reads: 0 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.allow_mmap_writes: 0 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.use_direct_reads: 0 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.create_missing_column_families: 0 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.db_log_dir: 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.wal_dir: 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-19T18:50:32.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.advise_random_on_open: 1 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.db_write_buffer_size: 0 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.write_buffer_manager: 0x56517eeba3c0 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.rate_limiter: (nil) 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.wal_recovery_mode: 2 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.enable_thread_tracking: 0 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.enable_pipelined_write: 0 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.unordered_write: 0 2024-02-19T18:50:32.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-19T18:50:32.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-19T18:50:32.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-19T18:50:32.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-19T18:50:32.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.row_cache: None 2024-02-19T18:50:32.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.wal_filter: None 2024-02-19T18:50:32.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-19T18:50:32.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.allow_ingest_behind: 0 2024-02-19T18:50:32.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.two_write_queues: 0 2024-02-19T18:50:32.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.manual_wal_flush: 0 2024-02-19T18:50:32.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.wal_compression: 0 2024-02-19T18:50:32.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.atomic_flush: 0 2024-02-19T18:50:32.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-19T18:50:32.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-19T18:50:32.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.log_readahead_size: 0 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.best_efforts_recovery: 0 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.allow_data_in_errors: 0 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.db_host_id: __hostname__ 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_background_jobs: 2 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_background_compactions: -1 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_subcompactions: 1 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-19T18:50:32.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_total_wal_size: 0 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_open_files: -1 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bytes_per_sync: 0 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_readahead_size: 0 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_background_flushes: -1 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Compression algorithms supported: 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: kZSTD supported: 0 2024-02-19T18:50:32.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: kXpressCompression supported: 0 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: kLZ4HCCompression supported: 1 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: kZlibCompression supported: 1 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: kSnappyCompression supported: 1 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: kLZ4Compression supported: 1 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: kBZip2Compression supported: 0 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi088/store.db/MANIFEST-000010 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.merge_operator: 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_filter: None 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_filter_factory: None 2024-02-19T18:50:32.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.sst_partitioner_factory: None 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56517fc493c0) 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: cache_index_and_filter_blocks: 1 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: pin_top_level_index_and_filter: 1 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_type: 0 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: data_block_index_type: 0 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_shortening: 1 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: checksum: 4 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: no_block_cache: 0 2024-02-19T18:50:32.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache: 0x56517ef5b1f0 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_name: BinnedLRUCache 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_options: 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: capacity : 536870912 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: num_shard_bits : 4 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: strict_capacity_limit : 0 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: high_pri_pool_ratio: 0.000 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_compressed: (nil) 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: persistent_cache: (nil) 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_size: 4096 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_size_deviation: 10 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_restart_interval: 16 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_block_restart_interval: 1 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: metadata_block_size: 4096 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: partition_filters: 0 2024-02-19T18:50:32.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: use_delta_encoding: 1 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: filter_policy: bloomfilter 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: whole_key_filtering: 1 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: verify_compression: 0 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: read_amp_bytes_per_bit: 0 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: format_version: 5 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: enable_index_compression: 1 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_align: 0 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: max_auto_readahead_size: 262144 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: prepopulate_block_cache: 0 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: initial_auto_readahead_size: 8192 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: num_file_reads_for_auto_readahead: 2 2024-02-19T18:50:32.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.write_buffer_size: 33554432 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_write_buffer_number: 2 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compression: NoCompression 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bottommost_compression: Disabled 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.prefix_extractor: nullptr 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.num_levels: 7 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-19T18:50:32.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compression_opts.level: 32767 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compression_opts.strategy: 0 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compression_opts.enabled: false 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-19T18:50:32.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.target_file_size_base: 67108864 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-19T18:50:32.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.arena_block_size: 1048576 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.disable_auto_compactions: 0 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-19T18:50:32.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.table_properties_collectors: 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.inplace_update_support: 0 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.bloom_locality: 0 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.max_successive_merges: 0 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.paranoid_file_checks: 0 2024-02-19T18:50:32.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.force_consistency_checks: 1 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.report_bg_io_stats: 0 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.ttl: 2592000 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.enable_blob_files: false 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.min_blob_size: 0 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.blob_file_size: 268435456 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-19T18:50:32.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.blob_file_starting_level: 0 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi088/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ba0b5dbd-2286-4f7c-8c91-1f2b765707c4 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708368631858010, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708368631859070, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92742, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 310, "table_properties": {"data_size": 90759, "index_size": 265, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13356, "raw_average_key_size": 49, "raw_value_size": 83474, "raw_average_value_size": 310, "num_data_blocks": 11, "num_entries": 269, "num_filter_entries": 269, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708368631, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ba0b5dbd-2286-4f7c-8c91-1f2b765707c4", "db_session_id": "ANV929TMG29BDS2YD2OE", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708368631859153, "job": 1, "event": "recovery_finished"} 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi088/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56517eff8000 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: DB pointer 0x56517efe4000 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-19T18:50:32.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** DB Stats ** 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** Compaction Stats [default] ** 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: L0 2/0 92.44 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 123.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Sum 2/0 92.44 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 123.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 123.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T18:50:32.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** Compaction Stats [default] ** 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 123.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative compaction: 0.00 GB write, 28.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval compaction: 0.00 GB write, 28.97 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T18:50:32.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Block cache BinnedLRUCache@0x56517ef5b1f0#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: starting mon.smithi088 rank 0 at public addrs [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] at bind addrs [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi088 fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: mon.smithi088@-1(???) e1 preinit fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: mon.smithi088@-1(???).mds e1 new map 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: mon.smithi088@-1(???).mds e1 print_map 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: e1 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: legacy client fscid: -1 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: No filesystems configured 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: mon.smithi088@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: mon.smithi088@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T18:50:32.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: mon.smithi088@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: mon.smithi088@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: mon.smithi088@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta expand map: {default=false} 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta from 'false' to 'false' 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta expanded map: {default=false} 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta expand map: {default=info} 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta from 'info' to 'info' 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta expanded map: {default=info} 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta expand map: {default=daemon} 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta from 'daemon' to 'daemon' 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta expanded map: {default=daemon} 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta expand map: {default=debug} 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta from 'debug' to 'debug' 2024-02-19T18:50:32.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: expand_channel_meta expanded map: {default=debug} 2024-02-19T18:50:32.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: mon.smithi088 is new leader, mons smithi088 in quorum (ranks 0) 2024-02-19T18:50:32.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: monmap e1: 1 mons at {smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0]} removed_ranks: {} 2024-02-19T18:50:32.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: fsmap 2024-02-19T18:50:32.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: osdmap e1: 0 total, 0 up, 0 in 2024-02-19T18:50:32.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:31 smithi088 ceph-mon[29272]: mgrmap e1: no daemons active 2024-02-19T18:50:32.656 INFO:teuthology.orchestra.run.smithi088.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-19T18:50:33.095 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:32 smithi088 bash[29254]: 3ea5108455bc6bd4da387b645ceea6ca82bc1ef71f39070e02e79bb88b5f3baf 2024-02-19T18:50:33.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:32 smithi088 systemd[1]: Started Ceph mon.smithi088 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T18:50:37.095 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:36 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2396949446' entity='client.admin' 2024-02-19T18:50:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.623+0000 7fc045909700 1 Processor -- start 2024-02-19T18:50:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.623+0000 7fc045909700 1 -- start start 2024-02-19T18:50:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.623+0000 7fc045909700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc0401085b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.623+0000 7fc045909700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc040108af0 con 0x7fc040079c70 2024-02-19T18:50:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.623+0000 7fc03effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc0401085b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.623+0000 7fc03effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc0401085b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48122/0 (socket says 172.21.15.88:48122) 2024-02-19T18:50:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.623+0000 7fc03effd700 1 -- 172.21.15.88:0/287140424 learned_addr learned my addr 172.21.15.88:0/287140424 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.624+0000 7fc03effd700 1 -- 172.21.15.88:0/287140424 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc040108c30 con 0x7fc040079c70 2024-02-19T18:50:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.624+0000 7fc03effd700 1 --2- 172.21.15.88:0/287140424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc0401085b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc0300098a0 tx=0x7fc030009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e63297d8717dac server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.624+0000 7fc03dffb700 1 -- 172.21.15.88:0/287140424 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc030004180 con 0x7fc040079c70 2024-02-19T18:50:41.870 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.624+0000 7fc03dffb700 1 -- 172.21.15.88:0/287140424 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc0300042e0 con 0x7fc040079c70 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.624+0000 7fc03dffb700 1 -- 172.21.15.88:0/287140424 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc030011a50 con 0x7fc040079c70 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.625+0000 7fc045909700 1 -- 172.21.15.88:0/287140424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 msgr2=0x7fc0401085b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.625+0000 7fc045909700 1 --2- 172.21.15.88:0/287140424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc0401085b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc0300098a0 tx=0x7fc030009bb0 comp rx=0 tx=0).stop 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.625+0000 7fc045909700 1 -- 172.21.15.88:0/287140424 shutdown_connections 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.625+0000 7fc045909700 1 --2- 172.21.15.88:0/287140424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc0401085b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.625+0000 7fc045909700 1 -- 172.21.15.88:0/287140424 >> 172.21.15.88:0/287140424 conn(0x7fc0400ffee0 msgr2=0x7fc040102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.625+0000 7fc045909700 1 -- 172.21.15.88:0/287140424 shutdown_connections 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.625+0000 7fc045909700 1 -- 172.21.15.88:0/287140424 wait complete. 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.625+0000 7fc045909700 1 Processor -- start 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc045909700 1 -- start start 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc045909700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc040199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc045909700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc040199c80 con 0x7fc040079c70 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc03effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc040199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc03effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc040199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48130/0 (socket says 172.21.15.88:48130) 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc03effd700 1 -- 172.21.15.88:0/2396949446 learned_addr learned my addr 172.21.15.88:0/2396949446 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:41.871 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc03effd700 1 -- 172.21.15.88:0/2396949446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc030009550 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc03effd700 1 --2- 172.21.15.88:0/2396949446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc040199740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc030003de0 tx=0x7fc030011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc044907700 1 -- 172.21.15.88:0/2396949446 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc030003930 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc045909700 1 -- 172.21.15.88:0/2396949446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc040199e80 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc044907700 1 -- 172.21.15.88:0/2396949446 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc030003a90 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc044907700 1 -- 172.21.15.88:0/2396949446 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc0300196c0 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.626+0000 7fc045909700 1 -- 172.21.15.88:0/2396949446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04019a320 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.627+0000 7fc044907700 1 -- 172.21.15.88:0/2396949446 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc030019ca0 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.627+0000 7fc044907700 1 -- 172.21.15.88:0/2396949446 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc03002c3a0 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.627+0000 7fc045909700 1 -- 172.21.15.88:0/2396949446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc040062380 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.630+0000 7fc044907700 1 -- 172.21.15.88:0/2396949446 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fc03001abc0 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.697+0000 7fc045909700 1 -- 172.21.15.88:0/2396949446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fc04019d150 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.699+0000 7fc044907700 1 -- 172.21.15.88:0/2396949446 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc030005a20 con 0x7fc040079c70 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.700+0000 7fc045909700 1 -- 172.21.15.88:0/2396949446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 msgr2=0x7fc040199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.700+0000 7fc045909700 1 --2- 172.21.15.88:0/2396949446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc040199740 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc030003de0 tx=0x7fc030011ab0 comp rx=0 tx=0).stop 2024-02-19T18:50:41.872 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.700+0000 7fc045909700 1 -- 172.21.15.88:0/2396949446 shutdown_connections 2024-02-19T18:50:41.873 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.700+0000 7fc045909700 1 --2- 172.21.15.88:0/2396949446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc040079c70 0x7fc040199740 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:41.873 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.700+0000 7fc045909700 1 -- 172.21.15.88:0/2396949446 >> 172.21.15.88:0/2396949446 conn(0x7fc0400ffee0 msgr2=0x7fc040101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:41.873 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.700+0000 7fc045909700 1 -- 172.21.15.88:0/2396949446 shutdown_connections 2024-02-19T18:50:41.873 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:35.700+0000 7fc045909700 1 -- 172.21.15.88:0/2396949446 wait complete. 2024-02-19T18:50:41.873 INFO:teuthology.orchestra.run.smithi088.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-19T18:50:42.120 INFO:teuthology.orchestra.run.smithi088.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-19T18:50:42.121 INFO:teuthology.orchestra.run.smithi088.stdout:Creating mgr... 2024-02-19T18:50:42.121 INFO:teuthology.orchestra.run.smithi088.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-19T18:50:42.122 INFO:teuthology.orchestra.run.smithi088.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-19T18:50:42.122 INFO:teuthology.orchestra.run.smithi088.stdout:Verifying port 0.0.0.0:8443 ... 2024-02-19T18:50:43.183 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mgr.smithi088.gyviqq 2024-02-19T18:50:43.184 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stderr Failed to reset failed state of unit ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mgr.smithi088.gyviqq.service: Unit ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mgr.smithi088.gyviqq.service not loaded. 2024-02-19T18:50:43.453 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715.target.wants/ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mgr.smithi088.gyviqq.service → /etc/systemd/system/ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@.service. 2024-02-19T18:50:45.752 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-19T18:50:45.752 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout disabled 2024-02-19T18:50:45.759 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-19T18:50:45.759 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout inactive 2024-02-19T18:50:45.759 INFO:teuthology.orchestra.run.smithi088.stdout:firewalld.service is not enabled 2024-02-19T18:50:45.759 INFO:teuthology.orchestra.run.smithi088.stdout:Not possible to enable service . firewalld.service is not available 2024-02-19T18:50:45.767 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-19T18:50:45.767 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout disabled 2024-02-19T18:50:45.774 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-19T18:50:45.774 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout inactive 2024-02-19T18:50:45.774 INFO:teuthology.orchestra.run.smithi088.stdout:firewalld.service is not enabled 2024-02-19T18:50:45.774 INFO:teuthology.orchestra.run.smithi088.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-02-19T18:50:45.774 INFO:teuthology.orchestra.run.smithi088.stdout:Waiting for mgr to start... 2024-02-19T18:50:45.774 INFO:teuthology.orchestra.run.smithi088.stdout:Waiting for mgr... 2024-02-19T18:50:46.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:46 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/282194260' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-19T18:50:46.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:46.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout { 2024-02-19T18:50:46.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "fsid": "a60b4d3a-cf57-11ee-95bb-87774f69a715", 2024-02-19T18:50:46.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "health": { 2024-02-19T18:50:46.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-19T18:50:46.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-19T18:50:46.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-19T18:50:46.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:46.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-19T18:50:46.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-19T18:50:46.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 0 2024-02-19T18:50:46.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:46.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-19T18:50:46.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "smithi088" 2024-02-19T18:50:46.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:46.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-02-19T18:50:46.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-19T18:50:46.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:46.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-19T18:50:46.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-19T18:50:46.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:46.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-19T18:50:46.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:46.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-19T18:50:46.953 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-19T18:50:46.953 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-19T18:50:46.961 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-19T18:50:46.961 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-19T18:50:46.961 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-19T18:50:46.961 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-19T18:50:46.962 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "restful" 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "modified": "2024-02-19T18:50:27.027520+0000", 2024-02-19T18:50:46.963 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T18:50:46.964 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:46.964 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-19T18:50:46.964 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout } 2024-02-19T18:50:46.964 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.218+0000 7f06602cd700 1 Processor -- start 2024-02-19T18:50:46.964 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.218+0000 7f06602cd700 1 -- start start 2024-02-19T18:50:46.964 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.218+0000 7f06602cd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f06581040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:46.964 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.218+0000 7f06602cd700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0658104600 con 0x7f0658103cb0 2024-02-19T18:50:46.964 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.218+0000 7f065e069700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f06581040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:46.965 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.218+0000 7f065e069700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f06581040c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37806/0 (socket says 172.21.15.88:37806) 2024-02-19T18:50:46.965 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.218+0000 7f065e069700 1 -- 172.21.15.88:0/371095430 learned_addr learned my addr 172.21.15.88:0/371095430 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:46.965 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.219+0000 7f065e069700 1 -- 172.21.15.88:0/371095430 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0658104740 con 0x7f0658103cb0 2024-02-19T18:50:46.965 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.219+0000 7f065e069700 1 --2- 172.21.15.88:0/371095430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f06581040c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f06540098a0 tx=0x7f0654009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a27f3ec815feaf0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:46.965 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.219+0000 7f065d067700 1 -- 172.21.15.88:0/371095430 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0654004180 con 0x7f0658103cb0 2024-02-19T18:50:46.965 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.219+0000 7f065d067700 1 -- 172.21.15.88:0/371095430 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f06540042e0 con 0x7f0658103cb0 2024-02-19T18:50:46.965 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.219+0000 7f065d067700 1 -- 172.21.15.88:0/371095430 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0654011a50 con 0x7f0658103cb0 2024-02-19T18:50:46.965 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.220+0000 7f06602cd700 1 -- 172.21.15.88:0/371095430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 msgr2=0x7f06581040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:46.965 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.220+0000 7f06602cd700 1 --2- 172.21.15.88:0/371095430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f06581040c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f06540098a0 tx=0x7f0654009bb0 comp rx=0 tx=0).stop 2024-02-19T18:50:46.965 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.220+0000 7f06602cd700 1 -- 172.21.15.88:0/371095430 shutdown_connections 2024-02-19T18:50:46.965 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.220+0000 7f06602cd700 1 --2- 172.21.15.88:0/371095430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f06581040c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.220+0000 7f06602cd700 1 -- 172.21.15.88:0/371095430 >> 172.21.15.88:0/371095430 conn(0x7f06580ff670 msgr2=0x7f0658101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.220+0000 7f06602cd700 1 -- 172.21.15.88:0/371095430 shutdown_connections 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.220+0000 7f06602cd700 1 -- 172.21.15.88:0/371095430 wait complete. 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.220+0000 7f06602cd700 1 Processor -- start 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f06602cd700 1 -- start start 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f06602cd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f0658106640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f06602cd700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0658106b80 con 0x7f0658103cb0 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f065e069700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f0658106640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f065e069700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f0658106640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37820/0 (socket says 172.21.15.88:37820) 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f065e069700 1 -- 172.21.15.88:0/282194260 learned_addr learned my addr 172.21.15.88:0/282194260 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f065e069700 1 -- 172.21.15.88:0/282194260 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0654009550 con 0x7f0658103cb0 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f065e069700 1 --2- 172.21.15.88:0/282194260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f0658106640 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0654009520 tx=0x7f0654011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:46.966 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f064f7fe700 1 -- 172.21.15.88:0/282194260 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0654003a80 con 0x7f0658103cb0 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f064f7fe700 1 -- 172.21.15.88:0/282194260 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0654003be0 con 0x7f0658103cb0 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f06602cd700 1 -- 172.21.15.88:0/282194260 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0658104c90 con 0x7f0658103cb0 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.221+0000 7f064f7fe700 1 -- 172.21.15.88:0/282194260 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f06540197b0 con 0x7f0658103cb0 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.222+0000 7f06602cd700 1 -- 172.21.15.88:0/282194260 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0658105130 con 0x7f0658103cb0 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.222+0000 7f064f7fe700 1 -- 172.21.15.88:0/282194260 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0654019d40 con 0x7f0658103cb0 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.222+0000 7f064f7fe700 1 -- 172.21.15.88:0/282194260 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f06540248a0 con 0x7f0658103cb0 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.223+0000 7f06602cd700 1 -- 172.21.15.88:0/282194260 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0658107870 con 0x7f0658103cb0 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.225+0000 7f064f7fe700 1 -- 172.21.15.88:0/282194260 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f065401ac30 con 0x7f0658103cb0 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.298+0000 7f06602cd700 1 -- 172.21.15.88:0/282194260 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0658107a90 con 0x7f0658103cb0 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.299+0000 7f064f7fe700 1 -- 172.21.15.88:0/282194260 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f0654024b00 con 0x7f0658103cb0 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.299+0000 7f06602cd700 1 -- 172.21.15.88:0/282194260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 msgr2=0x7f0658106640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.299+0000 7f06602cd700 1 --2- 172.21.15.88:0/282194260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f0658106640 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0654009520 tx=0x7f0654011ab0 comp rx=0 tx=0).stop 2024-02-19T18:50:46.967 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.300+0000 7f06602cd700 1 -- 172.21.15.88:0/282194260 shutdown_connections 2024-02-19T18:50:46.968 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.300+0000 7f06602cd700 1 --2- 172.21.15.88:0/282194260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0658103cb0 0x7f0658106640 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:46.968 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.300+0000 7f06602cd700 1 -- 172.21.15.88:0/282194260 >> 172.21.15.88:0/282194260 conn(0x7f06580ff670 msgr2=0x7f0658100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:46.968 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.300+0000 7f06602cd700 1 -- 172.21.15.88:0/282194260 shutdown_connections 2024-02-19T18:50:46.968 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:46.300+0000 7f06602cd700 1 -- 172.21.15.88:0/282194260 wait complete. 2024-02-19T18:50:46.968 INFO:teuthology.orchestra.run.smithi088.stdout:mgr not available, waiting (1/15)... 2024-02-19T18:50:49.595 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:49 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1551042213' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-19T18:50:50.181 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:50.181 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout { 2024-02-19T18:50:50.181 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "fsid": "a60b4d3a-cf57-11ee-95bb-87774f69a715", 2024-02-19T18:50:50.181 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "health": { 2024-02-19T18:50:50.181 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-19T18:50:50.181 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-19T18:50:50.181 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 0 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "smithi088" 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:50.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-19T18:50:50.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-19T18:50:50.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-19T18:50:50.190 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-19T18:50:50.190 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-19T18:50:50.190 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-19T18:50:50.190 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:50.190 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-19T18:50:50.190 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-19T18:50:50.191 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "restful" 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "modified": "2024-02-19T18:50:27.027520+0000", 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout } 2024-02-19T18:50:50.192 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.408+0000 7f1ca14a6700 1 Processor -- start 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.408+0000 7f1ca14a6700 1 -- start start 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.409+0000 7f1ca14a6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.409+0000 7f1ca14a6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c07a810 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.409+0000 7f1c9affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.409+0000 7f1c9affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37836/0 (socket says 172.21.15.88:37836) 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.409+0000 7f1c9affd700 1 -- 172.21.15.88:0/3422220448 learned_addr learned my addr 172.21.15.88:0/3422220448 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.409+0000 7f1c9affd700 1 -- 172.21.15.88:0/3422220448 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c07a950 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.409+0000 7f1c9affd700 1 --2- 172.21.15.88:0/3422220448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c07a2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009870 tx=0x7f1c8c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265ec5df16a6fa1e server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.409+0000 7f1c99ffb700 1 -- 172.21.15.88:0/3422220448 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c8c005c30 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.410+0000 7f1c99ffb700 1 -- 172.21.15.88:0/3422220448 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c8c005d90 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.410+0000 7f1c99ffb700 1 -- 172.21.15.88:0/3422220448 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c8c006030 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.410+0000 7f1ca14a6700 1 -- 172.21.15.88:0/3422220448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 msgr2=0x7f1c9c07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.410+0000 7f1ca14a6700 1 --2- 172.21.15.88:0/3422220448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c07a2d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c009870 tx=0x7f1c8c009b80 comp rx=0 tx=0).stop 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.410+0000 7f1ca14a6700 1 -- 172.21.15.88:0/3422220448 shutdown_connections 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.410+0000 7f1ca14a6700 1 --2- 172.21.15.88:0/3422220448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c07a2d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:50.193 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.410+0000 7f1ca14a6700 1 -- 172.21.15.88:0/3422220448 >> 172.21.15.88:0/3422220448 conn(0x7f1c9c101020 msgr2=0x7f1c9c103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.410+0000 7f1ca14a6700 1 -- 172.21.15.88:0/3422220448 shutdown_connections 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.410+0000 7f1ca14a6700 1 -- 172.21.15.88:0/3422220448 wait complete. 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.411+0000 7f1ca14a6700 1 Processor -- start 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.411+0000 7f1ca14a6700 1 -- start start 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.411+0000 7f1ca14a6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.411+0000 7f1ca14a6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c19bdb0 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.411+0000 7f1c9affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.411+0000 7f1c9affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c19b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37842/0 (socket says 172.21.15.88:37842) 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.412+0000 7f1c9affd700 1 -- 172.21.15.88:0/1551042213 learned_addr learned my addr 172.21.15.88:0/1551042213 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.412+0000 7f1c9affd700 1 -- 172.21.15.88:0/1551042213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8c009500 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.412+0000 7f1c9affd700 1 --2- 172.21.15.88:0/1551042213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c19b870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c006130 tx=0x7f1c8c006160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.412+0000 7f1c83fff700 1 -- 172.21.15.88:0/1551042213 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c8c0043b0 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.412+0000 7f1ca14a6700 1 -- 172.21.15.88:0/1551042213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c19bfb0 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.412+0000 7f1c83fff700 1 -- 172.21.15.88:0/1551042213 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f1c8c004510 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.412+0000 7f1c83fff700 1 -- 172.21.15.88:0/1551042213 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1c8c011c80 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.194 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.412+0000 7f1ca14a6700 1 -- 172.21.15.88:0/1551042213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c19c450 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.412+0000 7f1c83fff700 1 -- 172.21.15.88:0/1551042213 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1c8c018070 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.412+0000 7f1c83fff700 1 -- 172.21.15.88:0/1551042213 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f1c8c0249a0 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.413+0000 7f1ca14a6700 1 -- 172.21.15.88:0/1551042213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9c062380 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.415+0000 7f1c83fff700 1 -- 172.21.15.88:0/1551042213 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f1c8c016070 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.488+0000 7f1ca14a6700 1 -- 172.21.15.88:0/1551042213 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1c9c19ee70 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.489+0000 7f1c83fff700 1 -- 172.21.15.88:0/1551042213 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f1c8c024bb0 con 0x7f1c9c07bdd0 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.490+0000 7f1ca14a6700 1 -- 172.21.15.88:0/1551042213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 msgr2=0x7f1c9c19b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.490+0000 7f1ca14a6700 1 --2- 172.21.15.88:0/1551042213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c19b870 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1c8c006130 tx=0x7f1c8c006160 comp rx=0 tx=0).stop 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.490+0000 7f1ca14a6700 1 -- 172.21.15.88:0/1551042213 shutdown_connections 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.490+0000 7f1ca14a6700 1 --2- 172.21.15.88:0/1551042213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c9c07bdd0 0x7f1c9c19b870 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.490+0000 7f1ca14a6700 1 -- 172.21.15.88:0/1551042213 >> 172.21.15.88:0/1551042213 conn(0x7f1c9c101020 msgr2=0x7f1c9c101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.490+0000 7f1ca14a6700 1 -- 172.21.15.88:0/1551042213 shutdown_connections 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:49.490+0000 7f1ca14a6700 1 -- 172.21.15.88:0/1551042213 wait complete. 2024-02-19T18:50:50.195 INFO:teuthology.orchestra.run.smithi088.stdout:mgr not available, waiting (2/15)... 2024-02-19T18:50:53.166 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: Activating manager daemon smithi088.gyviqq 2024-02-19T18:50:53.166 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: mgrmap e2: smithi088.gyviqq(active, starting, since 0.00290064s) 2024-02-19T18:50:53.166 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: from='mgr.14100 172.21.15.88:0/1750221233' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T18:50:53.167 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: from='mgr.14100 172.21.15.88:0/1750221233' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T18:50:53.167 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: from='mgr.14100 172.21.15.88:0/1750221233' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T18:50:53.167 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: from='mgr.14100 172.21.15.88:0/1750221233' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T18:50:53.167 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: from='mgr.14100 172.21.15.88:0/1750221233' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi088.gyviqq", "id": "smithi088.gyviqq"}]: dispatch 2024-02-19T18:50:53.167 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: Manager daemon smithi088.gyviqq is now available 2024-02-19T18:50:53.167 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: from='mgr.14100 172.21.15.88:0/1750221233' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/mirror_snapshot_schedule"}]: dispatch 2024-02-19T18:50:53.167 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: from='mgr.14100 172.21.15.88:0/1750221233' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/trash_purge_schedule"}]: dispatch 2024-02-19T18:50:53.167 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: from='mgr.14100 172.21.15.88:0/1750221233' entity='mgr.smithi088.gyviqq' 2024-02-19T18:50:53.167 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: from='mgr.14100 172.21.15.88:0/1750221233' entity='mgr.smithi088.gyviqq' 2024-02-19T18:50:53.167 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: from='mgr.14100 172.21.15.88:0/1750221233' entity='mgr.smithi088.gyviqq' 2024-02-19T18:50:53.167 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:52 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1379810065' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-19T18:50:53.168 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:53.171 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout { 2024-02-19T18:50:53.171 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "fsid": "a60b4d3a-cf57-11ee-95bb-87774f69a715", 2024-02-19T18:50:53.171 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "health": { 2024-02-19T18:50:53.171 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-19T18:50:53.171 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-19T18:50:53.171 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 0 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "smithi088" 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum_age": 20, 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:53.172 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-19T18:50:53.173 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-19T18:50:53.173 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:53.173 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-19T18:50:53.173 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:53.173 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-19T18:50:53.173 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-19T18:50:53.173 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-19T18:50:53.182 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "restful" 2024-02-19T18:50:53.183 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "modified": "2024-02-19T18:50:27.027520+0000", 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout } 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.576+0000 7f7cd9e79700 1 Processor -- start 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.576+0000 7f7cd9e79700 1 -- start start 2024-02-19T18:50:53.184 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.576+0000 7f7cd9e79700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.576+0000 7f7cd9e79700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4079d10 con 0x7f7cd4106510 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.576+0000 7f7cd37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.576+0000 7f7cd37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37882/0 (socket says 172.21.15.88:37882) 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.576+0000 7f7cd37fe700 1 -- 172.21.15.88:0/1665693666 learned_addr learned my addr 172.21.15.88:0/1665693666 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.576+0000 7f7cd37fe700 1 -- 172.21.15.88:0/1665693666 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd4079e50 con 0x7f7cd4106510 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.576+0000 7f7cd37fe700 1 --2- 172.21.15.88:0/1665693666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41088f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40098a0 tx=0x7f7cc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c9c917f03e2d85 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.577+0000 7f7cd27fc700 1 -- 172.21.15.88:0/1665693666 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cc4004180 con 0x7f7cd4106510 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.577+0000 7f7cd27fc700 1 -- 172.21.15.88:0/1665693666 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7cc40042e0 con 0x7f7cd4106510 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.577+0000 7f7cd27fc700 1 -- 172.21.15.88:0/1665693666 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cc4011a50 con 0x7f7cd4106510 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.577+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1665693666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 msgr2=0x7f7cd41088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.577+0000 7f7cd9e79700 1 --2- 172.21.15.88:0/1665693666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41088f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7cc40098a0 tx=0x7f7cc4009bb0 comp rx=0 tx=0).stop 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.577+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1665693666 shutdown_connections 2024-02-19T18:50:53.185 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.577+0000 7f7cd9e79700 1 --2- 172.21.15.88:0/1665693666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41088f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.577+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1665693666 >> 172.21.15.88:0/1665693666 conn(0x7f7cd40ffed0 msgr2=0x7f7cd4102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.577+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1665693666 shutdown_connections 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.577+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1665693666 wait complete. 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.578+0000 7f7cd9e79700 1 Processor -- start 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.578+0000 7f7cd9e79700 1 -- start start 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.578+0000 7f7cd9e79700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.578+0000 7f7cd9e79700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cd4199c30 con 0x7f7cd4106510 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.578+0000 7f7cd37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.578+0000 7f7cd37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37884/0 (socket says 172.21.15.88:37884) 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.578+0000 7f7cd37fe700 1 -- 172.21.15.88:0/1379810065 learned_addr learned my addr 172.21.15.88:0/1379810065 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.578+0000 7f7cd37fe700 1 -- 172.21.15.88:0/1379810065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc4009550 con 0x7f7cd4106510 2024-02-19T18:50:53.186 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.579+0000 7f7cd37fe700 1 --2- 172.21.15.88:0/1379810065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41996f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4003de0 tx=0x7f7cc4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.579+0000 7f7cd0ff9700 1 -- 172.21.15.88:0/1379810065 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cc4003930 con 0x7f7cd4106510 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.579+0000 7f7cd0ff9700 1 -- 172.21.15.88:0/1379810065 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7cc4003a90 con 0x7f7cd4106510 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.579+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1379810065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cd4199e90 con 0x7f7cd4106510 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.579+0000 7f7cd0ff9700 1 -- 172.21.15.88:0/1379810065 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7cc4019690 con 0x7f7cd4106510 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.579+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1379810065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cd419cc90 con 0x7f7cd4106510 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.580+0000 7f7cd0ff9700 1 -- 172.21.15.88:0/1379810065 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 2) v1 ==== 44840+0+0 (secure 0 0 0) 0x7f7cc401abc0 con 0x7f7cd4106510 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.580+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1379810065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cd4062380 con 0x7f7cd4106510 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.580+0000 7f7cd0ff9700 1 -- 172.21.15.88:0/1379810065 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7cc404bc50 con 0x7f7cd4106510 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.582+0000 7f7cd0ff9700 1 -- 172.21.15.88:0/1379810065 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f7cc4010790 con 0x7f7cd4106510 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.652+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1379810065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7cd419ce70 con 0x7f7cd4106510 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.652+0000 7f7cd0ff9700 1 -- 172.21.15.88:0/1379810065 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f7cc4018350 con 0x7f7cd4106510 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.653+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1379810065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 msgr2=0x7f7cd41996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.653+0000 7f7cd9e79700 1 --2- 172.21.15.88:0/1379810065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41996f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7cc4003de0 tx=0x7f7cc4011ab0 comp rx=0 tx=0).stop 2024-02-19T18:50:53.187 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.653+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1379810065 shutdown_connections 2024-02-19T18:50:53.188 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.653+0000 7f7cd9e79700 1 --2- 172.21.15.88:0/1379810065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cd4106510 0x7f7cd41996f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:53.188 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.653+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1379810065 >> 172.21.15.88:0/1379810065 conn(0x7f7cd40ffed0 msgr2=0x7f7cd4101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:53.188 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.653+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1379810065 shutdown_connections 2024-02-19T18:50:53.188 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:52.653+0000 7f7cd9e79700 1 -- 172.21.15.88:0/1379810065 wait complete. 2024-02-19T18:50:53.188 INFO:teuthology.orchestra.run.smithi088.stdout:mgr not available, waiting (3/15)... 2024-02-19T18:50:54.595 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:54 smithi088 ceph-mon[29272]: mgrmap e3: smithi088.gyviqq(active, since 1.00635s) 2024-02-19T18:50:56.472 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:56 smithi088 ceph-mon[29272]: mgrmap e4: smithi088.gyviqq(active, since 3s) 2024-02-19T18:50:56.472 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:56 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2100308565' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-19T18:50:56.477 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:56.477 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout { 2024-02-19T18:50:56.477 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "fsid": "a60b4d3a-cf57-11ee-95bb-87774f69a715", 2024-02-19T18:50:56.477 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "health": { 2024-02-19T18:50:56.477 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-19T18:50:56.477 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-19T18:50:56.477 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 0 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "smithi088" 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "quorum_age": 23, 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-19T18:50:56.478 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:56.479 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-19T18:50:56.479 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:56.479 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-19T18:50:56.479 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-19T18:50:56.487 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-19T18:50:56.487 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-19T18:50:56.487 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:56.488 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "restful" 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ], 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T18:50:56.489 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "modified": "2024-02-19T18:50:27.027520+0000", 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout }, 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout } 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.599+0000 7f0228155700 1 Processor -- start 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.599+0000 7f0228155700 1 -- start start 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.599+0000 7f0228155700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220105dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.599+0000 7f0228155700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0220106310 con 0x7f02201059c0 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.599+0000 7f0225ef1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220105dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.599+0000 7f0225ef1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220105dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38056/0 (socket says 172.21.15.88:38056) 2024-02-19T18:50:56.490 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.599+0000 7f0225ef1700 1 -- 172.21.15.88:0/2525560222 learned_addr learned my addr 172.21.15.88:0/2525560222 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.600+0000 7f0225ef1700 1 -- 172.21.15.88:0/2525560222 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0220106450 con 0x7f02201059c0 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.600+0000 7f0225ef1700 1 --2- 172.21.15.88:0/2525560222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220105dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f021c0098a0 tx=0x7f021c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c57e68d0c652742 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.600+0000 7f0224eef700 1 -- 172.21.15.88:0/2525560222 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f021c004180 con 0x7f02201059c0 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.600+0000 7f0224eef700 1 -- 172.21.15.88:0/2525560222 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f021c0042e0 con 0x7f02201059c0 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.600+0000 7f0224eef700 1 -- 172.21.15.88:0/2525560222 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f021c011a50 con 0x7f02201059c0 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.600+0000 7f0228155700 1 -- 172.21.15.88:0/2525560222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 msgr2=0x7f0220105dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.600+0000 7f0228155700 1 --2- 172.21.15.88:0/2525560222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220105dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f021c0098a0 tx=0x7f021c009bb0 comp rx=0 tx=0).stop 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0228155700 1 -- 172.21.15.88:0/2525560222 shutdown_connections 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0228155700 1 --2- 172.21.15.88:0/2525560222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220105dd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0228155700 1 -- 172.21.15.88:0/2525560222 >> 172.21.15.88:0/2525560222 conn(0x7f0220100fd0 msgr2=0x7f0220103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0228155700 1 -- 172.21.15.88:0/2525560222 shutdown_connections 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0228155700 1 -- 172.21.15.88:0/2525560222 wait complete. 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0228155700 1 Processor -- start 2024-02-19T18:50:56.491 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0228155700 1 -- start start 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0228155700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0228155700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0220199c40 con 0x7f02201059c0 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0225ef1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0225ef1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38062/0 (socket says 172.21.15.88:38062) 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.601+0000 7f0225ef1700 1 -- 172.21.15.88:0/2100308565 learned_addr learned my addr 172.21.15.88:0/2100308565 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.602+0000 7f0225ef1700 1 -- 172.21.15.88:0/2100308565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c009550 con 0x7f02201059c0 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.602+0000 7f0225ef1700 1 --2- 172.21.15.88:0/2100308565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220199700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f021c003f80 tx=0x7f021c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.602+0000 7f0216ffd700 1 -- 172.21.15.88:0/2100308565 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f021c003a80 con 0x7f02201059c0 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.602+0000 7f0216ffd700 1 -- 172.21.15.88:0/2100308565 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f021c003be0 con 0x7f02201059c0 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.602+0000 7f0228155700 1 -- 172.21.15.88:0/2100308565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0220199e40 con 0x7f02201059c0 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.602+0000 7f0216ffd700 1 -- 172.21.15.88:0/2100308565 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f021c019820 con 0x7f02201059c0 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.602+0000 7f0228155700 1 -- 172.21.15.88:0/2100308565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022019a2e0 con 0x7f02201059c0 2024-02-19T18:50:56.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.603+0000 7f0216ffd700 1 -- 172.21.15.88:0/2100308565 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f021c019aa0 con 0x7f02201059c0 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.603+0000 7f0228155700 1 -- 172.21.15.88:0/2100308565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0220062380 con 0x7f02201059c0 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.603+0000 7f0216ffd700 1 --2- 172.21.15.88:0/2100308565 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f020c038350 0x7f020c03a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.603+0000 7f0216ffd700 1 -- 172.21.15.88:0/2100308565 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f021c04c750 con 0x7f02201059c0 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.603+0000 7f02256f0700 1 --2- 172.21.15.88:0/2100308565 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f020c038350 0x7f020c03a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.604+0000 7f02256f0700 1 --2- 172.21.15.88:0/2100308565 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f020c038350 0x7f020c03a800 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0210006fd0 tx=0x7f0210006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.608+0000 7f0216ffd700 1 -- 172.21.15.88:0/2100308565 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f021c018ce0 con 0x7f02201059c0 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.871+0000 7f0228155700 1 -- 172.21.15.88:0/2100308565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0220107830 con 0x7f02201059c0 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.871+0000 7f0216ffd700 1 -- 172.21.15.88:0/2100308565 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f021c018b00 con 0x7f02201059c0 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.874+0000 7f0228155700 1 -- 172.21.15.88:0/2100308565 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f020c038350 msgr2=0x7f020c03a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.874+0000 7f0228155700 1 --2- 172.21.15.88:0/2100308565 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f020c038350 0x7f020c03a800 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0210006fd0 tx=0x7f0210006e40 comp rx=0 tx=0).stop 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.874+0000 7f0228155700 1 -- 172.21.15.88:0/2100308565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 msgr2=0x7f0220199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.874+0000 7f0228155700 1 --2- 172.21.15.88:0/2100308565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220199700 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f021c003f80 tx=0x7f021c011ab0 comp rx=0 tx=0).stop 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.874+0000 7f0228155700 1 -- 172.21.15.88:0/2100308565 shutdown_connections 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.874+0000 7f0228155700 1 --2- 172.21.15.88:0/2100308565 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f020c038350 0x7f020c03a800 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.874+0000 7f0228155700 1 --2- 172.21.15.88:0/2100308565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02201059c0 0x7f0220199700 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:56.493 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.874+0000 7f0228155700 1 -- 172.21.15.88:0/2100308565 >> 172.21.15.88:0/2100308565 conn(0x7f0220100fd0 msgr2=0x7f0220102710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:56.494 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.874+0000 7f0228155700 1 -- 172.21.15.88:0/2100308565 shutdown_connections 2024-02-19T18:50:56.494 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:55.874+0000 7f0228155700 1 -- 172.21.15.88:0/2100308565 wait complete. 2024-02-19T18:50:56.494 INFO:teuthology.orchestra.run.smithi088.stdout:mgr is available 2024-02-19T18:50:57.625 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout [global] 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout fsid = a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout [osd] 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-19T18:50:57.626 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.856+0000 7f31c9825700 1 Processor -- start 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.856+0000 7f31c9825700 1 -- start start 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.856+0000 7f31c9825700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4108930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.856+0000 7f31c9825700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c40744b0 con 0x7f31c4106550 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.856+0000 7f31c2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4108930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.856+0000 7f31c2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4108930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38076/0 (socket says 172.21.15.88:38076) 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.856+0000 7f31c2ffd700 1 -- 172.21.15.88:0/3636972679 learned_addr learned my addr 172.21.15.88:0/3636972679 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.856+0000 7f31c2ffd700 1 -- 172.21.15.88:0/3636972679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c40745f0 con 0x7f31c4106550 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.857+0000 7f31c2ffd700 1 --2- 172.21.15.88:0/3636972679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4108930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f31b80098a0 tx=0x7f31b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e087a4b850fcca server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.857+0000 7f31c1ffb700 1 -- 172.21.15.88:0/3636972679 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f31b8004180 con 0x7f31c4106550 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.857+0000 7f31c1ffb700 1 -- 172.21.15.88:0/3636972679 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31b80042e0 con 0x7f31c4106550 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.857+0000 7f31c1ffb700 1 -- 172.21.15.88:0/3636972679 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f31b8011a50 con 0x7f31c4106550 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.857+0000 7f31c9825700 1 -- 172.21.15.88:0/3636972679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 msgr2=0x7f31c4108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.857+0000 7f31c9825700 1 --2- 172.21.15.88:0/3636972679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4108930 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f31b80098a0 tx=0x7f31b8009bb0 comp rx=0 tx=0).stop 2024-02-19T18:50:57.627 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.857+0000 7f31c9825700 1 -- 172.21.15.88:0/3636972679 shutdown_connections 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.857+0000 7f31c9825700 1 --2- 172.21.15.88:0/3636972679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4108930 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.857+0000 7f31c9825700 1 -- 172.21.15.88:0/3636972679 >> 172.21.15.88:0/3636972679 conn(0x7f31c40fff10 msgr2=0x7f31c4102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.857+0000 7f31c9825700 1 -- 172.21.15.88:0/3636972679 shutdown_connections 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.857+0000 7f31c9825700 1 -- 172.21.15.88:0/3636972679 wait complete. 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.858+0000 7f31c9825700 1 Processor -- start 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.858+0000 7f31c9825700 1 -- start start 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.858+0000 7f31c9825700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.858+0000 7f31c9825700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c41063f0 con 0x7f31c4106550 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.858+0000 7f31c2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.858+0000 7f31c2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4105eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38088/0 (socket says 172.21.15.88:38088) 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.858+0000 7f31c2ffd700 1 -- 172.21.15.88:0/4000900100 learned_addr learned my addr 172.21.15.88:0/4000900100 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.859+0000 7f31c2ffd700 1 -- 172.21.15.88:0/4000900100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b8009550 con 0x7f31c4106550 2024-02-19T18:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.859+0000 7f31c2ffd700 1 --2- 172.21.15.88:0/4000900100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4105eb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f31b8003de0 tx=0x7f31b8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.859+0000 7f31c8823700 1 -- 172.21.15.88:0/4000900100 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f31b8003930 con 0x7f31c4106550 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.859+0000 7f31c8823700 1 -- 172.21.15.88:0/4000900100 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f31b8003a90 con 0x7f31c4106550 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.859+0000 7f31c9825700 1 -- 172.21.15.88:0/4000900100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c4104500 con 0x7f31c4106550 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.859+0000 7f31c8823700 1 -- 172.21.15.88:0/4000900100 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f31b80196c0 con 0x7f31c4106550 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.859+0000 7f31c9825700 1 -- 172.21.15.88:0/4000900100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c41049a0 con 0x7f31c4106550 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.860+0000 7f31c8823700 1 -- 172.21.15.88:0/4000900100 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f31b8019930 con 0x7f31c4106550 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.860+0000 7f31c8823700 1 --2- 172.21.15.88:0/4000900100 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f31ac0383f0 0x7f31ac03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.860+0000 7f31c8823700 1 -- 172.21.15.88:0/4000900100 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f31b804c1c0 con 0x7f31c4106550 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.860+0000 7f31c9825700 1 -- 172.21.15.88:0/4000900100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c4109a60 con 0x7f31c4106550 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.860+0000 7f31c27fc700 1 --2- 172.21.15.88:0/4000900100 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f31ac0383f0 0x7f31ac03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.860+0000 7f31c27fc700 1 --2- 172.21.15.88:0/4000900100 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f31ac0383f0 0x7f31ac03a8a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006fd0 tx=0x7f31b4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:56.865+0000 7f31c8823700 1 -- 172.21.15.88:0/4000900100 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f31b8019be0 con 0x7f31c4106550 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.048+0000 7f31c9825700 1 -- 172.21.15.88:0/4000900100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f31c4109c80 con 0x7f31c4106550 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.049+0000 7f31c8823700 1 -- 172.21.15.88:0/4000900100 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f31b8010380 con 0x7f31c4106550 2024-02-19T18:50:57.629 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.051+0000 7f31c9825700 1 -- 172.21.15.88:0/4000900100 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f31ac0383f0 msgr2=0x7f31ac03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:57.630 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.051+0000 7f31c9825700 1 --2- 172.21.15.88:0/4000900100 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f31ac0383f0 0x7f31ac03a8a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006fd0 tx=0x7f31b4006e40 comp rx=0 tx=0).stop 2024-02-19T18:50:57.630 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.051+0000 7f31c9825700 1 -- 172.21.15.88:0/4000900100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 msgr2=0x7f31c4105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:57.630 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.051+0000 7f31c9825700 1 --2- 172.21.15.88:0/4000900100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4105eb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f31b8003de0 tx=0x7f31b8011ab0 comp rx=0 tx=0).stop 2024-02-19T18:50:57.630 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.052+0000 7f31c9825700 1 -- 172.21.15.88:0/4000900100 shutdown_connections 2024-02-19T18:50:57.630 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.052+0000 7f31c9825700 1 --2- 172.21.15.88:0/4000900100 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f31ac0383f0 0x7f31ac03a8a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:57.630 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.052+0000 7f31c9825700 1 --2- 172.21.15.88:0/4000900100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31c4106550 0x7f31c4105eb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:57.630 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.052+0000 7f31c9825700 1 -- 172.21.15.88:0/4000900100 >> 172.21.15.88:0/4000900100 conn(0x7f31c40fff10 msgr2=0x7f31c4101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:57.630 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.052+0000 7f31c9825700 1 -- 172.21.15.88:0/4000900100 shutdown_connections 2024-02-19T18:50:57.630 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:57.052+0000 7f31c9825700 1 -- 172.21.15.88:0/4000900100 wait complete. 2024-02-19T18:50:57.630 INFO:teuthology.orchestra.run.smithi088.stdout:Enabling cephadm module... 2024-02-19T18:50:58.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:57 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/4000900100' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-19T18:50:59.194 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:58 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1630201346' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-19T18:50:59.468 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.069+0000 7fd4f64da700 1 Processor -- start 2024-02-19T18:50:59.468 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.069+0000 7fd4f64da700 1 -- start start 2024-02-19T18:50:59.468 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.069+0000 7fd4f64da700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0104450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:59.468 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.069+0000 7fd4f64da700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0104990 con 0x7fd4f0104040 2024-02-19T18:50:59.468 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.069+0000 7fd4effff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:59.468 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.069+0000 7fd4effff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0104450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38102/0 (socket says 172.21.15.88:38102) 2024-02-19T18:50:59.468 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.069+0000 7fd4effff700 1 -- 172.21.15.88:0/2525622355 learned_addr learned my addr 172.21.15.88:0/2525622355 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:59.468 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.069+0000 7fd4effff700 1 -- 172.21.15.88:0/2525622355 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f0104ad0 con 0x7fd4f0104040 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.069+0000 7fd4effff700 1 --2- 172.21.15.88:0/2525622355 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0104450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00098a0 tx=0x7fd4e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a187d83d37d7a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.069+0000 7fd4eeffd700 1 -- 172.21.15.88:0/2525622355 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4e0004180 con 0x7fd4f0104040 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.070+0000 7fd4eeffd700 1 -- 172.21.15.88:0/2525622355 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd4e00042e0 con 0x7fd4f0104040 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.070+0000 7fd4eeffd700 1 -- 172.21.15.88:0/2525622355 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4e0011a50 con 0x7fd4f0104040 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.070+0000 7fd4f64da700 1 -- 172.21.15.88:0/2525622355 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 msgr2=0x7fd4f0104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.070+0000 7fd4f64da700 1 --2- 172.21.15.88:0/2525622355 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0104450 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00098a0 tx=0x7fd4e0009bb0 comp rx=0 tx=0).stop 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.070+0000 7fd4f64da700 1 -- 172.21.15.88:0/2525622355 shutdown_connections 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.070+0000 7fd4f64da700 1 --2- 172.21.15.88:0/2525622355 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0104450 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.070+0000 7fd4f64da700 1 -- 172.21.15.88:0/2525622355 >> 172.21.15.88:0/2525622355 conn(0x7fd4f00ff670 msgr2=0x7fd4f0101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.070+0000 7fd4f64da700 1 -- 172.21.15.88:0/2525622355 shutdown_connections 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.070+0000 7fd4f64da700 1 -- 172.21.15.88:0/2525622355 wait complete. 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.071+0000 7fd4f64da700 1 Processor -- start 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.071+0000 7fd4f64da700 1 -- start start 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.071+0000 7fd4f64da700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:59.469 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.071+0000 7fd4f64da700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0198b50 con 0x7fd4f0104040 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.071+0000 7fd4effff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.071+0000 7fd4effff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38104/0 (socket says 172.21.15.88:38104) 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.071+0000 7fd4effff700 1 -- 172.21.15.88:0/1630201346 learned_addr learned my addr 172.21.15.88:0/1630201346 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.071+0000 7fd4effff700 1 -- 172.21.15.88:0/1630201346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0009550 con 0x7fd4f0104040 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.072+0000 7fd4effff700 1 --2- 172.21.15.88:0/1630201346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0198610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0003f80 tx=0x7fd4e0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.072+0000 7fd4ed7fa700 1 -- 172.21.15.88:0/1630201346 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4e0003a80 con 0x7fd4f0104040 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.072+0000 7fd4ed7fa700 1 -- 172.21.15.88:0/1630201346 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd4e0003be0 con 0x7fd4f0104040 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.072+0000 7fd4f64da700 1 -- 172.21.15.88:0/1630201346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f0198d50 con 0x7fd4f0104040 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.072+0000 7fd4ed7fa700 1 -- 172.21.15.88:0/1630201346 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd4e0019820 con 0x7fd4f0104040 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.072+0000 7fd4f64da700 1 -- 172.21.15.88:0/1630201346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f01991f0 con 0x7fd4f0104040 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.072+0000 7fd4ed7fa700 1 -- 172.21.15.88:0/1630201346 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7fd4e0019a50 con 0x7fd4f0104040 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.073+0000 7fd4ed7fa700 1 --2- 172.21.15.88:0/1630201346 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7fd4dc0383a0 0x7fd4dc03a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.073+0000 7fd4ed7fa700 1 -- 172.21.15.88:0/1630201346 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd4e004c570 con 0x7fd4f0104040 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.073+0000 7fd4f64da700 1 -- 172.21.15.88:0/1630201346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f0192540 con 0x7fd4f0104040 2024-02-19T18:50:59.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.073+0000 7fd4ef7fe700 1 --2- 172.21.15.88:0/1630201346 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7fd4dc0383a0 0x7fd4dc03a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.073+0000 7fd4ef7fe700 1 --2- 172.21.15.88:0/1630201346 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7fd4dc0383a0 0x7fd4dc03a850 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8006fd0 tx=0x7fd4d8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.077+0000 7fd4ed7fa700 1 -- 172.21.15.88:0/1630201346 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd4e002c3a0 con 0x7fd4f0104040 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.320+0000 7fd4f64da700 1 -- 172.21.15.88:0/1630201346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fd4f002d050 con 0x7fd4f0104040 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.872+0000 7fd4ed7fa700 1 -- 172.21.15.88:0/1630201346 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fd4e000fda0 con 0x7fd4f0104040 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.873+0000 7fd4ed7fa700 1 -- 172.21.15.88:0/1630201346 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fd4e004bf90 con 0x7fd4f0104040 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.875+0000 7fd4f64da700 1 -- 172.21.15.88:0/1630201346 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7fd4dc0383a0 msgr2=0x7fd4dc03a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.875+0000 7fd4f64da700 1 --2- 172.21.15.88:0/1630201346 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7fd4dc0383a0 0x7fd4dc03a850 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8006fd0 tx=0x7fd4d8006e40 comp rx=0 tx=0).stop 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.875+0000 7fd4f64da700 1 -- 172.21.15.88:0/1630201346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 msgr2=0x7fd4f0198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.875+0000 7fd4f64da700 1 --2- 172.21.15.88:0/1630201346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0198610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0003f80 tx=0x7fd4e0011ab0 comp rx=0 tx=0).stop 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.875+0000 7fd4f64da700 1 -- 172.21.15.88:0/1630201346 shutdown_connections 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.875+0000 7fd4f64da700 1 --2- 172.21.15.88:0/1630201346 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7fd4dc0383a0 0x7fd4dc03a850 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.875+0000 7fd4f64da700 1 --2- 172.21.15.88:0/1630201346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0104040 0x7fd4f0198610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.875+0000 7fd4f64da700 1 -- 172.21.15.88:0/1630201346 >> 172.21.15.88:0/1630201346 conn(0x7fd4f00ff670 msgr2=0x7fd4f018f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.875+0000 7fd4f64da700 1 -- 172.21.15.88:0/1630201346 shutdown_connections 2024-02-19T18:50:59.471 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:58.875+0000 7fd4f64da700 1 -- 172.21.15.88:0/1630201346 wait complete. 2024-02-19T18:51:00.074 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:59 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1630201346' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-19T18:51:00.075 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:50:59 smithi088 ceph-mon[29272]: mgrmap e5: smithi088.gyviqq(active, since 6s) 2024-02-19T18:51:00.718 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout { 2024-02-19T18:51:00.718 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-19T18:51:00.718 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-19T18:51:00.718 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "active_name": "smithi088.gyviqq", 2024-02-19T18:51:00.718 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-19T18:51:00.718 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout } 2024-02-19T18:51:00.718 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.853+0000 7f22d8b86700 1 Processor -- start 2024-02-19T18:51:00.718 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.854+0000 7f22d8b86700 1 -- start start 2024-02-19T18:51:00.718 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.854+0000 7f22d8b86700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d41040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:00.718 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.854+0000 7f22d8b86700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4104600 con 0x7f22d4103cb0 2024-02-19T18:51:00.718 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.854+0000 7f22d259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d41040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.854+0000 7f22d259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d41040c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38142/0 (socket says 172.21.15.88:38142) 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.854+0000 7f22d259c700 1 -- 172.21.15.88:0/4114578063 learned_addr learned my addr 172.21.15.88:0/4114578063 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.854+0000 7f22d259c700 1 -- 172.21.15.88:0/4114578063 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d4104740 con 0x7f22d4103cb0 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.854+0000 7f22d259c700 1 --2- 172.21.15.88:0/4114578063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d41040c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22c40098a0 tx=0x7f22c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bdf7ae0aa7c6f59 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.855+0000 7f22d159a700 1 -- 172.21.15.88:0/4114578063 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f22c4004180 con 0x7f22d4103cb0 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.855+0000 7f22d159a700 1 -- 172.21.15.88:0/4114578063 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22c40042e0 con 0x7f22d4103cb0 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.855+0000 7f22d159a700 1 -- 172.21.15.88:0/4114578063 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f22c4011a50 con 0x7f22d4103cb0 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.855+0000 7f22d8b86700 1 -- 172.21.15.88:0/4114578063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 msgr2=0x7f22d41040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.855+0000 7f22d8b86700 1 --2- 172.21.15.88:0/4114578063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d41040c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22c40098a0 tx=0x7f22c4009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.855+0000 7f22d8b86700 1 -- 172.21.15.88:0/4114578063 shutdown_connections 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.855+0000 7f22d8b86700 1 --2- 172.21.15.88:0/4114578063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d41040c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.855+0000 7f22d8b86700 1 -- 172.21.15.88:0/4114578063 >> 172.21.15.88:0/4114578063 conn(0x7f22d40ff670 msgr2=0x7f22d4101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.855+0000 7f22d8b86700 1 -- 172.21.15.88:0/4114578063 shutdown_connections 2024-02-19T18:51:00.719 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.855+0000 7f22d8b86700 1 -- 172.21.15.88:0/4114578063 wait complete. 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.856+0000 7f22d8b86700 1 Processor -- start 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.856+0000 7f22d8b86700 1 -- start start 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.856+0000 7f22d8b86700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d4199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.856+0000 7f22d8b86700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4199c90 con 0x7f22d4103cb0 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.856+0000 7f22d259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d4199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.856+0000 7f22d259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d4199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38146/0 (socket says 172.21.15.88:38146) 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.856+0000 7f22d259c700 1 -- 172.21.15.88:0/1932833156 learned_addr learned my addr 172.21.15.88:0/1932833156 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.857+0000 7f22d259c700 1 -- 172.21.15.88:0/1932833156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22c4009550 con 0x7f22d4103cb0 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.857+0000 7f22d259c700 1 --2- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d4199750 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009520 tx=0x7f22c4011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.857+0000 7f22c37fe700 1 -- 172.21.15.88:0/1932833156 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f22c4003a80 con 0x7f22d4103cb0 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.857+0000 7f22c37fe700 1 -- 172.21.15.88:0/1932833156 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f22c4003be0 con 0x7f22d4103cb0 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.857+0000 7f22c37fe700 1 -- 172.21.15.88:0/1932833156 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f22c40197b0 con 0x7f22d4103cb0 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.857+0000 7f22d8b86700 1 -- 172.21.15.88:0/1932833156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d4199e90 con 0x7f22d4103cb0 2024-02-19T18:51:00.720 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.857+0000 7f22d8b86700 1 -- 172.21.15.88:0/1932833156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d419a330 con 0x7f22d4103cb0 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.858+0000 7f22c37fe700 1 -- 172.21.15.88:0/1932833156 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f22c4019a70 con 0x7f22d4103cb0 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.858+0000 7f22c37fe700 1 --2- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f22bc0383a0 0x7f22bc03a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.858+0000 7f22c37fe700 1 -- 172.21.15.88:0/1932833156 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f22c401a430 con 0x7f22d4103cb0 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.858+0000 7f22d1d9b700 1 -- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f22bc0383a0 msgr2=0x7f22bc03a850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/681311088 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.858+0000 7f22d1d9b700 1 --2- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f22bc0383a0 0x7f22bc03a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.858+0000 7f22d8b86700 1 -- 172.21.15.88:0/1932833156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d4062380 con 0x7f22d4103cb0 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:50:59.863+0000 7f22c37fe700 1 -- 172.21.15.88:0/1932833156 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f22c402c3a0 con 0x7f22d4103cb0 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.059+0000 7f22d1d9b700 1 -- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f22bc0383a0 msgr2=0x7f22bc03a850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/681311088 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.059+0000 7f22d1d9b700 1 --2- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f22bc0383a0 0x7f22bc03a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.117+0000 7f22d8b86700 1 -- 172.21.15.88:0/1932833156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f22d419ccf0 con 0x7f22d4103cb0 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.118+0000 7f22c37fe700 1 -- 172.21.15.88:0/1932833156 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f22c4018350 con 0x7f22d4103cb0 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.120+0000 7f22d8b86700 1 -- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f22bc0383a0 msgr2=0x7f22bc03a850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.120+0000 7f22d8b86700 1 --2- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f22bc0383a0 0x7f22bc03a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:00.721 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.120+0000 7f22d8b86700 1 -- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 msgr2=0x7f22d4199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:00.722 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.120+0000 7f22d8b86700 1 --2- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d4199750 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009520 tx=0x7f22c4011ab0 comp rx=0 tx=0).stop 2024-02-19T18:51:00.722 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.120+0000 7f22d8b86700 1 -- 172.21.15.88:0/1932833156 shutdown_connections 2024-02-19T18:51:00.722 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.120+0000 7f22d8b86700 1 --2- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f22bc0383a0 0x7f22bc03a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:00.722 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.120+0000 7f22d8b86700 1 --2- 172.21.15.88:0/1932833156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4103cb0 0x7f22d4199750 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:00.722 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.120+0000 7f22d8b86700 1 -- 172.21.15.88:0/1932833156 >> 172.21.15.88:0/1932833156 conn(0x7f22d40ff670 msgr2=0x7f22d4100da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:00.722 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.120+0000 7f22d8b86700 1 -- 172.21.15.88:0/1932833156 shutdown_connections 2024-02-19T18:51:00.722 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:00.120+0000 7f22d8b86700 1 -- 172.21.15.88:0/1932833156 wait complete. 2024-02-19T18:51:00.722 INFO:teuthology.orchestra.run.smithi088.stdout:Waiting for the mgr to restart... 2024-02-19T18:51:00.722 INFO:teuthology.orchestra.run.smithi088.stdout:Waiting for mgr epoch 5... 2024-02-19T18:51:00.963 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:00 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1932833156' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-19T18:51:05.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: Active manager daemon smithi088.gyviqq restarted 2024-02-19T18:51:05.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: Activating manager daemon smithi088.gyviqq 2024-02-19T18:51:05.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: osdmap e2: 0 total, 0 up, 0 in 2024-02-19T18:51:05.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: mgrmap e6: smithi088.gyviqq(active, starting, since 0.00334955s) 2024-02-19T18:51:05.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T18:51:05.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi088.gyviqq", "id": "smithi088.gyviqq"}]: dispatch 2024-02-19T18:51:05.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T18:51:05.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T18:51:05.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T18:51:05.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: Manager daemon smithi088.gyviqq is now available 2024-02-19T18:51:05.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:05.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:05.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:51:05.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:51:05.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:05 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:51:06.798 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:06 smithi088 ceph-mon[29272]: Found migration_current of "None". Setting to last migration. 2024-02-19T18:51:06.798 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:06 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/mirror_snapshot_schedule"}]: dispatch 2024-02-19T18:51:06.798 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:06 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/trash_purge_schedule"}]: dispatch 2024-02-19T18:51:06.798 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:06 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:06.798 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:06 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:06.798 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:06 smithi088 ceph-mon[29272]: mgrmap e7: smithi088.gyviqq(active, since 1.00516s) 2024-02-19T18:51:07.060 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout { 2024-02-19T18:51:07.060 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-19T18:51:07.060 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout } 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.128+0000 7f25fd903700 1 Processor -- start 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.128+0000 7f25fd903700 1 -- start start 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.128+0000 7f25fd903700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8074490 0x7f25f8108580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.128+0000 7f25fd903700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f8108ac0 con 0x7f25f8074490 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.128+0000 7f25f6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8074490 0x7f25f8108580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.129+0000 7f25f6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8074490 0x7f25f8108580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38150/0 (socket says 172.21.15.88:38150) 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.129+0000 7f25f6ffd700 1 -- 172.21.15.88:0/3609024165 learned_addr learned my addr 172.21.15.88:0/3609024165 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.129+0000 7f25f6ffd700 1 -- 172.21.15.88:0/3609024165 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f8108c00 con 0x7f25f8074490 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.129+0000 7f25f6ffd700 1 --2- 172.21.15.88:0/3609024165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8074490 0x7f25f8108580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0098a0 tx=0x7f25ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8539bf9f0fc21529 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.129+0000 7f25f5ffb700 1 -- 172.21.15.88:0/3609024165 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ec004180 con 0x7f25f8074490 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.129+0000 7f25f5ffb700 1 -- 172.21.15.88:0/3609024165 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25ec0042e0 con 0x7f25f8074490 2024-02-19T18:51:07.061 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.129+0000 7f25f5ffb700 1 -- 172.21.15.88:0/3609024165 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ec011a50 con 0x7f25f8074490 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.130+0000 7f25fd903700 1 -- 172.21.15.88:0/3609024165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8074490 msgr2=0x7f25f8108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.130+0000 7f25fd903700 1 --2- 172.21.15.88:0/3609024165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8074490 0x7f25f8108580 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f25ec0098a0 tx=0x7f25ec009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.130+0000 7f25fd903700 1 -- 172.21.15.88:0/3609024165 shutdown_connections 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.130+0000 7f25fd903700 1 --2- 172.21.15.88:0/3609024165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8074490 0x7f25f8108580 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.130+0000 7f25fd903700 1 -- 172.21.15.88:0/3609024165 >> 172.21.15.88:0/3609024165 conn(0x7f25f80fff10 msgr2=0x7f25f8102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.130+0000 7f25fd903700 1 -- 172.21.15.88:0/3609024165 shutdown_connections 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.130+0000 7f25fd903700 1 -- 172.21.15.88:0/3609024165 wait complete. 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.130+0000 7f25fd903700 1 Processor -- start 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.131+0000 7f25fd903700 1 -- start start 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.131+0000 7f25fd903700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8105e90 0x7f25f8104520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.131+0000 7f25fd903700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f81062a0 con 0x7f25f8105e90 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.131+0000 7f25f6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8105e90 0x7f25f8104520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.131+0000 7f25f6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8105e90 0x7f25f8104520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38162/0 (socket says 172.21.15.88:38162) 2024-02-19T18:51:07.062 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.131+0000 7f25f6ffd700 1 -- 172.21.15.88:0/4208301699 learned_addr learned my addr 172.21.15.88:0/4208301699 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.131+0000 7f25f6ffd700 1 -- 172.21.15.88:0/4208301699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ec009550 con 0x7f25f8105e90 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.131+0000 7f25f6ffd700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8105e90 0x7f25f8104520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25ec003de0 tx=0x7f25ec011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.131+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ec003930 con 0x7f25f8105e90 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.131+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f25ec003a90 con 0x7f25f8105e90 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.131+0000 7f25fd903700 1 -- 172.21.15.88:0/4208301699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f8104a60 con 0x7f25f8105e90 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.132+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25ec0196c0 con 0x7f25f8105e90 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.132+0000 7f25fd903700 1 -- 172.21.15.88:0/4208301699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f8104f00 con 0x7f25f8105e90 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.132+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f25ec019990 con 0x7f25f8105e90 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.132+0000 7f25fc901700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.132+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 --> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25e003b000 con 0x7f25e0038440 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.132+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f25ec04c060 con 0x7f25f8105e90 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.132+0000 7f25f67fc700 1 -- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 msgr2=0x7f25e003a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/681311088 2024-02-19T18:51:07.063 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.133+0000 7f25f67fc700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.334+0000 7f25f67fc700 1 -- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 msgr2=0x7f25e003a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/681311088 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.334+0000 7f25f67fc700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.735+0000 7f25f67fc700 1 -- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 msgr2=0x7f25e003a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/681311088 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:01.735+0000 7f25f67fc700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:02.536+0000 7f25f67fc700 1 -- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 msgr2=0x7f25e003a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/681311088 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:02.536+0000 7f25f67fc700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:04.137+0000 7f25f67fc700 1 -- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 msgr2=0x7f25e003a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/681311088 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:04.137+0000 7f25f67fc700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:05.486+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mgrmap(e 6) v1 ==== 44851+0+0 (secure 0 0 0) 0x7f25ec01abc0 con 0x7f25f8105e90 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:05.486+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 msgr2=0x7f25e003a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:05.486+0000 7f25fc901700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.487+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7f25ec04db30 con 0x7f25f8105e90 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.487+0000 7f25fc901700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f25e003bd10 0x7f25e003e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:07.064 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.487+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25e003b000 con 0x7f25e003bd10 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.488+0000 7f25f67fc700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f25e003bd10 0x7f25e003e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.488+0000 7f25f67fc700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f25e003bd10 0x7f25e003e0f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f25e8003a10 tx=0x7f25e80092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.489+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f25e003b000 con 0x7f25e003bd10 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.495+0000 7f25fd903700 1 -- 172.21.15.88:0/4208301699 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f25f8105a70 con 0x7f25e003bd10 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.495+0000 7f25fc901700 1 -- 172.21.15.88:0/4208301699 <== mgr.14120 v2:172.21.15.88:6800/3414266099 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f25f8105a70 con 0x7f25e003bd10 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.495+0000 7f25fd903700 1 -- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f25e003bd10 msgr2=0x7f25e003e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.495+0000 7f25fd903700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f25e003bd10 0x7f25e003e0f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f25e8003a10 tx=0x7f25e80092b0 comp rx=0 tx=0).stop 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.495+0000 7f25fd903700 1 -- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8105e90 msgr2=0x7f25f8104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.495+0000 7f25fd903700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8105e90 0x7f25f8104520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f25ec003de0 tx=0x7f25ec011ab0 comp rx=0 tx=0).stop 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.496+0000 7f25fd903700 1 -- 172.21.15.88:0/4208301699 shutdown_connections 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.496+0000 7f25fd903700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f25e003bd10 0x7f25e003e0f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.496+0000 7f25fd903700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25f8105e90 0x7f25f8104520 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:07.065 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.496+0000 7f25fd903700 1 --2- 172.21.15.88:0/4208301699 >> [v2:172.21.15.88:6800/681311088,v1:172.21.15.88:6801/681311088] conn(0x7f25e0038440 0x7f25e003a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:07.066 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.496+0000 7f25fd903700 1 -- 172.21.15.88:0/4208301699 >> 172.21.15.88:0/4208301699 conn(0x7f25f80fff10 msgr2=0x7f25f81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:07.066 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.496+0000 7f25fd903700 1 -- 172.21.15.88:0/4208301699 shutdown_connections 2024-02-19T18:51:07.066 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:06.496+0000 7f25fd903700 1 -- 172.21.15.88:0/4208301699 wait complete. 2024-02-19T18:51:07.066 INFO:teuthology.orchestra.run.smithi088.stdout:mgr epoch 5 is available 2024-02-19T18:51:07.066 INFO:teuthology.orchestra.run.smithi088.stdout:Setting orchestrator backend to cephadm... 2024-02-19T18:51:08.029 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:07 smithi088 ceph-mon[29272]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-19T18:51:08.030 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:07 smithi088 ceph-mon[29272]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-19T18:51:08.030 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:07 smithi088 ceph-mon[29272]: [19/Feb/2024:18:51:06] ENGINE Bus STARTING 2024-02-19T18:51:08.030 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:07 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:51:08.030 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:07 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:08.030 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:07 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:51:08.305 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.494+0000 7f6d48050700 1 Processor -- start 2024-02-19T18:51:08.306 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.494+0000 7f6d48050700 1 -- start start 2024-02-19T18:51:08.306 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.494+0000 7f6d48050700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:08.306 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.494+0000 7f6d48050700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d40105210 con 0x7f6d401048c0 2024-02-19T18:51:08.306 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.494+0000 7f6d45dec700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:08.306 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.494+0000 7f6d45dec700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42212/0 (socket says 172.21.15.88:42212) 2024-02-19T18:51:08.306 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.495+0000 7f6d45dec700 1 -- 172.21.15.88:0/941897546 learned_addr learned my addr 172.21.15.88:0/941897546 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:08.306 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.495+0000 7f6d45dec700 1 -- 172.21.15.88:0/941897546 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d40105350 con 0x7f6d401048c0 2024-02-19T18:51:08.306 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.495+0000 7f6d45dec700 1 --2- 172.21.15.88:0/941897546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40104cd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0098a0 tx=0x7f6d3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cb3cd96ee43ee5 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:08.306 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.495+0000 7f6d44dea700 1 -- 172.21.15.88:0/941897546 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d3c004180 con 0x7f6d401048c0 2024-02-19T18:51:08.306 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.495+0000 7f6d44dea700 1 -- 172.21.15.88:0/941897546 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6d3c0042e0 con 0x7f6d401048c0 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.495+0000 7f6d44dea700 1 -- 172.21.15.88:0/941897546 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d3c011a50 con 0x7f6d401048c0 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.496+0000 7f6d48050700 1 -- 172.21.15.88:0/941897546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 msgr2=0x7f6d40104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.496+0000 7f6d48050700 1 --2- 172.21.15.88:0/941897546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40104cd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0098a0 tx=0x7f6d3c009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.496+0000 7f6d48050700 1 -- 172.21.15.88:0/941897546 shutdown_connections 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.496+0000 7f6d48050700 1 --2- 172.21.15.88:0/941897546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40104cd0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.496+0000 7f6d48050700 1 -- 172.21.15.88:0/941897546 >> 172.21.15.88:0/941897546 conn(0x7f6d400ffed0 msgr2=0x7f6d40102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.496+0000 7f6d48050700 1 -- 172.21.15.88:0/941897546 shutdown_connections 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.496+0000 7f6d48050700 1 -- 172.21.15.88:0/941897546 wait complete. 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.496+0000 7f6d48050700 1 Processor -- start 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.496+0000 7f6d48050700 1 -- start start 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d48050700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40079b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d48050700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4019c400 con 0x7f6d401048c0 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d45dec700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40079b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:08.307 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d45dec700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40079b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42228/0 (socket says 172.21.15.88:42228) 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d45dec700 1 -- 172.21.15.88:0/2022960790 learned_addr learned my addr 172.21.15.88:0/2022960790 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d45dec700 1 -- 172.21.15.88:0/2022960790 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c009550 con 0x7f6d401048c0 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d45dec700 1 --2- 172.21.15.88:0/2022960790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40079b60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c003de0 tx=0x7f6d3c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d36ffd700 1 -- 172.21.15.88:0/2022960790 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d3c003930 con 0x7f6d401048c0 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d36ffd700 1 -- 172.21.15.88:0/2022960790 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6d3c003a90 con 0x7f6d401048c0 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d48050700 1 -- 172.21.15.88:0/2022960790 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4019c540 con 0x7f6d401048c0 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d36ffd700 1 -- 172.21.15.88:0/2022960790 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6d3c019730 con 0x7f6d401048c0 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.497+0000 7f6d48050700 1 -- 172.21.15.88:0/2022960790 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4019c9e0 con 0x7f6d401048c0 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.498+0000 7f6d36ffd700 1 -- 172.21.15.88:0/2022960790 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7f6d3c0199b0 con 0x7f6d401048c0 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.498+0000 7f6d36ffd700 1 --2- 172.21.15.88:0/2022960790 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f6d2c0382d0 0x7f6d2c03a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.498+0000 7f6d48050700 1 -- 172.21.15.88:0/2022960790 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d40062380 con 0x7f6d401048c0 2024-02-19T18:51:08.308 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.498+0000 7f6d36ffd700 1 -- 172.21.15.88:0/2022960790 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6d3c04c510 con 0x7f6d401048c0 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.498+0000 7f6d455eb700 1 --2- 172.21.15.88:0/2022960790 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f6d2c0382d0 0x7f6d2c03a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.499+0000 7f6d455eb700 1 --2- 172.21.15.88:0/2022960790 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f6d2c0382d0 0x7f6d2c03a780 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6d30006fd0 tx=0x7f6d30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.503+0000 7f6d36ffd700 1 -- 172.21.15.88:0/2022960790 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6d3c010dc0 con 0x7f6d401048c0 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.710+0000 7f6d48050700 1 -- 172.21.15.88:0/2022960790 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d40102210 con 0x7f6d2c0382d0 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.716+0000 7f6d36ffd700 1 -- 172.21.15.88:0/2022960790 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6d40102210 con 0x7f6d2c0382d0 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.719+0000 7f6d48050700 1 -- 172.21.15.88:0/2022960790 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f6d2c0382d0 msgr2=0x7f6d2c03a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.719+0000 7f6d48050700 1 --2- 172.21.15.88:0/2022960790 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f6d2c0382d0 0x7f6d2c03a780 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6d30006fd0 tx=0x7f6d30006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.719+0000 7f6d48050700 1 -- 172.21.15.88:0/2022960790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 msgr2=0x7f6d40079b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.719+0000 7f6d48050700 1 --2- 172.21.15.88:0/2022960790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40079b60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c003de0 tx=0x7f6d3c011ab0 comp rx=0 tx=0).stop 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.719+0000 7f6d48050700 1 -- 172.21.15.88:0/2022960790 shutdown_connections 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.719+0000 7f6d48050700 1 --2- 172.21.15.88:0/2022960790 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f6d2c0382d0 0x7f6d2c03a780 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:08.309 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.719+0000 7f6d48050700 1 --2- 172.21.15.88:0/2022960790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d401048c0 0x7f6d40079b60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:08.310 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.719+0000 7f6d48050700 1 -- 172.21.15.88:0/2022960790 >> 172.21.15.88:0/2022960790 conn(0x7f6d400ffed0 msgr2=0x7f6d401015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:08.310 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.719+0000 7f6d48050700 1 -- 172.21.15.88:0/2022960790 shutdown_connections 2024-02-19T18:51:08.310 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:07.719+0000 7f6d48050700 1 -- 172.21.15.88:0/2022960790 wait complete. 2024-02-19T18:51:09.177 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:08 smithi088 ceph-mon[29272]: [19/Feb/2024:18:51:06] ENGINE Serving on http://172.21.15.88:8765 2024-02-19T18:51:09.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:08 smithi088 ceph-mon[29272]: [19/Feb/2024:18:51:07] ENGINE Serving on https://172.21.15.88:7150 2024-02-19T18:51:09.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:08 smithi088 ceph-mon[29272]: [19/Feb/2024:18:51:07] ENGINE Bus STARTED 2024-02-19T18:51:09.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:08 smithi088 ceph-mon[29272]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:09.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:08 smithi088 ceph-mon[29272]: mgrmap e8: smithi088.gyviqq(active, since 2s) 2024-02-19T18:51:09.454 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-19T18:51:09.454 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.669+0000 7f44bec02700 1 Processor -- start 2024-02-19T18:51:09.454 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.669+0000 7f44bec02700 1 -- start start 2024-02-19T18:51:09.454 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.669+0000 7f44bec02700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b8105de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:09.454 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.669+0000 7f44bec02700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b8106320 con 0x7f44b81059d0 2024-02-19T18:51:09.454 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.669+0000 7f44bc99e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b8105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:09.454 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.669+0000 7f44bc99e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b8105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42232/0 (socket says 172.21.15.88:42232) 2024-02-19T18:51:09.454 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.669+0000 7f44bc99e700 1 -- 172.21.15.88:0/3433966090 learned_addr learned my addr 172.21.15.88:0/3433966090 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:09.455 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.669+0000 7f44bc99e700 1 -- 172.21.15.88:0/3433966090 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b8106460 con 0x7f44b81059d0 2024-02-19T18:51:09.455 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.670+0000 7f44bc99e700 1 --2- 172.21.15.88:0/3433966090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b8105de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009870 tx=0x7f44ac009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b02291589a69021 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:09.455 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.670+0000 7f44b77fe700 1 -- 172.21.15.88:0/3433966090 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44ac005c30 con 0x7f44b81059d0 2024-02-19T18:51:09.455 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.670+0000 7f44b77fe700 1 -- 172.21.15.88:0/3433966090 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44ac005d90 con 0x7f44b81059d0 2024-02-19T18:51:09.455 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.670+0000 7f44b77fe700 1 -- 172.21.15.88:0/3433966090 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44ac006030 con 0x7f44b81059d0 2024-02-19T18:51:09.455 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.670+0000 7f44bec02700 1 -- 172.21.15.88:0/3433966090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 msgr2=0x7f44b8105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:09.455 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.670+0000 7f44bec02700 1 --2- 172.21.15.88:0/3433966090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b8105de0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009870 tx=0x7f44ac009b80 comp rx=0 tx=0).stop 2024-02-19T18:51:09.455 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.670+0000 7f44bec02700 1 -- 172.21.15.88:0/3433966090 shutdown_connections 2024-02-19T18:51:09.455 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.670+0000 7f44bec02700 1 --2- 172.21.15.88:0/3433966090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b8105de0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:09.455 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.670+0000 7f44bec02700 1 -- 172.21.15.88:0/3433966090 >> 172.21.15.88:0/3433966090 conn(0x7f44b8101020 msgr2=0x7f44b8103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.670+0000 7f44bec02700 1 -- 172.21.15.88:0/3433966090 shutdown_connections 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.670+0000 7f44bec02700 1 -- 172.21.15.88:0/3433966090 wait complete. 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.671+0000 7f44bec02700 1 Processor -- start 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.671+0000 7f44bec02700 1 -- start start 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.671+0000 7f44bec02700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b81996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.671+0000 7f44bec02700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b8199bf0 con 0x7f44b81059d0 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.671+0000 7f44bc99e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b81996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.671+0000 7f44bc99e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b81996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42248/0 (socket says 172.21.15.88:42248) 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.671+0000 7f44bc99e700 1 -- 172.21.15.88:0/3809645881 learned_addr learned my addr 172.21.15.88:0/3809645881 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.671+0000 7f44bc99e700 1 -- 172.21.15.88:0/3809645881 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ac009500 con 0x7f44b81059d0 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.672+0000 7f44bc99e700 1 --2- 172.21.15.88:0/3809645881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b81996b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f44ac006130 tx=0x7f44ac003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:09.456 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.672+0000 7f44b5ffb700 1 -- 172.21.15.88:0/3809645881 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44ac0043d0 con 0x7f44b81059d0 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.672+0000 7f44b5ffb700 1 -- 172.21.15.88:0/3809645881 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f44ac004530 con 0x7f44b81059d0 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.672+0000 7f44bec02700 1 -- 172.21.15.88:0/3809645881 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b8199df0 con 0x7f44b81059d0 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.672+0000 7f44b5ffb700 1 -- 172.21.15.88:0/3809645881 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f44ac011c60 con 0x7f44b81059d0 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.672+0000 7f44bec02700 1 -- 172.21.15.88:0/3809645881 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b819a290 con 0x7f44b81059d0 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.673+0000 7f44b5ffb700 1 -- 172.21.15.88:0/3809645881 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f44ac018070 con 0x7f44b81059d0 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.673+0000 7f44bec02700 1 -- 172.21.15.88:0/3809645881 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b8062380 con 0x7f44b81059d0 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.673+0000 7f44b5ffb700 1 --2- 172.21.15.88:0/3809645881 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f44a8038400 0x7f44a803a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.673+0000 7f44b5ffb700 1 -- 172.21.15.88:0/3809645881 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f44ac04b8b0 con 0x7f44b81059d0 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.673+0000 7f44b7fff700 1 --2- 172.21.15.88:0/3809645881 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f44a8038400 0x7f44a803a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.673+0000 7f44b7fff700 1 --2- 172.21.15.88:0/3809645881 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f44a8038400 0x7f44a803a8b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44a4006fd0 tx=0x7f44a4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.678+0000 7f44b5ffb700 1 -- 172.21.15.88:0/3809645881 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f44ac022350 con 0x7f44b81059d0 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.868+0000 7f44bec02700 1 -- 172.21.15.88:0/3809645881 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f44b819d0d0 con 0x7f44a8038400 2024-02-19T18:51:09.457 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.869+0000 7f44b5ffb700 1 -- 172.21.15.88:0/3809645881 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f44b819d0d0 con 0x7f44a8038400 2024-02-19T18:51:09.458 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.871+0000 7f44bec02700 1 -- 172.21.15.88:0/3809645881 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f44a8038400 msgr2=0x7f44a803a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:09.458 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.871+0000 7f44bec02700 1 --2- 172.21.15.88:0/3809645881 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f44a8038400 0x7f44a803a8b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f44a4006fd0 tx=0x7f44a4006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:09.458 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.871+0000 7f44bec02700 1 -- 172.21.15.88:0/3809645881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 msgr2=0x7f44b81996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:09.458 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.871+0000 7f44bec02700 1 --2- 172.21.15.88:0/3809645881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b81996b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f44ac006130 tx=0x7f44ac003e30 comp rx=0 tx=0).stop 2024-02-19T18:51:09.458 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.871+0000 7f44bec02700 1 -- 172.21.15.88:0/3809645881 shutdown_connections 2024-02-19T18:51:09.458 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.871+0000 7f44bec02700 1 --2- 172.21.15.88:0/3809645881 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f44a8038400 0x7f44a803a8b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:09.458 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.871+0000 7f44bec02700 1 --2- 172.21.15.88:0/3809645881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b81059d0 0x7f44b81996b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:09.458 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.871+0000 7f44bec02700 1 -- 172.21.15.88:0/3809645881 >> 172.21.15.88:0/3809645881 conn(0x7f44b8101020 msgr2=0x7f44b8101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:09.459 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.871+0000 7f44bec02700 1 -- 172.21.15.88:0/3809645881 shutdown_connections 2024-02-19T18:51:09.459 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:08.871+0000 7f44bec02700 1 -- 172.21.15.88:0/3809645881 wait complete. 2024-02-19T18:51:09.459 INFO:teuthology.orchestra.run.smithi088.stdout:Generating ssh key... 2024-02-19T18:51:10.074 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:09 smithi088 ceph-mon[29272]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:10.970 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.863+0000 7fb427d27700 1 Processor -- start 2024-02-19T18:51:10.970 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.863+0000 7fb427d27700 1 -- start start 2024-02-19T18:51:10.970 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.863+0000 7fb427d27700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb4201088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:10.970 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.863+0000 7fb427d27700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb420079d10 con 0x7fb420106510 2024-02-19T18:51:10.970 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.863+0000 7fb425ac3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb4201088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:10.970 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.863+0000 7fb425ac3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb4201088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42260/0 (socket says 172.21.15.88:42260) 2024-02-19T18:51:10.970 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.863+0000 7fb425ac3700 1 -- 172.21.15.88:0/2991958167 learned_addr learned my addr 172.21.15.88:0/2991958167 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:10.970 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.864+0000 7fb425ac3700 1 -- 172.21.15.88:0/2991958167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb420079e50 con 0x7fb420106510 2024-02-19T18:51:10.970 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.864+0000 7fb425ac3700 1 --2- 172.21.15.88:0/2991958167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb4201088f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0098a0 tx=0x7fb41c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133dde2b3892d077 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:10.970 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.864+0000 7fb424ac1700 1 -- 172.21.15.88:0/2991958167 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb41c004180 con 0x7fb420106510 2024-02-19T18:51:10.970 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.864+0000 7fb424ac1700 1 -- 172.21.15.88:0/2991958167 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb41c0042e0 con 0x7fb420106510 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.864+0000 7fb424ac1700 1 -- 172.21.15.88:0/2991958167 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb41c011a50 con 0x7fb420106510 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.864+0000 7fb427d27700 1 -- 172.21.15.88:0/2991958167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 msgr2=0x7fb4201088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.864+0000 7fb427d27700 1 --2- 172.21.15.88:0/2991958167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb4201088f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb41c0098a0 tx=0x7fb41c009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.865+0000 7fb427d27700 1 -- 172.21.15.88:0/2991958167 shutdown_connections 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.865+0000 7fb427d27700 1 --2- 172.21.15.88:0/2991958167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb4201088f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.865+0000 7fb427d27700 1 -- 172.21.15.88:0/2991958167 >> 172.21.15.88:0/2991958167 conn(0x7fb4200ffed0 msgr2=0x7fb420102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.865+0000 7fb427d27700 1 -- 172.21.15.88:0/2991958167 shutdown_connections 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.865+0000 7fb427d27700 1 -- 172.21.15.88:0/2991958167 wait complete. 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.865+0000 7fb427d27700 1 Processor -- start 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.865+0000 7fb427d27700 1 -- start start 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.865+0000 7fb427d27700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb420105e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.865+0000 7fb427d27700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4201063b0 con 0x7fb420106510 2024-02-19T18:51:10.971 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.865+0000 7fb425ac3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb420105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.866+0000 7fb425ac3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb420105e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42262/0 (socket says 172.21.15.88:42262) 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.866+0000 7fb425ac3700 1 -- 172.21.15.88:0/2655286433 learned_addr learned my addr 172.21.15.88:0/2655286433 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.866+0000 7fb425ac3700 1 -- 172.21.15.88:0/2655286433 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c009550 con 0x7fb420106510 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.866+0000 7fb425ac3700 1 --2- 172.21.15.88:0/2655286433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb420105e70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009520 tx=0x7fb41c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.866+0000 7fb416ffd700 1 -- 172.21.15.88:0/2655286433 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb41c003a80 con 0x7fb420106510 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.866+0000 7fb416ffd700 1 -- 172.21.15.88:0/2655286433 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb41c003be0 con 0x7fb420106510 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.866+0000 7fb427d27700 1 -- 172.21.15.88:0/2655286433 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb420104520 con 0x7fb420106510 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.866+0000 7fb416ffd700 1 -- 172.21.15.88:0/2655286433 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb41c019750 con 0x7fb420106510 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.866+0000 7fb427d27700 1 -- 172.21.15.88:0/2655286433 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb420104a20 con 0x7fb420106510 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.867+0000 7fb416ffd700 1 -- 172.21.15.88:0/2655286433 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fb41c0199c0 con 0x7fb420106510 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.867+0000 7fb427d27700 1 -- 172.21.15.88:0/2655286433 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4201099f0 con 0x7fb420106510 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.867+0000 7fb416ffd700 1 --2- 172.21.15.88:0/2655286433 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fb40c0383f0 0x7fb40c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:10.972 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.867+0000 7fb416ffd700 1 -- 172.21.15.88:0/2655286433 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fb41c04c4f0 con 0x7fb420106510 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.867+0000 7fb4252c2700 1 --2- 172.21.15.88:0/2655286433 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fb40c0383f0 0x7fb40c03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.868+0000 7fb4252c2700 1 --2- 172.21.15.88:0/2655286433 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fb40c0383f0 0x7fb40c03a8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb410006fd0 tx=0x7fb410006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:09.872+0000 7fb416ffd700 1 -- 172.21.15.88:0/2655286433 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb41c018b40 con 0x7fb420106510 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.058+0000 7fb427d27700 1 -- 172.21.15.88:0/2655286433 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fb420102240 con 0x7fb40c0383f0 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.339+0000 7fb416ffd700 1 -- 172.21.15.88:0/2655286433 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb420102240 con 0x7fb40c0383f0 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.341+0000 7fb427d27700 1 -- 172.21.15.88:0/2655286433 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fb40c0383f0 msgr2=0x7fb40c03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.341+0000 7fb427d27700 1 --2- 172.21.15.88:0/2655286433 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fb40c0383f0 0x7fb40c03a8a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb410006fd0 tx=0x7fb410006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.341+0000 7fb427d27700 1 -- 172.21.15.88:0/2655286433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 msgr2=0x7fb420105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.341+0000 7fb427d27700 1 --2- 172.21.15.88:0/2655286433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb420105e70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009520 tx=0x7fb41c011ab0 comp rx=0 tx=0).stop 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.341+0000 7fb427d27700 1 -- 172.21.15.88:0/2655286433 shutdown_connections 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.341+0000 7fb427d27700 1 --2- 172.21.15.88:0/2655286433 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fb40c0383f0 0x7fb40c03a8a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.341+0000 7fb427d27700 1 --2- 172.21.15.88:0/2655286433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb420106510 0x7fb420105e70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.341+0000 7fb427d27700 1 -- 172.21.15.88:0/2655286433 >> 172.21.15.88:0/2655286433 conn(0x7fb4200ffed0 msgr2=0x7fb420101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:10.973 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.341+0000 7fb427d27700 1 -- 172.21.15.88:0/2655286433 shutdown_connections 2024-02-19T18:51:10.974 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:10.341+0000 7fb427d27700 1 -- 172.21.15.88:0/2655286433 wait complete. 2024-02-19T18:51:11.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:11 smithi088 ceph-mon[29272]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:11.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:11 smithi088 ceph-mon[29272]: Generating ssh key... 2024-02-19T18:51:11.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:11 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:11.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:11 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:12.196 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHbHg37EvC00L1Olql/fBtAg6CTlxVZS8fA27ktTQzlB576J6Zrss1tPdn7/R5m3QL/E9fSuVDMFja8LWW7hUy4PVygvCuEeoXWalz+wQa7Xj6IMRnnL4L6pJGlKZJ9d2XoAxUkYrTcSdy8Z4N8Q5R6cYsx/lBBV/PL7mxRq/Wx7TeevSKvDkDEsaY6uSfDtviBDDzXW0bmgKdUnVYA58ddb5JtzFKsfcTSZmPIETWE5/fb/iFqVSz0qphk8g+6PE3zww+1fVoNkROabHRWwTFpC5oiYg25V/879sbszrqWiyd7y+p5hAhwc0VSSTT0Hr5lhjB9AShx+4IVLJImhWW+keXH+P6LiwgdCRsjcS6YwsUt/vi+JMRTP98l6dVMP57x/VAZsMbZSoyTYdquzhe8u/4g0/AYHuxcyymhYvfjA7DQe+3jn4JKfyeeYLFBLMU776a7pM1UbrzVzjzzVJwLWV7YfmlAimklQpxG2YqVJVtnbGQaTKS0VsVFU2M5tE= ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.358+0000 7f3d624cc700 1 Processor -- start 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.358+0000 7f3d624cc700 1 -- start start 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.358+0000 7f3d624cc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c105dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.358+0000 7f3d624cc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c106300 con 0x7f3d5c1059b0 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.358+0000 7f3d5bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c105dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.358+0000 7f3d5bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c105dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42278/0 (socket says 172.21.15.88:42278) 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.358+0000 7f3d5bfff700 1 -- 172.21.15.88:0/407634241 learned_addr learned my addr 172.21.15.88:0/407634241 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.358+0000 7f3d5bfff700 1 -- 172.21.15.88:0/407634241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d5c106440 con 0x7f3d5c1059b0 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.359+0000 7f3d5bfff700 1 --2- 172.21.15.88:0/407634241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c105dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0098a0 tx=0x7f3d4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ef951c7e91100 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.359+0000 7f3d5affd700 1 -- 172.21.15.88:0/407634241 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d4c004180 con 0x7f3d5c1059b0 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.359+0000 7f3d5affd700 1 -- 172.21.15.88:0/407634241 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3d4c0042e0 con 0x7f3d5c1059b0 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.359+0000 7f3d5affd700 1 -- 172.21.15.88:0/407634241 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d4c011a50 con 0x7f3d5c1059b0 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.359+0000 7f3d624cc700 1 -- 172.21.15.88:0/407634241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 msgr2=0x7f3d5c105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:12.197 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.359+0000 7f3d624cc700 1 --2- 172.21.15.88:0/407634241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c105dc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0098a0 tx=0x7f3d4c009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.359+0000 7f3d624cc700 1 -- 172.21.15.88:0/407634241 shutdown_connections 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.359+0000 7f3d624cc700 1 --2- 172.21.15.88:0/407634241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c105dc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.359+0000 7f3d624cc700 1 -- 172.21.15.88:0/407634241 >> 172.21.15.88:0/407634241 conn(0x7f3d5c101020 msgr2=0x7f3d5c103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.360+0000 7f3d624cc700 1 -- 172.21.15.88:0/407634241 shutdown_connections 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.360+0000 7f3d624cc700 1 -- 172.21.15.88:0/407634241 wait complete. 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.360+0000 7f3d624cc700 1 Processor -- start 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.360+0000 7f3d624cc700 1 -- start start 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.360+0000 7f3d624cc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.360+0000 7f3d624cc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5c199bf0 con 0x7f3d5c1059b0 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.361+0000 7f3d5bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.361+0000 7f3d5bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42280/0 (socket says 172.21.15.88:42280) 2024-02-19T18:51:12.198 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.361+0000 7f3d5bfff700 1 -- 172.21.15.88:0/3840431387 learned_addr learned my addr 172.21.15.88:0/3840431387 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.361+0000 7f3d5bfff700 1 -- 172.21.15.88:0/3840431387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d4c009550 con 0x7f3d5c1059b0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.361+0000 7f3d5bfff700 1 --2- 172.21.15.88:0/3840431387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c1996b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0119e0 tx=0x7f3d4c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.361+0000 7f3d597fa700 1 -- 172.21.15.88:0/3840431387 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d4c003b90 con 0x7f3d5c1059b0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.361+0000 7f3d624cc700 1 -- 172.21.15.88:0/3840431387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5c199df0 con 0x7f3d5c1059b0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.361+0000 7f3d597fa700 1 -- 172.21.15.88:0/3840431387 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3d4c01a8a0 con 0x7f3d5c1059b0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.361+0000 7f3d597fa700 1 -- 172.21.15.88:0/3840431387 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3d4c019930 con 0x7f3d5c1059b0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.361+0000 7f3d624cc700 1 -- 172.21.15.88:0/3840431387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5c19a290 con 0x7f3d5c1059b0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.362+0000 7f3d597fa700 1 -- 172.21.15.88:0/3840431387 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f3d4c019ba0 con 0x7f3d5c1059b0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.362+0000 7f3d624cc700 1 -- 172.21.15.88:0/3840431387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5c062380 con 0x7f3d5c1059b0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.362+0000 7f3d597fa700 1 --2- 172.21.15.88:0/3840431387 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f3d44038400 0x7f3d4403a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.362+0000 7f3d597fa700 1 -- 172.21.15.88:0/3840431387 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3d4c04c950 con 0x7f3d5c1059b0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.362+0000 7f3d5b7fe700 1 --2- 172.21.15.88:0/3840431387 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f3d44038400 0x7f3d4403a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.363+0000 7f3d5b7fe700 1 --2- 172.21.15.88:0/3840431387 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f3d44038400 0x7f3d4403a8b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006fd0 tx=0x7f3d50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.367+0000 7f3d597fa700 1 -- 172.21.15.88:0/3840431387 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3d4c02d440 con 0x7f3d5c1059b0 2024-02-19T18:51:12.199 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.565+0000 7f3d624cc700 1 -- 172.21.15.88:0/3840431387 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d5c102490 con 0x7f3d44038400 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.565+0000 7f3d597fa700 1 -- 172.21.15.88:0/3840431387 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f3d5c102490 con 0x7f3d44038400 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.567+0000 7f3d624cc700 1 -- 172.21.15.88:0/3840431387 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f3d44038400 msgr2=0x7f3d4403a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.567+0000 7f3d624cc700 1 --2- 172.21.15.88:0/3840431387 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f3d44038400 0x7f3d4403a8b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3d50006fd0 tx=0x7f3d50006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.567+0000 7f3d624cc700 1 -- 172.21.15.88:0/3840431387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 msgr2=0x7f3d5c1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.567+0000 7f3d624cc700 1 --2- 172.21.15.88:0/3840431387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c1996b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c0119e0 tx=0x7f3d4c011ac0 comp rx=0 tx=0).stop 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.568+0000 7f3d624cc700 1 -- 172.21.15.88:0/3840431387 shutdown_connections 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.568+0000 7f3d624cc700 1 --2- 172.21.15.88:0/3840431387 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f3d44038400 0x7f3d4403a8b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.568+0000 7f3d624cc700 1 --2- 172.21.15.88:0/3840431387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d5c1059b0 0x7f3d5c1996b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.568+0000 7f3d624cc700 1 -- 172.21.15.88:0/3840431387 >> 172.21.15.88:0/3840431387 conn(0x7f3d5c101020 msgr2=0x7f3d5c101d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.568+0000 7f3d624cc700 1 -- 172.21.15.88:0/3840431387 shutdown_connections 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:11.568+0000 7f3d624cc700 1 -- 172.21.15.88:0/3840431387 wait complete. 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:Adding key to root@localhost authorized_keys... 2024-02-19T18:51:12.200 INFO:teuthology.orchestra.run.smithi088.stdout:Adding host smithi088... 2024-02-19T18:51:13.055 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:12 smithi088 ceph-mon[29272]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:13.959 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:13 smithi088 ceph-mon[29272]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi088", "addr": "172.21.15.88", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:15.095 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:14 smithi088 ceph-mon[29272]: Deploying cephadm binary to smithi088 2024-02-19T18:51:15.885 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout Added host 'smithi088' with addr '172.21.15.88' 2024-02-19T18:51:15.885 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.590+0000 7efd57b23700 1 Processor -- start 2024-02-19T18:51:15.885 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.590+0000 7efd57b23700 1 -- start start 2024-02-19T18:51:15.885 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.590+0000 7efd57b23700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5007a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:15.885 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.590+0000 7efd57b23700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5007a7f0 con 0x7efd5007bdb0 2024-02-19T18:51:15.885 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.590+0000 7efd558bf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5007a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:15.885 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.590+0000 7efd558bf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5007a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42288/0 (socket says 172.21.15.88:42288) 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.590+0000 7efd558bf700 1 -- 172.21.15.88:0/3591217468 learned_addr learned my addr 172.21.15.88:0/3591217468 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.591+0000 7efd558bf700 1 -- 172.21.15.88:0/3591217468 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5007a930 con 0x7efd5007bdb0 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.591+0000 7efd558bf700 1 --2- 172.21.15.88:0/3591217468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5007a2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009870 tx=0x7efd4c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ceeb83f05767d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.591+0000 7efd548bd700 1 -- 172.21.15.88:0/3591217468 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd4c005c30 con 0x7efd5007bdb0 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.591+0000 7efd548bd700 1 -- 172.21.15.88:0/3591217468 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd4c005d90 con 0x7efd5007bdb0 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.591+0000 7efd548bd700 1 -- 172.21.15.88:0/3591217468 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd4c006030 con 0x7efd5007bdb0 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.591+0000 7efd57b23700 1 -- 172.21.15.88:0/3591217468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 msgr2=0x7efd5007a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.591+0000 7efd57b23700 1 --2- 172.21.15.88:0/3591217468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5007a2b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7efd4c009870 tx=0x7efd4c009b80 comp rx=0 tx=0).stop 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd57b23700 1 -- 172.21.15.88:0/3591217468 shutdown_connections 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd57b23700 1 --2- 172.21.15.88:0/3591217468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5007a2b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd57b23700 1 -- 172.21.15.88:0/3591217468 >> 172.21.15.88:0/3591217468 conn(0x7efd50101020 msgr2=0x7efd50103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd57b23700 1 -- 172.21.15.88:0/3591217468 shutdown_connections 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd57b23700 1 -- 172.21.15.88:0/3591217468 wait complete. 2024-02-19T18:51:15.886 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd57b23700 1 Processor -- start 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd57b23700 1 -- start start 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd57b23700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd57b23700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5019be30 con 0x7efd5007bdb0 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd558bf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd558bf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5019b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42304/0 (socket says 172.21.15.88:42304) 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.592+0000 7efd558bf700 1 -- 172.21.15.88:0/2009462411 learned_addr learned my addr 172.21.15.88:0/2009462411 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.593+0000 7efd558bf700 1 -- 172.21.15.88:0/2009462411 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4c009500 con 0x7efd5007bdb0 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.593+0000 7efd558bf700 1 --2- 172.21.15.88:0/2009462411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5019b8f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efd4c006130 tx=0x7efd4c006160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.593+0000 7efd46ffd700 1 -- 172.21.15.88:0/2009462411 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd4c0043b0 con 0x7efd5007bdb0 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.593+0000 7efd46ffd700 1 -- 172.21.15.88:0/2009462411 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7efd4c004510 con 0x7efd5007bdb0 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.593+0000 7efd57b23700 1 -- 172.21.15.88:0/2009462411 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5019c030 con 0x7efd5007bdb0 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.593+0000 7efd46ffd700 1 -- 172.21.15.88:0/2009462411 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd4c011c80 con 0x7efd5007bdb0 2024-02-19T18:51:15.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.593+0000 7efd57b23700 1 -- 172.21.15.88:0/2009462411 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5019c4d0 con 0x7efd5007bdb0 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.594+0000 7efd46ffd700 1 -- 172.21.15.88:0/2009462411 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7efd4c018070 con 0x7efd5007bdb0 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.594+0000 7efd46ffd700 1 --2- 172.21.15.88:0/2009462411 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7efd3c0383b0 0x7efd3c03a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.594+0000 7efd46ffd700 1 -- 172.21.15.88:0/2009462411 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7efd4c04ba70 con 0x7efd5007bdb0 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.594+0000 7efd57b23700 1 -- 172.21.15.88:0/2009462411 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd5010bcc0 con 0x7efd5007bdb0 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.594+0000 7efd550be700 1 --2- 172.21.15.88:0/2009462411 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7efd3c0383b0 0x7efd3c03a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.594+0000 7efd550be700 1 --2- 172.21.15.88:0/2009462411 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7efd3c0383b0 0x7efd3c03a860 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efd40006fd0 tx=0x7efd40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.599+0000 7efd46ffd700 1 -- 172.21.15.88:0/2009462411 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efd4c022760 con 0x7efd5007bdb0 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:12.788+0000 7efd57b23700 1 -- 172.21.15.88:0/2009462411 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi088", "addr": "172.21.15.88", "target": ["mon-mgr", ""]}) v1 -- 0x7efd501024e0 con 0x7efd3c0383b0 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:15.304+0000 7efd46ffd700 1 -- 172.21.15.88:0/2009462411 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7efd501024e0 con 0x7efd3c0383b0 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:15.306+0000 7efd57b23700 1 -- 172.21.15.88:0/2009462411 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7efd3c0383b0 msgr2=0x7efd3c03a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:15.306+0000 7efd57b23700 1 --2- 172.21.15.88:0/2009462411 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7efd3c0383b0 0x7efd3c03a860 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7efd40006fd0 tx=0x7efd40006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:15.306+0000 7efd57b23700 1 -- 172.21.15.88:0/2009462411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 msgr2=0x7efd5019b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:15.306+0000 7efd57b23700 1 --2- 172.21.15.88:0/2009462411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5019b8f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7efd4c006130 tx=0x7efd4c006160 comp rx=0 tx=0).stop 2024-02-19T18:51:15.888 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:15.306+0000 7efd57b23700 1 -- 172.21.15.88:0/2009462411 shutdown_connections 2024-02-19T18:51:15.889 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:15.306+0000 7efd57b23700 1 --2- 172.21.15.88:0/2009462411 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7efd3c0383b0 0x7efd3c03a860 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:15.889 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:15.306+0000 7efd57b23700 1 --2- 172.21.15.88:0/2009462411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd5007bdb0 0x7efd5019b8f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:15.889 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:15.306+0000 7efd57b23700 1 -- 172.21.15.88:0/2009462411 >> 172.21.15.88:0/2009462411 conn(0x7efd50101020 msgr2=0x7efd50101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:15.889 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:15.306+0000 7efd57b23700 1 -- 172.21.15.88:0/2009462411 shutdown_connections 2024-02-19T18:51:15.889 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:15.306+0000 7efd57b23700 1 -- 172.21.15.88:0/2009462411 wait complete. 2024-02-19T18:51:15.889 INFO:teuthology.orchestra.run.smithi088.stdout:Deploying mon service with default placement... 2024-02-19T18:51:16.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:16 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:16.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:16 smithi088 ceph-mon[29272]: Added host smithi088 2024-02-19T18:51:16.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:16 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:51:17.750 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:17 smithi088 ceph-mon[29272]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:17.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:17 smithi088 ceph-mon[29272]: Saving service mon spec with placement count:5 2024-02-19T18:51:17.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:17 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:17.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:17 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:17.756 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-19T18:51:17.756 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.263+0000 7f5dc9f3a700 1 Processor -- start 2024-02-19T18:51:17.756 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.263+0000 7f5dc9f3a700 1 -- start start 2024-02-19T18:51:17.756 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.263+0000 7f5dc9f3a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:17.756 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.263+0000 7f5dc9f3a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc407a830 con 0x7f5dc407bdf0 2024-02-19T18:51:17.756 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.263+0000 7f5dc37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.263+0000 7f5dc37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49328/0 (socket says 172.21.15.88:49328) 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.263+0000 7f5dc37fe700 1 -- 172.21.15.88:0/3430200187 learned_addr learned my addr 172.21.15.88:0/3430200187 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.263+0000 7f5dc37fe700 1 -- 172.21.15.88:0/3430200187 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dc407a970 con 0x7f5dc407bdf0 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.264+0000 7f5dc37fe700 1 --2- 172.21.15.88:0/3430200187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc407a2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009870 tx=0x7f5db4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bb23fc9f8503ab server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.264+0000 7f5dc27fc700 1 -- 172.21.15.88:0/3430200187 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5db4005c30 con 0x7f5dc407bdf0 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.264+0000 7f5dc27fc700 1 -- 172.21.15.88:0/3430200187 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5db4005d90 con 0x7f5dc407bdf0 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.264+0000 7f5dc27fc700 1 -- 172.21.15.88:0/3430200187 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5db4006030 con 0x7f5dc407bdf0 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.264+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3430200187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 msgr2=0x7f5dc407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.264+0000 7f5dc9f3a700 1 --2- 172.21.15.88:0/3430200187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc407a2f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009870 tx=0x7f5db4009b80 comp rx=0 tx=0).stop 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.264+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3430200187 shutdown_connections 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.264+0000 7f5dc9f3a700 1 --2- 172.21.15.88:0/3430200187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc407a2f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.264+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3430200187 >> 172.21.15.88:0/3430200187 conn(0x7f5dc4101020 msgr2=0x7f5dc4103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.264+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3430200187 shutdown_connections 2024-02-19T18:51:17.757 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.264+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3430200187 wait complete. 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.265+0000 7f5dc9f3a700 1 Processor -- start 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.265+0000 7f5dc9f3a700 1 -- start start 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.265+0000 7f5dc9f3a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.265+0000 7f5dc9f3a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dc419be50 con 0x7f5dc407bdf0 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.265+0000 7f5dc37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.265+0000 7f5dc37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc419b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49340/0 (socket says 172.21.15.88:49340) 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.265+0000 7f5dc37fe700 1 -- 172.21.15.88:0/3584849423 learned_addr learned my addr 172.21.15.88:0/3584849423 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.266+0000 7f5dc37fe700 1 -- 172.21.15.88:0/3584849423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db4009500 con 0x7f5dc407bdf0 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.266+0000 7f5dc37fe700 1 --2- 172.21.15.88:0/3584849423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc419b910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5db40038f0 tx=0x7f5db40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.266+0000 7f5dc0ff9700 1 -- 172.21.15.88:0/3584849423 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5db4003e30 con 0x7f5dc407bdf0 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.266+0000 7f5dc0ff9700 1 -- 172.21.15.88:0/3584849423 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f5db4006580 con 0x7f5dc407bdf0 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.266+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3584849423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dc419c050 con 0x7f5dc407bdf0 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.266+0000 7f5dc0ff9700 1 -- 172.21.15.88:0/3584849423 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5db4011b90 con 0x7f5dc407bdf0 2024-02-19T18:51:17.758 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.266+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3584849423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dc419c4f0 con 0x7f5dc407bdf0 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.267+0000 7f5dc0ff9700 1 -- 172.21.15.88:0/3584849423 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f5db4018070 con 0x7f5dc407bdf0 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.267+0000 7f5dc0ff9700 1 --2- 172.21.15.88:0/3584849423 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f5dac0383b0 0x7f5dac03a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.267+0000 7f5dc0ff9700 1 -- 172.21.15.88:0/3584849423 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5db404baa0 con 0x7f5dc407bdf0 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.267+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3584849423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dc410bc80 con 0x7f5dc407bdf0 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.267+0000 7f5dc2ffd700 1 --2- 172.21.15.88:0/3584849423 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f5dac0383b0 0x7f5dac03a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.268+0000 7f5dc2ffd700 1 --2- 172.21.15.88:0/3584849423 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f5dac0383b0 0x7f5dac03a860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5db8006fd0 tx=0x7f5db8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.272+0000 7f5dc0ff9700 1 -- 172.21.15.88:0/3584849423 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5db4022350 con 0x7f5dc407bdf0 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.476+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3584849423 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f5dc419f2d0 con 0x7f5dac0383b0 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.483+0000 7f5dc0ff9700 1 -- 172.21.15.88:0/3584849423 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5dc419f2d0 con 0x7f5dac0383b0 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.485+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3584849423 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f5dac0383b0 msgr2=0x7f5dac03a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.485+0000 7f5dc9f3a700 1 --2- 172.21.15.88:0/3584849423 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f5dac0383b0 0x7f5dac03a860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5db8006fd0 tx=0x7f5db8006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.485+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3584849423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 msgr2=0x7f5dc419b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:17.759 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.485+0000 7f5dc9f3a700 1 --2- 172.21.15.88:0/3584849423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc419b910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5db40038f0 tx=0x7f5db40042e0 comp rx=0 tx=0).stop 2024-02-19T18:51:17.760 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.485+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3584849423 shutdown_connections 2024-02-19T18:51:17.760 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.485+0000 7f5dc9f3a700 1 --2- 172.21.15.88:0/3584849423 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f5dac0383b0 0x7f5dac03a860 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:17.760 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.485+0000 7f5dc9f3a700 1 --2- 172.21.15.88:0/3584849423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc407bdf0 0x7f5dc419b910 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:17.760 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.485+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3584849423 >> 172.21.15.88:0/3584849423 conn(0x7f5dc4101020 msgr2=0x7f5dc4101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:17.760 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.485+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3584849423 shutdown_connections 2024-02-19T18:51:17.760 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:16.485+0000 7f5dc9f3a700 1 -- 172.21.15.88:0/3584849423 wait complete. 2024-02-19T18:51:17.760 INFO:teuthology.orchestra.run.smithi088.stdout:Deploying mgr service with default placement... 2024-02-19T18:51:18.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-19T18:51:18.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.150+0000 7f98a6d0a700 1 Processor -- start 2024-02-19T18:51:18.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.150+0000 7f98a6d0a700 1 -- start start 2024-02-19T18:51:18.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.150+0000 7f98a6d0a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:18.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.150+0000 7f98a6d0a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0105210 con 0x7f98a01048c0 2024-02-19T18:51:18.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.151+0000 7f98a4aa6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:18.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.151+0000 7f98a4aa6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49342/0 (socket says 172.21.15.88:49342) 2024-02-19T18:51:18.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.151+0000 7f98a4aa6700 1 -- 172.21.15.88:0/2308850572 learned_addr learned my addr 172.21.15.88:0/2308850572 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.151+0000 7f98a4aa6700 1 -- 172.21.15.88:0/2308850572 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a0105350 con 0x7f98a01048c0 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.151+0000 7f98a4aa6700 1 --2- 172.21.15.88:0/2308850572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0104cd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98940098a0 tx=0x7f9894009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818c23c5a8d2b804 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.151+0000 7f989f7fe700 1 -- 172.21.15.88:0/2308850572 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9894004180 con 0x7f98a01048c0 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.151+0000 7f989f7fe700 1 -- 172.21.15.88:0/2308850572 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f98940042e0 con 0x7f98a01048c0 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.151+0000 7f989f7fe700 1 -- 172.21.15.88:0/2308850572 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9894011af0 con 0x7f98a01048c0 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.152+0000 7f98a6d0a700 1 -- 172.21.15.88:0/2308850572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 msgr2=0x7f98a0104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.152+0000 7f98a6d0a700 1 --2- 172.21.15.88:0/2308850572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0104cd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f98940098a0 tx=0x7f9894009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.152+0000 7f98a6d0a700 1 -- 172.21.15.88:0/2308850572 shutdown_connections 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.152+0000 7f98a6d0a700 1 --2- 172.21.15.88:0/2308850572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0104cd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.152+0000 7f98a6d0a700 1 -- 172.21.15.88:0/2308850572 >> 172.21.15.88:0/2308850572 conn(0x7f98a00ffed0 msgr2=0x7f98a0102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.152+0000 7f98a6d0a700 1 -- 172.21.15.88:0/2308850572 shutdown_connections 2024-02-19T18:51:18.998 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.152+0000 7f98a6d0a700 1 -- 172.21.15.88:0/2308850572 wait complete. 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.152+0000 7f98a6d0a700 1 Processor -- start 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.152+0000 7f98a6d0a700 1 -- start start 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.152+0000 7f98a6d0a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.152+0000 7f98a6d0a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a0199c80 con 0x7f98a01048c0 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.153+0000 7f98a4aa6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.153+0000 7f98a4aa6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49354/0 (socket says 172.21.15.88:49354) 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.153+0000 7f98a4aa6700 1 -- 172.21.15.88:0/192051181 learned_addr learned my addr 172.21.15.88:0/192051181 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.153+0000 7f98a4aa6700 1 -- 172.21.15.88:0/192051181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9894009550 con 0x7f98a01048c0 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.153+0000 7f98a4aa6700 1 --2- 172.21.15.88:0/192051181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0199740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9894011670 tx=0x7f9894011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.153+0000 7f989dffb700 1 -- 172.21.15.88:0/192051181 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9894003a40 con 0x7f98a01048c0 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.153+0000 7f989dffb700 1 -- 172.21.15.88:0/192051181 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9894003ba0 con 0x7f98a01048c0 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.153+0000 7f989dffb700 1 -- 172.21.15.88:0/192051181 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9894024820 con 0x7f98a01048c0 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.153+0000 7f98a6d0a700 1 -- 172.21.15.88:0/192051181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a0199e80 con 0x7f98a01048c0 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.153+0000 7f98a6d0a700 1 -- 172.21.15.88:0/192051181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a019a320 con 0x7f98a01048c0 2024-02-19T18:51:18.999 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.154+0000 7f989dffb700 1 -- 172.21.15.88:0/192051181 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f989401ac70 con 0x7f98a01048c0 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.154+0000 7f98a6d0a700 1 -- 172.21.15.88:0/192051181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a0062380 con 0x7f98a01048c0 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.154+0000 7f989dffb700 1 --2- 172.21.15.88:0/192051181 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f98900383b0 0x7f989003a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.154+0000 7f989dffb700 1 -- 172.21.15.88:0/192051181 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f989404c360 con 0x7f98a01048c0 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.154+0000 7f989ffff700 1 --2- 172.21.15.88:0/192051181 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f98900383b0 0x7f989003a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.155+0000 7f989ffff700 1 --2- 172.21.15.88:0/192051181 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f98900383b0 0x7f989003a860 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f988c006fd0 tx=0x7f988c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.159+0000 7f989dffb700 1 -- 172.21.15.88:0/192051181 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f989402cb40 con 0x7f98a01048c0 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.382+0000 7f98a6d0a700 1 -- 172.21.15.88:0/192051181 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f98a019d160 con 0x7f98900383b0 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.387+0000 7f989dffb700 1 -- 172.21.15.88:0/192051181 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f98a019d160 con 0x7f98900383b0 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.389+0000 7f98a6d0a700 1 -- 172.21.15.88:0/192051181 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f98900383b0 msgr2=0x7f989003a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.389+0000 7f98a6d0a700 1 --2- 172.21.15.88:0/192051181 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f98900383b0 0x7f989003a860 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f988c006fd0 tx=0x7f988c006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.389+0000 7f98a6d0a700 1 -- 172.21.15.88:0/192051181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 msgr2=0x7f98a0199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.389+0000 7f98a6d0a700 1 --2- 172.21.15.88:0/192051181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0199740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9894011670 tx=0x7f9894011aa0 comp rx=0 tx=0).stop 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.389+0000 7f98a6d0a700 1 -- 172.21.15.88:0/192051181 shutdown_connections 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.389+0000 7f98a6d0a700 1 --2- 172.21.15.88:0/192051181 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f98900383b0 0x7f989003a860 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:19.000 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.389+0000 7f98a6d0a700 1 --2- 172.21.15.88:0/192051181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98a01048c0 0x7f98a0199740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:19.001 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.389+0000 7f98a6d0a700 1 -- 172.21.15.88:0/192051181 >> 172.21.15.88:0/192051181 conn(0x7f98a00ffed0 msgr2=0x7f98a0079870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:19.001 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.390+0000 7f98a6d0a700 1 -- 172.21.15.88:0/192051181 shutdown_connections 2024-02-19T18:51:19.001 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:18.390+0000 7f98a6d0a700 1 -- 172.21.15.88:0/192051181 wait complete. 2024-02-19T18:51:19.001 INFO:teuthology.orchestra.run.smithi088.stdout:Deploying crash service with default placement... 2024-02-19T18:51:19.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:19 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:19.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:19 smithi088 ceph-mon[29272]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:19.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:19 smithi088 ceph-mon[29272]: Saving service mgr spec with placement count:2 2024-02-19T18:51:19.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:19 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:20.773 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-02-19T18:51:20.773 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.383+0000 7f03db4f4700 1 Processor -- start 2024-02-19T18:51:20.773 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.383+0000 7f03db4f4700 1 -- start start 2024-02-19T18:51:20.773 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.383+0000 7f03db4f4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d41085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:20.773 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.383+0000 7f03db4f4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4108b10 con 0x7f03d4079c70 2024-02-19T18:51:20.773 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.383+0000 7f03d9290700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d41085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:20.773 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.383+0000 7f03d9290700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d41085d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49358/0 (socket says 172.21.15.88:49358) 2024-02-19T18:51:20.773 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.383+0000 7f03d9290700 1 -- 172.21.15.88:0/4153671608 learned_addr learned my addr 172.21.15.88:0/4153671608 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:20.773 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.383+0000 7f03d9290700 1 -- 172.21.15.88:0/4153671608 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4108c50 con 0x7f03d4079c70 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.384+0000 7f03d9290700 1 --2- 172.21.15.88:0/4153671608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d41085d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f03d00098a0 tx=0x7f03d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b076de0455fb3d00 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.384+0000 7f03cbfff700 1 -- 172.21.15.88:0/4153671608 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03d0004180 con 0x7f03d4079c70 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.384+0000 7f03cbfff700 1 -- 172.21.15.88:0/4153671608 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f03d00042e0 con 0x7f03d4079c70 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.384+0000 7f03cbfff700 1 -- 172.21.15.88:0/4153671608 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03d0011af0 con 0x7f03d4079c70 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.384+0000 7f03db4f4700 1 -- 172.21.15.88:0/4153671608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 msgr2=0x7f03d41085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.384+0000 7f03db4f4700 1 --2- 172.21.15.88:0/4153671608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d41085d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f03d00098a0 tx=0x7f03d0009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.384+0000 7f03db4f4700 1 -- 172.21.15.88:0/4153671608 shutdown_connections 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.384+0000 7f03db4f4700 1 --2- 172.21.15.88:0/4153671608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d41085d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.384+0000 7f03db4f4700 1 -- 172.21.15.88:0/4153671608 >> 172.21.15.88:0/4153671608 conn(0x7f03d40fff00 msgr2=0x7f03d4102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.384+0000 7f03db4f4700 1 -- 172.21.15.88:0/4153671608 shutdown_connections 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.384+0000 7f03db4f4700 1 -- 172.21.15.88:0/4153671608 wait complete. 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.385+0000 7f03db4f4700 1 Processor -- start 2024-02-19T18:51:20.774 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.385+0000 7f03db4f4700 1 -- start start 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.385+0000 7f03db4f4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d4105f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.385+0000 7f03db4f4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4106460 con 0x7f03d4079c70 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.385+0000 7f03d9290700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d4105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.385+0000 7f03d9290700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d4105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49360/0 (socket says 172.21.15.88:49360) 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.385+0000 7f03d9290700 1 -- 172.21.15.88:0/3393998559 learned_addr learned my addr 172.21.15.88:0/3393998559 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.385+0000 7f03d9290700 1 -- 172.21.15.88:0/3393998559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d0009550 con 0x7f03d4079c70 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.386+0000 7f03d9290700 1 --2- 172.21.15.88:0/3393998559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d4105f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f03d0011670 tx=0x7f03d0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.386+0000 7f03ca7fc700 1 -- 172.21.15.88:0/3393998559 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03d00039c0 con 0x7f03d4079c70 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.386+0000 7f03ca7fc700 1 -- 172.21.15.88:0/3393998559 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f03d0003b20 con 0x7f03d4079c70 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.386+0000 7f03db4f4700 1 -- 172.21.15.88:0/3393998559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d4104570 con 0x7f03d4079c70 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.386+0000 7f03ca7fc700 1 -- 172.21.15.88:0/3393998559 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f03d00247f0 con 0x7f03d4079c70 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.386+0000 7f03db4f4700 1 -- 172.21.15.88:0/3393998559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d4104a10 con 0x7f03d4079c70 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.387+0000 7f03ca7fc700 1 -- 172.21.15.88:0/3393998559 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f03d0024ab0 con 0x7f03d4079c70 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.387+0000 7f03ca7fc700 1 --2- 172.21.15.88:0/3393998559 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f03c00383b0 0x7f03c003a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:20.775 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.387+0000 7f03ca7fc700 1 -- 172.21.15.88:0/3393998559 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f03d004c300 con 0x7f03d4079c70 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.387+0000 7f03db4f4700 1 -- 172.21.15.88:0/3393998559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d4109aa0 con 0x7f03d4079c70 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.387+0000 7f03d8a8f700 1 --2- 172.21.15.88:0/3393998559 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f03c00383b0 0x7f03c003a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.387+0000 7f03d8a8f700 1 --2- 172.21.15.88:0/3393998559 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f03c00383b0 0x7f03c003a860 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f03c4006fd0 tx=0x7f03c4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.392+0000 7f03ca7fc700 1 -- 172.21.15.88:0/3393998559 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f03d0024d60 con 0x7f03d4079c70 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.620+0000 7f03db4f4700 1 -- 172.21.15.88:0/3393998559 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f03d4102200 con 0x7f03c00383b0 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.625+0000 7f03ca7fc700 1 -- 172.21.15.88:0/3393998559 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f03d4102200 con 0x7f03c00383b0 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.627+0000 7f03db4f4700 1 -- 172.21.15.88:0/3393998559 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f03c00383b0 msgr2=0x7f03c003a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.627+0000 7f03db4f4700 1 --2- 172.21.15.88:0/3393998559 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f03c00383b0 0x7f03c003a860 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f03c4006fd0 tx=0x7f03c4006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.627+0000 7f03db4f4700 1 -- 172.21.15.88:0/3393998559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 msgr2=0x7f03d4105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.627+0000 7f03db4f4700 1 --2- 172.21.15.88:0/3393998559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d4105f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f03d0011670 tx=0x7f03d0011aa0 comp rx=0 tx=0).stop 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.627+0000 7f03db4f4700 1 -- 172.21.15.88:0/3393998559 shutdown_connections 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.627+0000 7f03db4f4700 1 --2- 172.21.15.88:0/3393998559 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f03c00383b0 0x7f03c003a860 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:20.776 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.627+0000 7f03db4f4700 1 --2- 172.21.15.88:0/3393998559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4079c70 0x7f03d4105f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:20.777 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.627+0000 7f03db4f4700 1 -- 172.21.15.88:0/3393998559 >> 172.21.15.88:0/3393998559 conn(0x7f03d40fff00 msgr2=0x7f03d41015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:20.777 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.627+0000 7f03db4f4700 1 -- 172.21.15.88:0/3393998559 shutdown_connections 2024-02-19T18:51:20.777 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:19.627+0000 7f03db4f4700 1 -- 172.21.15.88:0/3393998559 wait complete. 2024-02-19T18:51:20.777 INFO:teuthology.orchestra.run.smithi088.stdout:Deploying ceph-exporter service with default placement... 2024-02-19T18:51:20.811 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:20 smithi088 ceph-mon[29272]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:20.811 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:20 smithi088 ceph-mon[29272]: Saving service crash spec with placement * 2024-02-19T18:51:20.811 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:20 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:20.811 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:20 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:22.089 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-02-19T18:51:22.089 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.197+0000 7f2d3aac3700 1 Processor -- start 2024-02-19T18:51:22.089 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.197+0000 7f2d3aac3700 1 -- start start 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.197+0000 7f2d3aac3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.197+0000 7f2d3aac3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3407a830 con 0x7f2d3407bdf0 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.197+0000 7f2d3885f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.197+0000 7f2d3885f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49368/0 (socket says 172.21.15.88:49368) 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.197+0000 7f2d3885f700 1 -- 172.21.15.88:0/2548507331 learned_addr learned my addr 172.21.15.88:0/2548507331 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.198+0000 7f2d3885f700 1 -- 172.21.15.88:0/2548507331 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d3407a970 con 0x7f2d3407bdf0 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.198+0000 7f2d3885f700 1 --2- 172.21.15.88:0/2548507331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3407a2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2d280098a0 tx=0x7f2d28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bd0b6d6f810e60 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.198+0000 7f2d337fe700 1 -- 172.21.15.88:0/2548507331 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d28004180 con 0x7f2d3407bdf0 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.198+0000 7f2d337fe700 1 -- 172.21.15.88:0/2548507331 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2d280042e0 con 0x7f2d3407bdf0 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.198+0000 7f2d337fe700 1 -- 172.21.15.88:0/2548507331 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d28011af0 con 0x7f2d3407bdf0 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.198+0000 7f2d3aac3700 1 -- 172.21.15.88:0/2548507331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 msgr2=0x7f2d3407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:22.090 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.198+0000 7f2d3aac3700 1 --2- 172.21.15.88:0/2548507331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3407a2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2d280098a0 tx=0x7f2d28009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.199+0000 7f2d3aac3700 1 -- 172.21.15.88:0/2548507331 shutdown_connections 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.199+0000 7f2d3aac3700 1 --2- 172.21.15.88:0/2548507331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3407a2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.199+0000 7f2d3aac3700 1 -- 172.21.15.88:0/2548507331 >> 172.21.15.88:0/2548507331 conn(0x7f2d34101020 msgr2=0x7f2d34103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.199+0000 7f2d3aac3700 1 -- 172.21.15.88:0/2548507331 shutdown_connections 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.199+0000 7f2d3aac3700 1 -- 172.21.15.88:0/2548507331 wait complete. 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.199+0000 7f2d3aac3700 1 Processor -- start 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.199+0000 7f2d3aac3700 1 -- start start 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.200+0000 7f2d3aac3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3419b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.200+0000 7f2d3aac3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3419be60 con 0x7f2d3407bdf0 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.200+0000 7f2d3885f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3419b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.200+0000 7f2d3885f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3419b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49374/0 (socket says 172.21.15.88:49374) 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.200+0000 7f2d3885f700 1 -- 172.21.15.88:0/4246406359 learned_addr learned my addr 172.21.15.88:0/4246406359 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:22.091 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.200+0000 7f2d3885f700 1 -- 172.21.15.88:0/4246406359 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d28009550 con 0x7f2d3407bdf0 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.200+0000 7f2d3885f700 1 --2- 172.21.15.88:0/4246406359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3419b920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2d28000c00 tx=0x7f2d28011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.200+0000 7f2d31ffb700 1 -- 172.21.15.88:0/4246406359 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d28003a70 con 0x7f2d3407bdf0 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.200+0000 7f2d3aac3700 1 -- 172.21.15.88:0/4246406359 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d3419c060 con 0x7f2d3407bdf0 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.200+0000 7f2d31ffb700 1 -- 172.21.15.88:0/4246406359 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2d28003bd0 con 0x7f2d3407bdf0 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.200+0000 7f2d31ffb700 1 -- 172.21.15.88:0/4246406359 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d280238c0 con 0x7f2d3407bdf0 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.201+0000 7f2d3aac3700 1 -- 172.21.15.88:0/4246406359 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d3419c500 con 0x7f2d3407bdf0 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.201+0000 7f2d31ffb700 1 -- 172.21.15.88:0/4246406359 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f2d28023b80 con 0x7f2d3407bdf0 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.201+0000 7f2d3aac3700 1 -- 172.21.15.88:0/4246406359 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d3410bcc0 con 0x7f2d3407bdf0 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.202+0000 7f2d31ffb700 1 --2- 172.21.15.88:0/4246406359 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f2d240383f0 0x7f2d2403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.202+0000 7f2d31ffb700 1 -- 172.21.15.88:0/4246406359 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2d2804c220 con 0x7f2d3407bdf0 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.202+0000 7f2d33fff700 1 --2- 172.21.15.88:0/4246406359 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f2d240383f0 0x7f2d2403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:22.092 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.202+0000 7f2d33fff700 1 --2- 172.21.15.88:0/4246406359 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f2d240383f0 0x7f2d2403a8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2d20006fd0 tx=0x7f2d20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.206+0000 7f2d31ffb700 1 -- 172.21.15.88:0/4246406359 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2d2802b300 con 0x7f2d3407bdf0 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.427+0000 7f2d3aac3700 1 -- 172.21.15.88:0/4246406359 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d341032d0 con 0x7f2d240383f0 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.431+0000 7f2d31ffb700 1 -- 172.21.15.88:0/4246406359 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f2d341032d0 con 0x7f2d240383f0 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.433+0000 7f2d3aac3700 1 -- 172.21.15.88:0/4246406359 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f2d240383f0 msgr2=0x7f2d2403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.433+0000 7f2d3aac3700 1 --2- 172.21.15.88:0/4246406359 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f2d240383f0 0x7f2d2403a8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2d20006fd0 tx=0x7f2d20006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.433+0000 7f2d3aac3700 1 -- 172.21.15.88:0/4246406359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 msgr2=0x7f2d3419b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.433+0000 7f2d3aac3700 1 --2- 172.21.15.88:0/4246406359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3419b920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2d28000c00 tx=0x7f2d28011b50 comp rx=0 tx=0).stop 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.433+0000 7f2d3aac3700 1 -- 172.21.15.88:0/4246406359 shutdown_connections 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.433+0000 7f2d3aac3700 1 --2- 172.21.15.88:0/4246406359 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f2d240383f0 0x7f2d2403a8a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.433+0000 7f2d3aac3700 1 --2- 172.21.15.88:0/4246406359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d3407bdf0 0x7f2d3419b920 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.433+0000 7f2d3aac3700 1 -- 172.21.15.88:0/4246406359 >> 172.21.15.88:0/4246406359 conn(0x7f2d34101020 msgr2=0x7f2d34102670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.434+0000 7f2d3aac3700 1 -- 172.21.15.88:0/4246406359 shutdown_connections 2024-02-19T18:51:22.093 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:21.434+0000 7f2d3aac3700 1 -- 172.21.15.88:0/4246406359 wait complete. 2024-02-19T18:51:22.094 INFO:teuthology.orchestra.run.smithi088.stdout:Deploying prometheus service with default placement... 2024-02-19T18:51:22.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:22 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:22.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:22 smithi088 ceph-mon[29272]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:22.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:22 smithi088 ceph-mon[29272]: Saving service ceph-exporter spec with placement * 2024-02-19T18:51:22.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:22 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:23.397 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-02-19T18:51:23.398 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.482+0000 7fde9fb0f700 1 Processor -- start 2024-02-19T18:51:23.398 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.482+0000 7fde9fb0f700 1 -- start start 2024-02-19T18:51:23.398 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.482+0000 7fde9fb0f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde981088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:23.398 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.482+0000 7fde9fb0f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde98079d10 con 0x7fde98106510 2024-02-19T18:51:23.398 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.482+0000 7fde9d8ab700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde981088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:23.398 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.482+0000 7fde9d8ab700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde981088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49380/0 (socket says 172.21.15.88:49380) 2024-02-19T18:51:23.398 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.482+0000 7fde9d8ab700 1 -- 172.21.15.88:0/1099650144 learned_addr learned my addr 172.21.15.88:0/1099650144 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:23.398 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.483+0000 7fde9d8ab700 1 -- 172.21.15.88:0/1099650144 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde98079e50 con 0x7fde98106510 2024-02-19T18:51:23.398 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.483+0000 7fde9d8ab700 1 --2- 172.21.15.88:0/1099650144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde981088f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fde940098a0 tx=0x7fde94009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c443d665725543a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:23.398 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.483+0000 7fde9c8a9700 1 -- 172.21.15.88:0/1099650144 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde94004180 con 0x7fde98106510 2024-02-19T18:51:23.398 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.483+0000 7fde9c8a9700 1 -- 172.21.15.88:0/1099650144 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fde940042e0 con 0x7fde98106510 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.483+0000 7fde9c8a9700 1 -- 172.21.15.88:0/1099650144 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde94011af0 con 0x7fde98106510 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.483+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1099650144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 msgr2=0x7fde981088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.483+0000 7fde9fb0f700 1 --2- 172.21.15.88:0/1099650144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde981088f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fde940098a0 tx=0x7fde94009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.483+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1099650144 shutdown_connections 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.483+0000 7fde9fb0f700 1 --2- 172.21.15.88:0/1099650144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde981088f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.483+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1099650144 >> 172.21.15.88:0/1099650144 conn(0x7fde980ffed0 msgr2=0x7fde98102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.484+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1099650144 shutdown_connections 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.484+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1099650144 wait complete. 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.484+0000 7fde9fb0f700 1 Processor -- start 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.484+0000 7fde9fb0f700 1 -- start start 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.484+0000 7fde9fb0f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde98199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.484+0000 7fde9fb0f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde98199c60 con 0x7fde98106510 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.484+0000 7fde9d8ab700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde98199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:23.399 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.484+0000 7fde9d8ab700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde98199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49384/0 (socket says 172.21.15.88:49384) 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.484+0000 7fde9d8ab700 1 -- 172.21.15.88:0/1199341243 learned_addr learned my addr 172.21.15.88:0/1199341243 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.485+0000 7fde9d8ab700 1 -- 172.21.15.88:0/1199341243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde94009550 con 0x7fde98106510 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.485+0000 7fde9d8ab700 1 --2- 172.21.15.88:0/1199341243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde98199720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fde94011670 tx=0x7fde94011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.485+0000 7fde8effd700 1 -- 172.21.15.88:0/1199341243 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde940039c0 con 0x7fde98106510 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.485+0000 7fde8effd700 1 -- 172.21.15.88:0/1199341243 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fde94003b20 con 0x7fde98106510 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.485+0000 7fde8effd700 1 -- 172.21.15.88:0/1199341243 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fde940247f0 con 0x7fde98106510 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.485+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1199341243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde98199ec0 con 0x7fde98106510 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.485+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1199341243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9819ccc0 con 0x7fde98106510 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.486+0000 7fde8effd700 1 -- 172.21.15.88:0/1199341243 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fde9401ac70 con 0x7fde98106510 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.486+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1199341243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde98062380 con 0x7fde98106510 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.486+0000 7fde8effd700 1 --2- 172.21.15.88:0/1199341243 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fde84038400 0x7fde8403a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:23.400 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.486+0000 7fde8effd700 1 -- 172.21.15.88:0/1199341243 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fde9404c260 con 0x7fde98106510 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.486+0000 7fde9d0aa700 1 --2- 172.21.15.88:0/1199341243 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fde84038400 0x7fde8403a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.487+0000 7fde9d0aa700 1 --2- 172.21.15.88:0/1199341243 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fde84038400 0x7fde8403a8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fde88006fd0 tx=0x7fde88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.491+0000 7fde8effd700 1 -- 172.21.15.88:0/1199341243 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fde94024a60 con 0x7fde98106510 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.707+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1199341243 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fde98102240 con 0x7fde84038400 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.712+0000 7fde8effd700 1 -- 172.21.15.88:0/1199341243 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fde98102240 con 0x7fde84038400 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.714+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1199341243 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fde84038400 msgr2=0x7fde8403a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.714+0000 7fde9fb0f700 1 --2- 172.21.15.88:0/1199341243 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fde84038400 0x7fde8403a8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fde88006fd0 tx=0x7fde88006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.714+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1199341243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 msgr2=0x7fde98199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.714+0000 7fde9fb0f700 1 --2- 172.21.15.88:0/1199341243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde98199720 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fde94011670 tx=0x7fde94011aa0 comp rx=0 tx=0).stop 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.714+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1199341243 shutdown_connections 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.714+0000 7fde9fb0f700 1 --2- 172.21.15.88:0/1199341243 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fde84038400 0x7fde8403a8b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.714+0000 7fde9fb0f700 1 --2- 172.21.15.88:0/1199341243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde98106510 0x7fde98199720 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.714+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1199341243 >> 172.21.15.88:0/1199341243 conn(0x7fde980ffed0 msgr2=0x7fde98101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:23.401 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.714+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1199341243 shutdown_connections 2024-02-19T18:51:23.402 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:22.714+0000 7fde9fb0f700 1 -- 172.21.15.88:0/1199341243 wait complete. 2024-02-19T18:51:23.402 INFO:teuthology.orchestra.run.smithi088.stdout:Deploying grafana service with default placement... 2024-02-19T18:51:24.002 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:23 smithi088 ceph-mon[29272]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:24.003 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:23 smithi088 ceph-mon[29272]: Saving service prometheus spec with placement count:1 2024-02-19T18:51:24.003 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:23 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:24.993 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-02-19T18:51:24.993 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.784+0000 7f38870de700 1 Processor -- start 2024-02-19T18:51:24.993 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.784+0000 7f38870de700 1 -- start start 2024-02-19T18:51:24.993 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.784+0000 7f38870de700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880104ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:24.993 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.784+0000 7f38870de700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3880105220 con 0x7f38801048d0 2024-02-19T18:51:24.993 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.784+0000 7f3884e7a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880104ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.784+0000 7f3884e7a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880104ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49386/0 (socket says 172.21.15.88:49386) 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.784+0000 7f3884e7a700 1 -- 172.21.15.88:0/51736825 learned_addr learned my addr 172.21.15.88:0/51736825 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.784+0000 7f3884e7a700 1 -- 172.21.15.88:0/51736825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3880105360 con 0x7f38801048d0 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.784+0000 7f3884e7a700 1 --2- 172.21.15.88:0/51736825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880104ce0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38740098a0 tx=0x7f3874009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7b3a1cea5d5ae7 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.785+0000 7f387f7fe700 1 -- 172.21.15.88:0/51736825 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3874004180 con 0x7f38801048d0 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.785+0000 7f387f7fe700 1 -- 172.21.15.88:0/51736825 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f38740042e0 con 0x7f38801048d0 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.785+0000 7f387f7fe700 1 -- 172.21.15.88:0/51736825 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3874011af0 con 0x7f38801048d0 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.785+0000 7f38870de700 1 -- 172.21.15.88:0/51736825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 msgr2=0x7f3880104ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.785+0000 7f38870de700 1 --2- 172.21.15.88:0/51736825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880104ce0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f38740098a0 tx=0x7f3874009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.785+0000 7f38870de700 1 -- 172.21.15.88:0/51736825 shutdown_connections 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.785+0000 7f38870de700 1 --2- 172.21.15.88:0/51736825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880104ce0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.785+0000 7f38870de700 1 -- 172.21.15.88:0/51736825 >> 172.21.15.88:0/51736825 conn(0x7f38800fff00 msgr2=0x7f3880102350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.785+0000 7f38870de700 1 -- 172.21.15.88:0/51736825 shutdown_connections 2024-02-19T18:51:24.994 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.785+0000 7f38870de700 1 -- 172.21.15.88:0/51736825 wait complete. 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.786+0000 7f38870de700 1 Processor -- start 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.786+0000 7f38870de700 1 -- start start 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.786+0000 7f38870de700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.786+0000 7f38870de700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3880199c70 con 0x7f38801048d0 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.786+0000 7f3884e7a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.786+0000 7f3884e7a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49402/0 (socket says 172.21.15.88:49402) 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.786+0000 7f3884e7a700 1 -- 172.21.15.88:0/3843654541 learned_addr learned my addr 172.21.15.88:0/3843654541 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.786+0000 7f3884e7a700 1 -- 172.21.15.88:0/3843654541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3874009550 con 0x7f38801048d0 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.787+0000 7f3884e7a700 1 --2- 172.21.15.88:0/3843654541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880199730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3874011670 tx=0x7f3874011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.787+0000 7f387dffb700 1 -- 172.21.15.88:0/3843654541 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3874003a40 con 0x7f38801048d0 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.787+0000 7f387dffb700 1 -- 172.21.15.88:0/3843654541 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3874003ba0 con 0x7f38801048d0 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.787+0000 7f38870de700 1 -- 172.21.15.88:0/3843654541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3880199e70 con 0x7f38801048d0 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.787+0000 7f387dffb700 1 -- 172.21.15.88:0/3843654541 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3874024790 con 0x7f38801048d0 2024-02-19T18:51:24.995 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.787+0000 7f38870de700 1 -- 172.21.15.88:0/3843654541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388019a310 con 0x7f38801048d0 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.788+0000 7f387dffb700 1 -- 172.21.15.88:0/3843654541 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f387402d3f0 con 0x7f38801048d0 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.788+0000 7f387dffb700 1 --2- 172.21.15.88:0/3843654541 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f38700383f0 0x7f387003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.788+0000 7f38870de700 1 -- 172.21.15.88:0/3843654541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3880062380 con 0x7f38801048d0 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.788+0000 7f387dffb700 1 -- 172.21.15.88:0/3843654541 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f387404c760 con 0x7f38801048d0 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.788+0000 7f387ffff700 1 --2- 172.21.15.88:0/3843654541 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f38700383f0 0x7f387003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.788+0000 7f387ffff700 1 --2- 172.21.15.88:0/3843654541 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f38700383f0 0x7f387003a8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f386c006fd0 tx=0x7f386c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.793+0000 7f387dffb700 1 -- 172.21.15.88:0/3843654541 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3874019460 con 0x7f38801048d0 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:23.999+0000 7f38870de700 1 -- 172.21.15.88:0/3843654541 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f38801020d0 con 0x7f38700383f0 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:24.004+0000 7f387dffb700 1 -- 172.21.15.88:0/3843654541 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f38801020d0 con 0x7f38700383f0 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:24.006+0000 7f38870de700 1 -- 172.21.15.88:0/3843654541 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f38700383f0 msgr2=0x7f387003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:24.006+0000 7f38870de700 1 --2- 172.21.15.88:0/3843654541 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f38700383f0 0x7f387003a8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f386c006fd0 tx=0x7f386c006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:24.006+0000 7f38870de700 1 -- 172.21.15.88:0/3843654541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 msgr2=0x7f3880199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:24.996 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:24.006+0000 7f38870de700 1 --2- 172.21.15.88:0/3843654541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880199730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3874011670 tx=0x7f3874011b50 comp rx=0 tx=0).stop 2024-02-19T18:51:24.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:24.006+0000 7f38870de700 1 -- 172.21.15.88:0/3843654541 shutdown_connections 2024-02-19T18:51:24.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:24.006+0000 7f38870de700 1 --2- 172.21.15.88:0/3843654541 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f38700383f0 0x7f387003a8a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:24.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:24.006+0000 7f38870de700 1 --2- 172.21.15.88:0/3843654541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38801048d0 0x7f3880199730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:24.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:24.006+0000 7f38870de700 1 -- 172.21.15.88:0/3843654541 >> 172.21.15.88:0/3843654541 conn(0x7f38800fff00 msgr2=0x7f3880079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:24.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:24.006+0000 7f38870de700 1 -- 172.21.15.88:0/3843654541 shutdown_connections 2024-02-19T18:51:24.997 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:24.006+0000 7f38870de700 1 -- 172.21.15.88:0/3843654541 wait complete. 2024-02-19T18:51:24.997 INFO:teuthology.orchestra.run.smithi088.stdout:Deploying node-exporter service with default placement... 2024-02-19T18:51:25.095 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:25 smithi088 ceph-mon[29272]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:25.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:25 smithi088 ceph-mon[29272]: Saving service grafana spec with placement count:1 2024-02-19T18:51:25.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:25 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:26.207 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-02-19T18:51:26.207 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.393+0000 7f8bd006b700 1 Processor -- start 2024-02-19T18:51:26.207 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.393+0000 7f8bd006b700 1 -- start start 2024-02-19T18:51:26.207 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.394+0000 7f8bd006b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8074490 0x7f8bc8108580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.394+0000 7f8bd006b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc8108ac0 con 0x7f8bc8074490 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.394+0000 7f8bcde07700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8074490 0x7f8bc8108580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.394+0000 7f8bcde07700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8074490 0x7f8bc8108580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49410/0 (socket says 172.21.15.88:49410) 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.394+0000 7f8bcde07700 1 -- 172.21.15.88:0/1602290873 learned_addr learned my addr 172.21.15.88:0/1602290873 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.394+0000 7f8bcde07700 1 -- 172.21.15.88:0/1602290873 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc8108c00 con 0x7f8bc8074490 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.394+0000 7f8bcde07700 1 --2- 172.21.15.88:0/1602290873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8074490 0x7f8bc8108580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80098a0 tx=0x7f8bb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8d79e6d047c7ec server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.394+0000 7f8bcce05700 1 -- 172.21.15.88:0/1602290873 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bb8004180 con 0x7f8bc8074490 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.394+0000 7f8bcce05700 1 -- 172.21.15.88:0/1602290873 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8bb80042e0 con 0x7f8bc8074490 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.394+0000 7f8bcce05700 1 -- 172.21.15.88:0/1602290873 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bb8011af0 con 0x7f8bc8074490 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.395+0000 7f8bd006b700 1 -- 172.21.15.88:0/1602290873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8074490 msgr2=0x7f8bc8108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.395+0000 7f8bd006b700 1 --2- 172.21.15.88:0/1602290873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8074490 0x7f8bc8108580 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80098a0 tx=0x7f8bb8009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:26.208 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.395+0000 7f8bd006b700 1 -- 172.21.15.88:0/1602290873 shutdown_connections 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.395+0000 7f8bd006b700 1 --2- 172.21.15.88:0/1602290873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8074490 0x7f8bc8108580 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.395+0000 7f8bd006b700 1 -- 172.21.15.88:0/1602290873 >> 172.21.15.88:0/1602290873 conn(0x7f8bc80fff10 msgr2=0x7f8bc8102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.395+0000 7f8bd006b700 1 -- 172.21.15.88:0/1602290873 shutdown_connections 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.395+0000 7f8bd006b700 1 -- 172.21.15.88:0/1602290873 wait complete. 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.396+0000 7f8bd006b700 1 Processor -- start 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.396+0000 7f8bd006b700 1 -- start start 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.396+0000 7f8bd006b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8199730 0x7f8bc8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.396+0000 7f8bd006b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc819a080 con 0x7f8bc8199730 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.396+0000 7f8bcde07700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8199730 0x7f8bc8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.396+0000 7f8bcde07700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8199730 0x7f8bc8199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49424/0 (socket says 172.21.15.88:49424) 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.396+0000 7f8bcde07700 1 -- 172.21.15.88:0/2311787841 learned_addr learned my addr 172.21.15.88:0/2311787841 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.396+0000 7f8bcde07700 1 -- 172.21.15.88:0/2311787841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb8009550 con 0x7f8bc8199730 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.396+0000 7f8bcde07700 1 --2- 172.21.15.88:0/2311787841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8199730 0x7f8bc8199b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8011670 tx=0x7f8bb8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:26.209 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.397+0000 7f8bbeffd700 1 -- 172.21.15.88:0/2311787841 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bb80039f0 con 0x7f8bc8199730 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.397+0000 7f8bbeffd700 1 -- 172.21.15.88:0/2311787841 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8bb8003b50 con 0x7f8bc8199730 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.397+0000 7f8bd006b700 1 -- 172.21.15.88:0/2311787841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc819a280 con 0x7f8bc8199730 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.397+0000 7f8bbeffd700 1 -- 172.21.15.88:0/2311787841 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bb80247c0 con 0x7f8bc8199730 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.397+0000 7f8bd006b700 1 -- 172.21.15.88:0/2311787841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc819cee0 con 0x7f8bc8199730 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.397+0000 7f8bbeffd700 1 -- 172.21.15.88:0/2311787841 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f8bb802d3f0 con 0x7f8bc8199730 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.398+0000 7f8bd006b700 1 -- 172.21.15.88:0/2311787841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc8074490 con 0x7f8bc8199730 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.398+0000 7f8bbeffd700 1 --2- 172.21.15.88:0/2311787841 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f8bb4038440 0x7f8bb403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.398+0000 7f8bbeffd700 1 -- 172.21.15.88:0/2311787841 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8bb804c570 con 0x7f8bc8199730 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.398+0000 7f8bcd606700 1 --2- 172.21.15.88:0/2311787841 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f8bb4038440 0x7f8bb403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.398+0000 7f8bcd606700 1 --2- 172.21.15.88:0/2311787841 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f8bb4038440 0x7f8bb403a8f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4006fd0 tx=0x7f8bc4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.402+0000 7f8bbeffd700 1 -- 172.21.15.88:0/2311787841 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8bb8010970 con 0x7f8bc8199730 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.603+0000 7f8bd006b700 1 -- 172.21.15.88:0/2311787841 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bc81021f0 con 0x7f8bb4038440 2024-02-19T18:51:26.210 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.608+0000 7f8bbeffd700 1 -- 172.21.15.88:0/2311787841 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f8bc81021f0 con 0x7f8bb4038440 2024-02-19T18:51:26.211 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.610+0000 7f8bd006b700 1 -- 172.21.15.88:0/2311787841 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f8bb4038440 msgr2=0x7f8bb403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:26.211 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.610+0000 7f8bd006b700 1 --2- 172.21.15.88:0/2311787841 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f8bb4038440 0x7f8bb403a8f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4006fd0 tx=0x7f8bc4006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:26.211 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.610+0000 7f8bd006b700 1 -- 172.21.15.88:0/2311787841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8199730 msgr2=0x7f8bc8199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:26.211 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.610+0000 7f8bd006b700 1 --2- 172.21.15.88:0/2311787841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8199730 0x7f8bc8199b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8011670 tx=0x7f8bb8011b50 comp rx=0 tx=0).stop 2024-02-19T18:51:26.211 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.610+0000 7f8bd006b700 1 -- 172.21.15.88:0/2311787841 shutdown_connections 2024-02-19T18:51:26.211 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.610+0000 7f8bd006b700 1 --2- 172.21.15.88:0/2311787841 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f8bb4038440 0x7f8bb403a8f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:26.211 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.610+0000 7f8bd006b700 1 --2- 172.21.15.88:0/2311787841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bc8199730 0x7f8bc8199b40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:26.211 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.610+0000 7f8bd006b700 1 -- 172.21.15.88:0/2311787841 >> 172.21.15.88:0/2311787841 conn(0x7f8bc80fff10 msgr2=0x7f8bc81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:26.211 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.610+0000 7f8bd006b700 1 -- 172.21.15.88:0/2311787841 shutdown_connections 2024-02-19T18:51:26.211 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:25.610+0000 7f8bd006b700 1 -- 172.21.15.88:0/2311787841 wait complete. 2024-02-19T18:51:26.211 INFO:teuthology.orchestra.run.smithi088.stdout:Deploying alertmanager service with default placement... 2024-02-19T18:51:26.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:26 smithi088 ceph-mon[29272]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:51:26.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:26 smithi088 ceph-mon[29272]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:26.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:26 smithi088 ceph-mon[29272]: Saving service node-exporter spec with placement * 2024-02-19T18:51:26.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:26 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:27.431 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-02-19T18:51:27.431 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.593+0000 7fee98c30700 1 Processor -- start 2024-02-19T18:51:27.431 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.593+0000 7fee98c30700 1 -- start start 2024-02-19T18:51:27.431 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.593+0000 7fee98c30700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:27.431 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.593+0000 7fee98c30700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9407a810 con 0x7fee9407bdd0 2024-02-19T18:51:27.431 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.593+0000 7fee9259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:27.431 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.593+0000 7fee9259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50746/0 (socket says 172.21.15.88:50746) 2024-02-19T18:51:27.431 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.593+0000 7fee9259c700 1 -- 172.21.15.88:0/2704678790 learned_addr learned my addr 172.21.15.88:0/2704678790 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:27.431 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.594+0000 7fee9259c700 1 -- 172.21.15.88:0/2704678790 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9407a950 con 0x7fee9407bdd0 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.594+0000 7fee9259c700 1 --2- 172.21.15.88:0/2704678790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9407a2d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fee84009870 tx=0x7fee84009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4950d8a1903a94fc server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.594+0000 7fee9159a700 1 -- 172.21.15.88:0/2704678790 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee84005c30 con 0x7fee9407bdd0 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.594+0000 7fee9159a700 1 -- 172.21.15.88:0/2704678790 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fee840044e0 con 0x7fee9407bdd0 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.594+0000 7fee9159a700 1 -- 172.21.15.88:0/2704678790 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee84006030 con 0x7fee9407bdd0 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.595+0000 7fee98c30700 1 -- 172.21.15.88:0/2704678790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 msgr2=0x7fee9407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.595+0000 7fee98c30700 1 --2- 172.21.15.88:0/2704678790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9407a2d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fee84009870 tx=0x7fee84009b80 comp rx=0 tx=0).stop 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.595+0000 7fee98c30700 1 -- 172.21.15.88:0/2704678790 shutdown_connections 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.595+0000 7fee98c30700 1 --2- 172.21.15.88:0/2704678790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9407a2d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.595+0000 7fee98c30700 1 -- 172.21.15.88:0/2704678790 >> 172.21.15.88:0/2704678790 conn(0x7fee94101020 msgr2=0x7fee94103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.595+0000 7fee98c30700 1 -- 172.21.15.88:0/2704678790 shutdown_connections 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.595+0000 7fee98c30700 1 -- 172.21.15.88:0/2704678790 wait complete. 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.595+0000 7fee98c30700 1 Processor -- start 2024-02-19T18:51:27.432 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.595+0000 7fee98c30700 1 -- start start 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee98c30700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee98c30700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9419bed0 con 0x7fee9407bdd0 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee9259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee9259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9419b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50752/0 (socket says 172.21.15.88:50752) 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee9259c700 1 -- 172.21.15.88:0/564327271 learned_addr learned my addr 172.21.15.88:0/564327271 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee9259c700 1 -- 172.21.15.88:0/564327271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee84009500 con 0x7fee9407bdd0 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee9259c700 1 --2- 172.21.15.88:0/564327271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9419b990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fee84013000 tx=0x7fee840042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee837fe700 1 -- 172.21.15.88:0/564327271 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee84005e40 con 0x7fee9407bdd0 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee837fe700 1 -- 172.21.15.88:0/564327271 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fee84005fa0 con 0x7fee9407bdd0 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee837fe700 1 -- 172.21.15.88:0/564327271 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fee84011bc0 con 0x7fee9407bdd0 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee98c30700 1 -- 172.21.15.88:0/564327271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9419c0d0 con 0x7fee9407bdd0 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.596+0000 7fee98c30700 1 -- 172.21.15.88:0/564327271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9419c570 con 0x7fee9407bdd0 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.597+0000 7fee837fe700 1 -- 172.21.15.88:0/564327271 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fee84018070 con 0x7fee9407bdd0 2024-02-19T18:51:27.433 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.597+0000 7fee837fe700 1 --2- 172.21.15.88:0/564327271 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fee7c038400 0x7fee7c03a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:27.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.597+0000 7fee837fe700 1 -- 172.21.15.88:0/564327271 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fee8404b8b0 con 0x7fee9407bdd0 2024-02-19T18:51:27.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.597+0000 7fee98c30700 1 -- 172.21.15.88:0/564327271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee94062380 con 0x7fee9407bdd0 2024-02-19T18:51:27.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.597+0000 7fee91d9b700 1 --2- 172.21.15.88:0/564327271 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fee7c038400 0x7fee7c03a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:27.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.598+0000 7fee91d9b700 1 --2- 172.21.15.88:0/564327271 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fee7c038400 0x7fee7c03a8b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fee88006fd0 tx=0x7fee88006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:27.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.602+0000 7fee837fe700 1 -- 172.21.15.88:0/564327271 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fee8401f080 con 0x7fee9407bdd0 2024-02-19T18:51:27.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.802+0000 7fee98c30700 1 -- 172.21.15.88:0/564327271 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fee941032e0 con 0x7fee7c038400 2024-02-19T18:51:27.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.807+0000 7fee837fe700 1 -- 172.21.15.88:0/564327271 <== mgr.14120 v2:172.21.15.88:6800/3414266099 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fee941032e0 con 0x7fee7c038400 2024-02-19T18:51:27.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.808+0000 7fee98c30700 1 -- 172.21.15.88:0/564327271 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fee7c038400 msgr2=0x7fee7c03a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:27.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.808+0000 7fee98c30700 1 --2- 172.21.15.88:0/564327271 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fee7c038400 0x7fee7c03a8b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fee88006fd0 tx=0x7fee88006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:27.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.808+0000 7fee98c30700 1 -- 172.21.15.88:0/564327271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 msgr2=0x7fee9419b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:27.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.809+0000 7fee98c30700 1 --2- 172.21.15.88:0/564327271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9419b990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fee84013000 tx=0x7fee840042e0 comp rx=0 tx=0).stop 2024-02-19T18:51:27.435 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.809+0000 7fee98c30700 1 -- 172.21.15.88:0/564327271 shutdown_connections 2024-02-19T18:51:27.435 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.809+0000 7fee98c30700 1 --2- 172.21.15.88:0/564327271 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fee7c038400 0x7fee7c03a8b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:27.435 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.809+0000 7fee98c30700 1 --2- 172.21.15.88:0/564327271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee9407bdd0 0x7fee9419b990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:27.435 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.809+0000 7fee98c30700 1 -- 172.21.15.88:0/564327271 >> 172.21.15.88:0/564327271 conn(0x7fee94101020 msgr2=0x7fee941026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:27.435 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.809+0000 7fee98c30700 1 -- 172.21.15.88:0/564327271 shutdown_connections 2024-02-19T18:51:27.435 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:26.809+0000 7fee98c30700 1 -- 172.21.15.88:0/564327271 wait complete. 2024-02-19T18:51:28.071 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:27 smithi088 ceph-mon[29272]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:28.072 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:27 smithi088 ceph-mon[29272]: Saving service alertmanager spec with placement count:1 2024-02-19T18:51:28.072 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:27 smithi088 ceph-mon[29272]: from='mgr.14120 172.21.15.88:0/2890811423' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:28.620 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f8018b47700 1 Processor -- start 2024-02-19T18:51:28.620 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f8018b47700 1 -- start start 2024-02-19T18:51:28.620 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f8018b47700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f80141040c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:28.620 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f8018b47700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8014104600 con 0x7f8014103cb0 2024-02-19T18:51:28.620 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f801259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f80141040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:28.620 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f801259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f80141040c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50768/0 (socket says 172.21.15.88:50768) 2024-02-19T18:51:28.620 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f801259c700 1 -- 172.21.15.88:0/4233354051 learned_addr learned my addr 172.21.15.88:0/4233354051 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:28.620 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f801259c700 1 -- 172.21.15.88:0/4233354051 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8014104740 con 0x7f8014103cb0 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f801259c700 1 --2- 172.21.15.88:0/4233354051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f80141040c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80040098a0 tx=0x7f8004009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f67e13cfc87e44 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f801159a700 1 -- 172.21.15.88:0/4233354051 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8004004180 con 0x7f8014103cb0 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f801159a700 1 -- 172.21.15.88:0/4233354051 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f80040042e0 con 0x7f8014103cb0 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.857+0000 7f801159a700 1 -- 172.21.15.88:0/4233354051 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8004011af0 con 0x7f8014103cb0 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.858+0000 7f8018b47700 1 -- 172.21.15.88:0/4233354051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 msgr2=0x7f80141040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.858+0000 7f8018b47700 1 --2- 172.21.15.88:0/4233354051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f80141040c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80040098a0 tx=0x7f8004009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.858+0000 7f8018b47700 1 -- 172.21.15.88:0/4233354051 shutdown_connections 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.858+0000 7f8018b47700 1 --2- 172.21.15.88:0/4233354051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f80141040c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.858+0000 7f8018b47700 1 -- 172.21.15.88:0/4233354051 >> 172.21.15.88:0/4233354051 conn(0x7f80140ff670 msgr2=0x7f8014101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.858+0000 7f8018b47700 1 -- 172.21.15.88:0/4233354051 shutdown_connections 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.858+0000 7f8018b47700 1 -- 172.21.15.88:0/4233354051 wait complete. 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.859+0000 7f8018b47700 1 Processor -- start 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.859+0000 7f8018b47700 1 -- start start 2024-02-19T18:51:28.621 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.859+0000 7f8018b47700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f8014106640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.859+0000 7f8018b47700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8014106b80 con 0x7f8014103cb0 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.859+0000 7f801259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f8014106640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.859+0000 7f801259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f8014106640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50776/0 (socket says 172.21.15.88:50776) 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.859+0000 7f801259c700 1 -- 172.21.15.88:0/869061207 learned_addr learned my addr 172.21.15.88:0/869061207 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.859+0000 7f801259c700 1 -- 172.21.15.88:0/869061207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8004009550 con 0x7f8014103cb0 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.860+0000 7f801259c700 1 --2- 172.21.15.88:0/869061207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f8014106640 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8004011670 tx=0x7f8004011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.860+0000 7f80037fe700 1 -- 172.21.15.88:0/869061207 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80040039f0 con 0x7f8014103cb0 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.860+0000 7f80037fe700 1 -- 172.21.15.88:0/869061207 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8004003b50 con 0x7f8014103cb0 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.860+0000 7f8018b47700 1 -- 172.21.15.88:0/869061207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8014104c90 con 0x7f8014103cb0 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.860+0000 7f80037fe700 1 -- 172.21.15.88:0/869061207 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f80040247c0 con 0x7f8014103cb0 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.860+0000 7f8018b47700 1 -- 172.21.15.88:0/869061207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8014105130 con 0x7f8014103cb0 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.861+0000 7f80037fe700 1 -- 172.21.15.88:0/869061207 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f800402d3f0 con 0x7f8014103cb0 2024-02-19T18:51:28.622 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.861+0000 7f8018b47700 1 -- 172.21.15.88:0/869061207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8014062380 con 0x7f8014103cb0 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.861+0000 7f80037fe700 1 --2- 172.21.15.88:0/869061207 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f7ffc038440 0x7f7ffc03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.861+0000 7f80037fe700 1 -- 172.21.15.88:0/869061207 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f800404c590 con 0x7f8014103cb0 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.861+0000 7f8011d9b700 1 --2- 172.21.15.88:0/869061207 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f7ffc038440 0x7f7ffc03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.861+0000 7f8011d9b700 1 --2- 172.21.15.88:0/869061207 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f7ffc038440 0x7f7ffc03a8f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8008006fd0 tx=0x7f8008006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:27.866+0000 7f80037fe700 1 -- 172.21.15.88:0/869061207 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8004019410 con 0x7f8014103cb0 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.055+0000 7f8018b47700 1 -- 172.21.15.88:0/869061207 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f8014107930 con 0x7f8014103cb0 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.058+0000 7f80037fe700 1 -- 172.21.15.88:0/869061207 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8004010380 con 0x7f8014103cb0 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.061+0000 7f8018b47700 1 -- 172.21.15.88:0/869061207 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f7ffc038440 msgr2=0x7f7ffc03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.061+0000 7f8018b47700 1 --2- 172.21.15.88:0/869061207 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f7ffc038440 0x7f7ffc03a8f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8008006fd0 tx=0x7f8008006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.061+0000 7f8018b47700 1 -- 172.21.15.88:0/869061207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 msgr2=0x7f8014106640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.061+0000 7f8018b47700 1 --2- 172.21.15.88:0/869061207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f8014106640 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8004011670 tx=0x7f8004011b50 comp rx=0 tx=0).stop 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.061+0000 7f8018b47700 1 -- 172.21.15.88:0/869061207 shutdown_connections 2024-02-19T18:51:28.623 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.061+0000 7f8018b47700 1 --2- 172.21.15.88:0/869061207 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f7ffc038440 0x7f7ffc03a8f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:28.624 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.061+0000 7f8018b47700 1 --2- 172.21.15.88:0/869061207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8014103cb0 0x7f8014106640 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:28.624 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.061+0000 7f8018b47700 1 -- 172.21.15.88:0/869061207 >> 172.21.15.88:0/869061207 conn(0x7f80140ff670 msgr2=0x7f8014100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:28.624 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.061+0000 7f8018b47700 1 -- 172.21.15.88:0/869061207 shutdown_connections 2024-02-19T18:51:28.624 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:28.061+0000 7f8018b47700 1 -- 172.21.15.88:0/869061207 wait complete. 2024-02-19T18:51:28.992 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:28 smithi088 ceph-mon[29272]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:51:28.992 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:28 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/869061207' entity='client.admin' 2024-02-19T18:51:29.817 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.003+0000 7f262ca72700 1 Processor -- start 2024-02-19T18:51:29.818 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.003+0000 7f262ca72700 1 -- start start 2024-02-19T18:51:29.818 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.004+0000 7f262ca72700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628106280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:29.818 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.004+0000 7f262ca72700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26281067c0 con 0x7f2628105e70 2024-02-19T18:51:29.818 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.004+0000 7f262659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628106280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:29.818 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.004+0000 7f262659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628106280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50784/0 (socket says 172.21.15.88:50784) 2024-02-19T18:51:29.818 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.004+0000 7f262659c700 1 -- 172.21.15.88:0/1672189260 learned_addr learned my addr 172.21.15.88:0/1672189260 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:29.818 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.004+0000 7f262659c700 1 -- 172.21.15.88:0/1672189260 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2628106900 con 0x7f2628105e70 2024-02-19T18:51:29.818 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.004+0000 7f262659c700 1 --2- 172.21.15.88:0/1672189260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628106280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2620009e80 tx=0x7f262000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4f782eef7c2b9f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:29.819 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.005+0000 7f262559a700 1 -- 172.21.15.88:0/1672189260 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f262000e070 con 0x7f2628105e70 2024-02-19T18:51:29.819 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.005+0000 7f262559a700 1 -- 172.21.15.88:0/1672189260 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f262000b8c0 con 0x7f2628105e70 2024-02-19T18:51:29.819 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.005+0000 7f262559a700 1 -- 172.21.15.88:0/1672189260 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26200044d0 con 0x7f2628105e70 2024-02-19T18:51:29.819 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.005+0000 7f262ca72700 1 -- 172.21.15.88:0/1672189260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 msgr2=0x7f2628106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:29.819 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.005+0000 7f262ca72700 1 --2- 172.21.15.88:0/1672189260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628106280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2620009e80 tx=0x7f262000b330 comp rx=0 tx=0).stop 2024-02-19T18:51:29.819 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.005+0000 7f262ca72700 1 -- 172.21.15.88:0/1672189260 shutdown_connections 2024-02-19T18:51:29.819 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.005+0000 7f262ca72700 1 --2- 172.21.15.88:0/1672189260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628106280 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:29.820 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.005+0000 7f262ca72700 1 -- 172.21.15.88:0/1672189260 >> 172.21.15.88:0/1672189260 conn(0x7f26280ff670 msgr2=0x7f2628101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:29.820 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.005+0000 7f262ca72700 1 -- 172.21.15.88:0/1672189260 shutdown_connections 2024-02-19T18:51:29.820 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.005+0000 7f262ca72700 1 -- 172.21.15.88:0/1672189260 wait complete. 2024-02-19T18:51:29.820 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.006+0000 7f262ca72700 1 Processor -- start 2024-02-19T18:51:29.820 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.006+0000 7f262ca72700 1 -- start start 2024-02-19T18:51:29.820 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.006+0000 7f262ca72700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:29.820 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.006+0000 7f262ca72700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2628197ab0 con 0x7f2628105e70 2024-02-19T18:51:29.820 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.006+0000 7f262659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:29.821 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.006+0000 7f262659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50788/0 (socket says 172.21.15.88:50788) 2024-02-19T18:51:29.821 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.006+0000 7f262659c700 1 -- 172.21.15.88:0/2184680148 learned_addr learned my addr 172.21.15.88:0/2184680148 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:29.821 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.007+0000 7f262659c700 1 -- 172.21.15.88:0/2184680148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2620009b30 con 0x7f2628105e70 2024-02-19T18:51:29.821 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.007+0000 7f262659c700 1 --2- 172.21.15.88:0/2184680148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628197570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2620009e50 tx=0x7f2620003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:29.821 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.007+0000 7f26177fe700 1 -- 172.21.15.88:0/2184680148 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f262000e040 con 0x7f2628105e70 2024-02-19T18:51:29.821 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.007+0000 7f262ca72700 1 -- 172.21.15.88:0/2184680148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2628197cb0 con 0x7f2628105e70 2024-02-19T18:51:29.821 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.007+0000 7f26177fe700 1 -- 172.21.15.88:0/2184680148 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f262000bc40 con 0x7f2628105e70 2024-02-19T18:51:29.821 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.007+0000 7f26177fe700 1 -- 172.21.15.88:0/2184680148 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f26200125e0 con 0x7f2628105e70 2024-02-19T18:51:29.822 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.007+0000 7f262ca72700 1 -- 172.21.15.88:0/2184680148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26281981b0 con 0x7f2628105e70 2024-02-19T18:51:29.822 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.008+0000 7f26177fe700 1 -- 172.21.15.88:0/2184680148 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f2620019040 con 0x7f2628105e70 2024-02-19T18:51:29.822 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.008+0000 7f26177fe700 1 --2- 172.21.15.88:0/2184680148 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f261003a5b0 0x7f261003ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:29.822 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.008+0000 7f26177fe700 1 -- 172.21.15.88:0/2184680148 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f262004bd90 con 0x7f2628105e70 2024-02-19T18:51:29.822 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.008+0000 7f262ca72700 1 -- 172.21.15.88:0/2184680148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2628191210 con 0x7f2628105e70 2024-02-19T18:51:29.822 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.008+0000 7f2625d9b700 1 --2- 172.21.15.88:0/2184680148 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f261003a5b0 0x7f261003ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:29.822 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.008+0000 7f2625d9b700 1 --2- 172.21.15.88:0/2184680148 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f261003a5b0 0x7f261003ca60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f261c006fd0 tx=0x7f261c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:29.822 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.013+0000 7f26177fe700 1 -- 172.21.15.88:0/2184680148 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f262001f080 con 0x7f2628105e70 2024-02-19T18:51:29.823 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.202+0000 7f262ca72700 1 -- 172.21.15.88:0/2184680148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f2628061960 con 0x7f2628105e70 2024-02-19T18:51:29.823 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.206+0000 7f26177fe700 1 -- 172.21.15.88:0/2184680148 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f262004a0c0 con 0x7f2628105e70 2024-02-19T18:51:29.823 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.208+0000 7f262ca72700 1 -- 172.21.15.88:0/2184680148 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f261003a5b0 msgr2=0x7f261003ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:29.823 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.208+0000 7f262ca72700 1 --2- 172.21.15.88:0/2184680148 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f261003a5b0 0x7f261003ca60 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f261c006fd0 tx=0x7f261c006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:29.823 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.208+0000 7f262ca72700 1 -- 172.21.15.88:0/2184680148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 msgr2=0x7f2628197570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:29.823 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.208+0000 7f262ca72700 1 --2- 172.21.15.88:0/2184680148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628197570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2620009e50 tx=0x7f2620003d00 comp rx=0 tx=0).stop 2024-02-19T18:51:29.824 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.208+0000 7f262ca72700 1 -- 172.21.15.88:0/2184680148 shutdown_connections 2024-02-19T18:51:29.824 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.208+0000 7f262ca72700 1 --2- 172.21.15.88:0/2184680148 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f261003a5b0 0x7f261003ca60 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:29.824 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.208+0000 7f262ca72700 1 --2- 172.21.15.88:0/2184680148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2628105e70 0x7f2628197570 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:29.824 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.208+0000 7f262ca72700 1 -- 172.21.15.88:0/2184680148 >> 172.21.15.88:0/2184680148 conn(0x7f26280ff670 msgr2=0x7f262818e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:29.824 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.208+0000 7f262ca72700 1 -- 172.21.15.88:0/2184680148 shutdown_connections 2024-02-19T18:51:29.824 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:29.208+0000 7f262ca72700 1 -- 172.21.15.88:0/2184680148 wait complete. 2024-02-19T18:51:29.824 INFO:teuthology.orchestra.run.smithi088.stdout:Enabling the dashboard module... 2024-02-19T18:51:30.485 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:30 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2184680148' entity='client.admin' 2024-02-19T18:51:30.485 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:30 smithi088 ceph-mon[29272]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:51:31.552 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:31 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2762337994' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-19T18:51:31.893 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.216+0000 7fa542dc2700 1 Processor -- start 2024-02-19T18:51:31.893 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.216+0000 7fa542dc2700 1 -- start start 2024-02-19T18:51:31.893 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.216+0000 7fa542dc2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c104cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.216+0000 7fa542dc2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c105230 con 0x7fa53c1048e0 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.216+0000 7fa540b5e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c104cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.216+0000 7fa540b5e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c104cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50804/0 (socket says 172.21.15.88:50804) 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.216+0000 7fa540b5e700 1 -- 172.21.15.88:0/2724621229 learned_addr learned my addr 172.21.15.88:0/2724621229 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.216+0000 7fa540b5e700 1 -- 172.21.15.88:0/2724621229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa53c105370 con 0x7fa53c1048e0 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.216+0000 7fa540b5e700 1 --2- 172.21.15.88:0/2724621229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c104cf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5300098a0 tx=0x7fa530009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee65cb0d007ccb3 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.217+0000 7fa53b7fe700 1 -- 172.21.15.88:0/2724621229 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa530004180 con 0x7fa53c1048e0 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.217+0000 7fa53b7fe700 1 -- 172.21.15.88:0/2724621229 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa5300042e0 con 0x7fa53c1048e0 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.217+0000 7fa53b7fe700 1 -- 172.21.15.88:0/2724621229 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa530011af0 con 0x7fa53c1048e0 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.217+0000 7fa542dc2700 1 -- 172.21.15.88:0/2724621229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 msgr2=0x7fa53c104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.217+0000 7fa542dc2700 1 --2- 172.21.15.88:0/2724621229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c104cf0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa5300098a0 tx=0x7fa530009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.217+0000 7fa542dc2700 1 -- 172.21.15.88:0/2724621229 shutdown_connections 2024-02-19T18:51:31.894 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.217+0000 7fa542dc2700 1 --2- 172.21.15.88:0/2724621229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c104cf0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.217+0000 7fa542dc2700 1 -- 172.21.15.88:0/2724621229 >> 172.21.15.88:0/2724621229 conn(0x7fa53c0fff10 msgr2=0x7fa53c102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.217+0000 7fa542dc2700 1 -- 172.21.15.88:0/2724621229 shutdown_connections 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.217+0000 7fa542dc2700 1 -- 172.21.15.88:0/2724621229 wait complete. 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.218+0000 7fa542dc2700 1 Processor -- start 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.218+0000 7fa542dc2700 1 -- start start 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.218+0000 7fa542dc2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c18fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.218+0000 7fa542dc2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53c190000 con 0x7fa53c1048e0 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.218+0000 7fa540b5e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c18fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.218+0000 7fa540b5e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c18fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50812/0 (socket says 172.21.15.88:50812) 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.218+0000 7fa540b5e700 1 -- 172.21.15.88:0/2762337994 learned_addr learned my addr 172.21.15.88:0/2762337994 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.218+0000 7fa540b5e700 1 -- 172.21.15.88:0/2762337994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa530009550 con 0x7fa53c1048e0 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.218+0000 7fa540b5e700 1 --2- 172.21.15.88:0/2762337994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c18fac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa530011670 tx=0x7fa530011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:31.895 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.219+0000 7fa539ffb700 1 -- 172.21.15.88:0/2762337994 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa530003a40 con 0x7fa53c1048e0 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.219+0000 7fa539ffb700 1 -- 172.21.15.88:0/2762337994 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa530003ba0 con 0x7fa53c1048e0 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.219+0000 7fa542dc2700 1 -- 172.21.15.88:0/2762337994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53c18e110 con 0x7fa53c1048e0 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.219+0000 7fa539ffb700 1 -- 172.21.15.88:0/2762337994 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa530024790 con 0x7fa53c1048e0 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.219+0000 7fa542dc2700 1 -- 172.21.15.88:0/2762337994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53c18e5b0 con 0x7fa53c1048e0 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.220+0000 7fa539ffb700 1 -- 172.21.15.88:0/2762337994 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fa53002d3f0 con 0x7fa53c1048e0 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.220+0000 7fa542dc2700 1 -- 172.21.15.88:0/2762337994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53c062380 con 0x7fa53c1048e0 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.220+0000 7fa539ffb700 1 --2- 172.21.15.88:0/2762337994 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fa52c038440 0x7fa52c03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.220+0000 7fa539ffb700 1 -- 172.21.15.88:0/2762337994 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa53004c670 con 0x7fa53c1048e0 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.220+0000 7fa53bfff700 1 --2- 172.21.15.88:0/2762337994 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fa52c038440 0x7fa52c03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.220+0000 7fa53bfff700 1 --2- 172.21.15.88:0/2762337994 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fa52c038440 0x7fa52c03a8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa528006fd0 tx=0x7fa528006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.225+0000 7fa539ffb700 1 -- 172.21.15.88:0/2762337994 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa530018b00 con 0x7fa53c1048e0 2024-02-19T18:51:31.896 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:30.469+0000 7fa542dc2700 1 -- 172.21.15.88:0/2762337994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fa53c1067e0 con 0x7fa53c1048e0 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.220+0000 7fa539ffb700 1 -- 172.21.15.88:0/2762337994 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7fa53002d8d0 con 0x7fa53c1048e0 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.221+0000 7fa539ffb700 1 -- 172.21.15.88:0/2762337994 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fa53001ac50 con 0x7fa53c1048e0 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.225+0000 7fa542dc2700 1 -- 172.21.15.88:0/2762337994 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fa52c038440 msgr2=0x7fa52c03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.225+0000 7fa542dc2700 1 --2- 172.21.15.88:0/2762337994 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fa52c038440 0x7fa52c03a8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa528006fd0 tx=0x7fa528006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.225+0000 7fa542dc2700 1 -- 172.21.15.88:0/2762337994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 msgr2=0x7fa53c18fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.225+0000 7fa542dc2700 1 --2- 172.21.15.88:0/2762337994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c18fac0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa530011670 tx=0x7fa530011b50 comp rx=0 tx=0).stop 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.225+0000 7fa542dc2700 1 -- 172.21.15.88:0/2762337994 shutdown_connections 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.225+0000 7fa542dc2700 1 --2- 172.21.15.88:0/2762337994 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7fa52c038440 0x7fa52c03a8f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.225+0000 7fa542dc2700 1 --2- 172.21.15.88:0/2762337994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa53c1048e0 0x7fa53c18fac0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.225+0000 7fa542dc2700 1 -- 172.21.15.88:0/2762337994 >> 172.21.15.88:0/2762337994 conn(0x7fa53c0fff10 msgr2=0x7fa53c074070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.225+0000 7fa542dc2700 1 -- 172.21.15.88:0/2762337994 shutdown_connections 2024-02-19T18:51:31.897 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:31.225+0000 7fa542dc2700 1 -- 172.21.15.88:0/2762337994 wait complete. 2024-02-19T18:51:32.519 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:32 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2762337994' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-19T18:51:32.520 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:32 smithi088 ceph-mon[29272]: mgrmap e9: smithi088.gyviqq(active, since 25s) 2024-02-19T18:51:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout { 2024-02-19T18:51:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-19T18:51:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-19T18:51:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "active_name": "smithi088.gyviqq", 2024-02-19T18:51:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-19T18:51:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout } 2024-02-19T18:51:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.298+0000 7ff23e1a0700 1 Processor -- start 2024-02-19T18:51:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.298+0000 7ff23e1a0700 1 -- start start 2024-02-19T18:51:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.298+0000 7ff23e1a0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff238105bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.298+0000 7ff23e1a0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff238106130 con 0x7ff2381057e0 2024-02-19T18:51:33.125 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.298+0000 7ff23d19e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff238105bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.298+0000 7ff23d19e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff238105bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50840/0 (socket says 172.21.15.88:50840) 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.298+0000 7ff23d19e700 1 -- 172.21.15.88:0/183272172 learned_addr learned my addr 172.21.15.88:0/183272172 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.299+0000 7ff23d19e700 1 -- 172.21.15.88:0/183272172 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff238106270 con 0x7ff2381057e0 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.299+0000 7ff23d19e700 1 --2- 172.21.15.88:0/183272172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff238105bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2340098a0 tx=0x7ff234009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30942e82dea32fff server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.299+0000 7ff22ffff700 1 -- 172.21.15.88:0/183272172 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff234004180 con 0x7ff2381057e0 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.299+0000 7ff22ffff700 1 -- 172.21.15.88:0/183272172 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff2340042e0 con 0x7ff2381057e0 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.299+0000 7ff22ffff700 1 -- 172.21.15.88:0/183272172 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff234011af0 con 0x7ff2381057e0 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.300+0000 7ff23e1a0700 1 -- 172.21.15.88:0/183272172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 msgr2=0x7ff238105bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.300+0000 7ff23e1a0700 1 --2- 172.21.15.88:0/183272172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff238105bf0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2340098a0 tx=0x7ff234009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.300+0000 7ff23e1a0700 1 -- 172.21.15.88:0/183272172 shutdown_connections 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.300+0000 7ff23e1a0700 1 --2- 172.21.15.88:0/183272172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff238105bf0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.300+0000 7ff23e1a0700 1 -- 172.21.15.88:0/183272172 >> 172.21.15.88:0/183272172 conn(0x7ff238100e30 msgr2=0x7ff238103260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.300+0000 7ff23e1a0700 1 -- 172.21.15.88:0/183272172 shutdown_connections 2024-02-19T18:51:33.126 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.300+0000 7ff23e1a0700 1 -- 172.21.15.88:0/183272172 wait complete. 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.300+0000 7ff23e1a0700 1 Processor -- start 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff23e1a0700 1 -- start start 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff23e1a0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff2381994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff23e1a0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff238199a10 con 0x7ff2381057e0 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff23d19e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff2381994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff23d19e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff2381994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50842/0 (socket says 172.21.15.88:50842) 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff23d19e700 1 -- 172.21.15.88:0/1659995518 learned_addr learned my addr 172.21.15.88:0/1659995518 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff23d19e700 1 -- 172.21.15.88:0/1659995518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff234009550 con 0x7ff2381057e0 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff23d19e700 1 --2- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff2381994d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff234006ba0 tx=0x7ff234011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff22e7fc700 1 -- 172.21.15.88:0/1659995518 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff234003b00 con 0x7ff2381057e0 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff22e7fc700 1 -- 172.21.15.88:0/1659995518 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff23401a8e0 con 0x7ff2381057e0 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff22e7fc700 1 -- 172.21.15.88:0/1659995518 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff2340199c0 con 0x7ff2381057e0 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.301+0000 7ff23e1a0700 1 -- 172.21.15.88:0/1659995518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff238199c10 con 0x7ff2381057e0 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.302+0000 7ff23e1a0700 1 -- 172.21.15.88:0/1659995518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23819a0b0 con 0x7ff2381057e0 2024-02-19T18:51:33.127 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.302+0000 7ff22e7fc700 1 -- 172.21.15.88:0/1659995518 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7ff234019c30 con 0x7ff2381057e0 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.303+0000 7ff22e7fc700 1 --2- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7ff224038440 0x7ff22403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.303+0000 7ff23e1a0700 1 -- 172.21.15.88:0/1659995518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff238062380 con 0x7ff2381057e0 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.303+0000 7ff22e7fc700 1 -- 172.21.15.88:0/1659995518 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff23404c630 con 0x7ff2381057e0 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.303+0000 7ff23c99d700 1 -- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7ff224038440 msgr2=0x7ff22403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/3414266099 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.303+0000 7ff23c99d700 1 --2- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7ff224038440 0x7ff22403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.308+0000 7ff22e7fc700 1 -- 172.21.15.88:0/1659995518 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff23402d440 con 0x7ff2381057e0 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.504+0000 7ff23c99d700 1 -- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7ff224038440 msgr2=0x7ff22403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/3414266099 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.504+0000 7ff23c99d700 1 --2- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7ff224038440 0x7ff22403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.559+0000 7ff23e1a0700 1 -- 172.21.15.88:0/1659995518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff23819d240 con 0x7ff2381057e0 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.559+0000 7ff22e7fc700 1 -- 172.21.15.88:0/1659995518 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7ff23402b340 con 0x7ff2381057e0 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.561+0000 7ff23e1a0700 1 -- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7ff224038440 msgr2=0x7ff22403a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T18:51:33.128 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.561+0000 7ff23e1a0700 1 --2- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7ff224038440 0x7ff22403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.561+0000 7ff23e1a0700 1 -- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 msgr2=0x7ff2381994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.561+0000 7ff23e1a0700 1 --2- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff2381994d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff234006ba0 tx=0x7ff234011b50 comp rx=0 tx=0).stop 2024-02-19T18:51:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.561+0000 7ff23e1a0700 1 -- 172.21.15.88:0/1659995518 shutdown_connections 2024-02-19T18:51:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.561+0000 7ff23e1a0700 1 --2- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7ff224038440 0x7ff22403a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.561+0000 7ff23e1a0700 1 --2- 172.21.15.88:0/1659995518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2381057e0 0x7ff2381994d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.561+0000 7ff23e1a0700 1 -- 172.21.15.88:0/1659995518 >> 172.21.15.88:0/1659995518 conn(0x7ff238100e30 msgr2=0x7ff238101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.561+0000 7ff23e1a0700 1 -- 172.21.15.88:0/1659995518 shutdown_connections 2024-02-19T18:51:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:32.561+0000 7ff23e1a0700 1 -- 172.21.15.88:0/1659995518 wait complete. 2024-02-19T18:51:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:Waiting for the mgr to restart... 2024-02-19T18:51:33.129 INFO:teuthology.orchestra.run.smithi088.stdout:Waiting for mgr epoch 9... 2024-02-19T18:51:33.413 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:33 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1659995518' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-19T18:51:38.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:37 smithi088 ceph-mon[29272]: Active manager daemon smithi088.gyviqq restarted 2024-02-19T18:51:38.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:37 smithi088 ceph-mon[29272]: Activating manager daemon smithi088.gyviqq 2024-02-19T18:51:38.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:37 smithi088 ceph-mon[29272]: osdmap e3: 0 total, 0 up, 0 in 2024-02-19T18:51:38.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:37 smithi088 ceph-mon[29272]: mgrmap e10: smithi088.gyviqq(active, starting, since 0.00313183s) 2024-02-19T18:51:38.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:37 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T18:51:38.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:37 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi088.gyviqq", "id": "smithi088.gyviqq"}]: dispatch 2024-02-19T18:51:38.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:37 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T18:51:38.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:37 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T18:51:38.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:37 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T18:51:38.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:37 smithi088 ceph-mon[29272]: Manager daemon smithi088.gyviqq is now available 2024-02-19T18:51:38.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:38 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:51:38.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:38 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:51:38.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:38 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/mirror_snapshot_schedule"}]: dispatch 2024-02-19T18:51:38.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:38 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/trash_purge_schedule"}]: dispatch 2024-02-19T18:51:38.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:38 smithi088 ceph-mon[29272]: [19/Feb/2024:18:51:38] ENGINE Bus STARTING 2024-02-19T18:51:38.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:38 smithi088 ceph-mon[29272]: mgrmap e11: smithi088.gyviqq(active, since 1.00558s) 2024-02-19T18:51:39.551 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout { 2024-02-19T18:51:39.551 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-19T18:51:39.551 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout } 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.516+0000 7f455d170700 1 Processor -- start 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.516+0000 7f455d170700 1 -- start start 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.516+0000 7f455d170700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.516+0000 7f455d170700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45581045f0 con 0x7f4558103ca0 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.516+0000 7f4556d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.516+0000 7f4556d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581040b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50850/0 (socket says 172.21.15.88:50850) 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.516+0000 7f4556d9d700 1 -- 172.21.15.88:0/3952382428 learned_addr learned my addr 172.21.15.88:0/3952382428 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.516+0000 7f4556d9d700 1 -- 172.21.15.88:0/3952382428 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558104730 con 0x7f4558103ca0 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.516+0000 7f4556d9d700 1 --2- 172.21.15.88:0/3952382428 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581040b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f45480098a0 tx=0x7f4548009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b63cc9e448e511ac server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.516+0000 7f4555d9b700 1 -- 172.21.15.88:0/3952382428 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4548004180 con 0x7f4558103ca0 2024-02-19T18:51:39.552 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.517+0000 7f4555d9b700 1 -- 172.21.15.88:0/3952382428 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f45480042e0 con 0x7f4558103ca0 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.517+0000 7f4555d9b700 1 -- 172.21.15.88:0/3952382428 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4548011af0 con 0x7f4558103ca0 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.517+0000 7f455d170700 1 -- 172.21.15.88:0/3952382428 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 msgr2=0x7f45581040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.517+0000 7f455d170700 1 --2- 172.21.15.88:0/3952382428 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581040b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f45480098a0 tx=0x7f4548009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.517+0000 7f455d170700 1 -- 172.21.15.88:0/3952382428 shutdown_connections 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.517+0000 7f455d170700 1 --2- 172.21.15.88:0/3952382428 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581040b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.517+0000 7f455d170700 1 -- 172.21.15.88:0/3952382428 >> 172.21.15.88:0/3952382428 conn(0x7f45580ff6a0 msgr2=0x7f4558101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.517+0000 7f455d170700 1 -- 172.21.15.88:0/3952382428 shutdown_connections 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.517+0000 7f455d170700 1 -- 172.21.15.88:0/3952382428 wait complete. 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.518+0000 7f455d170700 1 Processor -- start 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.518+0000 7f455d170700 1 -- start start 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.518+0000 7f455d170700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.518+0000 7f455d170700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4558199ce0 con 0x7f4558103ca0 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.518+0000 7f4556d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.518+0000 7f4556d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50854/0 (socket says 172.21.15.88:50854) 2024-02-19T18:51:39.553 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.518+0000 7f4556d9d700 1 -- 172.21.15.88:0/4036547382 learned_addr learned my addr 172.21.15.88:0/4036547382 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.519+0000 7f4556d9d700 1 -- 172.21.15.88:0/4036547382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4548009550 con 0x7f4558103ca0 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.519+0000 7f4556d9d700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581997a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4548011670 tx=0x7f4548011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.519+0000 7f453ffff700 1 -- 172.21.15.88:0/4036547382 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45480039c0 con 0x7f4558103ca0 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.519+0000 7f453ffff700 1 -- 172.21.15.88:0/4036547382 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4548003b20 con 0x7f4558103ca0 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.519+0000 7f455d170700 1 -- 172.21.15.88:0/4036547382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4558199ee0 con 0x7f4558103ca0 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.519+0000 7f453ffff700 1 -- 172.21.15.88:0/4036547382 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45480247f0 con 0x7f4558103ca0 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.519+0000 7f455d170700 1 -- 172.21.15.88:0/4036547382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455819a380 con 0x7f4558103ca0 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.520+0000 7f453ffff700 1 -- 172.21.15.88:0/4036547382 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f4548024ab0 con 0x7f4558103ca0 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.520+0000 7f453ffff700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 0x7f454003a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.520+0000 7f455d170700 1 -- 172.21.15.88:0/4036547382 --> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4558062380 con 0x7f4540038400 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.520+0000 7f453ffff700 1 -- 172.21.15.88:0/4036547382 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f454804d410 con 0x7f4558103ca0 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.520+0000 7f455659c700 1 -- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 msgr2=0x7f454003a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/3414266099 2024-02-19T18:51:39.554 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.520+0000 7f455659c700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 0x7f454003a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.721+0000 7f455659c700 1 -- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 msgr2=0x7f454003a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/3414266099 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:33.721+0000 7f455659c700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 0x7f454003a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:34.122+0000 7f455659c700 1 -- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 msgr2=0x7f454003a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/3414266099 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:34.122+0000 7f455659c700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 0x7f454003a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:34.923+0000 7f455659c700 1 -- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 msgr2=0x7f454003a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/3414266099 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:34.923+0000 7f455659c700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 0x7f454003a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:36.524+0000 7f455659c700 1 -- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 msgr2=0x7f454003a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/3414266099 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:36.524+0000 7f455659c700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 0x7f454003a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:37.846+0000 7f453ffff700 1 -- 172.21.15.88:0/4036547382 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mgrmap(e 10) v1 ==== 44864+0+0 (secure 0 0 0) 0x7f454801ac70 con 0x7f4558103ca0 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:37.846+0000 7f453ffff700 1 -- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 msgr2=0x7f454003a8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:37.846+0000 7f453ffff700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 0x7f454003a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.847+0000 7f453ffff700 1 -- 172.21.15.88:0/4036547382 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7f454804df40 con 0x7f4558103ca0 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.848+0000 7f453ffff700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f454003bd00 0x7f454003e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:39.555 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.848+0000 7f453ffff700 1 -- 172.21.15.88:0/4036547382 --> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4558062380 con 0x7f454003bd00 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.848+0000 7f455659c700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f454003bd00 0x7f454003e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.848+0000 7f455659c700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f454003bd00 0x7f454003e0e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f454c003a10 tx=0x7f454c0092b0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.849+0000 7f453ffff700 1 -- 172.21.15.88:0/4036547382 <== mgr.14164 v2:172.21.15.88:6800/3043610921 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f4558062380 con 0x7f454003bd00 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.855+0000 7f455d170700 1 -- 172.21.15.88:0/4036547382 --> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f455819aae0 con 0x7f454003bd00 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.855+0000 7f453ffff700 1 -- 172.21.15.88:0/4036547382 <== mgr.14164 v2:172.21.15.88:6800/3043610921 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f455819aae0 con 0x7f454003bd00 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.855+0000 7f455d170700 1 -- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f454003bd00 msgr2=0x7f454003e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.855+0000 7f455d170700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f454003bd00 0x7f454003e0e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f454c003a10 tx=0x7f454c0092b0 comp rx=0 tx=0).stop 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.855+0000 7f455d170700 1 -- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 msgr2=0x7f45581997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.855+0000 7f455d170700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581997a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4548011670 tx=0x7f4548011aa0 comp rx=0 tx=0).stop 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.855+0000 7f455d170700 1 -- 172.21.15.88:0/4036547382 shutdown_connections 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.855+0000 7f455d170700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f454003bd00 0x7f454003e0e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.855+0000 7f455d170700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4558103ca0 0x7f45581997a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:39.556 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.855+0000 7f455d170700 1 --2- 172.21.15.88:0/4036547382 >> [v2:172.21.15.88:6800/3414266099,v1:172.21.15.88:6801/3414266099] conn(0x7f4540038400 0x7f454003a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:39.557 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.855+0000 7f455d170700 1 -- 172.21.15.88:0/4036547382 >> 172.21.15.88:0/4036547382 conn(0x7f45580ff6a0 msgr2=0x7f4558100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:39.557 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.856+0000 7f455d170700 1 -- 172.21.15.88:0/4036547382 shutdown_connections 2024-02-19T18:51:39.557 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:38.856+0000 7f455d170700 1 -- 172.21.15.88:0/4036547382 wait complete. 2024-02-19T18:51:39.557 INFO:teuthology.orchestra.run.smithi088.stdout:mgr epoch 9 is available 2024-02-19T18:51:39.557 INFO:teuthology.orchestra.run.smithi088.stdout:Generating a dashboard self-signed certificate... 2024-02-19T18:51:40.515 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:40 smithi088 ceph-mon[29272]: [19/Feb/2024:18:51:38] ENGINE Serving on https://172.21.15.88:7150 2024-02-19T18:51:40.515 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:40 smithi088 ceph-mon[29272]: [19/Feb/2024:18:51:38] ENGINE Serving on http://172.21.15.88:8765 2024-02-19T18:51:40.515 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:40 smithi088 ceph-mon[29272]: [19/Feb/2024:18:51:38] ENGINE Bus STARTED 2024-02-19T18:51:40.515 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:40 smithi088 ceph-mon[29272]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-19T18:51:40.515 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:40 smithi088 ceph-mon[29272]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-19T18:51:40.515 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:40 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:40.515 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:40 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:40.948 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-19T18:51:40.948 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.030+0000 7f9952f25700 1 Processor -- start 2024-02-19T18:51:40.948 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.030+0000 7f9952f25700 1 -- start start 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.030+0000 7f9952f25700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c1051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.030+0000 7f9952f25700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c105710 con 0x7f994c104dc0 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.030+0000 7f9950cc1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c1051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.031+0000 7f9950cc1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c1051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56044/0 (socket says 172.21.15.88:56044) 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.031+0000 7f9950cc1700 1 -- 172.21.15.88:0/601192434 learned_addr learned my addr 172.21.15.88:0/601192434 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.031+0000 7f9950cc1700 1 -- 172.21.15.88:0/601192434 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994c105850 con 0x7f994c104dc0 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.031+0000 7f9950cc1700 1 --2- 172.21.15.88:0/601192434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c1051d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f99400098a0 tx=0x7f9940009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77f70dcb997ae29f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.031+0000 7f994b7fe700 1 -- 172.21.15.88:0/601192434 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9940004180 con 0x7f994c104dc0 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.031+0000 7f994b7fe700 1 -- 172.21.15.88:0/601192434 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f99400042e0 con 0x7f994c104dc0 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.031+0000 7f994b7fe700 1 -- 172.21.15.88:0/601192434 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9940011af0 con 0x7f994c104dc0 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.032+0000 7f9952f25700 1 -- 172.21.15.88:0/601192434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 msgr2=0x7f994c1051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.032+0000 7f9952f25700 1 --2- 172.21.15.88:0/601192434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c1051d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f99400098a0 tx=0x7f9940009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:40.949 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.032+0000 7f9952f25700 1 -- 172.21.15.88:0/601192434 shutdown_connections 2024-02-19T18:51:40.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.032+0000 7f9952f25700 1 --2- 172.21.15.88:0/601192434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c1051d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:40.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.032+0000 7f9952f25700 1 -- 172.21.15.88:0/601192434 >> 172.21.15.88:0/601192434 conn(0x7f994c0ff670 msgr2=0x7f994c101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:40.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.032+0000 7f9952f25700 1 -- 172.21.15.88:0/601192434 shutdown_connections 2024-02-19T18:51:40.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.032+0000 7f9952f25700 1 -- 172.21.15.88:0/601192434 wait complete. 2024-02-19T18:51:40.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.033+0000 7f9952f25700 1 Processor -- start 2024-02-19T18:51:40.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.033+0000 7f9952f25700 1 -- start start 2024-02-19T18:51:40.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.033+0000 7f9952f25700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c104520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:40.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.033+0000 7f9952f25700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c19c4b0 con 0x7f994c104dc0 2024-02-19T18:51:40.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.033+0000 7f9950cc1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c104520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:40.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.033+0000 7f9950cc1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c104520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56046/0 (socket says 172.21.15.88:56046) 2024-02-19T18:51:40.950 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.033+0000 7f9950cc1700 1 -- 172.21.15.88:0/2706160796 learned_addr learned my addr 172.21.15.88:0/2706160796 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.033+0000 7f9950cc1700 1 -- 172.21.15.88:0/2706160796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9940009550 con 0x7f994c104dc0 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.033+0000 7f9950cc1700 1 --2- 172.21.15.88:0/2706160796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c104520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9940011670 tx=0x7f9940011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.033+0000 7f9949ffb700 1 -- 172.21.15.88:0/2706160796 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99400039c0 con 0x7f994c104dc0 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.034+0000 7f9949ffb700 1 -- 172.21.15.88:0/2706160796 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9940003b20 con 0x7f994c104dc0 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.034+0000 7f9952f25700 1 -- 172.21.15.88:0/2706160796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994c19c5f0 con 0x7f994c104dc0 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.034+0000 7f9949ffb700 1 -- 172.21.15.88:0/2706160796 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99400247f0 con 0x7f994c104dc0 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.034+0000 7f9952f25700 1 -- 172.21.15.88:0/2706160796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c19ca90 con 0x7f994c104dc0 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.034+0000 7f9949ffb700 1 -- 172.21.15.88:0/2706160796 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7f9940024a60 con 0x7f994c104dc0 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.035+0000 7f9949ffb700 1 --2- 172.21.15.88:0/2706160796 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f993c038330 0x7f993c03a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.035+0000 7f9949ffb700 1 -- 172.21.15.88:0/2706160796 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f994004d2b0 con 0x7f994c104dc0 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.035+0000 7f994bfff700 1 --2- 172.21.15.88:0/2706160796 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f993c038330 0x7f993c03a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.035+0000 7f9952f25700 1 -- 172.21.15.88:0/2706160796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994c062380 con 0x7f994c104dc0 2024-02-19T18:51:40.951 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.035+0000 7f994bfff700 1 --2- 172.21.15.88:0/2706160796 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f993c038330 0x7f993c03a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9938006fd0 tx=0x7f9938006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:40.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.040+0000 7f9949ffb700 1 -- 172.21.15.88:0/2706160796 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9940024d60 con 0x7f994c104dc0 2024-02-19T18:51:40.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.246+0000 7f9952f25700 1 -- 172.21.15.88:0/2706160796 --> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f994c100b00 con 0x7f993c038330 2024-02-19T18:51:40.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.261+0000 7f9949ffb700 1 -- 172.21.15.88:0/2706160796 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f994001ac70 con 0x7f994c104dc0 2024-02-19T18:51:40.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.314+0000 7f9949ffb700 1 -- 172.21.15.88:0/2706160796 <== mgr.14164 v2:172.21.15.88:6800/3043610921 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f994c100b00 con 0x7f993c038330 2024-02-19T18:51:40.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.318+0000 7f9952f25700 1 -- 172.21.15.88:0/2706160796 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f993c038330 msgr2=0x7f993c03a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:40.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.318+0000 7f9952f25700 1 --2- 172.21.15.88:0/2706160796 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f993c038330 0x7f993c03a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9938006fd0 tx=0x7f9938006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:40.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.318+0000 7f9952f25700 1 -- 172.21.15.88:0/2706160796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 msgr2=0x7f994c104520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:40.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.318+0000 7f9952f25700 1 --2- 172.21.15.88:0/2706160796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c104520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9940011670 tx=0x7f9940011aa0 comp rx=0 tx=0).stop 2024-02-19T18:51:40.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.318+0000 7f9952f25700 1 -- 172.21.15.88:0/2706160796 shutdown_connections 2024-02-19T18:51:40.952 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.318+0000 7f9952f25700 1 --2- 172.21.15.88:0/2706160796 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f993c038330 0x7f993c03a7e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:40.953 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.318+0000 7f9952f25700 1 --2- 172.21.15.88:0/2706160796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994c104dc0 0x7f994c104520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:40.953 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.318+0000 7f9952f25700 1 -- 172.21.15.88:0/2706160796 >> 172.21.15.88:0/2706160796 conn(0x7f994c0ff670 msgr2=0x7f994c101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:40.953 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.318+0000 7f9952f25700 1 -- 172.21.15.88:0/2706160796 shutdown_connections 2024-02-19T18:51:40.953 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:40.318+0000 7f9952f25700 1 -- 172.21.15.88:0/2706160796 wait complete. 2024-02-19T18:51:40.953 INFO:teuthology.orchestra.run.smithi088.stdout:Creating initial admin user... 2024-02-19T18:51:41.485 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:41 smithi088 ceph-mon[29272]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:41.485 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:41 smithi088 ceph-mon[29272]: mgrmap e12: smithi088.gyviqq(active, since 2s) 2024-02-19T18:51:41.486 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:41 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:41.486 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:41 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:43.011 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:42 smithi088 ceph-mon[29272]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:43.011 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:42 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:43.019 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$5ClFU98xDfvwK2Ri2APrXeiraYfT2cCOGdVVVUmmVP0j.VTgUnhd2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708368701, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-19T18:51:43.019 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.364+0000 7f32f52a9700 1 Processor -- start 2024-02-19T18:51:43.019 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.364+0000 7f32f52a9700 1 -- start start 2024-02-19T18:51:43.019 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.365+0000 7f32f52a9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01088e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:43.019 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.365+0000 7f32f52a9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f0079d10 con 0x7f32f0106500 2024-02-19T18:51:43.019 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.365+0000 7f32eeffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01088e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:43.019 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.365+0000 7f32eeffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01088e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56056/0 (socket says 172.21.15.88:56056) 2024-02-19T18:51:43.019 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.365+0000 7f32eeffd700 1 -- 172.21.15.88:0/607335003 learned_addr learned my addr 172.21.15.88:0/607335003 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:43.019 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.365+0000 7f32eeffd700 1 -- 172.21.15.88:0/607335003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f0079e50 con 0x7f32f0106500 2024-02-19T18:51:43.019 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.365+0000 7f32eeffd700 1 --2- 172.21.15.88:0/607335003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01088e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f32e00098a0 tx=0x7f32e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30f9af85ed30c42 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:43.020 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.365+0000 7f32edffb700 1 -- 172.21.15.88:0/607335003 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32e0004180 con 0x7f32f0106500 2024-02-19T18:51:43.020 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.365+0000 7f32edffb700 1 -- 172.21.15.88:0/607335003 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32e00042e0 con 0x7f32f0106500 2024-02-19T18:51:43.020 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.365+0000 7f32edffb700 1 -- 172.21.15.88:0/607335003 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32e0011af0 con 0x7f32f0106500 2024-02-19T18:51:43.020 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.366+0000 7f32f52a9700 1 -- 172.21.15.88:0/607335003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 msgr2=0x7f32f01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:43.020 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.366+0000 7f32f52a9700 1 --2- 172.21.15.88:0/607335003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01088e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f32e00098a0 tx=0x7f32e0009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:43.020 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.366+0000 7f32f52a9700 1 -- 172.21.15.88:0/607335003 shutdown_connections 2024-02-19T18:51:43.020 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.366+0000 7f32f52a9700 1 --2- 172.21.15.88:0/607335003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01088e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:43.020 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.366+0000 7f32f52a9700 1 -- 172.21.15.88:0/607335003 >> 172.21.15.88:0/607335003 conn(0x7f32f00ffee0 msgr2=0x7f32f0102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:43.020 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.366+0000 7f32f52a9700 1 -- 172.21.15.88:0/607335003 shutdown_connections 2024-02-19T18:51:43.020 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.366+0000 7f32f52a9700 1 -- 172.21.15.88:0/607335003 wait complete. 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.367+0000 7f32f52a9700 1 Processor -- start 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.367+0000 7f32f52a9700 1 -- start start 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.367+0000 7f32f52a9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.367+0000 7f32f52a9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f0199c30 con 0x7f32f0106500 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.367+0000 7f32eeffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.367+0000 7f32eeffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56068/0 (socket says 172.21.15.88:56068) 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.367+0000 7f32eeffd700 1 -- 172.21.15.88:0/3297285225 learned_addr learned my addr 172.21.15.88:0/3297285225 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.368+0000 7f32eeffd700 1 -- 172.21.15.88:0/3297285225 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0009550 con 0x7f32f0106500 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.368+0000 7f32eeffd700 1 --2- 172.21.15.88:0/3297285225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01996f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f32e0011670 tx=0x7f32e0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.368+0000 7f32d7fff700 1 -- 172.21.15.88:0/3297285225 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32e00039c0 con 0x7f32f0106500 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.368+0000 7f32d7fff700 1 -- 172.21.15.88:0/3297285225 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f32e0003b20 con 0x7f32f0106500 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.368+0000 7f32f52a9700 1 -- 172.21.15.88:0/3297285225 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f0199e90 con 0x7f32f0106500 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.368+0000 7f32d7fff700 1 -- 172.21.15.88:0/3297285225 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f32e00247f0 con 0x7f32f0106500 2024-02-19T18:51:43.021 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.368+0000 7f32f52a9700 1 -- 172.21.15.88:0/3297285225 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f019cc90 con 0x7f32f0106500 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.369+0000 7f32d7fff700 1 -- 172.21.15.88:0/3297285225 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f32e0024ab0 con 0x7f32f0106500 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.369+0000 7f32d7fff700 1 --2- 172.21.15.88:0/3297285225 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f32d8038400 0x7f32d803a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.369+0000 7f32d7fff700 1 -- 172.21.15.88:0/3297285225 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f32e001a430 con 0x7f32f0106500 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.369+0000 7f32ee7fc700 1 --2- 172.21.15.88:0/3297285225 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f32d8038400 0x7f32d803a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.369+0000 7f32ee7fc700 1 --2- 172.21.15.88:0/3297285225 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f32d8038400 0x7f32d803a8b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f32e4006fd0 tx=0x7f32e4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.370+0000 7f32f52a9700 1 -- 172.21.15.88:0/3297285225 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f0062380 con 0x7f32f0106500 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.375+0000 7f32d7fff700 1 -- 172.21.15.88:0/3297285225 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f32e0022310 con 0x7f32f0106500 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.586+0000 7f32f52a9700 1 -- 172.21.15.88:0/3297285225 --> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f32f00621a0 con 0x7f32d8038400 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.810+0000 7f32d7fff700 1 -- 172.21.15.88:0/3297285225 <== mgr.14164 v2:172.21.15.88:6800/3043610921 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f32f00621a0 con 0x7f32d8038400 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.812+0000 7f32f52a9700 1 -- 172.21.15.88:0/3297285225 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f32d8038400 msgr2=0x7f32d803a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.812+0000 7f32f52a9700 1 --2- 172.21.15.88:0/3297285225 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f32d8038400 0x7f32d803a8b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f32e4006fd0 tx=0x7f32e4006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.812+0000 7f32f52a9700 1 -- 172.21.15.88:0/3297285225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 msgr2=0x7f32f01996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.812+0000 7f32f52a9700 1 --2- 172.21.15.88:0/3297285225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01996f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f32e0011670 tx=0x7f32e0011aa0 comp rx=0 tx=0).stop 2024-02-19T18:51:43.022 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.812+0000 7f32f52a9700 1 -- 172.21.15.88:0/3297285225 shutdown_connections 2024-02-19T18:51:43.023 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.812+0000 7f32f52a9700 1 --2- 172.21.15.88:0/3297285225 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f32d8038400 0x7f32d803a8b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:43.023 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.812+0000 7f32f52a9700 1 --2- 172.21.15.88:0/3297285225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32f0106500 0x7f32f01996f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:43.023 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.812+0000 7f32f52a9700 1 -- 172.21.15.88:0/3297285225 >> 172.21.15.88:0/3297285225 conn(0x7f32f00ffee0 msgr2=0x7f32f0102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:43.023 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.813+0000 7f32f52a9700 1 -- 172.21.15.88:0/3297285225 shutdown_connections 2024-02-19T18:51:43.023 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:41.813+0000 7f32f52a9700 1 -- 172.21.15.88:0/3297285225 wait complete. 2024-02-19T18:51:43.023 INFO:teuthology.orchestra.run.smithi088.stdout:Fetching dashboard port number... 2024-02-19T18:51:43.959 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:43 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2072518929' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-19T18:51:44.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stdout 8443 2024-02-19T18:51:44.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.466+0000 7f593a37c700 1 Processor -- start 2024-02-19T18:51:44.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.466+0000 7f593a37c700 1 -- start start 2024-02-19T18:51:44.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.466+0000 7f593a37c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934105dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:44.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.466+0000 7f593a37c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5934106300 con 0x7f59341059b0 2024-02-19T18:51:44.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.467+0000 7f5933fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934105dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:44.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.467+0000 7f5933fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934105dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56080/0 (socket says 172.21.15.88:56080) 2024-02-19T18:51:44.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.467+0000 7f5933fff700 1 -- 172.21.15.88:0/663532149 learned_addr learned my addr 172.21.15.88:0/663532149 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:44.200 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.467+0000 7f5933fff700 1 -- 172.21.15.88:0/663532149 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5934106440 con 0x7f59341059b0 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.467+0000 7f5933fff700 1 --2- 172.21.15.88:0/663532149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934105dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59240098a0 tx=0x7f5924009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ef762ce2b446ce server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.467+0000 7f5932ffd700 1 -- 172.21.15.88:0/663532149 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5924004180 con 0x7f59341059b0 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.467+0000 7f5932ffd700 1 -- 172.21.15.88:0/663532149 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f59240042e0 con 0x7f59341059b0 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.467+0000 7f5932ffd700 1 -- 172.21.15.88:0/663532149 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5924011af0 con 0x7f59341059b0 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.468+0000 7f593a37c700 1 -- 172.21.15.88:0/663532149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 msgr2=0x7f5934105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.468+0000 7f593a37c700 1 --2- 172.21.15.88:0/663532149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934105dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f59240098a0 tx=0x7f5924009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.468+0000 7f593a37c700 1 -- 172.21.15.88:0/663532149 shutdown_connections 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.468+0000 7f593a37c700 1 --2- 172.21.15.88:0/663532149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934105dc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.468+0000 7f593a37c700 1 -- 172.21.15.88:0/663532149 >> 172.21.15.88:0/663532149 conn(0x7f5934101020 msgr2=0x7f5934103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.468+0000 7f593a37c700 1 -- 172.21.15.88:0/663532149 shutdown_connections 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.468+0000 7f593a37c700 1 -- 172.21.15.88:0/663532149 wait complete. 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.469+0000 7f593a37c700 1 Processor -- start 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.469+0000 7f593a37c700 1 -- start start 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.469+0000 7f593a37c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:44.201 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.469+0000 7f593a37c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5934199c40 con 0x7f59341059b0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.469+0000 7f5933fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.469+0000 7f5933fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934199700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56096/0 (socket says 172.21.15.88:56096) 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.469+0000 7f5933fff700 1 -- 172.21.15.88:0/2072518929 learned_addr learned my addr 172.21.15.88:0/2072518929 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.469+0000 7f5933fff700 1 -- 172.21.15.88:0/2072518929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5924009550 con 0x7f59341059b0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.469+0000 7f5933fff700 1 --2- 172.21.15.88:0/2072518929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934199700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5924006ba0 tx=0x7f5924011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.469+0000 7f59317fa700 1 -- 172.21.15.88:0/2072518929 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5924003b00 con 0x7f59341059b0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.470+0000 7f59317fa700 1 -- 172.21.15.88:0/2072518929 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f592401a8e0 con 0x7f59341059b0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.470+0000 7f593a37c700 1 -- 172.21.15.88:0/2072518929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5934199e40 con 0x7f59341059b0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.470+0000 7f59317fa700 1 -- 172.21.15.88:0/2072518929 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f59240199c0 con 0x7f59341059b0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.470+0000 7f593a37c700 1 -- 172.21.15.88:0/2072518929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593419a2e0 con 0x7f59341059b0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.470+0000 7f59317fa700 1 -- 172.21.15.88:0/2072518929 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f5924019c30 con 0x7f59341059b0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.471+0000 7f593a37c700 1 -- 172.21.15.88:0/2072518929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5934062380 con 0x7f59341059b0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.471+0000 7f59317fa700 1 --2- 172.21.15.88:0/2072518929 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f591c038440 0x7f591c03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.471+0000 7f59337fe700 1 --2- 172.21.15.88:0/2072518929 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f591c038440 0x7f591c03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:44.202 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.471+0000 7f59317fa700 1 -- 172.21.15.88:0/2072518929 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f592404c8a0 con 0x7f59341059b0 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.471+0000 7f59337fe700 1 --2- 172.21.15.88:0/2072518929 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f591c038440 0x7f591c03a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5928006fd0 tx=0x7f5928006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.475+0000 7f59317fa700 1 -- 172.21.15.88:0/2072518929 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5924018b00 con 0x7f59341059b0 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.665+0000 7f593a37c700 1 -- 172.21.15.88:0/2072518929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f593419d1e0 con 0x7f59341059b0 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.666+0000 7f59317fa700 1 -- 172.21.15.88:0/2072518929 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f5924018b00 con 0x7f59341059b0 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.668+0000 7f593a37c700 1 -- 172.21.15.88:0/2072518929 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f591c038440 msgr2=0x7f591c03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.668+0000 7f593a37c700 1 --2- 172.21.15.88:0/2072518929 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f591c038440 0x7f591c03a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5928006fd0 tx=0x7f5928006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.668+0000 7f593a37c700 1 -- 172.21.15.88:0/2072518929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 msgr2=0x7f5934199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.668+0000 7f593a37c700 1 --2- 172.21.15.88:0/2072518929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934199700 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5924006ba0 tx=0x7f5924011b50 comp rx=0 tx=0).stop 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.668+0000 7f593a37c700 1 -- 172.21.15.88:0/2072518929 shutdown_connections 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.668+0000 7f593a37c700 1 --2- 172.21.15.88:0/2072518929 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f591c038440 0x7f591c03a8f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.668+0000 7f593a37c700 1 --2- 172.21.15.88:0/2072518929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59341059b0 0x7f5934199700 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.668+0000 7f593a37c700 1 -- 172.21.15.88:0/2072518929 >> 172.21.15.88:0/2072518929 conn(0x7f5934101020 msgr2=0x7f5934101da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.668+0000 7f593a37c700 1 -- 172.21.15.88:0/2072518929 shutdown_connections 2024-02-19T18:51:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:43.668+0000 7f593a37c700 1 -- 172.21.15.88:0/2072518929 wait complete. 2024-02-19T18:51:44.207 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-19T18:51:44.207 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout disabled 2024-02-19T18:51:44.214 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-19T18:51:44.214 INFO:teuthology.orchestra.run.smithi088.stdout:systemctl: stdout inactive 2024-02-19T18:51:44.214 INFO:teuthology.orchestra.run.smithi088.stdout:firewalld.service is not enabled 2024-02-19T18:51:44.214 INFO:teuthology.orchestra.run.smithi088.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-19T18:51:44.216 INFO:teuthology.orchestra.run.smithi088.stdout:Ceph Dashboard is now available at: 2024-02-19T18:51:44.216 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:51:44.216 INFO:teuthology.orchestra.run.smithi088.stdout: URL: https://smithi088.front.sepia.ceph.com:8443/ 2024-02-19T18:51:44.216 INFO:teuthology.orchestra.run.smithi088.stdout: User: admin 2024-02-19T18:51:44.216 INFO:teuthology.orchestra.run.smithi088.stdout: Password: ghvqry4lsm 2024-02-19T18:51:44.216 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:51:44.217 INFO:teuthology.orchestra.run.smithi088.stdout:Saving cluster configuration to /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config directory 2024-02-19T18:51:44.217 INFO:teuthology.orchestra.run.smithi088.stdout:Enabling autotune for osd_memory_target 2024-02-19T18:51:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.615+0000 7f944de9e700 1 Processor -- start 2024-02-19T18:51:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.615+0000 7f944de9e700 1 -- start start 2024-02-19T18:51:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.615+0000 7f944de9e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448105de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.615+0000 7f944de9e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9448106320 con 0x7f94481059d0 2024-02-19T18:51:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.615+0000 7f94477fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.615+0000 7f94477fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56112/0 (socket says 172.21.15.88:56112) 2024-02-19T18:51:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.615+0000 7f94477fe700 1 -- 172.21.15.88:0/80764224 learned_addr learned my addr 172.21.15.88:0/80764224 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.615+0000 7f94477fe700 1 -- 172.21.15.88:0/80764224 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9448106460 con 0x7f94481059d0 2024-02-19T18:51:45.373 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.616+0000 7f94477fe700 1 --2- 172.21.15.88:0/80764224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448105de0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9438009870 tx=0x7f9438009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77645c5a5e46c45f server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.616+0000 7f94467fc700 1 -- 172.21.15.88:0/80764224 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9438005c30 con 0x7f94481059d0 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.616+0000 7f94467fc700 1 -- 172.21.15.88:0/80764224 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f94380044e0 con 0x7f94481059d0 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.616+0000 7f94467fc700 1 -- 172.21.15.88:0/80764224 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9438006030 con 0x7f94481059d0 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.616+0000 7f944de9e700 1 -- 172.21.15.88:0/80764224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 msgr2=0x7f9448105de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.616+0000 7f944de9e700 1 --2- 172.21.15.88:0/80764224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448105de0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9438009870 tx=0x7f9438009b80 comp rx=0 tx=0).stop 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.616+0000 7f944de9e700 1 -- 172.21.15.88:0/80764224 shutdown_connections 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.616+0000 7f944de9e700 1 --2- 172.21.15.88:0/80764224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448105de0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.616+0000 7f944de9e700 1 -- 172.21.15.88:0/80764224 >> 172.21.15.88:0/80764224 conn(0x7f9448101020 msgr2=0x7f9448103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.616+0000 7f944de9e700 1 -- 172.21.15.88:0/80764224 shutdown_connections 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.616+0000 7f944de9e700 1 -- 172.21.15.88:0/80764224 wait complete. 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.617+0000 7f944de9e700 1 Processor -- start 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.617+0000 7f944de9e700 1 -- start start 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.617+0000 7f944de9e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.617+0000 7f944de9e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9448199c60 con 0x7f94481059d0 2024-02-19T18:51:45.374 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.617+0000 7f94477fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.617+0000 7f94477fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56118/0 (socket says 172.21.15.88:56118) 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.617+0000 7f94477fe700 1 -- 172.21.15.88:0/3692222805 learned_addr learned my addr 172.21.15.88:0/3692222805 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.618+0000 7f94477fe700 1 -- 172.21.15.88:0/3692222805 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9438009500 con 0x7f94481059d0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.618+0000 7f94477fe700 1 --2- 172.21.15.88:0/3692222805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448199720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9438004840 tx=0x7f94380042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.618+0000 7f9444ff9700 1 -- 172.21.15.88:0/3692222805 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9438005e40 con 0x7f94481059d0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.618+0000 7f944de9e700 1 -- 172.21.15.88:0/3692222805 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9448199e60 con 0x7f94481059d0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.618+0000 7f9444ff9700 1 -- 172.21.15.88:0/3692222805 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9438005fa0 con 0x7f94481059d0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.618+0000 7f9444ff9700 1 -- 172.21.15.88:0/3692222805 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9438011c80 con 0x7f94481059d0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.618+0000 7f944de9e700 1 -- 172.21.15.88:0/3692222805 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f944819a300 con 0x7f94481059d0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.619+0000 7f9444ff9700 1 -- 172.21.15.88:0/3692222805 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f9438018070 con 0x7f94481059d0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.619+0000 7f944de9e700 1 -- 172.21.15.88:0/3692222805 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9448109e60 con 0x7f94481059d0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.619+0000 7f9444ff9700 1 --2- 172.21.15.88:0/3692222805 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f9430038400 0x7f943003a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.619+0000 7f9444ff9700 1 -- 172.21.15.88:0/3692222805 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f943804bc60 con 0x7f94481059d0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.619+0000 7f9446ffd700 1 --2- 172.21.15.88:0/3692222805 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f9430038400 0x7f943003a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:45.375 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.619+0000 7f9446ffd700 1 --2- 172.21.15.88:0/3692222805 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f9430038400 0x7f943003a8b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f943c006fd0 tx=0x7f943c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.624+0000 7f9444ff9700 1 -- 172.21.15.88:0/3692222805 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f943802c350 con 0x7f94481059d0 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.812+0000 7f9444ff9700 1 -- 172.21.15.88:0/3692222805 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f943807e720 con 0x7f94481059d0 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.814+0000 7f944de9e700 1 -- 172.21.15.88:0/3692222805 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f944819d0e0 con 0x7f94481059d0 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.814+0000 7f9444ff9700 1 -- 172.21.15.88:0/3692222805 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f943801f070 con 0x7f94481059d0 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.816+0000 7f944de9e700 1 -- 172.21.15.88:0/3692222805 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f9430038400 msgr2=0x7f943003a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.816+0000 7f944de9e700 1 --2- 172.21.15.88:0/3692222805 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f9430038400 0x7f943003a8b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f943c006fd0 tx=0x7f943c006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.816+0000 7f944de9e700 1 -- 172.21.15.88:0/3692222805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 msgr2=0x7f9448199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.816+0000 7f944de9e700 1 --2- 172.21.15.88:0/3692222805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448199720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9438004840 tx=0x7f94380042e0 comp rx=0 tx=0).stop 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.816+0000 7f944de9e700 1 -- 172.21.15.88:0/3692222805 shutdown_connections 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.816+0000 7f944de9e700 1 --2- 172.21.15.88:0/3692222805 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f9430038400 0x7f943003a8b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.816+0000 7f944de9e700 1 --2- 172.21.15.88:0/3692222805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94481059d0 0x7f9448199720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.817+0000 7f944de9e700 1 -- 172.21.15.88:0/3692222805 >> 172.21.15.88:0/3692222805 conn(0x7f9448101020 msgr2=0x7f9448101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.817+0000 7f944de9e700 1 -- 172.21.15.88:0/3692222805 shutdown_connections 2024-02-19T18:51:45.376 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:44.817+0000 7f944de9e700 1 -- 172.21.15.88:0/3692222805 wait complete. 2024-02-19T18:51:46.069 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:45 smithi088 ceph-mon[29272]: mgrmap e13: smithi088.gyviqq(active, since 6s) 2024-02-19T18:51:46.646 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.746+0000 7f0ce502b700 1 Processor -- start 2024-02-19T18:51:46.646 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.746+0000 7f0ce502b700 1 -- start start 2024-02-19T18:51:46.646 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.746+0000 7f0ce502b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0108900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:46.646 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.746+0000 7f0ce502b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0079cf0 con 0x7f0ce0106520 2024-02-19T18:51:46.646 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.747+0000 7f0cded9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0108900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:46.646 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.747+0000 7f0cded9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0108900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49500/0 (socket says 172.21.15.88:49500) 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.747+0000 7f0cded9d700 1 -- 172.21.15.88:0/1432188522 learned_addr learned my addr 172.21.15.88:0/1432188522 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.747+0000 7f0cded9d700 1 -- 172.21.15.88:0/1432188522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce0079e30 con 0x7f0ce0106520 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.747+0000 7f0cded9d700 1 --2- 172.21.15.88:0/1432188522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0108900 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00098a0 tx=0x7f0cd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5325f59ef5aab8d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.747+0000 7f0cddd9b700 1 -- 172.21.15.88:0/1432188522 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cd0004180 con 0x7f0ce0106520 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.747+0000 7f0cddd9b700 1 -- 172.21.15.88:0/1432188522 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0cd00042e0 con 0x7f0ce0106520 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.747+0000 7f0cddd9b700 1 -- 172.21.15.88:0/1432188522 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cd0011af0 con 0x7f0ce0106520 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.748+0000 7f0ce502b700 1 -- 172.21.15.88:0/1432188522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 msgr2=0x7f0ce0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.748+0000 7f0ce502b700 1 --2- 172.21.15.88:0/1432188522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0108900 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0cd00098a0 tx=0x7f0cd0009bb0 comp rx=0 tx=0).stop 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.748+0000 7f0ce502b700 1 -- 172.21.15.88:0/1432188522 shutdown_connections 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.748+0000 7f0ce502b700 1 --2- 172.21.15.88:0/1432188522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0108900 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.748+0000 7f0ce502b700 1 -- 172.21.15.88:0/1432188522 >> 172.21.15.88:0/1432188522 conn(0x7f0ce00ffee0 msgr2=0x7f0ce0102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.748+0000 7f0ce502b700 1 -- 172.21.15.88:0/1432188522 shutdown_connections 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.748+0000 7f0ce502b700 1 -- 172.21.15.88:0/1432188522 wait complete. 2024-02-19T18:51:46.647 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.748+0000 7f0ce502b700 1 Processor -- start 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0ce502b700 1 -- start start 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0ce502b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0ce502b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0199cb0 con 0x7f0ce0106520 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0cded9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0cded9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49514/0 (socket says 172.21.15.88:49514) 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0cded9d700 1 -- 172.21.15.88:0/67353513 learned_addr learned my addr 172.21.15.88:0/67353513 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0cded9d700 1 -- 172.21.15.88:0/67353513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd0009550 con 0x7f0ce0106520 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0cded9d700 1 --2- 172.21.15.88:0/67353513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0199770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0011670 tx=0x7f0cd0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0cc7fff700 1 -- 172.21.15.88:0/67353513 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cd00039f0 con 0x7f0ce0106520 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0cc7fff700 1 -- 172.21.15.88:0/67353513 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0cd0003b50 con 0x7f0ce0106520 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0ce502b700 1 -- 172.21.15.88:0/67353513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce0199eb0 con 0x7f0ce0106520 2024-02-19T18:51:46.648 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.749+0000 7f0cc7fff700 1 -- 172.21.15.88:0/67353513 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cd00247c0 con 0x7f0ce0106520 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.750+0000 7f0ce502b700 1 -- 172.21.15.88:0/67353513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce019a350 con 0x7f0ce0106520 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.750+0000 7f0cc7fff700 1 -- 172.21.15.88:0/67353513 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f0cd002d3f0 con 0x7f0ce0106520 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.750+0000 7f0ce502b700 1 -- 172.21.15.88:0/67353513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce0109aa0 con 0x7f0ce0106520 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.751+0000 7f0cc7fff700 1 --2- 172.21.15.88:0/67353513 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0cc80384e0 0x7f0cc803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.751+0000 7f0cc7fff700 1 -- 172.21.15.88:0/67353513 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0cd004c5f0 con 0x7f0ce0106520 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.751+0000 7f0cde59c700 1 --2- 172.21.15.88:0/67353513 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0cc80384e0 0x7f0cc803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.751+0000 7f0cde59c700 1 --2- 172.21.15.88:0/67353513 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0cc80384e0 0x7f0cc803a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4006fd0 tx=0x7f0cd4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:45.755+0000 7f0cc7fff700 1 -- 172.21.15.88:0/67353513 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0cd0010bd0 con 0x7f0ce0106520 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.053+0000 7f0ce502b700 1 -- 172.21.15.88:0/67353513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f0ce0109cc0 con 0x7f0ce0106520 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.058+0000 7f0cc7fff700 1 -- 172.21.15.88:0/67353513 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f0cd00107f0 con 0x7f0ce0106520 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.060+0000 7f0ce502b700 1 -- 172.21.15.88:0/67353513 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0cc80384e0 msgr2=0x7f0cc803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:46.649 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.060+0000 7f0ce502b700 1 --2- 172.21.15.88:0/67353513 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0cc80384e0 0x7f0cc803a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0cd4006fd0 tx=0x7f0cd4006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.060+0000 7f0ce502b700 1 -- 172.21.15.88:0/67353513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 msgr2=0x7f0ce0199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.060+0000 7f0ce502b700 1 --2- 172.21.15.88:0/67353513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0199770 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0011670 tx=0x7f0cd0011b50 comp rx=0 tx=0).stop 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.060+0000 7f0ce502b700 1 -- 172.21.15.88:0/67353513 shutdown_connections 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.060+0000 7f0ce502b700 1 --2- 172.21.15.88:0/67353513 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0cc80384e0 0x7f0cc803a990 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.060+0000 7f0ce502b700 1 --2- 172.21.15.88:0/67353513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ce0106520 0x7f0ce0199770 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.061+0000 7f0ce502b700 1 -- 172.21.15.88:0/67353513 >> 172.21.15.88:0/67353513 conn(0x7f0ce00ffee0 msgr2=0x7f0ce0101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.061+0000 7f0ce502b700 1 -- 172.21.15.88:0/67353513 shutdown_connections 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr 2024-02-19T18:51:46.061+0000 7f0ce502b700 1 -- 172.21.15.88:0/67353513 wait complete. 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout:Or, if you are only running a single cluster on this host: 2024-02-19T18:51:46.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:51:46.651 INFO:teuthology.orchestra.run.smithi088.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-19T18:51:46.651 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:51:46.651 INFO:teuthology.orchestra.run.smithi088.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-19T18:51:46.651 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:51:46.651 INFO:teuthology.orchestra.run.smithi088.stdout: ceph telemetry on 2024-02-19T18:51:46.651 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:51:46.651 INFO:teuthology.orchestra.run.smithi088.stdout:For more information see: 2024-02-19T18:51:46.651 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:51:46.651 INFO:teuthology.orchestra.run.smithi088.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-19T18:51:46.651 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:51:46.651 INFO:teuthology.orchestra.run.smithi088.stdout:Bootstrap complete. 2024-02-19T18:51:46.679 INFO:tasks.cephadm:Fetching config... 2024-02-19T18:51:46.680 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:51:46.680 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-19T18:51:46.695 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-19T18:51:46.696 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:51:46.696 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-19T18:51:46.753 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-19T18:51:46.754 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:51:46.754 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/keyring of=/dev/stdout 2024-02-19T18:51:46.820 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-19T18:51:46.820 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:51:46.820 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-19T18:51:46.875 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-19T18:51:46.875 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHbHg37EvC00L1Olql/fBtAg6CTlxVZS8fA27ktTQzlB576J6Zrss1tPdn7/R5m3QL/E9fSuVDMFja8LWW7hUy4PVygvCuEeoXWalz+wQa7Xj6IMRnnL4L6pJGlKZJ9d2XoAxUkYrTcSdy8Z4N8Q5R6cYsx/lBBV/PL7mxRq/Wx7TeevSKvDkDEsaY6uSfDtviBDDzXW0bmgKdUnVYA58ddb5JtzFKsfcTSZmPIETWE5/fb/iFqVSz0qphk8g+6PE3zww+1fVoNkROabHRWwTFpC5oiYg25V/879sbszrqWiyd7y+p5hAhwc0VSSTT0Hr5lhjB9AShx+4IVLJImhWW+keXH+P6LiwgdCRsjcS6YwsUt/vi+JMRTP98l6dVMP57x/VAZsMbZSoyTYdquzhe8u/4g0/AYHuxcyymhYvfjA7DQe+3jn4JKfyeeYLFBLMU776a7pM1UbrzVzjzzVJwLWV7YfmlAimklQpxG2YqVJVtnbGQaTKS0VsVFU2M5tE= ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-19T18:51:46.954 INFO:teuthology.orchestra.run.smithi088.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHbHg37EvC00L1Olql/fBtAg6CTlxVZS8fA27ktTQzlB576J6Zrss1tPdn7/R5m3QL/E9fSuVDMFja8LWW7hUy4PVygvCuEeoXWalz+wQa7Xj6IMRnnL4L6pJGlKZJ9d2XoAxUkYrTcSdy8Z4N8Q5R6cYsx/lBBV/PL7mxRq/Wx7TeevSKvDkDEsaY6uSfDtviBDDzXW0bmgKdUnVYA58ddb5JtzFKsfcTSZmPIETWE5/fb/iFqVSz0qphk8g+6PE3zww+1fVoNkROabHRWwTFpC5oiYg25V/879sbszrqWiyd7y+p5hAhwc0VSSTT0Hr5lhjB9AShx+4IVLJImhWW+keXH+P6LiwgdCRsjcS6YwsUt/vi+JMRTP98l6dVMP57x/VAZsMbZSoyTYdquzhe8u/4g0/AYHuxcyymhYvfjA7DQe+3jn4JKfyeeYLFBLMU776a7pM1UbrzVzjzzVJwLWV7YfmlAimklQpxG2YqVJVtnbGQaTKS0VsVFU2M5tE= ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:51:46.968 DEBUG:teuthology.orchestra.run.smithi132:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHbHg37EvC00L1Olql/fBtAg6CTlxVZS8fA27ktTQzlB576J6Zrss1tPdn7/R5m3QL/E9fSuVDMFja8LWW7hUy4PVygvCuEeoXWalz+wQa7Xj6IMRnnL4L6pJGlKZJ9d2XoAxUkYrTcSdy8Z4N8Q5R6cYsx/lBBV/PL7mxRq/Wx7TeevSKvDkDEsaY6uSfDtviBDDzXW0bmgKdUnVYA58ddb5JtzFKsfcTSZmPIETWE5/fb/iFqVSz0qphk8g+6PE3zww+1fVoNkROabHRWwTFpC5oiYg25V/879sbszrqWiyd7y+p5hAhwc0VSSTT0Hr5lhjB9AShx+4IVLJImhWW+keXH+P6LiwgdCRsjcS6YwsUt/vi+JMRTP98l6dVMP57x/VAZsMbZSoyTYdquzhe8u/4g0/AYHuxcyymhYvfjA7DQe+3jn4JKfyeeYLFBLMU776a7pM1UbrzVzjzzVJwLWV7YfmlAimklQpxG2YqVJVtnbGQaTKS0VsVFU2M5tE= ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-19T18:51:47.016 INFO:teuthology.orchestra.run.smithi132.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDHbHg37EvC00L1Olql/fBtAg6CTlxVZS8fA27ktTQzlB576J6Zrss1tPdn7/R5m3QL/E9fSuVDMFja8LWW7hUy4PVygvCuEeoXWalz+wQa7Xj6IMRnnL4L6pJGlKZJ9d2XoAxUkYrTcSdy8Z4N8Q5R6cYsx/lBBV/PL7mxRq/Wx7TeevSKvDkDEsaY6uSfDtviBDDzXW0bmgKdUnVYA58ddb5JtzFKsfcTSZmPIETWE5/fb/iFqVSz0qphk8g+6PE3zww+1fVoNkROabHRWwTFpC5oiYg25V/879sbszrqWiyd7y+p5hAhwc0VSSTT0Hr5lhjB9AShx+4IVLJImhWW+keXH+P6LiwgdCRsjcS6YwsUt/vi+JMRTP98l6dVMP57x/VAZsMbZSoyTYdquzhe8u/4g0/AYHuxcyymhYvfjA7DQe+3jn4JKfyeeYLFBLMU776a7pM1UbrzVzjzzVJwLWV7YfmlAimklQpxG2YqVJVtnbGQaTKS0VsVFU2M5tE= ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:51:47.031 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-19T18:51:47.191 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:47 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/67353513' entity='client.admin' 2024-02-19T18:51:47.255 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:51:49.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.788+0000 7f8df2cd5700 1 -- 172.21.15.88:0/1171278850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dec0ffe70 msgr2=0x7f8dec100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:49.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.788+0000 7f8df2cd5700 1 --2- 172.21.15.88:0/1171278850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dec0ffe70 0x7f8dec100280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009910 tx=0x7f8de8009c20 comp rx=0 tx=0).stop 2024-02-19T18:51:49.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.788+0000 7f8df2cd5700 1 -- 172.21.15.88:0/1171278850 shutdown_connections 2024-02-19T18:51:49.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.788+0000 7f8df2cd5700 1 --2- 172.21.15.88:0/1171278850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dec0ffe70 0x7f8dec100280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:49.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.788+0000 7f8df2cd5700 1 -- 172.21.15.88:0/1171278850 >> 172.21.15.88:0/1171278850 conn(0x7f8dec0fb470 msgr2=0x7f8dec0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:49.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.788+0000 7f8df2cd5700 1 -- 172.21.15.88:0/1171278850 shutdown_connections 2024-02-19T18:51:49.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.788+0000 7f8df2cd5700 1 -- 172.21.15.88:0/1171278850 wait complete. 2024-02-19T18:51:49.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.789+0000 7f8df2cd5700 1 Processor -- start 2024-02-19T18:51:49.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.789+0000 7f8df2cd5700 1 -- start start 2024-02-19T18:51:49.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.789+0000 7f8df2cd5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dec0ffe70 0x7f8dec074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:49.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.789+0000 7f8df2cd5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dec075030 con 0x7f8dec0ffe70 2024-02-19T18:51:49.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.789+0000 7f8df0a71700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dec0ffe70 0x7f8dec074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:49.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.789+0000 7f8df0a71700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dec0ffe70 0x7f8dec074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49524/0 (socket says 172.21.15.88:49524) 2024-02-19T18:51:49.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.789+0000 7f8df0a71700 1 -- 172.21.15.88:0/3340412607 learned_addr learned my addr 172.21.15.88:0/3340412607 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:49.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.789+0000 7f8df0a71700 1 -- 172.21.15.88:0/3340412607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de80095f0 con 0x7f8dec0ffe70 2024-02-19T18:51:49.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.789+0000 7f8df0a71700 1 --2- 172.21.15.88:0/3340412607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dec0ffe70 0x7f8dec074af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8de8005950 tx=0x7f8de8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:49.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.790+0000 7f8de1ffb700 1 -- 172.21.15.88:0/3340412607 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8de801d070 con 0x7f8dec0ffe70 2024-02-19T18:51:49.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.790+0000 7f8de1ffb700 1 -- 172.21.15.88:0/3340412607 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8de800be10 con 0x7f8dec0ffe70 2024-02-19T18:51:49.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.790+0000 7f8df2cd5700 1 -- 172.21.15.88:0/3340412607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dec073140 con 0x7f8dec0ffe70 2024-02-19T18:51:49.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.790+0000 7f8de1ffb700 1 -- 172.21.15.88:0/3340412607 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8de800f460 con 0x7f8dec0ffe70 2024-02-19T18:51:49.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.790+0000 7f8df2cd5700 1 -- 172.21.15.88:0/3340412607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dec0735e0 con 0x7f8dec0ffe70 2024-02-19T18:51:49.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.790+0000 7f8de1ffb700 1 -- 172.21.15.88:0/3340412607 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f8de800f690 con 0x7f8dec0ffe70 2024-02-19T18:51:49.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.791+0000 7f8de1ffb700 1 --2- 172.21.15.88:0/3340412607 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8dd4038490 0x7f8dd403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:49.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.791+0000 7f8de1ffb700 1 -- 172.21.15.88:0/3340412607 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8de804c750 con 0x7f8dec0ffe70 2024-02-19T18:51:49.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.791+0000 7f8df2cd5700 1 -- 172.21.15.88:0/3340412607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dec062380 con 0x7f8dec0ffe70 2024-02-19T18:51:49.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.791+0000 7f8de3fff700 1 --2- 172.21.15.88:0/3340412607 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8dd4038490 0x7f8dd403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:49.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.791+0000 7f8de3fff700 1 --2- 172.21.15.88:0/3340412607 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8dd4038490 0x7f8dd403a940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc006fd0 tx=0x7f8ddc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:49.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.796+0000 7f8de1ffb700 1 -- 172.21.15.88:0/3340412607 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8de8027070 con 0x7f8dec0ffe70 2024-02-19T18:51:50.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.990+0000 7f8df2cd5700 1 -- 172.21.15.88:0/3340412607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f8dec18e820 con 0x7f8dec0ffe70 2024-02-19T18:51:50.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.993+0000 7f8de1ffb700 1 -- 172.21.15.88:0/3340412607 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f8de802c440 con 0x7f8dec0ffe70 2024-02-19T18:51:50.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.995+0000 7f8df2cd5700 1 -- 172.21.15.88:0/3340412607 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8dd4038490 msgr2=0x7f8dd403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:50.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.995+0000 7f8df2cd5700 1 --2- 172.21.15.88:0/3340412607 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8dd4038490 0x7f8dd403a940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc006fd0 tx=0x7f8ddc006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:50.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.996+0000 7f8df2cd5700 1 -- 172.21.15.88:0/3340412607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dec0ffe70 msgr2=0x7f8dec074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:50.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.996+0000 7f8df2cd5700 1 --2- 172.21.15.88:0/3340412607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dec0ffe70 0x7f8dec074af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8de8005950 tx=0x7f8de8009e60 comp rx=0 tx=0).stop 2024-02-19T18:51:50.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.996+0000 7f8df2cd5700 1 -- 172.21.15.88:0/3340412607 shutdown_connections 2024-02-19T18:51:50.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.996+0000 7f8df2cd5700 1 --2- 172.21.15.88:0/3340412607 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8dd4038490 0x7f8dd403a940 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:50.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.996+0000 7f8df2cd5700 1 --2- 172.21.15.88:0/3340412607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dec0ffe70 0x7f8dec074af0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:50.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.996+0000 7f8df2cd5700 1 -- 172.21.15.88:0/3340412607 >> 172.21.15.88:0/3340412607 conn(0x7f8dec0fb470 msgr2=0x7f8dec0fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:50.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.996+0000 7f8df2cd5700 1 -- 172.21.15.88:0/3340412607 shutdown_connections 2024-02-19T18:51:50.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:49.996+0000 7f8df2cd5700 1 -- 172.21.15.88:0/3340412607 wait complete. 2024-02-19T18:51:51.002 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-19T18:51:51.002 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-19T18:51:51.232 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:50 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/3340412607' entity='client.admin' 2024-02-19T18:51:51.233 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:50 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:51.233 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:50 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:51.233 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:50 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd/host:smithi088", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:51:51.233 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:50 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:51.233 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:50 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T18:51:51.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:50 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-19T18:51:51.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:50 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:51:51.674 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:51:52.299 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:51 smithi088 ceph-mon[29272]: Deploying daemon ceph-exporter.smithi088 on smithi088 2024-02-19T18:51:53.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.951+0000 7f262929a700 1 -- 172.21.15.88:0/4048994086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240fe0e0 msgr2=0x7f26240fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:53.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.951+0000 7f262929a700 1 --2- 172.21.15.88:0/4048994086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240fe0e0 0x7f26240fe4f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2614009910 tx=0x7f2614009c20 comp rx=0 tx=0).stop 2024-02-19T18:51:53.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.951+0000 7f262929a700 1 -- 172.21.15.88:0/4048994086 shutdown_connections 2024-02-19T18:51:53.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.951+0000 7f262929a700 1 --2- 172.21.15.88:0/4048994086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240fe0e0 0x7f26240fe4f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:53.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.951+0000 7f262929a700 1 -- 172.21.15.88:0/4048994086 >> 172.21.15.88:0/4048994086 conn(0x7f26240f9a50 msgr2=0x7f26240fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:53.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.951+0000 7f262929a700 1 -- 172.21.15.88:0/4048994086 shutdown_connections 2024-02-19T18:51:53.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.951+0000 7f262929a700 1 -- 172.21.15.88:0/4048994086 wait complete. 2024-02-19T18:51:53.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.952+0000 7f262929a700 1 Processor -- start 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.952+0000 7f262929a700 1 -- start start 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.952+0000 7f262929a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240fe0e0 0x7f2624100a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.952+0000 7f262929a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2624100f60 con 0x7f26240fe0e0 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.952+0000 7f2622ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240fe0e0 0x7f2624100a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.952+0000 7f2622ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240fe0e0 0x7f2624100a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49548/0 (socket says 172.21.15.88:49548) 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.952+0000 7f2622ffd700 1 -- 172.21.15.88:0/3974026319 learned_addr learned my addr 172.21.15.88:0/3974026319 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.953+0000 7f2622ffd700 1 -- 172.21.15.88:0/3974026319 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26140095f0 con 0x7f26240fe0e0 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.953+0000 7f2622ffd700 1 --2- 172.21.15.88:0/3974026319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240fe0e0 0x7f2624100a20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2614005b40 tx=0x7f2614004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.953+0000 7f2620ff9700 1 -- 172.21.15.88:0/3974026319 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f261401d070 con 0x7f26240fe0e0 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.953+0000 7f2620ff9700 1 -- 172.21.15.88:0/3974026319 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f261400be10 con 0x7f26240fe0e0 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.953+0000 7f262929a700 1 -- 172.21.15.88:0/3974026319 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26240ff070 con 0x7f26240fe0e0 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.953+0000 7f2620ff9700 1 -- 172.21.15.88:0/3974026319 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f261400f460 con 0x7f26240fe0e0 2024-02-19T18:51:53.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.953+0000 7f262929a700 1 -- 172.21.15.88:0/3974026319 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26240ff510 con 0x7f26240fe0e0 2024-02-19T18:51:53.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.954+0000 7f2620ff9700 1 -- 172.21.15.88:0/3974026319 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f261400f5c0 con 0x7f26240fe0e0 2024-02-19T18:51:53.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.954+0000 7f2620ff9700 1 --2- 172.21.15.88:0/3974026319 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f26100384e0 0x7f261003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:53.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.954+0000 7f2620ff9700 1 -- 172.21.15.88:0/3974026319 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f261404c390 con 0x7f26240fe0e0 2024-02-19T18:51:53.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.954+0000 7f261a7fc700 1 --2- 172.21.15.88:0/3974026319 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f26100384e0 0x7f261003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:53.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.954+0000 7f262929a700 1 -- 172.21.15.88:0/3974026319 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2624062380 con 0x7f26240fe0e0 2024-02-19T18:51:53.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.955+0000 7f261a7fc700 1 --2- 172.21.15.88:0/3974026319 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f26100384e0 0x7f261003a990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f260c006fd0 tx=0x7f260c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:53.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:53.959+0000 7f2620ff9700 1 -- 172.21.15.88:0/3974026319 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2614027070 con 0x7f26240fe0e0 2024-02-19T18:51:54.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.163+0000 7f262929a700 1 -- 172.21.15.88:0/3974026319 --> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f26240ff750 con 0x7f26100384e0 2024-02-19T18:51:54.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.166+0000 7f2620ff9700 1 -- 172.21.15.88:0/3974026319 <== mgr.14164 v2:172.21.15.88:6800/3043610921 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f26240ff750 con 0x7f26100384e0 2024-02-19T18:51:54.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.168+0000 7f262929a700 1 -- 172.21.15.88:0/3974026319 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f26100384e0 msgr2=0x7f261003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:54.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.168+0000 7f262929a700 1 --2- 172.21.15.88:0/3974026319 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f26100384e0 0x7f261003a990 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f260c006fd0 tx=0x7f260c006e40 comp rx=0 tx=0).stop 2024-02-19T18:51:54.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.168+0000 7f262929a700 1 -- 172.21.15.88:0/3974026319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240fe0e0 msgr2=0x7f2624100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:54.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.168+0000 7f262929a700 1 --2- 172.21.15.88:0/3974026319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240fe0e0 0x7f2624100a20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2614005b40 tx=0x7f2614004740 comp rx=0 tx=0).stop 2024-02-19T18:51:54.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.168+0000 7f262929a700 1 -- 172.21.15.88:0/3974026319 shutdown_connections 2024-02-19T18:51:54.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.168+0000 7f262929a700 1 --2- 172.21.15.88:0/3974026319 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f26100384e0 0x7f261003a990 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:54.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.168+0000 7f262929a700 1 --2- 172.21.15.88:0/3974026319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240fe0e0 0x7f2624100a20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:54.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.168+0000 7f262929a700 1 -- 172.21.15.88:0/3974026319 >> 172.21.15.88:0/3974026319 conn(0x7f26240f9a50 msgr2=0x7f26240fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:54.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.169+0000 7f262929a700 1 -- 172.21.15.88:0/3974026319 shutdown_connections 2024-02-19T18:51:54.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:54.169+0000 7f262929a700 1 -- 172.21.15.88:0/3974026319 wait complete. 2024-02-19T18:51:54.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:54 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:54.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:54 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:54.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:54 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:54.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:54 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:54.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:54 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T18:51:54.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:54 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-19T18:51:54.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:54 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:51:54.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:54 smithi088 ceph-mon[29272]: Deploying daemon crash.smithi088 on smithi088 2024-02-19T18:51:54.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:54 smithi088 ceph-mon[29272]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:54.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:54 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:54.879 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi132 2024-02-19T18:51:54.879 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:51:54.880 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/etc/ceph/ceph.conf 2024-02-19T18:51:54.899 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:51:54.899 DEBUG:teuthology.orchestra.run.smithi132:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-19T18:51:54.956 INFO:tasks.cephadm:Adding host smithi132 to orchestrator... 2024-02-19T18:51:54.956 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph orch host add smithi132 2024-02-19T18:51:55.392 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:51:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.405+0000 7f233a553700 1 -- 172.21.15.88:0/3736282402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23340fed20 msgr2=0x7f23340ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:51:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.405+0000 7f233a553700 1 --2- 172.21.15.88:0/3736282402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23340fed20 0x7f23340ff130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2328009910 tx=0x7f2328009c20 comp rx=0 tx=0).stop 2024-02-19T18:51:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.405+0000 7f233a553700 1 -- 172.21.15.88:0/3736282402 shutdown_connections 2024-02-19T18:51:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.405+0000 7f233a553700 1 --2- 172.21.15.88:0/3736282402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23340fed20 0x7f23340ff130 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:51:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.405+0000 7f233a553700 1 -- 172.21.15.88:0/3736282402 >> 172.21.15.88:0/3736282402 conn(0x7f23340fa2b0 msgr2=0x7f23340fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:51:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.405+0000 7f233a553700 1 -- 172.21.15.88:0/3736282402 shutdown_connections 2024-02-19T18:51:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.405+0000 7f233a553700 1 -- 172.21.15.88:0/3736282402 wait complete. 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.406+0000 7f233a553700 1 Processor -- start 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.406+0000 7f233a553700 1 -- start start 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.406+0000 7f233a553700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23340fed20 0x7f2334068ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.406+0000 7f233a553700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2334197d30 con 0x7f23340fed20 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.406+0000 7f2333fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23340fed20 0x7f2334068ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.406+0000 7f2333fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23340fed20 0x7f2334068ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41988/0 (socket says 172.21.15.88:41988) 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.406+0000 7f2333fff700 1 -- 172.21.15.88:0/3285423481 learned_addr learned my addr 172.21.15.88:0/3285423481 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.406+0000 7f2333fff700 1 -- 172.21.15.88:0/3285423481 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23280095f0 con 0x7f23340fed20 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.407+0000 7f2333fff700 1 --2- 172.21.15.88:0/3285423481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23340fed20 0x7f2334068ab0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2328005950 tx=0x7f2328009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.407+0000 7f23317fa700 1 -- 172.21.15.88:0/3285423481 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f232801d070 con 0x7f23340fed20 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.407+0000 7f233a553700 1 -- 172.21.15.88:0/3285423481 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2334197e70 con 0x7f23340fed20 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.407+0000 7f23317fa700 1 -- 172.21.15.88:0/3285423481 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f232800be10 con 0x7f23340fed20 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.407+0000 7f23317fa700 1 -- 172.21.15.88:0/3285423481 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f232800f460 con 0x7f23340fed20 2024-02-19T18:51:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.407+0000 7f233a553700 1 -- 172.21.15.88:0/3285423481 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23341982b0 con 0x7f23340fed20 2024-02-19T18:51:57.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.408+0000 7f23317fa700 1 -- 172.21.15.88:0/3285423481 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f232800f690 con 0x7f23340fed20 2024-02-19T18:51:57.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.408+0000 7f23317fa700 1 --2- 172.21.15.88:0/3285423481 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f231c038490 0x7f231c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:51:57.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.408+0000 7f23317fa700 1 -- 172.21.15.88:0/3285423481 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f232804c3d0 con 0x7f23340fed20 2024-02-19T18:51:57.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.408+0000 7f233a553700 1 -- 172.21.15.88:0/3285423481 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233418da00 con 0x7f23340fed20 2024-02-19T18:51:57.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.408+0000 7f23337fe700 1 --2- 172.21.15.88:0/3285423481 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f231c038490 0x7f231c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:51:57.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.408+0000 7f23337fe700 1 --2- 172.21.15.88:0/3285423481 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f231c038490 0x7f231c03a940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2324006fd0 tx=0x7f2324006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:51:57.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.413+0000 7f23317fa700 1 -- 172.21.15.88:0/3285423481 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2328027020 con 0x7f23340fed20 2024-02-19T18:51:57.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:51:57.614+0000 7f233a553700 1 -- 172.21.15.88:0/3285423481 --> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi132", "target": ["mon-mgr", ""]}) v1 -- 0x7f2334061190 con 0x7f231c038490 2024-02-19T18:51:59.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:58 smithi088 ceph-mon[29272]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi132", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:51:59.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:58 smithi088 ceph-mon[29272]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:51:59.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:58 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:59.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:58 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:59.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:58 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:59.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:58 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:59.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:58 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:51:59.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:58 smithi088 ceph-mon[29272]: Deploying daemon node-exporter.smithi088 on smithi088 2024-02-19T18:52:00.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:51:59 smithi088 ceph-mon[29272]: Deploying cephadm binary to smithi132 2024-02-19T18:52:00.237 INFO:teuthology.orchestra.run.smithi088.stdout:Added host 'smithi132' with addr '172.21.15.132' 2024-02-19T18:52:00.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:00.220+0000 7f23317fa700 1 -- 172.21.15.88:0/3285423481 <== mgr.14164 v2:172.21.15.88:6800/3043610921 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f2334061190 con 0x7f231c038490 2024-02-19T18:52:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:00.223+0000 7f233a553700 1 -- 172.21.15.88:0/3285423481 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f231c038490 msgr2=0x7f231c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:00.223+0000 7f233a553700 1 --2- 172.21.15.88:0/3285423481 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f231c038490 0x7f231c03a940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2324006fd0 tx=0x7f2324006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:00.223+0000 7f233a553700 1 -- 172.21.15.88:0/3285423481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23340fed20 msgr2=0x7f2334068ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:00.223+0000 7f233a553700 1 --2- 172.21.15.88:0/3285423481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23340fed20 0x7f2334068ab0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2328005950 tx=0x7f2328009e60 comp rx=0 tx=0).stop 2024-02-19T18:52:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:00.223+0000 7f233a553700 1 -- 172.21.15.88:0/3285423481 shutdown_connections 2024-02-19T18:52:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:00.223+0000 7f233a553700 1 --2- 172.21.15.88:0/3285423481 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f231c038490 0x7f231c03a940 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:00.223+0000 7f233a553700 1 --2- 172.21.15.88:0/3285423481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23340fed20 0x7f2334068ab0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:00.223+0000 7f233a553700 1 -- 172.21.15.88:0/3285423481 >> 172.21.15.88:0/3285423481 conn(0x7f23340fa2b0 msgr2=0x7f23340faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:00.223+0000 7f233a553700 1 -- 172.21.15.88:0/3285423481 shutdown_connections 2024-02-19T18:52:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:00.223+0000 7f233a553700 1 -- 172.21.15.88:0/3285423481 wait complete. 2024-02-19T18:52:01.016 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph orch host ls --format=json 2024-02-19T18:52:01.221 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:52:01.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:01 smithi088 ceph-mon[29272]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:01.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:01 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:01.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:01 smithi088 ceph-mon[29272]: Added host smithi132 2024-02-19T18:52:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.071+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2612621322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1cc0ffe20 msgr2=0x7ff1cc100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.071+0000 7ff1d0e13700 1 --2- 172.21.15.88:0/2612621322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1cc0ffe20 0x7ff1cc100230 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009910 tx=0x7ff1bc009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.071+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2612621322 shutdown_connections 2024-02-19T18:52:03.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.071+0000 7ff1d0e13700 1 --2- 172.21.15.88:0/2612621322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1cc0ffe20 0x7ff1cc100230 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.071+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2612621322 >> 172.21.15.88:0/2612621322 conn(0x7ff1cc0fb3d0 msgr2=0x7ff1cc0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.071+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2612621322 shutdown_connections 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.071+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2612621322 wait complete. 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.072+0000 7ff1d0e13700 1 Processor -- start 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.072+0000 7ff1d0e13700 1 -- start start 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.072+0000 7ff1d0e13700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1cc0ffe20 0x7ff1cc194f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.072+0000 7ff1d0e13700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1cc1954a0 con 0x7ff1cc0ffe20 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.072+0000 7ff1ca59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1cc0ffe20 0x7ff1cc194f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.072+0000 7ff1ca59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1cc0ffe20 0x7ff1cc194f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42018/0 (socket says 172.21.15.88:42018) 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.072+0000 7ff1ca59c700 1 -- 172.21.15.88:0/2587608578 learned_addr learned my addr 172.21.15.88:0/2587608578 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.073+0000 7ff1ca59c700 1 -- 172.21.15.88:0/2587608578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc0095f0 con 0x7ff1cc0ffe20 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.073+0000 7ff1ca59c700 1 --2- 172.21.15.88:0/2587608578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1cc0ffe20 0x7ff1cc194f60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0092e0 tx=0x7ff1bc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:03.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.073+0000 7ff1bb7fe700 1 -- 172.21.15.88:0/2587608578 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1bc01d070 con 0x7ff1cc0ffe20 2024-02-19T18:52:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.073+0000 7ff1bb7fe700 1 -- 172.21.15.88:0/2587608578 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff1bc00be10 con 0x7ff1cc0ffe20 2024-02-19T18:52:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.073+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2587608578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1cc1956a0 con 0x7ff1cc0ffe20 2024-02-19T18:52:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.073+0000 7ff1bb7fe700 1 -- 172.21.15.88:0/2587608578 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1bc00f460 con 0x7ff1cc0ffe20 2024-02-19T18:52:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.073+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2587608578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1cc195b40 con 0x7ff1cc0ffe20 2024-02-19T18:52:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.074+0000 7ff1bb7fe700 1 -- 172.21.15.88:0/2587608578 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7ff1bc00f5c0 con 0x7ff1cc0ffe20 2024-02-19T18:52:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.074+0000 7ff1bb7fe700 1 --2- 172.21.15.88:0/2587608578 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7ff1b40384e0 0x7ff1b403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.074+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2587608578 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1cc18ea80 con 0x7ff1cc0ffe20 2024-02-19T18:52:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.074+0000 7ff1bb7fe700 1 -- 172.21.15.88:0/2587608578 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff1bc04c290 con 0x7ff1cc0ffe20 2024-02-19T18:52:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.074+0000 7ff1c9d9b700 1 --2- 172.21.15.88:0/2587608578 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7ff1b40384e0 0x7ff1b403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:03.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.074+0000 7ff1c9d9b700 1 --2- 172.21.15.88:0/2587608578 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7ff1b40384e0 0x7ff1b403a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0006fd0 tx=0x7ff1c0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:03.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.079+0000 7ff1bb7fe700 1 -- 172.21.15.88:0/2587608578 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff1bc02c440 con 0x7ff1cc0ffe20 2024-02-19T18:52:03.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.285+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2587608578 --> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff1cc1008b0 con 0x7ff1b40384e0 2024-02-19T18:52:03.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.286+0000 7ff1bb7fe700 1 -- 172.21.15.88:0/2587608578 <== mgr.14164 v2:172.21.15.88:6800/3043610921 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7ff1cc1008b0 con 0x7ff1b40384e0 2024-02-19T18:52:03.302 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:52:03.302 INFO:teuthology.orchestra.run.smithi088.stdout:[{"addr": "172.21.15.88", "hostname": "smithi088", "labels": [], "status": ""}, {"addr": "172.21.15.132", "hostname": "smithi132", "labels": [], "status": ""}] 2024-02-19T18:52:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.288+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2587608578 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7ff1b40384e0 msgr2=0x7ff1b403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.288+0000 7ff1d0e13700 1 --2- 172.21.15.88:0/2587608578 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7ff1b40384e0 0x7ff1b403a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0006fd0 tx=0x7ff1c0006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.288+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2587608578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1cc0ffe20 msgr2=0x7ff1cc194f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.288+0000 7ff1d0e13700 1 --2- 172.21.15.88:0/2587608578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1cc0ffe20 0x7ff1cc194f60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0092e0 tx=0x7ff1bc009e60 comp rx=0 tx=0).stop 2024-02-19T18:52:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.288+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2587608578 shutdown_connections 2024-02-19T18:52:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.288+0000 7ff1d0e13700 1 --2- 172.21.15.88:0/2587608578 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7ff1b40384e0 0x7ff1b403a990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.288+0000 7ff1d0e13700 1 --2- 172.21.15.88:0/2587608578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1cc0ffe20 0x7ff1cc194f60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.288+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2587608578 >> 172.21.15.88:0/2587608578 conn(0x7ff1cc0fb3d0 msgr2=0x7ff1cc0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.288+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2587608578 shutdown_connections 2024-02-19T18:52:03.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:03.288+0000 7ff1d0e13700 1 -- 172.21.15.88:0/2587608578 wait complete. 2024-02-19T18:52:03.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:03 smithi088 ceph-mon[29272]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:03.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:03 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:03.925 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-19T18:52:03.925 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd crush tunables default 2024-02-19T18:52:04.127 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:52:04.497 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:04 smithi088 ceph-mon[29272]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T18:52:05.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:05 smithi088 ceph-mon[29272]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:06.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.561+0000 7fd1b7984700 1 -- 172.21.15.88:0/1505071765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1b00fed30 msgr2=0x7fd1b00ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.561+0000 7fd1b7984700 1 --2- 172.21.15.88:0/1505071765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1b00fed30 0x7fd1b00ff140 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009910 tx=0x7fd1a8009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.561+0000 7fd1b7984700 1 -- 172.21.15.88:0/1505071765 shutdown_connections 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.561+0000 7fd1b7984700 1 --2- 172.21.15.88:0/1505071765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1b00fed30 0x7fd1b00ff140 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.561+0000 7fd1b7984700 1 -- 172.21.15.88:0/1505071765 >> 172.21.15.88:0/1505071765 conn(0x7fd1b00fa2c0 msgr2=0x7fd1b00fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.561+0000 7fd1b7984700 1 -- 172.21.15.88:0/1505071765 shutdown_connections 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.561+0000 7fd1b7984700 1 -- 172.21.15.88:0/1505071765 wait complete. 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.561+0000 7fd1b7984700 1 Processor -- start 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1b7984700 1 -- start start 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1b7984700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1b00fed30 0x7fd1b0193ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1b7984700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b0194410 con 0x7fd1b00fed30 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1b5720700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1b00fed30 0x7fd1b0193ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1b5720700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1b00fed30 0x7fd1b0193ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44414/0 (socket says 172.21.15.88:44414) 2024-02-19T18:52:06.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1b5720700 1 -- 172.21.15.88:0/2927920253 learned_addr learned my addr 172.21.15.88:0/2927920253 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:52:06.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1b5720700 1 -- 172.21.15.88:0/2927920253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a80095f0 con 0x7fd1b00fed30 2024-02-19T18:52:06.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1b5720700 1 --2- 172.21.15.88:0/2927920253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1b00fed30 0x7fd1b0193ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005b40 tx=0x7fd1a8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:06.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1a67fc700 1 -- 172.21.15.88:0/2927920253 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1a801d070 con 0x7fd1b00fed30 2024-02-19T18:52:06.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1a67fc700 1 -- 172.21.15.88:0/2927920253 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd1a800be10 con 0x7fd1b00fed30 2024-02-19T18:52:06.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1b7984700 1 -- 172.21.15.88:0/2927920253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b0194610 con 0x7fd1b00fed30 2024-02-19T18:52:06.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1a67fc700 1 -- 172.21.15.88:0/2927920253 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd1a800f460 con 0x7fd1b00fed30 2024-02-19T18:52:06.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.562+0000 7fd1b7984700 1 -- 172.21.15.88:0/2927920253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b0194ab0 con 0x7fd1b00fed30 2024-02-19T18:52:06.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.563+0000 7fd1a67fc700 1 -- 172.21.15.88:0/2927920253 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fd1a800b960 con 0x7fd1b00fed30 2024-02-19T18:52:06.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.563+0000 7fd1b7984700 1 -- 172.21.15.88:0/2927920253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b0104b60 con 0x7fd1b00fed30 2024-02-19T18:52:06.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.564+0000 7fd1a67fc700 1 --2- 172.21.15.88:0/2927920253 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd19c0384e0 0x7fd19c03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:06.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.564+0000 7fd1a67fc700 1 -- 172.21.15.88:0/2927920253 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd1a804c370 con 0x7fd1b00fed30 2024-02-19T18:52:06.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.564+0000 7fd1b4f1f700 1 --2- 172.21.15.88:0/2927920253 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd19c0384e0 0x7fd19c03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:06.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.564+0000 7fd1b4f1f700 1 --2- 172.21.15.88:0/2927920253 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd19c0384e0 0x7fd19c03a990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0006fd0 tx=0x7fd1a0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:06.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.570+0000 7fd1a67fc700 1 -- 172.21.15.88:0/2927920253 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd1a802bb60 con 0x7fd1b00fed30 2024-02-19T18:52:06.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:06.781+0000 7fd1b7984700 1 -- 172.21.15.88:0/2927920253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fd1b0104d50 con 0x7fd1b00fed30 2024-02-19T18:52:07.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:07.230+0000 7fd1a67fc700 1 -- 172.21.15.88:0/2927920253 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd1a8027070 con 0x7fd1b00fed30 2024-02-19T18:52:07.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:07.232+0000 7fd1b7984700 1 -- 172.21.15.88:0/2927920253 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd19c0384e0 msgr2=0x7fd19c03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:07.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:07.232+0000 7fd1b7984700 1 --2- 172.21.15.88:0/2927920253 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd19c0384e0 0x7fd19c03a990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0006fd0 tx=0x7fd1a0006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:07.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:07.232+0000 7fd1b7984700 1 -- 172.21.15.88:0/2927920253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1b00fed30 msgr2=0x7fd1b0193ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:07.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:07.232+0000 7fd1b7984700 1 --2- 172.21.15.88:0/2927920253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1b00fed30 0x7fd1b0193ed0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8005b40 tx=0x7fd1a8004740 comp rx=0 tx=0).stop 2024-02-19T18:52:07.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:07.232+0000 7fd1b7984700 1 -- 172.21.15.88:0/2927920253 shutdown_connections 2024-02-19T18:52:07.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:07.232+0000 7fd1b7984700 1 --2- 172.21.15.88:0/2927920253 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd19c0384e0 0x7fd19c03a990 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:07.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:07.232+0000 7fd1b7984700 1 --2- 172.21.15.88:0/2927920253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1b00fed30 0x7fd1b0193ed0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:07.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:07.232+0000 7fd1b7984700 1 -- 172.21.15.88:0/2927920253 >> 172.21.15.88:0/2927920253 conn(0x7fd1b00fa2c0 msgr2=0x7fd1b00faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:07.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:07.232+0000 7fd1b7984700 1 -- 172.21.15.88:0/2927920253 shutdown_connections 2024-02-19T18:52:07.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:52:07.232+0000 7fd1b7984700 1 -- 172.21.15.88:0/2927920253 wait complete. 2024-02-19T18:52:07.249 INFO:teuthology.orchestra.run.smithi088.stderr:adjusted tunables profile to default 2024-02-19T18:52:07.500 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:07 smithi088 ceph-mon[29272]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:07.500 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:07 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:07.500 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:07 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:07.500 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:07 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:07.501 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:07 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:07.501 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:07 smithi088 ceph-mon[29272]: Deploying daemon alertmanager.smithi088 on smithi088 2024-02-19T18:52:07.501 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:07 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2927920253' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-19T18:52:07.788 INFO:tasks.cephadm:Adding mon.smithi088 on smithi088 2024-02-19T18:52:07.788 INFO:tasks.cephadm:Adding mon.smithi132 on smithi132 2024-02-19T18:52:07.788 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph orch apply mon '2;smithi088:172.21.15.88=smithi088;smithi132:172.21.15.132=smithi132' 2024-02-19T18:52:07.989 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:08.037 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:08.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:08 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2927920253' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-19T18:52:08.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:08 smithi088 ceph-mon[29272]: osdmap e4: 0 total, 0 up, 0 in 2024-02-19T18:52:08.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:08 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:09.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:09 smithi088 ceph-mon[29272]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:09.718 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.701+0000 7f777abdb700 1 -- 172.21.15.132:0/2098994175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77740ffe20 msgr2=0x7f7774100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:09.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.701+0000 7f777abdb700 1 --2- 172.21.15.132:0/2098994175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77740ffe20 0x7f7774100230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7768009910 tx=0x7f7768009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:09.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.701+0000 7f777abdb700 1 -- 172.21.15.132:0/2098994175 shutdown_connections 2024-02-19T18:52:09.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.701+0000 7f777abdb700 1 --2- 172.21.15.132:0/2098994175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77740ffe20 0x7f7774100230 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:09.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.701+0000 7f777abdb700 1 -- 172.21.15.132:0/2098994175 >> 172.21.15.132:0/2098994175 conn(0x7f77740fb3d0 msgr2=0x7f77740fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:09.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.701+0000 7f777abdb700 1 -- 172.21.15.132:0/2098994175 shutdown_connections 2024-02-19T18:52:09.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.701+0000 7f777abdb700 1 -- 172.21.15.132:0/2098994175 wait complete. 2024-02-19T18:52:09.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.702+0000 7f777abdb700 1 Processor -- start 2024-02-19T18:52:09.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.702+0000 7f777abdb700 1 -- start start 2024-02-19T18:52:09.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.702+0000 7f777abdb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77740ffe20 0x7f7774194f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:09.719 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.702+0000 7f777abdb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77741954a0 con 0x7f77740ffe20 2024-02-19T18:52:09.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.702+0000 7f7778977700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77740ffe20 0x7f7774194f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:09.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.702+0000 7f7778977700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77740ffe20 0x7f7774194f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:43312/0 (socket says 172.21.15.132:43312) 2024-02-19T18:52:09.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.702+0000 7f7778977700 1 -- 172.21.15.132:0/2808690538 learned_addr learned my addr 172.21.15.132:0/2808690538 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:09.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.702+0000 7f7778977700 1 -- 172.21.15.132:0/2808690538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77680095f0 con 0x7f77740ffe20 2024-02-19T18:52:09.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.702+0000 7f7778977700 1 --2- 172.21.15.132:0/2808690538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77740ffe20 0x7f7774194f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7768005b40 tx=0x7f7768004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:09.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.703+0000 7f7771ffb700 1 -- 172.21.15.132:0/2808690538 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f776801d070 con 0x7f77740ffe20 2024-02-19T18:52:09.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.703+0000 7f777abdb700 1 -- 172.21.15.132:0/2808690538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77741956a0 con 0x7f77740ffe20 2024-02-19T18:52:09.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.703+0000 7f7771ffb700 1 -- 172.21.15.132:0/2808690538 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f776800be10 con 0x7f77740ffe20 2024-02-19T18:52:09.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.703+0000 7f7771ffb700 1 -- 172.21.15.132:0/2808690538 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f776800f460 con 0x7f77740ffe20 2024-02-19T18:52:09.720 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.703+0000 7f777abdb700 1 -- 172.21.15.132:0/2808690538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7774195b40 con 0x7f77740ffe20 2024-02-19T18:52:09.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.704+0000 7f777abdb700 1 -- 172.21.15.132:0/2808690538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f777418ea80 con 0x7f77740ffe20 2024-02-19T18:52:09.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.704+0000 7f7771ffb700 1 -- 172.21.15.132:0/2808690538 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f77680227e0 con 0x7f77740ffe20 2024-02-19T18:52:09.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.704+0000 7f7771ffb700 1 --2- 172.21.15.132:0/2808690538 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f7764038490 0x7f776403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:09.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.704+0000 7f7771ffb700 1 -- 172.21.15.132:0/2808690538 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f776804c330 con 0x7f77740ffe20 2024-02-19T18:52:09.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.704+0000 7f7773fff700 1 --2- 172.21.15.132:0/2808690538 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f7764038490 0x7f776403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:09.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.705+0000 7f7773fff700 1 --2- 172.21.15.132:0/2808690538 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f7764038490 0x7f776403a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7760006fd0 tx=0x7f7760006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:09.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.709+0000 7f7771ffb700 1 -- 172.21.15.132:0/2808690538 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f776802b590 con 0x7f77740ffe20 2024-02-19T18:52:09.922 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.906+0000 7f777abdb700 1 -- 172.21.15.132:0/2808690538 --> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi088:172.21.15.88=smithi088;smithi132:172.21.15.132=smithi132", "target": ["mon-mgr", ""]}) v1 -- 0x7f7774061190 con 0x7f7764038490 2024-02-19T18:52:09.928 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.913+0000 7f7771ffb700 1 -- 172.21.15.132:0/2808690538 <== mgr.14164 v2:172.21.15.88:6800/3043610921 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7774061190 con 0x7f7764038490 2024-02-19T18:52:09.929 INFO:teuthology.orchestra.run.smithi132.stdout:Scheduled mon update... 2024-02-19T18:52:09.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.916+0000 7f777abdb700 1 -- 172.21.15.132:0/2808690538 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f7764038490 msgr2=0x7f776403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:09.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.916+0000 7f777abdb700 1 --2- 172.21.15.132:0/2808690538 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f7764038490 0x7f776403a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7760006fd0 tx=0x7f7760006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:09.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.916+0000 7f777abdb700 1 -- 172.21.15.132:0/2808690538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77740ffe20 msgr2=0x7f7774194f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:09.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.916+0000 7f777abdb700 1 --2- 172.21.15.132:0/2808690538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77740ffe20 0x7f7774194f60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7768005b40 tx=0x7f7768004740 comp rx=0 tx=0).stop 2024-02-19T18:52:09.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.916+0000 7f777abdb700 1 -- 172.21.15.132:0/2808690538 shutdown_connections 2024-02-19T18:52:09.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.916+0000 7f777abdb700 1 --2- 172.21.15.132:0/2808690538 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f7764038490 0x7f776403a940 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:09.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.916+0000 7f777abdb700 1 --2- 172.21.15.132:0/2808690538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77740ffe20 0x7f7774194f60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:09.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.916+0000 7f777abdb700 1 -- 172.21.15.132:0/2808690538 >> 172.21.15.132:0/2808690538 conn(0x7f77740fb3d0 msgr2=0x7f77740fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:09.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.916+0000 7f777abdb700 1 -- 172.21.15.132:0/2808690538 shutdown_connections 2024-02-19T18:52:09.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:09.916+0000 7f777abdb700 1 -- 172.21.15.132:0/2808690538 wait complete. 2024-02-19T18:52:10.523 DEBUG:teuthology.orchestra.run.smithi132:mon.smithi132> sudo journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi132.service 2024-02-19T18:52:10.526 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:10.526 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:10.753 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:10.798 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:11.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:10 smithi088 ceph-mon[29272]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:11.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:10 smithi088 ceph-mon[29272]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi088:172.21.15.88=smithi088;smithi132:172.21.15.132=smithi132", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:52:11.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:10 smithi088 ceph-mon[29272]: Saving service mon spec with placement smithi088:172.21.15.88=smithi088;smithi132:172.21.15.132=smithi132;count:2 2024-02-19T18:52:11.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:10 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:12.412 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.396+0000 7f438645f700 1 -- 172.21.15.132:0/3617760369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43800ffe20 msgr2=0x7f4380100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.396+0000 7f438645f700 1 --2- 172.21.15.132:0/3617760369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43800ffe20 0x7f4380100230 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4370009910 tx=0x7f4370009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.396+0000 7f438645f700 1 -- 172.21.15.132:0/3617760369 shutdown_connections 2024-02-19T18:52:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.396+0000 7f438645f700 1 --2- 172.21.15.132:0/3617760369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43800ffe20 0x7f4380100230 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.396+0000 7f438645f700 1 -- 172.21.15.132:0/3617760369 >> 172.21.15.132:0/3617760369 conn(0x7f43800fb3d0 msgr2=0x7f43800fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.396+0000 7f438645f700 1 -- 172.21.15.132:0/3617760369 shutdown_connections 2024-02-19T18:52:12.413 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.396+0000 7f438645f700 1 -- 172.21.15.132:0/3617760369 wait complete. 2024-02-19T18:52:12.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.397+0000 7f438645f700 1 Processor -- start 2024-02-19T18:52:12.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.397+0000 7f438645f700 1 -- start start 2024-02-19T18:52:12.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.397+0000 7f438645f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43800ffe20 0x7f4380194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:12.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.397+0000 7f438645f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43801954b0 con 0x7f43800ffe20 2024-02-19T18:52:12.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.397+0000 7f437ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43800ffe20 0x7f4380194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:12.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.397+0000 7f437ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43800ffe20 0x7f4380194f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:43324/0 (socket says 172.21.15.132:43324) 2024-02-19T18:52:12.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.397+0000 7f437ffff700 1 -- 172.21.15.132:0/1746029229 learned_addr learned my addr 172.21.15.132:0/1746029229 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:12.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.398+0000 7f437ffff700 1 -- 172.21.15.132:0/1746029229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43700095f0 con 0x7f43800ffe20 2024-02-19T18:52:12.414 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.398+0000 7f437ffff700 1 --2- 172.21.15.132:0/1746029229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43800ffe20 0x7f4380194f70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4370005b40 tx=0x7f4370004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:12.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.398+0000 7f437d7fa700 1 -- 172.21.15.132:0/1746029229 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f437001d070 con 0x7f43800ffe20 2024-02-19T18:52:12.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.398+0000 7f437d7fa700 1 -- 172.21.15.132:0/1746029229 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f437000be10 con 0x7f43800ffe20 2024-02-19T18:52:12.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.398+0000 7f438645f700 1 -- 172.21.15.132:0/1746029229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43801956b0 con 0x7f43800ffe20 2024-02-19T18:52:12.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.398+0000 7f437d7fa700 1 -- 172.21.15.132:0/1746029229 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f437000f460 con 0x7f43800ffe20 2024-02-19T18:52:12.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.398+0000 7f438645f700 1 -- 172.21.15.132:0/1746029229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4380195b50 con 0x7f43800ffe20 2024-02-19T18:52:12.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.399+0000 7f437d7fa700 1 -- 172.21.15.132:0/1746029229 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f437000b960 con 0x7f43800ffe20 2024-02-19T18:52:12.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.399+0000 7f438645f700 1 -- 172.21.15.132:0/1746029229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f438018ea80 con 0x7f43800ffe20 2024-02-19T18:52:12.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.399+0000 7f437d7fa700 1 --2- 172.21.15.132:0/1746029229 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4368038490 0x7f436803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:12.416 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.399+0000 7f437d7fa700 1 -- 172.21.15.132:0/1746029229 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f437004c380 con 0x7f43800ffe20 2024-02-19T18:52:12.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.399+0000 7f437f7fe700 1 --2- 172.21.15.132:0/1746029229 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4368038490 0x7f436803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:12.417 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.400+0000 7f437f7fe700 1 --2- 172.21.15.132:0/1746029229 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4368038490 0x7f436803a940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4374006fd0 tx=0x7f4374006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:12.421 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.404+0000 7f437d7fa700 1 -- 172.21.15.132:0/1746029229 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f437002bba0 con 0x7f43800ffe20 2024-02-19T18:52:12.685 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.668+0000 7f438645f700 1 -- 172.21.15.132:0/1746029229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f438002cc30 con 0x7f43800ffe20 2024-02-19T18:52:12.686 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.669+0000 7f437d7fa700 1 -- 172.21.15.132:0/1746029229 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f4370027070 con 0x7f43800ffe20 2024-02-19T18:52:12.686 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:12.686 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:12.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.671+0000 7f438645f700 1 -- 172.21.15.132:0/1746029229 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4368038490 msgr2=0x7f436803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:12.688 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.671+0000 7f438645f700 1 --2- 172.21.15.132:0/1746029229 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4368038490 0x7f436803a940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4374006fd0 tx=0x7f4374006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:12.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.671+0000 7f438645f700 1 -- 172.21.15.132:0/1746029229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43800ffe20 msgr2=0x7f4380194f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:12.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.671+0000 7f438645f700 1 --2- 172.21.15.132:0/1746029229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43800ffe20 0x7f4380194f70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4370005b40 tx=0x7f4370004740 comp rx=0 tx=0).stop 2024-02-19T18:52:12.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.671+0000 7f438645f700 1 -- 172.21.15.132:0/1746029229 shutdown_connections 2024-02-19T18:52:12.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.671+0000 7f438645f700 1 --2- 172.21.15.132:0/1746029229 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4368038490 0x7f436803a940 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:12.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.671+0000 7f438645f700 1 --2- 172.21.15.132:0/1746029229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43800ffe20 0x7f4380194f70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:12.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.671+0000 7f438645f700 1 -- 172.21.15.132:0/1746029229 >> 172.21.15.132:0/1746029229 conn(0x7f43800fb3d0 msgr2=0x7f43800fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:12.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.671+0000 7f438645f700 1 -- 172.21.15.132:0/1746029229 shutdown_connections 2024-02-19T18:52:12.689 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:12.671+0000 7f438645f700 1 -- 172.21.15.132:0/1746029229 wait complete. 2024-02-19T18:52:12.689 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:13.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:12 smithi088 ceph-mon[29272]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:13.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:12 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1746029229' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:14.286 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:14.286 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:14.488 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:14.534 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:15.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:14 smithi088 ceph-mon[29272]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:15.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.853+0000 7f73eb156700 1 -- 172.21.15.132:0/4050666725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73e40ffe20 msgr2=0x7f73e4100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:15.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.853+0000 7f73eb156700 1 --2- 172.21.15.132:0/4050666725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73e40ffe20 0x7f73e4100230 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f73d8009910 tx=0x7f73d8009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:15.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.853+0000 7f73eb156700 1 -- 172.21.15.132:0/4050666725 shutdown_connections 2024-02-19T18:52:15.871 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.853+0000 7f73eb156700 1 --2- 172.21.15.132:0/4050666725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73e40ffe20 0x7f73e4100230 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:15.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.853+0000 7f73eb156700 1 -- 172.21.15.132:0/4050666725 >> 172.21.15.132:0/4050666725 conn(0x7f73e40fb3d0 msgr2=0x7f73e40fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:15.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.853+0000 7f73eb156700 1 -- 172.21.15.132:0/4050666725 shutdown_connections 2024-02-19T18:52:15.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.853+0000 7f73eb156700 1 -- 172.21.15.132:0/4050666725 wait complete. 2024-02-19T18:52:15.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.853+0000 7f73eb156700 1 Processor -- start 2024-02-19T18:52:15.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.853+0000 7f73eb156700 1 -- start start 2024-02-19T18:52:15.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.853+0000 7f73eb156700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73e40ffe20 0x7f73e4074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:15.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.854+0000 7f73eb156700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e4075030 con 0x7f73e40ffe20 2024-02-19T18:52:15.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.854+0000 7f73e8ef2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73e40ffe20 0x7f73e4074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:15.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.854+0000 7f73e8ef2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73e40ffe20 0x7f73e4074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:42258/0 (socket says 172.21.15.132:42258) 2024-02-19T18:52:15.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.854+0000 7f73e8ef2700 1 -- 172.21.15.132:0/1104333215 learned_addr learned my addr 172.21.15.132:0/1104333215 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:15.872 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.854+0000 7f73e8ef2700 1 -- 172.21.15.132:0/1104333215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d80095f0 con 0x7f73e40ffe20 2024-02-19T18:52:15.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.854+0000 7f73e8ef2700 1 --2- 172.21.15.132:0/1104333215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73e40ffe20 0x7f73e4074af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f73d8005b40 tx=0x7f73d8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:15.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.854+0000 7f73e1ffb700 1 -- 172.21.15.132:0/1104333215 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73d801d070 con 0x7f73e40ffe20 2024-02-19T18:52:15.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.854+0000 7f73eb156700 1 -- 172.21.15.132:0/1104333215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e4073140 con 0x7f73e40ffe20 2024-02-19T18:52:15.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.854+0000 7f73e1ffb700 1 -- 172.21.15.132:0/1104333215 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f73d800be10 con 0x7f73e40ffe20 2024-02-19T18:52:15.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.854+0000 7f73e1ffb700 1 -- 172.21.15.132:0/1104333215 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f73d800f460 con 0x7f73e40ffe20 2024-02-19T18:52:15.873 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.855+0000 7f73eb156700 1 -- 172.21.15.132:0/1104333215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e40735e0 con 0x7f73e40ffe20 2024-02-19T18:52:15.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.856+0000 7f73eb156700 1 -- 172.21.15.132:0/1104333215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e4062380 con 0x7f73e40ffe20 2024-02-19T18:52:15.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.856+0000 7f73e1ffb700 1 -- 172.21.15.132:0/1104333215 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f73d80227e0 con 0x7f73e40ffe20 2024-02-19T18:52:15.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.856+0000 7f73e1ffb700 1 --2- 172.21.15.132:0/1104333215 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f73d4038490 0x7f73d403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:15.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.856+0000 7f73e1ffb700 1 -- 172.21.15.132:0/1104333215 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f73d804c2a0 con 0x7f73e40ffe20 2024-02-19T18:52:15.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.856+0000 7f73e3fff700 1 --2- 172.21.15.132:0/1104333215 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f73d4038490 0x7f73d403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:15.874 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.857+0000 7f73e3fff700 1 --2- 172.21.15.132:0/1104333215 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f73d4038490 0x7f73d403a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006fd0 tx=0x7f73d0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:15.877 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:15.861+0000 7f73e1ffb700 1 -- 172.21.15.132:0/1104333215 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f73d802b590 con 0x7f73e40ffe20 2024-02-19T18:52:16.147 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.131+0000 7f73eb156700 1 -- 172.21.15.132:0/1104333215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f73e418e460 con 0x7f73e40ffe20 2024-02-19T18:52:16.148 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.131+0000 7f73e1ffb700 1 -- 172.21.15.132:0/1104333215 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f73d8027020 con 0x7f73e40ffe20 2024-02-19T18:52:16.148 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:16.148 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:16.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.133+0000 7f73eb156700 1 -- 172.21.15.132:0/1104333215 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f73d4038490 msgr2=0x7f73d403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:16.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.133+0000 7f73eb156700 1 --2- 172.21.15.132:0/1104333215 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f73d4038490 0x7f73d403a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f73d0006fd0 tx=0x7f73d0006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:16.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.133+0000 7f73eb156700 1 -- 172.21.15.132:0/1104333215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73e40ffe20 msgr2=0x7f73e4074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:16.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.133+0000 7f73eb156700 1 --2- 172.21.15.132:0/1104333215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73e40ffe20 0x7f73e4074af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f73d8005b40 tx=0x7f73d8004740 comp rx=0 tx=0).stop 2024-02-19T18:52:16.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.133+0000 7f73eb156700 1 -- 172.21.15.132:0/1104333215 shutdown_connections 2024-02-19T18:52:16.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.133+0000 7f73eb156700 1 --2- 172.21.15.132:0/1104333215 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f73d4038490 0x7f73d403a940 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:16.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.133+0000 7f73eb156700 1 --2- 172.21.15.132:0/1104333215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73e40ffe20 0x7f73e4074af0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:16.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.133+0000 7f73eb156700 1 -- 172.21.15.132:0/1104333215 >> 172.21.15.132:0/1104333215 conn(0x7f73e40fb3d0 msgr2=0x7f73e40fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:16.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.133+0000 7f73eb156700 1 -- 172.21.15.132:0/1104333215 shutdown_connections 2024-02-19T18:52:16.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:16.133+0000 7f73eb156700 1 -- 172.21.15.132:0/1104333215 wait complete. 2024-02-19T18:52:16.151 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:17.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:16 smithi088 ceph-mon[29272]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:17.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:16 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1104333215' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:17.764 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:17.765 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:17.966 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:18.012 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:19.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:18 smithi088 ceph-mon[29272]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.551+0000 7f398ef14700 1 -- 172.21.15.132:0/3773296625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39880fe0e0 msgr2=0x7f39880fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.551+0000 7f398ef14700 1 --2- 172.21.15.132:0/3773296625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39880fe0e0 0x7f39880fe4f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f397c009910 tx=0x7f397c009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.551+0000 7f398ef14700 1 -- 172.21.15.132:0/3773296625 shutdown_connections 2024-02-19T18:52:19.569 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.551+0000 7f398ef14700 1 --2- 172.21.15.132:0/3773296625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39880fe0e0 0x7f39880fe4f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.551+0000 7f398ef14700 1 -- 172.21.15.132:0/3773296625 >> 172.21.15.132:0/3773296625 conn(0x7f39880f9af0 msgr2=0x7f39880fbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.551+0000 7f398ef14700 1 -- 172.21.15.132:0/3773296625 shutdown_connections 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.551+0000 7f398ef14700 1 -- 172.21.15.132:0/3773296625 wait complete. 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.551+0000 7f398ef14700 1 Processor -- start 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.551+0000 7f398ef14700 1 -- start start 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.552+0000 7f398ef14700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39880fe0e0 0x7f3988194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.552+0000 7f398ef14700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39881954d0 con 0x7f39880fe0e0 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.552+0000 7f398ccb0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39880fe0e0 0x7f3988194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.552+0000 7f398ccb0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39880fe0e0 0x7f3988194f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:42286/0 (socket says 172.21.15.132:42286) 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.552+0000 7f398ccb0700 1 -- 172.21.15.132:0/4246589315 learned_addr learned my addr 172.21.15.132:0/4246589315 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.552+0000 7f398ccb0700 1 -- 172.21.15.132:0/4246589315 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c0095f0 con 0x7f39880fe0e0 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.552+0000 7f398ccb0700 1 --2- 172.21.15.132:0/4246589315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39880fe0e0 0x7f3988194f90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f397c0092e0 tx=0x7f397c009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:19.570 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.552+0000 7f3985ffb700 1 -- 172.21.15.132:0/4246589315 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f397c01d070 con 0x7f39880fe0e0 2024-02-19T18:52:19.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.552+0000 7f3985ffb700 1 -- 172.21.15.132:0/4246589315 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f397c00be10 con 0x7f39880fe0e0 2024-02-19T18:52:19.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.552+0000 7f398ef14700 1 -- 172.21.15.132:0/4246589315 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39881956d0 con 0x7f39880fe0e0 2024-02-19T18:52:19.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.552+0000 7f3985ffb700 1 -- 172.21.15.132:0/4246589315 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f397c00f460 con 0x7f39880fe0e0 2024-02-19T18:52:19.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.553+0000 7f398ef14700 1 -- 172.21.15.132:0/4246589315 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3988195b70 con 0x7f39880fe0e0 2024-02-19T18:52:19.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.554+0000 7f3985ffb700 1 -- 172.21.15.132:0/4246589315 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f397c022470 con 0x7f39880fe0e0 2024-02-19T18:52:19.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.554+0000 7f398ef14700 1 -- 172.21.15.132:0/4246589315 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398818ea80 con 0x7f39880fe0e0 2024-02-19T18:52:19.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.554+0000 7f3985ffb700 1 --2- 172.21.15.132:0/4246589315 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3978038490 0x7f397803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:19.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.554+0000 7f3985ffb700 1 -- 172.21.15.132:0/4246589315 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f397c04c4a0 con 0x7f39880fe0e0 2024-02-19T18:52:19.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.554+0000 7f3987fff700 1 --2- 172.21.15.132:0/4246589315 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3978038490 0x7f397803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:19.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.554+0000 7f3987fff700 1 --2- 172.21.15.132:0/4246589315 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3978038490 0x7f397803a940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3974006fd0 tx=0x7f3974006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:19.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.559+0000 7f3985ffb700 1 -- 172.21.15.132:0/4246589315 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f397c00b960 con 0x7f39880fe0e0 2024-02-19T18:52:19.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.824+0000 7f398ef14700 1 -- 172.21.15.132:0/4246589315 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f398802cc30 con 0x7f39880fe0e0 2024-02-19T18:52:19.841 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.825+0000 7f3985ffb700 1 -- 172.21.15.132:0/4246589315 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f397c027070 con 0x7f39880fe0e0 2024-02-19T18:52:19.842 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:19.842 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.827+0000 7f398ef14700 1 -- 172.21.15.132:0/4246589315 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3978038490 msgr2=0x7f397803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.827+0000 7f398ef14700 1 --2- 172.21.15.132:0/4246589315 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3978038490 0x7f397803a940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3974006fd0 tx=0x7f3974006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.827+0000 7f398ef14700 1 -- 172.21.15.132:0/4246589315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39880fe0e0 msgr2=0x7f3988194f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.827+0000 7f398ef14700 1 --2- 172.21.15.132:0/4246589315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39880fe0e0 0x7f3988194f90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f397c0092e0 tx=0x7f397c009e60 comp rx=0 tx=0).stop 2024-02-19T18:52:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.827+0000 7f398ef14700 1 -- 172.21.15.132:0/4246589315 shutdown_connections 2024-02-19T18:52:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.827+0000 7f398ef14700 1 --2- 172.21.15.132:0/4246589315 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3978038490 0x7f397803a940 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.827+0000 7f398ef14700 1 --2- 172.21.15.132:0/4246589315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39880fe0e0 0x7f3988194f90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.827+0000 7f398ef14700 1 -- 172.21.15.132:0/4246589315 >> 172.21.15.132:0/4246589315 conn(0x7f39880f9af0 msgr2=0x7f39880fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.827+0000 7f398ef14700 1 -- 172.21.15.132:0/4246589315 shutdown_connections 2024-02-19T18:52:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:19.827+0000 7f398ef14700 1 -- 172.21.15.132:0/4246589315 wait complete. 2024-02-19T18:52:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:20.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:19 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/4246589315' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:21.204 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:20 smithi088 ceph-mon[29272]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:21.438 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:21.438 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:21.646 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:21.693 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:23.026 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:22 smithi088 ceph-mon[29272]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:23.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.161+0000 7fc9fd120700 1 -- 172.21.15.132:0/771219573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f80ffe20 msgr2=0x7fc9f8100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:23.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.161+0000 7fc9fd120700 1 --2- 172.21.15.132:0/771219573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f80ffe20 0x7fc9f8100230 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8009910 tx=0x7fc9e8009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.161+0000 7fc9fd120700 1 -- 172.21.15.132:0/771219573 shutdown_connections 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.161+0000 7fc9fd120700 1 --2- 172.21.15.132:0/771219573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f80ffe20 0x7fc9f8100230 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.161+0000 7fc9fd120700 1 -- 172.21.15.132:0/771219573 >> 172.21.15.132:0/771219573 conn(0x7fc9f80fb3d0 msgr2=0x7fc9f80fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.161+0000 7fc9fd120700 1 -- 172.21.15.132:0/771219573 shutdown_connections 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.161+0000 7fc9fd120700 1 -- 172.21.15.132:0/771219573 wait complete. 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.161+0000 7fc9fd120700 1 Processor -- start 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.162+0000 7fc9fd120700 1 -- start start 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.162+0000 7fc9fd120700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f80ffe20 0x7fc9f8197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.162+0000 7fc9fd120700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8197690 con 0x7fc9f80ffe20 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.162+0000 7fc9f6d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f80ffe20 0x7fc9f8197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.162+0000 7fc9f6d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f80ffe20 0x7fc9f8197150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:42316/0 (socket says 172.21.15.132:42316) 2024-02-19T18:52:23.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.162+0000 7fc9f6d9d700 1 -- 172.21.15.132:0/1445181298 learned_addr learned my addr 172.21.15.132:0/1445181298 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:23.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.162+0000 7fc9f6d9d700 1 -- 172.21.15.132:0/1445181298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e80095f0 con 0x7fc9f80ffe20 2024-02-19T18:52:23.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.162+0000 7fc9f6d9d700 1 --2- 172.21.15.132:0/1445181298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f80ffe20 0x7fc9f8197150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8005b40 tx=0x7fc9e8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:23.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.162+0000 7fc9dffff700 1 -- 172.21.15.132:0/1445181298 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9e801d070 con 0x7fc9f80ffe20 2024-02-19T18:52:23.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.163+0000 7fc9dffff700 1 -- 172.21.15.132:0/1445181298 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc9e800be10 con 0x7fc9f80ffe20 2024-02-19T18:52:23.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.163+0000 7fc9fd120700 1 -- 172.21.15.132:0/1445181298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f8197890 con 0x7fc9f80ffe20 2024-02-19T18:52:23.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.163+0000 7fc9dffff700 1 -- 172.21.15.132:0/1445181298 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc9e800f460 con 0x7fc9f80ffe20 2024-02-19T18:52:23.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.163+0000 7fc9fd120700 1 -- 172.21.15.132:0/1445181298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f8197d30 con 0x7fc9f80ffe20 2024-02-19T18:52:23.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.163+0000 7fc9dffff700 1 -- 172.21.15.132:0/1445181298 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fc9e800b960 con 0x7fc9f80ffe20 2024-02-19T18:52:23.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.164+0000 7fc9fd120700 1 -- 172.21.15.132:0/1445181298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f8190e80 con 0x7fc9f80ffe20 2024-02-19T18:52:23.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.164+0000 7fc9dffff700 1 --2- 172.21.15.132:0/1445181298 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc9e0038490 0x7fc9e003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:23.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.164+0000 7fc9dffff700 1 -- 172.21.15.132:0/1445181298 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc9e804c380 con 0x7fc9f80ffe20 2024-02-19T18:52:23.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.164+0000 7fc9f659c700 1 --2- 172.21.15.132:0/1445181298 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc9e0038490 0x7fc9e003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:23.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.164+0000 7fc9f659c700 1 --2- 172.21.15.132:0/1445181298 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc9e0038490 0x7fc9e003a940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec006fd0 tx=0x7fc9ec006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:23.185 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.169+0000 7fc9dffff700 1 -- 172.21.15.132:0/1445181298 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc9e802bba0 con 0x7fc9f80ffe20 2024-02-19T18:52:23.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.436+0000 7fc9fd120700 1 -- 172.21.15.132:0/1445181298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc9f802cc30 con 0x7fc9f80ffe20 2024-02-19T18:52:23.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.436+0000 7fc9dffff700 1 -- 172.21.15.132:0/1445181298 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc9e8027070 con 0x7fc9f80ffe20 2024-02-19T18:52:23.453 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:23.453 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.439+0000 7fc9fd120700 1 -- 172.21.15.132:0/1445181298 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc9e0038490 msgr2=0x7fc9e003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.439+0000 7fc9fd120700 1 --2- 172.21.15.132:0/1445181298 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc9e0038490 0x7fc9e003a940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec006fd0 tx=0x7fc9ec006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.439+0000 7fc9fd120700 1 -- 172.21.15.132:0/1445181298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f80ffe20 msgr2=0x7fc9f8197150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.439+0000 7fc9fd120700 1 --2- 172.21.15.132:0/1445181298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f80ffe20 0x7fc9f8197150 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8005b40 tx=0x7fc9e8004740 comp rx=0 tx=0).stop 2024-02-19T18:52:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.439+0000 7fc9fd120700 1 -- 172.21.15.132:0/1445181298 shutdown_connections 2024-02-19T18:52:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.439+0000 7fc9fd120700 1 --2- 172.21.15.132:0/1445181298 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc9e0038490 0x7fc9e003a940 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.439+0000 7fc9fd120700 1 --2- 172.21.15.132:0/1445181298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f80ffe20 0x7fc9f8197150 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.439+0000 7fc9fd120700 1 -- 172.21.15.132:0/1445181298 >> 172.21.15.132:0/1445181298 conn(0x7fc9f80fb3d0 msgr2=0x7fc9f80fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.439+0000 7fc9fd120700 1 -- 172.21.15.132:0/1445181298 shutdown_connections 2024-02-19T18:52:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:23.439+0000 7fc9fd120700 1 -- 172.21.15.132:0/1445181298 wait complete. 2024-02-19T18:52:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:24.290 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:23 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1445181298' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:25.049 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:25.049 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:25.254 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:25.301 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:25.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:24 smithi088 ceph-mon[29272]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:25.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:24 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:25.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:24 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:26.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:25 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:26.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:25 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:26.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:25 smithi088 ceph-mon[29272]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-19T18:52:26.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:25 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:26.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:25 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:26.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:25 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-19T18:52:26.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:25 smithi088 ceph-mon[29272]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-19T18:52:26.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:25 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:26.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:25 smithi088 ceph-mon[29272]: Deploying daemon grafana.smithi088 on smithi088 2024-02-19T18:52:26.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.751+0000 7f197734f700 1 -- 172.21.15.132:0/187730785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19700fed20 msgr2=0x7f19700ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:26.767 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.751+0000 7f197734f700 1 --2- 172.21.15.132:0/187730785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19700fed20 0x7f19700ff130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1960009910 tx=0x7f1960009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.751+0000 7f197734f700 1 -- 172.21.15.132:0/187730785 shutdown_connections 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.751+0000 7f197734f700 1 --2- 172.21.15.132:0/187730785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19700fed20 0x7f19700ff130 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.751+0000 7f197734f700 1 -- 172.21.15.132:0/187730785 >> 172.21.15.132:0/187730785 conn(0x7f19700fa2b0 msgr2=0x7f19700fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.751+0000 7f197734f700 1 -- 172.21.15.132:0/187730785 shutdown_connections 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.751+0000 7f197734f700 1 -- 172.21.15.132:0/187730785 wait complete. 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.751+0000 7f197734f700 1 Processor -- start 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.751+0000 7f197734f700 1 -- start start 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.751+0000 7f197734f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19700fed20 0x7f19700fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.751+0000 7f197734f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19700fc7a0 con 0x7f19700fed20 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.752+0000 7f19750eb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19700fed20 0x7f19700fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.752+0000 7f19750eb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19700fed20 0x7f19700fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:44038/0 (socket says 172.21.15.132:44038) 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.752+0000 7f19750eb700 1 -- 172.21.15.132:0/1663528600 learned_addr learned my addr 172.21.15.132:0/1663528600 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:26.768 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.752+0000 7f19750eb700 1 -- 172.21.15.132:0/1663528600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19600095f0 con 0x7f19700fed20 2024-02-19T18:52:26.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.752+0000 7f19750eb700 1 --2- 172.21.15.132:0/1663528600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19700fed20 0x7f19700fc260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1960005b40 tx=0x7f1960004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:26.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.752+0000 7f19667fc700 1 -- 172.21.15.132:0/1663528600 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f196001d070 con 0x7f19700fed20 2024-02-19T18:52:26.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.752+0000 7f19667fc700 1 -- 172.21.15.132:0/1663528600 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f196000be10 con 0x7f19700fed20 2024-02-19T18:52:26.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.752+0000 7f19667fc700 1 -- 172.21.15.132:0/1663528600 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f196000f460 con 0x7f19700fed20 2024-02-19T18:52:26.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.752+0000 7f197734f700 1 -- 172.21.15.132:0/1663528600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19700fa8b0 con 0x7f19700fed20 2024-02-19T18:52:26.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.752+0000 7f197734f700 1 -- 172.21.15.132:0/1663528600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19700fad50 con 0x7f19700fed20 2024-02-19T18:52:26.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.753+0000 7f19667fc700 1 -- 172.21.15.132:0/1663528600 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f19600227e0 con 0x7f19700fed20 2024-02-19T18:52:26.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.753+0000 7f197734f700 1 -- 172.21.15.132:0/1663528600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1970062380 con 0x7f19700fed20 2024-02-19T18:52:26.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.753+0000 7f19667fc700 1 --2- 172.21.15.132:0/1663528600 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f195c038490 0x7f195c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:26.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.753+0000 7f19667fc700 1 -- 172.21.15.132:0/1663528600 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f196004c4a0 con 0x7f19700fed20 2024-02-19T18:52:26.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.754+0000 7f19748ea700 1 --2- 172.21.15.132:0/1663528600 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f195c038490 0x7f195c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:26.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.754+0000 7f19748ea700 1 --2- 172.21.15.132:0/1663528600 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f195c038490 0x7f195c03a940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f196c006fd0 tx=0x7f196c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:26.776 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:26.761+0000 7f19667fc700 1 -- 172.21.15.132:0/1663528600 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f196002bba0 con 0x7f19700fed20 2024-02-19T18:52:27.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.028+0000 7f197734f700 1 -- 172.21.15.132:0/1663528600 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f197018e3f0 con 0x7f19700fed20 2024-02-19T18:52:27.044 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.028+0000 7f19667fc700 1 -- 172.21.15.132:0/1663528600 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1960027030 con 0x7f19700fed20 2024-02-19T18:52:27.044 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:27.044 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:27.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.030+0000 7f197734f700 1 -- 172.21.15.132:0/1663528600 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f195c038490 msgr2=0x7f195c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:27.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.030+0000 7f197734f700 1 --2- 172.21.15.132:0/1663528600 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f195c038490 0x7f195c03a940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f196c006fd0 tx=0x7f196c006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:27.046 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.030+0000 7f197734f700 1 -- 172.21.15.132:0/1663528600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19700fed20 msgr2=0x7f19700fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:27.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.030+0000 7f197734f700 1 --2- 172.21.15.132:0/1663528600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19700fed20 0x7f19700fc260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1960005b40 tx=0x7f1960004740 comp rx=0 tx=0).stop 2024-02-19T18:52:27.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.030+0000 7f197734f700 1 -- 172.21.15.132:0/1663528600 shutdown_connections 2024-02-19T18:52:27.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.030+0000 7f197734f700 1 --2- 172.21.15.132:0/1663528600 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f195c038490 0x7f195c03a940 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:27.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.030+0000 7f197734f700 1 --2- 172.21.15.132:0/1663528600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19700fed20 0x7f19700fc260 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:27.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.030+0000 7f197734f700 1 -- 172.21.15.132:0/1663528600 >> 172.21.15.132:0/1663528600 conn(0x7f19700fa2b0 msgr2=0x7f19700690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:27.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.030+0000 7f197734f700 1 -- 172.21.15.132:0/1663528600 shutdown_connections 2024-02-19T18:52:27.047 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:27.031+0000 7f197734f700 1 -- 172.21.15.132:0/1663528600 wait complete. 2024-02-19T18:52:27.047 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:27.296 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:27 smithi088 ceph-mon[29272]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:28.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:28 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1663528600' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:28.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:28 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:52:28.626 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:28.627 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:28.833 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:28.879 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:29.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:29 smithi088 ceph-mon[29272]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:30.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.323+0000 7fc2a069a700 1 -- 172.21.15.132:0/1239115900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2980fe4a0 msgr2=0x7fc2980fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:30.340 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.323+0000 7fc2a069a700 1 --2- 172.21.15.132:0/1239115900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2980fe4a0 0x7fc2980fe8b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fc288009910 tx=0x7fc288009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.323+0000 7fc2a069a700 1 -- 172.21.15.132:0/1239115900 shutdown_connections 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.323+0000 7fc2a069a700 1 --2- 172.21.15.132:0/1239115900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2980fe4a0 0x7fc2980fe8b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.323+0000 7fc2a069a700 1 -- 172.21.15.132:0/1239115900 >> 172.21.15.132:0/1239115900 conn(0x7fc2980f9af0 msgr2=0x7fc2980fbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.323+0000 7fc2a069a700 1 -- 172.21.15.132:0/1239115900 shutdown_connections 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.323+0000 7fc2a069a700 1 -- 172.21.15.132:0/1239115900 wait complete. 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.324+0000 7fc2a069a700 1 Processor -- start 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.324+0000 7fc2a069a700 1 -- start start 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.324+0000 7fc2a069a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2980fe4a0 0x7fc2980faa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.324+0000 7fc2a069a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc298197cb0 con 0x7fc2980fe4a0 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.324+0000 7fc29e436700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2980fe4a0 0x7fc2980faa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.324+0000 7fc29e436700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2980fe4a0 0x7fc2980faa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:44044/0 (socket says 172.21.15.132:44044) 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.324+0000 7fc29e436700 1 -- 172.21.15.132:0/645180013 learned_addr learned my addr 172.21.15.132:0/645180013 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:30.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.324+0000 7fc29e436700 1 -- 172.21.15.132:0/645180013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2880095f0 con 0x7fc2980fe4a0 2024-02-19T18:52:30.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.324+0000 7fc29e436700 1 --2- 172.21.15.132:0/645180013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2980fe4a0 0x7fc2980faa90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc2880092e0 tx=0x7fc288009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:30.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.325+0000 7fc28f7fe700 1 -- 172.21.15.132:0/645180013 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc28801d070 con 0x7fc2980fe4a0 2024-02-19T18:52:30.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.325+0000 7fc28f7fe700 1 -- 172.21.15.132:0/645180013 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc28800be10 con 0x7fc2980fe4a0 2024-02-19T18:52:30.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.325+0000 7fc2a069a700 1 -- 172.21.15.132:0/645180013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc298197df0 con 0x7fc2980fe4a0 2024-02-19T18:52:30.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.325+0000 7fc28f7fe700 1 -- 172.21.15.132:0/645180013 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc28800f460 con 0x7fc2980fe4a0 2024-02-19T18:52:30.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.325+0000 7fc2a069a700 1 -- 172.21.15.132:0/645180013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc298198290 con 0x7fc2980fe4a0 2024-02-19T18:52:30.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.325+0000 7fc2a069a700 1 -- 172.21.15.132:0/645180013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2980623f0 con 0x7fc2980fe4a0 2024-02-19T18:52:30.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.325+0000 7fc28f7fe700 1 -- 172.21.15.132:0/645180013 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fc288022470 con 0x7fc2980fe4a0 2024-02-19T18:52:30.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.326+0000 7fc28f7fe700 1 --2- 172.21.15.132:0/645180013 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc2840384e0 0x7fc28403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:30.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.326+0000 7fc28f7fe700 1 -- 172.21.15.132:0/645180013 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc28804c450 con 0x7fc2980fe4a0 2024-02-19T18:52:30.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.326+0000 7fc29dc35700 1 --2- 172.21.15.132:0/645180013 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc2840384e0 0x7fc28403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:30.343 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.326+0000 7fc29dc35700 1 --2- 172.21.15.132:0/645180013 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc2840384e0 0x7fc28403a990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc294006fd0 tx=0x7fc294006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:30.346 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.331+0000 7fc28f7fe700 1 -- 172.21.15.132:0/645180013 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc2980623f0 con 0x7fc2980fe4a0 2024-02-19T18:52:30.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.594+0000 7fc2a069a700 1 -- 172.21.15.132:0/645180013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc298198540 con 0x7fc2980fe4a0 2024-02-19T18:52:30.610 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.595+0000 7fc28f7fe700 1 -- 172.21.15.132:0/645180013 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fc288027070 con 0x7fc2980fe4a0 2024-02-19T18:52:30.611 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:30.611 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:30.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.597+0000 7fc2a069a700 1 -- 172.21.15.132:0/645180013 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc2840384e0 msgr2=0x7fc28403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:30.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.597+0000 7fc2a069a700 1 --2- 172.21.15.132:0/645180013 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc2840384e0 0x7fc28403a990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc294006fd0 tx=0x7fc294006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:30.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.597+0000 7fc2a069a700 1 -- 172.21.15.132:0/645180013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2980fe4a0 msgr2=0x7fc2980faa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:30.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.597+0000 7fc2a069a700 1 --2- 172.21.15.132:0/645180013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2980fe4a0 0x7fc2980faa90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc2880092e0 tx=0x7fc288009e60 comp rx=0 tx=0).stop 2024-02-19T18:52:30.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.597+0000 7fc2a069a700 1 -- 172.21.15.132:0/645180013 shutdown_connections 2024-02-19T18:52:30.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.597+0000 7fc2a069a700 1 --2- 172.21.15.132:0/645180013 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fc2840384e0 0x7fc28403a990 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:30.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.597+0000 7fc2a069a700 1 --2- 172.21.15.132:0/645180013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2980fe4a0 0x7fc2980faa90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:30.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.597+0000 7fc2a069a700 1 -- 172.21.15.132:0/645180013 >> 172.21.15.132:0/645180013 conn(0x7fc2980f9af0 msgr2=0x7fc2980fb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:30.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.597+0000 7fc2a069a700 1 -- 172.21.15.132:0/645180013 shutdown_connections 2024-02-19T18:52:30.613 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:30.597+0000 7fc2a069a700 1 -- 172.21.15.132:0/645180013 wait complete. 2024-02-19T18:52:30.614 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:31.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:31 smithi088 ceph-mon[29272]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:31.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:31 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/645180013' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:32.182 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:32.183 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:32.386 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:32.432 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:33.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:33 smithi088 ceph-mon[29272]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:33.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.751+0000 7f10effd9700 1 -- 172.21.15.132:0/338096873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10e80ffe30 msgr2=0x7f10e8100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:33.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.751+0000 7f10effd9700 1 --2- 172.21.15.132:0/338096873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10e80ffe30 0x7f10e8100240 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f10e000b0a0 tx=0x7f10e000b3b0 comp rx=0 tx=0).stop 2024-02-19T18:52:33.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.751+0000 7f10effd9700 1 -- 172.21.15.132:0/338096873 shutdown_connections 2024-02-19T18:52:33.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.751+0000 7f10effd9700 1 --2- 172.21.15.132:0/338096873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10e80ffe30 0x7f10e8100240 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:33.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.751+0000 7f10effd9700 1 -- 172.21.15.132:0/338096873 >> 172.21.15.132:0/338096873 conn(0x7f10e80fb3c0 msgr2=0x7f10e80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:33.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.751+0000 7f10effd9700 1 -- 172.21.15.132:0/338096873 shutdown_connections 2024-02-19T18:52:33.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.751+0000 7f10effd9700 1 -- 172.21.15.132:0/338096873 wait complete. 2024-02-19T18:52:33.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.752+0000 7f10effd9700 1 Processor -- start 2024-02-19T18:52:33.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.752+0000 7f10effd9700 1 -- start start 2024-02-19T18:52:33.769 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.752+0000 7f10effd9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10e80ffe30 0x7f10e8074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:33.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.752+0000 7f10effd9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e8075030 con 0x7f10e80ffe30 2024-02-19T18:52:33.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.752+0000 7f10edd75700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10e80ffe30 0x7f10e8074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:33.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.752+0000 7f10edd75700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10e80ffe30 0x7f10e8074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:44872/0 (socket says 172.21.15.132:44872) 2024-02-19T18:52:33.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.752+0000 7f10edd75700 1 -- 172.21.15.132:0/4033557602 learned_addr learned my addr 172.21.15.132:0/4033557602 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:33.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.752+0000 7f10edd75700 1 -- 172.21.15.132:0/4033557602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e0009d00 con 0x7f10e80ffe30 2024-02-19T18:52:33.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.753+0000 7f10edd75700 1 --2- 172.21.15.132:0/4033557602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10e80ffe30 0x7f10e8074af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10e0015040 tx=0x7f10e000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:33.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.753+0000 7f10e6ffd700 1 -- 172.21.15.132:0/4033557602 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10e000e040 con 0x7f10e80ffe30 2024-02-19T18:52:33.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.753+0000 7f10e6ffd700 1 -- 172.21.15.132:0/4033557602 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f10e001d930 con 0x7f10e80ffe30 2024-02-19T18:52:33.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.753+0000 7f10effd9700 1 -- 172.21.15.132:0/4033557602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e8073140 con 0x7f10e80ffe30 2024-02-19T18:52:33.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.753+0000 7f10e6ffd700 1 -- 172.21.15.132:0/4033557602 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f10e00129d0 con 0x7f10e80ffe30 2024-02-19T18:52:33.770 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.753+0000 7f10effd9700 1 -- 172.21.15.132:0/4033557602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e8073560 con 0x7f10e80ffe30 2024-02-19T18:52:33.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.754+0000 7f10e6ffd700 1 -- 172.21.15.132:0/4033557602 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f10e0012c90 con 0x7f10e80ffe30 2024-02-19T18:52:33.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.754+0000 7f10e6ffd700 1 --2- 172.21.15.132:0/4033557602 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f10d0038490 0x7f10d003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:33.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.754+0000 7f10effd9700 1 -- 172.21.15.132:0/4033557602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e8062380 con 0x7f10e80ffe30 2024-02-19T18:52:33.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.754+0000 7f10e6ffd700 1 -- 172.21.15.132:0/4033557602 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f10e004c7a0 con 0x7f10e80ffe30 2024-02-19T18:52:33.771 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.754+0000 7f10ed574700 1 --2- 172.21.15.132:0/4033557602 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f10d0038490 0x7f10d003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:33.772 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.754+0000 7f10ed574700 1 --2- 172.21.15.132:0/4033557602 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f10d0038490 0x7f10d003a940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f10dc006fd0 tx=0x7f10dc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:33.775 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:33.759+0000 7f10e6ffd700 1 -- 172.21.15.132:0/4033557602 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f10e0025360 con 0x7f10e80ffe30 2024-02-19T18:52:34.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.023+0000 7f10effd9700 1 -- 172.21.15.132:0/4033557602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f10e818e470 con 0x7f10e80ffe30 2024-02-19T18:52:34.039 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.024+0000 7f10e6ffd700 1 -- 172.21.15.132:0/4033557602 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f10e004a090 con 0x7f10e80ffe30 2024-02-19T18:52:34.040 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:34.040 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:34.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.026+0000 7f10effd9700 1 -- 172.21.15.132:0/4033557602 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f10d0038490 msgr2=0x7f10d003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:34.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.026+0000 7f10effd9700 1 --2- 172.21.15.132:0/4033557602 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f10d0038490 0x7f10d003a940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f10dc006fd0 tx=0x7f10dc006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:34.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.026+0000 7f10effd9700 1 -- 172.21.15.132:0/4033557602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10e80ffe30 msgr2=0x7f10e8074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:34.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.026+0000 7f10effd9700 1 --2- 172.21.15.132:0/4033557602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10e80ffe30 0x7f10e8074af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f10e0015040 tx=0x7f10e000bb10 comp rx=0 tx=0).stop 2024-02-19T18:52:34.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.026+0000 7f10effd9700 1 -- 172.21.15.132:0/4033557602 shutdown_connections 2024-02-19T18:52:34.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.026+0000 7f10effd9700 1 --2- 172.21.15.132:0/4033557602 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f10d0038490 0x7f10d003a940 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:34.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.026+0000 7f10effd9700 1 --2- 172.21.15.132:0/4033557602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10e80ffe30 0x7f10e8074af0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:34.042 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.026+0000 7f10effd9700 1 -- 172.21.15.132:0/4033557602 >> 172.21.15.132:0/4033557602 conn(0x7f10e80fb3c0 msgr2=0x7f10e80fc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:34.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.026+0000 7f10effd9700 1 -- 172.21.15.132:0/4033557602 shutdown_connections 2024-02-19T18:52:34.043 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:34.026+0000 7f10effd9700 1 -- 172.21.15.132:0/4033557602 wait complete. 2024-02-19T18:52:34.043 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:35.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:35 smithi088 ceph-mon[29272]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:35.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:35 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/4033557602' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:35.637 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:35.637 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:35.839 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:35.885 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:37.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.314+0000 7f21d8025700 1 -- 172.21.15.132:0/1472085331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d01002b0 msgr2=0x7f21d01006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:37.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.314+0000 7f21d8025700 1 --2- 172.21.15.132:0/1472085331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d01002b0 0x7f21d01006c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00b0a0 tx=0x7f21cc00b3b0 comp rx=0 tx=0).stop 2024-02-19T18:52:37.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.314+0000 7f21d8025700 1 -- 172.21.15.132:0/1472085331 shutdown_connections 2024-02-19T18:52:37.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.314+0000 7f21d8025700 1 --2- 172.21.15.132:0/1472085331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d01002b0 0x7f21d01006c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:37.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.314+0000 7f21d8025700 1 -- 172.21.15.132:0/1472085331 >> 172.21.15.132:0/1472085331 conn(0x7f21d00f9a50 msgr2=0x7f21d00fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:37.332 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.314+0000 7f21d8025700 1 -- 172.21.15.132:0/1472085331 shutdown_connections 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.314+0000 7f21d8025700 1 -- 172.21.15.132:0/1472085331 wait complete. 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.315+0000 7f21d8025700 1 Processor -- start 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.315+0000 7f21d8025700 1 -- start start 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.315+0000 7f21d8025700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d01002b0 0x7f21d0194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.315+0000 7f21d8025700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d01954f0 con 0x7f21d01002b0 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.315+0000 7f21d5dc1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d01002b0 0x7f21d0194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.315+0000 7f21d5dc1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d01002b0 0x7f21d0194fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:44892/0 (socket says 172.21.15.132:44892) 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.315+0000 7f21d5dc1700 1 -- 172.21.15.132:0/3258116220 learned_addr learned my addr 172.21.15.132:0/3258116220 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.316+0000 7f21d5dc1700 1 -- 172.21.15.132:0/3258116220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc009d00 con 0x7f21d01002b0 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.316+0000 7f21d5dc1700 1 --2- 172.21.15.132:0/3258116220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d01002b0 0x7f21d0194fb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00b830 tx=0x7f21cc00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.316+0000 7f21c6ffd700 1 -- 172.21.15.132:0/3258116220 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21cc00e040 con 0x7f21d01002b0 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.316+0000 7f21d8025700 1 -- 172.21.15.132:0/3258116220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d01956f0 con 0x7f21d01002b0 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.316+0000 7f21c6ffd700 1 -- 172.21.15.132:0/3258116220 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f21cc012af0 con 0x7f21d01002b0 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.316+0000 7f21c6ffd700 1 -- 172.21.15.132:0/3258116220 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21cc01d9f0 con 0x7f21d01002b0 2024-02-19T18:52:37.333 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.316+0000 7f21d8025700 1 -- 172.21.15.132:0/3258116220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d0195b10 con 0x7f21d01002b0 2024-02-19T18:52:37.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.317+0000 7f21c6ffd700 1 -- 172.21.15.132:0/3258116220 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f21cc012c60 con 0x7f21d01002b0 2024-02-19T18:52:37.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.317+0000 7f21d8025700 1 -- 172.21.15.132:0/3258116220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d018ea10 con 0x7f21d01002b0 2024-02-19T18:52:37.334 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.317+0000 7f21c6ffd700 1 --2- 172.21.15.132:0/3258116220 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f21bc038490 0x7f21bc03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:37.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.317+0000 7f21c6ffd700 1 -- 172.21.15.132:0/3258116220 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f21cc04c990 con 0x7f21d01002b0 2024-02-19T18:52:37.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.317+0000 7f21d55c0700 1 --2- 172.21.15.132:0/3258116220 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f21bc038490 0x7f21bc03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:37.335 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.318+0000 7f21d55c0700 1 --2- 172.21.15.132:0/3258116220 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f21bc038490 0x7f21bc03a940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006fd0 tx=0x7f21c0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:37.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.322+0000 7f21c6ffd700 1 -- 172.21.15.132:0/3258116220 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f21cc01c5c0 con 0x7f21d01002b0 2024-02-19T18:52:37.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:37 smithi088 ceph-mon[29272]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:37.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.585+0000 7f21d8025700 1 -- 172.21.15.132:0/3258116220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f21d0062380 con 0x7f21d01002b0 2024-02-19T18:52:37.601 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.586+0000 7f21c6ffd700 1 -- 172.21.15.132:0/3258116220 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f21cc01c3e0 con 0x7f21d01002b0 2024-02-19T18:52:37.602 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:37.602 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:37.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.588+0000 7f21d8025700 1 -- 172.21.15.132:0/3258116220 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f21bc038490 msgr2=0x7f21bc03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:37.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.588+0000 7f21d8025700 1 --2- 172.21.15.132:0/3258116220 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f21bc038490 0x7f21bc03a940 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006fd0 tx=0x7f21c0006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:37.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.588+0000 7f21d8025700 1 -- 172.21.15.132:0/3258116220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d01002b0 msgr2=0x7f21d0194fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:37.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.588+0000 7f21d8025700 1 --2- 172.21.15.132:0/3258116220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d01002b0 0x7f21d0194fb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00b830 tx=0x7f21cc00bd40 comp rx=0 tx=0).stop 2024-02-19T18:52:37.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.588+0000 7f21d8025700 1 -- 172.21.15.132:0/3258116220 shutdown_connections 2024-02-19T18:52:37.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.588+0000 7f21d8025700 1 --2- 172.21.15.132:0/3258116220 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f21bc038490 0x7f21bc03a940 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:37.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.588+0000 7f21d8025700 1 --2- 172.21.15.132:0/3258116220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d01002b0 0x7f21d0194fb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:37.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.588+0000 7f21d8025700 1 -- 172.21.15.132:0/3258116220 >> 172.21.15.132:0/3258116220 conn(0x7f21d00f9a50 msgr2=0x7f21d00fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:37.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.588+0000 7f21d8025700 1 -- 172.21.15.132:0/3258116220 shutdown_connections 2024-02-19T18:52:37.604 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:37.588+0000 7f21d8025700 1 -- 172.21.15.132:0/3258116220 wait complete. 2024-02-19T18:52:37.605 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:38.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:38 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/3258116220' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:39.188 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:39.188 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:39.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:39 smithi088 ceph-mon[29272]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:39.390 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:39.435 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:40.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.819+0000 7f3cbdb28700 1 -- 172.21.15.132:0/1115477991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb80fed20 msgr2=0x7f3cb80ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:40.836 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.819+0000 7f3cbdb28700 1 --2- 172.21.15.132:0/1115477991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb80fed20 0x7f3cb80ff130 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009910 tx=0x7f3cac009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.819+0000 7f3cbdb28700 1 -- 172.21.15.132:0/1115477991 shutdown_connections 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.819+0000 7f3cbdb28700 1 --2- 172.21.15.132:0/1115477991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb80fed20 0x7f3cb80ff130 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.819+0000 7f3cbdb28700 1 -- 172.21.15.132:0/1115477991 >> 172.21.15.132:0/1115477991 conn(0x7f3cb80fa2b0 msgr2=0x7f3cb80fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.820+0000 7f3cbdb28700 1 -- 172.21.15.132:0/1115477991 shutdown_connections 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.820+0000 7f3cbdb28700 1 -- 172.21.15.132:0/1115477991 wait complete. 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.820+0000 7f3cbdb28700 1 Processor -- start 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.820+0000 7f3cbdb28700 1 -- start start 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.820+0000 7f3cbdb28700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb80fed20 0x7f3cb80fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.820+0000 7f3cbdb28700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb80fc7a0 con 0x7f3cb80fed20 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.821+0000 7f3cb77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb80fed20 0x7f3cb80fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.821+0000 7f3cb77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb80fed20 0x7f3cb80fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:44920/0 (socket says 172.21.15.132:44920) 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.821+0000 7f3cb77fe700 1 -- 172.21.15.132:0/2603928867 learned_addr learned my addr 172.21.15.132:0/2603928867 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:40.837 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.821+0000 7f3cb77fe700 1 -- 172.21.15.132:0/2603928867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac0095f0 con 0x7f3cb80fed20 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.821+0000 7f3cb77fe700 1 --2- 172.21.15.132:0/2603928867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb80fed20 0x7f3cb80fc260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3cac005950 tx=0x7f3cac009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.821+0000 7f3cb4ff9700 1 -- 172.21.15.132:0/2603928867 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cac01d070 con 0x7f3cb80fed20 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.821+0000 7f3cbdb28700 1 -- 172.21.15.132:0/2603928867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb80fa8b0 con 0x7f3cb80fed20 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.821+0000 7f3cb4ff9700 1 -- 172.21.15.132:0/2603928867 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3cac00be10 con 0x7f3cb80fed20 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.821+0000 7f3cb4ff9700 1 -- 172.21.15.132:0/2603928867 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3cac00f460 con 0x7f3cb80fed20 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.821+0000 7f3cbdb28700 1 -- 172.21.15.132:0/2603928867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb80fad50 con 0x7f3cb80fed20 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.822+0000 7f3cb4ff9700 1 -- 172.21.15.132:0/2603928867 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f3cac00b960 con 0x7f3cb80fed20 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.822+0000 7f3cbdb28700 1 -- 172.21.15.132:0/2603928867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb8062380 con 0x7f3cb80fed20 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.822+0000 7f3cb4ff9700 1 --2- 172.21.15.132:0/2603928867 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3ca0038490 0x7f3ca003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.822+0000 7f3cb4ff9700 1 -- 172.21.15.132:0/2603928867 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3cac04c4f0 con 0x7f3cb80fed20 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.822+0000 7f3cb6ffd700 1 --2- 172.21.15.132:0/2603928867 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3ca0038490 0x7f3ca003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:40.839 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.823+0000 7f3cb6ffd700 1 --2- 172.21.15.132:0/2603928867 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3ca0038490 0x7f3ca003a940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8006fd0 tx=0x7f3ca8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:40.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:40.827+0000 7f3cb4ff9700 1 -- 172.21.15.132:0/2603928867 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3cac013250 con 0x7f3cb80fed20 2024-02-19T18:52:41.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.090+0000 7f3cbdb28700 1 -- 172.21.15.132:0/2603928867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3cb818e3f0 con 0x7f3cb80fed20 2024-02-19T18:52:41.106 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.090+0000 7f3cb4ff9700 1 -- 172.21.15.132:0/2603928867 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f3cac027030 con 0x7f3cb80fed20 2024-02-19T18:52:41.106 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:41.107 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:41.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.092+0000 7f3cbdb28700 1 -- 172.21.15.132:0/2603928867 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3ca0038490 msgr2=0x7f3ca003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:41.108 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.092+0000 7f3cbdb28700 1 --2- 172.21.15.132:0/2603928867 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3ca0038490 0x7f3ca003a940 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8006fd0 tx=0x7f3ca8006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.092+0000 7f3cbdb28700 1 -- 172.21.15.132:0/2603928867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb80fed20 msgr2=0x7f3cb80fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.092+0000 7f3cbdb28700 1 --2- 172.21.15.132:0/2603928867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb80fed20 0x7f3cb80fc260 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3cac005950 tx=0x7f3cac009e60 comp rx=0 tx=0).stop 2024-02-19T18:52:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.092+0000 7f3cbdb28700 1 -- 172.21.15.132:0/2603928867 shutdown_connections 2024-02-19T18:52:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.092+0000 7f3cbdb28700 1 --2- 172.21.15.132:0/2603928867 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f3ca0038490 0x7f3ca003a940 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.092+0000 7f3cbdb28700 1 --2- 172.21.15.132:0/2603928867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb80fed20 0x7f3cb80fc260 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.092+0000 7f3cbdb28700 1 -- 172.21.15.132:0/2603928867 >> 172.21.15.132:0/2603928867 conn(0x7f3cb80fa2b0 msgr2=0x7f3cb80690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.093+0000 7f3cbdb28700 1 -- 172.21.15.132:0/2603928867 shutdown_connections 2024-02-19T18:52:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:41.093+0000 7f3cbdb28700 1 -- 172.21.15.132:0/2603928867 wait complete. 2024-02-19T18:52:41.109 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:41.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:41 smithi088 ceph-mon[29272]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:42.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:42 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/2603928867' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:42.692 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:42.692 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:42.894 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:42.940 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:43.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:43 smithi088 ceph-mon[29272]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:44.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.195+0000 7f1affa82700 1 -- 172.21.15.132:0/1546763789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1af80ffe20 msgr2=0x7f1af8100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:44.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.195+0000 7f1affa82700 1 --2- 172.21.15.132:0/1546763789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1af80ffe20 0x7f1af8100230 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1af4009910 tx=0x7f1af4009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:44.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.195+0000 7f1affa82700 1 -- 172.21.15.132:0/1546763789 shutdown_connections 2024-02-19T18:52:44.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.195+0000 7f1affa82700 1 --2- 172.21.15.132:0/1546763789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1af80ffe20 0x7f1af8100230 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:44.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.195+0000 7f1affa82700 1 -- 172.21.15.132:0/1546763789 >> 172.21.15.132:0/1546763789 conn(0x7f1af80fb3d0 msgr2=0x7f1af80fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:44.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.195+0000 7f1affa82700 1 -- 172.21.15.132:0/1546763789 shutdown_connections 2024-02-19T18:52:44.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.195+0000 7f1affa82700 1 -- 172.21.15.132:0/1546763789 wait complete. 2024-02-19T18:52:44.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.196+0000 7f1affa82700 1 Processor -- start 2024-02-19T18:52:44.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.196+0000 7f1affa82700 1 -- start start 2024-02-19T18:52:44.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.196+0000 7f1affa82700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1af80ffe20 0x7f1af8194f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:44.213 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.196+0000 7f1affa82700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af81954a0 con 0x7f1af80ffe20 2024-02-19T18:52:44.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.196+0000 7f1afd81e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1af80ffe20 0x7f1af8194f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:44.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.196+0000 7f1afd81e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1af80ffe20 0x7f1af8194f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:43692/0 (socket says 172.21.15.132:43692) 2024-02-19T18:52:44.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.196+0000 7f1afd81e700 1 -- 172.21.15.132:0/1220803409 learned_addr learned my addr 172.21.15.132:0/1220803409 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:44.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.196+0000 7f1afd81e700 1 -- 172.21.15.132:0/1220803409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af40095f0 con 0x7f1af80ffe20 2024-02-19T18:52:44.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.197+0000 7f1afd81e700 1 --2- 172.21.15.132:0/1220803409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1af80ffe20 0x7f1af8194f60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1af4005b40 tx=0x7f1af4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:44.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.197+0000 7f1aeeffd700 1 -- 172.21.15.132:0/1220803409 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1af401d070 con 0x7f1af80ffe20 2024-02-19T18:52:44.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.197+0000 7f1affa82700 1 -- 172.21.15.132:0/1220803409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af81956a0 con 0x7f1af80ffe20 2024-02-19T18:52:44.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.197+0000 7f1affa82700 1 -- 172.21.15.132:0/1220803409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af8195b40 con 0x7f1af80ffe20 2024-02-19T18:52:44.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.197+0000 7f1aeeffd700 1 -- 172.21.15.132:0/1220803409 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1af400be10 con 0x7f1af80ffe20 2024-02-19T18:52:44.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.197+0000 7f1aeeffd700 1 -- 172.21.15.132:0/1220803409 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1af400f460 con 0x7f1af80ffe20 2024-02-19T18:52:44.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.198+0000 7f1affa82700 1 -- 172.21.15.132:0/1220803409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1adc005320 con 0x7f1af80ffe20 2024-02-19T18:52:44.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.198+0000 7f1aeeffd700 1 -- 172.21.15.132:0/1220803409 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f1af40227c0 con 0x7f1af80ffe20 2024-02-19T18:52:44.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.198+0000 7f1aeeffd700 1 --2- 172.21.15.132:0/1220803409 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1ae4038490 0x7f1ae403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:44.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.198+0000 7f1aeeffd700 1 -- 172.21.15.132:0/1220803409 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1af404c210 con 0x7f1af80ffe20 2024-02-19T18:52:44.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.198+0000 7f1afd01d700 1 --2- 172.21.15.132:0/1220803409 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1ae4038490 0x7f1ae403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:44.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.199+0000 7f1afd01d700 1 --2- 172.21.15.132:0/1220803409 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1ae4038490 0x7f1ae403a940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8006fd0 tx=0x7f1ae8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:44.219 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.203+0000 7f1aeeffd700 1 -- 172.21.15.132:0/1220803409 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1af4022470 con 0x7f1af80ffe20 2024-02-19T18:52:44.482 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.466+0000 7f1affa82700 1 -- 172.21.15.132:0/1220803409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f1adc005190 con 0x7f1af80ffe20 2024-02-19T18:52:44.483 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.467+0000 7f1aeeffd700 1 -- 172.21.15.132:0/1220803409 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1af4027030 con 0x7f1af80ffe20 2024-02-19T18:52:44.483 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:44.483 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:44.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.469+0000 7f1affa82700 1 -- 172.21.15.132:0/1220803409 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1ae4038490 msgr2=0x7f1ae403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:44.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.469+0000 7f1affa82700 1 --2- 172.21.15.132:0/1220803409 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1ae4038490 0x7f1ae403a940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8006fd0 tx=0x7f1ae8006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:44.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.469+0000 7f1affa82700 1 -- 172.21.15.132:0/1220803409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1af80ffe20 msgr2=0x7f1af8194f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:44.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.469+0000 7f1affa82700 1 --2- 172.21.15.132:0/1220803409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1af80ffe20 0x7f1af8194f60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1af4005b40 tx=0x7f1af4004740 comp rx=0 tx=0).stop 2024-02-19T18:52:44.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.470+0000 7f1affa82700 1 -- 172.21.15.132:0/1220803409 shutdown_connections 2024-02-19T18:52:44.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.470+0000 7f1affa82700 1 --2- 172.21.15.132:0/1220803409 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1ae4038490 0x7f1ae403a940 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:44.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.470+0000 7f1affa82700 1 --2- 172.21.15.132:0/1220803409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1af80ffe20 0x7f1af8194f60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:44.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.470+0000 7f1affa82700 1 -- 172.21.15.132:0/1220803409 >> 172.21.15.132:0/1220803409 conn(0x7f1af80fb3d0 msgr2=0x7f1af80fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:44.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.470+0000 7f1affa82700 1 -- 172.21.15.132:0/1220803409 shutdown_connections 2024-02-19T18:52:44.486 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:44.470+0000 7f1affa82700 1 -- 172.21.15.132:0/1220803409 wait complete. 2024-02-19T18:52:44.486 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:45.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:45 smithi088 ceph-mon[29272]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:45.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:45 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1220803409' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:46.048 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:46.048 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:46.248 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:46.295 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:47.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:47 smithi088 ceph-mon[29272]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.828+0000 7f8d1d311700 1 -- 172.21.15.132:0/273764538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d180fe0e0 msgr2=0x7f8d180fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.828+0000 7f8d1d311700 1 --2- 172.21.15.132:0/273764538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d180fe0e0 0x7f8d180fe4f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009910 tx=0x7f8d0c009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.829+0000 7f8d1d311700 1 -- 172.21.15.132:0/273764538 shutdown_connections 2024-02-19T18:52:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.829+0000 7f8d1d311700 1 --2- 172.21.15.132:0/273764538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d180fe0e0 0x7f8d180fe4f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.829+0000 7f8d1d311700 1 -- 172.21.15.132:0/273764538 >> 172.21.15.132:0/273764538 conn(0x7f8d180f9a50 msgr2=0x7f8d180fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.829+0000 7f8d1d311700 1 -- 172.21.15.132:0/273764538 shutdown_connections 2024-02-19T18:52:47.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.829+0000 7f8d1d311700 1 -- 172.21.15.132:0/273764538 wait complete. 2024-02-19T18:52:47.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.829+0000 7f8d1d311700 1 Processor -- start 2024-02-19T18:52:47.846 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.829+0000 7f8d1d311700 1 -- start start 2024-02-19T18:52:47.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.829+0000 7f8d1d311700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d180fe0e0 0x7f8d18194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:47.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.829+0000 7f8d1d311700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d181954f0 con 0x7f8d180fe0e0 2024-02-19T18:52:47.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.830+0000 7f8d16ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d180fe0e0 0x7f8d18194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:47.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.830+0000 7f8d16ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d180fe0e0 0x7f8d18194fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:43714/0 (socket says 172.21.15.132:43714) 2024-02-19T18:52:47.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.830+0000 7f8d16ffd700 1 -- 172.21.15.132:0/3022438251 learned_addr learned my addr 172.21.15.132:0/3022438251 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:47.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.830+0000 7f8d16ffd700 1 -- 172.21.15.132:0/3022438251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0095f0 con 0x7f8d180fe0e0 2024-02-19T18:52:47.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.830+0000 7f8d16ffd700 1 --2- 172.21.15.132:0/3022438251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d180fe0e0 0x7f8d18194fb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005b40 tx=0x7f8d0c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:47.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.830+0000 7f8cfffff700 1 -- 172.21.15.132:0/3022438251 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d0c01d070 con 0x7f8d180fe0e0 2024-02-19T18:52:47.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.831+0000 7f8d1d311700 1 -- 172.21.15.132:0/3022438251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d181956f0 con 0x7f8d180fe0e0 2024-02-19T18:52:47.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.831+0000 7f8cfffff700 1 -- 172.21.15.132:0/3022438251 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8d0c00be10 con 0x7f8d180fe0e0 2024-02-19T18:52:47.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.831+0000 7f8cfffff700 1 -- 172.21.15.132:0/3022438251 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8d0c00f460 con 0x7f8d180fe0e0 2024-02-19T18:52:47.848 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.831+0000 7f8d1d311700 1 -- 172.21.15.132:0/3022438251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d18195b90 con 0x7f8d180fe0e0 2024-02-19T18:52:47.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.832+0000 7f8d1d311700 1 -- 172.21.15.132:0/3022438251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1818eb00 con 0x7f8d180fe0e0 2024-02-19T18:52:47.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.832+0000 7f8cfffff700 1 -- 172.21.15.132:0/3022438251 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f8d0c0227e0 con 0x7f8d180fe0e0 2024-02-19T18:52:47.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.832+0000 7f8cfffff700 1 --2- 172.21.15.132:0/3022438251 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8d00038490 0x7f8d0003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:47.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.832+0000 7f8cfffff700 1 -- 172.21.15.132:0/3022438251 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8d0c04c460 con 0x7f8d180fe0e0 2024-02-19T18:52:47.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.832+0000 7f8d167fc700 1 --2- 172.21.15.132:0/3022438251 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8d00038490 0x7f8d0003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:47.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.832+0000 7f8d167fc700 1 --2- 172.21.15.132:0/3022438251 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8d00038490 0x7f8d0003a940 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8d08006fd0 tx=0x7f8d08006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:47.853 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:47.837+0000 7f8cfffff700 1 -- 172.21.15.132:0/3022438251 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8d0c02bba0 con 0x7f8d180fe0e0 2024-02-19T18:52:48.115 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.099+0000 7f8d1d311700 1 -- 172.21.15.132:0/3022438251 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f8d1802cc30 con 0x7f8d180fe0e0 2024-02-19T18:52:48.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.100+0000 7f8cfffff700 1 -- 172.21.15.132:0/3022438251 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f8d0c027030 con 0x7f8d180fe0e0 2024-02-19T18:52:48.116 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:48.116 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:48.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.102+0000 7f8d1d311700 1 -- 172.21.15.132:0/3022438251 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8d00038490 msgr2=0x7f8d0003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:48.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.102+0000 7f8d1d311700 1 --2- 172.21.15.132:0/3022438251 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8d00038490 0x7f8d0003a940 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8d08006fd0 tx=0x7f8d08006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:48.118 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.102+0000 7f8d1d311700 1 -- 172.21.15.132:0/3022438251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d180fe0e0 msgr2=0x7f8d18194fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:48.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.102+0000 7f8d1d311700 1 --2- 172.21.15.132:0/3022438251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d180fe0e0 0x7f8d18194fb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005b40 tx=0x7f8d0c004740 comp rx=0 tx=0).stop 2024-02-19T18:52:48.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.102+0000 7f8d1d311700 1 -- 172.21.15.132:0/3022438251 shutdown_connections 2024-02-19T18:52:48.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.102+0000 7f8d1d311700 1 --2- 172.21.15.132:0/3022438251 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f8d00038490 0x7f8d0003a940 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:48.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.102+0000 7f8d1d311700 1 --2- 172.21.15.132:0/3022438251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d180fe0e0 0x7f8d18194fb0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:48.119 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.102+0000 7f8d1d311700 1 -- 172.21.15.132:0/3022438251 >> 172.21.15.132:0/3022438251 conn(0x7f8d180f9a50 msgr2=0x7f8d180fa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:48.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.102+0000 7f8d1d311700 1 -- 172.21.15.132:0/3022438251 shutdown_connections 2024-02-19T18:52:48.120 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:48.102+0000 7f8d1d311700 1 -- 172.21.15.132:0/3022438251 wait complete. 2024-02-19T18:52:48.120 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:49.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:49 smithi088 ceph-mon[29272]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:49.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:49 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/3022438251' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:49.726 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:49.726 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:49.929 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:49.975 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:51.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:51 smithi088 ceph-mon[29272]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:51.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.449+0000 7fd953537700 1 -- 172.21.15.132:0/3534842760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd94c0ffe20 msgr2=0x7fd94c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:51.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.449+0000 7fd953537700 1 --2- 172.21.15.132:0/3534842760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd94c0ffe20 0x7fd94c100230 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd94400b0a0 tx=0x7fd94400b3b0 comp rx=0 tx=0).stop 2024-02-19T18:52:51.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.449+0000 7fd953537700 1 -- 172.21.15.132:0/3534842760 shutdown_connections 2024-02-19T18:52:51.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.449+0000 7fd953537700 1 --2- 172.21.15.132:0/3534842760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd94c0ffe20 0x7fd94c100230 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:51.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.449+0000 7fd953537700 1 -- 172.21.15.132:0/3534842760 >> 172.21.15.132:0/3534842760 conn(0x7fd94c0fb3d0 msgr2=0x7fd94c0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:51.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.449+0000 7fd953537700 1 -- 172.21.15.132:0/3534842760 shutdown_connections 2024-02-19T18:52:51.468 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.449+0000 7fd953537700 1 -- 172.21.15.132:0/3534842760 wait complete. 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.450+0000 7fd953537700 1 Processor -- start 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.450+0000 7fd953537700 1 -- start start 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.450+0000 7fd953537700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd94c0ffe20 0x7fd94c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.450+0000 7fd953537700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd94c075030 con 0x7fd94c0ffe20 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.450+0000 7fd9512d3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd94c0ffe20 0x7fd94c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.450+0000 7fd9512d3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd94c0ffe20 0x7fd94c074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:43742/0 (socket says 172.21.15.132:43742) 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.450+0000 7fd9512d3700 1 -- 172.21.15.132:0/1085046418 learned_addr learned my addr 172.21.15.132:0/1085046418 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.450+0000 7fd9512d3700 1 -- 172.21.15.132:0/1085046418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd944009d00 con 0x7fd94c0ffe20 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.451+0000 7fd9512d3700 1 --2- 172.21.15.132:0/1085046418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd94c0ffe20 0x7fd94c074af0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd944015040 tx=0x7fd94400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.451+0000 7fd94a7fc700 1 -- 172.21.15.132:0/1085046418 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd94400e040 con 0x7fd94c0ffe20 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.451+0000 7fd953537700 1 -- 172.21.15.132:0/1085046418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94c073140 con 0x7fd94c0ffe20 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.451+0000 7fd94a7fc700 1 -- 172.21.15.132:0/1085046418 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd94401d930 con 0x7fd94c0ffe20 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.451+0000 7fd94a7fc700 1 -- 172.21.15.132:0/1085046418 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9440129d0 con 0x7fd94c0ffe20 2024-02-19T18:52:51.469 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.451+0000 7fd953537700 1 -- 172.21.15.132:0/1085046418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94c073560 con 0x7fd94c0ffe20 2024-02-19T18:52:51.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.452+0000 7fd953537700 1 -- 172.21.15.132:0/1085046418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd94c062380 con 0x7fd94c0ffe20 2024-02-19T18:52:51.470 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.452+0000 7fd94a7fc700 1 -- 172.21.15.132:0/1085046418 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fd944012c90 con 0x7fd94c0ffe20 2024-02-19T18:52:51.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.452+0000 7fd94a7fc700 1 --2- 172.21.15.132:0/1085046418 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd934038490 0x7fd93403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:51.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.452+0000 7fd94a7fc700 1 -- 172.21.15.132:0/1085046418 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd94404c780 con 0x7fd94c0ffe20 2024-02-19T18:52:51.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.452+0000 7fd950ad2700 1 --2- 172.21.15.132:0/1085046418 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd934038490 0x7fd93403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:51.471 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.453+0000 7fd950ad2700 1 --2- 172.21.15.132:0/1085046418 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd934038490 0x7fd93403a940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd940006fd0 tx=0x7fd940006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:51.473 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.457+0000 7fd94a7fc700 1 -- 172.21.15.132:0/1085046418 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd94401c3e0 con 0x7fd94c0ffe20 2024-02-19T18:52:51.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.723+0000 7fd953537700 1 -- 172.21.15.132:0/1085046418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd94c18e460 con 0x7fd94c0ffe20 2024-02-19T18:52:51.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.724+0000 7fd94a7fc700 1 -- 172.21.15.132:0/1085046418 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fd94401c3e0 con 0x7fd94c0ffe20 2024-02-19T18:52:51.741 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:51.741 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:51.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.726+0000 7fd953537700 1 -- 172.21.15.132:0/1085046418 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd934038490 msgr2=0x7fd93403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:51.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.726+0000 7fd953537700 1 --2- 172.21.15.132:0/1085046418 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd934038490 0x7fd93403a940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd940006fd0 tx=0x7fd940006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:51.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.726+0000 7fd953537700 1 -- 172.21.15.132:0/1085046418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd94c0ffe20 msgr2=0x7fd94c074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:51.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.726+0000 7fd953537700 1 --2- 172.21.15.132:0/1085046418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd94c0ffe20 0x7fd94c074af0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd944015040 tx=0x7fd94400bb10 comp rx=0 tx=0).stop 2024-02-19T18:52:51.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.727+0000 7fd953537700 1 -- 172.21.15.132:0/1085046418 shutdown_connections 2024-02-19T18:52:51.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.727+0000 7fd953537700 1 --2- 172.21.15.132:0/1085046418 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fd934038490 0x7fd93403a940 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:51.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.727+0000 7fd953537700 1 --2- 172.21.15.132:0/1085046418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd94c0ffe20 0x7fd94c074af0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:51.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.727+0000 7fd953537700 1 -- 172.21.15.132:0/1085046418 >> 172.21.15.132:0/1085046418 conn(0x7fd94c0fb3d0 msgr2=0x7fd94c0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:51.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.727+0000 7fd953537700 1 -- 172.21.15.132:0/1085046418 shutdown_connections 2024-02-19T18:52:51.744 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:51.727+0000 7fd953537700 1 -- 172.21.15.132:0/1085046418 wait complete. 2024-02-19T18:52:51.744 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:52.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:52 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1085046418' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:53.313 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:53.313 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:53.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:53 smithi088 ceph-mon[29272]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:53.510 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:53.558 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:54.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.918+0000 7f45fce33700 1 -- 172.21.15.132:0/486447944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f80fed20 msgr2=0x7f45f80ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.918+0000 7f45fce33700 1 --2- 172.21.15.132:0/486447944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f80fed20 0x7f45f80ff130 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009910 tx=0x7f45ec009c20 comp rx=0 tx=0).stop 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.918+0000 7f45fce33700 1 -- 172.21.15.132:0/486447944 shutdown_connections 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.918+0000 7f45fce33700 1 --2- 172.21.15.132:0/486447944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f80fed20 0x7f45f80ff130 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.918+0000 7f45fce33700 1 -- 172.21.15.132:0/486447944 >> 172.21.15.132:0/486447944 conn(0x7f45f80fa2b0 msgr2=0x7f45f80fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.918+0000 7f45fce33700 1 -- 172.21.15.132:0/486447944 shutdown_connections 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.918+0000 7f45fce33700 1 -- 172.21.15.132:0/486447944 wait complete. 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.918+0000 7f45fce33700 1 Processor -- start 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.918+0000 7f45fce33700 1 -- start start 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.919+0000 7f45fce33700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f80fed20 0x7f45f8194fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.919+0000 7f45fce33700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f8195500 con 0x7f45f80fed20 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.919+0000 7f45f659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f80fed20 0x7f45f8194fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:54.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.919+0000 7f45f659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f80fed20 0x7f45f8194fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:38978/0 (socket says 172.21.15.132:38978) 2024-02-19T18:52:54.937 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.919+0000 7f45f659c700 1 -- 172.21.15.132:0/2508248505 learned_addr learned my addr 172.21.15.132:0/2508248505 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:54.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.919+0000 7f45f659c700 1 -- 172.21.15.132:0/2508248505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ec0095f0 con 0x7f45f80fed20 2024-02-19T18:52:54.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.919+0000 7f45f659c700 1 --2- 172.21.15.132:0/2508248505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f80fed20 0x7f45f8194fc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f45ec005950 tx=0x7f45ec009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:54.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.919+0000 7f45e77fe700 1 -- 172.21.15.132:0/2508248505 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45ec01d070 con 0x7f45f80fed20 2024-02-19T18:52:54.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.919+0000 7f45e77fe700 1 -- 172.21.15.132:0/2508248505 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f45ec00be10 con 0x7f45f80fed20 2024-02-19T18:52:54.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.919+0000 7f45fce33700 1 -- 172.21.15.132:0/2508248505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f8195700 con 0x7f45f80fed20 2024-02-19T18:52:54.938 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.919+0000 7f45e77fe700 1 -- 172.21.15.132:0/2508248505 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f45ec00f460 con 0x7f45f80fed20 2024-02-19T18:52:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.920+0000 7f45fce33700 1 -- 172.21.15.132:0/2508248505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f8195ba0 con 0x7f45f80fed20 2024-02-19T18:52:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.920+0000 7f45e77fe700 1 -- 172.21.15.132:0/2508248505 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f45ec00b960 con 0x7f45f80fed20 2024-02-19T18:52:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.920+0000 7f45fce33700 1 -- 172.21.15.132:0/2508248505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f8104b50 con 0x7f45f80fed20 2024-02-19T18:52:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.920+0000 7f45e77fe700 1 --2- 172.21.15.132:0/2508248505 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f45e00384e0 0x7f45e003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.921+0000 7f45e77fe700 1 -- 172.21.15.132:0/2508248505 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f45ec04c4c0 con 0x7f45f80fed20 2024-02-19T18:52:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.921+0000 7f45f5d9b700 1 --2- 172.21.15.132:0/2508248505 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f45e00384e0 0x7f45e003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:54.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.921+0000 7f45f5d9b700 1 --2- 172.21.15.132:0/2508248505 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f45e00384e0 0x7f45e003a990 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f45e8006fd0 tx=0x7f45e8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:54.942 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:54.926+0000 7f45e77fe700 1 -- 172.21.15.132:0/2508248505 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f45ec02bd80 con 0x7f45f80fed20 2024-02-19T18:52:55.211 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.194+0000 7f45fce33700 1 -- 172.21.15.132:0/2508248505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f45f8104d40 con 0x7f45f80fed20 2024-02-19T18:52:55.212 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.195+0000 7f45e77fe700 1 -- 172.21.15.132:0/2508248505 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f45ec027070 con 0x7f45f80fed20 2024-02-19T18:52:55.212 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:55.212 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:55.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.197+0000 7f45fce33700 1 -- 172.21.15.132:0/2508248505 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f45e00384e0 msgr2=0x7f45e003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:55.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.197+0000 7f45fce33700 1 --2- 172.21.15.132:0/2508248505 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f45e00384e0 0x7f45e003a990 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f45e8006fd0 tx=0x7f45e8006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:55.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.197+0000 7f45fce33700 1 -- 172.21.15.132:0/2508248505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f80fed20 msgr2=0x7f45f8194fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:55.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.197+0000 7f45fce33700 1 --2- 172.21.15.132:0/2508248505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f80fed20 0x7f45f8194fc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f45ec005950 tx=0x7f45ec009e60 comp rx=0 tx=0).stop 2024-02-19T18:52:55.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.197+0000 7f45fce33700 1 -- 172.21.15.132:0/2508248505 shutdown_connections 2024-02-19T18:52:55.214 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.197+0000 7f45fce33700 1 --2- 172.21.15.132:0/2508248505 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f45e00384e0 0x7f45e003a990 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:55.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.197+0000 7f45fce33700 1 --2- 172.21.15.132:0/2508248505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f80fed20 0x7f45f8194fc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:55.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.197+0000 7f45fce33700 1 -- 172.21.15.132:0/2508248505 >> 172.21.15.132:0/2508248505 conn(0x7f45f80fa2b0 msgr2=0x7f45f80faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:55.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.197+0000 7f45fce33700 1 -- 172.21.15.132:0/2508248505 shutdown_connections 2024-02-19T18:52:55.215 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:55.197+0000 7f45fce33700 1 -- 172.21.15.132:0/2508248505 wait complete. 2024-02-19T18:52:55.215 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:55.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:55 smithi088 ceph-mon[29272]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:56.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:56 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/2508248505' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:52:56.741 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:52:56.742 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:52:56.939 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:56.986 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:52:57.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:57 smithi088 ceph-mon[29272]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.511+0000 7f065f694700 1 -- 172.21.15.132:0/1036873163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06580ffe20 msgr2=0x7f0658100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.511+0000 7f065f694700 1 --2- 172.21.15.132:0/1036873163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06580ffe20 0x7f0658100230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f065000b0a0 tx=0x7f065000b3b0 comp rx=0 tx=0).stop 2024-02-19T18:52:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.511+0000 7f065f694700 1 -- 172.21.15.132:0/1036873163 shutdown_connections 2024-02-19T18:52:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.511+0000 7f065f694700 1 --2- 172.21.15.132:0/1036873163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06580ffe20 0x7f0658100230 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:58.529 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.511+0000 7f065f694700 1 -- 172.21.15.132:0/1036873163 >> 172.21.15.132:0/1036873163 conn(0x7f06580fb3d0 msgr2=0x7f06580fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.511+0000 7f065f694700 1 -- 172.21.15.132:0/1036873163 shutdown_connections 2024-02-19T18:52:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.511+0000 7f065f694700 1 -- 172.21.15.132:0/1036873163 wait complete. 2024-02-19T18:52:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.512+0000 7f065f694700 1 Processor -- start 2024-02-19T18:52:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.512+0000 7f065f694700 1 -- start start 2024-02-19T18:52:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.512+0000 7f065f694700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06580ffe20 0x7f0658194f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:58.530 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.512+0000 7f065f694700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06581954c0 con 0x7f06580ffe20 2024-02-19T18:52:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.513+0000 7f065d430700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06580ffe20 0x7f0658194f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.513+0000 7f065d430700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06580ffe20 0x7f0658194f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:39000/0 (socket says 172.21.15.132:39000) 2024-02-19T18:52:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.513+0000 7f065d430700 1 -- 172.21.15.132:0/2393930916 learned_addr learned my addr 172.21.15.132:0/2393930916 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:52:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.513+0000 7f065d430700 1 -- 172.21.15.132:0/2393930916 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0650009d00 con 0x7f06580ffe20 2024-02-19T18:52:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.513+0000 7f065d430700 1 --2- 172.21.15.132:0/2393930916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06580ffe20 0x7f0658194f80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0650015040 tx=0x7f065000bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.513+0000 7f06567fc700 1 -- 172.21.15.132:0/2393930916 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f065000e040 con 0x7f06580ffe20 2024-02-19T18:52:58.532 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.513+0000 7f065f694700 1 -- 172.21.15.132:0/2393930916 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06581956c0 con 0x7f06580ffe20 2024-02-19T18:52:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.513+0000 7f06567fc700 1 -- 172.21.15.132:0/2393930916 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f065001d930 con 0x7f06580ffe20 2024-02-19T18:52:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.513+0000 7f06567fc700 1 -- 172.21.15.132:0/2393930916 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f06500129d0 con 0x7f06580ffe20 2024-02-19T18:52:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.513+0000 7f065f694700 1 -- 172.21.15.132:0/2393930916 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0658195ae0 con 0x7f06580ffe20 2024-02-19T18:52:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.514+0000 7f06567fc700 1 -- 172.21.15.132:0/2393930916 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f0650012c90 con 0x7f06580ffe20 2024-02-19T18:52:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.514+0000 7f065f694700 1 -- 172.21.15.132:0/2393930916 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f065818ea80 con 0x7f06580ffe20 2024-02-19T18:52:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.514+0000 7f06567fc700 1 --2- 172.21.15.132:0/2393930916 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0640038490 0x7f064003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:52:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.514+0000 7f06567fc700 1 -- 172.21.15.132:0/2393930916 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f065004c7e0 con 0x7f06580ffe20 2024-02-19T18:52:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.515+0000 7f065cc2f700 1 --2- 172.21.15.132:0/2393930916 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0640038490 0x7f064003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:52:58.533 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.515+0000 7f065cc2f700 1 --2- 172.21.15.132:0/2393930916 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0640038490 0x7f064003a940 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f064c006fd0 tx=0x7f064c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:52:58.536 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.520+0000 7f06567fc700 1 -- 172.21.15.132:0/2393930916 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0650012430 con 0x7f06580ffe20 2024-02-19T18:52:58.802 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.785+0000 7f065f694700 1 -- 172.21.15.132:0/2393930916 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f0658062380 con 0x7f06580ffe20 2024-02-19T18:52:58.803 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.786+0000 7f06567fc700 1 -- 172.21.15.132:0/2393930916 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f065001aba0 con 0x7f06580ffe20 2024-02-19T18:52:58.803 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:52:58.803 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:52:58.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.788+0000 7f065f694700 1 -- 172.21.15.132:0/2393930916 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0640038490 msgr2=0x7f064003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:58.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.788+0000 7f065f694700 1 --2- 172.21.15.132:0/2393930916 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0640038490 0x7f064003a940 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f064c006fd0 tx=0x7f064c006e40 comp rx=0 tx=0).stop 2024-02-19T18:52:58.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.788+0000 7f065f694700 1 -- 172.21.15.132:0/2393930916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06580ffe20 msgr2=0x7f0658194f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:52:58.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.788+0000 7f065f694700 1 --2- 172.21.15.132:0/2393930916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06580ffe20 0x7f0658194f80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0650015040 tx=0x7f065000bb10 comp rx=0 tx=0).stop 2024-02-19T18:52:58.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.788+0000 7f065f694700 1 -- 172.21.15.132:0/2393930916 shutdown_connections 2024-02-19T18:52:58.805 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.788+0000 7f065f694700 1 --2- 172.21.15.132:0/2393930916 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f0640038490 0x7f064003a940 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:58.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.788+0000 7f065f694700 1 --2- 172.21.15.132:0/2393930916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06580ffe20 0x7f0658194f80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:52:58.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.788+0000 7f065f694700 1 -- 172.21.15.132:0/2393930916 >> 172.21.15.132:0/2393930916 conn(0x7f06580fb3d0 msgr2=0x7f06580fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:52:58.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.788+0000 7f065f694700 1 -- 172.21.15.132:0/2393930916 shutdown_connections 2024-02-19T18:52:58.806 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:52:58.788+0000 7f065f694700 1 -- 172.21.15.132:0/2393930916 wait complete. 2024-02-19T18:52:58.806 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:52:59.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:59 smithi088 ceph-mon[29272]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:52:59.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:52:59 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/2393930916' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:00.380 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:00.381 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:00.583 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:00.628 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:01.273 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:01 smithi088 ceph-mon[29272]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:01.273 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:01 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:01.273 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:01 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:02.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.051+0000 7efeefcc5700 1 -- 172.21.15.132:0/939035968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80ffe70 msgr2=0x7efee8100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:02.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.051+0000 7efeefcc5700 1 --2- 172.21.15.132:0/939035968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80ffe70 0x7efee8100280 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efed8009910 tx=0x7efed8009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:02.069 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.051+0000 7efeefcc5700 1 -- 172.21.15.132:0/939035968 shutdown_connections 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.051+0000 7efeefcc5700 1 --2- 172.21.15.132:0/939035968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80ffe70 0x7efee8100280 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.051+0000 7efeefcc5700 1 -- 172.21.15.132:0/939035968 >> 172.21.15.132:0/939035968 conn(0x7efee80fb470 msgr2=0x7efee80fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.051+0000 7efeefcc5700 1 -- 172.21.15.132:0/939035968 shutdown_connections 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.051+0000 7efeefcc5700 1 -- 172.21.15.132:0/939035968 wait complete. 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.052+0000 7efeefcc5700 1 Processor -- start 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.052+0000 7efeefcc5700 1 -- start start 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.052+0000 7efeefcc5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80ffe70 0x7efee8194fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.052+0000 7efeefcc5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee81954e0 con 0x7efee80ffe70 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.052+0000 7efeeda61700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80ffe70 0x7efee8194fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.052+0000 7efeeda61700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80ffe70 0x7efee8194fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:39022/0 (socket says 172.21.15.132:39022) 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.052+0000 7efeeda61700 1 -- 172.21.15.132:0/1940724845 learned_addr learned my addr 172.21.15.132:0/1940724845 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.052+0000 7efeeda61700 1 -- 172.21.15.132:0/1940724845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80095f0 con 0x7efee80ffe70 2024-02-19T18:53:02.070 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.053+0000 7efeeda61700 1 --2- 172.21.15.132:0/1940724845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80ffe70 0x7efee8194fa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efed8005950 tx=0x7efed8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:02.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.053+0000 7efedeffd700 1 -- 172.21.15.132:0/1940724845 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efed801d070 con 0x7efee80ffe70 2024-02-19T18:53:02.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.053+0000 7efeefcc5700 1 -- 172.21.15.132:0/1940724845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee81956e0 con 0x7efee80ffe70 2024-02-19T18:53:02.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.053+0000 7efedeffd700 1 -- 172.21.15.132:0/1940724845 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efed800be10 con 0x7efee80ffe70 2024-02-19T18:53:02.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.053+0000 7efedeffd700 1 -- 172.21.15.132:0/1940724845 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efed800f460 con 0x7efee80ffe70 2024-02-19T18:53:02.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.053+0000 7efeefcc5700 1 -- 172.21.15.132:0/1940724845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee8195b80 con 0x7efee80ffe70 2024-02-19T18:53:02.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.054+0000 7efedeffd700 1 -- 172.21.15.132:0/1940724845 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7efed800f690 con 0x7efee80ffe70 2024-02-19T18:53:02.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.054+0000 7efedeffd700 1 --2- 172.21.15.132:0/1940724845 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efed4038490 0x7efed403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:02.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.054+0000 7efeefcc5700 1 -- 172.21.15.132:0/1940724845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee818ea70 con 0x7efee80ffe70 2024-02-19T18:53:02.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.054+0000 7efedeffd700 1 -- 172.21.15.132:0/1940724845 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efed804c510 con 0x7efee80ffe70 2024-02-19T18:53:02.072 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.054+0000 7efeed260700 1 --2- 172.21.15.132:0/1940724845 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efed4038490 0x7efed403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:02.073 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.054+0000 7efeed260700 1 --2- 172.21.15.132:0/1940724845 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efed4038490 0x7efed403a940 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efee4006fd0 tx=0x7efee4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:02.075 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.059+0000 7efedeffd700 1 -- 172.21.15.132:0/1940724845 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efed8027020 con 0x7efee80ffe70 2024-02-19T18:53:02.338 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.321+0000 7efeefcc5700 1 -- 172.21.15.132:0/1940724845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efee802cc30 con 0x7efee80ffe70 2024-02-19T18:53:02.339 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.322+0000 7efedeffd700 1 -- 172.21.15.132:0/1940724845 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7efed802c440 con 0x7efee80ffe70 2024-02-19T18:53:02.339 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:02.339 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:02.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.324+0000 7efeefcc5700 1 -- 172.21.15.132:0/1940724845 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efed4038490 msgr2=0x7efed403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:02.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.324+0000 7efeefcc5700 1 --2- 172.21.15.132:0/1940724845 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efed4038490 0x7efed403a940 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efee4006fd0 tx=0x7efee4006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:02.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.324+0000 7efeefcc5700 1 -- 172.21.15.132:0/1940724845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80ffe70 msgr2=0x7efee8194fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:02.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.324+0000 7efeefcc5700 1 --2- 172.21.15.132:0/1940724845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80ffe70 0x7efee8194fa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efed8005950 tx=0x7efed8009e60 comp rx=0 tx=0).stop 2024-02-19T18:53:02.341 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.324+0000 7efeefcc5700 1 -- 172.21.15.132:0/1940724845 shutdown_connections 2024-02-19T18:53:02.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.324+0000 7efeefcc5700 1 --2- 172.21.15.132:0/1940724845 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efed4038490 0x7efed403a940 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:02.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.324+0000 7efeefcc5700 1 --2- 172.21.15.132:0/1940724845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80ffe70 0x7efee8194fa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:02.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.324+0000 7efeefcc5700 1 -- 172.21.15.132:0/1940724845 >> 172.21.15.132:0/1940724845 conn(0x7efee80fb470 msgr2=0x7efee80fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:02.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.324+0000 7efeefcc5700 1 -- 172.21.15.132:0/1940724845 shutdown_connections 2024-02-19T18:53:02.342 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:02.324+0000 7efeefcc5700 1 -- 172.21.15.132:0/1940724845 wait complete. 2024-02-19T18:53:02.342 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:02.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:02 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:02.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:02 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:02.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:02 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:02.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:02 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:03.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:03 smithi088 ceph-mon[29272]: Deploying daemon prometheus.smithi088 on smithi088 2024-02-19T18:53:03.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:03 smithi088 ceph-mon[29272]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:03.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:03 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1940724845' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:03.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:03 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:03.886 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:03.886 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:04.089 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:04.135 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:05.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:05 smithi088 ceph-mon[29272]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.612+0000 7f14b7cbe700 1 -- 172.21.15.132:0/306932909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b0074d80 msgr2=0x7f14b00731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.612+0000 7f14b7cbe700 1 --2- 172.21.15.132:0/306932909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b0074d80 0x7f14b00731e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009910 tx=0x7f14ac009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.612+0000 7f14b7cbe700 1 -- 172.21.15.132:0/306932909 shutdown_connections 2024-02-19T18:53:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.612+0000 7f14b7cbe700 1 --2- 172.21.15.132:0/306932909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b0074d80 0x7f14b00731e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.612+0000 7f14b7cbe700 1 -- 172.21.15.132:0/306932909 >> 172.21.15.132:0/306932909 conn(0x7f14b00fb3d0 msgr2=0x7f14b00fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.612+0000 7f14b7cbe700 1 -- 172.21.15.132:0/306932909 shutdown_connections 2024-02-19T18:53:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.612+0000 7f14b7cbe700 1 -- 172.21.15.132:0/306932909 wait complete. 2024-02-19T18:53:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.613+0000 7f14b7cbe700 1 Processor -- start 2024-02-19T18:53:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.613+0000 7f14b7cbe700 1 -- start start 2024-02-19T18:53:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.613+0000 7f14b7cbe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b0074d80 0x7f14b019b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:05.631 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.613+0000 7f14b7cbe700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b019ba90 con 0x7f14b0074d80 2024-02-19T18:53:05.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.613+0000 7f14b5a5a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b0074d80 0x7f14b019b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:05.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.613+0000 7f14b5a5a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b0074d80 0x7f14b019b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:40356/0 (socket says 172.21.15.132:40356) 2024-02-19T18:53:05.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.613+0000 7f14b5a5a700 1 -- 172.21.15.132:0/1759169654 learned_addr learned my addr 172.21.15.132:0/1759169654 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:05.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.614+0000 7f14b5a5a700 1 -- 172.21.15.132:0/1759169654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac0095f0 con 0x7f14b0074d80 2024-02-19T18:53:05.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.614+0000 7f14b5a5a700 1 --2- 172.21.15.132:0/1759169654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b0074d80 0x7f14b019b550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f14ac005b40 tx=0x7f14ac004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:05.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.614+0000 7f14a6ffd700 1 -- 172.21.15.132:0/1759169654 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14ac01d070 con 0x7f14b0074d80 2024-02-19T18:53:05.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.614+0000 7f14a6ffd700 1 -- 172.21.15.132:0/1759169654 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f14ac00be10 con 0x7f14b0074d80 2024-02-19T18:53:05.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.614+0000 7f14a6ffd700 1 -- 172.21.15.132:0/1759169654 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f14ac00f460 con 0x7f14b0074d80 2024-02-19T18:53:05.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.614+0000 7f14b7cbe700 1 -- 172.21.15.132:0/1759169654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b019bc90 con 0x7f14b0074d80 2024-02-19T18:53:05.632 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.614+0000 7f14b7cbe700 1 -- 172.21.15.132:0/1759169654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b019c130 con 0x7f14b0074d80 2024-02-19T18:53:05.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.615+0000 7f14a6ffd700 1 -- 172.21.15.132:0/1759169654 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f14ac0227e0 con 0x7f14b0074d80 2024-02-19T18:53:05.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.615+0000 7f14b7cbe700 1 -- 172.21.15.132:0/1759169654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b0195100 con 0x7f14b0074d80 2024-02-19T18:53:05.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.615+0000 7f14a6ffd700 1 --2- 172.21.15.132:0/1759169654 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f149c038490 0x7f149c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:05.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.615+0000 7f14a6ffd700 1 -- 172.21.15.132:0/1759169654 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f14ac04c4b0 con 0x7f14b0074d80 2024-02-19T18:53:05.633 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.616+0000 7f14b5259700 1 --2- 172.21.15.132:0/1759169654 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f149c038490 0x7f149c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:05.634 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.616+0000 7f14b5259700 1 --2- 172.21.15.132:0/1759169654 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f149c038490 0x7f149c03a940 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006fd0 tx=0x7f14a0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:05.636 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.620+0000 7f14a6ffd700 1 -- 172.21.15.132:0/1759169654 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f14ac02bba0 con 0x7f14b0074d80 2024-02-19T18:53:05.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.881+0000 7f14b7cbe700 1 -- 172.21.15.132:0/1759169654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f14b002d050 con 0x7f14b0074d80 2024-02-19T18:53:05.898 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.882+0000 7f14a6ffd700 1 -- 172.21.15.132:0/1759169654 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f14ac027030 con 0x7f14b0074d80 2024-02-19T18:53:05.898 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:05.898 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:05.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.884+0000 7f14b7cbe700 1 -- 172.21.15.132:0/1759169654 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f149c038490 msgr2=0x7f149c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:05.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.884+0000 7f14b7cbe700 1 --2- 172.21.15.132:0/1759169654 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f149c038490 0x7f149c03a940 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f14a0006fd0 tx=0x7f14a0006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:05.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.884+0000 7f14b7cbe700 1 -- 172.21.15.132:0/1759169654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b0074d80 msgr2=0x7f14b019b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:05.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.884+0000 7f14b7cbe700 1 --2- 172.21.15.132:0/1759169654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b0074d80 0x7f14b019b550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f14ac005b40 tx=0x7f14ac004740 comp rx=0 tx=0).stop 2024-02-19T18:53:05.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.884+0000 7f14b7cbe700 1 -- 172.21.15.132:0/1759169654 shutdown_connections 2024-02-19T18:53:05.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.884+0000 7f14b7cbe700 1 --2- 172.21.15.132:0/1759169654 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f149c038490 0x7f149c03a940 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:05.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.884+0000 7f14b7cbe700 1 --2- 172.21.15.132:0/1759169654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14b0074d80 0x7f14b019b550 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:05.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.884+0000 7f14b7cbe700 1 -- 172.21.15.132:0/1759169654 >> 172.21.15.132:0/1759169654 conn(0x7f14b00fb3d0 msgr2=0x7f14b00fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:05.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.884+0000 7f14b7cbe700 1 -- 172.21.15.132:0/1759169654 shutdown_connections 2024-02-19T18:53:05.901 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:05.884+0000 7f14b7cbe700 1 -- 172.21.15.132:0/1759169654 wait complete. 2024-02-19T18:53:05.901 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:06.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:06 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1759169654' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:07.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:07 smithi088 ceph-mon[29272]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:07.448 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:07.448 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:07.650 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:07.696 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:09.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.162+0000 7f5a4d742700 1 -- 172.21.15.132:0/3705931264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a480fed20 msgr2=0x7f5a480ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:09.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.162+0000 7f5a4d742700 1 --2- 172.21.15.132:0/3705931264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a480fed20 0x7f5a480ff130 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5a4000b0a0 tx=0x7f5a4000b3b0 comp rx=0 tx=0).stop 2024-02-19T18:53:09.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.162+0000 7f5a4d742700 1 -- 172.21.15.132:0/3705931264 shutdown_connections 2024-02-19T18:53:09.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.162+0000 7f5a4d742700 1 --2- 172.21.15.132:0/3705931264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a480fed20 0x7f5a480ff130 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:09.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.162+0000 7f5a4d742700 1 -- 172.21.15.132:0/3705931264 >> 172.21.15.132:0/3705931264 conn(0x7f5a480fa2b0 msgr2=0x7f5a480fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:09.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.162+0000 7f5a4d742700 1 -- 172.21.15.132:0/3705931264 shutdown_connections 2024-02-19T18:53:09.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.162+0000 7f5a4d742700 1 -- 172.21.15.132:0/3705931264 wait complete. 2024-02-19T18:53:09.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.163+0000 7f5a4d742700 1 Processor -- start 2024-02-19T18:53:09.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.163+0000 7f5a4d742700 1 -- start start 2024-02-19T18:53:09.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.164+0000 7f5a4d742700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a48068ad0 0x7f5a48197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:09.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.164+0000 7f5a4d742700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48068ee0 con 0x7f5a48068ad0 2024-02-19T18:53:09.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.164+0000 7f5a46ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a48068ad0 0x7f5a48197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:09.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.164+0000 7f5a46ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a48068ad0 0x7f5a48197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:40380/0 (socket says 172.21.15.132:40380) 2024-02-19T18:53:09.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.164+0000 7f5a46ffd700 1 -- 172.21.15.132:0/3747241829 learned_addr learned my addr 172.21.15.132:0/3747241829 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:09.181 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.164+0000 7f5a46ffd700 1 -- 172.21.15.132:0/3747241829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a40009d00 con 0x7f5a48068ad0 2024-02-19T18:53:09.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.164+0000 7f5a46ffd700 1 --2- 172.21.15.132:0/3747241829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a48068ad0 0x7f5a48197ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009570 tx=0x7f5a40003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:09.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.164+0000 7f5a3ffff700 1 -- 172.21.15.132:0/3747241829 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a4000e040 con 0x7f5a48068ad0 2024-02-19T18:53:09.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.165+0000 7f5a3ffff700 1 -- 172.21.15.132:0/3747241829 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5a4000bc70 con 0x7f5a48068ad0 2024-02-19T18:53:09.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.165+0000 7f5a4d742700 1 -- 172.21.15.132:0/3747241829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a48198430 con 0x7f5a48068ad0 2024-02-19T18:53:09.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.165+0000 7f5a3ffff700 1 -- 172.21.15.132:0/3747241829 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5a40012670 con 0x7f5a48068ad0 2024-02-19T18:53:09.182 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.165+0000 7f5a4d742700 1 -- 172.21.15.132:0/3747241829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a481988b0 con 0x7f5a48068ad0 2024-02-19T18:53:09.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.166+0000 7f5a3ffff700 1 -- 172.21.15.132:0/3747241829 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f5a40019040 con 0x7f5a48068ad0 2024-02-19T18:53:09.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.166+0000 7f5a4d742700 1 -- 172.21.15.132:0/3747241829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4818da00 con 0x7f5a48068ad0 2024-02-19T18:53:09.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.166+0000 7f5a3ffff700 1 --2- 172.21.15.132:0/3747241829 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f5a340384e0 0x7f5a3403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:09.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.166+0000 7f5a3ffff700 1 -- 172.21.15.132:0/3747241829 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5a4004c130 con 0x7f5a48068ad0 2024-02-19T18:53:09.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.166+0000 7f5a467fc700 1 --2- 172.21.15.132:0/3747241829 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f5a340384e0 0x7f5a3403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:09.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.166+0000 7f5a467fc700 1 --2- 172.21.15.132:0/3747241829 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f5a340384e0 0x7f5a3403a990 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5a30006fd0 tx=0x7f5a30006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:09.186 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.171+0000 7f5a3ffff700 1 -- 172.21.15.132:0/3747241829 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5a4001d440 con 0x7f5a48068ad0 2024-02-19T18:53:09.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:09 smithi088 ceph-mon[29272]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:09.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.434+0000 7f5a4d742700 1 -- 172.21.15.132:0/3747241829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5a4802cc30 con 0x7f5a48068ad0 2024-02-19T18:53:09.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.435+0000 7f5a3ffff700 1 -- 172.21.15.132:0/3747241829 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f5a40021020 con 0x7f5a48068ad0 2024-02-19T18:53:09.451 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:09.451 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:09.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.437+0000 7f5a4d742700 1 -- 172.21.15.132:0/3747241829 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f5a340384e0 msgr2=0x7f5a3403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:09.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.437+0000 7f5a4d742700 1 --2- 172.21.15.132:0/3747241829 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f5a340384e0 0x7f5a3403a990 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5a30006fd0 tx=0x7f5a30006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:09.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.437+0000 7f5a4d742700 1 -- 172.21.15.132:0/3747241829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a48068ad0 msgr2=0x7f5a48197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:09.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.437+0000 7f5a4d742700 1 --2- 172.21.15.132:0/3747241829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a48068ad0 0x7f5a48197ef0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009570 tx=0x7f5a40003ce0 comp rx=0 tx=0).stop 2024-02-19T18:53:09.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.438+0000 7f5a4d742700 1 -- 172.21.15.132:0/3747241829 shutdown_connections 2024-02-19T18:53:09.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.438+0000 7f5a4d742700 1 --2- 172.21.15.132:0/3747241829 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f5a340384e0 0x7f5a3403a990 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:09.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.438+0000 7f5a4d742700 1 --2- 172.21.15.132:0/3747241829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a48068ad0 0x7f5a48197ef0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:09.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.438+0000 7f5a4d742700 1 -- 172.21.15.132:0/3747241829 >> 172.21.15.132:0/3747241829 conn(0x7f5a480fa2b0 msgr2=0x7f5a480fb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:09.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.438+0000 7f5a4d742700 1 -- 172.21.15.132:0/3747241829 shutdown_connections 2024-02-19T18:53:09.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:09.438+0000 7f5a4d742700 1 -- 172.21.15.132:0/3747241829 wait complete. 2024-02-19T18:53:09.454 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:10.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:10 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/3747241829' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:11.056 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:11.056 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:11.259 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:11.304 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:11.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:11 smithi088 ceph-mon[29272]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:12.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.763+0000 7f1334852700 1 -- 172.21.15.132:0/2501852915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13300fed20 msgr2=0x7f13300ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.763+0000 7f1334852700 1 --2- 172.21.15.132:0/2501852915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13300fed20 0x7f13300ff130 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1324009910 tx=0x7f1324009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.764+0000 7f1334852700 1 -- 172.21.15.132:0/2501852915 shutdown_connections 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.764+0000 7f1334852700 1 --2- 172.21.15.132:0/2501852915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13300fed20 0x7f13300ff130 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.764+0000 7f1334852700 1 -- 172.21.15.132:0/2501852915 >> 172.21.15.132:0/2501852915 conn(0x7f13300fa2b0 msgr2=0x7f13300fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.764+0000 7f1334852700 1 -- 172.21.15.132:0/2501852915 shutdown_connections 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.764+0000 7f1334852700 1 -- 172.21.15.132:0/2501852915 wait complete. 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.764+0000 7f1334852700 1 Processor -- start 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.764+0000 7f1334852700 1 -- start start 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.764+0000 7f1334852700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13300fed20 0x7f1330194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.764+0000 7f1334852700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13301954f0 con 0x7f13300fed20 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.765+0000 7f132e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13300fed20 0x7f1330194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:12.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.765+0000 7f132e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13300fed20 0x7f1330194fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:40396/0 (socket says 172.21.15.132:40396) 2024-02-19T18:53:12.782 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.765+0000 7f132e59c700 1 -- 172.21.15.132:0/1306658441 learned_addr learned my addr 172.21.15.132:0/1306658441 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:12.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.765+0000 7f132e59c700 1 -- 172.21.15.132:0/1306658441 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13240095f0 con 0x7f13300fed20 2024-02-19T18:53:12.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.765+0000 7f132e59c700 1 --2- 172.21.15.132:0/1306658441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13300fed20 0x7f1330194fb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1324005950 tx=0x7f1324009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:12.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.765+0000 7f131f7fe700 1 -- 172.21.15.132:0/1306658441 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f132401d070 con 0x7f13300fed20 2024-02-19T18:53:12.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.765+0000 7f131f7fe700 1 -- 172.21.15.132:0/1306658441 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f132400be10 con 0x7f13300fed20 2024-02-19T18:53:12.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.765+0000 7f1334852700 1 -- 172.21.15.132:0/1306658441 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13301956f0 con 0x7f13300fed20 2024-02-19T18:53:12.783 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.766+0000 7f131f7fe700 1 -- 172.21.15.132:0/1306658441 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f132400f460 con 0x7f13300fed20 2024-02-19T18:53:12.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.766+0000 7f1334852700 1 -- 172.21.15.132:0/1306658441 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1330195b90 con 0x7f13300fed20 2024-02-19T18:53:12.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.766+0000 7f131f7fe700 1 -- 172.21.15.132:0/1306658441 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f132400b960 con 0x7f13300fed20 2024-02-19T18:53:12.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.766+0000 7f1334852700 1 -- 172.21.15.132:0/1306658441 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133018ea10 con 0x7f13300fed20 2024-02-19T18:53:12.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.766+0000 7f131f7fe700 1 --2- 172.21.15.132:0/1306658441 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1318038490 0x7f131803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:12.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.767+0000 7f131f7fe700 1 -- 172.21.15.132:0/1306658441 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f132404c4f0 con 0x7f13300fed20 2024-02-19T18:53:12.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.767+0000 7f132dd9b700 1 --2- 172.21.15.132:0/1306658441 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1318038490 0x7f131803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:12.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.767+0000 7f132dd9b700 1 --2- 172.21.15.132:0/1306658441 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1318038490 0x7f131803a940 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1320006fd0 tx=0x7f1320006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:12.787 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:12.772+0000 7f131f7fe700 1 -- 172.21.15.132:0/1306658441 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1324013250 con 0x7f13300fed20 2024-02-19T18:53:13.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.039+0000 7f1334852700 1 -- 172.21.15.132:0/1306658441 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f133002cc30 con 0x7f13300fed20 2024-02-19T18:53:13.055 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.039+0000 7f131f7fe700 1 -- 172.21.15.132:0/1306658441 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f1324027030 con 0x7f13300fed20 2024-02-19T18:53:13.056 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:13.056 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:13.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.042+0000 7f1334852700 1 -- 172.21.15.132:0/1306658441 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1318038490 msgr2=0x7f131803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:13.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.042+0000 7f1334852700 1 --2- 172.21.15.132:0/1306658441 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1318038490 0x7f131803a940 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1320006fd0 tx=0x7f1320006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:13.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.042+0000 7f1334852700 1 -- 172.21.15.132:0/1306658441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13300fed20 msgr2=0x7f1330194fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:13.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.042+0000 7f1334852700 1 --2- 172.21.15.132:0/1306658441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13300fed20 0x7f1330194fb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1324005950 tx=0x7f1324009e60 comp rx=0 tx=0).stop 2024-02-19T18:53:13.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.042+0000 7f1334852700 1 -- 172.21.15.132:0/1306658441 shutdown_connections 2024-02-19T18:53:13.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.042+0000 7f1334852700 1 --2- 172.21.15.132:0/1306658441 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f1318038490 0x7f131803a940 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:13.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.042+0000 7f1334852700 1 --2- 172.21.15.132:0/1306658441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13300fed20 0x7f1330194fb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:13.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.042+0000 7f1334852700 1 -- 172.21.15.132:0/1306658441 >> 172.21.15.132:0/1306658441 conn(0x7f13300fa2b0 msgr2=0x7f13300690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:13.058 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.042+0000 7f1334852700 1 -- 172.21.15.132:0/1306658441 shutdown_connections 2024-02-19T18:53:13.059 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:13.042+0000 7f1334852700 1 -- 172.21.15.132:0/1306658441 wait complete. 2024-02-19T18:53:13.059 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:13.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:13 smithi088 ceph-mon[29272]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:13.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:13 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1306658441' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:14.578 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:14.578 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:14.778 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:14.825 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:15.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:15 smithi088 ceph-mon[29272]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:16.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.133+0000 7f4bd5746700 1 -- 172.21.15.132:0/761201976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bd00ffe70 msgr2=0x7f4bd0100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:16.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.133+0000 7f4bd5746700 1 --2- 172.21.15.132:0/761201976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bd00ffe70 0x7f4bd0100280 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0009910 tx=0x7f4bc0009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:16.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.133+0000 7f4bd5746700 1 -- 172.21.15.132:0/761201976 shutdown_connections 2024-02-19T18:53:16.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.133+0000 7f4bd5746700 1 --2- 172.21.15.132:0/761201976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bd00ffe70 0x7f4bd0100280 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:16.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.133+0000 7f4bd5746700 1 -- 172.21.15.132:0/761201976 >> 172.21.15.132:0/761201976 conn(0x7f4bd00fb470 msgr2=0x7f4bd00fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:16.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.133+0000 7f4bd5746700 1 -- 172.21.15.132:0/761201976 shutdown_connections 2024-02-19T18:53:16.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.133+0000 7f4bd5746700 1 -- 172.21.15.132:0/761201976 wait complete. 2024-02-19T18:53:16.149 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.133+0000 7f4bd5746700 1 Processor -- start 2024-02-19T18:53:16.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.133+0000 7f4bd5746700 1 -- start start 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.134+0000 7f4bd5746700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bd00ffe70 0x7f4bd0074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.134+0000 7f4bd5746700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd0075030 con 0x7f4bd00ffe70 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.134+0000 7f4bceffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bd00ffe70 0x7f4bd0074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.134+0000 7f4bceffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bd00ffe70 0x7f4bd0074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:58620/0 (socket says 172.21.15.132:58620) 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.134+0000 7f4bceffd700 1 -- 172.21.15.132:0/3286783967 learned_addr learned my addr 172.21.15.132:0/3286783967 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.134+0000 7f4bceffd700 1 -- 172.21.15.132:0/3286783967 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc00095f0 con 0x7f4bd00ffe70 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.134+0000 7f4bceffd700 1 --2- 172.21.15.132:0/3286783967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bd00ffe70 0x7f4bd0074af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0005950 tx=0x7f4bc0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.134+0000 7f4bb7fff700 1 -- 172.21.15.132:0/3286783967 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bc001d070 con 0x7f4bd00ffe70 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.135+0000 7f4bd5746700 1 -- 172.21.15.132:0/3286783967 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd0073140 con 0x7f4bd00ffe70 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.135+0000 7f4bd5746700 1 -- 172.21.15.132:0/3286783967 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd00735e0 con 0x7f4bd00ffe70 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.135+0000 7f4bb7fff700 1 -- 172.21.15.132:0/3286783967 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4bc000be10 con 0x7f4bd00ffe70 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.135+0000 7f4bb7fff700 1 -- 172.21.15.132:0/3286783967 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4bc0022d50 con 0x7f4bd00ffe70 2024-02-19T18:53:16.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.135+0000 7f4bb7fff700 1 -- 172.21.15.132:0/3286783967 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f4bc000b960 con 0x7f4bd00ffe70 2024-02-19T18:53:16.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.135+0000 7f4bd5746700 1 -- 172.21.15.132:0/3286783967 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd0062380 con 0x7f4bd00ffe70 2024-02-19T18:53:16.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.136+0000 7f4bb7fff700 1 --2- 172.21.15.132:0/3286783967 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4bb8038490 0x7f4bb803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:16.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.136+0000 7f4bb7fff700 1 -- 172.21.15.132:0/3286783967 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4bc004c280 con 0x7f4bd00ffe70 2024-02-19T18:53:16.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.136+0000 7f4bce7fc700 1 --2- 172.21.15.132:0/3286783967 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4bb8038490 0x7f4bb803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:16.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.136+0000 7f4bce7fc700 1 --2- 172.21.15.132:0/3286783967 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4bb8038490 0x7f4bb803a940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006fd0 tx=0x7f4bc4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:16.156 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.141+0000 7f4bb7fff700 1 -- 172.21.15.132:0/3286783967 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4bc00227c0 con 0x7f4bd00ffe70 2024-02-19T18:53:16.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.407+0000 7f4bd5746700 1 -- 172.21.15.132:0/3286783967 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4bd018e450 con 0x7f4bd00ffe70 2024-02-19T18:53:16.423 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.407+0000 7f4bb7fff700 1 -- 172.21.15.132:0/3286783967 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f4bc0027030 con 0x7f4bd00ffe70 2024-02-19T18:53:16.424 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:16.424 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:16.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.409+0000 7f4bd5746700 1 -- 172.21.15.132:0/3286783967 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4bb8038490 msgr2=0x7f4bb803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:16.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.409+0000 7f4bd5746700 1 --2- 172.21.15.132:0/3286783967 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4bb8038490 0x7f4bb803a940 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006fd0 tx=0x7f4bc4006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:16.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.409+0000 7f4bd5746700 1 -- 172.21.15.132:0/3286783967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bd00ffe70 msgr2=0x7f4bd0074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:16.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.409+0000 7f4bd5746700 1 --2- 172.21.15.132:0/3286783967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bd00ffe70 0x7f4bd0074af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4bc0005950 tx=0x7f4bc0009e60 comp rx=0 tx=0).stop 2024-02-19T18:53:16.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.410+0000 7f4bd5746700 1 -- 172.21.15.132:0/3286783967 shutdown_connections 2024-02-19T18:53:16.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.410+0000 7f4bd5746700 1 --2- 172.21.15.132:0/3286783967 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f4bb8038490 0x7f4bb803a940 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:16.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.410+0000 7f4bd5746700 1 --2- 172.21.15.132:0/3286783967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bd00ffe70 0x7f4bd0074af0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:16.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.410+0000 7f4bd5746700 1 -- 172.21.15.132:0/3286783967 >> 172.21.15.132:0/3286783967 conn(0x7f4bd00fb470 msgr2=0x7f4bd00fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:16.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.410+0000 7f4bd5746700 1 -- 172.21.15.132:0/3286783967 shutdown_connections 2024-02-19T18:53:16.426 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:16.410+0000 7f4bd5746700 1 -- 172.21.15.132:0/3286783967 wait complete. 2024-02-19T18:53:16.427 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:17.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:17 smithi088 ceph-mon[29272]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:17.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:17 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/3286783967' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:18.001 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:18.001 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:18.205 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:18.251 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:19.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:19 smithi088 ceph-mon[29272]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:19.842 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.823+0000 7fbc4733f700 1 -- 172.21.15.132:0/2247899585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc400ffe20 msgr2=0x7fbc40100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:19.842 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.823+0000 7fbc4733f700 1 --2- 172.21.15.132:0/2247899585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc400ffe20 0x7fbc40100230 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbc3800b0a0 tx=0x7fbc3800b3b0 comp rx=0 tx=0).stop 2024-02-19T18:53:19.842 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.824+0000 7fbc4733f700 1 -- 172.21.15.132:0/2247899585 shutdown_connections 2024-02-19T18:53:19.842 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.824+0000 7fbc4733f700 1 --2- 172.21.15.132:0/2247899585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc400ffe20 0x7fbc40100230 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:19.842 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.824+0000 7fbc4733f700 1 -- 172.21.15.132:0/2247899585 >> 172.21.15.132:0/2247899585 conn(0x7fbc400fb3d0 msgr2=0x7fbc400fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:19.842 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.824+0000 7fbc4733f700 1 -- 172.21.15.132:0/2247899585 shutdown_connections 2024-02-19T18:53:19.842 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.824+0000 7fbc4733f700 1 -- 172.21.15.132:0/2247899585 wait complete. 2024-02-19T18:53:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.824+0000 7fbc4733f700 1 Processor -- start 2024-02-19T18:53:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.824+0000 7fbc4733f700 1 -- start start 2024-02-19T18:53:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.824+0000 7fbc4733f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc400ffe20 0x7fbc40194f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.824+0000 7fbc4733f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc401954a0 con 0x7fbc400ffe20 2024-02-19T18:53:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.825+0000 7fbc450db700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc400ffe20 0x7fbc40194f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.825+0000 7fbc450db700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc400ffe20 0x7fbc40194f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:58646/0 (socket says 172.21.15.132:58646) 2024-02-19T18:53:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.825+0000 7fbc450db700 1 -- 172.21.15.132:0/1605323175 learned_addr learned my addr 172.21.15.132:0/1605323175 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.825+0000 7fbc450db700 1 -- 172.21.15.132:0/1605323175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc38009d00 con 0x7fbc400ffe20 2024-02-19T18:53:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.825+0000 7fbc450db700 1 --2- 172.21.15.132:0/1605323175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc400ffe20 0x7fbc40194f60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbc38015040 tx=0x7fbc3800bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.825+0000 7fbc3e7fc700 1 -- 172.21.15.132:0/1605323175 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc3800e040 con 0x7fbc400ffe20 2024-02-19T18:53:19.843 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.825+0000 7fbc4733f700 1 -- 172.21.15.132:0/1605323175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc401956a0 con 0x7fbc400ffe20 2024-02-19T18:53:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.825+0000 7fbc3e7fc700 1 -- 172.21.15.132:0/1605323175 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbc3801d930 con 0x7fbc400ffe20 2024-02-19T18:53:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.825+0000 7fbc4733f700 1 -- 172.21.15.132:0/1605323175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc40195ac0 con 0x7fbc400ffe20 2024-02-19T18:53:19.844 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.825+0000 7fbc3e7fc700 1 -- 172.21.15.132:0/1605323175 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc380129e0 con 0x7fbc400ffe20 2024-02-19T18:53:19.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.826+0000 7fbc4733f700 1 -- 172.21.15.132:0/1605323175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc4018ea80 con 0x7fbc400ffe20 2024-02-19T18:53:19.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.826+0000 7fbc3e7fc700 1 -- 172.21.15.132:0/1605323175 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fbc3801d480 con 0x7fbc400ffe20 2024-02-19T18:53:19.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.827+0000 7fbc3e7fc700 1 --2- 172.21.15.132:0/1605323175 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc28038490 0x7fbc2803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:19.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.827+0000 7fbc3e7fc700 1 -- 172.21.15.132:0/1605323175 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbc3804c6c0 con 0x7fbc400ffe20 2024-02-19T18:53:19.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.827+0000 7fbc448da700 1 --2- 172.21.15.132:0/1605323175 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc28038490 0x7fbc2803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:19.845 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.827+0000 7fbc448da700 1 --2- 172.21.15.132:0/1605323175 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc28038490 0x7fbc2803a940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbc34006fd0 tx=0x7fbc34006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:19.847 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:19.832+0000 7fbc3e7fc700 1 -- 172.21.15.132:0/1605323175 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbc3801c3e0 con 0x7fbc400ffe20 2024-02-19T18:53:20.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.097+0000 7fbc4733f700 1 -- 172.21.15.132:0/1605323175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbc40062380 con 0x7fbc400ffe20 2024-02-19T18:53:20.113 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.097+0000 7fbc3e7fc700 1 -- 172.21.15.132:0/1605323175 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fbc3801c3e0 con 0x7fbc400ffe20 2024-02-19T18:53:20.114 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:20.114 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:20.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.099+0000 7fbc4733f700 1 -- 172.21.15.132:0/1605323175 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc28038490 msgr2=0x7fbc2803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:20.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.099+0000 7fbc4733f700 1 --2- 172.21.15.132:0/1605323175 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc28038490 0x7fbc2803a940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbc34006fd0 tx=0x7fbc34006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:20.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.099+0000 7fbc4733f700 1 -- 172.21.15.132:0/1605323175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc400ffe20 msgr2=0x7fbc40194f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:20.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.099+0000 7fbc4733f700 1 --2- 172.21.15.132:0/1605323175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc400ffe20 0x7fbc40194f60 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbc38015040 tx=0x7fbc3800bb10 comp rx=0 tx=0).stop 2024-02-19T18:53:20.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.099+0000 7fbc4733f700 1 -- 172.21.15.132:0/1605323175 shutdown_connections 2024-02-19T18:53:20.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.099+0000 7fbc4733f700 1 --2- 172.21.15.132:0/1605323175 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc28038490 0x7fbc2803a940 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:20.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.099+0000 7fbc4733f700 1 --2- 172.21.15.132:0/1605323175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc400ffe20 0x7fbc40194f60 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:20.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.099+0000 7fbc4733f700 1 -- 172.21.15.132:0/1605323175 >> 172.21.15.132:0/1605323175 conn(0x7fbc400fb3d0 msgr2=0x7fbc400fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:20.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.099+0000 7fbc4733f700 1 -- 172.21.15.132:0/1605323175 shutdown_connections 2024-02-19T18:53:20.116 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:20.099+0000 7fbc4733f700 1 -- 172.21.15.132:0/1605323175 wait complete. 2024-02-19T18:53:20.117 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:21.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:21 smithi088 ceph-mon[29272]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:21.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:21 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1605323175' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:21.713 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:21.714 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:21.918 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:21.964 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:23.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.435+0000 7f676c1af700 1 -- 172.21.15.132:0/2127895591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67640fed20 msgr2=0x7f67640ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:23.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.435+0000 7f676c1af700 1 --2- 172.21.15.132:0/2127895591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67640fed20 0x7f67640ff130 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6760009910 tx=0x7f6760009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:23.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.435+0000 7f676c1af700 1 -- 172.21.15.132:0/2127895591 shutdown_connections 2024-02-19T18:53:23.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.435+0000 7f676c1af700 1 --2- 172.21.15.132:0/2127895591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67640fed20 0x7f67640ff130 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:23.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.435+0000 7f676c1af700 1 -- 172.21.15.132:0/2127895591 >> 172.21.15.132:0/2127895591 conn(0x7f67640fa2b0 msgr2=0x7f67640fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:23.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.436+0000 7f676c1af700 1 -- 172.21.15.132:0/2127895591 shutdown_connections 2024-02-19T18:53:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.436+0000 7f676c1af700 1 -- 172.21.15.132:0/2127895591 wait complete. 2024-02-19T18:53:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.436+0000 7f676c1af700 1 Processor -- start 2024-02-19T18:53:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.436+0000 7f676c1af700 1 -- start start 2024-02-19T18:53:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.436+0000 7f676c1af700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67640fed20 0x7f67640fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.436+0000 7f676c1af700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67640fc7a0 con 0x7f67640fed20 2024-02-19T18:53:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.437+0000 7f6769f4b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67640fed20 0x7f67640fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.437+0000 7f6769f4b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67640fed20 0x7f67640fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:58654/0 (socket says 172.21.15.132:58654) 2024-02-19T18:53:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.437+0000 7f6769f4b700 1 -- 172.21.15.132:0/2841737414 learned_addr learned my addr 172.21.15.132:0/2841737414 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.437+0000 7f6769f4b700 1 -- 172.21.15.132:0/2841737414 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67600095f0 con 0x7f67640fed20 2024-02-19T18:53:23.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.437+0000 7f6769f4b700 1 --2- 172.21.15.132:0/2841737414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67640fed20 0x7f67640fc260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6760005b40 tx=0x7f6760004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.437+0000 7f675affd700 1 -- 172.21.15.132:0/2841737414 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f676001d070 con 0x7f67640fed20 2024-02-19T18:53:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.437+0000 7f676c1af700 1 -- 172.21.15.132:0/2841737414 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67640fa8b0 con 0x7f67640fed20 2024-02-19T18:53:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.437+0000 7f675affd700 1 -- 172.21.15.132:0/2841737414 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f676000be10 con 0x7f67640fed20 2024-02-19T18:53:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.437+0000 7f676c1af700 1 -- 172.21.15.132:0/2841737414 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67640fad50 con 0x7f67640fed20 2024-02-19T18:53:23.456 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.437+0000 7f675affd700 1 -- 172.21.15.132:0/2841737414 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f676000f460 con 0x7f67640fed20 2024-02-19T18:53:23.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.438+0000 7f676c1af700 1 -- 172.21.15.132:0/2841737414 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6764062380 con 0x7f67640fed20 2024-02-19T18:53:23.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.438+0000 7f675affd700 1 -- 172.21.15.132:0/2841737414 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f67600227e0 con 0x7f67640fed20 2024-02-19T18:53:23.457 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.438+0000 7f675affd700 1 --2- 172.21.15.132:0/2841737414 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6750038490 0x7f675003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:23.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.439+0000 7f675affd700 1 -- 172.21.15.132:0/2841737414 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f676004c360 con 0x7f67640fed20 2024-02-19T18:53:23.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.439+0000 7f676974a700 1 --2- 172.21.15.132:0/2841737414 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6750038490 0x7f675003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:23.458 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.439+0000 7f676974a700 1 --2- 172.21.15.132:0/2841737414 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6750038490 0x7f675003a940 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6754006fd0 tx=0x7f6754006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:23.460 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.444+0000 7f675affd700 1 -- 172.21.15.132:0/2841737414 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6760022470 con 0x7f67640fed20 2024-02-19T18:53:23.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:23 smithi088 ceph-mon[29272]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:23.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.705+0000 7f676c1af700 1 -- 172.21.15.132:0/2841737414 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f676418e3f0 con 0x7f67640fed20 2024-02-19T18:53:23.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.706+0000 7f675affd700 1 -- 172.21.15.132:0/2841737414 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6760027030 con 0x7f67640fed20 2024-02-19T18:53:23.722 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:23.722 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:23.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.708+0000 7f676c1af700 1 -- 172.21.15.132:0/2841737414 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6750038490 msgr2=0x7f675003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:23.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.708+0000 7f676c1af700 1 --2- 172.21.15.132:0/2841737414 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6750038490 0x7f675003a940 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6754006fd0 tx=0x7f6754006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:23.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.708+0000 7f676c1af700 1 -- 172.21.15.132:0/2841737414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67640fed20 msgr2=0x7f67640fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:23.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.708+0000 7f676c1af700 1 --2- 172.21.15.132:0/2841737414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67640fed20 0x7f67640fc260 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6760005b40 tx=0x7f6760004740 comp rx=0 tx=0).stop 2024-02-19T18:53:23.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.708+0000 7f676c1af700 1 -- 172.21.15.132:0/2841737414 shutdown_connections 2024-02-19T18:53:23.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.708+0000 7f676c1af700 1 --2- 172.21.15.132:0/2841737414 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6750038490 0x7f675003a940 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:23.724 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.708+0000 7f676c1af700 1 --2- 172.21.15.132:0/2841737414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67640fed20 0x7f67640fc260 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:23.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.708+0000 7f676c1af700 1 -- 172.21.15.132:0/2841737414 >> 172.21.15.132:0/2841737414 conn(0x7f67640fa2b0 msgr2=0x7f67640687e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:23.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.708+0000 7f676c1af700 1 -- 172.21.15.132:0/2841737414 shutdown_connections 2024-02-19T18:53:23.725 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:23.708+0000 7f676c1af700 1 -- 172.21.15.132:0/2841737414 wait complete. 2024-02-19T18:53:23.725 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:24.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:24 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/2841737414' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:25.449 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:25.449 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:25.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:25 smithi088 ceph-mon[29272]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:25.654 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:25.699 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:27.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.159+0000 7fbc2203a700 1 -- 172.21.15.132:0/894922005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc1c0ffe20 msgr2=0x7fbc1c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:27.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.159+0000 7fbc2203a700 1 --2- 172.21.15.132:0/894922005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc1c0ffe20 0x7fbc1c100230 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c009910 tx=0x7fbc0c009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:27.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.159+0000 7fbc2203a700 1 -- 172.21.15.132:0/894922005 shutdown_connections 2024-02-19T18:53:27.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.159+0000 7fbc2203a700 1 --2- 172.21.15.132:0/894922005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc1c0ffe20 0x7fbc1c100230 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:27.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.159+0000 7fbc2203a700 1 -- 172.21.15.132:0/894922005 >> 172.21.15.132:0/894922005 conn(0x7fbc1c0fb3d0 msgr2=0x7fbc1c0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:27.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.159+0000 7fbc2203a700 1 -- 172.21.15.132:0/894922005 shutdown_connections 2024-02-19T18:53:27.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.159+0000 7fbc2203a700 1 -- 172.21.15.132:0/894922005 wait complete. 2024-02-19T18:53:27.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.159+0000 7fbc2203a700 1 Processor -- start 2024-02-19T18:53:27.177 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.159+0000 7fbc2203a700 1 -- start start 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc2203a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc1c0ffe20 0x7fbc1c194f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc2203a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1c1954a0 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc1b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc1c0ffe20 0x7fbc1c194f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc1b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc1c0ffe20 0x7fbc1c194f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:55376/0 (socket says 172.21.15.132:55376) 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc1b7fe700 1 -- 172.21.15.132:0/2896903112 learned_addr learned my addr 172.21.15.132:0/2896903112 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc1b7fe700 1 -- 172.21.15.132:0/2896903112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c0095f0 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc1b7fe700 1 --2- 172.21.15.132:0/2896903112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc1c0ffe20 0x7fbc1c194f60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c005b40 tx=0x7fbc0c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc18ff9700 1 -- 172.21.15.132:0/2896903112 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc0c01d070 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc18ff9700 1 -- 172.21.15.132:0/2896903112 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbc0c00be10 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc2203a700 1 -- 172.21.15.132:0/2896903112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c1956a0 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc18ff9700 1 -- 172.21.15.132:0/2896903112 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc0c00f460 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.178 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.160+0000 7fbc2203a700 1 -- 172.21.15.132:0/2896903112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c195b40 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.161+0000 7fbc18ff9700 1 -- 172.21.15.132:0/2896903112 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fbc0c00b960 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.161+0000 7fbc2203a700 1 -- 172.21.15.132:0/2896903112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c18ea80 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.161+0000 7fbc18ff9700 1 --2- 172.21.15.132:0/2896903112 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc04038490 0x7fbc0403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:27.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.161+0000 7fbc18ff9700 1 -- 172.21.15.132:0/2896903112 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbc0c04c380 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.162+0000 7fbc1affd700 1 --2- 172.21.15.132:0/2896903112 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc04038490 0x7fbc0403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:27.180 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.162+0000 7fbc1affd700 1 --2- 172.21.15.132:0/2896903112 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc04038490 0x7fbc0403a940 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbc10006fd0 tx=0x7fbc10006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:27.183 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.167+0000 7fbc18ff9700 1 -- 172.21.15.132:0/2896903112 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbc0c02bba0 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.450 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.434+0000 7fbc2203a700 1 -- 172.21.15.132:0/2896903112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbc1c02cc30 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.435+0000 7fbc18ff9700 1 -- 172.21.15.132:0/2896903112 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fbc0c027020 con 0x7fbc1c0ffe20 2024-02-19T18:53:27.451 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:27.451 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.437+0000 7fbc2203a700 1 -- 172.21.15.132:0/2896903112 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc04038490 msgr2=0x7fbc0403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.437+0000 7fbc2203a700 1 --2- 172.21.15.132:0/2896903112 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc04038490 0x7fbc0403a940 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbc10006fd0 tx=0x7fbc10006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.437+0000 7fbc2203a700 1 -- 172.21.15.132:0/2896903112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc1c0ffe20 msgr2=0x7fbc1c194f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.437+0000 7fbc2203a700 1 --2- 172.21.15.132:0/2896903112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc1c0ffe20 0x7fbc1c194f60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c005b40 tx=0x7fbc0c004740 comp rx=0 tx=0).stop 2024-02-19T18:53:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.437+0000 7fbc2203a700 1 -- 172.21.15.132:0/2896903112 shutdown_connections 2024-02-19T18:53:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.437+0000 7fbc2203a700 1 --2- 172.21.15.132:0/2896903112 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7fbc04038490 0x7fbc0403a940 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.437+0000 7fbc2203a700 1 --2- 172.21.15.132:0/2896903112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc1c0ffe20 0x7fbc1c194f60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.437+0000 7fbc2203a700 1 -- 172.21.15.132:0/2896903112 >> 172.21.15.132:0/2896903112 conn(0x7fbc1c0fb3d0 msgr2=0x7fbc1c0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:27.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.437+0000 7fbc2203a700 1 -- 172.21.15.132:0/2896903112 shutdown_connections 2024-02-19T18:53:27.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:27.437+0000 7fbc2203a700 1 -- 172.21.15.132:0/2896903112 wait complete. 2024-02-19T18:53:27.454 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:27.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:27 smithi088 ceph-mon[29272]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:28.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:28 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/2896903112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:29.025 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:29.025 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:29.158 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:29 smithi088 ceph-mon[29272]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:29.230 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:29.280 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:30.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.759+0000 7efd84133700 1 -- 172.21.15.132:0/1388099905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7c0fed20 msgr2=0x7efd7c0ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:30.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.759+0000 7efd84133700 1 --2- 172.21.15.132:0/1388099905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7c0fed20 0x7efd7c0ff130 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efd78009910 tx=0x7efd78009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:30.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.759+0000 7efd84133700 1 -- 172.21.15.132:0/1388099905 shutdown_connections 2024-02-19T18:53:30.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.759+0000 7efd84133700 1 --2- 172.21.15.132:0/1388099905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7c0fed20 0x7efd7c0ff130 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:30.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.759+0000 7efd84133700 1 -- 172.21.15.132:0/1388099905 >> 172.21.15.132:0/1388099905 conn(0x7efd7c0fa2b0 msgr2=0x7efd7c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:30.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.760+0000 7efd84133700 1 -- 172.21.15.132:0/1388099905 shutdown_connections 2024-02-19T18:53:30.777 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.760+0000 7efd84133700 1 -- 172.21.15.132:0/1388099905 wait complete. 2024-02-19T18:53:30.779 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.760+0000 7efd84133700 1 Processor -- start 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.760+0000 7efd84133700 1 -- start start 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.760+0000 7efd84133700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7c0fed20 0x7efd7c194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.760+0000 7efd84133700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c1954f0 con 0x7efd7c0fed20 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.761+0000 7efd81ecf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7c0fed20 0x7efd7c194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.761+0000 7efd81ecf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7c0fed20 0x7efd7c194fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:55400/0 (socket says 172.21.15.132:55400) 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.761+0000 7efd81ecf700 1 -- 172.21.15.132:0/3628397065 learned_addr learned my addr 172.21.15.132:0/3628397065 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.761+0000 7efd81ecf700 1 -- 172.21.15.132:0/3628397065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd780095f0 con 0x7efd7c0fed20 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.761+0000 7efd81ecf700 1 --2- 172.21.15.132:0/3628397065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7c0fed20 0x7efd7c194fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efd78005950 tx=0x7efd78009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.761+0000 7efd72ffd700 1 -- 172.21.15.132:0/3628397065 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd7801d070 con 0x7efd7c0fed20 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.761+0000 7efd84133700 1 -- 172.21.15.132:0/3628397065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c1956f0 con 0x7efd7c0fed20 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.761+0000 7efd72ffd700 1 -- 172.21.15.132:0/3628397065 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efd7800be10 con 0x7efd7c0fed20 2024-02-19T18:53:30.780 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.762+0000 7efd72ffd700 1 -- 172.21.15.132:0/3628397065 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efd7800f460 con 0x7efd7c0fed20 2024-02-19T18:53:30.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.762+0000 7efd84133700 1 -- 172.21.15.132:0/3628397065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c195b90 con 0x7efd7c0fed20 2024-02-19T18:53:30.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.762+0000 7efd84133700 1 -- 172.21.15.132:0/3628397065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7c18ea10 con 0x7efd7c0fed20 2024-02-19T18:53:30.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.762+0000 7efd72ffd700 1 -- 172.21.15.132:0/3628397065 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7efd7800b960 con 0x7efd7c0fed20 2024-02-19T18:53:30.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.763+0000 7efd72ffd700 1 --2- 172.21.15.132:0/3628397065 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efd68038490 0x7efd6803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:30.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.763+0000 7efd72ffd700 1 -- 172.21.15.132:0/3628397065 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efd7804c430 con 0x7efd7c0fed20 2024-02-19T18:53:30.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.763+0000 7efd816ce700 1 --2- 172.21.15.132:0/3628397065 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efd68038490 0x7efd6803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:30.781 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.763+0000 7efd816ce700 1 --2- 172.21.15.132:0/3628397065 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efd68038490 0x7efd6803a940 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd6c006fd0 tx=0x7efd6c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:30.784 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:30.768+0000 7efd72ffd700 1 -- 172.21.15.132:0/3628397065 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efd78013250 con 0x7efd7c0fed20 2024-02-19T18:53:31.049 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.032+0000 7efd84133700 1 -- 172.21.15.132:0/3628397065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7efd7c02cc30 con 0x7efd7c0fed20 2024-02-19T18:53:31.050 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.033+0000 7efd72ffd700 1 -- 172.21.15.132:0/3628397065 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7efd78027030 con 0x7efd7c0fed20 2024-02-19T18:53:31.050 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:31.050 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.035+0000 7efd84133700 1 -- 172.21.15.132:0/3628397065 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efd68038490 msgr2=0x7efd6803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.035+0000 7efd84133700 1 --2- 172.21.15.132:0/3628397065 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efd68038490 0x7efd6803a940 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efd6c006fd0 tx=0x7efd6c006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.035+0000 7efd84133700 1 -- 172.21.15.132:0/3628397065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7c0fed20 msgr2=0x7efd7c194fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.035+0000 7efd84133700 1 --2- 172.21.15.132:0/3628397065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7c0fed20 0x7efd7c194fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efd78005950 tx=0x7efd78009e60 comp rx=0 tx=0).stop 2024-02-19T18:53:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.035+0000 7efd84133700 1 -- 172.21.15.132:0/3628397065 shutdown_connections 2024-02-19T18:53:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.035+0000 7efd84133700 1 --2- 172.21.15.132:0/3628397065 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7efd68038490 0x7efd6803a940 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.035+0000 7efd84133700 1 --2- 172.21.15.132:0/3628397065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd7c0fed20 0x7efd7c194fb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.035+0000 7efd84133700 1 -- 172.21.15.132:0/3628397065 >> 172.21.15.132:0/3628397065 conn(0x7efd7c0fa2b0 msgr2=0x7efd7c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.035+0000 7efd84133700 1 -- 172.21.15.132:0/3628397065 shutdown_connections 2024-02-19T18:53:31.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:31.035+0000 7efd84133700 1 -- 172.21.15.132:0/3628397065 wait complete. 2024-02-19T18:53:31.053 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:31.528 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:31 smithi088 ceph-mon[29272]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:31.528 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:31 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/3628397065' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:32.648 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:32.648 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:32.850 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:32.896 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:33.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:32 smithi088 ceph-mon[29272]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:53:33.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:32 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:33.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:32 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:33.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:32 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:33.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:32 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-19T18:53:34.211 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:33 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:34.212 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:33 smithi088 ceph-mon[29272]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-19T18:53:34.212 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:33 smithi088 ceph-mon[29272]: mgrmap e14: smithi088.gyviqq(active, since 115s) 2024-02-19T18:53:34.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.380+0000 7f4120bfe700 1 -- 172.21.15.132:0/713360061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411c0fed20 msgr2=0x7f411c0ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:34.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.380+0000 7f4120bfe700 1 --2- 172.21.15.132:0/713360061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411c0fed20 0x7f411c0ff130 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4114009910 tx=0x7f4114009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:34.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.380+0000 7f4120bfe700 1 -- 172.21.15.132:0/713360061 shutdown_connections 2024-02-19T18:53:34.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.380+0000 7f4120bfe700 1 --2- 172.21.15.132:0/713360061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411c0fed20 0x7f411c0ff130 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:34.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.380+0000 7f4120bfe700 1 -- 172.21.15.132:0/713360061 >> 172.21.15.132:0/713360061 conn(0x7f411c0fa2b0 msgr2=0x7f411c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:34.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.380+0000 7f4120bfe700 1 -- 172.21.15.132:0/713360061 shutdown_connections 2024-02-19T18:53:34.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.380+0000 7f4120bfe700 1 -- 172.21.15.132:0/713360061 wait complete. 2024-02-19T18:53:34.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.381+0000 7f4120bfe700 1 Processor -- start 2024-02-19T18:53:34.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.381+0000 7f4120bfe700 1 -- start start 2024-02-19T18:53:34.401 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.381+0000 7f4120bfe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411c0fed20 0x7f411c194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.381+0000 7f4120bfe700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f411c1954f0 con 0x7f411c0fed20 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.382+0000 7f411a59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411c0fed20 0x7f411c194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.382+0000 7f411a59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411c0fed20 0x7f411c194fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:53854/0 (socket says 172.21.15.132:53854) 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.382+0000 7f411a59c700 1 -- 172.21.15.132:0/4101917898 learned_addr learned my addr 172.21.15.132:0/4101917898 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.382+0000 7f411a59c700 1 -- 172.21.15.132:0/4101917898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140095f0 con 0x7f411c0fed20 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.382+0000 7f411a59c700 1 --2- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411c0fed20 0x7f411c194fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4114005b40 tx=0x7f4114004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.382+0000 7f41137fe700 1 -- 172.21.15.132:0/4101917898 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f411401d070 con 0x7f411c0fed20 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.382+0000 7f41137fe700 1 -- 172.21.15.132:0/4101917898 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f411400be10 con 0x7f411c0fed20 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.382+0000 7f41137fe700 1 -- 172.21.15.132:0/4101917898 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f411400f460 con 0x7f411c0fed20 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.382+0000 7f4120bfe700 1 -- 172.21.15.132:0/4101917898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f411c1956f0 con 0x7f411c0fed20 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.382+0000 7f4120bfe700 1 -- 172.21.15.132:0/4101917898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f411c195b90 con 0x7f411c0fed20 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.383+0000 7f4120bfe700 1 -- 172.21.15.132:0/4101917898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f411c18ea10 con 0x7f411c0fed20 2024-02-19T18:53:34.402 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.383+0000 7f41137fe700 1 -- 172.21.15.132:0/4101917898 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 14) v1 ==== 45154+0+0 (secure 0 0 0) 0x7f41140227e0 con 0x7f411c0fed20 2024-02-19T18:53:34.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.384+0000 7f41137fe700 1 --2- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f41080384e0 0x7f410803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:34.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.384+0000 7f41137fe700 1 -- 172.21.15.132:0/4101917898 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f411404c4a0 con 0x7f411c0fed20 2024-02-19T18:53:34.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.384+0000 7f4119d9b700 1 -- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f41080384e0 msgr2=0x7f410803a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/3043610921 2024-02-19T18:53:34.403 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.384+0000 7f4119d9b700 1 --2- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f41080384e0 0x7f410803a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T18:53:34.405 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.389+0000 7f41137fe700 1 -- 172.21.15.132:0/4101917898 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f411402bba0 con 0x7f411c0fed20 2024-02-19T18:53:34.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.585+0000 7f4119d9b700 1 -- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f41080384e0 msgr2=0x7f410803a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/3043610921 2024-02-19T18:53:34.602 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.585+0000 7f4119d9b700 1 --2- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f41080384e0 0x7f410803a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-19T18:53:34.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.653+0000 7f4120bfe700 1 -- 172.21.15.132:0/4101917898 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f411c02cc30 con 0x7f411c0fed20 2024-02-19T18:53:34.670 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.653+0000 7f41137fe700 1 -- 172.21.15.132:0/4101917898 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f4114027030 con 0x7f411c0fed20 2024-02-19T18:53:34.670 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:34.671 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:34.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.655+0000 7f4120bfe700 1 -- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f41080384e0 msgr2=0x7f410803a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T18:53:34.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.655+0000 7f4120bfe700 1 --2- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f41080384e0 0x7f410803a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:34.672 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.655+0000 7f4120bfe700 1 -- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411c0fed20 msgr2=0x7f411c194fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:34.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.655+0000 7f4120bfe700 1 --2- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411c0fed20 0x7f411c194fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4114005b40 tx=0x7f4114004740 comp rx=0 tx=0).stop 2024-02-19T18:53:34.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.655+0000 7f4120bfe700 1 -- 172.21.15.132:0/4101917898 shutdown_connections 2024-02-19T18:53:34.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.655+0000 7f4120bfe700 1 --2- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f41080384e0 0x7f410803a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:34.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.655+0000 7f4120bfe700 1 --2- 172.21.15.132:0/4101917898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f411c0fed20 0x7f411c194fb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:34.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.655+0000 7f4120bfe700 1 -- 172.21.15.132:0/4101917898 >> 172.21.15.132:0/4101917898 conn(0x7f411c0fa2b0 msgr2=0x7f411c0690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:34.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.655+0000 7f4120bfe700 1 -- 172.21.15.132:0/4101917898 shutdown_connections 2024-02-19T18:53:34.673 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:34.655+0000 7f4120bfe700 1 -- 172.21.15.132:0/4101917898 wait complete. 2024-02-19T18:53:34.673 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:35.196 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:34 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/4101917898' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:36.227 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:36.227 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:36.431 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:36.476 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:37.987 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.970+0000 7f6b44e85700 1 -- 172.21.15.132:0/2868510937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b401008b0 msgr2=0x7f6b40102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:37.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.970+0000 7f6b44e85700 1 --2- 172.21.15.132:0/2868510937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b401008b0 0x7f6b40102c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6b30009910 tx=0x7f6b30009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:37.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.970+0000 7f6b44e85700 1 -- 172.21.15.132:0/2868510937 shutdown_connections 2024-02-19T18:53:37.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.970+0000 7f6b44e85700 1 --2- 172.21.15.132:0/2868510937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b401008b0 0x7f6b40102c90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:37.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.970+0000 7f6b44e85700 1 -- 172.21.15.132:0/2868510937 >> 172.21.15.132:0/2868510937 conn(0x7f6b400fa2b0 msgr2=0x7f6b400fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:37.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.970+0000 7f6b44e85700 1 -- 172.21.15.132:0/2868510937 shutdown_connections 2024-02-19T18:53:37.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.970+0000 7f6b44e85700 1 -- 172.21.15.132:0/2868510937 wait complete. 2024-02-19T18:53:37.988 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.971+0000 7f6b44e85700 1 Processor -- start 2024-02-19T18:53:37.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.971+0000 7f6b44e85700 1 -- start start 2024-02-19T18:53:37.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.971+0000 7f6b44e85700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b401008b0 0x7f6b40194fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:37.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.971+0000 7f6b44e85700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b401954e0 con 0x7f6b401008b0 2024-02-19T18:53:37.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.971+0000 7f6b3e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b401008b0 0x7f6b40194fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:37.990 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.971+0000 7f6b3e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b401008b0 0x7f6b40194fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:53886/0 (socket says 172.21.15.132:53886) 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.971+0000 7f6b3e59c700 1 -- 172.21.15.132:0/3087612972 learned_addr learned my addr 172.21.15.132:0/3087612972 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.971+0000 7f6b3e59c700 1 -- 172.21.15.132:0/3087612972 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b300095f0 con 0x7f6b401008b0 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.972+0000 7f6b3e59c700 1 --2- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b401008b0 0x7f6b40194fa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6b30005b40 tx=0x7f6b30004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.972+0000 7f6b37fff700 1 -- 172.21.15.132:0/3087612972 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b3001d070 con 0x7f6b401008b0 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.972+0000 7f6b37fff700 1 -- 172.21.15.132:0/3087612972 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6b3000be10 con 0x7f6b401008b0 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.972+0000 7f6b37fff700 1 -- 172.21.15.132:0/3087612972 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6b3000f460 con 0x7f6b401008b0 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.972+0000 7f6b44e85700 1 -- 172.21.15.132:0/3087612972 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b401956e0 con 0x7f6b401008b0 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.972+0000 7f6b44e85700 1 -- 172.21.15.132:0/3087612972 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b40195b80 con 0x7f6b401008b0 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.973+0000 7f6b37fff700 1 -- 172.21.15.132:0/3087612972 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 14) v1 ==== 45154+0+0 (secure 0 0 0) 0x7f6b3000b960 con 0x7f6b401008b0 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.973+0000 7f6b44e85700 1 -- 172.21.15.132:0/3087612972 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4018eb00 con 0x7f6b401008b0 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.973+0000 7f6b37fff700 1 --2- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6b2c038530 0x7f6b2c03a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.973+0000 7f6b37fff700 1 -- 172.21.15.132:0/3087612972 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6b3004c4c0 con 0x7f6b401008b0 2024-02-19T18:53:37.991 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.973+0000 7f6b35d9b700 1 -- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6b2c038530 msgr2=0x7f6b2c03a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/3043610921 2024-02-19T18:53:37.992 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.973+0000 7f6b35d9b700 1 --2- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6b2c038530 0x7f6b2c03a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T18:53:37.995 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:37.978+0000 7f6b37fff700 1 -- 172.21.15.132:0/3087612972 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6b3002bb60 con 0x7f6b401008b0 2024-02-19T18:53:38.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.174+0000 7f6b35d9b700 1 -- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6b2c038530 msgr2=0x7f6b2c03a9e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.88:6800/3043610921 2024-02-19T18:53:38.191 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.174+0000 7f6b35d9b700 1 --2- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6b2c038530 0x7f6b2c03a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-19T18:53:38.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.242+0000 7f6b44e85700 1 -- 172.21.15.132:0/3087612972 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6b4002cc30 con 0x7f6b401008b0 2024-02-19T18:53:38.259 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.242+0000 7f6b37fff700 1 -- 172.21.15.132:0/3087612972 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6b30027070 con 0x7f6b401008b0 2024-02-19T18:53:38.260 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:38.260 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:38.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.245+0000 7f6b44e85700 1 -- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6b2c038530 msgr2=0x7f6b2c03a9e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T18:53:38.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.245+0000 7f6b44e85700 1 --2- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6b2c038530 0x7f6b2c03a9e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:38.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.245+0000 7f6b44e85700 1 -- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b401008b0 msgr2=0x7f6b40194fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:38.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.245+0000 7f6b44e85700 1 --2- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b401008b0 0x7f6b40194fa0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6b30005b40 tx=0x7f6b30004740 comp rx=0 tx=0).stop 2024-02-19T18:53:38.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.245+0000 7f6b44e85700 1 -- 172.21.15.132:0/3087612972 shutdown_connections 2024-02-19T18:53:38.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.245+0000 7f6b44e85700 1 --2- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:6800/3043610921,v1:172.21.15.88:6801/3043610921] conn(0x7f6b2c038530 0x7f6b2c03a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:38.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.245+0000 7f6b44e85700 1 --2- 172.21.15.132:0/3087612972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b401008b0 0x7f6b40194fa0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:38.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.245+0000 7f6b44e85700 1 -- 172.21.15.132:0/3087612972 >> 172.21.15.132:0/3087612972 conn(0x7f6b400fa2b0 msgr2=0x7f6b400faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:38.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.245+0000 7f6b44e85700 1 -- 172.21.15.132:0/3087612972 shutdown_connections 2024-02-19T18:53:38.262 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:38.245+0000 7f6b44e85700 1 -- 172.21.15.132:0/3087612972 wait complete. 2024-02-19T18:53:38.263 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:38.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:38 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/3087612972' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:39.856 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:39.857 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:40.066 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:40.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: Active manager daemon smithi088.gyviqq restarted 2024-02-19T18:53:40.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: Activating manager daemon smithi088.gyviqq 2024-02-19T18:53:40.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: osdmap e5: 0 total, 0 up, 0 in 2024-02-19T18:53:40.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: mgrmap e15: smithi088.gyviqq(active, starting, since 0.00354374s) 2024-02-19T18:53:40.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T18:53:40.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi088.gyviqq", "id": "smithi088.gyviqq"}]: dispatch 2024-02-19T18:53:40.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T18:53:40.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T18:53:40.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T18:53:40.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: Manager daemon smithi088.gyviqq is now available 2024-02-19T18:53:40.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:53:40.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:39 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:53:40.118 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:40.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:40.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:53:40.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/mirror_snapshot_schedule"}]: dispatch 2024-02-19T18:53:40.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/trash_purge_schedule"}]: dispatch 2024-02-19T18:53:40.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:40 smithi088 ceph-mon[29272]: [19/Feb/2024:18:53:39] ENGINE Bus STARTING 2024-02-19T18:53:40.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:40.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:40 smithi088 ceph-mon[29272]: mgrmap e16: smithi088.gyviqq(active, since 1.00535s) 2024-02-19T18:53:41.652 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.635+0000 7f96c6fbc700 1 -- 172.21.15.132:0/2206596150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c00fe100 msgr2=0x7f96c00fe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:41.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.635+0000 7f96c6fbc700 1 --2- 172.21.15.132:0/2206596150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c00fe100 0x7f96c00fe510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009910 tx=0x7f96b4009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:41.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.635+0000 7f96c6fbc700 1 -- 172.21.15.132:0/2206596150 shutdown_connections 2024-02-19T18:53:41.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.635+0000 7f96c6fbc700 1 --2- 172.21.15.132:0/2206596150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c00fe100 0x7f96c00fe510 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:41.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.635+0000 7f96c6fbc700 1 -- 172.21.15.132:0/2206596150 >> 172.21.15.132:0/2206596150 conn(0x7f96c00f9a50 msgr2=0x7f96c00fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:41.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.635+0000 7f96c6fbc700 1 -- 172.21.15.132:0/2206596150 shutdown_connections 2024-02-19T18:53:41.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.635+0000 7f96c6fbc700 1 -- 172.21.15.132:0/2206596150 wait complete. 2024-02-19T18:53:41.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.635+0000 7f96c6fbc700 1 Processor -- start 2024-02-19T18:53:41.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.636+0000 7f96c6fbc700 1 -- start start 2024-02-19T18:53:41.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.636+0000 7f96c6fbc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c00fe100 0x7f96c0197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:41.653 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.636+0000 7f96c6fbc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96c01976d0 con 0x7f96c00fe100 2024-02-19T18:53:41.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.636+0000 7f96c4d58700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c00fe100 0x7f96c0197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:41.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.636+0000 7f96c4d58700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c00fe100 0x7f96c0197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:53900/0 (socket says 172.21.15.132:53900) 2024-02-19T18:53:41.654 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.636+0000 7f96c4d58700 1 -- 172.21.15.132:0/685083041 learned_addr learned my addr 172.21.15.132:0/685083041 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:41.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.636+0000 7f96c4d58700 1 -- 172.21.15.132:0/685083041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b40095f0 con 0x7f96c00fe100 2024-02-19T18:53:41.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.637+0000 7f96c4d58700 1 --2- 172.21.15.132:0/685083041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c00fe100 0x7f96c0197190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f96b4005b40 tx=0x7f96b4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:41.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.637+0000 7f96bdffb700 1 -- 172.21.15.132:0/685083041 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96b401d070 con 0x7f96c00fe100 2024-02-19T18:53:41.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.637+0000 7f96c6fbc700 1 -- 172.21.15.132:0/685083041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c01978d0 con 0x7f96c00fe100 2024-02-19T18:53:41.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.637+0000 7f96bdffb700 1 -- 172.21.15.132:0/685083041 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f96b400be10 con 0x7f96c00fe100 2024-02-19T18:53:41.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.637+0000 7f96bdffb700 1 -- 172.21.15.132:0/685083041 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96b400f460 con 0x7f96c00fe100 2024-02-19T18:53:41.655 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.637+0000 7f96c6fbc700 1 -- 172.21.15.132:0/685083041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96c0197d70 con 0x7f96c00fe100 2024-02-19T18:53:41.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.638+0000 7f96c6fbc700 1 -- 172.21.15.132:0/685083041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96c0190f10 con 0x7f96c00fe100 2024-02-19T18:53:41.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.638+0000 7f96bdffb700 1 -- 172.21.15.132:0/685083041 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) v1 ==== 45005+0+0 (secure 0 0 0) 0x7f96b400f5c0 con 0x7f96c00fe100 2024-02-19T18:53:41.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.638+0000 7f96bdffb700 1 --2- 172.21.15.132:0/685083041 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f96b0038380 0x7f96b003a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:41.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.638+0000 7f96bdffb700 1 -- 172.21.15.132:0/685083041 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f96b404c200 con 0x7f96c00fe100 2024-02-19T18:53:41.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.638+0000 7f96bffff700 1 --2- 172.21.15.132:0/685083041 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f96b0038380 0x7f96b003a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:41.656 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.639+0000 7f96bffff700 1 --2- 172.21.15.132:0/685083041 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f96b0038380 0x7f96b003a830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f96ac006fd0 tx=0x7f96ac006e40 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:41.661 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.644+0000 7f96bdffb700 1 -- 172.21.15.132:0/685083041 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f96b402a360 con 0x7f96c00fe100 2024-02-19T18:53:41.809 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.792+0000 7f96bdffb700 1 -- 172.21.15.132:0/685083041 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f96b40227e0 con 0x7f96c00fe100 2024-02-19T18:53:41.940 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.923+0000 7f96c6fbc700 1 -- 172.21.15.132:0/685083041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f96c002cc30 con 0x7f96c00fe100 2024-02-19T18:53:41.941 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.923+0000 7f96bdffb700 1 -- 172.21.15.132:0/685083041 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f96b4027030 con 0x7f96c00fe100 2024-02-19T18:53:41.941 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:41.941 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:41.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.926+0000 7f96c6fbc700 1 -- 172.21.15.132:0/685083041 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f96b0038380 msgr2=0x7f96b003a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:41.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.926+0000 7f96c6fbc700 1 --2- 172.21.15.132:0/685083041 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f96b0038380 0x7f96b003a830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f96ac006fd0 tx=0x7f96ac006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:41.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.926+0000 7f96c6fbc700 1 -- 172.21.15.132:0/685083041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c00fe100 msgr2=0x7f96c0197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:41.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.926+0000 7f96c6fbc700 1 --2- 172.21.15.132:0/685083041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c00fe100 0x7f96c0197190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f96b4005b40 tx=0x7f96b4004740 comp rx=0 tx=0).stop 2024-02-19T18:53:41.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.926+0000 7f96c6fbc700 1 -- 172.21.15.132:0/685083041 shutdown_connections 2024-02-19T18:53:41.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.926+0000 7f96c6fbc700 1 --2- 172.21.15.132:0/685083041 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f96b0038380 0x7f96b003a830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:41.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.926+0000 7f96c6fbc700 1 --2- 172.21.15.132:0/685083041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c00fe100 0x7f96c0197190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:41.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.926+0000 7f96c6fbc700 1 -- 172.21.15.132:0/685083041 >> 172.21.15.132:0/685083041 conn(0x7f96c00f9a50 msgr2=0x7f96c00fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:41.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.926+0000 7f96c6fbc700 1 -- 172.21.15.132:0/685083041 shutdown_connections 2024-02-19T18:53:41.943 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:41.926+0000 7f96c6fbc700 1 -- 172.21.15.132:0/685083041 wait complete. 2024-02-19T18:53:41.944 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:42.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:41 smithi088 ceph-mon[29272]: [19/Feb/2024:18:53:40] ENGINE Serving on http://172.21.15.88:8765 2024-02-19T18:53:42.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:41 smithi088 ceph-mon[29272]: [19/Feb/2024:18:53:40] ENGINE Serving on https://172.21.15.88:7150 2024-02-19T18:53:42.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:41 smithi088 ceph-mon[29272]: [19/Feb/2024:18:53:40] ENGINE Bus STARTED 2024-02-19T18:53:42.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:41 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:42.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:41 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:43.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:42 smithi088 ceph-mon[29272]: mgrmap e17: smithi088.gyviqq(active, since 2s) 2024-02-19T18:53:43.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:42 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/685083041' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:43.542 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:43.542 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:44.000 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:44.055 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /etc/ceph/ceph.conf 2024-02-19T18:53:44.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:44.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:44.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:44.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:44.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd/host:smithi088", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:53:46.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.153+0000 7f702e487700 1 -- 172.21.15.132:0/2207302266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70280fed20 msgr2=0x7f70280ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:46.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.153+0000 7f702e487700 1 --2- 172.21.15.132:0/2207302266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70280fed20 0x7f70280ff130 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7018009910 tx=0x7f7018009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:46.169 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.153+0000 7f702e487700 1 -- 172.21.15.132:0/2207302266 shutdown_connections 2024-02-19T18:53:46.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.153+0000 7f702e487700 1 --2- 172.21.15.132:0/2207302266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70280fed20 0x7f70280ff130 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:46.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.153+0000 7f702e487700 1 -- 172.21.15.132:0/2207302266 >> 172.21.15.132:0/2207302266 conn(0x7f70280fa2b0 msgr2=0x7f70280fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:46.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.153+0000 7f702e487700 1 -- 172.21.15.132:0/2207302266 shutdown_connections 2024-02-19T18:53:46.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.153+0000 7f702e487700 1 -- 172.21.15.132:0/2207302266 wait complete. 2024-02-19T18:53:46.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.153+0000 7f702e487700 1 Processor -- start 2024-02-19T18:53:46.170 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.153+0000 7f702e487700 1 -- start start 2024-02-19T18:53:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.154+0000 7f702e487700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70280fed20 0x7f7028068b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.154+0000 7f702e487700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7028197cd0 con 0x7f70280fed20 2024-02-19T18:53:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.154+0000 7f7027fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70280fed20 0x7f7028068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.154+0000 7f7027fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70280fed20 0x7f7028068b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:52928/0 (socket says 172.21.15.132:52928) 2024-02-19T18:53:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.154+0000 7f7027fff700 1 -- 172.21.15.132:0/1274573815 learned_addr learned my addr 172.21.15.132:0/1274573815 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.154+0000 7f7027fff700 1 -- 172.21.15.132:0/1274573815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70180095f0 con 0x7f70280fed20 2024-02-19T18:53:46.172 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.154+0000 7f7027fff700 1 --2- 172.21.15.132:0/1274573815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70280fed20 0x7f7028068b10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7018005b40 tx=0x7f7018004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:46.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.154+0000 7f70257fa700 1 -- 172.21.15.132:0/1274573815 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f701801d070 con 0x7f70280fed20 2024-02-19T18:53:46.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.155+0000 7f702e487700 1 -- 172.21.15.132:0/1274573815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7028197e10 con 0x7f70280fed20 2024-02-19T18:53:46.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.155+0000 7f70257fa700 1 -- 172.21.15.132:0/1274573815 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f701800be10 con 0x7f70280fed20 2024-02-19T18:53:46.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.155+0000 7f70257fa700 1 -- 172.21.15.132:0/1274573815 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f701800f460 con 0x7f70280fed20 2024-02-19T18:53:46.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.155+0000 7f702e487700 1 -- 172.21.15.132:0/1274573815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70281982b0 con 0x7f70280fed20 2024-02-19T18:53:46.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.155+0000 7f70257fa700 1 -- 172.21.15.132:0/1274573815 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f701800f6d0 con 0x7f70280fed20 2024-02-19T18:53:46.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.156+0000 7f702e487700 1 -- 172.21.15.132:0/1274573815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f702818da00 con 0x7f70280fed20 2024-02-19T18:53:46.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.156+0000 7f70257fa700 1 --2- 172.21.15.132:0/1274573815 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7010038580 0x7f701003aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:46.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.156+0000 7f70257fa700 1 -- 172.21.15.132:0/1274573815 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f701804c6c0 con 0x7f70280fed20 2024-02-19T18:53:46.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.156+0000 7f70277fe700 1 --2- 172.21.15.132:0/1274573815 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7010038580 0x7f701003aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:46.173 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.156+0000 7f70277fe700 1 --2- 172.21.15.132:0/1274573815 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7010038580 0x7f701003aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f701c006fd0 tx=0x7f701c006e40 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:46.179 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.163+0000 7f70257fa700 1 -- 172.21.15.132:0/1274573815 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f701802a360 con 0x7f70280fed20 2024-02-19T18:53:46.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.445+0000 7f702e487700 1 -- 172.21.15.132:0/1274573815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f702802cc30 con 0x7f70280fed20 2024-02-19T18:53:46.461 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.445+0000 7f70257fa700 1 -- 172.21.15.132:0/1274573815 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f7018027030 con 0x7f70280fed20 2024-02-19T18:53:46.461 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:46.462 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:46.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.447+0000 7f702e487700 1 -- 172.21.15.132:0/1274573815 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7010038580 msgr2=0x7f701003aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:46.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.447+0000 7f702e487700 1 --2- 172.21.15.132:0/1274573815 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7010038580 0x7f701003aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f701c006fd0 tx=0x7f701c006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:46.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.447+0000 7f702e487700 1 -- 172.21.15.132:0/1274573815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70280fed20 msgr2=0x7f7028068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:46.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.447+0000 7f702e487700 1 --2- 172.21.15.132:0/1274573815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70280fed20 0x7f7028068b10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7018005b40 tx=0x7f7018004740 comp rx=0 tx=0).stop 2024-02-19T18:53:46.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.448+0000 7f702e487700 1 -- 172.21.15.132:0/1274573815 shutdown_connections 2024-02-19T18:53:46.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.448+0000 7f702e487700 1 --2- 172.21.15.132:0/1274573815 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7010038580 0x7f701003aa30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:46.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.448+0000 7f702e487700 1 --2- 172.21.15.132:0/1274573815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70280fed20 0x7f7028068b10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:46.464 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.448+0000 7f702e487700 1 -- 172.21.15.132:0/1274573815 >> 172.21.15.132:0/1274573815 conn(0x7f70280fa2b0 msgr2=0x7f70280faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:46.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.448+0000 7f702e487700 1 -- 172.21.15.132:0/1274573815 shutdown_connections 2024-02-19T18:53:46.465 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:46.448+0000 7f702e487700 1 -- 172.21.15.132:0/1274573815 wait complete. 2024-02-19T18:53:46.465 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:47.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:47.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:47.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:47.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:47.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd/host:smithi132", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:53:47.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:53:47.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:53:47.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: Updating smithi088:/etc/ceph/ceph.conf 2024-02-19T18:53:47.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: Updating smithi132:/etc/ceph/ceph.conf 2024-02-19T18:53:47.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1274573815' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T18:53:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-19T18:53:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:53:48.118 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:48.119 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:48.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:47 smithi088 ceph-mon[29272]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:53:48.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:47 smithi088 ceph-mon[29272]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:53:48.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:47 smithi088 ceph-mon[29272]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-02-19T18:53:48.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:47 smithi088 ceph-mon[29272]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-02-19T18:53:48.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:47 smithi088 ceph-mon[29272]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T18:53:48.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:47 smithi088 ceph-mon[29272]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T18:53:48.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:47 smithi088 ceph-mon[29272]: Deploying daemon ceph-exporter.smithi132 on smithi132 2024-02-19T18:53:48.348 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:53:50.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.133+0000 7f7be246d700 1 -- 172.21.15.132:0/2343015296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bdc0fef90 msgr2=0x7f7bdc0ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:50.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.133+0000 7f7be246d700 1 --2- 172.21.15.132:0/2343015296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bdc0fef90 0x7f7bdc0ff360 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009910 tx=0x7f7bcc009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:50.150 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.133+0000 7f7be246d700 1 -- 172.21.15.132:0/2343015296 shutdown_connections 2024-02-19T18:53:50.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.133+0000 7f7be246d700 1 --2- 172.21.15.132:0/2343015296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bdc0fef90 0x7f7bdc0ff360 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:50.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.133+0000 7f7be246d700 1 -- 172.21.15.132:0/2343015296 >> 172.21.15.132:0/2343015296 conn(0x7f7bdc0fa850 msgr2=0x7f7bdc0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:50.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.133+0000 7f7be246d700 1 -- 172.21.15.132:0/2343015296 shutdown_connections 2024-02-19T18:53:50.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.133+0000 7f7be246d700 1 -- 172.21.15.132:0/2343015296 wait complete. 2024-02-19T18:53:50.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.133+0000 7f7be246d700 1 Processor -- start 2024-02-19T18:53:50.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.134+0000 7f7be246d700 1 -- start start 2024-02-19T18:53:50.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.134+0000 7f7be246d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bdc0fef90 0x7f7bdc194210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:50.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.134+0000 7f7be246d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc194750 con 0x7f7bdc0fef90 2024-02-19T18:53:50.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.134+0000 7f7bdbfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bdc0fef90 0x7f7bdc194210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:50.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.134+0000 7f7bdbfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bdc0fef90 0x7f7bdc194210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:52946/0 (socket says 172.21.15.132:52946) 2024-02-19T18:53:50.151 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.134+0000 7f7bdbfff700 1 -- 172.21.15.132:0/575037105 learned_addr learned my addr 172.21.15.132:0/575037105 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:50.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.135+0000 7f7bdbfff700 1 -- 172.21.15.132:0/575037105 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc0095f0 con 0x7f7bdc0fef90 2024-02-19T18:53:50.152 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.135+0000 7f7bdbfff700 1 --2- 172.21.15.132:0/575037105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bdc0fef90 0x7f7bdc194210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc005b40 tx=0x7f7bcc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:50.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.135+0000 7f7bd97fa700 1 -- 172.21.15.132:0/575037105 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7bcc01d070 con 0x7f7bdc0fef90 2024-02-19T18:53:50.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.135+0000 7f7bd97fa700 1 -- 172.21.15.132:0/575037105 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7bcc0044d0 con 0x7f7bdc0fef90 2024-02-19T18:53:50.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.135+0000 7f7be246d700 1 -- 172.21.15.132:0/575037105 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bdc194950 con 0x7f7bdc0fef90 2024-02-19T18:53:50.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.136+0000 7f7bd97fa700 1 -- 172.21.15.132:0/575037105 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7bcc003930 con 0x7f7bdc0fef90 2024-02-19T18:53:50.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.136+0000 7f7be246d700 1 -- 172.21.15.132:0/575037105 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc194d10 con 0x7f7bdc0fef90 2024-02-19T18:53:50.153 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.137+0000 7f7be246d700 1 -- 172.21.15.132:0/575037105 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bdc062380 con 0x7f7bdc0fef90 2024-02-19T18:53:50.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.137+0000 7f7bd97fa700 1 -- 172.21.15.132:0/575037105 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f7bcc00b960 con 0x7f7bdc0fef90 2024-02-19T18:53:50.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.137+0000 7f7bd97fa700 1 --2- 172.21.15.132:0/575037105 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7bc4038580 0x7f7bc403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:50.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.137+0000 7f7bd97fa700 1 -- 172.21.15.132:0/575037105 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7bcc04c410 con 0x7f7bdc0fef90 2024-02-19T18:53:50.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.137+0000 7f7bdb7fe700 1 --2- 172.21.15.132:0/575037105 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7bc4038580 0x7f7bc403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:50.154 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.138+0000 7f7bdb7fe700 1 --2- 172.21.15.132:0/575037105 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7bc4038580 0x7f7bc403aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0006fd0 tx=0x7f7bd0006e40 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:50.158 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.142+0000 7f7bd97fa700 1 -- 172.21.15.132:0/575037105 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7bcc02a540 con 0x7f7bdc0fef90 2024-02-19T18:53:50.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.435+0000 7f7be246d700 1 -- 172.21.15.132:0/575037105 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7bdc106d30 con 0x7f7bdc0fef90 2024-02-19T18:53:50.452 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.436+0000 7f7bd97fa700 1 -- 172.21.15.132:0/575037105 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f7bcc02a360 con 0x7f7bdc0fef90 2024-02-19T18:53:50.453 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:50.453 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:50.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.439+0000 7f7be246d700 1 -- 172.21.15.132:0/575037105 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7bc4038580 msgr2=0x7f7bc403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:50.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.439+0000 7f7be246d700 1 --2- 172.21.15.132:0/575037105 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7bc4038580 0x7f7bc403aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0006fd0 tx=0x7f7bd0006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:50.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.439+0000 7f7be246d700 1 -- 172.21.15.132:0/575037105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bdc0fef90 msgr2=0x7f7bdc194210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:50.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.439+0000 7f7be246d700 1 --2- 172.21.15.132:0/575037105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bdc0fef90 0x7f7bdc194210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc005b40 tx=0x7f7bcc004c80 comp rx=0 tx=0).stop 2024-02-19T18:53:50.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.439+0000 7f7be246d700 1 -- 172.21.15.132:0/575037105 shutdown_connections 2024-02-19T18:53:50.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.439+0000 7f7be246d700 1 --2- 172.21.15.132:0/575037105 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7bc4038580 0x7f7bc403aa30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:50.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.439+0000 7f7be246d700 1 --2- 172.21.15.132:0/575037105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7bdc0fef90 0x7f7bdc194210 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:50.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.439+0000 7f7be246d700 1 -- 172.21.15.132:0/575037105 >> 172.21.15.132:0/575037105 conn(0x7f7bdc0fa850 msgr2=0x7f7bdc103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:50.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.439+0000 7f7be246d700 1 -- 172.21.15.132:0/575037105 shutdown_connections 2024-02-19T18:53:50.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:50.439+0000 7f7be246d700 1 -- 172.21.15.132:0/575037105 wait complete. 2024-02-19T18:53:50.456 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:50.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:50 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/575037105' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:52.304 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:52.304 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:52.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:52.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:52.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:52.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:52.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi132", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T18:53:52.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi132", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-19T18:53:52.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:53:52.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:51 smithi088 ceph-mon[29272]: Deploying daemon crash.smithi132 on smithi132 2024-02-19T18:53:52.531 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:53:53.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.945+0000 7fdee8105700 1 -- 172.21.15.132:0/3741677237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee00fec90 msgr2=0x7fdee00ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:53.963 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.945+0000 7fdee8105700 1 --2- 172.21.15.132:0/3741677237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee00fec90 0x7fdee00ff060 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fdedc009910 tx=0x7fdedc009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.945+0000 7fdee8105700 1 -- 172.21.15.132:0/3741677237 shutdown_connections 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.945+0000 7fdee8105700 1 --2- 172.21.15.132:0/3741677237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee00fec90 0x7fdee00ff060 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.945+0000 7fdee8105700 1 -- 172.21.15.132:0/3741677237 >> 172.21.15.132:0/3741677237 conn(0x7fdee0075750 msgr2=0x7fdee0075b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.945+0000 7fdee8105700 1 -- 172.21.15.132:0/3741677237 shutdown_connections 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.945+0000 7fdee8105700 1 -- 172.21.15.132:0/3741677237 wait complete. 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.946+0000 7fdee8105700 1 Processor -- start 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.946+0000 7fdee8105700 1 -- start start 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.946+0000 7fdee8105700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee00fec90 0x7fdee0195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.946+0000 7fdee8105700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee0195880 con 0x7fdee00fec90 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.946+0000 7fdee5ea1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee00fec90 0x7fdee0195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.947+0000 7fdee5ea1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee00fec90 0x7fdee0195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:47616/0 (socket says 172.21.15.132:47616) 2024-02-19T18:53:53.964 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.947+0000 7fdee5ea1700 1 -- 172.21.15.132:0/4131692925 learned_addr learned my addr 172.21.15.132:0/4131692925 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:53.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.947+0000 7fdee5ea1700 1 -- 172.21.15.132:0/4131692925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdedc0095f0 con 0x7fdee00fec90 2024-02-19T18:53:53.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.947+0000 7fdee5ea1700 1 --2- 172.21.15.132:0/4131692925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee00fec90 0x7fdee0195340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdedc005b40 tx=0x7fdedc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:53.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.947+0000 7fded6ffd700 1 -- 172.21.15.132:0/4131692925 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdedc01d070 con 0x7fdee00fec90 2024-02-19T18:53:53.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.947+0000 7fded6ffd700 1 -- 172.21.15.132:0/4131692925 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdedc0044d0 con 0x7fdee00fec90 2024-02-19T18:53:53.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.947+0000 7fdee8105700 1 -- 172.21.15.132:0/4131692925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee0195a80 con 0x7fdee00fec90 2024-02-19T18:53:53.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.947+0000 7fded6ffd700 1 -- 172.21.15.132:0/4131692925 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdedc003930 con 0x7fdee00fec90 2024-02-19T18:53:53.965 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.947+0000 7fdee8105700 1 -- 172.21.15.132:0/4131692925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee0195e40 con 0x7fdee00fec90 2024-02-19T18:53:53.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.948+0000 7fded6ffd700 1 -- 172.21.15.132:0/4131692925 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7fdedc003bf0 con 0x7fdee00fec90 2024-02-19T18:53:53.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.948+0000 7fdee8105700 1 -- 172.21.15.132:0/4131692925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee0062380 con 0x7fdee00fec90 2024-02-19T18:53:53.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.948+0000 7fded6ffd700 1 --2- 172.21.15.132:0/4131692925 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdecc038580 0x7fdecc03aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:53.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.948+0000 7fded6ffd700 1 -- 172.21.15.132:0/4131692925 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fdedc04c6d0 con 0x7fdee00fec90 2024-02-19T18:53:53.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.949+0000 7fdee56a0700 1 --2- 172.21.15.132:0/4131692925 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdecc038580 0x7fdecc03aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:53.966 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.949+0000 7fdee56a0700 1 --2- 172.21.15.132:0/4131692925 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdecc038580 0x7fdecc03aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fded0006fd0 tx=0x7fded0006e40 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:53.969 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:53.953+0000 7fded6ffd700 1 -- 172.21.15.132:0/4131692925 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdedc02a950 con 0x7fdee00fec90 2024-02-19T18:53:54.242 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.226+0000 7fdee8105700 1 -- 172.21.15.132:0/4131692925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fdee0106fd0 con 0x7fdee00fec90 2024-02-19T18:53:54.243 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.227+0000 7fded6ffd700 1 -- 172.21.15.132:0/4131692925 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fdedc02a360 con 0x7fdee00fec90 2024-02-19T18:53:54.243 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:54.243 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.229+0000 7fdee8105700 1 -- 172.21.15.132:0/4131692925 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdecc038580 msgr2=0x7fdecc03aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.229+0000 7fdee8105700 1 --2- 172.21.15.132:0/4131692925 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdecc038580 0x7fdecc03aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fded0006fd0 tx=0x7fded0006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.229+0000 7fdee8105700 1 -- 172.21.15.132:0/4131692925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee00fec90 msgr2=0x7fdee0195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.229+0000 7fdee8105700 1 --2- 172.21.15.132:0/4131692925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee00fec90 0x7fdee0195340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdedc005b40 tx=0x7fdedc004c80 comp rx=0 tx=0).stop 2024-02-19T18:53:54.245 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.229+0000 7fdee8105700 1 -- 172.21.15.132:0/4131692925 shutdown_connections 2024-02-19T18:53:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.229+0000 7fdee8105700 1 --2- 172.21.15.132:0/4131692925 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdecc038580 0x7fdecc03aa30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.229+0000 7fdee8105700 1 --2- 172.21.15.132:0/4131692925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdee00fec90 0x7fdee0195340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.229+0000 7fdee8105700 1 -- 172.21.15.132:0/4131692925 >> 172.21.15.132:0/4131692925 conn(0x7fdee0075750 msgr2=0x7fdee01045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.229+0000 7fdee8105700 1 -- 172.21.15.132:0/4131692925 shutdown_connections 2024-02-19T18:53:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:54.229+0000 7fdee8105700 1 -- 172.21.15.132:0/4131692925 wait complete. 2024-02-19T18:53:54.246 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:54.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:54 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/4131692925' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:53:55.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:55.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:55.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:55.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:55.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:55 smithi088 ceph-mon[29272]: Deploying daemon node-exporter.smithi132 on smithi132 2024-02-19T18:53:55.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:53:55.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:53:56.046 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:53:56.047 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:53:56.249 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:53:58.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.434+0000 7fb21e137700 1 -- 172.21.15.132:0/3923120536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2180fef90 msgr2=0x7fb2180ff360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:58.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.434+0000 7fb21e137700 1 --2- 172.21.15.132:0/3923120536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2180fef90 0x7fb2180ff360 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb208009910 tx=0x7fb208009c20 comp rx=0 tx=0).stop 2024-02-19T18:53:58.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.434+0000 7fb21e137700 1 -- 172.21.15.132:0/3923120536 shutdown_connections 2024-02-19T18:53:58.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.434+0000 7fb21e137700 1 --2- 172.21.15.132:0/3923120536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2180fef90 0x7fb2180ff360 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:58.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.434+0000 7fb21e137700 1 -- 172.21.15.132:0/3923120536 >> 172.21.15.132:0/3923120536 conn(0x7fb2180fa850 msgr2=0x7fb2180fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:58.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.434+0000 7fb21e137700 1 -- 172.21.15.132:0/3923120536 shutdown_connections 2024-02-19T18:53:58.451 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.434+0000 7fb21e137700 1 -- 172.21.15.132:0/3923120536 wait complete. 2024-02-19T18:53:58.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.435+0000 7fb21e137700 1 Processor -- start 2024-02-19T18:53:58.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.435+0000 7fb21e137700 1 -- start start 2024-02-19T18:53:58.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.436+0000 7fb21e137700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2180fef90 0x7fb218196160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:58.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.436+0000 7fb21e137700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2181966a0 con 0x7fb2180fef90 2024-02-19T18:53:58.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.436+0000 7fb2177fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2180fef90 0x7fb218196160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:58.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.436+0000 7fb2177fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2180fef90 0x7fb218196160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:47646/0 (socket says 172.21.15.132:47646) 2024-02-19T18:53:58.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.436+0000 7fb2177fe700 1 -- 172.21.15.132:0/3322087598 learned_addr learned my addr 172.21.15.132:0/3322087598 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:53:58.453 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.436+0000 7fb2177fe700 1 -- 172.21.15.132:0/3322087598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2080095f0 con 0x7fb2180fef90 2024-02-19T18:53:58.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.436+0000 7fb2177fe700 1 --2- 172.21.15.132:0/3322087598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2180fef90 0x7fb218196160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb208005b40 tx=0x7fb208004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:58.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.436+0000 7fb214ff9700 1 -- 172.21.15.132:0/3322087598 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb20801d070 con 0x7fb2180fef90 2024-02-19T18:53:58.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.437+0000 7fb214ff9700 1 -- 172.21.15.132:0/3322087598 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb2080044d0 con 0x7fb2180fef90 2024-02-19T18:53:58.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.437+0000 7fb21e137700 1 -- 172.21.15.132:0/3322087598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2181968a0 con 0x7fb2180fef90 2024-02-19T18:53:58.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.437+0000 7fb214ff9700 1 -- 172.21.15.132:0/3322087598 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb208003930 con 0x7fb2180fef90 2024-02-19T18:53:58.454 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.437+0000 7fb21e137700 1 -- 172.21.15.132:0/3322087598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb218196c60 con 0x7fb2180fef90 2024-02-19T18:53:58.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.437+0000 7fb21e137700 1 -- 172.21.15.132:0/3322087598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb218107210 con 0x7fb2180fef90 2024-02-19T18:53:58.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.437+0000 7fb214ff9700 1 -- 172.21.15.132:0/3322087598 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7fb208003bf0 con 0x7fb2180fef90 2024-02-19T18:53:58.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.438+0000 7fb214ff9700 1 --2- 172.21.15.132:0/3322087598 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb200038580 0x7fb20003aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:53:58.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.438+0000 7fb214ff9700 1 -- 172.21.15.132:0/3322087598 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fb20804c480 con 0x7fb2180fef90 2024-02-19T18:53:58.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.438+0000 7fb216ffd700 1 --2- 172.21.15.132:0/3322087598 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb200038580 0x7fb20003aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:53:58.455 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.438+0000 7fb216ffd700 1 --2- 172.21.15.132:0/3322087598 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb200038580 0x7fb20003aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb20c006fd0 tx=0x7fb20c006e40 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:53:58.459 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.443+0000 7fb214ff9700 1 -- 172.21.15.132:0/3322087598 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb20802a540 con 0x7fb2180fef90 2024-02-19T18:53:58.740 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.724+0000 7fb21e137700 1 -- 172.21.15.132:0/3322087598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb218062380 con 0x7fb2180fef90 2024-02-19T18:53:58.741 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.724+0000 7fb214ff9700 1 -- 172.21.15.132:0/3322087598 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fb20802a360 con 0x7fb2180fef90 2024-02-19T18:53:58.741 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:53:58.741 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:53:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.726+0000 7fb21e137700 1 -- 172.21.15.132:0/3322087598 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb200038580 msgr2=0x7fb20003aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.726+0000 7fb21e137700 1 --2- 172.21.15.132:0/3322087598 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb200038580 0x7fb20003aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb20c006fd0 tx=0x7fb20c006e40 comp rx=0 tx=0).stop 2024-02-19T18:53:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.726+0000 7fb21e137700 1 -- 172.21.15.132:0/3322087598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2180fef90 msgr2=0x7fb218196160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:53:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.726+0000 7fb21e137700 1 --2- 172.21.15.132:0/3322087598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2180fef90 0x7fb218196160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb208005b40 tx=0x7fb208004c80 comp rx=0 tx=0).stop 2024-02-19T18:53:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.727+0000 7fb21e137700 1 -- 172.21.15.132:0/3322087598 shutdown_connections 2024-02-19T18:53:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.727+0000 7fb21e137700 1 --2- 172.21.15.132:0/3322087598 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb200038580 0x7fb20003aa30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.727+0000 7fb21e137700 1 --2- 172.21.15.132:0/3322087598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2180fef90 0x7fb218196160 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:53:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.727+0000 7fb21e137700 1 -- 172.21.15.132:0/3322087598 >> 172.21.15.132:0/3322087598 conn(0x7fb2180fa850 msgr2=0x7fb218103170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:53:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.727+0000 7fb21e137700 1 -- 172.21.15.132:0/3322087598 shutdown_connections 2024-02-19T18:53:58.743 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:53:58.727+0000 7fb21e137700 1 -- 172.21.15.132:0/3322087598 wait complete. 2024-02-19T18:53:58.744 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:53:59.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:53:58 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/3322087598' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:00.450 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:54:00.451 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:54:00.727 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:54:01.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:00 smithi088 ceph-mon[29272]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:03.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.263+0000 7f50e346d700 1 -- 172.21.15.132:0/520504794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50dc1000b0 msgr2=0x7f50dc100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:03.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.264+0000 7f50e346d700 1 --2- 172.21.15.132:0/520504794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50dc1000b0 0x7f50dc100480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009910 tx=0x7f50d8009c20 comp rx=0 tx=0).stop 2024-02-19T18:54:03.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.264+0000 7f50e346d700 1 -- 172.21.15.132:0/520504794 shutdown_connections 2024-02-19T18:54:03.280 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.264+0000 7f50e346d700 1 --2- 172.21.15.132:0/520504794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50dc1000b0 0x7f50dc100480 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:03.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.264+0000 7f50e346d700 1 -- 172.21.15.132:0/520504794 >> 172.21.15.132:0/520504794 conn(0x7f50dc0fb970 msgr2=0x7f50dc0fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:54:03.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.264+0000 7f50e346d700 1 -- 172.21.15.132:0/520504794 shutdown_connections 2024-02-19T18:54:03.281 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.264+0000 7f50e346d700 1 -- 172.21.15.132:0/520504794 wait complete. 2024-02-19T18:54:03.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.264+0000 7f50e346d700 1 Processor -- start 2024-02-19T18:54:03.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.265+0000 7f50e346d700 1 -- start start 2024-02-19T18:54:03.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.265+0000 7f50e346d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50dc1000b0 0x7f50dc195320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:54:03.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.265+0000 7f50e346d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50dc195860 con 0x7f50dc1000b0 2024-02-19T18:54:03.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.265+0000 7f50e1209700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50dc1000b0 0x7f50dc195320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:54:03.284 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.265+0000 7f50e1209700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50dc1000b0 0x7f50dc195320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:47664/0 (socket says 172.21.15.132:47664) 2024-02-19T18:54:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.265+0000 7f50e1209700 1 -- 172.21.15.132:0/2138796839 learned_addr learned my addr 172.21.15.132:0/2138796839 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:54:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.265+0000 7f50e1209700 1 -- 172.21.15.132:0/2138796839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d80095f0 con 0x7f50dc1000b0 2024-02-19T18:54:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.266+0000 7f50e1209700 1 --2- 172.21.15.132:0/2138796839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50dc1000b0 0x7f50dc195320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f50d8005b40 tx=0x7f50d8004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:54:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.266+0000 7f50d27fc700 1 -- 172.21.15.132:0/2138796839 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50d801d070 con 0x7f50dc1000b0 2024-02-19T18:54:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.266+0000 7f50d27fc700 1 -- 172.21.15.132:0/2138796839 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f50d80044d0 con 0x7f50dc1000b0 2024-02-19T18:54:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.266+0000 7f50e346d700 1 -- 172.21.15.132:0/2138796839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50dc195a60 con 0x7f50dc1000b0 2024-02-19T18:54:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.266+0000 7f50d27fc700 1 -- 172.21.15.132:0/2138796839 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f50d8003930 con 0x7f50dc1000b0 2024-02-19T18:54:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.266+0000 7f50e346d700 1 -- 172.21.15.132:0/2138796839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50dc195e20 con 0x7f50dc1000b0 2024-02-19T18:54:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.267+0000 7f50e346d700 1 -- 172.21.15.132:0/2138796839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50dc062380 con 0x7f50dc1000b0 2024-02-19T18:54:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.267+0000 7f50d27fc700 1 -- 172.21.15.132:0/2138796839 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f50d8003c30 con 0x7f50dc1000b0 2024-02-19T18:54:03.285 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.267+0000 7f50d27fc700 1 --2- 172.21.15.132:0/2138796839 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f50c8038580 0x7f50c803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:54:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.267+0000 7f50d27fc700 1 -- 172.21.15.132:0/2138796839 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f50d804c3e0 con 0x7f50dc1000b0 2024-02-19T18:54:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.267+0000 7f50e0a08700 1 --2- 172.21.15.132:0/2138796839 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f50c8038580 0x7f50c803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:54:03.286 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.268+0000 7f50e0a08700 1 --2- 172.21.15.132:0/2138796839 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f50c8038580 0x7f50c803aa30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50cc006fd0 tx=0x7f50cc006e40 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:54:03.288 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.272+0000 7f50d27fc700 1 -- 172.21.15.132:0/2138796839 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f50d802a540 con 0x7f50dc1000b0 2024-02-19T18:54:03.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:02 smithi088 ceph-mon[29272]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:03.571 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.555+0000 7f50e346d700 1 -- 172.21.15.132:0/2138796839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f50dc107e50 con 0x7f50dc1000b0 2024-02-19T18:54:03.572 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.556+0000 7f50d27fc700 1 -- 172.21.15.132:0/2138796839 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f50d802a360 con 0x7f50dc1000b0 2024-02-19T18:54:03.572 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:54:03.572 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":1,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:50:24.209693Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-19T18:54:03.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.558+0000 7f50e346d700 1 -- 172.21.15.132:0/2138796839 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f50c8038580 msgr2=0x7f50c803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:03.574 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.558+0000 7f50e346d700 1 --2- 172.21.15.132:0/2138796839 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f50c8038580 0x7f50c803aa30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50cc006fd0 tx=0x7f50cc006e40 comp rx=0 tx=0).stop 2024-02-19T18:54:03.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.558+0000 7f50e346d700 1 -- 172.21.15.132:0/2138796839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50dc1000b0 msgr2=0x7f50dc195320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:03.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.558+0000 7f50e346d700 1 --2- 172.21.15.132:0/2138796839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50dc1000b0 0x7f50dc195320 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f50d8005b40 tx=0x7f50d8004c80 comp rx=0 tx=0).stop 2024-02-19T18:54:03.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.558+0000 7f50e346d700 1 -- 172.21.15.132:0/2138796839 shutdown_connections 2024-02-19T18:54:03.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.558+0000 7f50e346d700 1 --2- 172.21.15.132:0/2138796839 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f50c8038580 0x7f50c803aa30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:03.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.558+0000 7f50e346d700 1 --2- 172.21.15.132:0/2138796839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50dc1000b0 0x7f50dc195320 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:03.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.558+0000 7f50e346d700 1 -- 172.21.15.132:0/2138796839 >> 172.21.15.132:0/2138796839 conn(0x7f50dc0fb970 msgr2=0x7f50dc104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:54:03.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.558+0000 7f50e346d700 1 -- 172.21.15.132:0/2138796839 shutdown_connections 2024-02-19T18:54:03.575 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:03.558+0000 7f50e346d700 1 -- 172.21.15.132:0/2138796839 wait complete. 2024-02-19T18:54:03.575 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 1 2024-02-19T18:54:04.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:04.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:04.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:04.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:04.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T18:54:04.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-19T18:54:04.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T18:54:04.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:04.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:04 smithi088 ceph-mon[29272]: Deploying daemon mgr.smithi132.psllsa on smithi132 2024-02-19T18:54:04.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:04 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/2138796839' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:04.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:04 smithi088 ceph-mon[29272]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:06.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:05 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:06.339 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-02-19T18:54:06.339 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mon dump -f json 2024-02-19T18:54:07.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:06 smithi088 ceph-mon[29272]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:07.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:06 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:07.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:06 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:07.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:06 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:07.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:06 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:07.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:06 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T18:54:07.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:06 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:07.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:06 smithi088 ceph-mon[29272]: Deploying daemon mon.smithi132 on smithi132 2024-02-19T18:54:07.438 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:54:09.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:08 smithi088 ceph-mon[29272]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:09 smithi132 podman[37913]: 2024-02-19 18:54:09.87968947 +0000 UTC m=+0.340671605 container create ba1c249c6017fc42706dce1d0b6d0f13df949c6cf99b9b3ebb870d9c5eb7d414 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, ceph=True, org.label-schema.vendor=CentOS) 2024-02-19T18:54:10.272 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 podman[37913]: 2024-02-19 18:54:10.19137619 +0000 UTC m=+0.652358326 container init ba1c249c6017fc42706dce1d0b6d0f13df949c6cf99b9b3ebb870d9c5eb7d414 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, ceph=True, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-02-19T18:54:10.272 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 podman[37913]: 2024-02-19 18:54:10.19463124 +0000 UTC m=+0.655613376 container start ba1c249c6017fc42706dce1d0b6d0f13df949c6cf99b9b3ebb870d9c5eb7d414 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS) 2024-02-19T18:54:10.272 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: set uid:gid to 167:167 (ceph:ceph) 2024-02-19T18:54:10.272 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-02-19T18:54:10.272 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pidfile_write: ignore empty --pid-file 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: load: jerasure load: lrc 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: RocksDB version: 7.9.2 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Git sha 0 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Compile date 2023-08-03 19:21:13 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: DB SUMMARY 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: DB Session ID: MWQNQMW20KE2DW0426JP 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: CURRENT file: CURRENT 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: IDENTITY file: IDENTITY 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi132/store.db dir, Total Num: 0, files: 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi132/store.db: 000004.log size: 511 ; 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.error_if_exists: 0 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.create_if_missing: 0 2024-02-19T18:54:10.273 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.paranoid_checks: 1 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.env: 0x556abca65720 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.fs: PosixFileSystem 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.info_log: 0x556abe5f92c0 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_file_opening_threads: 16 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.statistics: (nil) 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.use_fsync: 0 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_log_file_size: 0 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.keep_log_file_num: 1000 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.recycle_log_file_num: 0 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.allow_fallocate: 1 2024-02-19T18:54:10.274 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.allow_mmap_reads: 0 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.allow_mmap_writes: 0 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.use_direct_reads: 0 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.create_missing_column_families: 0 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.db_log_dir: 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.wal_dir: 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-19T18:54:10.275 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.advise_random_on_open: 1 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.db_write_buffer_size: 0 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.write_buffer_manager: 0x556abd86a3c0 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.rate_limiter: (nil) 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.wal_recovery_mode: 2 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.enable_thread_tracking: 0 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.enable_pipelined_write: 0 2024-02-19T18:54:10.276 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.unordered_write: 0 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.row_cache: None 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.wal_filter: None 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.allow_ingest_behind: 0 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.two_write_queues: 0 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.manual_wal_flush: 0 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.wal_compression: 0 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.atomic_flush: 0 2024-02-19T18:54:10.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.log_readahead_size: 0 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.best_efforts_recovery: 0 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.allow_data_in_errors: 0 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.db_host_id: __hostname__ 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_background_jobs: 2 2024-02-19T18:54:10.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_background_compactions: -1 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_subcompactions: 1 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_total_wal_size: 0 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_open_files: -1 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bytes_per_sync: 0 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-19T18:54:10.279 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_readahead_size: 0 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_background_flushes: -1 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Compression algorithms supported: 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: kZSTD supported: 0 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: kXpressCompression supported: 0 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: kLZ4HCCompression supported: 1 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: kZlibCompression supported: 1 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: kSnappyCompression supported: 1 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: kLZ4Compression supported: 1 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: kBZip2Compression supported: 0 2024-02-19T18:54:10.280 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi132/store.db/MANIFEST-000005 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.merge_operator: 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_filter: None 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_filter_factory: None 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.sst_partitioner_factory: None 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556abe5f93c0) 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: cache_index_and_filter_blocks: 1 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-19T18:54:10.281 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: pin_top_level_index_and_filter: 1 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: index_type: 0 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: data_block_index_type: 0 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: index_shortening: 1 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: checksum: 4 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: no_block_cache: 0 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_cache: 0x556abd90b1f0 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_cache_name: BinnedLRUCache 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_cache_options: 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: capacity : 536870912 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: num_shard_bits : 4 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: strict_capacity_limit : 0 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: high_pri_pool_ratio: 0.000 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_cache_compressed: (nil) 2024-02-19T18:54:10.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: persistent_cache: (nil) 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_size: 4096 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_size_deviation: 10 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_restart_interval: 16 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: index_block_restart_interval: 1 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: metadata_block_size: 4096 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: partition_filters: 0 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: use_delta_encoding: 1 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: filter_policy: bloomfilter 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: whole_key_filtering: 1 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: verify_compression: 0 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: read_amp_bytes_per_bit: 0 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: format_version: 5 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: enable_index_compression: 1 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_align: 0 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: max_auto_readahead_size: 262144 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: prepopulate_block_cache: 0 2024-02-19T18:54:10.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: initial_auto_readahead_size: 8192 2024-02-19T18:54:10.284 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: num_file_reads_for_auto_readahead: 2 2024-02-19T18:54:10.285 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.write_buffer_size: 33554432 2024-02-19T18:54:10.285 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_write_buffer_number: 2 2024-02-19T18:54:10.285 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compression: NoCompression 2024-02-19T18:54:10.285 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bottommost_compression: Disabled 2024-02-19T18:54:10.285 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.prefix_extractor: nullptr 2024-02-19T18:54:10.285 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-19T18:54:10.285 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.num_levels: 7 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compression_opts.level: 32767 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compression_opts.strategy: 0 2024-02-19T18:54:10.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compression_opts.enabled: false 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.target_file_size_base: 67108864 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-19T18:54:10.287 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.arena_block_size: 1048576 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.disable_auto_compactions: 0 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-19T18:54:10.288 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.table_properties_collectors: 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.inplace_update_support: 0 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.bloom_locality: 0 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.max_successive_merges: 0 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.paranoid_file_checks: 0 2024-02-19T18:54:10.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.force_consistency_checks: 1 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.report_bg_io_stats: 0 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.ttl: 2592000 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.enable_blob_files: false 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.min_blob_size: 0 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.blob_file_size: 268435456 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.blob_file_starting_level: 0 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-19T18:54:10.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi132/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6d76a768-93d4-4cfb-8a70-5188362b4b50 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708368850233261, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708368850233711, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708368850, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6d76a768-93d4-4cfb-8a70-5188362b4b50", "db_session_id": "MWQNQMW20KE2DW0426JP", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708368850233790, "job": 1, "event": "recovery_finished"} 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi132/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556abd9a8000 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: DB pointer 0x556abd994000 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: ** DB Stats ** 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-19T18:54:10.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: ** Compaction Stats [default] ** 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: ** Compaction Stats [default] ** 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T18:54:10.292 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-19T18:54:10.293 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Block cache BinnedLRUCache@0x556abd90b1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132 does not exist in monmap, will attempt to join an existing cluster 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: using public_addr v2:172.21.15.132:0/0 -> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: starting mon.smithi132 rank -1 at public addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] at bind addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi132 fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(???) e0 preinit fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).mds e1 new map 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).mds e1 print_map 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: e1 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T18:54:10.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: legacy client fscid: -1 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: No filesystems configured 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T18:54:10.295 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/3286783967' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/1605323175' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/2841737414' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/2896903112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/3628397065' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14164 172.21.15.88:0/1903183502' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mgrmap e14: smithi088.gyviqq(active, since 115s) 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/4101917898' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/3087612972' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.297 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Active manager daemon smithi088.gyviqq restarted 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Activating manager daemon smithi088.gyviqq 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: osdmap e5: 0 total, 0 up, 0 in 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mgrmap e15: smithi088.gyviqq(active, starting, since 0.00354374s) 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi088.gyviqq", "id": "smithi088.gyviqq"}]: dispatch 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Manager daemon smithi088.gyviqq is now available 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.298 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/mirror_snapshot_schedule"}]: dispatch 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/trash_purge_schedule"}]: dispatch 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: [19/Feb/2024:18:53:39] ENGINE Bus STARTING 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mgrmap e16: smithi088.gyviqq(active, since 1.00535s) 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: [19/Feb/2024:18:53:40] ENGINE Serving on http://172.21.15.88:8765 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: [19/Feb/2024:18:53:40] ENGINE Serving on https://172.21.15.88:7150 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: [19/Feb/2024:18:53:40] ENGINE Bus STARTED 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mgrmap e17: smithi088.gyviqq(active, since 2s) 2024-02-19T18:54:10.299 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/685083041' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd/host:smithi088", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd/host:smithi132", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:10.300 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Updating smithi088:/etc/ceph/ceph.conf 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Updating smithi132:/etc/ceph/ceph.conf 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/1274573815' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:54:10.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Deploying daemon ceph-exporter.smithi132 on smithi132 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/575037105' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi132", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T18:54:10.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi132", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Deploying daemon crash.smithi132 on smithi132 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/4131692925' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Deploying daemon node-exporter.smithi132 on smithi132 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/3322087598' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.303 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Deploying daemon mgr.smithi132.psllsa on smithi132 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/2138796839' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.304 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: Deploying daemon mon.smithi132 on smithi132 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.305 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: mon.smithi132@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta expand map: {default=false} 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta from 'false' to 'false' 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta expanded map: {default=false} 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta expand map: {default=info} 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta from 'info' to 'info' 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta expanded map: {default=info} 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta expand map: {default=daemon} 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta from 'daemon' to 'daemon' 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta expanded map: {default=daemon} 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta expand map: {default=debug} 2024-02-19T18:54:10.306 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta from 'debug' to 'debug' 2024-02-19T18:54:10.307 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 ceph-mon[37969]: expand_channel_meta expanded map: {default=debug} 2024-02-19T18:54:10.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:09 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:54:10.346 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:09 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:10.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.194+0000 7faa117b0700 1 -- 172.21.15.132:0/3501887807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa0c100ff0 msgr2=0x7faa0c1013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:10.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.194+0000 7faa117b0700 1 --2- 172.21.15.132:0/3501887807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa0c100ff0 0x7faa0c1013c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009910 tx=0x7fa9fc009c20 comp rx=0 tx=0).stop 2024-02-19T18:54:10.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.194+0000 7faa117b0700 1 -- 172.21.15.132:0/3501887807 shutdown_connections 2024-02-19T18:54:10.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.194+0000 7faa117b0700 1 --2- 172.21.15.132:0/3501887807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa0c100ff0 0x7faa0c1013c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:10.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.194+0000 7faa117b0700 1 -- 172.21.15.132:0/3501887807 >> 172.21.15.132:0/3501887807 conn(0x7faa0c0fa850 msgr2=0x7faa0c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:54:10.492 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.194+0000 7faa117b0700 1 -- 172.21.15.132:0/3501887807 shutdown_connections 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.194+0000 7faa117b0700 1 -- 172.21.15.132:0/3501887807 wait complete. 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.195+0000 7faa117b0700 1 Processor -- start 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.195+0000 7faa117b0700 1 -- start start 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.195+0000 7faa117b0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa0c100ff0 0x7faa0c1953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.195+0000 7faa117b0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c1958f0 con 0x7faa0c100ff0 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.196+0000 7faa0affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa0c100ff0 0x7faa0c1953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.196+0000 7faa0affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa0c100ff0 0x7faa0c1953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.132:38376/0 (socket says 172.21.15.132:38376) 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.196+0000 7faa0affd700 1 -- 172.21.15.132:0/3504327112 learned_addr learned my addr 172.21.15.132:0/3504327112 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.196+0000 7faa0affd700 1 -- 172.21.15.132:0/3504327112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9fc0095f0 con 0x7faa0c100ff0 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.196+0000 7faa0affd700 1 --2- 172.21.15.132:0/3504327112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa0c100ff0 0x7faa0c1953b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0092e0 tx=0x7fa9fc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.196+0000 7fa9f3fff700 1 -- 172.21.15.132:0/3504327112 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9fc01d070 con 0x7faa0c100ff0 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.196+0000 7fa9f3fff700 1 -- 172.21.15.132:0/3504327112 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa9fc00be10 con 0x7faa0c100ff0 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.196+0000 7fa9f3fff700 1 -- 172.21.15.132:0/3504327112 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9fc022e10 con 0x7faa0c100ff0 2024-02-19T18:54:10.493 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.196+0000 7faa117b0700 1 -- 172.21.15.132:0/3504327112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0c195af0 con 0x7faa0c100ff0 2024-02-19T18:54:10.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.196+0000 7faa117b0700 1 -- 172.21.15.132:0/3504327112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0c195f10 con 0x7faa0c100ff0 2024-02-19T18:54:10.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.197+0000 7fa9f3fff700 1 -- 172.21.15.132:0/3504327112 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7fa9fc00b960 con 0x7faa0c100ff0 2024-02-19T18:54:10.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.198+0000 7fa9f3fff700 1 --2- 172.21.15.132:0/3504327112 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa9f4038530 0x7fa9f403a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:54:10.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.198+0000 7faa117b0700 1 -- 172.21.15.132:0/3504327112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0c062380 con 0x7faa0c100ff0 2024-02-19T18:54:10.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.198+0000 7fa9f3fff700 1 -- 172.21.15.132:0/3504327112 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fa9fc04c5a0 con 0x7faa0c100ff0 2024-02-19T18:54:10.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.198+0000 7faa0a7fc700 1 --2- 172.21.15.132:0/3504327112 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa9f4038530 0x7fa9f403a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:54:10.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.198+0000 7faa0a7fc700 1 --2- 172.21.15.132:0/3504327112 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa9f4038530 0x7fa9f403a9e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa00006fd0 tx=0x7faa00006e40 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:54:10.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.203+0000 7fa9f3fff700 1 -- 172.21.15.132:0/3504327112 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa9fc02a540 con 0x7faa0c100ff0 2024-02-19T18:54:10.494 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.262+0000 7fa9f3fff700 1 -- 172.21.15.132:0/3504327112 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9fc02a540 con 0x7faa0c100ff0 2024-02-19T18:54:10.517 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:10.500+0000 7faa117b0700 1 -- 172.21.15.132:0/3504327112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7faa0c106f80 con 0x7faa0c100ff0 2024-02-19T18:54:10.585 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 bash[37913]: ba1c249c6017fc42706dce1d0b6d0f13df949c6cf99b9b3ebb870d9c5eb7d414 2024-02-19T18:54:10.585 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:10 smithi132 systemd[1]: Started Ceph mon.smithi132 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T18:54:15.325 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:15.308+0000 7fa9f3fff700 1 -- 172.21.15.132:0/3504327112 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1022 (secure 0 0 0) 0x7fa9fc027070 con 0x7faa0c100ff0 2024-02-19T18:54:15.325 INFO:teuthology.orchestra.run.smithi132.stdout: 2024-02-19T18:54:15.326 INFO:teuthology.orchestra.run.smithi132.stdout:{"epoch":2,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","modified":"2024-02-19T18:54:10.263127Z","created":"2024-02-19T18:50:24.209693Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi088","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi132","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:3300","nonce":0},{"type":"v1","addr":"172.21.15.132:6789","nonce":0}]},"addr":"172.21.15.132:6789/0","public_addr":"172.21.15.132:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-19T18:54:15.328 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:15.311+0000 7faa117b0700 1 -- 172.21.15.132:0/3504327112 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa9f4038530 msgr2=0x7fa9f403a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:15.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:15.311+0000 7faa117b0700 1 --2- 172.21.15.132:0/3504327112 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa9f4038530 0x7fa9f403a9e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa00006fd0 tx=0x7faa00006e40 comp rx=0 tx=0).stop 2024-02-19T18:54:15.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:15.311+0000 7faa117b0700 1 -- 172.21.15.132:0/3504327112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa0c100ff0 msgr2=0x7faa0c1953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:15.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:15.311+0000 7faa117b0700 1 --2- 172.21.15.132:0/3504327112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa0c100ff0 0x7faa0c1953b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc0092e0 tx=0x7fa9fc004c80 comp rx=0 tx=0).stop 2024-02-19T18:54:15.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:15.311+0000 7faa117b0700 1 -- 172.21.15.132:0/3504327112 shutdown_connections 2024-02-19T18:54:15.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:15.311+0000 7faa117b0700 1 --2- 172.21.15.132:0/3504327112 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa9f4038530 0x7fa9f403a9e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:15.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:15.311+0000 7faa117b0700 1 --2- 172.21.15.132:0/3504327112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa0c100ff0 0x7faa0c1953b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:15.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:15.311+0000 7faa117b0700 1 -- 172.21.15.132:0/3504327112 >> 172.21.15.132:0/3504327112 conn(0x7faa0c0fa850 msgr2=0x7faa0c104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:54:15.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:15.311+0000 7faa117b0700 1 -- 172.21.15.132:0/3504327112 shutdown_connections 2024-02-19T18:54:15.329 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:54:15.311+0000 7faa117b0700 1 -- 172.21.15.132:0/3504327112 wait complete. 2024-02-19T18:54:15.330 INFO:teuthology.orchestra.run.smithi132.stderr:dumped monmap epoch 2 2024-02-19T18:54:15.654 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T18:54:15.654 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: mon.smithi088 calling monitor election 2024-02-19T18:54:15.654 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.654 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/3504327112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:15.654 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.654 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:15.654 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.654 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: mon.smithi132 calling monitor election 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/2779901018' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/crt"}]: dispatch 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: mon.smithi088 is new leader, mons smithi088,smithi132 in quorum (ranks 0,1) 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: monmap e2: 2 mons at {smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0],smithi132=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0]} removed_ranks: {} 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: fsmap 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: osdmap e5: 0 total, 0 up, 0 in 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: mgrmap e17: smithi088.gyviqq(active, since 35s) 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: Standby manager daemon smithi132.psllsa started 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/2779901018' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: overall HEALTH_OK 2024-02-19T18:54:15.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/2779901018' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/key"}]: dispatch 2024-02-19T18:54:15.656 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/2779901018' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T18:54:15.656 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:15.656 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:15.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T18:54:15.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: mon.smithi088 calling monitor election 2024-02-19T18:54:15.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/3504327112' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-19T18:54:15.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:15.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: mon.smithi132 calling monitor election 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/2779901018' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/crt"}]: dispatch 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: mon.smithi088 is new leader, mons smithi088,smithi132 in quorum (ranks 0,1) 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: monmap e2: 2 mons at {smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0],smithi132=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0]} removed_ranks: {} 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: fsmap 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: osdmap e5: 0 total, 0 up, 0 in 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: mgrmap e17: smithi088.gyviqq(active, since 35s) 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: Standby manager daemon smithi132.psllsa started 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/2779901018' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: overall HEALTH_OK 2024-02-19T18:54:15.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/2779901018' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/key"}]: dispatch 2024-02-19T18:54:15.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/2779901018' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T18:54:15.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:15.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:15.917 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-19T18:54:15.918 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph config generate-minimal-conf 2024-02-19T18:54:16.134 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:54:16.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:16 smithi132 ceph-mon[37969]: mgrmap e18: smithi088.gyviqq(active, since 35s), standbys: smithi132.psllsa 2024-02-19T18:54:16.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi132.psllsa", "id": "smithi132.psllsa"}]: dispatch 2024-02-19T18:54:16.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:16.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:16.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:16.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:54:16.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:16 smithi132 ceph-mon[37969]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:16.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:16.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:16 smithi088 ceph-mon[29272]: mgrmap e18: smithi088.gyviqq(active, since 35s), standbys: smithi132.psllsa 2024-02-19T18:54:16.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi132.psllsa", "id": "smithi132.psllsa"}]: dispatch 2024-02-19T18:54:16.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:16.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:16.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:16.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:54:16.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:16 smithi088 ceph-mon[29272]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:16.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:54:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.844+0000 7ffb72ed1700 1 -- 172.21.15.88:0/2233229620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb6c0fec90 msgr2=0x7ffb6c0ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.844+0000 7ffb72ed1700 1 --2- 172.21.15.88:0/2233229620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb6c0fec90 0x7ffb6c0ff060 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009910 tx=0x7ffb60009c20 comp rx=0 tx=0).stop 2024-02-19T18:54:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.844+0000 7ffb72ed1700 1 -- 172.21.15.88:0/2233229620 shutdown_connections 2024-02-19T18:54:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.844+0000 7ffb72ed1700 1 --2- 172.21.15.88:0/2233229620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb6c0fec90 0x7ffb6c0ff060 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.844+0000 7ffb72ed1700 1 -- 172.21.15.88:0/2233229620 >> 172.21.15.88:0/2233229620 conn(0x7ffb6c075750 msgr2=0x7ffb6c075b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.844+0000 7ffb72ed1700 1 -- 172.21.15.88:0/2233229620 shutdown_connections 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.844+0000 7ffb72ed1700 1 -- 172.21.15.88:0/2233229620 wait complete. 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.845+0000 7ffb72ed1700 1 Processor -- start 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.845+0000 7ffb72ed1700 1 -- start start 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.845+0000 7ffb72ed1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb6c0fec90 0x7ffb6c195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.845+0000 7ffb72ed1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb6c1959b0 0x7ffb6c199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.845+0000 7ffb72ed1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c195fc0 con 0x7ffb6c1959b0 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.845+0000 7ffb72ed1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6c196130 con 0x7ffb6c0fec90 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.845+0000 7ffb6bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb6c1959b0 0x7ffb6c199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.845+0000 7ffb6bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb6c1959b0 0x7ffb6c199e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39894/0 (socket says 172.21.15.88:39894) 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.845+0000 7ffb6bfff700 1 -- 172.21.15.88:0/571991644 learned_addr learned my addr 172.21.15.88:0/571991644 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.845+0000 7ffb70c6d700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb6c0fec90 0x7ffb6c195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:54:17.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb70c6d700 1 -- 172.21.15.88:0/571991644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb6c0fec90 msgr2=0x7ffb6c195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb70c6d700 1 -- 172.21.15.88:0/571991644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb6c0fec90 msgr2=0x7ffb6c195470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb70c6d700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb6c0fec90 0x7ffb6c195470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb70c6d700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb6c0fec90 0x7ffb6c195470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb6bfff700 1 -- 172.21.15.88:0/571991644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb6c0fec90 msgr2=0x7ffb6c195470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb6bfff700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb6c0fec90 0x7ffb6c195470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb6bfff700 1 -- 172.21.15.88:0/571991644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb600095f0 con 0x7ffb6c1959b0 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb6bfff700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb6c1959b0 0x7ffb6c199e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800b730 tx=0x7ffb5800baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb69ffb700 1 -- 172.21.15.88:0/571991644 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb58004200 con 0x7ffb6c1959b0 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb72ed1700 1 -- 172.21.15.88:0/571991644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6c19a420 con 0x7ffb6c1959b0 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb69ffb700 1 -- 172.21.15.88:0/571991644 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb58004360 con 0x7ffb6c1959b0 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb69ffb700 1 -- 172.21.15.88:0/571991644 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb58011be0 con 0x7ffb6c1959b0 2024-02-19T18:54:17.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.846+0000 7ffb72ed1700 1 -- 172.21.15.88:0/571991644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb6c19a9f0 con 0x7ffb6c1959b0 2024-02-19T18:54:17.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.847+0000 7ffb72ed1700 1 -- 172.21.15.88:0/571991644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6c107550 con 0x7ffb6c1959b0 2024-02-19T18:54:17.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.847+0000 7ffb69ffb700 1 -- 172.21.15.88:0/571991644 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7ffb580036a0 con 0x7ffb6c1959b0 2024-02-19T18:54:17.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.848+0000 7ffb69ffb700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffb5c06c760 0x7ffb5c06ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:54:17.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.848+0000 7ffb70c6d700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffb5c06c760 0x7ffb5c06ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:54:17.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.848+0000 7ffb69ffb700 1 -- 172.21.15.88:0/571991644 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7ffb5808a3d0 con 0x7ffb6c1959b0 2024-02-19T18:54:17.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.849+0000 7ffb70c6d700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffb5c06c760 0x7ffb5c06ec10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000b5c0 tx=0x7ffb60005a90 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:54:17.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:17.853+0000 7ffb69ffb700 1 -- 172.21.15.88:0/571991644 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ffb58055b30 con 0x7ffb6c1959b0 2024-02-19T18:54:18.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.063+0000 7ffb72ed1700 1 -- 172.21.15.88:0/571991644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ffb6c1967d0 con 0x7ffb6c1959b0 2024-02-19T18:54:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.064+0000 7ffb69ffb700 1 -- 172.21.15.88:0/571991644 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7ffb58059150 con 0x7ffb6c1959b0 2024-02-19T18:54:18.081 INFO:teuthology.orchestra.run.smithi088.stdout:# minimal ceph.conf for a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:54:18.081 INFO:teuthology.orchestra.run.smithi088.stdout:[global] 2024-02-19T18:54:18.081 INFO:teuthology.orchestra.run.smithi088.stdout: fsid = a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T18:54:18.081 INFO:teuthology.orchestra.run.smithi088.stdout: mon_host = [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] 2024-02-19T18:54:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.066+0000 7ffb72ed1700 1 -- 172.21.15.88:0/571991644 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffb5c06c760 msgr2=0x7ffb5c06ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:18.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.066+0000 7ffb72ed1700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffb5c06c760 0x7ffb5c06ec10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000b5c0 tx=0x7ffb60005a90 comp rx=0 tx=0).stop 2024-02-19T18:54:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.066+0000 7ffb72ed1700 1 -- 172.21.15.88:0/571991644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb6c1959b0 msgr2=0x7ffb6c199e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.066+0000 7ffb72ed1700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb6c1959b0 0x7ffb6c199e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800b730 tx=0x7ffb5800baf0 comp rx=0 tx=0).stop 2024-02-19T18:54:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.066+0000 7ffb72ed1700 1 -- 172.21.15.88:0/571991644 shutdown_connections 2024-02-19T18:54:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.066+0000 7ffb72ed1700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffb5c06c760 0x7ffb5c06ec10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.066+0000 7ffb72ed1700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb6c0fec90 0x7ffb6c195470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.066+0000 7ffb72ed1700 1 --2- 172.21.15.88:0/571991644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb6c1959b0 0x7ffb6c199e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.066+0000 7ffb72ed1700 1 -- 172.21.15.88:0/571991644 >> 172.21.15.88:0/571991644 conn(0x7ffb6c075750 msgr2=0x7ffb6c104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:54:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.066+0000 7ffb72ed1700 1 -- 172.21.15.88:0/571991644 shutdown_connections 2024-02-19T18:54:18.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:18.066+0000 7ffb72ed1700 1 -- 172.21.15.88:0/571991644 wait complete. 2024-02-19T18:54:18.753 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.754 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.754 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:18.754 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:54:18.754 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: Updating smithi088:/etc/ceph/ceph.conf 2024-02-19T18:54:18.754 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: Updating smithi132:/etc/ceph/ceph.conf 2024-02-19T18:54:18.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:18.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:54:18.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:54:18.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/571991644' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:18.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.756 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.756 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.756 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T18:54:18.756 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T18:54:18.757 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:18.760 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-19T18:54:18.760 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:54:18.760 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.conf 2024-02-19T18:54:18.793 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:54:18.793 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-19T18:54:18.862 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:54:18.863 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.conf 2024-02-19T18:54:18.890 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.890 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.890 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:18.890 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:54:18.890 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: Updating smithi088:/etc/ceph/ceph.conf 2024-02-19T18:54:18.890 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: Updating smithi132:/etc/ceph/ceph.conf 2024-02-19T18:54:18.890 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:18.890 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:54:18.890 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T18:54:18.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/571991644' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:18.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:18.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T18:54:18.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T18:54:18.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:18.895 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:54:18.895 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-19T18:54:18.965 INFO:tasks.cephadm:Deploying OSDs... 2024-02-19T18:54:18.965 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:54:18.965 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/scratch_devs of=/dev/stdout 2024-02-19T18:54:18.981 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-19T18:54:18.981 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_1 2024-02-19T18:54:19.037 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-19T18:54:19.037 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T18:54:19.037 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-19T18:54:19.037 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T18:54:19.037 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T18:54:19.037 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-02-19 18:53:31.252382095 +0000 2024-02-19T18:54:19.037 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-02-19 18:46:34.050263720 +0000 2024-02-19T18:54:19.037 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-02-19 18:46:34.050263720 +0000 2024-02-19T18:54:19.037 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-02-19 18:46:34.050263720 +0000 2024-02-19T18:54:19.038 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-19T18:54:19.104 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-02-19T18:54:19.104 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-02-19T18:54:19.104 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000210701 s, 2.4 MB/s 2024-02-19T18:54:19.105 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-19T18:54:19.161 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_2 2024-02-19T18:54:19.216 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-19T18:54:19.216 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T18:54:19.216 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-19T18:54:19.216 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T18:54:19.216 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T18:54:19.216 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-02-19 18:53:31.253382074 +0000 2024-02-19T18:54:19.216 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-02-19 18:46:34.426256136 +0000 2024-02-19T18:54:19.216 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-02-19 18:46:34.426256136 +0000 2024-02-19T18:54:19.216 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-02-19 18:46:34.426256136 +0000 2024-02-19T18:54:19.217 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-19T18:54:19.282 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-02-19T18:54:19.282 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-02-19T18:54:19.282 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000224199 s, 2.3 MB/s 2024-02-19T18:54:19.283 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-19T18:54:19.338 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_3 2024-02-19T18:54:19.392 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-19T18:54:19.392 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T18:54:19.392 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-19T18:54:19.392 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T18:54:19.392 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T18:54:19.393 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-02-19 18:53:31.253382074 +0000 2024-02-19T18:54:19.393 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-02-19 18:46:34.834247940 +0000 2024-02-19T18:54:19.393 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-02-19 18:46:34.834247940 +0000 2024-02-19T18:54:19.393 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-02-19 18:46:34.834247940 +0000 2024-02-19T18:54:19.393 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-19T18:54:19.457 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-02-19T18:54:19.457 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-02-19T18:54:19.457 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000208741 s, 2.5 MB/s 2024-02-19T18:54:19.458 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-19T18:54:19.515 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_4 2024-02-19T18:54:19.570 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-19T18:54:19.570 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T18:54:19.570 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-02-19T18:54:19.570 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T18:54:19.570 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T18:54:19.570 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-02-19 18:53:31.253382074 +0000 2024-02-19T18:54:19.570 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-02-19 18:46:35.238239826 +0000 2024-02-19T18:54:19.570 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-02-19 18:46:35.238239826 +0000 2024-02-19T18:54:19.571 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-02-19 18:46:35.238239826 +0000 2024-02-19T18:54:19.571 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-19T18:54:19.635 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-02-19T18:54:19.636 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-02-19T18:54:19.636 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000250615 s, 2.0 MB/s 2024-02-19T18:54:19.637 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-19T18:54:19.692 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:54:19.692 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/scratch_devs of=/dev/stdout 2024-02-19T18:54:19.707 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-19T18:54:19.708 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_1 2024-02-19T18:54:19.762 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-19T18:54:19.762 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T18:54:19.762 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-19T18:54:19.762 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T18:54:19.763 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T18:54:19.763 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-02-19 18:54:08.941940260 +0000 2024-02-19T18:54:19.763 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-02-19 18:46:34.034436506 +0000 2024-02-19T18:54:19.763 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-02-19 18:46:34.034436506 +0000 2024-02-19T18:54:19.763 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-02-19 18:46:34.034436506 +0000 2024-02-19T18:54:19.763 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-19T18:54:19.826 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:19 smithi132 ceph-mon[37969]: Reconfiguring mon.smithi088 (unknown last config time)... 2024-02-19T18:54:19.826 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:19 smithi132 ceph-mon[37969]: Reconfiguring daemon mon.smithi088 on smithi088 2024-02-19T18:54:19.828 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-02-19T18:54:19.828 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-02-19T18:54:19.829 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000214699 s, 2.4 MB/s 2024-02-19T18:54:19.830 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-19T18:54:19.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:19 smithi088 ceph-mon[29272]: Reconfiguring mon.smithi088 (unknown last config time)... 2024-02-19T18:54:19.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:19 smithi088 ceph-mon[29272]: Reconfiguring daemon mon.smithi088 on smithi088 2024-02-19T18:54:19.885 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_2 2024-02-19T18:54:19.939 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-19T18:54:19.939 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T18:54:19.939 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-19T18:54:19.939 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T18:54:19.940 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T18:54:19.940 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-02-19 18:54:08.941940260 +0000 2024-02-19T18:54:19.940 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-02-19 18:46:34.423426496 +0000 2024-02-19T18:54:19.940 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-02-19 18:46:34.423426496 +0000 2024-02-19T18:54:19.940 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-02-19 18:46:34.423426496 +0000 2024-02-19T18:54:19.940 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-19T18:54:20.005 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-02-19T18:54:20.005 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-02-19T18:54:20.005 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000192865 s, 2.7 MB/s 2024-02-19T18:54:20.006 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-19T18:54:20.061 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_3 2024-02-19T18:54:20.116 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-19T18:54:20.116 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T18:54:20.116 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-19T18:54:20.117 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T18:54:20.117 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T18:54:20.117 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-02-19 18:54:08.942940235 +0000 2024-02-19T18:54:20.117 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-02-19 18:46:34.830416023 +0000 2024-02-19T18:54:20.117 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-02-19 18:46:34.830416023 +0000 2024-02-19T18:54:20.117 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-02-19 18:46:34.830416023 +0000 2024-02-19T18:54:20.117 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-19T18:54:20.182 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-02-19T18:54:20.182 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-02-19T18:54:20.182 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000212042 s, 2.4 MB/s 2024-02-19T18:54:20.184 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-19T18:54:20.239 DEBUG:teuthology.orchestra.run.smithi132:> stat /dev/vg_nvme/lv_4 2024-02-19T18:54:20.294 INFO:teuthology.orchestra.run.smithi132.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-19T18:54:20.294 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-19T18:54:20.294 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-19T18:54:20.294 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T18:54:20.294 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:device_t:s0 2024-02-19T18:54:20.294 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 2024-02-19 18:54:08.942940235 +0000 2024-02-19T18:54:20.294 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-02-19 18:46:35.243405396 +0000 2024-02-19T18:54:20.294 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-02-19 18:46:35.243405396 +0000 2024-02-19T18:54:20.294 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: 2024-02-19 18:46:35.243405396 +0000 2024-02-19T18:54:20.295 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-19T18:54:20.360 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records in 2024-02-19T18:54:20.360 INFO:teuthology.orchestra.run.smithi132.stderr:1+0 records out 2024-02-19T18:54:20.360 INFO:teuthology.orchestra.run.smithi132.stderr:512 bytes copied, 0.000308923 s, 1.7 MB/s 2024-02-19T18:54:20.361 DEBUG:teuthology.orchestra.run.smithi132:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-19T18:54:20.416 INFO:tasks.cephadm:Deploying osd.0 on smithi088 with /dev/vg_nvme/lv_4... 2024-02-19T18:54:20.417 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-19T18:54:20.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:20 smithi088 ceph-mon[29272]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:20.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:20 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:20.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:20 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:20.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:20 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:20.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:20 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.gyviqq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T18:54:20.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:20 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T18:54:20.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:20 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:20.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:20 smithi132 ceph-mon[37969]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:20.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:20 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:20.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:20 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:20.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:20 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:20.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:20 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.gyviqq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T18:54:20.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:20 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T18:54:20.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:20 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:21.417 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:54:21.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:21 smithi132 ceph-mon[37969]: Reconfiguring mgr.smithi088.gyviqq (unknown last config time)... 2024-02-19T18:54:21.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:21 smithi132 ceph-mon[37969]: Reconfiguring daemon mgr.smithi088.gyviqq on smithi088 2024-02-19T18:54:21.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:21 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:21.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:21 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:21.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:21 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T18:54:21.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:21 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:22.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:21 smithi088 ceph-mon[29272]: Reconfiguring mgr.smithi088.gyviqq (unknown last config time)... 2024-02-19T18:54:22.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:21 smithi088 ceph-mon[29272]: Reconfiguring daemon mgr.smithi088.gyviqq on smithi088 2024-02-19T18:54:22.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:21 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:22.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:21 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:22.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:21 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T18:54:22.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:21 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-19T18:54:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-19T18:54:23.164 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-02-19T18:54:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-02-19T18:54:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111162 s, 943 MB/s 2024-02-19T18:54:23.164 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 2024-02-19T18:54:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-02-19T18:54:23.185 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:22 smithi088 ceph-mon[29272]: Reconfiguring ceph-exporter.smithi088 (monmap changed)... 2024-02-19T18:54:23.185 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:22 smithi088 ceph-mon[29272]: Reconfiguring daemon ceph-exporter.smithi088 on smithi088 2024-02-19T18:54:23.185 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:22 smithi088 ceph-mon[29272]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:23.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:22 smithi132 ceph-mon[37969]: Reconfiguring ceph-exporter.smithi088 (monmap changed)... 2024-02-19T18:54:23.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:22 smithi132 ceph-mon[37969]: Reconfiguring daemon ceph-exporter.smithi088 on smithi088 2024-02-19T18:54:23.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:22 smithi132 ceph-mon[37969]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:24.146 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi088:vg_nvme/lv_4 2024-02-19T18:54:24.693 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:54:25.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:24 smithi088 ceph-mon[29272]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:25.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:24 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:25.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:24 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:25.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:24 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T18:54:25.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:24 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:25.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:24 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:54:25.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:24 smithi132 ceph-mon[37969]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:25.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:24 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:25.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:24 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:25.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:24 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T18:54:25.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:24 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:25.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:24 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:54:26.026 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:25 smithi088 ceph-mon[29272]: Reconfiguring crash.smithi088 (monmap changed)... 2024-02-19T18:54:26.026 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:25 smithi088 ceph-mon[29272]: Reconfiguring daemon crash.smithi088 on smithi088 2024-02-19T18:54:26.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:25 smithi132 ceph-mon[37969]: Reconfiguring crash.smithi088 (monmap changed)... 2024-02-19T18:54:26.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:25 smithi132 ceph-mon[37969]: Reconfiguring daemon crash.smithi088 on smithi088 2024-02-19T18:54:26.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.390+0000 7f3324a37700 1 -- 172.21.15.88:0/635144033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33201003d0 msgr2=0x7f3320100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:26.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.390+0000 7f3324a37700 1 --2- 172.21.15.88:0/635144033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33201003d0 0x7f3320100840 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3314009910 tx=0x7f3314009c20 comp rx=0 tx=0).stop 2024-02-19T18:54:26.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.390+0000 7f3324a37700 1 -- 172.21.15.88:0/635144033 shutdown_connections 2024-02-19T18:54:26.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.390+0000 7f3324a37700 1 --2- 172.21.15.88:0/635144033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33201003d0 0x7f3320100840 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:26.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.390+0000 7f3324a37700 1 --2- 172.21.15.88:0/635144033 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33200ff130 0x7f33200ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:26.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.390+0000 7f3324a37700 1 -- 172.21.15.88:0/635144033 >> 172.21.15.88:0/635144033 conn(0x7f33200fa780 msgr2=0x7f33200fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.390+0000 7f3324a37700 1 -- 172.21.15.88:0/635144033 shutdown_connections 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.390+0000 7f3324a37700 1 -- 172.21.15.88:0/635144033 wait complete. 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f3324a37700 1 Processor -- start 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f3324a37700 1 -- start start 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f3324a37700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33200ff130 0x7f33201949f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f3324a37700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33201003d0 0x7f3320194f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f3324a37700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3320195550 con 0x7f33200ff130 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f3324a37700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3320195690 con 0x7f33201003d0 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f331e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33200ff130 0x7f33201949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f331e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33200ff130 0x7f33201949f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48332/0 (socket says 172.21.15.88:48332) 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f331e59c700 1 -- 172.21.15.88:0/4020838072 learned_addr learned my addr 172.21.15.88:0/4020838072 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:54:26.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f331e59c700 1 -- 172.21.15.88:0/4020838072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33201003d0 msgr2=0x7f3320194f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f331e59c700 1 --2- 172.21.15.88:0/4020838072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33201003d0 0x7f3320194f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.391+0000 7f331e59c700 1 -- 172.21.15.88:0/4020838072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33140095f0 con 0x7f33200ff130 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.392+0000 7f331e59c700 1 --2- 172.21.15.88:0/4020838072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33200ff130 0x7f33201949f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f331000b4b0 tx=0x7f331000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.392+0000 7f330f7fe700 1 -- 172.21.15.88:0/4020838072 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3310011800 con 0x7f33200ff130 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.392+0000 7f330f7fe700 1 -- 172.21.15.88:0/4020838072 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3310011e10 con 0x7f33200ff130 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.392+0000 7f3324a37700 1 -- 172.21.15.88:0/4020838072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332019a140 con 0x7f33200ff130 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.392+0000 7f330f7fe700 1 -- 172.21.15.88:0/4020838072 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f331000f2e0 con 0x7f33200ff130 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.392+0000 7f3324a37700 1 -- 172.21.15.88:0/4020838072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332019a690 con 0x7f33200ff130 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.393+0000 7f3324a37700 1 -- 172.21.15.88:0/4020838072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3320107c90 con 0x7f33200ff130 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.393+0000 7f330f7fe700 1 -- 172.21.15.88:0/4020838072 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f3310011960 con 0x7f33200ff130 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.393+0000 7f330f7fe700 1 --2- 172.21.15.88:0/4020838072 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330806c610 0x7f330806eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.394+0000 7f330f7fe700 1 -- 172.21.15.88:0/4020838072 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f331008a3d0 con 0x7f33200ff130 2024-02-19T18:54:26.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.394+0000 7f331dd9b700 1 --2- 172.21.15.88:0/4020838072 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330806c610 0x7f330806eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:54:26.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.394+0000 7f331dd9b700 1 --2- 172.21.15.88:0/4020838072 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330806c610 0x7f330806eac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f331400b5c0 tx=0x7f3314005fd0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:54:26.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.398+0000 7f330f7fe700 1 -- 172.21.15.88:0/4020838072 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3310059620 con 0x7f33200ff130 2024-02-19T18:54:26.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:26.599+0000 7f3324a37700 1 -- 172.21.15.88:0/4020838072 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3320104d20 con 0x7f330806c610 2024-02-19T18:54:27.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:26 smithi132 ceph-mon[37969]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:27.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:26 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:54:27.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:26 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:54:27.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:26 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:27.309 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:26 smithi088 ceph-mon[29272]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:27.309 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:26 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:54:27.310 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:26 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:54:27.310 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:26 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:28.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:27 smithi088 ceph-mon[29272]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:54:28.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:27 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:28.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:27 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:28.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:27 smithi132 ceph-mon[37969]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:54:28.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:27 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:28.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:27 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:29.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:28 smithi132 ceph-mon[37969]: Reconfiguring alertmanager.smithi088 (dependencies changed)... 2024-02-19T18:54:29.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:28 smithi132 ceph-mon[37969]: Reconfiguring daemon alertmanager.smithi088 on smithi088 2024-02-19T18:54:29.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:28 smithi132 ceph-mon[37969]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:29.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:28 smithi088 ceph-mon[29272]: Reconfiguring alertmanager.smithi088 (dependencies changed)... 2024-02-19T18:54:29.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:28 smithi088 ceph-mon[29272]: Reconfiguring daemon alertmanager.smithi088 on smithi088 2024-02-19T18:54:29.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:28 smithi088 ceph-mon[29272]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:31.033 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:30 smithi088 ceph-mon[29272]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:31.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:30 smithi132 ceph-mon[37969]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:33.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:32 smithi132 ceph-mon[37969]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:33.275 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:32 smithi088 ceph-mon[29272]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:34.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:34.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:34.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:34 smithi088 ceph-mon[29272]: Reconfiguring grafana.smithi088 (dependencies changed)... 2024-02-19T18:54:34.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:34 smithi088 ceph-mon[29272]: Reconfiguring daemon grafana.smithi088 on smithi088 2024-02-19T18:54:34.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:34 smithi088 ceph-mon[29272]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:34.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:34.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:34.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:34 smithi132 ceph-mon[37969]: Reconfiguring grafana.smithi088 (dependencies changed)... 2024-02-19T18:54:34.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:34 smithi132 ceph-mon[37969]: Reconfiguring daemon grafana.smithi088 on smithi088 2024-02-19T18:54:34.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:34 smithi132 ceph-mon[37969]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:35.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:35 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1070334162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "787c5fec-c403-4975-92c3-8b68fcf9e399"}]: dispatch 2024-02-19T18:54:35.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:35 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1070334162' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "787c5fec-c403-4975-92c3-8b68fcf9e399"}]': finished 2024-02-19T18:54:35.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:35 smithi088 ceph-mon[29272]: osdmap e6: 1 total, 0 up, 1 in 2024-02-19T18:54:35.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:35 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T18:54:35.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:35 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/4203037903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:54:35.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:35 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1070334162' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "787c5fec-c403-4975-92c3-8b68fcf9e399"}]: dispatch 2024-02-19T18:54:35.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:35 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1070334162' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "787c5fec-c403-4975-92c3-8b68fcf9e399"}]': finished 2024-02-19T18:54:35.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:35 smithi132 ceph-mon[37969]: osdmap e6: 1 total, 0 up, 1 in 2024-02-19T18:54:35.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:35 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T18:54:35.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:35 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/4203037903' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:54:36.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:36 smithi132 ceph-mon[37969]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:36.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:36 smithi088 ceph-mon[29272]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:38.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:38 smithi088 ceph-mon[29272]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:39.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:38 smithi132 ceph-mon[37969]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:40.046 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:39 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:54:40.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:39 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:54:41.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:40 smithi132 ceph-mon[37969]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:41.297 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:40 smithi088 ceph-mon[29272]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:43.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:42 smithi088 ceph-mon[29272]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:43.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:42 smithi132 ceph-mon[37969]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:44.495 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:44.495 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:44.495 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:44 smithi088 ceph-mon[29272]: Reconfiguring prometheus.smithi088 (dependencies changed)... 2024-02-19T18:54:44.495 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:44 smithi088 ceph-mon[29272]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:44.495 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:44 smithi088 ceph-mon[29272]: Reconfiguring daemon prometheus.smithi088 on smithi088 2024-02-19T18:54:44.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:44 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:44.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:44 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:44.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:44 smithi132 ceph-mon[37969]: Reconfiguring prometheus.smithi088 (dependencies changed)... 2024-02-19T18:54:44.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:44 smithi132 ceph-mon[37969]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:44.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:44 smithi132 ceph-mon[37969]: Reconfiguring daemon prometheus.smithi088 on smithi088 2024-02-19T18:54:45.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:45 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-19T18:54:45.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:45 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:45.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:45 smithi088 ceph-mon[29272]: Deploying daemon osd.0 on smithi088 2024-02-19T18:54:45.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:45 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-19T18:54:45.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:45 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:45.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:45 smithi132 ceph-mon[37969]: Deploying daemon osd.0 on smithi088 2024-02-19T18:54:46.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:46 smithi088 ceph-mon[29272]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:46.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:46 smithi132 ceph-mon[37969]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:49.244 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:48 smithi088 ceph-mon[29272]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:49.248 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:48 smithi132 ceph-mon[37969]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:50.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:50 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:50.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:50 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:50.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:50 smithi088 ceph-mon[29272]: Reconfiguring ceph-exporter.smithi132 (monmap changed)... 2024-02-19T18:54:50.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:50 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T18:54:50.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:50 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:50.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:50 smithi088 ceph-mon[29272]: Reconfiguring daemon ceph-exporter.smithi132 on smithi132 2024-02-19T18:54:50.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:50 smithi088 ceph-mon[29272]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:50.417 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:50 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:50.417 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:50 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:50.417 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:50 smithi132 ceph-mon[37969]: Reconfiguring ceph-exporter.smithi132 (monmap changed)... 2024-02-19T18:54:50.417 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:50 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T18:54:50.417 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:50 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:50.417 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:50 smithi132 ceph-mon[37969]: Reconfiguring daemon ceph-exporter.smithi132 on smithi132 2024-02-19T18:54:50.417 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:50 smithi132 ceph-mon[37969]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:51.508 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:51.508 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:51.508 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:51 smithi088 ceph-mon[29272]: Reconfiguring crash.smithi132 (monmap changed)... 2024-02-19T18:54:51.508 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi132", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T18:54:51.508 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:51.508 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:51 smithi088 ceph-mon[29272]: Reconfiguring daemon crash.smithi132 on smithi132 2024-02-19T18:54:51.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:51 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:51.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:51 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:51.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:51 smithi132 ceph-mon[37969]: Reconfiguring crash.smithi132 (monmap changed)... 2024-02-19T18:54:51.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:51 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi132", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T18:54:51.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:51 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:51.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:51 smithi132 ceph-mon[37969]: Reconfiguring daemon crash.smithi132 on smithi132 2024-02-19T18:54:53.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:52 smithi132 ceph-mon[37969]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:53.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:53.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:53.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:52 smithi132 ceph-mon[37969]: Reconfiguring mgr.smithi132.psllsa (monmap changed)... 2024-02-19T18:54:53.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T18:54:53.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T18:54:53.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:53.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:52 smithi132 ceph-mon[37969]: Reconfiguring daemon mgr.smithi132.psllsa on smithi132 2024-02-19T18:54:53.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:52 smithi088 ceph-mon[29272]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:53.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:53.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:53.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:52 smithi088 ceph-mon[29272]: Reconfiguring mgr.smithi132.psllsa (monmap changed)... 2024-02-19T18:54:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T18:54:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T18:54:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:52 smithi088 ceph-mon[29272]: Reconfiguring daemon mgr.smithi132.psllsa on smithi132 2024-02-19T18:54:54.458 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:54.458 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:54.458 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:54 smithi088 ceph-mon[29272]: Reconfiguring mon.smithi132 (monmap changed)... 2024-02-19T18:54:54.458 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T18:54:54.458 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T18:54:54.458 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:54.458 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:54 smithi088 ceph-mon[29272]: Reconfiguring daemon mon.smithi132 on smithi132 2024-02-19T18:54:54.458 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:54 smithi088 ceph-mon[29272]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:54.458 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:54.459 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:54.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:54.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:54.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:54 smithi132 ceph-mon[37969]: Reconfiguring mon.smithi132 (monmap changed)... 2024-02-19T18:54:54.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T18:54:54.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T18:54:54.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:54:54.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:54 smithi132 ceph-mon[37969]: Reconfiguring daemon mon.smithi132 on smithi132 2024-02-19T18:54:54.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:54 smithi132 ceph-mon[37969]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:54.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:54.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:55.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:54:55.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:55.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:55.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-19T18:54:55.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-19T18:54:55.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi088.front.sepia.ceph.com:9093"}]: dispatch 2024-02-19T18:54:55.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi088.front.sepia.ceph.com:9093"}]: dispatch 2024-02-19T18:54:55.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:55.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-19T18:54:55.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-19T18:54:55.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi088.front.sepia.ceph.com:3000"}]: dispatch 2024-02-19T18:54:55.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi088.front.sepia.ceph.com:3000"}]: dispatch 2024-02-19T18:54:55.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:55.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T18:54:55.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T18:54:55.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi088.front.sepia.ceph.com:9095"}]: dispatch 2024-02-19T18:54:55.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi088.front.sepia.ceph.com:9095"}]: dispatch 2024-02-19T18:54:55.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:55.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:54:55.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:54:55.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:55.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:55.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-19T18:54:55.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-19T18:54:55.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi088.front.sepia.ceph.com:9093"}]: dispatch 2024-02-19T18:54:55.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi088.front.sepia.ceph.com:9093"}]: dispatch 2024-02-19T18:54:55.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:55.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-19T18:54:55.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-19T18:54:55.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi088.front.sepia.ceph.com:3000"}]: dispatch 2024-02-19T18:54:55.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi088.front.sepia.ceph.com:3000"}]: dispatch 2024-02-19T18:54:55.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:55.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T18:54:55.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T18:54:55.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi088.front.sepia.ceph.com:9095"}]: dispatch 2024-02-19T18:54:55.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi088.front.sepia.ceph.com:9095"}]: dispatch 2024-02-19T18:54:55.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:55.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:54:56.361 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:56 smithi132 ceph-mon[37969]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:56.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:56 smithi088 ceph-mon[29272]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:57.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.027+0000 7f330f7fe700 1 -- 172.21.15.88:0/4020838072 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3320104d20 con 0x7f330806c610 2024-02-19T18:54:57.045 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 0 on host 'smithi088' 2024-02-19T18:54:57.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.029+0000 7f3324a37700 1 -- 172.21.15.88:0/4020838072 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330806c610 msgr2=0x7f330806eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:57.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.029+0000 7f3324a37700 1 --2- 172.21.15.88:0/4020838072 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330806c610 0x7f330806eac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f331400b5c0 tx=0x7f3314005fd0 comp rx=0 tx=0).stop 2024-02-19T18:54:57.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.029+0000 7f3324a37700 1 -- 172.21.15.88:0/4020838072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33200ff130 msgr2=0x7f33201949f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:54:57.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.029+0000 7f3324a37700 1 --2- 172.21.15.88:0/4020838072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33200ff130 0x7f33201949f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f331000b4b0 tx=0x7f331000b870 comp rx=0 tx=0).stop 2024-02-19T18:54:57.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.030+0000 7f3324a37700 1 -- 172.21.15.88:0/4020838072 shutdown_connections 2024-02-19T18:54:57.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.030+0000 7f3324a37700 1 --2- 172.21.15.88:0/4020838072 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330806c610 0x7f330806eac0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:57.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.030+0000 7f3324a37700 1 --2- 172.21.15.88:0/4020838072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33200ff130 0x7f33201949f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:57.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.030+0000 7f3324a37700 1 --2- 172.21.15.88:0/4020838072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33201003d0 0x7f3320194f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:54:57.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.030+0000 7f3324a37700 1 -- 172.21.15.88:0/4020838072 >> 172.21.15.88:0/4020838072 conn(0x7f33200fa780 msgr2=0x7f3320103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:54:57.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.030+0000 7f3324a37700 1 -- 172.21.15.88:0/4020838072 shutdown_connections 2024-02-19T18:54:57.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:54:57.030+0000 7f3324a37700 1 -- 172.21.15.88:0/4020838072 wait complete. 2024-02-19T18:54:57.629 DEBUG:teuthology.orchestra.run.smithi088:osd.0> sudo journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.0.service 2024-02-19T18:54:57.631 INFO:tasks.cephadm:Deploying osd.1 on smithi088 with /dev/vg_nvme/lv_3... 2024-02-19T18:54:57.631 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-19T18:54:57.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:57 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:57.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:57 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:57.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:57 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:57.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:57 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:57.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:57 smithi132 ceph-mon[37969]: from='osd.0 [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-19T18:54:58.000 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:57 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:58.000 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:57 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:58.000 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:57 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:58.000 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:57 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:54:58.000 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:57 smithi088 ceph-mon[29272]: from='osd.0 [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-19T18:54:58.796 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:54:58.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:58 smithi132 ceph-mon[37969]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:58.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:58 smithi132 ceph-mon[37969]: from='osd.0 [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-19T18:54:58.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:58 smithi132 ceph-mon[37969]: osdmap e7: 1 total, 0 up, 1 in 2024-02-19T18:54:58.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:58 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T18:54:58.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:54:58 smithi132 ceph-mon[37969]: from='osd.0 [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T18:54:59.028 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:58 smithi088 ceph-mon[29272]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:54:59.028 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:58 smithi088 ceph-mon[29272]: from='osd.0 [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-19T18:54:59.028 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:58 smithi088 ceph-mon[29272]: osdmap e7: 1 total, 0 up, 1 in 2024-02-19T18:54:59.028 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:58 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T18:54:59.028 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:54:58 smithi088 ceph-mon[29272]: from='osd.0 [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T18:54:59.347 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 18:54:59 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[44165]: 2024-02-19T18:54:59.012+0000 7f11bc5f6700 -1 osd.0 0 waiting for initial osdmap 2024-02-19T18:54:59.347 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 18:54:59 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[44165]: 2024-02-19T18:54:59.015+0000 7f11b7bea700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T18:55:00.278 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:00 smithi088 ceph-mon[29272]: from='osd.0 [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-02-19T18:55:00.278 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:00 smithi088 ceph-mon[29272]: osdmap e8: 1 total, 0 up, 1 in 2024-02-19T18:55:00.278 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:00 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T18:55:00.278 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:00 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T18:55:00.278 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:00 smithi088 ceph-mon[29272]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:55:00.359 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:00 smithi132 ceph-mon[37969]: from='osd.0 [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-02-19T18:55:00.359 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:00 smithi132 ceph-mon[37969]: osdmap e8: 1 total, 0 up, 1 in 2024-02-19T18:55:00.359 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:00 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T18:55:00.359 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:00 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T18:55:00.359 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:00 smithi132 ceph-mon[37969]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-19T18:55:01.185 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-19T18:55:01.185 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-19T18:55:01.185 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-02-19T18:55:01.186 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-02-19T18:55:01.186 INFO:teuthology.orchestra.run.smithi088.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0123793 s, 847 MB/s 2024-02-19T18:55:01.186 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 2024-02-19T18:55:01.186 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-02-19T18:55:01.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:01 smithi132 ceph-mon[37969]: purged_snaps scrub starts 2024-02-19T18:55:01.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:01 smithi132 ceph-mon[37969]: purged_snaps scrub ok 2024-02-19T18:55:01.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:01 smithi132 ceph-mon[37969]: osd.0 [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937] boot 2024-02-19T18:55:01.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:01 smithi132 ceph-mon[37969]: osdmap e9: 1 total, 1 up, 1 in 2024-02-19T18:55:01.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T18:55:01.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:01 smithi088 ceph-mon[29272]: purged_snaps scrub starts 2024-02-19T18:55:01.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:01 smithi088 ceph-mon[29272]: purged_snaps scrub ok 2024-02-19T18:55:01.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:01 smithi088 ceph-mon[29272]: osd.0 [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937] boot 2024-02-19T18:55:01.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:01 smithi088 ceph-mon[29272]: osdmap e9: 1 total, 1 up, 1 in 2024-02-19T18:55:01.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T18:55:01.730 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi088:vg_nvme/lv_3 2024-02-19T18:55:02.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:02 smithi088 ceph-mon[29272]: osdmap e10: 1 total, 1 up, 1 in 2024-02-19T18:55:02.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:02 smithi088 ceph-mon[29272]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:02.450 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:55:02.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:02 smithi132 ceph-mon[37969]: osdmap e10: 1 total, 1 up, 1 in 2024-02-19T18:55:02.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:02 smithi132 ceph-mon[37969]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:03.525 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:03 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:03.525 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:03 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:03.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:03 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:03.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:03 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:04.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.652+0000 7f77ef921700 1 -- 172.21.15.88:0/4048472277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e80feb50 msgr2=0x7f77e8100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.652+0000 7f77ef921700 1 --2- 172.21.15.88:0/4048472277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e80feb50 0x7f77e8100f70 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f77e4009910 tx=0x7f77e4009c20 comp rx=0 tx=0).stop 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.652+0000 7f77ef921700 1 -- 172.21.15.88:0/4048472277 shutdown_connections 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.652+0000 7f77ef921700 1 --2- 172.21.15.88:0/4048472277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77e81014b0 0x7f77e8103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.652+0000 7f77ef921700 1 --2- 172.21.15.88:0/4048472277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e80feb50 0x7f77e8100f70 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.652+0000 7f77ef921700 1 -- 172.21.15.88:0/4048472277 >> 172.21.15.88:0/4048472277 conn(0x7f77e80fa780 msgr2=0x7f77e80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.652+0000 7f77ef921700 1 -- 172.21.15.88:0/4048472277 shutdown_connections 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.652+0000 7f77ef921700 1 -- 172.21.15.88:0/4048472277 wait complete. 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ef921700 1 Processor -- start 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ef921700 1 -- start start 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ef921700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77e80feb50 0x7f77e819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ef921700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81014b0 0x7f77e819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:55:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ef921700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e819ccb0 con 0x7f77e81014b0 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ef921700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e819cdf0 con 0x7f77e80feb50 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ecebc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81014b0 0x7f77e819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ecebc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81014b0 0x7f77e819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35590/0 (socket says 172.21.15.88:35590) 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ecebc700 1 -- 172.21.15.88:0/2513757714 learned_addr learned my addr 172.21.15.88:0/2513757714 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ed6bd700 1 --2- 172.21.15.88:0/2513757714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77e80feb50 0x7f77e819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ecebc700 1 -- 172.21.15.88:0/2513757714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77e80feb50 msgr2=0x7f77e819c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ecebc700 1 --2- 172.21.15.88:0/2513757714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77e80feb50 0x7f77e819c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ecebc700 1 -- 172.21.15.88:0/2513757714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e40095f0 con 0x7f77e81014b0 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.653+0000 7f77ecebc700 1 --2- 172.21.15.88:0/2513757714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81014b0 0x7f77e819c690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f77d800b4b0 tx=0x7f77d800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.654+0000 7f77de7fc700 1 -- 172.21.15.88:0/2513757714 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d8011800 con 0x7f77e81014b0 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.654+0000 7f77de7fc700 1 -- 172.21.15.88:0/2513757714 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f77d8011e10 con 0x7f77e81014b0 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.654+0000 7f77ef921700 1 -- 172.21.15.88:0/2513757714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e81a18a0 con 0x7f77e81014b0 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.654+0000 7f77de7fc700 1 -- 172.21.15.88:0/2513757714 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d800f2e0 con 0x7f77e81014b0 2024-02-19T18:55:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.654+0000 7f77ef921700 1 -- 172.21.15.88:0/2513757714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e81a1df0 con 0x7f77e81014b0 2024-02-19T18:55:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.654+0000 7f77ef921700 1 -- 172.21.15.88:0/2513757714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e8196240 con 0x7f77e81014b0 2024-02-19T18:55:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.655+0000 7f77de7fc700 1 -- 172.21.15.88:0/2513757714 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f77d8010380 con 0x7f77e81014b0 2024-02-19T18:55:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.655+0000 7f77de7fc700 1 --2- 172.21.15.88:0/2513757714 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f77d406c600 0x7f77d406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:55:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.655+0000 7f77de7fc700 1 -- 172.21.15.88:0/2513757714 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f77d808a5a0 con 0x7f77e81014b0 2024-02-19T18:55:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.655+0000 7f77ed6bd700 1 --2- 172.21.15.88:0/2513757714 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f77d406c600 0x7f77d406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:55:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.656+0000 7f77ed6bd700 1 --2- 172.21.15.88:0/2513757714 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f77d406c600 0x7f77d406eab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f77e4006010 tx=0x7f77e4005280 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:55:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.660+0000 7f77de7fc700 1 -- 172.21.15.88:0/2513757714 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f77d8059590 con 0x7f77e81014b0 2024-02-19T18:55:04.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:04 smithi132 ceph-mon[37969]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:04.793 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:04 smithi088 ceph-mon[29272]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:04.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:04.868+0000 7f77ef921700 1 -- 172.21.15.88:0/2513757714 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f77e8061190 con 0x7f77d406c600 2024-02-19T18:55:05.715 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:05 smithi088 ceph-mon[29272]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:55:05.715 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:05 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:55:05.715 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:05 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:55:05.716 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:05 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:05.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:05 smithi132 ceph-mon[37969]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:55:05.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:05 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:55:05.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:05 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:55:05.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:05 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:06.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:06 smithi132 ceph-mon[37969]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:06.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:06 smithi088 ceph-mon[29272]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:08.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:08 smithi088 ceph-mon[29272]: Detected new or changed devices on smithi088 2024-02-19T18:55:08.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:08 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:08.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:08 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:08.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:08 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:55:08.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:08 smithi088 ceph-mon[29272]: Adjusting osd_memory_target on smithi088 to 11840M 2024-02-19T18:55:08.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:08 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:08.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:08 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:08.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:08 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:55:08.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:08 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:08.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:08 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:55:08.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:08 smithi088 ceph-mon[29272]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:08.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:08 smithi132 ceph-mon[37969]: Detected new or changed devices on smithi088 2024-02-19T18:55:08.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:08 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:08.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:08 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:08.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:08 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:55:08.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:08 smithi132 ceph-mon[37969]: Adjusting osd_memory_target on smithi088 to 11840M 2024-02-19T18:55:08.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:08 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:08.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:08 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:08.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:08 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:55:08.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:08 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:08.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:08 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:55:08.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:08 smithi132 ceph-mon[37969]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:09.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:09 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1379052684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a459843c-43d7-480f-a76b-d52d6e8ca061"}]: dispatch 2024-02-19T18:55:09.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:09 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1379052684' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a459843c-43d7-480f-a76b-d52d6e8ca061"}]': finished 2024-02-19T18:55:09.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:09 smithi132 ceph-mon[37969]: osdmap e11: 2 total, 1 up, 2 in 2024-02-19T18:55:09.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:09 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T18:55:09.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:09 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/4160234518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:55:09.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:09 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:09.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:09 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:09.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:09 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1379052684' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a459843c-43d7-480f-a76b-d52d6e8ca061"}]: dispatch 2024-02-19T18:55:09.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:09 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1379052684' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a459843c-43d7-480f-a76b-d52d6e8ca061"}]': finished 2024-02-19T18:55:09.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:09 smithi088 ceph-mon[29272]: osdmap e11: 2 total, 1 up, 2 in 2024-02-19T18:55:09.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:09 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T18:55:09.835 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:09 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/4160234518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:55:09.835 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:09 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:09.835 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:09 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:10.708 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:10 smithi088 ceph-mon[29272]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:10.709 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:10 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:55:10.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:10 smithi132 ceph-mon[37969]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:10.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:55:12.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:12 smithi088 ceph-mon[29272]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:12.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:12 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:12.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:12 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:12.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:12 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:12.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:12 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:55:12.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:12 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:12.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:12 smithi132 ceph-mon[37969]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:13.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:12 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:13.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:12 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:13.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:12 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:13.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:12 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:55:13.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:12 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:14.904 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:14 smithi088 ceph-mon[29272]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:14.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:14 smithi132 ceph-mon[37969]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:16.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:16 smithi132 ceph-mon[37969]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:16.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-19T18:55:16.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:17.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:16 smithi132 ceph-mon[37969]: Deploying daemon osd.1 on smithi088 2024-02-19T18:55:17.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:16 smithi088 ceph-mon[29272]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:17.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-19T18:55:17.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:17.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:16 smithi088 ceph-mon[29272]: Deploying daemon osd.1 on smithi088 2024-02-19T18:55:19.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:18 smithi088 ceph-mon[29272]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:19.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:18 smithi132 ceph-mon[37969]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:21.075 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:20 smithi088 ceph-mon[29272]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:21.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:20 smithi132 ceph-mon[37969]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:22.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:22 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:22.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:22 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:22.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:22 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:55:22.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:22 smithi088 ceph-mon[29272]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:22.672 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:22 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:22.672 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:22 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:22.672 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:22 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:55:22.672 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:22 smithi132 ceph-mon[37969]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:24.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:24 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:24.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:24 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:24.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:24 smithi088 ceph-mon[29272]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:24.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:24 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:24.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:24 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:24.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:24 smithi132 ceph-mon[37969]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:25.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.023+0000 7f77de7fc700 1 -- 172.21.15.88:0/2513757714 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f77e8061190 con 0x7f77d406c600 2024-02-19T18:55:25.040 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 1 on host 'smithi088' 2024-02-19T18:55:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.025+0000 7f77ef921700 1 -- 172.21.15.88:0/2513757714 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f77d406c600 msgr2=0x7f77d406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:55:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.025+0000 7f77ef921700 1 --2- 172.21.15.88:0/2513757714 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f77d406c600 0x7f77d406eab0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f77e4006010 tx=0x7f77e4005280 comp rx=0 tx=0).stop 2024-02-19T18:55:25.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.025+0000 7f77ef921700 1 -- 172.21.15.88:0/2513757714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81014b0 msgr2=0x7f77e819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:55:25.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.025+0000 7f77ef921700 1 --2- 172.21.15.88:0/2513757714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81014b0 0x7f77e819c690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f77d800b4b0 tx=0x7f77d800b7c0 comp rx=0 tx=0).stop 2024-02-19T18:55:25.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.025+0000 7f77ef921700 1 -- 172.21.15.88:0/2513757714 shutdown_connections 2024-02-19T18:55:25.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.025+0000 7f77ef921700 1 --2- 172.21.15.88:0/2513757714 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f77d406c600 0x7f77d406eab0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:25.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.025+0000 7f77ef921700 1 --2- 172.21.15.88:0/2513757714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77e80feb50 0x7f77e819c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:25.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.025+0000 7f77ef921700 1 --2- 172.21.15.88:0/2513757714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81014b0 0x7f77e819c690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:25.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.025+0000 7f77ef921700 1 -- 172.21.15.88:0/2513757714 >> 172.21.15.88:0/2513757714 conn(0x7f77e80fa780 msgr2=0x7f77e80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:55:25.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.026+0000 7f77ef921700 1 -- 172.21.15.88:0/2513757714 shutdown_connections 2024-02-19T18:55:25.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:25.026+0000 7f77ef921700 1 -- 172.21.15.88:0/2513757714 wait complete. 2024-02-19T18:55:25.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:55:25.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:25.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:25.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:25 smithi088 ceph-mon[29272]: from='osd.1 [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-19T18:55:25.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:55:25.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:25.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:25.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:25 smithi132 ceph-mon[37969]: from='osd.1 [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-19T18:55:26.144 DEBUG:teuthology.orchestra.run.smithi088:osd.1> sudo journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.1.service 2024-02-19T18:55:26.146 INFO:tasks.cephadm:Deploying osd.2 on smithi088 with /dev/vg_nvme/lv_2... 2024-02-19T18:55:26.147 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-19T18:55:26.836 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:55:27.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:26 smithi088 ceph-mon[29272]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:27.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:26 smithi088 ceph-mon[29272]: from='osd.1 [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-19T18:55:27.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:26 smithi088 ceph-mon[29272]: osdmap e12: 2 total, 1 up, 2 in 2024-02-19T18:55:27.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:26 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T18:55:27.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:26 smithi088 ceph-mon[29272]: from='osd.1 [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T18:55:27.234 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 18:55:27 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[49795]: 2024-02-19T18:55:26.999+0000 7f9d46ed2700 -1 osd.1 0 waiting for initial osdmap 2024-02-19T18:55:27.234 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 18:55:27 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[49795]: 2024-02-19T18:55:27.003+0000 7f9d414c4700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T18:55:27.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:26 smithi132 ceph-mon[37969]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:27.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:26 smithi132 ceph-mon[37969]: from='osd.1 [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-19T18:55:27.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:26 smithi132 ceph-mon[37969]: osdmap e12: 2 total, 1 up, 2 in 2024-02-19T18:55:27.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:26 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T18:55:27.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:26 smithi132 ceph-mon[37969]: from='osd.1 [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T18:55:28.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:28 smithi088 ceph-mon[29272]: from='osd.1 [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-02-19T18:55:28.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:28 smithi088 ceph-mon[29272]: osdmap e13: 2 total, 1 up, 2 in 2024-02-19T18:55:28.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:28 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T18:55:28.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:28 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T18:55:28.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:28 smithi088 ceph-mon[29272]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:28.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:28 smithi132 ceph-mon[37969]: from='osd.1 [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-02-19T18:55:28.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:28 smithi132 ceph-mon[37969]: osdmap e13: 2 total, 1 up, 2 in 2024-02-19T18:55:28.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:28 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T18:55:28.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:28 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T18:55:28.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:28 smithi132 ceph-mon[37969]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-19T18:55:29.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:29 smithi088 ceph-mon[29272]: purged_snaps scrub starts 2024-02-19T18:55:29.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:29 smithi088 ceph-mon[29272]: purged_snaps scrub ok 2024-02-19T18:55:29.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:29 smithi088 ceph-mon[29272]: osd.1 [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945] boot 2024-02-19T18:55:29.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:29 smithi088 ceph-mon[29272]: osdmap e14: 2 total, 2 up, 2 in 2024-02-19T18:55:29.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:29 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T18:55:29.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:29 smithi132 ceph-mon[37969]: purged_snaps scrub starts 2024-02-19T18:55:29.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:29 smithi132 ceph-mon[37969]: purged_snaps scrub ok 2024-02-19T18:55:29.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:29 smithi132 ceph-mon[37969]: osd.1 [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945] boot 2024-02-19T18:55:29.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:29 smithi132 ceph-mon[37969]: osdmap e14: 2 total, 2 up, 2 in 2024-02-19T18:55:29.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:29 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T18:55:29.744 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-19T18:55:29.744 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-19T18:55:29.744 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-02-19T18:55:29.744 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-02-19T18:55:29.745 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0122473 s, 856 MB/s 2024-02-19T18:55:29.745 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-02-19T18:55:30.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:30 smithi132 ceph-mon[37969]: osdmap e15: 2 total, 2 up, 2 in 2024-02-19T18:55:30.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:30 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:30.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:30 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:30.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:30 smithi132 ceph-mon[37969]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:30.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:30 smithi088 ceph-mon[29272]: osdmap e15: 2 total, 2 up, 2 in 2024-02-19T18:55:30.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:30 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:30.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:30 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:30.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:30 smithi088 ceph-mon[29272]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:30.989 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi088:vg_nvme/lv_2 2024-02-19T18:55:31.941 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:55:33.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:32 smithi132 ceph-mon[37969]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:33.263 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:32 smithi088 ceph-mon[29272]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.973+0000 7f50af935700 1 -- 172.21.15.88:0/3741838677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a8100ce0 msgr2=0x7f50a81030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:55:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.973+0000 7f50af935700 1 --2- 172.21.15.88:0/3741838677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a8100ce0 0x7f50a81030c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009910 tx=0x7f50a4009c20 comp rx=0 tx=0).stop 2024-02-19T18:55:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.973+0000 7f50af935700 1 -- 172.21.15.88:0/3741838677 shutdown_connections 2024-02-19T18:55:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.973+0000 7f50af935700 1 --2- 172.21.15.88:0/3741838677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50a8103600 0x7f50a81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.973+0000 7f50af935700 1 --2- 172.21.15.88:0/3741838677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a8100ce0 0x7f50a81030c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.973+0000 7f50af935700 1 -- 172.21.15.88:0/3741838677 >> 172.21.15.88:0/3741838677 conn(0x7f50a80fa700 msgr2=0x7f50a80fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:55:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.973+0000 7f50af935700 1 -- 172.21.15.88:0/3741838677 shutdown_connections 2024-02-19T18:55:33.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.973+0000 7f50af935700 1 -- 172.21.15.88:0/3741838677 wait complete. 2024-02-19T18:55:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.973+0000 7f50af935700 1 Processor -- start 2024-02-19T18:55:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.973+0000 7f50af935700 1 -- start start 2024-02-19T18:55:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50af935700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50a8100ce0 0x7f50a8100640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:55:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50af935700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a8103600 0x7f50a80fec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:55:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50af935700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a80ff1d0 con 0x7f50a8103600 2024-02-19T18:55:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50af935700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a80ff310 con 0x7f50a8100ce0 2024-02-19T18:55:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50aced0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a8103600 0x7f50a80fec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:55:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50aced0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a8103600 0x7f50a80fec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36892/0 (socket says 172.21.15.88:36892) 2024-02-19T18:55:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50aced0700 1 -- 172.21.15.88:0/1819315104 learned_addr learned my addr 172.21.15.88:0/1819315104 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:55:33.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50ad6d1700 1 --2- 172.21.15.88:0/1819315104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50a8100ce0 0x7f50a8100640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:55:33.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50aced0700 1 -- 172.21.15.88:0/1819315104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50a8100ce0 msgr2=0x7f50a8100640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:55:33.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50aced0700 1 --2- 172.21.15.88:0/1819315104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50a8100ce0 0x7f50a8100640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:33.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50aced0700 1 -- 172.21.15.88:0/1819315104 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a40095f0 con 0x7f50a8103600 2024-02-19T18:55:33.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50aced0700 1 --2- 172.21.15.88:0/1819315104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a8103600 0x7f50a80fec90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f509800d8d0 tx=0x7f509800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:55:33.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f509e7fc700 1 -- 172.21.15.88:0/1819315104 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50980098a0 con 0x7f50a8103600 2024-02-19T18:55:33.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50af935700 1 -- 172.21.15.88:0/1819315104 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a80ff5f0 con 0x7f50a8103600 2024-02-19T18:55:33.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f509e7fc700 1 -- 172.21.15.88:0/1819315104 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5098010460 con 0x7f50a8103600 2024-02-19T18:55:33.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f509e7fc700 1 -- 172.21.15.88:0/1819315104 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f509800f5f0 con 0x7f50a8103600 2024-02-19T18:55:33.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.974+0000 7f50af935700 1 -- 172.21.15.88:0/1819315104 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a819db10 con 0x7f50a8103600 2024-02-19T18:55:33.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.975+0000 7f50af935700 1 -- 172.21.15.88:0/1819315104 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a818fc70 con 0x7f50a8103600 2024-02-19T18:55:33.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.976+0000 7f509e7fc700 1 -- 172.21.15.88:0/1819315104 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f5098009a00 con 0x7f50a8103600 2024-02-19T18:55:33.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.976+0000 7f509e7fc700 1 --2- 172.21.15.88:0/1819315104 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f509406c600 0x7f509406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:55:33.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.976+0000 7f509e7fc700 1 -- 172.21.15.88:0/1819315104 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f509808a9b0 con 0x7f50a8103600 2024-02-19T18:55:33.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.976+0000 7f50ad6d1700 1 --2- 172.21.15.88:0/1819315104 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f509406c600 0x7f509406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:55:33.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.977+0000 7f50ad6d1700 1 --2- 172.21.15.88:0/1819315104 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f509406c600 0x7f509406eab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f50a400b5c0 tx=0x7f50a4005ee0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:55:33.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:33.981+0000 7f509e7fc700 1 -- 172.21.15.88:0/1819315104 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5098059870 con 0x7f50a8103600 2024-02-19T18:55:34.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:34.205+0000 7f50af935700 1 -- 172.21.15.88:0/1819315104 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f50a8061190 con 0x7f509406c600 2024-02-19T18:55:34.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: Detected new or changed devices on smithi088 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: Adjusting osd_memory_target on smithi088 to 5920M 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:55:34.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:55:34.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:34 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:34.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: Detected new or changed devices on smithi088 2024-02-19T18:55:34.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:34.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:34.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:55:34.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:55:34.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: Adjusting osd_memory_target on smithi088 to 5920M 2024-02-19T18:55:34.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:34.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:34.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:55:34.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:34.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:34.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:55:34.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:55:34.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:55:34.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:34 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:35.887 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:35 smithi132 ceph-mon[37969]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:55:35.950 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:35 smithi088 ceph-mon[29272]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:55:36.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:36 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:36.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:36 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:36.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:36 smithi088 ceph-mon[29272]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:36.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:36 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:36.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:36 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:36.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:36 smithi132 ceph-mon[37969]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:38.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:37 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/242893199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66334f86-7411-4587-b96b-22b61bf5201a"}]: dispatch 2024-02-19T18:55:38.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:37 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/242893199' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66334f86-7411-4587-b96b-22b61bf5201a"}]': finished 2024-02-19T18:55:38.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:37 smithi132 ceph-mon[37969]: osdmap e16: 3 total, 2 up, 3 in 2024-02-19T18:55:38.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:37 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:38.300 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:37 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/242893199' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "66334f86-7411-4587-b96b-22b61bf5201a"}]: dispatch 2024-02-19T18:55:38.301 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:37 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/242893199' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "66334f86-7411-4587-b96b-22b61bf5201a"}]': finished 2024-02-19T18:55:38.301 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:37 smithi088 ceph-mon[29272]: osdmap e16: 3 total, 2 up, 3 in 2024-02-19T18:55:38.301 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:37 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:39.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:38 smithi088 ceph-mon[29272]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:39.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:38 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1395755175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:55:39.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:38 smithi132 ceph-mon[37969]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:39.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:38 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1395755175' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:55:40.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:40.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:40.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:40.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:55:40.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:40.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:40 smithi132 ceph-mon[37969]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:40.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:55:40.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:40.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:40.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:40.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:55:40.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:40.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:40 smithi088 ceph-mon[29272]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:40.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:55:43.001 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:42 smithi088 ceph-mon[29272]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:43.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:42 smithi132 ceph-mon[37969]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:45.022 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:44 smithi088 ceph-mon[29272]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:45.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:44 smithi132 ceph-mon[37969]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:46.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:45 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-19T18:55:46.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:45 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:46.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:45 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-19T18:55:46.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:45 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:55:47.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:46 smithi088 ceph-mon[29272]: Deploying daemon osd.2 on smithi088 2024-02-19T18:55:47.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:46 smithi088 ceph-mon[29272]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:47.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:46 smithi132 ceph-mon[37969]: Deploying daemon osd.2 on smithi088 2024-02-19T18:55:47.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:46 smithi132 ceph-mon[37969]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:49.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:48 smithi088 ceph-mon[29272]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:49.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:48 smithi132 ceph-mon[37969]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:51.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:50 smithi088 ceph-mon[29272]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:51.179 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:50 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:50 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:51.180 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:50 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:55:51.244 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:50 smithi132 ceph-mon[37969]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:51.245 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:50 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:51.245 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:50 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:51.245 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:50 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:55:53.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:52 smithi132 ceph-mon[37969]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:53.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:52 smithi088 ceph-mon[29272]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:54.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.621+0000 7f509e7fc700 1 -- 172.21.15.88:0/1819315104 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f50a8061190 con 0x7f509406c600 2024-02-19T18:55:54.638 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 2 on host 'smithi088' 2024-02-19T18:55:54.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.623+0000 7f50af935700 1 -- 172.21.15.88:0/1819315104 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f509406c600 msgr2=0x7f509406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:55:54.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.623+0000 7f50af935700 1 --2- 172.21.15.88:0/1819315104 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f509406c600 0x7f509406eab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f50a400b5c0 tx=0x7f50a4005ee0 comp rx=0 tx=0).stop 2024-02-19T18:55:54.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.624+0000 7f50af935700 1 -- 172.21.15.88:0/1819315104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a8103600 msgr2=0x7f50a80fec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:55:54.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.624+0000 7f50af935700 1 --2- 172.21.15.88:0/1819315104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a8103600 0x7f50a80fec90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f509800d8d0 tx=0x7f509800dbe0 comp rx=0 tx=0).stop 2024-02-19T18:55:54.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.624+0000 7f50af935700 1 -- 172.21.15.88:0/1819315104 shutdown_connections 2024-02-19T18:55:54.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.624+0000 7f50af935700 1 --2- 172.21.15.88:0/1819315104 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f509406c600 0x7f509406eab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:54.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.624+0000 7f50af935700 1 --2- 172.21.15.88:0/1819315104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f50a8100ce0 0x7f50a8100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:54.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.624+0000 7f50af935700 1 --2- 172.21.15.88:0/1819315104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a8103600 0x7f50a80fec90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:54.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.624+0000 7f50af935700 1 -- 172.21.15.88:0/1819315104 >> 172.21.15.88:0/1819315104 conn(0x7f50a80fa700 msgr2=0x7f50a80fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:55:54.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.624+0000 7f50af935700 1 -- 172.21.15.88:0/1819315104 shutdown_connections 2024-02-19T18:55:54.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:55:54.624+0000 7f50af935700 1 -- 172.21.15.88:0/1819315104 wait complete. 2024-02-19T18:55:55.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:54 smithi132 ceph-mon[37969]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:55.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:54 smithi132 ceph-mon[37969]: from='osd.2 [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-19T18:55:55.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:55.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:55.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:55:55.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:54 smithi088 ceph-mon[29272]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:55.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:54 smithi088 ceph-mon[29272]: from='osd.2 [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-19T18:55:55.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:55.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:55.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:55:55.371 DEBUG:teuthology.orchestra.run.smithi088:osd.2> sudo journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.2.service 2024-02-19T18:55:55.373 INFO:tasks.cephadm:Deploying osd.3 on smithi132 with /dev/vg_nvme/lv_4... 2024-02-19T18:55:55.373 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-19T18:55:55.585 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T18:55:56.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:55 smithi088 ceph-mon[29272]: from='osd.2 [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-19T18:55:56.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:55 smithi088 ceph-mon[29272]: osdmap e17: 3 total, 2 up, 3 in 2024-02-19T18:55:56.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:56.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:55 smithi088 ceph-mon[29272]: from='osd.2 [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T18:55:56.097 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 18:55:55 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[55263]: 2024-02-19T18:55:55.949+0000 7f98df7bb700 -1 osd.2 0 waiting for initial osdmap 2024-02-19T18:55:56.098 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 18:55:55 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[55263]: 2024-02-19T18:55:55.954+0000 7f98dadaf700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T18:55:56.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:55 smithi132 ceph-mon[37969]: from='osd.2 [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-19T18:55:56.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:55 smithi132 ceph-mon[37969]: osdmap e17: 3 total, 2 up, 3 in 2024-02-19T18:55:56.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:56.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:55 smithi132 ceph-mon[37969]: from='osd.2 [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T18:55:57.065 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:56 smithi132 ceph-mon[37969]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:57.065 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:56 smithi132 ceph-mon[37969]: from='osd.2 [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-02-19T18:55:57.065 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:56 smithi132 ceph-mon[37969]: osdmap e18: 3 total, 2 up, 3 in 2024-02-19T18:55:57.065 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:56 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:57.065 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:56 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:57.066 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:56 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:57.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:56 smithi088 ceph-mon[29272]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:57.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:56 smithi088 ceph-mon[29272]: from='osd.2 [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-02-19T18:55:57.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:56 smithi088 ceph-mon[29272]: osdmap e18: 3 total, 2 up, 3 in 2024-02-19T18:55:57.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:56 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:57.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:56 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:57.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:56 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:57.577 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-19T18:55:57.578 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-19T18:55:57.578 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-02-19T18:55:57.578 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-02-19T18:55:57.578 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114761 s, 914 MB/s 2024-02-19T18:55:57.578 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-02-19T18:55:58.190 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi132:vg_nvme/lv_4 2024-02-19T18:55:58.216 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:57 smithi132 ceph-mon[37969]: purged_snaps scrub starts 2024-02-19T18:55:58.216 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:57 smithi132 ceph-mon[37969]: purged_snaps scrub ok 2024-02-19T18:55:58.216 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:57 smithi132 ceph-mon[37969]: osd.2 [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249] boot 2024-02-19T18:55:58.216 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:57 smithi132 ceph-mon[37969]: osdmap e19: 3 total, 3 up, 3 in 2024-02-19T18:55:58.216 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:57 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:58.216 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:57 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:58.216 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:57 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:58.216 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:57 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-19T18:55:58.294 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:57 smithi088 ceph-mon[29272]: purged_snaps scrub starts 2024-02-19T18:55:58.294 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:57 smithi088 ceph-mon[29272]: purged_snaps scrub ok 2024-02-19T18:55:58.294 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:57 smithi088 ceph-mon[29272]: osd.2 [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249] boot 2024-02-19T18:55:58.295 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:57 smithi088 ceph-mon[29272]: osdmap e19: 3 total, 3 up, 3 in 2024-02-19T18:55:58.295 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:57 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T18:55:58.295 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:57 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:58.295 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:57 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:55:58.295 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:57 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-19T18:55:58.391 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T18:55:59.148 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:58 smithi132 ceph-mon[37969]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:59.149 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:58 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-19T18:55:59.149 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:58 smithi132 ceph-mon[37969]: osdmap e20: 3 total, 3 up, 3 in 2024-02-19T18:55:59.149 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:58 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-19T18:55:59.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:58 smithi088 ceph-mon[29272]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-19T18:55:59.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:58 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-19T18:55:59.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:58 smithi088 ceph-mon[29272]: osdmap e20: 3 total, 3 up, 3 in 2024-02-19T18:55:59.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:58 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-19T18:55:59.597 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 18:55:59 smithi088 sudo[57134]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-19T18:55:59.597 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 18:55:59 smithi088 sudo[57134]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T18:55:59.597 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 18:55:59 smithi088 sudo[57134]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T18:55:59.597 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 18:55:59 smithi088 sudo[57134]: pam_unix(sudo:session): session closed for user root 2024-02-19T18:55:59.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.913+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1071241450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dcc103650 msgr2=0x7f2dcc105a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:55:59.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.913+0000 7f2dd1bc9700 1 --2- 172.21.15.132:0/1071241450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dcc103650 0x7f2dcc105a30 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0009910 tx=0x7f2dc0009c20 comp rx=0 tx=0).stop 2024-02-19T18:55:59.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.913+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1071241450 shutdown_connections 2024-02-19T18:55:59.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.913+0000 7f2dd1bc9700 1 --2- 172.21.15.132:0/1071241450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dcc103650 0x7f2dcc105a30 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:59.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.914+0000 7f2dd1bc9700 1 --2- 172.21.15.132:0/1071241450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dcc100d30 0x7f2dcc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:59.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.914+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1071241450 >> 172.21.15.132:0/1071241450 conn(0x7f2dcc0fa6d0 msgr2=0x7f2dcc0fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:55:59.931 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.914+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1071241450 shutdown_connections 2024-02-19T18:55:59.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.914+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1071241450 wait complete. 2024-02-19T18:55:59.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.914+0000 7f2dd1bc9700 1 Processor -- start 2024-02-19T18:55:59.932 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.914+0000 7f2dd1bc9700 1 -- start start 2024-02-19T18:55:59.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dd1bc9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dcc100d30 0x7f2dcc195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:55:59.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dd1bc9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dcc103650 0x7f2dcc195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:55:59.933 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dd1bc9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc196610 con 0x7f2dcc103650 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dd1bc9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc196750 con 0x7f2dcc100d30 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dcb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dcc100d30 0x7f2dcc195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dcb7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dcc100d30 0x7f2dcc195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:54824/0 (socket says 172.21.15.132:54824) 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dcb7fe700 1 -- 172.21.15.132:0/1522799148 learned_addr learned my addr 172.21.15.132:0/1522799148 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dcaffd700 1 --2- 172.21.15.132:0/1522799148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dcc103650 0x7f2dcc195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dcb7fe700 1 -- 172.21.15.132:0/1522799148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dcc103650 msgr2=0x7f2dcc195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dcb7fe700 1 --2- 172.21.15.132:0/1522799148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dcc103650 0x7f2dcc195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dcb7fe700 1 -- 172.21.15.132:0/1522799148 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc00095f0 con 0x7f2dcc100d30 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dcb7fe700 1 --2- 172.21.15.132:0/1522799148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dcc100d30 0x7f2dcc195ab0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00d900 tx=0x7f2dbc00dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dc8ff9700 1 -- 172.21.15.132:0/1522799148 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dbc0041f0 con 0x7f2dcc100d30 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dc8ff9700 1 -- 172.21.15.132:0/1522799148 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2dbc004ce0 con 0x7f2dcc100d30 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1522799148 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc19b1b0 con 0x7f2dcc100d30 2024-02-19T18:55:59.934 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.915+0000 7f2dc8ff9700 1 -- 172.21.15.132:0/1522799148 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dbc00b750 con 0x7f2dcc100d30 2024-02-19T18:55:59.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.916+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1522799148 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc19b700 con 0x7f2dcc100d30 2024-02-19T18:55:59.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.916+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1522799148 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc18fc60 con 0x7f2dcc100d30 2024-02-19T18:55:59.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.917+0000 7f2dc8ff9700 1 -- 172.21.15.132:0/1522799148 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f2dbc004e50 con 0x7f2dcc100d30 2024-02-19T18:55:59.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.917+0000 7f2dc8ff9700 1 --2- 172.21.15.132:0/1522799148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f2db406c600 0x7f2db406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:55:59.935 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.917+0000 7f2dc8ff9700 1 -- 172.21.15.132:0/1522799148 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3232+0+0 (secure 0 0 0) 0x7f2dbc08acb0 con 0x7f2dcc100d30 2024-02-19T18:55:59.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.917+0000 7f2dcaffd700 1 --2- 172.21.15.132:0/1522799148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f2db406c600 0x7f2db406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:55:59.936 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.918+0000 7f2dcaffd700 1 --2- 172.21.15.132:0/1522799148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f2db406c600 0x7f2db406eab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006010 tx=0x7f2dc000b540 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:55:59.939 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:55:59.922+0000 7f2dc8ff9700 1 -- 172.21.15.132:0/1522799148 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2dbc059840 con 0x7f2dcc100d30 2024-02-19T18:55:59.955 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 18:55:59 smithi088 sudo[57145]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-19T18:55:59.955 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 18:55:59 smithi088 sudo[57145]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T18:55:59.955 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 18:55:59 smithi088 sudo[57145]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T18:55:59.955 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 18:55:59 smithi088 sudo[57145]: pam_unix(sudo:session): session closed for user root 2024-02-19T18:55:59.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:59 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-19T18:55:59.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:59 smithi132 ceph-mon[37969]: osdmap e21: 3 total, 3 up, 3 in 2024-02-19T18:55:59.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:55:59 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:00.137 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:00.120+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1522799148 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f2dcc061190 con 0x7f2db406c600 2024-02-19T18:56:00.267 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:59 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-19T18:56:00.267 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:59 smithi088 ceph-mon[29272]: osdmap e21: 3 total, 3 up, 3 in 2024-02-19T18:56:00.267 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:55:59 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:00.268 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 18:55:59 smithi088 sudo[57553]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-19T18:56:00.268 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 18:55:59 smithi088 sudo[57553]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T18:56:00.268 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 18:55:59 smithi088 sudo[57553]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T18:56:00.268 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57553]: pam_unix(sudo:session): session closed for user root 2024-02-19T18:56:00.517 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57562]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-19T18:56:00.518 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57562]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T18:56:00.518 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57562]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T18:56:00.518 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57562]: pam_unix(sudo:session): session closed for user root 2024-02-19T18:56:00.779 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57565]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-19T18:56:00.779 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57565]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T18:56:00.780 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57565]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T18:56:00.780 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57565]: pam_unix(sudo:session): session closed for user root 2024-02-19T18:56:01.041 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 sudo[57603]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-19T18:56:01.041 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 sudo[57603]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T18:56:01.041 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 sudo[57603]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T18:56:01.041 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 sudo[57603]: pam_unix(sudo:session): session closed for user root 2024-02-19T18:56:01.041 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57569]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-19T18:56:01.041 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57569]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T18:56:01.042 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57569]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T18:56:01.042 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 18:56:00 smithi088 sudo[57569]: pam_unix(sudo:session): session closed for user root 2024-02-19T18:56:01.304 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:01.287+0000 7f2dc8ff9700 1 -- 172.21.15.132:0/1522799148 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f2dbc056180 con 0x7f2dcc100d30 2024-02-19T18:56:01.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 sudo[57606]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-19T18:56:01.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 sudo[57606]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T18:56:01.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 sudo[57606]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T18:56:01.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 sudo[57606]: pam_unix(sudo:session): session closed for user root 2024-02-19T18:56:01.829 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 sudo[41700]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-19T18:56:01.829 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 sudo[41700]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T18:56:01.829 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 sudo[41700]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T18:56:01.829 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 sudo[41700]: pam_unix(sudo:session): session closed for user root 2024-02-19T18:56:02.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:02.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: osdmap e22: 3 total, 3 up, 3 in 2024-02-19T18:56:02.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:56:02.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:56:02.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:02.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:02.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:02.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:02.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:02.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:02.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:02.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:02.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:02.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:02.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:01 smithi088 ceph-mon[29272]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T18:56:02.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 sudo[41753]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 sudo[41753]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: pgmap v79: 1 pgs: 1 creating+activating; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 sudo[41753]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: osdmap e22: 3 total, 3 up, 3 in 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 sudo[41753]: pam_unix(sudo:session): session closed for user root 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:02.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:02.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:02.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:02.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:02.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:02.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:01 smithi132 ceph-mon[37969]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T18:56:03.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: Detected new or changed devices on smithi088 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: Adjusting osd_memory_target on smithi088 to 3946M 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: mgrmap e19: smithi088.gyviqq(active, since 2m), standbys: smithi132.psllsa 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: pgmap v81: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T18:56:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:03.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1105188760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "733318f2-be90-44d7-8c38-fb773f7da169"}]: dispatch 2024-02-19T18:56:03.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "733318f2-be90-44d7-8c38-fb773f7da169"}]: dispatch 2024-02-19T18:56:03.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "733318f2-be90-44d7-8c38-fb773f7da169"}]': finished 2024-02-19T18:56:03.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: osdmap e23: 4 total, 3 up, 4 in 2024-02-19T18:56:03.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:02 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T18:56:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:56:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: Detected new or changed devices on smithi088 2024-02-19T18:56:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: Adjusting osd_memory_target on smithi088 to 3946M 2024-02-19T18:56:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T18:56:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T18:56:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:56:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:56:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-19T18:56:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: mgrmap e19: smithi088.gyviqq(active, since 2m), standbys: smithi132.psllsa 2024-02-19T18:56:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T18:56:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T18:56:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: pgmap v81: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-19T18:56:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/1105188760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "733318f2-be90-44d7-8c38-fb773f7da169"}]: dispatch 2024-02-19T18:56:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "733318f2-be90-44d7-8c38-fb773f7da169"}]: dispatch 2024-02-19T18:56:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "733318f2-be90-44d7-8c38-fb773f7da169"}]': finished 2024-02-19T18:56:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: osdmap e23: 4 total, 3 up, 4 in 2024-02-19T18:56:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:02 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T18:56:04.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:03 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/3058326497' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:56:04.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:03 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/3058326497' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:56:05.062 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:04 smithi088 ceph-mon[29272]: pgmap v83: 1 pgs: 1 creating+activating; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:05.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:04 smithi132 ceph-mon[37969]: pgmap v83: 1 pgs: 1 creating+activating; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:06.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:06 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:06.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:06 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:06.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:06 smithi132 ceph-mon[37969]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:06.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:06 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:06.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:06 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:06.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:06 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:06.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:06 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:06.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:06 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:06.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:06 smithi088 ceph-mon[29272]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:06.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:06 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:06.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:06 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:06.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:06 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:09.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:08 smithi132 ceph-mon[37969]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:09.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:08 smithi088 ceph-mon[29272]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:10.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:09 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:56:10.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:09 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:56:10.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:10 smithi132 ceph-mon[37969]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:10.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-19T18:56:10.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:10 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:10.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:10 smithi132 ceph-mon[37969]: Deploying daemon osd.3 on smithi132 2024-02-19T18:56:11.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:10 smithi088 ceph-mon[29272]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:11.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:10 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-19T18:56:11.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:10 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:11.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:10 smithi088 ceph-mon[29272]: Deploying daemon osd.3 on smithi132 2024-02-19T18:56:13.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:12 smithi132 ceph-mon[37969]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:13.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:12 smithi088 ceph-mon[29272]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:15.081 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:14 smithi132 ceph-mon[37969]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:15.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:14 smithi088 ceph-mon[29272]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:16.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:16.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:16.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:56:16.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:16 smithi132 ceph-mon[37969]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:16.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:16.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:16.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:56:16.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:16 smithi088 ceph-mon[29272]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:18.405 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 3 on host 'smithi132' 2024-02-19T18:56:18.406 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.387+0000 7f2dc8ff9700 1 -- 172.21.15.132:0/1522799148 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2dcc061190 con 0x7f2db406c600 2024-02-19T18:56:18.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.390+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1522799148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f2db406c600 msgr2=0x7f2db406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:56:18.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.390+0000 7f2dd1bc9700 1 --2- 172.21.15.132:0/1522799148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f2db406c600 0x7f2db406eab0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006010 tx=0x7f2dc000b540 comp rx=0 tx=0).stop 2024-02-19T18:56:18.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.390+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1522799148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dcc100d30 msgr2=0x7f2dcc195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:56:18.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.390+0000 7f2dd1bc9700 1 --2- 172.21.15.132:0/1522799148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dcc100d30 0x7f2dcc195ab0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc00d900 tx=0x7f2dbc00dc10 comp rx=0 tx=0).stop 2024-02-19T18:56:18.407 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.390+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1522799148 shutdown_connections 2024-02-19T18:56:18.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.390+0000 7f2dd1bc9700 1 --2- 172.21.15.132:0/1522799148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f2db406c600 0x7f2db406eab0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:18.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.390+0000 7f2dd1bc9700 1 --2- 172.21.15.132:0/1522799148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dcc100d30 0x7f2dcc195ab0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:18.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.390+0000 7f2dd1bc9700 1 --2- 172.21.15.132:0/1522799148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dcc103650 0x7f2dcc195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:18.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.390+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1522799148 >> 172.21.15.132:0/1522799148 conn(0x7f2dcc0fa6d0 msgr2=0x7f2dcc0fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:56:18.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.390+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1522799148 shutdown_connections 2024-02-19T18:56:18.408 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:18.390+0000 7f2dd1bc9700 1 -- 172.21.15.132:0/1522799148 wait complete. 2024-02-19T18:56:19.052 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:18 smithi132 ceph-mon[37969]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:19.052 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:19.053 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:19.074 DEBUG:teuthology.orchestra.run.smithi132:osd.3> sudo journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.3.service 2024-02-19T18:56:19.118 INFO:tasks.cephadm:Deploying osd.4 on smithi132 with /dev/vg_nvme/lv_3... 2024-02-19T18:56:19.118 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-19T18:56:19.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:18 smithi088 ceph-mon[29272]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:19.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:19.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:19.860 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T18:56:20.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:19 smithi132 ceph-mon[37969]: from='osd.3 [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-19T18:56:20.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:19 smithi132 ceph-mon[37969]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-19T18:56:20.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:19 smithi088 ceph-mon[29272]: from='osd.3 [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-19T18:56:20.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:19 smithi088 ceph-mon[29272]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-19T18:56:21.186 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:20 smithi132 ceph-mon[37969]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:21.187 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:20 smithi132 ceph-mon[37969]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-19T18:56:21.187 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:20 smithi132 ceph-mon[37969]: osdmap e24: 4 total, 3 up, 4 in 2024-02-19T18:56:21.187 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:20 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T18:56:21.187 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:20 smithi132 ceph-mon[37969]: from='osd.3 [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:56:21.187 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:20 smithi132 ceph-mon[37969]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:56:21.187 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 18:56:20 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[43878]: 2024-02-19T18:56:20.947+0000 7f1082f40700 -1 osd.3 0 waiting for initial osdmap 2024-02-19T18:56:21.187 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 18:56:20 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[43878]: 2024-02-19T18:56:20.964+0000 7f107f536700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T18:56:21.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:20 smithi088 ceph-mon[29272]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:21.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:20 smithi088 ceph-mon[29272]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-19T18:56:21.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:20 smithi088 ceph-mon[29272]: osdmap e24: 4 total, 3 up, 4 in 2024-02-19T18:56:21.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:20 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T18:56:21.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:20 smithi088 ceph-mon[29272]: from='osd.3 [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:56:21.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:20 smithi088 ceph-mon[29272]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:56:21.695 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-19T18:56:21.695 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-19T18:56:21.695 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-02-19T18:56:21.696 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-02-19T18:56:21.696 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124643 s, 841 MB/s 2024-02-19T18:56:21.696 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-02-19T18:56:22.149 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:21 smithi132 ceph-mon[37969]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-02-19T18:56:22.150 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:21 smithi132 ceph-mon[37969]: osdmap e25: 4 total, 3 up, 4 in 2024-02-19T18:56:22.150 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:21 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T18:56:22.150 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:21 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T18:56:22.150 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:21 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:22.150 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:21 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:22.150 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:21 smithi132 ceph-mon[37969]: osd.3 [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174] boot 2024-02-19T18:56:22.150 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:21 smithi132 ceph-mon[37969]: osdmap e26: 4 total, 4 up, 4 in 2024-02-19T18:56:22.150 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:21 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T18:56:22.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:21 smithi088 ceph-mon[29272]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-02-19T18:56:22.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:21 smithi088 ceph-mon[29272]: osdmap e25: 4 total, 3 up, 4 in 2024-02-19T18:56:22.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:21 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T18:56:22.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:21 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T18:56:22.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:21 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:22.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:21 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:22.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:21 smithi088 ceph-mon[29272]: osd.3 [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174] boot 2024-02-19T18:56:22.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:21 smithi088 ceph-mon[29272]: osdmap e26: 4 total, 4 up, 4 in 2024-02-19T18:56:22.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:21 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T18:56:22.775 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi132:vg_nvme/lv_3 2024-02-19T18:56:23.211 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:22 smithi132 ceph-mon[37969]: purged_snaps scrub starts 2024-02-19T18:56:23.211 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:22 smithi132 ceph-mon[37969]: purged_snaps scrub ok 2024-02-19T18:56:23.211 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:22 smithi132 ceph-mon[37969]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:23.211 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:22 smithi132 ceph-mon[37969]: osdmap e27: 4 total, 4 up, 4 in 2024-02-19T18:56:23.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:22 smithi088 ceph-mon[29272]: purged_snaps scrub starts 2024-02-19T18:56:23.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:22 smithi088 ceph-mon[29272]: purged_snaps scrub ok 2024-02-19T18:56:23.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:22 smithi088 ceph-mon[29272]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-02-19T18:56:23.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:22 smithi088 ceph-mon[29272]: osdmap e27: 4 total, 4 up, 4 in 2024-02-19T18:56:23.753 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T18:56:25.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:24 smithi088 ceph-mon[29272]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:25.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:24 smithi088 ceph-mon[29272]: osdmap e28: 4 total, 4 up, 4 in 2024-02-19T18:56:25.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:24 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:56:25.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:24 smithi132 ceph-mon[37969]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:25.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:24 smithi132 ceph-mon[37969]: osdmap e28: 4 total, 4 up, 4 in 2024-02-19T18:56:25.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:24 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:56:26.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.032+0000 7fb961900700 1 -- 172.21.15.132:0/2459666582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb95c100d60 msgr2=0x7fb95c103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:56:26.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.032+0000 7fb961900700 1 --2- 172.21.15.132:0/2459666582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb95c100d60 0x7fb95c103140 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00b0a0 tx=0x7fb94c00b3b0 comp rx=0 tx=0).stop 2024-02-19T18:56:26.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.032+0000 7fb961900700 1 -- 172.21.15.132:0/2459666582 shutdown_connections 2024-02-19T18:56:26.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.032+0000 7fb961900700 1 --2- 172.21.15.132:0/2459666582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb95c103680 0x7fb95c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:26.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.032+0000 7fb961900700 1 --2- 172.21.15.132:0/2459666582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb95c100d60 0x7fb95c103140 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:26.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.032+0000 7fb961900700 1 -- 172.21.15.132:0/2459666582 >> 172.21.15.132:0/2459666582 conn(0x7fb95c0fa780 msgr2=0x7fb95c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:56:26.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.032+0000 7fb961900700 1 -- 172.21.15.132:0/2459666582 shutdown_connections 2024-02-19T18:56:26.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.032+0000 7fb961900700 1 -- 172.21.15.132:0/2459666582 wait complete. 2024-02-19T18:56:26.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb961900700 1 Processor -- start 2024-02-19T18:56:26.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb961900700 1 -- start start 2024-02-19T18:56:26.051 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb961900700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb95c100d60 0x7fb95c195ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb961900700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb95c103680 0x7fb95c1960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb961900700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c196700 con 0x7fb95c103680 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb961900700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95c196840 con 0x7fb95c100d60 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb95affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb95c100d60 0x7fb95c195ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb95affd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb95c100d60 0x7fb95c195ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:52018/0 (socket says 172.21.15.132:52018) 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb95affd700 1 -- 172.21.15.132:0/1922489339 learned_addr learned my addr 172.21.15.132:0/1922489339 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb95a7fc700 1 --2- 172.21.15.132:0/1922489339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb95c103680 0x7fb95c1960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb95affd700 1 -- 172.21.15.132:0/1922489339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb95c103680 msgr2=0x7fb95c1960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb95affd700 1 --2- 172.21.15.132:0/1922489339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb95c103680 0x7fb95c1960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb95affd700 1 -- 172.21.15.132:0/1922489339 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c009d00 con 0x7fb95c100d60 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.033+0000 7fb95affd700 1 --2- 172.21.15.132:0/1922489339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb95c100d60 0x7fb95c195ba0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0077b0 tx=0x7fb94c003ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.034+0000 7fb9608fe700 1 -- 172.21.15.132:0/1922489339 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb94c00e040 con 0x7fb95c100d60 2024-02-19T18:56:26.052 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.034+0000 7fb9608fe700 1 -- 172.21.15.132:0/1922489339 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb94c00ce60 con 0x7fb95c100d60 2024-02-19T18:56:26.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.034+0000 7fb961900700 1 -- 172.21.15.132:0/1922489339 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95c19b290 con 0x7fb95c100d60 2024-02-19T18:56:26.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.034+0000 7fb9608fe700 1 -- 172.21.15.132:0/1922489339 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb94c01db10 con 0x7fb95c100d60 2024-02-19T18:56:26.053 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.034+0000 7fb961900700 1 -- 172.21.15.132:0/1922489339 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95c19b780 con 0x7fb95c100d60 2024-02-19T18:56:26.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.035+0000 7fb961900700 1 -- 172.21.15.132:0/1922489339 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95c18fc40 con 0x7fb95c100d60 2024-02-19T18:56:26.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.035+0000 7fb9608fe700 1 -- 172.21.15.132:0/1922489339 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb94c01dc90 con 0x7fb95c100d60 2024-02-19T18:56:26.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.035+0000 7fb9608fe700 1 --2- 172.21.15.132:0/1922489339 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb94806c680 0x7fb94806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:56:26.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.035+0000 7fb9608fe700 1 -- 172.21.15.132:0/1922489339 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7fb94c08cc90 con 0x7fb95c100d60 2024-02-19T18:56:26.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.036+0000 7fb95a7fc700 1 --2- 172.21.15.132:0/1922489339 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb94806c680 0x7fb94806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:56:26.054 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.036+0000 7fb95a7fc700 1 --2- 172.21.15.132:0/1922489339 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb94806c680 0x7fb94806eb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb944006fd0 tx=0x7fb944008040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:56:26.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.040+0000 7fb9608fe700 1 -- 172.21.15.132:0/1922489339 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb94c05b4a0 con 0x7fb95c100d60 2024-02-19T18:56:26.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: osdmap e29: 4 total, 4 up, 4 in 2024-02-19T18:56:26.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:26.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:26.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:26.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:26.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:26.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:26.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:26.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:56:26.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:26.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:26.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:26.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:25 smithi132 ceph-mon[37969]: osdmap e30: 4 total, 4 up, 4 in 2024-02-19T18:56:26.250 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:26.234+0000 7fb961900700 1 -- 172.21.15.132:0/1922489339 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb95c061190 con 0x7fb94806c680 2024-02-19T18:56:26.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: osdmap e29: 4 total, 4 up, 4 in 2024-02-19T18:56:26.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:26.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:26.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:26.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:26.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:26.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:26.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:26.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:56:26.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:26.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:26.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:26.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:25 smithi088 ceph-mon[29272]: osdmap e30: 4 total, 4 up, 4 in 2024-02-19T18:56:26.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:26 smithi132 ceph-mon[37969]: Detected new or changed devices on smithi132 2024-02-19T18:56:27.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:26 smithi132 ceph-mon[37969]: Adjusting osd_memory_target on smithi132 to 14912M 2024-02-19T18:56:27.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:26 smithi132 ceph-mon[37969]: pgmap v100: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-19T18:56:27.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:26 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:56:27.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:26 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:56:27.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:26 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:27.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:26 smithi088 ceph-mon[29272]: Detected new or changed devices on smithi132 2024-02-19T18:56:27.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:26 smithi088 ceph-mon[29272]: Adjusting osd_memory_target on smithi132 to 14912M 2024-02-19T18:56:27.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:26 smithi088 ceph-mon[29272]: pgmap v100: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-19T18:56:27.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:26 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:56:27.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:26 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:56:27.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:26 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:28.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:27 smithi132 ceph-mon[37969]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:56:28.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:27 smithi088 ceph-mon[29272]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:56:29.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:28 smithi132 ceph-mon[37969]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-02-19T18:56:29.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:28 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/408770728' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b41c4ff-b4b1-4279-b973-0c3c1c62373e"}]: dispatch 2024-02-19T18:56:29.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:28 smithi132 ceph-mon[37969]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b41c4ff-b4b1-4279-b973-0c3c1c62373e"}]: dispatch 2024-02-19T18:56:29.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:28 smithi132 ceph-mon[37969]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b41c4ff-b4b1-4279-b973-0c3c1c62373e"}]': finished 2024-02-19T18:56:29.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:28 smithi132 ceph-mon[37969]: osdmap e31: 5 total, 4 up, 5 in 2024-02-19T18:56:29.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:28 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:29.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:28 smithi088 ceph-mon[29272]: pgmap v102: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 123 KiB/s, 0 objects/s recovering 2024-02-19T18:56:29.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:28 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/408770728' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b41c4ff-b4b1-4279-b973-0c3c1c62373e"}]: dispatch 2024-02-19T18:56:29.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:28 smithi088 ceph-mon[29272]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b41c4ff-b4b1-4279-b973-0c3c1c62373e"}]: dispatch 2024-02-19T18:56:29.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:28 smithi088 ceph-mon[29272]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b41c4ff-b4b1-4279-b973-0c3c1c62373e"}]': finished 2024-02-19T18:56:29.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:28 smithi088 ceph-mon[29272]: osdmap e31: 5 total, 4 up, 5 in 2024-02-19T18:56:29.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:28 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:30.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:29 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/1307010685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:56:30.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:29 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/1307010685' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:56:31.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:30 smithi132 ceph-mon[37969]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-02-19T18:56:31.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:30 smithi088 ceph-mon[29272]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-02-19T18:56:33.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:32 smithi132 ceph-mon[37969]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:33.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:32 smithi088 ceph-mon[29272]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:34.999 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:34 smithi132 ceph-mon[37969]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:35.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:34 smithi088 ceph-mon[29272]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:37.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:36 smithi132 ceph-mon[37969]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:37.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:36 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-19T18:56:37.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:36 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:37.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:36 smithi088 ceph-mon[29272]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:37.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:36 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-19T18:56:37.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:36 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:38.185 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:37 smithi132 ceph-mon[37969]: Deploying daemon osd.4 on smithi132 2024-02-19T18:56:38.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:37 smithi088 ceph-mon[29272]: Deploying daemon osd.4 on smithi132 2024-02-19T18:56:39.246 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:38 smithi132 ceph-mon[37969]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:39.284 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:38 smithi088 ceph-mon[29272]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:40.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:39 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:56:40.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:39 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:56:41.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:40 smithi132 ceph-mon[37969]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:41.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:40 smithi088 ceph-mon[29272]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:42.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:42 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:42.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:42 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:42.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:42 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:56:42.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:42 smithi132 ceph-mon[37969]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:42.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:42 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:42.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:42 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:42.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:42 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:56:42.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:42 smithi088 ceph-mon[29272]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:45.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:44 smithi132 ceph-mon[37969]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:45.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:44 smithi088 ceph-mon[29272]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:45.365 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.347+0000 7fb9608fe700 1 -- 172.21.15.132:0/1922489339 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb95c061190 con 0x7fb94806c680 2024-02-19T18:56:45.365 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 4 on host 'smithi132' 2024-02-19T18:56:45.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.349+0000 7fb961900700 1 -- 172.21.15.132:0/1922489339 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb94806c680 msgr2=0x7fb94806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:56:45.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.349+0000 7fb961900700 1 --2- 172.21.15.132:0/1922489339 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb94806c680 0x7fb94806eb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb944006fd0 tx=0x7fb944008040 comp rx=0 tx=0).stop 2024-02-19T18:56:45.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.349+0000 7fb961900700 1 -- 172.21.15.132:0/1922489339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb95c100d60 msgr2=0x7fb95c195ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:56:45.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.349+0000 7fb961900700 1 --2- 172.21.15.132:0/1922489339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb95c100d60 0x7fb95c195ba0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0077b0 tx=0x7fb94c003ce0 comp rx=0 tx=0).stop 2024-02-19T18:56:45.367 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.349+0000 7fb961900700 1 -- 172.21.15.132:0/1922489339 shutdown_connections 2024-02-19T18:56:45.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.349+0000 7fb961900700 1 --2- 172.21.15.132:0/1922489339 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fb94806c680 0x7fb94806eb30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:45.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.349+0000 7fb961900700 1 --2- 172.21.15.132:0/1922489339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb95c100d60 0x7fb95c195ba0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:45.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.349+0000 7fb961900700 1 --2- 172.21.15.132:0/1922489339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb95c103680 0x7fb95c1960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:45.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.349+0000 7fb961900700 1 -- 172.21.15.132:0/1922489339 >> 172.21.15.132:0/1922489339 conn(0x7fb95c0fa780 msgr2=0x7fb95c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:56:45.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.349+0000 7fb961900700 1 -- 172.21.15.132:0/1922489339 shutdown_connections 2024-02-19T18:56:45.368 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:45.349+0000 7fb961900700 1 -- 172.21.15.132:0/1922489339 wait complete. 2024-02-19T18:56:45.997 DEBUG:teuthology.orchestra.run.smithi132:osd.4> sudo journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.4.service 2024-02-19T18:56:45.999 INFO:tasks.cephadm:Deploying osd.5 on smithi132 with /dev/vg_nvme/lv_2... 2024-02-19T18:56:46.000 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-19T18:56:46.357 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T18:56:46.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:46.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:46 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:46.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:46 smithi088 ceph-mon[29272]: from='osd.4 [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T18:56:46.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:46 smithi088 ceph-mon[29272]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T18:56:46.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:46 smithi088 ceph-mon[29272]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:46.607 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:46 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:46.607 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:46 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:46.607 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:46 smithi132 ceph-mon[37969]: from='osd.4 [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T18:56:46.607 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:46 smithi132 ceph-mon[37969]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T18:56:46.607 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:46 smithi132 ceph-mon[37969]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:47.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:47 smithi088 ceph-mon[29272]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-19T18:56:47.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:47 smithi088 ceph-mon[29272]: osdmap e32: 5 total, 4 up, 5 in 2024-02-19T18:56:47.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:47 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:47.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:47 smithi088 ceph-mon[29272]: from='osd.4 [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:56:47.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:47 smithi088 ceph-mon[29272]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:56:47.598 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:47 smithi132 ceph-mon[37969]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-19T18:56:47.598 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:47 smithi132 ceph-mon[37969]: osdmap e32: 5 total, 4 up, 5 in 2024-02-19T18:56:47.598 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:47 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:47.598 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:47 smithi132 ceph-mon[37969]: from='osd.4 [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:56:47.598 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:47 smithi132 ceph-mon[37969]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:56:47.599 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 18:56:47 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[48560]: 2024-02-19T18:56:47.324+0000 7f20c5ecf700 -1 osd.4 0 waiting for initial osdmap 2024-02-19T18:56:47.599 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 18:56:47 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[48560]: 2024-02-19T18:56:47.331+0000 7f20c04c1700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T18:56:48.248 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-19T18:56:48.248 INFO:teuthology.orchestra.run.smithi132.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-19T18:56:48.248 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10+0 records in 2024-02-19T18:56:48.248 INFO:teuthology.orchestra.run.smithi132.stderr:10+0 records out 2024-02-19T18:56:48.248 INFO:teuthology.orchestra.run.smithi132.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0212219 s, 494 MB/s 2024-02-19T18:56:48.248 INFO:teuthology.orchestra.run.smithi132.stderr:--> Zapping successful for: 2024-02-19T18:56:48.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:48 smithi132 ceph-mon[37969]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-02-19T18:56:48.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:48 smithi132 ceph-mon[37969]: osdmap e33: 5 total, 4 up, 5 in 2024-02-19T18:56:48.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:48.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:48.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:48.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:48.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:48 smithi132 ceph-mon[37969]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:48.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:48 smithi088 ceph-mon[29272]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-02-19T18:56:48.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:48 smithi088 ceph-mon[29272]: osdmap e33: 5 total, 4 up, 5 in 2024-02-19T18:56:48.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:48.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:48.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:48.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:48.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:48 smithi088 ceph-mon[29272]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-02-19T18:56:48.921 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph orch daemon add osd smithi132:vg_nvme/lv_2 2024-02-19T18:56:49.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:49 smithi132 ceph-mon[37969]: purged_snaps scrub starts 2024-02-19T18:56:49.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:49 smithi132 ceph-mon[37969]: purged_snaps scrub ok 2024-02-19T18:56:49.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:49 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:49.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:49 smithi132 ceph-mon[37969]: osd.4 [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446] boot 2024-02-19T18:56:49.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:49 smithi132 ceph-mon[37969]: osdmap e34: 5 total, 5 up, 5 in 2024-02-19T18:56:49.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:49 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:49.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:49 smithi088 ceph-mon[29272]: purged_snaps scrub starts 2024-02-19T18:56:49.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:49 smithi088 ceph-mon[29272]: purged_snaps scrub ok 2024-02-19T18:56:49.597 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:49 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:49.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:49 smithi088 ceph-mon[29272]: osd.4 [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446] boot 2024-02-19T18:56:49.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:49 smithi088 ceph-mon[29272]: osdmap e34: 5 total, 5 up, 5 in 2024-02-19T18:56:49.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:49 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T18:56:50.431 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T18:56:50.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:50 smithi132 ceph-mon[37969]: osdmap e35: 5 total, 5 up, 5 in 2024-02-19T18:56:50.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:50 smithi132 ceph-mon[37969]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:56:50.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:50 smithi088 ceph-mon[29272]: osdmap e35: 5 total, 5 up, 5 in 2024-02-19T18:56:50.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:50 smithi088 ceph-mon[29272]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:56:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.694+0000 7f0a15834700 1 -- 172.21.15.132:0/2845397075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a101003d0 msgr2=0x7f0a10100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:56:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.694+0000 7f0a15834700 1 --2- 172.21.15.132:0/2845397075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a101003d0 0x7f0a10100840 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009910 tx=0x7f0a04009c20 comp rx=0 tx=0).stop 2024-02-19T18:56:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.694+0000 7f0a15834700 1 -- 172.21.15.132:0/2845397075 shutdown_connections 2024-02-19T18:56:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.694+0000 7f0a15834700 1 --2- 172.21.15.132:0/2845397075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a101003d0 0x7f0a10100840 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.694+0000 7f0a15834700 1 --2- 172.21.15.132:0/2845397075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a100ff130 0x7f0a100ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.694+0000 7f0a15834700 1 -- 172.21.15.132:0/2845397075 >> 172.21.15.132:0/2845397075 conn(0x7f0a100fa780 msgr2=0x7f0a100fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:56:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.694+0000 7f0a15834700 1 -- 172.21.15.132:0/2845397075 shutdown_connections 2024-02-19T18:56:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.694+0000 7f0a15834700 1 -- 172.21.15.132:0/2845397075 wait complete. 2024-02-19T18:56:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.694+0000 7f0a15834700 1 Processor -- start 2024-02-19T18:56:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a15834700 1 -- start start 2024-02-19T18:56:52.714 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a15834700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a100ff130 0x7f0a10195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a15834700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a101003d0 0x7f0a10195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a15834700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a101965b0 con 0x7f0a100ff130 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a15834700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a101966f0 con 0x7f0a101003d0 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a0e7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a101003d0 0x7f0a10195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a0e7fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a101003d0 0x7f0a10195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48488/0 (socket says 172.21.15.132:48488) 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a0e7fc700 1 -- 172.21.15.132:0/1252707852 learned_addr learned my addr 172.21.15.132:0/1252707852 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a0effd700 1 --2- 172.21.15.132:0/1252707852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a100ff130 0x7f0a10195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a0e7fc700 1 -- 172.21.15.132:0/1252707852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a100ff130 msgr2=0x7f0a10195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a0e7fc700 1 --2- 172.21.15.132:0/1252707852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a100ff130 0x7f0a10195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a0e7fc700 1 -- 172.21.15.132:0/1252707852 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040095f0 con 0x7f0a101003d0 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.695+0000 7f0a0e7fc700 1 --2- 172.21.15.132:0/1252707852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a101003d0 0x7f0a10195f90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0a04006010 tx=0x7f0a0400bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.696+0000 7f0a14832700 1 -- 172.21.15.132:0/1252707852 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a0401d070 con 0x7f0a101003d0 2024-02-19T18:56:52.715 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.696+0000 7f0a15834700 1 -- 172.21.15.132:0/1252707852 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1019b140 con 0x7f0a101003d0 2024-02-19T18:56:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.696+0000 7f0a14832700 1 -- 172.21.15.132:0/1252707852 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0a0400f460 con 0x7f0a101003d0 2024-02-19T18:56:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.696+0000 7f0a14832700 1 -- 172.21.15.132:0/1252707852 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a04021640 con 0x7f0a101003d0 2024-02-19T18:56:52.716 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.696+0000 7f0a15834700 1 -- 172.21.15.132:0/1252707852 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a10101950 con 0x7f0a101003d0 2024-02-19T18:56:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.697+0000 7f0a15834700 1 -- 172.21.15.132:0/1252707852 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a10107c10 con 0x7f0a101003d0 2024-02-19T18:56:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.697+0000 7f0a14832700 1 -- 172.21.15.132:0/1252707852 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f0a0400fab0 con 0x7f0a101003d0 2024-02-19T18:56:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.698+0000 7f0a14832700 1 --2- 172.21.15.132:0/1252707852 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f09f806c680 0x7f09f806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:56:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.698+0000 7f0a14832700 1 -- 172.21.15.132:0/1252707852 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4206+0+0 (secure 0 0 0) 0x7f0a0408c740 con 0x7f0a101003d0 2024-02-19T18:56:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.698+0000 7f0a0effd700 1 --2- 172.21.15.132:0/1252707852 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f09f806c680 0x7f09f806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:56:52.717 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.699+0000 7f0a0effd700 1 --2- 172.21.15.132:0/1252707852 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f09f806c680 0x7f09f806eb30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a080060b0 tx=0x7f0a0800c040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:56:52.721 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.703+0000 7f0a14832700 1 -- 172.21.15.132:0/1252707852 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0a0405ada0 con 0x7f0a101003d0 2024-02-19T18:56:52.924 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:56:52.906+0000 7f0a15834700 1 -- 172.21.15.132:0/1252707852 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a10068c50 con 0x7f09f806c680 2024-02-19T18:56:53.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:56:53.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:53.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:53.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:53.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:53.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:53.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:53.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:53.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:53.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:56:53.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:53.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:53.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:53.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:56:53.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:56:53.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:52 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:56:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:56:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:56:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:53.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:56:53.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:56:53.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-19T18:56:53.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-19T18:56:53.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:52 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:56:54.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:53 smithi132 ceph-mon[37969]: Detected new or changed devices on smithi132 2024-02-19T18:56:54.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:53 smithi132 ceph-mon[37969]: Adjusting osd_memory_target on smithi132 to 7456M 2024-02-19T18:56:54.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:53 smithi132 ceph-mon[37969]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:56:54.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:53 smithi088 ceph-mon[29272]: Detected new or changed devices on smithi132 2024-02-19T18:56:54.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:53 smithi088 ceph-mon[29272]: Adjusting osd_memory_target on smithi132 to 7456M 2024-02-19T18:56:54.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:53 smithi088 ceph-mon[29272]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi132:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:56:55.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:54 smithi132 ceph-mon[37969]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:56:55.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:56:55.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:54 smithi088 ceph-mon[29272]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:56:55.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:56:56.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:55 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/2717597456' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "367bd67a-e45d-40ff-9aac-1f88755641ad"}]: dispatch 2024-02-19T18:56:56.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:55 smithi132 ceph-mon[37969]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "367bd67a-e45d-40ff-9aac-1f88755641ad"}]: dispatch 2024-02-19T18:56:56.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:55 smithi132 ceph-mon[37969]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "367bd67a-e45d-40ff-9aac-1f88755641ad"}]': finished 2024-02-19T18:56:56.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:55 smithi132 ceph-mon[37969]: osdmap e36: 6 total, 5 up, 6 in 2024-02-19T18:56:56.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:56:56.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:55 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/2717597456' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "367bd67a-e45d-40ff-9aac-1f88755641ad"}]: dispatch 2024-02-19T18:56:56.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:55 smithi088 ceph-mon[29272]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "367bd67a-e45d-40ff-9aac-1f88755641ad"}]: dispatch 2024-02-19T18:56:56.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:55 smithi088 ceph-mon[29272]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "367bd67a-e45d-40ff-9aac-1f88755641ad"}]': finished 2024-02-19T18:56:56.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:55 smithi088 ceph-mon[29272]: osdmap e36: 6 total, 5 up, 6 in 2024-02-19T18:56:56.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:56:57.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:56 smithi132 ceph-mon[37969]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:56:57.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:56 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/978090524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:56:57.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:56 smithi088 ceph-mon[29272]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:56:57.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:56 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/978090524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-19T18:56:59.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:56:58 smithi132 ceph-mon[37969]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:56:59.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:56:58 smithi088 ceph-mon[29272]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:01.139 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:00 smithi132 ceph-mon[37969]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:01.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:00 smithi088 ceph-mon[29272]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:02 smithi132 ceph-mon[37969]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:02 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-19T18:57:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:02 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:57:03.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:02 smithi088 ceph-mon[29272]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:03.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:02 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-19T18:57:03.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:02 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:57:04.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:03 smithi132 ceph-mon[37969]: Deploying daemon osd.5 on smithi132 2024-02-19T18:57:04.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:03 smithi088 ceph-mon[29272]: Deploying daemon osd.5 on smithi132 2024-02-19T18:57:05.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:04 smithi132 ceph-mon[37969]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:05.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:04 smithi088 ceph-mon[29272]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:07.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:06 smithi132 ceph-mon[37969]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:07.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:06 smithi088 ceph-mon[29272]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:09.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:08 smithi132 ceph-mon[37969]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:09.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:08 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:09.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:08 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:09.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:08 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:57:09.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:08 smithi088 ceph-mon[29272]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:09.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:08 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:09.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:08 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:09.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:08 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:57:10.243 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:09 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:57:10.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:09 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:57:11.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:10 smithi132 ceph-mon[37969]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:11.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:10 smithi088 ceph-mon[29272]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:11.897 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.880+0000 7f0a14832700 1 -- 172.21.15.132:0/1252707852 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0a10068c50 con 0x7f09f806c680 2024-02-19T18:57:11.897 INFO:teuthology.orchestra.run.smithi132.stdout:Created osd(s) 5 on host 'smithi132' 2024-02-19T18:57:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.882+0000 7f0a15834700 1 -- 172.21.15.132:0/1252707852 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f09f806c680 msgr2=0x7f09f806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:11.899 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.882+0000 7f0a15834700 1 --2- 172.21.15.132:0/1252707852 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f09f806c680 0x7f09f806eb30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a080060b0 tx=0x7f0a0800c040 comp rx=0 tx=0).stop 2024-02-19T18:57:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.882+0000 7f0a15834700 1 -- 172.21.15.132:0/1252707852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a101003d0 msgr2=0x7f0a10195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.882+0000 7f0a15834700 1 --2- 172.21.15.132:0/1252707852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a101003d0 0x7f0a10195f90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0a04006010 tx=0x7f0a0400bd70 comp rx=0 tx=0).stop 2024-02-19T18:57:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.882+0000 7f0a15834700 1 -- 172.21.15.132:0/1252707852 shutdown_connections 2024-02-19T18:57:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.882+0000 7f0a15834700 1 --2- 172.21.15.132:0/1252707852 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f09f806c680 0x7f09f806eb30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.882+0000 7f0a15834700 1 --2- 172.21.15.132:0/1252707852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a100ff130 0x7f0a10195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.882+0000 7f0a15834700 1 --2- 172.21.15.132:0/1252707852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a101003d0 0x7f0a10195f90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.882+0000 7f0a15834700 1 -- 172.21.15.132:0/1252707852 >> 172.21.15.132:0/1252707852 conn(0x7f0a100fa780 msgr2=0x7f0a10068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.882+0000 7f0a15834700 1 -- 172.21.15.132:0/1252707852 shutdown_connections 2024-02-19T18:57:11.900 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:11.882+0000 7f0a15834700 1 -- 172.21.15.132:0/1252707852 wait complete. 2024-02-19T18:57:12.571 DEBUG:teuthology.orchestra.run.smithi132:osd.5> sudo journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.5.service 2024-02-19T18:57:12.574 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-02-19T18:57:12.574 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd stat -f json 2024-02-19T18:57:12.793 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:13.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:12 smithi088 ceph-mon[29272]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:13.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:12 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:13.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:12 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:13.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:12 smithi088 ceph-mon[29272]: from='osd.5 [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T18:57:13.151 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:12 smithi088 ceph-mon[29272]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T18:57:13.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:12 smithi132 ceph-mon[37969]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:13.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:12 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:13.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:12 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:13.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:12 smithi132 ceph-mon[37969]: from='osd.5 [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T18:57:13.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:12 smithi132 ceph-mon[37969]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T18:57:14.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:13 smithi088 ceph-mon[29272]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-19T18:57:14.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:13 smithi088 ceph-mon[29272]: osdmap e37: 6 total, 5 up, 6 in 2024-02-19T18:57:14.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:13 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:57:14.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:13 smithi088 ceph-mon[29272]: from='osd.5 [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:57:14.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:13 smithi088 ceph-mon[29272]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:57:14.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:13 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:14.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:13 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:14.360 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:13 smithi132 ceph-mon[37969]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-19T18:57:14.360 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:13 smithi132 ceph-mon[37969]: osdmap e37: 6 total, 5 up, 6 in 2024-02-19T18:57:14.361 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:13 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:57:14.361 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:13 smithi132 ceph-mon[37969]: from='osd.5 [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:57:14.361 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:13 smithi132 ceph-mon[37969]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T18:57:14.361 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:13 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:14.361 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:13 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:14.361 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 18:57:13 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[53201]: 2024-02-19T18:57:13.998+0000 7fec3c0d1700 -1 osd.5 0 waiting for initial osdmap 2024-02-19T18:57:14.361 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 18:57:14 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[53201]: 2024-02-19T18:57:14.006+0000 7fec35ec2700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T18:57:14.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.560+0000 7f9eb8714700 1 -- 172.21.15.88:0/2908098954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb0100240 msgr2=0x7f9eb0100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:14.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.560+0000 7f9eb8714700 1 --2- 172.21.15.88:0/2908098954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb0100240 0x7f9eb0100650 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009910 tx=0x7f9eac009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:14.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.560+0000 7f9eb8714700 1 -- 172.21.15.88:0/2908098954 shutdown_connections 2024-02-19T18:57:14.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.560+0000 7f9eb8714700 1 --2- 172.21.15.88:0/2908098954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eb0101440 0x7f9eb0101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:14.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.560+0000 7f9eb8714700 1 --2- 172.21.15.88:0/2908098954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb0100240 0x7f9eb0100650 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:14.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.560+0000 7f9eb8714700 1 -- 172.21.15.88:0/2908098954 >> 172.21.15.88:0/2908098954 conn(0x7f9eb00fb7f0 msgr2=0x7f9eb00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:14.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.560+0000 7f9eb8714700 1 -- 172.21.15.88:0/2908098954 shutdown_connections 2024-02-19T18:57:14.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.560+0000 7f9eb8714700 1 -- 172.21.15.88:0/2908098954 wait complete. 2024-02-19T18:57:14.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb8714700 1 Processor -- start 2024-02-19T18:57:14.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb8714700 1 -- start start 2024-02-19T18:57:14.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb8714700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eb0100240 0x7f9eb0195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:14.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb8714700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb0101440 0x7f9eb0196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:14.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb8714700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb0196770 con 0x7f9eb0101440 2024-02-19T18:57:14.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb8714700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb019b180 con 0x7f9eb0100240 2024-02-19T18:57:14.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb5caf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb0101440 0x7f9eb0196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:14.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb5caf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb0101440 0x7f9eb0196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59002/0 (socket says 172.21.15.88:59002) 2024-02-19T18:57:14.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb5caf700 1 -- 172.21.15.88:0/1671019912 learned_addr learned my addr 172.21.15.88:0/1671019912 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:14.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb64b0700 1 --2- 172.21.15.88:0/1671019912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eb0100240 0x7f9eb0195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:14.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb5caf700 1 -- 172.21.15.88:0/1671019912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eb0100240 msgr2=0x7f9eb0195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:14.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb5caf700 1 --2- 172.21.15.88:0/1671019912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eb0100240 0x7f9eb0195c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:14.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.561+0000 7f9eb5caf700 1 -- 172.21.15.88:0/1671019912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eac0095f0 con 0x7f9eb0101440 2024-02-19T18:57:14.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.562+0000 7f9eb5caf700 1 --2- 172.21.15.88:0/1671019912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb0101440 0x7f9eb0196150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000b4b0 tx=0x7f9ea000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:14.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.562+0000 7f9ea77fe700 1 -- 172.21.15.88:0/1671019912 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ea0011800 con 0x7f9eb0101440 2024-02-19T18:57:14.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.562+0000 7f9eb8714700 1 -- 172.21.15.88:0/1671019912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb019b380 con 0x7f9eb0101440 2024-02-19T18:57:14.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.562+0000 7f9eb8714700 1 -- 172.21.15.88:0/1671019912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb019b8a0 con 0x7f9eb0101440 2024-02-19T18:57:14.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.562+0000 7f9ea77fe700 1 -- 172.21.15.88:0/1671019912 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9ea0011e10 con 0x7f9eb0101440 2024-02-19T18:57:14.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.562+0000 7f9ea77fe700 1 -- 172.21.15.88:0/1671019912 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ea00193b0 con 0x7f9eb0101440 2024-02-19T18:57:14.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.563+0000 7f9eb8714700 1 -- 172.21.15.88:0/1671019912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb0108e10 con 0x7f9eb0101440 2024-02-19T18:57:14.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.563+0000 7f9ea77fe700 1 -- 172.21.15.88:0/1671019912 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f9ea001a7b0 con 0x7f9eb0101440 2024-02-19T18:57:14.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.564+0000 7f9ea77fe700 1 --2- 172.21.15.88:0/1671019912 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9e9c06c680 0x7f9e9c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:14.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.564+0000 7f9ea77fe700 1 -- 172.21.15.88:0/1671019912 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4358+0+0 (secure 0 0 0) 0x7f9ea008afb0 con 0x7f9eb0101440 2024-02-19T18:57:14.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.564+0000 7f9eb64b0700 1 --2- 172.21.15.88:0/1671019912 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9e9c06c680 0x7f9e9c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:14.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.564+0000 7f9eb64b0700 1 --2- 172.21.15.88:0/1671019912 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9e9c06c680 0x7f9e9c06eb30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005090 tx=0x7f9eac00b540 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:14.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.569+0000 7f9ea77fe700 1 -- 172.21.15.88:0/1671019912 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9ea0055ae0 con 0x7f9eb0101440 2024-02-19T18:57:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.760+0000 7f9eb8714700 1 -- 172.21.15.88:0/1671019912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f9eb019bf60 con 0x7f9eb0101440 2024-02-19T18:57:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.760+0000 7f9ea77fe700 1 -- 172.21.15.88:0/1671019912 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v38) v1 ==== 74+0+130 (secure 0 0 0) 0x7f9ea0059100 con 0x7f9eb0101440 2024-02-19T18:57:14.779 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:57:14.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.762+0000 7f9eb8714700 1 -- 172.21.15.88:0/1671019912 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9e9c06c680 msgr2=0x7f9e9c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.762+0000 7f9eb8714700 1 --2- 172.21.15.88:0/1671019912 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9e9c06c680 0x7f9e9c06eb30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9eac005090 tx=0x7f9eac00b540 comp rx=0 tx=0).stop 2024-02-19T18:57:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.762+0000 7f9eb8714700 1 -- 172.21.15.88:0/1671019912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb0101440 msgr2=0x7f9eb0196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.762+0000 7f9eb8714700 1 --2- 172.21.15.88:0/1671019912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb0101440 0x7f9eb0196150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000b4b0 tx=0x7f9ea000b870 comp rx=0 tx=0).stop 2024-02-19T18:57:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.762+0000 7f9eb8714700 1 -- 172.21.15.88:0/1671019912 shutdown_connections 2024-02-19T18:57:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.762+0000 7f9eb8714700 1 --2- 172.21.15.88:0/1671019912 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9e9c06c680 0x7f9e9c06eb30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.762+0000 7f9eb8714700 1 --2- 172.21.15.88:0/1671019912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9eb0100240 0x7f9eb0195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.762+0000 7f9eb8714700 1 --2- 172.21.15.88:0/1671019912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9eb0101440 0x7f9eb0196150 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.762+0000 7f9eb8714700 1 -- 172.21.15.88:0/1671019912 >> 172.21.15.88:0/1671019912 conn(0x7f9eb00fb7f0 msgr2=0x7f9eb0104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.762+0000 7f9eb8714700 1 -- 172.21.15.88:0/1671019912 shutdown_connections 2024-02-19T18:57:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:14.762+0000 7f9eb8714700 1 -- 172.21.15.88:0/1671019912 wait complete. 2024-02-19T18:57:15.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:15 smithi088 ceph-mon[29272]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:15.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:15 smithi088 ceph-mon[29272]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-02-19T18:57:15.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:15 smithi088 ceph-mon[29272]: osdmap e38: 6 total, 5 up, 6 in 2024-02-19T18:57:15.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:57:15.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:57:15.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:15 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1671019912' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-19T18:57:15.379 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:15 smithi132 ceph-mon[37969]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:15.379 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:15 smithi132 ceph-mon[37969]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]': finished 2024-02-19T18:57:15.379 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:15 smithi132 ceph-mon[37969]: osdmap e38: 6 total, 5 up, 6 in 2024-02-19T18:57:15.379 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:57:15.379 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:57:15.379 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:15 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1671019912' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-19T18:57:15.442 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":38,"num_osds":6,"num_up_osds":5,"osd_up_since":1708369008,"num_in_osds":6,"osd_in_since":1708369015,"num_remapped_pgs":0} 2024-02-19T18:57:16.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:16 smithi132 ceph-mon[37969]: purged_snaps scrub starts 2024-02-19T18:57:16.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:16 smithi132 ceph-mon[37969]: purged_snaps scrub ok 2024-02-19T18:57:16.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:57:16.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:16 smithi132 ceph-mon[37969]: osd.5 [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780] boot 2024-02-19T18:57:16.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:16 smithi132 ceph-mon[37969]: osdmap e39: 6 total, 6 up, 6 in 2024-02-19T18:57:16.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:16 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:57:16.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:16 smithi088 ceph-mon[29272]: purged_snaps scrub starts 2024-02-19T18:57:16.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:16 smithi088 ceph-mon[29272]: purged_snaps scrub ok 2024-02-19T18:57:16.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:57:16.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:16 smithi088 ceph-mon[29272]: osd.5 [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780] boot 2024-02-19T18:57:16.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:16 smithi088 ceph-mon[29272]: osdmap e39: 6 total, 6 up, 6 in 2024-02-19T18:57:16.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:16 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T18:57:16.443 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd stat -f json 2024-02-19T18:57:16.661 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:17.073 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:17 smithi088 ceph-mon[29272]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:17.073 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:17 smithi088 ceph-mon[29272]: osdmap e40: 6 total, 6 up, 6 in 2024-02-19T18:57:17.173 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:17 smithi132 ceph-mon[37969]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-02-19T18:57:17.173 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:17 smithi132 ceph-mon[37969]: osdmap e40: 6 total, 6 up, 6 in 2024-02-19T18:57:18.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.253+0000 7f5351dac700 1 -- 172.21.15.88:0/799271833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f534c1014d0 msgr2=0x7f534c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:18.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.253+0000 7f5351dac700 1 --2- 172.21.15.88:0/799271833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f534c1014d0 0x7f534c103950 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5340009910 tx=0x7f5340009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:18.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.253+0000 7f5351dac700 1 -- 172.21.15.88:0/799271833 shutdown_connections 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.253+0000 7f5351dac700 1 --2- 172.21.15.88:0/799271833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f534c1014d0 0x7f534c103950 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.253+0000 7f5351dac700 1 --2- 172.21.15.88:0/799271833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f534c0feb70 0x7f534c100f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.253+0000 7f5351dac700 1 -- 172.21.15.88:0/799271833 >> 172.21.15.88:0/799271833 conn(0x7f534c0fa7a0 msgr2=0x7f534c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.253+0000 7f5351dac700 1 -- 172.21.15.88:0/799271833 shutdown_connections 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.253+0000 7f5351dac700 1 -- 172.21.15.88:0/799271833 wait complete. 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.254+0000 7f5351dac700 1 Processor -- start 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.254+0000 7f5351dac700 1 -- start start 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.254+0000 7f5351dac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f534c0feb70 0x7f534c19c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.254+0000 7f5351dac700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f534c1014d0 0x7f534c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.254+0000 7f5351dac700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c19cf20 con 0x7f534c1014d0 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.254+0000 7f5351dac700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f534c19d060 con 0x7f534c0feb70 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.254+0000 7f534affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f534c1014d0 0x7f534c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.254+0000 7f534affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f534c1014d0 0x7f534c19c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52006/0 (socket says 172.21.15.88:52006) 2024-02-19T18:57:18.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.254+0000 7f534affd700 1 -- 172.21.15.88:0/300163391 learned_addr learned my addr 172.21.15.88:0/300163391 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:18.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.254+0000 7f534b7fe700 1 --2- 172.21.15.88:0/300163391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f534c0feb70 0x7f534c19c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:18.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.255+0000 7f534affd700 1 -- 172.21.15.88:0/300163391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f534c0feb70 msgr2=0x7f534c19c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:18.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.255+0000 7f534affd700 1 --2- 172.21.15.88:0/300163391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f534c0feb70 0x7f534c19c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:18.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.255+0000 7f534affd700 1 -- 172.21.15.88:0/300163391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53400095f0 con 0x7f534c1014d0 2024-02-19T18:57:18.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.255+0000 7f534affd700 1 --2- 172.21.15.88:0/300163391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f534c1014d0 0x7f534c19c900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5340006010 tx=0x7f534000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:18.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.255+0000 7f5348ff9700 1 -- 172.21.15.88:0/300163391 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f534001d070 con 0x7f534c1014d0 2024-02-19T18:57:18.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.255+0000 7f5351dac700 1 -- 172.21.15.88:0/300163391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f534c1a1ab0 con 0x7f534c1014d0 2024-02-19T18:57:18.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.255+0000 7f5348ff9700 1 -- 172.21.15.88:0/300163391 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f534000f460 con 0x7f534c1014d0 2024-02-19T18:57:18.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.255+0000 7f5348ff9700 1 -- 172.21.15.88:0/300163391 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5340021640 con 0x7f534c1014d0 2024-02-19T18:57:18.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.255+0000 7f5351dac700 1 -- 172.21.15.88:0/300163391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c1a1fa0 con 0x7f534c1014d0 2024-02-19T18:57:18.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.256+0000 7f5351dac700 1 -- 172.21.15.88:0/300163391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534c1961e0 con 0x7f534c1014d0 2024-02-19T18:57:18.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.256+0000 7f5348ff9700 1 -- 172.21.15.88:0/300163391 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f534000fab0 con 0x7f534c1014d0 2024-02-19T18:57:18.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.257+0000 7f5348ff9700 1 --2- 172.21.15.88:0/300163391 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f533406c680 0x7f533406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:18.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.257+0000 7f5348ff9700 1 -- 172.21.15.88:0/300163391 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f534008c8f0 con 0x7f534c1014d0 2024-02-19T18:57:18.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.257+0000 7f534b7fe700 1 --2- 172.21.15.88:0/300163391 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f533406c680 0x7f533406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:18.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.257+0000 7f534b7fe700 1 --2- 172.21.15.88:0/300163391 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f533406c680 0x7f533406eb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f534c069890 tx=0x7f533c00a380 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:18.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.261+0000 7f5348ff9700 1 -- 172.21.15.88:0/300163391 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f534005ada0 con 0x7f534c1014d0 2024-02-19T18:57:18.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: Detected new or changed devices on smithi132 2024-02-19T18:57:18.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:18.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:18.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:57:18.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:57:18.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:57:18.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: Adjusting osd_memory_target on smithi132 to 4970M 2024-02-19T18:57:18.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:18.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:57:18.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:57:18.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:18.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:57:18.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:57:18.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:57:18.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:18.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:18 smithi088 ceph-mon[29272]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:18.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.450+0000 7f5351dac700 1 -- 172.21.15.88:0/300163391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f534c02ce70 con 0x7f534c1014d0 2024-02-19T18:57:18.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.451+0000 7f5348ff9700 1 -- 172.21.15.88:0/300163391 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f534005a930 con 0x7f534c1014d0 2024-02-19T18:57:18.468 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:57:18.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.452+0000 7f5351dac700 1 -- 172.21.15.88:0/300163391 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f533406c680 msgr2=0x7f533406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:18.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.452+0000 7f5351dac700 1 --2- 172.21.15.88:0/300163391 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f533406c680 0x7f533406eb30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f534c069890 tx=0x7f533c00a380 comp rx=0 tx=0).stop 2024-02-19T18:57:18.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.452+0000 7f5351dac700 1 -- 172.21.15.88:0/300163391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f534c1014d0 msgr2=0x7f534c19c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:18.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.452+0000 7f5351dac700 1 --2- 172.21.15.88:0/300163391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f534c1014d0 0x7f534c19c900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5340006010 tx=0x7f534000bd70 comp rx=0 tx=0).stop 2024-02-19T18:57:18.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.453+0000 7f5351dac700 1 -- 172.21.15.88:0/300163391 shutdown_connections 2024-02-19T18:57:18.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.453+0000 7f5351dac700 1 --2- 172.21.15.88:0/300163391 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f533406c680 0x7f533406eb30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:18.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.453+0000 7f5351dac700 1 --2- 172.21.15.88:0/300163391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f534c0feb70 0x7f534c19c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:18.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.453+0000 7f5351dac700 1 --2- 172.21.15.88:0/300163391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f534c1014d0 0x7f534c19c900 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:18.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.453+0000 7f5351dac700 1 -- 172.21.15.88:0/300163391 >> 172.21.15.88:0/300163391 conn(0x7f534c0fa7a0 msgr2=0x7f534c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:18.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.453+0000 7f5351dac700 1 -- 172.21.15.88:0/300163391 shutdown_connections 2024-02-19T18:57:18.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:18.453+0000 7f5351dac700 1 -- 172.21.15.88:0/300163391 wait complete. 2024-02-19T18:57:18.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: Detected new or changed devices on smithi132 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: Adjusting osd_memory_target on smithi132 to 4970M 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:57:18.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:57:18.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:18 smithi132 ceph-mon[37969]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:19.116 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1708369034,"num_in_osds":6,"osd_in_since":1708369015,"num_remapped_pgs":0} 2024-02-19T18:57:19.117 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd dump --format=json 2024-02-19T18:57:19.320 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:19.677 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:19 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/300163391' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-19T18:57:19.694 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:19 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/300163391' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-19T18:57:20.486 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:20 smithi088 ceph-mon[29272]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:20.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:20 smithi132 ceph-mon[37969]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:20.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.958+0000 7fab74a1e700 1 -- 172.21.15.88:0/3019538382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab70104130 msgr2=0x7fab70104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:20.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.958+0000 7fab74a1e700 1 --2- 172.21.15.88:0/3019538382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab70104130 0x7fab70104500 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fab58009910 tx=0x7fab58009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:20.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.958+0000 7fab74a1e700 1 -- 172.21.15.88:0/3019538382 shutdown_connections 2024-02-19T18:57:20.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.958+0000 7fab74a1e700 1 --2- 172.21.15.88:0/3019538382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab700ffbd0 0x7fab70100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:20.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.958+0000 7fab74a1e700 1 --2- 172.21.15.88:0/3019538382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab70104130 0x7fab70104500 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:20.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.958+0000 7fab74a1e700 1 -- 172.21.15.88:0/3019538382 >> 172.21.15.88:0/3019538382 conn(0x7fab70075130 msgr2=0x7fab70075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:20.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.958+0000 7fab74a1e700 1 -- 172.21.15.88:0/3019538382 shutdown_connections 2024-02-19T18:57:20.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.958+0000 7fab74a1e700 1 -- 172.21.15.88:0/3019538382 wait complete. 2024-02-19T18:57:20.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.959+0000 7fab74a1e700 1 Processor -- start 2024-02-19T18:57:20.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.959+0000 7fab74a1e700 1 -- start start 2024-02-19T18:57:20.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.959+0000 7fab74a1e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab700ffbd0 0x7fab70194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:20.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.959+0000 7fab74a1e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab70104130 0x7fab70195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:20.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.959+0000 7fab74a1e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab701959f0 con 0x7fab70104130 2024-02-19T18:57:20.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.959+0000 7fab74a1e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab70199780 con 0x7fab700ffbd0 2024-02-19T18:57:20.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab6e59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab700ffbd0 0x7fab70194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:20.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab6dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab70104130 0x7fab70195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:20.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab6e59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab700ffbd0 0x7fab70194dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:52890/0 (socket says 172.21.15.88:52890) 2024-02-19T18:57:20.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab6e59c700 1 -- 172.21.15.88:0/4223105497 learned_addr learned my addr 172.21.15.88:0/4223105497 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:20.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab6dd9b700 1 -- 172.21.15.88:0/4223105497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab700ffbd0 msgr2=0x7fab70194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:20.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab6dd9b700 1 --2- 172.21.15.88:0/4223105497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab700ffbd0 0x7fab70194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:20.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab6dd9b700 1 -- 172.21.15.88:0/4223105497 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab580095f0 con 0x7fab70104130 2024-02-19T18:57:20.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab6dd9b700 1 --2- 172.21.15.88:0/4223105497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab70104130 0x7fab70195310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fab6000b3d0 tx=0x7fab6000b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:20.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab677fe700 1 -- 172.21.15.88:0/4223105497 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab60011800 con 0x7fab70104130 2024-02-19T18:57:20.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab677fe700 1 -- 172.21.15.88:0/4223105497 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fab6000be90 con 0x7fab70104130 2024-02-19T18:57:20.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab74a1e700 1 -- 172.21.15.88:0/4223105497 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab70199a60 con 0x7fab70104130 2024-02-19T18:57:20.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab677fe700 1 -- 172.21.15.88:0/4223105497 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab60010e70 con 0x7fab70104130 2024-02-19T18:57:20.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.960+0000 7fab74a1e700 1 -- 172.21.15.88:0/4223105497 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab70199fb0 con 0x7fab70104130 2024-02-19T18:57:20.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.962+0000 7fab74a1e700 1 -- 172.21.15.88:0/4223105497 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab701076a0 con 0x7fab70104130 2024-02-19T18:57:20.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.962+0000 7fab677fe700 1 -- 172.21.15.88:0/4223105497 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fab6001a440 con 0x7fab70104130 2024-02-19T18:57:20.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.962+0000 7fab677fe700 1 --2- 172.21.15.88:0/4223105497 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab5c06c680 0x7fab5c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:20.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.962+0000 7fab677fe700 1 -- 172.21.15.88:0/4223105497 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fab6008af70 con 0x7fab70104130 2024-02-19T18:57:20.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.962+0000 7fab6e59c700 1 --2- 172.21.15.88:0/4223105497 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab5c06c680 0x7fab5c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:20.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.962+0000 7fab6e59c700 1 --2- 172.21.15.88:0/4223105497 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab5c06c680 0x7fab5c06eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab58009fd0 tx=0x7fab5801a040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:20.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:20.966+0000 7fab677fe700 1 -- 172.21.15.88:0/4223105497 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fab60055990 con 0x7fab70104130 2024-02-19T18:57:21.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.156+0000 7fab74a1e700 1 -- 172.21.15.88:0/4223105497 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab701961d0 con 0x7fab70104130 2024-02-19T18:57:21.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.157+0000 7fab677fe700 1 -- 172.21.15.88:0/4223105497 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11336 (secure 0 0 0) 0x7fab60058fb0 con 0x7fab70104130 2024-02-19T18:57:21.175 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:57:21.175 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","created":"2024-02-19T18:50:27.026977+0000","modified":"2024-02-19T18:57:16.003216+0000","last_up_change":"2024-02-19T18:57:14.993225+0000","last_in_change":"2024-02-19T18:56:55.518417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T18:55:57.936009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"787c5fec-c403-4975-92c3-8b68fcf9e399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6803","nonce":2929682937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6805","nonce":2929682937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6809","nonce":2929682937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6807","nonce":2929682937}]},"public_addr":"172.21.15.88:6803/2929682937","cluster_addr":"172.21.15.88:6805/2929682937","heartbeat_back_addr":"172.21.15.88:6809/2929682937","heartbeat_front_addr":"172.21.15.88:6807/2929682937","state":["exists","up"]},{"osd":1,"uuid":"a459843c-43d7-480f-a76b-d52d6e8ca061","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6811","nonce":3005125945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6813","nonce":3005125945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6817","nonce":3005125945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6815","nonce":3005125945}]},"public_addr":"172.21.15.88:6811/3005125945","cluster_addr":"172.21.15.88:6813/3005125945","heartbeat_back_addr":"172.21.15.88:6817/3005125945","heartbeat_front_addr":"172.21.15.88:6815/3005125945","state":["exists","up"]},{"osd":2,"uuid":"66334f86-7411-4587-b96b-22b61bf5201a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6819","nonce":525694249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6821","nonce":525694249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6825","nonce":525694249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6823","nonce":525694249}]},"public_addr":"172.21.15.88:6819/525694249","cluster_addr":"172.21.15.88:6821/525694249","heartbeat_back_addr":"172.21.15.88:6825/525694249","heartbeat_front_addr":"172.21.15.88:6823/525694249","state":["exists","up"]},{"osd":3,"uuid":"733318f2-be90-44d7-8c38-fb773f7da169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6801","nonce":2008345174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6803","nonce":2008345174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6807","nonce":2008345174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6805","nonce":2008345174}]},"public_addr":"172.21.15.132:6801/2008345174","cluster_addr":"172.21.15.132:6803/2008345174","heartbeat_back_addr":"172.21.15.132:6807/2008345174","heartbeat_front_addr":"172.21.15.132:6805/2008345174","state":["exists","up"]},{"osd":4,"uuid":"3b41c4ff-b4b1-4279-b973-0c3c1c62373e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6809","nonce":3138829446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6811","nonce":3138829446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6815","nonce":3138829446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6813","nonce":3138829446}]},"public_addr":"172.21.15.132:6809/3138829446","cluster_addr":"172.21.15.132:6811/3138829446","heartbeat_back_addr":"172.21.15.132:6815/3138829446","heartbeat_front_addr":"172.21.15.132:6813/3138829446","state":["exists","up"]},{"osd":5,"uuid":"367bd67a-e45d-40ff-9aac-1f88755641ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6817","nonce":2563696780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6819","nonce":2563696780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6823","nonce":2563696780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6821","nonce":2563696780}]},"public_addr":"172.21.15.132:6817/2563696780","cluster_addr":"172.21.15.132:6819/2563696780","heartbeat_back_addr":"172.21.15.132:6823/2563696780","heartbeat_front_addr":"172.21.15.132:6821/2563696780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:54:58.653733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:26.141001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:55.352595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:19.994273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:46.399893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:57:13.114456+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:0/4069571809":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/2177498492":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6800/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6801/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/2372050564":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/3565328320":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/3237698093":"2024-02-20T18:51:05.484896+0000","172.21.15.88:0/2338689461":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/3838380819":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6800/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6800/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/3491684321":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6801/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6801/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/387402116":"2024-02-20T18:51:05.484896+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T18:57:21.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.159+0000 7fab74a1e700 1 -- 172.21.15.88:0/4223105497 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab5c06c680 msgr2=0x7fab5c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:21.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.159+0000 7fab74a1e700 1 --2- 172.21.15.88:0/4223105497 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab5c06c680 0x7fab5c06eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab58009fd0 tx=0x7fab5801a040 comp rx=0 tx=0).stop 2024-02-19T18:57:21.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.159+0000 7fab74a1e700 1 -- 172.21.15.88:0/4223105497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab70104130 msgr2=0x7fab70195310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:21.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.159+0000 7fab74a1e700 1 --2- 172.21.15.88:0/4223105497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab70104130 0x7fab70195310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fab6000b3d0 tx=0x7fab6000b6e0 comp rx=0 tx=0).stop 2024-02-19T18:57:21.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.159+0000 7fab74a1e700 1 -- 172.21.15.88:0/4223105497 shutdown_connections 2024-02-19T18:57:21.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.159+0000 7fab74a1e700 1 --2- 172.21.15.88:0/4223105497 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab5c06c680 0x7fab5c06eb30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:21.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.159+0000 7fab74a1e700 1 --2- 172.21.15.88:0/4223105497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab700ffbd0 0x7fab70194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:21.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.159+0000 7fab74a1e700 1 --2- 172.21.15.88:0/4223105497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab70104130 0x7fab70195310 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:21.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.159+0000 7fab74a1e700 1 -- 172.21.15.88:0/4223105497 >> 172.21.15.88:0/4223105497 conn(0x7fab70075130 msgr2=0x7fab700feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:21.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.159+0000 7fab74a1e700 1 -- 172.21.15.88:0/4223105497 shutdown_connections 2024-02-19T18:57:21.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:21.159+0000 7fab74a1e700 1 -- 172.21.15.88:0/4223105497 wait complete. 2024-02-19T18:57:21.496 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:21 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/4223105497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T18:57:21.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:21 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/4223105497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T18:57:21.880 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T18:55:57.936009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-19T18:57:21.880 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-19T18:57:22.090 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:23.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:22 smithi132 ceph-mon[37969]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:23.251 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:22 smithi088 ceph-mon[29272]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:23.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.633+0000 7f4445d78700 1 -- 172.21.15.88:0/900797986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4440104130 msgr2=0x7f4440104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.633+0000 7f4445d78700 1 --2- 172.21.15.88:0/900797986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4440104130 0x7f4440104500 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f4430009910 tx=0x7f4430009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.633+0000 7f4445d78700 1 -- 172.21.15.88:0/900797986 shutdown_connections 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.633+0000 7f4445d78700 1 --2- 172.21.15.88:0/900797986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44400ffbd0 0x7f4440100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.633+0000 7f4445d78700 1 --2- 172.21.15.88:0/900797986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4440104130 0x7f4440104500 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.633+0000 7f4445d78700 1 -- 172.21.15.88:0/900797986 >> 172.21.15.88:0/900797986 conn(0x7f4440075130 msgr2=0x7f4440075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.633+0000 7f4445d78700 1 -- 172.21.15.88:0/900797986 shutdown_connections 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.633+0000 7f4445d78700 1 -- 172.21.15.88:0/900797986 wait complete. 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4445d78700 1 Processor -- start 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4445d78700 1 -- start start 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4445d78700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44400ffbd0 0x7f4440100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4445d78700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4440104130 0x7f4440101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4445d78700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44401018b0 con 0x7f4440104130 2024-02-19T18:57:23.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4445d78700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44401019f0 con 0x7f44400ffbd0 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4436ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4440104130 0x7f4440101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4436ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4440104130 0x7f4440101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52046/0 (socket says 172.21.15.88:52046) 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4436ffd700 1 -- 172.21.15.88:0/4054693543 learned_addr learned my addr 172.21.15.88:0/4054693543 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f443f7fe700 1 --2- 172.21.15.88:0/4054693543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44400ffbd0 0x7f4440100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4436ffd700 1 -- 172.21.15.88:0/4054693543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44400ffbd0 msgr2=0x7f4440100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4436ffd700 1 --2- 172.21.15.88:0/4054693543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44400ffbd0 0x7f4440100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.634+0000 7f4436ffd700 1 -- 172.21.15.88:0/4054693543 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44300095f0 con 0x7f4440104130 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.635+0000 7f4436ffd700 1 --2- 172.21.15.88:0/4054693543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4440104130 0x7f4440101370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f442800b4b0 tx=0x7f442800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.635+0000 7f443d7fa700 1 -- 172.21.15.88:0/4054693543 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4428011800 con 0x7f4440104130 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.635+0000 7f443d7fa700 1 -- 172.21.15.88:0/4054693543 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4428011e10 con 0x7f4440104130 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.635+0000 7f4445d78700 1 -- 172.21.15.88:0/4054693543 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44401a2070 con 0x7f4440104130 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.635+0000 7f443d7fa700 1 -- 172.21.15.88:0/4054693543 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f442800f2e0 con 0x7f4440104130 2024-02-19T18:57:23.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.635+0000 7f4445d78700 1 -- 172.21.15.88:0/4054693543 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44401a2480 con 0x7f4440104130 2024-02-19T18:57:23.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.636+0000 7f4445d78700 1 -- 172.21.15.88:0/4054693543 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44401086a0 con 0x7f4440104130 2024-02-19T18:57:23.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.636+0000 7f443d7fa700 1 -- 172.21.15.88:0/4054693543 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f4428010380 con 0x7f4440104130 2024-02-19T18:57:23.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.637+0000 7f443d7fa700 1 --2- 172.21.15.88:0/4054693543 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f442c06c680 0x7f442c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:23.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.637+0000 7f443d7fa700 1 -- 172.21.15.88:0/4054693543 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f442808b1b0 con 0x7f4440104130 2024-02-19T18:57:23.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.637+0000 7f443f7fe700 1 --2- 172.21.15.88:0/4054693543 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f442c06c680 0x7f442c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:23.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.637+0000 7f443f7fe700 1 --2- 172.21.15.88:0/4054693543 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f442c06c680 0x7f442c06eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f443000b5c0 tx=0x7f443001a040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:23.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.641+0000 7f443d7fa700 1 -- 172.21.15.88:0/4054693543 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4428055bd0 con 0x7f4440104130 2024-02-19T18:57:23.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.834+0000 7f4445d78700 1 -- 172.21.15.88:0/4054693543 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f444002cc30 con 0x7f4440104130 2024-02-19T18:57:23.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.834+0000 7f443d7fa700 1 -- 172.21.15.88:0/4054693543 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f44280591f0 con 0x7f4440104130 2024-02-19T18:57:23.852 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-02-19T18:57:23.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.836+0000 7f4445d78700 1 -- 172.21.15.88:0/4054693543 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f442c06c680 msgr2=0x7f442c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:23.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.836+0000 7f4445d78700 1 --2- 172.21.15.88:0/4054693543 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f442c06c680 0x7f442c06eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f443000b5c0 tx=0x7f443001a040 comp rx=0 tx=0).stop 2024-02-19T18:57:23.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.836+0000 7f4445d78700 1 -- 172.21.15.88:0/4054693543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4440104130 msgr2=0x7f4440101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:23.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.836+0000 7f4445d78700 1 --2- 172.21.15.88:0/4054693543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4440104130 0x7f4440101370 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f442800b4b0 tx=0x7f442800b870 comp rx=0 tx=0).stop 2024-02-19T18:57:23.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.836+0000 7f4445d78700 1 -- 172.21.15.88:0/4054693543 shutdown_connections 2024-02-19T18:57:23.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.836+0000 7f4445d78700 1 --2- 172.21.15.88:0/4054693543 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f442c06c680 0x7f442c06eb30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:23.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.836+0000 7f4445d78700 1 --2- 172.21.15.88:0/4054693543 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44400ffbd0 0x7f4440100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:23.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.836+0000 7f4445d78700 1 --2- 172.21.15.88:0/4054693543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4440104130 0x7f4440101370 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:23.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.836+0000 7f4445d78700 1 -- 172.21.15.88:0/4054693543 >> 172.21.15.88:0/4054693543 conn(0x7f4440075130 msgr2=0x7f44400fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:23.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.836+0000 7f4445d78700 1 -- 172.21.15.88:0/4054693543 shutdown_connections 2024-02-19T18:57:23.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:23.836+0000 7f4445d78700 1 -- 172.21.15.88:0/4054693543 wait complete. 2024-02-19T18:57:24.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:23 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/4054693543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T18:57:24.250 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:23 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/4054693543' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T18:57:24.550 INFO:tasks.cephadm:Setting up client nodes... 2024-02-19T18:57:24.550 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-19T18:57:24.761 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:25.196 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:24 smithi088 ceph-mon[29272]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:25.197 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:24 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:57:25.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:24 smithi132 ceph-mon[37969]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:25.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:24 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:57:26.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.320+0000 7fc4d19db700 1 -- 172.21.15.88:0/3141700629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4cc1018a0 msgr2=0x7fc4cc101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:26.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.320+0000 7fc4d19db700 1 --2- 172.21.15.88:0/3141700629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc101cb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009910 tx=0x7fc4bc009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:26.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.320+0000 7fc4d19db700 1 -- 172.21.15.88:0/3141700629 shutdown_connections 2024-02-19T18:57:26.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.320+0000 7fc4d19db700 1 --2- 172.21.15.88:0/3141700629 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc1021f0 0x7fc4cc0fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:26.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.320+0000 7fc4d19db700 1 --2- 172.21.15.88:0/3141700629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc101cb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:26.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.320+0000 7fc4d19db700 1 -- 172.21.15.88:0/3141700629 >> 172.21.15.88:0/3141700629 conn(0x7fc4cc0f9f10 msgr2=0x7fc4cc0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:26.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.320+0000 7fc4d19db700 1 -- 172.21.15.88:0/3141700629 shutdown_connections 2024-02-19T18:57:26.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.320+0000 7fc4d19db700 1 -- 172.21.15.88:0/3141700629 wait complete. 2024-02-19T18:57:26.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.320+0000 7fc4d19db700 1 Processor -- start 2024-02-19T18:57:26.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.320+0000 7fc4d19db700 1 -- start start 2024-02-19T18:57:26.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4d19db700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:26.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4d19db700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4cc1021f0 0x7fc4cc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:26.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4d19db700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc1988b0 con 0x7fc4cc1021f0 2024-02-19T18:57:26.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4d19db700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4cc1989f0 con 0x7fc4cc1018a0 2024-02-19T18:57:26.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4c27fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4cc1021f0 0x7fc4cc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4c27fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4cc1021f0 0x7fc4cc198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60724/0 (socket says 172.21.15.88:60724) 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4c27fc700 1 -- 172.21.15.88:0/1464162337 learned_addr learned my addr 172.21.15.88:0/1464162337 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4caffd700 1 --2- 172.21.15.88:0/1464162337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4c27fc700 1 -- 172.21.15.88:0/1464162337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc1018a0 msgr2=0x7fc4cc197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4c27fc700 1 --2- 172.21.15.88:0/1464162337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4c27fc700 1 -- 172.21.15.88:0/1464162337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc0095f0 con 0x7fc4cc1021f0 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4c27fc700 1 --2- 172.21.15.88:0/1464162337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4cc1021f0 0x7fc4cc198290 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e8f0 tx=0x7fc4b400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.321+0000 7fc4d09d9700 1 -- 172.21.15.88:0/1464162337 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4b4009790 con 0x7fc4cc1021f0 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.322+0000 7fc4d09d9700 1 -- 172.21.15.88:0/1464162337 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc4b400c810 con 0x7fc4cc1021f0 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.322+0000 7fc4d19db700 1 -- 172.21.15.88:0/1464162337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4cc19d4a0 con 0x7fc4cc1021f0 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.322+0000 7fc4d09d9700 1 -- 172.21.15.88:0/1464162337 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4b40055b0 con 0x7fc4cc1021f0 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.322+0000 7fc4d19db700 1 -- 172.21.15.88:0/1464162337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4cc19d9f0 con 0x7fc4cc1021f0 2024-02-19T18:57:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.323+0000 7fc4d19db700 1 -- 172.21.15.88:0/1464162337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4cc105720 con 0x7fc4cc1021f0 2024-02-19T18:57:26.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.323+0000 7fc4d09d9700 1 -- 172.21.15.88:0/1464162337 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fc4b400c980 con 0x7fc4cc1021f0 2024-02-19T18:57:26.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.323+0000 7fc4d09d9700 1 --2- 172.21.15.88:0/1464162337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fc4b806c680 0x7fc4b806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:26.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.323+0000 7fc4d09d9700 1 -- 172.21.15.88:0/1464162337 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc4b4014070 con 0x7fc4cc1021f0 2024-02-19T18:57:26.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.323+0000 7fc4caffd700 1 --2- 172.21.15.88:0/1464162337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fc4b806c680 0x7fc4b806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:26.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.324+0000 7fc4caffd700 1 --2- 172.21.15.88:0/1464162337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fc4b806c680 0x7fc4b806eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00b6f0 tx=0x7fc4bc005a90 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:26.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.328+0000 7fc4d09d9700 1 -- 172.21.15.88:0/1464162337 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc4b4059d80 con 0x7fc4cc1021f0 2024-02-19T18:57:26.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.607+0000 7fc4d19db700 1 -- 172.21.15.88:0/1464162337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc4cc02d070 con 0x7fc4cc1021f0 2024-02-19T18:57:26.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.611+0000 7fc4d09d9700 1 -- 172.21.15.88:0/1464162337 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc4b4059910 con 0x7fc4cc1021f0 2024-02-19T18:57:26.629 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-02-19T18:57:26.629 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCWpNNldI5QJBAAKB7RJ3WA9hOCRmT3zcWBaA== 2024-02-19T18:57:26.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.613+0000 7fc4d19db700 1 -- 172.21.15.88:0/1464162337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fc4b806c680 msgr2=0x7fc4b806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:26.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.613+0000 7fc4d19db700 1 --2- 172.21.15.88:0/1464162337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fc4b806c680 0x7fc4b806eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00b6f0 tx=0x7fc4bc005a90 comp rx=0 tx=0).stop 2024-02-19T18:57:26.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.613+0000 7fc4d19db700 1 -- 172.21.15.88:0/1464162337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4cc1021f0 msgr2=0x7fc4cc198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:26.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.613+0000 7fc4d19db700 1 --2- 172.21.15.88:0/1464162337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4cc1021f0 0x7fc4cc198290 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc4b400e8f0 tx=0x7fc4b400ec00 comp rx=0 tx=0).stop 2024-02-19T18:57:26.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.614+0000 7fc4d19db700 1 -- 172.21.15.88:0/1464162337 shutdown_connections 2024-02-19T18:57:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.614+0000 7fc4d19db700 1 --2- 172.21.15.88:0/1464162337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fc4b806c680 0x7fc4b806eb30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.614+0000 7fc4d19db700 1 --2- 172.21.15.88:0/1464162337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc4cc1018a0 0x7fc4cc197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.614+0000 7fc4d19db700 1 --2- 172.21.15.88:0/1464162337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4cc1021f0 0x7fc4cc198290 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.614+0000 7fc4d19db700 1 -- 172.21.15.88:0/1464162337 >> 172.21.15.88:0/1464162337 conn(0x7fc4cc0f9f10 msgr2=0x7fc4cc102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.614+0000 7fc4d19db700 1 -- 172.21.15.88:0/1464162337 shutdown_connections 2024-02-19T18:57:26.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:26.614+0000 7fc4d19db700 1 -- 172.21.15.88:0/1464162337 wait complete. 2024-02-19T18:57:27.232 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:26 smithi088 ceph-mon[29272]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:27.232 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:26 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1464162337' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T18:57:27.232 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:26 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1464162337' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-19T18:57:27.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:26 smithi132 ceph-mon[37969]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:27.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:26 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1464162337' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T18:57:27.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:26 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1464162337' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-19T18:57:27.254 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T18:57:27.254 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-19T18:57:27.254 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-19T18:57:27.295 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-19T18:57:27.515 INFO:teuthology.orchestra.run.smithi132.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T18:57:29.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.175+0000 7f6e2431d700 1 -- 172.21.15.132:0/340819647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c1021f0 msgr2=0x7f6e1c0fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:29.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.175+0000 7f6e2431d700 1 --2- 172.21.15.132:0/340819647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c1021f0 0x7f6e1c0fe320 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c009910 tx=0x7f6e0c009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:29.196 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 -- 172.21.15.132:0/340819647 shutdown_connections 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 --2- 172.21.15.132:0/340819647 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c1021f0 0x7f6e1c0fe320 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 --2- 172.21.15.132:0/340819647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e1c1018a0 0x7f6e1c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 -- 172.21.15.132:0/340819647 >> 172.21.15.132:0/340819647 conn(0x7f6e1c0f9f10 msgr2=0x7f6e1c0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 -- 172.21.15.132:0/340819647 shutdown_connections 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 -- 172.21.15.132:0/340819647 wait complete. 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 Processor -- start 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 -- start start 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e1c1018a0 0x7f6e1c195b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c1021f0 0x7f6e1c1960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c1966d0 con 0x7f6e1c1018a0 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e2431d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1c196810 con 0x7f6e1c1021f0 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e218b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c1021f0 0x7f6e1c1960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.176+0000 7f6e218b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c1021f0 0x7f6e1c1960b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.132:48362/0 (socket says 172.21.15.132:48362) 2024-02-19T18:57:29.197 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e218b8700 1 -- 172.21.15.132:0/819017184 learned_addr learned my addr 172.21.15.132:0/819017184 (peer_addr_for_me v2:172.21.15.132:0/0) 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e220b9700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e1c1018a0 0x7f6e1c195b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e218b8700 1 -- 172.21.15.132:0/819017184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e1c1018a0 msgr2=0x7f6e1c195b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e218b8700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e1c1018a0 0x7f6e1c195b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e218b8700 1 -- 172.21.15.132:0/819017184 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c0095f0 con 0x7f6e1c1021f0 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e220b9700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e1c1018a0 0x7f6e1c195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e218b8700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c1021f0 0x7f6e1c1960b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00bda0 tx=0x7f6e0c00be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e137fe700 1 -- 172.21.15.132:0/819017184 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e0c01d070 con 0x7f6e1c1021f0 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e137fe700 1 -- 172.21.15.132:0/819017184 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6e0c004c50 con 0x7f6e1c1021f0 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e2431d700 1 -- 172.21.15.132:0/819017184 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e1c19b260 con 0x7f6e1c1021f0 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e137fe700 1 -- 172.21.15.132:0/819017184 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e0c00f690 con 0x7f6e1c1021f0 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.177+0000 7f6e2431d700 1 -- 172.21.15.132:0/819017184 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e1c19b750 con 0x7f6e1c1021f0 2024-02-19T18:57:29.198 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.178+0000 7f6e2431d700 1 -- 172.21.15.132:0/819017184 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e1c108d90 con 0x7f6e1c1021f0 2024-02-19T18:57:29.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.179+0000 7f6e137fe700 1 -- 172.21.15.132:0/819017184 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f6e0c0229b0 con 0x7f6e1c1021f0 2024-02-19T18:57:29.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.179+0000 7f6e137fe700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6e0806c680 0x7f6e0806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:29.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.179+0000 7f6e137fe700 1 -- 172.21.15.132:0/819017184 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f6e0c08cc70 con 0x7f6e1c1021f0 2024-02-19T18:57:29.199 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.179+0000 7f6e220b9700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6e0806c680 0x7f6e0806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:29.200 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.180+0000 7f6e220b9700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6e0806c680 0x7f6e0806eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e18007900 tx=0x7f6e18008040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:29.202 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.184+0000 7f6e137fe700 1 -- 172.21.15.132:0/819017184 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6e0c05b1d0 con 0x7f6e1c1021f0 2024-02-19T18:57:29.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:28 smithi132 ceph-mon[37969]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:29.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:28 smithi088 ceph-mon[29272]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:29.467 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.449+0000 7f6e2431d700 1 -- 172.21.15.132:0/819017184 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f6e1c19bd90 con 0x7f6e1c1021f0 2024-02-19T18:57:29.472 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.454+0000 7f6e137fe700 1 -- 172.21.15.132:0/819017184 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f6e0c027050 con 0x7f6e1c1021f0 2024-02-19T18:57:29.473 INFO:teuthology.orchestra.run.smithi132.stdout:[client.1] 2024-02-19T18:57:29.473 INFO:teuthology.orchestra.run.smithi132.stdout: key = AQCZpNNl7/jzGhAAVsza5wWgnq1D2BirX/GitA== 2024-02-19T18:57:29.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.456+0000 7f6e2431d700 1 -- 172.21.15.132:0/819017184 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6e0806c680 msgr2=0x7f6e0806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:29.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.456+0000 7f6e2431d700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6e0806c680 0x7f6e0806eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f6e18007900 tx=0x7f6e18008040 comp rx=0 tx=0).stop 2024-02-19T18:57:29.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.456+0000 7f6e2431d700 1 -- 172.21.15.132:0/819017184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c1021f0 msgr2=0x7f6e1c1960b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:29.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.456+0000 7f6e2431d700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c1021f0 0x7f6e1c1960b0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c00bda0 tx=0x7f6e0c00be80 comp rx=0 tx=0).stop 2024-02-19T18:57:29.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.456+0000 7f6e2431d700 1 -- 172.21.15.132:0/819017184 shutdown_connections 2024-02-19T18:57:29.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.457+0000 7f6e2431d700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6e0806c680 0x7f6e0806eb30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:29.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.457+0000 7f6e2431d700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e1c1018a0 0x7f6e1c195b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:29.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.457+0000 7f6e2431d700 1 --2- 172.21.15.132:0/819017184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e1c1021f0 0x7f6e1c1960b0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:29.475 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.457+0000 7f6e2431d700 1 -- 172.21.15.132:0/819017184 >> 172.21.15.132:0/819017184 conn(0x7f6e1c0f9f10 msgr2=0x7f6e1c104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:29.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.457+0000 7f6e2431d700 1 -- 172.21.15.132:0/819017184 shutdown_connections 2024-02-19T18:57:29.476 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:57:29.457+0000 7f6e2431d700 1 -- 172.21.15.132:0/819017184 wait complete. 2024-02-19T18:57:30.098 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:29 smithi132 ceph-mon[37969]: from='client.? 172.21.15.132:0/819017184' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T18:57:30.098 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:29 smithi132 ceph-mon[37969]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T18:57:30.098 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:29 smithi132 ceph-mon[37969]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-19T18:57:30.192 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T18:57:30.192 DEBUG:teuthology.orchestra.run.smithi132:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-19T18:57:30.193 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-19T18:57:30.235 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-19T18:57:30.235 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-19T18:57:30.236 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph mgr dump --format=json 2024-02-19T18:57:30.264 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:29 smithi088 ceph-mon[29272]: from='client.? 172.21.15.132:0/819017184' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T18:57:30.264 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:29 smithi088 ceph-mon[29272]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-19T18:57:30.264 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:29 smithi088 ceph-mon[29272]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-19T18:57:30.452 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:31.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:30 smithi132 ceph-mon[37969]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:31.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:30 smithi088 ceph-mon[29272]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:32.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.982+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2441227855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a240ffc60 msgr2=0x7f5a2410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:32.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.982+0000 7f5a29ea1700 1 --2- 172.21.15.88:0/2441227855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a240ffc60 0x7f5a2410a6d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009910 tx=0x7f5a18009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:32.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.983+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2441227855 shutdown_connections 2024-02-19T18:57:32.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.983+0000 7f5a29ea1700 1 --2- 172.21.15.88:0/2441227855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a240ffc60 0x7f5a2410a6d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:32.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.983+0000 7f5a29ea1700 1 --2- 172.21.15.88:0/2441227855 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a240ff350 0x7f5a240ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:32.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.983+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2441227855 >> 172.21.15.88:0/2441227855 conn(0x7f5a240748b0 msgr2=0x7f5a24074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:32.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.983+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2441227855 shutdown_connections 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.983+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2441227855 wait complete. 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.983+0000 7f5a29ea1700 1 Processor -- start 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.983+0000 7f5a29ea1700 1 -- start start 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a29ea1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a240ff350 0x7f5a24195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a29ea1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a240ffc60 0x7f5a24196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a29ea1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24196b40 con 0x7f5a240ffc60 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a29ea1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2419a8d0 con 0x7f5a240ff350 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a22ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a240ffc60 0x7f5a24196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a22ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a240ffc60 0x7f5a24196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60730/0 (socket says 172.21.15.88:60730) 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a22ffd700 1 -- 172.21.15.88:0/2159041829 learned_addr learned my addr 172.21.15.88:0/2159041829 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a237fe700 1 --2- 172.21.15.88:0/2159041829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a240ff350 0x7f5a24195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a22ffd700 1 -- 172.21.15.88:0/2159041829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a240ff350 msgr2=0x7f5a24195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a22ffd700 1 --2- 172.21.15.88:0/2159041829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a240ff350 0x7f5a24195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a22ffd700 1 -- 172.21.15.88:0/2159041829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a180095f0 con 0x7f5a240ffc60 2024-02-19T18:57:32.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a22ffd700 1 --2- 172.21.15.88:0/2159041829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a240ffc60 0x7f5a24196460 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5a18006010 tx=0x7f5a180056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:32.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a20ff9700 1 -- 172.21.15.88:0/2159041829 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a1801d070 con 0x7f5a240ffc60 2024-02-19T18:57:32.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2159041829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2419abb0 con 0x7f5a240ffc60 2024-02-19T18:57:32.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a20ff9700 1 -- 172.21.15.88:0/2159041829 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5a1800f460 con 0x7f5a240ffc60 2024-02-19T18:57:32.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a20ff9700 1 -- 172.21.15.88:0/2159041829 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a18021640 con 0x7f5a240ffc60 2024-02-19T18:57:32.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.984+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2159041829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2419b100 con 0x7f5a240ffc60 2024-02-19T18:57:32.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.985+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2159041829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a24107dd0 con 0x7f5a240ffc60 2024-02-19T18:57:32.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.986+0000 7f5a20ff9700 1 -- 172.21.15.88:0/2159041829 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f5a1800fad0 con 0x7f5a240ffc60 2024-02-19T18:57:32.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.986+0000 7f5a20ff9700 1 --2- 172.21.15.88:0/2159041829 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5a0c06c680 0x7f5a0c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:32.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.986+0000 7f5a237fe700 1 --2- 172.21.15.88:0/2159041829 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5a0c06c680 0x7f5a0c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:32.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.986+0000 7f5a20ff9700 1 -- 172.21.15.88:0/2159041829 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5a1808c740 con 0x7f5a240ffc60 2024-02-19T18:57:32.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.986+0000 7f5a237fe700 1 --2- 172.21.15.88:0/2159041829 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5a0c06c680 0x7f5a0c06eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5a14006fd0 tx=0x7f5a14008040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:32.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:31.991+0000 7f5a20ff9700 1 -- 172.21.15.88:0/2159041829 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5a180571e0 con 0x7f5a240ffc60 2024-02-19T18:57:32.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.233+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2159041829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5a2404f2a0 con 0x7f5a240ffc60 2024-02-19T18:57:32.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.236+0000 7f5a20ff9700 1 -- 172.21.15.88:0/2159041829 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+172855 (secure 0 0 0) 0x7f5a1805a800 con 0x7f5a240ffc60 2024-02-19T18:57:32.256 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:57:32.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.238+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2159041829 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5a0c06c680 msgr2=0x7f5a0c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:32.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.238+0000 7f5a29ea1700 1 --2- 172.21.15.88:0/2159041829 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5a0c06c680 0x7f5a0c06eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5a14006fd0 tx=0x7f5a14008040 comp rx=0 tx=0).stop 2024-02-19T18:57:32.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.238+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2159041829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a240ffc60 msgr2=0x7f5a24196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.238+0000 7f5a29ea1700 1 --2- 172.21.15.88:0/2159041829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a240ffc60 0x7f5a24196460 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5a18006010 tx=0x7f5a180056a0 comp rx=0 tx=0).stop 2024-02-19T18:57:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.238+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2159041829 shutdown_connections 2024-02-19T18:57:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.238+0000 7f5a29ea1700 1 --2- 172.21.15.88:0/2159041829 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5a0c06c680 0x7f5a0c06eb30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.238+0000 7f5a29ea1700 1 --2- 172.21.15.88:0/2159041829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a240ff350 0x7f5a24195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.238+0000 7f5a29ea1700 1 --2- 172.21.15.88:0/2159041829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a240ffc60 0x7f5a24196460 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.238+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2159041829 >> 172.21.15.88:0/2159041829 conn(0x7f5a240748b0 msgr2=0x7f5a24104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.239+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2159041829 shutdown_connections 2024-02-19T18:57:32.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:32.239+0000 7f5a29ea1700 1 -- 172.21.15.88:0/2159041829 wait complete. 2024-02-19T18:57:32.882 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"active_gid":14245,"active_name":"smithi088.gyviqq","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":2764007082},{"type":"v1","addr":"172.21.15.88:6801","nonce":2764007082}]},"active_addr":"172.21.15.88:6801/2764007082","active_change":"2024-02-19T18:53:39.596005+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14270,"name":"smithi132.psllsa","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.88:8443/","prometheus":"http://172.21.15.88:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":2389966380}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":2300938361}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":2418191043}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.88:0","nonce":3497052224}]}]} 2024-02-19T18:57:32.886 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-19T18:57:32.886 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-19T18:57:32.886 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd dump --format=json 2024-02-19T18:57:33.129 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:33.249 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:32 smithi088 ceph-mon[29272]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:33.250 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:32 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2159041829' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-19T18:57:33.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:32 smithi132 ceph-mon[37969]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:33.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:32 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2159041829' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-19T18:57:34.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.678+0000 7f6b78fec700 1 -- 172.21.15.88:0/3663100822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b74101d30 msgr2=0x7f6b7410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:34.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.678+0000 7f6b78fec700 1 --2- 172.21.15.88:0/3663100822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b74101d30 0x7f6b7410a230 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009910 tx=0x7f6b68009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:34.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.678+0000 7f6b78fec700 1 -- 172.21.15.88:0/3663100822 shutdown_connections 2024-02-19T18:57:34.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.678+0000 7f6b78fec700 1 --2- 172.21.15.88:0/3663100822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b74101d30 0x7f6b7410a230 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:34.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.678+0000 7f6b78fec700 1 --2- 172.21.15.88:0/3663100822 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b74101420 0x7f6b741017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:34.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.678+0000 7f6b78fec700 1 -- 172.21.15.88:0/3663100822 >> 172.21.15.88:0/3663100822 conn(0x7f6b740fac60 msgr2=0x7f6b740fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:34.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.678+0000 7f6b78fec700 1 -- 172.21.15.88:0/3663100822 shutdown_connections 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.678+0000 7f6b78fec700 1 -- 172.21.15.88:0/3663100822 wait complete. 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b78fec700 1 Processor -- start 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b78fec700 1 -- start start 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b78fec700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b74101420 0x7f6b74195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b78fec700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b74101d30 0x7f6b74196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b78fec700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b74196a70 con 0x7f6b74101d30 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b78fec700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7419a800 con 0x7f6b74101420 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b71d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b74101d30 0x7f6b74196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b71d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b74101d30 0x7f6b74196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60754/0 (socket says 172.21.15.88:60754) 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b71d9b700 1 -- 172.21.15.88:0/3330669407 learned_addr learned my addr 172.21.15.88:0/3330669407 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b7259c700 1 --2- 172.21.15.88:0/3330669407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b74101420 0x7f6b74195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:34.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b71d9b700 1 -- 172.21.15.88:0/3330669407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b74101420 msgr2=0x7f6b74195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:34.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b71d9b700 1 --2- 172.21.15.88:0/3330669407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b74101420 0x7f6b74195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:34.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b71d9b700 1 -- 172.21.15.88:0/3330669407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b680095f0 con 0x7f6b74101d30 2024-02-19T18:57:34.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.679+0000 7f6b71d9b700 1 --2- 172.21.15.88:0/3330669407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b74101d30 0x7f6b74196390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6b68006010 tx=0x7f6b6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:34.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.680+0000 7f6b637fe700 1 -- 172.21.15.88:0/3330669407 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b6801d070 con 0x7f6b74101d30 2024-02-19T18:57:34.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.680+0000 7f6b637fe700 1 -- 172.21.15.88:0/3330669407 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6b6800f460 con 0x7f6b74101d30 2024-02-19T18:57:34.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.680+0000 7f6b637fe700 1 -- 172.21.15.88:0/3330669407 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b68021640 con 0x7f6b74101d30 2024-02-19T18:57:34.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.680+0000 7f6b78fec700 1 -- 172.21.15.88:0/3330669407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7419aa80 con 0x7f6b74101d30 2024-02-19T18:57:34.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.680+0000 7f6b78fec700 1 -- 172.21.15.88:0/3330669407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7419af70 con 0x7f6b74101d30 2024-02-19T18:57:34.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.681+0000 7f6b78fec700 1 -- 172.21.15.88:0/3330669407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b741078a0 con 0x7f6b74101d30 2024-02-19T18:57:34.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.681+0000 7f6b637fe700 1 -- 172.21.15.88:0/3330669407 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f6b6800faf0 con 0x7f6b74101d30 2024-02-19T18:57:34.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.681+0000 7f6b637fe700 1 --2- 172.21.15.88:0/3330669407 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6b5c06c680 0x7f6b5c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:34.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.682+0000 7f6b637fe700 1 -- 172.21.15.88:0/3330669407 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f6b6808c740 con 0x7f6b74101d30 2024-02-19T18:57:34.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.682+0000 7f6b7259c700 1 --2- 172.21.15.88:0/3330669407 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6b5c06c680 0x7f6b5c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:34.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.682+0000 7f6b7259c700 1 --2- 172.21.15.88:0/3330669407 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6b5c06c680 0x7f6b5c06eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0060b0 tx=0x7f6b6c007b80 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:34.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.686+0000 7f6b637fe700 1 -- 172.21.15.88:0/3330669407 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6b680571e0 con 0x7f6b74101d30 2024-02-19T18:57:34.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.885+0000 7f6b78fec700 1 -- 172.21.15.88:0/3330669407 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6b7404f2a0 con 0x7f6b74101d30 2024-02-19T18:57:34.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.886+0000 7f6b637fe700 1 -- 172.21.15.88:0/3330669407 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11336 (secure 0 0 0) 0x7f6b6805a800 con 0x7f6b74101d30 2024-02-19T18:57:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:57:34.904 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","created":"2024-02-19T18:50:27.026977+0000","modified":"2024-02-19T18:57:16.003216+0000","last_up_change":"2024-02-19T18:57:14.993225+0000","last_in_change":"2024-02-19T18:56:55.518417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T18:55:57.936009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"787c5fec-c403-4975-92c3-8b68fcf9e399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6803","nonce":2929682937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6805","nonce":2929682937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6809","nonce":2929682937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6807","nonce":2929682937}]},"public_addr":"172.21.15.88:6803/2929682937","cluster_addr":"172.21.15.88:6805/2929682937","heartbeat_back_addr":"172.21.15.88:6809/2929682937","heartbeat_front_addr":"172.21.15.88:6807/2929682937","state":["exists","up"]},{"osd":1,"uuid":"a459843c-43d7-480f-a76b-d52d6e8ca061","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6811","nonce":3005125945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6813","nonce":3005125945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6817","nonce":3005125945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6815","nonce":3005125945}]},"public_addr":"172.21.15.88:6811/3005125945","cluster_addr":"172.21.15.88:6813/3005125945","heartbeat_back_addr":"172.21.15.88:6817/3005125945","heartbeat_front_addr":"172.21.15.88:6815/3005125945","state":["exists","up"]},{"osd":2,"uuid":"66334f86-7411-4587-b96b-22b61bf5201a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6819","nonce":525694249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6821","nonce":525694249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6825","nonce":525694249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6823","nonce":525694249}]},"public_addr":"172.21.15.88:6819/525694249","cluster_addr":"172.21.15.88:6821/525694249","heartbeat_back_addr":"172.21.15.88:6825/525694249","heartbeat_front_addr":"172.21.15.88:6823/525694249","state":["exists","up"]},{"osd":3,"uuid":"733318f2-be90-44d7-8c38-fb773f7da169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6801","nonce":2008345174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6803","nonce":2008345174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6807","nonce":2008345174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6805","nonce":2008345174}]},"public_addr":"172.21.15.132:6801/2008345174","cluster_addr":"172.21.15.132:6803/2008345174","heartbeat_back_addr":"172.21.15.132:6807/2008345174","heartbeat_front_addr":"172.21.15.132:6805/2008345174","state":["exists","up"]},{"osd":4,"uuid":"3b41c4ff-b4b1-4279-b973-0c3c1c62373e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6809","nonce":3138829446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6811","nonce":3138829446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6815","nonce":3138829446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6813","nonce":3138829446}]},"public_addr":"172.21.15.132:6809/3138829446","cluster_addr":"172.21.15.132:6811/3138829446","heartbeat_back_addr":"172.21.15.132:6815/3138829446","heartbeat_front_addr":"172.21.15.132:6813/3138829446","state":["exists","up"]},{"osd":5,"uuid":"367bd67a-e45d-40ff-9aac-1f88755641ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6817","nonce":2563696780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6819","nonce":2563696780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6823","nonce":2563696780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6821","nonce":2563696780}]},"public_addr":"172.21.15.132:6817/2563696780","cluster_addr":"172.21.15.132:6819/2563696780","heartbeat_back_addr":"172.21.15.132:6823/2563696780","heartbeat_front_addr":"172.21.15.132:6821/2563696780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:54:58.653733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:26.141001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:55.352595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:19.994273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:46.399893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:57:13.114456+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:0/4069571809":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/2177498492":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6800/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6801/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/2372050564":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/3565328320":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/3237698093":"2024-02-20T18:51:05.484896+0000","172.21.15.88:0/2338689461":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/3838380819":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6800/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6800/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/3491684321":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6801/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6801/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/387402116":"2024-02-20T18:51:05.484896+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T18:57:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.888+0000 7f6b78fec700 1 -- 172.21.15.88:0/3330669407 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6b5c06c680 msgr2=0x7f6b5c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.888+0000 7f6b78fec700 1 --2- 172.21.15.88:0/3330669407 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6b5c06c680 0x7f6b5c06eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c0060b0 tx=0x7f6b6c007b80 comp rx=0 tx=0).stop 2024-02-19T18:57:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.888+0000 7f6b78fec700 1 -- 172.21.15.88:0/3330669407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b74101d30 msgr2=0x7f6b74196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.888+0000 7f6b78fec700 1 --2- 172.21.15.88:0/3330669407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b74101d30 0x7f6b74196390 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6b68006010 tx=0x7f6b6800be20 comp rx=0 tx=0).stop 2024-02-19T18:57:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.889+0000 7f6b78fec700 1 -- 172.21.15.88:0/3330669407 shutdown_connections 2024-02-19T18:57:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.889+0000 7f6b78fec700 1 --2- 172.21.15.88:0/3330669407 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f6b5c06c680 0x7f6b5c06eb30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.889+0000 7f6b78fec700 1 --2- 172.21.15.88:0/3330669407 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6b74101420 0x7f6b74195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.889+0000 7f6b78fec700 1 --2- 172.21.15.88:0/3330669407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b74101d30 0x7f6b74196390 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.889+0000 7f6b78fec700 1 -- 172.21.15.88:0/3330669407 >> 172.21.15.88:0/3330669407 conn(0x7f6b740fac60 msgr2=0x7f6b741049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.889+0000 7f6b78fec700 1 -- 172.21.15.88:0/3330669407 shutdown_connections 2024-02-19T18:57:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:34.889+0000 7f6b78fec700 1 -- 172.21.15.88:0/3330669407 wait complete. 2024-02-19T18:57:35.207 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:34 smithi088 ceph-mon[29272]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:35.207 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:34 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/3330669407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T18:57:35.249 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:34 smithi132 ceph-mon[37969]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:35.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:34 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/3330669407' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T18:57:35.471 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-19T18:57:35.471 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd dump --format=json 2024-02-19T18:57:35.680 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:37.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:36 smithi088 ceph-mon[29272]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:37.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:36 smithi132 ceph-mon[37969]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:37.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.301+0000 7fe641c6b700 1 -- 172.21.15.88:0/341185114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe63c100ed0 msgr2=0x7fe63c104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:37.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.301+0000 7fe641c6b700 1 --2- 172.21.15.88:0/341185114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe63c100ed0 0x7fe63c104ea0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe624009910 tx=0x7fe624009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:37.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.302+0000 7fe641c6b700 1 -- 172.21.15.88:0/341185114 shutdown_connections 2024-02-19T18:57:37.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.302+0000 7fe641c6b700 1 --2- 172.21.15.88:0/341185114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe63c100ed0 0x7fe63c104ea0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:37.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.302+0000 7fe641c6b700 1 --2- 172.21.15.88:0/341185114 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe63c100530 0x7fe63c100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:37.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.302+0000 7fe641c6b700 1 -- 172.21.15.88:0/341185114 >> 172.21.15.88:0/341185114 conn(0x7fe63c0fbe10 msgr2=0x7fe63c0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:37.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.302+0000 7fe641c6b700 1 -- 172.21.15.88:0/341185114 shutdown_connections 2024-02-19T18:57:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.302+0000 7fe641c6b700 1 -- 172.21.15.88:0/341185114 wait complete. 2024-02-19T18:57:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.302+0000 7fe641c6b700 1 Processor -- start 2024-02-19T18:57:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.302+0000 7fe641c6b700 1 -- start start 2024-02-19T18:57:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe641c6b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe63c100530 0x7fe63c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe641c6b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe63c100ed0 0x7fe63c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe641c6b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c196bb0 con 0x7fe63c100530 2024-02-19T18:57:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe641c6b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe63c19a940 con 0x7fe63c100ed0 2024-02-19T18:57:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe63b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe63c100530 0x7fe63c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe63b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe63c100530 0x7fe63c195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53708/0 (socket says 172.21.15.88:53708) 2024-02-19T18:57:37.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe63b7fe700 1 -- 172.21.15.88:0/1501479952 learned_addr learned my addr 172.21.15.88:0/1501479952 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:37.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe632ffd700 1 --2- 172.21.15.88:0/1501479952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe63c100ed0 0x7fe63c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:37.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe63b7fe700 1 -- 172.21.15.88:0/1501479952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe63c100ed0 msgr2=0x7fe63c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:37.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe63b7fe700 1 --2- 172.21.15.88:0/1501479952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe63c100ed0 0x7fe63c1964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe63b7fe700 1 -- 172.21.15.88:0/1501479952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6240095f0 con 0x7fe63c100530 2024-02-19T18:57:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe63b7fe700 1 --2- 172.21.15.88:0/1501479952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe63c100530 0x7fe63c195f90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b4b0 tx=0x7fe62c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe6397fa700 1 -- 172.21.15.88:0/1501479952 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe62c011800 con 0x7fe63c100530 2024-02-19T18:57:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe641c6b700 1 -- 172.21.15.88:0/1501479952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe63c19ac20 con 0x7fe63c100530 2024-02-19T18:57:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe6397fa700 1 -- 172.21.15.88:0/1501479952 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe62c011e10 con 0x7fe63c100530 2024-02-19T18:57:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe6397fa700 1 -- 172.21.15.88:0/1501479952 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe62c00f2e0 con 0x7fe63c100530 2024-02-19T18:57:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.303+0000 7fe641c6b700 1 -- 172.21.15.88:0/1501479952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe63c19b170 con 0x7fe63c100530 2024-02-19T18:57:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.304+0000 7fe641c6b700 1 -- 172.21.15.88:0/1501479952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe63c1087d0 con 0x7fe63c100530 2024-02-19T18:57:37.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.305+0000 7fe6397fa700 1 -- 172.21.15.88:0/1501479952 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe62c010380 con 0x7fe63c100530 2024-02-19T18:57:37.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.305+0000 7fe6397fa700 1 --2- 172.21.15.88:0/1501479952 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe62806c680 0x7fe62806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:37.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.305+0000 7fe632ffd700 1 --2- 172.21.15.88:0/1501479952 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe62806c680 0x7fe62806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:37.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.305+0000 7fe6397fa700 1 -- 172.21.15.88:0/1501479952 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe62c08b1a0 con 0x7fe63c100530 2024-02-19T18:57:37.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.306+0000 7fe632ffd700 1 --2- 172.21.15.88:0/1501479952 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe62806c680 0x7fe62806eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe6240059d0 tx=0x7fe624005960 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:37.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.310+0000 7fe6397fa700 1 -- 172.21.15.88:0/1501479952 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe62c055bc0 con 0x7fe63c100530 2024-02-19T18:57:37.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.500+0000 7fe641c6b700 1 -- 172.21.15.88:0/1501479952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe63c19b3f0 con 0x7fe63c100530 2024-02-19T18:57:37.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.501+0000 7fe6397fa700 1 -- 172.21.15.88:0/1501479952 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11336 (secure 0 0 0) 0x7fe62c0591e0 con 0x7fe63c100530 2024-02-19T18:57:37.518 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:57:37.518 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","created":"2024-02-19T18:50:27.026977+0000","modified":"2024-02-19T18:57:16.003216+0000","last_up_change":"2024-02-19T18:57:14.993225+0000","last_in_change":"2024-02-19T18:56:55.518417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T18:55:57.936009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"787c5fec-c403-4975-92c3-8b68fcf9e399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6803","nonce":2929682937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6805","nonce":2929682937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6809","nonce":2929682937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6807","nonce":2929682937}]},"public_addr":"172.21.15.88:6803/2929682937","cluster_addr":"172.21.15.88:6805/2929682937","heartbeat_back_addr":"172.21.15.88:6809/2929682937","heartbeat_front_addr":"172.21.15.88:6807/2929682937","state":["exists","up"]},{"osd":1,"uuid":"a459843c-43d7-480f-a76b-d52d6e8ca061","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6811","nonce":3005125945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6813","nonce":3005125945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6817","nonce":3005125945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6815","nonce":3005125945}]},"public_addr":"172.21.15.88:6811/3005125945","cluster_addr":"172.21.15.88:6813/3005125945","heartbeat_back_addr":"172.21.15.88:6817/3005125945","heartbeat_front_addr":"172.21.15.88:6815/3005125945","state":["exists","up"]},{"osd":2,"uuid":"66334f86-7411-4587-b96b-22b61bf5201a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6819","nonce":525694249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6821","nonce":525694249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6825","nonce":525694249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6823","nonce":525694249}]},"public_addr":"172.21.15.88:6819/525694249","cluster_addr":"172.21.15.88:6821/525694249","heartbeat_back_addr":"172.21.15.88:6825/525694249","heartbeat_front_addr":"172.21.15.88:6823/525694249","state":["exists","up"]},{"osd":3,"uuid":"733318f2-be90-44d7-8c38-fb773f7da169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6801","nonce":2008345174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6803","nonce":2008345174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6807","nonce":2008345174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6805","nonce":2008345174}]},"public_addr":"172.21.15.132:6801/2008345174","cluster_addr":"172.21.15.132:6803/2008345174","heartbeat_back_addr":"172.21.15.132:6807/2008345174","heartbeat_front_addr":"172.21.15.132:6805/2008345174","state":["exists","up"]},{"osd":4,"uuid":"3b41c4ff-b4b1-4279-b973-0c3c1c62373e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6809","nonce":3138829446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6811","nonce":3138829446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6815","nonce":3138829446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6813","nonce":3138829446}]},"public_addr":"172.21.15.132:6809/3138829446","cluster_addr":"172.21.15.132:6811/3138829446","heartbeat_back_addr":"172.21.15.132:6815/3138829446","heartbeat_front_addr":"172.21.15.132:6813/3138829446","state":["exists","up"]},{"osd":5,"uuid":"367bd67a-e45d-40ff-9aac-1f88755641ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6817","nonce":2563696780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6819","nonce":2563696780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6823","nonce":2563696780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6821","nonce":2563696780}]},"public_addr":"172.21.15.132:6817/2563696780","cluster_addr":"172.21.15.132:6819/2563696780","heartbeat_back_addr":"172.21.15.132:6823/2563696780","heartbeat_front_addr":"172.21.15.132:6821/2563696780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:54:58.653733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:26.141001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:55.352595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:19.994273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:46.399893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:57:13.114456+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:0/4069571809":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/2177498492":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6800/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6801/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/2372050564":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/3565328320":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/3237698093":"2024-02-20T18:51:05.484896+0000","172.21.15.88:0/2338689461":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/3838380819":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6800/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6800/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/3491684321":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6801/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6801/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/387402116":"2024-02-20T18:51:05.484896+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T18:57:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.503+0000 7fe641c6b700 1 -- 172.21.15.88:0/1501479952 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe62806c680 msgr2=0x7fe62806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.503+0000 7fe641c6b700 1 --2- 172.21.15.88:0/1501479952 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe62806c680 0x7fe62806eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe6240059d0 tx=0x7fe624005960 comp rx=0 tx=0).stop 2024-02-19T18:57:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.503+0000 7fe641c6b700 1 -- 172.21.15.88:0/1501479952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe63c100530 msgr2=0x7fe63c195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.503+0000 7fe641c6b700 1 --2- 172.21.15.88:0/1501479952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe63c100530 0x7fe63c195f90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00b4b0 tx=0x7fe62c00b870 comp rx=0 tx=0).stop 2024-02-19T18:57:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.503+0000 7fe641c6b700 1 -- 172.21.15.88:0/1501479952 shutdown_connections 2024-02-19T18:57:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.503+0000 7fe641c6b700 1 --2- 172.21.15.88:0/1501479952 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe62806c680 0x7fe62806eb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.503+0000 7fe641c6b700 1 --2- 172.21.15.88:0/1501479952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe63c100530 0x7fe63c195f90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.503+0000 7fe641c6b700 1 --2- 172.21.15.88:0/1501479952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe63c100ed0 0x7fe63c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.503+0000 7fe641c6b700 1 -- 172.21.15.88:0/1501479952 >> 172.21.15.88:0/1501479952 conn(0x7fe63c0fbe10 msgr2=0x7fe63c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.503+0000 7fe641c6b700 1 -- 172.21.15.88:0/1501479952 shutdown_connections 2024-02-19T18:57:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:37.503+0000 7fe641c6b700 1 -- 172.21.15.88:0/1501479952 wait complete. 2024-02-19T18:57:38.158 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-19T18:57:38.159 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-19T18:57:38.159 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-19T18:57:38.160 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-19T18:57:38.160 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-19T18:57:38.160 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-19T18:57:38.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:37 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1501479952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T18:57:38.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:37 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1501479952' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T18:57:38.415 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:38.429 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:39.080 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:39.080 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:39.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:38 smithi088 ceph-mon[29272]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:39.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:38 smithi132 ceph-mon[37969]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:39.303 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:40.039 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:39 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:57:40.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:39 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:57:40.940 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:41.228 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:40 smithi088 ceph-mon[29272]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:41.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:40 smithi132 ceph-mon[37969]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:43.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:42 smithi132 ceph-mon[37969]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:43.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:42 smithi088 ceph-mon[29272]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:44.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.954+0000 7fda53b45700 1 -- 172.21.15.88:0/1224442804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0ff3c0 msgr2=0x7fda4c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:44.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.954+0000 7fda53b45700 1 --2- 172.21.15.88:0/1224442804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0ff3c0 0x7fda4c0ff790 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fda48009910 tx=0x7fda48009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:44.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.954+0000 7fda53b45700 1 -- 172.21.15.88:0/1224442804 shutdown_connections 2024-02-19T18:57:44.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.954+0000 7fda53b45700 1 --2- 172.21.15.88:0/1224442804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda4c0ffd60 0x7fda4c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:44.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.954+0000 7fda53b45700 1 --2- 172.21.15.88:0/1224442804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0ff3c0 0x7fda4c0ff790 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:44.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.954+0000 7fda53b45700 1 -- 172.21.15.88:0/1224442804 >> 172.21.15.88:0/1224442804 conn(0x7fda4c0fac80 msgr2=0x7fda4c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:44.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.954+0000 7fda53b45700 1 -- 172.21.15.88:0/1224442804 shutdown_connections 2024-02-19T18:57:44.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.954+0000 7fda53b45700 1 -- 172.21.15.88:0/1224442804 wait complete. 2024-02-19T18:57:44.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda53b45700 1 Processor -- start 2024-02-19T18:57:44.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda53b45700 1 -- start start 2024-02-19T18:57:44.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda53b45700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda4c0ff3c0 0x7fda4c100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:44.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda53b45700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0ffd60 0x7fda4c101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:44.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda53b45700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c101850 con 0x7fda4c0ffd60 2024-02-19T18:57:44.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda53b45700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda4c101990 con 0x7fda4c0ff3c0 2024-02-19T18:57:44.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda510e0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0ffd60 0x7fda4c101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda510e0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0ffd60 0x7fda4c101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53728/0 (socket says 172.21.15.88:53728) 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda510e0700 1 -- 172.21.15.88:0/702054337 learned_addr learned my addr 172.21.15.88:0/702054337 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda518e1700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda4c0ff3c0 0x7fda4c100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda510e0700 1 -- 172.21.15.88:0/702054337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda4c0ff3c0 msgr2=0x7fda4c100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda510e0700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda4c0ff3c0 0x7fda4c100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda510e0700 1 -- 172.21.15.88:0/702054337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda480095f0 con 0x7fda4c0ffd60 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda510e0700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0ffd60 0x7fda4c101310 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00e8f0 tx=0x7fda3c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda42ffd700 1 -- 172.21.15.88:0/702054337 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda3c009790 con 0x7fda4c0ffd60 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda42ffd700 1 -- 172.21.15.88:0/702054337 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fda3c00c810 con 0x7fda4c0ffd60 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda42ffd700 1 -- 172.21.15.88:0/702054337 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda3c0055b0 con 0x7fda4c0ffd60 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.955+0000 7fda53b45700 1 -- 172.21.15.88:0/702054337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda4c1a20d0 con 0x7fda4c0ffd60 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.956+0000 7fda53b45700 1 -- 172.21.15.88:0/702054337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4c1a24e0 con 0x7fda4c0ffd60 2024-02-19T18:57:44.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.956+0000 7fda53b45700 1 -- 172.21.15.88:0/702054337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda4c10c520 con 0x7fda4c0ffd60 2024-02-19T18:57:44.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.957+0000 7fda42ffd700 1 -- 172.21.15.88:0/702054337 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fda3c00c980 con 0x7fda4c0ffd60 2024-02-19T18:57:44.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.957+0000 7fda42ffd700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fda3806c680 0x7fda3806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:44.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.957+0000 7fda42ffd700 1 -- 172.21.15.88:0/702054337 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fda3c014070 con 0x7fda4c0ffd60 2024-02-19T18:57:44.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.957+0000 7fda518e1700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fda3806c680 0x7fda3806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:44.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.957+0000 7fda42ffd700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249] conn(0x7fda38072350 0x7fda38074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:44.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.957+0000 7fda42ffd700 1 -- 172.21.15.88:0/702054337 --> [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda38074e10 con 0x7fda38072350 2024-02-19T18:57:44.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.957+0000 7fda42ffd700 1 -- 172.21.15.88:0/702054337 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda3c08bf10 con 0x7fda4c0ffd60 2024-02-19T18:57:44.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.958+0000 7fda520e2700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249] conn(0x7fda38072350 0x7fda38074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:44.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.958+0000 7fda518e1700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fda3806c680 0x7fda3806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda480059d0 tx=0x7fda48005940 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:44.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.958+0000 7fda520e2700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249] conn(0x7fda38072350 0x7fda38074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:44.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.959+0000 7fda42ffd700 1 -- 172.21.15.88:0/702054337 <== osd.2 v2:172.21.15.88:6818/525694249 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fda38074e10 con 0x7fda38072350 2024-02-19T18:57:44.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.975+0000 7fda53b45700 1 -- 172.21.15.88:0/702054337 --> [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda4c04f2a0 con 0x7fda38072350 2024-02-19T18:57:44.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda42ffd700 1 -- 172.21.15.88:0/702054337 <== osd.2 v2:172.21.15.88:6818/525694249 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fda4c04f2a0 con 0x7fda38072350 2024-02-19T18:57:44.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 -- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249] conn(0x7fda38072350 msgr2=0x7fda38074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:44.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249] conn(0x7fda38072350 0x7fda38074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:44.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 -- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fda3806c680 msgr2=0x7fda3806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fda3806c680 0x7fda3806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fda480059d0 tx=0x7fda48005940 comp rx=0 tx=0).stop 2024-02-19T18:57:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 -- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0ffd60 msgr2=0x7fda4c101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0ffd60 0x7fda4c101310 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fda3c00e8f0 tx=0x7fda3c00ec00 comp rx=0 tx=0).stop 2024-02-19T18:57:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 -- 172.21.15.88:0/702054337 shutdown_connections 2024-02-19T18:57:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6818/525694249,v1:172.21.15.88:6819/525694249] conn(0x7fda38072350 0x7fda38074760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fda3806c680 0x7fda3806eb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda4c0ff3c0 0x7fda4c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 --2- 172.21.15.88:0/702054337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda4c0ffd60 0x7fda4c101310 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 -- 172.21.15.88:0/702054337 >> 172.21.15.88:0/702054337 conn(0x7fda4c0fac80 msgr2=0x7fda4c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 -- 172.21.15.88:0/702054337 shutdown_connections 2024-02-19T18:57:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:44.976+0000 7fda53b45700 1 -- 172.21.15.88:0/702054337 wait complete. 2024-02-19T18:57:45.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:44 smithi088 ceph-mon[29272]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:45.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:44 smithi132 ceph-mon[37969]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:45.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.440+0000 7fbf94896700 1 -- 172.21.15.88:0/1363360471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900ff3c0 msgr2=0x7fbf900ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:45.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.440+0000 7fbf94896700 1 --2- 172.21.15.88:0/1363360471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900ff3c0 0x7fbf900ff790 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009910 tx=0x7fbf84009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:45.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.440+0000 7fbf94896700 1 -- 172.21.15.88:0/1363360471 shutdown_connections 2024-02-19T18:57:45.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.440+0000 7fbf94896700 1 --2- 172.21.15.88:0/1363360471 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf900ffd60 0x7fbf90103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:45.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.440+0000 7fbf94896700 1 --2- 172.21.15.88:0/1363360471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900ff3c0 0x7fbf900ff790 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:45.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.440+0000 7fbf94896700 1 -- 172.21.15.88:0/1363360471 >> 172.21.15.88:0/1363360471 conn(0x7fbf900fac80 msgr2=0x7fbf900fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:45.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.440+0000 7fbf94896700 1 -- 172.21.15.88:0/1363360471 shutdown_connections 2024-02-19T18:57:45.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.440+0000 7fbf94896700 1 -- 172.21.15.88:0/1363360471 wait complete. 2024-02-19T18:57:45.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf94896700 1 Processor -- start 2024-02-19T18:57:45.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf94896700 1 -- start start 2024-02-19T18:57:45.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf94896700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf900ff3c0 0x7fbf90196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:45.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf94896700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900ffd60 0x7fbf901971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:45.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf94896700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf901978b0 con 0x7fbf900ffd60 2024-02-19T18:57:45.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf94896700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf901979f0 con 0x7fbf900ff3c0 2024-02-19T18:57:45.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf8dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900ffd60 0x7fbf901971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:45.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf8dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900ffd60 0x7fbf901971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60776/0 (socket says 172.21.15.88:60776) 2024-02-19T18:57:45.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf8dd9b700 1 -- 172.21.15.88:0/1436000844 learned_addr learned my addr 172.21.15.88:0/1436000844 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:45.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf8e59c700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf900ff3c0 0x7fbf90196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:45.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf8dd9b700 1 -- 172.21.15.88:0/1436000844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf900ff3c0 msgr2=0x7fbf90196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:45.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf8dd9b700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf900ff3c0 0x7fbf90196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:45.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf8dd9b700 1 -- 172.21.15.88:0/1436000844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf840095f0 con 0x7fbf900ffd60 2024-02-19T18:57:45.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.441+0000 7fbf8dd9b700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900ffd60 0x7fbf901971d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000d900 tx=0x7fbf8000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:45.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.442+0000 7fbf7f7fe700 1 -- 172.21.15.88:0/1436000844 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf80009900 con 0x7fbf900ffd60 2024-02-19T18:57:45.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.442+0000 7fbf7f7fe700 1 -- 172.21.15.88:0/1436000844 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbf80010460 con 0x7fbf900ffd60 2024-02-19T18:57:45.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.442+0000 7fbf94896700 1 -- 172.21.15.88:0/1436000844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf9019fd00 con 0x7fbf900ffd60 2024-02-19T18:57:45.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.442+0000 7fbf7f7fe700 1 -- 172.21.15.88:0/1436000844 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf8000f5f0 con 0x7fbf900ffd60 2024-02-19T18:57:45.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.442+0000 7fbf94896700 1 -- 172.21.15.88:0/1436000844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf901a0220 con 0x7fbf900ffd60 2024-02-19T18:57:45.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.442+0000 7fbf94896700 1 -- 172.21.15.88:0/1436000844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbf9010c520 con 0x7fbf900ffd60 2024-02-19T18:57:45.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.443+0000 7fbf7f7fe700 1 -- 172.21.15.88:0/1436000844 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fbf800105d0 con 0x7fbf900ffd60 2024-02-19T18:57:45.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.443+0000 7fbf7f7fe700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fbf7806c680 0x7fbf7806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:45.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.444+0000 7fbf8e59c700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fbf7806c680 0x7fbf7806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:45.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.444+0000 7fbf7f7fe700 1 -- 172.21.15.88:0/1436000844 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbf8008b080 con 0x7fbf900ffd60 2024-02-19T18:57:45.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.444+0000 7fbf7f7fe700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937] conn(0x7fbf78072350 0x7fbf78074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:45.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.444+0000 7fbf7f7fe700 1 -- 172.21.15.88:0/1436000844 --> [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbf78074e10 con 0x7fbf78072350 2024-02-19T18:57:45.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.444+0000 7fbf7f7fe700 1 -- 172.21.15.88:0/1436000844 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf8008d750 con 0x7fbf900ffd60 2024-02-19T18:57:45.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.444+0000 7fbf8ed9d700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937] conn(0x7fbf78072350 0x7fbf78074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:45.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.444+0000 7fbf8e59c700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fbf7806c680 0x7fbf7806eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbf84006010 tx=0x7fbf8401a040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:45.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.444+0000 7fbf8ed9d700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937] conn(0x7fbf78072350 0x7fbf78074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:45.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.445+0000 7fbf7f7fe700 1 -- 172.21.15.88:0/1436000844 <== osd.0 v2:172.21.15.88:6802/2929682937 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fbf78074e10 con 0x7fbf78072350 2024-02-19T18:57:45.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.460+0000 7fbf94896700 1 -- 172.21.15.88:0/1436000844 --> [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbf9004f2a0 con 0x7fbf78072350 2024-02-19T18:57:45.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf7f7fe700 1 -- 172.21.15.88:0/1436000844 <== osd.0 v2:172.21.15.88:6802/2929682937 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbf9004f2a0 con 0x7fbf78072350 2024-02-19T18:57:45.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 -- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937] conn(0x7fbf78072350 msgr2=0x7fbf78074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:45.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937] conn(0x7fbf78072350 0x7fbf78074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:45.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 -- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fbf7806c680 msgr2=0x7fbf7806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:45.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fbf7806c680 0x7fbf7806eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbf84006010 tx=0x7fbf8401a040 comp rx=0 tx=0).stop 2024-02-19T18:57:45.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 -- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900ffd60 msgr2=0x7fbf901971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:45.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900ffd60 0x7fbf901971d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000d900 tx=0x7fbf8000dc10 comp rx=0 tx=0).stop 2024-02-19T18:57:45.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 -- 172.21.15.88:0/1436000844 shutdown_connections 2024-02-19T18:57:45.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6802/2929682937,v1:172.21.15.88:6803/2929682937] conn(0x7fbf78072350 0x7fbf78074760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:45.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fbf7806c680 0x7fbf7806eb30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:45.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf900ff3c0 0x7fbf90196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:45.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 --2- 172.21.15.88:0/1436000844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf900ffd60 0x7fbf901971d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:45.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 -- 172.21.15.88:0/1436000844 >> 172.21.15.88:0/1436000844 conn(0x7fbf900fac80 msgr2=0x7fbf900fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:45.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 -- 172.21.15.88:0/1436000844 shutdown_connections 2024-02-19T18:57:45.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.461+0000 7fbf94896700 1 -- 172.21.15.88:0/1436000844 wait complete. 2024-02-19T18:57:46.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.675+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3782714916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e41004e0 msgr2=0x7f29e41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:46.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.675+0000 7f29e9cf3700 1 --2- 172.21.15.88:0/3782714916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e41004e0 0x7f29e41008b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009910 tx=0x7f29d4009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:46.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.675+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3782714916 shutdown_connections 2024-02-19T18:57:46.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.675+0000 7f29e9cf3700 1 --2- 172.21.15.88:0/3782714916 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e4100e80 0x7f29e4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.675+0000 7f29e9cf3700 1 --2- 172.21.15.88:0/3782714916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e41004e0 0x7f29e41008b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.675+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3782714916 >> 172.21.15.88:0/3782714916 conn(0x7f29e40fbda0 msgr2=0x7f29e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.675+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3782714916 shutdown_connections 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.675+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3782714916 wait complete. 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.675+0000 7f29e9cf3700 1 Processor -- start 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.675+0000 7f29e9cf3700 1 -- start start 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e9cf3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e41004e0 0x7f29e4195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e9cf3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e4100e80 0x7f29e4196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e9cf3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e4196b10 con 0x7f29e41004e0 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e9cf3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e419a8a0 con 0x7f29e4100e80 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e41004e0 0x7f29e4195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e41004e0 0x7f29e4195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60798/0 (socket says 172.21.15.88:60798) 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e37fe700 1 -- 172.21.15.88:0/3359694161 learned_addr learned my addr 172.21.15.88:0/3359694161 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:46.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e37fe700 1 -- 172.21.15.88:0/3359694161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e4100e80 msgr2=0x7f29e4196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:46.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e37fe700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e4100e80 0x7f29e4196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e37fe700 1 -- 172.21.15.88:0/3359694161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d40095f0 con 0x7f29e41004e0 2024-02-19T18:57:46.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e37fe700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e41004e0 0x7f29e4195ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f29d4006010 tx=0x7f29d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:46.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e0ff9700 1 -- 172.21.15.88:0/3359694161 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29d401d070 con 0x7f29e41004e0 2024-02-19T18:57:46.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e0ff9700 1 -- 172.21.15.88:0/3359694161 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f29d4004f50 con 0x7f29e41004e0 2024-02-19T18:57:46.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3359694161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e419ab20 con 0x7f29e41004e0 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e0ff9700 1 -- 172.21.15.88:0/3359694161 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29d4021640 con 0x7f29e41004e0 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.676+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3359694161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e419b010 con 0x7f29e41004e0 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.677+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3359694161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f29e410d670 con 0x7f29e41004e0 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.678+0000 7f29e0ff9700 1 -- 172.21.15.88:0/3359694161 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f29d4004aa0 con 0x7f29e41004e0 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.678+0000 7f29e0ff9700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f29cc06c680 0x7f29cc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.678+0000 7f29e2ffd700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f29cc06c680 0x7f29cc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.678+0000 7f29e0ff9700 1 -- 172.21.15.88:0/3359694161 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f29d408c790 con 0x7f29e41004e0 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.678+0000 7f29e0ff9700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446] conn(0x7f29cc072350 0x7f29cc074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.678+0000 7f29e0ff9700 1 -- 172.21.15.88:0/3359694161 --> [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f29cc074e10 con 0x7f29cc072350 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.678+0000 7f29e0ff9700 1 -- 172.21.15.88:0/3359694161 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29d40912a0 con 0x7f29e41004e0 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.678+0000 7f29e2ffd700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f29cc06c680 0x7f29cc06eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f29d8005950 tx=0x7f29d80058e0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.678+0000 7f29e3fff700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446] conn(0x7f29cc072350 0x7f29cc074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.679+0000 7f29e3fff700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446] conn(0x7f29cc072350 0x7f29cc074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.680+0000 7f29e0ff9700 1 -- 172.21.15.88:0/3359694161 <== osd.4 v2:172.21.15.132:6808/3138829446 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f29cc074e10 con 0x7f29cc072350 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.695+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3359694161 --> [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f29e404f2a0 con 0x7f29cc072350 2024-02-19T18:57:46.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.695+0000 7f29e0ff9700 1 -- 172.21.15.88:0/3359694161 <== osd.4 v2:172.21.15.132:6808/3138829446 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f29e404f2a0 con 0x7f29cc072350 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3359694161 >> [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446] conn(0x7f29cc072350 msgr2=0x7f29cc074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446] conn(0x7f29cc072350 0x7f29cc074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3359694161 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f29cc06c680 msgr2=0x7f29cc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f29cc06c680 0x7f29cc06eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f29d8005950 tx=0x7f29d80058e0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3359694161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e41004e0 msgr2=0x7f29e4195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e41004e0 0x7f29e4195ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f29d4006010 tx=0x7f29d400be20 comp rx=0 tx=0).stop 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3359694161 shutdown_connections 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.132:6808/3138829446,v1:172.21.15.132:6809/3138829446] conn(0x7f29cc072350 0x7f29cc074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f29cc06c680 0x7f29cc06eb30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e41004e0 0x7f29e4195ef0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 --2- 172.21.15.88:0/3359694161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29e4100e80 0x7f29e4196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3359694161 >> 172.21.15.88:0/3359694161 conn(0x7f29e40fbda0 msgr2=0x7f29e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3359694161 shutdown_connections 2024-02-19T18:57:46.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:45.696+0000 7f29e9cf3700 1 -- 172.21.15.88:0/3359694161 wait complete. 2024-02-19T18:57:46.757 INFO:teuthology.orchestra.run.smithi088.stdout:146028888079 2024-02-19T18:57:46.758 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-19T18:57:46.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 -- 172.21.15.88:0/2500840248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f963c100d70 msgr2=0x7f963c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 --2- 172.21.15.88:0/2500840248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f963c100d70 0x7f963c104e50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f962c009910 tx=0x7f962c009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 -- 172.21.15.88:0/2500840248 shutdown_connections 2024-02-19T18:57:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 --2- 172.21.15.88:0/2500840248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f963c100d70 0x7f963c104e50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 --2- 172.21.15.88:0/2500840248 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c1003d0 0x7f963c1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 -- 172.21.15.88:0/2500840248 >> 172.21.15.88:0/2500840248 conn(0x7f963c0fbcb0 msgr2=0x7f963c0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 -- 172.21.15.88:0/2500840248 shutdown_connections 2024-02-19T18:57:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 -- 172.21.15.88:0/2500840248 wait complete. 2024-02-19T18:57:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 Processor -- start 2024-02-19T18:57:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 -- start start 2024-02-19T18:57:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f963c1003d0 0x7f963c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c100d70 0x7f963c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:46.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.959+0000 7f96433e8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c073ad0 con 0x7f963c1003d0 2024-02-19T18:57:46.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.960+0000 7f96433e8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963c073c10 con 0x7f963c100d70 2024-02-19T18:57:46.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.960+0000 7f9641184700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f963c1003d0 0x7f963c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:46.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.960+0000 7f9641184700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f963c1003d0 0x7f963c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60810/0 (socket says 172.21.15.88:60810) 2024-02-19T18:57:46.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.960+0000 7f9641184700 1 -- 172.21.15.88:0/30638092 learned_addr learned my addr 172.21.15.88:0/30638092 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:46.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.960+0000 7f9640983700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c100d70 0x7f963c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:46.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.960+0000 7f9641184700 1 -- 172.21.15.88:0/30638092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c100d70 msgr2=0x7f963c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:46.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.960+0000 7f9641184700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c100d70 0x7f963c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.960+0000 7f9641184700 1 -- 172.21.15.88:0/30638092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c0095f0 con 0x7f963c1003d0 2024-02-19T18:57:46.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.960+0000 7f9640983700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c100d70 0x7f963c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T18:57:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.960+0000 7f9641184700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f963c1003d0 0x7f963c073050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f963800b4b0 tx=0x7f963800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.961+0000 7f96327fc700 1 -- 172.21.15.88:0/30638092 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9638011800 con 0x7f963c1003d0 2024-02-19T18:57:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.961+0000 7f96433e8700 1 -- 172.21.15.88:0/30638092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963c1a2120 con 0x7f963c1003d0 2024-02-19T18:57:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.961+0000 7f96327fc700 1 -- 172.21.15.88:0/30638092 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9638011e10 con 0x7f963c1003d0 2024-02-19T18:57:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.961+0000 7f96327fc700 1 -- 172.21.15.88:0/30638092 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f963800f2e0 con 0x7f963c1003d0 2024-02-19T18:57:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.961+0000 7f96433e8700 1 -- 172.21.15.88:0/30638092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f963c1a2530 con 0x7f963c1003d0 2024-02-19T18:57:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.961+0000 7f96433e8700 1 -- 172.21.15.88:0/30638092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f963c10d6d0 con 0x7f963c1003d0 2024-02-19T18:57:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.962+0000 7f96327fc700 1 -- 172.21.15.88:0/30638092 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f9638010380 con 0x7f963c1003d0 2024-02-19T18:57:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.963+0000 7f96327fc700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f962806c680 0x7f962806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.963+0000 7f9640983700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f962806c680 0x7f962806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.963+0000 7f96327fc700 1 -- 172.21.15.88:0/30638092 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f963808b7a0 con 0x7f963c1003d0 2024-02-19T18:57:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.963+0000 7f96327fc700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945] conn(0x7f9628072350 0x7f9628074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.963+0000 7f96327fc700 1 -- 172.21.15.88:0/30638092 --> [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9628074e10 con 0x7f9628072350 2024-02-19T18:57:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.963+0000 7f96327fc700 1 -- 172.21.15.88:0/30638092 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f963808bc00 con 0x7f963c1003d0 2024-02-19T18:57:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.963+0000 7f9641985700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945] conn(0x7f9628072350 0x7f9628074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.963+0000 7f9640983700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f962806c680 0x7f962806eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f963c074840 tx=0x7f962c01a040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.963+0000 7f9641985700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945] conn(0x7f9628072350 0x7f9628074760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.964+0000 7f96327fc700 1 -- 172.21.15.88:0/30638092 <== osd.1 v2:172.21.15.88:6810/3005125945 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9628074e10 con 0x7f9628072350 2024-02-19T18:57:46.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.980+0000 7f96433e8700 1 -- 172.21.15.88:0/30638092 --> [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f963c04f2a0 con 0x7f9628072350 2024-02-19T18:57:46.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96327fc700 1 -- 172.21.15.88:0/30638092 <== osd.1 v2:172.21.15.88:6810/3005125945 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f963c04f2a0 con 0x7f9628072350 2024-02-19T18:57:46.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 -- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945] conn(0x7f9628072350 msgr2=0x7f9628074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:46.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945] conn(0x7f9628072350 0x7f9628074760 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:46.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 -- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f962806c680 msgr2=0x7f962806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:46.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f962806c680 0x7f962806eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f963c074840 tx=0x7f962c01a040 comp rx=0 tx=0).stop 2024-02-19T18:57:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 -- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f963c1003d0 msgr2=0x7f963c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f963c1003d0 0x7f963c073050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f963800b4b0 tx=0x7f963800b870 comp rx=0 tx=0).stop 2024-02-19T18:57:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 -- 172.21.15.88:0/30638092 shutdown_connections 2024-02-19T18:57:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6810/3005125945,v1:172.21.15.88:6811/3005125945] conn(0x7f9628072350 0x7f9628074760 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f962806c680 0x7f962806eb30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f963c1003d0 0x7f963c073050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 --2- 172.21.15.88:0/30638092 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f963c100d70 0x7f963c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.981+0000 7f96433e8700 1 -- 172.21.15.88:0/30638092 >> 172.21.15.88:0/30638092 conn(0x7f963c0fbcb0 msgr2=0x7f963c0fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.982+0000 7f96433e8700 1 -- 172.21.15.88:0/30638092 shutdown_connections 2024-02-19T18:57:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:46.982+0000 7f96433e8700 1 -- 172.21.15.88:0/30638092 wait complete. 2024-02-19T18:57:47.261 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:46 smithi088 ceph-mon[29272]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:47.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:46 smithi132 ceph-mon[37969]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:47.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.486+0000 7f4d53119700 1 -- 172.21.15.88:0/604135397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d4c0ffd60 msgr2=0x7f4d4c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:47.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.486+0000 7f4d53119700 1 --2- 172.21.15.88:0/604135397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d4c0ffd60 0x7f4d4c103d30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4d38009910 tx=0x7f4d38009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:47.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.486+0000 7f4d53119700 1 -- 172.21.15.88:0/604135397 shutdown_connections 2024-02-19T18:57:47.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.486+0000 7f4d53119700 1 --2- 172.21.15.88:0/604135397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d4c0ffd60 0x7f4d4c103d30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.486+0000 7f4d53119700 1 --2- 172.21.15.88:0/604135397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d4c0ff3c0 0x7f4d4c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.486+0000 7f4d53119700 1 -- 172.21.15.88:0/604135397 >> 172.21.15.88:0/604135397 conn(0x7f4d4c0fac60 msgr2=0x7f4d4c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:47.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.486+0000 7f4d53119700 1 -- 172.21.15.88:0/604135397 shutdown_connections 2024-02-19T18:57:47.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.486+0000 7f4d53119700 1 -- 172.21.15.88:0/604135397 wait complete. 2024-02-19T18:57:47.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.486+0000 7f4d53119700 1 Processor -- start 2024-02-19T18:57:47.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.486+0000 7f4d53119700 1 -- start start 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d53119700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d4c0ff3c0 0x7f4d4c196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d53119700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d4c0ffd60 0x7f4d4c197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d53119700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c197880 con 0x7f4d4c0ff3c0 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d53119700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4c1979c0 con 0x7f4d4c0ffd60 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d50eb5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d4c0ff3c0 0x7f4d4c196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d50eb5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d4c0ff3c0 0x7f4d4c196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60818/0 (socket says 172.21.15.88:60818) 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d50eb5700 1 -- 172.21.15.88:0/3219424617 learned_addr learned my addr 172.21.15.88:0/3219424617 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d4bfff700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d4c0ffd60 0x7f4d4c197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d50eb5700 1 -- 172.21.15.88:0/3219424617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d4c0ffd60 msgr2=0x7f4d4c197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d50eb5700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d4c0ffd60 0x7f4d4c197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d50eb5700 1 -- 172.21.15.88:0/3219424617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d380095f0 con 0x7f4d4c0ff3c0 2024-02-19T18:57:47.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d50eb5700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d4c0ff3c0 0x7f4d4c196d20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000d900 tx=0x7f4d4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:47.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d49ffb700 1 -- 172.21.15.88:0/3219424617 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d400041f0 con 0x7f4d4c0ff3c0 2024-02-19T18:57:47.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d49ffb700 1 -- 172.21.15.88:0/3219424617 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4d40004ce0 con 0x7f4d4c0ff3c0 2024-02-19T18:57:47.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d53119700 1 -- 172.21.15.88:0/3219424617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4c19fd90 con 0x7f4d4c0ff3c0 2024-02-19T18:57:47.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.487+0000 7f4d49ffb700 1 -- 172.21.15.88:0/3219424617 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d4000b750 con 0x7f4d4c0ff3c0 2024-02-19T18:57:47.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.488+0000 7f4d53119700 1 -- 172.21.15.88:0/3219424617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4c1a0250 con 0x7f4d4c0ff3c0 2024-02-19T18:57:47.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.488+0000 7f4d53119700 1 -- 172.21.15.88:0/3219424617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4d4c10c4f0 con 0x7f4d4c0ff3c0 2024-02-19T18:57:47.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.489+0000 7f4d49ffb700 1 -- 172.21.15.88:0/3219424617 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f4d40004350 con 0x7f4d4c0ff3c0 2024-02-19T18:57:47.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.489+0000 7f4d49ffb700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f4d3c06c680 0x7f4d3c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:47.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.489+0000 7f4d4bfff700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f4d3c06c680 0x7f4d3c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:47.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.489+0000 7f4d49ffb700 1 -- 172.21.15.88:0/3219424617 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f4d4008b690 con 0x7f4d4c0ff3c0 2024-02-19T18:57:47.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.489+0000 7f4d49ffb700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174] conn(0x7f4d3c072350 0x7f4d3c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:47.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.489+0000 7f4d49ffb700 1 -- 172.21.15.88:0/3219424617 --> [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4d3c074e10 con 0x7f4d3c072350 2024-02-19T18:57:47.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.489+0000 7f4d49ffb700 1 -- 172.21.15.88:0/3219424617 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d4008dd60 con 0x7f4d4c0ff3c0 2024-02-19T18:57:47.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.490+0000 7f4d4bfff700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f4d3c06c680 0x7f4d3c06eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4d38006010 tx=0x7f4d3801a040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:47.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.490+0000 7f4d516b6700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174] conn(0x7f4d3c072350 0x7f4d3c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:47.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.490+0000 7f4d516b6700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174] conn(0x7f4d3c072350 0x7f4d3c074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:47.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.491+0000 7f4d49ffb700 1 -- 172.21.15.88:0/3219424617 <== osd.3 v2:172.21.15.132:6800/2008345174 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4d3c074e10 con 0x7f4d3c072350 2024-02-19T18:57:47.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.507+0000 7f4d53119700 1 -- 172.21.15.88:0/3219424617 --> [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4d4c067980 con 0x7f4d3c072350 2024-02-19T18:57:47.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.507+0000 7f4d49ffb700 1 -- 172.21.15.88:0/3219424617 <== osd.3 v2:172.21.15.132:6800/2008345174 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f4d4c067980 con 0x7f4d3c072350 2024-02-19T18:57:47.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.507+0000 7f4d53119700 1 -- 172.21.15.88:0/3219424617 >> [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174] conn(0x7f4d3c072350 msgr2=0x7f4d3c074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.507+0000 7f4d53119700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174] conn(0x7f4d3c072350 0x7f4d3c074760 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 -- 172.21.15.88:0/3219424617 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f4d3c06c680 msgr2=0x7f4d3c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f4d3c06c680 0x7f4d3c06eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4d38006010 tx=0x7f4d3801a040 comp rx=0 tx=0).stop 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 -- 172.21.15.88:0/3219424617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d4c0ff3c0 msgr2=0x7f4d4c196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d4c0ff3c0 0x7f4d4c196d20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4d4000d900 tx=0x7f4d4000dcc0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 -- 172.21.15.88:0/3219424617 shutdown_connections 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.132:6800/2008345174,v1:172.21.15.132:6801/2008345174] conn(0x7f4d3c072350 0x7f4d3c074760 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f4d3c06c680 0x7f4d3c06eb30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d4c0ff3c0 0x7f4d4c196d20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 --2- 172.21.15.88:0/3219424617 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d4c0ffd60 0x7f4d4c197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 -- 172.21.15.88:0/3219424617 >> 172.21.15.88:0/3219424617 conn(0x7f4d4c0fac60 msgr2=0x7f4d4c0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 -- 172.21.15.88:0/3219424617 shutdown_connections 2024-02-19T18:57:47.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:47.508+0000 7f4d53119700 1 -- 172.21.15.88:0/3219424617 wait complete. 2024-02-19T18:57:48.037 INFO:teuthology.orchestra.run.smithi088.stdout:38654705699 2024-02-19T18:57:48.038 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-19T18:57:49.000 INFO:teuthology.orchestra.run.smithi088.stdout:81604378649 2024-02-19T18:57:49.000 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-19T18:57:49.301 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:48 smithi132 ceph-mon[37969]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:49.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:48 smithi088 ceph-mon[29272]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:49.613 INFO:teuthology.orchestra.run.smithi088.stdout:60129542175 2024-02-19T18:57:49.614 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-19T18:57:50.250 INFO:teuthology.orchestra.run.smithi088.stdout:111669149716 2024-02-19T18:57:50.250 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.701+0000 7fac5be68700 1 -- 172.21.15.88:0/1825908485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac54100d70 msgr2=0x7fac541094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.701+0000 7fac5be68700 1 --2- 172.21.15.88:0/1825908485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac54100d70 0x7fac541094d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fac44009940 tx=0x7fac44009c50 comp rx=0 tx=0).stop 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.701+0000 7fac5be68700 1 -- 172.21.15.88:0/1825908485 shutdown_connections 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.701+0000 7fac5be68700 1 --2- 172.21.15.88:0/1825908485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac54100d70 0x7fac541094d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.701+0000 7fac5be68700 1 --2- 172.21.15.88:0/1825908485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac54100460 0x7fac54100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.701+0000 7fac5be68700 1 -- 172.21.15.88:0/1825908485 >> 172.21.15.88:0/1825908485 conn(0x7fac540748b0 msgr2=0x7fac54074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.701+0000 7fac5be68700 1 -- 172.21.15.88:0/1825908485 shutdown_connections 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.701+0000 7fac5be68700 1 -- 172.21.15.88:0/1825908485 wait complete. 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.702+0000 7fac5be68700 1 Processor -- start 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.702+0000 7fac5be68700 1 -- start start 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.702+0000 7fac5be68700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac54100460 0x7fac54195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.702+0000 7fac5be68700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac54100d70 0x7fac54196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.702+0000 7fac5be68700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac54196960 con 0x7fac54100d70 2024-02-19T18:57:50.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.702+0000 7fac5be68700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5419a6f0 con 0x7fac54100460 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.702+0000 7fac59403700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac54100d70 0x7fac54196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.702+0000 7fac59403700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac54100d70 0x7fac54196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60844/0 (socket says 172.21.15.88:60844) 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.702+0000 7fac59403700 1 -- 172.21.15.88:0/3427081865 learned_addr learned my addr 172.21.15.88:0/3427081865 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.702+0000 7fac59c04700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac54100460 0x7fac54195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.703+0000 7fac59403700 1 -- 172.21.15.88:0/3427081865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac54100460 msgr2=0x7fac54195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.703+0000 7fac59403700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac54100460 0x7fac54195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.703+0000 7fac59403700 1 -- 172.21.15.88:0/3427081865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac440095f0 con 0x7fac54100d70 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.703+0000 7fac59403700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac54100d70 0x7fac54196280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fac440055f0 tx=0x7fac44005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.703+0000 7fac4affd700 1 -- 172.21.15.88:0/3427081865 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac4401d070 con 0x7fac54100d70 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.703+0000 7fac4affd700 1 -- 172.21.15.88:0/3427081865 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fac4400f460 con 0x7fac54100d70 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.703+0000 7fac5be68700 1 -- 172.21.15.88:0/3427081865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac5419a9d0 con 0x7fac54100d70 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.703+0000 7fac4affd700 1 -- 172.21.15.88:0/3427081865 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac44021640 con 0x7fac54100d70 2024-02-19T18:57:50.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.703+0000 7fac5be68700 1 -- 172.21.15.88:0/3427081865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac5419af20 con 0x7fac54100d70 2024-02-19T18:57:50.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.704+0000 7fac5be68700 1 -- 172.21.15.88:0/3427081865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fac5410c570 con 0x7fac54100d70 2024-02-19T18:57:50.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.704+0000 7fac4affd700 1 -- 172.21.15.88:0/3427081865 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fac4400fab0 con 0x7fac54100d70 2024-02-19T18:57:50.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.705+0000 7fac4affd700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fac4006c680 0x7fac4006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:50.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.705+0000 7fac59c04700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fac4006c680 0x7fac4006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:50.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.705+0000 7fac4affd700 1 -- 172.21.15.88:0/3427081865 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fac4408c740 con 0x7fac54100d70 2024-02-19T18:57:50.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.705+0000 7fac4affd700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780] conn(0x7fac40072350 0x7fac40074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:50.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.705+0000 7fac4affd700 1 -- 172.21.15.88:0/3427081865 --> [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fac40074e10 con 0x7fac40072350 2024-02-19T18:57:50.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.705+0000 7fac4affd700 1 -- 172.21.15.88:0/3427081865 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac4408ee10 con 0x7fac54100d70 2024-02-19T18:57:50.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.705+0000 7fac59c04700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fac4006c680 0x7fac4006eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac50005950 tx=0x7fac5000b410 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:50.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.705+0000 7fac5a405700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780] conn(0x7fac40072350 0x7fac40074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:50.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.706+0000 7fac5a405700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780] conn(0x7fac40072350 0x7fac40074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:50.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.706+0000 7fac4affd700 1 -- 172.21.15.88:0/3427081865 <== osd.5 v2:172.21.15.132:6816/2563696780 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fac40074e10 con 0x7fac40072350 2024-02-19T18:57:50.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.722+0000 7fac5be68700 1 -- 172.21.15.88:0/3427081865 --> [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fac5404f2a0 con 0x7fac40072350 2024-02-19T18:57:50.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.722+0000 7fac4affd700 1 -- 172.21.15.88:0/3427081865 <== osd.5 v2:172.21.15.132:6816/2563696780 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fac5404f2a0 con 0x7fac40072350 2024-02-19T18:57:50.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 -- 172.21.15.88:0/3427081865 >> [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780] conn(0x7fac40072350 msgr2=0x7fac40074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:50.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780] conn(0x7fac40072350 0x7fac40074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:50.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 -- 172.21.15.88:0/3427081865 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fac4006c680 msgr2=0x7fac4006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:50.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fac4006c680 0x7fac4006eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fac50005950 tx=0x7fac5000b410 comp rx=0 tx=0).stop 2024-02-19T18:57:50.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 -- 172.21.15.88:0/3427081865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac54100d70 msgr2=0x7fac54196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:50.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac54100d70 0x7fac54196280 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fac440055f0 tx=0x7fac44005620 comp rx=0 tx=0).stop 2024-02-19T18:57:50.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 -- 172.21.15.88:0/3427081865 shutdown_connections 2024-02-19T18:57:50.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.132:6816/2563696780,v1:172.21.15.132:6817/2563696780] conn(0x7fac40072350 0x7fac40074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:50.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fac4006c680 0x7fac4006eb30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac54100460 0x7fac54195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 --2- 172.21.15.88:0/3427081865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac54100d70 0x7fac54196280 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 -- 172.21.15.88:0/3427081865 >> 172.21.15.88:0/3427081865 conn(0x7fac540748b0 msgr2=0x7fac540ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 -- 172.21.15.88:0/3427081865 shutdown_connections 2024-02-19T18:57:50.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:50.723+0000 7fac5be68700 1 -- 172.21.15.88:0/3427081865 wait complete. 2024-02-19T18:57:51.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:50 smithi088 ceph-mon[29272]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:51.340 INFO:teuthology.orchestra.run.smithi088.stdout:167503724554 2024-02-19T18:57:51.340 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-19T18:57:51.373 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:51.375 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:51.383 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:51.431 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:57:51.494 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:50 smithi132 ceph-mon[37969]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:53.314 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:52 smithi088 ceph-mon[29272]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:53.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:52 smithi132 ceph-mon[37969]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:55.283 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:54 smithi088 ceph-mon[29272]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:55.283 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:54 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:57:55.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:54 smithi132 ceph-mon[37969]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:55.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:54 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:57:57.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:56 smithi088 ceph-mon[29272]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:57.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:56 smithi132 ceph-mon[37969]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:57.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.683+0000 7f1a99606700 1 -- 172.21.15.88:0/183583839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a941004e0 msgr2=0x7f1a941008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:57.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.683+0000 7f1a99606700 1 --2- 172.21.15.88:0/183583839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a941004e0 0x7f1a941008b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a84009910 tx=0x7f1a84009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:57.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.683+0000 7f1a99606700 1 -- 172.21.15.88:0/183583839 shutdown_connections 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.683+0000 7f1a99606700 1 --2- 172.21.15.88:0/183583839 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a94100e80 0x7f1a94104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.683+0000 7f1a99606700 1 --2- 172.21.15.88:0/183583839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a941004e0 0x7f1a941008b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.683+0000 7f1a99606700 1 -- 172.21.15.88:0/183583839 >> 172.21.15.88:0/183583839 conn(0x7f1a940fbda0 msgr2=0x7f1a940fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.683+0000 7f1a99606700 1 -- 172.21.15.88:0/183583839 shutdown_connections 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.683+0000 7f1a99606700 1 -- 172.21.15.88:0/183583839 wait complete. 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.683+0000 7f1a99606700 1 Processor -- start 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a99606700 1 -- start start 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a99606700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a941004e0 0x7f1a94073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a99606700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a94100e80 0x7f1a94073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a99606700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94073ad0 con 0x7f1a941004e0 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a99606700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a94073c10 con 0x7f1a94100e80 2024-02-19T18:57:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a92ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a941004e0 0x7f1a94073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a92ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a941004e0 0x7f1a94073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37220/0 (socket says 172.21.15.88:37220) 2024-02-19T18:57:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a92ffd700 1 -- 172.21.15.88:0/2135730304 learned_addr learned my addr 172.21.15.88:0/2135730304 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a92ffd700 1 -- 172.21.15.88:0/2135730304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a94100e80 msgr2=0x7f1a94073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a92ffd700 1 --2- 172.21.15.88:0/2135730304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a94100e80 0x7f1a94073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a92ffd700 1 -- 172.21.15.88:0/2135730304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a840095f0 con 0x7f1a941004e0 2024-02-19T18:57:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a92ffd700 1 --2- 172.21.15.88:0/2135730304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a941004e0 0x7f1a94073050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1a84006010 tx=0x7f1a8400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a7bfff700 1 -- 172.21.15.88:0/2135730304 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a8401d070 con 0x7f1a941004e0 2024-02-19T18:57:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a99606700 1 -- 172.21.15.88:0/2135730304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a94073e90 con 0x7f1a941004e0 2024-02-19T18:57:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a7bfff700 1 -- 172.21.15.88:0/2135730304 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1a8400f460 con 0x7f1a941004e0 2024-02-19T18:57:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.684+0000 7f1a7bfff700 1 -- 172.21.15.88:0/2135730304 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a84021640 con 0x7f1a941004e0 2024-02-19T18:57:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.685+0000 7f1a99606700 1 -- 172.21.15.88:0/2135730304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a941a23f0 con 0x7f1a941004e0 2024-02-19T18:57:57.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.685+0000 7f1a99606700 1 -- 172.21.15.88:0/2135730304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a941087f0 con 0x7f1a941004e0 2024-02-19T18:57:57.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.686+0000 7f1a7bfff700 1 -- 172.21.15.88:0/2135730304 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f1a8400faf0 con 0x7f1a941004e0 2024-02-19T18:57:57.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.686+0000 7f1a7bfff700 1 --2- 172.21.15.88:0/2135730304 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f1a7c06c680 0x7f1a7c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:57.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.686+0000 7f1a7bfff700 1 -- 172.21.15.88:0/2135730304 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1a8408c860 con 0x7f1a941004e0 2024-02-19T18:57:57.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.686+0000 7f1a927fc700 1 --2- 172.21.15.88:0/2135730304 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f1a7c06c680 0x7f1a7c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:57.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.687+0000 7f1a927fc700 1 --2- 172.21.15.88:0/2135730304 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f1a7c06c680 0x7f1a7c06eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a94074840 tx=0x7f1a88005c80 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:57.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.691+0000 7f1a7bfff700 1 -- 172.21.15.88:0/2135730304 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1a84057280 con 0x7f1a941004e0 2024-02-19T18:57:57.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.867+0000 7f3e9b74a700 1 -- 172.21.15.88:0/4144519773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e940ff3c0 msgr2=0x7f3e940ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:57.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.867+0000 7f3e9b74a700 1 --2- 172.21.15.88:0/4144519773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e940ff3c0 0x7f3e940ff790 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009910 tx=0x7f3e84009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:57.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.867+0000 7f3e9b74a700 1 -- 172.21.15.88:0/4144519773 shutdown_connections 2024-02-19T18:57:57.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.867+0000 7f3e9b74a700 1 --2- 172.21.15.88:0/4144519773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e940ffd60 0x7f3e94103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:57.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.867+0000 7f3e9b74a700 1 --2- 172.21.15.88:0/4144519773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e940ff3c0 0x7f3e940ff790 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:57.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.867+0000 7f3e9b74a700 1 -- 172.21.15.88:0/4144519773 >> 172.21.15.88:0/4144519773 conn(0x7f3e940fac60 msgr2=0x7f3e940fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.867+0000 7f3e9b74a700 1 -- 172.21.15.88:0/4144519773 shutdown_connections 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.867+0000 7f3e9b74a700 1 -- 172.21.15.88:0/4144519773 wait complete. 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.868+0000 7f3e9b74a700 1 Processor -- start 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.868+0000 7f3e9b74a700 1 -- start start 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.868+0000 7f3e9b74a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e940ff3c0 0x7f3e94196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.868+0000 7f3e9b74a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e940ffd60 0x7f3e941972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.868+0000 7f3e9b74a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e941979c0 con 0x7f3e940ffd60 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.868+0000 7f3e9b74a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e94197b00 con 0x7f3e940ff3c0 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.868+0000 7f3e98ce5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e940ffd60 0x7f3e941972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.868+0000 7f3e98ce5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e940ffd60 0x7f3e941972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37238/0 (socket says 172.21.15.88:37238) 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.868+0000 7f3e98ce5700 1 -- 172.21.15.88:0/2431195708 learned_addr learned my addr 172.21.15.88:0/2431195708 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.868+0000 7f3e994e6700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e940ff3c0 0x7f3e94196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.869+0000 7f3e98ce5700 1 -- 172.21.15.88:0/2431195708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e940ff3c0 msgr2=0x7f3e94196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:57.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.869+0000 7f3e98ce5700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e940ff3c0 0x7f3e94196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:57.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.869+0000 7f3e98ce5700 1 -- 172.21.15.88:0/2431195708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e840095f0 con 0x7f3e940ffd60 2024-02-19T18:57:57.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.869+0000 7f3e994e6700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e940ff3c0 0x7f3e94196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T18:57:57.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.869+0000 7f3e98ce5700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e940ffd60 0x7f3e941972e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000b480 tx=0x7f3e9000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:57.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.869+0000 7f3e8a7fc700 1 -- 172.21.15.88:0/2431195708 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e900118c0 con 0x7f3e940ffd60 2024-02-19T18:57:57.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.869+0000 7f3e9b74a700 1 -- 172.21.15.88:0/2431195708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9419fe10 con 0x7f3e940ffd60 2024-02-19T18:57:57.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.869+0000 7f3e8a7fc700 1 -- 172.21.15.88:0/2431195708 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3e90010380 con 0x7f3e940ffd60 2024-02-19T18:57:57.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.869+0000 7f3e8a7fc700 1 -- 172.21.15.88:0/2431195708 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e9000f4f0 con 0x7f3e940ffd60 2024-02-19T18:57:57.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.869+0000 7f3e9b74a700 1 -- 172.21.15.88:0/2431195708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e941a0330 con 0x7f3e940ffd60 2024-02-19T18:57:57.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.870+0000 7f3e9b74a700 1 -- 172.21.15.88:0/2431195708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e941076d0 con 0x7f3e940ffd60 2024-02-19T18:57:57.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.871+0000 7f3e8a7fc700 1 -- 172.21.15.88:0/2431195708 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f3e900104f0 con 0x7f3e940ffd60 2024-02-19T18:57:57.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.871+0000 7f3e8a7fc700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3e8006c680 0x7f3e8006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:57.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.871+0000 7f3e8a7fc700 1 -- 172.21.15.88:0/2431195708 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3e9008b180 con 0x7f3e940ffd60 2024-02-19T18:57:57.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.871+0000 7f3e994e6700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3e8006c680 0x7f3e8006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:57.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.872+0000 7f3e994e6700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3e8006c680 0x7f3e8006eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400b5c0 tx=0x7f3e8401a040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:57.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.876+0000 7f3e8a7fc700 1 -- 172.21.15.88:0/2431195708 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3e90055ba0 con 0x7f3e940ffd60 2024-02-19T18:57:57.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.891+0000 7f1a99606700 1 -- 172.21.15.88:0/2135730304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1a9404f2a0 con 0x7f1a941004e0 2024-02-19T18:57:57.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.892+0000 7f1a7bfff700 1 -- 172.21.15.88:0/2135730304 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1a84026090 con 0x7f1a941004e0 2024-02-19T18:57:57.909 INFO:teuthology.orchestra.run.smithi088.stdout:38654705701 2024-02-19T18:57:57.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.894+0000 7f1a99606700 1 -- 172.21.15.88:0/2135730304 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f1a7c06c680 msgr2=0x7f1a7c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:57.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.894+0000 7f1a99606700 1 --2- 172.21.15.88:0/2135730304 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f1a7c06c680 0x7f1a7c06eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1a94074840 tx=0x7f1a88005c80 comp rx=0 tx=0).stop 2024-02-19T18:57:57.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.894+0000 7f1a99606700 1 -- 172.21.15.88:0/2135730304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a941004e0 msgr2=0x7f1a94073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:57.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.894+0000 7f1a99606700 1 --2- 172.21.15.88:0/2135730304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a941004e0 0x7f1a94073050 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1a84006010 tx=0x7f1a8400be20 comp rx=0 tx=0).stop 2024-02-19T18:57:57.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.894+0000 7f1a99606700 1 -- 172.21.15.88:0/2135730304 shutdown_connections 2024-02-19T18:57:57.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.894+0000 7f1a99606700 1 --2- 172.21.15.88:0/2135730304 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f1a7c06c680 0x7f1a7c06eb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:57.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.894+0000 7f1a99606700 1 --2- 172.21.15.88:0/2135730304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a941004e0 0x7f1a94073050 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:57.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.894+0000 7f1a99606700 1 --2- 172.21.15.88:0/2135730304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a94100e80 0x7f1a94073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:57.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.894+0000 7f1a99606700 1 -- 172.21.15.88:0/2135730304 >> 172.21.15.88:0/2135730304 conn(0x7f1a940fbda0 msgr2=0x7f1a940fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:57.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.894+0000 7f1a99606700 1 -- 172.21.15.88:0/2135730304 shutdown_connections 2024-02-19T18:57:57.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:57.894+0000 7f1a99606700 1 -- 172.21.15.88:0/2135730304 wait complete. 2024-02-19T18:57:58.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.076+0000 7f3e9b74a700 1 -- 172.21.15.88:0/2431195708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3e9404f2a0 con 0x7f3e940ffd60 2024-02-19T18:57:58.094 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:58 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2135730304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-19T18:57:58.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.076+0000 7f3e8a7fc700 1 -- 172.21.15.88:0/2431195708 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3e900591c0 con 0x7f3e940ffd60 2024-02-19T18:57:58.094 INFO:teuthology.orchestra.run.smithi088.stdout:81604378651 2024-02-19T18:57:58.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.079+0000 7f3e9b74a700 1 -- 172.21.15.88:0/2431195708 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3e8006c680 msgr2=0x7f3e8006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:58.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.079+0000 7f3e9b74a700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3e8006c680 0x7f3e8006eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3e8400b5c0 tx=0x7f3e8401a040 comp rx=0 tx=0).stop 2024-02-19T18:57:58.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.079+0000 7f3e9b74a700 1 -- 172.21.15.88:0/2431195708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e940ffd60 msgr2=0x7f3e941972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:58.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.079+0000 7f3e9b74a700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e940ffd60 0x7f3e941972e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000b480 tx=0x7f3e9000b840 comp rx=0 tx=0).stop 2024-02-19T18:57:58.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.079+0000 7f3e9b74a700 1 -- 172.21.15.88:0/2431195708 shutdown_connections 2024-02-19T18:57:58.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.079+0000 7f3e9b74a700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3e8006c680 0x7f3e8006eb30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.079+0000 7f3e9b74a700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e940ff3c0 0x7f3e94196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.079+0000 7f3e9b74a700 1 --2- 172.21.15.88:0/2431195708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e940ffd60 0x7f3e941972e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.079+0000 7f3e9b74a700 1 -- 172.21.15.88:0/2431195708 >> 172.21.15.88:0/2431195708 conn(0x7f3e940fac60 msgr2=0x7f3e940fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:58.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.079+0000 7f3e9b74a700 1 -- 172.21.15.88:0/2431195708 shutdown_connections 2024-02-19T18:57:58.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.079+0000 7f3e9b74a700 1 -- 172.21.15.88:0/2431195708 wait complete. 2024-02-19T18:57:58.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.094+0000 7ff5453a2700 1 -- 172.21.15.88:0/3316241993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5400ff210 msgr2=0x7ff540104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:58.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.094+0000 7ff5453a2700 1 --2- 172.21.15.88:0/3316241993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5400ff210 0x7ff540104cf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff534009910 tx=0x7ff534009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:58.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.094+0000 7ff5453a2700 1 -- 172.21.15.88:0/3316241993 shutdown_connections 2024-02-19T18:57:58.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.094+0000 7ff5453a2700 1 --2- 172.21.15.88:0/3316241993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5400ff210 0x7ff540104cf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.094+0000 7ff5453a2700 1 --2- 172.21.15.88:0/3316241993 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5400fe870 0x7ff5400fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.094+0000 7ff5453a2700 1 -- 172.21.15.88:0/3316241993 >> 172.21.15.88:0/3316241993 conn(0x7ff540075400 msgr2=0x7ff540075800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.094+0000 7ff5453a2700 1 -- 172.21.15.88:0/3316241993 shutdown_connections 2024-02-19T18:57:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.094+0000 7ff5453a2700 1 -- 172.21.15.88:0/3316241993 wait complete. 2024-02-19T18:57:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.095+0000 7ff5453a2700 1 Processor -- start 2024-02-19T18:57:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.095+0000 7ff5453a2700 1 -- start start 2024-02-19T18:57:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.095+0000 7ff5453a2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5400fe870 0x7ff54019b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.095+0000 7ff5453a2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5400ff210 0x7ff54019be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.095+0000 7ff5453a2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54019c520 con 0x7ff5400fe870 2024-02-19T18:57:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.095+0000 7ff5453a2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5401959d0 con 0x7ff5400ff210 2024-02-19T18:57:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.095+0000 7ff53ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5400fe870 0x7ff54019b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.095+0000 7ff53ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5400fe870 0x7ff54019b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37254/0 (socket says 172.21.15.88:37254) 2024-02-19T18:57:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.095+0000 7ff53ffff700 1 -- 172.21.15.88:0/4163322948 learned_addr learned my addr 172.21.15.88:0/4163322948 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.095+0000 7ff53f7fe700 1 --2- 172.21.15.88:0/4163322948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5400ff210 0x7ff54019be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.096+0000 7ff53ffff700 1 -- 172.21.15.88:0/4163322948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5400ff210 msgr2=0x7ff54019be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.096+0000 7ff53ffff700 1 --2- 172.21.15.88:0/4163322948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5400ff210 0x7ff54019be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.096+0000 7ff53ffff700 1 -- 172.21.15.88:0/4163322948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340095f0 con 0x7ff5400fe870 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.096+0000 7ff53ffff700 1 --2- 172.21.15.88:0/4163322948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5400fe870 0x7ff54019b950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff53000d8d0 tx=0x7ff53000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.096+0000 7ff53d7fa700 1 -- 172.21.15.88:0/4163322948 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5300098a0 con 0x7ff5400fe870 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.096+0000 7ff5453a2700 1 -- 172.21.15.88:0/4163322948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff540195cb0 con 0x7ff5400fe870 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.096+0000 7ff53d7fa700 1 -- 172.21.15.88:0/4163322948 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff530010460 con 0x7ff5400fe870 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.096+0000 7ff53d7fa700 1 -- 172.21.15.88:0/4163322948 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff53000f5f0 con 0x7ff5400fe870 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.096+0000 7ff5453a2700 1 -- 172.21.15.88:0/4163322948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff540196200 con 0x7ff5400fe870 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.097+0000 7ff5453a2700 1 -- 172.21.15.88:0/4163322948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff540062380 con 0x7ff5400fe870 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.098+0000 7ff53d7fa700 1 -- 172.21.15.88:0/4163322948 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff53000f820 con 0x7ff5400fe870 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.098+0000 7ff53d7fa700 1 --2- 172.21.15.88:0/4163322948 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff52806c680 0x7ff52806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.098+0000 7ff53d7fa700 1 -- 172.21.15.88:0/4163322948 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff53008b420 con 0x7ff5400fe870 2024-02-19T18:57:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.098+0000 7ff53f7fe700 1 --2- 172.21.15.88:0/4163322948 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff52806c680 0x7ff52806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:58.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.099+0000 7ff53f7fe700 1 --2- 172.21.15.88:0/4163322948 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff52806c680 0x7ff52806eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff5400fce60 tx=0x7ff53401a040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:58.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.103+0000 7ff53d7fa700 1 -- 172.21.15.88:0/4163322948 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff5300599b0 con 0x7ff5400fe870 2024-02-19T18:57:58.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.301+0000 7ff5453a2700 1 -- 172.21.15.88:0/4163322948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff540196ee0 con 0x7ff5400fe870 2024-02-19T18:57:58.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.301+0000 7ff53d7fa700 1 -- 172.21.15.88:0/4163322948 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff530059540 con 0x7ff5400fe870 2024-02-19T18:57:58.319 INFO:teuthology.orchestra.run.smithi088.stdout:111669149718 2024-02-19T18:57:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.303+0000 7ff5453a2700 1 -- 172.21.15.88:0/4163322948 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff52806c680 msgr2=0x7ff52806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.303+0000 7ff5453a2700 1 --2- 172.21.15.88:0/4163322948 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff52806c680 0x7ff52806eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff5400fce60 tx=0x7ff53401a040 comp rx=0 tx=0).stop 2024-02-19T18:57:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.303+0000 7ff5453a2700 1 -- 172.21.15.88:0/4163322948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5400fe870 msgr2=0x7ff54019b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.303+0000 7ff5453a2700 1 --2- 172.21.15.88:0/4163322948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5400fe870 0x7ff54019b950 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff53000d8d0 tx=0x7ff53000dbe0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.303+0000 7ff5453a2700 1 -- 172.21.15.88:0/4163322948 shutdown_connections 2024-02-19T18:57:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.303+0000 7ff5453a2700 1 --2- 172.21.15.88:0/4163322948 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff52806c680 0x7ff52806eb30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.303+0000 7ff5453a2700 1 --2- 172.21.15.88:0/4163322948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5400fe870 0x7ff54019b950 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.303+0000 7ff5453a2700 1 --2- 172.21.15.88:0/4163322948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5400ff210 0x7ff54019be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.303+0000 7ff5453a2700 1 -- 172.21.15.88:0/4163322948 >> 172.21.15.88:0/4163322948 conn(0x7ff540075400 msgr2=0x7ff5400fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:58.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.303+0000 7ff5453a2700 1 -- 172.21.15.88:0/4163322948 shutdown_connections 2024-02-19T18:57:58.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.303+0000 7ff5453a2700 1 -- 172.21.15.88:0/4163322948 wait complete. 2024-02-19T18:57:58.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:58 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2135730304' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-19T18:57:58.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.495+0000 7f0a218f1700 1 -- 172.21.15.88:0/1482405537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a1c100460 msgr2=0x7f0a1c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:58.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.495+0000 7f0a218f1700 1 --2- 172.21.15.88:0/1482405537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a1c100460 0x7f0a1c100830 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c009910 tx=0x7f0a0c009c20 comp rx=0 tx=0).stop 2024-02-19T18:57:58.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.495+0000 7f0a218f1700 1 -- 172.21.15.88:0/1482405537 shutdown_connections 2024-02-19T18:57:58.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.495+0000 7f0a218f1700 1 --2- 172.21.15.88:0/1482405537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a1c100d70 0x7f0a1c1094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.495+0000 7f0a218f1700 1 --2- 172.21.15.88:0/1482405537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a1c100460 0x7f0a1c100830 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.495+0000 7f0a218f1700 1 -- 172.21.15.88:0/1482405537 >> 172.21.15.88:0/1482405537 conn(0x7f0a1c0748b0 msgr2=0x7f0a1c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:58.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.495+0000 7f0a218f1700 1 -- 172.21.15.88:0/1482405537 shutdown_connections 2024-02-19T18:57:58.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.495+0000 7f0a218f1700 1 -- 172.21.15.88:0/1482405537 wait complete. 2024-02-19T18:57:58.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.495+0000 7f0a218f1700 1 Processor -- start 2024-02-19T18:57:58.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a218f1700 1 -- start start 2024-02-19T18:57:58.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a218f1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a1c100460 0x7f0a1c195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:58.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a218f1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a1c100d70 0x7f0a1c196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:58.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a218f1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c196b50 con 0x7f0a1c100d70 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a218f1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1c19a8e0 con 0x7f0a1c100460 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a127fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a1c100d70 0x7f0a1c196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a127fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a1c100d70 0x7f0a1c196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37282/0 (socket says 172.21.15.88:37282) 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a127fc700 1 -- 172.21.15.88:0/1878950093 learned_addr learned my addr 172.21.15.88:0/1878950093 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a1affd700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a1c100460 0x7f0a1c195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a127fc700 1 -- 172.21.15.88:0/1878950093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a1c100460 msgr2=0x7f0a1c195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a127fc700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a1c100460 0x7f0a1c195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a127fc700 1 -- 172.21.15.88:0/1878950093 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a0c0095f0 con 0x7f0a1c100d70 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a1affd700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a1c100460 0x7f0a1c195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.496+0000 7f0a127fc700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a1c100d70 0x7f0a1c196470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400e920 tx=0x7f0a0400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.497+0000 7f0a208ef700 1 -- 172.21.15.88:0/1878950093 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a04009790 con 0x7f0a1c100d70 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.497+0000 7f0a208ef700 1 -- 172.21.15.88:0/1878950093 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0a0400cba0 con 0x7f0a1c100d70 2024-02-19T18:57:58.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.497+0000 7f0a218f1700 1 -- 172.21.15.88:0/1878950093 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1c19abc0 con 0x7f0a1c100d70 2024-02-19T18:57:58.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.497+0000 7f0a208ef700 1 -- 172.21.15.88:0/1878950093 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a04018630 con 0x7f0a1c100d70 2024-02-19T18:57:58.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.497+0000 7f0a218f1700 1 -- 172.21.15.88:0/1878950093 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1c19b110 con 0x7f0a1c100d70 2024-02-19T18:57:58.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.498+0000 7f0a218f1700 1 -- 172.21.15.88:0/1878950093 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1c04f2a0 con 0x7f0a1c100d70 2024-02-19T18:57:58.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.498+0000 7f0a208ef700 1 -- 172.21.15.88:0/1878950093 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f0a0400cd10 con 0x7f0a1c100d70 2024-02-19T18:57:58.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.498+0000 7f0a208ef700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0a0806c680 0x7f0a0806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:57:58.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.499+0000 7f0a208ef700 1 -- 172.21.15.88:0/1878950093 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0a04014070 con 0x7f0a1c100d70 2024-02-19T18:57:58.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.499+0000 7f0a1affd700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0a0806c680 0x7f0a0806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:57:58.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.499+0000 7f0a1affd700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0a0806c680 0x7f0a0806eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00b760 tx=0x7f0a0c005940 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:57:58.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.503+0000 7f0a208ef700 1 -- 172.21.15.88:0/1878950093 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0a04051720 con 0x7f0a1c100d70 2024-02-19T18:57:58.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.697+0000 7f0a218f1700 1 -- 172.21.15.88:0/1878950093 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0a1c19b3f0 con 0x7f0a1c100d70 2024-02-19T18:57:58.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.697+0000 7f0a208ef700 1 -- 172.21.15.88:0/1878950093 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0a04059ea0 con 0x7f0a1c100d70 2024-02-19T18:57:58.715 INFO:teuthology.orchestra.run.smithi088.stdout:60129542177 2024-02-19T18:57:58.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.699+0000 7f0a218f1700 1 -- 172.21.15.88:0/1878950093 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0a0806c680 msgr2=0x7f0a0806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:58.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.699+0000 7f0a218f1700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0a0806c680 0x7f0a0806eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00b760 tx=0x7f0a0c005940 comp rx=0 tx=0).stop 2024-02-19T18:57:58.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.699+0000 7f0a218f1700 1 -- 172.21.15.88:0/1878950093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a1c100d70 msgr2=0x7f0a1c196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:57:58.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.699+0000 7f0a218f1700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a1c100d70 0x7f0a1c196470 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400e920 tx=0x7f0a0400ece0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.699+0000 7f0a218f1700 1 -- 172.21.15.88:0/1878950093 shutdown_connections 2024-02-19T18:57:58.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.699+0000 7f0a218f1700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0a0806c680 0x7f0a0806eb30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.699+0000 7f0a218f1700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a1c100460 0x7f0a1c195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.699+0000 7f0a218f1700 1 --2- 172.21.15.88:0/1878950093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a1c100d70 0x7f0a1c196470 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:57:58.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.699+0000 7f0a218f1700 1 -- 172.21.15.88:0/1878950093 >> 172.21.15.88:0/1878950093 conn(0x7f0a1c0748b0 msgr2=0x7f0a1c0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:57:58.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.700+0000 7f0a218f1700 1 -- 172.21.15.88:0/1878950093 shutdown_connections 2024-02-19T18:57:58.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:57:58.700+0000 7f0a218f1700 1 -- 172.21.15.88:0/1878950093 wait complete. 2024-02-19T18:57:58.987 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378651 for osd.2 2024-02-19T18:57:58.988 DEBUG:teuthology.parallel:result is None 2024-02-19T18:57:59.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:59 smithi088 ceph-mon[29272]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:59.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:59 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2431195708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-19T18:57:59.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:59 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/4163322948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-19T18:57:59.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:57:59 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1878950093' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-19T18:57:59.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:59 smithi132 ceph-mon[37969]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:57:59.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:59 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2431195708' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-19T18:57:59.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:59 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/4163322948' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-19T18:57:59.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:57:59 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1878950093' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-19T18:57:59.682 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-02-19T18:57:59.682 DEBUG:teuthology.parallel:result is None 2024-02-19T18:58:00.265 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149716 got 111669149718 for osd.3 2024-02-19T18:58:00.265 DEBUG:teuthology.parallel:result is None 2024-02-19T18:58:00.886 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542175 got 60129542177 for osd.1 2024-02-19T18:58:00.886 DEBUG:teuthology.parallel:result is None 2024-02-19T18:58:01.204 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:01 smithi088 ceph-mon[29272]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.289+0000 7fa462c3d700 1 -- 172.21.15.88:0/3118107841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c1004e0 msgr2=0x7fa45c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.289+0000 7fa462c3d700 1 --2- 172.21.15.88:0/3118107841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c1004e0 0x7fa45c1008b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fa450009910 tx=0x7fa450009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.290+0000 7fa462c3d700 1 -- 172.21.15.88:0/3118107841 shutdown_connections 2024-02-19T18:58:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.290+0000 7fa462c3d700 1 --2- 172.21.15.88:0/3118107841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa45c100e80 0x7fa45c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.290+0000 7fa462c3d700 1 --2- 172.21.15.88:0/3118107841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c1004e0 0x7fa45c1008b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.290+0000 7fa462c3d700 1 -- 172.21.15.88:0/3118107841 >> 172.21.15.88:0/3118107841 conn(0x7fa45c0fbda0 msgr2=0x7fa45c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.290+0000 7fa462c3d700 1 -- 172.21.15.88:0/3118107841 shutdown_connections 2024-02-19T18:58:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.290+0000 7fa462c3d700 1 -- 172.21.15.88:0/3118107841 wait complete. 2024-02-19T18:58:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.290+0000 7fa462c3d700 1 Processor -- start 2024-02-19T18:58:01.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.290+0000 7fa462c3d700 1 -- start start 2024-02-19T18:58:01.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.290+0000 7fa462c3d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c1004e0 0x7fa45c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:01.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.290+0000 7fa462c3d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa45c100e80 0x7fa45c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:01.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa462c3d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c196b10 con 0x7fa45c1004e0 2024-02-19T18:58:01.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa462c3d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa45c19a8a0 con 0x7fa45c100e80 2024-02-19T18:58:01.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa4609d9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c1004e0 0x7fa45c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa4609d9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c1004e0 0x7fa45c195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37300/0 (socket says 172.21.15.88:37300) 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa4609d9700 1 -- 172.21.15.88:0/1225887764 learned_addr learned my addr 172.21.15.88:0/1225887764 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa4609d9700 1 -- 172.21.15.88:0/1225887764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa45c100e80 msgr2=0x7fa45c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa4609d9700 1 --2- 172.21.15.88:0/1225887764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa45c100e80 0x7fa45c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa4609d9700 1 -- 172.21.15.88:0/1225887764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4500095f0 con 0x7fa45c1004e0 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa4609d9700 1 --2- 172.21.15.88:0/1225887764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c1004e0 0x7fa45c195ef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa450006010 tx=0x7fa45000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa459ffb700 1 -- 172.21.15.88:0/1225887764 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa45001d070 con 0x7fa45c1004e0 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa462c3d700 1 -- 172.21.15.88:0/1225887764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45c19ab20 con 0x7fa45c1004e0 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa459ffb700 1 -- 172.21.15.88:0/1225887764 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa45000f460 con 0x7fa45c1004e0 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa459ffb700 1 -- 172.21.15.88:0/1225887764 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa450017630 con 0x7fa45c1004e0 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.291+0000 7fa462c3d700 1 -- 172.21.15.88:0/1225887764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa45c19b010 con 0x7fa45c1004e0 2024-02-19T18:58:01.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.292+0000 7fa462c3d700 1 -- 172.21.15.88:0/1225887764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa45c1087f0 con 0x7fa45c1004e0 2024-02-19T18:58:01.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.293+0000 7fa459ffb700 1 -- 172.21.15.88:0/1225887764 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa45000f5d0 con 0x7fa45c1004e0 2024-02-19T18:58:01.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.293+0000 7fa459ffb700 1 --2- 172.21.15.88:0/1225887764 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa44c06c680 0x7fa44c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:01.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.293+0000 7fa459ffb700 1 -- 172.21.15.88:0/1225887764 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa45008ca90 con 0x7fa45c1004e0 2024-02-19T18:58:01.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.293+0000 7fa45bfff700 1 --2- 172.21.15.88:0/1225887764 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa44c06c680 0x7fa44c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:01.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.294+0000 7fa45bfff700 1 --2- 172.21.15.88:0/1225887764 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa44c06c680 0x7fa44c06eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa45c197510 tx=0x7fa448005c80 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:01.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.298+0000 7fa459ffb700 1 -- 172.21.15.88:0/1225887764 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa450057530 con 0x7fa45c1004e0 2024-02-19T18:58:01.403 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:01.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:01 smithi132 ceph-mon[37969]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:01.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.503+0000 7fa462c3d700 1 -- 172.21.15.88:0/1225887764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fa45c04f2a0 con 0x7fa45c1004e0 2024-02-19T18:58:01.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.504+0000 7fa459ffb700 1 -- 172.21.15.88:0/1225887764 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa45005ab50 con 0x7fa45c1004e0 2024-02-19T18:58:01.522 INFO:teuthology.orchestra.run.smithi088.stdout:146028888082 2024-02-19T18:58:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.506+0000 7fa462c3d700 1 -- 172.21.15.88:0/1225887764 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa44c06c680 msgr2=0x7fa44c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.506+0000 7fa462c3d700 1 --2- 172.21.15.88:0/1225887764 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa44c06c680 0x7fa44c06eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa45c197510 tx=0x7fa448005c80 comp rx=0 tx=0).stop 2024-02-19T18:58:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.506+0000 7fa462c3d700 1 -- 172.21.15.88:0/1225887764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c1004e0 msgr2=0x7fa45c195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.506+0000 7fa462c3d700 1 --2- 172.21.15.88:0/1225887764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c1004e0 0x7fa45c195ef0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa450006010 tx=0x7fa45000be20 comp rx=0 tx=0).stop 2024-02-19T18:58:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.506+0000 7fa462c3d700 1 -- 172.21.15.88:0/1225887764 shutdown_connections 2024-02-19T18:58:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.506+0000 7fa462c3d700 1 --2- 172.21.15.88:0/1225887764 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa44c06c680 0x7fa44c06eb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.506+0000 7fa462c3d700 1 --2- 172.21.15.88:0/1225887764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c1004e0 0x7fa45c195ef0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:01.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.506+0000 7fa462c3d700 1 --2- 172.21.15.88:0/1225887764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa45c100e80 0x7fa45c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:01.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.506+0000 7fa462c3d700 1 -- 172.21.15.88:0/1225887764 >> 172.21.15.88:0/1225887764 conn(0x7fa45c0fbda0 msgr2=0x7fa45c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:01.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.506+0000 7fa462c3d700 1 -- 172.21.15.88:0/1225887764 shutdown_connections 2024-02-19T18:58:01.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:01.506+0000 7fa462c3d700 1 -- 172.21.15.88:0/1225887764 wait complete. 2024-02-19T18:58:02.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:02 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1225887764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-19T18:58:02.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:02 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1225887764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-19T18:58:03.148 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:03 smithi088 ceph-mon[29272]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:03.173 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888079 got 146028888082 for osd.4 2024-02-19T18:58:03.173 DEBUG:teuthology.parallel:result is None 2024-02-19T18:58:03.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:03 smithi132 ceph-mon[37969]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.578+0000 7fce12c87700 1 -- 172.21.15.88:0/3440293583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce0c1004e0 msgr2=0x7fce0c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.578+0000 7fce12c87700 1 --2- 172.21.15.88:0/3440293583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce0c1004e0 0x7fce0c1008b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fce0400b0a0 tx=0x7fce0400b3b0 comp rx=0 tx=0).stop 2024-02-19T18:58:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.578+0000 7fce12c87700 1 -- 172.21.15.88:0/3440293583 shutdown_connections 2024-02-19T18:58:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.578+0000 7fce12c87700 1 --2- 172.21.15.88:0/3440293583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce0c100e80 0x7fce0c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.578+0000 7fce12c87700 1 --2- 172.21.15.88:0/3440293583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce0c1004e0 0x7fce0c1008b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:03.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.578+0000 7fce12c87700 1 -- 172.21.15.88:0/3440293583 >> 172.21.15.88:0/3440293583 conn(0x7fce0c0fbda0 msgr2=0x7fce0c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:03.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.578+0000 7fce12c87700 1 -- 172.21.15.88:0/3440293583 shutdown_connections 2024-02-19T18:58:03.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.578+0000 7fce12c87700 1 -- 172.21.15.88:0/3440293583 wait complete. 2024-02-19T18:58:03.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.579+0000 7fce12c87700 1 Processor -- start 2024-02-19T18:58:03.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.579+0000 7fce12c87700 1 -- start start 2024-02-19T18:58:03.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.579+0000 7fce12c87700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce0c1004e0 0x7fce0c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.579+0000 7fce12c87700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce0c100e80 0x7fce0c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.579+0000 7fce12c87700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c196bb0 con 0x7fce0c100e80 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.579+0000 7fce12c87700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce0c19a940 con 0x7fce0c1004e0 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.579+0000 7fce0bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce0c100e80 0x7fce0c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.579+0000 7fce0bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce0c100e80 0x7fce0c1964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37320/0 (socket says 172.21.15.88:37320) 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.579+0000 7fce0bfff700 1 -- 172.21.15.88:0/2016450706 learned_addr learned my addr 172.21.15.88:0/2016450706 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.579+0000 7fce10a23700 1 --2- 172.21.15.88:0/2016450706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce0c1004e0 0x7fce0c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.580+0000 7fce0bfff700 1 -- 172.21.15.88:0/2016450706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce0c1004e0 msgr2=0x7fce0c195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.580+0000 7fce0bfff700 1 --2- 172.21.15.88:0/2016450706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce0c1004e0 0x7fce0c195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.580+0000 7fce0bfff700 1 -- 172.21.15.88:0/2016450706 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce04009d00 con 0x7fce0c100e80 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.580+0000 7fce0bfff700 1 --2- 172.21.15.88:0/2016450706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce0c100e80 0x7fce0c1964d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00b480 tx=0x7fcdfc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:03.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.580+0000 7fce09ffb700 1 -- 172.21.15.88:0/2016450706 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdfc011800 con 0x7fce0c100e80 2024-02-19T18:58:03.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.580+0000 7fce09ffb700 1 -- 172.21.15.88:0/2016450706 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcdfc011e10 con 0x7fce0c100e80 2024-02-19T18:58:03.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.580+0000 7fce12c87700 1 -- 172.21.15.88:0/2016450706 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce0c19ac20 con 0x7fce0c100e80 2024-02-19T18:58:03.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.580+0000 7fce09ffb700 1 -- 172.21.15.88:0/2016450706 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdfc010ea0 con 0x7fce0c100e80 2024-02-19T18:58:03.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.580+0000 7fce12c87700 1 -- 172.21.15.88:0/2016450706 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce0c19b170 con 0x7fce0c100e80 2024-02-19T18:58:03.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.581+0000 7fce12c87700 1 -- 172.21.15.88:0/2016450706 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce0c1087f0 con 0x7fce0c100e80 2024-02-19T18:58:03.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.581+0000 7fce09ffb700 1 -- 172.21.15.88:0/2016450706 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fcdfc010380 con 0x7fce0c100e80 2024-02-19T18:58:03.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.582+0000 7fce09ffb700 1 --2- 172.21.15.88:0/2016450706 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcdf806c680 0x7fcdf806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:03.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.582+0000 7fce09ffb700 1 -- 172.21.15.88:0/2016450706 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fcdfc08b0b0 con 0x7fce0c100e80 2024-02-19T18:58:03.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.582+0000 7fce10a23700 1 --2- 172.21.15.88:0/2016450706 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcdf806c680 0x7fcdf806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:03.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.582+0000 7fce10a23700 1 --2- 172.21.15.88:0/2016450706 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcdf806c680 0x7fcdf806eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce04000f80 tx=0x7fce040060c0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:03.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.586+0000 7fce09ffb700 1 -- 172.21.15.88:0/2016450706 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcdfc055ad0 con 0x7fce0c100e80 2024-02-19T18:58:03.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.774+0000 7fce12c87700 1 -- 172.21.15.88:0/2016450706 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fce0c19b3f0 con 0x7fce0c100e80 2024-02-19T18:58:03.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.774+0000 7fce09ffb700 1 -- 172.21.15.88:0/2016450706 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcdfc0590f0 con 0x7fce0c100e80 2024-02-19T18:58:03.793 INFO:teuthology.orchestra.run.smithi088.stdout:167503724557 2024-02-19T18:58:03.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.776+0000 7fce12c87700 1 -- 172.21.15.88:0/2016450706 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcdf806c680 msgr2=0x7fcdf806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:03.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.776+0000 7fce12c87700 1 --2- 172.21.15.88:0/2016450706 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcdf806c680 0x7fcdf806eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fce04000f80 tx=0x7fce040060c0 comp rx=0 tx=0).stop 2024-02-19T18:58:03.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.776+0000 7fce12c87700 1 -- 172.21.15.88:0/2016450706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce0c100e80 msgr2=0x7fce0c1964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:03.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.776+0000 7fce12c87700 1 --2- 172.21.15.88:0/2016450706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce0c100e80 0x7fce0c1964d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00b480 tx=0x7fcdfc00b790 comp rx=0 tx=0).stop 2024-02-19T18:58:03.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.776+0000 7fce12c87700 1 -- 172.21.15.88:0/2016450706 shutdown_connections 2024-02-19T18:58:03.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.776+0000 7fce12c87700 1 --2- 172.21.15.88:0/2016450706 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcdf806c680 0x7fcdf806eb30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:03.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.776+0000 7fce12c87700 1 --2- 172.21.15.88:0/2016450706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce0c1004e0 0x7fce0c195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:03.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.776+0000 7fce12c87700 1 --2- 172.21.15.88:0/2016450706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce0c100e80 0x7fce0c1964d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:03.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.776+0000 7fce12c87700 1 -- 172.21.15.88:0/2016450706 >> 172.21.15.88:0/2016450706 conn(0x7fce0c0fbda0 msgr2=0x7fce0c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:03.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.777+0000 7fce12c87700 1 -- 172.21.15.88:0/2016450706 shutdown_connections 2024-02-19T18:58:03.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:03.777+0000 7fce12c87700 1 -- 172.21.15.88:0/2016450706 wait complete. 2024-02-19T18:58:04.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:04 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2016450706' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-19T18:58:04.403 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724554 got 167503724557 for osd.5 2024-02-19T18:58:04.403 DEBUG:teuthology.parallel:result is None 2024-02-19T18:58:04.403 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-19T18:58:04.404 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph pg dump --format=json 2024-02-19T18:58:04.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:04 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2016450706' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-19T18:58:04.612 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:05.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:05 smithi088 ceph-mon[29272]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:05.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:05 smithi132 ceph-mon[37969]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:06.270 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:06 smithi088 ceph-mon[29272]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:06.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.250+0000 7fabac7ae700 1 -- 172.21.15.88:0/2726743686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba4104130 msgr2=0x7faba4104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:06.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.250+0000 7fabac7ae700 1 --2- 172.21.15.88:0/2726743686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba4104130 0x7faba4104500 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fab94009910 tx=0x7fab94009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.250+0000 7fabac7ae700 1 -- 172.21.15.88:0/2726743686 shutdown_connections 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.250+0000 7fabac7ae700 1 --2- 172.21.15.88:0/2726743686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba40ffbd0 0x7faba4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.250+0000 7fabac7ae700 1 --2- 172.21.15.88:0/2726743686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba4104130 0x7faba4104500 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.250+0000 7fabac7ae700 1 -- 172.21.15.88:0/2726743686 >> 172.21.15.88:0/2726743686 conn(0x7faba4075130 msgr2=0x7faba4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.250+0000 7fabac7ae700 1 -- 172.21.15.88:0/2726743686 shutdown_connections 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.250+0000 7fabac7ae700 1 -- 172.21.15.88:0/2726743686 wait complete. 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.251+0000 7fabac7ae700 1 Processor -- start 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.251+0000 7fabac7ae700 1 -- start start 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.251+0000 7fabac7ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba40ffbd0 0x7faba4100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.251+0000 7fabac7ae700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba4104130 0x7faba4101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.251+0000 7fabac7ae700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba41018b0 con 0x7faba4104130 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.251+0000 7fabac7ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faba41019f0 con 0x7faba40ffbd0 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.251+0000 7faba9d49700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba4104130 0x7faba4101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.251+0000 7faba9d49700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba4104130 0x7faba4101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56600/0 (socket says 172.21.15.88:56600) 2024-02-19T18:58:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.251+0000 7faba9d49700 1 -- 172.21.15.88:0/2992782673 learned_addr learned my addr 172.21.15.88:0/2992782673 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.251+0000 7fabaa54a700 1 --2- 172.21.15.88:0/2992782673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba40ffbd0 0x7faba4100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.252+0000 7faba9d49700 1 -- 172.21.15.88:0/2992782673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba40ffbd0 msgr2=0x7faba4100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.252+0000 7faba9d49700 1 --2- 172.21.15.88:0/2992782673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba40ffbd0 0x7faba4100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.252+0000 7faba9d49700 1 -- 172.21.15.88:0/2992782673 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab940095f0 con 0x7faba4104130 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.252+0000 7faba9d49700 1 --2- 172.21.15.88:0/2992782673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba4104130 0x7faba4101370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faba000b4b0 tx=0x7faba000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.252+0000 7fab9b7fe700 1 -- 172.21.15.88:0/2992782673 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faba0011800 con 0x7faba4104130 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.252+0000 7fab9b7fe700 1 -- 172.21.15.88:0/2992782673 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faba0011e10 con 0x7faba4104130 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.252+0000 7fabac7ae700 1 -- 172.21.15.88:0/2992782673 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faba41a20a0 con 0x7faba4104130 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.252+0000 7fab9b7fe700 1 -- 172.21.15.88:0/2992782673 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faba000f2e0 con 0x7faba4104130 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.252+0000 7fabac7ae700 1 -- 172.21.15.88:0/2992782673 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faba41a24b0 con 0x7faba4104130 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.253+0000 7fabac7ae700 1 -- 172.21.15.88:0/2992782673 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faba41086a0 con 0x7faba4104130 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.253+0000 7fab9b7fe700 1 -- 172.21.15.88:0/2992782673 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7faba0010380 con 0x7faba4104130 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.254+0000 7fab9b7fe700 1 --2- 172.21.15.88:0/2992782673 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab9006c680 0x7fab9006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.254+0000 7fab9b7fe700 1 -- 172.21.15.88:0/2992782673 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7faba008aff0 con 0x7faba4104130 2024-02-19T18:58:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.254+0000 7fabaa54a700 1 --2- 172.21.15.88:0/2992782673 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab9006c680 0x7fab9006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.254+0000 7fabaa54a700 1 --2- 172.21.15.88:0/2992782673 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab9006c680 0x7fab9006eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab94006010 tx=0x7fab9401a040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:06.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.258+0000 7fab9b7fe700 1 -- 172.21.15.88:0/2992782673 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7faba0055a90 con 0x7faba4104130 2024-02-19T18:58:06.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.446+0000 7fabac7ae700 1 -- 172.21.15.88:0/2992782673 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faba402cca0 con 0x7fab9006c680 2024-02-19T18:58:06.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.448+0000 7fab9b7fe700 1 -- 172.21.15.88:0/2992782673 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28164 (secure 0 0 0) 0x7faba402cca0 con 0x7fab9006c680 2024-02-19T18:58:06.467 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:58:06.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.450+0000 7fabac7ae700 1 -- 172.21.15.88:0/2992782673 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab9006c680 msgr2=0x7fab9006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:06.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.450+0000 7fabac7ae700 1 --2- 172.21.15.88:0/2992782673 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab9006c680 0x7fab9006eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab94006010 tx=0x7fab9401a040 comp rx=0 tx=0).stop 2024-02-19T18:58:06.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.450+0000 7fabac7ae700 1 -- 172.21.15.88:0/2992782673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba4104130 msgr2=0x7faba4101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:06.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.450+0000 7fabac7ae700 1 --2- 172.21.15.88:0/2992782673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba4104130 0x7faba4101370 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faba000b4b0 tx=0x7faba000b7c0 comp rx=0 tx=0).stop 2024-02-19T18:58:06.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.450+0000 7fabac7ae700 1 -- 172.21.15.88:0/2992782673 shutdown_connections 2024-02-19T18:58:06.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.450+0000 7fabac7ae700 1 --2- 172.21.15.88:0/2992782673 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fab9006c680 0x7fab9006eb30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:06.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.450+0000 7fabac7ae700 1 --2- 172.21.15.88:0/2992782673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faba40ffbd0 0x7faba4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:06.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.450+0000 7fabac7ae700 1 --2- 172.21.15.88:0/2992782673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba4104130 0x7faba4101370 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:06.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.450+0000 7fabac7ae700 1 -- 172.21.15.88:0/2992782673 >> 172.21.15.88:0/2992782673 conn(0x7faba4075130 msgr2=0x7faba40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:06.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.450+0000 7fabac7ae700 1 -- 172.21.15.88:0/2992782673 shutdown_connections 2024-02-19T18:58:06.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:06.450+0000 7fabac7ae700 1 -- 172.21.15.88:0/2992782673 wait complete. 2024-02-19T18:58:06.470 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-02-19T18:58:06.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:06 smithi132 ceph-mon[37969]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:07.124 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-02-19T18:58:05.644168+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002046"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-02-19T18:57:15.000656+0000","last_change":"2024-02-19T18:56:48.338890+0000","last_active":"2024-02-19T18:57:15.000656+0000","last_peered":"2024-02-19T18:57:15.000656+0000","last_clean":"2024-02-19T18:57:15.000656+0000","last_became_active":"2024-02-19T18:56:48.338410+0000","last_became_peered":"2024-02-19T18:56:48.338410+0000","last_unstale":"2024-02-19T18:57:15.000656+0000","last_undegraded":"2024-02-19T18:57:15.000656+0000","last_fullsized":"2024-02-19T18:57:15.000656+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T18:55:57.959017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T18:55:57.959017+0000","last_clean_scrub_stamp":"2024-02-19T18:55:57.959017+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:27:48.058615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 18:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.433}]},{"osd":1,"last update":"Mon Feb 19 18:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.41699999999999998}]},{"osd":2,"last update":"Mon Feb 19 18:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.5},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Mon Feb 19 18:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.37},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":729714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.49299999999999999}]},{"osd":1,"last update":"Mon Feb 19 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.505},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.51400000000000001}]},{"osd":2,"last update":"Mon Feb 19 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.46899999999999997}]},{"osd":4,"last update":"Mon Feb 19 18:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.436},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.54900000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 18:57:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.34100000000000003,"15min":0.34499999999999997},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.46000000000000002,"5min":0.496,"15min":0.496},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.47599999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.26900000000000002}]},{"osd":1,"last update":"Mon Feb 19 18:57:58 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.44400000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.30099999999999999}]},{"osd":3,"last update":"Mon Feb 19 18:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58099999999999996}]},{"osd":4,"last update":"Mon Feb 19 18:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 19 18:57:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.32000000000000001,"15min":0.309},"min":{"1min":0.26400000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.25600000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.34599999999999997}]},{"osd":2,"last update":"Mon Feb 19 18:57:01 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.308},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.35899999999999999}]},{"osd":3,"last update":"Mon Feb 19 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.61499999999999999}]},{"osd":4,"last update":"Mon Feb 19 18:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 18:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.221,"5min":0.219,"15min":0.219},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33300000000000002,"15min":0.32900000000000001},"min":{"1min":0.25800000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.42699999999999999}]},{"osd":2,"last update":"Mon Feb 19 18:58:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.36099999999999999,"15min":0.36199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51200000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36799999999999999,"15min":0.36899999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.46899999999999997}]},{"osd":3,"last update":"Mon Feb 19 18:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.67000000000000004}]},{"osd":4,"last update":"Mon Feb 19 18:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-19T18:58:07.125 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph pg dump --format=json 2024-02-19T18:58:07.329 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:08.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:07 smithi132 ceph-mon[37969]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T18:58:08.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:07 smithi088 ceph-mon[29272]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T18:58:08.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.961+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1570322245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b01038b0 msgr2=0x7f36b0103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:08.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.961+0000 7f36b5ac1700 1 --2- 172.21.15.88:0/1570322245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b01038b0 0x7f36b0103c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f36a000aa20 tx=0x7f36a000ad30 comp rx=0 tx=0).stop 2024-02-19T18:58:08.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.961+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1570322245 shutdown_connections 2024-02-19T18:58:08.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.961+0000 7f36b5ac1700 1 --2- 172.21.15.88:0/1570322245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36b00ff350 0x7f36b00ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:08.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.961+0000 7f36b5ac1700 1 --2- 172.21.15.88:0/1570322245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b01038b0 0x7f36b0103c80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:08.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.961+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1570322245 >> 172.21.15.88:0/1570322245 conn(0x7f36b00748b0 msgr2=0x7f36b0074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:08.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.961+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1570322245 shutdown_connections 2024-02-19T18:58:08.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.961+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1570322245 wait complete. 2024-02-19T18:58:08.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.962+0000 7f36b5ac1700 1 Processor -- start 2024-02-19T18:58:08.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.962+0000 7f36b5ac1700 1 -- start start 2024-02-19T18:58:08.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.962+0000 7f36b5ac1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36b00ff350 0x7f36b0199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:08.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.962+0000 7f36b5ac1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b01038b0 0x7f36b0199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:08.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.962+0000 7f36b5ac1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0193a60 con 0x7f36b01038b0 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.962+0000 7f36b5ac1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0193bd0 con 0x7f36b00ff350 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.962+0000 7f36aeffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b01038b0 0x7f36b0199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.962+0000 7f36aeffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b01038b0 0x7f36b0199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56618/0 (socket says 172.21.15.88:56618) 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.962+0000 7f36aeffd700 1 -- 172.21.15.88:0/1227489557 learned_addr learned my addr 172.21.15.88:0/1227489557 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.963+0000 7f36af7fe700 1 --2- 172.21.15.88:0/1227489557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36b00ff350 0x7f36b0199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.963+0000 7f36aeffd700 1 -- 172.21.15.88:0/1227489557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36b00ff350 msgr2=0x7f36b0199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.963+0000 7f36aeffd700 1 --2- 172.21.15.88:0/1227489557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36b00ff350 0x7f36b0199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.963+0000 7f36aeffd700 1 -- 172.21.15.88:0/1227489557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a000a700 con 0x7f36b01038b0 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.963+0000 7f36aeffd700 1 --2- 172.21.15.88:0/1227489557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b01038b0 0x7f36b0199fb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009970 tx=0x7f36a4009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.963+0000 7f36acff9700 1 -- 172.21.15.88:0/1227489557 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a4012070 con 0x7f36b01038b0 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.963+0000 7f36acff9700 1 -- 172.21.15.88:0/1227489557 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f36a4007ae0 con 0x7f36b01038b0 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.963+0000 7f36acff9700 1 -- 172.21.15.88:0/1227489557 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a4017c20 con 0x7f36b01038b0 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.963+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1227489557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b0193eb0 con 0x7f36b01038b0 2024-02-19T18:58:08.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.963+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1227489557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b0194400 con 0x7f36b01038b0 2024-02-19T18:58:08.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.964+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1227489557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b0106e20 con 0x7f36b01038b0 2024-02-19T18:58:08.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.965+0000 7f36acff9700 1 -- 172.21.15.88:0/1227489557 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f36a4007c50 con 0x7f36b01038b0 2024-02-19T18:58:08.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.965+0000 7f36acff9700 1 --2- 172.21.15.88:0/1227489557 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f369806c680 0x7f369806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:08.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.965+0000 7f36acff9700 1 -- 172.21.15.88:0/1227489557 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f36a408ca20 con 0x7f36b01038b0 2024-02-19T18:58:08.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.965+0000 7f36af7fe700 1 --2- 172.21.15.88:0/1227489557 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f369806c680 0x7f369806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:08.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.966+0000 7f36af7fe700 1 --2- 172.21.15.88:0/1227489557 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f369806c680 0x7f369806eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c00 tx=0x7f36a0005de0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:08.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:08.970+0000 7f36acff9700 1 -- 172.21.15.88:0/1227489557 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f36a4051720 con 0x7f36b01038b0 2024-02-19T18:58:09.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:08 smithi088 ceph-mon[29272]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:09.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.160+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1227489557 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36b0195000 con 0x7f369806c680 2024-02-19T18:58:09.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.162+0000 7f36acff9700 1 -- 172.21.15.88:0/1227489557 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28134 (secure 0 0 0) 0x7f36b0195000 con 0x7f369806c680 2024-02-19T18:58:09.180 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:58:09.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.164+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1227489557 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f369806c680 msgr2=0x7f369806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:09.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.164+0000 7f36b5ac1700 1 --2- 172.21.15.88:0/1227489557 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f369806c680 0x7f369806eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f36a0000c00 tx=0x7f36a0005de0 comp rx=0 tx=0).stop 2024-02-19T18:58:09.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.164+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1227489557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b01038b0 msgr2=0x7f36b0199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:09.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.164+0000 7f36b5ac1700 1 --2- 172.21.15.88:0/1227489557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b01038b0 0x7f36b0199fb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f36a4009970 tx=0x7f36a4009c80 comp rx=0 tx=0).stop 2024-02-19T18:58:09.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.164+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1227489557 shutdown_connections 2024-02-19T18:58:09.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.164+0000 7f36b5ac1700 1 --2- 172.21.15.88:0/1227489557 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f369806c680 0x7f369806eb30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:09.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.164+0000 7f36b5ac1700 1 --2- 172.21.15.88:0/1227489557 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36b00ff350 0x7f36b0199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:09.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.164+0000 7f36b5ac1700 1 --2- 172.21.15.88:0/1227489557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b01038b0 0x7f36b0199fb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:09.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.164+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1227489557 >> 172.21.15.88:0/1227489557 conn(0x7f36b00748b0 msgr2=0x7f36b00fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:09.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.164+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1227489557 shutdown_connections 2024-02-19T18:58:09.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:09.164+0000 7f36b5ac1700 1 -- 172.21.15.88:0/1227489557 wait complete. 2024-02-19T18:58:09.183 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-02-19T18:58:09.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:08 smithi132 ceph-mon[37969]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:09.798 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-02-19T18:58:07.644560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165584,"kb_used_data":3872,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305328,"statfs":{"total":575970213888,"available":575800655872,"internally_reserved":0,"allocated":3964928,"data_stored":2608835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002066"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-02-19T18:57:15.000656+0000","last_change":"2024-02-19T18:56:48.338890+0000","last_active":"2024-02-19T18:57:15.000656+0000","last_peered":"2024-02-19T18:57:15.000656+0000","last_clean":"2024-02-19T18:57:15.000656+0000","last_became_active":"2024-02-19T18:56:48.338410+0000","last_became_peered":"2024-02-19T18:56:48.338410+0000","last_unstale":"2024-02-19T18:57:15.000656+0000","last_undegraded":"2024-02-19T18:57:15.000656+0000","last_fullsized":"2024-02-19T18:57:15.000656+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-19T18:55:57.959017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-19T18:55:57.959017+0000","last_clean_scrub_stamp":"2024-02-19T18:55:57.959017+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-20T19:27:48.058615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724558,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]}]},{"osd":4,"up_from":34,"seq":146028888083,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 18:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.433}]},{"osd":1,"last update":"Mon Feb 19 18:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.41699999999999998}]},{"osd":2,"last update":"Mon Feb 19 18:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.5},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.46100000000000002}]},{"osd":3,"last update":"Mon Feb 19 18:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.37},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.443}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":954368,"data_stored":729714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.55000000000000004}]},{"osd":1,"last update":"Mon Feb 19 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.623}]},{"osd":2,"last update":"Mon Feb 19 18:57:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.60799999999999998}]},{"osd":4,"last update":"Mon Feb 19 18:57:48 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"last":0.48099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":2,"up_from":19,"seq":81604378653,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 18:57:58 2024","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.34100000000000003,"15min":0.34499999999999997},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.46000000000000002,"5min":0.496,"15min":0.496},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.34399999999999997,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.47599999999999998,"5min":0.53300000000000003,"15min":0.53300000000000003},"last":0.26900000000000002}]},{"osd":1,"last update":"Mon Feb 19 18:57:58 2024","interfaces":[{"interface":"back","average":{"1min":0.34699999999999998,"5min":0.36099999999999999,"15min":0.36299999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36699999999999999,"15min":0.371},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.44400000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.30099999999999999}]},{"osd":3,"last update":"Mon Feb 19 18:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.58099999999999996}]},{"osd":4,"last update":"Mon Feb 19 18:57:51 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55600000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705704,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Feb 19 18:57:32 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.32000000000000001,"15min":0.309},"min":{"1min":0.26400000000000001,"5min":0.16500000000000001,"15min":0.16500000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.25600000000000001,"5min":0.19400000000000001,"15min":0.19400000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.41999999999999998}]},{"osd":2,"last update":"Mon Feb 19 18:58:04 2024","interfaces":[{"interface":"back","average":{"1min":0.35899999999999999,"5min":0.376,"15min":0.379},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.39300000000000002,"15min":0.39700000000000002},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.52500000000000002,"5min":0.54800000000000004,"15min":0.54800000000000004},"last":0.433}]},{"osd":3,"last update":"Mon Feb 19 18:57:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.69099999999999995}]},{"osd":4,"last update":"Mon Feb 19 18:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.624},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Feb 19 18:57:29 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.221,"5min":0.219,"15min":0.219},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33300000000000002,"15min":0.32900000000000001},"min":{"1min":0.25800000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.42699999999999999}]},{"osd":2,"last update":"Mon Feb 19 18:58:00 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.36099999999999999,"15min":0.36199999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.51200000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.36099999999999999,"5min":0.36799999999999999,"15min":0.36899999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.46899999999999997}]},{"osd":3,"last update":"Mon Feb 19 18:57:24 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.67000000000000004}]},{"osd":4,"last update":"Mon Feb 19 18:57:50 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.77600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.624}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-19T18:58:09.799 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-19T18:58:09.800 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-19T18:58:09.800 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-19T18:58:09.800 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph health --format=json 2024-02-19T18:58:10.011 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:10.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:09 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:58:10.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:09 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:58:11.074 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:10 smithi088 ceph-mon[29272]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T18:58:11.074 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:10 smithi088 ceph-mon[29272]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:11.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:10 smithi132 ceph-mon[37969]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-19T18:58:11.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:10 smithi132 ceph-mon[37969]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.608+0000 7f385bf16700 1 -- 172.21.15.88:0/743424160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38540687a0 msgr2=0x7f3854068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.608+0000 7f385bf16700 1 --2- 172.21.15.88:0/743424160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38540687a0 0x7f3854068b70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f384c00b0a0 tx=0x7f384c00b3b0 comp rx=0 tx=0).stop 2024-02-19T18:58:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.608+0000 7f385bf16700 1 -- 172.21.15.88:0/743424160 shutdown_connections 2024-02-19T18:58:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.608+0000 7f385bf16700 1 --2- 172.21.15.88:0/743424160 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3854069140 0x7f3854104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.608+0000 7f385bf16700 1 --2- 172.21.15.88:0/743424160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38540687a0 0x7f3854068b70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.608+0000 7f385bf16700 1 -- 172.21.15.88:0/743424160 >> 172.21.15.88:0/743424160 conn(0x7f3854075130 msgr2=0x7f3854075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.608+0000 7f385bf16700 1 -- 172.21.15.88:0/743424160 shutdown_connections 2024-02-19T18:58:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.608+0000 7f385bf16700 1 -- 172.21.15.88:0/743424160 wait complete. 2024-02-19T18:58:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f385bf16700 1 Processor -- start 2024-02-19T18:58:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f385bf16700 1 -- start start 2024-02-19T18:58:11.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f385bf16700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38540687a0 0x7f38540ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f385bf16700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3854069140 0x7f38541001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f385bf16700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3854100720 con 0x7f3854069140 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f385bf16700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3854100860 con 0x7f38540687a0 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f38594b1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3854069140 0x7f38541001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f38594b1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3854069140 0x7f38541001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56644/0 (socket says 172.21.15.88:56644) 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f38594b1700 1 -- 172.21.15.88:0/405373212 learned_addr learned my addr 172.21.15.88:0/405373212 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f3859cb2700 1 --2- 172.21.15.88:0/405373212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38540687a0 0x7f38540ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f38594b1700 1 -- 172.21.15.88:0/405373212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38540687a0 msgr2=0x7f38540ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f38594b1700 1 --2- 172.21.15.88:0/405373212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38540687a0 0x7f38540ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f38594b1700 1 -- 172.21.15.88:0/405373212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384c009d00 con 0x7f3854069140 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.609+0000 7f38594b1700 1 --2- 172.21.15.88:0/405373212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3854069140 0x7f38541001e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f384400b480 tx=0x7f384400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.610+0000 7f384affd700 1 -- 172.21.15.88:0/405373212 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38440118c0 con 0x7f3854069140 2024-02-19T18:58:11.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.610+0000 7f385bf16700 1 -- 172.21.15.88:0/405373212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38541a2130 con 0x7f3854069140 2024-02-19T18:58:11.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.610+0000 7f384affd700 1 -- 172.21.15.88:0/405373212 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3844010380 con 0x7f3854069140 2024-02-19T18:58:11.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.610+0000 7f384affd700 1 -- 172.21.15.88:0/405373212 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f384400f4f0 con 0x7f3854069140 2024-02-19T18:58:11.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.610+0000 7f385bf16700 1 -- 172.21.15.88:0/405373212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38541a2540 con 0x7f3854069140 2024-02-19T18:58:11.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.611+0000 7f385bf16700 1 -- 172.21.15.88:0/405373212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3854108770 con 0x7f3854069140 2024-02-19T18:58:11.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.611+0000 7f384affd700 1 -- 172.21.15.88:0/405373212 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f38440104f0 con 0x7f3854069140 2024-02-19T18:58:11.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.612+0000 7f384affd700 1 --2- 172.21.15.88:0/405373212 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f384006c680 0x7f384006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:11.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.612+0000 7f384affd700 1 -- 172.21.15.88:0/405373212 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f384408b180 con 0x7f3854069140 2024-02-19T18:58:11.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.612+0000 7f3859cb2700 1 --2- 172.21.15.88:0/405373212 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f384006c680 0x7f384006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:11.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.612+0000 7f3859cb2700 1 --2- 172.21.15.88:0/405373212 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f384006c680 0x7f384006eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f384c00b830 tx=0x7f384c0097c0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:11.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.616+0000 7f384affd700 1 -- 172.21.15.88:0/405373212 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3844055ba0 con 0x7f3854069140 2024-02-19T18:58:11.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.850+0000 7f385bf16700 1 -- 172.21.15.88:0/405373212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f385404f2a0 con 0x7f3854069140 2024-02-19T18:58:11.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.851+0000 7f384affd700 1 -- 172.21.15.88:0/405373212 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f38440591c0 con 0x7f3854069140 2024-02-19T18:58:11.869 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:58:11.869 INFO:teuthology.orchestra.run.smithi088.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-19T18:58:11.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.852+0000 7f385bf16700 1 -- 172.21.15.88:0/405373212 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f384006c680 msgr2=0x7f384006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:11.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.852+0000 7f385bf16700 1 --2- 172.21.15.88:0/405373212 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f384006c680 0x7f384006eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f384c00b830 tx=0x7f384c0097c0 comp rx=0 tx=0).stop 2024-02-19T18:58:11.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.852+0000 7f385bf16700 1 -- 172.21.15.88:0/405373212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3854069140 msgr2=0x7f38541001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:11.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.852+0000 7f385bf16700 1 --2- 172.21.15.88:0/405373212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3854069140 0x7f38541001e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f384400b480 tx=0x7f384400b840 comp rx=0 tx=0).stop 2024-02-19T18:58:11.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.853+0000 7f385bf16700 1 -- 172.21.15.88:0/405373212 shutdown_connections 2024-02-19T18:58:11.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.853+0000 7f385bf16700 1 --2- 172.21.15.88:0/405373212 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f384006c680 0x7f384006eb30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:11.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.853+0000 7f385bf16700 1 --2- 172.21.15.88:0/405373212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38540687a0 0x7f38540ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:11.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.853+0000 7f385bf16700 1 --2- 172.21.15.88:0/405373212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3854069140 0x7f38541001e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:11.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.853+0000 7f385bf16700 1 -- 172.21.15.88:0/405373212 >> 172.21.15.88:0/405373212 conn(0x7f3854075130 msgr2=0x7f38540fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:11.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.853+0000 7f385bf16700 1 -- 172.21.15.88:0/405373212 shutdown_connections 2024-02-19T18:58:11.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:11.853+0000 7f385bf16700 1 -- 172.21.15.88:0/405373212 wait complete. 2024-02-19T18:58:12.171 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:11 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/405373212' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-19T18:58:12.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:11 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/405373212' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-19T18:58:12.502 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-19T18:58:12.502 INFO:tasks.cephadm:Setup complete, yielding 2024-02-19T18:58:12.503 INFO:teuthology.run_tasks:Running task print... 2024-02-19T18:58:12.511 INFO:teuthology.task.print:**** done end installing v18.2.0 cephadm ... 2024-02-19T18:58:12.512 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-19T18:58:12.547 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T18:58:12.548 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-02-19T18:58:12.757 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:13.183 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:12 smithi088 ceph-mon[29272]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:13.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:12 smithi132 ceph-mon[37969]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:14.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.276+0000 7f6250adf700 1 -- 172.21.15.88:0/1648521009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f624c1006c0 msgr2=0x7f624c100ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:14.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.276+0000 7f6250adf700 1 --2- 172.21.15.88:0/1648521009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f624c1006c0 0x7f624c100ad0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f623c009910 tx=0x7f623c009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:14.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.276+0000 7f6250adf700 1 -- 172.21.15.88:0/1648521009 shutdown_connections 2024-02-19T18:58:14.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.276+0000 7f6250adf700 1 --2- 172.21.15.88:0/1648521009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f624c0ff290 0x7f624c0ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:14.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.276+0000 7f6250adf700 1 --2- 172.21.15.88:0/1648521009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f624c1006c0 0x7f624c100ad0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:14.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.276+0000 7f6250adf700 1 -- 172.21.15.88:0/1648521009 >> 172.21.15.88:0/1648521009 conn(0x7f624c0f9d70 msgr2=0x7f624c0fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:14.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.276+0000 7f6250adf700 1 -- 172.21.15.88:0/1648521009 shutdown_connections 2024-02-19T18:58:14.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.276+0000 7f6250adf700 1 -- 172.21.15.88:0/1648521009 wait complete. 2024-02-19T18:58:14.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.277+0000 7f6250adf700 1 Processor -- start 2024-02-19T18:58:14.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.277+0000 7f6250adf700 1 -- start start 2024-02-19T18:58:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.277+0000 7f6250adf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f624c0ff290 0x7f624c194890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f6250adf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f624c1006c0 0x7f624c194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f6250adf700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624c1953f0 con 0x7f624c0ff290 2024-02-19T18:58:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f6250adf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624c195530 con 0x7f624c1006c0 2024-02-19T18:58:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f624a59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f624c0ff290 0x7f624c194890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:14.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f624a59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f624c0ff290 0x7f624c194890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56658/0 (socket says 172.21.15.88:56658) 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f624a59c700 1 -- 172.21.15.88:0/4045666574 learned_addr learned my addr 172.21.15.88:0/4045666574 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f6249d9b700 1 --2- 172.21.15.88:0/4045666574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f624c1006c0 0x7f624c194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f624a59c700 1 -- 172.21.15.88:0/4045666574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f624c1006c0 msgr2=0x7f624c194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f624a59c700 1 --2- 172.21.15.88:0/4045666574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f624c1006c0 0x7f624c194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f624a59c700 1 -- 172.21.15.88:0/4045666574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c0095f0 con 0x7f624c0ff290 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f624a59c700 1 --2- 172.21.15.88:0/4045666574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f624c0ff290 0x7f624c194890 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f623c006010 tx=0x7f623c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f623b7fe700 1 -- 172.21.15.88:0/4045666574 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f623c01d070 con 0x7f624c0ff290 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f6250adf700 1 -- 172.21.15.88:0/4045666574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624c199f80 con 0x7f624c0ff290 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f623b7fe700 1 -- 172.21.15.88:0/4045666574 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f623c00f460 con 0x7f624c0ff290 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f623b7fe700 1 -- 172.21.15.88:0/4045666574 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f623c021640 con 0x7f624c0ff290 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.278+0000 7f6250adf700 1 -- 172.21.15.88:0/4045666574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624c19a470 con 0x7f624c0ff290 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.279+0000 7f6250adf700 1 -- 172.21.15.88:0/4045666574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624c107cb0 con 0x7f624c0ff290 2024-02-19T18:58:14.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.280+0000 7f623b7fe700 1 -- 172.21.15.88:0/4045666574 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f623c00fab0 con 0x7f624c0ff290 2024-02-19T18:58:14.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.280+0000 7f623b7fe700 1 --2- 172.21.15.88:0/4045666574 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f623406c680 0x7f623406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:14.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.280+0000 7f6249d9b700 1 --2- 172.21.15.88:0/4045666574 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f623406c680 0x7f623406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:14.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.280+0000 7f623b7fe700 1 -- 172.21.15.88:0/4045666574 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f623c08c970 con 0x7f624c0ff290 2024-02-19T18:58:14.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.280+0000 7f6249d9b700 1 --2- 172.21.15.88:0/4045666574 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f623406c680 0x7f623406eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6240009a10 tx=0x7f6240009190 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:14.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.284+0000 7f623b7fe700 1 -- 172.21.15.88:0/4045666574 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f623c05ae20 con 0x7f624c0ff290 2024-02-19T18:58:14.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.475+0000 7f6250adf700 1 -- 172.21.15.88:0/4045666574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f624c19ae90 con 0x7f624c0ff290 2024-02-19T18:58:14.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.479+0000 7f623b7fe700 1 -- 172.21.15.88:0/4045666574 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f623c05a9b0 con 0x7f624c0ff290 2024-02-19T18:58:14.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.481+0000 7f6250adf700 1 -- 172.21.15.88:0/4045666574 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f623406c680 msgr2=0x7f623406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:14.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.481+0000 7f6250adf700 1 --2- 172.21.15.88:0/4045666574 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f623406c680 0x7f623406eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6240009a10 tx=0x7f6240009190 comp rx=0 tx=0).stop 2024-02-19T18:58:14.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.481+0000 7f6250adf700 1 -- 172.21.15.88:0/4045666574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f624c0ff290 msgr2=0x7f624c194890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:14.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.481+0000 7f6250adf700 1 --2- 172.21.15.88:0/4045666574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f624c0ff290 0x7f624c194890 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f623c006010 tx=0x7f623c00bd70 comp rx=0 tx=0).stop 2024-02-19T18:58:14.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.481+0000 7f6250adf700 1 -- 172.21.15.88:0/4045666574 shutdown_connections 2024-02-19T18:58:14.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.481+0000 7f6250adf700 1 --2- 172.21.15.88:0/4045666574 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f623406c680 0x7f623406eb30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:14.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.481+0000 7f6250adf700 1 --2- 172.21.15.88:0/4045666574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f624c0ff290 0x7f624c194890 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:14.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.481+0000 7f6250adf700 1 --2- 172.21.15.88:0/4045666574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f624c1006c0 0x7f624c194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:14.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.482+0000 7f6250adf700 1 -- 172.21.15.88:0/4045666574 >> 172.21.15.88:0/4045666574 conn(0x7f624c0f9d70 msgr2=0x7f624c1034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:14.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.482+0000 7f6250adf700 1 -- 172.21.15.88:0/4045666574 shutdown_connections 2024-02-19T18:58:14.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:14.482+0000 7f6250adf700 1 -- 172.21.15.88:0/4045666574 wait complete. 2024-02-19T18:58:15.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:14 smithi088 ceph-mon[29272]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:15.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:14 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/4045666574' entity='client.admin' 2024-02-19T18:58:15.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:14 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:58:15.199 INFO:teuthology.run_tasks:Running task print... 2024-02-19T18:58:15.207 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-02-19T18:58:15.208 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-19T18:58:15.226 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T18:58:15.226 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph orch status' 2024-02-19T18:58:15.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:14 smithi132 ceph-mon[37969]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:15.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:14 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/4045666574' entity='client.admin' 2024-02-19T18:58:15.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:14 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:58:15.439 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:16.243 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:16.243 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:58:16.243 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:15 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:16.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:16.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:58:16.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:15 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:17.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.980+0000 7f3320c1d700 1 -- 172.21.15.88:0/2559400012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f331c0ff000 msgr2=0x7f331c0ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:17.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.980+0000 7f3320c1d700 1 --2- 172.21.15.88:0/2559400012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f331c0ff000 0x7f331c0ff410 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f330c009910 tx=0x7f330c009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:17.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.980+0000 7f3320c1d700 1 -- 172.21.15.88:0/2559400012 shutdown_connections 2024-02-19T18:58:17.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.980+0000 7f3320c1d700 1 --2- 172.21.15.88:0/2559400012 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c1002a0 0x7f331c100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:17.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.980+0000 7f3320c1d700 1 --2- 172.21.15.88:0/2559400012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f331c0ff000 0x7f331c0ff410 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:17.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.980+0000 7f3320c1d700 1 -- 172.21.15.88:0/2559400012 >> 172.21.15.88:0/2559400012 conn(0x7f331c0fa610 msgr2=0x7f331c0fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:17.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.980+0000 7f3320c1d700 1 -- 172.21.15.88:0/2559400012 shutdown_connections 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.980+0000 7f3320c1d700 1 -- 172.21.15.88:0/2559400012 wait complete. 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f3320c1d700 1 Processor -- start 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f3320c1d700 1 -- start start 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f3320c1d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f331c0ff000 0x7f331c1958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f3320c1d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c1002a0 0x7f331c195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f3320c1d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c196420 con 0x7f331c0ff000 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f3320c1d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c196560 con 0x7f331c1002a0 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f331a59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f331c0ff000 0x7f331c1958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f331a59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f331c0ff000 0x7f331c1958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35998/0 (socket says 172.21.15.88:35998) 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f331a59c700 1 -- 172.21.15.88:0/1632899918 learned_addr learned my addr 172.21.15.88:0/1632899918 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f3319d9b700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c1002a0 0x7f331c195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f331a59c700 1 -- 172.21.15.88:0/1632899918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c1002a0 msgr2=0x7f331c195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f331a59c700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c1002a0 0x7f331c195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:17.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f331a59c700 1 -- 172.21.15.88:0/1632899918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c0095f0 con 0x7f331c0ff000 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f3319d9b700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c1002a0 0x7f331c195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.981+0000 7f331a59c700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f331c0ff000 0x7f331c1958c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f330c006010 tx=0x7f330c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.982+0000 7f330b7fe700 1 -- 172.21.15.88:0/1632899918 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f330c01d070 con 0x7f331c0ff000 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.982+0000 7f330b7fe700 1 -- 172.21.15.88:0/1632899918 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f330c00f460 con 0x7f331c0ff000 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.982+0000 7f3320c1d700 1 -- 172.21.15.88:0/1632899918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331c19afb0 con 0x7f331c0ff000 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.982+0000 7f330b7fe700 1 -- 172.21.15.88:0/1632899918 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f330c017630 con 0x7f331c0ff000 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.982+0000 7f3320c1d700 1 -- 172.21.15.88:0/1632899918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331c19b420 con 0x7f331c0ff000 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.983+0000 7f3320c1d700 1 -- 172.21.15.88:0/1632899918 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331c107ae0 con 0x7f331c0ff000 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.983+0000 7f330b7fe700 1 -- 172.21.15.88:0/1632899918 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f330c00f5d0 con 0x7f331c0ff000 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.983+0000 7f330b7fe700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330406c680 0x7f330406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.984+0000 7f330b7fe700 1 -- 172.21.15.88:0/1632899918 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f330c08ca70 con 0x7f331c0ff000 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.984+0000 7f3319d9b700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330406c680 0x7f330406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:17.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.984+0000 7f3319d9b700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330406c680 0x7f330406eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3310005950 tx=0x7f33100058e0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:17.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:16.988+0000 7f330b7fe700 1 -- 172.21.15.88:0/1632899918 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f330c05af20 con 0x7f331c0ff000 2024-02-19T18:58:17.097 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:16 smithi088 ceph-mon[29272]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:17.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.197+0000 7f3320c1d700 1 -- 172.21.15.88:0/1632899918 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f331c103bb0 con 0x7f330406c680 2024-02-19T18:58:17.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.198+0000 7f330b7fe700 1 -- 172.21.15.88:0/1632899918 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f331c103bb0 con 0x7f330406c680 2024-02-19T18:58:17.216 INFO:teuthology.orchestra.run.smithi088.stdout:Backend: cephadm 2024-02-19T18:58:17.216 INFO:teuthology.orchestra.run.smithi088.stdout:Available: Yes 2024-02-19T18:58:17.216 INFO:teuthology.orchestra.run.smithi088.stdout:Paused: No 2024-02-19T18:58:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.200+0000 7f3320c1d700 1 -- 172.21.15.88:0/1632899918 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330406c680 msgr2=0x7f330406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.200+0000 7f3320c1d700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330406c680 0x7f330406eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3310005950 tx=0x7f33100058e0 comp rx=0 tx=0).stop 2024-02-19T18:58:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.200+0000 7f3320c1d700 1 -- 172.21.15.88:0/1632899918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f331c0ff000 msgr2=0x7f331c1958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.200+0000 7f3320c1d700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f331c0ff000 0x7f331c1958c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f330c006010 tx=0x7f330c00be20 comp rx=0 tx=0).stop 2024-02-19T18:58:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.200+0000 7f3320c1d700 1 -- 172.21.15.88:0/1632899918 shutdown_connections 2024-02-19T18:58:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.200+0000 7f3320c1d700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f330406c680 0x7f330406eb30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.200+0000 7f3320c1d700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f331c0ff000 0x7f331c1958c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:17.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.200+0000 7f3320c1d700 1 --2- 172.21.15.88:0/1632899918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f331c1002a0 0x7f331c195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:17.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.200+0000 7f3320c1d700 1 -- 172.21.15.88:0/1632899918 >> 172.21.15.88:0/1632899918 conn(0x7f331c0fa610 msgr2=0x7f331c0fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:17.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.200+0000 7f3320c1d700 1 -- 172.21.15.88:0/1632899918 shutdown_connections 2024-02-19T18:58:17.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:17.200+0000 7f3320c1d700 1 -- 172.21.15.88:0/1632899918 wait complete. 2024-02-19T18:58:17.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:16 smithi132 ceph-mon[37969]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:17.906 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-19T18:58:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:19.245 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:18 smithi088 ceph-mon[29272]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:19.245 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:18 smithi088 ceph-mon[29272]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:19.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:18 smithi132 ceph-mon[37969]: from='client.14466 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:19.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:18 smithi132 ceph-mon[37969]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.728+0000 7fe60e007700 1 -- 172.21.15.88:0/744805380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6081003f0 msgr2=0x7fe608100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.728+0000 7fe60e007700 1 --2- 172.21.15.88:0/744805380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6081003f0 0x7fe608100860 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc009910 tx=0x7fe5fc009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.728+0000 7fe60e007700 1 -- 172.21.15.88:0/744805380 shutdown_connections 2024-02-19T18:58:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.728+0000 7fe60e007700 1 --2- 172.21.15.88:0/744805380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6081003f0 0x7fe608100860 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.728+0000 7fe60e007700 1 --2- 172.21.15.88:0/744805380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6080ff150 0x7fe6080ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.728+0000 7fe60e007700 1 -- 172.21.15.88:0/744805380 >> 172.21.15.88:0/744805380 conn(0x7fe6080fa7a0 msgr2=0x7fe6080fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.728+0000 7fe60e007700 1 -- 172.21.15.88:0/744805380 shutdown_connections 2024-02-19T18:58:19.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.728+0000 7fe60e007700 1 -- 172.21.15.88:0/744805380 wait complete. 2024-02-19T18:58:19.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe60e007700 1 Processor -- start 2024-02-19T18:58:19.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe60e007700 1 -- start start 2024-02-19T18:58:19.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe60e007700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6080ff150 0x7fe608069950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe60e007700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6081003f0 0x7fe60819aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe60e007700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60819b0c0 con 0x7fe6080ff150 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe60e007700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60819b230 con 0x7fe6081003f0 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe6077fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6080ff150 0x7fe608069950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe6077fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6080ff150 0x7fe608069950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36008/0 (socket says 172.21.15.88:36008) 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe6077fe700 1 -- 172.21.15.88:0/3374643720 learned_addr learned my addr 172.21.15.88:0/3374643720 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe606ffd700 1 --2- 172.21.15.88:0/3374643720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6081003f0 0x7fe60819aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe6077fe700 1 -- 172.21.15.88:0/3374643720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6081003f0 msgr2=0x7fe60819aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe6077fe700 1 --2- 172.21.15.88:0/3374643720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6081003f0 0x7fe60819aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.729+0000 7fe6077fe700 1 -- 172.21.15.88:0/3374643720 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc0095f0 con 0x7fe6080ff150 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.730+0000 7fe6077fe700 1 --2- 172.21.15.88:0/3374643720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6080ff150 0x7fe608069950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800d8d0 tx=0x7fe5f800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:19.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.730+0000 7fe604ff9700 1 -- 172.21.15.88:0/3374643720 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5f80098a0 con 0x7fe6080ff150 2024-02-19T18:58:19.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.730+0000 7fe604ff9700 1 -- 172.21.15.88:0/3374643720 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe5f8010460 con 0x7fe6080ff150 2024-02-19T18:58:19.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.730+0000 7fe60e007700 1 -- 172.21.15.88:0/3374643720 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe60819b510 con 0x7fe6080ff150 2024-02-19T18:58:19.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.730+0000 7fe604ff9700 1 -- 172.21.15.88:0/3374643720 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5f800f5f0 con 0x7fe6080ff150 2024-02-19T18:58:19.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.730+0000 7fe60e007700 1 -- 172.21.15.88:0/3374643720 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe60819ba60 con 0x7fe6080ff150 2024-02-19T18:58:19.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.731+0000 7fe60e007700 1 -- 172.21.15.88:0/3374643720 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe608068ab0 con 0x7fe6080ff150 2024-02-19T18:58:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.731+0000 7fe604ff9700 1 -- 172.21.15.88:0/3374643720 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fe5f8009a00 con 0x7fe6080ff150 2024-02-19T18:58:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.732+0000 7fe604ff9700 1 --2- 172.21.15.88:0/3374643720 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe5f006c680 0x7fe5f006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.732+0000 7fe604ff9700 1 -- 172.21.15.88:0/3374643720 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fe5f808b420 con 0x7fe6080ff150 2024-02-19T18:58:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.732+0000 7fe606ffd700 1 --2- 172.21.15.88:0/3374643720 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe5f006c680 0x7fe5f006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:19.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.732+0000 7fe606ffd700 1 --2- 172.21.15.88:0/3374643720 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe5f006c680 0x7fe5f006eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe608069eb0 tx=0x7fe5fc00b540 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:19.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.736+0000 7fe604ff9700 1 -- 172.21.15.88:0/3374643720 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe5f80598d0 con 0x7fe6080ff150 2024-02-19T18:58:19.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.936+0000 7fe60e007700 1 -- 172.21.15.88:0/3374643720 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe608104e40 con 0x7fe5f006c680 2024-02-19T18:58:19.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.945+0000 7fe604ff9700 1 -- 172.21.15.88:0/3374643720 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2820 (secure 0 0 0) 0x7fe608104e40 con 0x7fe5f006c680 2024-02-19T18:58:19.963 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T18:58:19.963 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (3m) 2m ago 5m 25.8M - 0.25.0 c8568f914cd2 d6dd804f52d8 2024-02-19T18:58:19.963 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (6m) 2m ago 6m 7046k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T18:58:19.963 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (4m) 66s ago 4m 6933k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T18:58:19.963 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (6m) 2m ago 6m 6660k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T18:58:19.963 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (4m) 66s ago 4m 6656k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (3m) 2m ago 5m 79.9M - 9.4.7 954c08fa6188 54907dc6edb4 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:9283,8765,8443 running (7m) 2m ago 7m 479M - 18.2.0 dc2bc1663786 5b5e2fca6e0a 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (4m) 66s ago 4m 438M - 18.2.0 dc2bc1663786 95cabedbd6c4 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (7m) 2m ago 7m 46.2M 2048M 18.2.0 dc2bc1663786 3ea5108455bc 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (4m) 66s ago 4m 34.8M 2048M 18.2.0 dc2bc1663786 ba1c249c6017 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (6m) 2m ago 6m 14.2M - 1.5.0 0da6a335fe13 7cebce4a6d13 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (4m) 66s ago 4m 16.5M - 1.5.0 0da6a335fe13 f93d4349304d 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (3m) 2m ago 3m 43.7M 3946M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (2m) 2m ago 2m 40.4M 3946M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (2m) 2m ago 2m 40.1M 3946M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (2m) 66s ago 2m 43.9M 4970M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (98s) 66s ago 98s 41.3M 4970M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (72s) 66s ago 71s 11.5M 4970M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T18:58:19.964 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (3m) 2m ago 4m 41.7M - 2.43.0 a07b618ecd1d 17518e93393b 2024-02-19T18:58:19.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.947+0000 7fe60e007700 1 -- 172.21.15.88:0/3374643720 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe5f006c680 msgr2=0x7fe5f006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:19.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.947+0000 7fe60e007700 1 --2- 172.21.15.88:0/3374643720 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe5f006c680 0x7fe5f006eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe608069eb0 tx=0x7fe5fc00b540 comp rx=0 tx=0).stop 2024-02-19T18:58:19.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.947+0000 7fe60e007700 1 -- 172.21.15.88:0/3374643720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6080ff150 msgr2=0x7fe608069950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:19.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.947+0000 7fe60e007700 1 --2- 172.21.15.88:0/3374643720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6080ff150 0x7fe608069950 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe5f800d8d0 tx=0x7fe5f800dbe0 comp rx=0 tx=0).stop 2024-02-19T18:58:19.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.948+0000 7fe60e007700 1 -- 172.21.15.88:0/3374643720 shutdown_connections 2024-02-19T18:58:19.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.948+0000 7fe60e007700 1 --2- 172.21.15.88:0/3374643720 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fe5f006c680 0x7fe5f006eb30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:19.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.948+0000 7fe60e007700 1 --2- 172.21.15.88:0/3374643720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6080ff150 0x7fe608069950 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:19.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.948+0000 7fe60e007700 1 --2- 172.21.15.88:0/3374643720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6081003f0 0x7fe60819aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:19.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.948+0000 7fe60e007700 1 -- 172.21.15.88:0/3374643720 >> 172.21.15.88:0/3374643720 conn(0x7fe6080fa7a0 msgr2=0x7fe608103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:19.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.948+0000 7fe60e007700 1 -- 172.21.15.88:0/3374643720 shutdown_connections 2024-02-19T18:58:19.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:19.948+0000 7fe60e007700 1 -- 172.21.15.88:0/3374643720 wait complete. 2024-02-19T18:58:20.678 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-19T18:58:20.922 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:21.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:20 smithi132 ceph-mon[37969]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:21.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:20 smithi132 ceph-mon[37969]: from='client.14470 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:21.267 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:20 smithi088 ceph-mon[29272]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:21.267 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:20 smithi088 ceph-mon[29272]: from='client.14470 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:22.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.473+0000 7f2982945700 1 -- 172.21.15.88:0/3657163507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f297c100d60 msgr2=0x7f297c103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:22.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.473+0000 7f2982945700 1 --2- 172.21.15.88:0/3657163507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f297c100d60 0x7f297c103140 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f296c009910 tx=0x7f296c009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:22.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.473+0000 7f2982945700 1 -- 172.21.15.88:0/3657163507 shutdown_connections 2024-02-19T18:58:22.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.473+0000 7f2982945700 1 --2- 172.21.15.88:0/3657163507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f297c103680 0x7f297c105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:22.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.473+0000 7f2982945700 1 --2- 172.21.15.88:0/3657163507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f297c100d60 0x7f297c103140 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:22.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.473+0000 7f2982945700 1 -- 172.21.15.88:0/3657163507 >> 172.21.15.88:0/3657163507 conn(0x7f297c0fa780 msgr2=0x7f297c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:22.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.473+0000 7f2982945700 1 -- 172.21.15.88:0/3657163507 shutdown_connections 2024-02-19T18:58:22.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.473+0000 7f2982945700 1 -- 172.21.15.88:0/3657163507 wait complete. 2024-02-19T18:58:22.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.473+0000 7f2982945700 1 Processor -- start 2024-02-19T18:58:22.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.473+0000 7f2982945700 1 -- start start 2024-02-19T18:58:22.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f2982945700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f297c100d60 0x7f297c195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f2982945700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f297c103680 0x7f297c195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f2982945700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c196610 con 0x7f297c100d60 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f2982945700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f297c196750 con 0x7f297c103680 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f297bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f297c100d60 0x7f297c195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f297bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f297c100d60 0x7f297c195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36016/0 (socket says 172.21.15.88:36016) 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f297bfff700 1 -- 172.21.15.88:0/3731071174 learned_addr learned my addr 172.21.15.88:0/3731071174 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f297b7fe700 1 --2- 172.21.15.88:0/3731071174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f297c103680 0x7f297c195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f297bfff700 1 -- 172.21.15.88:0/3731071174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f297c103680 msgr2=0x7f297c195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f297bfff700 1 --2- 172.21.15.88:0/3731071174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f297c103680 0x7f297c195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f297bfff700 1 -- 172.21.15.88:0/3731071174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0095f0 con 0x7f297c100d60 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f297bfff700 1 --2- 172.21.15.88:0/3731071174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f297c100d60 0x7f297c195ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f296c00bda0 tx=0x7f296c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:22.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f29797fa700 1 -- 172.21.15.88:0/3731071174 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f296c01d070 con 0x7f297c100d60 2024-02-19T18:58:22.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f2982945700 1 -- 172.21.15.88:0/3731071174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297c19b1a0 con 0x7f297c100d60 2024-02-19T18:58:22.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f29797fa700 1 -- 172.21.15.88:0/3731071174 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f296c004c50 con 0x7f297c100d60 2024-02-19T18:58:22.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.474+0000 7f29797fa700 1 -- 172.21.15.88:0/3731071174 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f296c00f690 con 0x7f297c100d60 2024-02-19T18:58:22.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.475+0000 7f2982945700 1 -- 172.21.15.88:0/3731071174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297c19b690 con 0x7f297c100d60 2024-02-19T18:58:22.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.475+0000 7f2982945700 1 -- 172.21.15.88:0/3731071174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f297c18fbc0 con 0x7f297c100d60 2024-02-19T18:58:22.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.476+0000 7f29797fa700 1 -- 172.21.15.88:0/3731071174 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f296c0229b0 con 0x7f297c100d60 2024-02-19T18:58:22.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.476+0000 7f29797fa700 1 --2- 172.21.15.88:0/3731071174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f296406c680 0x7f296406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:22.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.476+0000 7f29797fa700 1 -- 172.21.15.88:0/3731071174 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f296c08cbb0 con 0x7f297c100d60 2024-02-19T18:58:22.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.476+0000 7f297b7fe700 1 --2- 172.21.15.88:0/3731071174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f296406c680 0x7f296406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:22.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.477+0000 7f297b7fe700 1 --2- 172.21.15.88:0/3731071174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f296406c680 0x7f296406eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2970005fd0 tx=0x7f297000a560 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:22.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.481+0000 7f29797fa700 1 -- 172.21.15.88:0/3731071174 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f296c093080 con 0x7f297c100d60 2024-02-19T18:58:22.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.683+0000 7f2982945700 1 -- 172.21.15.88:0/3731071174 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f297c061190 con 0x7f296406c680 2024-02-19T18:58:22.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.687+0000 7f29797fa700 1 -- 172.21.15.88:0/3731071174 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1300 (secure 0 0 0) 0x7f297c061190 con 0x7f296406c680 2024-02-19T18:58:22.705 INFO:teuthology.orchestra.run.smithi088.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-19T18:58:22.705 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager ?:9093,9094 1/1 2m ago 6m count:1 2024-02-19T18:58:22.706 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter 2/2 2m ago 7m * 2024-02-19T18:58:22.706 INFO:teuthology.orchestra.run.smithi088.stdout:crash 2/2 2m ago 7m * 2024-02-19T18:58:22.706 INFO:teuthology.orchestra.run.smithi088.stdout:grafana ?:3000 1/1 2m ago 6m count:1 2024-02-19T18:58:22.706 INFO:teuthology.orchestra.run.smithi088.stdout:mgr 2/2 2m ago 7m count:2 2024-02-19T18:58:22.706 INFO:teuthology.orchestra.run.smithi088.stdout:mon 2/2 2m ago 6m smithi088:172.21.15.88=smithi088;smithi132:172.21.15.132=smithi132;count:2 2024-02-19T18:58:22.706 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter ?:9100 2/2 2m ago 6m * 2024-02-19T18:58:22.706 INFO:teuthology.orchestra.run.smithi088.stdout:osd 6 2m ago - 2024-02-19T18:58:22.706 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus ?:9095 1/1 2m ago 6m count:1 2024-02-19T18:58:22.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.689+0000 7f2982945700 1 -- 172.21.15.88:0/3731071174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f296406c680 msgr2=0x7f296406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:22.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.689+0000 7f2982945700 1 --2- 172.21.15.88:0/3731071174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f296406c680 0x7f296406eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2970005fd0 tx=0x7f297000a560 comp rx=0 tx=0).stop 2024-02-19T18:58:22.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.689+0000 7f2982945700 1 -- 172.21.15.88:0/3731071174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f297c100d60 msgr2=0x7f297c195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:22.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.689+0000 7f2982945700 1 --2- 172.21.15.88:0/3731071174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f297c100d60 0x7f297c195ab0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f296c00bda0 tx=0x7f296c00be80 comp rx=0 tx=0).stop 2024-02-19T18:58:22.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.690+0000 7f2982945700 1 -- 172.21.15.88:0/3731071174 shutdown_connections 2024-02-19T18:58:22.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.690+0000 7f2982945700 1 --2- 172.21.15.88:0/3731071174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f296406c680 0x7f296406eb30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:22.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.690+0000 7f2982945700 1 --2- 172.21.15.88:0/3731071174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f297c100d60 0x7f297c195ab0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:22.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.690+0000 7f2982945700 1 --2- 172.21.15.88:0/3731071174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f297c103680 0x7f297c195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:22.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.690+0000 7f2982945700 1 -- 172.21.15.88:0/3731071174 >> 172.21.15.88:0/3731071174 conn(0x7f297c0fa780 msgr2=0x7f297c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:22.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.690+0000 7f2982945700 1 -- 172.21.15.88:0/3731071174 shutdown_connections 2024-02-19T18:58:22.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:22.690+0000 7f2982945700 1 -- 172.21.15.88:0/3731071174 wait complete. 2024-02-19T18:58:23.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:22 smithi132 ceph-mon[37969]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:23.284 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:22 smithi088 ceph-mon[29272]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:23.324 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-19T18:58:23.529 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:24.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:23 smithi088 ceph-mon[29272]: from='client.14474 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:24.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:23 smithi132 ceph-mon[37969]: from='client.14474 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.238+0000 7f93683e8700 1 -- 172.21.15.88:0/1409980618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9360100ce0 msgr2=0x7f93601030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.238+0000 7f93683e8700 1 --2- 172.21.15.88:0/1409980618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9360100ce0 0x7f93601030c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f935c009910 tx=0x7f935c009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.238+0000 7f93683e8700 1 -- 172.21.15.88:0/1409980618 shutdown_connections 2024-02-19T18:58:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.238+0000 7f93683e8700 1 --2- 172.21.15.88:0/1409980618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9360103600 0x7f93601059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.238+0000 7f93683e8700 1 --2- 172.21.15.88:0/1409980618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9360100ce0 0x7f93601030c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.238+0000 7f93683e8700 1 -- 172.21.15.88:0/1409980618 >> 172.21.15.88:0/1409980618 conn(0x7f93600fa700 msgr2=0x7f93600fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:25.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.238+0000 7f93683e8700 1 -- 172.21.15.88:0/1409980618 shutdown_connections 2024-02-19T18:58:25.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.238+0000 7f93683e8700 1 -- 172.21.15.88:0/1409980618 wait complete. 2024-02-19T18:58:25.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:24 smithi088 ceph-mon[29272]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:25.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:24 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:58:25.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.238+0000 7f93683e8700 1 Processor -- start 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.238+0000 7f93683e8700 1 -- start start 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f93683e8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9360100ce0 0x7f936019c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f93683e8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9360103600 0x7f936019c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f93683e8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936019cc00 con 0x7f9360100ce0 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f93683e8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936019cd40 con 0x7f9360103600 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f9366184700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9360100ce0 0x7f936019c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f9366184700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9360100ce0 0x7f936019c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36024/0 (socket says 172.21.15.88:36024) 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f9366184700 1 -- 172.21.15.88:0/1329314152 learned_addr learned my addr 172.21.15.88:0/1329314152 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f9365983700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9360103600 0x7f936019c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f9366184700 1 -- 172.21.15.88:0/1329314152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9360103600 msgr2=0x7f936019c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f9366184700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9360103600 0x7f936019c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f9366184700 1 -- 172.21.15.88:0/1329314152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f935c0095f0 con 0x7f9360100ce0 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f9365983700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9360103600 0x7f936019c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f9366184700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9360100ce0 0x7f936019c0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f935c00bd70 tx=0x7f935c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.239+0000 7f93577fe700 1 -- 172.21.15.88:0/1329314152 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935c01d070 con 0x7f9360100ce0 2024-02-19T18:58:25.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.240+0000 7f93683e8700 1 -- 172.21.15.88:0/1329314152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93601a1790 con 0x7f9360100ce0 2024-02-19T18:58:25.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.240+0000 7f93577fe700 1 -- 172.21.15.88:0/1329314152 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f935c022470 con 0x7f9360100ce0 2024-02-19T18:58:25.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.240+0000 7f93577fe700 1 -- 172.21.15.88:0/1329314152 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935c00f670 con 0x7f9360100ce0 2024-02-19T18:58:25.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.240+0000 7f93683e8700 1 -- 172.21.15.88:0/1329314152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93601a1c80 con 0x7f9360100ce0 2024-02-19T18:58:25.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.241+0000 7f93683e8700 1 -- 172.21.15.88:0/1329314152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93600fc2c0 con 0x7f9360100ce0 2024-02-19T18:58:25.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.241+0000 7f93577fe700 1 -- 172.21.15.88:0/1329314152 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f935c022a50 con 0x7f9360100ce0 2024-02-19T18:58:25.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.242+0000 7f93577fe700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f934c06c680 0x7f934c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:25.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.242+0000 7f93577fe700 1 -- 172.21.15.88:0/1329314152 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f935c08caa0 con 0x7f9360100ce0 2024-02-19T18:58:25.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.242+0000 7f9365983700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f934c06c680 0x7f934c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:25.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.242+0000 7f9365983700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f934c06c680 0x7f934c06eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9350005d90 tx=0x7f9350005d00 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:25.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.246+0000 7f93577fe700 1 -- 172.21.15.88:0/1329314152 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f935c05b000 con 0x7f9360100ce0 2024-02-19T18:58:25.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.445+0000 7f93683e8700 1 -- 172.21.15.88:0/1329314152 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f9360061190 con 0x7f934c06c680 2024-02-19T18:58:25.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.447+0000 7f93577fe700 1 -- 172.21.15.88:0/1329314152 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f9360061190 con 0x7f934c06c680 2024-02-19T18:58:25.464 INFO:teuthology.orchestra.run.smithi088.stdout:HOST ADDR LABELS STATUS 2024-02-19T18:58:25.465 INFO:teuthology.orchestra.run.smithi088.stdout:smithi088 172.21.15.88 2024-02-19T18:58:25.465 INFO:teuthology.orchestra.run.smithi088.stdout:smithi132 172.21.15.132 2024-02-19T18:58:25.465 INFO:teuthology.orchestra.run.smithi088.stdout:2 hosts in cluster 2024-02-19T18:58:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.449+0000 7f93683e8700 1 -- 172.21.15.88:0/1329314152 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f934c06c680 msgr2=0x7f934c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.449+0000 7f93683e8700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f934c06c680 0x7f934c06eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9350005d90 tx=0x7f9350005d00 comp rx=0 tx=0).stop 2024-02-19T18:58:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.449+0000 7f93683e8700 1 -- 172.21.15.88:0/1329314152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9360100ce0 msgr2=0x7f936019c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.449+0000 7f93683e8700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9360100ce0 0x7f936019c0a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f935c00bd70 tx=0x7f935c00be50 comp rx=0 tx=0).stop 2024-02-19T18:58:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.449+0000 7f93683e8700 1 -- 172.21.15.88:0/1329314152 shutdown_connections 2024-02-19T18:58:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.449+0000 7f93683e8700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f934c06c680 0x7f934c06eb30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.449+0000 7f93683e8700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9360100ce0 0x7f936019c0a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.449+0000 7f93683e8700 1 --2- 172.21.15.88:0/1329314152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9360103600 0x7f936019c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.449+0000 7f93683e8700 1 -- 172.21.15.88:0/1329314152 >> 172.21.15.88:0/1329314152 conn(0x7f93600fa700 msgr2=0x7f9360104260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.449+0000 7f93683e8700 1 -- 172.21.15.88:0/1329314152 shutdown_connections 2024-02-19T18:58:25.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:25.449+0000 7f93683e8700 1 -- 172.21.15.88:0/1329314152 wait complete. 2024-02-19T18:58:25.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:24 smithi132 ceph-mon[37969]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:25.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:24 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:58:26.122 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-19T18:58:26.327 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:27.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:26 smithi132 ceph-mon[37969]: from='client.14478 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:27.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:26 smithi132 ceph-mon[37969]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:27.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:26 smithi088 ceph-mon[29272]: from='client.14478 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:27.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:26 smithi088 ceph-mon[29272]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:27.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.819+0000 7f244833e700 1 -- 172.21.15.88:0/3389227268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24401003d0 msgr2=0x7f2440100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:27.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.819+0000 7f244833e700 1 --2- 172.21.15.88:0/3389227268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24401003d0 0x7f2440100840 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2430009910 tx=0x7f2430009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:27.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.819+0000 7f244833e700 1 -- 172.21.15.88:0/3389227268 shutdown_connections 2024-02-19T18:58:27.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.819+0000 7f244833e700 1 --2- 172.21.15.88:0/3389227268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24401003d0 0x7f2440100840 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:27.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.819+0000 7f244833e700 1 --2- 172.21.15.88:0/3389227268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24400ff130 0x7f24400ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:27.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.819+0000 7f244833e700 1 -- 172.21.15.88:0/3389227268 >> 172.21.15.88:0/3389227268 conn(0x7f24400fa780 msgr2=0x7f24400fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:27.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.819+0000 7f244833e700 1 -- 172.21.15.88:0/3389227268 shutdown_connections 2024-02-19T18:58:27.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.819+0000 7f244833e700 1 -- 172.21.15.88:0/3389227268 wait complete. 2024-02-19T18:58:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f244833e700 1 Processor -- start 2024-02-19T18:58:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f244833e700 1 -- start start 2024-02-19T18:58:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f244833e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24400ff130 0x7f24400690a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f244833e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24401003d0 0x7f244019a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f244833e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019ae20 con 0x7f24401003d0 2024-02-19T18:58:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f244833e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f244019af60 con 0x7f24400ff130 2024-02-19T18:58:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f24458d9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24401003d0 0x7f244019a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f24458d9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24401003d0 0x7f244019a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48572/0 (socket says 172.21.15.88:48572) 2024-02-19T18:58:27.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f24458d9700 1 -- 172.21.15.88:0/3355478126 learned_addr learned my addr 172.21.15.88:0/3355478126 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f24460da700 1 --2- 172.21.15.88:0/3355478126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24400ff130 0x7f24400690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f24458d9700 1 -- 172.21.15.88:0/3355478126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24400ff130 msgr2=0x7f24400690a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f24458d9700 1 --2- 172.21.15.88:0/3355478126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24400ff130 0x7f24400690a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.820+0000 7f24458d9700 1 -- 172.21.15.88:0/3355478126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24300095f0 con 0x7f24401003d0 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.821+0000 7f24458d9700 1 --2- 172.21.15.88:0/3355478126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24401003d0 0x7f244019a850 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2430006010 tx=0x7f243000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.821+0000 7f24377fe700 1 -- 172.21.15.88:0/3355478126 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f243001d070 con 0x7f24401003d0 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.821+0000 7f24377fe700 1 -- 172.21.15.88:0/3355478126 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f243000f460 con 0x7f24401003d0 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.821+0000 7f244833e700 1 -- 172.21.15.88:0/3355478126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f244019b1e0 con 0x7f24401003d0 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.821+0000 7f24377fe700 1 -- 172.21.15.88:0/3355478126 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2430021640 con 0x7f24401003d0 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.821+0000 7f244833e700 1 -- 172.21.15.88:0/3355478126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f244019b6d0 con 0x7f24401003d0 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.822+0000 7f244833e700 1 -- 172.21.15.88:0/3355478126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2440107c90 con 0x7f24401003d0 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.822+0000 7f24377fe700 1 -- 172.21.15.88:0/3355478126 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f2430004b20 con 0x7f24401003d0 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.823+0000 7f24377fe700 1 --2- 172.21.15.88:0/3355478126 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f242c06c680 0x7f242c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.823+0000 7f24460da700 1 --2- 172.21.15.88:0/3355478126 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f242c06c680 0x7f242c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:27.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.823+0000 7f24377fe700 1 -- 172.21.15.88:0/3355478126 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f243008c9e0 con 0x7f24401003d0 2024-02-19T18:58:27.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.823+0000 7f24460da700 1 --2- 172.21.15.88:0/3355478126 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f242c06c680 0x7f242c06eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f243c006fd0 tx=0x7f243c008040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:27.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:27.827+0000 7f24377fe700 1 -- 172.21.15.88:0/3355478126 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f243005ae90 con 0x7f24401003d0 2024-02-19T18:58:28.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.024+0000 7f244833e700 1 -- 172.21.15.88:0/3355478126 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f244019bdd0 con 0x7f242c06c680 2024-02-19T18:58:28.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.026+0000 7f24377fe700 1 -- 172.21.15.88:0/3355478126 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f244019bdd0 con 0x7f242c06c680 2024-02-19T18:58:28.044 INFO:teuthology.orchestra.run.smithi088.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-19T18:58:28.044 INFO:teuthology.orchestra.run.smithi088.stdout:smithi088 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XD400BGN 372G No 2m ago LVM detected, locked 2024-02-19T18:58:28.044 INFO:teuthology.orchestra.run.smithi088.stdout:smithi132 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6232000B400BGN 372G No 70s ago LVM detected, locked 2024-02-19T18:58:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.029+0000 7f244833e700 1 -- 172.21.15.88:0/3355478126 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f242c06c680 msgr2=0x7f242c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.029+0000 7f244833e700 1 --2- 172.21.15.88:0/3355478126 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f242c06c680 0x7f242c06eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f243c006fd0 tx=0x7f243c008040 comp rx=0 tx=0).stop 2024-02-19T18:58:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.029+0000 7f244833e700 1 -- 172.21.15.88:0/3355478126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24401003d0 msgr2=0x7f244019a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.029+0000 7f244833e700 1 --2- 172.21.15.88:0/3355478126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24401003d0 0x7f244019a850 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2430006010 tx=0x7f243000be20 comp rx=0 tx=0).stop 2024-02-19T18:58:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.029+0000 7f244833e700 1 -- 172.21.15.88:0/3355478126 shutdown_connections 2024-02-19T18:58:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.029+0000 7f244833e700 1 --2- 172.21.15.88:0/3355478126 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f242c06c680 0x7f242c06eb30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.029+0000 7f244833e700 1 --2- 172.21.15.88:0/3355478126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24400ff130 0x7f24400690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.029+0000 7f244833e700 1 --2- 172.21.15.88:0/3355478126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24401003d0 0x7f244019a850 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.029+0000 7f244833e700 1 -- 172.21.15.88:0/3355478126 >> 172.21.15.88:0/3355478126 conn(0x7f24400fa780 msgr2=0x7f2440103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.029+0000 7f244833e700 1 -- 172.21.15.88:0/3355478126 shutdown_connections 2024-02-19T18:58:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:28.029+0000 7f244833e700 1 -- 172.21.15.88:0/3355478126 wait complete. 2024-02-19T18:58:28.695 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-19T18:58:28.708 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-19T18:58:28.721 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T18:58:28.721 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-02-19T18:58:28.929 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:29.262 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:28 smithi088 ceph-mon[29272]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:29.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:28 smithi132 ceph-mon[37969]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:30.310 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:29 smithi132 ceph-mon[37969]: from='client.14482 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:30.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:29 smithi088 ceph-mon[29272]: from='client.14482 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:30.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.670+0000 7f3b474b1700 1 -- 172.21.15.88:0/2900485845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b401003d0 msgr2=0x7f3b40100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:30.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.670+0000 7f3b474b1700 1 --2- 172.21.15.88:0/2900485845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b401003d0 0x7f3b40100840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3b30009910 tx=0x7f3b30009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:30.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.670+0000 7f3b474b1700 1 -- 172.21.15.88:0/2900485845 shutdown_connections 2024-02-19T18:58:30.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.670+0000 7f3b474b1700 1 --2- 172.21.15.88:0/2900485845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b401003d0 0x7f3b40100840 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:30.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.670+0000 7f3b474b1700 1 --2- 172.21.15.88:0/2900485845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b400ff130 0x7f3b400ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:30.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.670+0000 7f3b474b1700 1 -- 172.21.15.88:0/2900485845 >> 172.21.15.88:0/2900485845 conn(0x7f3b400fa780 msgr2=0x7f3b400fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:30.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.670+0000 7f3b474b1700 1 -- 172.21.15.88:0/2900485845 shutdown_connections 2024-02-19T18:58:30.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.670+0000 7f3b474b1700 1 -- 172.21.15.88:0/2900485845 wait complete. 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b474b1700 1 Processor -- start 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b474b1700 1 -- start start 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b474b1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b400ff130 0x7f3b401949f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b474b1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b401003d0 0x7f3b40194f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b474b1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40195550 con 0x7f3b401003d0 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b474b1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b40195690 con 0x7f3b400ff130 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b44a4c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b401003d0 0x7f3b40194f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b44a4c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b401003d0 0x7f3b40194f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48590/0 (socket says 172.21.15.88:48590) 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b44a4c700 1 -- 172.21.15.88:0/2866912003 learned_addr learned my addr 172.21.15.88:0/2866912003 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b4524d700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b400ff130 0x7f3b401949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b44a4c700 1 -- 172.21.15.88:0/2866912003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b400ff130 msgr2=0x7f3b401949f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b44a4c700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b400ff130 0x7f3b401949f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b44a4c700 1 -- 172.21.15.88:0/2866912003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b300095f0 con 0x7f3b401003d0 2024-02-19T18:58:30.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b4524d700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b400ff130 0x7f3b401949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T18:58:30.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b44a4c700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b401003d0 0x7f3b40194f30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3b30006010 tx=0x7f3b3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:30.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.671+0000 7f3b367fc700 1 -- 172.21.15.88:0/2866912003 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b3001f070 con 0x7f3b401003d0 2024-02-19T18:58:30.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.672+0000 7f3b474b1700 1 -- 172.21.15.88:0/2866912003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b40068540 con 0x7f3b401003d0 2024-02-19T18:58:30.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.672+0000 7f3b367fc700 1 -- 172.21.15.88:0/2866912003 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3b30019610 con 0x7f3b401003d0 2024-02-19T18:58:30.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.672+0000 7f3b367fc700 1 -- 172.21.15.88:0/2866912003 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b3000f7f0 con 0x7f3b401003d0 2024-02-19T18:58:30.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.672+0000 7f3b474b1700 1 -- 172.21.15.88:0/2866912003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b40068a30 con 0x7f3b401003d0 2024-02-19T18:58:30.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.673+0000 7f3b474b1700 1 -- 172.21.15.88:0/2866912003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b40107c90 con 0x7f3b401003d0 2024-02-19T18:58:30.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.673+0000 7f3b367fc700 1 -- 172.21.15.88:0/2866912003 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f3b30019c60 con 0x7f3b401003d0 2024-02-19T18:58:30.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.673+0000 7f3b367fc700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3b2c06c680 0x7f3b2c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:30.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.673+0000 7f3b367fc700 1 -- 172.21.15.88:0/2866912003 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3b3008ebb0 con 0x7f3b401003d0 2024-02-19T18:58:30.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.673+0000 7f3b4524d700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3b2c06c680 0x7f3b2c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:30.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.674+0000 7f3b4524d700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3b2c06c680 0x7f3b2c06eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c007900 tx=0x7f3b3c008040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:30.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.678+0000 7f3b367fc700 1 -- 172.21.15.88:0/2866912003 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3b30095080 con 0x7f3b401003d0 2024-02-19T18:58:30.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:30.893+0000 7f3b474b1700 1 -- 172.21.15.88:0/2866912003 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b40104d20 con 0x7f3b2c06c680 2024-02-19T18:58:31.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:31 smithi132 ceph-mon[37969]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:31.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:31 smithi088 ceph-mon[29272]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:32.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.213+0000 7f3b367fc700 1 -- 172.21.15.88:0/2866912003 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3b40104d20 con 0x7f3b2c06c680 2024-02-19T18:58:32.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.216+0000 7f3b474b1700 1 -- 172.21.15.88:0/2866912003 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3b2c06c680 msgr2=0x7f3b2c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:32.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.216+0000 7f3b474b1700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3b2c06c680 0x7f3b2c06eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c007900 tx=0x7f3b3c008040 comp rx=0 tx=0).stop 2024-02-19T18:58:32.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.216+0000 7f3b474b1700 1 -- 172.21.15.88:0/2866912003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b401003d0 msgr2=0x7f3b40194f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:32.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.216+0000 7f3b474b1700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b401003d0 0x7f3b40194f30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3b30006010 tx=0x7f3b3000be20 comp rx=0 tx=0).stop 2024-02-19T18:58:32.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.216+0000 7f3b474b1700 1 -- 172.21.15.88:0/2866912003 shutdown_connections 2024-02-19T18:58:32.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.216+0000 7f3b474b1700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f3b2c06c680 0x7f3b2c06eb30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:32.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.216+0000 7f3b474b1700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b400ff130 0x7f3b401949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:32.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.216+0000 7f3b474b1700 1 --2- 172.21.15.88:0/2866912003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b401003d0 0x7f3b40194f30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:32.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.216+0000 7f3b474b1700 1 -- 172.21.15.88:0/2866912003 >> 172.21.15.88:0/2866912003 conn(0x7f3b400fa780 msgr2=0x7f3b40103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:32.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.216+0000 7f3b474b1700 1 -- 172.21.15.88:0/2866912003 shutdown_connections 2024-02-19T18:58:32.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:32.216+0000 7f3b474b1700 1 -- 172.21.15.88:0/2866912003 wait complete. 2024-02-19T18:58:32.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:32 smithi132 ceph-mon[37969]: from='client.14486 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:32.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:32 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-19T18:58:32.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:32 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-19T18:58:32.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:32 smithi132 ceph-mon[37969]: osdmap e41: 6 total, 6 up, 6 in 2024-02-19T18:58:32.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:32 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-19T18:58:32.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:32 smithi132 ceph-mon[37969]: pgmap v175: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:32.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:32 smithi088 ceph-mon[29272]: from='client.14486 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:58:32.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:32 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-02-19T18:58:32.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:32 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-02-19T18:58:32.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:32 smithi088 ceph-mon[29272]: osdmap e41: 6 total, 6 up, 6 in 2024-02-19T18:58:32.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:32 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-02-19T18:58:32.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:32 smithi088 ceph-mon[29272]: pgmap v175: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:32.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:32 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[29268]: 2024-02-19T18:58:32.191+0000 7f451f83f700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-19T18:58:33.018 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-19T18:58:33.286 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-19T18:58:33.286 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: osdmap e42: 6 total, 6 up, 6 in 2024-02-19T18:58:33.286 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-19T18:58:33.286 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-19T18:58:33.286 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-19T18:58:33.286 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: osdmap e43: 6 total, 6 up, 6 in 2024-02-19T18:58:33.286 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-19T18:58:33.286 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: fsmap cephfs:0 2024-02-19T18:58:33.286 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: Saving service mds.cephfs spec with placement count:4 2024-02-19T18:58:33.287 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:33.287 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:58:33.287 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:33.287 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:58:33.287 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:33.287 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.ajgvpz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T18:58:33.287 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.ajgvpz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T18:58:33.287 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:33.287 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:33 smithi088 ceph-mon[29272]: Deploying daemon mds.cephfs.smithi088.ajgvpz on smithi088 2024-02-19T18:58:33.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-02-19T18:58:33.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: osdmap e42: 6 total, 6 up, 6 in 2024-02-19T18:58:33.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-02-19T18:58:33.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: osdmap e43: 6 total, 6 up, 6 in 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: fsmap cephfs:0 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: Saving service mds.cephfs spec with placement count:4 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.ajgvpz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.ajgvpz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:33 smithi132 ceph-mon[37969]: Deploying daemon mds.cephfs.smithi088.ajgvpz on smithi088 2024-02-19T18:58:33.819 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:34.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:34 smithi132 ceph-mon[37969]: osdmap e44: 6 total, 6 up, 6 in 2024-02-19T18:58:34.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:34 smithi132 ceph-mon[37969]: pgmap v179: 65 pgs: 13 creating+peering, 46 unknown, 6 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:34.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:34 smithi088 ceph-mon[29272]: osdmap e44: 6 total, 6 up, 6 in 2024-02-19T18:58:34.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:34 smithi088 ceph-mon[29272]: pgmap v179: 65 pgs: 13 creating+peering, 46 unknown, 6 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:35.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:35 smithi132 ceph-mon[37969]: osdmap e45: 6 total, 6 up, 6 in 2024-02-19T18:58:35.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:35 smithi088 ceph-mon[29272]: osdmap e45: 6 total, 6 up, 6 in 2024-02-19T18:58:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.092+0000 7f164a8f9700 1 -- 172.21.15.88:0/2389525874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16441018a0 msgr2=0x7f1644101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.092+0000 7f164a8f9700 1 --2- 172.21.15.88:0/2389525874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16441018a0 0x7f1644101cb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1634009910 tx=0x7f1634009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.092+0000 7f164a8f9700 1 -- 172.21.15.88:0/2389525874 shutdown_connections 2024-02-19T18:58:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.092+0000 7f164a8f9700 1 --2- 172.21.15.88:0/2389525874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16441021f0 0x7f16440fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.092+0000 7f164a8f9700 1 --2- 172.21.15.88:0/2389525874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16441018a0 0x7f1644101cb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.092+0000 7f164a8f9700 1 -- 172.21.15.88:0/2389525874 >> 172.21.15.88:0/2389525874 conn(0x7f16440f9f10 msgr2=0x7f16440fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.092+0000 7f164a8f9700 1 -- 172.21.15.88:0/2389525874 shutdown_connections 2024-02-19T18:58:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.092+0000 7f164a8f9700 1 -- 172.21.15.88:0/2389525874 wait complete. 2024-02-19T18:58:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f164a8f9700 1 Processor -- start 2024-02-19T18:58:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f164a8f9700 1 -- start start 2024-02-19T18:58:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f164a8f9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16441018a0 0x7f1644195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f164a8f9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16441021f0 0x7f1644196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f164a8f9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1644196690 con 0x7f16441021f0 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f164a8f9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16441967d0 con 0x7f16441018a0 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f16437fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16441021f0 0x7f1644196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f16437fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16441021f0 0x7f1644196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48526/0 (socket says 172.21.15.88:48526) 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f16437fe700 1 -- 172.21.15.88:0/823573810 learned_addr learned my addr 172.21.15.88:0/823573810 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f1643fff700 1 --2- 172.21.15.88:0/823573810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16441018a0 0x7f1644195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f16437fe700 1 -- 172.21.15.88:0/823573810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16441018a0 msgr2=0x7f1644195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f16437fe700 1 --2- 172.21.15.88:0/823573810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16441018a0 0x7f1644195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f16437fe700 1 -- 172.21.15.88:0/823573810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16340095f0 con 0x7f16441021f0 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.093+0000 7f16437fe700 1 --2- 172.21.15.88:0/823573810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16441021f0 0x7f1644196070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f163800c8f0 tx=0x7f163800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.094+0000 7f16417fa700 1 -- 172.21.15.88:0/823573810 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1638007a50 con 0x7f16441021f0 2024-02-19T18:58:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.094+0000 7f16417fa700 1 -- 172.21.15.88:0/823573810 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1638007bb0 con 0x7f16441021f0 2024-02-19T18:58:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.094+0000 7f164a8f9700 1 -- 172.21.15.88:0/823573810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164419b280 con 0x7f16441021f0 2024-02-19T18:58:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.094+0000 7f16417fa700 1 -- 172.21.15.88:0/823573810 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1638018630 con 0x7f16441021f0 2024-02-19T18:58:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.094+0000 7f164a8f9700 1 -- 172.21.15.88:0/823573810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164419b7d0 con 0x7f16441021f0 2024-02-19T18:58:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.094+0000 7f164a8f9700 1 -- 172.21.15.88:0/823573810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1644108d30 con 0x7f16441021f0 2024-02-19T18:58:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.095+0000 7f16417fa700 1 -- 172.21.15.88:0/823573810 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f1638007d20 con 0x7f16441021f0 2024-02-19T18:58:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.095+0000 7f16417fa700 1 --2- 172.21.15.88:0/823573810 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f162c06c680 0x7f162c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.096+0000 7f16417fa700 1 -- 172.21.15.88:0/823573810 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f163808b840 con 0x7f16441021f0 2024-02-19T18:58:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.096+0000 7f1643fff700 1 --2- 172.21.15.88:0/823573810 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f162c06c680 0x7f162c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.096+0000 7f1643fff700 1 --2- 172.21.15.88:0/823573810 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f162c06c680 0x7f162c06eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f163400b5c0 tx=0x7f1634005960 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.100+0000 7f16417fa700 1 -- 172.21.15.88:0/823573810 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f16380599b0 con 0x7f16441021f0 2024-02-19T18:58:36.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.352+0000 7f164a8f9700 1 -- 172.21.15.88:0/823573810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f164419be70 con 0x7f16441021f0 2024-02-19T18:58:36.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.352+0000 7f16417fa700 1 -- 172.21.15.88:0/823573810 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1093 (secure 0 0 0) 0x7f1638059540 con 0x7f16441021f0 2024-02-19T18:58:36.370 INFO:teuthology.orchestra.run.smithi088.stdout:e2 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 2 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T18:58:32.192177+0000 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T18:58:36.371 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 0 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 1 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:in 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T18:58:36.372 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T18:58:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T18:58:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T18:58:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T18:58:36.373 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 0 2024-02-19T18:58:36.373 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:58:36.373 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:58:36.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.354+0000 7f164a8f9700 1 -- 172.21.15.88:0/823573810 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f162c06c680 msgr2=0x7f162c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:36.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.354+0000 7f164a8f9700 1 --2- 172.21.15.88:0/823573810 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f162c06c680 0x7f162c06eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f163400b5c0 tx=0x7f1634005960 comp rx=0 tx=0).stop 2024-02-19T18:58:36.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.354+0000 7f164a8f9700 1 -- 172.21.15.88:0/823573810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16441021f0 msgr2=0x7f1644196070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:36.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.354+0000 7f164a8f9700 1 --2- 172.21.15.88:0/823573810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16441021f0 0x7f1644196070 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f163800c8f0 tx=0x7f163800ccb0 comp rx=0 tx=0).stop 2024-02-19T18:58:36.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.354+0000 7f164a8f9700 1 -- 172.21.15.88:0/823573810 shutdown_connections 2024-02-19T18:58:36.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.354+0000 7f164a8f9700 1 --2- 172.21.15.88:0/823573810 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f162c06c680 0x7f162c06eb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:36.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.354+0000 7f164a8f9700 1 --2- 172.21.15.88:0/823573810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16441018a0 0x7f1644195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:36.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.354+0000 7f164a8f9700 1 --2- 172.21.15.88:0/823573810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16441021f0 0x7f1644196070 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:36.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.354+0000 7f164a8f9700 1 -- 172.21.15.88:0/823573810 >> 172.21.15.88:0/823573810 conn(0x7f16440f9f10 msgr2=0x7f1644104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:36.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.355+0000 7f164a8f9700 1 -- 172.21.15.88:0/823573810 shutdown_connections 2024-02-19T18:58:36.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:36.355+0000 7f164a8f9700 1 -- 172.21.15.88:0/823573810 wait complete. 2024-02-19T18:58:36.375 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 2 2024-02-19T18:58:36.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:36 smithi132 ceph-mon[37969]: pgmap v181: 65 pgs: 13 creating+peering, 28 unknown, 24 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:36.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:36 smithi088 ceph-mon[29272]: pgmap v181: 65 pgs: 13 creating+peering, 28 unknown, 24 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:37.035 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-19T18:58:37.046 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T18:58:37.046 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-02-19T18:58:37.340 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:37.347 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:37 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/823573810' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T18:58:37.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:37 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/823573810' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T18:58:38.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.afhgdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.afhgdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: Deploying daemon mds.cephfs.smithi132.afhgdi on smithi132 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: pgmap v182: 65 pgs: 4 creating+peering, 61 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.88:6826/694475098,v1:172.21.15.88:6827/694475098] up:boot 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: daemon mds.cephfs.smithi088.ajgvpz assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: Cluster is now healthy 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: fsmap cephfs:0 1 up:standby 2024-02-19T18:58:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T18:58:38.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: fsmap cephfs:1 {0=cephfs.smithi088.ajgvpz=up:creating} 2024-02-19T18:58:38.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:38 smithi088 ceph-mon[29272]: daemon mds.cephfs.smithi088.ajgvpz is now active in filesystem cephfs as rank 0 2024-02-19T18:58:38.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:38.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.afhgdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.afhgdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: Deploying daemon mds.cephfs.smithi132.afhgdi on smithi132 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: pgmap v182: 65 pgs: 4 creating+peering, 61 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.88:6826/694475098,v1:172.21.15.88:6827/694475098] up:boot 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: daemon mds.cephfs.smithi088.ajgvpz assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: Cluster is now healthy 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: fsmap cephfs:0 1 up:standby 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: fsmap cephfs:1 {0=cephfs.smithi088.ajgvpz=up:creating} 2024-02-19T18:58:38.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:38 smithi132 ceph-mon[37969]: daemon mds.cephfs.smithi088.ajgvpz is now active in filesystem cephfs as rank 0 2024-02-19T18:58:38.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 -- 172.21.15.88:0/895340634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2498100240 msgr2=0x7f2498100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:38.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 --2- 172.21.15.88:0/895340634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2498100240 0x7f2498100650 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f249400b0a0 tx=0x7f249400b3b0 comp rx=0 tx=0).stop 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 -- 172.21.15.88:0/895340634 shutdown_connections 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 --2- 172.21.15.88:0/895340634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2498101440 0x7f2498101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 --2- 172.21.15.88:0/895340634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2498100240 0x7f2498100650 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 -- 172.21.15.88:0/895340634 >> 172.21.15.88:0/895340634 conn(0x7f24980fb7f0 msgr2=0x7f24980fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 -- 172.21.15.88:0/895340634 shutdown_connections 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 -- 172.21.15.88:0/895340634 wait complete. 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 Processor -- start 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 -- start start 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2498100240 0x7f2498074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2498101440 0x7f2498073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2498073680 con 0x7f2498101440 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.828+0000 7f249ef48700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24980737c0 con 0x7f2498100240 2024-02-19T18:58:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f248ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2498101440 0x7f2498073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f248ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2498101440 0x7f2498073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48568/0 (socket says 172.21.15.88:48568) 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f248ffff700 1 -- 172.21.15.88:0/869744053 learned_addr learned my addr 172.21.15.88:0/869744053 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f249cce4700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2498100240 0x7f2498074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f248ffff700 1 -- 172.21.15.88:0/869744053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2498100240 msgr2=0x7f2498074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f248ffff700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2498100240 0x7f2498074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f248ffff700 1 -- 172.21.15.88:0/869744053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2494009d00 con 0x7f2498101440 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f249cce4700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2498100240 0x7f2498074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f248ffff700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2498101440 0x7f2498073140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f248800c8a0 tx=0x7f248800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f248dffb700 1 -- 172.21.15.88:0/869744053 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248800cea0 con 0x7f2498101440 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f248dffb700 1 -- 172.21.15.88:0/869744053 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2488004830 con 0x7f2498101440 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f248dffb700 1 -- 172.21.15.88:0/869744053 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24880055b0 con 0x7f2498101440 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f249ef48700 1 -- 172.21.15.88:0/869744053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2498073aa0 con 0x7f2498101440 2024-02-19T18:58:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.829+0000 7f249ef48700 1 -- 172.21.15.88:0/869744053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249819db30 con 0x7f2498101440 2024-02-19T18:58:38.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.830+0000 7f249ef48700 1 -- 172.21.15.88:0/869744053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2498108d90 con 0x7f2498101440 2024-02-19T18:58:38.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.831+0000 7f248dffb700 1 -- 172.21.15.88:0/869744053 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f2488007ad0 con 0x7f2498101440 2024-02-19T18:58:38.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.831+0000 7f248dffb700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f248006c680 0x7f248006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:38.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.831+0000 7f249cce4700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f248006c680 0x7f248006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:38.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.831+0000 7f248dffb700 1 -- 172.21.15.88:0/869744053 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f248808b5e0 con 0x7f2498101440 2024-02-19T18:58:38.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.831+0000 7f249cce4700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f248006c680 0x7f248006eb30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2494000f80 tx=0x7f2494006000 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:38.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:38.835+0000 7f248dffb700 1 -- 172.21.15.88:0/869744053 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2488059750 con 0x7f2498101440 2024-02-19T18:58:39.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.066+0000 7f249ef48700 1 -- 172.21.15.88:0/869744053 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f2498108f50 con 0x7f2498101440 2024-02-19T18:58:39.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.221+0000 7f248dffb700 1 -- 172.21.15.88:0/869744053 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v5) v1 ==== 105+0+0 (secure 0 0 0) 0x7f24880592e0 con 0x7f2498101440 2024-02-19T18:58:39.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.223+0000 7f249ef48700 1 -- 172.21.15.88:0/869744053 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f248006c680 msgr2=0x7f248006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:39.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.223+0000 7f249ef48700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f248006c680 0x7f248006eb30 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2494000f80 tx=0x7f2494006000 comp rx=0 tx=0).stop 2024-02-19T18:58:39.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.223+0000 7f249ef48700 1 -- 172.21.15.88:0/869744053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2498101440 msgr2=0x7f2498073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:39.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.223+0000 7f249ef48700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2498101440 0x7f2498073140 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f248800c8a0 tx=0x7f248800cc60 comp rx=0 tx=0).stop 2024-02-19T18:58:39.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.223+0000 7f249ef48700 1 -- 172.21.15.88:0/869744053 shutdown_connections 2024-02-19T18:58:39.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.223+0000 7f249ef48700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f248006c680 0x7f248006eb30 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:39.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.223+0000 7f249ef48700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2498100240 0x7f2498074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:39.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.223+0000 7f249ef48700 1 --2- 172.21.15.88:0/869744053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2498101440 0x7f2498073140 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:39.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.223+0000 7f249ef48700 1 -- 172.21.15.88:0/869744053 >> 172.21.15.88:0/869744053 conn(0x7f24980fb7f0 msgr2=0x7f2498104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:39.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.224+0000 7f249ef48700 1 -- 172.21.15.88:0/869744053 shutdown_connections 2024-02-19T18:58:39.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:39.224+0000 7f249ef48700 1 -- 172.21.15.88:0/869744053 wait complete. 2024-02-19T18:58:39.638 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:39 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/869744053' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-19T18:58:39.638 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:39 smithi132 ceph-mon[37969]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-19T18:58:39.638 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:39 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.88:6826/694475098,v1:172.21.15.88:6827/694475098] up:active 2024-02-19T18:58:39.638 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:39 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/869744053' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-19T18:58:39.638 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:39 smithi132 ceph-mon[37969]: fsmap cephfs:1 {0=cephfs.smithi088.ajgvpz=up:active} 2024-02-19T18:58:39.705 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:39 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/869744053' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-02-19T18:58:39.705 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:39 smithi088 ceph-mon[29272]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-19T18:58:39.705 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:39 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.88:6826/694475098,v1:172.21.15.88:6827/694475098] up:active 2024-02-19T18:58:39.705 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:39 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/869744053' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-02-19T18:58:39.705 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:39 smithi088 ceph-mon[29272]: fsmap cephfs:1 {0=cephfs.smithi088.ajgvpz=up:active} 2024-02-19T18:58:39.920 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-19T18:58:39.930 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T18:58:39.930 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-02-19T18:58:40.153 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:40.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:40 smithi088 ceph-mon[29272]: pgmap v183: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 412 B/s wr, 1 op/s 2024-02-19T18:58:40.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:40.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:58:40.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:40.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:40.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:40.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.psjiah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T18:58:40.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.psjiah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T18:58:40.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:40.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:40 smithi088 ceph-mon[29272]: Deploying daemon mds.cephfs.smithi088.psjiah on smithi088 2024-02-19T18:58:41.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:40 smithi132 ceph-mon[37969]: pgmap v183: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 412 B/s wr, 1 op/s 2024-02-19T18:58:41.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:41.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:58:41.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:41.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:41.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:41.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.psjiah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T18:58:41.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.psjiah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T18:58:41.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:41.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:40 smithi132 ceph-mon[37969]: Deploying daemon mds.cephfs.smithi088.psjiah on smithi088 2024-02-19T18:58:41.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:41 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] up:boot 2024-02-19T18:58:41.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:41 smithi088 ceph-mon[29272]: daemon mds.cephfs.smithi132.afhgdi assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-19T18:58:41.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:41 smithi088 ceph-mon[29272]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-19T18:58:41.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:41 smithi088 ceph-mon[29272]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-19T18:58:41.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:41 smithi088 ceph-mon[29272]: fsmap cephfs:1 {0=cephfs.smithi088.ajgvpz=up:active} 1 up:standby 2024-02-19T18:58:41.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:41 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T18:58:41.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:41 smithi088 ceph-mon[29272]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:creating} 2024-02-19T18:58:41.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:41 smithi088 ceph-mon[29272]: daemon mds.cephfs.smithi132.afhgdi is now active in filesystem cephfs as rank 1 2024-02-19T18:58:41.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.876+0000 7eff57628700 1 -- 172.21.15.88:0/735993712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff500fe500 msgr2=0x7eff500fe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:41.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.876+0000 7eff57628700 1 --2- 172.21.15.88:0/735993712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff500fe500 0x7eff500fe910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7eff40009910 tx=0x7eff40009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.876+0000 7eff57628700 1 -- 172.21.15.88:0/735993712 shutdown_connections 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.876+0000 7eff57628700 1 --2- 172.21.15.88:0/735993712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff500fee50 0x7eff50101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.876+0000 7eff57628700 1 --2- 172.21.15.88:0/735993712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff500fe500 0x7eff500fe910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.876+0000 7eff57628700 1 -- 172.21.15.88:0/735993712 >> 172.21.15.88:0/735993712 conn(0x7eff500f9f10 msgr2=0x7eff500fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.876+0000 7eff57628700 1 -- 172.21.15.88:0/735993712 shutdown_connections 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.876+0000 7eff57628700 1 -- 172.21.15.88:0/735993712 wait complete. 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff57628700 1 Processor -- start 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff57628700 1 -- start start 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff57628700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff500fe500 0x7eff50195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff57628700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff500fee50 0x7eff50196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff57628700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff50196660 con 0x7eff500fe500 2024-02-19T18:58:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff57628700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff501967a0 con 0x7eff500fee50 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff553c4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff500fe500 0x7eff50195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff553c4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff500fe500 0x7eff50195b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48576/0 (socket says 172.21.15.88:48576) 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff553c4700 1 -- 172.21.15.88:0/763745180 learned_addr learned my addr 172.21.15.88:0/763745180 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff54bc3700 1 --2- 172.21.15.88:0/763745180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff500fee50 0x7eff50196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff553c4700 1 -- 172.21.15.88:0/763745180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff500fee50 msgr2=0x7eff50196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff553c4700 1 --2- 172.21.15.88:0/763745180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff500fee50 0x7eff50196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff553c4700 1 -- 172.21.15.88:0/763745180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff400095f0 con 0x7eff500fe500 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff553c4700 1 --2- 172.21.15.88:0/763745180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff500fe500 0x7eff50195b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7eff40006010 tx=0x7eff4000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff467fc700 1 -- 172.21.15.88:0/763745180 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff4001d070 con 0x7eff500fe500 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff467fc700 1 -- 172.21.15.88:0/763745180 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7eff4000f460 con 0x7eff500fe500 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.877+0000 7eff57628700 1 -- 172.21.15.88:0/763745180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5019b1f0 con 0x7eff500fe500 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.878+0000 7eff467fc700 1 -- 172.21.15.88:0/763745180 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff40021640 con 0x7eff500fe500 2024-02-19T18:58:41.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.878+0000 7eff57628700 1 -- 172.21.15.88:0/763745180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5019b6e0 con 0x7eff500fe500 2024-02-19T18:58:41.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.878+0000 7eff57628700 1 -- 172.21.15.88:0/763745180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff50108d50 con 0x7eff500fe500 2024-02-19T18:58:41.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.879+0000 7eff467fc700 1 -- 172.21.15.88:0/763745180 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7eff4000fab0 con 0x7eff500fe500 2024-02-19T18:58:41.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.879+0000 7eff467fc700 1 --2- 172.21.15.88:0/763745180 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7eff3c06c680 0x7eff3c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:41.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.879+0000 7eff54bc3700 1 --2- 172.21.15.88:0/763745180 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7eff3c06c680 0x7eff3c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:41.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.879+0000 7eff467fc700 1 -- 172.21.15.88:0/763745180 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7eff4008cb00 con 0x7eff500fe500 2024-02-19T18:58:41.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.880+0000 7eff54bc3700 1 --2- 172.21.15.88:0/763745180 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7eff3c06c680 0x7eff3c06eb30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff4c005950 tx=0x7eff4c00b410 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:41.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:41.884+0000 7eff467fc700 1 -- 172.21.15.88:0/763745180 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7eff4005ac70 con 0x7eff500fe500 2024-02-19T18:58:42.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:41 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] up:boot 2024-02-19T18:58:42.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:41 smithi132 ceph-mon[37969]: daemon mds.cephfs.smithi132.afhgdi assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-19T18:58:42.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:41 smithi132 ceph-mon[37969]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-02-19T18:58:42.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:41 smithi132 ceph-mon[37969]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-19T18:58:42.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:41 smithi132 ceph-mon[37969]: fsmap cephfs:1 {0=cephfs.smithi088.ajgvpz=up:active} 1 up:standby 2024-02-19T18:58:42.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:41 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T18:58:42.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:41 smithi132 ceph-mon[37969]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:creating} 2024-02-19T18:58:42.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:41 smithi132 ceph-mon[37969]: daemon mds.cephfs.smithi132.afhgdi is now active in filesystem cephfs as rank 1 2024-02-19T18:58:42.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.127+0000 7eff57628700 1 -- 172.21.15.88:0/763745180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7eff50108f10 con 0x7eff500fe500 2024-02-19T18:58:42.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.962+0000 7eff467fc700 1 -- 172.21.15.88:0/763745180 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7eff4005a800 con 0x7eff500fe500 2024-02-19T18:58:42.981 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:42 smithi088 ceph-mon[29272]: pgmap v184: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 3 op/s 2024-02-19T18:58:42.981 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:42 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] up:active 2024-02-19T18:58:42.981 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:42 smithi088 ceph-mon[29272]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 2024-02-19T18:58:42.981 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:42 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/763745180' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-19T18:58:42.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.964+0000 7eff57628700 1 -- 172.21.15.88:0/763745180 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7eff3c06c680 msgr2=0x7eff3c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:42.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.964+0000 7eff57628700 1 --2- 172.21.15.88:0/763745180 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7eff3c06c680 0x7eff3c06eb30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7eff4c005950 tx=0x7eff4c00b410 comp rx=0 tx=0).stop 2024-02-19T18:58:42.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.964+0000 7eff57628700 1 -- 172.21.15.88:0/763745180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff500fe500 msgr2=0x7eff50195b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:42.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.964+0000 7eff57628700 1 --2- 172.21.15.88:0/763745180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff500fe500 0x7eff50195b00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7eff40006010 tx=0x7eff4000bd70 comp rx=0 tx=0).stop 2024-02-19T18:58:42.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.964+0000 7eff57628700 1 -- 172.21.15.88:0/763745180 shutdown_connections 2024-02-19T18:58:42.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.964+0000 7eff57628700 1 --2- 172.21.15.88:0/763745180 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7eff3c06c680 0x7eff3c06eb30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:42.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.965+0000 7eff57628700 1 --2- 172.21.15.88:0/763745180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff500fe500 0x7eff50195b00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:42.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.965+0000 7eff57628700 1 --2- 172.21.15.88:0/763745180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff500fee50 0x7eff50196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:42.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.965+0000 7eff57628700 1 -- 172.21.15.88:0/763745180 >> 172.21.15.88:0/763745180 conn(0x7eff500f9f10 msgr2=0x7eff501047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:42.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.965+0000 7eff57628700 1 -- 172.21.15.88:0/763745180 shutdown_connections 2024-02-19T18:58:42.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:42.965+0000 7eff57628700 1 -- 172.21.15.88:0/763745180 wait complete. 2024-02-19T18:58:43.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:42 smithi132 ceph-mon[37969]: pgmap v184: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 3 op/s 2024-02-19T18:58:43.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:42 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] up:active 2024-02-19T18:58:43.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:42 smithi132 ceph-mon[37969]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 2024-02-19T18:58:43.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:42 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/763745180' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-02-19T18:58:43.870 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-19T18:58:43.880 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T18:58:43.881 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph fs set cephfs inline_data false' 2024-02-19T18:58:44.132 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:43 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/763745180' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-19T18:58:44.132 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:43 smithi088 ceph-mon[29272]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 2024-02-19T18:58:44.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:43 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/763745180' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-02-19T18:58:44.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:43 smithi132 ceph-mon[37969]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 2024-02-19T18:58:44.509 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:45.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:44 smithi132 ceph-mon[37969]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-02-19T18:58:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:44 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:44 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:44 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:44 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.uyjxaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T18:58:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:44 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.uyjxaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T18:58:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:44 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:44 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:45.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:44 smithi088 ceph-mon[29272]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 7 op/s 2024-02-19T18:58:45.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:45.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:45.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:45.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.uyjxaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T18:58:45.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.uyjxaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-19T18:58:45.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:45.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:44 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:46.222 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:45 smithi088 ceph-mon[29272]: Deploying daemon mds.cephfs.smithi132.uyjxaz on smithi132 2024-02-19T18:58:46.222 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:45 smithi088 ceph-mon[29272]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-19T18:58:46.222 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:45 smithi088 ceph-mon[29272]: Cluster is now healthy 2024-02-19T18:58:46.222 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:45 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:boot 2024-02-19T18:58:46.222 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:45 smithi088 ceph-mon[29272]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 1 up:standby 2024-02-19T18:58:46.222 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:45 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T18:58:46.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:45 smithi132 ceph-mon[37969]: Deploying daemon mds.cephfs.smithi132.uyjxaz on smithi132 2024-02-19T18:58:46.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:45 smithi132 ceph-mon[37969]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-02-19T18:58:46.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:45 smithi132 ceph-mon[37969]: Cluster is now healthy 2024-02-19T18:58:46.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:45 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:boot 2024-02-19T18:58:46.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:45 smithi132 ceph-mon[37969]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 1 up:standby 2024-02-19T18:58:46.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:45 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T18:58:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.309+0000 7f9ae9c82700 1 -- 172.21.15.88:0/2004578127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4103680 msgr2=0x7f9ae4105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.309+0000 7f9ae9c82700 1 --2- 172.21.15.88:0/2004578127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4103680 0x7f9ae4105a60 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009910 tx=0x7f9acc009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.309+0000 7f9ae9c82700 1 -- 172.21.15.88:0/2004578127 shutdown_connections 2024-02-19T18:58:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.309+0000 7f9ae9c82700 1 --2- 172.21.15.88:0/2004578127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4103680 0x7f9ae4105a60 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.309+0000 7f9ae9c82700 1 --2- 172.21.15.88:0/2004578127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ae4100d60 0x7f9ae4103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.309+0000 7f9ae9c82700 1 -- 172.21.15.88:0/2004578127 >> 172.21.15.88:0/2004578127 conn(0x7f9ae40fa780 msgr2=0x7f9ae40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.309+0000 7f9ae9c82700 1 -- 172.21.15.88:0/2004578127 shutdown_connections 2024-02-19T18:58:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.309+0000 7f9ae9c82700 1 -- 172.21.15.88:0/2004578127 wait complete. 2024-02-19T18:58:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.309+0000 7f9ae9c82700 1 Processor -- start 2024-02-19T18:58:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9ae9c82700 1 -- start start 2024-02-19T18:58:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9ae9c82700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ae4100d60 0x7f9ae4195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9ae9c82700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4103680 0x7f9ae4196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9ae9c82700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae4196640 con 0x7f9ae4103680 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9ae9c82700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae4196780 con 0x7f9ae4100d60 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9adaffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4103680 0x7f9ae4196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9adaffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4103680 0x7f9ae4196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38128/0 (socket says 172.21.15.88:38128) 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9adaffd700 1 -- 172.21.15.88:0/4162847074 learned_addr learned my addr 172.21.15.88:0/4162847074 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9ae37fe700 1 --2- 172.21.15.88:0/4162847074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ae4100d60 0x7f9ae4195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9adaffd700 1 -- 172.21.15.88:0/4162847074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ae4100d60 msgr2=0x7f9ae4195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9adaffd700 1 --2- 172.21.15.88:0/4162847074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ae4100d60 0x7f9ae4195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9adaffd700 1 -- 172.21.15.88:0/4162847074 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0095f0 con 0x7f9ae4103680 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.310+0000 7f9adaffd700 1 --2- 172.21.15.88:0/4162847074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4103680 0x7f9ae4196020 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9acc006010 tx=0x7f9acc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.311+0000 7f9ae17fa700 1 -- 172.21.15.88:0/4162847074 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9acc01d070 con 0x7f9ae4103680 2024-02-19T18:58:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.311+0000 7f9ae17fa700 1 -- 172.21.15.88:0/4162847074 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9acc00f460 con 0x7f9ae4103680 2024-02-19T18:58:46.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.311+0000 7f9ae9c82700 1 -- 172.21.15.88:0/4162847074 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae419b1d0 con 0x7f9ae4103680 2024-02-19T18:58:46.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.311+0000 7f9ae17fa700 1 -- 172.21.15.88:0/4162847074 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9acc021640 con 0x7f9ae4103680 2024-02-19T18:58:46.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.311+0000 7f9ae9c82700 1 -- 172.21.15.88:0/4162847074 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae419b6c0 con 0x7f9ae4103680 2024-02-19T18:58:46.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.312+0000 7f9ae9c82700 1 -- 172.21.15.88:0/4162847074 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae418fbe0 con 0x7f9ae4103680 2024-02-19T18:58:46.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.312+0000 7f9ae17fa700 1 -- 172.21.15.88:0/4162847074 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f9acc00fab0 con 0x7f9ae4103680 2024-02-19T18:58:46.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.312+0000 7f9ae17fa700 1 --2- 172.21.15.88:0/4162847074 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9ad006c680 0x7f9ad006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:46.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.313+0000 7f9ae17fa700 1 -- 172.21.15.88:0/4162847074 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f9acc08cb30 con 0x7f9ae4103680 2024-02-19T18:58:46.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.313+0000 7f9ae37fe700 1 --2- 172.21.15.88:0/4162847074 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9ad006c680 0x7f9ad006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:46.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.313+0000 7f9ae37fe700 1 --2- 172.21.15.88:0/4162847074 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9ad006c680 0x7f9ad006eb30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4005950 tx=0x7f9ad400b410 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:46.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.317+0000 7f9ae17fa700 1 -- 172.21.15.88:0/4162847074 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9acc05ac40 con 0x7f9ae4103680 2024-02-19T18:58:46.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.553+0000 7f9ae9c82700 1 -- 172.21.15.88:0/4162847074 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f9ae4061960 con 0x7f9ae4103680 2024-02-19T18:58:46.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.979+0000 7f9ae17fa700 1 -- 172.21.15.88:0/4162847074 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v11) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9acc05a7d0 con 0x7f9ae4103680 2024-02-19T18:58:46.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.981+0000 7f9ae9c82700 1 -- 172.21.15.88:0/4162847074 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9ad006c680 msgr2=0x7f9ad006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.981+0000 7f9ae9c82700 1 --2- 172.21.15.88:0/4162847074 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9ad006c680 0x7f9ad006eb30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4005950 tx=0x7f9ad400b410 comp rx=0 tx=0).stop 2024-02-19T18:58:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.981+0000 7f9ae9c82700 1 -- 172.21.15.88:0/4162847074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4103680 msgr2=0x7f9ae4196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.981+0000 7f9ae9c82700 1 --2- 172.21.15.88:0/4162847074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4103680 0x7f9ae4196020 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9acc006010 tx=0x7f9acc00bd70 comp rx=0 tx=0).stop 2024-02-19T18:58:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.982+0000 7f9ae9c82700 1 -- 172.21.15.88:0/4162847074 shutdown_connections 2024-02-19T18:58:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.982+0000 7f9ae9c82700 1 --2- 172.21.15.88:0/4162847074 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9ad006c680 0x7f9ad006eb30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.982+0000 7f9ae9c82700 1 --2- 172.21.15.88:0/4162847074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9ae4100d60 0x7f9ae4195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.982+0000 7f9ae9c82700 1 --2- 172.21.15.88:0/4162847074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae4103680 0x7f9ae4196020 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.982+0000 7f9ae9c82700 1 -- 172.21.15.88:0/4162847074 >> 172.21.15.88:0/4162847074 conn(0x7f9ae40fa780 msgr2=0x7f9ae40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.982+0000 7f9ae9c82700 1 -- 172.21.15.88:0/4162847074 shutdown_connections 2024-02-19T18:58:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:46.982+0000 7f9ae9c82700 1 -- 172.21.15.88:0/4162847074 wait complete. 2024-02-19T18:58:47.000 INFO:teuthology.orchestra.run.smithi088.stderr:inline data disabled 2024-02-19T18:58:47.035 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:46 smithi132 ceph-mon[37969]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-02-19T18:58:47.035 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:46 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/4162847074' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-19T18:58:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:46 smithi088 ceph-mon[29272]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.3 KiB/s wr, 7 op/s 2024-02-19T18:58:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:46 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/4162847074' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-02-19T18:58:47.712 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-19T18:58:47.722 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T18:58:47.722 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-19T18:58:48.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:47 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/4162847074' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-19T18:58:48.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:47 smithi132 ceph-mon[37969]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 1 up:standby 2024-02-19T18:58:48.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:47 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:48.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:47 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:48.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:47 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:48.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:47 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:48.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:47 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:48.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:47 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:58:48.252 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:48.277 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:47 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/4162847074' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-02-19T18:58:48.277 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:47 smithi088 ceph-mon[29272]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 1 up:standby 2024-02-19T18:58:48.278 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:47 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:48.278 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:47 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:48.278 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:47 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:48.278 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:47 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:48.278 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:47 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:48.278 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:47 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:58:49.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:48 smithi088 ceph-mon[29272]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-02-19T18:58:49.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:48 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] up:boot 2024-02-19T18:58:49.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:48 smithi088 ceph-mon[29272]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:49.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T18:58:49.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:48 smithi132 ceph-mon[37969]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-02-19T18:58:49.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:48 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] up:boot 2024-02-19T18:58:49.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:48 smithi132 ceph-mon[37969]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:49.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T18:58:50.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:standby 2024-02-19T18:58:50.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] up:active 2024-02-19T18:58:50.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: Dropping low affinity active daemon mds.cephfs.smithi088.ajgvpz in favor of higher affinity standby. 2024-02-19T18:58:50.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: Replacing daemon mds.cephfs.smithi088.ajgvpz as rank 0 with standby daemon mds.cephfs.smithi088.psjiah 2024-02-19T18:58:50.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-19T18:58:50.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:50.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: osdmap e46: 6 total, 6 up, 6 in 2024-02-19T18:58:50.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: fsmap cephfs:2/2 {0=cephfs.smithi088.psjiah=up:replay,1=cephfs.smithi132.afhgdi=up:active} 1 up:standby 2024-02-19T18:58:50.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:50.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:50.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 2.3 KiB/s wr, 7 op/s 2024-02-19T18:58:50.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:50 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:standby 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] up:active 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: Dropping low affinity active daemon mds.cephfs.smithi088.ajgvpz in favor of higher affinity standby. 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: Replacing daemon mds.cephfs.smithi088.ajgvpz as rank 0 with standby daemon mds.cephfs.smithi088.psjiah 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: osdmap e46: 6 total, 6 up, 6 in 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: fsmap cephfs:2/2 {0=cephfs.smithi088.psjiah=up:replay,1=cephfs.smithi132.afhgdi=up:active} 1 up:standby 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 102 B/s rd, 2.3 KiB/s wr, 7 op/s 2024-02-19T18:58:50.533 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:50 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.961+0000 7f64df843700 1 -- 172.21.15.88:0/418652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d81003d0 msgr2=0x7f64d8100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.961+0000 7f64df843700 1 --2- 172.21.15.88:0/418652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d81003d0 0x7f64d8100840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009910 tx=0x7f64d4009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.961+0000 7f64df843700 1 -- 172.21.15.88:0/418652 shutdown_connections 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.961+0000 7f64df843700 1 --2- 172.21.15.88:0/418652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d81003d0 0x7f64d8100840 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.961+0000 7f64df843700 1 --2- 172.21.15.88:0/418652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d80ff130 0x7f64d80ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.961+0000 7f64df843700 1 -- 172.21.15.88:0/418652 >> 172.21.15.88:0/418652 conn(0x7f64d80fa780 msgr2=0x7f64d80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.961+0000 7f64df843700 1 -- 172.21.15.88:0/418652 shutdown_connections 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.961+0000 7f64df843700 1 -- 172.21.15.88:0/418652 wait complete. 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64df843700 1 Processor -- start 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64df843700 1 -- start start 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64df843700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d80ff130 0x7f64d8195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64df843700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81003d0 0x7f64d8195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64df843700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d81965c0 con 0x7f64d80ff130 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64df843700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d8196700 con 0x7f64d81003d0 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64dd5df700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d80ff130 0x7f64d8195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64dd5df700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d80ff130 0x7f64d8195a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38152/0 (socket says 172.21.15.88:38152) 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64dd5df700 1 -- 172.21.15.88:0/308170332 learned_addr learned my addr 172.21.15.88:0/308170332 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64dcdde700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81003d0 0x7f64d8195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64dd5df700 1 -- 172.21.15.88:0/308170332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81003d0 msgr2=0x7f64d8195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64dd5df700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81003d0 0x7f64d8195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64dd5df700 1 -- 172.21.15.88:0/308170332 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d40095f0 con 0x7f64d80ff130 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64dcdde700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81003d0 0x7f64d8195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.962+0000 7f64dd5df700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d80ff130 0x7f64d8195a60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f64c800d900 tx=0x7f64c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.963+0000 7f64ce7fc700 1 -- 172.21.15.88:0/308170332 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64c80041f0 con 0x7f64d80ff130 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.963+0000 7f64df843700 1 -- 172.21.15.88:0/308170332 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d819b1b0 con 0x7f64d80ff130 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.963+0000 7f64ce7fc700 1 -- 172.21.15.88:0/308170332 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f64c8004ce0 con 0x7f64d80ff130 2024-02-19T18:58:50.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.963+0000 7f64ce7fc700 1 -- 172.21.15.88:0/308170332 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64c800b750 con 0x7f64d80ff130 2024-02-19T18:58:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.963+0000 7f64df843700 1 -- 172.21.15.88:0/308170332 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d819b700 con 0x7f64d80ff130 2024-02-19T18:58:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.964+0000 7f64df843700 1 -- 172.21.15.88:0/308170332 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d80692b0 con 0x7f64d80ff130 2024-02-19T18:58:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.964+0000 7f64ce7fc700 1 -- 172.21.15.88:0/308170332 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f64c8004350 con 0x7f64d80ff130 2024-02-19T18:58:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.965+0000 7f64ce7fc700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f64c406c680 0x7f64c406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.965+0000 7f64ce7fc700 1 -- 172.21.15.88:0/308170332 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f64c808b760 con 0x7f64d80ff130 2024-02-19T18:58:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.965+0000 7f64dcdde700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f64c406c680 0x7f64c406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.965+0000 7f64dcdde700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f64c406c680 0x7f64c406eb30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f64d400b5c0 tx=0x7f64d4005960 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:50.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:50.970+0000 7f64ce7fc700 1 -- 172.21.15.88:0/308170332 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f64c80598f0 con 0x7f64d80ff130 2024-02-19T18:58:51.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.212+0000 7f64df843700 1 -- 172.21.15.88:0/308170332 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f64d819be00 con 0x7f64d80ff130 2024-02-19T18:58:51.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.212+0000 7f64ce7fc700 1 -- 172.21.15.88:0/308170332 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 16 v16) v1 ==== 76+0+1840 (secure 0 0 0) 0x7f64c8016070 con 0x7f64d80ff130 2024-02-19T18:58:51.231 INFO:teuthology.orchestra.run.smithi088.stdout:e16 2024-02-19T18:58:51.231 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T18:58:51.231 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T18:58:51.231 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 16 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T18:58:51.201809+0000 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T18:58:51.232 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T18:58:51.233 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:reconnect seq 4 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T18:58:51.234 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T18:58:51.234 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:58:51.234 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:58:51.234 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T18:58:51.234 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:58:51.234 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T18:58:51.234 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.215+0000 7f64df843700 1 -- 172.21.15.88:0/308170332 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f64c406c680 msgr2=0x7f64c406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.215+0000 7f64df843700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f64c406c680 0x7f64c406eb30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f64d400b5c0 tx=0x7f64d4005960 comp rx=0 tx=0).stop 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.215+0000 7f64df843700 1 -- 172.21.15.88:0/308170332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d80ff130 msgr2=0x7f64d8195a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.215+0000 7f64df843700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d80ff130 0x7f64d8195a60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f64c800d900 tx=0x7f64c800dcc0 comp rx=0 tx=0).stop 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.215+0000 7f64df843700 1 -- 172.21.15.88:0/308170332 shutdown_connections 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.215+0000 7f64df843700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f64c406c680 0x7f64c406eb30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.215+0000 7f64df843700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d80ff130 0x7f64d8195a60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.215+0000 7f64df843700 1 --2- 172.21.15.88:0/308170332 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81003d0 0x7f64d8195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.215+0000 7f64df843700 1 -- 172.21.15.88:0/308170332 >> 172.21.15.88:0/308170332 conn(0x7f64d80fa780 msgr2=0x7f64d8103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.215+0000 7f64df843700 1 -- 172.21.15.88:0/308170332 shutdown_connections 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:51.215+0000 7f64df843700 1 -- 172.21.15.88:0/308170332 wait complete. 2024-02-19T18:58:51.235 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 16 2024-02-19T18:58:51.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:51 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:resolve 2024-02-19T18:58:51.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:51 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] up:boot 2024-02-19T18:58:51.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:51 smithi132 ceph-mon[37969]: fsmap cephfs:2/2 {0=cephfs.smithi088.psjiah=up:resolve,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:51.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:51 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T18:58:51.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:51 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:resolve 2024-02-19T18:58:51.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:51 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] up:boot 2024-02-19T18:58:51.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:51 smithi088 ceph-mon[29272]: fsmap cephfs:2/2 {0=cephfs.smithi088.psjiah=up:resolve,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:51.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:51 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T18:58:52.293 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:52 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:reconnect 2024-02-19T18:58:52.293 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:52 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] up:standby 2024-02-19T18:58:52.293 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:52 smithi088 ceph-mon[29272]: fsmap cephfs:2/2 {0=cephfs.smithi088.psjiah=up:reconnect,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:52.293 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:52 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/308170332' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T18:58:52.293 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:52 smithi088 ceph-mon[29272]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 819 B/s wr, 5 op/s 2024-02-19T18:58:52.312 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-02-19T18:58:52.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:52 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:reconnect 2024-02-19T18:58:52.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:52 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] up:standby 2024-02-19T18:58:52.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:52 smithi132 ceph-mon[37969]: fsmap cephfs:2/2 {0=cephfs.smithi088.psjiah=up:reconnect,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:52.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:52 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/308170332' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T18:58:52.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:52 smithi132 ceph-mon[37969]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 307 B/s rd, 819 B/s wr, 5 op/s 2024-02-19T18:58:52.990 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:53.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:53 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:rejoin 2024-02-19T18:58:53.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:53 smithi132 ceph-mon[37969]: fsmap cephfs:2/2 {0=cephfs.smithi088.psjiah=up:rejoin,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:53.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:53 smithi132 ceph-mon[37969]: daemon mds.cephfs.smithi088.psjiah is now active in filesystem cephfs as rank 0 2024-02-19T18:58:53.515 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:53 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:rejoin 2024-02-19T18:58:53.515 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:53 smithi088 ceph-mon[29272]: fsmap cephfs:2/2 {0=cephfs.smithi088.psjiah=up:rejoin,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:53.515 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:53 smithi088 ceph-mon[29272]: daemon mds.cephfs.smithi088.psjiah is now active in filesystem cephfs as rank 0 2024-02-19T18:58:54.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:54 smithi132 ceph-mon[37969]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-19T18:58:54.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:54 smithi132 ceph-mon[37969]: Cluster is now healthy 2024-02-19T18:58:54.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:54 smithi132 ceph-mon[37969]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:active 2024-02-19T18:58:54.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:54 smithi132 ceph-mon[37969]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:54.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:54 smithi132 ceph-mon[37969]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 819 B/s rd, 409 B/s wr, 1 op/s 2024-02-19T18:58:54.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:54 smithi088 ceph-mon[29272]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-19T18:58:54.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:54 smithi088 ceph-mon[29272]: Cluster is now healthy 2024-02-19T18:58:54.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:54 smithi088 ceph-mon[29272]: mds.? [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] up:active 2024-02-19T18:58:54.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:54 smithi088 ceph-mon[29272]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T18:58:54.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:54 smithi088 ceph-mon[29272]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 819 B/s rd, 409 B/s wr, 1 op/s 2024-02-19T18:58:55.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.190+0000 7ffac4801700 1 -- 172.21.15.88:0/3213322202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffac00fef20 msgr2=0x7ffac00ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:55.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.190+0000 7ffac4801700 1 --2- 172.21.15.88:0/3213322202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffac00fef20 0x7ffac00ff330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009910 tx=0x7ffab0009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:55.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.190+0000 7ffac4801700 1 -- 172.21.15.88:0/3213322202 shutdown_connections 2024-02-19T18:58:55.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.190+0000 7ffac4801700 1 --2- 172.21.15.88:0/3213322202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffac0100080 0x7ffac01004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:55.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.190+0000 7ffac4801700 1 --2- 172.21.15.88:0/3213322202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffac00fef20 0x7ffac00ff330 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:55.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.190+0000 7ffac4801700 1 -- 172.21.15.88:0/3213322202 >> 172.21.15.88:0/3213322202 conn(0x7ffac00fa4d0 msgr2=0x7ffac00fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:55.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.190+0000 7ffac4801700 1 -- 172.21.15.88:0/3213322202 shutdown_connections 2024-02-19T18:58:55.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.190+0000 7ffac4801700 1 -- 172.21.15.88:0/3213322202 wait complete. 2024-02-19T18:58:55.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.190+0000 7ffac4801700 1 Processor -- start 2024-02-19T18:58:55.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffac4801700 1 -- start start 2024-02-19T18:58:55.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffac4801700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffac00fef20 0x7ffac01957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:55.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffac4801700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffac0100080 0x7ffac0195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:55.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffac4801700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac0196350 con 0x7ffac00fef20 2024-02-19T18:58:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffac4801700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac0196490 con 0x7ffac0100080 2024-02-19T18:58:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffabed9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffac00fef20 0x7ffac01957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffabed9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffac00fef20 0x7ffac01957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38170/0 (socket says 172.21.15.88:38170) 2024-02-19T18:58:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffabed9d700 1 -- 172.21.15.88:0/3107419798 learned_addr learned my addr 172.21.15.88:0/3107419798 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffabe59c700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffac0100080 0x7ffac0195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffabed9d700 1 -- 172.21.15.88:0/3107419798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffac0100080 msgr2=0x7ffac0195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffabed9d700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffac0100080 0x7ffac0195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffabed9d700 1 -- 172.21.15.88:0/3107419798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab00095f0 con 0x7ffac00fef20 2024-02-19T18:58:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffabe59c700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffac0100080 0x7ffac0195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T18:58:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffabed9d700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffac00fef20 0x7ffac01957f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffab000bd70 tx=0x7ffab000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffaa7fff700 1 -- 172.21.15.88:0/3107419798 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffab001d070 con 0x7ffac00fef20 2024-02-19T18:58:55.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffaa7fff700 1 -- 172.21.15.88:0/3107419798 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffab0022470 con 0x7ffac00fef20 2024-02-19T18:58:55.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffac4801700 1 -- 172.21.15.88:0/3107419798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac019aee0 con 0x7ffac00fef20 2024-02-19T18:58:55.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.191+0000 7ffaa7fff700 1 -- 172.21.15.88:0/3107419798 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffab000f670 con 0x7ffac00fef20 2024-02-19T18:58:55.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.192+0000 7ffac4801700 1 -- 172.21.15.88:0/3107419798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac019b350 con 0x7ffac00fef20 2024-02-19T18:58:55.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.193+0000 7ffac4801700 1 -- 172.21.15.88:0/3107419798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac01079f0 con 0x7ffac00fef20 2024-02-19T18:58:55.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.193+0000 7ffaa7fff700 1 -- 172.21.15.88:0/3107419798 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ffab0022a50 con 0x7ffac00fef20 2024-02-19T18:58:55.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.193+0000 7ffaa7fff700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffaa806c680 0x7ffaa806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:55.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.193+0000 7ffaa7fff700 1 -- 172.21.15.88:0/3107419798 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ffab008cfb0 con 0x7ffac00fef20 2024-02-19T18:58:55.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.193+0000 7ffabe59c700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffaa806c680 0x7ffaa806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:55.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.194+0000 7ffabe59c700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffaa806c680 0x7ffaa806eb30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffab4005fd0 tx=0x7ffab4005dc0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.198+0000 7ffaa7fff700 1 -- 172.21.15.88:0/3107419798 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ffab005b170 con 0x7ffac00fef20 2024-02-19T18:58:55.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.452+0000 7ffac4801700 1 -- 172.21.15.88:0/3107419798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffac019b990 con 0x7ffac00fef20 2024-02-19T18:58:55.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.453+0000 7ffaa7fff700 1 -- 172.21.15.88:0/3107419798 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4760 (secure 0 0 0) 0x7ffab005ad00 con 0x7ffac00fef20 2024-02-19T18:58:55.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.455+0000 7ffac4801700 1 -- 172.21.15.88:0/3107419798 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffaa806c680 msgr2=0x7ffaa806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:55.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.455+0000 7ffac4801700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffaa806c680 0x7ffaa806eb30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ffab4005fd0 tx=0x7ffab4005dc0 comp rx=0 tx=0).stop 2024-02-19T18:58:55.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.455+0000 7ffac4801700 1 -- 172.21.15.88:0/3107419798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffac00fef20 msgr2=0x7ffac01957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:55.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.455+0000 7ffac4801700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffac00fef20 0x7ffac01957f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffab000bd70 tx=0x7ffab000be50 comp rx=0 tx=0).stop 2024-02-19T18:58:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.455+0000 7ffac4801700 1 -- 172.21.15.88:0/3107419798 shutdown_connections 2024-02-19T18:58:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.455+0000 7ffac4801700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffaa806c680 0x7ffaa806eb30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.455+0000 7ffac4801700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffac00fef20 0x7ffac01957f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.455+0000 7ffac4801700 1 --2- 172.21.15.88:0/3107419798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffac0100080 0x7ffac0195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.455+0000 7ffac4801700 1 -- 172.21.15.88:0/3107419798 >> 172.21.15.88:0/3107419798 conn(0x7ffac00fa4d0 msgr2=0x7ffac0068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.455+0000 7ffac4801700 1 -- 172.21.15.88:0/3107419798 shutdown_connections 2024-02-19T18:58:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:55.455+0000 7ffac4801700 1 -- 172.21.15.88:0/3107419798 wait complete. 2024-02-19T18:58:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 18 2024-02-19T18:58:55.485 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T18:58:55.952 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:55.953 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:58:55.953 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:55.953 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:55.953 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:55.953 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:58:55.953 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:55.953 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:58:55.953 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:55 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/3107419798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T18:58:56.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:56.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:58:56.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:56.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:56.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:58:56.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:58:56.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:56.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:58:56.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:55 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/3107419798' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T18:58:56.324 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-02-19T18:58:56.556 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:58:57.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:56 smithi132 ceph-mon[37969]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-19T18:58:57.008 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:56 smithi088 ceph-mon[29272]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-19T18:58:58.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.884+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2564108154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb84100240 msgr2=0x7fdb84100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:58.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.884+0000 7fdb89fb6700 1 --2- 172.21.15.88:0/2564108154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb84100240 0x7fdb84100650 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fdb78009910 tx=0x7fdb78009c20 comp rx=0 tx=0).stop 2024-02-19T18:58:58.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.884+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2564108154 shutdown_connections 2024-02-19T18:58:58.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.884+0000 7fdb89fb6700 1 --2- 172.21.15.88:0/2564108154 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb84101440 0x7fdb84101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:58.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.884+0000 7fdb89fb6700 1 --2- 172.21.15.88:0/2564108154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb84100240 0x7fdb84100650 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:58.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.884+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2564108154 >> 172.21.15.88:0/2564108154 conn(0x7fdb840fb7f0 msgr2=0x7fdb840fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:58.903 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:58 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:58.903 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:58 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:58.903 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:58 smithi088 ceph-mon[29272]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-19T18:58:58.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.884+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2564108154 shutdown_connections 2024-02-19T18:58:58.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.884+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2564108154 wait complete. 2024-02-19T18:58:58.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.885+0000 7fdb89fb6700 1 Processor -- start 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.885+0000 7fdb89fb6700 1 -- start start 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.885+0000 7fdb89fb6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb84100240 0x7fdb84074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.885+0000 7fdb89fb6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb84101440 0x7fdb84073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.885+0000 7fdb89fb6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb84073680 con 0x7fdb84101440 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.885+0000 7fdb89fb6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb840737c0 con 0x7fdb84100240 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.885+0000 7fdb82ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb84101440 0x7fdb84073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.885+0000 7fdb82ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb84101440 0x7fdb84073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52846/0 (socket says 172.21.15.88:52846) 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.885+0000 7fdb82ffd700 1 -- 172.21.15.88:0/2866614148 learned_addr learned my addr 172.21.15.88:0/2866614148 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.885+0000 7fdb837fe700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb84100240 0x7fdb84074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.886+0000 7fdb82ffd700 1 -- 172.21.15.88:0/2866614148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb84100240 msgr2=0x7fdb84074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.886+0000 7fdb82ffd700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb84100240 0x7fdb84074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:58.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.886+0000 7fdb82ffd700 1 -- 172.21.15.88:0/2866614148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb780095f0 con 0x7fdb84101440 2024-02-19T18:58:58.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.886+0000 7fdb837fe700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb84100240 0x7fdb84074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T18:58:58.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.886+0000 7fdb82ffd700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb84101440 0x7fdb84073140 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400b4b0 tx=0x7fdb7400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:58.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.886+0000 7fdb80ff9700 1 -- 172.21.15.88:0/2866614148 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb74011800 con 0x7fdb84101440 2024-02-19T18:58:58.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.886+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2866614148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb84073aa0 con 0x7fdb84101440 2024-02-19T18:58:58.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.886+0000 7fdb80ff9700 1 -- 172.21.15.88:0/2866614148 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb74011e10 con 0x7fdb84101440 2024-02-19T18:58:58.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.886+0000 7fdb80ff9700 1 -- 172.21.15.88:0/2866614148 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7400f2e0 con 0x7fdb84101440 2024-02-19T18:58:58.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.886+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2866614148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8419dbc0 con 0x7fdb84101440 2024-02-19T18:58:58.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.887+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2866614148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb84108e10 con 0x7fdb84101440 2024-02-19T18:58:58.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.887+0000 7fdb80ff9700 1 -- 172.21.15.88:0/2866614148 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fdb74010380 con 0x7fdb84101440 2024-02-19T18:58:58.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.888+0000 7fdb80ff9700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdb6c06c680 0x7fdb6c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:58:58.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.888+0000 7fdb80ff9700 1 -- 172.21.15.88:0/2866614148 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdb7408b530 con 0x7fdb84101440 2024-02-19T18:58:58.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.888+0000 7fdb837fe700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdb6c06c680 0x7fdb6c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:58:58.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.888+0000 7fdb837fe700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdb6c06c680 0x7fdb6c06eb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb78006010 tx=0x7fdb78005ab0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:58:58.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:58.893+0000 7fdb80ff9700 1 -- 172.21.15.88:0/2866614148 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdb74059640 con 0x7fdb84101440 2024-02-19T18:58:59.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:58 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:59.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:58 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:58:59.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:58 smithi132 ceph-mon[37969]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 5.2 KiB/s rd, 0 B/s wr, 4 op/s 2024-02-19T18:58:59.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.184+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2866614148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fdb8419e3a0 con 0x7fdb84101440 2024-02-19T18:58:59.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.185+0000 7fdb80ff9700 1 -- 172.21.15.88:0/2866614148 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v18) v1 ==== 78+0+83 (secure 0 0 0) 0x7fdb740591d0 con 0x7fdb84101440 2024-02-19T18:58:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.186+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2866614148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdb6c06c680 msgr2=0x7fdb6c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.186+0000 7fdb89fb6700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdb6c06c680 0x7fdb6c06eb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb78006010 tx=0x7fdb78005ab0 comp rx=0 tx=0).stop 2024-02-19T18:58:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.187+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2866614148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb84101440 msgr2=0x7fdb84073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:58:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.187+0000 7fdb89fb6700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb84101440 0x7fdb84073140 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400b4b0 tx=0x7fdb7400b870 comp rx=0 tx=0).stop 2024-02-19T18:58:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.187+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2866614148 shutdown_connections 2024-02-19T18:58:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.187+0000 7fdb89fb6700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fdb6c06c680 0x7fdb6c06eb30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.187+0000 7fdb89fb6700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb84100240 0x7fdb84074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.187+0000 7fdb89fb6700 1 --2- 172.21.15.88:0/2866614148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb84101440 0x7fdb84073140 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:58:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.187+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2866614148 >> 172.21.15.88:0/2866614148 conn(0x7fdb840fb7f0 msgr2=0x7fdb84104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:58:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.187+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2866614148 shutdown_connections 2024-02-19T18:58:59.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:58:59.187+0000 7fdb89fb6700 1 -- 172.21.15.88:0/2866614148 wait complete. 2024-02-19T18:58:59.215 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T18:58:59.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:58:59 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2866614148' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-19T18:58:59.885 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-02-19T18:58:59.898 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T18:59:00.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:58:59 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2866614148' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-02-19T18:59:00.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.039+0000 7fa97ba77640 1 -- 172.21.15.88:0/959944401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa974100830 msgr2=0x7fa974100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.039+0000 7fa97ba77640 1 --2- 172.21.15.88:0/959944401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa974100830 0x7fa974100ca0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa964009880 tx=0x7fa96402bce0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.039+0000 7fa97ba77640 1 -- 172.21.15.88:0/959944401 shutdown_connections 2024-02-19T18:59:00.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.039+0000 7fa97ba77640 1 --2- 172.21.15.88:0/959944401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa974100830 0x7fa974100ca0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.039+0000 7fa97ba77640 1 --2- 172.21.15.88:0/959944401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff640 0x7fa9740ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.039+0000 7fa97ba77640 1 -- 172.21.15.88:0/959944401 >> 172.21.15.88:0/959944401 conn(0x7fa9740fabc0 msgr2=0x7fa9740fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:00.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.039+0000 7fa97ba77640 1 -- 172.21.15.88:0/959944401 shutdown_connections 2024-02-19T18:59:00.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.039+0000 7fa97ba77640 1 -- 172.21.15.88:0/959944401 wait complete. 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa97ba77640 1 Processor -- start 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa97ba77640 1 -- start start 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa97ba77640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff640 0x7fa974194f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa97ba77640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa974100830 0x7fa974195460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa97ba77640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa974195a30 con 0x7fa974100830 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa97ba77640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa974195ba0 con 0x7fa9740ff640 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa979012640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa974100830 0x7fa974195460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa979012640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa974100830 0x7fa974195460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52868/0 (socket says 172.21.15.88:52868) 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa979012640 1 -- 172.21.15.88:0/2270496528 learned_addr learned my addr 172.21.15.88:0/2270496528 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa979813640 1 --2- 172.21.15.88:0/2270496528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff640 0x7fa974194f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa979012640 1 -- 172.21.15.88:0/2270496528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff640 msgr2=0x7fa974194f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa979012640 1 --2- 172.21.15.88:0/2270496528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff640 0x7fa974194f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa979012640 1 -- 172.21.15.88:0/2270496528 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa964009560 con 0x7fa974100830 2024-02-19T18:59:00.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.040+0000 7fa979012640 1 --2- 172.21.15.88:0/2270496528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa974100830 0x7fa974195460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa964009430 tx=0x7fa96402cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.041+0000 7fa96affd640 1 -- 172.21.15.88:0/2270496528 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa964004520 con 0x7fa974100830 2024-02-19T18:59:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.041+0000 7fa96affd640 1 -- 172.21.15.88:0/2270496528 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa9640054b0 con 0x7fa974100830 2024-02-19T18:59:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.041+0000 7fa97ba77640 1 -- 172.21.15.88:0/2270496528 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa97419a5d0 con 0x7fa974100830 2024-02-19T18:59:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.041+0000 7fa96affd640 1 -- 172.21.15.88:0/2270496528 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa964034440 con 0x7fa974100830 2024-02-19T18:59:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.041+0000 7fa97ba77640 1 -- 172.21.15.88:0/2270496528 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa97419ab50 con 0x7fa974100830 2024-02-19T18:59:00.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.042+0000 7fa97ba77640 1 -- 172.21.15.88:0/2270496528 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9741083f0 con 0x7fa974100830 2024-02-19T18:59:00.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.042+0000 7fa96affd640 1 -- 172.21.15.88:0/2270496528 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa964005000 con 0x7fa974100830 2024-02-19T18:59:00.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.042+0000 7fa96affd640 1 --2- 172.21.15.88:0/2270496528 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa96006c6b0 0x7fa96006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.043+0000 7fa96affd640 1 -- 172.21.15.88:0/2270496528 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa9640adfe0 con 0x7fa974100830 2024-02-19T18:59:00.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.043+0000 7fa979813640 1 --2- 172.21.15.88:0/2270496528 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa96006c6b0 0x7fa96006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:00.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.043+0000 7fa979813640 1 --2- 172.21.15.88:0/2270496528 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa96006c6b0 0x7fa96006eb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa974100690 tx=0x7fa970008040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:00.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.047+0000 7fa96affd640 1 -- 172.21.15.88:0/2270496528 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa964078660 con 0x7fa974100830 2024-02-19T18:59:00.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.217+0000 7fa97ba77640 1 -- 172.21.15.88:0/2270496528 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa97419ad70 con 0x7fa974100830 2024-02-19T18:59:00.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.218+0000 7fa96affd640 1 -- 172.21.15.88:0/2270496528 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15127 (secure 0 0 0) 0x7fa96407bc80 con 0x7fa974100830 2024-02-19T18:59:00.236 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:59:00.236 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","created":"2024-02-19T18:50:27.026977+0000","modified":"2024-02-19T18:58:49.196080+0000","last_up_change":"2024-02-19T18:57:14.993225+0000","last_in_change":"2024-02-19T18:56:55.518417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T18:55:57.936009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-19T18:58:31.129412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-19T18:58:31.420377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"787c5fec-c403-4975-92c3-8b68fcf9e399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6803","nonce":2929682937}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6805","nonce":2929682937}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6809","nonce":2929682937}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":2929682937},{"type":"v1","addr":"172.21.15.88:6807","nonce":2929682937}]},"public_addr":"172.21.15.88:6803/2929682937","cluster_addr":"172.21.15.88:6805/2929682937","heartbeat_back_addr":"172.21.15.88:6809/2929682937","heartbeat_front_addr":"172.21.15.88:6807/2929682937","state":["exists","up"]},{"osd":1,"uuid":"a459843c-43d7-480f-a76b-d52d6e8ca061","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6811","nonce":3005125945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6813","nonce":3005125945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6817","nonce":3005125945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":3005125945},{"type":"v1","addr":"172.21.15.88:6815","nonce":3005125945}]},"public_addr":"172.21.15.88:6811/3005125945","cluster_addr":"172.21.15.88:6813/3005125945","heartbeat_back_addr":"172.21.15.88:6817/3005125945","heartbeat_front_addr":"172.21.15.88:6815/3005125945","state":["exists","up"]},{"osd":2,"uuid":"66334f86-7411-4587-b96b-22b61bf5201a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6819","nonce":525694249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6821","nonce":525694249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6825","nonce":525694249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":525694249},{"type":"v1","addr":"172.21.15.88:6823","nonce":525694249}]},"public_addr":"172.21.15.88:6819/525694249","cluster_addr":"172.21.15.88:6821/525694249","heartbeat_back_addr":"172.21.15.88:6825/525694249","heartbeat_front_addr":"172.21.15.88:6823/525694249","state":["exists","up"]},{"osd":3,"uuid":"733318f2-be90-44d7-8c38-fb773f7da169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6801","nonce":2008345174}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6803","nonce":2008345174}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6807","nonce":2008345174}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2008345174},{"type":"v1","addr":"172.21.15.132:6805","nonce":2008345174}]},"public_addr":"172.21.15.132:6801/2008345174","cluster_addr":"172.21.15.132:6803/2008345174","heartbeat_back_addr":"172.21.15.132:6807/2008345174","heartbeat_front_addr":"172.21.15.132:6805/2008345174","state":["exists","up"]},{"osd":4,"uuid":"3b41c4ff-b4b1-4279-b973-0c3c1c62373e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6809","nonce":3138829446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6811","nonce":3138829446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6815","nonce":3138829446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":3138829446},{"type":"v1","addr":"172.21.15.132:6813","nonce":3138829446}]},"public_addr":"172.21.15.132:6809/3138829446","cluster_addr":"172.21.15.132:6811/3138829446","heartbeat_back_addr":"172.21.15.132:6815/3138829446","heartbeat_front_addr":"172.21.15.132:6813/3138829446","state":["exists","up"]},{"osd":5,"uuid":"367bd67a-e45d-40ff-9aac-1f88755641ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6817","nonce":2563696780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6819","nonce":2563696780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6823","nonce":2563696780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":2563696780},{"type":"v1","addr":"172.21.15.132:6821","nonce":2563696780}]},"public_addr":"172.21.15.132:6817/2563696780","cluster_addr":"172.21.15.132:6819/2563696780","heartbeat_back_addr":"172.21.15.132:6823/2563696780","heartbeat_front_addr":"172.21.15.132:6821/2563696780","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:54:58.653733+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:26.141001+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:55.352595+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:19.994273+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:46.399893+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:57:13.114456+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6827/694475098":"2024-02-20T18:58:49.196048+0000","172.21.15.88:0/387402116":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6801/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6801/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:0/3491684321":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6800/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6800/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:0/3838380819":"2024-02-20T18:51:05.484896+0000","172.21.15.88:0/2338689461":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/3237698093":"2024-02-20T18:51:05.484896+0000","172.21.15.88:0/2372050564":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6801/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/3565328320":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6800/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/2177498492":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/4069571809":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6826/694475098":"2024-02-20T18:58:49.196048+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T18:59:00.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.220+0000 7fa97ba77640 1 -- 172.21.15.88:0/2270496528 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa96006c6b0 msgr2=0x7fa96006eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.220+0000 7fa97ba77640 1 --2- 172.21.15.88:0/2270496528 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa96006c6b0 0x7fa96006eb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa974100690 tx=0x7fa970008040 comp rx=0 tx=0).stop 2024-02-19T18:59:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.220+0000 7fa97ba77640 1 -- 172.21.15.88:0/2270496528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa974100830 msgr2=0x7fa974195460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.220+0000 7fa97ba77640 1 --2- 172.21.15.88:0/2270496528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa974100830 0x7fa974195460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa964009430 tx=0x7fa96402cec0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.220+0000 7fa97ba77640 1 -- 172.21.15.88:0/2270496528 shutdown_connections 2024-02-19T18:59:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.220+0000 7fa97ba77640 1 --2- 172.21.15.88:0/2270496528 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa96006c6b0 0x7fa96006eb60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.220+0000 7fa97ba77640 1 --2- 172.21.15.88:0/2270496528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa974100830 0x7fa974195460 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.220+0000 7fa97ba77640 1 --2- 172.21.15.88:0/2270496528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa9740ff640 0x7fa974194f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.220+0000 7fa97ba77640 1 -- 172.21.15.88:0/2270496528 >> 172.21.15.88:0/2270496528 conn(0x7fa9740fabc0 msgr2=0x7fa974103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.220+0000 7fa97ba77640 1 -- 172.21.15.88:0/2270496528 shutdown_connections 2024-02-19T18:59:00.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.220+0000 7fa97ba77640 1 -- 172.21.15.88:0/2270496528 wait complete. 2024-02-19T18:59:00.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T18:55:57.936009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-19T18:58:31.129412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-19T18:58:31.420377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-19T18:59:00.251 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T18:59:00.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.347+0000 7ffaa7550640 1 -- 172.21.15.88:0/1211026528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa00fe560 msgr2=0x7ffaa00fe970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.347+0000 7ffaa7550640 1 --2- 172.21.15.88:0/1211026528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa00fe560 0x7ffaa00fe970 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009880 tx=0x7ffa9c02bce0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.347+0000 7ffaa7550640 1 -- 172.21.15.88:0/1211026528 shutdown_connections 2024-02-19T18:59:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.347+0000 7ffaa7550640 1 --2- 172.21.15.88:0/1211026528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaa00ff750 0x7ffaa00ffbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.347+0000 7ffaa7550640 1 --2- 172.21.15.88:0/1211026528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa00fe560 0x7ffaa00fe970 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa7550640 1 -- 172.21.15.88:0/1211026528 >> 172.21.15.88:0/1211026528 conn(0x7ffaa00f9ac0 msgr2=0x7ffaa00fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa7550640 1 -- 172.21.15.88:0/1211026528 shutdown_connections 2024-02-19T18:59:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa7550640 1 -- 172.21.15.88:0/1211026528 wait complete. 2024-02-19T18:59:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa7550640 1 Processor -- start 2024-02-19T18:59:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa7550640 1 -- start start 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa7550640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa00fe560 0x7ffaa0068ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa7550640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaa00ff750 0x7ffaa0199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa7550640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa019a2a0 con 0x7ffaa00fe560 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa7550640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa019a3e0 con 0x7ffaa00ff750 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa52ec640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa00fe560 0x7ffaa0068ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa52ec640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa00fe560 0x7ffaa0068ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52894/0 (socket says 172.21.15.88:52894) 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.348+0000 7ffaa52ec640 1 -- 172.21.15.88:0/3407917346 learned_addr learned my addr 172.21.15.88:0/3407917346 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.349+0000 7ffaa52ec640 1 -- 172.21.15.88:0/3407917346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaa00ff750 msgr2=0x7ffaa0199cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.349+0000 7ffaa52ec640 1 --2- 172.21.15.88:0/3407917346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaa00ff750 0x7ffaa0199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.349+0000 7ffaa52ec640 1 -- 172.21.15.88:0/3407917346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c009560 con 0x7ffaa00fe560 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.349+0000 7ffaa52ec640 1 --2- 172.21.15.88:0/3407917346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa00fe560 0x7ffaa0068ed0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009230 tx=0x7ffa9c004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.349+0000 7ffa967fc640 1 -- 172.21.15.88:0/3407917346 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa9c005000 con 0x7ffaa00fe560 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.349+0000 7ffaa7550640 1 -- 172.21.15.88:0/3407917346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa019a630 con 0x7ffaa00fe560 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.349+0000 7ffa967fc640 1 -- 172.21.15.88:0/3407917346 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffa9c005610 con 0x7ffaa00fe560 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.349+0000 7ffa967fc640 1 -- 172.21.15.88:0/3407917346 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa9c034440 con 0x7ffaa00fe560 2024-02-19T18:59:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.349+0000 7ffaa7550640 1 -- 172.21.15.88:0/3407917346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa019ab30 con 0x7ffaa00fe560 2024-02-19T18:59:00.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.350+0000 7ffaa7550640 1 -- 172.21.15.88:0/3407917346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa0107290 con 0x7ffaa00fe560 2024-02-19T18:59:00.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.350+0000 7ffa967fc640 1 -- 172.21.15.88:0/3407917346 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ffa9c003880 con 0x7ffaa00fe560 2024-02-19T18:59:00.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.351+0000 7ffa967fc640 1 --2- 172.21.15.88:0/3407917346 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffa8c06c5f0 0x7ffa8c06eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.351+0000 7ffa967fc640 1 -- 172.21.15.88:0/3407917346 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ffa9c0ade50 con 0x7ffaa00fe560 2024-02-19T18:59:00.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.351+0000 7ffaa4aeb640 1 --2- 172.21.15.88:0/3407917346 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffa8c06c5f0 0x7ffa8c06eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:00.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.351+0000 7ffaa4aeb640 1 --2- 172.21.15.88:0/3407917346 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffa8c06c5f0 0x7ffa8c06eaa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffa90005fd0 tx=0x7ffa90005850 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:00.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.355+0000 7ffa967fc640 1 -- 172.21.15.88:0/3407917346 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ffa9c0784d0 con 0x7ffaa00fe560 2024-02-19T18:59:00.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.529+0000 7ffaa7550640 1 -- 172.21.15.88:0/3407917346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffaa019af80 con 0x7ffaa00fe560 2024-02-19T18:59:00.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.530+0000 7ffa967fc640 1 -- 172.21.15.88:0/3407917346 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa9c041020 con 0x7ffaa00fe560 2024-02-19T18:59:00.548 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-02-19T18:59:00.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.532+0000 7ffaa7550640 1 -- 172.21.15.88:0/3407917346 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffa8c06c5f0 msgr2=0x7ffa8c06eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.532+0000 7ffaa7550640 1 --2- 172.21.15.88:0/3407917346 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffa8c06c5f0 0x7ffa8c06eaa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffa90005fd0 tx=0x7ffa90005850 comp rx=0 tx=0).stop 2024-02-19T18:59:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.532+0000 7ffaa7550640 1 -- 172.21.15.88:0/3407917346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa00fe560 msgr2=0x7ffaa0068ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.532+0000 7ffaa7550640 1 --2- 172.21.15.88:0/3407917346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa00fe560 0x7ffaa0068ed0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c009230 tx=0x7ffa9c004e30 comp rx=0 tx=0).stop 2024-02-19T18:59:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.532+0000 7ffaa7550640 1 -- 172.21.15.88:0/3407917346 shutdown_connections 2024-02-19T18:59:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.532+0000 7ffaa7550640 1 --2- 172.21.15.88:0/3407917346 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ffa8c06c5f0 0x7ffa8c06eaa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.532+0000 7ffaa7550640 1 --2- 172.21.15.88:0/3407917346 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffaa00ff750 0x7ffaa0199cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.532+0000 7ffaa7550640 1 --2- 172.21.15.88:0/3407917346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa00fe560 0x7ffaa0068ed0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.532+0000 7ffaa7550640 1 -- 172.21.15.88:0/3407917346 >> 172.21.15.88:0/3407917346 conn(0x7ffaa00f9ac0 msgr2=0x7ffaa0102980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.532+0000 7ffaa7550640 1 -- 172.21.15.88:0/3407917346 shutdown_connections 2024-02-19T18:59:00.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.532+0000 7ffaa7550640 1 -- 172.21.15.88:0/3407917346 wait complete. 2024-02-19T18:59:00.562 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-19T18:59:00.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.653+0000 7f70e573c640 1 -- 172.21.15.88:0/2641160589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e0102a60 msgr2=0x7f70e0102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.653+0000 7f70e573c640 1 --2- 172.21.15.88:0/2641160589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e0102a60 0x7f70e0102ed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009880 tx=0x7f70c802bce0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.653+0000 7f70e573c640 1 -- 172.21.15.88:0/2641160589 shutdown_connections 2024-02-19T18:59:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70e573c640 1 --2- 172.21.15.88:0/2641160589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e0102a60 0x7f70e0102ed0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70e573c640 1 --2- 172.21.15.88:0/2641160589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e0101870 0x7f70e0101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70e573c640 1 -- 172.21.15.88:0/2641160589 >> 172.21.15.88:0/2641160589 conn(0x7f70e00fce10 msgr2=0x7f70e00ff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70e573c640 1 -- 172.21.15.88:0/2641160589 shutdown_connections 2024-02-19T18:59:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70e573c640 1 -- 172.21.15.88:0/2641160589 wait complete. 2024-02-19T18:59:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70e573c640 1 Processor -- start 2024-02-19T18:59:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70e573c640 1 -- start start 2024-02-19T18:59:00.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70e573c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e0101870 0x7f70e01971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70e573c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e0102a60 0x7f70e01976e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70e573c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e0197cb0 con 0x7f70e0102a60 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70e573c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e0197e20 con 0x7f70e0101870 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e0102a60 0x7f70e01976e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70d7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e0102a60 0x7f70e01976e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52906/0 (socket says 172.21.15.88:52906) 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70d7fff640 1 -- 172.21.15.88:0/2762816750 learned_addr learned my addr 172.21.15.88:0/2762816750 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.654+0000 7f70deffd640 1 --2- 172.21.15.88:0/2762816750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e0101870 0x7f70e01971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.655+0000 7f70d7fff640 1 -- 172.21.15.88:0/2762816750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e0101870 msgr2=0x7f70e01971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.655+0000 7f70d7fff640 1 --2- 172.21.15.88:0/2762816750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e0101870 0x7f70e01971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.655+0000 7f70d7fff640 1 -- 172.21.15.88:0/2762816750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c8009560 con 0x7f70e0102a60 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.655+0000 7f70d7fff640 1 --2- 172.21.15.88:0/2762816750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e0102a60 0x7f70e01976e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009230 tx=0x7f70c802cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.655+0000 7f70dcff9640 1 -- 172.21.15.88:0/2762816750 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70c8004420 con 0x7f70e0102a60 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.655+0000 7f70dcff9640 1 -- 172.21.15.88:0/2762816750 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f70c80054b0 con 0x7f70e0102a60 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.655+0000 7f70e573c640 1 -- 172.21.15.88:0/2762816750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e019c850 con 0x7f70e0102a60 2024-02-19T18:59:00.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.655+0000 7f70dcff9640 1 -- 172.21.15.88:0/2762816750 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70c8034440 con 0x7f70e0102a60 2024-02-19T18:59:00.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.655+0000 7f70e573c640 1 -- 172.21.15.88:0/2762816750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e019cd50 con 0x7f70e0102a60 2024-02-19T18:59:00.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.656+0000 7f70e573c640 1 -- 172.21.15.88:0/2762816750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e010a5b0 con 0x7f70e0102a60 2024-02-19T18:59:00.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.656+0000 7f70dcff9640 1 -- 172.21.15.88:0/2762816750 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f70c80037d0 con 0x7f70e0102a60 2024-02-19T18:59:00.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.657+0000 7f70dcff9640 1 --2- 172.21.15.88:0/2762816750 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70cc06c5f0 0x7f70cc06eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.657+0000 7f70deffd640 1 --2- 172.21.15.88:0/2762816750 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70cc06c5f0 0x7f70cc06eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:00.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.657+0000 7f70dcff9640 1 -- 172.21.15.88:0/2762816750 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f70c80adc60 con 0x7f70e0102a60 2024-02-19T18:59:00.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.657+0000 7f70deffd640 1 --2- 172.21.15.88:0/2762816750 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70cc06c5f0 0x7f70cc06eaa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f70e01028c0 tx=0x7f70d0005f90 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:00.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.661+0000 7f70dcff9640 1 -- 172.21.15.88:0/2762816750 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f70c8078410 con 0x7f70e0102a60 2024-02-19T18:59:00.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.829+0000 7f70e573c640 1 -- 172.21.15.88:0/2762816750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f70e019cfc0 con 0x7f70e0102a60 2024-02-19T18:59:00.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.830+0000 7f70dcff9640 1 -- 172.21.15.88:0/2762816750 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f70c804b020 con 0x7f70e0102a60 2024-02-19T18:59:00.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:00 smithi088 ceph-mon[29272]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-19T18:59:00.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:00 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2270496528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T18:59:00.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:00 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/3407917346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T18:59:00.848 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-02-19T18:59:00.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.832+0000 7f70e573c640 1 -- 172.21.15.88:0/2762816750 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70cc06c5f0 msgr2=0x7f70cc06eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.832+0000 7f70e573c640 1 --2- 172.21.15.88:0/2762816750 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70cc06c5f0 0x7f70cc06eaa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f70e01028c0 tx=0x7f70d0005f90 comp rx=0 tx=0).stop 2024-02-19T18:59:00.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.832+0000 7f70e573c640 1 -- 172.21.15.88:0/2762816750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e0102a60 msgr2=0x7f70e01976e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.832+0000 7f70e573c640 1 --2- 172.21.15.88:0/2762816750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e0102a60 0x7f70e01976e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009230 tx=0x7f70c802cec0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.832+0000 7f70e573c640 1 -- 172.21.15.88:0/2762816750 shutdown_connections 2024-02-19T18:59:00.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.832+0000 7f70e573c640 1 --2- 172.21.15.88:0/2762816750 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70cc06c5f0 0x7f70cc06eaa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.832+0000 7f70e573c640 1 --2- 172.21.15.88:0/2762816750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e0102a60 0x7f70e01976e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.832+0000 7f70e573c640 1 --2- 172.21.15.88:0/2762816750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e0101870 0x7f70e01971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.832+0000 7f70e573c640 1 -- 172.21.15.88:0/2762816750 >> 172.21.15.88:0/2762816750 conn(0x7f70e00fce10 msgr2=0x7f70e0105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:00.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.832+0000 7f70e573c640 1 -- 172.21.15.88:0/2762816750 shutdown_connections 2024-02-19T18:59:00.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.832+0000 7f70e573c640 1 -- 172.21.15.88:0/2762816750 wait complete. 2024-02-19T18:59:00.861 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-19T18:59:00.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3012682480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88f4100840 msgr2=0x7f88f4100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 --2- 172.21.15.88:0/3012682480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88f4100840 0x7f88f4100cb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009880 tx=0x7f88e402bce0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3012682480 shutdown_connections 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 --2- 172.21.15.88:0/3012682480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88f4100840 0x7f88f4100cb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 --2- 172.21.15.88:0/3012682480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88f40ff650 0x7f88f40ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3012682480 >> 172.21.15.88:0/3012682480 conn(0x7f88f40fabd0 msgr2=0x7f88f40fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3012682480 shutdown_connections 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3012682480 wait complete. 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 Processor -- start 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 -- start start 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88f40ff650 0x7f88f4194f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88f4100840 0x7f88f4195490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4195a60 con 0x7f88f40ff650 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.947+0000 7f88fc5fb640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88f4195bd0 con 0x7f88f4100840 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88fa397640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88f40ff650 0x7f88f4194f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88fa397640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88f40ff650 0x7f88f4194f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52928/0 (socket says 172.21.15.88:52928) 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88fa397640 1 -- 172.21.15.88:0/3159680823 learned_addr learned my addr 172.21.15.88:0/3159680823 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88f9b96640 1 --2- 172.21.15.88:0/3159680823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88f4100840 0x7f88f4195490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88fa397640 1 -- 172.21.15.88:0/3159680823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88f4100840 msgr2=0x7f88f4195490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88fa397640 1 --2- 172.21.15.88:0/3159680823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88f4100840 0x7f88f4195490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88fa397640 1 -- 172.21.15.88:0/3159680823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e4009560 con 0x7f88f40ff650 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88fa397640 1 --2- 172.21.15.88:0/3159680823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88f40ff650 0x7f88f4194f50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00ea00 tx=0x7f88ec00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88eb7fe640 1 -- 172.21.15.88:0/3159680823 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88ec00cd50 con 0x7f88f40ff650 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3159680823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f419a660 con 0x7f88f40ff650 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88eb7fe640 1 -- 172.21.15.88:0/3159680823 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f88ec011920 con 0x7f88f40ff650 2024-02-19T18:59:00.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88eb7fe640 1 -- 172.21.15.88:0/3159680823 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88ec01aa30 con 0x7f88f40ff650 2024-02-19T18:59:00.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.948+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3159680823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88f419ab90 con 0x7f88f40ff650 2024-02-19T18:59:00.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.949+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3159680823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88f4108220 con 0x7f88f40ff650 2024-02-19T18:59:00.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.949+0000 7f88eb7fe640 1 -- 172.21.15.88:0/3159680823 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f88ec01ac60 con 0x7f88f40ff650 2024-02-19T18:59:00.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.950+0000 7f88eb7fe640 1 --2- 172.21.15.88:0/3159680823 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f88e006c5f0 0x7f88e006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:00.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.950+0000 7f88eb7fe640 1 -- 172.21.15.88:0/3159680823 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f88ec08bef0 con 0x7f88f40ff650 2024-02-19T18:59:00.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.950+0000 7f88f9b96640 1 --2- 172.21.15.88:0/3159680823 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f88e006c5f0 0x7f88e006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:00.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.950+0000 7f88f9b96640 1 --2- 172.21.15.88:0/3159680823 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f88e006c5f0 0x7f88e006eaa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f88e4006010 tx=0x7f88e402c700 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:00.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:00.954+0000 7f88eb7fe640 1 -- 172.21.15.88:0/3159680823 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f88ec056570 con 0x7f88f40ff650 2024-02-19T18:59:01.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:00 smithi132 ceph-mon[37969]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-02-19T18:59:01.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:00 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2270496528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T18:59:01.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:00 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/3407917346' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T18:59:01.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.129+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3159680823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f88f4108500 con 0x7f88f40ff650 2024-02-19T18:59:01.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.130+0000 7f88eb7fe640 1 -- 172.21.15.88:0/3159680823 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f88f4108500 con 0x7f88f40ff650 2024-02-19T18:59:01.148 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-02-19T18:59:01.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.132+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3159680823 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f88e006c5f0 msgr2=0x7f88e006eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:01.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.132+0000 7f88fc5fb640 1 --2- 172.21.15.88:0/3159680823 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f88e006c5f0 0x7f88e006eaa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f88e4006010 tx=0x7f88e402c700 comp rx=0 tx=0).stop 2024-02-19T18:59:01.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.132+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3159680823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88f40ff650 msgr2=0x7f88f4194f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:01.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.132+0000 7f88fc5fb640 1 --2- 172.21.15.88:0/3159680823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88f40ff650 0x7f88f4194f50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f88ec00ea00 tx=0x7f88ec00eec0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.132+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3159680823 shutdown_connections 2024-02-19T18:59:01.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.132+0000 7f88fc5fb640 1 --2- 172.21.15.88:0/3159680823 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f88e006c5f0 0x7f88e006eaa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.132+0000 7f88fc5fb640 1 --2- 172.21.15.88:0/3159680823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88f4100840 0x7f88f4195490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.132+0000 7f88fc5fb640 1 --2- 172.21.15.88:0/3159680823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88f40ff650 0x7f88f4194f50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.132+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3159680823 >> 172.21.15.88:0/3159680823 conn(0x7f88f40fabd0 msgr2=0x7f88f4103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.132+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3159680823 shutdown_connections 2024-02-19T18:59:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.132+0000 7f88fc5fb640 1 -- 172.21.15.88:0/3159680823 wait complete. 2024-02-19T18:59:01.161 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T18:59:01.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.244+0000 7ff26e97e640 1 -- 172.21.15.88:0/3814342977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2681029f0 msgr2=0x7ff268104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:01.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.244+0000 7ff26e97e640 1 --2- 172.21.15.88:0/3814342977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2681029f0 0x7ff268104dd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff2580098b0 tx=0x7ff25802bcd0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.244+0000 7ff26e97e640 1 -- 172.21.15.88:0/3814342977 shutdown_connections 2024-02-19T18:59:01.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.244+0000 7ff26e97e640 1 --2- 172.21.15.88:0/3814342977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2681029f0 0x7ff268104dd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.244+0000 7ff26e97e640 1 --2- 172.21.15.88:0/3814342977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2681000d0 0x7ff2681024b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.244+0000 7ff26e97e640 1 -- 172.21.15.88:0/3814342977 >> 172.21.15.88:0/3814342977 conn(0x7ff2680f9af0 msgr2=0x7ff2680fbf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:01.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.244+0000 7ff26e97e640 1 -- 172.21.15.88:0/3814342977 shutdown_connections 2024-02-19T18:59:01.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.244+0000 7ff26e97e640 1 -- 172.21.15.88:0/3814342977 wait complete. 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff26e97e640 1 Processor -- start 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff26e97e640 1 -- start start 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff26e97e640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2681000d0 0x7ff268194f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff26e97e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2681029f0 0x7ff268195460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff26e97e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2681959a0 con 0x7ff2681029f0 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff26e97e640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff268195b10 con 0x7ff2681000d0 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff2677fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2681029f0 0x7ff268195460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff2677fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2681029f0 0x7ff268195460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52944/0 (socket says 172.21.15.88:52944) 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff2677fe640 1 -- 172.21.15.88:0/293478174 learned_addr learned my addr 172.21.15.88:0/293478174 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff267fff640 1 --2- 172.21.15.88:0/293478174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2681000d0 0x7ff268194f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff2677fe640 1 -- 172.21.15.88:0/293478174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2681000d0 msgr2=0x7ff268194f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:01.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff2677fe640 1 --2- 172.21.15.88:0/293478174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2681000d0 0x7ff268194f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff2677fe640 1 -- 172.21.15.88:0/293478174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff258009560 con 0x7ff2681029f0 2024-02-19T18:59:01.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff2677fe640 1 --2- 172.21.15.88:0/293478174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2681029f0 0x7ff268195460 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff258009880 tx=0x7ff258004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:01.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.245+0000 7ff2657fa640 1 -- 172.21.15.88:0/293478174 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff258005000 con 0x7ff2681029f0 2024-02-19T18:59:01.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.246+0000 7ff26e97e640 1 -- 172.21.15.88:0/293478174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26819a580 con 0x7ff2681029f0 2024-02-19T18:59:01.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.246+0000 7ff2657fa640 1 -- 172.21.15.88:0/293478174 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff258005610 con 0x7ff2681029f0 2024-02-19T18:59:01.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.246+0000 7ff2657fa640 1 -- 172.21.15.88:0/293478174 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff258034670 con 0x7ff2681029f0 2024-02-19T18:59:01.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.246+0000 7ff26e97e640 1 -- 172.21.15.88:0/293478174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26819aa50 con 0x7ff2681029f0 2024-02-19T18:59:01.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.247+0000 7ff26e97e640 1 -- 172.21.15.88:0/293478174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26818f060 con 0x7ff2681029f0 2024-02-19T18:59:01.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.247+0000 7ff2657fa640 1 -- 172.21.15.88:0/293478174 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff2580038b0 con 0x7ff2681029f0 2024-02-19T18:59:01.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.247+0000 7ff2657fa640 1 --2- 172.21.15.88:0/293478174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff25006c6b0 0x7ff25006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:01.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.247+0000 7ff2657fa640 1 -- 172.21.15.88:0/293478174 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff2580ade80 con 0x7ff2681029f0 2024-02-19T18:59:01.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.247+0000 7ff267fff640 1 --2- 172.21.15.88:0/293478174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff25006c6b0 0x7ff25006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:01.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.248+0000 7ff267fff640 1 --2- 172.21.15.88:0/293478174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff25006c6b0 0x7ff25006eb60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff25c005e10 tx=0x7ff25c005da0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:01.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.252+0000 7ff2657fa640 1 -- 172.21.15.88:0/293478174 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff258078500 con 0x7ff2681029f0 2024-02-19T18:59:01.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.470+0000 7ff26e97e640 1 -- 172.21.15.88:0/293478174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff268108210 con 0x7ff2681029f0 2024-02-19T18:59:01.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.470+0000 7ff2657fa640 1 -- 172.21.15.88:0/293478174 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4760 (secure 0 0 0) 0x7ff25807bb20 con 0x7ff2681029f0 2024-02-19T18:59:01.488 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T18:59:01.488 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14522,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/2124144962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2124144962},{"type":"v1","addr":"172.21.15.88:6827","nonce":2124144962}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T18:58:53.212790+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14510,"mds_1":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14510":{"gid":14510,"name":"cephfs.smithi088.psjiah","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6829/2325623973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2325623973},{"type":"v1","addr":"172.21.15.88:6829","nonce":2325623973}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24263":{"gid":24263,"name":"cephfs.smithi132.afhgdi","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.132:6825/2841430379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2841430379},{"type":"v1","addr":"172.21.15.132:6825","nonce":2841430379}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T18:59:01.489 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 18 2024-02-19T18:59:01.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.472+0000 7ff26e97e640 1 -- 172.21.15.88:0/293478174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff25006c6b0 msgr2=0x7ff25006eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.472+0000 7ff26e97e640 1 --2- 172.21.15.88:0/293478174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff25006c6b0 0x7ff25006eb60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff25c005e10 tx=0x7ff25c005da0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.472+0000 7ff26e97e640 1 -- 172.21.15.88:0/293478174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2681029f0 msgr2=0x7ff268195460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.472+0000 7ff26e97e640 1 --2- 172.21.15.88:0/293478174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2681029f0 0x7ff268195460 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff258009880 tx=0x7ff258004f00 comp rx=0 tx=0).stop 2024-02-19T18:59:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.472+0000 7ff26e97e640 1 -- 172.21.15.88:0/293478174 shutdown_connections 2024-02-19T18:59:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.472+0000 7ff26e97e640 1 --2- 172.21.15.88:0/293478174 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff25006c6b0 0x7ff25006eb60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.472+0000 7ff26e97e640 1 --2- 172.21.15.88:0/293478174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2681029f0 0x7ff268195460 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.472+0000 7ff26e97e640 1 --2- 172.21.15.88:0/293478174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2681000d0 0x7ff268194f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.472+0000 7ff26e97e640 1 -- 172.21.15.88:0/293478174 >> 172.21.15.88:0/293478174 conn(0x7ff2680f9af0 msgr2=0x7ff2680fbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.472+0000 7ff26e97e640 1 -- 172.21.15.88:0/293478174 shutdown_connections 2024-02-19T18:59:01.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:01.472+0000 7ff26e97e640 1 -- 172.21.15.88:0/293478174 wait complete. 2024-02-19T18:59:01.501 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 2, 'flags': 18} 2024-02-19T18:59:01.501 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-02-19T18:59:01.528 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-02-19T18:59:01.528 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-02-19T18:59:01.529 INFO:tasks.ceph_fuse:client.0 config is {} 2024-02-19T18:59:01.529 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-19T18:59:01.529 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-19T18:59:01.529 INFO:tasks.ceph_fuse:client.1 config is {} 2024-02-19T18:59:01.529 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-19T18:59:01.529 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-19T18:59:01.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:01.529 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-02-19T18:59:01.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:01.549 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link delete ceph-brx 2024-02-19T18:59:01.625 INFO:teuthology.orchestra.run.smithi132.stderr:Cannot find device "ceph-brx" 2024-02-19T18:59:01.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T18:59:01.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:01.627 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-02-19T18:59:01.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:01.641 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-02-19T18:59:01.705 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:01 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2762816750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-19T18:59:01.705 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:01 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/3159680823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-19T18:59:01.705 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:01 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/293478174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T18:59:01.710 INFO:teuthology.orchestra.run.smithi088.stderr:Cannot find device "ceph-brx" 2024-02-19T18:59:01.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T18:59:01.712 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-02-19T18:59:01.712 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph fs ls 2024-02-19T18:59:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:59:02.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:01 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2762816750' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-19T18:59:02.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:01 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/3159680823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-19T18:59:02.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:01 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/293478174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T18:59:03.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:02 smithi088 ceph-mon[29272]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-19T18:59:03.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:02 smithi132 ceph-mon[37969]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.7 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-19T18:59:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.271+0000 7f94faaab700 1 -- 172.21.15.88:0/3993554546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f4100e80 msgr2=0x7f94f4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.271+0000 7f94faaab700 1 --2- 172.21.15.88:0/3993554546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f4100e80 0x7f94f4104e50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f94e0009910 tx=0x7f94e0009c20 comp rx=0 tx=0).stop 2024-02-19T18:59:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.271+0000 7f94faaab700 1 -- 172.21.15.88:0/3993554546 shutdown_connections 2024-02-19T18:59:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.271+0000 7f94faaab700 1 --2- 172.21.15.88:0/3993554546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f4100e80 0x7f94f4104e50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.271+0000 7f94faaab700 1 --2- 172.21.15.88:0/3993554546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94f41004e0 0x7f94f41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.271+0000 7f94faaab700 1 -- 172.21.15.88:0/3993554546 >> 172.21.15.88:0/3993554546 conn(0x7f94f40fbda0 msgr2=0x7f94f40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.271+0000 7f94faaab700 1 -- 172.21.15.88:0/3993554546 shutdown_connections 2024-02-19T18:59:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.271+0000 7f94faaab700 1 -- 172.21.15.88:0/3993554546 wait complete. 2024-02-19T18:59:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.272+0000 7f94faaab700 1 Processor -- start 2024-02-19T18:59:04.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.272+0000 7f94faaab700 1 -- start start 2024-02-19T18:59:04.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.272+0000 7f94faaab700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f41004e0 0x7f94f4195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:04.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.272+0000 7f94faaab700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94f4100e80 0x7f94f41964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:04.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.272+0000 7f94faaab700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f4196bb0 con 0x7f94f41004e0 2024-02-19T18:59:04.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.272+0000 7f94faaab700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f419a940 con 0x7f94f4100e80 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.272+0000 7f94f8847700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f41004e0 0x7f94f4195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.272+0000 7f94f8847700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f41004e0 0x7f94f4195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52952/0 (socket says 172.21.15.88:52952) 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.272+0000 7f94f8847700 1 -- 172.21.15.88:0/2409715245 learned_addr learned my addr 172.21.15.88:0/2409715245 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.272+0000 7f94f3fff700 1 --2- 172.21.15.88:0/2409715245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94f4100e80 0x7f94f41964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.273+0000 7f94f8847700 1 -- 172.21.15.88:0/2409715245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94f4100e80 msgr2=0x7f94f41964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.273+0000 7f94f8847700 1 --2- 172.21.15.88:0/2409715245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94f4100e80 0x7f94f41964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.273+0000 7f94f8847700 1 -- 172.21.15.88:0/2409715245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e00095f0 con 0x7f94f41004e0 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.273+0000 7f94f8847700 1 --2- 172.21.15.88:0/2409715245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f41004e0 0x7f94f4195f90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f94e800b4b0 tx=0x7f94e800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.273+0000 7f94f1ffb700 1 -- 172.21.15.88:0/2409715245 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e8011800 con 0x7f94f41004e0 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.273+0000 7f94faaab700 1 -- 172.21.15.88:0/2409715245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f419ac20 con 0x7f94f41004e0 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.273+0000 7f94f1ffb700 1 -- 172.21.15.88:0/2409715245 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f94e8011e10 con 0x7f94f41004e0 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.273+0000 7f94f1ffb700 1 -- 172.21.15.88:0/2409715245 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94e800f2e0 con 0x7f94f41004e0 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.273+0000 7f94faaab700 1 -- 172.21.15.88:0/2409715245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f419b170 con 0x7f94f41004e0 2024-02-19T18:59:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.274+0000 7f94faaab700 1 -- 172.21.15.88:0/2409715245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f4108780 con 0x7f94f41004e0 2024-02-19T18:59:04.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.275+0000 7f94f1ffb700 1 -- 172.21.15.88:0/2409715245 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f94e8010380 con 0x7f94f41004e0 2024-02-19T18:59:04.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.275+0000 7f94f1ffb700 1 --2- 172.21.15.88:0/2409715245 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f94e406c680 0x7f94e406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:04.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.275+0000 7f94f1ffb700 1 -- 172.21.15.88:0/2409715245 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f94e808b4a0 con 0x7f94f41004e0 2024-02-19T18:59:04.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.275+0000 7f94f3fff700 1 --2- 172.21.15.88:0/2409715245 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f94e406c680 0x7f94e406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:04.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.275+0000 7f94f3fff700 1 --2- 172.21.15.88:0/2409715245 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f94e406c680 0x7f94e406eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f94f41975b0 tx=0x7f94e001a040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:04.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.279+0000 7f94f1ffb700 1 -- 172.21.15.88:0/2409715245 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f94e8055b20 con 0x7f94f41004e0 2024-02-19T18:59:04.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.513+0000 7f94faaab700 1 -- 172.21.15.88:0/2409715245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f94f419b3f0 con 0x7f94f41004e0 2024-02-19T18:59:04.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.513+0000 7f94f1ffb700 1 -- 172.21.15.88:0/2409715245 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7f94e8059140 con 0x7f94f41004e0 2024-02-19T18:59:04.531 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-19T18:59:04.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.515+0000 7f94faaab700 1 -- 172.21.15.88:0/2409715245 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f94e406c680 msgr2=0x7f94e406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.515+0000 7f94faaab700 1 --2- 172.21.15.88:0/2409715245 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f94e406c680 0x7f94e406eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f94f41975b0 tx=0x7f94e001a040 comp rx=0 tx=0).stop 2024-02-19T18:59:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.515+0000 7f94faaab700 1 -- 172.21.15.88:0/2409715245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f41004e0 msgr2=0x7f94f4195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.515+0000 7f94faaab700 1 --2- 172.21.15.88:0/2409715245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f41004e0 0x7f94f4195f90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f94e800b4b0 tx=0x7f94e800b7c0 comp rx=0 tx=0).stop 2024-02-19T18:59:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.515+0000 7f94faaab700 1 -- 172.21.15.88:0/2409715245 shutdown_connections 2024-02-19T18:59:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.515+0000 7f94faaab700 1 --2- 172.21.15.88:0/2409715245 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f94e406c680 0x7f94e406eb30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.515+0000 7f94faaab700 1 --2- 172.21.15.88:0/2409715245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94f41004e0 0x7f94f4195f90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.515+0000 7f94faaab700 1 --2- 172.21.15.88:0/2409715245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94f4100e80 0x7f94f41964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.515+0000 7f94faaab700 1 -- 172.21.15.88:0/2409715245 >> 172.21.15.88:0/2409715245 conn(0x7f94f40fbda0 msgr2=0x7f94f40fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.516+0000 7f94faaab700 1 -- 172.21.15.88:0/2409715245 shutdown_connections 2024-02-19T18:59:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:04.516+0000 7f94faaab700 1 -- 172.21.15.88:0/2409715245 wait complete. 2024-02-19T18:59:05.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:04 smithi088 ceph-mon[29272]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-19T18:59:05.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:59:05.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:59:05.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:59:05.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:59:05.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:04 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:59:05.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:04 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2409715245' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-19T18:59:05.163 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-19T18:59:05.163 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-19T18:59:05.164 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-02-19T18:59:05.164 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-19T18:59:05.164 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-19T18:59:05.164 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-19T18:59:05.164 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-19T18:59:05.164 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-19T18:59:05.164 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-19T18:59:05.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:05.164 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-02-19T18:59:05.181 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-19T18:59:05.181 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-19T18:59:05.181 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-02-19T18:59:05.181 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42119sec preferred_lft 42119sec 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link 2024-02-19T18:59:05.182 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-02-19T18:59:05.183 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-19T18:59:05.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:05.183 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2024-02-19T18:59:05.183 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-02-19T18:59:05.183 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add name ceph-brx type bridge 2024-02-19T18:59:05.183 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr flush dev ceph-brx 2024-02-19T18:59:05.183 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set ceph-brx up 2024-02-19T18:59:05.183 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-19T18:59:05.183 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-02-19T18:59:05.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:04 smithi132 ceph-mon[37969]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.6 KiB/s rd, 0 B/s wr, 6 op/s 2024-02-19T18:59:05.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:04 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:59:05.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:04 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:59:05.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:04 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:59:05.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:04 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:59:05.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:04 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:59:05.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:04 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2409715245' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-19T18:59:05.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:05.354 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-19T18:59:05.380 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-02-19T18:59:05.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:05.382 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-02-19T18:59:05.450 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-02-19T18:59:05.450 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-19T18:59:05.450 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-19T18:59:05.450 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-19T18:59:05.450 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-19T18:59:05.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:05.451 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2024-02-19T18:59:05.451 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-02-19T18:59:05.451 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-19T18:59:05.451 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-19T18:59:05.451 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-19T18:59:05.451 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-02-19T18:59:05.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:05.725 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-02-19T18:59:05.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:05.740 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2024-02-19T18:59:05.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:05.794 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2024-02-19T18:59:05.794 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-02-19T18:59:05.794 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-19T18:59:05.794 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-19T18:59:05.794 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-02-19T18:59:05.878 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-19T18:59:05.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:05.878 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2024-02-19T18:59:05.878 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-02-19T18:59:05.878 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-19T18:59:05.878 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-19T18:59:05.878 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-19T18:59:05.878 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-19T18:59:05.878 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-19T18:59:05.878 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-02-19T18:59:06.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:06.011 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec bash -c ' 2024-02-19T18:59:06.011 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-02-19T18:59:06.012 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-02-19T18:59:06.012 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.0 master ceph-brx 2024-02-19T18:59:06.012 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-02-19T18:59:06.053 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-02-19T18:59:06.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-19T18:59:06.053 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-19T18:59:06.107 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-19T18:59:06.108 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-19T18:59:06.108 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-19T18:59:06.162 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-02-19T18:59:06.229 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-02-19T18:59:06.286 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-02-19T18:59:06.287 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/d7bbd6ac72545f4a4eb32a3489844e41d62f0d62bf82db65f67803bec4be8b4e/merged 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/5d2b7a71927fc01a7b10b589493a1d5db584a41e28c51b915de04ad8992d67e5/merged 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/0 2024-02-19T18:59:06.288 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/5c34d64d2d09eecba62687a66661bcb850be7d3db0ca7efeb77b5dcbdbfc2c11/merged 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/117bf9a782ea12e98bc24546c1faa7b1551c76bbfba52dbb459e92c3925c7a61/merged 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/2dd743f46047e64876fecd6aebea6191db3dd7a27716d5d476b307dd4023f652/merged 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/d4dfa416007b7c0b1e6ea52ecca90ec7410b58fd185ecfb3c5ea4bb92c8cd9e2/merged 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/da7a88312014e287f90f2f27e57ac4cdf60a72205b89e8884123122e12d76d2e/merged 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/cd84d25389fe4ec061b3a6c7121fcbd651877313f2ce9dbb090536a4c80e4a03/merged 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/7cd50f90373cd44cc9848e75d17051c19b34a0a88863414d5337fdfc755ff2bf/merged 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/380006b1a61a6ac178c1eb7abd7c0884575deee1abc5ec5a3836edc9884ae746/merged 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/cdb6660dbdbf86aa75a6c39fd5d66a7629e3fa3f9aa2fb353cc138ff7b6c0e4a/merged 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/d9f91bfc0de287c320c79f1e92fe8a8c11ac6a4a6109af04cde5d2e7d72c32e8/merged 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/013cec26c3f5544ef85f0821638715a6981ec3a6181adb6a2185f721fff0a048/merged 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-19T18:59:06.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:06.290 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-02-19T18:59:06.341 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-19T18:59:06.342 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-02-19T18:59:06.384 DEBUG:teuthology.orchestra.run.smithi088:> sudo modprobe fuse 2024-02-19T18:59:06.410 DEBUG:teuthology.orchestra.run.smithi088:> cat /proc/self/mounts | awk '{print $2}' 2024-02-19T18:59:06.465 INFO:teuthology.orchestra.run.smithi088.stdout:/proc 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/sys 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/dev 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/security 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/shm 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/pts 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/run 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/cgroup 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/pstore 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/bpf 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/ 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/selinux 2024-02-19T18:59:06.466 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/hugepages 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/dev/mqueue 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/debug 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/tracing 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/fs/fuse/connections 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/sys/kernel/config 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-sysctl.service 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/nfs/rpc_pipefs 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/1000 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/ceph 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/d7bbd6ac72545f4a4eb32a3489844e41d62f0d62bf82db65f67803bec4be8b4e/merged 2024-02-19T18:59:06.467 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/5d2b7a71927fc01a7b10b589493a1d5db584a41e28c51b915de04ad8992d67e5/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/run/user/0 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/5c34d64d2d09eecba62687a66661bcb850be7d3db0ca7efeb77b5dcbdbfc2c11/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/117bf9a782ea12e98bc24546c1faa7b1551c76bbfba52dbb459e92c3925c7a61/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/2dd743f46047e64876fecd6aebea6191db3dd7a27716d5d476b307dd4023f652/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/d4dfa416007b7c0b1e6ea52ecca90ec7410b58fd185ecfb3c5ea4bb92c8cd9e2/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/da7a88312014e287f90f2f27e57ac4cdf60a72205b89e8884123122e12d76d2e/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/cd84d25389fe4ec061b3a6c7121fcbd651877313f2ce9dbb090536a4c80e4a03/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/7cd50f90373cd44cc9848e75d17051c19b34a0a88863414d5337fdfc755ff2bf/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/380006b1a61a6ac178c1eb7abd7c0884575deee1abc5ec5a3836edc9884ae746/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/cdb6660dbdbf86aa75a6c39fd5d66a7629e3fa3f9aa2fb353cc138ff7b6c0e4a/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/d9f91bfc0de287c320c79f1e92fe8a8c11ac6a4a6109af04cde5d2e7d72c32e8/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/var/lib/containers/storage/overlay/013cec26c3f5544ef85f0821638715a6981ec3a6181adb6a2185f721fff0a048/merged 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns 2024-02-19T18:59:06.468 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-19T18:59:06.469 INFO:teuthology.orchestra.run.smithi088.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-19T18:59:06.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:06.469 DEBUG:teuthology.orchestra.run.smithi088:> ls /sys/fs/fuse/connections 2024-02-19T18:59:06.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:2024-02-19T18:59:06.485+0000 7f620c2d8480 -1 init, newargv = 0x5595dedf95a0 newargc=15 2024-02-19T18:59:06.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[67419]: starting ceph client 2024-02-19T18:59:06.511 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[67419]: starting fuse 2024-02-19T18:59:06.518 INFO:teuthology.orchestra.run.smithi088.stdout:159 2024-02-19T18:59:06.518 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [159] 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> sudo stdin-killer -- python3 -c ' 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> import glob 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> import re 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> import os 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> import subprocess 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> def _find_admin_socket(client_name): 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> files = glob.glob(asok_path) 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> # Given a non-glob path, it better be there 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> if "*" not in asok_path: 2024-02-19T18:59:06.519 DEBUG:teuthology.orchestra.run.smithi088:> assert(len(files) == 1) 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> return files[0] 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> for f in files: 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> contents = proc_f.read() 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> if mountpoint in contents: 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> return f 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> print(_find_admin_socket("client.0")) 2024-02-19T18:59:06.520 DEBUG:teuthology.orchestra.run.smithi088:> ' 2024-02-19T18:59:06.637 INFO:teuthology.orchestra.run.smithi088.stdout:/var/run/ceph/ceph-client.0.67419.asok 2024-02-19T18:59:06.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-19T18:59:06.647 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67419.asok 2024-02-19T18:59:06.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:06.648 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67419.asok status 2024-02-19T18:59:06.783 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T18:59:06.783 INFO:teuthology.orchestra.run.smithi088.stdout: "metadata": { 2024-02-19T18:59:06.783 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-02-19T18:59:06.783 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "entity_id": "0", 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": "smithi088", 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "pid": "67419", 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "root": "/" 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_count": 0, 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "dentry_pinned_count": 0, 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 14562, 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "inst": { 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "name": { 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "client", 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: "num": 14562 2024-02-19T18:59:06.784 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 3373415141 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": { 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "192.168.0.1:0", 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 3373415141 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: "inst_str": "client.14562 192.168.0.1:0/3373415141", 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: "addr_str": "192.168.0.1:0/3373415141", 2024-02-19T18:59:06.785 INFO:teuthology.orchestra.run.smithi088.stdout: "inode_count": 1, 2024-02-19T18:59:06.786 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_epoch": 18, 2024-02-19T18:59:06.786 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch": 46, 2024-02-19T18:59:06.786 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_epoch_barrier": 0, 2024-02-19T18:59:06.786 INFO:teuthology.orchestra.run.smithi088.stdout: "blocklisted": false, 2024-02-19T18:59:06.786 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs" 2024-02-19T18:59:06.786 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T18:59:06.792 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- ceph fs ls 2024-02-19T18:59:07.000 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:59:07.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:06 smithi088 ceph-mon[29272]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-19T18:59:07.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:06 smithi132 ceph-mon[37969]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-19T18:59:08.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.677+0000 7ff27d0fc700 1 -- 172.21.15.88:0/3695176097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2781004e0 msgr2=0x7ff2781008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:08.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.677+0000 7ff27d0fc700 1 --2- 172.21.15.88:0/3695176097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2781004e0 0x7ff2781008b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff268009910 tx=0x7ff268009c20 comp rx=0 tx=0).stop 2024-02-19T18:59:08.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.677+0000 7ff27d0fc700 1 -- 172.21.15.88:0/3695176097 shutdown_connections 2024-02-19T18:59:08.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.677+0000 7ff27d0fc700 1 --2- 172.21.15.88:0/3695176097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff278100e80 0x7ff278104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:08.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.677+0000 7ff27d0fc700 1 --2- 172.21.15.88:0/3695176097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2781004e0 0x7ff2781008b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:08.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.677+0000 7ff27d0fc700 1 -- 172.21.15.88:0/3695176097 >> 172.21.15.88:0/3695176097 conn(0x7ff2780fbda0 msgr2=0x7ff2780fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:08.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.677+0000 7ff27d0fc700 1 -- 172.21.15.88:0/3695176097 shutdown_connections 2024-02-19T18:59:08.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.677+0000 7ff27d0fc700 1 -- 172.21.15.88:0/3695176097 wait complete. 2024-02-19T18:59:08.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.678+0000 7ff27d0fc700 1 Processor -- start 2024-02-19T18:59:08.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.678+0000 7ff27d0fc700 1 -- start start 2024-02-19T18:59:08.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.678+0000 7ff27d0fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2781004e0 0x7ff278198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:08.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.678+0000 7ff27d0fc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff278100e80 0x7ff278198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:08.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.678+0000 7ff27d0fc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278198d40 con 0x7ff2781004e0 2024-02-19T18:59:08.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.678+0000 7ff27d0fc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27819cad0 con 0x7ff278100e80 2024-02-19T18:59:08.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.678+0000 7ff276d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2781004e0 0x7ff278198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:08.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.678+0000 7ff276d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2781004e0 0x7ff278198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40768/0 (socket says 172.21.15.88:40768) 2024-02-19T18:59:08.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.678+0000 7ff276d9d700 1 -- 172.21.15.88:0/2735478602 learned_addr learned my addr 172.21.15.88:0/2735478602 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:59:08.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.678+0000 7ff27659c700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff278100e80 0x7ff278198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:08.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.679+0000 7ff276d9d700 1 -- 172.21.15.88:0/2735478602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff278100e80 msgr2=0x7ff278198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:08.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.679+0000 7ff276d9d700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff278100e80 0x7ff278198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:08.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.679+0000 7ff276d9d700 1 -- 172.21.15.88:0/2735478602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2680095f0 con 0x7ff2781004e0 2024-02-19T18:59:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.679+0000 7ff27659c700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff278100e80 0x7ff278198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T18:59:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.679+0000 7ff276d9d700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2781004e0 0x7ff278198120 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff268006010 tx=0x7ff26800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.679+0000 7ff25ffff700 1 -- 172.21.15.88:0/2735478602 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff26801d070 con 0x7ff2781004e0 2024-02-19T18:59:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.679+0000 7ff27d0fc700 1 -- 172.21.15.88:0/2735478602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27819cd50 con 0x7ff2781004e0 2024-02-19T18:59:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.679+0000 7ff25ffff700 1 -- 172.21.15.88:0/2735478602 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff26800f460 con 0x7ff2781004e0 2024-02-19T18:59:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.679+0000 7ff25ffff700 1 -- 172.21.15.88:0/2735478602 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff268021640 con 0x7ff2781004e0 2024-02-19T18:59:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.679+0000 7ff27d0fc700 1 -- 172.21.15.88:0/2735478602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27819d240 con 0x7ff2781004e0 2024-02-19T18:59:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.680+0000 7ff27d0fc700 1 -- 172.21.15.88:0/2735478602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2781045d0 con 0x7ff2781004e0 2024-02-19T18:59:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.680+0000 7ff25ffff700 1 -- 172.21.15.88:0/2735478602 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff26800faf0 con 0x7ff2781004e0 2024-02-19T18:59:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.681+0000 7ff25ffff700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff26006c680 0x7ff26006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:08.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.681+0000 7ff25ffff700 1 -- 172.21.15.88:0/2735478602 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff26808cc90 con 0x7ff2781004e0 2024-02-19T18:59:08.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.681+0000 7ff27659c700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff26006c680 0x7ff26006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:08.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.681+0000 7ff27659c700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff26006c680 0x7ff26006eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff278199740 tx=0x7ff26c008040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:08.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.685+0000 7ff25ffff700 1 -- 172.21.15.88:0/2735478602 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff268057310 con 0x7ff2781004e0 2024-02-19T18:59:08.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.916+0000 7ff27d0fc700 1 -- 172.21.15.88:0/2735478602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff278199580 con 0x7ff2781004e0 2024-02-19T18:59:08.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.917+0000 7ff25ffff700 1 -- 172.21.15.88:0/2735478602 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+83 (secure 0 0 0) 0x7ff268026070 con 0x7ff2781004e0 2024-02-19T18:59:08.935 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-02-19T18:59:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.919+0000 7ff27d0fc700 1 -- 172.21.15.88:0/2735478602 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff26006c680 msgr2=0x7ff26006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.919+0000 7ff27d0fc700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff26006c680 0x7ff26006eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff278199740 tx=0x7ff26c008040 comp rx=0 tx=0).stop 2024-02-19T18:59:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.919+0000 7ff27d0fc700 1 -- 172.21.15.88:0/2735478602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2781004e0 msgr2=0x7ff278198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.919+0000 7ff27d0fc700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2781004e0 0x7ff278198120 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff268006010 tx=0x7ff26800be20 comp rx=0 tx=0).stop 2024-02-19T18:59:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.919+0000 7ff27d0fc700 1 -- 172.21.15.88:0/2735478602 shutdown_connections 2024-02-19T18:59:08.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.919+0000 7ff27d0fc700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7ff26006c680 0x7ff26006eb30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:08.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.919+0000 7ff27d0fc700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2781004e0 0x7ff278198120 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:08.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.919+0000 7ff27d0fc700 1 --2- 172.21.15.88:0/2735478602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff278100e80 0x7ff278198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:08.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.919+0000 7ff27d0fc700 1 -- 172.21.15.88:0/2735478602 >> 172.21.15.88:0/2735478602 conn(0x7ff2780fbda0 msgr2=0x7ff2780fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:08.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.919+0000 7ff27d0fc700 1 -- 172.21.15.88:0/2735478602 shutdown_connections 2024-02-19T18:59:08.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:08.919+0000 7ff27d0fc700 1 -- 172.21.15.88:0/2735478602 wait complete. 2024-02-19T18:59:09.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:08 smithi132 ceph-mon[37969]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.8 KiB/s rd, 170 B/s wr, 4 op/s 2024-02-19T18:59:09.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:08 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2735478602' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-19T18:59:09.256 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:08 smithi088 ceph-mon[29272]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.8 KiB/s rd, 170 B/s wr, 4 op/s 2024-02-19T18:59:09.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:08 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2735478602' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-19T18:59:09.595 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-19T18:59:09.596 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-19T18:59:09.596 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi132.front.sepia.ceph.com 2024-02-19T18:59:09.596 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-02-19T18:59:09.596 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-02-19T18:59:09.596 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-19T18:59:09.596 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-19T18:59:09.596 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-19T18:59:09.596 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-02-19T18:59:09.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:09.596 DEBUG:teuthology.orchestra.run.smithi132:> ip addr 2024-02-19T18:59:09.615 INFO:teuthology.orchestra.run.smithi132.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-19T18:59:09.615 INFO:teuthology.orchestra.run.smithi132.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-19T18:59:09.615 INFO:teuthology.orchestra.run.smithi132.stdout: inet 127.0.0.1/8 scope host lo 2024-02-19T18:59:09.615 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-02-19T18:59:09.615 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 ::1/128 scope host 2024-02-19T18:59:09.615 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-02-19T18:59:09.615 INFO:teuthology.orchestra.run.smithi132.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T18:59:09.615 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cc brd ff:ff:ff:ff:ff:ff 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f0 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:d9:35:cd brd ff:ff:ff:ff:ff:ff 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout: altname enp7s0f1 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3e brd ff:ff:ff:ff:ff:ff 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout: link/ether 0c:c4:7a:88:77:3f brd ff:ff:ff:ff:ff:ff 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout: inet 172.21.15.132/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft 42092sec preferred_lft 42092sec 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout: inet6 fe80::ec4:7aff:fe88:773f/64 scope link 2024-02-19T18:59:09.616 INFO:teuthology.orchestra.run.smithi132.stdout: valid_lft forever preferred_lft forever 2024-02-19T18:59:09.616 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-19T18:59:09.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:09.617 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-02-19T18:59:09.617 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-02-19T18:59:09.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add name ceph-brx type bridge 2024-02-19T18:59:09.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr flush dev ceph-brx 2024-02-19T18:59:09.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set ceph-brx up 2024-02-19T18:59:09.617 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-19T18:59:09.617 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-02-19T18:59:09.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:09.783 DEBUG:teuthology.orchestra.run.smithi132:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-19T18:59:09.810 INFO:teuthology.orchestra.run.smithi132.stdout:1 2024-02-19T18:59:09.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:09.812 DEBUG:teuthology.orchestra.run.smithi132:> route 2024-02-19T18:59:09.878 INFO:teuthology.orchestra.run.smithi132.stdout:Kernel IP routing table 2024-02-19T18:59:09.879 INFO:teuthology.orchestra.run.smithi132.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-19T18:59:09.879 INFO:teuthology.orchestra.run.smithi132.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-02-19T18:59:09.879 INFO:teuthology.orchestra.run.smithi132.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-02-19T18:59:09.879 INFO:teuthology.orchestra.run.smithi132.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-19T18:59:09.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:09.879 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-02-19T18:59:09.879 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-02-19T18:59:09.879 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-02-19T18:59:09.879 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-02-19T18:59:09.879 DEBUG:teuthology.orchestra.run.smithi132:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-02-19T18:59:09.879 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-02-19T18:59:10.195 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:09 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:59:10.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:10.196 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list 2024-02-19T18:59:10.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:10.211 DEBUG:teuthology.orchestra.run.smithi132:> ip netns list-id 2024-02-19T18:59:10.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:10.266 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-02-19T18:59:10.266 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-02-19T18:59:10.266 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-19T18:59:10.266 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-02-19T18:59:10.266 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-02-19T18:59:10.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:09 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:59:10.352 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-02-19T18:59:10.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:10.352 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-02-19T18:59:10.352 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-02-19T18:59:10.352 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-02-19T18:59:10.352 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-19T18:59:10.352 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-02-19T18:59:10.353 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-02-19T18:59:10.353 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-02-19T18:59:10.353 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-02-19T18:59:10.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:10.468 DEBUG:teuthology.orchestra.run.smithi132:> (cd / && exec bash -c ' 2024-02-19T18:59:10.468 DEBUG:teuthology.orchestra.run.smithi132:> set -e 2024-02-19T18:59:10.468 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set brx.0 up 2024-02-19T18:59:10.468 DEBUG:teuthology.orchestra.run.smithi132:> sudo ip link set dev brx.0 master ceph-brx 2024-02-19T18:59:10.468 DEBUG:teuthology.orchestra.run.smithi132:> ') 2024-02-19T18:59:10.515 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-02-19T18:59:10.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-19T18:59:10.515 DEBUG:teuthology.orchestra.run.smithi132:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-02-19T18:59:10.528 INFO:teuthology.orchestra.run.smithi132.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-02-19T18:59:10.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-19T18:59:10.529 DEBUG:teuthology.orchestra.run.smithi132:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-02-19T18:59:10.583 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-02-19T18:59:10.652 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-02-19T18:59:10.708 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-02-19T18:59:10.708 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-02-19T18:59:10.708 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-02-19T18:59:10.708 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-02-19T18:59:10.708 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-02-19T18:59:10.708 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-02-19T18:59:10.708 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-02-19T18:59:10.708 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-02-19T18:59:10.708 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-02-19T18:59:10.709 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/0 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/8bc7eee5bd870a262ece0e79850b1c271c7ddc7324b7408db78724c0a9cb40c7/merged 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/d647a788ba3925c893d5cdef63ba4a0ef29eed78ac1b820cda1beed93ddaf364/merged 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/79847d344e99a7ecad3e1e85eb1d3ea13b8b6c1e865d95aa0042ea2f0decec1c/merged 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/bd8616cae235ee6e02de9715c516d381567ea0f039dfffe6c384e5c8f7be5e3c/merged 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/78fec7eb4a4b9980fa3d20fefac4adaa5953db92a1f26e21ca3b4af6cfe83c29/merged 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/9ad9121ba32d6d47521af3f5fa1db8d7b089706a8c1dd7adc6775e4ec6b0d354/merged 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/4f89aa0a9c8a7d9ecc2ac7143e9ec988da5867282bf0b2e164c31113d69d10da/merged 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/247a61bc922bf7780efba6e99568a928a217b6e4c8d276d12c904d6afe98d18b/merged 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/f983c22435ce93676766aaf3aeb2b837b47fc9b1ded9d5ae5e9357c8d60984b9/merged 2024-02-19T18:59:10.710 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/15220908e115abf36df601cf14b55aff81787415797299512be8e57bc9862ef3/merged 2024-02-19T18:59:10.711 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-02-19T18:59:10.711 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-19T18:59:10.711 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-19T18:59:10.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:10.711 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-02-19T18:59:10.763 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-19T18:59:10.763 DEBUG:teuthology.orchestra.run.smithi132:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-02-19T18:59:10.806 DEBUG:teuthology.orchestra.run.smithi132:> sudo modprobe fuse 2024-02-19T18:59:10.831 DEBUG:teuthology.orchestra.run.smithi132:> cat /proc/self/mounts | awk '{print $2}' 2024-02-19T18:59:10.886 INFO:teuthology.orchestra.run.smithi132.stdout:/proc 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/sys 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/dev 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/security 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/shm 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/pts 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/run 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/cgroup 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/pstore 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/bpf 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/ 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/selinux 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T18:59:10.887 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/hugepages 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/dev/mqueue 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/debug 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/tracing 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/fs/fuse/connections 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-sysctl.service 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/sys/kernel/config 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/nfs/rpc_pipefs 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/1000 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/ceph 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/run/user/0 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/proc/sys/fs/binfmt_misc 2024-02-19T18:59:10.888 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/8bc7eee5bd870a262ece0e79850b1c271c7ddc7324b7408db78724c0a9cb40c7/merged 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/d647a788ba3925c893d5cdef63ba4a0ef29eed78ac1b820cda1beed93ddaf364/merged 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/79847d344e99a7ecad3e1e85eb1d3ea13b8b6c1e865d95aa0042ea2f0decec1c/merged 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/bd8616cae235ee6e02de9715c516d381567ea0f039dfffe6c384e5c8f7be5e3c/merged 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/78fec7eb4a4b9980fa3d20fefac4adaa5953db92a1f26e21ca3b4af6cfe83c29/merged 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/9ad9121ba32d6d47521af3f5fa1db8d7b089706a8c1dd7adc6775e4ec6b0d354/merged 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/4f89aa0a9c8a7d9ecc2ac7143e9ec988da5867282bf0b2e164c31113d69d10da/merged 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/247a61bc922bf7780efba6e99568a928a217b6e4c8d276d12c904d6afe98d18b/merged 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/f983c22435ce93676766aaf3aeb2b837b47fc9b1ded9d5ae5e9357c8d60984b9/merged 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/var/lib/containers/storage/overlay/15220908e115abf36df601cf14b55aff81787415797299512be8e57bc9862ef3/merged 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-19T18:59:10.889 INFO:teuthology.orchestra.run.smithi132.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-02-19T18:59:10.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:10.890 DEBUG:teuthology.orchestra.run.smithi132:> ls /sys/fs/fuse/connections 2024-02-19T18:59:10.917 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:2024-02-19T18:59:10.899+0000 7f5340185480 -1 init, newargv = 0x55db7329d410 newargc=15 2024-02-19T18:59:10.917 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[57705]: starting ceph client 2024-02-19T18:59:10.923 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi132.stderr:ceph-fuse[57705]: starting fuse 2024-02-19T18:59:10.933 INFO:teuthology.orchestra.run.smithi132.stdout:126 2024-02-19T18:59:10.933 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [126] 2024-02-19T18:59:10.933 DEBUG:teuthology.orchestra.run.smithi132:> sudo stdin-killer -- python3 -c ' 2024-02-19T18:59:10.933 DEBUG:teuthology.orchestra.run.smithi132:> import glob 2024-02-19T18:59:10.933 DEBUG:teuthology.orchestra.run.smithi132:> import re 2024-02-19T18:59:10.933 DEBUG:teuthology.orchestra.run.smithi132:> import os 2024-02-19T18:59:10.933 DEBUG:teuthology.orchestra.run.smithi132:> import subprocess 2024-02-19T18:59:10.933 DEBUG:teuthology.orchestra.run.smithi132:> 2024-02-19T18:59:10.933 DEBUG:teuthology.orchestra.run.smithi132:> def _find_admin_socket(client_name): 2024-02-19T18:59:10.933 DEBUG:teuthology.orchestra.run.smithi132:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> files = glob.glob(asok_path) 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> # Given a non-glob path, it better be there 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> if "*" not in asok_path: 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> assert(len(files) == 1) 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> return files[0] 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> for f in files: 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> contents = proc_f.read() 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> if mountpoint in contents: 2024-02-19T18:59:10.934 DEBUG:teuthology.orchestra.run.smithi132:> return f 2024-02-19T18:59:10.935 DEBUG:teuthology.orchestra.run.smithi132:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-19T18:59:10.935 DEBUG:teuthology.orchestra.run.smithi132:> 2024-02-19T18:59:10.935 DEBUG:teuthology.orchestra.run.smithi132:> print(_find_admin_socket("client.1")) 2024-02-19T18:59:10.935 DEBUG:teuthology.orchestra.run.smithi132:> ' 2024-02-19T18:59:11.054 INFO:teuthology.orchestra.run.smithi132.stdout:/var/run/ceph/ceph-client.1.57705.asok 2024-02-19T18:59:11.057 INFO:teuthology.orchestra.run.smithi132.stderr:2024-02-19T18:59:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-02-19T18:59:11.064 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.57705.asok 2024-02-19T18:59:11.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:11.065 DEBUG:teuthology.orchestra.run.smithi132:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.57705.asok status 2024-02-19T18:59:11.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:10 smithi132 ceph-mon[37969]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-19T18:59:11.201 INFO:teuthology.orchestra.run.smithi132.stdout:{ 2024-02-19T18:59:11.201 INFO:teuthology.orchestra.run.smithi132.stdout: "metadata": { 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "entity_id": "1", 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "hostname": "smithi132", 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "pid": "57705", 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "root": "/" 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_count": 0, 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "dentry_pinned_count": 0, 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "id": 24305, 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "inst": { 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "name": { 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "client", 2024-02-19T18:59:11.202 INFO:teuthology.orchestra.run.smithi132.stdout: "num": 24305 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.1:0", 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 1879662666 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: } 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": { 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: "type": "v1", 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: "addr": "192.168.0.1:0", 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: "nonce": 1879662666 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: }, 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: "inst_str": "client.24305 192.168.0.1:0/1879662666", 2024-02-19T18:59:11.203 INFO:teuthology.orchestra.run.smithi132.stdout: "addr_str": "192.168.0.1:0/1879662666", 2024-02-19T18:59:11.204 INFO:teuthology.orchestra.run.smithi132.stdout: "inode_count": 1, 2024-02-19T18:59:11.204 INFO:teuthology.orchestra.run.smithi132.stdout: "mds_epoch": 18, 2024-02-19T18:59:11.204 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch": 46, 2024-02-19T18:59:11.204 INFO:teuthology.orchestra.run.smithi132.stdout: "osd_epoch_barrier": 0, 2024-02-19T18:59:11.204 INFO:teuthology.orchestra.run.smithi132.stdout: "blocklisted": false, 2024-02-19T18:59:11.204 INFO:teuthology.orchestra.run.smithi132.stdout: "fs_name": "cephfs" 2024-02-19T18:59:11.204 INFO:teuthology.orchestra.run.smithi132.stdout:} 2024-02-19T18:59:11.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:11.210 DEBUG:teuthology.orchestra.run.smithi088:> stat --file-system '--printf=%T 2024-02-19T18:59:11.210 DEBUG:teuthology.orchestra.run.smithi088:> ' -- /home/ubuntu/cephtest/mnt.0 2024-02-19T18:59:11.227 INFO:teuthology.orchestra.run.smithi088.stdout:fuseblk 2024-02-19T18:59:11.227 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-02-19T18:59:11.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:11.228 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-19T18:59:11.293 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:10 smithi088 ceph-mon[29272]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-19T18:59:11.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:11.300 DEBUG:teuthology.orchestra.run.smithi132:> stat --file-system '--printf=%T 2024-02-19T18:59:11.300 DEBUG:teuthology.orchestra.run.smithi132:> ' -- /home/ubuntu/cephtest/mnt.1 2024-02-19T18:59:11.316 INFO:teuthology.orchestra.run.smithi132.stdout:fuseblk 2024-02-19T18:59:11.316 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-02-19T18:59:11.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T18:59:11.316 DEBUG:teuthology.orchestra.run.smithi132:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-02-19T18:59:13.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:12 smithi132 ceph-mon[37969]: pgmap v200: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-19T18:59:13.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:12 smithi088 ceph-mon[29272]: pgmap v200: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-02-19T18:59:14.395 INFO:teuthology.run_tasks:Running task print... 2024-02-19T18:59:14.406 INFO:teuthology.task.print:**** done client 2024-02-19T18:59:14.406 INFO:teuthology.run_tasks:Running task parallel... 2024-02-19T18:59:14.445 INFO:teuthology.task.parallel:starting parallel... 2024-02-19T18:59:14.445 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-19T18:59:14.446 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-19T18:59:14.446 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T18:59:14.447 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-19T18:59:14.447 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-02-19T18:59:14.447 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-02-19T18:59:14.451 INFO:tasks.workunit:Pulling workunits from ref 1ce7ea8877c709fa862eac608550374bd96e166e 2024-02-19T18:59:14.451 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-19T18:59:14.451 INFO:tasks.workunit:timeout=3h 2024-02-19T18:59:14.451 INFO:tasks.workunit:cleanup=True 2024-02-19T18:59:14.451 DEBUG:teuthology.orchestra.run.smithi088:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-19T18:59:14.471 INFO:teuthology.orchestra.run.smithi088.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-19T18:59:14.471 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-19T18:59:14.471 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 9fh/159d Inode: 1 Links: 2 2024-02-19T18:59:14.471 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T18:59:14.471 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-19T18:59:14.471 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-19T18:59:14.471 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-02-19 18:58:38.220192901 +0000 2024-02-19T18:59:14.471 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-02-19 18:59:11.280116624 +0000 2024-02-19T18:59:14.471 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: - 2024-02-19T18:59:14.472 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-19T18:59:14.472 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-19T18:59:14.551 DEBUG:teuthology.orchestra.run.smithi132:> stat -- /home/ubuntu/cephtest/mnt.1 2024-02-19T18:59:14.570 INFO:teuthology.orchestra.run.smithi132.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-02-19T18:59:14.570 INFO:teuthology.orchestra.run.smithi132.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-02-19T18:59:14.571 INFO:teuthology.orchestra.run.smithi132.stdout:Device: 7eh/126d Inode: 1 Links: 3 2024-02-19T18:59:14.571 INFO:teuthology.orchestra.run.smithi132.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-19T18:59:14.571 INFO:teuthology.orchestra.run.smithi132.stdout:Context: system_u:object_r:fusefs_t:s0 2024-02-19T18:59:14.571 INFO:teuthology.orchestra.run.smithi132.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-02-19T18:59:14.571 INFO:teuthology.orchestra.run.smithi132.stdout:Modify: 2024-02-19 18:59:14.522938643 +0000 2024-02-19T18:59:14.571 INFO:teuthology.orchestra.run.smithi132.stdout:Change: 2024-02-19 18:59:14.522938643 +0000 2024-02-19T18:59:14.571 INFO:teuthology.orchestra.run.smithi132.stdout: Birth: - 2024-02-19T18:59:14.571 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-02-19T18:59:14.571 DEBUG:teuthology.orchestra.run.smithi132:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-02-19T18:59:14.646 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1ce7ea8877c709fa862eac608550374bd96e166e 2024-02-19T18:59:14.647 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://github.com/batrick/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 1ce7ea8877c709fa862eac608550374bd96e166e 2024-02-19T18:59:14.662 INFO:tasks.workunit.client.0.smithi088.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-19T18:59:14.679 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:59:14.703 INFO:tasks.workunit.client.1.smithi132.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-02-19T18:59:15.060 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:14 smithi088 ceph-mon[29272]: pgmap v201: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T18:59:15.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:14 smithi132 ceph-mon[37969]: pgmap v201: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T18:59:16.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.376+0000 7fcee785b700 1 -- 172.21.15.88:0/660802139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcee0074d00 msgr2=0x7fcee0073160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.376+0000 7fcee785b700 1 --2- 172.21.15.88:0/660802139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcee0074d00 0x7fcee0073160 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fced000b0a0 tx=0x7fced000b3b0 comp rx=0 tx=0).stop 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.376+0000 7fcee785b700 1 -- 172.21.15.88:0/660802139 shutdown_connections 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.376+0000 7fcee785b700 1 --2- 172.21.15.88:0/660802139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee0073730 0x7fcee0073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.376+0000 7fcee785b700 1 --2- 172.21.15.88:0/660802139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcee0074d00 0x7fcee0073160 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.376+0000 7fcee785b700 1 -- 172.21.15.88:0/660802139 >> 172.21.15.88:0/660802139 conn(0x7fcee00fb7f0 msgr2=0x7fcee00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.376+0000 7fcee785b700 1 -- 172.21.15.88:0/660802139 shutdown_connections 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.376+0000 7fcee785b700 1 -- 172.21.15.88:0/660802139 wait complete. 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.377+0000 7fcee785b700 1 Processor -- start 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.377+0000 7fcee785b700 1 -- start start 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.377+0000 7fcee785b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee0073730 0x7fcee0197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.377+0000 7fcee785b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcee0074d00 0x7fcee0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:16.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.377+0000 7fcee785b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0198920 con 0x7fcee0074d00 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.377+0000 7fcee785b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcee0198a60 con 0x7fcee0073730 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.377+0000 7fcee4df6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcee0074d00 0x7fcee0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.377+0000 7fcee4df6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcee0074d00 0x7fcee0198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37778/0 (socket says 172.21.15.88:37778) 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.377+0000 7fcee4df6700 1 -- 172.21.15.88:0/170883187 learned_addr learned my addr 172.21.15.88:0/170883187 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.377+0000 7fcee55f7700 1 --2- 172.21.15.88:0/170883187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee0073730 0x7fcee0197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.378+0000 7fcee4df6700 1 -- 172.21.15.88:0/170883187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee0073730 msgr2=0x7fcee0197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.378+0000 7fcee4df6700 1 --2- 172.21.15.88:0/170883187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee0073730 0x7fcee0197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.378+0000 7fcee4df6700 1 -- 172.21.15.88:0/170883187 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced0009d00 con 0x7fcee0074d00 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.378+0000 7fcee4df6700 1 --2- 172.21.15.88:0/170883187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcee0074d00 0x7fcee0198300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fced400b4b0 tx=0x7fced400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.378+0000 7fcede7fc700 1 -- 172.21.15.88:0/170883187 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fced4011800 con 0x7fcee0074d00 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.378+0000 7fcede7fc700 1 -- 172.21.15.88:0/170883187 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fced4011e10 con 0x7fcee0074d00 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.378+0000 7fcede7fc700 1 -- 172.21.15.88:0/170883187 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fced400f2e0 con 0x7fcee0074d00 2024-02-19T18:59:16.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.378+0000 7fcee785b700 1 -- 172.21.15.88:0/170883187 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcee019d510 con 0x7fcee0074d00 2024-02-19T18:59:16.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.378+0000 7fcee785b700 1 -- 172.21.15.88:0/170883187 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcee019da60 con 0x7fcee0074d00 2024-02-19T18:59:16.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.379+0000 7fcee785b700 1 -- 172.21.15.88:0/170883187 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee010aff0 con 0x7fcee0074d00 2024-02-19T18:59:16.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.380+0000 7fcede7fc700 1 -- 172.21.15.88:0/170883187 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fced4010380 con 0x7fcee0074d00 2024-02-19T18:59:16.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.381+0000 7fcede7fc700 1 --2- 172.21.15.88:0/170883187 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcec806c680 0x7fcec806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:16.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.381+0000 7fcede7fc700 1 -- 172.21.15.88:0/170883187 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fced408b920 con 0x7fcee0074d00 2024-02-19T18:59:16.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.381+0000 7fcee55f7700 1 --2- 172.21.15.88:0/170883187 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcec806c680 0x7fcec806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:16.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.381+0000 7fcee55f7700 1 --2- 172.21.15.88:0/170883187 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcec806c680 0x7fcec806eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fced0000f80 tx=0x7fced00060c0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:16.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.384+0000 7fcede7fc700 1 -- 172.21.15.88:0/170883187 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fced4053720 con 0x7fcee0074d00 2024-02-19T18:59:16.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.607+0000 7fcee785b700 1 -- 172.21.15.88:0/170883187 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fcee010b1e0 con 0x7fcee0074d00 2024-02-19T18:59:16.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.607+0000 7fcede7fc700 1 -- 172.21.15.88:0/170883187 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fced4059b50 con 0x7fcee0074d00 2024-02-19T18:59:16.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.609+0000 7fcee785b700 1 -- 172.21.15.88:0/170883187 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcec806c680 msgr2=0x7fcec806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:16.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.609+0000 7fcee785b700 1 --2- 172.21.15.88:0/170883187 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcec806c680 0x7fcec806eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fced0000f80 tx=0x7fced00060c0 comp rx=0 tx=0).stop 2024-02-19T18:59:16.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.609+0000 7fcee785b700 1 -- 172.21.15.88:0/170883187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcee0074d00 msgr2=0x7fcee0198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:16.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.609+0000 7fcee785b700 1 --2- 172.21.15.88:0/170883187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcee0074d00 0x7fcee0198300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fced400b4b0 tx=0x7fced400b7c0 comp rx=0 tx=0).stop 2024-02-19T18:59:16.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.609+0000 7fcee785b700 1 -- 172.21.15.88:0/170883187 shutdown_connections 2024-02-19T18:59:16.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.609+0000 7fcee785b700 1 --2- 172.21.15.88:0/170883187 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fcec806c680 0x7fcec806eb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:16.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.609+0000 7fcee785b700 1 --2- 172.21.15.88:0/170883187 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcee0073730 0x7fcee0197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:16.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.609+0000 7fcee785b700 1 --2- 172.21.15.88:0/170883187 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcee0074d00 0x7fcee0198300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:16.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.609+0000 7fcee785b700 1 -- 172.21.15.88:0/170883187 >> 172.21.15.88:0/170883187 conn(0x7fcee00fb7f0 msgr2=0x7fcee01060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:16.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.609+0000 7fcee785b700 1 -- 172.21.15.88:0/170883187 shutdown_connections 2024-02-19T18:59:16.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:16.609+0000 7fcee785b700 1 -- 172.21.15.88:0/170883187 wait complete. 2024-02-19T18:59:17.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:16 smithi132 ceph-mon[37969]: pgmap v202: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-19T18:59:17.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:16 smithi088 ceph-mon[29272]: pgmap v202: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-19T18:59:17.509 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-19T18:59:17.775 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:59:19.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:18 smithi132 ceph-mon[37969]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-02-19T18:59:19.268 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:18 smithi088 ceph-mon[29272]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-02-19T18:59:19.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.624+0000 7f9b32ae1700 1 -- 172.21.15.88:0/1958570910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b2c1018c0 msgr2=0x7f9b2c101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.624+0000 7f9b32ae1700 1 --2- 172.21.15.88:0/1958570910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b2c1018c0 0x7f9b2c101cd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009910 tx=0x7f9b20009c20 comp rx=0 tx=0).stop 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 -- 172.21.15.88:0/1958570910 shutdown_connections 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 --2- 172.21.15.88:0/1958570910 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b2c102210 0x7f9b2c0fe340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 --2- 172.21.15.88:0/1958570910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b2c1018c0 0x7f9b2c101cd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 -- 172.21.15.88:0/1958570910 >> 172.21.15.88:0/1958570910 conn(0x7f9b2c0f9f10 msgr2=0x7f9b2c0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 -- 172.21.15.88:0/1958570910 shutdown_connections 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 -- 172.21.15.88:0/1958570910 wait complete. 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 Processor -- start 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 -- start start 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b2c1018c0 0x7f9b2c195b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b2c102210 0x7f9b2c196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:19.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c196680 con 0x7f9b2c1018c0 2024-02-19T18:59:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.625+0000 7f9b32ae1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b2c1967c0 con 0x7f9b2c102210 2024-02-19T18:59:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b3087d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b2c1018c0 0x7f9b2c195b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b3087d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b2c1018c0 0x7f9b2c195b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37794/0 (socket says 172.21.15.88:37794) 2024-02-19T18:59:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b3087d700 1 -- 172.21.15.88:0/2631856013 learned_addr learned my addr 172.21.15.88:0/2631856013 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:59:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b2bfff700 1 --2- 172.21.15.88:0/2631856013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b2c102210 0x7f9b2c196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b3087d700 1 -- 172.21.15.88:0/2631856013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b2c102210 msgr2=0x7f9b2c196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b3087d700 1 --2- 172.21.15.88:0/2631856013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b2c102210 0x7f9b2c196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b3087d700 1 -- 172.21.15.88:0/2631856013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b200095f0 con 0x7f9b2c1018c0 2024-02-19T18:59:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b3087d700 1 --2- 172.21.15.88:0/2631856013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b2c1018c0 0x7f9b2c195b20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9b20006010 tx=0x7f9b2000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:19.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b29ffb700 1 -- 172.21.15.88:0/2631856013 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b2001d070 con 0x7f9b2c1018c0 2024-02-19T18:59:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b29ffb700 1 -- 172.21.15.88:0/2631856013 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9b2000f460 con 0x7f9b2c1018c0 2024-02-19T18:59:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b32ae1700 1 -- 172.21.15.88:0/2631856013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b2c19b210 con 0x7f9b2c1018c0 2024-02-19T18:59:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b29ffb700 1 -- 172.21.15.88:0/2631856013 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b20021640 con 0x7f9b2c1018c0 2024-02-19T18:59:19.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.626+0000 7f9b32ae1700 1 -- 172.21.15.88:0/2631856013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2c19b700 con 0x7f9b2c1018c0 2024-02-19T18:59:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.627+0000 7f9b32ae1700 1 -- 172.21.15.88:0/2631856013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2c108d50 con 0x7f9b2c1018c0 2024-02-19T18:59:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.628+0000 7f9b29ffb700 1 -- 172.21.15.88:0/2631856013 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f9b2000fab0 con 0x7f9b2c1018c0 2024-02-19T18:59:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.628+0000 7f9b29ffb700 1 --2- 172.21.15.88:0/2631856013 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9b1c06c680 0x7f9b1c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.628+0000 7f9b29ffb700 1 -- 172.21.15.88:0/2631856013 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9b2008ca30 con 0x7f9b2c1018c0 2024-02-19T18:59:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.628+0000 7f9b2bfff700 1 --2- 172.21.15.88:0/2631856013 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9b1c06c680 0x7f9b1c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:19.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.629+0000 7f9b2bfff700 1 --2- 172.21.15.88:0/2631856013 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9b1c06c680 0x7f9b1c06eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9b18005950 tx=0x7f9b1800b410 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:19.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.633+0000 7f9b29ffb700 1 -- 172.21.15.88:0/2631856013 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9b2005abc0 con 0x7f9b2c1018c0 2024-02-19T18:59:19.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.829+0000 7f9b32ae1700 1 -- 172.21.15.88:0/2631856013 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f9b2c108f10 con 0x7f9b2c1018c0 2024-02-19T18:59:19.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.829+0000 7f9b29ffb700 1 -- 172.21.15.88:0/2631856013 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f9b2005a750 con 0x7f9b2c1018c0 2024-02-19T18:59:19.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.832+0000 7f9b32ae1700 1 -- 172.21.15.88:0/2631856013 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9b1c06c680 msgr2=0x7f9b1c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:19.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.832+0000 7f9b32ae1700 1 --2- 172.21.15.88:0/2631856013 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9b1c06c680 0x7f9b1c06eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9b18005950 tx=0x7f9b1800b410 comp rx=0 tx=0).stop 2024-02-19T18:59:19.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.832+0000 7f9b32ae1700 1 -- 172.21.15.88:0/2631856013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b2c1018c0 msgr2=0x7f9b2c195b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:19.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.832+0000 7f9b32ae1700 1 --2- 172.21.15.88:0/2631856013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b2c1018c0 0x7f9b2c195b20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9b20006010 tx=0x7f9b2000bd70 comp rx=0 tx=0).stop 2024-02-19T18:59:19.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.832+0000 7f9b32ae1700 1 -- 172.21.15.88:0/2631856013 shutdown_connections 2024-02-19T18:59:19.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.832+0000 7f9b32ae1700 1 --2- 172.21.15.88:0/2631856013 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9b1c06c680 0x7f9b1c06eb30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:19.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.832+0000 7f9b32ae1700 1 --2- 172.21.15.88:0/2631856013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b2c1018c0 0x7f9b2c195b20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:19.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.832+0000 7f9b32ae1700 1 --2- 172.21.15.88:0/2631856013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b2c102210 0x7f9b2c196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:19.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.832+0000 7f9b32ae1700 1 -- 172.21.15.88:0/2631856013 >> 172.21.15.88:0/2631856013 conn(0x7f9b2c0f9f10 msgr2=0x7f9b2c104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:19.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.832+0000 7f9b32ae1700 1 -- 172.21.15.88:0/2631856013 shutdown_connections 2024-02-19T18:59:19.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:19.832+0000 7f9b32ae1700 1 -- 172.21.15.88:0/2631856013 wait complete. 2024-02-19T18:59:20.861 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-19T18:59:21.072 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:59:21.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:20 smithi088 ceph-mon[29272]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-02-19T18:59:21.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:20 smithi132 ceph-mon[37969]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-02-19T18:59:22.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 -- 172.21.15.88:0/152349574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad4069250 msgr2=0x7f5ad4102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:22.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 --2- 172.21.15.88:0/152349574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad4069250 0x7f5ad4102df0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4009910 tx=0x7f5ac4009c20 comp rx=0 tx=0).stop 2024-02-19T18:59:22.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 -- 172.21.15.88:0/152349574 shutdown_connections 2024-02-19T18:59:22.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 --2- 172.21.15.88:0/152349574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad4103330 0x7f5ad4105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:22.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 --2- 172.21.15.88:0/152349574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad4069250 0x7f5ad4102df0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 -- 172.21.15.88:0/152349574 >> 172.21.15.88:0/152349574 conn(0x7f5ad40fa780 msgr2=0x7f5ad40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 -- 172.21.15.88:0/152349574 shutdown_connections 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 -- 172.21.15.88:0/152349574 wait complete. 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 Processor -- start 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 -- start start 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad4069250 0x7f5ad4100740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad4103330 0x7f5ad40fed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.954+0000 7f5ada564700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40ff2d0 con 0x7f5ad4069250 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ada564700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad40ff410 con 0x7f5ad4103330 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ad3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad4069250 0x7f5ad4100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ad3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad4069250 0x7f5ad4100740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37810/0 (socket says 172.21.15.88:37810) 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ad3fff700 1 -- 172.21.15.88:0/2465522024 learned_addr learned my addr 172.21.15.88:0/2465522024 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ad37fe700 1 --2- 172.21.15.88:0/2465522024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad4103330 0x7f5ad40fed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ad3fff700 1 -- 172.21.15.88:0/2465522024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad4103330 msgr2=0x7f5ad40fed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ad3fff700 1 --2- 172.21.15.88:0/2465522024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad4103330 0x7f5ad40fed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:22.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ad3fff700 1 -- 172.21.15.88:0/2465522024 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac40095f0 con 0x7f5ad4069250 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ad3fff700 1 --2- 172.21.15.88:0/2465522024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad4069250 0x7f5ad4100740 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400bd70 tx=0x7f5ac400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ad17fa700 1 -- 172.21.15.88:0/2465522024 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ac401d070 con 0x7f5ad4069250 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ada564700 1 -- 172.21.15.88:0/2465522024 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad40ff690 con 0x7f5ad4069250 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ad17fa700 1 -- 172.21.15.88:0/2465522024 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ac4005dc0 con 0x7f5ad4069250 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ad17fa700 1 -- 172.21.15.88:0/2465522024 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ac4021410 con 0x7f5ad4069250 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.955+0000 7f5ada564700 1 -- 172.21.15.88:0/2465522024 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad40ffb80 con 0x7f5ad4069250 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.956+0000 7f5ada564700 1 -- 172.21.15.88:0/2465522024 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad418fbd0 con 0x7f5ad4069250 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.957+0000 7f5ad17fa700 1 -- 172.21.15.88:0/2465522024 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f5ac400f7b0 con 0x7f5ad4069250 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.957+0000 7f5ad17fa700 1 --2- 172.21.15.88:0/2465522024 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5abc06c680 0x7f5abc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.957+0000 7f5ad17fa700 1 -- 172.21.15.88:0/2465522024 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5ac408cbd0 con 0x7f5ad4069250 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.957+0000 7f5ad37fe700 1 --2- 172.21.15.88:0/2465522024 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5abc06c680 0x7f5abc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:22.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.958+0000 7f5ad37fe700 1 --2- 172.21.15.88:0/2465522024 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5abc06c680 0x7f5abc06eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8005950 tx=0x7f5ac800b410 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:22.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:22.962+0000 7f5ad17fa700 1 -- 172.21.15.88:0/2465522024 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5ac405ace0 con 0x7f5ad4069250 2024-02-19T18:59:23.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.181+0000 7f5ada564700 1 -- 172.21.15.88:0/2465522024 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f5ad402cc30 con 0x7f5ad4069250 2024-02-19T18:59:23.199 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:22 smithi088 ceph-mon[29272]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-19T18:59:23.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.181+0000 7f5ad17fa700 1 -- 172.21.15.88:0/2465522024 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f5ac405a870 con 0x7f5ad4069250 2024-02-19T18:59:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.183+0000 7f5ada564700 1 -- 172.21.15.88:0/2465522024 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5abc06c680 msgr2=0x7f5abc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.183+0000 7f5ada564700 1 --2- 172.21.15.88:0/2465522024 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5abc06c680 0x7f5abc06eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8005950 tx=0x7f5ac800b410 comp rx=0 tx=0).stop 2024-02-19T18:59:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.183+0000 7f5ada564700 1 -- 172.21.15.88:0/2465522024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad4069250 msgr2=0x7f5ad4100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.183+0000 7f5ada564700 1 --2- 172.21.15.88:0/2465522024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad4069250 0x7f5ad4100740 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5ac400bd70 tx=0x7f5ac400bda0 comp rx=0 tx=0).stop 2024-02-19T18:59:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.184+0000 7f5ada564700 1 -- 172.21.15.88:0/2465522024 shutdown_connections 2024-02-19T18:59:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.184+0000 7f5ada564700 1 --2- 172.21.15.88:0/2465522024 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f5abc06c680 0x7f5abc06eb30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.184+0000 7f5ada564700 1 --2- 172.21.15.88:0/2465522024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad4069250 0x7f5ad4100740 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.184+0000 7f5ada564700 1 --2- 172.21.15.88:0/2465522024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ad4103330 0x7f5ad40fed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.184+0000 7f5ada564700 1 -- 172.21.15.88:0/2465522024 >> 172.21.15.88:0/2465522024 conn(0x7f5ad40fa780 msgr2=0x7f5ad40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.184+0000 7f5ada564700 1 -- 172.21.15.88:0/2465522024 shutdown_connections 2024-02-19T18:59:23.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:23.184+0000 7f5ada564700 1 -- 172.21.15.88:0/2465522024 wait complete. 2024-02-19T18:59:23.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:22 smithi132 ceph-mon[37969]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-02-19T18:59:24.011 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-02-19T18:59:24.219 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:59:25.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:24 smithi088 ceph-mon[29272]: pgmap v206: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-19T18:59:25.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:24 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:59:25.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:24 smithi132 ceph-mon[37969]: pgmap v206: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-19T18:59:25.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:24 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:59:26.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.251+0000 7f7f433d5700 1 -- 172.21.15.88:0/3548225247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c074d80 msgr2=0x7f7f3c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:26.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.251+0000 7f7f433d5700 1 --2- 172.21.15.88:0/3548225247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c074d80 0x7f7f3c0731e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009910 tx=0x7f7f38009c20 comp rx=0 tx=0).stop 2024-02-19T18:59:26.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.251+0000 7f7f433d5700 1 -- 172.21.15.88:0/3548225247 shutdown_connections 2024-02-19T18:59:26.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.251+0000 7f7f433d5700 1 --2- 172.21.15.88:0/3548225247 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c0737b0 0x7f7f3c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:26.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.251+0000 7f7f433d5700 1 --2- 172.21.15.88:0/3548225247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c074d80 0x7f7f3c0731e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.251+0000 7f7f433d5700 1 -- 172.21.15.88:0/3548225247 >> 172.21.15.88:0/3548225247 conn(0x7f7f3c0fb7f0 msgr2=0x7f7f3c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.251+0000 7f7f433d5700 1 -- 172.21.15.88:0/3548225247 shutdown_connections 2024-02-19T18:59:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.251+0000 7f7f433d5700 1 -- 172.21.15.88:0/3548225247 wait complete. 2024-02-19T18:59:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.252+0000 7f7f433d5700 1 Processor -- start 2024-02-19T18:59:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.252+0000 7f7f433d5700 1 -- start start 2024-02-19T18:59:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.252+0000 7f7f433d5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c0737b0 0x7f7f3c19c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.252+0000 7f7f433d5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c074d80 0x7f7f3c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.252+0000 7f7f433d5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c19cc70 con 0x7f7f3c0737b0 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.252+0000 7f7f433d5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f3c19cdb0 con 0x7f7f3c074d80 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.252+0000 7f7f41171700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c0737b0 0x7f7f3c19c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.252+0000 7f7f41171700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c0737b0 0x7f7f3c19c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36534/0 (socket says 172.21.15.88:36534) 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.252+0000 7f7f41171700 1 -- 172.21.15.88:0/1483372401 learned_addr learned my addr 172.21.15.88:0/1483372401 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.252+0000 7f7f40970700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c074d80 0x7f7f3c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.253+0000 7f7f41171700 1 -- 172.21.15.88:0/1483372401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c074d80 msgr2=0x7f7f3c19c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.253+0000 7f7f41171700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c074d80 0x7f7f3c19c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.253+0000 7f7f41171700 1 -- 172.21.15.88:0/1483372401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f380095f0 con 0x7f7f3c0737b0 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.253+0000 7f7f40970700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c074d80 0x7f7f3c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.253+0000 7f7f41171700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c0737b0 0x7f7f3c19c110 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800b5c0 tx=0x7f7f3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.253+0000 7f7f327fc700 1 -- 172.21.15.88:0/1483372401 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f3801d070 con 0x7f7f3c0737b0 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.253+0000 7f7f327fc700 1 -- 172.21.15.88:0/1483372401 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7f38022470 con 0x7f7f3c0737b0 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.253+0000 7f7f433d5700 1 -- 172.21.15.88:0/1483372401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f3c1a1800 con 0x7f7f3c0737b0 2024-02-19T18:59:26.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.253+0000 7f7f327fc700 1 -- 172.21.15.88:0/1483372401 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f3800f690 con 0x7f7f3c0737b0 2024-02-19T18:59:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.253+0000 7f7f433d5700 1 -- 172.21.15.88:0/1483372401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3c1a1cf0 con 0x7f7f3c0737b0 2024-02-19T18:59:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.254+0000 7f7f433d5700 1 -- 172.21.15.88:0/1483372401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f3c10f3f0 con 0x7f7f3c0737b0 2024-02-19T18:59:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.254+0000 7f7f327fc700 1 -- 172.21.15.88:0/1483372401 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f7f38004b70 con 0x7f7f3c0737b0 2024-02-19T18:59:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.255+0000 7f7f327fc700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7f2806c680 0x7f7f2806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T18:59:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.255+0000 7f7f327fc700 1 -- 172.21.15.88:0/1483372401 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7f3808cf10 con 0x7f7f3c0737b0 2024-02-19T18:59:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.255+0000 7f7f40970700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7f2806c680 0x7f7f2806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T18:59:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.255+0000 7f7f40970700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7f2806c680 0x7f7f2806eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0099d0 tx=0x7f7f2c008040 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T18:59:26.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.259+0000 7f7f327fc700 1 -- 172.21.15.88:0/1483372401 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7f3805b150 con 0x7f7f3c0737b0 2024-02-19T18:59:26.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:26.470+0000 7f7f433d5700 1 -- 172.21.15.88:0/1483372401 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f3c1032f0 con 0x7f7f2806c680 2024-02-19T18:59:27.307 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:26 smithi132 ceph-mon[37969]: pgmap v207: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-19T18:59:27.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:26 smithi088 ceph-mon[29272]: pgmap v207: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-19T18:59:28.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:27 smithi088 ceph-mon[29272]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:59:28.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:27 smithi132 ceph-mon[37969]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T18:59:29.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:28 smithi088 ceph-mon[29272]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-19T18:59:29.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:28 smithi132 ceph-mon[37969]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-02-19T18:59:31.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:30 smithi088 ceph-mon[29272]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-19T18:59:31.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:30 smithi132 ceph-mon[37969]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-02-19T18:59:33.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:33 smithi088 ceph-mon[29272]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-19T18:59:33.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:33 smithi132 ceph-mon[37969]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-19T18:59:35.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:35 smithi088 ceph-mon[29272]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-19T18:59:35.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:35 smithi132 ceph-mon[37969]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-19T18:59:37.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:37 smithi088 ceph-mon[29272]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-19T18:59:37.403 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:37 smithi132 ceph-mon[37969]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-02-19T18:59:39.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:39 smithi088 ceph-mon[29272]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-19T18:59:39.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:39 smithi132 ceph-mon[37969]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-02-19T18:59:40.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:40 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:59:40.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:40 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:59:41.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:41 smithi088 ceph-mon[29272]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-02-19T18:59:41.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:41 smithi132 ceph-mon[37969]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-02-19T18:59:43.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:43 smithi088 ceph-mon[29272]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:43.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:43 smithi132 ceph-mon[37969]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:45.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:45 smithi088 ceph-mon[29272]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:45.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:45 smithi132 ceph-mon[37969]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:47 smithi088 ceph-mon[29272]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:47.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:47 smithi132 ceph-mon[37969]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:49.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:49 smithi088 ceph-mon[29272]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:49.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:49 smithi132 ceph-mon[37969]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:51.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:51 smithi088 ceph-mon[29272]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:51.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:51 smithi132 ceph-mon[37969]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:53.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:53 smithi088 ceph-mon[29272]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:53.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:53 smithi132 ceph-mon[37969]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:55.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:55 smithi088 ceph-mon[29272]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:55.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:55 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:59:55.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:55 smithi132 ceph-mon[37969]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:55.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:55 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T18:59:55.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.630+0000 7f7f327fc700 1 -- 172.21.15.88:0/1483372401 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f7f3c1032f0 con 0x7f7f2806c680 2024-02-19T18:59:55.649 INFO:teuthology.orchestra.run.smithi088.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:59:55.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.632+0000 7f7f433d5700 1 -- 172.21.15.88:0/1483372401 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7f2806c680 msgr2=0x7f7f2806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:55.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.632+0000 7f7f433d5700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7f2806c680 0x7f7f2806eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7f2c0099d0 tx=0x7f7f2c008040 comp rx=0 tx=0).stop 2024-02-19T18:59:55.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.632+0000 7f7f433d5700 1 -- 172.21.15.88:0/1483372401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c0737b0 msgr2=0x7f7f3c19c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T18:59:55.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.632+0000 7f7f433d5700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c0737b0 0x7f7f3c19c110 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800b5c0 tx=0x7f7f3800be20 comp rx=0 tx=0).stop 2024-02-19T18:59:55.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.632+0000 7f7f433d5700 1 -- 172.21.15.88:0/1483372401 shutdown_connections 2024-02-19T18:59:55.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.632+0000 7f7f433d5700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7f2806c680 0x7f7f2806eb30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:55.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.632+0000 7f7f433d5700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f3c0737b0 0x7f7f3c19c110 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:55.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.632+0000 7f7f433d5700 1 --2- 172.21.15.88:0/1483372401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f3c074d80 0x7f7f3c19c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T18:59:55.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.632+0000 7f7f433d5700 1 -- 172.21.15.88:0/1483372401 >> 172.21.15.88:0/1483372401 conn(0x7f7f3c0fb7f0 msgr2=0x7f7f3c101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T18:59:55.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.633+0000 7f7f433d5700 1 -- 172.21.15.88:0/1483372401 shutdown_connections 2024-02-19T18:59:55.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T18:59:55.633+0000 7f7f433d5700 1 -- 172.21.15.88:0/1483372401 wait complete. 2024-02-19T18:59:56.441 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-02-19T18:59:56.665 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T18:59:56.787 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:56 smithi088 ceph-mon[29272]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:59:56.787 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:56 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:59:56.787 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:56 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:59:56.787 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:56 smithi088 ceph-mon[29272]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:57.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:56 smithi132 ceph-mon[37969]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:59:57.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:56 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:59:57.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:56 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T18:59:57.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:56 smithi132 ceph-mon[37969]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:57.968 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:57 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:59:57.968 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:57 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:59:57.968 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:57 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:59:57.968 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:57 smithi088 ceph-mon[29272]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:59:58.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:57 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T18:59:58.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:57 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T18:59:58.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:57 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T18:59:58.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:57 smithi132 ceph-mon[37969]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T18:59:58.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 18:59:58 smithi088 ceph-mon[29272]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T18:59:59.000 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 18:59:58 smithi132 ceph-mon[37969]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:00.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:00 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:00.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:00 smithi132 ceph-mon[37969]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (unknown) 2024-02-19T19:00:00.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:00 smithi132 ceph-mon[37969]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-19T19:00:00.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:00 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:00:00.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:00 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:00.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:00 smithi132 ceph-mon[37969]: Upgrade: Need to upgrade myself (mgr.smithi088.gyviqq) 2024-02-19T19:00:00.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:00 smithi132 ceph-mon[37969]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:00.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:00 smithi132 ceph-mon[37969]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 on smithi132 2024-02-19T19:00:00.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:00 smithi132 ceph-mon[37969]: overall HEALTH_OK 2024-02-19T19:00:00.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:00 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:00.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:00 smithi088 ceph-mon[29272]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (unknown) 2024-02-19T19:00:00.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:00 smithi088 ceph-mon[29272]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-19T19:00:00.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:00 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:00:00.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:00 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:00.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:00 smithi088 ceph-mon[29272]: Upgrade: Need to upgrade myself (mgr.smithi088.gyviqq) 2024-02-19T19:00:00.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:00 smithi088 ceph-mon[29272]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:00.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:00 smithi088 ceph-mon[29272]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 on smithi132 2024-02-19T19:00:00.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:00 smithi088 ceph-mon[29272]: overall HEALTH_OK 2024-02-19T19:00:02.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:02 smithi132 ceph-mon[37969]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:03.094 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:02 smithi088 ceph-mon[29272]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.518+0000 7f9fee3bb700 1 -- 172.21.15.88:0/4076023571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fe80ffb60 msgr2=0x7f9fe80fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.518+0000 7f9fee3bb700 1 --2- 172.21.15.88:0/4076023571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fe80ffb60 0x7f9fe80fffd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009910 tx=0x7f9fdc009c20 comp rx=0 tx=0).stop 2024-02-19T19:00:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.518+0000 7f9fee3bb700 1 -- 172.21.15.88:0/4076023571 shutdown_connections 2024-02-19T19:00:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.518+0000 7f9fee3bb700 1 --2- 172.21.15.88:0/4076023571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fe80ffb60 0x7f9fe80fffd0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.518+0000 7f9fee3bb700 1 --2- 172.21.15.88:0/4076023571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fe80fe8c0 0x7f9fe80fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.518+0000 7f9fee3bb700 1 -- 172.21.15.88:0/4076023571 >> 172.21.15.88:0/4076023571 conn(0x7f9fe80f9f10 msgr2=0x7f9fe80fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.518+0000 7f9fee3bb700 1 -- 172.21.15.88:0/4076023571 shutdown_connections 2024-02-19T19:00:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.518+0000 7f9fee3bb700 1 -- 172.21.15.88:0/4076023571 wait complete. 2024-02-19T19:00:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.519+0000 7f9fee3bb700 1 Processor -- start 2024-02-19T19:00:03.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.519+0000 7f9fee3bb700 1 -- start start 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.519+0000 7f9fee3bb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fe80fe8c0 0x7f9fe81077c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.519+0000 7f9fee3bb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fe80ffb60 0x7f9fe819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.519+0000 7f9fee3bb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81966a0 con 0x7f9fe80fe8c0 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.519+0000 7f9fee3bb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe819d170 con 0x7f9fe80ffb60 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.519+0000 7f9fe7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fe80fe8c0 0x7f9fe81077c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.519+0000 7f9fe7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fe80fe8c0 0x7f9fe81077c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56958/0 (socket says 172.21.15.88:56958) 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.519+0000 7f9fe7fff700 1 -- 172.21.15.88:0/2852602065 learned_addr learned my addr 172.21.15.88:0/2852602065 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.519+0000 7f9fe77fe700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fe80ffb60 0x7f9fe819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.520+0000 7f9fe7fff700 1 -- 172.21.15.88:0/2852602065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fe80ffb60 msgr2=0x7f9fe819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.520+0000 7f9fe7fff700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fe80ffb60 0x7f9fe819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.520+0000 7f9fe7fff700 1 -- 172.21.15.88:0/2852602065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0095f0 con 0x7f9fe80fe8c0 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.520+0000 7f9fe77fe700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fe80ffb60 0x7f9fe819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.520+0000 7f9fe7fff700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fe80fe8c0 0x7f9fe81077c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800e950 tx=0x7f9fd800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:03.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.520+0000 7f9fe57fa700 1 -- 172.21.15.88:0/2852602065 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fd800cba0 con 0x7f9fe80fe8c0 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.520+0000 7f9fe57fa700 1 -- 172.21.15.88:0/2852602065 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9fd800eea0 con 0x7f9fe80fe8c0 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.520+0000 7f9fee3bb700 1 -- 172.21.15.88:0/2852602065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe819d420 con 0x7f9fe80fe8c0 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.520+0000 7f9fe57fa700 1 -- 172.21.15.88:0/2852602065 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fd8018650 con 0x7f9fe80fe8c0 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.520+0000 7f9fee3bb700 1 -- 172.21.15.88:0/2852602065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe819d940 con 0x7f9fe80fe8c0 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.521+0000 7f9fee3bb700 1 -- 172.21.15.88:0/2852602065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe8107d70 con 0x7f9fe80fe8c0 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.522+0000 7f9fe57fa700 1 -- 172.21.15.88:0/2852602065 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f9fd8010430 con 0x7f9fe80fe8c0 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.523+0000 7f9fe57fa700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9fd006c680 0x7f9fd006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.523+0000 7f9fe57fa700 1 -- 172.21.15.88:0/2852602065 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9fd8014070 con 0x7f9fe80fe8c0 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.523+0000 7f9fe77fe700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9fd006c680 0x7f9fd006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.523+0000 7f9fe77fe700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9fd006c680 0x7f9fd006eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009fd0 tx=0x7f9fdc005940 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.527+0000 7f9fe57fa700 1 -- 172.21.15.88:0/2852602065 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9fd8051720 con 0x7f9fe80fe8c0 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.743+0000 7f9fee3bb700 1 -- 172.21.15.88:0/2852602065 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fe81044b0 con 0x7f9fd006c680 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.744+0000 7f9fe57fa700 1 -- 172.21.15.88:0/2852602065 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f9fe81044b0 con 0x7f9fd006c680 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.746+0000 7f9fee3bb700 1 -- 172.21.15.88:0/2852602065 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9fd006c680 msgr2=0x7f9fd006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.746+0000 7f9fee3bb700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9fd006c680 0x7f9fd006eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc009fd0 tx=0x7f9fdc005940 comp rx=0 tx=0).stop 2024-02-19T19:00:03.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.746+0000 7f9fee3bb700 1 -- 172.21.15.88:0/2852602065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fe80fe8c0 msgr2=0x7f9fe81077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.746+0000 7f9fee3bb700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fe80fe8c0 0x7f9fe81077c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800e950 tx=0x7f9fd800ec60 comp rx=0 tx=0).stop 2024-02-19T19:00:03.551 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:00:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.746+0000 7f9fee3bb700 1 -- 172.21.15.88:0/2852602065 shutdown_connections 2024-02-19T19:00:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.746+0000 7f9fee3bb700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f9fd006c680 0x7f9fd006eb30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.746+0000 7f9fee3bb700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fe80fe8c0 0x7f9fe81077c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.746+0000 7f9fee3bb700 1 --2- 172.21.15.88:0/2852602065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9fe80ffb60 0x7f9fe819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.746+0000 7f9fee3bb700 1 -- 172.21.15.88:0/2852602065 >> 172.21.15.88:0/2852602065 conn(0x7f9fe80f9f10 msgr2=0x7f9fe8102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.746+0000 7f9fee3bb700 1 -- 172.21.15.88:0/2852602065 shutdown_connections 2024-02-19T19:00:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.746+0000 7f9fee3bb700 1 -- 172.21.15.88:0/2852602065 wait complete. 2024-02-19T19:00:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.854+0000 7f0c6564d700 1 -- 172.21.15.88:0/1384567583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60100240 msgr2=0x7f0c60100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.854+0000 7f0c6564d700 1 --2- 172.21.15.88:0/1384567583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60100240 0x7f0c60100650 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0c5000fa70 tx=0x7f0c5000fd80 comp rx=0 tx=0).stop 2024-02-19T19:00:03.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.854+0000 7f0c6564d700 1 -- 172.21.15.88:0/1384567583 shutdown_connections 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.854+0000 7f0c6564d700 1 --2- 172.21.15.88:0/1384567583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c60101440 0x7f0c60101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.854+0000 7f0c6564d700 1 --2- 172.21.15.88:0/1384567583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60100240 0x7f0c60100650 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.854+0000 7f0c6564d700 1 -- 172.21.15.88:0/1384567583 >> 172.21.15.88:0/1384567583 conn(0x7f0c600fb7f0 msgr2=0x7f0c600fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.854+0000 7f0c6564d700 1 -- 172.21.15.88:0/1384567583 shutdown_connections 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.854+0000 7f0c6564d700 1 -- 172.21.15.88:0/1384567583 wait complete. 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.854+0000 7f0c6564d700 1 Processor -- start 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c6564d700 1 -- start start 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c6564d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60100240 0x7f0c601938a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c6564d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c60101440 0x7f0c60193de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c6564d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60194400 con 0x7f0c60100240 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c6564d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60194540 con 0x7f0c60101440 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c5effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60100240 0x7f0c601938a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c5effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60100240 0x7f0c601938a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56972/0 (socket says 172.21.15.88:56972) 2024-02-19T19:00:03.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c5effd700 1 -- 172.21.15.88:0/203597576 learned_addr learned my addr 172.21.15.88:0/203597576 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c567fc700 1 --2- 172.21.15.88:0/203597576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c60101440 0x7f0c60193de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c5effd700 1 -- 172.21.15.88:0/203597576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c60101440 msgr2=0x7f0c60193de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c5effd700 1 --2- 172.21.15.88:0/203597576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c60101440 0x7f0c60193de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c5effd700 1 -- 172.21.15.88:0/203597576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5000f750 con 0x7f0c60100240 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c5effd700 1 --2- 172.21.15.88:0/203597576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60100240 0x7f0c601938a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0c50005820 tx=0x7f0c50017f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c5cff9700 1 -- 172.21.15.88:0/203597576 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c50009a80 con 0x7f0c60100240 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c6564d700 1 -- 172.21.15.88:0/203597576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c60198f90 con 0x7f0c60100240 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c5cff9700 1 -- 172.21.15.88:0/203597576 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0c50009be0 con 0x7f0c60100240 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c5cff9700 1 -- 172.21.15.88:0/203597576 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c50020af0 con 0x7f0c60100240 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.855+0000 7f0c6564d700 1 -- 172.21.15.88:0/203597576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c60199450 con 0x7f0c60100240 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.856+0000 7f0c6564d700 1 -- 172.21.15.88:0/203597576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c60106bc0 con 0x7f0c60100240 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.857+0000 7f0c5cff9700 1 -- 172.21.15.88:0/203597576 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f0c50020db0 con 0x7f0c60100240 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.857+0000 7f0c5cff9700 1 --2- 172.21.15.88:0/203597576 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0c4006c680 0x7f0c4006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.857+0000 7f0c5cff9700 1 -- 172.21.15.88:0/203597576 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0c50091630 con 0x7f0c60100240 2024-02-19T19:00:03.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.857+0000 7f0c567fc700 1 --2- 172.21.15.88:0/203597576 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0c4006c680 0x7f0c4006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.858+0000 7f0c567fc700 1 --2- 172.21.15.88:0/203597576 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0c4006c680 0x7f0c4006eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009c60 tx=0x7f0c48009260 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:02.862+0000 7f0c5cff9700 1 -- 172.21.15.88:0/203597576 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0c5005f740 con 0x7f0c60100240 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.075+0000 7f0c6564d700 1 -- 172.21.15.88:0/203597576 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c60105d90 con 0x7f0c4006c680 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.076+0000 7f0c5cff9700 1 -- 172.21.15.88:0/203597576 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f0c60105d90 con 0x7f0c4006c680 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.078+0000 7f0c6564d700 1 -- 172.21.15.88:0/203597576 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0c4006c680 msgr2=0x7f0c4006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.078+0000 7f0c6564d700 1 --2- 172.21.15.88:0/203597576 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0c4006c680 0x7f0c4006eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0c48009c60 tx=0x7f0c48009260 comp rx=0 tx=0).stop 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.078+0000 7f0c6564d700 1 -- 172.21.15.88:0/203597576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60100240 msgr2=0x7f0c601938a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.078+0000 7f0c6564d700 1 --2- 172.21.15.88:0/203597576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60100240 0x7f0c601938a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0c50005820 tx=0x7f0c50017f80 comp rx=0 tx=0).stop 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.079+0000 7f0c6564d700 1 -- 172.21.15.88:0/203597576 shutdown_connections 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.079+0000 7f0c6564d700 1 --2- 172.21.15.88:0/203597576 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f0c4006c680 0x7f0c4006eb30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.079+0000 7f0c6564d700 1 --2- 172.21.15.88:0/203597576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c60100240 0x7f0c601938a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.079+0000 7f0c6564d700 1 --2- 172.21.15.88:0/203597576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c60101440 0x7f0c60193de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.079+0000 7f0c6564d700 1 -- 172.21.15.88:0/203597576 >> 172.21.15.88:0/203597576 conn(0x7f0c600fb7f0 msgr2=0x7f0c60104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.079+0000 7f0c6564d700 1 -- 172.21.15.88:0/203597576 shutdown_connections 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.079+0000 7f0c6564d700 1 -- 172.21.15.88:0/203597576 wait complete. 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.183+0000 7f7d08dd0700 1 -- 172.21.15.88:0/3909768958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d041003d0 msgr2=0x7f7d04100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.183+0000 7f7d08dd0700 1 --2- 172.21.15.88:0/3909768958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d041003d0 0x7f7d04100840 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8009910 tx=0x7f7cf8009c20 comp rx=0 tx=0).stop 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.183+0000 7f7d08dd0700 1 -- 172.21.15.88:0/3909768958 shutdown_connections 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.183+0000 7f7d08dd0700 1 --2- 172.21.15.88:0/3909768958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d041003d0 0x7f7d04100840 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.183+0000 7f7d08dd0700 1 --2- 172.21.15.88:0/3909768958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040ff130 0x7f7d040ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.183+0000 7f7d08dd0700 1 -- 172.21.15.88:0/3909768958 >> 172.21.15.88:0/3909768958 conn(0x7f7d040fa780 msgr2=0x7f7d040fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.183+0000 7f7d08dd0700 1 -- 172.21.15.88:0/3909768958 shutdown_connections 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.183+0000 7f7d08dd0700 1 -- 172.21.15.88:0/3909768958 wait complete. 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.184+0000 7f7d08dd0700 1 Processor -- start 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.184+0000 7f7d08dd0700 1 -- start start 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.184+0000 7f7d08dd0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040ff130 0x7f7d040690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.184+0000 7f7d08dd0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d041003d0 0x7f7d0419a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.184+0000 7f7d08dd0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0419ad70 con 0x7f7d041003d0 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.184+0000 7f7d08dd0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0419aee0 con 0x7f7d040ff130 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.184+0000 7f7d01d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d041003d0 0x7f7d0419a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.184+0000 7f7d01d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d041003d0 0x7f7d0419a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56984/0 (socket says 172.21.15.88:56984) 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.184+0000 7f7d0259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040ff130 0x7f7d040690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.184+0000 7f7d01d9b700 1 -- 172.21.15.88:0/1993921568 learned_addr learned my addr 172.21.15.88:0/1993921568 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.185+0000 7f7d01d9b700 1 -- 172.21.15.88:0/1993921568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040ff130 msgr2=0x7f7d040690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.185+0000 7f7d01d9b700 1 --2- 172.21.15.88:0/1993921568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040ff130 0x7f7d040690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.185+0000 7f7d01d9b700 1 -- 172.21.15.88:0/1993921568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf80095f0 con 0x7f7d041003d0 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.185+0000 7f7d0259c700 1 --2- 172.21.15.88:0/1993921568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040ff130 0x7f7d040690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.185+0000 7f7d01d9b700 1 --2- 172.21.15.88:0/1993921568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d041003d0 0x7f7d0419a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8000c00 tx=0x7f7cf800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.185+0000 7f7cf37fe700 1 -- 172.21.15.88:0/1993921568 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cf801d070 con 0x7f7d041003d0 2024-02-19T19:00:03.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.185+0000 7f7cf37fe700 1 -- 172.21.15.88:0/1993921568 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7cf8005dc0 con 0x7f7d041003d0 2024-02-19T19:00:03.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.185+0000 7f7d08dd0700 1 -- 172.21.15.88:0/1993921568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0419b160 con 0x7f7d041003d0 2024-02-19T19:00:03.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.185+0000 7f7cf37fe700 1 -- 172.21.15.88:0/1993921568 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cf80226a0 con 0x7f7d041003d0 2024-02-19T19:00:03.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.185+0000 7f7d08dd0700 1 -- 172.21.15.88:0/1993921568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0419b650 con 0x7f7d041003d0 2024-02-19T19:00:03.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.186+0000 7f7d08dd0700 1 -- 172.21.15.88:0/1993921568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d04107c10 con 0x7f7d041003d0 2024-02-19T19:00:03.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.186+0000 7f7cf37fe700 1 -- 172.21.15.88:0/1993921568 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f7cf80038f0 con 0x7f7d041003d0 2024-02-19T19:00:03.558 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:00:03.558 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (5m) 59s ago 7m 24.7M - 0.25.0 c8568f914cd2 d6dd804f52d8 2024-02-19T19:00:03.558 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (8m) 59s ago 8m 7440k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:00:03.558 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (6m) 65s ago 6m 7340k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:00:03.558 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (8m) 59s ago 8m 6660k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (6m) 65s ago 6m 6656k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (5m) 59s ago 7m 86.6M - 9.4.7 954c08fa6188 54907dc6edb4 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (87s) 59s ago 86s 11.5M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (79s) 59s ago 78s 15.9M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (84s) 65s ago 83s 15.9M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (76s) 65s ago 76s 15.1M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:9283,8765,8443 running (9m) 59s ago 9m 488M - 18.2.0 dc2bc1663786 5b5e2fca6e0a 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (5m) 65s ago 5m 439M - 18.2.0 dc2bc1663786 95cabedbd6c4 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (9m) 59s ago 9m 46.0M 2048M 18.2.0 dc2bc1663786 3ea5108455bc 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (5m) 65s ago 5m 37.2M 2048M 18.2.0 dc2bc1663786 ba1c249c6017 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (7m) 59s ago 7m 14.3M - 1.5.0 0da6a335fe13 7cebce4a6d13 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (6m) 65s ago 6m 17.0M - 1.5.0 0da6a335fe13 f93d4349304d 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (5m) 59s ago 5m 49.9M 3946M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:00:03.559 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (4m) 59s ago 4m 48.3M 3946M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:00:03.560 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (4m) 59s ago 4m 46.7M 3946M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:00:03.560 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (3m) 65s ago 3m 48.7M 4970M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:00:03.560 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (3m) 65s ago 3m 46.4M 4970M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:00:03.560 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (2m) 65s ago 2m 44.3M 4970M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:00:03.560 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (5m) 59s ago 6m 58.4M - 2.43.0 a07b618ecd1d 17518e93393b 2024-02-19T19:00:03.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.187+0000 7f7cf37fe700 1 --2- 172.21.15.88:0/1993921568 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7cec06c680 0x7f7cec06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.187+0000 7f7d0259c700 1 --2- 172.21.15.88:0/1993921568 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7cec06c680 0x7f7cec06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.187+0000 7f7cf37fe700 1 -- 172.21.15.88:0/1993921568 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7cf808cd40 con 0x7f7d041003d0 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.187+0000 7f7d0259c700 1 --2- 172.21.15.88:0/1993921568 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7cec06c680 0x7f7cec06eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4011490 tx=0x7f7cf4009260 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.191+0000 7f7cf37fe700 1 -- 172.21.15.88:0/1993921568 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7cf805ae50 con 0x7f7d041003d0 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.396+0000 7f7d08dd0700 1 -- 172.21.15.88:0/1993921568 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d04104d20 con 0x7f7cec06c680 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.406+0000 7f7cf37fe700 1 -- 172.21.15.88:0/1993921568 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f7d04104d20 con 0x7f7cec06c680 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.408+0000 7f7d08dd0700 1 -- 172.21.15.88:0/1993921568 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7cec06c680 msgr2=0x7f7cec06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.408+0000 7f7d08dd0700 1 --2- 172.21.15.88:0/1993921568 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7cec06c680 0x7f7cec06eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4011490 tx=0x7f7cf4009260 comp rx=0 tx=0).stop 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.408+0000 7f7d08dd0700 1 -- 172.21.15.88:0/1993921568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d041003d0 msgr2=0x7f7d0419a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.408+0000 7f7d08dd0700 1 --2- 172.21.15.88:0/1993921568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d041003d0 0x7f7d0419a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7cf8000c00 tx=0x7f7cf800f690 comp rx=0 tx=0).stop 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.408+0000 7f7d08dd0700 1 -- 172.21.15.88:0/1993921568 shutdown_connections 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.408+0000 7f7d08dd0700 1 --2- 172.21.15.88:0/1993921568 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f7cec06c680 0x7f7cec06eb30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.408+0000 7f7d08dd0700 1 --2- 172.21.15.88:0/1993921568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d040ff130 0x7f7d040690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.408+0000 7f7d08dd0700 1 --2- 172.21.15.88:0/1993921568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d041003d0 0x7f7d0419a7a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.408+0000 7f7d08dd0700 1 -- 172.21.15.88:0/1993921568 >> 172.21.15.88:0/1993921568 conn(0x7f7d040fa780 msgr2=0x7f7d04103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.409+0000 7f7d08dd0700 1 -- 172.21.15.88:0/1993921568 shutdown_connections 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.409+0000 7f7d08dd0700 1 -- 172.21.15.88:0/1993921568 wait complete. 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.512+0000 7f353e0fa700 1 -- 172.21.15.88:0/2822164483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35380ff130 msgr2=0x7f35380ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.512+0000 7f353e0fa700 1 --2- 172.21.15.88:0/2822164483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35380ff130 0x7f35380ff540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3528009910 tx=0x7f3528009c20 comp rx=0 tx=0).stop 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.512+0000 7f353e0fa700 1 -- 172.21.15.88:0/2822164483 shutdown_connections 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.512+0000 7f353e0fa700 1 --2- 172.21.15.88:0/2822164483 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35381003d0 0x7f3538100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.512+0000 7f353e0fa700 1 --2- 172.21.15.88:0/2822164483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35380ff130 0x7f35380ff540 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.512+0000 7f353e0fa700 1 -- 172.21.15.88:0/2822164483 >> 172.21.15.88:0/2822164483 conn(0x7f35380fa780 msgr2=0x7f35380fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.512+0000 7f353e0fa700 1 -- 172.21.15.88:0/2822164483 shutdown_connections 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.512+0000 7f353e0fa700 1 -- 172.21.15.88:0/2822164483 wait complete. 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.512+0000 7f353e0fa700 1 Processor -- start 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.512+0000 7f353e0fa700 1 -- start start 2024-02-19T19:00:03.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f353e0fa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35380ff130 0x7f3538195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f353e0fa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35381003d0 0x7f3538195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f353e0fa700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3538196570 con 0x7f35381003d0 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f353e0fa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35381966b0 con 0x7f35380ff130 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f3536ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35381003d0 0x7f3538195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f3536ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35381003d0 0x7f3538195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57008/0 (socket says 172.21.15.88:57008) 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f3536ffd700 1 -- 172.21.15.88:0/853260779 learned_addr learned my addr 172.21.15.88:0/853260779 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f35377fe700 1 --2- 172.21.15.88:0/853260779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35380ff130 0x7f3538195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f3536ffd700 1 -- 172.21.15.88:0/853260779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35380ff130 msgr2=0x7f3538195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f3536ffd700 1 --2- 172.21.15.88:0/853260779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35380ff130 0x7f3538195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f3536ffd700 1 -- 172.21.15.88:0/853260779 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35280095f0 con 0x7f35381003d0 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f3536ffd700 1 --2- 172.21.15.88:0/853260779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35381003d0 0x7f3538195f50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e940 tx=0x7f352c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f3534ff9700 1 -- 172.21.15.88:0/853260779 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f352c00cba0 con 0x7f35381003d0 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f3534ff9700 1 -- 172.21.15.88:0/853260779 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f352c00ee90 con 0x7f35381003d0 2024-02-19T19:00:03.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f3534ff9700 1 -- 172.21.15.88:0/853260779 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f352c005650 con 0x7f35381003d0 2024-02-19T19:00:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f353e0fa700 1 -- 172.21.15.88:0/853260779 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353819b160 con 0x7f35381003d0 2024-02-19T19:00:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.513+0000 7f353e0fa700 1 -- 172.21.15.88:0/853260779 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353819b6b0 con 0x7f35381003d0 2024-02-19T19:00:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.514+0000 7f353e0fa700 1 -- 172.21.15.88:0/853260779 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3538108d10 con 0x7f35381003d0 2024-02-19T19:00:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.515+0000 7f3534ff9700 1 -- 172.21.15.88:0/853260779 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f352c004750 con 0x7f35381003d0 2024-02-19T19:00:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.515+0000 7f3534ff9700 1 --2- 172.21.15.88:0/853260779 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f352006c680 0x7f352006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.515+0000 7f3534ff9700 1 -- 172.21.15.88:0/853260779 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f352c014070 con 0x7f35381003d0 2024-02-19T19:00:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.515+0000 7f35377fe700 1 --2- 172.21.15.88:0/853260779 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f352006c680 0x7f352006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.516+0000 7f35377fe700 1 --2- 172.21.15.88:0/853260779 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f352006c680 0x7f352006eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3528009fd0 tx=0x7f3528005940 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:03.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.520+0000 7f3534ff9700 1 -- 172.21.15.88:0/853260779 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f352c059e30 con 0x7f35381003d0 2024-02-19T19:00:03.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.796+0000 7f353e0fa700 1 -- 172.21.15.88:0/853260779 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3538108ed0 con 0x7f35381003d0 2024-02-19T19:00:03.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.796+0000 7f3534ff9700 1 -- 172.21.15.88:0/853260779 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f352c0599c0 con 0x7f35381003d0 2024-02-19T19:00:03.815 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:00:03.815 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:00:03.815 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-19T19:00:03.815 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:00:03.815 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:00:03.815 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-19T19:00:03.815 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:00:03.815 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:00:03.815 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:00:03.816 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:00:03.816 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:00:03.816 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:00:03.816 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:00:03.816 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:00:03.816 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-02-19T19:00:03.816 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:00:03.816 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:00:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.799+0000 7f353e0fa700 1 -- 172.21.15.88:0/853260779 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f352006c680 msgr2=0x7f352006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.799+0000 7f353e0fa700 1 --2- 172.21.15.88:0/853260779 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f352006c680 0x7f352006eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3528009fd0 tx=0x7f3528005940 comp rx=0 tx=0).stop 2024-02-19T19:00:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.799+0000 7f353e0fa700 1 -- 172.21.15.88:0/853260779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35381003d0 msgr2=0x7f3538195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.799+0000 7f353e0fa700 1 --2- 172.21.15.88:0/853260779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35381003d0 0x7f3538195f50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f352c00e940 tx=0x7f352c00ec50 comp rx=0 tx=0).stop 2024-02-19T19:00:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.799+0000 7f353e0fa700 1 -- 172.21.15.88:0/853260779 shutdown_connections 2024-02-19T19:00:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.799+0000 7f353e0fa700 1 --2- 172.21.15.88:0/853260779 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f352006c680 0x7f352006eb30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.799+0000 7f353e0fa700 1 --2- 172.21.15.88:0/853260779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f35380ff130 0x7f3538195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.799+0000 7f353e0fa700 1 --2- 172.21.15.88:0/853260779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f35381003d0 0x7f3538195f50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.799+0000 7f353e0fa700 1 -- 172.21.15.88:0/853260779 >> 172.21.15.88:0/853260779 conn(0x7f35380fa780 msgr2=0x7f3538103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.799+0000 7f353e0fa700 1 -- 172.21.15.88:0/853260779 shutdown_connections 2024-02-19T19:00:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.799+0000 7f353e0fa700 1 -- 172.21.15.88:0/853260779 wait complete. 2024-02-19T19:00:03.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.905+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1547476890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a4101440 msgr2=0x7fa7a4101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.905+0000 7fa7aa5b5700 1 --2- 172.21.15.88:0/1547476890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a4101440 0x7fa7a4101890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa798009910 tx=0x7fa798009c20 comp rx=0 tx=0).stop 2024-02-19T19:00:03.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.905+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1547476890 shutdown_connections 2024-02-19T19:00:03.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.905+0000 7fa7aa5b5700 1 --2- 172.21.15.88:0/1547476890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a4101440 0x7fa7a4101890 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.905+0000 7fa7aa5b5700 1 --2- 172.21.15.88:0/1547476890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7a4100240 0x7fa7a4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.905+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1547476890 >> 172.21.15.88:0/1547476890 conn(0x7fa7a40fb7f0 msgr2=0x7fa7a40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:03.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.905+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1547476890 shutdown_connections 2024-02-19T19:00:03.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.905+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1547476890 wait complete. 2024-02-19T19:00:03.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.905+0000 7fa7aa5b5700 1 Processor -- start 2024-02-19T19:00:03.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.905+0000 7fa7aa5b5700 1 -- start start 2024-02-19T19:00:03.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7aa5b5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a4100240 0x7fa7a4195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7aa5b5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7a4101440 0x7fa7a4195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7aa5b5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4196610 con 0x7fa7a4100240 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7aa5b5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4196750 con 0x7fa7a4101440 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7a3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a4100240 0x7fa7a4195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7a3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a4100240 0x7fa7a4195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57016/0 (socket says 172.21.15.88:57016) 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7a3fff700 1 -- 172.21.15.88:0/1554215863 learned_addr learned my addr 172.21.15.88:0/1554215863 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7a37fe700 1 --2- 172.21.15.88:0/1554215863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7a4101440 0x7fa7a4195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7a3fff700 1 -- 172.21.15.88:0/1554215863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7a4101440 msgr2=0x7fa7a4195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7a3fff700 1 --2- 172.21.15.88:0/1554215863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7a4101440 0x7fa7a4195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7a3fff700 1 -- 172.21.15.88:0/1554215863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7980095f0 con 0x7fa7a4100240 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7a3fff700 1 --2- 172.21.15.88:0/1554215863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a4100240 0x7fa7a4195ab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa79400e8f0 tx=0x7fa79400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7a17fa700 1 -- 172.21.15.88:0/1554215863 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa794009790 con 0x7fa7a4100240 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1554215863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a419b200 con 0x7fa7a4100240 2024-02-19T19:00:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7a17fa700 1 -- 172.21.15.88:0/1554215863 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa79400c810 con 0x7fa7a4100240 2024-02-19T19:00:03.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7a17fa700 1 -- 172.21.15.88:0/1554215863 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa794003d20 con 0x7fa7a4100240 2024-02-19T19:00:03.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.906+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1554215863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a419b750 con 0x7fa7a4100240 2024-02-19T19:00:03.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.908+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1554215863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a4108d90 con 0x7fa7a4100240 2024-02-19T19:00:03.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.908+0000 7fa7a17fa700 1 -- 172.21.15.88:0/1554215863 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa794018430 con 0x7fa7a4100240 2024-02-19T19:00:03.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.908+0000 7fa7a17fa700 1 --2- 172.21.15.88:0/1554215863 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa78c06c680 0x7fa78c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:03.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.908+0000 7fa7a37fe700 1 --2- 172.21.15.88:0/1554215863 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa78c06c680 0x7fa78c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:03.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.908+0000 7fa7a17fa700 1 -- 172.21.15.88:0/1554215863 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa794014070 con 0x7fa7a4100240 2024-02-19T19:00:03.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.909+0000 7fa7a37fe700 1 --2- 172.21.15.88:0/1554215863 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa78c06c680 0x7fa78c06eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa79800b6f0 tx=0x7fa798005fb0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:03.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:03.913+0000 7fa7a17fa700 1 -- 172.21.15.88:0/1554215863 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa794059e50 con 0x7fa7a4100240 2024-02-19T19:00:04.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:03 smithi088 ceph-mon[29272]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:04.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:03 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/853260779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:00:04.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.124+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1554215863 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7a4105d90 con 0x7fa78c06c680 2024-02-19T19:00:04.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.125+0000 7fa7a17fa700 1 -- 172.21.15.88:0/1554215863 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fa7a4105d90 con 0x7fa78c06c680 2024-02-19T19:00:04.143 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:00:04.143 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:00:04.143 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:00:04.143 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-19T19:00:04.143 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [], 2024-02-19T19:00:04.143 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "0/2 daemons upgraded", 2024-02-19T19:00:04.144 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 image on host smithi132", 2024-02-19T19:00:04.144 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:00:04.144 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:00:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.127+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1554215863 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa78c06c680 msgr2=0x7fa78c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.127+0000 7fa7aa5b5700 1 --2- 172.21.15.88:0/1554215863 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa78c06c680 0x7fa78c06eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa79800b6f0 tx=0x7fa798005fb0 comp rx=0 tx=0).stop 2024-02-19T19:00:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.127+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1554215863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a4100240 msgr2=0x7fa7a4195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.127+0000 7fa7aa5b5700 1 --2- 172.21.15.88:0/1554215863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a4100240 0x7fa7a4195ab0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa79400e8f0 tx=0x7fa79400ec00 comp rx=0 tx=0).stop 2024-02-19T19:00:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.127+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1554215863 shutdown_connections 2024-02-19T19:00:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.127+0000 7fa7aa5b5700 1 --2- 172.21.15.88:0/1554215863 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fa78c06c680 0x7fa78c06eb30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.127+0000 7fa7aa5b5700 1 --2- 172.21.15.88:0/1554215863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7a4100240 0x7fa7a4195ab0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.127+0000 7fa7aa5b5700 1 --2- 172.21.15.88:0/1554215863 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa7a4101440 0x7fa7a4195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.127+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1554215863 >> 172.21.15.88:0/1554215863 conn(0x7fa7a40fb7f0 msgr2=0x7fa7a4104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.127+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1554215863 shutdown_connections 2024-02-19T19:00:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:04.127+0000 7fa7aa5b5700 1 -- 172.21.15.88:0/1554215863 wait complete. 2024-02-19T19:00:04.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:03 smithi132 ceph-mon[37969]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:04.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:03 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/853260779' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:00:05.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:04 smithi132 ceph-mon[37969]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:05.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:04 smithi132 ceph-mon[37969]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:05.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:04 smithi132 ceph-mon[37969]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:05.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:04 smithi088 ceph-mon[29272]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:05.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:04 smithi088 ceph-mon[29272]: from='client.14598 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:05.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:04 smithi088 ceph-mon[29272]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:06.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:05 smithi088 ceph-mon[29272]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:06.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:05 smithi132 ceph-mon[37969]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:07.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:06 smithi088 ceph-mon[29272]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:07.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:06 smithi132 ceph-mon[37969]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:09.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:08 smithi088 ceph-mon[29272]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:09.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:08 smithi132 ceph-mon[37969]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:10.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:09 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:00:10.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:09 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:00:11.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:11 smithi088 ceph-mon[29272]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:11.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:11 smithi132 ceph-mon[37969]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:13.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:13 smithi088 ceph-mon[29272]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:13.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:13 smithi132 ceph-mon[37969]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:15.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:15 smithi088 ceph-mon[29272]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:15.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:15 smithi132 ceph-mon[37969]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:17.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:17 smithi088 ceph-mon[29272]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:17.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:17 smithi132 ceph-mon[37969]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:19.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:19 smithi088 ceph-mon[29272]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:19.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:19 smithi132 ceph-mon[37969]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:21.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:21 smithi088 ceph-mon[29272]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:21.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:21 smithi132 ceph-mon[37969]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:23.250 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:23 smithi132 ceph-mon[37969]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:23.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:23 smithi088 ceph-mon[29272]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:25.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:25 smithi088 ceph-mon[29272]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:25.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:25 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:00:25.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:25 smithi132 ceph-mon[37969]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:25.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:25 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:00:27.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:27 smithi088 ceph-mon[29272]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:27.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:27 smithi132 ceph-mon[37969]: pgmap v237: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:29.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:29 smithi088 ceph-mon[29272]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:29.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:29 smithi132 ceph-mon[37969]: pgmap v238: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:31.302 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:31 smithi088 ceph-mon[29272]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:31.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:31 smithi132 ceph-mon[37969]: pgmap v239: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:33.308 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:33 smithi132 ceph-mon[37969]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:33.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:33 smithi088 ceph-mon[29272]: pgmap v240: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:34.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.243+0000 7efcaa81b700 1 -- 172.21.15.88:0/3679773123 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efca4100d60 msgr2=0x7efca4103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:34.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.243+0000 7efcaa81b700 1 --2- 172.21.15.88:0/3679773123 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efca4100d60 0x7efca4103140 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efc94009910 tx=0x7efc94009c20 comp rx=0 tx=0).stop 2024-02-19T19:00:34.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.243+0000 7efcaa81b700 1 -- 172.21.15.88:0/3679773123 shutdown_connections 2024-02-19T19:00:34.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.243+0000 7efcaa81b700 1 --2- 172.21.15.88:0/3679773123 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efca4103680 0x7efca4105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.243+0000 7efcaa81b700 1 --2- 172.21.15.88:0/3679773123 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efca4100d60 0x7efca4103140 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.243+0000 7efcaa81b700 1 -- 172.21.15.88:0/3679773123 >> 172.21.15.88:0/3679773123 conn(0x7efca40fa780 msgr2=0x7efca40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:34.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.243+0000 7efcaa81b700 1 -- 172.21.15.88:0/3679773123 shutdown_connections 2024-02-19T19:00:34.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.243+0000 7efcaa81b700 1 -- 172.21.15.88:0/3679773123 wait complete. 2024-02-19T19:00:34.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.244+0000 7efcaa81b700 1 Processor -- start 2024-02-19T19:00:34.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.244+0000 7efcaa81b700 1 -- start start 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.244+0000 7efcaa81b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efca4100d60 0x7efca41006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.244+0000 7efcaa81b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efca4103680 0x7efca40fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.244+0000 7efcaa81b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40ff250 con 0x7efca4100d60 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.244+0000 7efcaa81b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca40ff390 con 0x7efca4103680 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.244+0000 7efca3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efca4100d60 0x7efca41006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efca3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efca4100d60 0x7efca41006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:32842/0 (socket says 172.21.15.88:32842) 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efca3fff700 1 -- 172.21.15.88:0/1116658380 learned_addr learned my addr 172.21.15.88:0/1116658380 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efca37fe700 1 --2- 172.21.15.88:0/1116658380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efca4103680 0x7efca40fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efca3fff700 1 -- 172.21.15.88:0/1116658380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efca4103680 msgr2=0x7efca40fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efca3fff700 1 --2- 172.21.15.88:0/1116658380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efca4103680 0x7efca40fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efca3fff700 1 -- 172.21.15.88:0/1116658380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc940095f0 con 0x7efca4100d60 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efca3fff700 1 --2- 172.21.15.88:0/1116658380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efca4100d60 0x7efca41006c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc9400bda0 tx=0x7efc9400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efca17fa700 1 -- 172.21.15.88:0/1116658380 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc9401d070 con 0x7efca4100d60 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efca17fa700 1 -- 172.21.15.88:0/1116658380 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7efc9400f460 con 0x7efca4100d60 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efcaa81b700 1 -- 172.21.15.88:0/1116658380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca40ff610 con 0x7efca4100d60 2024-02-19T19:00:34.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efca17fa700 1 -- 172.21.15.88:0/1116658380 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc94021640 con 0x7efca4100d60 2024-02-19T19:00:34.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.245+0000 7efcaa81b700 1 -- 172.21.15.88:0/1116658380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca40ffb00 con 0x7efca4100d60 2024-02-19T19:00:34.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.246+0000 7efcaa81b700 1 -- 172.21.15.88:0/1116658380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca418fbc0 con 0x7efca4100d60 2024-02-19T19:00:34.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.247+0000 7efca17fa700 1 -- 172.21.15.88:0/1116658380 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7efc9400fa00 con 0x7efca4100d60 2024-02-19T19:00:34.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.247+0000 7efca17fa700 1 --2- 172.21.15.88:0/1116658380 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7efc8c06c680 0x7efc8c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:34.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.247+0000 7efca17fa700 1 -- 172.21.15.88:0/1116658380 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7efc9408cae0 con 0x7efca4100d60 2024-02-19T19:00:34.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.247+0000 7efca37fe700 1 --2- 172.21.15.88:0/1116658380 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7efc8c06c680 0x7efc8c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:34.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.248+0000 7efca37fe700 1 --2- 172.21.15.88:0/1116658380 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7efc8c06c680 0x7efc8c06eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc98005950 tx=0x7efc980058e0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:34.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.252+0000 7efca17fa700 1 -- 172.21.15.88:0/1116658380 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7efc9405abf0 con 0x7efca4100d60 2024-02-19T19:00:34.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.461+0000 7efcaa81b700 1 -- 172.21.15.88:0/1116658380 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efca4061190 con 0x7efc8c06c680 2024-02-19T19:00:34.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.462+0000 7efca17fa700 1 -- 172.21.15.88:0/1116658380 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7efca4061190 con 0x7efc8c06c680 2024-02-19T19:00:34.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.464+0000 7efcaa81b700 1 -- 172.21.15.88:0/1116658380 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7efc8c06c680 msgr2=0x7efc8c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:34.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.464+0000 7efcaa81b700 1 --2- 172.21.15.88:0/1116658380 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7efc8c06c680 0x7efc8c06eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7efc98005950 tx=0x7efc980058e0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.464+0000 7efcaa81b700 1 -- 172.21.15.88:0/1116658380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efca4100d60 msgr2=0x7efca41006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:34.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.464+0000 7efcaa81b700 1 --2- 172.21.15.88:0/1116658380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efca4100d60 0x7efca41006c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc9400bda0 tx=0x7efc9400be80 comp rx=0 tx=0).stop 2024-02-19T19:00:34.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.464+0000 7efcaa81b700 1 -- 172.21.15.88:0/1116658380 shutdown_connections 2024-02-19T19:00:34.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.464+0000 7efcaa81b700 1 --2- 172.21.15.88:0/1116658380 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7efc8c06c680 0x7efc8c06eb30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.464+0000 7efcaa81b700 1 --2- 172.21.15.88:0/1116658380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efca4100d60 0x7efca41006c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.464+0000 7efcaa81b700 1 --2- 172.21.15.88:0/1116658380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efca4103680 0x7efca40fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.464+0000 7efcaa81b700 1 -- 172.21.15.88:0/1116658380 >> 172.21.15.88:0/1116658380 conn(0x7efca40fa780 msgr2=0x7efca40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:34.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.464+0000 7efcaa81b700 1 -- 172.21.15.88:0/1116658380 shutdown_connections 2024-02-19T19:00:34.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.464+0000 7efcaa81b700 1 -- 172.21.15.88:0/1116658380 wait complete. 2024-02-19T19:00:34.494 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:00:34.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.568+0000 7f4722d96700 1 -- 172.21.15.88:0/1316172609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f471c0ff150 msgr2=0x7f471c0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:34.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.568+0000 7f4722d96700 1 --2- 172.21.15.88:0/1316172609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f471c0ff150 0x7f471c0ff560 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4710009910 tx=0x7f4710009c20 comp rx=0 tx=0).stop 2024-02-19T19:00:34.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.568+0000 7f4722d96700 1 -- 172.21.15.88:0/1316172609 shutdown_connections 2024-02-19T19:00:34.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.568+0000 7f4722d96700 1 --2- 172.21.15.88:0/1316172609 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f471c1003f0 0x7f471c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.568+0000 7f4722d96700 1 --2- 172.21.15.88:0/1316172609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f471c0ff150 0x7f471c0ff560 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.568+0000 7f4722d96700 1 -- 172.21.15.88:0/1316172609 >> 172.21.15.88:0/1316172609 conn(0x7f471c0fa7a0 msgr2=0x7f471c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:34.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.568+0000 7f4722d96700 1 -- 172.21.15.88:0/1316172609 shutdown_connections 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.569+0000 7f4722d96700 1 -- 172.21.15.88:0/1316172609 wait complete. 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.569+0000 7f4722d96700 1 Processor -- start 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f4722d96700 1 -- start start 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f4722d96700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f471c0ff150 0x7f471c103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f4722d96700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f471c1003f0 0x7f471c1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f4722d96700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c101d10 con 0x7f471c1003f0 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f4722d96700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c101e50 con 0x7f471c0ff150 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f471bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f471c1003f0 0x7f471c1017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f471bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f471c1003f0 0x7f471c1017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:32860/0 (socket says 172.21.15.88:32860) 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f471bfff700 1 -- 172.21.15.88:0/2416190783 learned_addr learned my addr 172.21.15.88:0/2416190783 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f4720b32700 1 --2- 172.21.15.88:0/2416190783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f471c0ff150 0x7f471c103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f471bfff700 1 -- 172.21.15.88:0/2416190783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f471c0ff150 msgr2=0x7f471c103180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f471bfff700 1 --2- 172.21.15.88:0/2416190783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f471c0ff150 0x7f471c103180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f471bfff700 1 -- 172.21.15.88:0/2416190783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47100095f0 con 0x7f471c1003f0 2024-02-19T19:00:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f471bfff700 1 --2- 172.21.15.88:0/2416190783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f471c1003f0 0x7f471c1017d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f470800d8d0 tx=0x7f470800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.570+0000 7f4719ffb700 1 -- 172.21.15.88:0/2416190783 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47080098a0 con 0x7f471c1003f0 2024-02-19T19:00:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.571+0000 7f4719ffb700 1 -- 172.21.15.88:0/2416190783 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4708010460 con 0x7f471c1003f0 2024-02-19T19:00:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.571+0000 7f4722d96700 1 -- 172.21.15.88:0/2416190783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471c102130 con 0x7f471c1003f0 2024-02-19T19:00:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.571+0000 7f4719ffb700 1 -- 172.21.15.88:0/2416190783 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f470800f5f0 con 0x7f471c1003f0 2024-02-19T19:00:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.571+0000 7f4722d96700 1 -- 172.21.15.88:0/2416190783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471c19de70 con 0x7f471c1003f0 2024-02-19T19:00:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.572+0000 7f4719ffb700 1 -- 172.21.15.88:0/2416190783 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f4708009a00 con 0x7f471c1003f0 2024-02-19T19:00:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.572+0000 7f4722d96700 1 -- 172.21.15.88:0/2416190783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471c068ab0 con 0x7f471c1003f0 2024-02-19T19:00:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.572+0000 7f4719ffb700 1 --2- 172.21.15.88:0/2416190783 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f470c06c680 0x7f470c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.573+0000 7f4719ffb700 1 -- 172.21.15.88:0/2416190783 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f470808b6f0 con 0x7f471c1003f0 2024-02-19T19:00:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.573+0000 7f4720b32700 1 --2- 172.21.15.88:0/2416190783 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f470c06c680 0x7f470c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:34.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.573+0000 7f4720b32700 1 --2- 172.21.15.88:0/2416190783 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f470c06c680 0x7f470c06eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f471000b5c0 tx=0x7f4710005ee0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:34.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.578+0000 7f4719ffb700 1 -- 172.21.15.88:0/2416190783 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4708059880 con 0x7f471c1003f0 2024-02-19T19:00:34.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.787+0000 7f4722d96700 1 -- 172.21.15.88:0/2416190783 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f471c0694e0 con 0x7f470c06c680 2024-02-19T19:00:34.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.788+0000 7f4719ffb700 1 -- 172.21.15.88:0/2416190783 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f471c0694e0 con 0x7f470c06c680 2024-02-19T19:00:34.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.790+0000 7f4722d96700 1 -- 172.21.15.88:0/2416190783 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f470c06c680 msgr2=0x7f470c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.790+0000 7f4722d96700 1 --2- 172.21.15.88:0/2416190783 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f470c06c680 0x7f470c06eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f471000b5c0 tx=0x7f4710005ee0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.790+0000 7f4722d96700 1 -- 172.21.15.88:0/2416190783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f471c1003f0 msgr2=0x7f471c1017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.790+0000 7f4722d96700 1 --2- 172.21.15.88:0/2416190783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f471c1003f0 0x7f471c1017d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f470800d8d0 tx=0x7f470800dbe0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.790+0000 7f4722d96700 1 -- 172.21.15.88:0/2416190783 shutdown_connections 2024-02-19T19:00:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.790+0000 7f4722d96700 1 --2- 172.21.15.88:0/2416190783 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f470c06c680 0x7f470c06eb30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.790+0000 7f4722d96700 1 --2- 172.21.15.88:0/2416190783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f471c0ff150 0x7f471c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.790+0000 7f4722d96700 1 --2- 172.21.15.88:0/2416190783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f471c1003f0 0x7f471c1017d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.790+0000 7f4722d96700 1 -- 172.21.15.88:0/2416190783 >> 172.21.15.88:0/2416190783 conn(0x7f471c0fa7a0 msgr2=0x7f471c068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.790+0000 7f4722d96700 1 -- 172.21.15.88:0/2416190783 shutdown_connections 2024-02-19T19:00:34.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.790+0000 7f4722d96700 1 -- 172.21.15.88:0/2416190783 wait complete. 2024-02-19T19:00:34.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.894+0000 7f70c3fca700 1 -- 172.21.15.88:0/2981656600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70bc0ff150 msgr2=0x7f70bc0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:34.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.894+0000 7f70c3fca700 1 --2- 172.21.15.88:0/2981656600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70bc0ff150 0x7f70bc0ff560 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f70b8009910 tx=0x7f70b8009c20 comp rx=0 tx=0).stop 2024-02-19T19:00:34.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 -- 172.21.15.88:0/2981656600 shutdown_connections 2024-02-19T19:00:34.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 --2- 172.21.15.88:0/2981656600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70bc1003f0 0x7f70bc100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 --2- 172.21.15.88:0/2981656600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70bc0ff150 0x7f70bc0ff560 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 -- 172.21.15.88:0/2981656600 >> 172.21.15.88:0/2981656600 conn(0x7f70bc0fa7a0 msgr2=0x7f70bc0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:34.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 -- 172.21.15.88:0/2981656600 shutdown_connections 2024-02-19T19:00:34.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 -- 172.21.15.88:0/2981656600 wait complete. 2024-02-19T19:00:34.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 Processor -- start 2024-02-19T19:00:34.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 -- start start 2024-02-19T19:00:34.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70bc0ff150 0x7f70bc195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:34.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70bc1003f0 0x7f70bc196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:34.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc196940 con 0x7f70bc0ff150 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.895+0000 7f70c3fca700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70bc196a80 con 0x7f70bc1003f0 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70c1d66700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70bc0ff150 0x7f70bc195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70c1d66700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70bc0ff150 0x7f70bc195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:32888/0 (socket says 172.21.15.88:32888) 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70c1d66700 1 -- 172.21.15.88:0/3020943995 learned_addr learned my addr 172.21.15.88:0/3020943995 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70c1565700 1 --2- 172.21.15.88:0/3020943995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70bc1003f0 0x7f70bc196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70c1d66700 1 -- 172.21.15.88:0/3020943995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70bc1003f0 msgr2=0x7f70bc196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70c1d66700 1 --2- 172.21.15.88:0/3020943995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70bc1003f0 0x7f70bc196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70c1d66700 1 -- 172.21.15.88:0/3020943995 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b80095f0 con 0x7f70bc0ff150 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70c1d66700 1 --2- 172.21.15.88:0/3020943995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70bc0ff150 0x7f70bc195de0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f70b8006010 tx=0x7f70b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70b2ffd700 1 -- 172.21.15.88:0/3020943995 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b801d070 con 0x7f70bc0ff150 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70b2ffd700 1 -- 172.21.15.88:0/3020943995 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f70b800f460 con 0x7f70bc0ff150 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70c3fca700 1 -- 172.21.15.88:0/3020943995 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70bc19b4d0 con 0x7f70bc0ff150 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70b2ffd700 1 -- 172.21.15.88:0/3020943995 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b8017630 con 0x7f70bc0ff150 2024-02-19T19:00:34.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.896+0000 7f70c3fca700 1 -- 172.21.15.88:0/3020943995 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70bc19b970 con 0x7f70bc0ff150 2024-02-19T19:00:34.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.897+0000 7f70c3fca700 1 -- 172.21.15.88:0/3020943995 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70bc068ab0 con 0x7f70bc0ff150 2024-02-19T19:00:34.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.898+0000 7f70b2ffd700 1 -- 172.21.15.88:0/3020943995 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f70b800f5d0 con 0x7f70bc0ff150 2024-02-19T19:00:34.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.898+0000 7f70b2ffd700 1 --2- 172.21.15.88:0/3020943995 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70a806c680 0x7f70a806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:34.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.898+0000 7f70b2ffd700 1 -- 172.21.15.88:0/3020943995 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f70b808ce20 con 0x7f70bc0ff150 2024-02-19T19:00:34.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.898+0000 7f70c1565700 1 --2- 172.21.15.88:0/3020943995 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70a806c680 0x7f70a806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:34.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.899+0000 7f70c1565700 1 --2- 172.21.15.88:0/3020943995 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70a806c680 0x7f70a806eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f70ac005950 tx=0x7f70ac0058e0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:34.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:34.903+0000 7f70b2ffd700 1 -- 172.21.15.88:0/3020943995 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f70b805ae90 con 0x7f70bc0ff150 2024-02-19T19:00:35.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.105+0000 7f70c3fca700 1 -- 172.21.15.88:0/3020943995 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f70bc0694e0 con 0x7f70a806c680 2024-02-19T19:00:35.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.115+0000 7f70b2ffd700 1 -- 172.21.15.88:0/3020943995 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7f70bc0694e0 con 0x7f70a806c680 2024-02-19T19:00:35.135 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:00:35.135 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (6m) 91s ago 8m 24.7M - 0.25.0 c8568f914cd2 d6dd804f52d8 2024-02-19T19:00:35.135 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (8m) 91s ago 8m 7440k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:00:35.135 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (6m) 97s ago 6m 7340k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:00:35.135 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (8m) 91s ago 8m 6660k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:00:35.135 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (6m) 97s ago 6m 6656k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:00:35.135 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (5m) 91s ago 7m 86.6M - 9.4.7 954c08fa6188 54907dc6edb4 2024-02-19T19:00:35.135 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (118s) 91s ago 117s 11.5M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (111s) 91s ago 110s 15.9M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (115s) 97s ago 115s 15.9M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (108s) 97s ago 107s 15.1M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:9283,8765,8443 running (9m) 91s ago 9m 488M - 18.2.0 dc2bc1663786 5b5e2fca6e0a 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (6m) 97s ago 6m 439M - 18.2.0 dc2bc1663786 95cabedbd6c4 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (10m) 91s ago 10m 46.0M 2048M 18.2.0 dc2bc1663786 3ea5108455bc 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (6m) 97s ago 6m 37.2M 2048M 18.2.0 dc2bc1663786 ba1c249c6017 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (8m) 91s ago 8m 14.3M - 1.5.0 0da6a335fe13 7cebce4a6d13 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (6m) 97s ago 6m 17.0M - 1.5.0 0da6a335fe13 f93d4349304d 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (5m) 91s ago 5m 49.9M 3946M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (5m) 91s ago 5m 48.3M 3946M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (4m) 91s ago 4m 46.7M 3946M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (4m) 97s ago 4m 48.7M 4970M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (3m) 97s ago 3m 46.4M 4970M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:00:35.136 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (3m) 97s ago 3m 44.3M 4970M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:00:35.137 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (5m) 91s ago 7m 58.4M - 2.43.0 a07b618ecd1d 17518e93393b 2024-02-19T19:00:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.117+0000 7f70c3fca700 1 -- 172.21.15.88:0/3020943995 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70a806c680 msgr2=0x7f70a806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.117+0000 7f70c3fca700 1 --2- 172.21.15.88:0/3020943995 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70a806c680 0x7f70a806eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f70ac005950 tx=0x7f70ac0058e0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.117+0000 7f70c3fca700 1 -- 172.21.15.88:0/3020943995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70bc0ff150 msgr2=0x7f70bc195de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.117+0000 7f70c3fca700 1 --2- 172.21.15.88:0/3020943995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70bc0ff150 0x7f70bc195de0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f70b8006010 tx=0x7f70b800be20 comp rx=0 tx=0).stop 2024-02-19T19:00:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.117+0000 7f70c3fca700 1 -- 172.21.15.88:0/3020943995 shutdown_connections 2024-02-19T19:00:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.117+0000 7f70c3fca700 1 --2- 172.21.15.88:0/3020943995 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f70a806c680 0x7f70a806eb30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.117+0000 7f70c3fca700 1 --2- 172.21.15.88:0/3020943995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70bc0ff150 0x7f70bc195de0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.117+0000 7f70c3fca700 1 --2- 172.21.15.88:0/3020943995 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70bc1003f0 0x7f70bc196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.117+0000 7f70c3fca700 1 -- 172.21.15.88:0/3020943995 >> 172.21.15.88:0/3020943995 conn(0x7f70bc0fa7a0 msgr2=0x7f70bc068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.118+0000 7f70c3fca700 1 -- 172.21.15.88:0/3020943995 shutdown_connections 2024-02-19T19:00:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.118+0000 7f70c3fca700 1 -- 172.21.15.88:0/3020943995 wait complete. 2024-02-19T19:00:35.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.222+0000 7fec956b9700 1 -- 172.21.15.88:0/2626879766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec90100220 msgr2=0x7fec90100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:35.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.222+0000 7fec956b9700 1 --2- 172.21.15.88:0/2626879766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec90100220 0x7fec90100670 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fec80009910 tx=0x7fec80009c20 comp rx=0 tx=0).stop 2024-02-19T19:00:35.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.222+0000 7fec956b9700 1 -- 172.21.15.88:0/2626879766 shutdown_connections 2024-02-19T19:00:35.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.222+0000 7fec956b9700 1 --2- 172.21.15.88:0/2626879766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec90100220 0x7fec90100670 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.222+0000 7fec956b9700 1 --2- 172.21.15.88:0/2626879766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec900ff020 0x7fec900ff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.222+0000 7fec956b9700 1 -- 172.21.15.88:0/2626879766 >> 172.21.15.88:0/2626879766 conn(0x7fec900fa5d0 msgr2=0x7fec900fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:35.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.222+0000 7fec956b9700 1 -- 172.21.15.88:0/2626879766 shutdown_connections 2024-02-19T19:00:35.242 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:35 smithi088 ceph-mon[29272]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:35.242 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:35 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:35.242 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:35 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:00:35.242 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:35 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T19:00:35.242 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:35 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:00:35.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.222+0000 7fec956b9700 1 -- 172.21.15.88:0/2626879766 wait complete. 2024-02-19T19:00:35.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.222+0000 7fec956b9700 1 Processor -- start 2024-02-19T19:00:35.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec956b9700 1 -- start start 2024-02-19T19:00:35.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec956b9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec900ff020 0x7fec90195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:35.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec956b9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec90100220 0x7fec90195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:35.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec956b9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec90196480 con 0x7fec900ff020 2024-02-19T19:00:35.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec956b9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec901965c0 con 0x7fec90100220 2024-02-19T19:00:35.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec8effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec900ff020 0x7fec90195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec8effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec900ff020 0x7fec90195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:32914/0 (socket says 172.21.15.88:32914) 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec8effd700 1 -- 172.21.15.88:0/3006456221 learned_addr learned my addr 172.21.15.88:0/3006456221 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec8e7fc700 1 --2- 172.21.15.88:0/3006456221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec90100220 0x7fec90195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec8effd700 1 -- 172.21.15.88:0/3006456221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec90100220 msgr2=0x7fec90195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec8effd700 1 --2- 172.21.15.88:0/3006456221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec90100220 0x7fec90195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.223+0000 7fec8effd700 1 -- 172.21.15.88:0/3006456221 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec800095f0 con 0x7fec900ff020 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.224+0000 7fec8effd700 1 --2- 172.21.15.88:0/3006456221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec900ff020 0x7fec90195920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fec8400c8f0 tx=0x7fec8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.224+0000 7fec77fff700 1 -- 172.21.15.88:0/3006456221 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec84007a70 con 0x7fec900ff020 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.224+0000 7fec77fff700 1 -- 172.21.15.88:0/3006456221 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fec84007bd0 con 0x7fec900ff020 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.224+0000 7fec956b9700 1 -- 172.21.15.88:0/3006456221 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec9019b070 con 0x7fec900ff020 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.224+0000 7fec77fff700 1 -- 172.21.15.88:0/3006456221 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec84018630 con 0x7fec900ff020 2024-02-19T19:00:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.224+0000 7fec956b9700 1 -- 172.21.15.88:0/3006456221 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9019b5c0 con 0x7fec900ff020 2024-02-19T19:00:35.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.225+0000 7fec956b9700 1 -- 172.21.15.88:0/3006456221 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec90107b70 con 0x7fec900ff020 2024-02-19T19:00:35.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.225+0000 7fec77fff700 1 -- 172.21.15.88:0/3006456221 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fec84018850 con 0x7fec900ff020 2024-02-19T19:00:35.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.226+0000 7fec77fff700 1 --2- 172.21.15.88:0/3006456221 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fec7806c680 0x7fec7806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:35.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.226+0000 7fec77fff700 1 -- 172.21.15.88:0/3006456221 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fec8408b880 con 0x7fec900ff020 2024-02-19T19:00:35.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.226+0000 7fec8e7fc700 1 --2- 172.21.15.88:0/3006456221 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fec7806c680 0x7fec7806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:35.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.226+0000 7fec8e7fc700 1 --2- 172.21.15.88:0/3006456221 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fec7806c680 0x7fec7806eb30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec800059d0 tx=0x7fec80005920 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:35.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.230+0000 7fec77fff700 1 -- 172.21.15.88:0/3006456221 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fec84059990 con 0x7fec900ff020 2024-02-19T19:00:35.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:35 smithi132 ceph-mon[37969]: pgmap v241: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:35.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:35 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:35.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:35 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:00:35.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:35 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T19:00:35.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:35 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.508+0000 7fec956b9700 1 -- 172.21.15.88:0/3006456221 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec9019bcc0 con 0x7fec900ff020 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.508+0000 7fec77fff700 1 -- 172.21.15.88:0/3006456221 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fec84059520 con 0x7fec900ff020 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:00:35.528 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:00:35.529 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:00:35.529 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:00:35.529 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:00:35.529 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:00:35.529 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-02-19T19:00:35.529 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:00:35.529 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:00:35.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.510+0000 7fec956b9700 1 -- 172.21.15.88:0/3006456221 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fec7806c680 msgr2=0x7fec7806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:35.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.510+0000 7fec956b9700 1 --2- 172.21.15.88:0/3006456221 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fec7806c680 0x7fec7806eb30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fec800059d0 tx=0x7fec80005920 comp rx=0 tx=0).stop 2024-02-19T19:00:35.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.510+0000 7fec956b9700 1 -- 172.21.15.88:0/3006456221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec900ff020 msgr2=0x7fec90195920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:35.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.510+0000 7fec956b9700 1 --2- 172.21.15.88:0/3006456221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec900ff020 0x7fec90195920 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fec8400c8f0 tx=0x7fec8400ccb0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.511+0000 7fec956b9700 1 -- 172.21.15.88:0/3006456221 shutdown_connections 2024-02-19T19:00:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.511+0000 7fec956b9700 1 --2- 172.21.15.88:0/3006456221 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7fec7806c680 0x7fec7806eb30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.511+0000 7fec956b9700 1 --2- 172.21.15.88:0/3006456221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec900ff020 0x7fec90195920 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.511+0000 7fec956b9700 1 --2- 172.21.15.88:0/3006456221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec90100220 0x7fec90195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.511+0000 7fec956b9700 1 -- 172.21.15.88:0/3006456221 >> 172.21.15.88:0/3006456221 conn(0x7fec900fa5d0 msgr2=0x7fec90068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.511+0000 7fec956b9700 1 -- 172.21.15.88:0/3006456221 shutdown_connections 2024-02-19T19:00:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.511+0000 7fec956b9700 1 -- 172.21.15.88:0/3006456221 wait complete. 2024-02-19T19:00:35.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.620+0000 7f65988a6700 1 -- 172.21.15.88:0/2843923679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65941003d0 msgr2=0x7f6594100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:35.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.620+0000 7f65988a6700 1 --2- 172.21.15.88:0/2843923679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65941003d0 0x7f6594100840 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6588009910 tx=0x7f6588009c20 comp rx=0 tx=0).stop 2024-02-19T19:00:35.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.620+0000 7f65988a6700 1 -- 172.21.15.88:0/2843923679 shutdown_connections 2024-02-19T19:00:35.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.620+0000 7f65988a6700 1 --2- 172.21.15.88:0/2843923679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65941003d0 0x7f6594100840 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.620+0000 7f65988a6700 1 --2- 172.21.15.88:0/2843923679 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65940ff130 0x7f65940ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.620+0000 7f65988a6700 1 -- 172.21.15.88:0/2843923679 >> 172.21.15.88:0/2843923679 conn(0x7f65940fa780 msgr2=0x7f65940fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:35.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.620+0000 7f65988a6700 1 -- 172.21.15.88:0/2843923679 shutdown_connections 2024-02-19T19:00:35.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.620+0000 7f65988a6700 1 -- 172.21.15.88:0/2843923679 wait complete. 2024-02-19T19:00:35.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f65988a6700 1 Processor -- start 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f65988a6700 1 -- start start 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f65988a6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65940ff130 0x7f6594194960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f65988a6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65941003d0 0x7f6594194ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f65988a6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65941954c0 con 0x7f65941003d0 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f65988a6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6594195600 con 0x7f65940ff130 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f6591d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65941003d0 0x7f6594194ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f6591d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65941003d0 0x7f6594194ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48300/0 (socket says 172.21.15.88:48300) 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f6591d9b700 1 -- 172.21.15.88:0/3177507005 learned_addr learned my addr 172.21.15.88:0/3177507005 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f659259c700 1 --2- 172.21.15.88:0/3177507005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65940ff130 0x7f6594194960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f6591d9b700 1 -- 172.21.15.88:0/3177507005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65940ff130 msgr2=0x7f6594194960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f6591d9b700 1 --2- 172.21.15.88:0/3177507005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65940ff130 0x7f6594194960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.621+0000 7f6591d9b700 1 -- 172.21.15.88:0/3177507005 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65880095f0 con 0x7f65941003d0 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.622+0000 7f6591d9b700 1 --2- 172.21.15.88:0/3177507005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65941003d0 0x7f6594194ea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f658800b5c0 tx=0x7f658800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.622+0000 7f65837fe700 1 -- 172.21.15.88:0/3177507005 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f658801d070 con 0x7f65941003d0 2024-02-19T19:00:35.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.622+0000 7f65988a6700 1 -- 172.21.15.88:0/3177507005 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659419a050 con 0x7f65941003d0 2024-02-19T19:00:35.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.622+0000 7f65837fe700 1 -- 172.21.15.88:0/3177507005 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f658800f460 con 0x7f65941003d0 2024-02-19T19:00:35.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.622+0000 7f65837fe700 1 -- 172.21.15.88:0/3177507005 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6588021640 con 0x7f65941003d0 2024-02-19T19:00:35.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.622+0000 7f65988a6700 1 -- 172.21.15.88:0/3177507005 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659419a540 con 0x7f65941003d0 2024-02-19T19:00:35.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.623+0000 7f65988a6700 1 -- 172.21.15.88:0/3177507005 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6594107c10 con 0x7f65941003d0 2024-02-19T19:00:35.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.623+0000 7f65837fe700 1 -- 172.21.15.88:0/3177507005 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f658800fa60 con 0x7f65941003d0 2024-02-19T19:00:35.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.624+0000 7f65837fe700 1 --2- 172.21.15.88:0/3177507005 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f657c06c680 0x7f657c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:00:35.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.624+0000 7f659259c700 1 --2- 172.21.15.88:0/3177507005 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f657c06c680 0x7f657c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:00:35.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.624+0000 7f65837fe700 1 -- 172.21.15.88:0/3177507005 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f658808cb30 con 0x7f65941003d0 2024-02-19T19:00:35.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.624+0000 7f659259c700 1 --2- 172.21.15.88:0/3177507005 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f657c06c680 0x7f657c06eb30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f658c009550 tx=0x7f658c0091b0 comp rx=0 tx=0).ready entity=mgr.14245 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:00:35.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.628+0000 7f65837fe700 1 -- 172.21.15.88:0/3177507005 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f658805ac40 con 0x7f65941003d0 2024-02-19T19:00:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.846+0000 7f65988a6700 1 -- 172.21.15.88:0/3177507005 --> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f659419ab80 con 0x7f657c06c680 2024-02-19T19:00:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.847+0000 7f65837fe700 1 -- 172.21.15.88:0/3177507005 <== mgr.14245 v2:172.21.15.88:6800/2764007082 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f659419ab80 con 0x7f657c06c680 2024-02-19T19:00:35.866 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:00:35.867 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:00:35.867 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:00:35.867 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-19T19:00:35.867 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [], 2024-02-19T19:00:35.867 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "0/2 daemons upgraded", 2024-02-19T19:00:35.867 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mgr daemons", 2024-02-19T19:00:35.867 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:00:35.867 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:00:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.849+0000 7f65988a6700 1 -- 172.21.15.88:0/3177507005 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f657c06c680 msgr2=0x7f657c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.849+0000 7f65988a6700 1 --2- 172.21.15.88:0/3177507005 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f657c06c680 0x7f657c06eb30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f658c009550 tx=0x7f658c0091b0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.849+0000 7f65988a6700 1 -- 172.21.15.88:0/3177507005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65941003d0 msgr2=0x7f6594194ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:00:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.849+0000 7f65988a6700 1 --2- 172.21.15.88:0/3177507005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65941003d0 0x7f6594194ea0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f658800b5c0 tx=0x7f658800bd70 comp rx=0 tx=0).stop 2024-02-19T19:00:35.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.849+0000 7f65988a6700 1 -- 172.21.15.88:0/3177507005 shutdown_connections 2024-02-19T19:00:35.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.849+0000 7f65988a6700 1 --2- 172.21.15.88:0/3177507005 >> [v2:172.21.15.88:6800/2764007082,v1:172.21.15.88:6801/2764007082] conn(0x7f657c06c680 0x7f657c06eb30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.849+0000 7f65988a6700 1 --2- 172.21.15.88:0/3177507005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65940ff130 0x7f6594194960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.849+0000 7f65988a6700 1 --2- 172.21.15.88:0/3177507005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65941003d0 0x7f6594194ea0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:00:35.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.849+0000 7f65988a6700 1 -- 172.21.15.88:0/3177507005 >> 172.21.15.88:0/3177507005 conn(0x7f65940fa780 msgr2=0x7f6594103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:00:35.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.849+0000 7f65988a6700 1 -- 172.21.15.88:0/3177507005 shutdown_connections 2024-02-19T19:00:35.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:00:35.849+0000 7f65988a6700 1 -- 172.21.15.88:0/3177507005 wait complete. 2024-02-19T19:00:36.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:36 smithi088 ceph-mon[29272]: Upgrade: Updating mgr.smithi132.psllsa 2024-02-19T19:00:36.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:36 smithi088 ceph-mon[29272]: Deploying daemon mgr.smithi132.psllsa on smithi132 2024-02-19T19:00:36.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:36 smithi088 ceph-mon[29272]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:36.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:36 smithi088 ceph-mon[29272]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:36.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:36 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/3006456221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:00:36.430 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:36 smithi132 ceph-mon[37969]: Upgrade: Updating mgr.smithi132.psllsa 2024-02-19T19:00:36.430 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:36 smithi132 ceph-mon[37969]: Deploying daemon mgr.smithi132.psllsa on smithi132 2024-02-19T19:00:36.431 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:36 smithi132 ceph-mon[37969]: from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:36.431 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:36 smithi132 ceph-mon[37969]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:36.431 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:36 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/3006456221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:00:37.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:37 smithi132 ceph-mon[37969]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:37.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:37 smithi132 ceph-mon[37969]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:37.135 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:37 smithi132 ceph-mon[37969]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:37.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:37 smithi088 ceph-mon[29272]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:37.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:37 smithi088 ceph-mon[29272]: pgmap v242: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:37.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:37 smithi088 ceph-mon[29272]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:00:39.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:38 smithi088 ceph-mon[29272]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:39.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:38 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:39.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:38 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:39.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:38 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:00:39.142 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:38 smithi132 ceph-mon[37969]: pgmap v243: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:39.142 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:38 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:39.142 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:38 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:39.142 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:38 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:00:40.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:39 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:00:40.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:39 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:00:41.098 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:40 smithi088 ceph-mon[29272]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:41.173 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:40 smithi132 ceph-mon[37969]: pgmap v244: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:43.141 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:42 smithi132 ceph-mon[37969]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:43.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:42 smithi088 ceph-mon[29272]: pgmap v245: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:44.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:43 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:44.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:43 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:44.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:43 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:44.302 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:43 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:44.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:43 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:44.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:43 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:44.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:43 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:44.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:43 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:44 smithi132 ceph-mon[37969]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:44 smithi132 ceph-mon[37969]: Standby manager daemon smithi132.psllsa restarted 2024-02-19T19:00:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:44 smithi132 ceph-mon[37969]: Standby manager daemon smithi132.psllsa started 2024-02-19T19:00:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:44 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/crt"}]: dispatch 2024-02-19T19:00:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:44 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:00:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:44 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/key"}]: dispatch 2024-02-19T19:00:45.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:44 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:00:45.272 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:44 smithi088 ceph-mon[29272]: pgmap v246: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:45.272 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:44 smithi088 ceph-mon[29272]: Standby manager daemon smithi132.psllsa restarted 2024-02-19T19:00:45.272 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:44 smithi088 ceph-mon[29272]: Standby manager daemon smithi132.psllsa started 2024-02-19T19:00:45.272 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:44 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/crt"}]: dispatch 2024-02-19T19:00:45.272 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:44 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:00:45.272 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:44 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/key"}]: dispatch 2024-02-19T19:00:45.272 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:44 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:00:46.140 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:46 smithi132 ceph-mon[37969]: mgrmap e20: smithi088.gyviqq(active, since 7m), standbys: smithi132.psllsa 2024-02-19T19:00:46.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:46 smithi088 ceph-mon[29272]: mgrmap e20: smithi088.gyviqq(active, since 7m), standbys: smithi132.psllsa 2024-02-19T19:00:47.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:47 smithi132 ceph-mon[37969]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:47.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:47 smithi088 ceph-mon[29272]: pgmap v247: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:48.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:48.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:48.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:00:48.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:00:48.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:48.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr fail", "who": "smithi088.gyviqq"}]: dispatch 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: osdmap e47: 6 total, 6 up, 6 in 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "mgr fail", "who": "smithi088.gyviqq"}]': finished 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: mgrmap e21: smithi132.psllsa(active, starting, since 0.00618898s) 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mgr metadata", "who": "smithi132.psllsa", "id": "smithi132.psllsa"}]: dispatch 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T19:00:48.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: Manager daemon smithi132.psllsa is now available 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi132.psllsa/mirror_snapshot_schedule"}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi132.psllsa/mirror_snapshot_schedule"}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi132.psllsa/trash_purge_schedule"}]: dispatch 2024-02-19T19:00:48.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:48 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi132.psllsa/trash_purge_schedule"}]: dispatch 2024-02-19T19:00:48.828 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:48.828 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:48.828 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:00:48.828 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr fail", "who": "smithi088.gyviqq"}]: dispatch 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: osdmap e47: 6 total, 6 up, 6 in 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.14245 172.21.15.88:0/1243353579' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "mgr fail", "who": "smithi088.gyviqq"}]': finished 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: mgrmap e21: smithi132.psllsa(active, starting, since 0.00618898s) 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:00:48.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mgr metadata", "who": "smithi132.psllsa", "id": "smithi132.psllsa"}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: Manager daemon smithi132.psllsa is now available 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:00:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi132.psllsa/mirror_snapshot_schedule"}]: dispatch 2024-02-19T19:00:48.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi132.psllsa/mirror_snapshot_schedule"}]: dispatch 2024-02-19T19:00:48.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi132.psllsa/trash_purge_schedule"}]: dispatch 2024-02-19T19:00:48.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:48 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi132.psllsa/trash_purge_schedule"}]: dispatch 2024-02-19T19:00:49.803 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:49 smithi088 ceph-mon[29272]: Deploying cephadm binary to smithi088 2024-02-19T19:00:49.803 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:49 smithi088 ceph-mon[29272]: Deploying cephadm binary to smithi132 2024-02-19T19:00:49.803 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:49 smithi088 ceph-mon[29272]: mgrmap e22: smithi132.psllsa(active, since 1.00785s) 2024-02-19T19:00:49.803 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:49 smithi088 ceph-mon[29272]: pgmap v3: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:49.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:49 smithi088 ceph-mon[29272]: [19/Feb/2024:19:00:48] ENGINE Bus STARTING 2024-02-19T19:00:50.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:49 smithi132 ceph-mon[37969]: Deploying cephadm binary to smithi088 2024-02-19T19:00:50.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:49 smithi132 ceph-mon[37969]: Deploying cephadm binary to smithi132 2024-02-19T19:00:50.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:49 smithi132 ceph-mon[37969]: mgrmap e22: smithi132.psllsa(active, since 1.00785s) 2024-02-19T19:00:50.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:49 smithi132 ceph-mon[37969]: pgmap v3: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:50.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:49 smithi132 ceph-mon[37969]: [19/Feb/2024:19:00:48] ENGINE Bus STARTING 2024-02-19T19:00:50.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:50 smithi088 ceph-mon[29272]: [19/Feb/2024:19:00:48] ENGINE Serving on https://172.21.15.132:7150 2024-02-19T19:00:50.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:50 smithi088 ceph-mon[29272]: [19/Feb/2024:19:00:48] ENGINE Serving on http://172.21.15.132:8765 2024-02-19T19:00:50.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:50 smithi088 ceph-mon[29272]: [19/Feb/2024:19:00:48] ENGINE Bus STARTED 2024-02-19T19:00:50.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:50 smithi088 ceph-mon[29272]: pgmap v4: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:50.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:50 smithi088 ceph-mon[29272]: mgrmap e23: smithi132.psllsa(active, since 2s) 2024-02-19T19:00:51.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:50 smithi132 ceph-mon[37969]: [19/Feb/2024:19:00:48] ENGINE Serving on https://172.21.15.132:7150 2024-02-19T19:00:51.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:50 smithi132 ceph-mon[37969]: [19/Feb/2024:19:00:48] ENGINE Serving on http://172.21.15.132:8765 2024-02-19T19:00:51.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:50 smithi132 ceph-mon[37969]: [19/Feb/2024:19:00:48] ENGINE Bus STARTED 2024-02-19T19:00:51.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:50 smithi132 ceph-mon[37969]: pgmap v4: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:51.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:50 smithi132 ceph-mon[37969]: mgrmap e23: smithi132.psllsa(active, since 2s) 2024-02-19T19:00:52.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:52 smithi088 ceph-mon[29272]: pgmap v5: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:52.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:52 smithi088 ceph-mon[29272]: mgrmap e24: smithi132.psllsa(active, since 4s) 2024-02-19T19:00:53.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:52 smithi132 ceph-mon[37969]: pgmap v5: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:53.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:52 smithi132 ceph-mon[37969]: mgrmap e24: smithi132.psllsa(active, since 4s) 2024-02-19T19:00:54.712 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:54 smithi088 ceph-mon[29272]: pgmap v6: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:54.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:54 smithi088 ceph-mon[29272]: Standby manager daemon smithi088.gyviqq started 2024-02-19T19:00:54.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:54 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.88:0/433809496' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.gyviqq/crt"}]: dispatch 2024-02-19T19:00:54.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:54 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.88:0/433809496' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:00:54.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:54 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.88:0/433809496' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.gyviqq/key"}]: dispatch 2024-02-19T19:00:54.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:54 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.88:0/433809496' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:00:54.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:54 smithi132 ceph-mon[37969]: pgmap v6: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:00:54.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:54 smithi132 ceph-mon[37969]: Standby manager daemon smithi088.gyviqq started 2024-02-19T19:00:54.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:54 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.88:0/433809496' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.gyviqq/crt"}]: dispatch 2024-02-19T19:00:54.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:54 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.88:0/433809496' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:00:54.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:54 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.88:0/433809496' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.gyviqq/key"}]: dispatch 2024-02-19T19:00:54.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:54 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.88:0/433809496' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:00:55.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:55 smithi088 ceph-mon[29272]: mgrmap e25: smithi132.psllsa(active, since 7s), standbys: smithi088.gyviqq 2024-02-19T19:00:55.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:55 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mgr metadata", "who": "smithi088.gyviqq", "id": "smithi088.gyviqq"}]: dispatch 2024-02-19T19:00:56.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:55 smithi132 ceph-mon[37969]: mgrmap e25: smithi132.psllsa(active, since 7s), standbys: smithi088.gyviqq 2024-02-19T19:00:56.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:55 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mgr metadata", "who": "smithi088.gyviqq", "id": "smithi088.gyviqq"}]: dispatch 2024-02-19T19:00:56.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:56 smithi088 ceph-mon[29272]: pgmap v7: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-19T19:00:56.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:56 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:56.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:56 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:57.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:56 smithi132 ceph-mon[37969]: pgmap v7: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-19T19:00:57.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:56 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:57.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:56 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:58.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:57 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:58.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:57 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:58.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:57 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:58.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:57 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:58.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:57 smithi132 ceph-mon[37969]: pgmap v8: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-19T19:00:58.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:57 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:58.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:57 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:58.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:57 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:58.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:57 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:58.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:57 smithi088 ceph-mon[29272]: pgmap v8: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-19T19:00:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:59 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:59 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:00:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:59 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:00:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:59 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:00:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:59 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:00:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:59 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:00:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:59 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:00:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:59 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:00:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:59 smithi088 ceph-mon[29272]: Adjusting osd_memory_target on smithi088 to 1216M 2024-02-19T19:00:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:00:59 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:00.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:59 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:00.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:59 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:00.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:59 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:00.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:59 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:00.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:59 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:00.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:59 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:00.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:59 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:00.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:59 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:00.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:59 smithi132 ceph-mon[37969]: Adjusting osd_memory_target on smithi088 to 1216M 2024-02-19T19:01:00.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:00:59 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:00.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:00 smithi088 ceph-mon[29272]: pgmap v9: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-19T19:01:00.900 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:00 smithi132 ceph-mon[37969]: pgmap v9: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-19T19:01:02.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:02 smithi088 ceph-mon[29272]: pgmap v10: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:03.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:02 smithi132 ceph-mon[37969]: pgmap v10: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:03.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:03 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:01:04.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:03 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:01:04.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:04 smithi088 ceph-mon[29272]: pgmap v11: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:05.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:04 smithi132 ceph-mon[37969]: pgmap v11: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:05.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.969+0000 7f5684bd7700 1 -- 172.21.15.88:0/1853736963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101440 msgr2=0x7f5680101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:05.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.969+0000 7f5684bd7700 1 --2- 172.21.15.88:0/1853736963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101440 0x7f5680101890 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5670009910 tx=0x7f5670009c20 comp rx=0 tx=0).stop 2024-02-19T19:01:05.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.970+0000 7f5684bd7700 1 -- 172.21.15.88:0/1853736963 shutdown_connections 2024-02-19T19:01:05.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.970+0000 7f5684bd7700 1 --2- 172.21.15.88:0/1853736963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101440 0x7f5680101890 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:05.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.970+0000 7f5684bd7700 1 --2- 172.21.15.88:0/1853736963 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5680100240 0x7f5680100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:05.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.970+0000 7f5684bd7700 1 -- 172.21.15.88:0/1853736963 >> 172.21.15.88:0/1853736963 conn(0x7f56800fb7f0 msgr2=0x7f56800fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:05.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.970+0000 7f5684bd7700 1 -- 172.21.15.88:0/1853736963 shutdown_connections 2024-02-19T19:01:05.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.970+0000 7f5684bd7700 1 -- 172.21.15.88:0/1853736963 wait complete. 2024-02-19T19:01:05.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.970+0000 7f5684bd7700 1 Processor -- start 2024-02-19T19:01:05.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.970+0000 7f5684bd7700 1 -- start start 2024-02-19T19:01:05.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f5684bd7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5680100240 0x7f5680195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:05.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f5684bd7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101440 0x7f56801960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f5684bd7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56801966e0 con 0x7f5680101440 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f5684bd7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5680196820 con 0x7f5680100240 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f567dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101440 0x7f56801960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f567dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101440 0x7f56801960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57276/0 (socket says 172.21.15.88:57276) 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f567dd9b700 1 -- 172.21.15.88:0/3610449896 learned_addr learned my addr 172.21.15.88:0/3610449896 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f567e59c700 1 --2- 172.21.15.88:0/3610449896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5680100240 0x7f5680195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f567dd9b700 1 -- 172.21.15.88:0/3610449896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5680100240 msgr2=0x7f5680195b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f567dd9b700 1 --2- 172.21.15.88:0/3610449896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5680100240 0x7f5680195b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f567dd9b700 1 -- 172.21.15.88:0/3610449896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56700095f0 con 0x7f5680101440 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f567dd9b700 1 --2- 172.21.15.88:0/3610449896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101440 0x7f56801960c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f567000bd70 tx=0x7f567000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f566f7fe700 1 -- 172.21.15.88:0/3610449896 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f567001d070 con 0x7f5680101440 2024-02-19T19:01:05.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f566f7fe700 1 -- 172.21.15.88:0/3610449896 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5670022470 con 0x7f5680101440 2024-02-19T19:01:05.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f5684bd7700 1 -- 172.21.15.88:0/3610449896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f568019b270 con 0x7f5680101440 2024-02-19T19:01:05.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f566f7fe700 1 -- 172.21.15.88:0/3610449896 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f567000f670 con 0x7f5680101440 2024-02-19T19:01:05.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.971+0000 7f5684bd7700 1 -- 172.21.15.88:0/3610449896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f568019b760 con 0x7f5680101440 2024-02-19T19:01:05.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.973+0000 7f5684bd7700 1 -- 172.21.15.88:0/3610449896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5680108e30 con 0x7f5680101440 2024-02-19T19:01:05.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.973+0000 7f566f7fe700 1 -- 172.21.15.88:0/3610449896 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7f5670022a50 con 0x7f5680101440 2024-02-19T19:01:05.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.973+0000 7f566f7fe700 1 --2- 172.21.15.88:0/3610449896 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f566806cf10 0x7f566806f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:05.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.973+0000 7f566f7fe700 1 -- 172.21.15.88:0/3610449896 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f567008dd50 con 0x7f5680101440 2024-02-19T19:01:05.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.974+0000 7f567e59c700 1 --2- 172.21.15.88:0/3610449896 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f566806cf10 0x7f566806f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:05.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.974+0000 7f567e59c700 1 --2- 172.21.15.88:0/3610449896 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f566806cf10 0x7f566806f3c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5674006fd0 tx=0x7f5674008040 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:05.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:05.978+0000 7f566f7fe700 1 -- 172.21.15.88:0/3610449896 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f567005b540 con 0x7f5680101440 2024-02-19T19:01:06.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.191+0000 7f5684bd7700 1 -- 172.21.15.88:0/3610449896 --> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f568019bda0 con 0x7f566806cf10 2024-02-19T19:01:06.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.193+0000 7f566f7fe700 1 -- 172.21.15.88:0/3610449896 <== mgr.24359 v2:172.21.15.132:6828/3141443905 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f568019bda0 con 0x7f566806cf10 2024-02-19T19:01:06.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.195+0000 7f5684bd7700 1 -- 172.21.15.88:0/3610449896 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f566806cf10 msgr2=0x7f566806f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.195+0000 7f5684bd7700 1 --2- 172.21.15.88:0/3610449896 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f566806cf10 0x7f566806f3c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5674006fd0 tx=0x7f5674008040 comp rx=0 tx=0).stop 2024-02-19T19:01:06.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.195+0000 7f5684bd7700 1 -- 172.21.15.88:0/3610449896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101440 msgr2=0x7f56801960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.195+0000 7f5684bd7700 1 --2- 172.21.15.88:0/3610449896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101440 0x7f56801960c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f567000bd70 tx=0x7f567000be50 comp rx=0 tx=0).stop 2024-02-19T19:01:06.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.195+0000 7f5684bd7700 1 -- 172.21.15.88:0/3610449896 shutdown_connections 2024-02-19T19:01:06.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.195+0000 7f5684bd7700 1 --2- 172.21.15.88:0/3610449896 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f566806cf10 0x7f566806f3c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.195+0000 7f5684bd7700 1 --2- 172.21.15.88:0/3610449896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5680100240 0x7f5680195b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.195+0000 7f5684bd7700 1 --2- 172.21.15.88:0/3610449896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5680101440 0x7f56801960c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.195+0000 7f5684bd7700 1 -- 172.21.15.88:0/3610449896 >> 172.21.15.88:0/3610449896 conn(0x7f56800fb7f0 msgr2=0x7f5680104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:06.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.195+0000 7f5684bd7700 1 -- 172.21.15.88:0/3610449896 shutdown_connections 2024-02-19T19:01:06.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.195+0000 7f5684bd7700 1 -- 172.21.15.88:0/3610449896 wait complete. 2024-02-19T19:01:06.224 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:01:06.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.300+0000 7f405ffff700 1 -- 172.21.15.88:0/2938440890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40600feff0 msgr2=0x7f40600ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.300+0000 7f405ffff700 1 --2- 172.21.15.88:0/2938440890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40600feff0 0x7f40600ff400 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4054009910 tx=0x7f4054009c20 comp rx=0 tx=0).stop 2024-02-19T19:01:06.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.300+0000 7f405ffff700 1 -- 172.21.15.88:0/2938440890 shutdown_connections 2024-02-19T19:01:06.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.300+0000 7f405ffff700 1 --2- 172.21.15.88:0/2938440890 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40601001f0 0x7f4060100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.300+0000 7f405ffff700 1 --2- 172.21.15.88:0/2938440890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40600feff0 0x7f40600ff400 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.300+0000 7f405ffff700 1 -- 172.21.15.88:0/2938440890 >> 172.21.15.88:0/2938440890 conn(0x7f40600fa5c0 msgr2=0x7f40600fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:06.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.300+0000 7f405ffff700 1 -- 172.21.15.88:0/2938440890 shutdown_connections 2024-02-19T19:01:06.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.300+0000 7f405ffff700 1 -- 172.21.15.88:0/2938440890 wait complete. 2024-02-19T19:01:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.301+0000 7f405ffff700 1 Processor -- start 2024-02-19T19:01:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.301+0000 7f405ffff700 1 -- start start 2024-02-19T19:01:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.301+0000 7f405ffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40600feff0 0x7f4060102f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.301+0000 7f405ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40601001f0 0x7f40601015d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.301+0000 7f405ffff700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4060101b10 con 0x7f40601001f0 2024-02-19T19:01:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.301+0000 7f405ffff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4060101c50 con 0x7f40600feff0 2024-02-19T19:01:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.301+0000 7f405e7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40601001f0 0x7f40601015d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.301+0000 7f405e7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40601001f0 0x7f40601015d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57284/0 (socket says 172.21.15.88:57284) 2024-02-19T19:01:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.301+0000 7f405e7fc700 1 -- 172.21.15.88:0/1897209397 learned_addr learned my addr 172.21.15.88:0/1897209397 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:01:06.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.301+0000 7f405effd700 1 --2- 172.21.15.88:0/1897209397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40600feff0 0x7f4060102f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.302+0000 7f405e7fc700 1 -- 172.21.15.88:0/1897209397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40600feff0 msgr2=0x7f4060102f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.302+0000 7f405e7fc700 1 --2- 172.21.15.88:0/1897209397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40600feff0 0x7f4060102f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.302+0000 7f405e7fc700 1 -- 172.21.15.88:0/1897209397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40540095f0 con 0x7f40601001f0 2024-02-19T19:01:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.302+0000 7f405e7fc700 1 --2- 172.21.15.88:0/1897209397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40601001f0 0x7f40601015d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f405000b4b0 tx=0x7f405000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.302+0000 7f4047fff700 1 -- 172.21.15.88:0/1897209397 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4050011800 con 0x7f40601001f0 2024-02-19T19:01:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.302+0000 7f4047fff700 1 -- 172.21.15.88:0/1897209397 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4050011e10 con 0x7f40601001f0 2024-02-19T19:01:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.302+0000 7f405ffff700 1 -- 172.21.15.88:0/1897209397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4060101f30 con 0x7f40601001f0 2024-02-19T19:01:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.302+0000 7f4047fff700 1 -- 172.21.15.88:0/1897209397 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f405000f2e0 con 0x7f40601001f0 2024-02-19T19:01:06.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.302+0000 7f405ffff700 1 -- 172.21.15.88:0/1897209397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406019d970 con 0x7f40601001f0 2024-02-19T19:01:06.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.303+0000 7f405ffff700 1 -- 172.21.15.88:0/1897209397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4060107bc0 con 0x7f40601001f0 2024-02-19T19:01:06.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.304+0000 7f4047fff700 1 -- 172.21.15.88:0/1897209397 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7f4050010380 con 0x7f40601001f0 2024-02-19T19:01:06.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.304+0000 7f4047fff700 1 --2- 172.21.15.88:0/1897209397 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f404806cf10 0x7f404806f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:06.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.304+0000 7f4047fff700 1 -- 172.21.15.88:0/1897209397 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f405008c020 con 0x7f40601001f0 2024-02-19T19:01:06.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.304+0000 7f405effd700 1 --2- 172.21.15.88:0/1897209397 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f404806cf10 0x7f404806f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:06.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.305+0000 7f405effd700 1 --2- 172.21.15.88:0/1897209397 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f404806cf10 0x7f404806f3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4054006010 tx=0x7f4054005280 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:06.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.309+0000 7f4047fff700 1 -- 172.21.15.88:0/1897209397 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f40500598f0 con 0x7f40601001f0 2024-02-19T19:01:06.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.523+0000 7f405ffff700 1 -- 172.21.15.88:0/1897209397 --> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f406019df20 con 0x7f404806cf10 2024-02-19T19:01:06.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.525+0000 7f4047fff700 1 -- 172.21.15.88:0/1897209397 <== mgr.24359 v2:172.21.15.132:6828/3141443905 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f406019df20 con 0x7f404806cf10 2024-02-19T19:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.527+0000 7f405ffff700 1 -- 172.21.15.88:0/1897209397 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f404806cf10 msgr2=0x7f404806f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.527+0000 7f405ffff700 1 --2- 172.21.15.88:0/1897209397 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f404806cf10 0x7f404806f3c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4054006010 tx=0x7f4054005280 comp rx=0 tx=0).stop 2024-02-19T19:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.527+0000 7f405ffff700 1 -- 172.21.15.88:0/1897209397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40601001f0 msgr2=0x7f40601015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.527+0000 7f405ffff700 1 --2- 172.21.15.88:0/1897209397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40601001f0 0x7f40601015d0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f405000b4b0 tx=0x7f405000b7c0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.527+0000 7f405ffff700 1 -- 172.21.15.88:0/1897209397 shutdown_connections 2024-02-19T19:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.527+0000 7f405ffff700 1 --2- 172.21.15.88:0/1897209397 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f404806cf10 0x7f404806f3c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.527+0000 7f405ffff700 1 --2- 172.21.15.88:0/1897209397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40600feff0 0x7f4060102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.527+0000 7f405ffff700 1 --2- 172.21.15.88:0/1897209397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40601001f0 0x7f40601015d0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.527+0000 7f405ffff700 1 -- 172.21.15.88:0/1897209397 >> 172.21.15.88:0/1897209397 conn(0x7f40600fa5c0 msgr2=0x7f4060068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:06.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.527+0000 7f405ffff700 1 -- 172.21.15.88:0/1897209397 shutdown_connections 2024-02-19T19:01:06.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.527+0000 7f405ffff700 1 -- 172.21.15.88:0/1897209397 wait complete. 2024-02-19T19:01:06.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.636+0000 7f5201596700 1 -- 172.21.15.88:0/2657186666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51fc103680 msgr2=0x7f51fc105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.636+0000 7f5201596700 1 --2- 172.21.15.88:0/2657186666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51fc103680 0x7f51fc105a60 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f51ec009910 tx=0x7f51ec009c20 comp rx=0 tx=0).stop 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.636+0000 7f5201596700 1 -- 172.21.15.88:0/2657186666 shutdown_connections 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.636+0000 7f5201596700 1 --2- 172.21.15.88:0/2657186666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51fc103680 0x7f51fc105a60 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.636+0000 7f5201596700 1 --2- 172.21.15.88:0/2657186666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51fc100d60 0x7f51fc103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.636+0000 7f5201596700 1 -- 172.21.15.88:0/2657186666 >> 172.21.15.88:0/2657186666 conn(0x7f51fc0fa780 msgr2=0x7f51fc0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.636+0000 7f5201596700 1 -- 172.21.15.88:0/2657186666 shutdown_connections 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.636+0000 7f5201596700 1 -- 172.21.15.88:0/2657186666 wait complete. 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.636+0000 7f5201596700 1 Processor -- start 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f5201596700 1 -- start start 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f5201596700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51fc100d60 0x7f51fc195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f5201596700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51fc103680 0x7f51fc196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f5201596700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc196640 con 0x7f51fc100d60 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f5201596700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc196780 con 0x7f51fc103680 2024-02-19T19:01:06.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f51faffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51fc100d60 0x7f51fc195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f51faffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51fc100d60 0x7f51fc195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57304/0 (socket says 172.21.15.88:57304) 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f51faffd700 1 -- 172.21.15.88:0/2875959325 learned_addr learned my addr 172.21.15.88:0/2875959325 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f51fa7fc700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51fc103680 0x7f51fc196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f51faffd700 1 -- 172.21.15.88:0/2875959325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51fc103680 msgr2=0x7f51fc196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f51faffd700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51fc103680 0x7f51fc196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f51faffd700 1 -- 172.21.15.88:0/2875959325 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0095f0 con 0x7f51fc100d60 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.637+0000 7f51fa7fc700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51fc103680 0x7f51fc196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.638+0000 7f51faffd700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51fc100d60 0x7f51fc195ae0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f51f000d900 tx=0x7f51f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.638+0000 7f51e3fff700 1 -- 172.21.15.88:0/2875959325 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f0009900 con 0x7f51fc100d60 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.638+0000 7f51e3fff700 1 -- 172.21.15.88:0/2875959325 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51f0010460 con 0x7f51fc100d60 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.638+0000 7f5201596700 1 -- 172.21.15.88:0/2875959325 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc19b230 con 0x7f51fc100d60 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.638+0000 7f51e3fff700 1 -- 172.21.15.88:0/2875959325 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f000f5f0 con 0x7f51fc100d60 2024-02-19T19:01:06.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.638+0000 7f5201596700 1 -- 172.21.15.88:0/2875959325 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc19b780 con 0x7f51fc100d60 2024-02-19T19:01:06.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.639+0000 7f5201596700 1 -- 172.21.15.88:0/2875959325 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc18fbe0 con 0x7f51fc100d60 2024-02-19T19:01:06.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.639+0000 7f51e3fff700 1 -- 172.21.15.88:0/2875959325 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7f51f000fa80 con 0x7f51fc100d60 2024-02-19T19:01:06.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.640+0000 7f51e3fff700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f51e406cf10 0x7f51e406f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:06.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.640+0000 7f51e3fff700 1 -- 172.21.15.88:0/2875959325 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f51f008c540 con 0x7f51fc100d60 2024-02-19T19:01:06.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.640+0000 7f51fa7fc700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f51e406cf10 0x7f51e406f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:06.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.640+0000 7f51fa7fc700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f51e406cf10 0x7f51e406f3c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f51ec005b40 tx=0x7f51ec005a90 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:06.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.645+0000 7f51e3fff700 1 -- 172.21.15.88:0/2875959325 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f51f0059d60 con 0x7f51fc100d60 2024-02-19T19:01:06.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:06 smithi088 ceph-mon[29272]: pgmap v12: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:06.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.854+0000 7f5201596700 1 -- 172.21.15.88:0/2875959325 --> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f51fc061190 con 0x7f51e406cf10 2024-02-19T19:01:06.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.866+0000 7f51e3fff700 1 -- 172.21.15.88:0/2875959325 <== mgr.24359 v2:172.21.15.132:6828/3141443905 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f51fc061190 con 0x7f51e406cf10 2024-02-19T19:01:06.885 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:01:06.885 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (6m) 9s ago 8m 24.7M - 0.25.0 c8568f914cd2 d6dd804f52d8 2024-02-19T19:01:06.885 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (9m) 9s ago 9m 7596k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:01:06.885 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (7m) 11s ago 7m 7499k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:01:06.885 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (9m) 9s ago 9m 6660k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (7m) 11s ago 7m 6656k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (6m) 9s ago 8m 86.8M - 9.4.7 954c08fa6188 54907dc6edb4 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (2m) 9s ago 2m 13.0M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (2m) 9s ago 2m 25.7M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (2m) 11s ago 2m 20.6M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (2m) 11s ago 2m 16.7M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:9283,8765,8443 running (10m) 9s ago 10m 374M - 18.2.0 dc2bc1663786 5b5e2fca6e0a 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (29s) 11s ago 7m 493M - 19.0.0-1450-gf78a58c0 f081f847070a 9bc43e19d9ce 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (10m) 9s ago 10m 51.3M 2048M 18.2.0 dc2bc1663786 3ea5108455bc 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (6m) 11s ago 6m 42.8M 2048M 18.2.0 dc2bc1663786 ba1c249c6017 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (9m) 9s ago 9m 13.8M - 1.5.0 0da6a335fe13 7cebce4a6d13 2024-02-19T19:01:06.886 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (7m) 11s ago 7m 17.0M - 1.5.0 0da6a335fe13 f93d4349304d 2024-02-19T19:01:06.887 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (6m) 9s ago 6m 50.9M 1216M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:01:06.887 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (5m) 9s ago 5m 48.6M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:01:06.887 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (5m) 9s ago 5m 48.4M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:01:06.887 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (4m) 11s ago 4m 49.7M 4970M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:01:06.887 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (4m) 11s ago 4m 48.4M 4970M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:01:06.887 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (3m) 11s ago 3m 46.3M 4970M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:01:06.887 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (6m) 9s ago 7m 66.9M - 2.43.0 a07b618ecd1d 17518e93393b 2024-02-19T19:01:06.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.868+0000 7f5201596700 1 -- 172.21.15.88:0/2875959325 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f51e406cf10 msgr2=0x7f51e406f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.868+0000 7f5201596700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f51e406cf10 0x7f51e406f3c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f51ec005b40 tx=0x7f51ec005a90 comp rx=0 tx=0).stop 2024-02-19T19:01:06.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.868+0000 7f5201596700 1 -- 172.21.15.88:0/2875959325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51fc100d60 msgr2=0x7f51fc195ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.868+0000 7f5201596700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51fc100d60 0x7f51fc195ae0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f51f000d900 tx=0x7f51f000dcc0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.868+0000 7f5201596700 1 -- 172.21.15.88:0/2875959325 shutdown_connections 2024-02-19T19:01:06.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.868+0000 7f5201596700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f51e406cf10 0x7f51e406f3c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.868+0000 7f5201596700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51fc100d60 0x7f51fc195ae0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.868+0000 7f5201596700 1 --2- 172.21.15.88:0/2875959325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f51fc103680 0x7f51fc196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.868+0000 7f5201596700 1 -- 172.21.15.88:0/2875959325 >> 172.21.15.88:0/2875959325 conn(0x7f51fc0fa780 msgr2=0x7f51fc0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:06.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.869+0000 7f5201596700 1 -- 172.21.15.88:0/2875959325 shutdown_connections 2024-02-19T19:01:06.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.869+0000 7f5201596700 1 -- 172.21.15.88:0/2875959325 wait complete. 2024-02-19T19:01:06.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.971+0000 7ffa76aa8700 1 -- 172.21.15.88:0/1196781902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa701003d0 msgr2=0x7ffa70100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.971+0000 7ffa76aa8700 1 --2- 172.21.15.88:0/1196781902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa701003d0 0x7ffa70100840 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000b0a0 tx=0x7ffa6000b3b0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.971+0000 7ffa76aa8700 1 -- 172.21.15.88:0/1196781902 shutdown_connections 2024-02-19T19:01:06.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.971+0000 7ffa76aa8700 1 --2- 172.21.15.88:0/1196781902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa701003d0 0x7ffa70100840 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.971+0000 7ffa76aa8700 1 --2- 172.21.15.88:0/1196781902 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa700ff130 0x7ffa700ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.971+0000 7ffa76aa8700 1 -- 172.21.15.88:0/1196781902 >> 172.21.15.88:0/1196781902 conn(0x7ffa700fa780 msgr2=0x7ffa700fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.971+0000 7ffa76aa8700 1 -- 172.21.15.88:0/1196781902 shutdown_connections 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.971+0000 7ffa76aa8700 1 -- 172.21.15.88:0/1196781902 wait complete. 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.971+0000 7ffa76aa8700 1 Processor -- start 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.971+0000 7ffa76aa8700 1 -- start start 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa76aa8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa700ff130 0x7ffa70197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa76aa8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa701003d0 0x7ffa701982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa76aa8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa70198900 con 0x7ffa701003d0 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa76aa8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa70198a40 con 0x7ffa700ff130 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa6ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa701003d0 0x7ffa701982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa6ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa701003d0 0x7ffa701982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57310/0 (socket says 172.21.15.88:57310) 2024-02-19T19:01:06.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa6ffff700 1 -- 172.21.15.88:0/2987318804 learned_addr learned my addr 172.21.15.88:0/2987318804 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:01:06.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa6ffff700 1 -- 172.21.15.88:0/2987318804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa700ff130 msgr2=0x7ffa70197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:06.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa6ffff700 1 --2- 172.21.15.88:0/2987318804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa700ff130 0x7ffa70197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:06.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa6ffff700 1 -- 172.21.15.88:0/2987318804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa60009d00 con 0x7ffa701003d0 2024-02-19T19:01:06.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa6ffff700 1 --2- 172.21.15.88:0/2987318804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa701003d0 0x7ffa701982e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffa600077b0 tx=0x7ffa60003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:06.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa6dffb700 1 -- 172.21.15.88:0/2987318804 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa6000e040 con 0x7ffa701003d0 2024-02-19T19:01:06.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa76aa8700 1 -- 172.21.15.88:0/2987318804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa7019d490 con 0x7ffa701003d0 2024-02-19T19:01:06.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa6dffb700 1 -- 172.21.15.88:0/2987318804 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffa6000ce60 con 0x7ffa701003d0 2024-02-19T19:01:06.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.972+0000 7ffa6dffb700 1 -- 172.21.15.88:0/2987318804 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa6001daf0 con 0x7ffa701003d0 2024-02-19T19:01:06.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.973+0000 7ffa76aa8700 1 -- 172.21.15.88:0/2987318804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa7019d980 con 0x7ffa701003d0 2024-02-19T19:01:06.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.973+0000 7ffa76aa8700 1 -- 172.21.15.88:0/2987318804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa70106b70 con 0x7ffa701003d0 2024-02-19T19:01:06.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.974+0000 7ffa6dffb700 1 -- 172.21.15.88:0/2987318804 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7ffa60004250 con 0x7ffa701003d0 2024-02-19T19:01:06.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.974+0000 7ffa6dffb700 1 --2- 172.21.15.88:0/2987318804 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7ffa5c06cef0 0x7ffa5c06f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:06.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.974+0000 7ffa6dffb700 1 -- 172.21.15.88:0/2987318804 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ffa6008e180 con 0x7ffa701003d0 2024-02-19T19:01:06.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.974+0000 7ffa74844700 1 --2- 172.21.15.88:0/2987318804 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7ffa5c06cef0 0x7ffa5c06f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:06.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.975+0000 7ffa74844700 1 --2- 172.21.15.88:0/2987318804 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7ffa5c06cef0 0x7ffa5c06f3a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffa680097b0 tx=0x7ffa68006d20 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:06.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:06.979+0000 7ffa6dffb700 1 -- 172.21.15.88:0/2987318804 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7ffa6005b8c0 con 0x7ffa701003d0 2024-02-19T19:01:07.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:06 smithi132 ceph-mon[37969]: pgmap v12: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:07.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.257+0000 7ffa76aa8700 1 -- 172.21.15.88:0/2987318804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffa70106d30 con 0x7ffa701003d0 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.258+0000 7ffa6dffb700 1 -- 172.21.15.88:0/2987318804 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7ffa6005b450 con 0x7ffa701003d0 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:01:07.277 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:01:07.278 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:01:07.278 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:01:07.278 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:01:07.278 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-02-19T19:01:07.278 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-19T19:01:07.278 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:01:07.278 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:01:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.260+0000 7ffa76aa8700 1 -- 172.21.15.88:0/2987318804 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7ffa5c06cef0 msgr2=0x7ffa5c06f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.260+0000 7ffa76aa8700 1 --2- 172.21.15.88:0/2987318804 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7ffa5c06cef0 0x7ffa5c06f3a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ffa680097b0 tx=0x7ffa68006d20 comp rx=0 tx=0).stop 2024-02-19T19:01:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.260+0000 7ffa76aa8700 1 -- 172.21.15.88:0/2987318804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa701003d0 msgr2=0x7ffa701982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.260+0000 7ffa76aa8700 1 --2- 172.21.15.88:0/2987318804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa701003d0 0x7ffa701982e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ffa600077b0 tx=0x7ffa60003ce0 comp rx=0 tx=0).stop 2024-02-19T19:01:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.260+0000 7ffa76aa8700 1 -- 172.21.15.88:0/2987318804 shutdown_connections 2024-02-19T19:01:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.260+0000 7ffa76aa8700 1 --2- 172.21.15.88:0/2987318804 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7ffa5c06cef0 0x7ffa5c06f3a0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.260+0000 7ffa76aa8700 1 --2- 172.21.15.88:0/2987318804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffa700ff130 0x7ffa70197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.260+0000 7ffa76aa8700 1 --2- 172.21.15.88:0/2987318804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa701003d0 0x7ffa701982e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.260+0000 7ffa76aa8700 1 -- 172.21.15.88:0/2987318804 >> 172.21.15.88:0/2987318804 conn(0x7ffa700fa780 msgr2=0x7ffa70103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.260+0000 7ffa76aa8700 1 -- 172.21.15.88:0/2987318804 shutdown_connections 2024-02-19T19:01:07.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.260+0000 7ffa76aa8700 1 -- 172.21.15.88:0/2987318804 wait complete. 2024-02-19T19:01:07.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.365+0000 7f78a13a1700 1 -- 172.21.15.88:0/2197049190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f789c100240 msgr2=0x7f789c100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:07.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.365+0000 7f78a13a1700 1 --2- 172.21.15.88:0/2197049190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f789c100240 0x7f789c100650 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f788c009910 tx=0x7f788c009c20 comp rx=0 tx=0).stop 2024-02-19T19:01:07.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.365+0000 7f78a13a1700 1 -- 172.21.15.88:0/2197049190 shutdown_connections 2024-02-19T19:01:07.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.365+0000 7f78a13a1700 1 --2- 172.21.15.88:0/2197049190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f789c1014e0 0x7f789c101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:07.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.365+0000 7f78a13a1700 1 --2- 172.21.15.88:0/2197049190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f789c100240 0x7f789c100650 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:07.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.365+0000 7f78a13a1700 1 -- 172.21.15.88:0/2197049190 >> 172.21.15.88:0/2197049190 conn(0x7f789c0fb890 msgr2=0x7f789c0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:07.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.366+0000 7f78a13a1700 1 -- 172.21.15.88:0/2197049190 shutdown_connections 2024-02-19T19:01:07.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.366+0000 7f78a13a1700 1 -- 172.21.15.88:0/2197049190 wait complete. 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.366+0000 7f78a13a1700 1 Processor -- start 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.366+0000 7f78a13a1700 1 -- start start 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.366+0000 7f78a13a1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f789c100240 0x7f789c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.366+0000 7f78a13a1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f789c1014e0 0x7f789c073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.366+0000 7f78a13a1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c073680 con 0x7f789c1014e0 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.366+0000 7f78a13a1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0737c0 con 0x7f789c100240 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f789a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f789c1014e0 0x7f789c073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f789a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f789c1014e0 0x7f789c073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57320/0 (socket says 172.21.15.88:57320) 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f789a7fc700 1 -- 172.21.15.88:0/1764095128 learned_addr learned my addr 172.21.15.88:0/1764095128 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f789affd700 1 --2- 172.21.15.88:0/1764095128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f789c100240 0x7f789c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f789a7fc700 1 -- 172.21.15.88:0/1764095128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f789c100240 msgr2=0x7f789c074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:07.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f789a7fc700 1 --2- 172.21.15.88:0/1764095128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f789c100240 0x7f789c074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f789a7fc700 1 -- 172.21.15.88:0/1764095128 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c0095f0 con 0x7f789c1014e0 2024-02-19T19:01:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f789a7fc700 1 --2- 172.21.15.88:0/1764095128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f789c1014e0 0x7f789c073140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f789000d8d0 tx=0x7f789000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f7883fff700 1 -- 172.21.15.88:0/1764095128 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78900098a0 con 0x7f789c1014e0 2024-02-19T19:01:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f7883fff700 1 -- 172.21.15.88:0/1764095128 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7890010460 con 0x7f789c1014e0 2024-02-19T19:01:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f78a13a1700 1 -- 172.21.15.88:0/1764095128 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c073aa0 con 0x7f789c1014e0 2024-02-19T19:01:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f7883fff700 1 -- 172.21.15.88:0/1764095128 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f789000f5f0 con 0x7f789c1014e0 2024-02-19T19:01:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.367+0000 7f78a13a1700 1 -- 172.21.15.88:0/1764095128 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789c19daf0 con 0x7f789c1014e0 2024-02-19T19:01:07.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.368+0000 7f78a13a1700 1 -- 172.21.15.88:0/1764095128 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789c108d20 con 0x7f789c1014e0 2024-02-19T19:01:07.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.369+0000 7f7883fff700 1 -- 172.21.15.88:0/1764095128 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) v1 ==== 90734+0+0 (secure 0 0 0) 0x7f7890009a00 con 0x7f789c1014e0 2024-02-19T19:01:07.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.369+0000 7f7883fff700 1 --2- 172.21.15.88:0/1764095128 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f788406cf10 0x7f788406f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:07.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.369+0000 7f7883fff700 1 -- 172.21.15.88:0/1764095128 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f789008c480 con 0x7f789c1014e0 2024-02-19T19:01:07.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.369+0000 7f789affd700 1 --2- 172.21.15.88:0/1764095128 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f788406cf10 0x7f788406f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:07.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.370+0000 7f789affd700 1 --2- 172.21.15.88:0/1764095128 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f788406cf10 0x7f788406f3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f788c006010 tx=0x7f788c00b540 comp rx=0 tx=0).ready entity=mgr.24359 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:07.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.374+0000 7f7883fff700 1 -- 172.21.15.88:0/1764095128 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f7890059c40 con 0x7f789c1014e0 2024-02-19T19:01:07.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.590+0000 7f78a13a1700 1 -- 172.21.15.88:0/1764095128 --> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f789c105e30 con 0x7f788406cf10 2024-02-19T19:01:07.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.591+0000 7f7883fff700 1 -- 172.21.15.88:0/1764095128 <== mgr.24359 v2:172.21.15.132:6828/3141443905 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f789c105e30 con 0x7f788406cf10 2024-02-19T19:01:07.610 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:01:07.610 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:01:07.610 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:01:07.610 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-19T19:01:07.610 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [], 2024-02-19T19:01:07.610 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "1/2 daemons upgraded", 2024-02-19T19:01:07.611 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "", 2024-02-19T19:01:07.611 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:01:07.611 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:01:07.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.593+0000 7f78a13a1700 1 -- 172.21.15.88:0/1764095128 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f788406cf10 msgr2=0x7f788406f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:07.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.593+0000 7f78a13a1700 1 --2- 172.21.15.88:0/1764095128 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f788406cf10 0x7f788406f3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f788c006010 tx=0x7f788c00b540 comp rx=0 tx=0).stop 2024-02-19T19:01:07.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.593+0000 7f78a13a1700 1 -- 172.21.15.88:0/1764095128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f789c1014e0 msgr2=0x7f789c073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:07.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.593+0000 7f78a13a1700 1 --2- 172.21.15.88:0/1764095128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f789c1014e0 0x7f789c073140 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f789000d8d0 tx=0x7f789000dbe0 comp rx=0 tx=0).stop 2024-02-19T19:01:07.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.593+0000 7f78a13a1700 1 -- 172.21.15.88:0/1764095128 shutdown_connections 2024-02-19T19:01:07.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.593+0000 7f78a13a1700 1 --2- 172.21.15.88:0/1764095128 >> [v2:172.21.15.132:6828/3141443905,v1:172.21.15.132:6829/3141443905] conn(0x7f788406cf10 0x7f788406f3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:07.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.593+0000 7f78a13a1700 1 --2- 172.21.15.88:0/1764095128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f789c100240 0x7f789c074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:07.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.593+0000 7f78a13a1700 1 --2- 172.21.15.88:0/1764095128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f789c1014e0 0x7f789c073140 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:07.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.593+0000 7f78a13a1700 1 -- 172.21.15.88:0/1764095128 >> 172.21.15.88:0/1764095128 conn(0x7f789c0fb890 msgr2=0x7f789c104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:07.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.594+0000 7f78a13a1700 1 -- 172.21.15.88:0/1764095128 shutdown_connections 2024-02-19T19:01:07.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:07.594+0000 7f78a13a1700 1 -- 172.21.15.88:0/1764095128 wait complete. 2024-02-19T19:01:07.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:07 smithi132 ceph-mon[37969]: from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:07.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:07 smithi132 ceph-mon[37969]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:07.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:07 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2987318804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:01:07.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:07 smithi088 ceph-mon[29272]: from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:07.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:07 smithi088 ceph-mon[29272]: from='client.14666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:07.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:07 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2987318804' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:01:08.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:08 smithi088 ceph-mon[29272]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:08.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:08 smithi088 ceph-mon[29272]: pgmap v13: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:08.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:08 smithi088 ceph-mon[29272]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:09.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:08 smithi132 ceph-mon[37969]: from='client.14670 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:09.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:08 smithi132 ceph-mon[37969]: pgmap v13: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:09.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:08 smithi132 ceph-mon[37969]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:10.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:10.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:10.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: Adjusting osd_memory_target on smithi132 to 2240M 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: Updating smithi088:/etc/ceph/ceph.conf 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: Updating smithi132:/etc/ceph/ceph.conf 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: pgmap v14: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:01:10.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:09 smithi088 ceph-mon[29272]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:01:10.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:10.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: Adjusting osd_memory_target on smithi132 to 2240M 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: Updating smithi088:/etc/ceph/ceph.conf 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: Updating smithi132:/etc/ceph/ceph.conf 2024-02-19T19:01:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:01:10.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:01:10.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:01:10.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:01:10.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: pgmap v14: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:10.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:01:10.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:09 smithi132 ceph-mon[37969]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:01:11.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:10 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:10 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:10 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:10 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:10 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:10 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:10 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:10 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:10 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:10 smithi088 ceph-mon[29272]: Reconfiguring prometheus.smithi088 (dependencies changed)... 2024-02-19T19:01:11.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:10 smithi088 ceph-mon[29272]: Reconfiguring daemon prometheus.smithi088 on smithi088 2024-02-19T19:01:11.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:10 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:10 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:10 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:10 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:10 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:10 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:10 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:10 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:10 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:11.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:10 smithi132 ceph-mon[37969]: Reconfiguring prometheus.smithi088 (dependencies changed)... 2024-02-19T19:01:11.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:10 smithi132 ceph-mon[37969]: Reconfiguring daemon prometheus.smithi088 on smithi088 2024-02-19T19:01:12.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:11 smithi132 ceph-mon[37969]: pgmap v15: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:12.348 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:11 smithi088 ceph-mon[29272]: pgmap v15: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:14.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:14 smithi088 ceph-mon[29272]: pgmap v16: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:15.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:14 smithi132 ceph-mon[37969]: pgmap v16: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:16.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:16 smithi088 ceph-mon[29272]: pgmap v17: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:16.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:16 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:16.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:16 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:16.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:16 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:01:16.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:16 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:17.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:16 smithi132 ceph-mon[37969]: pgmap v17: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:17.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:16 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:17.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:16 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:17.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:16 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:01:17.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:16 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:17.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:17 smithi088 ceph-mon[29272]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:01:17.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:17 smithi088 ceph-mon[29272]: Upgrade: Need to upgrade myself (mgr.smithi132.psllsa) 2024-02-19T19:01:17.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:17 smithi088 ceph-mon[29272]: Upgrade: Need to upgrade myself (mgr.smithi132.psllsa) 2024-02-19T19:01:18.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:17 smithi132 ceph-mon[37969]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:01:18.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:17 smithi132 ceph-mon[37969]: Upgrade: Need to upgrade myself (mgr.smithi132.psllsa) 2024-02-19T19:01:18.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:17 smithi132 ceph-mon[37969]: Upgrade: Need to upgrade myself (mgr.smithi132.psllsa) 2024-02-19T19:01:18.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:18 smithi132 ceph-mon[37969]: pgmap v18: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:18.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:18 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:01:18.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:18 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:18.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:18 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.gyviqq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:01:18.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:18 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.gyviqq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:01:18.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:18 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T19:01:18.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:18 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:01:18.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:18 smithi088 ceph-mon[29272]: pgmap v18: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:18.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:18 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:01:18.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:18 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:18.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:18 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.gyviqq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:01:18.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:18 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.gyviqq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:01:18.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:18 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T19:01:18.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:18 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:01:19.594 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:19 smithi132 ceph-mon[37969]: Upgrade: Updating mgr.smithi088.gyviqq 2024-02-19T19:01:19.594 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:19 smithi132 ceph-mon[37969]: Deploying daemon mgr.smithi088.gyviqq on smithi088 2024-02-19T19:01:19.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:19 smithi088 ceph-mon[29272]: Upgrade: Updating mgr.smithi088.gyviqq 2024-02-19T19:01:19.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:19 smithi088 ceph-mon[29272]: Deploying daemon mgr.smithi088.gyviqq on smithi088 2024-02-19T19:01:20.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:20 smithi088 ceph-mon[29272]: pgmap v19: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:21.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:20 smithi132 ceph-mon[37969]: pgmap v19: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:22.975 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:22 smithi088 ceph-mon[29272]: pgmap v20: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:22.976 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:22 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:22.976 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:22 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:22.976 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:22 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:22.976 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:22 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:22.976 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:22 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:23.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:22 smithi132 ceph-mon[37969]: pgmap v20: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:23.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:22 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:23.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:22 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:23.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:22 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:23.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:22 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:23.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:22 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:24.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:24 smithi088 ceph-mon[29272]: pgmap v21: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:25.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:24 smithi132 ceph-mon[37969]: pgmap v21: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:26.674 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:26 smithi088 ceph-mon[29272]: pgmap v22: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:27.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:26 smithi132 ceph-mon[37969]: pgmap v22: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:28.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:28 smithi088 ceph-mon[29272]: pgmap v23: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:28.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:28 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:28.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:28 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:28.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:28 smithi132 ceph-mon[37969]: pgmap v23: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:28.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:28 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:28.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:28 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:29.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:29 smithi088 ceph-mon[29272]: Standby manager daemon smithi088.gyviqq restarted 2024-02-19T19:01:29.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:29 smithi088 ceph-mon[29272]: Standby manager daemon smithi088.gyviqq started 2024-02-19T19:01:29.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:29 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.gyviqq/crt"}]: dispatch 2024-02-19T19:01:29.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:29 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:01:29.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:29 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.gyviqq/key"}]: dispatch 2024-02-19T19:01:29.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:29 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:01:29.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:29 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:29.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:29 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:29.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:29 smithi132 ceph-mon[37969]: Standby manager daemon smithi088.gyviqq restarted 2024-02-19T19:01:29.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:29 smithi132 ceph-mon[37969]: Standby manager daemon smithi088.gyviqq started 2024-02-19T19:01:29.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:29 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.gyviqq/crt"}]: dispatch 2024-02-19T19:01:29.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:29 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:01:29.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:29 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi088.gyviqq/key"}]: dispatch 2024-02-19T19:01:29.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:29 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:01:29.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:29 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:29.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:29 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:30.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:30 smithi132 ceph-mon[37969]: mgrmap e26: smithi132.psllsa(active, since 41s), standbys: smithi088.gyviqq 2024-02-19T19:01:30.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:30 smithi132 ceph-mon[37969]: pgmap v24: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:30.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:30 smithi088 ceph-mon[29272]: mgrmap e26: smithi132.psllsa(active, since 41s), standbys: smithi088.gyviqq 2024-02-19T19:01:30.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:30 smithi088 ceph-mon[29272]: pgmap v24: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:32.847 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: pgmap v25: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:32.847 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:32.847 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mgr fail", "who": "smithi132.psllsa"}]: dispatch 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mgr fail", "who": "smithi132.psllsa"}]: dispatch 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: osdmap e48: 6 total, 6 up, 6 in 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd='[{"prefix": "mgr fail", "who": "smithi132.psllsa"}]': finished 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: mgrmap e27: smithi088.gyviqq(active, starting, since 0.00742467s) 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T19:01:32.848 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi088.gyviqq", "id": "smithi088.gyviqq"}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T19:01:32.849 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: Manager daemon smithi088.gyviqq is now available 2024-02-19T19:01:32.850 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:32.850 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:32.850 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:01:32.850 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/mirror_snapshot_schedule"}]: dispatch 2024-02-19T19:01:32.850 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:32 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/trash_purge_schedule"}]: dispatch 2024-02-19T19:01:32.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: pgmap v25: 65 pgs: 65 active+clean; 591 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:32.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:32.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:32.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.24359 172.21.15.132:0/901117207' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mgr fail", "who": "smithi132.psllsa"}]: dispatch 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "mgr fail", "who": "smithi132.psllsa"}]: dispatch 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: osdmap e48: 6 total, 6 up, 6 in 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.24359 ' entity='mgr.smithi132.psllsa' cmd='[{"prefix": "mgr fail", "who": "smithi132.psllsa"}]': finished 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: mgrmap e27: smithi088.gyviqq(active, starting, since 0.00742467s) 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:01:32.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi088.gyviqq", "id": "smithi088.gyviqq"}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: Manager daemon smithi088.gyviqq is now available 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/mirror_snapshot_schedule"}]: dispatch 2024-02-19T19:01:32.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:32 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/trash_purge_schedule"}]: dispatch 2024-02-19T19:01:34.581 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:34 smithi132 ceph-mon[37969]: [19/Feb/2024:19:01:33] ENGINE Bus STARTING 2024-02-19T19:01:34.582 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:34 smithi132 ceph-mon[37969]: [19/Feb/2024:19:01:33] ENGINE Serving on https://172.21.15.88:7150 2024-02-19T19:01:34.582 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:34 smithi132 ceph-mon[37969]: mgrmap e28: smithi088.gyviqq(active, since 1.0098s) 2024-02-19T19:01:34.582 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:34 smithi132 ceph-mon[37969]: pgmap v3: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:34.582 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:34 smithi132 ceph-mon[37969]: [19/Feb/2024:19:01:33] ENGINE Serving on http://172.21.15.88:8765 2024-02-19T19:01:34.582 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:34 smithi132 ceph-mon[37969]: [19/Feb/2024:19:01:33] ENGINE Bus STARTED 2024-02-19T19:01:34.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:34 smithi088 ceph-mon[29272]: [19/Feb/2024:19:01:33] ENGINE Bus STARTING 2024-02-19T19:01:34.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:34 smithi088 ceph-mon[29272]: [19/Feb/2024:19:01:33] ENGINE Serving on https://172.21.15.88:7150 2024-02-19T19:01:34.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:34 smithi088 ceph-mon[29272]: mgrmap e28: smithi088.gyviqq(active, since 1.0098s) 2024-02-19T19:01:34.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:34 smithi088 ceph-mon[29272]: pgmap v3: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:34.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:34 smithi088 ceph-mon[29272]: [19/Feb/2024:19:01:33] ENGINE Serving on http://172.21.15.88:8765 2024-02-19T19:01:34.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:34 smithi088 ceph-mon[29272]: [19/Feb/2024:19:01:33] ENGINE Bus STARTED 2024-02-19T19:01:35.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:35 smithi132 ceph-mon[37969]: pgmap v4: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:35.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:35 smithi088 ceph-mon[29272]: pgmap v4: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:36.558 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:36 smithi132 ceph-mon[37969]: mgrmap e29: smithi088.gyviqq(active, since 3s) 2024-02-19T19:01:36.558 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:36 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:36.558 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:36 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:36.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:36 smithi088 ceph-mon[29272]: mgrmap e29: smithi088.gyviqq(active, since 3s) 2024-02-19T19:01:36.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:36 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:36.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:36 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:37.526 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:37 smithi132 ceph-mon[37969]: pgmap v5: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:37.526 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:37 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:37.526 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:37 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:37.535 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:37 smithi088 ceph-mon[29272]: pgmap v5: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:37.535 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:37 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:37.535 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:37 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:37.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.702+0000 7fee1ffc4700 1 -- 172.21.15.88:0/2595756061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee18101440 msgr2=0x7fee18101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:37.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.702+0000 7fee1ffc4700 1 --2- 172.21.15.88:0/2595756061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee18101440 0x7fee18101890 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fee08009910 tx=0x7fee08009c20 comp rx=0 tx=0).stop 2024-02-19T19:01:37.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.702+0000 7fee1ffc4700 1 -- 172.21.15.88:0/2595756061 shutdown_connections 2024-02-19T19:01:37.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.702+0000 7fee1ffc4700 1 --2- 172.21.15.88:0/2595756061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee18101440 0x7fee18101890 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:37.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.702+0000 7fee1ffc4700 1 --2- 172.21.15.88:0/2595756061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee18100240 0x7fee18100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:37.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.702+0000 7fee1ffc4700 1 -- 172.21.15.88:0/2595756061 >> 172.21.15.88:0/2595756061 conn(0x7fee180fb7f0 msgr2=0x7fee180fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:37.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.702+0000 7fee1ffc4700 1 -- 172.21.15.88:0/2595756061 shutdown_connections 2024-02-19T19:01:37.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.702+0000 7fee1ffc4700 1 -- 172.21.15.88:0/2595756061 wait complete. 2024-02-19T19:01:37.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.702+0000 7fee1ffc4700 1 Processor -- start 2024-02-19T19:01:37.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.702+0000 7fee1ffc4700 1 -- start start 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1ffc4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee18100240 0x7fee18195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1ffc4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee18101440 0x7fee18195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1ffc4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee181965f0 con 0x7fee18100240 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1ffc4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee18196730 con 0x7fee18101440 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1dd60700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee18100240 0x7fee18195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1dd60700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee18100240 0x7fee18195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58448/0 (socket says 172.21.15.88:58448) 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1dd60700 1 -- 172.21.15.88:0/3221132671 learned_addr learned my addr 172.21.15.88:0/3221132671 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1d55f700 1 --2- 172.21.15.88:0/3221132671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee18101440 0x7fee18195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1dd60700 1 -- 172.21.15.88:0/3221132671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee18101440 msgr2=0x7fee18195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1dd60700 1 --2- 172.21.15.88:0/3221132671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee18101440 0x7fee18195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1dd60700 1 -- 172.21.15.88:0/3221132671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee080095f0 con 0x7fee18100240 2024-02-19T19:01:37.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1dd60700 1 --2- 172.21.15.88:0/3221132671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee18100240 0x7fee18195a90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fee1400d900 tx=0x7fee1400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee0effd700 1 -- 172.21.15.88:0/3221132671 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee1400b920 con 0x7fee18100240 2024-02-19T19:01:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee0effd700 1 -- 172.21.15.88:0/3221132671 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fee14010460 con 0x7fee18100240 2024-02-19T19:01:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee0effd700 1 -- 172.21.15.88:0/3221132671 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee1400f590 con 0x7fee18100240 2024-02-19T19:01:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.703+0000 7fee1ffc4700 1 -- 172.21.15.88:0/3221132671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1819b1e0 con 0x7fee18100240 2024-02-19T19:01:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.704+0000 7fee1ffc4700 1 -- 172.21.15.88:0/3221132671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee1819b730 con 0x7fee18100240 2024-02-19T19:01:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.704+0000 7fee1ffc4700 1 -- 172.21.15.88:0/3221132671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee18108d90 con 0x7fee18100240 2024-02-19T19:01:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.705+0000 7fee0effd700 1 -- 172.21.15.88:0/3221132671 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 29) v1 ==== 46071+0+0 (secure 0 0 0) 0x7fee140105d0 con 0x7fee18100240 2024-02-19T19:01:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.705+0000 7fee0effd700 1 --2- 172.21.15.88:0/3221132671 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fee04038eb0 0x7fee0403b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.705+0000 7fee0effd700 1 -- 172.21.15.88:0/3221132671 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fee1404cc00 con 0x7fee18100240 2024-02-19T19:01:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.705+0000 7fee1d55f700 1 --2- 172.21.15.88:0/3221132671 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fee04038eb0 0x7fee0403b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:37.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.705+0000 7fee1d55f700 1 --2- 172.21.15.88:0/3221132671 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fee04038eb0 0x7fee0403b360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fee0800b5c0 tx=0x7fee08005ee0 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:37.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.714+0000 7fee0effd700 1 -- 172.21.15.88:0/3221132671 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fee1401aaa0 con 0x7fee18100240 2024-02-19T19:01:37.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.949+0000 7fee1ffc4700 1 -- 172.21.15.88:0/3221132671 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fee18105d90 con 0x7fee04038eb0 2024-02-19T19:01:37.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.951+0000 7fee0effd700 1 -- 172.21.15.88:0/3221132671 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fee18105d90 con 0x7fee04038eb0 2024-02-19T19:01:37.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.953+0000 7fee1ffc4700 1 -- 172.21.15.88:0/3221132671 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fee04038eb0 msgr2=0x7fee0403b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.953+0000 7fee1ffc4700 1 --2- 172.21.15.88:0/3221132671 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fee04038eb0 0x7fee0403b360 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fee0800b5c0 tx=0x7fee08005ee0 comp rx=0 tx=0).stop 2024-02-19T19:01:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.953+0000 7fee1ffc4700 1 -- 172.21.15.88:0/3221132671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee18100240 msgr2=0x7fee18195a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.953+0000 7fee1ffc4700 1 --2- 172.21.15.88:0/3221132671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee18100240 0x7fee18195a90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fee1400d900 tx=0x7fee1400dc10 comp rx=0 tx=0).stop 2024-02-19T19:01:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.953+0000 7fee1ffc4700 1 -- 172.21.15.88:0/3221132671 shutdown_connections 2024-02-19T19:01:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.953+0000 7fee1ffc4700 1 --2- 172.21.15.88:0/3221132671 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fee04038eb0 0x7fee0403b360 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.953+0000 7fee1ffc4700 1 --2- 172.21.15.88:0/3221132671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee18100240 0x7fee18195a90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.953+0000 7fee1ffc4700 1 --2- 172.21.15.88:0/3221132671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fee18101440 0x7fee18195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.953+0000 7fee1ffc4700 1 -- 172.21.15.88:0/3221132671 >> 172.21.15.88:0/3221132671 conn(0x7fee180fb7f0 msgr2=0x7fee18104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.954+0000 7fee1ffc4700 1 -- 172.21.15.88:0/3221132671 shutdown_connections 2024-02-19T19:01:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:37.954+0000 7fee1ffc4700 1 -- 172.21.15.88:0/3221132671 wait complete. 2024-02-19T19:01:37.984 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:01:38.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.065+0000 7f44991a6700 1 -- 172.21.15.88:0/740490885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44940fe500 msgr2=0x7f44940fe910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:38.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.065+0000 7f44991a6700 1 --2- 172.21.15.88:0/740490885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44940fe500 0x7f44940fe910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4484009910 tx=0x7f4484009c20 comp rx=0 tx=0).stop 2024-02-19T19:01:38.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.065+0000 7f44991a6700 1 -- 172.21.15.88:0/740490885 shutdown_connections 2024-02-19T19:01:38.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.065+0000 7f44991a6700 1 --2- 172.21.15.88:0/740490885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44940fee50 0x7f4494101580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.065+0000 7f44991a6700 1 --2- 172.21.15.88:0/740490885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44940fe500 0x7f44940fe910 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.065+0000 7f44991a6700 1 -- 172.21.15.88:0/740490885 >> 172.21.15.88:0/740490885 conn(0x7f44940f9f10 msgr2=0x7f44940fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:38.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.065+0000 7f44991a6700 1 -- 172.21.15.88:0/740490885 shutdown_connections 2024-02-19T19:01:38.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.065+0000 7f44991a6700 1 -- 172.21.15.88:0/740490885 wait complete. 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.065+0000 7f44991a6700 1 Processor -- start 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f44991a6700 1 -- start start 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f44991a6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44940fe500 0x7f4494100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f44991a6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44940fee50 0x7f44940ff530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f44991a6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44940ffa70 con 0x7f44940fe500 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f44991a6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44940ffbb0 con 0x7f44940fee50 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f4492d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44940fe500 0x7f4494100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f4492d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44940fe500 0x7f4494100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58468/0 (socket says 172.21.15.88:58468) 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f4492d9d700 1 -- 172.21.15.88:0/497571221 learned_addr learned my addr 172.21.15.88:0/497571221 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f449259c700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44940fee50 0x7f44940ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f449259c700 1 -- 172.21.15.88:0/497571221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44940fe500 msgr2=0x7f4494100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f449259c700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44940fe500 0x7f4494100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f449259c700 1 -- 172.21.15.88:0/497571221 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44840095f0 con 0x7f44940fee50 2024-02-19T19:01:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.066+0000 7f4492d9d700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44940fe500 0x7f4494100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:01:38.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.067+0000 7f449259c700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44940fee50 0x7f44940ff530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44940ff240 tx=0x7f448800ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:38.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.067+0000 7f447bfff700 1 -- 172.21.15.88:0/497571221 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4488004200 con 0x7f44940fee50 2024-02-19T19:01:38.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.067+0000 7f44991a6700 1 -- 172.21.15.88:0/497571221 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44940ffe90 con 0x7f44940fee50 2024-02-19T19:01:38.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.067+0000 7f447bfff700 1 -- 172.21.15.88:0/497571221 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4488004ce0 con 0x7f44940fee50 2024-02-19T19:01:38.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.067+0000 7f447bfff700 1 -- 172.21.15.88:0/497571221 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4488005020 con 0x7f44940fee50 2024-02-19T19:01:38.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.067+0000 7f44991a6700 1 -- 172.21.15.88:0/497571221 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449419dba0 con 0x7f44940fee50 2024-02-19T19:01:38.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.068+0000 7f447bfff700 1 -- 172.21.15.88:0/497571221 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 29) v1 ==== 46071+0+0 (secure 0 0 0) 0x7f4488005290 con 0x7f44940fee50 2024-02-19T19:01:38.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.068+0000 7f44991a6700 1 -- 172.21.15.88:0/497571221 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4494108d70 con 0x7f44940fee50 2024-02-19T19:01:38.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.068+0000 7f447bfff700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f447c038eb0 0x7f447c03b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:38.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.068+0000 7f447bfff700 1 -- 172.21.15.88:0/497571221 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4488014070 con 0x7f44940fee50 2024-02-19T19:01:38.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.068+0000 7f4492d9d700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f447c038eb0 0x7f447c03b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:38.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.069+0000 7f4492d9d700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f447c038eb0 0x7f447c03b360 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4484009fd0 tx=0x7f4484005a90 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:38.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.073+0000 7f447bfff700 1 -- 172.21.15.88:0/497571221 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f4488016c70 con 0x7f44940fee50 2024-02-19T19:01:38.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.301+0000 7f44991a6700 1 -- 172.21.15.88:0/497571221 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f449419e150 con 0x7f447c038eb0 2024-02-19T19:01:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.303+0000 7f447bfff700 1 -- 172.21.15.88:0/497571221 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f449419e150 con 0x7f447c038eb0 2024-02-19T19:01:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.306+0000 7f44991a6700 1 -- 172.21.15.88:0/497571221 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f447c038eb0 msgr2=0x7f447c03b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.306+0000 7f44991a6700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f447c038eb0 0x7f447c03b360 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4484009fd0 tx=0x7f4484005a90 comp rx=0 tx=0).stop 2024-02-19T19:01:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.306+0000 7f44991a6700 1 -- 172.21.15.88:0/497571221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44940fee50 msgr2=0x7f44940ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.306+0000 7f44991a6700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44940fee50 0x7f44940ff530 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f44940ff240 tx=0x7f448800ec20 comp rx=0 tx=0).stop 2024-02-19T19:01:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.306+0000 7f44991a6700 1 -- 172.21.15.88:0/497571221 shutdown_connections 2024-02-19T19:01:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.306+0000 7f44991a6700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f447c038eb0 0x7f447c03b360 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.306+0000 7f44991a6700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44940fe500 0x7f4494100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.306+0000 7f44991a6700 1 --2- 172.21.15.88:0/497571221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44940fee50 0x7f44940ff530 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.306+0000 7f44991a6700 1 -- 172.21.15.88:0/497571221 >> 172.21.15.88:0/497571221 conn(0x7f44940f9f10 msgr2=0x7f44941047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.306+0000 7f44991a6700 1 -- 172.21.15.88:0/497571221 shutdown_connections 2024-02-19T19:01:38.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.306+0000 7f44991a6700 1 -- 172.21.15.88:0/497571221 wait complete. 2024-02-19T19:01:38.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.415+0000 7fa334c74700 1 -- 172.21.15.88:0/21414016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3301003d0 msgr2=0x7fa330100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:38.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.415+0000 7fa334c74700 1 --2- 172.21.15.88:0/21414016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3301003d0 0x7fa330100840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa324009910 tx=0x7fa324009c20 comp rx=0 tx=0).stop 2024-02-19T19:01:38.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.415+0000 7fa334c74700 1 -- 172.21.15.88:0/21414016 shutdown_connections 2024-02-19T19:01:38.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.415+0000 7fa334c74700 1 --2- 172.21.15.88:0/21414016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3301003d0 0x7fa330100840 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.415+0000 7fa334c74700 1 --2- 172.21.15.88:0/21414016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3300ff130 0x7fa3300ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.415+0000 7fa334c74700 1 -- 172.21.15.88:0/21414016 >> 172.21.15.88:0/21414016 conn(0x7fa3300fa780 msgr2=0x7fa3300fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:38.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.415+0000 7fa334c74700 1 -- 172.21.15.88:0/21414016 shutdown_connections 2024-02-19T19:01:38.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.415+0000 7fa334c74700 1 -- 172.21.15.88:0/21414016 wait complete. 2024-02-19T19:01:38.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa334c74700 1 Processor -- start 2024-02-19T19:01:38.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa334c74700 1 -- start start 2024-02-19T19:01:38.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa334c74700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3300ff130 0x7fa3300690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:38.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa334c74700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3301003d0 0x7fa33019a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:38.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa334c74700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33019ad70 con 0x7fa3300ff130 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa334c74700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33019aee0 con 0x7fa3301003d0 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa32e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3300ff130 0x7fa3300690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa32e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3300ff130 0x7fa3300690c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58478/0 (socket says 172.21.15.88:58478) 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa32e59c700 1 -- 172.21.15.88:0/4183210003 learned_addr learned my addr 172.21.15.88:0/4183210003 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa32dd9b700 1 --2- 172.21.15.88:0/4183210003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3301003d0 0x7fa33019a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa32e59c700 1 -- 172.21.15.88:0/4183210003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3301003d0 msgr2=0x7fa33019a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.416+0000 7fa32e59c700 1 --2- 172.21.15.88:0/4183210003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3301003d0 0x7fa33019a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.417+0000 7fa32e59c700 1 -- 172.21.15.88:0/4183210003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3240095f0 con 0x7fa3300ff130 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.417+0000 7fa32e59c700 1 --2- 172.21.15.88:0/4183210003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3300ff130 0x7fa3300690c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa32000e920 tx=0x7fa32000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.417+0000 7fa31f7fe700 1 -- 172.21.15.88:0/4183210003 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa32000c9d0 con 0x7fa3300ff130 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.417+0000 7fa31f7fe700 1 -- 172.21.15.88:0/4183210003 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa320010430 con 0x7fa3300ff130 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.417+0000 7fa334c74700 1 -- 172.21.15.88:0/4183210003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33019b1c0 con 0x7fa3300ff130 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.417+0000 7fa31f7fe700 1 -- 172.21.15.88:0/4183210003 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa320018650 con 0x7fa3300ff130 2024-02-19T19:01:38.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.417+0000 7fa334c74700 1 -- 172.21.15.88:0/4183210003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33019b6e0 con 0x7fa3300ff130 2024-02-19T19:01:38.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.418+0000 7fa334c74700 1 -- 172.21.15.88:0/4183210003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa330107c10 con 0x7fa3300ff130 2024-02-19T19:01:38.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.418+0000 7fa31f7fe700 1 -- 172.21.15.88:0/4183210003 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 29) v1 ==== 46071+0+0 (secure 0 0 0) 0x7fa3200188c0 con 0x7fa3300ff130 2024-02-19T19:01:38.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.418+0000 7fa31f7fe700 1 --2- 172.21.15.88:0/4183210003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa318038eb0 0x7fa31803b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:38.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.418+0000 7fa31f7fe700 1 -- 172.21.15.88:0/4183210003 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa320014070 con 0x7fa3300ff130 2024-02-19T19:01:38.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.418+0000 7fa32dd9b700 1 --2- 172.21.15.88:0/4183210003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa318038eb0 0x7fa31803b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:38.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.419+0000 7fa32dd9b700 1 --2- 172.21.15.88:0/4183210003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa318038eb0 0x7fa31803b360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa324009fd0 tx=0x7fa324005940 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:38.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.423+0000 7fa31f7fe700 1 -- 172.21.15.88:0/4183210003 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fa32001fbe0 con 0x7fa3300ff130 2024-02-19T19:01:38.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.682+0000 7fa334c74700 1 -- 172.21.15.88:0/4183210003 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa330104d20 con 0x7fa318038eb0 2024-02-19T19:01:38.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.693+0000 7fa31f7fe700 1 -- 172.21.15.88:0/4183210003 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa330104d20 con 0x7fa318038eb0 2024-02-19T19:01:38.711 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (7m) 0s ago 9m 24.7M - 0.25.0 c8568f914cd2 d6dd804f52d8 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (9m) 0s ago 9m 7646k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (7m) 2s ago 7m 7554k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (9m) 0s ago 9m 6660k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (7m) 2s ago 7m 6656k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (6m) 0s ago 8m 87.7M - 9.4.7 954c08fa6188 54907dc6edb4 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (3m) 0s ago 3m 13.3M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (2m) 0s ago 2m 26.2M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (2m) 2s ago 2m 21.7M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (2m) 2s ago 2m 17.2M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (17s) 0s ago 10m 472M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (61s) 2s ago 7m 73.6M - 19.0.0-1450-gf78a58c0 f081f847070a 9bc43e19d9ce 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (11m) 0s ago 11m 51.2M 2048M 18.2.0 dc2bc1663786 3ea5108455bc 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (7m) 2s ago 7m 39.3M 2048M 18.2.0 dc2bc1663786 ba1c249c6017 2024-02-19T19:01:38.712 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (9m) 0s ago 9m 13.9M - 1.5.0 0da6a335fe13 7cebce4a6d13 2024-02-19T19:01:38.713 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (7m) 2s ago 7m 16.3M - 1.5.0 0da6a335fe13 f93d4349304d 2024-02-19T19:01:38.713 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (6m) 0s ago 6m 51.4M 1216M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:01:38.713 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (6m) 0s ago 6m 48.9M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:01:38.713 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (5m) 0s ago 5m 48.7M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:01:38.713 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (5m) 2s ago 5m 50.2M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:01:38.713 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (4m) 2s ago 4m 48.9M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:01:38.713 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (4m) 2s ago 4m 46.8M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:01:38.713 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (23s) 0s ago 8m 46.5M - 2.43.0 a07b618ecd1d bf82a56a1a55 2024-02-19T19:01:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.695+0000 7fa334c74700 1 -- 172.21.15.88:0/4183210003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa318038eb0 msgr2=0x7fa31803b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.695+0000 7fa334c74700 1 --2- 172.21.15.88:0/4183210003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa318038eb0 0x7fa31803b360 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa324009fd0 tx=0x7fa324005940 comp rx=0 tx=0).stop 2024-02-19T19:01:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.695+0000 7fa334c74700 1 -- 172.21.15.88:0/4183210003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3300ff130 msgr2=0x7fa3300690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.695+0000 7fa334c74700 1 --2- 172.21.15.88:0/4183210003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3300ff130 0x7fa3300690c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa32000e920 tx=0x7fa32000ec30 comp rx=0 tx=0).stop 2024-02-19T19:01:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.695+0000 7fa334c74700 1 -- 172.21.15.88:0/4183210003 shutdown_connections 2024-02-19T19:01:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.695+0000 7fa334c74700 1 --2- 172.21.15.88:0/4183210003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa318038eb0 0x7fa31803b360 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.695+0000 7fa334c74700 1 --2- 172.21.15.88:0/4183210003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3300ff130 0x7fa3300690c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.695+0000 7fa334c74700 1 --2- 172.21.15.88:0/4183210003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3301003d0 0x7fa33019a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.695+0000 7fa334c74700 1 -- 172.21.15.88:0/4183210003 >> 172.21.15.88:0/4183210003 conn(0x7fa3300fa780 msgr2=0x7fa330103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.695+0000 7fa334c74700 1 -- 172.21.15.88:0/4183210003 shutdown_connections 2024-02-19T19:01:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.695+0000 7fa334c74700 1 -- 172.21.15.88:0/4183210003 wait complete. 2024-02-19T19:01:38.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.802+0000 7fd115426700 1 -- 172.21.15.88:0/1289111179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1100ff130 msgr2=0x7fd1100ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:38.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.802+0000 7fd115426700 1 --2- 172.21.15.88:0/1289111179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1100ff130 0x7fd1100ff540 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd100009910 tx=0x7fd100009c20 comp rx=0 tx=0).stop 2024-02-19T19:01:38.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.802+0000 7fd115426700 1 -- 172.21.15.88:0/1289111179 shutdown_connections 2024-02-19T19:01:38.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.802+0000 7fd115426700 1 --2- 172.21.15.88:0/1289111179 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1101003d0 0x7fd110100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.802+0000 7fd115426700 1 --2- 172.21.15.88:0/1289111179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1100ff130 0x7fd1100ff540 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.802+0000 7fd115426700 1 -- 172.21.15.88:0/1289111179 >> 172.21.15.88:0/1289111179 conn(0x7fd1100fa780 msgr2=0x7fd1100fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:38.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.802+0000 7fd115426700 1 -- 172.21.15.88:0/1289111179 shutdown_connections 2024-02-19T19:01:38.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.802+0000 7fd115426700 1 -- 172.21.15.88:0/1289111179 wait complete. 2024-02-19T19:01:38.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd115426700 1 Processor -- start 2024-02-19T19:01:38.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd115426700 1 -- start start 2024-02-19T19:01:38.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd115426700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1100ff130 0x7fd1101949b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:38.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd115426700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1101003d0 0x7fd110194ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:38.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd115426700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd110195510 con 0x7fd1101003d0 2024-02-19T19:01:38.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd115426700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd110195650 con 0x7fd1100ff130 2024-02-19T19:01:38.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd10e7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1101003d0 0x7fd110194ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:38.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd10e7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1101003d0 0x7fd110194ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58510/0 (socket says 172.21.15.88:58510) 2024-02-19T19:01:38.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd10e7fc700 1 -- 172.21.15.88:0/2619677121 learned_addr learned my addr 172.21.15.88:0/2619677121 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:01:38.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd10effd700 1 --2- 172.21.15.88:0/2619677121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1100ff130 0x7fd1101949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:38.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd10e7fc700 1 -- 172.21.15.88:0/2619677121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1100ff130 msgr2=0x7fd1101949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:38.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.803+0000 7fd10e7fc700 1 --2- 172.21.15.88:0/2619677121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1100ff130 0x7fd1101949b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:38.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.804+0000 7fd10e7fc700 1 -- 172.21.15.88:0/2619677121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1000095f0 con 0x7fd1101003d0 2024-02-19T19:01:38.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.804+0000 7fd10e7fc700 1 --2- 172.21.15.88:0/2619677121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1101003d0 0x7fd110194ef0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd10400d8d0 tx=0x7fd10400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:38.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.804+0000 7fd0f7fff700 1 -- 172.21.15.88:0/2619677121 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1040098a0 con 0x7fd1101003d0 2024-02-19T19:01:38.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.804+0000 7fd115426700 1 -- 172.21.15.88:0/2619677121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11019a100 con 0x7fd1101003d0 2024-02-19T19:01:38.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.804+0000 7fd0f7fff700 1 -- 172.21.15.88:0/2619677121 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd104010460 con 0x7fd1101003d0 2024-02-19T19:01:38.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.804+0000 7fd0f7fff700 1 -- 172.21.15.88:0/2619677121 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd10400f5f0 con 0x7fd1101003d0 2024-02-19T19:01:38.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.804+0000 7fd115426700 1 -- 172.21.15.88:0/2619677121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11019a650 con 0x7fd1101003d0 2024-02-19T19:01:38.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.805+0000 7fd0f7fff700 1 -- 172.21.15.88:0/2619677121 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 29) v1 ==== 46071+0+0 (secure 0 0 0) 0x7fd10400f750 con 0x7fd1101003d0 2024-02-19T19:01:38.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.805+0000 7fd115426700 1 -- 172.21.15.88:0/2619677121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd110107c10 con 0x7fd1101003d0 2024-02-19T19:01:38.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.805+0000 7fd0f7fff700 1 --2- 172.21.15.88:0/2619677121 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd0f8038eb0 0x7fd0f803b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:38.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.805+0000 7fd0f7fff700 1 -- 172.21.15.88:0/2619677121 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd104010ad0 con 0x7fd1101003d0 2024-02-19T19:01:38.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.805+0000 7fd10effd700 1 --2- 172.21.15.88:0/2619677121 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd0f8038eb0 0x7fd0f803b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:38.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.806+0000 7fd10effd700 1 --2- 172.21.15.88:0/2619677121 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd0f8038eb0 0x7fd0f803b360 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd100006010 tx=0x7fd100005a90 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:38.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:38.810+0000 7fd0f7fff700 1 -- 172.21.15.88:0/2619677121 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fd104022680 con 0x7fd1101003d0 2024-02-19T19:01:39.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.115+0000 7fd115426700 1 -- 172.21.15.88:0/2619677121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd110107e30 con 0x7fd1101003d0 2024-02-19T19:01:39.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.115+0000 7fd0f7fff700 1 -- 172.21.15.88:0/2619677121 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fd104022680 con 0x7fd1101003d0 2024-02-19T19:01:39.134 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:01:39.134 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:01:39.135 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-19T19:01:39.136 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-19T19:01:39.136 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:01:39.136 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:01:39.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.117+0000 7fd115426700 1 -- 172.21.15.88:0/2619677121 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd0f8038eb0 msgr2=0x7fd0f803b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.117+0000 7fd115426700 1 --2- 172.21.15.88:0/2619677121 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd0f8038eb0 0x7fd0f803b360 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd100006010 tx=0x7fd100005a90 comp rx=0 tx=0).stop 2024-02-19T19:01:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.117+0000 7fd115426700 1 -- 172.21.15.88:0/2619677121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1101003d0 msgr2=0x7fd110194ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.117+0000 7fd115426700 1 --2- 172.21.15.88:0/2619677121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1101003d0 0x7fd110194ef0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd10400d8d0 tx=0x7fd10400dbe0 comp rx=0 tx=0).stop 2024-02-19T19:01:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.117+0000 7fd115426700 1 -- 172.21.15.88:0/2619677121 shutdown_connections 2024-02-19T19:01:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.117+0000 7fd115426700 1 --2- 172.21.15.88:0/2619677121 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd0f8038eb0 0x7fd0f803b360 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.117+0000 7fd115426700 1 --2- 172.21.15.88:0/2619677121 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1100ff130 0x7fd1101949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.117+0000 7fd115426700 1 --2- 172.21.15.88:0/2619677121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1101003d0 0x7fd110194ef0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.117+0000 7fd115426700 1 -- 172.21.15.88:0/2619677121 >> 172.21.15.88:0/2619677121 conn(0x7fd1100fa780 msgr2=0x7fd110103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.118+0000 7fd115426700 1 -- 172.21.15.88:0/2619677121 shutdown_connections 2024-02-19T19:01:39.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.118+0000 7fd115426700 1 -- 172.21.15.88:0/2619677121 wait complete. 2024-02-19T19:01:39.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.232+0000 7f44a8e5d700 1 -- 172.21.15.88:0/2262994510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a40fee50 msgr2=0x7f44a4101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:39.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.232+0000 7f44a8e5d700 1 --2- 172.21.15.88:0/2262994510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a40fee50 0x7f44a4101580 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f4498009910 tx=0x7f4498009c20 comp rx=0 tx=0).stop 2024-02-19T19:01:39.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.232+0000 7f44a8e5d700 1 -- 172.21.15.88:0/2262994510 shutdown_connections 2024-02-19T19:01:39.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.232+0000 7f44a8e5d700 1 --2- 172.21.15.88:0/2262994510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a40fee50 0x7f44a4101580 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:39.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.232+0000 7f44a8e5d700 1 --2- 172.21.15.88:0/2262994510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a40fe500 0x7f44a40fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:39.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.232+0000 7f44a8e5d700 1 -- 172.21.15.88:0/2262994510 >> 172.21.15.88:0/2262994510 conn(0x7f44a40f9f10 msgr2=0x7f44a40fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:39.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.232+0000 7f44a8e5d700 1 -- 172.21.15.88:0/2262994510 shutdown_connections 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.232+0000 7f44a8e5d700 1 -- 172.21.15.88:0/2262994510 wait complete. 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.233+0000 7f44a8e5d700 1 Processor -- start 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.233+0000 7f44a8e5d700 1 -- start start 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.233+0000 7f44a8e5d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a40fe500 0x7f44a4195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.233+0000 7f44a8e5d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a40fee50 0x7f44a4196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.233+0000 7f44a8e5d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a4196690 con 0x7f44a40fee50 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.233+0000 7f44a8e5d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a41967d0 con 0x7f44a40fe500 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.233+0000 7f44a1d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a40fee50 0x7f44a4196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.233+0000 7f44a1d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a40fee50 0x7f44a4196070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58532/0 (socket says 172.21.15.88:58532) 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.233+0000 7f44a1d9b700 1 -- 172.21.15.88:0/3808141003 learned_addr learned my addr 172.21.15.88:0/3808141003 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.233+0000 7f44a259c700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a40fe500 0x7f44a4195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.234+0000 7f44a1d9b700 1 -- 172.21.15.88:0/3808141003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a40fe500 msgr2=0x7f44a4195b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:39.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.234+0000 7f44a1d9b700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a40fe500 0x7f44a4195b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:39.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.234+0000 7f44a1d9b700 1 -- 172.21.15.88:0/3808141003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44980095f0 con 0x7f44a40fee50 2024-02-19T19:01:39.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.234+0000 7f44a259c700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a40fe500 0x7f44a4195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:01:39.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.234+0000 7f44a1d9b700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a40fee50 0x7f44a4196070 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4498005850 tx=0x7f449800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:39.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.234+0000 7f44937fe700 1 -- 172.21.15.88:0/3808141003 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f449801d070 con 0x7f44a40fee50 2024-02-19T19:01:39.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.234+0000 7f44937fe700 1 -- 172.21.15.88:0/3808141003 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f449800f460 con 0x7f44a40fee50 2024-02-19T19:01:39.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.234+0000 7f44937fe700 1 -- 172.21.15.88:0/3808141003 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4498021640 con 0x7f44a40fee50 2024-02-19T19:01:39.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.234+0000 7f44a8e5d700 1 -- 172.21.15.88:0/3808141003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a419b220 con 0x7f44a40fee50 2024-02-19T19:01:39.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.234+0000 7f44a8e5d700 1 -- 172.21.15.88:0/3808141003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a419b6c0 con 0x7f44a40fee50 2024-02-19T19:01:39.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.235+0000 7f44a8e5d700 1 -- 172.21.15.88:0/3808141003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a4108d10 con 0x7f44a40fee50 2024-02-19T19:01:39.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.235+0000 7f44937fe700 1 -- 172.21.15.88:0/3808141003 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 29) v1 ==== 46071+0+0 (secure 0 0 0) 0x7f449802b440 con 0x7f44a40fee50 2024-02-19T19:01:39.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.235+0000 7f44937fe700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f448c038eb0 0x7f448c03b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:01:39.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.235+0000 7f44937fe700 1 -- 172.21.15.88:0/3808141003 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f449804e290 con 0x7f44a40fee50 2024-02-19T19:01:39.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.235+0000 7f44a259c700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f448c038eb0 0x7f448c03b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:01:39.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.236+0000 7f44a259c700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f448c038eb0 0x7f448c03b360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4494005950 tx=0x7f449400b410 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:01:39.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.242+0000 7f44937fe700 1 -- 172.21.15.88:0/3808141003 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f4498029930 con 0x7f44a40fee50 2024-02-19T19:01:39.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:39 smithi088 ceph-mon[29272]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:39.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:39.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:39.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:39 smithi088 ceph-mon[29272]: pgmap v6: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:39.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:39 smithi088 ceph-mon[29272]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:39.492 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:39 smithi132 ceph-mon[37969]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:39.492 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:39.492 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:39.492 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:39 smithi132 ceph-mon[37969]: pgmap v6: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:39.492 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:39 smithi132 ceph-mon[37969]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:39.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.474+0000 7f44a8e5d700 1 -- 172.21.15.88:0/3808141003 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f44a419be10 con 0x7f448c038eb0 2024-02-19T19:01:39.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.475+0000 7f44937fe700 1 -- 172.21.15.88:0/3808141003 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f44a419be10 con 0x7f448c038eb0 2024-02-19T19:01:39.494 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:01:39.494 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:01:39.494 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:01:39.495 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-19T19:01:39.495 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:01:39.495 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:01:39.495 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:01:39.495 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "2/2 daemons upgraded", 2024-02-19T19:01:39.495 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "", 2024-02-19T19:01:39.495 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:01:39.495 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:01:39.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.478+0000 7f44a8e5d700 1 -- 172.21.15.88:0/3808141003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f448c038eb0 msgr2=0x7f448c03b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:39.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.478+0000 7f44a8e5d700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f448c038eb0 0x7f448c03b360 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4494005950 tx=0x7f449400b410 comp rx=0 tx=0).stop 2024-02-19T19:01:39.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.478+0000 7f44a8e5d700 1 -- 172.21.15.88:0/3808141003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a40fee50 msgr2=0x7f44a4196070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:01:39.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.478+0000 7f44a8e5d700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a40fee50 0x7f44a4196070 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f4498005850 tx=0x7f449800bd70 comp rx=0 tx=0).stop 2024-02-19T19:01:39.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.478+0000 7f44a8e5d700 1 -- 172.21.15.88:0/3808141003 shutdown_connections 2024-02-19T19:01:39.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.478+0000 7f44a8e5d700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f448c038eb0 0x7f448c03b360 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:39.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.478+0000 7f44a8e5d700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44a40fe500 0x7f44a4195b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:39.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.478+0000 7f44a8e5d700 1 --2- 172.21.15.88:0/3808141003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44a40fee50 0x7f44a4196070 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:01:39.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.478+0000 7f44a8e5d700 1 -- 172.21.15.88:0/3808141003 >> 172.21.15.88:0/3808141003 conn(0x7f44a40f9f10 msgr2=0x7f44a41047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:01:39.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.478+0000 7f44a8e5d700 1 -- 172.21.15.88:0/3808141003 shutdown_connections 2024-02-19T19:01:39.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:01:39.478+0000 7f44a8e5d700 1 -- 172.21.15.88:0/3808141003 wait complete. 2024-02-19T19:01:40.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:40 smithi088 ceph-mon[29272]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:40.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:40 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2619677121' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:01:40.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:40 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:40.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:40 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:40.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:40 smithi088 ceph-mon[29272]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:40.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:40 smithi088 ceph-mon[29272]: Standby manager daemon smithi132.psllsa started 2024-02-19T19:01:40.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:40 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/1163657488' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/crt"}]: dispatch 2024-02-19T19:01:40.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:40 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/1163657488' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:01:40.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:40 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/1163657488' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/key"}]: dispatch 2024-02-19T19:01:40.135 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:40 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/1163657488' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:01:40.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:40 smithi132 ceph-mon[37969]: from='client.14720 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:40.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:40 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2619677121' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:01:40.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:40 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:40.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:40 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:40.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:40 smithi132 ceph-mon[37969]: from='client.14728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:01:40.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:40 smithi132 ceph-mon[37969]: Standby manager daemon smithi132.psllsa started 2024-02-19T19:01:40.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:40 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/1163657488' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/crt"}]: dispatch 2024-02-19T19:01:40.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:40 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/1163657488' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:01:40.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:40 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/1163657488' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/key"}]: dispatch 2024-02-19T19:01:40.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:40 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/1163657488' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:01:41.500 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:41 smithi088 ceph-mon[29272]: mgrmap e30: smithi088.gyviqq(active, since 7s), standbys: smithi132.psllsa 2024-02-19T19:01:41.500 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi132.psllsa", "id": "smithi132.psllsa"}]: dispatch 2024-02-19T19:01:41.500 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:41 smithi088 ceph-mon[29272]: pgmap v7: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-19T19:01:41.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:41 smithi132 ceph-mon[37969]: mgrmap e30: smithi088.gyviqq(active, since 7s), standbys: smithi132.psllsa 2024-02-19T19:01:41.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi132.psllsa", "id": "smithi132.psllsa"}]: dispatch 2024-02-19T19:01:41.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:41 smithi132 ceph-mon[37969]: pgmap v7: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-19T19:01:43.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:43 smithi088 ceph-mon[29272]: pgmap v8: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-19T19:01:43.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:43 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:43.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:43 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:43.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:43 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:01:43.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:43 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:01:43.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:43 smithi132 ceph-mon[37969]: pgmap v8: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-19T19:01:43.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:43 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:43.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:43 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:43.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:43 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:01:43.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:43 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:01:44.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: Updating smithi088:/etc/ceph/ceph.conf 2024-02-19T19:01:44.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: Updating smithi132:/etc/ceph/ceph.conf 2024-02-19T19:01:44.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:01:44.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:01:44.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:01:44.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:01:44.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:01:44.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:01:44.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:44.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:44.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:44.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:44.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:44 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:44.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: Updating smithi088:/etc/ceph/ceph.conf 2024-02-19T19:01:44.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: Updating smithi132:/etc/ceph/ceph.conf 2024-02-19T19:01:44.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:01:44.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:01:44.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:01:44.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:01:44.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:01:44.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:01:44.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:44.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:44.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:44.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:44.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:44 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:45.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:45 smithi088 ceph-mon[29272]: Reconfiguring prometheus.smithi088 (dependencies changed)... 2024-02-19T19:01:45.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:45 smithi088 ceph-mon[29272]: Reconfiguring daemon prometheus.smithi088 on smithi088 2024-02-19T19:01:45.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:45 smithi088 ceph-mon[29272]: pgmap v9: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-19T19:01:45.496 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:45 smithi132 ceph-mon[37969]: Reconfiguring prometheus.smithi088 (dependencies changed)... 2024-02-19T19:01:45.496 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:45 smithi132 ceph-mon[37969]: Reconfiguring daemon prometheus.smithi088 on smithi088 2024-02-19T19:01:45.496 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:45 smithi132 ceph-mon[37969]: pgmap v9: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-19T19:01:47.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:47 smithi088 ceph-mon[29272]: pgmap v10: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:47.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:47 smithi132 ceph-mon[37969]: pgmap v10: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:48.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:48 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:01:48.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:48 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:01:49.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:49 smithi088 ceph-mon[29272]: pgmap v11: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:49.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:49 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:49.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:49 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:49.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:49 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:01:49.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:49 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:49.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:49 smithi132 ceph-mon[37969]: pgmap v11: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:49.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:49 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:49.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:49 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:49.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:49 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:01:49.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:49 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:50.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:50 smithi088 ceph-mon[29272]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:01:50.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:50 smithi132 ceph-mon[37969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:01:51.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:51 smithi132 ceph-mon[37969]: pgmap v12: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:51.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:51 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:01:51.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:51 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T19:01:51.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:51 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:01:51.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:51 smithi088 ceph-mon[29272]: pgmap v12: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-19T19:01:51.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:51 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:01:51.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:51 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T19:01:51.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:51 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:01:52.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:52 smithi088 ceph-mon[29272]: Upgrade: Updating mgr.smithi132.psllsa 2024-02-19T19:01:52.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:52 smithi088 ceph-mon[29272]: Deploying daemon mgr.smithi132.psllsa on smithi132 2024-02-19T19:01:52.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:52 smithi132 ceph-mon[37969]: Upgrade: Updating mgr.smithi132.psllsa 2024-02-19T19:01:52.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:52 smithi132 ceph-mon[37969]: Deploying daemon mgr.smithi132.psllsa on smithi132 2024-02-19T19:01:53.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:53 smithi088 ceph-mon[29272]: pgmap v13: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:53.628 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:53 smithi132 ceph-mon[37969]: pgmap v13: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:55.560 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:55 smithi132 ceph-mon[37969]: pgmap v14: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:55.560 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:55 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:55.560 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:55 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:55.560 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:55 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:55.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:55 smithi088 ceph-mon[29272]: pgmap v14: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:55.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:55 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:55.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:55 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:55.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:55 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:01:57.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:57 smithi132 ceph-mon[37969]: pgmap v15: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:57.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:57 smithi088 ceph-mon[29272]: pgmap v15: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:59.421 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:59 smithi132 ceph-mon[37969]: pgmap v16: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:59.421 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:59 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:59.421 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:01:59 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:59 smithi088 ceph-mon[29272]: pgmap v16: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:01:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:59 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:01:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:01:59 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:00.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:00 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:00.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:00 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:00.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:00 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:00.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:00 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:01.700 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:01 smithi132 ceph-mon[37969]: pgmap v17: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:01.700 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:01.701 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:01.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:01 smithi088 ceph-mon[29272]: pgmap v17: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:01.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:01.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: Standby manager daemon smithi132.psllsa restarted 2024-02-19T19:02:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: Standby manager daemon smithi132.psllsa started 2024-02-19T19:02:03.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/3479998188' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/crt"}]: dispatch 2024-02-19T19:02:03.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/3479998188' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:02:03.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/3479998188' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/key"}]: dispatch 2024-02-19T19:02:03.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/3479998188' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:02:03.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: pgmap v18: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:03.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:02:03.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:03.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:03.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:02:03.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:02:03.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:03 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:03.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: Standby manager daemon smithi132.psllsa restarted 2024-02-19T19:02:03.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: Standby manager daemon smithi132.psllsa started 2024-02-19T19:02:03.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/3479998188' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/crt"}]: dispatch 2024-02-19T19:02:03.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/3479998188' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:02:03.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/3479998188' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/key"}]: dispatch 2024-02-19T19:02:03.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: from='mgr.? 172.21.15.132:0/3479998188' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:02:03.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: pgmap v18: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:03.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:02:03.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:03.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:03.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:02:03.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:02:03.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:03 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: mgrmap e31: smithi088.gyviqq(active, since 30s), standbys: smithi132.psllsa 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: Upgrade: Setting container_image for all mgr 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi088.gyviqq"}]: dispatch 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi088.gyviqq"}]': finished 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi132.psllsa"}]: dispatch 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi132.psllsa"}]': finished 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: Upgrade: Setting container_image for all rgw 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: Upgrade: Setting container_image for all rbd-mirror 2024-02-19T19:02:04.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: Upgrade: Setting container_image for all cephfs-mirror 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: Upgrade: Setting container_image for all iscsi 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: Upgrade: Setting container_image for all nfs 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: Upgrade: Setting container_image for all nvmeof 2024-02-19T19:02:04.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:04 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: mgrmap e31: smithi088.gyviqq(active, since 30s), standbys: smithi132.psllsa 2024-02-19T19:02:04.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:04.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: Upgrade: Setting container_image for all mgr 2024-02-19T19:02:04.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi088.gyviqq"}]: dispatch 2024-02-19T19:02:04.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi088.gyviqq"}]': finished 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi132.psllsa"}]: dispatch 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi132.psllsa"}]': finished 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: Upgrade: Setting container_image for all rgw 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: Upgrade: Setting container_image for all rbd-mirror 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: Upgrade: Setting container_image for all cephfs-mirror 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: Upgrade: Setting container_image for all iscsi 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: Upgrade: Setting container_image for all nfs 2024-02-19T19:02:04.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:04.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:04.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: Upgrade: Setting container_image for all nvmeof 2024-02-19T19:02:04.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:04 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:05.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:05 smithi088 ceph-mon[29272]: pgmap v19: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:05.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:05 smithi132 ceph-mon[37969]: pgmap v19: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:06.185 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:06 smithi088 ceph-mon[29272]: Upgrade: Updating node-exporter.smithi088 (1/2) 2024-02-19T19:02:06.186 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:06 smithi088 ceph-mon[29272]: Deploying daemon node-exporter.smithi088 on smithi088 2024-02-19T19:02:06.327 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:06 smithi132 ceph-mon[37969]: Upgrade: Updating node-exporter.smithi088 (1/2) 2024-02-19T19:02:06.327 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:06 smithi132 ceph-mon[37969]: Deploying daemon node-exporter.smithi088 on smithi088 2024-02-19T19:02:07.218 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:07 smithi088 ceph-mon[29272]: pgmap v20: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:07.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:07 smithi132 ceph-mon[37969]: pgmap v20: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:08.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:08 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:08.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:08 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:08.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:08 smithi132 ceph-mon[37969]: pgmap v21: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:08.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:08 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:08.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:08 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:08.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:08 smithi088 ceph-mon[29272]: pgmap v21: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.598+0000 7f03e2d02700 1 -- 172.21.15.88:0/1667583980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03dc1003f0 msgr2=0x7f03dc100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.598+0000 7f03e2d02700 1 --2- 172.21.15.88:0/1667583980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03dc1003f0 0x7f03dc100860 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f03c8009910 tx=0x7f03c8009c20 comp rx=0 tx=0).stop 2024-02-19T19:02:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.598+0000 7f03e2d02700 1 -- 172.21.15.88:0/1667583980 shutdown_connections 2024-02-19T19:02:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.598+0000 7f03e2d02700 1 --2- 172.21.15.88:0/1667583980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03dc1003f0 0x7f03dc100860 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.598+0000 7f03e2d02700 1 --2- 172.21.15.88:0/1667583980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03dc0ff150 0x7f03dc0ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.598+0000 7f03e2d02700 1 -- 172.21.15.88:0/1667583980 >> 172.21.15.88:0/1667583980 conn(0x7f03dc0fa7a0 msgr2=0x7f03dc0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.598+0000 7f03e2d02700 1 -- 172.21.15.88:0/1667583980 shutdown_connections 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.598+0000 7f03e2d02700 1 -- 172.21.15.88:0/1667583980 wait complete. 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.598+0000 7f03e2d02700 1 Processor -- start 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.598+0000 7f03e2d02700 1 -- start start 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03e2d02700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03dc0ff150 0x7f03dc069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03e2d02700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03dc1003f0 0x7f03dc19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03e2d02700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc196980 con 0x7f03dc1003f0 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03e2d02700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc19d450 con 0x7f03dc0ff150 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03dbfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03dc1003f0 0x7f03dc19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03dbfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03dc1003f0 0x7f03dc19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44540/0 (socket says 172.21.15.88:44540) 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03dbfff700 1 -- 172.21.15.88:0/2064899833 learned_addr learned my addr 172.21.15.88:0/2064899833 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03e0a9e700 1 --2- 172.21.15.88:0/2064899833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03dc0ff150 0x7f03dc069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03dbfff700 1 -- 172.21.15.88:0/2064899833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03dc0ff150 msgr2=0x7f03dc069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03dbfff700 1 --2- 172.21.15.88:0/2064899833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03dc0ff150 0x7f03dc069930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03dbfff700 1 -- 172.21.15.88:0/2064899833 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c80095f0 con 0x7f03dc1003f0 2024-02-19T19:02:09.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03dbfff700 1 --2- 172.21.15.88:0/2064899833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03dc1003f0 0x7f03dc19cf10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f03c800bd70 tx=0x7f03c800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:09.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.599+0000 7f03d9ffb700 1 -- 172.21.15.88:0/2064899833 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03c801d070 con 0x7f03dc1003f0 2024-02-19T19:02:09.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.600+0000 7f03e2d02700 1 -- 172.21.15.88:0/2064899833 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc19d6a0 con 0x7f03dc1003f0 2024-02-19T19:02:09.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.600+0000 7f03d9ffb700 1 -- 172.21.15.88:0/2064899833 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f03c800f460 con 0x7f03dc1003f0 2024-02-19T19:02:09.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.600+0000 7f03d9ffb700 1 -- 172.21.15.88:0/2064899833 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03c8021640 con 0x7f03dc1003f0 2024-02-19T19:02:09.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.600+0000 7f03e2d02700 1 -- 172.21.15.88:0/2064899833 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc19db90 con 0x7f03dc1003f0 2024-02-19T19:02:09.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.601+0000 7f03e2d02700 1 -- 172.21.15.88:0/2064899833 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03dc068ab0 con 0x7f03dc1003f0 2024-02-19T19:02:09.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.601+0000 7f03d9ffb700 1 -- 172.21.15.88:0/2064899833 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f03c800fa40 con 0x7f03dc1003f0 2024-02-19T19:02:09.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.602+0000 7f03d9ffb700 1 --2- 172.21.15.88:0/2064899833 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f03cc06d660 0x7f03cc06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:09.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.602+0000 7f03d9ffb700 1 -- 172.21.15.88:0/2064899833 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f03c808e500 con 0x7f03dc1003f0 2024-02-19T19:02:09.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.602+0000 7f03e0a9e700 1 --2- 172.21.15.88:0/2064899833 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f03cc06d660 0x7f03cc06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:09.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.602+0000 7f03e0a9e700 1 --2- 172.21.15.88:0/2064899833 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f03cc06d660 0x7f03cc06fb10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03d0005950 tx=0x7f03d00058e0 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:09.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.606+0000 7f03d9ffb700 1 -- 172.21.15.88:0/2064899833 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f03c805baf0 con 0x7f03dc1003f0 2024-02-19T19:02:09.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.833+0000 7f03e2d02700 1 -- 172.21.15.88:0/2064899833 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03dc104e40 con 0x7f03cc06d660 2024-02-19T19:02:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.835+0000 7f03d9ffb700 1 -- 172.21.15.88:0/2064899833 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f03dc104e40 con 0x7f03cc06d660 2024-02-19T19:02:09.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.837+0000 7f03e2d02700 1 -- 172.21.15.88:0/2064899833 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f03cc06d660 msgr2=0x7f03cc06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:09.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.837+0000 7f03e2d02700 1 --2- 172.21.15.88:0/2064899833 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f03cc06d660 0x7f03cc06fb10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f03d0005950 tx=0x7f03d00058e0 comp rx=0 tx=0).stop 2024-02-19T19:02:09.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.837+0000 7f03e2d02700 1 -- 172.21.15.88:0/2064899833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03dc1003f0 msgr2=0x7f03dc19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:09.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.837+0000 7f03e2d02700 1 --2- 172.21.15.88:0/2064899833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03dc1003f0 0x7f03dc19cf10 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f03c800bd70 tx=0x7f03c800be50 comp rx=0 tx=0).stop 2024-02-19T19:02:09.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.837+0000 7f03e2d02700 1 -- 172.21.15.88:0/2064899833 shutdown_connections 2024-02-19T19:02:09.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.837+0000 7f03e2d02700 1 --2- 172.21.15.88:0/2064899833 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f03cc06d660 0x7f03cc06fb10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:09.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.837+0000 7f03e2d02700 1 --2- 172.21.15.88:0/2064899833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03dc0ff150 0x7f03dc069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:09.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.837+0000 7f03e2d02700 1 --2- 172.21.15.88:0/2064899833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03dc1003f0 0x7f03dc19cf10 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:09.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.837+0000 7f03e2d02700 1 -- 172.21.15.88:0/2064899833 >> 172.21.15.88:0/2064899833 conn(0x7f03dc0fa7a0 msgr2=0x7f03dc103720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:09.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.837+0000 7f03e2d02700 1 -- 172.21.15.88:0/2064899833 shutdown_connections 2024-02-19T19:02:09.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.837+0000 7f03e2d02700 1 -- 172.21.15.88:0/2064899833 wait complete. 2024-02-19T19:02:09.867 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:02:09.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.943+0000 7f625bfbc700 1 -- 172.21.15.88:0/3786625673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6254101440 msgr2=0x7f6254101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:09.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.943+0000 7f625bfbc700 1 --2- 172.21.15.88:0/3786625673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6254101440 0x7f6254101890 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6244009910 tx=0x7f6244009c20 comp rx=0 tx=0).stop 2024-02-19T19:02:09.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.943+0000 7f625bfbc700 1 -- 172.21.15.88:0/3786625673 shutdown_connections 2024-02-19T19:02:09.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.943+0000 7f625bfbc700 1 --2- 172.21.15.88:0/3786625673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6254101440 0x7f6254101890 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:09.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.943+0000 7f625bfbc700 1 --2- 172.21.15.88:0/3786625673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6254100240 0x7f6254100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:09.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.943+0000 7f625bfbc700 1 -- 172.21.15.88:0/3786625673 >> 172.21.15.88:0/3786625673 conn(0x7f62540fb7f0 msgr2=0x7f62540fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:09.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.943+0000 7f625bfbc700 1 -- 172.21.15.88:0/3786625673 shutdown_connections 2024-02-19T19:02:09.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.943+0000 7f625bfbc700 1 -- 172.21.15.88:0/3786625673 wait complete. 2024-02-19T19:02:09.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.944+0000 7f625bfbc700 1 Processor -- start 2024-02-19T19:02:09.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.944+0000 7f625bfbc700 1 -- start start 2024-02-19T19:02:09.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.944+0000 7f625bfbc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6254100240 0x7f6254074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:09.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.944+0000 7f625bfbc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6254101440 0x7f6254073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:09.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.944+0000 7f625bfbc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6254073680 con 0x7f6254100240 2024-02-19T19:02:09.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.944+0000 7f625bfbc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62540737c0 con 0x7f6254101440 2024-02-19T19:02:09.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.944+0000 7f6259d58700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6254100240 0x7f6254074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:09.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.944+0000 7f6259d58700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6254100240 0x7f6254074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44554/0 (socket says 172.21.15.88:44554) 2024-02-19T19:02:09.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.944+0000 7f6259d58700 1 -- 172.21.15.88:0/2984813894 learned_addr learned my addr 172.21.15.88:0/2984813894 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:02:09.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.944+0000 7f6259557700 1 --2- 172.21.15.88:0/2984813894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6254101440 0x7f6254073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:09.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.945+0000 7f6259d58700 1 -- 172.21.15.88:0/2984813894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6254101440 msgr2=0x7f6254073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:09.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.945+0000 7f6259d58700 1 --2- 172.21.15.88:0/2984813894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6254101440 0x7f6254073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:09.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.945+0000 7f6259d58700 1 -- 172.21.15.88:0/2984813894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440095f0 con 0x7f6254100240 2024-02-19T19:02:09.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.945+0000 7f6259d58700 1 --2- 172.21.15.88:0/2984813894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6254100240 0x7f6254074af0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f625000d900 tx=0x7f625000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:09.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.945+0000 7f624affd700 1 -- 172.21.15.88:0/2984813894 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62500041f0 con 0x7f6254100240 2024-02-19T19:02:09.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.945+0000 7f624affd700 1 -- 172.21.15.88:0/2984813894 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6250004350 con 0x7f6254100240 2024-02-19T19:02:09.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.945+0000 7f624affd700 1 -- 172.21.15.88:0/2984813894 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6250003d20 con 0x7f6254100240 2024-02-19T19:02:09.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.945+0000 7f625bfbc700 1 -- 172.21.15.88:0/2984813894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6254073aa0 con 0x7f6254100240 2024-02-19T19:02:09.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.945+0000 7f625bfbc700 1 -- 172.21.15.88:0/2984813894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6254073fa0 con 0x7f6254100240 2024-02-19T19:02:09.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.946+0000 7f625bfbc700 1 -- 172.21.15.88:0/2984813894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6254108d90 con 0x7f6254100240 2024-02-19T19:02:09.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.947+0000 7f624affd700 1 -- 172.21.15.88:0/2984813894 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f6250009730 con 0x7f6254100240 2024-02-19T19:02:09.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.947+0000 7f624affd700 1 --2- 172.21.15.88:0/2984813894 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f624006d660 0x7f624006fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:09.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.947+0000 7f624affd700 1 -- 172.21.15.88:0/2984813894 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f625008d0c0 con 0x7f6254100240 2024-02-19T19:02:09.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.947+0000 7f6259557700 1 --2- 172.21.15.88:0/2984813894 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f624006d660 0x7f624006fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:09.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.948+0000 7f6259557700 1 --2- 172.21.15.88:0/2984813894 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f624006d660 0x7f624006fb10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f624400b5c0 tx=0x7f6244005ee0 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:09.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:09.952+0000 7f624affd700 1 -- 172.21.15.88:0/2984813894 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f625005a6b0 con 0x7f6254100240 2024-02-19T19:02:10.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.170+0000 7f625bfbc700 1 -- 172.21.15.88:0/2984813894 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6254105d90 con 0x7f624006d660 2024-02-19T19:02:10.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.171+0000 7f624affd700 1 -- 172.21.15.88:0/2984813894 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f6254105d90 con 0x7f624006d660 2024-02-19T19:02:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.174+0000 7f625bfbc700 1 -- 172.21.15.88:0/2984813894 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f624006d660 msgr2=0x7f624006fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.174+0000 7f625bfbc700 1 --2- 172.21.15.88:0/2984813894 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f624006d660 0x7f624006fb10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f624400b5c0 tx=0x7f6244005ee0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.174+0000 7f625bfbc700 1 -- 172.21.15.88:0/2984813894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6254100240 msgr2=0x7f6254074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.174+0000 7f625bfbc700 1 --2- 172.21.15.88:0/2984813894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6254100240 0x7f6254074af0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f625000d900 tx=0x7f625000dc10 comp rx=0 tx=0).stop 2024-02-19T19:02:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.174+0000 7f625bfbc700 1 -- 172.21.15.88:0/2984813894 shutdown_connections 2024-02-19T19:02:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.174+0000 7f625bfbc700 1 --2- 172.21.15.88:0/2984813894 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f624006d660 0x7f624006fb10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.174+0000 7f625bfbc700 1 --2- 172.21.15.88:0/2984813894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6254100240 0x7f6254074af0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.174+0000 7f625bfbc700 1 --2- 172.21.15.88:0/2984813894 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6254101440 0x7f6254073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.174+0000 7f625bfbc700 1 -- 172.21.15.88:0/2984813894 >> 172.21.15.88:0/2984813894 conn(0x7f62540fb7f0 msgr2=0x7f6254104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.174+0000 7f625bfbc700 1 -- 172.21.15.88:0/2984813894 shutdown_connections 2024-02-19T19:02:10.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.174+0000 7f625bfbc700 1 -- 172.21.15.88:0/2984813894 wait complete. 2024-02-19T19:02:10.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.277+0000 7f47b2011700 1 -- 172.21.15.88:0/2439176318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ac100240 msgr2=0x7f47ac100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.277+0000 7f47b2011700 1 --2- 172.21.15.88:0/2439176318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ac100240 0x7f47ac100650 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f479c009910 tx=0x7f479c009c20 comp rx=0 tx=0).stop 2024-02-19T19:02:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.277+0000 7f47b2011700 1 -- 172.21.15.88:0/2439176318 shutdown_connections 2024-02-19T19:02:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.277+0000 7f47b2011700 1 --2- 172.21.15.88:0/2439176318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac101440 0x7f47ac101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.277+0000 7f47b2011700 1 --2- 172.21.15.88:0/2439176318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ac100240 0x7f47ac100650 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.277+0000 7f47b2011700 1 -- 172.21.15.88:0/2439176318 >> 172.21.15.88:0/2439176318 conn(0x7f47ac0fb7f0 msgr2=0x7f47ac0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.278+0000 7f47b2011700 1 -- 172.21.15.88:0/2439176318 shutdown_connections 2024-02-19T19:02:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.278+0000 7f47b2011700 1 -- 172.21.15.88:0/2439176318 wait complete. 2024-02-19T19:02:10.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.278+0000 7f47b2011700 1 Processor -- start 2024-02-19T19:02:10.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47b2011700 1 -- start start 2024-02-19T19:02:10.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47b2011700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ac100240 0x7f47ac074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:10.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47b2011700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac101440 0x7f47ac073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:10.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47b2011700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac073680 con 0x7f47ac100240 2024-02-19T19:02:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47b2011700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0737c0 con 0x7f47ac101440 2024-02-19T19:02:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ac100240 0x7f47ac074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47ab7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ac100240 0x7f47ac074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44564/0 (socket says 172.21.15.88:44564) 2024-02-19T19:02:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47ab7fe700 1 -- 172.21.15.88:0/829076562 learned_addr learned my addr 172.21.15.88:0/829076562 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:02:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47aaffd700 1 --2- 172.21.15.88:0/829076562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac101440 0x7f47ac073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47ab7fe700 1 -- 172.21.15.88:0/829076562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac101440 msgr2=0x7f47ac073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47ab7fe700 1 --2- 172.21.15.88:0/829076562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac101440 0x7f47ac073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47ab7fe700 1 -- 172.21.15.88:0/829076562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f479c0095f0 con 0x7f47ac100240 2024-02-19T19:02:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.279+0000 7f47ab7fe700 1 --2- 172.21.15.88:0/829076562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ac100240 0x7f47ac074af0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f479c000c00 tx=0x7f479c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.280+0000 7f47a8ff9700 1 -- 172.21.15.88:0/829076562 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f479c01d070 con 0x7f47ac100240 2024-02-19T19:02:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.280+0000 7f47b2011700 1 -- 172.21.15.88:0/829076562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac073a40 con 0x7f47ac100240 2024-02-19T19:02:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.280+0000 7f47a8ff9700 1 -- 172.21.15.88:0/829076562 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f479c005dc0 con 0x7f47ac100240 2024-02-19T19:02:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.280+0000 7f47a8ff9700 1 -- 172.21.15.88:0/829076562 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f479c022470 con 0x7f47ac100240 2024-02-19T19:02:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.280+0000 7f47b2011700 1 -- 172.21.15.88:0/829076562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac073f30 con 0x7f47ac100240 2024-02-19T19:02:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.281+0000 7f47b2011700 1 -- 172.21.15.88:0/829076562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ac108d90 con 0x7f47ac100240 2024-02-19T19:02:10.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.282+0000 7f47a8ff9700 1 -- 172.21.15.88:0/829076562 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f479c003810 con 0x7f47ac100240 2024-02-19T19:02:10.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.282+0000 7f47a8ff9700 1 --2- 172.21.15.88:0/829076562 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f479406d660 0x7f479406fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:10.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.282+0000 7f47a8ff9700 1 -- 172.21.15.88:0/829076562 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f479c08e6e0 con 0x7f47ac100240 2024-02-19T19:02:10.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.282+0000 7f47aaffd700 1 --2- 172.21.15.88:0/829076562 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f479406d660 0x7f479406fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:10.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.283+0000 7f47aaffd700 1 --2- 172.21.15.88:0/829076562 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f479406d660 0x7f479406fb10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009c60 tx=0x7f47a0009260 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:10.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.287+0000 7f47a8ff9700 1 -- 172.21.15.88:0/829076562 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f479c05bcd0 con 0x7f47ac100240 2024-02-19T19:02:10.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.495+0000 7f47b2011700 1 -- 172.21.15.88:0/829076562 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f47ac105d90 con 0x7f479406d660 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.506+0000 7f47a8ff9700 1 -- 172.21.15.88:0/829076562 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f47ac105d90 con 0x7f479406d660 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (7m) 10s ago 9m 24.7M - 0.25.0 c8568f914cd2 d6dd804f52d8 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (10m) 10s ago 10m 7688k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (8m) 12s ago 8m 7587k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (10m) 10s ago 10m 6660k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (8m) 12s ago 8m 6656k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (7m) 10s ago 9m 88.5M - 9.4.7 954c08fa6188 54907dc6edb4 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (3m) 10s ago 3m 13.6M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (3m) 10s ago 3m 26.4M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (3m) 12s ago 3m 22.7M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:02:10.526 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (3m) 12s ago 3m 17.5M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (49s) 10s ago 11m 492M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (15s) 12s ago 8m 66.7M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (11m) 10s ago 11m 53.2M 2048M 18.2.0 dc2bc1663786 3ea5108455bc 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (8m) 12s ago 8m 42.1M 2048M 18.2.0 dc2bc1663786 ba1c249c6017 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 starting - - - - 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (8m) 12s ago 8m 16.2M - 1.5.0 0da6a335fe13 f93d4349304d 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (7m) 10s ago 7m 51.5M 1216M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (6m) 10s ago 6m 49.1M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (6m) 10s ago 6m 48.9M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (5m) 12s ago 5m 50.5M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (5m) 12s ago 5m 49.0M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (5m) 12s ago 5m 47.0M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:02:10.527 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (21s) 10s ago 8m 49.0M - 2.43.0 a07b618ecd1d de7125b091db 2024-02-19T19:02:10.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.508+0000 7f47b2011700 1 -- 172.21.15.88:0/829076562 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f479406d660 msgr2=0x7f479406fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:10.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.508+0000 7f47b2011700 1 --2- 172.21.15.88:0/829076562 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f479406d660 0x7f479406fb10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009c60 tx=0x7f47a0009260 comp rx=0 tx=0).stop 2024-02-19T19:02:10.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.508+0000 7f47b2011700 1 -- 172.21.15.88:0/829076562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ac100240 msgr2=0x7f47ac074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:10.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.508+0000 7f47b2011700 1 --2- 172.21.15.88:0/829076562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ac100240 0x7f47ac074af0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f479c000c00 tx=0x7f479c00f690 comp rx=0 tx=0).stop 2024-02-19T19:02:10.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.508+0000 7f47b2011700 1 -- 172.21.15.88:0/829076562 shutdown_connections 2024-02-19T19:02:10.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.508+0000 7f47b2011700 1 --2- 172.21.15.88:0/829076562 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f479406d660 0x7f479406fb10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.508+0000 7f47b2011700 1 --2- 172.21.15.88:0/829076562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47ac100240 0x7f47ac074af0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.508+0000 7f47b2011700 1 --2- 172.21.15.88:0/829076562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47ac101440 0x7f47ac073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.508+0000 7f47b2011700 1 -- 172.21.15.88:0/829076562 >> 172.21.15.88:0/829076562 conn(0x7f47ac0fb7f0 msgr2=0x7f47ac104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:10.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.508+0000 7f47b2011700 1 -- 172.21.15.88:0/829076562 shutdown_connections 2024-02-19T19:02:10.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.508+0000 7f47b2011700 1 -- 172.21.15.88:0/829076562 wait complete. 2024-02-19T19:02:10.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.616+0000 7f38a9d83700 1 -- 172.21.15.88:0/2539686798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38a40ff150 msgr2=0x7f38a40ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:10.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.616+0000 7f38a9d83700 1 --2- 172.21.15.88:0/2539686798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38a40ff150 0x7f38a40ff560 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3894009910 tx=0x7f3894009c20 comp rx=0 tx=0).stop 2024-02-19T19:02:10.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.616+0000 7f38a9d83700 1 -- 172.21.15.88:0/2539686798 shutdown_connections 2024-02-19T19:02:10.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.616+0000 7f38a9d83700 1 --2- 172.21.15.88:0/2539686798 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38a4100350 0x7f38a41007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.616+0000 7f38a9d83700 1 --2- 172.21.15.88:0/2539686798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38a40ff150 0x7f38a40ff560 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.616+0000 7f38a9d83700 1 -- 172.21.15.88:0/2539686798 >> 172.21.15.88:0/2539686798 conn(0x7f38a40fa700 msgr2=0x7f38a40fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:10.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.616+0000 7f38a9d83700 1 -- 172.21.15.88:0/2539686798 shutdown_connections 2024-02-19T19:02:10.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.616+0000 7f38a9d83700 1 -- 172.21.15.88:0/2539686798 wait complete. 2024-02-19T19:02:10.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.616+0000 7f38a9d83700 1 Processor -- start 2024-02-19T19:02:10.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a9d83700 1 -- start start 2024-02-19T19:02:10.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a9d83700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38a40ff150 0x7f38a4069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:10.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a9d83700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38a4100350 0x7f38a419a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:10.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a9d83700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a419ae10 con 0x7f38a4100350 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a9d83700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a419af50 con 0x7f38a40ff150 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38a4100350 0x7f38a419a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38a4100350 0x7f38a419a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44578/0 (socket says 172.21.15.88:44578) 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a2ffd700 1 -- 172.21.15.88:0/1500635288 learned_addr learned my addr 172.21.15.88:0/1500635288 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a37fe700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38a40ff150 0x7f38a4069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a2ffd700 1 -- 172.21.15.88:0/1500635288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38a40ff150 msgr2=0x7f38a4069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a2ffd700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38a40ff150 0x7f38a4069930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a2ffd700 1 -- 172.21.15.88:0/1500635288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38940095f0 con 0x7f38a4100350 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a37fe700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38a40ff150 0x7f38a4069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a2ffd700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38a4100350 0x7f38a419a840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f389800e920 tx=0x7f389800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a0ff9700 1 -- 172.21.15.88:0/1500635288 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3898009790 con 0x7f38a4100350 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a0ff9700 1 -- 172.21.15.88:0/1500635288 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f389800cba0 con 0x7f38a4100350 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a9d83700 1 -- 172.21.15.88:0/1500635288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a419b230 con 0x7f38a4100350 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.617+0000 7f38a0ff9700 1 -- 172.21.15.88:0/1500635288 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38980186e0 con 0x7f38a4100350 2024-02-19T19:02:10.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.618+0000 7f38a9d83700 1 -- 172.21.15.88:0/1500635288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a419b780 con 0x7f38a4100350 2024-02-19T19:02:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.619+0000 7f38a9d83700 1 -- 172.21.15.88:0/1500635288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a4107ca0 con 0x7f38a4100350 2024-02-19T19:02:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.619+0000 7f38a0ff9700 1 -- 172.21.15.88:0/1500635288 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f389800cd10 con 0x7f38a4100350 2024-02-19T19:02:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.620+0000 7f38a0ff9700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f388c06d660 0x7f388c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.620+0000 7f38a37fe700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f388c06d660 0x7f388c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:10.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.620+0000 7f38a0ff9700 1 -- 172.21.15.88:0/1500635288 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3898014070 con 0x7f38a4100350 2024-02-19T19:02:10.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.620+0000 7f38a37fe700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f388c06d660 0x7f388c06fb10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f38940059d0 tx=0x7f3894005940 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:10.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.624+0000 7f38a0ff9700 1 -- 172.21.15.88:0/1500635288 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f3898052a50 con 0x7f38a4100350 2024-02-19T19:02:10.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.905+0000 7f38a9d83700 1 -- 172.21.15.88:0/1500635288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f38a4107ec0 con 0x7f38a4100350 2024-02-19T19:02:10.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.906+0000 7f38a0ff9700 1 -- 172.21.15.88:0/1500635288 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f389805ab10 con 0x7f38a4100350 2024-02-19T19:02:10.926 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:02:10.926 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:02:10.927 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-19T19:02:10.927 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:02:10.927 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:02:10.927 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:02:10.927 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:02:10.927 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:02:10.927 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:02:10.928 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:02:10.928 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:02:10.928 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:02:10.928 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:02:10.928 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:02:10.928 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-19T19:02:10.929 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:02:10.929 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:02:10.929 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:02:10.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.908+0000 7f38a9d83700 1 -- 172.21.15.88:0/1500635288 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f388c06d660 msgr2=0x7f388c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:10.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.908+0000 7f38a9d83700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f388c06d660 0x7f388c06fb10 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f38940059d0 tx=0x7f3894005940 comp rx=0 tx=0).stop 2024-02-19T19:02:10.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.908+0000 7f38a9d83700 1 -- 172.21.15.88:0/1500635288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38a4100350 msgr2=0x7f38a419a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:10.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.908+0000 7f38a9d83700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38a4100350 0x7f38a419a840 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f389800e920 tx=0x7f389800ece0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.908+0000 7f38a9d83700 1 -- 172.21.15.88:0/1500635288 shutdown_connections 2024-02-19T19:02:10.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.908+0000 7f38a9d83700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f388c06d660 0x7f388c06fb10 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.908+0000 7f38a9d83700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38a40ff150 0x7f38a4069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.908+0000 7f38a9d83700 1 --2- 172.21.15.88:0/1500635288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38a4100350 0x7f38a419a840 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:10.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.908+0000 7f38a9d83700 1 -- 172.21.15.88:0/1500635288 >> 172.21.15.88:0/1500635288 conn(0x7f38a40fa700 msgr2=0x7f38a4103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:10.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.908+0000 7f38a9d83700 1 -- 172.21.15.88:0/1500635288 shutdown_connections 2024-02-19T19:02:10.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:10.908+0000 7f38a9d83700 1 -- 172.21.15.88:0/1500635288 wait complete. 2024-02-19T19:02:11.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.013+0000 7fe0a1586700 1 -- 172.21.15.88:0/3309358486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe09c0ff150 msgr2=0x7fe09c0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:11.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.013+0000 7fe0a1586700 1 --2- 172.21.15.88:0/3309358486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe09c0ff150 0x7fe09c0ff560 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009910 tx=0x7fe08c009c20 comp rx=0 tx=0).stop 2024-02-19T19:02:11.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.013+0000 7fe0a1586700 1 -- 172.21.15.88:0/3309358486 shutdown_connections 2024-02-19T19:02:11.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.013+0000 7fe0a1586700 1 --2- 172.21.15.88:0/3309358486 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe09c1003f0 0x7fe09c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:11.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.013+0000 7fe0a1586700 1 --2- 172.21.15.88:0/3309358486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe09c0ff150 0x7fe09c0ff560 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:11.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.013+0000 7fe0a1586700 1 -- 172.21.15.88:0/3309358486 >> 172.21.15.88:0/3309358486 conn(0x7fe09c0fa7a0 msgr2=0x7fe09c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:11.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.013+0000 7fe0a1586700 1 -- 172.21.15.88:0/3309358486 shutdown_connections 2024-02-19T19:02:11.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.013+0000 7fe0a1586700 1 -- 172.21.15.88:0/3309358486 wait complete. 2024-02-19T19:02:11.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.014+0000 7fe0a1586700 1 Processor -- start 2024-02-19T19:02:11.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.014+0000 7fe0a1586700 1 -- start start 2024-02-19T19:02:11.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.014+0000 7fe0a1586700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe09c0ff150 0x7fe09c069990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:11.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.014+0000 7fe0a1586700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe09c1003f0 0x7fe09c19ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:11.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.014+0000 7fe0a1586700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c19b170 con 0x7fe09c0ff150 2024-02-19T19:02:11.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.014+0000 7fe0a1586700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c19b2b0 con 0x7fe09c1003f0 2024-02-19T19:02:11.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.014+0000 7fe09affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe09c0ff150 0x7fe09c069990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:11.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.014+0000 7fe09affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe09c0ff150 0x7fe09c069990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44604/0 (socket says 172.21.15.88:44604) 2024-02-19T19:02:11.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.014+0000 7fe09affd700 1 -- 172.21.15.88:0/2579622494 learned_addr learned my addr 172.21.15.88:0/2579622494 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:02:11.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.014+0000 7fe09a7fc700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe09c1003f0 0x7fe09c19ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:11.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.014+0000 7fe09affd700 1 -- 172.21.15.88:0/2579622494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe09c1003f0 msgr2=0x7fe09c19ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:11.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.015+0000 7fe09affd700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe09c1003f0 0x7fe09c19ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:11.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.015+0000 7fe09affd700 1 -- 172.21.15.88:0/2579622494 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c0095f0 con 0x7fe09c0ff150 2024-02-19T19:02:11.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.015+0000 7fe09a7fc700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe09c1003f0 0x7fe09c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:02:11.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.015+0000 7fe09affd700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe09c0ff150 0x7fe09c069990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00b5c0 tx=0x7fe08c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:11.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.015+0000 7fe083fff700 1 -- 172.21.15.88:0/2579622494 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe08c01d070 con 0x7fe09c0ff150 2024-02-19T19:02:11.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.015+0000 7fe083fff700 1 -- 172.21.15.88:0/2579622494 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe08c022470 con 0x7fe09c0ff150 2024-02-19T19:02:11.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.015+0000 7fe0a1586700 1 -- 172.21.15.88:0/2579622494 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c19b530 con 0x7fe09c0ff150 2024-02-19T19:02:11.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.015+0000 7fe083fff700 1 -- 172.21.15.88:0/2579622494 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe08c00f690 con 0x7fe09c0ff150 2024-02-19T19:02:11.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.015+0000 7fe0a1586700 1 -- 172.21.15.88:0/2579622494 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c19ba20 con 0x7fe09c0ff150 2024-02-19T19:02:11.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.016+0000 7fe0a1586700 1 -- 172.21.15.88:0/2579622494 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c068ab0 con 0x7fe09c0ff150 2024-02-19T19:02:11.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.016+0000 7fe083fff700 1 -- 172.21.15.88:0/2579622494 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7fe08c004b70 con 0x7fe09c0ff150 2024-02-19T19:02:11.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.017+0000 7fe083fff700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe08406d650 0x7fe08406fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:11.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.017+0000 7fe083fff700 1 -- 172.21.15.88:0/2579622494 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe08c08e950 con 0x7fe09c0ff150 2024-02-19T19:02:11.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.017+0000 7fe09a7fc700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe08406d650 0x7fe08406fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:11.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.017+0000 7fe09a7fc700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe08406d650 0x7fe08406fb00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0900099d0 tx=0x7fe090008040 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:11.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.021+0000 7fe083fff700 1 -- 172.21.15.88:0/2579622494 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fe08c05bff0 con 0x7fe09c0ff150 2024-02-19T19:02:11.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.245+0000 7fe0a1586700 1 -- 172.21.15.88:0/2579622494 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe09c104e00 con 0x7fe08406d650 2024-02-19T19:02:11.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.246+0000 7fe083fff700 1 -- 172.21.15.88:0/2579622494 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fe09c104e00 con 0x7fe08406d650 2024-02-19T19:02:11.266 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:02:11.266 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:02:11.266 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:02:11.266 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-19T19:02:11.266 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:02:11.267 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:02:11.267 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:02:11.267 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "2/2 daemons upgraded", 2024-02-19T19:02:11.267 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading node-exporter daemons", 2024-02-19T19:02:11.267 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:02:11.267 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:02:11.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.249+0000 7fe0a1586700 1 -- 172.21.15.88:0/2579622494 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe08406d650 msgr2=0x7fe08406fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:11.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.249+0000 7fe0a1586700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe08406d650 0x7fe08406fb00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0900099d0 tx=0x7fe090008040 comp rx=0 tx=0).stop 2024-02-19T19:02:11.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.249+0000 7fe0a1586700 1 -- 172.21.15.88:0/2579622494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe09c0ff150 msgr2=0x7fe09c069990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:11.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.249+0000 7fe0a1586700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe09c0ff150 0x7fe09c069990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fe08c00b5c0 tx=0x7fe08c00be20 comp rx=0 tx=0).stop 2024-02-19T19:02:11.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.249+0000 7fe0a1586700 1 -- 172.21.15.88:0/2579622494 shutdown_connections 2024-02-19T19:02:11.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.249+0000 7fe0a1586700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe08406d650 0x7fe08406fb00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:11.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.249+0000 7fe0a1586700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe09c0ff150 0x7fe09c069990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:11.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.249+0000 7fe0a1586700 1 --2- 172.21.15.88:0/2579622494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe09c1003f0 0x7fe09c19ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:11.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.249+0000 7fe0a1586700 1 -- 172.21.15.88:0/2579622494 >> 172.21.15.88:0/2579622494 conn(0x7fe09c0fa7a0 msgr2=0x7fe09c1036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:11.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.249+0000 7fe0a1586700 1 -- 172.21.15.88:0/2579622494 shutdown_connections 2024-02-19T19:02:11.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:11.249+0000 7fe0a1586700 1 -- 172.21.15.88:0/2579622494 wait complete. 2024-02-19T19:02:11.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:11 smithi088 ceph-mon[29272]: Upgrade: Updating node-exporter.smithi132 (2/2) 2024-02-19T19:02:11.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:11 smithi088 ceph-mon[29272]: Deploying daemon node-exporter.smithi132 on smithi132 2024-02-19T19:02:11.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:11 smithi088 ceph-mon[29272]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:11.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:11 smithi088 ceph-mon[29272]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:11.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:11 smithi088 ceph-mon[29272]: pgmap v22: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:11.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:11 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1500635288' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:11.669 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:11 smithi132 ceph-mon[37969]: Upgrade: Updating node-exporter.smithi132 (2/2) 2024-02-19T19:02:11.669 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:11 smithi132 ceph-mon[37969]: Deploying daemon node-exporter.smithi132 on smithi132 2024-02-19T19:02:11.669 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:11 smithi132 ceph-mon[37969]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:11.669 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:11 smithi132 ceph-mon[37969]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:11.670 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:11 smithi132 ceph-mon[37969]: pgmap v22: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:11.670 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:11 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1500635288' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:12.393 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:12 smithi132 ceph-mon[37969]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:12.393 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:12 smithi132 ceph-mon[37969]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:12.523 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:12 smithi088 ceph-mon[29272]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:12.523 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:12 smithi088 ceph-mon[29272]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:13.594 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:13 smithi088 ceph-mon[29272]: pgmap v23: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:13.594 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:13 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:13.594 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:13 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:13.594 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:13 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:13.620 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:13 smithi132 ceph-mon[37969]: pgmap v23: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:13.621 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:13 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:13.621 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:13 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:13.621 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:13 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:15.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:15 smithi088 ceph-mon[29272]: pgmap v24: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:15.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:15 smithi132 ceph-mon[37969]: pgmap v24: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:16.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:16 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:16.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:16 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:16.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:16 smithi088 ceph-mon[29272]: pgmap v25: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:17.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:16 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:17.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:16 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:17.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:16 smithi132 ceph-mon[37969]: pgmap v25: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:18.001 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:17 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:18.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:17 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:18.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:17 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:02:18.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:17 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:18.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:17 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:18.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:17 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:02:19.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:19 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:19.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:19 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:19.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:19 smithi132 ceph-mon[37969]: pgmap v26: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:19.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:19 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:19.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:19 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:19.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:19 smithi088 ceph-mon[29272]: pgmap v26: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:20.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:20 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:20.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:20 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:20.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:20 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:20.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:20 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:20.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:20 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:20.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:20 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:20.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:20 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:20.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:20 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:21.478 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:21 smithi088 ceph-mon[29272]: pgmap v27: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:21.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:21 smithi132 ceph-mon[37969]: pgmap v27: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:23.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: pgmap v28: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:23.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:23 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: pgmap v28: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:23 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:25.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:25 smithi088 ceph-mon[29272]: pgmap v29: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:25.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:25 smithi132 ceph-mon[37969]: pgmap v29: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:26.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:26 smithi088 ceph-mon[29272]: Upgrade: Updating prometheus.smithi088 2024-02-19T19:02:26.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:26 smithi088 ceph-mon[29272]: Deploying daemon prometheus.smithi088 on smithi088 2024-02-19T19:02:26.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:26 smithi132 ceph-mon[37969]: Upgrade: Updating prometheus.smithi088 2024-02-19T19:02:26.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:26 smithi132 ceph-mon[37969]: Deploying daemon prometheus.smithi088 on smithi088 2024-02-19T19:02:27.420 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:27 smithi088 ceph-mon[29272]: pgmap v30: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:27.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:27 smithi132 ceph-mon[37969]: pgmap v30: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:29.453 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:29 smithi088 ceph-mon[29272]: pgmap v31: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:29.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:29 smithi132 ceph-mon[37969]: pgmap v31: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:31.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:31 smithi088 ceph-mon[29272]: pgmap v32: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:31.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:31 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:31.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:31 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:31.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:31 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:31.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:31 smithi132 ceph-mon[37969]: pgmap v32: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:31.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:31 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:31.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:31 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:31.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:31 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:33.307 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:33 smithi088 ceph-mon[29272]: pgmap v33: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:33.307 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:33 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:02:33.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:33 smithi132 ceph-mon[37969]: pgmap v33: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:33.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:33 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:02:35.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:35 smithi088 ceph-mon[29272]: pgmap v34: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:35.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:35 smithi132 ceph-mon[37969]: pgmap v34: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:37.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:37 smithi088 ceph-mon[29272]: pgmap v35: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:37.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:37 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:37.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:37 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:37.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:37 smithi132 ceph-mon[37969]: pgmap v35: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:37.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:37 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:37.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:37 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:38.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:38 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:38.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:38 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:38.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:38 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:38.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:38 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:39.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:39 smithi132 ceph-mon[37969]: pgmap v36: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:39.794 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:39 smithi088 ceph-mon[29272]: pgmap v36: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:41.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.362+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1120077750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30a01036f0 msgr2=0x7f30a0103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:41.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.362+0000 7f30a7ef9700 1 --2- 172.21.15.88:0/1120077750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30a01036f0 0x7f30a0103b60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f3090009910 tx=0x7f3090009c20 comp rx=0 tx=0).stop 2024-02-19T19:02:41.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.363+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1120077750 shutdown_connections 2024-02-19T19:02:41.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.363+0000 7f30a7ef9700 1 --2- 172.21.15.88:0/1120077750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30a01036f0 0x7f30a0103b60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.363+0000 7f30a7ef9700 1 --2- 172.21.15.88:0/1120077750 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30a0102450 0x7f30a0102860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.363+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1120077750 >> 172.21.15.88:0/1120077750 conn(0x7f30a00fdac0 msgr2=0x7f30a00ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:41.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.363+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1120077750 shutdown_connections 2024-02-19T19:02:41.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.363+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1120077750 wait complete. 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.363+0000 7f30a7ef9700 1 Processor -- start 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.363+0000 7f30a7ef9700 1 -- start start 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.363+0000 7f30a7ef9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30a0102450 0x7f30a0197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a7ef9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30a01036f0 0x7f30a0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a7ef9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a0198890 con 0x7f30a01036f0 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a7ef9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a019d2a0 con 0x7f30a0102450 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a5494700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30a01036f0 0x7f30a0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a5494700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30a01036f0 0x7f30a0198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48260/0 (socket says 172.21.15.88:48260) 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a5494700 1 -- 172.21.15.88:0/1419545178 learned_addr learned my addr 172.21.15.88:0/1419545178 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a5c95700 1 --2- 172.21.15.88:0/1419545178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30a0102450 0x7f30a0197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a5494700 1 -- 172.21.15.88:0/1419545178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30a0102450 msgr2=0x7f30a0197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a5494700 1 --2- 172.21.15.88:0/1419545178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30a0102450 0x7f30a0197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a5494700 1 -- 172.21.15.88:0/1419545178 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30900095f0 con 0x7f30a01036f0 2024-02-19T19:02:41.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a5494700 1 --2- 172.21.15.88:0/1419545178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30a01036f0 0x7f30a0198270 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3090006010 tx=0x7f309000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:41.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f3096ffd700 1 -- 172.21.15.88:0/1419545178 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f309001d070 con 0x7f30a01036f0 2024-02-19T19:02:41.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f3096ffd700 1 -- 172.21.15.88:0/1419545178 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f309000f460 con 0x7f30a01036f0 2024-02-19T19:02:41.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.364+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1419545178 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a019d440 con 0x7f30a01036f0 2024-02-19T19:02:41.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.365+0000 7f3096ffd700 1 -- 172.21.15.88:0/1419545178 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3090021640 con 0x7f30a01036f0 2024-02-19T19:02:41.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.365+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1419545178 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a019d900 con 0x7f30a01036f0 2024-02-19T19:02:41.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.366+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1419545178 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a010afb0 con 0x7f30a01036f0 2024-02-19T19:02:41.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.367+0000 7f3096ffd700 1 -- 172.21.15.88:0/1419545178 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f309000fab0 con 0x7f30a01036f0 2024-02-19T19:02:41.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.367+0000 7f3096ffd700 1 --2- 172.21.15.88:0/1419545178 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f308c06d660 0x7f308c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:41.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.367+0000 7f3096ffd700 1 -- 172.21.15.88:0/1419545178 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f309008e5c0 con 0x7f30a01036f0 2024-02-19T19:02:41.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.367+0000 7f30a5c95700 1 --2- 172.21.15.88:0/1419545178 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f308c06d660 0x7f308c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:41.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.368+0000 7f30a5c95700 1 --2- 172.21.15.88:0/1419545178 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f308c06d660 0x7f308c06fb10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f30a0199210 tx=0x7f30980094f0 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:41.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.376+0000 7f3096ffd700 1 -- 172.21.15.88:0/1419545178 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f3090058100 con 0x7f30a01036f0 2024-02-19T19:02:41.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.615+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1419545178 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30a019df10 con 0x7f308c06d660 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: pgmap v37: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.635 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.636 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.636 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.636 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.636 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.617+0000 7f3096ffd700 1 -- 172.21.15.88:0/1419545178 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f30a019df10 con 0x7f308c06d660 2024-02-19T19:02:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.619+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1419545178 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f308c06d660 msgr2=0x7f308c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.619+0000 7f30a7ef9700 1 --2- 172.21.15.88:0/1419545178 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f308c06d660 0x7f308c06fb10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f30a0199210 tx=0x7f30980094f0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.619+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1419545178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30a01036f0 msgr2=0x7f30a0198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.619+0000 7f30a7ef9700 1 --2- 172.21.15.88:0/1419545178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30a01036f0 0x7f30a0198270 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3090006010 tx=0x7f309000be20 comp rx=0 tx=0).stop 2024-02-19T19:02:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.619+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1419545178 shutdown_connections 2024-02-19T19:02:41.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.619+0000 7f30a7ef9700 1 --2- 172.21.15.88:0/1419545178 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f308c06d660 0x7f308c06fb10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.619+0000 7f30a7ef9700 1 --2- 172.21.15.88:0/1419545178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30a0102450 0x7f30a0197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.619+0000 7f30a7ef9700 1 --2- 172.21.15.88:0/1419545178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30a01036f0 0x7f30a0198270 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.619+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1419545178 >> 172.21.15.88:0/1419545178 conn(0x7f30a00fdac0 msgr2=0x7f30a0106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:41.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.619+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1419545178 shutdown_connections 2024-02-19T19:02:41.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.619+0000 7f30a7ef9700 1 -- 172.21.15.88:0/1419545178 wait complete. 2024-02-19T19:02:41.649 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:02:41.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: pgmap v37: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:41.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:41.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:41.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.738+0000 7f7f87328700 1 -- 172.21.15.88:0/429209355 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f801018a0 msgr2=0x7f7f80101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:41.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.738+0000 7f7f87328700 1 --2- 172.21.15.88:0/429209355 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f801018a0 0x7f7f80101cb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009910 tx=0x7f7f70009c20 comp rx=0 tx=0).stop 2024-02-19T19:02:41.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.738+0000 7f7f87328700 1 -- 172.21.15.88:0/429209355 shutdown_connections 2024-02-19T19:02:41.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.738+0000 7f7f87328700 1 --2- 172.21.15.88:0/429209355 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f801021f0 0x7f7f800fe320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.738+0000 7f7f87328700 1 --2- 172.21.15.88:0/429209355 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f801018a0 0x7f7f80101cb0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.738+0000 7f7f87328700 1 -- 172.21.15.88:0/429209355 >> 172.21.15.88:0/429209355 conn(0x7f7f800f9f10 msgr2=0x7f7f800fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:41.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.738+0000 7f7f87328700 1 -- 172.21.15.88:0/429209355 shutdown_connections 2024-02-19T19:02:41.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.738+0000 7f7f87328700 1 -- 172.21.15.88:0/429209355 wait complete. 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.739+0000 7f7f87328700 1 Processor -- start 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.739+0000 7f7f87328700 1 -- start start 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.739+0000 7f7f87328700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f801018a0 0x7f7f80195b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.739+0000 7f7f87328700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f801021f0 0x7f7f801960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.739+0000 7f7f87328700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f801966e0 con 0x7f7f801018a0 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.739+0000 7f7f87328700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f80196820 con 0x7f7f801021f0 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.739+0000 7f7f850c4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f801018a0 0x7f7f80195b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.739+0000 7f7f850c4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f801018a0 0x7f7f80195b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48280/0 (socket says 172.21.15.88:48280) 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.739+0000 7f7f850c4700 1 -- 172.21.15.88:0/2519963618 learned_addr learned my addr 172.21.15.88:0/2519963618 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.739+0000 7f7f848c3700 1 --2- 172.21.15.88:0/2519963618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f801021f0 0x7f7f801960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.740+0000 7f7f850c4700 1 -- 172.21.15.88:0/2519963618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f801021f0 msgr2=0x7f7f801960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.740+0000 7f7f850c4700 1 --2- 172.21.15.88:0/2519963618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f801021f0 0x7f7f801960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.740+0000 7f7f850c4700 1 -- 172.21.15.88:0/2519963618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700095f0 con 0x7f7f801018a0 2024-02-19T19:02:41.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.740+0000 7f7f850c4700 1 --2- 172.21.15.88:0/2519963618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f801018a0 0x7f7f80195b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000bd70 tx=0x7f7f7000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:41.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.740+0000 7f7f767fc700 1 -- 172.21.15.88:0/2519963618 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f7001d070 con 0x7f7f801018a0 2024-02-19T19:02:41.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.740+0000 7f7f767fc700 1 -- 172.21.15.88:0/2519963618 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7f70005940 con 0x7f7f801018a0 2024-02-19T19:02:41.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.740+0000 7f7f87328700 1 -- 172.21.15.88:0/2519963618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8019b270 con 0x7f7f801018a0 2024-02-19T19:02:41.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.740+0000 7f7f767fc700 1 -- 172.21.15.88:0/2519963618 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f70021430 con 0x7f7f801018a0 2024-02-19T19:02:41.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.740+0000 7f7f87328700 1 -- 172.21.15.88:0/2519963618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8019b760 con 0x7f7f801018a0 2024-02-19T19:02:41.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.741+0000 7f7f87328700 1 -- 172.21.15.88:0/2519963618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f80108db0 con 0x7f7f801018a0 2024-02-19T19:02:41.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.742+0000 7f7f767fc700 1 -- 172.21.15.88:0/2519963618 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f7f7000f860 con 0x7f7f801018a0 2024-02-19T19:02:41.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.742+0000 7f7f767fc700 1 --2- 172.21.15.88:0/2519963618 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7f6c06d660 0x7f7f6c06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:41.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.742+0000 7f7f848c3700 1 --2- 172.21.15.88:0/2519963618 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7f6c06d660 0x7f7f6c06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:41.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.742+0000 7f7f767fc700 1 -- 172.21.15.88:0/2519963618 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7f7008e520 con 0x7f7f801018a0 2024-02-19T19:02:41.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.743+0000 7f7f848c3700 1 --2- 172.21.15.88:0/2519963618 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7f6c06d660 0x7f7f6c06fb10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7f80102620 tx=0x7f7f7c009190 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:41.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.747+0000 7f7f767fc700 1 -- 172.21.15.88:0/2519963618 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f7f7005bb10 con 0x7f7f801018a0 2024-02-19T19:02:41.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.970+0000 7f7f87328700 1 -- 172.21.15.88:0/2519963618 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f8019bdf0 con 0x7f7f6c06d660 2024-02-19T19:02:41.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.971+0000 7f7f767fc700 1 -- 172.21.15.88:0/2519963618 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f7f8019bdf0 con 0x7f7f6c06d660 2024-02-19T19:02:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.974+0000 7f7f87328700 1 -- 172.21.15.88:0/2519963618 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7f6c06d660 msgr2=0x7f7f6c06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.974+0000 7f7f87328700 1 --2- 172.21.15.88:0/2519963618 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7f6c06d660 0x7f7f6c06fb10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f7f80102620 tx=0x7f7f7c009190 comp rx=0 tx=0).stop 2024-02-19T19:02:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.974+0000 7f7f87328700 1 -- 172.21.15.88:0/2519963618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f801018a0 msgr2=0x7f7f80195b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.974+0000 7f7f87328700 1 --2- 172.21.15.88:0/2519963618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f801018a0 0x7f7f80195b80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7f7000bd70 tx=0x7f7f7000bda0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.974+0000 7f7f87328700 1 -- 172.21.15.88:0/2519963618 shutdown_connections 2024-02-19T19:02:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.974+0000 7f7f87328700 1 --2- 172.21.15.88:0/2519963618 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7f6c06d660 0x7f7f6c06fb10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.974+0000 7f7f87328700 1 --2- 172.21.15.88:0/2519963618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f801018a0 0x7f7f80195b80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.974+0000 7f7f87328700 1 --2- 172.21.15.88:0/2519963618 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f801021f0 0x7f7f801960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.974+0000 7f7f87328700 1 -- 172.21.15.88:0/2519963618 >> 172.21.15.88:0/2519963618 conn(0x7f7f800f9f10 msgr2=0x7f7f80104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.974+0000 7f7f87328700 1 -- 172.21.15.88:0/2519963618 shutdown_connections 2024-02-19T19:02:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:41.974+0000 7f7f87328700 1 -- 172.21.15.88:0/2519963618 wait complete. 2024-02-19T19:02:42.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.082+0000 7f49e9ed5700 1 -- 172.21.15.88:0/3942258209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e4103680 msgr2=0x7f49e4105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:42.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.082+0000 7f49e9ed5700 1 --2- 172.21.15.88:0/3942258209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e4103680 0x7f49e4105a60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009910 tx=0x7f49cc009c20 comp rx=0 tx=0).stop 2024-02-19T19:02:42.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.082+0000 7f49e9ed5700 1 -- 172.21.15.88:0/3942258209 shutdown_connections 2024-02-19T19:02:42.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.082+0000 7f49e9ed5700 1 --2- 172.21.15.88:0/3942258209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e4103680 0x7f49e4105a60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.082+0000 7f49e9ed5700 1 --2- 172.21.15.88:0/3942258209 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e4100d60 0x7f49e4103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.082+0000 7f49e9ed5700 1 -- 172.21.15.88:0/3942258209 >> 172.21.15.88:0/3942258209 conn(0x7f49e40fa780 msgr2=0x7f49e40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:42.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.082+0000 7f49e9ed5700 1 -- 172.21.15.88:0/3942258209 shutdown_connections 2024-02-19T19:02:42.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.082+0000 7f49e9ed5700 1 -- 172.21.15.88:0/3942258209 wait complete. 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.082+0000 7f49e9ed5700 1 Processor -- start 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.082+0000 7f49e9ed5700 1 -- start start 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e9ed5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e4100d60 0x7f49e41006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e9ed5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e4103680 0x7f49e40fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e9ed5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e40ff250 con 0x7f49e4100d60 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e9ed5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e40ff390 con 0x7f49e4103680 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e4100d60 0x7f49e41006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e4100d60 0x7f49e41006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48300/0 (socket says 172.21.15.88:48300) 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e37fe700 1 -- 172.21.15.88:0/4076401538 learned_addr learned my addr 172.21.15.88:0/4076401538 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49daffd700 1 --2- 172.21.15.88:0/4076401538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e4103680 0x7f49e40fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e37fe700 1 -- 172.21.15.88:0/4076401538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e4103680 msgr2=0x7f49e40fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e37fe700 1 --2- 172.21.15.88:0/4076401538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e4103680 0x7f49e40fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e37fe700 1 -- 172.21.15.88:0/4076401538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0095f0 con 0x7f49e4100d60 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e37fe700 1 --2- 172.21.15.88:0/4076401538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e4100d60 0x7f49e41006c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f49d400d900 tx=0x7f49d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:42.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e17fa700 1 -- 172.21.15.88:0/4076401538 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d4009900 con 0x7f49e4100d60 2024-02-19T19:02:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e17fa700 1 -- 172.21.15.88:0/4076401538 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f49d4010460 con 0x7f49e4100d60 2024-02-19T19:02:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e9ed5700 1 -- 172.21.15.88:0/4076401538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e40ff670 con 0x7f49e4100d60 2024-02-19T19:02:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e17fa700 1 -- 172.21.15.88:0/4076401538 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d400f5f0 con 0x7f49e4100d60 2024-02-19T19:02:42.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.083+0000 7f49e9ed5700 1 -- 172.21.15.88:0/4076401538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e419db40 con 0x7f49e4100d60 2024-02-19T19:02:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.084+0000 7f49e9ed5700 1 -- 172.21.15.88:0/4076401538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e418fbe0 con 0x7f49e4100d60 2024-02-19T19:02:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.085+0000 7f49e17fa700 1 -- 172.21.15.88:0/4076401538 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f49d400fa80 con 0x7f49e4100d60 2024-02-19T19:02:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.085+0000 7f49e17fa700 1 --2- 172.21.15.88:0/4076401538 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f49d006d660 0x7f49d006fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.085+0000 7f49e17fa700 1 -- 172.21.15.88:0/4076401538 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f49d408d0b0 con 0x7f49e4100d60 2024-02-19T19:02:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.085+0000 7f49daffd700 1 --2- 172.21.15.88:0/4076401538 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f49d006d660 0x7f49d006fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:42.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.086+0000 7f49daffd700 1 --2- 172.21.15.88:0/4076401538 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f49d006d660 0x7f49d006fb10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f49cc005b40 tx=0x7f49cc005a90 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:42.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.090+0000 7f49e17fa700 1 -- 172.21.15.88:0/4076401538 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f49d405a710 con 0x7f49e4100d60 2024-02-19T19:02:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.308+0000 7f49e9ed5700 1 -- 172.21.15.88:0/4076401538 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f49e4061190 con 0x7f49d006d660 2024-02-19T19:02:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.319+0000 7f49e17fa700 1 -- 172.21.15.88:0/4076401538 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f49e4061190 con 0x7f49d006d660 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (8m) 6s ago 10m 24.7M - 0.25.0 c8568f914cd2 d6dd804f52d8 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (10m) 6s ago 10m 7734k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (8m) 26s ago 8m 7608k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (10m) 6s ago 10m 6665k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (8m) 26s ago 8m 6656k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (7m) 6s ago 9m 85.9M - 9.4.7 954c08fa6188 54907dc6edb4 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (4m) 6s ago 4m 14.0M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (3m) 6s ago 3m 26.4M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (4m) 26s ago 4m 23.1M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (3m) 26s ago 3m 17.6M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (81s) 6s ago 11m 496M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (47s) 26s ago 8m 449M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:02:42.339 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (12m) 6s ago 12m 53.9M 2048M 18.2.0 dc2bc1663786 3ea5108455bc 2024-02-19T19:02:42.340 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (8m) 26s ago 8m 43.1M 2048M 18.2.0 dc2bc1663786 ba1c249c6017 2024-02-19T19:02:42.340 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (35s) 6s ago 10m 8190k - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:02:42.340 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (30s) 26s ago 8m 3573k - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:02:42.340 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (7m) 6s ago 7m 51.7M 1216M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:02:42.340 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (7m) 6s ago 7m 49.4M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:02:42.340 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (6m) 6s ago 6m 49.0M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:02:42.340 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (6m) 26s ago 6m 50.5M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:02:42.340 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (6m) 26s ago 6m 49.1M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:02:42.340 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (5m) 26s ago 5m 47.2M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:02:42.340 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (12s) 6s ago 9m 44.3M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:02:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.321+0000 7f49e9ed5700 1 -- 172.21.15.88:0/4076401538 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f49d006d660 msgr2=0x7f49d006fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.321+0000 7f49e9ed5700 1 --2- 172.21.15.88:0/4076401538 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f49d006d660 0x7f49d006fb10 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f49cc005b40 tx=0x7f49cc005a90 comp rx=0 tx=0).stop 2024-02-19T19:02:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.321+0000 7f49e9ed5700 1 -- 172.21.15.88:0/4076401538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e4100d60 msgr2=0x7f49e41006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.321+0000 7f49e9ed5700 1 --2- 172.21.15.88:0/4076401538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e4100d60 0x7f49e41006c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f49d400d900 tx=0x7f49d400dcc0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.322+0000 7f49e9ed5700 1 -- 172.21.15.88:0/4076401538 shutdown_connections 2024-02-19T19:02:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.322+0000 7f49e9ed5700 1 --2- 172.21.15.88:0/4076401538 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f49d006d660 0x7f49d006fb10 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.322+0000 7f49e9ed5700 1 --2- 172.21.15.88:0/4076401538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e4100d60 0x7f49e41006c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.322+0000 7f49e9ed5700 1 --2- 172.21.15.88:0/4076401538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e4103680 0x7f49e40fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.322+0000 7f49e9ed5700 1 -- 172.21.15.88:0/4076401538 >> 172.21.15.88:0/4076401538 conn(0x7f49e40fa780 msgr2=0x7f49e40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.322+0000 7f49e9ed5700 1 -- 172.21.15.88:0/4076401538 shutdown_connections 2024-02-19T19:02:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.322+0000 7f49e9ed5700 1 -- 172.21.15.88:0/4076401538 wait complete. 2024-02-19T19:02:42.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.426+0000 7f421c194700 1 -- 172.21.15.88:0/2852971825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42140ff130 msgr2=0x7f42140ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:42.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.426+0000 7f421c194700 1 --2- 172.21.15.88:0/2852971825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42140ff130 0x7f42140ff540 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f420c00b0a0 tx=0x7f420c00b3b0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.426+0000 7f421c194700 1 -- 172.21.15.88:0/2852971825 shutdown_connections 2024-02-19T19:02:42.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.426+0000 7f421c194700 1 --2- 172.21.15.88:0/2852971825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42141003d0 0x7f4214100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.426+0000 7f421c194700 1 --2- 172.21.15.88:0/2852971825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42140ff130 0x7f42140ff540 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.426+0000 7f421c194700 1 -- 172.21.15.88:0/2852971825 >> 172.21.15.88:0/2852971825 conn(0x7f42140fa780 msgr2=0x7f42140fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.426+0000 7f421c194700 1 -- 172.21.15.88:0/2852971825 shutdown_connections 2024-02-19T19:02:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.426+0000 7f421c194700 1 -- 172.21.15.88:0/2852971825 wait complete. 2024-02-19T19:02:42.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f421c194700 1 Processor -- start 2024-02-19T19:02:42.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f421c194700 1 -- start start 2024-02-19T19:02:42.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f421c194700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42140ff130 0x7f4214194a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:42.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f421c194700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42141003d0 0x7f4214194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:42.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f421c194700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42141955d0 con 0x7f42140ff130 2024-02-19T19:02:42.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f421c194700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214195710 con 0x7f42141003d0 2024-02-19T19:02:42.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f4219f30700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42140ff130 0x7f4214194a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:42.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f4219f30700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42140ff130 0x7f4214194a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48322/0 (socket says 172.21.15.88:48322) 2024-02-19T19:02:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f4219f30700 1 -- 172.21.15.88:0/1092736363 learned_addr learned my addr 172.21.15.88:0/1092736363 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:02:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f4219f30700 1 -- 172.21.15.88:0/1092736363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42141003d0 msgr2=0x7f4214194fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f4219f30700 1 --2- 172.21.15.88:0/1092736363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42141003d0 0x7f4214194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f4219f30700 1 -- 172.21.15.88:0/1092736363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c009d00 con 0x7f42140ff130 2024-02-19T19:02:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f4219f30700 1 --2- 172.21.15.88:0/1092736363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42140ff130 0x7f4214194a70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f420c0077b0 tx=0x7f420c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.427+0000 7f420affd700 1 -- 172.21.15.88:0/1092736363 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f420c00e040 con 0x7f42140ff130 2024-02-19T19:02:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.428+0000 7f420affd700 1 -- 172.21.15.88:0/1092736363 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f420c00ce60 con 0x7f42140ff130 2024-02-19T19:02:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.428+0000 7f421c194700 1 -- 172.21.15.88:0/1092736363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f421419a160 con 0x7f42140ff130 2024-02-19T19:02:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.428+0000 7f421c194700 1 -- 172.21.15.88:0/1092736363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f421419a650 con 0x7f42140ff130 2024-02-19T19:02:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.428+0000 7f420affd700 1 -- 172.21.15.88:0/1092736363 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f420c01daf0 con 0x7f42140ff130 2024-02-19T19:02:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.429+0000 7f421c194700 1 -- 172.21.15.88:0/1092736363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4214107c90 con 0x7f42140ff130 2024-02-19T19:02:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.429+0000 7f420affd700 1 -- 172.21.15.88:0/1092736363 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f420c004250 con 0x7f42140ff130 2024-02-19T19:02:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.429+0000 7f420affd700 1 --2- 172.21.15.88:0/1092736363 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f420006d660 0x7f420006fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.429+0000 7f420affd700 1 -- 172.21.15.88:0/1092736363 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f420c08ecf0 con 0x7f42140ff130 2024-02-19T19:02:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.429+0000 7f421972f700 1 --2- 172.21.15.88:0/1092736363 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f420006d660 0x7f420006fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.430+0000 7f421972f700 1 --2- 172.21.15.88:0/1092736363 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f420006d660 0x7f420006fb10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f42040097b0 tx=0x7f4204006d20 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:42.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.434+0000 7f420affd700 1 -- 172.21.15.88:0/1092736363 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f420c05c2e0 con 0x7f42140ff130 2024-02-19T19:02:42.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:42 smithi132 ceph-mon[37969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:02:42.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:42 smithi088 ceph-mon[29272]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-19T19:02:42.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.736+0000 7f421c194700 1 -- 172.21.15.88:0/1092736363 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f421419ace0 con 0x7f42140ff130 2024-02-19T19:02:42.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.736+0000 7f420affd700 1 -- 172.21.15.88:0/1092736363 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f420c05be70 con 0x7f42140ff130 2024-02-19T19:02:42.756 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:02:42.756 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:02:42.756 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-19T19:02:42.756 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:02:42.756 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:02:42.756 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:02:42.756 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:02:42.756 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:02:42.756 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:02:42.757 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:02:42.757 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:02:42.757 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:02:42.757 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:02:42.757 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:02:42.757 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-19T19:02:42.757 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:02:42.757 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:02:42.757 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:02:42.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.739+0000 7f421c194700 1 -- 172.21.15.88:0/1092736363 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f420006d660 msgr2=0x7f420006fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:42.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.739+0000 7f421c194700 1 --2- 172.21.15.88:0/1092736363 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f420006d660 0x7f420006fb10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f42040097b0 tx=0x7f4204006d20 comp rx=0 tx=0).stop 2024-02-19T19:02:42.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.739+0000 7f421c194700 1 -- 172.21.15.88:0/1092736363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42140ff130 msgr2=0x7f4214194a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:42.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.739+0000 7f421c194700 1 --2- 172.21.15.88:0/1092736363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42140ff130 0x7f4214194a70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f420c0077b0 tx=0x7f420c003ce0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.739+0000 7f421c194700 1 -- 172.21.15.88:0/1092736363 shutdown_connections 2024-02-19T19:02:42.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.739+0000 7f421c194700 1 --2- 172.21.15.88:0/1092736363 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f420006d660 0x7f420006fb10 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.739+0000 7f421c194700 1 --2- 172.21.15.88:0/1092736363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42140ff130 0x7f4214194a70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.739+0000 7f421c194700 1 --2- 172.21.15.88:0/1092736363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42141003d0 0x7f4214194fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.739+0000 7f421c194700 1 -- 172.21.15.88:0/1092736363 >> 172.21.15.88:0/1092736363 conn(0x7f42140fa780 msgr2=0x7f4214103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:42.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.739+0000 7f421c194700 1 -- 172.21.15.88:0/1092736363 shutdown_connections 2024-02-19T19:02:42.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.739+0000 7f421c194700 1 -- 172.21.15.88:0/1092736363 wait complete. 2024-02-19T19:02:42.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.845+0000 7fcff2ba4700 1 -- 172.21.15.88:0/4272560903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec0ff130 msgr2=0x7fcfec0ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:42.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.845+0000 7fcff2ba4700 1 --2- 172.21.15.88:0/4272560903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec0ff130 0x7fcfec0ff540 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009910 tx=0x7fcfe0009c20 comp rx=0 tx=0).stop 2024-02-19T19:02:42.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.845+0000 7fcff2ba4700 1 -- 172.21.15.88:0/4272560903 shutdown_connections 2024-02-19T19:02:42.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.845+0000 7fcff2ba4700 1 --2- 172.21.15.88:0/4272560903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfec1003d0 0x7fcfec100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.845+0000 7fcff2ba4700 1 --2- 172.21.15.88:0/4272560903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec0ff130 0x7fcfec0ff540 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.845+0000 7fcff2ba4700 1 -- 172.21.15.88:0/4272560903 >> 172.21.15.88:0/4272560903 conn(0x7fcfec0fa780 msgr2=0x7fcfec0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:42.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.845+0000 7fcff2ba4700 1 -- 172.21.15.88:0/4272560903 shutdown_connections 2024-02-19T19:02:42.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.845+0000 7fcff2ba4700 1 -- 172.21.15.88:0/4272560903 wait complete. 2024-02-19T19:02:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff2ba4700 1 Processor -- start 2024-02-19T19:02:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff2ba4700 1 -- start start 2024-02-19T19:02:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff2ba4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec0ff130 0x7fcfec0690a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff2ba4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfec1003d0 0x7fcfec19a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:42.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff2ba4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec19ad50 con 0x7fcfec0ff130 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff2ba4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec19ae90 con 0x7fcfec1003d0 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff0940700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec0ff130 0x7fcfec0690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff0940700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec0ff130 0x7fcfec0690a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48342/0 (socket says 172.21.15.88:48342) 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff0940700 1 -- 172.21.15.88:0/3049383513 learned_addr learned my addr 172.21.15.88:0/3049383513 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff0940700 1 -- 172.21.15.88:0/3049383513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfec1003d0 msgr2=0x7fcfec19a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff0940700 1 --2- 172.21.15.88:0/3049383513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfec1003d0 0x7fcfec19a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff0940700 1 -- 172.21.15.88:0/3049383513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe00095f0 con 0x7fcfec0ff130 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.846+0000 7fcff0940700 1 --2- 172.21.15.88:0/3049383513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec0ff130 0x7fcfec0690a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000bd70 tx=0x7fcfe000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.847+0000 7fcfe9ffb700 1 -- 172.21.15.88:0/3049383513 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfe001d070 con 0x7fcfec0ff130 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.847+0000 7fcfe9ffb700 1 -- 172.21.15.88:0/3049383513 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcfe0022470 con 0x7fcfec0ff130 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.847+0000 7fcff2ba4700 1 -- 172.21.15.88:0/3049383513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec19b110 con 0x7fcfec0ff130 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.847+0000 7fcfe9ffb700 1 -- 172.21.15.88:0/3049383513 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfe000f670 con 0x7fcfec0ff130 2024-02-19T19:02:42.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.847+0000 7fcff2ba4700 1 -- 172.21.15.88:0/3049383513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfec19b600 con 0x7fcfec0ff130 2024-02-19T19:02:42.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.848+0000 7fcff2ba4700 1 -- 172.21.15.88:0/3049383513 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfec107c10 con 0x7fcfec0ff130 2024-02-19T19:02:42.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.848+0000 7fcfe9ffb700 1 -- 172.21.15.88:0/3049383513 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 31) v1 ==== 91448+0+0 (secure 0 0 0) 0x7fcfe0022a50 con 0x7fcfec0ff130 2024-02-19T19:02:42.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.849+0000 7fcfe9ffb700 1 --2- 172.21.15.88:0/3049383513 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcfdc06d660 0x7fcfdc06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:02:42.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.849+0000 7fcfe9ffb700 1 -- 172.21.15.88:0/3049383513 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcfe008e990 con 0x7fcfec0ff130 2024-02-19T19:02:42.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.849+0000 7fcfebfff700 1 --2- 172.21.15.88:0/3049383513 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcfdc06d660 0x7fcfdc06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:02:42.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.849+0000 7fcfebfff700 1 --2- 172.21.15.88:0/3049383513 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcfdc06d660 0x7fcfdc06fb10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8005fd0 tx=0x7fcfd8005d00 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:02:42.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:42.853+0000 7fcfe9ffb700 1 -- 172.21.15.88:0/3049383513 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fcfe005c1d0 con 0x7fcfec0ff130 2024-02-19T19:02:43.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.082+0000 7fcff2ba4700 1 -- 172.21.15.88:0/3049383513 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfec104d20 con 0x7fcfdc06d660 2024-02-19T19:02:43.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.083+0000 7fcfe9ffb700 1 -- 172.21.15.88:0/3049383513 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fcfec104d20 con 0x7fcfdc06d660 2024-02-19T19:02:43.102 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:02:43.102 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:02:43.102 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:02:43.102 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-19T19:02:43.102 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:02:43.102 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:02:43.102 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:02:43.102 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "2/2 daemons upgraded", 2024-02-19T19:02:43.102 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading alertmanager daemons", 2024-02-19T19:02:43.103 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:02:43.103 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:02:43.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.085+0000 7fcff2ba4700 1 -- 172.21.15.88:0/3049383513 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcfdc06d660 msgr2=0x7fcfdc06fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:43.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.085+0000 7fcff2ba4700 1 --2- 172.21.15.88:0/3049383513 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcfdc06d660 0x7fcfdc06fb10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8005fd0 tx=0x7fcfd8005d00 comp rx=0 tx=0).stop 2024-02-19T19:02:43.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.085+0000 7fcff2ba4700 1 -- 172.21.15.88:0/3049383513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec0ff130 msgr2=0x7fcfec0690a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:02:43.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.085+0000 7fcff2ba4700 1 --2- 172.21.15.88:0/3049383513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec0ff130 0x7fcfec0690a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000bd70 tx=0x7fcfe000be50 comp rx=0 tx=0).stop 2024-02-19T19:02:43.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.085+0000 7fcff2ba4700 1 -- 172.21.15.88:0/3049383513 shutdown_connections 2024-02-19T19:02:43.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.085+0000 7fcff2ba4700 1 --2- 172.21.15.88:0/3049383513 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcfdc06d660 0x7fcfdc06fb10 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:43.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.085+0000 7fcff2ba4700 1 --2- 172.21.15.88:0/3049383513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfec0ff130 0x7fcfec0690a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:43.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.085+0000 7fcff2ba4700 1 --2- 172.21.15.88:0/3049383513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcfec1003d0 0x7fcfec19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:02:43.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.085+0000 7fcff2ba4700 1 -- 172.21.15.88:0/3049383513 >> 172.21.15.88:0/3049383513 conn(0x7fcfec0fa780 msgr2=0x7fcfec103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:02:43.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.085+0000 7fcff2ba4700 1 -- 172.21.15.88:0/3049383513 shutdown_connections 2024-02-19T19:02:43.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:02:43.085+0000 7fcff2ba4700 1 -- 172.21.15.88:0/3049383513 wait complete. 2024-02-19T19:02:43.703 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:43 smithi088 ceph-mon[29272]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:43.704 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:43 smithi088 ceph-mon[29272]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:43.704 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:43 smithi088 ceph-mon[29272]: pgmap v38: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:43.704 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:43 smithi088 ceph-mon[29272]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:43.704 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:43 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/1092736363' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:43.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:43 smithi132 ceph-mon[37969]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:43.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:43 smithi132 ceph-mon[37969]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:43.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:43 smithi132 ceph-mon[37969]: pgmap v38: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:43.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:43 smithi132 ceph-mon[37969]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:43.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:43 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/1092736363' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:02:44.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:44 smithi088 ceph-mon[29272]: Upgrade: Updating alertmanager.smithi088 2024-02-19T19:02:44.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:44 smithi088 ceph-mon[29272]: Deploying daemon alertmanager.smithi088 on smithi088 2024-02-19T19:02:44.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:44 smithi088 ceph-mon[29272]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:44.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:44 smithi132 ceph-mon[37969]: Upgrade: Updating alertmanager.smithi088 2024-02-19T19:02:44.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:44 smithi132 ceph-mon[37969]: Deploying daemon alertmanager.smithi088 on smithi088 2024-02-19T19:02:44.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:44 smithi132 ceph-mon[37969]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:02:45.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:45 smithi088 ceph-mon[29272]: pgmap v39: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:45.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:45 smithi132 ceph-mon[37969]: pgmap v39: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:47.694 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:47 smithi088 ceph-mon[29272]: pgmap v40: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:47.694 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:47 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:02:47.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:47 smithi132 ceph-mon[37969]: pgmap v40: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:47.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:47 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:02:49.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:49 smithi088 ceph-mon[29272]: pgmap v41: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:49.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:49 smithi132 ceph-mon[37969]: pgmap v41: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:51.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:50 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:51.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:50 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:51.099 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:50 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:51.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:50 smithi088 ceph-mon[29272]: pgmap v42: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:51.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:50 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:51.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:50 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:51.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:50 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:02:51.251 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:50 smithi132 ceph-mon[37969]: pgmap v42: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:53.310 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:53 smithi088 ceph-mon[29272]: pgmap v43: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:53.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:53 smithi132 ceph-mon[37969]: pgmap v43: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:55.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:55 smithi088 ceph-mon[29272]: pgmap v44: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:55.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:55 smithi132 ceph-mon[37969]: pgmap v44: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:56.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:56 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:56.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:56 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:56.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:56 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:56.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:56 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:57.725 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:57 smithi088 ceph-mon[29272]: pgmap v45: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:57.725 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:57 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:57.725 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:57 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:58.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:57 smithi132 ceph-mon[37969]: pgmap v45: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:58.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:57 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:58.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:57 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:02:59.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:02:59 smithi132 ceph-mon[37969]: pgmap v46: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:02:59.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:02:59 smithi088 ceph-mon[29272]: pgmap v46: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:03:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:03:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-19T19:03:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-19T19:03:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:03:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:01 smithi088 ceph-mon[29272]: pgmap v47: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:01.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:01 smithi132 ceph-mon[37969]: pgmap v47: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:03.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:03 smithi088 ceph-mon[29272]: Upgrade: Updating grafana.smithi088 2024-02-19T19:03:03.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:03 smithi088 ceph-mon[29272]: Deploying daemon grafana.smithi088 on smithi088 2024-02-19T19:03:03.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:03 smithi088 ceph-mon[29272]: pgmap v48: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:03.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:03 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:03:03.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:03 smithi132 ceph-mon[37969]: Upgrade: Updating grafana.smithi088 2024-02-19T19:03:03.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:03 smithi132 ceph-mon[37969]: Deploying daemon grafana.smithi088 on smithi088 2024-02-19T19:03:03.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:03 smithi132 ceph-mon[37969]: pgmap v48: 65 pgs: 65 active+clean; 593 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:03.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:03 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:03:05.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:05 smithi088 ceph-mon[29272]: mgrmap e32: smithi088.gyviqq(active, since 91s), standbys: smithi132.psllsa 2024-02-19T19:03:05.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:05 smithi088 ceph-mon[29272]: pgmap v49: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:05.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:05 smithi132 ceph-mon[37969]: mgrmap e32: smithi088.gyviqq(active, since 91s), standbys: smithi132.psllsa 2024-02-19T19:03:05.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:05 smithi132 ceph-mon[37969]: pgmap v49: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:07.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:07 smithi088 ceph-mon[29272]: pgmap v50: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:07.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:07 smithi132 ceph-mon[37969]: pgmap v50: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:09.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:09 smithi088 ceph-mon[29272]: pgmap v51: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:09.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:09 smithi132 ceph-mon[37969]: pgmap v51: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:11.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:11 smithi088 ceph-mon[29272]: pgmap v52: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:11.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:11 smithi132 ceph-mon[37969]: pgmap v52: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:13.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.201+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/2873939895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2e0103350 msgr2=0x7fe2e0105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:13.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.201+0000 7fe2e5cdd700 1 --2- 172.21.15.88:0/2873939895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2e0103350 0x7fe2e0105790 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4009910 tx=0x7fe2d4009c20 comp rx=0 tx=0).stop 2024-02-19T19:03:13.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.201+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/2873939895 shutdown_connections 2024-02-19T19:03:13.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.201+0000 7fe2e5cdd700 1 --2- 172.21.15.88:0/2873939895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2e0103350 0x7fe2e0105790 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.201+0000 7fe2e5cdd700 1 --2- 172.21.15.88:0/2873939895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2e0069230 0x7fe2e0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.201+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/2873939895 >> 172.21.15.88:0/2873939895 conn(0x7fe2e00fa780 msgr2=0x7fe2e00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:13.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.201+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/2873939895 shutdown_connections 2024-02-19T19:03:13.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.201+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/2873939895 wait complete. 2024-02-19T19:03:13.222 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:13 smithi088 ceph-mon[29272]: pgmap v53: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:13.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.202+0000 7fe2e5cdd700 1 Processor -- start 2024-02-19T19:03:13.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.202+0000 7fe2e5cdd700 1 -- start start 2024-02-19T19:03:13.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.202+0000 7fe2e5cdd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2e0069230 0x7fe2e0071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:13.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.202+0000 7fe2e5cdd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2e0103350 0x7fe2e0072330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:13.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.202+0000 7fe2e5cdd700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e0072950 con 0x7fe2e0069230 2024-02-19T19:03:13.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.202+0000 7fe2e5cdd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e01a3850 con 0x7fe2e0103350 2024-02-19T19:03:13.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.202+0000 7fe2df7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2e0069230 0x7fe2e0071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:13.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.202+0000 7fe2df7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2e0069230 0x7fe2e0071df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44128/0 (socket says 172.21.15.88:44128) 2024-02-19T19:03:13.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.202+0000 7fe2df7fe700 1 -- 172.21.15.88:0/3810885272 learned_addr learned my addr 172.21.15.88:0/3810885272 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:03:13.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.203+0000 7fe2deffd700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2e0103350 0x7fe2e0072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:13.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.203+0000 7fe2df7fe700 1 -- 172.21.15.88:0/3810885272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2e0103350 msgr2=0x7fe2e0072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.203+0000 7fe2df7fe700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2e0103350 0x7fe2e0072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.203+0000 7fe2df7fe700 1 -- 172.21.15.88:0/3810885272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d40095f0 con 0x7fe2e0069230 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.203+0000 7fe2deffd700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2e0103350 0x7fe2e0072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.203+0000 7fe2df7fe700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2e0069230 0x7fe2e0071df0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000c8f0 tx=0x7fe2d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.203+0000 7fe2dcff9700 1 -- 172.21.15.88:0/3810885272 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2d0007ad0 con 0x7fe2e0069230 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.203+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/3810885272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e01a3a50 con 0x7fe2e0069230 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.203+0000 7fe2dcff9700 1 -- 172.21.15.88:0/3810885272 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe2d000ce90 con 0x7fe2e0069230 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.203+0000 7fe2dcff9700 1 -- 172.21.15.88:0/3810885272 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2d0018630 con 0x7fe2e0069230 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.203+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/3810885272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e01a3f50 con 0x7fe2e0069230 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.204+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/3810885272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e006bf10 con 0x7fe2e0069230 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.204+0000 7fe2dcff9700 1 -- 172.21.15.88:0/3810885272 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fe2d0007c30 con 0x7fe2e0069230 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.205+0000 7fe2dcff9700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe2c806d520 0x7fe2c806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.205+0000 7fe2dcff9700 1 -- 172.21.15.88:0/3810885272 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe2d008cee0 con 0x7fe2e0069230 2024-02-19T19:03:13.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.205+0000 7fe2deffd700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe2c806d520 0x7fe2c806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:13.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.205+0000 7fe2deffd700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe2c806d520 0x7fe2c806f9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005090 tx=0x7fe2d400b540 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:13.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.210+0000 7fe2dcff9700 1 -- 172.21.15.88:0/3810885272 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fe2d0056bc0 con 0x7fe2e0069230 2024-02-19T19:03:13.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.450+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/3810885272 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2e0061190 con 0x7fe2c806d520 2024-02-19T19:03:13.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.452+0000 7fe2dcff9700 1 -- 172.21.15.88:0/3810885272 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fe2e0061190 con 0x7fe2c806d520 2024-02-19T19:03:13.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.454+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/3810885272 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe2c806d520 msgr2=0x7fe2c806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:13.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.454+0000 7fe2e5cdd700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe2c806d520 0x7fe2c806f9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe2d4005090 tx=0x7fe2d400b540 comp rx=0 tx=0).stop 2024-02-19T19:03:13.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.454+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/3810885272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2e0069230 msgr2=0x7fe2e0071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:13.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.454+0000 7fe2e5cdd700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2e0069230 0x7fe2e0071df0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe2d000c8f0 tx=0x7fe2d000ccb0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.454+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/3810885272 shutdown_connections 2024-02-19T19:03:13.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.454+0000 7fe2e5cdd700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe2c806d520 0x7fe2c806f9d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.454+0000 7fe2e5cdd700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2e0069230 0x7fe2e0071df0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.454+0000 7fe2e5cdd700 1 --2- 172.21.15.88:0/3810885272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2e0103350 0x7fe2e0072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.454+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/3810885272 >> 172.21.15.88:0/3810885272 conn(0x7fe2e00fa780 msgr2=0x7fe2e00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:13.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.454+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/3810885272 shutdown_connections 2024-02-19T19:03:13.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.454+0000 7fe2e5cdd700 1 -- 172.21.15.88:0/3810885272 wait complete. 2024-02-19T19:03:13.484 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:03:13.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:13 smithi132 ceph-mon[37969]: pgmap v53: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:13.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.567+0000 7f7c49661700 1 -- 172.21.15.88:0/839678731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c44100350 msgr2=0x7f7c441007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:13.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.567+0000 7f7c49661700 1 --2- 172.21.15.88:0/839678731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c44100350 0x7f7c441007a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009910 tx=0x7f7c38009c20 comp rx=0 tx=0).stop 2024-02-19T19:03:13.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.567+0000 7f7c49661700 1 -- 172.21.15.88:0/839678731 shutdown_connections 2024-02-19T19:03:13.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.567+0000 7f7c49661700 1 --2- 172.21.15.88:0/839678731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c44100350 0x7f7c441007a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.567+0000 7f7c49661700 1 --2- 172.21.15.88:0/839678731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c440ff150 0x7f7c440ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.567+0000 7f7c49661700 1 -- 172.21.15.88:0/839678731 >> 172.21.15.88:0/839678731 conn(0x7f7c440fa700 msgr2=0x7f7c440fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:13.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.567+0000 7f7c49661700 1 -- 172.21.15.88:0/839678731 shutdown_connections 2024-02-19T19:03:13.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.567+0000 7f7c49661700 1 -- 172.21.15.88:0/839678731 wait complete. 2024-02-19T19:03:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.567+0000 7f7c49661700 1 Processor -- start 2024-02-19T19:03:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.567+0000 7f7c49661700 1 -- start start 2024-02-19T19:03:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c49661700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c440ff150 0x7f7c44195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c49661700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c44100350 0x7f7c44195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c49661700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44196600 con 0x7f7c44100350 2024-02-19T19:03:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c49661700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c44196740 con 0x7f7c440ff150 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c427fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c44100350 0x7f7c44195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c427fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c44100350 0x7f7c44195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44144/0 (socket says 172.21.15.88:44144) 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c427fc700 1 -- 172.21.15.88:0/3219286574 learned_addr learned my addr 172.21.15.88:0/3219286574 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c427fc700 1 -- 172.21.15.88:0/3219286574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c440ff150 msgr2=0x7f7c44195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c427fc700 1 --2- 172.21.15.88:0/3219286574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c440ff150 0x7f7c44195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c427fc700 1 -- 172.21.15.88:0/3219286574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380095f0 con 0x7f7c44100350 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c427fc700 1 --2- 172.21.15.88:0/3219286574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c44100350 0x7f7c44195fe0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006010 tx=0x7f7c3800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c2bfff700 1 -- 172.21.15.88:0/3219286574 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3801d070 con 0x7f7c44100350 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c49661700 1 -- 172.21.15.88:0/3219286574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4419b190 con 0x7f7c44100350 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c2bfff700 1 -- 172.21.15.88:0/3219286574 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7c3800f460 con 0x7f7c44100350 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.568+0000 7f7c2bfff700 1 -- 172.21.15.88:0/3219286574 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c38017630 con 0x7f7c44100350 2024-02-19T19:03:13.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.569+0000 7f7c49661700 1 -- 172.21.15.88:0/3219286574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4419b680 con 0x7f7c44100350 2024-02-19T19:03:13.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.570+0000 7f7c49661700 1 -- 172.21.15.88:0/3219286574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c44107ca0 con 0x7f7c44100350 2024-02-19T19:03:13.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.571+0000 7f7c2bfff700 1 -- 172.21.15.88:0/3219286574 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f7c3800f5d0 con 0x7f7c44100350 2024-02-19T19:03:13.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.571+0000 7f7c2bfff700 1 --2- 172.21.15.88:0/3219286574 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7c2c06d520 0x7f7c2c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:13.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.571+0000 7f7c2bfff700 1 -- 172.21.15.88:0/3219286574 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7c3808e420 con 0x7f7c44100350 2024-02-19T19:03:13.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.571+0000 7f7c42ffd700 1 --2- 172.21.15.88:0/3219286574 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7c2c06d520 0x7f7c2c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:13.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.572+0000 7f7c42ffd700 1 --2- 172.21.15.88:0/3219286574 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7c2c06d520 0x7f7c2c06f9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7c441001b0 tx=0x7f7c34009470 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:13.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.576+0000 7f7c2bfff700 1 -- 172.21.15.88:0/3219286574 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f7c3805bc30 con 0x7f7c44100350 2024-02-19T19:03:13.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.834+0000 7f7c49661700 1 -- 172.21.15.88:0/3219286574 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c440694e0 con 0x7f7c2c06d520 2024-02-19T19:03:13.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.835+0000 7f7c2bfff700 1 -- 172.21.15.88:0/3219286574 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f7c440694e0 con 0x7f7c2c06d520 2024-02-19T19:03:13.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.837+0000 7f7c49661700 1 -- 172.21.15.88:0/3219286574 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7c2c06d520 msgr2=0x7f7c2c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:13.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.837+0000 7f7c49661700 1 --2- 172.21.15.88:0/3219286574 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7c2c06d520 0x7f7c2c06f9d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7c441001b0 tx=0x7f7c34009470 comp rx=0 tx=0).stop 2024-02-19T19:03:13.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.837+0000 7f7c49661700 1 -- 172.21.15.88:0/3219286574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c44100350 msgr2=0x7f7c44195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:13.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.837+0000 7f7c49661700 1 --2- 172.21.15.88:0/3219286574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c44100350 0x7f7c44195fe0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f7c38006010 tx=0x7f7c3800be20 comp rx=0 tx=0).stop 2024-02-19T19:03:13.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.837+0000 7f7c49661700 1 -- 172.21.15.88:0/3219286574 shutdown_connections 2024-02-19T19:03:13.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.837+0000 7f7c49661700 1 --2- 172.21.15.88:0/3219286574 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f7c2c06d520 0x7f7c2c06f9d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.837+0000 7f7c49661700 1 --2- 172.21.15.88:0/3219286574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c440ff150 0x7f7c44195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.837+0000 7f7c49661700 1 --2- 172.21.15.88:0/3219286574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c44100350 0x7f7c44195fe0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.837+0000 7f7c49661700 1 -- 172.21.15.88:0/3219286574 >> 172.21.15.88:0/3219286574 conn(0x7f7c440fa700 msgr2=0x7f7c44068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:13.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.838+0000 7f7c49661700 1 -- 172.21.15.88:0/3219286574 shutdown_connections 2024-02-19T19:03:13.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.838+0000 7f7c49661700 1 -- 172.21.15.88:0/3219286574 wait complete. 2024-02-19T19:03:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.949+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/3929269258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc41003d0 msgr2=0x7f4cc4100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.949+0000 7f4ccc4b8700 1 --2- 172.21.15.88:0/3929269258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc41003d0 0x7f4cc4100840 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009910 tx=0x7f4cb4009c20 comp rx=0 tx=0).stop 2024-02-19T19:03:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.949+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/3929269258 shutdown_connections 2024-02-19T19:03:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.949+0000 7f4ccc4b8700 1 --2- 172.21.15.88:0/3929269258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc41003d0 0x7f4cc4100840 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.949+0000 7f4ccc4b8700 1 --2- 172.21.15.88:0/3929269258 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc40ff130 0x7f4cc40ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.949+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/3929269258 >> 172.21.15.88:0/3929269258 conn(0x7f4cc40fa780 msgr2=0x7f4cc40fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.949+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/3929269258 shutdown_connections 2024-02-19T19:03:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.949+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/3929269258 wait complete. 2024-02-19T19:03:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.950+0000 7f4ccc4b8700 1 Processor -- start 2024-02-19T19:03:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.950+0000 7f4ccc4b8700 1 -- start start 2024-02-19T19:03:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.950+0000 7f4ccc4b8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc40ff130 0x7f4cc4195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.950+0000 7f4ccc4b8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc41003d0 0x7f4cc4195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.950+0000 7f4ccc4b8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc41965c0 con 0x7f4cc40ff130 2024-02-19T19:03:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.950+0000 7f4ccc4b8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc4196700 con 0x7f4cc41003d0 2024-02-19T19:03:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.950+0000 7f4cca254700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc40ff130 0x7f4cc4195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.950+0000 7f4cca254700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc40ff130 0x7f4cc4195a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44160/0 (socket says 172.21.15.88:44160) 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.950+0000 7f4cca254700 1 -- 172.21.15.88:0/2466378981 learned_addr learned my addr 172.21.15.88:0/2466378981 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.951+0000 7f4cc9a53700 1 --2- 172.21.15.88:0/2466378981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc41003d0 0x7f4cc4195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.951+0000 7f4cca254700 1 -- 172.21.15.88:0/2466378981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc41003d0 msgr2=0x7f4cc4195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.951+0000 7f4cca254700 1 --2- 172.21.15.88:0/2466378981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc41003d0 0x7f4cc4195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.951+0000 7f4cca254700 1 -- 172.21.15.88:0/2466378981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb40095f0 con 0x7f4cc40ff130 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.951+0000 7f4cca254700 1 --2- 172.21.15.88:0/2466378981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc40ff130 0x7f4cc4195a60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000d8d0 tx=0x7f4cc000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.951+0000 7f4cbb7fe700 1 -- 172.21.15.88:0/2466378981 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cc00098a0 con 0x7f4cc40ff130 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.951+0000 7f4cbb7fe700 1 -- 172.21.15.88:0/2466378981 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4cc0010460 con 0x7f4cc40ff130 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.951+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/2466378981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc419b1b0 con 0x7f4cc40ff130 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.951+0000 7f4cbb7fe700 1 -- 172.21.15.88:0/2466378981 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cc000f5f0 con 0x7f4cc40ff130 2024-02-19T19:03:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.951+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/2466378981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc419b700 con 0x7f4cc40ff130 2024-02-19T19:03:13.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.952+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/2466378981 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc4107c10 con 0x7f4cc40ff130 2024-02-19T19:03:13.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.953+0000 7f4cbb7fe700 1 -- 172.21.15.88:0/2466378981 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f4cc0009a00 con 0x7f4cc40ff130 2024-02-19T19:03:13.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.953+0000 7f4cbb7fe700 1 --2- 172.21.15.88:0/2466378981 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f4cb006d520 0x7f4cb006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:13.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.953+0000 7f4cbb7fe700 1 -- 172.21.15.88:0/2466378981 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4cc008ce40 con 0x7f4cc40ff130 2024-02-19T19:03:13.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.953+0000 7f4cc9a53700 1 --2- 172.21.15.88:0/2466378981 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f4cb006d520 0x7f4cb006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:13.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.954+0000 7f4cc9a53700 1 --2- 172.21.15.88:0/2466378981 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f4cb006d520 0x7f4cb006f9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006010 tx=0x7f4cb400b540 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:13.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:13.962+0000 7f4cbb7fe700 1 -- 172.21.15.88:0/2466378981 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f4cc005a650 con 0x7f4cc40ff130 2024-02-19T19:03:14.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.189+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/2466378981 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4cc4068c50 con 0x7f4cb006d520 2024-02-19T19:03:14.209 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:14 smithi088 ceph-mon[29272]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:03:14.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.201+0000 7f4cbb7fe700 1 -- 172.21.15.88:0/2466378981 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f4cc4068c50 con 0x7f4cb006d520 2024-02-19T19:03:14.220 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:03:14.220 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (25s) 18s ago 10m 15.6M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (11m) 18s ago 11m 7759k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (9m) 58s ago 9m 7608k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (11m) 18s ago 11m 6665k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (9m) 58s ago 9m 6656k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (8m) 18s ago 10m 86.1M - 9.4.7 954c08fa6188 54907dc6edb4 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (4m) 18s ago 4m 14.1M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (4m) 18s ago 4m 26.5M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (4m) 58s ago 4m 23.1M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (4m) 58s ago 4m 17.6M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (113s) 18s ago 12m 497M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (79s) 58s ago 9m 449M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (12m) 18s ago 12m 54.5M 2048M 18.2.0 dc2bc1663786 3ea5108455bc 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (9m) 58s ago 9m 43.1M 2048M 18.2.0 dc2bc1663786 ba1c249c6017 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (67s) 18s ago 11m 15.9M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (62s) 58s ago 9m 3573k - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:03:14.221 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (8m) 18s ago 8m 51.9M 1216M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:03:14.222 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (7m) 18s ago 7m 49.5M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:03:14.222 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (7m) 18s ago 7m 49.0M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:03:14.222 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (6m) 58s ago 6m 50.5M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:03:14.222 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (6m) 58s ago 6m 49.1M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:03:14.222 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (6m) 58s ago 6m 47.2M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:03:14.222 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (43s) 18s ago 9m 51.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:03:14.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.204+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/2466378981 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f4cb006d520 msgr2=0x7f4cb006f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:14.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.204+0000 7f4ccc4b8700 1 --2- 172.21.15.88:0/2466378981 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f4cb006d520 0x7f4cb006f9d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006010 tx=0x7f4cb400b540 comp rx=0 tx=0).stop 2024-02-19T19:03:14.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.204+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/2466378981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc40ff130 msgr2=0x7f4cc4195a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:14.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.204+0000 7f4ccc4b8700 1 --2- 172.21.15.88:0/2466378981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc40ff130 0x7f4cc4195a60 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4cc000d8d0 tx=0x7f4cc000dbe0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.204+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/2466378981 shutdown_connections 2024-02-19T19:03:14.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.204+0000 7f4ccc4b8700 1 --2- 172.21.15.88:0/2466378981 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f4cb006d520 0x7f4cb006f9d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.204+0000 7f4ccc4b8700 1 --2- 172.21.15.88:0/2466378981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc40ff130 0x7f4cc4195a60 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.204+0000 7f4ccc4b8700 1 --2- 172.21.15.88:0/2466378981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc41003d0 0x7f4cc4195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.204+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/2466378981 >> 172.21.15.88:0/2466378981 conn(0x7f4cc40fa780 msgr2=0x7f4cc4068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:14.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.204+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/2466378981 shutdown_connections 2024-02-19T19:03:14.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.204+0000 7f4ccc4b8700 1 -- 172.21.15.88:0/2466378981 wait complete. 2024-02-19T19:03:14.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.314+0000 7f22f22f0700 1 -- 172.21.15.88:0/909090874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22ec1003d0 msgr2=0x7f22ec100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:14.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.314+0000 7f22f22f0700 1 --2- 172.21.15.88:0/909090874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22ec1003d0 0x7f22ec100840 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009910 tx=0x7f22e0009c20 comp rx=0 tx=0).stop 2024-02-19T19:03:14.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.314+0000 7f22f22f0700 1 -- 172.21.15.88:0/909090874 shutdown_connections 2024-02-19T19:03:14.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.314+0000 7f22f22f0700 1 --2- 172.21.15.88:0/909090874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22ec1003d0 0x7f22ec100840 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.314+0000 7f22f22f0700 1 --2- 172.21.15.88:0/909090874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22ec0ff130 0x7f22ec0ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.314+0000 7f22f22f0700 1 -- 172.21.15.88:0/909090874 >> 172.21.15.88:0/909090874 conn(0x7f22ec0fa780 msgr2=0x7f22ec0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:14.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.314+0000 7f22f22f0700 1 -- 172.21.15.88:0/909090874 shutdown_connections 2024-02-19T19:03:14.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.314+0000 7f22f22f0700 1 -- 172.21.15.88:0/909090874 wait complete. 2024-02-19T19:03:14.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.315+0000 7f22f22f0700 1 Processor -- start 2024-02-19T19:03:14.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.315+0000 7f22f22f0700 1 -- start start 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.315+0000 7f22f22f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22ec0ff130 0x7f22ec0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.315+0000 7f22f22f0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22ec1003d0 0x7f22ec19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.315+0000 7f22f22f0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec19ad70 con 0x7f22ec1003d0 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.315+0000 7f22f22f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec19aee0 con 0x7f22ec0ff130 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.315+0000 7f22eb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22ec1003d0 0x7f22ec19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.316+0000 7f22eb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22ec1003d0 0x7f22ec19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44182/0 (socket says 172.21.15.88:44182) 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.316+0000 7f22eb7fe700 1 -- 172.21.15.88:0/3515763183 learned_addr learned my addr 172.21.15.88:0/3515763183 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.316+0000 7f22ebfff700 1 --2- 172.21.15.88:0/3515763183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22ec0ff130 0x7f22ec0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.316+0000 7f22eb7fe700 1 -- 172.21.15.88:0/3515763183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22ec0ff130 msgr2=0x7f22ec0690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.316+0000 7f22eb7fe700 1 --2- 172.21.15.88:0/3515763183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22ec0ff130 0x7f22ec0690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.316+0000 7f22eb7fe700 1 -- 172.21.15.88:0/3515763183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00095f0 con 0x7f22ec1003d0 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.316+0000 7f22eb7fe700 1 --2- 172.21.15.88:0/3515763183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22ec1003d0 0x7f22ec19a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f22e000b5c0 tx=0x7f22e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:14.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.316+0000 7f22e97fa700 1 -- 172.21.15.88:0/3515763183 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22e001d070 con 0x7f22ec1003d0 2024-02-19T19:03:14.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.316+0000 7f22e97fa700 1 -- 172.21.15.88:0/3515763183 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f22e000f460 con 0x7f22ec1003d0 2024-02-19T19:03:14.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.316+0000 7f22f22f0700 1 -- 172.21.15.88:0/3515763183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec19b160 con 0x7f22ec1003d0 2024-02-19T19:03:14.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.317+0000 7f22e97fa700 1 -- 172.21.15.88:0/3515763183 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22e0021640 con 0x7f22ec1003d0 2024-02-19T19:03:14.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.317+0000 7f22f22f0700 1 -- 172.21.15.88:0/3515763183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec19b650 con 0x7f22ec1003d0 2024-02-19T19:03:14.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.318+0000 7f22f22f0700 1 -- 172.21.15.88:0/3515763183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec107c10 con 0x7f22ec1003d0 2024-02-19T19:03:14.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.318+0000 7f22e97fa700 1 -- 172.21.15.88:0/3515763183 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f22e0003a60 con 0x7f22ec1003d0 2024-02-19T19:03:14.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.319+0000 7f22e97fa700 1 --2- 172.21.15.88:0/3515763183 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f22d406d520 0x7f22d406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:14.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.319+0000 7f22e97fa700 1 -- 172.21.15.88:0/3515763183 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f22e008e150 con 0x7f22ec1003d0 2024-02-19T19:03:14.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.319+0000 7f22ebfff700 1 --2- 172.21.15.88:0/3515763183 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f22d406d520 0x7f22d406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:14.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.319+0000 7f22ebfff700 1 --2- 172.21.15.88:0/3515763183 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f22d406d520 0x7f22d406f9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22dc005950 tx=0x7f22dc00b410 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:14.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.325+0000 7f22e97fa700 1 -- 172.21.15.88:0/3515763183 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f22e005b9e0 con 0x7f22ec1003d0 2024-02-19T19:03:14.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:14 smithi132 ceph-mon[37969]: from='client.14774 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:03:14.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.623+0000 7f22f22f0700 1 -- 172.21.15.88:0/3515763183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f22ec19bce0 con 0x7f22ec1003d0 2024-02-19T19:03:14.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.624+0000 7f22e97fa700 1 -- 172.21.15.88:0/3515763183 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f22e005b570 con 0x7f22ec1003d0 2024-02-19T19:03:14.643 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:03:14.643 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:03:14.644 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-19T19:03:14.645 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:03:14.645 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:03:14.645 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:03:14.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.626+0000 7f22f22f0700 1 -- 172.21.15.88:0/3515763183 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f22d406d520 msgr2=0x7f22d406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:14.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.626+0000 7f22f22f0700 1 --2- 172.21.15.88:0/3515763183 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f22d406d520 0x7f22d406f9d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f22dc005950 tx=0x7f22dc00b410 comp rx=0 tx=0).stop 2024-02-19T19:03:14.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.626+0000 7f22f22f0700 1 -- 172.21.15.88:0/3515763183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22ec1003d0 msgr2=0x7f22ec19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:14.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.626+0000 7f22f22f0700 1 --2- 172.21.15.88:0/3515763183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22ec1003d0 0x7f22ec19a7a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f22e000b5c0 tx=0x7f22e000bd70 comp rx=0 tx=0).stop 2024-02-19T19:03:14.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.627+0000 7f22f22f0700 1 -- 172.21.15.88:0/3515763183 shutdown_connections 2024-02-19T19:03:14.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.627+0000 7f22f22f0700 1 --2- 172.21.15.88:0/3515763183 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f22d406d520 0x7f22d406f9d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.627+0000 7f22f22f0700 1 --2- 172.21.15.88:0/3515763183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22ec0ff130 0x7f22ec0690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.627+0000 7f22f22f0700 1 --2- 172.21.15.88:0/3515763183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22ec1003d0 0x7f22ec19a7a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.627+0000 7f22f22f0700 1 -- 172.21.15.88:0/3515763183 >> 172.21.15.88:0/3515763183 conn(0x7f22ec0fa780 msgr2=0x7f22ec103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:14.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.627+0000 7f22f22f0700 1 -- 172.21.15.88:0/3515763183 shutdown_connections 2024-02-19T19:03:14.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.627+0000 7f22f22f0700 1 -- 172.21.15.88:0/3515763183 wait complete. 2024-02-19T19:03:14.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.755+0000 7f8cd664b700 1 -- 172.21.15.88:0/179429897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cd0100240 msgr2=0x7f8cd0100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:14.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.755+0000 7f8cd664b700 1 --2- 172.21.15.88:0/179429897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cd0100240 0x7f8cd0100650 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009910 tx=0x7f8cc0009c20 comp rx=0 tx=0).stop 2024-02-19T19:03:14.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.755+0000 7f8cd664b700 1 -- 172.21.15.88:0/179429897 shutdown_connections 2024-02-19T19:03:14.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.755+0000 7f8cd664b700 1 --2- 172.21.15.88:0/179429897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cd0101440 0x7f8cd0101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.755+0000 7f8cd664b700 1 --2- 172.21.15.88:0/179429897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cd0100240 0x7f8cd0100650 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.755+0000 7f8cd664b700 1 -- 172.21.15.88:0/179429897 >> 172.21.15.88:0/179429897 conn(0x7f8cd00fb7f0 msgr2=0x7f8cd00fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:14.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.755+0000 7f8cd664b700 1 -- 172.21.15.88:0/179429897 shutdown_connections 2024-02-19T19:03:14.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.755+0000 7f8cd664b700 1 -- 172.21.15.88:0/179429897 wait complete. 2024-02-19T19:03:14.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.756+0000 7f8cd664b700 1 Processor -- start 2024-02-19T19:03:14.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.756+0000 7f8cd664b700 1 -- start start 2024-02-19T19:03:14.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.756+0000 7f8cd664b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cd0100240 0x7f8cd0195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:14.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.756+0000 7f8cd664b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cd0101440 0x7f8cd0195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:14.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.756+0000 7f8cd664b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0196610 con 0x7f8cd0101440 2024-02-19T19:03:14.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.756+0000 7f8cd664b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0196750 con 0x7f8cd0100240 2024-02-19T19:03:14.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.756+0000 7f8ccf7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cd0101440 0x7f8cd0195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:14.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.756+0000 7f8ccf7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cd0101440 0x7f8cd0195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44208/0 (socket says 172.21.15.88:44208) 2024-02-19T19:03:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.756+0000 7f8ccf7fe700 1 -- 172.21.15.88:0/3340694531 learned_addr learned my addr 172.21.15.88:0/3340694531 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:03:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.756+0000 7f8ccffff700 1 --2- 172.21.15.88:0/3340694531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cd0100240 0x7f8cd0195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.757+0000 7f8ccf7fe700 1 -- 172.21.15.88:0/3340694531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cd0100240 msgr2=0x7f8cd0195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.757+0000 7f8ccf7fe700 1 --2- 172.21.15.88:0/3340694531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cd0100240 0x7f8cd0195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.757+0000 7f8ccf7fe700 1 -- 172.21.15.88:0/3340694531 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00095f0 con 0x7f8cd0101440 2024-02-19T19:03:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.757+0000 7f8ccf7fe700 1 --2- 172.21.15.88:0/3340694531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cd0101440 0x7f8cd0195ff0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400d8d0 tx=0x7f8cc400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.757+0000 7f8ccd7fa700 1 -- 172.21.15.88:0/3340694531 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cc40098a0 con 0x7f8cd0101440 2024-02-19T19:03:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.757+0000 7f8ccd7fa700 1 -- 172.21.15.88:0/3340694531 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8cc4010460 con 0x7f8cd0101440 2024-02-19T19:03:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.757+0000 7f8ccd7fa700 1 -- 172.21.15.88:0/3340694531 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cc400f5f0 con 0x7f8cd0101440 2024-02-19T19:03:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.757+0000 7f8cd664b700 1 -- 172.21.15.88:0/3340694531 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd019b200 con 0x7f8cd0101440 2024-02-19T19:03:14.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.757+0000 7f8cd664b700 1 -- 172.21.15.88:0/3340694531 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd019b750 con 0x7f8cd0101440 2024-02-19T19:03:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.758+0000 7f8cd664b700 1 -- 172.21.15.88:0/3340694531 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd0108d90 con 0x7f8cd0101440 2024-02-19T19:03:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.759+0000 7f8ccd7fa700 1 -- 172.21.15.88:0/3340694531 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f8cc4009a00 con 0x7f8cd0101440 2024-02-19T19:03:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.759+0000 7f8ccd7fa700 1 --2- 172.21.15.88:0/3340694531 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8cb806d520 0x7f8cb806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.759+0000 7f8ccd7fa700 1 -- 172.21.15.88:0/3340694531 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8cc408cd80 con 0x7f8cd0101440 2024-02-19T19:03:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.759+0000 7f8ccffff700 1 --2- 172.21.15.88:0/3340694531 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8cb806d520 0x7f8cb806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:14.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.760+0000 7f8ccffff700 1 --2- 172.21.15.88:0/3340694531 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8cb806d520 0x7f8cb806f9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0006010 tx=0x7f8cc000b540 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:14.764+0000 7f8ccd7fa700 1 -- 172.21.15.88:0/3340694531 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f8cc405a610 con 0x7f8cd0101440 2024-02-19T19:03:15.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.035+0000 7f8cd664b700 1 -- 172.21.15.88:0/3340694531 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cd0105d90 con 0x7f8cb806d520 2024-02-19T19:03:15.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.036+0000 7f8ccd7fa700 1 -- 172.21.15.88:0/3340694531 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f8cd0105d90 con 0x7f8cb806d520 2024-02-19T19:03:15.056 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:03:15.056 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:03:15.056 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:03:15.056 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-02-19T19:03:15.056 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:03:15.056 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:03:15.056 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:03:15.056 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "2/2 daemons upgraded", 2024-02-19T19:03:15.056 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading grafana daemons", 2024-02-19T19:03:15.056 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:03:15.056 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:03:15.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.039+0000 7f8cd664b700 1 -- 172.21.15.88:0/3340694531 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8cb806d520 msgr2=0x7f8cb806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:15.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.040+0000 7f8cd664b700 1 --2- 172.21.15.88:0/3340694531 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8cb806d520 0x7f8cb806f9d0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0006010 tx=0x7f8cc000b540 comp rx=0 tx=0).stop 2024-02-19T19:03:15.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.040+0000 7f8cd664b700 1 -- 172.21.15.88:0/3340694531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cd0101440 msgr2=0x7f8cd0195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:15.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.040+0000 7f8cd664b700 1 --2- 172.21.15.88:0/3340694531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cd0101440 0x7f8cd0195ff0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400d8d0 tx=0x7f8cc400dbe0 comp rx=0 tx=0).stop 2024-02-19T19:03:15.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.040+0000 7f8cd664b700 1 -- 172.21.15.88:0/3340694531 shutdown_connections 2024-02-19T19:03:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.040+0000 7f8cd664b700 1 --2- 172.21.15.88:0/3340694531 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8cb806d520 0x7f8cb806f9d0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.040+0000 7f8cd664b700 1 --2- 172.21.15.88:0/3340694531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8cd0100240 0x7f8cd0195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.040+0000 7f8cd664b700 1 --2- 172.21.15.88:0/3340694531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cd0101440 0x7f8cd0195ff0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.040+0000 7f8cd664b700 1 -- 172.21.15.88:0/3340694531 >> 172.21.15.88:0/3340694531 conn(0x7f8cd00fb7f0 msgr2=0x7f8cd0104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.040+0000 7f8cd664b700 1 -- 172.21.15.88:0/3340694531 shutdown_connections 2024-02-19T19:03:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:15.040+0000 7f8cd664b700 1 -- 172.21.15.88:0/3340694531 wait complete. 2024-02-19T19:03:15.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:15 smithi088 ceph-mon[29272]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:03:15.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:15 smithi088 ceph-mon[29272]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:03:15.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:15 smithi088 ceph-mon[29272]: pgmap v54: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:15.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:15 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/3515763183' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:15.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:15 smithi132 ceph-mon[37969]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:03:15.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:15 smithi132 ceph-mon[37969]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:03:15.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:15 smithi132 ceph-mon[37969]: pgmap v54: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-02-19T19:03:15.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:15 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/3515763183' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:16.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:16 smithi088 ceph-mon[29272]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:03:16.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:16 smithi132 ceph-mon[37969]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:03:17.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:17 smithi088 ceph-mon[29272]: pgmap v55: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:17.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:17 smithi132 ceph-mon[37969]: pgmap v55: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:18.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:18 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:03:18.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:18 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:03:19.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:19 smithi088 ceph-mon[29272]: pgmap v56: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:19.403 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:19 smithi132 ceph-mon[37969]: pgmap v56: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:21.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:21 smithi088 ceph-mon[29272]: pgmap v57: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:21.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:21 smithi132 ceph-mon[37969]: pgmap v57: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:23.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:23 smithi088 ceph-mon[29272]: pgmap v58: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:23 smithi132 ceph-mon[37969]: pgmap v58: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:25.420 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:25 smithi088 ceph-mon[29272]: pgmap v59: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:25.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:25 smithi132 ceph-mon[37969]: pgmap v59: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:27.412 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:27 smithi088 ceph-mon[29272]: pgmap v60: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:27.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:27 smithi132 ceph-mon[37969]: pgmap v60: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:29.624 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:29 smithi088 ceph-mon[29272]: pgmap v61: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:29.624 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:29 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:29.624 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:29 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:29.624 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:29 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:03:29.696 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:29 smithi132 ceph-mon[37969]: pgmap v61: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:29.697 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:29 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:29.697 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:29 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:29.697 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:29 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:03:31.402 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:31 smithi088 ceph-mon[29272]: pgmap v62: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:31.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:31 smithi132 ceph-mon[37969]: pgmap v62: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:33.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:33 smithi088 ceph-mon[29272]: pgmap v63: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:33.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:33 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:03:33.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:33 smithi132 ceph-mon[37969]: pgmap v63: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:33.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:33 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:03:35.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:35 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:35.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:35 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:35.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:35 smithi088 ceph-mon[29272]: pgmap v64: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:35.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:35 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:35.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:35 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:35.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:35 smithi132 ceph-mon[37969]: pgmap v64: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:36.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:36 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:36.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:36 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:36.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:36 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:36.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:36 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:37.548 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:37 smithi088 ceph-mon[29272]: pgmap v65: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:37.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:37 smithi132 ceph-mon[37969]: pgmap v65: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:39.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: pgmap v66: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:39.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:39.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:39.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:03:39.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:03:39.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:39.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-19T19:03:39.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-19T19:03:39.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-19T19:03:39.757 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:03:39.757 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:03:39.757 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:03:39.757 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:39.757 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:03:39.757 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:03:39.757 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:03:39.757 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:39.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: pgmap v66: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:39.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:39.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:39.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:03:39.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:03:39.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:39.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-19T19:03:39.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-19T19:03:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:03:39.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:39.854 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:03:39.854 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:03:39.854 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:03:39.854 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:40.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:40 smithi132 ceph-mon[37969]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-19T19:03:40.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:40 smithi132 ceph-mon[37969]: Upgrade: Finalizing container_image settings 2024-02-19T19:03:40.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:40 smithi132 ceph-mon[37969]: Upgrade: Complete! 2024-02-19T19:03:40.752 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:40 smithi088 ceph-mon[29272]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-19T19:03:40.753 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:40 smithi088 ceph-mon[29272]: Upgrade: Finalizing container_image settings 2024-02-19T19:03:40.753 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:40 smithi088 ceph-mon[29272]: Upgrade: Complete! 2024-02-19T19:03:41.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:41 smithi132 ceph-mon[37969]: pgmap v67: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:41.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:41 smithi088 ceph-mon[29272]: pgmap v67: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:43.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:43 smithi088 ceph-mon[29272]: pgmap v68: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:43.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:43 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:43.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:43 smithi132 ceph-mon[37969]: pgmap v68: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:43.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:43 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:03:45.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.153+0000 7fa2814bf700 1 -- 172.21.15.88:0/2542055277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c1003f0 msgr2=0x7fa27c100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:45.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.153+0000 7fa2814bf700 1 --2- 172.21.15.88:0/2542055277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c1003f0 0x7fa27c100860 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa264009910 tx=0x7fa264009c20 comp rx=0 tx=0).stop 2024-02-19T19:03:45.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.153+0000 7fa2814bf700 1 -- 172.21.15.88:0/2542055277 shutdown_connections 2024-02-19T19:03:45.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.153+0000 7fa2814bf700 1 --2- 172.21.15.88:0/2542055277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c1003f0 0x7fa27c100860 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:45.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.153+0000 7fa2814bf700 1 --2- 172.21.15.88:0/2542055277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa27c0ff150 0x7fa27c0ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:45.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.153+0000 7fa2814bf700 1 -- 172.21.15.88:0/2542055277 >> 172.21.15.88:0/2542055277 conn(0x7fa27c0fa7a0 msgr2=0x7fa27c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:45.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.153+0000 7fa2814bf700 1 -- 172.21.15.88:0/2542055277 shutdown_connections 2024-02-19T19:03:45.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.153+0000 7fa2814bf700 1 -- 172.21.15.88:0/2542055277 wait complete. 2024-02-19T19:03:45.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.153+0000 7fa2814bf700 1 Processor -- start 2024-02-19T19:03:45.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.153+0000 7fa2814bf700 1 -- start start 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa2814bf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c0ff150 0x7fa27c103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa2814bf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa27c1003f0 0x7fa27c1017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa2814bf700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c101d10 con 0x7fa27c0ff150 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa2814bf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c101e50 con 0x7fa27c1003f0 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa27affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c0ff150 0x7fa27c103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa27affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c0ff150 0x7fa27c103180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55546/0 (socket says 172.21.15.88:55546) 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa27affd700 1 -- 172.21.15.88:0/2370733467 learned_addr learned my addr 172.21.15.88:0/2370733467 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa273fff700 1 --2- 172.21.15.88:0/2370733467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa27c1003f0 0x7fa27c1017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa27affd700 1 -- 172.21.15.88:0/2370733467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa27c1003f0 msgr2=0x7fa27c1017d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa27affd700 1 --2- 172.21.15.88:0/2370733467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa27c1003f0 0x7fa27c1017d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa27affd700 1 -- 172.21.15.88:0/2370733467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640095f0 con 0x7fa27c0ff150 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa27affd700 1 --2- 172.21.15.88:0/2370733467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c0ff150 0x7fa27c103180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00d8d0 tx=0x7fa26c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa278ff9700 1 -- 172.21.15.88:0/2370733467 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26c0098a0 con 0x7fa27c0ff150 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa278ff9700 1 -- 172.21.15.88:0/2370733467 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa26c010460 con 0x7fa27c0ff150 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa2814bf700 1 -- 172.21.15.88:0/2370733467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c102130 con 0x7fa27c0ff150 2024-02-19T19:03:45.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa278ff9700 1 -- 172.21.15.88:0/2370733467 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa26c00f5f0 con 0x7fa27c0ff150 2024-02-19T19:03:45.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.154+0000 7fa2814bf700 1 -- 172.21.15.88:0/2370733467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c19de20 con 0x7fa27c0ff150 2024-02-19T19:03:45.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.155+0000 7fa2814bf700 1 -- 172.21.15.88:0/2370733467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c068ab0 con 0x7fa27c0ff150 2024-02-19T19:03:45.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.156+0000 7fa278ff9700 1 -- 172.21.15.88:0/2370733467 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fa26c009a00 con 0x7fa27c0ff150 2024-02-19T19:03:45.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.156+0000 7fa278ff9700 1 --2- 172.21.15.88:0/2370733467 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa26806d520 0x7fa26806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:45.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.156+0000 7fa278ff9700 1 -- 172.21.15.88:0/2370733467 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa26c08ce40 con 0x7fa27c0ff150 2024-02-19T19:03:45.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.156+0000 7fa273fff700 1 --2- 172.21.15.88:0/2370733467 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa26806d520 0x7fa26806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:45.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.157+0000 7fa273fff700 1 --2- 172.21.15.88:0/2370733467 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa26806d520 0x7fa26806f9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa264006010 tx=0x7fa26400b540 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:45.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.161+0000 7fa278ff9700 1 -- 172.21.15.88:0/2370733467 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fa26c05a650 con 0x7fa27c0ff150 2024-02-19T19:03:45.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.381+0000 7fa2814bf700 1 -- 172.21.15.88:0/2370733467 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa27c19dfb0 con 0x7fa26806d520 2024-02-19T19:03:45.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.382+0000 7fa278ff9700 1 -- 172.21.15.88:0/2370733467 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fa27c19dfb0 con 0x7fa26806d520 2024-02-19T19:03:45.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.385+0000 7fa2814bf700 1 -- 172.21.15.88:0/2370733467 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa26806d520 msgr2=0x7fa26806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:45.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.385+0000 7fa2814bf700 1 --2- 172.21.15.88:0/2370733467 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa26806d520 0x7fa26806f9d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa264006010 tx=0x7fa26400b540 comp rx=0 tx=0).stop 2024-02-19T19:03:45.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.385+0000 7fa2814bf700 1 -- 172.21.15.88:0/2370733467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c0ff150 msgr2=0x7fa27c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:45.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.385+0000 7fa2814bf700 1 --2- 172.21.15.88:0/2370733467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c0ff150 0x7fa27c103180 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00d8d0 tx=0x7fa26c00dbe0 comp rx=0 tx=0).stop 2024-02-19T19:03:45.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.385+0000 7fa2814bf700 1 -- 172.21.15.88:0/2370733467 shutdown_connections 2024-02-19T19:03:45.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.385+0000 7fa2814bf700 1 --2- 172.21.15.88:0/2370733467 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fa26806d520 0x7fa26806f9d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:45.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.385+0000 7fa2814bf700 1 --2- 172.21.15.88:0/2370733467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa27c0ff150 0x7fa27c103180 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:45.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.385+0000 7fa2814bf700 1 --2- 172.21.15.88:0/2370733467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa27c1003f0 0x7fa27c1017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:45.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.385+0000 7fa2814bf700 1 -- 172.21.15.88:0/2370733467 >> 172.21.15.88:0/2370733467 conn(0x7fa27c0fa7a0 msgr2=0x7fa27c068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:45.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.385+0000 7fa2814bf700 1 -- 172.21.15.88:0/2370733467 shutdown_connections 2024-02-19T19:03:45.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:45.385+0000 7fa2814bf700 1 -- 172.21.15.88:0/2370733467 wait complete. 2024-02-19T19:03:45.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:45 smithi088 ceph-mon[29272]: pgmap v69: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:45.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:45 smithi132 ceph-mon[37969]: pgmap v69: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:46.198 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-02-19T19:03:46.452 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:03:46.574 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:46 smithi088 ceph-mon[29272]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:03:46.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:46 smithi132 ceph-mon[37969]: from='client.14794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:03:47.733 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:47 smithi088 ceph-mon[29272]: pgmap v70: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:47.733 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:47 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:03:47.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:47 smithi132 ceph-mon[37969]: pgmap v70: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:47.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:47 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:03:48.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.255+0000 7f9f59d9a700 1 -- 172.21.15.88:0/171001955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f540ff3c0 msgr2=0x7f9f540ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:48.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.255+0000 7f9f59d9a700 1 --2- 172.21.15.88:0/171001955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f540ff3c0 0x7f9f540ff790 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f9f48009910 tx=0x7f9f48009c20 comp rx=0 tx=0).stop 2024-02-19T19:03:48.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.255+0000 7f9f59d9a700 1 -- 172.21.15.88:0/171001955 shutdown_connections 2024-02-19T19:03:48.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.256+0000 7f9f59d9a700 1 --2- 172.21.15.88:0/171001955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f540ffd60 0x7f9f54103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:48.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.256+0000 7f9f59d9a700 1 --2- 172.21.15.88:0/171001955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f540ff3c0 0x7f9f540ff790 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:48.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.256+0000 7f9f59d9a700 1 -- 172.21.15.88:0/171001955 >> 172.21.15.88:0/171001955 conn(0x7f9f540fac80 msgr2=0x7f9f540fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:48.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.256+0000 7f9f59d9a700 1 -- 172.21.15.88:0/171001955 shutdown_connections 2024-02-19T19:03:48.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.256+0000 7f9f59d9a700 1 -- 172.21.15.88:0/171001955 wait complete. 2024-02-19T19:03:48.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.256+0000 7f9f59d9a700 1 Processor -- start 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.256+0000 7f9f59d9a700 1 -- start start 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.256+0000 7f9f59d9a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f540ff3c0 0x7f9f54196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.256+0000 7f9f59d9a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f540ffd60 0x7f9f541972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.256+0000 7f9f59d9a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f541978f0 con 0x7f9f540ff3c0 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.256+0000 7f9f59d9a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f54197a60 con 0x7f9f540ffd60 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f537fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f540ff3c0 0x7f9f54196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f537fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f540ff3c0 0x7f9f54196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57102/0 (socket says 172.21.15.88:57102) 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f537fe700 1 -- 172.21.15.88:0/3180179301 learned_addr learned my addr 172.21.15.88:0/3180179301 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f52ffd700 1 --2- 172.21.15.88:0/3180179301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f540ffd60 0x7f9f541972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f537fe700 1 -- 172.21.15.88:0/3180179301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f540ffd60 msgr2=0x7f9f541972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f537fe700 1 --2- 172.21.15.88:0/3180179301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f540ffd60 0x7f9f541972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f537fe700 1 -- 172.21.15.88:0/3180179301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f480095f0 con 0x7f9f540ff3c0 2024-02-19T19:03:48.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f537fe700 1 --2- 172.21.15.88:0/3180179301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f540ff3c0 0x7f9f54196d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9f48006010 tx=0x7f9f4800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f50ff9700 1 -- 172.21.15.88:0/3180179301 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f4801d070 con 0x7f9f540ff3c0 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f59d9a700 1 -- 172.21.15.88:0/3180179301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5419fe00 con 0x7f9f540ff3c0 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f50ff9700 1 -- 172.21.15.88:0/3180179301 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f48005dc0 con 0x7f9f540ff3c0 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f50ff9700 1 -- 172.21.15.88:0/3180179301 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f48017400 con 0x7f9f540ff3c0 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.257+0000 7f9f59d9a700 1 -- 172.21.15.88:0/3180179301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f541a0270 con 0x7f9f540ff3c0 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.258+0000 7f9f59d9a700 1 -- 172.21.15.88:0/3180179301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f541076d0 con 0x7f9f540ff3c0 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.259+0000 7f9f50ff9700 1 -- 172.21.15.88:0/3180179301 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f9f48003a60 con 0x7f9f540ff3c0 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.259+0000 7f9f50ff9700 1 --2- 172.21.15.88:0/3180179301 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9f3c06d520 0x7f9f3c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.259+0000 7f9f50ff9700 1 -- 172.21.15.88:0/3180179301 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9f4808e3f0 con 0x7f9f540ff3c0 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.259+0000 7f9f52ffd700 1 --2- 172.21.15.88:0/3180179301 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9f3c06d520 0x7f9f3c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.260+0000 7f9f52ffd700 1 --2- 172.21.15.88:0/3180179301 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9f3c06d520 0x7f9f3c06f9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9f540696a0 tx=0x7f9f4400b3c0 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:48.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.264+0000 7f9f50ff9700 1 -- 172.21.15.88:0/3180179301 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f9f480580d0 con 0x7f9f540ff3c0 2024-02-19T19:03:48.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.551+0000 7f9f59d9a700 1 -- 172.21.15.88:0/3180179301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9f5404f2a0 con 0x7f9f540ff3c0 2024-02-19T19:03:48.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.552+0000 7f9f50ff9700 1 -- 172.21.15.88:0/3180179301 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f9f4805b790 con 0x7f9f540ff3c0 2024-02-19T19:03:48.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.553+0000 7f9f59d9a700 1 -- 172.21.15.88:0/3180179301 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9f3c06d520 msgr2=0x7f9f3c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:48.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.553+0000 7f9f59d9a700 1 --2- 172.21.15.88:0/3180179301 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9f3c06d520 0x7f9f3c06f9d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9f540696a0 tx=0x7f9f4400b3c0 comp rx=0 tx=0).stop 2024-02-19T19:03:48.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.554+0000 7f9f59d9a700 1 -- 172.21.15.88:0/3180179301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f540ff3c0 msgr2=0x7f9f54196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:48.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.554+0000 7f9f59d9a700 1 --2- 172.21.15.88:0/3180179301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f540ff3c0 0x7f9f54196d90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9f48006010 tx=0x7f9f4800bd70 comp rx=0 tx=0).stop 2024-02-19T19:03:48.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.554+0000 7f9f59d9a700 1 -- 172.21.15.88:0/3180179301 shutdown_connections 2024-02-19T19:03:48.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.554+0000 7f9f59d9a700 1 --2- 172.21.15.88:0/3180179301 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9f3c06d520 0x7f9f3c06f9d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:48.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.554+0000 7f9f59d9a700 1 --2- 172.21.15.88:0/3180179301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f540ff3c0 0x7f9f54196d90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:48.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.554+0000 7f9f59d9a700 1 --2- 172.21.15.88:0/3180179301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f540ffd60 0x7f9f541972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:48.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.554+0000 7f9f59d9a700 1 -- 172.21.15.88:0/3180179301 >> 172.21.15.88:0/3180179301 conn(0x7f9f540fac80 msgr2=0x7f9f540fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:48.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.554+0000 7f9f59d9a700 1 -- 172.21.15.88:0/3180179301 shutdown_connections 2024-02-19T19:03:48.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:48.554+0000 7f9f59d9a700 1 -- 172.21.15.88:0/3180179301 wait complete. 2024-02-19T19:03:48.584 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:03:49.313 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-02-19T19:03:49.526 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:03:49.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:49 smithi088 ceph-mon[29272]: pgmap v71: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:49.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:49 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/3180179301' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:49.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:49 smithi132 ceph-mon[37969]: pgmap v71: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:49.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:49 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/3180179301' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:51.722 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:51 smithi088 ceph-mon[29272]: pgmap v72: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:51.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:51 smithi132 ceph-mon[37969]: pgmap v72: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:53.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:53 smithi088 ceph-mon[29272]: pgmap v73: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:53.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:53 smithi132 ceph-mon[37969]: pgmap v73: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:55.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:55 smithi132 ceph-mon[37969]: pgmap v74: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:55.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:55 smithi088 ceph-mon[29272]: pgmap v74: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:55.873 INFO:tasks.workunit.client.1.smithi132.stderr:Note: switching to '1ce7ea8877c709fa862eac608550374bd96e166e'. 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr: 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr:state without impacting any branches by switching back to a branch. 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr: 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr: 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr: git switch -c 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr: 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr:Or undo this operation with: 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr: 2024-02-19T19:03:55.874 INFO:tasks.workunit.client.1.smithi132.stderr: git switch - 2024-02-19T19:03:55.875 INFO:tasks.workunit.client.1.smithi132.stderr: 2024-02-19T19:03:55.875 INFO:tasks.workunit.client.1.smithi132.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-19T19:03:55.875 INFO:tasks.workunit.client.1.smithi132.stderr: 2024-02-19T19:03:55.875 INFO:tasks.workunit.client.1.smithi132.stderr:HEAD is now at 1ce7ea8877c fix 2024-02-19T19:03:55.878 DEBUG:teuthology.orchestra.run.smithi132:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-02-19T19:03:55.904 INFO:tasks.workunit.client.1.smithi132.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-19T19:03:55.906 INFO:tasks.workunit.client.1.smithi132.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-19T19:03:55.906 INFO:tasks.workunit.client.1.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-19T19:03:56.658 INFO:tasks.workunit.client.1.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-19T19:03:56.729 INFO:tasks.workunit.client.1.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-19T19:03:56.771 INFO:tasks.workunit.client.1.smithi132.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-02-19T19:03:56.772 INFO:tasks.workunit.client.1.smithi132.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-19T19:03:56.772 INFO:tasks.workunit.client.1.smithi132.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-19T19:03:56.812 INFO:tasks.workunit.client.1.smithi132.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-02-19T19:03:56.815 DEBUG:teuthology.orchestra.run.smithi132:> set -ex 2024-02-19T19:03:56.815 DEBUG:teuthology.orchestra.run.smithi132:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-02-19T19:03:56.871 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-02-19T19:03:56.873 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-19T19:03:56.873 DEBUG:teuthology.orchestra.run.smithi132:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1ce7ea8877c709fa862eac608550374bd96e166e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-02-19T19:03:56.936 INFO:tasks.workunit.client.1.smithi132.stderr:+ mkdir -p fsstress 2024-02-19T19:03:56.938 INFO:tasks.workunit.client.1.smithi132.stderr:+ pushd fsstress 2024-02-19T19:03:56.939 INFO:tasks.workunit.client.1.smithi132.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-19T19:03:56.939 INFO:tasks.workunit.client.1.smithi132.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-19T19:03:57.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.006+0000 7fb18a5d2700 1 -- 172.21.15.88:0/3423361405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb184100e60 msgr2=0x7fb184104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:57.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.006+0000 7fb18a5d2700 1 --2- 172.21.15.88:0/3423361405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb184100e60 0x7fb184104e30 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb178009910 tx=0x7fb178009c20 comp rx=0 tx=0).stop 2024-02-19T19:03:57.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.006+0000 7fb18a5d2700 1 -- 172.21.15.88:0/3423361405 shutdown_connections 2024-02-19T19:03:57.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.006+0000 7fb18a5d2700 1 --2- 172.21.15.88:0/3423361405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb184100e60 0x7fb184104e30 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:57.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.006+0000 7fb18a5d2700 1 --2- 172.21.15.88:0/3423361405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb1841004c0 0x7fb184100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:57.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.006+0000 7fb18a5d2700 1 -- 172.21.15.88:0/3423361405 >> 172.21.15.88:0/3423361405 conn(0x7fb1840fbd80 msgr2=0x7fb1840fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:57.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.006+0000 7fb18a5d2700 1 -- 172.21.15.88:0/3423361405 shutdown_connections 2024-02-19T19:03:57.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.006+0000 7fb18a5d2700 1 -- 172.21.15.88:0/3423361405 wait complete. 2024-02-19T19:03:57.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.007+0000 7fb18a5d2700 1 Processor -- start 2024-02-19T19:03:57.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.007+0000 7fb18a5d2700 1 -- start start 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb18a5d2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1841004c0 0x7fb184195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb18a5d2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb184100e60 0x7fb184196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb18a5d2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb184196b10 con 0x7fb1841004c0 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb18a5d2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18419a830 con 0x7fb184100e60 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb183fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1841004c0 0x7fb184195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb183fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1841004c0 0x7fb184195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37424/0 (socket says 172.21.15.88:37424) 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb183fff700 1 -- 172.21.15.88:0/2990683104 learned_addr learned my addr 172.21.15.88:0/2990683104 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb1837fe700 1 --2- 172.21.15.88:0/2990683104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb184100e60 0x7fb184196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb183fff700 1 -- 172.21.15.88:0/2990683104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb184100e60 msgr2=0x7fb184196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb183fff700 1 --2- 172.21.15.88:0/2990683104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb184100e60 0x7fb184196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb183fff700 1 -- 172.21.15.88:0/2990683104 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1780095f0 con 0x7fb1841004c0 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb183fff700 1 --2- 172.21.15.88:0/2990683104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1841004c0 0x7fb184195ef0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb17400d8d0 tx=0x7fb17400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb1817fa700 1 -- 172.21.15.88:0/2990683104 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1740098a0 con 0x7fb1841004c0 2024-02-19T19:03:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb1817fa700 1 -- 172.21.15.88:0/2990683104 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb174010460 con 0x7fb1841004c0 2024-02-19T19:03:57.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb18a5d2700 1 -- 172.21.15.88:0/2990683104 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18419ab10 con 0x7fb1841004c0 2024-02-19T19:03:57.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb1817fa700 1 -- 172.21.15.88:0/2990683104 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb17400f5f0 con 0x7fb1841004c0 2024-02-19T19:03:57.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.008+0000 7fb18a5d2700 1 -- 172.21.15.88:0/2990683104 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18419b060 con 0x7fb1841004c0 2024-02-19T19:03:57.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.009+0000 7fb18a5d2700 1 -- 172.21.15.88:0/2990683104 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb184108830 con 0x7fb1841004c0 2024-02-19T19:03:57.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.010+0000 7fb1817fa700 1 -- 172.21.15.88:0/2990683104 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fb174009a00 con 0x7fb1841004c0 2024-02-19T19:03:57.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.010+0000 7fb1817fa700 1 --2- 172.21.15.88:0/2990683104 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fb16c06d520 0x7fb16c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:03:57.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.010+0000 7fb1817fa700 1 -- 172.21.15.88:0/2990683104 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb17408ce80 con 0x7fb1841004c0 2024-02-19T19:03:57.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.010+0000 7fb1837fe700 1 --2- 172.21.15.88:0/2990683104 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fb16c06d520 0x7fb16c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:03:57.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.011+0000 7fb1837fe700 1 --2- 172.21.15.88:0/2990683104 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fb16c06d520 0x7fb16c06f9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1841974a0 tx=0x7fb17801a040 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:03:57.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.015+0000 7fb1817fa700 1 -- 172.21.15.88:0/2990683104 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fb174056b60 con 0x7fb1841004c0 2024-02-19T19:03:57.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.302+0000 7fb18a5d2700 1 -- 172.21.15.88:0/2990683104 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb18419b280 con 0x7fb1841004c0 2024-02-19T19:03:57.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.302+0000 7fb1817fa700 1 -- 172.21.15.88:0/2990683104 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fb17405a220 con 0x7fb1841004c0 2024-02-19T19:03:57.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.304+0000 7fb18a5d2700 1 -- 172.21.15.88:0/2990683104 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fb16c06d520 msgr2=0x7fb16c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:57.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.304+0000 7fb18a5d2700 1 --2- 172.21.15.88:0/2990683104 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fb16c06d520 0x7fb16c06f9d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb1841974a0 tx=0x7fb17801a040 comp rx=0 tx=0).stop 2024-02-19T19:03:57.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.304+0000 7fb18a5d2700 1 -- 172.21.15.88:0/2990683104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1841004c0 msgr2=0x7fb184195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:03:57.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.304+0000 7fb18a5d2700 1 --2- 172.21.15.88:0/2990683104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1841004c0 0x7fb184195ef0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb17400d8d0 tx=0x7fb17400dbe0 comp rx=0 tx=0).stop 2024-02-19T19:03:57.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.304+0000 7fb18a5d2700 1 -- 172.21.15.88:0/2990683104 shutdown_connections 2024-02-19T19:03:57.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.304+0000 7fb18a5d2700 1 --2- 172.21.15.88:0/2990683104 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fb16c06d520 0x7fb16c06f9d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:57.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.304+0000 7fb18a5d2700 1 --2- 172.21.15.88:0/2990683104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1841004c0 0x7fb184195ef0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:57.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.304+0000 7fb18a5d2700 1 --2- 172.21.15.88:0/2990683104 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb184100e60 0x7fb184196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:03:57.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.304+0000 7fb18a5d2700 1 -- 172.21.15.88:0/2990683104 >> 172.21.15.88:0/2990683104 conn(0x7fb1840fbd80 msgr2=0x7fb1840fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:03:57.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.304+0000 7fb18a5d2700 1 -- 172.21.15.88:0/2990683104 shutdown_connections 2024-02-19T19:03:57.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:03:57.304+0000 7fb18a5d2700 1 -- 172.21.15.88:0/2990683104 wait complete. 2024-02-19T19:03:57.334 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)" 2024-02-19T19:03:57.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:57 smithi088 ceph-mon[29272]: pgmap v75: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:57.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:57 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/2990683104' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:57.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:57 smithi132 ceph-mon[37969]: pgmap v75: 65 pgs: 65 active+clean; 594 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-02-19T19:03:57.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:57 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2990683104' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:03:58.051 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-02-19T19:03:58.262 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:03:58.331 INFO:tasks.workunit.client.1.smithi132.stderr:+ tar xzf ltp-full.tgz 2024-02-19T19:03:59.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:03:59 smithi132 ceph-mon[37969]: pgmap v76: 65 pgs: 65 active+clean; 4.6 MiB data, 162 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s wr, 0 op/s 2024-02-19T19:03:59.756 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:03:59 smithi088 ceph-mon[29272]: pgmap v76: 65 pgs: 65 active+clean; 4.6 MiB data, 162 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s wr, 0 op/s 2024-02-19T19:04:00.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.346+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/18732261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d44069140 msgr2=0x7f8d44104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:00.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.346+0000 7f8d4a9c0700 1 --2- 172.21.15.88:0/18732261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d44069140 0x7f8d44104eb0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8d38009910 tx=0x7f8d38009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:00.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.346+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/18732261 shutdown_connections 2024-02-19T19:04:00.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.346+0000 7f8d4a9c0700 1 --2- 172.21.15.88:0/18732261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d44069140 0x7f8d44104eb0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:00.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.346+0000 7f8d4a9c0700 1 --2- 172.21.15.88:0/18732261 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d440687a0 0x7f8d44068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:00.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.346+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/18732261 >> 172.21.15.88:0/18732261 conn(0x7f8d44075130 msgr2=0x7f8d44075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:00.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.346+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/18732261 shutdown_connections 2024-02-19T19:04:00.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.346+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/18732261 wait complete. 2024-02-19T19:04:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.347+0000 7f8d4a9c0700 1 Processor -- start 2024-02-19T19:04:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.347+0000 7f8d4a9c0700 1 -- start start 2024-02-19T19:04:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d4a9c0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d440687a0 0x7f8d440ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d4a9c0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d44069140 0x7f8d441001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:00.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d4a9c0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d44100720 con 0x7f8d44069140 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d4a9c0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d44100860 con 0x7f8d440687a0 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d437fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d44069140 0x7f8d441001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d437fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d44069140 0x7f8d441001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37438/0 (socket says 172.21.15.88:37438) 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d437fe700 1 -- 172.21.15.88:0/3429823043 learned_addr learned my addr 172.21.15.88:0/3429823043 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d43fff700 1 --2- 172.21.15.88:0/3429823043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d440687a0 0x7f8d440ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d437fe700 1 -- 172.21.15.88:0/3429823043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d440687a0 msgr2=0x7f8d440ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d437fe700 1 --2- 172.21.15.88:0/3429823043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d440687a0 0x7f8d440ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d437fe700 1 -- 172.21.15.88:0/3429823043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d380095f0 con 0x7f8d44069140 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.348+0000 7f8d437fe700 1 --2- 172.21.15.88:0/3429823043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d44069140 0x7f8d441001e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006010 tx=0x7f8d3800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.349+0000 7f8d417fa700 1 -- 172.21.15.88:0/3429823043 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d3801d070 con 0x7f8d44069140 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.349+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/3429823043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d44100ae0 con 0x7f8d44069140 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.349+0000 7f8d417fa700 1 -- 172.21.15.88:0/3429823043 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d380050d0 con 0x7f8d44069140 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.349+0000 7f8d417fa700 1 -- 172.21.15.88:0/3429823043 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d38017400 con 0x7f8d44069140 2024-02-19T19:04:00.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.349+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/3429823043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d441a2360 con 0x7f8d44069140 2024-02-19T19:04:00.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.350+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/3429823043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d44108750 con 0x7f8d44069140 2024-02-19T19:04:00.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.350+0000 7f8d417fa700 1 -- 172.21.15.88:0/3429823043 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f8d3800faf0 con 0x7f8d44069140 2024-02-19T19:04:00.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.351+0000 7f8d417fa700 1 --2- 172.21.15.88:0/3429823043 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8d2c06d520 0x7f8d2c06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:00.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.351+0000 7f8d417fa700 1 -- 172.21.15.88:0/3429823043 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8d3808e180 con 0x7f8d44069140 2024-02-19T19:04:00.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.351+0000 7f8d43fff700 1 --2- 172.21.15.88:0/3429823043 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8d2c06d520 0x7f8d2c06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:00.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.351+0000 7f8d43fff700 1 --2- 172.21.15.88:0/3429823043 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8d2c06d520 0x7f8d2c06f9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009c60 tx=0x7f8d34009260 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:00.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.356+0000 7f8d417fa700 1 -- 172.21.15.88:0/3429823043 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f8d38057ee0 con 0x7f8d44069140 2024-02-19T19:04:00.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.717+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/3429823043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d4404f2a0 con 0x7f8d44069140 2024-02-19T19:04:00.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.717+0000 7f8d417fa700 1 -- 172.21.15.88:0/3429823043 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f8d3805b5a0 con 0x7f8d44069140 2024-02-19T19:04:00.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.720+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/3429823043 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8d2c06d520 msgr2=0x7f8d2c06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:00.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.720+0000 7f8d4a9c0700 1 --2- 172.21.15.88:0/3429823043 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8d2c06d520 0x7f8d2c06f9d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8d34009c60 tx=0x7f8d34009260 comp rx=0 tx=0).stop 2024-02-19T19:04:00.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.720+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/3429823043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d44069140 msgr2=0x7f8d441001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:00.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.720+0000 7f8d4a9c0700 1 --2- 172.21.15.88:0/3429823043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d44069140 0x7f8d441001e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f8d38006010 tx=0x7f8d3800f690 comp rx=0 tx=0).stop 2024-02-19T19:04:00.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.720+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/3429823043 shutdown_connections 2024-02-19T19:04:00.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.720+0000 7f8d4a9c0700 1 --2- 172.21.15.88:0/3429823043 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f8d2c06d520 0x7f8d2c06f9d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:00.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.720+0000 7f8d4a9c0700 1 --2- 172.21.15.88:0/3429823043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d440687a0 0x7f8d440ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:00.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.720+0000 7f8d4a9c0700 1 --2- 172.21.15.88:0/3429823043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d44069140 0x7f8d441001e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:00.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.720+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/3429823043 >> 172.21.15.88:0/3429823043 conn(0x7f8d44075130 msgr2=0x7f8d44104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:00.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.720+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/3429823043 shutdown_connections 2024-02-19T19:04:00.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:00.720+0000 7f8d4a9c0700 1 -- 172.21.15.88:0/3429823043 wait complete. 2024-02-19T19:04:00.752 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:04:01.467 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-02-19T19:04:01.711 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:04:01.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:01 smithi132 ceph-mon[37969]: pgmap v77: 65 pgs: 65 active+clean; 4.6 MiB data, 162 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s wr, 0 op/s 2024-02-19T19:04:01.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:01 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/3429823043' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:04:01.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:01 smithi088 ceph-mon[29272]: pgmap v77: 65 pgs: 65 active+clean; 4.6 MiB data, 162 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s wr, 0 op/s 2024-02-19T19:04:01.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:01 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/3429823043' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:04:03.287 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:03 smithi088 ceph-mon[29272]: pgmap v78: 65 pgs: 65 active+clean; 4.6 MiB data, 162 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s wr, 0 op/s 2024-02-19T19:04:03.287 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:03 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:04:03.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:03 smithi132 ceph-mon[37969]: pgmap v78: 65 pgs: 65 active+clean; 4.6 MiB data, 162 MiB used, 536 GiB / 536 GiB avail; 343 KiB/s wr, 0 op/s 2024-02-19T19:04:03.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:03 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:04:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.545+0000 7f3ee0da8700 1 -- 172.21.15.88:0/1923496738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3edc0687a0 msgr2=0x7f3edc068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.545+0000 7f3ee0da8700 1 --2- 172.21.15.88:0/1923496738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3edc0687a0 0x7f3edc068b70 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009910 tx=0x7f3ecc009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.546+0000 7f3ee0da8700 1 -- 172.21.15.88:0/1923496738 shutdown_connections 2024-02-19T19:04:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.546+0000 7f3ee0da8700 1 --2- 172.21.15.88:0/1923496738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3edc069140 0x7f3edc104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.546+0000 7f3ee0da8700 1 --2- 172.21.15.88:0/1923496738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3edc0687a0 0x7f3edc068b70 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.546+0000 7f3ee0da8700 1 -- 172.21.15.88:0/1923496738 >> 172.21.15.88:0/1923496738 conn(0x7f3edc075130 msgr2=0x7f3edc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.546+0000 7f3ee0da8700 1 -- 172.21.15.88:0/1923496738 shutdown_connections 2024-02-19T19:04:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.546+0000 7f3ee0da8700 1 -- 172.21.15.88:0/1923496738 wait complete. 2024-02-19T19:04:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.546+0000 7f3ee0da8700 1 Processor -- start 2024-02-19T19:04:03.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3ee0da8700 1 -- start start 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3ee0da8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3edc0687a0 0x7f3edc195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3ee0da8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3edc069140 0x7f3edc196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3ee0da8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc196af0 con 0x7f3edc069140 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3ee0da8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3edc19a880 con 0x7f3edc0687a0 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3ed9d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3edc069140 0x7f3edc196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3eda59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3edc0687a0 0x7f3edc195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3ed9d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3edc069140 0x7f3edc196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37452/0 (socket says 172.21.15.88:37452) 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3ed9d9b700 1 -- 172.21.15.88:0/2509008572 learned_addr learned my addr 172.21.15.88:0/2509008572 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3eda59c700 1 -- 172.21.15.88:0/2509008572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3edc069140 msgr2=0x7f3edc196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3eda59c700 1 --2- 172.21.15.88:0/2509008572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3edc069140 0x7f3edc196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3eda59c700 1 -- 172.21.15.88:0/2509008572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc0095f0 con 0x7f3edc0687a0 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.547+0000 7f3eda59c700 1 --2- 172.21.15.88:0/2509008572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3edc0687a0 0x7f3edc195ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009fd0 tx=0x7f3ecc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:03.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.548+0000 7f3ecb7fe700 1 -- 172.21.15.88:0/2509008572 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ecc01d070 con 0x7f3edc0687a0 2024-02-19T19:04:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.548+0000 7f3ee0da8700 1 -- 172.21.15.88:0/2509008572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3edc19ab00 con 0x7f3edc0687a0 2024-02-19T19:04:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.548+0000 7f3ecb7fe700 1 -- 172.21.15.88:0/2509008572 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ecc0050d0 con 0x7f3edc0687a0 2024-02-19T19:04:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.548+0000 7f3ecb7fe700 1 -- 172.21.15.88:0/2509008572 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ecc003e90 con 0x7f3edc0687a0 2024-02-19T19:04:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.548+0000 7f3ee0da8700 1 -- 172.21.15.88:0/2509008572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3edc19aff0 con 0x7f3edc0687a0 2024-02-19T19:04:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.549+0000 7f3ee0da8700 1 -- 172.21.15.88:0/2509008572 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3edc108790 con 0x7f3edc0687a0 2024-02-19T19:04:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.550+0000 7f3ecb7fe700 1 -- 172.21.15.88:0/2509008572 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f3ecc017400 con 0x7f3edc0687a0 2024-02-19T19:04:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.550+0000 7f3ecb7fe700 1 --2- 172.21.15.88:0/2509008572 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f3ec406d520 0x7f3ec406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.550+0000 7f3ecb7fe700 1 -- 172.21.15.88:0/2509008572 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3ecc08e390 con 0x7f3edc0687a0 2024-02-19T19:04:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.550+0000 7f3ed9d9b700 1 --2- 172.21.15.88:0/2509008572 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f3ec406d520 0x7f3ec406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.551+0000 7f3ed9d9b700 1 --2- 172.21.15.88:0/2509008572 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f3ec406d520 0x7f3ec406f9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3edc1974f0 tx=0x7f3ed0009260 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:03.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.556+0000 7f3ecb7fe700 1 -- 172.21.15.88:0/2509008572 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f3ecc0580f0 con 0x7f3edc0687a0 2024-02-19T19:04:03.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:03.821+0000 7f3ee0da8700 1 -- 172.21.15.88:0/2509008572 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}) v1 -- 0x7f3edc19b210 con 0x7f3ec406d520 2024-02-19T19:04:05.423 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:05 smithi088 ceph-mon[29272]: from='client.24511 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:05.424 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:05 smithi088 ceph-mon[29272]: pgmap v79: 65 pgs: 65 active+clean; 21 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s wr, 149 op/s 2024-02-19T19:04:05.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:05 smithi132 ceph-mon[37969]: from='client.24511 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:05.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:05 smithi132 ceph-mon[37969]: pgmap v79: 65 pgs: 65 active+clean; 21 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s wr, 149 op/s 2024-02-19T19:04:06.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.491+0000 7f3ecb7fe700 1 -- 172.21.15.88:0/2509008572 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7f3edc19b210 con 0x7f3ec406d520 2024-02-19T19:04:06.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.494+0000 7f3ee0da8700 1 -- 172.21.15.88:0/2509008572 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f3ec406d520 msgr2=0x7f3ec406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:06.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.494+0000 7f3ee0da8700 1 --2- 172.21.15.88:0/2509008572 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f3ec406d520 0x7f3ec406f9d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3edc1974f0 tx=0x7f3ed0009260 comp rx=0 tx=0).stop 2024-02-19T19:04:06.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.494+0000 7f3ee0da8700 1 -- 172.21.15.88:0/2509008572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3edc0687a0 msgr2=0x7f3edc195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:06.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.494+0000 7f3ee0da8700 1 --2- 172.21.15.88:0/2509008572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3edc0687a0 0x7f3edc195ed0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009fd0 tx=0x7f3ecc00f690 comp rx=0 tx=0).stop 2024-02-19T19:04:06.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.494+0000 7f3ee0da8700 1 -- 172.21.15.88:0/2509008572 shutdown_connections 2024-02-19T19:04:06.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.494+0000 7f3ee0da8700 1 --2- 172.21.15.88:0/2509008572 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f3ec406d520 0x7f3ec406f9d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:06.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.494+0000 7f3ee0da8700 1 --2- 172.21.15.88:0/2509008572 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3edc0687a0 0x7f3edc195ed0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:06.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.494+0000 7f3ee0da8700 1 --2- 172.21.15.88:0/2509008572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3edc069140 0x7f3edc196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:06.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.494+0000 7f3ee0da8700 1 -- 172.21.15.88:0/2509008572 >> 172.21.15.88:0/2509008572 conn(0x7f3edc075130 msgr2=0x7f3edc0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:06.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.494+0000 7f3ee0da8700 1 -- 172.21.15.88:0/2509008572 shutdown_connections 2024-02-19T19:04:06.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:06.494+0000 7f3ee0da8700 1 -- 172.21.15.88:0/2509008572 wait complete. 2024-02-19T19:04:06.525 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:04:07.355 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-19T19:04:07.595 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:04:07.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:07 smithi088 ceph-mon[29272]: pgmap v80: 65 pgs: 65 active+clean; 21 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s wr, 149 op/s 2024-02-19T19:04:07.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:07 smithi132 ceph-mon[37969]: pgmap v80: 65 pgs: 65 active+clean; 21 MiB data, 252 MiB used, 536 GiB / 536 GiB avail; 1.8 MiB/s wr, 149 op/s 2024-02-19T19:04:09.559 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:09 smithi088 ceph-mon[29272]: pgmap v81: 65 pgs: 65 active+clean; 37 MiB data, 321 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s wr, 293 op/s 2024-02-19T19:04:09.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:09 smithi132 ceph-mon[37969]: pgmap v81: 65 pgs: 65 active+clean; 37 MiB data, 321 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s wr, 293 op/s 2024-02-19T19:04:10.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.993+0000 7f42ed0e4700 1 -- 172.21.15.88:0/1296240621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42e8100e80 msgr2=0x7f42e8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:10.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.993+0000 7f42ed0e4700 1 --2- 172.21.15.88:0/1296240621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42e8100e80 0x7f42e8104e50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f42dc009910 tx=0x7f42dc009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:10.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.993+0000 7f42ed0e4700 1 -- 172.21.15.88:0/1296240621 shutdown_connections 2024-02-19T19:04:10.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.993+0000 7f42ed0e4700 1 --2- 172.21.15.88:0/1296240621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42e8100e80 0x7f42e8104e50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:10.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.993+0000 7f42ed0e4700 1 --2- 172.21.15.88:0/1296240621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42e81004e0 0x7f42e81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:10.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.993+0000 7f42ed0e4700 1 -- 172.21.15.88:0/1296240621 >> 172.21.15.88:0/1296240621 conn(0x7f42e80fbda0 msgr2=0x7f42e80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:10.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.993+0000 7f42ed0e4700 1 -- 172.21.15.88:0/1296240621 shutdown_connections 2024-02-19T19:04:10.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.993+0000 7f42ed0e4700 1 -- 172.21.15.88:0/1296240621 wait complete. 2024-02-19T19:04:10.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.994+0000 7f42ed0e4700 1 Processor -- start 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.994+0000 7f42ed0e4700 1 -- start start 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.994+0000 7f42ed0e4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42e81004e0 0x7f42e8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.994+0000 7f42ed0e4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42e8100e80 0x7f42e8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.994+0000 7f42ed0e4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e8073ad0 con 0x7f42e81004e0 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.994+0000 7f42ed0e4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e8073c10 con 0x7f42e8100e80 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.994+0000 7f42e6d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42e81004e0 0x7f42e8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.994+0000 7f42e6d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42e81004e0 0x7f42e8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52320/0 (socket says 172.21.15.88:52320) 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.994+0000 7f42e6d9d700 1 -- 172.21.15.88:0/2732773998 learned_addr learned my addr 172.21.15.88:0/2732773998 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.994+0000 7f42e659c700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42e8100e80 0x7f42e8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.995+0000 7f42e6d9d700 1 -- 172.21.15.88:0/2732773998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42e8100e80 msgr2=0x7f42e8073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.995+0000 7f42e6d9d700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42e8100e80 0x7f42e8073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.995+0000 7f42e6d9d700 1 -- 172.21.15.88:0/2732773998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc0095f0 con 0x7f42e81004e0 2024-02-19T19:04:10.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.995+0000 7f42e659c700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42e8100e80 0x7f42e8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:04:10.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.995+0000 7f42e6d9d700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42e81004e0 0x7f42e8073050 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f42d800e920 tx=0x7f42d800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:10.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.995+0000 7f42cffff700 1 -- 172.21.15.88:0/2732773998 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d800c9d0 con 0x7f42e81004e0 2024-02-19T19:04:10.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.995+0000 7f42cffff700 1 -- 172.21.15.88:0/2732773998 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42d8010430 con 0x7f42e81004e0 2024-02-19T19:04:10.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.995+0000 7f42ed0e4700 1 -- 172.21.15.88:0/2732773998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e81a2100 con 0x7f42e81004e0 2024-02-19T19:04:10.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.995+0000 7f42cffff700 1 -- 172.21.15.88:0/2732773998 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42d8018630 con 0x7f42e81004e0 2024-02-19T19:04:10.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.995+0000 7f42ed0e4700 1 -- 172.21.15.88:0/2732773998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e81a24b0 con 0x7f42e81004e0 2024-02-19T19:04:10.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.996+0000 7f42ed0e4700 1 -- 172.21.15.88:0/2732773998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e8108850 con 0x7f42e81004e0 2024-02-19T19:04:10.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.997+0000 7f42cffff700 1 -- 172.21.15.88:0/2732773998 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f42d80188f0 con 0x7f42e81004e0 2024-02-19T19:04:10.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.997+0000 7f42cffff700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f42d006d520 0x7f42d006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:10.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.997+0000 7f42cffff700 1 -- 172.21.15.88:0/2732773998 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f42d8014070 con 0x7f42e81004e0 2024-02-19T19:04:10.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.997+0000 7f42e659c700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f42d006d520 0x7f42d006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:10.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:09.998+0000 7f42e659c700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f42d006d520 0x7f42d006f9d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0059d0 tx=0x7f42dc005940 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:10.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.002+0000 7f42cffff700 1 -- 172.21.15.88:0/2732773998 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f42d8052a50 con 0x7f42e81004e0 2024-02-19T19:04:10.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.249+0000 7f42ed0e4700 1 -- 172.21.15.88:0/2732773998 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f42e80746f0 con 0x7f42d006d520 2024-02-19T19:04:10.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.263+0000 7f42cffff700 1 -- 172.21.15.88:0/2732773998 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f42dc01e040 con 0x7f42d006d520 2024-02-19T19:04:10.283 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:04:10.283 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (81s) 36s ago 11m 15.7M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:04:10.283 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (12m) 36s ago 12m 7843k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:04:10.283 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (10m) 114s ago 10m 7608k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:04:10.283 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (12m) 36s ago 12m 6665k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:04:10.283 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (10m) 114s ago 10m 6656k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:04:10.283 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (42s) 36s ago 11m 57.2M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (5m) 36s ago 5m 14.5M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (5m) 36s ago 5m 26.4M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (5m) 114s ago 5m 23.1M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (5m) 114s ago 5m 17.6M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (2m) 36s ago 13m 499M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (2m) 114s ago 10m 449M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (13m) 36s ago 13m 55.6M 2048M 18.2.0 dc2bc1663786 3ea5108455bc 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (10m) 114s ago 9m 43.1M 2048M 18.2.0 dc2bc1663786 ba1c249c6017 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (2m) 36s ago 12m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (118s) 114s ago 10m 3573k - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (9m) 36s ago 9m 52.2M 1216M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:04:10.284 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (8m) 36s ago 8m 49.8M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:04:10.285 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (8m) 36s ago 8m 49.3M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:04:10.285 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (7m) 114s ago 7m 50.5M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:04:10.285 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (7m) 114s ago 7m 49.1M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:04:10.285 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (7m) 114s ago 7m 47.2M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:04:10.285 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (99s) 36s ago 10m 54.5M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:04:10.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.265+0000 7f42ed0e4700 1 -- 172.21.15.88:0/2732773998 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f42d006d520 msgr2=0x7f42d006f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:10.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.265+0000 7f42ed0e4700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f42d006d520 0x7f42d006f9d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f42dc0059d0 tx=0x7f42dc005940 comp rx=0 tx=0).stop 2024-02-19T19:04:10.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.265+0000 7f42ed0e4700 1 -- 172.21.15.88:0/2732773998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42e81004e0 msgr2=0x7f42e8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:10.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.265+0000 7f42ed0e4700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42e81004e0 0x7f42e8073050 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f42d800e920 tx=0x7f42d800ec30 comp rx=0 tx=0).stop 2024-02-19T19:04:10.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.266+0000 7f42ed0e4700 1 -- 172.21.15.88:0/2732773998 shutdown_connections 2024-02-19T19:04:10.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.266+0000 7f42ed0e4700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f42d006d520 0x7f42d006f9d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:10.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.266+0000 7f42ed0e4700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42e81004e0 0x7f42e8073050 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:10.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.266+0000 7f42ed0e4700 1 --2- 172.21.15.88:0/2732773998 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42e8100e80 0x7f42e8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:10.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.266+0000 7f42ed0e4700 1 -- 172.21.15.88:0/2732773998 >> 172.21.15.88:0/2732773998 conn(0x7f42e80fbda0 msgr2=0x7f42e80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:10.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.266+0000 7f42ed0e4700 1 -- 172.21.15.88:0/2732773998 shutdown_connections 2024-02-19T19:04:10.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:10.266+0000 7f42ed0e4700 1 -- 172.21.15.88:0/2732773998 wait complete. 2024-02-19T19:04:10.985 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-19T19:04:10.986 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T19:04:10.987 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-02-19T19:04:11.231 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:04:11.623 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:11 smithi088 ceph-mon[29272]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:11.623 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:11 smithi088 ceph-mon[29272]: pgmap v82: 65 pgs: 65 active+clean; 37 MiB data, 321 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s wr, 293 op/s 2024-02-19T19:04:11.623 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:11 smithi088 ceph-mon[29272]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:04:11.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:11 smithi132 ceph-mon[37969]: from='client.14814 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:11.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:11 smithi132 ceph-mon[37969]: pgmap v82: 65 pgs: 65 active+clean; 37 MiB data, 321 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s wr, 293 op/s 2024-02-19T19:04:11.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:11 smithi132 ceph-mon[37969]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:04:12.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.959+0000 7f55030e1700 1 -- 172.21.15.88:0/2957252528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54fc068540 msgr2=0x7f54fc0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.959+0000 7f55030e1700 1 --2- 172.21.15.88:0/2957252528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54fc068540 0x7f54fc0689b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009910 tx=0x7f54e8009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.959+0000 7f55030e1700 1 -- 172.21.15.88:0/2957252528 shutdown_connections 2024-02-19T19:04:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.959+0000 7f55030e1700 1 --2- 172.21.15.88:0/2957252528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54fc068540 0x7f54fc0689b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.959+0000 7f55030e1700 1 --2- 172.21.15.88:0/2957252528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54fc105230 0x7f54fc105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.959+0000 7f55030e1700 1 -- 172.21.15.88:0/2957252528 >> 172.21.15.88:0/2957252528 conn(0x7f54fc075130 msgr2=0x7f54fc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.960+0000 7f55030e1700 1 -- 172.21.15.88:0/2957252528 shutdown_connections 2024-02-19T19:04:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.960+0000 7f55030e1700 1 -- 172.21.15.88:0/2957252528 wait complete. 2024-02-19T19:04:12.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.960+0000 7f55030e1700 1 Processor -- start 2024-02-19T19:04:12.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.960+0000 7f55030e1700 1 -- start start 2024-02-19T19:04:12.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f55030e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54fc068540 0x7f54fc195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:12.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f55030e1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54fc105230 0x7f54fc1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:12.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f55030e1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc1969c0 con 0x7f54fc105230 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f55030e1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc19a750 con 0x7f54fc068540 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f54fbfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54fc105230 0x7f54fc1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f5500e7d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54fc068540 0x7f54fc195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f5500e7d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54fc068540 0x7f54fc195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:50668/0 (socket says 172.21.15.88:50668) 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f5500e7d700 1 -- 172.21.15.88:0/4173297195 learned_addr learned my addr 172.21.15.88:0/4173297195 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f54fbfff700 1 -- 172.21.15.88:0/4173297195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54fc068540 msgr2=0x7f54fc195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f54fbfff700 1 --2- 172.21.15.88:0/4173297195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54fc068540 0x7f54fc195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f54fbfff700 1 -- 172.21.15.88:0/4173297195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e80095f0 con 0x7f54fc105230 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.961+0000 7f54fbfff700 1 --2- 172.21.15.88:0/4173297195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54fc105230 0x7f54fc1962e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009fd0 tx=0x7f54e800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.962+0000 7f54f9ffb700 1 -- 172.21.15.88:0/4173297195 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54e801d070 con 0x7f54fc105230 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.962+0000 7f55030e1700 1 -- 172.21.15.88:0/4173297195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc19a9d0 con 0x7f54fc105230 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.962+0000 7f54f9ffb700 1 -- 172.21.15.88:0/4173297195 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54e800b610 con 0x7f54fc105230 2024-02-19T19:04:12.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.962+0000 7f54f9ffb700 1 -- 172.21.15.88:0/4173297195 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54e8022470 con 0x7f54fc105230 2024-02-19T19:04:12.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.962+0000 7f55030e1700 1 -- 172.21.15.88:0/4173297195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc19ae90 con 0x7f54fc105230 2024-02-19T19:04:12.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.963+0000 7f55030e1700 1 -- 172.21.15.88:0/4173297195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc04f2a0 con 0x7f54fc105230 2024-02-19T19:04:12.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.963+0000 7f54f9ffb700 1 -- 172.21.15.88:0/4173297195 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f54e800fac0 con 0x7f54fc105230 2024-02-19T19:04:12.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.964+0000 7f54f9ffb700 1 --2- 172.21.15.88:0/4173297195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f54ec06d520 0x7f54ec06f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:12.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.964+0000 7f54f9ffb700 1 -- 172.21.15.88:0/4173297195 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f54e808e430 con 0x7f54fc105230 2024-02-19T19:04:12.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.964+0000 7f5500e7d700 1 --2- 172.21.15.88:0/4173297195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f54ec06d520 0x7f54ec06f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:12.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.965+0000 7f5500e7d700 1 --2- 172.21.15.88:0/4173297195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f54ec06d520 0x7f54ec06f9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009d20 tx=0x7f54f0009260 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:12.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:12.970+0000 7f54f9ffb700 1 -- 172.21.15.88:0/4173297195 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f54e8058110 con 0x7f54fc105230 2024-02-19T19:04:13.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.195+0000 7f55030e1700 1 -- 172.21.15.88:0/4173297195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f54fc197100 con 0x7f54fc105230 2024-02-19T19:04:13.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.200+0000 7f54f9ffb700 1 -- 172.21.15.88:0/4173297195 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v44)=0 v44) v1 ==== 125+0+0 (secure 0 0 0) 0x7f54e805b7d0 con 0x7f54fc105230 2024-02-19T19:04:13.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.202+0000 7f55030e1700 1 -- 172.21.15.88:0/4173297195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f54ec06d520 msgr2=0x7f54ec06f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:13.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.202+0000 7f55030e1700 1 --2- 172.21.15.88:0/4173297195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f54ec06d520 0x7f54ec06f9d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f54f0009d20 tx=0x7f54f0009260 comp rx=0 tx=0).stop 2024-02-19T19:04:13.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.202+0000 7f55030e1700 1 -- 172.21.15.88:0/4173297195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54fc105230 msgr2=0x7f54fc1962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:13.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.202+0000 7f55030e1700 1 --2- 172.21.15.88:0/4173297195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54fc105230 0x7f54fc1962e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f54e8009fd0 tx=0x7f54e800f690 comp rx=0 tx=0).stop 2024-02-19T19:04:13.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.202+0000 7f55030e1700 1 -- 172.21.15.88:0/4173297195 shutdown_connections 2024-02-19T19:04:13.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.202+0000 7f55030e1700 1 --2- 172.21.15.88:0/4173297195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f54ec06d520 0x7f54ec06f9d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:13.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.202+0000 7f55030e1700 1 --2- 172.21.15.88:0/4173297195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54fc068540 0x7f54fc195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:13.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.202+0000 7f55030e1700 1 --2- 172.21.15.88:0/4173297195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54fc105230 0x7f54fc1962e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:13.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.202+0000 7f55030e1700 1 -- 172.21.15.88:0/4173297195 >> 172.21.15.88:0/4173297195 conn(0x7f54fc075130 msgr2=0x7f54fc0fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:13.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.203+0000 7f55030e1700 1 -- 172.21.15.88:0/4173297195 shutdown_connections 2024-02-19T19:04:13.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:13.203+0000 7f55030e1700 1 -- 172.21.15.88:0/4173297195 wait complete. 2024-02-19T19:04:13.237 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:13 smithi088 ceph-mon[29272]: pgmap v83: 65 pgs: 65 active+clean; 37 MiB data, 321 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s wr, 293 op/s 2024-02-19T19:04:13.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:13 smithi132 ceph-mon[37969]: pgmap v83: 65 pgs: 65 active+clean; 37 MiB data, 321 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s wr, 293 op/s 2024-02-19T19:04:13.917 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-19T19:04:13.948 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T19:04:13.949 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-02-19T19:04:14.188 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:04:14.491 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:14 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/4173297195' entity='client.admin' 2024-02-19T19:04:14.491 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:14 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:04:14.491 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:14 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:14.491 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:14 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:14.491 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:14 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:04:14.491 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:14 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:04:14.491 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:14 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:14.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:14 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/4173297195' entity='client.admin' 2024-02-19T19:04:14.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:14 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:04:14.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:14 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:14.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:14 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:14.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:14 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:04:14.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:14 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:04:14.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:14 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:15.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:15 smithi132 ceph-mon[37969]: pgmap v84: 65 pgs: 65 active+clean; 51 MiB data, 404 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s wr, 418 op/s 2024-02-19T19:04:15.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:15 smithi088 ceph-mon[29272]: pgmap v84: 65 pgs: 65 active+clean; 51 MiB data, 404 MiB used, 536 GiB / 536 GiB avail; 4.1 MiB/s wr, 418 op/s 2024-02-19T19:04:17.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:17 smithi132 ceph-mon[37969]: pgmap v85: 65 pgs: 65 active+clean; 51 MiB data, 404 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s wr, 270 op/s 2024-02-19T19:04:17.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:17 smithi088 ceph-mon[29272]: pgmap v85: 65 pgs: 65 active+clean; 51 MiB data, 404 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s wr, 270 op/s 2024-02-19T19:04:18.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:18 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:04:18.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:18 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:04:19.652 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:19 smithi132 ceph-mon[37969]: pgmap v86: 65 pgs: 65 active+clean; 62 MiB data, 457 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s wr, 399 op/s 2024-02-19T19:04:19.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:19 smithi088 ceph-mon[29272]: pgmap v86: 65 pgs: 65 active+clean; 62 MiB data, 457 MiB used, 536 GiB / 536 GiB avail; 3.7 MiB/s wr, 399 op/s 2024-02-19T19:04:20.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.770+0000 7fddcc2a7700 1 -- 172.21.15.88:0/680325945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddc41038b0 msgr2=0x7fddc4103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:20.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.770+0000 7fddcc2a7700 1 --2- 172.21.15.88:0/680325945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddc41038b0 0x7fddc4103c80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009910 tx=0x7fddc0009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:20.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.770+0000 7fddcc2a7700 1 -- 172.21.15.88:0/680325945 shutdown_connections 2024-02-19T19:04:20.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.770+0000 7fddcc2a7700 1 --2- 172.21.15.88:0/680325945 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddc40ff350 0x7fddc40ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:20.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.770+0000 7fddcc2a7700 1 --2- 172.21.15.88:0/680325945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddc41038b0 0x7fddc4103c80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:20.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.770+0000 7fddcc2a7700 1 -- 172.21.15.88:0/680325945 >> 172.21.15.88:0/680325945 conn(0x7fddc40748b0 msgr2=0x7fddc4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:20.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.770+0000 7fddcc2a7700 1 -- 172.21.15.88:0/680325945 shutdown_connections 2024-02-19T19:04:20.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.770+0000 7fddcc2a7700 1 -- 172.21.15.88:0/680325945 wait complete. 2024-02-19T19:04:20.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.771+0000 7fddcc2a7700 1 Processor -- start 2024-02-19T19:04:20.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.771+0000 7fddcc2a7700 1 -- start start 2024-02-19T19:04:20.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.771+0000 7fddcc2a7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddc40ff350 0x7fddc4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:20.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.771+0000 7fddcc2a7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddc41038b0 0x7fddc4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:20.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.771+0000 7fddcc2a7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc4198c50 con 0x7fddc41038b0 2024-02-19T19:04:20.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.771+0000 7fddcc2a7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc419c9e0 con 0x7fddc40ff350 2024-02-19T19:04:20.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddc9842700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddc41038b0 0x7fddc4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:20.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddc9842700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddc41038b0 0x7fddc4198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59316/0 (socket says 172.21.15.88:59316) 2024-02-19T19:04:20.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddc9842700 1 -- 172.21.15.88:0/2772167195 learned_addr learned my addr 172.21.15.88:0/2772167195 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:20.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddca043700 1 --2- 172.21.15.88:0/2772167195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddc40ff350 0x7fddc4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:20.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddc9842700 1 -- 172.21.15.88:0/2772167195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddc40ff350 msgr2=0x7fddc4198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:20.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddc9842700 1 --2- 172.21.15.88:0/2772167195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddc40ff350 0x7fddc4198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:20.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddc9842700 1 -- 172.21.15.88:0/2772167195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc00095f0 con 0x7fddc41038b0 2024-02-19T19:04:20.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddc9842700 1 --2- 172.21.15.88:0/2772167195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddc41038b0 0x7fddc4198570 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fddb400e920 tx=0x7fddb400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:20.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddbb7fe700 1 -- 172.21.15.88:0/2772167195 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddb400c9d0 con 0x7fddc41038b0 2024-02-19T19:04:20.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddcc2a7700 1 -- 172.21.15.88:0/2772167195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc419ccc0 con 0x7fddc41038b0 2024-02-19T19:04:20.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddbb7fe700 1 -- 172.21.15.88:0/2772167195 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddb4010430 con 0x7fddc41038b0 2024-02-19T19:04:20.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddbb7fe700 1 -- 172.21.15.88:0/2772167195 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddb4018630 con 0x7fddc41038b0 2024-02-19T19:04:20.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.772+0000 7fddcc2a7700 1 -- 172.21.15.88:0/2772167195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc419d210 con 0x7fddc41038b0 2024-02-19T19:04:20.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.774+0000 7fddcc2a7700 1 -- 172.21.15.88:0/2772167195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc404f2a0 con 0x7fddc41038b0 2024-02-19T19:04:20.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.774+0000 7fddbb7fe700 1 -- 172.21.15.88:0/2772167195 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fddb40105a0 con 0x7fddc41038b0 2024-02-19T19:04:20.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.775+0000 7fddbb7fe700 1 --2- 172.21.15.88:0/2772167195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fddb006d520 0x7fddb006f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:20.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.775+0000 7fddbb7fe700 1 -- 172.21.15.88:0/2772167195 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fddb4014070 con 0x7fddc41038b0 2024-02-19T19:04:20.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.775+0000 7fddca043700 1 --2- 172.21.15.88:0/2772167195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fddb006d520 0x7fddb006f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:20.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.775+0000 7fddca043700 1 --2- 172.21.15.88:0/2772167195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fddb006d520 0x7fddb006f9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fddc0000c00 tx=0x7fddc0009f70 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:20.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:20.780+0000 7fddbb7fe700 1 -- 172.21.15.88:0/2772167195 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fddb4052a50 con 0x7fddc41038b0 2024-02-19T19:04:21.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.018+0000 7fddcc2a7700 1 -- 172.21.15.88:0/2772167195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fddc419ce50 con 0x7fddc41038b0 2024-02-19T19:04:21.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.018+0000 7fddbb7fe700 1 -- 172.21.15.88:0/2772167195 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v44)=0 v44) v1 ==== 155+0+0 (secure 0 0 0) 0x7fddc419ce50 con 0x7fddc41038b0 2024-02-19T19:04:21.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.020+0000 7fddcc2a7700 1 -- 172.21.15.88:0/2772167195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fddb006d520 msgr2=0x7fddb006f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:21.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.020+0000 7fddcc2a7700 1 --2- 172.21.15.88:0/2772167195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fddb006d520 0x7fddb006f9d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fddc0000c00 tx=0x7fddc0009f70 comp rx=0 tx=0).stop 2024-02-19T19:04:21.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.020+0000 7fddcc2a7700 1 -- 172.21.15.88:0/2772167195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddc41038b0 msgr2=0x7fddc4198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:21.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.020+0000 7fddcc2a7700 1 --2- 172.21.15.88:0/2772167195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddc41038b0 0x7fddc4198570 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fddb400e920 tx=0x7fddb400ec30 comp rx=0 tx=0).stop 2024-02-19T19:04:21.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.021+0000 7fddcc2a7700 1 -- 172.21.15.88:0/2772167195 shutdown_connections 2024-02-19T19:04:21.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.021+0000 7fddcc2a7700 1 --2- 172.21.15.88:0/2772167195 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fddb006d520 0x7fddb006f9d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:21.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.021+0000 7fddcc2a7700 1 --2- 172.21.15.88:0/2772167195 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fddc40ff350 0x7fddc4198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:21.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.021+0000 7fddcc2a7700 1 --2- 172.21.15.88:0/2772167195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddc41038b0 0x7fddc4198570 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:21.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.021+0000 7fddcc2a7700 1 -- 172.21.15.88:0/2772167195 >> 172.21.15.88:0/2772167195 conn(0x7fddc40748b0 msgr2=0x7fddc40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:21.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.021+0000 7fddcc2a7700 1 -- 172.21.15.88:0/2772167195 shutdown_connections 2024-02-19T19:04:21.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:21.021+0000 7fddcc2a7700 1 -- 172.21.15.88:0/2772167195 wait complete. 2024-02-19T19:04:21.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:21 smithi132 ceph-mon[37969]: pgmap v87: 65 pgs: 65 active+clean; 62 MiB data, 457 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s wr, 254 op/s 2024-02-19T19:04:21.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:21 smithi088 ceph-mon[29272]: pgmap v87: 65 pgs: 65 active+clean; 62 MiB data, 457 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s wr, 254 op/s 2024-02-19T19:04:22.035 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-02-19T19:04:22.274 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:04:23.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:23 smithi088 ceph-mon[29272]: pgmap v88: 65 pgs: 65 active+clean; 62 MiB data, 457 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s wr, 254 op/s 2024-02-19T19:04:23.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:23 smithi132 ceph-mon[37969]: pgmap v88: 65 pgs: 65 active+clean; 62 MiB data, 457 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s wr, 254 op/s 2024-02-19T19:04:24.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.376+0000 7f9a20979700 1 -- 172.21.15.88:0/4041969488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c0ffbd0 msgr2=0x7f9a1c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:24.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.376+0000 7f9a20979700 1 --2- 172.21.15.88:0/4041969488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c0ffbd0 0x7f9a1c100040 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009910 tx=0x7f9a04009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:24.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.376+0000 7f9a20979700 1 -- 172.21.15.88:0/4041969488 shutdown_connections 2024-02-19T19:04:24.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.376+0000 7f9a20979700 1 --2- 172.21.15.88:0/4041969488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c0ffbd0 0x7f9a1c100040 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:24.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.376+0000 7f9a20979700 1 --2- 172.21.15.88:0/4041969488 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c104130 0x7f9a1c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:24.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.376+0000 7f9a20979700 1 -- 172.21.15.88:0/4041969488 >> 172.21.15.88:0/4041969488 conn(0x7f9a1c075130 msgr2=0x7f9a1c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:24.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.376+0000 7f9a20979700 1 -- 172.21.15.88:0/4041969488 shutdown_connections 2024-02-19T19:04:24.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.376+0000 7f9a20979700 1 -- 172.21.15.88:0/4041969488 wait complete. 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.377+0000 7f9a20979700 1 Processor -- start 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.377+0000 7f9a20979700 1 -- start start 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.377+0000 7f9a20979700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c0ffbd0 0x7f9a1c194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.377+0000 7f9a20979700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c104130 0x7f9a1c195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.377+0000 7f9a20979700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c195950 con 0x7f9a1c104130 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.377+0000 7f9a20979700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1c1996e0 con 0x7f9a1c0ffbd0 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.377+0000 7f9a19d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c104130 0x7f9a1c195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.377+0000 7f9a1a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c0ffbd0 0x7f9a1c194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.377+0000 7f9a19d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c104130 0x7f9a1c195270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59330/0 (socket says 172.21.15.88:59330) 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.377+0000 7f9a19d9b700 1 -- 172.21.15.88:0/1492439126 learned_addr learned my addr 172.21.15.88:0/1492439126 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.377+0000 7f9a1a59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c0ffbd0 0x7f9a1c194d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:45962/0 (socket says 172.21.15.88:45962) 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.378+0000 7f9a19d9b700 1 -- 172.21.15.88:0/1492439126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c0ffbd0 msgr2=0x7f9a1c194d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.378+0000 7f9a19d9b700 1 --2- 172.21.15.88:0/1492439126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c0ffbd0 0x7f9a1c194d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:24.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.378+0000 7f9a19d9b700 1 -- 172.21.15.88:0/1492439126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a040095f0 con 0x7f9a1c104130 2024-02-19T19:04:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.378+0000 7f9a19d9b700 1 --2- 172.21.15.88:0/1492439126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c104130 0x7f9a1c195270 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400bd70 tx=0x7f9a0400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.378+0000 7f9a137fe700 1 -- 172.21.15.88:0/1492439126 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a0401d070 con 0x7f9a1c104130 2024-02-19T19:04:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.378+0000 7f9a137fe700 1 -- 172.21.15.88:0/1492439126 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a04005dc0 con 0x7f9a1c104130 2024-02-19T19:04:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.378+0000 7f9a20979700 1 -- 172.21.15.88:0/1492439126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a1c199960 con 0x7f9a1c104130 2024-02-19T19:04:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.378+0000 7f9a137fe700 1 -- 172.21.15.88:0/1492439126 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a04017400 con 0x7f9a1c104130 2024-02-19T19:04:24.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.378+0000 7f9a20979700 1 -- 172.21.15.88:0/1492439126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1c199e50 con 0x7f9a1c104130 2024-02-19T19:04:24.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.380+0000 7f9a20979700 1 -- 172.21.15.88:0/1492439126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a1c04f2a0 con 0x7f9a1c104130 2024-02-19T19:04:24.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.380+0000 7f9a137fe700 1 -- 172.21.15.88:0/1492439126 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f9a040175c0 con 0x7f9a1c104130 2024-02-19T19:04:24.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.381+0000 7f9a137fe700 1 --2- 172.21.15.88:0/1492439126 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9a0806d520 0x7f9a0806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:24.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.381+0000 7f9a137fe700 1 -- 172.21.15.88:0/1492439126 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9a0408e430 con 0x7f9a1c104130 2024-02-19T19:04:24.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.381+0000 7f9a1a59c700 1 --2- 172.21.15.88:0/1492439126 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9a0806d520 0x7f9a0806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:24.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.381+0000 7f9a1a59c700 1 --2- 172.21.15.88:0/1492439126 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9a0806d520 0x7f9a0806f9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9a14005950 tx=0x7f9a1400b410 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:24.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.388+0000 7f9a137fe700 1 -- 172.21.15.88:0/1492439126 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f9a0400f460 con 0x7f9a1c104130 2024-02-19T19:04:24.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.667+0000 7f9a20979700 1 -- 172.21.15.88:0/1492439126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f9a1c19a130 con 0x7f9a1c104130 2024-02-19T19:04:24.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.668+0000 7f9a137fe700 1 -- 172.21.15.88:0/1492439126 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v44)=0 v44) v1 ==== 163+0+0 (secure 0 0 0) 0x7f9a0405bc00 con 0x7f9a1c104130 2024-02-19T19:04:24.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.670+0000 7f9a20979700 1 -- 172.21.15.88:0/1492439126 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9a0806d520 msgr2=0x7f9a0806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:24.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.670+0000 7f9a20979700 1 --2- 172.21.15.88:0/1492439126 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9a0806d520 0x7f9a0806f9d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9a14005950 tx=0x7f9a1400b410 comp rx=0 tx=0).stop 2024-02-19T19:04:24.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.670+0000 7f9a20979700 1 -- 172.21.15.88:0/1492439126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c104130 msgr2=0x7f9a1c195270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:24.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.670+0000 7f9a20979700 1 --2- 172.21.15.88:0/1492439126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c104130 0x7f9a1c195270 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400bd70 tx=0x7f9a0400bda0 comp rx=0 tx=0).stop 2024-02-19T19:04:24.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.671+0000 7f9a20979700 1 -- 172.21.15.88:0/1492439126 shutdown_connections 2024-02-19T19:04:24.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.671+0000 7f9a20979700 1 --2- 172.21.15.88:0/1492439126 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f9a0806d520 0x7f9a0806f9d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:24.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.671+0000 7f9a20979700 1 --2- 172.21.15.88:0/1492439126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a1c0ffbd0 0x7f9a1c194d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:24.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.671+0000 7f9a20979700 1 --2- 172.21.15.88:0/1492439126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a1c104130 0x7f9a1c195270 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:24.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.671+0000 7f9a20979700 1 -- 172.21.15.88:0/1492439126 >> 172.21.15.88:0/1492439126 conn(0x7f9a1c075130 msgr2=0x7f9a1c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:24.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.671+0000 7f9a20979700 1 -- 172.21.15.88:0/1492439126 shutdown_connections 2024-02-19T19:04:24.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:24.671+0000 7f9a20979700 1 -- 172.21.15.88:0/1492439126 wait complete. 2024-02-19T19:04:25.621 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:25 smithi088 ceph-mon[29272]: pgmap v89: 65 pgs: 65 active+clean; 77 MiB data, 472 MiB used, 536 GiB / 536 GiB avail; 3.6 MiB/s wr, 363 op/s 2024-02-19T19:04:25.645 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph config set global log_to_journald false --force' 2024-02-19T19:04:25.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:25 smithi132 ceph-mon[37969]: pgmap v89: 65 pgs: 65 active+clean; 77 MiB data, 472 MiB used, 536 GiB / 536 GiB avail; 3.6 MiB/s wr, 363 op/s 2024-02-19T19:04:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:04:27.686 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:27 smithi088 ceph-mon[29272]: pgmap v90: 65 pgs: 65 active+clean; 77 MiB data, 472 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s wr, 238 op/s 2024-02-19T19:04:27.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.727+0000 7fd373b7c700 1 -- 172.21.15.88:0/3441916675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd36c100460 msgr2=0x7fd36c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:27.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.727+0000 7fd373b7c700 1 --2- 172.21.15.88:0/3441916675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd36c100460 0x7fd36c100830 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd368009910 tx=0x7fd368009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:27.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.727+0000 7fd373b7c700 1 -- 172.21.15.88:0/3441916675 shutdown_connections 2024-02-19T19:04:27.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.727+0000 7fd373b7c700 1 --2- 172.21.15.88:0/3441916675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd36c100d70 0x7fd36c1094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:27.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.727+0000 7fd373b7c700 1 --2- 172.21.15.88:0/3441916675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd36c100460 0x7fd36c100830 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd368009910 tx=0x7fd368009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:27.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.727+0000 7fd373b7c700 1 -- 172.21.15.88:0/3441916675 >> 172.21.15.88:0/3441916675 conn(0x7fd36c0748b0 msgr2=0x7fd36c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:27.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.727+0000 7fd373b7c700 1 -- 172.21.15.88:0/3441916675 shutdown_connections 2024-02-19T19:04:27.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.727+0000 7fd373b7c700 1 -- 172.21.15.88:0/3441916675 wait complete. 2024-02-19T19:04:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.728+0000 7fd373b7c700 1 Processor -- start 2024-02-19T19:04:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.728+0000 7fd373b7c700 1 -- start start 2024-02-19T19:04:27.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.728+0000 7fd373b7c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd36c100d70 0x7fd36c0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:27.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.728+0000 7fd373b7c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd36c06a840 0x7fd36c06acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:27.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.728+0000 7fd371918700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd36c100d70 0x7fd36c0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:27.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.728+0000 7fd371117700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd36c06a840 0x7fd36c06acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:27.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.728+0000 7fd371117700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd36c06a840 0x7fd36c06acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45426/0 (socket says 172.21.15.88:45426) 2024-02-19T19:04:27.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.728+0000 7fd371117700 1 -- 172.21.15.88:0/4016167926 learned_addr learned my addr 172.21.15.88:0/4016167926 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:27.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.728+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c06b1f0 con 0x7fd36c06a840 2024-02-19T19:04:27.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.728+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c06b360 con 0x7fd36c100d70 2024-02-19T19:04:27.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.729+0000 7fd371117700 1 -- 172.21.15.88:0/4016167926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd36c100d70 msgr2=0x7fd36c0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:27.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.729+0000 7fd371117700 1 --2- 172.21.15.88:0/4016167926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd36c100d70 0x7fd36c0728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:27.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.729+0000 7fd371117700 1 -- 172.21.15.88:0/4016167926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3680095f0 con 0x7fd36c06a840 2024-02-19T19:04:27.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.730+0000 7fd371117700 1 --2- 172.21.15.88:0/4016167926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd36c06a840 0x7fd36c06acb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00d900 tx=0x7fd35c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:27.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.730+0000 7fd362ffd700 1 -- 172.21.15.88:0/4016167926 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd35c0041f0 con 0x7fd36c06a840 2024-02-19T19:04:27.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.730+0000 7fd362ffd700 1 -- 172.21.15.88:0/4016167926 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd35c004cf0 con 0x7fd36c06a840 2024-02-19T19:04:27.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.730+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36c06b640 con 0x7fd36c06a840 2024-02-19T19:04:27.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.730+0000 7fd362ffd700 1 -- 172.21.15.88:0/4016167926 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd35c00b750 con 0x7fd36c06a840 2024-02-19T19:04:27.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.731+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36c1a9050 con 0x7fd36c06a840 2024-02-19T19:04:27.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:27 smithi132 ceph-mon[37969]: pgmap v90: 65 pgs: 65 active+clean; 77 MiB data, 472 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s wr, 238 op/s 2024-02-19T19:04:27.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.732+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36c107e40 con 0x7fd36c06a840 2024-02-19T19:04:27.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.733+0000 7fd362ffd700 1 -- 172.21.15.88:0/4016167926 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fd35c00b8b0 con 0x7fd36c06a840 2024-02-19T19:04:27.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.734+0000 7fd362ffd700 1 --2- 172.21.15.88:0/4016167926 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd35806d520 0x7fd35806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:27.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.734+0000 7fd362ffd700 1 -- 172.21.15.88:0/4016167926 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd35c05b220 con 0x7fd36c06a840 2024-02-19T19:04:27.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.735+0000 7fd371918700 1 --2- 172.21.15.88:0/4016167926 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd35806d520 0x7fd35806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:27.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.735+0000 7fd371918700 1 --2- 172.21.15.88:0/4016167926 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd35806d520 0x7fd35806f9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd368009fd0 tx=0x7fd36801a040 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:27.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:27.742+0000 7fd362ffd700 1 -- 172.21.15.88:0/4016167926 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fd35c092060 con 0x7fd36c06a840 2024-02-19T19:04:28.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.133+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fd36c04f2a0 con 0x7fd36c06a840 2024-02-19T19:04:28.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.134+0000 7fd362ffd700 1 -- 172.21.15.88:0/4016167926 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v44)=0 v44) v1 ==== 135+0+0 (secure 0 0 0) 0x7fd35c0920c0 con 0x7fd36c06a840 2024-02-19T19:04:28.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.136+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd35806d520 msgr2=0x7fd35806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:28.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.136+0000 7fd373b7c700 1 --2- 172.21.15.88:0/4016167926 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd35806d520 0x7fd35806f9d0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd368009fd0 tx=0x7fd36801a040 comp rx=0 tx=0).stop 2024-02-19T19:04:28.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.136+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd36c06a840 msgr2=0x7fd36c06acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:28.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.136+0000 7fd373b7c700 1 --2- 172.21.15.88:0/4016167926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd36c06a840 0x7fd36c06acb0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00d900 tx=0x7fd35c00dc10 comp rx=0 tx=0).stop 2024-02-19T19:04:28.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.136+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 shutdown_connections 2024-02-19T19:04:28.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.136+0000 7fd373b7c700 1 --2- 172.21.15.88:0/4016167926 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd35806d520 0x7fd35806f9d0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:28.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.136+0000 7fd373b7c700 1 --2- 172.21.15.88:0/4016167926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd36c100d70 0x7fd36c0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:28.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.136+0000 7fd373b7c700 1 --2- 172.21.15.88:0/4016167926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd36c06a840 0x7fd36c06acb0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:28.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.136+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 >> 172.21.15.88:0/4016167926 conn(0x7fd36c0748b0 msgr2=0x7fd36c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:28.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.136+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 shutdown_connections 2024-02-19T19:04:28.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:28.136+0000 7fd373b7c700 1 -- 172.21.15.88:0/4016167926 wait complete. 2024-02-19T19:04:28.858 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-02-19T19:04:29.171 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:04:29.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:29 smithi088 ceph-mon[29272]: pgmap v91: 65 pgs: 65 active+clean; 89 MiB data, 495 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s wr, 357 op/s 2024-02-19T19:04:29.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:29 smithi132 ceph-mon[37969]: pgmap v91: 65 pgs: 65 active+clean; 89 MiB data, 495 MiB used, 536 GiB / 536 GiB avail; 3.4 MiB/s wr, 357 op/s 2024-02-19T19:04:31.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:31 smithi132 ceph-mon[37969]: pgmap v92: 65 pgs: 65 active+clean; 89 MiB data, 495 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s wr, 228 op/s 2024-02-19T19:04:31.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:31 smithi088 ceph-mon[29272]: pgmap v92: 65 pgs: 65 active+clean; 89 MiB data, 495 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s wr, 228 op/s 2024-02-19T19:04:33.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:33 smithi088 ceph-mon[29272]: pgmap v93: 65 pgs: 65 active+clean; 89 MiB data, 495 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s wr, 228 op/s 2024-02-19T19:04:33.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:33 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:04:33.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.383+0000 7f69f12db700 1 -- 172.21.15.88:0/1858124467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ec100d70 msgr2=0x7f69ec1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:33.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.383+0000 7f69f12db700 1 --2- 172.21.15.88:0/1858124467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ec100d70 0x7f69ec1094d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009910 tx=0x7f69e0009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:33.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.383+0000 7f69f12db700 1 -- 172.21.15.88:0/1858124467 shutdown_connections 2024-02-19T19:04:33.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.383+0000 7f69f12db700 1 --2- 172.21.15.88:0/1858124467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ec100d70 0x7f69ec1094d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.383+0000 7f69f12db700 1 --2- 172.21.15.88:0/1858124467 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69ec100460 0x7f69ec100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.383+0000 7f69f12db700 1 -- 172.21.15.88:0/1858124467 >> 172.21.15.88:0/1858124467 conn(0x7f69ec0748b0 msgr2=0x7f69ec074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.384+0000 7f69f12db700 1 -- 172.21.15.88:0/1858124467 shutdown_connections 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.384+0000 7f69f12db700 1 -- 172.21.15.88:0/1858124467 wait complete. 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.384+0000 7f69f12db700 1 Processor -- start 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69f12db700 1 -- start start 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69f12db700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ec100460 0x7f69ec196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69f12db700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69ec100d70 0x7f69ec197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69f12db700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec197870 con 0x7f69ec100460 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69f12db700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec1979b0 con 0x7f69ec100d70 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69eaffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ec100460 0x7f69ec196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:33.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69eaffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ec100460 0x7f69ec196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45458/0 (socket says 172.21.15.88:45458) 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69eaffd700 1 -- 172.21.15.88:0/642512983 learned_addr learned my addr 172.21.15.88:0/642512983 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69eaffd700 1 -- 172.21.15.88:0/642512983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69ec100d70 msgr2=0x7f69ec197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69ea7fc700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69ec100d70 0x7f69ec197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69eaffd700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69ec100d70 0x7f69ec197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69eaffd700 1 -- 172.21.15.88:0/642512983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e00095f0 con 0x7f69ec100460 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.385+0000 7f69ea7fc700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69ec100d70 0x7f69ec197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.386+0000 7f69eaffd700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ec100460 0x7f69ec196c50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00d8d0 tx=0x7f69dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.386+0000 7f69d3fff700 1 -- 172.21.15.88:0/642512983 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69dc009940 con 0x7f69ec100460 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.386+0000 7f69d3fff700 1 -- 172.21.15.88:0/642512983 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69dc010460 con 0x7f69ec100460 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.386+0000 7f69f12db700 1 -- 172.21.15.88:0/642512983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec19fcc0 con 0x7f69ec100460 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.386+0000 7f69f12db700 1 -- 172.21.15.88:0/642512983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec1a01e0 con 0x7f69ec100460 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.386+0000 7f69d3fff700 1 -- 172.21.15.88:0/642512983 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69dc009940 con 0x7f69ec100460 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.387+0000 7f69f12db700 1 -- 172.21.15.88:0/642512983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ec106bd0 con 0x7f69ec100460 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.388+0000 7f69d3fff700 1 -- 172.21.15.88:0/642512983 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f69dc0105d0 con 0x7f69ec100460 2024-02-19T19:04:33.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.389+0000 7f69d3fff700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f69d406d590 0x7f69d406fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:33.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.389+0000 7f69d3fff700 1 -- 172.21.15.88:0/642512983 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f69dc08cbf0 con 0x7f69ec100460 2024-02-19T19:04:33.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.389+0000 7f69ea7fc700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f69d406d590 0x7f69d406fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:33.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.389+0000 7f69ea7fc700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f69d406d590 0x7f69d406fa40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69e000b5c0 tx=0x7f69e00053a0 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:33.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.395+0000 7f69d3fff700 1 -- 172.21.15.88:0/642512983 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f69dc0568d0 con 0x7f69ec100460 2024-02-19T19:04:33.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:33 smithi132 ceph-mon[37969]: pgmap v93: 65 pgs: 65 active+clean; 89 MiB data, 495 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s wr, 228 op/s 2024-02-19T19:04:33.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:33 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:04:33.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.682+0000 7f69f12db700 1 -- 172.21.15.88:0/642512983 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}) v1 -- 0x7f69ec061190 con 0x7f69d406d590 2024-02-19T19:04:33.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.690+0000 7f69d3fff700 1 -- 172.21.15.88:0/642512983 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f69ec061190 con 0x7f69d406d590 2024-02-19T19:04:33.710 INFO:teuthology.orchestra.run.smithi088.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T19:04:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.692+0000 7f69f12db700 1 -- 172.21.15.88:0/642512983 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f69d406d590 msgr2=0x7f69d406fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.692+0000 7f69f12db700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f69d406d590 0x7f69d406fa40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69e000b5c0 tx=0x7f69e00053a0 comp rx=0 tx=0).stop 2024-02-19T19:04:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.692+0000 7f69f12db700 1 -- 172.21.15.88:0/642512983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ec100460 msgr2=0x7f69ec196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:33.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.692+0000 7f69f12db700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ec100460 0x7f69ec196c50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00d8d0 tx=0x7f69dc00dc90 comp rx=0 tx=0).stop 2024-02-19T19:04:33.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.693+0000 7f69f12db700 1 -- 172.21.15.88:0/642512983 shutdown_connections 2024-02-19T19:04:33.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.693+0000 7f69f12db700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f69d406d590 0x7f69d406fa40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:33.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.693+0000 7f69f12db700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69ec100460 0x7f69ec196c50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:33.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.693+0000 7f69f12db700 1 --2- 172.21.15.88:0/642512983 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69ec100d70 0x7f69ec197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:33.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.693+0000 7f69f12db700 1 -- 172.21.15.88:0/642512983 >> 172.21.15.88:0/642512983 conn(0x7f69ec0748b0 msgr2=0x7f69ec103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:33.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.693+0000 7f69f12db700 1 -- 172.21.15.88:0/642512983 shutdown_connections 2024-02-19T19:04:33.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:33.693+0000 7f69f12db700 1 -- 172.21.15.88:0/642512983 wait complete. 2024-02-19T19:04:34.357 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-02-19T19:04:34.358 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T19:04:34.358 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-02-19T19:04:34.677 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:04:34.999 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:34 smithi088 ceph-mon[29272]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:34.999 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:34 smithi088 ceph-mon[29272]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T19:04:34.999 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:34 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:34.999 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:34 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:04:34.999 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:34 smithi088 ceph-mon[29272]: pgmap v94: 65 pgs: 65 active+clean; 93 MiB data, 515 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s wr, 258 op/s 2024-02-19T19:04:34.999 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:34 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:04:34.999 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:34 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:04:35.000 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:34 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:35.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:34 smithi132 ceph-mon[37969]: from='client.14834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:35.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:34 smithi132 ceph-mon[37969]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T19:04:35.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:34 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:35.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:34 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:04:35.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:34 smithi132 ceph-mon[37969]: pgmap v94: 65 pgs: 65 active+clean; 93 MiB data, 515 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s wr, 258 op/s 2024-02-19T19:04:35.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:34 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:04:35.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:34 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:04:35.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:34 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:36.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:35 smithi132 ceph-mon[37969]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T19:04:36.035 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:35 smithi088 ceph-mon[29272]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:f78a58c0ffd401d1493058a1022c35f011d65275 2024-02-19T19:04:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.582+0000 7fd74f532700 1 -- 172.21.15.88:0/552272818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7480ffbd0 msgr2=0x7fd748100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.582+0000 7fd74f532700 1 --2- 172.21.15.88:0/552272818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7480ffbd0 0x7fd748100040 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fd744009910 tx=0x7fd744009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.582+0000 7fd74f532700 1 -- 172.21.15.88:0/552272818 shutdown_connections 2024-02-19T19:04:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.582+0000 7fd74f532700 1 --2- 172.21.15.88:0/552272818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7480ffbd0 0x7fd748100040 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.582+0000 7fd74f532700 1 --2- 172.21.15.88:0/552272818 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd748104130 0x7fd748104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.582+0000 7fd74f532700 1 -- 172.21.15.88:0/552272818 >> 172.21.15.88:0/552272818 conn(0x7fd748075130 msgr2=0x7fd748075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.582+0000 7fd74f532700 1 -- 172.21.15.88:0/552272818 shutdown_connections 2024-02-19T19:04:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.582+0000 7fd74f532700 1 -- 172.21.15.88:0/552272818 wait complete. 2024-02-19T19:04:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.583+0000 7fd74f532700 1 Processor -- start 2024-02-19T19:04:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.583+0000 7fd74f532700 1 -- start start 2024-02-19T19:04:36.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.583+0000 7fd74f532700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7480ffbd0 0x7fd748194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.583+0000 7fd74f532700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd748104130 0x7fd748195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.584+0000 7fd74f532700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd748195950 con 0x7fd7480ffbd0 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.584+0000 7fd74f532700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7481996e0 con 0x7fd748104130 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.584+0000 7fd74d2ce700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7480ffbd0 0x7fd748194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.584+0000 7fd74cacd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd748104130 0x7fd748195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.584+0000 7fd74cacd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd748104130 0x7fd748195270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:51258/0 (socket says 172.21.15.88:51258) 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.584+0000 7fd74cacd700 1 -- 172.21.15.88:0/3782963730 learned_addr learned my addr 172.21.15.88:0/3782963730 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.584+0000 7fd74d2ce700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7480ffbd0 0x7fd748194d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46386/0 (socket says 172.21.15.88:46386) 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.584+0000 7fd74cacd700 1 -- 172.21.15.88:0/3782963730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7480ffbd0 msgr2=0x7fd748194d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.584+0000 7fd74cacd700 1 --2- 172.21.15.88:0/3782963730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7480ffbd0 0x7fd748194d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.584+0000 7fd74cacd700 1 -- 172.21.15.88:0/3782963730 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7440095f0 con 0x7fd748104130 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.584+0000 7fd74d2ce700 1 --2- 172.21.15.88:0/3782963730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7480ffbd0 0x7fd748194d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.585+0000 7fd74cacd700 1 --2- 172.21.15.88:0/3782963730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd748104130 0x7fd748195270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd74400b7d0 tx=0x7fd74400f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:36.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.585+0000 7fd73e7fc700 1 -- 172.21.15.88:0/3782963730 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd74401d070 con 0x7fd748104130 2024-02-19T19:04:36.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.585+0000 7fd74f532700 1 -- 172.21.15.88:0/3782963730 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd748199960 con 0x7fd748104130 2024-02-19T19:04:36.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.585+0000 7fd74f532700 1 -- 172.21.15.88:0/3782963730 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd748199e50 con 0x7fd748104130 2024-02-19T19:04:36.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.585+0000 7fd73e7fc700 1 -- 172.21.15.88:0/3782963730 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7440052a0 con 0x7fd748104130 2024-02-19T19:04:36.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.585+0000 7fd73e7fc700 1 -- 172.21.15.88:0/3782963730 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd744022470 con 0x7fd748104130 2024-02-19T19:04:36.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.586+0000 7fd74f532700 1 -- 172.21.15.88:0/3782963730 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72c005320 con 0x7fd748104130 2024-02-19T19:04:36.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.587+0000 7fd73e7fc700 1 -- 172.21.15.88:0/3782963730 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fd744003680 con 0x7fd748104130 2024-02-19T19:04:36.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.587+0000 7fd73e7fc700 1 --2- 172.21.15.88:0/3782963730 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd73406d520 0x7fd73406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:36.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.588+0000 7fd74d2ce700 1 --2- 172.21.15.88:0/3782963730 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd73406d520 0x7fd73406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:36.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.588+0000 7fd73e7fc700 1 -- 172.21.15.88:0/3782963730 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd74408e300 con 0x7fd748104130 2024-02-19T19:04:36.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.588+0000 7fd74d2ce700 1 --2- 172.21.15.88:0/3782963730 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd73406d520 0x7fd73406f9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd738009cb0 tx=0x7fd738009260 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:36.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.592+0000 7fd73e7fc700 1 -- 172.21.15.88:0/3782963730 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fd744057fc0 con 0x7fd748104130 2024-02-19T19:04:36.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.898+0000 7fd74f532700 1 -- 172.21.15.88:0/3782963730 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd72c000bf0 con 0x7fd73406d520 2024-02-19T19:04:36.919 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:36 smithi088 ceph-mon[29272]: pgmap v95: 65 pgs: 65 active+clean; 93 MiB data, 515 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s wr, 148 op/s 2024-02-19T19:04:36.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.899+0000 7fd73e7fc700 1 -- 172.21.15.88:0/3782963730 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fd72c000bf0 con 0x7fd73406d520 2024-02-19T19:04:36.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.902+0000 7fd74f532700 1 -- 172.21.15.88:0/3782963730 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd73406d520 msgr2=0x7fd73406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:36.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.902+0000 7fd74f532700 1 --2- 172.21.15.88:0/3782963730 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd73406d520 0x7fd73406f9d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd738009cb0 tx=0x7fd738009260 comp rx=0 tx=0).stop 2024-02-19T19:04:36.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.902+0000 7fd74f532700 1 -- 172.21.15.88:0/3782963730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd748104130 msgr2=0x7fd748195270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:36.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.902+0000 7fd74f532700 1 --2- 172.21.15.88:0/3782963730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd748104130 0x7fd748195270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd74400b7d0 tx=0x7fd74400f460 comp rx=0 tx=0).stop 2024-02-19T19:04:36.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.902+0000 7fd74f532700 1 -- 172.21.15.88:0/3782963730 shutdown_connections 2024-02-19T19:04:36.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.902+0000 7fd74f532700 1 --2- 172.21.15.88:0/3782963730 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fd73406d520 0x7fd73406f9d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:36.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.902+0000 7fd74f532700 1 --2- 172.21.15.88:0/3782963730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7480ffbd0 0x7fd748194d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:36.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.902+0000 7fd74f532700 1 --2- 172.21.15.88:0/3782963730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd748104130 0x7fd748195270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:36.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.902+0000 7fd74f532700 1 -- 172.21.15.88:0/3782963730 >> 172.21.15.88:0/3782963730 conn(0x7fd748075130 msgr2=0x7fd7480feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:36.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.902+0000 7fd74f532700 1 -- 172.21.15.88:0/3782963730 shutdown_connections 2024-02-19T19:04:36.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:36.902+0000 7fd74f532700 1 -- 172.21.15.88:0/3782963730 wait complete. 2024-02-19T19:04:36.935 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:04:37.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:36 smithi132 ceph-mon[37969]: pgmap v95: 65 pgs: 65 active+clean; 93 MiB data, 515 MiB used, 536 GiB / 536 GiB avail; 1.4 MiB/s wr, 148 op/s 2024-02-19T19:04:37.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.050+0000 7fca24d61700 1 -- 172.21.15.88:0/2057235635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca20102470 msgr2=0x7fca201028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:37.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.050+0000 7fca24d61700 1 --2- 172.21.15.88:0/2057235635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca20102470 0x7fca201028e0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fca14009910 tx=0x7fca14009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.050+0000 7fca24d61700 1 -- 172.21.15.88:0/2057235635 shutdown_connections 2024-02-19T19:04:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.050+0000 7fca24d61700 1 --2- 172.21.15.88:0/2057235635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca20102470 0x7fca201028e0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.050+0000 7fca24d61700 1 --2- 172.21.15.88:0/2057235635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca20108470 0x7fca20108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.050+0000 7fca24d61700 1 -- 172.21.15.88:0/2057235635 >> 172.21.15.88:0/2057235635 conn(0x7fca200fdf90 msgr2=0x7fca201003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.050+0000 7fca24d61700 1 -- 172.21.15.88:0/2057235635 shutdown_connections 2024-02-19T19:04:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.050+0000 7fca24d61700 1 -- 172.21.15.88:0/2057235635 wait complete. 2024-02-19T19:04:37.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.051+0000 7fca24d61700 1 Processor -- start 2024-02-19T19:04:37.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.051+0000 7fca24d61700 1 -- start start 2024-02-19T19:04:37.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.051+0000 7fca24d61700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca20102470 0x7fca20198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:37.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.051+0000 7fca24d61700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca20108470 0x7fca201987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:37.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.051+0000 7fca24d61700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca20198e90 con 0x7fca20108470 2024-02-19T19:04:37.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.051+0000 7fca24d61700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2019cc20 con 0x7fca20102470 2024-02-19T19:04:37.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.051+0000 7fca1dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca20108470 0x7fca201987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:37.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.052+0000 7fca1dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca20108470 0x7fca201987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46400/0 (socket says 172.21.15.88:46400) 2024-02-19T19:04:37.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.052+0000 7fca1dd9b700 1 -- 172.21.15.88:0/508533294 learned_addr learned my addr 172.21.15.88:0/508533294 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:37.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.052+0000 7fca1e59c700 1 --2- 172.21.15.88:0/508533294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca20102470 0x7fca20198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:37.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.052+0000 7fca1dd9b700 1 -- 172.21.15.88:0/508533294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca20102470 msgr2=0x7fca20198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:37.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.052+0000 7fca1dd9b700 1 --2- 172.21.15.88:0/508533294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca20102470 0x7fca20198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:37.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.052+0000 7fca1dd9b700 1 -- 172.21.15.88:0/508533294 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca140095f0 con 0x7fca20108470 2024-02-19T19:04:37.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.052+0000 7fca1dd9b700 1 --2- 172.21.15.88:0/508533294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca20108470 0x7fca201987b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fca14006010 tx=0x7fca1400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:37.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.052+0000 7fca0f7fe700 1 -- 172.21.15.88:0/508533294 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca1401d070 con 0x7fca20108470 2024-02-19T19:04:37.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.052+0000 7fca24d61700 1 -- 172.21.15.88:0/508533294 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2019cea0 con 0x7fca20108470 2024-02-19T19:04:37.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.052+0000 7fca24d61700 1 -- 172.21.15.88:0/508533294 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2019d390 con 0x7fca20108470 2024-02-19T19:04:37.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.052+0000 7fca0f7fe700 1 -- 172.21.15.88:0/508533294 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca1400f460 con 0x7fca20108470 2024-02-19T19:04:37.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.053+0000 7fca0f7fe700 1 -- 172.21.15.88:0/508533294 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca14017610 con 0x7fca20108470 2024-02-19T19:04:37.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.054+0000 7fca24d61700 1 -- 172.21.15.88:0/508533294 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2010a970 con 0x7fca20108470 2024-02-19T19:04:37.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.055+0000 7fca0f7fe700 1 -- 172.21.15.88:0/508533294 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fca1400f5d0 con 0x7fca20108470 2024-02-19T19:04:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.055+0000 7fca0f7fe700 1 --2- 172.21.15.88:0/508533294 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fca0806d520 0x7fca0806f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.056+0000 7fca0f7fe700 1 -- 172.21.15.88:0/508533294 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fca1408e3f0 con 0x7fca20108470 2024-02-19T19:04:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.056+0000 7fca1e59c700 1 --2- 172.21.15.88:0/508533294 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fca0806d520 0x7fca0806f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:37.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.056+0000 7fca1e59c700 1 --2- 172.21.15.88:0/508533294 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fca0806d520 0x7fca0806f9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca10005950 tx=0x7fca1000a300 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:37.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.065+0000 7fca0f7fe700 1 -- 172.21.15.88:0/508533294 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fca14058150 con 0x7fca20108470 2024-02-19T19:04:37.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.436+0000 7fca24d61700 1 -- 172.21.15.88:0/508533294 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fca201995d0 con 0x7fca0806d520 2024-02-19T19:04:37.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.438+0000 7fca0f7fe700 1 -- 172.21.15.88:0/508533294 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7fca201995d0 con 0x7fca0806d520 2024-02-19T19:04:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.441+0000 7fca24d61700 1 -- 172.21.15.88:0/508533294 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fca0806d520 msgr2=0x7fca0806f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.441+0000 7fca24d61700 1 --2- 172.21.15.88:0/508533294 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fca0806d520 0x7fca0806f9d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca10005950 tx=0x7fca1000a300 comp rx=0 tx=0).stop 2024-02-19T19:04:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.441+0000 7fca24d61700 1 -- 172.21.15.88:0/508533294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca20108470 msgr2=0x7fca201987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.441+0000 7fca24d61700 1 --2- 172.21.15.88:0/508533294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca20108470 0x7fca201987b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fca14006010 tx=0x7fca1400bd70 comp rx=0 tx=0).stop 2024-02-19T19:04:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.441+0000 7fca24d61700 1 -- 172.21.15.88:0/508533294 shutdown_connections 2024-02-19T19:04:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.441+0000 7fca24d61700 1 --2- 172.21.15.88:0/508533294 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fca0806d520 0x7fca0806f9d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.441+0000 7fca24d61700 1 --2- 172.21.15.88:0/508533294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca20102470 0x7fca20198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.441+0000 7fca24d61700 1 --2- 172.21.15.88:0/508533294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca20108470 0x7fca201987b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.441+0000 7fca24d61700 1 -- 172.21.15.88:0/508533294 >> 172.21.15.88:0/508533294 conn(0x7fca200fdf90 msgr2=0x7fca200ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.441+0000 7fca24d61700 1 -- 172.21.15.88:0/508533294 shutdown_connections 2024-02-19T19:04:37.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.441+0000 7fca24d61700 1 -- 172.21.15.88:0/508533294 wait complete. 2024-02-19T19:04:37.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.603+0000 7fcd4957c700 1 -- 172.21.15.88:0/3680090968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd441038b0 msgr2=0x7fcd44103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:37.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.603+0000 7fcd4957c700 1 --2- 172.21.15.88:0/3680090968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd441038b0 0x7fcd44103c80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009910 tx=0x7fcd34009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:37.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.603+0000 7fcd4957c700 1 -- 172.21.15.88:0/3680090968 shutdown_connections 2024-02-19T19:04:37.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.603+0000 7fcd4957c700 1 --2- 172.21.15.88:0/3680090968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd440ff350 0x7fcd440ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:37.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.604+0000 7fcd4957c700 1 --2- 172.21.15.88:0/3680090968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd441038b0 0x7fcd44103c80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:37.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.604+0000 7fcd4957c700 1 -- 172.21.15.88:0/3680090968 >> 172.21.15.88:0/3680090968 conn(0x7fcd440748b0 msgr2=0x7fcd44074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:37.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.604+0000 7fcd4957c700 1 -- 172.21.15.88:0/3680090968 shutdown_connections 2024-02-19T19:04:37.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.604+0000 7fcd4957c700 1 -- 172.21.15.88:0/3680090968 wait complete. 2024-02-19T19:04:37.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.605+0000 7fcd4957c700 1 Processor -- start 2024-02-19T19:04:37.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.605+0000 7fcd4957c700 1 -- start start 2024-02-19T19:04:37.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.606+0000 7fcd4957c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd440ff350 0x7fcd441999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:37.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.606+0000 7fcd4957c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd441038b0 0x7fcd44199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:37.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.606+0000 7fcd427fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd441038b0 0x7fcd44199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:37.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.606+0000 7fcd42ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd440ff350 0x7fcd441999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:37.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.606+0000 7fcd42ffd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd440ff350 0x7fcd441999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:51292/0 (socket says 172.21.15.88:51292) 2024-02-19T19:04:37.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.606+0000 7fcd427fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd441038b0 0x7fcd44199f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46424/0 (socket says 172.21.15.88:46424) 2024-02-19T19:04:37.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.606+0000 7fcd42ffd700 1 -- 172.21.15.88:0/2777629350 learned_addr learned my addr 172.21.15.88:0/2777629350 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:37.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.606+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4419a5b0 con 0x7fcd441038b0 2024-02-19T19:04:37.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.606+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd44193a60 con 0x7fcd440ff350 2024-02-19T19:04:37.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.607+0000 7fcd427fc700 1 -- 172.21.15.88:0/2777629350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd440ff350 msgr2=0x7fcd441999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:37.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.607+0000 7fcd427fc700 1 --2- 172.21.15.88:0/2777629350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd440ff350 0x7fcd441999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:37.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.607+0000 7fcd427fc700 1 -- 172.21.15.88:0/2777629350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd340095f0 con 0x7fcd441038b0 2024-02-19T19:04:37.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.607+0000 7fcd427fc700 1 --2- 172.21.15.88:0/2777629350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd441038b0 0x7fcd44199f20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800ead0 tx=0x7fcd3800ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:37.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.608+0000 7fcd2bfff700 1 -- 172.21.15.88:0/2777629350 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd380096d0 con 0x7fcd441038b0 2024-02-19T19:04:37.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.608+0000 7fcd2bfff700 1 -- 172.21.15.88:0/2777629350 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd38010430 con 0x7fcd441038b0 2024-02-19T19:04:37.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.608+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd44193d40 con 0x7fcd441038b0 2024-02-19T19:04:37.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.609+0000 7fcd2bfff700 1 -- 172.21.15.88:0/2777629350 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd38018650 con 0x7fcd441038b0 2024-02-19T19:04:37.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.610+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd44194290 con 0x7fcd441038b0 2024-02-19T19:04:37.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.610+0000 7fcd2bfff700 1 -- 172.21.15.88:0/2777629350 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fcd38018830 con 0x7fcd441038b0 2024-02-19T19:04:37.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.611+0000 7fcd2bfff700 1 --2- 172.21.15.88:0/2777629350 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcd2c06f570 0x7fcd2c071a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:37.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.611+0000 7fcd2bfff700 1 -- 172.21.15.88:0/2777629350 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcd38014070 con 0x7fcd441038b0 2024-02-19T19:04:37.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.611+0000 7fcd42ffd700 1 --2- 172.21.15.88:0/2777629350 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcd2c06f570 0x7fcd2c071a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:37.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.611+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd44106e20 con 0x7fcd441038b0 2024-02-19T19:04:37.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.613+0000 7fcd42ffd700 1 --2- 172.21.15.88:0/2777629350 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcd2c06f570 0x7fcd2c071a20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009fd0 tx=0x7fcd34005fb0 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:37.620+0000 7fcd2bfff700 1 -- 172.21.15.88:0/2777629350 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fcd38092060 con 0x7fcd441038b0 2024-02-19T19:04:38.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.041+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd44194fa0 con 0x7fcd2c06f570 2024-02-19T19:04:38.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.061+0000 7fcd2bfff700 1 -- 172.21.15.88:0/2777629350 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fcd44194fa0 con 0x7fcd2c06f570 2024-02-19T19:04:38.082 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:04:38.082 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (108s) 63s ago 12m 15.7M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:04:38.082 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (12m) 63s ago 12m 7843k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:04:38.082 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (10m) 2m ago 10m 7608k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:04:38.082 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (12m) 63s ago 12m 6665k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (10m) 2m ago 10m 6656k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (69s) 63s ago 11m 57.2M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (6m) 63s ago 6m 14.5M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (5m) 63s ago 5m 26.4M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (5m) 2m ago 5m 23.1M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (5m) 2m ago 5m 17.6M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (3m) 63s ago 13m 499M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (2m) 2m ago 10m 449M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (14m) 63s ago 14m 55.6M 2048M 18.2.0 dc2bc1663786 3ea5108455bc 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (10m) 2m ago 10m 43.1M 2048M 18.2.0 dc2bc1663786 ba1c249c6017 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (2m) 63s ago 12m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (2m) 2m ago 10m 3573k - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:04:38.083 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (9m) 63s ago 9m 52.2M 1216M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:04:38.084 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (9m) 63s ago 9m 49.8M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:04:38.084 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (8m) 63s ago 8m 49.3M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:04:38.084 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (8m) 2m ago 8m 50.5M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:04:38.084 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (7m) 2m ago 7m 49.1M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:04:38.084 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (7m) 2m ago 7m 47.2M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:04:38.084 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (2m) 63s ago 11m 54.5M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:04:38.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.064+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcd2c06f570 msgr2=0x7fcd2c071a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:38.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.064+0000 7fcd4957c700 1 --2- 172.21.15.88:0/2777629350 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcd2c06f570 0x7fcd2c071a20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009fd0 tx=0x7fcd34005fb0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.064+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd441038b0 msgr2=0x7fcd44199f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:38.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.064+0000 7fcd4957c700 1 --2- 172.21.15.88:0/2777629350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd441038b0 0x7fcd44199f20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800ead0 tx=0x7fcd3800ede0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.064+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 shutdown_connections 2024-02-19T19:04:38.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.064+0000 7fcd4957c700 1 --2- 172.21.15.88:0/2777629350 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fcd2c06f570 0x7fcd2c071a20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.064+0000 7fcd4957c700 1 --2- 172.21.15.88:0/2777629350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd440ff350 0x7fcd441999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.064+0000 7fcd4957c700 1 --2- 172.21.15.88:0/2777629350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd441038b0 0x7fcd44199f20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.064+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 >> 172.21.15.88:0/2777629350 conn(0x7fcd440748b0 msgr2=0x7fcd440fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.064+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 shutdown_connections 2024-02-19T19:04:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.064+0000 7fcd4957c700 1 -- 172.21.15.88:0/2777629350 wait complete. 2024-02-19T19:04:38.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.208+0000 7fe20559a700 1 -- 172.21.15.88:0/3248544208 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1fc004a00 con 0x7fe2080ff350 2024-02-19T19:04:38.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.209+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3248544208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2080ff350 msgr2=0x7fe2080ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:38.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.209+0000 7fe20ca8b700 1 --2- 172.21.15.88:0/3248544208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2080ff350 0x7fe2080ff7c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc009910 tx=0x7fe1fc009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:38.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.209+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3248544208 shutdown_connections 2024-02-19T19:04:38.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.209+0000 7fe20ca8b700 1 --2- 172.21.15.88:0/3248544208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2080ff350 0x7fe2080ff7c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.209+0000 7fe20ca8b700 1 --2- 172.21.15.88:0/3248544208 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2081038b0 0x7fe208103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.209+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3248544208 >> 172.21.15.88:0/3248544208 conn(0x7fe2080748b0 msgr2=0x7fe208074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:38.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.209+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3248544208 shutdown_connections 2024-02-19T19:04:38.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.209+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3248544208 wait complete. 2024-02-19T19:04:38.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.210+0000 7fe20ca8b700 1 Processor -- start 2024-02-19T19:04:38.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.210+0000 7fe20ca8b700 1 -- start start 2024-02-19T19:04:38.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.211+0000 7fe20ca8b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2080ff350 0x7fe20819c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:38.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.211+0000 7fe20ca8b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2081038b0 0x7fe20819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:38.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.211+0000 7fe20ca8b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20819d020 con 0x7fe2080ff350 2024-02-19T19:04:38.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.211+0000 7fe20ca8b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2081a0db0 con 0x7fe2081038b0 2024-02-19T19:04:38.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.211+0000 7fe20659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2080ff350 0x7fe20819c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:38.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.211+0000 7fe20659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2080ff350 0x7fe20819c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46450/0 (socket says 172.21.15.88:46450) 2024-02-19T19:04:38.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.211+0000 7fe20659c700 1 -- 172.21.15.88:0/3567854643 learned_addr learned my addr 172.21.15.88:0/3567854643 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:38.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.211+0000 7fe205d9b700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2081038b0 0x7fe20819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:38.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.212+0000 7fe20659c700 1 -- 172.21.15.88:0/3567854643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2081038b0 msgr2=0x7fe20819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:38.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.212+0000 7fe20659c700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2081038b0 0x7fe20819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.212+0000 7fe20659c700 1 -- 172.21.15.88:0/3567854643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0095f0 con 0x7fe2080ff350 2024-02-19T19:04:38.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.212+0000 7fe205d9b700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2081038b0 0x7fe20819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:04:38.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.212+0000 7fe20659c700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2080ff350 0x7fe20819c400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800e920 tx=0x7fe1f800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:38.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.212+0000 7fe1f77fe700 1 -- 172.21.15.88:0/3567854643 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1f800c9d0 con 0x7fe2080ff350 2024-02-19T19:04:38.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.212+0000 7fe1f77fe700 1 -- 172.21.15.88:0/3567854643 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1f8010430 con 0x7fe2080ff350 2024-02-19T19:04:38.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.212+0000 7fe1f77fe700 1 -- 172.21.15.88:0/3567854643 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1f800c9d0 con 0x7fe2080ff350 2024-02-19T19:04:38.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.212+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3567854643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2081a1090 con 0x7fe2080ff350 2024-02-19T19:04:38.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.212+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3567854643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2081a1590 con 0x7fe2080ff350 2024-02-19T19:04:38.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.214+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3567854643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe208102c90 con 0x7fe2080ff350 2024-02-19T19:04:38.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.215+0000 7fe1f77fe700 1 -- 172.21.15.88:0/3567854643 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fe1f80105a0 con 0x7fe2080ff350 2024-02-19T19:04:38.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.215+0000 7fe1f77fe700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe1f006d5f0 0x7fe1f006faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:38.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.215+0000 7fe1f77fe700 1 -- 172.21.15.88:0/3567854643 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe1f8014070 con 0x7fe2080ff350 2024-02-19T19:04:38.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.216+0000 7fe205d9b700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe1f006d5f0 0x7fe1f006faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:38.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.217+0000 7fe205d9b700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe1f006d5f0 0x7fe1f006faa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe20819da20 tx=0x7fe1fc005a90 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:38.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.221+0000 7fe1f77fe700 1 -- 172.21.15.88:0/3567854643 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fe1f8052a50 con 0x7fe2080ff350 2024-02-19T19:04:38.349 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:38 smithi088 ceph-mon[29272]: from='client.24537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:38.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:38 smithi088 ceph-mon[29272]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:38.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:38 smithi132 ceph-mon[37969]: from='client.24537 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:38.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:38 smithi132 ceph-mon[37969]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:38.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.763+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3567854643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe2081a1870 con 0x7fe2080ff350 2024-02-19T19:04:38.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.763+0000 7fe1f77fe700 1 -- 172.21.15.88:0/3567854643 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fe1f8052a50 con 0x7fe2080ff350 2024-02-19T19:04:38.785 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:04:38.785 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:04:38.785 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-02-19T19:04:38.785 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:04:38.785 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:04:38.785 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:04:38.786 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:04:38.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.767+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3567854643 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe1f006d5f0 msgr2=0x7fe1f006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:38.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.767+0000 7fe20ca8b700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe1f006d5f0 0x7fe1f006faa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe20819da20 tx=0x7fe1fc005a90 comp rx=0 tx=0).stop 2024-02-19T19:04:38.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.767+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3567854643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2080ff350 msgr2=0x7fe20819c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:38.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.767+0000 7fe20ca8b700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2080ff350 0x7fe20819c400 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe1f800e920 tx=0x7fe1f800ec30 comp rx=0 tx=0).stop 2024-02-19T19:04:38.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.767+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3567854643 shutdown_connections 2024-02-19T19:04:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.767+0000 7fe20ca8b700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe1f006d5f0 0x7fe1f006faa0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.767+0000 7fe20ca8b700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe2080ff350 0x7fe20819c400 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.767+0000 7fe20ca8b700 1 --2- 172.21.15.88:0/3567854643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe2081038b0 0x7fe20819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.767+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3567854643 >> 172.21.15.88:0/3567854643 conn(0x7fe2080748b0 msgr2=0x7fe2080febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.768+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3567854643 shutdown_connections 2024-02-19T19:04:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.768+0000 7fe20ca8b700 1 -- 172.21.15.88:0/3567854643 wait complete. 2024-02-19T19:04:38.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.924+0000 7f1af359e700 1 -- 172.21.15.88:0/880297141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aec094490 msgr2=0x7f1aec094860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:38.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.924+0000 7f1af359e700 1 --2- 172.21.15.88:0/880297141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aec094490 0x7f1aec094860 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4009910 tx=0x7f1ae4009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:38.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.925+0000 7f1af359e700 1 -- 172.21.15.88:0/880297141 shutdown_connections 2024-02-19T19:04:38.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.925+0000 7f1af359e700 1 --2- 172.21.15.88:0/880297141 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1aec094e30 0x7f1aec0983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.925+0000 7f1af359e700 1 --2- 172.21.15.88:0/880297141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aec094490 0x7f1aec094860 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.925+0000 7f1af359e700 1 -- 172.21.15.88:0/880297141 >> 172.21.15.88:0/880297141 conn(0x7f1aec0097f0 msgr2=0x7f1aec009bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:38.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.925+0000 7f1af359e700 1 -- 172.21.15.88:0/880297141 shutdown_connections 2024-02-19T19:04:38.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.925+0000 7f1af359e700 1 -- 172.21.15.88:0/880297141 wait complete. 2024-02-19T19:04:38.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.926+0000 7f1af359e700 1 Processor -- start 2024-02-19T19:04:38.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.926+0000 7f1af359e700 1 -- start start 2024-02-19T19:04:38.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.927+0000 7f1af359e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aec094490 0x7f1aec1292d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:38.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.927+0000 7f1af359e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1aec094e30 0x7f1aec129810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:38.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.927+0000 7f1af259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aec094490 0x7f1aec1292d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:38.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.927+0000 7f1af359e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec129ef0 con 0x7f1aec094490 2024-02-19T19:04:38.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.927+0000 7f1af359e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec003ef0 con 0x7f1aec094e30 2024-02-19T19:04:38.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.927+0000 7f1af259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aec094490 0x7f1aec1292d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46480/0 (socket says 172.21.15.88:46480) 2024-02-19T19:04:38.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.927+0000 7f1af259c700 1 -- 172.21.15.88:0/636983109 learned_addr learned my addr 172.21.15.88:0/636983109 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:38.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.927+0000 7f1af1d9b700 1 --2- 172.21.15.88:0/636983109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1aec094e30 0x7f1aec129810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:38.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.928+0000 7f1af1d9b700 1 -- 172.21.15.88:0/636983109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aec094490 msgr2=0x7f1aec1292d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:38.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.928+0000 7f1af1d9b700 1 --2- 172.21.15.88:0/636983109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aec094490 0x7f1aec1292d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:38.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.928+0000 7f1af1d9b700 1 -- 172.21.15.88:0/636983109 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae40095f0 con 0x7f1aec094e30 2024-02-19T19:04:38.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.929+0000 7f1af1d9b700 1 --2- 172.21.15.88:0/636983109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1aec094e30 0x7f1aec129810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800e940 tx=0x7f1ae800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:38.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.929+0000 7f1ae37fe700 1 -- 172.21.15.88:0/636983109 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ae800c990 con 0x7f1aec094e30 2024-02-19T19:04:38.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.929+0000 7f1ae37fe700 1 -- 172.21.15.88:0/636983109 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ae8010430 con 0x7f1aec094e30 2024-02-19T19:04:38.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.929+0000 7f1af359e700 1 -- 172.21.15.88:0/636983109 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec004110 con 0x7f1aec094e30 2024-02-19T19:04:38.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.929+0000 7f1af359e700 1 -- 172.21.15.88:0/636983109 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec12dc80 con 0x7f1aec094e30 2024-02-19T19:04:38.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.930+0000 7f1ae37fe700 1 -- 172.21.15.88:0/636983109 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ae8018650 con 0x7f1aec094e30 2024-02-19T19:04:38.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.932+0000 7f1ae37fe700 1 -- 172.21.15.88:0/636983109 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f1ae800caf0 con 0x7f1aec094e30 2024-02-19T19:04:38.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.933+0000 7f1ae37fe700 1 --2- 172.21.15.88:0/636983109 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f1adc06d690 0x7f1adc06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:38.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.933+0000 7f1af359e700 1 -- 172.21.15.88:0/636983109 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aec0054d0 con 0x7f1aec094e30 2024-02-19T19:04:38.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.933+0000 7f1ae37fe700 1 -- 172.21.15.88:0/636983109 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1ae8014070 con 0x7f1aec094e30 2024-02-19T19:04:38.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.933+0000 7f1af259c700 1 --2- 172.21.15.88:0/636983109 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f1adc06d690 0x7f1adc06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:38.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.934+0000 7f1af259c700 1 --2- 172.21.15.88:0/636983109 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f1adc06d690 0x7f1adc06fb40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b760 tx=0x7f1ae4005a90 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:38.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:38.938+0000 7f1ae37fe700 1 -- 172.21.15.88:0/636983109 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f1ae8026030 con 0x7f1aec094e30 2024-02-19T19:04:39.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.351+0000 7f1af359e700 1 -- 172.21.15.88:0/636983109 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1aec12dff0 con 0x7f1aec094e30 2024-02-19T19:04:39.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.351+0000 7f1ae37fe700 1 -- 172.21.15.88:0/636983109 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1854 (secure 0 0 0) 0x7f1ae8056e30 con 0x7f1aec094e30 2024-02-19T19:04:39.373 INFO:teuthology.orchestra.run.smithi088.stdout:e19 2024-02-19T19:04:39.373 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 19 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:04:10.337219+0000 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:04:39.374 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:04:39.375 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:04:39.376 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:04:39.376 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:04:39.376 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:04:39.376 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:04:39.376 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:04:39.376 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:04:39.376 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:04:39.376 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:04:39.376 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:04:39.376 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:04:39.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.355+0000 7f1af359e700 1 -- 172.21.15.88:0/636983109 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f1adc06d690 msgr2=0x7f1adc06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:39.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.355+0000 7f1af359e700 1 --2- 172.21.15.88:0/636983109 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f1adc06d690 0x7f1adc06fb40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b760 tx=0x7f1ae4005a90 comp rx=0 tx=0).stop 2024-02-19T19:04:39.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.355+0000 7f1af359e700 1 -- 172.21.15.88:0/636983109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1aec094e30 msgr2=0x7f1aec129810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:39.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.355+0000 7f1af359e700 1 --2- 172.21.15.88:0/636983109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1aec094e30 0x7f1aec129810 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1ae800e940 tx=0x7f1ae800ec50 comp rx=0 tx=0).stop 2024-02-19T19:04:39.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.356+0000 7f1af359e700 1 -- 172.21.15.88:0/636983109 shutdown_connections 2024-02-19T19:04:39.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.356+0000 7f1af359e700 1 --2- 172.21.15.88:0/636983109 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f1adc06d690 0x7f1adc06fb40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:39.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.356+0000 7f1af359e700 1 --2- 172.21.15.88:0/636983109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aec094490 0x7f1aec1292d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:39.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.356+0000 7f1af359e700 1 --2- 172.21.15.88:0/636983109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1aec094e30 0x7f1aec129810 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:39.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.356+0000 7f1af359e700 1 -- 172.21.15.88:0/636983109 >> 172.21.15.88:0/636983109 conn(0x7f1aec0097f0 msgr2=0x7f1aec093a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:39.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.356+0000 7f1af359e700 1 -- 172.21.15.88:0/636983109 shutdown_connections 2024-02-19T19:04:39.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.356+0000 7f1af359e700 1 -- 172.21.15.88:0/636983109 wait complete. 2024-02-19T19:04:39.378 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 19 2024-02-19T19:04:39.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.521+0000 7fe39159a700 1 -- 172.21.15.88:0/558032574 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe37c004a00 con 0x7fe39410e9a0 2024-02-19T19:04:39.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.522+0000 7fe398e7e700 1 -- 172.21.15.88:0/558032574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe39410e9a0 msgr2=0x7fe39410ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:39.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.522+0000 7fe398e7e700 1 --2- 172.21.15.88:0/558032574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe39410e9a0 0x7fe39410ed70 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fe37c009910 tx=0x7fe37c009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:39.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.522+0000 7fe398e7e700 1 -- 172.21.15.88:0/558032574 shutdown_connections 2024-02-19T19:04:39.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.522+0000 7fe398e7e700 1 --2- 172.21.15.88:0/558032574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe394071ad0 0x7fe394071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:39.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.522+0000 7fe398e7e700 1 --2- 172.21.15.88:0/558032574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe39410e9a0 0x7fe39410ed70 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:39.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.522+0000 7fe398e7e700 1 -- 172.21.15.88:0/558032574 >> 172.21.15.88:0/558032574 conn(0x7fe39406c630 msgr2=0x7fe39406ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:39.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.522+0000 7fe398e7e700 1 -- 172.21.15.88:0/558032574 shutdown_connections 2024-02-19T19:04:39.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.522+0000 7fe398e7e700 1 -- 172.21.15.88:0/558032574 wait complete. 2024-02-19T19:04:39.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.523+0000 7fe398e7e700 1 Processor -- start 2024-02-19T19:04:39.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.523+0000 7fe398e7e700 1 -- start start 2024-02-19T19:04:39.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.524+0000 7fe398e7e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe394071ad0 0x7fe3941aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:39.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.524+0000 7fe398e7e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe39410e9a0 0x7fe3941aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:39.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.524+0000 7fe398e7e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3941ab680 con 0x7fe39410e9a0 2024-02-19T19:04:39.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.524+0000 7fe398e7e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3941a4b30 con 0x7fe394071ad0 2024-02-19T19:04:39.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.524+0000 7fe391d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe39410e9a0 0x7fe3941aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:39.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.524+0000 7fe391d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe39410e9a0 0x7fe3941aaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46502/0 (socket says 172.21.15.88:46502) 2024-02-19T19:04:39.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.524+0000 7fe391d9b700 1 -- 172.21.15.88:0/2827249197 learned_addr learned my addr 172.21.15.88:0/2827249197 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:39.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.525+0000 7fe391d9b700 1 -- 172.21.15.88:0/2827249197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe394071ad0 msgr2=0x7fe3941aaab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-19T19:04:39.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.525+0000 7fe391d9b700 1 --2- 172.21.15.88:0/2827249197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe394071ad0 0x7fe3941aaab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:39.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.525+0000 7fe391d9b700 1 -- 172.21.15.88:0/2827249197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0095f0 con 0x7fe39410e9a0 2024-02-19T19:04:39.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.525+0000 7fe391d9b700 1 --2- 172.21.15.88:0/2827249197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe39410e9a0 0x7fe3941aaff0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe38400e920 tx=0x7fe38400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:39.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.526+0000 7fe38b7fe700 1 -- 172.21.15.88:0/2827249197 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe38400cba0 con 0x7fe39410e9a0 2024-02-19T19:04:39.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.526+0000 7fe38b7fe700 1 -- 172.21.15.88:0/2827249197 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe38400cd00 con 0x7fe39410e9a0 2024-02-19T19:04:39.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.526+0000 7fe38b7fe700 1 -- 172.21.15.88:0/2827249197 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe384018770 con 0x7fe39410e9a0 2024-02-19T19:04:39.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.526+0000 7fe398e7e700 1 -- 172.21.15.88:0/2827249197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3941a4e10 con 0x7fe39410e9a0 2024-02-19T19:04:39.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.526+0000 7fe398e7e700 1 -- 172.21.15.88:0/2827249197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3941a5360 con 0x7fe39410e9a0 2024-02-19T19:04:39.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.528+0000 7fe398e7e700 1 -- 172.21.15.88:0/2827249197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe374005320 con 0x7fe39410e9a0 2024-02-19T19:04:39.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.529+0000 7fe38b7fe700 1 -- 172.21.15.88:0/2827249197 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7fe3840188d0 con 0x7fe39410e9a0 2024-02-19T19:04:39.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.529+0000 7fe38b7fe700 1 --2- 172.21.15.88:0/2827249197 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe38006d640 0x7fe38006faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:39.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.530+0000 7fe38b7fe700 1 -- 172.21.15.88:0/2827249197 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe384014070 con 0x7fe39410e9a0 2024-02-19T19:04:39.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.531+0000 7fe39259c700 1 --2- 172.21.15.88:0/2827249197 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe38006d640 0x7fe38006faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:39.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.532+0000 7fe39259c700 1 --2- 172.21.15.88:0/2827249197 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe38006d640 0x7fe38006faf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe37c000c00 tx=0x7fe37c009f70 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:39.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.539+0000 7fe38b7fe700 1 -- 172.21.15.88:0/2827249197 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7fe384052a50 con 0x7fe39410e9a0 2024-02-19T19:04:39.680 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: pgmap v96: 65 pgs: 65 active+clean; 107 MiB data, 546 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s wr, 191 op/s 2024-02-19T19:04:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (squid) 2024-02-19T19:04:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-19T19:04:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:04:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:04:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: Upgrade: Setting container_image for all mgr 2024-02-19T19:04:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-19T19:04:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/3567854643' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:04:39.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:39 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/636983109' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:04:39.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:39.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: pgmap v96: 65 pgs: 65 active+clean; 107 MiB data, 546 MiB used, 536 GiB / 536 GiB avail; 2.6 MiB/s wr, 191 op/s 2024-02-19T19:04:39.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:39.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: Upgrade: Target is version 19.0.0-1450-gf78a58c0 (squid) 2024-02-19T19:04:39.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, digests ['quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526'] 2024-02-19T19:04:39.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:04:39.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:04:39.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: Upgrade: Setting container_image for all mgr 2024-02-19T19:04:39.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:39.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-19T19:04:39.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/3567854643' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:04:39.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:39 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/636983109' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:04:39.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.943+0000 7fe398e7e700 1 -- 172.21.15.88:0/2827249197 --> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe374000bf0 con 0x7fe38006d640 2024-02-19T19:04:39.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.944+0000 7fe38b7fe700 1 -- 172.21.15.88:0/2827249197 <== mgr.14682 v2:172.21.15.88:6800/221244694 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fe374000bf0 con 0x7fe38006d640 2024-02-19T19:04:39.966 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:04:39.966 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:04:39.966 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:04:39.966 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:04:39.966 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:04:39.966 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:04:39.966 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:04:39.966 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "2/23 daemons upgraded", 2024-02-19T19:04:39.967 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mon daemons", 2024-02-19T19:04:39.967 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:04:39.967 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:04:39.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.947+0000 7fe398e7e700 1 -- 172.21.15.88:0/2827249197 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe38006d640 msgr2=0x7fe38006faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:39.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.947+0000 7fe398e7e700 1 --2- 172.21.15.88:0/2827249197 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe38006d640 0x7fe38006faf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe37c000c00 tx=0x7fe37c009f70 comp rx=0 tx=0).stop 2024-02-19T19:04:39.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.947+0000 7fe398e7e700 1 -- 172.21.15.88:0/2827249197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe39410e9a0 msgr2=0x7fe3941aaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:39.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.947+0000 7fe398e7e700 1 --2- 172.21.15.88:0/2827249197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe39410e9a0 0x7fe3941aaff0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe38400e920 tx=0x7fe38400ece0 comp rx=0 tx=0).stop 2024-02-19T19:04:39.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.947+0000 7fe398e7e700 1 -- 172.21.15.88:0/2827249197 shutdown_connections 2024-02-19T19:04:39.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.947+0000 7fe398e7e700 1 --2- 172.21.15.88:0/2827249197 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7fe38006d640 0x7fe38006faf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:39.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.947+0000 7fe398e7e700 1 --2- 172.21.15.88:0/2827249197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe394071ad0 0x7fe3941aaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:39.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.947+0000 7fe398e7e700 1 --2- 172.21.15.88:0/2827249197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe39410e9a0 0x7fe3941aaff0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:39.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.947+0000 7fe398e7e700 1 -- 172.21.15.88:0/2827249197 >> 172.21.15.88:0/2827249197 conn(0x7fe39406c630 msgr2=0x7fe3940703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:39.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.948+0000 7fe398e7e700 1 -- 172.21.15.88:0/2827249197 shutdown_connections 2024-02-19T19:04:39.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:39.948+0000 7fe398e7e700 1 -- 172.21.15.88:0/2827249197 wait complete. 2024-02-19T19:04:40.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.103+0000 7f8299ffb700 1 -- 172.21.15.88:0/3494697203 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82900054b0 con 0x7f829c101d90 2024-02-19T19:04:40.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.103+0000 7f82a1649700 1 -- 172.21.15.88:0/3494697203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f829c101d90 msgr2=0x7f829c10a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:40.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.103+0000 7f82a1649700 1 --2- 172.21.15.88:0/3494697203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f829c101d90 0x7f829c10a290 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8290009910 tx=0x7f8290009c20 comp rx=0 tx=0).stop 2024-02-19T19:04:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.104+0000 7f82a1649700 1 -- 172.21.15.88:0/3494697203 shutdown_connections 2024-02-19T19:04:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.104+0000 7f82a1649700 1 --2- 172.21.15.88:0/3494697203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f829c101d90 0x7f829c10a290 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.104+0000 7f82a1649700 1 --2- 172.21.15.88:0/3494697203 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f829c101480 0x7f829c101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.104+0000 7f82a1649700 1 -- 172.21.15.88:0/3494697203 >> 172.21.15.88:0/3494697203 conn(0x7f829c0fac80 msgr2=0x7f829c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.104+0000 7f82a1649700 1 -- 172.21.15.88:0/3494697203 shutdown_connections 2024-02-19T19:04:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.104+0000 7f82a1649700 1 -- 172.21.15.88:0/3494697203 wait complete. 2024-02-19T19:04:40.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.105+0000 7f82a1649700 1 Processor -- start 2024-02-19T19:04:40.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.105+0000 7f82a1649700 1 -- start start 2024-02-19T19:04:40.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.106+0000 7f82a1649700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f829c101480 0x7f829c195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:40.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.106+0000 7f82a1649700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f829c101d90 0x7f829c1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:40.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.106+0000 7f82a1649700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c196bc0 con 0x7f829c101d90 2024-02-19T19:04:40.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.106+0000 7f82a1649700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829c19a740 con 0x7f829c101480 2024-02-19T19:04:40.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.106+0000 7f829a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f829c101d90 0x7f829c1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:40.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.106+0000 7f829a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f829c101d90 0x7f829c1964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46524/0 (socket says 172.21.15.88:46524) 2024-02-19T19:04:40.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.106+0000 7f829a7fc700 1 -- 172.21.15.88:0/4251228594 learned_addr learned my addr 172.21.15.88:0/4251228594 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:04:40.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.106+0000 7f829affd700 1 --2- 172.21.15.88:0/4251228594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f829c101480 0x7f829c195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:40.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.106+0000 7f829a7fc700 1 -- 172.21.15.88:0/4251228594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f829c101480 msgr2=0x7f829c195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:40.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.106+0000 7f829a7fc700 1 --2- 172.21.15.88:0/4251228594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f829c101480 0x7f829c195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:40.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.106+0000 7f829a7fc700 1 -- 172.21.15.88:0/4251228594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82900095f0 con 0x7f829c101d90 2024-02-19T19:04:40.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.107+0000 7f829a7fc700 1 --2- 172.21.15.88:0/4251228594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f829c101d90 0x7f829c1964e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f82900092e0 tx=0x7f8290004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:40.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.107+0000 7f8283fff700 1 -- 172.21.15.88:0/4251228594 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f829001d070 con 0x7f829c101d90 2024-02-19T19:04:40.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.107+0000 7f82a1649700 1 -- 172.21.15.88:0/4251228594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829c19aa20 con 0x7f829c101d90 2024-02-19T19:04:40.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.107+0000 7f82a1649700 1 -- 172.21.15.88:0/4251228594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c19af70 con 0x7f829c101d90 2024-02-19T19:04:40.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.107+0000 7f8283fff700 1 -- 172.21.15.88:0/4251228594 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f829000beb0 con 0x7f829c101d90 2024-02-19T19:04:40.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.107+0000 7f8283fff700 1 -- 172.21.15.88:0/4251228594 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f829000f7b0 con 0x7f829c101d90 2024-02-19T19:04:40.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.109+0000 7f82a1649700 1 -- 172.21.15.88:0/4251228594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829c107920 con 0x7f829c101d90 2024-02-19T19:04:40.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.110+0000 7f8283fff700 1 -- 172.21.15.88:0/4251228594 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 32) v1 ==== 91346+0+0 (secure 0 0 0) 0x7f8290022470 con 0x7f829c101d90 2024-02-19T19:04:40.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.110+0000 7f8283fff700 1 --2- 172.21.15.88:0/4251228594 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f828406d520 0x7f828406f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:04:40.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.110+0000 7f8283fff700 1 -- 172.21.15.88:0/4251228594 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f829008e630 con 0x7f829c101d90 2024-02-19T19:04:40.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.111+0000 7f829affd700 1 --2- 172.21.15.88:0/4251228594 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f828406d520 0x7f828406f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:04:40.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.111+0000 7f829affd700 1 --2- 172.21.15.88:0/4251228594 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f828406d520 0x7f828406f9d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f828c006fd0 tx=0x7f828c008040 comp rx=0 tx=0).ready entity=mgr.14682 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:04:40.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.117+0000 7f8283fff700 1 -- 172.21.15.88:0/4251228594 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182845 (secure 0 0 0) 0x7f8290058440 con 0x7f829c101d90 2024-02-19T19:04:40.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.627+0000 7f82a1649700 1 -- 172.21.15.88:0/4251228594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f829c069650 con 0x7f829c101d90 2024-02-19T19:04:40.650 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_OK 2024-02-19T19:04:40.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.629+0000 7f8283fff700 1 -- 172.21.15.88:0/4251228594 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f829005bb00 con 0x7f829c101d90 2024-02-19T19:04:40.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.632+0000 7f82a1649700 1 -- 172.21.15.88:0/4251228594 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f828406d520 msgr2=0x7f828406f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:40.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.632+0000 7f82a1649700 1 --2- 172.21.15.88:0/4251228594 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f828406d520 0x7f828406f9d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f828c006fd0 tx=0x7f828c008040 comp rx=0 tx=0).stop 2024-02-19T19:04:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.632+0000 7f82a1649700 1 -- 172.21.15.88:0/4251228594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f829c101d90 msgr2=0x7f829c1964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:04:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.632+0000 7f82a1649700 1 --2- 172.21.15.88:0/4251228594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f829c101d90 0x7f829c1964e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f82900092e0 tx=0x7f8290004970 comp rx=0 tx=0).stop 2024-02-19T19:04:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.633+0000 7f82a1649700 1 -- 172.21.15.88:0/4251228594 shutdown_connections 2024-02-19T19:04:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.633+0000 7f82a1649700 1 --2- 172.21.15.88:0/4251228594 >> [v2:172.21.15.88:6800/221244694,v1:172.21.15.88:6801/221244694] conn(0x7f828406d520 0x7f828406f9d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.633+0000 7f82a1649700 1 --2- 172.21.15.88:0/4251228594 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f829c101480 0x7f829c195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.633+0000 7f82a1649700 1 --2- 172.21.15.88:0/4251228594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f829c101d90 0x7f829c1964e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:04:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.633+0000 7f82a1649700 1 -- 172.21.15.88:0/4251228594 >> 172.21.15.88:0/4251228594 conn(0x7f829c0fac80 msgr2=0x7f829c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:04:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.633+0000 7f82a1649700 1 -- 172.21.15.88:0/4251228594 shutdown_connections 2024-02-19T19:04:40.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:04:40.633+0000 7f82a1649700 1 -- 172.21.15.88:0/4251228594 wait complete. 2024-02-19T19:04:40.729 INFO:tasks.workunit.client.1.smithi132.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-19T19:04:40.735 INFO:tasks.workunit.client.1.smithi132.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-19T19:04:40.735 INFO:tasks.workunit.client.1.smithi132.stderr:+ make 2024-02-19T19:04:40.961 INFO:tasks.workunit.client.1.smithi132.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-19T19:04:41.288 INFO:tasks.workunit.client.1.smithi132.stderr:++ readlink -f fsstress 2024-02-19T19:04:41.290 INFO:tasks.workunit.client.1.smithi132.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-19T19:04:41.290 INFO:tasks.workunit.client.1.smithi132.stderr:+ popd 2024-02-19T19:04:41.291 INFO:tasks.workunit.client.1.smithi132.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-02-19T19:04:41.291 INFO:tasks.workunit.client.1.smithi132.stderr:+ popd 2024-02-19T19:04:41.292 INFO:tasks.workunit.client.1.smithi132.stdout:~/cephtest/mnt.1/client.1/tmp 2024-02-19T19:04:41.292 INFO:tasks.workunit.client.1.smithi132.stderr:++ mktemp -d -p . 2024-02-19T19:04:41.295 INFO:tasks.workunit.client.1.smithi132.stderr:+ T=./tmp.eGsJfOogpO 2024-02-19T19:04:41.295 INFO:tasks.workunit.client.1.smithi132.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.eGsJfOogpO -l 1 -n 1000 -p 10 -v 2024-02-19T19:04:41.298 INFO:tasks.workunit.client.1.smithi132.stdout:seed = 1708647301 2024-02-19T19:04:41.302 INFO:tasks.workunit.client.1.smithi132.stdout:0/0: link - no file 2024-02-19T19:04:41.302 INFO:tasks.workunit.client.1.smithi132.stdout:0/1: readlink - no filename 2024-02-19T19:04:41.305 INFO:tasks.workunit.client.1.smithi132.stdout:0/2: creat f0 x:0 0 0 2024-02-19T19:04:41.305 INFO:tasks.workunit.client.1.smithi132.stdout:0/3: read - f0 zero size 2024-02-19T19:04:41.305 INFO:tasks.workunit.client.1.smithi132.stdout:0/4: dread - f0 zero size 2024-02-19T19:04:41.306 INFO:tasks.workunit.client.1.smithi132.stdout:0/5: dread - f0 zero size 2024-02-19T19:04:41.306 INFO:tasks.workunit.client.1.smithi132.stdout:2/0: chown . 4 1 2024-02-19T19:04:41.306 INFO:tasks.workunit.client.1.smithi132.stdout:0/6: truncate f0 89427 0 2024-02-19T19:04:41.307 INFO:tasks.workunit.client.1.smithi132.stdout:1/0: creat f0 x:0 0 0 2024-02-19T19:04:41.308 INFO:tasks.workunit.client.1.smithi132.stdout:0/7: write f0 [799824,117061] 0 2024-02-19T19:04:41.311 INFO:tasks.workunit.client.1.smithi132.stdout:3/0: truncate - no filename 2024-02-19T19:04:41.311 INFO:tasks.workunit.client.1.smithi132.stdout:3/1: read - no filename 2024-02-19T19:04:41.311 INFO:tasks.workunit.client.1.smithi132.stdout:0/8: write f0 [1498018,14357] 0 2024-02-19T19:04:41.311 INFO:tasks.workunit.client.1.smithi132.stdout:1/1: mkdir d1 0 2024-02-19T19:04:41.312 INFO:tasks.workunit.client.1.smithi132.stdout:2/1: mkdir d0 0 2024-02-19T19:04:41.313 INFO:tasks.workunit.client.1.smithi132.stdout:4/0: dread - no filename 2024-02-19T19:04:41.313 INFO:tasks.workunit.client.1.smithi132.stdout:4/1: dwrite - no filename 2024-02-19T19:04:41.313 INFO:tasks.workunit.client.1.smithi132.stdout:4/2: dwrite - no filename 2024-02-19T19:04:41.313 INFO:tasks.workunit.client.1.smithi132.stdout:4/3: dwrite - no filename 2024-02-19T19:04:41.313 INFO:tasks.workunit.client.1.smithi132.stdout:4/4: rmdir - no directory 2024-02-19T19:04:41.314 INFO:tasks.workunit.client.1.smithi132.stdout:3/2: creat f0 x:0 0 0 2024-02-19T19:04:41.314 INFO:tasks.workunit.client.1.smithi132.stdout:2/2: symlink d0/l1 0 2024-02-19T19:04:41.316 INFO:tasks.workunit.client.1.smithi132.stdout:3/3: creat f1 x:0 0 0 2024-02-19T19:04:41.317 INFO:tasks.workunit.client.1.smithi132.stdout:5/0: chown . 6975 1 2024-02-19T19:04:41.317 INFO:tasks.workunit.client.1.smithi132.stdout:3/4: stat f0 0 2024-02-19T19:04:41.318 INFO:tasks.workunit.client.1.smithi132.stdout:1/2: dwrite f0 [0,4194304] 0 2024-02-19T19:04:41.318 INFO:tasks.workunit.client.1.smithi132.stdout:2/3: mknod d0/c2 0 2024-02-19T19:04:41.319 INFO:tasks.workunit.client.1.smithi132.stdout:2/4: chown d0/l1 193233 1 2024-02-19T19:04:41.320 INFO:tasks.workunit.client.1.smithi132.stdout:3/5: mknod c2 0 2024-02-19T19:04:41.321 INFO:tasks.workunit.client.1.smithi132.stdout:3/6: chown c2 125 1 2024-02-19T19:04:41.322 INFO:tasks.workunit.client.1.smithi132.stdout:3/7: dread - f0 zero size 2024-02-19T19:04:41.323 INFO:tasks.workunit.client.1.smithi132.stdout:3/8: rmdir - no directory 2024-02-19T19:04:41.323 INFO:tasks.workunit.client.1.smithi132.stdout:5/1: creat f0 x:0 0 0 2024-02-19T19:04:41.328 INFO:tasks.workunit.client.1.smithi132.stdout:1/3: dwrite f0 [0,4194304] 0 2024-02-19T19:04:41.328 INFO:tasks.workunit.client.1.smithi132.stdout:2/5: creat d0/f3 x:0 0 0 2024-02-19T19:04:41.329 INFO:tasks.workunit.client.1.smithi132.stdout:3/9: creat f3 x:0 0 0 2024-02-19T19:04:41.330 INFO:tasks.workunit.client.1.smithi132.stdout:2/6: read - d0/f3 zero size 2024-02-19T19:04:41.332 INFO:tasks.workunit.client.1.smithi132.stdout:3/10: write f3 [386236,26112] 0 2024-02-19T19:04:41.333 INFO:tasks.workunit.client.1.smithi132.stdout:6/0: mkdir d0 0 2024-02-19T19:04:41.333 INFO:tasks.workunit.client.1.smithi132.stdout:6/1: fdatasync - no filename 2024-02-19T19:04:41.333 INFO:tasks.workunit.client.1.smithi132.stdout:6/2: dwrite - no filename 2024-02-19T19:04:41.333 INFO:tasks.workunit.client.1.smithi132.stdout:6/3: write - no filename 2024-02-19T19:04:41.334 INFO:tasks.workunit.client.1.smithi132.stdout:2/7: read - d0/f3 zero size 2024-02-19T19:04:41.334 INFO:tasks.workunit.client.1.smithi132.stdout:8/0: rename - no filename 2024-02-19T19:04:41.334 INFO:tasks.workunit.client.1.smithi132.stdout:8/1: fsync - no filename 2024-02-19T19:04:41.335 INFO:tasks.workunit.client.1.smithi132.stdout:8/2: chown . 1558463543 1 2024-02-19T19:04:41.335 INFO:tasks.workunit.client.1.smithi132.stdout:8/3: read - no filename 2024-02-19T19:04:41.337 INFO:tasks.workunit.client.1.smithi132.stdout:3/11: write f3 [654170,95770] 0 2024-02-19T19:04:41.337 INFO:tasks.workunit.client.1.smithi132.stdout:5/2: dwrite f0 [0,4194304] 0 2024-02-19T19:04:41.341 INFO:tasks.workunit.client.1.smithi132.stdout:7/0: mknod c0 0 2024-02-19T19:04:41.341 INFO:tasks.workunit.client.1.smithi132.stdout:7/1: write - no filename 2024-02-19T19:04:41.341 INFO:tasks.workunit.client.1.smithi132.stdout:7/2: truncate - no filename 2024-02-19T19:04:41.341 INFO:tasks.workunit.client.1.smithi132.stdout:7/3: write - no filename 2024-02-19T19:04:41.343 INFO:tasks.workunit.client.1.smithi132.stdout:2/8: mkdir d0/d4 0 2024-02-19T19:04:41.347 INFO:tasks.workunit.client.1.smithi132.stdout:2/9: write d0/f3 [879677,50323] 0 2024-02-19T19:04:41.349 INFO:tasks.workunit.client.1.smithi132.stdout:7/4: rename c0 to c1 0 2024-02-19T19:04:41.349 INFO:tasks.workunit.client.1.smithi132.stdout:7/5: write - no filename 2024-02-19T19:04:41.349 INFO:tasks.workunit.client.1.smithi132.stdout:2/10: chown d0/f3 32389927 1 2024-02-19T19:04:41.350 INFO:tasks.workunit.client.1.smithi132.stdout:6/4: mkdir d0/d1 0 2024-02-19T19:04:41.351 INFO:tasks.workunit.client.1.smithi132.stdout:5/3: dwrite f0 [0,4194304] 0 2024-02-19T19:04:41.351 INFO:tasks.workunit.client.1.smithi132.stdout:3/12: dwrite f0 [0,4194304] 0 2024-02-19T19:04:41.351 INFO:tasks.workunit.client.1.smithi132.stdout:8/4: creat f0 x:0 0 0 2024-02-19T19:04:41.353 INFO:tasks.workunit.client.1.smithi132.stdout:3/13: write f0 [810204,14542] 0 2024-02-19T19:04:41.353 INFO:tasks.workunit.client.1.smithi132.stdout:8/5: write f0 [168552,76383] 0 2024-02-19T19:04:41.354 INFO:tasks.workunit.client.1.smithi132.stdout:7/6: symlink l2 0 2024-02-19T19:04:41.354 INFO:tasks.workunit.client.1.smithi132.stdout:7/7: rmdir - no directory 2024-02-19T19:04:41.354 INFO:tasks.workunit.client.1.smithi132.stdout:1/4: fsync f0 0 2024-02-19T19:04:41.354 INFO:tasks.workunit.client.1.smithi132.stdout:9/0: mknod c0 0 2024-02-19T19:04:41.358 INFO:tasks.workunit.client.1.smithi132.stdout:2/11: read d0/f3 [438125,76233] 0 2024-02-19T19:04:41.358 INFO:tasks.workunit.client.1.smithi132.stdout:1/5: write f0 [3737677,84907] 0 2024-02-19T19:04:41.360 INFO:tasks.workunit.client.1.smithi132.stdout:5/4: symlink l1 0 2024-02-19T19:04:41.362 INFO:tasks.workunit.client.1.smithi132.stdout:5/5: stat l1 0 2024-02-19T19:04:41.363 INFO:tasks.workunit.client.1.smithi132.stdout:3/14: unlink f1 0 2024-02-19T19:04:41.364 INFO:tasks.workunit.client.1.smithi132.stdout:5/6: write f0 [1487228,65382] 0 2024-02-19T19:04:41.367 INFO:tasks.workunit.client.1.smithi132.stdout:7/8: rename l2 to l3 0 2024-02-19T19:04:41.367 INFO:tasks.workunit.client.1.smithi132.stdout:7/9: fsync - no filename 2024-02-19T19:04:41.367 INFO:tasks.workunit.client.1.smithi132.stdout:7/10: dread - no filename 2024-02-19T19:04:41.367 INFO:tasks.workunit.client.1.smithi132.stdout:8/6: dwrite f0 [0,4194304] 0 2024-02-19T19:04:41.368 INFO:tasks.workunit.client.1.smithi132.stdout:9/1: creat f1 x:0 0 0 2024-02-19T19:04:41.369 INFO:tasks.workunit.client.1.smithi132.stdout:9/2: truncate f1 151607 0 2024-02-19T19:04:41.369 INFO:tasks.workunit.client.1.smithi132.stdout:5/7: dread f0 [0,4194304] 0 2024-02-19T19:04:41.370 INFO:tasks.workunit.client.1.smithi132.stdout:6/5: mknod d0/d1/c2 0 2024-02-19T19:04:41.371 INFO:tasks.workunit.client.1.smithi132.stdout:6/6: stat d0/d1/c2 0 2024-02-19T19:04:41.372 INFO:tasks.workunit.client.1.smithi132.stdout:2/12: fsync d0/f3 0 2024-02-19T19:04:41.375 INFO:tasks.workunit.client.1.smithi132.stdout:1/6: rename f0 to d1/f2 0 2024-02-19T19:04:41.379 INFO:tasks.workunit.client.1.smithi132.stdout:3/15: unlink c2 0 2024-02-19T19:04:41.380 INFO:tasks.workunit.client.1.smithi132.stdout:3/16: chown f0 799 1 2024-02-19T19:04:41.384 INFO:tasks.workunit.client.1.smithi132.stdout:9/3: creat f2 x:0 0 0 2024-02-19T19:04:41.386 INFO:tasks.workunit.client.1.smithi132.stdout:6/7: mknod d0/d1/c3 0 2024-02-19T19:04:41.387 INFO:tasks.workunit.client.1.smithi132.stdout:6/8: chown d0 2064 1 2024-02-19T19:04:41.387 INFO:tasks.workunit.client.1.smithi132.stdout:6/9: write - no filename 2024-02-19T19:04:41.387 INFO:tasks.workunit.client.1.smithi132.stdout:2/13: creat d0/f5 x:0 0 0 2024-02-19T19:04:41.392 INFO:tasks.workunit.client.1.smithi132.stdout:3/17: creat f4 x:0 0 0 2024-02-19T19:04:41.393 INFO:tasks.workunit.client.1.smithi132.stdout:2/14: dwrite d0/f3 [0,4194304] 0 2024-02-19T19:04:41.396 INFO:tasks.workunit.client.1.smithi132.stdout:8/7: rename f0 to f1 0 2024-02-19T19:04:41.397 INFO:tasks.workunit.client.1.smithi132.stdout:9/4: mknod c3 0 2024-02-19T19:04:41.398 INFO:tasks.workunit.client.1.smithi132.stdout:9/5: truncate f2 806139 0 2024-02-19T19:04:41.399 INFO:tasks.workunit.client.1.smithi132.stdout:9/6: write f2 [1431897,129982] 0 2024-02-19T19:04:41.399 INFO:tasks.workunit.client.1.smithi132.stdout:9/7: write f2 [344967,111808] 0 2024-02-19T19:04:41.400 INFO:tasks.workunit.client.1.smithi132.stdout:9/8: truncate f2 1740691 0 2024-02-19T19:04:41.406 INFO:tasks.workunit.client.1.smithi132.stdout:7/11: rename l3 to l4 0 2024-02-19T19:04:41.406 INFO:tasks.workunit.client.1.smithi132.stdout:7/12: dread - no filename 2024-02-19T19:04:41.406 INFO:tasks.workunit.client.1.smithi132.stdout:7/13: write - no filename 2024-02-19T19:04:41.407 INFO:tasks.workunit.client.1.smithi132.stdout:2/15: creat d0/f6 x:0 0 0 2024-02-19T19:04:41.408 INFO:tasks.workunit.client.1.smithi132.stdout:9/9: dwrite f1 [0,4194304] 0 2024-02-19T19:04:41.408 INFO:tasks.workunit.client.1.smithi132.stdout:2/16: dread - d0/f6 zero size 2024-02-19T19:04:41.409 INFO:tasks.workunit.client.1.smithi132.stdout:5/8: link l1 l2 0 2024-02-19T19:04:41.412 INFO:tasks.workunit.client.1.smithi132.stdout:2/17: dread d0/f3 [0,4194304] 0 2024-02-19T19:04:41.414 INFO:tasks.workunit.client.1.smithi132.stdout:9/10: dread f2 [0,4194304] 0 2024-02-19T19:04:41.414 INFO:tasks.workunit.client.1.smithi132.stdout:9/11: chown c0 560524800 1 2024-02-19T19:04:41.415 INFO:tasks.workunit.client.1.smithi132.stdout:7/14: creat f5 x:0 0 0 2024-02-19T19:04:41.420 INFO:tasks.workunit.client.1.smithi132.stdout:9/12: dwrite f1 [0,4194304] 0 2024-02-19T19:04:41.425 INFO:tasks.workunit.client.1.smithi132.stdout:5/9: dread f0 [0,4194304] 0 2024-02-19T19:04:41.425 INFO:tasks.workunit.client.1.smithi132.stdout:7/15: creat f6 x:0 0 0 2024-02-19T19:04:41.426 INFO:tasks.workunit.client.1.smithi132.stdout:7/16: write f5 [739866,59462] 0 2024-02-19T19:04:41.429 INFO:tasks.workunit.client.1.smithi132.stdout:5/10: dread f0 [0,4194304] 0 2024-02-19T19:04:41.439 INFO:tasks.workunit.client.1.smithi132.stdout:9/13: creat f4 x:0 0 0 2024-02-19T19:04:41.439 INFO:tasks.workunit.client.1.smithi132.stdout:9/14: write f4 [575204,127575] 0 2024-02-19T19:04:41.440 INFO:tasks.workunit.client.1.smithi132.stdout:2/18: symlink d0/d4/l7 0 2024-02-19T19:04:41.443 INFO:tasks.workunit.client.1.smithi132.stdout:5/11: readlink l2 0 2024-02-19T19:04:41.443 INFO:tasks.workunit.client.1.smithi132.stdout:5/12: stat l1 0 2024-02-19T19:04:41.443 INFO:tasks.workunit.client.1.smithi132.stdout:8/8: getdents . 0 2024-02-19T19:04:41.445 INFO:tasks.workunit.client.1.smithi132.stdout:2/19: mknod d0/d4/c8 0 2024-02-19T19:04:41.445 INFO:tasks.workunit.client.1.smithi132.stdout:9/15: dread f2 [0,4194304] 0 2024-02-19T19:04:41.446 INFO:tasks.workunit.client.1.smithi132.stdout:5/13: dread f0 [0,4194304] 0 2024-02-19T19:04:41.446 INFO:tasks.workunit.client.1.smithi132.stdout:7/17: rename c1 to c7 0 2024-02-19T19:04:41.448 INFO:tasks.workunit.client.1.smithi132.stdout:8/9: unlink f1 0 2024-02-19T19:04:41.449 INFO:tasks.workunit.client.1.smithi132.stdout:9/16: creat f5 x:0 0 0 2024-02-19T19:04:41.450 INFO:tasks.workunit.client.1.smithi132.stdout:2/20: mknod d0/d4/c9 0 2024-02-19T19:04:41.452 INFO:tasks.workunit.client.1.smithi132.stdout:5/14: rename l1 to l3 0 2024-02-19T19:04:41.452 INFO:tasks.workunit.client.1.smithi132.stdout:5/15: rmdir - no directory 2024-02-19T19:04:41.453 INFO:tasks.workunit.client.1.smithi132.stdout:7/18: creat f8 x:0 0 0 2024-02-19T19:04:41.454 INFO:tasks.workunit.client.1.smithi132.stdout:8/10: creat f2 x:0 0 0 2024-02-19T19:04:41.454 INFO:tasks.workunit.client.1.smithi132.stdout:8/11: write f2 [44537,50011] 0 2024-02-19T19:04:41.456 INFO:tasks.workunit.client.1.smithi132.stdout:9/17: symlink l6 0 2024-02-19T19:04:41.457 INFO:tasks.workunit.client.1.smithi132.stdout:2/21: write d0/f3 [1187466,104927] 0 2024-02-19T19:04:41.457 INFO:tasks.workunit.client.1.smithi132.stdout:2/22: read - d0/f6 zero size 2024-02-19T19:04:41.461 INFO:tasks.workunit.client.1.smithi132.stdout:7/19: symlink l9 0 2024-02-19T19:04:41.461 INFO:tasks.workunit.client.1.smithi132.stdout:8/12: mknod c3 0 2024-02-19T19:04:41.462 INFO:tasks.workunit.client.1.smithi132.stdout:9/18: write f2 [2241685,36040] 0 2024-02-19T19:04:41.464 INFO:tasks.workunit.client.1.smithi132.stdout:2/23: mkdir d0/d4/da 0 2024-02-19T19:04:41.468 INFO:tasks.workunit.client.1.smithi132.stdout:8/13: mkdir d4 0 2024-02-19T19:04:41.469 INFO:tasks.workunit.client.1.smithi132.stdout:8/14: truncate f2 222127 0 2024-02-19T19:04:41.472 INFO:tasks.workunit.client.1.smithi132.stdout:9/19: mknod c7 0 2024-02-19T19:04:41.474 INFO:tasks.workunit.client.1.smithi132.stdout:2/24: creat d0/d4/fb x:0 0 0 2024-02-19T19:04:41.474 INFO:tasks.workunit.client.1.smithi132.stdout:2/25: readlink d0/l1 0 2024-02-19T19:04:41.477 INFO:tasks.workunit.client.1.smithi132.stdout:2/26: dread d0/f3 [0,4194304] 0 2024-02-19T19:04:41.477 INFO:tasks.workunit.client.1.smithi132.stdout:2/27: write d0/d4/fb [674963,46776] 0 2024-02-19T19:04:41.477 INFO:tasks.workunit.client.1.smithi132.stdout:2/28: stat d0 0 2024-02-19T19:04:41.480 INFO:tasks.workunit.client.1.smithi132.stdout:9/20: rename c3 to c8 0 2024-02-19T19:04:41.480 INFO:tasks.workunit.client.1.smithi132.stdout:9/21: write f5 [3661,126562] 0 2024-02-19T19:04:41.484 INFO:tasks.workunit.client.1.smithi132.stdout:8/15: creat d4/f5 x:0 0 0 2024-02-19T19:04:41.484 INFO:tasks.workunit.client.1.smithi132.stdout:5/16: write f0 [3827624,6002] 0 2024-02-19T19:04:41.486 INFO:tasks.workunit.client.1.smithi132.stdout:8/16: read f2 [123847,612] 0 2024-02-19T19:04:41.487 INFO:tasks.workunit.client.1.smithi132.stdout:8/17: dread - d4/f5 zero size 2024-02-19T19:04:41.487 INFO:tasks.workunit.client.1.smithi132.stdout:9/22: dwrite f4 [0,4194304] 0 2024-02-19T19:04:41.489 INFO:tasks.workunit.client.1.smithi132.stdout:7/20: link l4 la 0 2024-02-19T19:04:41.490 INFO:tasks.workunit.client.1.smithi132.stdout:8/18: dread f2 [0,4194304] 0 2024-02-19T19:04:41.490 INFO:tasks.workunit.client.1.smithi132.stdout:5/17: mkdir d4 0 2024-02-19T19:04:41.492 INFO:tasks.workunit.client.1.smithi132.stdout:7/21: read - f8 zero size 2024-02-19T19:04:41.492 INFO:tasks.workunit.client.1.smithi132.stdout:7/22: rmdir - no directory 2024-02-19T19:04:41.493 INFO:tasks.workunit.client.1.smithi132.stdout:2/29: rmdir d0/d4/da 0 2024-02-19T19:04:41.493 INFO:tasks.workunit.client.1.smithi132.stdout:8/19: dread - d4/f5 zero size 2024-02-19T19:04:41.497 INFO:tasks.workunit.client.1.smithi132.stdout:2/30: mknod d0/d4/cc 0 2024-02-19T19:04:41.498 INFO:tasks.workunit.client.1.smithi132.stdout:9/23: dwrite f2 [0,4194304] 0 2024-02-19T19:04:41.499 INFO:tasks.workunit.client.1.smithi132.stdout:5/18: dread f0 [0,4194304] 0 2024-02-19T19:04:41.499 INFO:tasks.workunit.client.1.smithi132.stdout:8/20: rename d4/f5 to d4/f6 0 2024-02-19T19:04:41.500 INFO:tasks.workunit.client.1.smithi132.stdout:7/23: chown l4 3 1 2024-02-19T19:04:41.505 INFO:tasks.workunit.client.1.smithi132.stdout:2/31: creat d0/fd x:0 0 0 2024-02-19T19:04:41.506 INFO:tasks.workunit.client.1.smithi132.stdout:5/19: write f0 [2758048,101855] 0 2024-02-19T19:04:41.507 INFO:tasks.workunit.client.1.smithi132.stdout:8/21: symlink d4/l7 0 2024-02-19T19:04:41.508 INFO:tasks.workunit.client.1.smithi132.stdout:8/22: readlink d4/l7 0 2024-02-19T19:04:41.508 INFO:tasks.workunit.client.1.smithi132.stdout:8/23: readlink d4/l7 0 2024-02-19T19:04:41.509 INFO:tasks.workunit.client.1.smithi132.stdout:8/24: dread - d4/f6 zero size 2024-02-19T19:04:41.509 INFO:tasks.workunit.client.1.smithi132.stdout:7/24: mknod cb 0 2024-02-19T19:04:41.513 INFO:tasks.workunit.client.1.smithi132.stdout:2/32: symlink d0/le 0 2024-02-19T19:04:41.513 INFO:tasks.workunit.client.1.smithi132.stdout:2/33: chown d0/l1 170598 1 2024-02-19T19:04:41.515 INFO:tasks.workunit.client.1.smithi132.stdout:7/25: symlink lc 0 2024-02-19T19:04:41.520 INFO:tasks.workunit.client.1.smithi132.stdout:2/34: mkdir d0/d4/df 0 2024-02-19T19:04:41.522 INFO:tasks.workunit.client.1.smithi132.stdout:8/25: link f2 d4/f8 0 2024-02-19T19:04:41.524 INFO:tasks.workunit.client.1.smithi132.stdout:7/26: unlink f5 0 2024-02-19T19:04:41.525 INFO:tasks.workunit.client.1.smithi132.stdout:2/35: mknod d0/d4/c10 0 2024-02-19T19:04:41.527 INFO:tasks.workunit.client.1.smithi132.stdout:9/24: truncate f4 1835936 0 2024-02-19T19:04:41.531 INFO:tasks.workunit.client.1.smithi132.stdout:7/27: creat fd x:0 0 0 2024-02-19T19:04:41.531 INFO:tasks.workunit.client.1.smithi132.stdout:7/28: write f8 [165388,67358] 0 2024-02-19T19:04:41.533 INFO:tasks.workunit.client.1.smithi132.stdout:2/36: mkdir d0/d4/d11 0 2024-02-19T19:04:41.535 INFO:tasks.workunit.client.1.smithi132.stdout:5/20: truncate f0 372044 0 2024-02-19T19:04:41.537 INFO:tasks.workunit.client.1.smithi132.stdout:7/29: mknod ce 0 2024-02-19T19:04:41.538 INFO:tasks.workunit.client.1.smithi132.stdout:2/37: dwrite d0/f6 [0,4194304] 0 2024-02-19T19:04:41.539 INFO:tasks.workunit.client.1.smithi132.stdout:9/25: rename c8 to c9 0 2024-02-19T19:04:41.549 INFO:tasks.workunit.client.1.smithi132.stdout:2/38: rename d0/fd to d0/d4/f12 0 2024-02-19T19:04:41.549 INFO:tasks.workunit.client.1.smithi132.stdout:2/39: readlink d0/l1 0 2024-02-19T19:04:41.550 INFO:tasks.workunit.client.1.smithi132.stdout:9/26: creat fa x:0 0 0 2024-02-19T19:04:41.552 INFO:tasks.workunit.client.1.smithi132.stdout:7/30: link fd ff 0 2024-02-19T19:04:41.554 INFO:tasks.workunit.client.1.smithi132.stdout:5/21: write f0 [693719,36305] 0 2024-02-19T19:04:41.557 INFO:tasks.workunit.client.1.smithi132.stdout:8/26: dwrite d4/f8 [0,4194304] 0 2024-02-19T19:04:41.557 INFO:tasks.workunit.client.1.smithi132.stdout:5/22: write f0 [788264,15581] 0 2024-02-19T19:04:41.560 INFO:tasks.workunit.client.1.smithi132.stdout:7/31: dread - ff zero size 2024-02-19T19:04:41.561 INFO:tasks.workunit.client.1.smithi132.stdout:9/27: dwrite f1 [0,4194304] 0 2024-02-19T19:04:41.573 INFO:tasks.workunit.client.1.smithi132.stdout:8/27: rmdir d4 39 2024-02-19T19:04:41.574 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:41 smithi088 ceph-mon[29272]: from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:41.574 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:41 smithi088 ceph-mon[29272]: pgmap v97: 65 pgs: 65 active+clean; 107 MiB data, 546 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s wr, 72 op/s 2024-02-19T19:04:41.574 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:41.574 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:41 smithi088 ceph-mon[29272]: from='client.? 172.21.15.88:0/4251228594' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:04:41.574 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T19:04:41.574 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T19:04:41.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:41 smithi088 ceph-mon[29272]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:04:41.576 INFO:tasks.workunit.client.1.smithi132.stdout:7/32: unlink fd 0 2024-02-19T19:04:41.576 INFO:tasks.workunit.client.1.smithi132.stdout:7/33: truncate f8 673286 0 2024-02-19T19:04:41.579 INFO:tasks.workunit.client.1.smithi132.stdout:2/40: link d0/d4/c10 d0/c13 0 2024-02-19T19:04:41.580 INFO:tasks.workunit.client.1.smithi132.stdout:2/41: chown d0/d4/c9 512315 1 2024-02-19T19:04:41.580 INFO:tasks.workunit.client.1.smithi132.stdout:2/42: stat d0/d4/c9 0 2024-02-19T19:04:41.582 INFO:tasks.workunit.client.1.smithi132.stdout:8/28: dread - d4/f6 zero size 2024-02-19T19:04:41.582 INFO:tasks.workunit.client.1.smithi132.stdout:8/29: fdatasync f2 0 2024-02-19T19:04:41.582 INFO:tasks.workunit.client.1.smithi132.stdout:7/34: dread - ff zero size 2024-02-19T19:04:41.584 INFO:tasks.workunit.client.1.smithi132.stdout:9/28: link f4 fb 0 2024-02-19T19:04:41.586 INFO:tasks.workunit.client.1.smithi132.stdout:9/29: dread f5 [0,4194304] 0 2024-02-19T19:04:41.590 INFO:tasks.workunit.client.1.smithi132.stdout:5/23: rmdir d4 0 2024-02-19T19:04:41.594 INFO:tasks.workunit.client.1.smithi132.stdout:2/43: creat d0/d4/f14 x:0 0 0 2024-02-19T19:04:41.595 INFO:tasks.workunit.client.1.smithi132.stdout:7/35: creat f10 x:0 0 0 2024-02-19T19:04:41.596 INFO:tasks.workunit.client.1.smithi132.stdout:7/36: write f6 [849119,109329] 0 2024-02-19T19:04:41.597 INFO:tasks.workunit.client.1.smithi132.stdout:9/30: symlink lc 0 2024-02-19T19:04:41.597 INFO:tasks.workunit.client.1.smithi132.stdout:7/37: read f8 [32834,117943] 0 2024-02-19T19:04:41.598 INFO:tasks.workunit.client.1.smithi132.stdout:5/24: chown l2 0 1 2024-02-19T19:04:41.598 INFO:tasks.workunit.client.1.smithi132.stdout:7/38: chown lc 92689090 1 2024-02-19T19:04:41.601 INFO:tasks.workunit.client.1.smithi132.stdout:5/25: dread f0 [0,4194304] 0 2024-02-19T19:04:41.601 INFO:tasks.workunit.client.1.smithi132.stdout:8/30: link d4/f8 d4/f9 0 2024-02-19T19:04:41.602 INFO:tasks.workunit.client.1.smithi132.stdout:5/26: fdatasync f0 0 2024-02-19T19:04:41.603 INFO:tasks.workunit.client.1.smithi132.stdout:9/31: dwrite f1 [0,4194304] 0 2024-02-19T19:04:41.604 INFO:tasks.workunit.client.1.smithi132.stdout:5/27: dread f0 [0,4194304] 0 2024-02-19T19:04:41.619 INFO:tasks.workunit.client.1.smithi132.stdout:7/39: creat f11 x:0 0 0 2024-02-19T19:04:41.619 INFO:tasks.workunit.client.1.smithi132.stdout:2/44: link d0/f3 d0/d4/d11/f15 0 2024-02-19T19:04:41.626 INFO:tasks.workunit.client.1.smithi132.stdout:5/28: unlink l2 0 2024-02-19T19:04:41.633 INFO:tasks.workunit.client.1.smithi132.stdout:2/45: unlink d0/f3 0 2024-02-19T19:04:41.633 INFO:tasks.workunit.client.1.smithi132.stdout:2/46: readlink d0/d4/l7 0 2024-02-19T19:04:41.636 INFO:tasks.workunit.client.1.smithi132.stdout:9/32: unlink f4 0 2024-02-19T19:04:41.639 INFO:tasks.workunit.client.1.smithi132.stdout:5/29: creat f5 x:0 0 0 2024-02-19T19:04:41.648 INFO:tasks.workunit.client.1.smithi132.stdout:5/30: creat f6 x:0 0 0 2024-02-19T19:04:41.648 INFO:tasks.workunit.client.1.smithi132.stdout:8/31: getdents d4 0 2024-02-19T19:04:41.651 INFO:tasks.workunit.client.1.smithi132.stdout:7/40: symlink l12 0 2024-02-19T19:04:41.657 INFO:tasks.workunit.client.1.smithi132.stdout:2/47: symlink d0/d4/df/l16 0 2024-02-19T19:04:41.665 INFO:tasks.workunit.client.1.smithi132.stdout:5/31: symlink l7 0 2024-02-19T19:04:41.672 INFO:tasks.workunit.client.1.smithi132.stdout:8/32: truncate f2 3946253 0 2024-02-19T19:04:41.673 INFO:tasks.workunit.client.1.smithi132.stdout:5/32: mkdir d8 0 2024-02-19T19:04:41.674 INFO:tasks.workunit.client.1.smithi132.stdout:5/33: write f5 [23489,3302] 0 2024-02-19T19:04:41.674 INFO:tasks.workunit.client.1.smithi132.stdout:9/33: write fb [2473719,47582] 0 2024-02-19T19:04:41.676 INFO:tasks.workunit.client.1.smithi132.stdout:7/41: symlink l13 0 2024-02-19T19:04:41.685 INFO:tasks.workunit.client.1.smithi132.stdout:5/34: mknod d8/c9 0 2024-02-19T19:04:41.685 INFO:tasks.workunit.client.1.smithi132.stdout:9/34: dwrite fa [0,4194304] 0 2024-02-19T19:04:41.685 INFO:tasks.workunit.client.1.smithi132.stdout:7/42: dread f8 [0,4194304] 0 2024-02-19T19:04:41.694 INFO:tasks.workunit.client.1.smithi132.stdout:5/35: dwrite f6 [0,4194304] 0 2024-02-19T19:04:41.694 INFO:tasks.workunit.client.1.smithi132.stdout:7/43: dwrite f11 [0,4194304] 0 2024-02-19T19:04:41.701 INFO:tasks.workunit.client.1.smithi132.stdout:5/36: mkdir d8/da 0 2024-02-19T19:04:41.709 INFO:tasks.workunit.client.1.smithi132.stdout:9/35: creat fd x:0 0 0 2024-02-19T19:04:41.709 INFO:tasks.workunit.client.1.smithi132.stdout:9/36: write fa [659896,36606] 0 2024-02-19T19:04:41.715 INFO:tasks.workunit.client.1.smithi132.stdout:7/44: getdents . 0 2024-02-19T19:04:41.720 INFO:tasks.workunit.client.1.smithi132.stdout:8/33: truncate d4/f8 4041916 0 2024-02-19T19:04:41.723 INFO:tasks.workunit.client.1.smithi132.stdout:8/34: chown d4/f6 12895541 1 2024-02-19T19:04:41.723 INFO:tasks.workunit.client.1.smithi132.stdout:7/45: dwrite f6 [0,4194304] 0 2024-02-19T19:04:41.723 INFO:tasks.workunit.client.1.smithi132.stdout:5/37: write f0 [1730804,86309] 0 2024-02-19T19:04:41.724 INFO:tasks.workunit.client.1.smithi132.stdout:5/38: chown f5 41741 1 2024-02-19T19:04:41.724 INFO:tasks.workunit.client.1.smithi132.stdout:8/35: symlink d4/la 0 2024-02-19T19:04:41.726 INFO:tasks.workunit.client.1.smithi132.stdout:9/37: mknod ce 0 2024-02-19T19:04:41.728 INFO:tasks.workunit.client.1.smithi132.stdout:7/46: mknod c14 0 2024-02-19T19:04:41.730 INFO:tasks.workunit.client.1.smithi132.stdout:8/36: creat d4/fb x:0 0 0 2024-02-19T19:04:41.732 INFO:tasks.workunit.client.1.smithi132.stdout:5/39: mkdir d8/da/db 0 2024-02-19T19:04:41.734 INFO:tasks.workunit.client.1.smithi132.stdout:5/40: read f5 [16859,77368] 0 2024-02-19T19:04:41.734 INFO:tasks.workunit.client.1.smithi132.stdout:8/37: mkdir d4/dc 0 2024-02-19T19:04:41.734 INFO:tasks.workunit.client.1.smithi132.stdout:5/41: read f6 [95683,45990] 0 2024-02-19T19:04:41.735 INFO:tasks.workunit.client.1.smithi132.stdout:5/42: stat d8/da/db 0 2024-02-19T19:04:41.735 INFO:tasks.workunit.client.1.smithi132.stdout:5/43: chown d8/c9 118898676 1 2024-02-19T19:04:41.735 INFO:tasks.workunit.client.1.smithi132.stdout:5/44: write f5 [146175,6903] 0 2024-02-19T19:04:41.739 INFO:tasks.workunit.client.1.smithi132.stdout:8/38: creat d4/fd x:0 0 0 2024-02-19T19:04:41.745 INFO:tasks.workunit.client.1.smithi132.stdout:9/38: rename lc to lf 0 2024-02-19T19:04:41.747 INFO:tasks.workunit.client.1.smithi132.stdout:8/39: write d4/f6 [921011,6841] 0 2024-02-19T19:04:41.748 INFO:tasks.workunit.client.1.smithi132.stdout:8/40: dread - d4/fb zero size 2024-02-19T19:04:41.748 INFO:tasks.workunit.client.1.smithi132.stdout:9/39: truncate f2 4367994 0 2024-02-19T19:04:41.750 INFO:tasks.workunit.client.1.smithi132.stdout:5/45: getdents d8 0 2024-02-19T19:04:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:41 smithi132 ceph-mon[37969]: from='client.14858 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:04:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:41 smithi132 ceph-mon[37969]: pgmap v97: 65 pgs: 65 active+clean; 107 MiB data, 546 MiB used, 536 GiB / 536 GiB avail; 1.6 MiB/s wr, 72 op/s 2024-02-19T19:04:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' 2024-02-19T19:04:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:41 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/4251228594' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:04:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T19:04:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T19:04:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:41 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:04:41.752 INFO:tasks.workunit.client.1.smithi132.stdout:7/47: rename f11 to f15 0 2024-02-19T19:04:41.753 INFO:tasks.workunit.client.1.smithi132.stdout:8/41: mknod d4/ce 0 2024-02-19T19:04:41.754 INFO:tasks.workunit.client.1.smithi132.stdout:8/42: dread - d4/fb zero size 2024-02-19T19:04:41.755 INFO:tasks.workunit.client.1.smithi132.stdout:9/40: symlink l10 0 2024-02-19T19:04:41.756 INFO:tasks.workunit.client.1.smithi132.stdout:5/46: symlink d8/da/lc 0 2024-02-19T19:04:41.756 INFO:tasks.workunit.client.1.smithi132.stdout:7/48: chown ce 154361352 1 2024-02-19T19:04:41.759 INFO:tasks.workunit.client.1.smithi132.stdout:8/43: fsync f2 0 2024-02-19T19:04:41.762 INFO:tasks.workunit.client.1.smithi132.stdout:5/47: getdents d8 0 2024-02-19T19:04:41.763 INFO:tasks.workunit.client.1.smithi132.stdout:9/41: dwrite fb [0,4194304] 0 2024-02-19T19:04:41.766 INFO:tasks.workunit.client.1.smithi132.stdout:5/48: symlink d8/da/ld 0 2024-02-19T19:04:41.766 INFO:tasks.workunit.client.1.smithi132.stdout:8/44: link d4/ce d4/cf 0 2024-02-19T19:04:41.767 INFO:tasks.workunit.client.1.smithi132.stdout:7/49: link la l16 0 2024-02-19T19:04:41.769 INFO:tasks.workunit.client.1.smithi132.stdout:5/49: creat d8/fe x:0 0 0 2024-02-19T19:04:41.771 INFO:tasks.workunit.client.1.smithi132.stdout:8/45: write d4/f8 [785081,32712] 0 2024-02-19T19:04:41.771 INFO:tasks.workunit.client.1.smithi132.stdout:7/50: readlink la 0 2024-02-19T19:04:41.772 INFO:tasks.workunit.client.1.smithi132.stdout:5/50: unlink f0 0 2024-02-19T19:04:41.774 INFO:tasks.workunit.client.1.smithi132.stdout:9/42: creat f11 x:0 0 0 2024-02-19T19:04:41.774 INFO:tasks.workunit.client.1.smithi132.stdout:5/51: creat d8/ff x:0 0 0 2024-02-19T19:04:41.775 INFO:tasks.workunit.client.1.smithi132.stdout:5/52: dread - d8/ff zero size 2024-02-19T19:04:41.775 INFO:tasks.workunit.client.1.smithi132.stdout:8/46: mknod d4/dc/c10 0 2024-02-19T19:04:41.778 INFO:tasks.workunit.client.1.smithi132.stdout:5/53: dread f5 [0,4194304] 0 2024-02-19T19:04:41.779 INFO:tasks.workunit.client.1.smithi132.stdout:9/43: mkdir d12 0 2024-02-19T19:04:41.780 INFO:tasks.workunit.client.1.smithi132.stdout:7/51: link ce c17 0 2024-02-19T19:04:41.780 INFO:tasks.workunit.client.1.smithi132.stdout:9/44: truncate f2 4759159 0 2024-02-19T19:04:41.781 INFO:tasks.workunit.client.1.smithi132.stdout:9/45: fsync fb 0 2024-02-19T19:04:41.783 INFO:tasks.workunit.client.1.smithi132.stdout:8/47: dwrite d4/fd [0,4194304] 0 2024-02-19T19:04:41.785 INFO:tasks.workunit.client.1.smithi132.stdout:5/54: rename l3 to d8/da/l10 0 2024-02-19T19:04:41.796 INFO:tasks.workunit.client.1.smithi132.stdout:9/46: mkdir d12/d13 0 2024-02-19T19:04:41.796 INFO:tasks.workunit.client.1.smithi132.stdout:9/47: chown fd 64629 1 2024-02-19T19:04:41.799 INFO:tasks.workunit.client.1.smithi132.stdout:8/48: link d4/ce d4/c11 0 2024-02-19T19:04:41.800 INFO:tasks.workunit.client.1.smithi132.stdout:9/48: symlink d12/d13/l14 0 2024-02-19T19:04:41.803 INFO:tasks.workunit.client.1.smithi132.stdout:5/55: dwrite f6 [0,4194304] 0 2024-02-19T19:04:41.806 INFO:tasks.workunit.client.1.smithi132.stdout:8/49: getdents d4 0 2024-02-19T19:04:41.808 INFO:tasks.workunit.client.1.smithi132.stdout:9/49: dwrite f11 [0,4194304] 0 2024-02-19T19:04:41.810 INFO:tasks.workunit.client.1.smithi132.stdout:8/50: symlink d4/l12 0 2024-02-19T19:04:41.811 INFO:tasks.workunit.client.1.smithi132.stdout:8/51: chown d4/fb 395 1 2024-02-19T19:04:41.812 INFO:tasks.workunit.client.1.smithi132.stdout:8/52: dread - d4/fb zero size 2024-02-19T19:04:41.814 INFO:tasks.workunit.client.1.smithi132.stdout:8/53: mknod d4/dc/c13 0 2024-02-19T19:04:41.821 INFO:tasks.workunit.client.1.smithi132.stdout:8/54: dwrite d4/fb [0,4194304] 0 2024-02-19T19:04:41.826 INFO:tasks.workunit.client.1.smithi132.stdout:8/55: dread d4/fd [0,4194304] 0 2024-02-19T19:04:41.827 INFO:tasks.workunit.client.1.smithi132.stdout:8/56: write d4/f6 [642357,128372] 0 2024-02-19T19:04:41.833 INFO:tasks.workunit.client.1.smithi132.stdout:9/50: write f5 [1005504,50012] 0 2024-02-19T19:04:41.834 INFO:tasks.workunit.client.1.smithi132.stdout:8/57: link d4/la d4/l14 0 2024-02-19T19:04:41.835 INFO:tasks.workunit.client.1.smithi132.stdout:8/58: write f2 [151269,44456] 0 2024-02-19T19:04:41.836 INFO:tasks.workunit.client.1.smithi132.stdout:9/51: mknod d12/d13/c15 0 2024-02-19T19:04:41.837 INFO:tasks.workunit.client.1.smithi132.stdout:9/52: read f11 [234783,105181] 0 2024-02-19T19:04:41.839 INFO:tasks.workunit.client.1.smithi132.stdout:8/59: mknod d4/c15 0 2024-02-19T19:04:41.842 INFO:tasks.workunit.client.1.smithi132.stdout:8/60: dread f2 [0,4194304] 0 2024-02-19T19:04:41.847 INFO:tasks.workunit.client.1.smithi132.stdout:8/61: creat d4/f16 x:0 0 0 2024-02-19T19:04:41.847 INFO:tasks.workunit.client.1.smithi132.stdout:8/62: stat d4/fd 0 2024-02-19T19:04:41.847 INFO:tasks.workunit.client.1.smithi132.stdout:9/53: getdents d12/d13 0 2024-02-19T19:04:41.850 INFO:tasks.workunit.client.1.smithi132.stdout:9/54: creat d12/d13/f16 x:0 0 0 2024-02-19T19:04:41.850 INFO:tasks.workunit.client.1.smithi132.stdout:9/55: readlink l10 0 2024-02-19T19:04:41.853 INFO:tasks.workunit.client.1.smithi132.stdout:8/63: dwrite d4/fd [0,4194304] 0 2024-02-19T19:04:41.855 INFO:tasks.workunit.client.1.smithi132.stdout:9/56: rename f1 to d12/d13/f17 0 2024-02-19T19:04:41.856 INFO:tasks.workunit.client.1.smithi132.stdout:9/57: chown d12/d13/l14 402712 1 2024-02-19T19:04:41.860 INFO:tasks.workunit.client.1.smithi132.stdout:8/64: dwrite d4/fd [0,4194304] 0 2024-02-19T19:04:41.864 INFO:tasks.workunit.client.1.smithi132.stdout:9/58: dwrite f2 [4194304,4194304] 0 2024-02-19T19:04:41.866 INFO:tasks.workunit.client.1.smithi132.stdout:8/65: mknod d4/c17 0 2024-02-19T19:04:41.868 INFO:tasks.workunit.client.1.smithi132.stdout:9/59: read f5 [509929,29929] 0 2024-02-19T19:04:41.874 INFO:tasks.workunit.client.1.smithi132.stdout:9/60: mknod d12/d13/c18 0 2024-02-19T19:04:41.875 INFO:tasks.workunit.client.1.smithi132.stdout:9/61: chown c0 939 1 2024-02-19T19:04:41.878 INFO:tasks.workunit.client.1.smithi132.stdout:9/62: unlink f2 0 2024-02-19T19:04:41.878 INFO:tasks.workunit.client.1.smithi132.stdout:9/63: stat c0 0 2024-02-19T19:04:41.880 INFO:tasks.workunit.client.1.smithi132.stdout:9/64: creat d12/f19 x:0 0 0 2024-02-19T19:04:41.882 INFO:tasks.workunit.client.1.smithi132.stdout:9/65: creat d12/f1a x:0 0 0 2024-02-19T19:04:41.889 INFO:tasks.workunit.client.1.smithi132.stdout:9/66: rename d12/d13 to d12/d1b 0 2024-02-19T19:04:41.889 INFO:tasks.workunit.client.1.smithi132.stdout:9/67: stat fa 0 2024-02-19T19:04:41.892 INFO:tasks.workunit.client.1.smithi132.stdout:9/68: dread f11 [0,4194304] 0 2024-02-19T19:04:41.893 INFO:tasks.workunit.client.1.smithi132.stdout:9/69: chown c9 3169 1 2024-02-19T19:04:41.895 INFO:tasks.workunit.client.1.smithi132.stdout:9/70: truncate f11 801268 0 2024-02-19T19:04:42.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:42 smithi132 ceph-mon[37969]: Upgrade: Updating mon.smithi088 2024-02-19T19:04:42.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:42 smithi132 ceph-mon[37969]: Deploying daemon mon.smithi088 on smithi088 2024-02-19T19:04:42.823 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:42 smithi088 ceph-mon[29272]: Upgrade: Updating mon.smithi088 2024-02-19T19:04:42.823 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:42 smithi088 ceph-mon[29272]: Deploying daemon mon.smithi088 on smithi088 2024-02-19T19:04:43.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:42 smithi088 systemd[1]: Stopping Ceph mon.smithi088 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:04:43.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:43 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[29268]: 2024-02-19T19:04:43.106+0000 7f452504a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi088 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T19:04:43.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:43 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[29268]: 2024-02-19T19:04:43.106+0000 7f452504a700 -1 mon.smithi088@0(leader) e2 *** Got Signal Terminated *** 2024-02-19T19:04:43.436 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:43 smithi088 podman[88486]: 2024-02-19 19:04:43.140093737 +0000 UTC m=+0.152519423 container died 3ea5108455bc6bd4da387b645ceea6ca82bc1ef71f39070e02e79bb88b5f3baf (image=quay.io/ceph/ceph:v18.2.0, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0) 2024-02-19T19:04:43.810 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:43 smithi088 podman[88486]: 2024-02-19 19:04:43.514995231 +0000 UTC m=+0.527420942 container cleanup 3ea5108455bc6bd4da387b645ceea6ca82bc1ef71f39070e02e79bb88b5f3baf (image=quay.io/ceph/ceph:v18.2.0, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-19T19:04:43.810 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:43 smithi088 bash[88486]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088 2024-02-19T19:04:44.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:43 smithi088 podman[88509]: 2024-02-19 19:04:43.791565209 +0000 UTC m=+0.648564537 container remove 3ea5108455bc6bd4da387b645ceea6ca82bc1ef71f39070e02e79bb88b5f3baf (image=quay.io/ceph/ceph:v18.2.0, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.build-date=20231212, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-19T19:04:44.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:43 smithi088 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi088.service: Deactivated successfully. 2024-02-19T19:04:44.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:43 smithi088 systemd[1]: Stopped Ceph mon.smithi088 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:04:44.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:43 smithi088 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi088.service: Consumed 13.666s CPU time. 2024-02-19T19:04:44.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 systemd[1]: Starting Ceph mon.smithi088 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:04:44.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 podman[88625]: 2024-02-19 19:04:44.417509676 +0000 UTC m=+0.031259731 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:04:44.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 podman[88625]: 2024-02-19 19:04:44.54308445 +0000 UTC m=+0.156834532 container create 8ff9c135b06187d3315d135fec91816e7d828be1c3326afb0f0deeae07081108 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T19:04:45.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 podman[88625]: 2024-02-19 19:04:44.804357671 +0000 UTC m=+0.418107713 container init 8ff9c135b06187d3315d135fec91816e7d828be1c3326afb0f0deeae07081108 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 podman[88625]: 2024-02-19 19:04:44.809172422 +0000 UTC m=+0.422922467 container start 8ff9c135b06187d3315d135fec91816e7d828be1c3326afb0f0deeae07081108 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: set uid:gid to 167:167 (ceph:ceph) 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev), process ceph-mon, pid 2 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: pidfile_write: ignore empty --pid-file 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: load: jerasure load: lrc 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: RocksDB version: 7.9.2 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Git sha 0 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Compile date 2024-02-15 18:05:23 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: DB SUMMARY 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: DB Session ID: ZM8SFBMJAD2ODLYCVHE6 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: CURRENT file: CURRENT 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: IDENTITY file: IDENTITY 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1509 Bytes 2024-02-19T19:04:45.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi088/store.db dir, Total Num: 1, files: 000032.sst 2024-02-19T19:04:45.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi088/store.db: 000030.log size: 4653228 ; 2024-02-19T19:04:45.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.error_if_exists: 0 2024-02-19T19:04:45.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.create_if_missing: 0 2024-02-19T19:04:45.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.paranoid_checks: 1 2024-02-19T19:04:45.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-19T19:04:45.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-19T19:04:45.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-19T19:04:45.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.env: 0x5613d5502ae0 2024-02-19T19:04:45.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.fs: PosixFileSystem 2024-02-19T19:04:45.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.info_log: 0x5613d6b7f760 2024-02-19T19:04:45.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_file_opening_threads: 16 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.statistics: (nil) 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.use_fsync: 0 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_log_file_size: 0 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.keep_log_file_num: 1000 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.recycle_log_file_num: 0 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.allow_fallocate: 1 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.allow_mmap_reads: 0 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.allow_mmap_writes: 0 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.use_direct_reads: 0 2024-02-19T19:04:45.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.create_missing_column_families: 0 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.db_log_dir: 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.wal_dir: 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.advise_random_on_open: 1 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.db_write_buffer_size: 0 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.write_buffer_manager: 0x5613d6b53400 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-19T19:04:45.104 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.rate_limiter: (nil) 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.wal_recovery_mode: 2 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.enable_thread_tracking: 0 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.enable_pipelined_write: 0 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.unordered_write: 0 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.row_cache: None 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.wal_filter: None 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.allow_ingest_behind: 0 2024-02-19T19:04:45.105 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.two_write_queues: 0 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.manual_wal_flush: 0 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.wal_compression: 0 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.atomic_flush: 0 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.log_readahead_size: 0 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.best_efforts_recovery: 0 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.allow_data_in_errors: 0 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.db_host_id: __hostname__ 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_background_jobs: 2 2024-02-19T19:04:45.106 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_background_compactions: -1 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_subcompactions: 1 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_total_wal_size: 0 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_open_files: -1 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bytes_per_sync: 0 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_readahead_size: 0 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_background_flushes: -1 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Compression algorithms supported: 2024-02-19T19:04:45.107 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: kZSTD supported: 0 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: kXpressCompression supported: 0 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: kLZ4HCCompression supported: 1 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: kZlibCompression supported: 1 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: kSnappyCompression supported: 1 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: kLZ4Compression supported: 1 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: kBZip2Compression supported: 0 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi088/store.db/MANIFEST-000015 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.merge_operator: 2024-02-19T19:04:45.108 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_filter: None 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_filter_factory: None 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.sst_partitioner_factory: None 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5613d6b7f880) 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: cache_index_and_filter_blocks: 1 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: pin_top_level_index_and_filter: 1 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_type: 0 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: data_block_index_type: 0 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_shortening: 1 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: checksum: 4 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: no_block_cache: 0 2024-02-19T19:04:45.109 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache: 0x5613d6b83090 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_name: BinnedLRUCache 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_options: 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: capacity : 536870912 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: num_shard_bits : 4 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: strict_capacity_limit : 0 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: high_pri_pool_ratio: 0.000 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_cache_compressed: (nil) 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: persistent_cache: (nil) 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_size: 4096 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_size_deviation: 10 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_restart_interval: 16 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: index_block_restart_interval: 1 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: metadata_block_size: 4096 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: partition_filters: 0 2024-02-19T19:04:45.110 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: use_delta_encoding: 1 2024-02-19T19:04:45.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: filter_policy: bloomfilter 2024-02-19T19:04:45.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: whole_key_filtering: 1 2024-02-19T19:04:45.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: verify_compression: 0 2024-02-19T19:04:45.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: read_amp_bytes_per_bit: 0 2024-02-19T19:04:45.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: format_version: 5 2024-02-19T19:04:45.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: enable_index_compression: 1 2024-02-19T19:04:45.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: block_align: 0 2024-02-19T19:04:45.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: max_auto_readahead_size: 262144 2024-02-19T19:04:45.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: prepopulate_block_cache: 0 2024-02-19T19:04:45.111 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: initial_auto_readahead_size: 8192 2024-02-19T19:04:45.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: num_file_reads_for_auto_readahead: 2 2024-02-19T19:04:45.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.write_buffer_size: 33554432 2024-02-19T19:04:45.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_write_buffer_number: 2 2024-02-19T19:04:45.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compression: NoCompression 2024-02-19T19:04:45.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bottommost_compression: Disabled 2024-02-19T19:04:45.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.prefix_extractor: nullptr 2024-02-19T19:04:45.112 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.num_levels: 7 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compression_opts.level: 32767 2024-02-19T19:04:45.113 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compression_opts.strategy: 0 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compression_opts.enabled: false 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.target_file_size_base: 67108864 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-19T19:04:45.114 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.arena_block_size: 1048576 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.disable_auto_compactions: 0 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-19T19:04:45.115 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.table_properties_collectors: 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.inplace_update_support: 0 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.bloom_locality: 0 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.max_successive_merges: 0 2024-02-19T19:04:45.116 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.paranoid_file_checks: 0 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.force_consistency_checks: 1 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.report_bg_io_stats: 0 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.ttl: 2592000 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.enable_blob_files: false 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.min_blob_size: 0 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.blob_file_size: 268435456 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-19T19:04:45.117 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.blob_file_starting_level: 0 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi088/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12013, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ba0b5dbd-2286-4f7c-8c91-1f2b765707c4 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708369484853441, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708369484874802, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 4112094, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12014, "largest_seqno": 15386, "table_properties": {"data_size": 4099036, "index_size": 8197, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3845, "raw_key_size": 33282, "raw_average_key_size": 22, "raw_value_size": 4069327, "raw_average_value_size": 2691, "num_data_blocks": 371, "num_entries": 1512, "num_filter_entries": 1512, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708369484, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ba0b5dbd-2286-4f7c-8c91-1f2b765707c4", "db_session_id": "ZM8SFBMJAD2ODLYCVHE6", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708369484874918, "job": 1, "event": "recovery_finished"} 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi088/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5613d6c72000 2024-02-19T19:04:45.118 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: DB pointer 0x5613d6c5e000 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** DB Stats ** 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** Compaction Stats [default] ** 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: L0 1/0 3.92 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 398.4 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-02-19T19:04:45.119 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: L6 1/0 5.09 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Sum 2/0 9.01 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 398.4 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 398.4 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** Compaction Stats [default] ** 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 398.4 0.01 0.00 1 0.010 0 0 0.0 0.0 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-02-19T19:04:45.120 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Cumulative compaction: 0.00 GB write, 156.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Interval compaction: 0.00 GB write, 156.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Block cache BinnedLRUCache@0x5613d6b83090#2 capacity: 512.00 MB usage: 13.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,9.08 KB,0.00173151%) Misc(1,0.00 KB,0%) 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: starting mon.smithi088 rank 0 at public addrs [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] at bind addrs [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi088 fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: mon.smithi088@-1(???) e2 preinit fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: mon.smithi088@-1(???).mds e19 new map 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: mon.smithi088@-1(???).mds e19 print_map 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: e19 2024-02-19T19:04:45.121 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: legacy client fscid: 1 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Filesystem 'cephfs' (1) 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: fs_name cephfs 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: epoch 19 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: modified 2024-02-19T19:04:10.337219+0000 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: tableserver 0 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: root 0 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: session_timeout 60 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: session_autoclose 300 2024-02-19T19:04:45.122 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: max_file_size 1099511627776 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: max_xattr_size 65536 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: required_client_features {} 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: last_failure 0 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: last_failure_osd_epoch 46 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: max_mds 2 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: in 0,1 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: up {0=14510,1=24263} 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: failed 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: damaged 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: stopped 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: data_pools [3] 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: metadata_pool 2 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: inline_data disabled 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: balancer 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: bal_rank_mask -1 2024-02-19T19:04:45.123 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: standby_count_wanted 1 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: [mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: [mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: Standby daemons: 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: [mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout: [mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: mon.smithi088@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: mon.smithi088@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: mon.smithi088@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: mon.smithi088@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 ceph-mon[88643]: mon.smithi088@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 bash[88625]: 8ff9c135b06187d3315d135fec91816e7d828be1c3326afb0f0deeae07081108 2024-02-19T19:04:45.124 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:44 smithi088 systemd[1]: Started Ceph mon.smithi088 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:04:46.176 INFO:tasks.workunit.client.1.smithi132.stdout:9/71: sync 2024-02-19T19:04:46.176 INFO:tasks.workunit.client.1.smithi132.stdout:2/48: sync 2024-02-19T19:04:46.176 INFO:tasks.workunit.client.1.smithi132.stdout:4/5: sync 2024-02-19T19:04:46.176 INFO:tasks.workunit.client.1.smithi132.stdout:6/10: sync 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:4/6: rmdir - no directory 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:4/7: write - no filename 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:4/8: read - no filename 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:4/9: write - no filename 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:4/10: dread - no filename 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:8/66: sync 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:5/56: sync 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:7/52: sync 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:1/7: sync 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:0/9: sync 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:3/18: sync 2024-02-19T19:04:46.177 INFO:tasks.workunit.client.1.smithi132.stdout:9/72: chown fb 21314 1 2024-02-19T19:04:46.178 INFO:tasks.workunit.client.1.smithi132.stdout:9/73: dread - fd zero size 2024-02-19T19:04:46.178 INFO:tasks.workunit.client.1.smithi132.stdout:0/10: read f0 [1034908,9678] 0 2024-02-19T19:04:46.179 INFO:tasks.workunit.client.1.smithi132.stdout:3/19: symlink l5 0 2024-02-19T19:04:46.184 INFO:tasks.workunit.client.1.smithi132.stdout:1/8: creat d1/f3 x:0 0 0 2024-02-19T19:04:46.185 INFO:tasks.workunit.client.1.smithi132.stdout:4/11: creat f0 x:0 0 0 2024-02-19T19:04:46.186 INFO:tasks.workunit.client.1.smithi132.stdout:6/11: unlink d0/d1/c2 0 2024-02-19T19:04:46.187 INFO:tasks.workunit.client.1.smithi132.stdout:0/11: dwrite f0 [0,4194304] 0 2024-02-19T19:04:46.188 INFO:tasks.workunit.client.1.smithi132.stdout:0/12: chown f0 1146 1 2024-02-19T19:04:46.189 INFO:tasks.workunit.client.1.smithi132.stdout:9/74: creat d12/d1b/f1c x:0 0 0 2024-02-19T19:04:46.190 INFO:tasks.workunit.client.1.smithi132.stdout:2/49: rmdir d0/d4/d11 39 2024-02-19T19:04:46.190 INFO:tasks.workunit.client.1.smithi132.stdout:8/67: dwrite d4/f8 [0,4194304] 0 2024-02-19T19:04:46.194 INFO:tasks.workunit.client.1.smithi132.stdout:1/9: creat d1/f4 x:0 0 0 2024-02-19T19:04:46.196 INFO:tasks.workunit.client.1.smithi132.stdout:4/12: rename f0 to f1 0 2024-02-19T19:04:46.197 INFO:tasks.workunit.client.1.smithi132.stdout:6/12: mkdir d0/d1/d4 0 2024-02-19T19:04:46.198 INFO:tasks.workunit.client.1.smithi132.stdout:6/13: write - no filename 2024-02-19T19:04:46.199 INFO:tasks.workunit.client.1.smithi132.stdout:1/10: dread d1/f2 [0,4194304] 0 2024-02-19T19:04:46.200 INFO:tasks.workunit.client.1.smithi132.stdout:9/75: write f5 [224347,36870] 0 2024-02-19T19:04:46.203 INFO:tasks.workunit.client.1.smithi132.stdout:2/50: mknod d0/d4/df/c17 0 2024-02-19T19:04:46.203 INFO:tasks.workunit.client.1.smithi132.stdout:8/68: mkdir d4/dc/d18 0 2024-02-19T19:04:46.204 INFO:tasks.workunit.client.1.smithi132.stdout:2/51: truncate d0/d4/f14 142940 0 2024-02-19T19:04:46.205 INFO:tasks.workunit.client.1.smithi132.stdout:7/53: rename l4 to l18 0 2024-02-19T19:04:46.213 INFO:tasks.workunit.client.1.smithi132.stdout:1/11: rename d1/f4 to d1/f5 0 2024-02-19T19:04:46.213 INFO:tasks.workunit.client.1.smithi132.stdout:8/69: creat d4/f19 x:0 0 0 2024-02-19T19:04:46.215 INFO:tasks.workunit.client.1.smithi132.stdout:1/12: truncate d1/f5 418750 0 2024-02-19T19:04:46.215 INFO:tasks.workunit.client.1.smithi132.stdout:8/70: truncate d4/f16 368180 0 2024-02-19T19:04:46.215 INFO:tasks.workunit.client.1.smithi132.stdout:7/54: write f6 [1338772,87285] 0 2024-02-19T19:04:46.215 INFO:tasks.workunit.client.1.smithi132.stdout:0/13: fdatasync f0 0 2024-02-19T19:04:46.218 INFO:tasks.workunit.client.1.smithi132.stdout:9/76: dread f5 [0,4194304] 0 2024-02-19T19:04:46.218 INFO:tasks.workunit.client.1.smithi132.stdout:9/77: write d12/f1a [242996,67289] 0 2024-02-19T19:04:46.220 INFO:tasks.workunit.client.1.smithi132.stdout:6/14: rmdir d0/d1/d4 0 2024-02-19T19:04:46.221 INFO:tasks.workunit.client.1.smithi132.stdout:7/55: chown l12 418349 1 2024-02-19T19:04:46.225 INFO:tasks.workunit.client.1.smithi132.stdout:2/52: link d0/c2 d0/c18 0 2024-02-19T19:04:46.227 INFO:tasks.workunit.client.1.smithi132.stdout:9/78: mknod d12/c1d 0 2024-02-19T19:04:46.229 INFO:tasks.workunit.client.1.smithi132.stdout:6/15: creat d0/f5 x:0 0 0 2024-02-19T19:04:46.315 INFO:tasks.workunit.client.1.smithi132.stdout:2/53: mkdir d0/d19 0 2024-02-19T19:04:46.315 INFO:tasks.workunit.client.1.smithi132.stdout:9/79: symlink d12/d1b/l1e 0 2024-02-19T19:04:46.315 INFO:tasks.workunit.client.1.smithi132.stdout:6/16: rename d0/d1 to d0/d1/d6 22 2024-02-19T19:04:46.316 INFO:tasks.workunit.client.1.smithi132.stdout:6/17: read - d0/f5 zero size 2024-02-19T19:04:46.317 INFO:tasks.workunit.client.1.smithi132.stdout:2/54: write d0/d4/fb [1533977,11816] 0 2024-02-19T19:04:46.317 INFO:tasks.workunit.client.1.smithi132.stdout:9/80: write d12/d1b/f16 [550422,50715] 0 2024-02-19T19:04:46.318 INFO:tasks.workunit.client.1.smithi132.stdout:2/55: stat d0/f5 0 2024-02-19T19:04:46.325 INFO:tasks.workunit.client.1.smithi132.stdout:9/81: dwrite fd [0,4194304] 0 2024-02-19T19:04:46.333 INFO:tasks.workunit.client.1.smithi132.stdout:3/20: getdents . 0 2024-02-19T19:04:46.334 INFO:tasks.workunit.client.1.smithi132.stdout:4/13: getdents . 0 2024-02-19T19:04:46.334 INFO:tasks.workunit.client.1.smithi132.stdout:3/21: write f0 [2559619,76028] 0 2024-02-19T19:04:46.334 INFO:tasks.workunit.client.1.smithi132.stdout:8/71: rmdir d4/dc/d18 0 2024-02-19T19:04:46.334 INFO:tasks.workunit.client.1.smithi132.stdout:6/18: symlink d0/l7 0 2024-02-19T19:04:46.335 INFO:tasks.workunit.client.1.smithi132.stdout:2/56: rename d0/d4/c9 to d0/d4/df/c1a 0 2024-02-19T19:04:46.337 INFO:tasks.workunit.client.1.smithi132.stdout:4/14: truncate f1 328159 0 2024-02-19T19:04:46.337 INFO:tasks.workunit.client.1.smithi132.stdout:9/82: write f5 [1429472,55355] 0 2024-02-19T19:04:46.338 INFO:tasks.workunit.client.1.smithi132.stdout:2/57: chown d0/d4/cc 737211 1 2024-02-19T19:04:46.340 INFO:tasks.workunit.client.1.smithi132.stdout:7/56: truncate f6 3643675 0 2024-02-19T19:04:46.341 INFO:tasks.workunit.client.1.smithi132.stdout:4/15: dread f1 [0,4194304] 0 2024-02-19T19:04:46.341 INFO:tasks.workunit.client.1.smithi132.stdout:8/72: mkdir d4/dc/d1a 0 2024-02-19T19:04:46.341 INFO:tasks.workunit.client.1.smithi132.stdout:5/57: sync 2024-02-19T19:04:46.343 INFO:tasks.workunit.client.1.smithi132.stdout:7/57: chown c14 2056325 1 2024-02-19T19:04:46.343 INFO:tasks.workunit.client.1.smithi132.stdout:6/19: creat d0/d1/f8 x:0 0 0 2024-02-19T19:04:46.345 INFO:tasks.workunit.client.1.smithi132.stdout:5/58: read - d8/ff zero size 2024-02-19T19:04:46.346 INFO:tasks.workunit.client.1.smithi132.stdout:7/58: read f15 [997396,77144] 0 2024-02-19T19:04:46.346 INFO:tasks.workunit.client.1.smithi132.stdout:9/83: rename d12/c1d to d12/c1f 0 2024-02-19T19:04:46.348 INFO:tasks.workunit.client.1.smithi132.stdout:4/16: fsync f1 0 2024-02-19T19:04:46.349 INFO:tasks.workunit.client.1.smithi132.stdout:3/22: dread f0 [0,4194304] 0 2024-02-19T19:04:46.350 INFO:tasks.workunit.client.1.smithi132.stdout:9/84: dread - d12/d1b/f1c zero size 2024-02-19T19:04:46.350 INFO:tasks.workunit.client.1.smithi132.stdout:0/14: dwrite f0 [4194304,4194304] 0 2024-02-19T19:04:46.350 INFO:tasks.workunit.client.1.smithi132.stdout:4/17: chown f1 3261743 1 2024-02-19T19:04:46.351 INFO:tasks.workunit.client.1.smithi132.stdout:4/18: fdatasync f1 0 2024-02-19T19:04:46.351 INFO:tasks.workunit.client.1.smithi132.stdout:4/19: readlink - no filename 2024-02-19T19:04:46.352 INFO:tasks.workunit.client.1.smithi132.stdout:8/73: dread d4/fb [0,4194304] 0 2024-02-19T19:04:46.352 INFO:tasks.workunit.client.1.smithi132.stdout:2/58: rename d0/d4/f14 to d0/d4/df/f1b 0 2024-02-19T19:04:46.357 INFO:tasks.workunit.client.1.smithi132.stdout:6/20: creat d0/d1/f9 x:0 0 0 2024-02-19T19:04:46.357 INFO:tasks.workunit.client.1.smithi132.stdout:6/21: readlink d0/l7 0 2024-02-19T19:04:46.358 INFO:tasks.workunit.client.1.smithi132.stdout:6/22: write d0/d1/f9 [292802,83644] 0 2024-02-19T19:04:46.363 INFO:tasks.workunit.client.1.smithi132.stdout:3/23: mknod c6 0 2024-02-19T19:04:46.365 INFO:tasks.workunit.client.1.smithi132.stdout:8/74: mkdir d4/dc/d1b 0 2024-02-19T19:04:46.367 INFO:tasks.workunit.client.1.smithi132.stdout:5/59: rename d8/da/lc to d8/da/db/l11 0 2024-02-19T19:04:46.368 INFO:tasks.workunit.client.1.smithi132.stdout:1/13: sync 2024-02-19T19:04:46.369 INFO:tasks.workunit.client.1.smithi132.stdout:5/60: write d8/fe [124768,47629] 0 2024-02-19T19:04:46.369 INFO:tasks.workunit.client.1.smithi132.stdout:5/61: chown d8 323750192 1 2024-02-19T19:04:46.372 INFO:tasks.workunit.client.1.smithi132.stdout:8/75: dwrite d4/f19 [0,4194304] 0 2024-02-19T19:04:46.373 INFO:tasks.workunit.client.1.smithi132.stdout:6/23: creat d0/fa x:0 0 0 2024-02-19T19:04:46.374 INFO:tasks.workunit.client.1.smithi132.stdout:8/76: read d4/fb [3833256,77215] 0 2024-02-19T19:04:46.374 INFO:tasks.workunit.client.1.smithi132.stdout:6/24: chown d0/d1/f9 7224 1 2024-02-19T19:04:46.375 INFO:tasks.workunit.client.1.smithi132.stdout:6/25: write d0/f5 [232685,79701] 0 2024-02-19T19:04:46.376 INFO:tasks.workunit.client.1.smithi132.stdout:3/24: mknod c7 0 2024-02-19T19:04:46.377 INFO:tasks.workunit.client.1.smithi132.stdout:3/25: write f4 [425211,47031] 0 2024-02-19T19:04:46.379 INFO:tasks.workunit.client.1.smithi132.stdout:3/26: write f3 [501892,98042] 0 2024-02-19T19:04:46.381 INFO:tasks.workunit.client.1.smithi132.stdout:4/20: link f1 f2 0 2024-02-19T19:04:46.385 INFO:tasks.workunit.client.1.smithi132.stdout:6/26: dwrite d0/fa [0,4194304] 0 2024-02-19T19:04:46.385 INFO:tasks.workunit.client.1.smithi132.stdout:5/62: write f5 [438441,105455] 0 2024-02-19T19:04:46.387 INFO:tasks.workunit.client.1.smithi132.stdout:5/63: write f6 [3779109,30494] 0 2024-02-19T19:04:46.390 INFO:tasks.workunit.client.1.smithi132.stdout:6/27: dread d0/f5 [0,4194304] 0 2024-02-19T19:04:46.395 INFO:tasks.workunit.client.1.smithi132.stdout:3/27: write f0 [1022457,85346] 0 2024-02-19T19:04:46.398 INFO:tasks.workunit.client.1.smithi132.stdout:3/28: dread f3 [0,4194304] 0 2024-02-19T19:04:46.399 INFO:tasks.workunit.client.1.smithi132.stdout:9/85: rename f11 to d12/d1b/f20 0 2024-02-19T19:04:46.400 INFO:tasks.workunit.client.1.smithi132.stdout:1/14: rename d1 to d1/d6 22 2024-02-19T19:04:46.400 INFO:tasks.workunit.client.1.smithi132.stdout:1/15: readlink - no filename 2024-02-19T19:04:46.400 INFO:tasks.workunit.client.1.smithi132.stdout:9/86: chown ce 7810 1 2024-02-19T19:04:46.403 INFO:tasks.workunit.client.1.smithi132.stdout:7/59: dread f6 [0,4194304] 0 2024-02-19T19:04:46.405 INFO:tasks.workunit.client.1.smithi132.stdout:8/77: unlink d4/c11 0 2024-02-19T19:04:46.406 INFO:tasks.workunit.client.1.smithi132.stdout:5/64: symlink d8/da/db/l12 0 2024-02-19T19:04:46.406 INFO:tasks.workunit.client.1.smithi132.stdout:0/15: getdents . 0 2024-02-19T19:04:46.407 INFO:tasks.workunit.client.1.smithi132.stdout:0/16: write f0 [6191565,44091] 0 2024-02-19T19:04:46.408 INFO:tasks.workunit.client.1.smithi132.stdout:6/28: mkdir d0/db 0 2024-02-19T19:04:46.411 INFO:tasks.workunit.client.1.smithi132.stdout:4/21: rename f2 to f3 0 2024-02-19T19:04:46.412 INFO:tasks.workunit.client.1.smithi132.stdout:3/29: mkdir d8 0 2024-02-19T19:04:46.413 INFO:tasks.workunit.client.1.smithi132.stdout:1/16: fdatasync d1/f2 0 2024-02-19T19:04:46.414 INFO:tasks.workunit.client.1.smithi132.stdout:5/65: dwrite d8/fe [0,4194304] 0 2024-02-19T19:04:46.414 INFO:tasks.workunit.client.1.smithi132.stdout:5/66: readlink d8/da/db/l12 0 2024-02-19T19:04:46.420 INFO:tasks.workunit.client.1.smithi132.stdout:8/78: creat d4/dc/f1c x:0 0 0 2024-02-19T19:04:46.429 INFO:tasks.workunit.client.1.smithi132.stdout:0/17: creat f1 x:0 0 0 2024-02-19T19:04:46.429 INFO:tasks.workunit.client.1.smithi132.stdout:6/29: mkdir d0/dc 0 2024-02-19T19:04:46.429 INFO:tasks.workunit.client.1.smithi132.stdout:4/22: mkdir d4 0 2024-02-19T19:04:46.430 INFO:tasks.workunit.client.1.smithi132.stdout:5/67: symlink d8/l13 0 2024-02-19T19:04:46.430 INFO:tasks.workunit.client.1.smithi132.stdout:0/18: dread - f1 zero size 2024-02-19T19:04:46.430 INFO:tasks.workunit.client.1.smithi132.stdout:0/19: readlink - no filename 2024-02-19T19:04:46.431 INFO:tasks.workunit.client.1.smithi132.stdout:6/30: write d0/fa [3889676,49106] 0 2024-02-19T19:04:46.431 INFO:tasks.workunit.client.1.smithi132.stdout:5/68: write d8/fe [3659190,15892] 0 2024-02-19T19:04:46.431 INFO:tasks.workunit.client.1.smithi132.stdout:2/59: rename d0/d4/df/f1b to d0/d4/d11/f1c 0 2024-02-19T19:04:46.432 INFO:tasks.workunit.client.1.smithi132.stdout:3/30: getdents d8 0 2024-02-19T19:04:46.434 INFO:tasks.workunit.client.1.smithi132.stdout:2/60: dread - d0/d4/f12 zero size 2024-02-19T19:04:46.434 INFO:tasks.workunit.client.1.smithi132.stdout:3/31: write f0 [1415424,10478] 0 2024-02-19T19:04:46.435 INFO:tasks.workunit.client.1.smithi132.stdout:8/79: mkdir d4/dc/d1b/d1d 0 2024-02-19T19:04:46.439 INFO:tasks.workunit.client.1.smithi132.stdout:8/80: read d4/f16 [44182,83688] 0 2024-02-19T19:04:46.439 INFO:tasks.workunit.client.1.smithi132.stdout:0/20: dread f0 [4194304,4194304] 0 2024-02-19T19:04:46.441 INFO:tasks.workunit.client.1.smithi132.stdout:0/21: dread - f1 zero size 2024-02-19T19:04:46.442 INFO:tasks.workunit.client.1.smithi132.stdout:5/69: unlink d8/l13 0 2024-02-19T19:04:46.443 INFO:tasks.workunit.client.1.smithi132.stdout:6/31: dwrite d0/d1/f9 [0,4194304] 0 2024-02-19T19:04:46.448 INFO:tasks.workunit.client.1.smithi132.stdout:8/81: dread d4/fb [0,4194304] 0 2024-02-19T19:04:46.449 INFO:tasks.workunit.client.1.smithi132.stdout:2/61: mknod d0/c1d 0 2024-02-19T19:04:46.451 INFO:tasks.workunit.client.1.smithi132.stdout:7/60: write ff [848834,103256] 0 2024-02-19T19:04:46.452 INFO:tasks.workunit.client.1.smithi132.stdout:0/22: dwrite f1 [0,4194304] 0 2024-02-19T19:04:46.453 INFO:tasks.workunit.client.1.smithi132.stdout:7/61: dread f8 [0,4194304] 0 2024-02-19T19:04:46.457 INFO:tasks.workunit.client.1.smithi132.stdout:9/87: rename c7 to d12/c21 0 2024-02-19T19:04:46.458 INFO:tasks.workunit.client.1.smithi132.stdout:5/70: mknod d8/c14 0 2024-02-19T19:04:46.460 INFO:tasks.workunit.client.1.smithi132.stdout:5/71: write d8/ff [493261,36823] 0 2024-02-19T19:04:46.460 INFO:tasks.workunit.client.1.smithi132.stdout:4/23: dread f3 [0,4194304] 0 2024-02-19T19:04:46.460 INFO:tasks.workunit.client.1.smithi132.stdout:9/88: read d12/f1a [239216,44717] 0 2024-02-19T19:04:46.460 INFO:tasks.workunit.client.1.smithi132.stdout:5/72: fdatasync d8/fe 0 2024-02-19T19:04:46.462 INFO:tasks.workunit.client.1.smithi132.stdout:3/32: creat d8/f9 x:0 0 0 2024-02-19T19:04:46.462 INFO:tasks.workunit.client.1.smithi132.stdout:4/24: dread f1 [0,4194304] 0 2024-02-19T19:04:46.463 INFO:tasks.workunit.client.1.smithi132.stdout:3/33: chown f0 235698031 1 2024-02-19T19:04:46.463 INFO:tasks.workunit.client.1.smithi132.stdout:0/23: mkdir d2 0 2024-02-19T19:04:46.465 INFO:tasks.workunit.client.1.smithi132.stdout:3/34: truncate d8/f9 939384 0 2024-02-19T19:04:46.467 INFO:tasks.workunit.client.1.smithi132.stdout:6/32: creat d0/db/fd x:0 0 0 2024-02-19T19:04:46.467 INFO:tasks.workunit.client.1.smithi132.stdout:2/62: mkdir d0/d19/d1e 0 2024-02-19T19:04:46.468 INFO:tasks.workunit.client.1.smithi132.stdout:6/33: chown d0/dc 3 1 2024-02-19T19:04:46.469 INFO:tasks.workunit.client.1.smithi132.stdout:2/63: stat d0/d4/c8 0 2024-02-19T19:04:46.469 INFO:tasks.workunit.client.1.smithi132.stdout:3/35: dread d8/f9 [0,4194304] 0 2024-02-19T19:04:46.470 INFO:tasks.workunit.client.1.smithi132.stdout:5/73: dwrite d8/fe [0,4194304] 0 2024-02-19T19:04:46.472 INFO:tasks.workunit.client.1.smithi132.stdout:8/82: link d4/f8 d4/dc/d1b/d1d/f1e 0 2024-02-19T19:04:46.473 INFO:tasks.workunit.client.1.smithi132.stdout:0/24: write f0 [7848385,16693] 0 2024-02-19T19:04:46.474 INFO:tasks.workunit.client.1.smithi132.stdout:0/25: chown d2 13236339 1 2024-02-19T19:04:46.482 INFO:tasks.workunit.client.1.smithi132.stdout:9/89: link d12/d1b/f17 d12/d1b/f22 0 2024-02-19T19:04:46.482 INFO:tasks.workunit.client.1.smithi132.stdout:6/34: read d0/f5 [24851,75867] 0 2024-02-19T19:04:46.484 INFO:tasks.workunit.client.1.smithi132.stdout:3/36: dread f3 [0,4194304] 0 2024-02-19T19:04:46.485 INFO:tasks.workunit.client.1.smithi132.stdout:5/74: creat d8/da/db/f15 x:0 0 0 2024-02-19T19:04:46.486 INFO:tasks.workunit.client.1.smithi132.stdout:8/83: rmdir d4/dc/d1b/d1d 39 2024-02-19T19:04:46.487 INFO:tasks.workunit.client.1.smithi132.stdout:4/25: creat d4/f5 x:0 0 0 2024-02-19T19:04:46.488 INFO:tasks.workunit.client.1.smithi132.stdout:4/26: chown f3 8 1 2024-02-19T19:04:46.489 INFO:tasks.workunit.client.1.smithi132.stdout:5/75: dread d8/fe [0,4194304] 0 2024-02-19T19:04:46.489 INFO:tasks.workunit.client.1.smithi132.stdout:9/90: creat d12/f23 x:0 0 0 2024-02-19T19:04:46.494 INFO:tasks.workunit.client.1.smithi132.stdout:3/37: rename f3 to d8/fa 0 2024-02-19T19:04:46.496 INFO:tasks.workunit.client.1.smithi132.stdout:9/91: dwrite d12/f19 [0,4194304] 0 2024-02-19T19:04:46.498 INFO:tasks.workunit.client.1.smithi132.stdout:0/26: mknod d2/c3 0 2024-02-19T19:04:46.504 INFO:tasks.workunit.client.1.smithi132.stdout:0/27: dwrite f0 [0,4194304] 0 2024-02-19T19:04:46.513 INFO:tasks.workunit.client.1.smithi132.stdout:5/76: rmdir d8/da/db 39 2024-02-19T19:04:46.517 INFO:tasks.workunit.client.1.smithi132.stdout:6/35: write d0/f5 [278029,46365] 0 2024-02-19T19:04:46.522 INFO:tasks.workunit.client.1.smithi132.stdout:9/92: symlink d12/d1b/l24 0 2024-02-19T19:04:46.524 INFO:tasks.workunit.client.1.smithi132.stdout:4/27: write f3 [1128361,46071] 0 2024-02-19T19:04:46.526 INFO:tasks.workunit.client.1.smithi132.stdout:4/28: truncate d4/f5 122383 0 2024-02-19T19:04:46.530 INFO:tasks.workunit.client.1.smithi132.stdout:0/28: rename f0 to d2/f4 0 2024-02-19T19:04:46.530 INFO:tasks.workunit.client.1.smithi132.stdout:9/93: dwrite d12/d1b/f1c [0,4194304] 0 2024-02-19T19:04:46.532 INFO:tasks.workunit.client.1.smithi132.stdout:3/38: unlink d8/fa 0 2024-02-19T19:04:46.533 INFO:tasks.workunit.client.1.smithi132.stdout:3/39: fdatasync f0 0 2024-02-19T19:04:46.534 INFO:tasks.workunit.client.1.smithi132.stdout:3/40: write f4 [592635,37124] 0 2024-02-19T19:04:46.534 INFO:tasks.workunit.client.1.smithi132.stdout:6/36: symlink d0/le 0 2024-02-19T19:04:46.535 INFO:tasks.workunit.client.1.smithi132.stdout:6/37: dread - d0/d1/f8 zero size 2024-02-19T19:04:46.538 INFO:tasks.workunit.client.1.smithi132.stdout:6/38: truncate d0/db/fd 809470 0 2024-02-19T19:04:46.538 INFO:tasks.workunit.client.1.smithi132.stdout:8/84: truncate d4/f9 3840679 0 2024-02-19T19:04:46.540 INFO:tasks.workunit.client.1.smithi132.stdout:3/41: dread f0 [0,4194304] 0 2024-02-19T19:04:46.541 INFO:tasks.workunit.client.1.smithi132.stdout:4/29: symlink d4/l6 0 2024-02-19T19:04:46.542 INFO:tasks.workunit.client.1.smithi132.stdout:0/29: symlink d2/l5 0 2024-02-19T19:04:46.543 INFO:tasks.workunit.client.1.smithi132.stdout:5/77: rename d8/c9 to d8/da/db/c16 0 2024-02-19T19:04:46.547 INFO:tasks.workunit.client.1.smithi132.stdout:2/64: link d0/d4/d11/f15 d0/d4/f1f 0 2024-02-19T19:04:46.547 INFO:tasks.workunit.client.1.smithi132.stdout:4/30: dwrite f3 [0,4194304] 0 2024-02-19T19:04:46.549 INFO:tasks.workunit.client.1.smithi132.stdout:9/94: fdatasync d12/d1b/f1c 0 2024-02-19T19:04:46.550 INFO:tasks.workunit.client.1.smithi132.stdout:9/95: read d12/d1b/f1c [1844418,95261] 0 2024-02-19T19:04:46.555 INFO:tasks.workunit.client.1.smithi132.stdout:9/96: dread d12/d1b/f22 [0,4194304] 0 2024-02-19T19:04:46.555 INFO:tasks.workunit.client.1.smithi132.stdout:9/97: fdatasync fb 0 2024-02-19T19:04:46.556 INFO:tasks.workunit.client.1.smithi132.stdout:0/30: rename d2/f4 to d2/f6 0 2024-02-19T19:04:46.559 INFO:tasks.workunit.client.1.smithi132.stdout:6/39: mknod d0/dc/cf 0 2024-02-19T19:04:46.562 INFO:tasks.workunit.client.1.smithi132.stdout:4/31: fsync f3 0 2024-02-19T19:04:46.564 INFO:tasks.workunit.client.1.smithi132.stdout:8/85: chown d4/dc/d1b/d1d/f1e 4 1 2024-02-19T19:04:46.571 INFO:tasks.workunit.client.1.smithi132.stdout:9/98: mkdir d12/d25 0 2024-02-19T19:04:46.572 INFO:tasks.workunit.client.1.smithi132.stdout:9/99: chown c9 6 1 2024-02-19T19:04:46.574 INFO:tasks.workunit.client.1.smithi132.stdout:0/31: rename d2/l5 to d2/l7 0 2024-02-19T19:04:46.574 INFO:tasks.workunit.client.1.smithi132.stdout:9/100: dread d12/d1b/f16 [0,4194304] 0 2024-02-19T19:04:46.578 INFO:tasks.workunit.client.1.smithi132.stdout:9/101: dread fd [0,4194304] 0 2024-02-19T19:04:46.579 INFO:tasks.workunit.client.1.smithi132.stdout:6/40: creat d0/f10 x:0 0 0 2024-02-19T19:04:46.579 INFO:tasks.workunit.client.1.smithi132.stdout:6/41: chown d0/d1 11406505 1 2024-02-19T19:04:46.580 INFO:tasks.workunit.client.1.smithi132.stdout:6/42: write d0/f10 [827927,31538] 0 2024-02-19T19:04:46.583 INFO:tasks.workunit.client.1.smithi132.stdout:5/78: truncate f6 3943404 0 2024-02-19T19:04:46.585 INFO:tasks.workunit.client.1.smithi132.stdout:0/32: symlink d2/l8 0 2024-02-19T19:04:46.594 INFO:tasks.workunit.client.1.smithi132.stdout:9/102: creat d12/f26 x:0 0 0 2024-02-19T19:04:46.594 INFO:tasks.workunit.client.1.smithi132.stdout:2/65: creat d0/d4/d11/f20 x:0 0 0 2024-02-19T19:04:46.594 INFO:tasks.workunit.client.1.smithi132.stdout:6/43: mknod d0/c11 0 2024-02-19T19:04:46.595 INFO:tasks.workunit.client.1.smithi132.stdout:5/79: mknod d8/da/db/c17 0 2024-02-19T19:04:46.597 INFO:tasks.workunit.client.1.smithi132.stdout:0/33: symlink d2/l9 0 2024-02-19T19:04:46.601 INFO:tasks.workunit.client.1.smithi132.stdout:4/32: truncate f1 2650119 0 2024-02-19T19:04:46.602 INFO:tasks.workunit.client.1.smithi132.stdout:1/17: sync 2024-02-19T19:04:46.603 INFO:tasks.workunit.client.1.smithi132.stdout:1/18: write d1/f3 [73929,81894] 0 2024-02-19T19:04:46.655 INFO:tasks.workunit.client.1.smithi132.stdout:6/44: creat d0/f12 x:0 0 0 2024-02-19T19:04:46.655 INFO:tasks.workunit.client.1.smithi132.stdout:9/103: creat d12/f27 x:0 0 0 2024-02-19T19:04:46.657 INFO:tasks.workunit.client.1.smithi132.stdout:5/80: mkdir d8/d18 0 2024-02-19T19:04:46.657 INFO:tasks.workunit.client.1.smithi132.stdout:5/81: fsync f5 0 2024-02-19T19:04:46.659 INFO:tasks.workunit.client.1.smithi132.stdout:0/34: rename d2/l9 to d2/la 0 2024-02-19T19:04:46.661 INFO:tasks.workunit.client.1.smithi132.stdout:6/45: dwrite d0/d1/f9 [0,4194304] 0 2024-02-19T19:04:46.661 INFO:tasks.workunit.client.1.smithi132.stdout:1/19: symlink d1/l7 0 2024-02-19T19:04:46.662 INFO:tasks.workunit.client.1.smithi132.stdout:6/46: fdatasync d0/fa 0 2024-02-19T19:04:46.663 INFO:tasks.workunit.client.1.smithi132.stdout:9/104: rmdir d12/d1b 39 2024-02-19T19:04:46.664 INFO:tasks.workunit.client.1.smithi132.stdout:1/20: write d1/f2 [4169000,110139] 0 2024-02-19T19:04:46.666 INFO:tasks.workunit.client.1.smithi132.stdout:2/66: fsync d0/d4/d11/f1c 0 2024-02-19T19:04:46.666 INFO:tasks.workunit.client.1.smithi132.stdout:6/47: read d0/fa [3886764,91775] 0 2024-02-19T19:04:46.666 INFO:tasks.workunit.client.1.smithi132.stdout:9/105: stat d12/f1a 0 2024-02-19T19:04:46.668 INFO:tasks.workunit.client.1.smithi132.stdout:1/21: readlink d1/l7 0 2024-02-19T19:04:46.669 INFO:tasks.workunit.client.1.smithi132.stdout:2/67: chown d0/d19/d1e 2044121 1 2024-02-19T19:04:46.670 INFO:tasks.workunit.client.1.smithi132.stdout:5/82: dwrite f5 [0,4194304] 0 2024-02-19T19:04:46.673 INFO:tasks.workunit.client.1.smithi132.stdout:6/48: rmdir d0/db 39 2024-02-19T19:04:46.674 INFO:tasks.workunit.client.1.smithi132.stdout:1/22: write d1/f5 [353733,70611] 0 2024-02-19T19:04:46.675 INFO:tasks.workunit.client.1.smithi132.stdout:1/23: rename d1 to d1/d8 22 2024-02-19T19:04:46.675 INFO:tasks.workunit.client.1.smithi132.stdout:1/24: read d1/f3 [92179,105651] 0 2024-02-19T19:04:46.680 INFO:tasks.workunit.client.1.smithi132.stdout:5/83: symlink d8/da/db/l19 0 2024-02-19T19:04:46.680 INFO:tasks.workunit.client.1.smithi132.stdout:7/62: sync 2024-02-19T19:04:46.685 INFO:tasks.workunit.client.1.smithi132.stdout:6/49: fsync d0/db/fd 0 2024-02-19T19:04:46.685 INFO:tasks.workunit.client.1.smithi132.stdout:9/106: creat d12/d25/f28 x:0 0 0 2024-02-19T19:04:46.687 INFO:tasks.workunit.client.1.smithi132.stdout:4/33: read f1 [1036729,82873] 0 2024-02-19T19:04:46.688 INFO:tasks.workunit.client.1.smithi132.stdout:1/25: symlink d1/l9 0 2024-02-19T19:04:46.689 INFO:tasks.workunit.client.1.smithi132.stdout:5/84: dwrite d8/da/db/f15 [0,4194304] 0 2024-02-19T19:04:46.689 INFO:tasks.workunit.client.1.smithi132.stdout:4/34: read f3 [1766733,114146] 0 2024-02-19T19:04:46.694 INFO:tasks.workunit.client.1.smithi132.stdout:7/63: write f10 [261811,18591] 0 2024-02-19T19:04:46.701 INFO:tasks.workunit.client.1.smithi132.stdout:7/64: dwrite ff [0,4194304] 0 2024-02-19T19:04:46.704 INFO:tasks.workunit.client.1.smithi132.stdout:9/107: creat d12/d25/f29 x:0 0 0 2024-02-19T19:04:46.704 INFO:tasks.workunit.client.1.smithi132.stdout:9/108: chown fd 10220636 1 2024-02-19T19:04:46.705 INFO:tasks.workunit.client.1.smithi132.stdout:5/85: mkdir d8/d1a 0 2024-02-19T19:04:46.712 INFO:tasks.workunit.client.1.smithi132.stdout:7/65: stat l9 0 2024-02-19T19:04:46.724 INFO:tasks.workunit.client.1.smithi132.stdout:1/26: link d1/l7 d1/la 0 2024-02-19T19:04:46.724 INFO:tasks.workunit.client.1.smithi132.stdout:6/50: getdents d0/db 0 2024-02-19T19:04:46.725 INFO:tasks.workunit.client.1.smithi132.stdout:7/66: mknod c19 0 2024-02-19T19:04:46.726 INFO:tasks.workunit.client.1.smithi132.stdout:7/67: read ff [2210050,53359] 0 2024-02-19T19:04:46.727 INFO:tasks.workunit.client.1.smithi132.stdout:1/27: readlink d1/l7 0 2024-02-19T19:04:46.732 INFO:tasks.workunit.client.1.smithi132.stdout:1/28: symlink d1/lb 0 2024-02-19T19:04:46.740 INFO:tasks.workunit.client.1.smithi132.stdout:6/51: link d0/dc/cf d0/dc/c13 0 2024-02-19T19:04:46.742 INFO:tasks.workunit.client.1.smithi132.stdout:7/68: creat f1a x:0 0 0 2024-02-19T19:04:46.744 INFO:tasks.workunit.client.1.smithi132.stdout:1/29: rename d1/la to d1/lc 0 2024-02-19T19:04:46.744 INFO:tasks.workunit.client.1.smithi132.stdout:1/30: chown d1 3064 1 2024-02-19T19:04:46.746 INFO:tasks.workunit.client.1.smithi132.stdout:6/52: creat d0/dc/f14 x:0 0 0 2024-02-19T19:04:46.746 INFO:tasks.workunit.client.1.smithi132.stdout:6/53: stat d0/f10 0 2024-02-19T19:04:46.748 INFO:tasks.workunit.client.1.smithi132.stdout:7/69: mkdir d1b 0 2024-02-19T19:04:46.750 INFO:tasks.workunit.client.1.smithi132.stdout:1/31: dwrite d1/f2 [0,4194304] 0 2024-02-19T19:04:46.750 INFO:tasks.workunit.client.1.smithi132.stdout:1/32: chown d1/l9 1627 1 2024-02-19T19:04:46.751 INFO:tasks.workunit.client.1.smithi132.stdout:6/54: creat d0/dc/f15 x:0 0 0 2024-02-19T19:04:46.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T19:04:46.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T19:04:46.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: pgmap v99: 65 pgs: 65 active+clean; 188 MiB data, 826 MiB used, 536 GiB / 536 GiB avail; 177 KiB/s rd, 12 MiB/s wr, 157 op/s 2024-02-19T19:04:46.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: mon.smithi088 calling monitor election 2024-02-19T19:04:46.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: mon.smithi088 is new leader, mons smithi088,smithi132 in quorum (ranks 0,1) 2024-02-19T19:04:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: monmap e2: 2 mons at {smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0],smithi132=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T19:04:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:04:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: osdmap e48: 6 total, 6 up, 6 in 2024-02-19T19:04:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: mgrmap e32: smithi088.gyviqq(active, since 3m), standbys: smithi132.psllsa 2024-02-19T19:04:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: overall HEALTH_OK 2024-02-19T19:04:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: from='mgr.14682 ' entity='' 2024-02-19T19:04:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:46 smithi132 ceph-mon[37969]: mgrmap e33: smithi088.gyviqq(active, since 3m), standbys: smithi132.psllsa 2024-02-19T19:04:46.753 INFO:tasks.workunit.client.1.smithi132.stdout:6/55: truncate d0/dc/f15 382015 0 2024-02-19T19:04:46.758 INFO:tasks.workunit.client.1.smithi132.stdout:0/35: truncate f1 472467 0 2024-02-19T19:04:46.760 INFO:tasks.workunit.client.1.smithi132.stdout:6/56: mkdir d0/db/d16 0 2024-02-19T19:04:46.760 INFO:tasks.workunit.client.1.smithi132.stdout:6/57: fdatasync d0/fa 0 2024-02-19T19:04:46.764 INFO:tasks.workunit.client.1.smithi132.stdout:0/36: creat d2/fb x:0 0 0 2024-02-19T19:04:46.765 INFO:tasks.workunit.client.1.smithi132.stdout:6/58: mknod d0/d1/c17 0 2024-02-19T19:04:46.768 INFO:tasks.workunit.client.1.smithi132.stdout:2/68: dwrite d0/d4/f1f [0,4194304] 0 2024-02-19T19:04:46.768 INFO:tasks.workunit.client.1.smithi132.stdout:4/35: dwrite f1 [0,4194304] 0 2024-02-19T19:04:46.770 INFO:tasks.workunit.client.1.smithi132.stdout:4/36: chown d4/l6 0 1 2024-02-19T19:04:46.773 INFO:tasks.workunit.client.1.smithi132.stdout:4/37: write f3 [645668,129979] 0 2024-02-19T19:04:46.773 INFO:tasks.workunit.client.1.smithi132.stdout:8/86: sync 2024-02-19T19:04:46.773 INFO:tasks.workunit.client.1.smithi132.stdout:3/42: sync 2024-02-19T19:04:46.776 INFO:tasks.workunit.client.1.smithi132.stdout:3/43: chown c7 167707892 1 2024-02-19T19:04:46.776 INFO:tasks.workunit.client.1.smithi132.stdout:2/69: rmdir d0 39 2024-02-19T19:04:46.779 INFO:tasks.workunit.client.1.smithi132.stdout:8/87: write d4/f19 [4282399,90036] 0 2024-02-19T19:04:46.782 INFO:tasks.workunit.client.1.smithi132.stdout:0/37: dwrite d2/f6 [4194304,4194304] 0 2024-02-19T19:04:46.784 INFO:tasks.workunit.client.1.smithi132.stdout:6/59: link d0/f5 d0/db/d16/f18 0 2024-02-19T19:04:46.786 INFO:tasks.workunit.client.1.smithi132.stdout:9/109: dwrite d12/d1b/f17 [0,4194304] 0 2024-02-19T19:04:46.791 INFO:tasks.workunit.client.1.smithi132.stdout:3/44: read f4 [21410,130966] 0 2024-02-19T19:04:46.792 INFO:tasks.workunit.client.1.smithi132.stdout:4/38: dwrite f1 [4194304,4194304] 0 2024-02-19T19:04:46.793 INFO:tasks.workunit.client.1.smithi132.stdout:0/38: mkdir d2/dc 0 2024-02-19T19:04:46.793 INFO:tasks.workunit.client.1.smithi132.stdout:3/45: read f0 [3415892,122074] 0 2024-02-19T19:04:46.795 INFO:tasks.workunit.client.1.smithi132.stdout:5/86: write f6 [4225165,39713] 0 2024-02-19T19:04:46.797 INFO:tasks.workunit.client.1.smithi132.stdout:0/39: write d2/fb [959956,128851] 0 2024-02-19T19:04:46.798 INFO:tasks.workunit.client.1.smithi132.stdout:8/88: rename d4/c15 to d4/dc/c1f 0 2024-02-19T19:04:46.803 INFO:tasks.workunit.client.1.smithi132.stdout:7/70: dwrite f15 [4194304,4194304] 0 2024-02-19T19:04:46.805 INFO:tasks.workunit.client.1.smithi132.stdout:1/33: truncate d1/f2 49947 0 2024-02-19T19:04:46.805 INFO:tasks.workunit.client.1.smithi132.stdout:6/60: creat d0/d1/f19 x:0 0 0 2024-02-19T19:04:46.806 INFO:tasks.workunit.client.1.smithi132.stdout:2/70: creat d0/d4/df/f21 x:0 0 0 2024-02-19T19:04:46.807 INFO:tasks.workunit.client.1.smithi132.stdout:4/39: creat d4/f7 x:0 0 0 2024-02-19T19:04:46.808 INFO:tasks.workunit.client.1.smithi132.stdout:9/110: dread d12/d1b/f16 [0,4194304] 0 2024-02-19T19:04:46.810 INFO:tasks.workunit.client.1.smithi132.stdout:6/61: write d0/f12 [734400,76663] 0 2024-02-19T19:04:46.813 INFO:tasks.workunit.client.1.smithi132.stdout:3/46: dread d8/f9 [0,4194304] 0 2024-02-19T19:04:46.813 INFO:tasks.workunit.client.1.smithi132.stdout:0/40: symlink d2/ld 0 2024-02-19T19:04:46.815 INFO:tasks.workunit.client.1.smithi132.stdout:7/71: fdatasync f6 0 2024-02-19T19:04:46.815 INFO:tasks.workunit.client.1.smithi132.stdout:6/62: chown d0/d1/f9 70921777 1 2024-02-19T19:04:46.816 INFO:tasks.workunit.client.1.smithi132.stdout:4/40: creat d4/f8 x:0 0 0 2024-02-19T19:04:46.817 INFO:tasks.workunit.client.1.smithi132.stdout:7/72: write f15 [6472051,92765] 0 2024-02-19T19:04:46.818 INFO:tasks.workunit.client.1.smithi132.stdout:1/34: dwrite d1/f5 [0,4194304] 0 2024-02-19T19:04:46.819 INFO:tasks.workunit.client.1.smithi132.stdout:4/41: truncate d4/f7 226586 0 2024-02-19T19:04:46.819 INFO:tasks.workunit.client.1.smithi132.stdout:9/111: unlink d12/d1b/l24 0 2024-02-19T19:04:46.820 INFO:tasks.workunit.client.1.smithi132.stdout:7/73: dread - f1a zero size 2024-02-19T19:04:46.821 INFO:tasks.workunit.client.1.smithi132.stdout:3/47: rmdir d8 39 2024-02-19T19:04:46.822 INFO:tasks.workunit.client.1.smithi132.stdout:9/112: truncate d12/d25/f28 656125 0 2024-02-19T19:04:46.824 INFO:tasks.workunit.client.1.smithi132.stdout:6/63: dread d0/fa [0,4194304] 0 2024-02-19T19:04:46.830 INFO:tasks.workunit.client.1.smithi132.stdout:4/42: dwrite d4/f8 [0,4194304] 0 2024-02-19T19:04:46.834 INFO:tasks.workunit.client.1.smithi132.stdout:3/48: dread d8/f9 [0,4194304] 0 2024-02-19T19:04:46.834 INFO:tasks.workunit.client.1.smithi132.stdout:0/41: creat d2/dc/fe x:0 0 0 2024-02-19T19:04:46.836 INFO:tasks.workunit.client.1.smithi132.stdout:8/89: write f2 [1123184,93713] 0 2024-02-19T19:04:46.838 INFO:tasks.workunit.client.1.smithi132.stdout:2/71: creat d0/d4/f22 x:0 0 0 2024-02-19T19:04:46.840 INFO:tasks.workunit.client.1.smithi132.stdout:8/90: dread d4/f6 [0,4194304] 0 2024-02-19T19:04:46.841 INFO:tasks.workunit.client.1.smithi132.stdout:8/91: write d4/f6 [1840113,118423] 0 2024-02-19T19:04:46.844 INFO:tasks.workunit.client.1.smithi132.stdout:6/64: symlink d0/dc/l1a 0 2024-02-19T19:04:46.844 INFO:tasks.workunit.client.1.smithi132.stdout:1/35: write d1/f3 [167059,115383] 0 2024-02-19T19:04:46.845 INFO:tasks.workunit.client.1.smithi132.stdout:7/74: creat d1b/f1c x:0 0 0 2024-02-19T19:04:46.845 INFO:tasks.workunit.client.1.smithi132.stdout:7/75: chown d1b 1269848 1 2024-02-19T19:04:46.848 INFO:tasks.workunit.client.1.smithi132.stdout:6/65: dread d0/f10 [0,4194304] 0 2024-02-19T19:04:46.849 INFO:tasks.workunit.client.1.smithi132.stdout:4/43: rename f1 to d4/f9 0 2024-02-19T19:04:46.849 INFO:tasks.workunit.client.1.smithi132.stdout:3/49: symlink d8/lb 0 2024-02-19T19:04:46.849 INFO:tasks.workunit.client.1.smithi132.stdout:0/42: unlink d2/ld 0 2024-02-19T19:04:46.850 INFO:tasks.workunit.client.1.smithi132.stdout:6/66: dread - d0/dc/f14 zero size 2024-02-19T19:04:46.850 INFO:tasks.workunit.client.1.smithi132.stdout:9/113: mkdir d12/d2a 0 2024-02-19T19:04:46.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T19:04:46.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: from='mgr.14682 172.21.15.88:0/2187068915' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T19:04:46.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: pgmap v99: 65 pgs: 65 active+clean; 188 MiB data, 826 MiB used, 536 GiB / 536 GiB avail; 177 KiB/s rd, 12 MiB/s wr, 157 op/s 2024-02-19T19:04:46.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: mon.smithi088 calling monitor election 2024-02-19T19:04:46.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: mon.smithi088 is new leader, mons smithi088,smithi132 in quorum (ranks 0,1) 2024-02-19T19:04:46.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: monmap e2: 2 mons at {smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0],smithi132=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T19:04:46.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:04:46.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: osdmap e48: 6 total, 6 up, 6 in 2024-02-19T19:04:46.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: mgrmap e32: smithi088.gyviqq(active, since 3m), standbys: smithi132.psllsa 2024-02-19T19:04:46.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: overall HEALTH_OK 2024-02-19T19:04:46.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: from='mgr.14682 ' entity='' 2024-02-19T19:04:46.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:46 smithi088 ceph-mon[88643]: mgrmap e33: smithi088.gyviqq(active, since 3m), standbys: smithi132.psllsa 2024-02-19T19:04:46.851 INFO:tasks.workunit.client.1.smithi132.stdout:6/67: readlink d0/le 0 2024-02-19T19:04:46.852 INFO:tasks.workunit.client.1.smithi132.stdout:2/72: symlink d0/d4/df/l23 0 2024-02-19T19:04:46.854 INFO:tasks.workunit.client.1.smithi132.stdout:6/68: dread d0/f10 [0,4194304] 0 2024-02-19T19:04:46.854 INFO:tasks.workunit.client.1.smithi132.stdout:8/92: chown d4/dc/c1f 98282 1 2024-02-19T19:04:46.857 INFO:tasks.workunit.client.1.smithi132.stdout:7/76: creat d1b/f1d x:0 0 0 2024-02-19T19:04:46.858 INFO:tasks.workunit.client.1.smithi132.stdout:1/36: fsync d1/f3 0 2024-02-19T19:04:46.858 INFO:tasks.workunit.client.1.smithi132.stdout:0/43: mkdir d2/dc/df 0 2024-02-19T19:04:46.860 INFO:tasks.workunit.client.1.smithi132.stdout:7/77: chown f1a 3 1 2024-02-19T19:04:46.862 INFO:tasks.workunit.client.1.smithi132.stdout:1/37: chown d1/l9 1581 1 2024-02-19T19:04:46.862 INFO:tasks.workunit.client.1.smithi132.stdout:3/50: dwrite f0 [0,4194304] 0 2024-02-19T19:04:46.863 INFO:tasks.workunit.client.1.smithi132.stdout:4/44: dwrite d4/f7 [0,4194304] 0 2024-02-19T19:04:46.864 INFO:tasks.workunit.client.1.smithi132.stdout:7/78: dread - d1b/f1d zero size 2024-02-19T19:04:46.864 INFO:tasks.workunit.client.1.smithi132.stdout:0/44: write d2/dc/fe [290343,86832] 0 2024-02-19T19:04:46.866 INFO:tasks.workunit.client.1.smithi132.stdout:4/45: stat d4 0 2024-02-19T19:04:46.869 INFO:tasks.workunit.client.1.smithi132.stdout:4/46: fdatasync d4/f5 0 2024-02-19T19:04:46.870 INFO:tasks.workunit.client.1.smithi132.stdout:9/114: creat d12/d2a/f2b x:0 0 0 2024-02-19T19:04:46.871 INFO:tasks.workunit.client.1.smithi132.stdout:6/69: mknod d0/c1b 0 2024-02-19T19:04:46.872 INFO:tasks.workunit.client.1.smithi132.stdout:2/73: dwrite d0/d4/d11/f15 [0,4194304] 0 2024-02-19T19:04:46.872 INFO:tasks.workunit.client.1.smithi132.stdout:6/70: chown d0 265397978 1 2024-02-19T19:04:46.875 INFO:tasks.workunit.client.1.smithi132.stdout:3/51: dread f0 [0,4194304] 0 2024-02-19T19:04:46.877 INFO:tasks.workunit.client.1.smithi132.stdout:1/38: creat d1/fd x:0 0 0 2024-02-19T19:04:46.880 INFO:tasks.workunit.client.1.smithi132.stdout:0/45: symlink d2/dc/l10 0 2024-02-19T19:04:46.882 INFO:tasks.workunit.client.1.smithi132.stdout:8/93: mkdir d4/dc/d1b/d20 0 2024-02-19T19:04:46.882 INFO:tasks.workunit.client.1.smithi132.stdout:4/47: write d4/f9 [7966473,79059] 0 2024-02-19T19:04:46.883 INFO:tasks.workunit.client.1.smithi132.stdout:8/94: write d4/dc/d1b/d1d/f1e [375394,112734] 0 2024-02-19T19:04:46.884 INFO:tasks.workunit.client.1.smithi132.stdout:2/74: mkdir d0/d24 0 2024-02-19T19:04:46.887 INFO:tasks.workunit.client.1.smithi132.stdout:9/115: rename d12/f19 to d12/d2a/f2c 0 2024-02-19T19:04:46.888 INFO:tasks.workunit.client.1.smithi132.stdout:2/75: dread d0/d4/d11/f15 [0,4194304] 0 2024-02-19T19:04:46.889 INFO:tasks.workunit.client.1.smithi132.stdout:1/39: rename d1/f3 to d1/fe 0 2024-02-19T19:04:46.891 INFO:tasks.workunit.client.1.smithi132.stdout:1/40: write d1/fd [557377,69853] 0 2024-02-19T19:04:46.891 INFO:tasks.workunit.client.1.smithi132.stdout:0/46: mknod d2/c11 0 2024-02-19T19:04:46.892 INFO:tasks.workunit.client.1.smithi132.stdout:1/41: readlink d1/l9 0 2024-02-19T19:04:46.893 INFO:tasks.workunit.client.1.smithi132.stdout:4/48: creat d4/fa x:0 0 0 2024-02-19T19:04:46.895 INFO:tasks.workunit.client.1.smithi132.stdout:7/79: write f6 [4348021,99700] 0 2024-02-19T19:04:46.896 INFO:tasks.workunit.client.1.smithi132.stdout:2/76: dwrite d0/d4/f22 [0,4194304] 0 2024-02-19T19:04:46.897 INFO:tasks.workunit.client.1.smithi132.stdout:2/77: chown d0/d4/cc 265833348 1 2024-02-19T19:04:46.903 INFO:tasks.workunit.client.1.smithi132.stdout:8/95: creat d4/dc/d1b/d1d/f21 x:0 0 0 2024-02-19T19:04:46.909 INFO:tasks.workunit.client.1.smithi132.stdout:6/71: dwrite d0/f5 [0,4194304] 0 2024-02-19T19:04:46.912 INFO:tasks.workunit.client.1.smithi132.stdout:7/80: fdatasync f8 0 2024-02-19T19:04:46.915 INFO:tasks.workunit.client.1.smithi132.stdout:8/96: creat d4/dc/f22 x:0 0 0 2024-02-19T19:04:46.919 INFO:tasks.workunit.client.1.smithi132.stdout:3/52: link d8/lb d8/lc 0 2024-02-19T19:04:46.920 INFO:tasks.workunit.client.1.smithi132.stdout:0/47: rename f1 to d2/f12 0 2024-02-19T19:04:46.920 INFO:tasks.workunit.client.1.smithi132.stdout:0/48: readlink d2/l8 0 2024-02-19T19:04:46.922 INFO:tasks.workunit.client.1.smithi132.stdout:0/49: write d2/fb [1822685,81143] 0 2024-02-19T19:04:46.922 INFO:tasks.workunit.client.1.smithi132.stdout:3/53: dread f4 [0,4194304] 0 2024-02-19T19:04:46.922 INFO:tasks.workunit.client.1.smithi132.stdout:7/81: mkdir d1b/d1e 0 2024-02-19T19:04:46.923 INFO:tasks.workunit.client.1.smithi132.stdout:8/97: chown d4/la 31 1 2024-02-19T19:04:46.924 INFO:tasks.workunit.client.1.smithi132.stdout:7/82: write d1b/f1d [997166,55940] 0 2024-02-19T19:04:46.927 INFO:tasks.workunit.client.1.smithi132.stdout:0/50: mknod d2/dc/c13 0 2024-02-19T19:04:46.929 INFO:tasks.workunit.client.1.smithi132.stdout:3/54: truncate d8/f9 1038082 0 2024-02-19T19:04:46.966 INFO:tasks.workunit.client.1.smithi132.stdout:7/83: unlink f1a 0 2024-02-19T19:04:46.967 INFO:tasks.workunit.client.1.smithi132.stdout:5/87: sync 2024-02-19T19:04:46.967 INFO:tasks.workunit.client.1.smithi132.stdout:0/51: mknod d2/c14 0 2024-02-19T19:04:46.967 INFO:tasks.workunit.client.1.smithi132.stdout:3/55: creat d8/fd x:0 0 0 2024-02-19T19:04:46.967 INFO:tasks.workunit.client.1.smithi132.stdout:9/116: link l6 d12/l2d 0 2024-02-19T19:04:46.968 INFO:tasks.workunit.client.1.smithi132.stdout:8/98: mkdir d4/dc/d1b/d20/d23 0 2024-02-19T19:04:46.969 INFO:tasks.workunit.client.1.smithi132.stdout:5/88: truncate f5 4645535 0 2024-02-19T19:04:46.970 INFO:tasks.workunit.client.1.smithi132.stdout:7/84: mknod d1b/c1f 0 2024-02-19T19:04:46.972 INFO:tasks.workunit.client.1.smithi132.stdout:2/78: dwrite d0/f5 [0,4194304] 0 2024-02-19T19:04:46.972 INFO:tasks.workunit.client.1.smithi132.stdout:8/99: write d4/f19 [2159603,78666] 0 2024-02-19T19:04:46.974 INFO:tasks.workunit.client.1.smithi132.stdout:3/56: write f4 [1394349,69451] 0 2024-02-19T19:04:46.975 INFO:tasks.workunit.client.1.smithi132.stdout:5/89: unlink d8/c14 0 2024-02-19T19:04:46.978 INFO:tasks.workunit.client.1.smithi132.stdout:3/57: fdatasync d8/fd 0 2024-02-19T19:04:46.985 INFO:tasks.workunit.client.1.smithi132.stdout:0/52: dwrite d2/dc/fe [0,4194304] 0 2024-02-19T19:04:46.985 INFO:tasks.workunit.client.1.smithi132.stdout:7/85: getdents d1b/d1e 0 2024-02-19T19:04:46.987 INFO:tasks.workunit.client.1.smithi132.stdout:9/117: dwrite d12/d2a/f2b [0,4194304] 0 2024-02-19T19:04:46.988 INFO:tasks.workunit.client.1.smithi132.stdout:2/79: fsync d0/d4/d11/f15 0 2024-02-19T19:04:46.989 INFO:tasks.workunit.client.1.smithi132.stdout:5/90: dwrite f5 [0,4194304] 0 2024-02-19T19:04:47.000 INFO:tasks.workunit.client.1.smithi132.stdout:8/100: link d4/f6 d4/f24 0 2024-02-19T19:04:47.001 INFO:tasks.workunit.client.1.smithi132.stdout:2/80: creat d0/d19/f25 x:0 0 0 2024-02-19T19:04:47.001 INFO:tasks.workunit.client.1.smithi132.stdout:8/101: stat d4/dc/d1b/d20 0 2024-02-19T19:04:47.002 INFO:tasks.workunit.client.1.smithi132.stdout:2/81: chown d0/d19/f25 175288613 1 2024-02-19T19:04:47.003 INFO:tasks.workunit.client.1.smithi132.stdout:2/82: read - d0/d19/f25 zero size 2024-02-19T19:04:47.004 INFO:tasks.workunit.client.1.smithi132.stdout:0/53: mkdir d2/dc/df/d15 0 2024-02-19T19:04:47.004 INFO:tasks.workunit.client.1.smithi132.stdout:7/86: read f10 [680,90394] 0 2024-02-19T19:04:47.012 INFO:tasks.workunit.client.1.smithi132.stdout:5/91: link d8/ff d8/da/db/f1b 0 2024-02-19T19:04:47.014 INFO:tasks.workunit.client.1.smithi132.stdout:7/87: mknod d1b/c20 0 2024-02-19T19:04:47.014 INFO:tasks.workunit.client.1.smithi132.stdout:5/92: write f6 [1652410,5188] 0 2024-02-19T19:04:47.016 INFO:tasks.workunit.client.1.smithi132.stdout:3/58: getdents d8 0 2024-02-19T19:04:47.018 INFO:tasks.workunit.client.1.smithi132.stdout:3/59: write d8/fd [286724,37333] 0 2024-02-19T19:04:47.021 INFO:tasks.workunit.client.1.smithi132.stdout:2/83: creat d0/d4/d11/f26 x:0 0 0 2024-02-19T19:04:47.021 INFO:tasks.workunit.client.1.smithi132.stdout:3/60: fdatasync f0 0 2024-02-19T19:04:47.023 INFO:tasks.workunit.client.1.smithi132.stdout:0/54: dread d2/fb [0,4194304] 0 2024-02-19T19:04:47.023 INFO:tasks.workunit.client.1.smithi132.stdout:3/61: chown l5 234410 1 2024-02-19T19:04:47.026 INFO:tasks.workunit.client.1.smithi132.stdout:0/55: dread d2/f12 [0,4194304] 0 2024-02-19T19:04:47.030 INFO:tasks.workunit.client.1.smithi132.stdout:5/93: dwrite d8/da/db/f1b [0,4194304] 0 2024-02-19T19:04:47.030 INFO:tasks.workunit.client.1.smithi132.stdout:3/62: symlink d8/le 0 2024-02-19T19:04:47.032 INFO:tasks.workunit.client.1.smithi132.stdout:5/94: fsync d8/da/db/f15 0 2024-02-19T19:04:47.042 INFO:tasks.workunit.client.1.smithi132.stdout:9/118: dwrite d12/d1b/f20 [0,4194304] 0 2024-02-19T19:04:47.044 INFO:tasks.workunit.client.1.smithi132.stdout:3/63: read f4 [747083,119918] 0 2024-02-19T19:04:47.045 INFO:tasks.workunit.client.1.smithi132.stdout:2/84: truncate d0/d4/f1f 1855552 0 2024-02-19T19:04:47.045 INFO:tasks.workunit.client.1.smithi132.stdout:8/102: truncate d4/f9 2788790 0 2024-02-19T19:04:47.046 INFO:tasks.workunit.client.1.smithi132.stdout:7/88: write ff [2710508,1344] 0 2024-02-19T19:04:47.047 INFO:tasks.workunit.client.1.smithi132.stdout:5/95: unlink d8/da/db/c17 0 2024-02-19T19:04:47.047 INFO:tasks.workunit.client.1.smithi132.stdout:2/85: dread - d0/d4/d11/f26 zero size 2024-02-19T19:04:47.048 INFO:tasks.workunit.client.1.smithi132.stdout:5/96: chown d8/d18 0 1 2024-02-19T19:04:47.048 INFO:tasks.workunit.client.1.smithi132.stdout:2/86: readlink d0/d4/l7 0 2024-02-19T19:04:47.048 INFO:tasks.workunit.client.1.smithi132.stdout:9/119: chown lf 670227464 1 2024-02-19T19:04:47.049 INFO:tasks.workunit.client.1.smithi132.stdout:9/120: fsync fb 0 2024-02-19T19:04:47.049 INFO:tasks.workunit.client.1.smithi132.stdout:2/87: fsync d0/d4/fb 0 2024-02-19T19:04:47.050 INFO:tasks.workunit.client.1.smithi132.stdout:3/64: rmdir d8 39 2024-02-19T19:04:47.051 INFO:tasks.workunit.client.1.smithi132.stdout:2/88: write d0/d4/d11/f20 [1000591,60904] 0 2024-02-19T19:04:47.059 INFO:tasks.workunit.client.1.smithi132.stdout:5/97: stat d8/da/l10 0 2024-02-19T19:04:47.061 INFO:tasks.workunit.client.1.smithi132.stdout:0/56: truncate d2/fb 29581 0 2024-02-19T19:04:47.063 INFO:tasks.workunit.client.1.smithi132.stdout:0/57: write d2/dc/fe [276124,127951] 0 2024-02-19T19:04:47.063 INFO:tasks.workunit.client.1.smithi132.stdout:9/121: rmdir d12/d2a 39 2024-02-19T19:04:47.065 INFO:tasks.workunit.client.1.smithi132.stdout:8/103: write d4/f24 [118305,116594] 0 2024-02-19T19:04:47.072 INFO:tasks.workunit.client.1.smithi132.stdout:0/58: symlink d2/l16 0 2024-02-19T19:04:47.073 INFO:tasks.workunit.client.1.smithi132.stdout:3/65: rmdir d8 39 2024-02-19T19:04:47.075 INFO:tasks.workunit.client.1.smithi132.stdout:5/98: write d8/ff [5240209,2581] 0 2024-02-19T19:04:47.082 INFO:tasks.workunit.client.1.smithi132.stdout:5/99: dwrite f5 [4194304,4194304] 0 2024-02-19T19:04:47.083 INFO:tasks.workunit.client.1.smithi132.stdout:8/104: symlink d4/dc/d1b/l25 0 2024-02-19T19:04:47.084 INFO:tasks.workunit.client.1.smithi132.stdout:7/89: getdents d1b 0 2024-02-19T19:04:47.085 INFO:tasks.workunit.client.1.smithi132.stdout:8/105: rename d4/dc to d4/dc/d1b/d20/d26 22 2024-02-19T19:04:47.086 INFO:tasks.workunit.client.1.smithi132.stdout:8/106: stat d4/l14 0 2024-02-19T19:04:47.087 INFO:tasks.workunit.client.1.smithi132.stdout:8/107: readlink d4/l7 0 2024-02-19T19:04:47.087 INFO:tasks.workunit.client.1.smithi132.stdout:0/59: dwrite d2/f12 [0,4194304] 0 2024-02-19T19:04:47.088 INFO:tasks.workunit.client.1.smithi132.stdout:9/122: getdents d12/d25 0 2024-02-19T19:04:47.088 INFO:tasks.workunit.client.1.smithi132.stdout:0/60: read d2/f12 [1988183,16217] 0 2024-02-19T19:04:47.093 INFO:tasks.workunit.client.1.smithi132.stdout:5/100: creat d8/d18/f1c x:0 0 0 2024-02-19T19:04:47.095 INFO:tasks.workunit.client.1.smithi132.stdout:3/66: link l5 d8/lf 0 2024-02-19T19:04:47.096 INFO:tasks.workunit.client.1.smithi132.stdout:8/108: dread d4/f24 [0,4194304] 0 2024-02-19T19:04:47.097 INFO:tasks.workunit.client.1.smithi132.stdout:5/101: chown d8/da/db/f15 28597 1 2024-02-19T19:04:47.099 INFO:tasks.workunit.client.1.smithi132.stdout:7/90: creat d1b/d1e/f21 x:0 0 0 2024-02-19T19:04:47.101 INFO:tasks.workunit.client.1.smithi132.stdout:9/123: dwrite fa [4194304,4194304] 0 2024-02-19T19:04:47.102 INFO:tasks.workunit.client.1.smithi132.stdout:3/67: read f4 [382452,42936] 0 2024-02-19T19:04:47.103 INFO:tasks.workunit.client.1.smithi132.stdout:0/61: dwrite d2/f6 [4194304,4194304] 0 2024-02-19T19:04:47.103 INFO:tasks.workunit.client.1.smithi132.stdout:9/124: chown c0 7626 1 2024-02-19T19:04:47.103 INFO:tasks.workunit.client.1.smithi132.stdout:8/109: read d4/f9 [2745159,70284] 0 2024-02-19T19:04:47.108 INFO:tasks.workunit.client.1.smithi132.stdout:5/102: creat d8/da/db/f1d x:0 0 0 2024-02-19T19:04:47.108 INFO:tasks.workunit.client.1.smithi132.stdout:7/91: unlink d1b/c20 0 2024-02-19T19:04:47.111 INFO:tasks.workunit.client.1.smithi132.stdout:9/125: creat d12/d25/f2e x:0 0 0 2024-02-19T19:04:47.117 INFO:tasks.workunit.client.1.smithi132.stdout:8/110: creat d4/f27 x:0 0 0 2024-02-19T19:04:47.118 INFO:tasks.workunit.client.1.smithi132.stdout:9/126: mknod d12/d25/c2f 0 2024-02-19T19:04:47.119 INFO:tasks.workunit.client.1.smithi132.stdout:0/62: symlink d2/dc/df/d15/l17 0 2024-02-19T19:04:47.120 INFO:tasks.workunit.client.1.smithi132.stdout:5/103: mkdir d8/d1a/d1e 0 2024-02-19T19:04:47.123 INFO:tasks.workunit.client.1.smithi132.stdout:9/127: rename d12/d1b/f22 to d12/d25/f30 0 2024-02-19T19:04:47.123 INFO:tasks.workunit.client.1.smithi132.stdout:7/92: dread d1b/f1d [0,4194304] 0 2024-02-19T19:04:47.124 INFO:tasks.workunit.client.1.smithi132.stdout:8/111: creat d4/dc/d1b/d20/d23/f28 x:0 0 0 2024-02-19T19:04:47.125 INFO:tasks.workunit.client.1.smithi132.stdout:8/112: fsync d4/f19 0 2024-02-19T19:04:47.125 INFO:tasks.workunit.client.1.smithi132.stdout:5/104: creat d8/d18/f1f x:0 0 0 2024-02-19T19:04:47.133 INFO:tasks.workunit.client.1.smithi132.stdout:8/113: creat d4/f29 x:0 0 0 2024-02-19T19:04:47.136 INFO:tasks.workunit.client.1.smithi132.stdout:9/128: rename l6 to d12/d25/l31 0 2024-02-19T19:04:47.139 INFO:tasks.workunit.client.1.smithi132.stdout:5/105: dwrite d8/d18/f1f [0,4194304] 0 2024-02-19T19:04:47.140 INFO:tasks.workunit.client.1.smithi132.stdout:7/93: rename f15 to d1b/d1e/f22 0 2024-02-19T19:04:47.142 INFO:tasks.workunit.client.1.smithi132.stdout:7/94: read - d1b/d1e/f21 zero size 2024-02-19T19:04:47.142 INFO:tasks.workunit.client.1.smithi132.stdout:8/114: creat d4/dc/d1b/f2a x:0 0 0 2024-02-19T19:04:47.143 INFO:tasks.workunit.client.1.smithi132.stdout:9/129: dread fb [0,4194304] 0 2024-02-19T19:04:47.145 INFO:tasks.workunit.client.1.smithi132.stdout:5/106: creat d8/da/db/f20 x:0 0 0 2024-02-19T19:04:47.146 INFO:tasks.workunit.client.1.smithi132.stdout:5/107: read d8/ff [4296950,65526] 0 2024-02-19T19:04:47.146 INFO:tasks.workunit.client.1.smithi132.stdout:1/42: sync 2024-02-19T19:04:47.149 INFO:tasks.workunit.client.1.smithi132.stdout:7/95: mknod d1b/d1e/c23 0 2024-02-19T19:04:47.150 INFO:tasks.workunit.client.1.smithi132.stdout:7/96: truncate ff 4740656 0 2024-02-19T19:04:47.156 INFO:tasks.workunit.client.1.smithi132.stdout:1/43: dwrite d1/fd [0,4194304] 0 2024-02-19T19:04:47.158 INFO:tasks.workunit.client.1.smithi132.stdout:7/97: dread d1b/d1e/f22 [0,4194304] 0 2024-02-19T19:04:47.161 INFO:tasks.workunit.client.1.smithi132.stdout:1/44: creat d1/ff x:0 0 0 2024-02-19T19:04:47.170 INFO:tasks.workunit.client.1.smithi132.stdout:7/98: chown ce 112371111 1 2024-02-19T19:04:47.171 INFO:tasks.workunit.client.1.smithi132.stdout:7/99: write ff [5661401,104163] 0 2024-02-19T19:04:47.172 INFO:tasks.workunit.client.1.smithi132.stdout:7/100: write d1b/d1e/f21 [626161,3184] 0 2024-02-19T19:04:47.174 INFO:tasks.workunit.client.1.smithi132.stdout:1/45: symlink d1/l10 0 2024-02-19T19:04:47.177 INFO:tasks.workunit.client.1.smithi132.stdout:9/130: getdents d12/d25 0 2024-02-19T19:04:47.180 INFO:tasks.workunit.client.1.smithi132.stdout:7/101: symlink d1b/d1e/l24 0 2024-02-19T19:04:47.180 INFO:tasks.workunit.client.1.smithi132.stdout:9/131: dread f5 [0,4194304] 0 2024-02-19T19:04:47.181 INFO:tasks.workunit.client.1.smithi132.stdout:1/46: symlink d1/l11 0 2024-02-19T19:04:47.182 INFO:tasks.workunit.client.1.smithi132.stdout:1/47: truncate d1/ff 147613 0 2024-02-19T19:04:47.182 INFO:tasks.workunit.client.1.smithi132.stdout:5/108: rmdir d8 39 2024-02-19T19:04:47.183 INFO:tasks.workunit.client.1.smithi132.stdout:1/48: read d1/f2 [20265,68605] 0 2024-02-19T19:04:47.185 INFO:tasks.workunit.client.1.smithi132.stdout:1/49: read d1/ff [132339,78696] 0 2024-02-19T19:04:47.186 INFO:tasks.workunit.client.1.smithi132.stdout:5/109: fsync d8/d18/f1c 0 2024-02-19T19:04:47.187 INFO:tasks.workunit.client.1.smithi132.stdout:9/132: rename l10 to d12/l32 0 2024-02-19T19:04:47.188 INFO:tasks.workunit.client.1.smithi132.stdout:7/102: dwrite ff [4194304,4194304] 0 2024-02-19T19:04:47.193 INFO:tasks.workunit.client.1.smithi132.stdout:1/50: unlink d1/l7 0 2024-02-19T19:04:47.194 INFO:tasks.workunit.client.1.smithi132.stdout:7/103: dwrite f6 [0,4194304] 0 2024-02-19T19:04:47.196 INFO:tasks.workunit.client.1.smithi132.stdout:1/51: unlink d1/fe 0 2024-02-19T19:04:47.203 INFO:tasks.workunit.client.1.smithi132.stdout:1/52: dread d1/f5 [0,4194304] 0 2024-02-19T19:04:47.204 INFO:tasks.workunit.client.1.smithi132.stdout:7/104: dwrite d1b/d1e/f21 [0,4194304] 0 2024-02-19T19:04:47.215 INFO:tasks.workunit.client.1.smithi132.stdout:7/105: rename d1b/f1d to d1b/f25 0 2024-02-19T19:04:47.215 INFO:tasks.workunit.client.1.smithi132.stdout:7/106: read f10 [156353,110622] 0 2024-02-19T19:04:47.218 INFO:tasks.workunit.client.1.smithi132.stdout:9/133: unlink d12/l32 0 2024-02-19T19:04:47.222 INFO:tasks.workunit.client.1.smithi132.stdout:7/107: rename c19 to d1b/d1e/c26 0 2024-02-19T19:04:47.223 INFO:tasks.workunit.client.1.smithi132.stdout:1/53: rmdir d1 39 2024-02-19T19:04:47.223 INFO:tasks.workunit.client.1.smithi132.stdout:5/110: dwrite d8/ff [0,4194304] 0 2024-02-19T19:04:47.227 INFO:tasks.workunit.client.1.smithi132.stdout:7/108: write f10 [92058,126569] 0 2024-02-19T19:04:47.228 INFO:tasks.workunit.client.1.smithi132.stdout:5/111: rmdir d8 39 2024-02-19T19:04:47.229 INFO:tasks.workunit.client.1.smithi132.stdout:7/109: dread d1b/f25 [0,4194304] 0 2024-02-19T19:04:47.238 INFO:tasks.workunit.client.1.smithi132.stdout:9/134: creat d12/f33 x:0 0 0 2024-02-19T19:04:47.242 INFO:tasks.workunit.client.1.smithi132.stdout:5/112: mkdir d8/da/db/d21 0 2024-02-19T19:04:47.242 INFO:tasks.workunit.client.1.smithi132.stdout:5/113: chown d8/da/db 120 1 2024-02-19T19:04:47.246 INFO:tasks.workunit.client.1.smithi132.stdout:5/114: symlink d8/l22 0 2024-02-19T19:04:47.247 INFO:tasks.workunit.client.1.smithi132.stdout:7/110: link c17 d1b/d1e/c27 0 2024-02-19T19:04:47.249 INFO:tasks.workunit.client.1.smithi132.stdout:7/111: write f6 [5036933,5498] 0 2024-02-19T19:04:47.249 INFO:tasks.workunit.client.1.smithi132.stdout:9/135: dwrite d12/d1b/f1c [4194304,4194304] 0 2024-02-19T19:04:47.250 INFO:tasks.workunit.client.1.smithi132.stdout:7/112: write d1b/d1e/f21 [2142112,66347] 0 2024-02-19T19:04:47.253 INFO:tasks.workunit.client.1.smithi132.stdout:5/115: mkdir d8/d23 0 2024-02-19T19:04:47.253 INFO:tasks.workunit.client.1.smithi132.stdout:1/54: mknod d1/c12 0 2024-02-19T19:04:47.260 INFO:tasks.workunit.client.1.smithi132.stdout:7/113: dwrite f10 [0,4194304] 0 2024-02-19T19:04:47.263 INFO:tasks.workunit.client.1.smithi132.stdout:7/114: mkdir d1b/d1e/d28 0 2024-02-19T19:04:47.268 INFO:tasks.workunit.client.1.smithi132.stdout:7/115: dread f10 [0,4194304] 0 2024-02-19T19:04:47.270 INFO:tasks.workunit.client.1.smithi132.stdout:7/116: dread d1b/f25 [0,4194304] 0 2024-02-19T19:04:47.275 INFO:tasks.workunit.client.1.smithi132.stdout:7/117: getdents d1b/d1e/d28 0 2024-02-19T19:04:47.275 INFO:tasks.workunit.client.1.smithi132.stdout:7/118: dread - d1b/f1c zero size 2024-02-19T19:04:47.277 INFO:tasks.workunit.client.1.smithi132.stdout:7/119: fsync d1b/f25 0 2024-02-19T19:04:47.277 INFO:tasks.workunit.client.1.smithi132.stdout:7/120: fdatasync d1b/d1e/f21 0 2024-02-19T19:04:47.279 INFO:tasks.workunit.client.1.smithi132.stdout:1/55: rename d1/ff to d1/f13 0 2024-02-19T19:04:47.280 INFO:tasks.workunit.client.1.smithi132.stdout:9/136: truncate d12/d2a/f2c 2216106 0 2024-02-19T19:04:47.281 INFO:tasks.workunit.client.1.smithi132.stdout:7/121: fdatasync f10 0 2024-02-19T19:04:47.282 INFO:tasks.workunit.client.1.smithi132.stdout:7/122: read f10 [1156771,43125] 0 2024-02-19T19:04:47.282 INFO:tasks.workunit.client.1.smithi132.stdout:7/123: chown ff 893804 1 2024-02-19T19:04:47.283 INFO:tasks.workunit.client.1.smithi132.stdout:7/124: write d1b/f25 [1301818,75047] 0 2024-02-19T19:04:47.284 INFO:tasks.workunit.client.1.smithi132.stdout:7/125: chown f8 28637 1 2024-02-19T19:04:47.284 INFO:tasks.workunit.client.1.smithi132.stdout:7/126: stat d1b/f25 0 2024-02-19T19:04:47.287 INFO:tasks.workunit.client.1.smithi132.stdout:7/127: dread f6 [0,4194304] 0 2024-02-19T19:04:47.290 INFO:tasks.workunit.client.1.smithi132.stdout:4/49: sync 2024-02-19T19:04:47.290 INFO:tasks.workunit.client.1.smithi132.stdout:2/89: sync 2024-02-19T19:04:47.290 INFO:tasks.workunit.client.1.smithi132.stdout:6/72: sync 2024-02-19T19:04:47.298 INFO:tasks.workunit.client.1.smithi132.stdout:2/90: rename d0/d4/df/l23 to d0/d19/l27 0 2024-02-19T19:04:47.298 INFO:tasks.workunit.client.1.smithi132.stdout:9/137: getdents d12/d1b 0 2024-02-19T19:04:47.299 INFO:tasks.workunit.client.1.smithi132.stdout:2/91: chown d0/l1 50193 1 2024-02-19T19:04:47.302 INFO:tasks.workunit.client.1.smithi132.stdout:9/138: rename c0 to d12/d25/c34 0 2024-02-19T19:04:47.303 INFO:tasks.workunit.client.1.smithi132.stdout:1/56: rename d1/l11 to d1/l14 0 2024-02-19T19:04:47.303 INFO:tasks.workunit.client.1.smithi132.stdout:9/139: fsync d12/d25/f30 0 2024-02-19T19:04:47.306 INFO:tasks.workunit.client.1.smithi132.stdout:9/140: fsync f5 0 2024-02-19T19:04:47.308 INFO:tasks.workunit.client.1.smithi132.stdout:4/50: truncate d4/f8 803438 0 2024-02-19T19:04:47.310 INFO:tasks.workunit.client.1.smithi132.stdout:1/57: stat d1 0 2024-02-19T19:04:47.310 INFO:tasks.workunit.client.1.smithi132.stdout:1/58: chown d1 524939753 1 2024-02-19T19:04:47.311 INFO:tasks.workunit.client.1.smithi132.stdout:4/51: symlink d4/lb 0 2024-02-19T19:04:47.314 INFO:tasks.workunit.client.1.smithi132.stdout:9/141: link d12/f26 d12/d25/f35 0 2024-02-19T19:04:47.314 INFO:tasks.workunit.client.1.smithi132.stdout:9/142: stat d12/d25/c2f 0 2024-02-19T19:04:47.315 INFO:tasks.workunit.client.1.smithi132.stdout:4/52: mkdir d4/dc 0 2024-02-19T19:04:47.315 INFO:tasks.workunit.client.1.smithi132.stdout:4/53: chown d4/fa 30673 1 2024-02-19T19:04:47.316 INFO:tasks.workunit.client.1.smithi132.stdout:4/54: write d4/f7 [2548322,54588] 0 2024-02-19T19:04:47.320 INFO:tasks.workunit.client.1.smithi132.stdout:9/143: getdents d12 0 2024-02-19T19:04:47.322 INFO:tasks.workunit.client.1.smithi132.stdout:1/59: dwrite d1/f5 [4194304,4194304] 0 2024-02-19T19:04:47.324 INFO:tasks.workunit.client.1.smithi132.stdout:9/144: chown d12/d25/l31 459 1 2024-02-19T19:04:47.324 INFO:tasks.workunit.client.1.smithi132.stdout:4/55: unlink d4/f8 0 2024-02-19T19:04:47.329 INFO:tasks.workunit.client.1.smithi132.stdout:9/145: rmdir d12/d2a 39 2024-02-19T19:04:47.332 INFO:tasks.workunit.client.1.smithi132.stdout:1/60: fdatasync d1/f13 0 2024-02-19T19:04:47.337 INFO:tasks.workunit.client.1.smithi132.stdout:4/56: mknod d4/dc/cd 0 2024-02-19T19:04:47.343 INFO:tasks.workunit.client.1.smithi132.stdout:4/57: dwrite d4/f9 [0,4194304] 0 2024-02-19T19:04:47.343 INFO:tasks.workunit.client.1.smithi132.stdout:9/146: rename fd to d12/d1b/f36 0 2024-02-19T19:04:47.344 INFO:tasks.workunit.client.1.smithi132.stdout:4/58: truncate d4/f9 8779256 0 2024-02-19T19:04:47.354 INFO:tasks.workunit.client.1.smithi132.stdout:1/61: dwrite d1/fd [4194304,4194304] 0 2024-02-19T19:04:47.356 INFO:tasks.workunit.client.1.smithi132.stdout:9/147: dwrite d12/d25/f29 [0,4194304] 0 2024-02-19T19:04:47.356 INFO:tasks.workunit.client.1.smithi132.stdout:4/59: truncate d4/f9 8348963 0 2024-02-19T19:04:47.358 INFO:tasks.workunit.client.1.smithi132.stdout:9/148: write d12/d25/f2e [957548,42455] 0 2024-02-19T19:04:47.361 INFO:tasks.workunit.client.1.smithi132.stdout:4/60: creat d4/dc/fe x:0 0 0 2024-02-19T19:04:47.361 INFO:tasks.workunit.client.1.smithi132.stdout:4/61: readlink d4/lb 0 2024-02-19T19:04:47.366 INFO:tasks.workunit.client.1.smithi132.stdout:9/149: dread d12/d2a/f2b [0,4194304] 0 2024-02-19T19:04:47.367 INFO:tasks.workunit.client.1.smithi132.stdout:9/150: chown d12/f27 377740549 1 2024-02-19T19:04:47.367 INFO:tasks.workunit.client.1.smithi132.stdout:9/151: chown ce 43880505 1 2024-02-19T19:04:47.375 INFO:tasks.workunit.client.1.smithi132.stdout:9/152: link d12/d2a/f2b d12/d2a/f37 0 2024-02-19T19:04:47.378 INFO:tasks.workunit.client.1.smithi132.stdout:9/153: unlink d12/d1b/c18 0 2024-02-19T19:04:47.380 INFO:tasks.workunit.client.1.smithi132.stdout:4/62: write f3 [5050236,58487] 0 2024-02-19T19:04:47.382 INFO:tasks.workunit.client.1.smithi132.stdout:9/154: dread d12/d1b/f16 [0,4194304] 0 2024-02-19T19:04:47.386 INFO:tasks.workunit.client.1.smithi132.stdout:9/155: mknod d12/d25/c38 0 2024-02-19T19:04:47.388 INFO:tasks.workunit.client.1.smithi132.stdout:1/62: dwrite d1/f13 [0,4194304] 0 2024-02-19T19:04:47.388 INFO:tasks.workunit.client.1.smithi132.stdout:9/156: read d12/d25/f28 [157295,62206] 0 2024-02-19T19:04:47.389 INFO:tasks.workunit.client.1.smithi132.stdout:4/63: dread d4/f9 [4194304,4194304] 0 2024-02-19T19:04:47.389 INFO:tasks.workunit.client.1.smithi132.stdout:9/157: stat d12/d25/f29 0 2024-02-19T19:04:47.393 INFO:tasks.workunit.client.1.smithi132.stdout:1/63: chown d1/l14 263777292 1 2024-02-19T19:04:47.394 INFO:tasks.workunit.client.1.smithi132.stdout:1/64: chown d1/fd 433075792 1 2024-02-19T19:04:47.398 INFO:tasks.workunit.client.1.smithi132.stdout:1/65: rmdir d1 39 2024-02-19T19:04:47.403 INFO:tasks.workunit.client.1.smithi132.stdout:1/66: dread d1/f5 [0,4194304] 0 2024-02-19T19:04:47.409 INFO:tasks.workunit.client.1.smithi132.stdout:9/158: write d12/f26 [162028,127127] 0 2024-02-19T19:04:47.410 INFO:tasks.workunit.client.1.smithi132.stdout:1/67: dwrite d1/f13 [0,4194304] 0 2024-02-19T19:04:47.412 INFO:tasks.workunit.client.1.smithi132.stdout:9/159: symlink d12/d1b/l39 0 2024-02-19T19:04:47.414 INFO:tasks.workunit.client.1.smithi132.stdout:9/160: mkdir d12/d25/d3a 0 2024-02-19T19:04:47.417 INFO:tasks.workunit.client.1.smithi132.stdout:3/68: sync 2024-02-19T19:04:47.424 INFO:tasks.workunit.client.1.smithi132.stdout:1/68: rename d1/fd to d1/f15 0 2024-02-19T19:04:47.424 INFO:tasks.workunit.client.1.smithi132.stdout:9/161: creat d12/d1b/f3b x:0 0 0 2024-02-19T19:04:47.428 INFO:tasks.workunit.client.1.smithi132.stdout:9/162: unlink d12/f27 0 2024-02-19T19:04:47.431 INFO:tasks.workunit.client.1.smithi132.stdout:9/163: truncate d12/f23 837648 0 2024-02-19T19:04:47.432 INFO:tasks.workunit.client.1.smithi132.stdout:1/69: creat d1/f16 x:0 0 0 2024-02-19T19:04:47.442 INFO:tasks.workunit.client.1.smithi132.stdout:1/70: rename d1/f13 to d1/f17 0 2024-02-19T19:04:47.446 INFO:tasks.workunit.client.1.smithi132.stdout:9/164: truncate fa 2242226 0 2024-02-19T19:04:47.449 INFO:tasks.workunit.client.1.smithi132.stdout:9/165: dread d12/d2a/f2c [0,4194304] 0 2024-02-19T19:04:47.449 INFO:tasks.workunit.client.1.smithi132.stdout:1/71: creat d1/f18 x:0 0 0 2024-02-19T19:04:47.451 INFO:tasks.workunit.client.1.smithi132.stdout:1/72: write d1/f15 [5441927,69027] 0 2024-02-19T19:04:47.452 INFO:tasks.workunit.client.1.smithi132.stdout:9/166: symlink d12/d25/d3a/l3c 0 2024-02-19T19:04:47.454 INFO:tasks.workunit.client.1.smithi132.stdout:9/167: mknod d12/d1b/c3d 0 2024-02-19T19:04:47.456 INFO:tasks.workunit.client.1.smithi132.stdout:1/73: mkdir d1/d19 0 2024-02-19T19:04:47.458 INFO:tasks.workunit.client.1.smithi132.stdout:1/74: dread d1/f2 [0,4194304] 0 2024-02-19T19:04:47.458 INFO:tasks.workunit.client.1.smithi132.stdout:1/75: truncate d1/f17 5016641 0 2024-02-19T19:04:47.459 INFO:tasks.workunit.client.1.smithi132.stdout:9/168: mkdir d12/d3e 0 2024-02-19T19:04:47.459 INFO:tasks.workunit.client.1.smithi132.stdout:9/169: readlink d12/d1b/l14 0 2024-02-19T19:04:47.459 INFO:tasks.workunit.client.1.smithi132.stdout:1/76: write d1/f17 [2673605,93529] 0 2024-02-19T19:04:47.462 INFO:tasks.workunit.client.1.smithi132.stdout:1/77: write d1/f2 [611297,2398] 0 2024-02-19T19:04:47.464 INFO:tasks.workunit.client.1.smithi132.stdout:1/78: chown d1/l14 112328 1 2024-02-19T19:04:47.468 INFO:tasks.workunit.client.1.smithi132.stdout:1/79: symlink d1/d19/l1a 0 2024-02-19T19:04:47.472 INFO:tasks.workunit.client.1.smithi132.stdout:9/170: getdents d12/d25/d3a 0 2024-02-19T19:04:47.473 INFO:tasks.workunit.client.1.smithi132.stdout:1/80: dwrite d1/f2 [0,4194304] 0 2024-02-19T19:04:47.474 INFO:tasks.workunit.client.1.smithi132.stdout:1/81: read - d1/f16 zero size 2024-02-19T19:04:47.476 INFO:tasks.workunit.client.1.smithi132.stdout:9/171: creat d12/d1b/f3f x:0 0 0 2024-02-19T19:04:47.479 INFO:tasks.workunit.client.1.smithi132.stdout:9/172: mkdir d12/d25/d3a/d40 0 2024-02-19T19:04:47.480 INFO:tasks.workunit.client.1.smithi132.stdout:9/173: chown d12/d25 5 1 2024-02-19T19:04:47.483 INFO:tasks.workunit.client.1.smithi132.stdout:9/174: truncate d12/d2a/f37 3422385 0 2024-02-19T19:04:47.488 INFO:tasks.workunit.client.1.smithi132.stdout:1/82: getdents d1/d19 0 2024-02-19T19:04:47.496 INFO:tasks.workunit.client.1.smithi132.stdout:1/83: creat d1/f1b x:0 0 0 2024-02-19T19:04:47.497 INFO:tasks.workunit.client.1.smithi132.stdout:9/175: write d12/d2a/f37 [2973115,67419] 0 2024-02-19T19:04:47.501 INFO:tasks.workunit.client.1.smithi132.stdout:1/84: mknod d1/c1c 0 2024-02-19T19:04:47.502 INFO:tasks.workunit.client.1.smithi132.stdout:9/176: symlink d12/l41 0 2024-02-19T19:04:47.506 INFO:tasks.workunit.client.1.smithi132.stdout:1/85: dwrite d1/f15 [0,4194304] 0 2024-02-19T19:04:47.508 INFO:tasks.workunit.client.1.smithi132.stdout:9/177: dread d12/d1b/f1c [0,4194304] 0 2024-02-19T19:04:47.511 INFO:tasks.workunit.client.1.smithi132.stdout:9/178: dread d12/d25/f35 [0,4194304] 0 2024-02-19T19:04:47.512 INFO:tasks.workunit.client.1.smithi132.stdout:9/179: dread - d12/d1b/f3f zero size 2024-02-19T19:04:47.514 INFO:tasks.workunit.client.1.smithi132.stdout:1/86: dwrite d1/f16 [0,4194304] 0 2024-02-19T19:04:47.523 INFO:tasks.workunit.client.1.smithi132.stdout:9/180: mknod d12/d25/d3a/d40/c42 0 2024-02-19T19:04:47.525 INFO:tasks.workunit.client.1.smithi132.stdout:9/181: dread d12/d2a/f2b [0,4194304] 0 2024-02-19T19:04:47.531 INFO:tasks.workunit.client.1.smithi132.stdout:9/182: dwrite d12/f23 [0,4194304] 0 2024-02-19T19:04:47.532 INFO:tasks.workunit.client.1.smithi132.stdout:8/115: sync 2024-02-19T19:04:47.532 INFO:tasks.workunit.client.1.smithi132.stdout:0/63: sync 2024-02-19T19:04:47.536 INFO:tasks.workunit.client.1.smithi132.stdout:9/183: truncate d12/f33 1043432 0 2024-02-19T19:04:47.537 INFO:tasks.workunit.client.1.smithi132.stdout:8/116: rmdir d4/dc/d1b/d20 39 2024-02-19T19:04:47.541 INFO:tasks.workunit.client.1.smithi132.stdout:1/87: truncate d1/f5 739173 0 2024-02-19T19:04:47.543 INFO:tasks.workunit.client.1.smithi132.stdout:8/117: dread d4/f16 [0,4194304] 0 2024-02-19T19:04:47.544 INFO:tasks.workunit.client.1.smithi132.stdout:0/64: dwrite d2/f12 [0,4194304] 0 2024-02-19T19:04:47.545 INFO:tasks.workunit.client.1.smithi132.stdout:1/88: creat d1/f1d x:0 0 0 2024-02-19T19:04:47.547 INFO:tasks.workunit.client.1.smithi132.stdout:1/89: write d1/f16 [4061555,91123] 0 2024-02-19T19:04:47.550 INFO:tasks.workunit.client.1.smithi132.stdout:9/184: dwrite d12/d1b/f36 [0,4194304] 0 2024-02-19T19:04:47.551 INFO:tasks.workunit.client.1.smithi132.stdout:0/65: unlink d2/dc/c13 0 2024-02-19T19:04:47.554 INFO:tasks.workunit.client.1.smithi132.stdout:0/66: mknod d2/dc/c18 0 2024-02-19T19:04:47.555 INFO:tasks.workunit.client.1.smithi132.stdout:9/185: dread d12/d25/f28 [0,4194304] 0 2024-02-19T19:04:47.556 INFO:tasks.workunit.client.1.smithi132.stdout:8/118: dwrite d4/f24 [0,4194304] 0 2024-02-19T19:04:47.556 INFO:tasks.workunit.client.1.smithi132.stdout:1/90: mkdir d1/d1e 0 2024-02-19T19:04:47.560 INFO:tasks.workunit.client.1.smithi132.stdout:8/119: fdatasync d4/dc/d1b/d20/d23/f28 0 2024-02-19T19:04:47.561 INFO:tasks.workunit.client.1.smithi132.stdout:0/67: dread d2/f12 [0,4194304] 0 2024-02-19T19:04:47.561 INFO:tasks.workunit.client.1.smithi132.stdout:1/91: mknod d1/d19/c1f 0 2024-02-19T19:04:47.561 INFO:tasks.workunit.client.1.smithi132.stdout:8/120: chown d4/dc/d1b/d20/d23 16 1 2024-02-19T19:04:47.562 INFO:tasks.workunit.client.1.smithi132.stdout:1/92: write d1/f15 [690978,73972] 0 2024-02-19T19:04:47.562 INFO:tasks.workunit.client.1.smithi132.stdout:8/121: fsync d4/f19 0 2024-02-19T19:04:47.572 INFO:tasks.workunit.client.1.smithi132.stdout:0/68: creat d2/f19 x:0 0 0 2024-02-19T19:04:47.573 INFO:tasks.workunit.client.1.smithi132.stdout:1/93: chown d1/lb 21 1 2024-02-19T19:04:47.574 INFO:tasks.workunit.client.1.smithi132.stdout:8/122: rename d4/f8 to d4/dc/d1b/d20/d23/f2b 0 2024-02-19T19:04:47.575 INFO:tasks.workunit.client.1.smithi132.stdout:1/94: write d1/f17 [2010782,13653] 0 2024-02-19T19:04:47.580 INFO:tasks.workunit.client.1.smithi132.stdout:0/69: creat d2/f1a x:0 0 0 2024-02-19T19:04:47.580 INFO:tasks.workunit.client.1.smithi132.stdout:8/123: creat d4/dc/f2c x:0 0 0 2024-02-19T19:04:47.581 INFO:tasks.workunit.client.1.smithi132.stdout:1/95: mknod d1/d19/c20 0 2024-02-19T19:04:47.582 INFO:tasks.workunit.client.1.smithi132.stdout:1/96: write d1/f18 [462167,78329] 0 2024-02-19T19:04:47.582 INFO:tasks.workunit.client.1.smithi132.stdout:0/70: mknod d2/c1b 0 2024-02-19T19:04:47.585 INFO:tasks.workunit.client.1.smithi132.stdout:8/124: mkdir d4/d2d 0 2024-02-19T19:04:47.589 INFO:tasks.workunit.client.1.smithi132.stdout:0/71: dwrite d2/dc/fe [0,4194304] 0 2024-02-19T19:04:47.593 INFO:tasks.workunit.client.1.smithi132.stdout:8/125: unlink d4/f9 0 2024-02-19T19:04:47.593 INFO:tasks.workunit.client.1.smithi132.stdout:8/126: chown d4/c17 1642841 1 2024-02-19T19:04:47.610 INFO:tasks.workunit.client.1.smithi132.stdout:8/127: chown d4/dc/d1b/d20/d23/f2b 2581 1 2024-02-19T19:04:47.611 INFO:tasks.workunit.client.1.smithi132.stdout:8/128: chown d4/la 0 1 2024-02-19T19:04:47.611 INFO:tasks.workunit.client.1.smithi132.stdout:8/129: dread - d4/dc/f2c zero size 2024-02-19T19:04:47.612 INFO:tasks.workunit.client.1.smithi132.stdout:1/97: getdents d1/d19 0 2024-02-19T19:04:47.614 INFO:tasks.workunit.client.1.smithi132.stdout:8/130: creat d4/f2e x:0 0 0 2024-02-19T19:04:47.614 INFO:tasks.workunit.client.1.smithi132.stdout:8/131: chown d4/dc/d1b/d1d/f21 0 1 2024-02-19T19:04:47.615 INFO:tasks.workunit.client.1.smithi132.stdout:8/132: chown d4/dc/f22 723828527 1 2024-02-19T19:04:47.616 INFO:tasks.workunit.client.1.smithi132.stdout:1/98: creat d1/f21 x:0 0 0 2024-02-19T19:04:47.616 INFO:tasks.workunit.client.1.smithi132.stdout:0/72: write d2/fb [811360,70236] 0 2024-02-19T19:04:47.617 INFO:tasks.workunit.client.1.smithi132.stdout:8/133: mknod d4/dc/d1b/d20/d23/c2f 0 2024-02-19T19:04:47.617 INFO:tasks.workunit.client.1.smithi132.stdout:1/99: dread - d1/f21 zero size 2024-02-19T19:04:47.619 INFO:tasks.workunit.client.1.smithi132.stdout:0/73: symlink d2/l1c 0 2024-02-19T19:04:47.619 INFO:tasks.workunit.client.1.smithi132.stdout:0/74: readlink d2/l1c 0 2024-02-19T19:04:47.619 INFO:tasks.workunit.client.1.smithi132.stdout:0/75: stat d2/l8 0 2024-02-19T19:04:47.620 INFO:tasks.workunit.client.1.smithi132.stdout:0/76: readlink d2/dc/l10 0 2024-02-19T19:04:47.620 INFO:tasks.workunit.client.1.smithi132.stdout:1/100: creat d1/d19/f22 x:0 0 0 2024-02-19T19:04:47.620 INFO:tasks.workunit.client.1.smithi132.stdout:0/77: read - d2/f1a zero size 2024-02-19T19:04:47.620 INFO:tasks.workunit.client.1.smithi132.stdout:1/101: fsync d1/f17 0 2024-02-19T19:04:47.621 INFO:tasks.workunit.client.1.smithi132.stdout:8/134: symlink d4/dc/d1b/d20/d23/l30 0 2024-02-19T19:04:47.621 INFO:tasks.workunit.client.1.smithi132.stdout:1/102: dread - d1/d19/f22 zero size 2024-02-19T19:04:47.623 INFO:tasks.workunit.client.1.smithi132.stdout:5/116: sync 2024-02-19T19:04:47.623 INFO:tasks.workunit.client.1.smithi132.stdout:7/128: sync 2024-02-19T19:04:47.624 INFO:tasks.workunit.client.1.smithi132.stdout:5/117: read f6 [4129287,10003] 0 2024-02-19T19:04:47.624 INFO:tasks.workunit.client.1.smithi132.stdout:8/135: rename d4/dc/d1b/f2a to d4/d2d/f31 0 2024-02-19T19:04:47.626 INFO:tasks.workunit.client.1.smithi132.stdout:0/78: dread d2/dc/fe [0,4194304] 0 2024-02-19T19:04:47.627 INFO:tasks.workunit.client.1.smithi132.stdout:7/129: creat d1b/d1e/d28/f29 x:0 0 0 2024-02-19T19:04:47.628 INFO:tasks.workunit.client.1.smithi132.stdout:5/118: unlink d8/d18/f1f 0 2024-02-19T19:04:47.629 INFO:tasks.workunit.client.1.smithi132.stdout:1/103: dwrite d1/f15 [4194304,4194304] 0 2024-02-19T19:04:47.629 INFO:tasks.workunit.client.1.smithi132.stdout:7/130: write d1b/d1e/f22 [1162008,64450] 0 2024-02-19T19:04:47.630 INFO:tasks.workunit.client.1.smithi132.stdout:7/131: chown c14 26058815 1 2024-02-19T19:04:47.631 INFO:tasks.workunit.client.1.smithi132.stdout:8/136: creat d4/dc/d1b/f32 x:0 0 0 2024-02-19T19:04:47.632 INFO:tasks.workunit.client.1.smithi132.stdout:8/137: readlink d4/dc/d1b/d20/d23/l30 0 2024-02-19T19:04:47.635 INFO:tasks.workunit.client.1.smithi132.stdout:5/119: creat d8/da/f24 x:0 0 0 2024-02-19T19:04:47.635 INFO:tasks.workunit.client.1.smithi132.stdout:8/138: fsync d4/f2e 0 2024-02-19T19:04:47.637 INFO:tasks.workunit.client.1.smithi132.stdout:7/132: dwrite d1b/d1e/f22 [0,4194304] 0 2024-02-19T19:04:47.650 INFO:tasks.workunit.client.1.smithi132.stdout:8/139: mkdir d4/dc/d1b/d33 0 2024-02-19T19:04:47.651 INFO:tasks.workunit.client.1.smithi132.stdout:8/140: chown d4/f27 43 1 2024-02-19T19:04:47.652 INFO:tasks.workunit.client.1.smithi132.stdout:5/120: creat d8/da/db/d21/f25 x:0 0 0 2024-02-19T19:04:47.653 INFO:tasks.workunit.client.1.smithi132.stdout:0/79: truncate d2/f12 2451300 0 2024-02-19T19:04:47.653 INFO:tasks.workunit.client.1.smithi132.stdout:0/80: write d2/f19 [496808,52177] 0 2024-02-19T19:04:47.654 INFO:tasks.workunit.client.1.smithi132.stdout:0/81: write d2/f19 [96814,50785] 0 2024-02-19T19:04:47.661 INFO:tasks.workunit.client.1.smithi132.stdout:0/82: mkdir d2/d1d 0 2024-02-19T19:04:47.662 INFO:tasks.workunit.client.1.smithi132.stdout:5/121: getdents d8/d1a/d1e 0 2024-02-19T19:04:47.662 INFO:tasks.workunit.client.1.smithi132.stdout:7/133: link ce d1b/d1e/d28/c2a 0 2024-02-19T19:04:47.662 INFO:tasks.workunit.client.1.smithi132.stdout:7/134: chown f6 3433 1 2024-02-19T19:04:47.663 INFO:tasks.workunit.client.1.smithi132.stdout:5/122: write d8/d18/f1c [385281,101354] 0 2024-02-19T19:04:47.666 INFO:tasks.workunit.client.1.smithi132.stdout:0/83: dread d2/dc/fe [0,4194304] 0 2024-02-19T19:04:47.668 INFO:tasks.workunit.client.1.smithi132.stdout:7/135: truncate f8 776129 0 2024-02-19T19:04:47.668 INFO:tasks.workunit.client.1.smithi132.stdout:7/136: stat d1b/f1c 0 2024-02-19T19:04:47.669 INFO:tasks.workunit.client.1.smithi132.stdout:5/123: creat d8/d1a/f26 x:0 0 0 2024-02-19T19:04:47.672 INFO:tasks.workunit.client.1.smithi132.stdout:8/141: dwrite d4/dc/d1b/d20/d23/f2b [0,4194304] 0 2024-02-19T19:04:47.673 INFO:tasks.workunit.client.1.smithi132.stdout:8/142: stat d4/dc/d1b/d20/d23/c2f 0 2024-02-19T19:04:47.675 INFO:tasks.workunit.client.1.smithi132.stdout:0/84: stat d2/f12 0 2024-02-19T19:04:47.677 INFO:tasks.workunit.client.1.smithi132.stdout:5/124: dread d8/fe [0,4194304] 0 2024-02-19T19:04:47.678 INFO:tasks.workunit.client.1.smithi132.stdout:5/125: truncate d8/da/f24 536828 0 2024-02-19T19:04:47.682 INFO:tasks.workunit.client.1.smithi132.stdout:8/143: creat d4/dc/d1b/d33/f34 x:0 0 0 2024-02-19T19:04:47.686 INFO:tasks.workunit.client.1.smithi132.stdout:0/85: link d2/l16 d2/dc/df/d15/l1e 0 2024-02-19T19:04:47.689 INFO:tasks.workunit.client.1.smithi132.stdout:8/144: dwrite f2 [4194304,4194304] 0 2024-02-19T19:04:47.693 INFO:tasks.workunit.client.1.smithi132.stdout:0/86: rename d2/dc/df/d15/l17 to d2/dc/df/l1f 0 2024-02-19T19:04:47.693 INFO:tasks.workunit.client.1.smithi132.stdout:8/145: symlink d4/l35 0 2024-02-19T19:04:47.695 INFO:tasks.workunit.client.1.smithi132.stdout:0/87: creat d2/dc/f20 x:0 0 0 2024-02-19T19:04:47.696 INFO:tasks.workunit.client.1.smithi132.stdout:7/137: dwrite f8 [0,4194304] 0 2024-02-19T19:04:47.696 INFO:tasks.workunit.client.1.smithi132.stdout:8/146: write d4/f29 [364931,103278] 0 2024-02-19T19:04:47.696 INFO:tasks.workunit.client.1.smithi132.stdout:0/88: stat d2/c11 0 2024-02-19T19:04:47.698 INFO:tasks.workunit.client.1.smithi132.stdout:0/89: write d2/f1a [372889,120837] 0 2024-02-19T19:04:47.698 INFO:tasks.workunit.client.1.smithi132.stdout:7/138: write d1b/d1e/f21 [2424688,57848] 0 2024-02-19T19:04:47.699 INFO:tasks.workunit.client.1.smithi132.stdout:8/147: write d4/dc/d1b/d33/f34 [21011,34731] 0 2024-02-19T19:04:47.703 INFO:tasks.workunit.client.1.smithi132.stdout:7/139: readlink l16 0 2024-02-19T19:04:47.704 INFO:tasks.workunit.client.1.smithi132.stdout:0/90: write d2/dc/fe [4054854,2121] 0 2024-02-19T19:04:47.705 INFO:tasks.workunit.client.1.smithi132.stdout:0/91: write d2/dc/fe [3860936,58716] 0 2024-02-19T19:04:47.705 INFO:tasks.workunit.client.1.smithi132.stdout:0/92: fsync d2/f19 0 2024-02-19T19:04:47.707 INFO:tasks.workunit.client.1.smithi132.stdout:7/140: mkdir d1b/d1e/d2b 0 2024-02-19T19:04:47.709 INFO:tasks.workunit.client.1.smithi132.stdout:0/93: dread d2/f1a [0,4194304] 0 2024-02-19T19:04:47.712 INFO:tasks.workunit.client.1.smithi132.stdout:0/94: dread d2/f19 [0,4194304] 0 2024-02-19T19:04:47.714 INFO:tasks.workunit.client.1.smithi132.stdout:0/95: mkdir d2/d21 0 2024-02-19T19:04:47.717 INFO:tasks.workunit.client.1.smithi132.stdout:7/141: write f10 [1603200,37028] 0 2024-02-19T19:04:47.721 INFO:tasks.workunit.client.1.smithi132.stdout:7/142: creat d1b/d1e/d28/f2c x:0 0 0 2024-02-19T19:04:47.721 INFO:tasks.workunit.client.1.smithi132.stdout:0/96: dwrite d2/dc/f20 [0,4194304] 0 2024-02-19T19:04:47.722 INFO:tasks.workunit.client.1.smithi132.stdout:0/97: fdatasync d2/dc/fe 0 2024-02-19T19:04:47.723 INFO:tasks.workunit.client.1.smithi132.stdout:2/92: sync 2024-02-19T19:04:47.723 INFO:tasks.workunit.client.1.smithi132.stdout:6/73: sync 2024-02-19T19:04:47.723 INFO:tasks.workunit.client.1.smithi132.stdout:3/69: sync 2024-02-19T19:04:47.723 INFO:tasks.workunit.client.1.smithi132.stdout:4/64: sync 2024-02-19T19:04:47.725 INFO:tasks.workunit.client.1.smithi132.stdout:7/143: symlink d1b/d1e/l2d 0 2024-02-19T19:04:47.727 INFO:tasks.workunit.client.1.smithi132.stdout:3/70: creat d8/f10 x:0 0 0 2024-02-19T19:04:47.728 INFO:tasks.workunit.client.1.smithi132.stdout:3/71: write d8/fd [137409,36756] 0 2024-02-19T19:04:47.730 INFO:tasks.workunit.client.1.smithi132.stdout:7/144: dread d1b/d1e/f21 [0,4194304] 0 2024-02-19T19:04:47.731 INFO:tasks.workunit.client.1.smithi132.stdout:4/65: creat d4/dc/ff x:0 0 0 2024-02-19T19:04:47.732 INFO:tasks.workunit.client.1.smithi132.stdout:3/72: dread f0 [0,4194304] 0 2024-02-19T19:04:47.732 INFO:tasks.workunit.client.1.smithi132.stdout:2/93: fdatasync d0/d4/d11/f15 0 2024-02-19T19:04:47.733 INFO:tasks.workunit.client.1.smithi132.stdout:2/94: fsync d0/d4/fb 0 2024-02-19T19:04:47.736 INFO:tasks.workunit.client.1.smithi132.stdout:7/145: mknod d1b/d1e/c2e 0 2024-02-19T19:04:47.741 INFO:tasks.workunit.client.1.smithi132.stdout:6/74: rename d0/c11 to d0/c1c 0 2024-02-19T19:04:47.742 INFO:tasks.workunit.client.1.smithi132.stdout:4/66: dwrite d4/f9 [8388608,4194304] 0 2024-02-19T19:04:47.743 INFO:tasks.workunit.client.1.smithi132.stdout:4/67: truncate f3 13140629 0 2024-02-19T19:04:47.743 INFO:tasks.workunit.client.1.smithi132.stdout:3/73: mknod d8/c11 0 2024-02-19T19:04:47.743 INFO:tasks.workunit.client.1.smithi132.stdout:4/68: dread - d4/fa zero size 2024-02-19T19:04:47.744 INFO:tasks.workunit.client.1.smithi132.stdout:3/74: chown d8 276487 1 2024-02-19T19:04:47.745 INFO:tasks.workunit.client.1.smithi132.stdout:4/69: write d4/f7 [4973321,100675] 0 2024-02-19T19:04:47.745 INFO:tasks.workunit.client.1.smithi132.stdout:2/95: readlink d0/d19/l27 0 2024-02-19T19:04:47.746 INFO:tasks.workunit.client.1.smithi132.stdout:4/70: write d4/fa [332665,49318] 0 2024-02-19T19:04:47.747 INFO:tasks.workunit.client.1.smithi132.stdout:0/98: truncate d2/dc/f20 2433639 0 2024-02-19T19:04:47.747 INFO:tasks.workunit.client.1.smithi132.stdout:4/71: fdatasync d4/dc/ff 0 2024-02-19T19:04:47.749 INFO:tasks.workunit.client.1.smithi132.stdout:7/146: dread d1b/d1e/f21 [0,4194304] 0 2024-02-19T19:04:47.750 INFO:tasks.workunit.client.1.smithi132.stdout:3/75: creat d8/f12 x:0 0 0 2024-02-19T19:04:47.751 INFO:tasks.workunit.client.1.smithi132.stdout:2/96: mknod d0/d4/df/c28 0 2024-02-19T19:04:47.751 INFO:tasks.workunit.client.1.smithi132.stdout:4/72: dread f3 [8388608,4194304] 0 2024-02-19T19:04:47.752 INFO:tasks.workunit.client.1.smithi132.stdout:6/75: creat d0/db/d16/f1d x:0 0 0 2024-02-19T19:04:47.753 INFO:tasks.workunit.client.1.smithi132.stdout:6/76: dread - d0/db/d16/f1d zero size 2024-02-19T19:04:47.757 INFO:tasks.workunit.client.1.smithi132.stdout:4/73: dwrite f3 [0,4194304] 0 2024-02-19T19:04:47.779 INFO:tasks.workunit.client.1.smithi132.stdout:2/97: mknod d0/d19/d1e/c29 0 2024-02-19T19:04:47.779 INFO:tasks.workunit.client.1.smithi132.stdout:4/74: fsync d4/dc/ff 0 2024-02-19T19:04:47.779 INFO:tasks.workunit.client.1.smithi132.stdout:6/77: mkdir d0/d1e 0 2024-02-19T19:04:47.780 INFO:tasks.workunit.client.1.smithi132.stdout:7/147: link f6 d1b/d1e/d2b/f2f 0 2024-02-19T19:04:47.780 INFO:tasks.workunit.client.1.smithi132.stdout:0/99: write d2/dc/f20 [2589394,12600] 0 2024-02-19T19:04:47.782 INFO:tasks.workunit.client.1.smithi132.stdout:7/148: creat d1b/d1e/f30 x:0 0 0 2024-02-19T19:04:47.783 INFO:tasks.workunit.client.1.smithi132.stdout:0/100: chown d2/dc/df/d15/l1e 149039 1 2024-02-19T19:04:47.784 INFO:tasks.workunit.client.1.smithi132.stdout:0/101: chown d2/fb 2216 1 2024-02-19T19:04:47.784 INFO:tasks.workunit.client.1.smithi132.stdout:6/78: mknod d0/c1f 0 2024-02-19T19:04:47.785 INFO:tasks.workunit.client.1.smithi132.stdout:6/79: dread - d0/d1/f8 zero size 2024-02-19T19:04:47.785 INFO:tasks.workunit.client.1.smithi132.stdout:2/98: mknod d0/d4/d11/c2a 0 2024-02-19T19:04:47.786 INFO:tasks.workunit.client.1.smithi132.stdout:2/99: fsync d0/f5 0 2024-02-19T19:04:47.787 INFO:tasks.workunit.client.1.smithi132.stdout:7/149: readlink lc 0 2024-02-19T19:04:47.787 INFO:tasks.workunit.client.1.smithi132.stdout:0/102: dread d2/dc/f20 [0,4194304] 0 2024-02-19T19:04:47.790 INFO:tasks.workunit.client.1.smithi132.stdout:0/103: stat d2/la 0 2024-02-19T19:04:47.791 INFO:tasks.workunit.client.1.smithi132.stdout:0/104: chown d2/c11 28393036 1 2024-02-19T19:04:47.791 INFO:tasks.workunit.client.1.smithi132.stdout:2/100: dread d0/d4/fb [0,4194304] 0 2024-02-19T19:04:47.792 INFO:tasks.workunit.client.1.smithi132.stdout:7/150: creat d1b/d1e/d28/f31 x:0 0 0 2024-02-19T19:04:47.792 INFO:tasks.workunit.client.1.smithi132.stdout:7/151: chown d1b 667 1 2024-02-19T19:04:47.792 INFO:tasks.workunit.client.1.smithi132.stdout:4/75: getdents d4/dc 0 2024-02-19T19:04:47.794 INFO:tasks.workunit.client.1.smithi132.stdout:4/76: dread - d4/dc/ff zero size 2024-02-19T19:04:47.798 INFO:tasks.workunit.client.1.smithi132.stdout:4/77: rmdir d4/dc 39 2024-02-19T19:04:47.800 INFO:tasks.workunit.client.1.smithi132.stdout:2/101: dwrite d0/d4/f22 [0,4194304] 0 2024-02-19T19:04:47.800 INFO:tasks.workunit.client.1.smithi132.stdout:7/152: dwrite f8 [0,4194304] 0 2024-02-19T19:04:47.801 INFO:tasks.workunit.client.1.smithi132.stdout:0/105: getdents d2/dc/df 0 2024-02-19T19:04:47.802 INFO:tasks.workunit.client.1.smithi132.stdout:2/102: write d0/d4/df/f21 [1301,24221] 0 2024-02-19T19:04:47.802 INFO:tasks.workunit.client.1.smithi132.stdout:7/153: chown d1b/d1e/d28/f29 961 1 2024-02-19T19:04:47.802 INFO:tasks.workunit.client.1.smithi132.stdout:4/78: creat d4/f10 x:0 0 0 2024-02-19T19:04:47.804 INFO:tasks.workunit.client.1.smithi132.stdout:7/154: dread - d1b/d1e/d28/f31 zero size 2024-02-19T19:04:47.809 INFO:tasks.workunit.client.1.smithi132.stdout:0/106: creat d2/dc/df/d15/f22 x:0 0 0 2024-02-19T19:04:47.810 INFO:tasks.workunit.client.1.smithi132.stdout:2/103: mknod d0/d4/c2b 0 2024-02-19T19:04:47.811 INFO:tasks.workunit.client.1.smithi132.stdout:4/79: symlink d4/dc/l11 0 2024-02-19T19:04:47.812 INFO:tasks.workunit.client.1.smithi132.stdout:4/80: stat d4/f9 0 2024-02-19T19:04:47.812 INFO:tasks.workunit.client.1.smithi132.stdout:2/104: read - d0/d4/f12 zero size 2024-02-19T19:04:47.814 INFO:tasks.workunit.client.1.smithi132.stdout:2/105: readlink d0/le 0 2024-02-19T19:04:47.814 INFO:tasks.workunit.client.1.smithi132.stdout:0/107: symlink d2/dc/df/l23 0 2024-02-19T19:04:47.814 INFO:tasks.workunit.client.1.smithi132.stdout:7/155: dwrite d1b/f25 [0,4194304] 0 2024-02-19T19:04:47.815 INFO:tasks.workunit.client.1.smithi132.stdout:4/81: rmdir d4/dc 39 2024-02-19T19:04:47.822 INFO:tasks.workunit.client.1.smithi132.stdout:7/156: dwrite d1b/f1c [0,4194304] 0 2024-02-19T19:04:47.824 INFO:tasks.workunit.client.1.smithi132.stdout:9/186: sync 2024-02-19T19:04:47.827 INFO:tasks.workunit.client.1.smithi132.stdout:0/108: rename d2/fb to d2/dc/df/d15/f24 0 2024-02-19T19:04:47.828 INFO:tasks.workunit.client.1.smithi132.stdout:4/82: mkdir d4/d12 0 2024-02-19T19:04:47.828 INFO:tasks.workunit.client.1.smithi132.stdout:7/157: dwrite f10 [0,4194304] 0 2024-02-19T19:04:47.835 INFO:tasks.workunit.client.1.smithi132.stdout:7/158: dwrite d1b/d1e/d28/f31 [0,4194304] 0 2024-02-19T19:04:47.839 INFO:tasks.workunit.client.1.smithi132.stdout:9/187: creat d12/f43 x:0 0 0 2024-02-19T19:04:47.839 INFO:tasks.workunit.client.1.smithi132.stdout:7/159: dread f8 [0,4194304] 0 2024-02-19T19:04:47.852 INFO:tasks.workunit.client.1.smithi132.stdout:7/160: mkdir d1b/d32 0 2024-02-19T19:04:47.855 INFO:tasks.workunit.client.1.smithi132.stdout:0/109: rmdir d2/dc 39 2024-02-19T19:04:47.857 INFO:tasks.workunit.client.1.smithi132.stdout:9/188: creat d12/d25/d3a/d40/f44 x:0 0 0 2024-02-19T19:04:47.858 INFO:tasks.workunit.client.1.smithi132.stdout:2/106: dwrite d0/d4/d11/f15 [0,4194304] 0 2024-02-19T19:04:47.858 INFO:tasks.workunit.client.1.smithi132.stdout:4/83: link d4/lb d4/l13 0 2024-02-19T19:04:47.860 INFO:tasks.workunit.client.1.smithi132.stdout:2/107: truncate d0/d4/df/f21 608870 0 2024-02-19T19:04:47.867 INFO:tasks.workunit.client.1.smithi132.stdout:9/189: fsync d12/d2a/f2c 0 2024-02-19T19:04:47.869 INFO:tasks.workunit.client.1.smithi132.stdout:2/108: rmdir d0 39 2024-02-19T19:04:47.872 INFO:tasks.workunit.client.1.smithi132.stdout:4/84: mkdir d4/dc/d14 0 2024-02-19T19:04:47.873 INFO:tasks.workunit.client.1.smithi132.stdout:9/190: chown d12/d25/c34 19816983 1 2024-02-19T19:04:47.874 INFO:tasks.workunit.client.1.smithi132.stdout:9/191: dread d12/d1b/f16 [0,4194304] 0 2024-02-19T19:04:47.876 INFO:tasks.workunit.client.1.smithi132.stdout:0/110: dread d2/dc/f20 [0,4194304] 0 2024-02-19T19:04:47.876 INFO:tasks.workunit.client.1.smithi132.stdout:0/111: write d2/dc/fe [5032937,55861] 0 2024-02-19T19:04:47.877 INFO:tasks.workunit.client.1.smithi132.stdout:4/85: unlink f3 0 2024-02-19T19:04:47.881 INFO:tasks.workunit.client.1.smithi132.stdout:0/112: dread d2/f6 [0,4194304] 0 2024-02-19T19:04:47.881 INFO:tasks.workunit.client.1.smithi132.stdout:2/109: creat d0/d19/f2c x:0 0 0 2024-02-19T19:04:47.883 INFO:tasks.workunit.client.1.smithi132.stdout:5/126: sync 2024-02-19T19:04:47.883 INFO:tasks.workunit.client.1.smithi132.stdout:8/148: sync 2024-02-19T19:04:47.883 INFO:tasks.workunit.client.1.smithi132.stdout:1/104: sync 2024-02-19T19:04:47.884 INFO:tasks.workunit.client.1.smithi132.stdout:1/105: chown d1/f1d 2 1 2024-02-19T19:04:47.885 INFO:tasks.workunit.client.1.smithi132.stdout:1/106: dread - d1/f21 zero size 2024-02-19T19:04:47.887 INFO:tasks.workunit.client.1.smithi132.stdout:4/86: mknod d4/d12/c15 0 2024-02-19T19:04:47.890 INFO:tasks.workunit.client.1.smithi132.stdout:7/161: dwrite f6 [0,4194304] 0 2024-02-19T19:04:47.891 INFO:tasks.workunit.client.1.smithi132.stdout:8/149: dread d4/dc/d1b/d20/d23/f2b [4194304,4194304] 0 2024-02-19T19:04:47.891 INFO:tasks.workunit.client.1.smithi132.stdout:5/127: rename d8/da/db/d21 to d8/d18/d27 0 2024-02-19T19:04:47.892 INFO:tasks.workunit.client.1.smithi132.stdout:7/162: write d1b/d1e/d28/f29 [931712,128686] 0 2024-02-19T19:04:47.893 INFO:tasks.workunit.client.1.smithi132.stdout:0/113: mkdir d2/d21/d25 0 2024-02-19T19:04:47.893 INFO:tasks.workunit.client.1.smithi132.stdout:7/163: stat d1b/d1e/d28 0 2024-02-19T19:04:47.896 INFO:tasks.workunit.client.1.smithi132.stdout:2/110: creat d0/d24/f2d x:0 0 0 2024-02-19T19:04:47.897 INFO:tasks.workunit.client.1.smithi132.stdout:9/192: write d12/f1a [1198702,75995] 0 2024-02-19T19:04:47.899 INFO:tasks.workunit.client.1.smithi132.stdout:5/128: dwrite d8/da/db/f20 [0,4194304] 0 2024-02-19T19:04:47.899 INFO:tasks.workunit.client.1.smithi132.stdout:2/111: write d0/d4/d11/f20 [557050,49466] 0 2024-02-19T19:04:47.899 INFO:tasks.workunit.client.1.smithi132.stdout:9/193: write d12/d1b/f36 [449238,72564] 0 2024-02-19T19:04:47.901 INFO:tasks.workunit.client.1.smithi132.stdout:2/112: chown d0/d4/df/f21 377 1 2024-02-19T19:04:47.901 INFO:tasks.workunit.client.1.smithi132.stdout:2/113: readlink d0/le 0 2024-02-19T19:04:47.901 INFO:tasks.workunit.client.1.smithi132.stdout:4/87: mkdir d4/d12/d16 0 2024-02-19T19:04:47.901 INFO:tasks.workunit.client.1.smithi132.stdout:8/150: creat d4/dc/d1b/d1d/f36 x:0 0 0 2024-02-19T19:04:47.902 INFO:tasks.workunit.client.1.smithi132.stdout:2/114: read - d0/d24/f2d zero size 2024-02-19T19:04:47.906 INFO:tasks.workunit.client.1.smithi132.stdout:1/107: symlink d1/d1e/l23 0 2024-02-19T19:04:47.906 INFO:tasks.workunit.client.1.smithi132.stdout:5/129: creat d8/da/db/f28 x:0 0 0 2024-02-19T19:04:47.908 INFO:tasks.workunit.client.1.smithi132.stdout:5/130: readlink d8/da/db/l19 0 2024-02-19T19:04:47.909 INFO:tasks.workunit.client.1.smithi132.stdout:4/88: dwrite d4/f10 [0,4194304] 0 2024-02-19T19:04:47.909 INFO:tasks.workunit.client.1.smithi132.stdout:5/131: truncate d8/da/f24 1190761 0 2024-02-19T19:04:47.914 INFO:tasks.workunit.client.1.smithi132.stdout:8/151: symlink d4/dc/d1b/d1d/l37 0 2024-02-19T19:04:47.922 INFO:tasks.workunit.client.1.smithi132.stdout:1/108: symlink d1/d19/l24 0 2024-02-19T19:04:47.923 INFO:tasks.workunit.client.1.smithi132.stdout:1/109: write d1/d19/f22 [735579,112332] 0 2024-02-19T19:04:47.924 INFO:tasks.workunit.client.1.smithi132.stdout:1/110: chown d1/f18 49852674 1 2024-02-19T19:04:47.925 INFO:tasks.workunit.client.1.smithi132.stdout:9/194: mknod d12/d3e/c45 0 2024-02-19T19:04:47.929 INFO:tasks.workunit.client.1.smithi132.stdout:4/89: fdatasync d4/f10 0 2024-02-19T19:04:47.931 INFO:tasks.workunit.client.1.smithi132.stdout:4/90: read d4/fa [213848,85549] 0 2024-02-19T19:04:47.932 INFO:tasks.workunit.client.1.smithi132.stdout:5/132: rename d8/d18 to d8/d23/d29 0 2024-02-19T19:04:47.933 INFO:tasks.workunit.client.1.smithi132.stdout:5/133: dread - d8/d23/d29/d27/f25 zero size 2024-02-19T19:04:47.934 INFO:tasks.workunit.client.1.smithi132.stdout:2/115: mkdir d0/d4/d11/d2e 0 2024-02-19T19:04:47.940 INFO:tasks.workunit.client.1.smithi132.stdout:0/114: getdents d2 0 2024-02-19T19:04:47.941 INFO:tasks.workunit.client.1.smithi132.stdout:9/195: readlink d12/l2d 0 2024-02-19T19:04:47.941 INFO:tasks.workunit.client.1.smithi132.stdout:0/115: stat d2/dc/df/d15/f22 0 2024-02-19T19:04:47.943 INFO:tasks.workunit.client.1.smithi132.stdout:4/91: unlink d4/dc/fe 0 2024-02-19T19:04:47.947 INFO:tasks.workunit.client.1.smithi132.stdout:2/116: mkdir d0/d19/d2f 0 2024-02-19T19:04:47.948 INFO:tasks.workunit.client.1.smithi132.stdout:8/152: mkdir d4/dc/d1b/d38 0 2024-02-19T19:04:47.948 INFO:tasks.workunit.client.1.smithi132.stdout:1/111: fdatasync d1/f5 0 2024-02-19T19:04:47.954 INFO:tasks.workunit.client.1.smithi132.stdout:5/134: write d8/fe [2273556,28911] 0 2024-02-19T19:04:47.956 INFO:tasks.workunit.client.1.smithi132.stdout:2/117: symlink d0/d19/d1e/l30 0 2024-02-19T19:04:47.956 INFO:tasks.workunit.client.1.smithi132.stdout:2/118: readlink d0/l1 0 2024-02-19T19:04:47.962 INFO:tasks.workunit.client.1.smithi132.stdout:9/196: rename d12/d1b/l39 to d12/l46 0 2024-02-19T19:04:47.978 INFO:tasks.workunit.client.1.smithi132.stdout:3/76: sync 2024-02-19T19:04:47.979 INFO:tasks.workunit.client.1.smithi132.stdout:6/80: sync 2024-02-19T19:04:47.991 INFO:tasks.workunit.client.1.smithi132.stdout:0/116: dwrite d2/dc/f20 [0,4194304] 0 2024-02-19T19:04:47.991 INFO:tasks.workunit.client.1.smithi132.stdout:2/119: symlink d0/d4/l31 0 2024-02-19T19:04:47.993 INFO:tasks.workunit.client.1.smithi132.stdout:9/197: fdatasync d12/d1b/f3f 0 2024-02-19T19:04:47.997 INFO:tasks.workunit.client.1.smithi132.stdout:4/92: rename d4/dc/ff to d4/d12/f17 0 2024-02-19T19:04:47.999 INFO:tasks.workunit.client.1.smithi132.stdout:2/120: dread d0/d4/f22 [0,4194304] 0 2024-02-19T19:04:48.001 INFO:tasks.workunit.client.1.smithi132.stdout:6/81: dread d0/db/d16/f18 [0,4194304] 0 2024-02-19T19:04:48.001 INFO:tasks.workunit.client.1.smithi132.stdout:2/121: readlink d0/d19/d1e/l30 0 2024-02-19T19:04:48.002 INFO:tasks.workunit.client.1.smithi132.stdout:6/82: write d0/dc/f15 [109754,73610] 0 2024-02-19T19:04:48.003 INFO:tasks.workunit.client.1.smithi132.stdout:3/77: readlink d8/lb 0 2024-02-19T19:04:48.003 INFO:tasks.workunit.client.1.smithi132.stdout:6/83: fdatasync d0/d1/f8 0 2024-02-19T19:04:48.004 INFO:tasks.workunit.client.1.smithi132.stdout:3/78: read - d8/f10 zero size 2024-02-19T19:04:48.004 INFO:tasks.workunit.client.1.smithi132.stdout:3/79: fdatasync d8/f10 0 2024-02-19T19:04:48.005 INFO:tasks.workunit.client.1.smithi132.stdout:5/135: dread f6 [0,4194304] 0 2024-02-19T19:04:48.005 INFO:tasks.workunit.client.1.smithi132.stdout:5/136: chown d8/d1a/d1e 60332787 1 2024-02-19T19:04:48.010 INFO:tasks.workunit.client.1.smithi132.stdout:9/198: fsync d12/d25/f35 0 2024-02-19T19:04:48.011 INFO:tasks.workunit.client.1.smithi132.stdout:9/199: stat d12/d2a/f37 0 2024-02-19T19:04:48.012 INFO:tasks.workunit.client.1.smithi132.stdout:0/117: dwrite d2/f1a [0,4194304] 0 2024-02-19T19:04:48.016 INFO:tasks.workunit.client.1.smithi132.stdout:6/84: creat d0/db/d16/f20 x:0 0 0 2024-02-19T19:04:48.016 INFO:tasks.workunit.client.1.smithi132.stdout:3/80: mkdir d8/d13 0 2024-02-19T19:04:48.022 INFO:tasks.workunit.client.1.smithi132.stdout:4/93: symlink d4/l18 0 2024-02-19T19:04:48.023 INFO:tasks.workunit.client.1.smithi132.stdout:0/118: rename d2/dc/f20 to d2/d21/d25/f26 0 2024-02-19T19:04:48.023 INFO:tasks.workunit.client.1.smithi132.stdout:3/81: creat d8/f14 x:0 0 0 2024-02-19T19:04:48.026 INFO:tasks.workunit.client.1.smithi132.stdout:2/122: link d0/d4/f22 d0/d4/d11/f32 0 2024-02-19T19:04:48.026 INFO:tasks.workunit.client.1.smithi132.stdout:2/123: chown d0/d4/fb 6999494 1 2024-02-19T19:04:48.027 INFO:tasks.workunit.client.1.smithi132.stdout:2/124: truncate d0/d4/d11/f26 1015610 0 2024-02-19T19:04:48.028 INFO:tasks.workunit.client.1.smithi132.stdout:2/125: fdatasync d0/d4/d11/f26 0 2024-02-19T19:04:48.028 INFO:tasks.workunit.client.1.smithi132.stdout:6/85: getdents d0/d1e 0 2024-02-19T19:04:48.029 INFO:tasks.workunit.client.1.smithi132.stdout:6/86: dread - d0/d1/f19 zero size 2024-02-19T19:04:48.030 INFO:tasks.workunit.client.1.smithi132.stdout:3/82: creat d8/f15 x:0 0 0 2024-02-19T19:04:48.030 INFO:tasks.workunit.client.1.smithi132.stdout:3/83: fdatasync f0 0 2024-02-19T19:04:48.032 INFO:tasks.workunit.client.1.smithi132.stdout:4/94: dread d4/fa [0,4194304] 0 2024-02-19T19:04:48.032 INFO:tasks.workunit.client.1.smithi132.stdout:9/200: getdents d12/d25/d3a 0 2024-02-19T19:04:48.033 INFO:tasks.workunit.client.1.smithi132.stdout:6/87: mknod d0/db/d16/c21 0 2024-02-19T19:04:48.033 INFO:tasks.workunit.client.1.smithi132.stdout:4/95: write d4/f5 [306686,101834] 0 2024-02-19T19:04:48.034 INFO:tasks.workunit.client.1.smithi132.stdout:3/84: rename f4 to d8/d13/f16 0 2024-02-19T19:04:48.038 INFO:tasks.workunit.client.1.smithi132.stdout:5/137: truncate d8/da/db/f1b 884783 0 2024-02-19T19:04:48.040 INFO:tasks.workunit.client.1.smithi132.stdout:9/201: creat d12/d25/f47 x:0 0 0 2024-02-19T19:04:48.040 INFO:tasks.workunit.client.1.smithi132.stdout:3/85: dwrite d8/f12 [0,4194304] 0 2024-02-19T19:04:48.042 INFO:tasks.workunit.client.1.smithi132.stdout:9/202: chown d12/d1b/f20 156039 1 2024-02-19T19:04:48.044 INFO:tasks.workunit.client.1.smithi132.stdout:9/203: dread - d12/d25/f47 zero size 2024-02-19T19:04:48.046 INFO:tasks.workunit.client.1.smithi132.stdout:9/204: stat d12/d1b/f1c 0 2024-02-19T19:04:48.047 INFO:tasks.workunit.client.1.smithi132.stdout:6/88: unlink d0/dc/c13 0 2024-02-19T19:04:48.047 INFO:tasks.workunit.client.1.smithi132.stdout:7/164: sync 2024-02-19T19:04:48.048 INFO:tasks.workunit.client.1.smithi132.stdout:6/89: chown d0 25956 1 2024-02-19T19:04:48.049 INFO:tasks.workunit.client.1.smithi132.stdout:5/138: dwrite d8/d23/d29/f1c [0,4194304] 0 2024-02-19T19:04:48.049 INFO:tasks.workunit.client.1.smithi132.stdout:6/90: readlink d0/le 0 2024-02-19T19:04:48.050 INFO:tasks.workunit.client.1.smithi132.stdout:5/139: chown d8/d1a/d1e 142 1 2024-02-19T19:04:48.051 INFO:tasks.workunit.client.1.smithi132.stdout:5/140: chown d8/da 3198694 1 2024-02-19T19:04:48.052 INFO:tasks.workunit.client.1.smithi132.stdout:9/205: creat d12/d3e/f48 x:0 0 0 2024-02-19T19:04:48.056 INFO:tasks.workunit.client.1.smithi132.stdout:5/141: write d8/fe [4260267,62031] 0 2024-02-19T19:04:48.057 INFO:tasks.workunit.client.1.smithi132.stdout:2/126: dwrite d0/f6 [4194304,4194304] 0 2024-02-19T19:04:48.060 INFO:tasks.workunit.client.1.smithi132.stdout:4/96: getdents d4 0 2024-02-19T19:04:48.061 INFO:tasks.workunit.client.1.smithi132.stdout:3/86: link d8/le d8/l17 0 2024-02-19T19:04:48.063 INFO:tasks.workunit.client.1.smithi132.stdout:4/97: write d4/f7 [3161547,2560] 0 2024-02-19T19:04:48.064 INFO:tasks.workunit.client.1.smithi132.stdout:9/206: dread d12/f23 [0,4194304] 0 2024-02-19T19:04:48.064 INFO:tasks.workunit.client.1.smithi132.stdout:5/142: dread f5 [4194304,4194304] 0 2024-02-19T19:04:48.067 INFO:tasks.workunit.client.1.smithi132.stdout:4/98: dread - d4/d12/f17 zero size 2024-02-19T19:04:48.067 INFO:tasks.workunit.client.1.smithi132.stdout:7/165: link d1b/c1f d1b/d32/c33 0 2024-02-19T19:04:48.069 INFO:tasks.workunit.client.1.smithi132.stdout:2/127: mkdir d0/d4/d33 0 2024-02-19T19:04:48.070 INFO:tasks.workunit.client.1.smithi132.stdout:5/143: mknod d8/d23/d29/c2a 0 2024-02-19T19:04:48.070 INFO:tasks.workunit.client.1.smithi132.stdout:5/144: stat d8/d1a 0 2024-02-19T19:04:48.072 INFO:tasks.workunit.client.1.smithi132.stdout:3/87: dwrite f0 [4194304,4194304] 0 2024-02-19T19:04:48.072 INFO:tasks.workunit.client.1.smithi132.stdout:9/207: creat d12/d25/d3a/d40/f49 x:0 0 0 2024-02-19T19:04:48.073 INFO:tasks.workunit.client.1.smithi132.stdout:3/88: stat d8/lc 0 2024-02-19T19:04:48.076 INFO:tasks.workunit.client.1.smithi132.stdout:4/99: mkdir d4/d19 0 2024-02-19T19:04:48.076 INFO:tasks.workunit.client.1.smithi132.stdout:7/166: symlink d1b/l34 0 2024-02-19T19:04:48.084 INFO:tasks.workunit.client.1.smithi132.stdout:3/89: readlink d8/le 0 2024-02-19T19:04:48.085 INFO:tasks.workunit.client.1.smithi132.stdout:7/167: creat d1b/d1e/d28/f35 x:0 0 0 2024-02-19T19:04:48.086 INFO:tasks.workunit.client.1.smithi132.stdout:6/91: dwrite d0/f10 [0,4194304] 0 2024-02-19T19:04:48.088 INFO:tasks.workunit.client.1.smithi132.stdout:2/128: dwrite d0/d4/f12 [0,4194304] 0 2024-02-19T19:04:48.088 INFO:tasks.workunit.client.1.smithi132.stdout:7/168: write ff [4682165,23510] 0 2024-02-19T19:04:48.093 INFO:tasks.workunit.client.1.smithi132.stdout:9/208: dwrite d12/f23 [0,4194304] 0 2024-02-19T19:04:48.093 INFO:tasks.workunit.client.1.smithi132.stdout:8/153: sync 2024-02-19T19:04:48.094 INFO:tasks.workunit.client.1.smithi132.stdout:8/154: dread - d4/f27 zero size 2024-02-19T19:04:48.094 INFO:tasks.workunit.client.1.smithi132.stdout:9/209: fsync d12/f1a 0 2024-02-19T19:04:48.098 INFO:tasks.workunit.client.1.smithi132.stdout:2/129: creat d0/d24/f34 x:0 0 0 2024-02-19T19:04:48.099 INFO:tasks.workunit.client.1.smithi132.stdout:2/130: write d0/d4/f12 [2089671,7194] 0 2024-02-19T19:04:48.101 INFO:tasks.workunit.client.1.smithi132.stdout:5/145: truncate d8/da/f24 613935 0 2024-02-19T19:04:48.104 INFO:tasks.workunit.client.1.smithi132.stdout:8/155: mknod d4/d2d/c39 0 2024-02-19T19:04:48.105 INFO:tasks.workunit.client.1.smithi132.stdout:9/210: fdatasync d12/d1b/f16 0 2024-02-19T19:04:48.105 INFO:tasks.workunit.client.1.smithi132.stdout:9/211: readlink d12/l41 0 2024-02-19T19:04:48.106 INFO:tasks.workunit.client.1.smithi132.stdout:2/131: dread d0/d4/d11/f15 [0,4194304] 0 2024-02-19T19:04:48.107 INFO:tasks.workunit.client.1.smithi132.stdout:2/132: chown d0/d4/d11/f26 183 1 2024-02-19T19:04:48.107 INFO:tasks.workunit.client.1.smithi132.stdout:5/146: dread d8/fe [0,4194304] 0 2024-02-19T19:04:48.109 INFO:tasks.workunit.client.1.smithi132.stdout:7/169: rename c7 to d1b/d32/c36 0 2024-02-19T19:04:48.111 INFO:tasks.workunit.client.1.smithi132.stdout:4/100: truncate d4/f9 13351178 0 2024-02-19T19:04:48.114 INFO:tasks.workunit.client.1.smithi132.stdout:2/133: dwrite d0/d24/f2d [0,4194304] 0 2024-02-19T19:04:48.121 INFO:tasks.workunit.client.1.smithi132.stdout:8/156: rename d4/l7 to d4/dc/d1a/l3a 0 2024-02-19T19:04:48.123 INFO:tasks.workunit.client.1.smithi132.stdout:9/212: dread d12/d2a/f37 [0,4194304] 0 2024-02-19T19:04:48.126 INFO:tasks.workunit.client.1.smithi132.stdout:7/170: creat d1b/d1e/f37 x:0 0 0 2024-02-19T19:04:48.128 INFO:tasks.workunit.client.1.smithi132.stdout:2/134: rmdir d0/d19/d1e 39 2024-02-19T19:04:48.136 INFO:tasks.workunit.client.1.smithi132.stdout:8/157: symlink d4/dc/d1b/d20/d23/l3b 0 2024-02-19T19:04:48.136 INFO:tasks.workunit.client.1.smithi132.stdout:8/158: stat d4/dc/d1b/d20/d23 0 2024-02-19T19:04:48.137 INFO:tasks.workunit.client.1.smithi132.stdout:8/159: truncate d4/f2e 767607 0 2024-02-19T19:04:48.138 INFO:tasks.workunit.client.1.smithi132.stdout:9/213: creat d12/d3e/f4a x:0 0 0 2024-02-19T19:04:48.138 INFO:tasks.workunit.client.1.smithi132.stdout:8/160: fsync d4/f19 0 2024-02-19T19:04:48.139 INFO:tasks.workunit.client.1.smithi132.stdout:9/214: write d12/f26 [10066,114127] 0 2024-02-19T19:04:48.139 INFO:tasks.workunit.client.1.smithi132.stdout:4/101: creat d4/d12/f1a x:0 0 0 2024-02-19T19:04:48.141 INFO:tasks.workunit.client.1.smithi132.stdout:8/161: truncate d4/dc/d1b/d20/d23/f28 172934 0 2024-02-19T19:04:48.142 INFO:tasks.workunit.client.1.smithi132.stdout:8/162: chown d4/dc/d1b/d20/d23/c2f 24013 1 2024-02-19T19:04:48.145 INFO:tasks.workunit.client.1.smithi132.stdout:9/215: fdatasync d12/d2a/f2c 0 2024-02-19T19:04:48.148 INFO:tasks.workunit.client.1.smithi132.stdout:8/163: creat d4/dc/d1b/d1d/f3c x:0 0 0 2024-02-19T19:04:48.154 INFO:tasks.workunit.client.1.smithi132.stdout:4/102: creat d4/d12/d16/f1b x:0 0 0 2024-02-19T19:04:48.154 INFO:tasks.workunit.client.1.smithi132.stdout:9/216: fdatasync d12/d1b/f1c 0 2024-02-19T19:04:48.155 INFO:tasks.workunit.client.1.smithi132.stdout:9/217: chown f5 5319176 1 2024-02-19T19:04:48.158 INFO:tasks.workunit.client.1.smithi132.stdout:8/164: mknod d4/dc/d1b/d33/c3d 0 2024-02-19T19:04:48.159 INFO:tasks.workunit.client.1.smithi132.stdout:8/165: rename d4/dc/d1b/d38 to d4/dc/d1b/d38/d3e 22 2024-02-19T19:04:48.159 INFO:tasks.workunit.client.1.smithi132.stdout:7/171: getdents d1b/d1e 0 2024-02-19T19:04:48.160 INFO:tasks.workunit.client.1.smithi132.stdout:4/103: fsync d4/d12/f17 0 2024-02-19T19:04:48.160 INFO:tasks.workunit.client.1.smithi132.stdout:8/166: write d4/dc/f22 [1014904,26108] 0 2024-02-19T19:04:48.162 INFO:tasks.workunit.client.1.smithi132.stdout:9/218: rmdir d12/d25/d3a/d40 39 2024-02-19T19:04:48.164 INFO:tasks.workunit.client.1.smithi132.stdout:7/172: dread d1b/f1c [0,4194304] 0 2024-02-19T19:04:48.164 INFO:tasks.workunit.client.1.smithi132.stdout:2/135: getdents d0/d4/d11 0 2024-02-19T19:04:48.165 INFO:tasks.workunit.client.1.smithi132.stdout:2/136: chown d0/c1d 3336103 1 2024-02-19T19:04:48.165 INFO:tasks.workunit.client.1.smithi132.stdout:2/137: stat d0/d24/f2d 0 2024-02-19T19:04:48.169 INFO:tasks.workunit.client.1.smithi132.stdout:2/138: dread d0/d4/d11/f26 [0,4194304] 0 2024-02-19T19:04:48.170 INFO:tasks.workunit.client.1.smithi132.stdout:9/219: creat d12/d1b/f4b x:0 0 0 2024-02-19T19:04:48.170 INFO:tasks.workunit.client.1.smithi132.stdout:7/173: dwrite d1b/f1c [0,4194304] 0 2024-02-19T19:04:48.173 INFO:tasks.workunit.client.1.smithi132.stdout:8/167: link d4/d2d/f31 d4/dc/f3f 0 2024-02-19T19:04:48.173 INFO:tasks.workunit.client.1.smithi132.stdout:4/104: truncate d4/fa 1075587 0 2024-02-19T19:04:48.175 INFO:tasks.workunit.client.1.smithi132.stdout:8/168: dread d4/dc/d1b/d33/f34 [0,4194304] 0 2024-02-19T19:04:48.175 INFO:tasks.workunit.client.1.smithi132.stdout:2/139: mknod d0/d4/d11/c35 0 2024-02-19T19:04:48.175 INFO:tasks.workunit.client.1.smithi132.stdout:9/220: chown d12/c1f 1406 1 2024-02-19T19:04:48.176 INFO:tasks.workunit.client.1.smithi132.stdout:8/169: fdatasync d4/dc/f2c 0 2024-02-19T19:04:48.177 INFO:tasks.workunit.client.1.smithi132.stdout:8/170: readlink d4/dc/d1b/l25 0 2024-02-19T19:04:48.177 INFO:tasks.workunit.client.1.smithi132.stdout:9/221: truncate d12/d25/f47 977294 0 2024-02-19T19:04:48.178 INFO:tasks.workunit.client.1.smithi132.stdout:7/174: rename f6 to d1b/d1e/f38 0 2024-02-19T19:04:48.179 INFO:tasks.workunit.client.1.smithi132.stdout:7/175: write f10 [4626865,85633] 0 2024-02-19T19:04:48.187 INFO:tasks.workunit.client.1.smithi132.stdout:8/171: fsync d4/fb 0 2024-02-19T19:04:48.187 INFO:tasks.workunit.client.1.smithi132.stdout:9/222: mkdir d12/d2a/d4c 0 2024-02-19T19:04:48.191 INFO:tasks.workunit.client.1.smithi132.stdout:7/176: creat d1b/d32/f39 x:0 0 0 2024-02-19T19:04:48.192 INFO:tasks.workunit.client.1.smithi132.stdout:4/105: symlink d4/dc/l1c 0 2024-02-19T19:04:48.194 INFO:tasks.workunit.client.1.smithi132.stdout:7/177: dread d1b/d1e/f22 [0,4194304] 0 2024-02-19T19:04:48.196 INFO:tasks.workunit.client.1.smithi132.stdout:8/172: mknod d4/dc/c40 0 2024-02-19T19:04:48.198 INFO:tasks.workunit.client.1.smithi132.stdout:7/178: dread d1b/d1e/d28/f29 [0,4194304] 0 2024-02-19T19:04:48.202 INFO:tasks.workunit.client.1.smithi132.stdout:1/112: sync 2024-02-19T19:04:48.202 INFO:tasks.workunit.client.1.smithi132.stdout:0/119: sync 2024-02-19T19:04:48.203 INFO:tasks.workunit.client.1.smithi132.stdout:9/223: rmdir d12/d1b 39 2024-02-19T19:04:48.205 INFO:tasks.workunit.client.1.smithi132.stdout:7/179: dwrite d1b/d1e/d28/f35 [0,4194304] 0 2024-02-19T19:04:48.206 INFO:tasks.workunit.client.1.smithi132.stdout:7/180: write d1b/f1c [3444950,101571] 0 2024-02-19T19:04:48.209 INFO:tasks.workunit.client.1.smithi132.stdout:7/181: dread d1b/f1c [0,4194304] 0 2024-02-19T19:04:48.212 INFO:tasks.workunit.client.1.smithi132.stdout:2/140: write d0/d4/f22 [3422974,35633] 0 2024-02-19T19:04:48.213 INFO:tasks.workunit.client.1.smithi132.stdout:1/113: rmdir d1/d1e 39 2024-02-19T19:04:48.218 INFO:tasks.workunit.client.1.smithi132.stdout:8/173: mknod d4/dc/d1b/c41 0 2024-02-19T19:04:48.223 INFO:tasks.workunit.client.1.smithi132.stdout:4/106: dwrite d4/f9 [0,4194304] 0 2024-02-19T19:04:48.226 INFO:tasks.workunit.client.1.smithi132.stdout:7/182: symlink d1b/d32/l3a 0 2024-02-19T19:04:48.228 INFO:tasks.workunit.client.1.smithi132.stdout:7/183: chown f8 221164 1 2024-02-19T19:04:48.230 INFO:tasks.workunit.client.1.smithi132.stdout:0/120: dwrite d2/f12 [0,4194304] 0 2024-02-19T19:04:48.233 INFO:tasks.workunit.client.1.smithi132.stdout:8/174: mkdir d4/dc/d42 0 2024-02-19T19:04:48.233 INFO:tasks.workunit.client.1.smithi132.stdout:9/224: symlink d12/d25/d3a/d40/l4d 0 2024-02-19T19:04:48.237 INFO:tasks.workunit.client.1.smithi132.stdout:4/107: unlink d4/d12/f17 0 2024-02-19T19:04:48.240 INFO:tasks.workunit.client.1.smithi132.stdout:0/121: dwrite d2/dc/df/d15/f22 [0,4194304] 0 2024-02-19T19:04:48.240 INFO:tasks.workunit.client.1.smithi132.stdout:2/141: link d0/f6 d0/d4/d33/f36 0 2024-02-19T19:04:48.244 INFO:tasks.workunit.client.1.smithi132.stdout:1/114: mkdir d1/d1e/d25 0 2024-02-19T19:04:48.250 INFO:tasks.workunit.client.1.smithi132.stdout:4/108: mknod d4/dc/c1d 0 2024-02-19T19:04:48.250 INFO:tasks.workunit.client.1.smithi132.stdout:9/225: chown d12/d1b 4538461 1 2024-02-19T19:04:48.252 INFO:tasks.workunit.client.1.smithi132.stdout:0/122: write d2/dc/df/d15/f24 [1669983,12029] 0 2024-02-19T19:04:48.252 INFO:tasks.workunit.client.1.smithi132.stdout:0/123: read d2/f6 [7463040,38027] 0 2024-02-19T19:04:48.256 INFO:tasks.workunit.client.1.smithi132.stdout:9/226: mkdir d12/d25/d3a/d40/d4e 0 2024-02-19T19:04:48.257 INFO:tasks.workunit.client.1.smithi132.stdout:0/124: mkdir d2/dc/df/d27 0 2024-02-19T19:04:48.258 INFO:tasks.workunit.client.1.smithi132.stdout:0/125: chown d2/f12 1261322741 1 2024-02-19T19:04:48.259 INFO:tasks.workunit.client.1.smithi132.stdout:4/109: truncate d4/fa 70619 0 2024-02-19T19:04:48.259 INFO:tasks.workunit.client.1.smithi132.stdout:0/126: chown d2/l1c 20431546 1 2024-02-19T19:04:48.260 INFO:tasks.workunit.client.1.smithi132.stdout:2/142: rmdir d0/d19/d2f 0 2024-02-19T19:04:48.260 INFO:tasks.workunit.client.1.smithi132.stdout:9/227: dread d12/d2a/f37 [0,4194304] 0 2024-02-19T19:04:48.261 INFO:tasks.workunit.client.1.smithi132.stdout:1/115: rmdir d1/d1e/d25 0 2024-02-19T19:04:48.265 INFO:tasks.workunit.client.1.smithi132.stdout:4/110: dread d4/f5 [0,4194304] 0 2024-02-19T19:04:48.267 INFO:tasks.workunit.client.1.smithi132.stdout:9/228: mknod d12/d2a/c4f 0 2024-02-19T19:04:48.267 INFO:tasks.workunit.client.1.smithi132.stdout:1/116: dread d1/f16 [0,4194304] 0 2024-02-19T19:04:48.270 INFO:tasks.workunit.client.1.smithi132.stdout:1/117: dread d1/f15 [0,4194304] 0 2024-02-19T19:04:48.271 INFO:tasks.workunit.client.1.smithi132.stdout:1/118: fdatasync d1/f15 0 2024-02-19T19:04:48.271 INFO:tasks.workunit.client.1.smithi132.stdout:0/127: fdatasync d2/dc/df/d15/f22 0 2024-02-19T19:04:48.273 INFO:tasks.workunit.client.1.smithi132.stdout:4/111: rename d4/d12/c15 to d4/d19/c1e 0 2024-02-19T19:04:48.274 INFO:tasks.workunit.client.1.smithi132.stdout:2/143: dread d0/d4/d11/f32 [0,4194304] 0 2024-02-19T19:04:48.277 INFO:tasks.workunit.client.1.smithi132.stdout:9/229: rename d12/d1b/f36 to d12/d1b/f50 0 2024-02-19T19:04:48.285 INFO:tasks.workunit.client.1.smithi132.stdout:4/112: mknod d4/d12/d16/c1f 0 2024-02-19T19:04:48.286 INFO:tasks.workunit.client.1.smithi132.stdout:4/113: readlink d4/dc/l1c 0 2024-02-19T19:04:48.287 INFO:tasks.workunit.client.1.smithi132.stdout:7/184: dwrite f8 [0,4194304] 0 2024-02-19T19:04:48.293 INFO:tasks.workunit.client.1.smithi132.stdout:9/230: rename d12/d1b/f4b to d12/d25/d3a/d40/f51 0 2024-02-19T19:04:48.293 INFO:tasks.workunit.client.1.smithi132.stdout:0/128: dwrite d2/d21/d25/f26 [4194304,4194304] 0 2024-02-19T19:04:48.299 INFO:tasks.workunit.client.1.smithi132.stdout:2/144: dwrite d0/d4/d11/f1c [0,4194304] 0 2024-02-19T19:04:48.300 INFO:tasks.workunit.client.1.smithi132.stdout:7/185: dwrite d1b/d32/f39 [0,4194304] 0 2024-02-19T19:04:48.310 INFO:tasks.workunit.client.1.smithi132.stdout:7/186: creat d1b/d1e/d28/f3b x:0 0 0 2024-02-19T19:04:48.312 INFO:tasks.workunit.client.1.smithi132.stdout:0/129: mknod d2/dc/df/d27/c28 0 2024-02-19T19:04:48.313 INFO:tasks.workunit.client.1.smithi132.stdout:9/231: creat d12/d25/d3a/f52 x:0 0 0 2024-02-19T19:04:48.314 INFO:tasks.workunit.client.1.smithi132.stdout:3/90: sync 2024-02-19T19:04:48.318 INFO:tasks.workunit.client.1.smithi132.stdout:6/92: sync 2024-02-19T19:04:48.318 INFO:tasks.workunit.client.1.smithi132.stdout:6/93: stat d0/db/d16/f18 0 2024-02-19T19:04:48.319 INFO:tasks.workunit.client.1.smithi132.stdout:7/187: rename d1b/d32/f39 to d1b/d1e/d28/f3c 0 2024-02-19T19:04:48.320 INFO:tasks.workunit.client.1.smithi132.stdout:0/130: symlink d2/dc/l29 0 2024-02-19T19:04:48.321 INFO:tasks.workunit.client.1.smithi132.stdout:0/131: chown d2/l8 3246 1 2024-02-19T19:04:48.327 INFO:tasks.workunit.client.1.smithi132.stdout:7/188: dwrite d1b/d1e/f30 [0,4194304] 0 2024-02-19T19:04:48.327 INFO:tasks.workunit.client.1.smithi132.stdout:2/145: link d0/d4/c10 d0/d4/c37 0 2024-02-19T19:04:48.332 INFO:tasks.workunit.client.1.smithi132.stdout:0/132: dwrite d2/dc/fe [0,4194304] 0 2024-02-19T19:04:48.337 INFO:tasks.workunit.client.1.smithi132.stdout:7/189: creat d1b/d1e/f3d x:0 0 0 2024-02-19T19:04:48.347 INFO:tasks.workunit.client.1.smithi132.stdout:3/91: getdents d8/d13 0 2024-02-19T19:04:48.348 INFO:tasks.workunit.client.1.smithi132.stdout:3/92: write d8/f15 [808165,88807] 0 2024-02-19T19:04:48.349 INFO:tasks.workunit.client.1.smithi132.stdout:3/93: write d8/f12 [1052856,27447] 0 2024-02-19T19:04:48.352 INFO:tasks.workunit.client.1.smithi132.stdout:9/232: rename d12/d25/f47 to d12/f53 0 2024-02-19T19:04:48.354 INFO:tasks.workunit.client.1.smithi132.stdout:7/190: write d1b/d1e/f22 [1167297,53023] 0 2024-02-19T19:04:48.354 INFO:tasks.workunit.client.1.smithi132.stdout:9/233: dread d12/d1b/f16 [0,4194304] 0 2024-02-19T19:04:48.355 INFO:tasks.workunit.client.1.smithi132.stdout:7/191: readlink l18 0 2024-02-19T19:04:48.356 INFO:tasks.workunit.client.1.smithi132.stdout:3/94: dwrite d8/f12 [0,4194304] 0 2024-02-19T19:04:48.360 INFO:tasks.workunit.client.1.smithi132.stdout:6/94: rename d0/d1 to d0/d1e/d22 0 2024-02-19T19:04:48.364 INFO:tasks.workunit.client.1.smithi132.stdout:9/234: write d12/d1b/f50 [1320392,439] 0 2024-02-19T19:04:48.365 INFO:tasks.workunit.client.1.smithi132.stdout:0/133: link d2/la d2/dc/l2a 0 2024-02-19T19:04:48.365 INFO:tasks.workunit.client.1.smithi132.stdout:6/95: write d0/f5 [1079304,122908] 0 2024-02-19T19:04:48.370 INFO:tasks.workunit.client.1.smithi132.stdout:3/95: dread d8/d13/f16 [0,4194304] 0 2024-02-19T19:04:48.371 INFO:tasks.workunit.client.1.smithi132.stdout:2/146: truncate d0/d24/f2d 1831930 0 2024-02-19T19:04:48.372 INFO:tasks.workunit.client.1.smithi132.stdout:6/96: dread d0/f12 [0,4194304] 0 2024-02-19T19:04:48.374 INFO:tasks.workunit.client.1.smithi132.stdout:0/134: creat d2/f2b x:0 0 0 2024-02-19T19:04:48.376 INFO:tasks.workunit.client.1.smithi132.stdout:7/192: link lc d1b/d1e/d2b/l3e 0 2024-02-19T19:04:48.377 INFO:tasks.workunit.client.1.smithi132.stdout:9/235: getdents d12/d25/d3a/d40/d4e 0 2024-02-19T19:04:48.378 INFO:tasks.workunit.client.1.smithi132.stdout:7/193: write d1b/d1e/f30 [2940990,59319] 0 2024-02-19T19:04:48.378 INFO:tasks.workunit.client.1.smithi132.stdout:3/96: symlink d8/l18 0 2024-02-19T19:04:48.379 INFO:tasks.workunit.client.1.smithi132.stdout:6/97: dwrite d0/fa [4194304,4194304] 0 2024-02-19T19:04:48.380 INFO:tasks.workunit.client.1.smithi132.stdout:9/236: write d12/d25/f30 [3944195,19396] 0 2024-02-19T19:04:48.382 INFO:tasks.workunit.client.1.smithi132.stdout:9/237: read d12/f53 [957651,25116] 0 2024-02-19T19:04:48.384 INFO:tasks.workunit.client.1.smithi132.stdout:9/238: truncate d12/d1b/f3b 72969 0 2024-02-19T19:04:48.384 INFO:tasks.workunit.client.1.smithi132.stdout:2/147: mknod d0/d4/d11/c38 0 2024-02-19T19:04:48.388 INFO:tasks.workunit.client.1.smithi132.stdout:3/97: mkdir d8/d13/d19 0 2024-02-19T19:04:48.394 INFO:tasks.workunit.client.1.smithi132.stdout:3/98: dwrite d8/f10 [0,4194304] 0 2024-02-19T19:04:48.394 INFO:tasks.workunit.client.1.smithi132.stdout:6/98: creat d0/d1e/d22/f23 x:0 0 0 2024-02-19T19:04:48.399 INFO:tasks.workunit.client.1.smithi132.stdout:7/194: symlink d1b/d1e/d2b/l3f 0 2024-02-19T19:04:48.402 INFO:tasks.workunit.client.1.smithi132.stdout:5/147: sync 2024-02-19T19:04:48.402 INFO:tasks.workunit.client.1.smithi132.stdout:8/175: sync 2024-02-19T19:04:48.405 INFO:tasks.workunit.client.1.smithi132.stdout:6/99: creat d0/dc/f24 x:0 0 0 2024-02-19T19:04:48.406 INFO:tasks.workunit.client.1.smithi132.stdout:6/100: chown d0/le 536929275 1 2024-02-19T19:04:48.409 INFO:tasks.workunit.client.1.smithi132.stdout:2/148: rmdir d0/d19/d1e 39 2024-02-19T19:04:48.413 INFO:tasks.workunit.client.1.smithi132.stdout:8/176: dread d4/f19 [0,4194304] 0 2024-02-19T19:04:48.414 INFO:tasks.workunit.client.1.smithi132.stdout:9/239: link d12/d1b/f20 d12/d25/d3a/d40/f54 0 2024-02-19T19:04:48.415 INFO:tasks.workunit.client.1.smithi132.stdout:3/99: mknod d8/d13/d19/c1a 0 2024-02-19T19:04:48.416 INFO:tasks.workunit.client.1.smithi132.stdout:3/100: dread d8/d13/f16 [0,4194304] 0 2024-02-19T19:04:48.417 INFO:tasks.workunit.client.1.smithi132.stdout:5/148: creat d8/d23/d29/d27/f2b x:0 0 0 2024-02-19T19:04:48.420 INFO:tasks.workunit.client.1.smithi132.stdout:6/101: mkdir d0/db/d25 0 2024-02-19T19:04:48.423 INFO:tasks.workunit.client.1.smithi132.stdout:3/101: dwrite d8/f15 [0,4194304] 0 2024-02-19T19:04:48.425 INFO:tasks.workunit.client.1.smithi132.stdout:9/240: mknod d12/d2a/c55 0 2024-02-19T19:04:48.427 INFO:tasks.workunit.client.1.smithi132.stdout:2/149: dwrite d0/d4/d11/f26 [0,4194304] 0 2024-02-19T19:04:48.432 INFO:tasks.workunit.client.1.smithi132.stdout:6/102: creat d0/db/d16/f26 x:0 0 0 2024-02-19T19:04:48.434 INFO:tasks.workunit.client.1.smithi132.stdout:6/103: write d0/fa [2417952,53215] 0 2024-02-19T19:04:48.435 INFO:tasks.workunit.client.1.smithi132.stdout:3/102: fsync d8/f9 0 2024-02-19T19:04:48.435 INFO:tasks.workunit.client.1.smithi132.stdout:6/104: write d0/dc/f15 [669122,129685] 0 2024-02-19T19:04:48.441 INFO:tasks.workunit.client.1.smithi132.stdout:5/149: mknod d8/da/c2c 0 2024-02-19T19:04:48.441 INFO:tasks.workunit.client.1.smithi132.stdout:8/177: dwrite d4/dc/f3f [0,4194304] 0 2024-02-19T19:04:48.441 INFO:tasks.workunit.client.1.smithi132.stdout:0/135: rmdir d2/d1d 0 2024-02-19T19:04:48.443 INFO:tasks.workunit.client.1.smithi132.stdout:8/178: fsync d4/dc/d1b/d20/d23/f28 0 2024-02-19T19:04:48.445 INFO:tasks.workunit.client.1.smithi132.stdout:3/103: symlink d8/d13/d19/l1b 0 2024-02-19T19:04:48.445 INFO:tasks.workunit.client.1.smithi132.stdout:9/241: mknod d12/d25/d3a/d40/d4e/c56 0 2024-02-19T19:04:48.448 INFO:tasks.workunit.client.1.smithi132.stdout:0/136: truncate d2/dc/df/d15/f24 2698502 0 2024-02-19T19:04:48.449 INFO:tasks.workunit.client.1.smithi132.stdout:0/137: stat d2/dc/df 0 2024-02-19T19:04:48.449 INFO:tasks.workunit.client.1.smithi132.stdout:5/150: creat d8/d23/d29/f2d x:0 0 0 2024-02-19T19:04:48.450 INFO:tasks.workunit.client.1.smithi132.stdout:5/151: chown d8/d23/d29/d27/f2b 128505964 1 2024-02-19T19:04:48.455 INFO:tasks.workunit.client.1.smithi132.stdout:5/152: mkdir d8/d2e 0 2024-02-19T19:04:48.465 INFO:tasks.workunit.client.1.smithi132.stdout:3/104: truncate f0 1727064 0 2024-02-19T19:04:48.466 INFO:tasks.workunit.client.1.smithi132.stdout:3/105: chown d8/fd 28 1 2024-02-19T19:04:48.466 INFO:tasks.workunit.client.1.smithi132.stdout:3/106: stat d8/l17 0 2024-02-19T19:04:48.467 INFO:tasks.workunit.client.1.smithi132.stdout:5/153: symlink d8/d23/d29/l2f 0 2024-02-19T19:04:48.468 INFO:tasks.workunit.client.1.smithi132.stdout:6/105: getdents d0/db/d16 0 2024-02-19T19:04:48.469 INFO:tasks.workunit.client.1.smithi132.stdout:9/242: dread d12/d1b/f50 [0,4194304] 0 2024-02-19T19:04:48.470 INFO:tasks.workunit.client.1.smithi132.stdout:0/138: rename d2/dc/df/l23 to d2/l2c 0 2024-02-19T19:04:48.470 INFO:tasks.workunit.client.1.smithi132.stdout:9/243: chown d12/d25/c34 72435849 1 2024-02-19T19:04:48.471 INFO:tasks.workunit.client.1.smithi132.stdout:9/244: dread - d12/f43 zero size 2024-02-19T19:04:48.472 INFO:tasks.workunit.client.1.smithi132.stdout:0/139: creat d2/dc/df/f2d x:0 0 0 2024-02-19T19:04:48.473 INFO:tasks.workunit.client.1.smithi132.stdout:0/140: write d2/f2b [390876,2933] 0 2024-02-19T19:04:48.475 INFO:tasks.workunit.client.1.smithi132.stdout:5/154: rename d8/da/db/f15 to d8/da/f30 0 2024-02-19T19:04:48.478 INFO:tasks.workunit.client.1.smithi132.stdout:6/106: rename d0 to d0/d27 22 2024-02-19T19:04:48.478 INFO:tasks.workunit.client.1.smithi132.stdout:5/155: creat d8/d23/d29/f31 x:0 0 0 2024-02-19T19:04:48.479 INFO:tasks.workunit.client.1.smithi132.stdout:5/156: readlink d8/da/l10 0 2024-02-19T19:04:48.484 INFO:tasks.workunit.client.1.smithi132.stdout:9/245: rename ce to d12/d25/d3a/d40/c57 0 2024-02-19T19:04:48.486 INFO:tasks.workunit.client.1.smithi132.stdout:6/107: creat d0/db/d16/f28 x:0 0 0 2024-02-19T19:04:48.494 INFO:tasks.workunit.client.1.smithi132.stdout:0/141: truncate d2/d21/d25/f26 1948073 0 2024-02-19T19:04:48.494 INFO:tasks.workunit.client.1.smithi132.stdout:0/142: write d2/dc/fe [4534773,15937] 0 2024-02-19T19:04:48.494 INFO:tasks.workunit.client.1.smithi132.stdout:5/157: dread d8/ff [0,4194304] 0 2024-02-19T19:04:48.495 INFO:tasks.workunit.client.1.smithi132.stdout:6/108: creat d0/db/d16/f29 x:0 0 0 2024-02-19T19:04:48.496 INFO:tasks.workunit.client.1.smithi132.stdout:6/109: dread - d0/d1e/d22/f8 zero size 2024-02-19T19:04:48.501 INFO:tasks.workunit.client.1.smithi132.stdout:6/110: symlink d0/d1e/l2a 0 2024-02-19T19:04:48.501 INFO:tasks.workunit.client.1.smithi132.stdout:9/246: getdents d12/d3e 0 2024-02-19T19:04:48.503 INFO:tasks.workunit.client.1.smithi132.stdout:5/158: dwrite d8/d23/d29/d27/f25 [0,4194304] 0 2024-02-19T19:04:48.504 INFO:tasks.workunit.client.1.smithi132.stdout:0/143: dwrite d2/dc/df/d15/f24 [0,4194304] 0 2024-02-19T19:04:48.507 INFO:tasks.workunit.client.1.smithi132.stdout:6/111: symlink d0/dc/l2b 0 2024-02-19T19:04:48.508 INFO:tasks.workunit.client.1.smithi132.stdout:6/112: chown d0/c1f 6939 1 2024-02-19T19:04:48.513 INFO:tasks.workunit.client.1.smithi132.stdout:5/159: symlink d8/d23/d29/d27/l32 0 2024-02-19T19:04:48.514 INFO:tasks.workunit.client.1.smithi132.stdout:0/144: dwrite d2/f2b [0,4194304] 0 2024-02-19T19:04:48.516 INFO:tasks.workunit.client.1.smithi132.stdout:6/113: unlink d0/db/d16/f28 0 2024-02-19T19:04:48.517 INFO:tasks.workunit.client.1.smithi132.stdout:0/145: readlink d2/dc/l2a 0 2024-02-19T19:04:48.518 INFO:tasks.workunit.client.1.smithi132.stdout:6/114: dread d0/db/fd [0,4194304] 0 2024-02-19T19:04:48.520 INFO:tasks.workunit.client.1.smithi132.stdout:9/247: write d12/d25/d3a/d40/f54 [5046277,84971] 0 2024-02-19T19:04:48.521 INFO:tasks.workunit.client.1.smithi132.stdout:9/248: write d12/d1b/f3b [389365,619] 0 2024-02-19T19:04:48.522 INFO:tasks.workunit.client.1.smithi132.stdout:0/146: mknod d2/d21/d25/c2e 0 2024-02-19T19:04:48.528 INFO:tasks.workunit.client.1.smithi132.stdout:5/160: rename d8/da/db/f20 to d8/da/f33 0 2024-02-19T19:04:48.530 INFO:tasks.workunit.client.1.smithi132.stdout:0/147: mkdir d2/dc/df/d15/d2f 0 2024-02-19T19:04:48.533 INFO:tasks.workunit.client.1.smithi132.stdout:6/115: rename d0/d1e/d22/c17 to d0/db/d25/c2c 0 2024-02-19T19:04:48.533 INFO:tasks.workunit.client.1.smithi132.stdout:5/161: dread d8/da/f33 [0,4194304] 0 2024-02-19T19:04:48.534 INFO:tasks.workunit.client.1.smithi132.stdout:0/148: symlink d2/dc/df/d15/l30 0 2024-02-19T19:04:48.537 INFO:tasks.workunit.client.1.smithi132.stdout:6/116: creat d0/dc/f2d x:0 0 0 2024-02-19T19:04:48.538 INFO:tasks.workunit.client.1.smithi132.stdout:5/162: rename d8/da/db/f28 to d8/da/db/f34 0 2024-02-19T19:04:48.539 INFO:tasks.workunit.client.1.smithi132.stdout:5/163: chown d8/d23/d29/f2d 29 1 2024-02-19T19:04:48.541 INFO:tasks.workunit.client.1.smithi132.stdout:0/149: rename d2/dc/l2a to d2/dc/df/l31 0 2024-02-19T19:04:48.542 INFO:tasks.workunit.client.1.smithi132.stdout:9/249: dread d12/d25/d3a/d40/f54 [0,4194304] 0 2024-02-19T19:04:48.543 INFO:tasks.workunit.client.1.smithi132.stdout:9/250: write d12/d25/f29 [3930362,21770] 0 2024-02-19T19:04:48.544 INFO:tasks.workunit.client.1.smithi132.stdout:5/164: mkdir d8/d1a/d35 0 2024-02-19T19:04:48.546 INFO:tasks.workunit.client.1.smithi132.stdout:5/165: truncate d8/d1a/f26 440170 0 2024-02-19T19:04:48.550 INFO:tasks.workunit.client.1.smithi132.stdout:9/251: dwrite d12/f26 [0,4194304] 0 2024-02-19T19:04:48.552 INFO:tasks.workunit.client.1.smithi132.stdout:0/150: dread d2/f6 [0,4194304] 0 2024-02-19T19:04:48.555 INFO:tasks.workunit.client.1.smithi132.stdout:9/252: mknod d12/d1b/c58 0 2024-02-19T19:04:48.558 INFO:tasks.workunit.client.1.smithi132.stdout:5/166: creat d8/d1a/d35/f36 x:0 0 0 2024-02-19T19:04:48.562 INFO:tasks.workunit.client.1.smithi132.stdout:9/253: symlink d12/d1b/l59 0 2024-02-19T19:04:48.564 INFO:tasks.workunit.client.1.smithi132.stdout:4/114: sync 2024-02-19T19:04:48.564 INFO:tasks.workunit.client.1.smithi132.stdout:1/119: sync 2024-02-19T19:04:48.564 INFO:tasks.workunit.client.1.smithi132.stdout:0/151: dwrite d2/f12 [0,4194304] 0 2024-02-19T19:04:48.565 INFO:tasks.workunit.client.1.smithi132.stdout:0/152: chown d2/d21/d25 5186022 1 2024-02-19T19:04:48.565 INFO:tasks.workunit.client.1.smithi132.stdout:4/115: rename d4/d12/d16 to d4/d12/d16/d20 22 2024-02-19T19:04:48.565 INFO:tasks.workunit.client.1.smithi132.stdout:5/167: mknod d8/d1a/c37 0 2024-02-19T19:04:48.566 INFO:tasks.workunit.client.1.smithi132.stdout:0/153: write d2/f12 [425944,26946] 0 2024-02-19T19:04:48.566 INFO:tasks.workunit.client.1.smithi132.stdout:5/168: chown d8/d23 43507703 1 2024-02-19T19:04:48.569 INFO:tasks.workunit.client.1.smithi132.stdout:9/254: creat d12/d1b/f5a x:0 0 0 2024-02-19T19:04:48.570 INFO:tasks.workunit.client.1.smithi132.stdout:1/120: rmdir d1 39 2024-02-19T19:04:48.571 INFO:tasks.workunit.client.1.smithi132.stdout:9/255: write d12/d3e/f4a [398832,31851] 0 2024-02-19T19:04:48.572 INFO:tasks.workunit.client.1.smithi132.stdout:9/256: readlink d12/l2d 0 2024-02-19T19:04:48.572 INFO:tasks.workunit.client.1.smithi132.stdout:0/154: dread d2/f2b [0,4194304] 0 2024-02-19T19:04:48.572 INFO:tasks.workunit.client.1.smithi132.stdout:4/116: mkdir d4/d19/d21 0 2024-02-19T19:04:48.574 INFO:tasks.workunit.client.1.smithi132.stdout:0/155: write d2/dc/df/d15/f22 [4123233,8133] 0 2024-02-19T19:04:48.575 INFO:tasks.workunit.client.1.smithi132.stdout:5/169: mknod d8/d23/c38 0 2024-02-19T19:04:48.576 INFO:tasks.workunit.client.1.smithi132.stdout:1/121: chown d1 845490 1 2024-02-19T19:04:48.577 INFO:tasks.workunit.client.1.smithi132.stdout:1/122: write d1/f18 [647665,86309] 0 2024-02-19T19:04:48.581 INFO:tasks.workunit.client.1.smithi132.stdout:9/257: chown d12/l46 0 1 2024-02-19T19:04:48.582 INFO:tasks.workunit.client.1.smithi132.stdout:4/117: write d4/f10 [2969037,38623] 0 2024-02-19T19:04:48.584 INFO:tasks.workunit.client.1.smithi132.stdout:0/156: fdatasync d2/f2b 0 2024-02-19T19:04:48.585 INFO:tasks.workunit.client.1.smithi132.stdout:1/123: dwrite d1/f21 [0,4194304] 0 2024-02-19T19:04:48.586 INFO:tasks.workunit.client.1.smithi132.stdout:4/118: write d4/d12/d16/f1b [567724,63990] 0 2024-02-19T19:04:48.587 INFO:tasks.workunit.client.1.smithi132.stdout:1/124: dread - d1/f1d zero size 2024-02-19T19:04:48.588 INFO:tasks.workunit.client.1.smithi132.stdout:4/119: read - d4/d12/f1a zero size 2024-02-19T19:04:48.600 INFO:tasks.workunit.client.1.smithi132.stdout:0/157: rmdir d2/d21 39 2024-02-19T19:04:48.605 INFO:tasks.workunit.client.1.smithi132.stdout:4/120: dread d4/f9 [4194304,4194304] 0 2024-02-19T19:04:48.606 INFO:tasks.workunit.client.1.smithi132.stdout:0/158: readlink d2/la 0 2024-02-19T19:04:48.608 INFO:tasks.workunit.client.1.smithi132.stdout:9/258: write f5 [1531031,17242] 0 2024-02-19T19:04:48.612 INFO:tasks.workunit.client.1.smithi132.stdout:5/170: getdents d8/da/db 0 2024-02-19T19:04:48.614 INFO:tasks.workunit.client.1.smithi132.stdout:0/159: symlink d2/dc/df/d15/l32 0 2024-02-19T19:04:48.615 INFO:tasks.workunit.client.1.smithi132.stdout:4/121: symlink d4/l22 0 2024-02-19T19:04:48.617 INFO:tasks.workunit.client.1.smithi132.stdout:1/125: dwrite d1/f15 [4194304,4194304] 0 2024-02-19T19:04:48.618 INFO:tasks.workunit.client.1.smithi132.stdout:5/171: rename d8/d1a/d35/f36 to d8/d2e/f39 0 2024-02-19T19:04:48.621 INFO:tasks.workunit.client.1.smithi132.stdout:9/259: link d12/d1b/f5a d12/d2a/f5b 0 2024-02-19T19:04:48.628 INFO:tasks.workunit.client.1.smithi132.stdout:5/172: mknod d8/d1a/c3a 0 2024-02-19T19:04:48.628 INFO:tasks.workunit.client.1.smithi132.stdout:9/260: symlink d12/d3e/l5c 0 2024-02-19T19:04:48.629 INFO:tasks.workunit.client.1.smithi132.stdout:0/160: mknod d2/d21/d25/c33 0 2024-02-19T19:04:48.640 INFO:tasks.workunit.client.1.smithi132.stdout:4/122: dwrite d4/f7 [4194304,4194304] 0 2024-02-19T19:04:48.641 INFO:tasks.workunit.client.1.smithi132.stdout:0/161: dread d2/d21/d25/f26 [0,4194304] 0 2024-02-19T19:04:48.642 INFO:tasks.workunit.client.1.smithi132.stdout:5/173: dwrite d8/d1a/f26 [0,4194304] 0 2024-02-19T19:04:48.643 INFO:tasks.workunit.client.1.smithi132.stdout:9/261: dwrite d12/d25/d3a/f52 [0,4194304] 0 2024-02-19T19:04:48.646 INFO:tasks.workunit.client.1.smithi132.stdout:4/123: creat d4/d19/f23 x:0 0 0 2024-02-19T19:04:48.648 INFO:tasks.workunit.client.1.smithi132.stdout:4/124: chown d4/l22 0 1 2024-02-19T19:04:48.648 INFO:tasks.workunit.client.1.smithi132.stdout:5/174: mkdir d8/d23/d3b 0 2024-02-19T19:04:48.651 INFO:tasks.workunit.client.1.smithi132.stdout:9/262: dread d12/f26 [0,4194304] 0 2024-02-19T19:04:48.652 INFO:tasks.workunit.client.1.smithi132.stdout:0/162: mkdir d2/dc/df/d15/d34 0 2024-02-19T19:04:48.653 INFO:tasks.workunit.client.1.smithi132.stdout:0/163: write d2/f2b [72983,27410] 0 2024-02-19T19:04:48.658 INFO:tasks.workunit.client.1.smithi132.stdout:9/263: creat d12/d25/d3a/d40/f5d x:0 0 0 2024-02-19T19:04:48.660 INFO:tasks.workunit.client.1.smithi132.stdout:0/164: creat d2/dc/df/d27/f35 x:0 0 0 2024-02-19T19:04:48.660 INFO:tasks.workunit.client.1.smithi132.stdout:0/165: chown d2/dc/df/d15/d34 35360983 1 2024-02-19T19:04:48.660 INFO:tasks.workunit.client.1.smithi132.stdout:4/125: mkdir d4/dc/d14/d24 0 2024-02-19T19:04:48.664 INFO:tasks.workunit.client.1.smithi132.stdout:0/166: truncate d2/d21/d25/f26 505912 0 2024-02-19T19:04:48.667 INFO:tasks.workunit.client.1.smithi132.stdout:4/126: symlink d4/dc/d14/d24/l25 0 2024-02-19T19:04:48.667 INFO:tasks.workunit.client.1.smithi132.stdout:4/127: fdatasync d4/d19/f23 0 2024-02-19T19:04:48.669 INFO:tasks.workunit.client.1.smithi132.stdout:0/167: chown d2/c14 7 1 2024-02-19T19:04:48.671 INFO:tasks.workunit.client.1.smithi132.stdout:4/128: mknod d4/dc/d14/d24/c26 0 2024-02-19T19:04:48.672 INFO:tasks.workunit.client.1.smithi132.stdout:0/168: mknod d2/d21/c36 0 2024-02-19T19:04:48.674 INFO:tasks.workunit.client.1.smithi132.stdout:0/169: fdatasync d2/f1a 0 2024-02-19T19:04:48.674 INFO:tasks.workunit.client.1.smithi132.stdout:0/170: stat d2/dc 0 2024-02-19T19:04:48.675 INFO:tasks.workunit.client.1.smithi132.stdout:4/129: mkdir d4/d27 0 2024-02-19T19:04:48.680 INFO:tasks.workunit.client.1.smithi132.stdout:4/130: dread d4/d12/d16/f1b [0,4194304] 0 2024-02-19T19:04:48.682 INFO:tasks.workunit.client.1.smithi132.stdout:9/264: dread d12/d1b/f17 [0,4194304] 0 2024-02-19T19:04:48.683 INFO:tasks.workunit.client.1.smithi132.stdout:4/131: mkdir d4/d12/d28 0 2024-02-19T19:04:48.687 INFO:tasks.workunit.client.1.smithi132.stdout:9/265: link d12/d25/c38 d12/d1b/c5e 0 2024-02-19T19:04:48.691 INFO:tasks.workunit.client.1.smithi132.stdout:4/132: dwrite d4/f9 [8388608,4194304] 0 2024-02-19T19:04:48.691 INFO:tasks.workunit.client.1.smithi132.stdout:9/266: creat d12/d1b/f5f x:0 0 0 2024-02-19T19:04:48.692 INFO:tasks.workunit.client.1.smithi132.stdout:4/133: write d4/d12/f1a [395474,117610] 0 2024-02-19T19:04:48.693 INFO:tasks.workunit.client.1.smithi132.stdout:9/267: mknod d12/d1b/c60 0 2024-02-19T19:04:48.694 INFO:tasks.workunit.client.1.smithi132.stdout:9/268: truncate d12/d2a/f5b 895163 0 2024-02-19T19:04:48.695 INFO:tasks.workunit.client.1.smithi132.stdout:4/134: creat d4/d12/d16/f29 x:0 0 0 2024-02-19T19:04:48.703 INFO:tasks.workunit.client.1.smithi132.stdout:4/135: dwrite d4/f10 [0,4194304] 0 2024-02-19T19:04:48.704 INFO:tasks.workunit.client.1.smithi132.stdout:9/269: mkdir d12/d2a/d4c/d61 0 2024-02-19T19:04:48.704 INFO:tasks.workunit.client.1.smithi132.stdout:7/195: sync 2024-02-19T19:04:48.706 INFO:tasks.workunit.client.1.smithi132.stdout:9/270: readlink d12/d1b/l14 0 2024-02-19T19:04:48.707 INFO:tasks.workunit.client.1.smithi132.stdout:4/136: symlink d4/d12/d16/l2a 0 2024-02-19T19:04:48.710 INFO:tasks.workunit.client.1.smithi132.stdout:4/137: chown d4/l13 0 1 2024-02-19T19:04:48.713 INFO:tasks.workunit.client.1.smithi132.stdout:4/138: symlink d4/d19/l2b 0 2024-02-19T19:04:48.714 INFO:tasks.workunit.client.1.smithi132.stdout:7/196: getdents d1b/d1e/d28 0 2024-02-19T19:04:48.714 INFO:tasks.workunit.client.1.smithi132.stdout:7/197: fdatasync d1b/d1e/f30 0 2024-02-19T19:04:48.718 INFO:tasks.workunit.client.1.smithi132.stdout:7/198: link l12 d1b/l40 0 2024-02-19T19:04:48.724 INFO:tasks.workunit.client.1.smithi132.stdout:7/199: dwrite d1b/d1e/d28/f2c [0,4194304] 0 2024-02-19T19:04:48.724 INFO:tasks.workunit.client.1.smithi132.stdout:9/271: dread d12/d2a/f2c [0,4194304] 0 2024-02-19T19:04:48.727 INFO:tasks.workunit.client.1.smithi132.stdout:7/200: mkdir d1b/d1e/d41 0 2024-02-19T19:04:48.767 INFO:tasks.workunit.client.1.smithi132.stdout:9/272: mknod d12/d2a/d4c/c62 0 2024-02-19T19:04:48.767 INFO:tasks.workunit.client.1.smithi132.stdout:7/201: chown d1b/d1e/f30 278873435 1 2024-02-19T19:04:48.768 INFO:tasks.workunit.client.1.smithi132.stdout:9/273: write d12/d1b/f5f [25960,24894] 0 2024-02-19T19:04:48.773 INFO:tasks.workunit.client.1.smithi132.stdout:9/274: creat d12/d2a/d4c/d61/f63 x:0 0 0 2024-02-19T19:04:48.773 INFO:tasks.workunit.client.1.smithi132.stdout:7/202: link d1b/d1e/f21 d1b/d32/f42 0 2024-02-19T19:04:48.775 INFO:tasks.workunit.client.1.smithi132.stdout:9/275: symlink d12/d2a/d4c/d61/l64 0 2024-02-19T19:04:48.776 INFO:tasks.workunit.client.1.smithi132.stdout:9/276: truncate d12/d1b/f3b 667535 0 2024-02-19T19:04:48.776 INFO:tasks.workunit.client.1.smithi132.stdout:7/203: creat d1b/d1e/f43 x:0 0 0 2024-02-19T19:04:48.777 INFO:tasks.workunit.client.1.smithi132.stdout:9/277: dread - d12/d25/d3a/d40/f49 zero size 2024-02-19T19:04:48.778 INFO:tasks.workunit.client.1.smithi132.stdout:7/204: creat d1b/d1e/d28/f44 x:0 0 0 2024-02-19T19:04:48.780 INFO:tasks.workunit.client.1.smithi132.stdout:7/205: mkdir d1b/d1e/d2b/d45 0 2024-02-19T19:04:48.782 INFO:tasks.workunit.client.1.smithi132.stdout:7/206: read d1b/f25 [1974612,101835] 0 2024-02-19T19:04:48.788 INFO:tasks.workunit.client.1.smithi132.stdout:7/207: truncate d1b/d1e/f30 10345 0 2024-02-19T19:04:48.790 INFO:tasks.workunit.client.1.smithi132.stdout:7/208: mkdir d1b/d46 0 2024-02-19T19:04:48.796 INFO:tasks.workunit.client.1.smithi132.stdout:7/209: dwrite d1b/d1e/d28/f3b [0,4194304] 0 2024-02-19T19:04:48.893 INFO:tasks.workunit.client.1.smithi132.stdout:8/179: sync 2024-02-19T19:04:48.894 INFO:tasks.workunit.client.1.smithi132.stdout:3/107: sync 2024-02-19T19:04:48.894 INFO:tasks.workunit.client.1.smithi132.stdout:2/150: sync 2024-02-19T19:04:48.899 INFO:tasks.workunit.client.1.smithi132.stdout:2/151: dread d0/d4/d11/f20 [0,4194304] 0 2024-02-19T19:04:48.899 INFO:tasks.workunit.client.1.smithi132.stdout:2/152: stat d0/d4/f22 0 2024-02-19T19:04:48.903 INFO:tasks.workunit.client.1.smithi132.stdout:8/180: truncate d4/dc/f3f 3068620 0 2024-02-19T19:04:48.904 INFO:tasks.workunit.client.1.smithi132.stdout:2/153: write d0/f6 [4549302,80581] 0 2024-02-19T19:04:48.904 INFO:tasks.workunit.client.1.smithi132.stdout:2/154: write d0/f6 [4583291,78163] 0 2024-02-19T19:04:48.910 INFO:tasks.workunit.client.1.smithi132.stdout:2/155: dwrite d0/d4/fb [0,4194304] 0 2024-02-19T19:04:48.916 INFO:tasks.workunit.client.1.smithi132.stdout:2/156: link d0/d19/f25 d0/d24/f39 0 2024-02-19T19:04:48.926 INFO:tasks.workunit.client.1.smithi132.stdout:2/157: link d0/c13 d0/d4/df/c3a 0 2024-02-19T19:04:48.927 INFO:tasks.workunit.client.1.smithi132.stdout:2/158: creat d0/d4/df/f3b x:0 0 0 2024-02-19T19:04:48.931 INFO:tasks.workunit.client.1.smithi132.stdout:2/159: mkdir d0/d3c 0 2024-02-19T19:04:48.934 INFO:tasks.workunit.client.1.smithi132.stdout:2/160: unlink d0/d4/df/f21 0 2024-02-19T19:04:48.937 INFO:tasks.workunit.client.1.smithi132.stdout:2/161: dread d0/d4/d11/f1c [0,4194304] 0 2024-02-19T19:04:48.938 INFO:tasks.workunit.client.1.smithi132.stdout:2/162: write d0/f5 [286504,79889] 0 2024-02-19T19:04:48.950 INFO:tasks.workunit.client.1.smithi132.stdout:2/163: write d0/d4/d11/f15 [3456350,78430] 0 2024-02-19T19:04:48.951 INFO:tasks.workunit.client.1.smithi132.stdout:2/164: write d0/d4/d11/f15 [3336386,59450] 0 2024-02-19T19:04:48.958 INFO:tasks.workunit.client.1.smithi132.stdout:2/165: link d0/d4/c10 d0/d24/c3d 0 2024-02-19T19:04:48.963 INFO:tasks.workunit.client.1.smithi132.stdout:2/166: dwrite d0/f5 [0,4194304] 0 2024-02-19T19:04:48.966 INFO:tasks.workunit.client.1.smithi132.stdout:2/167: unlink d0/c1d 0 2024-02-19T19:04:48.967 INFO:tasks.workunit.client.1.smithi132.stdout:2/168: write d0/d4/df/f3b [807568,62428] 0 2024-02-19T19:04:48.968 INFO:tasks.workunit.client.1.smithi132.stdout:2/169: chown d0/d19/f25 756722 1 2024-02-19T19:04:48.969 INFO:tasks.workunit.client.1.smithi132.stdout:9/278: sync 2024-02-19T19:04:48.969 INFO:tasks.workunit.client.1.smithi132.stdout:4/139: sync 2024-02-19T19:04:48.970 INFO:tasks.workunit.client.1.smithi132.stdout:0/171: sync 2024-02-19T19:04:48.970 INFO:tasks.workunit.client.1.smithi132.stdout:7/210: sync 2024-02-19T19:04:48.970 INFO:tasks.workunit.client.1.smithi132.stdout:5/175: sync 2024-02-19T19:04:48.970 INFO:tasks.workunit.client.1.smithi132.stdout:6/117: sync 2024-02-19T19:04:48.970 INFO:tasks.workunit.client.1.smithi132.stdout:1/126: sync 2024-02-19T19:04:48.970 INFO:tasks.workunit.client.1.smithi132.stdout:2/170: write d0/d4/d33/f36 [4200068,125838] 0 2024-02-19T19:04:48.970 INFO:tasks.workunit.client.1.smithi132.stdout:9/279: chown d12 10 1 2024-02-19T19:04:48.973 INFO:tasks.workunit.client.1.smithi132.stdout:9/280: dread d12/f33 [0,4194304] 0 2024-02-19T19:04:48.980 INFO:tasks.workunit.client.1.smithi132.stdout:6/118: creat d0/db/d16/f2e x:0 0 0 2024-02-19T19:04:48.984 INFO:tasks.workunit.client.1.smithi132.stdout:5/176: dread d8/d23/d29/f1c [0,4194304] 0 2024-02-19T19:04:48.986 INFO:tasks.workunit.client.1.smithi132.stdout:7/211: getdents d1b/d46 0 2024-02-19T19:04:48.991 INFO:tasks.workunit.client.1.smithi132.stdout:9/281: dread d12/f1a [0,4194304] 0 2024-02-19T19:04:48.991 INFO:tasks.workunit.client.1.smithi132.stdout:9/282: chown f5 0 1 2024-02-19T19:04:48.992 INFO:tasks.workunit.client.1.smithi132.stdout:1/127: rename d1/lc to d1/d19/l26 0 2024-02-19T19:04:48.996 INFO:tasks.workunit.client.1.smithi132.stdout:9/283: dwrite d12/d25/d3a/f52 [0,4194304] 0 2024-02-19T19:04:49.003 INFO:tasks.workunit.client.1.smithi132.stdout:2/171: creat d0/d4/d11/d2e/f3e x:0 0 0 2024-02-19T19:04:49.004 INFO:tasks.workunit.client.1.smithi132.stdout:7/212: fdatasync d1b/d1e/f30 0 2024-02-19T19:04:49.007 INFO:tasks.workunit.client.1.smithi132.stdout:9/284: dwrite d12/d2a/d4c/d61/f63 [0,4194304] 0 2024-02-19T19:04:49.009 INFO:tasks.workunit.client.1.smithi132.stdout:9/285: truncate d12/d25/d3a/d40/f51 231881 0 2024-02-19T19:04:49.011 INFO:tasks.workunit.client.1.smithi132.stdout:3/108: sync 2024-02-19T19:04:49.012 INFO:tasks.workunit.client.1.smithi132.stdout:2/172: chown d0/d19/d1e/c29 3 1 2024-02-19T19:04:49.014 INFO:tasks.workunit.client.1.smithi132.stdout:1/128: creat d1/f27 x:0 0 0 2024-02-19T19:04:49.015 INFO:tasks.workunit.client.1.smithi132.stdout:1/129: chown d1/f17 185509357 1 2024-02-19T19:04:49.019 INFO:tasks.workunit.client.1.smithi132.stdout:4/140: truncate d4/f9 6558655 0 2024-02-19T19:04:49.020 INFO:tasks.workunit.client.1.smithi132.stdout:0/172: truncate d2/dc/df/d15/f22 938627 0 2024-02-19T19:04:49.021 INFO:tasks.workunit.client.1.smithi132.stdout:1/130: mknod d1/d1e/c28 0 2024-02-19T19:04:49.022 INFO:tasks.workunit.client.1.smithi132.stdout:6/119: truncate d0/f5 2204751 0 2024-02-19T19:04:49.023 INFO:tasks.workunit.client.1.smithi132.stdout:3/109: read f0 [150308,97692] 0 2024-02-19T19:04:49.025 INFO:tasks.workunit.client.1.smithi132.stdout:4/141: dread d4/d12/f1a [0,4194304] 0 2024-02-19T19:04:49.026 INFO:tasks.workunit.client.1.smithi132.stdout:1/131: truncate d1/f1b 701300 0 2024-02-19T19:04:49.028 INFO:tasks.workunit.client.1.smithi132.stdout:2/173: write d0/d19/f25 [810921,68488] 0 2024-02-19T19:04:49.029 INFO:tasks.workunit.client.1.smithi132.stdout:3/110: symlink d8/l1c 0 2024-02-19T19:04:49.031 INFO:tasks.workunit.client.1.smithi132.stdout:5/177: dwrite d8/fe [0,4194304] 0 2024-02-19T19:04:49.033 INFO:tasks.workunit.client.1.smithi132.stdout:1/132: unlink d1/f16 0 2024-02-19T19:04:49.038 INFO:tasks.workunit.client.1.smithi132.stdout:2/174: stat d0/c2 0 2024-02-19T19:04:49.038 INFO:tasks.workunit.client.1.smithi132.stdout:0/173: dwrite d2/dc/df/d15/f24 [0,4194304] 0 2024-02-19T19:04:49.039 INFO:tasks.workunit.client.1.smithi132.stdout:9/286: dread fb [0,4194304] 0 2024-02-19T19:04:49.039 INFO:tasks.workunit.client.1.smithi132.stdout:3/111: symlink d8/d13/d19/l1d 0 2024-02-19T19:04:49.039 INFO:tasks.workunit.client.1.smithi132.stdout:4/142: dwrite d4/f10 [0,4194304] 0 2024-02-19T19:04:49.041 INFO:tasks.workunit.client.1.smithi132.stdout:9/287: chown d12/f33 153053 1 2024-02-19T19:04:49.045 INFO:tasks.workunit.client.1.smithi132.stdout:0/174: creat d2/dc/f37 x:0 0 0 2024-02-19T19:04:49.045 INFO:tasks.workunit.client.1.smithi132.stdout:2/175: write d0/d4/d11/f1c [544804,19822] 0 2024-02-19T19:04:49.046 INFO:tasks.workunit.client.1.smithi132.stdout:0/175: chown d2/dc/df/d15/d2f 7 1 2024-02-19T19:04:49.048 INFO:tasks.workunit.client.1.smithi132.stdout:0/176: write d2/f12 [174892,45335] 0 2024-02-19T19:04:49.050 INFO:tasks.workunit.client.1.smithi132.stdout:9/288: creat d12/d25/d3a/d40/f65 x:0 0 0 2024-02-19T19:04:49.051 INFO:tasks.workunit.client.1.smithi132.stdout:4/143: getdents d4/d12/d28 0 2024-02-19T19:04:49.053 INFO:tasks.workunit.client.1.smithi132.stdout:8/181: sync 2024-02-19T19:04:49.054 INFO:tasks.workunit.client.1.smithi132.stdout:8/182: write d4/dc/d1b/d1d/f36 [915342,127246] 0 2024-02-19T19:04:49.055 INFO:tasks.workunit.client.1.smithi132.stdout:2/176: dread d0/d4/f22 [0,4194304] 0 2024-02-19T19:04:49.056 INFO:tasks.workunit.client.1.smithi132.stdout:7/213: dread d1b/d1e/f21 [0,4194304] 0 2024-02-19T19:04:49.058 INFO:tasks.workunit.client.1.smithi132.stdout:7/214: write d1b/d1e/d28/f44 [658940,80275] 0 2024-02-19T19:04:49.060 INFO:tasks.workunit.client.1.smithi132.stdout:2/177: dread d0/f5 [0,4194304] 0 2024-02-19T19:04:49.062 INFO:tasks.workunit.client.1.smithi132.stdout:5/178: link d8/d23/d29/d27/f2b d8/da/f3c 0 2024-02-19T19:04:49.063 INFO:tasks.workunit.client.1.smithi132.stdout:9/289: fdatasync d12/f33 0 2024-02-19T19:04:49.064 INFO:tasks.workunit.client.1.smithi132.stdout:7/215: dread d1b/d1e/d28/f2c [0,4194304] 0 2024-02-19T19:04:49.067 INFO:tasks.workunit.client.1.smithi132.stdout:9/290: dread d12/d1b/f3b [0,4194304] 0 2024-02-19T19:04:49.069 INFO:tasks.workunit.client.1.smithi132.stdout:7/216: dread d1b/d32/f42 [0,4194304] 0 2024-02-19T19:04:49.072 INFO:tasks.workunit.client.1.smithi132.stdout:4/144: chown d4/f9 0 1 2024-02-19T19:04:49.085 INFO:tasks.workunit.client.1.smithi132.stdout:3/112: truncate d8/f12 3968605 0 2024-02-19T19:04:49.085 INFO:tasks.workunit.client.1.smithi132.stdout:4/145: creat d4/d12/f2c x:0 0 0 2024-02-19T19:04:49.086 INFO:tasks.workunit.client.1.smithi132.stdout:4/146: truncate d4/d12/f1a 1207702 0 2024-02-19T19:04:49.088 INFO:tasks.workunit.client.1.smithi132.stdout:2/178: dread d0/d4/f1f [0,4194304] 0 2024-02-19T19:04:49.089 INFO:tasks.workunit.client.1.smithi132.stdout:5/179: symlink d8/d1a/d1e/l3d 0 2024-02-19T19:04:49.089 INFO:tasks.workunit.client.1.smithi132.stdout:0/177: write d2/d21/d25/f26 [376745,120334] 0 2024-02-19T19:04:49.091 INFO:tasks.workunit.client.1.smithi132.stdout:5/180: write d8/d23/d29/f2d [523241,122940] 0 2024-02-19T19:04:49.092 INFO:tasks.workunit.client.1.smithi132.stdout:8/183: rename d4/dc/d1b/d1d/f1e to d4/dc/d1b/f43 0 2024-02-19T19:04:49.093 INFO:tasks.workunit.client.1.smithi132.stdout:7/217: creat d1b/d1e/d41/f47 x:0 0 0 2024-02-19T19:04:49.094 INFO:tasks.workunit.client.1.smithi132.stdout:3/113: dread f0 [0,4194304] 0 2024-02-19T19:04:49.095 INFO:tasks.workunit.client.1.smithi132.stdout:8/184: chown d4/dc/d1b/f32 991248 1 2024-02-19T19:04:49.097 INFO:tasks.workunit.client.1.smithi132.stdout:2/179: unlink d0/d24/f34 0 2024-02-19T19:04:49.099 INFO:tasks.workunit.client.1.smithi132.stdout:0/178: dwrite d2/dc/df/d27/f35 [0,4194304] 0 2024-02-19T19:04:49.099 INFO:tasks.workunit.client.1.smithi132.stdout:4/147: fdatasync d4/fa 0 2024-02-19T19:04:49.104 INFO:tasks.workunit.client.1.smithi132.stdout:8/185: creat d4/dc/f44 x:0 0 0 2024-02-19T19:04:49.106 INFO:tasks.workunit.client.1.smithi132.stdout:3/114: rename d8/l1c to d8/d13/d19/l1e 0 2024-02-19T19:04:49.108 INFO:tasks.workunit.client.1.smithi132.stdout:7/218: dwrite d1b/d1e/f3d [0,4194304] 0 2024-02-19T19:04:49.109 INFO:tasks.workunit.client.1.smithi132.stdout:8/186: fsync d4/dc/d1b/d1d/f21 0 2024-02-19T19:04:49.111 INFO:tasks.workunit.client.1.smithi132.stdout:3/115: write d8/f14 [1006094,52312] 0 2024-02-19T19:04:49.112 INFO:tasks.workunit.client.1.smithi132.stdout:9/291: truncate d12/d2a/f5b 884745 0 2024-02-19T19:04:49.114 INFO:tasks.workunit.client.1.smithi132.stdout:5/181: dwrite d8/da/db/f34 [0,4194304] 0 2024-02-19T19:04:49.116 INFO:tasks.workunit.client.1.smithi132.stdout:4/148: dwrite d4/d19/f23 [0,4194304] 0 2024-02-19T19:04:49.120 INFO:tasks.workunit.client.1.smithi132.stdout:2/180: mkdir d0/d3c/d3f 0 2024-02-19T19:04:49.126 INFO:tasks.workunit.client.1.smithi132.stdout:5/182: symlink d8/da/db/l3e 0 2024-02-19T19:04:49.129 INFO:tasks.workunit.client.1.smithi132.stdout:2/181: mkdir d0/d4/df/d40 0 2024-02-19T19:04:49.131 INFO:tasks.workunit.client.1.smithi132.stdout:7/219: mkdir d1b/d46/d48 0 2024-02-19T19:04:49.133 INFO:tasks.workunit.client.1.smithi132.stdout:5/183: dwrite d8/da/db/f1d [0,4194304] 0 2024-02-19T19:04:49.136 INFO:tasks.workunit.client.1.smithi132.stdout:0/179: write d2/f6 [2996885,11357] 0 2024-02-19T19:04:49.139 INFO:tasks.workunit.client.1.smithi132.stdout:0/180: chown d2/dc/df/d27/f35 214589 1 2024-02-19T19:04:49.142 INFO:tasks.workunit.client.1.smithi132.stdout:5/184: dwrite d8/fe [0,4194304] 0 2024-02-19T19:04:49.145 INFO:tasks.workunit.client.1.smithi132.stdout:2/182: dread d0/d24/f39 [0,4194304] 0 2024-02-19T19:04:49.146 INFO:tasks.workunit.client.1.smithi132.stdout:3/116: write d8/f9 [1851362,96467] 0 2024-02-19T19:04:49.146 INFO:tasks.workunit.client.1.smithi132.stdout:4/149: creat d4/f2d x:0 0 0 2024-02-19T19:04:49.147 INFO:tasks.workunit.client.1.smithi132.stdout:0/181: dread d2/dc/fe [0,4194304] 0 2024-02-19T19:04:49.148 INFO:tasks.workunit.client.1.smithi132.stdout:2/183: write d0/d4/df/f3b [699897,93086] 0 2024-02-19T19:04:49.149 INFO:tasks.workunit.client.1.smithi132.stdout:7/220: symlink d1b/d46/d48/l49 0 2024-02-19T19:04:49.150 INFO:tasks.workunit.client.1.smithi132.stdout:5/185: mkdir d8/d23/d29/d3f 0 2024-02-19T19:04:49.153 INFO:tasks.workunit.client.1.smithi132.stdout:1/133: sync 2024-02-19T19:04:49.153 INFO:tasks.workunit.client.1.smithi132.stdout:6/120: sync 2024-02-19T19:04:49.153 INFO:tasks.workunit.client.1.smithi132.stdout:6/121: readlink d0/dc/l1a 0 2024-02-19T19:04:49.154 INFO:tasks.workunit.client.1.smithi132.stdout:1/134: write d1/f18 [815526,34863] 0 2024-02-19T19:04:49.155 INFO:tasks.workunit.client.1.smithi132.stdout:1/135: write d1/f17 [5542484,63094] 0 2024-02-19T19:04:49.157 INFO:tasks.workunit.client.1.smithi132.stdout:4/150: fsync d4/f9 0 2024-02-19T19:04:49.157 INFO:tasks.workunit.client.1.smithi132.stdout:0/182: creat d2/d21/d25/f38 x:0 0 0 2024-02-19T19:04:49.160 INFO:tasks.workunit.client.1.smithi132.stdout:7/221: unlink d1b/d1e/d28/f2c 0 2024-02-19T19:04:49.161 INFO:tasks.workunit.client.1.smithi132.stdout:5/186: fsync f6 0 2024-02-19T19:04:49.163 INFO:tasks.workunit.client.1.smithi132.stdout:5/187: stat d8/d23/d29/d27 0 2024-02-19T19:04:49.164 INFO:tasks.workunit.client.1.smithi132.stdout:2/184: dwrite d0/d4/f1f [0,4194304] 0 2024-02-19T19:04:49.165 INFO:tasks.workunit.client.1.smithi132.stdout:6/122: mkdir d0/db/d25/d2f 0 2024-02-19T19:04:49.167 INFO:tasks.workunit.client.1.smithi132.stdout:4/151: mknod d4/dc/c2e 0 2024-02-19T19:04:49.172 INFO:tasks.workunit.client.1.smithi132.stdout:9/292: truncate d12/d1b/f20 1820694 0 2024-02-19T19:04:49.173 INFO:tasks.workunit.client.1.smithi132.stdout:1/136: dread d1/f2 [0,4194304] 0 2024-02-19T19:04:49.174 INFO:tasks.workunit.client.1.smithi132.stdout:8/187: dwrite d4/d2d/f31 [0,4194304] 0 2024-02-19T19:04:49.175 INFO:tasks.workunit.client.1.smithi132.stdout:7/222: symlink d1b/d1e/l4a 0 2024-02-19T19:04:49.179 INFO:tasks.workunit.client.1.smithi132.stdout:3/117: write d8/d13/f16 [1841045,106984] 0 2024-02-19T19:04:49.182 INFO:tasks.workunit.client.1.smithi132.stdout:6/123: symlink d0/db/l30 0 2024-02-19T19:04:49.186 INFO:tasks.workunit.client.1.smithi132.stdout:4/152: creat d4/d12/f2f x:0 0 0 2024-02-19T19:04:49.187 INFO:tasks.workunit.client.1.smithi132.stdout:0/183: mknod d2/dc/df/d15/d2f/c39 0 2024-02-19T19:04:49.189 INFO:tasks.workunit.client.1.smithi132.stdout:5/188: dwrite d8/da/f30 [0,4194304] 0 2024-02-19T19:04:49.189 INFO:tasks.workunit.client.1.smithi132.stdout:7/223: dwrite f10 [0,4194304] 0 2024-02-19T19:04:49.192 INFO:tasks.workunit.client.1.smithi132.stdout:2/185: dwrite d0/d19/f25 [0,4194304] 0 2024-02-19T19:04:49.195 INFO:tasks.workunit.client.1.smithi132.stdout:2/186: write d0/f6 [8116803,90016] 0 2024-02-19T19:04:49.195 INFO:tasks.workunit.client.1.smithi132.stdout:1/137: mkdir d1/d1e/d29 0 2024-02-19T19:04:49.197 INFO:tasks.workunit.client.1.smithi132.stdout:3/118: symlink d8/d13/l1f 0 2024-02-19T19:04:49.203 INFO:tasks.workunit.client.1.smithi132.stdout:2/187: dwrite d0/d4/d11/d2e/f3e [0,4194304] 0 2024-02-19T19:04:49.206 INFO:tasks.workunit.client.1.smithi132.stdout:4/153: creat d4/d12/f30 x:0 0 0 2024-02-19T19:04:49.206 INFO:tasks.workunit.client.1.smithi132.stdout:4/154: readlink d4/d12/d16/l2a 0 2024-02-19T19:04:49.209 INFO:tasks.workunit.client.1.smithi132.stdout:0/184: mkdir d2/dc/df/d27/d3a 0 2024-02-19T19:04:49.211 INFO:tasks.workunit.client.1.smithi132.stdout:5/189: unlink d8/d23/c38 0 2024-02-19T19:04:49.211 INFO:tasks.workunit.client.1.smithi132.stdout:5/190: readlink d8/da/db/l3e 0 2024-02-19T19:04:49.213 INFO:tasks.workunit.client.1.smithi132.stdout:7/224: mknod d1b/d1e/c4b 0 2024-02-19T19:04:49.220 INFO:tasks.workunit.client.1.smithi132.stdout:1/138: creat d1/d1e/f2a x:0 0 0 2024-02-19T19:04:49.221 INFO:tasks.workunit.client.1.smithi132.stdout:9/293: write d12/d25/f35 [4256934,130938] 0 2024-02-19T19:04:49.221 INFO:tasks.workunit.client.1.smithi132.stdout:1/139: write d1/f1d [195856,52604] 0 2024-02-19T19:04:49.223 INFO:tasks.workunit.client.1.smithi132.stdout:3/119: creat d8/d13/f20 x:0 0 0 2024-02-19T19:04:49.223 INFO:tasks.workunit.client.1.smithi132.stdout:3/120: fsync d8/f9 0 2024-02-19T19:04:49.225 INFO:tasks.workunit.client.1.smithi132.stdout:6/124: creat d0/db/d25/d2f/f31 x:0 0 0 2024-02-19T19:04:49.226 INFO:tasks.workunit.client.1.smithi132.stdout:6/125: read - d0/db/d16/f26 zero size 2024-02-19T19:04:49.226 INFO:tasks.workunit.client.1.smithi132.stdout:2/188: truncate d0/f5 854935 0 2024-02-19T19:04:49.228 INFO:tasks.workunit.client.1.smithi132.stdout:0/185: creat d2/dc/f3b x:0 0 0 2024-02-19T19:04:49.228 INFO:tasks.workunit.client.1.smithi132.stdout:5/191: write f5 [3832180,88174] 0 2024-02-19T19:04:49.231 INFO:tasks.workunit.client.1.smithi132.stdout:9/294: creat d12/d25/d3a/d40/f66 x:0 0 0 2024-02-19T19:04:49.231 INFO:tasks.workunit.client.1.smithi132.stdout:1/140: truncate d1/f5 74195 0 2024-02-19T19:04:49.233 INFO:tasks.workunit.client.1.smithi132.stdout:3/121: chown l5 0 1 2024-02-19T19:04:49.235 INFO:tasks.workunit.client.1.smithi132.stdout:6/126: fdatasync d0/f5 0 2024-02-19T19:04:49.236 INFO:tasks.workunit.client.1.smithi132.stdout:7/225: dwrite d1b/d1e/d2b/f2f [0,4194304] 0 2024-02-19T19:04:49.237 INFO:tasks.workunit.client.1.smithi132.stdout:7/226: fsync d1b/d1e/d28/f3b 0 2024-02-19T19:04:49.239 INFO:tasks.workunit.client.1.smithi132.stdout:7/227: truncate d1b/d1e/d28/f44 976489 0 2024-02-19T19:04:49.242 INFO:tasks.workunit.client.1.smithi132.stdout:9/295: creat d12/d25/d3a/d40/d4e/f67 x:0 0 0 2024-02-19T19:04:49.243 INFO:tasks.workunit.client.1.smithi132.stdout:9/296: chown d12/d25/l31 102154 1 2024-02-19T19:04:49.253 INFO:tasks.workunit.client.1.smithi132.stdout:8/188: dread d4/fd [0,4194304] 0 2024-02-19T19:04:49.253 INFO:tasks.workunit.client.1.smithi132.stdout:1/141: mknod d1/d1e/d29/c2b 0 2024-02-19T19:04:49.255 INFO:tasks.workunit.client.1.smithi132.stdout:5/192: write d8/da/f3c [739939,73451] 0 2024-02-19T19:04:49.256 INFO:tasks.workunit.client.1.smithi132.stdout:8/189: chown d4/dc/d1b/d33/c3d 43087 1 2024-02-19T19:04:49.258 INFO:tasks.workunit.client.1.smithi132.stdout:4/155: dwrite d4/fa [0,4194304] 0 2024-02-19T19:04:49.258 INFO:tasks.workunit.client.1.smithi132.stdout:5/193: fdatasync d8/da/db/f34 0 2024-02-19T19:04:49.258 INFO:tasks.workunit.client.1.smithi132.stdout:8/190: write d4/dc/f3f [279319,111432] 0 2024-02-19T19:04:49.259 INFO:tasks.workunit.client.1.smithi132.stdout:2/189: link d0/d4/d11/f1c d0/d4/d11/f41 0 2024-02-19T19:04:49.260 INFO:tasks.workunit.client.1.smithi132.stdout:7/228: creat d1b/d1e/d28/f4c x:0 0 0 2024-02-19T19:04:49.260 INFO:tasks.workunit.client.1.smithi132.stdout:2/190: chown d0/d19 5649420 1 2024-02-19T19:04:49.262 INFO:tasks.workunit.client.1.smithi132.stdout:7/229: readlink d1b/d1e/l2d 0 2024-02-19T19:04:49.263 INFO:tasks.workunit.client.1.smithi132.stdout:0/186: write d2/dc/df/d15/f22 [376558,40202] 0 2024-02-19T19:04:49.265 INFO:tasks.workunit.client.1.smithi132.stdout:0/187: stat d2/f6 0 2024-02-19T19:04:49.267 INFO:tasks.workunit.client.1.smithi132.stdout:4/156: dwrite d4/f7 [0,4194304] 0 2024-02-19T19:04:49.269 INFO:tasks.workunit.client.1.smithi132.stdout:2/191: dread d0/d4/d11/d2e/f3e [0,4194304] 0 2024-02-19T19:04:49.274 INFO:tasks.workunit.client.1.smithi132.stdout:5/194: creat d8/d1a/d35/f40 x:0 0 0 2024-02-19T19:04:49.279 INFO:tasks.workunit.client.1.smithi132.stdout:8/191: fdatasync d4/dc/d1b/f43 0 2024-02-19T19:04:49.279 INFO:tasks.workunit.client.1.smithi132.stdout:3/122: link d8/l18 d8/d13/l21 0 2024-02-19T19:04:49.280 INFO:tasks.workunit.client.1.smithi132.stdout:5/195: creat d8/d1a/f41 x:0 0 0 2024-02-19T19:04:49.281 INFO:tasks.workunit.client.1.smithi132.stdout:4/157: dwrite d4/fa [0,4194304] 0 2024-02-19T19:04:49.283 INFO:tasks.workunit.client.1.smithi132.stdout:2/192: dread d0/d19/f25 [0,4194304] 0 2024-02-19T19:04:49.284 INFO:tasks.workunit.client.1.smithi132.stdout:1/142: dwrite d1/f2 [0,4194304] 0 2024-02-19T19:04:49.285 INFO:tasks.workunit.client.1.smithi132.stdout:0/188: creat d2/dc/df/d15/d34/f3c x:0 0 0 2024-02-19T19:04:49.289 INFO:tasks.workunit.client.1.smithi132.stdout:3/123: rename d8/f9 to d8/f22 0 2024-02-19T19:04:49.292 INFO:tasks.workunit.client.1.smithi132.stdout:3/124: fdatasync d8/d13/f20 0 2024-02-19T19:04:49.293 INFO:tasks.workunit.client.1.smithi132.stdout:4/158: dwrite d4/d12/f2f [0,4194304] 0 2024-02-19T19:04:49.294 INFO:tasks.workunit.client.1.smithi132.stdout:0/189: dread d2/f2b [0,4194304] 0 2024-02-19T19:04:49.295 INFO:tasks.workunit.client.1.smithi132.stdout:0/190: stat d2/f2b 0 2024-02-19T19:04:49.297 INFO:tasks.workunit.client.1.smithi132.stdout:7/230: truncate d1b/d1e/d28/f3c 1491984 0 2024-02-19T19:04:49.298 INFO:tasks.workunit.client.1.smithi132.stdout:7/231: write d1b/d1e/f22 [8098217,102200] 0 2024-02-19T19:04:49.299 INFO:tasks.workunit.client.1.smithi132.stdout:5/196: dread d8/da/f24 [0,4194304] 0 2024-02-19T19:04:49.299 INFO:tasks.workunit.client.1.smithi132.stdout:2/193: mknod d0/d19/d1e/c42 0 2024-02-19T19:04:49.302 INFO:tasks.workunit.client.1.smithi132.stdout:1/143: mkdir d1/d1e/d29/d2c 0 2024-02-19T19:04:49.303 INFO:tasks.workunit.client.1.smithi132.stdout:8/192: mkdir d4/dc/d1b/d38/d45 0 2024-02-19T19:04:49.305 INFO:tasks.workunit.client.1.smithi132.stdout:3/125: mknod d8/d13/d19/c23 0 2024-02-19T19:04:49.307 INFO:tasks.workunit.client.1.smithi132.stdout:8/193: dread d4/dc/f3f [0,4194304] 0 2024-02-19T19:04:49.314 INFO:tasks.workunit.client.1.smithi132.stdout:8/194: dwrite d4/dc/d1b/d1d/f21 [0,4194304] 0 2024-02-19T19:04:49.315 INFO:tasks.workunit.client.1.smithi132.stdout:8/195: stat d4/fd 0 2024-02-19T19:04:49.316 INFO:tasks.workunit.client.1.smithi132.stdout:1/144: unlink d1/f27 0 2024-02-19T19:04:49.319 INFO:tasks.workunit.client.1.smithi132.stdout:0/191: mknod d2/c3d 0 2024-02-19T19:04:49.322 INFO:tasks.workunit.client.1.smithi132.stdout:2/194: dwrite d0/d19/f25 [0,4194304] 0 2024-02-19T19:04:49.327 INFO:tasks.workunit.client.1.smithi132.stdout:5/197: creat d8/d23/d29/d3f/f42 x:0 0 0 2024-02-19T19:04:49.329 INFO:tasks.workunit.client.1.smithi132.stdout:8/196: creat d4/d2d/f46 x:0 0 0 2024-02-19T19:04:49.334 INFO:tasks.workunit.client.1.smithi132.stdout:3/126: unlink d8/d13/d19/l1e 0 2024-02-19T19:04:49.339 INFO:tasks.workunit.client.1.smithi132.stdout:2/195: creat d0/f43 x:0 0 0 2024-02-19T19:04:49.340 INFO:tasks.workunit.client.1.smithi132.stdout:2/196: write d0/d24/f39 [744554,39405] 0 2024-02-19T19:04:49.341 INFO:tasks.workunit.client.1.smithi132.stdout:5/198: creat d8/d23/f43 x:0 0 0 2024-02-19T19:04:49.342 INFO:tasks.workunit.client.1.smithi132.stdout:2/197: chown d0/d19/f2c 428 1 2024-02-19T19:04:49.344 INFO:tasks.workunit.client.1.smithi132.stdout:7/232: dwrite d1b/d1e/d28/f3c [0,4194304] 0 2024-02-19T19:04:49.344 INFO:tasks.workunit.client.1.smithi132.stdout:1/145: creat d1/d1e/d29/d2c/f2d x:0 0 0 2024-02-19T19:04:49.345 INFO:tasks.workunit.client.1.smithi132.stdout:2/198: write d0/d19/f2c [145324,3840] 0 2024-02-19T19:04:49.346 INFO:tasks.workunit.client.1.smithi132.stdout:3/127: creat d8/d13/f24 x:0 0 0 2024-02-19T19:04:49.352 INFO:tasks.workunit.client.1.smithi132.stdout:8/197: dwrite d4/d2d/f31 [0,4194304] 0 2024-02-19T19:04:49.357 INFO:tasks.workunit.client.1.smithi132.stdout:1/146: creat d1/d1e/d29/f2e x:0 0 0 2024-02-19T19:04:49.358 INFO:tasks.workunit.client.1.smithi132.stdout:1/147: write d1/f18 [58874,68769] 0 2024-02-19T19:04:49.360 INFO:tasks.workunit.client.1.smithi132.stdout:3/128: truncate f0 1226665 0 2024-02-19T19:04:49.363 INFO:tasks.workunit.client.1.smithi132.stdout:8/198: rename c3 to d4/dc/d1b/d20/c47 0 2024-02-19T19:04:49.365 INFO:tasks.workunit.client.1.smithi132.stdout:0/192: truncate d2/f12 67866 0 2024-02-19T19:04:49.367 INFO:tasks.workunit.client.1.smithi132.stdout:7/233: unlink d1b/d32/c36 0 2024-02-19T19:04:49.368 INFO:tasks.workunit.client.1.smithi132.stdout:8/199: dwrite d4/d2d/f31 [0,4194304] 0 2024-02-19T19:04:49.369 INFO:tasks.workunit.client.1.smithi132.stdout:7/234: chown d1b/d1e/c23 98892771 1 2024-02-19T19:04:49.370 INFO:tasks.workunit.client.1.smithi132.stdout:7/235: read d1b/d1e/f21 [3802245,14528] 0 2024-02-19T19:04:49.372 INFO:tasks.workunit.client.1.smithi132.stdout:2/199: write d0/d4/d11/f41 [849906,27214] 0 2024-02-19T19:04:49.374 INFO:tasks.workunit.client.1.smithi132.stdout:1/148: mknod d1/c2f 0 2024-02-19T19:04:49.374 INFO:tasks.workunit.client.1.smithi132.stdout:8/200: truncate d4/fd 3483936 0 2024-02-19T19:04:49.376 INFO:tasks.workunit.client.1.smithi132.stdout:6/127: sync 2024-02-19T19:04:49.377 INFO:tasks.workunit.client.1.smithi132.stdout:3/129: link d8/d13/f16 d8/d13/f25 0 2024-02-19T19:04:49.377 INFO:tasks.workunit.client.1.smithi132.stdout:1/149: dread - d1/d1e/d29/f2e zero size 2024-02-19T19:04:49.378 INFO:tasks.workunit.client.1.smithi132.stdout:2/200: truncate d0/d4/d11/f20 1090888 0 2024-02-19T19:04:49.378 INFO:tasks.workunit.client.1.smithi132.stdout:5/199: dwrite d8/da/f24 [0,4194304] 0 2024-02-19T19:04:49.381 INFO:tasks.workunit.client.1.smithi132.stdout:7/236: creat d1b/d46/f4d x:0 0 0 2024-02-19T19:04:49.381 INFO:tasks.workunit.client.1.smithi132.stdout:0/193: mkdir d2/d3e 0 2024-02-19T19:04:49.382 INFO:tasks.workunit.client.1.smithi132.stdout:0/194: chown d2/d21 266662636 1 2024-02-19T19:04:49.382 INFO:tasks.workunit.client.1.smithi132.stdout:8/201: mknod d4/d2d/c48 0 2024-02-19T19:04:49.382 INFO:tasks.workunit.client.1.smithi132.stdout:0/195: readlink d2/dc/l10 0 2024-02-19T19:04:49.384 INFO:tasks.workunit.client.1.smithi132.stdout:1/150: creat d1/d19/f30 x:0 0 0 2024-02-19T19:04:49.385 INFO:tasks.workunit.client.1.smithi132.stdout:7/237: symlink d1b/d1e/d28/l4e 0 2024-02-19T19:04:49.388 INFO:tasks.workunit.client.1.smithi132.stdout:6/128: mknod d0/c32 0 2024-02-19T19:04:49.388 INFO:tasks.workunit.client.1.smithi132.stdout:8/202: rmdir d4/dc/d1a 39 2024-02-19T19:04:49.417 INFO:tasks.workunit.client.1.smithi132.stdout:2/201: mknod d0/d4/df/d40/c44 0 2024-02-19T19:04:49.418 INFO:tasks.workunit.client.1.smithi132.stdout:6/129: dread - d0/db/d16/f20 zero size 2024-02-19T19:04:49.418 INFO:tasks.workunit.client.1.smithi132.stdout:7/238: creat d1b/d1e/d28/f4f x:0 0 0 2024-02-19T19:04:49.419 INFO:tasks.workunit.client.1.smithi132.stdout:1/151: dwrite d1/f2 [0,4194304] 0 2024-02-19T19:04:49.419 INFO:tasks.workunit.client.1.smithi132.stdout:8/203: chown d4/dc/d1b/d33/f34 1239256 1 2024-02-19T19:04:49.420 INFO:tasks.workunit.client.1.smithi132.stdout:6/130: chown d0/fa 641955995 1 2024-02-19T19:04:49.428 INFO:tasks.workunit.client.1.smithi132.stdout:1/152: dwrite d1/f1d [0,4194304] 0 2024-02-19T19:04:49.435 INFO:tasks.workunit.client.1.smithi132.stdout:8/204: creat d4/dc/d1b/d33/f49 x:0 0 0 2024-02-19T19:04:49.436 INFO:tasks.workunit.client.1.smithi132.stdout:3/130: dwrite d8/f12 [0,4194304] 0 2024-02-19T19:04:49.438 INFO:tasks.workunit.client.1.smithi132.stdout:3/131: write d8/fd [899495,77794] 0 2024-02-19T19:04:49.439 INFO:tasks.workunit.client.1.smithi132.stdout:6/131: mknod d0/d1e/d22/c33 0 2024-02-19T19:04:49.444 INFO:tasks.workunit.client.1.smithi132.stdout:8/205: dwrite d4/f19 [0,4194304] 0 2024-02-19T19:04:49.449 INFO:tasks.workunit.client.1.smithi132.stdout:3/132: creat d8/f26 x:0 0 0 2024-02-19T19:04:49.450 INFO:tasks.workunit.client.1.smithi132.stdout:3/133: fdatasync d8/f14 0 2024-02-19T19:04:49.452 INFO:tasks.workunit.client.1.smithi132.stdout:3/134: write d8/f14 [308768,24859] 0 2024-02-19T19:04:49.453 INFO:tasks.workunit.client.1.smithi132.stdout:1/153: mknod d1/c31 0 2024-02-19T19:04:49.455 INFO:tasks.workunit.client.1.smithi132.stdout:3/135: chown d8/d13/f25 1386748 1 2024-02-19T19:04:49.456 INFO:tasks.workunit.client.1.smithi132.stdout:8/206: dwrite d4/dc/f2c [0,4194304] 0 2024-02-19T19:04:49.461 INFO:tasks.workunit.client.1.smithi132.stdout:3/136: read d8/f10 [3163908,41170] 0 2024-02-19T19:04:49.462 INFO:tasks.workunit.client.1.smithi132.stdout:6/132: dread d0/db/d16/f18 [0,4194304] 0 2024-02-19T19:04:49.464 INFO:tasks.workunit.client.1.smithi132.stdout:1/154: dwrite d1/d19/f22 [0,4194304] 0 2024-02-19T19:04:49.465 INFO:tasks.workunit.client.1.smithi132.stdout:1/155: chown d1/d19/l24 229810 1 2024-02-19T19:04:49.468 INFO:tasks.workunit.client.1.smithi132.stdout:9/297: sync 2024-02-19T19:04:49.468 INFO:tasks.workunit.client.1.smithi132.stdout:4/159: sync 2024-02-19T19:04:49.470 INFO:tasks.workunit.client.1.smithi132.stdout:3/137: symlink d8/l27 0 2024-02-19T19:04:49.471 INFO:tasks.workunit.client.1.smithi132.stdout:6/133: creat d0/db/d25/f34 x:0 0 0 2024-02-19T19:04:49.476 INFO:tasks.workunit.client.1.smithi132.stdout:8/207: creat d4/dc/d1a/f4a x:0 0 0 2024-02-19T19:04:49.484 INFO:tasks.workunit.client.1.smithi132.stdout:9/298: symlink d12/d2a/d4c/d61/l68 0 2024-02-19T19:04:49.487 INFO:tasks.workunit.client.1.smithi132.stdout:3/138: creat d8/f28 x:0 0 0 2024-02-19T19:04:49.489 INFO:tasks.workunit.client.1.smithi132.stdout:3/139: write d8/f28 [264020,114043] 0 2024-02-19T19:04:49.497 INFO:tasks.workunit.client.1.smithi132.stdout:9/299: symlink d12/d25/d3a/d40/d4e/l69 0 2024-02-19T19:04:49.543 INFO:tasks.workunit.client.1.smithi132.stdout:7/239: truncate d1b/f25 157585 0 2024-02-19T19:04:49.543 INFO:tasks.workunit.client.1.smithi132.stdout:9/300: chown d12/d25/d3a/d40/f5d 64878 1 2024-02-19T19:04:49.544 INFO:tasks.workunit.client.1.smithi132.stdout:3/140: write d8/f10 [630764,107283] 0 2024-02-19T19:04:49.545 INFO:tasks.workunit.client.1.smithi132.stdout:7/240: write d1b/d1e/d28/f35 [2370376,2875] 0 2024-02-19T19:04:49.546 INFO:tasks.workunit.client.1.smithi132.stdout:6/134: rename d0/le to d0/l35 0 2024-02-19T19:04:49.546 INFO:tasks.workunit.client.1.smithi132.stdout:1/156: link d1/f5 d1/d19/f32 0 2024-02-19T19:04:49.550 INFO:tasks.workunit.client.1.smithi132.stdout:9/301: creat d12/d2a/d4c/f6a x:0 0 0 2024-02-19T19:04:49.551 INFO:tasks.workunit.client.1.smithi132.stdout:4/160: dwrite d4/f5 [0,4194304] 0 2024-02-19T19:04:49.551 INFO:tasks.workunit.client.1.smithi132.stdout:3/141: symlink d8/d13/l29 0 2024-02-19T19:04:49.552 INFO:tasks.workunit.client.1.smithi132.stdout:3/142: chown d8/f22 39780 1 2024-02-19T19:04:49.552 INFO:tasks.workunit.client.1.smithi132.stdout:3/143: chown d8/d13 4 1 2024-02-19T19:04:49.553 INFO:tasks.workunit.client.1.smithi132.stdout:8/208: link d4/dc/d1a/l3a d4/dc/d1b/d20/l4b 0 2024-02-19T19:04:49.555 INFO:tasks.workunit.client.1.smithi132.stdout:6/135: creat d0/db/d16/f36 x:0 0 0 2024-02-19T19:04:49.559 INFO:tasks.workunit.client.1.smithi132.stdout:9/302: creat d12/d2a/d4c/d61/f6b x:0 0 0 2024-02-19T19:04:49.560 INFO:tasks.workunit.client.1.smithi132.stdout:3/144: creat d8/f2a x:0 0 0 2024-02-19T19:04:49.560 INFO:tasks.workunit.client.1.smithi132.stdout:9/303: write d12/f1a [1846860,106987] 0 2024-02-19T19:04:49.561 INFO:tasks.workunit.client.1.smithi132.stdout:8/209: dwrite d4/f19 [0,4194304] 0 2024-02-19T19:04:49.563 INFO:tasks.workunit.client.1.smithi132.stdout:7/241: link d1b/d46/f4d d1b/d1e/d28/f50 0 2024-02-19T19:04:49.568 INFO:tasks.workunit.client.1.smithi132.stdout:9/304: rmdir d12/d25 39 2024-02-19T19:04:49.569 INFO:tasks.workunit.client.1.smithi132.stdout:7/242: dread d1b/d1e/d28/f3c [0,4194304] 0 2024-02-19T19:04:49.570 INFO:tasks.workunit.client.1.smithi132.stdout:3/145: mkdir d8/d13/d2b 0 2024-02-19T19:04:49.570 INFO:tasks.workunit.client.1.smithi132.stdout:7/243: truncate d1b/d1e/d28/f4f 127734 0 2024-02-19T19:04:49.571 INFO:tasks.workunit.client.1.smithi132.stdout:3/146: chown d8/f10 134 1 2024-02-19T19:04:49.571 INFO:tasks.workunit.client.1.smithi132.stdout:8/210: symlink d4/dc/d1b/d20/d23/l4c 0 2024-02-19T19:04:49.575 INFO:tasks.workunit.client.1.smithi132.stdout:9/305: creat d12/d3e/f6c x:0 0 0 2024-02-19T19:04:49.576 INFO:tasks.workunit.client.1.smithi132.stdout:7/244: dwrite d1b/d1e/d28/f3c [0,4194304] 0 2024-02-19T19:04:49.581 INFO:tasks.workunit.client.1.smithi132.stdout:8/211: creat d4/dc/d1b/d33/f4d x:0 0 0 2024-02-19T19:04:49.582 INFO:tasks.workunit.client.1.smithi132.stdout:8/212: write d4/dc/f1c [191864,112130] 0 2024-02-19T19:04:49.583 INFO:tasks.workunit.client.1.smithi132.stdout:8/213: chown d4/dc/d1b/d33/f49 0 1 2024-02-19T19:04:49.586 INFO:tasks.workunit.client.1.smithi132.stdout:1/157: truncate d1/f2 651124 0 2024-02-19T19:04:49.587 INFO:tasks.workunit.client.1.smithi132.stdout:8/214: dwrite d4/dc/f2c [0,4194304] 0 2024-02-19T19:04:49.588 INFO:tasks.workunit.client.1.smithi132.stdout:4/161: truncate d4/d12/d16/f1b 317880 0 2024-02-19T19:04:49.589 INFO:tasks.workunit.client.1.smithi132.stdout:4/162: dread - d4/d12/f2c zero size 2024-02-19T19:04:49.589 INFO:tasks.workunit.client.1.smithi132.stdout:8/215: write d4/dc/d1b/d20/d23/f28 [1009477,11253] 0 2024-02-19T19:04:49.590 INFO:tasks.workunit.client.1.smithi132.stdout:8/216: write d4/f27 [853497,55926] 0 2024-02-19T19:04:49.593 INFO:tasks.workunit.client.1.smithi132.stdout:3/147: creat d8/d13/d2b/f2c x:0 0 0 2024-02-19T19:04:49.594 INFO:tasks.workunit.client.1.smithi132.stdout:3/148: fdatasync d8/f15 0 2024-02-19T19:04:49.594 INFO:tasks.workunit.client.1.smithi132.stdout:3/149: chown d8/l27 21 1 2024-02-19T19:04:49.595 INFO:tasks.workunit.client.1.smithi132.stdout:7/245: creat d1b/d1e/d2b/f51 x:0 0 0 2024-02-19T19:04:49.596 INFO:tasks.workunit.client.1.smithi132.stdout:7/246: write d1b/d1e/d2b/f2f [2666862,84359] 0 2024-02-19T19:04:49.596 INFO:tasks.workunit.client.1.smithi132.stdout:5/200: sync 2024-02-19T19:04:49.597 INFO:tasks.workunit.client.1.smithi132.stdout:5/201: readlink d8/da/ld 0 2024-02-19T19:04:49.598 INFO:tasks.workunit.client.1.smithi132.stdout:1/158: unlink d1/f17 0 2024-02-19T19:04:49.602 INFO:tasks.workunit.client.1.smithi132.stdout:4/163: unlink d4/dc/l1c 0 2024-02-19T19:04:49.603 INFO:tasks.workunit.client.1.smithi132.stdout:4/164: fdatasync d4/d12/f2f 0 2024-02-19T19:04:49.605 INFO:tasks.workunit.client.1.smithi132.stdout:4/165: write d4/d12/f2f [1515247,32770] 0 2024-02-19T19:04:49.606 INFO:tasks.workunit.client.1.smithi132.stdout:3/150: rename d8/f2a to d8/d13/d19/f2d 0 2024-02-19T19:04:49.607 INFO:tasks.workunit.client.1.smithi132.stdout:4/166: write d4/d19/f23 [1482828,78946] 0 2024-02-19T19:04:49.608 INFO:tasks.workunit.client.1.smithi132.stdout:7/247: dwrite d1b/d1e/d28/f4f [0,4194304] 0 2024-02-19T19:04:49.608 INFO:tasks.workunit.client.1.smithi132.stdout:4/167: stat d4/d19 0 2024-02-19T19:04:49.608 INFO:tasks.workunit.client.1.smithi132.stdout:7/248: chown l16 1 1 2024-02-19T19:04:49.613 INFO:tasks.workunit.client.1.smithi132.stdout:5/202: creat d8/da/db/f44 x:0 0 0 2024-02-19T19:04:49.614 INFO:tasks.workunit.client.1.smithi132.stdout:5/203: truncate d8/d23/d29/f2d 1013961 0 2024-02-19T19:04:49.615 INFO:tasks.workunit.client.1.smithi132.stdout:1/159: mknod d1/d1e/d29/d2c/c33 0 2024-02-19T19:04:49.615 INFO:tasks.workunit.client.1.smithi132.stdout:1/160: stat d1/d1e/d29/d2c/c33 0 2024-02-19T19:04:49.616 INFO:tasks.workunit.client.1.smithi132.stdout:5/204: truncate d8/da/db/f44 54764 0 2024-02-19T19:04:49.616 INFO:tasks.workunit.client.1.smithi132.stdout:1/161: chown d1/c2f 2975300 1 2024-02-19T19:04:49.616 INFO:tasks.workunit.client.1.smithi132.stdout:1/162: stat d1/d1e/d29/c2b 0 2024-02-19T19:04:49.617 INFO:tasks.workunit.client.1.smithi132.stdout:1/163: chown d1/d1e/l23 2 1 2024-02-19T19:04:49.618 INFO:tasks.workunit.client.1.smithi132.stdout:1/164: write d1/d19/f22 [2738023,122468] 0 2024-02-19T19:04:49.628 INFO:tasks.workunit.client.1.smithi132.stdout:4/168: truncate d4/f9 6045789 0 2024-02-19T19:04:49.629 INFO:tasks.workunit.client.1.smithi132.stdout:7/249: mknod d1b/d32/c52 0 2024-02-19T19:04:49.630 INFO:tasks.workunit.client.1.smithi132.stdout:7/250: chown d1b 22614396 1 2024-02-19T19:04:49.631 INFO:tasks.workunit.client.1.smithi132.stdout:9/306: creat d12/d25/d3a/f6d x:0 0 0 2024-02-19T19:04:49.632 INFO:tasks.workunit.client.1.smithi132.stdout:9/307: chown d12/d2a/d4c/d61/f63 268 1 2024-02-19T19:04:49.633 INFO:tasks.workunit.client.1.smithi132.stdout:8/217: creat d4/dc/f4e x:0 0 0 2024-02-19T19:04:49.634 INFO:tasks.workunit.client.1.smithi132.stdout:8/218: chown d4/dc/d1b 5711 1 2024-02-19T19:04:49.643 INFO:tasks.workunit.client.1.smithi132.stdout:7/251: mkdir d1b/d46/d48/d53 0 2024-02-19T19:04:49.644 INFO:tasks.workunit.client.1.smithi132.stdout:9/308: mkdir d12/d3e/d6e 0 2024-02-19T19:04:49.646 INFO:tasks.workunit.client.1.smithi132.stdout:4/169: mkdir d4/d27/d31 0 2024-02-19T19:04:49.649 INFO:tasks.workunit.client.1.smithi132.stdout:7/252: dwrite d1b/d1e/d28/f4f [0,4194304] 0 2024-02-19T19:04:49.651 INFO:tasks.workunit.client.1.smithi132.stdout:7/253: write d1b/d1e/f37 [928576,85034] 0 2024-02-19T19:04:49.654 INFO:tasks.workunit.client.1.smithi132.stdout:3/151: write f0 [996472,34874] 0 2024-02-19T19:04:49.658 INFO:tasks.workunit.client.1.smithi132.stdout:1/165: getdents d1/d1e 0 2024-02-19T19:04:49.658 INFO:tasks.workunit.client.1.smithi132.stdout:7/254: dwrite d1b/d1e/d41/f47 [0,4194304] 0 2024-02-19T19:04:49.659 INFO:tasks.workunit.client.1.smithi132.stdout:1/166: read - d1/d19/f30 zero size 2024-02-19T19:04:49.661 INFO:tasks.workunit.client.1.smithi132.stdout:9/309: symlink d12/d3e/d6e/l6f 0 2024-02-19T19:04:49.664 INFO:tasks.workunit.client.1.smithi132.stdout:4/170: mkdir d4/d27/d31/d32 0 2024-02-19T19:04:49.665 INFO:tasks.workunit.client.1.smithi132.stdout:4/171: chown d4/dc/l11 61285 1 2024-02-19T19:04:49.665 INFO:tasks.workunit.client.1.smithi132.stdout:7/255: mkdir d1b/d1e/d54 0 2024-02-19T19:04:49.666 INFO:tasks.workunit.client.1.smithi132.stdout:1/167: creat d1/d1e/d29/d2c/f34 x:0 0 0 2024-02-19T19:04:49.667 INFO:tasks.workunit.client.1.smithi132.stdout:7/256: truncate d1b/d1e/d2b/f51 584731 0 2024-02-19T19:04:49.668 INFO:tasks.workunit.client.1.smithi132.stdout:9/310: mkdir d12/d3e/d70 0 2024-02-19T19:04:49.670 INFO:tasks.workunit.client.1.smithi132.stdout:4/172: rmdir d4/d27/d31 39 2024-02-19T19:04:49.670 INFO:tasks.workunit.client.1.smithi132.stdout:4/173: dread - d4/d12/f2c zero size 2024-02-19T19:04:49.671 INFO:tasks.workunit.client.1.smithi132.stdout:1/168: dread d1/f21 [0,4194304] 0 2024-02-19T19:04:49.677 INFO:tasks.workunit.client.1.smithi132.stdout:1/169: rename d1/d19/f32 to d1/d1e/d29/d2c/f35 0 2024-02-19T19:04:49.680 INFO:tasks.workunit.client.1.smithi132.stdout:4/174: symlink d4/l33 0 2024-02-19T19:04:49.681 INFO:tasks.workunit.client.1.smithi132.stdout:4/175: fsync d4/d12/f2c 0 2024-02-19T19:04:49.683 INFO:tasks.workunit.client.1.smithi132.stdout:1/170: dwrite d1/f18 [0,4194304] 0 2024-02-19T19:04:49.687 INFO:tasks.workunit.client.1.smithi132.stdout:4/176: creat d4/d27/f34 x:0 0 0 2024-02-19T19:04:49.687 INFO:tasks.workunit.client.1.smithi132.stdout:3/152: truncate d8/f14 263418 0 2024-02-19T19:04:49.693 INFO:tasks.workunit.client.1.smithi132.stdout:4/177: unlink d4/f5 0 2024-02-19T19:04:49.694 INFO:tasks.workunit.client.1.smithi132.stdout:1/171: dwrite d1/d1e/d29/d2c/f2d [0,4194304] 0 2024-02-19T19:04:49.696 INFO:tasks.workunit.client.1.smithi132.stdout:4/178: chown d4/d12/d16/l2a 490 1 2024-02-19T19:04:49.697 INFO:tasks.workunit.client.1.smithi132.stdout:7/257: dwrite d1b/d1e/f21 [0,4194304] 0 2024-02-19T19:04:49.697 INFO:tasks.workunit.client.1.smithi132.stdout:9/311: dwrite d12/d2a/f37 [0,4194304] 0 2024-02-19T19:04:49.698 INFO:tasks.workunit.client.1.smithi132.stdout:3/153: rmdir d8/d13/d19 39 2024-02-19T19:04:49.702 INFO:tasks.workunit.client.1.smithi132.stdout:4/179: rename d4/d27 to d4/d27/d31/d35 22 2024-02-19T19:04:49.703 INFO:tasks.workunit.client.1.smithi132.stdout:4/180: truncate d4/f2d 214279 0 2024-02-19T19:04:49.709 INFO:tasks.workunit.client.1.smithi132.stdout:1/172: rmdir d1/d1e/d29 39 2024-02-19T19:04:49.712 INFO:tasks.workunit.client.1.smithi132.stdout:7/258: creat d1b/d1e/d28/f55 x:0 0 0 2024-02-19T19:04:49.712 INFO:tasks.workunit.client.1.smithi132.stdout:9/312: dread d12/d2a/f5b [0,4194304] 0 2024-02-19T19:04:49.713 INFO:tasks.workunit.client.1.smithi132.stdout:0/196: sync 2024-02-19T19:04:49.715 INFO:tasks.workunit.client.1.smithi132.stdout:3/154: mkdir d8/d13/d2b/d2e 0 2024-02-19T19:04:49.717 INFO:tasks.workunit.client.1.smithi132.stdout:1/173: dwrite d1/f15 [8388608,4194304] 0 2024-02-19T19:04:49.718 INFO:tasks.workunit.client.1.smithi132.stdout:1/174: stat d1/f1b 0 2024-02-19T19:04:49.728 INFO:tasks.workunit.client.1.smithi132.stdout:7/259: mkdir d1b/d46/d56 0 2024-02-19T19:04:49.728 INFO:tasks.workunit.client.1.smithi132.stdout:7/260: dread - d1b/d1e/d28/f4c zero size 2024-02-19T19:04:49.729 INFO:tasks.workunit.client.1.smithi132.stdout:7/261: dread - d1b/d1e/f43 zero size 2024-02-19T19:04:49.730 INFO:tasks.workunit.client.1.smithi132.stdout:0/197: mkdir d2/dc/df/d15/d2f/d3f 0 2024-02-19T19:04:49.733 INFO:tasks.workunit.client.1.smithi132.stdout:1/175: write d1/d1e/d29/d2c/f2d [1194836,56991] 0 2024-02-19T19:04:49.733 INFO:tasks.workunit.client.1.smithi132.stdout:4/181: creat d4/d19/d21/f36 x:0 0 0 2024-02-19T19:04:49.739 INFO:tasks.workunit.client.1.smithi132.stdout:9/313: rmdir d12/d3e/d70 0 2024-02-19T19:04:49.740 INFO:tasks.workunit.client.1.smithi132.stdout:9/314: dread - d12/d25/d3a/d40/d4e/f67 zero size 2024-02-19T19:04:49.741 INFO:tasks.workunit.client.1.smithi132.stdout:9/315: write d12/d25/f29 [1596297,119426] 0 2024-02-19T19:04:49.745 INFO:tasks.workunit.client.1.smithi132.stdout:4/182: mkdir d4/dc/d37 0 2024-02-19T19:04:49.746 INFO:tasks.workunit.client.1.smithi132.stdout:4/183: write d4/f2d [239114,93481] 0 2024-02-19T19:04:49.747 INFO:tasks.workunit.client.1.smithi132.stdout:0/198: getdents d2/dc/df/d27/d3a 0 2024-02-19T19:04:49.747 INFO:tasks.workunit.client.1.smithi132.stdout:9/316: dwrite d12/d2a/d4c/d61/f63 [0,4194304] 0 2024-02-19T19:04:49.748 INFO:tasks.workunit.client.1.smithi132.stdout:0/199: write d2/d21/d25/f26 [636114,46895] 0 2024-02-19T19:04:49.757 INFO:tasks.workunit.client.1.smithi132.stdout:7/262: dwrite d1b/d1e/f30 [0,4194304] 0 2024-02-19T19:04:49.761 INFO:tasks.workunit.client.1.smithi132.stdout:1/176: truncate d1/f21 257449 0 2024-02-19T19:04:49.765 INFO:tasks.workunit.client.1.smithi132.stdout:3/155: write d8/f14 [194489,11936] 0 2024-02-19T19:04:49.776 INFO:tasks.workunit.client.1.smithi132.stdout:9/317: truncate d12/d2a/f2b 3662286 0 2024-02-19T19:04:49.777 INFO:tasks.workunit.client.1.smithi132.stdout:3/156: mkdir d8/d13/d2f 0 2024-02-19T19:04:49.779 INFO:tasks.workunit.client.1.smithi132.stdout:7/263: write d1b/f25 [328664,68195] 0 2024-02-19T19:04:49.780 INFO:tasks.workunit.client.1.smithi132.stdout:4/184: dwrite d4/f9 [0,4194304] 0 2024-02-19T19:04:49.780 INFO:tasks.workunit.client.1.smithi132.stdout:1/177: write d1/f5 [468429,113702] 0 2024-02-19T19:04:49.783 INFO:tasks.workunit.client.1.smithi132.stdout:9/318: creat d12/d3e/f71 x:0 0 0 2024-02-19T19:04:49.785 INFO:tasks.workunit.client.1.smithi132.stdout:7/264: read d1b/d1e/d28/f31 [807897,57715] 0 2024-02-19T19:04:49.785 INFO:tasks.workunit.client.1.smithi132.stdout:1/178: rmdir d1/d19 39 2024-02-19T19:04:49.787 INFO:tasks.workunit.client.1.smithi132.stdout:0/200: dwrite d2/f12 [0,4194304] 0 2024-02-19T19:04:49.788 INFO:tasks.workunit.client.1.smithi132.stdout:4/185: unlink d4/d12/d16/f29 0 2024-02-19T19:04:49.788 INFO:tasks.workunit.client.1.smithi132.stdout:3/157: mknod d8/d13/c30 0 2024-02-19T19:04:49.794 INFO:tasks.workunit.client.1.smithi132.stdout:1/179: chown d1/d19/f30 3077 1 2024-02-19T19:04:49.794 INFO:tasks.workunit.client.1.smithi132.stdout:4/186: dread d4/f10 [0,4194304] 0 2024-02-19T19:04:49.796 INFO:tasks.workunit.client.1.smithi132.stdout:9/319: creat d12/d3e/f72 x:0 0 0 2024-02-19T19:04:49.798 INFO:tasks.workunit.client.1.smithi132.stdout:9/320: dread d12/f26 [4194304,4194304] 0 2024-02-19T19:04:49.799 INFO:tasks.workunit.client.1.smithi132.stdout:3/158: mkdir d8/d31 0 2024-02-19T19:04:49.799 INFO:tasks.workunit.client.1.smithi132.stdout:3/159: fsync d8/d13/f24 0 2024-02-19T19:04:49.801 INFO:tasks.workunit.client.1.smithi132.stdout:0/201: rename d2/dc/f37 to d2/dc/df/d15/d2f/f40 0 2024-02-19T19:04:49.803 INFO:tasks.workunit.client.1.smithi132.stdout:7/265: creat d1b/d46/d56/f57 x:0 0 0 2024-02-19T19:04:49.805 INFO:tasks.workunit.client.1.smithi132.stdout:2/202: sync 2024-02-19T19:04:49.805 INFO:tasks.workunit.client.1.smithi132.stdout:6/136: sync 2024-02-19T19:04:49.805 INFO:tasks.workunit.client.1.smithi132.stdout:0/202: dread d2/f12 [0,4194304] 0 2024-02-19T19:04:49.806 INFO:tasks.workunit.client.1.smithi132.stdout:6/137: chown d0/dc/f24 176 1 2024-02-19T19:04:49.806 INFO:tasks.workunit.client.1.smithi132.stdout:2/203: write d0/d4/d11/f1c [1604176,54045] 0 2024-02-19T19:04:49.807 INFO:tasks.workunit.client.1.smithi132.stdout:9/321: unlink d12/d1b/c60 0 2024-02-19T19:04:49.809 INFO:tasks.workunit.client.1.smithi132.stdout:3/160: unlink d8/c11 0 2024-02-19T19:04:49.811 INFO:tasks.workunit.client.1.smithi132.stdout:1/180: truncate d1/f2 1081111 0 2024-02-19T19:04:49.812 INFO:tasks.workunit.client.1.smithi132.stdout:1/181: fsync d1/d1e/d29/d2c/f2d 0 2024-02-19T19:04:49.813 INFO:tasks.workunit.client.1.smithi132.stdout:9/322: dwrite d12/d25/d3a/d40/f49 [0,4194304] 0 2024-02-19T19:04:49.814 INFO:tasks.workunit.client.1.smithi132.stdout:1/182: write d1/d1e/f2a [860001,117269] 0 2024-02-19T19:04:49.814 INFO:tasks.workunit.client.1.smithi132.stdout:1/183: readlink d1/d1e/l23 0 2024-02-19T19:04:49.815 INFO:tasks.workunit.client.1.smithi132.stdout:9/323: truncate d12/d25/d3a/d40/f51 300713 0 2024-02-19T19:04:49.848 INFO:tasks.workunit.client.1.smithi132.stdout:0/203: rmdir d2/dc/df/d15/d2f 39 2024-02-19T19:04:49.849 INFO:tasks.workunit.client.1.smithi132.stdout:6/138: creat d0/db/d16/f37 x:0 0 0 2024-02-19T19:04:49.849 INFO:tasks.workunit.client.1.smithi132.stdout:2/204: creat d0/d19/d1e/f45 x:0 0 0 2024-02-19T19:04:49.849 INFO:tasks.workunit.client.1.smithi132.stdout:4/187: link d4/d12/f2f d4/f38 0 2024-02-19T19:04:49.849 INFO:tasks.workunit.client.1.smithi132.stdout:6/139: fsync d0/dc/f2d 0 2024-02-19T19:04:49.851 INFO:tasks.workunit.client.1.smithi132.stdout:7/266: truncate d1b/d1e/d2b/f2f 2928 0 2024-02-19T19:04:49.851 INFO:tasks.workunit.client.1.smithi132.stdout:9/324: write d12/d25/f30 [373064,29427] 0 2024-02-19T19:04:49.852 INFO:tasks.workunit.client.1.smithi132.stdout:0/204: truncate d2/f2b 4125144 0 2024-02-19T19:04:49.852 INFO:tasks.workunit.client.1.smithi132.stdout:9/325: chown d12/d25/f2e 354 1 2024-02-19T19:04:49.854 INFO:tasks.workunit.client.1.smithi132.stdout:6/140: stat d0/db/d25/c2c 0 2024-02-19T19:04:49.855 INFO:tasks.workunit.client.1.smithi132.stdout:4/188: creat d4/dc/d14/d24/f39 x:0 0 0 2024-02-19T19:04:49.857 INFO:tasks.workunit.client.1.smithi132.stdout:7/267: creat d1b/f58 x:0 0 0 2024-02-19T19:04:49.859 INFO:tasks.workunit.client.1.smithi132.stdout:1/184: creat d1/d1e/f36 x:0 0 0 2024-02-19T19:04:49.859 INFO:tasks.workunit.client.1.smithi132.stdout:6/141: dwrite d0/dc/f24 [0,4194304] 0 2024-02-19T19:04:49.863 INFO:tasks.workunit.client.1.smithi132.stdout:0/205: creat d2/dc/df/d15/d34/f41 x:0 0 0 2024-02-19T19:04:49.869 INFO:tasks.workunit.client.1.smithi132.stdout:1/185: dwrite d1/d19/f22 [4194304,4194304] 0 2024-02-19T19:04:49.871 INFO:tasks.workunit.client.1.smithi132.stdout:6/142: dwrite d0/db/d16/f20 [0,4194304] 0 2024-02-19T19:04:49.871 INFO:tasks.workunit.client.1.smithi132.stdout:7/268: symlink d1b/d32/l59 0 2024-02-19T19:04:49.873 INFO:tasks.workunit.client.1.smithi132.stdout:1/186: truncate d1/d1e/d29/d2c/f2d 4837052 0 2024-02-19T19:04:49.873 INFO:tasks.workunit.client.1.smithi132.stdout:6/143: stat d0/db/d16/c21 0 2024-02-19T19:04:49.876 INFO:tasks.workunit.client.1.smithi132.stdout:4/189: creat d4/dc/d37/f3a x:0 0 0 2024-02-19T19:04:49.876 INFO:tasks.workunit.client.1.smithi132.stdout:9/326: write d12/d2a/f5b [26932,93485] 0 2024-02-19T19:04:49.876 INFO:tasks.workunit.client.1.smithi132.stdout:0/206: rmdir d2/d21/d25 39 2024-02-19T19:04:49.881 INFO:tasks.workunit.client.1.smithi132.stdout:2/205: link d0/d24/c3d d0/d3c/c46 0 2024-02-19T19:04:49.882 INFO:tasks.workunit.client.1.smithi132.stdout:6/144: mkdir d0/db/d25/d38 0 2024-02-19T19:04:49.884 INFO:tasks.workunit.client.1.smithi132.stdout:6/145: truncate d0/d1e/d22/f8 315356 0 2024-02-19T19:04:49.884 INFO:tasks.workunit.client.1.smithi132.stdout:4/190: creat d4/dc/d14/d24/f3b x:0 0 0 2024-02-19T19:04:49.885 INFO:tasks.workunit.client.1.smithi132.stdout:4/191: chown d4/d12/f30 843494 1 2024-02-19T19:04:49.886 INFO:tasks.workunit.client.1.smithi132.stdout:4/192: chown d4/dc/d14/d24/l25 20 1 2024-02-19T19:04:49.892 INFO:tasks.workunit.client.1.smithi132.stdout:6/146: link d0/d1e/d22/f23 d0/d1e/f39 0 2024-02-19T19:04:49.892 INFO:tasks.workunit.client.1.smithi132.stdout:9/327: getdents d12/d25 0 2024-02-19T19:04:49.895 INFO:tasks.workunit.client.1.smithi132.stdout:4/193: dwrite d4/dc/d14/d24/f3b [0,4194304] 0 2024-02-19T19:04:49.895 INFO:tasks.workunit.client.1.smithi132.stdout:9/328: read d12/d1b/f50 [2867926,87075] 0 2024-02-19T19:04:49.896 INFO:tasks.workunit.client.1.smithi132.stdout:6/147: mkdir d0/db/d25/d3a 0 2024-02-19T19:04:49.898 INFO:tasks.workunit.client.1.smithi132.stdout:4/194: truncate d4/dc/d14/d24/f39 981755 0 2024-02-19T19:04:49.899 INFO:tasks.workunit.client.1.smithi132.stdout:9/329: creat d12/d1b/f73 x:0 0 0 2024-02-19T19:04:49.900 INFO:tasks.workunit.client.1.smithi132.stdout:6/148: symlink d0/db/d25/l3b 0 2024-02-19T19:04:49.907 INFO:tasks.workunit.client.1.smithi132.stdout:1/187: write d1/f21 [1242323,41403] 0 2024-02-19T19:04:49.907 INFO:tasks.workunit.client.1.smithi132.stdout:2/206: dread d0/d4/d11/f26 [0,4194304] 0 2024-02-19T19:04:49.908 INFO:tasks.workunit.client.1.smithi132.stdout:4/195: dwrite d4/d19/d21/f36 [0,4194304] 0 2024-02-19T19:04:49.911 INFO:tasks.workunit.client.1.smithi132.stdout:6/149: dwrite d0/db/fd [0,4194304] 0 2024-02-19T19:04:49.916 INFO:tasks.workunit.client.1.smithi132.stdout:8/219: sync 2024-02-19T19:04:49.916 INFO:tasks.workunit.client.1.smithi132.stdout:0/207: chown d2/f2b 0 1 2024-02-19T19:04:49.916 INFO:tasks.workunit.client.1.smithi132.stdout:5/205: sync 2024-02-19T19:04:49.917 INFO:tasks.workunit.client.1.smithi132.stdout:1/188: mknod d1/d19/c37 0 2024-02-19T19:04:49.919 INFO:tasks.workunit.client.1.smithi132.stdout:9/330: link d12/d25/d3a/d40/d4e/l69 d12/d3e/l74 0 2024-02-19T19:04:49.919 INFO:tasks.workunit.client.1.smithi132.stdout:1/189: dread - d1/d1e/d29/d2c/f34 zero size 2024-02-19T19:04:49.924 INFO:tasks.workunit.client.1.smithi132.stdout:4/196: creat d4/d27/f3c x:0 0 0 2024-02-19T19:04:49.925 INFO:tasks.workunit.client.1.smithi132.stdout:2/207: dwrite d0/d4/fb [0,4194304] 0 2024-02-19T19:04:49.926 INFO:tasks.workunit.client.1.smithi132.stdout:2/208: chown d0/d4/f22 44726 1 2024-02-19T19:04:49.927 INFO:tasks.workunit.client.1.smithi132.stdout:2/209: fsync d0/d4/f1f 0 2024-02-19T19:04:49.927 INFO:tasks.workunit.client.1.smithi132.stdout:8/220: mknod d4/dc/d1a/c4f 0 2024-02-19T19:04:49.928 INFO:tasks.workunit.client.1.smithi132.stdout:1/190: creat d1/d1e/d29/d2c/f38 x:0 0 0 2024-02-19T19:04:49.929 INFO:tasks.workunit.client.1.smithi132.stdout:2/210: write d0/d24/f39 [4838057,75495] 0 2024-02-19T19:04:49.931 INFO:tasks.workunit.client.1.smithi132.stdout:6/150: rename d0/d1e to d0/d3c 0 2024-02-19T19:04:49.934 INFO:tasks.workunit.client.1.smithi132.stdout:8/221: creat d4/dc/d1a/f50 x:0 0 0 2024-02-19T19:04:49.936 INFO:tasks.workunit.client.1.smithi132.stdout:1/191: creat d1/d19/f39 x:0 0 0 2024-02-19T19:04:49.936 INFO:tasks.workunit.client.1.smithi132.stdout:4/197: symlink d4/l3d 0 2024-02-19T19:04:49.936 INFO:tasks.workunit.client.1.smithi132.stdout:9/331: getdents d12/d3e/d6e 0 2024-02-19T19:04:49.938 INFO:tasks.workunit.client.1.smithi132.stdout:9/332: readlink d12/l2d 0 2024-02-19T19:04:49.938 INFO:tasks.workunit.client.1.smithi132.stdout:2/211: dwrite d0/d4/d11/f15 [0,4194304] 0 2024-02-19T19:04:49.940 INFO:tasks.workunit.client.1.smithi132.stdout:6/151: mkdir d0/db/d25/d3a/d3d 0 2024-02-19T19:04:49.941 INFO:tasks.workunit.client.1.smithi132.stdout:6/152: dread - d0/db/d25/f34 zero size 2024-02-19T19:04:49.942 INFO:tasks.workunit.client.1.smithi132.stdout:1/192: dread d1/d19/f22 [4194304,4194304] 0 2024-02-19T19:04:49.942 INFO:tasks.workunit.client.1.smithi132.stdout:4/198: rename d4/d27/f34 to d4/d12/d28/f3e 0 2024-02-19T19:04:49.944 INFO:tasks.workunit.client.1.smithi132.stdout:2/212: unlink d0/c18 0 2024-02-19T19:04:50.018 INFO:tasks.workunit.client.1.smithi132.stdout:4/199: creat d4/d19/f3f x:0 0 0 2024-02-19T19:04:50.020 INFO:tasks.workunit.client.1.smithi132.stdout:8/222: dread d4/dc/d1b/d20/d23/f28 [0,4194304] 0 2024-02-19T19:04:50.022 INFO:tasks.workunit.client.1.smithi132.stdout:6/153: dwrite d0/db/d25/f34 [0,4194304] 0 2024-02-19T19:04:50.024 INFO:tasks.workunit.client.1.smithi132.stdout:2/213: mknod d0/d19/c47 0 2024-02-19T19:04:50.024 INFO:tasks.workunit.client.1.smithi132.stdout:0/208: dwrite d2/f2b [0,4194304] 0 2024-02-19T19:04:50.025 INFO:tasks.workunit.client.1.smithi132.stdout:2/214: fdatasync d0/f6 0 2024-02-19T19:04:50.025 INFO:tasks.workunit.client.1.smithi132.stdout:6/154: truncate d0/db/d16/f36 584406 0 2024-02-19T19:04:50.026 INFO:tasks.workunit.client.1.smithi132.stdout:2/215: stat d0/d4/f1f 0 2024-02-19T19:04:50.027 INFO:tasks.workunit.client.1.smithi132.stdout:6/155: chown d0/d3c/d22/f19 4625 1 2024-02-19T19:04:50.029 INFO:tasks.workunit.client.1.smithi132.stdout:1/193: getdents d1/d19 0 2024-02-19T19:04:50.029 INFO:tasks.workunit.client.1.smithi132.stdout:6/156: write d0/dc/f15 [1699507,51895] 0 2024-02-19T19:04:50.030 INFO:tasks.workunit.client.1.smithi132.stdout:8/223: creat d4/dc/d1a/f51 x:0 0 0 2024-02-19T19:04:50.031 INFO:tasks.workunit.client.1.smithi132.stdout:1/194: chown d1/f1b 6 1 2024-02-19T19:04:50.032 INFO:tasks.workunit.client.1.smithi132.stdout:1/195: write d1/f1d [1710949,18364] 0 2024-02-19T19:04:50.034 INFO:tasks.workunit.client.1.smithi132.stdout:2/216: dwrite d0/d4/d11/f1c [0,4194304] 0 2024-02-19T19:04:50.041 INFO:tasks.workunit.client.1.smithi132.stdout:0/209: dread d2/d21/d25/f26 [0,4194304] 0 2024-02-19T19:04:50.041 INFO:tasks.workunit.client.1.smithi132.stdout:8/224: mknod d4/dc/d42/c52 0 2024-02-19T19:04:50.042 INFO:tasks.workunit.client.1.smithi132.stdout:6/157: symlink d0/db/d25/d38/l3e 0 2024-02-19T19:04:50.046 INFO:tasks.workunit.client.1.smithi132.stdout:8/225: creat d4/dc/d1b/d20/f53 x:0 0 0 2024-02-19T19:04:50.046 INFO:tasks.workunit.client.1.smithi132.stdout:8/226: chown d4/dc/f3f 6561013 1 2024-02-19T19:04:50.048 INFO:tasks.workunit.client.1.smithi132.stdout:8/227: write d4/dc/d1b/d20/f53 [718221,1531] 0 2024-02-19T19:04:50.048 INFO:tasks.workunit.client.1.smithi132.stdout:1/196: rename d1/d19 to d1/d3a 0 2024-02-19T19:04:50.051 INFO:tasks.workunit.client.1.smithi132.stdout:3/161: sync 2024-02-19T19:04:50.052 INFO:tasks.workunit.client.1.smithi132.stdout:6/158: symlink d0/db/l3f 0 2024-02-19T19:04:50.052 INFO:tasks.workunit.client.1.smithi132.stdout:2/217: link d0/d19/d1e/c29 d0/d3c/c48 0 2024-02-19T19:04:50.055 INFO:tasks.workunit.client.1.smithi132.stdout:1/197: dwrite d1/d1e/d29/d2c/f35 [0,4194304] 0 2024-02-19T19:04:50.056 INFO:tasks.workunit.client.1.smithi132.stdout:3/162: unlink d8/f15 0 2024-02-19T19:04:50.057 INFO:tasks.workunit.client.1.smithi132.stdout:3/163: write d8/f14 [491585,96551] 0 2024-02-19T19:04:50.057 INFO:tasks.workunit.client.1.smithi132.stdout:0/210: creat d2/d21/f42 x:0 0 0 2024-02-19T19:04:50.059 INFO:tasks.workunit.client.1.smithi132.stdout:8/228: mkdir d4/dc/d1b/d54 0 2024-02-19T19:04:50.060 INFO:tasks.workunit.client.1.smithi132.stdout:6/159: creat d0/db/d25/f40 x:0 0 0 2024-02-19T19:04:50.063 INFO:tasks.workunit.client.1.smithi132.stdout:1/198: unlink d1/d3a/f22 0 2024-02-19T19:04:50.065 INFO:tasks.workunit.client.1.smithi132.stdout:3/164: symlink d8/d13/d2b/l32 0 2024-02-19T19:04:50.065 INFO:tasks.workunit.client.1.smithi132.stdout:1/199: write d1/f15 [8119961,128816] 0 2024-02-19T19:04:50.067 INFO:tasks.workunit.client.1.smithi132.stdout:2/218: creat d0/d4/d33/f49 x:0 0 0 2024-02-19T19:04:50.067 INFO:tasks.workunit.client.1.smithi132.stdout:0/211: dwrite d2/d21/f42 [0,4194304] 0 2024-02-19T19:04:50.067 INFO:tasks.workunit.client.1.smithi132.stdout:8/229: dread d4/dc/f3f [0,4194304] 0 2024-02-19T19:04:50.069 INFO:tasks.workunit.client.1.smithi132.stdout:0/212: fsync d2/dc/df/f2d 0 2024-02-19T19:04:50.069 INFO:tasks.workunit.client.1.smithi132.stdout:8/230: dread - d4/dc/d1b/d33/f4d zero size 2024-02-19T19:04:50.070 INFO:tasks.workunit.client.1.smithi132.stdout:3/165: creat d8/d13/d2b/f33 x:0 0 0 2024-02-19T19:04:50.073 INFO:tasks.workunit.client.1.smithi132.stdout:1/200: symlink d1/d1e/d29/d2c/l3b 0 2024-02-19T19:04:50.074 INFO:tasks.workunit.client.1.smithi132.stdout:2/219: write d0/d4/d11/f20 [1330192,84525] 0 2024-02-19T19:04:50.075 INFO:tasks.workunit.client.1.smithi132.stdout:6/160: creat d0/f41 x:0 0 0 2024-02-19T19:04:50.077 INFO:tasks.workunit.client.1.smithi132.stdout:8/231: mknod d4/dc/d1b/c55 0 2024-02-19T19:04:50.080 INFO:tasks.workunit.client.1.smithi132.stdout:0/213: dread d2/dc/fe [0,4194304] 0 2024-02-19T19:04:50.081 INFO:tasks.workunit.client.1.smithi132.stdout:2/220: mknod d0/d4/df/c4a 0 2024-02-19T19:04:50.082 INFO:tasks.workunit.client.1.smithi132.stdout:3/166: dwrite d8/d13/f20 [0,4194304] 0 2024-02-19T19:04:50.083 INFO:tasks.workunit.client.1.smithi132.stdout:3/167: write d8/f26 [133563,101810] 0 2024-02-19T19:04:50.084 INFO:tasks.workunit.client.1.smithi132.stdout:6/161: creat d0/d3c/f42 x:0 0 0 2024-02-19T19:04:50.084 INFO:tasks.workunit.client.1.smithi132.stdout:6/162: read - d0/db/d16/f29 zero size 2024-02-19T19:04:50.085 INFO:tasks.workunit.client.1.smithi132.stdout:2/221: read d0/d4/f12 [1968281,97510] 0 2024-02-19T19:04:50.086 INFO:tasks.workunit.client.1.smithi132.stdout:6/163: write d0/db/d16/f20 [3495089,121598] 0 2024-02-19T19:04:50.086 INFO:tasks.workunit.client.1.smithi132.stdout:2/222: fdatasync d0/d19/d1e/f45 0 2024-02-19T19:04:50.087 INFO:tasks.workunit.client.1.smithi132.stdout:2/223: readlink d0/le 0 2024-02-19T19:04:50.087 INFO:tasks.workunit.client.1.smithi132.stdout:6/164: truncate d0/db/d16/f2e 71130 0 2024-02-19T19:04:50.092 INFO:tasks.workunit.client.1.smithi132.stdout:8/232: creat d4/dc/d42/f56 x:0 0 0 2024-02-19T19:04:50.095 INFO:tasks.workunit.client.1.smithi132.stdout:2/224: dwrite d0/d4/d11/f1c [4194304,4194304] 0 2024-02-19T19:04:50.096 INFO:tasks.workunit.client.1.smithi132.stdout:2/225: fdatasync d0/d4/df/f3b 0 2024-02-19T19:04:50.098 INFO:tasks.workunit.client.1.smithi132.stdout:2/226: write d0/d19/f2c [1084639,61702] 0 2024-02-19T19:04:50.099 INFO:tasks.workunit.client.1.smithi132.stdout:2/227: fdatasync d0/d4/fb 0 2024-02-19T19:04:50.100 INFO:tasks.workunit.client.1.smithi132.stdout:8/233: dwrite d4/dc/d1b/d1d/f36 [0,4194304] 0 2024-02-19T19:04:50.102 INFO:tasks.workunit.client.1.smithi132.stdout:8/234: write d4/dc/d1b/f43 [4356994,65807] 0 2024-02-19T19:04:50.104 INFO:tasks.workunit.client.1.smithi132.stdout:6/165: mknod d0/db/d25/d38/c43 0 2024-02-19T19:04:50.106 INFO:tasks.workunit.client.1.smithi132.stdout:6/166: write d0/db/d16/f26 [518394,52680] 0 2024-02-19T19:04:50.115 INFO:tasks.workunit.client.1.smithi132.stdout:7/269: sync 2024-02-19T19:04:50.115 INFO:tasks.workunit.client.1.smithi132.stdout:5/206: sync 2024-02-19T19:04:50.116 INFO:tasks.workunit.client.1.smithi132.stdout:7/270: stat d1b/d1e/f43 0 2024-02-19T19:04:50.116 INFO:tasks.workunit.client.1.smithi132.stdout:3/168: mkdir d8/d31/d34 0 2024-02-19T19:04:50.118 INFO:tasks.workunit.client.1.smithi132.stdout:2/228: creat d0/d4/df/d40/f4b x:0 0 0 2024-02-19T19:04:50.119 INFO:tasks.workunit.client.1.smithi132.stdout:0/214: write d2/f12 [2836216,62142] 0 2024-02-19T19:04:50.119 INFO:tasks.workunit.client.1.smithi132.stdout:8/235: symlink d4/dc/d1b/d33/l57 0 2024-02-19T19:04:50.121 INFO:tasks.workunit.client.1.smithi132.stdout:1/201: dwrite d1/f2 [0,4194304] 0 2024-02-19T19:04:50.122 INFO:tasks.workunit.client.1.smithi132.stdout:0/215: write d2/dc/df/d15/f22 [40524,55444] 0 2024-02-19T19:04:50.124 INFO:tasks.workunit.client.1.smithi132.stdout:7/271: chown d1b/d1e/d2b/f2f 16357468 1 2024-02-19T19:04:50.124 INFO:tasks.workunit.client.1.smithi132.stdout:6/167: symlink d0/l44 0 2024-02-19T19:04:50.130 INFO:tasks.workunit.client.1.smithi132.stdout:6/168: stat d0/db/d25/d3a/d3d 0 2024-02-19T19:04:50.130 INFO:tasks.workunit.client.1.smithi132.stdout:7/272: dread d1b/d1e/f38 [0,4194304] 0 2024-02-19T19:04:50.131 INFO:tasks.workunit.client.1.smithi132.stdout:2/229: truncate d0/d4/d11/d2e/f3e 3705920 0 2024-02-19T19:04:50.134 INFO:tasks.workunit.client.1.smithi132.stdout:1/202: dwrite d1/d3a/f39 [0,4194304] 0 2024-02-19T19:04:50.135 INFO:tasks.workunit.client.1.smithi132.stdout:7/273: write d1b/d1e/d28/f4c [973968,73398] 0 2024-02-19T19:04:50.136 INFO:tasks.workunit.client.1.smithi132.stdout:8/236: getdents d4/dc/d1b/d1d 0 2024-02-19T19:04:50.137 INFO:tasks.workunit.client.1.smithi132.stdout:0/216: dread d2/f2b [0,4194304] 0 2024-02-19T19:04:50.138 INFO:tasks.workunit.client.1.smithi132.stdout:6/169: symlink d0/db/d16/l45 0 2024-02-19T19:04:50.139 INFO:tasks.workunit.client.1.smithi132.stdout:0/217: stat d2/c14 0 2024-02-19T19:04:50.139 INFO:tasks.workunit.client.1.smithi132.stdout:8/237: write d4/f27 [1407917,6448] 0 2024-02-19T19:04:50.142 INFO:tasks.workunit.client.1.smithi132.stdout:2/230: dread d0/d4/f1f [0,4194304] 0 2024-02-19T19:04:50.143 INFO:tasks.workunit.client.1.smithi132.stdout:8/238: stat d4/dc/d1b/d1d/l37 0 2024-02-19T19:04:50.143 INFO:tasks.workunit.client.1.smithi132.stdout:5/207: write d8/ff [1527127,91917] 0 2024-02-19T19:04:50.143 INFO:tasks.workunit.client.1.smithi132.stdout:6/170: dread d0/f5 [0,4194304] 0 2024-02-19T19:04:50.145 INFO:tasks.workunit.client.1.smithi132.stdout:1/203: dread d1/d3a/f39 [0,4194304] 0 2024-02-19T19:04:50.148 INFO:tasks.workunit.client.1.smithi132.stdout:7/274: rmdir d1b/d1e/d54 0 2024-02-19T19:04:50.154 INFO:tasks.workunit.client.1.smithi132.stdout:8/239: rmdir d4/dc/d1b 39 2024-02-19T19:04:50.156 INFO:tasks.workunit.client.1.smithi132.stdout:6/171: mkdir d0/d3c/d46 0 2024-02-19T19:04:50.156 INFO:tasks.workunit.client.1.smithi132.stdout:7/275: dwrite d1b/d1e/d41/f47 [4194304,4194304] 0 2024-02-19T19:04:50.157 INFO:tasks.workunit.client.1.smithi132.stdout:7/276: write d1b/d1e/f43 [374301,25807] 0 2024-02-19T19:04:50.160 INFO:tasks.workunit.client.1.smithi132.stdout:1/204: creat d1/d1e/d29/f3c x:0 0 0 2024-02-19T19:04:50.162 INFO:tasks.workunit.client.1.smithi132.stdout:0/218: fdatasync d2/dc/df/d15/d2f/f40 0 2024-02-19T19:04:50.165 INFO:tasks.workunit.client.1.smithi132.stdout:8/240: dread d4/f27 [0,4194304] 0 2024-02-19T19:04:50.165 INFO:tasks.workunit.client.1.smithi132.stdout:0/219: stat d2/dc/df/d15/d34 0 2024-02-19T19:04:50.167 INFO:tasks.workunit.client.1.smithi132.stdout:6/172: dwrite d0/db/d25/f40 [0,4194304] 0 2024-02-19T19:04:50.168 INFO:tasks.workunit.client.1.smithi132.stdout:2/231: symlink d0/d4/d11/l4c 0 2024-02-19T19:04:50.168 INFO:tasks.workunit.client.1.smithi132.stdout:7/277: mkdir d1b/d5a 0 2024-02-19T19:04:50.168 INFO:tasks.workunit.client.1.smithi132.stdout:1/205: mknod d1/d1e/d29/d2c/c3d 0 2024-02-19T19:04:50.170 INFO:tasks.workunit.client.1.smithi132.stdout:2/232: chown d0/d4/df/c4a 32711 1 2024-02-19T19:04:50.171 INFO:tasks.workunit.client.1.smithi132.stdout:0/220: mkdir d2/d21/d25/d43 0 2024-02-19T19:04:50.173 INFO:tasks.workunit.client.1.smithi132.stdout:6/173: mknod d0/d3c/d22/c47 0 2024-02-19T19:04:50.173 INFO:tasks.workunit.client.1.smithi132.stdout:2/233: stat d0/d4/d11 0 2024-02-19T19:04:50.175 INFO:tasks.workunit.client.1.smithi132.stdout:1/206: dread d1/f18 [0,4194304] 0 2024-02-19T19:04:50.178 INFO:tasks.workunit.client.1.smithi132.stdout:7/278: creat d1b/d1e/d28/f5b x:0 0 0 2024-02-19T19:04:50.182 INFO:tasks.workunit.client.1.smithi132.stdout:5/208: getdents d8/da 0 2024-02-19T19:04:50.182 INFO:tasks.workunit.client.1.smithi132.stdout:1/207: dwrite d1/f21 [0,4194304] 0 2024-02-19T19:04:50.183 INFO:tasks.workunit.client.1.smithi132.stdout:5/209: chown d8/da/db/f1d 1021 1 2024-02-19T19:04:50.183 INFO:tasks.workunit.client.1.smithi132.stdout:1/208: write d1/f2 [3808376,48303] 0 2024-02-19T19:04:50.184 INFO:tasks.workunit.client.1.smithi132.stdout:2/234: rename d0/d4/d11/f26 to d0/d4/df/f4d 0 2024-02-19T19:04:50.184 INFO:tasks.workunit.client.1.smithi132.stdout:5/210: readlink d8/d23/d29/d27/l32 0 2024-02-19T19:04:50.189 INFO:tasks.workunit.client.1.smithi132.stdout:7/279: chown d1b/d1e/d2b/l3e 127384 1 2024-02-19T19:04:50.191 INFO:tasks.workunit.client.1.smithi132.stdout:8/241: dread d4/dc/d1b/d20/d23/f2b [4194304,4194304] 0 2024-02-19T19:04:50.192 INFO:tasks.workunit.client.1.smithi132.stdout:8/242: chown d4/dc/d1b/d1d 105553 1 2024-02-19T19:04:50.194 INFO:tasks.workunit.client.1.smithi132.stdout:1/209: mknod d1/d1e/d29/c3e 0 2024-02-19T19:04:50.195 INFO:tasks.workunit.client.1.smithi132.stdout:2/235: mknod d0/d3c/c4e 0 2024-02-19T19:04:50.197 INFO:tasks.workunit.client.1.smithi132.stdout:5/211: creat d8/d23/f45 x:0 0 0 2024-02-19T19:04:50.198 INFO:tasks.workunit.client.1.smithi132.stdout:0/221: link d2/c3d d2/dc/df/d27/c44 0 2024-02-19T19:04:50.199 INFO:tasks.workunit.client.1.smithi132.stdout:9/333: sync 2024-02-19T19:04:50.199 INFO:tasks.workunit.client.1.smithi132.stdout:4/200: sync 2024-02-19T19:04:50.201 INFO:tasks.workunit.client.1.smithi132.stdout:1/210: creat d1/d1e/d29/f3f x:0 0 0 2024-02-19T19:04:50.202 INFO:tasks.workunit.client.1.smithi132.stdout:2/236: chown d0/d4/d11/c2a 23895716 1 2024-02-19T19:04:50.203 INFO:tasks.workunit.client.1.smithi132.stdout:0/222: fdatasync d2/f19 0 2024-02-19T19:04:50.204 INFO:tasks.workunit.client.1.smithi132.stdout:9/334: mkdir d12/d2a/d4c/d61/d75 0 2024-02-19T19:04:50.206 INFO:tasks.workunit.client.1.smithi132.stdout:5/212: dwrite d8/d23/d29/d27/f2b [0,4194304] 0 2024-02-19T19:04:50.207 INFO:tasks.workunit.client.1.smithi132.stdout:4/201: symlink d4/d27/l40 0 2024-02-19T19:04:50.208 INFO:tasks.workunit.client.1.smithi132.stdout:5/213: chown d8/d23/d29/d3f 128937 1 2024-02-19T19:04:50.209 INFO:tasks.workunit.client.1.smithi132.stdout:2/237: rename d0/d4/c2b to d0/d4/d11/d2e/c4f 0 2024-02-19T19:04:50.209 INFO:tasks.workunit.client.1.smithi132.stdout:5/214: stat d8/da/f33 0 2024-02-19T19:04:50.211 INFO:tasks.workunit.client.1.smithi132.stdout:9/335: creat d12/d25/f76 x:0 0 0 2024-02-19T19:04:50.213 INFO:tasks.workunit.client.1.smithi132.stdout:8/243: getdents d4/dc/d1b 0 2024-02-19T19:04:50.214 INFO:tasks.workunit.client.1.smithi132.stdout:4/202: mknod d4/d12/c41 0 2024-02-19T19:04:50.216 INFO:tasks.workunit.client.1.smithi132.stdout:4/203: dread - d4/d19/f3f zero size 2024-02-19T19:04:50.219 INFO:tasks.workunit.client.1.smithi132.stdout:5/215: dwrite d8/fe [0,4194304] 0 2024-02-19T19:04:50.219 INFO:tasks.workunit.client.1.smithi132.stdout:8/244: mknod d4/dc/d1b/d38/c58 0 2024-02-19T19:04:50.223 INFO:tasks.workunit.client.1.smithi132.stdout:8/245: readlink d4/la 0 2024-02-19T19:04:50.228 INFO:tasks.workunit.client.1.smithi132.stdout:2/238: getdents d0/d19/d1e 0 2024-02-19T19:04:50.229 INFO:tasks.workunit.client.1.smithi132.stdout:1/211: dwrite d1/f18 [0,4194304] 0 2024-02-19T19:04:50.231 INFO:tasks.workunit.client.1.smithi132.stdout:9/336: dwrite d12/f1a [0,4194304] 0 2024-02-19T19:04:50.234 INFO:tasks.workunit.client.1.smithi132.stdout:5/216: rename d8/d23/d29/c2a to d8/d1a/c46 0 2024-02-19T19:04:50.236 INFO:tasks.workunit.client.1.smithi132.stdout:7/280: dwrite d1b/d1e/d28/f50 [0,4194304] 0 2024-02-19T19:04:50.238 INFO:tasks.workunit.client.1.smithi132.stdout:1/212: fsync d1/d1e/d29/d2c/f35 0 2024-02-19T19:04:50.241 INFO:tasks.workunit.client.1.smithi132.stdout:0/223: dread d2/f1a [0,4194304] 0 2024-02-19T19:04:50.242 INFO:tasks.workunit.client.1.smithi132.stdout:4/204: dwrite d4/d12/f30 [0,4194304] 0 2024-02-19T19:04:50.243 INFO:tasks.workunit.client.1.smithi132.stdout:9/337: write d12/d25/d3a/d40/f65 [244066,129893] 0 2024-02-19T19:04:50.246 INFO:tasks.workunit.client.1.smithi132.stdout:7/281: write d1b/d1e/d28/f44 [1522705,60391] 0 2024-02-19T19:04:50.248 INFO:tasks.workunit.client.1.smithi132.stdout:9/338: chown d12/d1b/l59 49125725 1 2024-02-19T19:04:50.248 INFO:tasks.workunit.client.1.smithi132.stdout:4/205: write d4/f7 [6170625,29411] 0 2024-02-19T19:04:50.250 INFO:tasks.workunit.client.1.smithi132.stdout:0/224: chown d2/dc/df/d15/l30 4 1 2024-02-19T19:04:50.252 INFO:tasks.workunit.client.1.smithi132.stdout:7/282: read - d1b/d1e/d28/f55 zero size 2024-02-19T19:04:50.256 INFO:tasks.workunit.client.1.smithi132.stdout:8/246: dwrite d4/f6 [0,4194304] 0 2024-02-19T19:04:50.257 INFO:tasks.workunit.client.1.smithi132.stdout:9/339: write d12/d1b/f1c [8614304,53854] 0 2024-02-19T19:04:50.261 INFO:tasks.workunit.client.1.smithi132.stdout:5/217: dwrite f5 [8388608,4194304] 0 2024-02-19T19:04:50.262 INFO:tasks.workunit.client.1.smithi132.stdout:2/239: dwrite d0/d4/df/d40/f4b [0,4194304] 0 2024-02-19T19:04:50.263 INFO:tasks.workunit.client.1.smithi132.stdout:2/240: dread - d0/f43 zero size 2024-02-19T19:04:50.264 INFO:tasks.workunit.client.1.smithi132.stdout:0/225: stat d2/c3d 0 2024-02-19T19:04:50.266 INFO:tasks.workunit.client.1.smithi132.stdout:1/213: link d1/d1e/d29/f2e d1/d3a/f40 0 2024-02-19T19:04:50.266 INFO:tasks.workunit.client.1.smithi132.stdout:3/169: sync 2024-02-19T19:04:50.267 INFO:tasks.workunit.client.1.smithi132.stdout:9/340: write d12/d25/d3a/f52 [1059843,20854] 0 2024-02-19T19:04:50.267 INFO:tasks.workunit.client.1.smithi132.stdout:1/214: stat d1/c1c 0 2024-02-19T19:04:50.267 INFO:tasks.workunit.client.1.smithi132.stdout:8/247: fsync d4/dc/d1a/f4a 0 2024-02-19T19:04:50.268 INFO:tasks.workunit.client.1.smithi132.stdout:0/226: fdatasync d2/dc/f3b 0 2024-02-19T19:04:50.272 INFO:tasks.workunit.client.1.smithi132.stdout:5/218: chown d8/d23/d29/d27 31678 1 2024-02-19T19:04:50.272 INFO:tasks.workunit.client.1.smithi132.stdout:2/241: symlink d0/d3c/d3f/l50 0 2024-02-19T19:04:50.274 INFO:tasks.workunit.client.1.smithi132.stdout:4/206: link d4/l22 d4/dc/d14/d24/l42 0 2024-02-19T19:04:50.275 INFO:tasks.workunit.client.1.smithi132.stdout:9/341: mkdir d12/d25/d3a/d40/d4e/d77 0 2024-02-19T19:04:50.277 INFO:tasks.workunit.client.1.smithi132.stdout:7/283: dwrite d1b/d1e/d28/f55 [0,4194304] 0 2024-02-19T19:04:50.280 INFO:tasks.workunit.client.1.smithi132.stdout:0/227: write d2/dc/f3b [512521,6804] 0 2024-02-19T19:04:50.280 INFO:tasks.workunit.client.1.smithi132.stdout:4/207: chown d4/f10 238 1 2024-02-19T19:04:50.282 INFO:tasks.workunit.client.1.smithi132.stdout:2/242: creat d0/d4/df/f51 x:0 0 0 2024-02-19T19:04:50.282 INFO:tasks.workunit.client.1.smithi132.stdout:7/284: chown d1b/d1e/d2b/f51 13021200 1 2024-02-19T19:04:50.284 INFO:tasks.workunit.client.1.smithi132.stdout:9/342: dread d12/d2a/f5b [0,4194304] 0 2024-02-19T19:04:50.284 INFO:tasks.workunit.client.1.smithi132.stdout:5/219: creat d8/d23/d29/d3f/f47 x:0 0 0 2024-02-19T19:04:50.286 INFO:tasks.workunit.client.1.smithi132.stdout:3/170: dwrite d8/f14 [0,4194304] 0 2024-02-19T19:04:50.286 INFO:tasks.workunit.client.1.smithi132.stdout:5/220: chown d8/d1a/c37 45640572 1 2024-02-19T19:04:50.287 INFO:tasks.workunit.client.1.smithi132.stdout:3/171: read d8/d13/f20 [1427570,25821] 0 2024-02-19T19:04:50.290 INFO:tasks.workunit.client.1.smithi132.stdout:0/228: fsync d2/f2b 0 2024-02-19T19:04:50.291 INFO:tasks.workunit.client.1.smithi132.stdout:4/208: rename d4/dc/c2e to d4/dc/d14/c43 0 2024-02-19T19:04:50.294 INFO:tasks.workunit.client.1.smithi132.stdout:5/221: dwrite d8/da/db/f44 [0,4194304] 0 2024-02-19T19:04:50.297 INFO:tasks.workunit.client.1.smithi132.stdout:3/172: symlink d8/l35 0 2024-02-19T19:04:50.299 INFO:tasks.workunit.client.1.smithi132.stdout:7/285: mkdir d1b/d5a/d5c 0 2024-02-19T19:04:50.300 INFO:tasks.workunit.client.1.smithi132.stdout:2/243: getdents d0/d19/d1e 0 2024-02-19T19:04:50.300 INFO:tasks.workunit.client.1.smithi132.stdout:7/286: read d1b/d1e/d28/f50 [2047752,45348] 0 2024-02-19T19:04:50.302 INFO:tasks.workunit.client.1.smithi132.stdout:4/209: mknod d4/dc/d14/c44 0 2024-02-19T19:04:50.302 INFO:tasks.workunit.client.1.smithi132.stdout:9/343: mkdir d12/d25/d3a/d78 0 2024-02-19T19:04:50.303 INFO:tasks.workunit.client.1.smithi132.stdout:5/222: write d8/d2e/f39 [897002,35323] 0 2024-02-19T19:04:50.306 INFO:tasks.workunit.client.1.smithi132.stdout:5/223: truncate d8/da/db/f1b 1658490 0 2024-02-19T19:04:50.306 INFO:tasks.workunit.client.1.smithi132.stdout:0/229: mknod d2/dc/df/d27/d3a/c45 0 2024-02-19T19:04:50.307 INFO:tasks.workunit.client.1.smithi132.stdout:2/244: unlink d0/d4/f12 0 2024-02-19T19:04:50.309 INFO:tasks.workunit.client.1.smithi132.stdout:7/287: creat d1b/d1e/d2b/f5d x:0 0 0 2024-02-19T19:04:50.309 INFO:tasks.workunit.client.1.smithi132.stdout:4/210: mknod d4/d19/d21/c45 0 2024-02-19T19:04:50.311 INFO:tasks.workunit.client.1.smithi132.stdout:3/173: creat d8/d13/d2f/f36 x:0 0 0 2024-02-19T19:04:50.313 INFO:tasks.workunit.client.1.smithi132.stdout:9/344: rename d12/f1a to d12/d3e/d6e/f79 0 2024-02-19T19:04:50.315 INFO:tasks.workunit.client.1.smithi132.stdout:2/245: dwrite d0/d4/d11/f41 [4194304,4194304] 0 2024-02-19T19:04:50.319 INFO:tasks.workunit.client.1.smithi132.stdout:7/288: mkdir d1b/d1e/d28/d5e 0 2024-02-19T19:04:50.321 INFO:tasks.workunit.client.1.smithi132.stdout:9/345: dwrite d12/d25/d3a/d40/f66 [0,4194304] 0 2024-02-19T19:04:50.324 INFO:tasks.workunit.client.1.smithi132.stdout:4/211: mknod d4/d12/d16/c46 0 2024-02-19T19:04:50.328 INFO:tasks.workunit.client.1.smithi132.stdout:5/224: creat d8/d23/d3b/f48 x:0 0 0 2024-02-19T19:04:50.329 INFO:tasks.workunit.client.1.smithi132.stdout:5/225: read - d8/d23/f45 zero size 2024-02-19T19:04:50.330 INFO:tasks.workunit.client.1.smithi132.stdout:5/226: write d8/da/f3c [2049830,123138] 0 2024-02-19T19:04:50.331 INFO:tasks.workunit.client.1.smithi132.stdout:5/227: fsync d8/ff 0 2024-02-19T19:04:50.332 INFO:tasks.workunit.client.1.smithi132.stdout:5/228: chown d8/d23/d29/f1c 198455 1 2024-02-19T19:04:50.332 INFO:tasks.workunit.client.1.smithi132.stdout:5/229: readlink d8/l22 0 2024-02-19T19:04:50.333 INFO:tasks.workunit.client.1.smithi132.stdout:5/230: dread - d8/d23/d3b/f48 zero size 2024-02-19T19:04:50.334 INFO:tasks.workunit.client.1.smithi132.stdout:7/289: symlink d1b/d46/d48/l5f 0 2024-02-19T19:04:50.336 INFO:tasks.workunit.client.1.smithi132.stdout:9/346: creat d12/d2a/d4c/d61/f7a x:0 0 0 2024-02-19T19:04:50.338 INFO:tasks.workunit.client.1.smithi132.stdout:4/212: creat d4/d12/d16/f47 x:0 0 0 2024-02-19T19:04:50.341 INFO:tasks.workunit.client.1.smithi132.stdout:0/230: link d2/dc/df/f2d d2/dc/df/d15/f46 0 2024-02-19T19:04:50.342 INFO:tasks.workunit.client.1.smithi132.stdout:0/231: truncate d2/dc/df/d27/f35 4205943 0 2024-02-19T19:04:50.344 INFO:tasks.workunit.client.1.smithi132.stdout:0/232: write d2/f1a [2679671,76088] 0 2024-02-19T19:04:50.344 INFO:tasks.workunit.client.1.smithi132.stdout:2/246: fdatasync d0/d4/d11/d2e/f3e 0 2024-02-19T19:04:50.345 INFO:tasks.workunit.client.1.smithi132.stdout:6/174: sync 2024-02-19T19:04:50.345 INFO:tasks.workunit.client.1.smithi132.stdout:5/231: unlink d8/d1a/f41 0 2024-02-19T19:04:50.346 INFO:tasks.workunit.client.1.smithi132.stdout:6/175: dread - d0/d3c/d22/f19 zero size 2024-02-19T19:04:50.348 INFO:tasks.workunit.client.1.smithi132.stdout:3/174: creat d8/d13/f37 x:0 0 0 2024-02-19T19:04:50.349 INFO:tasks.workunit.client.1.smithi132.stdout:4/213: mkdir d4/d12/d48 0 2024-02-19T19:04:50.353 INFO:tasks.workunit.client.1.smithi132.stdout:0/233: mkdir d2/dc/df/d15/d2f/d47 0 2024-02-19T19:04:50.354 INFO:tasks.workunit.client.1.smithi132.stdout:0/234: write d2/dc/df/d15/d2f/f40 [195553,129029] 0 2024-02-19T19:04:50.355 INFO:tasks.workunit.client.1.smithi132.stdout:0/235: chown d2/dc/df/d15/d2f/c39 3011 1 2024-02-19T19:04:50.356 INFO:tasks.workunit.client.1.smithi132.stdout:0/236: write d2/d21/f42 [4811701,28503] 0 2024-02-19T19:04:50.393 INFO:tasks.workunit.client.1.smithi132.stdout:3/175: mkdir d8/d38 0 2024-02-19T19:04:50.396 INFO:tasks.workunit.client.1.smithi132.stdout:4/214: write d4/d12/d28/f3e [370841,93194] 0 2024-02-19T19:04:50.397 INFO:tasks.workunit.client.1.smithi132.stdout:3/176: truncate d8/d13/f16 2011766 0 2024-02-19T19:04:50.400 INFO:tasks.workunit.client.1.smithi132.stdout:0/237: symlink d2/dc/df/d27/l48 0 2024-02-19T19:04:50.404 INFO:tasks.workunit.client.1.smithi132.stdout:4/215: mkdir d4/dc/d14/d24/d49 0 2024-02-19T19:04:50.405 INFO:tasks.workunit.client.1.smithi132.stdout:3/177: symlink d8/d13/d19/l39 0 2024-02-19T19:04:50.408 INFO:tasks.workunit.client.1.smithi132.stdout:0/238: dread - d2/dc/df/f2d zero size 2024-02-19T19:04:50.409 INFO:tasks.workunit.client.1.smithi132.stdout:4/216: write d4/d19/d21/f36 [5080839,105990] 0 2024-02-19T19:04:50.412 INFO:tasks.workunit.client.1.smithi132.stdout:2/247: dwrite d0/f5 [0,4194304] 0 2024-02-19T19:04:50.412 INFO:tasks.workunit.client.1.smithi132.stdout:9/347: dwrite d12/d2a/f37 [4194304,4194304] 0 2024-02-19T19:04:50.413 INFO:tasks.workunit.client.1.smithi132.stdout:7/290: dread d1b/f1c [0,4194304] 0 2024-02-19T19:04:50.414 INFO:tasks.workunit.client.1.smithi132.stdout:1/215: sync 2024-02-19T19:04:50.415 INFO:tasks.workunit.client.1.smithi132.stdout:3/178: creat d8/d13/d2b/d2e/f3a x:0 0 0 2024-02-19T19:04:50.418 INFO:tasks.workunit.client.1.smithi132.stdout:1/216: stat d1/c1c 0 2024-02-19T19:04:50.420 INFO:tasks.workunit.client.1.smithi132.stdout:1/217: chown d1/d1e 397 1 2024-02-19T19:04:50.420 INFO:tasks.workunit.client.1.smithi132.stdout:9/348: chown d12/f23 2235855 1 2024-02-19T19:04:50.420 INFO:tasks.workunit.client.1.smithi132.stdout:2/248: creat d0/d19/d1e/f52 x:0 0 0 2024-02-19T19:04:50.426 INFO:tasks.workunit.client.1.smithi132.stdout:7/291: unlink d1b/d46/d48/l5f 0 2024-02-19T19:04:50.426 INFO:tasks.workunit.client.1.smithi132.stdout:1/218: write d1/d1e/f2a [760425,64946] 0 2024-02-19T19:04:50.428 INFO:tasks.workunit.client.1.smithi132.stdout:0/239: dread d2/f12 [0,4194304] 0 2024-02-19T19:04:50.428 INFO:tasks.workunit.client.1.smithi132.stdout:3/179: rename d8/f10 to d8/d13/d2b/d2e/f3b 0 2024-02-19T19:04:50.431 INFO:tasks.workunit.client.1.smithi132.stdout:0/240: chown d2/d21/d25/d43 0 1 2024-02-19T19:04:50.431 INFO:tasks.workunit.client.1.smithi132.stdout:3/180: stat d8/l35 0 2024-02-19T19:04:50.432 INFO:tasks.workunit.client.1.smithi132.stdout:0/241: fdatasync d2/dc/df/d15/d34/f3c 0 2024-02-19T19:04:50.433 INFO:tasks.workunit.client.1.smithi132.stdout:2/249: write d0/d4/f1f [968754,13221] 0 2024-02-19T19:04:50.435 INFO:tasks.workunit.client.1.smithi132.stdout:3/181: symlink d8/d31/l3c 0 2024-02-19T19:04:50.437 INFO:tasks.workunit.client.1.smithi132.stdout:0/242: dread d2/dc/f3b [0,4194304] 0 2024-02-19T19:04:50.439 INFO:tasks.workunit.client.1.smithi132.stdout:7/292: rename d1b/d1e/c27 to d1b/d46/c60 0 2024-02-19T19:04:50.440 INFO:tasks.workunit.client.1.smithi132.stdout:1/219: link d1/d3a/f39 d1/d1e/d29/f41 0 2024-02-19T19:04:50.442 INFO:tasks.workunit.client.1.smithi132.stdout:1/220: write d1/f1b [1142796,108185] 0 2024-02-19T19:04:50.442 INFO:tasks.workunit.client.1.smithi132.stdout:2/250: mkdir d0/d4/d33/d53 0 2024-02-19T19:04:50.444 INFO:tasks.workunit.client.1.smithi132.stdout:3/182: dwrite d8/f12 [0,4194304] 0 2024-02-19T19:04:50.445 INFO:tasks.workunit.client.1.smithi132.stdout:2/251: readlink d0/d4/df/l16 0 2024-02-19T19:04:50.445 INFO:tasks.workunit.client.1.smithi132.stdout:0/243: dwrite d2/f6 [4194304,4194304] 0 2024-02-19T19:04:50.447 INFO:tasks.workunit.client.1.smithi132.stdout:9/349: truncate fb 4055706 0 2024-02-19T19:04:50.448 INFO:tasks.workunit.client.1.smithi132.stdout:0/244: write d2/dc/df/d27/f35 [1747405,66519] 0 2024-02-19T19:04:50.449 INFO:tasks.workunit.client.1.smithi132.stdout:8/248: sync 2024-02-19T19:04:50.449 INFO:tasks.workunit.client.1.smithi132.stdout:7/293: mknod d1b/c61 0 2024-02-19T19:04:50.450 INFO:tasks.workunit.client.1.smithi132.stdout:0/245: fdatasync d2/dc/df/d15/d34/f3c 0 2024-02-19T19:04:50.452 INFO:tasks.workunit.client.1.smithi132.stdout:0/246: write d2/dc/df/d15/d34/f41 [72011,129389] 0 2024-02-19T19:04:50.458 INFO:tasks.workunit.client.1.smithi132.stdout:4/217: truncate d4/d19/d21/f36 3142582 0 2024-02-19T19:04:50.463 INFO:tasks.workunit.client.1.smithi132.stdout:3/183: creat d8/d38/f3d x:0 0 0 2024-02-19T19:04:50.464 INFO:tasks.workunit.client.1.smithi132.stdout:4/218: mknod d4/d19/c4a 0 2024-02-19T19:04:50.465 INFO:tasks.workunit.client.1.smithi132.stdout:4/219: write d4/d12/f2c [110677,61620] 0 2024-02-19T19:04:50.467 INFO:tasks.workunit.client.1.smithi132.stdout:1/221: rename d1/d3a/f39 to d1/f42 0 2024-02-19T19:04:50.470 INFO:tasks.workunit.client.1.smithi132.stdout:1/222: truncate d1/d1e/d29/d2c/f34 977610 0 2024-02-19T19:04:50.470 INFO:tasks.workunit.client.1.smithi132.stdout:2/252: link d0/d4/df/d40/c44 d0/d24/c54 0 2024-02-19T19:04:50.470 INFO:tasks.workunit.client.1.smithi132.stdout:1/223: chown d1/d1e/d29/f41 11218155 1 2024-02-19T19:04:50.472 INFO:tasks.workunit.client.1.smithi132.stdout:4/220: creat d4/d12/f4b x:0 0 0 2024-02-19T19:04:50.474 INFO:tasks.workunit.client.1.smithi132.stdout:9/350: rename d12/l46 to d12/d25/d3a/d40/d4e/d77/l7b 0 2024-02-19T19:04:50.474 INFO:tasks.workunit.client.1.smithi132.stdout:2/253: dread d0/d4/f1f [0,4194304] 0 2024-02-19T19:04:50.476 INFO:tasks.workunit.client.1.smithi132.stdout:2/254: stat d0/d4/d11/f15 0 2024-02-19T19:04:50.477 INFO:tasks.workunit.client.1.smithi132.stdout:1/224: dwrite d1/f5 [0,4194304] 0 2024-02-19T19:04:50.478 INFO:tasks.workunit.client.1.smithi132.stdout:4/221: mkdir d4/d12/d4c 0 2024-02-19T19:04:50.479 INFO:tasks.workunit.client.1.smithi132.stdout:4/222: write d4/dc/d14/d24/f39 [1362081,49115] 0 2024-02-19T19:04:50.483 INFO:tasks.workunit.client.1.smithi132.stdout:7/294: rename d1b/d46/d48 to d1b/d1e/d2b/d62 0 2024-02-19T19:04:50.483 INFO:tasks.workunit.client.1.smithi132.stdout:7/295: chown d1b/d1e/f21 2379 1 2024-02-19T19:04:50.487 INFO:tasks.workunit.client.1.smithi132.stdout:9/351: unlink d12/d2a/d4c/d61/l64 0 2024-02-19T19:04:50.488 INFO:tasks.workunit.client.1.smithi132.stdout:8/249: write d4/f16 [1011718,1955] 0 2024-02-19T19:04:50.500 INFO:tasks.workunit.client.1.smithi132.stdout:2/255: mknod d0/d4/d33/d53/c55 0 2024-02-19T19:04:50.502 INFO:tasks.workunit.client.1.smithi132.stdout:9/352: rename d12/d25/d3a/d40/l4d to d12/d25/l7c 0 2024-02-19T19:04:50.503 INFO:tasks.workunit.client.1.smithi132.stdout:9/353: chown d12/d25/f2e 0 1 2024-02-19T19:04:50.504 INFO:tasks.workunit.client.1.smithi132.stdout:7/296: rmdir d1b/d46 39 2024-02-19T19:04:50.506 INFO:tasks.workunit.client.1.smithi132.stdout:9/354: dread d12/d25/d3a/d40/f65 [0,4194304] 0 2024-02-19T19:04:50.506 INFO:tasks.workunit.client.1.smithi132.stdout:9/355: chown d12/f33 2 1 2024-02-19T19:04:50.509 INFO:tasks.workunit.client.1.smithi132.stdout:2/256: unlink d0/f43 0 2024-02-19T19:04:50.510 INFO:tasks.workunit.client.1.smithi132.stdout:1/225: write d1/f42 [81461,109497] 0 2024-02-19T19:04:50.511 INFO:tasks.workunit.client.1.smithi132.stdout:2/257: write d0/d4/d11/f20 [184791,71342] 0 2024-02-19T19:04:50.515 INFO:tasks.workunit.client.1.smithi132.stdout:8/250: link d4/dc/d1b/d20/d23/f2b d4/dc/d1b/d20/f59 0 2024-02-19T19:04:50.519 INFO:tasks.workunit.client.1.smithi132.stdout:9/356: write d12/d1b/f50 [3502395,3315] 0 2024-02-19T19:04:50.521 INFO:tasks.workunit.client.1.smithi132.stdout:9/357: truncate d12/d25/d3a/d40/f66 4949091 0 2024-02-19T19:04:50.521 INFO:tasks.workunit.client.1.smithi132.stdout:4/223: write d4/d19/d21/f36 [3849955,23347] 0 2024-02-19T19:04:50.522 INFO:tasks.workunit.client.1.smithi132.stdout:4/224: write d4/d27/f3c [370407,58293] 0 2024-02-19T19:04:50.525 INFO:tasks.workunit.client.1.smithi132.stdout:1/226: unlink d1/d1e/d29/d2c/c3d 0 2024-02-19T19:04:50.527 INFO:tasks.workunit.client.1.smithi132.stdout:2/258: unlink d0/d4/df/d40/f4b 0 2024-02-19T19:04:50.529 INFO:tasks.workunit.client.1.smithi132.stdout:8/251: rename d4/dc/d1b/d20/d23/l4c to d4/l5a 0 2024-02-19T19:04:50.530 INFO:tasks.workunit.client.1.smithi132.stdout:8/252: write d4/dc/d1b/d1d/f36 [1376083,10211] 0 2024-02-19T19:04:50.531 INFO:tasks.workunit.client.1.smithi132.stdout:7/297: getdents d1b/d5a/d5c 0 2024-02-19T19:04:50.535 INFO:tasks.workunit.client.1.smithi132.stdout:1/227: unlink d1/f2 0 2024-02-19T19:04:50.536 INFO:tasks.workunit.client.1.smithi132.stdout:2/259: symlink d0/d4/d33/l56 0 2024-02-19T19:04:50.538 INFO:tasks.workunit.client.1.smithi132.stdout:7/298: mkdir d1b/d5a/d63 0 2024-02-19T19:04:50.540 INFO:tasks.workunit.client.1.smithi132.stdout:8/253: creat d4/dc/d1b/d33/f5b x:0 0 0 2024-02-19T19:04:50.540 INFO:tasks.workunit.client.1.smithi132.stdout:8/254: write d4/f16 [302960,50311] 0 2024-02-19T19:04:50.542 INFO:tasks.workunit.client.1.smithi132.stdout:8/255: fdatasync d4/dc/d1b/d1d/f36 0 2024-02-19T19:04:50.544 INFO:tasks.workunit.client.1.smithi132.stdout:4/225: mkdir d4/d4d 0 2024-02-19T19:04:50.544 INFO:tasks.workunit.client.1.smithi132.stdout:4/226: fsync d4/d12/d28/f3e 0 2024-02-19T19:04:50.545 INFO:tasks.workunit.client.1.smithi132.stdout:4/227: truncate d4/d12/d28/f3e 1033693 0 2024-02-19T19:04:50.545 INFO:tasks.workunit.client.1.smithi132.stdout:4/228: truncate d4/f2d 459919 0 2024-02-19T19:04:50.547 INFO:tasks.workunit.client.1.smithi132.stdout:1/228: mkdir d1/d1e/d29/d2c/d43 0 2024-02-19T19:04:50.548 INFO:tasks.workunit.client.1.smithi132.stdout:1/229: chown d1/d3a/c20 278950872 1 2024-02-19T19:04:50.548 INFO:tasks.workunit.client.1.smithi132.stdout:2/260: symlink d0/l57 0 2024-02-19T19:04:50.549 INFO:tasks.workunit.client.1.smithi132.stdout:2/261: write d0/d19/f2c [580114,109677] 0 2024-02-19T19:04:50.551 INFO:tasks.workunit.client.1.smithi132.stdout:5/232: sync 2024-02-19T19:04:50.551 INFO:tasks.workunit.client.1.smithi132.stdout:6/176: sync 2024-02-19T19:04:50.552 INFO:tasks.workunit.client.1.smithi132.stdout:5/233: chown d8/da/db/f44 33 1 2024-02-19T19:04:50.552 INFO:tasks.workunit.client.1.smithi132.stdout:6/177: truncate d0/d3c/f42 509689 0 2024-02-19T19:04:50.553 INFO:tasks.workunit.client.1.smithi132.stdout:8/256: mknod d4/dc/d1b/d20/d23/c5c 0 2024-02-19T19:04:50.554 INFO:tasks.workunit.client.1.smithi132.stdout:8/257: write d4/f6 [793108,96815] 0 2024-02-19T19:04:50.554 INFO:tasks.workunit.client.1.smithi132.stdout:8/258: dread - d4/dc/d1b/d1d/f3c zero size 2024-02-19T19:04:50.560 INFO:tasks.workunit.client.1.smithi132.stdout:1/230: unlink d1/d1e/f36 0 2024-02-19T19:04:50.560 INFO:tasks.workunit.client.1.smithi132.stdout:8/259: dwrite d4/dc/d1a/f50 [0,4194304] 0 2024-02-19T19:04:50.561 INFO:tasks.workunit.client.1.smithi132.stdout:8/260: write d4/dc/d1b/d33/f49 [311360,19405] 0 2024-02-19T19:04:50.562 INFO:tasks.workunit.client.1.smithi132.stdout:8/261: chown d4/dc 14 1 2024-02-19T19:04:50.563 INFO:tasks.workunit.client.1.smithi132.stdout:8/262: write d4/fb [2661931,57303] 0 2024-02-19T19:04:50.564 INFO:tasks.workunit.client.1.smithi132.stdout:2/262: rename d0/d4/d33 to d0/d24/d58 0 2024-02-19T19:04:50.566 INFO:tasks.workunit.client.1.smithi132.stdout:5/234: mknod d8/d23/d29/c49 0 2024-02-19T19:04:50.568 INFO:tasks.workunit.client.1.smithi132.stdout:2/263: dread - d0/d19/d1e/f45 zero size 2024-02-19T19:04:50.568 INFO:tasks.workunit.client.1.smithi132.stdout:5/235: chown d8/da/f33 55903753 1 2024-02-19T19:04:50.569 INFO:tasks.workunit.client.1.smithi132.stdout:4/229: creat d4/d27/d31/d32/f4e x:0 0 0 2024-02-19T19:04:50.570 INFO:tasks.workunit.client.1.smithi132.stdout:9/358: dwrite fb [4194304,4194304] 0 2024-02-19T19:04:50.571 INFO:tasks.workunit.client.1.smithi132.stdout:4/230: chown d4/d12/d16/c1f 244939146 1 2024-02-19T19:04:50.578 INFO:tasks.workunit.client.1.smithi132.stdout:5/236: dwrite d8/d23/f45 [0,4194304] 0 2024-02-19T19:04:50.579 INFO:tasks.workunit.client.1.smithi132.stdout:2/264: dwrite d0/d4/d11/f1c [0,4194304] 0 2024-02-19T19:04:50.579 INFO:tasks.workunit.client.1.smithi132.stdout:1/231: rename d1/d1e/d29/c2b to d1/d1e/d29/c44 0 2024-02-19T19:04:50.583 INFO:tasks.workunit.client.1.smithi132.stdout:4/231: dwrite d4/d19/d21/f36 [0,4194304] 0 2024-02-19T19:04:50.585 INFO:tasks.workunit.client.1.smithi132.stdout:2/265: write d0/d24/d58/f36 [4311982,117692] 0 2024-02-19T19:04:50.590 INFO:tasks.workunit.client.1.smithi132.stdout:5/237: write d8/d1a/d35/f40 [885132,100335] 0 2024-02-19T19:04:50.598 INFO:tasks.workunit.client.1.smithi132.stdout:4/232: dwrite d4/d12/d16/f47 [0,4194304] 0 2024-02-19T19:04:50.608 INFO:tasks.workunit.client.1.smithi132.stdout:0/247: sync 2024-02-19T19:04:50.608 INFO:tasks.workunit.client.1.smithi132.stdout:8/263: mknod d4/dc/d1b/d54/c5d 0 2024-02-19T19:04:50.609 INFO:tasks.workunit.client.1.smithi132.stdout:0/248: write d2/f1a [4077123,88058] 0 2024-02-19T19:04:50.610 INFO:tasks.workunit.client.1.smithi132.stdout:8/264: fdatasync d4/dc/f2c 0 2024-02-19T19:04:50.611 INFO:tasks.workunit.client.1.smithi132.stdout:8/265: truncate d4/dc/d1a/f4a 353574 0 2024-02-19T19:04:50.616 INFO:tasks.workunit.client.1.smithi132.stdout:9/359: rename d12/d1b/f20 to d12/d2a/f7d 0 2024-02-19T19:04:50.619 INFO:tasks.workunit.client.1.smithi132.stdout:1/232: mknod d1/d1e/d29/d2c/c45 0 2024-02-19T19:04:50.622 INFO:tasks.workunit.client.1.smithi132.stdout:9/360: dwrite d12/d3e/f72 [0,4194304] 0 2024-02-19T19:04:50.630 INFO:tasks.workunit.client.1.smithi132.stdout:6/178: truncate d0/db/d25/f40 1723405 0 2024-02-19T19:04:50.637 INFO:tasks.workunit.client.1.smithi132.stdout:4/233: rmdir d4/dc 39 2024-02-19T19:04:50.645 INFO:tasks.workunit.client.1.smithi132.stdout:0/249: symlink d2/dc/l49 0 2024-02-19T19:04:50.646 INFO:tasks.workunit.client.1.smithi132.stdout:2/266: dread d0/d24/f2d [0,4194304] 0 2024-02-19T19:04:50.650 INFO:tasks.workunit.client.1.smithi132.stdout:1/233: symlink d1/d1e/d29/d2c/l46 0 2024-02-19T19:04:50.654 INFO:tasks.workunit.client.1.smithi132.stdout:2/267: dwrite d0/d19/d1e/f45 [0,4194304] 0 2024-02-19T19:04:50.656 INFO:tasks.workunit.client.1.smithi132.stdout:8/266: write f2 [6884587,99544] 0 2024-02-19T19:04:50.658 INFO:tasks.workunit.client.1.smithi132.stdout:6/179: unlink d0/c32 0 2024-02-19T19:04:50.660 INFO:tasks.workunit.client.1.smithi132.stdout:6/180: fdatasync d0/d3c/f42 0 2024-02-19T19:04:50.663 INFO:tasks.workunit.client.1.smithi132.stdout:1/234: dwrite d1/d1e/d29/f3f [0,4194304] 0 2024-02-19T19:04:50.666 INFO:tasks.workunit.client.1.smithi132.stdout:2/268: mknod d0/d4/d11/d2e/c59 0 2024-02-19T19:04:50.669 INFO:tasks.workunit.client.1.smithi132.stdout:8/267: rename d4/dc/d1a to d4/dc/d1b/d54/d5e 0 2024-02-19T19:04:50.671 INFO:tasks.workunit.client.1.smithi132.stdout:6/181: creat d0/d3c/f48 x:0 0 0 2024-02-19T19:04:50.673 INFO:tasks.workunit.client.1.smithi132.stdout:9/361: dread fa [0,4194304] 0 2024-02-19T19:04:50.673 INFO:tasks.workunit.client.1.smithi132.stdout:5/238: getdents d8/d23/d29 0 2024-02-19T19:04:50.674 INFO:tasks.workunit.client.1.smithi132.stdout:2/269: mkdir d0/d4/d11/d2e/d5a 0 2024-02-19T19:04:50.675 INFO:tasks.workunit.client.1.smithi132.stdout:5/239: write d8/fe [512940,46454] 0 2024-02-19T19:04:50.675 INFO:tasks.workunit.client.1.smithi132.stdout:6/182: dread d0/f5 [0,4194304] 0 2024-02-19T19:04:50.676 INFO:tasks.workunit.client.1.smithi132.stdout:4/234: rename d4/l13 to d4/dc/l4f 0 2024-02-19T19:04:50.677 INFO:tasks.workunit.client.1.smithi132.stdout:6/183: chown d0/d3c/f39 845656 1 2024-02-19T19:04:50.683 INFO:tasks.workunit.client.1.smithi132.stdout:6/184: dwrite d0/db/d16/f36 [0,4194304] 0 2024-02-19T19:04:50.703 INFO:tasks.workunit.client.1.smithi132.stdout:3/184: sync 2024-02-19T19:04:50.703 INFO:tasks.workunit.client.1.smithi132.stdout:7/299: sync 2024-02-19T19:04:50.703 INFO:tasks.workunit.client.1.smithi132.stdout:5/240: rename d8/d23/d29/d3f/f42 to d8/d1a/d35/f4a 0 2024-02-19T19:04:50.705 INFO:tasks.workunit.client.1.smithi132.stdout:6/185: chown d0/c1c 290612243 1 2024-02-19T19:04:50.705 INFO:tasks.workunit.client.1.smithi132.stdout:3/185: truncate d8/d13/f24 666988 0 2024-02-19T19:04:50.706 INFO:tasks.workunit.client.1.smithi132.stdout:9/362: truncate d12/d25/f30 1562251 0 2024-02-19T19:04:50.706 INFO:tasks.workunit.client.1.smithi132.stdout:3/186: fdatasync d8/d13/f24 0 2024-02-19T19:04:50.707 INFO:tasks.workunit.client.1.smithi132.stdout:6/186: dread d0/d3c/f42 [0,4194304] 0 2024-02-19T19:04:50.708 INFO:tasks.workunit.client.1.smithi132.stdout:1/235: getdents d1/d1e/d29/d2c 0 2024-02-19T19:04:50.709 INFO:tasks.workunit.client.1.smithi132.stdout:9/363: write d12/d25/f76 [847454,7900] 0 2024-02-19T19:04:50.715 INFO:tasks.workunit.client.1.smithi132.stdout:8/268: getdents d4/d2d 0 2024-02-19T19:04:50.715 INFO:tasks.workunit.client.1.smithi132.stdout:3/187: dwrite d8/f12 [0,4194304] 0 2024-02-19T19:04:50.715 INFO:tasks.workunit.client.1.smithi132.stdout:3/188: chown d8/d13/f20 523193448 1 2024-02-19T19:04:50.717 INFO:tasks.workunit.client.1.smithi132.stdout:2/270: dread d0/d24/f39 [0,4194304] 0 2024-02-19T19:04:50.717 INFO:tasks.workunit.client.1.smithi132.stdout:5/241: link d8/da/f30 d8/d1a/d35/f4b 0 2024-02-19T19:04:50.720 INFO:tasks.workunit.client.1.smithi132.stdout:5/242: write d8/d23/d29/f2d [1531759,119592] 0 2024-02-19T19:04:50.720 INFO:tasks.workunit.client.1.smithi132.stdout:7/300: rename l12 to d1b/d1e/d2b/d62/l64 0 2024-02-19T19:04:50.723 INFO:tasks.workunit.client.1.smithi132.stdout:6/187: write d0/f5 [2004178,94095] 0 2024-02-19T19:04:50.723 INFO:tasks.workunit.client.1.smithi132.stdout:9/364: symlink d12/d2a/d4c/d61/l7e 0 2024-02-19T19:04:50.727 INFO:tasks.workunit.client.1.smithi132.stdout:8/269: unlink d4/dc/c40 0 2024-02-19T19:04:50.728 INFO:tasks.workunit.client.1.smithi132.stdout:5/243: dwrite d8/d23/f43 [0,4194304] 0 2024-02-19T19:04:50.730 INFO:tasks.workunit.client.1.smithi132.stdout:3/189: rename d8/d13/c30 to d8/d13/d2f/c3e 0 2024-02-19T19:04:50.731 INFO:tasks.workunit.client.1.smithi132.stdout:6/188: symlink d0/db/d25/d38/l49 0 2024-02-19T19:04:50.732 INFO:tasks.workunit.client.1.smithi132.stdout:6/189: stat d0/db/d25/l3b 0 2024-02-19T19:04:50.732 INFO:tasks.workunit.client.1.smithi132.stdout:9/365: symlink d12/d3e/d6e/l7f 0 2024-02-19T19:04:50.733 INFO:tasks.workunit.client.1.smithi132.stdout:9/366: write d12/d3e/f72 [552406,54196] 0 2024-02-19T19:04:50.738 INFO:tasks.workunit.client.1.smithi132.stdout:2/271: link d0/d4/d11/f41 d0/d4/d11/d2e/d5a/f5b 0 2024-02-19T19:04:50.738 INFO:tasks.workunit.client.1.smithi132.stdout:8/270: symlink d4/dc/d1b/d54/d5e/l5f 0 2024-02-19T19:04:50.740 INFO:tasks.workunit.client.1.smithi132.stdout:1/236: dread d1/d1e/d29/d2c/f2d [0,4194304] 0 2024-02-19T19:04:50.741 INFO:tasks.workunit.client.1.smithi132.stdout:7/301: creat d1b/d1e/d2b/d62/d53/f65 x:0 0 0 2024-02-19T19:04:50.742 INFO:tasks.workunit.client.1.smithi132.stdout:3/190: symlink d8/d31/l3f 0 2024-02-19T19:04:50.742 INFO:tasks.workunit.client.1.smithi132.stdout:6/190: dwrite d0/dc/f14 [0,4194304] 0 2024-02-19T19:04:50.746 INFO:tasks.workunit.client.1.smithi132.stdout:2/272: truncate d0/d24/d58/f49 747203 0 2024-02-19T19:04:50.747 INFO:tasks.workunit.client.1.smithi132.stdout:7/302: read d1b/d1e/f21 [2108528,86411] 0 2024-02-19T19:04:50.748 INFO:tasks.workunit.client.1.smithi132.stdout:8/271: symlink d4/dc/d1b/l60 0 2024-02-19T19:04:50.751 INFO:tasks.workunit.client.1.smithi132.stdout:5/244: dread d8/d23/d29/d27/f25 [0,4194304] 0 2024-02-19T19:04:50.753 INFO:tasks.workunit.client.1.smithi132.stdout:9/367: dwrite d12/d3e/d6e/f79 [4194304,4194304] 0 2024-02-19T19:04:50.760 INFO:tasks.workunit.client.1.smithi132.stdout:3/191: dwrite d8/d13/d2b/d2e/f3a [0,4194304] 0 2024-02-19T19:04:50.763 INFO:tasks.workunit.client.1.smithi132.stdout:7/303: truncate d1b/d1e/d2b/f2f 985345 0 2024-02-19T19:04:50.767 INFO:tasks.workunit.client.1.smithi132.stdout:5/245: fdatasync d8/d23/d29/d27/f25 0 2024-02-19T19:04:50.768 INFO:tasks.workunit.client.1.smithi132.stdout:2/273: dwrite d0/d19/f25 [0,4194304] 0 2024-02-19T19:04:50.769 INFO:tasks.workunit.client.1.smithi132.stdout:2/274: stat d0/d4/df/c17 0 2024-02-19T19:04:50.770 INFO:tasks.workunit.client.1.smithi132.stdout:2/275: stat d0/d4/df 0 2024-02-19T19:04:50.771 INFO:tasks.workunit.client.1.smithi132.stdout:2/276: readlink d0/d4/df/l16 0 2024-02-19T19:04:50.774 INFO:tasks.workunit.client.1.smithi132.stdout:0/250: sync 2024-02-19T19:04:50.778 INFO:tasks.workunit.client.1.smithi132.stdout:3/192: symlink d8/d13/d19/l40 0 2024-02-19T19:04:50.782 INFO:tasks.workunit.client.1.smithi132.stdout:5/246: creat d8/f4c x:0 0 0 2024-02-19T19:04:50.784 INFO:tasks.workunit.client.1.smithi132.stdout:5/247: write d8/d23/d29/f2d [1817289,129634] 0 2024-02-19T19:04:50.785 INFO:tasks.workunit.client.1.smithi132.stdout:1/237: write d1/d1e/d29/f2e [62192,51636] 0 2024-02-19T19:04:50.788 INFO:tasks.workunit.client.1.smithi132.stdout:0/251: rename d2/d21 to d2/dc/d4a 0 2024-02-19T19:04:50.789 INFO:tasks.workunit.client.1.smithi132.stdout:7/304: mknod d1b/d1e/d28/d5e/c66 0 2024-02-19T19:04:50.791 INFO:tasks.workunit.client.1.smithi132.stdout:6/191: dwrite d0/d3c/f39 [0,4194304] 0 2024-02-19T19:04:50.791 INFO:tasks.workunit.client.1.smithi132.stdout:0/252: readlink d2/dc/df/d15/l1e 0 2024-02-19T19:04:50.792 INFO:tasks.workunit.client.1.smithi132.stdout:9/368: write d12/d1b/f16 [1216002,127468] 0 2024-02-19T19:04:50.793 INFO:tasks.workunit.client.1.smithi132.stdout:0/253: chown d2/f6 14916 1 2024-02-19T19:04:50.798 INFO:tasks.workunit.client.1.smithi132.stdout:1/238: dwrite d1/f5 [0,4194304] 0 2024-02-19T19:04:50.798 INFO:tasks.workunit.client.1.smithi132.stdout:5/248: creat d8/d23/f4d x:0 0 0 2024-02-19T19:04:50.799 INFO:tasks.workunit.client.1.smithi132.stdout:2/277: write d0/d4/d11/f32 [4664962,111379] 0 2024-02-19T19:04:50.802 INFO:tasks.workunit.client.1.smithi132.stdout:5/249: chown d8/d2e/f39 135324800 1 2024-02-19T19:04:50.802 INFO:tasks.workunit.client.1.smithi132.stdout:4/235: sync 2024-02-19T19:04:50.807 INFO:tasks.workunit.client.1.smithi132.stdout:7/305: creat d1b/d1e/d2b/d62/f67 x:0 0 0 2024-02-19T19:04:50.808 INFO:tasks.workunit.client.1.smithi132.stdout:8/272: symlink d4/dc/d1b/d1d/l61 0 2024-02-19T19:04:50.809 INFO:tasks.workunit.client.1.smithi132.stdout:7/306: dread - d1b/f58 zero size 2024-02-19T19:04:50.810 INFO:tasks.workunit.client.1.smithi132.stdout:9/369: dwrite d12/d3e/f48 [0,4194304] 0 2024-02-19T19:04:50.814 INFO:tasks.workunit.client.1.smithi132.stdout:6/192: rename d0/d3c/d22/f8 to d0/db/d25/d2f/f4a 0 2024-02-19T19:04:50.814 INFO:tasks.workunit.client.1.smithi132.stdout:3/193: dwrite d8/f14 [4194304,4194304] 0 2024-02-19T19:04:50.815 INFO:tasks.workunit.client.1.smithi132.stdout:1/239: mknod d1/d3a/c47 0 2024-02-19T19:04:50.816 INFO:tasks.workunit.client.1.smithi132.stdout:7/307: write d1b/d1e/d28/f4c [1062717,68899] 0 2024-02-19T19:04:50.816 INFO:tasks.workunit.client.1.smithi132.stdout:2/278: creat d0/d19/d1e/f5c x:0 0 0 2024-02-19T19:04:50.819 INFO:tasks.workunit.client.1.smithi132.stdout:5/250: symlink d8/l4e 0 2024-02-19T19:04:50.819 INFO:tasks.workunit.client.1.smithi132.stdout:2/279: chown d0/d24 17 1 2024-02-19T19:04:50.820 INFO:tasks.workunit.client.1.smithi132.stdout:4/236: dwrite d4/f2d [0,4194304] 0 2024-02-19T19:04:50.820 INFO:tasks.workunit.client.1.smithi132.stdout:3/194: dread - d8/d13/d2b/f2c zero size 2024-02-19T19:04:50.822 INFO:tasks.workunit.client.1.smithi132.stdout:8/273: chown d4/dc/d1b/d38 445 1 2024-02-19T19:04:50.823 INFO:tasks.workunit.client.1.smithi132.stdout:2/280: fdatasync d0/d4/d11/f20 0 2024-02-19T19:04:50.829 INFO:tasks.workunit.client.1.smithi132.stdout:6/193: dwrite d0/db/fd [0,4194304] 0 2024-02-19T19:04:50.841 INFO:tasks.workunit.client.1.smithi132.stdout:2/281: symlink d0/d3c/d3f/l5d 0 2024-02-19T19:04:50.842 INFO:tasks.workunit.client.1.smithi132.stdout:7/308: dread d1b/d1e/d2b/f2f [0,4194304] 0 2024-02-19T19:04:50.844 INFO:tasks.workunit.client.1.smithi132.stdout:6/194: mkdir d0/db/d25/d3a/d4b 0 2024-02-19T19:04:50.845 INFO:tasks.workunit.client.1.smithi132.stdout:9/370: link d12/d2a/c55 d12/d3e/c80 0 2024-02-19T19:04:50.846 INFO:tasks.workunit.client.1.smithi132.stdout:0/254: rename d2/f12 to d2/dc/df/d15/f4b 0 2024-02-19T19:04:50.847 INFO:tasks.workunit.client.1.smithi132.stdout:5/251: dwrite f6 [0,4194304] 0 2024-02-19T19:04:50.848 INFO:tasks.workunit.client.1.smithi132.stdout:8/274: mkdir d4/dc/d1b/d20/d62 0 2024-02-19T19:04:50.849 INFO:tasks.workunit.client.1.smithi132.stdout:5/252: write d8/da/f3c [1894946,43829] 0 2024-02-19T19:04:50.854 INFO:tasks.workunit.client.1.smithi132.stdout:6/195: creat d0/db/d25/d2f/f4c x:0 0 0 2024-02-19T19:04:50.855 INFO:tasks.workunit.client.1.smithi132.stdout:8/275: dread d4/dc/f1c [0,4194304] 0 2024-02-19T19:04:50.855 INFO:tasks.workunit.client.1.smithi132.stdout:0/255: symlink d2/dc/df/l4c 0 2024-02-19T19:04:50.858 INFO:tasks.workunit.client.1.smithi132.stdout:5/253: rmdir d8/d23/d29 39 2024-02-19T19:04:50.859 INFO:tasks.workunit.client.1.smithi132.stdout:9/371: truncate d12/d25/d3a/d40/f54 44071 0 2024-02-19T19:04:50.860 INFO:tasks.workunit.client.1.smithi132.stdout:5/254: fdatasync d8/ff 0 2024-02-19T19:04:50.862 INFO:tasks.workunit.client.1.smithi132.stdout:9/372: read d12/d25/d3a/d40/f51 [182993,22768] 0 2024-02-19T19:04:50.862 INFO:tasks.workunit.client.1.smithi132.stdout:8/276: dwrite d4/fb [0,4194304] 0 2024-02-19T19:04:50.867 INFO:tasks.workunit.client.1.smithi132.stdout:3/195: dread d8/d13/d2b/d2e/f3b [0,4194304] 0 2024-02-19T19:04:50.868 INFO:tasks.workunit.client.1.smithi132.stdout:3/196: chown d8/d31/d34 214274 1 2024-02-19T19:04:50.870 INFO:tasks.workunit.client.1.smithi132.stdout:0/256: creat d2/dc/d4a/d25/f4d x:0 0 0 2024-02-19T19:04:50.872 INFO:tasks.workunit.client.1.smithi132.stdout:4/237: truncate d4/f7 6999110 0 2024-02-19T19:04:50.875 INFO:tasks.workunit.client.1.smithi132.stdout:1/240: rename d1/d1e/d29/d2c/f35 to d1/f48 0 2024-02-19T19:04:50.877 INFO:tasks.workunit.client.1.smithi132.stdout:5/255: creat d8/da/db/f4f x:0 0 0 2024-02-19T19:04:50.878 INFO:tasks.workunit.client.1.smithi132.stdout:2/282: dwrite d0/d4/d11/f1c [4194304,4194304] 0 2024-02-19T19:04:50.882 INFO:tasks.workunit.client.1.smithi132.stdout:5/256: dwrite d8/d23/f45 [0,4194304] 0 2024-02-19T19:04:50.884 INFO:tasks.workunit.client.1.smithi132.stdout:3/197: rename d8/f26 to d8/d13/d19/f41 0 2024-02-19T19:04:50.887 INFO:tasks.workunit.client.1.smithi132.stdout:8/277: creat d4/dc/d1b/d54/d5e/f63 x:0 0 0 2024-02-19T19:04:50.888 INFO:tasks.workunit.client.1.smithi132.stdout:2/283: rmdir d0/d24/d58 39 2024-02-19T19:04:50.890 INFO:tasks.workunit.client.1.smithi132.stdout:6/196: truncate d0/d3c/f42 183548 0 2024-02-19T19:04:50.891 INFO:tasks.workunit.client.1.smithi132.stdout:7/309: getdents d1b/d46 0 2024-02-19T19:04:50.892 INFO:tasks.workunit.client.1.smithi132.stdout:7/310: write d1b/d1e/d41/f47 [1477472,2477] 0 2024-02-19T19:04:50.893 INFO:tasks.workunit.client.1.smithi132.stdout:5/257: dwrite f6 [0,4194304] 0 2024-02-19T19:04:50.894 INFO:tasks.workunit.client.1.smithi132.stdout:7/311: write d1b/d1e/d28/f3c [826660,73208] 0 2024-02-19T19:04:50.895 INFO:tasks.workunit.client.1.smithi132.stdout:9/373: creat d12/d25/d3a/f81 x:0 0 0 2024-02-19T19:04:50.897 INFO:tasks.workunit.client.1.smithi132.stdout:1/241: rename d1/d3a/c20 to d1/d1e/d29/d2c/c49 0 2024-02-19T19:04:50.899 INFO:tasks.workunit.client.1.smithi132.stdout:5/258: dread f6 [0,4194304] 0 2024-02-19T19:04:50.899 INFO:tasks.workunit.client.1.smithi132.stdout:5/259: stat d8/d1a 0 2024-02-19T19:04:50.903 INFO:tasks.workunit.client.1.smithi132.stdout:6/197: mkdir d0/d3c/d22/d4d 0 2024-02-19T19:04:50.905 INFO:tasks.workunit.client.1.smithi132.stdout:6/198: stat d0/db/d25/d3a/d3d 0 2024-02-19T19:04:50.906 INFO:tasks.workunit.client.1.smithi132.stdout:5/260: dwrite d8/d1a/f26 [0,4194304] 0 2024-02-19T19:04:50.907 INFO:tasks.workunit.client.1.smithi132.stdout:0/257: creat d2/dc/df/d15/f4e x:0 0 0 2024-02-19T19:04:50.920 INFO:tasks.workunit.client.1.smithi132.stdout:7/312: read d1b/f25 [208777,107743] 0 2024-02-19T19:04:50.921 INFO:tasks.workunit.client.1.smithi132.stdout:9/374: truncate d12/d2a/f5b 337954 0 2024-02-19T19:04:50.923 INFO:tasks.workunit.client.1.smithi132.stdout:3/198: mkdir d8/d31/d34/d42 0 2024-02-19T19:04:50.924 INFO:tasks.workunit.client.1.smithi132.stdout:8/278: symlink d4/dc/d1b/d33/l64 0 2024-02-19T19:04:50.925 INFO:tasks.workunit.client.1.smithi132.stdout:4/238: dread d4/d12/f2f [0,4194304] 0 2024-02-19T19:04:50.927 INFO:tasks.workunit.client.1.smithi132.stdout:2/284: dwrite d0/d24/d58/f36 [4194304,4194304] 0 2024-02-19T19:04:50.928 INFO:tasks.workunit.client.1.smithi132.stdout:7/313: read d1b/d1e/f3d [1285834,8919] 0 2024-02-19T19:04:50.929 INFO:tasks.workunit.client.1.smithi132.stdout:8/279: read - d4/dc/d1b/d33/f5b zero size 2024-02-19T19:04:50.931 INFO:tasks.workunit.client.1.smithi132.stdout:5/261: rmdir d8/d2e 39 2024-02-19T19:04:50.931 INFO:tasks.workunit.client.1.smithi132.stdout:8/280: dread - d4/d2d/f46 zero size 2024-02-19T19:04:50.933 INFO:tasks.workunit.client.1.smithi132.stdout:3/199: dread d8/f28 [0,4194304] 0 2024-02-19T19:04:50.935 INFO:tasks.workunit.client.1.smithi132.stdout:0/258: dwrite d2/dc/d4a/f42 [4194304,4194304] 0 2024-02-19T19:04:50.938 INFO:tasks.workunit.client.1.smithi132.stdout:3/200: write d8/f28 [889932,37081] 0 2024-02-19T19:04:50.941 INFO:tasks.workunit.client.1.smithi132.stdout:3/201: write d8/fd [1804855,96821] 0 2024-02-19T19:04:50.943 INFO:tasks.workunit.client.1.smithi132.stdout:7/314: dwrite d1b/d1e/f37 [0,4194304] 0 2024-02-19T19:04:50.948 INFO:tasks.workunit.client.1.smithi132.stdout:4/239: creat d4/d27/d31/d32/f50 x:0 0 0 2024-02-19T19:04:50.951 INFO:tasks.workunit.client.1.smithi132.stdout:2/285: rmdir d0/d3c/d3f 39 2024-02-19T19:04:50.951 INFO:tasks.workunit.client.1.smithi132.stdout:8/281: mknod d4/c65 0 2024-02-19T19:04:50.951 INFO:tasks.workunit.client.1.smithi132.stdout:6/199: truncate d0/db/d25/f40 2025399 0 2024-02-19T19:04:50.951 INFO:tasks.workunit.client.1.smithi132.stdout:2/286: readlink d0/l57 0 2024-02-19T19:04:50.953 INFO:tasks.workunit.client.1.smithi132.stdout:8/282: fdatasync d4/dc/d1b/d54/d5e/f63 0 2024-02-19T19:04:50.959 INFO:tasks.workunit.client.1.smithi132.stdout:9/375: dwrite d12/d2a/f2c [0,4194304] 0 2024-02-19T19:04:50.961 INFO:tasks.workunit.client.1.smithi132.stdout:5/262: dread d8/ff [0,4194304] 0 2024-02-19T19:04:50.962 INFO:tasks.workunit.client.1.smithi132.stdout:7/315: rename d1b/d32/f42 to d1b/d5a/f68 0 2024-02-19T19:04:50.964 INFO:tasks.workunit.client.1.smithi132.stdout:5/263: write d8/da/db/f1b [2004069,60433] 0 2024-02-19T19:04:50.965 INFO:tasks.workunit.client.1.smithi132.stdout:4/240: mkdir d4/d27/d31/d32/d51 0 2024-02-19T19:04:50.965 INFO:tasks.workunit.client.1.smithi132.stdout:0/259: mknod d2/d3e/c4f 0 2024-02-19T19:04:50.967 INFO:tasks.workunit.client.1.smithi132.stdout:2/287: fdatasync d0/d4/df/f4d 0 2024-02-19T19:04:50.968 INFO:tasks.workunit.client.1.smithi132.stdout:9/376: dwrite d12/d25/d3a/f52 [0,4194304] 0 2024-02-19T19:04:50.975 INFO:tasks.workunit.client.1.smithi132.stdout:6/200: creat d0/d3c/d22/d4d/f4e x:0 0 0 2024-02-19T19:04:50.976 INFO:tasks.workunit.client.1.smithi132.stdout:6/201: stat d0/db/d25/d38/l49 0 2024-02-19T19:04:50.978 INFO:tasks.workunit.client.1.smithi132.stdout:6/202: truncate d0/d3c/d22/f19 90896 0 2024-02-19T19:04:50.981 INFO:tasks.workunit.client.1.smithi132.stdout:5/264: dwrite d8/d23/d29/d27/f25 [0,4194304] 0 2024-02-19T19:04:50.981 INFO:tasks.workunit.client.1.smithi132.stdout:0/260: mknod d2/dc/d4a/d25/c50 0 2024-02-19T19:04:50.981 INFO:tasks.workunit.client.1.smithi132.stdout:3/202: write d8/d13/f20 [3201595,97867] 0 2024-02-19T19:04:50.985 INFO:tasks.workunit.client.1.smithi132.stdout:3/203: write d8/d13/d2f/f36 [574297,84581] 0 2024-02-19T19:04:50.985 INFO:tasks.workunit.client.1.smithi132.stdout:6/203: dread d0/f5 [0,4194304] 0 2024-02-19T19:04:50.987 INFO:tasks.workunit.client.1.smithi132.stdout:2/288: rmdir d0/d4/d11/d2e 39 2024-02-19T19:04:50.989 INFO:tasks.workunit.client.1.smithi132.stdout:2/289: chown d0/d4/df/c17 31195135 1 2024-02-19T19:04:50.991 INFO:tasks.workunit.client.1.smithi132.stdout:4/241: truncate d4/d12/d16/f1b 1105447 0 2024-02-19T19:04:50.991 INFO:tasks.workunit.client.1.smithi132.stdout:2/290: readlink d0/d24/d58/l56 0 2024-02-19T19:04:50.991 INFO:tasks.workunit.client.1.smithi132.stdout:0/261: mknod d2/d3e/c51 0 2024-02-19T19:04:50.991 INFO:tasks.workunit.client.1.smithi132.stdout:3/204: symlink d8/d13/d19/l43 0 2024-02-19T19:04:50.992 INFO:tasks.workunit.client.1.smithi132.stdout:6/204: creat d0/db/d25/f4f x:0 0 0 2024-02-19T19:04:50.993 INFO:tasks.workunit.client.1.smithi132.stdout:9/377: mkdir d12/d25/d3a/d78/d82 0 2024-02-19T19:04:50.995 INFO:tasks.workunit.client.1.smithi132.stdout:5/265: dwrite d8/da/f24 [0,4194304] 0 2024-02-19T19:04:50.995 INFO:tasks.workunit.client.1.smithi132.stdout:9/378: fsync d12/d3e/f48 0 2024-02-19T19:04:50.996 INFO:tasks.workunit.client.1.smithi132.stdout:5/266: write d8/da/f3c [5086092,18889] 0 2024-02-19T19:04:50.997 INFO:tasks.workunit.client.1.smithi132.stdout:9/379: dread - d12/d25/d3a/d40/f44 zero size 2024-02-19T19:04:50.999 INFO:tasks.workunit.client.1.smithi132.stdout:4/242: creat d4/d12/d28/f52 x:0 0 0 2024-02-19T19:04:50.999 INFO:tasks.workunit.client.1.smithi132.stdout:0/262: mkdir d2/dc/df/d27/d3a/d52 0 2024-02-19T19:04:51.002 INFO:tasks.workunit.client.1.smithi132.stdout:1/242: sync 2024-02-19T19:04:51.002 INFO:tasks.workunit.client.1.smithi132.stdout:8/283: dread d4/dc/d1b/f43 [0,4194304] 0 2024-02-19T19:04:51.004 INFO:tasks.workunit.client.1.smithi132.stdout:6/205: dwrite d0/d3c/d22/f19 [0,4194304] 0 2024-02-19T19:04:51.005 INFO:tasks.workunit.client.1.smithi132.stdout:9/380: mknod d12/d25/d3a/d40/d4e/c83 0 2024-02-19T19:04:51.008 INFO:tasks.workunit.client.1.smithi132.stdout:0/263: dread d2/f6 [4194304,4194304] 0 2024-02-19T19:04:51.013 INFO:tasks.workunit.client.1.smithi132.stdout:3/205: mkdir d8/d38/d44 0 2024-02-19T19:04:51.014 INFO:tasks.workunit.client.1.smithi132.stdout:3/206: fdatasync d8/d13/f37 0 2024-02-19T19:04:51.017 INFO:tasks.workunit.client.1.smithi132.stdout:0/264: dwrite d2/dc/df/d15/f22 [0,4194304] 0 2024-02-19T19:04:51.018 INFO:tasks.workunit.client.1.smithi132.stdout:6/206: rmdir d0/db/d16 39 2024-02-19T19:04:51.022 INFO:tasks.workunit.client.1.smithi132.stdout:9/381: creat d12/d3e/f84 x:0 0 0 2024-02-19T19:04:51.022 INFO:tasks.workunit.client.1.smithi132.stdout:9/382: fsync d12/d1b/f1c 0 2024-02-19T19:04:51.028 INFO:tasks.workunit.client.1.smithi132.stdout:7/316: dwrite d1b/d1e/d28/f29 [0,4194304] 0 2024-02-19T19:04:51.031 INFO:tasks.workunit.client.1.smithi132.stdout:2/291: link d0/d19/c47 d0/d24/d58/d53/c5e 0 2024-02-19T19:04:51.036 INFO:tasks.workunit.client.1.smithi132.stdout:6/207: rmdir d0 39 2024-02-19T19:04:51.038 INFO:tasks.workunit.client.1.smithi132.stdout:4/243: rmdir d4 39 2024-02-19T19:04:51.040 INFO:tasks.workunit.client.1.smithi132.stdout:7/317: dwrite d1b/d1e/f30 [4194304,4194304] 0 2024-02-19T19:04:51.042 INFO:tasks.workunit.client.1.smithi132.stdout:2/292: dwrite d0/d4/df/f3b [0,4194304] 0 2024-02-19T19:04:51.042 INFO:tasks.workunit.client.1.smithi132.stdout:8/284: getdents d4/dc/d1b/d38/d45 0 2024-02-19T19:04:51.051 INFO:tasks.workunit.client.1.smithi132.stdout:3/207: write d8/d13/f16 [27851,98582] 0 2024-02-19T19:04:51.060 INFO:tasks.workunit.client.1.smithi132.stdout:5/267: rename d8/d2e/f39 to d8/d23/f50 0 2024-02-19T19:04:51.062 INFO:tasks.workunit.client.1.smithi132.stdout:6/208: write d0/d3c/f48 [1025342,128855] 0 2024-02-19T19:04:51.065 INFO:tasks.workunit.client.1.smithi132.stdout:7/318: symlink d1b/d1e/d2b/d62/l69 0 2024-02-19T19:04:51.066 INFO:tasks.workunit.client.1.smithi132.stdout:9/383: mkdir d12/d25/d3a/d85 0 2024-02-19T19:04:51.066 INFO:tasks.workunit.client.1.smithi132.stdout:9/384: chown d12/d1b/l1e 38 1 2024-02-19T19:04:51.068 INFO:tasks.workunit.client.1.smithi132.stdout:2/293: write d0/d4/d11/d2e/d5a/f5b [8801675,55437] 0 2024-02-19T19:04:51.068 INFO:tasks.workunit.client.1.smithi132.stdout:9/385: chown d12/d25/d3a/d40/d4e/f67 326026 1 2024-02-19T19:04:51.069 INFO:tasks.workunit.client.1.smithi132.stdout:8/285: mknod d4/dc/d1b/d54/d5e/c66 0 2024-02-19T19:04:51.069 INFO:tasks.workunit.client.1.smithi132.stdout:9/386: chown d12/d25/d3a/d40/f5d 23 1 2024-02-19T19:04:51.070 INFO:tasks.workunit.client.1.smithi132.stdout:3/208: mkdir d8/d13/d2b/d2e/d45 0 2024-02-19T19:04:51.075 INFO:tasks.workunit.client.1.smithi132.stdout:5/268: mkdir d8/d1a/d1e/d51 0 2024-02-19T19:04:51.076 INFO:tasks.workunit.client.1.smithi132.stdout:6/209: mkdir d0/db/d25/d50 0 2024-02-19T19:04:51.076 INFO:tasks.workunit.client.1.smithi132.stdout:8/286: dwrite d4/dc/d1b/d54/d5e/f50 [4194304,4194304] 0 2024-02-19T19:04:51.099 INFO:tasks.workunit.client.1.smithi132.stdout:4/244: symlink d4/d27/d31/l53 0 2024-02-19T19:04:51.100 INFO:tasks.workunit.client.1.smithi132.stdout:3/209: symlink d8/d38/l46 0 2024-02-19T19:04:51.100 INFO:tasks.workunit.client.1.smithi132.stdout:5/269: truncate d8/d23/d29/f31 933617 0 2024-02-19T19:04:51.103 INFO:tasks.workunit.client.1.smithi132.stdout:3/210: truncate d8/d13/f37 684067 0 2024-02-19T19:04:51.106 INFO:tasks.workunit.client.1.smithi132.stdout:7/319: dwrite d1b/d1e/d28/f31 [0,4194304] 0 2024-02-19T19:04:51.109 INFO:tasks.workunit.client.1.smithi132.stdout:2/294: dread d0/f5 [0,4194304] 0 2024-02-19T19:04:51.113 INFO:tasks.workunit.client.1.smithi132.stdout:0/265: rename d2/dc/d4a/d25/f4d to d2/f53 0 2024-02-19T19:04:51.113 INFO:tasks.workunit.client.1.smithi132.stdout:4/245: mknod d4/d12/d16/c54 0 2024-02-19T19:04:51.113 INFO:tasks.workunit.client.1.smithi132.stdout:5/270: symlink d8/da/l52 0 2024-02-19T19:04:51.114 INFO:tasks.workunit.client.1.smithi132.stdout:5/271: chown d8/d23/d29/l2f 1 1 2024-02-19T19:04:51.116 INFO:tasks.workunit.client.1.smithi132.stdout:3/211: creat d8/d13/d2f/f47 x:0 0 0 2024-02-19T19:04:51.116 INFO:tasks.workunit.client.1.smithi132.stdout:3/212: fdatasync d8/d13/d2b/f2c 0 2024-02-19T19:04:51.120 INFO:tasks.workunit.client.1.smithi132.stdout:7/320: creat d1b/d1e/d41/f6a x:0 0 0 2024-02-19T19:04:51.123 INFO:tasks.workunit.client.1.smithi132.stdout:3/213: dread d8/d13/d2b/d2e/f3a [0,4194304] 0 2024-02-19T19:04:51.126 INFO:tasks.workunit.client.1.smithi132.stdout:8/287: link d4/dc/f2c d4/dc/d1b/d54/d5e/f67 0 2024-02-19T19:04:51.127 INFO:tasks.workunit.client.1.smithi132.stdout:6/210: dwrite d0/f5 [0,4194304] 0 2024-02-19T19:04:51.127 INFO:tasks.workunit.client.1.smithi132.stdout:4/246: rmdir d4/d12 39 2024-02-19T19:04:51.127 INFO:tasks.workunit.client.1.smithi132.stdout:6/211: readlink d0/db/l30 0 2024-02-19T19:04:51.128 INFO:tasks.workunit.client.1.smithi132.stdout:7/321: dwrite d1b/d1e/d28/f5b [0,4194304] 0 2024-02-19T19:04:51.129 INFO:tasks.workunit.client.1.smithi132.stdout:6/212: write d0/f5 [3543597,87120] 0 2024-02-19T19:04:51.131 INFO:tasks.workunit.client.1.smithi132.stdout:0/266: mknod d2/dc/c54 0 2024-02-19T19:04:51.144 INFO:tasks.workunit.client.1.smithi132.stdout:1/243: sync 2024-02-19T19:04:51.151 INFO:tasks.workunit.client.1.smithi132.stdout:3/214: write d8/f22 [1853324,58443] 0 2024-02-19T19:04:51.151 INFO:tasks.workunit.client.1.smithi132.stdout:3/215: read - d8/d13/d2f/f47 zero size 2024-02-19T19:04:51.155 INFO:tasks.workunit.client.1.smithi132.stdout:7/322: fdatasync d1b/f25 0 2024-02-19T19:04:51.155 INFO:tasks.workunit.client.1.smithi132.stdout:0/267: symlink d2/dc/l55 0 2024-02-19T19:04:51.156 INFO:tasks.workunit.client.1.smithi132.stdout:6/213: chown d0/db/d16/f29 3749844 1 2024-02-19T19:04:51.157 INFO:tasks.workunit.client.1.smithi132.stdout:6/214: readlink d0/db/d25/l3b 0 2024-02-19T19:04:51.163 INFO:tasks.workunit.client.1.smithi132.stdout:3/216: unlink c7 0 2024-02-19T19:04:51.165 INFO:tasks.workunit.client.1.smithi132.stdout:5/272: rmdir d8 39 2024-02-19T19:04:51.206 INFO:tasks.workunit.client.1.smithi132.stdout:0/268: creat d2/dc/df/d27/f56 x:0 0 0 2024-02-19T19:04:51.207 INFO:tasks.workunit.client.1.smithi132.stdout:3/217: mknod d8/d13/d19/c48 0 2024-02-19T19:04:51.207 INFO:tasks.workunit.client.1.smithi132.stdout:0/269: fsync d2/f1a 0 2024-02-19T19:04:51.208 INFO:tasks.workunit.client.1.smithi132.stdout:1/244: creat d1/f4a x:0 0 0 2024-02-19T19:04:51.209 INFO:tasks.workunit.client.1.smithi132.stdout:7/323: mkdir d1b/d5a/d63/d6b 0 2024-02-19T19:04:51.209 INFO:tasks.workunit.client.1.smithi132.stdout:6/215: mkdir d0/d51 0 2024-02-19T19:04:51.210 INFO:tasks.workunit.client.1.smithi132.stdout:1/245: chown d1/d3a/c47 3 1 2024-02-19T19:04:51.210 INFO:tasks.workunit.client.1.smithi132.stdout:3/218: dread d8/d13/f37 [0,4194304] 0 2024-02-19T19:04:51.211 INFO:tasks.workunit.client.1.smithi132.stdout:3/219: stat d8/d13/d2b/l32 0 2024-02-19T19:04:51.212 INFO:tasks.workunit.client.1.smithi132.stdout:4/247: rename d4/d12/d16/c46 to d4/c55 0 2024-02-19T19:04:51.212 INFO:tasks.workunit.client.1.smithi132.stdout:7/324: rename d1b/d1e to d1b/d1e/d28/d6c 22 2024-02-19T19:04:51.212 INFO:tasks.workunit.client.1.smithi132.stdout:3/220: write d8/d13/f16 [196982,50346] 0 2024-02-19T19:04:51.213 INFO:tasks.workunit.client.1.smithi132.stdout:0/270: fdatasync d2/dc/fe 0 2024-02-19T19:04:51.218 INFO:tasks.workunit.client.1.smithi132.stdout:4/248: mkdir d4/d19/d21/d56 0 2024-02-19T19:04:51.220 INFO:tasks.workunit.client.1.smithi132.stdout:3/221: creat d8/d13/d19/f49 x:0 0 0 2024-02-19T19:04:51.223 INFO:tasks.workunit.client.1.smithi132.stdout:1/246: mknod d1/d1e/c4b 0 2024-02-19T19:04:51.225 INFO:tasks.workunit.client.1.smithi132.stdout:7/325: truncate d1b/d1e/f21 2766812 0 2024-02-19T19:04:51.226 INFO:tasks.workunit.client.1.smithi132.stdout:7/326: write d1b/d1e/d41/f47 [2841605,48904] 0 2024-02-19T19:04:51.231 INFO:tasks.workunit.client.1.smithi132.stdout:1/247: dread d1/d1e/d29/d2c/f2d [4194304,4194304] 0 2024-02-19T19:04:51.231 INFO:tasks.workunit.client.1.smithi132.stdout:6/216: link d0/l7 d0/db/d25/l52 0 2024-02-19T19:04:51.231 INFO:tasks.workunit.client.1.smithi132.stdout:1/248: chown d1/d1e/d29 495 1 2024-02-19T19:04:51.232 INFO:tasks.workunit.client.1.smithi132.stdout:6/217: fdatasync d0/db/d25/d2f/f31 0 2024-02-19T19:04:51.237 INFO:tasks.workunit.client.1.smithi132.stdout:0/271: creat d2/dc/d4a/f57 x:0 0 0 2024-02-19T19:04:51.242 INFO:tasks.workunit.client.1.smithi132.stdout:5/273: truncate d8/d23/f45 3599929 0 2024-02-19T19:04:51.243 INFO:tasks.workunit.client.1.smithi132.stdout:0/272: dwrite d2/f2b [0,4194304] 0 2024-02-19T19:04:51.244 INFO:tasks.workunit.client.1.smithi132.stdout:9/387: sync 2024-02-19T19:04:51.244 INFO:tasks.workunit.client.1.smithi132.stdout:0/273: write d2/dc/df/d15/f4e [549775,120688] 0 2024-02-19T19:04:51.246 INFO:tasks.workunit.client.1.smithi132.stdout:7/327: dread d1b/d1e/d28/f4f [0,4194304] 0 2024-02-19T19:04:51.257 INFO:tasks.workunit.client.1.smithi132.stdout:6/218: mknod d0/db/d25/d3a/d3d/c53 0 2024-02-19T19:04:51.258 INFO:tasks.workunit.client.1.smithi132.stdout:9/388: readlink d12/d25/d3a/d40/d4e/d77/l7b 0 2024-02-19T19:04:51.259 INFO:tasks.workunit.client.1.smithi132.stdout:0/274: fsync d2/dc/f3b 0 2024-02-19T19:04:51.262 INFO:tasks.workunit.client.1.smithi132.stdout:7/328: creat d1b/d1e/d41/f6d x:0 0 0 2024-02-19T19:04:51.263 INFO:tasks.workunit.client.1.smithi132.stdout:4/249: dwrite d4/f10 [0,4194304] 0 2024-02-19T19:04:51.264 INFO:tasks.workunit.client.1.smithi132.stdout:4/250: dread - d4/dc/d37/f3a zero size 2024-02-19T19:04:51.268 INFO:tasks.workunit.client.1.smithi132.stdout:9/389: mkdir d12/d3e/d86 0 2024-02-19T19:04:51.270 INFO:tasks.workunit.client.1.smithi132.stdout:9/390: read - d12/d3e/f6c zero size 2024-02-19T19:04:51.270 INFO:tasks.workunit.client.1.smithi132.stdout:1/249: link d1/d3a/l24 d1/l4c 0 2024-02-19T19:04:51.270 INFO:tasks.workunit.client.1.smithi132.stdout:7/329: dwrite d1b/d1e/f43 [0,4194304] 0 2024-02-19T19:04:51.273 INFO:tasks.workunit.client.1.smithi132.stdout:5/274: rename d8/da/c2c to d8/d1a/c53 0 2024-02-19T19:04:51.276 INFO:tasks.workunit.client.1.smithi132.stdout:0/275: mkdir d2/dc/d4a/d58 0 2024-02-19T19:04:51.277 INFO:tasks.workunit.client.1.smithi132.stdout:0/276: chown d2/dc/l49 344101757 1 2024-02-19T19:04:51.278 INFO:tasks.workunit.client.1.smithi132.stdout:6/219: link d0/db/d16/f1d d0/db/d25/d3a/d4b/f54 0 2024-02-19T19:04:51.278 INFO:tasks.workunit.client.1.smithi132.stdout:1/250: dwrite d1/f4a [0,4194304] 0 2024-02-19T19:04:51.280 INFO:tasks.workunit.client.1.smithi132.stdout:0/277: fsync d2/dc/fe 0 2024-02-19T19:04:51.280 INFO:tasks.workunit.client.1.smithi132.stdout:7/330: symlink d1b/d46/d56/l6e 0 2024-02-19T19:04:51.281 INFO:tasks.workunit.client.1.smithi132.stdout:5/275: dread d8/d23/d29/d27/f25 [0,4194304] 0 2024-02-19T19:04:51.284 INFO:tasks.workunit.client.1.smithi132.stdout:9/391: unlink d12/d1b/f5a 0 2024-02-19T19:04:51.285 INFO:tasks.workunit.client.1.smithi132.stdout:1/251: fsync d1/f48 0 2024-02-19T19:04:51.287 INFO:tasks.workunit.client.1.smithi132.stdout:6/220: creat d0/d3c/d22/d4d/f55 x:0 0 0 2024-02-19T19:04:51.288 INFO:tasks.workunit.client.1.smithi132.stdout:7/331: creat d1b/d1e/d2b/f6f x:0 0 0 2024-02-19T19:04:51.289 INFO:tasks.workunit.client.1.smithi132.stdout:7/332: fdatasync d1b/d1e/d28/f4c 0 2024-02-19T19:04:51.293 INFO:tasks.workunit.client.1.smithi132.stdout:0/278: symlink d2/l59 0 2024-02-19T19:04:51.301 INFO:tasks.workunit.client.1.smithi132.stdout:6/221: symlink d0/db/d25/l56 0 2024-02-19T19:04:51.304 INFO:tasks.workunit.client.1.smithi132.stdout:5/276: creat d8/da/db/f54 x:0 0 0 2024-02-19T19:04:51.305 INFO:tasks.workunit.client.1.smithi132.stdout:6/222: dread d0/db/d16/f26 [0,4194304] 0 2024-02-19T19:04:51.305 INFO:tasks.workunit.client.1.smithi132.stdout:2/295: sync 2024-02-19T19:04:51.306 INFO:tasks.workunit.client.1.smithi132.stdout:6/223: chown d0/db/d25/d38/l3e 1877147 1 2024-02-19T19:04:51.306 INFO:tasks.workunit.client.1.smithi132.stdout:7/333: creat d1b/d1e/d2b/d62/d53/f70 x:0 0 0 2024-02-19T19:04:51.308 INFO:tasks.workunit.client.1.smithi132.stdout:1/252: mkdir d1/d4d 0 2024-02-19T19:04:51.309 INFO:tasks.workunit.client.1.smithi132.stdout:7/334: write d1b/d1e/d28/f3c [3895254,22163] 0 2024-02-19T19:04:51.310 INFO:tasks.workunit.client.1.smithi132.stdout:4/251: truncate d4/dc/d14/d24/f39 603109 0 2024-02-19T19:04:51.312 INFO:tasks.workunit.client.1.smithi132.stdout:9/392: mkdir d12/d25/d3a/d85/d87 0 2024-02-19T19:04:51.313 INFO:tasks.workunit.client.1.smithi132.stdout:5/277: dwrite d8/d23/d29/f31 [0,4194304] 0 2024-02-19T19:04:51.314 INFO:tasks.workunit.client.1.smithi132.stdout:9/393: read d12/d25/d3a/d40/f51 [220330,64047] 0 2024-02-19T19:04:51.315 INFO:tasks.workunit.client.1.smithi132.stdout:5/278: chown d8/d23/d29 162743 1 2024-02-19T19:04:51.316 INFO:tasks.workunit.client.1.smithi132.stdout:0/279: mknod d2/dc/df/d27/d3a/d52/c5a 0 2024-02-19T19:04:51.319 INFO:tasks.workunit.client.1.smithi132.stdout:0/280: write d2/dc/df/d15/f22 [391811,58262] 0 2024-02-19T19:04:51.320 INFO:tasks.workunit.client.1.smithi132.stdout:1/253: unlink d1/d1e/d29/f2e 0 2024-02-19T19:04:51.321 INFO:tasks.workunit.client.1.smithi132.stdout:7/335: dwrite d1b/d1e/d28/f4c [0,4194304] 0 2024-02-19T19:04:51.322 INFO:tasks.workunit.client.1.smithi132.stdout:0/281: write d2/dc/df/d15/d34/f41 [1107210,108528] 0 2024-02-19T19:04:51.323 INFO:tasks.workunit.client.1.smithi132.stdout:7/336: stat d1b/d1e/d41/f6d 0 2024-02-19T19:04:51.326 INFO:tasks.workunit.client.1.smithi132.stdout:2/296: link d0/d19/d1e/f5c d0/d4/df/f5f 0 2024-02-19T19:04:51.331 INFO:tasks.workunit.client.1.smithi132.stdout:1/254: dwrite d1/d1e/f2a [0,4194304] 0 2024-02-19T19:04:51.333 INFO:tasks.workunit.client.1.smithi132.stdout:0/282: dwrite d2/dc/fe [0,4194304] 0 2024-02-19T19:04:51.335 INFO:tasks.workunit.client.1.smithi132.stdout:8/288: sync 2024-02-19T19:04:51.335 INFO:tasks.workunit.client.1.smithi132.stdout:3/222: sync 2024-02-19T19:04:51.341 INFO:tasks.workunit.client.1.smithi132.stdout:1/255: dwrite d1/f15 [8388608,4194304] 0 2024-02-19T19:04:51.348 INFO:tasks.workunit.client.1.smithi132.stdout:8/289: dwrite d4/dc/f44 [0,4194304] 0 2024-02-19T19:04:51.349 INFO:tasks.workunit.client.1.smithi132.stdout:3/223: dwrite d8/f14 [4194304,4194304] 0 2024-02-19T19:04:51.353 INFO:tasks.workunit.client.1.smithi132.stdout:3/224: write d8/d13/f16 [1655667,114419] 0 2024-02-19T19:04:51.356 INFO:tasks.workunit.client.1.smithi132.stdout:1/256: dwrite d1/f48 [0,4194304] 0 2024-02-19T19:04:51.356 INFO:tasks.workunit.client.1.smithi132.stdout:3/225: chown d8/l17 118 1 2024-02-19T19:04:51.358 INFO:tasks.workunit.client.1.smithi132.stdout:1/257: chown d1/d1e/d29/d2c/c45 17841 1 2024-02-19T19:04:51.358 INFO:tasks.workunit.client.1.smithi132.stdout:1/258: readlink d1/d1e/d29/d2c/l46 0 2024-02-19T19:04:51.363 INFO:tasks.workunit.client.1.smithi132.stdout:4/252: symlink d4/d19/d21/d56/l57 0 2024-02-19T19:04:51.367 INFO:tasks.workunit.client.1.smithi132.stdout:0/283: rename d2/c14 to d2/dc/df/d15/d2f/d47/c5b 0 2024-02-19T19:04:51.368 INFO:tasks.workunit.client.1.smithi132.stdout:6/224: link d0/db/d25/d3a/d3d/c53 d0/d3c/d46/c57 0 2024-02-19T19:04:51.371 INFO:tasks.workunit.client.1.smithi132.stdout:3/226: dwrite f0 [0,4194304] 0 2024-02-19T19:04:51.373 INFO:tasks.workunit.client.1.smithi132.stdout:8/290: fdatasync d4/dc/f2c 0 2024-02-19T19:04:51.374 INFO:tasks.workunit.client.1.smithi132.stdout:3/227: chown d8/d13/d19/c48 1052163099 1 2024-02-19T19:04:51.375 INFO:tasks.workunit.client.1.smithi132.stdout:3/228: write d8/f22 [617746,125349] 0 2024-02-19T19:04:51.379 INFO:tasks.workunit.client.1.smithi132.stdout:1/259: mkdir d1/d1e/d29/d4e 0 2024-02-19T19:04:51.384 INFO:tasks.workunit.client.1.smithi132.stdout:4/253: symlink d4/d19/d21/l58 0 2024-02-19T19:04:51.388 INFO:tasks.workunit.client.1.smithi132.stdout:3/229: write d8/d13/d2b/d2e/f3b [2613772,103843] 0 2024-02-19T19:04:51.388 INFO:tasks.workunit.client.1.smithi132.stdout:7/337: link l16 d1b/d1e/d41/l71 0 2024-02-19T19:04:51.394 INFO:tasks.workunit.client.1.smithi132.stdout:0/284: unlink d2/l2c 0 2024-02-19T19:04:51.394 INFO:tasks.workunit.client.1.smithi132.stdout:2/297: creat d0/d3c/f60 x:0 0 0 2024-02-19T19:04:51.397 INFO:tasks.workunit.client.1.smithi132.stdout:2/298: chown d0/d24/d58/d53 100911 1 2024-02-19T19:04:51.397 INFO:tasks.workunit.client.1.smithi132.stdout:3/230: dwrite d8/d13/d2f/f47 [0,4194304] 0 2024-02-19T19:04:51.397 INFO:tasks.workunit.client.1.smithi132.stdout:6/225: creat d0/db/f58 x:0 0 0 2024-02-19T19:04:51.398 INFO:tasks.workunit.client.1.smithi132.stdout:7/338: dwrite d1b/d1e/f22 [0,4194304] 0 2024-02-19T19:04:51.400 INFO:tasks.workunit.client.1.smithi132.stdout:3/231: write d8/f22 [98056,34589] 0 2024-02-19T19:04:51.403 INFO:tasks.workunit.client.1.smithi132.stdout:4/254: link d4/f38 d4/f59 0 2024-02-19T19:04:51.405 INFO:tasks.workunit.client.1.smithi132.stdout:0/285: creat d2/f5c x:0 0 0 2024-02-19T19:04:51.407 INFO:tasks.workunit.client.1.smithi132.stdout:6/226: creat d0/d3c/d22/f59 x:0 0 0 2024-02-19T19:04:51.410 INFO:tasks.workunit.client.1.smithi132.stdout:3/232: dwrite d8/d13/f20 [4194304,4194304] 0 2024-02-19T19:04:51.420 INFO:tasks.workunit.client.1.smithi132.stdout:4/255: creat d4/d12/d16/f5a x:0 0 0 2024-02-19T19:04:51.429 INFO:tasks.workunit.client.1.smithi132.stdout:5/279: write d8/d23/f45 [1009331,10897] 0 2024-02-19T19:04:51.430 INFO:tasks.workunit.client.1.smithi132.stdout:0/286: mknod d2/dc/df/d27/d3a/c5d 0 2024-02-19T19:04:51.432 INFO:tasks.workunit.client.1.smithi132.stdout:0/287: write d2/f2b [1100463,42346] 0 2024-02-19T19:04:51.432 INFO:tasks.workunit.client.1.smithi132.stdout:9/394: dwrite d12/d1b/f17 [0,4194304] 0 2024-02-19T19:04:51.437 INFO:tasks.workunit.client.1.smithi132.stdout:6/227: rename d0/db/d16 to d0/d3c/d46/d5a 0 2024-02-19T19:04:51.440 INFO:tasks.workunit.client.1.smithi132.stdout:3/233: creat d8/d38/f4a x:0 0 0 2024-02-19T19:04:51.442 INFO:tasks.workunit.client.1.smithi132.stdout:7/339: truncate d1b/d1e/f21 2600537 0 2024-02-19T19:04:51.450 INFO:tasks.workunit.client.1.smithi132.stdout:4/256: symlink d4/d19/l5b 0 2024-02-19T19:04:51.455 INFO:tasks.workunit.client.1.smithi132.stdout:1/260: truncate d1/d1e/d29/f3f 289547 0 2024-02-19T19:04:51.458 INFO:tasks.workunit.client.1.smithi132.stdout:1/261: dread d1/f15 [8388608,4194304] 0 2024-02-19T19:04:51.462 INFO:tasks.workunit.client.1.smithi132.stdout:8/291: sync 2024-02-19T19:04:51.462 INFO:tasks.workunit.client.1.smithi132.stdout:6/228: symlink d0/dc/l5b 0 2024-02-19T19:04:51.462 INFO:tasks.workunit.client.1.smithi132.stdout:1/262: dread d1/f4a [0,4194304] 0 2024-02-19T19:04:51.463 INFO:tasks.workunit.client.1.smithi132.stdout:1/263: chown d1/f1d 2 1 2024-02-19T19:04:51.463 INFO:tasks.workunit.client.1.smithi132.stdout:3/234: creat d8/d38/f4b x:0 0 0 2024-02-19T19:04:51.465 INFO:tasks.workunit.client.1.smithi132.stdout:8/292: read d4/f16 [971601,70325] 0 2024-02-19T19:04:51.486 INFO:tasks.workunit.client.1.smithi132.stdout:4/257: mkdir d4/d27/d31/d32/d51/d5c 0 2024-02-19T19:04:51.491 INFO:tasks.workunit.client.1.smithi132.stdout:4/258: dwrite d4/d12/f30 [0,4194304] 0 2024-02-19T19:04:51.492 INFO:tasks.workunit.client.1.smithi132.stdout:4/259: stat d4/d27/d31/l53 0 2024-02-19T19:04:51.493 INFO:tasks.workunit.client.1.smithi132.stdout:8/293: mkdir d4/dc/d1b/d20/d23/d68 0 2024-02-19T19:04:51.497 INFO:tasks.workunit.client.1.smithi132.stdout:4/260: dwrite d4/d27/f3c [0,4194304] 0 2024-02-19T19:04:51.499 INFO:tasks.workunit.client.1.smithi132.stdout:4/261: read d4/d12/d16/f47 [2952154,5393] 0 2024-02-19T19:04:51.512 INFO:tasks.workunit.client.1.smithi132.stdout:2/299: link d0/d24/c3d d0/d24/c61 0 2024-02-19T19:04:51.513 INFO:tasks.workunit.client.1.smithi132.stdout:2/300: truncate d0/d4/df/f5f 725930 0 2024-02-19T19:04:51.518 INFO:tasks.workunit.client.1.smithi132.stdout:2/301: dwrite d0/d4/d11/f1c [4194304,4194304] 0 2024-02-19T19:04:51.519 INFO:tasks.workunit.client.1.smithi132.stdout:0/288: creat d2/dc/d4a/f5e x:0 0 0 2024-02-19T19:04:51.521 INFO:tasks.workunit.client.1.smithi132.stdout:0/289: chown d2/dc/df/d15/d2f/c39 119748601 1 2024-02-19T19:04:51.525 INFO:tasks.workunit.client.1.smithi132.stdout:2/302: dwrite d0/d3c/f60 [0,4194304] 0 2024-02-19T19:04:51.531 INFO:tasks.workunit.client.1.smithi132.stdout:7/340: rename d1b/d1e/d28/f5b to d1b/f72 0 2024-02-19T19:04:51.531 INFO:tasks.workunit.client.1.smithi132.stdout:3/235: dwrite d8/d13/d2b/d2e/f3a [0,4194304] 0 2024-02-19T19:04:51.532 INFO:tasks.workunit.client.1.smithi132.stdout:3/236: dread - d8/d13/d2b/f2c zero size 2024-02-19T19:04:51.532 INFO:tasks.workunit.client.1.smithi132.stdout:1/264: creat d1/d1e/d29/d4e/f4f x:0 0 0 2024-02-19T19:04:51.540 INFO:tasks.workunit.client.1.smithi132.stdout:1/265: dwrite d1/f42 [0,4194304] 0 2024-02-19T19:04:51.544 INFO:tasks.workunit.client.1.smithi132.stdout:5/280: getdents d8/d23/d29/d27 0 2024-02-19T19:04:51.545 INFO:tasks.workunit.client.1.smithi132.stdout:2/303: rmdir d0/d4 39 2024-02-19T19:04:51.545 INFO:tasks.workunit.client.1.smithi132.stdout:5/281: chown d8/l22 7 1 2024-02-19T19:04:51.548 INFO:tasks.workunit.client.1.smithi132.stdout:7/341: symlink d1b/d5a/l73 0 2024-02-19T19:04:51.551 INFO:tasks.workunit.client.1.smithi132.stdout:8/294: getdents d4/dc/d1b/d20/d23/d68 0 2024-02-19T19:04:51.552 INFO:tasks.workunit.client.1.smithi132.stdout:8/295: write d4/dc/d42/f56 [205709,51956] 0 2024-02-19T19:04:51.553 INFO:tasks.workunit.client.1.smithi132.stdout:8/296: write d4/f24 [3531737,71325] 0 2024-02-19T19:04:51.566 INFO:tasks.workunit.client.1.smithi132.stdout:0/290: mkdir d2/dc/d4a/d25/d43/d5f 0 2024-02-19T19:04:51.566 INFO:tasks.workunit.client.1.smithi132.stdout:6/229: getdents d0/d3c/d46 0 2024-02-19T19:04:51.566 INFO:tasks.workunit.client.1.smithi132.stdout:6/230: fdatasync d0/f5 0 2024-02-19T19:04:51.567 INFO:tasks.workunit.client.1.smithi132.stdout:0/291: truncate d2/f53 990571 0 2024-02-19T19:04:51.567 INFO:tasks.workunit.client.1.smithi132.stdout:0/292: chown d2/dc/df 33 1 2024-02-19T19:04:51.567 INFO:tasks.workunit.client.1.smithi132.stdout:6/231: chown d0/d3c/d46/d5a/f37 10 1 2024-02-19T19:04:51.568 INFO:tasks.workunit.client.1.smithi132.stdout:0/293: write d2/f1a [4373955,56498] 0 2024-02-19T19:04:51.569 INFO:tasks.workunit.client.1.smithi132.stdout:0/294: dread - d2/dc/d4a/f57 zero size 2024-02-19T19:04:51.573 INFO:tasks.workunit.client.1.smithi132.stdout:0/295: dread d2/f2b [0,4194304] 0 2024-02-19T19:04:51.575 INFO:tasks.workunit.client.1.smithi132.stdout:5/282: write f6 [994810,81076] 0 2024-02-19T19:04:51.578 INFO:tasks.workunit.client.1.smithi132.stdout:4/262: truncate d4/d19/f23 1092735 0 2024-02-19T19:04:51.579 INFO:tasks.workunit.client.1.smithi132.stdout:4/263: chown d4/dc/d14/d24/l25 1 1 2024-02-19T19:04:51.584 INFO:tasks.workunit.client.1.smithi132.stdout:8/297: rmdir d4/d2d 39 2024-02-19T19:04:51.586 INFO:tasks.workunit.client.1.smithi132.stdout:6/232: creat d0/d3c/d22/d4d/f5c x:0 0 0 2024-02-19T19:04:51.587 INFO:tasks.workunit.client.1.smithi132.stdout:6/233: write d0/d3c/d46/d5a/f20 [2040010,35837] 0 2024-02-19T19:04:51.593 INFO:tasks.workunit.client.1.smithi132.stdout:6/234: dwrite d0/d3c/d22/f9 [0,4194304] 0 2024-02-19T19:04:51.594 INFO:tasks.workunit.client.1.smithi132.stdout:5/283: write d8/d23/f50 [955764,65302] 0 2024-02-19T19:04:51.595 INFO:tasks.workunit.client.1.smithi132.stdout:0/296: dread d2/dc/df/d15/f24 [0,4194304] 0 2024-02-19T19:04:51.602 INFO:tasks.workunit.client.1.smithi132.stdout:9/395: rename d12/d3e/d86 to d12/d2a/d88 0 2024-02-19T19:04:51.608 INFO:tasks.workunit.client.1.smithi132.stdout:7/342: mknod d1b/d1e/d2b/d45/c74 0 2024-02-19T19:04:51.609 INFO:tasks.workunit.client.1.smithi132.stdout:2/304: write d0/d4/f1f [180911,80176] 0 2024-02-19T19:04:51.612 INFO:tasks.workunit.client.1.smithi132.stdout:8/298: unlink d4/dc/f22 0 2024-02-19T19:04:51.620 INFO:tasks.workunit.client.1.smithi132.stdout:0/297: mknod d2/dc/df/d15/d34/c60 0 2024-02-19T19:04:51.621 INFO:tasks.workunit.client.1.smithi132.stdout:4/264: dread d4/d12/d16/f47 [0,4194304] 0 2024-02-19T19:04:51.621 INFO:tasks.workunit.client.1.smithi132.stdout:0/298: fdatasync d2/dc/df/d15/d2f/f40 0 2024-02-19T19:04:51.621 INFO:tasks.workunit.client.1.smithi132.stdout:2/305: dread d0/d19/f2c [0,4194304] 0 2024-02-19T19:04:51.622 INFO:tasks.workunit.client.1.smithi132.stdout:3/237: rename d8/d13/f16 to d8/d38/d44/f4c 0 2024-02-19T19:04:51.624 INFO:tasks.workunit.client.1.smithi132.stdout:9/396: mkdir d12/d25/d3a/d40/d4e/d77/d89 0 2024-02-19T19:04:51.625 INFO:tasks.workunit.client.1.smithi132.stdout:9/397: fsync d12/d3e/f6c 0 2024-02-19T19:04:51.626 INFO:tasks.workunit.client.1.smithi132.stdout:7/343: creat d1b/d46/d56/f75 x:0 0 0 2024-02-19T19:04:51.627 INFO:tasks.workunit.client.1.smithi132.stdout:7/344: write d1b/d1e/f30 [4067559,106238] 0 2024-02-19T19:04:51.633 INFO:tasks.workunit.client.1.smithi132.stdout:5/284: link d8/d1a/d35/f4a d8/d2e/f55 0 2024-02-19T19:04:51.639 INFO:tasks.workunit.client.1.smithi132.stdout:3/238: unlink d8/d13/d2b/d2e/f3a 0 2024-02-19T19:04:51.646 INFO:tasks.workunit.client.1.smithi132.stdout:5/285: creat d8/d23/d3b/f56 x:0 0 0 2024-02-19T19:04:51.647 INFO:tasks.workunit.client.1.smithi132.stdout:4/265: creat d4/d27/d31/d32/d51/d5c/f5d x:0 0 0 2024-02-19T19:04:51.647 INFO:tasks.workunit.client.1.smithi132.stdout:4/266: chown d4/d12/f2c 490661 1 2024-02-19T19:04:51.648 INFO:tasks.workunit.client.1.smithi132.stdout:4/267: fsync d4/d12/d28/f3e 0 2024-02-19T19:04:51.653 INFO:tasks.workunit.client.1.smithi132.stdout:0/299: link d2/dc/df/d15/d34/f41 d2/dc/df/d27/f61 0 2024-02-19T19:04:51.654 INFO:tasks.workunit.client.1.smithi132.stdout:0/300: dread - d2/dc/d4a/f57 zero size 2024-02-19T19:04:51.659 INFO:tasks.workunit.client.1.smithi132.stdout:5/286: fsync d8/d1a/d35/f4b 0 2024-02-19T19:04:51.660 INFO:tasks.workunit.client.1.smithi132.stdout:5/287: dread - d8/d23/d3b/f48 zero size 2024-02-19T19:04:51.662 INFO:tasks.workunit.client.1.smithi132.stdout:4/268: symlink d4/d19/d21/d56/l5e 0 2024-02-19T19:04:51.663 INFO:tasks.workunit.client.1.smithi132.stdout:4/269: write d4/f10 [1135330,90605] 0 2024-02-19T19:04:51.665 INFO:tasks.workunit.client.1.smithi132.stdout:1/266: rename d1/d4d to d1/d3a/d50 0 2024-02-19T19:04:51.667 INFO:tasks.workunit.client.1.smithi132.stdout:4/270: dread d4/d12/d28/f3e [0,4194304] 0 2024-02-19T19:04:51.668 INFO:tasks.workunit.client.1.smithi132.stdout:0/301: creat d2/dc/df/d27/d3a/f62 x:0 0 0 2024-02-19T19:04:51.668 INFO:tasks.workunit.client.1.smithi132.stdout:8/299: getdents d4/dc/d42 0 2024-02-19T19:04:51.675 INFO:tasks.workunit.client.1.smithi132.stdout:9/398: rename d12/d25/d3a/d78 to d12/d25/d3a/d40/d4e/d77/d8a 0 2024-02-19T19:04:51.678 INFO:tasks.workunit.client.1.smithi132.stdout:6/235: sync 2024-02-19T19:04:51.682 INFO:tasks.workunit.client.1.smithi132.stdout:4/271: mkdir d4/d12/d48/d5f 0 2024-02-19T19:04:51.682 INFO:tasks.workunit.client.1.smithi132.stdout:1/267: dread d1/f1d [0,4194304] 0 2024-02-19T19:04:51.683 INFO:tasks.workunit.client.1.smithi132.stdout:3/239: rename d8/d38/f4a to d8/d13/d2f/f4d 0 2024-02-19T19:04:51.685 INFO:tasks.workunit.client.1.smithi132.stdout:0/302: mkdir d2/dc/df/d15/d2f/d3f/d63 0 2024-02-19T19:04:51.686 INFO:tasks.workunit.client.1.smithi132.stdout:2/306: truncate d0/d4/d11/f41 6960297 0 2024-02-19T19:04:51.686 INFO:tasks.workunit.client.1.smithi132.stdout:6/236: mkdir d0/db/d25/d3a/d5d 0 2024-02-19T19:04:51.687 INFO:tasks.workunit.client.1.smithi132.stdout:0/303: chown d2/dc/d4a/f57 1372 1 2024-02-19T19:04:51.687 INFO:tasks.workunit.client.1.smithi132.stdout:7/345: dwrite d1b/d1e/d2b/f2f [0,4194304] 0 2024-02-19T19:04:51.692 INFO:tasks.workunit.client.1.smithi132.stdout:4/272: symlink d4/d12/d48/l60 0 2024-02-19T19:04:51.694 INFO:tasks.workunit.client.1.smithi132.stdout:5/288: rename d8/da/db/f1d to d8/d23/d29/d27/f57 0 2024-02-19T19:04:51.695 INFO:tasks.workunit.client.1.smithi132.stdout:5/289: dread - d8/d2e/f55 zero size 2024-02-19T19:04:51.695 INFO:tasks.workunit.client.1.smithi132.stdout:3/240: write d8/d13/d2f/f4d [1035388,2852] 0 2024-02-19T19:04:51.696 INFO:tasks.workunit.client.1.smithi132.stdout:3/241: fdatasync d8/f22 0 2024-02-19T19:04:51.700 INFO:tasks.workunit.client.1.smithi132.stdout:0/304: creat d2/dc/df/d15/d2f/f64 x:0 0 0 2024-02-19T19:04:51.701 INFO:tasks.workunit.client.1.smithi132.stdout:0/305: write d2/dc/df/d15/d34/f3c [481464,24502] 0 2024-02-19T19:04:51.701 INFO:tasks.workunit.client.1.smithi132.stdout:8/300: rmdir d4/dc/d1b/d38/d45 0 2024-02-19T19:04:51.702 INFO:tasks.workunit.client.1.smithi132.stdout:9/399: mknod d12/d2a/c8b 0 2024-02-19T19:04:51.703 INFO:tasks.workunit.client.1.smithi132.stdout:9/400: fdatasync d12/d3e/f4a 0 2024-02-19T19:04:51.704 INFO:tasks.workunit.client.1.smithi132.stdout:8/301: read d4/dc/d1b/d1d/f36 [1736914,24716] 0 2024-02-19T19:04:51.704 INFO:tasks.workunit.client.1.smithi132.stdout:1/268: mknod d1/c51 0 2024-02-19T19:04:51.708 INFO:tasks.workunit.client.1.smithi132.stdout:4/273: rename d4/dc/d37/f3a to d4/d19/f61 0 2024-02-19T19:04:51.715 INFO:tasks.workunit.client.1.smithi132.stdout:5/290: symlink d8/da/db/l58 0 2024-02-19T19:04:51.717 INFO:tasks.workunit.client.1.smithi132.stdout:3/242: mknod d8/d13/d19/c4e 0 2024-02-19T19:04:51.718 INFO:tasks.workunit.client.1.smithi132.stdout:3/243: truncate d8/d13/d2b/f33 681153 0 2024-02-19T19:04:51.719 INFO:tasks.workunit.client.1.smithi132.stdout:0/306: mknod d2/d3e/c65 0 2024-02-19T19:04:51.720 INFO:tasks.workunit.client.1.smithi132.stdout:0/307: fdatasync d2/dc/df/d27/f35 0 2024-02-19T19:04:51.723 INFO:tasks.workunit.client.1.smithi132.stdout:9/401: truncate d12/d25/d3a/d40/f65 935399 0 2024-02-19T19:04:51.727 INFO:tasks.workunit.client.1.smithi132.stdout:1/269: write d1/f4a [3503776,130682] 0 2024-02-19T19:04:51.728 INFO:tasks.workunit.client.1.smithi132.stdout:1/270: chown d1/d1e/d29/d2c/l46 11 1 2024-02-19T19:04:51.730 INFO:tasks.workunit.client.1.smithi132.stdout:8/302: dwrite d4/d2d/f46 [0,4194304] 0 2024-02-19T19:04:51.732 INFO:tasks.workunit.client.1.smithi132.stdout:7/346: getdents d1b/d46 0 2024-02-19T19:04:51.733 INFO:tasks.workunit.client.1.smithi132.stdout:7/347: dread - d1b/d1e/d2b/d62/f67 zero size 2024-02-19T19:04:51.734 INFO:tasks.workunit.client.1.smithi132.stdout:7/348: write d1b/d1e/d28/f4c [175719,94234] 0 2024-02-19T19:04:51.735 INFO:tasks.workunit.client.1.smithi132.stdout:3/244: mkdir d8/d13/d19/d4f 0 2024-02-19T19:04:51.735 INFO:tasks.workunit.client.1.smithi132.stdout:7/349: fdatasync d1b/d1e/d28/f31 0 2024-02-19T19:04:51.736 INFO:tasks.workunit.client.1.smithi132.stdout:7/350: chown d1b/d1e/d28/f31 683844 1 2024-02-19T19:04:51.738 INFO:tasks.workunit.client.1.smithi132.stdout:0/308: symlink d2/dc/df/l66 0 2024-02-19T19:04:51.744 INFO:tasks.workunit.client.1.smithi132.stdout:0/309: dwrite d2/dc/df/d15/d34/f3c [0,4194304] 0 2024-02-19T19:04:51.745 INFO:tasks.workunit.client.1.smithi132.stdout:0/310: chown d2/dc/d4a/d25/c50 1909 1 2024-02-19T19:04:51.754 INFO:tasks.workunit.client.1.smithi132.stdout:4/274: fdatasync d4/d19/f23 0 2024-02-19T19:04:51.755 INFO:tasks.workunit.client.1.smithi132.stdout:6/237: getdents d0 0 2024-02-19T19:04:51.758 INFO:tasks.workunit.client.1.smithi132.stdout:7/351: mkdir d1b/d46/d76 0 2024-02-19T19:04:51.760 INFO:tasks.workunit.client.1.smithi132.stdout:1/271: rmdir d1/d3a 39 2024-02-19T19:04:51.763 INFO:tasks.workunit.client.1.smithi132.stdout:4/275: creat d4/d12/d28/f62 x:0 0 0 2024-02-19T19:04:51.764 INFO:tasks.workunit.client.1.smithi132.stdout:4/276: write d4/d12/d28/f62 [548509,30180] 0 2024-02-19T19:04:51.764 INFO:tasks.workunit.client.1.smithi132.stdout:6/238: rename d0/dc/f15 to d0/db/d25/d3a/d5d/f5e 0 2024-02-19T19:04:51.765 INFO:tasks.workunit.client.1.smithi132.stdout:6/239: fsync d0/db/d25/d2f/f31 0 2024-02-19T19:04:51.766 INFO:tasks.workunit.client.1.smithi132.stdout:9/402: link d12/d1b/f3b d12/d25/d3a/d40/d4e/d77/d8a/d82/f8c 0 2024-02-19T19:04:51.768 INFO:tasks.workunit.client.1.smithi132.stdout:7/352: creat d1b/d1e/d28/f77 x:0 0 0 2024-02-19T19:04:51.773 INFO:tasks.workunit.client.1.smithi132.stdout:7/353: dwrite ff [4194304,4194304] 0 2024-02-19T19:04:51.778 INFO:tasks.workunit.client.1.smithi132.stdout:0/311: write d2/f2b [984926,19863] 0 2024-02-19T19:04:51.780 INFO:tasks.workunit.client.1.smithi132.stdout:6/240: unlink d0/db/d25/d38/l49 0 2024-02-19T19:04:51.784 INFO:tasks.workunit.client.1.smithi132.stdout:3/245: truncate d8/d13/f25 424659 0 2024-02-19T19:04:51.786 INFO:tasks.workunit.client.1.smithi132.stdout:7/354: mknod d1b/d5a/d63/c78 0 2024-02-19T19:04:51.789 INFO:tasks.workunit.client.1.smithi132.stdout:6/241: creat d0/d3c/d22/d4d/f5f x:0 0 0 2024-02-19T19:04:51.791 INFO:tasks.workunit.client.1.smithi132.stdout:0/312: mkdir d2/dc/d4a/d67 0 2024-02-19T19:04:51.791 INFO:tasks.workunit.client.1.smithi132.stdout:3/246: dread d8/d13/f37 [0,4194304] 0 2024-02-19T19:04:51.793 INFO:tasks.workunit.client.1.smithi132.stdout:6/242: creat d0/d51/f60 x:0 0 0 2024-02-19T19:04:51.795 INFO:tasks.workunit.client.1.smithi132.stdout:7/355: dwrite d1b/d1e/d28/f31 [0,4194304] 0 2024-02-19T19:04:51.797 INFO:tasks.workunit.client.1.smithi132.stdout:0/313: creat d2/dc/df/d27/d3a/d52/f68 x:0 0 0 2024-02-19T19:04:51.801 INFO:tasks.workunit.client.1.smithi132.stdout:3/247: dwrite d8/d13/d19/f41 [0,4194304] 0 2024-02-19T19:04:51.801 INFO:tasks.workunit.client.1.smithi132.stdout:6/243: unlink d0/dc/l2b 0 2024-02-19T19:04:51.803 INFO:tasks.workunit.client.1.smithi132.stdout:6/244: write d0/d3c/d46/d5a/f18 [5107161,124249] 0 2024-02-19T19:04:51.807 INFO:tasks.workunit.client.1.smithi132.stdout:3/248: dwrite d8/d13/d2f/f4d [0,4194304] 0 2024-02-19T19:04:51.809 INFO:tasks.workunit.client.1.smithi132.stdout:3/249: write d8/d13/f24 [430965,67056] 0 2024-02-19T19:04:51.810 INFO:tasks.workunit.client.1.smithi132.stdout:7/356: unlink d1b/d1e/d2b/f6f 0 2024-02-19T19:04:51.819 INFO:tasks.workunit.client.1.smithi132.stdout:4/277: write d4/f7 [1403238,11298] 0 2024-02-19T19:04:51.821 INFO:tasks.workunit.client.1.smithi132.stdout:6/245: rmdir d0/db/d25/d3a 39 2024-02-19T19:04:51.822 INFO:tasks.workunit.client.1.smithi132.stdout:4/278: write d4/d12/f30 [3663182,54741] 0 2024-02-19T19:04:51.823 INFO:tasks.workunit.client.1.smithi132.stdout:1/272: dwrite d1/d1e/d29/f3f [0,4194304] 0 2024-02-19T19:04:51.826 INFO:tasks.workunit.client.1.smithi132.stdout:2/307: sync 2024-02-19T19:04:51.827 INFO:tasks.workunit.client.1.smithi132.stdout:1/273: truncate d1/d1e/d29/d2c/f38 695608 0 2024-02-19T19:04:51.828 INFO:tasks.workunit.client.1.smithi132.stdout:3/250: dread d8/d13/d2b/d2e/f3b [0,4194304] 0 2024-02-19T19:04:51.828 INFO:tasks.workunit.client.1.smithi132.stdout:4/279: creat d4/d27/d31/d32/f63 x:0 0 0 2024-02-19T19:04:51.830 INFO:tasks.workunit.client.1.smithi132.stdout:6/246: creat d0/db/d25/d3a/d3d/f61 x:0 0 0 2024-02-19T19:04:51.832 INFO:tasks.workunit.client.1.smithi132.stdout:1/274: mkdir d1/d1e/d29/d52 0 2024-02-19T19:04:51.836 INFO:tasks.workunit.client.1.smithi132.stdout:4/280: creat d4/d4d/f64 x:0 0 0 2024-02-19T19:04:51.838 INFO:tasks.workunit.client.1.smithi132.stdout:2/308: dwrite d0/d19/d1e/f52 [0,4194304] 0 2024-02-19T19:04:51.839 INFO:tasks.workunit.client.1.smithi132.stdout:3/251: dwrite d8/d13/d19/f49 [0,4194304] 0 2024-02-19T19:04:51.841 INFO:tasks.workunit.client.1.smithi132.stdout:3/252: read d8/d13/d19/f49 [296141,84652] 0 2024-02-19T19:04:51.841 INFO:tasks.workunit.client.1.smithi132.stdout:4/281: truncate d4/d12/d16/f5a 993638 0 2024-02-19T19:04:51.843 INFO:tasks.workunit.client.1.smithi132.stdout:4/282: dread - d4/d27/d31/d32/f63 zero size 2024-02-19T19:04:51.845 INFO:tasks.workunit.client.1.smithi132.stdout:0/314: truncate d2/dc/df/d15/f4b 3972887 0 2024-02-19T19:04:51.850 INFO:tasks.workunit.client.1.smithi132.stdout:1/275: creat d1/f53 x:0 0 0 2024-02-19T19:04:51.855 INFO:tasks.workunit.client.1.smithi132.stdout:4/283: dwrite d4/f2d [0,4194304] 0 2024-02-19T19:04:51.856 INFO:tasks.workunit.client.1.smithi132.stdout:4/284: chown d4 97146824 1 2024-02-19T19:04:51.856 INFO:tasks.workunit.client.1.smithi132.stdout:7/357: dread ff [0,4194304] 0 2024-02-19T19:04:51.857 INFO:tasks.workunit.client.1.smithi132.stdout:7/358: chown d1b/d1e/d28/f29 205366411 1 2024-02-19T19:04:51.858 INFO:tasks.workunit.client.1.smithi132.stdout:3/253: mkdir d8/d31/d34/d50 0 2024-02-19T19:04:51.859 INFO:tasks.workunit.client.1.smithi132.stdout:3/254: stat d8/le 0 2024-02-19T19:04:51.859 INFO:tasks.workunit.client.1.smithi132.stdout:4/285: dread d4/d12/d16/f5a [0,4194304] 0 2024-02-19T19:04:51.859 INFO:tasks.workunit.client.1.smithi132.stdout:2/309: truncate d0/d4/fb 3372799 0 2024-02-19T19:04:51.867 INFO:tasks.workunit.client.1.smithi132.stdout:1/276: creat d1/d1e/d29/d2c/d43/f54 x:0 0 0 2024-02-19T19:04:51.873 INFO:tasks.workunit.client.1.smithi132.stdout:7/359: mkdir d1b/d5a/d63/d6b/d79 0 2024-02-19T19:04:51.874 INFO:tasks.workunit.client.1.smithi132.stdout:3/255: dwrite d8/d13/d19/f2d [0,4194304] 0 2024-02-19T19:04:51.876 INFO:tasks.workunit.client.1.smithi132.stdout:6/247: dwrite d0/d3c/f42 [0,4194304] 0 2024-02-19T19:04:51.879 INFO:tasks.workunit.client.1.smithi132.stdout:5/291: sync 2024-02-19T19:04:51.881 INFO:tasks.workunit.client.1.smithi132.stdout:0/315: truncate d2/f1a 3083024 0 2024-02-19T19:04:51.888 INFO:tasks.workunit.client.1.smithi132.stdout:7/360: mknod d1b/d1e/d2b/d45/c7a 0 2024-02-19T19:04:51.889 INFO:tasks.workunit.client.1.smithi132.stdout:6/248: unlink d0/d3c/d22/c3 0 2024-02-19T19:04:51.895 INFO:tasks.workunit.client.1.smithi132.stdout:4/286: truncate d4/d12/f1a 804732 0 2024-02-19T19:04:51.898 INFO:tasks.workunit.client.1.smithi132.stdout:4/287: dread d4/d12/d16/f47 [0,4194304] 0 2024-02-19T19:04:51.903 INFO:tasks.workunit.client.1.smithi132.stdout:2/310: getdents d0/d3c 0 2024-02-19T19:04:51.904 INFO:tasks.workunit.client.1.smithi132.stdout:2/311: chown d0/d19/d1e/f45 14 1 2024-02-19T19:04:51.906 INFO:tasks.workunit.client.1.smithi132.stdout:3/256: link d8/d13/d19/f41 d8/f51 0 2024-02-19T19:04:51.907 INFO:tasks.workunit.client.1.smithi132.stdout:3/257: stat d8/d13/f20 0 2024-02-19T19:04:51.907 INFO:tasks.workunit.client.1.smithi132.stdout:5/292: dwrite d8/d23/d29/d27/f25 [0,4194304] 0 2024-02-19T19:04:51.908 INFO:tasks.workunit.client.1.smithi132.stdout:3/258: chown d8/d13/d2f/f4d 53933 1 2024-02-19T19:04:51.913 INFO:tasks.workunit.client.1.smithi132.stdout:1/277: truncate d1/f42 2416595 0 2024-02-19T19:04:51.915 INFO:tasks.workunit.client.1.smithi132.stdout:6/249: symlink d0/db/d25/d3a/d3d/l62 0 2024-02-19T19:04:51.918 INFO:tasks.workunit.client.1.smithi132.stdout:7/361: write d1b/d46/f4d [2458253,103177] 0 2024-02-19T19:04:51.920 INFO:tasks.workunit.client.1.smithi132.stdout:1/278: dwrite d1/d1e/d29/d4e/f4f [0,4194304] 0 2024-02-19T19:04:51.921 INFO:tasks.workunit.client.1.smithi132.stdout:4/288: creat d4/d12/d48/f65 x:0 0 0 2024-02-19T19:04:51.922 INFO:tasks.workunit.client.1.smithi132.stdout:2/312: creat d0/d19/d1e/f62 x:0 0 0 2024-02-19T19:04:51.923 INFO:tasks.workunit.client.1.smithi132.stdout:1/279: stat d1/d1e/d29/d2c/l3b 0 2024-02-19T19:04:51.924 INFO:tasks.workunit.client.1.smithi132.stdout:5/293: creat d8/d23/d29/d3f/f59 x:0 0 0 2024-02-19T19:04:51.927 INFO:tasks.workunit.client.1.smithi132.stdout:6/250: dwrite d0/d3c/d22/d4d/f5f [0,4194304] 0 2024-02-19T19:04:51.928 INFO:tasks.workunit.client.1.smithi132.stdout:5/294: chown d8/da/db/f54 6949 1 2024-02-19T19:04:51.928 INFO:tasks.workunit.client.1.smithi132.stdout:8/303: sync 2024-02-19T19:04:51.928 INFO:tasks.workunit.client.1.smithi132.stdout:9/403: sync 2024-02-19T19:04:51.930 INFO:tasks.workunit.client.1.smithi132.stdout:6/251: write d0/f10 [5097482,100333] 0 2024-02-19T19:04:51.930 INFO:tasks.workunit.client.1.smithi132.stdout:7/362: rename d1b/d1e/c2e to d1b/d5a/d63/d6b/c7b 0 2024-02-19T19:04:51.931 INFO:tasks.workunit.client.1.smithi132.stdout:8/304: write d4/dc/d1b/d1d/f21 [767065,58452] 0 2024-02-19T19:04:51.933 INFO:tasks.workunit.client.1.smithi132.stdout:2/313: dread d0/d4/df/f3b [0,4194304] 0 2024-02-19T19:04:51.933 INFO:tasks.workunit.client.1.smithi132.stdout:4/289: unlink d4/d12/d16/c54 0 2024-02-19T19:04:51.934 INFO:tasks.workunit.client.1.smithi132.stdout:3/259: getdents d8/d31/d34/d50 0 2024-02-19T19:04:51.936 INFO:tasks.workunit.client.1.smithi132.stdout:3/260: write d8/d13/d2b/f33 [453842,94541] 0 2024-02-19T19:04:51.942 INFO:tasks.workunit.client.1.smithi132.stdout:1/280: dwrite d1/f1d [0,4194304] 0 2024-02-19T19:04:51.942 INFO:tasks.workunit.client.1.smithi132.stdout:7/363: rmdir d1b/d46/d56 39 2024-02-19T19:04:51.945 INFO:tasks.workunit.client.1.smithi132.stdout:3/261: dread d8/d13/d2f/f4d [0,4194304] 0 2024-02-19T19:04:51.946 INFO:tasks.workunit.client.1.smithi132.stdout:6/252: unlink d0/db/d25/d3a/d3d/l62 0 2024-02-19T19:04:51.949 INFO:tasks.workunit.client.1.smithi132.stdout:8/305: mkdir d4/dc/d1b/d38/d69 0 2024-02-19T19:04:51.952 INFO:tasks.workunit.client.1.smithi132.stdout:3/262: dwrite d8/d13/d19/f2d [0,4194304] 0 2024-02-19T19:04:51.955 INFO:tasks.workunit.client.1.smithi132.stdout:8/306: dwrite d4/dc/d1b/d1d/f36 [4194304,4194304] 0 2024-02-19T19:04:51.956 INFO:tasks.workunit.client.1.smithi132.stdout:8/307: write d4/dc/f44 [1502549,24105] 0 2024-02-19T19:04:51.956 INFO:tasks.workunit.client.1.smithi132.stdout:8/308: chown d4/dc/f1c 2971 1 2024-02-19T19:04:51.957 INFO:tasks.workunit.client.1.smithi132.stdout:4/290: mkdir d4/dc/d37/d66 0 2024-02-19T19:04:51.967 INFO:tasks.workunit.client.1.smithi132.stdout:0/316: truncate d2/f1a 1521141 0 2024-02-19T19:04:51.967 INFO:tasks.workunit.client.1.smithi132.stdout:7/364: readlink d1b/d1e/d41/l71 0 2024-02-19T19:04:51.967 INFO:tasks.workunit.client.1.smithi132.stdout:6/253: unlink d0/d3c/l2a 0 2024-02-19T19:04:51.968 INFO:tasks.workunit.client.1.smithi132.stdout:0/317: fsync d2/dc/df/d15/d34/f3c 0 2024-02-19T19:04:51.969 INFO:tasks.workunit.client.1.smithi132.stdout:0/318: fsync d2/f19 0 2024-02-19T19:04:51.970 INFO:tasks.workunit.client.1.smithi132.stdout:8/309: unlink d4/dc/d1b/d1d/f3c 0 2024-02-19T19:04:51.970 INFO:tasks.workunit.client.1.smithi132.stdout:7/365: truncate d1b/d1e/d2b/d62/f67 824664 0 2024-02-19T19:04:51.972 INFO:tasks.workunit.client.1.smithi132.stdout:9/404: rename d12/d25 to d12/d8d 0 2024-02-19T19:04:51.973 INFO:tasks.workunit.client.1.smithi132.stdout:8/310: dread - d4/dc/d1b/d54/d5e/f63 zero size 2024-02-19T19:04:51.973 INFO:tasks.workunit.client.1.smithi132.stdout:7/366: write d1b/d1e/d2b/f5d [376169,31848] 0 2024-02-19T19:04:51.975 INFO:tasks.workunit.client.1.smithi132.stdout:5/295: rename d8/d1a/d1e to d8/da/d5a 0 2024-02-19T19:04:51.976 INFO:tasks.workunit.client.1.smithi132.stdout:6/254: dwrite d0/db/d25/f34 [0,4194304] 0 2024-02-19T19:04:51.979 INFO:tasks.workunit.client.1.smithi132.stdout:4/291: symlink d4/dc/d37/d66/l67 0 2024-02-19T19:04:51.980 INFO:tasks.workunit.client.1.smithi132.stdout:0/319: mkdir d2/dc/df/d15/d2f/d69 0 2024-02-19T19:04:51.983 INFO:tasks.workunit.client.1.smithi132.stdout:7/367: mknod d1b/d5a/d63/c7c 0 2024-02-19T19:04:51.985 INFO:tasks.workunit.client.1.smithi132.stdout:7/368: dread - d1b/d1e/d2b/d62/d53/f70 zero size 2024-02-19T19:04:51.985 INFO:tasks.workunit.client.1.smithi132.stdout:5/296: mkdir d8/d1a/d35/d5b 0 2024-02-19T19:04:51.987 INFO:tasks.workunit.client.1.smithi132.stdout:7/369: chown d1b/d46/d76 31679123 1 2024-02-19T19:04:51.990 INFO:tasks.workunit.client.1.smithi132.stdout:9/405: write d12/d2a/f2b [7562222,11900] 0 2024-02-19T19:04:51.991 INFO:tasks.workunit.client.1.smithi132.stdout:0/320: dwrite d2/dc/df/d15/d34/f3c [0,4194304] 0 2024-02-19T19:04:51.995 INFO:tasks.workunit.client.1.smithi132.stdout:9/406: write d12/d1b/f1c [2602094,121697] 0 2024-02-19T19:04:51.996 INFO:tasks.workunit.client.1.smithi132.stdout:8/311: dread d4/dc/d1b/f43 [4194304,4194304] 0 2024-02-19T19:04:51.998 INFO:tasks.workunit.client.1.smithi132.stdout:1/281: dread d1/d1e/d29/f41 [0,4194304] 0 2024-02-19T19:04:51.999 INFO:tasks.workunit.client.1.smithi132.stdout:7/370: dwrite d1b/d1e/d28/f3b [0,4194304] 0 2024-02-19T19:04:52.000 INFO:tasks.workunit.client.1.smithi132.stdout:7/371: readlink d1b/d1e/l4a 0 2024-02-19T19:04:52.001 INFO:tasks.workunit.client.1.smithi132.stdout:0/321: rmdir d2/dc/d4a/d25 39 2024-02-19T19:04:52.008 INFO:tasks.workunit.client.1.smithi132.stdout:7/372: dwrite d1b/d1e/d28/f44 [0,4194304] 0 2024-02-19T19:04:52.009 INFO:tasks.workunit.client.1.smithi132.stdout:0/322: dwrite d2/dc/d4a/f57 [0,4194304] 0 2024-02-19T19:04:52.009 INFO:tasks.workunit.client.1.smithi132.stdout:7/373: stat d1b/d1e/d2b/f2f 0 2024-02-19T19:04:52.010 INFO:tasks.workunit.client.1.smithi132.stdout:7/374: write d1b/f25 [438429,109042] 0 2024-02-19T19:04:52.015 INFO:tasks.workunit.client.1.smithi132.stdout:9/407: readlink d12/d3e/l74 0 2024-02-19T19:04:52.017 INFO:tasks.workunit.client.1.smithi132.stdout:2/314: rename d0/d19/c47 to d0/d4/d11/c63 0 2024-02-19T19:04:52.028 INFO:tasks.workunit.client.1.smithi132.stdout:4/292: write d4/f59 [1813025,40599] 0 2024-02-19T19:04:52.029 INFO:tasks.workunit.client.1.smithi132.stdout:3/263: rename d8/d31 to d8/d13/d2b/d52 0 2024-02-19T19:04:52.031 INFO:tasks.workunit.client.1.smithi132.stdout:3/264: stat d8/d13/d2b/f2c 0 2024-02-19T19:04:52.031 INFO:tasks.workunit.client.1.smithi132.stdout:0/323: fsync d2/dc/df/d15/f4b 0 2024-02-19T19:04:52.033 INFO:tasks.workunit.client.1.smithi132.stdout:7/375: symlink d1b/d5a/d63/d6b/d79/l7d 0 2024-02-19T19:04:52.034 INFO:tasks.workunit.client.1.smithi132.stdout:5/297: dwrite d8/d1a/d35/f4b [0,4194304] 0 2024-02-19T19:04:52.035 INFO:tasks.workunit.client.1.smithi132.stdout:2/315: mkdir d0/d64 0 2024-02-19T19:04:52.036 INFO:tasks.workunit.client.1.smithi132.stdout:6/255: rename d0/l44 to d0/d3c/d22/d4d/l63 0 2024-02-19T19:04:52.039 INFO:tasks.workunit.client.1.smithi132.stdout:3/265: symlink d8/d13/d2b/d52/l53 0 2024-02-19T19:04:52.041 INFO:tasks.workunit.client.1.smithi132.stdout:7/376: dread d1b/d1e/d28/f4c [0,4194304] 0 2024-02-19T19:04:52.043 INFO:tasks.workunit.client.1.smithi132.stdout:4/293: link d4/d27/d31/d32/f63 d4/d27/d31/f68 0 2024-02-19T19:04:52.048 INFO:tasks.workunit.client.1.smithi132.stdout:3/266: dread d8/f22 [0,4194304] 0 2024-02-19T19:04:52.048 INFO:tasks.workunit.client.1.smithi132.stdout:7/377: dwrite d1b/d1e/f43 [0,4194304] 0 2024-02-19T19:04:52.050 INFO:tasks.workunit.client.1.smithi132.stdout:1/282: dread d1/f21 [0,4194304] 0 2024-02-19T19:04:52.051 INFO:tasks.workunit.client.1.smithi132.stdout:1/283: readlink d1/d1e/d29/d2c/l46 0 2024-02-19T19:04:52.051 INFO:tasks.workunit.client.1.smithi132.stdout:7/378: dread d1b/d1e/d2b/f51 [0,4194304] 0 2024-02-19T19:04:52.052 INFO:tasks.workunit.client.1.smithi132.stdout:8/312: truncate f2 464126 0 2024-02-19T19:04:52.053 INFO:tasks.workunit.client.1.smithi132.stdout:8/313: write d4/f24 [3914056,114060] 0 2024-02-19T19:04:52.054 INFO:tasks.workunit.client.1.smithi132.stdout:8/314: stat d4/dc/d1b/d54/d5e/l5f 0 2024-02-19T19:04:52.054 INFO:tasks.workunit.client.1.smithi132.stdout:9/408: rename d12/d2a/f7d to d12/d8d/d3a/d40/d4e/d77/f8e 0 2024-02-19T19:04:52.055 INFO:tasks.workunit.client.1.smithi132.stdout:8/315: fdatasync d4/f29 0 2024-02-19T19:04:52.055 INFO:tasks.workunit.client.1.smithi132.stdout:0/324: rename d2/dc/df/d15 to d2/dc/df/d15/d34/d6a 22 2024-02-19T19:04:52.056 INFO:tasks.workunit.client.1.smithi132.stdout:4/294: rmdir d4/d27/d31/d32/d51/d5c 39 2024-02-19T19:04:52.056 INFO:tasks.workunit.client.1.smithi132.stdout:9/409: write fb [1431772,104587] 0 2024-02-19T19:04:52.057 INFO:tasks.workunit.client.1.smithi132.stdout:2/316: truncate d0/d4/d11/f1c 7076712 0 2024-02-19T19:04:52.057 INFO:tasks.workunit.client.1.smithi132.stdout:4/295: stat d4/dc/d14/d24 0 2024-02-19T19:04:52.058 INFO:tasks.workunit.client.1.smithi132.stdout:4/296: fsync d4/f38 0 2024-02-19T19:04:52.060 INFO:tasks.workunit.client.1.smithi132.stdout:4/297: chown d4/dc/d37/d66/l67 133 1 2024-02-19T19:04:52.060 INFO:tasks.workunit.client.1.smithi132.stdout:1/284: rmdir d1/d1e 39 2024-02-19T19:04:52.063 INFO:tasks.workunit.client.1.smithi132.stdout:7/379: symlink d1b/d32/l7e 0 2024-02-19T19:04:52.069 INFO:tasks.workunit.client.1.smithi132.stdout:3/267: rename d8/f14 to d8/d13/d2b/d52/d34/d42/f54 0 2024-02-19T19:04:52.079 INFO:tasks.workunit.client.1.smithi132.stdout:9/410: mknod d12/d3e/c8f 0 2024-02-19T19:04:52.089 INFO:tasks.workunit.client.1.smithi132.stdout:4/298: symlink d4/dc/d37/l69 0 2024-02-19T19:04:52.091 INFO:tasks.workunit.client.1.smithi132.stdout:7/380: unlink f10 0 2024-02-19T19:04:52.091 INFO:tasks.workunit.client.1.smithi132.stdout:4/299: truncate d4/d12/d48/f65 648275 0 2024-02-19T19:04:52.092 INFO:tasks.workunit.client.1.smithi132.stdout:1/285: dwrite d1/d1e/d29/d2c/f34 [0,4194304] 0 2024-02-19T19:04:52.092 INFO:tasks.workunit.client.1.smithi132.stdout:2/317: write d0/d19/f2c [1021592,51646] 0 2024-02-19T19:04:52.094 INFO:tasks.workunit.client.1.smithi132.stdout:4/300: read d4/d12/d28/f3e [898993,96118] 0 2024-02-19T19:04:52.098 INFO:tasks.workunit.client.1.smithi132.stdout:8/316: fdatasync d4/fd 0 2024-02-19T19:04:52.099 INFO:tasks.workunit.client.1.smithi132.stdout:0/325: mkdir d2/dc/df/d15/d2f/d3f/d63/d6b 0 2024-02-19T19:04:52.099 INFO:tasks.workunit.client.1.smithi132.stdout:8/317: readlink d4/dc/d1b/d1d/l61 0 2024-02-19T19:04:52.101 INFO:tasks.workunit.client.1.smithi132.stdout:7/381: dwrite d1b/d1e/d28/f3c [4194304,4194304] 0 2024-02-19T19:04:52.101 INFO:tasks.workunit.client.1.smithi132.stdout:9/411: read d12/d8d/d3a/d40/f54 [17992,10813] 0 2024-02-19T19:04:52.102 INFO:tasks.workunit.client.1.smithi132.stdout:8/318: read - d4/dc/d1b/d33/f5b zero size 2024-02-19T19:04:52.106 INFO:tasks.workunit.client.1.smithi132.stdout:6/256: chown d0/db/d25/f40 1613896 1 2024-02-19T19:04:52.110 INFO:tasks.workunit.client.1.smithi132.stdout:0/326: chown d2/dc/d4a/d25/d43 55028385 1 2024-02-19T19:04:52.112 INFO:tasks.workunit.client.1.smithi132.stdout:7/382: chown d1b/d1e/d28/c2a 0 1 2024-02-19T19:04:52.113 INFO:tasks.workunit.client.1.smithi132.stdout:1/286: symlink d1/d1e/d29/d52/l55 0 2024-02-19T19:04:52.116 INFO:tasks.workunit.client.1.smithi132.stdout:9/412: creat d12/d8d/d3a/d40/d4e/d77/d8a/d82/f90 x:0 0 0 2024-02-19T19:04:52.119 INFO:tasks.workunit.client.1.smithi132.stdout:1/287: rename d1/f21 to d1/d1e/d29/d52/f56 0 2024-02-19T19:04:52.120 INFO:tasks.workunit.client.1.smithi132.stdout:6/257: dwrite d0/dc/f24 [0,4194304] 0 2024-02-19T19:04:52.123 INFO:tasks.workunit.client.1.smithi132.stdout:2/318: getdents d0/d24/d58 0 2024-02-19T19:04:52.127 INFO:tasks.workunit.client.1.smithi132.stdout:1/288: dwrite d1/f48 [0,4194304] 0 2024-02-19T19:04:52.128 INFO:tasks.workunit.client.1.smithi132.stdout:9/413: symlink d12/d2a/d88/l91 0 2024-02-19T19:04:52.130 INFO:tasks.workunit.client.1.smithi132.stdout:9/414: read - d12/d8d/d3a/d40/d4e/f67 zero size 2024-02-19T19:04:52.130 INFO:tasks.workunit.client.1.smithi132.stdout:3/268: dwrite d8/d13/d19/f41 [0,4194304] 0 2024-02-19T19:04:52.132 INFO:tasks.workunit.client.1.smithi132.stdout:6/258: creat d0/d3c/d22/f64 x:0 0 0 2024-02-19T19:04:52.133 INFO:tasks.workunit.client.1.smithi132.stdout:6/259: fdatasync d0/d3c/d22/f64 0 2024-02-19T19:04:52.134 INFO:tasks.workunit.client.1.smithi132.stdout:1/289: readlink d1/l10 0 2024-02-19T19:04:52.151 INFO:tasks.workunit.client.1.smithi132.stdout:3/269: unlink d8/d13/d2b/d52/l3f 0 2024-02-19T19:04:52.153 INFO:tasks.workunit.client.1.smithi132.stdout:3/270: readlink d8/l27 0 2024-02-19T19:04:52.153 INFO:tasks.workunit.client.1.smithi132.stdout:2/319: rename d0/d24/f39 to d0/d4/d11/f65 0 2024-02-19T19:04:52.153 INFO:tasks.workunit.client.1.smithi132.stdout:9/415: symlink d12/d8d/d3a/d40/d4e/d77/d89/l92 0 2024-02-19T19:04:52.154 INFO:tasks.workunit.client.1.smithi132.stdout:2/320: fdatasync d0/d19/f2c 0 2024-02-19T19:04:52.155 INFO:tasks.workunit.client.1.smithi132.stdout:8/319: write f2 [1404627,11957] 0 2024-02-19T19:04:52.156 INFO:tasks.workunit.client.1.smithi132.stdout:8/320: fsync d4/f19 0 2024-02-19T19:04:52.160 INFO:tasks.workunit.client.1.smithi132.stdout:9/416: write d12/d2a/d4c/d61/f6b [254327,97759] 0 2024-02-19T19:04:52.161 INFO:tasks.workunit.client.1.smithi132.stdout:7/383: dwrite d1b/d5a/f68 [0,4194304] 0 2024-02-19T19:04:52.163 INFO:tasks.workunit.client.1.smithi132.stdout:1/290: symlink d1/d3a/d50/l57 0 2024-02-19T19:04:52.163 INFO:tasks.workunit.client.1.smithi132.stdout:7/384: read d1b/d1e/d28/f31 [773146,108940] 0 2024-02-19T19:04:52.164 INFO:tasks.workunit.client.1.smithi132.stdout:1/291: fsync d1/f18 0 2024-02-19T19:04:52.165 INFO:tasks.workunit.client.1.smithi132.stdout:2/321: mkdir d0/d4/d11/d2e/d5a/d66 0 2024-02-19T19:04:52.165 INFO:tasks.workunit.client.1.smithi132.stdout:1/292: fdatasync d1/d1e/f2a 0 2024-02-19T19:04:52.166 INFO:tasks.workunit.client.1.smithi132.stdout:9/417: creat d12/d3e/f93 x:0 0 0 2024-02-19T19:04:52.167 INFO:tasks.workunit.client.1.smithi132.stdout:2/322: chown d0/d19/f2c 5518218 1 2024-02-19T19:04:52.170 INFO:tasks.workunit.client.1.smithi132.stdout:9/418: truncate d12/d8d/d3a/f81 817171 0 2024-02-19T19:04:52.171 INFO:tasks.workunit.client.1.smithi132.stdout:8/321: mknod d4/dc/d1b/d38/d69/c6a 0 2024-02-19T19:04:52.172 INFO:tasks.workunit.client.1.smithi132.stdout:7/385: dwrite d1b/d1e/f30 [0,4194304] 0 2024-02-19T19:04:52.176 INFO:tasks.workunit.client.1.smithi132.stdout:3/271: rmdir d8/d13/d19/d4f 0 2024-02-19T19:04:52.176 INFO:tasks.workunit.client.1.smithi132.stdout:7/386: write d1b/d1e/f21 [1594563,102456] 0 2024-02-19T19:04:52.176 INFO:tasks.workunit.client.1.smithi132.stdout:1/293: unlink d1/l14 0 2024-02-19T19:04:52.178 INFO:tasks.workunit.client.1.smithi132.stdout:2/323: creat d0/d24/d58/f67 x:0 0 0 2024-02-19T19:04:52.180 INFO:tasks.workunit.client.1.smithi132.stdout:9/419: creat d12/d3e/f94 x:0 0 0 2024-02-19T19:04:52.180 INFO:tasks.workunit.client.1.smithi132.stdout:5/298: sync 2024-02-19T19:04:52.181 INFO:tasks.workunit.client.1.smithi132.stdout:7/387: write d1b/d1e/d2b/f2f [1169761,30878] 0 2024-02-19T19:04:52.182 INFO:tasks.workunit.client.1.smithi132.stdout:8/322: dread d4/dc/d1b/d20/f53 [0,4194304] 0 2024-02-19T19:04:52.182 INFO:tasks.workunit.client.1.smithi132.stdout:6/260: dread d0/fa [0,4194304] 0 2024-02-19T19:04:52.185 INFO:tasks.workunit.client.1.smithi132.stdout:2/324: mkdir d0/d4/df/d40/d68 0 2024-02-19T19:04:52.186 INFO:tasks.workunit.client.1.smithi132.stdout:9/420: creat d12/d1b/f95 x:0 0 0 2024-02-19T19:04:52.187 INFO:tasks.workunit.client.1.smithi132.stdout:1/294: dread d1/d1e/d29/f41 [0,4194304] 0 2024-02-19T19:04:52.187 INFO:tasks.workunit.client.1.smithi132.stdout:5/299: fsync d8/d1a/d35/f4a 0 2024-02-19T19:04:52.187 INFO:tasks.workunit.client.1.smithi132.stdout:2/325: stat d0/d19/d1e/f52 0 2024-02-19T19:04:52.190 INFO:tasks.workunit.client.1.smithi132.stdout:9/421: mknod d12/d3e/c96 0 2024-02-19T19:04:52.191 INFO:tasks.workunit.client.1.smithi132.stdout:6/261: dread d0/dc/f24 [0,4194304] 0 2024-02-19T19:04:52.193 INFO:tasks.workunit.client.1.smithi132.stdout:7/388: rename d1b/d5a/d5c to d1b/d46/d7f 0 2024-02-19T19:04:52.193 INFO:tasks.workunit.client.1.smithi132.stdout:2/326: mknod d0/d24/d58/c69 0 2024-02-19T19:04:52.197 INFO:tasks.workunit.client.1.smithi132.stdout:6/262: dwrite d0/d3c/f39 [0,4194304] 0 2024-02-19T19:04:52.207 INFO:tasks.workunit.client.1.smithi132.stdout:6/263: unlink d0/db/d25/f40 0 2024-02-19T19:04:52.208 INFO:tasks.workunit.client.1.smithi132.stdout:6/264: fdatasync d0/d3c/d22/f64 0 2024-02-19T19:04:52.209 INFO:tasks.workunit.client.1.smithi132.stdout:6/265: dread - d0/db/d25/d3a/d3d/f61 zero size 2024-02-19T19:04:52.209 INFO:tasks.workunit.client.1.smithi132.stdout:5/300: link d8/d23/d3b/f56 d8/d23/f5c 0 2024-02-19T19:04:52.210 INFO:tasks.workunit.client.1.smithi132.stdout:6/266: write d0/d3c/d46/d5a/f37 [591849,107284] 0 2024-02-19T19:04:52.210 INFO:tasks.workunit.client.1.smithi132.stdout:2/327: mkdir d0/d19/d6a 0 2024-02-19T19:04:52.215 INFO:tasks.workunit.client.1.smithi132.stdout:6/267: mknod d0/db/c65 0 2024-02-19T19:04:52.218 INFO:tasks.workunit.client.1.smithi132.stdout:8/323: dread d4/dc/f2c [0,4194304] 0 2024-02-19T19:04:52.218 INFO:tasks.workunit.client.1.smithi132.stdout:2/328: mknod d0/d4/d11/d2e/d5a/d66/c6b 0 2024-02-19T19:04:52.219 INFO:tasks.workunit.client.1.smithi132.stdout:3/272: truncate d8/d13/f20 1594141 0 2024-02-19T19:04:52.219 INFO:tasks.workunit.client.1.smithi132.stdout:8/324: truncate d4/dc/d1b/f43 1420073 0 2024-02-19T19:04:52.220 INFO:tasks.workunit.client.1.smithi132.stdout:6/268: creat d0/dc/f66 x:0 0 0 2024-02-19T19:04:52.221 INFO:tasks.workunit.client.1.smithi132.stdout:6/269: chown d0/d3c/d22/f23 0 1 2024-02-19T19:04:52.222 INFO:tasks.workunit.client.1.smithi132.stdout:3/273: creat d8/d13/d2b/d52/f55 x:0 0 0 2024-02-19T19:04:52.227 INFO:tasks.workunit.client.1.smithi132.stdout:3/274: mknod d8/d13/d2b/d52/d34/d50/c56 0 2024-02-19T19:04:52.228 INFO:tasks.workunit.client.1.smithi132.stdout:3/275: truncate d8/d38/f4b 110723 0 2024-02-19T19:04:52.230 INFO:tasks.workunit.client.1.smithi132.stdout:6/270: rename d0/db/d25/l3b to d0/db/d25/l67 0 2024-02-19T19:04:52.232 INFO:tasks.workunit.client.1.smithi132.stdout:3/276: dread d8/d13/d19/f49 [0,4194304] 0 2024-02-19T19:04:52.232 INFO:tasks.workunit.client.1.smithi132.stdout:1/295: truncate d1/d1e/d29/d4e/f4f 1396513 0 2024-02-19T19:04:52.232 INFO:tasks.workunit.client.1.smithi132.stdout:6/271: dread d0/db/d25/d2f/f4a [0,4194304] 0 2024-02-19T19:04:52.233 INFO:tasks.workunit.client.1.smithi132.stdout:6/272: fdatasync d0/dc/f66 0 2024-02-19T19:04:52.233 INFO:tasks.workunit.client.1.smithi132.stdout:9/422: truncate d12/d1b/f1c 5685873 0 2024-02-19T19:04:52.234 INFO:tasks.workunit.client.1.smithi132.stdout:9/423: truncate d12/d8d/d3a/d40/f49 5073139 0 2024-02-19T19:04:52.235 INFO:tasks.workunit.client.1.smithi132.stdout:9/424: truncate d12/d1b/f95 514087 0 2024-02-19T19:04:52.237 INFO:tasks.workunit.client.1.smithi132.stdout:1/296: mknod d1/d3a/c58 0 2024-02-19T19:04:52.239 INFO:tasks.workunit.client.1.smithi132.stdout:6/273: write d0/dc/f24 [1552433,38571] 0 2024-02-19T19:04:52.240 INFO:tasks.workunit.client.1.smithi132.stdout:9/425: mknod d12/d8d/d3a/d85/c97 0 2024-02-19T19:04:52.242 INFO:tasks.workunit.client.1.smithi132.stdout:5/301: truncate d8/d1a/d35/f40 111263 0 2024-02-19T19:04:52.242 INFO:tasks.workunit.client.1.smithi132.stdout:1/297: chown d1/d3a/l24 0 1 2024-02-19T19:04:52.249 INFO:tasks.workunit.client.1.smithi132.stdout:1/298: read - d1/d3a/f30 zero size 2024-02-19T19:04:52.301 INFO:tasks.workunit.client.1.smithi132.stdout:4/301: sync 2024-02-19T19:04:52.302 INFO:tasks.workunit.client.1.smithi132.stdout:6/274: rename d0/f5 to d0/db/d25/d38/f68 0 2024-02-19T19:04:52.303 INFO:tasks.workunit.client.1.smithi132.stdout:3/277: write d8/d13/f37 [1474707,46506] 0 2024-02-19T19:04:52.303 INFO:tasks.workunit.client.1.smithi132.stdout:9/426: dwrite d12/d2a/f37 [0,4194304] 0 2024-02-19T19:04:52.304 INFO:tasks.workunit.client.1.smithi132.stdout:5/302: mkdir d8/d23/d5d 0 2024-02-19T19:04:52.306 INFO:tasks.workunit.client.1.smithi132.stdout:5/303: write d8/d23/f45 [2395421,1332] 0 2024-02-19T19:04:52.308 INFO:tasks.workunit.client.1.smithi132.stdout:4/302: creat d4/dc/d37/f6a x:0 0 0 2024-02-19T19:04:52.310 INFO:tasks.workunit.client.1.smithi132.stdout:1/299: dwrite d1/d1e/d29/d2c/f34 [0,4194304] 0 2024-02-19T19:04:52.319 INFO:tasks.workunit.client.1.smithi132.stdout:3/278: dwrite d8/f28 [0,4194304] 0 2024-02-19T19:04:52.319 INFO:tasks.workunit.client.1.smithi132.stdout:4/303: dwrite d4/d12/d28/f3e [0,4194304] 0 2024-02-19T19:04:52.319 INFO:tasks.workunit.client.1.smithi132.stdout:5/304: creat d8/d1a/f5e x:0 0 0 2024-02-19T19:04:52.322 INFO:tasks.workunit.client.1.smithi132.stdout:3/279: truncate d8/d13/d2b/d52/d34/d42/f54 9037326 0 2024-02-19T19:04:52.325 INFO:tasks.workunit.client.1.smithi132.stdout:3/280: fdatasync d8/d13/d2f/f36 0 2024-02-19T19:04:52.338 INFO:tasks.workunit.client.1.smithi132.stdout:9/427: dread d12/d8d/f35 [0,4194304] 0 2024-02-19T19:04:52.342 INFO:tasks.workunit.client.1.smithi132.stdout:3/281: mkdir d8/d13/d57 0 2024-02-19T19:04:52.345 INFO:tasks.workunit.client.1.smithi132.stdout:6/275: rename d0/d3c/d22/d4d to d0/db/d25/d3a/d3d/d69 0 2024-02-19T19:04:52.348 INFO:tasks.workunit.client.1.smithi132.stdout:3/282: unlink d8/d13/d19/c23 0 2024-02-19T19:04:52.349 INFO:tasks.workunit.client.1.smithi132.stdout:3/283: write f0 [3095916,71360] 0 2024-02-19T19:04:52.350 INFO:tasks.workunit.client.1.smithi132.stdout:3/284: chown d8/f22 2483 1 2024-02-19T19:04:52.353 INFO:tasks.workunit.client.1.smithi132.stdout:9/428: mknod d12/d8d/d3a/d85/d87/c98 0 2024-02-19T19:04:52.353 INFO:tasks.workunit.client.1.smithi132.stdout:1/300: write d1/d1e/d29/d2c/f2d [3995069,122021] 0 2024-02-19T19:04:52.354 INFO:tasks.workunit.client.1.smithi132.stdout:1/301: read - d1/d3a/f30 zero size 2024-02-19T19:04:52.355 INFO:tasks.workunit.client.1.smithi132.stdout:1/302: readlink d1/d3a/l24 0 2024-02-19T19:04:52.356 INFO:tasks.workunit.client.1.smithi132.stdout:6/276: creat d0/dc/f6a x:0 0 0 2024-02-19T19:04:52.358 INFO:tasks.workunit.client.1.smithi132.stdout:1/303: creat d1/d1e/d29/d2c/f59 x:0 0 0 2024-02-19T19:04:52.359 INFO:tasks.workunit.client.1.smithi132.stdout:1/304: stat d1/d3a/d50/l57 0 2024-02-19T19:04:52.361 INFO:tasks.workunit.client.1.smithi132.stdout:3/285: read d8/d13/f20 [1475431,125890] 0 2024-02-19T19:04:52.361 INFO:tasks.workunit.client.1.smithi132.stdout:6/277: symlink d0/db/l6b 0 2024-02-19T19:04:52.361 INFO:tasks.workunit.client.1.smithi132.stdout:1/305: read d1/f5 [202626,128793] 0 2024-02-19T19:04:52.362 INFO:tasks.workunit.client.1.smithi132.stdout:4/304: dwrite d4/d12/d16/f5a [0,4194304] 0 2024-02-19T19:04:52.363 INFO:tasks.workunit.client.1.smithi132.stdout:9/429: creat d12/f99 x:0 0 0 2024-02-19T19:04:52.363 INFO:tasks.workunit.client.1.smithi132.stdout:4/305: chown d4/f2d 68 1 2024-02-19T19:04:52.367 INFO:tasks.workunit.client.1.smithi132.stdout:5/305: dread d8/da/f24 [0,4194304] 0 2024-02-19T19:04:52.369 INFO:tasks.workunit.client.1.smithi132.stdout:3/286: dwrite d8/d13/d2f/f47 [4194304,4194304] 0 2024-02-19T19:04:52.373 INFO:tasks.workunit.client.1.smithi132.stdout:9/430: symlink d12/d2a/d4c/d61/l9a 0 2024-02-19T19:04:52.376 INFO:tasks.workunit.client.1.smithi132.stdout:4/306: unlink d4/d19/d21/c45 0 2024-02-19T19:04:52.380 INFO:tasks.workunit.client.1.smithi132.stdout:9/431: dwrite d12/d8d/d3a/f6d [0,4194304] 0 2024-02-19T19:04:52.381 INFO:tasks.workunit.client.1.smithi132.stdout:9/432: stat d12/f53 0 2024-02-19T19:04:52.382 INFO:tasks.workunit.client.1.smithi132.stdout:5/306: rename d8/da/db/l58 to d8/da/db/l5f 0 2024-02-19T19:04:52.382 INFO:tasks.workunit.client.1.smithi132.stdout:5/307: chown d8/d23/d3b 44088290 1 2024-02-19T19:04:52.384 INFO:tasks.workunit.client.1.smithi132.stdout:3/287: creat d8/d13/d2f/f58 x:0 0 0 2024-02-19T19:04:52.384 INFO:tasks.workunit.client.1.smithi132.stdout:5/308: truncate d8/d23/d29/d27/f2b 5281139 0 2024-02-19T19:04:52.385 INFO:tasks.workunit.client.1.smithi132.stdout:0/327: sync 2024-02-19T19:04:52.385 INFO:tasks.workunit.client.1.smithi132.stdout:4/307: creat d4/d12/d16/f6b x:0 0 0 2024-02-19T19:04:52.388 INFO:tasks.workunit.client.1.smithi132.stdout:1/306: creat d1/d1e/f5a x:0 0 0 2024-02-19T19:04:52.389 INFO:tasks.workunit.client.1.smithi132.stdout:9/433: creat d12/d2a/d4c/f9b x:0 0 0 2024-02-19T19:04:52.392 INFO:tasks.workunit.client.1.smithi132.stdout:3/288: creat d8/d13/d2b/d52/d34/f59 x:0 0 0 2024-02-19T19:04:52.393 INFO:tasks.workunit.client.1.smithi132.stdout:3/289: write d8/d13/d2b/f2c [802435,114616] 0 2024-02-19T19:04:52.395 INFO:tasks.workunit.client.1.smithi132.stdout:4/308: creat d4/d12/d16/f6c x:0 0 0 2024-02-19T19:04:52.397 INFO:tasks.workunit.client.1.smithi132.stdout:9/434: creat d12/d8d/d3a/d40/d4e/d77/f9c x:0 0 0 2024-02-19T19:04:52.397 INFO:tasks.workunit.client.1.smithi132.stdout:9/435: readlink d12/l2d 0 2024-02-19T19:04:52.397 INFO:tasks.workunit.client.1.smithi132.stdout:6/278: link d0/db/d25/l56 d0/d51/l6c 0 2024-02-19T19:04:52.399 INFO:tasks.workunit.client.1.smithi132.stdout:9/436: dread - d12/d3e/f71 zero size 2024-02-19T19:04:52.400 INFO:tasks.workunit.client.1.smithi132.stdout:4/309: mknod d4/d19/d21/c6d 0 2024-02-19T19:04:52.400 INFO:tasks.workunit.client.1.smithi132.stdout:6/279: dread - d0/db/d25/d3a/d3d/d69/f55 zero size 2024-02-19T19:04:52.401 INFO:tasks.workunit.client.1.smithi132.stdout:3/290: dwrite d8/d38/f3d [0,4194304] 0 2024-02-19T19:04:52.401 INFO:tasks.workunit.client.1.smithi132.stdout:4/310: stat d4/fa 0 2024-02-19T19:04:52.403 INFO:tasks.workunit.client.1.smithi132.stdout:6/280: read - d0/db/d25/d3a/d3d/d69/f4e zero size 2024-02-19T19:04:52.403 INFO:tasks.workunit.client.1.smithi132.stdout:4/311: write d4/d4d/f64 [515415,9878] 0 2024-02-19T19:04:52.403 INFO:tasks.workunit.client.1.smithi132.stdout:0/328: creat d2/dc/d4a/f6c x:0 0 0 2024-02-19T19:04:52.406 INFO:tasks.workunit.client.1.smithi132.stdout:9/437: mkdir d12/d8d/d3a/d40/d4e/d77/d9d 0 2024-02-19T19:04:52.406 INFO:tasks.workunit.client.1.smithi132.stdout:4/312: chown d4/dc/d14/d24/f3b 102 1 2024-02-19T19:04:52.410 INFO:tasks.workunit.client.1.smithi132.stdout:3/291: dwrite f0 [0,4194304] 0 2024-02-19T19:04:52.414 INFO:tasks.workunit.client.1.smithi132.stdout:3/292: readlink d8/d13/d19/l1d 0 2024-02-19T19:04:52.415 INFO:tasks.workunit.client.1.smithi132.stdout:0/329: dwrite d2/dc/d4a/f6c [0,4194304] 0 2024-02-19T19:04:52.415 INFO:tasks.workunit.client.1.smithi132.stdout:6/281: unlink d0/dc/f6a 0 2024-02-19T19:04:52.419 INFO:tasks.workunit.client.1.smithi132.stdout:0/330: chown d2/d3e 893 1 2024-02-19T19:04:52.422 INFO:tasks.workunit.client.1.smithi132.stdout:5/309: dwrite d8/d23/f5c [0,4194304] 0 2024-02-19T19:04:52.425 INFO:tasks.workunit.client.1.smithi132.stdout:3/293: unlink d8/f28 0 2024-02-19T19:04:52.425 INFO:tasks.workunit.client.1.smithi132.stdout:0/331: chown d2/dc/d4a 9763 1 2024-02-19T19:04:52.425 INFO:tasks.workunit.client.1.smithi132.stdout:9/438: rename d12/d8d/d3a/d40/c57 to d12/d2a/d4c/d61/c9e 0 2024-02-19T19:04:52.428 INFO:tasks.workunit.client.1.smithi132.stdout:3/294: stat d8/d13/f20 0 2024-02-19T19:04:52.428 INFO:tasks.workunit.client.1.smithi132.stdout:9/439: chown d12/d1b/f17 2639673 1 2024-02-19T19:04:52.429 INFO:tasks.workunit.client.1.smithi132.stdout:0/332: creat d2/dc/f6d x:0 0 0 2024-02-19T19:04:52.429 INFO:tasks.workunit.client.1.smithi132.stdout:3/295: chown d8/d13/d2b/f2c 234 1 2024-02-19T19:04:52.431 INFO:tasks.workunit.client.1.smithi132.stdout:4/313: getdents d4/dc/d37 0 2024-02-19T19:04:52.432 INFO:tasks.workunit.client.1.smithi132.stdout:5/310: symlink d8/d23/d5d/l60 0 2024-02-19T19:04:52.433 INFO:tasks.workunit.client.1.smithi132.stdout:2/329: sync 2024-02-19T19:04:52.433 INFO:tasks.workunit.client.1.smithi132.stdout:7/389: sync 2024-02-19T19:04:52.438 INFO:tasks.workunit.client.1.smithi132.stdout:9/440: creat d12/d3e/f9f x:0 0 0 2024-02-19T19:04:52.438 INFO:tasks.workunit.client.1.smithi132.stdout:1/307: dread d1/f15 [4194304,4194304] 0 2024-02-19T19:04:52.439 INFO:tasks.workunit.client.1.smithi132.stdout:3/296: creat d8/f5a x:0 0 0 2024-02-19T19:04:52.439 INFO:tasks.workunit.client.1.smithi132.stdout:4/314: dwrite d4/d12/d48/f65 [0,4194304] 0 2024-02-19T19:04:52.440 INFO:tasks.workunit.client.1.smithi132.stdout:2/330: creat d0/d4/d11/d2e/f6c x:0 0 0 2024-02-19T19:04:52.442 INFO:tasks.workunit.client.1.smithi132.stdout:8/325: sync 2024-02-19T19:04:52.442 INFO:tasks.workunit.client.1.smithi132.stdout:5/311: fsync d8/da/f33 0 2024-02-19T19:04:52.443 INFO:tasks.workunit.client.1.smithi132.stdout:3/297: write d8/d13/d19/f41 [1915577,24246] 0 2024-02-19T19:04:52.445 INFO:tasks.workunit.client.1.smithi132.stdout:5/312: chown d8/l22 25 1 2024-02-19T19:04:52.446 INFO:tasks.workunit.client.1.smithi132.stdout:5/313: write d8/d23/f4d [899448,14686] 0 2024-02-19T19:04:52.451 INFO:tasks.workunit.client.1.smithi132.stdout:2/331: read d0/d4/fb [2638171,115082] 0 2024-02-19T19:04:52.454 INFO:tasks.workunit.client.1.smithi132.stdout:4/315: symlink d4/d12/d28/l6e 0 2024-02-19T19:04:52.456 INFO:tasks.workunit.client.1.smithi132.stdout:9/441: mknod d12/d8d/d3a/d40/d4e/d77/d89/ca0 0 2024-02-19T19:04:52.458 INFO:tasks.workunit.client.1.smithi132.stdout:8/326: chown d4/dc/d1b/c55 0 1 2024-02-19T19:04:52.458 INFO:tasks.workunit.client.1.smithi132.stdout:6/282: dwrite d0/d3c/d46/d5a/f26 [0,4194304] 0 2024-02-19T19:04:52.459 INFO:tasks.workunit.client.1.smithi132.stdout:6/283: chown d0/db/d25/f4f 19680468 1 2024-02-19T19:04:52.459 INFO:tasks.workunit.client.1.smithi132.stdout:8/327: chown d4/dc/d1b/d38/c58 1146068363 1 2024-02-19T19:04:52.461 INFO:tasks.workunit.client.1.smithi132.stdout:3/298: creat d8/d13/d2b/d52/d34/d50/f5b x:0 0 0 2024-02-19T19:04:52.465 INFO:tasks.workunit.client.1.smithi132.stdout:5/314: symlink d8/d1a/d35/l61 0 2024-02-19T19:04:52.467 INFO:tasks.workunit.client.1.smithi132.stdout:9/442: dwrite d12/d1b/f16 [0,4194304] 0 2024-02-19T19:04:52.470 INFO:tasks.workunit.client.1.smithi132.stdout:7/390: getdents d1b/d1e/d2b/d45 0 2024-02-19T19:04:52.471 INFO:tasks.workunit.client.1.smithi132.stdout:8/328: creat d4/dc/d1b/d20/f6b x:0 0 0 2024-02-19T19:04:52.471 INFO:tasks.workunit.client.1.smithi132.stdout:9/443: write f5 [2234146,51247] 0 2024-02-19T19:04:52.474 INFO:tasks.workunit.client.1.smithi132.stdout:4/316: mknod d4/c6f 0 2024-02-19T19:04:52.476 INFO:tasks.workunit.client.1.smithi132.stdout:1/308: dwrite d1/f48 [0,4194304] 0 2024-02-19T19:04:52.478 INFO:tasks.workunit.client.1.smithi132.stdout:8/329: fsync d4/dc/d1b/d1d/f21 0 2024-02-19T19:04:52.478 INFO:tasks.workunit.client.1.smithi132.stdout:6/284: rename d0/d3c/d46/d5a/f36 to d0/db/d25/d3a/f6d 0 2024-02-19T19:04:52.480 INFO:tasks.workunit.client.1.smithi132.stdout:9/444: truncate d12/f99 652425 0 2024-02-19T19:04:52.480 INFO:tasks.workunit.client.1.smithi132.stdout:2/332: dwrite d0/d19/f25 [0,4194304] 0 2024-02-19T19:04:52.481 INFO:tasks.workunit.client.1.smithi132.stdout:3/299: dread d8/d13/d19/f41 [0,4194304] 0 2024-02-19T19:04:52.484 INFO:tasks.workunit.client.1.smithi132.stdout:3/300: readlink d8/lb 0 2024-02-19T19:04:52.485 INFO:tasks.workunit.client.1.smithi132.stdout:9/445: chown d12/d1b/l59 0 1 2024-02-19T19:04:52.486 INFO:tasks.workunit.client.1.smithi132.stdout:5/315: creat d8/f62 x:0 0 0 2024-02-19T19:04:52.489 INFO:tasks.workunit.client.1.smithi132.stdout:1/309: dwrite d1/d1e/d29/d2c/f59 [0,4194304] 0 2024-02-19T19:04:52.493 INFO:tasks.workunit.client.1.smithi132.stdout:4/317: creat d4/dc/d37/d66/f70 x:0 0 0 2024-02-19T19:04:52.496 INFO:tasks.workunit.client.1.smithi132.stdout:7/391: link d1b/d46/f4d d1b/d1e/d2b/f80 0 2024-02-19T19:04:52.499 INFO:tasks.workunit.client.1.smithi132.stdout:2/333: mkdir d0/d64/d6d 0 2024-02-19T19:04:52.500 INFO:tasks.workunit.client.1.smithi132.stdout:1/310: dwrite d1/f18 [4194304,4194304] 0 2024-02-19T19:04:52.501 INFO:tasks.workunit.client.1.smithi132.stdout:9/446: symlink d12/d8d/d3a/d40/d4e/la1 0 2024-02-19T19:04:52.501 INFO:tasks.workunit.client.1.smithi132.stdout:7/392: dread d1b/d1e/d2b/d62/f67 [0,4194304] 0 2024-02-19T19:04:52.502 INFO:tasks.workunit.client.1.smithi132.stdout:5/316: creat d8/da/d5a/f63 x:0 0 0 2024-02-19T19:04:52.503 INFO:tasks.workunit.client.1.smithi132.stdout:9/447: chown d12/d8d/d3a/d40/f66 2022760 1 2024-02-19T19:04:52.503 INFO:tasks.workunit.client.1.smithi132.stdout:7/393: write d1b/d1e/f37 [1472755,81939] 0 2024-02-19T19:04:52.504 INFO:tasks.workunit.client.1.smithi132.stdout:3/301: getdents d8/d13/d57 0 2024-02-19T19:04:52.510 INFO:tasks.workunit.client.1.smithi132.stdout:7/394: dwrite d1b/d1e/d2b/f5d [0,4194304] 0 2024-02-19T19:04:52.512 INFO:tasks.workunit.client.1.smithi132.stdout:9/448: chown d12/d1b/f1c 855229862 1 2024-02-19T19:04:52.516 INFO:tasks.workunit.client.1.smithi132.stdout:2/334: mkdir d0/d19/d6e 0 2024-02-19T19:04:52.519 INFO:tasks.workunit.client.1.smithi132.stdout:4/318: rename d4/dc to d4/d27/d31/d71 0 2024-02-19T19:04:52.521 INFO:tasks.workunit.client.1.smithi132.stdout:6/285: getdents d0/db/d25/d3a/d5d 0 2024-02-19T19:04:52.528 INFO:tasks.workunit.client.1.smithi132.stdout:6/286: mknod d0/db/d25/d3a/d3d/c6e 0 2024-02-19T19:04:52.528 INFO:tasks.workunit.client.1.smithi132.stdout:1/311: getdents d1/d1e/d29/d2c/d43 0 2024-02-19T19:04:52.529 INFO:tasks.workunit.client.1.smithi132.stdout:6/287: write d0/d3c/f42 [2744835,73770] 0 2024-02-19T19:04:52.530 INFO:tasks.workunit.client.1.smithi132.stdout:4/319: mkdir d4/d27/d31/d32/d51/d72 0 2024-02-19T19:04:52.532 INFO:tasks.workunit.client.1.smithi132.stdout:5/317: getdents d8/da/db 0 2024-02-19T19:04:52.533 INFO:tasks.workunit.client.1.smithi132.stdout:5/318: chown d8/d1a/f26 997650 1 2024-02-19T19:04:52.534 INFO:tasks.workunit.client.1.smithi132.stdout:5/319: stat d8/da/db/l3e 0 2024-02-19T19:04:52.536 INFO:tasks.workunit.client.1.smithi132.stdout:6/288: dwrite d0/db/d25/f34 [4194304,4194304] 0 2024-02-19T19:04:52.537 INFO:tasks.workunit.client.1.smithi132.stdout:4/320: rmdir d4/d27/d31/d32/d51 39 2024-02-19T19:04:52.540 INFO:tasks.workunit.client.1.smithi132.stdout:5/320: mkdir d8/d1a/d64 0 2024-02-19T19:04:52.541 INFO:tasks.workunit.client.1.smithi132.stdout:2/335: link d0/d24/c3d d0/d24/c6f 0 2024-02-19T19:04:52.546 INFO:tasks.workunit.client.1.smithi132.stdout:5/321: dwrite d8/d23/f43 [0,4194304] 0 2024-02-19T19:04:52.548 INFO:tasks.workunit.client.1.smithi132.stdout:2/336: rmdir d0/d4/df 39 2024-02-19T19:04:52.548 INFO:tasks.workunit.client.1.smithi132.stdout:1/312: creat d1/d1e/f5b x:0 0 0 2024-02-19T19:04:52.549 INFO:tasks.workunit.client.1.smithi132.stdout:1/313: write d1/f18 [3975426,44214] 0 2024-02-19T19:04:52.551 INFO:tasks.workunit.client.1.smithi132.stdout:4/321: mkdir d4/d27/d73 0 2024-02-19T19:04:52.552 INFO:tasks.workunit.client.1.smithi132.stdout:5/322: creat d8/d23/d3b/f65 x:0 0 0 2024-02-19T19:04:52.553 INFO:tasks.workunit.client.1.smithi132.stdout:4/322: chown d4/d27/d31/d71/d37 111392 1 2024-02-19T19:04:52.555 INFO:tasks.workunit.client.1.smithi132.stdout:2/337: chown d0/d24/d58/d53/c5e 4 1 2024-02-19T19:04:52.561 INFO:tasks.workunit.client.1.smithi132.stdout:6/289: dread d0/f10 [4194304,4194304] 0 2024-02-19T19:04:52.564 INFO:tasks.workunit.client.1.smithi132.stdout:2/338: symlink d0/d19/d6a/l70 0 2024-02-19T19:04:52.566 INFO:tasks.workunit.client.1.smithi132.stdout:6/290: rename d0/db/d25/f4f to d0/db/d25/d2f/f6f 0 2024-02-19T19:04:52.567 INFO:tasks.workunit.client.1.smithi132.stdout:2/339: mkdir d0/d24/d58/d71 0 2024-02-19T19:04:52.570 INFO:tasks.workunit.client.1.smithi132.stdout:4/323: getdents d4/d19 0 2024-02-19T19:04:52.616 INFO:tasks.workunit.client.1.smithi132.stdout:0/333: sync 2024-02-19T19:04:52.630 INFO:tasks.workunit.client.1.smithi132.stdout:1/314: link d1/d1e/d29/d4e/f4f d1/d1e/f5c 0 2024-02-19T19:04:52.630 INFO:tasks.workunit.client.1.smithi132.stdout:5/323: dread d8/da/db/f1b [0,4194304] 0 2024-02-19T19:04:52.631 INFO:tasks.workunit.client.1.smithi132.stdout:6/291: write d0/d3c/d46/d5a/f18 [2399609,116711] 0 2024-02-19T19:04:52.631 INFO:tasks.workunit.client.1.smithi132.stdout:4/324: stat d4/d12/d28/l6e 0 2024-02-19T19:04:52.632 INFO:tasks.workunit.client.1.smithi132.stdout:6/292: stat d0/dc 0 2024-02-19T19:04:52.632 INFO:tasks.workunit.client.1.smithi132.stdout:0/334: write d2/dc/fe [5289527,80823] 0 2024-02-19T19:04:52.633 INFO:tasks.workunit.client.1.smithi132.stdout:2/340: getdents d0 0 2024-02-19T19:04:52.633 INFO:tasks.workunit.client.1.smithi132.stdout:4/325: chown d4/d27/d31/d71/d37/d66 1821602878 1 2024-02-19T19:04:52.635 INFO:tasks.workunit.client.1.smithi132.stdout:5/324: mknod d8/d23/d5d/c66 0 2024-02-19T19:04:52.636 INFO:tasks.workunit.client.1.smithi132.stdout:0/335: write d2/f2b [1518186,84141] 0 2024-02-19T19:04:52.638 INFO:tasks.workunit.client.1.smithi132.stdout:6/293: dread d0/db/d25/d2f/f4a [0,4194304] 0 2024-02-19T19:04:52.638 INFO:tasks.workunit.client.1.smithi132.stdout:2/341: rmdir d0/d4/d11/d2e 39 2024-02-19T19:04:52.640 INFO:tasks.workunit.client.1.smithi132.stdout:1/315: dwrite d1/d1e/d29/f3f [0,4194304] 0 2024-02-19T19:04:52.642 INFO:tasks.workunit.client.1.smithi132.stdout:1/316: write d1/d1e/d29/d2c/f59 [2440270,119188] 0 2024-02-19T19:04:52.646 INFO:tasks.workunit.client.1.smithi132.stdout:2/342: dwrite d0/d4/f22 [0,4194304] 0 2024-02-19T19:04:52.646 INFO:tasks.workunit.client.1.smithi132.stdout:2/343: chown d0/d3c/f60 150571 1 2024-02-19T19:04:52.655 INFO:tasks.workunit.client.1.smithi132.stdout:0/336: mkdir d2/d6e 0 2024-02-19T19:04:52.660 INFO:tasks.workunit.client.1.smithi132.stdout:1/317: creat d1/d1e/d29/d2c/d43/f5d x:0 0 0 2024-02-19T19:04:52.666 INFO:tasks.workunit.client.1.smithi132.stdout:1/318: dwrite d1/f53 [0,4194304] 0 2024-02-19T19:04:52.668 INFO:tasks.workunit.client.1.smithi132.stdout:6/294: rename d0/db/d25/d3a/f6d to d0/f70 0 2024-02-19T19:04:52.670 INFO:tasks.workunit.client.1.smithi132.stdout:5/325: getdents d8/d1a/d35 0 2024-02-19T19:04:52.677 INFO:tasks.workunit.client.1.smithi132.stdout:0/337: dwrite d2/f6 [0,4194304] 0 2024-02-19T19:04:52.678 INFO:tasks.workunit.client.1.smithi132.stdout:0/338: readlink d2/dc/l55 0 2024-02-19T19:04:52.679 INFO:tasks.workunit.client.1.smithi132.stdout:2/344: dread d0/d19/d1e/f45 [0,4194304] 0 2024-02-19T19:04:52.680 INFO:tasks.workunit.client.1.smithi132.stdout:5/326: rename d8/d23/d29/f31 to d8/d23/d5d/f67 0 2024-02-19T19:04:52.682 INFO:tasks.workunit.client.1.smithi132.stdout:1/319: link d1/f18 d1/d1e/d29/f5e 0 2024-02-19T19:04:52.690 INFO:tasks.workunit.client.1.smithi132.stdout:6/295: creat d0/db/d25/d2f/f71 x:0 0 0 2024-02-19T19:04:52.690 INFO:tasks.workunit.client.1.smithi132.stdout:5/327: symlink d8/d1a/d35/l68 0 2024-02-19T19:04:52.690 INFO:tasks.workunit.client.1.smithi132.stdout:2/345: chown d0/d4/df/f51 61069 1 2024-02-19T19:04:52.691 INFO:tasks.workunit.client.1.smithi132.stdout:1/320: creat d1/d1e/d29/d4e/f5f x:0 0 0 2024-02-19T19:04:52.693 INFO:tasks.workunit.client.1.smithi132.stdout:6/296: fdatasync d0/d3c/d46/d5a/f20 0 2024-02-19T19:04:52.696 INFO:tasks.workunit.client.1.smithi132.stdout:2/346: mkdir d0/d19/d72 0 2024-02-19T19:04:52.699 INFO:tasks.workunit.client.1.smithi132.stdout:0/339: getdents d2 0 2024-02-19T19:04:52.700 INFO:tasks.workunit.client.1.smithi132.stdout:1/321: dwrite d1/f5 [4194304,4194304] 0 2024-02-19T19:04:52.700 INFO:tasks.workunit.client.1.smithi132.stdout:5/328: link d8/d23/f50 d8/d23/d29/d27/f69 0 2024-02-19T19:04:52.703 INFO:tasks.workunit.client.1.smithi132.stdout:0/340: dread - d2/dc/d4a/f5e zero size 2024-02-19T19:04:52.705 INFO:tasks.workunit.client.1.smithi132.stdout:6/297: dwrite d0/d3c/d22/f64 [0,4194304] 0 2024-02-19T19:04:52.711 INFO:tasks.workunit.client.1.smithi132.stdout:2/347: chown d0/d4/d11/d2e/d5a/f5b 79979 1 2024-02-19T19:04:52.715 INFO:tasks.workunit.client.1.smithi132.stdout:6/298: symlink d0/d3c/d22/l72 0 2024-02-19T19:04:52.718 INFO:tasks.workunit.client.1.smithi132.stdout:5/329: link d8/fe d8/d1a/f6a 0 2024-02-19T19:04:52.718 INFO:tasks.workunit.client.1.smithi132.stdout:5/330: chown d8/d1a/c3a 22983 1 2024-02-19T19:04:52.719 INFO:tasks.workunit.client.1.smithi132.stdout:5/331: dread - d8/f62 zero size 2024-02-19T19:04:52.721 INFO:tasks.workunit.client.1.smithi132.stdout:2/348: symlink d0/l73 0 2024-02-19T19:04:52.724 INFO:tasks.workunit.client.1.smithi132.stdout:5/332: creat d8/d23/d5d/f6b x:0 0 0 2024-02-19T19:04:52.728 INFO:tasks.workunit.client.1.smithi132.stdout:5/333: rename d8/f4c to d8/d1a/f6c 0 2024-02-19T19:04:52.729 INFO:tasks.workunit.client.1.smithi132.stdout:5/334: readlink d8/d23/d29/l2f 0 2024-02-19T19:04:52.730 INFO:tasks.workunit.client.1.smithi132.stdout:6/299: creat d0/f73 x:0 0 0 2024-02-19T19:04:52.730 INFO:tasks.workunit.client.1.smithi132.stdout:0/341: write d2/dc/df/f2d [1035505,25017] 0 2024-02-19T19:04:52.732 INFO:tasks.workunit.client.1.smithi132.stdout:5/335: truncate d8/d1a/f5e 19314 0 2024-02-19T19:04:52.734 INFO:tasks.workunit.client.1.smithi132.stdout:1/322: dwrite d1/f18 [4194304,4194304] 0 2024-02-19T19:04:52.735 INFO:tasks.workunit.client.1.smithi132.stdout:6/300: creat d0/db/d25/d3a/d3d/f74 x:0 0 0 2024-02-19T19:04:52.736 INFO:tasks.workunit.client.1.smithi132.stdout:0/342: creat d2/dc/d4a/d25/d43/f6f x:0 0 0 2024-02-19T19:04:52.738 INFO:tasks.workunit.client.1.smithi132.stdout:5/336: symlink d8/d23/d29/d27/l6d 0 2024-02-19T19:04:52.739 INFO:tasks.workunit.client.1.smithi132.stdout:5/337: chown d8/d1a/d35/f4b 869537 1 2024-02-19T19:04:52.739 INFO:tasks.workunit.client.1.smithi132.stdout:5/338: chown d8/d1a/d35 5935 1 2024-02-19T19:04:52.739 INFO:tasks.workunit.client.1.smithi132.stdout:5/339: chown d8/da/db/l12 37 1 2024-02-19T19:04:52.744 INFO:tasks.workunit.client.1.smithi132.stdout:0/343: rename d2/c11 to d2/dc/d4a/d25/d43/c70 0 2024-02-19T19:04:52.744 INFO:tasks.workunit.client.1.smithi132.stdout:5/340: creat d8/da/d5a/f6e x:0 0 0 2024-02-19T19:04:52.747 INFO:tasks.workunit.client.1.smithi132.stdout:1/323: mknod d1/d3a/c60 0 2024-02-19T19:04:52.747 INFO:tasks.workunit.client.1.smithi132.stdout:6/301: creat d0/dc/f75 x:0 0 0 2024-02-19T19:04:52.748 INFO:tasks.workunit.client.1.smithi132.stdout:6/302: dread - d0/db/f58 zero size 2024-02-19T19:04:52.750 INFO:tasks.workunit.client.1.smithi132.stdout:6/303: readlink d0/l7 0 2024-02-19T19:04:52.751 INFO:tasks.workunit.client.1.smithi132.stdout:1/324: mkdir d1/d61 0 2024-02-19T19:04:52.753 INFO:tasks.workunit.client.1.smithi132.stdout:5/341: creat d8/d23/f6f x:0 0 0 2024-02-19T19:04:52.756 INFO:tasks.workunit.client.1.smithi132.stdout:1/325: mkdir d1/d1e/d29/d2c/d62 0 2024-02-19T19:04:52.759 INFO:tasks.workunit.client.1.smithi132.stdout:6/304: dwrite d0/f41 [0,4194304] 0 2024-02-19T19:04:52.759 INFO:tasks.workunit.client.1.smithi132.stdout:5/342: creat d8/d23/d3b/f70 x:0 0 0 2024-02-19T19:04:52.760 INFO:tasks.workunit.client.1.smithi132.stdout:5/343: dread - d8/d23/d29/d3f/f59 zero size 2024-02-19T19:04:52.761 INFO:tasks.workunit.client.1.smithi132.stdout:1/326: stat d1/c12 0 2024-02-19T19:04:52.762 INFO:tasks.workunit.client.1.smithi132.stdout:6/305: truncate d0/db/d25/d2f/f31 199535 0 2024-02-19T19:04:52.765 INFO:tasks.workunit.client.1.smithi132.stdout:5/344: mkdir d8/d2e/d71 0 2024-02-19T19:04:52.771 INFO:tasks.workunit.client.1.smithi132.stdout:6/306: symlink d0/d3c/l76 0 2024-02-19T19:04:52.776 INFO:tasks.workunit.client.1.smithi132.stdout:6/307: dread d0/d3c/f39 [0,4194304] 0 2024-02-19T19:04:52.777 INFO:tasks.workunit.client.1.smithi132.stdout:2/349: dread d0/d4/f1f [0,4194304] 0 2024-02-19T19:04:52.778 INFO:tasks.workunit.client.1.smithi132.stdout:5/345: creat d8/d1a/d64/f72 x:0 0 0 2024-02-19T19:04:52.778 INFO:tasks.workunit.client.1.smithi132.stdout:0/344: write d2/dc/df/d15/f24 [3298692,4012] 0 2024-02-19T19:04:52.779 INFO:tasks.workunit.client.1.smithi132.stdout:5/346: write d8/d23/d3b/f56 [5193071,99232] 0 2024-02-19T19:04:52.780 INFO:tasks.workunit.client.1.smithi132.stdout:0/345: dread - d2/dc/d4a/d25/d43/f6f zero size 2024-02-19T19:04:52.780 INFO:tasks.workunit.client.1.smithi132.stdout:6/308: mkdir d0/db/d25/d2f/d77 0 2024-02-19T19:04:52.781 INFO:tasks.workunit.client.1.smithi132.stdout:5/347: readlink d8/da/db/l11 0 2024-02-19T19:04:52.782 INFO:tasks.workunit.client.1.smithi132.stdout:2/350: dread - d0/d19/d1e/f62 zero size 2024-02-19T19:04:52.782 INFO:tasks.workunit.client.1.smithi132.stdout:6/309: dread - d0/db/f58 zero size 2024-02-19T19:04:52.783 INFO:tasks.workunit.client.1.smithi132.stdout:1/327: getdents d1 0 2024-02-19T19:04:52.787 INFO:tasks.workunit.client.1.smithi132.stdout:8/330: sync 2024-02-19T19:04:52.788 INFO:tasks.workunit.client.1.smithi132.stdout:0/346: rename d2/dc/d4a/f5e to d2/dc/df/f71 0 2024-02-19T19:04:52.789 INFO:tasks.workunit.client.1.smithi132.stdout:5/348: symlink d8/d23/d5d/l73 0 2024-02-19T19:04:52.790 INFO:tasks.workunit.client.1.smithi132.stdout:1/328: symlink d1/d1e/d29/l63 0 2024-02-19T19:04:52.795 INFO:tasks.workunit.client.1.smithi132.stdout:0/347: chown d2/d3e/c4f 71398 1 2024-02-19T19:04:52.795 INFO:tasks.workunit.client.1.smithi132.stdout:5/349: dwrite d8/d23/f4d [0,4194304] 0 2024-02-19T19:04:52.799 INFO:tasks.workunit.client.1.smithi132.stdout:6/310: link d0/f41 d0/db/d25/d38/f78 0 2024-02-19T19:04:52.808 INFO:tasks.workunit.client.1.smithi132.stdout:6/311: mkdir d0/d3c/d22/d79 0 2024-02-19T19:04:52.809 INFO:tasks.workunit.client.1.smithi132.stdout:8/331: dwrite d4/f27 [0,4194304] 0 2024-02-19T19:04:52.811 INFO:tasks.workunit.client.1.smithi132.stdout:8/332: fsync d4/fb 0 2024-02-19T19:04:52.811 INFO:tasks.workunit.client.1.smithi132.stdout:6/312: chown d0/d3c/d22/c47 4774 1 2024-02-19T19:04:52.814 INFO:tasks.workunit.client.1.smithi132.stdout:5/350: dwrite d8/da/db/f1b [0,4194304] 0 2024-02-19T19:04:52.814 INFO:tasks.workunit.client.1.smithi132.stdout:0/348: dread d2/f2b [0,4194304] 0 2024-02-19T19:04:52.819 INFO:tasks.workunit.client.1.smithi132.stdout:6/313: symlink d0/db/d25/d38/l7a 0 2024-02-19T19:04:52.821 INFO:tasks.workunit.client.1.smithi132.stdout:5/351: dwrite d8/d1a/f26 [0,4194304] 0 2024-02-19T19:04:52.822 INFO:tasks.workunit.client.1.smithi132.stdout:8/333: rename d4/dc/d1b/d20/d23/l30 to d4/dc/d1b/d1d/l6c 0 2024-02-19T19:04:52.825 INFO:tasks.workunit.client.1.smithi132.stdout:6/314: creat d0/d3c/d46/f7b x:0 0 0 2024-02-19T19:04:52.829 INFO:tasks.workunit.client.1.smithi132.stdout:8/334: creat d4/dc/d1b/d38/d69/f6d x:0 0 0 2024-02-19T19:04:52.836 INFO:tasks.workunit.client.1.smithi132.stdout:6/315: truncate d0/d3c/d46/d5a/f20 1972174 0 2024-02-19T19:04:52.838 INFO:tasks.workunit.client.1.smithi132.stdout:6/316: fsync d0/d3c/d46/d5a/f29 0 2024-02-19T19:04:52.840 INFO:tasks.workunit.client.1.smithi132.stdout:1/329: truncate d1/d1e/d29/d2c/f34 1963082 0 2024-02-19T19:04:52.840 INFO:tasks.workunit.client.1.smithi132.stdout:8/335: dwrite d4/dc/f3f [0,4194304] 0 2024-02-19T19:04:52.842 INFO:tasks.workunit.client.1.smithi132.stdout:6/317: unlink d0/dc/l1a 0 2024-02-19T19:04:52.855 INFO:tasks.workunit.client.1.smithi132.stdout:6/318: mkdir d0/db/d25/d3a/d4b/d7c 0 2024-02-19T19:04:52.855 INFO:tasks.workunit.client.1.smithi132.stdout:5/352: write d8/d23/f50 [1596554,87823] 0 2024-02-19T19:04:52.856 INFO:tasks.workunit.client.1.smithi132.stdout:6/319: chown d0/db/d25/d2f 91413 1 2024-02-19T19:04:52.856 INFO:tasks.workunit.client.1.smithi132.stdout:0/349: truncate d2/dc/df/d15/f22 1361310 0 2024-02-19T19:04:52.857 INFO:tasks.workunit.client.1.smithi132.stdout:5/353: fsync d8/d23/f45 0 2024-02-19T19:04:52.857 INFO:tasks.workunit.client.1.smithi132.stdout:5/354: chown d8/d23/f43 1 1 2024-02-19T19:04:52.858 INFO:tasks.workunit.client.1.smithi132.stdout:5/355: stat d8/d23 0 2024-02-19T19:04:52.860 INFO:tasks.workunit.client.1.smithi132.stdout:0/350: creat d2/dc/df/d15/d2f/d3f/d63/d6b/f72 x:0 0 0 2024-02-19T19:04:52.862 INFO:tasks.workunit.client.1.smithi132.stdout:5/356: getdents d8/da/d5a/d51 0 2024-02-19T19:04:52.863 INFO:tasks.workunit.client.1.smithi132.stdout:6/320: getdents d0/db/d25/d2f 0 2024-02-19T19:04:52.869 INFO:tasks.workunit.client.1.smithi132.stdout:0/351: dread d2/dc/df/d15/d2f/f40 [0,4194304] 0 2024-02-19T19:04:52.869 INFO:tasks.workunit.client.1.smithi132.stdout:8/336: write d4/dc/d1b/d33/f34 [701445,3707] 0 2024-02-19T19:04:52.869 INFO:tasks.workunit.client.1.smithi132.stdout:1/330: write d1/d1e/d29/d4e/f4f [2119878,43042] 0 2024-02-19T19:04:52.873 INFO:tasks.workunit.client.1.smithi132.stdout:6/321: creat d0/d3c/d46/d5a/f7d x:0 0 0 2024-02-19T19:04:52.873 INFO:tasks.workunit.client.1.smithi132.stdout:1/331: truncate d1/d1e/d29/d2c/f59 5046410 0 2024-02-19T19:04:52.875 INFO:tasks.workunit.client.1.smithi132.stdout:6/322: truncate d0/dc/f2d 318728 0 2024-02-19T19:04:52.876 INFO:tasks.workunit.client.1.smithi132.stdout:0/352: dwrite d2/dc/df/d15/f24 [0,4194304] 0 2024-02-19T19:04:52.878 INFO:tasks.workunit.client.1.smithi132.stdout:8/337: creat d4/dc/d1b/d20/f6e x:0 0 0 2024-02-19T19:04:52.882 INFO:tasks.workunit.client.1.smithi132.stdout:1/332: mkdir d1/d1e/d29/d52/d64 0 2024-02-19T19:04:52.882 INFO:tasks.workunit.client.1.smithi132.stdout:1/333: fsync d1/f5 0 2024-02-19T19:04:52.886 INFO:tasks.workunit.client.1.smithi132.stdout:8/338: rename d4/dc/d1b/d54/d5e/f4a to d4/dc/d42/f6f 0 2024-02-19T19:04:52.889 INFO:tasks.workunit.client.1.smithi132.stdout:0/353: creat d2/dc/d4a/d25/d43/d5f/f73 x:0 0 0 2024-02-19T19:04:52.890 INFO:tasks.workunit.client.1.smithi132.stdout:1/334: creat d1/d1e/d29/d52/f65 x:0 0 0 2024-02-19T19:04:52.891 INFO:tasks.workunit.client.1.smithi132.stdout:1/335: write d1/f53 [3543543,71895] 0 2024-02-19T19:04:52.892 INFO:tasks.workunit.client.1.smithi132.stdout:8/339: symlink d4/dc/d1b/d20/d23/l70 0 2024-02-19T19:04:52.900 INFO:tasks.workunit.client.1.smithi132.stdout:1/336: dwrite d1/f48 [0,4194304] 0 2024-02-19T19:04:52.901 INFO:tasks.workunit.client.1.smithi132.stdout:3/302: sync 2024-02-19T19:04:52.901 INFO:tasks.workunit.client.1.smithi132.stdout:7/395: sync 2024-02-19T19:04:52.903 INFO:tasks.workunit.client.1.smithi132.stdout:0/354: dwrite d2/f2b [0,4194304] 0 2024-02-19T19:04:52.907 INFO:tasks.workunit.client.1.smithi132.stdout:5/357: dread d8/da/db/f34 [0,4194304] 0 2024-02-19T19:04:52.911 INFO:tasks.workunit.client.1.smithi132.stdout:8/340: link d4/ce d4/dc/d1b/d38/d69/c71 0 2024-02-19T19:04:52.912 INFO:tasks.workunit.client.1.smithi132.stdout:0/355: mkdir d2/dc/d4a/d74 0 2024-02-19T19:04:52.913 INFO:tasks.workunit.client.1.smithi132.stdout:3/303: dwrite d8/fd [0,4194304] 0 2024-02-19T19:04:52.913 INFO:tasks.workunit.client.1.smithi132.stdout:1/337: dread d1/f5 [4194304,4194304] 0 2024-02-19T19:04:52.916 INFO:tasks.workunit.client.1.smithi132.stdout:3/304: read - d8/f5a zero size 2024-02-19T19:04:52.921 INFO:tasks.workunit.client.1.smithi132.stdout:5/358: dwrite d8/f62 [0,4194304] 0 2024-02-19T19:04:52.928 INFO:tasks.workunit.client.1.smithi132.stdout:5/359: rename d8/da/d5a/f6e to d8/d1a/d35/f74 0 2024-02-19T19:04:52.933 INFO:tasks.workunit.client.1.smithi132.stdout:8/341: dwrite d4/dc/d1b/d33/f5b [0,4194304] 0 2024-02-19T19:04:52.933 INFO:tasks.workunit.client.1.smithi132.stdout:7/396: dwrite d1b/d1e/d2b/f51 [0,4194304] 0 2024-02-19T19:04:52.934 INFO:tasks.workunit.client.1.smithi132.stdout:5/360: mknod d8/d1a/d35/c75 0 2024-02-19T19:04:52.935 INFO:tasks.workunit.client.1.smithi132.stdout:5/361: fsync d8/d23/d3b/f65 0 2024-02-19T19:04:52.936 INFO:tasks.workunit.client.1.smithi132.stdout:8/342: readlink d4/dc/d1b/l25 0 2024-02-19T19:04:52.937 INFO:tasks.workunit.client.1.smithi132.stdout:0/356: write d2/dc/df/d15/d34/f41 [1613482,44307] 0 2024-02-19T19:04:52.938 INFO:tasks.workunit.client.1.smithi132.stdout:8/343: chown f2 338 1 2024-02-19T19:04:52.944 INFO:tasks.workunit.client.1.smithi132.stdout:1/338: truncate d1/d1e/d29/f3f 2728962 0 2024-02-19T19:04:52.945 INFO:tasks.workunit.client.1.smithi132.stdout:3/305: dwrite d8/d13/d2f/f4d [0,4194304] 0 2024-02-19T19:04:52.946 INFO:tasks.workunit.client.1.smithi132.stdout:3/306: chown d8/fd 33750055 1 2024-02-19T19:04:52.948 INFO:tasks.workunit.client.1.smithi132.stdout:8/344: symlink d4/l72 0 2024-02-19T19:04:52.953 INFO:tasks.workunit.client.1.smithi132.stdout:3/307: rmdir d8/d13/d2b/d52 39 2024-02-19T19:04:52.953 INFO:tasks.workunit.client.1.smithi132.stdout:8/345: write d4/dc/d1b/d33/f49 [542847,51595] 0 2024-02-19T19:04:52.955 INFO:tasks.workunit.client.1.smithi132.stdout:1/339: dwrite d1/d1e/d29/d52/f65 [0,4194304] 0 2024-02-19T19:04:52.956 INFO:tasks.workunit.client.1.smithi132.stdout:7/397: link d1b/d1e/f3d d1b/d1e/d2b/d62/d53/f81 0 2024-02-19T19:04:52.959 INFO:tasks.workunit.client.1.smithi132.stdout:8/346: fsync d4/fd 0 2024-02-19T19:04:52.963 INFO:tasks.workunit.client.1.smithi132.stdout:3/308: mknod d8/d38/d44/c5c 0 2024-02-19T19:04:52.963 INFO:tasks.workunit.client.1.smithi132.stdout:1/340: symlink d1/d1e/d29/d2c/l66 0 2024-02-19T19:04:52.967 INFO:tasks.workunit.client.1.smithi132.stdout:8/347: dwrite d4/dc/d1b/d33/f34 [0,4194304] 0 2024-02-19T19:04:52.968 INFO:tasks.workunit.client.1.smithi132.stdout:8/348: rename d4/dc/d1b/d33 to d4/dc/d1b/d33/d73 22 2024-02-19T19:04:52.969 INFO:tasks.workunit.client.1.smithi132.stdout:1/341: creat d1/d3a/d50/f67 x:0 0 0 2024-02-19T19:04:52.973 INFO:tasks.workunit.client.1.smithi132.stdout:1/342: symlink d1/d1e/l68 0 2024-02-19T19:04:52.974 INFO:tasks.workunit.client.1.smithi132.stdout:5/362: dread d8/d1a/f6a [0,4194304] 0 2024-02-19T19:04:52.977 INFO:tasks.workunit.client.1.smithi132.stdout:8/349: dwrite f2 [0,4194304] 0 2024-02-19T19:04:52.977 INFO:tasks.workunit.client.1.smithi132.stdout:1/343: creat d1/d1e/d29/f69 x:0 0 0 2024-02-19T19:04:52.982 INFO:tasks.workunit.client.1.smithi132.stdout:8/350: dread d4/f27 [0,4194304] 0 2024-02-19T19:04:52.984 INFO:tasks.workunit.client.1.smithi132.stdout:5/363: symlink d8/d23/l76 0 2024-02-19T19:04:52.985 INFO:tasks.workunit.client.1.smithi132.stdout:7/398: dread d1b/d1e/f38 [0,4194304] 0 2024-02-19T19:04:52.985 INFO:tasks.workunit.client.1.smithi132.stdout:5/364: write d8/f62 [510416,63956] 0 2024-02-19T19:04:52.986 INFO:tasks.workunit.client.1.smithi132.stdout:1/344: mkdir d1/d1e/d6a 0 2024-02-19T19:04:52.988 INFO:tasks.workunit.client.1.smithi132.stdout:8/351: mkdir d4/dc/d1b/d38/d69/d74 0 2024-02-19T19:04:52.989 INFO:tasks.workunit.client.1.smithi132.stdout:8/352: dread - d4/dc/d1b/d20/f6b zero size 2024-02-19T19:04:52.991 INFO:tasks.workunit.client.1.smithi132.stdout:7/399: fsync d1b/d1e/d2b/f80 0 2024-02-19T19:04:52.994 INFO:tasks.workunit.client.1.smithi132.stdout:1/345: mkdir d1/d1e/d29/d2c/d62/d6b 0 2024-02-19T19:04:52.996 INFO:tasks.workunit.client.1.smithi132.stdout:1/346: read - d1/d1e/d29/f3c zero size 2024-02-19T19:04:52.997 INFO:tasks.workunit.client.1.smithi132.stdout:8/353: dwrite d4/dc/d1b/d1d/f21 [0,4194304] 0 2024-02-19T19:04:52.997 INFO:tasks.workunit.client.1.smithi132.stdout:7/400: link d1b/l34 d1b/d5a/d63/l82 0 2024-02-19T19:04:52.998 INFO:tasks.workunit.client.1.smithi132.stdout:1/347: truncate d1/d1e/f5c 2227733 0 2024-02-19T19:04:52.999 INFO:tasks.workunit.client.1.smithi132.stdout:8/354: dread - d4/dc/d1b/d54/d5e/f63 zero size 2024-02-19T19:04:53.001 INFO:tasks.workunit.client.1.smithi132.stdout:5/365: dwrite d8/d23/d29/d27/f57 [0,4194304] 0 2024-02-19T19:04:53.003 INFO:tasks.workunit.client.1.smithi132.stdout:8/355: fdatasync d4/d2d/f46 0 2024-02-19T19:04:53.004 INFO:tasks.workunit.client.1.smithi132.stdout:1/348: truncate d1/d1e/d29/d2c/d43/f54 901684 0 2024-02-19T19:04:53.011 INFO:tasks.workunit.client.1.smithi132.stdout:1/349: dread d1/f48 [4194304,4194304] 0 2024-02-19T19:04:53.012 INFO:tasks.workunit.client.1.smithi132.stdout:1/350: readlink d1/d1e/l68 0 2024-02-19T19:04:53.012 INFO:tasks.workunit.client.1.smithi132.stdout:8/356: dwrite f2 [0,4194304] 0 2024-02-19T19:04:53.020 INFO:tasks.workunit.client.1.smithi132.stdout:7/401: link lc d1b/d32/l83 0 2024-02-19T19:04:53.020 INFO:tasks.workunit.client.1.smithi132.stdout:5/366: rmdir d8 39 2024-02-19T19:04:53.023 INFO:tasks.workunit.client.1.smithi132.stdout:1/351: dwrite d1/d1e/d29/d4e/f5f [0,4194304] 0 2024-02-19T19:04:53.023 INFO:tasks.workunit.client.1.smithi132.stdout:8/357: dwrite d4/f29 [0,4194304] 0 2024-02-19T19:04:53.026 INFO:tasks.workunit.client.1.smithi132.stdout:5/367: read - d8/d1a/d35/f74 zero size 2024-02-19T19:04:53.027 INFO:tasks.workunit.client.1.smithi132.stdout:5/368: chown d8/d1a/f26 1779398961 1 2024-02-19T19:04:53.028 INFO:tasks.workunit.client.1.smithi132.stdout:7/402: mkdir d1b/d5a/d63/d6b/d79/d84 0 2024-02-19T19:04:53.037 INFO:tasks.workunit.client.1.smithi132.stdout:5/369: rename d8/d23/d5d to d8/d1a/d35/d77 0 2024-02-19T19:04:53.040 INFO:tasks.workunit.client.1.smithi132.stdout:8/358: write d4/dc/d1b/d20/d23/f2b [4144607,63251] 0 2024-02-19T19:04:53.042 INFO:tasks.workunit.client.1.smithi132.stdout:5/370: truncate d8/d2e/f55 545366 0 2024-02-19T19:04:53.043 INFO:tasks.workunit.client.1.smithi132.stdout:5/371: readlink d8/d1a/d35/d77/l60 0 2024-02-19T19:04:53.049 INFO:tasks.workunit.client.1.smithi132.stdout:1/352: getdents d1/d1e/d29/d4e 0 2024-02-19T19:04:53.050 INFO:tasks.workunit.client.1.smithi132.stdout:1/353: write d1/f4a [5070073,84841] 0 2024-02-19T19:04:53.051 INFO:tasks.workunit.client.1.smithi132.stdout:8/359: mknod d4/dc/d1b/d20/d23/d68/c75 0 2024-02-19T19:04:53.052 INFO:tasks.workunit.client.1.smithi132.stdout:4/326: sync 2024-02-19T19:04:53.052 INFO:tasks.workunit.client.1.smithi132.stdout:2/351: sync 2024-02-19T19:04:53.053 INFO:tasks.workunit.client.1.smithi132.stdout:9/449: sync 2024-02-19T19:04:53.053 INFO:tasks.workunit.client.1.smithi132.stdout:4/327: chown d4/d12/d16 303590 1 2024-02-19T19:04:53.053 INFO:tasks.workunit.client.1.smithi132.stdout:9/450: chown d12/d8d/l31 223 1 2024-02-19T19:04:53.054 INFO:tasks.workunit.client.1.smithi132.stdout:7/403: link d1b/d32/c33 d1b/d5a/c85 0 2024-02-19T19:04:53.058 INFO:tasks.workunit.client.1.smithi132.stdout:8/360: rmdir d4/dc/d1b/d20/d23/d68 39 2024-02-19T19:04:53.062 INFO:tasks.workunit.client.1.smithi132.stdout:4/328: creat d4/d27/d31/d71/d37/f74 x:0 0 0 2024-02-19T19:04:53.064 INFO:tasks.workunit.client.1.smithi132.stdout:9/451: creat d12/d8d/d3a/d40/d4e/d77/fa2 x:0 0 0 2024-02-19T19:04:53.065 INFO:tasks.workunit.client.1.smithi132.stdout:9/452: fsync d12/d3e/f84 0 2024-02-19T19:04:53.065 INFO:tasks.workunit.client.1.smithi132.stdout:7/404: mkdir d1b/d5a/d63/d6b/d86 0 2024-02-19T19:04:53.067 INFO:tasks.workunit.client.1.smithi132.stdout:9/453: write d12/d8d/d3a/f6d [2468715,130107] 0 2024-02-19T19:04:53.068 INFO:tasks.workunit.client.1.smithi132.stdout:1/354: fsync d1/d1e/d29/d2c/f34 0 2024-02-19T19:04:53.069 INFO:tasks.workunit.client.1.smithi132.stdout:5/372: write d8/d1a/d35/f4a [521405,15825] 0 2024-02-19T19:04:53.070 INFO:tasks.workunit.client.1.smithi132.stdout:4/329: dwrite d4/d12/d28/f62 [0,4194304] 0 2024-02-19T19:04:53.070 INFO:tasks.workunit.client.1.smithi132.stdout:8/361: mknod d4/dc/d1b/d38/d69/c76 0 2024-02-19T19:04:53.074 INFO:tasks.workunit.client.1.smithi132.stdout:2/352: rename d0/d24/d58/f67 to d0/d4/df/d40/f74 0 2024-02-19T19:04:53.076 INFO:tasks.workunit.client.1.smithi132.stdout:2/353: truncate d0/d19/d1e/f62 708090 0 2024-02-19T19:04:53.079 INFO:tasks.workunit.client.1.smithi132.stdout:7/405: symlink d1b/d1e/d2b/d45/l87 0 2024-02-19T19:04:53.080 INFO:tasks.workunit.client.1.smithi132.stdout:9/454: symlink d12/d8d/d3a/d40/d4e/d77/d89/la3 0 2024-02-19T19:04:53.081 INFO:tasks.workunit.client.1.smithi132.stdout:5/373: symlink d8/d23/d3b/l78 0 2024-02-19T19:04:53.083 INFO:tasks.workunit.client.1.smithi132.stdout:8/362: creat d4/dc/d1b/d1d/f77 x:0 0 0 2024-02-19T19:04:53.085 INFO:tasks.workunit.client.1.smithi132.stdout:2/354: creat d0/d4/d11/d2e/f75 x:0 0 0 2024-02-19T19:04:53.087 INFO:tasks.workunit.client.1.smithi132.stdout:9/455: dwrite d12/d1b/f50 [0,4194304] 0 2024-02-19T19:04:53.090 INFO:tasks.workunit.client.1.smithi132.stdout:5/374: dwrite d8/d1a/d35/f4a [0,4194304] 0 2024-02-19T19:04:53.092 INFO:tasks.workunit.client.1.smithi132.stdout:5/375: chown d8/d1a/c3a 611146 1 2024-02-19T19:04:53.092 INFO:tasks.workunit.client.1.smithi132.stdout:2/355: dread d0/d19/d1e/f45 [0,4194304] 0 2024-02-19T19:04:53.095 INFO:tasks.workunit.client.1.smithi132.stdout:8/363: symlink d4/dc/l78 0 2024-02-19T19:04:53.096 INFO:tasks.workunit.client.1.smithi132.stdout:1/355: getdents d1/d3a/d50 0 2024-02-19T19:04:53.097 INFO:tasks.workunit.client.1.smithi132.stdout:2/356: read d0/d19/f2c [500954,111163] 0 2024-02-19T19:04:53.098 INFO:tasks.workunit.client.1.smithi132.stdout:5/376: write d8/d1a/d35/f74 [1002589,53254] 0 2024-02-19T19:04:53.099 INFO:tasks.workunit.client.1.smithi132.stdout:2/357: truncate d0/d4/df/f51 5612 0 2024-02-19T19:04:53.101 INFO:tasks.workunit.client.1.smithi132.stdout:5/377: dread - d8/da/db/f4f zero size 2024-02-19T19:04:53.102 INFO:tasks.workunit.client.1.smithi132.stdout:1/356: dread d1/f53 [0,4194304] 0 2024-02-19T19:04:53.102 INFO:tasks.workunit.client.1.smithi132.stdout:5/378: dread - d8/d23/f6f zero size 2024-02-19T19:04:53.102 INFO:tasks.workunit.client.1.smithi132.stdout:8/364: rmdir d4/dc/d42 39 2024-02-19T19:04:53.103 INFO:tasks.workunit.client.1.smithi132.stdout:1/357: write d1/f1d [1757424,27677] 0 2024-02-19T19:04:53.104 INFO:tasks.workunit.client.1.smithi132.stdout:5/379: chown d8/d1a/f5e 71265 1 2024-02-19T19:04:53.110 INFO:tasks.workunit.client.1.smithi132.stdout:7/406: truncate d1b/d1e/d28/f4f 356323 0 2024-02-19T19:04:53.111 INFO:tasks.workunit.client.1.smithi132.stdout:4/330: link d4/d19/d21/d56/l5e d4/d27/d31/d32/d51/d5c/l75 0 2024-02-19T19:04:53.115 INFO:tasks.workunit.client.1.smithi132.stdout:5/380: unlink d8/d1a/d35/c75 0 2024-02-19T19:04:53.116 INFO:tasks.workunit.client.1.smithi132.stdout:9/456: link d12/f33 d12/fa4 0 2024-02-19T19:04:53.116 INFO:tasks.workunit.client.1.smithi132.stdout:9/457: truncate d12/d3e/f6c 160097 0 2024-02-19T19:04:53.119 INFO:tasks.workunit.client.1.smithi132.stdout:7/407: mknod d1b/d1e/d2b/d62/c88 0 2024-02-19T19:04:53.120 INFO:tasks.workunit.client.1.smithi132.stdout:7/408: chown d1b/d1e/f21 59369 1 2024-02-19T19:04:53.120 INFO:tasks.workunit.client.1.smithi132.stdout:8/365: mkdir d4/dc/d1b/d20/d23/d68/d79 0 2024-02-19T19:04:53.123 INFO:tasks.workunit.client.1.smithi132.stdout:4/331: getdents d4/d19 0 2024-02-19T19:04:53.124 INFO:tasks.workunit.client.1.smithi132.stdout:4/332: write d4/d12/f2c [920332,9482] 0 2024-02-19T19:04:53.125 INFO:tasks.workunit.client.1.smithi132.stdout:7/409: symlink d1b/d46/d76/l89 0 2024-02-19T19:04:53.125 INFO:tasks.workunit.client.1.smithi132.stdout:2/358: dwrite d0/d4/d11/f1c [0,4194304] 0 2024-02-19T19:04:53.126 INFO:tasks.workunit.client.1.smithi132.stdout:7/410: dread - d1b/d1e/d41/f6a zero size 2024-02-19T19:04:53.126 INFO:tasks.workunit.client.1.smithi132.stdout:8/366: mkdir d4/dc/d1b/d20/d23/d68/d7a 0 2024-02-19T19:04:53.127 INFO:tasks.workunit.client.1.smithi132.stdout:2/359: chown d0/d19/f2c 2 1 2024-02-19T19:04:53.128 INFO:tasks.workunit.client.1.smithi132.stdout:8/367: write d4/dc/d1b/d20/f6b [739966,8042] 0 2024-02-19T19:04:53.133 INFO:tasks.workunit.client.1.smithi132.stdout:2/360: write d0/d19/d1e/f62 [550198,87267] 0 2024-02-19T19:04:53.133 INFO:tasks.workunit.client.1.smithi132.stdout:9/458: write d12/fa4 [1947084,56149] 0 2024-02-19T19:04:53.134 INFO:tasks.workunit.client.1.smithi132.stdout:5/381: getdents d8/d23/d29 0 2024-02-19T19:04:53.134 INFO:tasks.workunit.client.1.smithi132.stdout:2/361: dread - d0/d4/d11/d2e/f75 zero size 2024-02-19T19:04:53.135 INFO:tasks.workunit.client.1.smithi132.stdout:5/382: readlink d8/d23/l76 0 2024-02-19T19:04:53.135 INFO:tasks.workunit.client.1.smithi132.stdout:5/383: readlink d8/d23/l76 0 2024-02-19T19:04:53.136 INFO:tasks.workunit.client.1.smithi132.stdout:8/368: dread d4/f27 [0,4194304] 0 2024-02-19T19:04:53.137 INFO:tasks.workunit.client.1.smithi132.stdout:5/384: truncate d8/d1a/d35/f74 1797571 0 2024-02-19T19:04:53.137 INFO:tasks.workunit.client.1.smithi132.stdout:8/369: chown d4/dc/d1b/d20/f6e 7 1 2024-02-19T19:04:53.141 INFO:tasks.workunit.client.1.smithi132.stdout:9/459: rename d12/d8d/d3a/d40/f54 to d12/d8d/d3a/d40/d4e/d77/d89/fa5 0 2024-02-19T19:04:53.142 INFO:tasks.workunit.client.1.smithi132.stdout:9/460: write d12/d1b/f95 [1087574,97906] 0 2024-02-19T19:04:53.143 INFO:tasks.workunit.client.1.smithi132.stdout:8/370: rmdir d4/dc/d1b/d54/d5e 39 2024-02-19T19:04:53.144 INFO:tasks.workunit.client.1.smithi132.stdout:5/385: getdents d8/d2e/d71 0 2024-02-19T19:04:53.149 INFO:tasks.workunit.client.1.smithi132.stdout:7/411: truncate d1b/d1e/d41/f47 1813578 0 2024-02-19T19:04:53.151 INFO:tasks.workunit.client.1.smithi132.stdout:7/412: dread - d1b/d1e/d41/f6d zero size 2024-02-19T19:04:53.151 INFO:tasks.workunit.client.1.smithi132.stdout:9/461: dread d12/d8d/d3a/d40/f65 [0,4194304] 0 2024-02-19T19:04:53.154 INFO:tasks.workunit.client.1.smithi132.stdout:2/362: dwrite d0/d4/fb [0,4194304] 0 2024-02-19T19:04:53.154 INFO:tasks.workunit.client.1.smithi132.stdout:9/462: readlink d12/d8d/d3a/d40/d4e/d77/d89/l92 0 2024-02-19T19:04:53.156 INFO:tasks.workunit.client.1.smithi132.stdout:9/463: fdatasync d12/d2a/d4c/d61/f63 0 2024-02-19T19:04:53.156 INFO:tasks.workunit.client.1.smithi132.stdout:7/413: mknod d1b/d1e/d28/c8a 0 2024-02-19T19:04:53.157 INFO:tasks.workunit.client.1.smithi132.stdout:5/386: dwrite d8/fe [0,4194304] 0 2024-02-19T19:04:53.159 INFO:tasks.workunit.client.1.smithi132.stdout:5/387: write d8/d23/d3b/f70 [479207,1090] 0 2024-02-19T19:04:53.160 INFO:tasks.workunit.client.1.smithi132.stdout:5/388: fdatasync d8/d23/d29/d27/f57 0 2024-02-19T19:04:53.161 INFO:tasks.workunit.client.1.smithi132.stdout:2/363: unlink d0/d4/f1f 0 2024-02-19T19:04:53.168 INFO:tasks.workunit.client.1.smithi132.stdout:2/364: rename d0/d19/d6e to d0/d64/d6d/d76 0 2024-02-19T19:04:53.169 INFO:tasks.workunit.client.1.smithi132.stdout:9/464: link d12/d1b/c58 d12/d8d/d3a/d85/d87/ca6 0 2024-02-19T19:04:53.170 INFO:tasks.workunit.client.1.smithi132.stdout:7/414: dwrite d1b/d46/d56/f75 [0,4194304] 0 2024-02-19T19:04:53.172 INFO:tasks.workunit.client.1.smithi132.stdout:5/389: dwrite d8/da/f30 [0,4194304] 0 2024-02-19T19:04:53.177 INFO:tasks.workunit.client.1.smithi132.stdout:2/365: mknod d0/d24/d58/d53/c77 0 2024-02-19T19:04:53.179 INFO:tasks.workunit.client.1.smithi132.stdout:7/415: mknod d1b/d5a/c8b 0 2024-02-19T19:04:53.179 INFO:tasks.workunit.client.1.smithi132.stdout:5/390: symlink d8/d1a/d64/l79 0 2024-02-19T19:04:53.179 INFO:tasks.workunit.client.1.smithi132.stdout:2/366: write d0/d4/d11/d2e/d5a/f5b [2068366,62417] 0 2024-02-19T19:04:53.181 INFO:tasks.workunit.client.1.smithi132.stdout:7/416: chown l16 106663482 1 2024-02-19T19:04:53.182 INFO:tasks.workunit.client.1.smithi132.stdout:9/465: rmdir d12/d2a/d4c/d61/d75 0 2024-02-19T19:04:53.182 INFO:tasks.workunit.client.1.smithi132.stdout:8/371: dread d4/dc/d1b/d1d/f36 [0,4194304] 0 2024-02-19T19:04:53.183 INFO:tasks.workunit.client.1.smithi132.stdout:5/391: readlink d8/da/d5a/l3d 0 2024-02-19T19:04:53.183 INFO:tasks.workunit.client.1.smithi132.stdout:7/417: dread - d1b/d1e/d2b/d62/d53/f65 zero size 2024-02-19T19:04:53.184 INFO:tasks.workunit.client.1.smithi132.stdout:2/367: rmdir d0/d4/d11/d2e/d5a 39 2024-02-19T19:04:53.185 INFO:tasks.workunit.client.1.smithi132.stdout:2/368: fsync d0/d24/d58/f49 0 2024-02-19T19:04:53.186 INFO:tasks.workunit.client.1.smithi132.stdout:5/392: creat d8/d1a/f7a x:0 0 0 2024-02-19T19:04:53.186 INFO:tasks.workunit.client.1.smithi132.stdout:2/369: write d0/d4/d11/d2e/f75 [57588,54080] 0 2024-02-19T19:04:53.190 INFO:tasks.workunit.client.1.smithi132.stdout:8/372: write d4/dc/d1b/d20/d23/f28 [1912989,86087] 0 2024-02-19T19:04:53.195 INFO:tasks.workunit.client.1.smithi132.stdout:6/323: sync 2024-02-19T19:04:53.197 INFO:tasks.workunit.client.1.smithi132.stdout:5/393: mkdir d8/da/d5a/d7b 0 2024-02-19T19:04:53.199 INFO:tasks.workunit.client.1.smithi132.stdout:5/394: write d8/fe [1959523,59489] 0 2024-02-19T19:04:53.200 INFO:tasks.workunit.client.1.smithi132.stdout:9/466: creat d12/d8d/d3a/d40/d4e/d77/d9d/fa7 x:0 0 0 2024-02-19T19:04:53.203 INFO:tasks.workunit.client.1.smithi132.stdout:7/418: dwrite d1b/d1e/d2b/f2f [0,4194304] 0 2024-02-19T19:04:53.203 INFO:tasks.workunit.client.1.smithi132.stdout:8/373: symlink d4/dc/d1b/d54/l7b 0 2024-02-19T19:04:53.206 INFO:tasks.workunit.client.1.smithi132.stdout:6/324: unlink d0/l35 0 2024-02-19T19:04:53.207 INFO:tasks.workunit.client.1.smithi132.stdout:2/370: creat d0/d19/f78 x:0 0 0 2024-02-19T19:04:53.213 INFO:tasks.workunit.client.1.smithi132.stdout:6/325: fsync d0/db/d25/d3a/d3d/d69/f5f 0 2024-02-19T19:04:53.213 INFO:tasks.workunit.client.1.smithi132.stdout:9/467: dwrite d12/d3e/f72 [0,4194304] 0 2024-02-19T19:04:53.214 INFO:tasks.workunit.client.1.smithi132.stdout:5/395: dwrite d8/d1a/f5e [0,4194304] 0 2024-02-19T19:04:53.214 INFO:tasks.workunit.client.1.smithi132.stdout:8/374: write d4/dc/d1b/f32 [559299,92265] 0 2024-02-19T19:04:53.216 INFO:tasks.workunit.client.1.smithi132.stdout:8/375: stat d4/d2d 0 2024-02-19T19:04:53.221 INFO:tasks.workunit.client.1.smithi132.stdout:7/419: dwrite d1b/f72 [0,4194304] 0 2024-02-19T19:04:53.223 INFO:tasks.workunit.client.1.smithi132.stdout:2/371: dread d0/d19/f25 [0,4194304] 0 2024-02-19T19:04:53.228 INFO:tasks.workunit.client.1.smithi132.stdout:9/468: dwrite d12/d8d/f76 [0,4194304] 0 2024-02-19T19:04:53.229 INFO:tasks.workunit.client.1.smithi132.stdout:8/376: unlink d4/dc/d1b/d54/d5e/l3a 0 2024-02-19T19:04:53.234 INFO:tasks.workunit.client.1.smithi132.stdout:7/420: mknod d1b/d46/c8c 0 2024-02-19T19:04:53.236 INFO:tasks.workunit.client.1.smithi132.stdout:5/396: dwrite d8/d1a/d35/d77/f67 [0,4194304] 0 2024-02-19T19:04:53.241 INFO:tasks.workunit.client.1.smithi132.stdout:9/469: creat d12/d2a/d4c/d61/fa8 x:0 0 0 2024-02-19T19:04:53.243 INFO:tasks.workunit.client.1.smithi132.stdout:2/372: dwrite d0/d4/d11/d2e/f6c [0,4194304] 0 2024-02-19T19:04:53.249 INFO:tasks.workunit.client.1.smithi132.stdout:5/397: symlink d8/d2e/l7c 0 2024-02-19T19:04:53.251 INFO:tasks.workunit.client.1.smithi132.stdout:8/377: getdents d4/d2d 0 2024-02-19T19:04:53.252 INFO:tasks.workunit.client.1.smithi132.stdout:7/421: fsync d1b/d1e/d28/f4f 0 2024-02-19T19:04:53.253 INFO:tasks.workunit.client.1.smithi132.stdout:9/470: dwrite d12/d1b/f95 [0,4194304] 0 2024-02-19T19:04:53.253 INFO:tasks.workunit.client.1.smithi132.stdout:9/471: stat d12/d3e/f4a 0 2024-02-19T19:04:53.254 INFO:tasks.workunit.client.1.smithi132.stdout:9/472: chown d12/d8d/l31 471956 1 2024-02-19T19:04:53.260 INFO:tasks.workunit.client.1.smithi132.stdout:5/398: unlink d8/d1a/f26 0 2024-02-19T19:04:53.260 INFO:tasks.workunit.client.1.smithi132.stdout:8/378: symlink d4/dc/d1b/d1d/l7c 0 2024-02-19T19:04:53.261 INFO:tasks.workunit.client.1.smithi132.stdout:7/422: dwrite f8 [0,4194304] 0 2024-02-19T19:04:53.262 INFO:tasks.workunit.client.1.smithi132.stdout:9/473: creat d12/d8d/d3a/d40/d4e/d77/d8a/d82/fa9 x:0 0 0 2024-02-19T19:04:53.264 INFO:tasks.workunit.client.1.smithi132.stdout:9/474: readlink lf 0 2024-02-19T19:04:53.275 INFO:tasks.workunit.client.1.smithi132.stdout:5/399: link d8/d1a/d35/d77/c66 d8/d2e/d71/c7d 0 2024-02-19T19:04:53.276 INFO:tasks.workunit.client.1.smithi132.stdout:2/373: dwrite d0/d4/d11/f15 [0,4194304] 0 2024-02-19T19:04:53.279 INFO:tasks.workunit.client.1.smithi132.stdout:8/379: dwrite d4/dc/d1b/d54/d5e/f50 [4194304,4194304] 0 2024-02-19T19:04:53.281 INFO:tasks.workunit.client.1.smithi132.stdout:9/475: dwrite d12/d2a/d4c/f9b [0,4194304] 0 2024-02-19T19:04:53.282 INFO:tasks.workunit.client.1.smithi132.stdout:8/380: dread - d4/dc/d1b/d54/d5e/f51 zero size 2024-02-19T19:04:53.283 INFO:tasks.workunit.client.1.smithi132.stdout:9/476: write d12/d8d/d3a/f6d [2454394,130037] 0 2024-02-19T19:04:53.286 INFO:tasks.workunit.client.1.smithi132.stdout:1/358: sync 2024-02-19T19:04:53.286 INFO:tasks.workunit.client.1.smithi132.stdout:3/309: sync 2024-02-19T19:04:53.287 INFO:tasks.workunit.client.1.smithi132.stdout:0/357: sync 2024-02-19T19:04:53.287 INFO:tasks.workunit.client.1.smithi132.stdout:2/374: readlink d0/d3c/d3f/l5d 0 2024-02-19T19:04:53.294 INFO:tasks.workunit.client.1.smithi132.stdout:7/423: dread d1b/d1e/d28/f29 [0,4194304] 0 2024-02-19T19:04:53.296 INFO:tasks.workunit.client.1.smithi132.stdout:8/381: rename d4/l72 to d4/dc/d1b/d1d/l7d 0 2024-02-19T19:04:53.297 INFO:tasks.workunit.client.1.smithi132.stdout:3/310: dwrite d8/f5a [0,4194304] 0 2024-02-19T19:04:53.298 INFO:tasks.workunit.client.1.smithi132.stdout:8/382: stat d4/dc/d1b/d38/d69/c6a 0 2024-02-19T19:04:53.298 INFO:tasks.workunit.client.1.smithi132.stdout:9/477: symlink d12/d8d/laa 0 2024-02-19T19:04:53.299 INFO:tasks.workunit.client.1.smithi132.stdout:3/311: readlink d8/d13/d19/l39 0 2024-02-19T19:04:53.303 INFO:tasks.workunit.client.1.smithi132.stdout:0/358: rmdir d2/dc/d4a/d25 39 2024-02-19T19:04:53.303 INFO:tasks.workunit.client.1.smithi132.stdout:2/375: stat d0/d3c/c46 0 2024-02-19T19:04:53.304 INFO:tasks.workunit.client.1.smithi132.stdout:1/359: fdatasync d1/f15 0 2024-02-19T19:04:53.307 INFO:tasks.workunit.client.1.smithi132.stdout:9/478: mknod d12/d2a/d88/cab 0 2024-02-19T19:04:53.309 INFO:tasks.workunit.client.1.smithi132.stdout:5/400: dread d8/d23/f5c [0,4194304] 0 2024-02-19T19:04:53.309 INFO:tasks.workunit.client.1.smithi132.stdout:7/424: read d1b/d1e/d2b/d62/d53/f81 [971907,95362] 0 2024-02-19T19:04:53.310 INFO:tasks.workunit.client.1.smithi132.stdout:2/376: unlink d0/d24/d58/l56 0 2024-02-19T19:04:53.315 INFO:tasks.workunit.client.1.smithi132.stdout:3/312: dwrite d8/d13/d2b/f2c [0,4194304] 0 2024-02-19T19:04:53.315 INFO:tasks.workunit.client.1.smithi132.stdout:0/359: write d2/dc/df/d27/d3a/d52/f68 [254300,84964] 0 2024-02-19T19:04:53.315 INFO:tasks.workunit.client.1.smithi132.stdout:9/479: creat d12/d3e/d6e/fac x:0 0 0 2024-02-19T19:04:53.316 INFO:tasks.workunit.client.1.smithi132.stdout:2/377: write d0/d4/fb [181333,96840] 0 2024-02-19T19:04:53.318 INFO:tasks.workunit.client.1.smithi132.stdout:4/333: sync 2024-02-19T19:04:53.319 INFO:tasks.workunit.client.1.smithi132.stdout:7/425: creat d1b/d1e/d2b/d45/f8d x:0 0 0 2024-02-19T19:04:53.321 INFO:tasks.workunit.client.1.smithi132.stdout:1/360: getdents d1/d1e/d29/d52/d64 0 2024-02-19T19:04:53.321 INFO:tasks.workunit.client.1.smithi132.stdout:5/401: symlink d8/d1a/d35/d77/l7e 0 2024-02-19T19:04:53.322 INFO:tasks.workunit.client.1.smithi132.stdout:9/480: mkdir d12/d2a/d4c/dad 0 2024-02-19T19:04:53.322 INFO:tasks.workunit.client.1.smithi132.stdout:0/360: chown d2/dc/df/f71 335723745 1 2024-02-19T19:04:53.324 INFO:tasks.workunit.client.1.smithi132.stdout:2/378: stat d0/c13 0 2024-02-19T19:04:53.324 INFO:tasks.workunit.client.1.smithi132.stdout:7/426: chown d1b/d5a/d63/d6b/d79/d84 1 1 2024-02-19T19:04:53.324 INFO:tasks.workunit.client.1.smithi132.stdout:1/361: chown d1/l4c 37555 1 2024-02-19T19:04:53.324 INFO:tasks.workunit.client.1.smithi132.stdout:3/313: mkdir d8/d5d 0 2024-02-19T19:04:53.325 INFO:tasks.workunit.client.1.smithi132.stdout:9/481: write d12/d1b/f16 [3394690,66085] 0 2024-02-19T19:04:53.326 INFO:tasks.workunit.client.1.smithi132.stdout:4/334: mknod d4/d27/d31/d71/d14/d24/c76 0 2024-02-19T19:04:53.326 INFO:tasks.workunit.client.1.smithi132.stdout:3/314: read d8/d13/d2b/f2c [1570476,85062] 0 2024-02-19T19:04:53.327 INFO:tasks.workunit.client.1.smithi132.stdout:7/427: chown d1b/d1e/d28/f3c 772188 1 2024-02-19T19:04:53.329 INFO:tasks.workunit.client.1.smithi132.stdout:3/315: chown d8/d38 245 1 2024-02-19T19:04:53.331 INFO:tasks.workunit.client.1.smithi132.stdout:1/362: mknod d1/d3a/d50/c6c 0 2024-02-19T19:04:53.338 INFO:tasks.workunit.client.1.smithi132.stdout:2/379: dwrite d0/d4/d11/f32 [4194304,4194304] 0 2024-02-19T19:04:53.342 INFO:tasks.workunit.client.1.smithi132.stdout:8/383: truncate d4/dc/d1b/d1d/f36 4830391 0 2024-02-19T19:04:53.342 INFO:tasks.workunit.client.1.smithi132.stdout:9/482: dwrite d12/f43 [0,4194304] 0 2024-02-19T19:04:53.343 INFO:tasks.workunit.client.1.smithi132.stdout:3/316: creat d8/d38/d44/f5e x:0 0 0 2024-02-19T19:04:53.344 INFO:tasks.workunit.client.1.smithi132.stdout:1/363: rmdir d1 39 2024-02-19T19:04:53.346 INFO:tasks.workunit.client.1.smithi132.stdout:5/402: rmdir d8/da/d5a/d51 0 2024-02-19T19:04:53.346 INFO:tasks.workunit.client.1.smithi132.stdout:7/428: dwrite d1b/d1e/d28/f3c [4194304,4194304] 0 2024-02-19T19:04:53.346 INFO:tasks.workunit.client.1.smithi132.stdout:9/483: chown d12/d3e/f48 335045871 1 2024-02-19T19:04:53.347 INFO:tasks.workunit.client.1.smithi132.stdout:8/384: truncate d4/dc/d1b/f32 1591584 0 2024-02-19T19:04:53.347 INFO:tasks.workunit.client.1.smithi132.stdout:7/429: write d1b/d1e/f38 [5102023,5154] 0 2024-02-19T19:04:53.354 INFO:tasks.workunit.client.1.smithi132.stdout:8/385: dwrite d4/dc/d1b/d1d/f77 [0,4194304] 0 2024-02-19T19:04:53.359 INFO:tasks.workunit.client.1.smithi132.stdout:0/361: link d2/dc/d4a/d25/d43/d5f/f73 d2/dc/df/d15/d2f/d3f/d63/d6b/f75 0 2024-02-19T19:04:53.360 INFO:tasks.workunit.client.1.smithi132.stdout:0/362: stat d2/d3e/c51 0 2024-02-19T19:04:53.362 INFO:tasks.workunit.client.1.smithi132.stdout:5/403: creat d8/d23/d29/f7f x:0 0 0 2024-02-19T19:04:53.363 INFO:tasks.workunit.client.1.smithi132.stdout:4/335: dwrite d4/d12/d16/f47 [4194304,4194304] 0 2024-02-19T19:04:53.364 INFO:tasks.workunit.client.1.smithi132.stdout:0/363: write d2/dc/d4a/f57 [3148808,4966] 0 2024-02-19T19:04:53.366 INFO:tasks.workunit.client.1.smithi132.stdout:7/430: unlink d1b/d1e/d2b/d45/f8d 0 2024-02-19T19:04:53.371 INFO:tasks.workunit.client.1.smithi132.stdout:2/380: mknod d0/d24/d58/d71/c79 0 2024-02-19T19:04:53.372 INFO:tasks.workunit.client.1.smithi132.stdout:1/364: creat d1/d1e/d29/d4e/f6d x:0 0 0 2024-02-19T19:04:53.372 INFO:tasks.workunit.client.1.smithi132.stdout:2/381: write d0/d19/d1e/f52 [2780340,76711] 0 2024-02-19T19:04:53.372 INFO:tasks.workunit.client.1.smithi132.stdout:7/431: fsync d1b/d1e/d2b/f2f 0 2024-02-19T19:04:53.374 INFO:tasks.workunit.client.1.smithi132.stdout:6/326: sync 2024-02-19T19:04:53.374 INFO:tasks.workunit.client.1.smithi132.stdout:3/317: rename d8/d38/l46 to d8/d13/d2b/d52/d34/d50/l5f 0 2024-02-19T19:04:53.384 INFO:tasks.workunit.client.1.smithi132.stdout:4/336: rmdir d4/d19/d21 39 2024-02-19T19:04:53.387 INFO:tasks.workunit.client.1.smithi132.stdout:0/364: stat d2/c1b 0 2024-02-19T19:04:53.393 INFO:tasks.workunit.client.1.smithi132.stdout:8/386: readlink d4/dc/d1b/d20/d23/l70 0 2024-02-19T19:04:53.394 INFO:tasks.workunit.client.1.smithi132.stdout:8/387: chown d4/dc/d1b/d54/d5e/f50 2 1 2024-02-19T19:04:53.396 INFO:tasks.workunit.client.1.smithi132.stdout:7/432: chown d1b/d1e/d2b/d62/l64 34199278 1 2024-02-19T19:04:53.401 INFO:tasks.workunit.client.1.smithi132.stdout:9/484: link d12/d3e/f48 d12/fae 0 2024-02-19T19:04:53.401 INFO:tasks.workunit.client.1.smithi132.stdout:6/327: write d0/db/d25/d38/f78 [1186977,60758] 0 2024-02-19T19:04:53.401 INFO:tasks.workunit.client.1.smithi132.stdout:2/382: dwrite d0/d24/f2d [0,4194304] 0 2024-02-19T19:04:53.403 INFO:tasks.workunit.client.1.smithi132.stdout:2/383: stat d0/d4/df/f4d 0 2024-02-19T19:04:53.403 INFO:tasks.workunit.client.1.smithi132.stdout:6/328: fdatasync d0/d3c/d22/f9 0 2024-02-19T19:04:53.403 INFO:tasks.workunit.client.1.smithi132.stdout:4/337: symlink d4/d12/d16/l77 0 2024-02-19T19:04:53.405 INFO:tasks.workunit.client.1.smithi132.stdout:5/404: creat d8/d2e/d71/f80 x:0 0 0 2024-02-19T19:04:53.406 INFO:tasks.workunit.client.1.smithi132.stdout:0/365: chown d2/f5c 5552 1 2024-02-19T19:04:53.407 INFO:tasks.workunit.client.1.smithi132.stdout:0/366: write d2/dc/df/d27/f35 [3822243,115301] 0 2024-02-19T19:04:53.412 INFO:tasks.workunit.client.1.smithi132.stdout:2/384: mkdir d0/d4/df/d7a 0 2024-02-19T19:04:53.413 INFO:tasks.workunit.client.1.smithi132.stdout:0/367: dread d2/dc/d4a/d25/f26 [0,4194304] 0 2024-02-19T19:04:53.415 INFO:tasks.workunit.client.1.smithi132.stdout:9/485: creat d12/d8d/d3a/d40/d4e/d77/d9d/faf x:0 0 0 2024-02-19T19:04:53.418 INFO:tasks.workunit.client.1.smithi132.stdout:5/405: rename d8/d23/d3b/f56 to d8/da/d5a/f81 0 2024-02-19T19:04:53.418 INFO:tasks.workunit.client.1.smithi132.stdout:5/406: chown d8/d1a/d35/d77/f67 5 1 2024-02-19T19:04:53.421 INFO:tasks.workunit.client.1.smithi132.stdout:1/365: rename d1/d3a/c60 to d1/d3a/c6e 0 2024-02-19T19:04:53.424 INFO:tasks.workunit.client.1.smithi132.stdout:7/433: creat d1b/d5a/d63/d6b/d79/d84/f8e x:0 0 0 2024-02-19T19:04:53.424 INFO:tasks.workunit.client.1.smithi132.stdout:6/329: write d0/d3c/d46/d5a/f1d [14232,110265] 0 2024-02-19T19:04:53.426 INFO:tasks.workunit.client.1.smithi132.stdout:3/318: dwrite d8/d13/f25 [0,4194304] 0 2024-02-19T19:04:53.426 INFO:tasks.workunit.client.1.smithi132.stdout:2/385: rmdir d0/d24/d58 39 2024-02-19T19:04:53.428 INFO:tasks.workunit.client.1.smithi132.stdout:0/368: mknod d2/dc/df/c76 0 2024-02-19T19:04:53.428 INFO:tasks.workunit.client.1.smithi132.stdout:3/319: truncate d8/d13/d2b/d52/f55 830766 0 2024-02-19T19:04:53.428 INFO:tasks.workunit.client.1.smithi132.stdout:4/338: mknod d4/d27/c78 0 2024-02-19T19:04:53.433 INFO:tasks.workunit.client.1.smithi132.stdout:2/386: dwrite d0/d4/d11/f41 [4194304,4194304] 0 2024-02-19T19:04:53.433 INFO:tasks.workunit.client.1.smithi132.stdout:9/486: dread d12/d8d/d3a/d40/d4e/d77/f8e [0,4194304] 0 2024-02-19T19:04:53.434 INFO:tasks.workunit.client.1.smithi132.stdout:9/487: chown d12/d2a/d4c/d61/l68 0 1 2024-02-19T19:04:53.434 INFO:tasks.workunit.client.1.smithi132.stdout:5/407: rename d8/da/db/f54 to d8/d2e/d71/f82 0 2024-02-19T19:04:53.436 INFO:tasks.workunit.client.1.smithi132.stdout:8/388: getdents d4/dc/d1b 0 2024-02-19T19:04:53.436 INFO:tasks.workunit.client.1.smithi132.stdout:9/488: write d12/d3e/d6e/fac [373697,127805] 0 2024-02-19T19:04:53.437 INFO:tasks.workunit.client.1.smithi132.stdout:1/366: creat d1/d3a/d50/f6f x:0 0 0 2024-02-19T19:04:53.439 INFO:tasks.workunit.client.1.smithi132.stdout:7/434: mkdir d1b/d1e/d2b/d8f 0 2024-02-19T19:04:53.440 INFO:tasks.workunit.client.1.smithi132.stdout:6/330: creat d0/d3c/d22/f7e x:0 0 0 2024-02-19T19:04:53.440 INFO:tasks.workunit.client.1.smithi132.stdout:7/435: chown d1b/d5a/f68 1797105 1 2024-02-19T19:04:53.443 INFO:tasks.workunit.client.1.smithi132.stdout:7/436: chown d1b/d1e/d28/f29 7817 1 2024-02-19T19:04:53.443 INFO:tasks.workunit.client.1.smithi132.stdout:3/320: unlink d8/l17 0 2024-02-19T19:04:53.447 INFO:tasks.workunit.client.1.smithi132.stdout:4/339: rmdir d4/d12/d48 39 2024-02-19T19:04:53.450 INFO:tasks.workunit.client.1.smithi132.stdout:6/331: dwrite d0/d3c/d46/f7b [0,4194304] 0 2024-02-19T19:04:53.451 INFO:tasks.workunit.client.1.smithi132.stdout:5/408: creat d8/da/db/f83 x:0 0 0 2024-02-19T19:04:53.451 INFO:tasks.workunit.client.1.smithi132.stdout:8/389: symlink d4/dc/d1b/d1d/l7e 0 2024-02-19T19:04:53.457 INFO:tasks.workunit.client.1.smithi132.stdout:7/437: creat d1b/d1e/f90 x:0 0 0 2024-02-19T19:04:53.457 INFO:tasks.workunit.client.1.smithi132.stdout:9/489: chown d12/d8d/d3a/d40/d4e/d77/d8a/d82/f8c 0 1 2024-02-19T19:04:53.458 INFO:tasks.workunit.client.1.smithi132.stdout:6/332: dread d0/d3c/d46/d5a/f2e [0,4194304] 0 2024-02-19T19:04:53.459 INFO:tasks.workunit.client.1.smithi132.stdout:8/390: chown d4/dc/d1b/d20/d23/c2f 26344 1 2024-02-19T19:04:53.460 INFO:tasks.workunit.client.1.smithi132.stdout:5/409: mkdir d8/da/d5a/d84 0 2024-02-19T19:04:53.460 INFO:tasks.workunit.client.1.smithi132.stdout:3/321: dwrite d8/d13/d2b/d52/f55 [0,4194304] 0 2024-02-19T19:04:53.461 INFO:tasks.workunit.client.1.smithi132.stdout:6/333: stat d0/f73 0 2024-02-19T19:04:53.463 INFO:tasks.workunit.client.1.smithi132.stdout:4/340: link d4/d12/d16/f5a d4/d27/d31/f79 0 2024-02-19T19:04:53.463 INFO:tasks.workunit.client.1.smithi132.stdout:0/369: link d2/dc/l10 d2/dc/df/d15/d2f/d3f/d63/l77 0 2024-02-19T19:04:53.464 INFO:tasks.workunit.client.1.smithi132.stdout:5/410: write d8/d1a/d35/d77/f6b [500107,47497] 0 2024-02-19T19:04:53.465 INFO:tasks.workunit.client.1.smithi132.stdout:6/334: write d0/d3c/d46/d5a/f18 [4047599,12572] 0 2024-02-19T19:04:53.467 INFO:tasks.workunit.client.1.smithi132.stdout:6/335: readlink d0/db/l3f 0 2024-02-19T19:04:53.467 INFO:tasks.workunit.client.1.smithi132.stdout:7/438: unlink d1b/d1e/f3d 0 2024-02-19T19:04:53.467 INFO:tasks.workunit.client.1.smithi132.stdout:3/322: truncate d8/d38/d44/f5e 367355 0 2024-02-19T19:04:53.468 INFO:tasks.workunit.client.1.smithi132.stdout:6/336: fsync d0/dc/f24 0 2024-02-19T19:04:53.469 INFO:tasks.workunit.client.1.smithi132.stdout:1/367: dread d1/f48 [0,4194304] 0 2024-02-19T19:04:53.469 INFO:tasks.workunit.client.1.smithi132.stdout:9/490: unlink d12/d8d/d3a/d40/d4e/d77/d89/fa5 0 2024-02-19T19:04:53.469 INFO:tasks.workunit.client.1.smithi132.stdout:3/323: chown d8/d13/d2b/f33 6970847 1 2024-02-19T19:04:53.470 INFO:tasks.workunit.client.1.smithi132.stdout:1/368: chown d1/d61 172765106 1 2024-02-19T19:04:53.472 INFO:tasks.workunit.client.1.smithi132.stdout:8/391: symlink d4/dc/d1b/d20/d23/d68/l7f 0 2024-02-19T19:04:53.473 INFO:tasks.workunit.client.1.smithi132.stdout:5/411: mknod d8/d1a/c85 0 2024-02-19T19:04:53.477 INFO:tasks.workunit.client.1.smithi132.stdout:0/370: write d2/dc/d4a/d25/f26 [1719600,45904] 0 2024-02-19T19:04:53.479 INFO:tasks.workunit.client.1.smithi132.stdout:2/387: dread d0/d4/d11/d2e/f3e [0,4194304] 0 2024-02-19T19:04:53.480 INFO:tasks.workunit.client.1.smithi132.stdout:8/392: dwrite f2 [0,4194304] 0 2024-02-19T19:04:53.481 INFO:tasks.workunit.client.1.smithi132.stdout:8/393: chown d4/dc/d1b/d38 0 1 2024-02-19T19:04:53.482 INFO:tasks.workunit.client.1.smithi132.stdout:1/369: dread d1/d3a/f40 [0,4194304] 0 2024-02-19T19:04:53.483 INFO:tasks.workunit.client.1.smithi132.stdout:5/412: symlink d8/d2e/l86 0 2024-02-19T19:04:53.484 INFO:tasks.workunit.client.1.smithi132.stdout:7/439: mknod d1b/d1e/d2b/d8f/c91 0 2024-02-19T19:04:53.486 INFO:tasks.workunit.client.1.smithi132.stdout:0/371: mkdir d2/dc/df/d78 0 2024-02-19T19:04:53.487 INFO:tasks.workunit.client.1.smithi132.stdout:5/413: readlink d8/da/db/l5f 0 2024-02-19T19:04:53.489 INFO:tasks.workunit.client.1.smithi132.stdout:1/370: write d1/d1e/d29/d4e/f4f [2348188,108821] 0 2024-02-19T19:04:53.491 INFO:tasks.workunit.client.1.smithi132.stdout:5/414: chown d8/d1a/d35/d77/l60 735211227 1 2024-02-19T19:04:53.492 INFO:tasks.workunit.client.1.smithi132.stdout:2/388: dwrite d0/d19/d1e/f62 [0,4194304] 0 2024-02-19T19:04:53.494 INFO:tasks.workunit.client.1.smithi132.stdout:8/394: creat d4/d2d/f80 x:0 0 0 2024-02-19T19:04:53.505 INFO:tasks.workunit.client.1.smithi132.stdout:4/341: rename d4/d12/d16/f6b to d4/d19/f7a 0 2024-02-19T19:04:53.508 INFO:tasks.workunit.client.1.smithi132.stdout:7/440: dread ff [0,4194304] 0 2024-02-19T19:04:53.509 INFO:tasks.workunit.client.1.smithi132.stdout:6/337: truncate d0/d3c/d22/f9 452758 0 2024-02-19T19:04:53.510 INFO:tasks.workunit.client.1.smithi132.stdout:6/338: readlink d0/db/d25/d38/l7a 0 2024-02-19T19:04:53.511 INFO:tasks.workunit.client.1.smithi132.stdout:9/491: dwrite d12/d3e/f48 [0,4194304] 0 2024-02-19T19:04:53.512 INFO:tasks.workunit.client.1.smithi132.stdout:9/492: chown d12/f26 0 1 2024-02-19T19:04:53.512 INFO:tasks.workunit.client.1.smithi132.stdout:6/339: write d0/db/d25/d3a/d3d/d69/f5f [572635,24687] 0 2024-02-19T19:04:53.513 INFO:tasks.workunit.client.1.smithi132.stdout:1/371: symlink d1/l70 0 2024-02-19T19:04:53.513 INFO:tasks.workunit.client.1.smithi132.stdout:6/340: truncate d0/f70 4593634 0 2024-02-19T19:04:53.514 INFO:tasks.workunit.client.1.smithi132.stdout:9/493: write d12/d8d/d3a/d40/d4e/d77/d9d/fa7 [538655,13097] 0 2024-02-19T19:04:53.515 INFO:tasks.workunit.client.1.smithi132.stdout:5/415: rmdir d8/d2e 39 2024-02-19T19:04:53.520 INFO:tasks.workunit.client.1.smithi132.stdout:9/494: dwrite d12/d8d/d3a/f6d [0,4194304] 0 2024-02-19T19:04:53.528 INFO:tasks.workunit.client.1.smithi132.stdout:0/372: write d2/dc/df/d15/f4b [1983510,112733] 0 2024-02-19T19:04:53.531 INFO:tasks.workunit.client.1.smithi132.stdout:7/441: creat d1b/d5a/d63/d6b/d79/d84/f92 x:0 0 0 2024-02-19T19:04:53.531 INFO:tasks.workunit.client.1.smithi132.stdout:6/341: creat d0/db/d25/d3a/d3d/d69/f7f x:0 0 0 2024-02-19T19:04:53.532 INFO:tasks.workunit.client.1.smithi132.stdout:7/442: chown d1b/d46/d76/l89 1 1 2024-02-19T19:04:53.534 INFO:tasks.workunit.client.1.smithi132.stdout:0/373: read d2/dc/df/d27/f35 [1349646,26756] 0 2024-02-19T19:04:53.535 INFO:tasks.workunit.client.1.smithi132.stdout:0/374: fdatasync d2/dc/df/d15/f4e 0 2024-02-19T19:04:53.537 INFO:tasks.workunit.client.1.smithi132.stdout:2/389: write d0/d4/df/f5f [1478172,4773] 0 2024-02-19T19:04:53.540 INFO:tasks.workunit.client.1.smithi132.stdout:8/395: mkdir d4/dc/d1b/d54/d81 0 2024-02-19T19:04:53.545 INFO:tasks.workunit.client.1.smithi132.stdout:4/342: creat d4/d19/d21/d56/f7b x:0 0 0 2024-02-19T19:04:53.546 INFO:tasks.workunit.client.1.smithi132.stdout:4/343: chown d4/d19/d21/c6d 6 1 2024-02-19T19:04:53.548 INFO:tasks.workunit.client.1.smithi132.stdout:6/342: readlink d0/db/d25/d3a/d3d/d69/l63 0 2024-02-19T19:04:53.555 INFO:tasks.workunit.client.1.smithi132.stdout:0/375: creat d2/dc/df/d15/d2f/d3f/f79 x:0 0 0 2024-02-19T19:04:53.555 INFO:tasks.workunit.client.1.smithi132.stdout:7/443: write d1b/d1e/d2b/f80 [137527,27904] 0 2024-02-19T19:04:53.556 INFO:tasks.workunit.client.1.smithi132.stdout:7/444: write d1b/d46/f4d [2167760,20274] 0 2024-02-19T19:04:53.556 INFO:tasks.workunit.client.1.smithi132.stdout:7/445: stat f8 0 2024-02-19T19:04:53.557 INFO:tasks.workunit.client.1.smithi132.stdout:5/416: link d8/d23/d29/d27/f69 d8/d23/d29/d27/f87 0 2024-02-19T19:04:53.559 INFO:tasks.workunit.client.1.smithi132.stdout:1/372: dread - d1/d1e/d29/d2c/d43/f5d zero size 2024-02-19T19:04:53.561 INFO:tasks.workunit.client.1.smithi132.stdout:9/495: link d12/d8d/d3a/f6d d12/d8d/d3a/d85/d87/fb0 0 2024-02-19T19:04:53.561 INFO:tasks.workunit.client.1.smithi132.stdout:9/496: dread - d12/d2a/d4c/d61/f7a zero size 2024-02-19T19:04:53.562 INFO:tasks.workunit.client.1.smithi132.stdout:9/497: chown d12/d3e/f84 1605698 1 2024-02-19T19:04:53.563 INFO:tasks.workunit.client.1.smithi132.stdout:3/324: rename d8/d13/d2b/d2e/d45 to d8/d13/d2b/d60 0 2024-02-19T19:04:53.566 INFO:tasks.workunit.client.1.smithi132.stdout:8/396: dread d4/dc/f1c [0,4194304] 0 2024-02-19T19:04:53.566 INFO:tasks.workunit.client.1.smithi132.stdout:0/376: readlink d2/l7 0 2024-02-19T19:04:53.569 INFO:tasks.workunit.client.1.smithi132.stdout:7/446: symlink d1b/d5a/d63/d6b/d79/l93 0 2024-02-19T19:04:53.569 INFO:tasks.workunit.client.1.smithi132.stdout:7/447: readlink d1b/l40 0 2024-02-19T19:04:53.571 INFO:tasks.workunit.client.1.smithi132.stdout:1/373: unlink d1/d1e/d29/l63 0 2024-02-19T19:04:53.574 INFO:tasks.workunit.client.1.smithi132.stdout:3/325: write d8/d13/d2b/d52/d34/d42/f54 [7456746,125229] 0 2024-02-19T19:04:53.576 INFO:tasks.workunit.client.1.smithi132.stdout:8/397: mkdir d4/d2d/d82 0 2024-02-19T19:04:53.577 INFO:tasks.workunit.client.1.smithi132.stdout:5/417: creat d8/da/d5a/d7b/f88 x:0 0 0 2024-02-19T19:04:53.577 INFO:tasks.workunit.client.1.smithi132.stdout:8/398: write d4/dc/d1b/f32 [2086428,89688] 0 2024-02-19T19:04:53.581 INFO:tasks.workunit.client.1.smithi132.stdout:9/498: getdents d12/d2a/d4c/dad 0 2024-02-19T19:04:53.581 INFO:tasks.workunit.client.1.smithi132.stdout:9/499: dread - d12/d8d/d3a/d40/f44 zero size 2024-02-19T19:04:53.586 INFO:tasks.workunit.client.1.smithi132.stdout:3/326: rename d8/d13/d19/l1d to d8/d13/d2b/d52/l61 0 2024-02-19T19:04:53.592 INFO:tasks.workunit.client.1.smithi132.stdout:7/448: fsync d1b/d1e/d2b/d62/d53/f70 0 2024-02-19T19:04:53.593 INFO:tasks.workunit.client.1.smithi132.stdout:1/374: dread d1/d1e/d29/d2c/f34 [0,4194304] 0 2024-02-19T19:04:53.593 INFO:tasks.workunit.client.1.smithi132.stdout:7/449: dread - d1b/d1e/d2b/d62/d53/f70 zero size 2024-02-19T19:04:53.594 INFO:tasks.workunit.client.1.smithi132.stdout:9/500: mknod d12/d8d/d3a/d40/d4e/d77/d8a/cb1 0 2024-02-19T19:04:53.594 INFO:tasks.workunit.client.1.smithi132.stdout:6/343: getdents d0/db/d25/d3a/d3d 0 2024-02-19T19:04:53.595 INFO:tasks.workunit.client.1.smithi132.stdout:6/344: chown d0/f70 7 1 2024-02-19T19:04:53.596 INFO:tasks.workunit.client.1.smithi132.stdout:3/327: rmdir d8/d13/d2b/d52/d34/d50 39 2024-02-19T19:04:53.599 INFO:tasks.workunit.client.1.smithi132.stdout:5/418: truncate d8/d23/d29/d27/f87 1186385 0 2024-02-19T19:04:53.600 INFO:tasks.workunit.client.1.smithi132.stdout:7/450: dwrite d1b/d1e/d28/f35 [4194304,4194304] 0 2024-02-19T19:04:53.600 INFO:tasks.workunit.client.1.smithi132.stdout:1/375: readlink d1/l9 0 2024-02-19T19:04:53.601 INFO:tasks.workunit.client.1.smithi132.stdout:5/419: readlink d8/d23/d29/d27/l32 0 2024-02-19T19:04:53.602 INFO:tasks.workunit.client.1.smithi132.stdout:5/420: write d8/d23/f6f [253326,55892] 0 2024-02-19T19:04:53.602 INFO:tasks.workunit.client.1.smithi132.stdout:9/501: unlink d12/d1b/f3b 0 2024-02-19T19:04:53.605 INFO:tasks.workunit.client.1.smithi132.stdout:0/377: write d2/dc/df/d15/d2f/d3f/d63/d6b/f75 [990089,130001] 0 2024-02-19T19:04:53.606 INFO:tasks.workunit.client.1.smithi132.stdout:2/390: truncate d0/f6 7641466 0 2024-02-19T19:04:53.608 INFO:tasks.workunit.client.1.smithi132.stdout:4/344: truncate d4/d12/d48/f65 1249746 0 2024-02-19T19:04:53.609 INFO:tasks.workunit.client.1.smithi132.stdout:8/399: creat d4/f83 x:0 0 0 2024-02-19T19:04:53.610 INFO:tasks.workunit.client.1.smithi132.stdout:4/345: chown d4/d27/d31/d71/d37 113 1 2024-02-19T19:04:53.611 INFO:tasks.workunit.client.1.smithi132.stdout:4/346: fsync d4/d12/f2f 0 2024-02-19T19:04:53.611 INFO:tasks.workunit.client.1.smithi132.stdout:5/421: dread - d8/d2e/d71/f82 zero size 2024-02-19T19:04:53.612 INFO:tasks.workunit.client.1.smithi132.stdout:5/422: write d8/f62 [4120867,16707] 0 2024-02-19T19:04:53.614 INFO:tasks.workunit.client.1.smithi132.stdout:2/391: dread d0/d4/d11/f65 [0,4194304] 0 2024-02-19T19:04:53.615 INFO:tasks.workunit.client.1.smithi132.stdout:6/345: mknod d0/db/d25/c80 0 2024-02-19T19:04:53.616 INFO:tasks.workunit.client.1.smithi132.stdout:2/392: readlink d0/d4/l31 0 2024-02-19T19:04:53.619 INFO:tasks.workunit.client.1.smithi132.stdout:3/328: dread d8/f12 [0,4194304] 0 2024-02-19T19:04:53.619 INFO:tasks.workunit.client.1.smithi132.stdout:9/502: mkdir d12/d2a/d4c/dad/db2 0 2024-02-19T19:04:53.622 INFO:tasks.workunit.client.1.smithi132.stdout:4/347: unlink d4/d12/d28/f52 0 2024-02-19T19:04:53.623 INFO:tasks.workunit.client.1.smithi132.stdout:0/378: dread d2/f19 [0,4194304] 0 2024-02-19T19:04:53.624 INFO:tasks.workunit.client.1.smithi132.stdout:9/503: write d12/d8d/d3a/f81 [879244,101995] 0 2024-02-19T19:04:53.626 INFO:tasks.workunit.client.1.smithi132.stdout:4/348: chown d4/d27/d31/d32/d51/d5c 0 1 2024-02-19T19:04:53.627 INFO:tasks.workunit.client.1.smithi132.stdout:5/423: dwrite d8/d23/d29/f2d [0,4194304] 0 2024-02-19T19:04:53.628 INFO:tasks.workunit.client.1.smithi132.stdout:9/504: fdatasync d12/d2a/d4c/d61/f6b 0 2024-02-19T19:04:53.629 INFO:tasks.workunit.client.1.smithi132.stdout:0/379: read d2/dc/df/d27/f61 [1039789,19930] 0 2024-02-19T19:04:53.635 INFO:tasks.workunit.client.1.smithi132.stdout:5/424: write d8/d23/f43 [1071417,58056] 0 2024-02-19T19:04:53.636 INFO:tasks.workunit.client.1.smithi132.stdout:4/349: read d4/fa [2866679,92201] 0 2024-02-19T19:04:53.637 INFO:tasks.workunit.client.1.smithi132.stdout:6/346: dwrite d0/db/d25/d3a/d3d/d69/f5f [0,4194304] 0 2024-02-19T19:04:53.638 INFO:tasks.workunit.client.1.smithi132.stdout:5/425: chown d8/d2e/l86 8388114 1 2024-02-19T19:04:53.638 INFO:tasks.workunit.client.1.smithi132.stdout:3/329: creat d8/d13/d2f/f62 x:0 0 0 2024-02-19T19:04:53.640 INFO:tasks.workunit.client.1.smithi132.stdout:1/376: rename d1/l10 to d1/d1e/l71 0 2024-02-19T19:04:53.640 INFO:tasks.workunit.client.1.smithi132.stdout:0/380: write d2/dc/df/d27/f56 [993479,22477] 0 2024-02-19T19:04:53.642 INFO:tasks.workunit.client.1.smithi132.stdout:3/330: write d8/d38/d44/f4c [2196590,108677] 0 2024-02-19T19:04:53.645 INFO:tasks.workunit.client.1.smithi132.stdout:8/400: dwrite d4/fd [0,4194304] 0 2024-02-19T19:04:53.659 INFO:tasks.workunit.client.1.smithi132.stdout:6/347: chown d0/f12 37674219 1 2024-02-19T19:04:53.664 INFO:tasks.workunit.client.1.smithi132.stdout:9/505: rename d12/d8d/d3a/d40/d4e/d77/d9d/fa7 to d12/d2a/d4c/fb3 0 2024-02-19T19:04:53.667 INFO:tasks.workunit.client.1.smithi132.stdout:8/401: mknod d4/dc/d1b/d1d/c84 0 2024-02-19T19:04:53.668 INFO:tasks.workunit.client.1.smithi132.stdout:8/402: fsync d4/dc/d1b/d1d/f77 0 2024-02-19T19:04:53.670 INFO:tasks.workunit.client.1.smithi132.stdout:2/393: getdents d0/d19 0 2024-02-19T19:04:53.672 INFO:tasks.workunit.client.1.smithi132.stdout:6/348: unlink d0/f70 0 2024-02-19T19:04:53.672 INFO:tasks.workunit.client.1.smithi132.stdout:2/394: read d0/d4/d11/f32 [990567,63308] 0 2024-02-19T19:04:53.674 INFO:tasks.workunit.client.1.smithi132.stdout:6/349: truncate d0/d3c/d22/f59 261805 0 2024-02-19T19:04:53.674 INFO:tasks.workunit.client.1.smithi132.stdout:8/403: dwrite d4/dc/d1b/d54/d5e/f51 [0,4194304] 0 2024-02-19T19:04:53.675 INFO:tasks.workunit.client.1.smithi132.stdout:6/350: chown d0/d3c/l76 89156075 1 2024-02-19T19:04:53.675 INFO:tasks.workunit.client.1.smithi132.stdout:4/350: rename d4/d27/d31/d32/f4e to d4/d12/d48/f7c 0 2024-02-19T19:04:53.678 INFO:tasks.workunit.client.1.smithi132.stdout:2/395: dread d0/d19/d1e/f62 [0,4194304] 0 2024-02-19T19:04:53.679 INFO:tasks.workunit.client.1.smithi132.stdout:9/506: creat d12/d8d/d3a/d40/d4e/fb4 x:0 0 0 2024-02-19T19:04:53.691 INFO:tasks.workunit.client.1.smithi132.stdout:0/381: mkdir d2/d6e/d7a 0 2024-02-19T19:04:53.694 INFO:tasks.workunit.client.1.smithi132.stdout:6/351: rmdir d0/db/d25/d3a/d3d/d69 39 2024-02-19T19:04:53.699 INFO:tasks.workunit.client.1.smithi132.stdout:9/507: creat d12/d8d/d3a/d85/d87/fb5 x:0 0 0 2024-02-19T19:04:53.699 INFO:tasks.workunit.client.1.smithi132.stdout:1/377: rmdir d1/d1e/d29/d2c/d62 39 2024-02-19T19:04:53.700 INFO:tasks.workunit.client.1.smithi132.stdout:1/378: dread - d1/d1e/d29/d2c/d43/f5d zero size 2024-02-19T19:04:53.702 INFO:tasks.workunit.client.1.smithi132.stdout:5/426: rename d8/d1a/c53 to d8/da/d5a/d7b/c89 0 2024-02-19T19:04:53.702 INFO:tasks.workunit.client.1.smithi132.stdout:2/396: write d0/d4/d11/d2e/f3e [464401,47228] 0 2024-02-19T19:04:53.703 INFO:tasks.workunit.client.1.smithi132.stdout:8/404: creat d4/dc/d1b/d20/d23/d68/d79/f85 x:0 0 0 2024-02-19T19:04:53.704 INFO:tasks.workunit.client.1.smithi132.stdout:8/405: chown d4/dc/d1b/d1d/l61 26716 1 2024-02-19T19:04:53.705 INFO:tasks.workunit.client.1.smithi132.stdout:3/331: getdents d8/d13/d2f 0 2024-02-19T19:04:53.705 INFO:tasks.workunit.client.1.smithi132.stdout:9/508: dread - d12/d1b/f3f zero size 2024-02-19T19:04:53.706 INFO:tasks.workunit.client.1.smithi132.stdout:0/382: getdents d2/dc/d4a/d74 0 2024-02-19T19:04:53.707 INFO:tasks.workunit.client.1.smithi132.stdout:2/397: mkdir d0/d4/df/d40/d7b 0 2024-02-19T19:04:53.710 INFO:tasks.workunit.client.1.smithi132.stdout:0/383: write d2/dc/df/d27/d3a/f62 [808633,27100] 0 2024-02-19T19:04:53.711 INFO:tasks.workunit.client.1.smithi132.stdout:5/427: dwrite d8/d1a/f7a [0,4194304] 0 2024-02-19T19:04:53.714 INFO:tasks.workunit.client.1.smithi132.stdout:4/351: link d4/d27/d31/f79 d4/f7d 0 2024-02-19T19:04:53.714 INFO:tasks.workunit.client.1.smithi132.stdout:5/428: write d8/da/db/f4f [1008073,103926] 0 2024-02-19T19:04:53.715 INFO:tasks.workunit.client.1.smithi132.stdout:1/379: creat d1/f72 x:0 0 0 2024-02-19T19:04:53.717 INFO:tasks.workunit.client.1.smithi132.stdout:5/429: fdatasync d8/d23/f6f 0 2024-02-19T19:04:53.717 INFO:tasks.workunit.client.1.smithi132.stdout:8/406: dwrite d4/dc/d1b/d33/f34 [0,4194304] 0 2024-02-19T19:04:53.719 INFO:tasks.workunit.client.1.smithi132.stdout:5/430: readlink d8/d1a/d35/d77/l7e 0 2024-02-19T19:04:53.719 INFO:tasks.workunit.client.1.smithi132.stdout:3/332: symlink d8/d13/d2b/d52/d34/l63 0 2024-02-19T19:04:53.720 INFO:tasks.workunit.client.1.smithi132.stdout:5/431: read d8/d1a/d35/f4b [2264133,102065] 0 2024-02-19T19:04:53.721 INFO:tasks.workunit.client.1.smithi132.stdout:5/432: stat d8/da/db/f44 0 2024-02-19T19:04:53.723 INFO:tasks.workunit.client.1.smithi132.stdout:5/433: read - d8/d2e/d71/f80 zero size 2024-02-19T19:04:53.724 INFO:tasks.workunit.client.1.smithi132.stdout:0/384: dwrite d2/f6 [4194304,4194304] 0 2024-02-19T19:04:53.725 INFO:tasks.workunit.client.1.smithi132.stdout:0/385: truncate d2/dc/d4a/d25/d43/f6f 424280 0 2024-02-19T19:04:53.736 INFO:tasks.workunit.client.1.smithi132.stdout:6/352: truncate d0/db/d25/d38/f68 2138866 0 2024-02-19T19:04:53.738 INFO:tasks.workunit.client.1.smithi132.stdout:9/509: write d12/d1b/f3f [839707,74250] 0 2024-02-19T19:04:53.738 INFO:tasks.workunit.client.1.smithi132.stdout:1/380: creat d1/d1e/d29/f73 x:0 0 0 2024-02-19T19:04:53.739 INFO:tasks.workunit.client.1.smithi132.stdout:3/333: creat d8/d13/d2b/d52/d34/d42/f64 x:0 0 0 2024-02-19T19:04:53.740 INFO:tasks.workunit.client.1.smithi132.stdout:5/434: rmdir d8/da/d5a/d7b 39 2024-02-19T19:04:53.740 INFO:tasks.workunit.client.1.smithi132.stdout:3/334: fsync d8/d13/d2b/d52/f55 0 2024-02-19T19:04:53.741 INFO:tasks.workunit.client.1.smithi132.stdout:0/386: mknod d2/dc/df/d15/d34/c7b 0 2024-02-19T19:04:53.745 INFO:tasks.workunit.client.1.smithi132.stdout:4/352: mkdir d4/d27/d31/d71/d14/d24/d49/d7e 0 2024-02-19T19:04:53.752 INFO:tasks.workunit.client.1.smithi132.stdout:7/451: sync 2024-02-19T19:04:53.752 INFO:tasks.workunit.client.1.smithi132.stdout:5/435: symlink d8/d1a/d35/d77/l8a 0 2024-02-19T19:04:53.753 INFO:tasks.workunit.client.1.smithi132.stdout:5/436: fsync d8/d23/d29/d3f/f59 0 2024-02-19T19:04:53.753 INFO:tasks.workunit.client.1.smithi132.stdout:0/387: creat d2/dc/df/d27/f7c x:0 0 0 2024-02-19T19:04:53.754 INFO:tasks.workunit.client.1.smithi132.stdout:2/398: link d0/d4/c10 d0/d24/d58/d53/c7c 0 2024-02-19T19:04:53.754 INFO:tasks.workunit.client.1.smithi132.stdout:0/388: chown d2/dc/d4a/f42 80 1 2024-02-19T19:04:53.756 INFO:tasks.workunit.client.1.smithi132.stdout:6/353: unlink d0/dc/cf 0 2024-02-19T19:04:53.758 INFO:tasks.workunit.client.1.smithi132.stdout:4/353: write d4/d12/d48/f7c [587722,130164] 0 2024-02-19T19:04:53.758 INFO:tasks.workunit.client.1.smithi132.stdout:6/354: truncate d0/d3c/f48 2012507 0 2024-02-19T19:04:53.760 INFO:tasks.workunit.client.1.smithi132.stdout:8/407: rename d4/l14 to d4/dc/d1b/d20/l86 0 2024-02-19T19:04:53.761 INFO:tasks.workunit.client.1.smithi132.stdout:2/399: dwrite d0/d3c/f60 [0,4194304] 0 2024-02-19T19:04:53.763 INFO:tasks.workunit.client.1.smithi132.stdout:9/510: mkdir d12/db6 0 2024-02-19T19:04:53.763 INFO:tasks.workunit.client.1.smithi132.stdout:2/400: write d0/d4/d11/f1c [2954851,113311] 0 2024-02-19T19:04:53.766 INFO:tasks.workunit.client.1.smithi132.stdout:7/452: mkdir d1b/d46/d76/d94 0 2024-02-19T19:04:53.767 INFO:tasks.workunit.client.1.smithi132.stdout:4/354: dread d4/d12/f2c [0,4194304] 0 2024-02-19T19:04:53.767 INFO:tasks.workunit.client.1.smithi132.stdout:5/437: creat d8/d1a/f8b x:0 0 0 2024-02-19T19:04:53.770 INFO:tasks.workunit.client.1.smithi132.stdout:7/453: dread - d1b/d5a/d63/d6b/d79/d84/f92 zero size 2024-02-19T19:04:53.771 INFO:tasks.workunit.client.1.smithi132.stdout:8/408: dwrite d4/dc/f4e [0,4194304] 0 2024-02-19T19:04:53.772 INFO:tasks.workunit.client.1.smithi132.stdout:7/454: chown d1b/d1e/d2b/d8f/c91 3 1 2024-02-19T19:04:53.772 INFO:tasks.workunit.client.1.smithi132.stdout:6/355: read - d0/db/d25/d3a/d3d/d69/f4e zero size 2024-02-19T19:04:53.774 INFO:tasks.workunit.client.1.smithi132.stdout:3/335: rename d8/d13/l1f to d8/d5d/l65 0 2024-02-19T19:04:53.779 INFO:tasks.workunit.client.1.smithi132.stdout:1/381: creat d1/d1e/f74 x:0 0 0 2024-02-19T19:04:53.781 INFO:tasks.workunit.client.1.smithi132.stdout:8/409: dwrite d4/f6 [0,4194304] 0 2024-02-19T19:04:53.782 INFO:tasks.workunit.client.1.smithi132.stdout:0/389: symlink d2/dc/d4a/d58/l7d 0 2024-02-19T19:04:53.783 INFO:tasks.workunit.client.1.smithi132.stdout:8/410: readlink d4/dc/d1b/d33/l64 0 2024-02-19T19:04:53.786 INFO:tasks.workunit.client.1.smithi132.stdout:8/411: write d4/f6 [3934871,126033] 0 2024-02-19T19:04:53.786 INFO:tasks.workunit.client.1.smithi132.stdout:3/336: dwrite d8/d13/d19/f2d [0,4194304] 0 2024-02-19T19:04:53.788 INFO:tasks.workunit.client.1.smithi132.stdout:3/337: readlink l5 0 2024-02-19T19:04:53.789 INFO:tasks.workunit.client.1.smithi132.stdout:8/412: fdatasync d4/dc/d1b/d1d/f21 0 2024-02-19T19:04:53.791 INFO:tasks.workunit.client.1.smithi132.stdout:8/413: write d4/d2d/f80 [458878,41275] 0 2024-02-19T19:04:53.791 INFO:tasks.workunit.client.1.smithi132.stdout:7/455: rename d1b/d1e/d2b/d62/d53/f70 to d1b/d1e/d2b/f95 0 2024-02-19T19:04:53.795 INFO:tasks.workunit.client.1.smithi132.stdout:2/401: unlink d0/l57 0 2024-02-19T19:04:53.802 INFO:tasks.workunit.client.1.smithi132.stdout:9/511: mknod d12/d2a/cb7 0 2024-02-19T19:04:53.802 INFO:tasks.workunit.client.1.smithi132.stdout:0/390: symlink d2/dc/df/d15/d2f/d3f/d63/l7e 0 2024-02-19T19:04:53.804 INFO:tasks.workunit.client.1.smithi132.stdout:3/338: creat d8/d13/d2f/f66 x:0 0 0 2024-02-19T19:04:53.805 INFO:tasks.workunit.client.1.smithi132.stdout:8/414: chown d4/dc/d1b/d20/l86 183 1 2024-02-19T19:04:53.808 INFO:tasks.workunit.client.1.smithi132.stdout:5/438: creat d8/da/f8c x:0 0 0 2024-02-19T19:04:53.809 INFO:tasks.workunit.client.1.smithi132.stdout:0/391: dwrite d2/dc/df/d15/f4b [0,4194304] 0 2024-02-19T19:04:53.812 INFO:tasks.workunit.client.1.smithi132.stdout:2/402: dread d0/d4/f22 [4194304,4194304] 0 2024-02-19T19:04:53.813 INFO:tasks.workunit.client.1.smithi132.stdout:3/339: mknod d8/d13/d2b/d52/c67 0 2024-02-19T19:04:53.815 INFO:tasks.workunit.client.1.smithi132.stdout:8/415: unlink d4/dc/d1b/d20/l86 0 2024-02-19T19:04:53.818 INFO:tasks.workunit.client.1.smithi132.stdout:4/355: getdents d4/d12 0 2024-02-19T19:04:53.819 INFO:tasks.workunit.client.1.smithi132.stdout:6/356: getdents d0 0 2024-02-19T19:04:53.819 INFO:tasks.workunit.client.1.smithi132.stdout:4/356: chown d4/f10 0 1 2024-02-19T19:04:53.825 INFO:tasks.workunit.client.1.smithi132.stdout:4/357: dwrite d4/f9 [0,4194304] 0 2024-02-19T19:04:53.825 INFO:tasks.workunit.client.1.smithi132.stdout:3/340: symlink d8/d13/d2b/d60/l68 0 2024-02-19T19:04:53.826 INFO:tasks.workunit.client.1.smithi132.stdout:3/341: fsync d8/f22 0 2024-02-19T19:04:53.826 INFO:tasks.workunit.client.1.smithi132.stdout:7/456: write d1b/d1e/d2b/d62/f67 [745725,33407] 0 2024-02-19T19:04:53.829 INFO:tasks.workunit.client.1.smithi132.stdout:6/357: creat d0/dc/f81 x:0 0 0 2024-02-19T19:04:53.835 INFO:tasks.workunit.client.1.smithi132.stdout:0/392: dread d2/dc/df/d15/d34/f41 [0,4194304] 0 2024-02-19T19:04:53.835 INFO:tasks.workunit.client.1.smithi132.stdout:2/403: rmdir d0/d4/d11/d2e/d5a 39 2024-02-19T19:04:53.836 INFO:tasks.workunit.client.1.smithi132.stdout:7/457: mkdir d1b/d5a/d63/d96 0 2024-02-19T19:04:53.840 INFO:tasks.workunit.client.1.smithi132.stdout:8/416: creat d4/dc/d1b/d54/f87 x:0 0 0 2024-02-19T19:04:53.841 INFO:tasks.workunit.client.1.smithi132.stdout:3/342: mkdir d8/d13/d2b/d69 0 2024-02-19T19:04:53.841 INFO:tasks.workunit.client.1.smithi132.stdout:5/439: dwrite d8/d1a/d35/f40 [0,4194304] 0 2024-02-19T19:04:53.843 INFO:tasks.workunit.client.1.smithi132.stdout:0/393: mkdir d2/dc/d4a/d25/d7f 0 2024-02-19T19:04:53.844 INFO:tasks.workunit.client.1.smithi132.stdout:2/404: symlink d0/d4/df/d40/l7d 0 2024-02-19T19:04:53.845 INFO:tasks.workunit.client.1.smithi132.stdout:7/458: creat d1b/d1e/d28/f97 x:0 0 0 2024-02-19T19:04:53.845 INFO:tasks.workunit.client.1.smithi132.stdout:0/394: truncate d2/dc/f6d 275222 0 2024-02-19T19:04:53.846 INFO:tasks.workunit.client.1.smithi132.stdout:4/358: dread d4/f38 [0,4194304] 0 2024-02-19T19:04:53.847 INFO:tasks.workunit.client.1.smithi132.stdout:6/358: write d0/d3c/d22/f23 [436953,112249] 0 2024-02-19T19:04:53.848 INFO:tasks.workunit.client.1.smithi132.stdout:6/359: fsync d0/d3c/d22/f59 0 2024-02-19T19:04:53.848 INFO:tasks.workunit.client.1.smithi132.stdout:8/417: mkdir d4/dc/d1b/d38/d69/d88 0 2024-02-19T19:04:53.848 INFO:tasks.workunit.client.1.smithi132.stdout:8/418: readlink d4/dc/l78 0 2024-02-19T19:04:53.849 INFO:tasks.workunit.client.1.smithi132.stdout:7/459: read d1b/d1e/f22 [5521366,90409] 0 2024-02-19T19:04:53.851 INFO:tasks.workunit.client.1.smithi132.stdout:8/419: stat d4/dc/d1b/d54/d5e/f67 0 2024-02-19T19:04:53.852 INFO:tasks.workunit.client.1.smithi132.stdout:7/460: readlink d1b/d1e/l2d 0 2024-02-19T19:04:53.852 INFO:tasks.workunit.client.1.smithi132.stdout:0/395: read d2/f53 [344034,80061] 0 2024-02-19T19:04:53.856 INFO:tasks.workunit.client.1.smithi132.stdout:8/420: write d4/dc/d1b/d20/d23/f28 [2084669,43286] 0 2024-02-19T19:04:53.856 INFO:tasks.workunit.client.1.smithi132.stdout:7/461: write d1b/d1e/d41/f6d [308698,70211] 0 2024-02-19T19:04:53.856 INFO:tasks.workunit.client.1.smithi132.stdout:2/405: rmdir d0/d4/d11/d2e/d5a/d66 39 2024-02-19T19:04:53.859 INFO:tasks.workunit.client.1.smithi132.stdout:5/440: dwrite d8/d1a/f6c [0,4194304] 0 2024-02-19T19:04:53.859 INFO:tasks.workunit.client.1.smithi132.stdout:3/343: dread d8/d13/f20 [0,4194304] 0 2024-02-19T19:04:53.859 INFO:tasks.workunit.client.1.smithi132.stdout:0/396: write d2/dc/df/d15/d34/f3c [3788961,75840] 0 2024-02-19T19:04:53.860 INFO:tasks.workunit.client.1.smithi132.stdout:4/359: mkdir d4/d27/d7f 0 2024-02-19T19:04:53.861 INFO:tasks.workunit.client.1.smithi132.stdout:6/360: symlink d0/d3c/d22/d79/l82 0 2024-02-19T19:04:53.861 INFO:tasks.workunit.client.1.smithi132.stdout:4/360: stat d4/d19/d21/d56/f7b 0 2024-02-19T19:04:53.862 INFO:tasks.workunit.client.1.smithi132.stdout:3/344: truncate d8/d13/f24 1187364 0 2024-02-19T19:04:53.863 INFO:tasks.workunit.client.1.smithi132.stdout:4/361: readlink d4/l3d 0 2024-02-19T19:04:53.866 INFO:tasks.workunit.client.1.smithi132.stdout:7/462: mkdir d1b/d5a/d63/d98 0 2024-02-19T19:04:53.867 INFO:tasks.workunit.client.1.smithi132.stdout:5/441: mknod d8/d1a/d35/d77/c8d 0 2024-02-19T19:04:53.870 INFO:tasks.workunit.client.1.smithi132.stdout:8/421: symlink d4/dc/d1b/d20/d23/d68/d79/l89 0 2024-02-19T19:04:53.871 INFO:tasks.workunit.client.1.smithi132.stdout:6/361: dwrite d0/d3c/d46/f7b [4194304,4194304] 0 2024-02-19T19:04:53.872 INFO:tasks.workunit.client.1.smithi132.stdout:0/397: dwrite d2/dc/d4a/d25/d43/d5f/f73 [0,4194304] 0 2024-02-19T19:04:53.874 INFO:tasks.workunit.client.1.smithi132.stdout:6/362: truncate d0/dc/f66 371856 0 2024-02-19T19:04:53.875 INFO:tasks.workunit.client.1.smithi132.stdout:3/345: mkdir d8/d38/d44/d6a 0 2024-02-19T19:04:53.875 INFO:tasks.workunit.client.1.smithi132.stdout:7/463: mkdir d1b/d1e/d28/d5e/d99 0 2024-02-19T19:04:53.877 INFO:tasks.workunit.client.1.smithi132.stdout:8/422: unlink d4/dc/d1b/d20/d23/d68/l7f 0 2024-02-19T19:04:53.879 INFO:tasks.workunit.client.1.smithi132.stdout:0/398: mknod d2/d6e/c80 0 2024-02-19T19:04:53.880 INFO:tasks.workunit.client.1.smithi132.stdout:4/362: rename d4/l18 to d4/d27/d31/l80 0 2024-02-19T19:04:53.883 INFO:tasks.workunit.client.1.smithi132.stdout:3/346: symlink d8/d5d/l6b 0 2024-02-19T19:04:53.886 INFO:tasks.workunit.client.1.smithi132.stdout:6/363: getdents d0/db/d25/d2f/d77 0 2024-02-19T19:04:53.888 INFO:tasks.workunit.client.1.smithi132.stdout:0/399: mknod d2/dc/d4a/d25/c81 0 2024-02-19T19:04:53.889 INFO:tasks.workunit.client.1.smithi132.stdout:3/347: read d8/d13/d2f/f47 [594771,123055] 0 2024-02-19T19:04:53.891 INFO:tasks.workunit.client.1.smithi132.stdout:5/442: rename d8/d23/d29/c49 to d8/d23/d29/d3f/c8e 0 2024-02-19T19:04:53.892 INFO:tasks.workunit.client.1.smithi132.stdout:3/348: write d8/fd [1935923,12381] 0 2024-02-19T19:04:53.895 INFO:tasks.workunit.client.1.smithi132.stdout:6/364: dwrite d0/d3c/d22/f7e [0,4194304] 0 2024-02-19T19:04:53.897 INFO:tasks.workunit.client.1.smithi132.stdout:8/423: mkdir d4/dc/d1b/d20/d8a 0 2024-02-19T19:04:53.903 INFO:tasks.workunit.client.1.smithi132.stdout:0/400: mknod d2/dc/df/d15/d2f/d47/c82 0 2024-02-19T19:04:53.904 INFO:tasks.workunit.client.1.smithi132.stdout:0/401: dread - d2/dc/df/d27/f7c zero size 2024-02-19T19:04:53.909 INFO:tasks.workunit.client.1.smithi132.stdout:2/406: dread d0/d4/df/f3b [0,4194304] 0 2024-02-19T19:04:53.909 INFO:tasks.workunit.client.1.smithi132.stdout:5/443: rename d8/d23/d29/f1c to d8/d1a/d35/d77/f8f 0 2024-02-19T19:04:53.911 INFO:tasks.workunit.client.1.smithi132.stdout:0/402: dwrite d2/dc/df/d15/f4e [0,4194304] 0 2024-02-19T19:04:53.911 INFO:tasks.workunit.client.1.smithi132.stdout:3/349: mkdir d8/d13/d2b/d52/d34/d42/d6c 0 2024-02-19T19:04:53.913 INFO:tasks.workunit.client.1.smithi132.stdout:7/464: link d1b/d1e/d28/l4e d1b/d5a/d63/d98/l9a 0 2024-02-19T19:04:53.913 INFO:tasks.workunit.client.1.smithi132.stdout:0/403: chown d2/dc/df/d27/d3a/d52/f68 221403160 1 2024-02-19T19:04:53.914 INFO:tasks.workunit.client.1.smithi132.stdout:6/365: symlink d0/dc/l83 0 2024-02-19T19:04:53.915 INFO:tasks.workunit.client.1.smithi132.stdout:6/366: truncate d0/f10 5833839 0 2024-02-19T19:04:53.918 INFO:tasks.workunit.client.1.smithi132.stdout:5/444: dread d8/d1a/d35/d77/f8f [0,4194304] 0 2024-02-19T19:04:53.920 INFO:tasks.workunit.client.1.smithi132.stdout:1/382: sync 2024-02-19T19:04:53.920 INFO:tasks.workunit.client.1.smithi132.stdout:1/383: chown d1/d1e/f5b 2191 1 2024-02-19T19:04:53.922 INFO:tasks.workunit.client.1.smithi132.stdout:4/363: truncate d4/d12/d28/f62 1748162 0 2024-02-19T19:04:53.922 INFO:tasks.workunit.client.1.smithi132.stdout:3/350: rename d8/d13/f24 to d8/d13/d2b/d69/f6d 0 2024-02-19T19:04:53.922 INFO:tasks.workunit.client.1.smithi132.stdout:4/364: fsync d4/f2d 0 2024-02-19T19:04:53.924 INFO:tasks.workunit.client.1.smithi132.stdout:3/351: truncate d8/d13/d2b/d69/f6d 1192363 0 2024-02-19T19:04:53.926 INFO:tasks.workunit.client.1.smithi132.stdout:4/365: read d4/d27/d31/d71/d14/d24/f3b [1995404,85378] 0 2024-02-19T19:04:53.926 INFO:tasks.workunit.client.1.smithi132.stdout:0/404: rmdir d2/dc/df/d15 39 2024-02-19T19:04:53.932 INFO:tasks.workunit.client.1.smithi132.stdout:2/407: creat d0/d64/d6d/d76/f7e x:0 0 0 2024-02-19T19:04:53.943 INFO:tasks.workunit.client.1.smithi132.stdout:4/366: creat d4/d12/d28/f81 x:0 0 0 2024-02-19T19:04:53.946 INFO:tasks.workunit.client.1.smithi132.stdout:8/424: creat d4/dc/d1b/d20/f8b x:0 0 0 2024-02-19T19:04:53.948 INFO:tasks.workunit.client.1.smithi132.stdout:4/367: dwrite d4/f10 [0,4194304] 0 2024-02-19T19:04:53.949 INFO:tasks.workunit.client.1.smithi132.stdout:6/367: link d0/db/d25/d3a/d3d/f61 d0/db/d25/d2f/f84 0 2024-02-19T19:04:53.949 INFO:tasks.workunit.client.1.smithi132.stdout:7/465: truncate d1b/d1e/f37 663261 0 2024-02-19T19:04:53.952 INFO:tasks.workunit.client.1.smithi132.stdout:1/384: mkdir d1/d1e/d29/d52/d64/d75 0 2024-02-19T19:04:53.953 INFO:tasks.workunit.client.1.smithi132.stdout:3/352: getdents d8/d13/d57 0 2024-02-19T19:04:53.956 INFO:tasks.workunit.client.1.smithi132.stdout:7/466: symlink d1b/d5a/l9b 0 2024-02-19T19:04:53.957 INFO:tasks.workunit.client.1.smithi132.stdout:7/467: chown d1b/d1e/d28/f3c 0 1 2024-02-19T19:04:53.958 INFO:tasks.workunit.client.1.smithi132.stdout:5/445: rmdir d8/d1a/d35/d5b 0 2024-02-19T19:04:53.959 INFO:tasks.workunit.client.1.smithi132.stdout:3/353: dread d8/d13/f37 [0,4194304] 0 2024-02-19T19:04:53.959 INFO:tasks.workunit.client.1.smithi132.stdout:5/446: dread - d8/d1a/d64/f72 zero size 2024-02-19T19:04:53.960 INFO:tasks.workunit.client.1.smithi132.stdout:2/408: creat d0/d4/d11/d2e/d5a/d66/f7f x:0 0 0 2024-02-19T19:04:53.962 INFO:tasks.workunit.client.1.smithi132.stdout:8/425: link d4/f6 d4/dc/d1b/d33/f8c 0 2024-02-19T19:04:53.964 INFO:tasks.workunit.client.1.smithi132.stdout:6/368: unlink d0/d51/l6c 0 2024-02-19T19:04:53.964 INFO:tasks.workunit.client.1.smithi132.stdout:5/447: mknod d8/d2e/c90 0 2024-02-19T19:04:53.966 INFO:tasks.workunit.client.1.smithi132.stdout:7/468: dwrite d1b/d5a/d63/d6b/d79/d84/f92 [0,4194304] 0 2024-02-19T19:04:54.007 INFO:tasks.workunit.client.1.smithi132.stdout:8/426: mknod d4/dc/d1b/d1d/c8d 0 2024-02-19T19:04:54.008 INFO:tasks.workunit.client.1.smithi132.stdout:9/512: sync 2024-02-19T19:04:54.010 INFO:tasks.workunit.client.1.smithi132.stdout:8/427: write d4/fd [61600,94893] 0 2024-02-19T19:04:54.010 INFO:tasks.workunit.client.1.smithi132.stdout:3/354: dwrite d8/d38/d44/f5e [0,4194304] 0 2024-02-19T19:04:54.010 INFO:tasks.workunit.client.1.smithi132.stdout:5/448: dread d8/d23/f6f [0,4194304] 0 2024-02-19T19:04:54.013 INFO:tasks.workunit.client.1.smithi132.stdout:8/428: stat d4/dc/d1b/d20/d23/l3b 0 2024-02-19T19:04:54.013 INFO:tasks.workunit.client.1.smithi132.stdout:3/355: chown d8/d13/d2b/f33 1321 1 2024-02-19T19:04:54.013 INFO:tasks.workunit.client.1.smithi132.stdout:4/368: link d4/d12/d16/c1f d4/d27/d31/d32/d51/c82 0 2024-02-19T19:04:54.015 INFO:tasks.workunit.client.1.smithi132.stdout:2/409: creat d0/f80 x:0 0 0 2024-02-19T19:04:54.016 INFO:tasks.workunit.client.1.smithi132.stdout:9/513: unlink f5 0 2024-02-19T19:04:54.018 INFO:tasks.workunit.client.1.smithi132.stdout:5/449: creat d8/d2e/f91 x:0 0 0 2024-02-19T19:04:54.018 INFO:tasks.workunit.client.1.smithi132.stdout:7/469: dread d1b/d5a/d63/d6b/d79/d84/f92 [0,4194304] 0 2024-02-19T19:04:54.019 INFO:tasks.workunit.client.1.smithi132.stdout:3/356: mkdir d8/d38/d44/d6e 0 2024-02-19T19:04:54.020 INFO:tasks.workunit.client.1.smithi132.stdout:5/450: truncate d8/d2e/d71/f82 925195 0 2024-02-19T19:04:54.022 INFO:tasks.workunit.client.1.smithi132.stdout:5/451: truncate d8/d1a/d64/f72 651571 0 2024-02-19T19:04:54.022 INFO:tasks.workunit.client.1.smithi132.stdout:8/429: unlink d4/d2d/f80 0 2024-02-19T19:04:54.025 INFO:tasks.workunit.client.1.smithi132.stdout:5/452: truncate d8/d23/d3b/f65 146068 0 2024-02-19T19:04:54.026 INFO:tasks.workunit.client.1.smithi132.stdout:7/470: dwrite d1b/d1e/f21 [0,4194304] 0 2024-02-19T19:04:54.027 INFO:tasks.workunit.client.1.smithi132.stdout:7/471: write d1b/d5a/f68 [3797738,91404] 0 2024-02-19T19:04:54.029 INFO:tasks.workunit.client.1.smithi132.stdout:9/514: mkdir d12/d2a/d88/db8 0 2024-02-19T19:04:54.033 INFO:tasks.workunit.client.1.smithi132.stdout:3/357: dread - d8/d13/d2b/d52/d34/d50/f5b zero size 2024-02-19T19:04:54.036 INFO:tasks.workunit.client.1.smithi132.stdout:2/410: creat d0/d3c/d3f/f81 x:0 0 0 2024-02-19T19:04:54.037 INFO:tasks.workunit.client.1.smithi132.stdout:5/453: mknod d8/d1a/d35/d77/c92 0 2024-02-19T19:04:54.037 INFO:tasks.workunit.client.1.smithi132.stdout:4/369: dread d4/f2d [0,4194304] 0 2024-02-19T19:04:54.038 INFO:tasks.workunit.client.1.smithi132.stdout:6/369: getdents d0/db/d25/d2f 0 2024-02-19T19:04:54.039 INFO:tasks.workunit.client.1.smithi132.stdout:4/370: fdatasync d4/d12/d16/f6c 0 2024-02-19T19:04:54.047 INFO:tasks.workunit.client.1.smithi132.stdout:4/371: dwrite d4/d4d/f64 [0,4194304] 0 2024-02-19T19:04:54.047 INFO:tasks.workunit.client.1.smithi132.stdout:6/370: write d0/db/d25/d2f/f4a [206318,78221] 0 2024-02-19T19:04:54.047 INFO:tasks.workunit.client.1.smithi132.stdout:3/358: dread f0 [0,4194304] 0 2024-02-19T19:04:54.048 INFO:tasks.workunit.client.1.smithi132.stdout:5/454: rename d8/da/db/f1b to d8/d23/d29/d3f/f93 0 2024-02-19T19:04:54.050 INFO:tasks.workunit.client.1.smithi132.stdout:2/411: creat d0/d19/f82 x:0 0 0 2024-02-19T19:04:54.052 INFO:tasks.workunit.client.1.smithi132.stdout:3/359: creat d8/d13/d2b/d2e/f6f x:0 0 0 2024-02-19T19:04:54.055 INFO:tasks.workunit.client.1.smithi132.stdout:4/372: creat d4/d27/d31/d71/d14/d24/d49/f83 x:0 0 0 2024-02-19T19:04:54.055 INFO:tasks.workunit.client.1.smithi132.stdout:5/455: symlink d8/d23/d29/l94 0 2024-02-19T19:04:54.056 INFO:tasks.workunit.client.1.smithi132.stdout:4/373: fsync d4/d19/f3f 0 2024-02-19T19:04:54.056 INFO:tasks.workunit.client.1.smithi132.stdout:5/456: write d8/d23/f6f [1346033,115313] 0 2024-02-19T19:04:54.058 INFO:tasks.workunit.client.1.smithi132.stdout:2/412: mknod d0/d4/d11/d2e/d5a/d66/c83 0 2024-02-19T19:04:54.060 INFO:tasks.workunit.client.1.smithi132.stdout:2/413: readlink d0/d4/df/d40/l7d 0 2024-02-19T19:04:54.062 INFO:tasks.workunit.client.1.smithi132.stdout:4/374: dwrite d4/d12/d48/f7c [0,4194304] 0 2024-02-19T19:04:54.064 INFO:tasks.workunit.client.1.smithi132.stdout:4/375: write d4/d12/d16/f6c [446548,109870] 0 2024-02-19T19:04:54.064 INFO:tasks.workunit.client.1.smithi132.stdout:3/360: rename d8/d13/d19/l39 to d8/d13/d19/l70 0 2024-02-19T19:04:54.068 INFO:tasks.workunit.client.1.smithi132.stdout:8/430: rmdir d4/d2d 39 2024-02-19T19:04:54.072 INFO:tasks.workunit.client.1.smithi132.stdout:6/371: dread d0/db/fd [0,4194304] 0 2024-02-19T19:04:54.075 INFO:tasks.workunit.client.1.smithi132.stdout:3/361: unlink d8/d38/d44/c5c 0 2024-02-19T19:04:54.076 INFO:tasks.workunit.client.1.smithi132.stdout:7/472: write d1b/d1e/f37 [1623008,23753] 0 2024-02-19T19:04:54.076 INFO:tasks.workunit.client.1.smithi132.stdout:3/362: chown d8/d13/d2b/d52/d34/d42/f54 293 1 2024-02-19T19:04:54.082 INFO:tasks.workunit.client.1.smithi132.stdout:3/363: dwrite d8/f22 [0,4194304] 0 2024-02-19T19:04:54.084 INFO:tasks.workunit.client.1.smithi132.stdout:3/364: read d8/d13/d2b/f33 [631050,50617] 0 2024-02-19T19:04:54.084 INFO:tasks.workunit.client.1.smithi132.stdout:7/473: mknod d1b/d1e/d2b/d62/c9c 0 2024-02-19T19:04:54.086 INFO:tasks.workunit.client.1.smithi132.stdout:7/474: write d1b/f58 [701719,129875] 0 2024-02-19T19:04:54.089 INFO:tasks.workunit.client.1.smithi132.stdout:7/475: read d1b/d1e/d28/f50 [3418221,13934] 0 2024-02-19T19:04:54.093 INFO:tasks.workunit.client.1.smithi132.stdout:3/365: dwrite d8/d13/d2f/f36 [0,4194304] 0 2024-02-19T19:04:54.094 INFO:tasks.workunit.client.1.smithi132.stdout:3/366: chown d8/d13/d2b/d52/l53 2 1 2024-02-19T19:04:54.095 INFO:tasks.workunit.client.1.smithi132.stdout:3/367: write d8/d13/d2f/f66 [286977,87506] 0 2024-02-19T19:04:54.096 INFO:tasks.workunit.client.1.smithi132.stdout:4/376: write d4/d12/d16/f5a [1675522,86979] 0 2024-02-19T19:04:54.097 INFO:tasks.workunit.client.1.smithi132.stdout:6/372: link d0/d3c/d22/c33 d0/db/d25/d3a/d5d/c85 0 2024-02-19T19:04:54.099 INFO:tasks.workunit.client.1.smithi132.stdout:8/431: getdents d4/dc/d1b/d54 0 2024-02-19T19:04:54.099 INFO:tasks.workunit.client.1.smithi132.stdout:4/377: readlink d4/d27/d31/d71/d37/l69 0 2024-02-19T19:04:54.099 INFO:tasks.workunit.client.1.smithi132.stdout:6/373: dread - d0/d3c/d46/d5a/f7d zero size 2024-02-19T19:04:54.101 INFO:tasks.workunit.client.1.smithi132.stdout:8/432: fdatasync d4/dc/d1b/d54/f87 0 2024-02-19T19:04:54.102 INFO:tasks.workunit.client.1.smithi132.stdout:7/476: rename d1b/f58 to d1b/d46/d76/d94/f9d 0 2024-02-19T19:04:54.103 INFO:tasks.workunit.client.1.smithi132.stdout:0/405: sync 2024-02-19T19:04:54.103 INFO:tasks.workunit.client.1.smithi132.stdout:3/368: readlink d8/d13/l21 0 2024-02-19T19:04:54.104 INFO:tasks.workunit.client.1.smithi132.stdout:2/414: dwrite d0/d24/d58/f36 [0,4194304] 0 2024-02-19T19:04:54.104 INFO:tasks.workunit.client.1.smithi132.stdout:8/433: write d4/dc/d1b/d33/f4d [414637,44260] 0 2024-02-19T19:04:54.106 INFO:tasks.workunit.client.1.smithi132.stdout:2/415: write d0/d3c/f60 [4375500,107924] 0 2024-02-19T19:04:54.106 INFO:tasks.workunit.client.1.smithi132.stdout:4/378: fsync d4/d27/d31/f79 0 2024-02-19T19:04:54.109 INFO:tasks.workunit.client.1.smithi132.stdout:7/477: symlink d1b/d1e/d2b/d62/l9e 0 2024-02-19T19:04:54.110 INFO:tasks.workunit.client.1.smithi132.stdout:6/374: mkdir d0/db/d25/d50/d86 0 2024-02-19T19:04:54.111 INFO:tasks.workunit.client.1.smithi132.stdout:6/375: dread - d0/dc/f81 zero size 2024-02-19T19:04:54.111 INFO:tasks.workunit.client.1.smithi132.stdout:6/376: chown d0/dc/f2d 4 1 2024-02-19T19:04:54.112 INFO:tasks.workunit.client.1.smithi132.stdout:8/434: read d4/dc/d1b/d20/f59 [1029626,12101] 0 2024-02-19T19:04:54.115 INFO:tasks.workunit.client.1.smithi132.stdout:0/406: symlink d2/dc/df/d15/d2f/d3f/d63/l83 0 2024-02-19T19:04:54.116 INFO:tasks.workunit.client.1.smithi132.stdout:4/379: mknod d4/d19/d21/d56/c84 0 2024-02-19T19:04:54.119 INFO:tasks.workunit.client.1.smithi132.stdout:6/377: mkdir d0/d3c/d87 0 2024-02-19T19:04:54.121 INFO:tasks.workunit.client.1.smithi132.stdout:0/407: mkdir d2/dc/df/d15/d34/d84 0 2024-02-19T19:04:54.122 INFO:tasks.workunit.client.1.smithi132.stdout:3/369: link d8/lb d8/l71 0 2024-02-19T19:04:54.123 INFO:tasks.workunit.client.1.smithi132.stdout:4/380: write d4/d19/f61 [741577,95889] 0 2024-02-19T19:04:54.128 INFO:tasks.workunit.client.1.smithi132.stdout:6/378: dwrite d0/db/d25/f34 [0,4194304] 0 2024-02-19T19:04:54.132 INFO:tasks.workunit.client.1.smithi132.stdout:4/381: fdatasync d4/d19/f7a 0 2024-02-19T19:04:54.133 INFO:tasks.workunit.client.1.smithi132.stdout:2/416: mkdir d0/d19/d72/d84 0 2024-02-19T19:04:54.134 INFO:tasks.workunit.client.1.smithi132.stdout:3/370: dwrite d8/d38/d44/f4c [0,4194304] 0 2024-02-19T19:04:54.136 INFO:tasks.workunit.client.1.smithi132.stdout:0/408: creat d2/dc/d4a/d74/f85 x:0 0 0 2024-02-19T19:04:54.138 INFO:tasks.workunit.client.1.smithi132.stdout:7/478: dread f8 [0,4194304] 0 2024-02-19T19:04:54.138 INFO:tasks.workunit.client.1.smithi132.stdout:2/417: chown d0/d4/d11/d2e/f6c 42 1 2024-02-19T19:04:54.139 INFO:tasks.workunit.client.1.smithi132.stdout:8/435: truncate d4/dc/d1b/d33/f34 1917161 0 2024-02-19T19:04:54.140 INFO:tasks.workunit.client.1.smithi132.stdout:7/479: stat d1b/d1e/d2b/f51 0 2024-02-19T19:04:54.141 INFO:tasks.workunit.client.1.smithi132.stdout:6/379: dwrite d0/d3c/d46/f7b [4194304,4194304] 0 2024-02-19T19:04:54.143 INFO:tasks.workunit.client.1.smithi132.stdout:6/380: truncate d0/db/d25/d2f/f4a 858284 0 2024-02-19T19:04:54.144 INFO:tasks.workunit.client.1.smithi132.stdout:0/409: creat d2/dc/d4a/d74/f86 x:0 0 0 2024-02-19T19:04:54.144 INFO:tasks.workunit.client.1.smithi132.stdout:6/381: chown d0/dc/l5b 45156482 1 2024-02-19T19:04:54.149 INFO:tasks.workunit.client.1.smithi132.stdout:2/418: mknod d0/d3c/d3f/c85 0 2024-02-19T19:04:54.151 INFO:tasks.workunit.client.1.smithi132.stdout:0/410: dread d2/dc/df/d27/d3a/d52/f68 [0,4194304] 0 2024-02-19T19:04:54.152 INFO:tasks.workunit.client.1.smithi132.stdout:0/411: chown d2 22465 1 2024-02-19T19:04:54.152 INFO:tasks.workunit.client.1.smithi132.stdout:7/480: write d1b/d1e/d2b/f95 [1031594,34169] 0 2024-02-19T19:04:54.153 INFO:tasks.workunit.client.1.smithi132.stdout:6/382: dwrite d0/d3c/d22/f59 [0,4194304] 0 2024-02-19T19:04:54.154 INFO:tasks.workunit.client.1.smithi132.stdout:3/371: link d8/d13/d19/l43 d8/d13/d2b/d2e/l72 0 2024-02-19T19:04:54.156 INFO:tasks.workunit.client.1.smithi132.stdout:3/372: readlink d8/lf 0 2024-02-19T19:04:54.160 INFO:tasks.workunit.client.1.smithi132.stdout:6/383: rename d0/c1f to d0/db/d25/d3a/d5d/c88 0 2024-02-19T19:04:54.163 INFO:tasks.workunit.client.1.smithi132.stdout:2/419: creat d0/d4/df/d40/d68/f86 x:0 0 0 2024-02-19T19:04:54.163 INFO:tasks.workunit.client.1.smithi132.stdout:0/412: getdents d2/dc/d4a/d25/d7f 0 2024-02-19T19:04:54.163 INFO:tasks.workunit.client.1.smithi132.stdout:7/481: dwrite d1b/d1e/d41/f6a [0,4194304] 0 2024-02-19T19:04:54.163 INFO:tasks.workunit.client.1.smithi132.stdout:3/373: creat d8/d38/d44/f73 x:0 0 0 2024-02-19T19:04:54.165 INFO:tasks.workunit.client.1.smithi132.stdout:3/374: dread - d8/d13/d2f/f58 zero size 2024-02-19T19:04:54.166 INFO:tasks.workunit.client.1.smithi132.stdout:3/375: readlink d8/l35 0 2024-02-19T19:04:54.167 INFO:tasks.workunit.client.1.smithi132.stdout:7/482: read d1b/f1c [1273186,119435] 0 2024-02-19T19:04:54.167 INFO:tasks.workunit.client.1.smithi132.stdout:9/515: sync 2024-02-19T19:04:54.167 INFO:tasks.workunit.client.1.smithi132.stdout:1/385: sync 2024-02-19T19:04:54.173 INFO:tasks.workunit.client.1.smithi132.stdout:0/413: creat d2/dc/d4a/d25/d43/d5f/f87 x:0 0 0 2024-02-19T19:04:54.174 INFO:tasks.workunit.client.1.smithi132.stdout:4/382: truncate d4/f10 3947498 0 2024-02-19T19:04:54.175 INFO:tasks.workunit.client.1.smithi132.stdout:8/436: write d4/dc/f1c [685521,108481] 0 2024-02-19T19:04:54.180 INFO:tasks.workunit.client.1.smithi132.stdout:7/483: creat d1b/d46/d76/f9f x:0 0 0 2024-02-19T19:04:54.182 INFO:tasks.workunit.client.1.smithi132.stdout:4/383: dwrite d4/d4d/f64 [0,4194304] 0 2024-02-19T19:04:54.182 INFO:tasks.workunit.client.1.smithi132.stdout:9/516: rename d12/d3e/f72 to d12/d2a/d88/db8/fb9 0 2024-02-19T19:04:54.183 INFO:tasks.workunit.client.1.smithi132.stdout:9/517: chown d12/f23 21364 1 2024-02-19T19:04:54.184 INFO:tasks.workunit.client.1.smithi132.stdout:9/518: chown d12/d8d/d3a/d40/d4e/c83 952 1 2024-02-19T19:04:54.185 INFO:tasks.workunit.client.1.smithi132.stdout:9/519: chown d12/d8d/d3a/f81 3 1 2024-02-19T19:04:54.186 INFO:tasks.workunit.client.1.smithi132.stdout:1/386: truncate d1/d1e/d29/d2c/f34 314611 0 2024-02-19T19:04:54.187 INFO:tasks.workunit.client.1.smithi132.stdout:9/520: write d12/d8d/d3a/d40/f66 [3441186,62064] 0 2024-02-19T19:04:54.191 INFO:tasks.workunit.client.1.smithi132.stdout:3/376: symlink d8/d13/d57/l74 0 2024-02-19T19:04:54.194 INFO:tasks.workunit.client.1.smithi132.stdout:0/414: rename d2/dc/df/d27/d3a/f62 to d2/dc/df/d15/d2f/d69/f88 0 2024-02-19T19:04:54.194 INFO:tasks.workunit.client.1.smithi132.stdout:3/377: rename d8/d13 to d8/d13/d2b/d52/d34/d50/d75 22 2024-02-19T19:04:54.196 INFO:tasks.workunit.client.1.smithi132.stdout:4/384: read d4/d27/d31/d71/d14/d24/f39 [204067,30129] 0 2024-02-19T19:04:54.197 INFO:tasks.workunit.client.1.smithi132.stdout:2/420: dread d0/d4/d11/d2e/d5a/f5b [4194304,4194304] 0 2024-02-19T19:04:54.198 INFO:tasks.workunit.client.1.smithi132.stdout:6/384: fdatasync d0/db/d25/d38/f78 0 2024-02-19T19:04:54.204 INFO:tasks.workunit.client.1.smithi132.stdout:8/437: mknod d4/dc/d1b/d38/d69/d74/c8e 0 2024-02-19T19:04:54.212 INFO:tasks.workunit.client.1.smithi132.stdout:3/378: rename d8/d13/l29 to d8/d38/l76 0 2024-02-19T19:04:54.212 INFO:tasks.workunit.client.1.smithi132.stdout:3/379: chown d8/d38/d44/d6e 57041 1 2024-02-19T19:04:54.217 INFO:tasks.workunit.client.1.smithi132.stdout:2/421: unlink d0/d4/fb 0 2024-02-19T19:04:54.221 INFO:tasks.workunit.client.1.smithi132.stdout:1/387: chown d1/d3a/c47 801644266 1 2024-02-19T19:04:54.222 INFO:tasks.workunit.client.1.smithi132.stdout:9/521: mknod d12/d2a/d4c/dad/db2/cba 0 2024-02-19T19:04:54.224 INFO:tasks.workunit.client.1.smithi132.stdout:8/438: mkdir d4/dc/d1b/d8f 0 2024-02-19T19:04:54.224 INFO:tasks.workunit.client.1.smithi132.stdout:6/385: dwrite d0/d3c/d46/f7b [8388608,4194304] 0 2024-02-19T19:04:54.226 INFO:tasks.workunit.client.1.smithi132.stdout:0/415: symlink d2/d6e/d7a/l89 0 2024-02-19T19:04:54.226 INFO:tasks.workunit.client.1.smithi132.stdout:4/385: creat d4/d27/d31/d32/d51/d72/f85 x:0 0 0 2024-02-19T19:04:54.226 INFO:tasks.workunit.client.1.smithi132.stdout:4/386: stat d4/f38 0 2024-02-19T19:04:54.229 INFO:tasks.workunit.client.1.smithi132.stdout:3/380: unlink d8/f12 0 2024-02-19T19:04:54.233 INFO:tasks.workunit.client.1.smithi132.stdout:8/439: write d4/dc/d1b/d20/f6e [678607,85421] 0 2024-02-19T19:04:54.233 INFO:tasks.workunit.client.1.smithi132.stdout:6/386: mkdir d0/db/d25/d3a/d3d/d69/d89 0 2024-02-19T19:04:54.236 INFO:tasks.workunit.client.1.smithi132.stdout:1/388: mknod d1/d3a/c76 0 2024-02-19T19:04:54.238 INFO:tasks.workunit.client.1.smithi132.stdout:9/522: symlink d12/d2a/lbb 0 2024-02-19T19:04:54.240 INFO:tasks.workunit.client.1.smithi132.stdout:1/389: truncate d1/d1e/d29/f3c 386385 0 2024-02-19T19:04:54.240 INFO:tasks.workunit.client.1.smithi132.stdout:6/387: dwrite d0/db/d25/d3a/d3d/d69/f5f [4194304,4194304] 0 2024-02-19T19:04:54.240 INFO:tasks.workunit.client.1.smithi132.stdout:0/416: mknod d2/dc/d4a/c8a 0 2024-02-19T19:04:54.242 INFO:tasks.workunit.client.1.smithi132.stdout:3/381: mkdir d8/d13/d2b/d52/d34/d42/d6c/d77 0 2024-02-19T19:04:54.242 INFO:tasks.workunit.client.1.smithi132.stdout:2/422: creat d0/d24/f87 x:0 0 0 2024-02-19T19:04:54.243 INFO:tasks.workunit.client.1.smithi132.stdout:8/440: mknod d4/dc/d1b/d20/d8a/c90 0 2024-02-19T19:04:54.248 INFO:tasks.workunit.client.1.smithi132.stdout:3/382: mknod d8/d13/d2b/d60/c78 0 2024-02-19T19:04:54.248 INFO:tasks.workunit.client.1.smithi132.stdout:6/388: dwrite d0/db/f58 [0,4194304] 0 2024-02-19T19:04:54.250 INFO:tasks.workunit.client.1.smithi132.stdout:5/457: sync 2024-02-19T19:04:54.250 INFO:tasks.workunit.client.1.smithi132.stdout:3/383: fdatasync d8/d13/d2b/d2e/f6f 0 2024-02-19T19:04:54.254 INFO:tasks.workunit.client.1.smithi132.stdout:4/387: dread d4/d12/d16/f5a [0,4194304] 0 2024-02-19T19:04:54.256 INFO:tasks.workunit.client.1.smithi132.stdout:4/388: stat d4/d27/d31/d32/d51 0 2024-02-19T19:04:54.256 INFO:tasks.workunit.client.1.smithi132.stdout:1/390: dwrite d1/f4a [4194304,4194304] 0 2024-02-19T19:04:54.257 INFO:tasks.workunit.client.1.smithi132.stdout:6/389: dread d0/d3c/d22/f59 [0,4194304] 0 2024-02-19T19:04:54.259 INFO:tasks.workunit.client.1.smithi132.stdout:2/423: rename d0/d64/d6d/d76/f7e to d0/d4/df/d7a/f88 0 2024-02-19T19:04:54.260 INFO:tasks.workunit.client.1.smithi132.stdout:6/390: chown d0/db/f58 152915639 1 2024-02-19T19:04:54.263 INFO:tasks.workunit.client.1.smithi132.stdout:6/391: write d0/d3c/d46/d5a/f26 [4270086,99609] 0 2024-02-19T19:04:54.267 INFO:tasks.workunit.client.1.smithi132.stdout:1/391: dwrite d1/d1e/d29/d2c/f2d [0,4194304] 0 2024-02-19T19:04:54.268 INFO:tasks.workunit.client.1.smithi132.stdout:7/484: sync 2024-02-19T19:04:54.272 INFO:tasks.workunit.client.1.smithi132.stdout:3/384: fsync d8/d13/d2b/f33 0 2024-02-19T19:04:54.277 INFO:tasks.workunit.client.1.smithi132.stdout:4/389: mknod d4/d19/d21/c86 0 2024-02-19T19:04:54.278 INFO:tasks.workunit.client.1.smithi132.stdout:4/390: truncate d4/f2d 4657174 0 2024-02-19T19:04:54.279 INFO:tasks.workunit.client.1.smithi132.stdout:2/424: mkdir d0/d3c/d3f/d89 0 2024-02-19T19:04:54.279 INFO:tasks.workunit.client.1.smithi132.stdout:9/523: dwrite d12/f26 [0,4194304] 0 2024-02-19T19:04:54.284 INFO:tasks.workunit.client.1.smithi132.stdout:8/441: rename d4/dc/d1b/d1d to d4/dc/d1b/d20/d62/d91 0 2024-02-19T19:04:54.287 INFO:tasks.workunit.client.1.smithi132.stdout:5/458: symlink d8/da/d5a/l95 0 2024-02-19T19:04:54.287 INFO:tasks.workunit.client.1.smithi132.stdout:6/392: read - d0/f73 zero size 2024-02-19T19:04:54.289 INFO:tasks.workunit.client.1.smithi132.stdout:7/485: mknod d1b/d46/d76/ca0 0 2024-02-19T19:04:54.292 INFO:tasks.workunit.client.1.smithi132.stdout:0/417: dread d2/f6 [0,4194304] 0 2024-02-19T19:04:54.293 INFO:tasks.workunit.client.1.smithi132.stdout:6/393: dread d0/d3c/d46/d5a/f1d [0,4194304] 0 2024-02-19T19:04:54.295 INFO:tasks.workunit.client.1.smithi132.stdout:1/392: dread d1/d1e/d29/f41 [0,4194304] 0 2024-02-19T19:04:54.297 INFO:tasks.workunit.client.1.smithi132.stdout:4/391: unlink d4/d19/d21/c6d 0 2024-02-19T19:04:54.297 INFO:tasks.workunit.client.1.smithi132.stdout:3/385: dwrite d8/d13/d2b/d69/f6d [0,4194304] 0 2024-02-19T19:04:54.299 INFO:tasks.workunit.client.1.smithi132.stdout:2/425: mkdir d0/d19/d72/d8a 0 2024-02-19T19:04:54.304 INFO:tasks.workunit.client.1.smithi132.stdout:8/442: rename d4/dc/d1b/d20/d23/c5c to d4/dc/d1b/d20/d23/d68/c92 0 2024-02-19T19:04:54.307 INFO:tasks.workunit.client.1.smithi132.stdout:7/486: mknod d1b/d1e/d2b/d8f/ca1 0 2024-02-19T19:04:54.310 INFO:tasks.workunit.client.1.smithi132.stdout:0/418: symlink d2/dc/d4a/d58/l8b 0 2024-02-19T19:04:54.311 INFO:tasks.workunit.client.1.smithi132.stdout:6/394: rmdir d0 39 2024-02-19T19:04:54.311 INFO:tasks.workunit.client.1.smithi132.stdout:4/392: mkdir d4/d4d/d87 0 2024-02-19T19:04:54.312 INFO:tasks.workunit.client.1.smithi132.stdout:0/419: write d2/dc/df/d15/d2f/f40 [1317427,56424] 0 2024-02-19T19:04:54.312 INFO:tasks.workunit.client.1.smithi132.stdout:2/426: symlink d0/d3c/d3f/l8b 0 2024-02-19T19:04:54.312 INFO:tasks.workunit.client.1.smithi132.stdout:4/393: truncate d4/d12/f2f 5002786 0 2024-02-19T19:04:54.313 INFO:tasks.workunit.client.1.smithi132.stdout:2/427: chown d0/d19/d72/d84 244256 1 2024-02-19T19:04:54.314 INFO:tasks.workunit.client.1.smithi132.stdout:4/394: write d4/d12/d16/f5a [3277240,3229] 0 2024-02-19T19:04:54.315 INFO:tasks.workunit.client.1.smithi132.stdout:2/428: write d0/d24/d58/f36 [8627278,10753] 0 2024-02-19T19:04:54.316 INFO:tasks.workunit.client.1.smithi132.stdout:4/395: chown d4/d27/d31/d71/d14/d24/f39 1356 1 2024-02-19T19:04:54.317 INFO:tasks.workunit.client.1.smithi132.stdout:2/429: truncate d0/d4/df/d40/d68/f86 495477 0 2024-02-19T19:04:54.318 INFO:tasks.workunit.client.1.smithi132.stdout:5/459: mkdir d8/da/d5a/d7b/d96 0 2024-02-19T19:04:54.319 INFO:tasks.workunit.client.1.smithi132.stdout:7/487: creat d1b/d1e/d28/d5e/fa2 x:0 0 0 2024-02-19T19:04:54.322 INFO:tasks.workunit.client.1.smithi132.stdout:7/488: write d1b/d1e/d2b/f95 [988974,10735] 0 2024-02-19T19:04:54.323 INFO:tasks.workunit.client.1.smithi132.stdout:9/524: link d12/d2a/d4c/d61/l7e d12/d2a/d4c/dad/db2/lbc 0 2024-02-19T19:04:54.324 INFO:tasks.workunit.client.1.smithi132.stdout:7/489: readlink l9 0 2024-02-19T19:04:54.324 INFO:tasks.workunit.client.1.smithi132.stdout:8/443: dread d4/dc/d1b/d20/d23/f28 [0,4194304] 0 2024-02-19T19:04:54.325 INFO:tasks.workunit.client.1.smithi132.stdout:9/525: read - d12/d3e/f93 zero size 2024-02-19T19:04:54.326 INFO:tasks.workunit.client.1.smithi132.stdout:2/430: dwrite d0/d19/f82 [0,4194304] 0 2024-02-19T19:04:54.326 INFO:tasks.workunit.client.1.smithi132.stdout:8/444: fdatasync d4/dc/f3f 0 2024-02-19T19:04:54.327 INFO:tasks.workunit.client.1.smithi132.stdout:8/445: fsync d4/dc/f4e 0 2024-02-19T19:04:54.328 INFO:tasks.workunit.client.1.smithi132.stdout:2/431: truncate d0/d4/d11/d2e/f75 288605 0 2024-02-19T19:04:54.332 INFO:tasks.workunit.client.1.smithi132.stdout:1/393: rename d1/d1e/d29 to d1/d1e/d77 0 2024-02-19T19:04:54.336 INFO:tasks.workunit.client.1.smithi132.stdout:3/386: creat d8/d13/d2b/d52/d34/f79 x:0 0 0 2024-02-19T19:04:54.338 INFO:tasks.workunit.client.1.smithi132.stdout:1/394: dwrite d1/d1e/d77/d52/f65 [0,4194304] 0 2024-02-19T19:04:54.342 INFO:tasks.workunit.client.1.smithi132.stdout:5/460: rename d8/d23/d29/d3f/f47 to d8/d1a/f97 0 2024-02-19T19:04:54.344 INFO:tasks.workunit.client.1.smithi132.stdout:1/395: read d1/d3a/f40 [49239,125770] 0 2024-02-19T19:04:54.347 INFO:tasks.workunit.client.1.smithi132.stdout:4/396: write d4/d12/f2c [1772511,20696] 0 2024-02-19T19:04:54.349 INFO:tasks.workunit.client.1.smithi132.stdout:7/490: mkdir d1b/d1e/d41/da3 0 2024-02-19T19:04:54.351 INFO:tasks.workunit.client.1.smithi132.stdout:1/396: dwrite d1/d3a/d50/f6f [0,4194304] 0 2024-02-19T19:04:54.355 INFO:tasks.workunit.client.1.smithi132.stdout:8/446: mkdir d4/dc/d1b/d38/d69/d74/d93 0 2024-02-19T19:04:54.355 INFO:tasks.workunit.client.1.smithi132.stdout:0/420: creat d2/f8c x:0 0 0 2024-02-19T19:04:54.357 INFO:tasks.workunit.client.1.smithi132.stdout:8/447: stat d4/dc/d1b/d20/f6e 0 2024-02-19T19:04:54.358 INFO:tasks.workunit.client.1.smithi132.stdout:3/387: rename d8/d13/d19/c48 to d8/d38/d44/d6e/c7a 0 2024-02-19T19:04:54.364 INFO:tasks.workunit.client.1.smithi132.stdout:4/397: rmdir d4/d19/d21/d56 39 2024-02-19T19:04:54.365 INFO:tasks.workunit.client.1.smithi132.stdout:5/461: dread d8/d23/f45 [0,4194304] 0 2024-02-19T19:04:54.367 INFO:tasks.workunit.client.1.smithi132.stdout:3/388: dwrite d8/d38/f3d [0,4194304] 0 2024-02-19T19:04:54.370 INFO:tasks.workunit.client.1.smithi132.stdout:4/398: dread d4/d12/f2f [0,4194304] 0 2024-02-19T19:04:54.374 INFO:tasks.workunit.client.1.smithi132.stdout:2/432: dwrite d0/d19/d1e/f62 [0,4194304] 0 2024-02-19T19:04:54.376 INFO:tasks.workunit.client.1.smithi132.stdout:7/491: unlink cb 0 2024-02-19T19:04:54.378 INFO:tasks.workunit.client.1.smithi132.stdout:6/395: dread d0/dc/f24 [0,4194304] 0 2024-02-19T19:04:54.378 INFO:tasks.workunit.client.1.smithi132.stdout:7/492: chown d1b/d1e/d41/da3 47127074 1 2024-02-19T19:04:54.379 INFO:tasks.workunit.client.1.smithi132.stdout:7/493: write d1b/f72 [5075229,129563] 0 2024-02-19T19:04:54.383 INFO:tasks.workunit.client.1.smithi132.stdout:8/448: creat d4/dc/d1b/d20/d23/d68/d7a/f94 x:0 0 0 2024-02-19T19:04:54.388 INFO:tasks.workunit.client.1.smithi132.stdout:2/433: write d0/d19/f78 [505304,29525] 0 2024-02-19T19:04:54.390 INFO:tasks.workunit.client.1.smithi132.stdout:3/389: symlink d8/d13/d2b/l7b 0 2024-02-19T19:04:54.392 INFO:tasks.workunit.client.1.smithi132.stdout:9/526: link d12/d2a/d4c/d61/c9e d12/d2a/d88/cbd 0 2024-02-19T19:04:54.392 INFO:tasks.workunit.client.1.smithi132.stdout:3/390: dread - d8/d13/d2b/d52/d34/d42/f64 zero size 2024-02-19T19:04:54.395 INFO:tasks.workunit.client.1.smithi132.stdout:4/399: mknod d4/d4d/d87/c88 0 2024-02-19T19:04:54.395 INFO:tasks.workunit.client.1.smithi132.stdout:6/396: dread d0/d3c/f42 [0,4194304] 0 2024-02-19T19:04:54.397 INFO:tasks.workunit.client.1.smithi132.stdout:9/527: dread d12/f26 [0,4194304] 0 2024-02-19T19:04:54.399 INFO:tasks.workunit.client.1.smithi132.stdout:4/400: write d4/d27/d31/d71/d37/f74 [250286,127729] 0 2024-02-19T19:04:54.401 INFO:tasks.workunit.client.1.smithi132.stdout:1/397: getdents d1/d1e/d77/d52 0 2024-02-19T19:04:54.402 INFO:tasks.workunit.client.1.smithi132.stdout:8/449: rename d4/dc/d1b/d33/f34 to d4/dc/d1b/d20/d23/d68/d7a/f95 0 2024-02-19T19:04:54.404 INFO:tasks.workunit.client.1.smithi132.stdout:1/398: read d1/d1e/d77/d2c/f2d [3890365,39977] 0 2024-02-19T19:04:54.404 INFO:tasks.workunit.client.1.smithi132.stdout:2/434: dwrite d0/d19/d1e/f45 [0,4194304] 0 2024-02-19T19:04:54.407 INFO:tasks.workunit.client.1.smithi132.stdout:1/399: readlink d1/d1e/d77/d52/l55 0 2024-02-19T19:04:54.411 INFO:tasks.workunit.client.1.smithi132.stdout:8/450: dwrite d4/dc/d1b/d20/d23/d68/d79/f85 [0,4194304] 0 2024-02-19T19:04:54.412 INFO:tasks.workunit.client.1.smithi132.stdout:8/451: chown d4/dc/d1b/d38/d69/c76 177 1 2024-02-19T19:04:54.417 INFO:tasks.workunit.client.1.smithi132.stdout:5/462: dread d8/d1a/f7a [0,4194304] 0 2024-02-19T19:04:54.431 INFO:tasks.workunit.client.1.smithi132.stdout:6/397: rmdir d0/db/d25/d3a/d3d 39 2024-02-19T19:04:54.441 INFO:tasks.workunit.client.1.smithi132.stdout:0/421: sync 2024-02-19T19:04:54.461 INFO:tasks.workunit.client.1.smithi132.stdout:3/391: dread d8/f5a [0,4194304] 0 2024-02-19T19:04:54.462 INFO:tasks.workunit.client.1.smithi132.stdout:3/392: write d8/f22 [4217676,1258] 0 2024-02-19T19:04:54.488 INFO:tasks.workunit.client.1.smithi132.stdout:7/494: creat d1b/d1e/d2b/d62/d53/fa4 x:0 0 0 2024-02-19T19:04:54.488 INFO:tasks.workunit.client.1.smithi132.stdout:7/495: fdatasync d1b/d5a/f68 0 2024-02-19T19:04:54.489 INFO:tasks.workunit.client.1.smithi132.stdout:7/496: chown d1b/d1e/d41/l71 27810 1 2024-02-19T19:04:54.495 INFO:tasks.workunit.client.1.smithi132.stdout:9/528: mknod d12/d8d/d3a/d40/d4e/d77/d9d/cbe 0 2024-02-19T19:04:54.500 INFO:tasks.workunit.client.1.smithi132.stdout:2/435: creat d0/d24/d58/f8c x:0 0 0 2024-02-19T19:04:54.504 INFO:tasks.workunit.client.1.smithi132.stdout:2/436: dread d0/d4/d11/f65 [4194304,4194304] 0 2024-02-19T19:04:54.510 INFO:tasks.workunit.client.1.smithi132.stdout:8/452: chown d4/dc/d1b/d20/d23/d68/d7a/f95 2312530 1 2024-02-19T19:04:54.513 INFO:tasks.workunit.client.1.smithi132.stdout:2/437: dwrite d0/d19/f78 [0,4194304] 0 2024-02-19T19:04:54.517 INFO:tasks.workunit.client.1.smithi132.stdout:7/497: mkdir d1b/d1e/d2b/d8f/da5 0 2024-02-19T19:04:54.526 INFO:tasks.workunit.client.1.smithi132.stdout:5/463: creat d8/da/db/f98 x:0 0 0 2024-02-19T19:04:54.529 INFO:tasks.workunit.client.1.smithi132.stdout:0/422: unlink d2/dc/d4a/d58/l8b 0 2024-02-19T19:04:54.531 INFO:tasks.workunit.client.1.smithi132.stdout:0/423: read - d2/dc/df/d15/d2f/d3f/f79 zero size 2024-02-19T19:04:54.532 INFO:tasks.workunit.client.1.smithi132.stdout:7/498: mknod d1b/d46/d76/ca6 0 2024-02-19T19:04:54.534 INFO:tasks.workunit.client.1.smithi132.stdout:5/464: rmdir d8 39 2024-02-19T19:04:54.535 INFO:tasks.workunit.client.1.smithi132.stdout:6/398: sync 2024-02-19T19:04:54.536 INFO:tasks.workunit.client.1.smithi132.stdout:6/399: stat d0/db/c65 0 2024-02-19T19:04:54.536 INFO:tasks.workunit.client.1.smithi132.stdout:2/438: creat d0/d4/df/d40/d7b/f8d x:0 0 0 2024-02-19T19:04:54.538 INFO:tasks.workunit.client.1.smithi132.stdout:0/424: chown d2/f19 201062804 1 2024-02-19T19:04:54.541 INFO:tasks.workunit.client.1.smithi132.stdout:9/529: fsync d12/d8d/d3a/d40/d4e/d77/d8a/d82/f8c 0 2024-02-19T19:04:54.542 INFO:tasks.workunit.client.1.smithi132.stdout:9/530: write d12/d8d/d3a/d40/d4e/d77/fa2 [363241,105028] 0 2024-02-19T19:04:54.545 INFO:tasks.workunit.client.1.smithi132.stdout:6/400: dread - d0/db/d25/d3a/d3d/d69/f7f zero size 2024-02-19T19:04:54.549 INFO:tasks.workunit.client.1.smithi132.stdout:2/439: unlink d0/d4/df/f4d 0 2024-02-19T19:04:54.551 INFO:tasks.workunit.client.1.smithi132.stdout:0/425: readlink d2/l59 0 2024-02-19T19:04:54.553 INFO:tasks.workunit.client.1.smithi132.stdout:7/499: mknod d1b/d5a/d63/d98/ca7 0 2024-02-19T19:04:54.557 INFO:tasks.workunit.client.1.smithi132.stdout:5/465: unlink d8/d23/d29/d27/l32 0 2024-02-19T19:04:54.561 INFO:tasks.workunit.client.1.smithi132.stdout:2/440: rmdir d0/d19/d72 39 2024-02-19T19:04:54.564 INFO:tasks.workunit.client.1.smithi132.stdout:0/426: unlink d2/dc/df/d15/d2f/d3f/d63/l7e 0 2024-02-19T19:04:54.565 INFO:tasks.workunit.client.1.smithi132.stdout:4/401: write d4/d19/f23 [1615335,68075] 0 2024-02-19T19:04:54.573 INFO:tasks.workunit.client.1.smithi132.stdout:4/402: creat d4/d12/f89 x:0 0 0 2024-02-19T19:04:54.576 INFO:tasks.workunit.client.1.smithi132.stdout:2/441: stat d0/d19/f78 0 2024-02-19T19:04:54.577 INFO:tasks.workunit.client.1.smithi132.stdout:4/403: symlink d4/d27/d31/l8a 0 2024-02-19T19:04:54.577 INFO:tasks.workunit.client.1.smithi132.stdout:2/442: write d0/d24/f87 [837312,74366] 0 2024-02-19T19:04:54.578 INFO:tasks.workunit.client.1.smithi132.stdout:5/466: dread d8/d1a/d35/f40 [0,4194304] 0 2024-02-19T19:04:54.579 INFO:tasks.workunit.client.1.smithi132.stdout:1/400: mkdir d1/d1e/d77/d78 0 2024-02-19T19:04:54.580 INFO:tasks.workunit.client.1.smithi132.stdout:5/467: truncate d8/d23/d3b/f65 776479 0 2024-02-19T19:04:54.581 INFO:tasks.workunit.client.1.smithi132.stdout:7/500: dread d1b/d1e/f22 [4194304,4194304] 0 2024-02-19T19:04:54.582 INFO:tasks.workunit.client.1.smithi132.stdout:4/404: fdatasync d4/d27/d31/d71/d14/d24/f3b 0 2024-02-19T19:04:54.585 INFO:tasks.workunit.client.1.smithi132.stdout:5/468: rmdir d8 39 2024-02-19T19:04:54.587 INFO:tasks.workunit.client.1.smithi132.stdout:2/443: dread d0/d19/f2c [0,4194304] 0 2024-02-19T19:04:54.588 INFO:tasks.workunit.client.1.smithi132.stdout:4/405: dwrite d4/f9 [0,4194304] 0 2024-02-19T19:04:54.591 INFO:tasks.workunit.client.1.smithi132.stdout:1/401: mkdir d1/d3a/d79 0 2024-02-19T19:04:54.594 INFO:tasks.workunit.client.1.smithi132.stdout:2/444: write d0/f6 [9611385,2139] 0 2024-02-19T19:04:54.594 INFO:tasks.workunit.client.1.smithi132.stdout:3/393: sync 2024-02-19T19:04:54.595 INFO:tasks.workunit.client.1.smithi132.stdout:3/394: dread - d8/d13/d2f/f62 zero size 2024-02-19T19:04:54.596 INFO:tasks.workunit.client.1.smithi132.stdout:3/395: chown d8/f51 38 1 2024-02-19T19:04:54.596 INFO:tasks.workunit.client.1.smithi132.stdout:5/469: creat d8/da/d5a/d7b/f99 x:0 0 0 2024-02-19T19:04:54.597 INFO:tasks.workunit.client.1.smithi132.stdout:5/470: truncate d8/da/d5a/d7b/f99 514971 0 2024-02-19T19:04:54.602 INFO:tasks.workunit.client.1.smithi132.stdout:4/406: symlink d4/d27/l8b 0 2024-02-19T19:04:54.602 INFO:tasks.workunit.client.1.smithi132.stdout:4/407: stat d4/d19/c4a 0 2024-02-19T19:04:54.602 INFO:tasks.workunit.client.1.smithi132.stdout:5/471: symlink d8/d2e/d71/l9a 0 2024-02-19T19:04:54.621 INFO:tasks.workunit.client.1.smithi132.stdout:2/445: creat d0/d4/f8e x:0 0 0 2024-02-19T19:04:54.622 INFO:tasks.workunit.client.1.smithi132.stdout:8/453: dwrite d4/f2e [0,4194304] 0 2024-02-19T19:04:54.623 INFO:tasks.workunit.client.1.smithi132.stdout:6/401: write d0/db/d25/d2f/f84 [766333,69556] 0 2024-02-19T19:04:54.624 INFO:tasks.workunit.client.1.smithi132.stdout:2/446: fdatasync d0/d24/f87 0 2024-02-19T19:04:54.625 INFO:tasks.workunit.client.1.smithi132.stdout:7/501: write d1b/d1e/d28/f4c [2489722,128707] 0 2024-02-19T19:04:54.627 INFO:tasks.workunit.client.1.smithi132.stdout:6/402: stat d0/d3c/d46/d5a/f1d 0 2024-02-19T19:04:54.627 INFO:tasks.workunit.client.1.smithi132.stdout:8/454: write d4/dc/d1b/f32 [941265,46519] 0 2024-02-19T19:04:54.627 INFO:tasks.workunit.client.1.smithi132.stdout:4/408: dwrite d4/f7d [0,4194304] 0 2024-02-19T19:04:54.627 INFO:tasks.workunit.client.1.smithi132.stdout:5/472: rename d8/d1a/d35/d77/f8f to d8/f9b 0 2024-02-19T19:04:54.641 INFO:tasks.workunit.client.1.smithi132.stdout:5/473: creat d8/d1a/f9c x:0 0 0 2024-02-19T19:04:54.654 INFO:tasks.workunit.client.1.smithi132.stdout:8/455: rename d4/dc/d42/c52 to d4/dc/d1b/d33/c96 0 2024-02-19T19:04:54.659 INFO:tasks.workunit.client.1.smithi132.stdout:9/531: sync 2024-02-19T19:04:54.663 INFO:tasks.workunit.client.1.smithi132.stdout:7/502: creat d1b/d1e/d2b/d8f/da5/fa8 x:0 0 0 2024-02-19T19:04:54.666 INFO:tasks.workunit.client.1.smithi132.stdout:2/447: link d0/d19/d1e/f45 d0/d4/df/d40/d68/f8f 0 2024-02-19T19:04:54.669 INFO:tasks.workunit.client.1.smithi132.stdout:6/403: link d0/db/d25/d2f/f4c d0/f8a 0 2024-02-19T19:04:54.672 INFO:tasks.workunit.client.1.smithi132.stdout:5/474: dread d8/da/f33 [0,4194304] 0 2024-02-19T19:04:54.674 INFO:tasks.workunit.client.1.smithi132.stdout:7/503: mkdir d1b/d1e/d28/da9 0 2024-02-19T19:04:54.679 INFO:tasks.workunit.client.1.smithi132.stdout:4/409: getdents d4/d27/d31/d71/d37 0 2024-02-19T19:04:54.680 INFO:tasks.workunit.client.1.smithi132.stdout:7/504: dwrite d1b/d1e/d28/d5e/fa2 [0,4194304] 0 2024-02-19T19:04:54.681 INFO:tasks.workunit.client.1.smithi132.stdout:2/448: dread d0/d4/d11/f41 [0,4194304] 0 2024-02-19T19:04:54.682 INFO:tasks.workunit.client.1.smithi132.stdout:2/449: fsync d0/d4/d11/d2e/f75 0 2024-02-19T19:04:54.683 INFO:tasks.workunit.client.1.smithi132.stdout:5/475: creat d8/d1a/f9d x:0 0 0 2024-02-19T19:04:54.696 INFO:tasks.workunit.client.1.smithi132.stdout:2/450: dread d0/d3c/f60 [0,4194304] 0 2024-02-19T19:04:54.698 INFO:tasks.workunit.client.1.smithi132.stdout:2/451: dread d0/d24/d58/f49 [0,4194304] 0 2024-02-19T19:04:54.699 INFO:tasks.workunit.client.1.smithi132.stdout:9/532: link d12/d8d/d3a/d40/c42 d12/d8d/d3a/d40/d4e/d77/d8a/d82/cbf 0 2024-02-19T19:04:54.699 INFO:tasks.workunit.client.1.smithi132.stdout:9/533: fdatasync d12/d3e/f6c 0 2024-02-19T19:04:54.699 INFO:tasks.workunit.client.1.smithi132.stdout:9/534: readlink d12/d1b/l59 0 2024-02-19T19:04:54.703 INFO:tasks.workunit.client.1.smithi132.stdout:6/404: rename d0/db/d25/d2f to d0/d8b 0 2024-02-19T19:04:54.704 INFO:tasks.workunit.client.1.smithi132.stdout:4/410: creat d4/d19/d21/d56/f8c x:0 0 0 2024-02-19T19:04:54.708 INFO:tasks.workunit.client.1.smithi132.stdout:2/452: mknod d0/d3c/d3f/c90 0 2024-02-19T19:04:54.709 INFO:tasks.workunit.client.1.smithi132.stdout:2/453: read d0/d19/d1e/f45 [3857396,68743] 0 2024-02-19T19:04:54.710 INFO:tasks.workunit.client.1.smithi132.stdout:6/405: mknod d0/d3c/d22/d79/c8c 0 2024-02-19T19:04:54.710 INFO:tasks.workunit.client.1.smithi132.stdout:5/476: dwrite d8/d1a/d35/f4b [0,4194304] 0 2024-02-19T19:04:54.717 INFO:tasks.workunit.client.1.smithi132.stdout:5/477: dwrite d8/da/db/f44 [0,4194304] 0 2024-02-19T19:04:54.718 INFO:tasks.workunit.client.1.smithi132.stdout:5/478: dread - d8/da/db/f83 zero size 2024-02-19T19:04:54.718 INFO:tasks.workunit.client.1.smithi132.stdout:9/535: link fb d12/d8d/fc0 0 2024-02-19T19:04:54.720 INFO:tasks.workunit.client.1.smithi132.stdout:9/536: chown d12/d1b/f16 25082 1 2024-02-19T19:04:54.721 INFO:tasks.workunit.client.1.smithi132.stdout:7/505: getdents d1b/d1e/d2b/d62 0 2024-02-19T19:04:54.723 INFO:tasks.workunit.client.1.smithi132.stdout:2/454: truncate d0/d4/df/f3b 2758346 0 2024-02-19T19:04:54.724 INFO:tasks.workunit.client.1.smithi132.stdout:2/455: fsync d0/d4/f8e 0 2024-02-19T19:04:54.731 INFO:tasks.workunit.client.1.smithi132.stdout:2/456: rmdir d0/d64/d6d 39 2024-02-19T19:04:54.736 INFO:tasks.workunit.client.1.smithi132.stdout:2/457: creat d0/d24/d58/d53/f91 x:0 0 0 2024-02-19T19:04:54.736 INFO:tasks.workunit.client.1.smithi132.stdout:7/506: dwrite f8 [0,4194304] 0 2024-02-19T19:04:54.739 INFO:tasks.workunit.client.1.smithi132.stdout:2/458: truncate d0/d4/df/d7a/f88 190709 0 2024-02-19T19:04:54.739 INFO:tasks.workunit.client.1.smithi132.stdout:0/427: sync 2024-02-19T19:04:54.744 INFO:tasks.workunit.client.1.smithi132.stdout:5/479: getdents d8/d1a 0 2024-02-19T19:04:54.746 INFO:tasks.workunit.client.1.smithi132.stdout:9/537: getdents d12/d8d/d3a/d40/d4e/d77 0 2024-02-19T19:04:54.751 INFO:tasks.workunit.client.1.smithi132.stdout:2/459: truncate d0/d19/d1e/f45 2849093 0 2024-02-19T19:04:54.752 INFO:tasks.workunit.client.1.smithi132.stdout:2/460: fdatasync d0/d19/f82 0 2024-02-19T19:04:54.752 INFO:tasks.workunit.client.1.smithi132.stdout:0/428: creat d2/dc/df/d15/d2f/d3f/f8d x:0 0 0 2024-02-19T19:04:54.753 INFO:tasks.workunit.client.1.smithi132.stdout:2/461: fdatasync d0/d19/d1e/f62 0 2024-02-19T19:04:54.753 INFO:tasks.workunit.client.1.smithi132.stdout:6/406: truncate d0/d3c/f39 3166141 0 2024-02-19T19:04:54.756 INFO:tasks.workunit.client.1.smithi132.stdout:7/507: link d1b/d1e/d41/f6a d1b/d1e/d28/faa 0 2024-02-19T19:04:54.759 INFO:tasks.workunit.client.1.smithi132.stdout:4/411: write d4/d12/d28/f62 [2369660,120326] 0 2024-02-19T19:04:54.763 INFO:tasks.workunit.client.1.smithi132.stdout:2/462: creat d0/d3c/f92 x:0 0 0 2024-02-19T19:04:54.766 INFO:tasks.workunit.client.1.smithi132.stdout:6/407: dwrite d0/d51/f60 [0,4194304] 0 2024-02-19T19:04:54.766 INFO:tasks.workunit.client.1.smithi132.stdout:0/429: symlink d2/dc/df/d15/d2f/d3f/l8e 0 2024-02-19T19:04:54.770 INFO:tasks.workunit.client.1.smithi132.stdout:7/508: symlink d1b/d5a/d63/d98/lab 0 2024-02-19T19:04:54.771 INFO:tasks.workunit.client.1.smithi132.stdout:4/412: rmdir d4/d4d/d87 39 2024-02-19T19:04:54.771 INFO:tasks.workunit.client.1.smithi132.stdout:3/396: sync 2024-02-19T19:04:54.771 INFO:tasks.workunit.client.1.smithi132.stdout:8/456: sync 2024-02-19T19:04:54.771 INFO:tasks.workunit.client.1.smithi132.stdout:1/402: sync 2024-02-19T19:04:54.771 INFO:tasks.workunit.client.1.smithi132.stdout:2/463: dwrite d0/d24/d58/f49 [0,4194304] 0 2024-02-19T19:04:54.773 INFO:tasks.workunit.client.1.smithi132.stdout:8/457: write d4/dc/f3f [283572,116514] 0 2024-02-19T19:04:54.775 INFO:tasks.workunit.client.1.smithi132.stdout:1/403: read d1/d1e/d77/d52/f56 [3517229,110081] 0 2024-02-19T19:04:54.776 INFO:tasks.workunit.client.1.smithi132.stdout:7/509: dread f8 [0,4194304] 0 2024-02-19T19:04:54.794 INFO:tasks.workunit.client.1.smithi132.stdout:5/480: getdents d8/d1a 0 2024-02-19T19:04:54.794 INFO:tasks.workunit.client.1.smithi132.stdout:8/458: creat d4/dc/d42/f97 x:0 0 0 2024-02-19T19:04:54.795 INFO:tasks.workunit.client.1.smithi132.stdout:0/430: dread d2/dc/df/d15/f4e [0,4194304] 0 2024-02-19T19:04:54.796 INFO:tasks.workunit.client.1.smithi132.stdout:1/404: symlink d1/d1e/d77/d52/l7a 0 2024-02-19T19:04:54.797 INFO:tasks.workunit.client.1.smithi132.stdout:9/538: link d12/d8d/d3a/d85/c97 d12/cc1 0 2024-02-19T19:04:54.799 INFO:tasks.workunit.client.1.smithi132.stdout:9/539: write d12/d1b/f95 [2361271,30275] 0 2024-02-19T19:04:54.799 INFO:tasks.workunit.client.1.smithi132.stdout:4/413: mknod d4/d27/d31/d32/c8d 0 2024-02-19T19:04:54.800 INFO:tasks.workunit.client.1.smithi132.stdout:2/464: mknod d0/d19/d6a/c93 0 2024-02-19T19:04:54.802 INFO:tasks.workunit.client.1.smithi132.stdout:8/459: dwrite d4/dc/f44 [0,4194304] 0 2024-02-19T19:04:54.803 INFO:tasks.workunit.client.1.smithi132.stdout:2/465: dread d0/d4/df/d7a/f88 [0,4194304] 0 2024-02-19T19:04:54.804 INFO:tasks.workunit.client.1.smithi132.stdout:2/466: stat d0/d19/d6a 0 2024-02-19T19:04:54.806 INFO:tasks.workunit.client.1.smithi132.stdout:6/408: creat d0/db/d25/f8d x:0 0 0 2024-02-19T19:04:54.807 INFO:tasks.workunit.client.1.smithi132.stdout:7/510: creat d1b/d5a/d63/d6b/d86/fac x:0 0 0 2024-02-19T19:04:54.809 INFO:tasks.workunit.client.1.smithi132.stdout:3/397: rmdir d8/d38/d44/d6a 0 2024-02-19T19:04:54.812 INFO:tasks.workunit.client.1.smithi132.stdout:6/409: fdatasync d0/d3c/d22/f59 0 2024-02-19T19:04:54.812 INFO:tasks.workunit.client.1.smithi132.stdout:4/414: mknod d4/d19/c8e 0 2024-02-19T19:04:54.813 INFO:tasks.workunit.client.1.smithi132.stdout:2/467: write d0/d4/df/d40/f74 [844027,45119] 0 2024-02-19T19:04:54.813 INFO:tasks.workunit.client.1.smithi132.stdout:1/405: creat d1/f7b x:0 0 0 2024-02-19T19:04:54.814 INFO:tasks.workunit.client.1.smithi132.stdout:1/406: dread - d1/f72 zero size 2024-02-19T19:04:54.818 INFO:tasks.workunit.client.1.smithi132.stdout:5/481: dread d8/d23/d29/d27/f25 [0,4194304] 0 2024-02-19T19:04:54.820 INFO:tasks.workunit.client.1.smithi132.stdout:8/460: rename d4/dc/d1b/d20/d62/d91/f77 to d4/d2d/d82/f98 0 2024-02-19T19:04:54.821 INFO:tasks.workunit.client.1.smithi132.stdout:4/415: creat d4/d19/d21/d56/f8f x:0 0 0 2024-02-19T19:04:54.823 INFO:tasks.workunit.client.1.smithi132.stdout:2/468: dread d0/d4/d11/f20 [0,4194304] 0 2024-02-19T19:04:54.825 INFO:tasks.workunit.client.1.smithi132.stdout:2/469: truncate d0/d4/d11/f65 5684042 0 2024-02-19T19:04:54.825 INFO:tasks.workunit.client.1.smithi132.stdout:2/470: stat d0/d19/f2c 0 2024-02-19T19:04:54.827 INFO:tasks.workunit.client.1.smithi132.stdout:3/398: rename d8/d13/d2b/d69/f6d to d8/d13/d2b/d2e/f7c 0 2024-02-19T19:04:54.828 INFO:tasks.workunit.client.1.smithi132.stdout:6/410: mknod d0/db/d25/d3a/d3d/d69/d89/c8e 0 2024-02-19T19:04:54.830 INFO:tasks.workunit.client.1.smithi132.stdout:4/416: readlink d4/d27/d31/d71/d14/d24/l42 0 2024-02-19T19:04:54.830 INFO:tasks.workunit.client.1.smithi132.stdout:5/482: dwrite d8/d2e/d71/f82 [0,4194304] 0 2024-02-19T19:04:54.830 INFO:tasks.workunit.client.1.smithi132.stdout:9/540: link d12/d8d/d3a/d40/c42 d12/d2a/d4c/cc2 0 2024-02-19T19:04:54.831 INFO:tasks.workunit.client.1.smithi132.stdout:5/483: stat d8/d1a/d35/d77/f6b 0 2024-02-19T19:04:54.831 INFO:tasks.workunit.client.1.smithi132.stdout:0/431: getdents d2/d6e 0 2024-02-19T19:04:54.835 INFO:tasks.workunit.client.1.smithi132.stdout:2/471: write d0/d4/df/d7a/f88 [1132361,55074] 0 2024-02-19T19:04:54.836 INFO:tasks.workunit.client.1.smithi132.stdout:1/407: creat d1/d61/f7c x:0 0 0 2024-02-19T19:04:54.838 INFO:tasks.workunit.client.1.smithi132.stdout:5/484: dwrite d8/d1a/d35/f74 [0,4194304] 0 2024-02-19T19:04:54.839 INFO:tasks.workunit.client.1.smithi132.stdout:8/461: rename d4/dc/c13 to d4/dc/d1b/d20/d23/d68/d79/c99 0 2024-02-19T19:04:54.842 INFO:tasks.workunit.client.1.smithi132.stdout:8/462: write d4/dc/d1b/d33/f49 [765693,124375] 0 2024-02-19T19:04:54.843 INFO:tasks.workunit.client.1.smithi132.stdout:3/399: dwrite f0 [0,4194304] 0 2024-02-19T19:04:54.845 INFO:tasks.workunit.client.1.smithi132.stdout:8/463: fdatasync d4/dc/f1c 0 2024-02-19T19:04:54.848 INFO:tasks.workunit.client.1.smithi132.stdout:4/417: readlink d4/d27/d31/d71/l4f 0 2024-02-19T19:04:54.854 INFO:tasks.workunit.client.1.smithi132.stdout:8/464: fdatasync d4/dc/d1b/d33/f49 0 2024-02-19T19:04:54.854 INFO:tasks.workunit.client.1.smithi132.stdout:8/465: write d4/f83 [327756,29546] 0 2024-02-19T19:04:54.855 INFO:tasks.workunit.client.1.smithi132.stdout:8/466: chown d4/dc/d42/f6f 706333 1 2024-02-19T19:04:54.856 INFO:tasks.workunit.client.1.smithi132.stdout:0/432: creat d2/d6e/d7a/f8f x:0 0 0 2024-02-19T19:04:54.856 INFO:tasks.workunit.client.1.smithi132.stdout:2/472: chown d0/d4/df/c3a 7 1 2024-02-19T19:04:54.866 INFO:tasks.workunit.client.1.smithi132.stdout:8/467: symlink d4/dc/d1b/d33/l9a 0 2024-02-19T19:04:54.869 INFO:tasks.workunit.client.1.smithi132.stdout:9/541: write d12/d2a/d4c/f6a [208648,104834] 0 2024-02-19T19:04:54.871 INFO:tasks.workunit.client.1.smithi132.stdout:7/511: sync 2024-02-19T19:04:54.872 INFO:tasks.workunit.client.1.smithi132.stdout:9/542: read d12/d8d/d3a/d40/f49 [1766705,116777] 0 2024-02-19T19:04:54.872 INFO:tasks.workunit.client.1.smithi132.stdout:1/408: write d1/d1e/d77/f41 [160007,47314] 0 2024-02-19T19:04:54.875 INFO:tasks.workunit.client.1.smithi132.stdout:5/485: unlink d8/d23/d29/d27/f69 0 2024-02-19T19:04:54.876 INFO:tasks.workunit.client.1.smithi132.stdout:6/411: write d0/db/d25/d38/f68 [1097451,1963] 0 2024-02-19T19:04:54.876 INFO:tasks.workunit.client.1.smithi132.stdout:5/486: write d8/d1a/d35/d77/f6b [1539193,4577] 0 2024-02-19T19:04:54.880 INFO:tasks.workunit.client.1.smithi132.stdout:0/433: dread d2/f1a [0,4194304] 0 2024-02-19T19:04:54.881 INFO:tasks.workunit.client.1.smithi132.stdout:2/473: creat d0/d64/d6d/f94 x:0 0 0 2024-02-19T19:04:54.881 INFO:tasks.workunit.client.1.smithi132.stdout:7/512: chown d1b/d5a/d63/d6b/c7b 13124 1 2024-02-19T19:04:54.882 INFO:tasks.workunit.client.1.smithi132.stdout:2/474: write d0/d19/f2c [963495,60421] 0 2024-02-19T19:04:54.882 INFO:tasks.workunit.client.1.smithi132.stdout:9/543: truncate d12/d8d/d3a/f6d 5119639 0 2024-02-19T19:04:54.883 INFO:tasks.workunit.client.1.smithi132.stdout:2/475: stat d0/d19/f2c 0 2024-02-19T19:04:54.885 INFO:tasks.workunit.client.1.smithi132.stdout:3/400: creat d8/d13/f7d x:0 0 0 2024-02-19T19:04:54.885 INFO:tasks.workunit.client.1.smithi132.stdout:4/418: creat d4/d19/d21/f90 x:0 0 0 2024-02-19T19:04:54.887 INFO:tasks.workunit.client.1.smithi132.stdout:8/468: write f2 [3058322,44700] 0 2024-02-19T19:04:54.890 INFO:tasks.workunit.client.1.smithi132.stdout:6/412: rename d0/d3c to d0/db/d25/d3a/d5d/d8f 0 2024-02-19T19:04:54.891 INFO:tasks.workunit.client.1.smithi132.stdout:0/434: rmdir d2/dc/df/d15 39 2024-02-19T19:04:54.894 INFO:tasks.workunit.client.1.smithi132.stdout:9/544: creat d12/d2a/d88/fc3 x:0 0 0 2024-02-19T19:04:54.895 INFO:tasks.workunit.client.1.smithi132.stdout:8/469: dwrite d4/dc/d1b/d20/d23/d68/d79/f85 [0,4194304] 0 2024-02-19T19:04:54.896 INFO:tasks.workunit.client.1.smithi132.stdout:3/401: unlink d8/d13/d2f/f62 0 2024-02-19T19:04:54.899 INFO:tasks.workunit.client.1.smithi132.stdout:4/419: dread d4/d12/d28/f3e [0,4194304] 0 2024-02-19T19:04:54.901 INFO:tasks.workunit.client.1.smithi132.stdout:4/420: write d4/f2d [1490476,18093] 0 2024-02-19T19:04:54.901 INFO:tasks.workunit.client.1.smithi132.stdout:5/487: mkdir d8/da/d5a/d7b/d96/d9e 0 2024-02-19T19:04:54.904 INFO:tasks.workunit.client.1.smithi132.stdout:6/413: creat d0/dc/f90 x:0 0 0 2024-02-19T19:04:54.904 INFO:tasks.workunit.client.1.smithi132.stdout:6/414: chown d0/d51/f60 8253169 1 2024-02-19T19:04:54.905 INFO:tasks.workunit.client.1.smithi132.stdout:1/409: creat d1/d1e/d77/d2c/f7d x:0 0 0 2024-02-19T19:04:54.906 INFO:tasks.workunit.client.1.smithi132.stdout:7/513: creat d1b/d5a/d63/d96/fad x:0 0 0 2024-02-19T19:04:54.907 INFO:tasks.workunit.client.1.smithi132.stdout:0/435: fsync d2/dc/df/d15/f46 0 2024-02-19T19:04:54.909 INFO:tasks.workunit.client.1.smithi132.stdout:4/421: dwrite d4/d4d/f64 [0,4194304] 0 2024-02-19T19:04:54.910 INFO:tasks.workunit.client.1.smithi132.stdout:9/545: mkdir d12/d8d/d3a/d40/d4e/d77/d89/dc4 0 2024-02-19T19:04:54.912 INFO:tasks.workunit.client.1.smithi132.stdout:3/402: rename d8/d5d to d8/d13/d2b/d69/d7e 0 2024-02-19T19:04:54.912 INFO:tasks.workunit.client.1.smithi132.stdout:9/546: write d12/d3e/d6e/fac [212371,103881] 0 2024-02-19T19:04:54.914 INFO:tasks.workunit.client.1.smithi132.stdout:3/403: fsync d8/f22 0 2024-02-19T19:04:54.917 INFO:tasks.workunit.client.1.smithi132.stdout:5/488: mknod d8/da/d5a/d7b/d96/c9f 0 2024-02-19T19:04:54.917 INFO:tasks.workunit.client.1.smithi132.stdout:7/514: dwrite d1b/d1e/d2b/f95 [0,4194304] 0 2024-02-19T19:04:54.920 INFO:tasks.workunit.client.1.smithi132.stdout:9/547: fsync d12/d3e/d6e/fac 0 2024-02-19T19:04:54.920 INFO:tasks.workunit.client.1.smithi132.stdout:8/470: mknod d4/c9b 0 2024-02-19T19:04:54.924 INFO:tasks.workunit.client.1.smithi132.stdout:0/436: rename d2/dc/d4a/f6c to d2/dc/df/d15/d2f/f90 0 2024-02-19T19:04:54.925 INFO:tasks.workunit.client.1.smithi132.stdout:0/437: write d2/dc/df/d15/d2f/f64 [360123,105199] 0 2024-02-19T19:04:54.933 INFO:tasks.workunit.client.1.smithi132.stdout:2/476: truncate d0/d4/d11/f32 3905264 0 2024-02-19T19:04:54.934 INFO:tasks.workunit.client.1.smithi132.stdout:2/477: write d0/d24/d58/f8c [783914,60931] 0 2024-02-19T19:04:54.937 INFO:tasks.workunit.client.1.smithi132.stdout:6/415: creat d0/db/d25/d3a/d4b/d7c/f91 x:0 0 0 2024-02-19T19:04:54.939 INFO:tasks.workunit.client.1.smithi132.stdout:7/515: mkdir d1b/d1e/d2b/d8f/dae 0 2024-02-19T19:04:54.940 INFO:tasks.workunit.client.1.smithi132.stdout:1/410: truncate d1/f15 3185440 0 2024-02-19T19:04:54.941 INFO:tasks.workunit.client.1.smithi132.stdout:7/516: truncate d1b/d5a/d63/d6b/d79/d84/f8e 280923 0 2024-02-19T19:04:54.945 INFO:tasks.workunit.client.1.smithi132.stdout:8/471: creat d4/dc/d1b/d54/f9c x:0 0 0 2024-02-19T19:04:54.962 INFO:tasks.workunit.client.1.smithi132.stdout:2/478: fdatasync d0/d4/d11/f32 0 2024-02-19T19:04:54.964 INFO:tasks.workunit.client.1.smithi132.stdout:2/479: write d0/d4/df/f51 [442478,58223] 0 2024-02-19T19:04:54.965 INFO:tasks.workunit.client.1.smithi132.stdout:1/411: mknod d1/d3a/d79/c7e 0 2024-02-19T19:04:54.968 INFO:tasks.workunit.client.1.smithi132.stdout:5/489: link d8/da/db/l3e d8/d1a/d64/la0 0 2024-02-19T19:04:54.971 INFO:tasks.workunit.client.1.smithi132.stdout:4/422: rename d4/d19/c4a to d4/c91 0 2024-02-19T19:04:54.974 INFO:tasks.workunit.client.1.smithi132.stdout:3/404: getdents d8/d13/d2b/d52/d34 0 2024-02-19T19:04:54.974 INFO:tasks.workunit.client.1.smithi132.stdout:3/405: chown d8/d13/d19/l40 9241933 1 2024-02-19T19:04:54.976 INFO:tasks.workunit.client.1.smithi132.stdout:2/480: symlink d0/d64/d6d/l95 0 2024-02-19T19:04:54.979 INFO:tasks.workunit.client.1.smithi132.stdout:7/517: link d1b/d5a/l9b d1b/d5a/d63/d6b/d86/laf 0 2024-02-19T19:04:54.982 INFO:tasks.workunit.client.1.smithi132.stdout:2/481: dwrite d0/d24/f2d [0,4194304] 0 2024-02-19T19:04:54.982 INFO:tasks.workunit.client.1.smithi132.stdout:0/438: truncate d2/f6 551001 0 2024-02-19T19:04:54.985 INFO:tasks.workunit.client.1.smithi132.stdout:6/416: rename d0/db/d25/f34 to d0/d51/f92 0 2024-02-19T19:04:54.985 INFO:tasks.workunit.client.1.smithi132.stdout:8/472: rename d4 to d4/dc/d1b/d20/d23/d68/d79/d9d 22 2024-02-19T19:04:54.987 INFO:tasks.workunit.client.1.smithi132.stdout:1/412: creat d1/d1e/d77/d78/f7f x:0 0 0 2024-02-19T19:04:54.987 INFO:tasks.workunit.client.1.smithi132.stdout:1/413: stat d1/d1e/c28 0 2024-02-19T19:04:54.989 INFO:tasks.workunit.client.1.smithi132.stdout:9/548: sync 2024-02-19T19:04:54.993 INFO:tasks.workunit.client.1.smithi132.stdout:4/423: truncate d4/f10 630756 0 2024-02-19T19:04:54.994 INFO:tasks.workunit.client.1.smithi132.stdout:1/414: dwrite d1/f4a [4194304,4194304] 0 2024-02-19T19:04:54.995 INFO:tasks.workunit.client.1.smithi132.stdout:8/473: creat d4/dc/d1b/d20/d62/f9e x:0 0 0 2024-02-19T19:04:54.996 INFO:tasks.workunit.client.1.smithi132.stdout:9/549: dread d12/d8d/f35 [0,4194304] 0 2024-02-19T19:04:55.000 INFO:tasks.workunit.client.1.smithi132.stdout:8/474: chown d4/dc/d1b/d33/f5b 131915698 1 2024-02-19T19:04:55.001 INFO:tasks.workunit.client.1.smithi132.stdout:5/490: rename d8/d23/f50 to d8/d23/d29/fa1 0 2024-02-19T19:04:55.002 INFO:tasks.workunit.client.1.smithi132.stdout:9/550: chown d12/d3e/f94 52606 1 2024-02-19T19:04:55.003 INFO:tasks.workunit.client.1.smithi132.stdout:6/417: creat d0/db/d25/d3a/f93 x:0 0 0 2024-02-19T19:04:55.004 INFO:tasks.workunit.client.1.smithi132.stdout:4/424: symlink d4/d19/d21/l92 0 2024-02-19T19:04:55.007 INFO:tasks.workunit.client.1.smithi132.stdout:2/482: rename d0/d19/d6a/c93 to d0/d24/d58/c96 0 2024-02-19T19:04:55.008 INFO:tasks.workunit.client.1.smithi132.stdout:3/406: dread d8/d38/d44/f4c [0,4194304] 0 2024-02-19T19:04:55.009 INFO:tasks.workunit.client.1.smithi132.stdout:9/551: fdatasync d12/d8d/d3a/d40/f49 0 2024-02-19T19:04:55.015 INFO:tasks.workunit.client.1.smithi132.stdout:6/418: unlink d0/db/d25/d3a/d5d/d8f/d22/c47 0 2024-02-19T19:04:55.016 INFO:tasks.workunit.client.1.smithi132.stdout:6/419: write d0/dc/f81 [300976,27965] 0 2024-02-19T19:04:55.016 INFO:tasks.workunit.client.1.smithi132.stdout:7/518: write d1b/d46/f4d [290353,50728] 0 2024-02-19T19:04:55.021 INFO:tasks.workunit.client.1.smithi132.stdout:3/407: read d8/d13/d2f/f47 [7053721,15197] 0 2024-02-19T19:04:55.023 INFO:tasks.workunit.client.1.smithi132.stdout:0/439: dwrite d2/f1a [0,4194304] 0 2024-02-19T19:04:55.026 INFO:tasks.workunit.client.1.smithi132.stdout:9/552: creat d12/d2a/d4c/d61/fc5 x:0 0 0 2024-02-19T19:04:55.027 INFO:tasks.workunit.client.1.smithi132.stdout:9/553: chown d12/d1b/l14 306 1 2024-02-19T19:04:55.028 INFO:tasks.workunit.client.1.smithi132.stdout:5/491: mkdir d8/da/da2 0 2024-02-19T19:04:55.029 INFO:tasks.workunit.client.1.smithi132.stdout:8/475: creat d4/dc/d1b/d20/f9f x:0 0 0 2024-02-19T19:04:55.033 INFO:tasks.workunit.client.1.smithi132.stdout:1/415: getdents d1/d1e/d77/d52 0 2024-02-19T19:04:55.034 INFO:tasks.workunit.client.1.smithi132.stdout:6/420: creat d0/dc/f94 x:0 0 0 2024-02-19T19:04:55.035 INFO:tasks.workunit.client.1.smithi132.stdout:6/421: chown d0/db/d25/d38/f68 1513382 1 2024-02-19T19:04:55.037 INFO:tasks.workunit.client.1.smithi132.stdout:3/408: write d8/f51 [4493751,50257] 0 2024-02-19T19:04:55.038 INFO:tasks.workunit.client.1.smithi132.stdout:0/440: chown d2/c3 3827686 1 2024-02-19T19:04:55.038 INFO:tasks.workunit.client.1.smithi132.stdout:3/409: chown d8/d13/d2b/d2e/f6f 49439 1 2024-02-19T19:04:55.042 INFO:tasks.workunit.client.1.smithi132.stdout:4/425: creat d4/d27/d31/d32/d51/f93 x:0 0 0 2024-02-19T19:04:55.043 INFO:tasks.workunit.client.1.smithi132.stdout:8/476: chown d4/dc/d1b/d20/d23/d68/d79/c99 2 1 2024-02-19T19:04:55.045 INFO:tasks.workunit.client.1.smithi132.stdout:7/519: mknod d1b/d1e/d28/da9/cb0 0 2024-02-19T19:04:55.047 INFO:tasks.workunit.client.1.smithi132.stdout:6/422: symlink d0/d8b/l95 0 2024-02-19T19:04:55.048 INFO:tasks.workunit.client.1.smithi132.stdout:2/483: rmdir d0/d3c/d3f/d89 0 2024-02-19T19:04:55.049 INFO:tasks.workunit.client.1.smithi132.stdout:0/441: fsync d2/f5c 0 2024-02-19T19:04:55.050 INFO:tasks.workunit.client.1.smithi132.stdout:6/423: write d0/db/d25/d3a/d3d/d69/f5f [2464975,75406] 0 2024-02-19T19:04:55.053 INFO:tasks.workunit.client.1.smithi132.stdout:9/554: creat d12/d8d/d3a/d40/d4e/d77/d8a/d82/fc6 x:0 0 0 2024-02-19T19:04:55.054 INFO:tasks.workunit.client.1.smithi132.stdout:7/520: dwrite d1b/d1e/d28/f3b [0,4194304] 0 2024-02-19T19:04:55.057 INFO:tasks.workunit.client.1.smithi132.stdout:1/416: dread d1/d1e/d77/d52/f56 [0,4194304] 0 2024-02-19T19:04:55.059 INFO:tasks.workunit.client.1.smithi132.stdout:9/555: unlink d12/cc1 0 2024-02-19T19:04:55.060 INFO:tasks.workunit.client.1.smithi132.stdout:0/442: dwrite d2/dc/d4a/d25/d43/d5f/f73 [0,4194304] 0 2024-02-19T19:04:55.060 INFO:tasks.workunit.client.1.smithi132.stdout:9/556: chown d12/d8d/f30 1286198361 1 2024-02-19T19:04:55.061 INFO:tasks.workunit.client.1.smithi132.stdout:0/443: write d2/dc/d4a/d74/f86 [159091,131042] 0 2024-02-19T19:04:55.063 INFO:tasks.workunit.client.1.smithi132.stdout:4/426: rename d4/d27/d31/d71/c1d to d4/d4d/c94 0 2024-02-19T19:04:55.064 INFO:tasks.workunit.client.1.smithi132.stdout:9/557: write d12/d1b/f50 [2623860,115760] 0 2024-02-19T19:04:55.067 INFO:tasks.workunit.client.1.smithi132.stdout:5/492: truncate d8/d23/f43 1502904 0 2024-02-19T19:04:55.068 INFO:tasks.workunit.client.1.smithi132.stdout:5/493: chown d8/da/db 749465954 1 2024-02-19T19:04:55.072 INFO:tasks.workunit.client.1.smithi132.stdout:4/427: dwrite d4/d19/f3f [0,4194304] 0 2024-02-19T19:04:55.074 INFO:tasks.workunit.client.1.smithi132.stdout:8/477: rename d4/dc/d1b/d20/d23/d68/d79/c99 to d4/dc/d1b/d38/ca0 0 2024-02-19T19:04:55.077 INFO:tasks.workunit.client.1.smithi132.stdout:7/521: unlink d1b/d1e/d2b/l3e 0 2024-02-19T19:04:55.077 INFO:tasks.workunit.client.1.smithi132.stdout:8/478: dread d4/dc/d42/f6f [0,4194304] 0 2024-02-19T19:04:55.080 INFO:tasks.workunit.client.1.smithi132.stdout:7/522: fsync d1b/d5a/d63/d96/fad 0 2024-02-19T19:04:55.080 INFO:tasks.workunit.client.1.smithi132.stdout:2/484: dwrite d0/d4/df/f3b [0,4194304] 0 2024-02-19T19:04:55.081 INFO:tasks.workunit.client.1.smithi132.stdout:2/485: chown d0/d4/c8 60676714 1 2024-02-19T19:04:55.082 INFO:tasks.workunit.client.1.smithi132.stdout:0/444: symlink d2/dc/df/d15/d2f/d3f/d63/l91 0 2024-02-19T19:04:55.084 INFO:tasks.workunit.client.1.smithi132.stdout:8/479: mknod d4/d2d/d82/ca1 0 2024-02-19T19:04:55.087 INFO:tasks.workunit.client.1.smithi132.stdout:1/417: creat d1/d3a/f80 x:0 0 0 2024-02-19T19:04:55.088 INFO:tasks.workunit.client.1.smithi132.stdout:6/424: rename d0/f12 to d0/db/d25/d3a/d5d/f96 0 2024-02-19T19:04:55.088 INFO:tasks.workunit.client.1.smithi132.stdout:9/558: dwrite d12/f26 [0,4194304] 0 2024-02-19T19:04:55.092 INFO:tasks.workunit.client.1.smithi132.stdout:4/428: truncate d4/d12/f2f 3606078 0 2024-02-19T19:04:55.094 INFO:tasks.workunit.client.1.smithi132.stdout:0/445: mkdir d2/dc/d4a/d25/d43/d92 0 2024-02-19T19:04:55.096 INFO:tasks.workunit.client.1.smithi132.stdout:9/559: mknod d12/d8d/d3a/d40/d4e/cc7 0 2024-02-19T19:04:55.096 INFO:tasks.workunit.client.1.smithi132.stdout:2/486: write d0/d4/d11/d2e/d5a/f5b [7542209,90174] 0 2024-02-19T19:04:55.099 INFO:tasks.workunit.client.1.smithi132.stdout:2/487: chown d0/d4/l31 10991215 1 2024-02-19T19:04:55.099 INFO:tasks.workunit.client.1.smithi132.stdout:6/425: symlink d0/db/d25/l97 0 2024-02-19T19:04:55.099 INFO:tasks.workunit.client.1.smithi132.stdout:7/523: dwrite d1b/d1e/d2b/f95 [0,4194304] 0 2024-02-19T19:04:55.103 INFO:tasks.workunit.client.1.smithi132.stdout:2/488: write d0/d64/d6d/f94 [803299,68991] 0 2024-02-19T19:04:55.104 INFO:tasks.workunit.client.1.smithi132.stdout:1/418: dwrite d1/d1e/d77/d2c/d43/f5d [0,4194304] 0 2024-02-19T19:04:55.108 INFO:tasks.workunit.client.1.smithi132.stdout:9/560: mkdir d12/d8d/d3a/d85/d87/dc8 0 2024-02-19T19:04:55.109 INFO:tasks.workunit.client.1.smithi132.stdout:6/426: creat d0/d51/f98 x:0 0 0 2024-02-19T19:04:55.111 INFO:tasks.workunit.client.1.smithi132.stdout:7/524: dwrite d1b/d1e/d2b/f2f [0,4194304] 0 2024-02-19T19:04:55.112 INFO:tasks.workunit.client.1.smithi132.stdout:2/489: rename d0/d4/d11/f41 to d0/d19/d1e/f97 0 2024-02-19T19:04:55.114 INFO:tasks.workunit.client.1.smithi132.stdout:6/427: chown d0/db/d25/d3a/d5d/d8f/d46/d5a/f18 65581376 1 2024-02-19T19:04:55.114 INFO:tasks.workunit.client.1.smithi132.stdout:1/419: mkdir d1/d1e/d77/d2c/d43/d81 0 2024-02-19T19:04:55.118 INFO:tasks.workunit.client.1.smithi132.stdout:4/429: creat d4/d4d/d87/f95 x:0 0 0 2024-02-19T19:04:55.119 INFO:tasks.workunit.client.1.smithi132.stdout:3/410: sync 2024-02-19T19:04:55.119 INFO:tasks.workunit.client.1.smithi132.stdout:4/430: fdatasync d4/d12/d28/f62 0 2024-02-19T19:04:55.120 INFO:tasks.workunit.client.1.smithi132.stdout:6/428: truncate d0/db/d25/d3a/d3d/d69/f5c 419950 0 2024-02-19T19:04:55.123 INFO:tasks.workunit.client.1.smithi132.stdout:6/429: chown d0/db/d25/l52 487 1 2024-02-19T19:04:55.124 INFO:tasks.workunit.client.1.smithi132.stdout:2/490: dwrite d0/d4/d11/d2e/f6c [0,4194304] 0 2024-02-19T19:04:55.130 INFO:tasks.workunit.client.1.smithi132.stdout:9/561: rename d12/d2a/cb7 to d12/d2a/d4c/cc9 0 2024-02-19T19:04:55.131 INFO:tasks.workunit.client.1.smithi132.stdout:8/480: write d4/dc/f2c [1022239,93096] 0 2024-02-19T19:04:55.133 INFO:tasks.workunit.client.1.smithi132.stdout:6/430: dwrite d0/dc/f14 [4194304,4194304] 0 2024-02-19T19:04:55.135 INFO:tasks.workunit.client.1.smithi132.stdout:0/446: dread d2/dc/d4a/f42 [0,4194304] 0 2024-02-19T19:04:55.136 INFO:tasks.workunit.client.1.smithi132.stdout:3/411: write d8/d13/f25 [2183462,17090] 0 2024-02-19T19:04:55.137 INFO:tasks.workunit.client.1.smithi132.stdout:0/447: dread - d2/dc/df/d15/d2f/d3f/f8d zero size 2024-02-19T19:04:55.139 INFO:tasks.workunit.client.1.smithi132.stdout:2/491: creat d0/d24/d58/f98 x:0 0 0 2024-02-19T19:04:55.139 INFO:tasks.workunit.client.1.smithi132.stdout:1/420: link d1/d1e/f5a d1/d1e/d77/d4e/f82 0 2024-02-19T19:04:55.141 INFO:tasks.workunit.client.1.smithi132.stdout:2/492: write d0/d3c/d3f/f81 [695044,68708] 0 2024-02-19T19:04:55.143 INFO:tasks.workunit.client.1.smithi132.stdout:8/481: readlink d4/la 0 2024-02-19T19:04:55.144 INFO:tasks.workunit.client.1.smithi132.stdout:3/412: symlink d8/d13/d2f/l7f 0 2024-02-19T19:04:55.145 INFO:tasks.workunit.client.1.smithi132.stdout:0/448: dwrite d2/dc/df/d15/f24 [0,4194304] 0 2024-02-19T19:04:55.147 INFO:tasks.workunit.client.1.smithi132.stdout:8/482: dread - d4/dc/d1b/d20/d23/d68/d7a/f94 zero size 2024-02-19T19:04:55.148 INFO:tasks.workunit.client.1.smithi132.stdout:8/483: chown d4/fd 195 1 2024-02-19T19:04:55.149 INFO:tasks.workunit.client.1.smithi132.stdout:8/484: chown d4/dc/d1b/d20/d23/f2b 449 1 2024-02-19T19:04:55.154 INFO:tasks.workunit.client.1.smithi132.stdout:0/449: dwrite d2/dc/d4a/d25/d43/f6f [0,4194304] 0 2024-02-19T19:04:55.154 INFO:tasks.workunit.client.1.smithi132.stdout:0/450: chown d2/dc/df 2 1 2024-02-19T19:04:55.156 INFO:tasks.workunit.client.1.smithi132.stdout:5/494: sync 2024-02-19T19:04:55.163 INFO:tasks.workunit.client.1.smithi132.stdout:7/525: fsync d1b/d1e/d2b/f2f 0 2024-02-19T19:04:55.164 INFO:tasks.workunit.client.1.smithi132.stdout:2/493: rmdir d0 39 2024-02-19T19:04:55.167 INFO:tasks.workunit.client.1.smithi132.stdout:6/431: mknod d0/db/c99 0 2024-02-19T19:04:55.168 INFO:tasks.workunit.client.1.smithi132.stdout:1/421: creat d1/d1e/d77/d52/d64/d75/f83 x:0 0 0 2024-02-19T19:04:55.173 INFO:tasks.workunit.client.1.smithi132.stdout:0/451: chown d2/l16 48 1 2024-02-19T19:04:55.175 INFO:tasks.workunit.client.1.smithi132.stdout:5/495: mknod d8/d1a/ca3 0 2024-02-19T19:04:55.177 INFO:tasks.workunit.client.1.smithi132.stdout:5/496: truncate d8/d23/f6f 1652756 0 2024-02-19T19:04:55.178 INFO:tasks.workunit.client.1.smithi132.stdout:2/494: dread - d0/d24/d58/d53/f91 zero size 2024-02-19T19:04:55.180 INFO:tasks.workunit.client.1.smithi132.stdout:7/526: dwrite ff [0,4194304] 0 2024-02-19T19:04:55.182 INFO:tasks.workunit.client.1.smithi132.stdout:7/527: write d1b/d1e/d2b/d8f/da5/fa8 [269598,51494] 0 2024-02-19T19:04:55.182 INFO:tasks.workunit.client.1.smithi132.stdout:7/528: fdatasync d1b/d1e/d28/f3b 0 2024-02-19T19:04:55.183 INFO:tasks.workunit.client.1.smithi132.stdout:7/529: write d1b/d1e/d28/f4c [2092391,52214] 0 2024-02-19T19:04:55.184 INFO:tasks.workunit.client.1.smithi132.stdout:9/562: getdents d12/d8d/d3a 0 2024-02-19T19:04:55.185 INFO:tasks.workunit.client.1.smithi132.stdout:9/563: fdatasync d12/d2a/f37 0 2024-02-19T19:04:55.186 INFO:tasks.workunit.client.1.smithi132.stdout:5/497: creat d8/d23/d3b/fa4 x:0 0 0 2024-02-19T19:04:55.189 INFO:tasks.workunit.client.1.smithi132.stdout:5/498: dread d8/d23/d3b/f65 [0,4194304] 0 2024-02-19T19:04:55.191 INFO:tasks.workunit.client.1.smithi132.stdout:2/495: creat d0/d4/df/d7a/f99 x:0 0 0 2024-02-19T19:04:55.194 INFO:tasks.workunit.client.1.smithi132.stdout:4/431: link d4/d12/d48/f65 d4/d27/d31/d71/d37/f96 0 2024-02-19T19:04:55.195 INFO:tasks.workunit.client.1.smithi132.stdout:8/485: truncate d4/dc/d1b/d33/f5b 3562937 0 2024-02-19T19:04:55.196 INFO:tasks.workunit.client.1.smithi132.stdout:8/486: truncate d4/dc/d42/f56 995510 0 2024-02-19T19:04:55.197 INFO:tasks.workunit.client.1.smithi132.stdout:3/413: rmdir d8/d13/d2b/d52/d34/d42/d6c/d77 0 2024-02-19T19:04:55.201 INFO:tasks.workunit.client.1.smithi132.stdout:6/432: dwrite d0/db/d25/d3a/d5d/d8f/d22/f9 [0,4194304] 0 2024-02-19T19:04:55.203 INFO:tasks.workunit.client.1.smithi132.stdout:7/530: symlink d1b/d5a/d63/d96/lb1 0 2024-02-19T19:04:55.206 INFO:tasks.workunit.client.1.smithi132.stdout:9/564: fsync d12/f53 0 2024-02-19T19:04:55.208 INFO:tasks.workunit.client.1.smithi132.stdout:5/499: creat d8/d2e/fa5 x:0 0 0 2024-02-19T19:04:55.208 INFO:tasks.workunit.client.1.smithi132.stdout:5/500: chown d8/d1a/d35/f4a 3 1 2024-02-19T19:04:55.209 INFO:tasks.workunit.client.1.smithi132.stdout:1/422: creat d1/d1e/f84 x:0 0 0 2024-02-19T19:04:55.209 INFO:tasks.workunit.client.1.smithi132.stdout:5/501: stat d8/da/d5a/d7b/f88 0 2024-02-19T19:04:55.211 INFO:tasks.workunit.client.1.smithi132.stdout:1/423: write d1/d1e/d77/d2c/f59 [1068324,107089] 0 2024-02-19T19:04:55.213 INFO:tasks.workunit.client.1.smithi132.stdout:2/496: dread d0/d19/f25 [4194304,4194304] 0 2024-02-19T19:04:55.215 INFO:tasks.workunit.client.1.smithi132.stdout:3/414: unlink l5 0 2024-02-19T19:04:55.217 INFO:tasks.workunit.client.1.smithi132.stdout:1/424: dwrite d1/d1e/d77/d4e/f5f [0,4194304] 0 2024-02-19T19:04:55.222 INFO:tasks.workunit.client.1.smithi132.stdout:0/452: truncate d2/dc/fe 1223664 0 2024-02-19T19:04:55.236 INFO:tasks.workunit.client.1.smithi132.stdout:8/487: write d4/dc/d1b/d54/f87 [604710,89618] 0 2024-02-19T19:04:55.238 INFO:tasks.workunit.client.1.smithi132.stdout:2/497: rename d0/d19/d6a/l70 to d0/d64/d6d/l9a 0 2024-02-19T19:04:55.243 INFO:tasks.workunit.client.1.smithi132.stdout:1/425: truncate d1/f42 2778748 0 2024-02-19T19:04:55.244 INFO:tasks.workunit.client.1.smithi132.stdout:1/426: fsync d1/d1e/d77/d2c/d43/f5d 0 2024-02-19T19:04:55.249 INFO:tasks.workunit.client.1.smithi132.stdout:9/565: dread d12/d2a/f2c [0,4194304] 0 2024-02-19T19:04:55.249 INFO:tasks.workunit.client.1.smithi132.stdout:4/432: creat d4/d27/d31/d32/d51/f97 x:0 0 0 2024-02-19T19:04:55.251 INFO:tasks.workunit.client.1.smithi132.stdout:9/566: read d12/d2a/f2b [2086710,103397] 0 2024-02-19T19:04:55.251 INFO:tasks.workunit.client.1.smithi132.stdout:9/567: stat d12/d2a/d4c/f9b 0 2024-02-19T19:04:55.252 INFO:tasks.workunit.client.1.smithi132.stdout:8/488: creat d4/dc/d1b/d20/d8a/fa2 x:0 0 0 2024-02-19T19:04:55.261 INFO:tasks.workunit.client.1.smithi132.stdout:4/433: unlink d4/d27/f3c 0 2024-02-19T19:04:55.264 INFO:tasks.workunit.client.1.smithi132.stdout:7/531: write d1b/d1e/d2b/d62/d53/f81 [4614104,89457] 0 2024-02-19T19:04:55.265 INFO:tasks.workunit.client.1.smithi132.stdout:7/532: chown d1b/f72 591 1 2024-02-19T19:04:55.268 INFO:tasks.workunit.client.1.smithi132.stdout:2/498: dwrite d0/f5 [0,4194304] 0 2024-02-19T19:04:55.268 INFO:tasks.workunit.client.1.smithi132.stdout:6/433: getdents d0/db/d25/d3a/d5d/d8f/d22 0 2024-02-19T19:04:55.269 INFO:tasks.workunit.client.1.smithi132.stdout:2/499: fdatasync d0/d4/d11/f20 0 2024-02-19T19:04:55.270 INFO:tasks.workunit.client.1.smithi132.stdout:1/427: write d1/f48 [3413879,5152] 0 2024-02-19T19:04:55.272 INFO:tasks.workunit.client.1.smithi132.stdout:5/502: getdents d8/d1a/d35/d77 0 2024-02-19T19:04:55.273 INFO:tasks.workunit.client.1.smithi132.stdout:4/434: mkdir d4/d27/d31/d71/d37/d66/d98 0 2024-02-19T19:04:55.273 INFO:tasks.workunit.client.1.smithi132.stdout:1/428: write d1/d1e/d77/f41 [1053537,53082] 0 2024-02-19T19:04:55.275 INFO:tasks.workunit.client.1.smithi132.stdout:8/489: mkdir d4/dc/d1b/d20/d23/da3 0 2024-02-19T19:04:55.277 INFO:tasks.workunit.client.1.smithi132.stdout:0/453: getdents d2/dc/df/d15/d2f 0 2024-02-19T19:04:55.281 INFO:tasks.workunit.client.1.smithi132.stdout:8/490: dwrite d4/dc/d1b/d20/f6e [0,4194304] 0 2024-02-19T19:04:55.281 INFO:tasks.workunit.client.1.smithi132.stdout:2/500: rmdir d0/d4/d11 39 2024-02-19T19:04:55.282 INFO:tasks.workunit.client.1.smithi132.stdout:9/568: write d12/d8d/d3a/d40/d4e/d77/f8e [719234,68909] 0 2024-02-19T19:04:55.283 INFO:tasks.workunit.client.1.smithi132.stdout:5/503: mkdir d8/d2e/d71/da6 0 2024-02-19T19:04:55.285 INFO:tasks.workunit.client.1.smithi132.stdout:4/435: chown d4/d12/f30 39146 1 2024-02-19T19:04:55.285 INFO:tasks.workunit.client.1.smithi132.stdout:8/491: read d4/dc/d1b/d20/d23/f28 [404304,6715] 0 2024-02-19T19:04:55.285 INFO:tasks.workunit.client.1.smithi132.stdout:9/569: fsync d12/d8d/d3a/d40/d4e/d77/d8a/d82/fc6 0 2024-02-19T19:04:55.285 INFO:tasks.workunit.client.1.smithi132.stdout:9/570: chown d12 108057458 1 2024-02-19T19:04:55.289 INFO:tasks.workunit.client.1.smithi132.stdout:0/454: symlink d2/dc/df/d15/d2f/d69/l93 0 2024-02-19T19:04:55.291 INFO:tasks.workunit.client.1.smithi132.stdout:2/501: symlink d0/d4/df/d7a/l9b 0 2024-02-19T19:04:55.293 INFO:tasks.workunit.client.1.smithi132.stdout:4/436: fdatasync d4/d12/f4b 0 2024-02-19T19:04:55.293 INFO:tasks.workunit.client.1.smithi132.stdout:8/492: mknod d4/dc/d1b/d38/ca4 0 2024-02-19T19:04:55.296 INFO:tasks.workunit.client.1.smithi132.stdout:0/455: dwrite d2/f5c [0,4194304] 0 2024-02-19T19:04:55.296 INFO:tasks.workunit.client.1.smithi132.stdout:9/571: mkdir d12/d8d/d3a/d40/d4e/d77/d8a/dca 0 2024-02-19T19:04:55.297 INFO:tasks.workunit.client.1.smithi132.stdout:6/434: truncate d0/dc/f24 3176443 0 2024-02-19T19:04:55.300 INFO:tasks.workunit.client.1.smithi132.stdout:5/504: symlink d8/la7 0 2024-02-19T19:04:55.300 INFO:tasks.workunit.client.1.smithi132.stdout:3/415: sync 2024-02-19T19:04:55.300 INFO:tasks.workunit.client.1.smithi132.stdout:0/456: write d2/dc/d4a/d25/d43/d5f/f73 [827392,90903] 0 2024-02-19T19:04:55.301 INFO:tasks.workunit.client.1.smithi132.stdout:4/437: symlink d4/d12/d28/l99 0 2024-02-19T19:04:55.302 INFO:tasks.workunit.client.1.smithi132.stdout:2/502: dwrite d0/d3c/d3f/f81 [0,4194304] 0 2024-02-19T19:04:55.303 INFO:tasks.workunit.client.1.smithi132.stdout:1/429: mkdir d1/d1e/d77/d2c/d62/d6b/d85 0 2024-02-19T19:04:55.303 INFO:tasks.workunit.client.1.smithi132.stdout:8/493: creat d4/dc/d1b/d33/fa5 x:0 0 0 2024-02-19T19:04:55.305 INFO:tasks.workunit.client.1.smithi132.stdout:2/503: fdatasync d0/d4/df/d40/f74 0 2024-02-19T19:04:55.306 INFO:tasks.workunit.client.1.smithi132.stdout:4/438: chown d4/d27/d31/d32/c8d 413 1 2024-02-19T19:04:55.308 INFO:tasks.workunit.client.1.smithi132.stdout:4/439: fdatasync d4/d19/f23 0 2024-02-19T19:04:55.308 INFO:tasks.workunit.client.1.smithi132.stdout:9/572: unlink d12/d8d/f35 0 2024-02-19T19:04:55.310 INFO:tasks.workunit.client.1.smithi132.stdout:6/435: dwrite d0/db/d25/d3a/f93 [0,4194304] 0 2024-02-19T19:04:55.314 INFO:tasks.workunit.client.1.smithi132.stdout:5/505: unlink d8/d1a/d35/d77/f6b 0 2024-02-19T19:04:55.317 INFO:tasks.workunit.client.1.smithi132.stdout:5/506: write d8/da/f30 [4072959,41681] 0 2024-02-19T19:04:55.317 INFO:tasks.workunit.client.1.smithi132.stdout:0/457: mknod d2/dc/df/d15/d2f/c94 0 2024-02-19T19:04:55.320 INFO:tasks.workunit.client.1.smithi132.stdout:9/573: dwrite d12/d3e/f6c [0,4194304] 0 2024-02-19T19:04:55.321 INFO:tasks.workunit.client.1.smithi132.stdout:7/533: truncate d1b/d1e/d28/f4c 798598 0 2024-02-19T19:04:55.323 INFO:tasks.workunit.client.1.smithi132.stdout:7/534: stat d1b/d1e/d28/faa 0 2024-02-19T19:04:55.331 INFO:tasks.workunit.client.1.smithi132.stdout:0/458: dwrite d2/dc/df/d15/d2f/d3f/d63/d6b/f72 [0,4194304] 0 2024-02-19T19:04:55.331 INFO:tasks.workunit.client.1.smithi132.stdout:8/494: dread d4/d2d/d82/f98 [0,4194304] 0 2024-02-19T19:04:55.331 INFO:tasks.workunit.client.1.smithi132.stdout:0/459: stat d2/dc 0 2024-02-19T19:04:55.332 INFO:tasks.workunit.client.1.smithi132.stdout:4/440: rename d4/d19/f23 to d4/d4d/d87/f9a 0 2024-02-19T19:04:55.334 INFO:tasks.workunit.client.1.smithi132.stdout:6/436: creat d0/db/d25/d3a/d4b/f9a x:0 0 0 2024-02-19T19:04:55.344 INFO:tasks.workunit.client.1.smithi132.stdout:5/507: unlink d8/d23/d29/f2d 0 2024-02-19T19:04:55.345 INFO:tasks.workunit.client.1.smithi132.stdout:5/508: dread - d8/da/d5a/d7b/f88 zero size 2024-02-19T19:04:55.347 INFO:tasks.workunit.client.1.smithi132.stdout:9/574: symlink d12/d2a/d4c/lcb 0 2024-02-19T19:04:55.347 INFO:tasks.workunit.client.1.smithi132.stdout:9/575: read - d12/d2a/d4c/d61/fa8 zero size 2024-02-19T19:04:55.348 INFO:tasks.workunit.client.1.smithi132.stdout:9/576: fsync d12/d2a/d4c/f6a 0 2024-02-19T19:04:55.350 INFO:tasks.workunit.client.1.smithi132.stdout:7/535: unlink d1b/d1e/d2b/d45/c7a 0 2024-02-19T19:04:55.356 INFO:tasks.workunit.client.1.smithi132.stdout:0/460: chown d2/d3e/c4f 95601389 1 2024-02-19T19:04:55.357 INFO:tasks.workunit.client.1.smithi132.stdout:7/536: dwrite d1b/d46/d56/f57 [0,4194304] 0 2024-02-19T19:04:55.358 INFO:tasks.workunit.client.1.smithi132.stdout:4/441: mknod d4/d27/d31/d71/c9b 0 2024-02-19T19:04:55.363 INFO:tasks.workunit.client.1.smithi132.stdout:6/437: fsync d0/db/fd 0 2024-02-19T19:04:55.365 INFO:tasks.workunit.client.1.smithi132.stdout:5/509: dread d8/f62 [0,4194304] 0 2024-02-19T19:04:55.366 INFO:tasks.workunit.client.1.smithi132.stdout:6/438: dread d0/dc/f81 [0,4194304] 0 2024-02-19T19:04:55.366 INFO:tasks.workunit.client.1.smithi132.stdout:5/510: dread - d8/d23/d29/d3f/f59 zero size 2024-02-19T19:04:55.367 INFO:tasks.workunit.client.1.smithi132.stdout:6/439: chown d0/dc 0 1 2024-02-19T19:04:55.367 INFO:tasks.workunit.client.1.smithi132.stdout:5/511: stat d8/da/db/f83 0 2024-02-19T19:04:55.368 INFO:tasks.workunit.client.1.smithi132.stdout:9/577: symlink d12/d8d/d3a/d40/d4e/d77/d89/lcc 0 2024-02-19T19:04:55.371 INFO:tasks.workunit.client.1.smithi132.stdout:0/461: creat d2/dc/d4a/d58/f95 x:0 0 0 2024-02-19T19:04:55.373 INFO:tasks.workunit.client.1.smithi132.stdout:7/537: unlink d1b/f72 0 2024-02-19T19:04:55.374 INFO:tasks.workunit.client.1.smithi132.stdout:4/442: chown d4/f10 5 1 2024-02-19T19:04:55.374 INFO:tasks.workunit.client.1.smithi132.stdout:4/443: chown d4/d27/d31/f68 127195500 1 2024-02-19T19:04:55.382 INFO:tasks.workunit.client.1.smithi132.stdout:1/430: getdents d1/d1e/d77/d2c 0 2024-02-19T19:04:55.383 INFO:tasks.workunit.client.1.smithi132.stdout:9/578: creat d12/d8d/d3a/d40/d4e/d77/d9d/fcd x:0 0 0 2024-02-19T19:04:55.384 INFO:tasks.workunit.client.1.smithi132.stdout:1/431: write d1/d1e/d77/f5e [5116845,37457] 0 2024-02-19T19:04:55.384 INFO:tasks.workunit.client.1.smithi132.stdout:9/579: readlink d12/l41 0 2024-02-19T19:04:55.384 INFO:tasks.workunit.client.1.smithi132.stdout:1/432: fsync d1/d1e/f5c 0 2024-02-19T19:04:55.387 INFO:tasks.workunit.client.1.smithi132.stdout:0/462: creat d2/dc/df/d15/d2f/d47/f96 x:0 0 0 2024-02-19T19:04:55.388 INFO:tasks.workunit.client.1.smithi132.stdout:8/495: link d4/dc/d1b/c55 d4/dc/d1b/d38/d69/ca6 0 2024-02-19T19:04:55.390 INFO:tasks.workunit.client.1.smithi132.stdout:0/463: dread d2/dc/df/d15/d2f/f64 [0,4194304] 0 2024-02-19T19:04:55.391 INFO:tasks.workunit.client.1.smithi132.stdout:4/444: dread d4/d12/f1a [0,4194304] 0 2024-02-19T19:04:55.391 INFO:tasks.workunit.client.1.smithi132.stdout:3/416: rename d8/d13/d2b/d52/l53 to d8/d13/l80 0 2024-02-19T19:04:55.393 INFO:tasks.workunit.client.1.smithi132.stdout:6/440: unlink d0/db/d25/d3a/d5d/f96 0 2024-02-19T19:04:55.396 INFO:tasks.workunit.client.1.smithi132.stdout:2/504: link d0/d4/df/c28 d0/d4/d11/d2e/c9c 0 2024-02-19T19:04:55.397 INFO:tasks.workunit.client.1.smithi132.stdout:0/464: dwrite d2/dc/df/d15/d2f/d47/f96 [0,4194304] 0 2024-02-19T19:04:55.399 INFO:tasks.workunit.client.1.smithi132.stdout:7/538: getdents d1b/d1e/d2b/d8f/dae 0 2024-02-19T19:04:55.399 INFO:tasks.workunit.client.1.smithi132.stdout:7/539: fdatasync d1b/d1e/d2b/d62/d53/f81 0 2024-02-19T19:04:55.405 INFO:tasks.workunit.client.1.smithi132.stdout:3/417: chown d8/l35 31 1 2024-02-19T19:04:55.406 INFO:tasks.workunit.client.1.smithi132.stdout:3/418: write d8/d13/d2b/d2e/f6f [336274,61311] 0 2024-02-19T19:04:55.407 INFO:tasks.workunit.client.1.smithi132.stdout:1/433: rename d1/d1e/d77/d52/l7a to d1/d1e/d77/l86 0 2024-02-19T19:04:55.412 INFO:tasks.workunit.client.1.smithi132.stdout:5/512: write d8/d23/d29/d27/f87 [1566017,107727] 0 2024-02-19T19:04:55.415 INFO:tasks.workunit.client.1.smithi132.stdout:2/505: creat d0/d19/d1e/f9d x:0 0 0 2024-02-19T19:04:55.417 INFO:tasks.workunit.client.1.smithi132.stdout:9/580: write d12/d8d/d3a/d40/f49 [852112,29215] 0 2024-02-19T19:04:55.420 INFO:tasks.workunit.client.1.smithi132.stdout:7/540: fsync d1b/d1e/d28/faa 0 2024-02-19T19:04:55.421 INFO:tasks.workunit.client.1.smithi132.stdout:7/541: write d1b/d1e/f37 [2172383,77066] 0 2024-02-19T19:04:55.421 INFO:tasks.workunit.client.1.smithi132.stdout:7/542: fdatasync d1b/d1e/d28/f77 0 2024-02-19T19:04:55.422 INFO:tasks.workunit.client.1.smithi132.stdout:8/496: creat d4/dc/d1b/d8f/fa7 x:0 0 0 2024-02-19T19:04:55.423 INFO:tasks.workunit.client.1.smithi132.stdout:8/497: write d4/dc/d1b/d33/f49 [1667575,8735] 0 2024-02-19T19:04:55.424 INFO:tasks.workunit.client.1.smithi132.stdout:3/419: creat d8/d13/d2b/d52/d34/f81 x:0 0 0 2024-02-19T19:04:55.426 INFO:tasks.workunit.client.1.smithi132.stdout:6/441: rename d0/db/c65 to d0/db/d25/d50/d86/c9b 0 2024-02-19T19:04:55.427 INFO:tasks.workunit.client.1.smithi132.stdout:1/434: creat d1/d1e/d77/d2c/d62/f87 x:0 0 0 2024-02-19T19:04:55.431 INFO:tasks.workunit.client.1.smithi132.stdout:0/465: mknod d2/dc/d4a/d25/d7f/c97 0 2024-02-19T19:04:55.433 INFO:tasks.workunit.client.1.smithi132.stdout:1/435: dwrite d1/d1e/d77/d2c/f59 [0,4194304] 0 2024-02-19T19:04:55.435 INFO:tasks.workunit.client.1.smithi132.stdout:0/466: dread d2/dc/d4a/f42 [4194304,4194304] 0 2024-02-19T19:04:55.436 INFO:tasks.workunit.client.1.smithi132.stdout:0/467: dread - d2/f8c zero size 2024-02-19T19:04:55.437 INFO:tasks.workunit.client.1.smithi132.stdout:7/543: truncate d1b/f1c 3713532 0 2024-02-19T19:04:55.438 INFO:tasks.workunit.client.1.smithi132.stdout:4/445: truncate d4/d27/d31/f79 34684 0 2024-02-19T19:04:55.442 INFO:tasks.workunit.client.1.smithi132.stdout:9/581: write d12/d8d/f28 [486286,99094] 0 2024-02-19T19:04:55.442 INFO:tasks.workunit.client.1.smithi132.stdout:3/420: write d8/f51 [4906109,108614] 0 2024-02-19T19:04:55.444 INFO:tasks.workunit.client.1.smithi132.stdout:9/582: chown d12/d8d/d3a/d40/d4e/d77/d89/lcc 1836 1 2024-02-19T19:04:55.447 INFO:tasks.workunit.client.1.smithi132.stdout:8/498: dwrite d4/dc/d1b/d54/d5e/f50 [0,4194304] 0 2024-02-19T19:04:55.447 INFO:tasks.workunit.client.1.smithi132.stdout:9/583: write d12/d8d/d3a/d40/d4e/f67 [957455,92728] 0 2024-02-19T19:04:55.450 INFO:tasks.workunit.client.1.smithi132.stdout:8/499: stat d4/dc/d1b/d20/d8a/fa2 0 2024-02-19T19:04:55.451 INFO:tasks.workunit.client.1.smithi132.stdout:2/506: rename d0/d24/d58/d71 to d0/d4/df/d40/d7b/d9e 0 2024-02-19T19:04:55.451 INFO:tasks.workunit.client.1.smithi132.stdout:6/442: fdatasync d0/d51/f92 0 2024-02-19T19:04:55.453 INFO:tasks.workunit.client.1.smithi132.stdout:5/513: mknod d8/da/d5a/ca8 0 2024-02-19T19:04:55.454 INFO:tasks.workunit.client.1.smithi132.stdout:3/421: dwrite d8/d13/f25 [4194304,4194304] 0 2024-02-19T19:04:55.454 INFO:tasks.workunit.client.1.smithi132.stdout:0/468: mkdir d2/dc/d98 0 2024-02-19T19:04:55.456 INFO:tasks.workunit.client.1.smithi132.stdout:1/436: chown d1/d3a/c1f 261958654 1 2024-02-19T19:04:55.457 INFO:tasks.workunit.client.1.smithi132.stdout:7/544: rename d1b/d1e/d2b/f80 to d1b/d1e/fb2 0 2024-02-19T19:04:55.462 INFO:tasks.workunit.client.1.smithi132.stdout:1/437: chown d1/d3a/d50/f67 2 1 2024-02-19T19:04:55.462 INFO:tasks.workunit.client.1.smithi132.stdout:7/545: fdatasync d1b/d46/d56/f57 0 2024-02-19T19:04:55.463 INFO:tasks.workunit.client.1.smithi132.stdout:2/507: creat d0/d19/d1e/f9f x:0 0 0 2024-02-19T19:04:55.467 INFO:tasks.workunit.client.1.smithi132.stdout:9/584: dwrite d12/d8d/d3a/d40/d4e/f67 [0,4194304] 0 2024-02-19T19:04:55.468 INFO:tasks.workunit.client.1.smithi132.stdout:5/514: mkdir d8/d1a/d35/d77/da9 0 2024-02-19T19:04:55.468 INFO:tasks.workunit.client.1.smithi132.stdout:6/443: rmdir d0/db/d25/d3a/d5d/d8f/d46/d5a 39 2024-02-19T19:04:55.469 INFO:tasks.workunit.client.1.smithi132.stdout:8/500: dwrite d4/dc/d1b/d33/f49 [0,4194304] 0 2024-02-19T19:04:55.470 INFO:tasks.workunit.client.1.smithi132.stdout:2/508: dread d0/d4/d11/d2e/f75 [0,4194304] 0 2024-02-19T19:04:55.472 INFO:tasks.workunit.client.1.smithi132.stdout:3/422: dread d8/d38/d44/f4c [4194304,4194304] 0 2024-02-19T19:04:55.474 INFO:tasks.workunit.client.1.smithi132.stdout:1/438: mkdir d1/d1e/d77/d2c/d62/d88 0 2024-02-19T19:04:55.476 INFO:tasks.workunit.client.1.smithi132.stdout:3/423: write d8/d13/d2f/f4d [2883186,65231] 0 2024-02-19T19:04:55.476 INFO:tasks.workunit.client.1.smithi132.stdout:2/509: rmdir d0/d24/d58 39 2024-02-19T19:04:55.478 INFO:tasks.workunit.client.1.smithi132.stdout:6/444: creat d0/db/d25/d3a/d5d/d8f/d22/f9c x:0 0 0 2024-02-19T19:04:55.479 INFO:tasks.workunit.client.1.smithi132.stdout:6/445: chown d0/db/d25/d3a/d3d/d69/f5f 299452229 1 2024-02-19T19:04:55.484 INFO:tasks.workunit.client.1.smithi132.stdout:3/424: symlink d8/d13/l82 0 2024-02-19T19:04:55.488 INFO:tasks.workunit.client.1.smithi132.stdout:8/501: rename d4/dc/f4e to d4/dc/d1b/d20/fa8 0 2024-02-19T19:04:55.493 INFO:tasks.workunit.client.1.smithi132.stdout:2/510: mknod d0/d19/d72/d84/ca0 0 2024-02-19T19:04:55.496 INFO:tasks.workunit.client.1.smithi132.stdout:6/446: creat d0/db/f9d x:0 0 0 2024-02-19T19:04:55.505 INFO:tasks.workunit.client.1.smithi132.stdout:5/515: rename d8/d23/d29/d3f to d8/d1a/d35/d77/daa 0 2024-02-19T19:04:55.507 INFO:tasks.workunit.client.1.smithi132.stdout:3/425: fsync d8/d13/d2b/d52/d34/d50/f5b 0 2024-02-19T19:04:55.508 INFO:tasks.workunit.client.1.smithi132.stdout:2/511: creat d0/d19/d72/d84/fa1 x:0 0 0 2024-02-19T19:04:55.510 INFO:tasks.workunit.client.1.smithi132.stdout:6/447: mknod d0/d51/c9e 0 2024-02-19T19:04:55.511 INFO:tasks.workunit.client.1.smithi132.stdout:7/546: write f8 [3634797,72084] 0 2024-02-19T19:04:55.515 INFO:tasks.workunit.client.1.smithi132.stdout:9/585: dwrite d12/f23 [0,4194304] 0 2024-02-19T19:04:55.516 INFO:tasks.workunit.client.1.smithi132.stdout:9/586: write d12/d3e/f93 [729243,32307] 0 2024-02-19T19:04:55.518 INFO:tasks.workunit.client.1.smithi132.stdout:6/448: symlink d0/db/d25/d3a/d4b/d7c/l9f 0 2024-02-19T19:04:55.518 INFO:tasks.workunit.client.1.smithi132.stdout:2/512: chown d0/d24/d58/d53/f91 888867 1 2024-02-19T19:04:55.541 INFO:tasks.workunit.client.1.smithi132.stdout:8/502: creat d4/dc/fa9 x:0 0 0 2024-02-19T19:04:55.542 INFO:tasks.workunit.client.1.smithi132.stdout:6/449: chown d0/db/d25/d3a/d5d/d8f/d22/l72 7 1 2024-02-19T19:04:55.545 INFO:tasks.workunit.client.1.smithi132.stdout:5/516: rename d8/d1a/d35/d77/c8d to d8/d23/cab 0 2024-02-19T19:04:55.546 INFO:tasks.workunit.client.1.smithi132.stdout:5/517: fdatasync d8/d23/d3b/f70 0 2024-02-19T19:04:55.548 INFO:tasks.workunit.client.1.smithi132.stdout:2/513: dwrite d0/d19/f2c [0,4194304] 0 2024-02-19T19:04:55.551 INFO:tasks.workunit.client.1.smithi132.stdout:8/503: write d4/dc/d1b/d20/f53 [178181,55244] 0 2024-02-19T19:04:55.552 INFO:tasks.workunit.client.1.smithi132.stdout:2/514: dread d0/d4/df/f51 [0,4194304] 0 2024-02-19T19:04:55.555 INFO:tasks.workunit.client.1.smithi132.stdout:9/587: write d12/d8d/d3a/d40/d4e/d77/d8a/d82/f8c [121691,22357] 0 2024-02-19T19:04:55.555 INFO:tasks.workunit.client.1.smithi132.stdout:9/588: chown d12/d8d/c34 42053347 1 2024-02-19T19:04:55.559 INFO:tasks.workunit.client.1.smithi132.stdout:3/426: dread d8/f22 [0,4194304] 0 2024-02-19T19:04:55.560 INFO:tasks.workunit.client.1.smithi132.stdout:6/450: link d0/db/d25/c2c d0/db/d25/d3a/d5d/d8f/d22/d79/ca0 0 2024-02-19T19:04:55.563 INFO:tasks.workunit.client.1.smithi132.stdout:9/589: creat d12/d2a/d4c/dad/db2/fce x:0 0 0 2024-02-19T19:04:55.565 INFO:tasks.workunit.client.1.smithi132.stdout:9/590: write d12/d1b/f50 [2373934,107847] 0 2024-02-19T19:04:55.565 INFO:tasks.workunit.client.1.smithi132.stdout:5/518: link d8/d23/d29/d27/f25 d8/d23/fac 0 2024-02-19T19:04:55.568 INFO:tasks.workunit.client.1.smithi132.stdout:5/519: fsync d8/da/db/f98 0 2024-02-19T19:04:55.569 INFO:tasks.workunit.client.1.smithi132.stdout:6/451: dwrite d0/db/d25/d3a/d3d/f61 [0,4194304] 0 2024-02-19T19:04:55.569 INFO:tasks.workunit.client.1.smithi132.stdout:5/520: chown d8/d23 15999525 1 2024-02-19T19:04:55.569 INFO:tasks.workunit.client.1.smithi132.stdout:8/504: creat d4/dc/d1b/d38/d69/d74/d93/faa x:0 0 0 2024-02-19T19:04:55.571 INFO:tasks.workunit.client.1.smithi132.stdout:3/427: rename d8/d13/d2f/f36 to d8/d13/d2b/d60/f83 0 2024-02-19T19:04:55.572 INFO:tasks.workunit.client.1.smithi132.stdout:4/446: sync 2024-02-19T19:04:55.573 INFO:tasks.workunit.client.1.smithi132.stdout:0/469: sync 2024-02-19T19:04:55.573 INFO:tasks.workunit.client.1.smithi132.stdout:1/439: sync 2024-02-19T19:04:55.577 INFO:tasks.workunit.client.1.smithi132.stdout:9/591: dread - d12/d8d/d3a/d40/d4e/d77/d8a/d82/fa9 zero size 2024-02-19T19:04:55.580 INFO:tasks.workunit.client.1.smithi132.stdout:3/428: dwrite d8/d38/d44/f5e [0,4194304] 0 2024-02-19T19:04:55.581 INFO:tasks.workunit.client.1.smithi132.stdout:4/447: dwrite d4/d27/d31/d71/d37/d66/f70 [0,4194304] 0 2024-02-19T19:04:55.581 INFO:tasks.workunit.client.1.smithi132.stdout:3/429: chown d8/d13/d2b/d52/d34 1483911 1 2024-02-19T19:04:55.583 INFO:tasks.workunit.client.1.smithi132.stdout:4/448: write d4/d12/f4b [684435,111744] 0 2024-02-19T19:04:55.587 INFO:tasks.workunit.client.1.smithi132.stdout:8/505: rmdir d4/dc/d1b/d20/d23/d68/d79 39 2024-02-19T19:04:55.594 INFO:tasks.workunit.client.1.smithi132.stdout:1/440: rename d1/d3a/d79 to d1/d1e/d77/d2c/d89 0 2024-02-19T19:04:55.595 INFO:tasks.workunit.client.1.smithi132.stdout:2/515: dwrite d0/d4/df/d40/d68/f8f [0,4194304] 0 2024-02-19T19:04:55.598 INFO:tasks.workunit.client.1.smithi132.stdout:3/430: symlink d8/d13/d2b/d52/l84 0 2024-02-19T19:04:55.600 INFO:tasks.workunit.client.1.smithi132.stdout:5/521: symlink d8/d1a/d35/d77/da9/lad 0 2024-02-19T19:04:55.604 INFO:tasks.workunit.client.1.smithi132.stdout:8/506: dread d4/dc/d1b/d20/d62/d91/f36 [4194304,4194304] 0 2024-02-19T19:04:55.605 INFO:tasks.workunit.client.1.smithi132.stdout:8/507: write d4/dc/d1b/d20/d23/d68/d7a/f94 [42597,20321] 0 2024-02-19T19:04:55.608 INFO:tasks.workunit.client.1.smithi132.stdout:2/516: mknod d0/d19/d72/d8a/ca2 0 2024-02-19T19:04:55.609 INFO:tasks.workunit.client.1.smithi132.stdout:2/517: fdatasync d0/d19/d1e/f9d 0 2024-02-19T19:04:55.614 INFO:tasks.workunit.client.1.smithi132.stdout:4/449: creat d4/d27/f9c x:0 0 0 2024-02-19T19:04:55.614 INFO:tasks.workunit.client.1.smithi132.stdout:6/452: getdents d0/db/d25/d3a 0 2024-02-19T19:04:55.616 INFO:tasks.workunit.client.1.smithi132.stdout:5/522: mkdir d8/da/dae 0 2024-02-19T19:04:55.618 INFO:tasks.workunit.client.1.smithi132.stdout:9/592: getdents d12 0 2024-02-19T19:04:55.620 INFO:tasks.workunit.client.1.smithi132.stdout:9/593: write d12/d3e/f4a [204324,112987] 0 2024-02-19T19:04:55.621 INFO:tasks.workunit.client.1.smithi132.stdout:4/450: dwrite d4/d19/d21/f90 [0,4194304] 0 2024-02-19T19:04:55.621 INFO:tasks.workunit.client.1.smithi132.stdout:1/441: stat d1/d1e/d77/d2c/f34 0 2024-02-19T19:04:55.630 INFO:tasks.workunit.client.1.smithi132.stdout:2/518: dread d0/d3c/f60 [0,4194304] 0 2024-02-19T19:04:55.632 INFO:tasks.workunit.client.1.smithi132.stdout:4/451: symlink d4/d12/d48/l9d 0 2024-02-19T19:04:55.632 INFO:tasks.workunit.client.1.smithi132.stdout:5/523: mknod d8/da/dae/caf 0 2024-02-19T19:04:55.634 INFO:tasks.workunit.client.1.smithi132.stdout:4/452: write d4/d19/f7a [129596,78887] 0 2024-02-19T19:04:55.635 INFO:tasks.workunit.client.1.smithi132.stdout:1/442: symlink d1/d1e/d77/d2c/d43/d81/l8a 0 2024-02-19T19:04:55.644 INFO:tasks.workunit.client.1.smithi132.stdout:4/453: rename d4/d27/d31/d71/cd to d4/d12/d48/c9e 0 2024-02-19T19:04:55.647 INFO:tasks.workunit.client.1.smithi132.stdout:6/453: getdents d0/db/d25/d3a/d5d/d8f/d46/d5a 0 2024-02-19T19:04:55.647 INFO:tasks.workunit.client.1.smithi132.stdout:1/443: symlink d1/d3a/l8b 0 2024-02-19T19:04:55.648 INFO:tasks.workunit.client.1.smithi132.stdout:9/594: dwrite fa [0,4194304] 0 2024-02-19T19:04:55.654 INFO:tasks.workunit.client.1.smithi132.stdout:5/524: creat d8/d23/fb0 x:0 0 0 2024-02-19T19:04:55.654 INFO:tasks.workunit.client.1.smithi132.stdout:2/519: link d0/d3c/d3f/l50 d0/d4/la3 0 2024-02-19T19:04:55.654 INFO:tasks.workunit.client.1.smithi132.stdout:7/547: sync 2024-02-19T19:04:55.655 INFO:tasks.workunit.client.1.smithi132.stdout:1/444: dwrite d1/d61/f7c [0,4194304] 0 2024-02-19T19:04:55.656 INFO:tasks.workunit.client.1.smithi132.stdout:5/525: stat d8/d23/d3b/f70 0 2024-02-19T19:04:55.656 INFO:tasks.workunit.client.1.smithi132.stdout:7/548: read - d1b/d1e/d28/f97 zero size 2024-02-19T19:04:55.657 INFO:tasks.workunit.client.1.smithi132.stdout:5/526: chown d8/d1a/f7a 1143 1 2024-02-19T19:04:55.658 INFO:tasks.workunit.client.1.smithi132.stdout:7/549: readlink d1b/d5a/l73 0 2024-02-19T19:04:55.658 INFO:tasks.workunit.client.1.smithi132.stdout:6/454: mknod d0/ca1 0 2024-02-19T19:04:55.658 INFO:tasks.workunit.client.1.smithi132.stdout:1/445: write d1/d1e/d77/d4e/f6d [834993,40328] 0 2024-02-19T19:04:55.659 INFO:tasks.workunit.client.1.smithi132.stdout:6/455: readlink d0/db/l30 0 2024-02-19T19:04:55.659 INFO:tasks.workunit.client.1.smithi132.stdout:9/595: link d12/d8d/d3a/d40/d4e/d77/d8a/d82/fc6 d12/d8d/d3a/d40/d4e/d77/d89/fcf 0 2024-02-19T19:04:55.661 INFO:tasks.workunit.client.1.smithi132.stdout:9/596: chown d12/d1b/f73 45688766 1 2024-02-19T19:04:55.663 INFO:tasks.workunit.client.1.smithi132.stdout:9/597: chown d12/d1b/c15 139380 1 2024-02-19T19:04:55.668 INFO:tasks.workunit.client.1.smithi132.stdout:7/550: dwrite d1b/d1e/d41/f6d [0,4194304] 0 2024-02-19T19:04:55.673 INFO:tasks.workunit.client.1.smithi132.stdout:1/446: dread d1/d1e/d77/d4e/f4f [0,4194304] 0 2024-02-19T19:04:55.677 INFO:tasks.workunit.client.1.smithi132.stdout:7/551: dwrite d1b/d1e/f38 [0,4194304] 0 2024-02-19T19:04:55.679 INFO:tasks.workunit.client.1.smithi132.stdout:9/598: rename d12/d2a/d4c/d61/c9e to d12/cd0 0 2024-02-19T19:04:55.679 INFO:tasks.workunit.client.1.smithi132.stdout:7/552: truncate d1b/d1e/d2b/f95 4515321 0 2024-02-19T19:04:55.679 INFO:tasks.workunit.client.1.smithi132.stdout:7/553: stat d1b/d1e/d2b/d8f 0 2024-02-19T19:04:55.682 INFO:tasks.workunit.client.1.smithi132.stdout:6/456: truncate d0/f41 992419 0 2024-02-19T19:04:55.689 INFO:tasks.workunit.client.1.smithi132.stdout:6/457: dwrite d0/dc/f66 [0,4194304] 0 2024-02-19T19:04:55.702 INFO:tasks.workunit.client.1.smithi132.stdout:5/527: dread d8/ff [0,4194304] 0 2024-02-19T19:04:55.707 INFO:tasks.workunit.client.1.smithi132.stdout:9/599: symlink d12/d8d/d3a/d40/d4e/ld1 0 2024-02-19T19:04:55.708 INFO:tasks.workunit.client.1.smithi132.stdout:5/528: getdents d8/da/d5a/d84 0 2024-02-19T19:04:55.709 INFO:tasks.workunit.client.1.smithi132.stdout:5/529: write d8/d2e/f55 [128031,111699] 0 2024-02-19T19:04:55.711 INFO:tasks.workunit.client.1.smithi132.stdout:9/600: write d12/d3e/f6c [3228352,110267] 0 2024-02-19T19:04:55.713 INFO:tasks.workunit.client.1.smithi132.stdout:5/530: mknod d8/d2e/d71/cb1 0 2024-02-19T19:04:55.714 INFO:tasks.workunit.client.1.smithi132.stdout:9/601: fdatasync d12/d2a/d4c/d61/fc5 0 2024-02-19T19:04:55.716 INFO:tasks.workunit.client.1.smithi132.stdout:5/531: symlink d8/d23/d3b/lb2 0 2024-02-19T19:04:55.716 INFO:tasks.workunit.client.1.smithi132.stdout:9/602: write d12/d8d/d3a/f52 [3020859,81840] 0 2024-02-19T19:04:55.717 INFO:tasks.workunit.client.1.smithi132.stdout:5/532: write d8/d23/d29/d27/f57 [5061230,46913] 0 2024-02-19T19:04:55.719 INFO:tasks.workunit.client.1.smithi132.stdout:9/603: chown d12/d2a/c55 26352 1 2024-02-19T19:04:55.720 INFO:tasks.workunit.client.1.smithi132.stdout:5/533: read - d8/da/d5a/d7b/f88 zero size 2024-02-19T19:04:55.720 INFO:tasks.workunit.client.1.smithi132.stdout:2/520: dread d0/d4/d11/f15 [0,4194304] 0 2024-02-19T19:04:55.722 INFO:tasks.workunit.client.1.smithi132.stdout:2/521: write d0/d64/d6d/f94 [1752222,26354] 0 2024-02-19T19:04:55.723 INFO:tasks.workunit.client.1.smithi132.stdout:9/604: creat d12/d2a/fd2 x:0 0 0 2024-02-19T19:04:55.726 INFO:tasks.workunit.client.1.smithi132.stdout:9/605: fdatasync d12/d8d/d3a/d40/d4e/d77/d9d/faf 0 2024-02-19T19:04:55.728 INFO:tasks.workunit.client.1.smithi132.stdout:5/534: dwrite d8/da/f8c [0,4194304] 0 2024-02-19T19:04:55.729 INFO:tasks.workunit.client.1.smithi132.stdout:5/535: fsync d8/d23/d3b/fa4 0 2024-02-19T19:04:55.732 INFO:tasks.workunit.client.1.smithi132.stdout:5/536: write d8/da/db/f34 [1954749,85381] 0 2024-02-19T19:04:55.732 INFO:tasks.workunit.client.1.smithi132.stdout:9/606: mknod d12/d8d/d3a/d85/d87/cd3 0 2024-02-19T19:04:55.732 INFO:tasks.workunit.client.1.smithi132.stdout:5/537: chown d8/d23/fb0 929989 1 2024-02-19T19:04:55.734 INFO:tasks.workunit.client.1.smithi132.stdout:9/607: readlink d12/d3e/d6e/l7f 0 2024-02-19T19:04:55.737 INFO:tasks.workunit.client.1.smithi132.stdout:2/522: dread d0/d4/d11/f32 [0,4194304] 0 2024-02-19T19:04:55.738 INFO:tasks.workunit.client.1.smithi132.stdout:5/538: rename d8/da/da2 to d8/d2e/db3 0 2024-02-19T19:04:55.738 INFO:tasks.workunit.client.1.smithi132.stdout:3/431: sync 2024-02-19T19:04:55.738 INFO:tasks.workunit.client.1.smithi132.stdout:0/470: sync 2024-02-19T19:04:55.740 INFO:tasks.workunit.client.1.smithi132.stdout:5/539: write d8/d1a/d35/f4a [1380823,114874] 0 2024-02-19T19:04:55.740 INFO:tasks.workunit.client.1.smithi132.stdout:2/523: chown d0/d4/df/d40/d68/f86 90668 1 2024-02-19T19:04:55.740 INFO:tasks.workunit.client.1.smithi132.stdout:3/432: chown d8/d13/d2b/d52/l3c 10 1 2024-02-19T19:04:55.741 INFO:tasks.workunit.client.1.smithi132.stdout:5/540: stat d8/d1a/f6a 0 2024-02-19T19:04:55.741 INFO:tasks.workunit.client.1.smithi132.stdout:9/608: dread d12/f53 [0,4194304] 0 2024-02-19T19:04:55.742 INFO:tasks.workunit.client.1.smithi132.stdout:2/524: fsync d0/d24/d58/f8c 0 2024-02-19T19:04:55.743 INFO:tasks.workunit.client.1.smithi132.stdout:3/433: dread - d8/d13/d2b/d52/d34/d50/f5b zero size 2024-02-19T19:04:55.744 INFO:tasks.workunit.client.1.smithi132.stdout:5/541: creat d8/d2e/fb4 x:0 0 0 2024-02-19T19:04:55.745 INFO:tasks.workunit.client.1.smithi132.stdout:9/609: creat d12/d3e/fd4 x:0 0 0 2024-02-19T19:04:55.746 INFO:tasks.workunit.client.1.smithi132.stdout:2/525: creat d0/d19/d72/d8a/fa4 x:0 0 0 2024-02-19T19:04:55.747 INFO:tasks.workunit.client.1.smithi132.stdout:3/434: write d8/d38/d44/f4c [1388863,77087] 0 2024-02-19T19:04:55.748 INFO:tasks.workunit.client.1.smithi132.stdout:5/542: creat d8/da/dae/fb5 x:0 0 0 2024-02-19T19:04:55.749 INFO:tasks.workunit.client.1.smithi132.stdout:9/610: symlink d12/d2a/d4c/dad/db2/ld5 0 2024-02-19T19:04:55.752 INFO:tasks.workunit.client.1.smithi132.stdout:2/526: unlink d0/d3c/c48 0 2024-02-19T19:04:55.753 INFO:tasks.workunit.client.1.smithi132.stdout:9/611: creat d12/d8d/d3a/d40/d4e/fd6 x:0 0 0 2024-02-19T19:04:55.754 INFO:tasks.workunit.client.1.smithi132.stdout:5/543: mknod d8/da/cb6 0 2024-02-19T19:04:55.758 INFO:tasks.workunit.client.1.smithi132.stdout:5/544: mknod d8/d1a/cb7 0 2024-02-19T19:04:55.759 INFO:tasks.workunit.client.1.smithi132.stdout:2/527: dwrite d0/d24/d58/f36 [4194304,4194304] 0 2024-02-19T19:04:55.760 INFO:tasks.workunit.client.1.smithi132.stdout:3/435: dread d8/d13/d2f/f4d [0,4194304] 0 2024-02-19T19:04:55.761 INFO:tasks.workunit.client.1.smithi132.stdout:2/528: write d0/d19/d1e/f9d [952414,9165] 0 2024-02-19T19:04:55.766 INFO:tasks.workunit.client.1.smithi132.stdout:2/529: readlink d0/d4/d11/l4c 0 2024-02-19T19:04:55.768 INFO:tasks.workunit.client.1.smithi132.stdout:2/530: stat d0/d4/d11/d2e 0 2024-02-19T19:04:55.770 INFO:tasks.workunit.client.1.smithi132.stdout:3/436: dread d8/d13/f20 [0,4194304] 0 2024-02-19T19:04:55.771 INFO:tasks.workunit.client.1.smithi132.stdout:5/545: dwrite d8/f9b [0,4194304] 0 2024-02-19T19:04:55.776 INFO:tasks.workunit.client.1.smithi132.stdout:9/612: dread d12/d8d/f30 [0,4194304] 0 2024-02-19T19:04:55.777 INFO:tasks.workunit.client.1.smithi132.stdout:3/437: mknod d8/d13/d2b/c85 0 2024-02-19T19:04:55.778 INFO:tasks.workunit.client.1.smithi132.stdout:5/546: mkdir d8/da/d5a/db8 0 2024-02-19T19:04:55.778 INFO:tasks.workunit.client.1.smithi132.stdout:2/531: dwrite d0/d4/f8e [0,4194304] 0 2024-02-19T19:04:55.780 INFO:tasks.workunit.client.1.smithi132.stdout:3/438: write d8/f51 [296354,128083] 0 2024-02-19T19:04:55.781 INFO:tasks.workunit.client.1.smithi132.stdout:9/613: creat d12/d3e/fd7 x:0 0 0 2024-02-19T19:04:55.783 INFO:tasks.workunit.client.1.smithi132.stdout:2/532: mkdir d0/d19/d72/d8a/da5 0 2024-02-19T19:04:55.786 INFO:tasks.workunit.client.1.smithi132.stdout:9/614: rmdir d12/d8d 39 2024-02-19T19:04:55.788 INFO:tasks.workunit.client.1.smithi132.stdout:5/547: creat d8/da/fb9 x:0 0 0 2024-02-19T19:04:55.793 INFO:tasks.workunit.client.1.smithi132.stdout:3/439: getdents d8/d13/d57 0 2024-02-19T19:04:55.794 INFO:tasks.workunit.client.1.smithi132.stdout:9/615: dread - d12/d8d/d3a/d40/d4e/d77/d9d/fcd zero size 2024-02-19T19:04:55.794 INFO:tasks.workunit.client.1.smithi132.stdout:2/533: link d0/d64/d6d/l95 d0/d19/d72/d8a/la6 0 2024-02-19T19:04:55.795 INFO:tasks.workunit.client.1.smithi132.stdout:9/616: dread - d12/d8d/d3a/d40/f44 zero size 2024-02-19T19:04:55.796 INFO:tasks.workunit.client.1.smithi132.stdout:3/440: dread - d8/d13/d2b/d52/d34/f59 zero size 2024-02-19T19:04:55.797 INFO:tasks.workunit.client.1.smithi132.stdout:5/548: getdents d8 0 2024-02-19T19:04:55.799 INFO:tasks.workunit.client.1.smithi132.stdout:5/549: mkdir d8/d23/d29/dba 0 2024-02-19T19:04:55.802 INFO:tasks.workunit.client.1.smithi132.stdout:3/441: dwrite d8/d13/d2b/d52/f55 [0,4194304] 0 2024-02-19T19:04:55.805 INFO:tasks.workunit.client.1.smithi132.stdout:9/617: dwrite d12/d1b/f3f [0,4194304] 0 2024-02-19T19:04:55.810 INFO:tasks.workunit.client.1.smithi132.stdout:9/618: write d12/d2a/d4c/f9b [5003424,34727] 0 2024-02-19T19:04:55.812 INFO:tasks.workunit.client.1.smithi132.stdout:3/442: unlink d8/d38/d44/d6e/c7a 0 2024-02-19T19:04:55.815 INFO:tasks.workunit.client.1.smithi132.stdout:3/443: write d8/d13/d2b/d2e/f7c [613460,29244] 0 2024-02-19T19:04:55.818 INFO:tasks.workunit.client.1.smithi132.stdout:5/550: write d8/d23/f43 [1249401,54843] 0 2024-02-19T19:04:55.819 INFO:tasks.workunit.client.1.smithi132.stdout:5/551: stat d8/da/dae/fb5 0 2024-02-19T19:04:55.819 INFO:tasks.workunit.client.1.smithi132.stdout:3/444: mknod d8/d13/d2b/d2e/c86 0 2024-02-19T19:04:55.820 INFO:tasks.workunit.client.1.smithi132.stdout:2/534: dread d0/d19/f82 [0,4194304] 0 2024-02-19T19:04:55.821 INFO:tasks.workunit.client.1.smithi132.stdout:5/552: creat d8/da/db/fbb x:0 0 0 2024-02-19T19:04:55.821 INFO:tasks.workunit.client.1.smithi132.stdout:2/535: fsync d0/d19/f2c 0 2024-02-19T19:04:55.824 INFO:tasks.workunit.client.1.smithi132.stdout:5/553: mkdir d8/d2e/d71/dbc 0 2024-02-19T19:04:55.826 INFO:tasks.workunit.client.1.smithi132.stdout:4/454: sync 2024-02-19T19:04:55.826 INFO:tasks.workunit.client.1.smithi132.stdout:8/508: sync 2024-02-19T19:04:55.827 INFO:tasks.workunit.client.1.smithi132.stdout:2/536: mknod d0/d24/ca7 0 2024-02-19T19:04:55.827 INFO:tasks.workunit.client.1.smithi132.stdout:5/554: symlink d8/lbd 0 2024-02-19T19:04:55.828 INFO:tasks.workunit.client.1.smithi132.stdout:4/455: dread - d4/d27/d31/d32/d51/d5c/f5d zero size 2024-02-19T19:04:55.829 INFO:tasks.workunit.client.1.smithi132.stdout:2/537: chown d0/c2 59778 1 2024-02-19T19:04:55.830 INFO:tasks.workunit.client.1.smithi132.stdout:2/538: truncate d0/d19/f78 5048704 0 2024-02-19T19:04:55.834 INFO:tasks.workunit.client.1.smithi132.stdout:4/456: dwrite d4/f2d [0,4194304] 0 2024-02-19T19:04:55.842 INFO:tasks.workunit.client.1.smithi132.stdout:3/445: dread d8/d13/d2b/d2e/f3b [0,4194304] 0 2024-02-19T19:04:55.846 INFO:tasks.workunit.client.1.smithi132.stdout:4/457: unlink d4/d27/d31/d71/d14/d24/l42 0 2024-02-19T19:04:55.848 INFO:tasks.workunit.client.1.smithi132.stdout:8/509: truncate d4/dc/d1b/d33/f49 2209142 0 2024-02-19T19:04:55.849 INFO:tasks.workunit.client.1.smithi132.stdout:8/510: write d4/fb [2319897,3537] 0 2024-02-19T19:04:55.850 INFO:tasks.workunit.client.1.smithi132.stdout:5/555: chown d8/d1a/d35/d77/da9/lad 244236722 1 2024-02-19T19:04:55.850 INFO:tasks.workunit.client.1.smithi132.stdout:3/446: mknod d8/d38/d44/c87 0 2024-02-19T19:04:55.852 INFO:tasks.workunit.client.1.smithi132.stdout:2/539: rename d0/d4/d11/d2e/d5a/d66/c6b to d0/d4/d11/d2e/ca8 0 2024-02-19T19:04:55.853 INFO:tasks.workunit.client.1.smithi132.stdout:5/556: readlink d8/da/d5a/l95 0 2024-02-19T19:04:55.856 INFO:tasks.workunit.client.1.smithi132.stdout:4/458: mkdir d4/d12/d9f 0 2024-02-19T19:04:55.860 INFO:tasks.workunit.client.1.smithi132.stdout:5/557: rmdir d8/d23 39 2024-02-19T19:04:55.860 INFO:tasks.workunit.client.1.smithi132.stdout:4/459: dwrite d4/d12/f89 [0,4194304] 0 2024-02-19T19:04:55.862 INFO:tasks.workunit.client.1.smithi132.stdout:2/540: creat d0/d4/fa9 x:0 0 0 2024-02-19T19:04:55.871 INFO:tasks.workunit.client.1.smithi132.stdout:3/447: getdents d8 0 2024-02-19T19:04:55.875 INFO:tasks.workunit.client.1.smithi132.stdout:3/448: dwrite d8/d38/f4b [0,4194304] 0 2024-02-19T19:04:55.877 INFO:tasks.workunit.client.1.smithi132.stdout:8/511: rename d4/dc/d1b/d20/f6b to d4/fab 0 2024-02-19T19:04:55.882 INFO:tasks.workunit.client.1.smithi132.stdout:4/460: mknod d4/d12/d48/ca0 0 2024-02-19T19:04:55.882 INFO:tasks.workunit.client.1.smithi132.stdout:5/558: dread d8/d1a/d64/f72 [0,4194304] 0 2024-02-19T19:04:55.884 INFO:tasks.workunit.client.1.smithi132.stdout:3/449: symlink d8/d13/d2b/d52/d34/d42/d6c/l88 0 2024-02-19T19:04:55.885 INFO:tasks.workunit.client.1.smithi132.stdout:8/512: mkdir d4/dac 0 2024-02-19T19:04:55.886 INFO:tasks.workunit.client.1.smithi132.stdout:8/513: chown d4/fd 5336976 1 2024-02-19T19:04:55.887 INFO:tasks.workunit.client.1.smithi132.stdout:2/541: link d0/d4/d11/f32 d0/d24/faa 0 2024-02-19T19:04:55.891 INFO:tasks.workunit.client.1.smithi132.stdout:4/461: symlink d4/d27/d31/d71/d37/d66/d98/la1 0 2024-02-19T19:04:55.891 INFO:tasks.workunit.client.1.smithi132.stdout:7/554: sync 2024-02-19T19:04:55.891 INFO:tasks.workunit.client.1.smithi132.stdout:1/447: sync 2024-02-19T19:04:55.891 INFO:tasks.workunit.client.1.smithi132.stdout:0/471: sync 2024-02-19T19:04:55.891 INFO:tasks.workunit.client.1.smithi132.stdout:6/458: sync 2024-02-19T19:04:55.903 INFO:tasks.workunit.client.1.smithi132.stdout:2/542: rename d0/d19/d72 to d0/d24/d58/d53/dab 0 2024-02-19T19:04:55.905 INFO:tasks.workunit.client.1.smithi132.stdout:0/472: read d2/dc/d4a/d74/f86 [136005,91600] 0 2024-02-19T19:04:55.905 INFO:tasks.workunit.client.1.smithi132.stdout:5/559: dwrite d8/d23/d3b/f48 [0,4194304] 0 2024-02-19T19:04:55.908 INFO:tasks.workunit.client.1.smithi132.stdout:7/555: truncate d1b/d1e/d28/f55 5188951 0 2024-02-19T19:04:55.908 INFO:tasks.workunit.client.1.smithi132.stdout:3/450: creat d8/d13/d2b/d52/d34/f89 x:0 0 0 2024-02-19T19:04:55.908 INFO:tasks.workunit.client.1.smithi132.stdout:2/543: truncate d0/d19/d1e/f52 4461682 0 2024-02-19T19:04:55.908 INFO:tasks.workunit.client.1.smithi132.stdout:4/462: fsync d4/f38 0 2024-02-19T19:04:55.911 INFO:tasks.workunit.client.1.smithi132.stdout:5/560: dread - d8/d2e/f91 zero size 2024-02-19T19:04:55.912 INFO:tasks.workunit.client.1.smithi132.stdout:1/448: creat d1/f8c x:0 0 0 2024-02-19T19:04:55.913 INFO:tasks.workunit.client.1.smithi132.stdout:1/449: dread - d1/d3a/f30 zero size 2024-02-19T19:04:55.914 INFO:tasks.workunit.client.1.smithi132.stdout:0/473: dwrite d2/dc/d4a/f57 [0,4194304] 0 2024-02-19T19:04:55.917 INFO:tasks.workunit.client.1.smithi132.stdout:7/556: unlink d1b/d1e/d28/f29 0 2024-02-19T19:04:55.919 INFO:tasks.workunit.client.1.smithi132.stdout:5/561: creat d8/d2e/fbe x:0 0 0 2024-02-19T19:04:55.921 INFO:tasks.workunit.client.1.smithi132.stdout:5/562: write d8/d23/f43 [2244789,106129] 0 2024-02-19T19:04:55.922 INFO:tasks.workunit.client.1.smithi132.stdout:5/563: write d8/d2e/d71/f82 [4954097,115450] 0 2024-02-19T19:04:55.924 INFO:tasks.workunit.client.1.smithi132.stdout:0/474: write d2/dc/df/d15/f4e [1712910,67320] 0 2024-02-19T19:04:55.928 INFO:tasks.workunit.client.1.smithi132.stdout:2/544: creat d0/d4/d11/d2e/fac x:0 0 0 2024-02-19T19:04:55.929 INFO:tasks.workunit.client.1.smithi132.stdout:2/545: write d0/d24/f87 [1292596,99452] 0 2024-02-19T19:04:55.929 INFO:tasks.workunit.client.1.smithi132.stdout:3/451: link d8/l27 d8/d38/l8a 0 2024-02-19T19:04:55.931 INFO:tasks.workunit.client.1.smithi132.stdout:4/463: link d4/d27/d31/l53 d4/d12/d9f/la2 0 2024-02-19T19:04:55.935 INFO:tasks.workunit.client.1.smithi132.stdout:3/452: chown d8/lc 7333574 1 2024-02-19T19:04:55.936 INFO:tasks.workunit.client.1.smithi132.stdout:2/546: fdatasync d0/d4/df/f51 0 2024-02-19T19:04:55.939 INFO:tasks.workunit.client.1.smithi132.stdout:4/464: unlink d4/f9 0 2024-02-19T19:04:55.940 INFO:tasks.workunit.client.1.smithi132.stdout:4/465: read - d4/d27/d31/d32/d51/f93 zero size 2024-02-19T19:04:55.940 INFO:tasks.workunit.client.1.smithi132.stdout:1/450: link d1/d3a/f40 d1/d1e/d77/d2c/d89/f8d 0 2024-02-19T19:04:55.942 INFO:tasks.workunit.client.1.smithi132.stdout:1/451: read d1/d3a/d50/f6f [671839,112409] 0 2024-02-19T19:04:55.943 INFO:tasks.workunit.client.1.smithi132.stdout:0/475: creat d2/dc/df/d15/f99 x:0 0 0 2024-02-19T19:04:55.945 INFO:tasks.workunit.client.1.smithi132.stdout:0/476: write d2/dc/df/d15/d34/f3c [3728176,78275] 0 2024-02-19T19:04:55.946 INFO:tasks.workunit.client.1.smithi132.stdout:6/459: truncate d0/db/d25/d3a/d5d/d8f/d46/d5a/f18 1916829 0 2024-02-19T19:04:55.948 INFO:tasks.workunit.client.1.smithi132.stdout:1/452: dwrite d1/f48 [0,4194304] 0 2024-02-19T19:04:55.952 INFO:tasks.workunit.client.1.smithi132.stdout:2/547: rename d0/d64/d6d to d0/d4/d11/d2e/d5a/d66/dad 0 2024-02-19T19:04:55.956 INFO:tasks.workunit.client.1.smithi132.stdout:2/548: dread - d0/d24/d58/f98 zero size 2024-02-19T19:04:55.958 INFO:tasks.workunit.client.1.smithi132.stdout:7/557: dwrite d1b/d1e/d28/f31 [4194304,4194304] 0 2024-02-19T19:04:55.959 INFO:tasks.workunit.client.1.smithi132.stdout:2/549: fsync d0/d4/d11/d2e/d5a/d66/dad/f94 0 2024-02-19T19:04:55.960 INFO:tasks.workunit.client.1.smithi132.stdout:5/564: truncate d8/d23/f6f 1242575 0 2024-02-19T19:04:55.960 INFO:tasks.workunit.client.1.smithi132.stdout:1/453: dwrite d1/d1e/d77/d2c/d43/f5d [0,4194304] 0 2024-02-19T19:04:55.963 INFO:tasks.workunit.client.1.smithi132.stdout:6/460: rename d0/db/d25/d3a/d5d/d8f/d22/d79 to d0/db/d25/d3a/d4b/d7c/da2 0 2024-02-19T19:04:55.965 INFO:tasks.workunit.client.1.smithi132.stdout:9/619: sync 2024-02-19T19:04:55.966 INFO:tasks.workunit.client.1.smithi132.stdout:9/620: stat d12/d8d/d3a/d85/d87 0 2024-02-19T19:04:55.966 INFO:tasks.workunit.client.1.smithi132.stdout:4/466: symlink d4/d12/la3 0 2024-02-19T19:04:55.968 INFO:tasks.workunit.client.1.smithi132.stdout:7/558: dread d1b/d1e/d2b/d62/f67 [0,4194304] 0 2024-02-19T19:04:55.969 INFO:tasks.workunit.client.1.smithi132.stdout:4/467: truncate d4/d27/d31/d32/f50 637722 0 2024-02-19T19:04:55.970 INFO:tasks.workunit.client.1.smithi132.stdout:3/453: write d8/d13/f20 [2077727,68005] 0 2024-02-19T19:04:55.980 INFO:tasks.workunit.client.1.smithi132.stdout:0/477: symlink d2/dc/d98/l9a 0 2024-02-19T19:04:55.984 INFO:tasks.workunit.client.1.smithi132.stdout:2/550: creat d0/d4/df/d40/fae x:0 0 0 2024-02-19T19:04:55.987 INFO:tasks.workunit.client.1.smithi132.stdout:7/559: fdatasync d1b/d1e/fb2 0 2024-02-19T19:04:55.988 INFO:tasks.workunit.client.1.smithi132.stdout:3/454: fsync d8/d13/d2f/f47 0 2024-02-19T19:04:55.990 INFO:tasks.workunit.client.1.smithi132.stdout:7/560: write d1b/d1e/f37 [3058964,123520] 0 2024-02-19T19:04:55.992 INFO:tasks.workunit.client.1.smithi132.stdout:2/551: dwrite d0/d19/d1e/f45 [0,4194304] 0 2024-02-19T19:04:55.995 INFO:tasks.workunit.client.1.smithi132.stdout:2/552: write d0/d4/d11/f20 [646436,62177] 0 2024-02-19T19:04:55.998 INFO:tasks.workunit.client.1.smithi132.stdout:1/454: dread d1/f4a [0,4194304] 0 2024-02-19T19:04:55.998 INFO:tasks.workunit.client.1.smithi132.stdout:8/514: sync 2024-02-19T19:04:56.002 INFO:tasks.workunit.client.1.smithi132.stdout:9/621: write d12/d8d/f76 [4445530,23766] 0 2024-02-19T19:04:56.003 INFO:tasks.workunit.client.1.smithi132.stdout:4/468: mkdir d4/d12/da4 0 2024-02-19T19:04:56.006 INFO:tasks.workunit.client.1.smithi132.stdout:3/455: write d8/d13/d2b/d52/d34/f59 [455942,31015] 0 2024-02-19T19:04:56.007 INFO:tasks.workunit.client.1.smithi132.stdout:4/469: read d4/d19/f3f [3143507,8254] 0 2024-02-19T19:04:56.009 INFO:tasks.workunit.client.1.smithi132.stdout:7/561: rename d1b/d1e/l24 to d1b/d1e/d2b/d62/lb3 0 2024-02-19T19:04:56.009 INFO:tasks.workunit.client.1.smithi132.stdout:0/478: mknod d2/dc/d4a/d67/c9b 0 2024-02-19T19:04:56.009 INFO:tasks.workunit.client.1.smithi132.stdout:5/565: mkdir d8/da/d5a/d84/dbf 0 2024-02-19T19:04:56.011 INFO:tasks.workunit.client.1.smithi132.stdout:1/455: chown d1/d3a/l26 31970 1 2024-02-19T19:04:56.011 INFO:tasks.workunit.client.1.smithi132.stdout:0/479: chown d2/dc/df/d27/c28 23615 1 2024-02-19T19:04:56.012 INFO:tasks.workunit.client.1.smithi132.stdout:8/515: creat d4/dc/d1b/fad x:0 0 0 2024-02-19T19:04:56.013 INFO:tasks.workunit.client.1.smithi132.stdout:0/480: readlink d2/dc/l55 0 2024-02-19T19:04:56.015 INFO:tasks.workunit.client.1.smithi132.stdout:4/470: dwrite d4/d12/f89 [4194304,4194304] 0 2024-02-19T19:04:56.018 INFO:tasks.workunit.client.1.smithi132.stdout:3/456: creat d8/d38/f8b x:0 0 0 2024-02-19T19:04:56.025 INFO:tasks.workunit.client.1.smithi132.stdout:6/461: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f18 [221909,100042] 0 2024-02-19T19:04:56.028 INFO:tasks.workunit.client.1.smithi132.stdout:4/471: mknod d4/d19/d21/d56/ca5 0 2024-02-19T19:04:56.029 INFO:tasks.workunit.client.1.smithi132.stdout:4/472: read d4/d19/d21/f90 [3288036,59875] 0 2024-02-19T19:04:56.035 INFO:tasks.workunit.client.1.smithi132.stdout:3/457: link d8/d13/f25 d8/d13/d2b/d52/f8c 0 2024-02-19T19:04:56.035 INFO:tasks.workunit.client.1.smithi132.stdout:3/458: stat d8/f51 0 2024-02-19T19:04:56.036 INFO:tasks.workunit.client.1.smithi132.stdout:3/459: write d8/d13/d2b/d52/d34/f81 [116092,7882] 0 2024-02-19T19:04:56.038 INFO:tasks.workunit.client.1.smithi132.stdout:2/553: rename d0/d4/d11/d2e/d5a/d66/dad/d76 to d0/d24/daf 0 2024-02-19T19:04:56.039 INFO:tasks.workunit.client.1.smithi132.stdout:1/456: symlink d1/d1e/d77/d2c/d62/d6b/d85/l8e 0 2024-02-19T19:04:56.041 INFO:tasks.workunit.client.1.smithi132.stdout:4/473: chown d4/d27/d31/d32/d51/d5c/l75 763 1 2024-02-19T19:04:56.041 INFO:tasks.workunit.client.1.smithi132.stdout:7/562: write d1b/d1e/f22 [4355352,1091] 0 2024-02-19T19:04:56.042 INFO:tasks.workunit.client.1.smithi132.stdout:9/622: link d12/d8d/c38 d12/d2a/d88/cd8 0 2024-02-19T19:04:56.044 INFO:tasks.workunit.client.1.smithi132.stdout:7/563: truncate d1b/d1e/d2b/d62/d53/fa4 483781 0 2024-02-19T19:04:56.045 INFO:tasks.workunit.client.1.smithi132.stdout:1/457: dwrite d1/d1e/f2a [0,4194304] 0 2024-02-19T19:04:56.045 INFO:tasks.workunit.client.1.smithi132.stdout:6/462: link d0/db/d25/d3a/d3d/d69/f7f d0/d8b/fa3 0 2024-02-19T19:04:56.051 INFO:tasks.workunit.client.1.smithi132.stdout:0/481: truncate d2/dc/df/d15/f4e 1189976 0 2024-02-19T19:04:56.051 INFO:tasks.workunit.client.1.smithi132.stdout:3/460: rmdir d8/d13/d2b/d52 39 2024-02-19T19:04:56.051 INFO:tasks.workunit.client.1.smithi132.stdout:5/566: rename d8/d23/l76 to d8/d1a/lc0 0 2024-02-19T19:04:56.052 INFO:tasks.workunit.client.1.smithi132.stdout:7/564: read d1b/d46/d76/d94/f9d [646273,78398] 0 2024-02-19T19:04:56.052 INFO:tasks.workunit.client.1.smithi132.stdout:4/474: chown d4/d12/d28 17203924 1 2024-02-19T19:04:56.052 INFO:tasks.workunit.client.1.smithi132.stdout:6/463: write d0/db/fd [4342050,112675] 0 2024-02-19T19:04:56.054 INFO:tasks.workunit.client.1.smithi132.stdout:0/482: write d2/dc/df/d15/f24 [5182864,13758] 0 2024-02-19T19:04:56.055 INFO:tasks.workunit.client.1.smithi132.stdout:3/461: read d8/d13/f20 [2134937,21375] 0 2024-02-19T19:04:56.056 INFO:tasks.workunit.client.1.smithi132.stdout:7/565: write d1b/d1e/f43 [4913506,83008] 0 2024-02-19T19:04:56.058 INFO:tasks.workunit.client.1.smithi132.stdout:8/516: rename d4/dc/d1b/d54/d5e/f51 to d4/dac/fae 0 2024-02-19T19:04:56.059 INFO:tasks.workunit.client.1.smithi132.stdout:6/464: read d0/db/d25/d3a/d5d/d8f/d46/d5a/f37 [358542,124086] 0 2024-02-19T19:04:56.059 INFO:tasks.workunit.client.1.smithi132.stdout:1/458: symlink d1/d3a/l8f 0 2024-02-19T19:04:56.059 INFO:tasks.workunit.client.1.smithi132.stdout:3/462: write d8/d13/d2b/d2e/f6f [481915,27397] 0 2024-02-19T19:04:56.060 INFO:tasks.workunit.client.1.smithi132.stdout:5/567: mknod d8/d23/d3b/cc1 0 2024-02-19T19:04:56.060 INFO:tasks.workunit.client.1.smithi132.stdout:7/566: truncate d1b/d46/d56/f57 4821163 0 2024-02-19T19:04:56.061 INFO:tasks.workunit.client.1.smithi132.stdout:7/567: fsync d1b/d1e/f21 0 2024-02-19T19:04:56.062 INFO:tasks.workunit.client.1.smithi132.stdout:4/475: fdatasync d4/f59 0 2024-02-19T19:04:56.063 INFO:tasks.workunit.client.1.smithi132.stdout:6/465: readlink d0/db/d25/d3a/d4b/d7c/l9f 0 2024-02-19T19:04:56.063 INFO:tasks.workunit.client.1.smithi132.stdout:5/568: write d8/da/db/f44 [702648,60890] 0 2024-02-19T19:04:56.069 INFO:tasks.workunit.client.1.smithi132.stdout:8/517: mkdir d4/dc/d1b/d8f/daf 0 2024-02-19T19:04:56.071 INFO:tasks.workunit.client.1.smithi132.stdout:1/459: dwrite d1/d1e/d77/d2c/d43/f5d [0,4194304] 0 2024-02-19T19:04:56.073 INFO:tasks.workunit.client.1.smithi132.stdout:7/568: dwrite d1b/d46/f4d [4194304,4194304] 0 2024-02-19T19:04:56.074 INFO:tasks.workunit.client.1.smithi132.stdout:8/518: write d4/dc/d1b/d20/f8b [902922,75401] 0 2024-02-19T19:04:56.081 INFO:tasks.workunit.client.1.smithi132.stdout:3/463: read d8/d13/d2b/d52/d34/f59 [270976,23007] 0 2024-02-19T19:04:56.083 INFO:tasks.workunit.client.1.smithi132.stdout:3/464: fsync d8/d13/d2b/d52/f55 0 2024-02-19T19:04:56.086 INFO:tasks.workunit.client.1.smithi132.stdout:5/569: dread d8/da/f33 [0,4194304] 0 2024-02-19T19:04:56.086 INFO:tasks.workunit.client.1.smithi132.stdout:5/570: truncate d8/d2e/fbe 759904 0 2024-02-19T19:04:56.088 INFO:tasks.workunit.client.1.smithi132.stdout:1/460: mknod d1/d1e/d77/d52/d64/c90 0 2024-02-19T19:04:56.093 INFO:tasks.workunit.client.1.smithi132.stdout:3/465: mkdir d8/d13/d2b/d52/d8d 0 2024-02-19T19:04:56.093 INFO:tasks.workunit.client.1.smithi132.stdout:0/483: link d2/dc/df/d27/d3a/c5d d2/d6e/d7a/c9c 0 2024-02-19T19:04:56.099 INFO:tasks.workunit.client.1.smithi132.stdout:8/519: symlink d4/lb0 0 2024-02-19T19:04:56.101 INFO:tasks.workunit.client.1.smithi132.stdout:1/461: creat d1/d1e/d77/d52/d64/f91 x:0 0 0 2024-02-19T19:04:56.101 INFO:tasks.workunit.client.1.smithi132.stdout:5/571: dwrite d8/d1a/d35/f40 [0,4194304] 0 2024-02-19T19:04:56.103 INFO:tasks.workunit.client.1.smithi132.stdout:1/462: write d1/d1e/d77/d4e/f6d [885459,62176] 0 2024-02-19T19:04:56.104 INFO:tasks.workunit.client.1.smithi132.stdout:6/466: creat d0/db/d25/d3a/d3d/fa4 x:0 0 0 2024-02-19T19:04:56.105 INFO:tasks.workunit.client.1.smithi132.stdout:4/476: link d4/d19/d21/f36 d4/d27/d73/fa6 0 2024-02-19T19:04:56.106 INFO:tasks.workunit.client.1.smithi132.stdout:6/467: write d0/db/d25/d3a/d5d/d8f/d22/f64 [3193723,27005] 0 2024-02-19T19:04:56.107 INFO:tasks.workunit.client.1.smithi132.stdout:4/477: write d4/d12/d48/f7c [426131,107039] 0 2024-02-19T19:04:56.109 INFO:tasks.workunit.client.1.smithi132.stdout:6/468: write d0/db/d25/d3a/d3d/d69/f55 [835389,33166] 0 2024-02-19T19:04:56.111 INFO:tasks.workunit.client.1.smithi132.stdout:6/469: write d0/db/d25/d3a/d3d/fa4 [432343,100461] 0 2024-02-19T19:04:56.114 INFO:tasks.workunit.client.1.smithi132.stdout:6/470: truncate d0/db/d25/d3a/d5d/d8f/d22/f64 4752930 0 2024-02-19T19:04:56.115 INFO:tasks.workunit.client.1.smithi132.stdout:7/569: dread d1b/f1c [0,4194304] 0 2024-02-19T19:04:56.116 INFO:tasks.workunit.client.1.smithi132.stdout:3/466: link d8/d13/d2f/f66 d8/d38/d44/d6e/f8e 0 2024-02-19T19:04:56.117 INFO:tasks.workunit.client.1.smithi132.stdout:1/463: truncate d1/f53 1846219 0 2024-02-19T19:04:56.119 INFO:tasks.workunit.client.1.smithi132.stdout:0/484: rmdir d2/dc/d4a/d25/d43/d92 0 2024-02-19T19:04:56.121 INFO:tasks.workunit.client.1.smithi132.stdout:3/467: dread d8/d38/d44/f5e [0,4194304] 0 2024-02-19T19:04:56.122 INFO:tasks.workunit.client.1.smithi132.stdout:7/570: truncate d1b/d1e/d41/f6a 4043070 0 2024-02-19T19:04:56.124 INFO:tasks.workunit.client.1.smithi132.stdout:2/554: sync 2024-02-19T19:04:56.124 INFO:tasks.workunit.client.1.smithi132.stdout:1/464: symlink d1/d1e/d77/d2c/d43/l92 0 2024-02-19T19:04:56.126 INFO:tasks.workunit.client.1.smithi132.stdout:2/555: fsync d0/d24/f87 0 2024-02-19T19:04:56.129 INFO:tasks.workunit.client.1.smithi132.stdout:0/485: dwrite d2/dc/df/d27/f56 [0,4194304] 0 2024-02-19T19:04:56.130 INFO:tasks.workunit.client.1.smithi132.stdout:8/520: link d4/la d4/dc/d1b/d20/d23/lb1 0 2024-02-19T19:04:56.133 INFO:tasks.workunit.client.1.smithi132.stdout:7/571: dwrite d1b/d1e/d28/f35 [4194304,4194304] 0 2024-02-19T19:04:56.137 INFO:tasks.workunit.client.1.smithi132.stdout:6/471: creat d0/db/d25/d3a/d5d/d8f/d46/fa5 x:0 0 0 2024-02-19T19:04:56.140 INFO:tasks.workunit.client.1.smithi132.stdout:1/465: mknod d1/d1e/d77/d2c/d62/c93 0 2024-02-19T19:04:56.142 INFO:tasks.workunit.client.1.smithi132.stdout:5/572: dread d8/da/f24 [0,4194304] 0 2024-02-19T19:04:56.144 INFO:tasks.workunit.client.1.smithi132.stdout:0/486: creat d2/dc/df/d27/d3a/d52/f9d x:0 0 0 2024-02-19T19:04:56.147 INFO:tasks.workunit.client.1.smithi132.stdout:8/521: creat d4/dc/d1b/d20/d62/d91/fb2 x:0 0 0 2024-02-19T19:04:56.149 INFO:tasks.workunit.client.1.smithi132.stdout:4/478: dwrite d4/d27/d31/f79 [0,4194304] 0 2024-02-19T19:04:56.149 INFO:tasks.workunit.client.1.smithi132.stdout:8/522: fdatasync d4/dc/d1b/f32 0 2024-02-19T19:04:56.154 INFO:tasks.workunit.client.1.smithi132.stdout:8/523: dread d4/f83 [0,4194304] 0 2024-02-19T19:04:56.157 INFO:tasks.workunit.client.1.smithi132.stdout:6/472: creat d0/db/fa6 x:0 0 0 2024-02-19T19:04:56.157 INFO:tasks.workunit.client.1.smithi132.stdout:7/572: getdents d1b/d46/d56 0 2024-02-19T19:04:56.158 INFO:tasks.workunit.client.1.smithi132.stdout:7/573: write d1b/d1e/d28/f3b [901269,44562] 0 2024-02-19T19:04:56.160 INFO:tasks.workunit.client.1.smithi132.stdout:4/479: creat d4/d4d/fa7 x:0 0 0 2024-02-19T19:04:56.165 INFO:tasks.workunit.client.1.smithi132.stdout:8/524: creat d4/dc/d1b/d54/d81/fb3 x:0 0 0 2024-02-19T19:04:56.166 INFO:tasks.workunit.client.1.smithi132.stdout:7/574: getdents d1b/d46/d56 0 2024-02-19T19:04:56.167 INFO:tasks.workunit.client.1.smithi132.stdout:1/466: dread d1/d1e/d77/d52/f65 [0,4194304] 0 2024-02-19T19:04:56.168 INFO:tasks.workunit.client.1.smithi132.stdout:4/480: link d4/d4d/f64 d4/d27/d31/d32/d51/d5c/fa8 0 2024-02-19T19:04:56.173 INFO:tasks.workunit.client.1.smithi132.stdout:7/575: creat d1b/d1e/d2b/fb4 x:0 0 0 2024-02-19T19:04:56.175 INFO:tasks.workunit.client.1.smithi132.stdout:7/576: read d1b/d5a/d63/d6b/d79/d84/f92 [1355175,80622] 0 2024-02-19T19:04:56.175 INFO:tasks.workunit.client.1.smithi132.stdout:1/467: unlink d1/f7b 0 2024-02-19T19:04:56.178 INFO:tasks.workunit.client.1.smithi132.stdout:8/525: mkdir d4/dc/d1b/d20/d23/d68/d79/db4 0 2024-02-19T19:04:56.182 INFO:tasks.workunit.client.1.smithi132.stdout:7/577: mkdir d1b/d1e/d41/db5 0 2024-02-19T19:04:56.183 INFO:tasks.workunit.client.1.smithi132.stdout:3/468: truncate d8/d13/d19/f2d 3391726 0 2024-02-19T19:04:56.184 INFO:tasks.workunit.client.1.smithi132.stdout:2/556: truncate d0/d19/f25 4107439 0 2024-02-19T19:04:56.185 INFO:tasks.workunit.client.1.smithi132.stdout:4/481: creat d4/d27/d7f/fa9 x:0 0 0 2024-02-19T19:04:56.186 INFO:tasks.workunit.client.1.smithi132.stdout:0/487: write d2/f6 [868046,94947] 0 2024-02-19T19:04:56.188 INFO:tasks.workunit.client.1.smithi132.stdout:6/473: write d0/db/d25/d38/f78 [1409981,27648] 0 2024-02-19T19:04:56.192 INFO:tasks.workunit.client.1.smithi132.stdout:1/468: creat d1/d1e/d77/d2c/d62/d6b/d85/f94 x:0 0 0 2024-02-19T19:04:56.194 INFO:tasks.workunit.client.1.smithi132.stdout:8/526: creat d4/dc/d1b/d38/d69/d74/fb5 x:0 0 0 2024-02-19T19:04:56.196 INFO:tasks.workunit.client.1.smithi132.stdout:7/578: creat d1b/d5a/d63/d6b/d86/fb6 x:0 0 0 2024-02-19T19:04:56.197 INFO:tasks.workunit.client.1.smithi132.stdout:1/469: dwrite d1/d1e/d77/d2c/d62/f87 [0,4194304] 0 2024-02-19T19:04:56.198 INFO:tasks.workunit.client.1.smithi132.stdout:7/579: write d1b/d5a/d63/d96/fad [284909,24443] 0 2024-02-19T19:04:56.199 INFO:tasks.workunit.client.1.smithi132.stdout:7/580: read - d1b/d1e/d2b/fb4 zero size 2024-02-19T19:04:56.200 INFO:tasks.workunit.client.1.smithi132.stdout:2/557: mknod d0/d4/df/d40/d68/cb0 0 2024-02-19T19:04:56.202 INFO:tasks.workunit.client.1.smithi132.stdout:0/488: symlink d2/dc/df/d27/d3a/l9e 0 2024-02-19T19:04:56.202 INFO:tasks.workunit.client.1.smithi132.stdout:6/474: rmdir d0/db/d25/d50/d86 39 2024-02-19T19:04:56.207 INFO:tasks.workunit.client.1.smithi132.stdout:6/475: mkdir d0/d51/da7 0 2024-02-19T19:04:56.208 INFO:tasks.workunit.client.1.smithi132.stdout:0/489: fsync d2/dc/df/d15/d2f/d3f/f8d 0 2024-02-19T19:04:56.210 INFO:tasks.workunit.client.1.smithi132.stdout:4/482: rename d4/d27/d31/d71/d14/d24/d49/d7e to d4/d19/daa 0 2024-02-19T19:04:56.211 INFO:tasks.workunit.client.1.smithi132.stdout:1/470: dwrite d1/f48 [4194304,4194304] 0 2024-02-19T19:04:56.216 INFO:tasks.workunit.client.1.smithi132.stdout:0/490: fdatasync d2/dc/df/d15/d2f/d3f/f79 0 2024-02-19T19:04:56.218 INFO:tasks.workunit.client.1.smithi132.stdout:7/581: dwrite d1b/d1e/f38 [0,4194304] 0 2024-02-19T19:04:56.222 INFO:tasks.workunit.client.1.smithi132.stdout:2/558: dwrite d0/f80 [0,4194304] 0 2024-02-19T19:04:56.222 INFO:tasks.workunit.client.1.smithi132.stdout:8/527: dwrite d4/f83 [0,4194304] 0 2024-02-19T19:04:56.224 INFO:tasks.workunit.client.1.smithi132.stdout:3/469: rename d8/lb to d8/d13/d2b/d52/d34/d50/l8f 0 2024-02-19T19:04:56.232 INFO:tasks.workunit.client.1.smithi132.stdout:6/476: unlink d0/db/d25/d3a/d4b/d7c/da2/c8c 0 2024-02-19T19:04:56.232 INFO:tasks.workunit.client.1.smithi132.stdout:8/528: write d4/dc/d1b/d8f/fa7 [771184,91553] 0 2024-02-19T19:04:56.234 INFO:tasks.workunit.client.1.smithi132.stdout:9/623: sync 2024-02-19T19:04:56.234 INFO:tasks.workunit.client.1.smithi132.stdout:1/471: dread d1/d1e/d77/d2c/d43/f5d [0,4194304] 0 2024-02-19T19:04:56.239 INFO:tasks.workunit.client.1.smithi132.stdout:3/470: write d8/d13/d2b/d52/d34/d50/f5b [231735,8002] 0 2024-02-19T19:04:56.240 INFO:tasks.workunit.client.1.smithi132.stdout:8/529: fdatasync d4/dc/d1b/d20/f6e 0 2024-02-19T19:04:56.243 INFO:tasks.workunit.client.1.smithi132.stdout:3/471: write d8/d38/f3d [40210,63728] 0 2024-02-19T19:04:56.243 INFO:tasks.workunit.client.1.smithi132.stdout:6/477: dwrite d0/f73 [0,4194304] 0 2024-02-19T19:04:56.244 INFO:tasks.workunit.client.1.smithi132.stdout:4/483: getdents d4/d12/d48/d5f 0 2024-02-19T19:04:56.246 INFO:tasks.workunit.client.1.smithi132.stdout:3/472: fsync d8/d13/d2b/d2e/f7c 0 2024-02-19T19:04:56.247 INFO:tasks.workunit.client.1.smithi132.stdout:2/559: mkdir d0/d24/d58/d53/dab/d8a/da5/db1 0 2024-02-19T19:04:56.251 INFO:tasks.workunit.client.1.smithi132.stdout:9/624: dwrite d12/f43 [0,4194304] 0 2024-02-19T19:04:56.262 INFO:tasks.workunit.client.1.smithi132.stdout:7/582: dread ff [0,4194304] 0 2024-02-19T19:04:56.269 INFO:tasks.workunit.client.1.smithi132.stdout:6/478: rmdir d0/db/d25/d3a/d4b 39 2024-02-19T19:04:56.271 INFO:tasks.workunit.client.1.smithi132.stdout:2/560: rmdir d0/d4/d11/d2e/d5a/d66/dad 39 2024-02-19T19:04:56.273 INFO:tasks.workunit.client.1.smithi132.stdout:2/561: write d0/d4/d11/d2e/fac [185391,65752] 0 2024-02-19T19:04:56.274 INFO:tasks.workunit.client.1.smithi132.stdout:2/562: write d0/f80 [4538014,10794] 0 2024-02-19T19:04:56.274 INFO:tasks.workunit.client.1.smithi132.stdout:0/491: truncate d2/dc/d4a/f57 3848655 0 2024-02-19T19:04:56.275 INFO:tasks.workunit.client.1.smithi132.stdout:2/563: write d0/d19/d1e/f9f [20022,33228] 0 2024-02-19T19:04:56.282 INFO:tasks.workunit.client.1.smithi132.stdout:8/530: dwrite d4/f16 [0,4194304] 0 2024-02-19T19:04:56.283 INFO:tasks.workunit.client.1.smithi132.stdout:8/531: readlink d4/dc/d1b/d33/l64 0 2024-02-19T19:04:56.286 INFO:tasks.workunit.client.1.smithi132.stdout:8/532: chown d4/dc/d1b/d20/d62/d91/c8d 469 1 2024-02-19T19:04:56.291 INFO:tasks.workunit.client.1.smithi132.stdout:9/625: write d12/d2a/f5b [1206839,58992] 0 2024-02-19T19:04:56.292 INFO:tasks.workunit.client.1.smithi132.stdout:7/583: link d1b/d1e/d2b/d62/l69 d1b/d32/lb7 0 2024-02-19T19:04:56.295 INFO:tasks.workunit.client.1.smithi132.stdout:5/573: sync 2024-02-19T19:04:56.295 INFO:tasks.workunit.client.1.smithi132.stdout:8/533: creat d4/dc/d42/fb6 x:0 0 0 2024-02-19T19:04:56.295 INFO:tasks.workunit.client.1.smithi132.stdout:3/473: dwrite d8/d13/d2b/d52/d34/f59 [0,4194304] 0 2024-02-19T19:04:56.299 INFO:tasks.workunit.client.1.smithi132.stdout:0/492: dwrite d2/dc/d4a/f42 [4194304,4194304] 0 2024-02-19T19:04:56.299 INFO:tasks.workunit.client.1.smithi132.stdout:4/484: dwrite d4/d27/d31/d32/d51/d5c/fa8 [0,4194304] 0 2024-02-19T19:04:56.302 INFO:tasks.workunit.client.1.smithi132.stdout:9/626: write d12/f99 [758642,122653] 0 2024-02-19T19:04:56.311 INFO:tasks.workunit.client.1.smithi132.stdout:3/474: rmdir d8/d13/d2b 39 2024-02-19T19:04:56.317 INFO:tasks.workunit.client.1.smithi132.stdout:7/584: mknod d1b/d1e/d2b/d8f/dae/cb8 0 2024-02-19T19:04:56.318 INFO:tasks.workunit.client.1.smithi132.stdout:7/585: fdatasync d1b/d1e/d2b/f5d 0 2024-02-19T19:04:56.321 INFO:tasks.workunit.client.1.smithi132.stdout:6/479: dread d0/db/d25/d3a/d5d/d8f/d22/f23 [0,4194304] 0 2024-02-19T19:04:56.322 INFO:tasks.workunit.client.1.smithi132.stdout:6/480: write d0/d51/f92 [3646798,1523] 0 2024-02-19T19:04:56.325 INFO:tasks.workunit.client.1.smithi132.stdout:0/493: link d2/dc/df/d27/d3a/d52/f9d d2/dc/df/d78/f9f 0 2024-02-19T19:04:56.327 INFO:tasks.workunit.client.1.smithi132.stdout:9/627: mkdir d12/d8d/d3a/d40/d4e/d77/d8a/d82/dd9 0 2024-02-19T19:04:56.328 INFO:tasks.workunit.client.1.smithi132.stdout:3/475: symlink d8/d13/d2b/d52/l90 0 2024-02-19T19:04:56.329 INFO:tasks.workunit.client.1.smithi132.stdout:4/485: creat d4/fab x:0 0 0 2024-02-19T19:04:56.330 INFO:tasks.workunit.client.1.smithi132.stdout:5/574: write d8/d23/d29/d27/f25 [2393686,17831] 0 2024-02-19T19:04:56.332 INFO:tasks.workunit.client.1.smithi132.stdout:7/586: write d1b/f1c [4065111,41187] 0 2024-02-19T19:04:56.333 INFO:tasks.workunit.client.1.smithi132.stdout:0/494: symlink d2/dc/d98/la0 0 2024-02-19T19:04:56.335 INFO:tasks.workunit.client.1.smithi132.stdout:7/587: chown d1b/d1e/d28/f50 4 1 2024-02-19T19:04:56.335 INFO:tasks.workunit.client.1.smithi132.stdout:9/628: dwrite d12/d8d/f28 [0,4194304] 0 2024-02-19T19:04:56.335 INFO:tasks.workunit.client.1.smithi132.stdout:3/476: unlink d8/d13/f20 0 2024-02-19T19:04:56.338 INFO:tasks.workunit.client.1.smithi132.stdout:4/486: write d4/d12/d16/f47 [1668720,3612] 0 2024-02-19T19:04:56.342 INFO:tasks.workunit.client.1.smithi132.stdout:5/575: mkdir d8/da/dc2 0 2024-02-19T19:04:56.343 INFO:tasks.workunit.client.1.smithi132.stdout:8/534: dread d4/d2d/f46 [0,4194304] 0 2024-02-19T19:04:56.343 INFO:tasks.workunit.client.1.smithi132.stdout:4/487: dread - d4/d27/d31/d32/d51/f93 zero size 2024-02-19T19:04:56.345 INFO:tasks.workunit.client.1.smithi132.stdout:7/588: dread ff [0,4194304] 0 2024-02-19T19:04:56.346 INFO:tasks.workunit.client.1.smithi132.stdout:3/477: symlink d8/d13/d2b/l91 0 2024-02-19T19:04:56.346 INFO:tasks.workunit.client.1.smithi132.stdout:6/481: dread d0/db/f58 [0,4194304] 0 2024-02-19T19:04:56.347 INFO:tasks.workunit.client.1.smithi132.stdout:5/576: symlink d8/da/d5a/d84/lc3 0 2024-02-19T19:04:56.349 INFO:tasks.workunit.client.1.smithi132.stdout:0/495: getdents d2/d3e 0 2024-02-19T19:04:56.351 INFO:tasks.workunit.client.1.smithi132.stdout:3/478: dread d8/d13/d2b/d52/d34/d50/f5b [0,4194304] 0 2024-02-19T19:04:56.353 INFO:tasks.workunit.client.1.smithi132.stdout:3/479: stat d8/d13/d2f/f4d 0 2024-02-19T19:04:56.353 INFO:tasks.workunit.client.1.smithi132.stdout:7/589: getdents d1b/d46/d7f 0 2024-02-19T19:04:56.353 INFO:tasks.workunit.client.1.smithi132.stdout:5/577: chown d8/d2e/d71/c7d 3537 1 2024-02-19T19:04:56.354 INFO:tasks.workunit.client.1.smithi132.stdout:8/535: dwrite d4/dc/d1b/d54/f87 [0,4194304] 0 2024-02-19T19:04:56.356 INFO:tasks.workunit.client.1.smithi132.stdout:8/536: truncate d4/fab 1503019 0 2024-02-19T19:04:56.359 INFO:tasks.workunit.client.1.smithi132.stdout:0/496: creat d2/dc/df/d78/fa1 x:0 0 0 2024-02-19T19:04:56.359 INFO:tasks.workunit.client.1.smithi132.stdout:5/578: mkdir d8/d2e/dc4 0 2024-02-19T19:04:56.361 INFO:tasks.workunit.client.1.smithi132.stdout:7/590: mknod d1b/d1e/d28/d5e/cb9 0 2024-02-19T19:04:56.362 INFO:tasks.workunit.client.1.smithi132.stdout:3/480: rename d8/d13/d2b/d52 to d8/d13/d2b/d69/d92 0 2024-02-19T19:04:56.362 INFO:tasks.workunit.client.1.smithi132.stdout:7/591: readlink l16 0 2024-02-19T19:04:56.363 INFO:tasks.workunit.client.1.smithi132.stdout:8/537: creat d4/dc/d1b/d33/fb7 x:0 0 0 2024-02-19T19:04:56.365 INFO:tasks.workunit.client.1.smithi132.stdout:7/592: write d1b/d1e/d28/f44 [2473117,63784] 0 2024-02-19T19:04:56.365 INFO:tasks.workunit.client.1.smithi132.stdout:4/488: dread d4/d12/d48/f7c [0,4194304] 0 2024-02-19T19:04:56.365 INFO:tasks.workunit.client.1.smithi132.stdout:9/629: dwrite d12/d8d/fc0 [4194304,4194304] 0 2024-02-19T19:04:56.367 INFO:tasks.workunit.client.1.smithi132.stdout:8/538: truncate d4/dc/d1b/d54/d81/fb3 235610 0 2024-02-19T19:04:56.367 INFO:tasks.workunit.client.1.smithi132.stdout:3/481: fdatasync d8/d13/d2b/f2c 0 2024-02-19T19:04:56.368 INFO:tasks.workunit.client.1.smithi132.stdout:8/539: fsync d4/dc/d1b/d20/f6e 0 2024-02-19T19:04:56.369 INFO:tasks.workunit.client.1.smithi132.stdout:4/489: mkdir d4/d19/d21/d56/dac 0 2024-02-19T19:04:56.369 INFO:tasks.workunit.client.1.smithi132.stdout:0/497: dread d2/f6 [0,4194304] 0 2024-02-19T19:04:56.370 INFO:tasks.workunit.client.1.smithi132.stdout:8/540: readlink d4/dc/d1b/d20/d23/l70 0 2024-02-19T19:04:56.370 INFO:tasks.workunit.client.1.smithi132.stdout:7/593: mknod d1b/cba 0 2024-02-19T19:04:56.372 INFO:tasks.workunit.client.1.smithi132.stdout:3/482: rename d8/d13/d2b/d2e to d8/d93 0 2024-02-19T19:04:56.375 INFO:tasks.workunit.client.1.smithi132.stdout:8/541: creat d4/dc/d1b/d38/fb8 x:0 0 0 2024-02-19T19:04:56.377 INFO:tasks.workunit.client.1.smithi132.stdout:8/542: read - d4/dc/d1b/d33/fb7 zero size 2024-02-19T19:04:56.377 INFO:tasks.workunit.client.1.smithi132.stdout:4/490: dwrite d4/d27/d31/d32/d51/d5c/f5d [0,4194304] 0 2024-02-19T19:04:56.378 INFO:tasks.workunit.client.1.smithi132.stdout:4/491: chown d4/d12/d9f 19839 1 2024-02-19T19:04:56.379 INFO:tasks.workunit.client.1.smithi132.stdout:0/498: getdents d2/dc/df/d27 0 2024-02-19T19:04:56.380 INFO:tasks.workunit.client.1.smithi132.stdout:0/499: stat d2/d6e/d7a 0 2024-02-19T19:04:56.383 INFO:tasks.workunit.client.1.smithi132.stdout:3/483: rename d8/d13/d2b/d69/d92/d34/d50/c56 to d8/d13/d2b/d69/d92/d34/d42/c94 0 2024-02-19T19:04:56.384 INFO:tasks.workunit.client.1.smithi132.stdout:4/492: readlink d4/l6 0 2024-02-19T19:04:56.385 INFO:tasks.workunit.client.1.smithi132.stdout:8/543: mkdir d4/dc/d1b/d20/d23/d68/d79/db9 0 2024-02-19T19:04:56.385 INFO:tasks.workunit.client.1.smithi132.stdout:8/544: fsync d4/f29 0 2024-02-19T19:04:56.387 INFO:tasks.workunit.client.1.smithi132.stdout:3/484: creat d8/d13/d2b/d60/f95 x:0 0 0 2024-02-19T19:04:56.390 INFO:tasks.workunit.client.1.smithi132.stdout:4/493: dread d4/d27/d31/d32/d51/d5c/fa8 [0,4194304] 0 2024-02-19T19:04:56.392 INFO:tasks.workunit.client.1.smithi132.stdout:4/494: fsync d4/d12/f2c 0 2024-02-19T19:04:56.393 INFO:tasks.workunit.client.1.smithi132.stdout:8/545: dwrite d4/fb [0,4194304] 0 2024-02-19T19:04:56.398 INFO:tasks.workunit.client.1.smithi132.stdout:5/579: dread d8/da/db/f44 [0,4194304] 0 2024-02-19T19:04:56.399 INFO:tasks.workunit.client.1.smithi132.stdout:4/495: readlink d4/l22 0 2024-02-19T19:04:56.402 INFO:tasks.workunit.client.1.smithi132.stdout:4/496: dread d4/d4d/f64 [0,4194304] 0 2024-02-19T19:04:56.404 INFO:tasks.workunit.client.1.smithi132.stdout:3/485: getdents d8/d13/d2b/d69/d92 0 2024-02-19T19:04:56.411 INFO:tasks.workunit.client.1.smithi132.stdout:8/546: getdents d4/dc/d1b/d8f/daf 0 2024-02-19T19:04:56.417 INFO:tasks.workunit.client.1.smithi132.stdout:8/547: dwrite d4/dc/d1b/d38/d69/f6d [0,4194304] 0 2024-02-19T19:04:56.419 INFO:tasks.workunit.client.1.smithi132.stdout:3/486: dread d8/d13/d2f/f66 [0,4194304] 0 2024-02-19T19:04:56.422 INFO:tasks.workunit.client.1.smithi132.stdout:5/580: truncate d8/da/f33 2471184 0 2024-02-19T19:04:56.422 INFO:tasks.workunit.client.1.smithi132.stdout:0/500: dwrite d2/dc/df/d15/f22 [0,4194304] 0 2024-02-19T19:04:56.423 INFO:tasks.workunit.client.1.smithi132.stdout:4/497: creat d4/fad x:0 0 0 2024-02-19T19:04:56.424 INFO:tasks.workunit.client.1.smithi132.stdout:7/594: dread d1b/d1e/d28/f3b [0,4194304] 0 2024-02-19T19:04:56.427 INFO:tasks.workunit.client.1.smithi132.stdout:8/548: dread d4/dc/d1b/f32 [0,4194304] 0 2024-02-19T19:04:56.429 INFO:tasks.workunit.client.1.smithi132.stdout:3/487: creat d8/d13/d19/f96 x:0 0 0 2024-02-19T19:04:56.431 INFO:tasks.workunit.client.1.smithi132.stdout:0/501: mknod d2/dc/d4a/d58/ca2 0 2024-02-19T19:04:56.432 INFO:tasks.workunit.client.1.smithi132.stdout:1/472: sync 2024-02-19T19:04:56.433 INFO:tasks.workunit.client.1.smithi132.stdout:1/473: dread - d1/d1e/d77/f73 zero size 2024-02-19T19:04:56.435 INFO:tasks.workunit.client.1.smithi132.stdout:7/595: creat d1b/d1e/d28/fbb x:0 0 0 2024-02-19T19:04:56.439 INFO:tasks.workunit.client.1.smithi132.stdout:0/502: dwrite d2/dc/df/d15/d34/f3c [0,4194304] 0 2024-02-19T19:04:56.442 INFO:tasks.workunit.client.1.smithi132.stdout:7/596: rmdir d1b/d46/d56 39 2024-02-19T19:04:56.444 INFO:tasks.workunit.client.1.smithi132.stdout:7/597: read d1b/d1e/d2b/f95 [4377395,5010] 0 2024-02-19T19:04:56.445 INFO:tasks.workunit.client.1.smithi132.stdout:0/503: creat d2/dc/fa3 x:0 0 0 2024-02-19T19:04:56.447 INFO:tasks.workunit.client.1.smithi132.stdout:0/504: chown d2/dc/df/d15/d2f/d3f/f8d 337 1 2024-02-19T19:04:56.448 INFO:tasks.workunit.client.1.smithi132.stdout:4/498: link d4/d19/l2b d4/d27/d31/d32/d51/d72/lae 0 2024-02-19T19:04:56.448 INFO:tasks.workunit.client.1.smithi132.stdout:0/505: chown d2/dc/df/d15/d2f/d69/f88 391 1 2024-02-19T19:04:56.450 INFO:tasks.workunit.client.1.smithi132.stdout:4/499: dread - d4/d27/d31/d32/d51/f97 zero size 2024-02-19T19:04:56.451 INFO:tasks.workunit.client.1.smithi132.stdout:7/598: chown d1b/d1e/d41/f6a 1529 1 2024-02-19T19:04:56.453 INFO:tasks.workunit.client.1.smithi132.stdout:3/488: getdents d8/d13/d2b/d69 0 2024-02-19T19:04:56.453 INFO:tasks.workunit.client.1.smithi132.stdout:8/549: dwrite d4/f6 [0,4194304] 0 2024-02-19T19:04:56.455 INFO:tasks.workunit.client.1.smithi132.stdout:0/506: creat d2/dc/d4a/d25/d43/d5f/fa4 x:0 0 0 2024-02-19T19:04:56.456 INFO:tasks.workunit.client.1.smithi132.stdout:7/599: mknod d1b/d1e/d28/da9/cbc 0 2024-02-19T19:04:56.457 INFO:tasks.workunit.client.1.smithi132.stdout:7/600: truncate d1b/d1e/d28/f97 361900 0 2024-02-19T19:04:56.461 INFO:tasks.workunit.client.1.smithi132.stdout:7/601: write d1b/d1e/d2b/f5d [4565741,65701] 0 2024-02-19T19:04:56.462 INFO:tasks.workunit.client.1.smithi132.stdout:4/500: creat d4/d27/d31/d32/faf x:0 0 0 2024-02-19T19:04:56.462 INFO:tasks.workunit.client.1.smithi132.stdout:8/550: dread d4/dc/d1b/d8f/fa7 [0,4194304] 0 2024-02-19T19:04:56.463 INFO:tasks.workunit.client.1.smithi132.stdout:1/474: write d1/d3a/f40 [236518,97675] 0 2024-02-19T19:04:56.466 INFO:tasks.workunit.client.1.smithi132.stdout:8/551: write d4/dc/d1b/d20/f59 [386934,94818] 0 2024-02-19T19:04:56.467 INFO:tasks.workunit.client.1.smithi132.stdout:0/507: dwrite d2/dc/df/d78/fa1 [0,4194304] 0 2024-02-19T19:04:56.468 INFO:tasks.workunit.client.1.smithi132.stdout:7/602: mkdir d1b/d1e/d41/da3/dbd 0 2024-02-19T19:04:56.470 INFO:tasks.workunit.client.1.smithi132.stdout:1/475: mkdir d1/d95 0 2024-02-19T19:04:56.471 INFO:tasks.workunit.client.1.smithi132.stdout:4/501: mknod d4/d12/d48/d5f/cb0 0 2024-02-19T19:04:56.474 INFO:tasks.workunit.client.1.smithi132.stdout:0/508: rename d2/dc/df/d15/d2f/d69/l93 to d2/dc/df/d15/d2f/la5 0 2024-02-19T19:04:56.477 INFO:tasks.workunit.client.1.smithi132.stdout:1/476: symlink d1/d1e/d77/d52/d64/l96 0 2024-02-19T19:04:56.479 INFO:tasks.workunit.client.1.smithi132.stdout:3/489: truncate f0 2281826 0 2024-02-19T19:04:56.481 INFO:tasks.workunit.client.1.smithi132.stdout:4/502: rename d4/d12/d48/f65 to d4/d27/d31/d71/d14/d24/fb1 0 2024-02-19T19:04:56.482 INFO:tasks.workunit.client.1.smithi132.stdout:4/503: truncate d4/d27/f9c 930630 0 2024-02-19T19:04:56.484 INFO:tasks.workunit.client.1.smithi132.stdout:1/477: unlink d1/d1e/d77/d52/d64/c90 0 2024-02-19T19:04:56.485 INFO:tasks.workunit.client.1.smithi132.stdout:1/478: fsync d1/d1e/d77/d52/d64/f91 0 2024-02-19T19:04:56.486 INFO:tasks.workunit.client.1.smithi132.stdout:8/552: dread d4/dc/d1b/d20/fa8 [0,4194304] 0 2024-02-19T19:04:56.486 INFO:tasks.workunit.client.1.smithi132.stdout:0/509: link d2/dc/d4a/d25/f26 d2/dc/d4a/d58/fa6 0 2024-02-19T19:04:56.487 INFO:tasks.workunit.client.1.smithi132.stdout:8/553: write d4/dc/d1b/d20/f8b [697315,30715] 0 2024-02-19T19:04:56.487 INFO:tasks.workunit.client.1.smithi132.stdout:3/490: mkdir d8/d13/d2b/d69/d92/d34/d97 0 2024-02-19T19:04:56.488 INFO:tasks.workunit.client.1.smithi132.stdout:3/491: chown d8/d13/d2f 56 1 2024-02-19T19:04:56.489 INFO:tasks.workunit.client.1.smithi132.stdout:3/492: chown d8/d13/d2b/f2c 412699 1 2024-02-19T19:04:56.490 INFO:tasks.workunit.client.1.smithi132.stdout:8/554: write d4/dc/d1b/d20/d62/d91/fb2 [367842,28869] 0 2024-02-19T19:04:56.490 INFO:tasks.workunit.client.1.smithi132.stdout:1/479: unlink d1/d1e/d77/d4e/f4f 0 2024-02-19T19:04:56.492 INFO:tasks.workunit.client.1.smithi132.stdout:7/603: dread d1b/d1e/d28/f3c [0,4194304] 0 2024-02-19T19:04:56.493 INFO:tasks.workunit.client.1.smithi132.stdout:3/493: creat d8/d13/d57/f98 x:0 0 0 2024-02-19T19:04:56.495 INFO:tasks.workunit.client.1.smithi132.stdout:0/510: mkdir d2/dc/df/d15/d34/d84/da7 0 2024-02-19T19:04:56.497 INFO:tasks.workunit.client.1.smithi132.stdout:3/494: unlink d8/d38/d44/d6e/f8e 0 2024-02-19T19:04:56.500 INFO:tasks.workunit.client.1.smithi132.stdout:1/480: dread d1/f53 [0,4194304] 0 2024-02-19T19:04:56.501 INFO:tasks.workunit.client.1.smithi132.stdout:0/511: mkdir d2/dc/df/d27/d3a/d52/da8 0 2024-02-19T19:04:56.502 INFO:tasks.workunit.client.1.smithi132.stdout:3/495: rmdir d8/d93 39 2024-02-19T19:04:56.512 INFO:tasks.workunit.client.1.smithi132.stdout:0/512: creat d2/dc/df/fa9 x:0 0 0 2024-02-19T19:04:56.519 INFO:tasks.workunit.client.1.smithi132.stdout:3/496: creat d8/d13/d2b/f99 x:0 0 0 2024-02-19T19:04:56.523 INFO:tasks.workunit.client.1.smithi132.stdout:8/555: truncate d4/dc/f3f 2458072 0 2024-02-19T19:04:56.524 INFO:tasks.workunit.client.1.smithi132.stdout:7/604: truncate d1b/d1e/f38 4736986 0 2024-02-19T19:04:56.524 INFO:tasks.workunit.client.1.smithi132.stdout:7/605: stat d1b/d46/d76/l89 0 2024-02-19T19:04:56.527 INFO:tasks.workunit.client.1.smithi132.stdout:7/606: dread ff [0,4194304] 0 2024-02-19T19:04:56.527 INFO:tasks.workunit.client.1.smithi132.stdout:3/497: creat d8/d13/d2f/f9a x:0 0 0 2024-02-19T19:04:56.527 INFO:tasks.workunit.client.1.smithi132.stdout:2/564: sync 2024-02-19T19:04:56.529 INFO:tasks.workunit.client.1.smithi132.stdout:0/513: rename d2/f2b to d2/dc/df/faa 0 2024-02-19T19:04:56.530 INFO:tasks.workunit.client.1.smithi132.stdout:8/556: creat d4/dc/d1b/d38/d69/d88/fba x:0 0 0 2024-02-19T19:04:56.533 INFO:tasks.workunit.client.1.smithi132.stdout:8/557: chown d4/dc/d1b/d54/d5e/f50 611228 1 2024-02-19T19:04:56.534 INFO:tasks.workunit.client.1.smithi132.stdout:2/565: dread d0/f80 [0,4194304] 0 2024-02-19T19:04:56.534 INFO:tasks.workunit.client.1.smithi132.stdout:7/607: dwrite d1b/d1e/fb2 [4194304,4194304] 0 2024-02-19T19:04:56.538 INFO:tasks.workunit.client.1.smithi132.stdout:3/498: creat d8/d13/d2b/d60/f9b x:0 0 0 2024-02-19T19:04:56.540 INFO:tasks.workunit.client.1.smithi132.stdout:0/514: dwrite d2/dc/d4a/d58/f95 [0,4194304] 0 2024-02-19T19:04:56.541 INFO:tasks.workunit.client.1.smithi132.stdout:8/558: symlink d4/dac/lbb 0 2024-02-19T19:04:56.547 INFO:tasks.workunit.client.1.smithi132.stdout:2/566: mknod d0/d4/df/d40/d68/cb2 0 2024-02-19T19:04:56.550 INFO:tasks.workunit.client.1.smithi132.stdout:8/559: symlink d4/dac/lbc 0 2024-02-19T19:04:56.550 INFO:tasks.workunit.client.1.smithi132.stdout:8/560: stat d4/dc/d1b/d33/l9a 0 2024-02-19T19:04:56.554 INFO:tasks.workunit.client.1.smithi132.stdout:7/608: unlink d1b/d5a/d63/l82 0 2024-02-19T19:04:56.558 INFO:tasks.workunit.client.1.smithi132.stdout:8/561: mkdir d4/dc/d1b/d54/d81/dbd 0 2024-02-19T19:04:56.558 INFO:tasks.workunit.client.1.smithi132.stdout:8/562: fsync d4/dc/d42/f97 0 2024-02-19T19:04:56.560 INFO:tasks.workunit.client.1.smithi132.stdout:0/515: link d2/dc/df/d27/f61 d2/d6e/d7a/fab 0 2024-02-19T19:04:56.563 INFO:tasks.workunit.client.1.smithi132.stdout:0/516: creat d2/d6e/fac x:0 0 0 2024-02-19T19:04:56.566 INFO:tasks.workunit.client.1.smithi132.stdout:8/563: mkdir d4/dc/d1b/d20/dbe 0 2024-02-19T19:04:56.566 INFO:tasks.workunit.client.1.smithi132.stdout:8/564: write d4/d2d/f46 [3395146,23265] 0 2024-02-19T19:04:56.567 INFO:tasks.workunit.client.1.smithi132.stdout:7/609: link d1b/d5a/d63/d98/ca7 d1b/d1e/d41/da3/dbd/cbe 0 2024-02-19T19:04:56.568 INFO:tasks.workunit.client.1.smithi132.stdout:7/610: fdatasync d1b/d1e/f90 0 2024-02-19T19:04:56.571 INFO:tasks.workunit.client.1.smithi132.stdout:0/517: symlink d2/d6e/lad 0 2024-02-19T19:04:56.573 INFO:tasks.workunit.client.1.smithi132.stdout:8/565: creat d4/d2d/fbf x:0 0 0 2024-02-19T19:04:56.573 INFO:tasks.workunit.client.1.smithi132.stdout:8/566: stat d4/fb 0 2024-02-19T19:04:56.576 INFO:tasks.workunit.client.1.smithi132.stdout:0/518: rmdir d2/dc/df/d27/d3a 39 2024-02-19T19:04:56.577 INFO:tasks.workunit.client.1.smithi132.stdout:7/611: dread d1b/d1e/d2b/d62/f67 [0,4194304] 0 2024-02-19T19:04:56.577 INFO:tasks.workunit.client.1.smithi132.stdout:3/499: dwrite f0 [0,4194304] 0 2024-02-19T19:04:56.579 INFO:tasks.workunit.client.1.smithi132.stdout:2/567: truncate d0/d4/df/f5f 555855 0 2024-02-19T19:04:56.579 INFO:tasks.workunit.client.1.smithi132.stdout:3/500: fsync d8/d13/d2f/f58 0 2024-02-19T19:04:56.580 INFO:tasks.workunit.client.1.smithi132.stdout:8/567: creat d4/d2d/fc0 x:0 0 0 2024-02-19T19:04:56.582 INFO:tasks.workunit.client.1.smithi132.stdout:8/568: fdatasync d4/dc/d1b/f43 0 2024-02-19T19:04:56.582 INFO:tasks.workunit.client.1.smithi132.stdout:7/612: creat d1b/d1e/d41/fbf x:0 0 0 2024-02-19T19:04:56.583 INFO:tasks.workunit.client.1.smithi132.stdout:7/613: fsync d1b/d1e/d28/f97 0 2024-02-19T19:04:56.587 INFO:tasks.workunit.client.1.smithi132.stdout:8/569: dwrite f2 [0,4194304] 0 2024-02-19T19:04:56.595 INFO:tasks.workunit.client.1.smithi132.stdout:7/614: creat d1b/d1e/d2b/fc0 x:0 0 0 2024-02-19T19:04:56.596 INFO:tasks.workunit.client.1.smithi132.stdout:8/570: stat d4/cf 0 2024-02-19T19:04:56.597 INFO:tasks.workunit.client.1.smithi132.stdout:2/568: creat d0/d24/fb3 x:0 0 0 2024-02-19T19:04:56.600 INFO:tasks.workunit.client.1.smithi132.stdout:3/501: getdents d8/d13/d57 0 2024-02-19T19:04:56.601 INFO:tasks.workunit.client.1.smithi132.stdout:2/569: mknod d0/d19/d6a/cb4 0 2024-02-19T19:04:56.602 INFO:tasks.workunit.client.1.smithi132.stdout:2/570: chown d0/d4/df 80 1 2024-02-19T19:04:56.606 INFO:tasks.workunit.client.1.smithi132.stdout:3/502: symlink d8/d13/d2b/d69/d92/l9c 0 2024-02-19T19:04:56.606 INFO:tasks.workunit.client.1.smithi132.stdout:8/571: link d4/dc/d1b/d20/d62/d91/l7e d4/d2d/d82/lc1 0 2024-02-19T19:04:56.607 INFO:tasks.workunit.client.1.smithi132.stdout:2/571: dwrite d0/d4/df/d40/f74 [0,4194304] 0 2024-02-19T19:04:56.608 INFO:tasks.workunit.client.1.smithi132.stdout:8/572: dread - d4/dc/d1b/d33/fa5 zero size 2024-02-19T19:04:56.612 INFO:tasks.workunit.client.1.smithi132.stdout:8/573: dread d4/dc/d1b/d20/d23/f2b [0,4194304] 0 2024-02-19T19:04:56.619 INFO:tasks.workunit.client.1.smithi132.stdout:3/503: rename d8/d13/d2b/d60/c78 to d8/d93/c9d 0 2024-02-19T19:04:56.621 INFO:tasks.workunit.client.1.smithi132.stdout:7/615: truncate d1b/d1e/fb2 7670388 0 2024-02-19T19:04:56.623 INFO:tasks.workunit.client.1.smithi132.stdout:8/574: creat d4/dc/d1b/d38/d69/d88/fc2 x:0 0 0 2024-02-19T19:04:56.629 INFO:tasks.workunit.client.1.smithi132.stdout:2/572: creat d0/d4/d11/d2e/d5a/fb5 x:0 0 0 2024-02-19T19:04:56.631 INFO:tasks.workunit.client.1.smithi132.stdout:3/504: rename d8/d38/l8a to d8/d13/d19/l9e 0 2024-02-19T19:04:56.636 INFO:tasks.workunit.client.1.smithi132.stdout:3/505: fsync d8/d13/d2b/d69/d92/d34/f59 0 2024-02-19T19:04:56.638 INFO:tasks.workunit.client.1.smithi132.stdout:2/573: dwrite d0/d24/d58/d53/dab/d8a/fa4 [0,4194304] 0 2024-02-19T19:04:56.642 INFO:tasks.workunit.client.1.smithi132.stdout:3/506: dread f0 [0,4194304] 0 2024-02-19T19:04:56.644 INFO:tasks.workunit.client.1.smithi132.stdout:8/575: dread d4/dc/d1b/d20/d62/d91/f21 [0,4194304] 0 2024-02-19T19:04:56.644 INFO:tasks.workunit.client.1.smithi132.stdout:3/507: read d8/d93/f6f [106646,110249] 0 2024-02-19T19:04:56.664 INFO:tasks.workunit.client.1.smithi132.stdout:3/508: readlink d8/lf 0 2024-02-19T19:04:56.664 INFO:tasks.workunit.client.1.smithi132.stdout:8/576: creat d4/dc/d1b/d38/d69/fc3 x:0 0 0 2024-02-19T19:04:56.672 INFO:tasks.workunit.client.1.smithi132.stdout:8/577: creat d4/dc/d1b/d20/d23/fc4 x:0 0 0 2024-02-19T19:04:56.675 INFO:tasks.workunit.client.1.smithi132.stdout:8/578: truncate d4/dc/d1b/d54/d81/fb3 458733 0 2024-02-19T19:04:56.679 INFO:tasks.workunit.client.1.smithi132.stdout:8/579: rmdir d4/dc/d1b/d20/d23/d68/d79/db9 0 2024-02-19T19:04:56.679 INFO:tasks.workunit.client.1.smithi132.stdout:2/574: truncate d0/d4/df/f3b 1259333 0 2024-02-19T19:04:56.681 INFO:tasks.workunit.client.1.smithi132.stdout:3/509: dread d8/d13/d2b/d69/d92/f8c [4194304,4194304] 0 2024-02-19T19:04:56.681 INFO:tasks.workunit.client.1.smithi132.stdout:8/580: symlink d4/dc/d1b/d20/d8a/lc5 0 2024-02-19T19:04:56.682 INFO:tasks.workunit.client.1.smithi132.stdout:3/510: read d8/d13/d2b/f33 [570341,68769] 0 2024-02-19T19:04:56.682 INFO:tasks.workunit.client.1.smithi132.stdout:2/575: creat d0/d24/d58/d53/dab/d8a/da5/db1/fb6 x:0 0 0 2024-02-19T19:04:56.683 INFO:tasks.workunit.client.1.smithi132.stdout:8/581: creat d4/dc/d1b/d38/fc6 x:0 0 0 2024-02-19T19:04:56.683 INFO:tasks.workunit.client.1.smithi132.stdout:5/581: sync 2024-02-19T19:04:56.683 INFO:tasks.workunit.client.1.smithi132.stdout:6/482: sync 2024-02-19T19:04:56.684 INFO:tasks.workunit.client.1.smithi132.stdout:9/630: sync 2024-02-19T19:04:56.684 INFO:tasks.workunit.client.1.smithi132.stdout:3/511: write d8/d38/f8b [1037362,5172] 0 2024-02-19T19:04:56.686 INFO:tasks.workunit.client.1.smithi132.stdout:9/631: rmdir d12/d2a/d4c 39 2024-02-19T19:04:56.687 INFO:tasks.workunit.client.1.smithi132.stdout:9/632: readlink d12/d1b/l1e 0 2024-02-19T19:04:56.687 INFO:tasks.workunit.client.1.smithi132.stdout:8/582: unlink d4/dc/c10 0 2024-02-19T19:04:56.689 INFO:tasks.workunit.client.1.smithi132.stdout:2/576: readlink d0/d4/d11/d2e/d5a/d66/dad/l9a 0 2024-02-19T19:04:56.690 INFO:tasks.workunit.client.1.smithi132.stdout:8/583: chown d4/dc/d1b/d20/f53 2 1 2024-02-19T19:04:56.690 INFO:tasks.workunit.client.1.smithi132.stdout:2/577: stat d0/d24/d58/c69 0 2024-02-19T19:04:56.691 INFO:tasks.workunit.client.1.smithi132.stdout:3/512: mkdir d8/d13/d2b/d69/d7e/d9f 0 2024-02-19T19:04:56.691 INFO:tasks.workunit.client.1.smithi132.stdout:6/483: dwrite d0/db/d25/d38/f78 [0,4194304] 0 2024-02-19T19:04:56.694 INFO:tasks.workunit.client.1.smithi132.stdout:5/582: rename d8/da/d5a/db8 to d8/d23/d29/dc5 0 2024-02-19T19:04:56.700 INFO:tasks.workunit.client.1.smithi132.stdout:9/633: dwrite d12/d2a/d4c/dad/db2/fce [0,4194304] 0 2024-02-19T19:04:56.702 INFO:tasks.workunit.client.1.smithi132.stdout:9/634: read - d12/d2a/d4c/d61/fa8 zero size 2024-02-19T19:04:56.704 INFO:tasks.workunit.client.1.smithi132.stdout:5/583: mknod d8/d1a/d35/d77/da9/cc6 0 2024-02-19T19:04:56.708 INFO:tasks.workunit.client.1.smithi132.stdout:9/635: read - d12/d3e/f9f zero size 2024-02-19T19:04:56.709 INFO:tasks.workunit.client.1.smithi132.stdout:9/636: readlink d12/d8d/d3a/d40/d4e/d77/l7b 0 2024-02-19T19:04:56.709 INFO:tasks.workunit.client.1.smithi132.stdout:5/584: rename d8/da/d5a/l95 to d8/d23/d29/d27/lc7 0 2024-02-19T19:04:56.713 INFO:tasks.workunit.client.1.smithi132.stdout:9/637: rmdir d12/d8d/d3a/d40/d4e/d77 39 2024-02-19T19:04:56.718 INFO:tasks.workunit.client.1.smithi132.stdout:9/638: dwrite fb [0,4194304] 0 2024-02-19T19:04:56.721 INFO:tasks.workunit.client.1.smithi132.stdout:2/578: write d0/d3c/d3f/f81 [4715868,125222] 0 2024-02-19T19:04:56.726 INFO:tasks.workunit.client.1.smithi132.stdout:2/579: chown d0/d4/d11/c63 117 1 2024-02-19T19:04:56.726 INFO:tasks.workunit.client.1.smithi132.stdout:8/584: truncate d4/dc/d1b/d54/d5e/f50 925398 0 2024-02-19T19:04:56.729 INFO:tasks.workunit.client.1.smithi132.stdout:9/639: truncate d12/d8d/d3a/d40/d4e/d77/f9c 752300 0 2024-02-19T19:04:56.729 INFO:tasks.workunit.client.1.smithi132.stdout:2/580: symlink d0/d3c/d3f/lb7 0 2024-02-19T19:04:56.730 INFO:tasks.workunit.client.1.smithi132.stdout:6/484: dwrite d0/dc/f24 [0,4194304] 0 2024-02-19T19:04:56.732 INFO:tasks.workunit.client.1.smithi132.stdout:3/513: dwrite d8/d13/d19/f2d [0,4194304] 0 2024-02-19T19:04:56.733 INFO:tasks.workunit.client.1.smithi132.stdout:3/514: fsync d8/fd 0 2024-02-19T19:04:56.734 INFO:tasks.workunit.client.1.smithi132.stdout:8/585: truncate d4/dc/d1b/d20/d62/d91/fb2 1160059 0 2024-02-19T19:04:56.735 INFO:tasks.workunit.client.1.smithi132.stdout:2/581: mknod d0/d4/df/d40/d68/cb8 0 2024-02-19T19:04:56.736 INFO:tasks.workunit.client.1.smithi132.stdout:3/515: write d8/d13/d2b/d60/f9b [862030,64845] 0 2024-02-19T19:04:56.737 INFO:tasks.workunit.client.1.smithi132.stdout:8/586: chown d4/dc/d1b/d20/d62/f9e 3808112 1 2024-02-19T19:04:56.740 INFO:tasks.workunit.client.1.smithi132.stdout:3/516: readlink d8/d13/d2b/d69/d92/d34/d42/d6c/l88 0 2024-02-19T19:04:56.741 INFO:tasks.workunit.client.1.smithi132.stdout:6/485: dwrite d0/db/d25/f8d [0,4194304] 0 2024-02-19T19:04:56.743 INFO:tasks.workunit.client.1.smithi132.stdout:8/587: mknod d4/dc/d1b/cc7 0 2024-02-19T19:04:56.744 INFO:tasks.workunit.client.1.smithi132.stdout:6/486: write d0/db/d25/d3a/d5d/d8f/f48 [428476,45913] 0 2024-02-19T19:04:56.746 INFO:tasks.workunit.client.1.smithi132.stdout:2/582: mkdir d0/d4/db9 0 2024-02-19T19:04:56.748 INFO:tasks.workunit.client.1.smithi132.stdout:6/487: fsync d0/db/d25/d3a/d5d/d8f/d46/fa5 0 2024-02-19T19:04:56.750 INFO:tasks.workunit.client.1.smithi132.stdout:5/585: dread d8/fe [0,4194304] 0 2024-02-19T19:04:56.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:56 smithi132 ceph-mon[37969]: Standby manager daemon smithi132.psllsa restarted 2024-02-19T19:04:56.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:56 smithi132 ceph-mon[37969]: Standby manager daemon smithi132.psllsa started 2024-02-19T19:04:56.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:56 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/1523056338' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/crt"}]: dispatch 2024-02-19T19:04:56.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:56 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/1523056338' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:04:56.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:56 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/1523056338' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/key"}]: dispatch 2024-02-19T19:04:56.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:56 smithi132 ceph-mon[37969]: from='mgr.? 172.21.15.132:0/1523056338' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:04:56.753 INFO:tasks.workunit.client.1.smithi132.stdout:8/588: creat d4/dc/d1b/d20/d62/d91/fc8 x:0 0 0 2024-02-19T19:04:56.754 INFO:tasks.workunit.client.1.smithi132.stdout:3/517: dwrite d8/d93/f3b [4194304,4194304] 0 2024-02-19T19:04:56.755 INFO:tasks.workunit.client.1.smithi132.stdout:8/589: chown d4/dc/d1b/d38/fc6 3974 1 2024-02-19T19:04:56.755 INFO:tasks.workunit.client.1.smithi132.stdout:5/586: chown d8/d1a/lc0 4583 1 2024-02-19T19:04:56.756 INFO:tasks.workunit.client.1.smithi132.stdout:8/590: chown d4/dc/d1b 0 1 2024-02-19T19:04:56.757 INFO:tasks.workunit.client.1.smithi132.stdout:5/587: truncate d8/d1a/f9d 1023041 0 2024-02-19T19:04:56.764 INFO:tasks.workunit.client.1.smithi132.stdout:9/640: write d12/d8d/d3a/d85/d87/fb0 [2414868,64515] 0 2024-02-19T19:04:56.765 INFO:tasks.workunit.client.1.smithi132.stdout:8/591: rmdir d4/dc/d1b/d38/d69/d88 39 2024-02-19T19:04:56.771 INFO:tasks.workunit.client.1.smithi132.stdout:5/588: rename d8/d1a/d35/d77 to d8/d2e/dc4/dc8 0 2024-02-19T19:04:56.773 INFO:tasks.workunit.client.1.smithi132.stdout:5/589: write d8/d1a/d35/f4b [595932,117980] 0 2024-02-19T19:04:56.775 INFO:tasks.workunit.client.1.smithi132.stdout:2/583: write d0/d4/df/f3b [2143863,36512] 0 2024-02-19T19:04:56.776 INFO:tasks.workunit.client.1.smithi132.stdout:9/641: fsync d12/f53 0 2024-02-19T19:04:56.778 INFO:tasks.workunit.client.1.smithi132.stdout:5/590: truncate d8/d23/d3b/f65 1659873 0 2024-02-19T19:04:56.779 INFO:tasks.workunit.client.1.smithi132.stdout:5/591: chown d8/d2e 6 1 2024-02-19T19:04:56.779 INFO:tasks.workunit.client.1.smithi132.stdout:2/584: write d0/d4/d11/f15 [3039759,100240] 0 2024-02-19T19:04:56.781 INFO:tasks.workunit.client.1.smithi132.stdout:8/592: creat d4/dc/d1b/d38/d69/fc9 x:0 0 0 2024-02-19T19:04:56.782 INFO:tasks.workunit.client.1.smithi132.stdout:2/585: dread - d0/d3c/f92 zero size 2024-02-19T19:04:56.783 INFO:tasks.workunit.client.1.smithi132.stdout:3/518: dwrite d8/d13/d2b/f33 [0,4194304] 0 2024-02-19T19:04:56.785 INFO:tasks.workunit.client.1.smithi132.stdout:8/593: creat d4/d2d/fca x:0 0 0 2024-02-19T19:04:56.786 INFO:tasks.workunit.client.1.smithi132.stdout:2/586: write d0/d19/d1e/f52 [1473980,28324] 0 2024-02-19T19:04:56.786 INFO:tasks.workunit.client.1.smithi132.stdout:3/519: fdatasync d8/d13/d2f/f9a 0 2024-02-19T19:04:56.788 INFO:tasks.workunit.client.1.smithi132.stdout:9/642: getdents d12/d8d/d3a/d40/d4e/d77/d9d 0 2024-02-19T19:04:56.788 INFO:tasks.workunit.client.1.smithi132.stdout:8/594: chown d4/d2d/f46 190 1 2024-02-19T19:04:56.791 INFO:tasks.workunit.client.1.smithi132.stdout:8/595: write d4/dc/d1b/d38/d69/d74/fb5 [669446,18629] 0 2024-02-19T19:04:56.792 INFO:tasks.workunit.client.1.smithi132.stdout:5/592: dwrite d8/d23/d29/fa1 [0,4194304] 0 2024-02-19T19:04:56.799 INFO:tasks.workunit.client.1.smithi132.stdout:2/587: dwrite d0/d19/d1e/f97 [0,4194304] 0 2024-02-19T19:04:56.799 INFO:tasks.workunit.client.1.smithi132.stdout:8/596: stat d4/dc/d1b/d33/f5b 0 2024-02-19T19:04:56.802 INFO:tasks.workunit.client.1.smithi132.stdout:3/520: dwrite d8/d13/d2b/d69/d92/d34/d42/f64 [0,4194304] 0 2024-02-19T19:04:56.803 INFO:tasks.workunit.client.1.smithi132.stdout:3/521: stat d8/d13/d2b/d69/d92/d34/d97 0 2024-02-19T19:04:56.806 INFO:tasks.workunit.client.1.smithi132.stdout:9/643: link d12/d8d/d3a/d40/f66 d12/fda 0 2024-02-19T19:04:56.809 INFO:tasks.workunit.client.1.smithi132.stdout:8/597: creat d4/dc/d42/fcb x:0 0 0 2024-02-19T19:04:56.810 INFO:tasks.workunit.client.1.smithi132.stdout:8/598: readlink d4/dc/d1b/l60 0 2024-02-19T19:04:56.813 INFO:tasks.workunit.client.1.smithi132.stdout:8/599: mkdir d4/d2d/dcc 0 2024-02-19T19:04:56.814 INFO:tasks.workunit.client.1.smithi132.stdout:9/644: symlink d12/d2a/d88/db8/ldb 0 2024-02-19T19:04:56.827 INFO:tasks.workunit.client.1.smithi132.stdout:1/481: sync 2024-02-19T19:04:56.827 INFO:tasks.workunit.client.1.smithi132.stdout:4/504: sync 2024-02-19T19:04:56.827 INFO:tasks.workunit.client.1.smithi132.stdout:9/645: read d12/d8d/d3a/d40/d4e/f67 [3326570,42805] 0 2024-02-19T19:04:56.828 INFO:tasks.workunit.client.1.smithi132.stdout:5/593: dwrite d8/d1a/d35/f74 [4194304,4194304] 0 2024-02-19T19:04:56.829 INFO:tasks.workunit.client.1.smithi132.stdout:8/600: rename d4/dc/d1b/d38/d69/fc3 to d4/d2d/dcc/fcd 0 2024-02-19T19:04:56.837 INFO:tasks.workunit.client.1.smithi132.stdout:9/646: mkdir d12/d8d/d3a/ddc 0 2024-02-19T19:04:56.839 INFO:tasks.workunit.client.1.smithi132.stdout:9/647: chown d12/d2a/d88/db8 100 1 2024-02-19T19:04:56.840 INFO:tasks.workunit.client.1.smithi132.stdout:4/505: dwrite d4/d27/d31/d71/d14/d24/d49/f83 [0,4194304] 0 2024-02-19T19:04:56.843 INFO:tasks.workunit.client.1.smithi132.stdout:1/482: fdatasync d1/d1e/d77/d2c/d62/d6b/d85/f94 0 2024-02-19T19:04:56.844 INFO:tasks.workunit.client.1.smithi132.stdout:8/601: dwrite d4/dc/d1b/d38/d69/d74/d93/faa [0,4194304] 0 2024-02-19T19:04:56.847 INFO:tasks.workunit.client.1.smithi132.stdout:5/594: creat d8/da/d5a/d7b/d96/d9e/fc9 x:0 0 0 2024-02-19T19:04:56.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:56 smithi088 ceph-mon[88643]: Standby manager daemon smithi132.psllsa restarted 2024-02-19T19:04:56.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:56 smithi088 ceph-mon[88643]: Standby manager daemon smithi132.psllsa started 2024-02-19T19:04:56.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:56 smithi088 ceph-mon[88643]: from='mgr.? 172.21.15.132:0/1523056338' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/crt"}]: dispatch 2024-02-19T19:04:56.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:56 smithi088 ceph-mon[88643]: from='mgr.? 172.21.15.132:0/1523056338' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-19T19:04:56.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:56 smithi088 ceph-mon[88643]: from='mgr.? 172.21.15.132:0/1523056338' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi132.psllsa/key"}]: dispatch 2024-02-19T19:04:56.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:56 smithi088 ceph-mon[88643]: from='mgr.? 172.21.15.132:0/1523056338' entity='mgr.smithi132.psllsa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-19T19:04:56.853 INFO:tasks.workunit.client.1.smithi132.stdout:5/595: dread - d8/da/db/f98 zero size 2024-02-19T19:04:56.853 INFO:tasks.workunit.client.1.smithi132.stdout:9/648: creat d12/d2a/d88/db8/fdd x:0 0 0 2024-02-19T19:04:56.856 INFO:tasks.workunit.client.1.smithi132.stdout:1/483: mkdir d1/d95/d97 0 2024-02-19T19:04:56.856 INFO:tasks.workunit.client.1.smithi132.stdout:3/522: dread d8/d93/f7c [0,4194304] 0 2024-02-19T19:04:56.857 INFO:tasks.workunit.client.1.smithi132.stdout:9/649: write fb [5128491,58134] 0 2024-02-19T19:04:56.859 INFO:tasks.workunit.client.1.smithi132.stdout:4/506: dwrite d4/d19/f7a [0,4194304] 0 2024-02-19T19:04:56.861 INFO:tasks.workunit.client.1.smithi132.stdout:5/596: rmdir d8/d2e 39 2024-02-19T19:04:56.865 INFO:tasks.workunit.client.1.smithi132.stdout:8/602: dwrite d4/dc/d42/f6f [0,4194304] 0 2024-02-19T19:04:56.867 INFO:tasks.workunit.client.1.smithi132.stdout:4/507: symlink d4/d27/d31/d71/d37/d66/d98/lb2 0 2024-02-19T19:04:56.869 INFO:tasks.workunit.client.1.smithi132.stdout:9/650: fdatasync d12/d8d/fc0 0 2024-02-19T19:04:56.871 INFO:tasks.workunit.client.1.smithi132.stdout:5/597: dwrite d8/d1a/f6c [4194304,4194304] 0 2024-02-19T19:04:56.872 INFO:tasks.workunit.client.1.smithi132.stdout:5/598: read - d8/d1a/f8b zero size 2024-02-19T19:04:56.873 INFO:tasks.workunit.client.1.smithi132.stdout:4/508: write d4/d27/d31/d71/d14/d24/f3b [2994120,4392] 0 2024-02-19T19:04:56.879 INFO:tasks.workunit.client.1.smithi132.stdout:3/523: symlink d8/d13/d2b/la0 0 2024-02-19T19:04:56.879 INFO:tasks.workunit.client.1.smithi132.stdout:8/603: mkdir d4/dc/d1b/d38/d69/d74/dce 0 2024-02-19T19:04:56.885 INFO:tasks.workunit.client.1.smithi132.stdout:3/524: unlink d8/d13/d2b/l32 0 2024-02-19T19:04:56.887 INFO:tasks.workunit.client.1.smithi132.stdout:4/509: dwrite d4/d27/d31/d71/d37/f74 [0,4194304] 0 2024-02-19T19:04:56.888 INFO:tasks.workunit.client.1.smithi132.stdout:8/604: rename d4/dc/d1b/fad to d4/dc/d1b/d8f/daf/fcf 0 2024-02-19T19:04:56.888 INFO:tasks.workunit.client.1.smithi132.stdout:9/651: dwrite d12/fae [0,4194304] 0 2024-02-19T19:04:56.888 INFO:tasks.workunit.client.1.smithi132.stdout:3/525: chown d8/d13/d2b/d69/d7e/l6b 398 1 2024-02-19T19:04:56.890 INFO:tasks.workunit.client.1.smithi132.stdout:4/510: readlink d4/d12/d16/l77 0 2024-02-19T19:04:56.894 INFO:tasks.workunit.client.1.smithi132.stdout:4/511: write d4/d27/d7f/fa9 [759500,496] 0 2024-02-19T19:04:56.902 INFO:tasks.workunit.client.1.smithi132.stdout:4/512: write d4/d27/d31/d32/faf [643220,70275] 0 2024-02-19T19:04:56.904 INFO:tasks.workunit.client.1.smithi132.stdout:3/526: dread d8/d13/d2b/d60/f9b [0,4194304] 0 2024-02-19T19:04:56.906 INFO:tasks.workunit.client.1.smithi132.stdout:8/605: rename d4/dc/d1b/d20/d62/d91/l7d to d4/dc/d1b/d20/d23/da3/ld0 0 2024-02-19T19:04:56.907 INFO:tasks.workunit.client.1.smithi132.stdout:9/652: dwrite d12/d1b/f50 [0,4194304] 0 2024-02-19T19:04:56.908 INFO:tasks.workunit.client.1.smithi132.stdout:5/599: dwrite f5 [12582912,4194304] 0 2024-02-19T19:04:56.909 INFO:tasks.workunit.client.1.smithi132.stdout:1/484: dwrite d1/f1b [0,4194304] 0 2024-02-19T19:04:56.915 INFO:tasks.workunit.client.1.smithi132.stdout:4/513: rename d4/d19/d21/c86 to d4/d27/d73/cb3 0 2024-02-19T19:04:56.917 INFO:tasks.workunit.client.1.smithi132.stdout:4/514: chown d4/d4d/d87/c88 200295773 1 2024-02-19T19:04:56.920 INFO:tasks.workunit.client.1.smithi132.stdout:8/606: dread d4/dc/d1b/d38/d69/d74/d93/faa [0,4194304] 0 2024-02-19T19:04:56.921 INFO:tasks.workunit.client.1.smithi132.stdout:4/515: truncate d4/d12/f2c 2173404 0 2024-02-19T19:04:56.922 INFO:tasks.workunit.client.1.smithi132.stdout:5/600: dwrite d8/d1a/f6a [0,4194304] 0 2024-02-19T19:04:56.922 INFO:tasks.workunit.client.1.smithi132.stdout:1/485: readlink d1/d1e/d77/d2c/l66 0 2024-02-19T19:04:56.923 INFO:tasks.workunit.client.1.smithi132.stdout:8/607: write d4/dc/fa9 [544944,98312] 0 2024-02-19T19:04:56.924 INFO:tasks.workunit.client.1.smithi132.stdout:4/516: write d4/d27/d31/d71/d37/f6a [668541,21709] 0 2024-02-19T19:04:56.926 INFO:tasks.workunit.client.1.smithi132.stdout:1/486: chown d1/d1e/d77/c3e 12274 1 2024-02-19T19:04:56.932 INFO:tasks.workunit.client.1.smithi132.stdout:9/653: rename fa to d12/d2a/d4c/dad/fde 0 2024-02-19T19:04:56.935 INFO:tasks.workunit.client.1.smithi132.stdout:8/608: dwrite d4/dc/d1b/d54/d5e/f63 [0,4194304] 0 2024-02-19T19:04:56.940 INFO:tasks.workunit.client.1.smithi132.stdout:1/487: creat d1/d1e/d77/d2c/d62/d6b/d85/f98 x:0 0 0 2024-02-19T19:04:56.942 INFO:tasks.workunit.client.1.smithi132.stdout:5/601: fsync d8/d23/d3b/f65 0 2024-02-19T19:04:56.942 INFO:tasks.workunit.client.1.smithi132.stdout:3/527: link d8/d13/d2b/d69/d92/d34/d42/d6c/l88 d8/d13/d2f/la1 0 2024-02-19T19:04:56.943 INFO:tasks.workunit.client.1.smithi132.stdout:4/517: creat d4/d12/d9f/fb4 x:0 0 0 2024-02-19T19:04:56.944 INFO:tasks.workunit.client.1.smithi132.stdout:1/488: write d1/f4a [7980971,77456] 0 2024-02-19T19:04:56.947 INFO:tasks.workunit.client.1.smithi132.stdout:9/654: creat d12/d8d/d3a/ddc/fdf x:0 0 0 2024-02-19T19:04:56.947 INFO:tasks.workunit.client.1.smithi132.stdout:3/528: truncate d8/d13/d2b/d69/d92/d34/f81 338219 0 2024-02-19T19:04:56.948 INFO:tasks.workunit.client.1.smithi132.stdout:5/602: mknod d8/d1a/d35/cca 0 2024-02-19T19:04:56.950 INFO:tasks.workunit.client.1.smithi132.stdout:8/609: truncate d4/dc/d1b/d38/d69/d74/d93/faa 1190228 0 2024-02-19T19:04:56.950 INFO:tasks.workunit.client.1.smithi132.stdout:3/529: write d8/d13/d2b/f33 [2576138,16166] 0 2024-02-19T19:04:56.952 INFO:tasks.workunit.client.1.smithi132.stdout:3/530: chown c6 1336 1 2024-02-19T19:04:56.952 INFO:tasks.workunit.client.1.smithi132.stdout:8/610: stat d4/dc/d1b/cc7 0 2024-02-19T19:04:56.953 INFO:tasks.workunit.client.1.smithi132.stdout:4/518: dread d4/d12/d48/f7c [0,4194304] 0 2024-02-19T19:04:56.960 INFO:tasks.workunit.client.1.smithi132.stdout:4/519: unlink d4/d27/d31/d32/f63 0 2024-02-19T19:04:56.961 INFO:tasks.workunit.client.1.smithi132.stdout:1/489: dwrite d1/d1e/f5b [0,4194304] 0 2024-02-19T19:04:56.961 INFO:tasks.workunit.client.1.smithi132.stdout:9/655: dwrite d12/d2a/d4c/dad/fde [0,4194304] 0 2024-02-19T19:04:56.963 INFO:tasks.workunit.client.1.smithi132.stdout:5/603: creat d8/da/d5a/fcb x:0 0 0 2024-02-19T19:04:56.964 INFO:tasks.workunit.client.1.smithi132.stdout:1/490: stat d1/d1e/d77/d2c/d62/f87 0 2024-02-19T19:04:56.969 INFO:tasks.workunit.client.1.smithi132.stdout:0/519: sync 2024-02-19T19:04:56.969 INFO:tasks.workunit.client.1.smithi132.stdout:3/531: mkdir d8/d13/d2b/d69/d7e/d9f/da2 0 2024-02-19T19:04:56.970 INFO:tasks.workunit.client.1.smithi132.stdout:4/520: unlink d4/d27/d31/d71/l4f 0 2024-02-19T19:04:56.972 INFO:tasks.workunit.client.1.smithi132.stdout:0/520: write d2/dc/df/d15/f46 [1580885,59150] 0 2024-02-19T19:04:56.973 INFO:tasks.workunit.client.1.smithi132.stdout:4/521: chown d4/d27/l8b 1024808 1 2024-02-19T19:04:56.974 INFO:tasks.workunit.client.1.smithi132.stdout:5/604: mknod d8/ccc 0 2024-02-19T19:04:56.985 INFO:tasks.workunit.client.1.smithi132.stdout:8/611: getdents d4/dc/d1b/d38/d69/d88 0 2024-02-19T19:04:56.988 INFO:tasks.workunit.client.1.smithi132.stdout:4/522: mknod d4/d4d/cb5 0 2024-02-19T19:04:56.988 INFO:tasks.workunit.client.1.smithi132.stdout:1/491: dwrite d1/f18 [4194304,4194304] 0 2024-02-19T19:04:56.990 INFO:tasks.workunit.client.1.smithi132.stdout:9/656: getdents d12/d8d/d3a/d85/d87 0 2024-02-19T19:04:56.992 INFO:tasks.workunit.client.1.smithi132.stdout:4/523: stat d4/l6 0 2024-02-19T19:04:56.992 INFO:tasks.workunit.client.1.smithi132.stdout:3/532: dwrite d8/d13/d2b/d69/d92/d34/f89 [0,4194304] 0 2024-02-19T19:04:56.998 INFO:tasks.workunit.client.1.smithi132.stdout:0/521: dwrite d2/dc/df/d15/d2f/d47/f96 [4194304,4194304] 0 2024-02-19T19:04:57.001 INFO:tasks.workunit.client.1.smithi132.stdout:8/612: dwrite d4/f29 [0,4194304] 0 2024-02-19T19:04:57.002 INFO:tasks.workunit.client.1.smithi132.stdout:5/605: rename d8/d2e/l7c to d8/d23/d29/lcd 0 2024-02-19T19:04:57.006 INFO:tasks.workunit.client.1.smithi132.stdout:5/606: truncate d8/da/d5a/d7b/d96/d9e/fc9 658785 0 2024-02-19T19:04:57.011 INFO:tasks.workunit.client.1.smithi132.stdout:1/492: mkdir d1/d1e/d77/d4e/d99 0 2024-02-19T19:04:57.012 INFO:tasks.workunit.client.1.smithi132.stdout:1/493: write d1/d1e/d77/f3c [1142267,99656] 0 2024-02-19T19:04:57.012 INFO:tasks.workunit.client.1.smithi132.stdout:7/616: sync 2024-02-19T19:04:57.013 INFO:tasks.workunit.client.1.smithi132.stdout:7/617: chown l9 14689719 1 2024-02-19T19:04:57.013 INFO:tasks.workunit.client.1.smithi132.stdout:7/618: stat d1b/d1e/f43 0 2024-02-19T19:04:57.014 INFO:tasks.workunit.client.1.smithi132.stdout:7/619: fsync d1b/d1e/d28/f31 0 2024-02-19T19:04:57.019 INFO:tasks.workunit.client.1.smithi132.stdout:8/613: symlink d4/dc/d1b/d33/ld1 0 2024-02-19T19:04:57.023 INFO:tasks.workunit.client.1.smithi132.stdout:7/620: dwrite d1b/d1e/d2b/fb4 [0,4194304] 0 2024-02-19T19:04:57.026 INFO:tasks.workunit.client.1.smithi132.stdout:1/494: unlink d1/d1e/d77/d2c/d62/d6b/d85/f98 0 2024-02-19T19:04:57.028 INFO:tasks.workunit.client.1.smithi132.stdout:9/657: symlink d12/d8d/d3a/d40/d4e/le0 0 2024-02-19T19:04:57.030 INFO:tasks.workunit.client.1.smithi132.stdout:0/522: mkdir d2/dc/d4a/dae 0 2024-02-19T19:04:57.035 INFO:tasks.workunit.client.1.smithi132.stdout:9/658: mknod d12/d2a/d4c/dad/ce1 0 2024-02-19T19:04:57.037 INFO:tasks.workunit.client.1.smithi132.stdout:8/614: symlink d4/dc/d1b/d20/d23/d68/ld2 0 2024-02-19T19:04:57.039 INFO:tasks.workunit.client.1.smithi132.stdout:7/621: truncate d1b/d46/f4d 6517631 0 2024-02-19T19:04:57.040 INFO:tasks.workunit.client.1.smithi132.stdout:0/523: dread d2/f53 [0,4194304] 0 2024-02-19T19:04:57.042 INFO:tasks.workunit.client.1.smithi132.stdout:5/607: getdents d8 0 2024-02-19T19:04:57.042 INFO:tasks.workunit.client.1.smithi132.stdout:7/622: chown d1b/d1e/d2b/d8f/da5/fa8 45 1 2024-02-19T19:04:57.043 INFO:tasks.workunit.client.1.smithi132.stdout:4/524: creat d4/d27/d31/d32/fb6 x:0 0 0 2024-02-19T19:04:57.044 INFO:tasks.workunit.client.1.smithi132.stdout:9/659: creat d12/d2a/d4c/d61/fe2 x:0 0 0 2024-02-19T19:04:57.044 INFO:tasks.workunit.client.1.smithi132.stdout:4/525: chown d4/d12 3346 1 2024-02-19T19:04:57.046 INFO:tasks.workunit.client.1.smithi132.stdout:8/615: creat d4/dc/d1b/fd3 x:0 0 0 2024-02-19T19:04:57.046 INFO:tasks.workunit.client.1.smithi132.stdout:1/495: symlink d1/d1e/d6a/l9a 0 2024-02-19T19:04:57.047 INFO:tasks.workunit.client.1.smithi132.stdout:5/608: dread d8/d1a/f9d [0,4194304] 0 2024-02-19T19:04:57.047 INFO:tasks.workunit.client.1.smithi132.stdout:3/533: dwrite d8/d93/f6f [0,4194304] 0 2024-02-19T19:04:57.049 INFO:tasks.workunit.client.1.smithi132.stdout:8/616: write d4/dc/d42/f56 [298647,111363] 0 2024-02-19T19:04:57.049 INFO:tasks.workunit.client.1.smithi132.stdout:0/524: creat d2/dc/df/d15/d2f/d69/faf x:0 0 0 2024-02-19T19:04:57.052 INFO:tasks.workunit.client.1.smithi132.stdout:8/617: write d4/f16 [2995443,56867] 0 2024-02-19T19:04:57.053 INFO:tasks.workunit.client.1.smithi132.stdout:7/623: fdatasync d1b/d1e/d2b/d62/f67 0 2024-02-19T19:04:57.054 INFO:tasks.workunit.client.1.smithi132.stdout:0/525: write d2/dc/df/d78/fa1 [410930,3552] 0 2024-02-19T19:04:57.054 INFO:tasks.workunit.client.1.smithi132.stdout:1/496: truncate d1/d1e/d77/d52/d64/f91 230086 0 2024-02-19T19:04:57.056 INFO:tasks.workunit.client.1.smithi132.stdout:7/624: write d1b/d1e/d28/f31 [7031602,12671] 0 2024-02-19T19:04:57.057 INFO:tasks.workunit.client.1.smithi132.stdout:5/609: dwrite d8/d1a/d35/f74 [4194304,4194304] 0 2024-02-19T19:04:57.062 INFO:tasks.workunit.client.1.smithi132.stdout:1/497: dwrite d1/d1e/d77/d4e/f5f [4194304,4194304] 0 2024-02-19T19:04:57.064 INFO:tasks.workunit.client.1.smithi132.stdout:1/498: truncate d1/f42 3468416 0 2024-02-19T19:04:57.064 INFO:tasks.workunit.client.1.smithi132.stdout:4/526: symlink d4/d19/d21/lb7 0 2024-02-19T19:04:57.064 INFO:tasks.workunit.client.1.smithi132.stdout:1/499: fdatasync d1/d1e/d77/f73 0 2024-02-19T19:04:57.065 INFO:tasks.workunit.client.1.smithi132.stdout:9/660: creat d12/d8d/d3a/d40/d4e/d77/d89/dc4/fe3 x:0 0 0 2024-02-19T19:04:57.066 INFO:tasks.workunit.client.1.smithi132.stdout:8/618: symlink d4/dc/d1b/d20/d62/d91/ld4 0 2024-02-19T19:04:57.068 INFO:tasks.workunit.client.1.smithi132.stdout:6/488: sync 2024-02-19T19:04:57.071 INFO:tasks.workunit.client.1.smithi132.stdout:0/526: fsync d2/f53 0 2024-02-19T19:04:57.072 INFO:tasks.workunit.client.1.smithi132.stdout:0/527: chown d2/dc/d4a 1 1 2024-02-19T19:04:57.082 INFO:tasks.workunit.client.1.smithi132.stdout:5/610: rename d8/d23/d29/dc5 to d8/d2e/d71/dce 0 2024-02-19T19:04:57.086 INFO:tasks.workunit.client.1.smithi132.stdout:4/527: write d4/d27/d31/d32/d51/d72/f85 [471,5651] 0 2024-02-19T19:04:57.087 INFO:tasks.workunit.client.1.smithi132.stdout:1/500: symlink d1/d1e/d77/d52/l9b 0 2024-02-19T19:04:57.089 INFO:tasks.workunit.client.1.smithi132.stdout:9/661: creat d12/d8d/d3a/d85/d87/fe4 x:0 0 0 2024-02-19T19:04:57.091 INFO:tasks.workunit.client.1.smithi132.stdout:8/619: rmdir d4/dc/d1b/d38/d69/d74 39 2024-02-19T19:04:57.097 INFO:tasks.workunit.client.1.smithi132.stdout:3/534: creat d8/d13/d2b/d69/d92/d34/fa3 x:0 0 0 2024-02-19T19:04:57.098 INFO:tasks.workunit.client.1.smithi132.stdout:7/625: getdents d1b/d1e/d28/d5e/d99 0 2024-02-19T19:04:57.101 INFO:tasks.workunit.client.1.smithi132.stdout:5/611: rename d8/d2e/l86 to d8/da/d5a/d7b/d96/lcf 0 2024-02-19T19:04:57.103 INFO:tasks.workunit.client.1.smithi132.stdout:5/612: write d8/da/d5a/d7b/f99 [1312591,62007] 0 2024-02-19T19:04:57.103 INFO:tasks.workunit.client.1.smithi132.stdout:7/626: dwrite d1b/d1e/d41/f6d [0,4194304] 0 2024-02-19T19:04:57.104 INFO:tasks.workunit.client.1.smithi132.stdout:4/528: fsync d4/f38 0 2024-02-19T19:04:57.104 INFO:tasks.workunit.client.1.smithi132.stdout:5/613: write d8/d23/d29/d27/f87 [556207,24801] 0 2024-02-19T19:04:57.108 INFO:tasks.workunit.client.1.smithi132.stdout:1/501: creat d1/d1e/d77/d2c/d89/f9c x:0 0 0 2024-02-19T19:04:57.109 INFO:tasks.workunit.client.1.smithi132.stdout:1/502: chown d1/d1e/d77/d2c/l66 5447 1 2024-02-19T19:04:57.113 INFO:tasks.workunit.client.1.smithi132.stdout:1/503: read d1/d3a/f40 [279027,95127] 0 2024-02-19T19:04:57.114 INFO:tasks.workunit.client.1.smithi132.stdout:9/662: creat d12/d8d/d3a/d40/d4e/d77/d8a/fe5 x:0 0 0 2024-02-19T19:04:57.116 INFO:tasks.workunit.client.1.smithi132.stdout:8/620: mkdir d4/d2d/dd5 0 2024-02-19T19:04:57.116 INFO:tasks.workunit.client.1.smithi132.stdout:4/529: read d4/f7d [2060871,68397] 0 2024-02-19T19:04:57.117 INFO:tasks.workunit.client.1.smithi132.stdout:1/504: dread d1/d1e/d77/d4e/f6d [0,4194304] 0 2024-02-19T19:04:57.118 INFO:tasks.workunit.client.1.smithi132.stdout:6/489: write d0/f10 [620998,83309] 0 2024-02-19T19:04:57.119 INFO:tasks.workunit.client.1.smithi132.stdout:1/505: chown d1/d1e/d77/d2c/d62 6406 1 2024-02-19T19:04:57.122 INFO:tasks.workunit.client.1.smithi132.stdout:3/535: mkdir d8/d13/d2b/d69/d92/d34/d50/da4 0 2024-02-19T19:04:57.123 INFO:tasks.workunit.client.1.smithi132.stdout:3/536: dread - d8/d13/d2f/f58 zero size 2024-02-19T19:04:57.135 INFO:tasks.workunit.client.1.smithi132.stdout:5/614: creat d8/d23/d3b/fd0 x:0 0 0 2024-02-19T19:04:57.135 INFO:tasks.workunit.client.1.smithi132.stdout:4/530: dread d4/d27/d31/d32/d51/d5c/fa8 [0,4194304] 0 2024-02-19T19:04:57.136 INFO:tasks.workunit.client.1.smithi132.stdout:9/663: rename d12/d8d/d3a/d40/d4e/le0 to d12/d2a/d4c/d61/le6 0 2024-02-19T19:04:57.138 INFO:tasks.workunit.client.1.smithi132.stdout:7/627: write d1b/d1e/d41/f6a [2268405,110963] 0 2024-02-19T19:04:57.138 INFO:tasks.workunit.client.1.smithi132.stdout:9/664: truncate d12/d8d/d3a/d40/d4e/d77/d8a/d82/fa9 392252 0 2024-02-19T19:04:57.142 INFO:tasks.workunit.client.1.smithi132.stdout:3/537: dread d8/d38/f4b [0,4194304] 0 2024-02-19T19:04:57.147 INFO:tasks.workunit.client.1.smithi132.stdout:0/528: getdents d2/dc/df/d15/d34/d84 0 2024-02-19T19:04:57.148 INFO:tasks.workunit.client.1.smithi132.stdout:3/538: dwrite d8/d13/d19/f2d [4194304,4194304] 0 2024-02-19T19:04:57.149 INFO:tasks.workunit.client.1.smithi132.stdout:4/531: dwrite d4/d12/f30 [0,4194304] 0 2024-02-19T19:04:57.150 INFO:tasks.workunit.client.1.smithi132.stdout:3/539: write d8/fd [174917,111178] 0 2024-02-19T19:04:57.150 INFO:tasks.workunit.client.1.smithi132.stdout:4/532: read - d4/d12/d28/f81 zero size 2024-02-19T19:04:57.152 INFO:tasks.workunit.client.1.smithi132.stdout:1/506: link d1/d1e/d77/d52/f65 d1/d1e/d77/d78/f9d 0 2024-02-19T19:04:57.152 INFO:tasks.workunit.client.1.smithi132.stdout:7/628: mknod d1b/d1e/d2b/cc1 0 2024-02-19T19:04:57.156 INFO:tasks.workunit.client.1.smithi132.stdout:3/540: dread d8/d13/d2b/d69/d92/d34/d50/f5b [0,4194304] 0 2024-02-19T19:04:57.159 INFO:tasks.workunit.client.1.smithi132.stdout:8/621: dwrite d4/dc/f3f [0,4194304] 0 2024-02-19T19:04:57.159 INFO:tasks.workunit.client.1.smithi132.stdout:5/615: mknod d8/d2e/d71/da6/cd1 0 2024-02-19T19:04:57.163 INFO:tasks.workunit.client.1.smithi132.stdout:1/507: read d1/d1e/d77/d2c/f59 [4109711,78487] 0 2024-02-19T19:04:57.164 INFO:tasks.workunit.client.1.smithi132.stdout:1/508: write d1/d3a/f30 [849597,64492] 0 2024-02-19T19:04:57.165 INFO:tasks.workunit.client.1.smithi132.stdout:7/629: mknod d1b/d1e/d2b/d8f/dae/cc2 0 2024-02-19T19:04:57.166 INFO:tasks.workunit.client.1.smithi132.stdout:9/665: rename d12/d2a/d88/cbd to d12/d3e/ce7 0 2024-02-19T19:04:57.167 INFO:tasks.workunit.client.1.smithi132.stdout:3/541: creat d8/d13/d2b/d69/d92/d34/d50/fa5 x:0 0 0 2024-02-19T19:04:57.168 INFO:tasks.workunit.client.1.smithi132.stdout:6/490: creat d0/db/d25/d3a/d3d/fa8 x:0 0 0 2024-02-19T19:04:57.169 INFO:tasks.workunit.client.1.smithi132.stdout:3/542: chown d8/d13/d2b/d69/d92/l9c 2 1 2024-02-19T19:04:57.170 INFO:tasks.workunit.client.1.smithi132.stdout:3/543: chown d8/d13/d2b/f2c 4020930 1 2024-02-19T19:04:57.171 INFO:tasks.workunit.client.1.smithi132.stdout:1/509: dread - d1/d1e/d77/d2c/f7d zero size 2024-02-19T19:04:57.173 INFO:tasks.workunit.client.1.smithi132.stdout:5/616: creat d8/d2e/d71/dbc/fd2 x:0 0 0 2024-02-19T19:04:57.175 INFO:tasks.workunit.client.1.smithi132.stdout:9/666: rename d12/d2a/d4c/dad/ce1 to d12/d2a/d4c/d61/ce8 0 2024-02-19T19:04:57.180 INFO:tasks.workunit.client.1.smithi132.stdout:8/622: rename d4/dc/d1b/d33/l9a to d4/dc/d1b/d38/d69/d74/dce/ld6 0 2024-02-19T19:04:57.180 INFO:tasks.workunit.client.1.smithi132.stdout:1/510: mknod d1/d1e/d77/d4e/d99/c9e 0 2024-02-19T19:04:57.181 INFO:tasks.workunit.client.1.smithi132.stdout:8/623: chown d4/dc/d42/f56 16 1 2024-02-19T19:04:57.181 INFO:tasks.workunit.client.1.smithi132.stdout:1/511: write d1/d1e/d77/f73 [824961,8024] 0 2024-02-19T19:04:57.182 INFO:tasks.workunit.client.1.smithi132.stdout:6/491: dread d0/db/d25/d3a/d5d/d8f/d46/d5a/f1d [0,4194304] 0 2024-02-19T19:04:57.182 INFO:tasks.workunit.client.1.smithi132.stdout:9/667: symlink d12/le9 0 2024-02-19T19:04:57.185 INFO:tasks.workunit.client.1.smithi132.stdout:6/492: write d0/db/fd [1786784,120356] 0 2024-02-19T19:04:57.187 INFO:tasks.workunit.client.1.smithi132.stdout:5/617: dread d8/da/f3c [4194304,4194304] 0 2024-02-19T19:04:57.188 INFO:tasks.workunit.client.1.smithi132.stdout:5/618: dread - d8/d2e/f91 zero size 2024-02-19T19:04:57.189 INFO:tasks.workunit.client.1.smithi132.stdout:3/544: rename d8/d13/d2b/d69/d92/d34/d50 to d8/d13/d2b/d69/da6 0 2024-02-19T19:04:57.190 INFO:tasks.workunit.client.1.smithi132.stdout:4/533: write d4/d19/d21/f36 [4672791,50655] 0 2024-02-19T19:04:57.191 INFO:tasks.workunit.client.1.smithi132.stdout:1/512: creat d1/d1e/d77/d2c/d62/d6b/d85/f9f x:0 0 0 2024-02-19T19:04:57.192 INFO:tasks.workunit.client.1.smithi132.stdout:7/630: write d1b/d1e/d2b/d62/f67 [454531,10786] 0 2024-02-19T19:04:57.194 INFO:tasks.workunit.client.1.smithi132.stdout:9/668: write d12/d1b/f17 [4200567,11673] 0 2024-02-19T19:04:57.195 INFO:tasks.workunit.client.1.smithi132.stdout:8/624: symlink d4/dc/d1b/d20/d23/d68/ld7 0 2024-02-19T19:04:57.195 INFO:tasks.workunit.client.1.smithi132.stdout:6/493: fdatasync d0/db/d25/d3a/d4b/d7c/f91 0 2024-02-19T19:04:57.197 INFO:tasks.workunit.client.1.smithi132.stdout:5/619: dwrite d8/da/f8c [4194304,4194304] 0 2024-02-19T19:04:57.200 INFO:tasks.workunit.client.1.smithi132.stdout:5/620: readlink d8/d23/d29/l2f 0 2024-02-19T19:04:57.203 INFO:tasks.workunit.client.1.smithi132.stdout:7/631: fdatasync d1b/d1e/d2b/f95 0 2024-02-19T19:04:57.205 INFO:tasks.workunit.client.1.smithi132.stdout:8/625: dread d4/dc/d42/f56 [0,4194304] 0 2024-02-19T19:04:57.206 INFO:tasks.workunit.client.1.smithi132.stdout:4/534: dwrite d4/d27/d31/d32/f50 [0,4194304] 0 2024-02-19T19:04:57.207 INFO:tasks.workunit.client.1.smithi132.stdout:3/545: rename d8/d13/d2b/d69/d7e/l65 to d8/d13/d57/la7 0 2024-02-19T19:04:57.211 INFO:tasks.workunit.client.1.smithi132.stdout:5/621: symlink d8/da/d5a/d7b/d96/d9e/ld3 0 2024-02-19T19:04:57.214 INFO:tasks.workunit.client.1.smithi132.stdout:7/632: rmdir d1b/d1e/d41/da3/dbd 39 2024-02-19T19:04:57.220 INFO:tasks.workunit.client.1.smithi132.stdout:1/513: dwrite d1/d1e/d77/d78/f9d [0,4194304] 0 2024-02-19T19:04:57.224 INFO:tasks.workunit.client.1.smithi132.stdout:7/633: mkdir d1b/d46/d76/dc3 0 2024-02-19T19:04:57.224 INFO:tasks.workunit.client.1.smithi132.stdout:4/535: symlink d4/d27/d31/d71/lb8 0 2024-02-19T19:04:57.225 INFO:tasks.workunit.client.1.smithi132.stdout:8/626: symlink d4/dc/d1b/d54/d81/dbd/ld8 0 2024-02-19T19:04:57.226 INFO:tasks.workunit.client.1.smithi132.stdout:4/536: write d4/d19/d21/d56/f8c [269636,34583] 0 2024-02-19T19:04:57.228 INFO:tasks.workunit.client.1.smithi132.stdout:2/588: sync 2024-02-19T19:04:57.229 INFO:tasks.workunit.client.1.smithi132.stdout:3/546: rename d8/d13/d19/l9e to d8/d13/d2f/la8 0 2024-02-19T19:04:57.230 INFO:tasks.workunit.client.1.smithi132.stdout:9/669: getdents d12/d8d/d3a/d40/d4e/d77/d8a/d82 0 2024-02-19T19:04:57.232 INFO:tasks.workunit.client.1.smithi132.stdout:7/634: symlink d1b/d5a/d63/d98/lc4 0 2024-02-19T19:04:57.233 INFO:tasks.workunit.client.1.smithi132.stdout:8/627: creat d4/dc/d1b/d20/d23/da3/fd9 x:0 0 0 2024-02-19T19:04:57.234 INFO:tasks.workunit.client.1.smithi132.stdout:6/494: link d0/db/d25/c80 d0/db/d25/d3a/d3d/ca9 0 2024-02-19T19:04:57.235 INFO:tasks.workunit.client.1.smithi132.stdout:4/537: write d4/f7d [2145024,126593] 0 2024-02-19T19:04:57.235 INFO:tasks.workunit.client.1.smithi132.stdout:2/589: symlink d0/d19/d6a/lba 0 2024-02-19T19:04:57.235 INFO:tasks.workunit.client.1.smithi132.stdout:4/538: stat d4/f2d 0 2024-02-19T19:04:57.236 INFO:tasks.workunit.client.1.smithi132.stdout:2/590: chown d0/d24/daf 13 1 2024-02-19T19:04:57.236 INFO:tasks.workunit.client.1.smithi132.stdout:6/495: fsync d0/db/d25/d3a/d5d/d8f/d46/d5a/f29 0 2024-02-19T19:04:57.237 INFO:tasks.workunit.client.1.smithi132.stdout:3/547: fdatasync d8/d13/d2b/d69/d92/f8c 0 2024-02-19T19:04:57.241 INFO:tasks.workunit.client.1.smithi132.stdout:3/548: dread d8/d13/d2b/f33 [0,4194304] 0 2024-02-19T19:04:57.243 INFO:tasks.workunit.client.1.smithi132.stdout:9/670: rename d12/d8d/l7c to d12/d8d/d3a/d40/d4e/d77/d89/dc4/lea 0 2024-02-19T19:04:57.247 INFO:tasks.workunit.client.1.smithi132.stdout:8/628: symlink d4/dc/d1b/d20/d23/d68/d79/db4/lda 0 2024-02-19T19:04:57.248 INFO:tasks.workunit.client.1.smithi132.stdout:8/629: fsync d4/dc/d1b/d20/f9f 0 2024-02-19T19:04:57.251 INFO:tasks.workunit.client.1.smithi132.stdout:5/622: truncate d8/d2e/fbe 600545 0 2024-02-19T19:04:57.254 INFO:tasks.workunit.client.1.smithi132.stdout:4/539: rename d4/d19/d21/d56/c84 to d4/d19/daa/cb9 0 2024-02-19T19:04:57.257 INFO:tasks.workunit.client.1.smithi132.stdout:9/671: write d12/d8d/d3a/f81 [267491,105384] 0 2024-02-19T19:04:57.257 INFO:tasks.workunit.client.1.smithi132.stdout:1/514: truncate d1/d61/f7c 2343387 0 2024-02-19T19:04:57.259 INFO:tasks.workunit.client.1.smithi132.stdout:1/515: read d1/d1e/d77/d4e/f5f [8031889,105213] 0 2024-02-19T19:04:57.260 INFO:tasks.workunit.client.1.smithi132.stdout:8/630: dread d4/dc/d1b/d33/f8c [0,4194304] 0 2024-02-19T19:04:57.262 INFO:tasks.workunit.client.1.smithi132.stdout:6/496: link d0/db/d25/d3a/d3d/d69/f55 d0/db/d25/d3a/d3d/d69/faa 0 2024-02-19T19:04:57.264 INFO:tasks.workunit.client.1.smithi132.stdout:7/635: getdents d1b 0 2024-02-19T19:04:57.265 INFO:tasks.workunit.client.1.smithi132.stdout:4/540: mknod d4/d12/d16/cba 0 2024-02-19T19:04:57.265 INFO:tasks.workunit.client.1.smithi132.stdout:2/591: dwrite d0/d4/d11/f32 [0,4194304] 0 2024-02-19T19:04:57.267 INFO:tasks.workunit.client.1.smithi132.stdout:9/672: dwrite d12/d1b/f3f [0,4194304] 0 2024-02-19T19:04:57.268 INFO:tasks.workunit.client.1.smithi132.stdout:6/497: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f18 [335397,124807] 0 2024-02-19T19:04:57.273 INFO:tasks.workunit.client.1.smithi132.stdout:6/498: chown d0/db/d25/d3a/d5d/d8f/f42 193182 1 2024-02-19T19:04:57.274 INFO:tasks.workunit.client.1.smithi132.stdout:0/529: sync 2024-02-19T19:04:57.274 INFO:tasks.workunit.client.1.smithi132.stdout:7/636: mknod d1b/d1e/cc5 0 2024-02-19T19:04:57.275 INFO:tasks.workunit.client.1.smithi132.stdout:8/631: dwrite d4/dc/d1b/d20/d62/d91/f21 [0,4194304] 0 2024-02-19T19:04:57.275 INFO:tasks.workunit.client.1.smithi132.stdout:2/592: stat d0/c13 0 2024-02-19T19:04:57.279 INFO:tasks.workunit.client.1.smithi132.stdout:3/549: rename d8/d13/d2b/d69/d92/d34/d42/f64 to d8/d13/d2b/fa9 0 2024-02-19T19:04:57.279 INFO:tasks.workunit.client.1.smithi132.stdout:9/673: mkdir d12/d8d/d3a/d85/d87/dc8/deb 0 2024-02-19T19:04:57.282 INFO:tasks.workunit.client.1.smithi132.stdout:2/593: write d0/d24/d58/d53/dab/d8a/da5/db1/fb6 [768804,45033] 0 2024-02-19T19:04:57.285 INFO:tasks.workunit.client.1.smithi132.stdout:8/632: write d4/dc/f2c [710554,58209] 0 2024-02-19T19:04:57.288 INFO:tasks.workunit.client.1.smithi132.stdout:9/674: write d12/d3e/fd4 [428006,82392] 0 2024-02-19T19:04:57.288 INFO:tasks.workunit.client.1.smithi132.stdout:1/516: getdents d1/d1e/d77/d2c/d62 0 2024-02-19T19:04:57.290 INFO:tasks.workunit.client.1.smithi132.stdout:6/499: dwrite d0/db/d25/d3a/d3d/fa4 [0,4194304] 0 2024-02-19T19:04:57.291 INFO:tasks.workunit.client.1.smithi132.stdout:3/550: readlink d8/d13/d19/l43 0 2024-02-19T19:04:57.294 INFO:tasks.workunit.client.1.smithi132.stdout:5/623: rename d8/d2e/d71/cb1 to d8/da/d5a/d7b/cd4 0 2024-02-19T19:04:57.296 INFO:tasks.workunit.client.1.smithi132.stdout:0/530: mknod d2/dc/df/d15/cb0 0 2024-02-19T19:04:57.297 INFO:tasks.workunit.client.1.smithi132.stdout:8/633: mkdir d4/dc/d1b/d38/ddb 0 2024-02-19T19:04:57.298 INFO:tasks.workunit.client.1.smithi132.stdout:1/517: stat d1/d1e/f2a 0 2024-02-19T19:04:57.302 INFO:tasks.workunit.client.1.smithi132.stdout:6/500: unlink d0/dc/f75 0 2024-02-19T19:04:57.302 INFO:tasks.workunit.client.1.smithi132.stdout:2/594: dwrite d0/d3c/d3f/f81 [4194304,4194304] 0 2024-02-19T19:04:57.305 INFO:tasks.workunit.client.1.smithi132.stdout:9/675: dwrite d12/d2a/d4c/d61/f7a [0,4194304] 0 2024-02-19T19:04:57.305 INFO:tasks.workunit.client.1.smithi132.stdout:4/541: rename d4/l3d to d4/d12/d9f/lbb 0 2024-02-19T19:04:57.307 INFO:tasks.workunit.client.1.smithi132.stdout:5/624: creat d8/da/d5a/d84/fd5 x:0 0 0 2024-02-19T19:04:57.307 INFO:tasks.workunit.client.1.smithi132.stdout:2/595: stat d0/d19/d1e/f52 0 2024-02-19T19:04:57.309 INFO:tasks.workunit.client.1.smithi132.stdout:9/676: dread - d12/d3e/f9f zero size 2024-02-19T19:04:57.310 INFO:tasks.workunit.client.1.smithi132.stdout:6/501: truncate d0/d8b/f84 4326307 0 2024-02-19T19:04:57.310 INFO:tasks.workunit.client.1.smithi132.stdout:7/637: dread d1b/d1e/d28/f3b [0,4194304] 0 2024-02-19T19:04:57.313 INFO:tasks.workunit.client.1.smithi132.stdout:0/531: dwrite d2/dc/f6d [0,4194304] 0 2024-02-19T19:04:57.314 INFO:tasks.workunit.client.1.smithi132.stdout:1/518: dread d1/d1e/f5b [0,4194304] 0 2024-02-19T19:04:57.321 INFO:tasks.workunit.client.1.smithi132.stdout:8/634: rename d4/dc/d1b/d38/d69/fc9 to d4/dc/d1b/d54/d5e/fdc 0 2024-02-19T19:04:57.327 INFO:tasks.workunit.client.1.smithi132.stdout:4/542: mkdir d4/d27/d31/d32/d51/d5c/dbc 0 2024-02-19T19:04:57.327 INFO:tasks.workunit.client.1.smithi132.stdout:3/551: write d8/d13/f37 [1978013,16568] 0 2024-02-19T19:04:57.331 INFO:tasks.workunit.client.1.smithi132.stdout:2/596: symlink d0/d24/d58/d53/dab/d84/lbb 0 2024-02-19T19:04:57.335 INFO:tasks.workunit.client.1.smithi132.stdout:7/638: mkdir d1b/d5a/d63/d6b/d79/dc6 0 2024-02-19T19:04:57.337 INFO:tasks.workunit.client.1.smithi132.stdout:7/639: write d1b/d46/d76/f9f [760700,37902] 0 2024-02-19T19:04:57.338 INFO:tasks.workunit.client.1.smithi132.stdout:0/532: unlink d2/dc/df/d15/l1e 0 2024-02-19T19:04:57.338 INFO:tasks.workunit.client.1.smithi132.stdout:2/597: dwrite d0/d4/d11/d2e/f6c [0,4194304] 0 2024-02-19T19:04:57.339 INFO:tasks.workunit.client.1.smithi132.stdout:7/640: chown d1b/d46/c60 1499 1 2024-02-19T19:04:57.339 INFO:tasks.workunit.client.1.smithi132.stdout:0/533: fsync d2/f1a 0 2024-02-19T19:04:57.340 INFO:tasks.workunit.client.1.smithi132.stdout:6/502: dread d0/db/d25/d3a/d5d/d8f/d46/d5a/f26 [0,4194304] 0 2024-02-19T19:04:57.340 INFO:tasks.workunit.client.1.smithi132.stdout:7/641: chown d1b/d1e/d2b/d8f/da5 1 1 2024-02-19T19:04:57.341 INFO:tasks.workunit.client.1.smithi132.stdout:5/625: rename d8/da/d5a/d84/lc3 to d8/d2e/d71/dbc/ld6 0 2024-02-19T19:04:57.341 INFO:tasks.workunit.client.1.smithi132.stdout:2/598: write d0/d19/d1e/f9d [1822465,29143] 0 2024-02-19T19:04:57.342 INFO:tasks.workunit.client.1.smithi132.stdout:3/552: fdatasync d8/d13/d2b/d60/f83 0 2024-02-19T19:04:57.345 INFO:tasks.workunit.client.1.smithi132.stdout:3/553: truncate d8/d13/d2b/d69/da6/fa5 755682 0 2024-02-19T19:04:57.347 INFO:tasks.workunit.client.1.smithi132.stdout:9/677: write d12/d8d/f29 [3811812,79159] 0 2024-02-19T19:04:57.347 INFO:tasks.workunit.client.1.smithi132.stdout:7/642: dread d1b/d1e/d2b/d62/f67 [0,4194304] 0 2024-02-19T19:04:57.349 INFO:tasks.workunit.client.1.smithi132.stdout:9/678: chown d12/d8d/d3a/d40/d4e/d77/d8a/fe5 108 1 2024-02-19T19:04:57.349 INFO:tasks.workunit.client.1.smithi132.stdout:1/519: creat d1/d95/d97/fa0 x:0 0 0 2024-02-19T19:04:57.353 INFO:tasks.workunit.client.1.smithi132.stdout:4/543: unlink d4/d12/d16/f1b 0 2024-02-19T19:04:57.355 INFO:tasks.workunit.client.1.smithi132.stdout:2/599: rename d0/d19/d1e/f9d to d0/d24/d58/fbc 0 2024-02-19T19:04:57.357 INFO:tasks.workunit.client.1.smithi132.stdout:8/635: dread d4/dc/d1b/d20/d23/d68/d7a/f95 [0,4194304] 0 2024-02-19T19:04:57.358 INFO:tasks.workunit.client.1.smithi132.stdout:1/520: mkdir d1/d1e/d77/d2c/d43/d81/da1 0 2024-02-19T19:04:57.360 INFO:tasks.workunit.client.1.smithi132.stdout:6/503: symlink d0/db/lab 0 2024-02-19T19:04:57.361 INFO:tasks.workunit.client.1.smithi132.stdout:5/626: mknod d8/d23/d29/dba/cd7 0 2024-02-19T19:04:57.366 INFO:tasks.workunit.client.1.smithi132.stdout:4/544: rename d4/d19/d21/f90 to d4/d27/d31/d32/d51/d5c/dbc/fbd 0 2024-02-19T19:04:57.367 INFO:tasks.workunit.client.1.smithi132.stdout:0/534: dread d2/dc/df/faa [0,4194304] 0 2024-02-19T19:04:57.368 INFO:tasks.workunit.client.1.smithi132.stdout:4/545: truncate d4/d27/d31/d32/d51/f97 269473 0 2024-02-19T19:04:57.368 INFO:tasks.workunit.client.1.smithi132.stdout:2/600: mkdir d0/d4/df/d40/dbd 0 2024-02-19T19:04:57.374 INFO:tasks.workunit.client.1.smithi132.stdout:1/521: chown d1/c31 0 1 2024-02-19T19:04:57.375 INFO:tasks.workunit.client.1.smithi132.stdout:1/522: truncate d1/d1e/d77/d52/d64/d75/f83 398445 0 2024-02-19T19:04:57.379 INFO:tasks.workunit.client.1.smithi132.stdout:8/636: dread d4/dc/d1b/d20/d62/d91/f36 [0,4194304] 0 2024-02-19T19:04:57.384 INFO:tasks.workunit.client.1.smithi132.stdout:0/535: mkdir d2/d3e/db1 0 2024-02-19T19:04:57.387 INFO:tasks.workunit.client.1.smithi132.stdout:4/546: rename d4/d12/d48/l60 to d4/d27/d31/d32/d51/d72/lbe 0 2024-02-19T19:04:57.387 INFO:tasks.workunit.client.1.smithi132.stdout:6/504: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f37 [401825,5948] 0 2024-02-19T19:04:57.388 INFO:tasks.workunit.client.1.smithi132.stdout:6/505: chown d0/f8a 0 1 2024-02-19T19:04:57.388 INFO:tasks.workunit.client.1.smithi132.stdout:3/554: truncate d8/d13/d2b/d60/f83 1901863 0 2024-02-19T19:04:57.389 INFO:tasks.workunit.client.1.smithi132.stdout:7/643: link d1b/d1e/d28/d5e/c66 d1b/d5a/d63/d6b/d79/cc7 0 2024-02-19T19:04:57.391 INFO:tasks.workunit.client.1.smithi132.stdout:8/637: write d4/f27 [2033708,56709] 0 2024-02-19T19:04:57.393 INFO:tasks.workunit.client.1.smithi132.stdout:0/536: rename d2/dc/l29 to d2/dc/df/d15/d2f/d69/lb2 0 2024-02-19T19:04:57.393 INFO:tasks.workunit.client.1.smithi132.stdout:1/523: dread d1/d1e/d77/d2c/d89/f8d [0,4194304] 0 2024-02-19T19:04:57.394 INFO:tasks.workunit.client.1.smithi132.stdout:4/547: fsync d4/d12/f4b 0 2024-02-19T19:04:57.395 INFO:tasks.workunit.client.1.smithi132.stdout:6/506: rmdir d0/db/d25/d38 39 2024-02-19T19:04:57.396 INFO:tasks.workunit.client.1.smithi132.stdout:4/548: chown d4/d19/d21/d56 3363405 1 2024-02-19T19:04:57.396 INFO:tasks.workunit.client.1.smithi132.stdout:1/524: readlink d1/d1e/d77/d2c/d43/d81/l8a 0 2024-02-19T19:04:57.396 INFO:tasks.workunit.client.1.smithi132.stdout:3/555: mkdir d8/d13/d2b/d69/daa 0 2024-02-19T19:04:57.398 INFO:tasks.workunit.client.1.smithi132.stdout:8/638: dwrite d4/dc/d1b/d33/f4d [0,4194304] 0 2024-02-19T19:04:57.399 INFO:tasks.workunit.client.1.smithi132.stdout:6/507: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f37 [385123,128354] 0 2024-02-19T19:04:57.403 INFO:tasks.workunit.client.1.smithi132.stdout:8/639: chown d4/dc/d1b/d20/d23/d68/d7a/f95 58128 1 2024-02-19T19:04:57.404 INFO:tasks.workunit.client.1.smithi132.stdout:3/556: dread d8/d13/d2b/d69/da6/fa5 [0,4194304] 0 2024-02-19T19:04:57.408 INFO:tasks.workunit.client.1.smithi132.stdout:7/644: mkdir d1b/d1e/d41/da3/dc8 0 2024-02-19T19:04:57.409 INFO:tasks.workunit.client.1.smithi132.stdout:7/645: fsync d1b/d46/d76/f9f 0 2024-02-19T19:04:57.410 INFO:tasks.workunit.client.1.smithi132.stdout:1/525: read d1/d1e/d77/f3f [2041616,65487] 0 2024-02-19T19:04:57.411 INFO:tasks.workunit.client.1.smithi132.stdout:1/526: readlink d1/d1e/d77/d52/l55 0 2024-02-19T19:04:57.412 INFO:tasks.workunit.client.1.smithi132.stdout:1/527: read d1/f42 [3093507,90287] 0 2024-02-19T19:04:57.413 INFO:tasks.workunit.client.1.smithi132.stdout:5/627: getdents d8/da 0 2024-02-19T19:04:57.415 INFO:tasks.workunit.client.1.smithi132.stdout:3/557: creat d8/d13/d2b/d69/d7e/fab x:0 0 0 2024-02-19T19:04:57.416 INFO:tasks.workunit.client.1.smithi132.stdout:0/537: creat d2/dc/df/d15/d34/d84/da7/fb3 x:0 0 0 2024-02-19T19:04:57.419 INFO:tasks.workunit.client.1.smithi132.stdout:7/646: mknod d1b/d5a/d63/d6b/d86/cc9 0 2024-02-19T19:04:57.419 INFO:tasks.workunit.client.1.smithi132.stdout:3/558: dwrite d8/fd [0,4194304] 0 2024-02-19T19:04:57.423 INFO:tasks.workunit.client.1.smithi132.stdout:5/628: rmdir d8/d1a/d35 39 2024-02-19T19:04:57.425 INFO:tasks.workunit.client.1.smithi132.stdout:6/508: dread d0/d51/f92 [4194304,4194304] 0 2024-02-19T19:04:57.426 INFO:tasks.workunit.client.1.smithi132.stdout:3/559: creat d8/d13/d2f/fac x:0 0 0 2024-02-19T19:04:57.428 INFO:tasks.workunit.client.1.smithi132.stdout:4/549: rename d4/d12/f89 to d4/d27/d31/fbf 0 2024-02-19T19:04:57.430 INFO:tasks.workunit.client.1.smithi132.stdout:5/629: unlink d8/d23/f5c 0 2024-02-19T19:04:57.432 INFO:tasks.workunit.client.1.smithi132.stdout:0/538: creat d2/dc/df/d27/d3a/fb4 x:0 0 0 2024-02-19T19:04:57.437 INFO:tasks.workunit.client.1.smithi132.stdout:8/640: rename d4/ce to d4/dc/d1b/d54/d81/cdd 0 2024-02-19T19:04:57.438 INFO:tasks.workunit.client.1.smithi132.stdout:4/550: chown d4/d27/d31/l80 269 1 2024-02-19T19:04:57.441 INFO:tasks.workunit.client.1.smithi132.stdout:7/647: dread d1b/d1e/d28/f50 [4194304,4194304] 0 2024-02-19T19:04:57.442 INFO:tasks.workunit.client.1.smithi132.stdout:8/641: write d4/dc/d1b/d20/d23/f28 [2647468,57272] 0 2024-02-19T19:04:57.445 INFO:tasks.workunit.client.1.smithi132.stdout:4/551: mkdir d4/d27/d7f/dc0 0 2024-02-19T19:04:57.448 INFO:tasks.workunit.client.1.smithi132.stdout:8/642: dwrite d4/dc/d1b/d54/d5e/f67 [0,4194304] 0 2024-02-19T19:04:57.451 INFO:tasks.workunit.client.1.smithi132.stdout:1/528: rename d1/d1e/l71 to d1/d1e/d77/d2c/d62/d6b/la2 0 2024-02-19T19:04:57.453 INFO:tasks.workunit.client.1.smithi132.stdout:5/630: creat d8/d23/fd8 x:0 0 0 2024-02-19T19:04:57.453 INFO:tasks.workunit.client.1.smithi132.stdout:3/560: truncate d8/d13/d2b/d60/f9b 594811 0 2024-02-19T19:04:57.455 INFO:tasks.workunit.client.1.smithi132.stdout:6/509: link d0/db/d25/d38/l3e d0/db/d25/d3a/d4b/lac 0 2024-02-19T19:04:57.456 INFO:tasks.workunit.client.1.smithi132.stdout:5/631: stat d8/d23/d29/d27 0 2024-02-19T19:04:57.458 INFO:tasks.workunit.client.1.smithi132.stdout:3/561: rmdir d8/d93 39 2024-02-19T19:04:57.460 INFO:tasks.workunit.client.1.smithi132.stdout:8/643: dwrite d4/dc/d1b/d20/d23/d68/d7a/f94 [0,4194304] 0 2024-02-19T19:04:57.462 INFO:tasks.workunit.client.1.smithi132.stdout:0/539: dwrite d2/dc/df/d15/d2f/f64 [0,4194304] 0 2024-02-19T19:04:57.463 INFO:tasks.workunit.client.1.smithi132.stdout:1/529: creat d1/d3a/fa3 x:0 0 0 2024-02-19T19:04:57.463 INFO:tasks.workunit.client.1.smithi132.stdout:4/552: dread d4/d12/d16/f47 [0,4194304] 0 2024-02-19T19:04:57.466 INFO:tasks.workunit.client.1.smithi132.stdout:3/562: unlink d8/d13/f25 0 2024-02-19T19:04:57.469 INFO:tasks.workunit.client.1.smithi132.stdout:5/632: creat d8/d2e/dc4/dc8/daa/fd9 x:0 0 0 2024-02-19T19:04:57.470 INFO:tasks.workunit.client.1.smithi132.stdout:1/530: symlink d1/d61/la4 0 2024-02-19T19:04:57.472 INFO:tasks.workunit.client.1.smithi132.stdout:9/679: sync 2024-02-19T19:04:57.478 INFO:tasks.workunit.client.1.smithi132.stdout:5/633: rename d8/da/db/l12 to d8/d2e/lda 0 2024-02-19T19:04:57.480 INFO:tasks.workunit.client.1.smithi132.stdout:8/644: dwrite d4/dc/f44 [4194304,4194304] 0 2024-02-19T19:04:57.481 INFO:tasks.workunit.client.1.smithi132.stdout:7/648: write d1b/d1e/d41/f47 [184607,87658] 0 2024-02-19T19:04:57.484 INFO:tasks.workunit.client.1.smithi132.stdout:5/634: write d8/d23/fd8 [1048171,71558] 0 2024-02-19T19:04:57.486 INFO:tasks.workunit.client.1.smithi132.stdout:4/553: dwrite d4/fad [0,4194304] 0 2024-02-19T19:04:57.492 INFO:tasks.workunit.client.1.smithi132.stdout:4/554: write d4/d12/f2c [2219464,74169] 0 2024-02-19T19:04:57.493 INFO:tasks.workunit.client.1.smithi132.stdout:3/563: dwrite d8/d13/d2b/d69/d92/d34/fa3 [0,4194304] 0 2024-02-19T19:04:57.495 INFO:tasks.workunit.client.1.smithi132.stdout:5/635: truncate d8/d2e/f91 345700 0 2024-02-19T19:04:57.502 INFO:tasks.workunit.client.1.smithi132.stdout:4/555: chown d4/d27/d31/d71/d37/d66/d98 0 1 2024-02-19T19:04:57.503 INFO:tasks.workunit.client.1.smithi132.stdout:6/510: dwrite d0/db/d25/d3a/d4b/f54 [0,4194304] 0 2024-02-19T19:04:57.506 INFO:tasks.workunit.client.1.smithi132.stdout:5/636: mkdir d8/d2e/d71/da6/ddb 0 2024-02-19T19:04:57.507 INFO:tasks.workunit.client.1.smithi132.stdout:4/556: fdatasync d4/d19/f61 0 2024-02-19T19:04:57.507 INFO:tasks.workunit.client.1.smithi132.stdout:0/540: dwrite d2/dc/fe [0,4194304] 0 2024-02-19T19:04:57.508 INFO:tasks.workunit.client.1.smithi132.stdout:6/511: readlink d0/db/d25/d38/l7a 0 2024-02-19T19:04:57.509 INFO:tasks.workunit.client.1.smithi132.stdout:1/531: dwrite d1/f15 [0,4194304] 0 2024-02-19T19:04:57.510 INFO:tasks.workunit.client.1.smithi132.stdout:8/645: dwrite d4/dc/f3f [0,4194304] 0 2024-02-19T19:04:57.511 INFO:tasks.workunit.client.1.smithi132.stdout:1/532: chown d1/d1e/d77/d2c/d89/c7e 0 1 2024-02-19T19:04:57.512 INFO:tasks.workunit.client.1.smithi132.stdout:5/637: unlink f6 0 2024-02-19T19:04:57.513 INFO:tasks.workunit.client.1.smithi132.stdout:7/649: truncate d1b/d1e/d28/f44 2849604 0 2024-02-19T19:04:57.514 INFO:tasks.workunit.client.1.smithi132.stdout:3/564: link d8/d13/d19/l40 d8/d13/d2f/lad 0 2024-02-19T19:04:57.519 INFO:tasks.workunit.client.1.smithi132.stdout:1/533: unlink d1/d1e/d77/d52/l55 0 2024-02-19T19:04:57.520 INFO:tasks.workunit.client.1.smithi132.stdout:8/646: dread d4/dc/d1b/d54/d5e/f67 [0,4194304] 0 2024-02-19T19:04:57.521 INFO:tasks.workunit.client.1.smithi132.stdout:5/638: truncate d8/da/f33 2142938 0 2024-02-19T19:04:57.523 INFO:tasks.workunit.client.1.smithi132.stdout:4/557: symlink d4/d19/lc1 0 2024-02-19T19:04:57.524 INFO:tasks.workunit.client.1.smithi132.stdout:7/650: dwrite d1b/d1e/d28/fbb [0,4194304] 0 2024-02-19T19:04:57.526 INFO:tasks.workunit.client.1.smithi132.stdout:1/534: creat d1/d61/fa5 x:0 0 0 2024-02-19T19:04:57.528 INFO:tasks.workunit.client.1.smithi132.stdout:6/512: creat d0/fad x:0 0 0 2024-02-19T19:04:57.528 INFO:tasks.workunit.client.1.smithi132.stdout:7/651: chown d1b/d1e/d41/l71 38 1 2024-02-19T19:04:57.529 INFO:tasks.workunit.client.1.smithi132.stdout:4/558: write d4/d27/d31/d32/d51/d5c/f5d [440529,90987] 0 2024-02-19T19:04:57.531 INFO:tasks.workunit.client.1.smithi132.stdout:7/652: chown d1b/d46/c8c 1570 1 2024-02-19T19:04:57.532 INFO:tasks.workunit.client.1.smithi132.stdout:5/639: fdatasync d8/d1a/f9d 0 2024-02-19T19:04:57.533 INFO:tasks.workunit.client.1.smithi132.stdout:0/541: dread d2/dc/df/d15/f24 [4194304,4194304] 0 2024-02-19T19:04:57.535 INFO:tasks.workunit.client.1.smithi132.stdout:1/535: chown d1/d1e/d77/d2c/d89/c7e 178141696 1 2024-02-19T19:04:57.535 INFO:tasks.workunit.client.1.smithi132.stdout:0/542: readlink d2/dc/l55 0 2024-02-19T19:04:57.535 INFO:tasks.workunit.client.1.smithi132.stdout:8/647: dwrite d4/d2d/fca [0,4194304] 0 2024-02-19T19:04:57.536 INFO:tasks.workunit.client.1.smithi132.stdout:0/543: fdatasync d2/dc/f6d 0 2024-02-19T19:04:57.536 INFO:tasks.workunit.client.1.smithi132.stdout:6/513: mknod d0/db/d25/d3a/d3d/d69/d89/cae 0 2024-02-19T19:04:57.538 INFO:tasks.workunit.client.1.smithi132.stdout:5/640: mkdir d8/d2e/dc4/dc8/ddc 0 2024-02-19T19:04:57.539 INFO:tasks.workunit.client.1.smithi132.stdout:5/641: write d8/d1a/f9c [116718,106189] 0 2024-02-19T19:04:57.540 INFO:tasks.workunit.client.1.smithi132.stdout:4/559: rename d4/d19/l5b to d4/d12/d9f/lc2 0 2024-02-19T19:04:57.540 INFO:tasks.workunit.client.1.smithi132.stdout:5/642: chown d8/fe 93468346 1 2024-02-19T19:04:57.541 INFO:tasks.workunit.client.1.smithi132.stdout:1/536: mknod d1/d1e/d77/ca6 0 2024-02-19T19:04:57.543 INFO:tasks.workunit.client.1.smithi132.stdout:7/653: creat d1b/d1e/d28/d5e/d99/fca x:0 0 0 2024-02-19T19:04:57.543 INFO:tasks.workunit.client.1.smithi132.stdout:7/654: fdatasync d1b/d1e/f43 0 2024-02-19T19:04:57.544 INFO:tasks.workunit.client.1.smithi132.stdout:8/648: creat d4/dc/d1b/d33/fde x:0 0 0 2024-02-19T19:04:57.546 INFO:tasks.workunit.client.1.smithi132.stdout:6/514: mkdir d0/db/d25/d3a/d5d/d8f/d46/d5a/daf 0 2024-02-19T19:04:57.548 INFO:tasks.workunit.client.1.smithi132.stdout:1/537: write d1/d1e/f84 [33644,40049] 0 2024-02-19T19:04:57.549 INFO:tasks.workunit.client.1.smithi132.stdout:1/538: write d1/d1e/d77/d2c/d89/f9c [243937,73934] 0 2024-02-19T19:04:57.555 INFO:tasks.workunit.client.1.smithi132.stdout:0/544: rmdir d2/dc/d4a/dae 0 2024-02-19T19:04:57.556 INFO:tasks.workunit.client.1.smithi132.stdout:2/601: sync 2024-02-19T19:04:57.557 INFO:tasks.workunit.client.1.smithi132.stdout:1/539: dread d1/f4a [4194304,4194304] 0 2024-02-19T19:04:57.558 INFO:tasks.workunit.client.1.smithi132.stdout:1/540: write d1/f18 [2898583,76576] 0 2024-02-19T19:04:57.560 INFO:tasks.workunit.client.1.smithi132.stdout:4/560: creat d4/d12/fc3 x:0 0 0 2024-02-19T19:04:57.562 INFO:tasks.workunit.client.1.smithi132.stdout:5/643: link d8/da/db/l5f d8/d2e/d71/ldd 0 2024-02-19T19:04:57.563 INFO:tasks.workunit.client.1.smithi132.stdout:4/561: dread d4/d12/d16/f47 [0,4194304] 0 2024-02-19T19:04:57.565 INFO:tasks.workunit.client.1.smithi132.stdout:0/545: creat d2/dc/d4a/d74/fb5 x:0 0 0 2024-02-19T19:04:57.565 INFO:tasks.workunit.client.1.smithi132.stdout:7/655: link d1b/d46/d76/ca6 d1b/d46/d76/ccb 0 2024-02-19T19:04:57.566 INFO:tasks.workunit.client.1.smithi132.stdout:7/656: write d1b/d1e/d28/d5e/d99/fca [624715,8396] 0 2024-02-19T19:04:57.567 INFO:tasks.workunit.client.1.smithi132.stdout:2/602: truncate d0/d3c/f60 4250697 0 2024-02-19T19:04:57.570 INFO:tasks.workunit.client.1.smithi132.stdout:1/541: mknod d1/d1e/d6a/ca7 0 2024-02-19T19:04:57.571 INFO:tasks.workunit.client.1.smithi132.stdout:8/649: link d4/dc/d1b/d33/c3d d4/dc/d42/cdf 0 2024-02-19T19:04:57.571 INFO:tasks.workunit.client.1.smithi132.stdout:1/542: dread - d1/d1e/d77/d78/f7f zero size 2024-02-19T19:04:57.571 INFO:tasks.workunit.client.1.smithi132.stdout:8/650: dread - d4/d2d/fbf zero size 2024-02-19T19:04:57.575 INFO:tasks.workunit.client.1.smithi132.stdout:0/546: mknod d2/dc/df/d15/d2f/d69/cb6 0 2024-02-19T19:04:57.576 INFO:tasks.workunit.client.1.smithi132.stdout:0/547: write d2/dc/d4a/d58/f95 [1309881,78098] 0 2024-02-19T19:04:57.578 INFO:tasks.workunit.client.1.smithi132.stdout:7/657: mkdir d1b/d5a/d63/d96/dcc 0 2024-02-19T19:04:57.580 INFO:tasks.workunit.client.1.smithi132.stdout:8/651: creat d4/dc/d1b/d33/fe0 x:0 0 0 2024-02-19T19:04:57.588 INFO:tasks.workunit.client.1.smithi132.stdout:8/652: creat d4/dc/d1b/d38/d69/d74/fe1 x:0 0 0 2024-02-19T19:04:57.589 INFO:tasks.workunit.client.1.smithi132.stdout:0/548: dwrite d2/f6 [0,4194304] 0 2024-02-19T19:04:57.591 INFO:tasks.workunit.client.1.smithi132.stdout:4/562: getdents d4/d27/d31/d71/d14/d24 0 2024-02-19T19:04:57.592 INFO:tasks.workunit.client.1.smithi132.stdout:4/563: read - d4/d19/d21/d56/f8f zero size 2024-02-19T19:04:57.593 INFO:tasks.workunit.client.1.smithi132.stdout:0/549: chown d2/dc/df/d15/d2f/d3f/d63/l91 157 1 2024-02-19T19:04:57.593 INFO:tasks.workunit.client.1.smithi132.stdout:8/653: creat d4/dc/d1b/d38/d69/d74/dce/fe2 x:0 0 0 2024-02-19T19:04:57.594 INFO:tasks.workunit.client.1.smithi132.stdout:6/515: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f2e [896455,105103] 0 2024-02-19T19:04:57.596 INFO:tasks.workunit.client.1.smithi132.stdout:4/564: chown d4/d27/l8b 271113 1 2024-02-19T19:04:57.598 INFO:tasks.workunit.client.1.smithi132.stdout:6/516: write d0/db/d25/d3a/d5d/d8f/f48 [1933357,27901] 0 2024-02-19T19:04:57.600 INFO:tasks.workunit.client.1.smithi132.stdout:0/550: creat d2/dc/d4a/d25/d43/d5f/fb7 x:0 0 0 2024-02-19T19:04:57.601 INFO:tasks.workunit.client.1.smithi132.stdout:8/654: dread d4/dc/f44 [4194304,4194304] 0 2024-02-19T19:04:57.601 INFO:tasks.workunit.client.1.smithi132.stdout:4/565: fsync d4/d12/d16/f47 0 2024-02-19T19:04:57.602 INFO:tasks.workunit.client.1.smithi132.stdout:8/655: stat d4/fab 0 2024-02-19T19:04:57.604 INFO:tasks.workunit.client.1.smithi132.stdout:5/644: dread d8/d1a/d35/f4b [0,4194304] 0 2024-02-19T19:04:57.605 INFO:tasks.workunit.client.1.smithi132.stdout:4/566: write d4/d27/d31/d32/d51/f93 [97934,23674] 0 2024-02-19T19:04:57.605 INFO:tasks.workunit.client.1.smithi132.stdout:5/645: fdatasync d8/d2e/fa5 0 2024-02-19T19:04:57.606 INFO:tasks.workunit.client.1.smithi132.stdout:6/517: symlink d0/db/d25/d50/lb0 0 2024-02-19T19:04:57.607 INFO:tasks.workunit.client.1.smithi132.stdout:8/656: mknod d4/dc/d1b/d20/d62/ce3 0 2024-02-19T19:04:57.610 INFO:tasks.workunit.client.1.smithi132.stdout:6/518: mknod d0/db/d25/d3a/d4b/cb1 0 2024-02-19T19:04:57.613 INFO:tasks.workunit.client.1.smithi132.stdout:5/646: read d8/da/d5a/f81 [4159127,106605] 0 2024-02-19T19:04:57.616 INFO:tasks.workunit.client.1.smithi132.stdout:8/657: mknod d4/dc/d1b/d8f/ce4 0 2024-02-19T19:04:57.616 INFO:tasks.workunit.client.1.smithi132.stdout:8/658: readlink d4/lb0 0 2024-02-19T19:04:57.617 INFO:tasks.workunit.client.1.smithi132.stdout:8/659: dread - d4/dc/d1b/d20/f9f zero size 2024-02-19T19:04:57.618 INFO:tasks.workunit.client.1.smithi132.stdout:2/603: dread d0/d3c/f60 [0,4194304] 0 2024-02-19T19:04:57.623 INFO:tasks.workunit.client.1.smithi132.stdout:2/604: dwrite d0/d3c/f92 [0,4194304] 0 2024-02-19T19:04:57.623 INFO:tasks.workunit.client.1.smithi132.stdout:8/660: mkdir d4/dc/d1b/d20/de5 0 2024-02-19T19:04:57.624 INFO:tasks.workunit.client.1.smithi132.stdout:2/605: dread - d0/d4/df/d40/fae zero size 2024-02-19T19:04:57.627 INFO:tasks.workunit.client.1.smithi132.stdout:4/567: rename d4/d27/d31/l8a to d4/d27/d31/d32/lc4 0 2024-02-19T19:04:57.628 INFO:tasks.workunit.client.1.smithi132.stdout:4/568: chown d4/d12/fc3 3 1 2024-02-19T19:04:57.630 INFO:tasks.workunit.client.1.smithi132.stdout:8/661: unlink d4/dc/d1b/d20/d23/d68/d79/f85 0 2024-02-19T19:04:57.638 INFO:tasks.workunit.client.1.smithi132.stdout:0/551: truncate d2/dc/d4a/d25/d43/f6f 3586746 0 2024-02-19T19:04:57.640 INFO:tasks.workunit.client.1.smithi132.stdout:2/606: creat d0/d4/db9/fbe x:0 0 0 2024-02-19T19:04:57.642 INFO:tasks.workunit.client.1.smithi132.stdout:0/552: dread d2/f6 [0,4194304] 0 2024-02-19T19:04:57.643 INFO:tasks.workunit.client.1.smithi132.stdout:5/647: rename d8/d1a/d64/la0 to d8/d2e/dc4/dc8/da9/lde 0 2024-02-19T19:04:57.644 INFO:tasks.workunit.client.1.smithi132.stdout:4/569: symlink d4/d12/da4/lc5 0 2024-02-19T19:04:57.644 INFO:tasks.workunit.client.1.smithi132.stdout:5/648: chown d8/d2e/d71/f80 4681 1 2024-02-19T19:04:57.646 INFO:tasks.workunit.client.1.smithi132.stdout:5/649: truncate d8/da/d5a/d84/fd5 875900 0 2024-02-19T19:04:57.647 INFO:tasks.workunit.client.1.smithi132.stdout:9/680: sync 2024-02-19T19:04:57.648 INFO:tasks.workunit.client.1.smithi132.stdout:2/607: creat d0/d24/d58/d53/fbf x:0 0 0 2024-02-19T19:04:57.654 INFO:tasks.workunit.client.1.smithi132.stdout:6/519: rename d0/db/l30 to d0/db/d25/d50/d86/lb2 0 2024-02-19T19:04:57.654 INFO:tasks.workunit.client.1.smithi132.stdout:5/650: mkdir d8/da/d5a/d7b/d96/d9e/ddf 0 2024-02-19T19:04:57.654 INFO:tasks.workunit.client.1.smithi132.stdout:9/681: mknod d12/d2a/d88/cec 0 2024-02-19T19:04:57.655 INFO:tasks.workunit.client.1.smithi132.stdout:5/651: dread - d8/da/d5a/fcb zero size 2024-02-19T19:04:57.656 INFO:tasks.workunit.client.1.smithi132.stdout:5/652: dread - d8/d2e/fb4 zero size 2024-02-19T19:04:57.667 INFO:tasks.workunit.client.1.smithi132.stdout:4/570: symlink d4/d4d/lc6 0 2024-02-19T19:04:57.668 INFO:tasks.workunit.client.1.smithi132.stdout:0/553: write d2/dc/d4a/d74/f86 [1049348,96339] 0 2024-02-19T19:04:57.668 INFO:tasks.workunit.client.1.smithi132.stdout:2/608: dread d0/d19/d1e/f52 [0,4194304] 0 2024-02-19T19:04:57.668 INFO:tasks.workunit.client.1.smithi132.stdout:9/682: rmdir d12/d8d/d3a/d40/d4e/d77 39 2024-02-19T19:04:57.669 INFO:tasks.workunit.client.1.smithi132.stdout:8/662: dwrite d4/dc/d1b/d54/d5e/f50 [0,4194304] 0 2024-02-19T19:04:57.670 INFO:tasks.workunit.client.1.smithi132.stdout:2/609: write d0/d4/d11/f15 [4510523,8569] 0 2024-02-19T19:04:57.671 INFO:tasks.workunit.client.1.smithi132.stdout:0/554: fdatasync d2/dc/d4a/d25/d43/d5f/fb7 0 2024-02-19T19:04:57.671 INFO:tasks.workunit.client.1.smithi132.stdout:4/571: truncate d4/d12/d16/f47 8469471 0 2024-02-19T19:04:57.674 INFO:tasks.workunit.client.1.smithi132.stdout:8/663: chown d4/lb0 0 1 2024-02-19T19:04:57.674 INFO:tasks.workunit.client.1.smithi132.stdout:9/683: write d12/d2a/d4c/d61/f6b [1082303,68662] 0 2024-02-19T19:04:57.676 INFO:tasks.workunit.client.1.smithi132.stdout:6/520: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f20 [241987,42081] 0 2024-02-19T19:04:57.677 INFO:tasks.workunit.client.1.smithi132.stdout:0/555: chown d2/dc/df/fa9 146 1 2024-02-19T19:04:57.678 INFO:tasks.workunit.client.1.smithi132.stdout:9/684: write d12/fae [3931952,128513] 0 2024-02-19T19:04:57.680 INFO:tasks.workunit.client.1.smithi132.stdout:5/653: dwrite d8/ff [4194304,4194304] 0 2024-02-19T19:04:57.680 INFO:tasks.workunit.client.1.smithi132.stdout:8/664: write d4/f6 [2927951,73538] 0 2024-02-19T19:04:57.680 INFO:tasks.workunit.client.1.smithi132.stdout:2/610: creat d0/d24/d58/d53/dab/d8a/fc0 x:0 0 0 2024-02-19T19:04:57.682 INFO:tasks.workunit.client.1.smithi132.stdout:6/521: unlink d0/d51/f98 0 2024-02-19T19:04:57.685 INFO:tasks.workunit.client.1.smithi132.stdout:4/572: rename d4/d27/d31/d71/d37/d66 to d4/d19/daa/dc7 0 2024-02-19T19:04:57.685 INFO:tasks.workunit.client.1.smithi132.stdout:6/522: chown d0/d51/f60 1511636 1 2024-02-19T19:04:57.686 INFO:tasks.workunit.client.1.smithi132.stdout:0/556: symlink d2/dc/df/d15/d34/d84/lb8 0 2024-02-19T19:04:57.689 INFO:tasks.workunit.client.1.smithi132.stdout:8/665: creat d4/dc/d1b/d38/d69/d74/fe6 x:0 0 0 2024-02-19T19:04:57.689 INFO:tasks.workunit.client.1.smithi132.stdout:6/523: read d0/db/d25/f8d [2941535,70549] 0 2024-02-19T19:04:57.690 INFO:tasks.workunit.client.1.smithi132.stdout:0/557: creat d2/dc/df/d15/d2f/d69/fb9 x:0 0 0 2024-02-19T19:04:57.690 INFO:tasks.workunit.client.1.smithi132.stdout:6/524: stat d0/db/d25/d3a/d5d/d8f/d22/f64 0 2024-02-19T19:04:57.691 INFO:tasks.workunit.client.1.smithi132.stdout:4/573: fsync d4/d4d/d87/f9a 0 2024-02-19T19:04:57.691 INFO:tasks.workunit.client.1.smithi132.stdout:9/685: mkdir d12/d8d/d3a/d40/d4e/ded 0 2024-02-19T19:04:57.692 INFO:tasks.workunit.client.1.smithi132.stdout:8/666: creat d4/d2d/fe7 x:0 0 0 2024-02-19T19:04:57.692 INFO:tasks.workunit.client.1.smithi132.stdout:3/565: sync 2024-02-19T19:04:57.694 INFO:tasks.workunit.client.1.smithi132.stdout:0/558: symlink d2/d6e/d7a/lba 0 2024-02-19T19:04:57.695 INFO:tasks.workunit.client.1.smithi132.stdout:8/667: truncate d4/d2d/dcc/fcd 829002 0 2024-02-19T19:04:57.696 INFO:tasks.workunit.client.1.smithi132.stdout:4/574: symlink d4/d19/d21/d56/dac/lc8 0 2024-02-19T19:04:57.697 INFO:tasks.workunit.client.1.smithi132.stdout:4/575: chown d4/d27/c78 14879907 1 2024-02-19T19:04:57.698 INFO:tasks.workunit.client.1.smithi132.stdout:9/686: creat d12/d8d/d3a/d40/d4e/d77/d89/dc4/fee x:0 0 0 2024-02-19T19:04:57.699 INFO:tasks.workunit.client.1.smithi132.stdout:3/566: getdents d8/d13/d2b/d69/d7e/d9f/da2 0 2024-02-19T19:04:57.700 INFO:tasks.workunit.client.1.smithi132.stdout:8/668: mkdir d4/dc/d1b/d20/d23/d68/d7a/de8 0 2024-02-19T19:04:57.703 INFO:tasks.workunit.client.1.smithi132.stdout:4/576: truncate d4/fa 1940601 0 2024-02-19T19:04:57.704 INFO:tasks.workunit.client.1.smithi132.stdout:9/687: mknod d12/d2a/d88/db8/cef 0 2024-02-19T19:04:57.706 INFO:tasks.workunit.client.1.smithi132.stdout:0/559: dwrite d2/dc/df/d15/d2f/d69/faf [0,4194304] 0 2024-02-19T19:04:57.707 INFO:tasks.workunit.client.1.smithi132.stdout:3/567: rename d8/d13/d2b/f33 to d8/d38/d44/fae 0 2024-02-19T19:04:57.708 INFO:tasks.workunit.client.1.smithi132.stdout:9/688: fsync d12/d3e/d6e/fac 0 2024-02-19T19:04:57.711 INFO:tasks.workunit.client.1.smithi132.stdout:4/577: rename d4/d19/daa/dc7 to d4/d19/daa/dc7/d98/dc9 22 2024-02-19T19:04:57.712 INFO:tasks.workunit.client.1.smithi132.stdout:2/611: dread d0/d19/d1e/f45 [0,4194304] 0 2024-02-19T19:04:57.713 INFO:tasks.workunit.client.1.smithi132.stdout:5/654: dwrite d8/d23/d29/d27/f2b [0,4194304] 0 2024-02-19T19:04:57.714 INFO:tasks.workunit.client.1.smithi132.stdout:6/525: dread d0/fa [0,4194304] 0 2024-02-19T19:04:57.716 INFO:tasks.workunit.client.1.smithi132.stdout:3/568: mkdir d8/d13/d2b/d69/d92/daf 0 2024-02-19T19:04:57.716 INFO:tasks.workunit.client.1.smithi132.stdout:8/669: write d4/f19 [3734396,110346] 0 2024-02-19T19:04:57.716 INFO:tasks.workunit.client.1.smithi132.stdout:2/612: fsync d0/d4/d11/d2e/f6c 0 2024-02-19T19:04:57.718 INFO:tasks.workunit.client.1.smithi132.stdout:6/526: stat d0/db/d25/d3a/d4b/d7c/da2 0 2024-02-19T19:04:57.719 INFO:tasks.workunit.client.1.smithi132.stdout:8/670: chown d4/dc/d1b/d20/f8b 826 1 2024-02-19T19:04:57.719 INFO:tasks.workunit.client.1.smithi132.stdout:2/613: read - d0/d24/d58/d53/dab/d8a/fc0 zero size 2024-02-19T19:04:57.719 INFO:tasks.workunit.client.1.smithi132.stdout:4/578: mknod d4/d12/d48/d5f/cca 0 2024-02-19T19:04:57.721 INFO:tasks.workunit.client.1.smithi132.stdout:6/527: chown d0/db/d25/d3a/d5d/d8f/d46/f7b 13012 1 2024-02-19T19:04:57.722 INFO:tasks.workunit.client.1.smithi132.stdout:5/655: mknod d8/d23/d29/ce0 0 2024-02-19T19:04:57.724 INFO:tasks.workunit.client.1.smithi132.stdout:9/689: creat d12/d8d/d3a/ff0 x:0 0 0 2024-02-19T19:04:57.729 INFO:tasks.workunit.client.1.smithi132.stdout:9/690: dwrite d12/d3e/f6c [0,4194304] 0 2024-02-19T19:04:57.730 INFO:tasks.workunit.client.1.smithi132.stdout:9/691: fsync d12/d2a/f2b 0 2024-02-19T19:04:57.730 INFO:tasks.workunit.client.1.smithi132.stdout:1/543: sync 2024-02-19T19:04:57.730 INFO:tasks.workunit.client.1.smithi132.stdout:7/658: sync 2024-02-19T19:04:57.733 INFO:tasks.workunit.client.1.smithi132.stdout:7/659: truncate d1b/d1e/d28/f77 471793 0 2024-02-19T19:04:57.734 INFO:tasks.workunit.client.1.smithi132.stdout:2/614: creat d0/d24/d58/d53/dab/d8a/da5/fc1 x:0 0 0 2024-02-19T19:04:57.735 INFO:tasks.workunit.client.1.smithi132.stdout:2/615: stat d0/d4/df/d40/fae 0 2024-02-19T19:04:57.735 INFO:tasks.workunit.client.1.smithi132.stdout:9/692: dread d12/d1b/f3f [0,4194304] 0 2024-02-19T19:04:57.738 INFO:tasks.workunit.client.1.smithi132.stdout:6/528: truncate d0/db/d25/d3a/d5d/d8f/d22/f19 3786948 0 2024-02-19T19:04:57.740 INFO:tasks.workunit.client.1.smithi132.stdout:3/569: creat d8/d13/d2b/d69/daa/fb0 x:0 0 0 2024-02-19T19:04:57.741 INFO:tasks.workunit.client.1.smithi132.stdout:4/579: rmdir d4 39 2024-02-19T19:04:57.743 INFO:tasks.workunit.client.1.smithi132.stdout:0/560: dread d2/dc/df/d78/fa1 [0,4194304] 0 2024-02-19T19:04:57.745 INFO:tasks.workunit.client.1.smithi132.stdout:0/561: write d2/dc/f6d [3359984,92497] 0 2024-02-19T19:04:57.745 INFO:tasks.workunit.client.1.smithi132.stdout:0/562: fdatasync d2/f8c 0 2024-02-19T19:04:57.745 INFO:tasks.workunit.client.1.smithi132.stdout:1/544: mknod d1/d1e/d77/d4e/d99/ca8 0 2024-02-19T19:04:57.747 INFO:tasks.workunit.client.1.smithi132.stdout:3/570: dread d8/d38/f8b [0,4194304] 0 2024-02-19T19:04:57.748 INFO:tasks.workunit.client.1.smithi132.stdout:7/660: unlink d1b/d1e/d2b/cc1 0 2024-02-19T19:04:57.751 INFO:tasks.workunit.client.1.smithi132.stdout:2/616: rename d0/d4/d11/d2e/d5a/d66 to d0/d4/df/dc2 0 2024-02-19T19:04:57.753 INFO:tasks.workunit.client.1.smithi132.stdout:6/529: rmdir d0/db/d25/d3a 39 2024-02-19T19:04:57.753 INFO:tasks.workunit.client.1.smithi132.stdout:0/563: dwrite d2/dc/df/fa9 [0,4194304] 0 2024-02-19T19:04:57.753 INFO:tasks.workunit.client.1.smithi132.stdout:6/530: stat d0/d8b/f4c 0 2024-02-19T19:04:57.759 INFO:tasks.workunit.client.1.smithi132.stdout:4/580: read d4/d12/d48/f7c [3001153,40546] 0 2024-02-19T19:04:57.759 INFO:tasks.workunit.client.1.smithi132.stdout:8/671: creat d4/dc/d1b/d20/d23/fe9 x:0 0 0 2024-02-19T19:04:57.759 INFO:tasks.workunit.client.1.smithi132.stdout:5/656: dread d8/d23/d29/d27/f87 [0,4194304] 0 2024-02-19T19:04:57.762 INFO:tasks.workunit.client.1.smithi132.stdout:9/693: write d12/d3e/f71 [15067,124300] 0 2024-02-19T19:04:57.762 INFO:tasks.workunit.client.1.smithi132.stdout:9/694: chown d12/d8d/d3a/d85/d87/fb0 124091 1 2024-02-19T19:04:57.763 INFO:tasks.workunit.client.1.smithi132.stdout:9/695: chown d12/d8d/f76 425 1 2024-02-19T19:04:57.763 INFO:tasks.workunit.client.1.smithi132.stdout:7/661: unlink d1b/d1e/d28/c8a 0 2024-02-19T19:04:57.763 INFO:tasks.workunit.client.1.smithi132.stdout:9/696: readlink d12/d2a/d88/l91 0 2024-02-19T19:04:57.764 INFO:tasks.workunit.client.1.smithi132.stdout:3/571: truncate d8/d13/d19/f49 2134458 0 2024-02-19T19:04:57.767 INFO:tasks.workunit.client.1.smithi132.stdout:2/617: rename d0/d3c/f60 to d0/d4/df/fc3 0 2024-02-19T19:04:57.770 INFO:tasks.workunit.client.1.smithi132.stdout:0/564: unlink d2/dc/df/d27/f56 0 2024-02-19T19:04:57.771 INFO:tasks.workunit.client.1.smithi132.stdout:3/572: dwrite d8/d13/d19/f2d [8388608,4194304] 0 2024-02-19T19:04:57.771 INFO:tasks.workunit.client.1.smithi132.stdout:3/573: chown d8/d13/d2f 1073 1 2024-02-19T19:04:57.772 INFO:tasks.workunit.client.1.smithi132.stdout:6/531: mknod d0/d51/cb3 0 2024-02-19T19:04:57.774 INFO:tasks.workunit.client.1.smithi132.stdout:5/657: symlink d8/da/le1 0 2024-02-19T19:04:57.775 INFO:tasks.workunit.client.1.smithi132.stdout:3/574: write d8/d13/d2b/d69/d92/d34/d42/f54 [6101960,87014] 0 2024-02-19T19:04:57.775 INFO:tasks.workunit.client.1.smithi132.stdout:8/672: fsync d4/dc/d1b/d20/d23/fe9 0 2024-02-19T19:04:57.776 INFO:tasks.workunit.client.1.smithi132.stdout:9/697: symlink d12/d3e/d6e/lf1 0 2024-02-19T19:04:57.780 INFO:tasks.workunit.client.1.smithi132.stdout:9/698: write d12/d8d/d3a/d40/d4e/d77/d9d/faf [210598,42557] 0 2024-02-19T19:04:57.781 INFO:tasks.workunit.client.1.smithi132.stdout:0/565: rename d2/dc/df/d27/d3a/d52/f68 to d2/dc/d4a/d67/fbb 0 2024-02-19T19:04:57.785 INFO:tasks.workunit.client.1.smithi132.stdout:7/662: dread ff [0,4194304] 0 2024-02-19T19:04:57.786 INFO:tasks.workunit.client.1.smithi132.stdout:5/658: mkdir d8/d1a/d64/de2 0 2024-02-19T19:04:57.791 INFO:tasks.workunit.client.1.smithi132.stdout:3/575: write d8/d38/f8b [615339,32780] 0 2024-02-19T19:04:57.792 INFO:tasks.workunit.client.1.smithi132.stdout:6/532: dwrite d0/db/d25/d3a/d5d/d8f/d46/d5a/f20 [0,4194304] 0 2024-02-19T19:04:57.793 INFO:tasks.workunit.client.1.smithi132.stdout:1/545: getdents d1 0 2024-02-19T19:04:57.794 INFO:tasks.workunit.client.1.smithi132.stdout:3/576: chown d8/d13/d2f/l7f 9344 1 2024-02-19T19:04:57.795 INFO:tasks.workunit.client.1.smithi132.stdout:6/533: chown d0/db/d25/d50/d86 20 1 2024-02-19T19:04:57.795 INFO:tasks.workunit.client.1.smithi132.stdout:0/566: rename d2/dc/df/l66 to d2/dc/df/d27/d3a/lbc 0 2024-02-19T19:04:57.797 INFO:tasks.workunit.client.1.smithi132.stdout:4/581: creat d4/d27/d31/d32/d51/fcb x:0 0 0 2024-02-19T19:04:57.798 INFO:tasks.workunit.client.1.smithi132.stdout:7/663: readlink d1b/d32/lb7 0 2024-02-19T19:04:57.800 INFO:tasks.workunit.client.1.smithi132.stdout:8/673: creat d4/dc/d1b/d20/dbe/fea x:0 0 0 2024-02-19T19:04:57.801 INFO:tasks.workunit.client.1.smithi132.stdout:4/582: chown d4/d27/d31/d32/d51/d5c/fa8 12354 1 2024-02-19T19:04:57.804 INFO:tasks.workunit.client.1.smithi132.stdout:9/699: link d12/d2a/d4c/d61/f6b d12/d8d/d3a/d85/d87/dc8/deb/ff2 0 2024-02-19T19:04:57.804 INFO:tasks.workunit.client.1.smithi132.stdout:1/546: dwrite d1/d1e/d77/f3c [0,4194304] 0 2024-02-19T19:04:57.805 INFO:tasks.workunit.client.1.smithi132.stdout:0/567: dwrite d2/dc/df/d15/d2f/d3f/d63/d6b/f72 [0,4194304] 0 2024-02-19T19:04:57.805 INFO:tasks.workunit.client.1.smithi132.stdout:9/700: stat d12/d1b/f3f 0 2024-02-19T19:04:57.809 INFO:tasks.workunit.client.1.smithi132.stdout:0/568: write d2/dc/d4a/d25/d43/d5f/fa4 [126636,131052] 0 2024-02-19T19:04:57.810 INFO:tasks.workunit.client.1.smithi132.stdout:7/664: symlink d1b/d1e/d28/d5e/lcd 0 2024-02-19T19:04:57.811 INFO:tasks.workunit.client.1.smithi132.stdout:9/701: write d12/d8d/d3a/d85/d87/fb5 [931497,113557] 0 2024-02-19T19:04:57.811 INFO:tasks.workunit.client.1.smithi132.stdout:5/659: fdatasync d8/da/db/fbb 0 2024-02-19T19:04:57.813 INFO:tasks.workunit.client.1.smithi132.stdout:5/660: readlink d8/da/l52 0 2024-02-19T19:04:57.814 INFO:tasks.workunit.client.1.smithi132.stdout:2/618: truncate d0/d4/df/d40/f74 1501495 0 2024-02-19T19:04:57.815 INFO:tasks.workunit.client.1.smithi132.stdout:3/577: mknod d8/d13/cb1 0 2024-02-19T19:04:57.820 INFO:tasks.workunit.client.1.smithi132.stdout:4/583: creat d4/d27/d7f/dc0/fcc x:0 0 0 2024-02-19T19:04:57.820 INFO:tasks.workunit.client.1.smithi132.stdout:7/665: dwrite d1b/d1e/f22 [8388608,4194304] 0 2024-02-19T19:04:57.823 INFO:tasks.workunit.client.1.smithi132.stdout:8/674: symlink d4/dc/d1b/d38/ddb/leb 0 2024-02-19T19:04:57.825 INFO:tasks.workunit.client.1.smithi132.stdout:4/584: chown d4/d27/d31/d71/l11 1 1 2024-02-19T19:04:57.827 INFO:tasks.workunit.client.1.smithi132.stdout:0/569: mkdir d2/dc/df/dbd 0 2024-02-19T19:04:57.829 INFO:tasks.workunit.client.1.smithi132.stdout:9/702: dwrite d12/d8d/d3a/d40/d4e/d77/d8a/d82/f8c [0,4194304] 0 2024-02-19T19:04:57.830 INFO:tasks.workunit.client.1.smithi132.stdout:1/547: link d1/d3a/l8b d1/d95/d97/la9 0 2024-02-19T19:04:57.830 INFO:tasks.workunit.client.1.smithi132.stdout:2/619: dwrite d0/d4/d11/d2e/d5a/f5b [4194304,4194304] 0 2024-02-19T19:04:57.832 INFO:tasks.workunit.client.1.smithi132.stdout:4/585: read d4/d27/d31/d71/d37/f74 [2438138,70499] 0 2024-02-19T19:04:57.833 INFO:tasks.workunit.client.1.smithi132.stdout:7/666: rename d1b/d1e/d28/c2a to d1b/d46/d7f/cce 0 2024-02-19T19:04:57.833 INFO:tasks.workunit.client.1.smithi132.stdout:1/548: chown d1/d1e/f5b 11 1 2024-02-19T19:04:57.836 INFO:tasks.workunit.client.1.smithi132.stdout:3/578: mkdir d8/d13/d2b/d69/d92/d8d/db2 0 2024-02-19T19:04:57.838 INFO:tasks.workunit.client.1.smithi132.stdout:5/661: creat d8/d23/fe3 x:0 0 0 2024-02-19T19:04:57.839 INFO:tasks.workunit.client.1.smithi132.stdout:2/620: chown d0/d24/d58/d53/c77 3 1 2024-02-19T19:04:57.839 INFO:tasks.workunit.client.1.smithi132.stdout:8/675: write d4/dc/d1b/d54/d5e/f67 [1527751,67162] 0 2024-02-19T19:04:57.840 INFO:tasks.workunit.client.1.smithi132.stdout:9/703: symlink d12/d8d/d3a/d40/lf3 0 2024-02-19T19:04:57.842 INFO:tasks.workunit.client.1.smithi132.stdout:3/579: readlink d8/d13/d2b/d69/d92/l84 0 2024-02-19T19:04:57.843 INFO:tasks.workunit.client.1.smithi132.stdout:4/586: creat d4/d27/d31/d32/d51/d5c/dbc/fcd x:0 0 0 2024-02-19T19:04:57.844 INFO:tasks.workunit.client.1.smithi132.stdout:9/704: chown d12/d2a/d4c/d61/l9a 103710 1 2024-02-19T19:04:57.845 INFO:tasks.workunit.client.1.smithi132.stdout:0/570: dwrite d2/dc/df/d27/d3a/fb4 [0,4194304] 0 2024-02-19T19:04:57.848 INFO:tasks.workunit.client.1.smithi132.stdout:1/549: mkdir d1/d1e/daa 0 2024-02-19T19:04:57.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:57 smithi088 ceph-mon[88643]: mgrmap e34: smithi088.gyviqq(active, since 3m), standbys: smithi132.psllsa 2024-02-19T19:04:57.851 INFO:tasks.workunit.client.1.smithi132.stdout:2/621: creat d0/d19/d6a/fc4 x:0 0 0 2024-02-19T19:04:57.853 INFO:tasks.workunit.client.1.smithi132.stdout:5/662: mknod d8/d2e/d71/dbc/ce4 0 2024-02-19T19:04:57.853 INFO:tasks.workunit.client.1.smithi132.stdout:8/676: symlink d4/dc/d1b/d20/d62/d91/lec 0 2024-02-19T19:04:57.855 INFO:tasks.workunit.client.1.smithi132.stdout:4/587: dwrite d4/d27/d7f/fa9 [0,4194304] 0 2024-02-19T19:04:57.857 INFO:tasks.workunit.client.1.smithi132.stdout:8/677: fsync d4/d2d/fe7 0 2024-02-19T19:04:57.857 INFO:tasks.workunit.client.1.smithi132.stdout:1/550: dread d1/d1e/d77/d52/d64/d75/f83 [0,4194304] 0 2024-02-19T19:04:57.860 INFO:tasks.workunit.client.1.smithi132.stdout:4/588: chown d4/d27/d31/d71/d37 52519905 1 2024-02-19T19:04:57.861 INFO:tasks.workunit.client.1.smithi132.stdout:3/580: symlink d8/d13/d2b/d69/d92/d34/lb3 0 2024-02-19T19:04:57.862 INFO:tasks.workunit.client.1.smithi132.stdout:7/667: dwrite d1b/d46/d76/d94/f9d [0,4194304] 0 2024-02-19T19:04:57.864 INFO:tasks.workunit.client.1.smithi132.stdout:7/668: read d1b/d1e/fb2 [6340844,2686] 0 2024-02-19T19:04:57.866 INFO:tasks.workunit.client.1.smithi132.stdout:9/705: rename d12/d2a/d4c/d61/le6 to d12/d8d/d3a/d40/d4e/d77/d8a/d82/dd9/lf4 0 2024-02-19T19:04:57.868 INFO:tasks.workunit.client.1.smithi132.stdout:1/551: mknod d1/d1e/cab 0 2024-02-19T19:04:57.869 INFO:tasks.workunit.client.1.smithi132.stdout:8/678: read d4/dc/d1b/d54/f87 [26641,126443] 0 2024-02-19T19:04:57.872 INFO:tasks.workunit.client.1.smithi132.stdout:2/622: dwrite d0/d4/db9/fbe [0,4194304] 0 2024-02-19T19:04:57.872 INFO:tasks.workunit.client.1.smithi132.stdout:9/706: dread - d12/d8d/d3a/d85/d87/fe4 zero size 2024-02-19T19:04:57.874 INFO:tasks.workunit.client.1.smithi132.stdout:9/707: fdatasync d12/d2a/d88/db8/fdd 0 2024-02-19T19:04:57.875 INFO:tasks.workunit.client.1.smithi132.stdout:5/663: rename d8/d2e/fa5 to d8/d2e/dc4/dc8/daa/fe5 0 2024-02-19T19:04:57.875 INFO:tasks.workunit.client.1.smithi132.stdout:3/581: read d8/d13/d19/f49 [1313595,23822] 0 2024-02-19T19:04:57.875 INFO:tasks.workunit.client.1.smithi132.stdout:5/664: write d8/d23/d3b/f48 [690974,94774] 0 2024-02-19T19:04:57.877 INFO:tasks.workunit.client.1.smithi132.stdout:8/679: creat d4/dc/d1b/d8f/fed x:0 0 0 2024-02-19T19:04:57.877 INFO:tasks.workunit.client.1.smithi132.stdout:7/669: truncate d1b/d1e/d28/f55 804281 0 2024-02-19T19:04:57.879 INFO:tasks.workunit.client.1.smithi132.stdout:2/623: symlink d0/d4/db9/lc5 0 2024-02-19T19:04:57.879 INFO:tasks.workunit.client.1.smithi132.stdout:7/670: write f8 [2557953,126515] 0 2024-02-19T19:04:57.879 INFO:tasks.workunit.client.1.smithi132.stdout:8/680: chown d4/dc/d1b/d20/d62/d91/fc8 4487 1 2024-02-19T19:04:57.884 INFO:tasks.workunit.client.1.smithi132.stdout:3/582: dwrite d8/d13/d2b/f2c [0,4194304] 0 2024-02-19T19:04:57.885 INFO:tasks.workunit.client.1.smithi132.stdout:5/665: symlink d8/d2e/dc4/dc8/da9/le6 0 2024-02-19T19:04:57.885 INFO:tasks.workunit.client.1.smithi132.stdout:3/583: chown d8 5254 1 2024-02-19T19:04:57.886 INFO:tasks.workunit.client.1.smithi132.stdout:8/681: chown d4/dc/d1b/d20/d23/d68/d79 5127 1 2024-02-19T19:04:57.890 INFO:tasks.workunit.client.1.smithi132.stdout:9/708: creat d12/db6/ff5 x:0 0 0 2024-02-19T19:04:57.894 INFO:tasks.workunit.client.1.smithi132.stdout:2/624: dread d0/d19/d1e/f45 [0,4194304] 0 2024-02-19T19:04:57.895 INFO:tasks.workunit.client.1.smithi132.stdout:8/682: mknod d4/cee 0 2024-02-19T19:04:57.895 INFO:tasks.workunit.client.1.smithi132.stdout:1/552: write d1/d1e/d77/d2c/f38 [1400517,102987] 0 2024-02-19T19:04:57.896 INFO:tasks.workunit.client.1.smithi132.stdout:3/584: write d8/d13/d2f/f4d [1289579,53730] 0 2024-02-19T19:04:57.897 INFO:tasks.workunit.client.1.smithi132.stdout:9/709: write d12/d2a/d4c/d61/fc5 [173353,43457] 0 2024-02-19T19:04:57.897 INFO:tasks.workunit.client.1.smithi132.stdout:1/553: stat d1/d3a/d50 0 2024-02-19T19:04:57.898 INFO:tasks.workunit.client.1.smithi132.stdout:5/666: dwrite d8/d2e/d71/f82 [0,4194304] 0 2024-02-19T19:04:57.899 INFO:tasks.workunit.client.1.smithi132.stdout:8/683: readlink d4/dc/d1b/d20/d23/d68/d79/l89 0 2024-02-19T19:04:57.900 INFO:tasks.workunit.client.1.smithi132.stdout:1/554: write d1/d1e/d77/d78/f9d [2208588,76733] 0 2024-02-19T19:04:57.901 INFO:tasks.workunit.client.1.smithi132.stdout:8/684: dread - d4/d2d/fe7 zero size 2024-02-19T19:04:57.901 INFO:tasks.workunit.client.1.smithi132.stdout:3/585: write d8/d13/d2b/d69/daa/fb0 [782176,7661] 0 2024-02-19T19:04:57.903 INFO:tasks.workunit.client.1.smithi132.stdout:5/667: truncate d8/da/dae/fb5 198350 0 2024-02-19T19:04:57.903 INFO:tasks.workunit.client.1.smithi132.stdout:3/586: chown d8/d13/f37 124364335 1 2024-02-19T19:04:57.906 INFO:tasks.workunit.client.1.smithi132.stdout:5/668: stat d8/d2e/dc4/dc8/daa/fe5 0 2024-02-19T19:04:57.907 INFO:tasks.workunit.client.1.smithi132.stdout:3/587: mkdir d8/d13/d2b/d69/d92/db4 0 2024-02-19T19:04:57.909 INFO:tasks.workunit.client.1.smithi132.stdout:7/671: write d1b/d1e/d2b/f95 [2008246,6949] 0 2024-02-19T19:04:57.912 INFO:tasks.workunit.client.1.smithi132.stdout:1/555: link d1/f48 d1/d3a/fac 0 2024-02-19T19:04:57.913 INFO:tasks.workunit.client.1.smithi132.stdout:3/588: creat d8/d13/d2b/d69/d92/daf/fb5 x:0 0 0 2024-02-19T19:04:57.914 INFO:tasks.workunit.client.1.smithi132.stdout:7/672: creat d1b/d5a/d63/d6b/fcf x:0 0 0 2024-02-19T19:04:57.916 INFO:tasks.workunit.client.1.smithi132.stdout:1/556: rename d1/c2f to d1/d61/cad 0 2024-02-19T19:04:57.917 INFO:tasks.workunit.client.1.smithi132.stdout:3/589: creat d8/d13/d2f/fb6 x:0 0 0 2024-02-19T19:04:57.917 INFO:tasks.workunit.client.1.smithi132.stdout:9/710: write d12/d8d/d3a/d40/f66 [379493,17191] 0 2024-02-19T19:04:57.918 INFO:tasks.workunit.client.1.smithi132.stdout:7/673: mkdir d1b/d1e/d2b/dd0 0 2024-02-19T19:04:57.919 INFO:tasks.workunit.client.1.smithi132.stdout:2/625: truncate d0/d3c/d3f/f81 2987478 0 2024-02-19T19:04:57.921 INFO:tasks.workunit.client.1.smithi132.stdout:9/711: symlink d12/d2a/d4c/dad/db2/lf6 0 2024-02-19T19:04:57.922 INFO:tasks.workunit.client.1.smithi132.stdout:5/669: dread d8/d23/f45 [0,4194304] 0 2024-02-19T19:04:57.923 INFO:tasks.workunit.client.1.smithi132.stdout:3/590: mknod d8/d13/d2b/d69/d92/d34/d42/d6c/cb7 0 2024-02-19T19:04:57.923 INFO:tasks.workunit.client.1.smithi132.stdout:7/674: dread d1b/d1e/d2b/d62/d53/fa4 [0,4194304] 0 2024-02-19T19:04:57.925 INFO:tasks.workunit.client.1.smithi132.stdout:8/685: dread d4/fd [0,4194304] 0 2024-02-19T19:04:57.926 INFO:tasks.workunit.client.1.smithi132.stdout:2/626: mknod d0/d4/df/d40/dbd/cc6 0 2024-02-19T19:04:57.927 INFO:tasks.workunit.client.1.smithi132.stdout:9/712: mknod d12/d2a/cf7 0 2024-02-19T19:04:57.929 INFO:tasks.workunit.client.1.smithi132.stdout:7/675: rename d1b/d46/d76/dc3 to d1b/d46/d76/d94/dd1 0 2024-02-19T19:04:57.930 INFO:tasks.workunit.client.1.smithi132.stdout:9/713: dread - d12/d8d/d3a/d40/d4e/d77/d89/dc4/fe3 zero size 2024-02-19T19:04:57.933 INFO:tasks.workunit.client.1.smithi132.stdout:7/676: dread - d1b/d5a/d63/d6b/fcf zero size 2024-02-19T19:04:57.934 INFO:tasks.workunit.client.1.smithi132.stdout:9/714: chown d12/d2a/d88/db8 11318 1 2024-02-19T19:04:57.936 INFO:tasks.workunit.client.1.smithi132.stdout:2/627: dwrite d0/d4/d11/d2e/d5a/fb5 [0,4194304] 0 2024-02-19T19:04:57.937 INFO:tasks.workunit.client.1.smithi132.stdout:1/557: fdatasync d1/d1e/d77/f3c 0 2024-02-19T19:04:57.938 INFO:tasks.workunit.client.1.smithi132.stdout:3/591: rmdir d8/d13/d2b/d69/da6/da4 0 2024-02-19T19:04:57.945 INFO:tasks.workunit.client.1.smithi132.stdout:3/592: mkdir d8/d13/d2b/d60/db8 0 2024-02-19T19:04:57.945 INFO:tasks.workunit.client.1.smithi132.stdout:6/534: sync 2024-02-19T19:04:57.947 INFO:tasks.workunit.client.1.smithi132.stdout:8/686: dread d4/dc/d1b/d33/f4d [0,4194304] 0 2024-02-19T19:04:57.950 INFO:tasks.workunit.client.1.smithi132.stdout:9/715: getdents d12/d8d/d3a/d40/d4e/d77/d89 0 2024-02-19T19:04:57.952 INFO:tasks.workunit.client.1.smithi132.stdout:7/677: dwrite d1b/d5a/d63/d6b/d86/fac [0,4194304] 0 2024-02-19T19:04:57.957 INFO:tasks.workunit.client.1.smithi132.stdout:5/670: dwrite d8/d23/f6f [0,4194304] 0 2024-02-19T19:04:57.960 INFO:tasks.workunit.client.1.smithi132.stdout:9/716: creat d12/d8d/d3a/d40/d4e/d77/d89/dc4/ff8 x:0 0 0 2024-02-19T19:04:57.960 INFO:tasks.workunit.client.1.smithi132.stdout:1/558: dwrite d1/d3a/f80 [0,4194304] 0 2024-02-19T19:04:57.962 INFO:tasks.workunit.client.1.smithi132.stdout:7/678: write d1b/d1e/d41/f6d [5095972,125107] 0 2024-02-19T19:04:57.966 INFO:tasks.workunit.client.1.smithi132.stdout:5/671: readlink d8/d2e/lda 0 2024-02-19T19:04:57.967 INFO:tasks.workunit.client.1.smithi132.stdout:7/679: dread - d1b/d1e/d41/fbf zero size 2024-02-19T19:04:57.968 INFO:tasks.workunit.client.1.smithi132.stdout:2/628: dread d0/d19/d1e/f97 [0,4194304] 0 2024-02-19T19:04:57.969 INFO:tasks.workunit.client.1.smithi132.stdout:3/593: dwrite d8/d13/d2b/d69/da6/f5b [0,4194304] 0 2024-02-19T19:04:57.971 INFO:tasks.workunit.client.1.smithi132.stdout:6/535: dread d0/db/d25/d3a/d5d/d8f/f39 [0,4194304] 0 2024-02-19T19:04:57.971 INFO:tasks.workunit.client.1.smithi132.stdout:2/629: chown d0/d4/d11/d2e/d5a 0 1 2024-02-19T19:04:57.976 INFO:tasks.workunit.client.1.smithi132.stdout:9/717: truncate d12/d1b/f1c 6625613 0 2024-02-19T19:04:57.977 INFO:tasks.workunit.client.1.smithi132.stdout:2/630: write d0/d4/df/d7a/f88 [26910,52269] 0 2024-02-19T19:04:57.978 INFO:tasks.workunit.client.1.smithi132.stdout:7/680: dread d1b/d1e/d28/f77 [0,4194304] 0 2024-02-19T19:04:57.979 INFO:tasks.workunit.client.1.smithi132.stdout:5/672: chown d8/d2e/d71/ldd 488 1 2024-02-19T19:04:57.980 INFO:tasks.workunit.client.1.smithi132.stdout:3/594: dwrite d8/fd [0,4194304] 0 2024-02-19T19:04:57.980 INFO:tasks.workunit.client.1.smithi132.stdout:9/718: mknod d12/db6/cf9 0 2024-02-19T19:04:57.982 INFO:tasks.workunit.client.1.smithi132.stdout:1/559: write d1/d1e/d77/d2c/d89/f8d [432780,32714] 0 2024-02-19T19:04:57.984 INFO:tasks.workunit.client.1.smithi132.stdout:2/631: rmdir d0/d4/df/d40 39 2024-02-19T19:04:57.986 INFO:tasks.workunit.client.1.smithi132.stdout:7/681: mkdir d1b/d1e/d28/dd2 0 2024-02-19T19:04:57.987 INFO:tasks.workunit.client.1.smithi132.stdout:5/673: creat d8/d23/d29/d27/fe7 x:0 0 0 2024-02-19T19:04:57.990 INFO:tasks.workunit.client.1.smithi132.stdout:2/632: rmdir d0/d4/d11 39 2024-02-19T19:04:57.991 INFO:tasks.workunit.client.1.smithi132.stdout:5/674: mknod d8/d2e/ce8 0 2024-02-19T19:04:57.992 INFO:tasks.workunit.client.1.smithi132.stdout:6/536: dwrite d0/db/d25/d3a/d3d/fa4 [0,4194304] 0 2024-02-19T19:04:57.996 INFO:tasks.workunit.client.1.smithi132.stdout:3/595: dwrite d8/d13/d2f/f58 [0,4194304] 0 2024-02-19T19:04:57.997 INFO:tasks.workunit.client.1.smithi132.stdout:7/682: truncate d1b/d1e/d2b/d62/d53/fa4 1129824 0 2024-02-19T19:04:57.999 INFO:tasks.workunit.client.1.smithi132.stdout:6/537: chown d0/db/d25/d38/l3e 1626 1 2024-02-19T19:04:57.999 INFO:tasks.workunit.client.1.smithi132.stdout:2/633: mknod d0/d4/df/d40/d68/cc7 0 2024-02-19T19:04:58.001 INFO:tasks.workunit.client.1.smithi132.stdout:5/675: getdents d8/d2e/dc4/dc8/ddc 0 2024-02-19T19:04:58.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:57 smithi132 ceph-mon[37969]: mgrmap e34: smithi088.gyviqq(active, since 3m), standbys: smithi132.psllsa 2024-02-19T19:04:58.002 INFO:tasks.workunit.client.1.smithi132.stdout:7/683: mkdir d1b/d1e/d28/da9/dd3 0 2024-02-19T19:04:58.003 INFO:tasks.workunit.client.1.smithi132.stdout:3/596: mknod d8/d13/d2b/d69/d92/d34/d42/d6c/cb9 0 2024-02-19T19:04:58.004 INFO:tasks.workunit.client.1.smithi132.stdout:3/597: readlink d8/d13/d2b/d60/l68 0 2024-02-19T19:04:58.008 INFO:tasks.workunit.client.1.smithi132.stdout:6/538: fdatasync d0/db/d25/d3a/d5d/d8f/d22/f59 0 2024-02-19T19:04:58.011 INFO:tasks.workunit.client.1.smithi132.stdout:5/676: mkdir d8/d23/d29/dba/de9 0 2024-02-19T19:04:58.011 INFO:tasks.workunit.client.1.smithi132.stdout:9/719: dread d12/d3e/d6e/f79 [4194304,4194304] 0 2024-02-19T19:04:58.011 INFO:tasks.workunit.client.1.smithi132.stdout:1/560: write d1/d1e/d77/d4e/f6d [1889523,130690] 0 2024-02-19T19:04:58.013 INFO:tasks.workunit.client.1.smithi132.stdout:7/684: unlink d1b/d1e/d28/f77 0 2024-02-19T19:04:58.013 INFO:tasks.workunit.client.1.smithi132.stdout:5/677: truncate d8/d2e/d71/f80 11593 0 2024-02-19T19:04:58.014 INFO:tasks.workunit.client.1.smithi132.stdout:2/634: mkdir d0/d64/dc8 0 2024-02-19T19:04:58.015 INFO:tasks.workunit.client.1.smithi132.stdout:3/598: chown d8/d13/d2b/d69/da6/l8f 5944 1 2024-02-19T19:04:58.017 INFO:tasks.workunit.client.1.smithi132.stdout:6/539: write d0/db/f58 [4206971,94982] 0 2024-02-19T19:04:58.018 INFO:tasks.workunit.client.1.smithi132.stdout:1/561: mknod d1/d3a/d50/cae 0 2024-02-19T19:04:58.020 INFO:tasks.workunit.client.1.smithi132.stdout:7/685: symlink d1b/d46/d76/d94/ld4 0 2024-02-19T19:04:58.022 INFO:tasks.workunit.client.1.smithi132.stdout:6/540: symlink d0/db/d25/d3a/d4b/lb4 0 2024-02-19T19:04:58.023 INFO:tasks.workunit.client.1.smithi132.stdout:1/562: creat d1/d3a/d50/faf x:0 0 0 2024-02-19T19:04:58.023 INFO:tasks.workunit.client.1.smithi132.stdout:5/678: mkdir d8/d2e/d71/da6/ddb/dea 0 2024-02-19T19:04:58.027 INFO:tasks.workunit.client.1.smithi132.stdout:7/686: dwrite d1b/d5a/d63/d6b/d86/fac [0,4194304] 0 2024-02-19T19:04:58.029 INFO:tasks.workunit.client.1.smithi132.stdout:1/563: creat d1/d1e/d77/d52/d64/fb0 x:0 0 0 2024-02-19T19:04:58.030 INFO:tasks.workunit.client.1.smithi132.stdout:6/541: creat d0/db/d25/d3a/d3d/fb5 x:0 0 0 2024-02-19T19:04:58.033 INFO:tasks.workunit.client.1.smithi132.stdout:6/542: creat d0/db/d25/d3a/d3d/d69/d89/fb6 x:0 0 0 2024-02-19T19:04:58.034 INFO:tasks.workunit.client.1.smithi132.stdout:1/564: mknod d1/d95/cb1 0 2024-02-19T19:04:58.035 INFO:tasks.workunit.client.1.smithi132.stdout:3/599: write d8/d13/d2b/d69/d92/f8c [1747706,38442] 0 2024-02-19T19:04:58.035 INFO:tasks.workunit.client.1.smithi132.stdout:7/687: getdents d1b/d46/d76/d94 0 2024-02-19T19:04:58.036 INFO:tasks.workunit.client.1.smithi132.stdout:2/635: truncate d0/d24/d58/d53/dab/d8a/fa4 1081346 0 2024-02-19T19:04:58.036 INFO:tasks.workunit.client.1.smithi132.stdout:6/543: truncate d0/db/d25/d3a/d3d/d69/d89/fb6 510136 0 2024-02-19T19:04:58.037 INFO:tasks.workunit.client.1.smithi132.stdout:3/600: fdatasync d8/d13/d2b/d69/d92/f55 0 2024-02-19T19:04:58.037 INFO:tasks.workunit.client.1.smithi132.stdout:1/565: mkdir d1/d3a/d50/db2 0 2024-02-19T19:04:58.038 INFO:tasks.workunit.client.1.smithi132.stdout:2/636: write d0/d19/d6a/fc4 [258423,96293] 0 2024-02-19T19:04:58.040 INFO:tasks.workunit.client.1.smithi132.stdout:3/601: creat d8/d38/fba x:0 0 0 2024-02-19T19:04:58.042 INFO:tasks.workunit.client.1.smithi132.stdout:1/566: symlink d1/d1e/d77/d2c/d62/d6b/d85/lb3 0 2024-02-19T19:04:58.043 INFO:tasks.workunit.client.1.smithi132.stdout:6/544: creat d0/fb7 x:0 0 0 2024-02-19T19:04:58.044 INFO:tasks.workunit.client.1.smithi132.stdout:2/637: mkdir d0/d24/d58/d53/dab/d84/dc9 0 2024-02-19T19:04:58.045 INFO:tasks.workunit.client.1.smithi132.stdout:5/679: dwrite d8/da/f33 [0,4194304] 0 2024-02-19T19:04:58.045 INFO:tasks.workunit.client.1.smithi132.stdout:1/567: readlink d1/d1e/d77/d2c/d43/l92 0 2024-02-19T19:04:58.047 INFO:tasks.workunit.client.1.smithi132.stdout:7/688: dwrite ff [0,4194304] 0 2024-02-19T19:04:58.047 INFO:tasks.workunit.client.1.smithi132.stdout:5/680: dread - d8/d23/d29/f7f zero size 2024-02-19T19:04:58.048 INFO:tasks.workunit.client.1.smithi132.stdout:3/602: unlink d8/d13/d2f/fac 0 2024-02-19T19:04:58.049 INFO:tasks.workunit.client.1.smithi132.stdout:6/545: mknod d0/d8b/cb8 0 2024-02-19T19:04:58.050 INFO:tasks.workunit.client.1.smithi132.stdout:3/603: write d8/d13/d2f/f9a [448386,49200] 0 2024-02-19T19:04:58.051 INFO:tasks.workunit.client.1.smithi132.stdout:5/681: fsync d8/da/fb9 0 2024-02-19T19:04:58.053 INFO:tasks.workunit.client.1.smithi132.stdout:5/682: chown d8/da/le1 1 1 2024-02-19T19:04:58.053 INFO:tasks.workunit.client.1.smithi132.stdout:7/689: dread ff [0,4194304] 0 2024-02-19T19:04:58.054 INFO:tasks.workunit.client.1.smithi132.stdout:5/683: readlink d8/d1a/lc0 0 2024-02-19T19:04:58.055 INFO:tasks.workunit.client.1.smithi132.stdout:5/684: write d8/d1a/f9c [791356,121618] 0 2024-02-19T19:04:58.058 INFO:tasks.workunit.client.1.smithi132.stdout:1/568: dread d1/d3a/f30 [0,4194304] 0 2024-02-19T19:04:58.059 INFO:tasks.workunit.client.1.smithi132.stdout:5/685: dread d8/da/dae/fb5 [0,4194304] 0 2024-02-19T19:04:58.059 INFO:tasks.workunit.client.1.smithi132.stdout:5/686: readlink l7 0 2024-02-19T19:04:58.064 INFO:tasks.workunit.client.1.smithi132.stdout:2/638: link d0/d24/d58/d53/dab/d84/lbb d0/d4/df/dc2/lca 0 2024-02-19T19:04:58.065 INFO:tasks.workunit.client.1.smithi132.stdout:5/687: dwrite d8/da/db/f34 [0,4194304] 0 2024-02-19T19:04:58.075 INFO:tasks.workunit.client.1.smithi132.stdout:3/604: getdents d8/d13/d2b/d69/d92 0 2024-02-19T19:04:58.076 INFO:tasks.workunit.client.1.smithi132.stdout:1/569: unlink d1/d95/d97/la9 0 2024-02-19T19:04:58.077 INFO:tasks.workunit.client.1.smithi132.stdout:7/690: dread f8 [0,4194304] 0 2024-02-19T19:04:58.077 INFO:tasks.workunit.client.1.smithi132.stdout:2/639: mkdir d0/d24/dcb 0 2024-02-19T19:04:58.080 INFO:tasks.workunit.client.1.smithi132.stdout:1/570: unlink d1/d1e/f5b 0 2024-02-19T19:04:58.082 INFO:tasks.workunit.client.1.smithi132.stdout:5/688: creat d8/d23/feb x:0 0 0 2024-02-19T19:04:58.082 INFO:tasks.workunit.client.1.smithi132.stdout:1/571: stat d1/d1e/d77/d78 0 2024-02-19T19:04:58.083 INFO:tasks.workunit.client.1.smithi132.stdout:7/691: mkdir d1b/d1e/d41/da3/dd5 0 2024-02-19T19:04:58.084 INFO:tasks.workunit.client.1.smithi132.stdout:6/546: dwrite d0/db/d25/d38/f78 [4194304,4194304] 0 2024-02-19T19:04:58.085 INFO:tasks.workunit.client.1.smithi132.stdout:3/605: getdents d8/d13/d2b/d69/d92/d34 0 2024-02-19T19:04:58.088 INFO:tasks.workunit.client.1.smithi132.stdout:1/572: write d1/d1e/d77/d2c/d62/d6b/d85/f94 [961562,575] 0 2024-02-19T19:04:58.088 INFO:tasks.workunit.client.1.smithi132.stdout:5/689: mkdir d8/d1a/d64/de2/dec 0 2024-02-19T19:04:58.091 INFO:tasks.workunit.client.1.smithi132.stdout:2/640: dwrite d0/d4/df/d40/d68/f8f [0,4194304] 0 2024-02-19T19:04:58.093 INFO:tasks.workunit.client.1.smithi132.stdout:7/692: unlink d1b/d5a/d63/d6b/d79/l93 0 2024-02-19T19:04:58.094 INFO:tasks.workunit.client.1.smithi132.stdout:6/547: unlink d0/db/d25/d3a/d4b/d7c/f91 0 2024-02-19T19:04:58.094 INFO:tasks.workunit.client.1.smithi132.stdout:5/690: dread d8/ff [4194304,4194304] 0 2024-02-19T19:04:58.099 INFO:tasks.workunit.client.1.smithi132.stdout:1/573: creat d1/d1e/d77/d2c/d62/fb4 x:0 0 0 2024-02-19T19:04:58.100 INFO:tasks.workunit.client.1.smithi132.stdout:1/574: chown d1/d3a/d50/f67 12716 1 2024-02-19T19:04:58.103 INFO:tasks.workunit.client.1.smithi132.stdout:7/693: creat d1b/d1e/fd6 x:0 0 0 2024-02-19T19:04:58.104 INFO:tasks.workunit.client.1.smithi132.stdout:5/691: symlink d8/d1a/d35/led 0 2024-02-19T19:04:58.106 INFO:tasks.workunit.client.1.smithi132.stdout:3/606: rename d8/d13/d2b/c85 to d8/d13/cbb 0 2024-02-19T19:04:58.107 INFO:tasks.workunit.client.1.smithi132.stdout:6/548: fdatasync d0/db/d25/d3a/d5d/d8f/d46/d5a/f26 0 2024-02-19T19:04:58.108 INFO:tasks.workunit.client.1.smithi132.stdout:6/549: fdatasync d0/db/d25/d3a/d3d/d69/d89/fb6 0 2024-02-19T19:04:58.110 INFO:tasks.workunit.client.1.smithi132.stdout:5/692: creat d8/d2e/d71/da6/fee x:0 0 0 2024-02-19T19:04:58.111 INFO:tasks.workunit.client.1.smithi132.stdout:5/693: chown d8/d1a/d64 22000 1 2024-02-19T19:04:58.115 INFO:tasks.workunit.client.1.smithi132.stdout:7/694: rename d1b/d46/d7f/cce to d1b/d1e/d41/da3/dc8/cd7 0 2024-02-19T19:04:58.117 INFO:tasks.workunit.client.1.smithi132.stdout:3/607: dread d8/d38/f4b [0,4194304] 0 2024-02-19T19:04:58.122 INFO:tasks.workunit.client.1.smithi132.stdout:5/694: rmdir d8/da/d5a/d84 39 2024-02-19T19:04:58.126 INFO:tasks.workunit.client.1.smithi132.stdout:6/550: creat d0/db/d25/d3a/d5d/d8f/d46/fb9 x:0 0 0 2024-02-19T19:04:58.126 INFO:tasks.workunit.client.1.smithi132.stdout:2/641: write d0/d4/df/f5f [69119,44941] 0 2024-02-19T19:04:58.126 INFO:tasks.workunit.client.1.smithi132.stdout:1/575: write d1/d1e/d77/d4e/f5f [7534422,59803] 0 2024-02-19T19:04:58.128 INFO:tasks.workunit.client.1.smithi132.stdout:1/576: fdatasync d1/d1e/d77/d2c/d89/f9c 0 2024-02-19T19:04:58.129 INFO:tasks.workunit.client.1.smithi132.stdout:2/642: dread - d0/d24/d58/d53/fbf zero size 2024-02-19T19:04:58.130 INFO:tasks.workunit.client.1.smithi132.stdout:0/571: sync 2024-02-19T19:04:58.131 INFO:tasks.workunit.client.1.smithi132.stdout:7/695: write d1b/f25 [1265149,67518] 0 2024-02-19T19:04:58.132 INFO:tasks.workunit.client.1.smithi132.stdout:2/643: write d0/d4/f8e [2455000,62017] 0 2024-02-19T19:04:58.132 INFO:tasks.workunit.client.1.smithi132.stdout:3/608: getdents d8/d13/d2b/d69/d7e/d9f 0 2024-02-19T19:04:58.134 INFO:tasks.workunit.client.1.smithi132.stdout:5/695: dwrite d8/d23/d29/fa1 [0,4194304] 0 2024-02-19T19:04:58.136 INFO:tasks.workunit.client.1.smithi132.stdout:0/572: symlink d2/dc/d4a/d25/d43/lbe 0 2024-02-19T19:04:58.137 INFO:tasks.workunit.client.1.smithi132.stdout:0/573: chown d2/dc/d4a/d25/d43/d5f/f87 5858 1 2024-02-19T19:04:58.139 INFO:tasks.workunit.client.1.smithi132.stdout:7/696: symlink d1b/d1e/d41/da3/dbd/ld8 0 2024-02-19T19:04:58.140 INFO:tasks.workunit.client.1.smithi132.stdout:2/644: mknod d0/ccc 0 2024-02-19T19:04:58.141 INFO:tasks.workunit.client.1.smithi132.stdout:5/696: dwrite d8/d23/d29/d27/f57 [0,4194304] 0 2024-02-19T19:04:58.146 INFO:tasks.workunit.client.1.smithi132.stdout:1/577: rename d1/d61/la4 to d1/d1e/d77/d4e/lb5 0 2024-02-19T19:04:58.146 INFO:tasks.workunit.client.1.smithi132.stdout:3/609: symlink d8/d13/d2b/d69/d92/d34/d42/lbc 0 2024-02-19T19:04:58.146 INFO:tasks.workunit.client.1.smithi132.stdout:6/551: link d0/c1c d0/db/d25/d3a/d3d/d69/d89/cba 0 2024-02-19T19:04:58.147 INFO:tasks.workunit.client.1.smithi132.stdout:0/574: symlink d2/dc/d4a/d25/d7f/lbf 0 2024-02-19T19:04:58.148 INFO:tasks.workunit.client.1.smithi132.stdout:5/697: mkdir d8/d23/d29/def 0 2024-02-19T19:04:58.149 INFO:tasks.workunit.client.1.smithi132.stdout:7/697: mknod d1b/d5a/d63/d96/dcc/cd9 0 2024-02-19T19:04:58.149 INFO:tasks.workunit.client.1.smithi132.stdout:2/645: dwrite d0/d4/df/d40/d68/f8f [0,4194304] 0 2024-02-19T19:04:58.152 INFO:tasks.workunit.client.1.smithi132.stdout:2/646: fsync d0/d4/df/f5f 0 2024-02-19T19:04:58.152 INFO:tasks.workunit.client.1.smithi132.stdout:3/610: truncate d8/d13/d2b/d69/d92/d34/f79 98498 0 2024-02-19T19:04:58.154 INFO:tasks.workunit.client.1.smithi132.stdout:5/698: mkdir d8/d2e/d71/dce/df0 0 2024-02-19T19:04:58.154 INFO:tasks.workunit.client.1.smithi132.stdout:7/698: creat d1b/d1e/d28/d5e/fda x:0 0 0 2024-02-19T19:04:58.155 INFO:tasks.workunit.client.1.smithi132.stdout:3/611: write d8/d13/d19/f41 [4672195,52817] 0 2024-02-19T19:04:58.156 INFO:tasks.workunit.client.1.smithi132.stdout:7/699: chown d1b/d1e/c26 184520575 1 2024-02-19T19:04:58.159 INFO:tasks.workunit.client.1.smithi132.stdout:3/612: readlink d8/d93/l72 0 2024-02-19T19:04:58.160 INFO:tasks.workunit.client.1.smithi132.stdout:5/699: mkdir d8/d2e/dc4/dc8/df1 0 2024-02-19T19:04:58.160 INFO:tasks.workunit.client.1.smithi132.stdout:1/578: dwrite d1/d1e/d77/d78/f7f [0,4194304] 0 2024-02-19T19:04:58.164 INFO:tasks.workunit.client.1.smithi132.stdout:3/613: dread d8/d38/f8b [0,4194304] 0 2024-02-19T19:04:58.166 INFO:tasks.workunit.client.1.smithi132.stdout:1/579: write d1/d1e/d77/f3c [4857803,61683] 0 2024-02-19T19:04:58.169 INFO:tasks.workunit.client.1.smithi132.stdout:3/614: chown d8/d13/d2b/d69/d92/d34/f89 1428 1 2024-02-19T19:04:58.169 INFO:tasks.workunit.client.1.smithi132.stdout:2/647: dwrite d0/d24/d58/d53/dab/d84/fa1 [0,4194304] 0 2024-02-19T19:04:58.171 INFO:tasks.workunit.client.1.smithi132.stdout:3/615: write d8/d38/d44/f4c [4229105,65573] 0 2024-02-19T19:04:58.171 INFO:tasks.workunit.client.1.smithi132.stdout:5/700: truncate d8/d23/d3b/f65 1009333 0 2024-02-19T19:04:58.173 INFO:tasks.workunit.client.1.smithi132.stdout:5/701: write d8/da/d5a/d7b/f99 [652009,121956] 0 2024-02-19T19:04:58.176 INFO:tasks.workunit.client.1.smithi132.stdout:2/648: unlink d0/d4/df/d40/d68/cc7 0 2024-02-19T19:04:58.179 INFO:tasks.workunit.client.1.smithi132.stdout:7/700: write d1b/d1e/d28/f4c [1689877,116102] 0 2024-02-19T19:04:58.180 INFO:tasks.workunit.client.1.smithi132.stdout:3/616: mknod d8/d13/d2b/d69/da6/cbd 0 2024-02-19T19:04:58.181 INFO:tasks.workunit.client.1.smithi132.stdout:5/702: symlink d8/d2e/dc4/dc8/daa/lf2 0 2024-02-19T19:04:58.184 INFO:tasks.workunit.client.1.smithi132.stdout:2/649: fdatasync d0/d4/d11/d2e/f75 0 2024-02-19T19:04:58.186 INFO:tasks.workunit.client.1.smithi132.stdout:5/703: read d8/d1a/f5e [743765,74850] 0 2024-02-19T19:04:58.187 INFO:tasks.workunit.client.1.smithi132.stdout:1/580: truncate d1/d3a/f80 20823 0 2024-02-19T19:04:58.192 INFO:tasks.workunit.client.1.smithi132.stdout:5/704: dread d8/da/db/f4f [0,4194304] 0 2024-02-19T19:04:58.194 INFO:tasks.workunit.client.1.smithi132.stdout:6/552: dread d0/db/d25/f8d [0,4194304] 0 2024-02-19T19:04:58.196 INFO:tasks.workunit.client.1.smithi132.stdout:1/581: creat d1/d1e/d77/d2c/d62/d6b/fb6 x:0 0 0 2024-02-19T19:04:58.197 INFO:tasks.workunit.client.1.smithi132.stdout:2/650: mknod d0/d4/d11/ccd 0 2024-02-19T19:04:58.198 INFO:tasks.workunit.client.1.smithi132.stdout:2/651: fdatasync d0/d4/df/d40/d68/f86 0 2024-02-19T19:04:58.199 INFO:tasks.workunit.client.1.smithi132.stdout:3/617: link d8/d13/l82 d8/d13/d2b/d69/d92/lbe 0 2024-02-19T19:04:58.199 INFO:tasks.workunit.client.1.smithi132.stdout:2/652: chown d0/l73 174047923 1 2024-02-19T19:04:58.202 INFO:tasks.workunit.client.1.smithi132.stdout:1/582: creat d1/d1e/d77/d4e/fb7 x:0 0 0 2024-02-19T19:04:58.203 INFO:tasks.workunit.client.1.smithi132.stdout:7/701: dwrite d1b/d1e/d28/f44 [0,4194304] 0 2024-02-19T19:04:58.204 INFO:tasks.workunit.client.1.smithi132.stdout:2/653: dread d0/d4/df/f5f [0,4194304] 0 2024-02-19T19:04:58.205 INFO:tasks.workunit.client.1.smithi132.stdout:6/553: getdents d0/db/d25/d3a/d5d/d8f/d87 0 2024-02-19T19:04:58.207 INFO:tasks.workunit.client.1.smithi132.stdout:3/618: dwrite d8/d13/d2f/f58 [0,4194304] 0 2024-02-19T19:04:58.208 INFO:tasks.workunit.client.1.smithi132.stdout:3/619: chown d8/d13/d2f/f58 56892 1 2024-02-19T19:04:58.209 INFO:tasks.workunit.client.1.smithi132.stdout:5/705: getdents d8/d2e/d71 0 2024-02-19T19:04:58.213 INFO:tasks.workunit.client.1.smithi132.stdout:7/702: mkdir d1b/d1e/d28/da9/dd3/ddb 0 2024-02-19T19:04:58.215 INFO:tasks.workunit.client.1.smithi132.stdout:6/554: rename d0/db/d25/f8d to d0/db/d25/d50/d86/fbb 0 2024-02-19T19:04:58.218 INFO:tasks.workunit.client.1.smithi132.stdout:3/620: unlink d8/d13/d57/f98 0 2024-02-19T19:04:58.224 INFO:tasks.workunit.client.1.smithi132.stdout:7/703: mknod d1b/d5a/d63/cdc 0 2024-02-19T19:04:58.224 INFO:tasks.workunit.client.1.smithi132.stdout:3/621: dwrite d8/d13/d2b/d69/d92/daf/fb5 [0,4194304] 0 2024-02-19T19:04:58.233 INFO:tasks.workunit.client.1.smithi132.stdout:3/622: creat d8/d13/d2b/d69/d92/daf/fbf x:0 0 0 2024-02-19T19:04:58.236 INFO:tasks.workunit.client.1.smithi132.stdout:2/654: getdents d0/d19/d6a 0 2024-02-19T19:04:58.237 INFO:tasks.workunit.client.1.smithi132.stdout:5/706: dwrite d8/d1a/f9d [0,4194304] 0 2024-02-19T19:04:58.239 INFO:tasks.workunit.client.1.smithi132.stdout:3/623: truncate d8/d13/d2b/d69/d92/d34/f79 892889 0 2024-02-19T19:04:58.239 INFO:tasks.workunit.client.1.smithi132.stdout:2/655: read d0/d4/df/f5f [523086,82887] 0 2024-02-19T19:04:58.239 INFO:tasks.workunit.client.1.smithi132.stdout:5/707: dread - d8/da/db/fbb zero size 2024-02-19T19:04:58.239 INFO:tasks.workunit.client.1.smithi132.stdout:6/555: getdents d0/db/d25/d3a/d4b/d7c/da2 0 2024-02-19T19:04:58.241 INFO:tasks.workunit.client.1.smithi132.stdout:6/556: write d0/db/f9d [21470,102418] 0 2024-02-19T19:04:58.241 INFO:tasks.workunit.client.1.smithi132.stdout:5/708: chown d8/d2e/dc4/dc8/daa/f93 10043395 1 2024-02-19T19:04:58.243 INFO:tasks.workunit.client.1.smithi132.stdout:5/709: readlink d8/d1a/d35/led 0 2024-02-19T19:04:58.244 INFO:tasks.workunit.client.1.smithi132.stdout:5/710: chown d8/d2e/dc4/dc8 2054065330 1 2024-02-19T19:04:58.244 INFO:tasks.workunit.client.1.smithi132.stdout:6/557: read d0/db/d25/d3a/d3d/d69/d89/fb6 [417389,31607] 0 2024-02-19T19:04:58.244 INFO:tasks.workunit.client.1.smithi132.stdout:6/558: readlink d0/db/d25/d3a/d4b/lac 0 2024-02-19T19:04:58.245 INFO:tasks.workunit.client.1.smithi132.stdout:2/656: dread d0/d4/d11/d2e/d5a/fb5 [0,4194304] 0 2024-02-19T19:04:58.247 INFO:tasks.workunit.client.1.smithi132.stdout:3/624: rename d8/d38/d44/fae to d8/fc0 0 2024-02-19T19:04:58.248 INFO:tasks.workunit.client.1.smithi132.stdout:3/625: chown d8/d13/d2b/d60/l68 956208 1 2024-02-19T19:04:58.249 INFO:tasks.workunit.client.1.smithi132.stdout:5/711: write d8/da/db/f44 [1523159,21008] 0 2024-02-19T19:04:58.250 INFO:tasks.workunit.client.1.smithi132.stdout:2/657: readlink d0/d19/l27 0 2024-02-19T19:04:58.252 INFO:tasks.workunit.client.1.smithi132.stdout:3/626: creat d8/d13/d2f/fc1 x:0 0 0 2024-02-19T19:04:58.253 INFO:tasks.workunit.client.1.smithi132.stdout:5/712: symlink d8/lf3 0 2024-02-19T19:04:58.254 INFO:tasks.workunit.client.1.smithi132.stdout:2/658: creat d0/d4/df/d7a/fce x:0 0 0 2024-02-19T19:04:58.255 INFO:tasks.workunit.client.1.smithi132.stdout:6/559: rename d0/db/d25/d3a/d3d/ca9 to d0/db/d25/d3a/d5d/d8f/cbc 0 2024-02-19T19:04:58.255 INFO:tasks.workunit.client.1.smithi132.stdout:3/627: mkdir d8/d13/d2b/d60/dc2 0 2024-02-19T19:04:58.265 INFO:tasks.workunit.client.1.smithi132.stdout:6/560: getdents d0/db/d25/d3a/d4b 0 2024-02-19T19:04:58.266 INFO:tasks.workunit.client.1.smithi132.stdout:9/720: sync 2024-02-19T19:04:58.266 INFO:tasks.workunit.client.1.smithi132.stdout:8/687: sync 2024-02-19T19:04:58.267 INFO:tasks.workunit.client.1.smithi132.stdout:4/589: sync 2024-02-19T19:04:58.267 INFO:tasks.workunit.client.1.smithi132.stdout:9/721: fdatasync d12/d2a/d4c/d61/fa8 0 2024-02-19T19:04:58.268 INFO:tasks.workunit.client.1.smithi132.stdout:4/590: write d4/d19/d21/d56/f8f [604509,2561] 0 2024-02-19T19:04:58.268 INFO:tasks.workunit.client.1.smithi132.stdout:3/628: getdents d8/d13/d57 0 2024-02-19T19:04:58.269 INFO:tasks.workunit.client.1.smithi132.stdout:2/659: link d0/d4/df/dc2/dad/l95 d0/d19/lcf 0 2024-02-19T19:04:58.270 INFO:tasks.workunit.client.1.smithi132.stdout:9/722: chown d12/d2a/d4c/d61/ce8 0 1 2024-02-19T19:04:58.271 INFO:tasks.workunit.client.1.smithi132.stdout:5/713: dread f5 [12582912,4194304] 0 2024-02-19T19:04:58.273 INFO:tasks.workunit.client.1.smithi132.stdout:2/660: mkdir d0/d24/dd0 0 2024-02-19T19:04:58.276 INFO:tasks.workunit.client.1.smithi132.stdout:4/591: creat d4/d12/d4c/fce x:0 0 0 2024-02-19T19:04:58.276 INFO:tasks.workunit.client.1.smithi132.stdout:5/714: write d8/d2e/dc4/dc8/daa/fe5 [622754,11496] 0 2024-02-19T19:04:58.279 INFO:tasks.workunit.client.1.smithi132.stdout:3/629: link d8/d13/d2b/d69/d92/d34/f81 d8/d13/d2b/d69/d92/daf/fc3 0 2024-02-19T19:04:58.280 INFO:tasks.workunit.client.1.smithi132.stdout:3/630: write d8/d13/d2b/d69/d92/f55 [2955851,87507] 0 2024-02-19T19:04:58.281 INFO:tasks.workunit.client.1.smithi132.stdout:5/715: write d8/d1a/d35/f4b [944339,54770] 0 2024-02-19T19:04:58.283 INFO:tasks.workunit.client.1.smithi132.stdout:4/592: mknod d4/d27/d31/d32/d51/ccf 0 2024-02-19T19:04:58.284 INFO:tasks.workunit.client.1.smithi132.stdout:3/631: symlink d8/d38/d44/d6e/lc4 0 2024-02-19T19:04:58.347 INFO:tasks.workunit.client.1.smithi132.stdout:9/723: getdents d12/d2a/d88 0 2024-02-19T19:04:58.349 INFO:tasks.workunit.client.1.smithi132.stdout:5/716: symlink d8/d2e/dc4/dc8/df1/lf4 0 2024-02-19T19:04:58.353 INFO:tasks.workunit.client.1.smithi132.stdout:2/661: dread d0/d19/f25 [0,4194304] 0 2024-02-19T19:04:58.353 INFO:tasks.workunit.client.1.smithi132.stdout:3/632: rename d8/l27 to d8/d13/d2b/d69/da6/lc5 0 2024-02-19T19:04:58.356 INFO:tasks.workunit.client.1.smithi132.stdout:3/633: dread d8/d13/d2b/d69/d92/d34/f79 [0,4194304] 0 2024-02-19T19:04:58.356 INFO:tasks.workunit.client.1.smithi132.stdout:9/724: mknod d12/d8d/d3a/cfa 0 2024-02-19T19:04:58.357 INFO:tasks.workunit.client.1.smithi132.stdout:3/634: stat d8/d13/d2b/fa9 0 2024-02-19T19:04:58.358 INFO:tasks.workunit.client.1.smithi132.stdout:5/717: mkdir d8/d2e/dc4/dc8/ddc/df5 0 2024-02-19T19:04:58.359 INFO:tasks.workunit.client.1.smithi132.stdout:4/593: dwrite d4/d27/d31/d32/d51/d5c/dbc/fbd [0,4194304] 0 2024-02-19T19:04:58.361 INFO:tasks.workunit.client.1.smithi132.stdout:3/635: readlink d8/d13/d2b/d69/da6/l5f 0 2024-02-19T19:04:58.364 INFO:tasks.workunit.client.1.smithi132.stdout:2/662: creat d0/d4/fd1 x:0 0 0 2024-02-19T19:04:58.364 INFO:tasks.workunit.client.1.smithi132.stdout:4/594: rmdir d4/d27/d31/d71/d37 39 2024-02-19T19:04:58.367 INFO:tasks.workunit.client.1.smithi132.stdout:5/718: rename d8/da/db/l19 to d8/da/d5a/d7b/d96/d9e/lf6 0 2024-02-19T19:04:58.368 INFO:tasks.workunit.client.1.smithi132.stdout:9/725: dwrite d12/d8d/d3a/f52 [0,4194304] 0 2024-02-19T19:04:58.369 INFO:tasks.workunit.client.1.smithi132.stdout:3/636: dread d8/d13/d2f/f58 [0,4194304] 0 2024-02-19T19:04:58.369 INFO:tasks.workunit.client.1.smithi132.stdout:4/595: creat d4/d12/da4/fd0 x:0 0 0 2024-02-19T19:04:58.373 INFO:tasks.workunit.client.1.smithi132.stdout:2/663: symlink d0/d24/d58/d53/dab/d8a/da5/db1/ld2 0 2024-02-19T19:04:58.373 INFO:tasks.workunit.client.1.smithi132.stdout:3/637: rename d8/d13 to d8/d13/d19/dc6 22 2024-02-19T19:04:58.373 INFO:tasks.workunit.client.1.smithi132.stdout:9/726: write d12/d8d/d3a/d40/d4e/d77/d89/dc4/fee [823226,127858] 0 2024-02-19T19:04:58.373 INFO:tasks.workunit.client.1.smithi132.stdout:5/719: mknod d8/d2e/d71/da6/ddb/cf7 0 2024-02-19T19:04:58.374 INFO:tasks.workunit.client.1.smithi132.stdout:2/664: write d0/f6 [4231571,102599] 0 2024-02-19T19:04:58.378 INFO:tasks.workunit.client.1.smithi132.stdout:5/720: write d8/d23/d29/d27/f87 [1024899,7993] 0 2024-02-19T19:04:58.378 INFO:tasks.workunit.client.1.smithi132.stdout:3/638: creat d8/d13/d2b/d69/da6/fc7 x:0 0 0 2024-02-19T19:04:58.378 INFO:tasks.workunit.client.1.smithi132.stdout:9/727: mknod d12/d8d/d3a/d40/d4e/d77/d9d/cfb 0 2024-02-19T19:04:58.380 INFO:tasks.workunit.client.1.smithi132.stdout:4/596: dread d4/d27/d31/d32/d51/d5c/dbc/fbd [0,4194304] 0 2024-02-19T19:04:58.380 INFO:tasks.workunit.client.1.smithi132.stdout:2/665: mknod d0/d4/df/d40/d7b/d9e/cd3 0 2024-02-19T19:04:58.381 INFO:tasks.workunit.client.1.smithi132.stdout:2/666: write d0/d4/df/d7a/f99 [477705,95859] 0 2024-02-19T19:04:58.382 INFO:tasks.workunit.client.1.smithi132.stdout:3/639: chown d8/d13/l80 1887088527 1 2024-02-19T19:04:58.384 INFO:tasks.workunit.client.1.smithi132.stdout:9/728: creat d12/d8d/d3a/d85/ffc x:0 0 0 2024-02-19T19:04:58.386 INFO:tasks.workunit.client.1.smithi132.stdout:5/721: unlink d8/da/d5a/d7b/d96/d9e/ld3 0 2024-02-19T19:04:58.388 INFO:tasks.workunit.client.1.smithi132.stdout:4/597: creat d4/d19/daa/fd1 x:0 0 0 2024-02-19T19:04:58.389 INFO:tasks.workunit.client.1.smithi132.stdout:2/667: rmdir d0/d4/df/d40/d68 39 2024-02-19T19:04:58.391 INFO:tasks.workunit.client.1.smithi132.stdout:3/640: creat d8/d13/d57/fc8 x:0 0 0 2024-02-19T19:04:58.392 INFO:tasks.workunit.client.1.smithi132.stdout:9/729: dwrite d12/d8d/d3a/d40/d4e/d77/d89/dc4/ff8 [0,4194304] 0 2024-02-19T19:04:58.392 INFO:tasks.workunit.client.1.smithi132.stdout:5/722: mkdir d8/d23/d29/d27/df8 0 2024-02-19T19:04:58.399 INFO:tasks.workunit.client.1.smithi132.stdout:4/598: symlink d4/d12/d48/d5f/ld2 0 2024-02-19T19:04:58.400 INFO:tasks.workunit.client.1.smithi132.stdout:4/599: stat d4/d27/d31/d71/d14/d24 0 2024-02-19T19:04:58.401 INFO:tasks.workunit.client.1.smithi132.stdout:0/575: sync 2024-02-19T19:04:58.404 INFO:tasks.workunit.client.1.smithi132.stdout:9/730: unlink d12/d8d/d3a/f81 0 2024-02-19T19:04:58.411 INFO:tasks.workunit.client.1.smithi132.stdout:0/576: creat d2/dc/df/d27/d3a/fc0 x:0 0 0 2024-02-19T19:04:58.414 INFO:tasks.workunit.client.1.smithi132.stdout:2/668: creat d0/d4/d11/d2e/d5a/fd4 x:0 0 0 2024-02-19T19:04:58.416 INFO:tasks.workunit.client.1.smithi132.stdout:2/669: write d0/d24/d58/d53/dab/d84/fa1 [3904717,57796] 0 2024-02-19T19:04:58.418 INFO:tasks.workunit.client.1.smithi132.stdout:4/600: symlink d4/ld3 0 2024-02-19T19:04:58.426 INFO:tasks.workunit.client.1.smithi132.stdout:9/731: dread d12/f23 [0,4194304] 0 2024-02-19T19:04:58.426 INFO:tasks.workunit.client.1.smithi132.stdout:0/577: dread d2/dc/df/d15/d2f/d3f/d63/d6b/f75 [0,4194304] 0 2024-02-19T19:04:58.426 INFO:tasks.workunit.client.1.smithi132.stdout:2/670: unlink d0/d19/d1e/f52 0 2024-02-19T19:04:58.427 INFO:tasks.workunit.client.1.smithi132.stdout:3/641: link d8/d13/cbb d8/d13/d2b/d69/d92/d34/d97/cc9 0 2024-02-19T19:04:58.428 INFO:tasks.workunit.client.1.smithi132.stdout:2/671: dread - d0/d24/d58/d53/dab/d8a/fc0 zero size 2024-02-19T19:04:58.428 INFO:tasks.workunit.client.1.smithi132.stdout:0/578: write d2/dc/df/d15/d2f/d3f/d63/d6b/f72 [4348072,60169] 0 2024-02-19T19:04:58.429 INFO:tasks.workunit.client.1.smithi132.stdout:2/672: dread - d0/d4/d11/d2e/d5a/fd4 zero size 2024-02-19T19:04:58.431 INFO:tasks.workunit.client.1.smithi132.stdout:5/723: getdents d8/d23/d29/dba 0 2024-02-19T19:04:58.432 INFO:tasks.workunit.client.1.smithi132.stdout:9/732: mkdir d12/d8d/d3a/d85/d87/dfd 0 2024-02-19T19:04:58.433 INFO:tasks.workunit.client.1.smithi132.stdout:9/733: write d12/d3e/f9f [846343,84098] 0 2024-02-19T19:04:58.434 INFO:tasks.workunit.client.1.smithi132.stdout:9/734: stat d12/d2a/d4c/d61 0 2024-02-19T19:04:58.435 INFO:tasks.workunit.client.1.smithi132.stdout:4/601: mkdir d4/d19/dd4 0 2024-02-19T19:04:58.436 INFO:tasks.workunit.client.1.smithi132.stdout:0/579: symlink d2/dc/d4a/d25/d43/lc1 0 2024-02-19T19:04:58.436 INFO:tasks.workunit.client.1.smithi132.stdout:0/580: chown d2/dc/df/dbd 167859686 1 2024-02-19T19:04:58.439 INFO:tasks.workunit.client.1.smithi132.stdout:3/642: unlink d8/d13/d2b/d60/f83 0 2024-02-19T19:04:58.440 INFO:tasks.workunit.client.1.smithi132.stdout:9/735: mkdir d12/d2a/d4c/d61/dfe 0 2024-02-19T19:04:58.441 INFO:tasks.workunit.client.1.smithi132.stdout:2/673: rename d0/d4/df/fc3 to d0/d3c/d3f/fd5 0 2024-02-19T19:04:58.442 INFO:tasks.workunit.client.1.smithi132.stdout:0/581: rmdir d2/dc/d4a/d67 39 2024-02-19T19:04:58.442 INFO:tasks.workunit.client.1.smithi132.stdout:3/643: rmdir d8/d13/d2b/d69/d92/d34/d42 39 2024-02-19T19:04:58.445 INFO:tasks.workunit.client.1.smithi132.stdout:2/674: truncate d0/d19/d1e/f5c 1122687 0 2024-02-19T19:04:58.446 INFO:tasks.workunit.client.1.smithi132.stdout:5/724: getdents d8/da/d5a/d84 0 2024-02-19T19:04:58.447 INFO:tasks.workunit.client.1.smithi132.stdout:0/582: symlink d2/dc/df/d15/d34/lc2 0 2024-02-19T19:04:58.450 INFO:tasks.workunit.client.1.smithi132.stdout:9/736: dwrite d12/d2a/d4c/d61/fa8 [0,4194304] 0 2024-02-19T19:04:58.453 INFO:tasks.workunit.client.1.smithi132.stdout:5/725: truncate d8/da/db/fbb 254853 0 2024-02-19T19:04:58.454 INFO:tasks.workunit.client.1.smithi132.stdout:9/737: creat d12/d2a/d4c/fff x:0 0 0 2024-02-19T19:04:58.454 INFO:tasks.workunit.client.1.smithi132.stdout:3/644: dread d8/d13/d2f/f58 [0,4194304] 0 2024-02-19T19:04:58.455 INFO:tasks.workunit.client.1.smithi132.stdout:4/602: dread d4/f2d [0,4194304] 0 2024-02-19T19:04:58.455 INFO:tasks.workunit.client.1.smithi132.stdout:2/675: dwrite d0/d24/faa [0,4194304] 0 2024-02-19T19:04:58.457 INFO:tasks.workunit.client.1.smithi132.stdout:9/738: chown d12/d8d/d3a/d40/d4e/cc7 948 1 2024-02-19T19:04:58.457 INFO:tasks.workunit.client.1.smithi132.stdout:5/726: truncate d8/da/d5a/d84/fd5 1866087 0 2024-02-19T19:04:58.459 INFO:tasks.workunit.client.1.smithi132.stdout:2/676: truncate d0/d4/df/d7a/f88 1657884 0 2024-02-19T19:04:58.465 INFO:tasks.workunit.client.1.smithi132.stdout:0/583: link d2/d6e/fac d2/dc/df/d15/fc3 0 2024-02-19T19:04:58.466 INFO:tasks.workunit.client.1.smithi132.stdout:4/603: creat d4/d4d/d87/fd5 x:0 0 0 2024-02-19T19:04:58.467 INFO:tasks.workunit.client.1.smithi132.stdout:3/645: dwrite d8/d13/d2f/fb6 [0,4194304] 0 2024-02-19T19:04:58.468 INFO:tasks.workunit.client.1.smithi132.stdout:2/677: mknod d0/d24/daf/cd6 0 2024-02-19T19:04:58.471 INFO:tasks.workunit.client.1.smithi132.stdout:9/739: symlink d12/d2a/d4c/d61/dfe/l100 0 2024-02-19T19:04:58.472 INFO:tasks.workunit.client.1.smithi132.stdout:0/584: mknod d2/cc4 0 2024-02-19T19:04:58.473 INFO:tasks.workunit.client.1.smithi132.stdout:9/740: write d12/d2a/f5b [2066077,88113] 0 2024-02-19T19:04:58.474 INFO:tasks.workunit.client.1.smithi132.stdout:2/678: rename d0/d4/db9/lc5 to d0/d24/d58/d53/dab/d84/ld7 0 2024-02-19T19:04:58.476 INFO:tasks.workunit.client.1.smithi132.stdout:3/646: mknod d8/cca 0 2024-02-19T19:04:58.478 INFO:tasks.workunit.client.1.smithi132.stdout:2/679: dread - d0/d4/d11/d2e/d5a/fd4 zero size 2024-02-19T19:04:58.479 INFO:tasks.workunit.client.1.smithi132.stdout:9/741: symlink d12/d8d/d3a/l101 0 2024-02-19T19:04:58.479 INFO:tasks.workunit.client.1.smithi132.stdout:3/647: chown d8/d13/d2b/d69/d7e/d9f/da2 1 1 2024-02-19T19:04:58.480 INFO:tasks.workunit.client.1.smithi132.stdout:4/604: getdents d4/d27/d31/d71 0 2024-02-19T19:04:58.481 INFO:tasks.workunit.client.1.smithi132.stdout:2/680: dread - d0/d24/d58/f98 zero size 2024-02-19T19:04:58.482 INFO:tasks.workunit.client.1.smithi132.stdout:0/585: dwrite d2/dc/df/d15/d2f/d3f/f8d [0,4194304] 0 2024-02-19T19:04:58.485 INFO:tasks.workunit.client.1.smithi132.stdout:0/586: chown d2/dc/df/d27/d3a/l9e 372 1 2024-02-19T19:04:58.488 INFO:tasks.workunit.client.1.smithi132.stdout:3/648: symlink d8/d13/d2b/lcb 0 2024-02-19T19:04:58.493 INFO:tasks.workunit.client.1.smithi132.stdout:9/742: rename d12/d8d/d3a/d40/d4e/d77/d8a/d82/fc6 to d12/d2a/d88/f102 0 2024-02-19T19:04:58.495 INFO:tasks.workunit.client.1.smithi132.stdout:4/605: creat d4/fd6 x:0 0 0 2024-02-19T19:04:58.497 INFO:tasks.workunit.client.1.smithi132.stdout:0/587: unlink d2/dc/d4a/d74/f86 0 2024-02-19T19:04:58.497 INFO:tasks.workunit.client.1.smithi132.stdout:5/727: dread d8/d1a/d35/f74 [4194304,4194304] 0 2024-02-19T19:04:58.498 INFO:tasks.workunit.client.1.smithi132.stdout:3/649: mkdir d8/d38/d44/d6e/dcc 0 2024-02-19T19:04:58.500 INFO:tasks.workunit.client.1.smithi132.stdout:9/743: dwrite d12/d3e/f4a [0,4194304] 0 2024-02-19T19:04:58.502 INFO:tasks.workunit.client.1.smithi132.stdout:0/588: mknod d2/dc/d4a/d25/cc5 0 2024-02-19T19:04:58.504 INFO:tasks.workunit.client.1.smithi132.stdout:5/728: mkdir d8/d23/d29/d27/df9 0 2024-02-19T19:04:58.504 INFO:tasks.workunit.client.1.smithi132.stdout:0/589: truncate d2/dc/df/f2d 2676603 0 2024-02-19T19:04:58.506 INFO:tasks.workunit.client.1.smithi132.stdout:0/590: chown d2/d6e/d7a/l89 99 1 2024-02-19T19:04:58.509 INFO:tasks.workunit.client.1.smithi132.stdout:1/583: sync 2024-02-19T19:04:58.511 INFO:tasks.workunit.client.1.smithi132.stdout:0/591: write d2/dc/df/d15/d2f/d3f/d63/d6b/f75 [939012,45305] 0 2024-02-19T19:04:58.513 INFO:tasks.workunit.client.1.smithi132.stdout:5/729: dwrite d8/d23/d29/d27/f2b [0,4194304] 0 2024-02-19T19:04:58.515 INFO:tasks.workunit.client.1.smithi132.stdout:1/584: unlink d1/d1e/d77/d2c/f2d 0 2024-02-19T19:04:58.516 INFO:tasks.workunit.client.1.smithi132.stdout:3/650: getdents d8/d13/d2b 0 2024-02-19T19:04:58.516 INFO:tasks.workunit.client.1.smithi132.stdout:3/651: chown d8 3 1 2024-02-19T19:04:58.517 INFO:tasks.workunit.client.1.smithi132.stdout:0/592: write d2/dc/df/d15/f24 [1160544,29275] 0 2024-02-19T19:04:58.519 INFO:tasks.workunit.client.1.smithi132.stdout:5/730: mkdir d8/da/d5a/d7b/dfa 0 2024-02-19T19:04:58.525 INFO:tasks.workunit.client.1.smithi132.stdout:9/744: dread d12/f26 [0,4194304] 0 2024-02-19T19:04:58.526 INFO:tasks.workunit.client.1.smithi132.stdout:9/745: write d12/d3e/fd7 [800919,59703] 0 2024-02-19T19:04:58.526 INFO:tasks.workunit.client.1.smithi132.stdout:4/606: write d4/d12/f2f [550206,9369] 0 2024-02-19T19:04:58.527 INFO:tasks.workunit.client.1.smithi132.stdout:4/607: stat d4/d4d/d87/c88 0 2024-02-19T19:04:58.529 INFO:tasks.workunit.client.1.smithi132.stdout:3/652: mkdir d8/d13/d2b/d69/d7e/d9f/da2/dcd 0 2024-02-19T19:04:58.529 INFO:tasks.workunit.client.1.smithi132.stdout:3/653: chown d8/l71 7315 1 2024-02-19T19:04:58.530 INFO:tasks.workunit.client.1.smithi132.stdout:1/585: mkdir d1/d1e/d77/d2c/d43/db8 0 2024-02-19T19:04:58.533 INFO:tasks.workunit.client.1.smithi132.stdout:9/746: symlink d12/d8d/d3a/d40/d4e/d77/d8a/l103 0 2024-02-19T19:04:58.536 INFO:tasks.workunit.client.1.smithi132.stdout:4/608: rename d4/d27/d31/d71/d14/d24/c76 to d4/d27/d73/cd7 0 2024-02-19T19:04:58.538 INFO:tasks.workunit.client.1.smithi132.stdout:3/654: rmdir d8/d13/d2b/d69/d7e/d9f 39 2024-02-19T19:04:58.542 INFO:tasks.workunit.client.1.smithi132.stdout:5/731: truncate d8/f9b 1227647 0 2024-02-19T19:04:58.543 INFO:tasks.workunit.client.1.smithi132.stdout:1/586: mkdir d1/d1e/d77/d52/db9 0 2024-02-19T19:04:58.547 INFO:tasks.workunit.client.1.smithi132.stdout:0/593: getdents d2/dc/df/d15/d2f/d69 0 2024-02-19T19:04:58.548 INFO:tasks.workunit.client.1.smithi132.stdout:0/594: dread - d2/dc/d4a/d25/d43/d5f/fb7 zero size 2024-02-19T19:04:58.548 INFO:tasks.workunit.client.1.smithi132.stdout:5/732: mknod d8/d2e/db3/cfb 0 2024-02-19T19:04:58.552 INFO:tasks.workunit.client.1.smithi132.stdout:5/733: rmdir d8/d2e/dc4 39 2024-02-19T19:04:58.553 INFO:tasks.workunit.client.1.smithi132.stdout:0/595: dwrite d2/dc/d4a/f42 [0,4194304] 0 2024-02-19T19:04:58.554 INFO:tasks.workunit.client.1.smithi132.stdout:0/596: write d2/f1a [2126960,76197] 0 2024-02-19T19:04:58.554 INFO:tasks.workunit.client.1.smithi132.stdout:1/587: mkdir d1/d1e/d77/d2c/d62/d6b/dba 0 2024-02-19T19:04:58.555 INFO:tasks.workunit.client.1.smithi132.stdout:1/588: dread - d1/d3a/fa3 zero size 2024-02-19T19:04:58.555 INFO:tasks.workunit.client.1.smithi132.stdout:4/609: creat d4/d19/fd8 x:0 0 0 2024-02-19T19:04:58.557 INFO:tasks.workunit.client.1.smithi132.stdout:4/610: chown d4/d12/d16/f5a 1585519488 1 2024-02-19T19:04:58.560 INFO:tasks.workunit.client.1.smithi132.stdout:9/747: getdents d12/d8d 0 2024-02-19T19:04:58.563 INFO:tasks.workunit.client.1.smithi132.stdout:0/597: symlink d2/dc/df/d15/lc6 0 2024-02-19T19:04:58.566 INFO:tasks.workunit.client.1.smithi132.stdout:9/748: creat d12/d2a/d4c/d61/dfe/f104 x:0 0 0 2024-02-19T19:04:58.566 INFO:tasks.workunit.client.1.smithi132.stdout:5/734: link d8/d23/d29/l94 d8/d2e/dc4/dc8/daa/lfc 0 2024-02-19T19:04:58.567 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: Active manager daemon smithi088.gyviqq restarted 2024-02-19T19:04:58.567 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: Activating manager daemon smithi088.gyviqq 2024-02-19T19:04:58.567 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: osdmap e49: 6 total, 6 up, 6 in 2024-02-19T19:04:58.567 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: mgrmap e35: smithi088.gyviqq(active, starting, since 0.0192531s), standbys: smithi132.psllsa 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi088.gyviqq", "id": "smithi088.gyviqq"}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi132.psllsa", "id": "smithi132.psllsa"}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T19:04:58.568 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T19:04:58.569 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T19:04:58.569 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T19:04:58.569 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T19:04:58.569 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T19:04:58.569 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: Manager daemon smithi088.gyviqq is now available 2024-02-19T19:04:58.569 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:04:58.569 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:04:58.569 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:04:58.569 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/mirror_snapshot_schedule"}]: dispatch 2024-02-19T19:04:58.569 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:04:58 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/trash_purge_schedule"}]: dispatch 2024-02-19T19:04:58.569 INFO:tasks.workunit.client.1.smithi132.stdout:0/598: symlink d2/dc/d4a/d25/d7f/lc7 0 2024-02-19T19:04:58.570 INFO:tasks.workunit.client.1.smithi132.stdout:9/749: rmdir d12/d1b 39 2024-02-19T19:04:58.574 INFO:tasks.workunit.client.1.smithi132.stdout:1/589: dwrite d1/d1e/d77/d2c/f7d [0,4194304] 0 2024-02-19T19:04:58.574 INFO:tasks.workunit.client.1.smithi132.stdout:0/599: mkdir d2/d6e/dc8 0 2024-02-19T19:04:58.574 INFO:tasks.workunit.client.1.smithi132.stdout:5/735: read d8/da/f24 [1320978,105259] 0 2024-02-19T19:04:58.575 INFO:tasks.workunit.client.1.smithi132.stdout:4/611: dwrite d4/d27/d31/fbf [4194304,4194304] 0 2024-02-19T19:04:58.578 INFO:tasks.workunit.client.1.smithi132.stdout:0/600: dread - d2/dc/d4a/d25/d43/d5f/fb7 zero size 2024-02-19T19:04:58.578 INFO:tasks.workunit.client.1.smithi132.stdout:9/750: read d12/d2a/d4c/d61/f6b [349806,78554] 0 2024-02-19T19:04:58.579 INFO:tasks.workunit.client.1.smithi132.stdout:0/601: chown d2/d3e/db1 874956 1 2024-02-19T19:04:58.581 INFO:tasks.workunit.client.1.smithi132.stdout:5/736: chown d8/d23/cab 0 1 2024-02-19T19:04:58.586 INFO:tasks.workunit.client.1.smithi132.stdout:4/612: creat d4/d27/d7f/fd9 x:0 0 0 2024-02-19T19:04:58.588 INFO:tasks.workunit.client.1.smithi132.stdout:4/613: write d4/d12/d16/f6c [649812,64959] 0 2024-02-19T19:04:58.592 INFO:tasks.workunit.client.1.smithi132.stdout:0/602: mkdir d2/dc/d4a/d25/dc9 0 2024-02-19T19:04:58.594 INFO:tasks.workunit.client.1.smithi132.stdout:5/737: mknod d8/d2e/d71/da6/cfd 0 2024-02-19T19:04:58.595 INFO:tasks.workunit.client.1.smithi132.stdout:1/590: creat d1/d1e/d77/d2c/d43/d81/da1/fbb x:0 0 0 2024-02-19T19:04:58.600 INFO:tasks.workunit.client.1.smithi132.stdout:0/603: dread d2/dc/f3b [0,4194304] 0 2024-02-19T19:04:58.603 INFO:tasks.workunit.client.1.smithi132.stdout:9/751: dread d12/d8d/f28 [0,4194304] 0 2024-02-19T19:04:58.605 INFO:tasks.workunit.client.1.smithi132.stdout:5/738: link d8/d2e/dc4/dc8/l73 d8/da/d5a/d7b/d96/lfe 0 2024-02-19T19:04:58.605 INFO:tasks.workunit.client.1.smithi132.stdout:1/591: dwrite d1/d1e/d77/d78/f7f [0,4194304] 0 2024-02-19T19:04:58.606 INFO:tasks.workunit.client.1.smithi132.stdout:9/752: dread - d12/d8d/d3a/d40/d4e/d77/d89/dc4/fe3 zero size 2024-02-19T19:04:58.607 INFO:tasks.workunit.client.1.smithi132.stdout:9/753: chown d12/d8d/d3a/d40/d4e/d77 173984 1 2024-02-19T19:04:58.611 INFO:tasks.workunit.client.1.smithi132.stdout:0/604: rename d2/dc/d4a/f57 to d2/dc/df/d15/d2f/fca 0 2024-02-19T19:04:58.611 INFO:tasks.workunit.client.1.smithi132.stdout:5/739: dread d8/da/db/f34 [0,4194304] 0 2024-02-19T19:04:58.613 INFO:tasks.workunit.client.1.smithi132.stdout:5/740: write d8/d23/d29/d27/fe7 [924890,4709] 0 2024-02-19T19:04:58.613 INFO:tasks.workunit.client.1.smithi132.stdout:9/754: creat d12/d8d/d3a/d40/d4e/d77/d8a/dca/f105 x:0 0 0 2024-02-19T19:04:58.654 INFO:tasks.workunit.client.1.smithi132.stdout:8/688: sync 2024-02-19T19:04:58.654 INFO:tasks.workunit.client.1.smithi132.stdout:6/561: sync 2024-02-19T19:04:58.654 INFO:tasks.workunit.client.1.smithi132.stdout:7/704: sync 2024-02-19T19:04:58.688 INFO:tasks.workunit.client.1.smithi132.stdout:4/614: write d4/d27/d31/d71/d14/d24/fb1 [373958,127686] 0 2024-02-19T19:04:58.688 INFO:tasks.workunit.client.1.smithi132.stdout:8/689: fdatasync d4/f2e 0 2024-02-19T19:04:58.690 INFO:tasks.workunit.client.1.smithi132.stdout:7/705: chown d1b/d1e/d41/l71 3165849 1 2024-02-19T19:04:58.692 INFO:tasks.workunit.client.1.smithi132.stdout:5/741: truncate d8/da/d5a/fcb 338772 0 2024-02-19T19:04:58.692 INFO:tasks.workunit.client.1.smithi132.stdout:4/615: mknod d4/d19/daa/cda 0 2024-02-19T19:04:58.693 INFO:tasks.workunit.client.1.smithi132.stdout:6/562: creat d0/db/d25/d3a/d4b/d7c/fbd x:0 0 0 2024-02-19T19:04:58.693 INFO:tasks.workunit.client.1.smithi132.stdout:4/616: write d4/d19/fd8 [1029303,41495] 0 2024-02-19T19:04:58.695 INFO:tasks.workunit.client.1.smithi132.stdout:8/690: creat d4/dc/d1b/d54/d81/dbd/fef x:0 0 0 2024-02-19T19:04:58.696 INFO:tasks.workunit.client.1.smithi132.stdout:9/755: truncate fb 2764480 0 2024-02-19T19:04:58.697 INFO:tasks.workunit.client.1.smithi132.stdout:0/605: dread d2/dc/df/d15/d2f/d47/f96 [4194304,4194304] 0 2024-02-19T19:04:58.699 INFO:tasks.workunit.client.1.smithi132.stdout:0/606: write d2/dc/d4a/d74/fb5 [960883,72776] 0 2024-02-19T19:04:58.703 INFO:tasks.workunit.client.1.smithi132.stdout:8/691: mknod d4/dc/d1b/d20/d8a/cf0 0 2024-02-19T19:04:58.706 INFO:tasks.workunit.client.1.smithi132.stdout:5/742: dwrite d8/da/db/f34 [0,4194304] 0 2024-02-19T19:04:58.709 INFO:tasks.workunit.client.1.smithi132.stdout:8/692: mknod d4/dc/d1b/d20/dbe/cf1 0 2024-02-19T19:04:58.709 INFO:tasks.workunit.client.1.smithi132.stdout:0/607: dwrite d2/dc/df/d27/d3a/fb4 [0,4194304] 0 2024-02-19T19:04:58.711 INFO:tasks.workunit.client.1.smithi132.stdout:5/743: rmdir d8/da 39 2024-02-19T19:04:58.713 INFO:tasks.workunit.client.1.smithi132.stdout:8/693: chown d4/dc/d1b/d54/c5d 57510 1 2024-02-19T19:04:58.713 INFO:tasks.workunit.client.1.smithi132.stdout:5/744: chown d8/d23/d29/d27/f25 1 1 2024-02-19T19:04:58.713 INFO:tasks.workunit.client.1.smithi132.stdout:0/608: write d2/dc/d4a/d25/d43/d5f/fa4 [1289263,60034] 0 2024-02-19T19:04:58.724 INFO:tasks.workunit.client.1.smithi132.stdout:8/694: symlink d4/dc/d1b/d8f/lf2 0 2024-02-19T19:04:58.724 INFO:tasks.workunit.client.1.smithi132.stdout:9/756: link d12/d2a/d4c/d61/l7e d12/d8d/d3a/d40/d4e/d77/d8a/dca/l106 0 2024-02-19T19:04:58.724 INFO:tasks.workunit.client.1.smithi132.stdout:7/706: write d1b/d1e/fb2 [3105110,126379] 0 2024-02-19T19:04:58.726 INFO:tasks.workunit.client.1.smithi132.stdout:7/707: readlink d1b/d1e/d41/da3/dbd/ld8 0 2024-02-19T19:04:58.727 INFO:tasks.workunit.client.1.smithi132.stdout:4/617: truncate d4/d4d/f64 185744 0 2024-02-19T19:04:58.729 INFO:tasks.workunit.client.1.smithi132.stdout:5/745: rmdir d8/d23/d29/d27/df9 0 2024-02-19T19:04:58.730 INFO:tasks.workunit.client.1.smithi132.stdout:8/695: write d4/dc/d42/f56 [1930960,104492] 0 2024-02-19T19:04:58.730 INFO:tasks.workunit.client.1.smithi132.stdout:4/618: chown d4/d12/d16/f5a 3375 1 2024-02-19T19:04:58.731 INFO:tasks.workunit.client.1.smithi132.stdout:7/708: truncate d1b/d5a/d63/d6b/fcf 556584 0 2024-02-19T19:04:58.733 INFO:tasks.workunit.client.1.smithi132.stdout:6/563: dwrite d0/db/d25/d3a/d5d/d8f/f39 [0,4194304] 0 2024-02-19T19:04:58.735 INFO:tasks.workunit.client.1.smithi132.stdout:8/696: read d4/dc/d1b/d33/f4d [2994546,16197] 0 2024-02-19T19:04:58.735 INFO:tasks.workunit.client.1.smithi132.stdout:7/709: write d1b/d1e/d2b/fb4 [3067879,130628] 0 2024-02-19T19:04:58.736 INFO:tasks.workunit.client.1.smithi132.stdout:4/619: write d4/d27/d31/d71/d37/f6a [531080,84396] 0 2024-02-19T19:04:58.736 INFO:tasks.workunit.client.1.smithi132.stdout:9/757: readlink d12/d8d/d3a/d40/d4e/d77/d8a/d82/dd9/lf4 0 2024-02-19T19:04:58.739 INFO:tasks.workunit.client.1.smithi132.stdout:5/746: creat d8/da/d5a/d7b/d96/d9e/fff x:0 0 0 2024-02-19T19:04:58.741 INFO:tasks.workunit.client.1.smithi132.stdout:8/697: fdatasync d4/dc/d42/f97 0 2024-02-19T19:04:58.744 INFO:tasks.workunit.client.1.smithi132.stdout:7/710: fdatasync d1b/d1e/d2b/d62/d53/fa4 0 2024-02-19T19:04:58.745 INFO:tasks.workunit.client.1.smithi132.stdout:8/698: fdatasync d4/fd 0 2024-02-19T19:04:58.745 INFO:tasks.workunit.client.1.smithi132.stdout:9/758: dread d12/d8d/d3a/d40/d4e/d77/f9c [0,4194304] 0 2024-02-19T19:04:58.746 INFO:tasks.workunit.client.1.smithi132.stdout:6/564: rename d0/d8b/d77 to d0/db/d25/d3a/d5d/d8f/dbe 0 2024-02-19T19:04:58.747 INFO:tasks.workunit.client.1.smithi132.stdout:5/747: symlink d8/d2e/d71/da6/ddb/l100 0 2024-02-19T19:04:58.749 INFO:tasks.workunit.client.1.smithi132.stdout:9/759: creat d12/d2a/d4c/dad/f107 x:0 0 0 2024-02-19T19:04:58.749 INFO:tasks.workunit.client.1.smithi132.stdout:5/748: stat d8/d23/d3b/lb2 0 2024-02-19T19:04:58.750 INFO:tasks.workunit.client.1.smithi132.stdout:0/609: dread d2/f6 [0,4194304] 0 2024-02-19T19:04:58.750 INFO:tasks.workunit.client.1.smithi132.stdout:8/699: truncate d4/dc/d1b/d20/f8b 1481753 0 2024-02-19T19:04:58.751 INFO:tasks.workunit.client.1.smithi132.stdout:0/610: stat d2/dc/df/faa 0 2024-02-19T19:04:58.752 INFO:tasks.workunit.client.1.smithi132.stdout:8/700: read - d4/dc/d1b/d33/fe0 zero size 2024-02-19T19:04:58.753 INFO:tasks.workunit.client.1.smithi132.stdout:7/711: link d1b/d1e/d2b/d8f/dae/cc2 d1b/d32/cdd 0 2024-02-19T19:04:58.755 INFO:tasks.workunit.client.1.smithi132.stdout:7/712: write d1b/d1e/d28/f97 [1120986,123416] 0 2024-02-19T19:04:58.757 INFO:tasks.workunit.client.1.smithi132.stdout:5/749: creat d8/d1a/d64/de2/dec/f101 x:0 0 0 2024-02-19T19:04:58.758 INFO:tasks.workunit.client.1.smithi132.stdout:8/701: symlink d4/dc/d1b/d20/dbe/lf3 0 2024-02-19T19:04:58.759 INFO:tasks.workunit.client.1.smithi132.stdout:0/611: creat d2/dc/df/d27/d3a/d52/fcb x:0 0 0 2024-02-19T19:04:58.760 INFO:tasks.workunit.client.1.smithi132.stdout:8/702: chown d4/dc/d1b/d20/d62/d91/lec 2300859 1 2024-02-19T19:04:58.764 INFO:tasks.workunit.client.1.smithi132.stdout:0/612: symlink d2/dc/d4a/d25/lcc 0 2024-02-19T19:04:58.764 INFO:tasks.workunit.client.1.smithi132.stdout:5/750: write d8/da/f8c [790395,80817] 0 2024-02-19T19:04:58.767 INFO:tasks.workunit.client.1.smithi132.stdout:4/620: dread d4/d27/d7f/fa9 [0,4194304] 0 2024-02-19T19:04:58.768 INFO:tasks.workunit.client.1.smithi132.stdout:5/751: dread - d8/d2e/dc4/dc8/daa/fd9 zero size 2024-02-19T19:04:58.768 INFO:tasks.workunit.client.1.smithi132.stdout:0/613: creat d2/d6e/d7a/fcd x:0 0 0 2024-02-19T19:04:58.769 INFO:tasks.workunit.client.1.smithi132.stdout:7/713: dread d1b/f1c [0,4194304] 0 2024-02-19T19:04:58.769 INFO:tasks.workunit.client.1.smithi132.stdout:0/614: chown d2/dc/l55 51172304 1 2024-02-19T19:04:58.769 INFO:tasks.workunit.client.1.smithi132.stdout:4/621: fsync d4/d27/d31/d32/d51/d5c/dbc/fcd 0 2024-02-19T19:04:58.769 INFO:tasks.workunit.client.1.smithi132.stdout:5/752: chown d8/d2e/d71/dce/df0 145360 1 2024-02-19T19:04:58.770 INFO:tasks.workunit.client.1.smithi132.stdout:0/615: chown d2/dc 161 1 2024-02-19T19:04:58.771 INFO:tasks.workunit.client.1.smithi132.stdout:4/622: write d4/d12/f30 [3451763,33391] 0 2024-02-19T19:04:58.772 INFO:tasks.workunit.client.1.smithi132.stdout:9/760: truncate d12/d2a/f2c 3346910 0 2024-02-19T19:04:58.772 INFO:tasks.workunit.client.1.smithi132.stdout:4/623: chown d4/d27/d31/d71/d14/d24 0 1 2024-02-19T19:04:58.773 INFO:tasks.workunit.client.1.smithi132.stdout:7/714: creat d1b/d1e/d41/db5/fde x:0 0 0 2024-02-19T19:04:58.774 INFO:tasks.workunit.client.1.smithi132.stdout:9/761: fsync d12/d8d/d3a/d40/f66 0 2024-02-19T19:04:58.777 INFO:tasks.workunit.client.1.smithi132.stdout:9/762: truncate d12/d2a/fd2 551778 0 2024-02-19T19:04:58.777 INFO:tasks.workunit.client.1.smithi132.stdout:4/624: mkdir d4/d12/d16/ddb 0 2024-02-19T19:04:58.779 INFO:tasks.workunit.client.1.smithi132.stdout:5/753: creat d8/d2e/d71/da6/ddb/dea/f102 x:0 0 0 2024-02-19T19:04:58.783 INFO:tasks.workunit.client.1.smithi132.stdout:0/616: dwrite d2/dc/df/faa [0,4194304] 0 2024-02-19T19:04:58.783 INFO:tasks.workunit.client.1.smithi132.stdout:8/703: dread d4/dc/f44 [0,4194304] 0 2024-02-19T19:04:58.785 INFO:tasks.workunit.client.1.smithi132.stdout:7/715: dwrite d1b/f25 [0,4194304] 0 2024-02-19T19:04:58.787 INFO:tasks.workunit.client.1.smithi132.stdout:8/704: fdatasync d4/dc/d1b/d38/d69/d74/dce/fe2 0 2024-02-19T19:04:58.789 INFO:tasks.workunit.client.1.smithi132.stdout:4/625: dwrite d4/d12/d28/f62 [0,4194304] 0 2024-02-19T19:04:58.790 INFO:tasks.workunit.client.1.smithi132.stdout:8/705: read d4/dc/d1b/d20/f53 [573666,110399] 0 2024-02-19T19:04:58.791 INFO:tasks.workunit.client.1.smithi132.stdout:4/626: write d4/d27/d31/d32/d51/fcb [265280,72868] 0 2024-02-19T19:04:58.793 INFO:tasks.workunit.client.1.smithi132.stdout:9/763: rename d12/d8d/d3a/d40/d4e/d77/fa2 to d12/d2a/d88/db8/f108 0 2024-02-19T19:04:58.794 INFO:tasks.workunit.client.1.smithi132.stdout:0/617: creat d2/d6e/d7a/fce x:0 0 0 2024-02-19T19:04:58.798 INFO:tasks.workunit.client.1.smithi132.stdout:7/716: mknod d1b/d5a/d63/d6b/d86/cdf 0 2024-02-19T19:04:58.799 INFO:tasks.workunit.client.1.smithi132.stdout:7/717: write d1b/d1e/d28/fbb [5067586,109362] 0 2024-02-19T19:04:58.800 INFO:tasks.workunit.client.1.smithi132.stdout:4/627: unlink d4/d12/d16/l2a 0 2024-02-19T19:04:58.802 INFO:tasks.workunit.client.1.smithi132.stdout:8/706: creat d4/dc/d42/ff4 x:0 0 0 2024-02-19T19:04:58.802 INFO:tasks.workunit.client.1.smithi132.stdout:9/764: mkdir d12/db6/d109 0 2024-02-19T19:04:58.803 INFO:tasks.workunit.client.1.smithi132.stdout:8/707: dread - d4/d2d/fc0 zero size 2024-02-19T19:04:58.803 INFO:tasks.workunit.client.1.smithi132.stdout:0/618: creat d2/dc/d4a/d74/fcf x:0 0 0 2024-02-19T19:04:58.805 INFO:tasks.workunit.client.1.smithi132.stdout:5/754: rmdir d8/d2e/dc4/dc8/ddc/df5 0 2024-02-19T19:04:58.807 INFO:tasks.workunit.client.1.smithi132.stdout:4/628: chown d4/d27/d31/d32/d51/d72/lbe 142 1 2024-02-19T19:04:58.808 INFO:tasks.workunit.client.1.smithi132.stdout:4/629: fsync d4/d4d/fa7 0 2024-02-19T19:04:58.809 INFO:tasks.workunit.client.1.smithi132.stdout:9/765: symlink d12/d8d/d3a/d40/d4e/d77/d89/dc4/l10a 0 2024-02-19T19:04:58.811 INFO:tasks.workunit.client.1.smithi132.stdout:8/708: rename d4/dc/d1b/d20/d23/c2f to d4/dac/cf5 0 2024-02-19T19:04:58.811 INFO:tasks.workunit.client.1.smithi132.stdout:0/619: creat d2/dc/df/d27/d3a/d52/fd0 x:0 0 0 2024-02-19T19:04:58.813 INFO:tasks.workunit.client.1.smithi132.stdout:5/755: unlink d8/da/d5a/f63 0 2024-02-19T19:04:58.817 INFO:tasks.workunit.client.1.smithi132.stdout:8/709: creat d4/d2d/dcc/ff6 x:0 0 0 2024-02-19T19:04:58.818 INFO:tasks.workunit.client.1.smithi132.stdout:5/756: mknod d8/d2e/dc4/dc8/ddc/c103 0 2024-02-19T19:04:58.823 INFO:tasks.workunit.client.1.smithi132.stdout:5/757: unlink d8/da/db/l5f 0 2024-02-19T19:04:58.824 INFO:tasks.workunit.client.1.smithi132.stdout:5/758: write d8/d23/d3b/f70 [180718,64869] 0 2024-02-19T19:04:58.825 INFO:tasks.workunit.client.1.smithi132.stdout:9/766: rmdir d12/d8d/d3a/d85/d87/dfd 0 2024-02-19T19:04:58.826 INFO:tasks.workunit.client.1.smithi132.stdout:8/710: mknod d4/dc/d1b/d20/d23/cf7 0 2024-02-19T19:04:58.832 INFO:tasks.workunit.client.1.smithi132.stdout:8/711: mkdir d4/dc/d1b/d38/d69/d74/dce/df8 0 2024-02-19T19:04:58.833 INFO:tasks.workunit.client.1.smithi132.stdout:2/681: sync 2024-02-19T19:04:58.833 INFO:tasks.workunit.client.1.smithi132.stdout:3/655: sync 2024-02-19T19:04:58.841 INFO:tasks.workunit.client.1.smithi132.stdout:8/712: getdents d4/dc/d1b/d8f 0 2024-02-19T19:04:58.841 INFO:tasks.workunit.client.1.smithi132.stdout:3/656: dwrite d8/d13/d2b/fa9 [4194304,4194304] 0 2024-02-19T19:04:58.845 INFO:tasks.workunit.client.1.smithi132.stdout:8/713: creat d4/dc/d1b/d54/d81/dbd/ff9 x:0 0 0 2024-02-19T19:04:58.846 INFO:tasks.workunit.client.1.smithi132.stdout:8/714: write d4/d2d/f46 [1575731,25153] 0 2024-02-19T19:04:58.847 INFO:tasks.workunit.client.1.smithi132.stdout:3/657: dread d8/d13/d2b/d69/d92/daf/fc3 [0,4194304] 0 2024-02-19T19:04:58.848 INFO:tasks.workunit.client.1.smithi132.stdout:5/759: dread d8/d1a/f5e [0,4194304] 0 2024-02-19T19:04:58.848 INFO:tasks.workunit.client.1.smithi132.stdout:5/760: readlink d8/d23/d3b/l78 0 2024-02-19T19:04:58.849 INFO:tasks.workunit.client.1.smithi132.stdout:5/761: write d8/d1a/f6a [3070376,70360] 0 2024-02-19T19:04:58.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: Active manager daemon smithi088.gyviqq restarted 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: Activating manager daemon smithi088.gyviqq 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: osdmap e49: 6 total, 6 up, 6 in 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: mgrmap e35: smithi088.gyviqq(active, starting, since 0.0192531s), standbys: smithi132.psllsa 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi088.gyviqq", "id": "smithi088.gyviqq"}]: dispatch 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr metadata", "who": "smithi132.psllsa", "id": "smithi132.psllsa"}]: dispatch 2024-02-19T19:04:58.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: Manager daemon smithi088.gyviqq is now available 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:04:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/mirror_snapshot_schedule"}]: dispatch 2024-02-19T19:04:58.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi088.gyviqq/trash_purge_schedule"}]: dispatch 2024-02-19T19:04:58.853 INFO:tasks.workunit.client.1.smithi132.stdout:3/658: creat d8/d13/d2b/d69/da6/fce x:0 0 0 2024-02-19T19:04:58.853 INFO:tasks.workunit.client.1.smithi132.stdout:7/718: truncate d1b/d5a/d63/d6b/d86/fac 1730765 0 2024-02-19T19:04:58.856 INFO:tasks.workunit.client.1.smithi132.stdout:4/630: truncate d4/d19/daa/dc7/f70 1840598 0 2024-02-19T19:04:58.857 INFO:tasks.workunit.client.1.smithi132.stdout:5/762: dwrite d8/d23/f45 [0,4194304] 0 2024-02-19T19:04:58.858 INFO:tasks.workunit.client.1.smithi132.stdout:5/763: write d8/d1a/f9d [712659,55535] 0 2024-02-19T19:04:58.859 INFO:tasks.workunit.client.1.smithi132.stdout:5/764: dread - d8/d2e/fb4 zero size 2024-02-19T19:04:58.866 INFO:tasks.workunit.client.1.smithi132.stdout:4/631: dread d4/d27/d31/d71/d37/f6a [0,4194304] 0 2024-02-19T19:04:58.867 INFO:tasks.workunit.client.1.smithi132.stdout:5/765: creat d8/d2e/dc4/dc8/f104 x:0 0 0 2024-02-19T19:04:58.869 INFO:tasks.workunit.client.1.smithi132.stdout:4/632: stat d4/d12/d4c/fce 0 2024-02-19T19:04:58.871 INFO:tasks.workunit.client.1.smithi132.stdout:4/633: fdatasync d4/fab 0 2024-02-19T19:04:58.872 INFO:tasks.workunit.client.1.smithi132.stdout:9/767: dwrite d12/d8d/d3a/d40/d4e/d77/d8a/d82/f90 [0,4194304] 0 2024-02-19T19:04:58.874 INFO:tasks.workunit.client.1.smithi132.stdout:2/682: dwrite d0/d4/df/f51 [0,4194304] 0 2024-02-19T19:04:58.881 INFO:tasks.workunit.client.1.smithi132.stdout:5/766: dread d8/d23/d29/d27/f57 [0,4194304] 0 2024-02-19T19:04:58.884 INFO:tasks.workunit.client.1.smithi132.stdout:7/719: truncate d1b/d1e/d28/f31 4945684 0 2024-02-19T19:04:58.885 INFO:tasks.workunit.client.1.smithi132.stdout:7/720: stat d1b/d5a/d63/c78 0 2024-02-19T19:04:58.889 INFO:tasks.workunit.client.1.smithi132.stdout:5/767: mkdir d8/d23/d29/d27/df8/d105 0 2024-02-19T19:04:58.891 INFO:tasks.workunit.client.1.smithi132.stdout:3/659: dread d8/d13/d2b/f2c [0,4194304] 0 2024-02-19T19:04:58.892 INFO:tasks.workunit.client.1.smithi132.stdout:7/721: read d1b/d1e/d2b/f51 [2317561,12936] 0 2024-02-19T19:04:58.893 INFO:tasks.workunit.client.1.smithi132.stdout:2/683: getdents d0/d24/d58/d53/dab/d8a 0 2024-02-19T19:04:58.894 INFO:tasks.workunit.client.1.smithi132.stdout:5/768: creat d8/da/dc2/f106 x:0 0 0 2024-02-19T19:04:58.896 INFO:tasks.workunit.client.1.smithi132.stdout:7/722: read - d1b/d1e/d2b/d62/d53/f65 zero size 2024-02-19T19:04:58.898 INFO:tasks.workunit.client.1.smithi132.stdout:9/768: dread d12/d8d/d3a/d40/f65 [0,4194304] 0 2024-02-19T19:04:58.898 INFO:tasks.workunit.client.1.smithi132.stdout:2/684: link d0/d24/d58/f98 d0/d24/dd0/fd8 0 2024-02-19T19:04:58.899 INFO:tasks.workunit.client.1.smithi132.stdout:9/769: fdatasync d12/d2a/d4c/d61/fc5 0 2024-02-19T19:04:58.899 INFO:tasks.workunit.client.1.smithi132.stdout:2/685: chown d0/d4/df/d40/d7b/d9e 18084 1 2024-02-19T19:04:58.901 INFO:tasks.workunit.client.1.smithi132.stdout:7/723: symlink d1b/d1e/d2b/d8f/da5/le0 0 2024-02-19T19:04:58.903 INFO:tasks.workunit.client.1.smithi132.stdout:4/634: truncate d4/d27/d31/d32/d51/d5c/dbc/fbd 886627 0 2024-02-19T19:04:58.905 INFO:tasks.workunit.client.1.smithi132.stdout:2/686: fsync d0/d4/d11/d2e/d5a/f5b 0 2024-02-19T19:04:58.906 INFO:tasks.workunit.client.1.smithi132.stdout:7/724: symlink d1b/d46/d7f/le1 0 2024-02-19T19:04:58.908 INFO:tasks.workunit.client.1.smithi132.stdout:4/635: truncate d4/f10 628573 0 2024-02-19T19:04:58.909 INFO:tasks.workunit.client.1.smithi132.stdout:2/687: mkdir d0/d24/d58/d53/dab/d84/dd9 0 2024-02-19T19:04:58.910 INFO:tasks.workunit.client.1.smithi132.stdout:9/770: link d12/d3e/d6e/f79 d12/d8d/f10b 0 2024-02-19T19:04:58.911 INFO:tasks.workunit.client.1.smithi132.stdout:5/769: write d8/d23/d3b/f65 [644095,47884] 0 2024-02-19T19:04:58.911 INFO:tasks.workunit.client.1.smithi132.stdout:3/660: dwrite d8/d13/d2b/d69/da6/fa5 [0,4194304] 0 2024-02-19T19:04:58.911 INFO:tasks.workunit.client.1.smithi132.stdout:7/725: mknod d1b/d1e/d2b/d8f/da5/ce2 0 2024-02-19T19:04:58.913 INFO:tasks.workunit.client.1.smithi132.stdout:2/688: fdatasync d0/d4/d11/d2e/f75 0 2024-02-19T19:04:58.915 INFO:tasks.workunit.client.1.smithi132.stdout:9/771: write d12/d2a/d4c/f9b [1329197,88341] 0 2024-02-19T19:04:58.917 INFO:tasks.workunit.client.1.smithi132.stdout:5/770: creat d8/d2e/dc4/dc8/df1/f107 x:0 0 0 2024-02-19T19:04:58.920 INFO:tasks.workunit.client.1.smithi132.stdout:3/661: rename d8/d13/d2f/fb6 to d8/d38/d44/fcf 0 2024-02-19T19:04:58.920 INFO:tasks.workunit.client.1.smithi132.stdout:9/772: write d12/d3e/f48 [2808078,108928] 0 2024-02-19T19:04:58.921 INFO:tasks.workunit.client.1.smithi132.stdout:7/726: mkdir d1b/d1e/d41/da3/de3 0 2024-02-19T19:04:58.922 INFO:tasks.workunit.client.1.smithi132.stdout:3/662: write d8/d13/d2b/d69/d7e/fab [960661,73028] 0 2024-02-19T19:04:58.928 INFO:tasks.workunit.client.1.smithi132.stdout:9/773: rmdir d12/d8d/d3a/d85 39 2024-02-19T19:04:58.929 INFO:tasks.workunit.client.1.smithi132.stdout:9/774: fdatasync d12/d8d/d3a/d40/f44 0 2024-02-19T19:04:58.930 INFO:tasks.workunit.client.1.smithi132.stdout:4/636: truncate d4/d19/f3f 1391149 0 2024-02-19T19:04:58.931 INFO:tasks.workunit.client.1.smithi132.stdout:2/689: truncate d0/d4/d11/f1c 7548733 0 2024-02-19T19:04:58.931 INFO:tasks.workunit.client.1.smithi132.stdout:9/775: creat d12/d2a/d4c/dad/db2/f10c x:0 0 0 2024-02-19T19:04:58.932 INFO:tasks.workunit.client.1.smithi132.stdout:2/690: read d0/d24/d58/fbc [1489247,3951] 0 2024-02-19T19:04:58.965 INFO:tasks.workunit.client.1.smithi132.stdout:4/637: creat d4/d27/d31/d32/d51/d72/fdc x:0 0 0 2024-02-19T19:04:58.966 INFO:tasks.workunit.client.1.smithi132.stdout:9/776: rmdir d12/d8d/d3a/d40/d4e/d77/d9d 39 2024-02-19T19:04:58.968 INFO:tasks.workunit.client.1.smithi132.stdout:4/638: symlink d4/d27/d31/d32/d51/d5c/dbc/ldd 0 2024-02-19T19:04:58.969 INFO:tasks.workunit.client.1.smithi132.stdout:9/777: truncate d12/d2a/d4c/fff 774259 0 2024-02-19T19:04:58.970 INFO:tasks.workunit.client.1.smithi132.stdout:5/771: dwrite d8/d1a/f5e [0,4194304] 0 2024-02-19T19:04:58.971 INFO:tasks.workunit.client.1.smithi132.stdout:9/778: fdatasync d12/d8d/d3a/d40/f66 0 2024-02-19T19:04:58.971 INFO:tasks.workunit.client.1.smithi132.stdout:5/772: chown d8/d23/d29/ce0 844067324 1 2024-02-19T19:04:58.972 INFO:tasks.workunit.client.1.smithi132.stdout:4/639: unlink d4/d4d/d87/f95 0 2024-02-19T19:04:58.976 INFO:tasks.workunit.client.1.smithi132.stdout:4/640: unlink d4/d27/d31/d71/d37/f74 0 2024-02-19T19:04:58.983 INFO:tasks.workunit.client.1.smithi132.stdout:9/779: rename d12/d1b/c58 to d12/d8d/d3a/d40/d4e/d77/d89/c10d 0 2024-02-19T19:04:58.988 INFO:tasks.workunit.client.1.smithi132.stdout:4/641: rename d4/d19/f7a to d4/d12/d48/fde 0 2024-02-19T19:04:58.989 INFO:tasks.workunit.client.1.smithi132.stdout:4/642: write d4/fad [4901639,106312] 0 2024-02-19T19:04:58.991 INFO:tasks.workunit.client.1.smithi132.stdout:9/780: creat d12/d8d/f10e x:0 0 0 2024-02-19T19:04:58.992 INFO:tasks.workunit.client.1.smithi132.stdout:9/781: dread - d12/d8d/d3a/d40/d4e/fb4 zero size 2024-02-19T19:04:58.996 INFO:tasks.workunit.client.1.smithi132.stdout:2/691: dread d0/d4/df/d40/d68/f8f [0,4194304] 0 2024-02-19T19:04:58.998 INFO:tasks.workunit.client.1.smithi132.stdout:5/773: truncate d8/d23/fd8 124583 0 2024-02-19T19:04:59.001 INFO:tasks.workunit.client.1.smithi132.stdout:5/774: mknod d8/da/db/c108 0 2024-02-19T19:04:59.004 INFO:tasks.workunit.client.1.smithi132.stdout:4/643: getdents d4/d19/d21/d56 0 2024-02-19T19:04:59.006 INFO:tasks.workunit.client.1.smithi132.stdout:5/775: creat d8/da/db/f109 x:0 0 0 2024-02-19T19:04:59.006 INFO:tasks.workunit.client.1.smithi132.stdout:9/782: link d12/d8d/d3a/d40/d4e/d77/d8a/d82/dd9/lf4 d12/d8d/d3a/d85/l10f 0 2024-02-19T19:04:59.007 INFO:tasks.workunit.client.1.smithi132.stdout:2/692: rmdir d0/d24/d58/d53/dab/d84/dd9 0 2024-02-19T19:04:59.011 INFO:tasks.workunit.client.1.smithi132.stdout:9/783: mkdir d12/d8d/d3a/d40/d4e/d77/d89/dc4/d110 0 2024-02-19T19:04:59.014 INFO:tasks.workunit.client.1.smithi132.stdout:4/644: dwrite d4/d12/d4c/fce [0,4194304] 0 2024-02-19T19:04:59.016 INFO:tasks.workunit.client.1.smithi132.stdout:5/776: dwrite d8/d23/d3b/f48 [0,4194304] 0 2024-02-19T19:04:59.018 INFO:tasks.workunit.client.1.smithi132.stdout:4/645: symlink d4/d12/da4/ldf 0 2024-02-19T19:04:59.019 INFO:tasks.workunit.client.1.smithi132.stdout:4/646: read d4/d27/d31/d71/d37/f6a [507903,40451] 0 2024-02-19T19:04:59.020 INFO:tasks.workunit.client.1.smithi132.stdout:2/693: write d0/d24/d58/d53/dab/d8a/fa4 [2091223,65866] 0 2024-02-19T19:04:59.022 INFO:tasks.workunit.client.1.smithi132.stdout:9/784: link d12/db6/cf9 d12/d8d/c111 0 2024-02-19T19:04:59.028 INFO:tasks.workunit.client.1.smithi132.stdout:2/694: creat d0/d24/fda x:0 0 0 2024-02-19T19:04:59.032 INFO:tasks.workunit.client.1.smithi132.stdout:4/647: dwrite d4/d27/d31/d71/d37/f96 [0,4194304] 0 2024-02-19T19:04:59.034 INFO:tasks.workunit.client.1.smithi132.stdout:5/777: dwrite d8/da/db/f4f [0,4194304] 0 2024-02-19T19:04:59.036 INFO:tasks.workunit.client.1.smithi132.stdout:9/785: dwrite d12/d2a/f5b [0,4194304] 0 2024-02-19T19:04:59.044 INFO:tasks.workunit.client.1.smithi132.stdout:9/786: write d12/d8d/d3a/d40/d4e/d77/d89/dc4/ff8 [3131612,39648] 0 2024-02-19T19:04:59.045 INFO:tasks.workunit.client.1.smithi132.stdout:2/695: dwrite d0/d24/d58/d53/dab/d84/fa1 [0,4194304] 0 2024-02-19T19:04:59.045 INFO:tasks.workunit.client.1.smithi132.stdout:4/648: mknod d4/d27/d31/d32/d51/d5c/ce0 0 2024-02-19T19:04:59.051 INFO:tasks.workunit.client.1.smithi132.stdout:2/696: symlink d0/d4/df/dc2/dad/ldb 0 2024-02-19T19:04:59.054 INFO:tasks.workunit.client.1.smithi132.stdout:2/697: read d0/d4/d11/d2e/f6c [2128210,66798] 0 2024-02-19T19:04:59.054 INFO:tasks.workunit.client.1.smithi132.stdout:4/649: creat d4/d19/d21/fe1 x:0 0 0 2024-02-19T19:04:59.056 INFO:tasks.workunit.client.1.smithi132.stdout:9/787: creat d12/d8d/d3a/d40/d4e/d77/f112 x:0 0 0 2024-02-19T19:04:59.056 INFO:tasks.workunit.client.1.smithi132.stdout:8/715: sync 2024-02-19T19:04:59.057 INFO:tasks.workunit.client.1.smithi132.stdout:0/620: sync 2024-02-19T19:04:59.057 INFO:tasks.workunit.client.1.smithi132.stdout:1/592: sync 2024-02-19T19:04:59.057 INFO:tasks.workunit.client.1.smithi132.stdout:6/565: sync 2024-02-19T19:04:59.058 INFO:tasks.workunit.client.1.smithi132.stdout:2/698: mknod d0/d24/d58/d53/dab/d8a/da5/cdc 0 2024-02-19T19:04:59.062 INFO:tasks.workunit.client.1.smithi132.stdout:4/650: mknod d4/d12/d16/ddb/ce2 0 2024-02-19T19:04:59.063 INFO:tasks.workunit.client.1.smithi132.stdout:6/566: symlink d0/db/d25/d3a/d5d/lbf 0 2024-02-19T19:04:59.065 INFO:tasks.workunit.client.1.smithi132.stdout:0/621: creat d2/dc/df/d15/d34/d84/fd1 x:0 0 0 2024-02-19T19:04:59.067 INFO:tasks.workunit.client.1.smithi132.stdout:1/593: rename d1/d1e/d77/d52/l9b to d1/d1e/d77/d2c/d62/d88/lbc 0 2024-02-19T19:04:59.068 INFO:tasks.workunit.client.1.smithi132.stdout:4/651: unlink d4/d19/d21/l58 0 2024-02-19T19:04:59.071 INFO:tasks.workunit.client.1.smithi132.stdout:6/567: symlink d0/d51/da7/lc0 0 2024-02-19T19:04:59.071 INFO:tasks.workunit.client.1.smithi132.stdout:2/699: dread d0/d4/d11/f32 [0,4194304] 0 2024-02-19T19:04:59.072 INFO:tasks.workunit.client.1.smithi132.stdout:8/716: dwrite d4/d2d/f31 [4194304,4194304] 0 2024-02-19T19:04:59.076 INFO:tasks.workunit.client.1.smithi132.stdout:6/568: symlink d0/d8b/lc1 0 2024-02-19T19:04:59.076 INFO:tasks.workunit.client.1.smithi132.stdout:5/778: dread d8/da/d5a/f81 [0,4194304] 0 2024-02-19T19:04:59.076 INFO:tasks.workunit.client.1.smithi132.stdout:8/717: chown d4/dc/d1b/d33/f5b 633 1 2024-02-19T19:04:59.076 INFO:tasks.workunit.client.1.smithi132.stdout:4/652: truncate d4/d27/d31/d32/d51/f93 642078 0 2024-02-19T19:04:59.078 INFO:tasks.workunit.client.1.smithi132.stdout:1/594: link d1/d3a/f80 d1/d3a/d50/fbd 0 2024-02-19T19:04:59.079 INFO:tasks.workunit.client.1.smithi132.stdout:2/700: chown d0/d4/df/dc2/dad/l95 112211569 1 2024-02-19T19:04:59.079 INFO:tasks.workunit.client.1.smithi132.stdout:8/718: chown d4/dc/d1b/d33 1922 1 2024-02-19T19:04:59.079 INFO:tasks.workunit.client.1.smithi132.stdout:6/569: chown d0/db/d25/d3a/d5d/f5e 100 1 2024-02-19T19:04:59.082 INFO:tasks.workunit.client.1.smithi132.stdout:6/570: readlink d0/db/d25/d3a/d4b/d7c/l9f 0 2024-02-19T19:04:59.083 INFO:tasks.workunit.client.1.smithi132.stdout:5/779: write d8/d2e/d71/f80 [480357,83405] 0 2024-02-19T19:04:59.084 INFO:tasks.workunit.client.1.smithi132.stdout:8/719: mkdir d4/dc/d1b/d38/dfa 0 2024-02-19T19:04:59.084 INFO:tasks.workunit.client.1.smithi132.stdout:9/788: dread d12/d8d/f10b [4194304,4194304] 0 2024-02-19T19:04:59.085 INFO:tasks.workunit.client.1.smithi132.stdout:6/571: truncate d0/dc/f94 208612 0 2024-02-19T19:04:59.087 INFO:tasks.workunit.client.1.smithi132.stdout:6/572: stat d0/dc/f94 0 2024-02-19T19:04:59.087 INFO:tasks.workunit.client.1.smithi132.stdout:8/720: write d4/d2d/fbf [886694,32889] 0 2024-02-19T19:04:59.088 INFO:tasks.workunit.client.1.smithi132.stdout:0/622: dread d2/dc/df/d27/f35 [0,4194304] 0 2024-02-19T19:04:59.089 INFO:tasks.workunit.client.1.smithi132.stdout:1/595: rmdir d1/d1e/d77/d2c/d62/d6b 39 2024-02-19T19:04:59.091 INFO:tasks.workunit.client.1.smithi132.stdout:1/596: truncate d1/d3a/fa3 855704 0 2024-02-19T19:04:59.092 INFO:tasks.workunit.client.1.smithi132.stdout:5/780: symlink d8/da/dae/l10a 0 2024-02-19T19:04:59.094 INFO:tasks.workunit.client.1.smithi132.stdout:6/573: unlink d0/db/d25/d3a/d5d/d8f/d46/fa5 0 2024-02-19T19:04:59.095 INFO:tasks.workunit.client.1.smithi132.stdout:8/721: dwrite d4/dc/d1b/d20/f8b [0,4194304] 0 2024-02-19T19:04:59.098 INFO:tasks.workunit.client.1.smithi132.stdout:1/597: mknod d1/d61/cbe 0 2024-02-19T19:04:59.100 INFO:tasks.workunit.client.1.smithi132.stdout:9/789: rename d12/cd0 to d12/d3e/d6e/c113 0 2024-02-19T19:04:59.100 INFO:tasks.workunit.client.1.smithi132.stdout:5/781: dwrite d8/d23/d29/d27/f25 [0,4194304] 0 2024-02-19T19:04:59.108 INFO:tasks.workunit.client.1.smithi132.stdout:2/701: getdents d0/d4/df/dc2 0 2024-02-19T19:04:59.108 INFO:tasks.workunit.client.1.smithi132.stdout:5/782: dread d8/d23/d29/d27/f2b [0,4194304] 0 2024-02-19T19:04:59.109 INFO:tasks.workunit.client.1.smithi132.stdout:8/722: mkdir d4/dc/d1b/d54/d81/dbd/dfb 0 2024-02-19T19:04:59.109 INFO:tasks.workunit.client.1.smithi132.stdout:2/702: readlink d0/d19/l27 0 2024-02-19T19:04:59.111 INFO:tasks.workunit.client.1.smithi132.stdout:2/703: truncate d0/d24/fb3 595913 0 2024-02-19T19:04:59.112 INFO:tasks.workunit.client.1.smithi132.stdout:2/704: fsync d0/d4/df/d40/d68/f86 0 2024-02-19T19:04:59.114 INFO:tasks.workunit.client.1.smithi132.stdout:4/653: truncate d4/d12/f30 1487261 0 2024-02-19T19:04:59.116 INFO:tasks.workunit.client.1.smithi132.stdout:1/598: mknod d1/d1e/d77/d2c/cbf 0 2024-02-19T19:04:59.117 INFO:tasks.workunit.client.1.smithi132.stdout:5/783: mknod d8/d2e/dc4/dc8/c10b 0 2024-02-19T19:04:59.118 INFO:tasks.workunit.client.1.smithi132.stdout:2/705: creat d0/d3c/fdd x:0 0 0 2024-02-19T19:04:59.121 INFO:tasks.workunit.client.1.smithi132.stdout:0/623: rename d2/dc/df/d15/f22 to d2/dc/df/fd2 0 2024-02-19T19:04:59.121 INFO:tasks.workunit.client.1.smithi132.stdout:3/663: sync 2024-02-19T19:04:59.121 INFO:tasks.workunit.client.1.smithi132.stdout:7/727: sync 2024-02-19T19:04:59.122 INFO:tasks.workunit.client.1.smithi132.stdout:5/784: mknod d8/d1a/d64/c10c 0 2024-02-19T19:04:59.123 INFO:tasks.workunit.client.1.smithi132.stdout:0/624: fsync d2/dc/d4a/f42 0 2024-02-19T19:04:59.124 INFO:tasks.workunit.client.1.smithi132.stdout:6/574: getdents d0/d51/da7 0 2024-02-19T19:04:59.126 INFO:tasks.workunit.client.1.smithi132.stdout:2/706: creat d0/d19/d1e/fde x:0 0 0 2024-02-19T19:04:59.129 INFO:tasks.workunit.client.1.smithi132.stdout:6/575: chown d0/d8b/cb8 26 1 2024-02-19T19:04:59.129 INFO:tasks.workunit.client.1.smithi132.stdout:8/723: dwrite d4/dc/d1b/d8f/fa7 [0,4194304] 0 2024-02-19T19:04:59.131 INFO:tasks.workunit.client.1.smithi132.stdout:7/728: creat d1b/d1e/d28/fe4 x:0 0 0 2024-02-19T19:04:59.132 INFO:tasks.workunit.client.1.smithi132.stdout:6/576: readlink d0/db/d25/l52 0 2024-02-19T19:04:59.133 INFO:tasks.workunit.client.1.smithi132.stdout:4/654: creat d4/d27/d31/fe3 x:0 0 0 2024-02-19T19:04:59.133 INFO:tasks.workunit.client.1.smithi132.stdout:7/729: fdatasync d1b/d1e/d2b/fb4 0 2024-02-19T19:04:59.133 INFO:tasks.workunit.client.1.smithi132.stdout:9/790: dread d12/d8d/f29 [0,4194304] 0 2024-02-19T19:04:59.137 INFO:tasks.workunit.client.1.smithi132.stdout:9/791: chown d12/d3e/f84 106 1 2024-02-19T19:04:59.137 INFO:tasks.workunit.client.1.smithi132.stdout:5/785: chown d8/da/f30 111135273 1 2024-02-19T19:04:59.137 INFO:tasks.workunit.client.1.smithi132.stdout:0/625: mkdir d2/dc/df/d15/d2f/d3f/dd3 0 2024-02-19T19:04:59.142 INFO:tasks.workunit.client.1.smithi132.stdout:0/626: truncate d2/dc/df/d27/f7c 911820 0 2024-02-19T19:04:59.143 INFO:tasks.workunit.client.1.smithi132.stdout:3/664: rename d8/d13/d2b/d69/d92/f55 to d8/d13/d2b/d69/d92/d8d/db2/fd0 0 2024-02-19T19:04:59.143 INFO:tasks.workunit.client.1.smithi132.stdout:9/792: dwrite d12/d8d/f30 [0,4194304] 0 2024-02-19T19:04:59.144 INFO:tasks.workunit.client.1.smithi132.stdout:9/793: stat d12/d8d/d3a/cfa 0 2024-02-19T19:04:59.144 INFO:tasks.workunit.client.1.smithi132.stdout:3/665: chown d8/d13/d2b/d69/da6/l5f 60096380 1 2024-02-19T19:04:59.145 INFO:tasks.workunit.client.1.smithi132.stdout:9/794: chown d12/d2a/d88/db8/fb9 94633 1 2024-02-19T19:04:59.146 INFO:tasks.workunit.client.1.smithi132.stdout:1/599: dread d1/d1e/d77/f3f [0,4194304] 0 2024-02-19T19:04:59.147 INFO:tasks.workunit.client.1.smithi132.stdout:8/724: mkdir d4/d2d/dcc/dfc 0 2024-02-19T19:04:59.147 INFO:tasks.workunit.client.1.smithi132.stdout:8/725: stat d4/dc/d1b/d20/d8a/c90 0 2024-02-19T19:04:59.152 INFO:tasks.workunit.client.1.smithi132.stdout:6/577: mknod d0/db/d25/d3a/d3d/d69/cc2 0 2024-02-19T19:04:59.155 INFO:tasks.workunit.client.1.smithi132.stdout:2/707: mknod d0/d24/d58/d53/dab/d8a/da5/db1/cdf 0 2024-02-19T19:04:59.158 INFO:tasks.workunit.client.1.smithi132.stdout:5/786: truncate d8/d2e/d71/da6/ddb/dea/f102 524941 0 2024-02-19T19:04:59.159 INFO:tasks.workunit.client.1.smithi132.stdout:9/795: unlink d12/d8d/d3a/d40/d4e/d77/d89/lcc 0 2024-02-19T19:04:59.161 INFO:tasks.workunit.client.1.smithi132.stdout:1/600: mkdir d1/d1e/d77/d78/dc0 0 2024-02-19T19:04:59.163 INFO:tasks.workunit.client.1.smithi132.stdout:7/730: mkdir d1b/d1e/d41/da3/de5 0 2024-02-19T19:04:59.163 INFO:tasks.workunit.client.1.smithi132.stdout:0/627: dread d2/dc/df/d27/f61 [0,4194304] 0 2024-02-19T19:04:59.163 INFO:tasks.workunit.client.1.smithi132.stdout:0/628: chown d2/dc/df 17207071 1 2024-02-19T19:04:59.164 INFO:tasks.workunit.client.1.smithi132.stdout:6/578: mkdir d0/db/d25/d50/d86/dc3 0 2024-02-19T19:04:59.166 INFO:tasks.workunit.client.1.smithi132.stdout:3/666: mkdir d8/dd1 0 2024-02-19T19:04:59.167 INFO:tasks.workunit.client.1.smithi132.stdout:5/787: rmdir d8/d1a/d64/de2 39 2024-02-19T19:04:59.167 INFO:tasks.workunit.client.1.smithi132.stdout:5/788: fsync d8/d2e/f91 0 2024-02-19T19:04:59.168 INFO:tasks.workunit.client.1.smithi132.stdout:5/789: write d8/da/d5a/d7b/f99 [695190,45155] 0 2024-02-19T19:04:59.168 INFO:tasks.workunit.client.1.smithi132.stdout:9/796: mkdir d12/d8d/d3a/d85/d87/d114 0 2024-02-19T19:04:59.169 INFO:tasks.workunit.client.1.smithi132.stdout:8/726: symlink d4/d2d/dd5/lfd 0 2024-02-19T19:04:59.170 INFO:tasks.workunit.client.1.smithi132.stdout:8/727: chown d4/dc/d1b/d38/fc6 452 1 2024-02-19T19:04:59.171 INFO:tasks.workunit.client.1.smithi132.stdout:7/731: creat d1b/d1e/d41/db5/fe6 x:0 0 0 2024-02-19T19:04:59.171 INFO:tasks.workunit.client.1.smithi132.stdout:0/629: rmdir d2/dc/d4a 39 2024-02-19T19:04:59.173 INFO:tasks.workunit.client.1.smithi132.stdout:0/630: write d2/dc/df/d15/d2f/f40 [2232271,108953] 0 2024-02-19T19:04:59.175 INFO:tasks.workunit.client.1.smithi132.stdout:6/579: unlink d0/db/d25/d3a/d5d/d8f/d46/d5a/f29 0 2024-02-19T19:04:59.176 INFO:tasks.workunit.client.1.smithi132.stdout:8/728: dwrite d4/f19 [0,4194304] 0 2024-02-19T19:04:59.177 INFO:tasks.workunit.client.1.smithi132.stdout:8/729: chown d4/dc/d42/fcb 15682 1 2024-02-19T19:04:59.178 INFO:tasks.workunit.client.1.smithi132.stdout:5/790: unlink d8/lf3 0 2024-02-19T19:04:59.181 INFO:tasks.workunit.client.1.smithi132.stdout:9/797: rename d12/d8d/d3a/d85/ffc to d12/d8d/f115 0 2024-02-19T19:04:59.185 INFO:tasks.workunit.client.1.smithi132.stdout:2/708: link d0/d19/d6a/cb4 d0/d4/df/d40/d68/ce0 0 2024-02-19T19:04:59.187 INFO:tasks.workunit.client.1.smithi132.stdout:0/631: chown d2/dc/d4a/d25 142945 1 2024-02-19T19:04:59.224 INFO:tasks.workunit.client.1.smithi132.stdout:3/667: mkdir d8/dd1/dd2 0 2024-02-19T19:04:59.225 INFO:tasks.workunit.client.1.smithi132.stdout:9/798: unlink d12/d2a/d4c/d61/f63 0 2024-02-19T19:04:59.225 INFO:tasks.workunit.client.1.smithi132.stdout:2/709: unlink d0/d24/d58/d53/dab/d8a/da5/cdc 0 2024-02-19T19:04:59.225 INFO:tasks.workunit.client.1.smithi132.stdout:0/632: write d2/dc/df/d27/d3a/fb4 [1088967,26625] 0 2024-02-19T19:04:59.228 INFO:tasks.workunit.client.1.smithi132.stdout:1/601: write d1/f42 [376070,21986] 0 2024-02-19T19:04:59.229 INFO:tasks.workunit.client.1.smithi132.stdout:8/730: mknod d4/dc/d1b/d20/d23/cfe 0 2024-02-19T19:04:59.230 INFO:tasks.workunit.client.1.smithi132.stdout:3/668: write d8/d38/f4b [2208627,128729] 0 2024-02-19T19:04:59.233 INFO:tasks.workunit.client.1.smithi132.stdout:5/791: truncate d8/d23/d29/d27/f57 2377571 0 2024-02-19T19:04:59.234 INFO:tasks.workunit.client.1.smithi132.stdout:0/633: symlink d2/d3e/db1/ld4 0 2024-02-19T19:04:59.235 INFO:tasks.workunit.client.1.smithi132.stdout:2/710: rmdir d0/d24/dcb 0 2024-02-19T19:04:59.239 INFO:tasks.workunit.client.1.smithi132.stdout:7/732: dwrite d1b/d1e/d2b/d62/d53/f65 [0,4194304] 0 2024-02-19T19:04:59.240 INFO:tasks.workunit.client.1.smithi132.stdout:5/792: creat d8/d2e/db3/f10d x:0 0 0 2024-02-19T19:04:59.248 INFO:tasks.workunit.client.1.smithi132.stdout:3/669: dread d8/d13/f37 [0,4194304] 0 2024-02-19T19:04:59.248 INFO:tasks.workunit.client.1.smithi132.stdout:1/602: truncate d1/f4a 141627 0 2024-02-19T19:04:59.253 INFO:tasks.workunit.client.1.smithi132.stdout:9/799: dwrite d12/d2a/d4c/d61/f6b [0,4194304] 0 2024-02-19T19:04:59.253 INFO:tasks.workunit.client.1.smithi132.stdout:7/733: creat d1b/d1e/d28/d5e/d99/fe7 x:0 0 0 2024-02-19T19:04:59.253 INFO:tasks.workunit.client.1.smithi132.stdout:5/793: creat d8/d2e/d71/da6/f10e x:0 0 0 2024-02-19T19:04:59.253 INFO:tasks.workunit.client.1.smithi132.stdout:0/634: rename d2/l8 to d2/d6e/dc8/ld5 0 2024-02-19T19:04:59.254 INFO:tasks.workunit.client.1.smithi132.stdout:5/794: stat f5 0 2024-02-19T19:04:59.254 INFO:tasks.workunit.client.1.smithi132.stdout:2/711: dread d0/d19/d1e/f45 [0,4194304] 0 2024-02-19T19:04:59.254 INFO:tasks.workunit.client.1.smithi132.stdout:9/800: chown d12/d8d/d3a/d40/d4e/d77/d8a/dca/l106 307590 1 2024-02-19T19:04:59.255 INFO:tasks.workunit.client.1.smithi132.stdout:1/603: chown d1/d3a/f80 6086 1 2024-02-19T19:04:59.257 INFO:tasks.workunit.client.1.smithi132.stdout:2/712: write d0/d24/d58/f36 [357938,101082] 0 2024-02-19T19:04:59.257 INFO:tasks.workunit.client.1.smithi132.stdout:7/734: write d1b/d46/d56/f57 [3467525,83376] 0 2024-02-19T19:04:59.258 INFO:tasks.workunit.client.1.smithi132.stdout:1/604: stat d1/d3a 0 2024-02-19T19:04:59.260 INFO:tasks.workunit.client.1.smithi132.stdout:9/801: dread - d12/d2a/d4c/dad/db2/f10c zero size 2024-02-19T19:04:59.262 INFO:tasks.workunit.client.1.smithi132.stdout:9/802: chown d12/d2a/d4c/fb3 10 1 2024-02-19T19:04:59.262 INFO:tasks.workunit.client.1.smithi132.stdout:7/735: write d1b/d1e/d28/f44 [1584433,24206] 0 2024-02-19T19:04:59.264 INFO:tasks.workunit.client.1.smithi132.stdout:9/803: readlink d12/d2a/d4c/dad/db2/lf6 0 2024-02-19T19:04:59.265 INFO:tasks.workunit.client.1.smithi132.stdout:8/731: dread f2 [0,4194304] 0 2024-02-19T19:04:59.267 INFO:tasks.workunit.client.1.smithi132.stdout:0/635: rename d2/dc/d98/la0 to d2/dc/d98/ld6 0 2024-02-19T19:04:59.269 INFO:tasks.workunit.client.1.smithi132.stdout:2/713: mkdir d0/d24/d58/d53/dab/d84/de1 0 2024-02-19T19:04:59.270 INFO:tasks.workunit.client.1.smithi132.stdout:3/670: truncate d8/d13/d2b/d69/d92/daf/fc3 251075 0 2024-02-19T19:04:59.271 INFO:tasks.workunit.client.1.smithi132.stdout:1/605: creat d1/d1e/d77/d2c/d43/d81/da1/fc1 x:0 0 0 2024-02-19T19:04:59.273 INFO:tasks.workunit.client.1.smithi132.stdout:9/804: write d12/d2a/f5b [5103646,10054] 0 2024-02-19T19:04:59.275 INFO:tasks.workunit.client.1.smithi132.stdout:4/655: sync 2024-02-19T19:04:59.276 INFO:tasks.workunit.client.1.smithi132.stdout:7/736: rename d1b/d1e/d2b/d62/d53/fa4 to d1b/d1e/d41/fe8 0 2024-02-19T19:04:59.277 INFO:tasks.workunit.client.1.smithi132.stdout:8/732: creat d4/dc/d1b/d20/d8a/fff x:0 0 0 2024-02-19T19:04:59.278 INFO:tasks.workunit.client.1.smithi132.stdout:0/636: dwrite d2/dc/df/d15/d2f/f40 [0,4194304] 0 2024-02-19T19:04:59.278 INFO:tasks.workunit.client.1.smithi132.stdout:2/714: stat d0/d3c/d3f/l50 0 2024-02-19T19:04:59.282 INFO:tasks.workunit.client.1.smithi132.stdout:0/637: mkdir d2/d6e/dc8/dd7 0 2024-02-19T19:04:59.283 INFO:tasks.workunit.client.1.smithi132.stdout:7/737: write d1b/d1e/d28/d5e/d99/fca [1238239,20315] 0 2024-02-19T19:04:59.284 INFO:tasks.workunit.client.1.smithi132.stdout:4/656: mkdir d4/d27/d31/d71/d14/de4 0 2024-02-19T19:04:59.284 INFO:tasks.workunit.client.1.smithi132.stdout:5/795: dread d8/d23/d29/d27/f87 [0,4194304] 0 2024-02-19T19:04:59.285 INFO:tasks.workunit.client.1.smithi132.stdout:7/738: stat d1b/d1e/d28/d5e 0 2024-02-19T19:04:59.286 INFO:tasks.workunit.client.1.smithi132.stdout:5/796: fdatasync d8/da/f8c 0 2024-02-19T19:04:59.287 INFO:tasks.workunit.client.1.smithi132.stdout:4/657: write d4/d27/d7f/dc0/fcc [325859,79657] 0 2024-02-19T19:04:59.288 INFO:tasks.workunit.client.1.smithi132.stdout:8/733: creat d4/dc/d1b/d38/d69/d74/d93/f100 x:0 0 0 2024-02-19T19:04:59.292 INFO:tasks.workunit.client.1.smithi132.stdout:2/715: creat d0/d4/df/d40/d7b/fe2 x:0 0 0 2024-02-19T19:04:59.293 INFO:tasks.workunit.client.1.smithi132.stdout:9/805: symlink d12/d8d/d3a/d40/d4e/d77/d89/dc4/d110/l116 0 2024-02-19T19:04:59.297 INFO:tasks.workunit.client.1.smithi132.stdout:7/739: creat d1b/d1e/d2b/d62/fe9 x:0 0 0 2024-02-19T19:04:59.298 INFO:tasks.workunit.client.1.smithi132.stdout:3/671: rename d8/d13/d2b/d69/d92/d34/f81 to d8/d13/d2b/d69/fd3 0 2024-02-19T19:04:59.304 INFO:tasks.workunit.client.1.smithi132.stdout:0/638: rename d2/dc/df/f2d to d2/dc/df/dbd/fd8 0 2024-02-19T19:04:59.304 INFO:tasks.workunit.client.1.smithi132.stdout:8/734: dwrite d4/d2d/dcc/ff6 [0,4194304] 0 2024-02-19T19:04:59.306 INFO:tasks.workunit.client.1.smithi132.stdout:9/806: mknod d12/d8d/d3a/d40/d4e/c117 0 2024-02-19T19:04:59.307 INFO:tasks.workunit.client.1.smithi132.stdout:3/672: write d8/d13/d2f/fc1 [362593,94881] 0 2024-02-19T19:04:59.307 INFO:tasks.workunit.client.1.smithi132.stdout:7/740: fdatasync d1b/d1e/d28/f3c 0 2024-02-19T19:04:59.310 INFO:tasks.workunit.client.1.smithi132.stdout:5/797: dread d8/f62 [0,4194304] 0 2024-02-19T19:04:59.310 INFO:tasks.workunit.client.1.smithi132.stdout:6/580: sync 2024-02-19T19:04:59.313 INFO:tasks.workunit.client.1.smithi132.stdout:0/639: fsync d2/dc/df/d78/f9f 0 2024-02-19T19:04:59.314 INFO:tasks.workunit.client.1.smithi132.stdout:5/798: chown d8/d23/d3b/l78 2464844 1 2024-02-19T19:04:59.316 INFO:tasks.workunit.client.1.smithi132.stdout:2/716: dwrite d0/d24/d58/d53/dab/d8a/da5/db1/fb6 [0,4194304] 0 2024-02-19T19:04:59.317 INFO:tasks.workunit.client.1.smithi132.stdout:8/735: mkdir d4/dc/d1b/d20/d62/d91/d101 0 2024-02-19T19:04:59.320 INFO:tasks.workunit.client.1.smithi132.stdout:1/606: dwrite d1/d1e/d77/d4e/f82 [0,4194304] 0 2024-02-19T19:04:59.325 INFO:tasks.workunit.client.1.smithi132.stdout:7/741: getdents d1b/d1e/d2b/dd0 0 2024-02-19T19:04:59.325 INFO:tasks.workunit.client.1.smithi132.stdout:3/673: creat d8/fd4 x:0 0 0 2024-02-19T19:04:59.328 INFO:tasks.workunit.client.1.smithi132.stdout:2/717: symlink d0/d4/df/dc2/dad/le3 0 2024-02-19T19:04:59.329 INFO:tasks.workunit.client.1.smithi132.stdout:7/742: write d1b/d1e/d28/f97 [629990,118134] 0 2024-02-19T19:04:59.330 INFO:tasks.workunit.client.1.smithi132.stdout:1/607: write d1/d1e/f74 [618278,82639] 0 2024-02-19T19:04:59.331 INFO:tasks.workunit.client.1.smithi132.stdout:4/658: rename d4/d19/l2b to d4/d19/d21/le5 0 2024-02-19T19:04:59.334 INFO:tasks.workunit.client.1.smithi132.stdout:2/718: write d0/d4/df/f3b [599761,87483] 0 2024-02-19T19:04:59.334 INFO:tasks.workunit.client.1.smithi132.stdout:5/799: dwrite d8/da/f30 [0,4194304] 0 2024-02-19T19:04:59.337 INFO:tasks.workunit.client.1.smithi132.stdout:6/581: getdents d0/db/d25 0 2024-02-19T19:04:59.338 INFO:tasks.workunit.client.1.smithi132.stdout:1/608: dread - d1/d1e/d77/d2c/d62/fb4 zero size 2024-02-19T19:04:59.342 INFO:tasks.workunit.client.1.smithi132.stdout:5/800: dread - d8/da/db/f109 zero size 2024-02-19T19:04:59.343 INFO:tasks.workunit.client.1.smithi132.stdout:0/640: dread d2/dc/d4a/d58/fa6 [0,4194304] 0 2024-02-19T19:04:59.345 INFO:tasks.workunit.client.1.smithi132.stdout:9/807: rename d12/d3e to d12/d8d/d3a/d85/d87/d114/d118 0 2024-02-19T19:04:59.347 INFO:tasks.workunit.client.1.smithi132.stdout:6/582: stat d0/db/d25/d3a/d3d/f74 0 2024-02-19T19:04:59.348 INFO:tasks.workunit.client.1.smithi132.stdout:1/609: dread d1/d1e/f84 [0,4194304] 0 2024-02-19T19:04:59.349 INFO:tasks.workunit.client.1.smithi132.stdout:2/719: read d0/d4/d11/f15 [4407489,40203] 0 2024-02-19T19:04:59.355 INFO:tasks.workunit.client.1.smithi132.stdout:0/641: mkdir d2/dc/df/d15/dd9 0 2024-02-19T19:04:59.356 INFO:tasks.workunit.client.1.smithi132.stdout:8/736: dread d4/f83 [0,4194304] 0 2024-02-19T19:04:59.357 INFO:tasks.workunit.client.1.smithi132.stdout:0/642: write d2/dc/df/d15/d2f/f64 [2742916,49594] 0 2024-02-19T19:04:59.357 INFO:tasks.workunit.client.1.smithi132.stdout:2/720: rename d0/d4/f22 to d0/d24/d58/d53/dab/fe4 0 2024-02-19T19:04:59.359 INFO:tasks.workunit.client.1.smithi132.stdout:3/674: link d8/d13/d2b/d69/da6/lc5 d8/d13/d2b/d60/dc2/ld5 0 2024-02-19T19:04:59.361 INFO:tasks.workunit.client.1.smithi132.stdout:5/801: getdents d8/da/d5a/d7b/dfa 0 2024-02-19T19:04:59.362 INFO:tasks.workunit.client.1.smithi132.stdout:6/583: dread d0/db/d25/d3a/d5d/f5e [0,4194304] 0 2024-02-19T19:04:59.366 INFO:tasks.workunit.client.1.smithi132.stdout:2/721: truncate d0/d24/d58/d53/f91 323014 0 2024-02-19T19:04:59.367 INFO:tasks.workunit.client.1.smithi132.stdout:8/737: dread d4/dc/f1c [0,4194304] 0 2024-02-19T19:04:59.368 INFO:tasks.workunit.client.1.smithi132.stdout:3/675: write d8/d13/d2b/d69/d92/d34/fa3 [363315,87975] 0 2024-02-19T19:04:59.369 INFO:tasks.workunit.client.1.smithi132.stdout:7/743: dread d1b/d1e/d2b/f95 [0,4194304] 0 2024-02-19T19:04:59.369 INFO:tasks.workunit.client.1.smithi132.stdout:6/584: unlink d0/dc/f24 0 2024-02-19T19:04:59.378 INFO:tasks.workunit.client.1.smithi132.stdout:4/659: dread d4/d27/d31/d32/d51/d5c/f5d [0,4194304] 0 2024-02-19T19:04:59.379 INFO:tasks.workunit.client.1.smithi132.stdout:0/643: dwrite d2/dc/df/d15/f46 [0,4194304] 0 2024-02-19T19:04:59.381 INFO:tasks.workunit.client.1.smithi132.stdout:1/610: dread d1/d3a/fac [0,4194304] 0 2024-02-19T19:04:59.381 INFO:tasks.workunit.client.1.smithi132.stdout:8/738: truncate d4/dc/d1b/d33/fb7 471172 0 2024-02-19T19:04:59.381 INFO:tasks.workunit.client.1.smithi132.stdout:5/802: truncate f5 8889532 0 2024-02-19T19:04:59.387 INFO:tasks.workunit.client.1.smithi132.stdout:3/676: mkdir d8/d13/d2b/d69/d7e/dd6 0 2024-02-19T19:04:59.388 INFO:tasks.workunit.client.1.smithi132.stdout:6/585: dwrite d0/dc/f66 [0,4194304] 0 2024-02-19T19:04:59.390 INFO:tasks.workunit.client.1.smithi132.stdout:5/803: creat d8/d23/d3b/f10f x:0 0 0 2024-02-19T19:04:59.393 INFO:tasks.workunit.client.1.smithi132.stdout:8/739: mknod d4/d2d/dcc/dfc/c102 0 2024-02-19T19:04:59.395 INFO:tasks.workunit.client.1.smithi132.stdout:1/611: creat d1/d1e/d77/d4e/d99/fc2 x:0 0 0 2024-02-19T19:04:59.397 INFO:tasks.workunit.client.1.smithi132.stdout:2/722: dread d0/d24/d58/d53/dab/d8a/da5/db1/fb6 [0,4194304] 0 2024-02-19T19:04:59.401 INFO:tasks.workunit.client.1.smithi132.stdout:7/744: dwrite d1b/d1e/f38 [4194304,4194304] 0 2024-02-19T19:04:59.405 INFO:tasks.workunit.client.1.smithi132.stdout:7/745: dread - d1b/d1e/d41/fbf zero size 2024-02-19T19:04:59.406 INFO:tasks.workunit.client.1.smithi132.stdout:2/723: mknod d0/d3c/ce5 0 2024-02-19T19:04:59.406 INFO:tasks.workunit.client.1.smithi132.stdout:3/677: rename d8/d13/d2b/d69/d7e/d9f to d8/d13/d2b/d69/d92/d8d/dd7 0 2024-02-19T19:04:59.408 INFO:tasks.workunit.client.1.smithi132.stdout:6/586: dwrite d0/db/d25/d3a/d5d/d8f/d46/d5a/f26 [0,4194304] 0 2024-02-19T19:04:59.421 INFO:tasks.workunit.client.1.smithi132.stdout:2/724: dwrite d0/f6 [4194304,4194304] 0 2024-02-19T19:04:59.430 INFO:tasks.workunit.client.1.smithi132.stdout:0/644: dwrite d2/dc/f3b [0,4194304] 0 2024-02-19T19:04:59.434 INFO:tasks.workunit.client.1.smithi132.stdout:4/660: dread d4/d12/d48/fde [0,4194304] 0 2024-02-19T19:04:59.442 INFO:tasks.workunit.client.0.smithi088.stderr:Note: switching to '1ce7ea8877c709fa862eac608550374bd96e166e'. 2024-02-19T19:04:59.442 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr:state without impacting any branches by switching back to a branch. 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr: git switch -c 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr:Or undo this operation with: 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr: git switch - 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-19T19:04:59.443 INFO:tasks.workunit.client.0.smithi088.stderr: 2024-02-19T19:04:59.444 INFO:tasks.workunit.client.0.smithi088.stderr:HEAD is now at 1ce7ea8877c fix 2024-02-19T19:04:59.446 INFO:tasks.workunit.client.1.smithi132.stdout:8/740: dread d4/dc/d1b/d33/f49 [0,4194304] 0 2024-02-19T19:04:59.448 INFO:tasks.workunit.client.1.smithi132.stdout:8/741: truncate d4/dc/d1b/d20/f9f 263751 0 2024-02-19T19:04:59.451 DEBUG:teuthology.orchestra.run.smithi088:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-19T19:04:59.455 INFO:tasks.workunit.client.1.smithi132.stdout:8/742: dread d4/dc/d1b/d20/d62/d91/fb2 [0,4194304] 0 2024-02-19T19:04:59.456 INFO:tasks.workunit.client.1.smithi132.stdout:8/743: chown d4/dc/d1b/fd3 0 1 2024-02-19T19:04:59.456 INFO:tasks.workunit.client.1.smithi132.stdout:5/804: getdents d8/d2e/dc4 0 2024-02-19T19:04:59.457 INFO:tasks.workunit.client.1.smithi132.stdout:8/744: dread - d4/dc/d1b/d38/d69/d74/fe1 zero size 2024-02-19T19:04:59.458 INFO:tasks.workunit.client.1.smithi132.stdout:8/745: fsync d4/dc/d1b/d38/fb8 0 2024-02-19T19:04:59.461 INFO:tasks.workunit.client.1.smithi132.stdout:3/678: creat d8/d13/d2b/d69/d92/d8d/fd8 x:0 0 0 2024-02-19T19:04:59.463 INFO:tasks.workunit.client.1.smithi132.stdout:3/679: write d8/d13/d2b/d69/d92/d8d/fd8 [969081,30984] 0 2024-02-19T19:04:59.478 INFO:tasks.workunit.client.1.smithi132.stdout:8/746: getdents d4/dc/d1b/d38/dfa 0 2024-02-19T19:04:59.479 INFO:tasks.workunit.client.1.smithi132.stdout:8/747: chown d4/f6 14 1 2024-02-19T19:04:59.481 INFO:tasks.workunit.client.1.smithi132.stdout:2/725: link d0/d3c/d3f/c85 d0/d19/d6a/ce6 0 2024-02-19T19:04:59.485 INFO:tasks.workunit.client.1.smithi132.stdout:5/805: getdents d8/d1a/d64 0 2024-02-19T19:04:59.491 INFO:tasks.workunit.client.1.smithi132.stdout:3/680: creat d8/d13/d2b/d69/d92/d34/d42/fd9 x:0 0 0 2024-02-19T19:04:59.504 INFO:tasks.workunit.client.1.smithi132.stdout:9/808: sync 2024-02-19T19:04:59.507 INFO:tasks.workunit.client.1.smithi132.stdout:9/809: creat d12/d8d/d3a/d85/d87/f119 x:0 0 0 2024-02-19T19:04:59.511 INFO:tasks.workunit.client.1.smithi132.stdout:7/746: rename d1b/d1e/d2b/d62/c9c to d1b/d5a/d63/d6b/d79/cea 0 2024-02-19T19:04:59.512 INFO:tasks.workunit.client.1.smithi132.stdout:5/806: rmdir d8/da/d5a/d7b 39 2024-02-19T19:04:59.513 INFO:tasks.workunit.client.1.smithi132.stdout:8/748: creat d4/dc/d1b/d20/d23/d68/f103 x:0 0 0 2024-02-19T19:04:59.517 INFO:tasks.workunit.client.1.smithi132.stdout:9/810: mkdir d12/d2a/d4c/dad/d11a 0 2024-02-19T19:04:59.520 INFO:tasks.workunit.client.1.smithi132.stdout:6/587: write d0/dc/f2d [688230,60420] 0 2024-02-19T19:04:59.523 INFO:tasks.workunit.client.1.smithi132.stdout:4/661: write d4/d12/d28/f3e [2768850,50538] 0 2024-02-19T19:04:59.523 INFO:tasks.workunit.client.1.smithi132.stdout:8/749: dwrite d4/dc/d1b/d38/d69/d88/fc2 [0,4194304] 0 2024-02-19T19:04:59.526 INFO:tasks.workunit.client.1.smithi132.stdout:9/811: unlink d12/f23 0 2024-02-19T19:04:59.528 INFO:tasks.workunit.client.1.smithi132.stdout:9/812: write d12/d1b/f17 [4342329,27667] 0 2024-02-19T19:04:59.530 INFO:tasks.workunit.client.1.smithi132.stdout:8/750: symlink d4/d2d/dcc/l104 0 2024-02-19T19:04:59.530 INFO:tasks.workunit.client.1.smithi132.stdout:9/813: chown d12/c21 120 1 2024-02-19T19:04:59.533 INFO:tasks.workunit.client.1.smithi132.stdout:8/751: symlink d4/dc/d1b/d38/d69/d88/l105 0 2024-02-19T19:04:59.534 INFO:tasks.workunit.client.1.smithi132.stdout:8/752: write d4/d2d/f46 [1394696,85483] 0 2024-02-19T19:04:59.535 INFO:tasks.workunit.client.1.smithi132.stdout:9/814: mknod d12/d8d/d3a/d85/d87/d114/c11b 0 2024-02-19T19:04:59.537 INFO:tasks.workunit.client.1.smithi132.stdout:0/645: rename d2/dc/df/d15/d2f/d3f/d63/d6b to d2/dc/d4a/d25/dda 0 2024-02-19T19:04:59.537 INFO:tasks.workunit.client.1.smithi132.stdout:8/753: truncate d4/dc/d1b/d38/d69/d74/dce/fe2 925774 0 2024-02-19T19:04:59.539 INFO:tasks.workunit.client.1.smithi132.stdout:9/815: rmdir d12/d2a/d4c/d61/dfe 39 2024-02-19T19:04:59.540 INFO:tasks.workunit.client.1.smithi132.stdout:2/726: truncate d0/d4/d11/f20 1322757 0 2024-02-19T19:04:59.541 INFO:tasks.workunit.client.1.smithi132.stdout:8/754: write d4/dc/d1b/d8f/daf/fcf [781591,69264] 0 2024-02-19T19:04:59.545 INFO:tasks.workunit.client.1.smithi132.stdout:3/681: write d8/d38/d44/f5e [2349892,81403] 0 2024-02-19T19:04:59.545 INFO:tasks.workunit.client.1.smithi132.stdout:2/727: mknod d0/d4/df/d7a/ce7 0 2024-02-19T19:04:59.547 INFO:tasks.workunit.client.1.smithi132.stdout:8/755: symlink d4/dc/d1b/d54/d81/l106 0 2024-02-19T19:04:59.548 INFO:tasks.workunit.client.1.smithi132.stdout:5/807: rmdir d8/da 39 2024-02-19T19:04:59.548 INFO:tasks.workunit.client.1.smithi132.stdout:2/728: creat d0/d19/d6a/fe8 x:0 0 0 2024-02-19T19:04:59.551 INFO:tasks.workunit.client.1.smithi132.stdout:2/729: write d0/d4/df/d7a/f88 [358607,31785] 0 2024-02-19T19:04:59.551 INFO:tasks.workunit.client.1.smithi132.stdout:8/756: creat d4/dac/f107 x:0 0 0 2024-02-19T19:04:59.555 INFO:tasks.workunit.client.1.smithi132.stdout:3/682: getdents d8/d13/d2b/d69/d92/d34/d42 0 2024-02-19T19:04:59.556 INFO:tasks.workunit.client.1.smithi132.stdout:5/808: dwrite d8/d23/d29/d27/f25 [0,4194304] 0 2024-02-19T19:04:59.559 INFO:tasks.workunit.client.1.smithi132.stdout:0/646: dread d2/dc/df/d15/f24 [0,4194304] 0 2024-02-19T19:04:59.560 INFO:tasks.workunit.client.1.smithi132.stdout:7/747: write d1b/d1e/d41/fe8 [2102332,86054] 0 2024-02-19T19:04:59.560 INFO:tasks.workunit.client.1.smithi132.stdout:9/816: rmdir d12/d2a/d4c 39 2024-02-19T19:04:59.563 INFO:tasks.workunit.client.1.smithi132.stdout:2/730: unlink d0/l73 0 2024-02-19T19:04:59.563 INFO:tasks.workunit.client.1.smithi132.stdout:0/647: fdatasync d2/dc/df/d15/d2f/d3f/f79 0 2024-02-19T19:04:59.565 INFO:tasks.workunit.client.1.smithi132.stdout:4/662: write d4/d27/d31/f68 [632596,108340] 0 2024-02-19T19:04:59.565 INFO:tasks.workunit.client.1.smithi132.stdout:7/748: fdatasync f8 0 2024-02-19T19:04:59.566 INFO:tasks.workunit.client.1.smithi132.stdout:3/683: truncate d8/fc0 1878166 0 2024-02-19T19:04:59.567 INFO:tasks.workunit.client.1.smithi132.stdout:0/648: chown d2/d6e/dc8/dd7 0 1 2024-02-19T19:04:59.569 INFO:tasks.workunit.client.1.smithi132.stdout:8/757: dwrite d4/d2d/d82/f98 [0,4194304] 0 2024-02-19T19:04:59.569 INFO:tasks.workunit.client.1.smithi132.stdout:7/749: readlink d1b/d1e/d28/d5e/lcd 0 2024-02-19T19:04:59.569 INFO:tasks.workunit.client.1.smithi132.stdout:9/817: unlink d12/d8d/d3a/d40/d4e/d77/f8e 0 2024-02-19T19:04:59.571 INFO:tasks.workunit.client.1.smithi132.stdout:3/684: write d8/d13/d2b/d69/d92/f8c [282069,37678] 0 2024-02-19T19:04:59.575 INFO:tasks.workunit.client.1.smithi132.stdout:4/663: unlink d4/d19/d21/d56/f8f 0 2024-02-19T19:04:59.577 INFO:tasks.workunit.client.1.smithi132.stdout:0/649: creat d2/dc/df/d15/d34/fdb x:0 0 0 2024-02-19T19:04:59.577 INFO:tasks.workunit.client.1.smithi132.stdout:7/750: write ff [4813342,26916] 0 2024-02-19T19:04:59.578 INFO:tasks.workunit.client.1.smithi132.stdout:4/664: fsync d4/d27/d31/fbf 0 2024-02-19T19:04:59.578 INFO:tasks.workunit.client.1.smithi132.stdout:8/758: creat d4/dac/f108 x:0 0 0 2024-02-19T19:04:59.580 INFO:tasks.workunit.client.1.smithi132.stdout:2/731: creat d0/fe9 x:0 0 0 2024-02-19T19:04:59.582 INFO:tasks.workunit.client.1.smithi132.stdout:0/650: fsync d2/dc/df/f71 0 2024-02-19T19:04:59.583 INFO:tasks.workunit.client.1.smithi132.stdout:3/685: chown d8/d13/d2b/d69/d92/d34/f59 29 1 2024-02-19T19:04:59.585 INFO:tasks.workunit.client.1.smithi132.stdout:5/809: link d8/da/d5a/d7b/cd4 d8/d2e/d71/da6/ddb/c110 0 2024-02-19T19:04:59.587 INFO:tasks.workunit.client.1.smithi132.stdout:9/818: dwrite d12/d8d/d3a/d40/d4e/d77/f112 [0,4194304] 0 2024-02-19T19:04:59.589 INFO:tasks.workunit.client.1.smithi132.stdout:0/651: dread - d2/f8c zero size 2024-02-19T19:04:59.593 INFO:tasks.workunit.client.1.smithi132.stdout:3/686: write d8/d13/d2b/d60/f95 [1018358,101009] 0 2024-02-19T19:04:59.595 INFO:tasks.workunit.client.1.smithi132.stdout:8/759: link d4/dac/lbb d4/dc/d1b/d54/d5e/l109 0 2024-02-19T19:04:59.595 INFO:tasks.workunit.client.1.smithi132.stdout:9/819: readlink d12/d2a/d4c/d61/l68 0 2024-02-19T19:04:59.597 INFO:tasks.workunit.client.1.smithi132.stdout:4/665: rename d4/d19/f61 to d4/d27/d31/d71/d14/de4/fe6 0 2024-02-19T19:04:59.597 INFO:tasks.workunit.client.1.smithi132.stdout:5/810: mkdir d8/d2e/dc4/dc8/ddc/d111 0 2024-02-19T19:04:59.598 INFO:tasks.workunit.client.1.smithi132.stdout:8/760: dread - d4/dac/f108 zero size 2024-02-19T19:04:59.598 INFO:tasks.workunit.client.1.smithi132.stdout:0/652: creat d2/d3e/fdc x:0 0 0 2024-02-19T19:04:59.601 INFO:tasks.workunit.client.1.smithi132.stdout:5/811: unlink d8/d1a/c46 0 2024-02-19T19:04:59.604 INFO:tasks.workunit.client.1.smithi132.stdout:7/751: dwrite d1b/d1e/d2b/d62/d53/f65 [0,4194304] 0 2024-02-19T19:04:59.604 INFO:tasks.workunit.client.1.smithi132.stdout:0/653: symlink d2/dc/df/dbd/ldd 0 2024-02-19T19:04:59.615 INFO:tasks.workunit.client.1.smithi132.stdout:3/687: dwrite d8/d13/d2b/d69/d92/d34/fa3 [0,4194304] 0 2024-02-19T19:04:59.617 INFO:tasks.workunit.client.1.smithi132.stdout:7/752: mknod d1b/d5a/d63/ceb 0 2024-02-19T19:04:59.618 INFO:tasks.workunit.client.1.smithi132.stdout:0/654: read - d2/dc/d4a/d25/d43/d5f/fb7 zero size 2024-02-19T19:04:59.619 INFO:tasks.workunit.client.1.smithi132.stdout:9/820: getdents d12/d8d/d3a/ddc 0 2024-02-19T19:04:59.620 INFO:tasks.workunit.client.1.smithi132.stdout:2/732: dread d0/d4/d11/d2e/d5a/fb5 [0,4194304] 0 2024-02-19T19:04:59.622 INFO:tasks.workunit.client.1.smithi132.stdout:8/761: dwrite d4/dc/d1b/d38/d69/d74/fe1 [0,4194304] 0 2024-02-19T19:04:59.629 INFO:tasks.workunit.client.1.smithi132.stdout:7/753: creat d1b/d1e/d41/da3/dbd/fec x:0 0 0 2024-02-19T19:04:59.632 INFO:tasks.workunit.client.1.smithi132.stdout:5/812: dwrite d8/d1a/f9d [0,4194304] 0 2024-02-19T19:04:59.633 INFO:tasks.workunit.client.1.smithi132.stdout:3/688: dread d8/d13/d2b/d69/d92/daf/fc3 [0,4194304] 0 2024-02-19T19:04:59.634 INFO:tasks.workunit.client.1.smithi132.stdout:4/666: dread d4/d12/d28/f62 [0,4194304] 0 2024-02-19T19:04:59.635 INFO:tasks.workunit.client.1.smithi132.stdout:2/733: dwrite d0/d24/fda [0,4194304] 0 2024-02-19T19:04:59.635 INFO:tasks.workunit.client.1.smithi132.stdout:9/821: rename d12/d2a/d88/fc3 to d12/d8d/d3a/d85/d87/d114/f11c 0 2024-02-19T19:04:59.637 INFO:tasks.workunit.client.1.smithi132.stdout:5/813: write d8/d23/d29/d27/f25 [574493,74105] 0 2024-02-19T19:04:59.638 INFO:tasks.workunit.client.1.smithi132.stdout:8/762: symlink d4/dc/d1b/d20/d62/d91/d101/l10a 0 2024-02-19T19:04:59.641 INFO:tasks.workunit.client.1.smithi132.stdout:9/822: mkdir d12/db6/d11d 0 2024-02-19T19:04:59.650 INFO:tasks.workunit.client.1.smithi132.stdout:8/763: mkdir d4/d10b 0 2024-02-19T19:04:59.650 INFO:tasks.workunit.client.1.smithi132.stdout:2/734: write d0/d4/df/d40/d68/f8f [364234,107885] 0 2024-02-19T19:04:59.651 INFO:tasks.workunit.client.1.smithi132.stdout:7/754: dwrite d1b/d1e/d28/f3b [0,4194304] 0 2024-02-19T19:04:59.653 INFO:tasks.workunit.client.1.smithi132.stdout:5/814: creat d8/d2e/d71/dce/df0/f112 x:0 0 0 2024-02-19T19:04:59.656 INFO:tasks.workunit.client.1.smithi132.stdout:4/667: getdents d4/d19/daa 0 2024-02-19T19:04:59.656 INFO:tasks.workunit.client.1.smithi132.stdout:8/764: chown d4/dc/d1b/d20/f53 50262 1 2024-02-19T19:04:59.656 INFO:tasks.workunit.client.1.smithi132.stdout:3/689: dwrite d8/d13/d2b/d69/da6/fc7 [0,4194304] 0 2024-02-19T19:04:59.658 INFO:tasks.workunit.client.1.smithi132.stdout:3/690: readlink d8/d13/d2b/d60/l68 0 2024-02-19T19:04:59.658 INFO:tasks.workunit.client.1.smithi132.stdout:2/735: symlink d0/d19/lea 0 2024-02-19T19:04:59.660 INFO:tasks.workunit.client.1.smithi132.stdout:5/815: fsync f5 0 2024-02-19T19:04:59.663 INFO:tasks.workunit.client.1.smithi132.stdout:4/668: symlink d4/d12/d16/ddb/le7 0 2024-02-19T19:04:59.663 INFO:tasks.workunit.client.1.smithi132.stdout:2/736: chown d0/d4/d11/d2e/f6c 722889157 1 2024-02-19T19:04:59.664 INFO:tasks.workunit.client.1.smithi132.stdout:7/755: mkdir d1b/d1e/d28/da9/dd3/ddb/ded 0 2024-02-19T19:04:59.665 INFO:tasks.workunit.client.1.smithi132.stdout:1/612: sync 2024-02-19T19:04:59.665 INFO:tasks.workunit.client.1.smithi132.stdout:3/691: write d8/d13/d2b/d69/d92/daf/fb5 [1802071,122874] 0 2024-02-19T19:04:59.666 INFO:tasks.workunit.client.1.smithi132.stdout:8/765: rename d4/dc/l78 to d4/dc/d1b/d20/d62/d91/d101/l10c 0 2024-02-19T19:04:59.667 INFO:tasks.workunit.client.1.smithi132.stdout:4/669: read d4/d27/f9c [40131,60442] 0 2024-02-19T19:04:59.670 INFO:tasks.workunit.client.1.smithi132.stdout:9/823: link d12/d8d/fc0 d12/d8d/d3a/d40/d4e/d77/d89/dc4/d110/f11e 0 2024-02-19T19:04:59.672 INFO:tasks.workunit.client.1.smithi132.stdout:3/692: rmdir d8/d38/d44/d6e 39 2024-02-19T19:04:59.673 INFO:tasks.workunit.client.1.smithi132.stdout:5/816: dwrite d8/da/d5a/fcb [0,4194304] 0 2024-02-19T19:04:59.680 INFO:tasks.workunit.client.1.smithi132.stdout:1/613: creat d1/d1e/d77/d52/fc3 x:0 0 0 2024-02-19T19:04:59.680 INFO:tasks.workunit.client.1.smithi132.stdout:8/766: mknod d4/dc/d1b/d8f/c10d 0 2024-02-19T19:04:59.681 INFO:tasks.workunit.client.1.smithi132.stdout:4/670: mkdir d4/d12/de8 0 2024-02-19T19:04:59.681 INFO:tasks.workunit.client.1.smithi132.stdout:2/737: truncate d0/d4/d11/d2e/d5a/f5b 1972465 0 2024-02-19T19:04:59.681 INFO:tasks.workunit.client.1.smithi132.stdout:1/614: readlink d1/d3a/d50/l57 0 2024-02-19T19:04:59.682 INFO:tasks.workunit.client.1.smithi132.stdout:5/817: fdatasync d8/d2e/dc4/dc8/f67 0 2024-02-19T19:04:59.684 INFO:tasks.workunit.client.1.smithi132.stdout:3/693: getdents d8/dd1/dd2 0 2024-02-19T19:04:59.684 INFO:tasks.workunit.client.1.smithi132.stdout:9/824: dwrite d12/d8d/d3a/d85/d87/d114/d118/f93 [0,4194304] 0 2024-02-19T19:04:59.685 INFO:tasks.workunit.client.1.smithi132.stdout:2/738: write d0/d4/df/d40/d7b/fe2 [1009944,111467] 0 2024-02-19T19:04:59.686 INFO:tasks.workunit.client.1.smithi132.stdout:8/767: mkdir d4/dc/d10e 0 2024-02-19T19:04:59.686 INFO:tasks.workunit.client.1.smithi132.stdout:1/615: truncate d1/f18 8423070 0 2024-02-19T19:04:59.686 INFO:tasks.workunit.client.1.smithi132.stdout:0/655: dread d2/dc/df/d15/d34/f3c [0,4194304] 0 2024-02-19T19:04:59.692 INFO:tasks.workunit.client.1.smithi132.stdout:4/671: unlink d4/d27/d31/f68 0 2024-02-19T19:04:59.692 INFO:tasks.workunit.client.1.smithi132.stdout:5/818: mkdir d8/d2e/dc4/d113 0 2024-02-19T19:04:59.693 INFO:tasks.workunit.client.1.smithi132.stdout:9/825: creat d12/d8d/f11f x:0 0 0 2024-02-19T19:04:59.693 INFO:tasks.workunit.client.1.smithi132.stdout:2/739: readlink d0/d19/d1e/l30 0 2024-02-19T19:04:59.696 INFO:tasks.workunit.client.1.smithi132.stdout:2/740: readlink d0/d4/df/d7a/l9b 0 2024-02-19T19:04:59.697 INFO:tasks.workunit.client.1.smithi132.stdout:9/826: chown d12/d8d/d3a/d40/f51 11 1 2024-02-19T19:04:59.701 INFO:tasks.workunit.client.1.smithi132.stdout:9/827: readlink d12/d2a/d4c/dad/db2/lf6 0 2024-02-19T19:04:59.702 INFO:tasks.workunit.client.1.smithi132.stdout:3/694: dwrite d8/d13/d2b/d69/da6/fc7 [0,4194304] 0 2024-02-19T19:04:59.703 INFO:tasks.workunit.client.1.smithi132.stdout:9/828: stat d12/d2a/d4c/d61/fc5 0 2024-02-19T19:04:59.706 INFO:tasks.workunit.client.1.smithi132.stdout:1/616: dwrite d1/d1e/d77/d2c/d62/fb4 [0,4194304] 0 2024-02-19T19:04:59.708 INFO:tasks.workunit.client.1.smithi132.stdout:4/672: mknod d4/d27/d31/d71/d14/d24/d49/ce9 0 2024-02-19T19:04:59.711 INFO:tasks.workunit.client.1.smithi132.stdout:3/695: write d8/d13/d2f/f58 [170321,83820] 0 2024-02-19T19:04:59.713 INFO:tasks.workunit.client.1.smithi132.stdout:1/617: mkdir d1/d61/dc4 0 2024-02-19T19:04:59.714 INFO:tasks.workunit.client.1.smithi132.stdout:2/741: getdents d0/d24/d58/d53/dab/d84/de1 0 2024-02-19T19:04:59.715 INFO:tasks.workunit.client.1.smithi132.stdout:4/673: creat d4/d12/d16/ddb/fea x:0 0 0 2024-02-19T19:04:59.718 INFO:tasks.workunit.client.1.smithi132.stdout:1/618: truncate d1/d1e/d77/d52/d64/fb0 609193 0 2024-02-19T19:04:59.718 INFO:tasks.workunit.client.1.smithi132.stdout:2/742: dread d0/d24/fb3 [0,4194304] 0 2024-02-19T19:04:59.718 INFO:tasks.workunit.client.1.smithi132.stdout:3/696: rmdir d8/d13/d2b/d69/d92/d34/d42/d6c 39 2024-02-19T19:04:59.721 INFO:tasks.workunit.client.1.smithi132.stdout:8/768: getdents d4/dc/d1b/d20/d23 0 2024-02-19T19:04:59.724 INFO:tasks.workunit.client.1.smithi132.stdout:9/829: getdents d12/d8d/d3a/d40/d4e/d77/d89 0 2024-02-19T19:04:59.726 INFO:tasks.workunit.client.1.smithi132.stdout:2/743: rename d0/d4/df/d40/d7b/d9e to d0/d24/d58/d53/dab/deb 0 2024-02-19T19:04:59.727 INFO:tasks.workunit.client.1.smithi132.stdout:4/674: rename d4/d27/d31/d71/d14/c43 to d4/d27/d31/d71/d14/ceb 0 2024-02-19T19:04:59.728 INFO:tasks.workunit.client.1.smithi132.stdout:3/697: mkdir d8/d13/d2b/d69/d7e/dda 0 2024-02-19T19:04:59.731 INFO:tasks.workunit.client.1.smithi132.stdout:8/769: dwrite d4/dc/d1b/d20/d62/f9e [0,4194304] 0 2024-02-19T19:04:59.732 INFO:tasks.workunit.client.1.smithi132.stdout:9/830: rename d12/d8d/d3a/d85/d87/d114/d118/f71 to d12/d2a/d88/f120 0 2024-02-19T19:04:59.734 INFO:tasks.workunit.client.1.smithi132.stdout:0/656: dread d2/dc/d4a/d25/d43/f6f [0,4194304] 0 2024-02-19T19:04:59.734 INFO:tasks.workunit.client.1.smithi132.stdout:5/819: dread d8/d2e/f55 [0,4194304] 0 2024-02-19T19:04:59.736 INFO:tasks.workunit.client.1.smithi132.stdout:3/698: mknod d8/d13/d57/cdb 0 2024-02-19T19:04:59.736 INFO:tasks.workunit.client.1.smithi132.stdout:2/744: dwrite d0/d4/df/f51 [0,4194304] 0 2024-02-19T19:04:59.739 INFO:tasks.workunit.client.1.smithi132.stdout:8/770: unlink d4/dc/d42/fcb 0 2024-02-19T19:04:59.741 INFO:tasks.workunit.client.1.smithi132.stdout:9/831: creat d12/d8d/d3a/d85/d87/dc8/f121 x:0 0 0 2024-02-19T19:04:59.743 INFO:tasks.workunit.client.1.smithi132.stdout:8/771: read d4/dc/d42/f6f [3371423,121792] 0 2024-02-19T19:04:59.744 INFO:tasks.workunit.client.1.smithi132.stdout:0/657: rename d2/dc/d4a/d25/c33 to d2/d3e/db1/cde 0 2024-02-19T19:04:59.746 INFO:tasks.workunit.client.1.smithi132.stdout:2/745: symlink d0/d24/d58/d53/dab/lec 0 2024-02-19T19:04:59.747 INFO:tasks.workunit.client.1.smithi132.stdout:6/588: sync 2024-02-19T19:04:59.748 INFO:tasks.workunit.client.1.smithi132.stdout:5/820: dwrite d8/d2e/d71/da6/ddb/dea/f102 [0,4194304] 0 2024-02-19T19:04:59.749 INFO:tasks.workunit.client.1.smithi132.stdout:3/699: dwrite d8/d13/d19/f2d [4194304,4194304] 0 2024-02-19T19:04:59.750 INFO:tasks.workunit.client.1.smithi132.stdout:5/821: fsync d8/fe 0 2024-02-19T19:04:59.752 INFO:tasks.workunit.client.1.smithi132.stdout:8/772: creat d4/dc/d1b/d20/d23/d68/d7a/f10f x:0 0 0 2024-02-19T19:04:59.754 INFO:tasks.workunit.client.1.smithi132.stdout:9/832: getdents d12/d8d/d3a/d85/d87/d114/d118/d6e 0 2024-02-19T19:04:59.755 INFO:tasks.workunit.client.1.smithi132.stdout:8/773: write d4/fb [910152,123860] 0 2024-02-19T19:04:59.757 INFO:tasks.workunit.client.1.smithi132.stdout:6/589: unlink d0/c1c 0 2024-02-19T19:04:59.758 INFO:tasks.workunit.client.1.smithi132.stdout:3/700: creat d8/d13/d2b/d69/da6/fdc x:0 0 0 2024-02-19T19:04:59.758 INFO:tasks.workunit.client.1.smithi132.stdout:0/658: dread d2/dc/df/d15/f46 [0,4194304] 0 2024-02-19T19:04:59.759 INFO:tasks.workunit.client.1.smithi132.stdout:5/822: chown d8/f9b 15 1 2024-02-19T19:04:59.761 INFO:tasks.workunit.client.1.smithi132.stdout:6/590: dread - d0/d8b/f71 zero size 2024-02-19T19:04:59.762 INFO:tasks.workunit.client.1.smithi132.stdout:2/746: dwrite d0/d4/d11/f32 [0,4194304] 0 2024-02-19T19:04:59.764 INFO:tasks.workunit.client.1.smithi132.stdout:9/833: unlink d12/d8d/d3a/d85/d87/d114/d118/c80 0 2024-02-19T19:04:59.844 INFO:tasks.workunit.client.1.smithi132.stdout:6/591: creat d0/db/d25/d3a/d4b/fc4 x:0 0 0 2024-02-19T19:04:59.844 INFO:tasks.workunit.client.1.smithi132.stdout:5/823: dread d8/da/dae/fb5 [0,4194304] 0 2024-02-19T19:04:59.845 INFO:tasks.workunit.client.1.smithi132.stdout:8/774: getdents d4/dc/d1b/d20/d23/d68/d7a/de8 0 2024-02-19T19:04:59.845 INFO:tasks.workunit.client.1.smithi132.stdout:3/701: mkdir d8/d13/d2b/d69/d7e/ddd 0 2024-02-19T19:04:59.849 INFO:tasks.workunit.client.1.smithi132.stdout:8/775: chown d4/d2d/dcc/l104 117 1 2024-02-19T19:04:59.849 INFO:tasks.workunit.client.1.smithi132.stdout:3/702: readlink d8/d13/d2b/d69/d92/l9c 0 2024-02-19T19:04:59.850 INFO:tasks.workunit.client.1.smithi132.stdout:2/747: dwrite d0/d19/d1e/f62 [0,4194304] 0 2024-02-19T19:04:59.856 INFO:tasks.workunit.client.1.smithi132.stdout:0/659: dwrite d2/dc/df/d15/d2f/d69/f88 [0,4194304] 0 2024-02-19T19:04:59.856 INFO:tasks.workunit.client.1.smithi132.stdout:9/834: dwrite d12/d8d/d3a/d85/d87/fb5 [0,4194304] 0 2024-02-19T19:04:59.856 INFO:tasks.workunit.client.1.smithi132.stdout:5/824: dread d8/d23/fac [0,4194304] 0 2024-02-19T19:04:59.858 INFO:tasks.workunit.client.1.smithi132.stdout:0/660: readlink d2/dc/df/d15/d2f/d3f/l8e 0 2024-02-19T19:04:59.865 INFO:tasks.workunit.client.0.smithi088.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-19T19:04:59.868 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-19T19:04:59.869 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-19T19:04:59.870 INFO:tasks.workunit.client.1.smithi132.stdout:8/776: rename d4/dc/d1b/d38/fb8 to d4/dac/f110 0 2024-02-19T19:04:59.877 INFO:tasks.workunit.client.1.smithi132.stdout:6/592: link d0/db/d25/l52 d0/db/d25/d3a/d5d/d8f/d46/d5a/lc5 0 2024-02-19T19:04:59.883 INFO:tasks.workunit.client.1.smithi132.stdout:8/777: creat d4/dc/d1b/d20/d62/d91/d101/f111 x:0 0 0 2024-02-19T19:04:59.885 INFO:tasks.workunit.client.1.smithi132.stdout:7/756: sync 2024-02-19T19:04:59.886 INFO:tasks.workunit.client.1.smithi132.stdout:9/835: creat d12/d8d/d3a/d40/d4e/ded/f122 x:0 0 0 2024-02-19T19:04:59.890 INFO:tasks.workunit.client.1.smithi132.stdout:5/825: link d8/d2e/d71/ldd d8/d1a/l114 0 2024-02-19T19:04:59.891 INFO:tasks.workunit.client.1.smithi132.stdout:0/661: dwrite d2/f19 [0,4194304] 0 2024-02-19T19:04:59.891 INFO:tasks.workunit.client.1.smithi132.stdout:9/836: creat d12/d2a/d88/db8/f123 x:0 0 0 2024-02-19T19:04:59.893 INFO:tasks.workunit.client.1.smithi132.stdout:2/748: truncate d0/d4/d11/d2e/d5a/fb5 964970 0 2024-02-19T19:04:59.893 INFO:tasks.workunit.client.1.smithi132.stdout:9/837: stat d12/d8d/f10b 0 2024-02-19T19:04:59.899 INFO:tasks.workunit.client.1.smithi132.stdout:2/749: chown d0/d4/db9/fbe 400 1 2024-02-19T19:04:59.899 INFO:tasks.workunit.client.1.smithi132.stdout:5/826: rename d8/l22 to d8/d2e/dc4/dc8/daa/l115 0 2024-02-19T19:04:59.900 INFO:tasks.workunit.client.1.smithi132.stdout:7/757: dwrite f8 [4194304,4194304] 0 2024-02-19T19:04:59.904 INFO:tasks.workunit.client.1.smithi132.stdout:3/703: dread d8/d13/d2b/fa9 [4194304,4194304] 0 2024-02-19T19:04:59.904 INFO:tasks.workunit.client.1.smithi132.stdout:9/838: fdatasync d12/d8d/d3a/d40/d4e/d77/d8a/d82/f90 0 2024-02-19T19:04:59.906 INFO:tasks.workunit.client.1.smithi132.stdout:3/704: readlink d8/d13/l80 0 2024-02-19T19:04:59.906 INFO:tasks.workunit.client.1.smithi132.stdout:9/839: fdatasync d12/d8d/f30 0 2024-02-19T19:04:59.907 INFO:tasks.workunit.client.1.smithi132.stdout:5/827: chown d8/d23/d29/lcd 41 1 2024-02-19T19:04:59.910 INFO:tasks.workunit.client.1.smithi132.stdout:3/705: chown d8/d13/d19 1843617 1 2024-02-19T19:04:59.912 INFO:tasks.workunit.client.1.smithi132.stdout:6/593: dwrite d0/f8a [0,4194304] 0 2024-02-19T19:04:59.915 INFO:tasks.workunit.client.1.smithi132.stdout:0/662: dwrite d2/dc/d4a/d25/dda/f75 [0,4194304] 0 2024-02-19T19:04:59.915 INFO:tasks.workunit.client.1.smithi132.stdout:1/619: sync 2024-02-19T19:04:59.920 INFO:tasks.workunit.client.1.smithi132.stdout:3/706: write d8/d13/d2b/d69/da6/fc7 [3266388,121878] 0 2024-02-19T19:04:59.923 INFO:tasks.workunit.client.1.smithi132.stdout:8/778: dread d4/dc/d1b/d54/d5e/f63 [0,4194304] 0 2024-02-19T19:04:59.924 INFO:tasks.workunit.client.1.smithi132.stdout:1/620: chown d1/d1e/d77/d4e/f82 30 1 2024-02-19T19:04:59.924 INFO:tasks.workunit.client.1.smithi132.stdout:6/594: creat d0/db/d25/d50/d86/dc3/fc6 x:0 0 0 2024-02-19T19:04:59.925 INFO:tasks.workunit.client.1.smithi132.stdout:5/828: getdents d8/d23 0 2024-02-19T19:04:59.928 INFO:tasks.workunit.client.1.smithi132.stdout:5/829: mknod d8/d23/d29/dba/c116 0 2024-02-19T19:04:59.929 INFO:tasks.workunit.client.1.smithi132.stdout:9/840: dread d12/d8d/d3a/d85/d87/d114/d118/f93 [0,4194304] 0 2024-02-19T19:04:59.931 INFO:tasks.workunit.client.1.smithi132.stdout:1/621: chown d1/c51 27 1 2024-02-19T19:04:59.932 INFO:tasks.workunit.client.1.smithi132.stdout:0/663: read d2/dc/d4a/d67/fbb [190479,86732] 0 2024-02-19T19:04:59.932 INFO:tasks.workunit.client.1.smithi132.stdout:5/830: mkdir d8/d2e/d71/dce/d117 0 2024-02-19T19:04:59.935 INFO:tasks.workunit.client.1.smithi132.stdout:3/707: creat d8/d13/d2b/d69/d92/d8d/db2/fde x:0 0 0 2024-02-19T19:04:59.939 INFO:tasks.workunit.client.1.smithi132.stdout:5/831: rename d8/d23/d29/dba/de9 to d8/d1a/d64/de2/d118 0 2024-02-19T19:04:59.939 INFO:tasks.workunit.client.1.smithi132.stdout:9/841: mkdir d12/d2a/d4c/d61/dfe/d124 0 2024-02-19T19:04:59.941 INFO:tasks.workunit.client.1.smithi132.stdout:3/708: read d8/d38/f4b [562559,128434] 0 2024-02-19T19:04:59.942 INFO:tasks.workunit.client.1.smithi132.stdout:7/758: dread d1b/d1e/d2b/fb4 [0,4194304] 0 2024-02-19T19:04:59.942 INFO:tasks.workunit.client.1.smithi132.stdout:6/595: getdents d0/db/d25/d3a/d3d/d69 0 2024-02-19T19:04:59.943 INFO:tasks.workunit.client.1.smithi132.stdout:0/664: rename d2/dc/df/c76 to d2/dc/df/d15/d34/d84/da7/cdf 0 2024-02-19T19:04:59.945 INFO:tasks.workunit.client.1.smithi132.stdout:5/832: creat d8/d2e/d71/dce/df0/f119 x:0 0 0 2024-02-19T19:04:59.946 INFO:tasks.workunit.client.1.smithi132.stdout:8/779: link d4/dc/d1b/d33/c3d d4/dc/d1b/d38/c112 0 2024-02-19T19:04:59.948 INFO:tasks.workunit.client.1.smithi132.stdout:5/833: chown d8/d2e/db3 40867 1 2024-02-19T19:04:59.949 INFO:tasks.workunit.client.1.smithi132.stdout:3/709: read d8/d13/d2b/d69/d92/daf/fb5 [2081940,54865] 0 2024-02-19T19:04:59.949 INFO:tasks.workunit.client.1.smithi132.stdout:2/750: dread d0/d3c/f92 [0,4194304] 0 2024-02-19T19:04:59.950 INFO:tasks.workunit.client.1.smithi132.stdout:0/665: chown d2/dc/df/d15/d2f/c94 952765137 1 2024-02-19T19:04:59.951 INFO:tasks.workunit.client.1.smithi132.stdout:1/622: write d1/f1d [4619506,82585] 0 2024-02-19T19:04:59.951 INFO:tasks.workunit.client.1.smithi132.stdout:9/842: dwrite d12/d8d/d3a/ff0 [0,4194304] 0 2024-02-19T19:04:59.953 INFO:tasks.workunit.client.1.smithi132.stdout:2/751: chown d0/d4/d11 84278932 1 2024-02-19T19:04:59.953 INFO:tasks.workunit.client.1.smithi132.stdout:8/780: chown d4/dc/d1b/d20/d62/d91/ld4 14233352 1 2024-02-19T19:04:59.957 INFO:tasks.workunit.client.1.smithi132.stdout:7/759: dwrite d1b/d1e/d41/fe8 [0,4194304] 0 2024-02-19T19:04:59.958 INFO:tasks.workunit.client.1.smithi132.stdout:9/843: chown d12/d8d/d3a/l3c 99 1 2024-02-19T19:04:59.959 INFO:tasks.workunit.client.1.smithi132.stdout:6/596: truncate d0/db/d25/d3a/d5d/d8f/d46/d5a/f7d 889233 0 2024-02-19T19:04:59.960 INFO:tasks.workunit.client.1.smithi132.stdout:5/834: mkdir d8/d2e/dc4/d113/d11a 0 2024-02-19T19:04:59.960 INFO:tasks.workunit.client.1.smithi132.stdout:0/666: chown d2/dc/df/d15/d2f/d47/c5b 389 1 2024-02-19T19:04:59.961 INFO:tasks.workunit.client.1.smithi132.stdout:1/623: write d1/f5 [8957184,15329] 0 2024-02-19T19:04:59.961 INFO:tasks.workunit.client.1.smithi132.stdout:8/781: creat d4/dc/d1b/d38/d69/d74/d93/f113 x:0 0 0 2024-02-19T19:04:59.961 INFO:tasks.workunit.client.1.smithi132.stdout:2/752: write d0/d24/d58/fbc [714538,47821] 0 2024-02-19T19:04:59.963 INFO:tasks.workunit.client.1.smithi132.stdout:0/667: readlink d2/l7 0 2024-02-19T19:04:59.964 INFO:tasks.workunit.client.1.smithi132.stdout:2/753: fdatasync d0/d4/df/d7a/f99 0 2024-02-19T19:04:59.965 INFO:tasks.workunit.client.1.smithi132.stdout:6/597: mkdir d0/db/dc7 0 2024-02-19T19:04:59.986 INFO:tasks.workunit.client.1.smithi132.stdout:2/754: chown d0/d19/d1e/f45 0 1 2024-02-19T19:04:59.986 INFO:tasks.workunit.client.1.smithi132.stdout:7/760: dread d1b/d46/d76/f9f [0,4194304] 0 2024-02-19T19:04:59.987 INFO:tasks.workunit.client.1.smithi132.stdout:1/624: mknod d1/d1e/d77/d2c/d43/db8/cc5 0 2024-02-19T19:04:59.989 INFO:tasks.workunit.client.1.smithi132.stdout:7/761: symlink d1b/d1e/lee 0 2024-02-19T19:04:59.990 INFO:tasks.workunit.client.1.smithi132.stdout:2/755: unlink d0/d24/d58/f98 0 2024-02-19T19:04:59.992 INFO:tasks.workunit.client.1.smithi132.stdout:8/782: dwrite d4/dc/d1b/d20/d23/f28 [0,4194304] 0 2024-02-19T19:04:59.993 INFO:tasks.workunit.client.1.smithi132.stdout:0/668: dread d2/dc/df/dbd/fd8 [0,4194304] 0 2024-02-19T19:04:59.998 INFO:tasks.workunit.client.1.smithi132.stdout:9/844: dwrite d12/d8d/d3a/d40/f5d [0,4194304] 0 2024-02-19T19:04:59.999 INFO:tasks.workunit.client.1.smithi132.stdout:7/762: rename d1b/d1e/d28/f50 to d1b/d46/d76/d94/dd1/fef 0 2024-02-19T19:05:00.000 INFO:tasks.workunit.client.1.smithi132.stdout:3/710: dread d8/d38/d44/f4c [0,4194304] 0 2024-02-19T19:05:00.005 INFO:tasks.workunit.client.1.smithi132.stdout:9/845: read d12/d8d/d3a/d85/d87/dc8/deb/ff2 [383731,58669] 0 2024-02-19T19:05:00.005 INFO:tasks.workunit.client.1.smithi132.stdout:2/756: dread d0/d19/d1e/f62 [0,4194304] 0 2024-02-19T19:05:00.006 INFO:tasks.workunit.client.1.smithi132.stdout:0/669: mkdir d2/de0 0 2024-02-19T19:05:00.010 INFO:tasks.workunit.client.1.smithi132.stdout:7/763: mknod d1b/d46/d76/d94/dd1/cf0 0 2024-02-19T19:05:00.011 INFO:tasks.workunit.client.1.smithi132.stdout:8/783: write d4/dc/d1b/d20/d23/f2b [5035487,90387] 0 2024-02-19T19:05:00.011 INFO:tasks.workunit.client.1.smithi132.stdout:1/625: truncate d1/d1e/d77/d78/f7f 202090 0 2024-02-19T19:05:00.013 INFO:tasks.workunit.client.1.smithi132.stdout:9/846: chown d12/d8d/d3a/d85/l10f 6219 1 2024-02-19T19:05:00.016 INFO:tasks.workunit.client.1.smithi132.stdout:3/711: mknod d8/d38/d44/cdf 0 2024-02-19T19:05:00.017 INFO:tasks.workunit.client.1.smithi132.stdout:9/847: read d12/d8d/f28 [3618941,73688] 0 2024-02-19T19:05:00.019 INFO:tasks.workunit.client.1.smithi132.stdout:0/670: link d2/d6e/d7a/fcd d2/dc/df/d15/d2f/d3f/d63/fe1 0 2024-02-19T19:05:00.024 INFO:tasks.workunit.client.1.smithi132.stdout:0/671: chown d2/dc/df/d15/f4e 3062705 1 2024-02-19T19:05:00.025 INFO:tasks.workunit.client.1.smithi132.stdout:3/712: dwrite d8/d38/f3d [0,4194304] 0 2024-02-19T19:05:00.027 INFO:tasks.workunit.client.1.smithi132.stdout:3/713: dread - d8/d38/fba zero size 2024-02-19T19:05:00.027 INFO:tasks.workunit.client.1.smithi132.stdout:8/784: truncate d4/d2d/fbf 345449 0 2024-02-19T19:05:00.028 INFO:tasks.workunit.client.1.smithi132.stdout:1/626: truncate d1/d1e/d77/f73 393672 0 2024-02-19T19:05:00.029 INFO:tasks.workunit.client.1.smithi132.stdout:0/672: mknod d2/dc/d4a/d25/d7f/ce2 0 2024-02-19T19:05:00.032 INFO:tasks.workunit.client.1.smithi132.stdout:1/627: rename d1/d1e/d77/ca6 to d1/d61/cc6 0 2024-02-19T19:05:00.033 INFO:tasks.workunit.client.1.smithi132.stdout:2/757: dread d0/f80 [0,4194304] 0 2024-02-19T19:05:00.034 INFO:tasks.workunit.client.1.smithi132.stdout:0/673: unlink d2/dc/d4a/d25/dda/f75 0 2024-02-19T19:05:00.035 INFO:tasks.workunit.client.1.smithi132.stdout:7/764: dread d1b/d46/d56/f75 [0,4194304] 0 2024-02-19T19:05:00.036 INFO:tasks.workunit.client.1.smithi132.stdout:1/628: fdatasync d1/f15 0 2024-02-19T19:05:00.037 INFO:tasks.workunit.client.1.smithi132.stdout:2/758: dread d0/d4/df/d40/d68/f86 [0,4194304] 0 2024-02-19T19:05:00.039 INFO:tasks.workunit.client.1.smithi132.stdout:8/785: creat d4/dc/d1b/d54/f114 x:0 0 0 2024-02-19T19:05:00.041 INFO:tasks.workunit.client.1.smithi132.stdout:0/674: creat d2/dc/df/d15/d2f/d69/fe3 x:0 0 0 2024-02-19T19:05:00.043 INFO:tasks.workunit.client.1.smithi132.stdout:0/675: chown d2/dc/df/l31 857 1 2024-02-19T19:05:00.043 INFO:tasks.workunit.client.1.smithi132.stdout:4/675: sync 2024-02-19T19:05:00.044 INFO:tasks.workunit.client.1.smithi132.stdout:7/765: creat d1b/d32/ff1 x:0 0 0 2024-02-19T19:05:00.044 INFO:tasks.workunit.client.1.smithi132.stdout:1/629: creat d1/d95/fc7 x:0 0 0 2024-02-19T19:05:00.046 INFO:tasks.workunit.client.1.smithi132.stdout:8/786: mknod d4/dc/d1b/d20/d23/d68/d79/c115 0 2024-02-19T19:05:00.046 INFO:tasks.workunit.client.1.smithi132.stdout:0/676: chown d2/f19 85650080 1 2024-02-19T19:05:00.046 INFO:tasks.workunit.client.1.smithi132.stdout:2/759: chown d0/d19/d1e/c29 5383257 1 2024-02-19T19:05:00.047 INFO:tasks.workunit.client.1.smithi132.stdout:4/676: stat d4/d12/d48/ca0 0 2024-02-19T19:05:00.048 INFO:tasks.workunit.client.1.smithi132.stdout:3/714: dwrite d8/d13/d2b/d69/fd3 [0,4194304] 0 2024-02-19T19:05:00.048 INFO:tasks.workunit.client.1.smithi132.stdout:7/766: write d1b/d5a/f68 [5045028,105693] 0 2024-02-19T19:05:00.049 INFO:tasks.workunit.client.1.smithi132.stdout:2/760: chown d0/d4/df/d7a 15955 1 2024-02-19T19:05:00.050 INFO:tasks.workunit.client.1.smithi132.stdout:4/677: fdatasync d4/d27/d7f/fd9 0 2024-02-19T19:05:00.050 INFO:tasks.workunit.client.1.smithi132.stdout:2/761: chown d0/d64 2985111 1 2024-02-19T19:05:00.050 INFO:tasks.workunit.client.1.smithi132.stdout:1/630: dread d1/d3a/d50/fbd [0,4194304] 0 2024-02-19T19:05:00.061 INFO:tasks.workunit.client.1.smithi132.stdout:8/787: creat d4/dc/d1b/d20/de5/f116 x:0 0 0 2024-02-19T19:05:00.061 INFO:tasks.workunit.client.1.smithi132.stdout:0/677: read d2/dc/df/d15/d34/f41 [1313954,4048] 0 2024-02-19T19:05:00.066 INFO:tasks.workunit.client.1.smithi132.stdout:2/762: mknod d0/d4/df/ced 0 2024-02-19T19:05:00.066 INFO:tasks.workunit.client.1.smithi132.stdout:8/788: mknod d4/dc/d1b/d54/c117 0 2024-02-19T19:05:00.068 INFO:tasks.workunit.client.1.smithi132.stdout:3/715: dwrite d8/fd [0,4194304] 0 2024-02-19T19:05:00.070 INFO:tasks.workunit.client.1.smithi132.stdout:0/678: creat d2/dc/d4a/fe4 x:0 0 0 2024-02-19T19:05:00.073 INFO:tasks.workunit.client.1.smithi132.stdout:7/767: truncate d1b/d1e/d2b/d62/d53/f65 1905331 0 2024-02-19T19:05:00.074 INFO:tasks.workunit.client.1.smithi132.stdout:4/678: link d4/d27/d31/d71/l11 d4/d19/d21/lec 0 2024-02-19T19:05:00.074 INFO:tasks.workunit.client.1.smithi132.stdout:0/679: dread - d2/dc/df/d15/d34/d84/da7/fb3 zero size 2024-02-19T19:05:00.074 INFO:tasks.workunit.client.1.smithi132.stdout:2/763: mkdir d0/d4/d11/dee 0 2024-02-19T19:05:00.075 INFO:tasks.workunit.client.1.smithi132.stdout:8/789: mkdir d4/dc/d1b/d54/d81/dbd/d118 0 2024-02-19T19:05:00.076 INFO:tasks.workunit.client.1.smithi132.stdout:3/716: creat d8/d13/d2b/d69/d92/db4/fe0 x:0 0 0 2024-02-19T19:05:00.077 INFO:tasks.workunit.client.1.smithi132.stdout:7/768: dread - d1b/d5a/d63/d6b/d86/fb6 zero size 2024-02-19T19:05:00.078 INFO:tasks.workunit.client.1.smithi132.stdout:1/631: dwrite d1/d61/f7c [0,4194304] 0 2024-02-19T19:05:00.078 INFO:tasks.workunit.client.1.smithi132.stdout:2/764: chown d0/d4/d11/f20 10633 1 2024-02-19T19:05:00.080 INFO:tasks.workunit.client.1.smithi132.stdout:4/679: creat d4/d27/d31/d71/fed x:0 0 0 2024-02-19T19:05:00.081 INFO:tasks.workunit.client.1.smithi132.stdout:8/790: mknod d4/dc/c119 0 2024-02-19T19:05:00.083 INFO:tasks.workunit.client.1.smithi132.stdout:1/632: mknod d1/d95/d97/cc8 0 2024-02-19T19:05:00.085 INFO:tasks.workunit.client.1.smithi132.stdout:2/765: creat d0/d4/df/dc2/dad/fef x:0 0 0 2024-02-19T19:05:00.088 INFO:tasks.workunit.client.1.smithi132.stdout:8/791: write d4/dac/f110 [662817,15846] 0 2024-02-19T19:05:00.091 INFO:tasks.workunit.client.1.smithi132.stdout:0/680: write d2/d6e/fac [205921,68556] 0 2024-02-19T19:05:00.095 INFO:tasks.workunit.client.1.smithi132.stdout:8/792: rename d4/dc/d1b/d8f/ce4 to d4/d2d/d82/c11a 0 2024-02-19T19:05:00.095 INFO:tasks.workunit.client.1.smithi132.stdout:2/766: dread d0/d4/df/dc2/dad/f94 [0,4194304] 0 2024-02-19T19:05:00.096 INFO:tasks.workunit.client.1.smithi132.stdout:4/680: creat d4/fee x:0 0 0 2024-02-19T19:05:00.096 INFO:tasks.workunit.client.1.smithi132.stdout:7/769: getdents d1b/d1e/d2b/d62 0 2024-02-19T19:05:00.097 INFO:tasks.workunit.client.1.smithi132.stdout:0/681: creat d2/dc/d4a/d25/d43/d5f/fe5 x:0 0 0 2024-02-19T19:05:00.098 INFO:tasks.workunit.client.1.smithi132.stdout:1/633: truncate d1/d1e/d77/d52/d64/f91 143 0 2024-02-19T19:05:00.100 INFO:tasks.workunit.client.1.smithi132.stdout:0/682: chown d2/d3e/fdc 3303 1 2024-02-19T19:05:00.101 INFO:tasks.workunit.client.1.smithi132.stdout:1/634: chown d1/f18 29190740 1 2024-02-19T19:05:00.102 INFO:tasks.workunit.client.1.smithi132.stdout:3/717: dwrite d8/d38/d44/f73 [0,4194304] 0 2024-02-19T19:05:00.103 INFO:tasks.workunit.client.1.smithi132.stdout:2/767: creat d0/d24/d58/d53/dab/d84/dc9/ff0 x:0 0 0 2024-02-19T19:05:00.106 INFO:tasks.workunit.client.1.smithi132.stdout:1/635: write d1/d1e/d77/d2c/d43/d81/da1/fc1 [311192,10387] 0 2024-02-19T19:05:00.106 INFO:tasks.workunit.client.1.smithi132.stdout:4/681: rename d4/d19/d21/d56/dac/lc8 to d4/d19/dd4/lef 0 2024-02-19T19:05:00.107 INFO:tasks.workunit.client.1.smithi132.stdout:7/770: truncate d1b/d1e/d28/f31 773339 0 2024-02-19T19:05:00.107 INFO:tasks.workunit.client.1.smithi132.stdout:4/682: dread - d4/d12/fc3 zero size 2024-02-19T19:05:00.108 INFO:tasks.workunit.client.1.smithi132.stdout:8/793: dread d4/dc/d1b/d33/f4d [0,4194304] 0 2024-02-19T19:05:00.109 INFO:tasks.workunit.client.1.smithi132.stdout:7/771: chown d1b/d1e/d41/f6a 28193932 1 2024-02-19T19:05:00.110 INFO:tasks.workunit.client.1.smithi132.stdout:2/768: unlink d0/d4/df/d7a/l9b 0 2024-02-19T19:05:00.112 INFO:tasks.workunit.client.1.smithi132.stdout:3/718: creat d8/d13/d2b/d60/fe1 x:0 0 0 2024-02-19T19:05:00.113 INFO:tasks.workunit.client.1.smithi132.stdout:2/769: chown d0/d19/f25 35670 1 2024-02-19T19:05:00.113 INFO:tasks.workunit.client.1.smithi132.stdout:0/683: getdents d2/dc/df/d15/d2f/d3f/dd3 0 2024-02-19T19:05:00.114 INFO:tasks.workunit.client.1.smithi132.stdout:8/794: readlink d4/dac/lbb 0 2024-02-19T19:05:00.118 INFO:tasks.workunit.client.1.smithi132.stdout:2/770: mkdir d0/d3c/df1 0 2024-02-19T19:05:00.119 INFO:tasks.workunit.client.1.smithi132.stdout:1/636: dread d1/d1e/d77/f3f [0,4194304] 0 2024-02-19T19:05:00.121 INFO:tasks.workunit.client.1.smithi132.stdout:7/772: dwrite d1b/d32/ff1 [0,4194304] 0 2024-02-19T19:05:00.127 INFO:tasks.workunit.client.1.smithi132.stdout:3/719: dwrite d8/d38/d44/f73 [0,4194304] 0 2024-02-19T19:05:00.128 INFO:tasks.workunit.client.1.smithi132.stdout:7/773: stat d1b/d1e/d28/f55 0 2024-02-19T19:05:00.128 INFO:tasks.workunit.client.1.smithi132.stdout:4/683: dread d4/d19/d21/f36 [0,4194304] 0 2024-02-19T19:05:00.132 INFO:tasks.workunit.client.1.smithi132.stdout:4/684: write d4/d27/d31/fe3 [129665,110984] 0 2024-02-19T19:05:00.132 INFO:tasks.workunit.client.1.smithi132.stdout:2/771: rename d0/d19/d6a/fc4 to d0/d24/d58/d53/dab/ff2 0 2024-02-19T19:05:00.133 INFO:tasks.workunit.client.1.smithi132.stdout:2/772: readlink d0/d4/df/l16 0 2024-02-19T19:05:00.134 INFO:tasks.workunit.client.1.smithi132.stdout:4/685: write d4/d12/f2c [701596,15142] 0 2024-02-19T19:05:00.136 INFO:tasks.workunit.client.1.smithi132.stdout:2/773: chown d0/d4/fa9 101 1 2024-02-19T19:05:00.139 INFO:tasks.workunit.client.1.smithi132.stdout:3/720: dread d8/d13/d2b/d69/fd3 [0,4194304] 0 2024-02-19T19:05:00.140 INFO:tasks.workunit.client.1.smithi132.stdout:3/721: fdatasync d8/d13/d2f/f9a 0 2024-02-19T19:05:00.141 INFO:tasks.workunit.client.1.smithi132.stdout:8/795: write d4/dc/d1b/d20/d23/d68/d7a/f95 [1123359,30162] 0 2024-02-19T19:05:00.141 INFO:tasks.workunit.client.1.smithi132.stdout:0/684: link d2/dc/df/d15/d2f/d69/lb2 d2/dc/df/d15/d2f/d3f/dd3/le6 0 2024-02-19T19:05:00.143 INFO:tasks.workunit.client.1.smithi132.stdout:3/722: write d8/d93/f6f [1317064,63642] 0 2024-02-19T19:05:00.146 INFO:tasks.workunit.client.1.smithi132.stdout:1/637: write d1/d3a/d50/f6f [5018390,63231] 0 2024-02-19T19:05:00.146 INFO:tasks.workunit.client.1.smithi132.stdout:4/686: write d4/d27/d31/d32/d51/d5c/f5d [4609699,38782] 0 2024-02-19T19:05:00.147 INFO:tasks.workunit.client.1.smithi132.stdout:8/796: mknod d4/d2d/c11b 0 2024-02-19T19:05:00.147 INFO:tasks.workunit.client.1.smithi132.stdout:1/638: chown d1/d3a/f40 100026244 1 2024-02-19T19:05:00.150 INFO:tasks.workunit.client.1.smithi132.stdout:0/685: truncate d2/d6e/d7a/fab 193701 0 2024-02-19T19:05:00.152 INFO:tasks.workunit.client.1.smithi132.stdout:4/687: creat d4/d27/d7f/dc0/ff0 x:0 0 0 2024-02-19T19:05:00.154 INFO:tasks.workunit.client.1.smithi132.stdout:7/774: getdents d1b/d5a/d63/d96 0 2024-02-19T19:05:00.154 INFO:tasks.workunit.client.1.smithi132.stdout:1/639: dwrite d1/d1e/d77/d2c/d62/f87 [0,4194304] 0 2024-02-19T19:05:00.158 INFO:tasks.workunit.client.1.smithi132.stdout:0/686: creat d2/dc/df/d15/d2f/fe7 x:0 0 0 2024-02-19T19:05:00.160 INFO:tasks.workunit.client.1.smithi132.stdout:1/640: read - d1/d1e/d77/d2c/d62/d6b/d85/f9f zero size 2024-02-19T19:05:00.162 INFO:tasks.workunit.client.1.smithi132.stdout:2/774: link d0/d4/df/d40/c44 d0/d24/d58/cf3 0 2024-02-19T19:05:00.166 INFO:tasks.workunit.client.1.smithi132.stdout:8/797: getdents d4/dc/d1b/d20/dbe 0 2024-02-19T19:05:00.168 INFO:tasks.workunit.client.1.smithi132.stdout:8/798: truncate d4/dc/d1b/d20/d8a/fff 38792 0 2024-02-19T19:05:00.169 INFO:tasks.workunit.client.1.smithi132.stdout:8/799: chown d4/dc/d1b/d38/d69/d74/d93 5 1 2024-02-19T19:05:00.170 INFO:tasks.workunit.client.1.smithi132.stdout:7/775: link d1b/d1e/d28/f4f d1b/d1e/d41/da3/de3/ff2 0 2024-02-19T19:05:00.173 INFO:tasks.workunit.client.1.smithi132.stdout:7/776: write d1b/d5a/d63/d6b/d86/fb6 [533403,77507] 0 2024-02-19T19:05:00.176 INFO:tasks.workunit.client.1.smithi132.stdout:2/775: write d0/d3c/d3f/f81 [1813093,58062] 0 2024-02-19T19:05:00.176 INFO:tasks.workunit.client.1.smithi132.stdout:1/641: truncate d1/d3a/f30 518468 0 2024-02-19T19:05:00.177 INFO:tasks.workunit.client.1.smithi132.stdout:7/777: mknod d1b/d1e/d2b/dd0/cf3 0 2024-02-19T19:05:00.179 INFO:tasks.workunit.client.1.smithi132.stdout:7/778: fdatasync d1b/d1e/f22 0 2024-02-19T19:05:00.179 INFO:tasks.workunit.client.1.smithi132.stdout:3/723: dread d8/d93/f6f [0,4194304] 0 2024-02-19T19:05:00.179 INFO:tasks.workunit.client.1.smithi132.stdout:1/642: creat d1/d1e/d77/d52/d64/fc9 x:0 0 0 2024-02-19T19:05:00.182 INFO:tasks.workunit.client.1.smithi132.stdout:2/776: getdents d0/d24/dd0 0 2024-02-19T19:05:00.184 INFO:tasks.workunit.client.1.smithi132.stdout:1/643: mkdir d1/d1e/d77/d52/dca 0 2024-02-19T19:05:00.185 INFO:tasks.workunit.client.1.smithi132.stdout:0/687: dread d2/dc/df/d27/d3a/fb4 [0,4194304] 0 2024-02-19T19:05:00.187 INFO:tasks.workunit.client.1.smithi132.stdout:0/688: stat d2/dc/df/d27/d3a/d52 0 2024-02-19T19:05:00.188 INFO:tasks.workunit.client.1.smithi132.stdout:7/779: link d1b/d1e/d2b/d8f/da5/le0 d1b/d5a/d63/d6b/d79/dc6/lf4 0 2024-02-19T19:05:00.190 INFO:tasks.workunit.client.1.smithi132.stdout:0/689: unlink d2/dc/df/d27/d3a/d52/f9d 0 2024-02-19T19:05:00.191 INFO:tasks.workunit.client.1.smithi132.stdout:2/777: dwrite d0/d24/d58/f49 [0,4194304] 0 2024-02-19T19:05:00.192 INFO:tasks.workunit.client.1.smithi132.stdout:7/780: creat d1b/d1e/d2b/d8f/dae/ff5 x:0 0 0 2024-02-19T19:05:00.194 INFO:tasks.workunit.client.1.smithi132.stdout:1/644: getdents d1/d3a 0 2024-02-19T19:05:00.195 INFO:tasks.workunit.client.1.smithi132.stdout:0/690: mkdir d2/dc/df/d27/de8 0 2024-02-19T19:05:00.198 INFO:tasks.workunit.client.1.smithi132.stdout:1/645: symlink d1/d1e/d77/d2c/d43/lcb 0 2024-02-19T19:05:00.198 INFO:tasks.workunit.client.1.smithi132.stdout:3/724: dread d8/d13/d2f/f4d [0,4194304] 0 2024-02-19T19:05:00.200 INFO:tasks.workunit.client.1.smithi132.stdout:0/691: symlink d2/dc/df/d27/d3a/d52/le9 0 2024-02-19T19:05:00.205 INFO:tasks.workunit.client.1.smithi132.stdout:0/692: mknod d2/dc/df/d15/d2f/cea 0 2024-02-19T19:05:00.209 INFO:tasks.workunit.client.1.smithi132.stdout:3/725: rename d8/d13/d2b/d69/d7e/l6b to d8/d13/d2b/d69/d92/d34/d42/d6c/le2 0 2024-02-19T19:05:00.209 INFO:tasks.workunit.client.1.smithi132.stdout:1/646: dread d1/d1e/d77/d2c/d89/f9c [0,4194304] 0 2024-02-19T19:05:00.211 INFO:tasks.workunit.client.1.smithi132.stdout:7/781: dwrite d1b/d1e/d2b/d62/f67 [0,4194304] 0 2024-02-19T19:05:00.213 INFO:tasks.workunit.client.1.smithi132.stdout:0/693: creat d2/d6e/dc8/dd7/feb x:0 0 0 2024-02-19T19:05:00.213 INFO:tasks.workunit.client.1.smithi132.stdout:1/647: dread - d1/f8c zero size 2024-02-19T19:05:00.215 INFO:tasks.workunit.client.1.smithi132.stdout:7/782: dread - d1b/d1e/fd6 zero size 2024-02-19T19:05:00.216 INFO:tasks.workunit.client.1.smithi132.stdout:3/726: mknod d8/d13/d2b/d69/d92/d8d/dd7/da2/ce3 0 2024-02-19T19:05:00.217 INFO:tasks.workunit.client.1.smithi132.stdout:1/648: dread d1/d1e/d77/d2c/d89/f9c [0,4194304] 0 2024-02-19T19:05:00.218 INFO:tasks.workunit.client.1.smithi132.stdout:2/778: dread d0/d4/d11/f65 [0,4194304] 0 2024-02-19T19:05:00.218 INFO:tasks.workunit.client.1.smithi132.stdout:0/694: creat d2/d3e/db1/fec x:0 0 0 2024-02-19T19:05:00.222 INFO:tasks.workunit.client.1.smithi132.stdout:3/727: mkdir d8/d13/d2b/d60/dc2/de4 0 2024-02-19T19:05:00.223 INFO:tasks.workunit.client.1.smithi132.stdout:3/728: read d8/d93/f6f [2784110,54599] 0 2024-02-19T19:05:00.224 INFO:tasks.workunit.client.1.smithi132.stdout:7/783: creat d1b/d1e/d28/da9/dd3/ddb/ded/ff6 x:0 0 0 2024-02-19T19:05:00.226 INFO:tasks.workunit.client.1.smithi132.stdout:2/779: rename d0/d4/df/d40/d68/cb2 to d0/d3c/df1/cf4 0 2024-02-19T19:05:00.227 INFO:tasks.workunit.client.1.smithi132.stdout:1/649: mknod d1/d1e/d77/d2c/ccc 0 2024-02-19T19:05:00.227 INFO:tasks.workunit.client.1.smithi132.stdout:0/695: dwrite d2/d6e/fac [0,4194304] 0 2024-02-19T19:05:00.229 INFO:tasks.workunit.client.1.smithi132.stdout:7/784: getdents d1b/d1e/d2b/d62 0 2024-02-19T19:05:00.231 INFO:tasks.workunit.client.1.smithi132.stdout:1/650: mkdir d1/d1e/dcd 0 2024-02-19T19:05:00.232 INFO:tasks.workunit.client.1.smithi132.stdout:1/651: dread - d1/d95/fc7 zero size 2024-02-19T19:05:00.234 INFO:tasks.workunit.client.1.smithi132.stdout:2/780: rename d0/d4/d11/f1c to d0/d4/df/dc2/ff5 0 2024-02-19T19:05:00.234 INFO:tasks.workunit.client.1.smithi132.stdout:3/729: link d8/d13/d2b/la0 d8/d13/d2b/d69/daa/le5 0 2024-02-19T19:05:00.235 INFO:tasks.workunit.client.1.smithi132.stdout:3/730: fsync d8/d13/d57/fc8 0 2024-02-19T19:05:00.235 INFO:tasks.workunit.client.1.smithi132.stdout:7/785: fsync d1b/d1e/d28/f4f 0 2024-02-19T19:05:00.237 INFO:tasks.workunit.client.1.smithi132.stdout:2/781: read d0/d24/f87 [1095777,129869] 0 2024-02-19T19:05:00.238 INFO:tasks.workunit.client.1.smithi132.stdout:7/786: fsync d1b/d1e/d2b/f2f 0 2024-02-19T19:05:00.238 INFO:tasks.workunit.client.1.smithi132.stdout:7/787: stat d1b/d1e/d41/f6d 0 2024-02-19T19:05:00.239 INFO:tasks.workunit.client.1.smithi132.stdout:3/731: mkdir d8/d13/d2b/d69/d92/d34/d42/d6c/de6 0 2024-02-19T19:05:00.240 INFO:tasks.workunit.client.1.smithi132.stdout:3/732: readlink d8/d13/d2b/d69/da6/l8f 0 2024-02-19T19:05:00.242 INFO:tasks.workunit.client.1.smithi132.stdout:7/788: read d1b/d1e/f38 [103105,23327] 0 2024-02-19T19:05:00.242 INFO:tasks.workunit.client.1.smithi132.stdout:7/789: readlink d1b/d1e/l2d 0 2024-02-19T19:05:00.243 INFO:tasks.workunit.client.1.smithi132.stdout:3/733: mkdir d8/d13/d2b/d69/d7e/dd6/de7 0 2024-02-19T19:05:00.244 INFO:tasks.workunit.client.1.smithi132.stdout:3/734: write d8/d13/d2b/d69/da6/f5b [3687105,40400] 0 2024-02-19T19:05:00.246 INFO:tasks.workunit.client.1.smithi132.stdout:3/735: chown d8/d13/d2b/d60/dc2/de4 3403 1 2024-02-19T19:05:00.247 INFO:tasks.workunit.client.1.smithi132.stdout:3/736: write d8/d13/d2b/d69/da6/f5b [3630778,55726] 0 2024-02-19T19:05:00.248 INFO:tasks.workunit.client.1.smithi132.stdout:7/790: rename d1b/c61 to d1b/d46/cf7 0 2024-02-19T19:05:00.254 INFO:tasks.workunit.client.1.smithi132.stdout:3/737: getdents d8/d38/d44 0 2024-02-19T19:05:00.257 INFO:tasks.workunit.client.1.smithi132.stdout:1/652: dwrite d1/d1e/d77/f69 [0,4194304] 0 2024-02-19T19:05:00.261 INFO:tasks.workunit.client.1.smithi132.stdout:3/738: mknod d8/d13/d2b/d69/d92/d34/ce8 0 2024-02-19T19:05:00.262 INFO:tasks.workunit.client.1.smithi132.stdout:7/791: creat d1b/d1e/d2b/d8f/dae/ff8 x:0 0 0 2024-02-19T19:05:00.264 INFO:tasks.workunit.client.1.smithi132.stdout:3/739: chown d8/d13/d2b/d69/d7e/dd6/de7 17 1 2024-02-19T19:05:00.266 INFO:tasks.workunit.client.1.smithi132.stdout:1/653: dwrite d1/f18 [8388608,4194304] 0 2024-02-19T19:05:00.268 INFO:tasks.workunit.client.1.smithi132.stdout:3/740: readlink d8/d13/d2b/d69/d92/lbe 0 2024-02-19T19:05:00.269 INFO:tasks.workunit.client.1.smithi132.stdout:1/654: symlink d1/d1e/d77/d4e/d99/lce 0 2024-02-19T19:05:00.270 INFO:tasks.workunit.client.1.smithi132.stdout:3/741: read - d8/d13/d2b/d69/d92/daf/fbf zero size 2024-02-19T19:05:00.271 INFO:tasks.workunit.client.1.smithi132.stdout:1/655: write d1/d1e/d77/d2c/d89/f8d [1271410,20600] 0 2024-02-19T19:05:00.271 INFO:tasks.workunit.client.1.smithi132.stdout:3/742: symlink d8/dd1/le9 0 2024-02-19T19:05:00.272 INFO:tasks.workunit.client.1.smithi132.stdout:3/743: chown d8/d13/d57/fc8 899985 1 2024-02-19T19:05:00.329 INFO:tasks.workunit.client.1.smithi132.stdout:5/835: sync 2024-02-19T19:05:00.329 INFO:tasks.workunit.client.1.smithi132.stdout:9/848: sync 2024-02-19T19:05:00.329 INFO:tasks.workunit.client.1.smithi132.stdout:6/598: sync 2024-02-19T19:05:00.347 INFO:tasks.workunit.client.1.smithi132.stdout:7/792: dread f8 [0,4194304] 0 2024-02-19T19:05:00.347 INFO:tasks.workunit.client.1.smithi132.stdout:1/656: truncate d1/d1e/d77/d2c/d62/fb4 877659 0 2024-02-19T19:05:00.349 INFO:tasks.workunit.client.1.smithi132.stdout:9/849: write d12/d2a/d4c/fff [588217,111692] 0 2024-02-19T19:05:00.352 INFO:tasks.workunit.client.1.smithi132.stdout:3/744: dwrite d8/d13/d2b/f99 [0,4194304] 0 2024-02-19T19:05:00.355 INFO:tasks.workunit.client.1.smithi132.stdout:9/850: creat d12/d8d/d3a/ddc/f125 x:0 0 0 2024-02-19T19:05:00.355 INFO:tasks.workunit.client.1.smithi132.stdout:1/657: read d1/d1e/d77/d4e/f82 [349674,81227] 0 2024-02-19T19:05:00.355 INFO:tasks.workunit.client.1.smithi132.stdout:7/793: dread d1b/d5a/d63/d6b/fcf [0,4194304] 0 2024-02-19T19:05:00.357 INFO:tasks.workunit.client.1.smithi132.stdout:5/836: rename d8/d2e/db3 to d8/da/d5a/d7b/d96/d9e/ddf/d11b 0 2024-02-19T19:05:00.359 INFO:tasks.workunit.client.1.smithi132.stdout:5/837: chown d8/d1a/d35/l68 51 1 2024-02-19T19:05:00.360 INFO:tasks.workunit.client.1.smithi132.stdout:6/599: link d0/db/d25/d3a/d5d/f5e d0/db/d25/d3a/d5d/d8f/d87/fc8 0 2024-02-19T19:05:00.362 INFO:tasks.workunit.client.1.smithi132.stdout:6/600: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f18 [109302,127253] 0 2024-02-19T19:05:00.364 INFO:tasks.workunit.client.1.smithi132.stdout:7/794: rename d1b/l34 to d1b/d5a/d63/d6b/d79/lf9 0 2024-02-19T19:05:00.365 INFO:tasks.workunit.client.1.smithi132.stdout:1/658: getdents d1/d1e/d77 0 2024-02-19T19:05:00.367 INFO:tasks.workunit.client.1.smithi132.stdout:6/601: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f18 [1029592,15016] 0 2024-02-19T19:05:00.368 INFO:tasks.workunit.client.1.smithi132.stdout:5/838: rename d8/da/db/fbb to d8/da/d5a/d7b/d96/f11c 0 2024-02-19T19:05:00.368 INFO:tasks.workunit.client.1.smithi132.stdout:9/851: dwrite d12/d8d/d3a/d85/d87/d114/d118/f93 [4194304,4194304] 0 2024-02-19T19:05:00.373 INFO:tasks.workunit.client.1.smithi132.stdout:5/839: stat d8/da/d5a/d7b/d96/d9e/ddf/d11b/cfb 0 2024-02-19T19:05:00.376 INFO:tasks.workunit.client.1.smithi132.stdout:1/659: dwrite d1/d1e/d77/f3c [4194304,4194304] 0 2024-02-19T19:05:00.377 INFO:tasks.workunit.client.1.smithi132.stdout:5/840: read d8/d23/f6f [2516324,14587] 0 2024-02-19T19:05:00.377 INFO:tasks.workunit.client.1.smithi132.stdout:9/852: dread d12/f99 [0,4194304] 0 2024-02-19T19:05:00.377 INFO:tasks.workunit.client.1.smithi132.stdout:7/795: truncate d1b/d1e/d28/f3b 5110869 0 2024-02-19T19:05:00.378 INFO:tasks.workunit.client.1.smithi132.stdout:1/660: chown d1/d95/d97 36 1 2024-02-19T19:05:00.379 INFO:tasks.workunit.client.1.smithi132.stdout:7/796: dread - d1b/d1e/d41/db5/fde zero size 2024-02-19T19:05:00.380 INFO:tasks.workunit.client.1.smithi132.stdout:1/661: stat d1/d1e/d77/d2c/d62/c93 0 2024-02-19T19:05:00.381 INFO:tasks.workunit.client.1.smithi132.stdout:5/841: creat d8/d2e/dc4/dc8/da9/f11d x:0 0 0 2024-02-19T19:05:00.382 INFO:tasks.workunit.client.1.smithi132.stdout:5/842: dread - d8/d23/fb0 zero size 2024-02-19T19:05:00.383 INFO:tasks.workunit.client.1.smithi132.stdout:9/853: symlink d12/d8d/d3a/d40/d4e/d77/d8a/l126 0 2024-02-19T19:05:00.383 INFO:tasks.workunit.client.1.smithi132.stdout:5/843: write d8/d1a/f5e [2280651,35214] 0 2024-02-19T19:05:00.385 INFO:tasks.workunit.client.1.smithi132.stdout:6/602: getdents d0/db/d25/d3a/d3d/d69/d89 0 2024-02-19T19:05:00.385 INFO:tasks.workunit.client.1.smithi132.stdout:7/797: stat d1b/d5a/d63/d96/dcc/cd9 0 2024-02-19T19:05:00.388 INFO:tasks.workunit.client.1.smithi132.stdout:1/662: rename d1/f53 to d1/d1e/d77/d2c/d62/d6b/d85/fcf 0 2024-02-19T19:05:00.389 INFO:tasks.workunit.client.1.smithi132.stdout:9/854: symlink d12/d8d/d3a/d85/d87/d114/d118/d6e/l127 0 2024-02-19T19:05:00.391 INFO:tasks.workunit.client.1.smithi132.stdout:7/798: creat d1b/d5a/d63/d6b/d79/d84/ffa x:0 0 0 2024-02-19T19:05:00.394 INFO:tasks.workunit.client.1.smithi132.stdout:5/844: dwrite d8/d2e/d71/f82 [0,4194304] 0 2024-02-19T19:05:00.395 INFO:tasks.workunit.client.1.smithi132.stdout:9/855: write d12/d8d/d3a/d40/d4e/d77/d9d/fcd [142241,126060] 0 2024-02-19T19:05:00.395 INFO:tasks.workunit.client.1.smithi132.stdout:6/603: dwrite d0/dc/f14 [0,4194304] 0 2024-02-19T19:05:00.399 INFO:tasks.workunit.client.1.smithi132.stdout:5/845: creat d8/d2e/dc4/dc8/daa/f11e x:0 0 0 2024-02-19T19:05:00.402 INFO:tasks.workunit.client.1.smithi132.stdout:6/604: mkdir d0/db/dc9 0 2024-02-19T19:05:00.404 INFO:tasks.workunit.client.1.smithi132.stdout:9/856: dwrite d12/d8d/d3a/d85/d87/d114/d118/fd7 [0,4194304] 0 2024-02-19T19:05:00.404 INFO:tasks.workunit.client.1.smithi132.stdout:5/846: mkdir d8/d1a/d11f 0 2024-02-19T19:05:00.409 INFO:tasks.workunit.client.1.smithi132.stdout:5/847: mkdir d8/da/d5a/d7b/d96/d9e/ddf/d11b/d120 0 2024-02-19T19:05:00.411 INFO:tasks.workunit.client.1.smithi132.stdout:9/857: symlink d12/d2a/d4c/d61/dfe/d124/l128 0 2024-02-19T19:05:00.412 INFO:tasks.workunit.client.1.smithi132.stdout:6/605: dwrite d0/db/d25/d3a/d4b/f9a [0,4194304] 0 2024-02-19T19:05:00.442 INFO:tasks.workunit.client.1.smithi132.stdout:5/848: rename d8/d2e/dc4/dc8/daa/lfc to d8/l121 0 2024-02-19T19:05:00.445 INFO:tasks.workunit.client.1.smithi132.stdout:6/606: rename d0/db/d25/d3a/d3d/d69/f55 to d0/db/d25/d3a/d5d/d8f/d87/fca 0 2024-02-19T19:05:00.450 INFO:tasks.workunit.client.1.smithi132.stdout:6/607: dwrite d0/fb7 [0,4194304] 0 2024-02-19T19:05:00.451 INFO:tasks.workunit.client.1.smithi132.stdout:6/608: fsync d0/f73 0 2024-02-19T19:05:00.465 INFO:tasks.workunit.client.1.smithi132.stdout:9/858: truncate d12/d2a/d4c/dad/fde 1976779 0 2024-02-19T19:05:00.467 INFO:tasks.workunit.client.1.smithi132.stdout:6/609: creat d0/db/d25/fcb x:0 0 0 2024-02-19T19:05:00.468 INFO:tasks.workunit.client.1.smithi132.stdout:6/610: dread - d0/d8b/f6f zero size 2024-02-19T19:05:00.469 INFO:tasks.workunit.client.1.smithi132.stdout:9/859: rename d12/d8d/d3a/d40/d4e/f67 to d12/d8d/d3a/d40/f129 0 2024-02-19T19:05:00.472 INFO:tasks.workunit.client.1.smithi132.stdout:9/860: unlink d12/d8d/d3a/d40/f49 0 2024-02-19T19:05:00.473 INFO:tasks.workunit.client.1.smithi132.stdout:6/611: link d0/db/fa6 d0/d51/fcc 0 2024-02-19T19:05:00.476 INFO:tasks.workunit.client.1.smithi132.stdout:9/861: creat d12/d8d/d3a/d85/f12a x:0 0 0 2024-02-19T19:05:00.476 INFO:tasks.workunit.client.1.smithi132.stdout:6/612: write d0/d51/f92 [5859381,22194] 0 2024-02-19T19:05:00.482 INFO:tasks.workunit.client.1.smithi132.stdout:9/862: dwrite d12/d8d/d3a/d40/f44 [0,4194304] 0 2024-02-19T19:05:00.484 INFO:tasks.workunit.client.1.smithi132.stdout:9/863: truncate d12/d8d/f11f 182074 0 2024-02-19T19:05:00.485 INFO:tasks.workunit.client.1.smithi132.stdout:6/613: read - d0/db/d25/d3a/d5d/d8f/d46/fb9 zero size 2024-02-19T19:05:00.488 INFO:tasks.workunit.client.1.smithi132.stdout:9/864: getdents d12/db6 0 2024-02-19T19:05:00.489 INFO:tasks.workunit.client.1.smithi132.stdout:9/865: write d12/d2a/fd2 [817354,111909] 0 2024-02-19T19:05:00.490 INFO:tasks.workunit.client.1.smithi132.stdout:6/614: dread d0/db/d25/d3a/d5d/d8f/d46/d5a/f2e [0,4194304] 0 2024-02-19T19:05:00.491 INFO:tasks.workunit.client.1.smithi132.stdout:6/615: chown d0/db/d25/fcb 415 1 2024-02-19T19:05:00.494 INFO:tasks.workunit.client.1.smithi132.stdout:6/616: mknod d0/db/d25/d3a/d3d/ccd 0 2024-02-19T19:05:00.494 INFO:tasks.workunit.client.1.smithi132.stdout:9/866: link d12/d8d/d3a/d40/d4e/ld1 d12/d8d/d3a/d85/d87/d114/l12b 0 2024-02-19T19:05:00.495 INFO:tasks.workunit.client.1.smithi132.stdout:9/867: dread - d12/d2a/d4c/dad/db2/f10c zero size 2024-02-19T19:05:00.496 INFO:tasks.workunit.client.1.smithi132.stdout:9/868: fdatasync d12/d8d/d3a/d40/f66 0 2024-02-19T19:05:00.496 INFO:tasks.workunit.client.1.smithi132.stdout:6/617: creat d0/db/d25/d3a/d5d/fce x:0 0 0 2024-02-19T19:05:00.497 INFO:tasks.workunit.client.1.smithi132.stdout:9/869: chown d12/d8d/d3a/d40/d4e/d77/d8a/d82/f90 3 1 2024-02-19T19:05:00.499 INFO:tasks.workunit.client.1.smithi132.stdout:6/618: creat d0/db/d25/d50/d86/fcf x:0 0 0 2024-02-19T19:05:00.500 INFO:tasks.workunit.client.1.smithi132.stdout:9/870: rename d12/d8d/d3a/l3c to d12/d2a/d4c/dad/l12c 0 2024-02-19T19:05:00.501 INFO:tasks.workunit.client.1.smithi132.stdout:9/871: chown d12/d1b/f73 594955600 1 2024-02-19T19:05:00.502 INFO:tasks.workunit.client.1.smithi132.stdout:9/872: readlink d12/d8d/d3a/d40/d4e/d77/d8a/d82/dd9/lf4 0 2024-02-19T19:05:00.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:00 smithi132 ceph-mon[37969]: [19/Feb/2024:19:04:58] ENGINE Bus STARTING 2024-02-19T19:05:00.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:00 smithi132 ceph-mon[37969]: [19/Feb/2024:19:04:59] ENGINE Serving on https://172.21.15.88:7150 2024-02-19T19:05:00.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:00 smithi132 ceph-mon[37969]: mgrmap e36: smithi088.gyviqq(active, since 1.03356s), standbys: smithi132.psllsa 2024-02-19T19:05:00.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:00 smithi132 ceph-mon[37969]: pgmap v3: 65 pgs: 65 active+clean; 1.5 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail 2024-02-19T19:05:00.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:00 smithi132 ceph-mon[37969]: [19/Feb/2024:19:04:59] ENGINE Serving on http://172.21.15.88:8765 2024-02-19T19:05:00.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:00 smithi132 ceph-mon[37969]: [19/Feb/2024:19:04:59] ENGINE Bus STARTED 2024-02-19T19:05:00.503 INFO:tasks.workunit.client.1.smithi132.stdout:6/619: rename d0/db/d25/d50/lb0 to d0/db/d25/d3a/d5d/d8f/d46/d5a/ld0 0 2024-02-19T19:05:00.504 INFO:tasks.workunit.client.1.smithi132.stdout:6/620: mkdir d0/dd1 0 2024-02-19T19:05:00.505 INFO:tasks.workunit.client.1.smithi132.stdout:6/621: write d0/d51/f60 [3227656,38196] 0 2024-02-19T19:05:00.506 INFO:tasks.workunit.client.1.smithi132.stdout:6/622: chown d0/db/d25/d3a/d4b/d7c/fbd 8 1 2024-02-19T19:05:00.506 INFO:tasks.workunit.client.1.smithi132.stdout:9/873: rename d12/d8d/d3a/d85/l10f to d12/d8d/d3a/d40/d4e/d77/d8a/d82/l12d 0 2024-02-19T19:05:00.508 INFO:tasks.workunit.client.1.smithi132.stdout:6/623: symlink d0/db/d25/d3a/d4b/ld2 0 2024-02-19T19:05:00.509 INFO:tasks.workunit.client.1.smithi132.stdout:9/874: mknod d12/d2a/d4c/dad/c12e 0 2024-02-19T19:05:00.510 INFO:tasks.workunit.client.1.smithi132.stdout:6/624: chown d0/l7 118013459 1 2024-02-19T19:05:00.514 INFO:tasks.workunit.client.1.smithi132.stdout:9/875: getdents d12/d8d/d3a/d40/d4e/d77/d89/dc4 0 2024-02-19T19:05:00.517 INFO:tasks.workunit.client.1.smithi132.stdout:4/688: sync 2024-02-19T19:05:00.518 INFO:tasks.workunit.client.1.smithi132.stdout:9/876: rmdir d12/d8d/d3a 39 2024-02-19T19:05:00.519 INFO:tasks.workunit.client.1.smithi132.stdout:4/689: mknod d4/d19/d21/d56/cf1 0 2024-02-19T19:05:00.521 INFO:tasks.workunit.client.1.smithi132.stdout:4/690: stat d4/d27/d31/d71/d14/d24/d49 0 2024-02-19T19:05:00.522 INFO:tasks.workunit.client.1.smithi132.stdout:9/877: link d12/d1b/f50 d12/d2a/f12f 0 2024-02-19T19:05:00.523 INFO:tasks.workunit.client.1.smithi132.stdout:4/691: creat d4/d4d/d87/ff2 x:0 0 0 2024-02-19T19:05:00.524 INFO:tasks.workunit.client.1.smithi132.stdout:4/692: chown d4/d19/daa/dc7/d98 2864317 1 2024-02-19T19:05:00.527 INFO:tasks.workunit.client.1.smithi132.stdout:9/878: symlink d12/d8d/d3a/d40/d4e/d77/d8a/dca/l130 0 2024-02-19T19:05:00.527 INFO:tasks.workunit.client.1.smithi132.stdout:6/625: dread d0/db/d25/d3a/d5d/d8f/d22/f7e [0,4194304] 0 2024-02-19T19:05:00.528 INFO:tasks.workunit.client.1.smithi132.stdout:4/693: rename d4/d4d/f64 to d4/d4d/d87/ff3 0 2024-02-19T19:05:00.530 INFO:tasks.workunit.client.1.smithi132.stdout:4/694: mknod d4/d12/da4/cf4 0 2024-02-19T19:05:00.530 INFO:tasks.workunit.client.1.smithi132.stdout:4/695: readlink d4/l33 0 2024-02-19T19:05:00.531 INFO:tasks.workunit.client.1.smithi132.stdout:4/696: chown d4/l22 1901347991 1 2024-02-19T19:05:00.532 INFO:tasks.workunit.client.1.smithi132.stdout:6/626: link d0/db/d25/d3a/d3d/d69/f7f d0/db/d25/d3a/d3d/fd3 0 2024-02-19T19:05:00.532 INFO:tasks.workunit.client.1.smithi132.stdout:9/879: dwrite d12/d8d/d3a/d40/d4e/d77/d89/dc4/ff8 [0,4194304] 0 2024-02-19T19:05:00.532 INFO:tasks.workunit.client.1.smithi132.stdout:4/697: mkdir d4/d12/d9f/df5 0 2024-02-19T19:05:00.533 INFO:tasks.workunit.client.1.smithi132.stdout:6/627: readlink d0/db/d25/d3a/d5d/d8f/d22/l72 0 2024-02-19T19:05:00.535 INFO:tasks.workunit.client.1.smithi132.stdout:4/698: write d4/d27/d31/d32/d51/d5c/dbc/fcd [513991,3415] 0 2024-02-19T19:05:00.538 INFO:tasks.workunit.client.1.smithi132.stdout:4/699: write d4/d27/d31/fe3 [243760,63257] 0 2024-02-19T19:05:00.538 INFO:tasks.workunit.client.1.smithi132.stdout:6/628: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f1d [756567,64210] 0 2024-02-19T19:05:00.544 INFO:tasks.workunit.client.1.smithi132.stdout:4/700: dwrite d4/d12/d16/f5a [0,4194304] 0 2024-02-19T19:05:00.546 INFO:tasks.workunit.client.1.smithi132.stdout:4/701: fdatasync d4/d27/d7f/fa9 0 2024-02-19T19:05:00.548 INFO:tasks.workunit.client.1.smithi132.stdout:4/702: mkdir d4/d19/dd4/df6 0 2024-02-19T19:05:00.551 INFO:tasks.workunit.client.1.smithi132.stdout:4/703: truncate d4/d27/d7f/fd9 791838 0 2024-02-19T19:05:00.551 INFO:tasks.workunit.client.1.smithi132.stdout:9/880: dwrite d12/d8d/d3a/d40/d4e/d77/d89/fcf [0,4194304] 0 2024-02-19T19:05:00.553 INFO:tasks.workunit.client.1.smithi132.stdout:9/881: write d12/d8d/d3a/f52 [4742482,73220] 0 2024-02-19T19:05:00.554 INFO:tasks.workunit.client.1.smithi132.stdout:4/704: stat d4/d12/d9f/la2 0 2024-02-19T19:05:00.557 INFO:tasks.workunit.client.1.smithi132.stdout:4/705: mknod d4/d12/d48/d5f/cf7 0 2024-02-19T19:05:00.560 INFO:tasks.workunit.client.1.smithi132.stdout:9/882: link d12/d8d/d3a/d40/d4e/d77/d8a/d82/f90 d12/d8d/d3a/d40/d4e/d77/d89/f131 0 2024-02-19T19:05:00.561 INFO:tasks.workunit.client.1.smithi132.stdout:4/706: stat d4/d27/d31/d71/l11 0 2024-02-19T19:05:00.561 INFO:tasks.workunit.client.1.smithi132.stdout:4/707: chown d4/d19/dd4/df6 0 1 2024-02-19T19:05:00.562 INFO:tasks.workunit.client.1.smithi132.stdout:9/883: truncate d12/d8d/f10b 1916524 0 2024-02-19T19:05:00.563 INFO:tasks.workunit.client.1.smithi132.stdout:4/708: mknod d4/d27/d7f/cf8 0 2024-02-19T19:05:00.565 INFO:tasks.workunit.client.1.smithi132.stdout:4/709: read d4/d27/d31/d32/f50 [662990,13706] 0 2024-02-19T19:05:00.566 INFO:tasks.workunit.client.1.smithi132.stdout:9/884: read d12/d2a/d88/db8/fb9 [3844214,14159] 0 2024-02-19T19:05:00.567 INFO:tasks.workunit.client.1.smithi132.stdout:6/629: dread d0/db/d25/d38/f78 [4194304,4194304] 0 2024-02-19T19:05:00.567 INFO:tasks.workunit.client.1.smithi132.stdout:6/630: chown d0/d51/f92 347695560 1 2024-02-19T19:05:00.571 INFO:tasks.workunit.client.1.smithi132.stdout:6/631: mknod d0/db/d25/d3a/d5d/d8f/d46/d5a/cd4 0 2024-02-19T19:05:00.573 INFO:tasks.workunit.client.1.smithi132.stdout:4/710: dwrite d4/d27/d73/fa6 [0,4194304] 0 2024-02-19T19:05:00.574 INFO:tasks.workunit.client.1.smithi132.stdout:6/632: chown d0/db/d25/d3a/d5d/d8f/d46/d5a/lc5 190193 1 2024-02-19T19:05:00.581 INFO:tasks.workunit.client.1.smithi132.stdout:6/633: truncate d0/db/d25/d3a/d5d/d8f/d46/f7b 13037709 0 2024-02-19T19:05:00.583 INFO:tasks.workunit.client.1.smithi132.stdout:6/634: symlink d0/d51/da7/ld5 0 2024-02-19T19:05:00.584 INFO:tasks.workunit.client.1.smithi132.stdout:4/711: dwrite d4/d12/d16/f5a [4194304,4194304] 0 2024-02-19T19:05:00.585 INFO:tasks.workunit.client.1.smithi132.stdout:4/712: write d4/f7d [5365990,59076] 0 2024-02-19T19:05:00.588 INFO:tasks.workunit.client.1.smithi132.stdout:6/635: mknod d0/db/d25/d3a/d5d/d8f/d22/cd6 0 2024-02-19T19:05:00.590 INFO:tasks.workunit.client.1.smithi132.stdout:4/713: symlink d4/d27/d31/d71/d14/lf9 0 2024-02-19T19:05:00.592 INFO:tasks.workunit.client.1.smithi132.stdout:6/636: chown d0/db/d25/d3a/d5d/d8f/d22/f19 769 1 2024-02-19T19:05:00.593 INFO:tasks.workunit.client.1.smithi132.stdout:6/637: write d0/f10 [4162048,96645] 0 2024-02-19T19:05:00.594 INFO:tasks.workunit.client.1.smithi132.stdout:4/714: mkdir d4/d19/d21/dfa 0 2024-02-19T19:05:00.598 INFO:tasks.workunit.client.1.smithi132.stdout:4/715: rename d4/d19/daa/cb9 to d4/d12/d16/cfb 0 2024-02-19T19:05:00.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:00 smithi088 ceph-mon[88643]: [19/Feb/2024:19:04:58] ENGINE Bus STARTING 2024-02-19T19:05:00.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:00 smithi088 ceph-mon[88643]: [19/Feb/2024:19:04:59] ENGINE Serving on https://172.21.15.88:7150 2024-02-19T19:05:00.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:00 smithi088 ceph-mon[88643]: mgrmap e36: smithi088.gyviqq(active, since 1.03356s), standbys: smithi132.psllsa 2024-02-19T19:05:00.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:00 smithi088 ceph-mon[88643]: pgmap v3: 65 pgs: 65 active+clean; 1.5 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail 2024-02-19T19:05:00.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:00 smithi088 ceph-mon[88643]: [19/Feb/2024:19:04:59] ENGINE Serving on http://172.21.15.88:8765 2024-02-19T19:05:00.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:00 smithi088 ceph-mon[88643]: [19/Feb/2024:19:04:59] ENGINE Bus STARTED 2024-02-19T19:05:00.601 INFO:tasks.workunit.client.1.smithi132.stdout:6/638: dwrite d0/db/d25/d3a/d3d/d69/d89/fb6 [0,4194304] 0 2024-02-19T19:05:00.601 INFO:tasks.workunit.client.1.smithi132.stdout:9/885: dread d12/d1b/f95 [0,4194304] 0 2024-02-19T19:05:00.602 INFO:tasks.workunit.client.1.smithi132.stdout:4/716: symlink d4/d19/d21/d56/dac/lfc 0 2024-02-19T19:05:00.603 INFO:tasks.workunit.client.1.smithi132.stdout:6/639: dread - d0/db/d25/d3a/d3d/f74 zero size 2024-02-19T19:05:00.605 INFO:tasks.workunit.client.1.smithi132.stdout:9/886: fdatasync d12/d1b/f1c 0 2024-02-19T19:05:00.606 INFO:tasks.workunit.client.1.smithi132.stdout:6/640: symlink d0/db/d25/d50/d86/ld7 0 2024-02-19T19:05:00.606 INFO:tasks.workunit.client.1.smithi132.stdout:0/696: sync 2024-02-19T19:05:00.606 INFO:tasks.workunit.client.1.smithi132.stdout:8/800: sync 2024-02-19T19:05:00.609 INFO:tasks.workunit.client.1.smithi132.stdout:9/887: truncate d12/d8d/d3a/ddc/f125 777721 0 2024-02-19T19:05:00.611 INFO:tasks.workunit.client.1.smithi132.stdout:4/717: dwrite d4/d27/d31/d32/f50 [0,4194304] 0 2024-02-19T19:05:00.613 INFO:tasks.workunit.client.1.smithi132.stdout:8/801: rmdir d4/dc/d10e 0 2024-02-19T19:05:00.613 INFO:tasks.workunit.client.1.smithi132.stdout:6/641: creat d0/db/d25/d3a/d5d/d8f/d46/d5a/daf/fd8 x:0 0 0 2024-02-19T19:05:00.614 INFO:tasks.workunit.client.1.smithi132.stdout:4/718: rename d4/d12/d9f/la2 to d4/d27/d31/d71/d14/lfd 0 2024-02-19T19:05:00.616 INFO:tasks.workunit.client.1.smithi132.stdout:6/642: mkdir d0/db/d25/d3a/d3d/dd9 0 2024-02-19T19:05:00.619 INFO:tasks.workunit.client.1.smithi132.stdout:6/643: creat d0/db/d25/d3a/d5d/d8f/d87/fda x:0 0 0 2024-02-19T19:05:00.620 INFO:tasks.workunit.client.1.smithi132.stdout:6/644: write d0/db/d25/d3a/d5d/d8f/d22/f23 [3137798,35451] 0 2024-02-19T19:05:00.637 INFO:tasks.workunit.client.1.smithi132.stdout:8/802: write d4/dc/d1b/d20/f53 [921177,42829] 0 2024-02-19T19:05:00.638 INFO:tasks.workunit.client.1.smithi132.stdout:6/645: truncate d0/d51/f60 2402339 0 2024-02-19T19:05:00.639 INFO:tasks.workunit.client.1.smithi132.stdout:8/803: symlink d4/dc/d1b/d38/dfa/l11c 0 2024-02-19T19:05:00.640 INFO:tasks.workunit.client.1.smithi132.stdout:6/646: creat d0/db/d25/d3a/d3d/d69/d89/fdb x:0 0 0 2024-02-19T19:05:00.641 INFO:tasks.workunit.client.1.smithi132.stdout:6/647: chown d0/db/d25/d3a/d5d 985706042 1 2024-02-19T19:05:00.642 INFO:tasks.workunit.client.1.smithi132.stdout:8/804: fdatasync d4/dc/d1b/d54/d5e/f63 0 2024-02-19T19:05:00.642 INFO:tasks.workunit.client.1.smithi132.stdout:6/648: symlink d0/dc/ldc 0 2024-02-19T19:05:00.648 INFO:tasks.workunit.client.1.smithi132.stdout:8/805: dwrite d4/dc/d1b/d8f/daf/fcf [0,4194304] 0 2024-02-19T19:05:00.651 INFO:tasks.workunit.client.1.smithi132.stdout:8/806: mknod d4/d2d/c11d 0 2024-02-19T19:05:00.652 INFO:tasks.workunit.client.1.smithi132.stdout:6/649: truncate d0/fa 2200300 0 2024-02-19T19:05:00.652 INFO:tasks.workunit.client.1.smithi132.stdout:8/807: chown d4/dc/d1b/d38/dfa/l11c 916021 1 2024-02-19T19:05:00.655 INFO:tasks.workunit.client.1.smithi132.stdout:6/650: fsync d0/db/d25/d3a/d5d/d8f/d22/f59 0 2024-02-19T19:05:00.658 INFO:tasks.workunit.client.1.smithi132.stdout:6/651: creat d0/db/d25/d3a/d5d/d8f/fdd x:0 0 0 2024-02-19T19:05:00.666 INFO:tasks.workunit.client.1.smithi132.stdout:8/808: dread d4/dc/d1b/d38/d69/d74/d93/faa [0,4194304] 0 2024-02-19T19:05:00.669 INFO:tasks.workunit.client.1.smithi132.stdout:8/809: dread d4/dac/f110 [0,4194304] 0 2024-02-19T19:05:00.673 INFO:tasks.workunit.client.1.smithi132.stdout:8/810: link d4/la d4/dc/d1b/d38/d69/d74/d93/l11e 0 2024-02-19T19:05:00.676 INFO:tasks.workunit.client.1.smithi132.stdout:8/811: creat d4/dc/d1b/d20/d23/d68/d7a/de8/f11f x:0 0 0 2024-02-19T19:05:00.680 INFO:tasks.workunit.client.1.smithi132.stdout:6/652: dread d0/db/d25/d38/f68 [0,4194304] 0 2024-02-19T19:05:00.681 INFO:tasks.workunit.client.1.smithi132.stdout:8/812: dwrite d4/d2d/f46 [0,4194304] 0 2024-02-19T19:05:00.682 INFO:tasks.workunit.client.1.smithi132.stdout:6/653: truncate d0/db/d25/d3a/d5d/d8f/fdd 380600 0 2024-02-19T19:05:00.685 INFO:tasks.workunit.client.1.smithi132.stdout:6/654: mkdir d0/db/d25/d3a/d5d/d8f/dbe/dde 0 2024-02-19T19:05:00.689 INFO:tasks.workunit.client.1.smithi132.stdout:6/655: creat d0/db/d25/d50/fdf x:0 0 0 2024-02-19T19:05:00.696 INFO:tasks.workunit.client.1.smithi132.stdout:6/656: creat d0/db/d25/d3a/d5d/d8f/d87/fe0 x:0 0 0 2024-02-19T19:05:00.703 INFO:tasks.workunit.client.1.smithi132.stdout:6/657: mknod d0/db/d25/d3a/d3d/ce1 0 2024-02-19T19:05:00.707 INFO:tasks.workunit.client.1.smithi132.stdout:8/813: dread d4/f2e [0,4194304] 0 2024-02-19T19:05:00.708 INFO:tasks.workunit.client.1.smithi132.stdout:8/814: write d4/dc/d1b/d20/f59 [1902452,44439] 0 2024-02-19T19:05:00.711 INFO:tasks.workunit.client.1.smithi132.stdout:8/815: mknod d4/dc/d1b/d38/d69/d74/dce/c120 0 2024-02-19T19:05:00.712 INFO:tasks.workunit.client.1.smithi132.stdout:8/816: chown d4/dc/d1b/d54/d5e/f50 310737215 1 2024-02-19T19:05:00.713 INFO:tasks.workunit.client.1.smithi132.stdout:8/817: truncate d4/dc/d1b/d33/fb7 949438 0 2024-02-19T19:05:00.715 INFO:tasks.workunit.client.1.smithi132.stdout:8/818: mkdir d4/dc/d1b/d20/d62/d91/d121 0 2024-02-19T19:05:00.716 INFO:tasks.workunit.client.1.smithi132.stdout:8/819: truncate d4/dc/fa9 699870 0 2024-02-19T19:05:00.717 INFO:tasks.workunit.client.1.smithi132.stdout:6/658: dread d0/db/d25/d3a/d5d/d8f/d46/d5a/f20 [0,4194304] 0 2024-02-19T19:05:00.720 INFO:tasks.workunit.client.1.smithi132.stdout:8/820: rename d4/dc/d1b/d20/d62/d91/fb2 to d4/dc/d1b/d20/d23/d68/d7a/f122 0 2024-02-19T19:05:00.721 INFO:tasks.workunit.client.1.smithi132.stdout:8/821: creat d4/d2d/dcc/f123 x:0 0 0 2024-02-19T19:05:00.724 INFO:tasks.workunit.client.1.smithi132.stdout:8/822: getdents d4/dc/d1b/d20/d23/da3 0 2024-02-19T19:05:00.751 INFO:tasks.workunit.client.1.smithi132.stdout:3/745: sync 2024-02-19T19:05:00.751 INFO:tasks.workunit.client.1.smithi132.stdout:5/849: sync 2024-02-19T19:05:00.751 INFO:tasks.workunit.client.1.smithi132.stdout:2/782: sync 2024-02-19T19:05:00.751 INFO:tasks.workunit.client.1.smithi132.stdout:1/663: sync 2024-02-19T19:05:00.751 INFO:tasks.workunit.client.1.smithi132.stdout:7/799: sync 2024-02-19T19:05:00.752 INFO:tasks.workunit.client.1.smithi132.stdout:3/746: readlink d8/l35 0 2024-02-19T19:05:00.752 INFO:tasks.workunit.client.1.smithi132.stdout:5/850: chown d8/d2e/d71/da6 1 1 2024-02-19T19:05:00.753 INFO:tasks.workunit.client.1.smithi132.stdout:2/783: readlink d0/l1 0 2024-02-19T19:05:00.755 INFO:tasks.workunit.client.1.smithi132.stdout:1/664: unlink d1/f1b 0 2024-02-19T19:05:00.766 INFO:tasks.workunit.client.1.smithi132.stdout:5/851: dread d8/d23/f4d [0,4194304] 0 2024-02-19T19:05:00.767 INFO:tasks.workunit.client.1.smithi132.stdout:5/852: chown d8/da/db/l11 518 1 2024-02-19T19:05:00.828 INFO:tasks.workunit.client.1.smithi132.stdout:7/800: link d1b/d1e/d28/f4f d1b/d1e/d28/d5e/ffb 0 2024-02-19T19:05:00.830 INFO:tasks.workunit.client.1.smithi132.stdout:3/747: getdents d8/d13/d2b/d60 0 2024-02-19T19:05:00.831 INFO:tasks.workunit.client.1.smithi132.stdout:1/665: mkdir d1/d1e/d77/d2c/d62/dd0 0 2024-02-19T19:05:00.832 INFO:tasks.workunit.client.1.smithi132.stdout:2/784: getdents d0/d24/d58/d53/dab/d84 0 2024-02-19T19:05:00.833 INFO:tasks.workunit.client.1.smithi132.stdout:5/853: dwrite d8/da/d5a/d7b/d96/f11c [0,4194304] 0 2024-02-19T19:05:00.833 INFO:tasks.workunit.client.1.smithi132.stdout:7/801: mknod d1b/d1e/d2b/d8f/dae/cfc 0 2024-02-19T19:05:00.838 INFO:tasks.workunit.client.1.smithi132.stdout:7/802: creat d1b/d1e/d2b/d45/ffd x:0 0 0 2024-02-19T19:05:00.840 INFO:tasks.workunit.client.1.smithi132.stdout:5/854: symlink d8/d23/d29/d27/df8/d105/l122 0 2024-02-19T19:05:00.841 INFO:tasks.workunit.client.1.smithi132.stdout:3/748: dwrite d8/d13/d2b/d69/da6/fa5 [0,4194304] 0 2024-02-19T19:05:00.841 INFO:tasks.workunit.client.1.smithi132.stdout:7/803: chown d1b/d1e/d28/da9/cbc 2526 1 2024-02-19T19:05:00.843 INFO:tasks.workunit.client.1.smithi132.stdout:5/855: mknod d8/d2e/d71/dce/c123 0 2024-02-19T19:05:00.845 INFO:tasks.workunit.client.1.smithi132.stdout:3/749: creat d8/d13/d2b/d69/d92/d8d/db2/fea x:0 0 0 2024-02-19T19:05:00.854 INFO:tasks.workunit.client.1.smithi132.stdout:5/856: dread d8/d2e/f55 [0,4194304] 0 2024-02-19T19:05:00.856 INFO:tasks.workunit.client.1.smithi132.stdout:1/666: dwrite d1/d1e/d77/d2c/f34 [0,4194304] 0 2024-02-19T19:05:00.859 INFO:tasks.workunit.client.1.smithi132.stdout:1/667: chown d1/d1e/d77/f41 8412363 1 2024-02-19T19:05:00.859 INFO:tasks.workunit.client.1.smithi132.stdout:2/785: dread d0/d19/f82 [0,4194304] 0 2024-02-19T19:05:00.862 INFO:tasks.workunit.client.1.smithi132.stdout:1/668: read - d1/d1e/d77/d52/fc3 zero size 2024-02-19T19:05:00.863 INFO:tasks.workunit.client.1.smithi132.stdout:5/857: dwrite d8/d1a/f6c [0,4194304] 0 2024-02-19T19:05:00.864 INFO:tasks.workunit.client.1.smithi132.stdout:2/786: unlink d0/d4/d11/c2a 0 2024-02-19T19:05:00.866 INFO:tasks.workunit.client.1.smithi132.stdout:1/669: mkdir d1/d1e/d77/d2c/d62/d6b/dd1 0 2024-02-19T19:05:00.867 INFO:tasks.workunit.client.1.smithi132.stdout:5/858: unlink d8/d23/d29/d27/fe7 0 2024-02-19T19:05:00.869 INFO:tasks.workunit.client.1.smithi132.stdout:2/787: creat d0/d24/dd0/ff6 x:0 0 0 2024-02-19T19:05:00.870 INFO:tasks.workunit.client.1.smithi132.stdout:7/804: dread d1b/d1e/f43 [0,4194304] 0 2024-02-19T19:05:00.870 INFO:tasks.workunit.client.1.smithi132.stdout:1/670: write d1/d95/fc7 [830119,44572] 0 2024-02-19T19:05:00.873 INFO:tasks.workunit.client.1.smithi132.stdout:7/805: stat d1b/c1f 0 2024-02-19T19:05:00.874 INFO:tasks.workunit.client.1.smithi132.stdout:3/750: dread d8/fc0 [0,4194304] 0 2024-02-19T19:05:00.876 INFO:tasks.workunit.client.1.smithi132.stdout:5/859: getdents d8/da/d5a 0 2024-02-19T19:05:00.877 INFO:tasks.workunit.client.1.smithi132.stdout:1/671: dread d1/d1e/d77/d2c/d43/f54 [0,4194304] 0 2024-02-19T19:05:00.879 INFO:tasks.workunit.client.1.smithi132.stdout:3/751: mkdir d8/d13/deb 0 2024-02-19T19:05:00.880 INFO:tasks.workunit.client.1.smithi132.stdout:5/860: mknod d8/da/d5a/d7b/d96/d9e/ddf/c124 0 2024-02-19T19:05:00.881 INFO:tasks.workunit.client.1.smithi132.stdout:7/806: dread d1b/d1e/f43 [0,4194304] 0 2024-02-19T19:05:00.881 INFO:tasks.workunit.client.1.smithi132.stdout:1/672: mkdir d1/dd2 0 2024-02-19T19:05:00.883 INFO:tasks.workunit.client.1.smithi132.stdout:5/861: mkdir d8/d125 0 2024-02-19T19:05:00.885 INFO:tasks.workunit.client.1.smithi132.stdout:2/788: dwrite d0/d4/d11/d2e/f3e [0,4194304] 0 2024-02-19T19:05:00.886 INFO:tasks.workunit.client.1.smithi132.stdout:1/673: truncate d1/d1e/d77/f3f 695308 0 2024-02-19T19:05:00.891 INFO:tasks.workunit.client.1.smithi132.stdout:3/752: dwrite d8/d38/d44/f73 [4194304,4194304] 0 2024-02-19T19:05:00.894 INFO:tasks.workunit.client.1.smithi132.stdout:1/674: mknod d1/cd3 0 2024-02-19T19:05:00.895 INFO:tasks.workunit.client.1.smithi132.stdout:2/789: dwrite d0/d19/d1e/f9f [0,4194304] 0 2024-02-19T19:05:00.897 INFO:tasks.workunit.client.1.smithi132.stdout:5/862: write d8/d23/f6f [2070552,89939] 0 2024-02-19T19:05:00.898 INFO:tasks.workunit.client.1.smithi132.stdout:1/675: chown d1/d3a/c1f 13816 1 2024-02-19T19:05:00.900 INFO:tasks.workunit.client.1.smithi132.stdout:2/790: symlink d0/d24/d58/d53/dab/deb/lf7 0 2024-02-19T19:05:00.901 INFO:tasks.workunit.client.1.smithi132.stdout:7/807: dread d1b/d1e/d41/f47 [0,4194304] 0 2024-02-19T19:05:00.903 INFO:tasks.workunit.client.1.smithi132.stdout:5/863: dread d8/d1a/f6c [0,4194304] 0 2024-02-19T19:05:00.906 INFO:tasks.workunit.client.1.smithi132.stdout:3/753: truncate d8/d38/f3d 2637701 0 2024-02-19T19:05:00.908 INFO:tasks.workunit.client.1.smithi132.stdout:7/808: rmdir d1b/d5a/d63/d96 39 2024-02-19T19:05:00.909 INFO:tasks.workunit.client.1.smithi132.stdout:1/676: mkdir d1/d61/dd4 0 2024-02-19T19:05:00.910 INFO:tasks.workunit.client.1.smithi132.stdout:5/864: unlink d8/fe 0 2024-02-19T19:05:00.910 INFO:tasks.workunit.client.1.smithi132.stdout:2/791: fdatasync d0/d19/d1e/f97 0 2024-02-19T19:05:00.912 INFO:tasks.workunit.client.1.smithi132.stdout:3/754: symlink d8/d13/d2b/d69/d92/d8d/dd7/da2/lec 0 2024-02-19T19:05:00.913 INFO:tasks.workunit.client.1.smithi132.stdout:1/677: stat d1/l9 0 2024-02-19T19:05:00.914 INFO:tasks.workunit.client.1.smithi132.stdout:2/792: read d0/d4/f8e [3303800,85460] 0 2024-02-19T19:05:00.916 INFO:tasks.workunit.client.1.smithi132.stdout:5/865: mknod d8/da/d5a/d7b/dfa/c126 0 2024-02-19T19:05:00.916 INFO:tasks.workunit.client.1.smithi132.stdout:3/755: write d8/d13/d2b/d69/d92/d34/f89 [603993,86611] 0 2024-02-19T19:05:00.921 INFO:tasks.workunit.client.1.smithi132.stdout:1/678: dwrite d1/d61/fa5 [0,4194304] 0 2024-02-19T19:05:00.922 INFO:tasks.workunit.client.1.smithi132.stdout:5/866: creat d8/d23/d29/f127 x:0 0 0 2024-02-19T19:05:00.925 INFO:tasks.workunit.client.1.smithi132.stdout:3/756: unlink d8/d13/d19/l70 0 2024-02-19T19:05:00.926 INFO:tasks.workunit.client.1.smithi132.stdout:3/757: write d8/d13/d57/fc8 [977395,80889] 0 2024-02-19T19:05:00.929 INFO:tasks.workunit.client.1.smithi132.stdout:3/758: read d8/d13/d2b/d69/d7e/fab [267479,70999] 0 2024-02-19T19:05:00.930 INFO:tasks.workunit.client.1.smithi132.stdout:5/867: readlink d8/d23/d29/l94 0 2024-02-19T19:05:00.931 INFO:tasks.workunit.client.1.smithi132.stdout:2/793: link d0/d24/d58/d53/dab/lec d0/d24/d58/d53/dab/d84/dc9/lf8 0 2024-02-19T19:05:00.937 INFO:tasks.workunit.client.1.smithi132.stdout:3/759: mknod d8/d13/d2b/d69/daa/ced 0 2024-02-19T19:05:00.938 INFO:tasks.workunit.client.1.smithi132.stdout:7/809: dread d1b/d1e/d28/f3b [0,4194304] 0 2024-02-19T19:05:00.939 INFO:tasks.workunit.client.1.smithi132.stdout:0/697: sync 2024-02-19T19:05:00.940 INFO:tasks.workunit.client.1.smithi132.stdout:0/698: truncate d2/dc/df/d15/f99 537279 0 2024-02-19T19:05:00.942 INFO:tasks.workunit.client.1.smithi132.stdout:0/699: write d2/dc/df/d15/d2f/d3f/f79 [299345,89699] 0 2024-02-19T19:05:00.942 INFO:tasks.workunit.client.1.smithi132.stdout:0/700: stat d2/d6e/dc8 0 2024-02-19T19:05:00.943 INFO:tasks.workunit.client.1.smithi132.stdout:5/868: rename d8/d2e/d71/dce to d8/da/d5a/d7b/dfa/d128 0 2024-02-19T19:05:00.944 INFO:tasks.workunit.client.1.smithi132.stdout:5/869: write d8/d23/feb [181073,30147] 0 2024-02-19T19:05:00.947 INFO:tasks.workunit.client.1.smithi132.stdout:3/760: write d8/d13/d2f/f4d [2305226,78748] 0 2024-02-19T19:05:00.948 INFO:tasks.workunit.client.1.smithi132.stdout:7/810: mkdir d1b/d1e/d41/da3/dc8/dfe 0 2024-02-19T19:05:00.949 INFO:tasks.workunit.client.1.smithi132.stdout:0/701: readlink d2/dc/df/d15/d2f/d69/lb2 0 2024-02-19T19:05:00.951 INFO:tasks.workunit.client.1.smithi132.stdout:5/870: unlink d8/d23/fb0 0 2024-02-19T19:05:00.952 INFO:tasks.workunit.client.1.smithi132.stdout:1/679: getdents d1/d1e/d77/d2c/d62/d88 0 2024-02-19T19:05:00.952 INFO:tasks.workunit.client.1.smithi132.stdout:0/702: dread - d2/dc/df/d15/d2f/d3f/d63/fe1 zero size 2024-02-19T19:05:00.955 INFO:tasks.workunit.client.1.smithi132.stdout:7/811: creat d1b/d5a/d63/d6b/d79/dc6/fff x:0 0 0 2024-02-19T19:05:00.955 INFO:tasks.workunit.client.1.smithi132.stdout:2/794: dwrite d0/d19/d1e/f9f [4194304,4194304] 0 2024-02-19T19:05:00.955 INFO:tasks.workunit.client.1.smithi132.stdout:3/761: symlink d8/d13/d2b/d69/d92/d34/d42/d6c/de6/lee 0 2024-02-19T19:05:00.956 INFO:tasks.workunit.client.1.smithi132.stdout:3/762: chown d8/lf 1 1 2024-02-19T19:05:00.957 INFO:tasks.workunit.client.1.smithi132.stdout:3/763: fdatasync d8/d13/d19/f96 0 2024-02-19T19:05:00.958 INFO:tasks.workunit.client.1.smithi132.stdout:0/703: creat d2/dc/df/dbd/fed x:0 0 0 2024-02-19T19:05:00.960 INFO:tasks.workunit.client.1.smithi132.stdout:5/871: truncate d8/da/f3c 2435536 0 2024-02-19T19:05:00.960 INFO:tasks.workunit.client.1.smithi132.stdout:2/795: mkdir d0/d3c/d3f/df9 0 2024-02-19T19:05:00.962 INFO:tasks.workunit.client.1.smithi132.stdout:0/704: unlink d2/dc/d4a/d25/f26 0 2024-02-19T19:05:00.966 INFO:tasks.workunit.client.1.smithi132.stdout:7/812: getdents d1b/d1e/d2b/d8f/dae 0 2024-02-19T19:05:00.966 INFO:tasks.workunit.client.1.smithi132.stdout:1/680: write d1/d1e/d77/d2c/d43/f54 [1476356,108160] 0 2024-02-19T19:05:00.968 INFO:tasks.workunit.client.1.smithi132.stdout:0/705: getdents d2/dc/df/d27/d3a/d52/da8 0 2024-02-19T19:05:00.972 INFO:tasks.workunit.client.1.smithi132.stdout:2/796: dwrite d0/d24/d58/d53/dab/d8a/fc0 [0,4194304] 0 2024-02-19T19:05:00.973 INFO:tasks.workunit.client.1.smithi132.stdout:1/681: write d1/d1e/d77/d52/f65 [272363,18274] 0 2024-02-19T19:05:00.973 INFO:tasks.workunit.client.1.smithi132.stdout:5/872: dwrite d8/d2e/d71/f80 [0,4194304] 0 2024-02-19T19:05:00.984 INFO:tasks.workunit.client.1.smithi132.stdout:7/813: getdents d1b/d1e/d28/da9/dd3 0 2024-02-19T19:05:00.985 INFO:tasks.workunit.client.1.smithi132.stdout:3/764: dwrite d8/fc0 [0,4194304] 0 2024-02-19T19:05:00.986 INFO:tasks.workunit.client.1.smithi132.stdout:2/797: dwrite d0/f6 [4194304,4194304] 0 2024-02-19T19:05:00.987 INFO:tasks.workunit.client.1.smithi132.stdout:5/873: dread d8/d2e/dc4/dc8/daa/fe5 [0,4194304] 0 2024-02-19T19:05:00.987 INFO:tasks.workunit.client.1.smithi132.stdout:3/765: readlink d8/d13/d2f/l7f 0 2024-02-19T19:05:00.989 INFO:tasks.workunit.client.1.smithi132.stdout:0/706: getdents d2/d3e/db1 0 2024-02-19T19:05:00.990 INFO:tasks.workunit.client.1.smithi132.stdout:7/814: mknod d1b/d5a/d63/d6b/d79/dc6/c100 0 2024-02-19T19:05:00.998 INFO:tasks.workunit.client.1.smithi132.stdout:3/766: dwrite d8/d13/d2b/d69/d92/d34/f59 [0,4194304] 0 2024-02-19T19:05:00.999 INFO:tasks.workunit.client.1.smithi132.stdout:0/707: rename d2/dc/d4a/fe4 to d2/dc/df/d15/d2f/d3f/fee 0 2024-02-19T19:05:01.001 INFO:tasks.workunit.client.1.smithi132.stdout:3/767: dread - d8/fd4 zero size 2024-02-19T19:05:01.003 INFO:tasks.workunit.client.1.smithi132.stdout:7/815: dwrite d1b/d1e/d41/f47 [0,4194304] 0 2024-02-19T19:05:01.005 INFO:tasks.workunit.client.1.smithi132.stdout:0/708: mkdir d2/dc/df/d15/d2f/def 0 2024-02-19T19:05:01.006 INFO:tasks.workunit.client.1.smithi132.stdout:0/709: write d2/dc/fe [4394359,71619] 0 2024-02-19T19:05:01.008 INFO:tasks.workunit.client.1.smithi132.stdout:7/816: unlink l9 0 2024-02-19T19:05:01.014 INFO:tasks.workunit.client.1.smithi132.stdout:7/817: mknod d1b/d46/d56/c101 0 2024-02-19T19:05:01.014 INFO:tasks.workunit.client.1.smithi132.stdout:2/798: truncate d0/d24/d58/fbc 804318 0 2024-02-19T19:05:01.017 INFO:tasks.workunit.client.1.smithi132.stdout:0/710: dwrite d2/dc/df/d15/d34/d84/da7/fb3 [0,4194304] 0 2024-02-19T19:05:01.019 INFO:tasks.workunit.client.1.smithi132.stdout:0/711: stat d2/dc/df/d15/d2f/fe7 0 2024-02-19T19:05:01.021 INFO:tasks.workunit.client.1.smithi132.stdout:0/712: mknod d2/dc/d4a/d25/dda/cf0 0 2024-02-19T19:05:01.029 INFO:tasks.workunit.client.1.smithi132.stdout:5/874: dread d8/da/f30 [0,4194304] 0 2024-02-19T19:05:01.032 INFO:tasks.workunit.client.1.smithi132.stdout:5/875: mkdir d8/d1a/d35/d129 0 2024-02-19T19:05:01.034 INFO:tasks.workunit.client.1.smithi132.stdout:5/876: mkdir d8/d2e/d71/da6/ddb/dea/d12a 0 2024-02-19T19:05:01.035 INFO:tasks.workunit.client.1.smithi132.stdout:5/877: truncate d8/d1a/d35/f4b 4856716 0 2024-02-19T19:05:01.036 INFO:tasks.workunit.client.1.smithi132.stdout:2/799: write d0/d4/df/d40/d7b/f8d [1026868,33676] 0 2024-02-19T19:05:01.036 INFO:tasks.workunit.client.1.smithi132.stdout:7/818: truncate f8 921790 0 2024-02-19T19:05:01.037 INFO:tasks.workunit.client.1.smithi132.stdout:5/878: chown d8/da/d5a/d84/fd5 2728906 1 2024-02-19T19:05:01.039 INFO:tasks.workunit.client.1.smithi132.stdout:7/819: rmdir d1b/d1e/d28 39 2024-02-19T19:05:01.046 INFO:tasks.workunit.client.1.smithi132.stdout:0/713: dwrite d2/dc/df/d15/d2f/d3f/d63/fe1 [0,4194304] 0 2024-02-19T19:05:01.050 INFO:tasks.workunit.client.1.smithi132.stdout:5/879: dwrite d8/da/db/f109 [0,4194304] 0 2024-02-19T19:05:01.051 INFO:tasks.workunit.client.1.smithi132.stdout:2/800: dwrite d0/d3c/f92 [0,4194304] 0 2024-02-19T19:05:01.053 INFO:tasks.workunit.client.1.smithi132.stdout:5/880: truncate d8/d2e/fb4 300743 0 2024-02-19T19:05:01.054 INFO:tasks.workunit.client.1.smithi132.stdout:7/820: truncate d1b/d1e/d28/f97 749435 0 2024-02-19T19:05:01.056 INFO:tasks.workunit.client.1.smithi132.stdout:7/821: dread - d1b/d1e/d2b/fc0 zero size 2024-02-19T19:05:01.059 INFO:tasks.workunit.client.1.smithi132.stdout:0/714: rmdir d2/dc/df/d15/d2f/def 0 2024-02-19T19:05:01.059 INFO:tasks.workunit.client.1.smithi132.stdout:8/823: sync 2024-02-19T19:05:01.059 INFO:tasks.workunit.client.1.smithi132.stdout:4/719: sync 2024-02-19T19:05:01.059 INFO:tasks.workunit.client.1.smithi132.stdout:6/659: sync 2024-02-19T19:05:01.059 INFO:tasks.workunit.client.1.smithi132.stdout:9/888: sync 2024-02-19T19:05:01.063 INFO:tasks.workunit.client.1.smithi132.stdout:7/822: mknod d1b/d1e/c102 0 2024-02-19T19:05:01.064 INFO:tasks.workunit.client.1.smithi132.stdout:4/720: symlink d4/d19/daa/dc7/lfe 0 2024-02-19T19:05:01.064 INFO:tasks.workunit.client.1.smithi132.stdout:0/715: fdatasync d2/dc/df/d15/d2f/d47/f96 0 2024-02-19T19:05:01.065 INFO:tasks.workunit.client.1.smithi132.stdout:8/824: creat d4/dc/d1b/d20/d23/d68/d7a/de8/f124 x:0 0 0 2024-02-19T19:05:01.067 INFO:tasks.workunit.client.1.smithi132.stdout:5/881: creat d8/d23/d29/dba/f12b x:0 0 0 2024-02-19T19:05:01.067 INFO:tasks.workunit.client.1.smithi132.stdout:0/716: readlink d2/dc/df/d15/d2f/d3f/d63/l91 0 2024-02-19T19:05:01.067 INFO:tasks.workunit.client.1.smithi132.stdout:8/825: write d4/d2d/f31 [6345999,27488] 0 2024-02-19T19:05:01.069 INFO:tasks.workunit.client.1.smithi132.stdout:7/823: mknod d1b/d1e/c103 0 2024-02-19T19:05:01.069 INFO:tasks.workunit.client.1.smithi132.stdout:9/889: dread d12/d8d/d3a/d40/d4e/d77/d89/dc4/fee [0,4194304] 0 2024-02-19T19:05:01.071 INFO:tasks.workunit.client.1.smithi132.stdout:8/826: mknod d4/dc/d1b/c125 0 2024-02-19T19:05:01.071 INFO:tasks.workunit.client.1.smithi132.stdout:9/890: chown d12/d8d/l31 35999 1 2024-02-19T19:05:01.072 INFO:tasks.workunit.client.1.smithi132.stdout:5/882: symlink d8/d2e/dc4/dc8/df1/l12c 0 2024-02-19T19:05:01.073 INFO:tasks.workunit.client.1.smithi132.stdout:7/824: chown d1b/d32 940957 1 2024-02-19T19:05:01.073 INFO:tasks.workunit.client.1.smithi132.stdout:4/721: dwrite d4/f7d [4194304,4194304] 0 2024-02-19T19:05:01.075 INFO:tasks.workunit.client.1.smithi132.stdout:0/717: rename d2/dc/df/d15/f99 to d2/dc/d4a/d25/dda/ff1 0 2024-02-19T19:05:01.076 INFO:tasks.workunit.client.1.smithi132.stdout:4/722: dread - d4/d19/d21/fe1 zero size 2024-02-19T19:05:01.078 INFO:tasks.workunit.client.1.smithi132.stdout:2/801: write d0/d19/f25 [795572,92835] 0 2024-02-19T19:05:01.079 INFO:tasks.workunit.client.1.smithi132.stdout:0/718: write d2/dc/fe [4170641,64067] 0 2024-02-19T19:05:01.083 INFO:tasks.workunit.client.1.smithi132.stdout:7/825: dread d1b/d1e/d2b/d8f/da5/fa8 [0,4194304] 0 2024-02-19T19:05:01.085 INFO:tasks.workunit.client.1.smithi132.stdout:6/660: dwrite d0/d8b/f4a [0,4194304] 0 2024-02-19T19:05:01.086 INFO:tasks.workunit.client.1.smithi132.stdout:6/661: chown d0/db/d25/d3a/d5d/d8f/d46/d5a/f18 7426 1 2024-02-19T19:05:01.087 INFO:tasks.workunit.client.1.smithi132.stdout:6/662: write d0/fb7 [5118313,627] 0 2024-02-19T19:05:01.104 INFO:tasks.workunit.client.1.smithi132.stdout:5/883: symlink d8/d1a/d64/l12d 0 2024-02-19T19:05:01.106 INFO:tasks.workunit.client.1.smithi132.stdout:8/827: rename d4/dc/d1b/d20/d23 to d4/dc/d1b/d38/d126 0 2024-02-19T19:05:01.123 INFO:tasks.workunit.client.1.smithi132.stdout:1/682: sync 2024-02-19T19:05:01.124 INFO:tasks.workunit.client.1.smithi132.stdout:1/683: fsync d1/d1e/d77/d2c/d43/f54 0 2024-02-19T19:05:01.124 INFO:tasks.workunit.client.1.smithi132.stdout:6/663: mkdir d0/db/d25/d3a/d3d/d69/de2 0 2024-02-19T19:05:01.125 INFO:tasks.workunit.client.1.smithi132.stdout:9/891: link d12/d2a/d88/l91 d12/d8d/d3a/d85/l132 0 2024-02-19T19:05:01.129 INFO:tasks.workunit.client.1.smithi132.stdout:1/684: dwrite d1/d95/fc7 [0,4194304] 0 2024-02-19T19:05:01.130 INFO:tasks.workunit.client.1.smithi132.stdout:8/828: mkdir d4/dc/d1b/d38/d126/d127 0 2024-02-19T19:05:01.133 INFO:tasks.workunit.client.1.smithi132.stdout:7/826: fdatasync d1b/d1e/d41/da3/de3/ff2 0 2024-02-19T19:05:01.134 INFO:tasks.workunit.client.1.smithi132.stdout:8/829: truncate d4/dc/d1b/d20/d8a/fff 1025326 0 2024-02-19T19:05:01.136 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:01 smithi132 ceph-mon[37969]: pgmap v4: 65 pgs: 65 active+clean; 1.5 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail 2024-02-19T19:05:01.136 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:01 smithi132 ceph-mon[37969]: mgrmap e37: smithi088.gyviqq(active, since 2s), standbys: smithi132.psllsa 2024-02-19T19:05:01.136 INFO:tasks.workunit.client.1.smithi132.stdout:8/830: read d4/fd [3041456,29068] 0 2024-02-19T19:05:01.137 INFO:tasks.workunit.client.1.smithi132.stdout:2/802: truncate d0/f6 8136580 0 2024-02-19T19:05:01.138 INFO:tasks.workunit.client.1.smithi132.stdout:5/884: mknod d8/d125/c12e 0 2024-02-19T19:05:01.139 INFO:tasks.workunit.client.1.smithi132.stdout:0/719: link d2/dc/df/dbd/fed d2/ff2 0 2024-02-19T19:05:01.143 INFO:tasks.workunit.client.1.smithi132.stdout:1/685: unlink d1/d1e/d77/d52/f56 0 2024-02-19T19:05:01.144 INFO:tasks.workunit.client.1.smithi132.stdout:7/827: write d1b/d1e/d2b/f95 [1716622,44677] 0 2024-02-19T19:05:01.144 INFO:tasks.workunit.client.1.smithi132.stdout:6/664: mknod d0/db/d25/ce3 0 2024-02-19T19:05:01.144 INFO:tasks.workunit.client.1.smithi132.stdout:4/723: getdents d4/d12/d48/d5f 0 2024-02-19T19:05:01.144 INFO:tasks.workunit.client.1.smithi132.stdout:9/892: mkdir d12/db6/d109/d133 0 2024-02-19T19:05:01.165 INFO:tasks.workunit.client.1.smithi132.stdout:0/720: mknod d2/dc/df/d15/d2f/d47/cf3 0 2024-02-19T19:05:01.166 INFO:tasks.workunit.client.1.smithi132.stdout:5/885: creat d8/da/d5a/d7b/d96/d9e/ddf/f12f x:0 0 0 2024-02-19T19:05:01.167 INFO:tasks.workunit.client.1.smithi132.stdout:1/686: chown d1/f5 1 1 2024-02-19T19:05:01.169 INFO:tasks.workunit.client.1.smithi132.stdout:2/803: link d0/d24/d58/d53/dab/fe4 d0/ffa 0 2024-02-19T19:05:01.169 INFO:tasks.workunit.client.1.smithi132.stdout:0/721: chown d2/dc/df/d15/d2f/d3f/l8e 9476 1 2024-02-19T19:05:01.170 INFO:tasks.workunit.client.1.smithi132.stdout:7/828: dread d1b/d1e/d28/f4f [0,4194304] 0 2024-02-19T19:05:01.171 INFO:tasks.workunit.client.1.smithi132.stdout:5/886: symlink d8/da/d5a/d7b/dfa/d128/l130 0 2024-02-19T19:05:01.171 INFO:tasks.workunit.client.1.smithi132.stdout:6/665: truncate d0/db/d25/d3a/d5d/fce 581426 0 2024-02-19T19:05:01.172 INFO:tasks.workunit.client.1.smithi132.stdout:0/722: creat d2/d3e/ff4 x:0 0 0 2024-02-19T19:05:01.175 INFO:tasks.workunit.client.1.smithi132.stdout:2/804: symlink d0/d24/d58/d53/dab/d84/dc9/lfb 0 2024-02-19T19:05:01.175 INFO:tasks.workunit.client.1.smithi132.stdout:5/887: rename d8/d23/d29/d27 to d8/d23/d29/d27/df8/d105/d131 22 2024-02-19T19:05:01.177 INFO:tasks.workunit.client.1.smithi132.stdout:4/724: link d4/d27/d31/d32/d51/c82 d4/cff 0 2024-02-19T19:05:01.178 INFO:tasks.workunit.client.1.smithi132.stdout:1/687: dwrite d1/d1e/d77/d4e/f6d [0,4194304] 0 2024-02-19T19:05:01.179 INFO:tasks.workunit.client.1.smithi132.stdout:6/666: mkdir d0/db/d25/d3a/d4b/d7c/de4 0 2024-02-19T19:05:01.180 INFO:tasks.workunit.client.1.smithi132.stdout:5/888: rmdir d8/da/d5a 39 2024-02-19T19:05:01.182 INFO:tasks.workunit.client.1.smithi132.stdout:4/725: creat d4/d12/da4/f100 x:0 0 0 2024-02-19T19:05:01.184 INFO:tasks.workunit.client.1.smithi132.stdout:6/667: chown d0/db/d25/d3a/d5d/d8f/f39 0 1 2024-02-19T19:05:01.184 INFO:tasks.workunit.client.1.smithi132.stdout:1/688: mknod d1/d1e/d77/d2c/d62/d6b/d85/cd5 0 2024-02-19T19:05:01.186 INFO:tasks.workunit.client.1.smithi132.stdout:5/889: fsync d8/d2e/dc4/dc8/f67 0 2024-02-19T19:05:01.188 INFO:tasks.workunit.client.1.smithi132.stdout:0/723: getdents d2/dc/df/d27/d3a/d52 0 2024-02-19T19:05:01.188 INFO:tasks.workunit.client.1.smithi132.stdout:1/689: unlink d1/d1e/d6a/ca7 0 2024-02-19T19:05:01.190 INFO:tasks.workunit.client.1.smithi132.stdout:4/726: creat d4/d27/d31/d32/d51/d5c/f101 x:0 0 0 2024-02-19T19:05:01.192 INFO:tasks.workunit.client.1.smithi132.stdout:7/829: dread d1b/d1e/d2b/f5d [0,4194304] 0 2024-02-19T19:05:01.192 INFO:tasks.workunit.client.1.smithi132.stdout:0/724: unlink d2/dc/d4a/d25/c50 0 2024-02-19T19:05:01.193 INFO:tasks.workunit.client.1.smithi132.stdout:9/893: dwrite d12/fa4 [0,4194304] 0 2024-02-19T19:05:01.196 INFO:tasks.workunit.client.1.smithi132.stdout:4/727: mknod d4/d19/dd4/c102 0 2024-02-19T19:05:01.197 INFO:tasks.workunit.client.1.smithi132.stdout:7/830: unlink d1b/c1f 0 2024-02-19T19:05:01.199 INFO:tasks.workunit.client.1.smithi132.stdout:5/890: creat d8/d2e/dc4/d113/d11a/f132 x:0 0 0 2024-02-19T19:05:01.200 INFO:tasks.workunit.client.1.smithi132.stdout:1/690: dread d1/d1e/d77/f3f [0,4194304] 0 2024-02-19T19:05:01.200 INFO:tasks.workunit.client.1.smithi132.stdout:9/894: fdatasync d12/d1b/f95 0 2024-02-19T19:05:01.201 INFO:tasks.workunit.client.1.smithi132.stdout:2/805: truncate d0/d24/d58/d53/dab/d8a/da5/db1/fb6 834744 0 2024-02-19T19:05:01.203 INFO:tasks.workunit.client.1.smithi132.stdout:4/728: write d4/d27/d31/d32/fb6 [43017,5375] 0 2024-02-19T19:05:01.204 INFO:tasks.workunit.client.1.smithi132.stdout:6/668: truncate d0/db/f9d 86994 0 2024-02-19T19:05:01.205 INFO:tasks.workunit.client.1.smithi132.stdout:5/891: write d8/d2e/d71/da6/fee [27266,76441] 0 2024-02-19T19:05:01.205 INFO:tasks.workunit.client.1.smithi132.stdout:1/691: unlink d1/f72 0 2024-02-19T19:05:01.207 INFO:tasks.workunit.client.1.smithi132.stdout:4/729: stat d4/d12/d16/cfb 0 2024-02-19T19:05:01.209 INFO:tasks.workunit.client.1.smithi132.stdout:0/725: dwrite d2/d3e/fdc [0,4194304] 0 2024-02-19T19:05:01.210 INFO:tasks.workunit.client.1.smithi132.stdout:9/895: truncate d12/d1b/f1c 4999410 0 2024-02-19T19:05:01.212 INFO:tasks.workunit.client.1.smithi132.stdout:4/730: chown d4/d12/d9f/lc2 20807259 1 2024-02-19T19:05:01.215 INFO:tasks.workunit.client.1.smithi132.stdout:0/726: write d2/dc/d4a/d74/fb5 [1757156,130507] 0 2024-02-19T19:05:01.216 INFO:tasks.workunit.client.1.smithi132.stdout:0/727: write d2/dc/d4a/d25/d43/d5f/fe5 [466996,92058] 0 2024-02-19T19:05:01.217 INFO:tasks.workunit.client.1.smithi132.stdout:6/669: creat d0/db/d25/d3a/d5d/d8f/d46/d5a/fe5 x:0 0 0 2024-02-19T19:05:01.219 INFO:tasks.workunit.client.1.smithi132.stdout:1/692: creat d1/d1e/d77/d52/d64/d75/fd6 x:0 0 0 2024-02-19T19:05:01.220 INFO:tasks.workunit.client.1.smithi132.stdout:1/693: read - d1/d95/d97/fa0 zero size 2024-02-19T19:05:01.221 INFO:tasks.workunit.client.1.smithi132.stdout:5/892: rename d8/d23/d29/l94 to d8/da/d5a/d7b/d96/d9e/l133 0 2024-02-19T19:05:01.223 INFO:tasks.workunit.client.1.smithi132.stdout:7/831: truncate d1b/d1e/d2b/d62/d53/f65 419302 0 2024-02-19T19:05:01.224 INFO:tasks.workunit.client.1.smithi132.stdout:7/832: chown d1b/d1e/d2b/d45 3 1 2024-02-19T19:05:01.228 INFO:tasks.workunit.client.1.smithi132.stdout:1/694: fdatasync d1/d1e/d77/d2c/f38 0 2024-02-19T19:05:01.229 INFO:tasks.workunit.client.1.smithi132.stdout:5/893: mkdir d8/da/db/d134 0 2024-02-19T19:05:01.236 INFO:tasks.workunit.client.1.smithi132.stdout:2/806: dread d0/d24/f2d [0,4194304] 0 2024-02-19T19:05:01.236 INFO:tasks.workunit.client.1.smithi132.stdout:1/695: dwrite d1/d1e/d77/d2c/f7d [0,4194304] 0 2024-02-19T19:05:01.238 INFO:tasks.workunit.client.1.smithi132.stdout:1/696: chown d1/d95/fc7 163 1 2024-02-19T19:05:01.238 INFO:tasks.workunit.client.1.smithi132.stdout:0/728: rmdir d2/de0 0 2024-02-19T19:05:01.238 INFO:tasks.workunit.client.1.smithi132.stdout:3/768: sync 2024-02-19T19:05:01.240 INFO:tasks.workunit.client.1.smithi132.stdout:0/729: write d2/dc/df/faa [2324960,50686] 0 2024-02-19T19:05:01.240 INFO:tasks.workunit.client.1.smithi132.stdout:3/769: write d8/d13/d2f/f9a [168378,21379] 0 2024-02-19T19:05:01.241 INFO:tasks.workunit.client.1.smithi132.stdout:5/894: creat d8/da/d5a/d7b/dfa/d128/d117/f135 x:0 0 0 2024-02-19T19:05:01.241 INFO:tasks.workunit.client.1.smithi132.stdout:4/731: truncate d4/d12/d4c/fce 3526433 0 2024-02-19T19:05:01.242 INFO:tasks.workunit.client.1.smithi132.stdout:4/732: chown d4/d12/f4b 8 1 2024-02-19T19:05:01.243 INFO:tasks.workunit.client.1.smithi132.stdout:5/895: read d8/d2e/f91 [269780,49460] 0 2024-02-19T19:05:01.244 INFO:tasks.workunit.client.1.smithi132.stdout:4/733: fdatasync d4/d27/d31/d32/d51/f93 0 2024-02-19T19:05:01.245 INFO:tasks.workunit.client.1.smithi132.stdout:1/697: mknod d1/d1e/d6a/cd7 0 2024-02-19T19:05:01.246 INFO:tasks.workunit.client.1.smithi132.stdout:4/734: fdatasync d4/d27/d31/d71/d14/d24/f3b 0 2024-02-19T19:05:01.247 INFO:tasks.workunit.client.1.smithi132.stdout:4/735: write d4/d4d/fa7 [354793,126647] 0 2024-02-19T19:05:01.247 INFO:tasks.workunit.client.1.smithi132.stdout:6/670: getdents d0/d8b 0 2024-02-19T19:05:01.249 INFO:tasks.workunit.client.1.smithi132.stdout:0/730: rename d2/d6e/lad to d2/dc/d4a/d25/dc9/lf5 0 2024-02-19T19:05:01.250 INFO:tasks.workunit.client.1.smithi132.stdout:3/770: mknod d8/d13/d2b/d60/cef 0 2024-02-19T19:05:01.252 INFO:tasks.workunit.client.1.smithi132.stdout:5/896: unlink d8/ccc 0 2024-02-19T19:05:01.254 INFO:tasks.workunit.client.1.smithi132.stdout:5/897: stat d8/f62 0 2024-02-19T19:05:01.255 INFO:tasks.workunit.client.1.smithi132.stdout:6/671: readlink d0/db/d25/l67 0 2024-02-19T19:05:01.257 INFO:tasks.workunit.client.1.smithi132.stdout:2/807: creat d0/d4/d11/d2e/ffc x:0 0 0 2024-02-19T19:05:01.258 INFO:tasks.workunit.client.1.smithi132.stdout:1/698: rename d1/d3a/f80 to d1/d1e/d77/d2c/d89/fd8 0 2024-02-19T19:05:01.258 INFO:tasks.workunit.client.1.smithi132.stdout:4/736: dwrite d4/d4d/fa7 [0,4194304] 0 2024-02-19T19:05:01.261 INFO:tasks.workunit.client.1.smithi132.stdout:7/833: getdents d1b/d5a/d63/d96/dcc 0 2024-02-19T19:05:01.261 INFO:tasks.workunit.client.1.smithi132.stdout:2/808: chown d0/d4/df/f5f 922153270 1 2024-02-19T19:05:01.262 INFO:tasks.workunit.client.1.smithi132.stdout:1/699: write d1/d1e/d77/f3c [365837,129261] 0 2024-02-19T19:05:01.262 INFO:tasks.workunit.client.1.smithi132.stdout:3/771: dwrite d8/d13/d2b/d69/d92/d8d/fd8 [0,4194304] 0 2024-02-19T19:05:01.265 INFO:tasks.workunit.client.1.smithi132.stdout:6/672: unlink d0/db/d25/d3a/d5d/d8f/f42 0 2024-02-19T19:05:01.270 INFO:tasks.workunit.client.1.smithi132.stdout:5/898: link d8/da/d5a/d7b/d96/d9e/fff d8/d2e/dc4/dc8/df1/f136 0 2024-02-19T19:05:01.271 INFO:tasks.workunit.client.1.smithi132.stdout:5/899: write d8/d1a/d35/f40 [966554,90374] 0 2024-02-19T19:05:01.273 INFO:tasks.workunit.client.1.smithi132.stdout:5/900: chown d8/da/d5a/d7b/d96/d9e/ddf/d11b/d120 486984146 1 2024-02-19T19:05:01.274 INFO:tasks.workunit.client.1.smithi132.stdout:2/809: creat d0/d24/d58/d53/dab/ffd x:0 0 0 2024-02-19T19:05:01.274 INFO:tasks.workunit.client.1.smithi132.stdout:3/772: creat d8/d13/d2b/d69/d7e/dd6/ff0 x:0 0 0 2024-02-19T19:05:01.276 INFO:tasks.workunit.client.1.smithi132.stdout:0/731: getdents d2/dc/df/d15/d2f/d69 0 2024-02-19T19:05:01.277 INFO:tasks.workunit.client.1.smithi132.stdout:7/834: mkdir d1b/d46/d76/d94/dd1/d104 0 2024-02-19T19:05:01.279 INFO:tasks.workunit.client.1.smithi132.stdout:4/737: link d4/d4d/fa7 d4/d27/d31/d71/d14/d24/f103 0 2024-02-19T19:05:01.280 INFO:tasks.workunit.client.1.smithi132.stdout:1/700: creat d1/d1e/daa/fd9 x:0 0 0 2024-02-19T19:05:01.287 INFO:tasks.workunit.client.1.smithi132.stdout:0/732: creat d2/dc/d4a/d25/dc9/ff6 x:0 0 0 2024-02-19T19:05:01.288 INFO:tasks.workunit.client.1.smithi132.stdout:1/701: stat d1/c51 0 2024-02-19T19:05:01.291 INFO:tasks.workunit.client.1.smithi132.stdout:6/673: creat d0/db/fe6 x:0 0 0 2024-02-19T19:05:01.296 INFO:tasks.workunit.client.1.smithi132.stdout:5/901: mkdir d8/d1a/d64/de2/d118/d137 0 2024-02-19T19:05:01.299 INFO:tasks.workunit.client.1.smithi132.stdout:6/674: dwrite d0/db/d25/d3a/d4b/f54 [0,4194304] 0 2024-02-19T19:05:01.300 INFO:tasks.workunit.client.1.smithi132.stdout:3/773: dread d8/d13/d2f/f58 [0,4194304] 0 2024-02-19T19:05:01.300 INFO:tasks.workunit.client.1.smithi132.stdout:5/902: unlink d8/d2e/dc4/dc8/c66 0 2024-02-19T19:05:01.302 INFO:tasks.workunit.client.1.smithi132.stdout:7/835: dread d1b/d1e/f22 [0,4194304] 0 2024-02-19T19:05:01.302 INFO:tasks.workunit.client.1.smithi132.stdout:0/733: rename d2/dc/df/d15/d2f/f40 to d2/dc/df/d15/ff7 0 2024-02-19T19:05:01.308 INFO:tasks.workunit.client.1.smithi132.stdout:2/810: rename d0/d4/d11/f65 to d0/d4/df/dc2/dad/ffe 0 2024-02-19T19:05:01.309 INFO:tasks.workunit.client.1.smithi132.stdout:0/734: rmdir d2/dc/df/dbd 39 2024-02-19T19:05:01.311 INFO:tasks.workunit.client.1.smithi132.stdout:6/675: mkdir d0/db/d25/d50/d86/de7 0 2024-02-19T19:05:01.311 INFO:tasks.workunit.client.1.smithi132.stdout:4/738: write d4/d19/daa/dc7/f70 [1075538,6465] 0 2024-02-19T19:05:01.312 INFO:tasks.workunit.client.1.smithi132.stdout:7/836: creat d1b/d5a/d63/d6b/f105 x:0 0 0 2024-02-19T19:05:01.313 INFO:tasks.workunit.client.1.smithi132.stdout:3/774: symlink d8/d13/d2b/d69/d92/d34/d42/lf1 0 2024-02-19T19:05:01.314 INFO:tasks.workunit.client.1.smithi132.stdout:4/739: read d4/d12/f2f [3050062,51407] 0 2024-02-19T19:05:01.315 INFO:tasks.workunit.client.1.smithi132.stdout:2/811: rename d0/d3c/f92 to d0/d24/daf/fff 0 2024-02-19T19:05:01.315 INFO:tasks.workunit.client.1.smithi132.stdout:0/735: mkdir d2/d6e/df8 0 2024-02-19T19:05:01.317 INFO:tasks.workunit.client.1.smithi132.stdout:6/676: mknod d0/db/d25/d3a/d3d/d69/ce8 0 2024-02-19T19:05:01.317 INFO:tasks.workunit.client.1.smithi132.stdout:2/812: fdatasync d0/d4/df/f5f 0 2024-02-19T19:05:01.319 INFO:tasks.workunit.client.1.smithi132.stdout:7/837: creat d1b/d46/d7f/f106 x:0 0 0 2024-02-19T19:05:01.320 INFO:tasks.workunit.client.1.smithi132.stdout:2/813: read - d0/d4/df/dc2/dad/fef zero size 2024-02-19T19:05:01.320 INFO:tasks.workunit.client.1.smithi132.stdout:3/775: creat d8/d13/d2b/d69/d92/daf/ff2 x:0 0 0 2024-02-19T19:05:01.322 INFO:tasks.workunit.client.1.smithi132.stdout:4/740: dwrite d4/d19/daa/fd1 [0,4194304] 0 2024-02-19T19:05:01.323 INFO:tasks.workunit.client.1.smithi132.stdout:5/903: link d8/d23/d29/l2f d8/d1a/l138 0 2024-02-19T19:05:01.330 INFO:tasks.workunit.client.1.smithi132.stdout:2/814: mknod d0/d24/daf/c100 0 2024-02-19T19:05:01.332 INFO:tasks.workunit.client.1.smithi132.stdout:4/741: creat d4/d27/d31/d32/d51/d5c/dbc/f104 x:0 0 0 2024-02-19T19:05:01.338 INFO:tasks.workunit.client.1.smithi132.stdout:7/838: dread d1b/d1e/d41/f6d [0,4194304] 0 2024-02-19T19:05:01.341 INFO:tasks.workunit.client.1.smithi132.stdout:4/742: rename d4/d4d/d87/c88 to d4/d27/d7f/dc0/c105 0 2024-02-19T19:05:01.342 INFO:tasks.workunit.client.1.smithi132.stdout:2/815: mknod d0/d4/d11/d2e/c101 0 2024-02-19T19:05:01.347 INFO:tasks.workunit.client.1.smithi132.stdout:3/776: rmdir d8/d13/d2b/d69/d92/d8d/dd7/da2/dcd 0 2024-02-19T19:05:01.347 INFO:tasks.workunit.client.1.smithi132.stdout:4/743: creat d4/d27/d31/d32/d51/d72/f106 x:0 0 0 2024-02-19T19:05:01.348 INFO:tasks.workunit.client.1.smithi132.stdout:2/816: readlink d0/d24/d58/d53/dab/lec 0 2024-02-19T19:05:01.348 INFO:tasks.workunit.client.1.smithi132.stdout:3/777: unlink d8/d13/d2b/d69/d92/f8c 0 2024-02-19T19:05:01.349 INFO:tasks.workunit.client.1.smithi132.stdout:2/817: mkdir d0/d4/df/d40/d68/d102 0 2024-02-19T19:05:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:01 smithi088 ceph-mon[88643]: pgmap v4: 65 pgs: 65 active+clean; 1.5 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail 2024-02-19T19:05:01.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:01 smithi088 ceph-mon[88643]: mgrmap e37: smithi088.gyviqq(active, since 2s), standbys: smithi132.psllsa 2024-02-19T19:05:01.350 INFO:tasks.workunit.client.1.smithi132.stdout:2/818: read - d0/d19/d6a/fe8 zero size 2024-02-19T19:05:01.351 INFO:tasks.workunit.client.1.smithi132.stdout:3/778: creat d8/d38/ff3 x:0 0 0 2024-02-19T19:05:01.351 INFO:tasks.workunit.client.1.smithi132.stdout:0/736: write d2/dc/d4a/d25/d43/d5f/f87 [1035976,90548] 0 2024-02-19T19:05:01.355 INFO:tasks.workunit.client.1.smithi132.stdout:0/737: mkdir d2/dc/df/d27/d3a/d52/df9 0 2024-02-19T19:05:01.360 INFO:tasks.workunit.client.1.smithi132.stdout:6/677: dwrite d0/db/d25/d3a/d5d/f5e [0,4194304] 0 2024-02-19T19:05:01.364 INFO:tasks.workunit.client.1.smithi132.stdout:5/904: dwrite d8/d23/d29/d27/f57 [0,4194304] 0 2024-02-19T19:05:01.365 INFO:tasks.workunit.client.1.smithi132.stdout:2/819: dread d0/d3c/d3f/fd5 [0,4194304] 0 2024-02-19T19:05:01.365 INFO:tasks.workunit.client.1.smithi132.stdout:0/738: getdents d2/dc/df/d15/d2f/d47 0 2024-02-19T19:05:01.366 INFO:tasks.workunit.client.1.smithi132.stdout:6/678: creat d0/db/d25/d3a/d5d/d8f/fe9 x:0 0 0 2024-02-19T19:05:01.368 INFO:tasks.workunit.client.1.smithi132.stdout:2/820: write d0/d4/df/d7a/fce [767963,67044] 0 2024-02-19T19:05:01.369 INFO:tasks.workunit.client.1.smithi132.stdout:0/739: unlink d2/dc/df/l1f 0 2024-02-19T19:05:01.372 INFO:tasks.workunit.client.1.smithi132.stdout:0/740: write d2/dc/fe [1039581,36077] 0 2024-02-19T19:05:01.373 INFO:tasks.workunit.client.1.smithi132.stdout:3/779: dread d8/fd [0,4194304] 0 2024-02-19T19:05:01.373 INFO:tasks.workunit.client.1.smithi132.stdout:2/821: mknod d0/d4/df/d40/dbd/c103 0 2024-02-19T19:05:01.375 INFO:tasks.workunit.client.1.smithi132.stdout:2/822: write d0/d4/df/f51 [4352939,103131] 0 2024-02-19T19:05:01.376 INFO:tasks.workunit.client.1.smithi132.stdout:6/679: link d0/db/d25/d3a/d5d/d8f/d22/f23 d0/db/d25/d3a/d3d/d69/fea 0 2024-02-19T19:05:01.377 INFO:tasks.workunit.client.1.smithi132.stdout:5/905: dwrite d8/d2e/dc4/dc8/df1/f107 [0,4194304] 0 2024-02-19T19:05:01.377 INFO:tasks.workunit.client.1.smithi132.stdout:0/741: symlink d2/dc/d98/lfa 0 2024-02-19T19:05:01.378 INFO:tasks.workunit.client.1.smithi132.stdout:3/780: unlink d8/d13/d19/c1a 0 2024-02-19T19:05:01.380 INFO:tasks.workunit.client.1.smithi132.stdout:5/906: chown d8/da/ld 1703440 1 2024-02-19T19:05:01.380 INFO:tasks.workunit.client.1.smithi132.stdout:6/680: fdatasync d0/db/d25/d3a/d3d/d69/f4e 0 2024-02-19T19:05:01.382 INFO:tasks.workunit.client.1.smithi132.stdout:3/781: creat d8/d13/d2b/d60/dc2/ff4 x:0 0 0 2024-02-19T19:05:01.383 INFO:tasks.workunit.client.1.smithi132.stdout:6/681: write d0/db/f58 [6935,14360] 0 2024-02-19T19:05:01.386 INFO:tasks.workunit.client.1.smithi132.stdout:2/823: getdents d0/d24/d58/d53 0 2024-02-19T19:05:01.387 INFO:tasks.workunit.client.1.smithi132.stdout:3/782: creat d8/d38/ff5 x:0 0 0 2024-02-19T19:05:01.388 INFO:tasks.workunit.client.1.smithi132.stdout:6/682: mknod d0/db/d25/d3a/d5d/d8f/d87/ceb 0 2024-02-19T19:05:01.390 INFO:tasks.workunit.client.1.smithi132.stdout:0/742: getdents d2/dc/df/d15/d2f 0 2024-02-19T19:05:01.393 INFO:tasks.workunit.client.1.smithi132.stdout:6/683: mkdir d0/dc/dec 0 2024-02-19T19:05:01.393 INFO:tasks.workunit.client.1.smithi132.stdout:3/783: read d8/d13/d2f/f47 [5841968,71743] 0 2024-02-19T19:05:01.395 INFO:tasks.workunit.client.1.smithi132.stdout:0/743: unlink d2/dc/df/faa 0 2024-02-19T19:05:01.399 INFO:tasks.workunit.client.1.smithi132.stdout:2/824: creat d0/d4/d11/d2e/d5a/f104 x:0 0 0 2024-02-19T19:05:01.399 INFO:tasks.workunit.client.1.smithi132.stdout:5/907: dread d8/ff [4194304,4194304] 0 2024-02-19T19:05:01.400 INFO:tasks.workunit.client.1.smithi132.stdout:0/744: dread - d2/dc/d4a/d74/f85 zero size 2024-02-19T19:05:01.400 INFO:tasks.workunit.client.1.smithi132.stdout:6/684: dwrite d0/db/d25/d3a/d5d/d8f/d46/d5a/f26 [0,4194304] 0 2024-02-19T19:05:01.402 INFO:tasks.workunit.client.1.smithi132.stdout:3/784: link d8/d38/d44/fcf d8/d13/d2b/d69/d92/daf/ff6 0 2024-02-19T19:05:01.402 INFO:tasks.workunit.client.1.smithi132.stdout:2/825: read d0/d4/df/f3b [161838,21538] 0 2024-02-19T19:05:01.405 INFO:tasks.workunit.client.1.smithi132.stdout:6/685: creat d0/db/d25/d3a/d4b/d7c/fed x:0 0 0 2024-02-19T19:05:01.406 INFO:tasks.workunit.client.1.smithi132.stdout:0/745: mknod d2/dc/df/d27/d3a/d52/da8/cfb 0 2024-02-19T19:05:01.410 INFO:tasks.workunit.client.1.smithi132.stdout:6/686: fdatasync d0/db/d25/d38/f68 0 2024-02-19T19:05:01.411 INFO:tasks.workunit.client.1.smithi132.stdout:5/908: write f5 [5156160,37800] 0 2024-02-19T19:05:01.412 INFO:tasks.workunit.client.1.smithi132.stdout:3/785: dwrite d8/d38/d44/f5e [4194304,4194304] 0 2024-02-19T19:05:01.412 INFO:tasks.workunit.client.1.smithi132.stdout:0/746: dread d2/dc/df/d27/f7c [0,4194304] 0 2024-02-19T19:05:01.412 INFO:tasks.workunit.client.1.smithi132.stdout:2/826: dread d0/d24/d58/fbc [0,4194304] 0 2024-02-19T19:05:01.413 INFO:tasks.workunit.client.1.smithi132.stdout:6/687: fsync d0/d8b/f31 0 2024-02-19T19:05:01.415 INFO:tasks.workunit.client.1.smithi132.stdout:6/688: stat d0/db/d25/d50/d86/dc3 0 2024-02-19T19:05:01.415 INFO:tasks.workunit.client.1.smithi132.stdout:3/786: unlink d8/f5a 0 2024-02-19T19:05:01.418 INFO:tasks.workunit.client.1.smithi132.stdout:3/787: write d8/d13/d19/f2d [8950492,54503] 0 2024-02-19T19:05:01.419 INFO:tasks.workunit.client.1.smithi132.stdout:0/747: truncate d2/dc/d4a/d25/d43/d5f/fb7 575861 0 2024-02-19T19:05:01.421 INFO:tasks.workunit.client.1.smithi132.stdout:3/788: chown d8/d38/d44/cdf 2 1 2024-02-19T19:05:01.422 INFO:tasks.workunit.client.1.smithi132.stdout:5/909: rename d8/d2e/d71/da6/ddb to d8/d2e/dc4/d113/d11a/d139 0 2024-02-19T19:05:01.422 INFO:tasks.workunit.client.1.smithi132.stdout:3/789: readlink d8/d13/l80 0 2024-02-19T19:05:01.424 INFO:tasks.workunit.client.1.smithi132.stdout:6/689: dwrite d0/dc/f66 [4194304,4194304] 0 2024-02-19T19:05:01.424 INFO:tasks.workunit.client.1.smithi132.stdout:3/790: creat d8/d13/d57/ff7 x:0 0 0 2024-02-19T19:05:01.427 INFO:tasks.workunit.client.1.smithi132.stdout:6/690: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f18 [1360048,57955] 0 2024-02-19T19:05:01.428 INFO:tasks.workunit.client.1.smithi132.stdout:3/791: creat d8/d13/d2b/d69/d7e/dd6/ff8 x:0 0 0 2024-02-19T19:05:01.430 INFO:tasks.workunit.client.1.smithi132.stdout:3/792: read - d8/d13/d2b/d69/d92/daf/ff2 zero size 2024-02-19T19:05:01.433 INFO:tasks.workunit.client.1.smithi132.stdout:3/793: symlink d8/d13/d2b/d69/d92/d8d/db2/lf9 0 2024-02-19T19:05:01.435 INFO:tasks.workunit.client.1.smithi132.stdout:3/794: creat d8/d13/deb/ffa x:0 0 0 2024-02-19T19:05:01.436 INFO:tasks.workunit.client.1.smithi132.stdout:3/795: write d8/d13/deb/ffa [431099,73322] 0 2024-02-19T19:05:01.438 INFO:tasks.workunit.client.1.smithi132.stdout:3/796: unlink d8/d13/d57/l74 0 2024-02-19T19:05:01.440 INFO:tasks.workunit.client.1.smithi132.stdout:0/748: dread d2/dc/df/d15/f46 [0,4194304] 0 2024-02-19T19:05:01.444 INFO:tasks.workunit.client.1.smithi132.stdout:6/691: dwrite d0/db/d25/d3a/d3d/d69/faa [0,4194304] 0 2024-02-19T19:05:01.446 INFO:tasks.workunit.client.1.smithi132.stdout:6/692: symlink d0/d8b/lee 0 2024-02-19T19:05:01.451 INFO:tasks.workunit.client.1.smithi132.stdout:0/749: link d2/dc/df/d15/d2f/d3f/dd3/le6 d2/dc/d4a/d25/dda/lfc 0 2024-02-19T19:05:01.453 INFO:tasks.workunit.client.1.smithi132.stdout:3/797: dread d8/d13/d2b/fa9 [0,4194304] 0 2024-02-19T19:05:01.453 INFO:tasks.workunit.client.1.smithi132.stdout:0/750: symlink d2/dc/df/d27/d3a/d52/da8/lfd 0 2024-02-19T19:05:01.454 INFO:tasks.workunit.client.1.smithi132.stdout:6/693: truncate d0/db/d25/d3a/d5d/d8f/d46/d5a/f7d 1718973 0 2024-02-19T19:05:01.455 INFO:tasks.workunit.client.1.smithi132.stdout:6/694: fsync d0/db/d25/d3a/d3d/fa8 0 2024-02-19T19:05:01.456 INFO:tasks.workunit.client.1.smithi132.stdout:0/751: mkdir d2/dc/d4a/d25/d43/d5f/dfe 0 2024-02-19T19:05:01.457 INFO:tasks.workunit.client.1.smithi132.stdout:3/798: dread d8/fc0 [0,4194304] 0 2024-02-19T19:05:01.457 INFO:tasks.workunit.client.1.smithi132.stdout:6/695: symlink d0/db/lef 0 2024-02-19T19:05:01.458 INFO:tasks.workunit.client.1.smithi132.stdout:0/752: rmdir d2/dc/d98 39 2024-02-19T19:05:01.459 INFO:tasks.workunit.client.1.smithi132.stdout:3/799: write d8/d13/d2b/d60/f95 [1015021,125712] 0 2024-02-19T19:05:01.459 INFO:tasks.workunit.client.1.smithi132.stdout:6/696: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f18 [2799255,105634] 0 2024-02-19T19:05:01.462 INFO:tasks.workunit.client.1.smithi132.stdout:6/697: chown d0/db/d25/d38/c43 42808 1 2024-02-19T19:05:01.463 INFO:tasks.workunit.client.1.smithi132.stdout:3/800: unlink d8/d13/d19/c4e 0 2024-02-19T19:05:01.464 INFO:tasks.workunit.client.1.smithi132.stdout:0/753: fsync d2/dc/df/d78/fa1 0 2024-02-19T19:05:01.465 INFO:tasks.workunit.client.1.smithi132.stdout:0/754: write d2/d6e/d7a/fcd [3965887,79616] 0 2024-02-19T19:05:01.466 INFO:tasks.workunit.client.1.smithi132.stdout:3/801: mkdir d8/d13/d2b/d60/db8/dfb 0 2024-02-19T19:05:01.470 INFO:tasks.workunit.client.1.smithi132.stdout:6/698: link d0/db/d25/d3a/d3d/c53 d0/db/d25/d3a/d5d/d8f/d46/d5a/daf/cf0 0 2024-02-19T19:05:01.471 INFO:tasks.workunit.client.1.smithi132.stdout:6/699: chown d0/dc 481340 1 2024-02-19T19:05:01.471 INFO:tasks.workunit.client.1.smithi132.stdout:6/700: chown d0/db/dc9 472 1 2024-02-19T19:05:01.471 INFO:tasks.workunit.client.1.smithi132.stdout:3/802: truncate d8/d93/f7c 2862829 0 2024-02-19T19:05:01.472 INFO:tasks.workunit.client.1.smithi132.stdout:3/803: write d8/d13/d2b/d69/d7e/dd6/ff0 [519424,108621] 0 2024-02-19T19:05:01.474 INFO:tasks.workunit.client.1.smithi132.stdout:6/701: chown d0/db/d25/d3a/d3d/c53 1 1 2024-02-19T19:05:01.475 INFO:tasks.workunit.client.1.smithi132.stdout:3/804: creat d8/d13/d2b/d69/d92/daf/ffc x:0 0 0 2024-02-19T19:05:01.476 INFO:tasks.workunit.client.1.smithi132.stdout:6/702: creat d0/db/d25/d3a/d3d/d69/de2/ff1 x:0 0 0 2024-02-19T19:05:01.476 INFO:tasks.workunit.client.1.smithi132.stdout:3/805: dread - d8/d13/f7d zero size 2024-02-19T19:05:01.478 INFO:tasks.workunit.client.1.smithi132.stdout:6/703: read d0/fa [1716547,101016] 0 2024-02-19T19:05:01.478 INFO:tasks.workunit.client.1.smithi132.stdout:3/806: fdatasync d8/d13/d2f/f66 0 2024-02-19T19:05:01.479 INFO:tasks.workunit.client.1.smithi132.stdout:6/704: write d0/f73 [892269,7039] 0 2024-02-19T19:05:01.481 INFO:tasks.workunit.client.1.smithi132.stdout:6/705: creat d0/db/d25/d3a/d4b/d7c/da2/ff2 x:0 0 0 2024-02-19T19:05:01.482 INFO:tasks.workunit.client.1.smithi132.stdout:3/807: mkdir d8/dfd 0 2024-02-19T19:05:01.484 INFO:tasks.workunit.client.1.smithi132.stdout:3/808: read d8/d13/d2b/d69/da6/f5b [1434294,97209] 0 2024-02-19T19:05:01.484 INFO:tasks.workunit.client.1.smithi132.stdout:6/706: mkdir d0/db/d25/d3a/df3 0 2024-02-19T19:05:01.486 INFO:tasks.workunit.client.1.smithi132.stdout:3/809: mknod d8/d13/d2b/d69/d92/daf/cfe 0 2024-02-19T19:05:01.488 INFO:tasks.workunit.client.1.smithi132.stdout:6/707: rmdir d0/db/d25/d3a/d4b/d7c/de4 0 2024-02-19T19:05:01.489 INFO:tasks.workunit.client.1.smithi132.stdout:3/810: rmdir d8/dfd 0 2024-02-19T19:05:01.489 INFO:tasks.workunit.client.1.smithi132.stdout:6/708: creat d0/d51/ff4 x:0 0 0 2024-02-19T19:05:01.491 INFO:tasks.workunit.client.1.smithi132.stdout:6/709: creat d0/db/dc9/ff5 x:0 0 0 2024-02-19T19:05:01.493 INFO:tasks.workunit.client.1.smithi132.stdout:6/710: mkdir d0/db/d25/d3a/d4b/df6 0 2024-02-19T19:05:01.494 INFO:tasks.workunit.client.1.smithi132.stdout:6/711: mkdir d0/db/dc9/df7 0 2024-02-19T19:05:01.499 INFO:tasks.workunit.client.1.smithi132.stdout:6/712: dwrite d0/db/d25/d3a/d4b/f9a [0,4194304] 0 2024-02-19T19:05:01.500 INFO:tasks.workunit.client.1.smithi132.stdout:6/713: stat d0/db/d25/d3a/d3d/dd9 0 2024-02-19T19:05:01.505 INFO:tasks.workunit.client.1.smithi132.stdout:6/714: dwrite d0/f10 [0,4194304] 0 2024-02-19T19:05:01.508 INFO:tasks.workunit.client.1.smithi132.stdout:6/715: mkdir d0/db/d25/d3a/df3/df8 0 2024-02-19T19:05:01.509 INFO:tasks.workunit.client.1.smithi132.stdout:6/716: write d0/fb7 [2538754,66984] 0 2024-02-19T19:05:01.510 INFO:tasks.workunit.client.1.smithi132.stdout:6/717: chown d0/dc/f2d 7921635 1 2024-02-19T19:05:01.525 INFO:tasks.workunit.client.1.smithi132.stdout:6/718: write d0/db/d25/d3a/d5d/d8f/d22/f7e [3869918,64430] 0 2024-02-19T19:05:01.531 INFO:tasks.workunit.client.1.smithi132.stdout:6/719: dwrite d0/db/d25/d3a/d5d/f5e [0,4194304] 0 2024-02-19T19:05:01.532 INFO:tasks.workunit.client.1.smithi132.stdout:6/720: truncate d0/db/d25/fcb 538534 0 2024-02-19T19:05:01.540 INFO:tasks.workunit.client.1.smithi132.stdout:6/721: dwrite d0/db/d25/d3a/d5d/d8f/d22/f9 [4194304,4194304] 0 2024-02-19T19:05:01.540 INFO:tasks.workunit.client.1.smithi132.stdout:6/722: readlink d0/d8b/lee 0 2024-02-19T19:05:01.546 INFO:tasks.workunit.client.1.smithi132.stdout:6/723: dwrite d0/dc/f66 [0,4194304] 0 2024-02-19T19:05:01.547 INFO:tasks.workunit.client.1.smithi132.stdout:6/724: dread - d0/db/fe6 zero size 2024-02-19T19:05:01.551 INFO:tasks.workunit.client.1.smithi132.stdout:8/831: sync 2024-02-19T19:05:01.551 INFO:tasks.workunit.client.1.smithi132.stdout:9/896: sync 2024-02-19T19:05:01.554 INFO:tasks.workunit.client.1.smithi132.stdout:6/725: dwrite d0/f8a [4194304,4194304] 0 2024-02-19T19:05:01.559 INFO:tasks.workunit.client.1.smithi132.stdout:9/897: rename d12/db6/d109/d133 to d12/d8d/d3a/d40/d4e/d77/d134 0 2024-02-19T19:05:01.560 INFO:tasks.workunit.client.1.smithi132.stdout:9/898: stat d12/d8d/d3a/d40/d4e/c56 0 2024-02-19T19:05:01.561 INFO:tasks.workunit.client.1.smithi132.stdout:6/726: mkdir d0/db/d25/df9 0 2024-02-19T19:05:01.564 INFO:tasks.workunit.client.1.smithi132.stdout:8/832: rename d4/dc/d1b/d33 to d4/dc/d1b/d38/d126/d128 0 2024-02-19T19:05:01.566 INFO:tasks.workunit.client.1.smithi132.stdout:6/727: creat d0/db/d25/d3a/df3/ffa x:0 0 0 2024-02-19T19:05:01.567 INFO:tasks.workunit.client.1.smithi132.stdout:8/833: mkdir d4/d2d/dcc/d129 0 2024-02-19T19:05:01.578 INFO:tasks.workunit.client.1.smithi132.stdout:9/899: write d12/d1b/f1c [992657,26129] 0 2024-02-19T19:05:01.579 INFO:tasks.workunit.client.1.smithi132.stdout:6/728: write d0/db/d25/d3a/d3d/d69/f7f [129715,53257] 0 2024-02-19T19:05:01.580 INFO:tasks.workunit.client.1.smithi132.stdout:6/729: chown d0/f41 943261615 1 2024-02-19T19:05:01.581 INFO:tasks.workunit.client.1.smithi132.stdout:6/730: dread - d0/db/d25/d3a/d5d/d8f/d87/fe0 zero size 2024-02-19T19:05:01.583 INFO:tasks.workunit.client.1.smithi132.stdout:6/731: truncate d0/db/d25/d3a/d4b/d7c/fed 344171 0 2024-02-19T19:05:01.586 INFO:tasks.workunit.client.1.smithi132.stdout:6/732: mknod d0/db/d25/d3a/d4b/cfb 0 2024-02-19T19:05:01.588 INFO:tasks.workunit.client.1.smithi132.stdout:8/834: dread d4/dc/d1b/f43 [0,4194304] 0 2024-02-19T19:05:01.595 INFO:tasks.workunit.client.1.smithi132.stdout:9/900: dread d12/d2a/f2b [0,4194304] 0 2024-02-19T19:05:01.595 INFO:tasks.workunit.client.1.smithi132.stdout:8/835: dwrite d4/dc/d1b/d38/d69/f6d [0,4194304] 0 2024-02-19T19:05:01.597 INFO:tasks.workunit.client.1.smithi132.stdout:8/836: write d4/dc/d1b/d8f/fed [213222,103083] 0 2024-02-19T19:05:01.597 INFO:tasks.workunit.client.1.smithi132.stdout:8/837: fdatasync d4/dc/f1c 0 2024-02-19T19:05:01.599 INFO:tasks.workunit.client.1.smithi132.stdout:9/901: creat d12/f135 x:0 0 0 2024-02-19T19:05:01.601 INFO:tasks.workunit.client.1.smithi132.stdout:6/733: dread d0/fb7 [0,4194304] 0 2024-02-19T19:05:01.603 INFO:tasks.workunit.client.1.smithi132.stdout:9/902: stat d12/d8d/d3a/d40/d4e/d77/d8a/d82/f8c 0 2024-02-19T19:05:01.603 INFO:tasks.workunit.client.1.smithi132.stdout:9/903: write d12/d1b/f1c [3549556,37990] 0 2024-02-19T19:05:01.606 INFO:tasks.workunit.client.1.smithi132.stdout:8/838: link d4/dc/d1b/d38/d126/l70 d4/dc/d1b/d54/d81/dbd/d118/l12a 0 2024-02-19T19:05:01.608 INFO:tasks.workunit.client.1.smithi132.stdout:6/734: creat d0/db/d25/d3a/d5d/d8f/d46/ffc x:0 0 0 2024-02-19T19:05:01.609 INFO:tasks.workunit.client.1.smithi132.stdout:6/735: chown d0/db/l6b 64325 1 2024-02-19T19:05:01.613 INFO:tasks.workunit.client.1.smithi132.stdout:6/736: symlink d0/db/d25/d3a/d4b/lfd 0 2024-02-19T19:05:01.613 INFO:tasks.workunit.client.1.smithi132.stdout:8/839: dwrite d4/dc/d1b/d54/d81/dbd/fef [0,4194304] 0 2024-02-19T19:05:01.618 INFO:tasks.workunit.client.1.smithi132.stdout:9/904: dread d12/d2a/d4c/d61/f6b [0,4194304] 0 2024-02-19T19:05:01.623 INFO:tasks.workunit.client.1.smithi132.stdout:6/737: rename d0/db/d25/d3a/f93 to d0/ffe 0 2024-02-19T19:05:01.626 INFO:tasks.workunit.client.1.smithi132.stdout:8/840: truncate d4/dc/d1b/d38/d69/d74/fe1 967663 0 2024-02-19T19:05:01.627 INFO:tasks.workunit.client.1.smithi132.stdout:8/841: chown d4/dc/d1b/d20/f6e 106478 1 2024-02-19T19:05:01.627 INFO:tasks.workunit.client.1.smithi132.stdout:6/738: symlink d0/lff 0 2024-02-19T19:05:01.628 INFO:tasks.workunit.client.1.smithi132.stdout:8/842: write d4/f29 [1103992,31728] 0 2024-02-19T19:05:01.629 INFO:tasks.workunit.client.1.smithi132.stdout:9/905: getdents d12/d8d/d3a/d40/d4e/d77/d89/dc4/d110 0 2024-02-19T19:05:01.629 INFO:tasks.workunit.client.1.smithi132.stdout:6/739: stat d0/db/d25/d3a/d5d/d8f/d87/fda 0 2024-02-19T19:05:01.630 INFO:tasks.workunit.client.1.smithi132.stdout:9/906: write d12/d2a/d4c/d61/fa8 [1321858,12886] 0 2024-02-19T19:05:01.631 INFO:tasks.workunit.client.1.smithi132.stdout:8/843: mkdir d4/dc/d1b/d54/d81/dbd/dfb/d12b 0 2024-02-19T19:05:01.633 INFO:tasks.workunit.client.1.smithi132.stdout:6/740: creat d0/db/d25/d3a/d5d/d8f/dbe/f100 x:0 0 0 2024-02-19T19:05:01.634 INFO:tasks.workunit.client.1.smithi132.stdout:6/741: write d0/db/d25/d3a/d4b/d7c/fbd [242256,45987] 0 2024-02-19T19:05:01.635 INFO:tasks.workunit.client.1.smithi132.stdout:8/844: mkdir d4/dc/d1b/d20/d12c 0 2024-02-19T19:05:01.637 INFO:tasks.workunit.client.1.smithi132.stdout:6/742: truncate d0/db/d25/d3a/d5d/d8f/d87/fc8 5211484 0 2024-02-19T19:05:01.638 INFO:tasks.workunit.client.1.smithi132.stdout:8/845: readlink d4/dc/d1b/d20/l4b 0 2024-02-19T19:05:01.640 INFO:tasks.workunit.client.1.smithi132.stdout:9/907: rename d12/d8d/d3a/d85 to d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136 0 2024-02-19T19:05:01.640 INFO:tasks.workunit.client.1.smithi132.stdout:6/743: creat d0/d8b/f101 x:0 0 0 2024-02-19T19:05:01.642 INFO:tasks.workunit.client.1.smithi132.stdout:8/846: rename d4/f29 to d4/dc/d1b/d38/d126/da3/f12d 0 2024-02-19T19:05:01.643 INFO:tasks.workunit.client.1.smithi132.stdout:9/908: unlink d12/d8d/f2e 0 2024-02-19T19:05:01.645 INFO:tasks.workunit.client.1.smithi132.stdout:8/847: mknod d4/d2d/c12e 0 2024-02-19T19:05:01.645 INFO:tasks.workunit.client.1.smithi132.stdout:6/744: symlink d0/db/dc9/df7/l102 0 2024-02-19T19:05:01.646 INFO:tasks.workunit.client.1.smithi132.stdout:9/909: mknod d12/db6/d11d/c137 0 2024-02-19T19:05:01.649 INFO:tasks.workunit.client.1.smithi132.stdout:9/910: dread - d12/d2a/d88/db8/f123 zero size 2024-02-19T19:05:01.649 INFO:tasks.workunit.client.1.smithi132.stdout:6/745: mkdir d0/dd1/d103 0 2024-02-19T19:05:01.667 INFO:tasks.workunit.client.1.smithi132.stdout:7/839: sync 2024-02-19T19:05:01.667 INFO:tasks.workunit.client.1.smithi132.stdout:1/702: sync 2024-02-19T19:05:01.667 INFO:tasks.workunit.client.1.smithi132.stdout:7/840: dread - d1b/d1e/f90 zero size 2024-02-19T19:05:01.668 INFO:tasks.workunit.client.1.smithi132.stdout:7/841: chown la 15 1 2024-02-19T19:05:01.672 INFO:tasks.workunit.client.1.smithi132.stdout:1/703: dwrite d1/d1e/d77/d2c/d43/d81/da1/fbb [0,4194304] 0 2024-02-19T19:05:01.693 INFO:tasks.workunit.client.1.smithi132.stdout:6/746: symlink d0/db/dc9/df7/l104 0 2024-02-19T19:05:01.694 INFO:tasks.workunit.client.1.smithi132.stdout:8/848: getdents d4/dc/d1b/d38/d126/d68/d7a 0 2024-02-19T19:05:01.702 INFO:tasks.workunit.client.1.smithi132.stdout:9/911: creat d12/d2a/d4c/d61/f138 x:0 0 0 2024-02-19T19:05:01.703 INFO:tasks.workunit.client.1.smithi132.stdout:6/747: creat d0/db/d25/d50/d86/dc3/f105 x:0 0 0 2024-02-19T19:05:01.716 INFO:tasks.workunit.client.1.smithi132.stdout:7/842: rename d1b/d46/cf7 to d1b/d1e/d28/c107 0 2024-02-19T19:05:01.717 INFO:tasks.workunit.client.1.smithi132.stdout:8/849: mkdir d4/dc/d1b/d8f/daf/d12f 0 2024-02-19T19:05:01.719 INFO:tasks.workunit.client.1.smithi132.stdout:9/912: unlink d12/d1b/f73 0 2024-02-19T19:05:01.722 INFO:tasks.workunit.client.1.smithi132.stdout:1/704: rename d1/d95/d97/fa0 to d1/d1e/d77/d2c/d43/fda 0 2024-02-19T19:05:01.725 INFO:tasks.workunit.client.1.smithi132.stdout:7/843: symlink d1b/l108 0 2024-02-19T19:05:01.727 INFO:tasks.workunit.client.1.smithi132.stdout:7/844: read d1b/d46/d76/d94/f9d [2657,121432] 0 2024-02-19T19:05:01.728 INFO:tasks.workunit.client.1.smithi132.stdout:6/748: mknod d0/db/d25/d3a/d3d/d69/d89/c106 0 2024-02-19T19:05:01.728 INFO:tasks.workunit.client.1.smithi132.stdout:7/845: dread - d1b/d1e/d2b/d62/fe9 zero size 2024-02-19T19:05:01.728 INFO:tasks.workunit.client.1.smithi132.stdout:8/850: creat d4/dc/f130 x:0 0 0 2024-02-19T19:05:01.733 INFO:tasks.workunit.client.1.smithi132.stdout:7/846: rename d1b/d1e/d2b/d62/fe9 to d1b/d1e/d41/da3/dbd/f109 0 2024-02-19T19:05:01.736 INFO:tasks.workunit.client.1.smithi132.stdout:7/847: symlink d1b/d1e/d41/da3/l10a 0 2024-02-19T19:05:01.737 INFO:tasks.workunit.client.1.smithi132.stdout:8/851: dread d4/f83 [0,4194304] 0 2024-02-19T19:05:01.739 INFO:tasks.workunit.client.1.smithi132.stdout:8/852: write d4/d2d/dcc/fcd [505010,19263] 0 2024-02-19T19:05:01.741 INFO:tasks.workunit.client.1.smithi132.stdout:7/848: dread d1b/d1e/d28/f97 [0,4194304] 0 2024-02-19T19:05:01.741 INFO:tasks.workunit.client.1.smithi132.stdout:8/853: symlink d4/dc/d1b/d54/d81/dbd/dfb/l131 0 2024-02-19T19:05:01.742 INFO:tasks.workunit.client.1.smithi132.stdout:7/849: rename d1b/d1e/d28 to d1b/d1e/d28/da9/d10b 22 2024-02-19T19:05:01.742 INFO:tasks.workunit.client.1.smithi132.stdout:8/854: readlink d4/la 0 2024-02-19T19:05:01.743 INFO:tasks.workunit.client.1.smithi132.stdout:8/855: chown d4/dc/d1b/d20/f9f 1 1 2024-02-19T19:05:01.745 INFO:tasks.workunit.client.1.smithi132.stdout:4/744: sync 2024-02-19T19:05:01.746 INFO:tasks.workunit.client.1.smithi132.stdout:4/745: write d4/d27/d31/fbf [3732998,69802] 0 2024-02-19T19:05:01.746 INFO:tasks.workunit.client.1.smithi132.stdout:8/856: unlink d4/dc/d1b/d54/d81/dbd/ff9 0 2024-02-19T19:05:01.747 INFO:tasks.workunit.client.1.smithi132.stdout:8/857: fdatasync d4/f24 0 2024-02-19T19:05:01.748 INFO:tasks.workunit.client.1.smithi132.stdout:7/850: getdents d1b/d1e/d28 0 2024-02-19T19:05:01.749 INFO:tasks.workunit.client.1.smithi132.stdout:7/851: read - d1b/d1e/d41/da3/dbd/fec zero size 2024-02-19T19:05:01.749 INFO:tasks.workunit.client.1.smithi132.stdout:9/913: write d12/d8d/d3a/d40/d4e/d77/d8a/d82/f8c [5223186,83253] 0 2024-02-19T19:05:01.750 INFO:tasks.workunit.client.1.smithi132.stdout:1/705: write d1/d1e/d77/f3f [732782,42180] 0 2024-02-19T19:05:01.752 INFO:tasks.workunit.client.1.smithi132.stdout:8/858: unlink d4/dc/f44 0 2024-02-19T19:05:01.755 INFO:tasks.workunit.client.1.smithi132.stdout:9/914: creat d12/d8d/d3a/d40/d4e/d77/d134/f139 x:0 0 0 2024-02-19T19:05:01.756 INFO:tasks.workunit.client.1.smithi132.stdout:1/706: mknod d1/d1e/d77/d78/cdb 0 2024-02-19T19:05:01.757 INFO:tasks.workunit.client.1.smithi132.stdout:8/859: dread - d4/dc/d1b/d20/d8a/fa2 zero size 2024-02-19T19:05:01.760 INFO:tasks.workunit.client.1.smithi132.stdout:7/852: dwrite d1b/d1e/d2b/fc0 [0,4194304] 0 2024-02-19T19:05:01.761 INFO:tasks.workunit.client.1.smithi132.stdout:4/746: dwrite d4/d12/d48/fde [4194304,4194304] 0 2024-02-19T19:05:01.762 INFO:tasks.workunit.client.1.smithi132.stdout:1/707: write d1/d1e/d77/d2c/d62/d6b/d85/f94 [269143,124779] 0 2024-02-19T19:05:01.767 INFO:tasks.workunit.client.1.smithi132.stdout:7/853: write d1b/d1e/d41/f47 [2404729,76443] 0 2024-02-19T19:05:01.767 INFO:tasks.workunit.client.1.smithi132.stdout:6/749: dwrite d0/db/d25/d3a/d5d/d8f/d22/f23 [4194304,4194304] 0 2024-02-19T19:05:01.771 INFO:tasks.workunit.client.1.smithi132.stdout:4/747: creat d4/d12/d9f/f107 x:0 0 0 2024-02-19T19:05:01.772 INFO:tasks.workunit.client.1.smithi132.stdout:9/915: dwrite d12/d8d/d3a/d40/f66 [4194304,4194304] 0 2024-02-19T19:05:01.779 INFO:tasks.workunit.client.1.smithi132.stdout:9/916: chown d12/d8d/d3a/d40/d4e/d77/d9d/fcd 515773 1 2024-02-19T19:05:01.779 INFO:tasks.workunit.client.1.smithi132.stdout:1/708: dwrite d1/f42 [0,4194304] 0 2024-02-19T19:05:01.783 INFO:tasks.workunit.client.1.smithi132.stdout:4/748: rename d4/d27/d31/d71/d14/d24/f3b to d4/d27/d7f/f108 0 2024-02-19T19:05:01.784 INFO:tasks.workunit.client.1.smithi132.stdout:7/854: dwrite d1b/d1e/fd6 [0,4194304] 0 2024-02-19T19:05:01.786 INFO:tasks.workunit.client.1.smithi132.stdout:7/855: rename d1b/d1e/d28/da9 to d1b/d1e/d28/da9/dd3/ddb/ded/d10c 22 2024-02-19T19:05:01.788 INFO:tasks.workunit.client.1.smithi132.stdout:1/709: creat d1/fdc x:0 0 0 2024-02-19T19:05:01.794 INFO:tasks.workunit.client.1.smithi132.stdout:7/856: dwrite d1b/d1e/d2b/fb4 [0,4194304] 0 2024-02-19T19:05:01.818 INFO:tasks.workunit.client.1.smithi132.stdout:6/750: truncate d0/db/d25/d3a/d3d/d69/faa 2898734 0 2024-02-19T19:05:01.819 INFO:tasks.workunit.client.1.smithi132.stdout:4/749: truncate d4/d27/d73/fa6 4259596 0 2024-02-19T19:05:01.820 INFO:tasks.workunit.client.1.smithi132.stdout:4/750: readlink d4/d19/daa/dc7/lfe 0 2024-02-19T19:05:01.820 INFO:tasks.workunit.client.1.smithi132.stdout:6/751: mkdir d0/db/d25/d3a/d3d/d69/d107 0 2024-02-19T19:05:01.821 INFO:tasks.workunit.client.1.smithi132.stdout:5/910: sync 2024-02-19T19:05:01.821 INFO:tasks.workunit.client.1.smithi132.stdout:0/755: sync 2024-02-19T19:05:01.821 INFO:tasks.workunit.client.1.smithi132.stdout:2/827: sync 2024-02-19T19:05:01.821 INFO:tasks.workunit.client.1.smithi132.stdout:4/751: chown d4/d4d/d87 1913 1 2024-02-19T19:05:01.823 INFO:tasks.workunit.client.1.smithi132.stdout:5/911: readlink d8/d2e/dc4/dc8/l7e 0 2024-02-19T19:05:01.826 INFO:tasks.workunit.client.1.smithi132.stdout:0/756: write d2/dc/d4a/d25/d43/d5f/fb7 [1072998,61260] 0 2024-02-19T19:05:01.826 INFO:tasks.workunit.client.1.smithi132.stdout:7/857: dwrite d1b/d5a/d63/d6b/fcf [0,4194304] 0 2024-02-19T19:05:01.828 INFO:tasks.workunit.client.1.smithi132.stdout:1/710: dread d1/d1e/f2a [0,4194304] 0 2024-02-19T19:05:01.829 INFO:tasks.workunit.client.1.smithi132.stdout:0/757: write d2/dc/d4a/d25/dc9/ff6 [932506,10960] 0 2024-02-19T19:05:01.832 INFO:tasks.workunit.client.1.smithi132.stdout:6/752: symlink d0/db/d25/d3a/d4b/l108 0 2024-02-19T19:05:01.834 INFO:tasks.workunit.client.1.smithi132.stdout:5/912: readlink d8/da/db/l3e 0 2024-02-19T19:05:01.837 INFO:tasks.workunit.client.1.smithi132.stdout:7/858: read d1b/d5a/d63/d6b/d86/fac [1592432,92273] 0 2024-02-19T19:05:01.838 INFO:tasks.workunit.client.1.smithi132.stdout:0/758: dread d2/dc/d4a/d25/d43/d5f/fa4 [0,4194304] 0 2024-02-19T19:05:01.839 INFO:tasks.workunit.client.1.smithi132.stdout:0/759: readlink d2/dc/df/d27/l48 0 2024-02-19T19:05:01.840 INFO:tasks.workunit.client.1.smithi132.stdout:0/760: truncate d2/dc/df/d15/d2f/f64 5239432 0 2024-02-19T19:05:01.869 INFO:tasks.workunit.client.1.smithi132.stdout:1/711: rename d1/c31 to d1/d1e/d77/d2c/d62/d6b/d85/cdd 0 2024-02-19T19:05:01.870 INFO:tasks.workunit.client.1.smithi132.stdout:2/828: getdents d0/d24/d58/d53/dab/d8a/da5 0 2024-02-19T19:05:01.871 INFO:tasks.workunit.client.1.smithi132.stdout:7/859: rmdir d1b/d46/d76/d94/dd1/d104 0 2024-02-19T19:05:01.872 INFO:tasks.workunit.client.1.smithi132.stdout:1/712: write d1/d1e/d77/d2c/d43/d81/da1/fc1 [926371,68464] 0 2024-02-19T19:05:01.873 INFO:tasks.workunit.client.1.smithi132.stdout:0/761: mkdir d2/dc/d4a/d25/d43/d5f/dfe/dff 0 2024-02-19T19:05:01.877 INFO:tasks.workunit.client.1.smithi132.stdout:4/752: dread d4/d12/d28/f62 [0,4194304] 0 2024-02-19T19:05:01.878 INFO:tasks.workunit.client.1.smithi132.stdout:0/762: mknod d2/dc/d4a/d25/dc9/c100 0 2024-02-19T19:05:01.879 INFO:tasks.workunit.client.1.smithi132.stdout:4/753: fsync d4/d27/d31/d32/d51/d72/fdc 0 2024-02-19T19:05:01.880 INFO:tasks.workunit.client.1.smithi132.stdout:1/713: write d1/d1e/d77/d52/d64/d75/f83 [1378022,18567] 0 2024-02-19T19:05:01.881 INFO:tasks.workunit.client.1.smithi132.stdout:0/763: mknod d2/c101 0 2024-02-19T19:05:01.882 INFO:tasks.workunit.client.1.smithi132.stdout:1/714: rmdir d1/d95 39 2024-02-19T19:05:01.883 INFO:tasks.workunit.client.1.smithi132.stdout:4/754: mkdir d4/d27/d31/d32/d51/d5c/dbc/d109 0 2024-02-19T19:05:01.884 INFO:tasks.workunit.client.1.smithi132.stdout:1/715: readlink d1/l70 0 2024-02-19T19:05:01.884 INFO:tasks.workunit.client.1.smithi132.stdout:5/913: dread d8/da/db/f4f [0,4194304] 0 2024-02-19T19:05:01.884 INFO:tasks.workunit.client.1.smithi132.stdout:2/829: dwrite d0/d19/f2c [0,4194304] 0 2024-02-19T19:05:01.886 INFO:tasks.workunit.client.1.smithi132.stdout:4/755: dread - d4/d27/d7f/dc0/ff0 zero size 2024-02-19T19:05:01.888 INFO:tasks.workunit.client.1.smithi132.stdout:0/764: rmdir d2/dc/df/d27/d3a/d52/df9 0 2024-02-19T19:05:01.889 INFO:tasks.workunit.client.1.smithi132.stdout:0/765: read d2/dc/fe [211932,46016] 0 2024-02-19T19:05:01.890 INFO:tasks.workunit.client.1.smithi132.stdout:5/914: creat d8/da/d5a/d84/dbf/f13a x:0 0 0 2024-02-19T19:05:01.891 INFO:tasks.workunit.client.1.smithi132.stdout:1/716: getdents d1/d1e/d77/d2c/d62/d6b/dd1 0 2024-02-19T19:05:01.892 INFO:tasks.workunit.client.1.smithi132.stdout:4/756: mknod d4/d19/d21/c10a 0 2024-02-19T19:05:01.893 INFO:tasks.workunit.client.1.smithi132.stdout:1/717: write d1/d1e/f74 [1535170,7504] 0 2024-02-19T19:05:01.893 INFO:tasks.workunit.client.1.smithi132.stdout:5/915: truncate d8/f9b 939317 0 2024-02-19T19:05:01.896 INFO:tasks.workunit.client.1.smithi132.stdout:4/757: creat d4/d12/d4c/f10b x:0 0 0 2024-02-19T19:05:01.899 INFO:tasks.workunit.client.1.smithi132.stdout:4/758: mkdir d4/d19/daa/dc7/d10c 0 2024-02-19T19:05:01.901 INFO:tasks.workunit.client.1.smithi132.stdout:2/830: dread d0/d24/f2d [0,4194304] 0 2024-02-19T19:05:01.902 INFO:tasks.workunit.client.1.smithi132.stdout:5/916: dwrite d8/d23/d29/dba/f12b [0,4194304] 0 2024-02-19T19:05:01.904 INFO:tasks.workunit.client.1.smithi132.stdout:1/718: rmdir d1/d3a/d50/db2 0 2024-02-19T19:05:01.905 INFO:tasks.workunit.client.1.smithi132.stdout:2/831: mknod d0/d24/d58/d53/dab/d84/c105 0 2024-02-19T19:05:01.905 INFO:tasks.workunit.client.1.smithi132.stdout:4/759: write d4/d27/d31/d71/d37/f96 [2500873,14226] 0 2024-02-19T19:05:01.907 INFO:tasks.workunit.client.1.smithi132.stdout:5/917: mknod d8/da/db/c13b 0 2024-02-19T19:05:01.911 INFO:tasks.workunit.client.1.smithi132.stdout:0/766: dwrite d2/dc/d4a/d74/f85 [0,4194304] 0 2024-02-19T19:05:01.912 INFO:tasks.workunit.client.1.smithi132.stdout:4/760: rmdir d4/d19/d21/d56 39 2024-02-19T19:05:01.913 INFO:tasks.workunit.client.1.smithi132.stdout:2/832: mknod d0/d4/df/d40/d68/d102/c106 0 2024-02-19T19:05:01.913 INFO:tasks.workunit.client.1.smithi132.stdout:5/918: mkdir d8/d2e/dc4/dc8/da9/d13c 0 2024-02-19T19:05:01.914 INFO:tasks.workunit.client.1.smithi132.stdout:1/719: dread d1/d1e/f2a [0,4194304] 0 2024-02-19T19:05:01.915 INFO:tasks.workunit.client.1.smithi132.stdout:2/833: mknod d0/d24/d58/d53/c107 0 2024-02-19T19:05:01.915 INFO:tasks.workunit.client.1.smithi132.stdout:3/811: sync 2024-02-19T19:05:01.915 INFO:tasks.workunit.client.1.smithi132.stdout:2/834: stat d0/d4/d11/f20 0 2024-02-19T19:05:01.917 INFO:tasks.workunit.client.1.smithi132.stdout:5/919: dread - d8/d23/d3b/fd0 zero size 2024-02-19T19:05:01.918 INFO:tasks.workunit.client.1.smithi132.stdout:0/767: creat d2/dc/df/d15/d2f/d3f/dd3/f102 x:0 0 0 2024-02-19T19:05:01.919 INFO:tasks.workunit.client.1.smithi132.stdout:1/720: dread d1/d1e/d77/d52/d64/d75/f83 [0,4194304] 0 2024-02-19T19:05:01.920 INFO:tasks.workunit.client.1.smithi132.stdout:1/721: dread - d1/d1e/d77/d52/fc3 zero size 2024-02-19T19:05:01.920 INFO:tasks.workunit.client.1.smithi132.stdout:0/768: write d2/dc/df/d27/d3a/d52/fd0 [470230,102448] 0 2024-02-19T19:05:01.922 INFO:tasks.workunit.client.1.smithi132.stdout:1/722: dread - d1/d1e/d77/d2c/d62/d6b/fb6 zero size 2024-02-19T19:05:01.923 INFO:tasks.workunit.client.1.smithi132.stdout:1/723: chown d1/d1e/d6a/l9a 110857 1 2024-02-19T19:05:01.925 INFO:tasks.workunit.client.1.smithi132.stdout:3/812: rmdir d8 39 2024-02-19T19:05:01.926 INFO:tasks.workunit.client.1.smithi132.stdout:5/920: readlink d8/d2e/dc4/dc8/l73 0 2024-02-19T19:05:01.928 INFO:tasks.workunit.client.1.smithi132.stdout:0/769: dwrite d2/dc/df/d15/fc3 [0,4194304] 0 2024-02-19T19:05:01.930 INFO:tasks.workunit.client.1.smithi132.stdout:1/724: mknod d1/d1e/d77/d2c/d43/d81/da1/cde 0 2024-02-19T19:05:01.930 INFO:tasks.workunit.client.1.smithi132.stdout:4/761: dread d4/d12/f30 [0,4194304] 0 2024-02-19T19:05:01.930 INFO:tasks.workunit.client.1.smithi132.stdout:2/835: mknod d0/d4/df/d40/c108 0 2024-02-19T19:05:01.933 INFO:tasks.workunit.client.1.smithi132.stdout:1/725: read - d1/d1e/d77/d2c/d43/fda zero size 2024-02-19T19:05:01.934 INFO:tasks.workunit.client.1.smithi132.stdout:5/921: creat d8/d2e/d71/f13d x:0 0 0 2024-02-19T19:05:01.936 INFO:tasks.workunit.client.1.smithi132.stdout:5/922: stat d8/da/l52 0 2024-02-19T19:05:01.936 INFO:tasks.workunit.client.1.smithi132.stdout:0/770: rename d2/dc/l49 to d2/l103 0 2024-02-19T19:05:01.937 INFO:tasks.workunit.client.1.smithi132.stdout:1/726: write d1/d1e/d77/d4e/f5f [7703927,123301] 0 2024-02-19T19:05:01.937 INFO:tasks.workunit.client.1.smithi132.stdout:2/836: creat d0/d24/d58/d53/dab/d8a/f109 x:0 0 0 2024-02-19T19:05:01.939 INFO:tasks.workunit.client.1.smithi132.stdout:5/923: chown d8/d1a/cb7 4778 1 2024-02-19T19:05:01.940 INFO:tasks.workunit.client.1.smithi132.stdout:0/771: write d2/dc/df/d15/d2f/d3f/f79 [529520,95693] 0 2024-02-19T19:05:01.940 INFO:tasks.workunit.client.1.smithi132.stdout:4/762: rename d4/d12/f1a to d4/d27/d31/d32/d51/d72/f10d 0 2024-02-19T19:05:01.941 INFO:tasks.workunit.client.1.smithi132.stdout:5/924: chown d8/d2e/dc4/dc8/daa/f11e 8251334 1 2024-02-19T19:05:01.941 INFO:tasks.workunit.client.1.smithi132.stdout:1/727: rmdir d1/d1e/d77/d2c/d62/d6b/d85 39 2024-02-19T19:05:01.942 INFO:tasks.workunit.client.1.smithi132.stdout:4/763: stat d4/d12/d48/d5f/cf7 0 2024-02-19T19:05:01.943 INFO:tasks.workunit.client.1.smithi132.stdout:4/764: write d4/d19/daa/fd1 [490589,41515] 0 2024-02-19T19:05:01.943 INFO:tasks.workunit.client.1.smithi132.stdout:2/837: link d0/d19/d1e/f9f d0/d4/df/d40/dbd/f10a 0 2024-02-19T19:05:01.944 INFO:tasks.workunit.client.1.smithi132.stdout:4/765: readlink d4/d12/d16/l77 0 2024-02-19T19:05:01.946 INFO:tasks.workunit.client.1.smithi132.stdout:0/772: rename d2/dc/d4a/d25/d43/d5f/fe5 to d2/dc/d4a/d25/d43/d5f/f104 0 2024-02-19T19:05:01.946 INFO:tasks.workunit.client.1.smithi132.stdout:4/766: write d4/d12/d16/f6c [1107209,104686] 0 2024-02-19T19:05:01.947 INFO:tasks.workunit.client.1.smithi132.stdout:1/728: mkdir d1/d1e/d77/d2c/d43/db8/ddf 0 2024-02-19T19:05:01.949 INFO:tasks.workunit.client.1.smithi132.stdout:1/729: write d1/d1e/d77/d2c/f7d [3096086,12224] 0 2024-02-19T19:05:01.949 INFO:tasks.workunit.client.1.smithi132.stdout:2/838: chown d0/d24/d58/d53/dab/d8a/da5/db1/fb6 19 1 2024-02-19T19:05:01.951 INFO:tasks.workunit.client.1.smithi132.stdout:4/767: mkdir d4/d27/d31/d71/d14/d24/d49/d10e 0 2024-02-19T19:05:01.952 INFO:tasks.workunit.client.1.smithi132.stdout:4/768: truncate d4/d12/d28/f3e 4833878 0 2024-02-19T19:05:01.954 INFO:tasks.workunit.client.1.smithi132.stdout:0/773: link d2/dc/df/d15/d2f/cea d2/dc/df/dbd/c105 0 2024-02-19T19:05:01.956 INFO:tasks.workunit.client.1.smithi132.stdout:4/769: creat d4/d12/d48/d5f/f10f x:0 0 0 2024-02-19T19:05:01.958 INFO:tasks.workunit.client.1.smithi132.stdout:0/774: symlink d2/d6e/df8/l106 0 2024-02-19T19:05:01.959 INFO:tasks.workunit.client.1.smithi132.stdout:4/770: mknod d4/d12/d48/d5f/c110 0 2024-02-19T19:05:01.962 INFO:tasks.workunit.client.1.smithi132.stdout:3/813: dwrite d8/d13/d2b/d69/d92/d34/f79 [0,4194304] 0 2024-02-19T19:05:01.964 INFO:tasks.workunit.client.1.smithi132.stdout:5/925: dread d8/d2e/dc4/d113/d11a/d139/dea/f102 [0,4194304] 0 2024-02-19T19:05:01.964 INFO:tasks.workunit.client.1.smithi132.stdout:0/775: creat d2/dc/df/d78/f107 x:0 0 0 2024-02-19T19:05:01.965 INFO:tasks.workunit.client.1.smithi132.stdout:4/771: creat d4/d27/d31/d71/f111 x:0 0 0 2024-02-19T19:05:01.967 INFO:tasks.workunit.client.1.smithi132.stdout:0/776: unlink d2/dc/l55 0 2024-02-19T19:05:01.967 INFO:tasks.workunit.client.1.smithi132.stdout:2/839: write d0/d24/d58/d53/dab/fe4 [2749663,106667] 0 2024-02-19T19:05:01.968 INFO:tasks.workunit.client.1.smithi132.stdout:3/814: mkdir d8/d13/d2b/d69/d7e/dda/dff 0 2024-02-19T19:05:01.968 INFO:tasks.workunit.client.1.smithi132.stdout:0/777: fdatasync d2/dc/df/d15/d34/fdb 0 2024-02-19T19:05:01.970 INFO:tasks.workunit.client.1.smithi132.stdout:5/926: mkdir d8/da/d5a/d7b/d13e 0 2024-02-19T19:05:01.971 INFO:tasks.workunit.client.1.smithi132.stdout:1/730: truncate d1/f42 1120147 0 2024-02-19T19:05:01.971 INFO:tasks.workunit.client.1.smithi132.stdout:4/772: fsync d4/d12/d48/d5f/f10f 0 2024-02-19T19:05:01.974 INFO:tasks.workunit.client.1.smithi132.stdout:3/815: rmdir d8/d13/d2b/d69/d92/d34/d97 39 2024-02-19T19:05:01.975 INFO:tasks.workunit.client.1.smithi132.stdout:5/927: mkdir d8/d2e/dc4/dc8/daa/d13f 0 2024-02-19T19:05:01.976 INFO:tasks.workunit.client.1.smithi132.stdout:6/753: sync 2024-02-19T19:05:01.976 INFO:tasks.workunit.client.1.smithi132.stdout:8/860: sync 2024-02-19T19:05:01.976 INFO:tasks.workunit.client.1.smithi132.stdout:9/917: sync 2024-02-19T19:05:01.978 INFO:tasks.workunit.client.1.smithi132.stdout:0/778: creat d2/dc/df/d78/f108 x:0 0 0 2024-02-19T19:05:01.978 INFO:tasks.workunit.client.1.smithi132.stdout:2/840: dwrite d0/d4/df/dc2/dad/fef [0,4194304] 0 2024-02-19T19:05:01.979 INFO:tasks.workunit.client.1.smithi132.stdout:4/773: write d4/d19/d21/d56/f8c [221404,24231] 0 2024-02-19T19:05:01.981 INFO:tasks.workunit.client.1.smithi132.stdout:8/861: chown d4/dc/d1b/d38/d126/l3b 103 1 2024-02-19T19:05:01.983 INFO:tasks.workunit.client.1.smithi132.stdout:3/816: mknod d8/d13/d2b/d69/d92/d34/c100 0 2024-02-19T19:05:02.032 INFO:tasks.workunit.client.1.smithi132.stdout:9/918: creat d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/dc8/f13a x:0 0 0 2024-02-19T19:05:02.033 INFO:tasks.workunit.client.1.smithi132.stdout:8/862: creat d4/dc/d1b/f132 x:0 0 0 2024-02-19T19:05:02.034 INFO:tasks.workunit.client.1.smithi132.stdout:1/731: dread d1/d1e/d77/d2c/d89/f8d [0,4194304] 0 2024-02-19T19:05:02.034 INFO:tasks.workunit.client.1.smithi132.stdout:5/928: dwrite d8/da/dc2/f106 [0,4194304] 0 2024-02-19T19:05:02.039 INFO:tasks.workunit.client.1.smithi132.stdout:8/863: fdatasync d4/dc/d1b/d38/d126/f28 0 2024-02-19T19:05:02.039 INFO:tasks.workunit.client.1.smithi132.stdout:0/779: dwrite d2/dc/d4a/d25/dc9/ff6 [0,4194304] 0 2024-02-19T19:05:02.039 INFO:tasks.workunit.client.1.smithi132.stdout:4/774: symlink d4/d27/l112 0 2024-02-19T19:05:02.040 INFO:tasks.workunit.client.1.smithi132.stdout:9/919: rmdir d12/d8d/d3a 39 2024-02-19T19:05:02.041 INFO:tasks.workunit.client.1.smithi132.stdout:3/817: unlink d8/d93/c9d 0 2024-02-19T19:05:02.042 INFO:tasks.workunit.client.1.smithi132.stdout:6/754: truncate d0/db/d25/d3a/d5d/d8f/d46/d5a/f7d 26150 0 2024-02-19T19:05:02.045 INFO:tasks.workunit.client.1.smithi132.stdout:3/818: write d8/d38/ff5 [880094,116146] 0 2024-02-19T19:05:02.046 INFO:tasks.workunit.client.1.smithi132.stdout:4/775: creat d4/d12/d48/d5f/f113 x:0 0 0 2024-02-19T19:05:02.046 INFO:tasks.workunit.client.1.smithi132.stdout:5/929: write d8/d2e/dc4/d113/d11a/d139/dea/f102 [935901,56872] 0 2024-02-19T19:05:02.048 INFO:tasks.workunit.client.1.smithi132.stdout:1/732: creat d1/d1e/d77/d78/dc0/fe0 x:0 0 0 2024-02-19T19:05:02.052 INFO:tasks.workunit.client.1.smithi132.stdout:3/819: dread - d8/d13/d2b/d69/d92/d8d/db2/fde zero size 2024-02-19T19:05:02.054 INFO:tasks.workunit.client.1.smithi132.stdout:9/920: symlink d12/d8d/d3a/l13b 0 2024-02-19T19:05:02.055 INFO:tasks.workunit.client.1.smithi132.stdout:3/820: read d8/d38/f4b [704579,1889] 0 2024-02-19T19:05:02.055 INFO:tasks.workunit.client.1.smithi132.stdout:0/780: rename d2/dc/d4a/d25/f38 to d2/f109 0 2024-02-19T19:05:02.056 INFO:tasks.workunit.client.1.smithi132.stdout:6/755: mknod d0/db/d25/d3a/d3d/dd9/c109 0 2024-02-19T19:05:02.059 INFO:tasks.workunit.client.1.smithi132.stdout:4/776: dwrite d4/d19/daa/dc7/f70 [0,4194304] 0 2024-02-19T19:05:02.060 INFO:tasks.workunit.client.1.smithi132.stdout:9/921: write d12/d8d/d3a/d40/d4e/d77/d134/f139 [775584,93773] 0 2024-02-19T19:05:02.066 INFO:tasks.workunit.client.1.smithi132.stdout:3/821: chown d8/d13/d2b/d69/daa/le5 19937544 1 2024-02-19T19:05:02.067 INFO:tasks.workunit.client.1.smithi132.stdout:5/930: dwrite d8/da/d5a/f81 [4194304,4194304] 0 2024-02-19T19:05:02.071 INFO:tasks.workunit.client.1.smithi132.stdout:9/922: symlink d12/d8d/d3a/d40/d4e/d77/d9d/l13c 0 2024-02-19T19:05:02.074 INFO:tasks.workunit.client.1.smithi132.stdout:5/931: getdents d8/da/dae 0 2024-02-19T19:05:02.075 INFO:tasks.workunit.client.1.smithi132.stdout:8/864: dread d4/dc/d1b/d20/d62/d91/f21 [0,4194304] 0 2024-02-19T19:05:02.077 INFO:tasks.workunit.client.1.smithi132.stdout:3/822: dread d8/fc0 [0,4194304] 0 2024-02-19T19:05:02.078 INFO:tasks.workunit.client.1.smithi132.stdout:1/733: dwrite d1/d1e/d77/d2c/d62/d6b/d85/fcf [0,4194304] 0 2024-02-19T19:05:02.080 INFO:tasks.workunit.client.1.smithi132.stdout:0/781: write d2/dc/d4a/d25/d43/d5f/f73 [4184689,60979] 0 2024-02-19T19:05:02.083 INFO:tasks.workunit.client.1.smithi132.stdout:5/932: creat d8/d23/f140 x:0 0 0 2024-02-19T19:05:02.083 INFO:tasks.workunit.client.1.smithi132.stdout:8/865: write d4/dc/d1b/d20/d62/d91/f21 [4141824,115482] 0 2024-02-19T19:05:02.084 INFO:tasks.workunit.client.1.smithi132.stdout:0/782: symlink d2/dc/d4a/d25/dda/l10a 0 2024-02-19T19:05:02.087 INFO:tasks.workunit.client.1.smithi132.stdout:9/923: write d12/d1b/f50 [415497,116561] 0 2024-02-19T19:05:02.090 INFO:tasks.workunit.client.1.smithi132.stdout:1/734: write d1/d3a/d50/f67 [242314,44074] 0 2024-02-19T19:05:02.091 INFO:tasks.workunit.client.1.smithi132.stdout:9/924: write d12/fda [2745869,101077] 0 2024-02-19T19:05:02.091 INFO:tasks.workunit.client.1.smithi132.stdout:0/783: truncate d2/dc/fa3 545960 0 2024-02-19T19:05:02.092 INFO:tasks.workunit.client.1.smithi132.stdout:3/823: link d8/d13/d2f/la8 d8/d13/d2b/d69/daa/l101 0 2024-02-19T19:05:02.094 INFO:tasks.workunit.client.1.smithi132.stdout:5/933: link d8/d2e/f91 d8/d2e/d71/f141 0 2024-02-19T19:05:02.094 INFO:tasks.workunit.client.1.smithi132.stdout:3/824: chown d8/d93/f6f 296787 1 2024-02-19T19:05:02.095 INFO:tasks.workunit.client.1.smithi132.stdout:1/735: stat d1/d1e/d77/d2c/d43/lcb 0 2024-02-19T19:05:02.095 INFO:tasks.workunit.client.1.smithi132.stdout:7/860: sync 2024-02-19T19:05:02.095 INFO:tasks.workunit.client.1.smithi132.stdout:4/777: dread d4/d12/d48/fde [0,4194304] 0 2024-02-19T19:05:02.096 INFO:tasks.workunit.client.1.smithi132.stdout:0/784: rmdir d2/dc/d4a/d74 39 2024-02-19T19:05:02.097 INFO:tasks.workunit.client.1.smithi132.stdout:8/866: dwrite d4/f6 [4194304,4194304] 0 2024-02-19T19:05:02.098 INFO:tasks.workunit.client.1.smithi132.stdout:9/925: write d12/d2a/d4c/d61/fa8 [1553308,1839] 0 2024-02-19T19:05:02.104 INFO:tasks.workunit.client.1.smithi132.stdout:1/736: unlink d1/d1e/d77/d4e/d99/fc2 0 2024-02-19T19:05:02.109 INFO:tasks.workunit.client.1.smithi132.stdout:9/926: creat d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/dc8/f13d x:0 0 0 2024-02-19T19:05:02.109 INFO:tasks.workunit.client.1.smithi132.stdout:3/825: dwrite d8/d13/d2b/d69/da6/fce [0,4194304] 0 2024-02-19T19:05:02.110 INFO:tasks.workunit.client.1.smithi132.stdout:8/867: dwrite d4/f27 [0,4194304] 0 2024-02-19T19:05:02.114 INFO:tasks.workunit.client.1.smithi132.stdout:3/826: chown d8/d13/d2b/d69/d92/l3c 2958 1 2024-02-19T19:05:02.115 INFO:tasks.workunit.client.1.smithi132.stdout:1/737: dread d1/d1e/f74 [0,4194304] 0 2024-02-19T19:05:02.115 INFO:tasks.workunit.client.1.smithi132.stdout:9/927: mkdir d12/d8d/d3a/d40/d4e/d77/d134/d13e 0 2024-02-19T19:05:02.118 INFO:tasks.workunit.client.1.smithi132.stdout:5/934: link d8/da/dae/l10a d8/da/d5a/l142 0 2024-02-19T19:05:02.119 INFO:tasks.workunit.client.1.smithi132.stdout:4/778: getdents d4/d12/d16 0 2024-02-19T19:05:02.120 INFO:tasks.workunit.client.1.smithi132.stdout:0/785: getdents d2/dc/d4a/d74 0 2024-02-19T19:05:02.121 INFO:tasks.workunit.client.1.smithi132.stdout:3/827: mkdir d8/d13/d19/d102 0 2024-02-19T19:05:02.123 INFO:tasks.workunit.client.1.smithi132.stdout:9/928: creat d12/d1b/f13f x:0 0 0 2024-02-19T19:05:02.126 INFO:tasks.workunit.client.1.smithi132.stdout:7/861: truncate d1b/d1e/d28/faa 3968493 0 2024-02-19T19:05:02.127 INFO:tasks.workunit.client.1.smithi132.stdout:4/779: creat d4/d27/d7f/f114 x:0 0 0 2024-02-19T19:05:02.129 INFO:tasks.workunit.client.1.smithi132.stdout:0/786: symlink d2/dc/df/d27/d3a/d52/l10b 0 2024-02-19T19:05:02.130 INFO:tasks.workunit.client.1.smithi132.stdout:9/929: symlink d12/d2a/d88/db8/l140 0 2024-02-19T19:05:02.132 INFO:tasks.workunit.client.1.smithi132.stdout:9/930: fsync d12/fda 0 2024-02-19T19:05:02.132 INFO:tasks.workunit.client.1.smithi132.stdout:3/828: mknod d8/c103 0 2024-02-19T19:05:02.133 INFO:tasks.workunit.client.1.smithi132.stdout:5/935: dwrite d8/da/db/f4f [4194304,4194304] 0 2024-02-19T19:05:02.134 INFO:tasks.workunit.client.1.smithi132.stdout:0/787: unlink d2/dc/d4a/d25/d43/d5f/fb7 0 2024-02-19T19:05:02.134 INFO:tasks.workunit.client.1.smithi132.stdout:8/868: write d4/dc/d1b/d20/d62/d91/f36 [5828914,17500] 0 2024-02-19T19:05:02.138 INFO:tasks.workunit.client.1.smithi132.stdout:3/829: truncate d8/d13/d2b/d60/f95 1849610 0 2024-02-19T19:05:02.138 INFO:tasks.workunit.client.1.smithi132.stdout:1/738: write d1/d1e/f2a [2789761,45142] 0 2024-02-19T19:05:02.141 INFO:tasks.workunit.client.1.smithi132.stdout:4/780: creat d4/d27/d31/d71/d14/de4/f115 x:0 0 0 2024-02-19T19:05:02.141 INFO:tasks.workunit.client.1.smithi132.stdout:1/739: fdatasync d1/d1e/d77/d78/f9d 0 2024-02-19T19:05:02.142 INFO:tasks.workunit.client.1.smithi132.stdout:0/788: symlink d2/dc/df/d15/d2f/d3f/d63/l10c 0 2024-02-19T19:05:02.142 INFO:tasks.workunit.client.1.smithi132.stdout:3/830: write d8/d13/d2b/d69/d92/d34/f89 [1058504,68821] 0 2024-02-19T19:05:02.143 INFO:tasks.workunit.client.1.smithi132.stdout:4/781: stat d4/d27/d7f/fa9 0 2024-02-19T19:05:02.146 INFO:tasks.workunit.client.1.smithi132.stdout:5/936: read - d8/da/d5a/d7b/dfa/d128/df0/f119 zero size 2024-02-19T19:05:02.149 INFO:tasks.workunit.client.1.smithi132.stdout:8/869: dwrite d4/dc/d1b/d20/de5/f116 [0,4194304] 0 2024-02-19T19:05:02.155 INFO:tasks.workunit.client.1.smithi132.stdout:0/789: truncate d2/f6 287644 0 2024-02-19T19:05:02.156 INFO:tasks.workunit.client.1.smithi132.stdout:0/790: truncate d2/f19 4673283 0 2024-02-19T19:05:02.158 INFO:tasks.workunit.client.1.smithi132.stdout:7/862: dwrite f8 [0,4194304] 0 2024-02-19T19:05:02.159 INFO:tasks.workunit.client.1.smithi132.stdout:7/863: dread - d1b/d1e/d41/fbf zero size 2024-02-19T19:05:02.159 INFO:tasks.workunit.client.1.smithi132.stdout:0/791: truncate d2/dc/d4a/d74/fcf 358569 0 2024-02-19T19:05:02.164 INFO:tasks.workunit.client.1.smithi132.stdout:7/864: read d1b/d1e/f21 [1729083,18318] 0 2024-02-19T19:05:02.164 INFO:tasks.workunit.client.1.smithi132.stdout:7/865: dread - d1b/d1e/d28/d5e/fda zero size 2024-02-19T19:05:02.165 INFO:tasks.workunit.client.1.smithi132.stdout:7/866: dread - d1b/d5a/d63/d6b/f105 zero size 2024-02-19T19:05:02.196 INFO:tasks.workunit.client.1.smithi132.stdout:6/756: sync 2024-02-19T19:05:02.196 INFO:tasks.workunit.client.1.smithi132.stdout:2/841: sync 2024-02-19T19:05:02.214 INFO:tasks.workunit.client.1.smithi132.stdout:1/740: mkdir d1/d95/d97/de1 0 2024-02-19T19:05:02.215 INFO:tasks.workunit.client.1.smithi132.stdout:8/870: fdatasync d4/dc/d1b/d38/d126/da3/f12d 0 2024-02-19T19:05:02.215 INFO:tasks.workunit.client.1.smithi132.stdout:9/931: getdents d12/d8d/d3a/ddc 0 2024-02-19T19:05:02.216 INFO:tasks.workunit.client.1.smithi132.stdout:8/871: read d4/dc/d1b/d20/d8a/fff [67695,18584] 0 2024-02-19T19:05:02.218 INFO:tasks.workunit.client.1.smithi132.stdout:9/932: chown d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114/d118/d6e/l127 17368 1 2024-02-19T19:05:02.218 INFO:tasks.workunit.client.1.smithi132.stdout:8/872: dread - d4/dc/d1b/d38/d126/d68/d7a/de8/f124 zero size 2024-02-19T19:05:02.218 INFO:tasks.workunit.client.1.smithi132.stdout:8/873: fdatasync d4/f6 0 2024-02-19T19:05:02.219 INFO:tasks.workunit.client.1.smithi132.stdout:8/874: fsync d4/dc/d1b/d54/f114 0 2024-02-19T19:05:02.220 INFO:tasks.workunit.client.1.smithi132.stdout:8/875: chown d4/dc/d1b/d54/d81/dbd 6273 1 2024-02-19T19:05:02.222 INFO:tasks.workunit.client.1.smithi132.stdout:7/867: mknod d1b/d1e/d28/d5e/c10d 0 2024-02-19T19:05:02.223 INFO:tasks.workunit.client.1.smithi132.stdout:2/842: fsync d0/d4/d11/f15 0 2024-02-19T19:05:02.223 INFO:tasks.workunit.client.1.smithi132.stdout:8/876: fdatasync d4/dc/d1b/d38/d69/d74/d93/f100 0 2024-02-19T19:05:02.225 INFO:tasks.workunit.client.1.smithi132.stdout:0/792: write d2/ff2 [355700,111618] 0 2024-02-19T19:05:02.225 INFO:tasks.workunit.client.1.smithi132.stdout:5/937: rmdir d8/d2e/dc4/dc8/da9/d13c 0 2024-02-19T19:05:02.227 INFO:tasks.workunit.client.1.smithi132.stdout:3/831: dwrite d8/d13/d2b/d69/d92/d8d/fd8 [4194304,4194304] 0 2024-02-19T19:05:02.227 INFO:tasks.workunit.client.1.smithi132.stdout:8/877: chown d4/dc/d1b/d38/d126/fe9 586262 1 2024-02-19T19:05:02.227 INFO:tasks.workunit.client.1.smithi132.stdout:4/782: rename d4/d19/d21/f36 to d4/d27/d31/d71/d37/f116 0 2024-02-19T19:05:02.229 INFO:tasks.workunit.client.1.smithi132.stdout:9/933: write d12/f33 [474651,3793] 0 2024-02-19T19:05:02.235 INFO:tasks.workunit.client.1.smithi132.stdout:2/843: creat d0/d24/d58/d53/dab/d8a/da5/db1/f10b x:0 0 0 2024-02-19T19:05:02.235 INFO:tasks.workunit.client.1.smithi132.stdout:3/832: mkdir d8/dd1/d104 0 2024-02-19T19:05:02.236 INFO:tasks.workunit.client.1.smithi132.stdout:2/844: write d0/d4/d11/d2e/d5a/fd4 [180529,128357] 0 2024-02-19T19:05:02.236 INFO:tasks.workunit.client.1.smithi132.stdout:4/783: mknod d4/d12/d9f/c117 0 2024-02-19T19:05:02.238 INFO:tasks.workunit.client.1.smithi132.stdout:5/938: creat d8/da/d5a/d7b/dfa/d128/d117/f143 x:0 0 0 2024-02-19T19:05:02.239 INFO:tasks.workunit.client.1.smithi132.stdout:8/878: mkdir d4/dc/d1b/d38/d69/d74/d93/d133 0 2024-02-19T19:05:02.239 INFO:tasks.workunit.client.1.smithi132.stdout:4/784: readlink d4/d12/d16/ddb/le7 0 2024-02-19T19:05:02.240 INFO:tasks.workunit.client.1.smithi132.stdout:7/868: rmdir d1b/d32 39 2024-02-19T19:05:02.241 INFO:tasks.workunit.client.1.smithi132.stdout:5/939: fsync d8/d2e/dc4/dc8/da9/f11d 0 2024-02-19T19:05:02.242 INFO:tasks.workunit.client.1.smithi132.stdout:7/869: stat d1b/d5a/d63/d98 0 2024-02-19T19:05:02.242 INFO:tasks.workunit.client.1.smithi132.stdout:0/793: dwrite d2/dc/df/dbd/fd8 [4194304,4194304] 0 2024-02-19T19:05:02.243 INFO:tasks.workunit.client.1.smithi132.stdout:6/757: getdents d0/d51/da7 0 2024-02-19T19:05:02.243 INFO:tasks.workunit.client.1.smithi132.stdout:2/845: mknod d0/d24/d58/d53/c10c 0 2024-02-19T19:05:02.244 INFO:tasks.workunit.client.1.smithi132.stdout:3/833: creat d8/d13/d2b/d60/db8/dfb/f105 x:0 0 0 2024-02-19T19:05:02.247 INFO:tasks.workunit.client.1.smithi132.stdout:8/879: unlink d4/dc/d1b/d38/d126/d68/ld2 0 2024-02-19T19:05:02.251 INFO:tasks.workunit.client.1.smithi132.stdout:3/834: dread - d8/d13/d2b/d69/d92/d8d/db2/fde zero size 2024-02-19T19:05:02.251 INFO:tasks.workunit.client.1.smithi132.stdout:6/758: write d0/db/d25/d3a/d5d/d8f/f39 [107566,54171] 0 2024-02-19T19:05:02.251 INFO:tasks.workunit.client.1.smithi132.stdout:2/846: mknod d0/d19/d1e/c10d 0 2024-02-19T19:05:02.253 INFO:tasks.workunit.client.1.smithi132.stdout:0/794: mknod d2/dc/d4a/d25/d43/d5f/c10d 0 2024-02-19T19:05:02.256 INFO:tasks.workunit.client.1.smithi132.stdout:8/880: creat d4/dc/d1b/d54/d81/dbd/dfb/f134 x:0 0 0 2024-02-19T19:05:02.258 INFO:tasks.workunit.client.1.smithi132.stdout:1/741: dwrite d1/d1e/d77/f41 [0,4194304] 0 2024-02-19T19:05:02.258 INFO:tasks.workunit.client.1.smithi132.stdout:5/940: dread d8/d23/d3b/f70 [0,4194304] 0 2024-02-19T19:05:02.258 INFO:tasks.workunit.client.1.smithi132.stdout:6/759: creat d0/d51/da7/f10a x:0 0 0 2024-02-19T19:05:02.262 INFO:tasks.workunit.client.1.smithi132.stdout:7/870: dwrite d1b/d1e/d41/fe8 [0,4194304] 0 2024-02-19T19:05:02.264 INFO:tasks.workunit.client.1.smithi132.stdout:3/835: write d8/d13/d2b/d69/fd3 [2259205,126968] 0 2024-02-19T19:05:02.264 INFO:tasks.workunit.client.1.smithi132.stdout:4/785: dread d4/d27/d31/d32/d51/d5c/dbc/fbd [0,4194304] 0 2024-02-19T19:05:02.264 INFO:tasks.workunit.client.1.smithi132.stdout:6/760: readlink d0/db/d25/d3a/d5d/d8f/l76 0 2024-02-19T19:05:02.266 INFO:tasks.workunit.client.1.smithi132.stdout:6/761: chown d0/fa 1234 1 2024-02-19T19:05:02.267 INFO:tasks.workunit.client.1.smithi132.stdout:7/871: fdatasync d1b/d1e/d2b/d8f/dae/ff8 0 2024-02-19T19:05:02.268 INFO:tasks.workunit.client.1.smithi132.stdout:3/836: dread - d8/d13/d2b/d69/da6/fdc zero size 2024-02-19T19:05:02.269 INFO:tasks.workunit.client.1.smithi132.stdout:1/742: unlink d1/d1e/d77/c44 0 2024-02-19T19:05:02.271 INFO:tasks.workunit.client.1.smithi132.stdout:4/786: dread d4/d27/d7f/dc0/fcc [0,4194304] 0 2024-02-19T19:05:02.273 INFO:tasks.workunit.client.1.smithi132.stdout:0/795: creat d2/dc/d4a/d58/f10e x:0 0 0 2024-02-19T19:05:02.274 INFO:tasks.workunit.client.1.smithi132.stdout:7/872: symlink d1b/d1e/d2b/d8f/l10e 0 2024-02-19T19:05:02.276 INFO:tasks.workunit.client.1.smithi132.stdout:5/941: creat d8/d1a/d64/f144 x:0 0 0 2024-02-19T19:05:02.277 INFO:tasks.workunit.client.1.smithi132.stdout:4/787: dread - d4/d27/d31/d32/d51/d5c/f101 zero size 2024-02-19T19:05:02.277 INFO:tasks.workunit.client.1.smithi132.stdout:0/796: stat d2/dc/df/d15/d2f/d3f/fee 0 2024-02-19T19:05:02.280 INFO:tasks.workunit.client.1.smithi132.stdout:0/797: stat d2/dc/df/d15/d34/d84/lb8 0 2024-02-19T19:05:02.281 INFO:tasks.workunit.client.1.smithi132.stdout:8/881: dread d4/dc/d1b/d8f/fa7 [0,4194304] 0 2024-02-19T19:05:02.282 INFO:tasks.workunit.client.1.smithi132.stdout:5/942: creat d8/d1a/f145 x:0 0 0 2024-02-19T19:05:02.283 INFO:tasks.workunit.client.1.smithi132.stdout:1/743: dwrite d1/d3a/fac [0,4194304] 0 2024-02-19T19:05:02.283 INFO:tasks.workunit.client.1.smithi132.stdout:0/798: chown d2/dc/d4a/d25/d7f/lc7 47 1 2024-02-19T19:05:02.283 INFO:tasks.workunit.client.1.smithi132.stdout:4/788: creat d4/d27/d31/d32/d51/f118 x:0 0 0 2024-02-19T19:05:02.285 INFO:tasks.workunit.client.1.smithi132.stdout:5/943: chown d8/da/d5a/d7b/dfa/d128/df0/f112 9 1 2024-02-19T19:05:02.286 INFO:tasks.workunit.client.1.smithi132.stdout:8/882: read d4/dc/d1b/d20/fa8 [1023383,39399] 0 2024-02-19T19:05:02.288 INFO:tasks.workunit.client.1.smithi132.stdout:1/744: mkdir d1/d1e/d77/d2c/d43/d81/da1/de2 0 2024-02-19T19:05:02.288 INFO:tasks.workunit.client.1.smithi132.stdout:0/799: creat d2/dc/df/d15/d34/d84/da7/f10f x:0 0 0 2024-02-19T19:05:02.291 INFO:tasks.workunit.client.1.smithi132.stdout:8/883: creat d4/dc/d1b/d8f/f135 x:0 0 0 2024-02-19T19:05:02.292 INFO:tasks.workunit.client.1.smithi132.stdout:0/800: write d2/dc/df/f71 [678569,21088] 0 2024-02-19T19:05:02.293 INFO:tasks.workunit.client.1.smithi132.stdout:4/789: dread d4/d19/daa/dc7/f70 [0,4194304] 0 2024-02-19T19:05:02.296 INFO:tasks.workunit.client.1.smithi132.stdout:5/944: rename d8/d1a/d11f to d8/d2e/dc4/d113/d11a/d139/dea/d12a/d146 0 2024-02-19T19:05:02.296 INFO:tasks.workunit.client.1.smithi132.stdout:3/837: write d8/d38/f3d [94795,22136] 0 2024-02-19T19:05:02.298 INFO:tasks.workunit.client.1.smithi132.stdout:8/884: mknod d4/dc/d1b/d54/d81/dbd/dfb/c136 0 2024-02-19T19:05:02.299 INFO:tasks.workunit.client.1.smithi132.stdout:8/885: write d4/dc/fa9 [798033,12726] 0 2024-02-19T19:05:02.299 INFO:tasks.workunit.client.1.smithi132.stdout:4/790: rmdir d4/d19/d21/d56/dac 39 2024-02-19T19:05:02.300 INFO:tasks.workunit.client.1.smithi132.stdout:3/838: stat d8/d13/d2f/la1 0 2024-02-19T19:05:02.302 INFO:tasks.workunit.client.1.smithi132.stdout:3/839: chown d8/d13/deb/ffa 42601093 1 2024-02-19T19:05:02.302 INFO:tasks.workunit.client.1.smithi132.stdout:4/791: write d4/d12/d48/fde [7627386,116988] 0 2024-02-19T19:05:02.303 INFO:tasks.workunit.client.1.smithi132.stdout:1/745: rename d1/d1e/daa/fd9 to d1/d1e/d77/d2c/d62/d6b/fe3 0 2024-02-19T19:05:02.303 INFO:tasks.workunit.client.1.smithi132.stdout:6/762: dread d0/db/d25/d3a/d5d/d8f/d22/f19 [0,4194304] 0 2024-02-19T19:05:02.304 INFO:tasks.workunit.client.1.smithi132.stdout:8/886: creat d4/dc/d1b/d20/d62/d91/d101/f137 x:0 0 0 2024-02-19T19:05:02.305 INFO:tasks.workunit.client.1.smithi132.stdout:0/801: getdents d2/dc/d4a/d25/d43 0 2024-02-19T19:05:02.307 INFO:tasks.workunit.client.1.smithi132.stdout:3/840: truncate d8/d13/d2b/d60/dc2/ff4 893257 0 2024-02-19T19:05:02.308 INFO:tasks.workunit.client.1.smithi132.stdout:6/763: rename d0/d51/ff4 to d0/dc/f10b 0 2024-02-19T19:05:02.309 INFO:tasks.workunit.client.1.smithi132.stdout:6/764: dread - d0/d8b/f6f zero size 2024-02-19T19:05:02.310 INFO:tasks.workunit.client.1.smithi132.stdout:8/887: dread d4/dc/fa9 [0,4194304] 0 2024-02-19T19:05:02.311 INFO:tasks.workunit.client.1.smithi132.stdout:6/765: fdatasync d0/db/d25/d3a/d5d/d8f/d22/f9 0 2024-02-19T19:05:02.311 INFO:tasks.workunit.client.1.smithi132.stdout:0/802: mkdir d2/dc/df/d15/d2f/d3f/d63/d110 0 2024-02-19T19:05:02.312 INFO:tasks.workunit.client.1.smithi132.stdout:8/888: write d4/dc/d1b/d38/d69/f6d [3819325,8499] 0 2024-02-19T19:05:02.312 INFO:tasks.workunit.client.1.smithi132.stdout:6/766: fsync d0/d8b/f4c 0 2024-02-19T19:05:02.313 INFO:tasks.workunit.client.1.smithi132.stdout:0/803: write d2/dc/df/d15/d2f/d3f/d63/fe1 [3175226,31320] 0 2024-02-19T19:05:02.314 INFO:tasks.workunit.client.1.smithi132.stdout:3/841: creat d8/d13/d2b/d60/f106 x:0 0 0 2024-02-19T19:05:02.317 INFO:tasks.workunit.client.1.smithi132.stdout:4/792: write d4/d19/d21/d56/f7b [1020477,123165] 0 2024-02-19T19:05:02.319 INFO:tasks.workunit.client.1.smithi132.stdout:6/767: dwrite d0/db/d25/d3a/d3d/f61 [0,4194304] 0 2024-02-19T19:05:02.320 INFO:tasks.workunit.client.1.smithi132.stdout:8/889: rename d4/d2d/c48 to d4/dc/d1b/d54/d81/dbd/dfb/c138 0 2024-02-19T19:05:02.321 INFO:tasks.workunit.client.1.smithi132.stdout:8/890: fdatasync d4/d2d/fca 0 2024-02-19T19:05:02.323 INFO:tasks.workunit.client.1.smithi132.stdout:6/768: write d0/db/d25/d3a/d5d/d8f/d46/d5a/daf/fd8 [897933,112360] 0 2024-02-19T19:05:02.324 INFO:tasks.workunit.client.1.smithi132.stdout:1/746: truncate d1/d95/fc7 2358099 0 2024-02-19T19:05:02.324 INFO:tasks.workunit.client.1.smithi132.stdout:8/891: write d4/dc/d1b/d8f/f135 [377441,43665] 0 2024-02-19T19:05:02.325 INFO:tasks.workunit.client.1.smithi132.stdout:3/842: rmdir d8/d38 39 2024-02-19T19:05:02.325 INFO:tasks.workunit.client.1.smithi132.stdout:1/747: chown d1/d1e/d77/d2c 21 1 2024-02-19T19:05:02.327 INFO:tasks.workunit.client.1.smithi132.stdout:4/793: write d4/d27/d7f/dc0/fcc [1129195,25817] 0 2024-02-19T19:05:02.328 INFO:tasks.workunit.client.1.smithi132.stdout:1/748: chown d1/d1e/d77/d2c/d43/lcb 5990 1 2024-02-19T19:05:02.331 INFO:tasks.workunit.client.1.smithi132.stdout:3/843: rename d8/d13/d2b/d69/d92/d8d/db2 to d8/d13/d2b/d69/d92/d107 0 2024-02-19T19:05:02.334 INFO:tasks.workunit.client.1.smithi132.stdout:4/794: creat d4/d19/d21/d56/f119 x:0 0 0 2024-02-19T19:05:02.335 INFO:tasks.workunit.client.1.smithi132.stdout:1/749: rename d1/d1e/d77/d2c/d62/d6b/d85/lb3 to d1/d1e/d77/d52/d64/d75/le4 0 2024-02-19T19:05:02.336 INFO:tasks.workunit.client.1.smithi132.stdout:0/804: dwrite d2/dc/d4a/d58/f95 [0,4194304] 0 2024-02-19T19:05:02.337 INFO:tasks.workunit.client.1.smithi132.stdout:4/795: creat d4/d19/daa/f11a x:0 0 0 2024-02-19T19:05:02.337 INFO:tasks.workunit.client.1.smithi132.stdout:6/769: dread d0/db/f9d [0,4194304] 0 2024-02-19T19:05:02.337 INFO:tasks.workunit.client.1.smithi132.stdout:3/844: rename d8/d38/f4b to d8/d13/d2b/d60/f108 0 2024-02-19T19:05:02.338 INFO:tasks.workunit.client.1.smithi132.stdout:0/805: chown d2/dc/df/d15/d2f/d69/cb6 4023055 1 2024-02-19T19:05:02.340 INFO:tasks.workunit.client.1.smithi132.stdout:1/750: symlink d1/d3a/d50/le5 0 2024-02-19T19:05:02.342 INFO:tasks.workunit.client.1.smithi132.stdout:8/892: write d4/dc/d1b/d38/d69/d88/fba [747744,21772] 0 2024-02-19T19:05:02.342 INFO:tasks.workunit.client.1.smithi132.stdout:0/806: write d2/dc/d4a/f42 [7692554,110449] 0 2024-02-19T19:05:02.342 INFO:tasks.workunit.client.1.smithi132.stdout:6/770: mknod d0/db/d25/d3a/d3d/dd9/c10c 0 2024-02-19T19:05:02.345 INFO:tasks.workunit.client.1.smithi132.stdout:4/796: creat d4/d19/d21/f11b x:0 0 0 2024-02-19T19:05:02.346 INFO:tasks.workunit.client.1.smithi132.stdout:3/845: dread d8/d38/ff5 [0,4194304] 0 2024-02-19T19:05:02.349 INFO:tasks.workunit.client.1.smithi132.stdout:6/771: rename d0/d8b/l95 to d0/db/l10d 0 2024-02-19T19:05:02.350 INFO:tasks.workunit.client.1.smithi132.stdout:1/751: dwrite d1/d1e/d77/d52/fc3 [0,4194304] 0 2024-02-19T19:05:02.352 INFO:tasks.workunit.client.1.smithi132.stdout:4/797: chown d4/d27/d31/d32/d51/d72/f10d 0 1 2024-02-19T19:05:02.352 INFO:tasks.workunit.client.1.smithi132.stdout:3/846: symlink d8/d13/d2b/d69/d92/l109 0 2024-02-19T19:05:02.354 INFO:tasks.workunit.client.1.smithi132.stdout:0/807: getdents d2/dc/d4a/d25/d7f 0 2024-02-19T19:05:02.355 INFO:tasks.workunit.client.1.smithi132.stdout:8/893: rename d4/dc/d1b/d54/f114 to d4/dc/d1b/d38/d126/d68/d79/f139 0 2024-02-19T19:05:02.355 INFO:tasks.workunit.client.1.smithi132.stdout:0/808: chown d2/dc/df/d15/l30 83807100 1 2024-02-19T19:05:02.355 INFO:tasks.workunit.client.1.smithi132.stdout:6/772: creat d0/db/d25/d3a/d3d/d69/de2/f10e x:0 0 0 2024-02-19T19:05:02.358 INFO:tasks.workunit.client.1.smithi132.stdout:1/752: mkdir d1/d1e/d77/d52/d64/de6 0 2024-02-19T19:05:02.361 INFO:tasks.workunit.client.1.smithi132.stdout:3/847: rename d8/l71 to d8/dd1/d104/l10a 0 2024-02-19T19:05:02.362 INFO:tasks.workunit.client.1.smithi132.stdout:6/773: creat d0/dc/f10f x:0 0 0 2024-02-19T19:05:02.368 INFO:tasks.workunit.client.1.smithi132.stdout:3/848: creat d8/d13/d19/d102/f10b x:0 0 0 2024-02-19T19:05:02.369 INFO:tasks.workunit.client.1.smithi132.stdout:6/774: mknod d0/dd1/d103/c110 0 2024-02-19T19:05:02.370 INFO:tasks.workunit.client.1.smithi132.stdout:4/798: getdents d4/d27/d31/d71/d14/d24/d49 0 2024-02-19T19:05:02.371 INFO:tasks.workunit.client.1.smithi132.stdout:3/849: fsync d8/d13/d2b/d69/d92/daf/fb5 0 2024-02-19T19:05:02.373 INFO:tasks.workunit.client.1.smithi132.stdout:6/775: mkdir d0/db/dc9/d111 0 2024-02-19T19:05:02.374 INFO:tasks.workunit.client.1.smithi132.stdout:0/809: dread d2/dc/df/d15/d2f/fca [0,4194304] 0 2024-02-19T19:05:02.376 INFO:tasks.workunit.client.1.smithi132.stdout:4/799: mkdir d4/d27/d31/d32/d51/d11c 0 2024-02-19T19:05:02.376 INFO:tasks.workunit.client.1.smithi132.stdout:1/753: write d1/d1e/f84 [1053903,11101] 0 2024-02-19T19:05:02.378 INFO:tasks.workunit.client.1.smithi132.stdout:4/800: truncate d4/d27/d31/d32/d51/f97 275976 0 2024-02-19T19:05:02.378 INFO:tasks.workunit.client.1.smithi132.stdout:0/810: unlink d2/dc/d4a/d25/d7f/ce2 0 2024-02-19T19:05:02.380 INFO:tasks.workunit.client.1.smithi132.stdout:3/850: rename d8/d13/d19 to d8/d13/d2b/d10c 0 2024-02-19T19:05:02.380 INFO:tasks.workunit.client.1.smithi132.stdout:0/811: chown d2/dc/df/d15/d2f/d47 126044 1 2024-02-19T19:05:02.380 INFO:tasks.workunit.client.1.smithi132.stdout:6/776: mkdir d0/db/d25/d3a/d3d/d69/d89/d112 0 2024-02-19T19:05:02.381 INFO:tasks.workunit.client.1.smithi132.stdout:1/754: symlink d1/d1e/d77/d4e/le7 0 2024-02-19T19:05:02.381 INFO:tasks.workunit.client.1.smithi132.stdout:1/755: fdatasync d1/d1e/f2a 0 2024-02-19T19:05:02.382 INFO:tasks.workunit.client.1.smithi132.stdout:4/801: truncate d4/f38 2271635 0 2024-02-19T19:05:02.382 INFO:tasks.workunit.client.1.smithi132.stdout:3/851: chown d8/d13/d2b/d60/dc2 1 1 2024-02-19T19:05:02.383 INFO:tasks.workunit.client.1.smithi132.stdout:4/802: chown d4/d19/daa/cda 6618948 1 2024-02-19T19:05:02.383 INFO:tasks.workunit.client.1.smithi132.stdout:0/812: creat d2/dc/df/d27/d3a/d52/da8/f111 x:0 0 0 2024-02-19T19:05:02.385 INFO:tasks.workunit.client.1.smithi132.stdout:6/777: link d0/db/d25/d3a/d5d/d8f/d46/d5a/cd4 d0/db/d25/d3a/df3/c113 0 2024-02-19T19:05:02.388 INFO:tasks.workunit.client.1.smithi132.stdout:6/778: creat d0/db/dc9/f114 x:0 0 0 2024-02-19T19:05:02.388 INFO:tasks.workunit.client.1.smithi132.stdout:1/756: link d1/d1e/d77/l86 d1/d1e/d77/d4e/d99/le8 0 2024-02-19T19:05:02.390 INFO:tasks.workunit.client.1.smithi132.stdout:0/813: rename d2/dc/d4a/d74/fb5 to d2/dc/df/f112 0 2024-02-19T19:05:02.390 INFO:tasks.workunit.client.1.smithi132.stdout:4/803: dwrite d4/d27/d31/d71/d14/de4/f115 [0,4194304] 0 2024-02-19T19:05:02.394 INFO:tasks.workunit.client.1.smithi132.stdout:0/814: rmdir d2 39 2024-02-19T19:05:02.397 INFO:tasks.workunit.client.1.smithi132.stdout:6/779: dwrite d0/db/d25/d38/f68 [0,4194304] 0 2024-02-19T19:05:02.403 INFO:tasks.workunit.client.1.smithi132.stdout:1/757: rename d1/d1e/d77/d2c/d43/f54 to d1/fe9 0 2024-02-19T19:05:02.404 INFO:tasks.workunit.client.1.smithi132.stdout:3/852: truncate d8/d38/d44/f73 2447378 0 2024-02-19T19:05:02.404 INFO:tasks.workunit.client.1.smithi132.stdout:4/804: dwrite d4/f2d [4194304,4194304] 0 2024-02-19T19:05:02.409 INFO:tasks.workunit.client.1.smithi132.stdout:4/805: fdatasync d4/d4d/d87/fd5 0 2024-02-19T19:05:02.409 INFO:tasks.workunit.client.1.smithi132.stdout:1/758: dread d1/d1e/f74 [0,4194304] 0 2024-02-19T19:05:02.409 INFO:tasks.workunit.client.1.smithi132.stdout:3/853: readlink d8/d13/d2b/d69/da6/l5f 0 2024-02-19T19:05:02.410 INFO:tasks.workunit.client.1.smithi132.stdout:0/815: unlink d2/dc/df/d15/d2f/d3f/d63/l77 0 2024-02-19T19:05:02.410 INFO:tasks.workunit.client.1.smithi132.stdout:1/759: stat d1/d1e/d77/f69 0 2024-02-19T19:05:02.411 INFO:tasks.workunit.client.1.smithi132.stdout:3/854: truncate d8/d13/f37 2005661 0 2024-02-19T19:05:02.413 INFO:tasks.workunit.client.1.smithi132.stdout:0/816: write d2/dc/df/d15/d34/d84/da7/fb3 [4700108,119512] 0 2024-02-19T19:05:02.416 INFO:tasks.workunit.client.1.smithi132.stdout:3/855: creat d8/d38/d44/d6e/dcc/f10d x:0 0 0 2024-02-19T19:05:02.417 INFO:tasks.workunit.client.1.smithi132.stdout:0/817: creat d2/dc/df/f113 x:0 0 0 2024-02-19T19:05:02.418 INFO:tasks.workunit.client.1.smithi132.stdout:1/760: dwrite d1/f1d [0,4194304] 0 2024-02-19T19:05:02.418 INFO:tasks.workunit.client.1.smithi132.stdout:3/856: fdatasync d8/d13/d2b/f99 0 2024-02-19T19:05:02.420 INFO:tasks.workunit.client.1.smithi132.stdout:3/857: write d8/d13/d2b/d60/db8/dfb/f105 [487344,39584] 0 2024-02-19T19:05:02.421 INFO:tasks.workunit.client.1.smithi132.stdout:1/761: stat d1/d1e/f5a 0 2024-02-19T19:05:02.424 INFO:tasks.workunit.client.1.smithi132.stdout:0/818: dread d2/dc/df/d15/d2f/fca [0,4194304] 0 2024-02-19T19:05:02.424 INFO:tasks.workunit.client.1.smithi132.stdout:3/858: symlink d8/d13/deb/l10e 0 2024-02-19T19:05:02.425 INFO:tasks.workunit.client.1.smithi132.stdout:6/780: write d0/d8b/f71 [529789,94729] 0 2024-02-19T19:05:02.425 INFO:tasks.workunit.client.1.smithi132.stdout:0/819: chown d2/dc/df/d15/d34 91453100 1 2024-02-19T19:05:02.430 INFO:tasks.workunit.client.1.smithi132.stdout:3/859: creat d8/d13/d2b/d69/d92/daf/f10f x:0 0 0 2024-02-19T19:05:02.431 INFO:tasks.workunit.client.1.smithi132.stdout:1/762: fdatasync d1/d1e/d77/d78/f7f 0 2024-02-19T19:05:02.432 INFO:tasks.workunit.client.1.smithi132.stdout:6/781: symlink d0/db/d25/d3a/d5d/l115 0 2024-02-19T19:05:02.433 INFO:tasks.workunit.client.1.smithi132.stdout:3/860: truncate d8/d13/deb/ffa 1544758 0 2024-02-19T19:05:02.434 INFO:tasks.workunit.client.1.smithi132.stdout:1/763: fdatasync d1/fdc 0 2024-02-19T19:05:02.435 INFO:tasks.workunit.client.1.smithi132.stdout:0/820: creat d2/f114 x:0 0 0 2024-02-19T19:05:02.435 INFO:tasks.workunit.client.1.smithi132.stdout:6/782: creat d0/db/d25/d3a/df3/f116 x:0 0 0 2024-02-19T19:05:02.436 INFO:tasks.workunit.client.1.smithi132.stdout:1/764: rmdir d1/d1e/d77/d52/d64 39 2024-02-19T19:05:02.438 INFO:tasks.workunit.client.1.smithi132.stdout:3/861: rename d8/d13/d2b/d69/d92/d34/fa3 to d8/d13/d2b/d69/d7e/dda/f110 0 2024-02-19T19:05:02.440 INFO:tasks.workunit.client.1.smithi132.stdout:4/806: dread d4/d27/d73/fa6 [0,4194304] 0 2024-02-19T19:05:02.441 INFO:tasks.workunit.client.1.smithi132.stdout:6/783: dread d0/db/d25/d3a/d5d/d8f/d46/d5a/f7d [0,4194304] 0 2024-02-19T19:05:02.444 INFO:tasks.workunit.client.1.smithi132.stdout:6/784: chown d0/c1b 1579 1 2024-02-19T19:05:02.451 INFO:tasks.workunit.client.1.smithi132.stdout:1/765: dwrite d1/d3a/f40 [0,4194304] 0 2024-02-19T19:05:02.453 INFO:tasks.workunit.client.1.smithi132.stdout:4/807: dwrite d4/d27/d31/d32/fb6 [0,4194304] 0 2024-02-19T19:05:02.453 INFO:tasks.workunit.client.1.smithi132.stdout:6/785: unlink d0/db/d25/d3a/d3d/d69/d89/c8e 0 2024-02-19T19:05:02.454 INFO:tasks.workunit.client.1.smithi132.stdout:1/766: read d1/d1e/f74 [1184872,22198] 0 2024-02-19T19:05:02.455 INFO:tasks.workunit.client.1.smithi132.stdout:4/808: readlink d4/d19/d21/d56/l5e 0 2024-02-19T19:05:02.456 INFO:tasks.workunit.client.1.smithi132.stdout:0/821: dread d2/dc/d4a/f42 [0,4194304] 0 2024-02-19T19:05:02.458 INFO:tasks.workunit.client.1.smithi132.stdout:0/822: chown d2/d3e/db1/ld4 17280 1 2024-02-19T19:05:02.459 INFO:tasks.workunit.client.1.smithi132.stdout:3/862: dread d8/d13/d2b/d69/d92/d34/f59 [0,4194304] 0 2024-02-19T19:05:02.462 INFO:tasks.workunit.client.1.smithi132.stdout:0/823: fdatasync d2/dc/df/d27/f35 0 2024-02-19T19:05:02.467 INFO:tasks.workunit.client.1.smithi132.stdout:3/863: mknod d8/d13/d2b/d69/d92/d8d/dd7/c111 0 2024-02-19T19:05:02.467 INFO:tasks.workunit.client.1.smithi132.stdout:4/809: dwrite d4/d12/f4b [0,4194304] 0 2024-02-19T19:05:02.473 INFO:tasks.workunit.client.1.smithi132.stdout:3/864: dread - d8/d13/d2b/d60/fe1 zero size 2024-02-19T19:05:02.476 INFO:tasks.workunit.client.1.smithi132.stdout:0/824: dwrite d2/dc/d4a/d25/d43/d5f/f104 [0,4194304] 0 2024-02-19T19:05:02.486 INFO:tasks.workunit.client.1.smithi132.stdout:4/810: mkdir d4/d27/d31/d71/d14/d24/d49/d10e/d11d 0 2024-02-19T19:05:02.487 INFO:tasks.workunit.client.1.smithi132.stdout:3/865: chown d8/dd1/d104/l10a 2 1 2024-02-19T19:05:02.489 INFO:tasks.workunit.client.1.smithi132.stdout:0/825: mknod d2/dc/d4a/d25/dc9/c115 0 2024-02-19T19:05:02.491 INFO:tasks.workunit.client.1.smithi132.stdout:1/767: dread d1/d1e/d77/d52/f65 [0,4194304] 0 2024-02-19T19:05:02.492 INFO:tasks.workunit.client.1.smithi132.stdout:4/811: symlink d4/d27/d31/l11e 0 2024-02-19T19:05:02.493 INFO:tasks.workunit.client.1.smithi132.stdout:0/826: creat d2/dc/df/d15/d2f/f116 x:0 0 0 2024-02-19T19:05:02.494 INFO:tasks.workunit.client.1.smithi132.stdout:4/812: rmdir d4/d12/da4 39 2024-02-19T19:05:02.496 INFO:tasks.workunit.client.1.smithi132.stdout:4/813: chown d4/d27/d31/d71/d14/d24/d49/ce9 0 1 2024-02-19T19:05:02.497 INFO:tasks.workunit.client.1.smithi132.stdout:0/827: mknod d2/dc/d98/c117 0 2024-02-19T19:05:02.499 INFO:tasks.workunit.client.1.smithi132.stdout:4/814: rename d4/d19/d21/d56/dac to d4/d19/d21/dfa/d11f 0 2024-02-19T19:05:02.502 INFO:tasks.workunit.client.1.smithi132.stdout:4/815: mknod d4/d27/d31/c120 0 2024-02-19T19:05:02.502 INFO:tasks.workunit.client.1.smithi132.stdout:6/786: write d0/db/d25/d3a/d5d/d8f/d87/fca [305983,69794] 0 2024-02-19T19:05:02.503 INFO:tasks.workunit.client.1.smithi132.stdout:9/934: sync 2024-02-19T19:05:02.504 INFO:tasks.workunit.client.1.smithi132.stdout:6/787: chown d0/db/d25/d3a/d5d/d8f/d22/f9c 26849 1 2024-02-19T19:05:02.507 INFO:tasks.workunit.client.1.smithi132.stdout:1/768: write d1/f4a [259404,56403] 0 2024-02-19T19:05:02.509 INFO:tasks.workunit.client.1.smithi132.stdout:4/816: write d4/d27/d31/d32/fb6 [4399314,34437] 0 2024-02-19T19:05:02.511 INFO:tasks.workunit.client.1.smithi132.stdout:1/769: mkdir d1/d95/d97/de1/dea 0 2024-02-19T19:05:02.512 INFO:tasks.workunit.client.1.smithi132.stdout:4/817: readlink d4/d19/d21/le5 0 2024-02-19T19:05:02.513 INFO:tasks.workunit.client.1.smithi132.stdout:6/788: dwrite d0/f10 [0,4194304] 0 2024-02-19T19:05:02.515 INFO:tasks.workunit.client.1.smithi132.stdout:0/828: dread d2/f1a [0,4194304] 0 2024-02-19T19:05:02.515 INFO:tasks.workunit.client.1.smithi132.stdout:6/789: stat d0/db/d25/d3a/d3d/d69/d89 0 2024-02-19T19:05:02.517 INFO:tasks.workunit.client.1.smithi132.stdout:4/818: creat d4/d27/d31/d32/d51/d5c/dbc/f121 x:0 0 0 2024-02-19T19:05:02.519 INFO:tasks.workunit.client.1.smithi132.stdout:0/829: truncate d2/dc/df/d27/d3a/d52/fcb 834830 0 2024-02-19T19:05:02.520 INFO:tasks.workunit.client.1.smithi132.stdout:4/819: rmdir d4/d12/d48/d5f 39 2024-02-19T19:05:02.522 INFO:tasks.workunit.client.1.smithi132.stdout:9/935: dread d12/d8d/f76 [0,4194304] 0 2024-02-19T19:05:02.522 INFO:tasks.workunit.client.1.smithi132.stdout:0/830: mkdir d2/dc/d4a/d25/d7f/d118 0 2024-02-19T19:05:02.524 INFO:tasks.workunit.client.1.smithi132.stdout:6/790: rename d0/db/d25/d3a/d5d/d8f/d22/f59 to d0/db/d25/d3a/f117 0 2024-02-19T19:05:02.525 INFO:tasks.workunit.client.1.smithi132.stdout:9/936: stat d12/d2a/d88/l91 0 2024-02-19T19:05:02.526 INFO:tasks.workunit.client.1.smithi132.stdout:0/831: mkdir d2/dc/df/d15/d2f/d3f/d119 0 2024-02-19T19:05:02.528 INFO:tasks.workunit.client.1.smithi132.stdout:0/832: rmdir d2/dc/df/d15/d2f/d47 39 2024-02-19T19:05:02.529 INFO:tasks.workunit.client.1.smithi132.stdout:4/820: dread d4/d27/d31/d32/d51/d72/f10d [0,4194304] 0 2024-02-19T19:05:02.530 INFO:tasks.workunit.client.1.smithi132.stdout:9/937: unlink d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114/d118/d6e/fac 0 2024-02-19T19:05:02.530 INFO:tasks.workunit.client.1.smithi132.stdout:4/821: chown d4/d4d/d87/f9a 1977121931 1 2024-02-19T19:05:02.531 INFO:tasks.workunit.client.1.smithi132.stdout:6/791: dwrite d0/d8b/f4a [0,4194304] 0 2024-02-19T19:05:02.532 INFO:tasks.workunit.client.1.smithi132.stdout:6/792: readlink d0/db/d25/d3a/d4b/lb4 0 2024-02-19T19:05:02.535 INFO:tasks.workunit.client.1.smithi132.stdout:9/938: symlink d12/d8d/d3a/d40/d4e/d77/d8a/d82/dd9/l141 0 2024-02-19T19:05:02.535 INFO:tasks.workunit.client.1.smithi132.stdout:6/793: write d0/db/d25/d3a/d5d/d8f/d46/d5a/daf/fd8 [104107,71545] 0 2024-02-19T19:05:02.536 INFO:tasks.workunit.client.1.smithi132.stdout:4/822: creat d4/d19/daa/dc7/d98/f122 x:0 0 0 2024-02-19T19:05:02.539 INFO:tasks.workunit.client.1.smithi132.stdout:6/794: rmdir d0/db/d25/d50/d86/dc3 39 2024-02-19T19:05:02.541 INFO:tasks.workunit.client.1.smithi132.stdout:0/833: write d2/dc/df/d27/f61 [1230707,83481] 0 2024-02-19T19:05:02.541 INFO:tasks.workunit.client.1.smithi132.stdout:9/939: mkdir d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114/d142 0 2024-02-19T19:05:02.543 INFO:tasks.workunit.client.1.smithi132.stdout:6/795: creat d0/db/d25/d38/f118 x:0 0 0 2024-02-19T19:05:02.545 INFO:tasks.workunit.client.1.smithi132.stdout:4/823: rename d4/d19/c1e to d4/d19/c123 0 2024-02-19T19:05:02.546 INFO:tasks.workunit.client.1.smithi132.stdout:6/796: write d0/db/d25/d3a/d4b/d7c/fbd [522380,97273] 0 2024-02-19T19:05:02.548 INFO:tasks.workunit.client.1.smithi132.stdout:6/797: fsync d0/fad 0 2024-02-19T19:05:02.549 INFO:tasks.workunit.client.1.smithi132.stdout:9/940: dwrite d12/d8d/d3a/ff0 [0,4194304] 0 2024-02-19T19:05:02.551 INFO:tasks.workunit.client.1.smithi132.stdout:9/941: write d12/d2a/f5b [4553555,26496] 0 2024-02-19T19:05:02.556 INFO:tasks.workunit.client.1.smithi132.stdout:9/942: unlink d12/d2a/d88/db8/f108 0 2024-02-19T19:05:02.559 INFO:tasks.workunit.client.1.smithi132.stdout:6/798: dwrite d0/db/d25/d3a/d3d/d69/f7f [0,4194304] 0 2024-02-19T19:05:02.561 INFO:tasks.workunit.client.1.smithi132.stdout:4/824: dread d4/d12/f2c [0,4194304] 0 2024-02-19T19:05:02.562 INFO:tasks.workunit.client.1.smithi132.stdout:0/834: dread d2/dc/df/d15/d2f/d69/faf [0,4194304] 0 2024-02-19T19:05:02.562 INFO:tasks.workunit.client.1.smithi132.stdout:6/799: dread d0/d8b/f71 [0,4194304] 0 2024-02-19T19:05:02.563 INFO:tasks.workunit.client.1.smithi132.stdout:9/943: link d12/d2a/d4c/lcb d12/d8d/d3a/ddc/l143 0 2024-02-19T19:05:02.588 INFO:tasks.workunit.client.1.smithi132.stdout:9/944: dread d12/d8d/f30 [0,4194304] 0 2024-02-19T19:05:02.598 INFO:tasks.workunit.client.1.smithi132.stdout:9/945: dwrite d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114/d118/d6e/f79 [0,4194304] 0 2024-02-19T19:05:02.603 INFO:tasks.workunit.client.1.smithi132.stdout:9/946: symlink d12/d8d/d3a/d40/d4e/d77/d89/l144 0 2024-02-19T19:05:02.605 INFO:tasks.workunit.client.1.smithi132.stdout:9/947: truncate d12/d8d/d3a/d40/d4e/fd6 501071 0 2024-02-19T19:05:02.607 INFO:tasks.workunit.client.1.smithi132.stdout:9/948: mkdir d12/d8d/d3a/d40/d4e/d77/d134/d145 0 2024-02-19T19:05:02.609 INFO:tasks.workunit.client.1.smithi132.stdout:9/949: mkdir d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114/d118/d6e/d146 0 2024-02-19T19:05:02.615 INFO:tasks.workunit.client.1.smithi132.stdout:9/950: dwrite d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/dc8/f121 [0,4194304] 0 2024-02-19T19:05:02.617 INFO:tasks.workunit.client.1.smithi132.stdout:9/951: rmdir d12/d8d/d3a/d40/d4e 39 2024-02-19T19:05:02.625 INFO:tasks.workunit.client.1.smithi132.stdout:9/952: truncate d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114/d118/fd7 3825236 0 2024-02-19T19:05:02.627 INFO:tasks.workunit.client.1.smithi132.stdout:9/953: symlink d12/d8d/d3a/d40/d4e/d77/d134/d13e/l147 0 2024-02-19T19:05:02.645 INFO:tasks.workunit.client.1.smithi132.stdout:9/954: dread d12/d2a/f2b [0,4194304] 0 2024-02-19T19:05:02.646 INFO:tasks.workunit.client.1.smithi132.stdout:9/955: write d12/d2a/d4c/d61/f138 [461411,86206] 0 2024-02-19T19:05:02.648 INFO:tasks.workunit.client.1.smithi132.stdout:9/956: chown d12/d2a/d88/cd8 1880290828 1 2024-02-19T19:05:02.650 INFO:tasks.workunit.client.1.smithi132.stdout:9/957: symlink d12/d8d/d3a/d40/d4e/d77/d8a/l148 0 2024-02-19T19:05:02.651 INFO:tasks.workunit.client.1.smithi132.stdout:9/958: chown d12/d1b/c15 506464 1 2024-02-19T19:05:02.656 INFO:tasks.workunit.client.1.smithi132.stdout:2/847: sync 2024-02-19T19:05:02.657 INFO:tasks.workunit.client.1.smithi132.stdout:9/959: truncate d12/d2a/d4c/f9b 4446289 0 2024-02-19T19:05:02.657 INFO:tasks.workunit.client.1.smithi132.stdout:2/848: symlink d0/d24/d58/d53/dab/d8a/l10e 0 2024-02-19T19:05:02.658 INFO:tasks.workunit.client.1.smithi132.stdout:9/960: fdatasync d12/d8d/d3a/d40/d4e/d77/d89/dc4/ff8 0 2024-02-19T19:05:02.662 INFO:tasks.workunit.client.1.smithi132.stdout:2/849: dread d0/d4/df/d40/d7b/fe2 [0,4194304] 0 2024-02-19T19:05:02.669 INFO:tasks.workunit.client.1.smithi132.stdout:9/961: dread d12/d8d/d3a/d40/d4e/d77/d89/fcf [0,4194304] 0 2024-02-19T19:05:02.672 INFO:tasks.workunit.client.1.smithi132.stdout:9/962: rmdir d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114 39 2024-02-19T19:05:02.674 INFO:tasks.workunit.client.1.smithi132.stdout:2/850: dwrite d0/d4/d11/f20 [0,4194304] 0 2024-02-19T19:05:02.676 INFO:tasks.workunit.client.1.smithi132.stdout:9/963: creat d12/d8d/d3a/d40/d4e/d77/d89/dc4/f149 x:0 0 0 2024-02-19T19:05:02.678 INFO:tasks.workunit.client.1.smithi132.stdout:9/964: chown d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136 1918 1 2024-02-19T19:05:02.679 INFO:tasks.workunit.client.1.smithi132.stdout:2/851: rename d0/d4/df/dc2/dad/le3 to d0/d24/d58/d53/dab/d84/de1/l10f 0 2024-02-19T19:05:02.776 INFO:tasks.workunit.client.1.smithi132.stdout:7/873: sync 2024-02-19T19:05:02.779 INFO:tasks.workunit.client.1.smithi132.stdout:3/866: sync 2024-02-19T19:05:02.779 INFO:tasks.workunit.client.1.smithi132.stdout:7/874: creat d1b/d1e/d2b/f10f x:0 0 0 2024-02-19T19:05:02.790 INFO:tasks.workunit.client.1.smithi132.stdout:3/867: dread d8/d38/d44/f5e [4194304,4194304] 0 2024-02-19T19:05:02.800 INFO:tasks.workunit.client.1.smithi132.stdout:7/875: dwrite d1b/d1e/d41/da3/de3/ff2 [0,4194304] 0 2024-02-19T19:05:02.802 INFO:tasks.workunit.client.1.smithi132.stdout:3/868: dread d8/d38/d44/fcf [0,4194304] 0 2024-02-19T19:05:02.804 INFO:tasks.workunit.client.1.smithi132.stdout:3/869: mkdir d8/d13/d2b/d60/db8/dfb/d112 0 2024-02-19T19:05:02.806 INFO:tasks.workunit.client.1.smithi132.stdout:7/876: creat d1b/d46/f110 x:0 0 0 2024-02-19T19:05:02.810 INFO:tasks.workunit.client.1.smithi132.stdout:7/877: rename d1b/d46/d56 to d1b/d1e/d28/dd2/d111 0 2024-02-19T19:05:02.810 INFO:tasks.workunit.client.1.smithi132.stdout:3/870: dwrite d8/d13/d2b/d69/d92/d34/f89 [4194304,4194304] 0 2024-02-19T19:05:02.811 INFO:tasks.workunit.client.1.smithi132.stdout:3/871: truncate d8/fc0 4877266 0 2024-02-19T19:05:02.814 INFO:tasks.workunit.client.1.smithi132.stdout:7/878: symlink d1b/d1e/d2b/d62/d53/l112 0 2024-02-19T19:05:02.819 INFO:tasks.workunit.client.1.smithi132.stdout:3/872: dwrite d8/d38/f8b [0,4194304] 0 2024-02-19T19:05:02.821 INFO:tasks.workunit.client.1.smithi132.stdout:7/879: link d1b/d1e/d2b/d45/c74 d1b/d1e/d28/d5e/d99/c113 0 2024-02-19T19:05:02.823 INFO:tasks.workunit.client.1.smithi132.stdout:3/873: truncate d8/d13/d2f/f66 59663 0 2024-02-19T19:05:02.824 INFO:tasks.workunit.client.1.smithi132.stdout:3/874: fsync d8/d13/d2b/d69/d92/daf/ff2 0 2024-02-19T19:05:02.828 INFO:tasks.workunit.client.1.smithi132.stdout:3/875: mknod d8/d38/d44/d6e/dcc/c113 0 2024-02-19T19:05:02.830 INFO:tasks.workunit.client.1.smithi132.stdout:3/876: creat d8/d13/d2b/d10c/f114 x:0 0 0 2024-02-19T19:05:02.834 INFO:tasks.workunit.client.1.smithi132.stdout:3/877: symlink d8/d13/d2b/d69/d92/d34/d42/l115 0 2024-02-19T19:05:02.835 INFO:tasks.workunit.client.1.smithi132.stdout:7/880: truncate d1b/d1e/d28/f4c 984227 0 2024-02-19T19:05:02.838 INFO:tasks.workunit.client.1.smithi132.stdout:7/881: unlink d1b/d1e/d2b/d62/l64 0 2024-02-19T19:05:02.838 INFO:tasks.workunit.client.1.smithi132.stdout:7/882: stat l18 0 2024-02-19T19:05:02.838 INFO:tasks.workunit.client.1.smithi132.stdout:3/878: read d8/d13/d2b/d10c/f49 [2028036,77514] 0 2024-02-19T19:05:02.839 INFO:tasks.workunit.client.1.smithi132.stdout:8/894: sync 2024-02-19T19:05:02.839 INFO:tasks.workunit.client.1.smithi132.stdout:5/945: sync 2024-02-19T19:05:02.840 INFO:tasks.workunit.client.1.smithi132.stdout:7/883: creat d1b/d1e/d2b/d8f/dae/f114 x:0 0 0 2024-02-19T19:05:02.841 INFO:tasks.workunit.client.1.smithi132.stdout:3/879: symlink d8/d13/d2b/d10c/d102/l116 0 2024-02-19T19:05:02.841 INFO:tasks.workunit.client.1.smithi132.stdout:8/895: creat d4/dc/d42/f13a x:0 0 0 2024-02-19T19:05:02.842 INFO:tasks.workunit.client.1.smithi132.stdout:8/896: fsync d4/dc/d1b/d20/f53 0 2024-02-19T19:05:02.843 INFO:tasks.workunit.client.1.smithi132.stdout:7/884: read d1b/d1e/f30 [2517789,43901] 0 2024-02-19T19:05:02.849 INFO:tasks.workunit.client.1.smithi132.stdout:5/946: dwrite d8/d2e/dc4/dc8/daa/fe5 [0,4194304] 0 2024-02-19T19:05:02.851 INFO:tasks.workunit.client.1.smithi132.stdout:5/947: fsync d8/d2e/dc4/dc8/daa/f11e 0 2024-02-19T19:05:02.855 INFO:tasks.workunit.client.1.smithi132.stdout:3/880: truncate d8/d13/d2b/d69/d92/daf/fc3 2348336 0 2024-02-19T19:05:02.855 INFO:tasks.workunit.client.1.smithi132.stdout:8/897: dwrite d4/d2d/dcc/ff6 [4194304,4194304] 0 2024-02-19T19:05:02.864 INFO:tasks.workunit.client.1.smithi132.stdout:7/885: dread d1b/d1e/d2b/f2f [0,4194304] 0 2024-02-19T19:05:02.865 INFO:tasks.workunit.client.1.smithi132.stdout:8/898: dread d4/dc/d1b/d20/f59 [0,4194304] 0 2024-02-19T19:05:02.867 INFO:tasks.workunit.client.1.smithi132.stdout:8/899: mknod d4/dc/d1b/d8f/c13b 0 2024-02-19T19:05:02.867 INFO:tasks.workunit.client.1.smithi132.stdout:8/900: readlink d4/dc/d1b/d20/d62/d91/l7c 0 2024-02-19T19:05:02.868 INFO:tasks.workunit.client.1.smithi132.stdout:7/886: mkdir d1b/d1e/d41/da3/d115 0 2024-02-19T19:05:02.869 INFO:tasks.workunit.client.1.smithi132.stdout:8/901: creat d4/d2d/dcc/d129/f13c x:0 0 0 2024-02-19T19:05:02.870 INFO:tasks.workunit.client.1.smithi132.stdout:8/902: chown d4/dc/d1b/d54/d81/dbd/dfb/f134 2151 1 2024-02-19T19:05:02.872 INFO:tasks.workunit.client.1.smithi132.stdout:8/903: creat d4/dc/d1b/d38/d69/d74/d93/f13d x:0 0 0 2024-02-19T19:05:02.873 INFO:tasks.workunit.client.1.smithi132.stdout:8/904: rmdir d4 39 2024-02-19T19:05:02.875 INFO:tasks.workunit.client.1.smithi132.stdout:8/905: chown d4/dc/d1b/d20/d62/d91/f21 390 1 2024-02-19T19:05:02.876 INFO:tasks.workunit.client.1.smithi132.stdout:8/906: dread - d4/dc/d1b/d54/f9c zero size 2024-02-19T19:05:02.879 INFO:tasks.workunit.client.1.smithi132.stdout:8/907: dread d4/dc/d1b/d38/d69/d88/fba [0,4194304] 0 2024-02-19T19:05:02.880 INFO:tasks.workunit.client.1.smithi132.stdout:8/908: stat d4/dc/d1b/d38/d126/d68/d7a/de8/f124 0 2024-02-19T19:05:02.882 INFO:tasks.workunit.client.1.smithi132.stdout:8/909: unlink d4/dc/d1b/d54/d5e/c4f 0 2024-02-19T19:05:02.883 INFO:tasks.workunit.client.1.smithi132.stdout:8/910: dread - d4/dc/d1b/fd3 zero size 2024-02-19T19:05:02.883 INFO:tasks.workunit.client.1.smithi132.stdout:8/911: dread - d4/d2d/dcc/d129/f13c zero size 2024-02-19T19:05:02.885 INFO:tasks.workunit.client.1.smithi132.stdout:8/912: truncate d4/dc/d1b/d38/d126/f2b 5736353 0 2024-02-19T19:05:02.886 INFO:tasks.workunit.client.1.smithi132.stdout:8/913: fsync d4/dc/d1b/d20/f6e 0 2024-02-19T19:05:02.887 INFO:tasks.workunit.client.1.smithi132.stdout:8/914: stat d4/dc/d42/f56 0 2024-02-19T19:05:02.889 INFO:tasks.workunit.client.1.smithi132.stdout:8/915: truncate d4/dc/d1b/fd3 980524 0 2024-02-19T19:05:02.892 INFO:tasks.workunit.client.1.smithi132.stdout:8/916: mknod d4/dc/d1b/d20/d8a/c13e 0 2024-02-19T19:05:02.893 INFO:tasks.workunit.client.1.smithi132.stdout:3/881: write d8/d13/d2f/f66 [517590,56101] 0 2024-02-19T19:05:02.894 INFO:tasks.workunit.client.1.smithi132.stdout:3/882: chown d8/d13/d2b/d60 523579 1 2024-02-19T19:05:02.895 INFO:tasks.workunit.client.1.smithi132.stdout:3/883: readlink d8/d13/d2b/d69/d92/l9c 0 2024-02-19T19:05:02.897 INFO:tasks.workunit.client.1.smithi132.stdout:3/884: chown d8/d13/d2b/d69/d92/d34/d97 74028144 1 2024-02-19T19:05:02.899 INFO:tasks.workunit.client.1.smithi132.stdout:8/917: dwrite d4/dac/fae [0,4194304] 0 2024-02-19T19:05:02.899 INFO:tasks.workunit.client.1.smithi132.stdout:7/887: dwrite d1b/d1e/f37 [0,4194304] 0 2024-02-19T19:05:02.900 INFO:tasks.workunit.client.1.smithi132.stdout:8/918: read - d4/dc/d1b/d54/d81/dbd/dfb/f134 zero size 2024-02-19T19:05:02.905 INFO:tasks.workunit.client.1.smithi132.stdout:8/919: rmdir d4/dc/d1b/d38/d126/d127 0 2024-02-19T19:05:02.936 INFO:tasks.workunit.client.1.smithi132.stdout:1/770: sync 2024-02-19T19:05:02.936 INFO:tasks.workunit.client.1.smithi132.stdout:4/825: sync 2024-02-19T19:05:02.937 INFO:tasks.workunit.client.1.smithi132.stdout:9/965: sync 2024-02-19T19:05:02.937 INFO:tasks.workunit.client.1.smithi132.stdout:0/835: sync 2024-02-19T19:05:02.937 INFO:tasks.workunit.client.1.smithi132.stdout:2/852: sync 2024-02-19T19:05:02.937 INFO:tasks.workunit.client.1.smithi132.stdout:6/800: sync 2024-02-19T19:05:02.943 INFO:tasks.workunit.client.1.smithi132.stdout:7/888: dwrite d1b/d1e/d28/fbb [0,4194304] 0 2024-02-19T19:05:02.943 INFO:tasks.workunit.client.1.smithi132.stdout:2/853: chown d0/d4/d11/d2e/ffc 528716704 1 2024-02-19T19:05:02.944 INFO:tasks.workunit.client.1.smithi132.stdout:8/920: write d4/dc/d1b/d38/d69/d74/d93/f113 [407534,75582] 0 2024-02-19T19:05:02.947 INFO:tasks.workunit.client.1.smithi132.stdout:4/826: unlink d4/d27/d31/d71/d14/d24/l25 0 2024-02-19T19:05:02.947 INFO:tasks.workunit.client.1.smithi132.stdout:1/771: dread d1/d1e/f74 [0,4194304] 0 2024-02-19T19:05:02.947 INFO:tasks.workunit.client.1.smithi132.stdout:6/801: truncate d0/db/d25/d50/d86/fcf 577195 0 2024-02-19T19:05:02.950 INFO:tasks.workunit.client.1.smithi132.stdout:1/772: stat d1/d3a/c76 0 2024-02-19T19:05:02.950 INFO:tasks.workunit.client.1.smithi132.stdout:2/854: stat d0/d24/d58/d53/dab/d84/de1/l10f 0 2024-02-19T19:05:02.950 INFO:tasks.workunit.client.1.smithi132.stdout:4/827: creat d4/d12/d4c/f124 x:0 0 0 2024-02-19T19:05:02.951 INFO:tasks.workunit.client.1.smithi132.stdout:0/836: dread d2/dc/df/d15/d2f/d3f/f79 [0,4194304] 0 2024-02-19T19:05:02.951 INFO:tasks.workunit.client.1.smithi132.stdout:6/802: chown d0/db/d25/d3a/d3d/d69/ce8 6120 1 2024-02-19T19:05:02.953 INFO:tasks.workunit.client.1.smithi132.stdout:2/855: write d0/d4/d11/f15 [2560083,6360] 0 2024-02-19T19:05:02.954 INFO:tasks.workunit.client.1.smithi132.stdout:2/856: chown d0/d19/d1e 111674091 1 2024-02-19T19:05:02.954 INFO:tasks.workunit.client.1.smithi132.stdout:0/837: write d2/dc/d4a/d58/f10e [386676,106495] 0 2024-02-19T19:05:02.955 INFO:tasks.workunit.client.1.smithi132.stdout:2/857: chown d0/d24/d58/d53/dab/ffd 56718118 1 2024-02-19T19:05:02.960 INFO:tasks.workunit.client.1.smithi132.stdout:2/858: dwrite d0/d4/df/d7a/fce [0,4194304] 0 2024-02-19T19:05:02.988 INFO:tasks.workunit.client.1.smithi132.stdout:5/948: sync 2024-02-19T19:05:02.988 INFO:tasks.workunit.client.1.smithi132.stdout:7/889: creat d1b/d1e/d41/da3/de3/f116 x:0 0 0 2024-02-19T19:05:02.989 INFO:tasks.workunit.client.1.smithi132.stdout:5/949: dread - d8/da/d5a/d7b/dfa/d128/df0/f112 zero size 2024-02-19T19:05:02.990 INFO:tasks.workunit.client.1.smithi132.stdout:5/950: chown d8/da/db/c108 550 1 2024-02-19T19:05:02.991 INFO:tasks.workunit.client.1.smithi132.stdout:0/838: creat d2/dc/d4a/d74/f11a x:0 0 0 2024-02-19T19:05:02.993 INFO:tasks.workunit.client.1.smithi132.stdout:2/859: mknod d0/d24/d58/d53/dab/d84/dc9/c110 0 2024-02-19T19:05:02.995 INFO:tasks.workunit.client.1.smithi132.stdout:8/921: creat d4/dc/d1b/d8f/daf/d12f/f13f x:0 0 0 2024-02-19T19:05:02.996 INFO:tasks.workunit.client.1.smithi132.stdout:4/828: creat d4/d27/d31/d32/d51/d5c/dbc/d109/f125 x:0 0 0 2024-02-19T19:05:02.996 INFO:tasks.workunit.client.1.smithi132.stdout:8/922: dread d4/dc/d1b/d20/f9f [0,4194304] 0 2024-02-19T19:05:02.997 INFO:tasks.workunit.client.1.smithi132.stdout:7/890: unlink l16 0 2024-02-19T19:05:02.997 INFO:tasks.workunit.client.1.smithi132.stdout:9/966: write d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114/d118/f84 [984613,41978] 0 2024-02-19T19:05:02.999 INFO:tasks.workunit.client.1.smithi132.stdout:8/923: dread - d4/dc/d1b/d8f/daf/d12f/f13f zero size 2024-02-19T19:05:02.999 INFO:tasks.workunit.client.1.smithi132.stdout:6/803: creat d0/db/dc7/f119 x:0 0 0 2024-02-19T19:05:03.000 INFO:tasks.workunit.client.1.smithi132.stdout:5/951: creat d8/d2e/dc4/dc8/df1/f147 x:0 0 0 2024-02-19T19:05:03.002 INFO:tasks.workunit.client.1.smithi132.stdout:2/860: mkdir d0/d4/df/d7a/d111 0 2024-02-19T19:05:03.004 INFO:tasks.workunit.client.1.smithi132.stdout:7/891: dwrite ff [4194304,4194304] 0 2024-02-19T19:05:03.006 INFO:tasks.workunit.client.1.smithi132.stdout:1/773: link d1/d1e/d77/d4e/le7 d1/d1e/d77/d52/dca/leb 0 2024-02-19T19:05:03.008 INFO:tasks.workunit.client.1.smithi132.stdout:1/774: write d1/d1e/d77/f3f [330938,78580] 0 2024-02-19T19:05:03.009 INFO:tasks.workunit.client.1.smithi132.stdout:6/804: dread d0/db/d25/d3a/d3d/d69/f5c [0,4194304] 0 2024-02-19T19:05:03.013 INFO:tasks.workunit.client.1.smithi132.stdout:6/805: dread d0/d8b/fa3 [0,4194304] 0 2024-02-19T19:05:03.015 INFO:tasks.workunit.client.1.smithi132.stdout:9/967: creat d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114/d142/f14a x:0 0 0 2024-02-19T19:05:03.016 INFO:tasks.workunit.client.1.smithi132.stdout:6/806: write d0/db/d25/d3a/d3d/d69/de2/f10e [459345,28108] 0 2024-02-19T19:05:03.019 INFO:tasks.workunit.client.1.smithi132.stdout:7/892: rmdir d1b/d46/d76/d94/dd1 39 2024-02-19T19:05:03.019 INFO:tasks.workunit.client.1.smithi132.stdout:6/807: chown d0/db/d25/d3a/d3d/ce1 107 1 2024-02-19T19:05:03.020 INFO:tasks.workunit.client.1.smithi132.stdout:9/968: dread - d12/d8d/d3a/ddc/fdf zero size 2024-02-19T19:05:03.022 INFO:tasks.workunit.client.1.smithi132.stdout:2/861: creat d0/d4/f112 x:0 0 0 2024-02-19T19:05:03.024 INFO:tasks.workunit.client.1.smithi132.stdout:4/829: write d4/d4d/d87/ff3 [775552,65725] 0 2024-02-19T19:05:03.025 INFO:tasks.workunit.client.1.smithi132.stdout:5/952: getdents d8/d23/d29/dba 0 2024-02-19T19:05:03.025 INFO:tasks.workunit.client.1.smithi132.stdout:7/893: unlink d1b/d1e/d2b/d8f/dae/cc2 0 2024-02-19T19:05:03.028 INFO:tasks.workunit.client.1.smithi132.stdout:0/839: dwrite d2/dc/df/d27/f35 [4194304,4194304] 0 2024-02-19T19:05:03.033 INFO:tasks.workunit.client.1.smithi132.stdout:6/808: write d0/d51/fcc [479238,45354] 0 2024-02-19T19:05:03.034 INFO:tasks.workunit.client.1.smithi132.stdout:9/969: link d12/d8d/d3a/d40/d4e/d77/d8a/cb1 d12/d2a/d4c/d61/dfe/c14b 0 2024-02-19T19:05:03.036 INFO:tasks.workunit.client.1.smithi132.stdout:1/775: dread d1/f5 [4194304,4194304] 0 2024-02-19T19:05:03.037 INFO:tasks.workunit.client.1.smithi132.stdout:2/862: link d0/d24/d58/d53/dab/d8a/da5/fc1 d0/d19/d6a/f113 0 2024-02-19T19:05:03.039 INFO:tasks.workunit.client.1.smithi132.stdout:5/953: write d8/d1a/d64/f72 [805118,47106] 0 2024-02-19T19:05:03.039 INFO:tasks.workunit.client.1.smithi132.stdout:4/830: dread d4/d27/d31/d32/fb6 [0,4194304] 0 2024-02-19T19:05:03.042 INFO:tasks.workunit.client.1.smithi132.stdout:7/894: dwrite d1b/d5a/d63/d6b/d86/fb6 [0,4194304] 0 2024-02-19T19:05:03.044 INFO:tasks.workunit.client.1.smithi132.stdout:2/863: truncate d0/d24/dd0/ff6 856293 0 2024-02-19T19:05:03.044 INFO:tasks.workunit.client.1.smithi132.stdout:7/895: write d1b/d1e/fd6 [995371,21933] 0 2024-02-19T19:05:03.044 INFO:tasks.workunit.client.1.smithi132.stdout:5/954: mknod d8/da/db/c148 0 2024-02-19T19:05:03.046 INFO:tasks.workunit.client.1.smithi132.stdout:2/864: chown d0/d19/lea 905471570 1 2024-02-19T19:05:03.046 INFO:tasks.workunit.client.1.smithi132.stdout:1/776: symlink d1/d61/dd4/lec 0 2024-02-19T19:05:03.048 INFO:tasks.workunit.client.1.smithi132.stdout:9/970: rename d12/d8d/d3a/d40/d4e/d77/d89/dc4/lea to d12/d8d/d3a/d40/d4e/d77/d8a/dca/l14c 0 2024-02-19T19:05:03.051 INFO:tasks.workunit.client.1.smithi132.stdout:6/809: rmdir d0/db/d25/d3a/d5d/d8f/dbe/dde 0 2024-02-19T19:05:03.055 INFO:tasks.workunit.client.1.smithi132.stdout:9/971: dwrite d12/d2a/d4c/dad/f107 [0,4194304] 0 2024-02-19T19:05:03.058 INFO:tasks.workunit.client.1.smithi132.stdout:6/810: chown d0/db/d25/c2c 47 1 2024-02-19T19:05:03.059 INFO:tasks.workunit.client.1.smithi132.stdout:5/955: mkdir d8/d1a/d35/d129/d149 0 2024-02-19T19:05:03.061 INFO:tasks.workunit.client.1.smithi132.stdout:1/777: fdatasync d1/d1e/d77/d2c/d89/fd8 0 2024-02-19T19:05:03.062 INFO:tasks.workunit.client.1.smithi132.stdout:6/811: stat d0/db/d25/d3a/d3d/d69/d89/fdb 0 2024-02-19T19:05:03.064 INFO:tasks.workunit.client.1.smithi132.stdout:0/840: dread d2/dc/f6d [0,4194304] 0 2024-02-19T19:05:03.064 INFO:tasks.workunit.client.1.smithi132.stdout:9/972: dwrite d12/fa4 [4194304,4194304] 0 2024-02-19T19:05:03.065 INFO:tasks.workunit.client.1.smithi132.stdout:0/841: dread - d2/f8c zero size 2024-02-19T19:05:03.067 INFO:tasks.workunit.client.1.smithi132.stdout:5/956: creat d8/da/d5a/d7b/d96/f14a x:0 0 0 2024-02-19T19:05:03.068 INFO:tasks.workunit.client.1.smithi132.stdout:0/842: write d2/dc/df/d15/d34/d84/da7/f10f [64510,56639] 0 2024-02-19T19:05:03.068 INFO:tasks.workunit.client.1.smithi132.stdout:4/831: truncate d4/d27/d31/d32/d51/f97 75536 0 2024-02-19T19:05:03.069 INFO:tasks.workunit.client.1.smithi132.stdout:4/832: dread - d4/d19/d21/f11b zero size 2024-02-19T19:05:03.070 INFO:tasks.workunit.client.1.smithi132.stdout:2/865: rename d0/d4/df/d40/d68/f8f to d0/d4/d11/d2e/f114 0 2024-02-19T19:05:03.071 INFO:tasks.workunit.client.1.smithi132.stdout:0/843: chown d2/dc/df/d15/d2f/d3f/f8d 24005773 1 2024-02-19T19:05:03.071 INFO:tasks.workunit.client.1.smithi132.stdout:0/844: readlink d2/d6e/d7a/l89 0 2024-02-19T19:05:03.073 INFO:tasks.workunit.client.1.smithi132.stdout:7/896: creat d1b/d32/f117 x:0 0 0 2024-02-19T19:05:03.073 INFO:tasks.workunit.client.1.smithi132.stdout:3/885: sync 2024-02-19T19:05:03.074 INFO:tasks.workunit.client.1.smithi132.stdout:9/973: creat d12/d1b/f14d x:0 0 0 2024-02-19T19:05:03.076 INFO:tasks.workunit.client.1.smithi132.stdout:5/957: mknod d8/d2e/dc4/d113/d11a/d139/c14b 0 2024-02-19T19:05:03.076 INFO:tasks.workunit.client.1.smithi132.stdout:9/974: dread d12/d8d/d3a/ddc/f125 [0,4194304] 0 2024-02-19T19:05:03.077 INFO:tasks.workunit.client.1.smithi132.stdout:1/778: rename d1/d1e/d77/d2c/ccc to d1/d3a/d50/ced 0 2024-02-19T19:05:03.078 INFO:tasks.workunit.client.1.smithi132.stdout:4/833: creat d4/d27/d31/d71/d14/de4/f126 x:0 0 0 2024-02-19T19:05:03.079 INFO:tasks.workunit.client.1.smithi132.stdout:3/886: mknod d8/d13/deb/c117 0 2024-02-19T19:05:03.080 INFO:tasks.workunit.client.1.smithi132.stdout:1/779: write d1/d1e/d77/f3f [1488778,110066] 0 2024-02-19T19:05:03.082 INFO:tasks.workunit.client.1.smithi132.stdout:2/866: rename d0/d4/df/d40/d7b/fe2 to d0/d4/df/d40/d68/d102/f115 0 2024-02-19T19:05:03.085 INFO:tasks.workunit.client.1.smithi132.stdout:4/834: symlink d4/d27/d31/d32/d51/d5c/l127 0 2024-02-19T19:05:03.088 INFO:tasks.workunit.client.1.smithi132.stdout:0/845: dwrite d2/dc/df/d15/d2f/f90 [0,4194304] 0 2024-02-19T19:05:03.089 INFO:tasks.workunit.client.1.smithi132.stdout:7/897: getdents d1b/d1e/d41 0 2024-02-19T19:05:03.089 INFO:tasks.workunit.client.1.smithi132.stdout:6/812: dread d0/db/d25/d3a/d5d/d8f/d22/f23 [0,4194304] 0 2024-02-19T19:05:03.089 INFO:tasks.workunit.client.1.smithi132.stdout:8/924: sync 2024-02-19T19:05:03.090 INFO:tasks.workunit.client.1.smithi132.stdout:1/780: mknod d1/d1e/d77/d52/db9/cee 0 2024-02-19T19:05:03.095 INFO:tasks.workunit.client.1.smithi132.stdout:2/867: creat d0/d24/f116 x:0 0 0 2024-02-19T19:05:03.096 INFO:tasks.workunit.client.1.smithi132.stdout:3/887: dwrite d8/d13/d2b/d69/d92/daf/f10f [0,4194304] 0 2024-02-19T19:05:03.101 INFO:tasks.workunit.client.1.smithi132.stdout:9/975: dread d12/d2a/f12f [0,4194304] 0 2024-02-19T19:05:03.102 INFO:tasks.workunit.client.1.smithi132.stdout:9/976: chown d12/d1b/c5e 203 1 2024-02-19T19:05:03.105 INFO:tasks.workunit.client.1.smithi132.stdout:3/888: dwrite d8/d13/d2b/d60/f95 [0,4194304] 0 2024-02-19T19:05:03.106 INFO:tasks.workunit.client.1.smithi132.stdout:3/889: chown d8/d13 98052193 1 2024-02-19T19:05:03.107 INFO:tasks.workunit.client.1.smithi132.stdout:3/890: fdatasync d8/d13/d2b/d10c/f2d 0 2024-02-19T19:05:03.111 INFO:tasks.workunit.client.1.smithi132.stdout:9/977: dwrite d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/fb5 [0,4194304] 0 2024-02-19T19:05:03.113 INFO:tasks.workunit.client.1.smithi132.stdout:9/978: write d12/d2a/d4c/dad/f107 [1252211,29343] 0 2024-02-19T19:05:03.131 INFO:tasks.workunit.client.1.smithi132.stdout:7/898: rmdir d1b/d1e/d2b/d8f/da5 39 2024-02-19T19:05:03.133 INFO:tasks.workunit.client.1.smithi132.stdout:1/781: write d1/d1e/d77/d2c/d43/fda [714045,2109] 0 2024-02-19T19:05:03.136 INFO:tasks.workunit.client.1.smithi132.stdout:8/925: rename d4/dc/d1b/d38/d126/da3/ld0 to d4/d2d/l140 0 2024-02-19T19:05:03.141 INFO:tasks.workunit.client.1.smithi132.stdout:5/958: truncate d8/d23/d29/d27/f87 1832812 0 2024-02-19T19:05:03.144 INFO:tasks.workunit.client.1.smithi132.stdout:4/835: creat d4/d27/d31/d71/f128 x:0 0 0 2024-02-19T19:05:03.147 INFO:tasks.workunit.client.1.smithi132.stdout:4/836: dread d4/d27/d31/d71/d14/d24/f39 [0,4194304] 0 2024-02-19T19:05:03.149 INFO:tasks.workunit.client.1.smithi132.stdout:6/813: creat d0/db/f11a x:0 0 0 2024-02-19T19:05:03.150 INFO:tasks.workunit.client.1.smithi132.stdout:6/814: chown d0/db/d25/d3a/d5d/d8f/d46/d5a/l45 38839449 1 2024-02-19T19:05:03.152 INFO:tasks.workunit.client.1.smithi132.stdout:5/959: dread d8/d2e/fbe [0,4194304] 0 2024-02-19T19:05:03.153 INFO:tasks.workunit.client.1.smithi132.stdout:5/960: fsync d8/d2e/dc4/dc8/daa/fe5 0 2024-02-19T19:05:03.153 INFO:tasks.workunit.client.1.smithi132.stdout:2/868: creat d0/d24/f117 x:0 0 0 2024-02-19T19:05:03.154 INFO:tasks.workunit.client.1.smithi132.stdout:0/846: truncate d2/dc/d4a/f42 4561023 0 2024-02-19T19:05:03.155 INFO:tasks.workunit.client.1.smithi132.stdout:8/926: write d4/dc/d1b/d54/f9c [476297,51640] 0 2024-02-19T19:05:03.155 INFO:tasks.workunit.client.1.smithi132.stdout:5/961: write d8/d2e/dc4/d113/d11a/d139/dea/f102 [608916,48817] 0 2024-02-19T19:05:03.155 INFO:tasks.workunit.client.1.smithi132.stdout:7/899: creat d1b/d1e/d2b/d62/d53/f118 x:0 0 0 2024-02-19T19:05:03.157 INFO:tasks.workunit.client.1.smithi132.stdout:3/891: truncate d8/d13/d2b/d69/d92/daf/f10f 3807419 0 2024-02-19T19:05:03.159 INFO:tasks.workunit.client.1.smithi132.stdout:2/869: fsync d0/d24/daf/fff 0 2024-02-19T19:05:03.160 INFO:tasks.workunit.client.1.smithi132.stdout:8/927: dread f2 [0,4194304] 0 2024-02-19T19:05:03.160 INFO:tasks.workunit.client.1.smithi132.stdout:0/847: readlink d2/la 0 2024-02-19T19:05:03.162 INFO:tasks.workunit.client.1.smithi132.stdout:0/848: read - d2/dc/df/d27/d3a/d52/da8/f111 zero size 2024-02-19T19:05:03.163 INFO:tasks.workunit.client.1.smithi132.stdout:9/979: rename d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114/d118/fd7 to d12/d8d/d3a/f14e 0 2024-02-19T19:05:03.166 INFO:tasks.workunit.client.1.smithi132.stdout:7/900: dread d1b/d5a/d63/d6b/d79/d84/f8e [0,4194304] 0 2024-02-19T19:05:03.167 INFO:tasks.workunit.client.1.smithi132.stdout:2/870: creat d0/d4/df/dc2/dad/f118 x:0 0 0 2024-02-19T19:05:03.168 INFO:tasks.workunit.client.1.smithi132.stdout:5/962: mknod d8/d23/c14c 0 2024-02-19T19:05:03.169 INFO:tasks.workunit.client.1.smithi132.stdout:8/928: creat d4/dac/f141 x:0 0 0 2024-02-19T19:05:03.170 INFO:tasks.workunit.client.1.smithi132.stdout:1/782: rename d1/d1e/f2a to d1/d3a/d50/fef 0 2024-02-19T19:05:03.170 INFO:tasks.workunit.client.1.smithi132.stdout:6/815: link d0/db/d25/d3a/d4b/lb4 d0/db/dc9/l11b 0 2024-02-19T19:05:03.171 INFO:tasks.workunit.client.1.smithi132.stdout:9/980: mkdir d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d14f 0 2024-02-19T19:05:03.173 INFO:tasks.workunit.client.1.smithi132.stdout:7/901: rmdir d1b/d5a/d63/d6b/d79/d84 39 2024-02-19T19:05:03.174 INFO:tasks.workunit.client.1.smithi132.stdout:2/871: symlink d0/d4/df/d40/dbd/l119 0 2024-02-19T19:05:03.176 INFO:tasks.workunit.client.1.smithi132.stdout:5/963: fsync d8/da/db/f83 0 2024-02-19T19:05:03.178 INFO:tasks.workunit.client.1.smithi132.stdout:5/964: chown d8/da/d5a/d7b/d96/c9f 127471365 1 2024-02-19T19:05:03.178 INFO:tasks.workunit.client.1.smithi132.stdout:9/981: dwrite d12/d2a/d4c/dad/f107 [0,4194304] 0 2024-02-19T19:05:03.180 INFO:tasks.workunit.client.1.smithi132.stdout:8/929: creat d4/dc/d42/f142 x:0 0 0 2024-02-19T19:05:03.181 INFO:tasks.workunit.client.1.smithi132.stdout:4/837: rename d4/d19/daa/dc7/d98/f122 to d4/d27/d31/d32/d51/d5c/dbc/d109/f129 0 2024-02-19T19:05:03.181 INFO:tasks.workunit.client.1.smithi132.stdout:1/783: rename d1/d1e/d77 to d1/d1e/d77/d2c/d43/d81/df0 22 2024-02-19T19:05:03.185 INFO:tasks.workunit.client.1.smithi132.stdout:2/872: symlink d0/d3c/l11a 0 2024-02-19T19:05:03.186 INFO:tasks.workunit.client.1.smithi132.stdout:5/965: mknod d8/da/d5a/d7b/d96/d9e/c14d 0 2024-02-19T19:05:03.188 INFO:tasks.workunit.client.1.smithi132.stdout:5/966: chown d8/da/d5a/d7b/dfa/d128/df0 6316 1 2024-02-19T19:05:03.188 INFO:tasks.workunit.client.1.smithi132.stdout:3/892: truncate d8/d13/d2b/d69/da6/f5b 1270283 0 2024-02-19T19:05:03.189 INFO:tasks.workunit.client.1.smithi132.stdout:9/982: dread d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/dc8/deb/ff2 [0,4194304] 0 2024-02-19T19:05:03.190 INFO:tasks.workunit.client.1.smithi132.stdout:6/816: mkdir d0/dc/dec/d11c 0 2024-02-19T19:05:03.193 INFO:tasks.workunit.client.1.smithi132.stdout:1/784: unlink d1/d1e/d77/d78/cdb 0 2024-02-19T19:05:03.193 INFO:tasks.workunit.client.1.smithi132.stdout:9/983: write d12/d8d/d3a/ff0 [405918,4839] 0 2024-02-19T19:05:03.198 INFO:tasks.workunit.client.1.smithi132.stdout:5/967: dwrite d8/d2e/dc4/dc8/daa/fe5 [4194304,4194304] 0 2024-02-19T19:05:03.201 INFO:tasks.workunit.client.1.smithi132.stdout:9/984: symlink d12/d2a/d4c/d61/dfe/d124/l150 0 2024-02-19T19:05:03.202 INFO:tasks.workunit.client.1.smithi132.stdout:7/902: getdents d1b/d1e/d28/da9/dd3/ddb 0 2024-02-19T19:05:03.203 INFO:tasks.workunit.client.1.smithi132.stdout:7/903: chown d1b/d46 47 1 2024-02-19T19:05:03.203 INFO:tasks.workunit.client.1.smithi132.stdout:6/817: rename d0/db/d25/d3a/d5d/d8f/d46/d5a/c21 to d0/db/d25/d3a/d5d/d8f/d46/d5a/daf/c11d 0 2024-02-19T19:05:03.206 INFO:tasks.workunit.client.1.smithi132.stdout:2/873: write d0/d19/f78 [3305800,87055] 0 2024-02-19T19:05:03.210 INFO:tasks.workunit.client.1.smithi132.stdout:6/818: write d0/d8b/f4c [4217323,45514] 0 2024-02-19T19:05:03.210 INFO:tasks.workunit.client.1.smithi132.stdout:8/930: dwrite d4/dc/fa9 [0,4194304] 0 2024-02-19T19:05:03.210 INFO:tasks.workunit.client.1.smithi132.stdout:3/893: write d8/d13/d2b/d69/da6/f5b [2173421,73538] 0 2024-02-19T19:05:03.212 INFO:tasks.workunit.client.1.smithi132.stdout:5/968: rename d8/d2e/dc4/dc8/c92 to d8/d2e/dc4/d113/d11a/c14e 0 2024-02-19T19:05:03.218 INFO:tasks.workunit.client.1.smithi132.stdout:7/904: read d1b/d1e/d41/f6a [3110982,61749] 0 2024-02-19T19:05:03.218 INFO:tasks.workunit.client.1.smithi132.stdout:9/985: rename d12/d2a/d88/f120 to d12/db6/f151 0 2024-02-19T19:05:03.219 INFO:tasks.workunit.client.1.smithi132.stdout:2/874: mknod d0/c11b 0 2024-02-19T19:05:03.219 INFO:tasks.workunit.client.1.smithi132.stdout:4/838: dwrite d4/d27/d31/d71/d14/d24/f39 [0,4194304] 0 2024-02-19T19:05:03.223 INFO:tasks.workunit.client.1.smithi132.stdout:3/894: rmdir d8/d13/d2b/d60/db8/dfb 39 2024-02-19T19:05:03.223 INFO:tasks.workunit.client.1.smithi132.stdout:4/839: fsync d4/d19/daa/f11a 0 2024-02-19T19:05:03.223 INFO:tasks.workunit.client.1.smithi132.stdout:8/931: write d4/dc/d1b/d8f/daf/d12f/f13f [301306,40927] 0 2024-02-19T19:05:03.226 INFO:tasks.workunit.client.1.smithi132.stdout:5/969: mknod d8/d1a/d35/d129/d149/c14f 0 2024-02-19T19:05:03.226 INFO:tasks.workunit.client.1.smithi132.stdout:2/875: rmdir d0/d19/d6a 39 2024-02-19T19:05:03.226 INFO:tasks.workunit.client.1.smithi132.stdout:6/819: rmdir d0/db/d25/d3a/d4b/df6 0 2024-02-19T19:05:03.228 INFO:tasks.workunit.client.1.smithi132.stdout:5/970: fdatasync d8/d2e/dc4/dc8/df1/f107 0 2024-02-19T19:05:03.228 INFO:tasks.workunit.client.1.smithi132.stdout:9/986: mknod d12/d8d/d3a/d40/d4e/d77/d134/c152 0 2024-02-19T19:05:03.229 INFO:tasks.workunit.client.1.smithi132.stdout:4/840: mkdir d4/d12/d28/d12a 0 2024-02-19T19:05:03.230 INFO:tasks.workunit.client.1.smithi132.stdout:8/932: creat d4/d2d/dcc/dfc/f143 x:0 0 0 2024-02-19T19:05:03.232 INFO:tasks.workunit.client.1.smithi132.stdout:2/876: symlink d0/d4/l11c 0 2024-02-19T19:05:03.234 INFO:tasks.workunit.client.1.smithi132.stdout:6/820: read d0/db/d25/d3a/d5d/d8f/d87/fca [383738,34024] 0 2024-02-19T19:05:03.235 INFO:tasks.workunit.client.1.smithi132.stdout:1/785: dread d1/f1d [0,4194304] 0 2024-02-19T19:05:03.235 INFO:tasks.workunit.client.1.smithi132.stdout:7/905: creat d1b/d32/f119 x:0 0 0 2024-02-19T19:05:03.237 INFO:tasks.workunit.client.1.smithi132.stdout:3/895: dwrite d8/d13/d2b/d60/f106 [0,4194304] 0 2024-02-19T19:05:03.238 INFO:tasks.workunit.client.1.smithi132.stdout:4/841: creat d4/d27/d31/d71/d14/d24/d49/f12b x:0 0 0 2024-02-19T19:05:03.238 INFO:tasks.workunit.client.1.smithi132.stdout:9/987: creat d12/d8d/d3a/d40/d4e/d77/d134/d145/f153 x:0 0 0 2024-02-19T19:05:03.239 INFO:tasks.workunit.client.1.smithi132.stdout:1/786: truncate d1/f4a 350138 0 2024-02-19T19:05:03.240 INFO:tasks.workunit.client.1.smithi132.stdout:4/842: stat d4/d12/f2c 0 2024-02-19T19:05:03.240 INFO:tasks.workunit.client.1.smithi132.stdout:7/906: unlink la 0 2024-02-19T19:05:03.243 INFO:tasks.workunit.client.1.smithi132.stdout:8/933: rename d4/dc/d1b/d38/d69/d88 to d4/dc/d1b/d38/d126/d68/d79/d144 0 2024-02-19T19:05:03.244 INFO:tasks.workunit.client.1.smithi132.stdout:7/907: dread - d1b/d1e/d2b/d8f/dae/f114 zero size 2024-02-19T19:05:03.244 INFO:tasks.workunit.client.1.smithi132.stdout:3/896: readlink d8/d13/d2b/d69/d92/d34/d42/l115 0 2024-02-19T19:05:03.247 INFO:tasks.workunit.client.1.smithi132.stdout:6/821: dread d0/dc/f81 [0,4194304] 0 2024-02-19T19:05:03.247 INFO:tasks.workunit.client.1.smithi132.stdout:7/908: dread - d1b/d32/f119 zero size 2024-02-19T19:05:03.248 INFO:tasks.workunit.client.1.smithi132.stdout:4/843: write d4/d27/d7f/dc0/ff0 [819792,27177] 0 2024-02-19T19:05:03.249 INFO:tasks.workunit.client.1.smithi132.stdout:8/934: chown d4/dc/d1b/d38/d69/d74/d93 49041833 1 2024-02-19T19:05:03.251 INFO:tasks.workunit.client.1.smithi132.stdout:6/822: fdatasync d0/db/d25/d3a/d3d/f74 0 2024-02-19T19:05:03.253 INFO:tasks.workunit.client.1.smithi132.stdout:7/909: mknod d1b/d1e/d41/da3/dc8/c11a 0 2024-02-19T19:05:03.255 INFO:tasks.workunit.client.1.smithi132.stdout:4/844: creat d4/d12/d9f/f12c x:0 0 0 2024-02-19T19:05:03.255 INFO:tasks.workunit.client.1.smithi132.stdout:8/935: mkdir d4/dc/d1b/d54/d5e/d145 0 2024-02-19T19:05:03.257 INFO:tasks.workunit.client.1.smithi132.stdout:9/988: rename d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114/d118/c96 to d12/d8d/d3a/d40/d4e/d77/d89/dc4/c154 0 2024-02-19T19:05:03.258 INFO:tasks.workunit.client.1.smithi132.stdout:7/910: mkdir d1b/d1e/d28/d11b 0 2024-02-19T19:05:03.260 INFO:tasks.workunit.client.1.smithi132.stdout:6/823: chown d0/db/d25/d3a/d5d/c88 3 1 2024-02-19T19:05:03.261 INFO:tasks.workunit.client.1.smithi132.stdout:5/971: dwrite d8/d23/d29/fa1 [0,4194304] 0 2024-02-19T19:05:03.264 INFO:tasks.workunit.client.1.smithi132.stdout:9/989: write d12/d2a/d4c/dad/db2/f10c [243018,111680] 0 2024-02-19T19:05:03.266 INFO:tasks.workunit.client.1.smithi132.stdout:6/824: chown d0/db/d25/fcb 303080 1 2024-02-19T19:05:03.269 INFO:tasks.workunit.client.1.smithi132.stdout:2/877: dread d0/d4/d11/f32 [0,4194304] 0 2024-02-19T19:05:03.271 INFO:tasks.workunit.client.1.smithi132.stdout:7/911: truncate d1b/d46/d76/d94/dd1/fef 7346641 0 2024-02-19T19:05:03.272 INFO:tasks.workunit.client.1.smithi132.stdout:8/936: link d4/dc/d1b/d20/d62/d91/d101/f137 d4/dc/d1b/d20/d62/f146 0 2024-02-19T19:05:03.273 INFO:tasks.workunit.client.1.smithi132.stdout:1/787: dread d1/d1e/d77/d2c/d43/d81/da1/fbb [0,4194304] 0 2024-02-19T19:05:03.274 INFO:tasks.workunit.client.1.smithi132.stdout:3/897: dread d8/d13/d2b/f99 [0,4194304] 0 2024-02-19T19:05:03.275 INFO:tasks.workunit.client.1.smithi132.stdout:6/825: mkdir d0/db/d25/d50/d86/de7/d11e 0 2024-02-19T19:05:03.275 INFO:tasks.workunit.client.1.smithi132.stdout:4/845: dwrite d4/d27/d31/d71/d37/f116 [0,4194304] 0 2024-02-19T19:05:03.276 INFO:tasks.workunit.client.1.smithi132.stdout:5/972: rename d8/d23/d29/dba/c116 to d8/d1a/c150 0 2024-02-19T19:05:03.276 INFO:tasks.workunit.client.1.smithi132.stdout:2/878: mknod d0/c11d 0 2024-02-19T19:05:03.279 INFO:tasks.workunit.client.1.smithi132.stdout:3/898: symlink d8/d38/d44/d6e/l118 0 2024-02-19T19:05:03.281 INFO:tasks.workunit.client.1.smithi132.stdout:1/788: write d1/d1e/d77/d4e/f5f [480652,34018] 0 2024-02-19T19:05:03.283 INFO:tasks.workunit.client.1.smithi132.stdout:5/973: symlink d8/d1a/d64/de2/l151 0 2024-02-19T19:05:03.283 INFO:tasks.workunit.client.1.smithi132.stdout:9/990: dwrite d12/d8d/f115 [0,4194304] 0 2024-02-19T19:05:03.285 INFO:tasks.workunit.client.1.smithi132.stdout:6/826: truncate d0/db/d25/fcb 1359128 0 2024-02-19T19:05:03.286 INFO:tasks.workunit.client.1.smithi132.stdout:7/912: rename d1b/d1e/d2b/d62 to d1b/d1e/d41/da3/dbd/d11c 0 2024-02-19T19:05:03.288 INFO:tasks.workunit.client.1.smithi132.stdout:7/913: write d1b/d1e/d2b/d8f/dae/f114 [467490,106188] 0 2024-02-19T19:05:03.288 INFO:tasks.workunit.client.1.smithi132.stdout:2/879: rmdir d0/d24/d58/d53/dab 39 2024-02-19T19:05:03.291 INFO:tasks.workunit.client.1.smithi132.stdout:2/880: fsync d0/d24/d58/f49 0 2024-02-19T19:05:03.291 INFO:tasks.workunit.client.1.smithi132.stdout:9/991: readlink d12/l2d 0 2024-02-19T19:05:03.292 INFO:tasks.workunit.client.1.smithi132.stdout:2/881: read - d0/d19/d1e/fde zero size 2024-02-19T19:05:03.295 INFO:tasks.workunit.client.1.smithi132.stdout:6/827: symlink d0/d8b/l11f 0 2024-02-19T19:05:03.296 INFO:tasks.workunit.client.1.smithi132.stdout:3/899: dread d8/d13/d2b/d69/d92/daf/f10f [0,4194304] 0 2024-02-19T19:05:03.297 INFO:tasks.workunit.client.1.smithi132.stdout:7/914: unlink d1b/d1e/d2b/dd0/cf3 0 2024-02-19T19:05:03.299 INFO:tasks.workunit.client.1.smithi132.stdout:9/992: creat d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/d114/d142/f155 x:0 0 0 2024-02-19T19:05:03.299 INFO:tasks.workunit.client.1.smithi132.stdout:1/789: getdents d1/d1e/d77/d2c/d43/d81/da1/de2 0 2024-02-19T19:05:03.300 INFO:tasks.workunit.client.1.smithi132.stdout:5/974: mkdir d8/d23/d29/def/d152 0 2024-02-19T19:05:03.302 INFO:tasks.workunit.client.1.smithi132.stdout:5/975: write d8/d2e/dc4/dc8/f104 [836747,39973] 0 2024-02-19T19:05:03.302 INFO:tasks.workunit.client.1.smithi132.stdout:3/900: mkdir d8/d13/d2b/d69/d92/d34/d42/d6c/de6/d119 0 2024-02-19T19:05:03.302 INFO:tasks.workunit.client.1.smithi132.stdout:4/846: write d4/f59 [2067750,101280] 0 2024-02-19T19:05:03.302 INFO:tasks.workunit.client.1.smithi132.stdout:9/993: chown d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/fb0 15968513 1 2024-02-19T19:05:03.303 INFO:tasks.workunit.client.1.smithi132.stdout:7/915: rename d1b/d1e/d28/f55 to d1b/d1e/d28/f11d 0 2024-02-19T19:05:03.304 INFO:tasks.workunit.client.1.smithi132.stdout:7/916: fdatasync d1b/d1e/d41/db5/fe6 0 2024-02-19T19:05:03.304 INFO:tasks.workunit.client.1.smithi132.stdout:8/937: truncate d4/dc/d1b/d38/d126/d128/f8c 6830086 0 2024-02-19T19:05:03.305 INFO:tasks.workunit.client.1.smithi132.stdout:8/938: stat d4/d2d/d82/f98 0 2024-02-19T19:05:03.305 INFO:tasks.workunit.client.1.smithi132.stdout:7/917: write d1b/d1e/d41/db5/fe6 [241198,117258] 0 2024-02-19T19:05:03.307 INFO:tasks.workunit.client.1.smithi132.stdout:6/828: getdents d0/db/d25/d3a/df3/df8 0 2024-02-19T19:05:03.308 INFO:tasks.workunit.client.1.smithi132.stdout:1/790: creat d1/d95/d97/ff1 x:0 0 0 2024-02-19T19:05:03.310 INFO:tasks.workunit.client.1.smithi132.stdout:3/901: truncate d8/f22 2212547 0 2024-02-19T19:05:03.311 INFO:tasks.workunit.client.1.smithi132.stdout:9/994: creat d12/d2a/d4c/dad/db2/f156 x:0 0 0 2024-02-19T19:05:03.314 INFO:tasks.workunit.client.1.smithi132.stdout:2/882: write d0/d4/df/d40/d68/d102/f115 [982810,39417] 0 2024-02-19T19:05:03.314 INFO:tasks.workunit.client.1.smithi132.stdout:6/829: dread d0/db/d25/d3a/d5d/d8f/d46/d5a/daf/fd8 [0,4194304] 0 2024-02-19T19:05:03.318 INFO:tasks.workunit.client.1.smithi132.stdout:3/902: write d8/d38/ff5 [1619448,71685] 0 2024-02-19T19:05:03.319 INFO:tasks.workunit.client.1.smithi132.stdout:4/847: write d4/f10 [498463,98981] 0 2024-02-19T19:05:03.322 INFO:tasks.workunit.client.1.smithi132.stdout:1/791: dread d1/d1e/d77/d52/d64/fb0 [0,4194304] 0 2024-02-19T19:05:03.323 INFO:tasks.workunit.client.1.smithi132.stdout:9/995: dwrite d12/d2a/d88/db8/f123 [0,4194304] 0 2024-02-19T19:05:03.325 INFO:tasks.workunit.client.1.smithi132.stdout:7/918: rename d1b/d5a/d63/d6b to d1b/d1e/d11e 0 2024-02-19T19:05:03.326 INFO:tasks.workunit.client.1.smithi132.stdout:0/849: sync 2024-02-19T19:05:03.326 INFO:tasks.workunit.client.1.smithi132.stdout:5/976: truncate d8/d23/d29/d27/f25 2115580 0 2024-02-19T19:05:03.328 INFO:tasks.workunit.client.1.smithi132.stdout:7/919: write d1b/d1e/d2b/d8f/dae/f114 [1157186,102282] 0 2024-02-19T19:05:03.330 INFO:tasks.workunit.client.1.smithi132.stdout:3/903: write d8/d38/d44/f5e [4917899,73659] 0 2024-02-19T19:05:03.330 INFO:tasks.workunit.client.1.smithi132.stdout:2/883: fsync d0/d24/dd0/fd8 0 2024-02-19T19:05:03.332 INFO:tasks.workunit.client.1.smithi132.stdout:1/792: write d1/d1e/d77/d52/d64/d75/f83 [1690913,60325] 0 2024-02-19T19:05:03.333 INFO:tasks.workunit.client.1.smithi132.stdout:6/830: rename d0/d51/da7/f10a to d0/db/d25/d3a/df3/f120 0 2024-02-19T19:05:03.341 INFO:tasks.workunit.client.1.smithi132.stdout:5/977: read d8/d2e/d71/f80 [589607,32866] 0 2024-02-19T19:05:03.342 INFO:tasks.workunit.client.1.smithi132.stdout:7/920: mknod d1b/d1e/d41/da3/de3/c11f 0 2024-02-19T19:05:03.342 INFO:tasks.workunit.client.1.smithi132.stdout:6/831: creat d0/db/d25/d3a/df3/f121 x:0 0 0 2024-02-19T19:05:03.343 INFO:tasks.workunit.client.1.smithi132.stdout:8/939: link d4/l5a d4/l147 0 2024-02-19T19:05:03.345 INFO:tasks.workunit.client.1.smithi132.stdout:4/848: dread d4/f7d [0,4194304] 0 2024-02-19T19:05:03.346 INFO:tasks.workunit.client.1.smithi132.stdout:6/832: fdatasync d0/db/d25/d3a/d4b/fc4 0 2024-02-19T19:05:03.346 INFO:tasks.workunit.client.1.smithi132.stdout:2/884: symlink d0/d4/d11/l11e 0 2024-02-19T19:05:03.348 INFO:tasks.workunit.client.1.smithi132.stdout:0/850: dwrite d2/dc/df/fd2 [0,4194304] 0 2024-02-19T19:05:03.350 INFO:tasks.workunit.client.1.smithi132.stdout:8/940: dread d4/dc/d1b/f43 [4194304,4194304] 0 2024-02-19T19:05:03.354 INFO:tasks.workunit.client.1.smithi132.stdout:3/904: rename d8/d13/d2b/d69/d92/l109 to d8/l11a 0 2024-02-19T19:05:03.355 INFO:tasks.workunit.client.1.smithi132.stdout:1/793: truncate d1/d1e/d77/f5e 1355207 0 2024-02-19T19:05:03.356 INFO:tasks.workunit.client.1.smithi132.stdout:9/996: dwrite d12/d8d/d3a/f6d [0,4194304] 0 2024-02-19T19:05:03.358 INFO:tasks.workunit.client.1.smithi132.stdout:5/978: unlink d8/da/d5a/ca8 0 2024-02-19T19:05:03.359 INFO:tasks.workunit.client.1.smithi132.stdout:3/905: write d8/d13/d2b/d69/d92/daf/fb5 [489908,111357] 0 2024-02-19T19:05:03.361 INFO:tasks.workunit.client.1.smithi132.stdout:7/921: creat d1b/d1e/d41/da3/dd5/f120 x:0 0 0 2024-02-19T19:05:03.363 INFO:tasks.workunit.client.1.smithi132.stdout:0/851: mkdir d2/dc/df/d15/d34/d11b 0 2024-02-19T19:05:03.367 INFO:tasks.workunit.client.1.smithi132.stdout:8/941: read - d4/dc/d1b/d20/d62/d91/fc8 zero size 2024-02-19T19:05:03.367 INFO:tasks.workunit.client.1.smithi132.stdout:9/997: chown d12/d8d/d3a/d40/d4e/d77/d8a/l126 59172 1 2024-02-19T19:05:03.367 INFO:tasks.workunit.client.1.smithi132.stdout:4/849: rename d4/d12/da4/fd0 to d4/d12/da4/f12d 0 2024-02-19T19:05:03.367 INFO:tasks.workunit.client.1.smithi132.stdout:6/833: dwrite d0/db/d25/d3a/d5d/d8f/d87/fda [0,4194304] 0 2024-02-19T19:05:03.367 INFO:tasks.workunit.client.1.smithi132.stdout:2/885: rmdir d0/d24/d58/d53/dab/d84/de1 39 2024-02-19T19:05:03.370 INFO:tasks.workunit.client.1.smithi132.stdout:7/922: symlink d1b/d1e/d11e/d86/l121 0 2024-02-19T19:05:03.372 INFO:tasks.workunit.client.1.smithi132.stdout:5/979: chown d8/d2e/d71/dbc/fd2 1882267584 1 2024-02-19T19:05:03.373 INFO:tasks.workunit.client.1.smithi132.stdout:8/942: read d4/fab [338913,122546] 0 2024-02-19T19:05:03.375 INFO:tasks.workunit.client.1.smithi132.stdout:2/886: unlink d0/d24/d58/f8c 0 2024-02-19T19:05:03.377 INFO:tasks.workunit.client.1.smithi132.stdout:7/923: mknod d1b/d1e/d41/da3/dd5/c122 0 2024-02-19T19:05:03.377 INFO:tasks.workunit.client.1.smithi132.stdout:3/906: dwrite d8/fd4 [0,4194304] 0 2024-02-19T19:05:03.378 INFO:tasks.workunit.client.1.smithi132.stdout:4/850: rename d4/d12/d16/ddb/le7 to d4/d12/de8/l12e 0 2024-02-19T19:05:03.379 INFO:tasks.workunit.client.1.smithi132.stdout:0/852: dread d2/dc/d4a/d67/fbb [0,4194304] 0 2024-02-19T19:05:03.381 INFO:tasks.workunit.client.1.smithi132.stdout:8/943: readlink d4/dc/d1b/d38/d69/d74/dce/ld6 0 2024-02-19T19:05:03.386 INFO:tasks.workunit.client.1.smithi132.stdout:7/924: dread d1b/d1e/d28/f4f [0,4194304] 0 2024-02-19T19:05:03.391 INFO:tasks.workunit.client.1.smithi132.stdout:7/925: stat d1b/d1e/d28/dd2 0 2024-02-19T19:05:03.391 INFO:tasks.workunit.client.1.smithi132.stdout:8/944: dwrite d4/dac/f141 [0,4194304] 0 2024-02-19T19:05:03.391 INFO:tasks.workunit.client.1.smithi132.stdout:3/907: creat d8/d13/f11b x:0 0 0 2024-02-19T19:05:03.392 INFO:tasks.workunit.client.1.smithi132.stdout:5/980: write d8/da/db/f83 [920587,101506] 0 2024-02-19T19:05:03.393 INFO:tasks.workunit.client.1.smithi132.stdout:2/887: creat d0/d4/d11/d2e/d5a/f11f x:0 0 0 2024-02-19T19:05:03.394 INFO:tasks.workunit.client.1.smithi132.stdout:9/998: rename d12/d8d/d3a/d40/d4e/d77/d8a/d82/d136/d87/fb5 to d12/d8d/d3a/d40/d4e/d77/d8a/d82/f157 0 2024-02-19T19:05:03.395 INFO:tasks.workunit.client.1.smithi132.stdout:7/926: fdatasync d1b/d1e/d41/da3/de3/ff2 0 2024-02-19T19:05:03.397 INFO:tasks.workunit.client.1.smithi132.stdout:8/945: unlink d4/dc/d1b/d38/d126/cf7 0 2024-02-19T19:05:03.398 INFO:tasks.workunit.client.1.smithi132.stdout:8/946: chown d4/dc/d1b/d38/d69/c76 11601262 1 2024-02-19T19:05:03.400 INFO:tasks.workunit.client.1.smithi132.stdout:2/888: creat d0/d3c/df1/f120 x:0 0 0 2024-02-19T19:05:03.401 INFO:tasks.workunit.client.1.smithi132.stdout:5/981: stat d8/da/d5a/l3d 0 2024-02-19T19:05:03.402 INFO:tasks.workunit.client.1.smithi132.stdout:7/927: mknod d1b/d1e/d41/da3/dd5/c123 0 2024-02-19T19:05:03.403 INFO:tasks.workunit.client.1.smithi132.stdout:8/947: creat d4/d2d/dcc/f148 x:0 0 0 2024-02-19T19:05:03.403 INFO:tasks.workunit.client.1.smithi132.stdout:7/928: chown d1b/d1e/d41 210 1 2024-02-19T19:05:03.404 INFO:tasks.workunit.client.1.smithi132.stdout:0/853: dwrite d2/dc/df/d15/f4b [0,4194304] 0 2024-02-19T19:05:03.413 INFO:tasks.workunit.client.1.smithi132.stdout:8/948: dwrite d4/dc/d1b/d38/d126/d128/fe0 [0,4194304] 0 2024-02-19T19:05:03.413 INFO:tasks.workunit.client.1.smithi132.stdout:0/854: dwrite d2/d6e/fac [0,4194304] 0 2024-02-19T19:05:03.415 INFO:tasks.workunit.client.1.smithi132.stdout:4/851: dread d4/d27/d31/d32/d51/d5c/f5d [0,4194304] 0 2024-02-19T19:05:03.419 INFO:tasks.workunit.client.1.smithi132.stdout:0/855: truncate d2/dc/df/d15/d2f/d69/f88 4701545 0 2024-02-19T19:05:03.420 INFO:tasks.workunit.client.1.smithi132.stdout:0/856: write d2/dc/d4a/d74/f11a [637736,61565] 0 2024-02-19T19:05:03.424 INFO:tasks.workunit.client.1.smithi132.stdout:8/949: dread d4/fab [0,4194304] 0 2024-02-19T19:05:03.427 INFO:tasks.workunit.client.1.smithi132.stdout:0/857: dwrite d2/d3e/ff4 [0,4194304] 0 2024-02-19T19:05:03.445 INFO:tasks.workunit.client.1.smithi132.stdout:3/908: link d8/d13/d2b/d69/d92/daf/ff6 d8/d38/d44/f11c 0 2024-02-19T19:05:03.452 INFO:tasks.workunit.client.1.smithi132.stdout:4/852: creat d4/d19/dd4/f12f x:0 0 0 2024-02-19T19:05:03.455 INFO:tasks.workunit.client.1.smithi132.stdout:8/950: creat d4/dc/d1b/d38/d126/d68/d79/db4/f149 x:0 0 0 2024-02-19T19:05:03.459 INFO:tasks.workunit.client.1.smithi132.stdout:9/999: rename d12/d8d/d3a/ddc to d12/d2a/d158 0 2024-02-19T19:05:03.461 INFO:tasks.workunit.client.1.smithi132.stdout:5/982: unlink d8/d2e/dc4/d113/d11a/c14e 0 2024-02-19T19:05:03.462 INFO:tasks.workunit.client.1.smithi132.stdout:7/929: write d1b/d1e/d28/f35 [4588809,34108] 0 2024-02-19T19:05:03.463 INFO:tasks.workunit.client.1.smithi132.stdout:4/853: creat d4/d27/d31/d71/d14/d24/d49/d10e/f130 x:0 0 0 2024-02-19T19:05:03.464 INFO:tasks.workunit.client.1.smithi132.stdout:8/951: mkdir d4/d2d/dcc/d129/d14a 0 2024-02-19T19:05:03.467 INFO:tasks.workunit.client.1.smithi132.stdout:4/854: write d4/d27/d73/fa6 [3552364,94681] 0 2024-02-19T19:05:03.467 INFO:tasks.workunit.client.1.smithi132.stdout:2/889: dwrite d0/d24/f2d [0,4194304] 0 2024-02-19T19:05:03.468 INFO:tasks.workunit.client.1.smithi132.stdout:7/930: write d1b/d1e/d41/f6d [2289765,23772] 0 2024-02-19T19:05:03.468 INFO:tasks.workunit.client.1.smithi132.stdout:5/983: read d8/d1a/f9d [1402489,1391] 0 2024-02-19T19:05:03.472 INFO:tasks.workunit.client.1.smithi132.stdout:0/858: write d2/f109 [726370,98699] 0 2024-02-19T19:05:03.472 INFO:tasks.workunit.client.1.smithi132.stdout:4/855: mkdir d4/d12/d16/ddb/d131 0 2024-02-19T19:05:03.474 INFO:tasks.workunit.client.1.smithi132.stdout:8/952: rename d4/c65 to d4/c14b 0 2024-02-19T19:05:03.474 INFO:tasks.workunit.client.1.smithi132.stdout:4/856: write d4/d27/d31/d32/d51/d5c/fa8 [1390277,29130] 0 2024-02-19T19:05:03.474 INFO:tasks.workunit.client.1.smithi132.stdout:7/931: mknod d1b/d1e/d28/d5e/c124 0 2024-02-19T19:05:03.477 INFO:tasks.workunit.client.1.smithi132.stdout:5/984: unlink d8/d2e/dc4/d113/d11a/f132 0 2024-02-19T19:05:03.477 INFO:tasks.workunit.client.1.smithi132.stdout:3/909: dread d8/d93/f3b [0,4194304] 0 2024-02-19T19:05:03.478 INFO:tasks.workunit.client.1.smithi132.stdout:0/859: dread d2/dc/df/d15/fc3 [0,4194304] 0 2024-02-19T19:05:03.480 INFO:tasks.workunit.client.1.smithi132.stdout:7/932: stat d1b/d32/c33 0 2024-02-19T19:05:03.480 INFO:tasks.workunit.client.1.smithi132.stdout:0/860: read d2/dc/df/d15/d34/d84/da7/fb3 [1395318,26674] 0 2024-02-19T19:05:03.482 INFO:tasks.workunit.client.1.smithi132.stdout:0/861: readlink d2/dc/d4a/d25/d43/lbe 0 2024-02-19T19:05:03.484 INFO:tasks.workunit.client.1.smithi132.stdout:0/862: truncate d2/dc/df/d15/d2f/f64 5977136 0 2024-02-19T19:05:03.485 INFO:tasks.workunit.client.1.smithi132.stdout:3/910: rmdir d8/d13 39 2024-02-19T19:05:03.486 INFO:tasks.workunit.client.1.smithi132.stdout:8/953: fdatasync d4/dc/d1b/d54/f87 0 2024-02-19T19:05:03.487 INFO:tasks.workunit.client.1.smithi132.stdout:7/933: dwrite d1b/d1e/d2b/d8f/dae/f114 [0,4194304] 0 2024-02-19T19:05:03.488 INFO:tasks.workunit.client.1.smithi132.stdout:2/890: rename d0/d24/d58/d53/dab/lec to d0/d24/dd0/l121 0 2024-02-19T19:05:03.490 INFO:tasks.workunit.client.1.smithi132.stdout:7/934: write d1b/d46/f110 [905598,44843] 0 2024-02-19T19:05:03.490 INFO:tasks.workunit.client.1.smithi132.stdout:4/857: creat d4/d19/d21/d56/f132 x:0 0 0 2024-02-19T19:05:03.490 INFO:tasks.workunit.client.1.smithi132.stdout:8/954: fdatasync d4/f83 0 2024-02-19T19:05:03.492 INFO:tasks.workunit.client.1.smithi132.stdout:8/955: fsync d4/dc/d1b/d38/d126/d68/d7a/f10f 0 2024-02-19T19:05:03.492 INFO:tasks.workunit.client.1.smithi132.stdout:0/863: mkdir d2/dc/df/d15/d34/d11b/d11c 0 2024-02-19T19:05:03.493 INFO:tasks.workunit.client.1.smithi132.stdout:0/864: stat d2/dc/d4a/d25/dda/cf0 0 2024-02-19T19:05:03.494 INFO:tasks.workunit.client.1.smithi132.stdout:3/911: mknod d8/d13/d2b/c11d 0 2024-02-19T19:05:03.494 INFO:tasks.workunit.client.1.smithi132.stdout:3/912: chown d8/l35 3 1 2024-02-19T19:05:03.495 INFO:tasks.workunit.client.1.smithi132.stdout:0/865: write d2/dc/d4a/d25/d43/d5f/f104 [1701883,21644] 0 2024-02-19T19:05:03.498 INFO:tasks.workunit.client.1.smithi132.stdout:4/858: write d4/d27/d31/d71/d14/d24/f103 [4341292,17671] 0 2024-02-19T19:05:03.498 INFO:tasks.workunit.client.1.smithi132.stdout:8/956: fdatasync d4/dc/d1b/d38/fc6 0 2024-02-19T19:05:03.500 INFO:tasks.workunit.client.1.smithi132.stdout:4/859: read - d4/fee zero size 2024-02-19T19:05:03.501 INFO:tasks.workunit.client.1.smithi132.stdout:2/891: creat d0/d4/f122 x:0 0 0 2024-02-19T19:05:03.501 INFO:tasks.workunit.client.1.smithi132.stdout:3/913: mknod d8/d13/d2b/d69/d7e/dd6/c11e 0 2024-02-19T19:05:03.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:03 smithi132 ceph-mon[37969]: pgmap v5: 65 pgs: 65 active+clean; 1.5 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail 2024-02-19T19:05:03.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:03 smithi132 ceph-mon[37969]: mgrmap e38: smithi088.gyviqq(active, since 4s), standbys: smithi132.psllsa 2024-02-19T19:05:03.503 INFO:tasks.workunit.client.1.smithi132.stdout:8/957: chown d4/dc/d1b/d38/d69/d74/dce/fe2 3228 1 2024-02-19T19:05:03.504 INFO:tasks.workunit.client.1.smithi132.stdout:4/860: fdatasync d4/d27/d7f/f108 0 2024-02-19T19:05:03.505 INFO:tasks.workunit.client.1.smithi132.stdout:7/935: getdents d1b/d1e/d2b/dd0 0 2024-02-19T19:05:03.505 INFO:tasks.workunit.client.1.smithi132.stdout:0/866: dwrite d2/dc/df/dbd/fed [0,4194304] 0 2024-02-19T19:05:03.505 INFO:tasks.workunit.client.1.smithi132.stdout:2/892: rmdir d0/d19/d6a 39 2024-02-19T19:05:03.507 INFO:tasks.workunit.client.1.smithi132.stdout:8/958: write d4/dc/f2c [1216326,129793] 0 2024-02-19T19:05:03.507 INFO:tasks.workunit.client.1.smithi132.stdout:3/914: symlink d8/d13/d2b/d69/d7e/dd6/l11f 0 2024-02-19T19:05:03.512 INFO:tasks.workunit.client.1.smithi132.stdout:4/861: dread d4/f7d [0,4194304] 0 2024-02-19T19:05:03.512 INFO:tasks.workunit.client.1.smithi132.stdout:8/959: unlink d4/dc/d1b/d38/fc6 0 2024-02-19T19:05:03.512 INFO:tasks.workunit.client.1.smithi132.stdout:4/862: truncate d4/fd6 277240 0 2024-02-19T19:05:03.513 INFO:tasks.workunit.client.1.smithi132.stdout:8/960: readlink d4/dc/d1b/d20/d62/d91/l37 0 2024-02-19T19:05:03.513 INFO:tasks.workunit.client.1.smithi132.stdout:8/961: stat f2 0 2024-02-19T19:05:03.517 INFO:tasks.workunit.client.1.smithi132.stdout:0/867: rename d2/dc/df/d15 to d2/dc/d4a/d25/dda/d11d 0 2024-02-19T19:05:03.520 INFO:tasks.workunit.client.1.smithi132.stdout:4/863: unlink d4/d27/d7f/dc0/fcc 0 2024-02-19T19:05:03.525 INFO:tasks.workunit.client.1.smithi132.stdout:0/868: rename d2/dc/d4a/d25/dda/d11d/d2f/la5 to d2/dc/d4a/d25/d43/d5f/dfe/dff/l11e 0 2024-02-19T19:05:03.526 INFO:tasks.workunit.client.1.smithi132.stdout:0/869: write d2/dc/d4a/d25/dda/d11d/d2f/f64 [4646646,72204] 0 2024-02-19T19:05:03.527 INFO:tasks.workunit.client.1.smithi132.stdout:7/936: write d1b/d5a/d63/d96/fad [240076,74063] 0 2024-02-19T19:05:03.530 INFO:tasks.workunit.client.1.smithi132.stdout:2/893: getdents d0/d64 0 2024-02-19T19:05:03.533 INFO:tasks.workunit.client.1.smithi132.stdout:3/915: link d8/d13/d2b/d69/d92/d107/lf9 d8/d13/d2b/d60/dc2/l120 0 2024-02-19T19:05:03.534 INFO:tasks.workunit.client.1.smithi132.stdout:0/870: symlink d2/dc/df/d27/d3a/l11f 0 2024-02-19T19:05:03.535 INFO:tasks.workunit.client.1.smithi132.stdout:0/871: fsync d2/dc/df/d78/f107 0 2024-02-19T19:05:03.535 INFO:tasks.workunit.client.1.smithi132.stdout:4/864: symlink d4/d19/d21/d56/l133 0 2024-02-19T19:05:03.536 INFO:tasks.workunit.client.1.smithi132.stdout:4/865: write d4/d27/d7f/f114 [764814,45419] 0 2024-02-19T19:05:03.537 INFO:tasks.workunit.client.1.smithi132.stdout:7/937: mknod d1b/d5a/c125 0 2024-02-19T19:05:03.538 INFO:tasks.workunit.client.1.smithi132.stdout:2/894: creat d0/d4/df/d40/d7b/f123 x:0 0 0 2024-02-19T19:05:03.539 INFO:tasks.workunit.client.1.smithi132.stdout:0/872: chown d2/dc/df/dbd/c105 55 1 2024-02-19T19:05:03.540 INFO:tasks.workunit.client.1.smithi132.stdout:8/962: dread d4/dc/d1b/d8f/daf/fcf [0,4194304] 0 2024-02-19T19:05:03.540 INFO:tasks.workunit.client.1.smithi132.stdout:4/866: mkdir d4/d12/da4/d134 0 2024-02-19T19:05:03.541 INFO:tasks.workunit.client.1.smithi132.stdout:3/916: symlink d8/d13/d2b/d60/db8/dfb/l121 0 2024-02-19T19:05:03.542 INFO:tasks.workunit.client.1.smithi132.stdout:7/938: mkdir d1b/d1e/d11e/d79/d126 0 2024-02-19T19:05:03.543 INFO:tasks.workunit.client.1.smithi132.stdout:0/873: unlink d2/dc/d4a/d58/f10e 0 2024-02-19T19:05:03.543 INFO:tasks.workunit.client.1.smithi132.stdout:2/895: mkdir d0/d64/dc8/d124 0 2024-02-19T19:05:03.544 INFO:tasks.workunit.client.1.smithi132.stdout:8/963: creat d4/dc/d1b/d38/dfa/f14c x:0 0 0 2024-02-19T19:05:03.545 INFO:tasks.workunit.client.1.smithi132.stdout:2/896: chown d0/d4/df/d40/d68/d102/c106 6127783 1 2024-02-19T19:05:03.545 INFO:tasks.workunit.client.1.smithi132.stdout:4/867: symlink d4/d19/d21/dfa/l135 0 2024-02-19T19:05:03.547 INFO:tasks.workunit.client.1.smithi132.stdout:3/917: mkdir d8/d13/d2b/d60/db8/d122 0 2024-02-19T19:05:03.548 INFO:tasks.workunit.client.1.smithi132.stdout:2/897: fsync d0/d24/d58/d53/dab/ff2 0 2024-02-19T19:05:03.549 INFO:tasks.workunit.client.1.smithi132.stdout:4/868: rmdir d4/d27/d31/d71/d14/de4 39 2024-02-19T19:05:03.550 INFO:tasks.workunit.client.1.smithi132.stdout:0/874: mknod d2/dc/d4a/d25/dda/d11d/d2f/d3f/d119/c120 0 2024-02-19T19:05:03.551 INFO:tasks.workunit.client.1.smithi132.stdout:3/918: dread d8/d13/d2b/d69/d92/daf/f10f [0,4194304] 0 2024-02-19T19:05:03.553 INFO:tasks.workunit.client.1.smithi132.stdout:4/869: symlink d4/d12/de8/l136 0 2024-02-19T19:05:03.554 INFO:tasks.workunit.client.1.smithi132.stdout:7/939: link d1b/d1e/c23 d1b/d1e/d28/da9/c127 0 2024-02-19T19:05:03.555 INFO:tasks.workunit.client.1.smithi132.stdout:0/875: mknod d2/dc/df/d27/d3a/c121 0 2024-02-19T19:05:03.556 INFO:tasks.workunit.client.1.smithi132.stdout:0/876: write d2/dc/df/f71 [419657,91370] 0 2024-02-19T19:05:03.559 INFO:tasks.workunit.client.1.smithi132.stdout:4/870: unlink d4/d27/d31/d32/d51/d72/lae 0 2024-02-19T19:05:03.560 INFO:tasks.workunit.client.1.smithi132.stdout:4/871: fsync d4/d19/daa/f11a 0 2024-02-19T19:05:03.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:03 smithi088 ceph-mon[88643]: pgmap v5: 65 pgs: 65 active+clean; 1.5 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail 2024-02-19T19:05:03.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:03 smithi088 ceph-mon[88643]: mgrmap e38: smithi088.gyviqq(active, since 4s), standbys: smithi132.psllsa 2024-02-19T19:05:03.621 INFO:tasks.workunit.client.1.smithi132.stdout:8/964: getdents d4 0 2024-02-19T19:05:03.621 INFO:tasks.workunit.client.1.smithi132.stdout:2/898: link d0/d4/df/d40/d68/f86 d0/d4/df/f125 0 2024-02-19T19:05:03.623 INFO:tasks.workunit.client.1.smithi132.stdout:0/877: dwrite d2/dc/d4a/d25/dda/d11d/d2f/d3f/d63/fe1 [0,4194304] 0 2024-02-19T19:05:03.625 INFO:tasks.workunit.client.1.smithi132.stdout:8/965: truncate d4/fab 618081 0 2024-02-19T19:05:03.626 INFO:tasks.workunit.client.1.smithi132.stdout:4/872: dread d4/d27/d31/d32/d51/f97 [0,4194304] 0 2024-02-19T19:05:03.627 INFO:tasks.workunit.client.1.smithi132.stdout:0/878: read d2/dc/d4a/d74/f85 [43859,45568] 0 2024-02-19T19:05:03.628 INFO:tasks.workunit.client.1.smithi132.stdout:3/919: dread d8/d38/f8b [0,4194304] 0 2024-02-19T19:05:03.628 INFO:tasks.workunit.client.1.smithi132.stdout:3/920: chown d8/d13/l82 1 1 2024-02-19T19:05:03.629 INFO:tasks.workunit.client.1.smithi132.stdout:2/899: mknod d0/d24/c126 0 2024-02-19T19:05:03.633 INFO:tasks.workunit.client.1.smithi132.stdout:8/966: unlink d4/d2d/c39 0 2024-02-19T19:05:03.639 INFO:tasks.workunit.client.1.smithi132.stdout:2/900: mkdir d0/d24/d58/d53/dab/d8a/d127 0 2024-02-19T19:05:03.640 INFO:tasks.workunit.client.1.smithi132.stdout:8/967: rmdir d4/dc/d1b/d38/d126/da3 39 2024-02-19T19:05:03.641 INFO:tasks.workunit.client.1.smithi132.stdout:2/901: fdatasync d0/d4/d11/d2e/d5a/f104 0 2024-02-19T19:05:03.642 INFO:tasks.workunit.client.1.smithi132.stdout:0/879: dwrite d2/dc/df/f112 [0,4194304] 0 2024-02-19T19:05:03.642 INFO:tasks.workunit.client.1.smithi132.stdout:4/873: symlink d4/d27/d31/d32/d51/l137 0 2024-02-19T19:05:03.642 INFO:tasks.workunit.client.1.smithi132.stdout:8/968: write d4/dac/fae [330057,88164] 0 2024-02-19T19:05:03.646 INFO:tasks.workunit.client.1.smithi132.stdout:0/880: fsync d2/dc/df/d27/d3a/d52/da8/f111 0 2024-02-19T19:05:03.646 INFO:tasks.workunit.client.1.smithi132.stdout:3/921: read d8/d38/d44/f5e [5669355,21582] 0 2024-02-19T19:05:03.652 INFO:tasks.workunit.client.1.smithi132.stdout:4/874: unlink d4/d27/d31/d32/d51/d5c/dbc/f104 0 2024-02-19T19:05:03.652 INFO:tasks.workunit.client.1.smithi132.stdout:3/922: dwrite d8/d13/d2b/d60/dc2/ff4 [0,4194304] 0 2024-02-19T19:05:03.656 INFO:tasks.workunit.client.1.smithi132.stdout:0/881: read d2/dc/d4a/f42 [3465324,121783] 0 2024-02-19T19:05:03.657 INFO:tasks.workunit.client.1.smithi132.stdout:2/902: mkdir d0/d4/d11/dee/d128 0 2024-02-19T19:05:03.666 INFO:tasks.workunit.client.1.smithi132.stdout:0/882: creat d2/dc/d4a/d25/dda/d11d/d34/f122 x:0 0 0 2024-02-19T19:05:03.668 INFO:tasks.workunit.client.1.smithi132.stdout:1/794: sync 2024-02-19T19:05:03.668 INFO:tasks.workunit.client.1.smithi132.stdout:0/883: chown d2/dc/d4a/d25/dda/d11d/d2f/d3f/d119 93632600 1 2024-02-19T19:05:03.668 INFO:tasks.workunit.client.1.smithi132.stdout:2/903: fsync d0/d4/f8e 0 2024-02-19T19:05:03.668 INFO:tasks.workunit.client.1.smithi132.stdout:4/875: symlink d4/d12/d16/ddb/d131/l138 0 2024-02-19T19:05:03.670 INFO:tasks.workunit.client.1.smithi132.stdout:1/795: write d1/d3a/d50/f67 [1301671,48059] 0 2024-02-19T19:05:03.670 INFO:tasks.workunit.client.1.smithi132.stdout:2/904: write d0/d4/f112 [636749,23419] 0 2024-02-19T19:05:03.671 INFO:tasks.workunit.client.1.smithi132.stdout:8/969: link d4/c14b d4/dc/d1b/d54/d5e/c14d 0 2024-02-19T19:05:03.671 INFO:tasks.workunit.client.1.smithi132.stdout:4/876: fsync d4/d27/d31/d32/d51/d5c/f101 0 2024-02-19T19:05:03.674 INFO:tasks.workunit.client.1.smithi132.stdout:0/884: creat d2/dc/d4a/d25/dda/d11d/d34/f123 x:0 0 0 2024-02-19T19:05:03.676 INFO:tasks.workunit.client.1.smithi132.stdout:2/905: dread d0/d24/d58/d53/dab/fe4 [0,4194304] 0 2024-02-19T19:05:03.728 INFO:tasks.workunit.client.1.smithi132.stdout:0/885: rename d2/dc/d4a/d25 to d2/dc/df/d27/de8/d124 0 2024-02-19T19:05:03.731 INFO:tasks.workunit.client.1.smithi132.stdout:8/970: dwrite d4/dc/d1b/d20/f59 [0,4194304] 0 2024-02-19T19:05:03.732 INFO:tasks.workunit.client.1.smithi132.stdout:0/886: write d2/dc/df/d27/de8/d124/d43/d5f/f87 [170635,69216] 0 2024-02-19T19:05:03.735 INFO:tasks.workunit.client.1.smithi132.stdout:3/923: dwrite d8/d13/d2b/d69/fd3 [0,4194304] 0 2024-02-19T19:05:03.735 INFO:tasks.workunit.client.1.smithi132.stdout:8/971: write d4/d2d/d82/f98 [3928885,47648] 0 2024-02-19T19:05:03.739 INFO:tasks.workunit.client.1.smithi132.stdout:4/877: dread d4/d27/d7f/fa9 [0,4194304] 0 2024-02-19T19:05:03.747 INFO:tasks.workunit.client.1.smithi132.stdout:3/924: creat d8/d13/d2b/d69/d92/d8d/dd7/f123 x:0 0 0 2024-02-19T19:05:03.750 INFO:tasks.workunit.client.1.smithi132.stdout:2/906: mkdir d0/d24/dd0/d129 0 2024-02-19T19:05:03.751 INFO:tasks.workunit.client.1.smithi132.stdout:8/972: creat d4/dc/d1b/d38/d126/da3/f14e x:0 0 0 2024-02-19T19:05:03.752 INFO:tasks.workunit.client.1.smithi132.stdout:8/973: fsync d4/dc/d42/f13a 0 2024-02-19T19:05:03.753 INFO:tasks.workunit.client.1.smithi132.stdout:4/878: creat d4/d27/f139 x:0 0 0 2024-02-19T19:05:03.754 INFO:tasks.workunit.client.1.smithi132.stdout:3/925: creat d8/d38/d44/f124 x:0 0 0 2024-02-19T19:05:03.755 INFO:tasks.workunit.client.1.smithi132.stdout:5/985: sync 2024-02-19T19:05:03.756 INFO:tasks.workunit.client.1.smithi132.stdout:6/834: sync 2024-02-19T19:05:03.756 INFO:tasks.workunit.client.1.smithi132.stdout:8/974: mknod d4/dc/d1b/d38/d69/d74/c14f 0 2024-02-19T19:05:03.757 INFO:tasks.workunit.client.1.smithi132.stdout:4/879: chown d4/d12/d48/d5f/f113 9075 1 2024-02-19T19:05:03.760 INFO:tasks.workunit.client.1.smithi132.stdout:5/986: unlink d8/l4e 0 2024-02-19T19:05:03.760 INFO:tasks.workunit.client.1.smithi132.stdout:2/907: getdents d0/d24/d58/d53/dab/d8a/da5 0 2024-02-19T19:05:03.761 INFO:tasks.workunit.client.1.smithi132.stdout:8/975: creat d4/dc/d1b/d54/d81/f150 x:0 0 0 2024-02-19T19:05:03.762 INFO:tasks.workunit.client.1.smithi132.stdout:3/926: getdents d8/d13/d2b/d69/d92/d34/d42/d6c/de6/d119 0 2024-02-19T19:05:03.762 INFO:tasks.workunit.client.1.smithi132.stdout:8/976: dread - d4/dc/d1b/d54/d5e/fdc zero size 2024-02-19T19:05:03.764 INFO:tasks.workunit.client.1.smithi132.stdout:6/835: rename d0/d51/da7/lc0 to d0/db/d25/d3a/d4b/d7c/da2/l122 0 2024-02-19T19:05:03.764 INFO:tasks.workunit.client.1.smithi132.stdout:8/977: readlink d4/dc/d1b/d54/d81/dbd/ld8 0 2024-02-19T19:05:03.764 INFO:tasks.workunit.client.1.smithi132.stdout:8/978: chown d4/dc/fa9 159516731 1 2024-02-19T19:05:03.765 INFO:tasks.workunit.client.1.smithi132.stdout:5/987: mknod d8/d1a/d35/d129/d149/c153 0 2024-02-19T19:05:03.768 INFO:tasks.workunit.client.1.smithi132.stdout:3/927: mknod d8/d13/d2b/d60/c125 0 2024-02-19T19:05:03.772 INFO:tasks.workunit.client.1.smithi132.stdout:8/979: symlink d4/dc/d1b/d54/l151 0 2024-02-19T19:05:03.773 INFO:tasks.workunit.client.1.smithi132.stdout:3/928: mkdir d8/d13/d2b/d69/d92/d126 0 2024-02-19T19:05:03.774 INFO:tasks.workunit.client.1.smithi132.stdout:2/908: dwrite d0/d19/d6a/fe8 [0,4194304] 0 2024-02-19T19:05:03.775 INFO:tasks.workunit.client.1.smithi132.stdout:6/836: link d0/db/d25/d3a/d3d/fb5 d0/db/d25/d50/f123 0 2024-02-19T19:05:03.777 INFO:tasks.workunit.client.1.smithi132.stdout:8/980: creat d4/d2d/dcc/dfc/f152 x:0 0 0 2024-02-19T19:05:03.783 INFO:tasks.workunit.client.1.smithi132.stdout:6/837: link d0/db/d25/d3a/d5d/d8f/f39 d0/db/d25/d3a/d5d/d8f/dbe/f124 0 2024-02-19T19:05:03.784 INFO:tasks.workunit.client.1.smithi132.stdout:8/981: write d4/dc/d1b/d38/d126/d68/d79/f139 [809560,80394] 0 2024-02-19T19:05:03.785 INFO:tasks.workunit.client.1.smithi132.stdout:6/838: chown d0/db/d25/d3a/d4b/ld2 2699885 1 2024-02-19T19:05:03.790 INFO:tasks.workunit.client.1.smithi132.stdout:6/839: read d0/db/d25/d3a/d5d/d8f/d46/d5a/f1d [2399182,66777] 0 2024-02-19T19:05:03.790 INFO:tasks.workunit.client.1.smithi132.stdout:7/940: sync 2024-02-19T19:05:03.791 INFO:tasks.workunit.client.1.smithi132.stdout:4/880: write d4/d27/d31/d32/d51/d72/f85 [853389,109036] 0 2024-02-19T19:05:03.794 INFO:tasks.workunit.client.1.smithi132.stdout:0/887: dwrite d2/dc/df/d27/de8/d124/dda/d11d/d2f/d69/fb9 [0,4194304] 0 2024-02-19T19:05:03.795 INFO:tasks.workunit.client.1.smithi132.stdout:0/888: read - d2/f114 zero size 2024-02-19T19:05:03.796 INFO:tasks.workunit.client.1.smithi132.stdout:7/941: mknod d1b/d1e/d28/c128 0 2024-02-19T19:05:03.799 INFO:tasks.workunit.client.1.smithi132.stdout:2/909: write d0/d4/fa9 [715210,64900] 0 2024-02-19T19:05:03.800 INFO:tasks.workunit.client.1.smithi132.stdout:5/988: dwrite d8/d2e/fbe [0,4194304] 0 2024-02-19T19:05:03.802 INFO:tasks.workunit.client.1.smithi132.stdout:6/840: creat d0/db/d25/d3a/d3d/d69/de2/f125 x:0 0 0 2024-02-19T19:05:03.803 INFO:tasks.workunit.client.1.smithi132.stdout:4/881: mkdir d4/d19/d13a 0 2024-02-19T19:05:03.806 INFO:tasks.workunit.client.1.smithi132.stdout:2/910: creat d0/d24/daf/f12a x:0 0 0 2024-02-19T19:05:03.807 INFO:tasks.workunit.client.1.smithi132.stdout:6/841: read - d0/db/d25/d3a/d5d/d8f/d46/ffc zero size 2024-02-19T19:05:03.807 INFO:tasks.workunit.client.1.smithi132.stdout:7/942: rename d1b/d32/f117 to d1b/d1e/d28/da9/dd3/ddb/f129 0 2024-02-19T19:05:03.809 INFO:tasks.workunit.client.1.smithi132.stdout:2/911: write d0/d4/d11/d2e/d5a/fd4 [1119530,67688] 0 2024-02-19T19:05:03.810 INFO:tasks.workunit.client.1.smithi132.stdout:4/882: unlink d4/d27/d31/d32/d51/f93 0 2024-02-19T19:05:03.810 INFO:tasks.workunit.client.1.smithi132.stdout:0/889: write d2/dc/df/d27/f7c [987976,54556] 0 2024-02-19T19:05:03.812 INFO:tasks.workunit.client.1.smithi132.stdout:5/989: write d8/da/d5a/d7b/f88 [714727,6888] 0 2024-02-19T19:05:03.814 INFO:tasks.workunit.client.1.smithi132.stdout:4/883: mknod d4/d19/daa/dc7/c13b 0 2024-02-19T19:05:03.818 INFO:tasks.workunit.client.1.smithi132.stdout:0/890: creat d2/dc/df/d27/de8/d124/d7f/f125 x:0 0 0 2024-02-19T19:05:03.818 INFO:tasks.workunit.client.1.smithi132.stdout:7/943: write d1b/f1c [5142515,65301] 0 2024-02-19T19:05:03.819 INFO:tasks.workunit.client.1.smithi132.stdout:4/884: dread - d4/d4d/d87/fd5 zero size 2024-02-19T19:05:03.821 INFO:tasks.workunit.client.1.smithi132.stdout:7/944: chown d1b/d1e/d41/f47 121 1 2024-02-19T19:05:03.821 INFO:tasks.workunit.client.1.smithi132.stdout:6/842: dread d0/db/d25/d3a/d5d/d8f/d46/d5a/f37 [0,4194304] 0 2024-02-19T19:05:03.823 INFO:tasks.workunit.client.1.smithi132.stdout:6/843: readlink d0/db/l6b 0 2024-02-19T19:05:03.824 INFO:tasks.workunit.client.1.smithi132.stdout:2/912: dwrite d0/d24/f117 [0,4194304] 0 2024-02-19T19:05:03.827 INFO:tasks.workunit.client.1.smithi132.stdout:0/891: creat d2/dc/df/d27/de8/d124/d43/d5f/dfe/f126 x:0 0 0 2024-02-19T19:05:03.829 INFO:tasks.workunit.client.1.smithi132.stdout:7/945: creat d1b/d1e/d41/da3/de5/f12a x:0 0 0 2024-02-19T19:05:03.829 INFO:tasks.workunit.client.1.smithi132.stdout:5/990: dwrite d8/da/db/f109 [0,4194304] 0 2024-02-19T19:05:03.829 INFO:tasks.workunit.client.1.smithi132.stdout:4/885: dread d4/d27/d7f/f114 [0,4194304] 0 2024-02-19T19:05:03.831 INFO:tasks.workunit.client.1.smithi132.stdout:7/946: readlink d1b/d32/l7e 0 2024-02-19T19:05:03.831 INFO:tasks.workunit.client.1.smithi132.stdout:0/892: write d2/dc/f6d [5042602,15940] 0 2024-02-19T19:05:03.836 INFO:tasks.workunit.client.1.smithi132.stdout:7/947: mkdir d1b/d1e/d41/da3/dbd/d11c/d12b 0 2024-02-19T19:05:03.838 INFO:tasks.workunit.client.1.smithi132.stdout:5/991: creat d8/d1a/d64/f154 x:0 0 0 2024-02-19T19:05:03.838 INFO:tasks.workunit.client.1.smithi132.stdout:4/886: write d4/d27/d31/d71/d37/f96 [2644936,18435] 0 2024-02-19T19:05:03.839 INFO:tasks.workunit.client.1.smithi132.stdout:0/893: mkdir d2/dc/df/d27/d127 0 2024-02-19T19:05:03.840 INFO:tasks.workunit.client.1.smithi132.stdout:7/948: chown d1b/d1e/d2b/d8f/dae/ff5 2853318 1 2024-02-19T19:05:03.843 INFO:tasks.workunit.client.1.smithi132.stdout:7/949: write d1b/d1e/fd6 [14207,12667] 0 2024-02-19T19:05:03.843 INFO:tasks.workunit.client.1.smithi132.stdout:4/887: rmdir d4/d27/d31/d32/d51/d72 39 2024-02-19T19:05:03.844 INFO:tasks.workunit.client.1.smithi132.stdout:2/913: dread d0/d24/f87 [0,4194304] 0 2024-02-19T19:05:03.844 INFO:tasks.workunit.client.1.smithi132.stdout:0/894: truncate d2/d6e/d7a/fab 1465888 0 2024-02-19T19:05:03.845 INFO:tasks.workunit.client.1.smithi132.stdout:4/888: chown d4/d19/d21/c10a 1829 1 2024-02-19T19:05:03.846 INFO:tasks.workunit.client.1.smithi132.stdout:7/950: fdatasync d1b/d1e/d28/da9/dd3/ddb/ded/ff6 0 2024-02-19T19:05:03.846 INFO:tasks.workunit.client.1.smithi132.stdout:4/889: chown d4/d12/da4/d134 2 1 2024-02-19T19:05:03.847 INFO:tasks.workunit.client.1.smithi132.stdout:7/951: dread - d1b/d1e/d28/da9/dd3/ddb/ded/ff6 zero size 2024-02-19T19:05:03.847 INFO:tasks.workunit.client.1.smithi132.stdout:4/890: chown d4/d27/d31/d32/d51/d5c/f101 6389869 1 2024-02-19T19:05:03.848 INFO:tasks.workunit.client.1.smithi132.stdout:4/891: chown d4/f2d 10289 1 2024-02-19T19:05:03.848 INFO:tasks.workunit.client.1.smithi132.stdout:4/892: dread - d4/d27/d31/d32/d51/d5c/dbc/d109/f129 zero size 2024-02-19T19:05:03.860 INFO:tasks.workunit.client.1.smithi132.stdout:6/844: dread d0/db/d25/d3a/d5d/d8f/d46/d5a/f26 [0,4194304] 0 2024-02-19T19:05:03.860 INFO:tasks.workunit.client.1.smithi132.stdout:6/845: chown d0/d51/da7 179899520 1 2024-02-19T19:05:03.868 INFO:tasks.workunit.client.1.smithi132.stdout:2/914: dread d0/d19/d1e/f9f [4194304,4194304] 0 2024-02-19T19:05:03.869 INFO:tasks.workunit.client.1.smithi132.stdout:2/915: write d0/d24/d58/d53/dab/d8a/fc0 [1931312,47733] 0 2024-02-19T19:05:03.884 INFO:tasks.workunit.client.1.smithi132.stdout:0/895: dread - d2/d6e/d7a/f8f zero size 2024-02-19T19:05:03.886 INFO:tasks.workunit.client.1.smithi132.stdout:7/952: readlink d1b/l108 0 2024-02-19T19:05:03.886 INFO:tasks.workunit.client.1.smithi132.stdout:0/896: chown d2/dc/df/d27/de8/d124/dda/d11d/f24 7258447 1 2024-02-19T19:05:03.886 INFO:tasks.workunit.client.1.smithi132.stdout:4/893: fdatasync d4/d27/d31/d32/d51/d5c/dbc/d109/f129 0 2024-02-19T19:05:03.888 INFO:tasks.workunit.client.1.smithi132.stdout:4/894: chown d4/d27/d31/d71/d14/d24/d49/d10e 826 1 2024-02-19T19:05:03.888 INFO:tasks.workunit.client.1.smithi132.stdout:1/796: sync 2024-02-19T19:05:03.888 INFO:tasks.workunit.client.1.smithi132.stdout:6/846: mkdir d0/db/d25/d3a/d4b/d126 0 2024-02-19T19:05:03.890 INFO:tasks.workunit.client.1.smithi132.stdout:2/916: dread - d0/d24/d58/d53/fbf zero size 2024-02-19T19:05:03.892 INFO:tasks.workunit.client.1.smithi132.stdout:7/953: creat d1b/d1e/d11e/f12c x:0 0 0 2024-02-19T19:05:03.893 INFO:tasks.workunit.client.1.smithi132.stdout:0/897: creat d2/dc/df/d27/de8/d124/dda/d11d/d2f/d3f/dd3/f128 x:0 0 0 2024-02-19T19:05:03.895 INFO:tasks.workunit.client.1.smithi132.stdout:6/847: dwrite d0/db/fe6 [0,4194304] 0 2024-02-19T19:05:03.895 INFO:tasks.workunit.client.1.smithi132.stdout:1/797: readlink d1/d3a/l1a 0 2024-02-19T19:05:03.896 INFO:tasks.workunit.client.1.smithi132.stdout:1/798: write d1/d61/f7c [2774697,96214] 0 2024-02-19T19:05:03.896 INFO:tasks.workunit.client.1.smithi132.stdout:5/992: rename d8/d2e/dc4/dc8/l73 to d8/d2e/d71/l155 0 2024-02-19T19:05:03.897 INFO:tasks.workunit.client.1.smithi132.stdout:1/799: readlink d1/d3a/d50/l57 0 2024-02-19T19:05:03.898 INFO:tasks.workunit.client.1.smithi132.stdout:2/917: creat d0/d24/daf/f12b x:0 0 0 2024-02-19T19:05:03.901 INFO:tasks.workunit.client.1.smithi132.stdout:4/895: write d4/d27/d31/f79 [5721941,51045] 0 2024-02-19T19:05:03.903 INFO:tasks.workunit.client.1.smithi132.stdout:0/898: dwrite d2/dc/df/d27/de8/d124/dda/d11d/d2f/d69/fb9 [0,4194304] 0 2024-02-19T19:05:03.904 INFO:tasks.workunit.client.1.smithi132.stdout:7/954: getdents d1b/d1e/d41/da3/dbd/d11c/d12b 0 2024-02-19T19:05:03.905 INFO:tasks.workunit.client.1.smithi132.stdout:6/848: write d0/db/d25/d3a/d5d/d8f/d46/d5a/f26 [1202997,36185] 0 2024-02-19T19:05:03.909 INFO:tasks.workunit.client.1.smithi132.stdout:7/955: rmdir d1b/d1e/d41/da3/dbd 39 2024-02-19T19:05:03.910 INFO:tasks.workunit.client.1.smithi132.stdout:7/956: write d1b/d1e/d41/db5/fe6 [1405321,8236] 0 2024-02-19T19:05:03.912 INFO:tasks.workunit.client.1.smithi132.stdout:4/896: mkdir d4/d12/d9f/df5/d13c 0 2024-02-19T19:05:03.913 INFO:tasks.workunit.client.1.smithi132.stdout:2/918: creat d0/d24/d58/d53/dab/d84/de1/f12c x:0 0 0 2024-02-19T19:05:03.916 INFO:tasks.workunit.client.1.smithi132.stdout:2/919: chown d0/d19/f78 217175 1 2024-02-19T19:05:03.916 INFO:tasks.workunit.client.1.smithi132.stdout:1/800: creat d1/d1e/d77/d2c/ff2 x:0 0 0 2024-02-19T19:05:03.917 INFO:tasks.workunit.client.1.smithi132.stdout:0/899: creat d2/dc/df/d27/de8/d124/dda/d11d/d34/f129 x:0 0 0 2024-02-19T19:05:03.917 INFO:tasks.workunit.client.1.smithi132.stdout:5/993: link d8/d2e/dc4/dc8/l60 d8/d2e/dc4/d113/d11a/d139/l156 0 2024-02-19T19:05:03.918 INFO:tasks.workunit.client.1.smithi132.stdout:7/957: dwrite f8 [4194304,4194304] 0 2024-02-19T19:05:03.921 INFO:tasks.workunit.client.1.smithi132.stdout:4/897: rename d4/d27/d31/d32/d51/d5c to d4/d12/d4c/d13d 0 2024-02-19T19:05:03.922 INFO:tasks.workunit.client.1.smithi132.stdout:2/920: mkdir d0/d24/daf/d12d 0 2024-02-19T19:05:03.926 INFO:tasks.workunit.client.1.smithi132.stdout:5/994: dwrite d8/da/db/f109 [0,4194304] 0 2024-02-19T19:05:03.926 INFO:tasks.workunit.client.1.smithi132.stdout:1/801: mkdir d1/d1e/d77/d2c/d43/db8/df3 0 2024-02-19T19:05:03.927 INFO:tasks.workunit.client.1.smithi132.stdout:6/849: write d0/db/d25/d3a/d3d/f74 [920413,118225] 0 2024-02-19T19:05:03.928 INFO:tasks.workunit.client.1.smithi132.stdout:4/898: creat d4/d12/d48/d5f/f13e x:0 0 0 2024-02-19T19:05:03.930 INFO:tasks.workunit.client.1.smithi132.stdout:4/899: fsync d4/d19/d21/f11b 0 2024-02-19T19:05:03.930 INFO:tasks.workunit.client.1.smithi132.stdout:2/921: fsync d0/d4/fd1 0 2024-02-19T19:05:03.932 INFO:tasks.workunit.client.1.smithi132.stdout:1/802: unlink d1/d3a/d50/cae 0 2024-02-19T19:05:03.932 INFO:tasks.workunit.client.1.smithi132.stdout:7/958: symlink d1b/d32/l12d 0 2024-02-19T19:05:03.933 INFO:tasks.workunit.client.1.smithi132.stdout:5/995: mkdir d8/da/db/d157 0 2024-02-19T19:05:03.933 INFO:tasks.workunit.client.1.smithi132.stdout:4/900: mkdir d4/d12/d9f/d13f 0 2024-02-19T19:05:03.935 INFO:tasks.workunit.client.1.smithi132.stdout:1/803: write d1/d1e/d77/d2c/d62/d6b/fb6 [682742,122877] 0 2024-02-19T19:05:03.936 INFO:tasks.workunit.client.1.smithi132.stdout:4/901: chown d4/d19/daa/dc7/l67 120 1 2024-02-19T19:05:03.936 INFO:tasks.workunit.client.1.smithi132.stdout:0/900: link d2/dc/fa3 d2/dc/df/d27/de8/d124/dda/d11d/d2f/d69/f12a 0 2024-02-19T19:05:03.936 INFO:tasks.workunit.client.1.smithi132.stdout:7/959: write d1b/d1e/d28/da9/dd3/ddb/ded/ff6 [964701,103416] 0 2024-02-19T19:05:03.937 INFO:tasks.workunit.client.1.smithi132.stdout:3/929: sync 2024-02-19T19:05:03.937 INFO:tasks.workunit.client.1.smithi132.stdout:8/982: sync 2024-02-19T19:05:03.938 INFO:tasks.workunit.client.1.smithi132.stdout:5/996: creat d8/d1a/d64/de2/dec/f158 x:0 0 0 2024-02-19T19:05:03.940 INFO:tasks.workunit.client.1.smithi132.stdout:1/804: fsync d1/d1e/d77/d78/f7f 0 2024-02-19T19:05:03.942 INFO:tasks.workunit.client.1.smithi132.stdout:7/960: write d1b/d1e/d41/da3/dbd/d11c/f67 [38165,115149] 0 2024-02-19T19:05:03.942 INFO:tasks.workunit.client.1.smithi132.stdout:4/902: creat d4/d27/d31/d71/d14/d24/f140 x:0 0 0 2024-02-19T19:05:03.945 INFO:tasks.workunit.client.1.smithi132.stdout:5/997: read d8/da/d5a/fcb [1758696,24845] 0 2024-02-19T19:05:03.946 INFO:tasks.workunit.client.1.smithi132.stdout:3/930: write d8/d13/d2b/d69/d92/d107/fd0 [527384,114950] 0 2024-02-19T19:05:03.947 INFO:tasks.workunit.client.1.smithi132.stdout:0/901: dwrite d2/dc/df/d27/de8/d124/d43/d5f/f73 [0,4194304] 0 2024-02-19T19:05:03.948 INFO:tasks.workunit.client.1.smithi132.stdout:1/805: chown d1/d1e/d77/d2c/c33 0 1 2024-02-19T19:05:03.948 INFO:tasks.workunit.client.1.smithi132.stdout:7/961: write d1b/d1e/d28/f3b [5011385,5012] 0 2024-02-19T19:05:03.950 INFO:tasks.workunit.client.1.smithi132.stdout:4/903: creat d4/d27/d31/d71/d14/d24/f141 x:0 0 0 2024-02-19T19:05:03.951 INFO:tasks.workunit.client.1.smithi132.stdout:4/904: dread - d4/d4d/d87/fd5 zero size 2024-02-19T19:05:03.954 INFO:tasks.workunit.client.1.smithi132.stdout:8/983: mkdir d4/dc/d1b/d38/d69/d153 0 2024-02-19T19:05:03.956 INFO:tasks.workunit.client.1.smithi132.stdout:5/998: rename d8/da/f8c to d8/da/db/d157/f159 0 2024-02-19T19:05:03.957 INFO:tasks.workunit.client.1.smithi132.stdout:7/962: mknod d1b/d1e/d11e/c12e 0 2024-02-19T19:05:03.960 INFO:tasks.workunit.client.1.smithi132.stdout:4/905: creat d4/d27/d31/d71/d37/f142 x:0 0 0 2024-02-19T19:05:03.961 INFO:tasks.workunit.client.1.smithi132.stdout:5/999: fsync d8/d1a/f9d 0 2024-02-19T19:05:03.964 INFO:tasks.workunit.client.1.smithi132.stdout:4/906: fsync d4/d12/d4c/f10b 0 2024-02-19T19:05:03.964 INFO:tasks.workunit.client.1.smithi132.stdout:1/806: dwrite d1/d61/fa5 [0,4194304] 0 2024-02-19T19:05:03.964 INFO:tasks.workunit.client.1.smithi132.stdout:3/931: dread d8/d13/d2b/f99 [0,4194304] 0 2024-02-19T19:05:03.967 INFO:tasks.workunit.client.1.smithi132.stdout:2/922: dread d0/d19/f82 [0,4194304] 0 2024-02-19T19:05:03.970 INFO:tasks.workunit.client.1.smithi132.stdout:4/907: unlink d4/f7d 0 2024-02-19T19:05:03.972 INFO:tasks.workunit.client.1.smithi132.stdout:8/984: dwrite d4/dc/d42/f13a [0,4194304] 0 2024-02-19T19:05:03.972 INFO:tasks.workunit.client.1.smithi132.stdout:4/908: stat d4/d12/d48/fde 0 2024-02-19T19:05:03.975 INFO:tasks.workunit.client.1.smithi132.stdout:7/963: link d1b/d1e/d11e/d86/cdf d1b/d1e/d28/da9/dd3/ddb/ded/c12f 0 2024-02-19T19:05:03.976 INFO:tasks.workunit.client.1.smithi132.stdout:0/902: dwrite d2/dc/df/d27/de8/d124/d43/f6f [0,4194304] 0 2024-02-19T19:05:03.977 INFO:tasks.workunit.client.1.smithi132.stdout:1/807: write d1/d1e/d77/d2c/d43/d81/da1/fbb [3843333,96673] 0 2024-02-19T19:05:03.978 INFO:tasks.workunit.client.1.smithi132.stdout:2/923: symlink d0/d4/df/d40/d68/d102/l12e 0 2024-02-19T19:05:03.980 INFO:tasks.workunit.client.1.smithi132.stdout:2/924: read - d0/d24/f116 zero size 2024-02-19T19:05:03.982 INFO:tasks.workunit.client.1.smithi132.stdout:7/964: mknod d1b/d5a/d63/d98/c130 0 2024-02-19T19:05:03.984 INFO:tasks.workunit.client.1.smithi132.stdout:2/925: chown d0/d24/daf/f12b 420 1 2024-02-19T19:05:03.985 INFO:tasks.workunit.client.1.smithi132.stdout:7/965: fdatasync d1b/d1e/d28/f4f 0 2024-02-19T19:05:03.987 INFO:tasks.workunit.client.1.smithi132.stdout:2/926: mknod d0/d4/db9/c12f 0 2024-02-19T19:05:03.988 INFO:tasks.workunit.client.1.smithi132.stdout:1/808: mkdir d1/d1e/d77/d2c/d62/d6b/dba/df4 0 2024-02-19T19:05:03.988 INFO:tasks.workunit.client.1.smithi132.stdout:4/909: dread d4/d27/d31/d71/d14/de4/fe6 [0,4194304] 0 2024-02-19T19:05:03.991 INFO:tasks.workunit.client.1.smithi132.stdout:2/927: creat d0/d64/f130 x:0 0 0 2024-02-19T19:05:03.992 INFO:tasks.workunit.client.1.smithi132.stdout:4/910: mknod d4/d12/d48/d5f/c143 0 2024-02-19T19:05:03.993 INFO:tasks.workunit.client.1.smithi132.stdout:1/809: write d1/d1e/d77/d2c/d62/d6b/d85/fcf [21241,72761] 0 2024-02-19T19:05:03.994 INFO:tasks.workunit.client.1.smithi132.stdout:3/932: dread d8/d38/d44/f73 [0,4194304] 0 2024-02-19T19:05:03.994 INFO:tasks.workunit.client.1.smithi132.stdout:2/928: rmdir d0/d24/d58 39 2024-02-19T19:05:03.995 INFO:tasks.workunit.client.1.smithi132.stdout:7/966: dwrite d1b/d1e/d2b/f10f [0,4194304] 0 2024-02-19T19:05:03.997 INFO:tasks.workunit.client.1.smithi132.stdout:2/929: read d0/d3c/d3f/fd5 [1076431,94297] 0 2024-02-19T19:05:03.999 INFO:tasks.workunit.client.1.smithi132.stdout:7/967: creat d1b/d1e/d2b/dd0/f131 x:0 0 0 2024-02-19T19:05:04.004 INFO:tasks.workunit.client.1.smithi132.stdout:3/933: dwrite d8/d13/d2b/d69/da6/fc7 [0,4194304] 0 2024-02-19T19:05:04.007 INFO:tasks.workunit.client.1.smithi132.stdout:3/934: mkdir d8/d13/d2b/d69/d92/d8d/dd7/da2/d127 0 2024-02-19T19:05:04.013 INFO:tasks.workunit.client.1.smithi132.stdout:0/903: dwrite d2/dc/df/d27/de8/d124/dda/d11d/d2f/d69/f12a [0,4194304] 0 2024-02-19T19:05:04.013 INFO:tasks.workunit.client.1.smithi132.stdout:3/935: write d8/d13/d2b/d69/d92/d8d/dd7/f123 [976737,22740] 0 2024-02-19T19:05:04.015 INFO:tasks.workunit.client.1.smithi132.stdout:2/930: creat d0/d64/dc8/f131 x:0 0 0 2024-02-19T19:05:04.018 INFO:tasks.workunit.client.1.smithi132.stdout:3/936: creat d8/d13/d2b/d69/d92/d126/f128 x:0 0 0 2024-02-19T19:05:04.020 INFO:tasks.workunit.client.1.smithi132.stdout:2/931: chown d0/d24/d58/d53/dab/d8a/fa4 165 1 2024-02-19T19:05:04.020 INFO:tasks.workunit.client.1.smithi132.stdout:3/937: creat d8/d13/deb/f129 x:0 0 0 2024-02-19T19:05:04.023 INFO:tasks.workunit.client.1.smithi132.stdout:3/938: write d8/d13/d2b/d69/d92/d34/f79 [52649,99572] 0 2024-02-19T19:05:04.025 INFO:tasks.workunit.client.1.smithi132.stdout:1/810: dread d1/d1e/d77/f69 [0,4194304] 0 2024-02-19T19:05:04.027 INFO:tasks.workunit.client.1.smithi132.stdout:2/932: dwrite d0/d24/d58/d53/dab/d84/dc9/ff0 [0,4194304] 0 2024-02-19T19:05:04.029 INFO:tasks.workunit.client.1.smithi132.stdout:1/811: mknod d1/d1e/d77/cf5 0 2024-02-19T19:05:04.031 INFO:tasks.workunit.client.1.smithi132.stdout:1/812: fsync d1/d1e/d77/d52/d64/d75/fd6 0 2024-02-19T19:05:04.033 INFO:tasks.workunit.client.1.smithi132.stdout:1/813: write d1/d1e/d77/d2c/f7d [3789278,33777] 0 2024-02-19T19:05:04.035 INFO:tasks.workunit.client.1.smithi132.stdout:3/939: truncate d8/d13/d2b/d69/d92/d34/f89 120601 0 2024-02-19T19:05:04.036 INFO:tasks.workunit.client.1.smithi132.stdout:2/933: dwrite d0/d19/d1e/f5c [0,4194304] 0 2024-02-19T19:05:04.037 INFO:tasks.workunit.client.1.smithi132.stdout:1/814: symlink d1/d1e/d77/lf6 0 2024-02-19T19:05:04.038 INFO:tasks.workunit.client.1.smithi132.stdout:3/940: write d8/d13/d2b/d60/f108 [37366,43002] 0 2024-02-19T19:05:04.041 INFO:tasks.workunit.client.1.smithi132.stdout:3/941: dread - d8/d13/d2b/d69/d92/daf/ff2 zero size 2024-02-19T19:05:04.044 INFO:tasks.workunit.client.1.smithi132.stdout:3/942: chown d8/d13/d2b/d69/d92/daf/f10f 34182481 1 2024-02-19T19:05:04.046 INFO:tasks.workunit.client.1.smithi132.stdout:1/815: dwrite d1/f1d [0,4194304] 0 2024-02-19T19:05:04.050 INFO:tasks.workunit.client.1.smithi132.stdout:3/943: mknod d8/d13/d2b/d69/d92/d34/c12a 0 2024-02-19T19:05:04.051 INFO:tasks.workunit.client.1.smithi132.stdout:3/944: stat d8/d13/d2b/d60/db8/dfb/d112 0 2024-02-19T19:05:04.054 INFO:tasks.workunit.client.1.smithi132.stdout:2/934: dwrite d0/d19/d1e/f97 [0,4194304] 0 2024-02-19T19:05:04.056 INFO:tasks.workunit.client.1.smithi132.stdout:2/935: truncate d0/d24/dd0/ff6 1777312 0 2024-02-19T19:05:04.059 INFO:tasks.workunit.client.1.smithi132.stdout:3/945: getdents d8/d13/d2b/d69/d7e/dd6 0 2024-02-19T19:05:04.060 INFO:tasks.workunit.client.1.smithi132.stdout:3/946: readlink d8/d13/d2b/d69/d92/l61 0 2024-02-19T19:05:04.062 INFO:tasks.workunit.client.1.smithi132.stdout:3/947: write d8/d13/d2b/d69/d92/daf/ff2 [769549,25427] 0 2024-02-19T19:05:04.064 INFO:tasks.workunit.client.1.smithi132.stdout:3/948: fdatasync d8/d38/d44/d6e/dcc/f10d 0 2024-02-19T19:05:04.065 INFO:tasks.workunit.client.1.smithi132.stdout:3/949: dread - d8/d13/d2b/d69/d92/daf/ffc zero size 2024-02-19T19:05:04.065 INFO:tasks.workunit.client.1.smithi132.stdout:1/816: dread d1/d1e/d77/f5e [0,4194304] 0 2024-02-19T19:05:04.067 INFO:tasks.workunit.client.1.smithi132.stdout:3/950: creat d8/d13/d2b/d69/d92/d8d/f12b x:0 0 0 2024-02-19T19:05:04.068 INFO:tasks.workunit.client.1.smithi132.stdout:1/817: mknod d1/d1e/d77/d2c/d62/d88/cf7 0 2024-02-19T19:05:04.070 INFO:tasks.workunit.client.1.smithi132.stdout:3/951: unlink d8/d13/d2b/d69/da6/fce 0 2024-02-19T19:05:04.075 INFO:tasks.workunit.client.1.smithi132.stdout:3/952: link d8/d13/d2b/l91 d8/d13/d2b/d69/d7e/ddd/l12c 0 2024-02-19T19:05:04.080 INFO:tasks.workunit.client.1.smithi132.stdout:2/936: dwrite d0/d24/dd0/fd8 [0,4194304] 0 2024-02-19T19:05:04.083 INFO:tasks.workunit.client.1.smithi132.stdout:1/818: dwrite d1/d1e/d77/d52/d64/fb0 [0,4194304] 0 2024-02-19T19:05:04.089 INFO:tasks.workunit.client.1.smithi132.stdout:2/937: dwrite d0/d19/f82 [0,4194304] 0 2024-02-19T19:05:04.089 INFO:tasks.workunit.client.1.smithi132.stdout:1/819: rename d1/d1e/d77/d2c/d62/d6b/dba/df4 to d1/d1e/d77/d2c/d62/dd0/df8 0 2024-02-19T19:05:04.092 INFO:tasks.workunit.client.1.smithi132.stdout:1/820: creat d1/d1e/d77/d2c/d43/d81/da1/ff9 x:0 0 0 2024-02-19T19:05:04.094 INFO:tasks.workunit.client.1.smithi132.stdout:2/938: symlink d0/d24/d58/d53/l132 0 2024-02-19T19:05:04.098 INFO:tasks.workunit.client.1.smithi132.stdout:1/821: unlink d1/l4c 0 2024-02-19T19:05:04.100 INFO:tasks.workunit.client.1.smithi132.stdout:1/822: write d1/d1e/d77/d2c/f34 [1385811,118456] 0 2024-02-19T19:05:04.103 INFO:tasks.workunit.client.1.smithi132.stdout:2/939: link d0/d24/d58/d53/dab/d8a/da5/db1/cdf d0/d24/daf/d12d/c133 0 2024-02-19T19:05:04.104 INFO:tasks.workunit.client.1.smithi132.stdout:3/953: dread d8/d13/d2b/f2c [0,4194304] 0 2024-02-19T19:05:04.107 INFO:tasks.workunit.client.1.smithi132.stdout:2/940: dread d0/d4/fa9 [0,4194304] 0 2024-02-19T19:05:04.111 INFO:tasks.workunit.client.1.smithi132.stdout:1/823: dwrite d1/f48 [4194304,4194304] 0 2024-02-19T19:05:04.113 INFO:tasks.workunit.client.1.smithi132.stdout:2/941: truncate d0/f6 216693 0 2024-02-19T19:05:04.114 INFO:tasks.workunit.client.1.smithi132.stdout:3/954: dwrite d8/d13/d2b/d10c/f114 [0,4194304] 0 2024-02-19T19:05:04.115 INFO:tasks.workunit.client.1.smithi132.stdout:1/824: chown d1/d1e/d77/d4e/le7 241718 1 2024-02-19T19:05:04.120 INFO:tasks.workunit.client.1.smithi132.stdout:1/825: unlink d1/d3a/d50/l57 0 2024-02-19T19:05:04.122 INFO:tasks.workunit.client.1.smithi132.stdout:2/942: dread d0/d24/f117 [0,4194304] 0 2024-02-19T19:05:04.122 INFO:tasks.workunit.client.1.smithi132.stdout:1/826: truncate d1/d95/d97/ff1 280823 0 2024-02-19T19:05:04.125 INFO:tasks.workunit.client.1.smithi132.stdout:1/827: creat d1/d1e/dcd/ffa x:0 0 0 2024-02-19T19:05:04.126 INFO:tasks.workunit.client.1.smithi132.stdout:1/828: readlink d1/d1e/d77/d2c/d43/l92 0 2024-02-19T19:05:04.130 INFO:tasks.workunit.client.1.smithi132.stdout:3/955: write d8/d93/f7c [2742211,93460] 0 2024-02-19T19:05:04.141 INFO:tasks.workunit.client.1.smithi132.stdout:1/829: dwrite d1/d3a/d50/faf [0,4194304] 0 2024-02-19T19:05:04.141 INFO:tasks.workunit.client.1.smithi132.stdout:3/956: dwrite d8/d13/d2b/d60/f9b [0,4194304] 0 2024-02-19T19:05:04.158 INFO:tasks.workunit.client.1.smithi132.stdout:3/957: creat d8/d13/d2b/d60/f12d x:0 0 0 2024-02-19T19:05:04.164 INFO:tasks.workunit.client.1.smithi132.stdout:3/958: dwrite d8/d93/f6f [0,4194304] 0 2024-02-19T19:05:04.166 INFO:tasks.workunit.client.1.smithi132.stdout:3/959: mkdir d8/d13/d2f/d12e 0 2024-02-19T19:05:04.167 INFO:tasks.workunit.client.1.smithi132.stdout:1/830: write d1/d1e/d77/d2c/f59 [1202692,107878] 0 2024-02-19T19:05:04.168 INFO:tasks.workunit.client.1.smithi132.stdout:3/960: rmdir d8/d13/d2b/d60/db8 39 2024-02-19T19:05:04.171 INFO:tasks.workunit.client.1.smithi132.stdout:1/831: rename d1/f4a to d1/d1e/d77/d2c/d43/db8/ffb 0 2024-02-19T19:05:04.174 INFO:tasks.workunit.client.1.smithi132.stdout:1/832: mkdir d1/dfc 0 2024-02-19T19:05:04.175 INFO:tasks.workunit.client.1.smithi132.stdout:3/961: creat d8/d13/f12f x:0 0 0 2024-02-19T19:05:04.180 INFO:tasks.workunit.client.1.smithi132.stdout:1/833: rmdir d1/d1e/d77/d2c/d43/d81/da1/de2 0 2024-02-19T19:05:04.183 INFO:tasks.workunit.client.1.smithi132.stdout:1/834: dread d1/d61/fa5 [0,4194304] 0 2024-02-19T19:05:04.185 INFO:tasks.workunit.client.1.smithi132.stdout:1/835: mkdir d1/d1e/d6a/dfd 0 2024-02-19T19:05:04.190 INFO:tasks.workunit.client.1.smithi132.stdout:1/836: dread d1/fe9 [0,4194304] 0 2024-02-19T19:05:04.190 INFO:tasks.workunit.client.1.smithi132.stdout:3/962: dread d8/d13/d2b/d69/d92/d8d/fd8 [0,4194304] 0 2024-02-19T19:05:04.193 INFO:tasks.workunit.client.1.smithi132.stdout:3/963: symlink d8/d13/d2f/d12e/l130 0 2024-02-19T19:05:04.194 INFO:tasks.workunit.client.1.smithi132.stdout:3/964: rmdir d8/d13/d2b/d69 39 2024-02-19T19:05:04.197 INFO:tasks.workunit.client.1.smithi132.stdout:3/965: chown d8/d13/d2b/d69/d92/d34/d42/f54 432089795 1 2024-02-19T19:05:04.199 INFO:tasks.workunit.client.1.smithi132.stdout:3/966: write d8/d93/f3b [1748954,89339] 0 2024-02-19T19:05:04.203 INFO:tasks.workunit.client.1.smithi132.stdout:1/837: dwrite d1/d1e/f5c [0,4194304] 0 2024-02-19T19:05:04.203 INFO:tasks.workunit.client.1.smithi132.stdout:6/850: sync 2024-02-19T19:05:04.203 INFO:tasks.workunit.client.1.smithi132.stdout:8/985: sync 2024-02-19T19:05:04.207 INFO:tasks.workunit.client.1.smithi132.stdout:6/851: chown d0/db/d25/d3a/d5d/fce 13483849 1 2024-02-19T19:05:04.208 INFO:tasks.workunit.client.1.smithi132.stdout:3/967: mkdir d8/d13/d2b/d60/db8/d131 0 2024-02-19T19:05:04.208 INFO:tasks.workunit.client.1.smithi132.stdout:8/986: stat d4/dc/d1b/fd3 0 2024-02-19T19:05:04.208 INFO:tasks.workunit.client.1.smithi132.stdout:6/852: dread d0/dc/f94 [0,4194304] 0 2024-02-19T19:05:04.208 INFO:tasks.workunit.client.1.smithi132.stdout:6/853: readlink d0/db/l6b 0 2024-02-19T19:05:04.210 INFO:tasks.workunit.client.1.smithi132.stdout:1/838: rename d1/d1e/d77/d2c/d43/db8/ddf to d1/d1e/dfe 0 2024-02-19T19:05:04.213 INFO:tasks.workunit.client.1.smithi132.stdout:8/987: mknod d4/dc/d1b/d38/d126/d68/d79/c154 0 2024-02-19T19:05:04.214 INFO:tasks.workunit.client.1.smithi132.stdout:8/988: fsync d4/dc/d1b/d38/d126/d68/d7a/de8/f124 0 2024-02-19T19:05:04.214 INFO:tasks.workunit.client.1.smithi132.stdout:1/839: rename d1/f1d to d1/dfc/fff 0 2024-02-19T19:05:04.215 INFO:tasks.workunit.client.1.smithi132.stdout:6/854: mkdir d0/db/d25/d3a/d4b/d126/d127 0 2024-02-19T19:05:04.217 INFO:tasks.workunit.client.1.smithi132.stdout:6/855: dread d0/dc/f94 [0,4194304] 0 2024-02-19T19:05:04.218 INFO:tasks.workunit.client.1.smithi132.stdout:8/989: dread d4/dc/d1b/d20/f59 [0,4194304] 0 2024-02-19T19:05:04.219 INFO:tasks.workunit.client.1.smithi132.stdout:3/968: write d8/d13/d2b/d69/d92/d34/d42/f54 [1770774,69572] 0 2024-02-19T19:05:04.225 INFO:tasks.workunit.client.1.smithi132.stdout:1/840: truncate d1/f48 3715365 0 2024-02-19T19:05:04.226 INFO:tasks.workunit.client.1.smithi132.stdout:3/969: creat d8/d13/d2b/d60/db8/d122/f132 x:0 0 0 2024-02-19T19:05:04.227 INFO:tasks.workunit.client.1.smithi132.stdout:1/841: creat d1/d1e/d77/d2c/d89/f100 x:0 0 0 2024-02-19T19:05:04.230 INFO:tasks.workunit.client.1.smithi132.stdout:1/842: chown d1/d1e/d77/d2c/d62/d6b/la2 15948636 1 2024-02-19T19:05:04.233 INFO:tasks.workunit.client.1.smithi132.stdout:8/990: dwrite d4/dc/d1b/d38/d126/d128/f8c [4194304,4194304] 0 2024-02-19T19:05:04.234 INFO:tasks.workunit.client.1.smithi132.stdout:3/970: truncate d8/d13/d2b/d60/f106 811044 0 2024-02-19T19:05:04.236 INFO:tasks.workunit.client.1.smithi132.stdout:1/843: creat d1/d1e/d77/d2c/d43/d81/da1/f101 x:0 0 0 2024-02-19T19:05:04.237 INFO:tasks.workunit.client.1.smithi132.stdout:8/991: read - d4/dc/d1b/d38/d126/d128/fde zero size 2024-02-19T19:05:04.238 INFO:tasks.workunit.client.1.smithi132.stdout:6/856: dread d0/db/d25/d3a/d5d/d8f/d87/fda [0,4194304] 0 2024-02-19T19:05:04.239 INFO:tasks.workunit.client.1.smithi132.stdout:3/971: creat d8/d13/d2b/d69/d92/d8d/dd7/f133 x:0 0 0 2024-02-19T19:05:04.239 INFO:tasks.workunit.client.1.smithi132.stdout:1/844: chown d1/d61/cad 1348 1 2024-02-19T19:05:04.243 INFO:tasks.workunit.client.1.smithi132.stdout:8/992: rename d4/dc/d1b/d54/l7b to d4/d2d/dcc/l155 0 2024-02-19T19:05:04.244 INFO:tasks.workunit.client.1.smithi132.stdout:1/845: truncate d1/f18 87262 0 2024-02-19T19:05:04.245 INFO:tasks.workunit.client.1.smithi132.stdout:6/857: getdents d0/db/d25/d3a/d3d/d69/de2 0 2024-02-19T19:05:04.246 INFO:tasks.workunit.client.1.smithi132.stdout:8/993: mkdir d4/dc/d1b/d38/d69/d74/dce/d156 0 2024-02-19T19:05:04.247 INFO:tasks.workunit.client.1.smithi132.stdout:8/994: write d4/dc/d1b/d54/d81/f150 [282014,21067] 0 2024-02-19T19:05:04.248 INFO:tasks.workunit.client.1.smithi132.stdout:6/858: getdents d0/db/d25/d3a/d3d/d69/d107 0 2024-02-19T19:05:04.251 INFO:tasks.workunit.client.1.smithi132.stdout:6/859: creat d0/db/d25/d3a/d4b/d126/f128 x:0 0 0 2024-02-19T19:05:04.255 INFO:tasks.workunit.client.1.smithi132.stdout:8/995: dwrite d4/dc/d1b/f132 [0,4194304] 0 2024-02-19T19:05:04.261 INFO:tasks.workunit.client.1.smithi132.stdout:8/996: creat d4/dc/d1b/d38/d126/d68/f157 x:0 0 0 2024-02-19T19:05:04.263 INFO:tasks.workunit.client.1.smithi132.stdout:1/846: dwrite d1/d1e/d77/d2c/d43/f5d [4194304,4194304] 0 2024-02-19T19:05:04.266 INFO:tasks.workunit.client.1.smithi132.stdout:8/997: rename d4/d2d/fc0 to d4/dc/d1b/d38/f158 0 2024-02-19T19:05:04.268 INFO:tasks.workunit.client.1.smithi132.stdout:1/847: unlink d1/d1e/d77/d2c/f38 0 2024-02-19T19:05:04.269 INFO:tasks.workunit.client.1.smithi132.stdout:8/998: chown d4/dc/d1b/d38/d69/d74/fe1 431561 1 2024-02-19T19:05:04.269 INFO:tasks.workunit.client.1.smithi132.stdout:1/848: readlink d1/d1e/l23 0 2024-02-19T19:05:04.269 INFO:tasks.workunit.client.1.smithi132.stdout:1/849: fdatasync d1/d61/f7c 0 2024-02-19T19:05:04.270 INFO:tasks.workunit.client.1.smithi132.stdout:8/999: dread - d4/dc/d1b/d38/d126/d68/d79/db4/f149 zero size 2024-02-19T19:05:04.271 INFO:tasks.workunit.client.1.smithi132.stdout:1/850: fsync d1/dfc/fff 0 2024-02-19T19:05:04.273 INFO:tasks.workunit.client.1.smithi132.stdout:1/851: fdatasync d1/d1e/d77/d2c/d89/fd8 0 2024-02-19T19:05:04.275 INFO:tasks.workunit.client.1.smithi132.stdout:6/860: dread d0/dc/f66 [0,4194304] 0 2024-02-19T19:05:04.275 INFO:tasks.workunit.client.1.smithi132.stdout:1/852: mknod d1/d1e/d77/d2c/d62/d88/c102 0 2024-02-19T19:05:04.277 INFO:tasks.workunit.client.1.smithi132.stdout:1/853: mknod d1/d1e/dcd/c103 0 2024-02-19T19:05:04.277 INFO:tasks.workunit.client.1.smithi132.stdout:6/861: mknod d0/db/d25/d3a/d3d/dd9/c129 0 2024-02-19T19:05:04.278 INFO:tasks.workunit.client.1.smithi132.stdout:6/862: readlink d0/d8b/lee 0 2024-02-19T19:05:04.278 INFO:tasks.workunit.client.1.smithi132.stdout:1/854: mkdir d1/d1e/dfe/d104 0 2024-02-19T19:05:04.279 INFO:tasks.workunit.client.1.smithi132.stdout:1/855: write d1/d1e/d77/f41 [555356,116225] 0 2024-02-19T19:05:04.281 INFO:tasks.workunit.client.1.smithi132.stdout:6/863: mkdir d0/db/d25/d3a/d5d/d8f/d46/d5a/daf/d12a 0 2024-02-19T19:05:04.281 INFO:tasks.workunit.client.1.smithi132.stdout:1/856: dread - d1/d1e/d77/d4e/fb7 zero size 2024-02-19T19:05:04.282 INFO:tasks.workunit.client.1.smithi132.stdout:1/857: chown d1/d1e/d77 210 1 2024-02-19T19:05:04.283 INFO:tasks.workunit.client.1.smithi132.stdout:1/858: write d1/d1e/d77/d52/d64/d75/fd6 [1042751,112425] 0 2024-02-19T19:05:04.284 INFO:tasks.workunit.client.1.smithi132.stdout:6/864: rename d0/f41 to d0/db/d25/d3a/d4b/f12b 0 2024-02-19T19:05:04.285 INFO:tasks.workunit.client.1.smithi132.stdout:1/859: write d1/d1e/d77/f3c [3897140,13015] 0 2024-02-19T19:05:04.287 INFO:tasks.workunit.client.1.smithi132.stdout:6/865: mkdir d0/db/d25/d38/d12c 0 2024-02-19T19:05:04.287 INFO:tasks.workunit.client.1.smithi132.stdout:1/860: mkdir d1/d1e/d77/d4e/d99/d105 0 2024-02-19T19:05:04.288 INFO:tasks.workunit.client.1.smithi132.stdout:6/866: write d0/db/d25/d3a/d5d/d8f/d46/d5a/fe5 [578258,76963] 0 2024-02-19T19:05:04.289 INFO:tasks.workunit.client.1.smithi132.stdout:6/867: readlink d0/db/d25/d3a/d5d/d8f/l76 0 2024-02-19T19:05:04.290 INFO:tasks.workunit.client.1.smithi132.stdout:1/861: mkdir d1/d95/d97/de1/dea/d106 0 2024-02-19T19:05:04.291 INFO:tasks.workunit.client.1.smithi132.stdout:1/862: chown d1/d1e/d77/d2c/d43/lcb 0 1 2024-02-19T19:05:04.292 INFO:tasks.workunit.client.1.smithi132.stdout:1/863: write d1/d1e/d77/d2c/f34 [3377207,81486] 0 2024-02-19T19:05:04.293 INFO:tasks.workunit.client.1.smithi132.stdout:6/868: rename d0/dd1/d103/c110 to d0/db/d25/d3a/d3d/d69/de2/c12d 0 2024-02-19T19:05:04.293 INFO:tasks.workunit.client.1.smithi132.stdout:6/869: write d0/db/d25/d3a/df3/f121 [718669,76446] 0 2024-02-19T19:05:04.295 INFO:tasks.workunit.client.1.smithi132.stdout:1/864: rmdir d1/d1e/d6a 39 2024-02-19T19:05:04.297 INFO:tasks.workunit.client.1.smithi132.stdout:1/865: write d1/d3a/d50/fef [3032686,88447] 0 2024-02-19T19:05:04.298 INFO:tasks.workunit.client.1.smithi132.stdout:1/866: readlink d1/d1e/d77/d2c/d43/lcb 0 2024-02-19T19:05:04.301 INFO:tasks.workunit.client.1.smithi132.stdout:6/870: dwrite d0/db/d25/d3a/d5d/d8f/d46/fb9 [0,4194304] 0 2024-02-19T19:05:04.301 INFO:tasks.workunit.client.1.smithi132.stdout:1/867: write d1/d1e/d77/d2c/d43/db8/ffb [420565,20838] 0 2024-02-19T19:05:04.306 INFO:tasks.workunit.client.1.smithi132.stdout:6/871: getdents d0/db/d25/d3a/d4b/d7c/da2 0 2024-02-19T19:05:04.306 INFO:tasks.workunit.client.1.smithi132.stdout:1/868: getdents d1/d1e/d77/d2c 0 2024-02-19T19:05:04.308 INFO:tasks.workunit.client.1.smithi132.stdout:1/869: truncate d1/f8c 1009349 0 2024-02-19T19:05:04.313 INFO:tasks.workunit.client.1.smithi132.stdout:1/870: dwrite d1/d1e/d77/d2c/f34 [0,4194304] 0 2024-02-19T19:05:04.314 INFO:tasks.workunit.client.1.smithi132.stdout:6/872: write d0/db/d25/d3a/d3d/fb5 [789528,130884] 0 2024-02-19T19:05:04.321 INFO:tasks.workunit.client.1.smithi132.stdout:4/911: sync 2024-02-19T19:05:04.323 INFO:tasks.workunit.client.1.smithi132.stdout:4/912: fdatasync d4/d27/d31/d71/d14/de4/f115 0 2024-02-19T19:05:04.324 INFO:tasks.workunit.client.1.smithi132.stdout:1/871: dread d1/d1e/d77/f3f [0,4194304] 0 2024-02-19T19:05:04.325 INFO:tasks.workunit.client.1.smithi132.stdout:4/913: getdents d4/d19/daa/dc7/d10c 0 2024-02-19T19:05:04.326 INFO:tasks.workunit.client.1.smithi132.stdout:1/872: rmdir d1/d95/d97 39 2024-02-19T19:05:04.326 INFO:tasks.workunit.client.1.smithi132.stdout:4/914: fsync d4/d27/d31/d71/d14/de4/f115 0 2024-02-19T19:05:04.328 INFO:tasks.workunit.client.1.smithi132.stdout:1/873: creat d1/d1e/f107 x:0 0 0 2024-02-19T19:05:04.329 INFO:tasks.workunit.client.1.smithi132.stdout:4/915: creat d4/d27/d31/d32/f144 x:0 0 0 2024-02-19T19:05:04.336 INFO:tasks.workunit.client.1.smithi132.stdout:6/873: dread d0/db/d25/d3a/d3d/fa4 [0,4194304] 0 2024-02-19T19:05:04.364 INFO:tasks.workunit.client.1.smithi132.stdout:1/874: dread d1/d1e/d77/d2c/d62/fb4 [0,4194304] 0 2024-02-19T19:05:04.366 INFO:tasks.workunit.client.1.smithi132.stdout:1/875: truncate d1/d1e/d77/d2c/f34 5104124 0 2024-02-19T19:05:04.366 INFO:tasks.workunit.client.1.smithi132.stdout:6/874: rmdir d0 39 2024-02-19T19:05:04.367 INFO:tasks.workunit.client.1.smithi132.stdout:4/916: symlink d4/d27/d31/d32/d51/l145 0 2024-02-19T19:05:04.368 INFO:tasks.workunit.client.1.smithi132.stdout:1/876: truncate d1/d1e/d77/d4e/fb7 954454 0 2024-02-19T19:05:04.369 INFO:tasks.workunit.client.1.smithi132.stdout:4/917: mkdir d4/d27/d7f/d146 0 2024-02-19T19:05:04.369 INFO:tasks.workunit.client.1.smithi132.stdout:6/875: rmdir d0/db/d25/d3a/d5d/d8f/d87 39 2024-02-19T19:05:04.369 INFO:tasks.workunit.client.1.smithi132.stdout:1/877: write d1/d1e/d77/d2c/d89/f8d [1580037,82253] 0 2024-02-19T19:05:04.370 INFO:tasks.workunit.client.1.smithi132.stdout:7/968: sync 2024-02-19T19:05:04.370 INFO:tasks.workunit.client.1.smithi132.stdout:0/904: sync 2024-02-19T19:05:04.370 INFO:tasks.workunit.client.1.smithi132.stdout:2/943: sync 2024-02-19T19:05:04.372 INFO:tasks.workunit.client.1.smithi132.stdout:7/969: chown d1b/d46/d76/d94 30035 1 2024-02-19T19:05:04.372 INFO:tasks.workunit.client.1.smithi132.stdout:4/918: mkdir d4/d19/d21/dfa/d147 0 2024-02-19T19:05:04.373 INFO:tasks.workunit.client.1.smithi132.stdout:0/905: write d2/dc/df/d27/de8/d124/d43/d5f/dfe/f126 [776755,108151] 0 2024-02-19T19:05:04.376 INFO:tasks.workunit.client.1.smithi132.stdout:7/970: readlink d1b/d1e/d28/l4e 0 2024-02-19T19:05:04.377 INFO:tasks.workunit.client.1.smithi132.stdout:6/876: getdents d0/db/d25/d50/d86/de7/d11e 0 2024-02-19T19:05:04.378 INFO:tasks.workunit.client.1.smithi132.stdout:4/919: write d4/d27/d7f/f114 [1857762,119276] 0 2024-02-19T19:05:04.380 INFO:tasks.workunit.client.1.smithi132.stdout:0/906: dwrite d2/dc/df/d27/d3a/d52/fd0 [0,4194304] 0 2024-02-19T19:05:04.380 INFO:tasks.workunit.client.1.smithi132.stdout:7/971: mknod d1b/d46/d7f/c132 0 2024-02-19T19:05:04.384 INFO:tasks.workunit.client.1.smithi132.stdout:2/944: rename d0/d24/c54 to d0/d4/df/dc2/c134 0 2024-02-19T19:05:04.384 INFO:tasks.workunit.client.1.smithi132.stdout:1/878: getdents d1/d95/d97/de1/dea 0 2024-02-19T19:05:04.388 INFO:tasks.workunit.client.1.smithi132.stdout:1/879: dread d1/d1e/d77/d2c/f34 [0,4194304] 0 2024-02-19T19:05:04.388 INFO:tasks.workunit.client.1.smithi132.stdout:6/877: rename d0/fb7 to d0/db/d25/d3a/d5d/d8f/d22/f12e 0 2024-02-19T19:05:04.390 INFO:tasks.workunit.client.1.smithi132.stdout:4/920: creat d4/d27/d31/f148 x:0 0 0 2024-02-19T19:05:04.391 INFO:tasks.workunit.client.1.smithi132.stdout:7/972: creat d1b/d1e/d28/d11b/f133 x:0 0 0 2024-02-19T19:05:04.392 INFO:tasks.workunit.client.1.smithi132.stdout:6/878: dread d0/db/fa6 [0,4194304] 0 2024-02-19T19:05:04.394 INFO:tasks.workunit.client.1.smithi132.stdout:2/945: creat d0/d4/df/f135 x:0 0 0 2024-02-19T19:05:04.395 INFO:tasks.workunit.client.1.smithi132.stdout:4/921: mknod d4/d4d/c149 0 2024-02-19T19:05:04.396 INFO:tasks.workunit.client.1.smithi132.stdout:6/879: creat d0/db/d25/d38/f12f x:0 0 0 2024-02-19T19:05:04.397 INFO:tasks.workunit.client.1.smithi132.stdout:7/973: mkdir d1b/d1e/d41/da3/d134 0 2024-02-19T19:05:04.398 INFO:tasks.workunit.client.1.smithi132.stdout:4/922: creat d4/d27/d31/d71/d14/d24/f14a x:0 0 0 2024-02-19T19:05:04.401 INFO:tasks.workunit.client.1.smithi132.stdout:6/880: creat d0/db/d25/d3a/d5d/d8f/d87/f130 x:0 0 0 2024-02-19T19:05:04.402 INFO:tasks.workunit.client.1.smithi132.stdout:4/923: creat d4/d19/d21/d56/f14b x:0 0 0 2024-02-19T19:05:04.411 INFO:tasks.workunit.client.1.smithi132.stdout:6/881: dwrite d0/dc/f66 [8388608,4194304] 0 2024-02-19T19:05:04.412 INFO:tasks.workunit.client.1.smithi132.stdout:0/907: dread d2/f1a [0,4194304] 0 2024-02-19T19:05:04.414 INFO:tasks.workunit.client.1.smithi132.stdout:0/908: read d2/dc/df/d27/de8/d124/dda/d11d/f46 [2646706,41562] 0 2024-02-19T19:05:04.429 INFO:tasks.workunit.client.1.smithi132.stdout:7/974: dread d1b/d1e/d41/fe8 [0,4194304] 0 2024-02-19T19:05:04.435 INFO:tasks.workunit.client.1.smithi132.stdout:7/975: dwrite d1b/d1e/d28/f35 [0,4194304] 0 2024-02-19T19:05:04.435 INFO:tasks.workunit.client.1.smithi132.stdout:7/976: readlink d1b/d5a/l73 0 2024-02-19T19:05:04.441 INFO:tasks.workunit.client.1.smithi132.stdout:7/977: dwrite d1b/d46/f110 [0,4194304] 0 2024-02-19T19:05:04.444 INFO:tasks.workunit.client.1.smithi132.stdout:7/978: write d1b/d1e/f90 [877942,43049] 0 2024-02-19T19:05:04.445 INFO:tasks.workunit.client.1.smithi132.stdout:7/979: chown d1b/d1e/d41/fe8 336560084 1 2024-02-19T19:05:04.459 INFO:tasks.workunit.client.1.smithi132.stdout:6/882: symlink d0/db/d25/d3a/d3d/d69/de2/l131 0 2024-02-19T19:05:04.460 INFO:tasks.workunit.client.1.smithi132.stdout:6/883: chown d0/db/d25/d3a/d5d 947294958 1 2024-02-19T19:05:04.464 INFO:tasks.workunit.client.1.smithi132.stdout:4/924: mkdir d4/d19/daa/dc7/d10c/d14c 0 2024-02-19T19:05:04.465 INFO:tasks.workunit.client.1.smithi132.stdout:4/925: readlink d4/d12/d16/ddb/d131/l138 0 2024-02-19T19:05:04.466 INFO:tasks.workunit.client.1.smithi132.stdout:1/880: write d1/d3a/d50/fbd [694715,71331] 0 2024-02-19T19:05:04.473 INFO:tasks.workunit.client.1.smithi132.stdout:4/926: dwrite d4/d27/d31/d71/d14/de4/f126 [0,4194304] 0 2024-02-19T19:05:04.475 INFO:tasks.workunit.client.1.smithi132.stdout:4/927: chown d4/d27/d31/fbf 103800608 1 2024-02-19T19:05:04.476 INFO:tasks.workunit.client.1.smithi132.stdout:2/946: truncate d0/d24/f2d 2538599 0 2024-02-19T19:05:04.478 INFO:tasks.workunit.client.1.smithi132.stdout:2/947: truncate d0/d24/f116 240964 0 2024-02-19T19:05:04.478 INFO:tasks.workunit.client.1.smithi132.stdout:2/948: stat d0/d4/l7 0 2024-02-19T19:05:04.486 INFO:tasks.workunit.client.1.smithi132.stdout:2/949: dwrite d0/d24/d58/d53/dab/d8a/fc0 [0,4194304] 0 2024-02-19T19:05:04.490 INFO:tasks.workunit.client.1.smithi132.stdout:4/928: rmdir d4/d19/daa 39 2024-02-19T19:05:04.490 INFO:tasks.workunit.client.1.smithi132.stdout:4/929: read d4/fd6 [134693,71957] 0 2024-02-19T19:05:04.498 INFO:tasks.workunit.client.1.smithi132.stdout:7/980: getdents d1b/d1e/d2b/d8f 0 2024-02-19T19:05:04.502 INFO:tasks.workunit.client.1.smithi132.stdout:2/950: creat d0/d4/df/d40/f136 x:0 0 0 2024-02-19T19:05:04.503 INFO:tasks.workunit.client.1.smithi132.stdout:7/981: rmdir d1b/d1e/d28/dd2/d111 39 2024-02-19T19:05:04.505 INFO:tasks.workunit.client.1.smithi132.stdout:4/930: dwrite d4/d19/daa/f11a [0,4194304] 0 2024-02-19T19:05:04.513 INFO:tasks.workunit.client.1.smithi132.stdout:2/951: dwrite d0/d4/d11/d2e/d5a/fd4 [0,4194304] 0 2024-02-19T19:05:04.514 INFO:tasks.workunit.client.1.smithi132.stdout:2/952: chown d0/d4/df/d40/d68/d102/c106 15808634 1 2024-02-19T19:05:04.515 INFO:tasks.workunit.client.1.smithi132.stdout:2/953: chown d0/d24/daf 40609685 1 2024-02-19T19:05:04.519 INFO:tasks.workunit.client.1.smithi132.stdout:2/954: mknod d0/d4/c137 0 2024-02-19T19:05:04.520 INFO:tasks.workunit.client.1.smithi132.stdout:2/955: write d0/d4/f122 [484133,72226] 0 2024-02-19T19:05:04.523 INFO:tasks.workunit.client.1.smithi132.stdout:2/956: mkdir d0/d64/dc8/d138 0 2024-02-19T19:05:04.533 INFO:tasks.workunit.client.1.smithi132.stdout:3/972: sync 2024-02-19T19:05:04.535 INFO:tasks.workunit.client.1.smithi132.stdout:3/973: readlink d8/l18 0 2024-02-19T19:05:04.541 INFO:tasks.workunit.client.1.smithi132.stdout:3/974: dread d8/d38/ff5 [0,4194304] 0 2024-02-19T19:05:04.542 INFO:tasks.workunit.client.1.smithi132.stdout:3/975: chown d8/d38/d44/d6e/dcc 38973 1 2024-02-19T19:05:04.547 INFO:tasks.workunit.client.1.smithi132.stdout:3/976: dwrite d8/d13/d2b/d69/d92/d34/f59 [0,4194304] 0 2024-02-19T19:05:04.586 INFO:tasks.workunit.client.1.smithi132.stdout:0/909: sync 2024-02-19T19:05:04.661 INFO:tasks.workunit.client.1.smithi132.stdout:4/931: truncate d4/d27/d31/d71/d37/f116 907445 0 2024-02-19T19:05:04.665 INFO:tasks.workunit.client.1.smithi132.stdout:2/957: write d0/d24/d58/fbc [4510,107565] 0 2024-02-19T19:05:04.671 INFO:tasks.workunit.client.1.smithi132.stdout:2/958: dwrite d0/d4/d11/d2e/d5a/f11f [0,4194304] 0 2024-02-19T19:05:04.671 INFO:tasks.workunit.client.1.smithi132.stdout:6/884: rename d0/db/d25/d50 to d0/db/d25/d3a/d3d/d69/d132 0 2024-02-19T19:05:04.672 INFO:tasks.workunit.client.1.smithi132.stdout:0/910: write d2/dc/df/d27/de8/d124/dda/d11d/d34/d84/da7/fb3 [4768480,42088] 0 2024-02-19T19:05:04.673 INFO:tasks.workunit.client.1.smithi132.stdout:2/959: chown d0/d4/df/d40/d7b/f123 14374 1 2024-02-19T19:05:04.675 INFO:tasks.workunit.client.1.smithi132.stdout:7/982: rename d1b/d1e/d11e/d86/fb6 to d1b/d1e/d28/d5e/f135 0 2024-02-19T19:05:04.680 INFO:tasks.workunit.client.1.smithi132.stdout:0/911: mknod d2/dc/df/d27/c12b 0 2024-02-19T19:05:04.681 INFO:tasks.workunit.client.1.smithi132.stdout:0/912: fdatasync d2/d3e/ff4 0 2024-02-19T19:05:04.683 INFO:tasks.workunit.client.1.smithi132.stdout:6/885: dwrite d0/db/d25/d38/f68 [4194304,4194304] 0 2024-02-19T19:05:04.689 INFO:tasks.workunit.client.1.smithi132.stdout:0/913: fdatasync d2/f53 0 2024-02-19T19:05:04.691 INFO:tasks.workunit.client.1.smithi132.stdout:6/886: dwrite d0/db/d25/d3a/d5d/d8f/d22/f7e [0,4194304] 0 2024-02-19T19:05:04.695 INFO:tasks.workunit.client.1.smithi132.stdout:0/914: symlink d2/dc/df/d27/d3a/d52/da8/l12c 0 2024-02-19T19:05:04.698 INFO:tasks.workunit.client.1.smithi132.stdout:2/960: dread d0/d4/d11/f32 [0,4194304] 0 2024-02-19T19:05:04.703 INFO:tasks.workunit.client.1.smithi132.stdout:2/961: symlink d0/d24/daf/d12d/l139 0 2024-02-19T19:05:04.704 INFO:tasks.workunit.client.1.smithi132.stdout:6/887: getdents d0/db/d25/d38 0 2024-02-19T19:05:04.704 INFO:tasks.workunit.client.1.smithi132.stdout:2/962: chown d0/d64 9797 1 2024-02-19T19:05:04.706 INFO:tasks.workunit.client.1.smithi132.stdout:2/963: dread - d0/d24/d58/d53/dab/d8a/f109 zero size 2024-02-19T19:05:04.708 INFO:tasks.workunit.client.1.smithi132.stdout:7/983: dread d1b/d1e/d28/dd2/d111/f75 [0,4194304] 0 2024-02-19T19:05:04.718 INFO:tasks.workunit.client.1.smithi132.stdout:7/984: write d1b/d5a/f68 [253896,55738] 0 2024-02-19T19:05:04.721 INFO:tasks.workunit.client.1.smithi132.stdout:7/985: mknod d1b/d46/d76/d94/c136 0 2024-02-19T19:05:04.781 INFO:tasks.workunit.client.1.smithi132.stdout:7/986: write d1b/d1e/d28/dd2/d111/f75 [4436195,97809] 0 2024-02-19T19:05:04.808 INFO:tasks.workunit.client.1.smithi132.stdout:1/881: write d1/f18 [655742,51247] 0 2024-02-19T19:05:04.811 INFO:tasks.workunit.client.1.smithi132.stdout:3/977: rename d8/d13/d2b/d60/f106 to d8/dd1/f134 0 2024-02-19T19:05:04.812 INFO:tasks.workunit.client.1.smithi132.stdout:3/978: readlink d8/d13/l82 0 2024-02-19T19:05:04.812 INFO:tasks.workunit.client.1.smithi132.stdout:7/987: dread d1b/d1e/d28/f4c [0,4194304] 0 2024-02-19T19:05:04.814 INFO:tasks.workunit.client.1.smithi132.stdout:4/932: rename d4/d27/d31/fe3 to d4/d12/d9f/d13f/f14d 0 2024-02-19T19:05:04.815 INFO:tasks.workunit.client.1.smithi132.stdout:7/988: readlink d1b/d1e/d41/l71 0 2024-02-19T19:05:04.816 INFO:tasks.workunit.client.1.smithi132.stdout:2/964: rename d0/d24/dd0/fd8 to d0/d3c/d3f/f13a 0 2024-02-19T19:05:04.818 INFO:tasks.workunit.client.1.smithi132.stdout:1/882: dwrite d1/d1e/d77/d52/d64/fc9 [0,4194304] 0 2024-02-19T19:05:04.820 INFO:tasks.workunit.client.1.smithi132.stdout:1/883: stat d1/d1e/d77/d78 0 2024-02-19T19:05:04.821 INFO:tasks.workunit.client.1.smithi132.stdout:3/979: rename d8/fc0 to d8/d13/d2b/d60/db8/dfb/d112/f135 0 2024-02-19T19:05:04.824 INFO:tasks.workunit.client.1.smithi132.stdout:7/989: rename d1b/l108 to d1b/d1e/d2b/d8f/dae/l137 0 2024-02-19T19:05:04.826 INFO:tasks.workunit.client.1.smithi132.stdout:2/965: dwrite d0/d19/f2c [0,4194304] 0 2024-02-19T19:05:04.829 INFO:tasks.workunit.client.1.smithi132.stdout:1/884: write d1/d1e/d77/d4e/fb7 [1368113,14011] 0 2024-02-19T19:05:04.831 INFO:tasks.workunit.client.1.smithi132.stdout:2/966: rename d0/d3c/d3f/l5d to d0/d3c/d3f/l13b 0 2024-02-19T19:05:04.836 INFO:tasks.workunit.client.1.smithi132.stdout:2/967: creat d0/d4/df/d7a/f13c x:0 0 0 2024-02-19T19:05:04.838 INFO:tasks.workunit.client.1.smithi132.stdout:1/885: dwrite d1/d1e/d77/d2c/d62/d6b/d85/f94 [0,4194304] 0 2024-02-19T19:05:04.841 INFO:tasks.workunit.client.1.smithi132.stdout:1/886: mknod d1/d95/d97/de1/c108 0 2024-02-19T19:05:04.843 INFO:tasks.workunit.client.1.smithi132.stdout:3/980: rename d8/d38/d44/f11c to d8/dd1/dd2/f136 0 2024-02-19T19:05:04.849 INFO:tasks.workunit.client.1.smithi132.stdout:3/981: link d8/d38/ff5 d8/d13/d2b/d69/d7e/dd6/de7/f137 0 2024-02-19T19:05:04.851 INFO:tasks.workunit.client.1.smithi132.stdout:1/887: truncate d1/d1e/d77/d52/d64/d75/f83 1603191 0 2024-02-19T19:05:04.853 INFO:tasks.workunit.client.1.smithi132.stdout:1/888: stat d1/d1e/d77/f41 0 2024-02-19T19:05:04.856 INFO:tasks.workunit.client.1.smithi132.stdout:4/933: dwrite d4/d12/d48/f7c [0,4194304] 0 2024-02-19T19:05:04.856 INFO:tasks.workunit.client.1.smithi132.stdout:3/982: symlink d8/d13/d2b/d69/d92/d8d/dd7/da2/l138 0 2024-02-19T19:05:04.860 INFO:tasks.workunit.client.1.smithi132.stdout:2/968: dwrite d0/d4/f8e [0,4194304] 0 2024-02-19T19:05:04.861 INFO:tasks.workunit.client.1.smithi132.stdout:1/889: dread - d1/d1e/d77/d2c/d43/d81/da1/f101 zero size 2024-02-19T19:05:04.862 INFO:tasks.workunit.client.1.smithi132.stdout:7/990: dwrite d1b/d1e/d41/da3/dbd/d11c/d53/f65 [0,4194304] 0 2024-02-19T19:05:04.863 INFO:tasks.workunit.client.1.smithi132.stdout:4/934: truncate d4/d4d/d87/ff2 682103 0 2024-02-19T19:05:04.864 INFO:tasks.workunit.client.1.smithi132.stdout:3/983: chown d8/d13/d2b/d60/db8/dfb/d112/f135 2137 1 2024-02-19T19:05:04.866 INFO:tasks.workunit.client.1.smithi132.stdout:7/991: fsync d1b/d1e/d28/d5e/f135 0 2024-02-19T19:05:04.867 INFO:tasks.workunit.client.1.smithi132.stdout:4/935: mknod d4/d19/dd4/c14e 0 2024-02-19T19:05:04.867 INFO:tasks.workunit.client.1.smithi132.stdout:7/992: chown d1b/d5a/l73 208 1 2024-02-19T19:05:04.870 INFO:tasks.workunit.client.1.smithi132.stdout:2/969: mkdir d0/d13d 0 2024-02-19T19:05:04.875 INFO:tasks.workunit.client.1.smithi132.stdout:7/993: rename d1b/d1e/d41/da3/dbd/d11c/d53/f118 to d1b/d1e/d28/da9/dd3/f138 0 2024-02-19T19:05:04.875 INFO:tasks.workunit.client.1.smithi132.stdout:4/936: creat d4/d12/d48/f14f x:0 0 0 2024-02-19T19:05:04.876 INFO:tasks.workunit.client.1.smithi132.stdout:7/994: dread - d1b/d1e/d41/da3/dd5/f120 zero size 2024-02-19T19:05:04.878 INFO:tasks.workunit.client.1.smithi132.stdout:4/937: symlink d4/d12/d48/l150 0 2024-02-19T19:05:04.883 INFO:tasks.workunit.client.1.smithi132.stdout:4/938: rename d4/d12/c41 to d4/d27/d31/c151 0 2024-02-19T19:05:04.885 INFO:tasks.workunit.client.1.smithi132.stdout:1/890: creat d1/d1e/d77/d78/f109 x:0 0 0 2024-02-19T19:05:04.887 INFO:tasks.workunit.client.1.smithi132.stdout:4/939: rename d4/d12/d9f/lbb to d4/d12/d28/d12a/l152 0 2024-02-19T19:05:04.889 INFO:tasks.workunit.client.1.smithi132.stdout:4/940: chown d4/d19/daa/dc7/lfe 0 1 2024-02-19T19:05:04.889 INFO:tasks.workunit.client.1.smithi132.stdout:2/970: write d0/d4/df/dc2/f7f [745123,72946] 0 2024-02-19T19:05:04.890 INFO:tasks.workunit.client.1.smithi132.stdout:7/995: truncate d1b/d46/f110 3935324 0 2024-02-19T19:05:04.892 INFO:tasks.workunit.client.1.smithi132.stdout:7/996: unlink d1b/d1e/d28/faa 0 2024-02-19T19:05:04.894 INFO:tasks.workunit.client.1.smithi132.stdout:2/971: unlink d0/d24/d58/d53/dab/d84/dc9/ff0 0 2024-02-19T19:05:04.897 INFO:tasks.workunit.client.1.smithi132.stdout:3/984: dread d8/dd1/dd2/f136 [0,4194304] 0 2024-02-19T19:05:04.898 INFO:tasks.workunit.client.1.smithi132.stdout:3/985: fdatasync d8/d38/d44/d6e/dcc/f10d 0 2024-02-19T19:05:04.899 INFO:tasks.workunit.client.1.smithi132.stdout:7/997: getdents d1b/d1e/d28/d11b 0 2024-02-19T19:05:04.902 INFO:tasks.workunit.client.1.smithi132.stdout:3/986: creat d8/d13/d2b/d69/d7e/dd6/f139 x:0 0 0 2024-02-19T19:05:04.903 INFO:tasks.workunit.client.1.smithi132.stdout:1/891: mknod d1/d1e/c10a 0 2024-02-19T19:05:04.904 INFO:tasks.workunit.client.1.smithi132.stdout:7/998: symlink d1b/d1e/l139 0 2024-02-19T19:05:04.905 INFO:tasks.workunit.client.1.smithi132.stdout:4/941: truncate d4/d12/d48/fde 2943412 0 2024-02-19T19:05:04.908 INFO:tasks.workunit.client.1.smithi132.stdout:1/892: chown d1/d3a/d50/fef 945698597 1 2024-02-19T19:05:04.912 INFO:tasks.workunit.client.1.smithi132.stdout:4/942: write d4/f59 [631904,27597] 0 2024-02-19T19:05:04.916 INFO:tasks.workunit.client.1.smithi132.stdout:3/987: dwrite d8/d13/deb/ffa [0,4194304] 0 2024-02-19T19:05:04.916 INFO:tasks.workunit.client.1.smithi132.stdout:4/943: write d4/d19/d21/d56/f14b [757053,69052] 0 2024-02-19T19:05:04.918 INFO:tasks.workunit.client.1.smithi132.stdout:2/972: dread d0/d4/df/dc2/dad/fef [0,4194304] 0 2024-02-19T19:05:04.922 INFO:tasks.workunit.client.1.smithi132.stdout:6/888: sync 2024-02-19T19:05:04.922 INFO:tasks.workunit.client.1.smithi132.stdout:0/915: sync 2024-02-19T19:05:04.935 INFO:tasks.workunit.client.1.smithi132.stdout:1/893: creat d1/d1e/d77/d52/dca/f10b x:0 0 0 2024-02-19T19:05:04.938 INFO:tasks.workunit.client.1.smithi132.stdout:3/988: rename d8/d38/d44/cdf to d8/d13/d2b/d69/d92/d107/c13a 0 2024-02-19T19:05:04.938 INFO:tasks.workunit.client.1.smithi132.stdout:1/894: mkdir d1/d1e/dcd/d10c 0 2024-02-19T19:05:04.942 INFO:tasks.workunit.client.1.smithi132.stdout:0/916: dread d2/dc/df/d27/de8/d124/dda/d11d/d2f/d69/fb9 [0,4194304] 0 2024-02-19T19:05:04.963 INFO:tasks.workunit.client.1.smithi132.stdout:1/895: rename d1/d3a/l8f to d1/d1e/d77/d4e/l10d 0 2024-02-19T19:05:04.968 INFO:tasks.workunit.client.1.smithi132.stdout:1/896: truncate d1/d1e/f5a 2292962 0 2024-02-19T19:05:04.970 INFO:tasks.workunit.client.1.smithi132.stdout:1/897: mkdir d1/d1e/d77/d2c/d62/d10e 0 2024-02-19T19:05:04.971 INFO:tasks.workunit.client.1.smithi132.stdout:1/898: chown d1/d1e/d77/d2c/d62/d6b/dd1 32227807 1 2024-02-19T19:05:04.978 INFO:tasks.workunit.client.1.smithi132.stdout:1/899: link d1/d1e/dcd/c103 d1/d1e/d77/c10f 0 2024-02-19T19:05:04.981 INFO:tasks.workunit.client.1.smithi132.stdout:1/900: rmdir d1/d61/dc4 0 2024-02-19T19:05:04.982 INFO:tasks.workunit.client.1.smithi132.stdout:1/901: dread - d1/d1e/d77/d2c/d43/d81/da1/ff9 zero size 2024-02-19T19:05:04.984 INFO:tasks.workunit.client.1.smithi132.stdout:1/902: mkdir d1/d1e/d77/d2c/d62/dd0/d110 0 2024-02-19T19:05:04.984 INFO:tasks.workunit.client.1.smithi132.stdout:1/903: write d1/d3a/d50/f67 [1528166,67725] 0 2024-02-19T19:05:04.985 INFO:tasks.workunit.client.1.smithi132.stdout:1/904: read - d1/d1e/d77/d2c/d62/d6b/fe3 zero size 2024-02-19T19:05:04.986 INFO:tasks.workunit.client.1.smithi132.stdout:1/905: chown d1/d1e/d77/d4e/d99/lce 729 1 2024-02-19T19:05:05.029 INFO:tasks.workunit.client.1.smithi132.stdout:7/999: creat d1b/d5a/d63/d96/f13a x:0 0 0 2024-02-19T19:05:05.033 INFO:tasks.workunit.client.1.smithi132.stdout:2/973: link d0/d19/d6a/fe8 d0/d4/df/d7a/d111/f13e 0 2024-02-19T19:05:05.035 INFO:tasks.workunit.client.1.smithi132.stdout:4/944: creat d4/f153 x:0 0 0 2024-02-19T19:05:05.036 INFO:tasks.workunit.client.1.smithi132.stdout:2/974: write d0/d4/df/dc2/dad/fef [4035284,28079] 0 2024-02-19T19:05:05.038 INFO:tasks.workunit.client.1.smithi132.stdout:2/975: write d0/d24/f117 [4977627,73861] 0 2024-02-19T19:05:05.040 INFO:tasks.workunit.client.1.smithi132.stdout:1/906: mknod d1/d1e/d77/d52/d64/c111 0 2024-02-19T19:05:05.040 INFO:tasks.workunit.client.1.smithi132.stdout:2/976: dread - d0/fe9 zero size 2024-02-19T19:05:05.042 INFO:tasks.workunit.client.1.smithi132.stdout:4/945: creat d4/d27/d31/d71/d14/f154 x:0 0 0 2024-02-19T19:05:05.043 INFO:tasks.workunit.client.1.smithi132.stdout:4/946: fdatasync d4/d12/d48/d5f/f13e 0 2024-02-19T19:05:05.046 INFO:tasks.workunit.client.1.smithi132.stdout:1/907: dwrite d1/d61/f7c [0,4194304] 0 2024-02-19T19:05:05.047 INFO:tasks.workunit.client.1.smithi132.stdout:6/889: read - d0/db/d25/d3a/d5d/d8f/fe9 zero size 2024-02-19T19:05:05.048 INFO:tasks.workunit.client.1.smithi132.stdout:2/977: creat d0/d24/f13f x:0 0 0 2024-02-19T19:05:05.049 INFO:tasks.workunit.client.1.smithi132.stdout:6/890: fsync d0/db/dc9/f114 0 2024-02-19T19:05:05.052 INFO:tasks.workunit.client.1.smithi132.stdout:6/891: read - d0/dc/f90 zero size 2024-02-19T19:05:05.053 INFO:tasks.workunit.client.1.smithi132.stdout:4/947: dwrite d4/d27/d31/d71/d14/d24/f39 [0,4194304] 0 2024-02-19T19:05:05.053 INFO:tasks.workunit.client.1.smithi132.stdout:2/978: symlink d0/d4/d11/dee/l140 0 2024-02-19T19:05:05.053 INFO:tasks.workunit.client.1.smithi132.stdout:4/948: readlink d4/l22 0 2024-02-19T19:05:05.054 INFO:tasks.workunit.client.1.smithi132.stdout:6/892: read - d0/db/d25/d3a/d3d/d69/d89/fdb zero size 2024-02-19T19:05:05.054 INFO:tasks.workunit.client.1.smithi132.stdout:2/979: chown d0/d4/df/d40/d68 5293131 1 2024-02-19T19:05:05.055 INFO:tasks.workunit.client.1.smithi132.stdout:1/908: link d1/d3a/c58 d1/d95/d97/de1/dea/d106/c112 0 2024-02-19T19:05:05.059 INFO:tasks.workunit.client.1.smithi132.stdout:2/980: mknod d0/d24/dd0/d129/c141 0 2024-02-19T19:05:05.061 INFO:tasks.workunit.client.1.smithi132.stdout:6/893: symlink d0/db/d25/d3a/d5d/d8f/d87/l133 0 2024-02-19T19:05:05.062 INFO:tasks.workunit.client.1.smithi132.stdout:2/981: truncate d0/d24/fda 4723920 0 2024-02-19T19:05:05.064 INFO:tasks.workunit.client.1.smithi132.stdout:1/909: dwrite d1/fe9 [0,4194304] 0 2024-02-19T19:05:05.066 INFO:tasks.workunit.client.1.smithi132.stdout:1/910: readlink d1/d3a/l26 0 2024-02-19T19:05:05.068 INFO:tasks.workunit.client.1.smithi132.stdout:2/982: link d0/d24/d58/d53/dab/d8a/da5/db1/f10b d0/d64/dc8/d124/f142 0 2024-02-19T19:05:05.069 INFO:tasks.workunit.client.1.smithi132.stdout:1/911: creat d1/d95/d97/de1/f113 x:0 0 0 2024-02-19T19:05:05.079 INFO:tasks.workunit.client.1.smithi132.stdout:1/912: mkdir d1/d1e/d77/d2c/d43/d114 0 2024-02-19T19:05:05.081 INFO:tasks.workunit.client.1.smithi132.stdout:1/913: creat d1/d1e/d77/d4e/d99/f115 x:0 0 0 2024-02-19T19:05:05.082 INFO:tasks.workunit.client.1.smithi132.stdout:1/914: fdatasync d1/d1e/d77/d2c/ff2 0 2024-02-19T19:05:05.085 INFO:tasks.workunit.client.1.smithi132.stdout:2/983: write d0/d19/d6a/fe8 [1442085,76411] 0 2024-02-19T19:05:05.088 INFO:tasks.workunit.client.1.smithi132.stdout:2/984: mkdir d0/d4/df/d40/d68/d102/d143 0 2024-02-19T19:05:05.189 INFO:tasks.workunit.client.1.smithi132.stdout:4/949: dread d4/d27/d31/fbf [4194304,4194304] 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:4/950: truncate d4/d27/d7f/dc0/ff0 1396233 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:1/915: dwrite d1/d1e/d77/d52/d64/f91 [0,4194304] 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:1/916: read d1/f8c [974869,70630] 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:1/917: readlink d1/d1e/d77/d52/d64/l96 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:1/918: symlink d1/d1e/d77/d2c/d89/l116 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:1/919: rename d1/d1e/d77/d2c/d43/d81/da1/fbb to d1/d1e/d77/d2c/d62/d6b/d85/f117 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:2/985: link d0/d3c/df1/cf4 d0/d24/d58/d53/dab/d8a/da5/db1/c144 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:4/951: link d4/d12/da4/cf4 d4/d27/d31/d71/d14/d24/c155 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:1/920: fsync d1/f5 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:2/986: creat d0/d4/df/f145 x:0 0 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:1/921: getdents d1/dfc 0 2024-02-19T19:05:05.190 INFO:tasks.workunit.client.1.smithi132.stdout:2/987: dwrite d0/d4/f8e [0,4194304] 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:1/922: dwrite d1/d1e/d77/d78/f109 [0,4194304] 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:1/923: mkdir d1/d1e/d77/d2c/d43/d118 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:1/924: getdents d1/d1e/d77/d52 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:4/952: link d4/d12/d48/f14f d4/d12/d4c/d13d/f156 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:4/953: chown d4/d4d/d87/ff3 44070724 1 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:2/988: creat d0/d24/f146 x:0 0 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:1/925: unlink d1/d95/d97/de1/f113 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:2/989: truncate d0/d4/df/dc2/dad/f118 415594 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:2/990: mknod d0/d4/df/d40/d68/d102/d143/c147 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:2/991: getdents d0/d64/dc8 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:4/954: truncate d4/d19/daa/dc7/f70 659573 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:1/926: rename d1/d95 to d1/d1e/d77/d2c/d62/dd0/d110/d119 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:4/955: dwrite d4/d19/fd8 [0,4194304] 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:4/956: write d4/f2d [8733875,111090] 0 2024-02-19T19:05:05.191 INFO:tasks.workunit.client.1.smithi132.stdout:4/957: read - d4/d27/d31/d71/d14/d24/d49/f12b zero size 2024-02-19T19:05:05.192 INFO:tasks.workunit.client.1.smithi132.stdout:4/958: symlink d4/d12/d4c/l157 0 2024-02-19T19:05:05.192 INFO:tasks.workunit.client.1.smithi132.stdout:4/959: write d4/f7 [1381138,44821] 0 2024-02-19T19:05:05.192 INFO:tasks.workunit.client.1.smithi132.stdout:4/960: read d4/fad [1259170,114153] 0 2024-02-19T19:05:05.192 INFO:tasks.workunit.client.1.smithi132.stdout:4/961: rename d4/d4d/d87/fd5 to d4/d27/d7f/d146/f158 0 2024-02-19T19:05:05.192 INFO:tasks.workunit.client.1.smithi132.stdout:4/962: fsync d4/d27/d31/d32/fb6 0 2024-02-19T19:05:05.194 INFO:tasks.workunit.client.1.smithi132.stdout:1/927: dread d1/d1e/d77/d52/d64/fb0 [0,4194304] 0 2024-02-19T19:05:05.194 INFO:tasks.workunit.client.1.smithi132.stdout:4/963: rename d4/d12/d16/ddb/d131 to d4/d12/d9f/d159 0 2024-02-19T19:05:05.196 INFO:tasks.workunit.client.1.smithi132.stdout:4/964: creat d4/d27/d7f/d146/f15a x:0 0 0 2024-02-19T19:05:05.197 INFO:tasks.workunit.client.1.smithi132.stdout:4/965: rename d4/d12/d9f to d4/d12/d9f/d159/d15b 22 2024-02-19T19:05:05.198 INFO:tasks.workunit.client.1.smithi132.stdout:4/966: symlink d4/d12/d9f/l15c 0 2024-02-19T19:05:05.198 INFO:tasks.workunit.client.1.smithi132.stdout:1/928: link d1/d1e/d77/d2c/d62/dd0/d110/d119/d97/de1/c108 d1/d1e/d77/d2c/d62/dd0/d110/d119/d97/de1/dea/c11a 0 2024-02-19T19:05:05.200 INFO:tasks.workunit.client.1.smithi132.stdout:1/929: dread - d1/d1e/d77/d2c/d43/d81/da1/f101 zero size 2024-02-19T19:05:05.205 INFO:tasks.workunit.client.1.smithi132.stdout:4/967: dwrite d4/d27/d31/d71/d37/f142 [0,4194304] 0 2024-02-19T19:05:05.207 INFO:tasks.workunit.client.1.smithi132.stdout:1/930: dwrite d1/d1e/d77/d4e/d99/f115 [0,4194304] 0 2024-02-19T19:05:05.217 INFO:tasks.workunit.client.1.smithi132.stdout:1/931: dwrite d1/d1e/d77/d2c/f59 [0,4194304] 0 2024-02-19T19:05:05.217 INFO:tasks.workunit.client.1.smithi132.stdout:4/968: dwrite d4/d12/d4c/d13d/dbc/d109/f129 [0,4194304] 0 2024-02-19T19:05:05.223 INFO:tasks.workunit.client.1.smithi132.stdout:4/969: dread d4/d19/fd8 [0,4194304] 0 2024-02-19T19:05:05.224 INFO:tasks.workunit.client.1.smithi132.stdout:4/970: chown d4/d12/d9f/df5/d13c 2184312 1 2024-02-19T19:05:05.227 INFO:tasks.workunit.client.1.smithi132.stdout:4/971: readlink d4/d12/d4c/d13d/dbc/ldd 0 2024-02-19T19:05:05.230 INFO:tasks.workunit.client.1.smithi132.stdout:4/972: rename d4/d19/d21/dfa/d11f to d4/d12/d48/d15d 0 2024-02-19T19:05:05.232 INFO:tasks.workunit.client.1.smithi132.stdout:1/932: creat d1/d1e/d77/d2c/d43/d81/f11b x:0 0 0 2024-02-19T19:05:05.232 INFO:tasks.workunit.client.1.smithi132.stdout:4/973: chown d4/d12/d48/fde 503576512 1 2024-02-19T19:05:05.235 INFO:tasks.workunit.client.1.smithi132.stdout:1/933: unlink d1/d1e/d77/d52/d64/fc9 0 2024-02-19T19:05:05.237 INFO:tasks.workunit.client.1.smithi132.stdout:4/974: dread d4/d12/d4c/d13d/dbc/d109/f129 [0,4194304] 0 2024-02-19T19:05:05.237 INFO:tasks.workunit.client.1.smithi132.stdout:4/975: chown d4/d19/d21/lec 5 1 2024-02-19T19:05:05.238 INFO:tasks.workunit.client.1.smithi132.stdout:1/934: creat d1/d1e/d77/d2c/d62/dd0/df8/f11c x:0 0 0 2024-02-19T19:05:05.240 INFO:tasks.workunit.client.1.smithi132.stdout:4/976: mkdir d4/d19/d21/d15e 0 2024-02-19T19:05:05.241 INFO:tasks.workunit.client.1.smithi132.stdout:4/977: write d4/d12/d48/f7c [1695322,43628] 0 2024-02-19T19:05:05.242 INFO:tasks.workunit.client.1.smithi132.stdout:4/978: write d4/d12/da4/f100 [533147,107877] 0 2024-02-19T19:05:05.246 INFO:tasks.workunit.client.1.smithi132.stdout:4/979: mkdir d4/d27/d31/d71/d14/d24/d49/d10e/d11d/d15f 0 2024-02-19T19:05:05.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:05 smithi132 ceph-mon[37969]: pgmap v6: 65 pgs: 65 active+clean; 1.5 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail 2024-02-19T19:05:05.256 INFO:tasks.workunit.client.1.smithi132.stdout:4/980: dwrite d4/d19/daa/dc7/f70 [0,4194304] 0 2024-02-19T19:05:05.256 INFO:tasks.workunit.client.1.smithi132.stdout:1/935: dread d1/d1e/d77/d78/f9d [0,4194304] 0 2024-02-19T19:05:05.259 INFO:tasks.workunit.client.1.smithi132.stdout:4/981: chown d4/d19/f3f 129 1 2024-02-19T19:05:05.260 INFO:tasks.workunit.client.1.smithi132.stdout:1/936: creat d1/d1e/d77/d2c/d89/f11d x:0 0 0 2024-02-19T19:05:05.262 INFO:tasks.workunit.client.1.smithi132.stdout:1/937: mknod d1/d1e/d77/d78/dc0/c11e 0 2024-02-19T19:05:05.263 INFO:tasks.workunit.client.1.smithi132.stdout:4/982: mknod d4/d27/d31/d32/d51/c160 0 2024-02-19T19:05:05.263 INFO:tasks.workunit.client.1.smithi132.stdout:1/938: write d1/d1e/d77/d2c/d43/db8/ffb [830003,81760] 0 2024-02-19T19:05:05.266 INFO:tasks.workunit.client.1.smithi132.stdout:1/939: mkdir d1/d1e/d6a/dfd/d11f 0 2024-02-19T19:05:05.267 INFO:tasks.workunit.client.1.smithi132.stdout:4/983: rename d4/d27/d7f/fd9 to d4/d12/d48/d5f/f161 0 2024-02-19T19:05:05.273 INFO:tasks.workunit.client.1.smithi132.stdout:4/984: dwrite d4/d12/d4c/d13d/dbc/f121 [0,4194304] 0 2024-02-19T19:05:05.277 INFO:tasks.workunit.client.1.smithi132.stdout:1/940: fdatasync d1/f18 0 2024-02-19T19:05:05.278 INFO:tasks.workunit.client.1.smithi132.stdout:4/985: creat d4/d27/d31/d32/d51/d72/f162 x:0 0 0 2024-02-19T19:05:05.279 INFO:tasks.workunit.client.1.smithi132.stdout:1/941: chown d1/d1e/d77/d2c/f7d 1128 1 2024-02-19T19:05:05.282 INFO:tasks.workunit.client.1.smithi132.stdout:4/986: creat d4/d12/d9f/df5/d13c/f163 x:0 0 0 2024-02-19T19:05:05.283 INFO:tasks.workunit.client.1.smithi132.stdout:1/942: unlink d1/d3a/c76 0 2024-02-19T19:05:05.287 INFO:tasks.workunit.client.1.smithi132.stdout:1/943: rename d1/d1e/d77/d2c/c45 to d1/d1e/d77/d4e/c120 0 2024-02-19T19:05:05.289 INFO:tasks.workunit.client.1.smithi132.stdout:4/987: dwrite d4/d12/d48/d5f/f161 [0,4194304] 0 2024-02-19T19:05:05.289 INFO:tasks.workunit.client.1.smithi132.stdout:1/944: mkdir d1/d1e/d77/d52/d64/de6/d121 0 2024-02-19T19:05:05.291 INFO:tasks.workunit.client.1.smithi132.stdout:4/988: write d4/d27/d31/d71/d14/d24/d49/f12b [344867,97730] 0 2024-02-19T19:05:05.294 INFO:tasks.workunit.client.1.smithi132.stdout:4/989: creat d4/d12/da4/d134/f164 x:0 0 0 2024-02-19T19:05:05.297 INFO:tasks.workunit.client.1.smithi132.stdout:4/990: mkdir d4/d19/daa/dc7/d10c/d165 0 2024-02-19T19:05:05.299 INFO:tasks.workunit.client.1.smithi132.stdout:4/991: symlink d4/d19/d21/d56/l166 0 2024-02-19T19:05:05.299 INFO:tasks.workunit.client.1.smithi132.stdout:4/992: chown d4/d12/d9f 493379678 1 2024-02-19T19:05:05.300 INFO:tasks.workunit.client.1.smithi132.stdout:1/945: dwrite d1/d3a/fac [0,4194304] 0 2024-02-19T19:05:05.306 INFO:tasks.workunit.client.1.smithi132.stdout:1/946: mkdir d1/d1e/d77/d2c/d122 0 2024-02-19T19:05:05.308 INFO:tasks.workunit.client.1.smithi132.stdout:4/993: dwrite d4/d12/d4c/d13d/fa8 [0,4194304] 0 2024-02-19T19:05:05.311 INFO:tasks.workunit.client.1.smithi132.stdout:4/994: link d4/d12/d9f/f12c d4/d12/d9f/f167 0 2024-02-19T19:05:05.312 INFO:tasks.workunit.client.1.smithi132.stdout:4/995: readlink d4/d19/d21/lb7 0 2024-02-19T19:05:05.319 INFO:tasks.workunit.client.1.smithi132.stdout:4/996: truncate d4/d12/d4c/d13d/dbc/fbd 1797769 0 2024-02-19T19:05:05.321 INFO:tasks.workunit.client.1.smithi132.stdout:4/997: mknod d4/d12/d48/d5f/c168 0 2024-02-19T19:05:05.322 INFO:tasks.workunit.client.1.smithi132.stdout:4/998: write d4/d4d/d87/ff3 [3540903,105874] 0 2024-02-19T19:05:05.322 INFO:tasks.workunit.client.1.smithi132.stdout:1/947: write d1/d1e/d77/d2c/f7d [3013129,49441] 0 2024-02-19T19:05:05.323 INFO:tasks.workunit.client.1.smithi132.stdout:4/999: chown d4/d19/d21/dfa 60522 1 2024-02-19T19:05:05.324 INFO:tasks.workunit.client.1.smithi132.stdout:1/948: truncate d1/d1e/d77/d4e/f5f 9180346 0 2024-02-19T19:05:05.330 INFO:tasks.workunit.client.1.smithi132.stdout:1/949: dwrite d1/d1e/dcd/ffa [0,4194304] 0 2024-02-19T19:05:05.334 INFO:tasks.workunit.client.1.smithi132.stdout:1/950: mknod d1/d1e/d77/d2c/d62/d10e/c123 0 2024-02-19T19:05:05.338 INFO:tasks.workunit.client.1.smithi132.stdout:1/951: rename d1/d1e/d77/d4e/l10d to d1/d1e/d77/d78/dc0/l124 0 2024-02-19T19:05:05.340 INFO:tasks.workunit.client.1.smithi132.stdout:1/952: chown d1/d1e/d77/d2c/d89/f9c 698 1 2024-02-19T19:05:05.342 INFO:tasks.workunit.client.1.smithi132.stdout:1/953: mkdir d1/d1e/d77/d2c/d62/d6b/dd1/d125 0 2024-02-19T19:05:05.359 INFO:tasks.workunit.client.1.smithi132.stdout:1/954: dread d1/d1e/d77/f69 [0,4194304] 0 2024-02-19T19:05:05.360 INFO:tasks.workunit.client.1.smithi132.stdout:1/955: mkdir d1/d1e/d77/d52/dca/d126 0 2024-02-19T19:05:05.362 INFO:tasks.workunit.client.1.smithi132.stdout:1/956: mkdir d1/d1e/dcd/d10c/d127 0 2024-02-19T19:05:05.366 INFO:tasks.workunit.client.1.smithi132.stdout:1/957: dwrite d1/d3a/f40 [0,4194304] 0 2024-02-19T19:05:05.379 INFO:tasks.workunit.client.1.smithi132.stdout:1/958: dread d1/d61/fa5 [0,4194304] 0 2024-02-19T19:05:05.386 INFO:tasks.workunit.client.1.smithi132.stdout:1/959: dwrite d1/d3a/d50/fbd [0,4194304] 0 2024-02-19T19:05:05.389 INFO:tasks.workunit.client.1.smithi132.stdout:1/960: creat d1/d1e/d77/d52/dca/f128 x:0 0 0 2024-02-19T19:05:05.396 INFO:tasks.workunit.client.1.smithi132.stdout:1/961: dwrite d1/d1e/d77/d2c/d62/d6b/fe3 [0,4194304] 0 2024-02-19T19:05:05.466 INFO:tasks.workunit.client.1.smithi132.stdout:3/989: sync 2024-02-19T19:05:05.467 INFO:tasks.workunit.client.1.smithi132.stdout:3/990: fsync d8/d13/d2b/d69/d92/d8d/f12b 0 2024-02-19T19:05:05.468 INFO:tasks.workunit.client.1.smithi132.stdout:3/991: chown d8/d13/d2b/d60 44 1 2024-02-19T19:05:05.470 INFO:tasks.workunit.client.1.smithi132.stdout:3/992: mkdir d8/d13/d2b/d69/d92/d34/d42/d13b 0 2024-02-19T19:05:05.471 INFO:tasks.workunit.client.1.smithi132.stdout:3/993: mkdir d8/d13/d2b/d60/db8/dfb/d13c 0 2024-02-19T19:05:05.473 INFO:tasks.workunit.client.1.smithi132.stdout:3/994: creat d8/d13/d2f/f13d x:0 0 0 2024-02-19T19:05:05.474 INFO:tasks.workunit.client.1.smithi132.stdout:3/995: write d8/d13/d2b/d60/db8/d122/f132 [704384,38481] 0 2024-02-19T19:05:05.475 INFO:tasks.workunit.client.1.smithi132.stdout:3/996: write d8/d13/d2b/d69/da6/fc7 [1170031,98344] 0 2024-02-19T19:05:05.477 INFO:tasks.workunit.client.1.smithi132.stdout:3/997: write d8/d13/d2b/d60/db8/dfb/f105 [1328367,106236] 0 2024-02-19T19:05:05.478 INFO:tasks.workunit.client.1.smithi132.stdout:3/998: dread - d8/d13/d2b/d69/d92/d34/d42/fd9 zero size 2024-02-19T19:05:05.479 INFO:tasks.workunit.client.1.smithi132.stdout:3/999: dread - d8/d13/d2b/d69/d92/d107/fde zero size 2024-02-19T19:05:05.562 INFO:tasks.workunit.client.1.smithi132.stdout:2/992: sync 2024-02-19T19:05:05.562 INFO:tasks.workunit.client.1.smithi132.stdout:6/894: sync 2024-02-19T19:05:05.562 INFO:tasks.workunit.client.1.smithi132.stdout:0/917: sync 2024-02-19T19:05:05.565 INFO:tasks.workunit.client.1.smithi132.stdout:6/895: rename d0/db/d25/d3a/d5d/d8f/d87/fe0 to d0/d51/f134 0 2024-02-19T19:05:05.566 INFO:tasks.workunit.client.1.smithi132.stdout:6/896: write d0/db/d25/fcb [2098489,61338] 0 2024-02-19T19:05:05.566 INFO:tasks.workunit.client.1.smithi132.stdout:0/918: link d2/dc/df/d27/de8/d124/dda/d11d/fc3 d2/dc/df/f12d 0 2024-02-19T19:05:05.567 INFO:tasks.workunit.client.1.smithi132.stdout:2/993: getdents d0/d4/df/d40/d68/d102/d143 0 2024-02-19T19:05:05.570 INFO:tasks.workunit.client.1.smithi132.stdout:6/897: dread - d0/db/d25/d3a/d3d/d69/d132/d86/dc3/f105 zero size 2024-02-19T19:05:05.574 INFO:tasks.workunit.client.1.smithi132.stdout:0/919: truncate d2/ff2 734729 0 2024-02-19T19:05:05.576 INFO:tasks.workunit.client.1.smithi132.stdout:0/920: truncate d2/dc/d4a/d74/f85 4916217 0 2024-02-19T19:05:05.577 INFO:tasks.workunit.client.1.smithi132.stdout:2/994: dwrite d0/d4/df/f125 [0,4194304] 0 2024-02-19T19:05:05.582 INFO:tasks.workunit.client.1.smithi132.stdout:2/995: rename d0/d4/df/d40/d68/d102/f115 to d0/d4/d11/dee/f148 0 2024-02-19T19:05:05.583 INFO:tasks.workunit.client.1.smithi132.stdout:2/996: chown d0/d24/daf/d12d/l139 59080 1 2024-02-19T19:05:05.584 INFO:tasks.workunit.client.1.smithi132.stdout:2/997: stat d0/d4/d11/dee 0 2024-02-19T19:05:05.587 INFO:tasks.workunit.client.1.smithi132.stdout:0/921: creat d2/dc/df/f12e x:0 0 0 2024-02-19T19:05:05.590 INFO:tasks.workunit.client.1.smithi132.stdout:0/922: creat d2/d6e/dc8/dd7/f12f x:0 0 0 2024-02-19T19:05:05.591 INFO:tasks.workunit.client.1.smithi132.stdout:0/923: read - d2/dc/df/f113 zero size 2024-02-19T19:05:05.592 INFO:tasks.workunit.client.1.smithi132.stdout:0/924: readlink d2/dc/d98/lfa 0 2024-02-19T19:05:05.594 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:05 smithi088 ceph-mon[88643]: pgmap v6: 65 pgs: 65 active+clean; 1.5 GiB data, 4.5 GiB used, 532 GiB / 536 GiB avail 2024-02-19T19:05:05.595 INFO:tasks.workunit.client.1.smithi132.stdout:0/925: symlink d2/dc/df/d27/d3a/d52/da8/l130 0 2024-02-19T19:05:05.596 INFO:tasks.workunit.client.1.smithi132.stdout:0/926: mknod d2/dc/d4a/d74/c131 0 2024-02-19T19:05:05.597 INFO:tasks.workunit.client.1.smithi132.stdout:0/927: stat d2/dc/df/d27/de8/d124/dda/d11d/d34/c60 0 2024-02-19T19:05:05.599 INFO:tasks.workunit.client.1.smithi132.stdout:0/928: truncate d2/dc/d4a/d58/f95 5089067 0 2024-02-19T19:05:05.600 INFO:tasks.workunit.client.1.smithi132.stdout:0/929: readlink d2/dc/l10 0 2024-02-19T19:05:05.603 INFO:tasks.workunit.client.1.smithi132.stdout:0/930: dread d2/dc/f6d [4194304,4194304] 0 2024-02-19T19:05:05.607 INFO:tasks.workunit.client.1.smithi132.stdout:2/998: dread d0/d4/df/dc2/ff5 [0,4194304] 0 2024-02-19T19:05:05.608 INFO:tasks.workunit.client.1.smithi132.stdout:0/931: rename d2/dc/df/d27/de8/d124/d43/d5f/fa4 to d2/dc/df/d27/de8/d124/dda/d11d/d2f/f132 0 2024-02-19T19:05:05.609 INFO:tasks.workunit.client.1.smithi132.stdout:2/999: mkdir d0/d64/dc8/d138/d149 0 2024-02-19T19:05:05.610 INFO:tasks.workunit.client.1.smithi132.stdout:0/932: mknod d2/dc/df/d27/de8/d124/c133 0 2024-02-19T19:05:05.610 INFO:tasks.workunit.client.1.smithi132.stdout:0/933: stat d2/d3e/ff4 0 2024-02-19T19:05:05.611 INFO:tasks.workunit.client.1.smithi132.stdout:0/934: chown d2/dc/df/d27/de8/d124/dda/d11d/f4b 3 1 2024-02-19T19:05:05.612 INFO:tasks.workunit.client.1.smithi132.stdout:0/935: read - d2/dc/df/d27/de8/d124/dda/d11d/d34/f123 zero size 2024-02-19T19:05:05.614 INFO:tasks.workunit.client.1.smithi132.stdout:0/936: symlink d2/dc/df/d27/d3a/d52/l134 0 2024-02-19T19:05:05.615 INFO:tasks.workunit.client.1.smithi132.stdout:0/937: chown d2/d3e/db1/cde 33 1 2024-02-19T19:05:05.616 INFO:tasks.workunit.client.1.smithi132.stdout:0/938: stat d2/dc/d4a/d58/l7d 0 2024-02-19T19:05:05.616 INFO:tasks.workunit.client.1.smithi132.stdout:0/939: chown d2/dc/df/l4c 20697712 1 2024-02-19T19:05:05.618 INFO:tasks.workunit.client.1.smithi132.stdout:0/940: chown d2/dc/df/d27/de8/d124/dda/d11d/d34/d84/da7/cdf 29570 1 2024-02-19T19:05:05.620 INFO:tasks.workunit.client.1.smithi132.stdout:0/941: mknod d2/dc/d4a/c135 0 2024-02-19T19:05:05.621 INFO:tasks.workunit.client.1.smithi132.stdout:0/942: fsync d2/dc/df/f12d 0 2024-02-19T19:05:05.626 INFO:tasks.workunit.client.1.smithi132.stdout:0/943: getdents d2/dc/df/d27/de8/d124/dda/d11d/d2f/d3f/d119 0 2024-02-19T19:05:05.632 INFO:tasks.workunit.client.1.smithi132.stdout:1/962: sync 2024-02-19T19:05:05.635 INFO:tasks.workunit.client.1.smithi132.stdout:1/963: symlink d1/d1e/d77/d2c/d62/dd0/df8/l129 0 2024-02-19T19:05:05.635 INFO:tasks.workunit.client.1.smithi132.stdout:1/964: write d1/d1e/f74 [113808,122343] 0 2024-02-19T19:05:05.638 INFO:tasks.workunit.client.1.smithi132.stdout:1/965: mkdir d1/d1e/d77/d2c/d62/dd0/df8/d12a 0 2024-02-19T19:05:05.642 INFO:tasks.workunit.client.1.smithi132.stdout:0/944: dread d2/dc/df/d27/de8/d124/dda/d11d/d34/f3c [0,4194304] 0 2024-02-19T19:05:05.650 INFO:tasks.workunit.client.1.smithi132.stdout:0/945: dwrite d2/dc/df/fa9 [0,4194304] 0 2024-02-19T19:05:05.651 INFO:tasks.workunit.client.1.smithi132.stdout:1/966: dwrite d1/d1e/d77/d2c/d62/dd0/d110/d119/fc7 [0,4194304] 0 2024-02-19T19:05:05.652 INFO:tasks.workunit.client.1.smithi132.stdout:0/946: chown d2/dc/df/d27/de8/d124/dda/d11d/d34/c7b 2 1 2024-02-19T19:05:05.656 INFO:tasks.workunit.client.1.smithi132.stdout:1/967: symlink d1/d1e/d77/d2c/d62/dd0/d110/d119/d97/de1/dea/l12b 0 2024-02-19T19:05:05.658 INFO:tasks.workunit.client.1.smithi132.stdout:0/947: mknod d2/dc/df/d27/de8/d124/c136 0 2024-02-19T19:05:05.663 INFO:tasks.workunit.client.1.smithi132.stdout:0/948: truncate d2/dc/df/d27/de8/d124/dda/d11d/d2f/d3f/f8d 947097 0 2024-02-19T19:05:05.663 INFO:tasks.workunit.client.1.smithi132.stdout:1/968: link d1/d1e/d77/d2c/d43/f5d d1/d1e/d77/d4e/d99/d105/f12c 0 2024-02-19T19:05:05.664 INFO:tasks.workunit.client.1.smithi132.stdout:1/969: fdatasync d1/d1e/d77/f41 0 2024-02-19T19:05:05.664 INFO:tasks.workunit.client.1.smithi132.stdout:0/949: chown d2/dc/df/d27/de8/d124/dda/d11d/d2f/d3f/l8e 91 1 2024-02-19T19:05:05.665 INFO:tasks.workunit.client.1.smithi132.stdout:1/970: stat d1/d1e/d77/d2c/d62/d6b/dd1/d125 0 2024-02-19T19:05:05.666 INFO:tasks.workunit.client.1.smithi132.stdout:1/971: dread - d1/d1e/d77/d2c/d62/dd0/df8/f11c zero size 2024-02-19T19:05:05.666 INFO:tasks.workunit.client.1.smithi132.stdout:0/950: mkdir d2/d6e/df8/d137 0 2024-02-19T19:05:05.667 INFO:tasks.workunit.client.1.smithi132.stdout:1/972: chown d1/lb 1575 1 2024-02-19T19:05:05.668 INFO:tasks.workunit.client.1.smithi132.stdout:0/951: getdents d2/dc/df/d27/de8/d124/d7f/d118 0 2024-02-19T19:05:05.669 INFO:tasks.workunit.client.1.smithi132.stdout:1/973: creat d1/d1e/d77/d2c/d62/d6b/dd1/f12d x:0 0 0 2024-02-19T19:05:05.669 INFO:tasks.workunit.client.1.smithi132.stdout:0/952: rmdir d2/d6e/df8 39 2024-02-19T19:05:05.670 INFO:tasks.workunit.client.1.smithi132.stdout:0/953: write d2/f8c [341193,31114] 0 2024-02-19T19:05:05.673 INFO:tasks.workunit.client.1.smithi132.stdout:1/974: creat d1/dfc/f12e x:0 0 0 2024-02-19T19:05:05.674 INFO:tasks.workunit.client.1.smithi132.stdout:0/954: getdents d2/dc/df/d27/de8/d124/dda 0 2024-02-19T19:05:05.675 INFO:tasks.workunit.client.1.smithi132.stdout:1/975: truncate d1/d1e/d77/d2c/d43/d81/f11b 820693 0 2024-02-19T19:05:05.675 INFO:tasks.workunit.client.1.smithi132.stdout:0/955: dread - d2/dc/df/d27/de8/d124/dda/d11d/d34/f129 zero size 2024-02-19T19:05:05.676 INFO:tasks.workunit.client.1.smithi132.stdout:0/956: dread - d2/d6e/dc8/dd7/feb zero size 2024-02-19T19:05:05.676 INFO:tasks.workunit.client.1.smithi132.stdout:1/976: truncate d1/d1e/d77/d52/fc3 4777585 0 2024-02-19T19:05:05.678 INFO:tasks.workunit.client.1.smithi132.stdout:0/957: creat d2/dc/df/d27/d3a/d52/da8/f138 x:0 0 0 2024-02-19T19:05:05.680 INFO:tasks.workunit.client.1.smithi132.stdout:0/958: write d2/dc/df/f113 [140702,98824] 0 2024-02-19T19:05:05.683 INFO:tasks.workunit.client.1.smithi132.stdout:0/959: rmdir d2/dc/df/d27/de8/d124/d7f 39 2024-02-19T19:05:05.684 INFO:tasks.workunit.client.1.smithi132.stdout:1/977: dwrite d1/d1e/d77/d4e/d99/f115 [4194304,4194304] 0 2024-02-19T19:05:05.689 INFO:tasks.workunit.client.1.smithi132.stdout:0/960: mkdir d2/dc/df/d27/de8/d124/dda/d11d/dd9/d139 0 2024-02-19T19:05:05.692 INFO:tasks.workunit.client.1.smithi132.stdout:0/961: mknod d2/dc/df/d27/de8/d124/dda/d11d/c13a 0 2024-02-19T19:05:05.692 INFO:tasks.workunit.client.1.smithi132.stdout:0/962: chown d2/d6e/c80 11854 1 2024-02-19T19:05:05.696 INFO:tasks.workunit.client.1.smithi132.stdout:0/963: rename d2/dc/df/d27/de8/d124/lcc to d2/dc/df/d27/de8/d124/dda/d11d/d34/l13b 0 2024-02-19T19:05:05.697 INFO:tasks.workunit.client.1.smithi132.stdout:6/898: sync 2024-02-19T19:05:05.725 INFO:tasks.workunit.client.1.smithi132.stdout:6/899: fdatasync d0/db/d25/d3a/d5d/d8f/d87/fda 0 2024-02-19T19:05:05.728 INFO:tasks.workunit.client.1.smithi132.stdout:6/900: creat d0/db/d25/d3a/df3/f135 x:0 0 0 2024-02-19T19:05:05.730 INFO:tasks.workunit.client.1.smithi132.stdout:0/964: dwrite d2/dc/df/dbd/fed [0,4194304] 0 2024-02-19T19:05:05.742 INFO:tasks.workunit.client.1.smithi132.stdout:6/901: dwrite d0/db/d25/d3a/df3/f121 [0,4194304] 0 2024-02-19T19:05:05.749 INFO:tasks.workunit.client.1.smithi132.stdout:6/902: dwrite d0/db/d25/d3a/d3d/d69/d132/fdf [0,4194304] 0 2024-02-19T19:05:05.751 INFO:tasks.workunit.client.1.smithi132.stdout:6/903: dread d0/dc/f94 [0,4194304] 0 2024-02-19T19:05:05.791 INFO:tasks.workunit.client.1.smithi132.stdout:0/965: creat d2/d6e/dc8/dd7/f13c x:0 0 0 2024-02-19T19:05:05.791 INFO:tasks.workunit.client.1.smithi132.stdout:0/966: stat d2/f5c 0 2024-02-19T19:05:05.793 INFO:tasks.workunit.client.1.smithi132.stdout:6/904: link d0/db/d25/d3a/d4b/fc4 d0/f136 0 2024-02-19T19:05:05.793 INFO:tasks.workunit.client.1.smithi132.stdout:0/967: fdatasync d2/dc/df/d78/f9f 0 2024-02-19T19:05:05.795 INFO:tasks.workunit.client.1.smithi132.stdout:0/968: rmdir d2/dc/df/d27/de8/d124/dda/d11d/d34/d84 39 2024-02-19T19:05:05.797 INFO:tasks.workunit.client.1.smithi132.stdout:0/969: dread - d2/dc/df/d27/de8/d124/dda/d11d/d34/f123 zero size 2024-02-19T19:05:05.798 INFO:tasks.workunit.client.1.smithi132.stdout:0/970: chown d2/dc/d4a/d58/ca2 7097406 1 2024-02-19T19:05:05.799 INFO:tasks.workunit.client.1.smithi132.stdout:0/971: chown d2/d6e/c80 161376619 1 2024-02-19T19:05:05.800 INFO:tasks.workunit.client.1.smithi132.stdout:6/905: getdents d0/db/d25/d3a/d3d/d69 0 2024-02-19T19:05:05.807 INFO:tasks.workunit.client.1.smithi132.stdout:6/906: dwrite d0/db/fd [0,4194304] 0 2024-02-19T19:05:05.808 INFO:tasks.workunit.client.1.smithi132.stdout:0/972: write d2/f6 [525787,13882] 0 2024-02-19T19:05:05.812 INFO:tasks.workunit.client.1.smithi132.stdout:0/973: symlink d2/dc/df/d27/d3a/d52/da8/l13d 0 2024-02-19T19:05:05.813 INFO:tasks.workunit.client.1.smithi132.stdout:6/907: getdents d0/db/d25/d3a/d3d/d69/de2 0 2024-02-19T19:05:05.816 INFO:tasks.workunit.client.1.smithi132.stdout:0/974: truncate d2/dc/df/d27/de8/d124/dda/d11d/d2f/d3f/f79 985241 0 2024-02-19T19:05:05.817 INFO:tasks.workunit.client.1.smithi132.stdout:1/978: sync 2024-02-19T19:05:05.822 INFO:tasks.workunit.client.1.smithi132.stdout:0/975: dwrite d2/dc/df/d27/de8/d124/dc9/ff6 [0,4194304] 0 2024-02-19T19:05:05.827 INFO:tasks.workunit.client.1.smithi132.stdout:0/976: getdents d2/dc/d4a/d58 0 2024-02-19T19:05:05.829 INFO:tasks.workunit.client.1.smithi132.stdout:1/979: dwrite d1/d1e/d77/d78/f7f [0,4194304] 0 2024-02-19T19:05:05.831 INFO:tasks.workunit.client.1.smithi132.stdout:0/977: read d2/d3e/ff4 [2243017,107448] 0 2024-02-19T19:05:05.833 INFO:tasks.workunit.client.1.smithi132.stdout:0/978: mknod d2/dc/d4a/d74/c13e 0 2024-02-19T19:05:05.834 INFO:tasks.workunit.client.1.smithi132.stdout:1/980: mkdir d1/d1e/d77/d2c/d62/dd0/d110/d12f 0 2024-02-19T19:05:05.836 INFO:tasks.workunit.client.1.smithi132.stdout:6/908: dread d0/f73 [0,4194304] 0 2024-02-19T19:05:05.836 INFO:tasks.workunit.client.1.smithi132.stdout:1/981: write d1/d1e/d77/f3c [1224690,58614] 0 2024-02-19T19:05:05.841 INFO:tasks.workunit.client.1.smithi132.stdout:6/909: dread d0/db/f9d [0,4194304] 0 2024-02-19T19:05:05.842 INFO:tasks.workunit.client.1.smithi132.stdout:1/982: link d1/d1e/d77/f5e d1/d1e/dfe/d104/f130 0 2024-02-19T19:05:05.843 INFO:tasks.workunit.client.1.smithi132.stdout:6/910: chown d0/db/d25/d3a/d5d/d8f 26 1 2024-02-19T19:05:05.844 INFO:tasks.workunit.client.1.smithi132.stdout:6/911: write d0/db/d25/d38/f68 [5618970,67389] 0 2024-02-19T19:05:05.850 INFO:tasks.workunit.client.1.smithi132.stdout:1/983: dwrite d1/d1e/d77/d4e/d99/f115 [4194304,4194304] 0 2024-02-19T19:05:05.853 INFO:tasks.workunit.client.1.smithi132.stdout:1/984: chown d1/d1e/d77/d2c/d62/dd0/d110/d119/d97/de1/dea/d106 1 1 2024-02-19T19:05:05.855 INFO:tasks.workunit.client.1.smithi132.stdout:6/912: dwrite d0/f10 [4194304,4194304] 0 2024-02-19T19:05:05.856 INFO:tasks.workunit.client.1.smithi132.stdout:1/985: read d1/d1e/dcd/ffa [3020347,57215] 0 2024-02-19T19:05:05.862 INFO:tasks.workunit.client.1.smithi132.stdout:1/986: dread d1/d1e/d77/d78/f9d [0,4194304] 0 2024-02-19T19:05:05.864 INFO:tasks.workunit.client.1.smithi132.stdout:6/913: link d0/db/d25/c2c d0/db/d25/d3a/d3d/d69/d132/d86/c137 0 2024-02-19T19:05:05.866 INFO:tasks.workunit.client.1.smithi132.stdout:0/979: dread d2/dc/df/d27/de8/d124/dda/d11d/d2f/d47/f96 [4194304,4194304] 0 2024-02-19T19:05:05.868 INFO:tasks.workunit.client.1.smithi132.stdout:1/987: rename d1/d3a/l8b to d1/d1e/d77/d2c/d43/d114/l131 0 2024-02-19T19:05:05.869 INFO:tasks.workunit.client.1.smithi132.stdout:6/914: symlink d0/db/d25/d38/d12c/l138 0 2024-02-19T19:05:05.872 INFO:tasks.workunit.client.1.smithi132.stdout:1/988: rename d1/d3a/f40 to d1/d1e/dcd/f132 0 2024-02-19T19:05:05.875 INFO:tasks.workunit.client.1.smithi132.stdout:0/980: creat d2/dc/df/d27/de8/d124/dda/d11d/d34/f13f x:0 0 0 2024-02-19T19:05:05.877 INFO:tasks.workunit.client.1.smithi132.stdout:6/915: rmdir d0/db/d25/df9 0 2024-02-19T19:05:05.877 INFO:tasks.workunit.client.1.smithi132.stdout:0/981: write d2/dc/df/d27/de8/d124/dda/d11d/f24 [3289474,87077] 0 2024-02-19T19:05:05.879 INFO:tasks.workunit.client.1.smithi132.stdout:6/916: fsync d0/d8b/f6f 0 2024-02-19T19:05:05.880 INFO:tasks.workunit.client.1.smithi132.stdout:6/917: read - d0/db/d25/d3a/d3d/fa8 zero size 2024-02-19T19:05:05.886 INFO:tasks.workunit.client.1.smithi132.stdout:0/982: dwrite d2/dc/d4a/d58/fa6 [0,4194304] 0 2024-02-19T19:05:05.886 INFO:tasks.workunit.client.1.smithi132.stdout:6/918: rmdir d0/dc 39 2024-02-19T19:05:05.889 INFO:tasks.workunit.client.1.smithi132.stdout:6/919: write d0/db/fe6 [670089,45825] 0 2024-02-19T19:05:05.890 INFO:tasks.workunit.client.1.smithi132.stdout:0/983: fdatasync d2/dc/df/d27/d3a/d52/da8/f138 0 2024-02-19T19:05:05.891 INFO:tasks.workunit.client.1.smithi132.stdout:1/989: dwrite d1/d1e/d77/d2c/d43/f5d [0,4194304] 0 2024-02-19T19:05:05.894 INFO:tasks.workunit.client.1.smithi132.stdout:6/920: dread d0/db/d25/d3a/d5d/d8f/d46/d5a/f7d [0,4194304] 0 2024-02-19T19:05:05.898 INFO:tasks.workunit.client.1.smithi132.stdout:6/921: readlink d0/db/d25/d3a/d4b/lb4 0 2024-02-19T19:05:05.898 INFO:tasks.workunit.client.1.smithi132.stdout:1/990: truncate d1/d61/f7c 1673036 0 2024-02-19T19:05:05.900 INFO:tasks.workunit.client.1.smithi132.stdout:1/991: chown d1/d1e/d77/d2c/d62/d6b/la2 28 1 2024-02-19T19:05:05.901 INFO:tasks.workunit.client.1.smithi132.stdout:6/922: symlink d0/d51/da7/l139 0 2024-02-19T19:05:05.902 INFO:tasks.workunit.client.1.smithi132.stdout:1/992: fdatasync d1/d1e/d77/d2c/d89/f8d 0 2024-02-19T19:05:05.903 INFO:tasks.workunit.client.1.smithi132.stdout:1/993: write d1/d1e/d77/d52/dca/f10b [230698,78710] 0 2024-02-19T19:05:05.905 INFO:tasks.workunit.client.1.smithi132.stdout:1/994: creat d1/d61/dd4/f133 x:0 0 0 2024-02-19T19:05:05.911 INFO:tasks.workunit.client.1.smithi132.stdout:1/995: dwrite d1/d1e/dcd/f132 [0,4194304] 0 2024-02-19T19:05:05.911 INFO:tasks.workunit.client.1.smithi132.stdout:6/923: dread d0/db/d25/d3a/d4b/d7c/fbd [0,4194304] 0 2024-02-19T19:05:05.913 INFO:tasks.workunit.client.1.smithi132.stdout:6/924: write d0/db/d25/d3a/d5d/d8f/d22/f7e [995501,6327] 0 2024-02-19T19:05:05.915 INFO:tasks.workunit.client.1.smithi132.stdout:1/996: chown d1/d3a/c58 145246543 1 2024-02-19T19:05:05.917 INFO:tasks.workunit.client.1.smithi132.stdout:6/925: fdatasync d0/db/d25/d3a/d4b/fc4 0 2024-02-19T19:05:05.919 INFO:tasks.workunit.client.1.smithi132.stdout:1/997: rename d1/d1e/dfe/d104 to d1/d1e/d77/d2c/d62/d6b/dd1/d125/d134 0 2024-02-19T19:05:05.923 INFO:tasks.workunit.client.1.smithi132.stdout:1/998: truncate d1/d1e/d77/d2c/d62/d6b/dd1/d125/d134/f130 712140 0 2024-02-19T19:05:05.925 INFO:tasks.workunit.client.1.smithi132.stdout:0/984: dread d2/dc/df/d27/de8/d124/dda/d11d/d34/d84/da7/fb3 [0,4194304] 0 2024-02-19T19:05:05.925 INFO:tasks.workunit.client.1.smithi132.stdout:1/999: read d1/d1e/f84 [593474,24158] 0 2024-02-19T19:05:05.928 INFO:tasks.workunit.client.1.smithi132.stdout:0/985: dread - d2/dc/df/d27/de8/d124/dda/d11d/d34/d84/fd1 zero size 2024-02-19T19:05:05.928 INFO:tasks.workunit.client.1.smithi132.stdout:0/986: dread - d2/f114 zero size 2024-02-19T19:05:05.930 INFO:tasks.workunit.client.1.smithi132.stdout:0/987: write d2/dc/df/d27/de8/d124/d43/d5f/dfe/f126 [1519555,33293] 0 2024-02-19T19:05:05.930 INFO:tasks.workunit.client.1.smithi132.stdout:6/926: write d0/d51/f60 [2945340,24907] 0 2024-02-19T19:05:05.935 INFO:tasks.workunit.client.1.smithi132.stdout:0/988: link d2/dc/df/d27/de8/d124/dda/d11d/d2f/d3f/f8d d2/dc/df/d27/d127/f140 0 2024-02-19T19:05:05.942 INFO:tasks.workunit.client.1.smithi132.stdout:0/989: dwrite d2/dc/df/d27/de8/d124/dda/d11d/f46 [0,4194304] 0 2024-02-19T19:05:05.945 INFO:tasks.workunit.client.1.smithi132.stdout:0/990: mkdir d2/dc/df/d27/de8/d124/dda/d11d/d2f/d3f/dd3/d141 0 2024-02-19T19:05:05.998 INFO:tasks.workunit.client.1.smithi132.stdout:0/991: dread d2/d3e/fdc [0,4194304] 0 2024-02-19T19:05:05.999 INFO:tasks.workunit.client.1.smithi132.stdout:0/992: truncate d2/dc/df/d27/de8/d124/dda/d11d/d34/f122 1035536 0 2024-02-19T19:05:06.002 INFO:tasks.workunit.client.1.smithi132.stdout:0/993: unlink d2/dc/df/d27/de8/d124/dda/d11d/d34/f3c 0 2024-02-19T19:05:06.004 INFO:tasks.workunit.client.1.smithi132.stdout:0/994: write d2/dc/df/d27/de8/d124/dda/d11d/d2f/d3f/d63/fe1 [347231,51027] 0 2024-02-19T19:05:06.005 INFO:tasks.workunit.client.1.smithi132.stdout:0/995: symlink d2/dc/d98/l142 0 2024-02-19T19:05:06.007 INFO:tasks.workunit.client.1.smithi132.stdout:0/996: mknod d2/dc/df/d27/de8/d124/dda/d11d/d2f/d3f/c143 0 2024-02-19T19:05:06.008 INFO:tasks.workunit.client.1.smithi132.stdout:0/997: write d2/d3e/fdc [940644,89316] 0 2024-02-19T19:05:06.012 INFO:tasks.workunit.client.1.smithi132.stdout:0/998: rename d2/d3e/db1/cde to d2/dc/df/d78/c144 0 2024-02-19T19:05:06.017 INFO:tasks.workunit.client.1.smithi132.stdout:0/999: write d2/dc/f3b [2112697,34656] 0 2024-02-19T19:05:06.077 INFO:tasks.workunit.client.1.smithi132.stdout:6/927: sync 2024-02-19T19:05:06.108 INFO:tasks.workunit.client.1.smithi132.stdout:6/928: dread d0/db/d25/d3a/d5d/d8f/d22/f23 [4194304,4194304] 0 2024-02-19T19:05:06.112 INFO:tasks.workunit.client.1.smithi132.stdout:6/929: dread d0/db/d25/d3a/d3d/f74 [0,4194304] 0 2024-02-19T19:05:06.113 INFO:tasks.workunit.client.1.smithi132.stdout:6/930: mknod d0/db/d25/d3a/d5d/d8f/d87/c13a 0 2024-02-19T19:05:06.116 INFO:tasks.workunit.client.1.smithi132.stdout:6/931: creat d0/dd1/d103/f13b x:0 0 0 2024-02-19T19:05:06.116 INFO:tasks.workunit.client.1.smithi132.stdout:6/932: stat d0/dd1/d103/f13b 0 2024-02-19T19:05:06.117 INFO:tasks.workunit.client.1.smithi132.stdout:6/933: chown d0/db/d25/d3a/df3/df8 517 1 2024-02-19T19:05:06.123 INFO:tasks.workunit.client.1.smithi132.stdout:6/934: truncate d0/db/d25/d3a/d5d/d8f/d46/d5a/f26 3971733 0 2024-02-19T19:05:06.124 INFO:tasks.workunit.client.1.smithi132.stdout:6/935: write d0/db/d25/d3a/d4b/d126/f128 [53253,121366] 0 2024-02-19T19:05:06.134 INFO:tasks.workunit.client.1.smithi132.stdout:6/936: dwrite d0/ffe [0,4194304] 0 2024-02-19T19:05:06.138 INFO:tasks.workunit.client.1.smithi132.stdout:6/937: rename d0/db/d25/d3a/d3d/c53 to d0/db/d25/d3a/d5d/c13c 0 2024-02-19T19:05:06.138 INFO:tasks.workunit.client.1.smithi132.stdout:6/938: chown d0/db/d25/d3a/d5d/d8f/dbe/f124 3441652 1 2024-02-19T19:05:06.140 INFO:tasks.workunit.client.1.smithi132.stdout:6/939: dread d0/d51/fcc [0,4194304] 0 2024-02-19T19:05:06.142 INFO:tasks.workunit.client.1.smithi132.stdout:6/940: truncate d0/db/d25/d3a/df3/ffa 530674 0 2024-02-19T19:05:06.145 INFO:tasks.workunit.client.1.smithi132.stdout:6/941: rename d0/d51 to d0/db/d25/d3a/d3d/d69/d132/d86/de7/d13d 0 2024-02-19T19:05:06.148 INFO:tasks.workunit.client.1.smithi132.stdout:6/942: getdents d0/dd1 0 2024-02-19T19:05:06.153 INFO:tasks.workunit.client.1.smithi132.stdout:6/943: dread d0/db/d25/d3a/d5d/d8f/d46/d5a/fe5 [0,4194304] 0 2024-02-19T19:05:06.167 INFO:tasks.workunit.client.1.smithi132.stdout:6/944: dread d0/d8b/f84 [0,4194304] 0 2024-02-19T19:05:06.167 INFO:tasks.workunit.client.1.smithi132.stdout:6/945: chown d0/db/d25/d3a/d3d/d69/d132/d86/fcf 15843404 1 2024-02-19T19:05:06.173 INFO:tasks.workunit.client.1.smithi132.stdout:6/946: write d0/db/d25/d3a/d3d/fd3 [1991055,114264] 0 2024-02-19T19:05:06.189 INFO:tasks.workunit.client.1.smithi132.stdout:6/947: dread d0/db/d25/d3a/d3d/d69/d132/d86/de7/d13d/f92 [4194304,4194304] 0 2024-02-19T19:05:06.191 INFO:tasks.workunit.client.1.smithi132.stdout:6/948: symlink d0/db/dc9/d111/l13e 0 2024-02-19T19:05:06.192 INFO:tasks.workunit.client.1.smithi132.stdout:6/949: mknod d0/dd1/c13f 0 2024-02-19T19:05:06.193 INFO:tasks.workunit.client.1.smithi132.stdout:6/950: stat d0/db/d25/d3a/d4b/d7c/da2/ff2 0 2024-02-19T19:05:06.194 INFO:tasks.workunit.client.1.smithi132.stdout:6/951: fsync d0/db/d25/d3a/d4b/d7c/fed 0 2024-02-19T19:05:06.195 INFO:tasks.workunit.client.1.smithi132.stdout:6/952: write d0/db/d25/d3a/d3d/d69/faa [1452254,101467] 0 2024-02-19T19:05:06.197 INFO:tasks.workunit.client.1.smithi132.stdout:6/953: mknod d0/db/d25/d3a/d4b/c140 0 2024-02-19T19:05:06.203 INFO:tasks.workunit.client.1.smithi132.stdout:6/954: dread d0/db/d25/d3a/d5d/d8f/f48 [0,4194304] 0 2024-02-19T19:05:06.204 INFO:tasks.workunit.client.1.smithi132.stdout:6/955: write d0/db/f58 [5308814,30709] 0 2024-02-19T19:05:06.205 INFO:tasks.workunit.client.1.smithi132.stdout:6/956: chown d0/db/d25/d3a/d4b/lfd 991 1 2024-02-19T19:05:06.205 INFO:tasks.workunit.client.1.smithi132.stdout:6/957: stat d0/db/d25/d38/f118 0 2024-02-19T19:05:06.328 INFO:tasks.workunit.client.1.smithi132.stdout:6/958: sync 2024-02-19T19:05:06.333 INFO:tasks.workunit.client.1.smithi132.stdout:6/959: dwrite d0/d8b/f6f [0,4194304] 0 2024-02-19T19:05:06.336 INFO:tasks.workunit.client.1.smithi132.stdout:6/960: creat d0/db/d25/d3a/df3/f141 x:0 0 0 2024-02-19T19:05:06.338 INFO:tasks.workunit.client.1.smithi132.stdout:6/961: rename d0/dc/l5b to d0/db/d25/d3a/d4b/d126/d127/l142 0 2024-02-19T19:05:06.341 INFO:tasks.workunit.client.1.smithi132.stdout:6/962: truncate d0/db/d25/d3a/d3d/d69/f5f 8050114 0 2024-02-19T19:05:06.342 INFO:tasks.workunit.client.1.smithi132.stdout:6/963: mknod d0/db/d25/d38/c143 0 2024-02-19T19:05:06.343 INFO:tasks.workunit.client.1.smithi132.stdout:6/964: rmdir d0/db/dc9/df7 39 2024-02-19T19:05:06.344 INFO:tasks.workunit.client.1.smithi132.stdout:6/965: chown d0/db/d25/d3a/d5d/d8f/cbc 157 1 2024-02-19T19:05:06.345 INFO:tasks.workunit.client.1.smithi132.stdout:6/966: write d0/db/d25/d3a/d5d/d8f/d87/fda [1278096,67266] 0 2024-02-19T19:05:06.348 INFO:tasks.workunit.client.1.smithi132.stdout:6/967: unlink d0/db/d25/d3a/d3d/d69/de2/ff1 0 2024-02-19T19:05:06.349 INFO:tasks.workunit.client.1.smithi132.stdout:6/968: symlink d0/db/d25/d3a/d5d/l144 0 2024-02-19T19:05:06.351 INFO:tasks.workunit.client.1.smithi132.stdout:6/969: symlink d0/db/d25/d3a/d5d/d8f/l145 0 2024-02-19T19:05:06.371 INFO:tasks.workunit.client.1.smithi132.stdout:6/970: dread d0/db/d25/d3a/d3d/d69/f5f [0,4194304] 0 2024-02-19T19:05:06.372 INFO:tasks.workunit.client.1.smithi132.stdout:6/971: truncate d0/db/d25/d3a/df3/ffa 1574100 0 2024-02-19T19:05:06.375 INFO:tasks.workunit.client.1.smithi132.stdout:6/972: write d0/f136 [688249,48609] 0 2024-02-19T19:05:06.381 INFO:tasks.workunit.client.1.smithi132.stdout:6/973: dwrite d0/db/d25/d3a/d4b/f12b [0,4194304] 0 2024-02-19T19:05:06.384 INFO:tasks.workunit.client.1.smithi132.stdout:6/974: symlink d0/db/d25/d3a/l146 0 2024-02-19T19:05:06.385 INFO:tasks.workunit.client.1.smithi132.stdout:6/975: fdatasync d0/db/fa6 0 2024-02-19T19:05:06.387 INFO:tasks.workunit.client.1.smithi132.stdout:6/976: dread d0/db/d25/d3a/d3d/d69/d132/d86/de7/d13d/fcc [0,4194304] 0 2024-02-19T19:05:06.388 INFO:tasks.workunit.client.1.smithi132.stdout:6/977: chown d0/db/d25/d3a/d5d/d8f/d22/l72 17749877 1 2024-02-19T19:05:06.391 INFO:tasks.workunit.client.1.smithi132.stdout:6/978: creat d0/db/d25/d3a/d3d/d69/d132/d86/de7/f147 x:0 0 0 2024-02-19T19:05:06.395 INFO:tasks.workunit.client.1.smithi132.stdout:6/979: rename d0/db/d25/d3a/d5d/d8f/d46/d5a/l45 to d0/db/d25/d3a/d5d/d8f/d46/l148 0 2024-02-19T19:05:06.395 INFO:tasks.workunit.client.1.smithi132.stdout:6/980: rename d0/db to d0/db/d25/d3a/d4b/d7c/d149 22 2024-02-19T19:05:06.399 INFO:tasks.workunit.client.1.smithi132.stdout:6/981: dread d0/db/d25/d3a/d3d/f61 [0,4194304] 0 2024-02-19T19:05:06.399 INFO:tasks.workunit.client.1.smithi132.stdout:6/982: write d0/db/fe6 [4895459,106234] 0 2024-02-19T19:05:06.408 INFO:tasks.workunit.client.1.smithi132.stdout:6/983: dwrite d0/db/d25/d3a/d5d/d8f/d46/d5a/f37 [0,4194304] 0 2024-02-19T19:05:06.409 INFO:tasks.workunit.client.1.smithi132.stdout:6/984: dread - d0/db/d25/d3a/d4b/d7c/da2/ff2 zero size 2024-02-19T19:05:06.412 INFO:tasks.workunit.client.1.smithi132.stdout:6/985: dread d0/db/d25/d3a/d5d/d8f/d46/d5a/f7d [0,4194304] 0 2024-02-19T19:05:06.413 INFO:tasks.workunit.client.1.smithi132.stdout:6/986: chown d0/db/d25/d3a/d3d/f61 2382 1 2024-02-19T19:05:06.414 INFO:tasks.workunit.client.1.smithi132.stdout:6/987: write d0/db/d25/d3a/d3d/fd3 [2580871,68239] 0 2024-02-19T19:05:06.417 INFO:tasks.workunit.client.1.smithi132.stdout:6/988: chown d0/db/d25/d3a/d3d/d69/d89/cae 13909 1 2024-02-19T19:05:06.418 INFO:tasks.workunit.client.1.smithi132.stdout:6/989: write d0/db/d25/d3a/d4b/f12b [3692452,39478] 0 2024-02-19T19:05:06.419 INFO:tasks.workunit.client.1.smithi132.stdout:6/990: write d0/db/d25/d3a/d3d/d69/d132/d86/fcf [1088129,68003] 0 2024-02-19T19:05:06.422 INFO:tasks.workunit.client.1.smithi132.stdout:6/991: read d0/d8b/f4a [3547181,100403] 0 2024-02-19T19:05:06.423 INFO:tasks.workunit.client.1.smithi132.stdout:6/992: write d0/db/d25/d38/f78 [1574105,72056] 0 2024-02-19T19:05:06.425 INFO:tasks.workunit.client.1.smithi132.stdout:6/993: fsync d0/db/d25/d3a/d3d/d69/d132/d86/dc3/f105 0 2024-02-19T19:05:06.425 INFO:tasks.workunit.client.1.smithi132.stdout:6/994: dread - d0/db/dc9/f114 zero size 2024-02-19T19:05:06.427 INFO:tasks.workunit.client.1.smithi132.stdout:6/995: creat d0/db/d25/d38/d12c/f14a x:0 0 0 2024-02-19T19:05:06.429 INFO:tasks.workunit.client.1.smithi132.stdout:6/996: unlink d0/d8b/lc1 0 2024-02-19T19:05:06.430 INFO:tasks.workunit.client.1.smithi132.stdout:6/997: read - d0/db/d25/d3a/d5d/d8f/dbe/f100 zero size 2024-02-19T19:05:06.430 INFO:tasks.workunit.client.1.smithi132.stdout:6/998: chown d0/db/d25/d38/l3e 46386154 1 2024-02-19T19:05:06.431 INFO:tasks.workunit.client.1.smithi132.stdout:6/999: dread - d0/db/d25/d38/d12c/f14a zero size 2024-02-19T19:05:06.434 INFO:tasks.workunit.client.1.smithi132.stderr:+ rm -rf -- ./tmp.eGsJfOogpO 2024-02-19T19:05:07.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:07 smithi132 ceph-mon[37969]: pgmap v7: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 73 MiB/s rd, 141 MiB/s wr, 335 op/s 2024-02-19T19:05:07.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:07 smithi088 ceph-mon[88643]: pgmap v7: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 73 MiB/s rd, 141 MiB/s wr, 335 op/s 2024-02-19T19:05:08.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:08.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:08.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:08 smithi088 ceph-mon[88643]: pgmap v8: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 56 MiB/s rd, 110 MiB/s wr, 260 op/s 2024-02-19T19:05:09.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:08 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:09.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:08 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:09.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:08 smithi132 ceph-mon[37969]: pgmap v8: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 56 MiB/s rd, 110 MiB/s wr, 260 op/s 2024-02-19T19:05:10.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:10.096 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:09 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:10.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:09 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:10.268 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-19T19:05:10.414 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-19T19:05:10.463 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-19T19:05:10.465 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-19T19:05:10.465 INFO:tasks.workunit.client.0.smithi088.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-19T19:05:10.510 INFO:tasks.workunit.client.0.smithi088.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-19T19:05:10.513 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T19:05:10.513 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-19T19:05:10.530 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-02-19T19:05:10.531 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-02-19T19:05:10.531 DEBUG:teuthology.orchestra.run.smithi088:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1ce7ea8877c709fa862eac608550374bd96e166e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-02-19T19:05:10.593 INFO:tasks.workunit.client.0.smithi088.stderr:+ mkdir -p fsstress 2024-02-19T19:05:10.595 INFO:tasks.workunit.client.0.smithi088.stderr:+ pushd fsstress 2024-02-19T19:05:10.596 INFO:tasks.workunit.client.0.smithi088.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-19T19:05:10.597 INFO:tasks.workunit.client.0.smithi088.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-02-19T19:05:10.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.808+0000 7f90c0bf6700 1 -- 172.21.15.88:0/512077464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90bc0687a0 msgr2=0x7f90bc068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:10.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.808+0000 7f90c0bf6700 1 --2- 172.21.15.88:0/512077464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90bc0687a0 0x7f90bc068b70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009910 tx=0x7f90ac009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:10.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.808+0000 7f90c0bf6700 1 -- 172.21.15.88:0/512077464 shutdown_connections 2024-02-19T19:05:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.808+0000 7f90c0bf6700 1 --2- 172.21.15.88:0/512077464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90bc069140 0x7f90bc104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.808+0000 7f90c0bf6700 1 --2- 172.21.15.88:0/512077464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90bc0687a0 0x7f90bc068b70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.808+0000 7f90c0bf6700 1 -- 172.21.15.88:0/512077464 >> 172.21.15.88:0/512077464 conn(0x7f90bc075130 msgr2=0x7f90bc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.808+0000 7f90c0bf6700 1 -- 172.21.15.88:0/512077464 shutdown_connections 2024-02-19T19:05:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.808+0000 7f90c0bf6700 1 -- 172.21.15.88:0/512077464 wait complete. 2024-02-19T19:05:10.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.809+0000 7f90c0bf6700 1 Processor -- start 2024-02-19T19:05:10.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.809+0000 7f90c0bf6700 1 -- start start 2024-02-19T19:05:10.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.809+0000 7f90c0bf6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90bc0687a0 0x7f90bc0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:10.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.809+0000 7f90c0bf6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90bc069140 0x7f90bc1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:10.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.810+0000 7f90c0bf6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc100720 con 0x7f90bc069140 2024-02-19T19:05:10.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.810+0000 7f90c0bf6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90bc100860 con 0x7f90bc0687a0 2024-02-19T19:05:10.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.810+0000 7f90ba59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90bc0687a0 0x7f90bc0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.810+0000 7f90ba59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90bc0687a0 0x7f90bc0ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:41070/0 (socket says 172.21.15.88:41070) 2024-02-19T19:05:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.810+0000 7f90ba59c700 1 -- 172.21.15.88:0/4048647214 learned_addr learned my addr 172.21.15.88:0/4048647214 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.810+0000 7f90b9d9b700 1 --2- 172.21.15.88:0/4048647214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90bc069140 0x7f90bc1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.810+0000 7f90ba59c700 1 -- 172.21.15.88:0/4048647214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90bc069140 msgr2=0x7f90bc1001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.810+0000 7f90ba59c700 1 --2- 172.21.15.88:0/4048647214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90bc069140 0x7f90bc1001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.810+0000 7f90ba59c700 1 -- 172.21.15.88:0/4048647214 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ac0095f0 con 0x7f90bc0687a0 2024-02-19T19:05:10.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.811+0000 7f90ba59c700 1 --2- 172.21.15.88:0/4048647214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90bc0687a0 0x7f90bc0ffca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0059d0 tx=0x7f90ac00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:10.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.811+0000 7f90ab7fe700 1 -- 172.21.15.88:0/4048647214 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ac01d070 con 0x7f90bc0687a0 2024-02-19T19:05:10.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.811+0000 7f90ab7fe700 1 -- 172.21.15.88:0/4048647214 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90ac00b610 con 0x7f90bc0687a0 2024-02-19T19:05:10.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.811+0000 7f90c0bf6700 1 -- 172.21.15.88:0/4048647214 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90bc100ae0 con 0x7f90bc0687a0 2024-02-19T19:05:10.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.811+0000 7f90ab7fe700 1 -- 172.21.15.88:0/4048647214 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ac022470 con 0x7f90bc0687a0 2024-02-19T19:05:10.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.811+0000 7f90c0bf6700 1 -- 172.21.15.88:0/4048647214 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90bc1a23b0 con 0x7f90bc0687a0 2024-02-19T19:05:10.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.812+0000 7f90c0bf6700 1 -- 172.21.15.88:0/4048647214 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90bc108820 con 0x7f90bc0687a0 2024-02-19T19:05:10.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.813+0000 7f90ab7fe700 1 -- 172.21.15.88:0/4048647214 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 38) v1 ==== 91587+0+0 (secure 0 0 0) 0x7f90ac0226e0 con 0x7f90bc0687a0 2024-02-19T19:05:10.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.813+0000 7f90ab7fe700 1 --2- 172.21.15.88:0/4048647214 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90a406d9e0 0x7f90a406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:10.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.813+0000 7f90b9d9b700 1 --2- 172.21.15.88:0/4048647214 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90a406d9e0 0x7f90a406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:10.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.813+0000 7f90ab7fe700 1 -- 172.21.15.88:0/4048647214 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f90ac08ebb0 con 0x7f90bc0687a0 2024-02-19T19:05:10.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.813+0000 7f90b9d9b700 1 --2- 172.21.15.88:0/4048647214 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90a406d9e0 0x7f90a406fe90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90bc101490 tx=0x7f90b0009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:10.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:10.820+0000 7f90ab7fe700 1 -- 172.21.15.88:0/4048647214 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f90ac058190 con 0x7f90bc0687a0 2024-02-19T19:05:11.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:10 smithi088 ceph-mon[88643]: pgmap v9: 65 pgs: 65 active+clean; 1.8 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 58 MiB/s rd, 116 MiB/s wr, 304 op/s 2024-02-19T19:05:11.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.129+0000 7f90c0bf6700 1 -- 172.21.15.88:0/4048647214 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f90bc061190 con 0x7f90a406d9e0 2024-02-19T19:05:11.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.131+0000 7f90ab7fe700 1 -- 172.21.15.88:0/4048647214 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f90bc061190 con 0x7f90a406d9e0 2024-02-19T19:05:11.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.133+0000 7f90c0bf6700 1 -- 172.21.15.88:0/4048647214 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90a406d9e0 msgr2=0x7f90a406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:11.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.133+0000 7f90c0bf6700 1 --2- 172.21.15.88:0/4048647214 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90a406d9e0 0x7f90a406fe90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f90bc101490 tx=0x7f90b0009260 comp rx=0 tx=0).stop 2024-02-19T19:05:11.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.133+0000 7f90c0bf6700 1 -- 172.21.15.88:0/4048647214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90bc0687a0 msgr2=0x7f90bc0ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:11.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.133+0000 7f90c0bf6700 1 --2- 172.21.15.88:0/4048647214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90bc0687a0 0x7f90bc0ffca0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0059d0 tx=0x7f90ac00f690 comp rx=0 tx=0).stop 2024-02-19T19:05:11.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.133+0000 7f90c0bf6700 1 -- 172.21.15.88:0/4048647214 shutdown_connections 2024-02-19T19:05:11.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.133+0000 7f90c0bf6700 1 --2- 172.21.15.88:0/4048647214 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90a406d9e0 0x7f90a406fe90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.133+0000 7f90c0bf6700 1 --2- 172.21.15.88:0/4048647214 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90bc0687a0 0x7f90bc0ffca0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.133+0000 7f90c0bf6700 1 --2- 172.21.15.88:0/4048647214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90bc069140 0x7f90bc1001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.133+0000 7f90c0bf6700 1 -- 172.21.15.88:0/4048647214 >> 172.21.15.88:0/4048647214 conn(0x7f90bc075130 msgr2=0x7f90bc0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:11.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.133+0000 7f90c0bf6700 1 -- 172.21.15.88:0/4048647214 shutdown_connections 2024-02-19T19:05:11.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.133+0000 7f90c0bf6700 1 -- 172.21.15.88:0/4048647214 wait complete. 2024-02-19T19:05:11.164 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:05:11.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:10 smithi132 ceph-mon[37969]: pgmap v9: 65 pgs: 65 active+clean; 1.8 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 58 MiB/s rd, 116 MiB/s wr, 304 op/s 2024-02-19T19:05:11.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.258+0000 7f67e3e6a700 1 -- 172.21.15.88:0/1964959352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67dc100460 msgr2=0x7f67dc100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:11.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.258+0000 7f67e3e6a700 1 --2- 172.21.15.88:0/1964959352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67dc100460 0x7f67dc100830 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009910 tx=0x7f67cc009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:11.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.258+0000 7f67e3e6a700 1 -- 172.21.15.88:0/1964959352 shutdown_connections 2024-02-19T19:05:11.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.258+0000 7f67e3e6a700 1 --2- 172.21.15.88:0/1964959352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67dc100d70 0x7f67dc1094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.258+0000 7f67e3e6a700 1 --2- 172.21.15.88:0/1964959352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67dc100460 0x7f67dc100830 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.258+0000 7f67e3e6a700 1 -- 172.21.15.88:0/1964959352 >> 172.21.15.88:0/1964959352 conn(0x7f67dc0748b0 msgr2=0x7f67dc074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:11.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.258+0000 7f67e3e6a700 1 -- 172.21.15.88:0/1964959352 shutdown_connections 2024-02-19T19:05:11.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.259+0000 7f67e3e6a700 1 -- 172.21.15.88:0/1964959352 wait complete. 2024-02-19T19:05:11.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.259+0000 7f67e3e6a700 1 Processor -- start 2024-02-19T19:05:11.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.259+0000 7f67e3e6a700 1 -- start start 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.260+0000 7f67e3e6a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67dc100460 0x7f67dc196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.260+0000 7f67e3e6a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67dc100d70 0x7f67dc1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.260+0000 7f67e3e6a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc1977f0 con 0x7f67dc100d70 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.260+0000 7f67e3e6a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc197930 con 0x7f67dc100460 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.260+0000 7f67e1405700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67dc100d70 0x7f67dc1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.260+0000 7f67e1405700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67dc100d70 0x7f67dc1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48954/0 (socket says 172.21.15.88:48954) 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.260+0000 7f67e1405700 1 -- 172.21.15.88:0/3724749780 learned_addr learned my addr 172.21.15.88:0/3724749780 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.260+0000 7f67e1c06700 1 --2- 172.21.15.88:0/3724749780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67dc100460 0x7f67dc196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.260+0000 7f67e1405700 1 -- 172.21.15.88:0/3724749780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67dc100460 msgr2=0x7f67dc196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.260+0000 7f67e1405700 1 --2- 172.21.15.88:0/3724749780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67dc100460 0x7f67dc196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.260+0000 7f67e1405700 1 -- 172.21.15.88:0/3724749780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc0095f0 con 0x7f67dc100d70 2024-02-19T19:05:11.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.261+0000 7f67e1405700 1 --2- 172.21.15.88:0/3724749780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67dc100d70 0x7f67dc1971d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67d800e940 tx=0x7f67d800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:11.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.261+0000 7f67d2ffd700 1 -- 172.21.15.88:0/3724749780 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67d800cba0 con 0x7f67dc100d70 2024-02-19T19:05:11.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.261+0000 7f67d2ffd700 1 -- 172.21.15.88:0/3724749780 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67d8004cf0 con 0x7f67dc100d70 2024-02-19T19:05:11.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.261+0000 7f67e3e6a700 1 -- 172.21.15.88:0/3724749780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc19fd00 con 0x7f67dc100d70 2024-02-19T19:05:11.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.261+0000 7f67d2ffd700 1 -- 172.21.15.88:0/3724749780 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67d8010430 con 0x7f67dc100d70 2024-02-19T19:05:11.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.261+0000 7f67e3e6a700 1 -- 172.21.15.88:0/3724749780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67dc1a0160 con 0x7f67dc100d70 2024-02-19T19:05:11.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.262+0000 7f67e3e6a700 1 -- 172.21.15.88:0/3724749780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc106bd0 con 0x7f67dc100d70 2024-02-19T19:05:11.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.263+0000 7f67d2ffd700 1 -- 172.21.15.88:0/3724749780 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f67d8004750 con 0x7f67dc100d70 2024-02-19T19:05:11.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.264+0000 7f67d2ffd700 1 --2- 172.21.15.88:0/3724749780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67c806da30 0x7f67c806fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:11.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.264+0000 7f67d2ffd700 1 -- 172.21.15.88:0/3724749780 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f67d8014070 con 0x7f67dc100d70 2024-02-19T19:05:11.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.264+0000 7f67e1c06700 1 --2- 172.21.15.88:0/3724749780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67c806da30 0x7f67c806fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:11.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.265+0000 7f67e1c06700 1 --2- 172.21.15.88:0/3724749780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67c806da30 0x7f67c806fee0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00b760 tx=0x7f67cc005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:11.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.272+0000 7f67d2ffd700 1 -- 172.21.15.88:0/3724749780 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f67d8057090 con 0x7f67dc100d70 2024-02-19T19:05:11.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.620+0000 7f67e3e6a700 1 -- 172.21.15.88:0/3724749780 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67dc02cca0 con 0x7f67c806da30 2024-02-19T19:05:11.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.622+0000 7f67d2ffd700 1 -- 172.21.15.88:0/3724749780 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f67dc02cca0 con 0x7f67c806da30 2024-02-19T19:05:11.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.624+0000 7f67e3e6a700 1 -- 172.21.15.88:0/3724749780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67c806da30 msgr2=0x7f67c806fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:11.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.624+0000 7f67e3e6a700 1 --2- 172.21.15.88:0/3724749780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67c806da30 0x7f67c806fee0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00b760 tx=0x7f67cc005940 comp rx=0 tx=0).stop 2024-02-19T19:05:11.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.624+0000 7f67e3e6a700 1 -- 172.21.15.88:0/3724749780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67dc100d70 msgr2=0x7f67dc1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:11.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.624+0000 7f67e3e6a700 1 --2- 172.21.15.88:0/3724749780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67dc100d70 0x7f67dc1971d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f67d800e940 tx=0x7f67d800ed00 comp rx=0 tx=0).stop 2024-02-19T19:05:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.624+0000 7f67e3e6a700 1 -- 172.21.15.88:0/3724749780 shutdown_connections 2024-02-19T19:05:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.624+0000 7f67e3e6a700 1 --2- 172.21.15.88:0/3724749780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67c806da30 0x7f67c806fee0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.624+0000 7f67e3e6a700 1 --2- 172.21.15.88:0/3724749780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67dc100460 0x7f67dc196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.624+0000 7f67e3e6a700 1 --2- 172.21.15.88:0/3724749780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67dc100d70 0x7f67dc1971d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.624+0000 7f67e3e6a700 1 -- 172.21.15.88:0/3724749780 >> 172.21.15.88:0/3724749780 conn(0x7f67dc0748b0 msgr2=0x7f67dc103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.625+0000 7f67e3e6a700 1 -- 172.21.15.88:0/3724749780 shutdown_connections 2024-02-19T19:05:11.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.625+0000 7f67e3e6a700 1 -- 172.21.15.88:0/3724749780 wait complete. 2024-02-19T19:05:11.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.750+0000 7f1d43510700 1 -- 172.21.15.88:0/236380531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d3c0ffbd0 msgr2=0x7f1d3c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:11.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.750+0000 7f1d43510700 1 --2- 172.21.15.88:0/236380531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d3c0ffbd0 0x7f1d3c100040 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c009910 tx=0x7f1d2c009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:11.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.750+0000 7f1d43510700 1 -- 172.21.15.88:0/236380531 shutdown_connections 2024-02-19T19:05:11.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.750+0000 7f1d43510700 1 --2- 172.21.15.88:0/236380531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d3c0ffbd0 0x7f1d3c100040 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.750+0000 7f1d43510700 1 --2- 172.21.15.88:0/236380531 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c104130 0x7f1d3c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.750+0000 7f1d43510700 1 -- 172.21.15.88:0/236380531 >> 172.21.15.88:0/236380531 conn(0x7f1d3c075130 msgr2=0x7f1d3c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:11.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.750+0000 7f1d43510700 1 -- 172.21.15.88:0/236380531 shutdown_connections 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.750+0000 7f1d43510700 1 -- 172.21.15.88:0/236380531 wait complete. 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.751+0000 7f1d43510700 1 Processor -- start 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.751+0000 7f1d43510700 1 -- start start 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.751+0000 7f1d43510700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d3c0ffbd0 0x7f1d3c104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.751+0000 7f1d43510700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c104130 0x7f1d3c105500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.751+0000 7f1d43510700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c106d70 con 0x7f1d3c0ffbd0 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.751+0000 7f1d43510700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c106ee0 con 0x7f1d3c104130 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.752+0000 7f1d412ac700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d3c0ffbd0 0x7f1d3c104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.752+0000 7f1d412ac700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d3c0ffbd0 0x7f1d3c104fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48968/0 (socket says 172.21.15.88:48968) 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.752+0000 7f1d412ac700 1 -- 172.21.15.88:0/3240255260 learned_addr learned my addr 172.21.15.88:0/3240255260 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.752+0000 7f1d40aab700 1 --2- 172.21.15.88:0/3240255260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c104130 0x7f1d3c105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.752+0000 7f1d412ac700 1 -- 172.21.15.88:0/3240255260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c104130 msgr2=0x7f1d3c105500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:11.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.752+0000 7f1d412ac700 1 --2- 172.21.15.88:0/3240255260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c104130 0x7f1d3c105500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:11.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.752+0000 7f1d412ac700 1 -- 172.21.15.88:0/3240255260 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c0095f0 con 0x7f1d3c0ffbd0 2024-02-19T19:05:11.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.752+0000 7f1d412ac700 1 --2- 172.21.15.88:0/3240255260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d3c0ffbd0 0x7f1d3c104fc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800e940 tx=0x7f1d3800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:11.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.753+0000 7f1d327fc700 1 -- 172.21.15.88:0/3240255260 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d380097b0 con 0x7f1d3c0ffbd0 2024-02-19T19:05:11.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.753+0000 7f1d43510700 1 -- 172.21.15.88:0/3240255260 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3c105b00 con 0x7f1d3c0ffbd0 2024-02-19T19:05:11.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.753+0000 7f1d327fc700 1 -- 172.21.15.88:0/3240255260 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d38004cf0 con 0x7f1d3c0ffbd0 2024-02-19T19:05:11.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.753+0000 7f1d43510700 1 -- 172.21.15.88:0/3240255260 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c1a69e0 con 0x7f1d3c0ffbd0 2024-02-19T19:05:11.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.753+0000 7f1d327fc700 1 -- 172.21.15.88:0/3240255260 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d380097b0 con 0x7f1d3c0ffbd0 2024-02-19T19:05:11.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.754+0000 7f1d43510700 1 -- 172.21.15.88:0/3240255260 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c04f2a0 con 0x7f1d3c0ffbd0 2024-02-19T19:05:11.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.755+0000 7f1d327fc700 1 -- 172.21.15.88:0/3240255260 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f1d3800cba0 con 0x7f1d3c0ffbd0 2024-02-19T19:05:11.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.756+0000 7f1d327fc700 1 --2- 172.21.15.88:0/3240255260 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d2806da30 0x7f1d2806fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:11.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.756+0000 7f1d327fc700 1 -- 172.21.15.88:0/3240255260 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1d38014070 con 0x7f1d3c0ffbd0 2024-02-19T19:05:11.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.756+0000 7f1d40aab700 1 --2- 172.21.15.88:0/3240255260 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d2806da30 0x7f1d2806fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:11.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.757+0000 7f1d40aab700 1 --2- 172.21.15.88:0/3240255260 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d2806da30 0x7f1d2806fee0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c106860 tx=0x7f1d2c009f90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:11.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:11.761+0000 7f1d327fc700 1 -- 172.21.15.88:0/3240255260 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d38057770 con 0x7f1d3c0ffbd0 2024-02-19T19:05:12.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.050+0000 7f1d43510700 1 -- 172.21.15.88:0/3240255260 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d3c0ff150 con 0x7f1d2806da30 2024-02-19T19:05:12.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.062+0000 7f1d327fc700 1 -- 172.21.15.88:0/3240255260 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f1d3c0ff150 con 0x7f1d2806da30 2024-02-19T19:05:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:05:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (2m) 97s ago 12m 15.7M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (13m) 97s ago 13m 7843k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (11m) 4s ago 11m 7852k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (13m) 97s ago 13m 6665k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (11m) 4s ago 11m 6660k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (103s) 97s ago 12m 57.2M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (6m) 97s ago 6m 14.5M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (6m) 97s ago 6m 26.4M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (6m) 4s ago 6m 67.8M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (6m) 4s ago 6m 19.0M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (3m) 97s ago 14m 499M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (3m) 4s ago 11m 447M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (14m) 97s ago 14m 55.6M 2048M 18.2.0 dc2bc1663786 3ea5108455bc 2024-02-19T19:05:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (11m) 4s ago 11m 54.2M 2048M 18.2.0 dc2bc1663786 ba1c249c6017 2024-02-19T19:05:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (3m) 97s ago 13m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:05:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (3m) 4s ago 11m 15.9M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:05:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (10m) 97s ago 10m 52.2M 1216M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:05:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (9m) 97s ago 9m 49.8M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:05:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (9m) 97s ago 9m 49.3M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:05:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (8m) 4s ago 8m 172M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:05:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (8m) 4s ago 8m 175M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:05:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (8m) 4s ago 8m 192M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:05:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (2m) 97s ago 11m 54.5M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:05:12.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.065+0000 7f1d43510700 1 -- 172.21.15.88:0/3240255260 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d2806da30 msgr2=0x7f1d2806fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.065+0000 7f1d43510700 1 --2- 172.21.15.88:0/3240255260 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d2806da30 0x7f1d2806fee0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c106860 tx=0x7f1d2c009f90 comp rx=0 tx=0).stop 2024-02-19T19:05:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.065+0000 7f1d43510700 1 -- 172.21.15.88:0/3240255260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d3c0ffbd0 msgr2=0x7f1d3c104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.065+0000 7f1d43510700 1 --2- 172.21.15.88:0/3240255260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d3c0ffbd0 0x7f1d3c104fc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800e940 tx=0x7f1d3800ed00 comp rx=0 tx=0).stop 2024-02-19T19:05:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.065+0000 7f1d43510700 1 -- 172.21.15.88:0/3240255260 shutdown_connections 2024-02-19T19:05:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.065+0000 7f1d43510700 1 --2- 172.21.15.88:0/3240255260 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d2806da30 0x7f1d2806fee0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.065+0000 7f1d43510700 1 --2- 172.21.15.88:0/3240255260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d3c0ffbd0 0x7f1d3c104fc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.065+0000 7f1d43510700 1 --2- 172.21.15.88:0/3240255260 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d3c104130 0x7f1d3c105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.065+0000 7f1d43510700 1 -- 172.21.15.88:0/3240255260 >> 172.21.15.88:0/3240255260 conn(0x7f1d3c075130 msgr2=0x7f1d3c0fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.065+0000 7f1d43510700 1 -- 172.21.15.88:0/3240255260 shutdown_connections 2024-02-19T19:05:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.066+0000 7f1d43510700 1 -- 172.21.15.88:0/3240255260 wait complete. 2024-02-19T19:05:12.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:11 smithi088 ceph-mon[88643]: from='client.24587 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:12.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.198+0000 7fb456413700 1 -- 172.21.15.88:0/3472565942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4500687a0 msgr2=0x7fb450068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:12.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.198+0000 7fb456413700 1 --2- 172.21.15.88:0/3472565942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4500687a0 0x7fb450068b70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb440009910 tx=0x7fb440009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:12.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.198+0000 7fb456413700 1 -- 172.21.15.88:0/3472565942 shutdown_connections 2024-02-19T19:05:12.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.198+0000 7fb456413700 1 --2- 172.21.15.88:0/3472565942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb450069140 0x7fb450104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.198+0000 7fb456413700 1 --2- 172.21.15.88:0/3472565942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4500687a0 0x7fb450068b70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.198+0000 7fb456413700 1 -- 172.21.15.88:0/3472565942 >> 172.21.15.88:0/3472565942 conn(0x7fb450075130 msgr2=0x7fb450075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:12.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.198+0000 7fb456413700 1 -- 172.21.15.88:0/3472565942 shutdown_connections 2024-02-19T19:05:12.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.198+0000 7fb456413700 1 -- 172.21.15.88:0/3472565942 wait complete. 2024-02-19T19:05:12.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.199+0000 7fb456413700 1 Processor -- start 2024-02-19T19:05:12.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.199+0000 7fb456413700 1 -- start start 2024-02-19T19:05:12.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.199+0000 7fb456413700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4500687a0 0x7fb4500ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:12.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.199+0000 7fb456413700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb450069140 0x7fb4501001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:12.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.199+0000 7fb456413700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb450100720 con 0x7fb4500687a0 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.199+0000 7fb456413700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb450100860 con 0x7fb450069140 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.199+0000 7fb44ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4500687a0 0x7fb4500ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.199+0000 7fb44ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4500687a0 0x7fb4500ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48984/0 (socket says 172.21.15.88:48984) 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.199+0000 7fb44ffff700 1 -- 172.21.15.88:0/914801801 learned_addr learned my addr 172.21.15.88:0/914801801 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.200+0000 7fb44f7fe700 1 --2- 172.21.15.88:0/914801801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb450069140 0x7fb4501001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.200+0000 7fb44ffff700 1 -- 172.21.15.88:0/914801801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb450069140 msgr2=0x7fb4501001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.200+0000 7fb44ffff700 1 --2- 172.21.15.88:0/914801801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb450069140 0x7fb4501001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.200+0000 7fb44ffff700 1 -- 172.21.15.88:0/914801801 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4400095f0 con 0x7fb4500687a0 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.200+0000 7fb44ffff700 1 --2- 172.21.15.88:0/914801801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4500687a0 0x7fb4500ffca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb440005f50 tx=0x7fb44000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.200+0000 7fb44d7fa700 1 -- 172.21.15.88:0/914801801 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb44001d070 con 0x7fb4500687a0 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.200+0000 7fb44d7fa700 1 -- 172.21.15.88:0/914801801 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb44000b610 con 0x7fb4500687a0 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.200+0000 7fb456413700 1 -- 172.21.15.88:0/914801801 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb450100ae0 con 0x7fb4500687a0 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.200+0000 7fb44d7fa700 1 -- 172.21.15.88:0/914801801 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb440022470 con 0x7fb4500687a0 2024-02-19T19:05:12.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.200+0000 7fb456413700 1 -- 172.21.15.88:0/914801801 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4501a23e0 con 0x7fb4500687a0 2024-02-19T19:05:12.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.201+0000 7fb456413700 1 -- 172.21.15.88:0/914801801 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb450108790 con 0x7fb4500687a0 2024-02-19T19:05:12.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.202+0000 7fb44d7fa700 1 -- 172.21.15.88:0/914801801 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fb44000fca0 con 0x7fb4500687a0 2024-02-19T19:05:12.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.202+0000 7fb44d7fa700 1 --2- 172.21.15.88:0/914801801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb43806d9e0 0x7fb43806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:12.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.202+0000 7fb44d7fa700 1 -- 172.21.15.88:0/914801801 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb44008eb60 con 0x7fb4500687a0 2024-02-19T19:05:12.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.202+0000 7fb44f7fe700 1 --2- 172.21.15.88:0/914801801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb43806d9e0 0x7fb43806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:12.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.202+0000 7fb44f7fe700 1 --2- 172.21.15.88:0/914801801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb43806d9e0 0x7fb43806fe90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb450101490 tx=0x7fb444009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:12.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.207+0000 7fb44d7fa700 1 -- 172.21.15.88:0/914801801 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb4400581a0 con 0x7fb4500687a0 2024-02-19T19:05:12.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:11 smithi132 ceph-mon[37969]: from='client.24587 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:12.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.507+0000 7fb456413700 1 -- 172.21.15.88:0/914801801 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb45004f2a0 con 0x7fb4500687a0 2024-02-19T19:05:12.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.508+0000 7fb44d7fa700 1 -- 172.21.15.88:0/914801801 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+779 (secure 0 0 0) 0x7fb44005b860 con 0x7fb4500687a0 2024-02-19T19:05:12.528 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:05:12.528 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:05:12.528 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:05:12.528 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-19T19:05:12.528 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:05:12.528 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:05:12.528 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:05:12.528 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:05:12.528 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:05:12.528 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:05:12.529 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:05:12.529 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:05:12.529 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:05:12.529 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:05:12.529 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:05:12.529 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 11, 2024-02-19T19:05:12.529 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:05:12.529 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:05:12.529 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:05:12.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.510+0000 7fb456413700 1 -- 172.21.15.88:0/914801801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb43806d9e0 msgr2=0x7fb43806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:12.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.510+0000 7fb456413700 1 --2- 172.21.15.88:0/914801801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb43806d9e0 0x7fb43806fe90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb450101490 tx=0x7fb444009260 comp rx=0 tx=0).stop 2024-02-19T19:05:12.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.510+0000 7fb456413700 1 -- 172.21.15.88:0/914801801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4500687a0 msgr2=0x7fb4500ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:12.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.510+0000 7fb456413700 1 --2- 172.21.15.88:0/914801801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4500687a0 0x7fb4500ffca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb440005f50 tx=0x7fb44000f740 comp rx=0 tx=0).stop 2024-02-19T19:05:12.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.510+0000 7fb456413700 1 -- 172.21.15.88:0/914801801 shutdown_connections 2024-02-19T19:05:12.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.510+0000 7fb456413700 1 --2- 172.21.15.88:0/914801801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb43806d9e0 0x7fb43806fe90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.510+0000 7fb456413700 1 --2- 172.21.15.88:0/914801801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4500687a0 0x7fb4500ffca0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.510+0000 7fb456413700 1 --2- 172.21.15.88:0/914801801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb450069140 0x7fb4501001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.510+0000 7fb456413700 1 -- 172.21.15.88:0/914801801 >> 172.21.15.88:0/914801801 conn(0x7fb450075130 msgr2=0x7fb4500fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:12.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.510+0000 7fb456413700 1 -- 172.21.15.88:0/914801801 shutdown_connections 2024-02-19T19:05:12.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.510+0000 7fb456413700 1 -- 172.21.15.88:0/914801801 wait complete. 2024-02-19T19:05:12.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.618+0000 7f46970ba700 1 -- 172.21.15.88:0/2456010669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4690101420 msgr2=0x7f46901017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:12.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.618+0000 7f46970ba700 1 --2- 172.21.15.88:0/2456010669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4690101420 0x7f46901017f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4684009910 tx=0x7f4684009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:12.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.618+0000 7f46970ba700 1 -- 172.21.15.88:0/2456010669 shutdown_connections 2024-02-19T19:05:12.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.618+0000 7f46970ba700 1 --2- 172.21.15.88:0/2456010669 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4690101d30 0x7f469010a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.618+0000 7f46970ba700 1 --2- 172.21.15.88:0/2456010669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4690101420 0x7f46901017f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.618+0000 7f46970ba700 1 -- 172.21.15.88:0/2456010669 >> 172.21.15.88:0/2456010669 conn(0x7f46900fac60 msgr2=0x7f46900fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.618+0000 7f46970ba700 1 -- 172.21.15.88:0/2456010669 shutdown_connections 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.618+0000 7f46970ba700 1 -- 172.21.15.88:0/2456010669 wait complete. 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.618+0000 7f46970ba700 1 Processor -- start 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.618+0000 7f46970ba700 1 -- start start 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f46970ba700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4690101420 0x7f4690195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f46970ba700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4690101d30 0x7f4690196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f46970ba700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4690196af0 con 0x7f4690101420 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f46970ba700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469019a830 con 0x7f4690101d30 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f4694e56700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4690101420 0x7f4690195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f4694e56700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4690101420 0x7f4690195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48992/0 (socket says 172.21.15.88:48992) 2024-02-19T19:05:12.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f4694e56700 1 -- 172.21.15.88:0/2809189690 learned_addr learned my addr 172.21.15.88:0/2809189690 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:12.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f468ffff700 1 --2- 172.21.15.88:0/2809189690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4690101d30 0x7f4690196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:12.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f4694e56700 1 -- 172.21.15.88:0/2809189690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4690101d30 msgr2=0x7f4690196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:12.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f4694e56700 1 --2- 172.21.15.88:0/2809189690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4690101d30 0x7f4690196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f4694e56700 1 -- 172.21.15.88:0/2809189690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46840095f0 con 0x7f4690101420 2024-02-19T19:05:12.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f4694e56700 1 --2- 172.21.15.88:0/2809189690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4690101420 0x7f4690195ed0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4684005f50 tx=0x7f468400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:12.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f468dffb700 1 -- 172.21.15.88:0/2809189690 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f468401d070 con 0x7f4690101420 2024-02-19T19:05:12.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f46970ba700 1 -- 172.21.15.88:0/2809189690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f469019aab0 con 0x7f4690101420 2024-02-19T19:05:12.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f468dffb700 1 -- 172.21.15.88:0/2809189690 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f468400b610 con 0x7f4690101420 2024-02-19T19:05:12.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.619+0000 7f468dffb700 1 -- 172.21.15.88:0/2809189690 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4684022470 con 0x7f4690101420 2024-02-19T19:05:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.620+0000 7f46970ba700 1 -- 172.21.15.88:0/2809189690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469019afa0 con 0x7f4690101420 2024-02-19T19:05:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.621+0000 7f46970ba700 1 -- 172.21.15.88:0/2809189690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46901078d0 con 0x7f4690101420 2024-02-19T19:05:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.621+0000 7f468dffb700 1 -- 172.21.15.88:0/2809189690 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f4684003780 con 0x7f4690101420 2024-02-19T19:05:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.622+0000 7f468dffb700 1 --2- 172.21.15.88:0/2809189690 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f468006d9e0 0x7f468006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.622+0000 7f468dffb700 1 -- 172.21.15.88:0/2809189690 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f468408ead0 con 0x7f4690101420 2024-02-19T19:05:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.622+0000 7f468ffff700 1 --2- 172.21.15.88:0/2809189690 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f468006d9e0 0x7f468006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:12.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.622+0000 7f468ffff700 1 --2- 172.21.15.88:0/2809189690 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f468006d9e0 0x7f468006fe90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f46901974a0 tx=0x7f467c009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:12.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.627+0000 7f468dffb700 1 -- 172.21.15.88:0/2809189690 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4684058110 con 0x7f4690101420 2024-02-19T19:05:12.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.882+0000 7f46970ba700 1 -- 172.21.15.88:0/2809189690 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f469004f2a0 con 0x7f4690101420 2024-02-19T19:05:12.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.883+0000 7f468dffb700 1 -- 172.21.15.88:0/2809189690 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1875 (secure 0 0 0) 0x7f468405b7d0 con 0x7f4690101420 2024-02-19T19:05:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:e19 2024-02-19T19:05:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:05:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:05:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:05:12.903 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:05:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:05:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:05:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 19 2024-02-19T19:05:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:05:12.903 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:04:10.337219+0000 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:05:12.904 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:05:12.905 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:05:12.906 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:05:12.906 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:05:12.906 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:12.906 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.885+0000 7f46970ba700 1 -- 172.21.15.88:0/2809189690 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f468006d9e0 msgr2=0x7f468006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.885+0000 7f46970ba700 1 --2- 172.21.15.88:0/2809189690 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f468006d9e0 0x7f468006fe90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f46901974a0 tx=0x7f467c009260 comp rx=0 tx=0).stop 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.885+0000 7f46970ba700 1 -- 172.21.15.88:0/2809189690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4690101420 msgr2=0x7f4690195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.885+0000 7f46970ba700 1 --2- 172.21.15.88:0/2809189690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4690101420 0x7f4690195ed0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4684005f50 tx=0x7f468400f690 comp rx=0 tx=0).stop 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.885+0000 7f46970ba700 1 -- 172.21.15.88:0/2809189690 shutdown_connections 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.885+0000 7f46970ba700 1 --2- 172.21.15.88:0/2809189690 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f468006d9e0 0x7f468006fe90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.885+0000 7f46970ba700 1 --2- 172.21.15.88:0/2809189690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4690101420 0x7f4690195ed0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.885+0000 7f46970ba700 1 --2- 172.21.15.88:0/2809189690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4690101d30 0x7f4690196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.885+0000 7f46970ba700 1 -- 172.21.15.88:0/2809189690 >> 172.21.15.88:0/2809189690 conn(0x7f46900fac60 msgr2=0x7f4690104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.886+0000 7f46970ba700 1 -- 172.21.15.88:0/2809189690 shutdown_connections 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:12.886+0000 7f46970ba700 1 -- 172.21.15.88:0/2809189690 wait complete. 2024-02-19T19:05:12.907 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 19 2024-02-19T19:05:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.005+0000 7f9168ecd700 1 -- 172.21.15.88:0/1775663757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91641038b0 msgr2=0x7f9164103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.005+0000 7f9168ecd700 1 --2- 172.21.15.88:0/1775663757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91641038b0 0x7f9164103c80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9154009910 tx=0x7f9154009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.006+0000 7f9168ecd700 1 -- 172.21.15.88:0/1775663757 shutdown_connections 2024-02-19T19:05:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.006+0000 7f9168ecd700 1 --2- 172.21.15.88:0/1775663757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91640ff350 0x7f91640ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.006+0000 7f9168ecd700 1 --2- 172.21.15.88:0/1775663757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91641038b0 0x7f9164103c80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.006+0000 7f9168ecd700 1 -- 172.21.15.88:0/1775663757 >> 172.21.15.88:0/1775663757 conn(0x7f91640748b0 msgr2=0x7f9164074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.006+0000 7f9168ecd700 1 -- 172.21.15.88:0/1775663757 shutdown_connections 2024-02-19T19:05:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.006+0000 7f9168ecd700 1 -- 172.21.15.88:0/1775663757 wait complete. 2024-02-19T19:05:13.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.006+0000 7f9168ecd700 1 Processor -- start 2024-02-19T19:05:13.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.006+0000 7f9168ecd700 1 -- start start 2024-02-19T19:05:13.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9168ecd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91640ff350 0x7f9164196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:13.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9168ecd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91641038b0 0x7f91641971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:13.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9168ecd700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91641978b0 con 0x7f91641038b0 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9168ecd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9164197a20 con 0x7f91640ff350 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9161d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91641038b0 0x7f91641971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9161d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91641038b0 0x7f91641971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49010/0 (socket says 172.21.15.88:49010) 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9161d9b700 1 -- 172.21.15.88:0/2843845782 learned_addr learned my addr 172.21.15.88:0/2843845782 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f916259c700 1 --2- 172.21.15.88:0/2843845782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91640ff350 0x7f9164196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9161d9b700 1 -- 172.21.15.88:0/2843845782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91640ff350 msgr2=0x7f9164196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9161d9b700 1 --2- 172.21.15.88:0/2843845782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91640ff350 0x7f9164196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9161d9b700 1 -- 172.21.15.88:0/2843845782 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91540095f0 con 0x7f91641038b0 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9161d9b700 1 --2- 172.21.15.88:0/2843845782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91641038b0 0x7f91641971d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f915800d8d0 tx=0x7f915800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f91537fe700 1 -- 172.21.15.88:0/2843845782 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91580098a0 con 0x7f91641038b0 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f91537fe700 1 -- 172.21.15.88:0/2843845782 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9158010460 con 0x7f91641038b0 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9168ecd700 1 -- 172.21.15.88:0/2843845782 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916419fd00 con 0x7f91641038b0 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f91537fe700 1 -- 172.21.15.88:0/2843845782 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f915800f5f0 con 0x7f91641038b0 2024-02-19T19:05:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.007+0000 7f9168ecd700 1 -- 172.21.15.88:0/2843845782 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91641a0250 con 0x7f91641038b0 2024-02-19T19:05:13.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.008+0000 7f9168ecd700 1 -- 172.21.15.88:0/2843845782 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9164107530 con 0x7f91641038b0 2024-02-19T19:05:13.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.009+0000 7f91537fe700 1 -- 172.21.15.88:0/2843845782 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f9158009a00 con 0x7f91641038b0 2024-02-19T19:05:13.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.009+0000 7f91537fe700 1 --2- 172.21.15.88:0/2843845782 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f914c06d9d0 0x7f914c06fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:13.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.009+0000 7f91537fe700 1 -- 172.21.15.88:0/2843845782 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f915808d5b0 con 0x7f91641038b0 2024-02-19T19:05:13.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.009+0000 7f916259c700 1 --2- 172.21.15.88:0/2843845782 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f914c06d9d0 0x7f914c06fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:13.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.010+0000 7f916259c700 1 --2- 172.21.15.88:0/2843845782 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f914c06d9d0 0x7f914c06fe80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9154006010 tx=0x7f915401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:13.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.014+0000 7f91537fe700 1 -- 172.21.15.88:0/2843845782 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9158056bf0 con 0x7f91641038b0 2024-02-19T19:05:13.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:12 smithi132 ceph-mon[37969]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:13.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:12 smithi132 ceph-mon[37969]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:13.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:12 smithi132 ceph-mon[37969]: pgmap v10: 65 pgs: 65 active+clean; 1.8 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 53 MiB/s rd, 106 MiB/s wr, 278 op/s 2024-02-19T19:05:13.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:12 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/914801801' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:05:13.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.286+0000 7f9168ecd700 1 -- 172.21.15.88:0/2843845782 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9164061190 con 0x7f914c06d9d0 2024-02-19T19:05:13.306 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:12 smithi088 ceph-mon[88643]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:13.306 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:12 smithi088 ceph-mon[88643]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:13.306 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:12 smithi088 ceph-mon[88643]: pgmap v10: 65 pgs: 65 active+clean; 1.8 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 53 MiB/s rd, 106 MiB/s wr, 278 op/s 2024-02-19T19:05:13.306 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:12 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/914801801' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:05:13.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.287+0000 7f91537fe700 1 -- 172.21.15.88:0/2843845782 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f9164061190 con 0x7f914c06d9d0 2024-02-19T19:05:13.307 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:05:13.307 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:05:13.307 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:05:13.307 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:05:13.307 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:05:13.307 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:05:13.307 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:05:13.307 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "2/23 daemons upgraded", 2024-02-19T19:05:13.308 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "", 2024-02-19T19:05:13.308 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:05:13.308 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:05:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.290+0000 7f9168ecd700 1 -- 172.21.15.88:0/2843845782 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f914c06d9d0 msgr2=0x7f914c06fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.290+0000 7f9168ecd700 1 --2- 172.21.15.88:0/2843845782 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f914c06d9d0 0x7f914c06fe80 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9154006010 tx=0x7f915401a040 comp rx=0 tx=0).stop 2024-02-19T19:05:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.290+0000 7f9168ecd700 1 -- 172.21.15.88:0/2843845782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91641038b0 msgr2=0x7f91641971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.290+0000 7f9168ecd700 1 --2- 172.21.15.88:0/2843845782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91641038b0 0x7f91641971d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f915800d8d0 tx=0x7f915800dbe0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.290+0000 7f9168ecd700 1 -- 172.21.15.88:0/2843845782 shutdown_connections 2024-02-19T19:05:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.290+0000 7f9168ecd700 1 --2- 172.21.15.88:0/2843845782 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f914c06d9d0 0x7f914c06fe80 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.290+0000 7f9168ecd700 1 --2- 172.21.15.88:0/2843845782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f91640ff350 0x7f9164196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.290+0000 7f9168ecd700 1 --2- 172.21.15.88:0/2843845782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91641038b0 0x7f91641971d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.290+0000 7f9168ecd700 1 -- 172.21.15.88:0/2843845782 >> 172.21.15.88:0/2843845782 conn(0x7f91640748b0 msgr2=0x7f9164104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.290+0000 7f9168ecd700 1 -- 172.21.15.88:0/2843845782 shutdown_connections 2024-02-19T19:05:13.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.290+0000 7f9168ecd700 1 -- 172.21.15.88:0/2843845782 wait complete. 2024-02-19T19:05:13.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.409+0000 7f7702a61700 1 -- 172.21.15.88:0/3833166463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc104130 msgr2=0x7f76fc104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:13.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.409+0000 7f7702a61700 1 --2- 172.21.15.88:0/3833166463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc104130 0x7f76fc104500 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009910 tx=0x7f76ec009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:13.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.409+0000 7f7702a61700 1 -- 172.21.15.88:0/3833166463 shutdown_connections 2024-02-19T19:05:13.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.409+0000 7f7702a61700 1 --2- 172.21.15.88:0/3833166463 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76fc0ffbd0 0x7f76fc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.409+0000 7f7702a61700 1 --2- 172.21.15.88:0/3833166463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc104130 0x7f76fc104500 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.409+0000 7f7702a61700 1 -- 172.21.15.88:0/3833166463 >> 172.21.15.88:0/3833166463 conn(0x7f76fc075130 msgr2=0x7f76fc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:13.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.409+0000 7f7702a61700 1 -- 172.21.15.88:0/3833166463 shutdown_connections 2024-02-19T19:05:13.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.409+0000 7f7702a61700 1 -- 172.21.15.88:0/3833166463 wait complete. 2024-02-19T19:05:13.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f7702a61700 1 Processor -- start 2024-02-19T19:05:13.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f7702a61700 1 -- start start 2024-02-19T19:05:13.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f7702a61700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76fc0ffbd0 0x7f76fc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:13.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f7702a61700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc104130 0x7f76fc198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:13.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f7702a61700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc198c70 con 0x7f76fc104130 2024-02-19T19:05:13.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f7702a61700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76fc19ca00 con 0x7f76fc0ffbd0 2024-02-19T19:05:13.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f76fb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc104130 0x7f76fc198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:13.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f76fb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc104130 0x7f76fc198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49026/0 (socket says 172.21.15.88:49026) 2024-02-19T19:05:13.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f76fb7fe700 1 -- 172.21.15.88:0/601058390 learned_addr learned my addr 172.21.15.88:0/601058390 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:13.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f76fb7fe700 1 -- 172.21.15.88:0/601058390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76fc0ffbd0 msgr2=0x7f76fc198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:13.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f76fb7fe700 1 --2- 172.21.15.88:0/601058390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76fc0ffbd0 0x7f76fc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.410+0000 7f76fb7fe700 1 -- 172.21.15.88:0/601058390 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ec0095f0 con 0x7f76fc104130 2024-02-19T19:05:13.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.411+0000 7f76fb7fe700 1 --2- 172.21.15.88:0/601058390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc104130 0x7f76fc198590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f76f000e920 tx=0x7f76f000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:13.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.411+0000 7f76f97fa700 1 -- 172.21.15.88:0/601058390 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76f0009790 con 0x7f76fc104130 2024-02-19T19:05:13.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.411+0000 7f76f97fa700 1 -- 172.21.15.88:0/601058390 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f76f000cba0 con 0x7f76fc104130 2024-02-19T19:05:13.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.411+0000 7f76f97fa700 1 -- 172.21.15.88:0/601058390 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76f0018650 con 0x7f76fc104130 2024-02-19T19:05:13.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.411+0000 7f7702a61700 1 -- 172.21.15.88:0/601058390 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76fc19cce0 con 0x7f76fc104130 2024-02-19T19:05:13.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.411+0000 7f7702a61700 1 -- 172.21.15.88:0/601058390 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76fc19d1e0 con 0x7f76fc104130 2024-02-19T19:05:13.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.412+0000 7f7702a61700 1 -- 172.21.15.88:0/601058390 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76fc192040 con 0x7f76fc104130 2024-02-19T19:05:13.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.413+0000 7f76f97fa700 1 -- 172.21.15.88:0/601058390 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f76f000cd10 con 0x7f76fc104130 2024-02-19T19:05:13.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.413+0000 7f76f97fa700 1 --2- 172.21.15.88:0/601058390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f76e406d9c0 0x7f76e406fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:13.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.413+0000 7f76f97fa700 1 -- 172.21.15.88:0/601058390 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f76f0014070 con 0x7f76fc104130 2024-02-19T19:05:13.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.413+0000 7f76fbfff700 1 --2- 172.21.15.88:0/601058390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f76e406d9c0 0x7f76e406fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:13.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.414+0000 7f76fbfff700 1 --2- 172.21.15.88:0/601058390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f76e406d9c0 0x7f76e406fe70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009fd0 tx=0x7f76ec000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:13.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.418+0000 7f76f97fa700 1 -- 172.21.15.88:0/601058390 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f76f0052d20 con 0x7f76fc104130 2024-02-19T19:05:13.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.779+0000 7f7702a61700 1 -- 172.21.15.88:0/601058390 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f76fc04f2a0 con 0x7f76fc104130 2024-02-19T19:05:13.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.780+0000 7f76f97fa700 1 -- 172.21.15.88:0/601058390 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f76f005acc0 con 0x7f76fc104130 2024-02-19T19:05:13.800 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_OK 2024-02-19T19:05:13.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.782+0000 7f7702a61700 1 -- 172.21.15.88:0/601058390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f76e406d9c0 msgr2=0x7f76e406fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:13.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.782+0000 7f7702a61700 1 --2- 172.21.15.88:0/601058390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f76e406d9c0 0x7f76e406fe70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009fd0 tx=0x7f76ec000bc0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.782+0000 7f7702a61700 1 -- 172.21.15.88:0/601058390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc104130 msgr2=0x7f76fc198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:13.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.782+0000 7f7702a61700 1 --2- 172.21.15.88:0/601058390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc104130 0x7f76fc198590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f76f000e920 tx=0x7f76f000ece0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.782+0000 7f7702a61700 1 -- 172.21.15.88:0/601058390 shutdown_connections 2024-02-19T19:05:13.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.782+0000 7f7702a61700 1 --2- 172.21.15.88:0/601058390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f76e406d9c0 0x7f76e406fe70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.782+0000 7f7702a61700 1 --2- 172.21.15.88:0/601058390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f76fc0ffbd0 0x7f76fc198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.782+0000 7f7702a61700 1 --2- 172.21.15.88:0/601058390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76fc104130 0x7f76fc198590 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:13.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.782+0000 7f7702a61700 1 -- 172.21.15.88:0/601058390 >> 172.21.15.88:0/601058390 conn(0x7f76fc075130 msgr2=0x7f76fc1097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:13.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.782+0000 7f7702a61700 1 -- 172.21.15.88:0/601058390 shutdown_connections 2024-02-19T19:05:13.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:13.782+0000 7f7702a61700 1 -- 172.21.15.88:0/601058390 wait complete. 2024-02-19T19:05:14.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:13 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2809189690' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:05:14.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:05:14.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:13 smithi088 ceph-mon[88643]: from='client.34148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:14.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:13 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/601058390' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:05:14.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:13 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/2809189690' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:05:14.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:13 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:05:14.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:13 smithi132 ceph-mon[37969]: from='client.34148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:14.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:13 smithi132 ceph-mon[37969]: from='client.? 172.21.15.88:0/601058390' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:05:15.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:14 smithi088 ceph-mon[88643]: pgmap v11: 65 pgs: 65 active+clean; 1.8 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 53 MiB/s rd, 106 MiB/s wr, 278 op/s 2024-02-19T19:05:15.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:14 smithi132 ceph-mon[37969]: pgmap v11: 65 pgs: 65 active+clean; 1.8 GiB data, 6.0 GiB used, 530 GiB / 536 GiB avail; 53 MiB/s rd, 106 MiB/s wr, 278 op/s 2024-02-19T19:05:16.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:15 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:16.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:15 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:16.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:16.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:17.143 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:16 smithi088 ceph-mon[88643]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:05:17.143 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:16 smithi088 ceph-mon[88643]: pgmap v12: 65 pgs: 65 active+clean; 1.0 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 53 MiB/s rd, 106 MiB/s wr, 339 op/s 2024-02-19T19:05:17.143 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:17.143 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:17.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:16 smithi132 ceph-mon[37969]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:05:17.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:16 smithi132 ceph-mon[37969]: pgmap v12: 65 pgs: 65 active+clean; 1.0 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 53 MiB/s rd, 106 MiB/s wr, 339 op/s 2024-02-19T19:05:17.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:16 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:17.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:16 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:18.040 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-19T19:05:18.040 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-02-19T19:05:19.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:19 smithi132 ceph-mon[37969]: pgmap v13: 65 pgs: 65 active+clean; 1.0 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 11 MiB/s rd, 24 MiB/s wr, 144 op/s 2024-02-19T19:05:19.510 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:19 smithi088 ceph-mon[88643]: pgmap v13: 65 pgs: 65 active+clean; 1.0 GiB data, 3.8 GiB used, 533 GiB / 536 GiB avail; 11 MiB/s rd, 24 MiB/s wr, 144 op/s 2024-02-19T19:05:21.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:21 smithi132 ceph-mon[37969]: pgmap v14: 65 pgs: 65 active+clean; 161 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 24 MiB/s wr, 209 op/s 2024-02-19T19:05:21.434 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:21 smithi088 ceph-mon[88643]: pgmap v14: 65 pgs: 65 active+clean; 161 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 24 MiB/s wr, 209 op/s 2024-02-19T19:05:22.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: Detected new or changed devices on smithi088 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: Updating smithi088:/etc/ceph/ceph.conf 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: Updating smithi132:/etc/ceph/ceph.conf 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: pgmap v15: 65 pgs: 65 active+clean; 161 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 911 KiB/s wr, 125 op/s 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:05:22.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:22 smithi132 ceph-mon[37969]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:05:22.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: Detected new or changed devices on smithi088 2024-02-19T19:05:22.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:22.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:22.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:22.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:05:22.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: Updating smithi088:/etc/ceph/ceph.conf 2024-02-19T19:05:22.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: Updating smithi132:/etc/ceph/ceph.conf 2024-02-19T19:05:22.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:05:22.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.conf 2024-02-19T19:05:22.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: Updating smithi132:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:05:22.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-02-19T19:05:22.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: pgmap v15: 65 pgs: 65 active+clean; 161 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 911 KiB/s wr, 125 op/s 2024-02-19T19:05:22.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: Updating smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:05:22.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:22 smithi088 ceph-mon[88643]: Updating smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/config/ceph.client.admin.keyring 2024-02-19T19:05:23.601 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:23 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:23.601 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:23 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:23.601 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:23 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:23.601 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:23 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:23.601 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:23 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:23.601 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:23 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:05:23.601 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:23 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:05:23.601 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:23 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-19T19:05:23.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:23.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:23.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:23.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:23.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:23.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:05:23.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:05:23.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-02-19T19:05:24.704 INFO:tasks.workunit.client.0.smithi088.stderr:+ tar xzf ltp-full.tgz 2024-02-19T19:05:24.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:24 smithi088 ceph-mon[88643]: pgmap v16: 65 pgs: 65 active+clean; 161 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 911 KiB/s wr, 125 op/s 2024-02-19T19:05:24.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:24.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T19:05:24.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T19:05:24.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:24.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:24 smithi132 ceph-mon[37969]: pgmap v16: 65 pgs: 65 active+clean; 161 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 911 KiB/s wr, 125 op/s 2024-02-19T19:05:24.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:24 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:24.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:24 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T19:05:24.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:24 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T19:05:24.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:24 smithi132 ceph-mon[37969]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:25.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:25 smithi132 ceph-mon[37969]: Upgrade: Updating mon.smithi132 2024-02-19T19:05:25.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:25 smithi132 ceph-mon[37969]: Deploying daemon mon.smithi132 on smithi132 2024-02-19T19:05:25.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:25 smithi088 ceph-mon[88643]: Upgrade: Updating mon.smithi132 2024-02-19T19:05:25.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:25 smithi088 ceph-mon[88643]: Deploying daemon mon.smithi132 on smithi132 2024-02-19T19:05:26.820 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:26 smithi132 systemd[1]: Stopping Ceph mon.smithi132 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:05:26.821 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:26 smithi132 ceph-mon[37969]: pgmap v17: 65 pgs: 65 active+clean; 149 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-02-19T19:05:26.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:26 smithi088 ceph-mon[88643]: pgmap v17: 65 pgs: 65 active+clean; 149 MiB data, 824 MiB used, 536 GiB / 536 GiB avail; 30 KiB/s rd, 1.6 MiB/s wr, 233 op/s 2024-02-19T19:05:27.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:26 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132[37950]: 2024-02-19T19:05:26.800+0000 7f61e143a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi132 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T19:05:27.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:26 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132[37950]: 2024-02-19T19:05:26.800+0000 7f61e143a700 -1 mon.smithi132@1(peon) e2 *** Got Signal Terminated *** 2024-02-19T19:05:27.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:27 smithi132 podman[70516]: 2024-02-19 19:05:27.004210736 +0000 UTC m=+0.332797829 container died ba1c249c6017fc42706dce1d0b6d0f13df949c6cf99b9b3ebb870d9c5eb7d414 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD) 2024-02-19T19:05:30.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:29 smithi132 podman[70516]: 2024-02-19 19:05:29.552294839 +0000 UTC m=+2.880881929 container cleanup ba1c249c6017fc42706dce1d0b6d0f13df949c6cf99b9b3ebb870d9c5eb7d414 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-19T19:05:30.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:29 smithi132 bash[70516]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132 2024-02-19T19:05:30.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:29 smithi132 podman[70530]: 2024-02-19 19:05:29.770614439 +0000 UTC m=+2.764232501 container remove ba1c249c6017fc42706dce1d0b6d0f13df949c6cf99b9b3ebb870d9c5eb7d414 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0) 2024-02-19T19:05:30.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi132.service: Deactivated successfully. 2024-02-19T19:05:30.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 systemd[1]: Stopped Ceph mon.smithi132 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:05:30.296 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi132.service: Consumed 7.442s CPU time. 2024-02-19T19:05:30.566 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 systemd[1]: Starting Ceph mon.smithi132 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:05:30.566 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 podman[70658]: 2024-02-19 19:05:30.415129618 +0000 UTC m=+0.021543396 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:05:30.836 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 podman[70658]: 2024-02-19 19:05:30.547159246 +0000 UTC m=+0.153573006 container create dabaa14880db943321b7811ddf25b67634b11e3f2ba4729f86377603cceb603b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-02-19T19:05:30.837 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 podman[70658]: 2024-02-19 19:05:30.816950869 +0000 UTC m=+0.423364626 container init dabaa14880db943321b7811ddf25b67634b11e3f2ba4729f86377603cceb603b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-02-19T19:05:31.108 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 podman[70658]: 2024-02-19 19:05:30.820878121 +0000 UTC m=+0.427291887 container start dabaa14880db943321b7811ddf25b67634b11e3f2ba4729f86377603cceb603b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-02-19T19:05:31.108 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: set uid:gid to 167:167 (ceph:ceph) 2024-02-19T19:05:31.108 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev), process ceph-mon, pid 2 2024-02-19T19:05:31.108 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: pidfile_write: ignore empty --pid-file 2024-02-19T19:05:31.108 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: load: jerasure load: lrc 2024-02-19T19:05:31.108 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: RocksDB version: 7.9.2 2024-02-19T19:05:31.108 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Git sha 0 2024-02-19T19:05:31.108 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Compile date 2024-02-15 18:05:23 2024-02-19T19:05:31.108 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: DB SUMMARY 2024-02-19T19:05:31.108 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: DB Session ID: WJGAAKX9EE4YJ1ZI8ZVJ 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: CURRENT file: CURRENT 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: IDENTITY file: IDENTITY 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1403 Bytes 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi132/store.db dir, Total Num: 1, files: 000027.sst 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi132/store.db: 000025.log size: 9992169 ; 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.error_if_exists: 0 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.create_if_missing: 0 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.paranoid_checks: 1 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.env: 0x561b17809ae0 2024-02-19T19:05:31.109 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.fs: PosixFileSystem 2024-02-19T19:05:31.110 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.info_log: 0x561b19edf760 2024-02-19T19:05:31.110 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_file_opening_threads: 16 2024-02-19T19:05:31.110 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.statistics: (nil) 2024-02-19T19:05:31.110 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.use_fsync: 0 2024-02-19T19:05:31.110 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_log_file_size: 0 2024-02-19T19:05:31.110 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-19T19:05:31.110 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-19T19:05:31.110 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.keep_log_file_num: 1000 2024-02-19T19:05:31.110 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.recycle_log_file_num: 0 2024-02-19T19:05:31.110 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.allow_fallocate: 1 2024-02-19T19:05:31.111 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.allow_mmap_reads: 0 2024-02-19T19:05:31.111 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.allow_mmap_writes: 0 2024-02-19T19:05:31.111 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.use_direct_reads: 0 2024-02-19T19:05:31.111 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-19T19:05:31.111 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.create_missing_column_families: 0 2024-02-19T19:05:31.111 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.db_log_dir: 2024-02-19T19:05:31.111 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.wal_dir: 2024-02-19T19:05:31.111 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-19T19:05:31.111 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-19T19:05:31.111 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-19T19:05:31.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-19T19:05:31.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-19T19:05:31.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-19T19:05:31.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.advise_random_on_open: 1 2024-02-19T19:05:31.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.db_write_buffer_size: 0 2024-02-19T19:05:31.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.write_buffer_manager: 0x561b19eb3400 2024-02-19T19:05:31.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-19T19:05:31.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-19T19:05:31.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-19T19:05:31.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.rate_limiter: (nil) 2024-02-19T19:05:31.112 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.wal_recovery_mode: 2 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.enable_thread_tracking: 0 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.enable_pipelined_write: 0 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.unordered_write: 0 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.row_cache: None 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.wal_filter: None 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.allow_ingest_behind: 0 2024-02-19T19:05:31.113 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.two_write_queues: 0 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.manual_wal_flush: 0 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.wal_compression: 0 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.atomic_flush: 0 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.log_readahead_size: 0 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.best_efforts_recovery: 0 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.allow_data_in_errors: 0 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.db_host_id: __hostname__ 2024-02-19T19:05:31.114 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_background_jobs: 2 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_background_compactions: -1 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_subcompactions: 1 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_total_wal_size: 0 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_open_files: -1 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bytes_per_sync: 0 2024-02-19T19:05:31.115 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_readahead_size: 0 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_background_flushes: -1 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Compression algorithms supported: 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: kZSTD supported: 0 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: kXpressCompression supported: 0 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: kLZ4HCCompression supported: 1 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: kZlibCompression supported: 1 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: kSnappyCompression supported: 1 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: kLZ4Compression supported: 1 2024-02-19T19:05:31.116 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: kBZip2Compression supported: 0 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi132/store.db/MANIFEST-000010 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.merge_operator: 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_filter: None 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_filter_factory: None 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.sst_partitioner_factory: None 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561b19edf880) 2024-02-19T19:05:31.117 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: cache_index_and_filter_blocks: 1 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: pin_top_level_index_and_filter: 1 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: index_type: 0 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: data_block_index_type: 0 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: index_shortening: 1 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: checksum: 4 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: no_block_cache: 0 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_cache: 0x561b19ee3090 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_cache_name: BinnedLRUCache 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_cache_options: 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: capacity : 536870912 2024-02-19T19:05:31.118 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: num_shard_bits : 4 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: strict_capacity_limit : 0 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: high_pri_pool_ratio: 0.000 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_cache_compressed: (nil) 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: persistent_cache: (nil) 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_size: 4096 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_size_deviation: 10 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_restart_interval: 16 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: index_block_restart_interval: 1 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: metadata_block_size: 4096 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: partition_filters: 0 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: use_delta_encoding: 1 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: filter_policy: bloomfilter 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: whole_key_filtering: 1 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: verify_compression: 0 2024-02-19T19:05:31.119 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: read_amp_bytes_per_bit: 0 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: format_version: 5 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: enable_index_compression: 1 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: block_align: 0 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: max_auto_readahead_size: 262144 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: prepopulate_block_cache: 0 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: initial_auto_readahead_size: 8192 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: num_file_reads_for_auto_readahead: 2 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.write_buffer_size: 33554432 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_write_buffer_number: 2 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compression: NoCompression 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bottommost_compression: Disabled 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.prefix_extractor: nullptr 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-19T19:05:31.120 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.num_levels: 7 2024-02-19T19:05:31.121 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-19T19:05:31.121 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-19T19:05:31.121 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-19T19:05:31.121 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compression_opts.level: 32767 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compression_opts.strategy: 0 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-19T19:05:31.122 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compression_opts.enabled: false 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.target_file_size_base: 67108864 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-19T19:05:31.123 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.arena_block_size: 1048576 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.disable_auto_compactions: 0 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-19T19:05:31.124 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.table_properties_collectors: 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.inplace_update_support: 0 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-19T19:05:31.125 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.bloom_locality: 0 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.max_successive_merges: 0 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.paranoid_file_checks: 0 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.force_consistency_checks: 1 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.report_bg_io_stats: 0 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.ttl: 2592000 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.enable_blob_files: false 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.min_blob_size: 0 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.blob_file_size: 268435456 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-19T19:05:31.126 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.blob_file_starting_level: 0 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi132/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 11763, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6d76a768-93d4-4cfb-8a70-5188362b4b50 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708369530856785, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708369530882180, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 5908882, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 11768, "largest_seqno": 16138, "table_properties": {"data_size": 5893351, "index_size": 10094, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 4421, "raw_key_size": 38918, "raw_average_key_size": 22, "raw_value_size": 5858389, "raw_average_value_size": 3326, "num_data_blocks": 453, "num_entries": 1761, "num_filter_entries": 1761, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1708369530, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6d76a768-93d4-4cfb-8a70-5188362b4b50", "db_session_id": "WJGAAKX9EE4YJ1ZI8ZVJ", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: EVENT_LOG_v1 {"time_micros": 1708369530882275, "job": 1, "event": "recovery_finished"} 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-02-19T19:05:31.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi132/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561b19fd2000 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: DB pointer 0x561b19fbe000 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: ** DB Stats ** 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-19T19:05:31.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: ** Compaction Stats [default] ** 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: L0 1/0 5.64 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 506.2 0.01 0.00 1 0.011 0 0 0.0 0.0 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: L6 1/0 5.09 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Sum 2/0 10.73 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 506.2 0.01 0.00 1 0.011 0 0 0.0 0.0 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 506.2 0.01 0.00 1 0.011 0 0 0.0 0.0 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: ** Compaction Stats [default] ** 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 506.2 0.01 0.00 1 0.011 0 0 0.0 0.0 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T19:05:31.129 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Flush(GB): cumulative 0.006, interval 0.006 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Cumulative compaction: 0.01 GB write, 177.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Interval compaction: 0.01 GB write, 177.87 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Block cache BinnedLRUCache@0x561b19ee3090#2 capacity: 512.00 MB usage: 14.61 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.53 KB,0.000864267%) IndexBlock(1,10.08 KB,0.00192225%) Misc(1,0.00 KB,0%) 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T19:05:31.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: starting mon.smithi132 rank 1 at public addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] at bind addrs [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi132 fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: mon.smithi132@-1(???) e2 preinit fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: mon.smithi132@-1(???).mds e20 new map 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: mon.smithi132@-1(???).mds e20 print_map 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: e20 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: legacy client fscid: 1 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Filesystem 'cephfs' (1) 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: fs_name cephfs 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: epoch 20 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:05:31.131 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: modified 2024-02-19T19:05:14.913778+0000 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: tableserver 0 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: root 0 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: session_timeout 60 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: session_autoclose 300 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: max_file_size 1099511627776 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: max_xattr_size 65536 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: required_client_features {} 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: last_failure 0 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: last_failure_osd_epoch 46 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: max_mds 2 2024-02-19T19:05:31.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: in 0,1 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: up {0=14510,1=24263} 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: failed 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: damaged 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: stopped 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: data_pools [3] 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: metadata_pool 2 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: inline_data disabled 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: balancer 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: bal_rank_mask -1 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: standby_count_wanted 1 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: [mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:31.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: [mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: Standby daemons: 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: [mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout: [mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: mon.smithi132@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: mon.smithi132@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: mon.smithi132@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: mon.smithi132@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 ceph-mon[70676]: mon.smithi132@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 bash[70658]: dabaa14880db943321b7811ddf25b67634b11e3f2ba4729f86377603cceb603b 2024-02-19T19:05:31.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:30 smithi132 systemd[1]: Started Ceph mon.smithi132 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:05:34.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T19:05:34.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T19:05:34.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: mon.smithi088 calling monitor election 2024-02-19T19:05:34.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: mon.smithi132 calling monitor election 2024-02-19T19:05:34.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: mon.smithi088 is new leader, mons smithi088,smithi132 in quorum (ranks 0,1) 2024-02-19T19:05:34.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: monmap e3: 2 mons at {smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0],smithi132=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T19:05:34.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:05:34.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: osdmap e49: 6 total, 6 up, 6 in 2024-02-19T19:05:34.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: mgrmap e38: smithi088.gyviqq(active, since 35s), standbys: smithi132.psllsa 2024-02-19T19:05:34.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: overall HEALTH_OK 2024-02-19T19:05:34.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:34.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:34.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:05:34.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:34 smithi088 ceph-mon[88643]: pgmap v21: 65 pgs: 65 active+clean; 164 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 21 KiB/s rd, 2.3 MiB/s wr, 314 op/s 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi088"}]: dispatch 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mon metadata", "id": "smithi132"}]: dispatch 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: mon.smithi088 calling monitor election 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: mon.smithi132 calling monitor election 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: mon.smithi088 is new leader, mons smithi088,smithi132 in quorum (ranks 0,1) 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: monmap e3: 2 mons at {smithi088=[v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0],smithi132=[v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: osdmap e49: 6 total, 6 up, 6 in 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: mgrmap e38: smithi088.gyviqq(active, since 35s), standbys: smithi132.psllsa 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: overall HEALTH_OK 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:35.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:35.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:05:35.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:34 smithi132 ceph-mon[70676]: pgmap v21: 65 pgs: 65 active+clean; 164 MiB data, 859 MiB used, 536 GiB / 536 GiB avail; 21 KiB/s rd, 2.3 MiB/s wr, 314 op/s 2024-02-19T19:05:37.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:36 smithi132 ceph-mon[70676]: pgmap v22: 65 pgs: 65 active+clean; 176 MiB data, 887 MiB used, 536 GiB / 536 GiB avail; 33 KiB/s rd, 3.6 MiB/s wr, 513 op/s 2024-02-19T19:05:37.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:36 smithi088 ceph-mon[88643]: pgmap v22: 65 pgs: 65 active+clean; 176 MiB data, 887 MiB used, 536 GiB / 536 GiB avail; 33 KiB/s rd, 3.6 MiB/s wr, 513 op/s 2024-02-19T19:05:39.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:38 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:39.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:38 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:39.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:38 smithi132 ceph-mon[70676]: pgmap v23: 65 pgs: 65 active+clean; 176 MiB data, 887 MiB used, 536 GiB / 536 GiB avail; 22 KiB/s rd, 2.8 MiB/s wr, 405 op/s 2024-02-19T19:05:39.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:38 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:39.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:38 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:39.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:38 smithi088 ceph-mon[88643]: pgmap v23: 65 pgs: 65 active+clean; 176 MiB data, 887 MiB used, 536 GiB / 536 GiB avail; 22 KiB/s rd, 2.8 MiB/s wr, 405 op/s 2024-02-19T19:05:40.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:40 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:40.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:40 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:40.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:40 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:40.752 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:40 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:41.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:41 smithi132 ceph-mon[70676]: pgmap v24: 65 pgs: 65 active+clean; 189 MiB data, 960 MiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 4.5 MiB/s wr, 604 op/s 2024-02-19T19:05:41.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:41 smithi088 ceph-mon[88643]: pgmap v24: 65 pgs: 65 active+clean; 189 MiB data, 960 MiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 4.5 MiB/s wr, 604 op/s 2024-02-19T19:05:43.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:43 smithi132 ceph-mon[70676]: pgmap v25: 65 pgs: 65 active+clean; 189 MiB data, 960 MiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 3.0 MiB/s wr, 398 op/s 2024-02-19T19:05:43.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:05:43.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:43 smithi088 ceph-mon[88643]: pgmap v25: 65 pgs: 65 active+clean; 189 MiB data, 960 MiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 3.0 MiB/s wr, 398 op/s 2024-02-19T19:05:43.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:05:43.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.895+0000 7ff8bba7f700 1 -- 172.21.15.88:0/4213778157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b4104130 msgr2=0x7ff8b4104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:43.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.895+0000 7ff8bba7f700 1 --2- 172.21.15.88:0/4213778157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b4104130 0x7ff8b4104500 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009910 tx=0x7ff8b0009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:43.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.895+0000 7ff8bba7f700 1 -- 172.21.15.88:0/4213778157 shutdown_connections 2024-02-19T19:05:43.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.895+0000 7ff8bba7f700 1 --2- 172.21.15.88:0/4213778157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8b40ffbd0 0x7ff8b4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:43.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.895+0000 7ff8bba7f700 1 --2- 172.21.15.88:0/4213778157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b4104130 0x7ff8b4104500 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:43.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.895+0000 7ff8bba7f700 1 -- 172.21.15.88:0/4213778157 >> 172.21.15.88:0/4213778157 conn(0x7ff8b4075130 msgr2=0x7ff8b4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:43.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.895+0000 7ff8bba7f700 1 -- 172.21.15.88:0/4213778157 shutdown_connections 2024-02-19T19:05:43.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.895+0000 7ff8bba7f700 1 -- 172.21.15.88:0/4213778157 wait complete. 2024-02-19T19:05:43.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.895+0000 7ff8bba7f700 1 Processor -- start 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.895+0000 7ff8bba7f700 1 -- start start 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8bba7f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8b40ffbd0 0x7ff8b4195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8bba7f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b4104130 0x7ff8b4196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8bba7f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b4196a70 con 0x7ff8b4104130 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8bba7f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8b419a800 con 0x7ff8b40ffbd0 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8b901a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b4104130 0x7ff8b4196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8b901a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b4104130 0x7ff8b4196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36854/0 (socket says 172.21.15.88:36854) 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8b901a700 1 -- 172.21.15.88:0/2052902077 learned_addr learned my addr 172.21.15.88:0/2052902077 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8b981b700 1 --2- 172.21.15.88:0/2052902077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8b40ffbd0 0x7ff8b4195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8b901a700 1 -- 172.21.15.88:0/2052902077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8b40ffbd0 msgr2=0x7ff8b4195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8b901a700 1 --2- 172.21.15.88:0/2052902077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8b40ffbd0 0x7ff8b4195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:43.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8b901a700 1 -- 172.21.15.88:0/2052902077 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b00095f0 con 0x7ff8b4104130 2024-02-19T19:05:43.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8b901a700 1 --2- 172.21.15.88:0/2052902077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b4104130 0x7ff8b4196390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400e920 tx=0x7ff8a400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:43.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8aaffd700 1 -- 172.21.15.88:0/2052902077 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8a4009790 con 0x7ff8b4104130 2024-02-19T19:05:43.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8aaffd700 1 -- 172.21.15.88:0/2052902077 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8a400cba0 con 0x7ff8b4104130 2024-02-19T19:05:43.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8bba7f700 1 -- 172.21.15.88:0/2052902077 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8b419aae0 con 0x7ff8b4104130 2024-02-19T19:05:43.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.896+0000 7ff8aaffd700 1 -- 172.21.15.88:0/2052902077 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8a4018650 con 0x7ff8b4104130 2024-02-19T19:05:43.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.897+0000 7ff8bba7f700 1 -- 172.21.15.88:0/2052902077 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8b419b030 con 0x7ff8b4104130 2024-02-19T19:05:43.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.898+0000 7ff8bba7f700 1 -- 172.21.15.88:0/2052902077 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8b41086a0 con 0x7ff8b4104130 2024-02-19T19:05:43.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.898+0000 7ff8aaffd700 1 -- 172.21.15.88:0/2052902077 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7ff8a400cd10 con 0x7ff8b4104130 2024-02-19T19:05:43.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.898+0000 7ff8aaffd700 1 --2- 172.21.15.88:0/2052902077 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8a006d9e0 0x7ff8a006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:43.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.899+0000 7ff8aaffd700 1 -- 172.21.15.88:0/2052902077 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff8a4014070 con 0x7ff8b4104130 2024-02-19T19:05:43.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.899+0000 7ff8b981b700 1 --2- 172.21.15.88:0/2052902077 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8a006d9e0 0x7ff8a006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:43.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.899+0000 7ff8b981b700 1 --2- 172.21.15.88:0/2052902077 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8a006d9e0 0x7ff8a006fe90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005f50 tx=0x7ff8b0005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:43.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:43.904+0000 7ff8aaffd700 1 -- 172.21.15.88:0/2052902077 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff8a4052d20 con 0x7ff8b4104130 2024-02-19T19:05:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.196+0000 7ff8bba7f700 1 -- 172.21.15.88:0/2052902077 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8b419b220 con 0x7ff8a006d9e0 2024-02-19T19:05:44.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.198+0000 7ff8aaffd700 1 -- 172.21.15.88:0/2052902077 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7ff8b419b220 con 0x7ff8a006d9e0 2024-02-19T19:05:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.200+0000 7ff8bba7f700 1 -- 172.21.15.88:0/2052902077 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8a006d9e0 msgr2=0x7ff8a006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.200+0000 7ff8bba7f700 1 --2- 172.21.15.88:0/2052902077 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8a006d9e0 0x7ff8a006fe90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0005f50 tx=0x7ff8b0005e20 comp rx=0 tx=0).stop 2024-02-19T19:05:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.200+0000 7ff8bba7f700 1 -- 172.21.15.88:0/2052902077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b4104130 msgr2=0x7ff8b4196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.200+0000 7ff8bba7f700 1 --2- 172.21.15.88:0/2052902077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b4104130 0x7ff8b4196390 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff8a400e920 tx=0x7ff8a400ece0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.200+0000 7ff8bba7f700 1 -- 172.21.15.88:0/2052902077 shutdown_connections 2024-02-19T19:05:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.200+0000 7ff8bba7f700 1 --2- 172.21.15.88:0/2052902077 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8a006d9e0 0x7ff8a006fe90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.200+0000 7ff8bba7f700 1 --2- 172.21.15.88:0/2052902077 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8b40ffbd0 0x7ff8b4195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.200+0000 7ff8bba7f700 1 --2- 172.21.15.88:0/2052902077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8b4104130 0x7ff8b4196390 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.200+0000 7ff8bba7f700 1 -- 172.21.15.88:0/2052902077 >> 172.21.15.88:0/2052902077 conn(0x7ff8b4075130 msgr2=0x7ff8b40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.201+0000 7ff8bba7f700 1 -- 172.21.15.88:0/2052902077 shutdown_connections 2024-02-19T19:05:44.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.201+0000 7ff8bba7f700 1 -- 172.21.15.88:0/2052902077 wait complete. 2024-02-19T19:05:44.233 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:05:44.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.332+0000 7f56cf299700 1 -- 172.21.15.88:0/2521042842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c80ffd60 msgr2=0x7f56c8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:44.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.332+0000 7f56cf299700 1 --2- 172.21.15.88:0/2521042842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c80ffd60 0x7f56c8103d30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009910 tx=0x7f56b8009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:44.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.332+0000 7f56cf299700 1 -- 172.21.15.88:0/2521042842 shutdown_connections 2024-02-19T19:05:44.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.332+0000 7f56cf299700 1 --2- 172.21.15.88:0/2521042842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c80ffd60 0x7f56c8103d30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.332+0000 7f56cf299700 1 --2- 172.21.15.88:0/2521042842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c80ff3c0 0x7f56c80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.332+0000 7f56cf299700 1 -- 172.21.15.88:0/2521042842 >> 172.21.15.88:0/2521042842 conn(0x7f56c80fac60 msgr2=0x7f56c80fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:44.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.333+0000 7f56cf299700 1 -- 172.21.15.88:0/2521042842 shutdown_connections 2024-02-19T19:05:44.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.333+0000 7f56cf299700 1 -- 172.21.15.88:0/2521042842 wait complete. 2024-02-19T19:05:44.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.333+0000 7f56cf299700 1 Processor -- start 2024-02-19T19:05:44.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cf299700 1 -- start start 2024-02-19T19:05:44.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cf299700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c80ff3c0 0x7f56c8192bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:44.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cf299700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c80ffd60 0x7f56c8193110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:44.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cf299700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c8196d60 con 0x7f56c80ff3c0 2024-02-19T19:05:44.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cf299700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c8193650 con 0x7f56c80ffd60 2024-02-19T19:05:44.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cd035700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c80ff3c0 0x7f56c8192bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:44.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cd035700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c80ff3c0 0x7f56c8192bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36870/0 (socket says 172.21.15.88:36870) 2024-02-19T19:05:44.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cd035700 1 -- 172.21.15.88:0/2879429891 learned_addr learned my addr 172.21.15.88:0/2879429891 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:44.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cc834700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c80ffd60 0x7f56c8193110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:44.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cd035700 1 -- 172.21.15.88:0/2879429891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c80ffd60 msgr2=0x7f56c8193110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:44.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cd035700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c80ffd60 0x7f56c8193110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cd035700 1 -- 172.21.15.88:0/2879429891 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80095f0 con 0x7f56c80ff3c0 2024-02-19T19:05:44.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.334+0000 7f56cc834700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c80ffd60 0x7f56c8193110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:05:44.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.335+0000 7f56cd035700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c80ff3c0 0x7f56c8192bd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f56c400e920 tx=0x7f56c400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:44.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.335+0000 7f56be7fc700 1 -- 172.21.15.88:0/2879429891 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56c400c9d0 con 0x7f56c80ff3c0 2024-02-19T19:05:44.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.335+0000 7f56cf299700 1 -- 172.21.15.88:0/2879429891 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c8193930 con 0x7f56c80ff3c0 2024-02-19T19:05:44.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.335+0000 7f56be7fc700 1 -- 172.21.15.88:0/2879429891 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56c4010430 con 0x7f56c80ff3c0 2024-02-19T19:05:44.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.335+0000 7f56be7fc700 1 -- 172.21.15.88:0/2879429891 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56c4018630 con 0x7f56c80ff3c0 2024-02-19T19:05:44.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.335+0000 7f56cf299700 1 -- 172.21.15.88:0/2879429891 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c81a68f0 con 0x7f56c80ff3c0 2024-02-19T19:05:44.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.336+0000 7f56cf299700 1 -- 172.21.15.88:0/2879429891 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c8107730 con 0x7f56c80ff3c0 2024-02-19T19:05:44.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.336+0000 7f56be7fc700 1 -- 172.21.15.88:0/2879429891 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f56c40105a0 con 0x7f56c80ff3c0 2024-02-19T19:05:44.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.337+0000 7f56be7fc700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f56b406d9d0 0x7f56b406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:44.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.337+0000 7f56be7fc700 1 -- 172.21.15.88:0/2879429891 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f56c4014070 con 0x7f56c80ff3c0 2024-02-19T19:05:44.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.337+0000 7f56cc834700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f56b406d9d0 0x7f56b406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:44.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.338+0000 7f56cc834700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f56b406d9d0 0x7f56b406fe80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f56b8005b40 tx=0x7f56b8005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:44.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.343+0000 7f56be7fc700 1 -- 172.21.15.88:0/2879429891 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f56c4052d20 con 0x7f56c80ff3c0 2024-02-19T19:05:44.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.653+0000 7f56cf299700 1 -- 172.21.15.88:0/2879429891 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f56c8194270 con 0x7f56b406d9d0 2024-02-19T19:05:44.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.654+0000 7f56be7fc700 1 -- 172.21.15.88:0/2879429891 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f56b801e040 con 0x7f56b406d9d0 2024-02-19T19:05:44.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.656+0000 7f56cf299700 1 -- 172.21.15.88:0/2879429891 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f56b406d9d0 msgr2=0x7f56b406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:44.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.656+0000 7f56cf299700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f56b406d9d0 0x7f56b406fe80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f56b8005b40 tx=0x7f56b8005a90 comp rx=0 tx=0).stop 2024-02-19T19:05:44.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.656+0000 7f56cf299700 1 -- 172.21.15.88:0/2879429891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c80ff3c0 msgr2=0x7f56c8192bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:44.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.656+0000 7f56cf299700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c80ff3c0 0x7f56c8192bd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f56c400e920 tx=0x7f56c400ec30 comp rx=0 tx=0).stop 2024-02-19T19:05:44.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.656+0000 7f56cf299700 1 -- 172.21.15.88:0/2879429891 shutdown_connections 2024-02-19T19:05:44.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.656+0000 7f56cf299700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f56b406d9d0 0x7f56b406fe80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.656+0000 7f56cf299700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56c80ff3c0 0x7f56c8192bd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.656+0000 7f56cf299700 1 --2- 172.21.15.88:0/2879429891 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f56c80ffd60 0x7f56c8193110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.656+0000 7f56cf299700 1 -- 172.21.15.88:0/2879429891 >> 172.21.15.88:0/2879429891 conn(0x7f56c80fac60 msgr2=0x7f56c80fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:44.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.657+0000 7f56cf299700 1 -- 172.21.15.88:0/2879429891 shutdown_connections 2024-02-19T19:05:44.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.657+0000 7f56cf299700 1 -- 172.21.15.88:0/2879429891 wait complete. 2024-02-19T19:05:44.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: Detected new or changed devices on smithi132 2024-02-19T19:05:44.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:44.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:44.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:44.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:05:44.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:44.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: Reconfiguring mon.smithi088 (monmap changed)... 2024-02-19T19:05:44.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T19:05:44.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T19:05:44.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:44.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: Reconfiguring daemon mon.smithi088 on smithi088 2024-02-19T19:05:44.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: pgmap v26: 65 pgs: 65 active+clean; 189 MiB data, 960 MiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 3.0 MiB/s wr, 398 op/s 2024-02-19T19:05:44.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:44 smithi132 ceph-mon[70676]: from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:44.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.782+0000 7fedc1111700 1 -- 172.21.15.88:0/14115875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc1004e0 msgr2=0x7fedbc1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:44.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.782+0000 7fedc1111700 1 --2- 172.21.15.88:0/14115875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc1004e0 0x7fedbc1008b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fedac009910 tx=0x7fedac009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:44.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.782+0000 7fedc1111700 1 -- 172.21.15.88:0/14115875 shutdown_connections 2024-02-19T19:05:44.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.782+0000 7fedc1111700 1 --2- 172.21.15.88:0/14115875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedbc100e80 0x7fedbc104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.782+0000 7fedc1111700 1 --2- 172.21.15.88:0/14115875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc1004e0 0x7fedbc1008b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.782+0000 7fedc1111700 1 -- 172.21.15.88:0/14115875 >> 172.21.15.88:0/14115875 conn(0x7fedbc0fbda0 msgr2=0x7fedbc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:44.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.782+0000 7fedc1111700 1 -- 172.21.15.88:0/14115875 shutdown_connections 2024-02-19T19:05:44.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.782+0000 7fedc1111700 1 -- 172.21.15.88:0/14115875 wait complete. 2024-02-19T19:05:44.803 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: Detected new or changed devices on smithi132 2024-02-19T19:05:44.803 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:44.803 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:44.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:44.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:05:44.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:44.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: Reconfiguring mon.smithi088 (monmap changed)... 2024-02-19T19:05:44.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T19:05:44.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T19:05:44.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:44.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: Reconfiguring daemon mon.smithi088 on smithi088 2024-02-19T19:05:44.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: pgmap v26: 65 pgs: 65 active+clean; 189 MiB data, 960 MiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 3.0 MiB/s wr, 398 op/s 2024-02-19T19:05:44.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:44 smithi088 ceph-mon[88643]: from='client.34156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.783+0000 7fedc1111700 1 Processor -- start 2024-02-19T19:05:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.783+0000 7fedc1111700 1 -- start start 2024-02-19T19:05:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.783+0000 7fedc1111700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc1004e0 0x7fedbc195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.783+0000 7fedc1111700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedbc100e80 0x7fedbc1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.783+0000 7fedc1111700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc196bb0 con 0x7fedbc1004e0 2024-02-19T19:05:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.783+0000 7fedc1111700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedbc19a940 con 0x7fedbc100e80 2024-02-19T19:05:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7fedbad9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc1004e0 0x7fedbc195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7fedba59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedbc100e80 0x7fedbc1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7fedbad9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc1004e0 0x7fedbc195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36884/0 (socket says 172.21.15.88:36884) 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7fedbad9d700 1 -- 172.21.15.88:0/2565466145 learned_addr learned my addr 172.21.15.88:0/2565466145 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7fedbad9d700 1 -- 172.21.15.88:0/2565466145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedbc100e80 msgr2=0x7fedbc1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7fedbad9d700 1 --2- 172.21.15.88:0/2565466145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedbc100e80 0x7fedbc1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7fedbad9d700 1 -- 172.21.15.88:0/2565466145 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac0095f0 con 0x7fedbc1004e0 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7fedbad9d700 1 --2- 172.21.15.88:0/2565466145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc1004e0 0x7fedbc195f90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedac006010 tx=0x7fedac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7feda3fff700 1 -- 172.21.15.88:0/2565466145 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedac01d070 con 0x7fedbc1004e0 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7feda3fff700 1 -- 172.21.15.88:0/2565466145 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fedac00f460 con 0x7fedbc1004e0 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7fedc1111700 1 -- 172.21.15.88:0/2565466145 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedbc19abc0 con 0x7fedbc1004e0 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7feda3fff700 1 -- 172.21.15.88:0/2565466145 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fedac005de0 con 0x7fedbc1004e0 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.784+0000 7fedc1111700 1 -- 172.21.15.88:0/2565466145 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedbc19b0b0 con 0x7fedbc1004e0 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.785+0000 7fedc1111700 1 -- 172.21.15.88:0/2565466145 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedbc1087f0 con 0x7fedbc1004e0 2024-02-19T19:05:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.786+0000 7feda3fff700 1 -- 172.21.15.88:0/2565466145 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fedac00f5d0 con 0x7fedbc1004e0 2024-02-19T19:05:44.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.786+0000 7feda3fff700 1 --2- 172.21.15.88:0/2565466145 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feda406d9d0 0x7feda406fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:44.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.786+0000 7fedba59c700 1 --2- 172.21.15.88:0/2565466145 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feda406d9d0 0x7feda406fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:44.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.786+0000 7feda3fff700 1 -- 172.21.15.88:0/2565466145 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fedac08ebf0 con 0x7fedbc1004e0 2024-02-19T19:05:44.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.787+0000 7fedba59c700 1 --2- 172.21.15.88:0/2565466145 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feda406d9d0 0x7feda406fe80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fedbc1975b0 tx=0x7fedb0009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:44.791+0000 7feda3fff700 1 -- 172.21.15.88:0/2565466145 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fedac058230 con 0x7fedbc1004e0 2024-02-19T19:05:45.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.015+0000 7fedc1111700 1 -- 172.21.15.88:0/2565466145 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fedbc19b390 con 0x7feda406d9d0 2024-02-19T19:05:45.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.026+0000 7feda3fff700 1 -- 172.21.15.88:0/2565466145 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fedbc19b390 con 0x7feda406d9d0 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (2m) 30s ago 13m 25.0M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (13m) 30s ago 13m 7960k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (11m) 7s ago 11m 7902k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (13m) 30s ago 13m 6665k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (11m) 7s ago 11m 6660k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (2m) 30s ago 12m 83.1M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (7m) 30s ago 7m 15.3M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (7m) 30s ago 7m 107M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (7m) 7s ago 7m 119M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (6m) 7s ago 6m 19.6M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (4m) 30s ago 14m 476M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:05:45.047 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (3m) 7s ago 11m 448M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:05:45.048 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (60s) 30s ago 15m 32.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:05:45.048 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (14s) 7s ago 11m 26.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:05:45.048 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (3m) 30s ago 13m 15.8M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:05:45.048 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (3m) 7s ago 11m 15.2M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:05:45.048 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (10m) 30s ago 10m 176M 1216M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:05:45.048 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (10m) 30s ago 10m 181M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:05:45.048 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (9m) 30s ago 9m 147M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:05:45.048 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (9m) 7s ago 9m 164M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:05:45.048 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (9m) 7s ago 9m 149M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:05:45.048 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (8m) 7s ago 8m 144M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:05:45.048 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (3m) 30s ago 12m 64.8M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.029+0000 7fedc1111700 1 -- 172.21.15.88:0/2565466145 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feda406d9d0 msgr2=0x7feda406fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.029+0000 7fedc1111700 1 --2- 172.21.15.88:0/2565466145 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feda406d9d0 0x7feda406fe80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fedbc1975b0 tx=0x7fedb0009190 comp rx=0 tx=0).stop 2024-02-19T19:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.030+0000 7fedc1111700 1 -- 172.21.15.88:0/2565466145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc1004e0 msgr2=0x7fedbc195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.030+0000 7fedc1111700 1 --2- 172.21.15.88:0/2565466145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc1004e0 0x7fedbc195f90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedac006010 tx=0x7fedac00bd70 comp rx=0 tx=0).stop 2024-02-19T19:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.030+0000 7fedc1111700 1 -- 172.21.15.88:0/2565466145 shutdown_connections 2024-02-19T19:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.030+0000 7fedc1111700 1 --2- 172.21.15.88:0/2565466145 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feda406d9d0 0x7feda406fe80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.030+0000 7fedc1111700 1 --2- 172.21.15.88:0/2565466145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedbc1004e0 0x7fedbc195f90 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.030+0000 7fedc1111700 1 --2- 172.21.15.88:0/2565466145 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fedbc100e80 0x7fedbc1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.030+0000 7fedc1111700 1 -- 172.21.15.88:0/2565466145 >> 172.21.15.88:0/2565466145 conn(0x7fedbc0fbda0 msgr2=0x7fedbc0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:45.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.030+0000 7fedc1111700 1 -- 172.21.15.88:0/2565466145 shutdown_connections 2024-02-19T19:05:45.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.030+0000 7fedc1111700 1 -- 172.21.15.88:0/2565466145 wait complete. 2024-02-19T19:05:45.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.158+0000 7f6caedde700 1 -- 172.21.15.88:0/3640682497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca80ffc60 msgr2=0x7f6ca810a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:45.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.158+0000 7f6caedde700 1 --2- 172.21.15.88:0/3640682497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca80ffc60 0x7f6ca810a6d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c94009910 tx=0x7f6c94009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:45.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.158+0000 7f6caedde700 1 -- 172.21.15.88:0/3640682497 shutdown_connections 2024-02-19T19:05:45.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.158+0000 7f6caedde700 1 --2- 172.21.15.88:0/3640682497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca80ffc60 0x7f6ca810a6d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.158+0000 7f6caedde700 1 --2- 172.21.15.88:0/3640682497 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca80ff350 0x7f6ca80ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.158+0000 7f6caedde700 1 -- 172.21.15.88:0/3640682497 >> 172.21.15.88:0/3640682497 conn(0x7f6ca80748b0 msgr2=0x7f6ca8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:45.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.158+0000 7f6caedde700 1 -- 172.21.15.88:0/3640682497 shutdown_connections 2024-02-19T19:05:45.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.158+0000 7f6caedde700 1 -- 172.21.15.88:0/3640682497 wait complete. 2024-02-19T19:05:45.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.159+0000 7f6caedde700 1 Processor -- start 2024-02-19T19:05:45.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.159+0000 7f6caedde700 1 -- start start 2024-02-19T19:05:45.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.159+0000 7f6caedde700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca80ff350 0x7f6ca8100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:45.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.159+0000 7f6caedde700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca80ffc60 0x7f6ca8100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:45.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.159+0000 7f6caedde700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca8100fb0 con 0x7f6ca80ffc60 2024-02-19T19:05:45.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.159+0000 7f6caedde700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca81010f0 con 0x7f6ca80ff350 2024-02-19T19:05:45.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6ca7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca80ffc60 0x7f6ca8100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:45.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6ca7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca80ffc60 0x7f6ca8100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36894/0 (socket says 172.21.15.88:36894) 2024-02-19T19:05:45.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6ca7fff700 1 -- 172.21.15.88:0/141342950 learned_addr learned my addr 172.21.15.88:0/141342950 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:45.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6cacb7a700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca80ff350 0x7f6ca8100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:45.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6ca7fff700 1 -- 172.21.15.88:0/141342950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca80ff350 msgr2=0x7f6ca8100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:45.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6ca7fff700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca80ff350 0x7f6ca8100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6ca7fff700 1 -- 172.21.15.88:0/141342950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c940095f0 con 0x7f6ca80ffc60 2024-02-19T19:05:45.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6cacb7a700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca80ff350 0x7f6ca8100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:05:45.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6ca7fff700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca80ffc60 0x7f6ca8100a70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c94005f50 tx=0x7f6c9400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:45.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6ca5ffb700 1 -- 172.21.15.88:0/141342950 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c9401d070 con 0x7f6ca80ffc60 2024-02-19T19:05:45.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6ca5ffb700 1 -- 172.21.15.88:0/141342950 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c9400b610 con 0x7f6ca80ffc60 2024-02-19T19:05:45.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6caedde700 1 -- 172.21.15.88:0/141342950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca8101370 con 0x7f6ca80ffc60 2024-02-19T19:05:45.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.160+0000 7f6ca5ffb700 1 -- 172.21.15.88:0/141342950 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c94022470 con 0x7f6ca80ffc60 2024-02-19T19:05:45.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.161+0000 7f6caedde700 1 -- 172.21.15.88:0/141342950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca81a23b0 con 0x7f6ca80ffc60 2024-02-19T19:05:45.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.162+0000 7f6caedde700 1 -- 172.21.15.88:0/141342950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca8107dd0 con 0x7f6ca80ffc60 2024-02-19T19:05:45.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.162+0000 7f6ca5ffb700 1 -- 172.21.15.88:0/141342950 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f6c9400fca0 con 0x7f6ca80ffc60 2024-02-19T19:05:45.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.163+0000 7f6ca5ffb700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c9806d9e0 0x7f6c9806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:45.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.163+0000 7f6ca5ffb700 1 -- 172.21.15.88:0/141342950 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6c9408eb30 con 0x7f6ca80ffc60 2024-02-19T19:05:45.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.163+0000 7f6cacb7a700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c9806d9e0 0x7f6c9806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:45.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.164+0000 7f6cacb7a700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c9806d9e0 0x7f6c9806fe90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6ca806a640 tx=0x7f6c9c009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:45.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.169+0000 7f6ca5ffb700 1 -- 172.21.15.88:0/141342950 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c94058190 con 0x7f6ca80ffc60 2024-02-19T19:05:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.560+0000 7f6caedde700 1 -- 172.21.15.88:0/141342950 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ca804f2a0 con 0x7f6ca80ffc60 2024-02-19T19:05:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.561+0000 7f6ca5ffb700 1 -- 172.21.15.88:0/141342950 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f6c9405b850 con 0x7f6ca80ffc60 2024-02-19T19:05:45.581 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:05:45.581 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:05:45.582 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-02-19T19:05:45.583 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:05:45.583 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:05:45.583 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:05:45.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.564+0000 7f6caedde700 1 -- 172.21.15.88:0/141342950 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c9806d9e0 msgr2=0x7f6c9806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:45.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.564+0000 7f6caedde700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c9806d9e0 0x7f6c9806fe90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6ca806a640 tx=0x7f6c9c009260 comp rx=0 tx=0).stop 2024-02-19T19:05:45.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.564+0000 7f6caedde700 1 -- 172.21.15.88:0/141342950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca80ffc60 msgr2=0x7f6ca8100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:45.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.564+0000 7f6caedde700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca80ffc60 0x7f6ca8100a70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6c94005f50 tx=0x7f6c9400f740 comp rx=0 tx=0).stop 2024-02-19T19:05:45.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.564+0000 7f6caedde700 1 -- 172.21.15.88:0/141342950 shutdown_connections 2024-02-19T19:05:45.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.564+0000 7f6caedde700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c9806d9e0 0x7f6c9806fe90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.564+0000 7f6caedde700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ca80ff350 0x7f6ca8100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.564+0000 7f6caedde700 1 --2- 172.21.15.88:0/141342950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ca80ffc60 0x7f6ca8100a70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.564+0000 7f6caedde700 1 -- 172.21.15.88:0/141342950 >> 172.21.15.88:0/141342950 conn(0x7f6ca80748b0 msgr2=0x7f6ca8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:45.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.564+0000 7f6caedde700 1 -- 172.21.15.88:0/141342950 shutdown_connections 2024-02-19T19:05:45.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.564+0000 7f6caedde700 1 -- 172.21.15.88:0/141342950 wait complete. 2024-02-19T19:05:45.726 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:45 smithi088 ceph-mon[88643]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:45.726 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:45 smithi088 ceph-mon[88643]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:45.726 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:45.726 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.705+0000 7f43a3a58700 1 -- 172.21.15.88:0/1378700571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f439c100e60 msgr2=0x7f439c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.705+0000 7f43a3a58700 1 --2- 172.21.15.88:0/1378700571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f439c100e60 0x7f439c104e30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f438c009910 tx=0x7f438c009c20 comp rx=0 tx=0).stop 2024-02-19T19:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.705+0000 7f43a3a58700 1 -- 172.21.15.88:0/1378700571 shutdown_connections 2024-02-19T19:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.705+0000 7f43a3a58700 1 --2- 172.21.15.88:0/1378700571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f439c100e60 0x7f439c104e30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.705+0000 7f43a3a58700 1 --2- 172.21.15.88:0/1378700571 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f439c1004c0 0x7f439c100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.705+0000 7f43a3a58700 1 -- 172.21.15.88:0/1378700571 >> 172.21.15.88:0/1378700571 conn(0x7f439c0fbd80 msgr2=0x7f439c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.706+0000 7f43a3a58700 1 -- 172.21.15.88:0/1378700571 shutdown_connections 2024-02-19T19:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.706+0000 7f43a3a58700 1 -- 172.21.15.88:0/1378700571 wait complete. 2024-02-19T19:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.706+0000 7f43a3a58700 1 Processor -- start 2024-02-19T19:05:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.706+0000 7f43a3a58700 1 -- start start 2024-02-19T19:05:45.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.707+0000 7f43a3a58700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f439c1004c0 0x7f439c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:45.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.707+0000 7f43a3a58700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f439c100e60 0x7f439c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:45.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.707+0000 7f43a3a58700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c198cf0 con 0x7f439c1004c0 2024-02-19T19:05:45.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.707+0000 7f43a3a58700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c19ca80 con 0x7f439c100e60 2024-02-19T19:05:45.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.707+0000 7f43a17f4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f439c1004c0 0x7f439c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:45.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.708+0000 7f43a17f4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f439c1004c0 0x7f439c1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58104/0 (socket says 172.21.15.88:58104) 2024-02-19T19:05:45.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.708+0000 7f43a17f4700 1 -- 172.21.15.88:0/1920098036 learned_addr learned my addr 172.21.15.88:0/1920098036 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:45.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.708+0000 7f43a17f4700 1 -- 172.21.15.88:0/1920098036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f439c100e60 msgr2=0x7f439c198610 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-19T19:05:45.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.708+0000 7f43a17f4700 1 --2- 172.21.15.88:0/1920098036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f439c100e60 0x7f439c198610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:45.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.708+0000 7f43a17f4700 1 -- 172.21.15.88:0/1920098036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f438c0095f0 con 0x7f439c1004c0 2024-02-19T19:05:45.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.708+0000 7f43a17f4700 1 --2- 172.21.15.88:0/1920098036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f439c1004c0 0x7f439c1980d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f439800e920 tx=0x7f439800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:45.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.708+0000 7f43927fc700 1 -- 172.21.15.88:0/1920098036 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f439800cba0 con 0x7f439c1004c0 2024-02-19T19:05:45.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.709+0000 7f43927fc700 1 -- 172.21.15.88:0/1920098036 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f439800cd00 con 0x7f439c1004c0 2024-02-19T19:05:45.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.709+0000 7f43927fc700 1 -- 172.21.15.88:0/1920098036 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4398018750 con 0x7f439c1004c0 2024-02-19T19:05:45.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.709+0000 7f43a3a58700 1 -- 172.21.15.88:0/1920098036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439c19cd60 con 0x7f439c1004c0 2024-02-19T19:05:45.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.709+0000 7f43a3a58700 1 -- 172.21.15.88:0/1920098036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c19d2b0 con 0x7f439c1004c0 2024-02-19T19:05:45.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.710+0000 7f43a3a58700 1 -- 172.21.15.88:0/1920098036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f439c1045b0 con 0x7f439c1004c0 2024-02-19T19:05:45.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.711+0000 7f43927fc700 1 -- 172.21.15.88:0/1920098036 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f4398010430 con 0x7f439c1004c0 2024-02-19T19:05:45.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.712+0000 7f43927fc700 1 --2- 172.21.15.88:0/1920098036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806da20 0x7f438806fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:45.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.712+0000 7f43927fc700 1 -- 172.21.15.88:0/1920098036 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4398014070 con 0x7f439c1004c0 2024-02-19T19:05:45.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.712+0000 7f43a0ff3700 1 --2- 172.21.15.88:0/1920098036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806da20 0x7f438806fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:45.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.713+0000 7f43a0ff3700 1 --2- 172.21.15.88:0/1920098036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806da20 0x7f438806fed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f438c00b760 tx=0x7f438c005e00 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:45.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:45.719+0000 7f43927fc700 1 -- 172.21.15.88:0/1920098036 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4398057220 con 0x7f439c1004c0 2024-02-19T19:05:46.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:45 smithi132 ceph-mon[70676]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:46.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:45 smithi132 ceph-mon[70676]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:46.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:46.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:46.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.066+0000 7f43a3a58700 1 -- 172.21.15.88:0/1920098036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f439c19d520 con 0x7f439c1004c0 2024-02-19T19:05:46.087 INFO:teuthology.orchestra.run.smithi088.stdout:e20 2024-02-19T19:05:46.087 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:05:46.087 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 20 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:05:14.913778+0000 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:05:46.088 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:05:46.089 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:05:46.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.067+0000 7f43927fc700 1 -- 172.21.15.88:0/1920098036 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1892 (secure 0 0 0) 0x7f439805a8e0 con 0x7f439c1004c0 2024-02-19T19:05:46.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.071+0000 7f43a3a58700 1 -- 172.21.15.88:0/1920098036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806da20 msgr2=0x7f438806fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:46.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.071+0000 7f43a3a58700 1 --2- 172.21.15.88:0/1920098036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806da20 0x7f438806fed0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f438c00b760 tx=0x7f438c005e00 comp rx=0 tx=0).stop 2024-02-19T19:05:46.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.071+0000 7f43a3a58700 1 -- 172.21.15.88:0/1920098036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f439c1004c0 msgr2=0x7f439c1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:46.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.071+0000 7f43a3a58700 1 --2- 172.21.15.88:0/1920098036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f439c1004c0 0x7f439c1980d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f439800e920 tx=0x7f439800ece0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.071+0000 7f43a3a58700 1 -- 172.21.15.88:0/1920098036 shutdown_connections 2024-02-19T19:05:46.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.071+0000 7f43a3a58700 1 --2- 172.21.15.88:0/1920098036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806da20 0x7f438806fed0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.071+0000 7f43a3a58700 1 --2- 172.21.15.88:0/1920098036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f439c1004c0 0x7f439c1980d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.071+0000 7f43a3a58700 1 --2- 172.21.15.88:0/1920098036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f439c100e60 0x7f439c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.071+0000 7f43a3a58700 1 -- 172.21.15.88:0/1920098036 >> 172.21.15.88:0/1920098036 conn(0x7f439c0fbd80 msgr2=0x7f439c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:46.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.071+0000 7f43a3a58700 1 -- 172.21.15.88:0/1920098036 shutdown_connections 2024-02-19T19:05:46.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.072+0000 7f43a3a58700 1 -- 172.21.15.88:0/1920098036 wait complete. 2024-02-19T19:05:46.093 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-02-19T19:05:46.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.235+0000 7f9524aab700 1 -- 172.21.15.88:0/2071983030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9520100530 msgr2=0x7f9520100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:46.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.235+0000 7f9524aab700 1 --2- 172.21.15.88:0/2071983030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9520100530 0x7f9520100900 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9510009870 tx=0x7f9510009b80 comp rx=0 tx=0).stop 2024-02-19T19:05:46.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.236+0000 7f9524aab700 1 -- 172.21.15.88:0/2071983030 shutdown_connections 2024-02-19T19:05:46.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.236+0000 7f9524aab700 1 --2- 172.21.15.88:0/2071983030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9520100ed0 0x7f9520104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.236+0000 7f9524aab700 1 --2- 172.21.15.88:0/2071983030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9520100530 0x7f9520100900 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.236+0000 7f9524aab700 1 -- 172.21.15.88:0/2071983030 >> 172.21.15.88:0/2071983030 conn(0x7f95200fbe10 msgr2=0x7f95200fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:46.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.236+0000 7f9524aab700 1 -- 172.21.15.88:0/2071983030 shutdown_connections 2024-02-19T19:05:46.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.236+0000 7f9524aab700 1 -- 172.21.15.88:0/2071983030 wait complete. 2024-02-19T19:05:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.238+0000 7f9524aab700 1 Processor -- start 2024-02-19T19:05:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.238+0000 7f9524aab700 1 -- start start 2024-02-19T19:05:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.238+0000 7f9524aab700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9520100530 0x7f9520193af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.238+0000 7f9524aab700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9520100ed0 0x7f9520194030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.238+0000 7f9524aab700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9520196c70 con 0x7f9520100530 2024-02-19T19:05:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.238+0000 7f9524aab700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9520194570 con 0x7f9520100ed0 2024-02-19T19:05:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.238+0000 7f951dd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9520100ed0 0x7f9520194030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.239+0000 7f951dd9b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9520100ed0 0x7f9520194030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:46134/0 (socket says 172.21.15.88:46134) 2024-02-19T19:05:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.239+0000 7f951dd9b700 1 -- 172.21.15.88:0/3552222830 learned_addr learned my addr 172.21.15.88:0/3552222830 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.239+0000 7f951e59c700 1 --2- 172.21.15.88:0/3552222830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9520100530 0x7f9520193af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:46.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.239+0000 7f951dd9b700 1 -- 172.21.15.88:0/3552222830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9520100530 msgr2=0x7f9520193af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:46.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.239+0000 7f951dd9b700 1 --2- 172.21.15.88:0/3552222830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9520100530 0x7f9520193af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.239+0000 7f951dd9b700 1 -- 172.21.15.88:0/3552222830 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9510009520 con 0x7f9520100ed0 2024-02-19T19:05:46.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.242+0000 7f951dd9b700 1 --2- 172.21.15.88:0/3552222830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9520100ed0 0x7f9520194030 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f951400e970 tx=0x7f951400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:46.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.242+0000 7f950f7fe700 1 -- 172.21.15.88:0/3552222830 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f951400c9d0 con 0x7f9520100ed0 2024-02-19T19:05:46.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.242+0000 7f950f7fe700 1 -- 172.21.15.88:0/3552222830 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9514010430 con 0x7f9520100ed0 2024-02-19T19:05:46.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.242+0000 7f950f7fe700 1 -- 172.21.15.88:0/3552222830 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9514018650 con 0x7f9520100ed0 2024-02-19T19:05:46.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.242+0000 7f9524aab700 1 -- 172.21.15.88:0/3552222830 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9520194850 con 0x7f9520100ed0 2024-02-19T19:05:46.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.242+0000 7f9524aab700 1 -- 172.21.15.88:0/3552222830 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952006a940 con 0x7f9520100ed0 2024-02-19T19:05:46.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.244+0000 7f9524aab700 1 -- 172.21.15.88:0/3552222830 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9520104620 con 0x7f9520100ed0 2024-02-19T19:05:46.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.245+0000 7f950f7fe700 1 -- 172.21.15.88:0/3552222830 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f95140105a0 con 0x7f9520100ed0 2024-02-19T19:05:46.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.246+0000 7f950f7fe700 1 --2- 172.21.15.88:0/3552222830 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da10 0x7f950806fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:46.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.246+0000 7f950f7fe700 1 -- 172.21.15.88:0/3552222830 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9514014070 con 0x7f9520100ed0 2024-02-19T19:05:46.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.246+0000 7f951e59c700 1 --2- 172.21.15.88:0/3552222830 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da10 0x7f950806fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:46.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.247+0000 7f951e59c700 1 --2- 172.21.15.88:0/3552222830 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da10 0x7f950806fec0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f951000b760 tx=0x7f9510019040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:46.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.255+0000 7f950f7fe700 1 -- 172.21.15.88:0/3552222830 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9514052d20 con 0x7f9520100ed0 2024-02-19T19:05:46.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.668+0000 7f9524aab700 1 -- 172.21.15.88:0/3552222830 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9520195020 con 0x7f950806da10 2024-02-19T19:05:46.689 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:46 smithi088 ceph-mon[88643]: Reconfiguring mgr.smithi088.gyviqq (monmap changed)... 2024-02-19T19:05:46.689 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.gyviqq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:05:46.689 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T19:05:46.689 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:46.689 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:46 smithi088 ceph-mon[88643]: Reconfiguring daemon mgr.smithi088.gyviqq on smithi088 2024-02-19T19:05:46.689 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:46 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/141342950' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:05:46.689 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:46 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1920098036' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:05:46.689 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:46 smithi088 ceph-mon[88643]: pgmap v27: 65 pgs: 65 active+clean; 193 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 4.2 MiB/s wr, 579 op/s 2024-02-19T19:05:46.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.670+0000 7f950f7fe700 1 -- 172.21.15.88:0/3552222830 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f9520195020 con 0x7f950806da10 2024-02-19T19:05:46.690 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:05:46.690 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:05:46.690 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:05:46.690 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:05:46.691 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:05:46.691 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:05:46.691 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:05:46.691 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:05:46.691 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "4/23 daemons upgraded", 2024-02-19T19:05:46.691 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mon daemons", 2024-02-19T19:05:46.691 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:05:46.691 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:05:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.673+0000 7f9524aab700 1 -- 172.21.15.88:0/3552222830 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da10 msgr2=0x7f950806fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:46.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.673+0000 7f9524aab700 1 --2- 172.21.15.88:0/3552222830 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da10 0x7f950806fec0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f951000b760 tx=0x7f9510019040 comp rx=0 tx=0).stop 2024-02-19T19:05:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.673+0000 7f9524aab700 1 -- 172.21.15.88:0/3552222830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9520100ed0 msgr2=0x7f9520194030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.673+0000 7f9524aab700 1 --2- 172.21.15.88:0/3552222830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9520100ed0 0x7f9520194030 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f951400e970 tx=0x7f951400ed30 comp rx=0 tx=0).stop 2024-02-19T19:05:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.673+0000 7f9524aab700 1 -- 172.21.15.88:0/3552222830 shutdown_connections 2024-02-19T19:05:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.673+0000 7f9524aab700 1 --2- 172.21.15.88:0/3552222830 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da10 0x7f950806fec0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.673+0000 7f9524aab700 1 --2- 172.21.15.88:0/3552222830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9520100530 0x7f9520193af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.673+0000 7f9524aab700 1 --2- 172.21.15.88:0/3552222830 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9520100ed0 0x7f9520194030 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.673+0000 7f9524aab700 1 -- 172.21.15.88:0/3552222830 >> 172.21.15.88:0/3552222830 conn(0x7f95200fbe10 msgr2=0x7f95200fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.673+0000 7f9524aab700 1 -- 172.21.15.88:0/3552222830 shutdown_connections 2024-02-19T19:05:46.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.673+0000 7f9524aab700 1 -- 172.21.15.88:0/3552222830 wait complete. 2024-02-19T19:05:46.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:46 smithi132 ceph-mon[70676]: Reconfiguring mgr.smithi088.gyviqq (monmap changed)... 2024-02-19T19:05:46.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi088.gyviqq", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:05:46.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T19:05:46.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:46 smithi132 ceph-mon[70676]: Reconfiguring daemon mgr.smithi088.gyviqq on smithi088 2024-02-19T19:05:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:46 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/141342950' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:05:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:46 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1920098036' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:05:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:46 smithi132 ceph-mon[70676]: pgmap v27: 65 pgs: 65 active+clean; 193 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 4.2 MiB/s wr, 579 op/s 2024-02-19T19:05:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.811+0000 7feda3e13700 1 -- 172.21.15.88:0/361196712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed9c100d70 msgr2=0x7fed9c1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.811+0000 7feda3e13700 1 --2- 172.21.15.88:0/361196712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed9c100d70 0x7fed9c1094d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009940 tx=0x7fed8c009c50 comp rx=0 tx=0).stop 2024-02-19T19:05:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.812+0000 7feda3e13700 1 -- 172.21.15.88:0/361196712 shutdown_connections 2024-02-19T19:05:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.812+0000 7feda3e13700 1 --2- 172.21.15.88:0/361196712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed9c100d70 0x7fed9c1094d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.812+0000 7feda3e13700 1 --2- 172.21.15.88:0/361196712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed9c100460 0x7fed9c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.812+0000 7feda3e13700 1 -- 172.21.15.88:0/361196712 >> 172.21.15.88:0/361196712 conn(0x7fed9c0748b0 msgr2=0x7fed9c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:46.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.812+0000 7feda3e13700 1 -- 172.21.15.88:0/361196712 shutdown_connections 2024-02-19T19:05:46.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.812+0000 7feda3e13700 1 -- 172.21.15.88:0/361196712 wait complete. 2024-02-19T19:05:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.812+0000 7feda3e13700 1 Processor -- start 2024-02-19T19:05:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda3e13700 1 -- start start 2024-02-19T19:05:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda3e13700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed9c100460 0x7fed9c101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda3e13700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed9c100d70 0x7fed9c102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda3e13700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c102850 con 0x7fed9c100460 2024-02-19T19:05:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda3e13700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c102990 con 0x7fed9c100d70 2024-02-19T19:05:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda1baf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed9c100460 0x7fed9c101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda1baf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed9c100460 0x7fed9c101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58132/0 (socket says 172.21.15.88:58132) 2024-02-19T19:05:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda1baf700 1 -- 172.21.15.88:0/1411561122 learned_addr learned my addr 172.21.15.88:0/1411561122 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:05:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda13ae700 1 --2- 172.21.15.88:0/1411561122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed9c100d70 0x7fed9c102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda1baf700 1 -- 172.21.15.88:0/1411561122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed9c100d70 msgr2=0x7fed9c102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda1baf700 1 --2- 172.21.15.88:0/1411561122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed9c100d70 0x7fed9c102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda1baf700 1 -- 172.21.15.88:0/1411561122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8c0095f0 con 0x7fed9c100460 2024-02-19T19:05:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7feda1baf700 1 --2- 172.21.15.88:0/1411561122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed9c100460 0x7fed9c101dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed9800e920 tx=0x7fed9800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.813+0000 7fed92ffd700 1 -- 172.21.15.88:0/1411561122 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed9800c9d0 con 0x7fed9c100460 2024-02-19T19:05:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.814+0000 7feda3e13700 1 -- 172.21.15.88:0/1411561122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9c1a2080 con 0x7fed9c100460 2024-02-19T19:05:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.814+0000 7fed92ffd700 1 -- 172.21.15.88:0/1411561122 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed98010430 con 0x7fed9c100460 2024-02-19T19:05:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.814+0000 7fed92ffd700 1 -- 172.21.15.88:0/1411561122 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed98018630 con 0x7fed9c100460 2024-02-19T19:05:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.814+0000 7feda3e13700 1 -- 172.21.15.88:0/1411561122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c1a2430 con 0x7fed9c100460 2024-02-19T19:05:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.815+0000 7feda3e13700 1 -- 172.21.15.88:0/1411561122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9c04f2a0 con 0x7fed9c100460 2024-02-19T19:05:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.815+0000 7fed92ffd700 1 -- 172.21.15.88:0/1411561122 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fed980105a0 con 0x7fed9c100460 2024-02-19T19:05:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.816+0000 7fed92ffd700 1 --2- 172.21.15.88:0/1411561122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed8806d9e0 0x7fed8806fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:05:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.816+0000 7fed92ffd700 1 -- 172.21.15.88:0/1411561122 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fed98014070 con 0x7fed9c100460 2024-02-19T19:05:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.816+0000 7feda13ae700 1 --2- 172.21.15.88:0/1411561122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed8806d9e0 0x7fed8806fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:05:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.816+0000 7feda13ae700 1 --2- 172.21.15.88:0/1411561122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed8806d9e0 0x7fed8806fe90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fed8c006010 tx=0x7fed8c005ad0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:05:46.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:46.821+0000 7fed92ffd700 1 -- 172.21.15.88:0/1411561122 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed98052d20 con 0x7fed9c100460 2024-02-19T19:05:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.255+0000 7feda3e13700 1 -- 172.21.15.88:0/1411561122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fed9c102c40 con 0x7fed9c100460 2024-02-19T19:05:47.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.255+0000 7fed92ffd700 1 -- 172.21.15.88:0/1411561122 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fed9c102c40 con 0x7fed9c100460 2024-02-19T19:05:47.276 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_OK 2024-02-19T19:05:47.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.258+0000 7feda3e13700 1 -- 172.21.15.88:0/1411561122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed8806d9e0 msgr2=0x7fed8806fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:47.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.258+0000 7feda3e13700 1 --2- 172.21.15.88:0/1411561122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed8806d9e0 0x7fed8806fe90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fed8c006010 tx=0x7fed8c005ad0 comp rx=0 tx=0).stop 2024-02-19T19:05:47.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.258+0000 7feda3e13700 1 -- 172.21.15.88:0/1411561122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed9c100460 msgr2=0x7fed9c101dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:05:47.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.258+0000 7feda3e13700 1 --2- 172.21.15.88:0/1411561122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed9c100460 0x7fed9c101dd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fed9800e920 tx=0x7fed9800ec30 comp rx=0 tx=0).stop 2024-02-19T19:05:47.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.259+0000 7feda3e13700 1 -- 172.21.15.88:0/1411561122 shutdown_connections 2024-02-19T19:05:47.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.259+0000 7feda3e13700 1 --2- 172.21.15.88:0/1411561122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed8806d9e0 0x7fed8806fe90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:47.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.259+0000 7feda3e13700 1 --2- 172.21.15.88:0/1411561122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed9c100460 0x7fed9c101dd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:47.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.259+0000 7feda3e13700 1 --2- 172.21.15.88:0/1411561122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed9c100d70 0x7fed9c102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:05:47.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.259+0000 7feda3e13700 1 -- 172.21.15.88:0/1411561122 >> 172.21.15.88:0/1411561122 conn(0x7fed9c0748b0 msgr2=0x7fed9c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:05:47.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.259+0000 7feda3e13700 1 -- 172.21.15.88:0/1411561122 shutdown_connections 2024-02-19T19:05:47.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:05:47.259+0000 7feda3e13700 1 -- 172.21.15.88:0/1411561122 wait complete. 2024-02-19T19:05:47.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:47 smithi088 ceph-mon[88643]: from='client.44117 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:47.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:47 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1411561122' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:05:47.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:47.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:47.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:47 smithi088 ceph-mon[88643]: Reconfiguring ceph-exporter.smithi088 (monmap changed)... 2024-02-19T19:05:47.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:05:47.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:05:47.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-19T19:05:47.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi088"}]: dispatch 2024-02-19T19:05:47.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:48.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:47 smithi132 ceph-mon[70676]: from='client.44117 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:05:48.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:47 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1411561122' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:05:48.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:48.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:48.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:47 smithi132 ceph-mon[70676]: Reconfiguring ceph-exporter.smithi088 (monmap changed)... 2024-02-19T19:05:48.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:05:48.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:05:48.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-19T19:05:48.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi088"}]: dispatch 2024-02-19T19:05:48.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:48.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:48 smithi088 ceph-mon[88643]: Unable to update caps for client.ceph-exporter.smithi088 2024-02-19T19:05:48.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:48 smithi088 ceph-mon[88643]: Reconfiguring daemon ceph-exporter.smithi088 on smithi088 2024-02-19T19:05:48.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:48 smithi088 ceph-mon[88643]: pgmap v28: 65 pgs: 65 active+clean; 193 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 2.9 MiB/s wr, 380 op/s 2024-02-19T19:05:49.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:48 smithi132 ceph-mon[70676]: Unable to update caps for client.ceph-exporter.smithi088 2024-02-19T19:05:49.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:48 smithi132 ceph-mon[70676]: Reconfiguring daemon ceph-exporter.smithi088 on smithi088 2024-02-19T19:05:49.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:48 smithi132 ceph-mon[70676]: pgmap v28: 65 pgs: 65 active+clean; 193 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 2.9 MiB/s wr, 380 op/s 2024-02-19T19:05:50.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:50.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:50.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:50 smithi132 ceph-mon[70676]: Reconfiguring crash.smithi088 (monmap changed)... 2024-02-19T19:05:50.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T19:05:50.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:50.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:50 smithi132 ceph-mon[70676]: Reconfiguring daemon crash.smithi088 on smithi088 2024-02-19T19:05:50.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:50 smithi132 ceph-mon[70676]: pgmap v29: 65 pgs: 65 active+clean; 200 MiB data, 1000 MiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 4.0 MiB/s wr, 597 op/s 2024-02-19T19:05:50.794 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:50.794 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:50.794 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:50 smithi088 ceph-mon[88643]: Reconfiguring crash.smithi088 (monmap changed)... 2024-02-19T19:05:50.794 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T19:05:50.795 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:50.795 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:50 smithi088 ceph-mon[88643]: Reconfiguring daemon crash.smithi088 on smithi088 2024-02-19T19:05:50.795 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:50 smithi088 ceph-mon[88643]: pgmap v29: 65 pgs: 65 active+clean; 200 MiB data, 1000 MiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 4.0 MiB/s wr, 597 op/s 2024-02-19T19:05:52.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:52.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:52.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:52 smithi088 ceph-mon[88643]: Reconfiguring osd.0 (monmap changed)... 2024-02-19T19:05:52.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-19T19:05:52.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:52.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:52 smithi088 ceph-mon[88643]: Reconfiguring daemon osd.0 on smithi088 2024-02-19T19:05:52.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:52 smithi088 ceph-mon[88643]: pgmap v30: 65 pgs: 65 active+clean; 200 MiB data, 1000 MiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 2.2 MiB/s wr, 398 op/s 2024-02-19T19:05:53.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:53.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:53.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:52 smithi132 ceph-mon[70676]: Reconfiguring osd.0 (monmap changed)... 2024-02-19T19:05:53.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-19T19:05:53.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:53.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:52 smithi132 ceph-mon[70676]: Reconfiguring daemon osd.0 on smithi088 2024-02-19T19:05:53.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:52 smithi132 ceph-mon[70676]: pgmap v30: 65 pgs: 65 active+clean; 200 MiB data, 1000 MiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 2.2 MiB/s wr, 398 op/s 2024-02-19T19:05:54.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:54.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:54.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:54 smithi088 ceph-mon[88643]: Reconfiguring osd.1 (monmap changed)... 2024-02-19T19:05:54.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-19T19:05:54.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:54.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:54 smithi088 ceph-mon[88643]: Reconfiguring daemon osd.1 on smithi088 2024-02-19T19:05:54.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:54 smithi088 ceph-mon[88643]: pgmap v31: 65 pgs: 65 active+clean; 200 MiB data, 1000 MiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 2.2 MiB/s wr, 398 op/s 2024-02-19T19:05:55.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:55.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:55.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:54 smithi132 ceph-mon[70676]: Reconfiguring osd.1 (monmap changed)... 2024-02-19T19:05:55.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-19T19:05:55.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:55.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:54 smithi132 ceph-mon[70676]: Reconfiguring daemon osd.1 on smithi088 2024-02-19T19:05:55.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:54 smithi132 ceph-mon[70676]: pgmap v31: 65 pgs: 65 active+clean; 200 MiB data, 1000 MiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 2.2 MiB/s wr, 398 op/s 2024-02-19T19:05:56.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:56 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:56.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:56 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:56.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:56 smithi132 ceph-mon[70676]: Reconfiguring osd.2 (monmap changed)... 2024-02-19T19:05:56.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:56 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-19T19:05:56.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:56 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:56.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:56 smithi132 ceph-mon[70676]: Reconfiguring daemon osd.2 on smithi088 2024-02-19T19:05:56.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:56 smithi132 ceph-mon[70676]: pgmap v32: 65 pgs: 65 active+clean; 211 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 3.5 MiB/s wr, 575 op/s 2024-02-19T19:05:56.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:56 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:56.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:56 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:56.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:56 smithi088 ceph-mon[88643]: Reconfiguring osd.2 (monmap changed)... 2024-02-19T19:05:56.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:56 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-19T19:05:56.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:56 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:56.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:56 smithi088 ceph-mon[88643]: Reconfiguring daemon osd.2 on smithi088 2024-02-19T19:05:56.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:56 smithi088 ceph-mon[88643]: pgmap v32: 65 pgs: 65 active+clean; 211 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 27 KiB/s rd, 3.5 MiB/s wr, 575 op/s 2024-02-19T19:05:59.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:59 smithi088 ceph-mon[88643]: Reconfiguring mds.cephfs.smithi088.ajgvpz (monmap changed)... 2024-02-19T19:05:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.ajgvpz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:05:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:59 smithi088 ceph-mon[88643]: Reconfiguring daemon mds.cephfs.smithi088.ajgvpz on smithi088 2024-02-19T19:05:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:59 smithi088 ceph-mon[88643]: pgmap v33: 65 pgs: 65 active+clean; 211 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 2.3 MiB/s wr, 394 op/s 2024-02-19T19:05:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:05:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:05:59.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:05:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:59 smithi132 ceph-mon[70676]: Reconfiguring mds.cephfs.smithi088.ajgvpz (monmap changed)... 2024-02-19T19:05:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.ajgvpz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:05:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:05:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:59 smithi132 ceph-mon[70676]: Reconfiguring daemon mds.cephfs.smithi088.ajgvpz on smithi088 2024-02-19T19:05:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:59 smithi132 ceph-mon[70676]: pgmap v33: 65 pgs: 65 active+clean; 211 MiB data, 1.0 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 2.3 MiB/s wr, 394 op/s 2024-02-19T19:05:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:05:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:06:01.047 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:01.048 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:01.048 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:00 smithi088 ceph-mon[88643]: Reconfiguring mds.cephfs.smithi088.psjiah (monmap changed)... 2024-02-19T19:06:01.048 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.psjiah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:06:01.048 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:01.048 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:00 smithi088 ceph-mon[88643]: Reconfiguring daemon mds.cephfs.smithi088.psjiah on smithi088 2024-02-19T19:06:01.048 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:00 smithi088 ceph-mon[88643]: pgmap v34: 65 pgs: 65 active+clean; 227 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 3.8 MiB/s wr, 568 op/s 2024-02-19T19:06:01.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:01.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:01.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:00 smithi132 ceph-mon[70676]: Reconfiguring mds.cephfs.smithi088.psjiah (monmap changed)... 2024-02-19T19:06:01.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.psjiah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:06:01.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:01.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:00 smithi132 ceph-mon[70676]: Reconfiguring daemon mds.cephfs.smithi088.psjiah on smithi088 2024-02-19T19:06:01.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:00 smithi132 ceph-mon[70676]: pgmap v34: 65 pgs: 65 active+clean; 227 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 3.8 MiB/s wr, 568 op/s 2024-02-19T19:06:03.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:03 smithi132 ceph-mon[70676]: pgmap v35: 65 pgs: 65 active+clean; 227 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-02-19T19:06:03.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:03 smithi088 ceph-mon[88643]: pgmap v35: 65 pgs: 65 active+clean; 227 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-02-19T19:06:05.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:05 smithi132 ceph-mon[70676]: pgmap v36: 65 pgs: 65 active+clean; 227 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-02-19T19:06:05.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:05 smithi088 ceph-mon[88643]: pgmap v36: 65 pgs: 65 active+clean; 227 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 2.8 MiB/s wr, 351 op/s 2024-02-19T19:06:07.317 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:07 smithi132 ceph-mon[70676]: pgmap v37: 65 pgs: 65 active+clean; 240 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 4.2 MiB/s wr, 547 op/s 2024-02-19T19:06:07.317 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:07.318 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:07.318 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:07 smithi132 ceph-mon[70676]: Reconfiguring ceph-exporter.smithi132 (monmap changed)... 2024-02-19T19:06:07.318 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:06:07.318 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:06:07.318 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-19T19:06:07.318 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:07 smithi132 ceph-mon[70676]: Unable to update caps for client.ceph-exporter.smithi132 2024-02-19T19:06:07.318 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi132"}]: dispatch 2024-02-19T19:06:07.318 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:07.318 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:07 smithi132 ceph-mon[70676]: Reconfiguring daemon ceph-exporter.smithi132 on smithi132 2024-02-19T19:06:07.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:07 smithi088 ceph-mon[88643]: pgmap v37: 65 pgs: 65 active+clean; 240 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 4.2 MiB/s wr, 547 op/s 2024-02-19T19:06:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:07 smithi088 ceph-mon[88643]: Reconfiguring ceph-exporter.smithi132 (monmap changed)... 2024-02-19T19:06:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:06:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:06:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-02-19T19:06:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:07 smithi088 ceph-mon[88643]: Unable to update caps for client.ceph-exporter.smithi132 2024-02-19T19:06:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi132"}]: dispatch 2024-02-19T19:06:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:07 smithi088 ceph-mon[88643]: Reconfiguring daemon ceph-exporter.smithi132 on smithi132 2024-02-19T19:06:09.036 INFO:tasks.workunit.client.0.smithi088.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-02-19T19:06:09.049 INFO:tasks.workunit.client.0.smithi088.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-19T19:06:09.049 INFO:tasks.workunit.client.0.smithi088.stderr:+ make 2024-02-19T19:06:09.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:09 smithi132 ceph-mon[70676]: pgmap v38: 65 pgs: 65 active+clean; 240 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 3.0 MiB/s wr, 369 op/s 2024-02-19T19:06:09.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:09.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:09.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:09 smithi132 ceph-mon[70676]: Reconfiguring crash.smithi132 (monmap changed)... 2024-02-19T19:06:09.277 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi132", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T19:06:09.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:09.278 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:09 smithi132 ceph-mon[70676]: Reconfiguring daemon crash.smithi132 on smithi132 2024-02-19T19:06:09.345 INFO:tasks.workunit.client.0.smithi088.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-02-19T19:06:09.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:09 smithi088 ceph-mon[88643]: pgmap v38: 65 pgs: 65 active+clean; 240 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 3.0 MiB/s wr, 369 op/s 2024-02-19T19:06:09.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:09.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:09.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:09 smithi088 ceph-mon[88643]: Reconfiguring crash.smithi132 (monmap changed)... 2024-02-19T19:06:09.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi132", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T19:06:09.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:09.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:09 smithi088 ceph-mon[88643]: Reconfiguring daemon crash.smithi132 on smithi132 2024-02-19T19:06:09.792 INFO:tasks.workunit.client.0.smithi088.stderr:++ readlink -f fsstress 2024-02-19T19:06:09.793 INFO:tasks.workunit.client.0.smithi088.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-02-19T19:06:09.793 INFO:tasks.workunit.client.0.smithi088.stderr:+ popd 2024-02-19T19:06:09.794 INFO:tasks.workunit.client.0.smithi088.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-02-19T19:06:09.794 INFO:tasks.workunit.client.0.smithi088.stderr:+ popd 2024-02-19T19:06:09.795 INFO:tasks.workunit.client.0.smithi088.stdout:~/cephtest/mnt.0/client.0/tmp 2024-02-19T19:06:09.795 INFO:tasks.workunit.client.0.smithi088.stderr:++ mktemp -d -p . 2024-02-19T19:06:09.797 INFO:tasks.workunit.client.0.smithi088.stderr:+ T=./tmp.eME568FDig 2024-02-19T19:06:09.798 INFO:tasks.workunit.client.0.smithi088.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.eME568FDig -l 1 -n 1000 -p 10 -v 2024-02-19T19:06:09.801 INFO:tasks.workunit.client.0.smithi088.stdout:seed = 1708673147 2024-02-19T19:06:09.827 INFO:tasks.workunit.client.0.smithi088.stdout:0/0: creat f0 x:0 0 0 2024-02-19T19:06:09.828 INFO:tasks.workunit.client.0.smithi088.stdout:2/0: dread - no filename 2024-02-19T19:06:09.828 INFO:tasks.workunit.client.0.smithi088.stdout:2/1: fsync - no filename 2024-02-19T19:06:09.828 INFO:tasks.workunit.client.0.smithi088.stdout:2/2: readlink - no filename 2024-02-19T19:06:09.828 INFO:tasks.workunit.client.0.smithi088.stdout:2/3: link - no file 2024-02-19T19:06:09.829 INFO:tasks.workunit.client.0.smithi088.stdout:0/1: creat f1 x:0 0 0 2024-02-19T19:06:09.829 INFO:tasks.workunit.client.0.smithi088.stdout:1/0: getdents . 0 2024-02-19T19:06:09.829 INFO:tasks.workunit.client.0.smithi088.stdout:0/2: chown f1 13769943 1 2024-02-19T19:06:09.832 INFO:tasks.workunit.client.0.smithi088.stdout:0/3: creat f2 x:0 0 0 2024-02-19T19:06:09.832 INFO:tasks.workunit.client.0.smithi088.stdout:0/4: chown f1 271410732 1 2024-02-19T19:06:09.832 INFO:tasks.workunit.client.0.smithi088.stdout:2/4: symlink l0 0 2024-02-19T19:06:09.833 INFO:tasks.workunit.client.0.smithi088.stdout:2/5: dwrite - no filename 2024-02-19T19:06:09.833 INFO:tasks.workunit.client.0.smithi088.stdout:2/6: fdatasync - no filename 2024-02-19T19:06:09.833 INFO:tasks.workunit.client.0.smithi088.stdout:2/7: stat l0 0 2024-02-19T19:06:09.834 INFO:tasks.workunit.client.0.smithi088.stdout:1/1: creat f0 x:0 0 0 2024-02-19T19:06:09.834 INFO:tasks.workunit.client.0.smithi088.stdout:1/2: readlink - no filename 2024-02-19T19:06:09.835 INFO:tasks.workunit.client.0.smithi088.stdout:1/3: chown f0 7 1 2024-02-19T19:06:09.835 INFO:tasks.workunit.client.0.smithi088.stdout:0/5: creat f3 x:0 0 0 2024-02-19T19:06:09.838 INFO:tasks.workunit.client.0.smithi088.stdout:3/0: symlink l0 0 2024-02-19T19:06:09.839 INFO:tasks.workunit.client.0.smithi088.stdout:1/4: symlink l1 0 2024-02-19T19:06:09.841 INFO:tasks.workunit.client.0.smithi088.stdout:4/0: creat f0 x:0 0 0 2024-02-19T19:06:09.843 INFO:tasks.workunit.client.0.smithi088.stdout:3/1: creat f1 x:0 0 0 2024-02-19T19:06:09.844 INFO:tasks.workunit.client.0.smithi088.stdout:2/8: link l0 l1 0 2024-02-19T19:06:09.846 INFO:tasks.workunit.client.0.smithi088.stdout:4/1: mknod c1 0 2024-02-19T19:06:09.850 INFO:tasks.workunit.client.0.smithi088.stdout:0/6: dwrite f3 [0,4194304] 0 2024-02-19T19:06:09.850 INFO:tasks.workunit.client.0.smithi088.stdout:5/0: creat f0 x:0 0 0 2024-02-19T19:06:09.850 INFO:tasks.workunit.client.0.smithi088.stdout:6/0: truncate - no filename 2024-02-19T19:06:09.850 INFO:tasks.workunit.client.0.smithi088.stdout:6/1: read - no filename 2024-02-19T19:06:09.850 INFO:tasks.workunit.client.0.smithi088.stdout:6/2: fdatasync - no filename 2024-02-19T19:06:09.850 INFO:tasks.workunit.client.0.smithi088.stdout:6/3: dwrite - no filename 2024-02-19T19:06:09.850 INFO:tasks.workunit.client.0.smithi088.stdout:6/4: dwrite - no filename 2024-02-19T19:06:09.850 INFO:tasks.workunit.client.0.smithi088.stdout:6/5: rename - no filename 2024-02-19T19:06:09.851 INFO:tasks.workunit.client.0.smithi088.stdout:6/6: dwrite - no filename 2024-02-19T19:06:09.851 INFO:tasks.workunit.client.0.smithi088.stdout:6/7: chown . 2265 1 2024-02-19T19:06:09.851 INFO:tasks.workunit.client.0.smithi088.stdout:6/8: rmdir - no directory 2024-02-19T19:06:09.851 INFO:tasks.workunit.client.0.smithi088.stdout:6/9: write - no filename 2024-02-19T19:06:09.853 INFO:tasks.workunit.client.0.smithi088.stdout:0/7: chown f1 6912538 1 2024-02-19T19:06:09.853 INFO:tasks.workunit.client.0.smithi088.stdout:0/8: rmdir - no directory 2024-02-19T19:06:09.857 INFO:tasks.workunit.client.0.smithi088.stdout:2/9: rename l1 to l2 0 2024-02-19T19:06:09.857 INFO:tasks.workunit.client.0.smithi088.stdout:2/10: dwrite - no filename 2024-02-19T19:06:09.857 INFO:tasks.workunit.client.0.smithi088.stdout:2/11: write - no filename 2024-02-19T19:06:09.857 INFO:tasks.workunit.client.0.smithi088.stdout:5/1: write f0 [338581,18161] 0 2024-02-19T19:06:09.858 INFO:tasks.workunit.client.0.smithi088.stdout:0/9: write f0 [197201,1308] 0 2024-02-19T19:06:09.860 INFO:tasks.workunit.client.0.smithi088.stdout:7/0: getdents . 0 2024-02-19T19:06:09.863 INFO:tasks.workunit.client.0.smithi088.stdout:1/5: dwrite f0 [0,4194304] 0 2024-02-19T19:06:09.873 INFO:tasks.workunit.client.0.smithi088.stdout:3/2: dwrite f1 [0,4194304] 0 2024-02-19T19:06:09.874 INFO:tasks.workunit.client.0.smithi088.stdout:4/2: dwrite f0 [0,4194304] 0 2024-02-19T19:06:09.878 INFO:tasks.workunit.client.0.smithi088.stdout:0/10: creat f4 x:0 0 0 2024-02-19T19:06:09.880 INFO:tasks.workunit.client.0.smithi088.stdout:6/10: getdents . 0 2024-02-19T19:06:09.880 INFO:tasks.workunit.client.0.smithi088.stdout:6/11: write - no filename 2024-02-19T19:06:09.880 INFO:tasks.workunit.client.0.smithi088.stdout:6/12: rename - no filename 2024-02-19T19:06:09.880 INFO:tasks.workunit.client.0.smithi088.stdout:6/13: dread - no filename 2024-02-19T19:06:09.880 INFO:tasks.workunit.client.0.smithi088.stdout:5/2: dwrite f0 [0,4194304] 0 2024-02-19T19:06:09.881 INFO:tasks.workunit.client.0.smithi088.stdout:1/6: dread f0 [0,4194304] 0 2024-02-19T19:06:09.884 INFO:tasks.workunit.client.0.smithi088.stdout:3/3: rename f1 to f2 0 2024-02-19T19:06:09.885 INFO:tasks.workunit.client.0.smithi088.stdout:4/3: mkdir d2 0 2024-02-19T19:06:09.886 INFO:tasks.workunit.client.0.smithi088.stdout:8/0: creat f0 x:0 0 0 2024-02-19T19:06:09.889 INFO:tasks.workunit.client.0.smithi088.stdout:7/1: creat f0 x:0 0 0 2024-02-19T19:06:09.889 INFO:tasks.workunit.client.0.smithi088.stdout:2/12: chown l0 11 1 2024-02-19T19:06:09.889 INFO:tasks.workunit.client.0.smithi088.stdout:2/13: fsync - no filename 2024-02-19T19:06:09.889 INFO:tasks.workunit.client.0.smithi088.stdout:2/14: fsync - no filename 2024-02-19T19:06:09.889 INFO:tasks.workunit.client.0.smithi088.stdout:0/11: rename f1 to f5 0 2024-02-19T19:06:09.889 INFO:tasks.workunit.client.0.smithi088.stdout:0/12: rmdir - no directory 2024-02-19T19:06:09.892 INFO:tasks.workunit.client.0.smithi088.stdout:7/2: write f0 [370631,49108] 0 2024-02-19T19:06:09.893 INFO:tasks.workunit.client.0.smithi088.stdout:9/0: mknod c0 0 2024-02-19T19:06:09.893 INFO:tasks.workunit.client.0.smithi088.stdout:5/3: dwrite f0 [0,4194304] 0 2024-02-19T19:06:09.900 INFO:tasks.workunit.client.0.smithi088.stdout:8/1: symlink l1 0 2024-02-19T19:06:09.900 INFO:tasks.workunit.client.0.smithi088.stdout:8/2: readlink l1 0 2024-02-19T19:06:09.901 INFO:tasks.workunit.client.0.smithi088.stdout:8/3: readlink l1 0 2024-02-19T19:06:09.902 INFO:tasks.workunit.client.0.smithi088.stdout:8/4: stat l1 0 2024-02-19T19:06:09.904 INFO:tasks.workunit.client.0.smithi088.stdout:6/14: mknod c0 0 2024-02-19T19:06:09.904 INFO:tasks.workunit.client.0.smithi088.stdout:3/4: dwrite f2 [0,4194304] 0 2024-02-19T19:06:09.908 INFO:tasks.workunit.client.0.smithi088.stdout:0/13: rename f2 to f6 0 2024-02-19T19:06:09.909 INFO:tasks.workunit.client.0.smithi088.stdout:5/4: mknod c1 0 2024-02-19T19:06:09.909 INFO:tasks.workunit.client.0.smithi088.stdout:9/1: mkdir d1 0 2024-02-19T19:06:09.910 INFO:tasks.workunit.client.0.smithi088.stdout:9/2: dwrite - no filename 2024-02-19T19:06:09.910 INFO:tasks.workunit.client.0.smithi088.stdout:9/3: write - no filename 2024-02-19T19:06:09.911 INFO:tasks.workunit.client.0.smithi088.stdout:7/3: creat f1 x:0 0 0 2024-02-19T19:06:09.912 INFO:tasks.workunit.client.0.smithi088.stdout:4/4: mknod d2/c3 0 2024-02-19T19:06:09.914 INFO:tasks.workunit.client.0.smithi088.stdout:7/4: write f1 [991811,76486] 0 2024-02-19T19:06:09.915 INFO:tasks.workunit.client.0.smithi088.stdout:2/15: chown l0 88550 1 2024-02-19T19:06:09.916 INFO:tasks.workunit.client.0.smithi088.stdout:6/15: mkdir d1 0 2024-02-19T19:06:09.922 INFO:tasks.workunit.client.0.smithi088.stdout:8/5: dwrite f0 [0,4194304] 0 2024-02-19T19:06:09.927 INFO:tasks.workunit.client.0.smithi088.stdout:4/5: creat d2/f4 x:0 0 0 2024-02-19T19:06:09.930 INFO:tasks.workunit.client.0.smithi088.stdout:1/7: getdents . 0 2024-02-19T19:06:09.930 INFO:tasks.workunit.client.0.smithi088.stdout:0/14: dwrite f4 [0,4194304] 0 2024-02-19T19:06:09.930 INFO:tasks.workunit.client.0.smithi088.stdout:2/16: mkdir d3 0 2024-02-19T19:06:09.931 INFO:tasks.workunit.client.0.smithi088.stdout:1/8: readlink l1 0 2024-02-19T19:06:09.936 INFO:tasks.workunit.client.0.smithi088.stdout:5/5: dwrite f0 [0,4194304] 0 2024-02-19T19:06:09.938 INFO:tasks.workunit.client.0.smithi088.stdout:9/4: mknod d1/c2 0 2024-02-19T19:06:09.940 INFO:tasks.workunit.client.0.smithi088.stdout:7/5: dwrite f0 [0,4194304] 0 2024-02-19T19:06:09.940 INFO:tasks.workunit.client.0.smithi088.stdout:8/6: unlink f0 0 2024-02-19T19:06:09.942 INFO:tasks.workunit.client.0.smithi088.stdout:8/7: chown l1 0 1 2024-02-19T19:06:09.943 INFO:tasks.workunit.client.0.smithi088.stdout:1/9: creat f2 x:0 0 0 2024-02-19T19:06:09.944 INFO:tasks.workunit.client.0.smithi088.stdout:8/8: stat l1 0 2024-02-19T19:06:09.944 INFO:tasks.workunit.client.0.smithi088.stdout:8/9: dread - no filename 2024-02-19T19:06:09.944 INFO:tasks.workunit.client.0.smithi088.stdout:8/10: dwrite - no filename 2024-02-19T19:06:09.944 INFO:tasks.workunit.client.0.smithi088.stdout:8/11: rmdir - no directory 2024-02-19T19:06:09.944 INFO:tasks.workunit.client.0.smithi088.stdout:0/15: creat f7 x:0 0 0 2024-02-19T19:06:09.947 INFO:tasks.workunit.client.0.smithi088.stdout:5/6: unlink c1 0 2024-02-19T19:06:09.948 INFO:tasks.workunit.client.0.smithi088.stdout:8/12: chown l1 20423123 1 2024-02-19T19:06:09.948 INFO:tasks.workunit.client.0.smithi088.stdout:8/13: dread - no filename 2024-02-19T19:06:09.948 INFO:tasks.workunit.client.0.smithi088.stdout:4/6: dwrite d2/f4 [0,4194304] 0 2024-02-19T19:06:09.948 INFO:tasks.workunit.client.0.smithi088.stdout:9/5: creat d1/f3 x:0 0 0 2024-02-19T19:06:09.950 INFO:tasks.workunit.client.0.smithi088.stdout:2/17: getdents d3 0 2024-02-19T19:06:09.950 INFO:tasks.workunit.client.0.smithi088.stdout:2/18: dread - no filename 2024-02-19T19:06:09.950 INFO:tasks.workunit.client.0.smithi088.stdout:0/16: creat f8 x:0 0 0 2024-02-19T19:06:09.951 INFO:tasks.workunit.client.0.smithi088.stdout:6/16: rmdir d1 0 2024-02-19T19:06:09.951 INFO:tasks.workunit.client.0.smithi088.stdout:6/17: fsync - no filename 2024-02-19T19:06:09.952 INFO:tasks.workunit.client.0.smithi088.stdout:9/6: write d1/f3 [379811,110287] 0 2024-02-19T19:06:09.952 INFO:tasks.workunit.client.0.smithi088.stdout:3/5: getdents . 0 2024-02-19T19:06:09.953 INFO:tasks.workunit.client.0.smithi088.stdout:5/7: creat f2 x:0 0 0 2024-02-19T19:06:09.954 INFO:tasks.workunit.client.0.smithi088.stdout:8/14: creat f2 x:0 0 0 2024-02-19T19:06:09.955 INFO:tasks.workunit.client.0.smithi088.stdout:8/15: write f2 [444720,41069] 0 2024-02-19T19:06:09.959 INFO:tasks.workunit.client.0.smithi088.stdout:4/7: mkdir d2/d5 0 2024-02-19T19:06:09.964 INFO:tasks.workunit.client.0.smithi088.stdout:0/17: mknod c9 0 2024-02-19T19:06:09.966 INFO:tasks.workunit.client.0.smithi088.stdout:6/18: symlink l2 0 2024-02-19T19:06:09.968 INFO:tasks.workunit.client.0.smithi088.stdout:9/7: mkdir d1/d4 0 2024-02-19T19:06:09.969 INFO:tasks.workunit.client.0.smithi088.stdout:5/8: symlink l3 0 2024-02-19T19:06:09.969 INFO:tasks.workunit.client.0.smithi088.stdout:5/9: read - f2 zero size 2024-02-19T19:06:09.972 INFO:tasks.workunit.client.0.smithi088.stdout:8/16: fsync f2 0 2024-02-19T19:06:09.972 INFO:tasks.workunit.client.0.smithi088.stdout:8/17: chown f2 14042 1 2024-02-19T19:06:09.973 INFO:tasks.workunit.client.0.smithi088.stdout:8/18: chown l1 474035 1 2024-02-19T19:06:09.973 INFO:tasks.workunit.client.0.smithi088.stdout:2/19: unlink l0 0 2024-02-19T19:06:09.975 INFO:tasks.workunit.client.0.smithi088.stdout:6/19: creat f3 x:0 0 0 2024-02-19T19:06:09.976 INFO:tasks.workunit.client.0.smithi088.stdout:7/6: truncate f1 793694 0 2024-02-19T19:06:09.979 INFO:tasks.workunit.client.0.smithi088.stdout:5/10: mkdir d4 0 2024-02-19T19:06:09.979 INFO:tasks.workunit.client.0.smithi088.stdout:9/8: rmdir d1 39 2024-02-19T19:06:09.980 INFO:tasks.workunit.client.0.smithi088.stdout:4/8: fdatasync d2/f4 0 2024-02-19T19:06:09.980 INFO:tasks.workunit.client.0.smithi088.stdout:4/9: readlink - no filename 2024-02-19T19:06:09.981 INFO:tasks.workunit.client.0.smithi088.stdout:1/10: rename l1 to l3 0 2024-02-19T19:06:09.981 INFO:tasks.workunit.client.0.smithi088.stdout:1/11: fsync f2 0 2024-02-19T19:06:09.982 INFO:tasks.workunit.client.0.smithi088.stdout:8/19: symlink l3 0 2024-02-19T19:06:09.982 INFO:tasks.workunit.client.0.smithi088.stdout:8/20: readlink l1 0 2024-02-19T19:06:09.982 INFO:tasks.workunit.client.0.smithi088.stdout:8/21: fdatasync f2 0 2024-02-19T19:06:09.984 INFO:tasks.workunit.client.0.smithi088.stdout:0/18: rename f8 to fa 0 2024-02-19T19:06:09.987 INFO:tasks.workunit.client.0.smithi088.stdout:3/6: truncate f2 4013846 0 2024-02-19T19:06:10.279 INFO:tasks.workunit.client.0.smithi088.stdout:9/9: dread d1/f3 [0,4194304] 0 2024-02-19T19:06:10.279 INFO:tasks.workunit.client.0.smithi088.stdout:5/11: rename f2 to d4/f5 0 2024-02-19T19:06:10.291 INFO:tasks.workunit.client.0.smithi088.stdout:4/10: creat d2/f6 x:0 0 0 2024-02-19T19:06:10.293 INFO:tasks.workunit.client.0.smithi088.stdout:2/20: creat d3/f4 x:0 0 0 2024-02-19T19:06:10.293 INFO:tasks.workunit.client.0.smithi088.stdout:5/12: write f0 [4347971,46203] 0 2024-02-19T19:06:10.294 INFO:tasks.workunit.client.0.smithi088.stdout:4/11: chown d2/c3 10567 1 2024-02-19T19:06:10.295 INFO:tasks.workunit.client.0.smithi088.stdout:5/13: chown f0 30 1 2024-02-19T19:06:10.295 INFO:tasks.workunit.client.0.smithi088.stdout:0/19: dwrite f7 [0,4194304] 0 2024-02-19T19:06:10.301 INFO:tasks.workunit.client.0.smithi088.stdout:5/14: dwrite f0 [0,4194304] 0 2024-02-19T19:06:10.397 INFO:tasks.workunit.client.0.smithi088.stdout:1/12: mknod c4 0 2024-02-19T19:06:10.399 INFO:tasks.workunit.client.0.smithi088.stdout:2/21: dread - d3/f4 zero size 2024-02-19T19:06:10.401 INFO:tasks.workunit.client.0.smithi088.stdout:0/20: fdatasync f5 0 2024-02-19T19:06:10.402 INFO:tasks.workunit.client.0.smithi088.stdout:5/15: mkdir d4/d6 0 2024-02-19T19:06:10.402 INFO:tasks.workunit.client.0.smithi088.stdout:3/7: getdents . 0 2024-02-19T19:06:10.404 INFO:tasks.workunit.client.0.smithi088.stdout:1/13: unlink c4 0 2024-02-19T19:06:10.405 INFO:tasks.workunit.client.0.smithi088.stdout:9/10: creat d1/d4/f5 x:0 0 0 2024-02-19T19:06:10.406 INFO:tasks.workunit.client.0.smithi088.stdout:9/11: chown d1/d4/f5 30739 1 2024-02-19T19:06:10.406 INFO:tasks.workunit.client.0.smithi088.stdout:9/12: read - d1/d4/f5 zero size 2024-02-19T19:06:10.407 INFO:tasks.workunit.client.0.smithi088.stdout:2/22: symlink d3/l5 0 2024-02-19T19:06:10.411 INFO:tasks.workunit.client.0.smithi088.stdout:1/14: chown f0 275 1 2024-02-19T19:06:10.412 INFO:tasks.workunit.client.0.smithi088.stdout:9/13: creat d1/f6 x:0 0 0 2024-02-19T19:06:10.413 INFO:tasks.workunit.client.0.smithi088.stdout:5/16: getdents d4/d6 0 2024-02-19T19:06:10.415 INFO:tasks.workunit.client.0.smithi088.stdout:1/15: stat l3 0 2024-02-19T19:06:10.421 INFO:tasks.workunit.client.0.smithi088.stdout:1/16: creat f5 x:0 0 0 2024-02-19T19:06:10.422 INFO:tasks.workunit.client.0.smithi088.stdout:1/17: chown l3 269664 1 2024-02-19T19:06:10.422 INFO:tasks.workunit.client.0.smithi088.stdout:1/18: rmdir - no directory 2024-02-19T19:06:10.423 INFO:tasks.workunit.client.0.smithi088.stdout:1/19: fdatasync f2 0 2024-02-19T19:06:10.423 INFO:tasks.workunit.client.0.smithi088.stdout:1/20: dread - f2 zero size 2024-02-19T19:06:10.426 INFO:tasks.workunit.client.0.smithi088.stdout:1/21: mkdir d6 0 2024-02-19T19:06:10.429 INFO:tasks.workunit.client.0.smithi088.stdout:1/22: mknod d6/c7 0 2024-02-19T19:06:10.430 INFO:tasks.workunit.client.0.smithi088.stdout:1/23: dread - f2 zero size 2024-02-19T19:06:10.433 INFO:tasks.workunit.client.0.smithi088.stdout:1/24: unlink f2 0 2024-02-19T19:06:10.434 INFO:tasks.workunit.client.0.smithi088.stdout:1/25: stat f0 0 2024-02-19T19:06:10.436 INFO:tasks.workunit.client.0.smithi088.stdout:1/26: fdatasync f0 0 2024-02-19T19:06:10.438 INFO:tasks.workunit.client.0.smithi088.stdout:1/27: rmdir d6 39 2024-02-19T19:06:10.438 INFO:tasks.workunit.client.0.smithi088.stdout:1/28: fdatasync f0 0 2024-02-19T19:06:10.439 INFO:tasks.workunit.client.0.smithi088.stdout:1/29: write f0 [5144408,126778] 0 2024-02-19T19:06:10.476 INFO:tasks.workunit.client.0.smithi088.stdout:5/17: getdents d4 0 2024-02-19T19:06:10.477 INFO:tasks.workunit.client.0.smithi088.stdout:3/8: truncate f2 909353 0 2024-02-19T19:06:10.477 INFO:tasks.workunit.client.0.smithi088.stdout:5/18: write f0 [1943350,126732] 0 2024-02-19T19:06:10.478 INFO:tasks.workunit.client.0.smithi088.stdout:5/19: truncate d4/f5 53152 0 2024-02-19T19:06:10.478 INFO:tasks.workunit.client.0.smithi088.stdout:9/14: fsync d1/d4/f5 0 2024-02-19T19:06:10.478 INFO:tasks.workunit.client.0.smithi088.stdout:9/15: dread - d1/f6 zero size 2024-02-19T19:06:10.479 INFO:tasks.workunit.client.0.smithi088.stdout:3/9: mknod c3 0 2024-02-19T19:06:10.479 INFO:tasks.workunit.client.0.smithi088.stdout:9/16: stat c0 0 2024-02-19T19:06:10.480 INFO:tasks.workunit.client.0.smithi088.stdout:5/20: rename d4/d6 to d4/d7 0 2024-02-19T19:06:10.481 INFO:tasks.workunit.client.0.smithi088.stdout:5/21: stat d4/f5 0 2024-02-19T19:06:10.482 INFO:tasks.workunit.client.0.smithi088.stdout:3/10: mkdir d4 0 2024-02-19T19:06:10.484 INFO:tasks.workunit.client.0.smithi088.stdout:9/17: mknod d1/c7 0 2024-02-19T19:06:10.486 INFO:tasks.workunit.client.0.smithi088.stdout:1/30: creat d6/f8 x:0 0 0 2024-02-19T19:06:10.488 INFO:tasks.workunit.client.0.smithi088.stdout:5/22: link d4/f5 d4/f8 0 2024-02-19T19:06:10.488 INFO:tasks.workunit.client.0.smithi088.stdout:3/11: symlink d4/l5 0 2024-02-19T19:06:10.491 INFO:tasks.workunit.client.0.smithi088.stdout:1/31: unlink l3 0 2024-02-19T19:06:10.491 INFO:tasks.workunit.client.0.smithi088.stdout:1/32: chown d6/f8 1211 1 2024-02-19T19:06:10.491 INFO:tasks.workunit.client.0.smithi088.stdout:5/23: creat d4/f9 x:0 0 0 2024-02-19T19:06:10.492 INFO:tasks.workunit.client.0.smithi088.stdout:3/12: creat d4/f6 x:0 0 0 2024-02-19T19:06:10.493 INFO:tasks.workunit.client.0.smithi088.stdout:1/33: mknod d6/c9 0 2024-02-19T19:06:10.493 INFO:tasks.workunit.client.0.smithi088.stdout:9/18: getdents d1/d4 0 2024-02-19T19:06:10.493 INFO:tasks.workunit.client.0.smithi088.stdout:1/34: fsync f5 0 2024-02-19T19:06:10.495 INFO:tasks.workunit.client.0.smithi088.stdout:1/35: dread - d6/f8 zero size 2024-02-19T19:06:10.496 INFO:tasks.workunit.client.0.smithi088.stdout:3/13: creat d4/f7 x:0 0 0 2024-02-19T19:06:10.497 INFO:tasks.workunit.client.0.smithi088.stdout:3/14: readlink d4/l5 0 2024-02-19T19:06:10.499 INFO:tasks.workunit.client.0.smithi088.stdout:5/24: dwrite f0 [0,4194304] 0 2024-02-19T19:06:10.500 INFO:tasks.workunit.client.0.smithi088.stdout:3/15: creat d4/f8 x:0 0 0 2024-02-19T19:06:10.502 INFO:tasks.workunit.client.0.smithi088.stdout:3/16: creat d4/f9 x:0 0 0 2024-02-19T19:06:10.504 INFO:tasks.workunit.client.0.smithi088.stdout:5/25: mkdir d4/d7/da 0 2024-02-19T19:06:10.504 INFO:tasks.workunit.client.0.smithi088.stdout:3/17: fsync d4/f6 0 2024-02-19T19:06:10.505 INFO:tasks.workunit.client.0.smithi088.stdout:1/36: dwrite f0 [0,4194304] 0 2024-02-19T19:06:10.507 INFO:tasks.workunit.client.0.smithi088.stdout:5/26: mknod d4/d7/cb 0 2024-02-19T19:06:10.508 INFO:tasks.workunit.client.0.smithi088.stdout:1/37: write f5 [330043,35523] 0 2024-02-19T19:06:10.509 INFO:tasks.workunit.client.0.smithi088.stdout:3/18: creat d4/fa x:0 0 0 2024-02-19T19:06:10.512 INFO:tasks.workunit.client.0.smithi088.stdout:1/38: rmdir d6 39 2024-02-19T19:06:10.513 INFO:tasks.workunit.client.0.smithi088.stdout:1/39: chown f5 25429492 1 2024-02-19T19:06:10.514 INFO:tasks.workunit.client.0.smithi088.stdout:9/19: dwrite d1/f3 [0,4194304] 0 2024-02-19T19:06:10.514 INFO:tasks.workunit.client.0.smithi088.stdout:3/19: creat d4/fb x:0 0 0 2024-02-19T19:06:10.515 INFO:tasks.workunit.client.0.smithi088.stdout:1/40: chown d6/c9 6177 1 2024-02-19T19:06:10.517 INFO:tasks.workunit.client.0.smithi088.stdout:9/20: mkdir d1/d8 0 2024-02-19T19:06:10.518 INFO:tasks.workunit.client.0.smithi088.stdout:1/41: symlink d6/la 0 2024-02-19T19:06:10.519 INFO:tasks.workunit.client.0.smithi088.stdout:1/42: chown d6/c9 1009124828 1 2024-02-19T19:06:10.520 INFO:tasks.workunit.client.0.smithi088.stdout:9/21: mknod d1/d8/c9 0 2024-02-19T19:06:10.522 INFO:tasks.workunit.client.0.smithi088.stdout:1/43: creat d6/fb x:0 0 0 2024-02-19T19:06:10.523 INFO:tasks.workunit.client.0.smithi088.stdout:1/44: write f5 [1272967,75284] 0 2024-02-19T19:06:10.525 INFO:tasks.workunit.client.0.smithi088.stdout:9/22: rename d1/c7 to d1/ca 0 2024-02-19T19:06:10.532 INFO:tasks.workunit.client.0.smithi088.stdout:1/45: dwrite d6/f8 [0,4194304] 0 2024-02-19T19:06:10.533 INFO:tasks.workunit.client.0.smithi088.stdout:5/27: truncate f0 2615797 0 2024-02-19T19:06:10.538 INFO:tasks.workunit.client.0.smithi088.stdout:3/20: write f2 [1397728,95741] 0 2024-02-19T19:06:10.538 INFO:tasks.workunit.client.0.smithi088.stdout:1/46: link d6/la d6/lc 0 2024-02-19T19:06:10.539 INFO:tasks.workunit.client.0.smithi088.stdout:9/23: rmdir d1 39 2024-02-19T19:06:10.539 INFO:tasks.workunit.client.0.smithi088.stdout:1/47: write f0 [5883747,39286] 0 2024-02-19T19:06:10.539 INFO:tasks.workunit.client.0.smithi088.stdout:3/21: write d4/f8 [303112,53800] 0 2024-02-19T19:06:10.541 INFO:tasks.workunit.client.0.smithi088.stdout:3/22: read - d4/fb zero size 2024-02-19T19:06:10.541 INFO:tasks.workunit.client.0.smithi088.stdout:1/48: truncate d6/fb 761594 0 2024-02-19T19:06:10.541 INFO:tasks.workunit.client.0.smithi088.stdout:3/23: read - d4/f7 zero size 2024-02-19T19:06:10.543 INFO:tasks.workunit.client.0.smithi088.stdout:9/24: dread - d1/d4/f5 zero size 2024-02-19T19:06:10.546 INFO:tasks.workunit.client.0.smithi088.stdout:1/49: rename f0 to d6/fd 0 2024-02-19T19:06:10.548 INFO:tasks.workunit.client.0.smithi088.stdout:5/28: write d4/f8 [838144,78423] 0 2024-02-19T19:06:10.548 INFO:tasks.workunit.client.0.smithi088.stdout:9/25: symlink d1/lb 0 2024-02-19T19:06:10.549 INFO:tasks.workunit.client.0.smithi088.stdout:3/24: dwrite f2 [0,4194304] 0 2024-02-19T19:06:10.552 INFO:tasks.workunit.client.0.smithi088.stdout:3/25: write d4/fa [39266,28043] 0 2024-02-19T19:06:10.552 INFO:tasks.workunit.client.0.smithi088.stdout:9/26: rename d1/f3 to d1/fc 0 2024-02-19T19:06:10.555 INFO:tasks.workunit.client.0.smithi088.stdout:3/26: creat d4/fc x:0 0 0 2024-02-19T19:06:10.556 INFO:tasks.workunit.client.0.smithi088.stdout:1/50: dwrite f5 [0,4194304] 0 2024-02-19T19:06:10.557 INFO:tasks.workunit.client.0.smithi088.stdout:3/27: dread - d4/f6 zero size 2024-02-19T19:06:10.558 INFO:tasks.workunit.client.0.smithi088.stdout:9/27: creat d1/d4/fd x:0 0 0 2024-02-19T19:06:10.561 INFO:tasks.workunit.client.0.smithi088.stdout:3/28: rename d4/f9 to d4/fd 0 2024-02-19T19:06:10.562 INFO:tasks.workunit.client.0.smithi088.stdout:9/28: rmdir d1/d8 39 2024-02-19T19:06:10.562 INFO:tasks.workunit.client.0.smithi088.stdout:1/51: dread f5 [0,4194304] 0 2024-02-19T19:06:10.565 INFO:tasks.workunit.client.0.smithi088.stdout:9/29: unlink c0 0 2024-02-19T19:06:10.567 INFO:tasks.workunit.client.0.smithi088.stdout:3/29: rename c3 to d4/ce 0 2024-02-19T19:06:10.571 INFO:tasks.workunit.client.0.smithi088.stdout:1/52: dwrite d6/f8 [0,4194304] 0 2024-02-19T19:06:10.573 INFO:tasks.workunit.client.0.smithi088.stdout:1/53: rmdir d6 39 2024-02-19T19:06:10.574 INFO:tasks.workunit.client.0.smithi088.stdout:5/29: truncate d4/f5 198497 0 2024-02-19T19:06:10.576 INFO:tasks.workunit.client.0.smithi088.stdout:9/30: dwrite d1/fc [0,4194304] 0 2024-02-19T19:06:10.576 INFO:tasks.workunit.client.0.smithi088.stdout:5/30: truncate d4/f9 167749 0 2024-02-19T19:06:10.577 INFO:tasks.workunit.client.0.smithi088.stdout:5/31: chown d4/d7/da 1353637 1 2024-02-19T19:06:10.578 INFO:tasks.workunit.client.0.smithi088.stdout:1/54: dread d6/fb [0,4194304] 0 2024-02-19T19:06:10.579 INFO:tasks.workunit.client.0.smithi088.stdout:9/31: mknod d1/d4/ce 0 2024-02-19T19:06:10.579 INFO:tasks.workunit.client.0.smithi088.stdout:9/32: chown d1/d4 461674 1 2024-02-19T19:06:10.580 INFO:tasks.workunit.client.0.smithi088.stdout:5/32: symlink d4/d7/da/lc 0 2024-02-19T19:06:10.580 INFO:tasks.workunit.client.0.smithi088.stdout:5/33: chown d4/d7/da 0 1 2024-02-19T19:06:10.584 INFO:tasks.workunit.client.0.smithi088.stdout:9/33: unlink d1/c2 0 2024-02-19T19:06:10.593 INFO:tasks.workunit.client.0.smithi088.stdout:9/34: dwrite d1/d4/f5 [0,4194304] 0 2024-02-19T19:06:10.595 INFO:tasks.workunit.client.0.smithi088.stdout:9/35: chown d1/fc 3 1 2024-02-19T19:06:10.596 INFO:tasks.workunit.client.0.smithi088.stdout:3/30: dwrite f2 [4194304,4194304] 0 2024-02-19T19:06:10.597 INFO:tasks.workunit.client.0.smithi088.stdout:9/36: write d1/f6 [217659,4518] 0 2024-02-19T19:06:10.599 INFO:tasks.workunit.client.0.smithi088.stdout:3/31: creat d4/ff x:0 0 0 2024-02-19T19:06:10.602 INFO:tasks.workunit.client.0.smithi088.stdout:3/32: chown d4/ce 43646368 1 2024-02-19T19:06:10.602 INFO:tasks.workunit.client.0.smithi088.stdout:9/37: symlink d1/d8/lf 0 2024-02-19T19:06:10.602 INFO:tasks.workunit.client.0.smithi088.stdout:3/33: fdatasync d4/f7 0 2024-02-19T19:06:10.603 INFO:tasks.workunit.client.0.smithi088.stdout:9/38: chown d1/d8/c9 225534 1 2024-02-19T19:06:10.604 INFO:tasks.workunit.client.0.smithi088.stdout:9/39: chown d1/d8/c9 58914726 1 2024-02-19T19:06:10.606 INFO:tasks.workunit.client.0.smithi088.stdout:3/34: symlink d4/l10 0 2024-02-19T19:06:10.606 INFO:tasks.workunit.client.0.smithi088.stdout:3/35: stat l0 0 2024-02-19T19:06:10.608 INFO:tasks.workunit.client.0.smithi088.stdout:3/36: mknod d4/c11 0 2024-02-19T19:06:10.609 INFO:tasks.workunit.client.0.smithi088.stdout:9/40: creat d1/f10 x:0 0 0 2024-02-19T19:06:10.610 INFO:tasks.workunit.client.0.smithi088.stdout:9/41: dread - d1/f10 zero size 2024-02-19T19:06:10.610 INFO:tasks.workunit.client.0.smithi088.stdout:3/37: mkdir d4/d12 0 2024-02-19T19:06:10.611 INFO:tasks.workunit.client.0.smithi088.stdout:9/42: write d1/fc [5198158,53660] 0 2024-02-19T19:06:10.613 INFO:tasks.workunit.client.0.smithi088.stdout:3/38: mknod d4/d12/c13 0 2024-02-19T19:06:10.615 INFO:tasks.workunit.client.0.smithi088.stdout:9/43: mknod d1/d8/c11 0 2024-02-19T19:06:10.615 INFO:tasks.workunit.client.0.smithi088.stdout:3/39: rmdir d4/d12 39 2024-02-19T19:06:10.615 INFO:tasks.workunit.client.0.smithi088.stdout:9/44: stat d1/f10 0 2024-02-19T19:06:10.618 INFO:tasks.workunit.client.0.smithi088.stdout:9/45: dread d1/fc [0,4194304] 0 2024-02-19T19:06:10.618 INFO:tasks.workunit.client.0.smithi088.stdout:5/34: read d4/f8 [125751,70179] 0 2024-02-19T19:06:10.619 INFO:tasks.workunit.client.0.smithi088.stdout:3/40: link f2 d4/f14 0 2024-02-19T19:06:10.620 INFO:tasks.workunit.client.0.smithi088.stdout:9/46: fsync d1/d4/fd 0 2024-02-19T19:06:10.620 INFO:tasks.workunit.client.0.smithi088.stdout:5/35: rmdir d4 39 2024-02-19T19:06:10.622 INFO:tasks.workunit.client.0.smithi088.stdout:3/41: unlink d4/f8 0 2024-02-19T19:06:10.624 INFO:tasks.workunit.client.0.smithi088.stdout:3/42: write d4/fb [808172,73538] 0 2024-02-19T19:06:10.624 INFO:tasks.workunit.client.0.smithi088.stdout:9/47: creat d1/d4/f12 x:0 0 0 2024-02-19T19:06:10.626 INFO:tasks.workunit.client.0.smithi088.stdout:5/36: symlink d4/d7/da/ld 0 2024-02-19T19:06:10.628 INFO:tasks.workunit.client.0.smithi088.stdout:3/43: rename d4/d12 to d4/d12/d15 22 2024-02-19T19:06:10.629 INFO:tasks.workunit.client.0.smithi088.stdout:5/37: dread d4/f8 [0,4194304] 0 2024-02-19T19:06:10.629 INFO:tasks.workunit.client.0.smithi088.stdout:3/44: stat d4/fd 0 2024-02-19T19:06:10.630 INFO:tasks.workunit.client.0.smithi088.stdout:3/45: write d4/f6 [616387,54105] 0 2024-02-19T19:06:10.631 INFO:tasks.workunit.client.0.smithi088.stdout:5/38: creat d4/fe x:0 0 0 2024-02-19T19:06:10.632 INFO:tasks.workunit.client.0.smithi088.stdout:5/39: write d4/fe [921678,95042] 0 2024-02-19T19:06:10.633 INFO:tasks.workunit.client.0.smithi088.stdout:5/40: chown d4/f5 14 1 2024-02-19T19:06:10.635 INFO:tasks.workunit.client.0.smithi088.stdout:3/46: rename d4/fb to d4/f16 0 2024-02-19T19:06:10.751 INFO:tasks.workunit.client.0.smithi088.stdout:9/48: rename d1/d4 to d1/d13 0 2024-02-19T19:06:10.753 INFO:tasks.workunit.client.0.smithi088.stdout:9/49: write d1/d13/f12 [760747,30294] 0 2024-02-19T19:06:10.756 INFO:tasks.workunit.client.0.smithi088.stdout:3/47: dread d4/f16 [0,4194304] 0 2024-02-19T19:06:10.757 INFO:tasks.workunit.client.0.smithi088.stdout:5/41: dwrite f0 [0,4194304] 0 2024-02-19T19:06:10.757 INFO:tasks.workunit.client.0.smithi088.stdout:3/48: chown d4/fc 62 1 2024-02-19T19:06:10.762 INFO:tasks.workunit.client.0.smithi088.stdout:3/49: creat d4/d12/f17 x:0 0 0 2024-02-19T19:06:10.763 INFO:tasks.workunit.client.0.smithi088.stdout:5/42: dread d4/fe [0,4194304] 0 2024-02-19T19:06:10.764 INFO:tasks.workunit.client.0.smithi088.stdout:9/50: dwrite d1/d13/f12 [0,4194304] 0 2024-02-19T19:06:10.766 INFO:tasks.workunit.client.0.smithi088.stdout:5/43: dread d4/fe [0,4194304] 0 2024-02-19T19:06:10.768 INFO:tasks.workunit.client.0.smithi088.stdout:3/50: creat d4/f18 x:0 0 0 2024-02-19T19:06:10.768 INFO:tasks.workunit.client.0.smithi088.stdout:5/44: truncate d4/f9 595263 0 2024-02-19T19:06:10.771 INFO:tasks.workunit.client.0.smithi088.stdout:3/51: creat d4/d12/f19 x:0 0 0 2024-02-19T19:06:10.771 INFO:tasks.workunit.client.0.smithi088.stdout:5/45: dread d4/f5 [0,4194304] 0 2024-02-19T19:06:10.775 INFO:tasks.workunit.client.0.smithi088.stdout:9/51: dwrite d1/f6 [0,4194304] 0 2024-02-19T19:06:10.776 INFO:tasks.workunit.client.0.smithi088.stdout:5/46: creat d4/d7/da/ff x:0 0 0 2024-02-19T19:06:10.778 INFO:tasks.workunit.client.0.smithi088.stdout:5/47: symlink d4/d7/l10 0 2024-02-19T19:06:10.784 INFO:tasks.workunit.client.0.smithi088.stdout:9/52: dwrite d1/d13/f5 [0,4194304] 0 2024-02-19T19:06:10.787 INFO:tasks.workunit.client.0.smithi088.stdout:9/53: mkdir d1/d8/d14 0 2024-02-19T19:06:10.789 INFO:tasks.workunit.client.0.smithi088.stdout:3/52: getdents d4 0 2024-02-19T19:06:10.792 INFO:tasks.workunit.client.0.smithi088.stdout:3/53: fsync d4/f16 0 2024-02-19T19:06:10.795 INFO:tasks.workunit.client.0.smithi088.stdout:9/54: dwrite d1/f6 [0,4194304] 0 2024-02-19T19:06:10.795 INFO:tasks.workunit.client.0.smithi088.stdout:3/54: mkdir d4/d1a 0 2024-02-19T19:06:10.796 INFO:tasks.workunit.client.0.smithi088.stdout:5/48: getdents d4/d7 0 2024-02-19T19:06:10.799 INFO:tasks.workunit.client.0.smithi088.stdout:3/55: mknod d4/c1b 0 2024-02-19T19:06:10.802 INFO:tasks.workunit.client.0.smithi088.stdout:5/49: dread d4/f9 [0,4194304] 0 2024-02-19T19:06:10.802 INFO:tasks.workunit.client.0.smithi088.stdout:9/55: dread d1/d13/f12 [0,4194304] 0 2024-02-19T19:06:10.802 INFO:tasks.workunit.client.0.smithi088.stdout:3/56: dread - d4/d12/f19 zero size 2024-02-19T19:06:10.803 INFO:tasks.workunit.client.0.smithi088.stdout:9/56: truncate d1/f10 196672 0 2024-02-19T19:06:10.804 INFO:tasks.workunit.client.0.smithi088.stdout:5/50: creat d4/f11 x:0 0 0 2024-02-19T19:06:10.804 INFO:tasks.workunit.client.0.smithi088.stdout:3/57: fdatasync d4/ff 0 2024-02-19T19:06:10.805 INFO:tasks.workunit.client.0.smithi088.stdout:5/51: chown d4/d7/cb 1088 1 2024-02-19T19:06:10.805 INFO:tasks.workunit.client.0.smithi088.stdout:3/58: fdatasync d4/d12/f19 0 2024-02-19T19:06:10.806 INFO:tasks.workunit.client.0.smithi088.stdout:9/57: symlink d1/l15 0 2024-02-19T19:06:10.806 INFO:tasks.workunit.client.0.smithi088.stdout:9/58: chown d1/fc 31 1 2024-02-19T19:06:10.807 INFO:tasks.workunit.client.0.smithi088.stdout:3/59: unlink f2 0 2024-02-19T19:06:10.809 INFO:tasks.workunit.client.0.smithi088.stdout:3/60: write d4/f18 [62375,52550] 0 2024-02-19T19:06:10.810 INFO:tasks.workunit.client.0.smithi088.stdout:9/59: dread d1/d13/f12 [0,4194304] 0 2024-02-19T19:06:10.810 INFO:tasks.workunit.client.0.smithi088.stdout:5/52: getdents d4/d7/da 0 2024-02-19T19:06:10.811 INFO:tasks.workunit.client.0.smithi088.stdout:5/53: chown d4 0 1 2024-02-19T19:06:10.812 INFO:tasks.workunit.client.0.smithi088.stdout:3/61: mknod d4/c1c 0 2024-02-19T19:06:10.814 INFO:tasks.workunit.client.0.smithi088.stdout:5/54: link d4/f9 d4/f12 0 2024-02-19T19:06:10.815 INFO:tasks.workunit.client.0.smithi088.stdout:9/60: dread d1/d13/f12 [0,4194304] 0 2024-02-19T19:06:10.817 INFO:tasks.workunit.client.0.smithi088.stdout:3/62: rename d4/fa to d4/d12/f1d 0 2024-02-19T19:06:10.818 INFO:tasks.workunit.client.0.smithi088.stdout:5/55: mknod d4/d7/c13 0 2024-02-19T19:06:10.818 INFO:tasks.workunit.client.0.smithi088.stdout:5/56: write d4/d7/da/ff [976888,46878] 0 2024-02-19T19:06:10.823 INFO:tasks.workunit.client.0.smithi088.stdout:5/57: symlink d4/d7/da/l14 0 2024-02-19T19:06:10.824 INFO:tasks.workunit.client.0.smithi088.stdout:3/63: mkdir d4/d1a/d1e 0 2024-02-19T19:06:10.828 INFO:tasks.workunit.client.0.smithi088.stdout:9/61: getdents d1/d8 0 2024-02-19T19:06:10.831 INFO:tasks.workunit.client.0.smithi088.stdout:9/62: rmdir d1/d13 39 2024-02-19T19:06:10.833 INFO:tasks.workunit.client.0.smithi088.stdout:9/63: rmdir d1 39 2024-02-19T19:06:10.837 INFO:tasks.workunit.client.0.smithi088.stdout:5/58: dwrite d4/f9 [0,4194304] 0 2024-02-19T19:06:10.839 INFO:tasks.workunit.client.0.smithi088.stdout:9/64: creat d1/d8/d14/f16 x:0 0 0 2024-02-19T19:06:10.840 INFO:tasks.workunit.client.0.smithi088.stdout:3/64: dread d4/f16 [0,4194304] 0 2024-02-19T19:06:10.840 INFO:tasks.workunit.client.0.smithi088.stdout:5/59: creat d4/d7/f15 x:0 0 0 2024-02-19T19:06:10.842 INFO:tasks.workunit.client.0.smithi088.stdout:9/65: write d1/d13/f12 [5066925,47189] 0 2024-02-19T19:06:10.844 INFO:tasks.workunit.client.0.smithi088.stdout:3/65: write d4/f6 [235497,476] 0 2024-02-19T19:06:10.848 INFO:tasks.workunit.client.0.smithi088.stdout:3/66: write d4/f18 [184949,120943] 0 2024-02-19T19:06:10.850 INFO:tasks.workunit.client.0.smithi088.stdout:5/60: dwrite d4/d7/da/ff [0,4194304] 0 2024-02-19T19:06:10.851 INFO:tasks.workunit.client.0.smithi088.stdout:3/67: readlink d4/l10 0 2024-02-19T19:06:10.851 INFO:tasks.workunit.client.0.smithi088.stdout:5/61: dread - d4/f11 zero size 2024-02-19T19:06:10.852 INFO:tasks.workunit.client.0.smithi088.stdout:5/62: write d4/d7/f15 [700357,11537] 0 2024-02-19T19:06:10.855 INFO:tasks.workunit.client.0.smithi088.stdout:9/66: truncate d1/d13/f12 1902728 0 2024-02-19T19:06:10.855 INFO:tasks.workunit.client.0.smithi088.stdout:3/68: dread d4/f18 [0,4194304] 0 2024-02-19T19:06:10.859 INFO:tasks.workunit.client.0.smithi088.stdout:3/69: chown d4/fd 0 1 2024-02-19T19:06:10.864 INFO:tasks.workunit.client.0.smithi088.stdout:9/67: dwrite d1/d13/fd [0,4194304] 0 2024-02-19T19:06:10.866 INFO:tasks.workunit.client.0.smithi088.stdout:9/68: unlink d1/d13/f5 0 2024-02-19T19:06:10.868 INFO:tasks.workunit.client.0.smithi088.stdout:9/69: chown d1/d8/d14 0 1 2024-02-19T19:06:10.871 INFO:tasks.workunit.client.0.smithi088.stdout:3/70: dwrite d4/fd [0,4194304] 0 2024-02-19T19:06:10.875 INFO:tasks.workunit.client.0.smithi088.stdout:3/71: symlink d4/d12/l1f 0 2024-02-19T19:06:10.876 INFO:tasks.workunit.client.0.smithi088.stdout:9/70: read d1/d13/f12 [1314757,86943] 0 2024-02-19T19:06:10.878 INFO:tasks.workunit.client.0.smithi088.stdout:5/63: truncate d4/d7/da/ff 2606239 0 2024-02-19T19:06:10.880 INFO:tasks.workunit.client.0.smithi088.stdout:3/72: unlink d4/d12/f19 0 2024-02-19T19:06:10.881 INFO:tasks.workunit.client.0.smithi088.stdout:9/71: write d1/d13/f12 [2150424,14918] 0 2024-02-19T19:06:10.882 INFO:tasks.workunit.client.0.smithi088.stdout:9/72: truncate d1/fc 5684557 0 2024-02-19T19:06:10.884 INFO:tasks.workunit.client.0.smithi088.stdout:5/64: creat d4/f16 x:0 0 0 2024-02-19T19:06:10.887 INFO:tasks.workunit.client.0.smithi088.stdout:9/73: symlink d1/d13/l17 0 2024-02-19T19:06:10.890 INFO:tasks.workunit.client.0.smithi088.stdout:3/73: link d4/c11 d4/d12/c20 0 2024-02-19T19:06:10.897 INFO:tasks.workunit.client.0.smithi088.stdout:5/65: truncate d4/d7/f15 365403 0 2024-02-19T19:06:10.898 INFO:tasks.workunit.client.0.smithi088.stdout:3/74: dwrite d4/ff [0,4194304] 0 2024-02-19T19:06:10.899 INFO:tasks.workunit.client.0.smithi088.stdout:5/66: truncate d4/f5 507511 0 2024-02-19T19:06:10.900 INFO:tasks.workunit.client.0.smithi088.stdout:3/75: write d4/f7 [287093,107319] 0 2024-02-19T19:06:10.904 INFO:tasks.workunit.client.0.smithi088.stdout:3/76: creat d4/d12/f21 x:0 0 0 2024-02-19T19:06:10.904 INFO:tasks.workunit.client.0.smithi088.stdout:5/67: dread f0 [0,4194304] 0 2024-02-19T19:06:10.906 INFO:tasks.workunit.client.0.smithi088.stdout:3/77: mkdir d4/d22 0 2024-02-19T19:06:10.906 INFO:tasks.workunit.client.0.smithi088.stdout:5/68: fsync d4/d7/da/ff 0 2024-02-19T19:06:10.907 INFO:tasks.workunit.client.0.smithi088.stdout:5/69: fdatasync d4/f5 0 2024-02-19T19:06:10.908 INFO:tasks.workunit.client.0.smithi088.stdout:3/78: mkdir d4/d1a/d23 0 2024-02-19T19:06:10.910 INFO:tasks.workunit.client.0.smithi088.stdout:3/79: getdents d4/d22 0 2024-02-19T19:06:10.912 INFO:tasks.workunit.client.0.smithi088.stdout:3/80: creat d4/d1a/f24 x:0 0 0 2024-02-19T19:06:10.912 INFO:tasks.workunit.client.0.smithi088.stdout:3/81: chown d4/d1a 19 1 2024-02-19T19:06:10.912 INFO:tasks.workunit.client.0.smithi088.stdout:5/70: write d4/d7/f15 [1098941,97963] 0 2024-02-19T19:06:10.913 INFO:tasks.workunit.client.0.smithi088.stdout:3/82: write d4/f6 [332288,61740] 0 2024-02-19T19:06:10.921 INFO:tasks.workunit.client.0.smithi088.stdout:5/71: dwrite d4/f8 [0,4194304] 0 2024-02-19T19:06:10.921 INFO:tasks.workunit.client.0.smithi088.stdout:5/72: readlink d4/d7/da/lc 0 2024-02-19T19:06:11.025 INFO:tasks.workunit.client.0.smithi088.stdout:5/73: mknod d4/d7/c17 0 2024-02-19T19:06:11.026 INFO:tasks.workunit.client.0.smithi088.stdout:3/83: mknod d4/d1a/d23/c25 0 2024-02-19T19:06:11.028 INFO:tasks.workunit.client.0.smithi088.stdout:5/74: rename d4/d7/c13 to d4/d7/da/c18 0 2024-02-19T19:06:11.029 INFO:tasks.workunit.client.0.smithi088.stdout:5/75: write d4/f12 [2175715,32906] 0 2024-02-19T19:06:11.031 INFO:tasks.workunit.client.0.smithi088.stdout:3/84: getdents d4/d12 0 2024-02-19T19:06:11.031 INFO:tasks.workunit.client.0.smithi088.stdout:3/85: write d4/ff [2052138,116201] 0 2024-02-19T19:06:11.032 INFO:tasks.workunit.client.0.smithi088.stdout:5/76: creat d4/d7/da/f19 x:0 0 0 2024-02-19T19:06:11.033 INFO:tasks.workunit.client.0.smithi088.stdout:5/77: read d4/f8 [734784,107480] 0 2024-02-19T19:06:11.036 INFO:tasks.workunit.client.0.smithi088.stdout:3/86: symlink d4/l26 0 2024-02-19T19:06:11.037 INFO:tasks.workunit.client.0.smithi088.stdout:3/87: read - d4/d12/f17 zero size 2024-02-19T19:06:11.037 INFO:tasks.workunit.client.0.smithi088.stdout:5/78: dread d4/d7/f15 [0,4194304] 0 2024-02-19T19:06:11.039 INFO:tasks.workunit.client.0.smithi088.stdout:3/88: creat d4/d12/f27 x:0 0 0 2024-02-19T19:06:11.039 INFO:tasks.workunit.client.0.smithi088.stdout:3/89: read - d4/d12/f21 zero size 2024-02-19T19:06:11.046 INFO:tasks.workunit.client.0.smithi088.stdout:3/90: dread d4/ff [0,4194304] 0 2024-02-19T19:06:11.046 INFO:tasks.workunit.client.0.smithi088.stdout:5/79: dwrite d4/fe [0,4194304] 0 2024-02-19T19:06:11.049 INFO:tasks.workunit.client.0.smithi088.stdout:3/91: creat d4/f28 x:0 0 0 2024-02-19T19:06:11.051 INFO:tasks.workunit.client.0.smithi088.stdout:3/92: stat d4/f14 0 2024-02-19T19:06:11.053 INFO:tasks.workunit.client.0.smithi088.stdout:3/93: symlink d4/d1a/l29 0 2024-02-19T19:06:11.059 INFO:tasks.workunit.client.0.smithi088.stdout:5/80: fsync d4/fe 0 2024-02-19T19:06:11.062 INFO:tasks.workunit.client.0.smithi088.stdout:5/81: dread f0 [0,4194304] 0 2024-02-19T19:06:11.062 INFO:tasks.workunit.client.0.smithi088.stdout:5/82: fdatasync d4/f12 0 2024-02-19T19:06:11.064 INFO:tasks.workunit.client.0.smithi088.stdout:5/83: mkdir d4/d1a 0 2024-02-19T19:06:11.064 INFO:tasks.workunit.client.0.smithi088.stdout:5/84: chown d4/f16 203783 1 2024-02-19T19:06:11.069 INFO:tasks.workunit.client.0.smithi088.stdout:5/85: dwrite d4/f12 [0,4194304] 0 2024-02-19T19:06:11.071 INFO:tasks.workunit.client.0.smithi088.stdout:5/86: creat d4/d1a/f1b x:0 0 0 2024-02-19T19:06:11.071 INFO:tasks.workunit.client.0.smithi088.stdout:5/87: stat d4/d7 0 2024-02-19T19:06:11.072 INFO:tasks.workunit.client.0.smithi088.stdout:5/88: fdatasync d4/fe 0 2024-02-19T19:06:11.094 INFO:tasks.workunit.client.0.smithi088.stdout:5/89: dread d4/f12 [0,4194304] 0 2024-02-19T19:06:11.099 INFO:tasks.workunit.client.0.smithi088.stdout:5/90: dwrite f0 [0,4194304] 0 2024-02-19T19:06:11.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:11 smithi088 ceph-mon[88643]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 540 op/s 2024-02-19T19:06:11.620 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:11 smithi132 ceph-mon[70676]: pgmap v39: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 4.9 MiB/s wr, 540 op/s 2024-02-19T19:06:12.223 INFO:tasks.workunit.client.0.smithi088.stdout:6/20: sync 2024-02-19T19:06:12.224 INFO:tasks.workunit.client.0.smithi088.stdout:7/7: sync 2024-02-19T19:06:12.225 INFO:tasks.workunit.client.0.smithi088.stdout:6/21: rename l2 to l4 0 2024-02-19T19:06:12.226 INFO:tasks.workunit.client.0.smithi088.stdout:6/22: truncate f3 865482 0 2024-02-19T19:06:12.226 INFO:tasks.workunit.client.0.smithi088.stdout:7/8: rename f1 to f2 0 2024-02-19T19:06:12.263 INFO:tasks.workunit.client.0.smithi088.stdout:6/23: stat l4 0 2024-02-19T19:06:12.264 INFO:tasks.workunit.client.0.smithi088.stdout:6/24: fsync f3 0 2024-02-19T19:06:12.266 INFO:tasks.workunit.client.0.smithi088.stdout:6/25: mkdir d5 0 2024-02-19T19:06:12.266 INFO:tasks.workunit.client.0.smithi088.stdout:7/9: chown f2 370655933 1 2024-02-19T19:06:12.266 INFO:tasks.workunit.client.0.smithi088.stdout:6/26: fsync f3 0 2024-02-19T19:06:12.268 INFO:tasks.workunit.client.0.smithi088.stdout:7/10: unlink f0 0 2024-02-19T19:06:12.269 INFO:tasks.workunit.client.0.smithi088.stdout:6/27: unlink f3 0 2024-02-19T19:06:12.269 INFO:tasks.workunit.client.0.smithi088.stdout:6/28: dread - no filename 2024-02-19T19:06:12.271 INFO:tasks.workunit.client.0.smithi088.stdout:7/11: mknod c3 0 2024-02-19T19:06:12.273 INFO:tasks.workunit.client.0.smithi088.stdout:6/29: unlink c0 0 2024-02-19T19:06:12.273 INFO:tasks.workunit.client.0.smithi088.stdout:6/30: write - no filename 2024-02-19T19:06:12.274 INFO:tasks.workunit.client.0.smithi088.stdout:6/31: getdents d5 0 2024-02-19T19:06:12.276 INFO:tasks.workunit.client.0.smithi088.stdout:6/32: symlink d5/l6 0 2024-02-19T19:06:12.276 INFO:tasks.workunit.client.0.smithi088.stdout:6/33: dwrite - no filename 2024-02-19T19:06:12.278 INFO:tasks.workunit.client.0.smithi088.stdout:6/34: mkdir d5/d7 0 2024-02-19T19:06:12.278 INFO:tasks.workunit.client.0.smithi088.stdout:6/35: chown d5/d7 3764 1 2024-02-19T19:06:12.280 INFO:tasks.workunit.client.0.smithi088.stdout:6/36: mkdir d5/d8 0 2024-02-19T19:06:12.281 INFO:tasks.workunit.client.0.smithi088.stdout:7/12: write f2 [1554458,52013] 0 2024-02-19T19:06:12.281 INFO:tasks.workunit.client.0.smithi088.stdout:6/37: mkdir d5/d7/d9 0 2024-02-19T19:06:12.283 INFO:tasks.workunit.client.0.smithi088.stdout:7/13: unlink c3 0 2024-02-19T19:06:12.283 INFO:tasks.workunit.client.0.smithi088.stdout:7/14: write f2 [768244,1776] 0 2024-02-19T19:06:12.284 INFO:tasks.workunit.client.0.smithi088.stdout:6/38: mkdir d5/d7/da 0 2024-02-19T19:06:12.284 INFO:tasks.workunit.client.0.smithi088.stdout:6/39: fsync - no filename 2024-02-19T19:06:12.284 INFO:tasks.workunit.client.0.smithi088.stdout:6/40: dwrite - no filename 2024-02-19T19:06:12.284 INFO:tasks.workunit.client.0.smithi088.stdout:6/41: dread - no filename 2024-02-19T19:06:12.285 INFO:tasks.workunit.client.0.smithi088.stdout:7/15: symlink l4 0 2024-02-19T19:06:12.286 INFO:tasks.workunit.client.0.smithi088.stdout:6/42: mknod d5/cb 0 2024-02-19T19:06:12.286 INFO:tasks.workunit.client.0.smithi088.stdout:6/43: write - no filename 2024-02-19T19:06:12.286 INFO:tasks.workunit.client.0.smithi088.stdout:6/44: dread - no filename 2024-02-19T19:06:12.287 INFO:tasks.workunit.client.0.smithi088.stdout:6/45: rename d5 to d5/d7/da/dc 22 2024-02-19T19:06:12.287 INFO:tasks.workunit.client.0.smithi088.stdout:6/46: dwrite - no filename 2024-02-19T19:06:12.287 INFO:tasks.workunit.client.0.smithi088.stdout:7/16: mknod c5 0 2024-02-19T19:06:12.288 INFO:tasks.workunit.client.0.smithi088.stdout:7/17: creat f6 x:0 0 0 2024-02-19T19:06:12.294 INFO:tasks.workunit.client.0.smithi088.stdout:7/18: dread f2 [0,4194304] 0 2024-02-19T19:06:12.296 INFO:tasks.workunit.client.0.smithi088.stdout:7/19: mknod c7 0 2024-02-19T19:06:12.297 INFO:tasks.workunit.client.0.smithi088.stdout:7/20: fsync f2 0 2024-02-19T19:06:12.322 INFO:tasks.workunit.client.0.smithi088.stdout:7/21: dwrite f6 [0,4194304] 0 2024-02-19T19:06:12.432 INFO:tasks.workunit.client.0.smithi088.stdout:8/22: sync 2024-02-19T19:06:12.432 INFO:tasks.workunit.client.0.smithi088.stdout:2/23: sync 2024-02-19T19:06:12.432 INFO:tasks.workunit.client.0.smithi088.stdout:5/91: sync 2024-02-19T19:06:12.433 INFO:tasks.workunit.client.0.smithi088.stdout:0/21: sync 2024-02-19T19:06:12.433 INFO:tasks.workunit.client.0.smithi088.stdout:9/74: sync 2024-02-19T19:06:12.433 INFO:tasks.workunit.client.0.smithi088.stdout:4/12: sync 2024-02-19T19:06:12.433 INFO:tasks.workunit.client.0.smithi088.stdout:1/55: sync 2024-02-19T19:06:12.433 INFO:tasks.workunit.client.0.smithi088.stdout:3/94: sync 2024-02-19T19:06:12.433 INFO:tasks.workunit.client.0.smithi088.stdout:0/22: stat f3 0 2024-02-19T19:06:12.436 INFO:tasks.workunit.client.0.smithi088.stdout:8/23: read f2 [84090,31882] 0 2024-02-19T19:06:12.439 INFO:tasks.workunit.client.0.smithi088.stdout:8/24: write f2 [737782,91974] 0 2024-02-19T19:06:12.444 INFO:tasks.workunit.client.0.smithi088.stdout:8/25: write f2 [210233,98167] 0 2024-02-19T19:06:12.447 INFO:tasks.workunit.client.0.smithi088.stdout:4/13: dwrite f0 [4194304,4194304] 0 2024-02-19T19:06:12.448 INFO:tasks.workunit.client.0.smithi088.stdout:8/26: chown f2 3623 1 2024-02-19T19:06:12.449 INFO:tasks.workunit.client.0.smithi088.stdout:5/92: dwrite d4/fe [0,4194304] 0 2024-02-19T19:06:12.472 INFO:tasks.workunit.client.0.smithi088.stdout:8/27: fsync f2 0 2024-02-19T19:06:12.473 INFO:tasks.workunit.client.0.smithi088.stdout:8/28: chown l1 19266478 1 2024-02-19T19:06:12.501 INFO:tasks.workunit.client.0.smithi088.stdout:7/22: creat f8 x:0 0 0 2024-02-19T19:06:12.501 INFO:tasks.workunit.client.0.smithi088.stdout:3/95: readlink d4/l5 0 2024-02-19T19:06:12.502 INFO:tasks.workunit.client.0.smithi088.stdout:1/56: mknod d6/ce 0 2024-02-19T19:06:12.504 INFO:tasks.workunit.client.0.smithi088.stdout:0/23: unlink f0 0 2024-02-19T19:06:12.507 INFO:tasks.workunit.client.0.smithi088.stdout:9/75: mkdir d1/d8/d14/d18 0 2024-02-19T19:06:12.510 INFO:tasks.workunit.client.0.smithi088.stdout:5/93: mknod d4/d1a/c1c 0 2024-02-19T19:06:12.512 INFO:tasks.workunit.client.0.smithi088.stdout:0/24: dread f3 [0,4194304] 0 2024-02-19T19:06:12.512 INFO:tasks.workunit.client.0.smithi088.stdout:5/94: read d4/fe [1997756,66213] 0 2024-02-19T19:06:12.514 INFO:tasks.workunit.client.0.smithi088.stdout:3/96: dwrite d4/fd [0,4194304] 0 2024-02-19T19:06:12.515 INFO:tasks.workunit.client.0.smithi088.stdout:3/97: chown d4/f6 1867 1 2024-02-19T19:06:12.516 INFO:tasks.workunit.client.0.smithi088.stdout:1/57: creat d6/ff x:0 0 0 2024-02-19T19:06:12.516 INFO:tasks.workunit.client.0.smithi088.stdout:5/95: truncate d4/f12 5213748 0 2024-02-19T19:06:12.516 INFO:tasks.workunit.client.0.smithi088.stdout:3/98: readlink l0 0 2024-02-19T19:06:12.517 INFO:tasks.workunit.client.0.smithi088.stdout:9/76: unlink d1/d8/c9 0 2024-02-19T19:06:12.518 INFO:tasks.workunit.client.0.smithi088.stdout:9/77: write d1/f10 [1009699,13427] 0 2024-02-19T19:06:12.519 INFO:tasks.workunit.client.0.smithi088.stdout:1/58: chown d6/ff 2 1 2024-02-19T19:06:12.519 INFO:tasks.workunit.client.0.smithi088.stdout:9/78: dread - d1/d8/d14/f16 zero size 2024-02-19T19:06:12.524 INFO:tasks.workunit.client.0.smithi088.stdout:0/25: rename f6 to fb 0 2024-02-19T19:06:12.524 INFO:tasks.workunit.client.0.smithi088.stdout:0/26: rmdir - no directory 2024-02-19T19:06:12.525 INFO:tasks.workunit.client.0.smithi088.stdout:5/96: creat d4/d7/f1d x:0 0 0 2024-02-19T19:06:12.526 INFO:tasks.workunit.client.0.smithi088.stdout:9/79: dwrite d1/d13/fd [0,4194304] 0 2024-02-19T19:06:12.527 INFO:tasks.workunit.client.0.smithi088.stdout:5/97: dread - d4/d1a/f1b zero size 2024-02-19T19:06:12.614 INFO:tasks.workunit.client.0.smithi088.stdout:0/27: symlink lc 0 2024-02-19T19:06:12.624 INFO:tasks.workunit.client.0.smithi088.stdout:9/80: symlink d1/d13/l19 0 2024-02-19T19:06:12.625 INFO:tasks.workunit.client.0.smithi088.stdout:4/14: write f0 [8373993,28920] 0 2024-02-19T19:06:12.625 INFO:tasks.workunit.client.0.smithi088.stdout:9/81: read d1/fc [2702476,128527] 0 2024-02-19T19:06:12.628 INFO:tasks.workunit.client.0.smithi088.stdout:7/23: truncate f6 161555 0 2024-02-19T19:06:12.628 INFO:tasks.workunit.client.0.smithi088.stdout:4/15: fsync d2/f6 0 2024-02-19T19:06:12.629 INFO:tasks.workunit.client.0.smithi088.stdout:7/24: chown c7 25 1 2024-02-19T19:06:12.630 INFO:tasks.workunit.client.0.smithi088.stdout:4/16: fdatasync f0 0 2024-02-19T19:06:12.632 INFO:tasks.workunit.client.0.smithi088.stdout:1/59: getdents d6 0 2024-02-19T19:06:12.634 INFO:tasks.workunit.client.0.smithi088.stdout:0/28: dwrite f3 [0,4194304] 0 2024-02-19T19:06:12.635 INFO:tasks.workunit.client.0.smithi088.stdout:5/98: getdents d4/d7/da 0 2024-02-19T19:06:12.635 INFO:tasks.workunit.client.0.smithi088.stdout:0/29: dread - fa zero size 2024-02-19T19:06:12.636 INFO:tasks.workunit.client.0.smithi088.stdout:1/60: mkdir d6/d10 0 2024-02-19T19:06:12.638 INFO:tasks.workunit.client.0.smithi088.stdout:0/30: symlink ld 0 2024-02-19T19:06:12.639 INFO:tasks.workunit.client.0.smithi088.stdout:5/99: mknod d4/d7/da/c1e 0 2024-02-19T19:06:12.639 INFO:tasks.workunit.client.0.smithi088.stdout:9/82: truncate d1/fc 5121304 0 2024-02-19T19:06:12.641 INFO:tasks.workunit.client.0.smithi088.stdout:9/83: write d1/d8/d14/f16 [710537,127345] 0 2024-02-19T19:06:12.642 INFO:tasks.workunit.client.0.smithi088.stdout:1/61: unlink d6/f8 0 2024-02-19T19:06:12.642 INFO:tasks.workunit.client.0.smithi088.stdout:1/62: write d6/ff [369479,75146] 0 2024-02-19T19:06:12.643 INFO:tasks.workunit.client.0.smithi088.stdout:0/31: creat fe x:0 0 0 2024-02-19T19:06:12.644 INFO:tasks.workunit.client.0.smithi088.stdout:0/32: chown fa 103168485 1 2024-02-19T19:06:12.645 INFO:tasks.workunit.client.0.smithi088.stdout:5/100: mknod d4/c1f 0 2024-02-19T19:06:12.646 INFO:tasks.workunit.client.0.smithi088.stdout:0/33: symlink lf 0 2024-02-19T19:06:12.647 INFO:tasks.workunit.client.0.smithi088.stdout:0/34: chown lc 16882 1 2024-02-19T19:06:12.647 INFO:tasks.workunit.client.0.smithi088.stdout:1/63: creat d6/d10/f11 x:0 0 0 2024-02-19T19:06:12.648 INFO:tasks.workunit.client.0.smithi088.stdout:5/101: symlink d4/l20 0 2024-02-19T19:06:12.650 INFO:tasks.workunit.client.0.smithi088.stdout:0/35: rename ld to l10 0 2024-02-19T19:06:12.652 INFO:tasks.workunit.client.0.smithi088.stdout:1/64: mknod d6/d10/c12 0 2024-02-19T19:06:12.654 INFO:tasks.workunit.client.0.smithi088.stdout:1/65: write d6/ff [802788,12695] 0 2024-02-19T19:06:12.658 INFO:tasks.workunit.client.0.smithi088.stdout:1/66: symlink d6/d10/l13 0 2024-02-19T19:06:12.658 INFO:tasks.workunit.client.0.smithi088.stdout:5/102: dwrite d4/d7/f1d [0,4194304] 0 2024-02-19T19:06:12.662 INFO:tasks.workunit.client.0.smithi088.stdout:5/103: creat d4/d1a/f21 x:0 0 0 2024-02-19T19:06:12.665 INFO:tasks.workunit.client.0.smithi088.stdout:0/36: dwrite fb [0,4194304] 0 2024-02-19T19:06:12.667 INFO:tasks.workunit.client.0.smithi088.stdout:0/37: write fb [1583160,48745] 0 2024-02-19T19:06:12.670 INFO:tasks.workunit.client.0.smithi088.stdout:0/38: mknod c11 0 2024-02-19T19:06:12.673 INFO:tasks.workunit.client.0.smithi088.stdout:1/67: dwrite d6/ff [0,4194304] 0 2024-02-19T19:06:12.675 INFO:tasks.workunit.client.0.smithi088.stdout:9/84: truncate d1/fc 2861041 0 2024-02-19T19:06:12.676 INFO:tasks.workunit.client.0.smithi088.stdout:5/104: dwrite f0 [0,4194304] 0 2024-02-19T19:06:12.677 INFO:tasks.workunit.client.0.smithi088.stdout:0/39: rename lc to l12 0 2024-02-19T19:06:12.679 INFO:tasks.workunit.client.0.smithi088.stdout:1/68: creat d6/f14 x:0 0 0 2024-02-19T19:06:12.681 INFO:tasks.workunit.client.0.smithi088.stdout:0/40: mkdir d13 0 2024-02-19T19:06:12.683 INFO:tasks.workunit.client.0.smithi088.stdout:9/85: creat d1/d8/d14/d18/f1a x:0 0 0 2024-02-19T19:06:12.684 INFO:tasks.workunit.client.0.smithi088.stdout:9/86: write d1/d13/fd [423138,109143] 0 2024-02-19T19:06:12.685 INFO:tasks.workunit.client.0.smithi088.stdout:5/105: rename d4/d7/da to d4/d1a/d22 0 2024-02-19T19:06:12.685 INFO:tasks.workunit.client.0.smithi088.stdout:6/47: sync 2024-02-19T19:06:12.686 INFO:tasks.workunit.client.0.smithi088.stdout:1/69: readlink d6/lc 0 2024-02-19T19:06:12.688 INFO:tasks.workunit.client.0.smithi088.stdout:0/41: symlink d13/l14 0 2024-02-19T19:06:12.689 INFO:tasks.workunit.client.0.smithi088.stdout:5/106: creat d4/d7/f23 x:0 0 0 2024-02-19T19:06:12.689 INFO:tasks.workunit.client.0.smithi088.stdout:6/48: rename d5/cb to d5/d8/cd 0 2024-02-19T19:06:12.695 INFO:tasks.workunit.client.0.smithi088.stdout:6/49: rename d5/d7/d9 to d5/de 0 2024-02-19T19:06:12.695 INFO:tasks.workunit.client.0.smithi088.stdout:6/50: dread - no filename 2024-02-19T19:06:12.696 INFO:tasks.workunit.client.0.smithi088.stdout:1/70: dwrite d6/f14 [0,4194304] 0 2024-02-19T19:06:12.696 INFO:tasks.workunit.client.0.smithi088.stdout:5/107: rename d4/d1a/d22/f19 to d4/d1a/d22/f24 0 2024-02-19T19:06:12.697 INFO:tasks.workunit.client.0.smithi088.stdout:1/71: stat d6/d10/f11 0 2024-02-19T19:06:12.697 INFO:tasks.workunit.client.0.smithi088.stdout:6/51: unlink l4 0 2024-02-19T19:06:12.697 INFO:tasks.workunit.client.0.smithi088.stdout:6/52: write - no filename 2024-02-19T19:06:12.698 INFO:tasks.workunit.client.0.smithi088.stdout:6/53: dread - no filename 2024-02-19T19:06:12.699 INFO:tasks.workunit.client.0.smithi088.stdout:5/108: write d4/f12 [4622022,82811] 0 2024-02-19T19:06:12.700 INFO:tasks.workunit.client.0.smithi088.stdout:1/72: creat d6/d10/f15 x:0 0 0 2024-02-19T19:06:12.700 INFO:tasks.workunit.client.0.smithi088.stdout:5/109: dread - d4/f11 zero size 2024-02-19T19:06:12.703 INFO:tasks.workunit.client.0.smithi088.stdout:6/54: creat d5/ff x:0 0 0 2024-02-19T19:06:12.707 INFO:tasks.workunit.client.0.smithi088.stdout:1/73: creat d6/d10/f16 x:0 0 0 2024-02-19T19:06:12.708 INFO:tasks.workunit.client.0.smithi088.stdout:6/55: creat d5/de/f10 x:0 0 0 2024-02-19T19:06:12.710 INFO:tasks.workunit.client.0.smithi088.stdout:6/56: chown d5/d7/da 5 1 2024-02-19T19:06:12.710 INFO:tasks.workunit.client.0.smithi088.stdout:0/42: getdents d13 0 2024-02-19T19:06:12.711 INFO:tasks.workunit.client.0.smithi088.stdout:6/57: chown d5/de 4142786 1 2024-02-19T19:06:12.713 INFO:tasks.workunit.client.0.smithi088.stdout:6/58: chown d5/d7/da 235067122 1 2024-02-19T19:06:12.714 INFO:tasks.workunit.client.0.smithi088.stdout:0/43: unlink fa 0 2024-02-19T19:06:12.715 INFO:tasks.workunit.client.0.smithi088.stdout:0/44: write f3 [4071457,31400] 0 2024-02-19T19:06:12.716 INFO:tasks.workunit.client.0.smithi088.stdout:1/74: dwrite d6/d10/f15 [0,4194304] 0 2024-02-19T19:06:12.718 INFO:tasks.workunit.client.0.smithi088.stdout:6/59: mknod d5/d7/c11 0 2024-02-19T19:06:12.718 INFO:tasks.workunit.client.0.smithi088.stdout:0/45: write f5 [566121,116244] 0 2024-02-19T19:06:12.719 INFO:tasks.workunit.client.0.smithi088.stdout:6/60: chown d5/d7/da 158000926 1 2024-02-19T19:06:12.719 INFO:tasks.workunit.client.0.smithi088.stdout:1/75: creat d6/f17 x:0 0 0 2024-02-19T19:06:12.721 INFO:tasks.workunit.client.0.smithi088.stdout:5/110: write d4/fe [1932062,4175] 0 2024-02-19T19:06:12.723 INFO:tasks.workunit.client.0.smithi088.stdout:6/61: mknod d5/de/c12 0 2024-02-19T19:06:12.725 INFO:tasks.workunit.client.0.smithi088.stdout:0/46: getdents d13 0 2024-02-19T19:06:12.727 INFO:tasks.workunit.client.0.smithi088.stdout:6/62: mknod d5/de/c13 0 2024-02-19T19:06:12.729 INFO:tasks.workunit.client.0.smithi088.stdout:0/47: creat d13/f15 x:0 0 0 2024-02-19T19:06:12.732 INFO:tasks.workunit.client.0.smithi088.stdout:1/76: dwrite f5 [4194304,4194304] 0 2024-02-19T19:06:12.734 INFO:tasks.workunit.client.0.smithi088.stdout:1/77: dread - d6/f17 zero size 2024-02-19T19:06:12.739 INFO:tasks.workunit.client.0.smithi088.stdout:5/111: dwrite d4/d7/f15 [0,4194304] 0 2024-02-19T19:06:12.739 INFO:tasks.workunit.client.0.smithi088.stdout:1/78: mknod d6/d10/c18 0 2024-02-19T19:06:12.741 INFO:tasks.workunit.client.0.smithi088.stdout:0/48: getdents d13 0 2024-02-19T19:06:12.744 INFO:tasks.workunit.client.0.smithi088.stdout:5/112: symlink d4/d1a/l25 0 2024-02-19T19:06:12.746 INFO:tasks.workunit.client.0.smithi088.stdout:1/79: dread d6/d10/f15 [0,4194304] 0 2024-02-19T19:06:12.747 INFO:tasks.workunit.client.0.smithi088.stdout:1/80: dread - d6/d10/f11 zero size 2024-02-19T19:06:12.810 INFO:tasks.workunit.client.0.smithi088.stdout:5/113: creat d4/d1a/d22/f26 x:0 0 0 2024-02-19T19:06:12.811 INFO:tasks.workunit.client.0.smithi088.stdout:1/81: creat d6/d10/f19 x:0 0 0 2024-02-19T19:06:12.811 INFO:tasks.workunit.client.0.smithi088.stdout:1/82: chown d6/fd 434 1 2024-02-19T19:06:12.813 INFO:tasks.workunit.client.0.smithi088.stdout:0/49: truncate fb 98826 0 2024-02-19T19:06:12.814 INFO:tasks.workunit.client.0.smithi088.stdout:1/83: creat d6/d10/f1a x:0 0 0 2024-02-19T19:06:12.816 INFO:tasks.workunit.client.0.smithi088.stdout:1/84: creat d6/f1b x:0 0 0 2024-02-19T19:06:12.816 INFO:tasks.workunit.client.0.smithi088.stdout:5/114: link l3 d4/d7/l27 0 2024-02-19T19:06:12.817 INFO:tasks.workunit.client.0.smithi088.stdout:0/50: rename l10 to d13/l16 0 2024-02-19T19:06:12.818 INFO:tasks.workunit.client.0.smithi088.stdout:0/51: dread - fe zero size 2024-02-19T19:06:12.818 INFO:tasks.workunit.client.0.smithi088.stdout:1/85: creat d6/f1c x:0 0 0 2024-02-19T19:06:12.819 INFO:tasks.workunit.client.0.smithi088.stdout:1/86: fdatasync d6/f17 0 2024-02-19T19:06:12.819 INFO:tasks.workunit.client.0.smithi088.stdout:5/115: unlink d4/d1a/c1c 0 2024-02-19T19:06:12.820 INFO:tasks.workunit.client.0.smithi088.stdout:5/116: dread - d4/f16 zero size 2024-02-19T19:06:12.822 INFO:tasks.workunit.client.0.smithi088.stdout:5/117: unlink d4/l20 0 2024-02-19T19:06:12.824 INFO:tasks.workunit.client.0.smithi088.stdout:5/118: dread - d4/d7/f23 zero size 2024-02-19T19:06:12.827 INFO:tasks.workunit.client.0.smithi088.stdout:1/87: dwrite d6/d10/f11 [0,4194304] 0 2024-02-19T19:06:12.828 INFO:tasks.workunit.client.0.smithi088.stdout:1/88: truncate d6/f17 259690 0 2024-02-19T19:06:12.832 INFO:tasks.workunit.client.0.smithi088.stdout:5/119: dread d4/fe [0,4194304] 0 2024-02-19T19:06:12.833 INFO:tasks.workunit.client.0.smithi088.stdout:5/120: chown d4/f8 174660358 1 2024-02-19T19:06:12.835 INFO:tasks.workunit.client.0.smithi088.stdout:5/121: dread - d4/d1a/f1b zero size 2024-02-19T19:06:12.838 INFO:tasks.workunit.client.0.smithi088.stdout:5/122: creat d4/d1a/d22/f28 x:0 0 0 2024-02-19T19:06:12.838 INFO:tasks.workunit.client.0.smithi088.stdout:1/89: dwrite d6/f17 [0,4194304] 0 2024-02-19T19:06:12.839 INFO:tasks.workunit.client.0.smithi088.stdout:1/90: dread - d6/f1b zero size 2024-02-19T19:06:12.839 INFO:tasks.workunit.client.0.smithi088.stdout:5/123: fdatasync d4/d1a/d22/ff 0 2024-02-19T19:06:12.840 INFO:tasks.workunit.client.0.smithi088.stdout:1/91: fsync d6/ff 0 2024-02-19T19:06:12.850 INFO:tasks.workunit.client.0.smithi088.stdout:2/24: sync 2024-02-19T19:06:12.850 INFO:tasks.workunit.client.0.smithi088.stdout:8/29: sync 2024-02-19T19:06:12.851 INFO:tasks.workunit.client.0.smithi088.stdout:1/92: dwrite d6/d10/f16 [0,4194304] 0 2024-02-19T19:06:12.852 INFO:tasks.workunit.client.0.smithi088.stdout:2/25: fsync d3/f4 0 2024-02-19T19:06:12.853 INFO:tasks.workunit.client.0.smithi088.stdout:5/124: dwrite d4/f8 [4194304,4194304] 0 2024-02-19T19:06:12.855 INFO:tasks.workunit.client.0.smithi088.stdout:1/93: dread - d6/d10/f19 zero size 2024-02-19T19:06:12.857 INFO:tasks.workunit.client.0.smithi088.stdout:2/26: creat d3/f6 x:0 0 0 2024-02-19T19:06:12.859 INFO:tasks.workunit.client.0.smithi088.stdout:1/94: creat d6/d10/f1d x:0 0 0 2024-02-19T19:06:12.861 INFO:tasks.workunit.client.0.smithi088.stdout:5/125: creat d4/d1a/d22/f29 x:0 0 0 2024-02-19T19:06:12.863 INFO:tasks.workunit.client.0.smithi088.stdout:1/95: chown d6/lc 99435243 1 2024-02-19T19:06:12.867 INFO:tasks.workunit.client.0.smithi088.stdout:1/96: dread - d6/f1b zero size 2024-02-19T19:06:12.867 INFO:tasks.workunit.client.0.smithi088.stdout:8/30: dwrite f2 [0,4194304] 0 2024-02-19T19:06:12.869 INFO:tasks.workunit.client.0.smithi088.stdout:5/126: unlink d4/d1a/d22/ff 0 2024-02-19T19:06:12.872 INFO:tasks.workunit.client.0.smithi088.stdout:1/97: creat d6/f1e x:0 0 0 2024-02-19T19:06:12.873 INFO:tasks.workunit.client.0.smithi088.stdout:5/127: mknod d4/d1a/c2a 0 2024-02-19T19:06:12.875 INFO:tasks.workunit.client.0.smithi088.stdout:1/98: fdatasync d6/ff 0 2024-02-19T19:06:12.876 INFO:tasks.workunit.client.0.smithi088.stdout:2/27: dwrite d3/f4 [0,4194304] 0 2024-02-19T19:06:12.879 INFO:tasks.workunit.client.0.smithi088.stdout:2/28: read - d3/f6 zero size 2024-02-19T19:06:12.881 INFO:tasks.workunit.client.0.smithi088.stdout:1/99: write d6/ff [3825604,104433] 0 2024-02-19T19:06:12.884 INFO:tasks.workunit.client.0.smithi088.stdout:2/29: mknod d3/c7 0 2024-02-19T19:06:12.885 INFO:tasks.workunit.client.0.smithi088.stdout:8/31: dwrite f2 [0,4194304] 0 2024-02-19T19:06:12.885 INFO:tasks.workunit.client.0.smithi088.stdout:1/100: write d6/f17 [1332489,45454] 0 2024-02-19T19:06:12.888 INFO:tasks.workunit.client.0.smithi088.stdout:1/101: fsync d6/d10/f1d 0 2024-02-19T19:06:12.889 INFO:tasks.workunit.client.0.smithi088.stdout:8/32: rename l3 to l4 0 2024-02-19T19:06:12.891 INFO:tasks.workunit.client.0.smithi088.stdout:5/128: dwrite d4/d1a/f1b [0,4194304] 0 2024-02-19T19:06:12.893 INFO:tasks.workunit.client.0.smithi088.stdout:1/102: rename d6/d10/c18 to d6/c1f 0 2024-02-19T19:06:12.895 INFO:tasks.workunit.client.0.smithi088.stdout:5/129: creat d4/d1a/f2b x:0 0 0 2024-02-19T19:06:12.896 INFO:tasks.workunit.client.0.smithi088.stdout:1/103: mknod d6/d10/c20 0 2024-02-19T19:06:12.899 INFO:tasks.workunit.client.0.smithi088.stdout:8/33: stat l4 0 2024-02-19T19:06:12.900 INFO:tasks.workunit.client.0.smithi088.stdout:5/130: creat d4/d7/f2c x:0 0 0 2024-02-19T19:06:12.900 INFO:tasks.workunit.client.0.smithi088.stdout:1/104: dread d6/ff [0,4194304] 0 2024-02-19T19:06:12.901 INFO:tasks.workunit.client.0.smithi088.stdout:8/34: creat f5 x:0 0 0 2024-02-19T19:06:12.901 INFO:tasks.workunit.client.0.smithi088.stdout:1/105: fdatasync d6/d10/f16 0 2024-02-19T19:06:12.903 INFO:tasks.workunit.client.0.smithi088.stdout:8/35: unlink f2 0 2024-02-19T19:06:12.904 INFO:tasks.workunit.client.0.smithi088.stdout:8/36: dread - f5 zero size 2024-02-19T19:06:12.912 INFO:tasks.workunit.client.0.smithi088.stdout:5/131: dwrite d4/d7/f2c [0,4194304] 0 2024-02-19T19:06:12.913 INFO:tasks.workunit.client.0.smithi088.stdout:8/37: getdents . 0 2024-02-19T19:06:12.913 INFO:tasks.workunit.client.0.smithi088.stdout:1/106: dread d6/d10/f11 [0,4194304] 0 2024-02-19T19:06:12.915 INFO:tasks.workunit.client.0.smithi088.stdout:1/107: write d6/f1c [478964,117996] 0 2024-02-19T19:06:12.916 INFO:tasks.workunit.client.0.smithi088.stdout:8/38: creat f6 x:0 0 0 2024-02-19T19:06:12.917 INFO:tasks.workunit.client.0.smithi088.stdout:1/108: mkdir d6/d21 0 2024-02-19T19:06:12.918 INFO:tasks.workunit.client.0.smithi088.stdout:1/109: truncate d6/d10/f1a 985241 0 2024-02-19T19:06:12.918 INFO:tasks.workunit.client.0.smithi088.stdout:1/110: chown d6/d10 4496 1 2024-02-19T19:06:12.927 INFO:tasks.workunit.client.0.smithi088.stdout:1/111: dwrite d6/fd [4194304,4194304] 0 2024-02-19T19:06:12.929 INFO:tasks.workunit.client.0.smithi088.stdout:1/112: mkdir d6/d10/d22 0 2024-02-19T19:06:12.931 INFO:tasks.workunit.client.0.smithi088.stdout:1/113: creat d6/d10/f23 x:0 0 0 2024-02-19T19:06:12.932 INFO:tasks.workunit.client.0.smithi088.stdout:1/114: write d6/f17 [2008013,61191] 0 2024-02-19T19:06:12.933 INFO:tasks.workunit.client.0.smithi088.stdout:1/115: dread - d6/d10/f1d zero size 2024-02-19T19:06:12.936 INFO:tasks.workunit.client.0.smithi088.stdout:1/116: rename d6/d10/f1a to d6/d10/f24 0 2024-02-19T19:06:12.943 INFO:tasks.workunit.client.0.smithi088.stdout:1/117: dwrite f5 [4194304,4194304] 0 2024-02-19T19:06:12.944 INFO:tasks.workunit.client.0.smithi088.stdout:1/118: write d6/d10/f16 [1100692,106167] 0 2024-02-19T19:06:12.945 INFO:tasks.workunit.client.0.smithi088.stdout:1/119: chown d6/d10 59 1 2024-02-19T19:06:12.954 INFO:tasks.workunit.client.0.smithi088.stdout:1/120: truncate d6/f14 2220649 0 2024-02-19T19:06:12.956 INFO:tasks.workunit.client.0.smithi088.stdout:1/121: dread d6/fb [0,4194304] 0 2024-02-19T19:06:12.957 INFO:tasks.workunit.client.0.smithi088.stdout:1/122: getdents d6/d10/d22 0 2024-02-19T19:06:12.964 INFO:tasks.workunit.client.0.smithi088.stdout:1/123: dwrite d6/f17 [4194304,4194304] 0 2024-02-19T19:06:12.966 INFO:tasks.workunit.client.0.smithi088.stdout:1/124: mknod d6/c25 0 2024-02-19T19:06:12.967 INFO:tasks.workunit.client.0.smithi088.stdout:1/125: chown d6/d10/f11 121011 1 2024-02-19T19:06:12.968 INFO:tasks.workunit.client.0.smithi088.stdout:1/126: write d6/d10/f23 [249295,18017] 0 2024-02-19T19:06:12.979 INFO:tasks.workunit.client.0.smithi088.stdout:1/127: truncate d6/fd 8079039 0 2024-02-19T19:06:12.979 INFO:tasks.workunit.client.0.smithi088.stdout:1/128: chown d6/d10/f11 172568 1 2024-02-19T19:06:12.980 INFO:tasks.workunit.client.0.smithi088.stdout:1/129: stat d6/c7 0 2024-02-19T19:06:12.981 INFO:tasks.workunit.client.0.smithi088.stdout:1/130: mkdir d6/d10/d26 0 2024-02-19T19:06:12.983 INFO:tasks.workunit.client.0.smithi088.stdout:1/131: write d6/ff [4466769,70156] 0 2024-02-19T19:06:12.989 INFO:tasks.workunit.client.0.smithi088.stdout:1/132: dread d6/fd [4194304,4194304] 0 2024-02-19T19:06:12.993 INFO:tasks.workunit.client.0.smithi088.stdout:1/133: dread f5 [0,4194304] 0 2024-02-19T19:06:12.993 INFO:tasks.workunit.client.0.smithi088.stdout:1/134: stat d6/c25 0 2024-02-19T19:06:12.996 INFO:tasks.workunit.client.0.smithi088.stdout:1/135: rmdir d6/d10/d22 0 2024-02-19T19:06:12.998 INFO:tasks.workunit.client.0.smithi088.stdout:1/136: mkdir d6/d10/d27 0 2024-02-19T19:06:12.998 INFO:tasks.workunit.client.0.smithi088.stdout:1/137: read f5 [2487829,46538] 0 2024-02-19T19:06:12.999 INFO:tasks.workunit.client.0.smithi088.stdout:1/138: write d6/f17 [59605,19691] 0 2024-02-19T19:06:13.000 INFO:tasks.workunit.client.0.smithi088.stdout:1/139: write d6/d10/f16 [3575958,29235] 0 2024-02-19T19:06:13.002 INFO:tasks.workunit.client.0.smithi088.stdout:1/140: mkdir d6/d10/d28 0 2024-02-19T19:06:13.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:12 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:13.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:12 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:13.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:12 smithi132 ceph-mon[70676]: Reconfiguring mgr.smithi132.psllsa (monmap changed)... 2024-02-19T19:06:13.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:12 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:06:13.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:12 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T19:06:13.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:12 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:13.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:12 smithi132 ceph-mon[70676]: Reconfiguring daemon mgr.smithi132.psllsa on smithi132 2024-02-19T19:06:13.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:12 smithi132 ceph-mon[70676]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 366 op/s 2024-02-19T19:06:13.008 INFO:tasks.workunit.client.0.smithi088.stdout:1/141: dwrite d6/d10/f15 [0,4194304] 0 2024-02-19T19:06:13.010 INFO:tasks.workunit.client.0.smithi088.stdout:1/142: mkdir d6/d10/d28/d29 0 2024-02-19T19:06:13.013 INFO:tasks.workunit.client.0.smithi088.stdout:1/143: creat d6/d10/d27/f2a x:0 0 0 2024-02-19T19:06:13.017 INFO:tasks.workunit.client.0.smithi088.stdout:1/144: dread d6/d10/f23 [0,4194304] 0 2024-02-19T19:06:13.019 INFO:tasks.workunit.client.0.smithi088.stdout:1/145: symlink d6/d21/l2b 0 2024-02-19T19:06:13.019 INFO:tasks.workunit.client.0.smithi088.stdout:1/146: readlink d6/lc 0 2024-02-19T19:06:13.020 INFO:tasks.workunit.client.0.smithi088.stdout:1/147: write d6/f1e [457934,2320] 0 2024-02-19T19:06:13.022 INFO:tasks.workunit.client.0.smithi088.stdout:1/148: creat d6/d10/d27/f2c x:0 0 0 2024-02-19T19:06:13.027 INFO:tasks.workunit.client.0.smithi088.stdout:1/149: write d6/fb [192137,130896] 0 2024-02-19T19:06:13.027 INFO:tasks.workunit.client.0.smithi088.stdout:1/150: chown d6/ce 33069171 1 2024-02-19T19:06:13.035 INFO:tasks.workunit.client.0.smithi088.stdout:1/151: dwrite d6/d10/f16 [4194304,4194304] 0 2024-02-19T19:06:13.036 INFO:tasks.workunit.client.0.smithi088.stdout:1/152: creat d6/d10/f2d x:0 0 0 2024-02-19T19:06:13.039 INFO:tasks.workunit.client.0.smithi088.stdout:1/153: write f5 [5592331,96427] 0 2024-02-19T19:06:13.046 INFO:tasks.workunit.client.0.smithi088.stdout:3/99: sync 2024-02-19T19:06:13.049 INFO:tasks.workunit.client.0.smithi088.stdout:1/154: creat d6/d10/d28/d29/f2e x:0 0 0 2024-02-19T19:06:13.049 INFO:tasks.workunit.client.0.smithi088.stdout:3/100: symlink d4/d12/l2a 0 2024-02-19T19:06:13.050 INFO:tasks.workunit.client.0.smithi088.stdout:3/101: write d4/d12/f21 [223671,84110] 0 2024-02-19T19:06:13.051 INFO:tasks.workunit.client.0.smithi088.stdout:1/155: creat d6/d21/f2f x:0 0 0 2024-02-19T19:06:13.052 INFO:tasks.workunit.client.0.smithi088.stdout:1/156: dread - d6/d10/d27/f2a zero size 2024-02-19T19:06:13.052 INFO:tasks.workunit.client.0.smithi088.stdout:1/157: chown d6/f1e 15 1 2024-02-19T19:06:13.053 INFO:tasks.workunit.client.0.smithi088.stdout:3/102: creat d4/d22/f2b x:0 0 0 2024-02-19T19:06:13.053 INFO:tasks.workunit.client.0.smithi088.stdout:3/103: readlink d4/l5 0 2024-02-19T19:06:13.055 INFO:tasks.workunit.client.0.smithi088.stdout:3/104: creat d4/d22/f2c x:0 0 0 2024-02-19T19:06:13.058 INFO:tasks.workunit.client.0.smithi088.stdout:1/158: getdents d6/d10/d28/d29 0 2024-02-19T19:06:13.063 INFO:tasks.workunit.client.0.smithi088.stdout:3/105: dwrite d4/d22/f2b [0,4194304] 0 2024-02-19T19:06:13.065 INFO:tasks.workunit.client.0.smithi088.stdout:1/159: getdents d6 0 2024-02-19T19:06:13.067 INFO:tasks.workunit.client.0.smithi088.stdout:3/106: creat d4/f2d x:0 0 0 2024-02-19T19:06:13.069 INFO:tasks.workunit.client.0.smithi088.stdout:3/107: creat d4/d12/f2e x:0 0 0 2024-02-19T19:06:13.070 INFO:tasks.workunit.client.0.smithi088.stdout:3/108: fsync d4/d1a/f24 0 2024-02-19T19:06:13.073 INFO:tasks.workunit.client.0.smithi088.stdout:1/160: dwrite d6/f1b [0,4194304] 0 2024-02-19T19:06:13.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:12 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:13.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:12 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:13.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:12 smithi088 ceph-mon[88643]: Reconfiguring mgr.smithi132.psllsa (monmap changed)... 2024-02-19T19:06:13.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:12 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi132.psllsa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-19T19:06:13.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:12 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-19T19:06:13.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:12 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:13.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:12 smithi088 ceph-mon[88643]: Reconfiguring daemon mgr.smithi132.psllsa on smithi132 2024-02-19T19:06:13.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:12 smithi088 ceph-mon[88643]: pgmap v40: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 366 op/s 2024-02-19T19:06:13.157 INFO:tasks.workunit.client.0.smithi088.stdout:3/109: getdents d4/d22 0 2024-02-19T19:06:13.163 INFO:tasks.workunit.client.0.smithi088.stdout:3/110: link d4/f14 d4/d1a/d1e/f2f 0 2024-02-19T19:06:13.167 INFO:tasks.workunit.client.0.smithi088.stdout:1/161: dwrite d6/f1b [0,4194304] 0 2024-02-19T19:06:13.167 INFO:tasks.workunit.client.0.smithi088.stdout:3/111: unlink d4/c11 0 2024-02-19T19:06:13.169 INFO:tasks.workunit.client.0.smithi088.stdout:3/112: creat d4/d12/f30 x:0 0 0 2024-02-19T19:06:13.170 INFO:tasks.workunit.client.0.smithi088.stdout:1/162: creat d6/d10/d28/d29/f30 x:0 0 0 2024-02-19T19:06:13.172 INFO:tasks.workunit.client.0.smithi088.stdout:1/163: dread d6/fb [0,4194304] 0 2024-02-19T19:06:13.174 INFO:tasks.workunit.client.0.smithi088.stdout:1/164: symlink d6/d10/d27/l31 0 2024-02-19T19:06:13.176 INFO:tasks.workunit.client.0.smithi088.stdout:1/165: symlink d6/d10/d28/l32 0 2024-02-19T19:06:13.179 INFO:tasks.workunit.client.0.smithi088.stdout:1/166: mknod d6/d10/d28/c33 0 2024-02-19T19:06:13.180 INFO:tasks.workunit.client.0.smithi088.stdout:1/167: write d6/d10/f1d [951033,76552] 0 2024-02-19T19:06:13.182 INFO:tasks.workunit.client.0.smithi088.stdout:3/113: rmdir d4 39 2024-02-19T19:06:13.182 INFO:tasks.workunit.client.0.smithi088.stdout:1/168: truncate d6/d10/d27/f2c 369823 0 2024-02-19T19:06:13.184 INFO:tasks.workunit.client.0.smithi088.stdout:3/114: write d4/fd [3068220,65378] 0 2024-02-19T19:06:13.185 INFO:tasks.workunit.client.0.smithi088.stdout:3/115: write d4/d22/f2c [55109,78271] 0 2024-02-19T19:06:13.186 INFO:tasks.workunit.client.0.smithi088.stdout:1/169: rmdir d6/d21 39 2024-02-19T19:06:13.188 INFO:tasks.workunit.client.0.smithi088.stdout:3/116: truncate d4/fc 514582 0 2024-02-19T19:06:13.192 INFO:tasks.workunit.client.0.smithi088.stdout:3/117: mkdir d4/d1a/d1e/d31 0 2024-02-19T19:06:13.195 INFO:tasks.workunit.client.0.smithi088.stdout:1/170: dwrite d6/d10/f23 [0,4194304] 0 2024-02-19T19:06:13.195 INFO:tasks.workunit.client.0.smithi088.stdout:3/118: mkdir d4/d1a/d1e/d31/d32 0 2024-02-19T19:06:13.196 INFO:tasks.workunit.client.0.smithi088.stdout:1/171: write d6/f1b [3714275,18642] 0 2024-02-19T19:06:13.196 INFO:tasks.workunit.client.0.smithi088.stdout:3/119: write d4/f2d [519812,11655] 0 2024-02-19T19:06:13.199 INFO:tasks.workunit.client.0.smithi088.stdout:1/172: symlink d6/l34 0 2024-02-19T19:06:13.203 INFO:tasks.workunit.client.0.smithi088.stdout:1/173: unlink d6/d10/c20 0 2024-02-19T19:06:13.205 INFO:tasks.workunit.client.0.smithi088.stdout:1/174: creat d6/d10/d27/f35 x:0 0 0 2024-02-19T19:06:13.210 INFO:tasks.workunit.client.0.smithi088.stdout:3/120: dwrite d4/f14 [0,4194304] 0 2024-02-19T19:06:13.210 INFO:tasks.workunit.client.0.smithi088.stdout:1/175: rmdir d6/d10 39 2024-02-19T19:06:13.211 INFO:tasks.workunit.client.0.smithi088.stdout:3/121: fsync d4/d1a/f24 0 2024-02-19T19:06:13.211 INFO:tasks.workunit.client.0.smithi088.stdout:3/122: readlink d4/d1a/l29 0 2024-02-19T19:06:13.212 INFO:tasks.workunit.client.0.smithi088.stdout:3/123: truncate d4/ff 5190927 0 2024-02-19T19:06:13.215 INFO:tasks.workunit.client.0.smithi088.stdout:3/124: dread d4/d1a/d1e/f2f [4194304,4194304] 0 2024-02-19T19:06:13.218 INFO:tasks.workunit.client.0.smithi088.stdout:7/25: sync 2024-02-19T19:06:13.220 INFO:tasks.workunit.client.0.smithi088.stdout:3/125: mkdir d4/d12/d33 0 2024-02-19T19:06:13.223 INFO:tasks.workunit.client.0.smithi088.stdout:7/26: getdents . 0 2024-02-19T19:06:13.224 INFO:tasks.workunit.client.0.smithi088.stdout:7/27: fsync f2 0 2024-02-19T19:06:13.224 INFO:tasks.workunit.client.0.smithi088.stdout:7/28: fsync f8 0 2024-02-19T19:06:13.225 INFO:tasks.workunit.client.0.smithi088.stdout:3/126: rmdir d4/d12 39 2024-02-19T19:06:13.226 INFO:tasks.workunit.client.0.smithi088.stdout:7/29: fsync f6 0 2024-02-19T19:06:13.227 INFO:tasks.workunit.client.0.smithi088.stdout:7/30: read f2 [434707,74037] 0 2024-02-19T19:06:13.228 INFO:tasks.workunit.client.0.smithi088.stdout:3/127: rename d4/c1c to d4/d1a/d1e/d31/d32/c34 0 2024-02-19T19:06:13.229 INFO:tasks.workunit.client.0.smithi088.stdout:7/31: creat f9 x:0 0 0 2024-02-19T19:06:13.230 INFO:tasks.workunit.client.0.smithi088.stdout:3/128: mknod d4/d1a/d1e/d31/c35 0 2024-02-19T19:06:13.231 INFO:tasks.workunit.client.0.smithi088.stdout:7/32: symlink la 0 2024-02-19T19:06:13.231 INFO:tasks.workunit.client.0.smithi088.stdout:3/129: dread - d4/f28 zero size 2024-02-19T19:06:13.233 INFO:tasks.workunit.client.0.smithi088.stdout:3/130: symlink d4/d1a/d1e/d31/l36 0 2024-02-19T19:06:13.235 INFO:tasks.workunit.client.0.smithi088.stdout:3/131: symlink d4/d1a/d1e/l37 0 2024-02-19T19:06:13.236 INFO:tasks.workunit.client.0.smithi088.stdout:3/132: readlink d4/d1a/d1e/d31/l36 0 2024-02-19T19:06:13.240 INFO:tasks.workunit.client.0.smithi088.stdout:7/33: getdents . 0 2024-02-19T19:06:13.242 INFO:tasks.workunit.client.0.smithi088.stdout:7/34: creat fb x:0 0 0 2024-02-19T19:06:13.245 INFO:tasks.workunit.client.0.smithi088.stdout:7/35: creat fc x:0 0 0 2024-02-19T19:06:13.246 INFO:tasks.workunit.client.0.smithi088.stdout:3/133: dwrite d4/f7 [0,4194304] 0 2024-02-19T19:06:13.250 INFO:tasks.workunit.client.0.smithi088.stdout:7/36: link f6 fd 0 2024-02-19T19:06:13.250 INFO:tasks.workunit.client.0.smithi088.stdout:7/37: rmdir - no directory 2024-02-19T19:06:13.250 INFO:tasks.workunit.client.0.smithi088.stdout:7/38: dread - f8 zero size 2024-02-19T19:06:13.250 INFO:tasks.workunit.client.0.smithi088.stdout:7/39: rmdir - no directory 2024-02-19T19:06:13.250 INFO:tasks.workunit.client.0.smithi088.stdout:3/134: rmdir d4/d12 39 2024-02-19T19:06:13.255 INFO:tasks.workunit.client.0.smithi088.stdout:3/135: creat d4/f38 x:0 0 0 2024-02-19T19:06:13.256 INFO:tasks.workunit.client.0.smithi088.stdout:3/136: write d4/f38 [330686,111367] 0 2024-02-19T19:06:13.257 INFO:tasks.workunit.client.0.smithi088.stdout:3/137: fsync d4/f2d 0 2024-02-19T19:06:13.258 INFO:tasks.workunit.client.0.smithi088.stdout:7/40: write fd [495488,35021] 0 2024-02-19T19:06:13.258 INFO:tasks.workunit.client.0.smithi088.stdout:7/41: dread - fc zero size 2024-02-19T19:06:13.259 INFO:tasks.workunit.client.0.smithi088.stdout:7/42: chown f9 235410 1 2024-02-19T19:06:13.259 INFO:tasks.workunit.client.0.smithi088.stdout:7/43: chown c7 2 1 2024-02-19T19:06:13.260 INFO:tasks.workunit.client.0.smithi088.stdout:3/138: chown d4/d12/c20 7 1 2024-02-19T19:06:13.260 INFO:tasks.workunit.client.0.smithi088.stdout:7/44: write f2 [2189035,62311] 0 2024-02-19T19:06:13.263 INFO:tasks.workunit.client.0.smithi088.stdout:7/45: creat fe x:0 0 0 2024-02-19T19:06:13.263 INFO:tasks.workunit.client.0.smithi088.stdout:3/139: fdatasync d4/f38 0 2024-02-19T19:06:13.265 INFO:tasks.workunit.client.0.smithi088.stdout:7/46: link c5 cf 0 2024-02-19T19:06:13.267 INFO:tasks.workunit.client.0.smithi088.stdout:7/47: unlink la 0 2024-02-19T19:06:13.269 INFO:tasks.workunit.client.0.smithi088.stdout:7/48: rename f9 to f10 0 2024-02-19T19:06:13.271 INFO:tasks.workunit.client.0.smithi088.stdout:7/49: rename l4 to l11 0 2024-02-19T19:06:13.272 INFO:tasks.workunit.client.0.smithi088.stdout:3/140: truncate d4/ff 757242 0 2024-02-19T19:06:13.273 INFO:tasks.workunit.client.0.smithi088.stdout:7/50: creat f12 x:0 0 0 2024-02-19T19:06:13.274 INFO:tasks.workunit.client.0.smithi088.stdout:7/51: chown cf 80387 1 2024-02-19T19:06:13.275 INFO:tasks.workunit.client.0.smithi088.stdout:3/141: rename d4/d12/l1f to d4/d1a/d1e/d31/l39 0 2024-02-19T19:06:13.276 INFO:tasks.workunit.client.0.smithi088.stdout:3/142: chown d4/d1a/d23/c25 10732 1 2024-02-19T19:06:13.276 INFO:tasks.workunit.client.0.smithi088.stdout:3/143: readlink d4/d1a/l29 0 2024-02-19T19:06:13.277 INFO:tasks.workunit.client.0.smithi088.stdout:7/52: link f2 f13 0 2024-02-19T19:06:13.278 INFO:tasks.workunit.client.0.smithi088.stdout:7/53: mkdir d14 0 2024-02-19T19:06:13.279 INFO:tasks.workunit.client.0.smithi088.stdout:3/144: write d4/d12/f1d [891512,43718] 0 2024-02-19T19:06:13.280 INFO:tasks.workunit.client.0.smithi088.stdout:7/54: creat d14/f15 x:0 0 0 2024-02-19T19:06:13.281 INFO:tasks.workunit.client.0.smithi088.stdout:3/145: mknod d4/d1a/d1e/d31/d32/c3a 0 2024-02-19T19:06:13.285 INFO:tasks.workunit.client.0.smithi088.stdout:3/146: link d4/d1a/d1e/d31/d32/c3a d4/d1a/d1e/d31/c3b 0 2024-02-19T19:06:13.285 INFO:tasks.workunit.client.0.smithi088.stdout:3/147: read d4/f16 [729449,72060] 0 2024-02-19T19:06:13.286 INFO:tasks.workunit.client.0.smithi088.stdout:3/148: chown d4/d22/f2c 61382 1 2024-02-19T19:06:13.290 INFO:tasks.workunit.client.0.smithi088.stdout:3/149: mkdir d4/d3c 0 2024-02-19T19:06:13.293 INFO:tasks.workunit.client.0.smithi088.stdout:3/150: symlink d4/d1a/d1e/d31/d32/l3d 0 2024-02-19T19:06:13.294 INFO:tasks.workunit.client.0.smithi088.stdout:7/55: truncate f6 39578 0 2024-02-19T19:06:13.298 INFO:tasks.workunit.client.0.smithi088.stdout:3/151: rmdir d4/d3c 0 2024-02-19T19:06:13.300 INFO:tasks.workunit.client.0.smithi088.stdout:3/152: mknod d4/d12/d33/c3e 0 2024-02-19T19:06:13.301 INFO:tasks.workunit.client.0.smithi088.stdout:3/153: read d4/f16 [561283,91064] 0 2024-02-19T19:06:13.301 INFO:tasks.workunit.client.0.smithi088.stdout:3/154: chown d4 921212 1 2024-02-19T19:06:13.303 INFO:tasks.workunit.client.0.smithi088.stdout:3/155: write d4/d12/f27 [759532,17617] 0 2024-02-19T19:06:13.303 INFO:tasks.workunit.client.0.smithi088.stdout:7/56: truncate f6 621639 0 2024-02-19T19:06:13.306 INFO:tasks.workunit.client.0.smithi088.stdout:7/57: rename f12 to d14/f16 0 2024-02-19T19:06:13.307 INFO:tasks.workunit.client.0.smithi088.stdout:7/58: write fe [471111,100174] 0 2024-02-19T19:06:13.307 INFO:tasks.workunit.client.0.smithi088.stdout:3/156: mknod d4/d1a/d1e/d31/d32/c3f 0 2024-02-19T19:06:13.309 INFO:tasks.workunit.client.0.smithi088.stdout:7/59: creat d14/f17 x:0 0 0 2024-02-19T19:06:13.310 INFO:tasks.workunit.client.0.smithi088.stdout:3/157: fdatasync d4/ff 0 2024-02-19T19:06:13.312 INFO:tasks.workunit.client.0.smithi088.stdout:7/60: dread fe [0,4194304] 0 2024-02-19T19:06:13.313 INFO:tasks.workunit.client.0.smithi088.stdout:7/61: chown f8 588372 1 2024-02-19T19:06:13.313 INFO:tasks.workunit.client.0.smithi088.stdout:3/158: creat d4/f40 x:0 0 0 2024-02-19T19:06:13.314 INFO:tasks.workunit.client.0.smithi088.stdout:3/159: chown d4/f40 1984550731 1 2024-02-19T19:06:13.314 INFO:tasks.workunit.client.0.smithi088.stdout:7/62: readlink l11 0 2024-02-19T19:06:13.315 INFO:tasks.workunit.client.0.smithi088.stdout:3/160: dread d4/f16 [0,4194304] 0 2024-02-19T19:06:13.317 INFO:tasks.workunit.client.0.smithi088.stdout:7/63: rmdir d14 39 2024-02-19T19:06:13.319 INFO:tasks.workunit.client.0.smithi088.stdout:7/64: fsync f13 0 2024-02-19T19:06:13.415 INFO:tasks.workunit.client.0.smithi088.stdout:6/63: sync 2024-02-19T19:06:13.416 INFO:tasks.workunit.client.0.smithi088.stdout:4/17: sync 2024-02-19T19:06:13.416 INFO:tasks.workunit.client.0.smithi088.stdout:6/64: write d5/ff [1035117,28564] 0 2024-02-19T19:06:13.417 INFO:tasks.workunit.client.0.smithi088.stdout:4/18: write f0 [2233637,2357] 0 2024-02-19T19:06:13.418 INFO:tasks.workunit.client.0.smithi088.stdout:6/65: dread - d5/de/f10 zero size 2024-02-19T19:06:13.418 INFO:tasks.workunit.client.0.smithi088.stdout:6/66: chown d5/d7 27212459 1 2024-02-19T19:06:13.418 INFO:tasks.workunit.client.0.smithi088.stdout:4/19: dread - d2/f6 zero size 2024-02-19T19:06:13.419 INFO:tasks.workunit.client.0.smithi088.stdout:6/67: chown d5/d7 1913883 1 2024-02-19T19:06:13.420 INFO:tasks.workunit.client.0.smithi088.stdout:6/68: write d5/de/f10 [419579,84500] 0 2024-02-19T19:06:13.427 INFO:tasks.workunit.client.0.smithi088.stdout:6/69: dwrite d5/ff [0,4194304] 0 2024-02-19T19:06:13.525 INFO:tasks.workunit.client.0.smithi088.stdout:4/20: creat d2/f7 x:0 0 0 2024-02-19T19:06:13.526 INFO:tasks.workunit.client.0.smithi088.stdout:6/70: creat d5/d7/f14 x:0 0 0 2024-02-19T19:06:13.526 INFO:tasks.workunit.client.0.smithi088.stdout:3/161: link d4/d12/c13 d4/c41 0 2024-02-19T19:06:13.526 INFO:tasks.workunit.client.0.smithi088.stdout:6/71: fsync d5/ff 0 2024-02-19T19:06:13.526 INFO:tasks.workunit.client.0.smithi088.stdout:3/162: fdatasync d4/fc 0 2024-02-19T19:06:13.527 INFO:tasks.workunit.client.0.smithi088.stdout:7/65: mknod d14/c18 0 2024-02-19T19:06:13.527 INFO:tasks.workunit.client.0.smithi088.stdout:3/163: write d4/f6 [1161207,70526] 0 2024-02-19T19:06:13.528 INFO:tasks.workunit.client.0.smithi088.stdout:4/21: mkdir d2/d8 0 2024-02-19T19:06:13.531 INFO:tasks.workunit.client.0.smithi088.stdout:7/66: creat d14/f19 x:0 0 0 2024-02-19T19:06:13.532 INFO:tasks.workunit.client.0.smithi088.stdout:3/164: mknod d4/d22/c42 0 2024-02-19T19:06:13.533 INFO:tasks.workunit.client.0.smithi088.stdout:7/67: fdatasync fe 0 2024-02-19T19:06:13.534 INFO:tasks.workunit.client.0.smithi088.stdout:4/22: mkdir d2/d5/d9 0 2024-02-19T19:06:13.534 INFO:tasks.workunit.client.0.smithi088.stdout:3/165: read d4/f14 [2242822,81554] 0 2024-02-19T19:06:13.535 INFO:tasks.workunit.client.0.smithi088.stdout:6/72: dwrite d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:13.537 INFO:tasks.workunit.client.0.smithi088.stdout:3/166: rmdir d4/d1a/d23 39 2024-02-19T19:06:13.538 INFO:tasks.workunit.client.0.smithi088.stdout:4/23: symlink d2/d5/d9/la 0 2024-02-19T19:06:13.540 INFO:tasks.workunit.client.0.smithi088.stdout:4/24: readlink d2/d5/d9/la 0 2024-02-19T19:06:13.543 INFO:tasks.workunit.client.0.smithi088.stdout:7/68: dwrite fc [0,4194304] 0 2024-02-19T19:06:13.548 INFO:tasks.workunit.client.0.smithi088.stdout:9/87: sync 2024-02-19T19:06:13.558 INFO:tasks.workunit.client.0.smithi088.stdout:4/25: dwrite f0 [0,4194304] 0 2024-02-19T19:06:13.558 INFO:tasks.workunit.client.0.smithi088.stdout:3/167: dwrite d4/d12/f21 [0,4194304] 0 2024-02-19T19:06:13.562 INFO:tasks.workunit.client.0.smithi088.stdout:7/69: dwrite d14/f17 [0,4194304] 0 2024-02-19T19:06:13.563 INFO:tasks.workunit.client.0.smithi088.stdout:3/168: chown d4/l5 2678 1 2024-02-19T19:06:13.567 INFO:tasks.workunit.client.0.smithi088.stdout:3/169: write d4/d12/f1d [1353309,118845] 0 2024-02-19T19:06:13.570 INFO:tasks.workunit.client.0.smithi088.stdout:7/70: mknod d14/c1a 0 2024-02-19T19:06:13.571 INFO:tasks.workunit.client.0.smithi088.stdout:9/88: dwrite d1/d13/f12 [0,4194304] 0 2024-02-19T19:06:13.576 INFO:tasks.workunit.client.0.smithi088.stdout:4/26: dwrite d2/f7 [0,4194304] 0 2024-02-19T19:06:13.579 INFO:tasks.workunit.client.0.smithi088.stdout:4/27: chown f0 16410849 1 2024-02-19T19:06:13.579 INFO:tasks.workunit.client.0.smithi088.stdout:9/89: mknod d1/d13/c1b 0 2024-02-19T19:06:13.583 INFO:tasks.workunit.client.0.smithi088.stdout:3/170: dwrite d4/d22/f2c [0,4194304] 0 2024-02-19T19:06:13.584 INFO:tasks.workunit.client.0.smithi088.stdout:9/90: symlink d1/d13/l1c 0 2024-02-19T19:06:13.590 INFO:tasks.workunit.client.0.smithi088.stdout:9/91: symlink d1/d13/l1d 0 2024-02-19T19:06:13.593 INFO:tasks.workunit.client.0.smithi088.stdout:0/52: sync 2024-02-19T19:06:13.593 INFO:tasks.workunit.client.0.smithi088.stdout:4/28: dwrite d2/f4 [0,4194304] 0 2024-02-19T19:06:13.596 INFO:tasks.workunit.client.0.smithi088.stdout:3/171: dwrite d4/d22/f2b [0,4194304] 0 2024-02-19T19:06:13.599 INFO:tasks.workunit.client.0.smithi088.stdout:3/172: readlink d4/d1a/d1e/l37 0 2024-02-19T19:06:13.606 INFO:tasks.workunit.client.0.smithi088.stdout:0/53: dwrite f7 [0,4194304] 0 2024-02-19T19:06:13.609 INFO:tasks.workunit.client.0.smithi088.stdout:9/92: dwrite d1/d8/d14/f16 [0,4194304] 0 2024-02-19T19:06:13.612 INFO:tasks.workunit.client.0.smithi088.stdout:0/54: symlink d13/l17 0 2024-02-19T19:06:13.614 INFO:tasks.workunit.client.0.smithi088.stdout:9/93: chown d1/d8/d14/f16 1726313108 1 2024-02-19T19:06:13.615 INFO:tasks.workunit.client.0.smithi088.stdout:3/173: dwrite d4/f16 [0,4194304] 0 2024-02-19T19:06:13.616 INFO:tasks.workunit.client.0.smithi088.stdout:0/55: dread - d13/f15 zero size 2024-02-19T19:06:13.616 INFO:tasks.workunit.client.0.smithi088.stdout:0/56: chown c11 2 1 2024-02-19T19:06:13.618 INFO:tasks.workunit.client.0.smithi088.stdout:0/57: write fe [332644,14096] 0 2024-02-19T19:06:13.619 INFO:tasks.workunit.client.0.smithi088.stdout:3/174: mkdir d4/d12/d33/d43 0 2024-02-19T19:06:13.620 INFO:tasks.workunit.client.0.smithi088.stdout:9/94: fsync d1/fc 0 2024-02-19T19:06:13.621 INFO:tasks.workunit.client.0.smithi088.stdout:0/58: symlink d13/l18 0 2024-02-19T19:06:13.627 INFO:tasks.workunit.client.0.smithi088.stdout:0/59: dread f7 [0,4194304] 0 2024-02-19T19:06:13.627 INFO:tasks.workunit.client.0.smithi088.stdout:9/95: mknod d1/c1e 0 2024-02-19T19:06:13.628 INFO:tasks.workunit.client.0.smithi088.stdout:9/96: fdatasync d1/f10 0 2024-02-19T19:06:13.628 INFO:tasks.workunit.client.0.smithi088.stdout:0/60: read f3 [635534,42025] 0 2024-02-19T19:06:13.629 INFO:tasks.workunit.client.0.smithi088.stdout:3/175: rename d4/d1a/d23/c25 to d4/d22/c44 0 2024-02-19T19:06:13.632 INFO:tasks.workunit.client.0.smithi088.stdout:0/61: mknod d13/c19 0 2024-02-19T19:06:13.632 INFO:tasks.workunit.client.0.smithi088.stdout:9/97: dread d1/d13/fd [0,4194304] 0 2024-02-19T19:06:13.633 INFO:tasks.workunit.client.0.smithi088.stdout:3/176: stat d4/d1a/d1e/d31/d32/c3a 0 2024-02-19T19:06:13.635 INFO:tasks.workunit.client.0.smithi088.stdout:0/62: creat d13/f1a x:0 0 0 2024-02-19T19:06:13.639 INFO:tasks.workunit.client.0.smithi088.stdout:3/177: symlink d4/d1a/l45 0 2024-02-19T19:06:13.646 INFO:tasks.workunit.client.0.smithi088.stdout:5/132: sync 2024-02-19T19:06:13.646 INFO:tasks.workunit.client.0.smithi088.stdout:1/176: sync 2024-02-19T19:06:13.646 INFO:tasks.workunit.client.0.smithi088.stdout:2/30: sync 2024-02-19T19:06:13.646 INFO:tasks.workunit.client.0.smithi088.stdout:8/39: sync 2024-02-19T19:06:13.646 INFO:tasks.workunit.client.0.smithi088.stdout:8/40: rmdir - no directory 2024-02-19T19:06:13.646 INFO:tasks.workunit.client.0.smithi088.stdout:3/178: dread d4/d12/f27 [0,4194304] 0 2024-02-19T19:06:13.646 INFO:tasks.workunit.client.0.smithi088.stdout:0/63: dwrite f7 [0,4194304] 0 2024-02-19T19:06:13.651 INFO:tasks.workunit.client.0.smithi088.stdout:2/31: rename d3/c7 to d3/c8 0 2024-02-19T19:06:13.652 INFO:tasks.workunit.client.0.smithi088.stdout:2/32: dread - d3/f6 zero size 2024-02-19T19:06:13.652 INFO:tasks.workunit.client.0.smithi088.stdout:4/29: fdatasync d2/f4 0 2024-02-19T19:06:13.652 INFO:tasks.workunit.client.0.smithi088.stdout:3/179: fsync d4/d22/f2b 0 2024-02-19T19:06:13.659 INFO:tasks.workunit.client.0.smithi088.stdout:5/133: mknod d4/d1a/d22/c2d 0 2024-02-19T19:06:13.660 INFO:tasks.workunit.client.0.smithi088.stdout:5/134: dread - d4/d1a/d22/f26 zero size 2024-02-19T19:06:13.663 INFO:tasks.workunit.client.0.smithi088.stdout:3/180: dread d4/f7 [0,4194304] 0 2024-02-19T19:06:13.664 INFO:tasks.workunit.client.0.smithi088.stdout:2/33: rename l2 to d3/l9 0 2024-02-19T19:06:13.666 INFO:tasks.workunit.client.0.smithi088.stdout:5/135: dread f0 [0,4194304] 0 2024-02-19T19:06:13.667 INFO:tasks.workunit.client.0.smithi088.stdout:2/34: creat d3/fa x:0 0 0 2024-02-19T19:06:13.667 INFO:tasks.workunit.client.0.smithi088.stdout:3/181: dread d4/f38 [0,4194304] 0 2024-02-19T19:06:13.667 INFO:tasks.workunit.client.0.smithi088.stdout:3/182: stat d4/d1a 0 2024-02-19T19:06:13.668 INFO:tasks.workunit.client.0.smithi088.stdout:1/177: rename d6/d10/d27/f2a to d6/d21/f36 0 2024-02-19T19:06:13.668 INFO:tasks.workunit.client.0.smithi088.stdout:1/178: chown d6/la 755402 1 2024-02-19T19:06:13.672 INFO:tasks.workunit.client.0.smithi088.stdout:2/35: creat d3/fb x:0 0 0 2024-02-19T19:06:13.673 INFO:tasks.workunit.client.0.smithi088.stdout:3/183: symlink d4/d1a/d1e/d31/d32/l46 0 2024-02-19T19:06:13.676 INFO:tasks.workunit.client.0.smithi088.stdout:2/36: mknod d3/cc 0 2024-02-19T19:06:13.678 INFO:tasks.workunit.client.0.smithi088.stdout:1/179: link d6/d10/f23 d6/d10/d28/d29/f37 0 2024-02-19T19:06:13.680 INFO:tasks.workunit.client.0.smithi088.stdout:3/184: dwrite d4/f40 [0,4194304] 0 2024-02-19T19:06:13.686 INFO:tasks.workunit.client.0.smithi088.stdout:3/185: creat d4/d12/f47 x:0 0 0 2024-02-19T19:06:13.686 INFO:tasks.workunit.client.0.smithi088.stdout:3/186: fdatasync d4/fd 0 2024-02-19T19:06:13.691 INFO:tasks.workunit.client.0.smithi088.stdout:3/187: rename d4/d1a/d1e/d31/l39 to d4/d1a/d23/l48 0 2024-02-19T19:06:13.718 INFO:tasks.workunit.client.0.smithi088.stdout:4/30: fsync d2/f7 0 2024-02-19T19:06:13.719 INFO:tasks.workunit.client.0.smithi088.stdout:4/31: fsync d2/f6 0 2024-02-19T19:06:13.722 INFO:tasks.workunit.client.0.smithi088.stdout:4/32: dread d2/f4 [0,4194304] 0 2024-02-19T19:06:13.723 INFO:tasks.workunit.client.0.smithi088.stdout:6/73: truncate d5/d7/f14 2473585 0 2024-02-19T19:06:13.723 INFO:tasks.workunit.client.0.smithi088.stdout:6/74: stat d5/d7 0 2024-02-19T19:06:13.759 INFO:tasks.workunit.client.0.smithi088.stdout:4/33: dread d2/f4 [0,4194304] 0 2024-02-19T19:06:13.760 INFO:tasks.workunit.client.0.smithi088.stdout:9/98: truncate d1/f10 363891 0 2024-02-19T19:06:13.764 INFO:tasks.workunit.client.0.smithi088.stdout:0/64: truncate f7 3326313 0 2024-02-19T19:06:13.764 INFO:tasks.workunit.client.0.smithi088.stdout:9/99: write d1/d8/d14/d18/f1a [94561,46338] 0 2024-02-19T19:06:13.766 INFO:tasks.workunit.client.0.smithi088.stdout:5/136: truncate d4/f12 4731463 0 2024-02-19T19:06:13.766 INFO:tasks.workunit.client.0.smithi088.stdout:6/75: dwrite d5/de/f10 [0,4194304] 0 2024-02-19T19:06:13.767 INFO:tasks.workunit.client.0.smithi088.stdout:7/71: dwrite fd [0,4194304] 0 2024-02-19T19:06:13.771 INFO:tasks.workunit.client.0.smithi088.stdout:2/37: rmdir d3 39 2024-02-19T19:06:13.774 INFO:tasks.workunit.client.0.smithi088.stdout:1/180: truncate d6/f1e 119779 0 2024-02-19T19:06:13.776 INFO:tasks.workunit.client.0.smithi088.stdout:7/72: dread d14/f17 [0,4194304] 0 2024-02-19T19:06:13.776 INFO:tasks.workunit.client.0.smithi088.stdout:3/188: rmdir d4 39 2024-02-19T19:06:13.776 INFO:tasks.workunit.client.0.smithi088.stdout:4/34: creat d2/d5/fb x:0 0 0 2024-02-19T19:06:13.779 INFO:tasks.workunit.client.0.smithi088.stdout:4/35: dread - d2/d5/fb zero size 2024-02-19T19:06:13.783 INFO:tasks.workunit.client.0.smithi088.stdout:9/100: mkdir d1/d8/d14/d18/d1f 0 2024-02-19T19:06:13.783 INFO:tasks.workunit.client.0.smithi088.stdout:4/36: readlink d2/d5/d9/la 0 2024-02-19T19:06:13.783 INFO:tasks.workunit.client.0.smithi088.stdout:6/76: creat d5/d7/f15 x:0 0 0 2024-02-19T19:06:13.784 INFO:tasks.workunit.client.0.smithi088.stdout:5/137: dwrite d4/d1a/d22/f28 [0,4194304] 0 2024-02-19T19:06:13.784 INFO:tasks.workunit.client.0.smithi088.stdout:9/101: stat d1/l15 0 2024-02-19T19:06:13.785 INFO:tasks.workunit.client.0.smithi088.stdout:4/37: fsync d2/d5/fb 0 2024-02-19T19:06:13.787 INFO:tasks.workunit.client.0.smithi088.stdout:6/77: mknod d5/c16 0 2024-02-19T19:06:13.791 INFO:tasks.workunit.client.0.smithi088.stdout:6/78: write d5/ff [1588395,9823] 0 2024-02-19T19:06:13.792 INFO:tasks.workunit.client.0.smithi088.stdout:5/138: truncate d4/d1a/f1b 4675232 0 2024-02-19T19:06:13.792 INFO:tasks.workunit.client.0.smithi088.stdout:3/189: dread - d4/d12/f2e zero size 2024-02-19T19:06:13.793 INFO:tasks.workunit.client.0.smithi088.stdout:4/38: creat d2/d5/fc x:0 0 0 2024-02-19T19:06:13.794 INFO:tasks.workunit.client.0.smithi088.stdout:9/102: creat d1/f20 x:0 0 0 2024-02-19T19:06:13.794 INFO:tasks.workunit.client.0.smithi088.stdout:1/181: dwrite d6/d10/d28/d29/f30 [0,4194304] 0 2024-02-19T19:06:13.797 INFO:tasks.workunit.client.0.smithi088.stdout:6/79: fdatasync d5/d7/f15 0 2024-02-19T19:06:13.799 INFO:tasks.workunit.client.0.smithi088.stdout:5/139: chown d4/d7/c17 1911 1 2024-02-19T19:06:13.802 INFO:tasks.workunit.client.0.smithi088.stdout:9/103: rmdir d1/d13 39 2024-02-19T19:06:13.805 INFO:tasks.workunit.client.0.smithi088.stdout:5/140: chown d4/d7/f2c 4 1 2024-02-19T19:06:13.810 INFO:tasks.workunit.client.0.smithi088.stdout:7/73: dwrite d14/f16 [0,4194304] 0 2024-02-19T19:06:13.813 INFO:tasks.workunit.client.0.smithi088.stdout:2/38: dwrite d3/fa [0,4194304] 0 2024-02-19T19:06:13.814 INFO:tasks.workunit.client.0.smithi088.stdout:1/182: creat d6/d10/f38 x:0 0 0 2024-02-19T19:06:13.816 INFO:tasks.workunit.client.0.smithi088.stdout:3/190: dwrite d4/f40 [0,4194304] 0 2024-02-19T19:06:13.817 INFO:tasks.workunit.client.0.smithi088.stdout:5/141: creat d4/f2e x:0 0 0 2024-02-19T19:06:13.818 INFO:tasks.workunit.client.0.smithi088.stdout:1/183: read - d6/d10/f38 zero size 2024-02-19T19:06:13.818 INFO:tasks.workunit.client.0.smithi088.stdout:7/74: symlink d14/l1b 0 2024-02-19T19:06:13.821 INFO:tasks.workunit.client.0.smithi088.stdout:4/39: dwrite d2/f4 [0,4194304] 0 2024-02-19T19:06:13.823 INFO:tasks.workunit.client.0.smithi088.stdout:1/184: chown d6/c7 37753 1 2024-02-19T19:06:13.823 INFO:tasks.workunit.client.0.smithi088.stdout:2/39: symlink d3/ld 0 2024-02-19T19:06:13.824 INFO:tasks.workunit.client.0.smithi088.stdout:7/75: write fb [786772,106439] 0 2024-02-19T19:06:13.824 INFO:tasks.workunit.client.0.smithi088.stdout:3/191: truncate d4/d1a/d1e/f2f 9336138 0 2024-02-19T19:06:13.824 INFO:tasks.workunit.client.0.smithi088.stdout:9/104: mkdir d1/d21 0 2024-02-19T19:06:13.825 INFO:tasks.workunit.client.0.smithi088.stdout:6/80: rmdir d5/d7/da 0 2024-02-19T19:06:13.825 INFO:tasks.workunit.client.0.smithi088.stdout:1/185: truncate d6/d21/f2f 316534 0 2024-02-19T19:06:13.826 INFO:tasks.workunit.client.0.smithi088.stdout:5/142: rename d4/f16 to d4/d7/f2f 0 2024-02-19T19:06:13.826 INFO:tasks.workunit.client.0.smithi088.stdout:1/186: readlink d6/la 0 2024-02-19T19:06:13.827 INFO:tasks.workunit.client.0.smithi088.stdout:4/40: creat d2/fd x:0 0 0 2024-02-19T19:06:13.831 INFO:tasks.workunit.client.0.smithi088.stdout:5/143: read - d4/d1a/d22/f26 zero size 2024-02-19T19:06:13.832 INFO:tasks.workunit.client.0.smithi088.stdout:9/105: symlink d1/d8/l22 0 2024-02-19T19:06:13.837 INFO:tasks.workunit.client.0.smithi088.stdout:5/144: mknod d4/d7/c30 0 2024-02-19T19:06:13.837 INFO:tasks.workunit.client.0.smithi088.stdout:7/76: dread f6 [0,4194304] 0 2024-02-19T19:06:13.842 INFO:tasks.workunit.client.0.smithi088.stdout:2/40: dwrite d3/f6 [0,4194304] 0 2024-02-19T19:06:13.842 INFO:tasks.workunit.client.0.smithi088.stdout:6/81: dread d5/ff [0,4194304] 0 2024-02-19T19:06:13.843 INFO:tasks.workunit.client.0.smithi088.stdout:3/192: dread d4/fd [0,4194304] 0 2024-02-19T19:06:13.845 INFO:tasks.workunit.client.0.smithi088.stdout:7/77: rmdir d14 39 2024-02-19T19:06:13.846 INFO:tasks.workunit.client.0.smithi088.stdout:4/41: dread d2/f4 [0,4194304] 0 2024-02-19T19:06:13.847 INFO:tasks.workunit.client.0.smithi088.stdout:5/145: creat d4/d7/f31 x:0 0 0 2024-02-19T19:06:13.847 INFO:tasks.workunit.client.0.smithi088.stdout:6/82: dread - d5/d7/f15 zero size 2024-02-19T19:06:13.848 INFO:tasks.workunit.client.0.smithi088.stdout:1/187: dwrite d6/d10/f15 [0,4194304] 0 2024-02-19T19:06:13.848 INFO:tasks.workunit.client.0.smithi088.stdout:2/41: rmdir d3 39 2024-02-19T19:06:13.850 INFO:tasks.workunit.client.0.smithi088.stdout:3/193: symlink d4/d22/l49 0 2024-02-19T19:06:13.850 INFO:tasks.workunit.client.0.smithi088.stdout:5/146: chown d4/d1a/d22/f24 2532 1 2024-02-19T19:06:13.851 INFO:tasks.workunit.client.0.smithi088.stdout:6/83: unlink d5/l6 0 2024-02-19T19:06:13.854 INFO:tasks.workunit.client.0.smithi088.stdout:4/42: mknod d2/d8/ce 0 2024-02-19T19:06:13.855 INFO:tasks.workunit.client.0.smithi088.stdout:3/194: read - d4/d1a/f24 zero size 2024-02-19T19:06:13.858 INFO:tasks.workunit.client.0.smithi088.stdout:5/147: symlink d4/l32 0 2024-02-19T19:06:13.858 INFO:tasks.workunit.client.0.smithi088.stdout:3/195: chown d4/d12/f21 32351 1 2024-02-19T19:06:13.858 INFO:tasks.workunit.client.0.smithi088.stdout:6/84: read - d5/d7/f15 zero size 2024-02-19T19:06:13.859 INFO:tasks.workunit.client.0.smithi088.stdout:1/188: creat d6/d10/d26/f39 x:0 0 0 2024-02-19T19:06:13.861 INFO:tasks.workunit.client.0.smithi088.stdout:7/78: dread f2 [0,4194304] 0 2024-02-19T19:06:13.862 INFO:tasks.workunit.client.0.smithi088.stdout:3/196: fdatasync d4/f38 0 2024-02-19T19:06:13.862 INFO:tasks.workunit.client.0.smithi088.stdout:9/106: dwrite d1/d8/d14/f16 [4194304,4194304] 0 2024-02-19T19:06:13.866 INFO:tasks.workunit.client.0.smithi088.stdout:6/85: rmdir d5/d7 39 2024-02-19T19:06:13.866 INFO:tasks.workunit.client.0.smithi088.stdout:3/197: dread - d4/f28 zero size 2024-02-19T19:06:13.868 INFO:tasks.workunit.client.0.smithi088.stdout:9/107: chown d1/f6 285951 1 2024-02-19T19:06:13.870 INFO:tasks.workunit.client.0.smithi088.stdout:1/189: rename d6/f14 to d6/d21/f3a 0 2024-02-19T19:06:13.872 INFO:tasks.workunit.client.0.smithi088.stdout:4/43: dwrite f0 [4194304,4194304] 0 2024-02-19T19:06:13.872 INFO:tasks.workunit.client.0.smithi088.stdout:6/86: mknod d5/de/c17 0 2024-02-19T19:06:13.874 INFO:tasks.workunit.client.0.smithi088.stdout:4/44: dread - d2/fd zero size 2024-02-19T19:06:13.874 INFO:tasks.workunit.client.0.smithi088.stdout:9/108: mkdir d1/d8/d14/d23 0 2024-02-19T19:06:13.874 INFO:tasks.workunit.client.0.smithi088.stdout:2/42: dwrite d3/fb [0,4194304] 0 2024-02-19T19:06:13.876 INFO:tasks.workunit.client.0.smithi088.stdout:9/109: fdatasync d1/f6 0 2024-02-19T19:06:13.879 INFO:tasks.workunit.client.0.smithi088.stdout:1/190: write d6/fd [2458656,56931] 0 2024-02-19T19:06:13.883 INFO:tasks.workunit.client.0.smithi088.stdout:1/191: fdatasync d6/d21/f2f 0 2024-02-19T19:06:13.883 INFO:tasks.workunit.client.0.smithi088.stdout:7/79: dread fd [0,4194304] 0 2024-02-19T19:06:13.884 INFO:tasks.workunit.client.0.smithi088.stdout:4/45: write d2/f4 [2477044,107195] 0 2024-02-19T19:06:13.884 INFO:tasks.workunit.client.0.smithi088.stdout:6/87: link d5/d7/f15 d5/de/f18 0 2024-02-19T19:06:13.885 INFO:tasks.workunit.client.0.smithi088.stdout:6/88: readlink - no filename 2024-02-19T19:06:13.886 INFO:tasks.workunit.client.0.smithi088.stdout:2/43: dread d3/f6 [0,4194304] 0 2024-02-19T19:06:13.887 INFO:tasks.workunit.client.0.smithi088.stdout:1/192: creat d6/d10/f3b x:0 0 0 2024-02-19T19:06:13.888 INFO:tasks.workunit.client.0.smithi088.stdout:4/46: symlink d2/d8/lf 0 2024-02-19T19:06:13.889 INFO:tasks.workunit.client.0.smithi088.stdout:3/198: link d4/d1a/d1e/d31/d32/c3f d4/d12/c4a 0 2024-02-19T19:06:13.894 INFO:tasks.workunit.client.0.smithi088.stdout:2/44: rename d3/ld to d3/le 0 2024-02-19T19:06:13.895 INFO:tasks.workunit.client.0.smithi088.stdout:1/193: creat d6/d21/f3c x:0 0 0 2024-02-19T19:06:13.896 INFO:tasks.workunit.client.0.smithi088.stdout:6/89: mkdir d5/d19 0 2024-02-19T19:06:13.897 INFO:tasks.workunit.client.0.smithi088.stdout:4/47: mknod d2/d5/d9/c10 0 2024-02-19T19:06:13.897 INFO:tasks.workunit.client.0.smithi088.stdout:4/48: readlink d2/d8/lf 0 2024-02-19T19:06:13.898 INFO:tasks.workunit.client.0.smithi088.stdout:3/199: creat d4/d1a/d1e/d31/f4b x:0 0 0 2024-02-19T19:06:13.898 INFO:tasks.workunit.client.0.smithi088.stdout:9/110: link d1/d13/ce d1/d8/d14/d18/c24 0 2024-02-19T19:06:13.899 INFO:tasks.workunit.client.0.smithi088.stdout:3/200: chown d4/d1a/d1e/d31/d32 4365125 1 2024-02-19T19:06:13.899 INFO:tasks.workunit.client.0.smithi088.stdout:9/111: readlink d1/d8/lf 0 2024-02-19T19:06:13.900 INFO:tasks.workunit.client.0.smithi088.stdout:7/80: truncate fe 177587 0 2024-02-19T19:06:13.902 INFO:tasks.workunit.client.0.smithi088.stdout:2/45: mkdir d3/df 0 2024-02-19T19:06:13.903 INFO:tasks.workunit.client.0.smithi088.stdout:1/194: write d6/fb [107103,107619] 0 2024-02-19T19:06:13.904 INFO:tasks.workunit.client.0.smithi088.stdout:1/195: chown d6/lc 24 1 2024-02-19T19:06:13.905 INFO:tasks.workunit.client.0.smithi088.stdout:6/90: unlink d5/d7/c11 0 2024-02-19T19:06:13.905 INFO:tasks.workunit.client.0.smithi088.stdout:6/91: rename d5/d8 to d5/d8/d1a 22 2024-02-19T19:06:13.906 INFO:tasks.workunit.client.0.smithi088.stdout:4/49: mknod d2/d5/d9/c11 0 2024-02-19T19:06:13.907 INFO:tasks.workunit.client.0.smithi088.stdout:4/50: write d2/d5/fb [94423,102569] 0 2024-02-19T19:06:13.908 INFO:tasks.workunit.client.0.smithi088.stdout:3/201: unlink d4/d1a/d1e/d31/c3b 0 2024-02-19T19:06:13.910 INFO:tasks.workunit.client.0.smithi088.stdout:6/92: symlink d5/d8/l1b 0 2024-02-19T19:06:13.911 INFO:tasks.workunit.client.0.smithi088.stdout:4/51: truncate d2/d5/fc 605510 0 2024-02-19T19:06:13.913 INFO:tasks.workunit.client.0.smithi088.stdout:1/196: dwrite d6/d21/f3c [0,4194304] 0 2024-02-19T19:06:13.914 INFO:tasks.workunit.client.0.smithi088.stdout:1/197: stat d6/d10/d26 0 2024-02-19T19:06:13.915 INFO:tasks.workunit.client.0.smithi088.stdout:3/202: creat d4/d1a/d1e/d31/d32/f4c x:0 0 0 2024-02-19T19:06:13.917 INFO:tasks.workunit.client.0.smithi088.stdout:3/203: readlink d4/d1a/d1e/d31/d32/l3d 0 2024-02-19T19:06:13.918 INFO:tasks.workunit.client.0.smithi088.stdout:8/41: sync 2024-02-19T19:06:13.919 INFO:tasks.workunit.client.0.smithi088.stdout:6/93: rename d5/de/c17 to d5/de/c1c 0 2024-02-19T19:06:13.919 INFO:tasks.workunit.client.0.smithi088.stdout:6/94: read d5/de/f10 [3902293,13817] 0 2024-02-19T19:06:13.920 INFO:tasks.workunit.client.0.smithi088.stdout:9/112: getdents d1/d8/d14 0 2024-02-19T19:06:13.922 INFO:tasks.workunit.client.0.smithi088.stdout:4/52: symlink d2/d5/l12 0 2024-02-19T19:06:13.924 INFO:tasks.workunit.client.0.smithi088.stdout:7/81: rename fe to d14/f1c 0 2024-02-19T19:06:13.926 INFO:tasks.workunit.client.0.smithi088.stdout:2/46: getdents d3 0 2024-02-19T19:06:13.927 INFO:tasks.workunit.client.0.smithi088.stdout:8/42: mknod c7 0 2024-02-19T19:06:13.929 INFO:tasks.workunit.client.0.smithi088.stdout:3/204: unlink d4/d1a/d1e/d31/l36 0 2024-02-19T19:06:13.932 INFO:tasks.workunit.client.0.smithi088.stdout:6/95: symlink d5/d8/l1d 0 2024-02-19T19:06:13.932 INFO:tasks.workunit.client.0.smithi088.stdout:6/96: chown d5/d7/f14 21142193 1 2024-02-19T19:06:13.937 INFO:tasks.workunit.client.0.smithi088.stdout:4/53: rename c1 to d2/c13 0 2024-02-19T19:06:13.937 INFO:tasks.workunit.client.0.smithi088.stdout:3/205: rename d4/d12/d33 to d4/d12/d33/d4d 22 2024-02-19T19:06:13.937 INFO:tasks.workunit.client.0.smithi088.stdout:4/54: readlink d2/d8/lf 0 2024-02-19T19:06:13.937 INFO:tasks.workunit.client.0.smithi088.stdout:2/47: stat d3/l9 0 2024-02-19T19:06:13.938 INFO:tasks.workunit.client.0.smithi088.stdout:4/55: fdatasync d2/f4 0 2024-02-19T19:06:13.938 INFO:tasks.workunit.client.0.smithi088.stdout:3/206: chown d4/d1a/d1e/d31/c35 12 1 2024-02-19T19:06:13.938 INFO:tasks.workunit.client.0.smithi088.stdout:8/43: creat f8 x:0 0 0 2024-02-19T19:06:13.940 INFO:tasks.workunit.client.0.smithi088.stdout:4/56: write d2/f6 [621922,35765] 0 2024-02-19T19:06:13.942 INFO:tasks.workunit.client.0.smithi088.stdout:1/198: dread d6/fd [0,4194304] 0 2024-02-19T19:06:13.944 INFO:tasks.workunit.client.0.smithi088.stdout:4/57: truncate d2/d5/fb 652807 0 2024-02-19T19:06:13.944 INFO:tasks.workunit.client.0.smithi088.stdout:2/48: dread d3/fb [0,4194304] 0 2024-02-19T19:06:13.946 INFO:tasks.workunit.client.0.smithi088.stdout:8/44: mkdir d9 0 2024-02-19T19:06:13.948 INFO:tasks.workunit.client.0.smithi088.stdout:9/113: dwrite d1/d13/fd [0,4194304] 0 2024-02-19T19:06:13.948 INFO:tasks.workunit.client.0.smithi088.stdout:3/207: creat d4/d12/d33/f4e x:0 0 0 2024-02-19T19:06:13.949 INFO:tasks.workunit.client.0.smithi088.stdout:9/114: chown d1/d13/fd 80 1 2024-02-19T19:06:13.950 INFO:tasks.workunit.client.0.smithi088.stdout:3/208: dread - d4/d1a/d1e/d31/f4b zero size 2024-02-19T19:06:13.951 INFO:tasks.workunit.client.0.smithi088.stdout:4/58: creat d2/d8/f14 x:0 0 0 2024-02-19T19:06:13.953 INFO:tasks.workunit.client.0.smithi088.stdout:7/82: truncate d14/f1c 34519 0 2024-02-19T19:06:13.954 INFO:tasks.workunit.client.0.smithi088.stdout:6/97: creat d5/f1e x:0 0 0 2024-02-19T19:06:13.955 INFO:tasks.workunit.client.0.smithi088.stdout:1/199: mknod d6/c3d 0 2024-02-19T19:06:13.956 INFO:tasks.workunit.client.0.smithi088.stdout:3/209: mknod d4/d1a/d1e/d31/c4f 0 2024-02-19T19:06:13.956 INFO:tasks.workunit.client.0.smithi088.stdout:2/49: getdents d3/df 0 2024-02-19T19:06:13.956 INFO:tasks.workunit.client.0.smithi088.stdout:8/45: symlink d9/la 0 2024-02-19T19:06:13.957 INFO:tasks.workunit.client.0.smithi088.stdout:4/59: dread d2/f4 [0,4194304] 0 2024-02-19T19:06:13.958 INFO:tasks.workunit.client.0.smithi088.stdout:4/60: read - d2/d8/f14 zero size 2024-02-19T19:06:13.959 INFO:tasks.workunit.client.0.smithi088.stdout:9/115: creat d1/d8/d14/d23/f25 x:0 0 0 2024-02-19T19:06:13.959 INFO:tasks.workunit.client.0.smithi088.stdout:6/98: truncate d5/de/f10 5073340 0 2024-02-19T19:06:13.960 INFO:tasks.workunit.client.0.smithi088.stdout:8/46: symlink d9/lb 0 2024-02-19T19:06:13.963 INFO:tasks.workunit.client.0.smithi088.stdout:3/210: creat d4/d1a/d1e/d31/d32/f50 x:0 0 0 2024-02-19T19:06:13.964 INFO:tasks.workunit.client.0.smithi088.stdout:3/211: stat d4/f40 0 2024-02-19T19:06:13.966 INFO:tasks.workunit.client.0.smithi088.stdout:4/61: creat d2/f15 x:0 0 0 2024-02-19T19:06:13.967 INFO:tasks.workunit.client.0.smithi088.stdout:9/116: creat d1/d8/d14/d23/f26 x:0 0 0 2024-02-19T19:06:13.969 INFO:tasks.workunit.client.0.smithi088.stdout:9/117: fsync d1/d8/d14/d18/f1a 0 2024-02-19T19:06:13.970 INFO:tasks.workunit.client.0.smithi088.stdout:8/47: symlink d9/lc 0 2024-02-19T19:06:13.971 INFO:tasks.workunit.client.0.smithi088.stdout:8/48: read - f6 zero size 2024-02-19T19:06:13.972 INFO:tasks.workunit.client.0.smithi088.stdout:9/118: truncate d1/d13/fd 4978020 0 2024-02-19T19:06:13.972 INFO:tasks.workunit.client.0.smithi088.stdout:8/49: readlink l1 0 2024-02-19T19:06:13.973 INFO:tasks.workunit.client.0.smithi088.stdout:3/212: creat d4/d1a/d1e/d31/d32/f51 x:0 0 0 2024-02-19T19:06:13.974 INFO:tasks.workunit.client.0.smithi088.stdout:4/62: creat d2/d8/f16 x:0 0 0 2024-02-19T19:06:13.974 INFO:tasks.workunit.client.0.smithi088.stdout:6/99: dwrite d5/ff [0,4194304] 0 2024-02-19T19:06:13.977 INFO:tasks.workunit.client.0.smithi088.stdout:9/119: symlink d1/d8/l27 0 2024-02-19T19:06:13.977 INFO:tasks.workunit.client.0.smithi088.stdout:7/83: dwrite f13 [0,4194304] 0 2024-02-19T19:06:13.978 INFO:tasks.workunit.client.0.smithi088.stdout:3/213: write d4/d1a/d1e/d31/d32/f50 [785737,10202] 0 2024-02-19T19:06:13.981 INFO:tasks.workunit.client.0.smithi088.stdout:1/200: truncate d6/d21/f3c 3439672 0 2024-02-19T19:06:13.983 INFO:tasks.workunit.client.0.smithi088.stdout:6/100: dread d5/ff [0,4194304] 0 2024-02-19T19:06:13.985 INFO:tasks.workunit.client.0.smithi088.stdout:4/63: rename d2/d5/d9/la to d2/d8/l17 0 2024-02-19T19:06:13.985 INFO:tasks.workunit.client.0.smithi088.stdout:3/214: dread d4/d1a/d1e/f2f [0,4194304] 0 2024-02-19T19:06:13.986 INFO:tasks.workunit.client.0.smithi088.stdout:3/215: readlink d4/l5 0 2024-02-19T19:06:13.986 INFO:tasks.workunit.client.0.smithi088.stdout:2/50: truncate d3/f6 3722690 0 2024-02-19T19:06:13.988 INFO:tasks.workunit.client.0.smithi088.stdout:9/120: creat d1/d8/f28 x:0 0 0 2024-02-19T19:06:13.989 INFO:tasks.workunit.client.0.smithi088.stdout:4/64: read d2/f6 [293461,111630] 0 2024-02-19T19:06:13.990 INFO:tasks.workunit.client.0.smithi088.stdout:7/84: mkdir d14/d1d 0 2024-02-19T19:06:13.991 INFO:tasks.workunit.client.0.smithi088.stdout:4/65: write d2/d5/fb [1223951,79500] 0 2024-02-19T19:06:13.992 INFO:tasks.workunit.client.0.smithi088.stdout:6/101: chown d5/de/f10 14696 1 2024-02-19T19:06:13.997 INFO:tasks.workunit.client.0.smithi088.stdout:7/85: unlink d14/f15 0 2024-02-19T19:06:13.998 INFO:tasks.workunit.client.0.smithi088.stdout:4/66: dwrite f0 [0,4194304] 0 2024-02-19T19:06:13.998 INFO:tasks.workunit.client.0.smithi088.stdout:8/50: getdents d9 0 2024-02-19T19:06:13.999 INFO:tasks.workunit.client.0.smithi088.stdout:6/102: write d5/ff [3253895,9411] 0 2024-02-19T19:06:14.002 INFO:tasks.workunit.client.0.smithi088.stdout:3/216: symlink d4/d1a/l52 0 2024-02-19T19:06:14.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:06:14.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:14.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:14.003 INFO:tasks.workunit.client.0.smithi088.stdout:2/51: chown d3/f6 1 1 2024-02-19T19:06:14.005 INFO:tasks.workunit.client.0.smithi088.stdout:4/67: creat d2/d5/f18 x:0 0 0 2024-02-19T19:06:14.007 INFO:tasks.workunit.client.0.smithi088.stdout:1/201: rename d6/d10/c12 to d6/c3e 0 2024-02-19T19:06:14.007 INFO:tasks.workunit.client.0.smithi088.stdout:7/86: rename d14 to d14/d1d/d1e 22 2024-02-19T19:06:14.008 INFO:tasks.workunit.client.0.smithi088.stdout:3/217: symlink d4/d1a/d1e/d31/d32/l53 0 2024-02-19T19:06:14.010 INFO:tasks.workunit.client.0.smithi088.stdout:2/52: write d3/fb [1410701,111161] 0 2024-02-19T19:06:14.011 INFO:tasks.workunit.client.0.smithi088.stdout:4/68: unlink d2/f6 0 2024-02-19T19:06:14.011 INFO:tasks.workunit.client.0.smithi088.stdout:6/103: mkdir d5/d19/d1f 0 2024-02-19T19:06:14.012 INFO:tasks.workunit.client.0.smithi088.stdout:1/202: dread d6/d10/f11 [0,4194304] 0 2024-02-19T19:06:14.013 INFO:tasks.workunit.client.0.smithi088.stdout:3/218: mknod d4/d1a/d1e/c54 0 2024-02-19T19:06:14.013 INFO:tasks.workunit.client.0.smithi088.stdout:7/87: write fd [4203998,27732] 0 2024-02-19T19:06:14.014 INFO:tasks.workunit.client.0.smithi088.stdout:3/219: readlink d4/l5 0 2024-02-19T19:06:14.015 INFO:tasks.workunit.client.0.smithi088.stdout:4/69: rename d2/d8/lf to d2/d5/d9/l19 0 2024-02-19T19:06:14.016 INFO:tasks.workunit.client.0.smithi088.stdout:4/70: write d2/d5/f18 [185723,43115] 0 2024-02-19T19:06:14.016 INFO:tasks.workunit.client.0.smithi088.stdout:6/104: creat d5/d19/f20 x:0 0 0 2024-02-19T19:06:14.017 INFO:tasks.workunit.client.0.smithi088.stdout:2/53: fdatasync d3/f6 0 2024-02-19T19:06:14.017 INFO:tasks.workunit.client.0.smithi088.stdout:2/54: stat d3 0 2024-02-19T19:06:14.019 INFO:tasks.workunit.client.0.smithi088.stdout:3/220: mkdir d4/d1a/d1e/d31/d32/d55 0 2024-02-19T19:06:14.020 INFO:tasks.workunit.client.0.smithi088.stdout:3/221: stat d4/d12/f47 0 2024-02-19T19:06:14.020 INFO:tasks.workunit.client.0.smithi088.stdout:6/105: mkdir d5/d19/d21 0 2024-02-19T19:06:14.021 INFO:tasks.workunit.client.0.smithi088.stdout:6/106: chown d5/d19/d1f 48 1 2024-02-19T19:06:14.021 INFO:tasks.workunit.client.0.smithi088.stdout:3/222: write d4/d1a/d1e/d31/d32/f50 [887072,10309] 0 2024-02-19T19:06:14.021 INFO:tasks.workunit.client.0.smithi088.stdout:6/107: dread - d5/f1e zero size 2024-02-19T19:06:14.022 INFO:tasks.workunit.client.0.smithi088.stdout:2/55: rename d3/cc to d3/df/c10 0 2024-02-19T19:06:14.023 INFO:tasks.workunit.client.0.smithi088.stdout:7/88: link f10 d14/f1f 0 2024-02-19T19:06:14.025 INFO:tasks.workunit.client.0.smithi088.stdout:6/108: dread d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:14.027 INFO:tasks.workunit.client.0.smithi088.stdout:2/56: creat d3/df/f11 x:0 0 0 2024-02-19T19:06:14.031 INFO:tasks.workunit.client.0.smithi088.stdout:7/89: dwrite fc [0,4194304] 0 2024-02-19T19:06:14.036 INFO:tasks.workunit.client.0.smithi088.stdout:3/223: rename d4/d1a/d1e/d31/d32/c3f to d4/d1a/d23/c56 0 2024-02-19T19:06:14.036 INFO:tasks.workunit.client.0.smithi088.stdout:3/224: stat d4/d12/f2e 0 2024-02-19T19:06:14.041 INFO:tasks.workunit.client.0.smithi088.stdout:7/90: symlink d14/d1d/l20 0 2024-02-19T19:06:14.042 INFO:tasks.workunit.client.0.smithi088.stdout:7/91: fdatasync fb 0 2024-02-19T19:06:14.042 INFO:tasks.workunit.client.0.smithi088.stdout:7/92: chown d14/f17 496006461 1 2024-02-19T19:06:14.043 INFO:tasks.workunit.client.0.smithi088.stdout:7/93: chown d14/f16 0 1 2024-02-19T19:06:14.045 INFO:tasks.workunit.client.0.smithi088.stdout:3/225: link d4/fc d4/d1a/d1e/d31/d32/d55/f57 0 2024-02-19T19:06:14.048 INFO:tasks.workunit.client.0.smithi088.stdout:7/94: dread d14/f1c [0,4194304] 0 2024-02-19T19:06:14.049 INFO:tasks.workunit.client.0.smithi088.stdout:7/95: chown d14/c18 13538459 1 2024-02-19T19:06:14.049 INFO:tasks.workunit.client.0.smithi088.stdout:7/96: read - d14/f19 zero size 2024-02-19T19:06:14.052 INFO:tasks.workunit.client.0.smithi088.stdout:3/226: dwrite d4/f16 [0,4194304] 0 2024-02-19T19:06:14.054 INFO:tasks.workunit.client.0.smithi088.stdout:3/227: truncate d4/fc 1475142 0 2024-02-19T19:06:14.057 INFO:tasks.workunit.client.0.smithi088.stdout:3/228: stat d4/d1a/d23/l48 0 2024-02-19T19:06:14.059 INFO:tasks.workunit.client.0.smithi088.stdout:2/57: truncate d3/fa 1172526 0 2024-02-19T19:06:14.060 INFO:tasks.workunit.client.0.smithi088.stdout:3/229: mkdir d4/d12/d33/d58 0 2024-02-19T19:06:14.062 INFO:tasks.workunit.client.0.smithi088.stdout:1/203: dwrite d6/d21/f3a [0,4194304] 0 2024-02-19T19:06:14.065 INFO:tasks.workunit.client.0.smithi088.stdout:6/109: dwrite d5/d7/f15 [0,4194304] 0 2024-02-19T19:06:14.067 INFO:tasks.workunit.client.0.smithi088.stdout:3/230: symlink d4/d1a/d1e/d31/l59 0 2024-02-19T19:06:14.070 INFO:tasks.workunit.client.0.smithi088.stdout:1/204: symlink d6/d10/d28/d29/l3f 0 2024-02-19T19:06:14.075 INFO:tasks.workunit.client.0.smithi088.stdout:3/231: mkdir d4/d1a/d1e/d5a 0 2024-02-19T19:06:14.076 INFO:tasks.workunit.client.0.smithi088.stdout:7/97: dwrite f10 [0,4194304] 0 2024-02-19T19:06:14.080 INFO:tasks.workunit.client.0.smithi088.stdout:3/232: mknod d4/c5b 0 2024-02-19T19:06:14.083 INFO:tasks.workunit.client.0.smithi088.stdout:7/98: dwrite f6 [0,4194304] 0 2024-02-19T19:06:14.083 INFO:tasks.workunit.client.0.smithi088.stdout:7/99: readlink d14/l1b 0 2024-02-19T19:06:14.084 INFO:tasks.workunit.client.0.smithi088.stdout:3/233: mknod d4/d12/d33/d58/c5c 0 2024-02-19T19:06:14.086 INFO:tasks.workunit.client.0.smithi088.stdout:7/100: mkdir d14/d21 0 2024-02-19T19:06:14.086 INFO:tasks.workunit.client.0.smithi088.stdout:7/101: stat c5 0 2024-02-19T19:06:14.087 INFO:tasks.workunit.client.0.smithi088.stdout:3/234: read - d4/d12/f17 zero size 2024-02-19T19:06:14.089 INFO:tasks.workunit.client.0.smithi088.stdout:3/235: read d4/d1a/d1e/d31/d32/d55/f57 [784358,61526] 0 2024-02-19T19:06:14.091 INFO:tasks.workunit.client.0.smithi088.stdout:3/236: dread d4/d1a/d1e/d31/d32/d55/f57 [0,4194304] 0 2024-02-19T19:06:14.092 INFO:tasks.workunit.client.0.smithi088.stdout:3/237: read d4/d12/f1d [181039,80728] 0 2024-02-19T19:06:14.093 INFO:tasks.workunit.client.0.smithi088.stdout:3/238: truncate d4/d12/f2e 351989 0 2024-02-19T19:06:14.095 INFO:tasks.workunit.client.0.smithi088.stdout:3/239: write d4/f18 [581083,45303] 0 2024-02-19T19:06:14.099 INFO:tasks.workunit.client.0.smithi088.stdout:2/58: truncate d3/fa 592153 0 2024-02-19T19:06:14.100 INFO:tasks.workunit.client.0.smithi088.stdout:0/65: sync 2024-02-19T19:06:14.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:06:14.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:14.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:14.101 INFO:tasks.workunit.client.0.smithi088.stdout:3/240: chown d4/d1a/d23/c56 225 1 2024-02-19T19:06:14.102 INFO:tasks.workunit.client.0.smithi088.stdout:0/66: rename f7 to d13/f1b 0 2024-02-19T19:06:14.105 INFO:tasks.workunit.client.0.smithi088.stdout:0/67: creat d13/f1c x:0 0 0 2024-02-19T19:06:14.106 INFO:tasks.workunit.client.0.smithi088.stdout:3/241: link d4/fd d4/d1a/d23/f5d 0 2024-02-19T19:06:14.108 INFO:tasks.workunit.client.0.smithi088.stdout:0/68: link d13/f1c d13/f1d 0 2024-02-19T19:06:14.110 INFO:tasks.workunit.client.0.smithi088.stdout:3/242: creat d4/d1a/d1e/d5a/f5e x:0 0 0 2024-02-19T19:06:14.111 INFO:tasks.workunit.client.0.smithi088.stdout:0/69: unlink c11 0 2024-02-19T19:06:14.111 INFO:tasks.workunit.client.0.smithi088.stdout:0/70: truncate d13/f1a 348542 0 2024-02-19T19:06:14.113 INFO:tasks.workunit.client.0.smithi088.stdout:7/102: truncate fc 2290944 0 2024-02-19T19:06:14.113 INFO:tasks.workunit.client.0.smithi088.stdout:7/103: chown d14 57672 1 2024-02-19T19:06:14.114 INFO:tasks.workunit.client.0.smithi088.stdout:0/71: creat d13/f1e x:0 0 0 2024-02-19T19:06:14.115 INFO:tasks.workunit.client.0.smithi088.stdout:0/72: stat d13/f15 0 2024-02-19T19:06:14.115 INFO:tasks.workunit.client.0.smithi088.stdout:7/104: creat d14/f22 x:0 0 0 2024-02-19T19:06:14.123 INFO:tasks.workunit.client.0.smithi088.stdout:0/73: dwrite d13/f1d [0,4194304] 0 2024-02-19T19:06:14.124 INFO:tasks.workunit.client.0.smithi088.stdout:0/74: write f5 [1641405,128261] 0 2024-02-19T19:06:14.126 INFO:tasks.workunit.client.0.smithi088.stdout:0/75: read fe [80499,19335] 0 2024-02-19T19:06:14.130 INFO:tasks.workunit.client.0.smithi088.stdout:0/76: mkdir d13/d1f 0 2024-02-19T19:06:14.131 INFO:tasks.workunit.client.0.smithi088.stdout:7/105: truncate fc 2938906 0 2024-02-19T19:06:14.132 INFO:tasks.workunit.client.0.smithi088.stdout:7/106: read f10 [1438680,127565] 0 2024-02-19T19:06:14.138 INFO:tasks.workunit.client.0.smithi088.stdout:7/107: symlink d14/d21/l23 0 2024-02-19T19:06:14.139 INFO:tasks.workunit.client.0.smithi088.stdout:7/108: write f6 [3337793,68732] 0 2024-02-19T19:06:14.140 INFO:tasks.workunit.client.0.smithi088.stdout:7/109: chown f8 35992758 1 2024-02-19T19:06:14.141 INFO:tasks.workunit.client.0.smithi088.stdout:2/59: dwrite d3/fa [0,4194304] 0 2024-02-19T19:06:14.145 INFO:tasks.workunit.client.0.smithi088.stdout:0/77: truncate f4 2077055 0 2024-02-19T19:06:14.146 INFO:tasks.workunit.client.0.smithi088.stdout:7/110: rmdir d14 39 2024-02-19T19:06:14.147 INFO:tasks.workunit.client.0.smithi088.stdout:0/78: read f3 [3332076,128550] 0 2024-02-19T19:06:14.148 INFO:tasks.workunit.client.0.smithi088.stdout:0/79: truncate d13/f1e 14855 0 2024-02-19T19:06:14.150 INFO:tasks.workunit.client.0.smithi088.stdout:0/80: read fe [55475,115827] 0 2024-02-19T19:06:14.152 INFO:tasks.workunit.client.0.smithi088.stdout:7/111: chown fc 2737 1 2024-02-19T19:06:14.158 INFO:tasks.workunit.client.0.smithi088.stdout:7/112: dwrite f6 [0,4194304] 0 2024-02-19T19:06:14.162 INFO:tasks.workunit.client.0.smithi088.stdout:7/113: rename d14/c18 to d14/d21/c24 0 2024-02-19T19:06:14.168 INFO:tasks.workunit.client.0.smithi088.stdout:7/114: write f10 [2305106,25960] 0 2024-02-19T19:06:14.172 INFO:tasks.workunit.client.0.smithi088.stdout:0/81: dwrite f4 [0,4194304] 0 2024-02-19T19:06:14.173 INFO:tasks.workunit.client.0.smithi088.stdout:0/82: chown f4 1 1 2024-02-19T19:06:14.173 INFO:tasks.workunit.client.0.smithi088.stdout:0/83: readlink lf 0 2024-02-19T19:06:14.173 INFO:tasks.workunit.client.0.smithi088.stdout:0/84: chown fe 8700 1 2024-02-19T19:06:14.178 INFO:tasks.workunit.client.0.smithi088.stdout:0/85: rmdir d13/d1f 0 2024-02-19T19:06:14.178 INFO:tasks.workunit.client.0.smithi088.stdout:7/115: dwrite fc [0,4194304] 0 2024-02-19T19:06:14.181 INFO:tasks.workunit.client.0.smithi088.stdout:7/116: creat d14/f25 x:0 0 0 2024-02-19T19:06:14.183 INFO:tasks.workunit.client.0.smithi088.stdout:7/117: rmdir d14/d1d 39 2024-02-19T19:06:14.189 INFO:tasks.workunit.client.0.smithi088.stdout:7/118: dread d14/f17 [0,4194304] 0 2024-02-19T19:06:14.191 INFO:tasks.workunit.client.0.smithi088.stdout:0/86: dwrite fe [0,4194304] 0 2024-02-19T19:06:14.193 INFO:tasks.workunit.client.0.smithi088.stdout:0/87: mknod d13/c20 0 2024-02-19T19:06:14.197 INFO:tasks.workunit.client.0.smithi088.stdout:7/119: dwrite f6 [0,4194304] 0 2024-02-19T19:06:14.197 INFO:tasks.workunit.client.0.smithi088.stdout:7/120: readlink d14/l1b 0 2024-02-19T19:06:14.202 INFO:tasks.workunit.client.0.smithi088.stdout:7/121: creat d14/d1d/f26 x:0 0 0 2024-02-19T19:06:14.207 INFO:tasks.workunit.client.0.smithi088.stdout:7/122: rename d14/d21 to d14/d27 0 2024-02-19T19:06:14.209 INFO:tasks.workunit.client.0.smithi088.stdout:0/88: dread f5 [0,4194304] 0 2024-02-19T19:06:14.212 INFO:tasks.workunit.client.0.smithi088.stdout:0/89: rename d13/c20 to d13/c21 0 2024-02-19T19:06:14.212 INFO:tasks.workunit.client.0.smithi088.stdout:7/123: dwrite fc [0,4194304] 0 2024-02-19T19:06:14.214 INFO:tasks.workunit.client.0.smithi088.stdout:7/124: write d14/f25 [1000021,24877] 0 2024-02-19T19:06:14.214 INFO:tasks.workunit.client.0.smithi088.stdout:7/125: stat c7 0 2024-02-19T19:06:14.216 INFO:tasks.workunit.client.0.smithi088.stdout:0/90: rename d13/l18 to d13/l22 0 2024-02-19T19:06:14.219 INFO:tasks.workunit.client.0.smithi088.stdout:7/126: mkdir d14/d28 0 2024-02-19T19:06:14.228 INFO:tasks.workunit.client.0.smithi088.stdout:0/91: write fb [817394,93793] 0 2024-02-19T19:06:14.229 INFO:tasks.workunit.client.0.smithi088.stdout:7/127: truncate fc 2186841 0 2024-02-19T19:06:14.230 INFO:tasks.workunit.client.0.smithi088.stdout:0/92: creat d13/f23 x:0 0 0 2024-02-19T19:06:14.233 INFO:tasks.workunit.client.0.smithi088.stdout:7/128: creat d14/f29 x:0 0 0 2024-02-19T19:06:14.237 INFO:tasks.workunit.client.0.smithi088.stdout:0/93: dwrite d13/f1e [0,4194304] 0 2024-02-19T19:06:14.238 INFO:tasks.workunit.client.0.smithi088.stdout:7/129: rename f10 to d14/f2a 0 2024-02-19T19:06:14.239 INFO:tasks.workunit.client.0.smithi088.stdout:7/130: dread - d14/d1d/f26 zero size 2024-02-19T19:06:14.241 INFO:tasks.workunit.client.0.smithi088.stdout:0/94: fdatasync f3 0 2024-02-19T19:06:14.242 INFO:tasks.workunit.client.0.smithi088.stdout:0/95: fsync d13/f1d 0 2024-02-19T19:06:14.243 INFO:tasks.workunit.client.0.smithi088.stdout:7/131: mknod d14/d27/c2b 0 2024-02-19T19:06:14.249 INFO:tasks.workunit.client.0.smithi088.stdout:7/132: symlink d14/l2c 0 2024-02-19T19:06:14.250 INFO:tasks.workunit.client.0.smithi088.stdout:7/133: read - d14/d1d/f26 zero size 2024-02-19T19:06:14.250 INFO:tasks.workunit.client.0.smithi088.stdout:7/134: stat f6 0 2024-02-19T19:06:14.252 INFO:tasks.workunit.client.0.smithi088.stdout:7/135: symlink d14/d28/l2d 0 2024-02-19T19:06:14.253 INFO:tasks.workunit.client.0.smithi088.stdout:7/136: write d14/f16 [3552201,29708] 0 2024-02-19T19:06:14.257 INFO:tasks.workunit.client.0.smithi088.stdout:0/96: truncate fb 56675 0 2024-02-19T19:06:14.258 INFO:tasks.workunit.client.0.smithi088.stdout:7/137: dread f6 [0,4194304] 0 2024-02-19T19:06:14.258 INFO:tasks.workunit.client.0.smithi088.stdout:0/97: mkdir d13/d24 0 2024-02-19T19:06:14.259 INFO:tasks.workunit.client.0.smithi088.stdout:7/138: write f6 [753692,10914] 0 2024-02-19T19:06:14.262 INFO:tasks.workunit.client.0.smithi088.stdout:0/98: dread f4 [0,4194304] 0 2024-02-19T19:06:14.264 INFO:tasks.workunit.client.0.smithi088.stdout:7/139: dread fb [0,4194304] 0 2024-02-19T19:06:14.268 INFO:tasks.workunit.client.0.smithi088.stdout:7/140: rename d14/d1d/f26 to d14/f2e 0 2024-02-19T19:06:14.268 INFO:tasks.workunit.client.0.smithi088.stdout:0/99: dwrite d13/f1a [0,4194304] 0 2024-02-19T19:06:14.269 INFO:tasks.workunit.client.0.smithi088.stdout:0/100: read d13/f1b [2420472,27274] 0 2024-02-19T19:06:14.273 INFO:tasks.workunit.client.0.smithi088.stdout:5/148: sync 2024-02-19T19:06:14.273 INFO:tasks.workunit.client.0.smithi088.stdout:7/141: dread fd [0,4194304] 0 2024-02-19T19:06:14.276 INFO:tasks.workunit.client.0.smithi088.stdout:5/149: creat d4/d7/f33 x:0 0 0 2024-02-19T19:06:14.277 INFO:tasks.workunit.client.0.smithi088.stdout:0/101: dwrite d13/f1e [0,4194304] 0 2024-02-19T19:06:14.280 INFO:tasks.workunit.client.0.smithi088.stdout:7/142: link l11 d14/d28/l2f 0 2024-02-19T19:06:14.281 INFO:tasks.workunit.client.0.smithi088.stdout:7/143: fsync d14/f19 0 2024-02-19T19:06:14.288 INFO:tasks.workunit.client.0.smithi088.stdout:0/102: getdents d13 0 2024-02-19T19:06:14.288 INFO:tasks.workunit.client.0.smithi088.stdout:7/144: creat d14/f30 x:0 0 0 2024-02-19T19:06:14.292 INFO:tasks.workunit.client.0.smithi088.stdout:5/150: write d4/f12 [1014712,888] 0 2024-02-19T19:06:14.292 INFO:tasks.workunit.client.0.smithi088.stdout:7/145: dread f13 [0,4194304] 0 2024-02-19T19:06:14.293 INFO:tasks.workunit.client.0.smithi088.stdout:0/103: mkdir d13/d24/d25 0 2024-02-19T19:06:14.293 INFO:tasks.workunit.client.0.smithi088.stdout:7/146: chown d14/f17 3724 1 2024-02-19T19:06:14.294 INFO:tasks.workunit.client.0.smithi088.stdout:5/151: write d4/d7/f15 [5125104,74776] 0 2024-02-19T19:06:14.294 INFO:tasks.workunit.client.0.smithi088.stdout:0/104: write d13/f23 [58824,71972] 0 2024-02-19T19:06:14.295 INFO:tasks.workunit.client.0.smithi088.stdout:0/105: fdatasync d13/f15 0 2024-02-19T19:06:14.296 INFO:tasks.workunit.client.0.smithi088.stdout:7/147: fsync d14/f2a 0 2024-02-19T19:06:14.298 INFO:tasks.workunit.client.0.smithi088.stdout:0/106: creat d13/d24/d25/f26 x:0 0 0 2024-02-19T19:06:14.298 INFO:tasks.workunit.client.0.smithi088.stdout:7/148: symlink d14/d27/l31 0 2024-02-19T19:06:14.299 INFO:tasks.workunit.client.0.smithi088.stdout:0/107: rmdir d13/d24 39 2024-02-19T19:06:14.302 INFO:tasks.workunit.client.0.smithi088.stdout:5/152: dwrite d4/d1a/f2b [0,4194304] 0 2024-02-19T19:06:14.302 INFO:tasks.workunit.client.0.smithi088.stdout:0/108: write d13/d24/d25/f26 [382390,10925] 0 2024-02-19T19:06:14.303 INFO:tasks.workunit.client.0.smithi088.stdout:7/149: rename d14/f25 to d14/d27/f32 0 2024-02-19T19:06:14.305 INFO:tasks.workunit.client.0.smithi088.stdout:0/109: creat d13/d24/d25/f27 x:0 0 0 2024-02-19T19:06:14.306 INFO:tasks.workunit.client.0.smithi088.stdout:5/153: rename d4/d7/c30 to d4/c34 0 2024-02-19T19:06:14.309 INFO:tasks.workunit.client.0.smithi088.stdout:0/110: unlink d13/l14 0 2024-02-19T19:06:14.310 INFO:tasks.workunit.client.0.smithi088.stdout:7/150: dwrite d14/f1f [4194304,4194304] 0 2024-02-19T19:06:14.311 INFO:tasks.workunit.client.0.smithi088.stdout:5/154: symlink d4/d1a/l35 0 2024-02-19T19:06:14.312 INFO:tasks.workunit.client.0.smithi088.stdout:5/155: dread - d4/d7/f31 zero size 2024-02-19T19:06:14.314 INFO:tasks.workunit.client.0.smithi088.stdout:5/156: write d4/f5 [3882670,48647] 0 2024-02-19T19:06:14.316 INFO:tasks.workunit.client.0.smithi088.stdout:7/151: mknod d14/c33 0 2024-02-19T19:06:14.317 INFO:tasks.workunit.client.0.smithi088.stdout:5/157: creat d4/d7/f36 x:0 0 0 2024-02-19T19:06:14.318 INFO:tasks.workunit.client.0.smithi088.stdout:7/152: write d14/f22 [1002136,48945] 0 2024-02-19T19:06:14.320 INFO:tasks.workunit.client.0.smithi088.stdout:0/111: dwrite d13/f1a [4194304,4194304] 0 2024-02-19T19:06:14.321 INFO:tasks.workunit.client.0.smithi088.stdout:0/112: chown f5 1449571 1 2024-02-19T19:06:14.321 INFO:tasks.workunit.client.0.smithi088.stdout:0/113: readlink lf 0 2024-02-19T19:06:14.322 INFO:tasks.workunit.client.0.smithi088.stdout:0/114: readlink d13/l17 0 2024-02-19T19:06:14.322 INFO:tasks.workunit.client.0.smithi088.stdout:5/158: unlink d4/d7/cb 0 2024-02-19T19:06:14.322 INFO:tasks.workunit.client.0.smithi088.stdout:0/115: fsync d13/f1c 0 2024-02-19T19:06:14.323 INFO:tasks.workunit.client.0.smithi088.stdout:0/116: fdatasync d13/f1e 0 2024-02-19T19:06:14.326 INFO:tasks.workunit.client.0.smithi088.stdout:5/159: dread d4/f12 [4194304,4194304] 0 2024-02-19T19:06:14.341 INFO:tasks.workunit.client.0.smithi088.stdout:5/160: rmdir d4 39 2024-02-19T19:06:14.342 INFO:tasks.workunit.client.0.smithi088.stdout:5/161: chown f0 867061875 1 2024-02-19T19:06:14.352 INFO:tasks.workunit.client.0.smithi088.stdout:7/153: truncate f2 4150735 0 2024-02-19T19:06:14.358 INFO:tasks.workunit.client.0.smithi088.stdout:0/117: dwrite fb [0,4194304] 0 2024-02-19T19:06:14.361 INFO:tasks.workunit.client.0.smithi088.stdout:7/154: dwrite fb [0,4194304] 0 2024-02-19T19:06:14.366 INFO:tasks.workunit.client.0.smithi088.stdout:0/118: dwrite d13/f23 [0,4194304] 0 2024-02-19T19:06:14.421 INFO:tasks.workunit.client.0.smithi088.stdout:9/121: sync 2024-02-19T19:06:14.563 INFO:tasks.workunit.client.0.smithi088.stdout:8/51: sync 2024-02-19T19:06:14.563 INFO:tasks.workunit.client.0.smithi088.stdout:8/52: fsync f8 0 2024-02-19T19:06:14.563 INFO:tasks.workunit.client.0.smithi088.stdout:8/53: write f6 [858497,17983] 0 2024-02-19T19:06:14.568 INFO:tasks.workunit.client.0.smithi088.stdout:8/54: dwrite f8 [0,4194304] 0 2024-02-19T19:06:14.572 INFO:tasks.workunit.client.0.smithi088.stdout:5/162: write d4/fe [57607,115950] 0 2024-02-19T19:06:14.573 INFO:tasks.workunit.client.0.smithi088.stdout:5/163: chown d4/f5 40544943 1 2024-02-19T19:06:14.574 INFO:tasks.workunit.client.0.smithi088.stdout:5/164: write d4/d1a/f1b [2919550,106569] 0 2024-02-19T19:06:14.575 INFO:tasks.workunit.client.0.smithi088.stdout:7/155: creat d14/d1d/f34 x:0 0 0 2024-02-19T19:06:14.575 INFO:tasks.workunit.client.0.smithi088.stdout:5/165: readlink d4/d7/l10 0 2024-02-19T19:06:14.576 INFO:tasks.workunit.client.0.smithi088.stdout:0/119: symlink d13/d24/l28 0 2024-02-19T19:06:14.577 INFO:tasks.workunit.client.0.smithi088.stdout:5/166: truncate d4/f2e 860586 0 2024-02-19T19:06:14.578 INFO:tasks.workunit.client.0.smithi088.stdout:9/122: unlink d1/d8/l27 0 2024-02-19T19:06:14.578 INFO:tasks.workunit.client.0.smithi088.stdout:5/167: write d4/f5 [8571165,124217] 0 2024-02-19T19:06:14.579 INFO:tasks.workunit.client.0.smithi088.stdout:5/168: fdatasync d4/f2e 0 2024-02-19T19:06:14.707 INFO:tasks.workunit.client.0.smithi088.stdout:7/156: rmdir d14/d28 39 2024-02-19T19:06:14.712 INFO:tasks.workunit.client.0.smithi088.stdout:0/120: creat d13/f29 x:0 0 0 2024-02-19T19:06:14.713 INFO:tasks.workunit.client.0.smithi088.stdout:7/157: getdents d14/d27 0 2024-02-19T19:06:14.715 INFO:tasks.workunit.client.0.smithi088.stdout:8/55: symlink d9/ld 0 2024-02-19T19:06:14.716 INFO:tasks.workunit.client.0.smithi088.stdout:0/121: mkdir d13/d24/d25/d2a 0 2024-02-19T19:06:14.716 INFO:tasks.workunit.client.0.smithi088.stdout:0/122: chown d13/f1c 1022 1 2024-02-19T19:06:14.717 INFO:tasks.workunit.client.0.smithi088.stdout:0/123: chown d13/f29 15603 1 2024-02-19T19:06:14.720 INFO:tasks.workunit.client.0.smithi088.stdout:5/169: creat d4/d1a/f37 x:0 0 0 2024-02-19T19:06:14.720 INFO:tasks.workunit.client.0.smithi088.stdout:5/170: write d4/f11 [59700,48945] 0 2024-02-19T19:06:14.721 INFO:tasks.workunit.client.0.smithi088.stdout:8/56: write f8 [2468595,86967] 0 2024-02-19T19:06:14.724 INFO:tasks.workunit.client.0.smithi088.stdout:0/124: rename d13/f1e to d13/d24/d25/f2b 0 2024-02-19T19:06:14.725 INFO:tasks.workunit.client.0.smithi088.stdout:7/158: creat d14/d28/f35 x:0 0 0 2024-02-19T19:06:14.728 INFO:tasks.workunit.client.0.smithi088.stdout:8/57: mknod d9/ce 0 2024-02-19T19:06:14.731 INFO:tasks.workunit.client.0.smithi088.stdout:0/125: dwrite d13/d24/d25/f27 [0,4194304] 0 2024-02-19T19:06:14.732 INFO:tasks.workunit.client.0.smithi088.stdout:5/171: rename d4/d1a/d22/f29 to d4/d1a/f38 0 2024-02-19T19:06:14.736 INFO:tasks.workunit.client.0.smithi088.stdout:5/172: dread f0 [0,4194304] 0 2024-02-19T19:06:14.739 INFO:tasks.workunit.client.0.smithi088.stdout:0/126: unlink d13/c21 0 2024-02-19T19:06:14.743 INFO:tasks.workunit.client.0.smithi088.stdout:0/127: dread d13/f1a [4194304,4194304] 0 2024-02-19T19:06:14.744 INFO:tasks.workunit.client.0.smithi088.stdout:0/128: write d13/f29 [786758,34607] 0 2024-02-19T19:06:14.744 INFO:tasks.workunit.client.0.smithi088.stdout:5/173: mknod d4/c39 0 2024-02-19T19:06:14.750 INFO:tasks.workunit.client.0.smithi088.stdout:5/174: creat d4/f3a x:0 0 0 2024-02-19T19:06:14.751 INFO:tasks.workunit.client.0.smithi088.stdout:6/110: sync 2024-02-19T19:06:14.752 INFO:tasks.workunit.client.0.smithi088.stdout:4/71: sync 2024-02-19T19:06:14.752 INFO:tasks.workunit.client.0.smithi088.stdout:3/243: sync 2024-02-19T19:06:14.752 INFO:tasks.workunit.client.0.smithi088.stdout:2/60: sync 2024-02-19T19:06:14.752 INFO:tasks.workunit.client.0.smithi088.stdout:1/205: sync 2024-02-19T19:06:14.753 INFO:tasks.workunit.client.0.smithi088.stdout:8/58: truncate f8 711943 0 2024-02-19T19:06:14.753 INFO:tasks.workunit.client.0.smithi088.stdout:2/61: write d3/fb [904883,50705] 0 2024-02-19T19:06:14.753 INFO:tasks.workunit.client.0.smithi088.stdout:0/129: dread d13/f29 [0,4194304] 0 2024-02-19T19:06:14.754 INFO:tasks.workunit.client.0.smithi088.stdout:5/175: mkdir d4/d3b 0 2024-02-19T19:06:14.754 INFO:tasks.workunit.client.0.smithi088.stdout:5/176: readlink d4/l32 0 2024-02-19T19:06:14.760 INFO:tasks.workunit.client.0.smithi088.stdout:1/206: creat d6/d10/d27/f40 x:0 0 0 2024-02-19T19:06:14.760 INFO:tasks.workunit.client.0.smithi088.stdout:0/130: dwrite d13/d24/d25/f26 [0,4194304] 0 2024-02-19T19:06:14.761 INFO:tasks.workunit.client.0.smithi088.stdout:8/59: mkdir d9/df 0 2024-02-19T19:06:14.762 INFO:tasks.workunit.client.0.smithi088.stdout:2/62: mknod d3/df/c12 0 2024-02-19T19:06:14.764 INFO:tasks.workunit.client.0.smithi088.stdout:5/177: creat d4/d1a/f3c x:0 0 0 2024-02-19T19:06:14.765 INFO:tasks.workunit.client.0.smithi088.stdout:3/244: mknod d4/c5f 0 2024-02-19T19:06:14.765 INFO:tasks.workunit.client.0.smithi088.stdout:5/178: write d4/fe [596744,121826] 0 2024-02-19T19:06:14.766 INFO:tasks.workunit.client.0.smithi088.stdout:3/245: chown d4/d1a/d1e/d31/f4b 407274 1 2024-02-19T19:06:14.767 INFO:tasks.workunit.client.0.smithi088.stdout:4/72: link d2/f7 d2/d8/f1a 0 2024-02-19T19:06:14.771 INFO:tasks.workunit.client.0.smithi088.stdout:2/63: unlink d3/c8 0 2024-02-19T19:06:14.774 INFO:tasks.workunit.client.0.smithi088.stdout:5/179: dwrite d4/d1a/f21 [0,4194304] 0 2024-02-19T19:06:14.775 INFO:tasks.workunit.client.0.smithi088.stdout:4/73: mknod d2/d5/d9/c1b 0 2024-02-19T19:06:14.776 INFO:tasks.workunit.client.0.smithi088.stdout:5/180: dread - d4/d1a/f3c zero size 2024-02-19T19:06:14.776 INFO:tasks.workunit.client.0.smithi088.stdout:8/60: symlink d9/df/l10 0 2024-02-19T19:06:14.781 INFO:tasks.workunit.client.0.smithi088.stdout:0/131: creat d13/f2c x:0 0 0 2024-02-19T19:06:14.782 INFO:tasks.workunit.client.0.smithi088.stdout:0/132: truncate d13/f15 887171 0 2024-02-19T19:06:14.784 INFO:tasks.workunit.client.0.smithi088.stdout:4/74: write d2/f7 [2320125,40032] 0 2024-02-19T19:06:14.786 INFO:tasks.workunit.client.0.smithi088.stdout:5/181: rename f0 to d4/d1a/d22/f3d 0 2024-02-19T19:06:14.789 INFO:tasks.workunit.client.0.smithi088.stdout:8/61: unlink l1 0 2024-02-19T19:06:14.791 INFO:tasks.workunit.client.0.smithi088.stdout:4/75: dwrite d2/d8/f1a [4194304,4194304] 0 2024-02-19T19:06:14.792 INFO:tasks.workunit.client.0.smithi088.stdout:4/76: fdatasync d2/f4 0 2024-02-19T19:06:14.799 INFO:tasks.workunit.client.0.smithi088.stdout:5/182: unlink d4/f2e 0 2024-02-19T19:06:14.799 INFO:tasks.workunit.client.0.smithi088.stdout:1/207: getdents d6/d10/d26 0 2024-02-19T19:06:14.800 INFO:tasks.workunit.client.0.smithi088.stdout:5/183: read d4/d1a/d22/f28 [2612007,71114] 0 2024-02-19T19:06:14.801 INFO:tasks.workunit.client.0.smithi088.stdout:4/77: dwrite d2/d5/fb [0,4194304] 0 2024-02-19T19:06:14.805 INFO:tasks.workunit.client.0.smithi088.stdout:0/133: symlink d13/l2d 0 2024-02-19T19:06:14.812 INFO:tasks.workunit.client.0.smithi088.stdout:5/184: symlink d4/d1a/l3e 0 2024-02-19T19:06:14.813 INFO:tasks.workunit.client.0.smithi088.stdout:2/64: getdents d3/df 0 2024-02-19T19:06:14.815 INFO:tasks.workunit.client.0.smithi088.stdout:4/78: creat d2/d5/f1c x:0 0 0 2024-02-19T19:06:14.819 INFO:tasks.workunit.client.0.smithi088.stdout:0/134: readlink l12 0 2024-02-19T19:06:14.820 INFO:tasks.workunit.client.0.smithi088.stdout:5/185: dwrite d4/d7/f31 [0,4194304] 0 2024-02-19T19:06:14.821 INFO:tasks.workunit.client.0.smithi088.stdout:1/208: symlink d6/l41 0 2024-02-19T19:06:14.821 INFO:tasks.workunit.client.0.smithi088.stdout:2/65: dread d3/fa [0,4194304] 0 2024-02-19T19:06:14.822 INFO:tasks.workunit.client.0.smithi088.stdout:2/66: readlink d3/l9 0 2024-02-19T19:06:14.824 INFO:tasks.workunit.client.0.smithi088.stdout:2/67: chown d3/le 28567901 1 2024-02-19T19:06:14.824 INFO:tasks.workunit.client.0.smithi088.stdout:1/209: chown d6/d10/d28/l32 230 1 2024-02-19T19:06:14.827 INFO:tasks.workunit.client.0.smithi088.stdout:0/135: dwrite d13/f1c [0,4194304] 0 2024-02-19T19:06:14.827 INFO:tasks.workunit.client.0.smithi088.stdout:4/79: symlink d2/d5/d9/l1d 0 2024-02-19T19:06:14.829 INFO:tasks.workunit.client.0.smithi088.stdout:4/80: dread - d2/d8/f16 zero size 2024-02-19T19:06:14.829 INFO:tasks.workunit.client.0.smithi088.stdout:0/136: write fb [3465194,89252] 0 2024-02-19T19:06:14.832 INFO:tasks.workunit.client.0.smithi088.stdout:4/81: write d2/d8/f14 [531468,123994] 0 2024-02-19T19:06:14.833 INFO:tasks.workunit.client.0.smithi088.stdout:5/186: unlink d4/d1a/f38 0 2024-02-19T19:06:14.835 INFO:tasks.workunit.client.0.smithi088.stdout:4/82: chown d2/d5/d9/l1d 2 1 2024-02-19T19:06:14.836 INFO:tasks.workunit.client.0.smithi088.stdout:2/68: dwrite d3/f4 [4194304,4194304] 0 2024-02-19T19:06:14.844 INFO:tasks.workunit.client.0.smithi088.stdout:1/210: write d6/d10/f11 [3773043,24712] 0 2024-02-19T19:06:14.845 INFO:tasks.workunit.client.0.smithi088.stdout:4/83: dread f0 [0,4194304] 0 2024-02-19T19:06:14.847 INFO:tasks.workunit.client.0.smithi088.stdout:4/84: stat d2/d5/l12 0 2024-02-19T19:06:14.847 INFO:tasks.workunit.client.0.smithi088.stdout:2/69: dwrite d3/df/f11 [0,4194304] 0 2024-02-19T19:06:14.853 INFO:tasks.workunit.client.0.smithi088.stdout:2/70: dread d3/f6 [0,4194304] 0 2024-02-19T19:06:14.856 INFO:tasks.workunit.client.0.smithi088.stdout:0/137: symlink d13/l2e 0 2024-02-19T19:06:14.857 INFO:tasks.workunit.client.0.smithi088.stdout:0/138: stat d13/f1a 0 2024-02-19T19:06:14.859 INFO:tasks.workunit.client.0.smithi088.stdout:2/71: dwrite d3/fb [0,4194304] 0 2024-02-19T19:06:14.860 INFO:tasks.workunit.client.0.smithi088.stdout:4/85: rename d2/d5/d9/l1d to d2/d5/d9/l1e 0 2024-02-19T19:06:14.862 INFO:tasks.workunit.client.0.smithi088.stdout:4/86: write d2/d8/f16 [381252,66907] 0 2024-02-19T19:06:14.869 INFO:tasks.workunit.client.0.smithi088.stdout:5/187: dread d4/d7/f1d [0,4194304] 0 2024-02-19T19:06:14.871 INFO:tasks.workunit.client.0.smithi088.stdout:5/188: dread - d4/d7/f2f zero size 2024-02-19T19:06:14.871 INFO:tasks.workunit.client.0.smithi088.stdout:2/72: symlink d3/l13 0 2024-02-19T19:06:14.872 INFO:tasks.workunit.client.0.smithi088.stdout:8/62: dwrite f8 [0,4194304] 0 2024-02-19T19:06:14.873 INFO:tasks.workunit.client.0.smithi088.stdout:8/63: readlink d9/lc 0 2024-02-19T19:06:14.873 INFO:tasks.workunit.client.0.smithi088.stdout:8/64: dread - f5 zero size 2024-02-19T19:06:14.877 INFO:tasks.workunit.client.0.smithi088.stdout:1/211: truncate f5 3042885 0 2024-02-19T19:06:14.881 INFO:tasks.workunit.client.0.smithi088.stdout:4/87: link d2/d8/f16 d2/d5/f1f 0 2024-02-19T19:06:14.881 INFO:tasks.workunit.client.0.smithi088.stdout:4/88: write d2/fd [100564,79840] 0 2024-02-19T19:06:14.882 INFO:tasks.workunit.client.0.smithi088.stdout:2/73: unlink d3/fa 0 2024-02-19T19:06:14.883 INFO:tasks.workunit.client.0.smithi088.stdout:0/139: truncate d13/d24/d25/f26 2299378 0 2024-02-19T19:06:14.884 INFO:tasks.workunit.client.0.smithi088.stdout:2/74: fsync d3/f6 0 2024-02-19T19:06:14.886 INFO:tasks.workunit.client.0.smithi088.stdout:8/65: symlink d9/l11 0 2024-02-19T19:06:14.887 INFO:tasks.workunit.client.0.smithi088.stdout:8/66: chown d9/lc 208063 1 2024-02-19T19:06:14.887 INFO:tasks.workunit.client.0.smithi088.stdout:1/212: symlink d6/d21/l42 0 2024-02-19T19:06:14.889 INFO:tasks.workunit.client.0.smithi088.stdout:4/89: dwrite d2/d5/f1c [0,4194304] 0 2024-02-19T19:06:14.891 INFO:tasks.workunit.client.0.smithi088.stdout:2/75: mkdir d3/d14 0 2024-02-19T19:06:14.893 INFO:tasks.workunit.client.0.smithi088.stdout:1/213: dread d6/fd [4194304,4194304] 0 2024-02-19T19:06:14.895 INFO:tasks.workunit.client.0.smithi088.stdout:8/67: mkdir d9/df/d12 0 2024-02-19T19:06:14.896 INFO:tasks.workunit.client.0.smithi088.stdout:5/189: dread d4/f9 [0,4194304] 0 2024-02-19T19:06:14.898 INFO:tasks.workunit.client.0.smithi088.stdout:2/76: dread d3/df/f11 [0,4194304] 0 2024-02-19T19:06:14.899 INFO:tasks.workunit.client.0.smithi088.stdout:0/140: mkdir d13/d24/d25/d2a/d2f 0 2024-02-19T19:06:14.901 INFO:tasks.workunit.client.0.smithi088.stdout:2/77: write d3/fb [867750,83675] 0 2024-02-19T19:06:14.901 INFO:tasks.workunit.client.0.smithi088.stdout:0/141: write fb [3382778,94282] 0 2024-02-19T19:06:14.901 INFO:tasks.workunit.client.0.smithi088.stdout:4/90: creat d2/d5/d9/f20 x:0 0 0 2024-02-19T19:06:14.902 INFO:tasks.workunit.client.0.smithi088.stdout:0/142: stat lf 0 2024-02-19T19:06:14.935 INFO:tasks.workunit.client.0.smithi088.stdout:4/91: fsync d2/d5/f1c 0 2024-02-19T19:06:14.937 INFO:tasks.workunit.client.0.smithi088.stdout:8/68: dwrite f6 [0,4194304] 0 2024-02-19T19:06:14.937 INFO:tasks.workunit.client.0.smithi088.stdout:8/69: stat l4 0 2024-02-19T19:06:14.938 INFO:tasks.workunit.client.0.smithi088.stdout:1/214: write d6/d10/d28/d29/f37 [3699695,22069] 0 2024-02-19T19:06:14.939 INFO:tasks.workunit.client.0.smithi088.stdout:0/143: write d13/d24/d25/f2b [1925248,124535] 0 2024-02-19T19:06:14.940 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:14 smithi132 ceph-mon[70676]: Reconfiguring mon.smithi132 (monmap changed)... 2024-02-19T19:06:14.940 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T19:06:14.940 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T19:06:14.940 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:14.940 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:14 smithi132 ceph-mon[70676]: Reconfiguring daemon mon.smithi132 on smithi132 2024-02-19T19:06:14.940 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:14 smithi132 ceph-mon[70676]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 366 op/s 2024-02-19T19:06:14.968 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:14 smithi088 ceph-mon[88643]: Reconfiguring mon.smithi132 (monmap changed)... 2024-02-19T19:06:14.968 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-19T19:06:14.968 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-19T19:06:14.969 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:14.969 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:14 smithi088 ceph-mon[88643]: Reconfiguring daemon mon.smithi132 on smithi132 2024-02-19T19:06:14.969 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:14 smithi088 ceph-mon[88643]: pgmap v41: 65 pgs: 65 active+clean; 257 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 3.4 MiB/s wr, 366 op/s 2024-02-19T19:06:14.985 INFO:tasks.workunit.client.0.smithi088.stdout:9/123: sync 2024-02-19T19:06:15.015 INFO:tasks.workunit.client.0.smithi088.stdout:2/78: rename d3/l5 to d3/d14/l15 0 2024-02-19T19:06:15.016 INFO:tasks.workunit.client.0.smithi088.stdout:4/92: rename d2/d5 to d2/d5/d21 22 2024-02-19T19:06:15.017 INFO:tasks.workunit.client.0.smithi088.stdout:2/79: chown d3/l9 2 1 2024-02-19T19:06:15.017 INFO:tasks.workunit.client.0.smithi088.stdout:5/190: dwrite d4/f8 [4194304,4194304] 0 2024-02-19T19:06:15.017 INFO:tasks.workunit.client.0.smithi088.stdout:8/70: creat d9/df/d12/f13 x:0 0 0 2024-02-19T19:06:15.018 INFO:tasks.workunit.client.0.smithi088.stdout:4/93: write d2/fd [1092914,67360] 0 2024-02-19T19:06:15.019 INFO:tasks.workunit.client.0.smithi088.stdout:5/191: dread - d4/f3a zero size 2024-02-19T19:06:15.020 INFO:tasks.workunit.client.0.smithi088.stdout:1/215: mknod d6/c43 0 2024-02-19T19:06:15.023 INFO:tasks.workunit.client.0.smithi088.stdout:5/192: read d4/d1a/d22/f3d [2564281,93488] 0 2024-02-19T19:06:15.023 INFO:tasks.workunit.client.0.smithi088.stdout:1/216: read d6/d10/f23 [1543617,12595] 0 2024-02-19T19:06:15.025 INFO:tasks.workunit.client.0.smithi088.stdout:8/71: symlink d9/df/l14 0 2024-02-19T19:06:15.028 INFO:tasks.workunit.client.0.smithi088.stdout:2/80: dread d3/f6 [0,4194304] 0 2024-02-19T19:06:15.031 INFO:tasks.workunit.client.0.smithi088.stdout:4/94: dwrite d2/f7 [0,4194304] 0 2024-02-19T19:06:15.032 INFO:tasks.workunit.client.0.smithi088.stdout:4/95: chown d2/d5 354601 1 2024-02-19T19:06:15.033 INFO:tasks.workunit.client.0.smithi088.stdout:1/217: dwrite d6/f1c [0,4194304] 0 2024-02-19T19:06:15.034 INFO:tasks.workunit.client.0.smithi088.stdout:1/218: fsync d6/ff 0 2024-02-19T19:06:15.034 INFO:tasks.workunit.client.0.smithi088.stdout:8/72: mknod d9/df/c15 0 2024-02-19T19:06:15.038 INFO:tasks.workunit.client.0.smithi088.stdout:4/96: dread d2/d8/f16 [0,4194304] 0 2024-02-19T19:06:15.041 INFO:tasks.workunit.client.0.smithi088.stdout:2/81: rename d3/l9 to d3/df/l16 0 2024-02-19T19:06:15.042 INFO:tasks.workunit.client.0.smithi088.stdout:5/193: mkdir d4/d3b/d3f 0 2024-02-19T19:06:15.046 INFO:tasks.workunit.client.0.smithi088.stdout:2/82: dread d3/fb [0,4194304] 0 2024-02-19T19:06:15.048 INFO:tasks.workunit.client.0.smithi088.stdout:1/219: symlink d6/d10/l44 0 2024-02-19T19:06:15.048 INFO:tasks.workunit.client.0.smithi088.stdout:2/83: write d3/f6 [2362736,126144] 0 2024-02-19T19:06:15.050 INFO:tasks.workunit.client.0.smithi088.stdout:5/194: dwrite d4/d7/f2c [0,4194304] 0 2024-02-19T19:06:15.051 INFO:tasks.workunit.client.0.smithi088.stdout:5/195: chown d4/d1a/l3e 5078 1 2024-02-19T19:06:15.052 INFO:tasks.workunit.client.0.smithi088.stdout:1/220: read d6/d10/d27/f2c [117250,33444] 0 2024-02-19T19:06:15.053 INFO:tasks.workunit.client.0.smithi088.stdout:8/73: creat d9/df/d12/f16 x:0 0 0 2024-02-19T19:06:15.058 INFO:tasks.workunit.client.0.smithi088.stdout:4/97: chown d2/d5/d9/l19 270910 1 2024-02-19T19:06:15.058 INFO:tasks.workunit.client.0.smithi088.stdout:9/124: dread d1/f6 [0,4194304] 0 2024-02-19T19:06:15.075 INFO:tasks.workunit.client.0.smithi088.stdout:8/74: dwrite d9/df/d12/f16 [0,4194304] 0 2024-02-19T19:06:15.077 INFO:tasks.workunit.client.0.smithi088.stdout:1/221: unlink d6/d10/f38 0 2024-02-19T19:06:15.077 INFO:tasks.workunit.client.0.smithi088.stdout:5/196: write d4/d7/f1d [229871,11272] 0 2024-02-19T19:06:15.078 INFO:tasks.workunit.client.0.smithi088.stdout:4/98: symlink d2/d8/l22 0 2024-02-19T19:06:15.078 INFO:tasks.workunit.client.0.smithi088.stdout:8/75: write f6 [3824731,121976] 0 2024-02-19T19:06:15.085 INFO:tasks.workunit.client.0.smithi088.stdout:9/125: dwrite d1/d8/d14/d23/f25 [0,4194304] 0 2024-02-19T19:06:15.092 INFO:tasks.workunit.client.0.smithi088.stdout:4/99: creat d2/d5/f23 x:0 0 0 2024-02-19T19:06:15.093 INFO:tasks.workunit.client.0.smithi088.stdout:4/100: chown d2/d5/d9 5925263 1 2024-02-19T19:06:15.094 INFO:tasks.workunit.client.0.smithi088.stdout:4/101: truncate d2/d5/f23 439731 0 2024-02-19T19:06:15.098 INFO:tasks.workunit.client.0.smithi088.stdout:8/76: rename d9/la to d9/df/l17 0 2024-02-19T19:06:15.102 INFO:tasks.workunit.client.0.smithi088.stdout:8/77: dread f6 [0,4194304] 0 2024-02-19T19:06:15.109 INFO:tasks.workunit.client.0.smithi088.stdout:2/84: truncate d3/f4 8375652 0 2024-02-19T19:06:15.110 INFO:tasks.workunit.client.0.smithi088.stdout:8/78: dwrite f6 [0,4194304] 0 2024-02-19T19:06:15.113 INFO:tasks.workunit.client.0.smithi088.stdout:5/197: dread d4/d7/f15 [0,4194304] 0 2024-02-19T19:06:15.127 INFO:tasks.workunit.client.0.smithi088.stdout:8/79: mknod d9/df/d12/c18 0 2024-02-19T19:06:15.128 INFO:tasks.workunit.client.0.smithi088.stdout:8/80: chown d9/df/l14 9661 1 2024-02-19T19:06:15.128 INFO:tasks.workunit.client.0.smithi088.stdout:8/81: dread - f5 zero size 2024-02-19T19:06:15.129 INFO:tasks.workunit.client.0.smithi088.stdout:5/198: symlink d4/l40 0 2024-02-19T19:06:15.130 INFO:tasks.workunit.client.0.smithi088.stdout:9/126: truncate d1/d8/d14/d18/f1a 45624 0 2024-02-19T19:06:15.132 INFO:tasks.workunit.client.0.smithi088.stdout:1/222: getdents d6/d10/d28/d29 0 2024-02-19T19:06:15.135 INFO:tasks.workunit.client.0.smithi088.stdout:4/102: truncate d2/f4 3376271 0 2024-02-19T19:06:15.137 INFO:tasks.workunit.client.0.smithi088.stdout:8/82: unlink d9/df/l10 0 2024-02-19T19:06:15.142 INFO:tasks.workunit.client.0.smithi088.stdout:9/127: unlink d1/d13/l19 0 2024-02-19T19:06:15.142 INFO:tasks.workunit.client.0.smithi088.stdout:9/128: fsync d1/d13/fd 0 2024-02-19T19:06:15.143 INFO:tasks.workunit.client.0.smithi088.stdout:9/129: write d1/d8/d14/d23/f26 [458868,58006] 0 2024-02-19T19:06:15.143 INFO:tasks.workunit.client.0.smithi088.stdout:1/223: creat d6/d10/f45 x:0 0 0 2024-02-19T19:06:15.150 INFO:tasks.workunit.client.0.smithi088.stdout:4/103: mknod d2/d8/c24 0 2024-02-19T19:06:15.150 INFO:tasks.workunit.client.0.smithi088.stdout:9/130: dwrite d1/d8/f28 [0,4194304] 0 2024-02-19T19:06:15.152 INFO:tasks.workunit.client.0.smithi088.stdout:8/83: creat d9/df/f19 x:0 0 0 2024-02-19T19:06:15.152 INFO:tasks.workunit.client.0.smithi088.stdout:8/84: write f8 [3035153,25402] 0 2024-02-19T19:06:15.158 INFO:tasks.workunit.client.0.smithi088.stdout:2/85: truncate d3/f4 5692949 0 2024-02-19T19:06:15.159 INFO:tasks.workunit.client.0.smithi088.stdout:4/104: mkdir d2/d5/d25 0 2024-02-19T19:06:15.160 INFO:tasks.workunit.client.0.smithi088.stdout:9/131: mknod d1/d8/d14/d23/c29 0 2024-02-19T19:06:15.162 INFO:tasks.workunit.client.0.smithi088.stdout:8/85: creat d9/df/f1a x:0 0 0 2024-02-19T19:06:15.163 INFO:tasks.workunit.client.0.smithi088.stdout:8/86: write f8 [2094565,109381] 0 2024-02-19T19:06:15.163 INFO:tasks.workunit.client.0.smithi088.stdout:5/199: rmdir d4/d3b/d3f 0 2024-02-19T19:06:15.167 INFO:tasks.workunit.client.0.smithi088.stdout:2/86: creat d3/df/f17 x:0 0 0 2024-02-19T19:06:15.170 INFO:tasks.workunit.client.0.smithi088.stdout:4/105: unlink d2/d5/f1f 0 2024-02-19T19:06:15.170 INFO:tasks.workunit.client.0.smithi088.stdout:4/106: fsync d2/f15 0 2024-02-19T19:06:15.171 INFO:tasks.workunit.client.0.smithi088.stdout:9/132: creat d1/d8/d14/f2a x:0 0 0 2024-02-19T19:06:15.175 INFO:tasks.workunit.client.0.smithi088.stdout:5/200: mkdir d4/d1a/d41 0 2024-02-19T19:06:15.179 INFO:tasks.workunit.client.0.smithi088.stdout:8/87: creat d9/df/d12/f1b x:0 0 0 2024-02-19T19:06:15.181 INFO:tasks.workunit.client.0.smithi088.stdout:8/88: truncate d9/df/d12/f13 750283 0 2024-02-19T19:06:15.182 INFO:tasks.workunit.client.0.smithi088.stdout:5/201: dwrite d4/f8 [8388608,4194304] 0 2024-02-19T19:06:15.188 INFO:tasks.workunit.client.0.smithi088.stdout:1/224: write d6/d21/f3c [1968265,91124] 0 2024-02-19T19:06:15.190 INFO:tasks.workunit.client.0.smithi088.stdout:8/89: dwrite d9/df/f19 [0,4194304] 0 2024-02-19T19:06:15.191 INFO:tasks.workunit.client.0.smithi088.stdout:8/90: write f5 [1035086,113310] 0 2024-02-19T19:06:15.193 INFO:tasks.workunit.client.0.smithi088.stdout:5/202: rename d4/f3a to d4/f42 0 2024-02-19T19:06:15.196 INFO:tasks.workunit.client.0.smithi088.stdout:4/107: mkdir d2/d5/d25/d26 0 2024-02-19T19:06:15.204 INFO:tasks.workunit.client.0.smithi088.stdout:8/91: rename d9/df/d12 to d9/df/d1c 0 2024-02-19T19:06:15.205 INFO:tasks.workunit.client.0.smithi088.stdout:8/92: read - d9/df/f1a zero size 2024-02-19T19:06:15.205 INFO:tasks.workunit.client.0.smithi088.stdout:8/93: chown d9/l11 47 1 2024-02-19T19:06:15.206 INFO:tasks.workunit.client.0.smithi088.stdout:1/225: fdatasync d6/f1e 0 2024-02-19T19:06:15.207 INFO:tasks.workunit.client.0.smithi088.stdout:2/87: write d3/fb [4746735,90577] 0 2024-02-19T19:06:15.208 INFO:tasks.workunit.client.0.smithi088.stdout:5/203: mknod d4/d1a/d41/c43 0 2024-02-19T19:06:15.210 INFO:tasks.workunit.client.0.smithi088.stdout:4/108: rename d2/d8/l17 to d2/d5/l27 0 2024-02-19T19:06:15.211 INFO:tasks.workunit.client.0.smithi088.stdout:4/109: fdatasync d2/d8/f1a 0 2024-02-19T19:06:15.212 INFO:tasks.workunit.client.0.smithi088.stdout:4/110: write d2/d5/f23 [1242457,113814] 0 2024-02-19T19:06:15.214 INFO:tasks.workunit.client.0.smithi088.stdout:5/204: mknod d4/d1a/d41/c44 0 2024-02-19T19:06:15.215 INFO:tasks.workunit.client.0.smithi088.stdout:9/133: truncate d1/d8/d14/d23/f26 333428 0 2024-02-19T19:06:15.216 INFO:tasks.workunit.client.0.smithi088.stdout:9/134: chown d1/d13/l1d 0 1 2024-02-19T19:06:15.219 INFO:tasks.workunit.client.0.smithi088.stdout:1/226: rename d6/c25 to d6/d10/d26/c46 0 2024-02-19T19:06:15.220 INFO:tasks.workunit.client.0.smithi088.stdout:5/205: dwrite d4/d1a/f21 [0,4194304] 0 2024-02-19T19:06:15.222 INFO:tasks.workunit.client.0.smithi088.stdout:4/111: mkdir d2/d28 0 2024-02-19T19:06:15.227 INFO:tasks.workunit.client.0.smithi088.stdout:1/227: symlink d6/d10/d28/d29/l47 0 2024-02-19T19:06:15.229 INFO:tasks.workunit.client.0.smithi088.stdout:5/206: mknod d4/c45 0 2024-02-19T19:06:15.229 INFO:tasks.workunit.client.0.smithi088.stdout:1/228: write d6/d10/f15 [971288,126174] 0 2024-02-19T19:06:15.230 INFO:tasks.workunit.client.0.smithi088.stdout:5/207: write d4/d7/f33 [348794,34497] 0 2024-02-19T19:06:15.233 INFO:tasks.workunit.client.0.smithi088.stdout:4/112: rename d2/d5/fc to d2/d8/f29 0 2024-02-19T19:06:15.240 INFO:tasks.workunit.client.0.smithi088.stdout:1/229: symlink d6/d10/d28/l48 0 2024-02-19T19:06:15.240 INFO:tasks.workunit.client.0.smithi088.stdout:1/230: chown d6/d10/d27/l31 118 1 2024-02-19T19:06:15.241 INFO:tasks.workunit.client.0.smithi088.stdout:5/208: creat d4/d1a/f46 x:0 0 0 2024-02-19T19:06:15.241 INFO:tasks.workunit.client.0.smithi088.stdout:1/231: chown d6/f1c 1867697 1 2024-02-19T19:06:15.241 INFO:tasks.workunit.client.0.smithi088.stdout:5/209: stat d4/f42 0 2024-02-19T19:06:15.242 INFO:tasks.workunit.client.0.smithi088.stdout:2/88: link d3/df/l16 d3/l18 0 2024-02-19T19:06:15.245 INFO:tasks.workunit.client.0.smithi088.stdout:4/113: mknod d2/d5/d9/c2a 0 2024-02-19T19:06:15.249 INFO:tasks.workunit.client.0.smithi088.stdout:9/135: link d1/d8/d14/f2a d1/f2b 0 2024-02-19T19:06:15.250 INFO:tasks.workunit.client.0.smithi088.stdout:1/232: mknod d6/d10/d28/c49 0 2024-02-19T19:06:15.254 INFO:tasks.workunit.client.0.smithi088.stdout:2/89: creat d3/d14/f19 x:0 0 0 2024-02-19T19:06:15.255 INFO:tasks.workunit.client.0.smithi088.stdout:4/114: mknod d2/d5/c2b 0 2024-02-19T19:06:15.258 INFO:tasks.workunit.client.0.smithi088.stdout:1/233: mkdir d6/d10/d26/d4a 0 2024-02-19T19:06:15.259 INFO:tasks.workunit.client.0.smithi088.stdout:5/210: dwrite d4/f42 [0,4194304] 0 2024-02-19T19:06:15.265 INFO:tasks.workunit.client.0.smithi088.stdout:1/234: creat d6/d10/f4b x:0 0 0 2024-02-19T19:06:15.268 INFO:tasks.workunit.client.0.smithi088.stdout:4/115: dwrite d2/d5/f18 [0,4194304] 0 2024-02-19T19:06:15.269 INFO:tasks.workunit.client.0.smithi088.stdout:9/136: creat d1/f2c x:0 0 0 2024-02-19T19:06:15.270 INFO:tasks.workunit.client.0.smithi088.stdout:1/235: read - d6/d10/d27/f40 zero size 2024-02-19T19:06:15.270 INFO:tasks.workunit.client.0.smithi088.stdout:9/137: chown d1/d21 260437205 1 2024-02-19T19:06:15.270 INFO:tasks.workunit.client.0.smithi088.stdout:2/90: dread d3/df/f11 [0,4194304] 0 2024-02-19T19:06:15.271 INFO:tasks.workunit.client.0.smithi088.stdout:1/236: stat d6/c43 0 2024-02-19T19:06:15.272 INFO:tasks.workunit.client.0.smithi088.stdout:4/116: mknod d2/c2c 0 2024-02-19T19:06:15.273 INFO:tasks.workunit.client.0.smithi088.stdout:5/211: dwrite d4/d7/f1d [0,4194304] 0 2024-02-19T19:06:15.275 INFO:tasks.workunit.client.0.smithi088.stdout:1/237: write d6/d21/f2f [870890,125499] 0 2024-02-19T19:06:15.279 INFO:tasks.workunit.client.0.smithi088.stdout:4/117: symlink d2/d5/d25/l2d 0 2024-02-19T19:06:15.284 INFO:tasks.workunit.client.0.smithi088.stdout:1/238: rename d6/d10/d27 to d6/d10/d26/d4c 0 2024-02-19T19:06:15.286 INFO:tasks.workunit.client.0.smithi088.stdout:5/212: dwrite d4/d1a/f3c [0,4194304] 0 2024-02-19T19:06:15.288 INFO:tasks.workunit.client.0.smithi088.stdout:5/213: write d4/f42 [3058669,114127] 0 2024-02-19T19:06:15.291 INFO:tasks.workunit.client.0.smithi088.stdout:1/239: creat d6/d10/f4d x:0 0 0 2024-02-19T19:06:15.296 INFO:tasks.workunit.client.0.smithi088.stdout:5/214: truncate d4/d1a/d22/f28 3831499 0 2024-02-19T19:06:15.298 INFO:tasks.workunit.client.0.smithi088.stdout:1/240: mkdir d6/d10/d28/d29/d4e 0 2024-02-19T19:06:15.300 INFO:tasks.workunit.client.0.smithi088.stdout:9/138: getdents d1/d8/d14/d23 0 2024-02-19T19:06:15.301 INFO:tasks.workunit.client.0.smithi088.stdout:5/215: mknod d4/d3b/c47 0 2024-02-19T19:06:15.301 INFO:tasks.workunit.client.0.smithi088.stdout:2/91: write d3/f4 [5413349,109545] 0 2024-02-19T19:06:15.302 INFO:tasks.workunit.client.0.smithi088.stdout:5/216: chown d4/c45 23866408 1 2024-02-19T19:06:15.304 INFO:tasks.workunit.client.0.smithi088.stdout:5/217: truncate d4/d1a/d22/f26 714501 0 2024-02-19T19:06:15.305 INFO:tasks.workunit.client.0.smithi088.stdout:1/241: rename d6/d21/f2f to d6/d10/d28/f4f 0 2024-02-19T19:06:15.306 INFO:tasks.workunit.client.0.smithi088.stdout:1/242: write d6/d21/f3c [1508327,338] 0 2024-02-19T19:06:15.309 INFO:tasks.workunit.client.0.smithi088.stdout:2/92: stat d3/df/l16 0 2024-02-19T19:06:15.312 INFO:tasks.workunit.client.0.smithi088.stdout:5/218: rename d4/c1f to d4/d1a/d41/c48 0 2024-02-19T19:06:15.315 INFO:tasks.workunit.client.0.smithi088.stdout:4/118: dwrite d2/f4 [0,4194304] 0 2024-02-19T19:06:15.315 INFO:tasks.workunit.client.0.smithi088.stdout:2/93: mknod d3/d14/c1a 0 2024-02-19T19:06:15.317 INFO:tasks.workunit.client.0.smithi088.stdout:1/243: dwrite d6/d10/d26/d4c/f40 [0,4194304] 0 2024-02-19T19:06:15.318 INFO:tasks.workunit.client.0.smithi088.stdout:5/219: mknod d4/d3b/c49 0 2024-02-19T19:06:15.319 INFO:tasks.workunit.client.0.smithi088.stdout:2/94: mkdir d3/d1b 0 2024-02-19T19:06:15.324 INFO:tasks.workunit.client.0.smithi088.stdout:5/220: mknod d4/c4a 0 2024-02-19T19:06:15.325 INFO:tasks.workunit.client.0.smithi088.stdout:9/139: dread d1/d8/d14/f16 [0,4194304] 0 2024-02-19T19:06:15.325 INFO:tasks.workunit.client.0.smithi088.stdout:4/119: read d2/fd [233495,52653] 0 2024-02-19T19:06:15.326 INFO:tasks.workunit.client.0.smithi088.stdout:4/120: chown d2/d8/ce 2 1 2024-02-19T19:06:15.329 INFO:tasks.workunit.client.0.smithi088.stdout:1/244: dread d6/d10/d26/d4c/f2c [0,4194304] 0 2024-02-19T19:06:15.333 INFO:tasks.workunit.client.0.smithi088.stdout:5/221: dwrite d4/d7/f31 [0,4194304] 0 2024-02-19T19:06:15.336 INFO:tasks.workunit.client.0.smithi088.stdout:2/95: rmdir d3/d14 39 2024-02-19T19:06:15.343 INFO:tasks.workunit.client.0.smithi088.stdout:1/245: mknod d6/d21/c50 0 2024-02-19T19:06:15.356 INFO:tasks.workunit.client.0.smithi088.stdout:2/96: creat d3/d1b/f1c x:0 0 0 2024-02-19T19:06:15.356 INFO:tasks.workunit.client.0.smithi088.stdout:4/121: rmdir d2/d28 0 2024-02-19T19:06:15.357 INFO:tasks.workunit.client.0.smithi088.stdout:9/140: rename d1/d8/d14 to d1/d21/d2d 0 2024-02-19T19:06:15.358 INFO:tasks.workunit.client.0.smithi088.stdout:1/246: mknod d6/c51 0 2024-02-19T19:06:15.359 INFO:tasks.workunit.client.0.smithi088.stdout:9/141: write d1/d13/fd [4822317,109901] 0 2024-02-19T19:06:15.360 INFO:tasks.workunit.client.0.smithi088.stdout:9/142: fsync d1/d21/d2d/d23/f25 0 2024-02-19T19:06:15.360 INFO:tasks.workunit.client.0.smithi088.stdout:9/143: chown d1/lb 2769627 1 2024-02-19T19:06:15.362 INFO:tasks.workunit.client.0.smithi088.stdout:4/122: dread d2/fd [0,4194304] 0 2024-02-19T19:06:15.363 INFO:tasks.workunit.client.0.smithi088.stdout:5/222: dread d4/f8 [0,4194304] 0 2024-02-19T19:06:15.364 INFO:tasks.workunit.client.0.smithi088.stdout:9/144: creat d1/d21/d2d/d18/d1f/f2e x:0 0 0 2024-02-19T19:06:15.364 INFO:tasks.workunit.client.0.smithi088.stdout:5/223: dread - d4/d7/f36 zero size 2024-02-19T19:06:15.365 INFO:tasks.workunit.client.0.smithi088.stdout:2/97: creat d3/d14/f1d x:0 0 0 2024-02-19T19:06:15.365 INFO:tasks.workunit.client.0.smithi088.stdout:9/145: fdatasync d1/d21/d2d/d18/d1f/f2e 0 2024-02-19T19:06:15.365 INFO:tasks.workunit.client.0.smithi088.stdout:5/224: chown d4/d1a/f37 5422 1 2024-02-19T19:06:15.365 INFO:tasks.workunit.client.0.smithi088.stdout:2/98: stat d3/f6 0 2024-02-19T19:06:15.375 INFO:tasks.workunit.client.0.smithi088.stdout:7/159: sync 2024-02-19T19:06:15.375 INFO:tasks.workunit.client.0.smithi088.stdout:6/111: sync 2024-02-19T19:06:15.384 INFO:tasks.workunit.client.0.smithi088.stdout:4/123: rmdir d2/d5 39 2024-02-19T19:06:15.385 INFO:tasks.workunit.client.0.smithi088.stdout:2/99: rename d3/d1b to d3/d1b/d1e 22 2024-02-19T19:06:15.385 INFO:tasks.workunit.client.0.smithi088.stdout:6/112: chown d5/c16 45964227 1 2024-02-19T19:06:15.386 INFO:tasks.workunit.client.0.smithi088.stdout:2/100: read d3/f6 [438862,21776] 0 2024-02-19T19:06:15.386 INFO:tasks.workunit.client.0.smithi088.stdout:7/160: creat d14/d27/f36 x:0 0 0 2024-02-19T19:06:15.390 INFO:tasks.workunit.client.0.smithi088.stdout:9/146: mkdir d1/d21/d2f 0 2024-02-19T19:06:15.510 INFO:tasks.workunit.client.0.smithi088.stdout:3/246: sync 2024-02-19T19:06:15.511 INFO:tasks.workunit.client.0.smithi088.stdout:0/144: sync 2024-02-19T19:06:15.528 INFO:tasks.workunit.client.0.smithi088.stdout:6/113: rename d5/d19/f20 to d5/de/f22 0 2024-02-19T19:06:15.528 INFO:tasks.workunit.client.0.smithi088.stdout:7/161: rmdir d14/d28 39 2024-02-19T19:06:15.528 INFO:tasks.workunit.client.0.smithi088.stdout:4/124: creat d2/d5/d25/f2e x:0 0 0 2024-02-19T19:06:15.530 INFO:tasks.workunit.client.0.smithi088.stdout:2/101: mkdir d3/d1f 0 2024-02-19T19:06:15.532 INFO:tasks.workunit.client.0.smithi088.stdout:0/145: mknod d13/d24/d25/d2a/c30 0 2024-02-19T19:06:15.534 INFO:tasks.workunit.client.0.smithi088.stdout:4/125: rmdir d2/d5/d9 39 2024-02-19T19:06:15.536 INFO:tasks.workunit.client.0.smithi088.stdout:2/102: symlink d3/df/l20 0 2024-02-19T19:06:15.537 INFO:tasks.workunit.client.0.smithi088.stdout:7/162: symlink d14/l37 0 2024-02-19T19:06:15.537 INFO:tasks.workunit.client.0.smithi088.stdout:9/147: dwrite d1/d21/d2d/d18/d1f/f2e [0,4194304] 0 2024-02-19T19:06:15.540 INFO:tasks.workunit.client.0.smithi088.stdout:7/163: stat d14/f22 0 2024-02-19T19:06:15.541 INFO:tasks.workunit.client.0.smithi088.stdout:0/146: truncate f3 1742660 0 2024-02-19T19:06:15.541 INFO:tasks.workunit.client.0.smithi088.stdout:2/103: dread - d3/d14/f1d zero size 2024-02-19T19:06:15.543 INFO:tasks.workunit.client.0.smithi088.stdout:3/247: dwrite d4/d1a/d1e/d31/d32/f4c [0,4194304] 0 2024-02-19T19:06:15.545 INFO:tasks.workunit.client.0.smithi088.stdout:4/126: mknod d2/d8/c2f 0 2024-02-19T19:06:15.546 INFO:tasks.workunit.client.0.smithi088.stdout:5/225: dread d4/d1a/f1b [0,4194304] 0 2024-02-19T19:06:15.550 INFO:tasks.workunit.client.0.smithi088.stdout:2/104: dread d3/fb [0,4194304] 0 2024-02-19T19:06:15.550 INFO:tasks.workunit.client.0.smithi088.stdout:9/148: symlink d1/d21/d2d/d23/l30 0 2024-02-19T19:06:15.554 INFO:tasks.workunit.client.0.smithi088.stdout:7/164: creat d14/d1d/f38 x:0 0 0 2024-02-19T19:06:15.557 INFO:tasks.workunit.client.0.smithi088.stdout:9/149: dwrite d1/d21/d2d/d23/f25 [0,4194304] 0 2024-02-19T19:06:15.563 INFO:tasks.workunit.client.0.smithi088.stdout:3/248: rename d4/d1a/d1e/d5a/f5e to d4/d1a/d23/f60 0 2024-02-19T19:06:15.563 INFO:tasks.workunit.client.0.smithi088.stdout:7/165: dwrite d14/f30 [0,4194304] 0 2024-02-19T19:06:15.564 INFO:tasks.workunit.client.0.smithi088.stdout:4/127: creat d2/d5/d25/f30 x:0 0 0 2024-02-19T19:06:15.569 INFO:tasks.workunit.client.0.smithi088.stdout:4/128: dread d2/d5/fb [0,4194304] 0 2024-02-19T19:06:15.570 INFO:tasks.workunit.client.0.smithi088.stdout:5/226: rename d4/d7/f23 to d4/d7/f4b 0 2024-02-19T19:06:15.570 INFO:tasks.workunit.client.0.smithi088.stdout:4/129: write d2/d5/fb [3804857,45986] 0 2024-02-19T19:06:15.571 INFO:tasks.workunit.client.0.smithi088.stdout:5/227: chown d4/d7/f1d 12579124 1 2024-02-19T19:06:15.571 INFO:tasks.workunit.client.0.smithi088.stdout:4/130: fdatasync d2/d5/d25/f2e 0 2024-02-19T19:06:15.572 INFO:tasks.workunit.client.0.smithi088.stdout:6/114: link d5/de/c1c d5/d19/c23 0 2024-02-19T19:06:15.575 INFO:tasks.workunit.client.0.smithi088.stdout:6/115: read d5/d7/f14 [2146921,28230] 0 2024-02-19T19:06:15.576 INFO:tasks.workunit.client.0.smithi088.stdout:0/147: mknod d13/c31 0 2024-02-19T19:06:15.579 INFO:tasks.workunit.client.0.smithi088.stdout:4/131: dwrite d2/d5/f1c [4194304,4194304] 0 2024-02-19T19:06:15.581 INFO:tasks.workunit.client.0.smithi088.stdout:9/150: rename d1/d8/c11 to d1/d21/c31 0 2024-02-19T19:06:15.582 INFO:tasks.workunit.client.0.smithi088.stdout:3/249: fsync d4/ff 0 2024-02-19T19:06:15.583 INFO:tasks.workunit.client.0.smithi088.stdout:7/166: creat d14/d1d/f39 x:0 0 0 2024-02-19T19:06:15.587 INFO:tasks.workunit.client.0.smithi088.stdout:5/228: creat d4/d1a/d22/f4c x:0 0 0 2024-02-19T19:06:15.589 INFO:tasks.workunit.client.0.smithi088.stdout:2/105: symlink d3/l21 0 2024-02-19T19:06:15.590 INFO:tasks.workunit.client.0.smithi088.stdout:8/94: sync 2024-02-19T19:06:15.590 INFO:tasks.workunit.client.0.smithi088.stdout:8/95: readlink d9/ld 0 2024-02-19T19:06:15.591 INFO:tasks.workunit.client.0.smithi088.stdout:8/96: truncate d9/df/d1c/f13 965640 0 2024-02-19T19:06:15.599 INFO:tasks.workunit.client.0.smithi088.stdout:4/132: rmdir d2 39 2024-02-19T19:06:15.600 INFO:tasks.workunit.client.0.smithi088.stdout:3/250: fsync d4/d1a/d23/f60 0 2024-02-19T19:06:15.600 INFO:tasks.workunit.client.0.smithi088.stdout:9/151: dwrite d1/d21/d2d/f16 [0,4194304] 0 2024-02-19T19:06:15.604 INFO:tasks.workunit.client.0.smithi088.stdout:0/148: read f3 [616830,126288] 0 2024-02-19T19:06:15.605 INFO:tasks.workunit.client.0.smithi088.stdout:7/167: rename d14/c33 to d14/d1d/c3a 0 2024-02-19T19:06:15.606 INFO:tasks.workunit.client.0.smithi088.stdout:0/149: write d13/f1c [3033081,14592] 0 2024-02-19T19:06:15.609 INFO:tasks.workunit.client.0.smithi088.stdout:5/229: creat d4/f4d x:0 0 0 2024-02-19T19:06:15.609 INFO:tasks.workunit.client.0.smithi088.stdout:5/230: readlink d4/l40 0 2024-02-19T19:06:15.611 INFO:tasks.workunit.client.0.smithi088.stdout:8/97: symlink d9/df/d1c/l1d 0 2024-02-19T19:06:15.612 INFO:tasks.workunit.client.0.smithi088.stdout:8/98: chown f5 386721 1 2024-02-19T19:06:15.615 INFO:tasks.workunit.client.0.smithi088.stdout:3/251: unlink d4/d1a/d1e/d31/f4b 0 2024-02-19T19:06:15.616 INFO:tasks.workunit.client.0.smithi088.stdout:3/252: chown d4/d1a/d1e/d31/d32/f50 2 1 2024-02-19T19:06:15.620 INFO:tasks.workunit.client.0.smithi088.stdout:9/152: rename d1/l15 to d1/d21/l32 0 2024-02-19T19:06:15.621 INFO:tasks.workunit.client.0.smithi088.stdout:3/253: dwrite d4/f6 [0,4194304] 0 2024-02-19T19:06:15.623 INFO:tasks.workunit.client.0.smithi088.stdout:5/231: creat d4/d1a/d22/f4e x:0 0 0 2024-02-19T19:06:15.623 INFO:tasks.workunit.client.0.smithi088.stdout:6/116: link d5/de/c12 d5/d8/c24 0 2024-02-19T19:06:15.624 INFO:tasks.workunit.client.0.smithi088.stdout:6/117: chown d5 94246525 1 2024-02-19T19:06:15.624 INFO:tasks.workunit.client.0.smithi088.stdout:5/232: read d4/f11 [106300,24451] 0 2024-02-19T19:06:15.631 INFO:tasks.workunit.client.0.smithi088.stdout:7/168: mkdir d14/d28/d3b 0 2024-02-19T19:06:15.632 INFO:tasks.workunit.client.0.smithi088.stdout:7/169: dread - d14/d27/f36 zero size 2024-02-19T19:06:15.633 INFO:tasks.workunit.client.0.smithi088.stdout:7/170: chown d14/f29 56 1 2024-02-19T19:06:15.633 INFO:tasks.workunit.client.0.smithi088.stdout:7/171: chown d14/f17 4804721 1 2024-02-19T19:06:15.634 INFO:tasks.workunit.client.0.smithi088.stdout:2/106: link d3/fb d3/df/f22 0 2024-02-19T19:06:15.636 INFO:tasks.workunit.client.0.smithi088.stdout:3/254: creat d4/d1a/d23/f61 x:0 0 0 2024-02-19T19:06:15.638 INFO:tasks.workunit.client.0.smithi088.stdout:8/99: rmdir d9/df 39 2024-02-19T19:06:15.639 INFO:tasks.workunit.client.0.smithi088.stdout:9/153: truncate d1/d21/d2d/d23/f26 1065921 0 2024-02-19T19:06:15.641 INFO:tasks.workunit.client.0.smithi088.stdout:6/118: write d5/de/f22 [584608,65754] 0 2024-02-19T19:06:15.642 INFO:tasks.workunit.client.0.smithi088.stdout:6/119: chown d5/d8/l1d 4 1 2024-02-19T19:06:15.643 INFO:tasks.workunit.client.0.smithi088.stdout:5/233: symlink d4/d7/l4f 0 2024-02-19T19:06:15.644 INFO:tasks.workunit.client.0.smithi088.stdout:6/120: fsync d5/de/f22 0 2024-02-19T19:06:15.646 INFO:tasks.workunit.client.0.smithi088.stdout:7/172: mkdir d14/d1d/d3c 0 2024-02-19T19:06:15.647 INFO:tasks.workunit.client.0.smithi088.stdout:7/173: truncate d14/d1d/f39 154591 0 2024-02-19T19:06:15.647 INFO:tasks.workunit.client.0.smithi088.stdout:2/107: chown d3/df/f22 35 1 2024-02-19T19:06:15.648 INFO:tasks.workunit.client.0.smithi088.stdout:3/255: symlink d4/d1a/d1e/l62 0 2024-02-19T19:06:15.649 INFO:tasks.workunit.client.0.smithi088.stdout:7/174: read d14/f2a [2954938,90916] 0 2024-02-19T19:06:15.651 INFO:tasks.workunit.client.0.smithi088.stdout:9/154: truncate d1/fc 3630709 0 2024-02-19T19:06:15.651 INFO:tasks.workunit.client.0.smithi088.stdout:3/256: dread d4/d12/f2e [0,4194304] 0 2024-02-19T19:06:15.655 INFO:tasks.workunit.client.0.smithi088.stdout:5/234: write d4/f11 [1031999,2343] 0 2024-02-19T19:06:15.656 INFO:tasks.workunit.client.0.smithi088.stdout:5/235: readlink d4/d1a/d22/lc 0 2024-02-19T19:06:15.656 INFO:tasks.workunit.client.0.smithi088.stdout:5/236: chown d4/d7/f33 0 1 2024-02-19T19:06:15.658 INFO:tasks.workunit.client.0.smithi088.stdout:6/121: dread d5/d7/f15 [0,4194304] 0 2024-02-19T19:06:15.659 INFO:tasks.workunit.client.0.smithi088.stdout:4/133: truncate d2/d5/f23 324718 0 2024-02-19T19:06:15.660 INFO:tasks.workunit.client.0.smithi088.stdout:4/134: write d2/f7 [6949314,63338] 0 2024-02-19T19:06:15.661 INFO:tasks.workunit.client.0.smithi088.stdout:4/135: truncate d2/d8/f29 747398 0 2024-02-19T19:06:15.666 INFO:tasks.workunit.client.0.smithi088.stdout:8/100: mkdir d9/df/d1c/d1e 0 2024-02-19T19:06:15.669 INFO:tasks.workunit.client.0.smithi088.stdout:5/237: symlink d4/d1a/l50 0 2024-02-19T19:06:15.677 INFO:tasks.workunit.client.0.smithi088.stdout:4/136: chown d2/d5/d9/c1b 2051859 1 2024-02-19T19:06:15.678 INFO:tasks.workunit.client.0.smithi088.stdout:6/122: write d5/de/f10 [1321645,68660] 0 2024-02-19T19:06:15.679 INFO:tasks.workunit.client.0.smithi088.stdout:7/175: creat d14/d1d/d3c/f3d x:0 0 0 2024-02-19T19:06:15.681 INFO:tasks.workunit.client.0.smithi088.stdout:9/155: rename d1/d21/d2d/d18/f1a to d1/d13/f33 0 2024-02-19T19:06:15.682 INFO:tasks.workunit.client.0.smithi088.stdout:8/101: rmdir d9 39 2024-02-19T19:06:15.685 INFO:tasks.workunit.client.0.smithi088.stdout:5/238: creat d4/f51 x:0 0 0 2024-02-19T19:06:15.686 INFO:tasks.workunit.client.0.smithi088.stdout:2/108: creat d3/f23 x:0 0 0 2024-02-19T19:06:15.687 INFO:tasks.workunit.client.0.smithi088.stdout:3/257: truncate d4/d1a/d1e/d31/d32/f50 94355 0 2024-02-19T19:06:15.688 INFO:tasks.workunit.client.0.smithi088.stdout:6/123: write d5/d7/f14 [396369,54385] 0 2024-02-19T19:06:15.692 INFO:tasks.workunit.client.0.smithi088.stdout:3/258: dread d4/d1a/d1e/d31/d32/d55/f57 [0,4194304] 0 2024-02-19T19:06:15.694 INFO:tasks.workunit.client.0.smithi088.stdout:8/102: dread d9/df/d1c/f16 [0,4194304] 0 2024-02-19T19:06:15.696 INFO:tasks.workunit.client.0.smithi088.stdout:6/124: rename d5/d19/d1f to d5/d19/d21/d25 0 2024-02-19T19:06:15.697 INFO:tasks.workunit.client.0.smithi088.stdout:6/125: write d5/de/f22 [1086776,80138] 0 2024-02-19T19:06:15.699 INFO:tasks.workunit.client.0.smithi088.stdout:2/109: symlink d3/l24 0 2024-02-19T19:06:15.701 INFO:tasks.workunit.client.0.smithi088.stdout:7/176: creat d14/f3e x:0 0 0 2024-02-19T19:06:15.705 INFO:tasks.workunit.client.0.smithi088.stdout:6/126: dwrite d5/f1e [0,4194304] 0 2024-02-19T19:06:15.708 INFO:tasks.workunit.client.0.smithi088.stdout:5/239: truncate d4/d1a/f2b 63428 0 2024-02-19T19:06:15.713 INFO:tasks.workunit.client.0.smithi088.stdout:6/127: dwrite d5/ff [0,4194304] 0 2024-02-19T19:06:15.713 INFO:tasks.workunit.client.0.smithi088.stdout:8/103: write d9/df/d1c/f16 [258589,66692] 0 2024-02-19T19:06:15.716 INFO:tasks.workunit.client.0.smithi088.stdout:7/177: rename d14/d1d/l20 to d14/d28/l3f 0 2024-02-19T19:06:15.720 INFO:tasks.workunit.client.0.smithi088.stdout:3/259: symlink d4/l63 0 2024-02-19T19:06:15.723 INFO:tasks.workunit.client.0.smithi088.stdout:2/110: dread d3/f4 [0,4194304] 0 2024-02-19T19:06:15.724 INFO:tasks.workunit.client.0.smithi088.stdout:2/111: chown d3 101252316 1 2024-02-19T19:06:15.726 INFO:tasks.workunit.client.0.smithi088.stdout:6/128: mkdir d5/d19/d26 0 2024-02-19T19:06:15.731 INFO:tasks.workunit.client.0.smithi088.stdout:5/240: dwrite d4/f12 [0,4194304] 0 2024-02-19T19:06:15.731 INFO:tasks.workunit.client.0.smithi088.stdout:3/260: mkdir d4/d1a/d1e/d31/d64 0 2024-02-19T19:06:15.732 INFO:tasks.workunit.client.0.smithi088.stdout:8/104: dwrite f6 [0,4194304] 0 2024-02-19T19:06:15.734 INFO:tasks.workunit.client.0.smithi088.stdout:8/105: stat d9/df/l14 0 2024-02-19T19:06:15.734 INFO:tasks.workunit.client.0.smithi088.stdout:2/112: mknod d3/df/c25 0 2024-02-19T19:06:15.735 INFO:tasks.workunit.client.0.smithi088.stdout:2/113: dread - d3/f23 zero size 2024-02-19T19:06:15.736 INFO:tasks.workunit.client.0.smithi088.stdout:6/129: write d5/de/f18 [2081802,67820] 0 2024-02-19T19:06:15.740 INFO:tasks.workunit.client.0.smithi088.stdout:7/178: link d14/d1d/d3c/f3d d14/d28/d3b/f40 0 2024-02-19T19:06:15.742 INFO:tasks.workunit.client.0.smithi088.stdout:3/261: creat d4/d12/d33/d43/f65 x:0 0 0 2024-02-19T19:06:15.743 INFO:tasks.workunit.client.0.smithi088.stdout:3/262: dread - d4/f28 zero size 2024-02-19T19:06:15.746 INFO:tasks.workunit.client.0.smithi088.stdout:6/130: creat d5/d19/d26/f27 x:0 0 0 2024-02-19T19:06:15.748 INFO:tasks.workunit.client.0.smithi088.stdout:8/106: dwrite d9/df/d1c/f1b [0,4194304] 0 2024-02-19T19:06:15.750 INFO:tasks.workunit.client.0.smithi088.stdout:7/179: rename d14/d28/l2d to d14/d28/l41 0 2024-02-19T19:06:15.750 INFO:tasks.workunit.client.0.smithi088.stdout:5/241: link d4/d3b/c49 d4/d1a/c52 0 2024-02-19T19:06:15.752 INFO:tasks.workunit.client.0.smithi088.stdout:6/131: unlink d5/de/c13 0 2024-02-19T19:06:15.754 INFO:tasks.workunit.client.0.smithi088.stdout:7/180: readlink l11 0 2024-02-19T19:06:15.755 INFO:tasks.workunit.client.0.smithi088.stdout:5/242: write d4/d7/f2c [3486335,121220] 0 2024-02-19T19:06:15.756 INFO:tasks.workunit.client.0.smithi088.stdout:2/114: dwrite d3/f4 [4194304,4194304] 0 2024-02-19T19:06:15.756 INFO:tasks.workunit.client.0.smithi088.stdout:1/247: sync 2024-02-19T19:06:15.757 INFO:tasks.workunit.client.0.smithi088.stdout:7/181: chown d14/f1f 0 1 2024-02-19T19:06:15.758 INFO:tasks.workunit.client.0.smithi088.stdout:1/248: fsync d6/d21/f3c 0 2024-02-19T19:06:15.760 INFO:tasks.workunit.client.0.smithi088.stdout:6/132: symlink d5/d7/l28 0 2024-02-19T19:06:15.762 INFO:tasks.workunit.client.0.smithi088.stdout:3/263: dwrite d4/d12/f17 [0,4194304] 0 2024-02-19T19:06:15.765 INFO:tasks.workunit.client.0.smithi088.stdout:5/243: dread d4/d7/f15 [0,4194304] 0 2024-02-19T19:06:15.767 INFO:tasks.workunit.client.0.smithi088.stdout:3/264: readlink d4/d1a/d1e/d31/d32/l46 0 2024-02-19T19:06:15.769 INFO:tasks.workunit.client.0.smithi088.stdout:6/133: mknod d5/de/c29 0 2024-02-19T19:06:15.773 INFO:tasks.workunit.client.0.smithi088.stdout:6/134: write d5/d7/f15 [2423049,71189] 0 2024-02-19T19:06:15.774 INFO:tasks.workunit.client.0.smithi088.stdout:2/115: dread d3/df/f22 [0,4194304] 0 2024-02-19T19:06:15.775 INFO:tasks.workunit.client.0.smithi088.stdout:1/249: creat d6/f52 x:0 0 0 2024-02-19T19:06:15.783 INFO:tasks.workunit.client.0.smithi088.stdout:5/244: chown d4/d3b/c49 760 1 2024-02-19T19:06:15.785 INFO:tasks.workunit.client.0.smithi088.stdout:2/116: fsync d3/f4 0 2024-02-19T19:06:15.786 INFO:tasks.workunit.client.0.smithi088.stdout:2/117: truncate d3/d1b/f1c 894584 0 2024-02-19T19:06:15.787 INFO:tasks.workunit.client.0.smithi088.stdout:2/118: truncate d3/d14/f1d 803638 0 2024-02-19T19:06:15.788 INFO:tasks.workunit.client.0.smithi088.stdout:1/250: chown d6/d10/d26/c46 685 1 2024-02-19T19:06:15.790 INFO:tasks.workunit.client.0.smithi088.stdout:5/245: creat d4/d1a/f53 x:0 0 0 2024-02-19T19:06:15.791 INFO:tasks.workunit.client.0.smithi088.stdout:8/107: truncate f8 411335 0 2024-02-19T19:06:15.791 INFO:tasks.workunit.client.0.smithi088.stdout:5/246: write d4/f4d [836153,67174] 0 2024-02-19T19:06:15.794 INFO:tasks.workunit.client.0.smithi088.stdout:7/182: getdents d14/d1d 0 2024-02-19T19:06:15.797 INFO:tasks.workunit.client.0.smithi088.stdout:3/265: rename d4/d1a/d1e/d5a to d4/d22/d66 0 2024-02-19T19:06:15.798 INFO:tasks.workunit.client.0.smithi088.stdout:8/108: creat d9/df/d1c/f1f x:0 0 0 2024-02-19T19:06:15.801 INFO:tasks.workunit.client.0.smithi088.stdout:7/183: creat d14/d1d/f42 x:0 0 0 2024-02-19T19:06:15.803 INFO:tasks.workunit.client.0.smithi088.stdout:5/247: unlink d4/d1a/f2b 0 2024-02-19T19:06:15.804 INFO:tasks.workunit.client.0.smithi088.stdout:8/109: dwrite f6 [0,4194304] 0 2024-02-19T19:06:15.805 INFO:tasks.workunit.client.0.smithi088.stdout:8/110: dread - d9/df/f1a zero size 2024-02-19T19:06:15.807 INFO:tasks.workunit.client.0.smithi088.stdout:5/248: creat d4/d1a/f54 x:0 0 0 2024-02-19T19:06:15.809 INFO:tasks.workunit.client.0.smithi088.stdout:5/249: dread - d4/d1a/d22/f4e zero size 2024-02-19T19:06:15.811 INFO:tasks.workunit.client.0.smithi088.stdout:5/250: write d4/d1a/f37 [84826,71234] 0 2024-02-19T19:06:15.812 INFO:tasks.workunit.client.0.smithi088.stdout:6/135: truncate d5/ff 1395978 0 2024-02-19T19:06:15.814 INFO:tasks.workunit.client.0.smithi088.stdout:8/111: dwrite d9/df/f1a [0,4194304] 0 2024-02-19T19:06:15.814 INFO:tasks.workunit.client.0.smithi088.stdout:5/251: truncate d4/d7/f2f 424494 0 2024-02-19T19:06:15.816 INFO:tasks.workunit.client.0.smithi088.stdout:6/136: rmdir d5/d8 39 2024-02-19T19:06:15.819 INFO:tasks.workunit.client.0.smithi088.stdout:6/137: stat d5/d8/l1d 0 2024-02-19T19:06:15.821 INFO:tasks.workunit.client.0.smithi088.stdout:5/252: dread d4/d1a/d22/f28 [0,4194304] 0 2024-02-19T19:06:15.821 INFO:tasks.workunit.client.0.smithi088.stdout:8/112: rmdir d9/df/d1c/d1e 0 2024-02-19T19:06:15.822 INFO:tasks.workunit.client.0.smithi088.stdout:5/253: chown d4/d1a/d41/c44 3766 1 2024-02-19T19:06:15.823 INFO:tasks.workunit.client.0.smithi088.stdout:5/254: read - d4/d7/f36 zero size 2024-02-19T19:06:15.826 INFO:tasks.workunit.client.0.smithi088.stdout:8/113: symlink d9/df/d1c/l20 0 2024-02-19T19:06:15.827 INFO:tasks.workunit.client.0.smithi088.stdout:5/255: dread d4/f9 [0,4194304] 0 2024-02-19T19:06:15.833 INFO:tasks.workunit.client.0.smithi088.stdout:5/256: rename d4/f51 to d4/d1a/f55 0 2024-02-19T19:06:15.835 INFO:tasks.workunit.client.0.smithi088.stdout:8/114: symlink d9/l21 0 2024-02-19T19:06:15.836 INFO:tasks.workunit.client.0.smithi088.stdout:8/115: truncate d9/df/d1c/f1f 118752 0 2024-02-19T19:06:15.843 INFO:tasks.workunit.client.0.smithi088.stdout:8/116: dwrite d9/df/d1c/f1f [0,4194304] 0 2024-02-19T19:06:15.907 INFO:tasks.workunit.client.0.smithi088.stdout:6/138: truncate d5/f1e 705992 0 2024-02-19T19:06:15.909 INFO:tasks.workunit.client.0.smithi088.stdout:5/257: unlink d4/d1a/f53 0 2024-02-19T19:06:15.912 INFO:tasks.workunit.client.0.smithi088.stdout:8/117: mkdir d9/df/d1c/d22 0 2024-02-19T19:06:15.914 INFO:tasks.workunit.client.0.smithi088.stdout:5/258: rename d4/c45 to d4/d1a/c56 0 2024-02-19T19:06:15.915 INFO:tasks.workunit.client.0.smithi088.stdout:5/259: write d4/d7/f36 [991319,52424] 0 2024-02-19T19:06:15.916 INFO:tasks.workunit.client.0.smithi088.stdout:5/260: write d4/d1a/d22/f26 [1444508,128862] 0 2024-02-19T19:06:15.917 INFO:tasks.workunit.client.0.smithi088.stdout:8/118: creat d9/df/d1c/f23 x:0 0 0 2024-02-19T19:06:15.919 INFO:tasks.workunit.client.0.smithi088.stdout:6/139: link d5/d19/c23 d5/de/c2a 0 2024-02-19T19:06:15.922 INFO:tasks.workunit.client.0.smithi088.stdout:5/261: dwrite d4/d1a/f21 [0,4194304] 0 2024-02-19T19:06:15.923 INFO:tasks.workunit.client.0.smithi088.stdout:6/140: mkdir d5/d19/d21/d2b 0 2024-02-19T19:06:15.930 INFO:tasks.workunit.client.0.smithi088.stdout:5/262: rename d4/d3b to d4/d1a/d22/d57 0 2024-02-19T19:06:15.932 INFO:tasks.workunit.client.0.smithi088.stdout:8/119: link d9/l11 d9/df/d1c/d22/l24 0 2024-02-19T19:06:15.935 INFO:tasks.workunit.client.0.smithi088.stdout:6/141: rename d5/d19/d21/d2b to d5/d8/d2c 0 2024-02-19T19:06:15.936 INFO:tasks.workunit.client.0.smithi088.stdout:5/263: mknod d4/d7/c58 0 2024-02-19T19:06:15.938 INFO:tasks.workunit.client.0.smithi088.stdout:8/120: rmdir d9/df 39 2024-02-19T19:06:15.940 INFO:tasks.workunit.client.0.smithi088.stdout:5/264: mknod d4/d1a/d22/c59 0 2024-02-19T19:06:15.941 INFO:tasks.workunit.client.0.smithi088.stdout:5/265: fsync d4/d1a/f55 0 2024-02-19T19:06:15.942 INFO:tasks.workunit.client.0.smithi088.stdout:8/121: creat d9/df/d1c/f25 x:0 0 0 2024-02-19T19:06:15.943 INFO:tasks.workunit.client.0.smithi088.stdout:8/122: chown d9/df/d1c/l20 174869608 1 2024-02-19T19:06:15.943 INFO:tasks.workunit.client.0.smithi088.stdout:0/150: sync 2024-02-19T19:06:15.944 INFO:tasks.workunit.client.0.smithi088.stdout:5/266: write d4/d1a/d22/f24 [482263,13778] 0 2024-02-19T19:06:15.945 INFO:tasks.workunit.client.0.smithi088.stdout:8/123: read f8 [242084,58451] 0 2024-02-19T19:06:15.946 INFO:tasks.workunit.client.0.smithi088.stdout:8/124: read f8 [45706,108959] 0 2024-02-19T19:06:15.946 INFO:tasks.workunit.client.0.smithi088.stdout:0/151: creat d13/d24/f32 x:0 0 0 2024-02-19T19:06:15.948 INFO:tasks.workunit.client.0.smithi088.stdout:6/142: dread d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:15.949 INFO:tasks.workunit.client.0.smithi088.stdout:8/125: write d9/df/d1c/f23 [146067,78659] 0 2024-02-19T19:06:15.952 INFO:tasks.workunit.client.0.smithi088.stdout:6/143: unlink d5/d7/l28 0 2024-02-19T19:06:15.953 INFO:tasks.workunit.client.0.smithi088.stdout:5/267: dwrite d4/f11 [0,4194304] 0 2024-02-19T19:06:15.955 INFO:tasks.workunit.client.0.smithi088.stdout:8/126: mknod d9/df/d1c/c26 0 2024-02-19T19:06:15.955 INFO:tasks.workunit.client.0.smithi088.stdout:8/127: readlink l4 0 2024-02-19T19:06:15.959 INFO:tasks.workunit.client.0.smithi088.stdout:5/268: mknod d4/d7/c5a 0 2024-02-19T19:06:15.961 INFO:tasks.workunit.client.0.smithi088.stdout:5/269: truncate d4/d1a/d22/f26 2081969 0 2024-02-19T19:06:15.963 INFO:tasks.workunit.client.0.smithi088.stdout:8/128: dwrite d9/df/d1c/f1b [4194304,4194304] 0 2024-02-19T19:06:15.965 INFO:tasks.workunit.client.0.smithi088.stdout:6/144: rename d5/d19/c23 to d5/d19/d21/c2d 0 2024-02-19T19:06:15.967 INFO:tasks.workunit.client.0.smithi088.stdout:6/145: read d5/d7/f14 [2124005,43349] 0 2024-02-19T19:06:15.970 INFO:tasks.workunit.client.0.smithi088.stdout:8/129: dwrite d9/df/f1a [4194304,4194304] 0 2024-02-19T19:06:15.978 INFO:tasks.workunit.client.0.smithi088.stdout:6/146: dwrite d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:15.983 INFO:tasks.workunit.client.0.smithi088.stdout:6/147: creat d5/d19/f2e x:0 0 0 2024-02-19T19:06:15.985 INFO:tasks.workunit.client.0.smithi088.stdout:6/148: getdents d5/d19/d21/d25 0 2024-02-19T19:06:15.989 INFO:tasks.workunit.client.0.smithi088.stdout:6/149: unlink d5/d8/cd 0 2024-02-19T19:06:15.991 INFO:tasks.workunit.client.0.smithi088.stdout:6/150: creat d5/d19/f2f x:0 0 0 2024-02-19T19:06:15.994 INFO:tasks.workunit.client.0.smithi088.stdout:0/152: truncate d13/f23 3380545 0 2024-02-19T19:06:15.995 INFO:tasks.workunit.client.0.smithi088.stdout:6/151: rename d5/de/c12 to d5/d19/d26/c30 0 2024-02-19T19:06:16.002 INFO:tasks.workunit.client.0.smithi088.stdout:6/152: rename d5/d19/d21 to d5/d31 0 2024-02-19T19:06:16.004 INFO:tasks.workunit.client.0.smithi088.stdout:8/130: dwrite d9/df/d1c/f1b [8388608,4194304] 0 2024-02-19T19:06:16.009 INFO:tasks.workunit.client.0.smithi088.stdout:8/131: dread d9/df/d1c/f1b [8388608,4194304] 0 2024-02-19T19:06:16.013 INFO:tasks.workunit.client.0.smithi088.stdout:8/132: dread d9/df/d1c/f1b [0,4194304] 0 2024-02-19T19:06:16.013 INFO:tasks.workunit.client.0.smithi088.stdout:0/153: write d13/d24/d25/f26 [3196841,124707] 0 2024-02-19T19:06:16.016 INFO:tasks.workunit.client.0.smithi088.stdout:0/154: dread d13/f15 [0,4194304] 0 2024-02-19T19:06:16.017 INFO:tasks.workunit.client.0.smithi088.stdout:8/133: creat d9/df/f27 x:0 0 0 2024-02-19T19:06:16.018 INFO:tasks.workunit.client.0.smithi088.stdout:6/153: truncate d5/d7/f14 1408782 0 2024-02-19T19:06:16.020 INFO:tasks.workunit.client.0.smithi088.stdout:8/134: symlink d9/df/l28 0 2024-02-19T19:06:16.021 INFO:tasks.workunit.client.0.smithi088.stdout:6/154: mkdir d5/d7/d32 0 2024-02-19T19:06:16.024 INFO:tasks.workunit.client.0.smithi088.stdout:8/135: creat d9/df/d1c/f29 x:0 0 0 2024-02-19T19:06:16.024 INFO:tasks.workunit.client.0.smithi088.stdout:6/155: read d5/ff [895625,107609] 0 2024-02-19T19:06:16.025 INFO:tasks.workunit.client.0.smithi088.stdout:0/155: creat d13/f33 x:0 0 0 2024-02-19T19:06:16.026 INFO:tasks.workunit.client.0.smithi088.stdout:8/136: write d9/df/d1c/f23 [549123,77448] 0 2024-02-19T19:06:16.027 INFO:tasks.workunit.client.0.smithi088.stdout:8/137: truncate d9/df/d1c/f16 4723555 0 2024-02-19T19:06:16.030 INFO:tasks.workunit.client.0.smithi088.stdout:6/156: dread d5/de/f22 [0,4194304] 0 2024-02-19T19:06:16.033 INFO:tasks.workunit.client.0.smithi088.stdout:6/157: symlink d5/d8/l33 0 2024-02-19T19:06:16.034 INFO:tasks.workunit.client.0.smithi088.stdout:6/158: write d5/d19/f2f [143054,69528] 0 2024-02-19T19:06:16.036 INFO:tasks.workunit.client.0.smithi088.stdout:6/159: write d5/d19/d26/f27 [374865,28278] 0 2024-02-19T19:06:16.036 INFO:tasks.workunit.client.0.smithi088.stdout:6/160: readlink d5/d8/l1b 0 2024-02-19T19:06:16.038 INFO:tasks.workunit.client.0.smithi088.stdout:0/156: truncate d13/f23 2313497 0 2024-02-19T19:06:16.041 INFO:tasks.workunit.client.0.smithi088.stdout:0/157: read fe [1180887,121654] 0 2024-02-19T19:06:16.041 INFO:tasks.workunit.client.0.smithi088.stdout:6/161: mknod d5/d19/c34 0 2024-02-19T19:06:16.047 INFO:tasks.workunit.client.0.smithi088.stdout:6/162: stat d5/d31/c2d 0 2024-02-19T19:06:16.050 INFO:tasks.workunit.client.0.smithi088.stdout:6/163: dwrite d5/de/f10 [0,4194304] 0 2024-02-19T19:06:16.051 INFO:tasks.workunit.client.0.smithi088.stdout:6/164: chown d5/d19/d26 5 1 2024-02-19T19:06:16.054 INFO:tasks.workunit.client.0.smithi088.stdout:4/137: sync 2024-02-19T19:06:16.054 INFO:tasks.workunit.client.0.smithi088.stdout:2/119: sync 2024-02-19T19:06:16.054 INFO:tasks.workunit.client.0.smithi088.stdout:9/156: sync 2024-02-19T19:06:16.055 INFO:tasks.workunit.client.0.smithi088.stdout:9/157: chown d1/d21/d2d/d23/l30 63 1 2024-02-19T19:06:16.056 INFO:tasks.workunit.client.0.smithi088.stdout:9/158: fdatasync d1/d21/d2d/d23/f25 0 2024-02-19T19:06:16.059 INFO:tasks.workunit.client.0.smithi088.stdout:2/120: truncate d3/f6 409855 0 2024-02-19T19:06:16.061 INFO:tasks.workunit.client.0.smithi088.stdout:2/121: truncate d3/df/f17 14322 0 2024-02-19T19:06:16.062 INFO:tasks.workunit.client.0.smithi088.stdout:0/158: dwrite f3 [0,4194304] 0 2024-02-19T19:06:16.065 INFO:tasks.workunit.client.0.smithi088.stdout:9/159: creat d1/d13/f34 x:0 0 0 2024-02-19T19:06:16.066 INFO:tasks.workunit.client.0.smithi088.stdout:9/160: fsync d1/d21/d2d/d23/f25 0 2024-02-19T19:06:16.066 INFO:tasks.workunit.client.0.smithi088.stdout:6/165: link d5/d8/l33 d5/d8/d2c/l35 0 2024-02-19T19:06:16.068 INFO:tasks.workunit.client.0.smithi088.stdout:6/166: write d5/de/f10 [1400459,12693] 0 2024-02-19T19:06:16.070 INFO:tasks.workunit.client.0.smithi088.stdout:4/138: write d2/d5/f23 [689872,111019] 0 2024-02-19T19:06:16.070 INFO:tasks.workunit.client.0.smithi088.stdout:6/167: chown d5/de/c29 299 1 2024-02-19T19:06:16.073 INFO:tasks.workunit.client.0.smithi088.stdout:2/122: unlink d3/l24 0 2024-02-19T19:06:16.077 INFO:tasks.workunit.client.0.smithi088.stdout:9/161: creat d1/d21/d2d/d23/f35 x:0 0 0 2024-02-19T19:06:16.078 INFO:tasks.workunit.client.0.smithi088.stdout:4/139: rmdir d2/d5/d9 39 2024-02-19T19:06:16.081 INFO:tasks.workunit.client.0.smithi088.stdout:6/168: mknod d5/d19/c36 0 2024-02-19T19:06:16.081 INFO:tasks.workunit.client.0.smithi088.stdout:6/169: stat d5/d7 0 2024-02-19T19:06:16.082 INFO:tasks.workunit.client.0.smithi088.stdout:2/123: symlink d3/df/l26 0 2024-02-19T19:06:16.084 INFO:tasks.workunit.client.0.smithi088.stdout:4/140: dwrite d2/d5/f1c [4194304,4194304] 0 2024-02-19T19:06:16.085 INFO:tasks.workunit.client.0.smithi088.stdout:9/162: creat d1/d13/f36 x:0 0 0 2024-02-19T19:06:16.089 INFO:tasks.workunit.client.0.smithi088.stdout:2/124: creat d3/d1b/f27 x:0 0 0 2024-02-19T19:06:16.092 INFO:tasks.workunit.client.0.smithi088.stdout:0/159: dread fb [0,4194304] 0 2024-02-19T19:06:16.094 INFO:tasks.workunit.client.0.smithi088.stdout:4/141: mknod d2/d5/d25/d26/c31 0 2024-02-19T19:06:16.099 INFO:tasks.workunit.client.0.smithi088.stdout:0/160: mkdir d13/d24/d25/d2a/d34 0 2024-02-19T19:06:16.100 INFO:tasks.workunit.client.0.smithi088.stdout:2/125: symlink d3/l28 0 2024-02-19T19:06:16.101 INFO:tasks.workunit.client.0.smithi088.stdout:2/126: read d3/f4 [2412403,31744] 0 2024-02-19T19:06:16.106 INFO:tasks.workunit.client.0.smithi088.stdout:0/161: dwrite d13/d24/d25/f27 [4194304,4194304] 0 2024-02-19T19:06:16.109 INFO:tasks.workunit.client.0.smithi088.stdout:6/170: write d5/d7/f14 [1858004,30869] 0 2024-02-19T19:06:16.110 INFO:tasks.workunit.client.0.smithi088.stdout:0/162: rmdir d13/d24/d25/d2a 39 2024-02-19T19:06:16.112 INFO:tasks.workunit.client.0.smithi088.stdout:6/171: creat d5/d31/f37 x:0 0 0 2024-02-19T19:06:16.113 INFO:tasks.workunit.client.0.smithi088.stdout:6/172: stat d5/d7 0 2024-02-19T19:06:16.115 INFO:tasks.workunit.client.0.smithi088.stdout:2/127: getdents d3/df 0 2024-02-19T19:06:16.116 INFO:tasks.workunit.client.0.smithi088.stdout:4/142: getdents d2/d5/d25/d26 0 2024-02-19T19:06:16.117 INFO:tasks.workunit.client.0.smithi088.stdout:9/163: write d1/d21/d2d/d23/f26 [660324,8801] 0 2024-02-19T19:06:16.119 INFO:tasks.workunit.client.0.smithi088.stdout:6/173: mknod d5/de/c38 0 2024-02-19T19:06:16.121 INFO:tasks.workunit.client.0.smithi088.stdout:4/143: dread d2/f7 [0,4194304] 0 2024-02-19T19:06:16.122 INFO:tasks.workunit.client.0.smithi088.stdout:4/144: readlink d2/d8/l22 0 2024-02-19T19:06:16.122 INFO:tasks.workunit.client.0.smithi088.stdout:4/145: chown d2/d8/c2f 45693 1 2024-02-19T19:06:16.123 INFO:tasks.workunit.client.0.smithi088.stdout:0/163: creat d13/d24/d25/d2a/d34/f35 x:0 0 0 2024-02-19T19:06:16.125 INFO:tasks.workunit.client.0.smithi088.stdout:9/164: mkdir d1/d8/d37 0 2024-02-19T19:06:16.129 INFO:tasks.workunit.client.0.smithi088.stdout:0/164: mknod d13/d24/c36 0 2024-02-19T19:06:16.131 INFO:tasks.workunit.client.0.smithi088.stdout:6/174: link d5/d19/f2f d5/d31/d25/f39 0 2024-02-19T19:06:16.136 INFO:tasks.workunit.client.0.smithi088.stdout:6/175: rmdir d5/d19/d26 39 2024-02-19T19:06:16.141 INFO:tasks.workunit.client.0.smithi088.stdout:6/176: mkdir d5/d8/d3a 0 2024-02-19T19:06:16.142 INFO:tasks.workunit.client.0.smithi088.stdout:6/177: readlink d5/d8/l1d 0 2024-02-19T19:06:16.143 INFO:tasks.workunit.client.0.smithi088.stdout:0/165: write d13/f1a [4561781,69132] 0 2024-02-19T19:06:16.143 INFO:tasks.workunit.client.0.smithi088.stdout:4/146: dwrite d2/d5/f18 [4194304,4194304] 0 2024-02-19T19:06:16.145 INFO:tasks.workunit.client.0.smithi088.stdout:0/166: chown d13/f1b 43313473 1 2024-02-19T19:06:16.148 INFO:tasks.workunit.client.0.smithi088.stdout:0/167: fsync d13/f1c 0 2024-02-19T19:06:16.149 INFO:tasks.workunit.client.0.smithi088.stdout:9/165: dwrite d1/d21/d2d/f2a [0,4194304] 0 2024-02-19T19:06:16.151 INFO:tasks.workunit.client.0.smithi088.stdout:9/166: write d1/d21/d2d/d18/d1f/f2e [1778330,111208] 0 2024-02-19T19:06:16.159 INFO:tasks.workunit.client.0.smithi088.stdout:9/167: mkdir d1/d21/d2d/d23/d38 0 2024-02-19T19:06:16.159 INFO:tasks.workunit.client.0.smithi088.stdout:6/178: dwrite d5/d19/f2f [0,4194304] 0 2024-02-19T19:06:16.164 INFO:tasks.workunit.client.0.smithi088.stdout:9/168: symlink d1/d21/d2d/l39 0 2024-02-19T19:06:16.165 INFO:tasks.workunit.client.0.smithi088.stdout:0/168: creat d13/f37 x:0 0 0 2024-02-19T19:06:16.166 INFO:tasks.workunit.client.0.smithi088.stdout:0/169: dread - d13/f33 zero size 2024-02-19T19:06:16.166 INFO:tasks.workunit.client.0.smithi088.stdout:4/147: rename d2/d5 to d2/d32 0 2024-02-19T19:06:16.169 INFO:tasks.workunit.client.0.smithi088.stdout:9/169: mknod d1/d21/d2f/c3a 0 2024-02-19T19:06:16.170 INFO:tasks.workunit.client.0.smithi088.stdout:6/179: dread d5/d19/d26/f27 [0,4194304] 0 2024-02-19T19:06:16.172 INFO:tasks.workunit.client.0.smithi088.stdout:4/148: dread - d2/d32/d9/f20 zero size 2024-02-19T19:06:16.174 INFO:tasks.workunit.client.0.smithi088.stdout:4/149: write d2/d32/d9/f20 [627503,97933] 0 2024-02-19T19:06:16.176 INFO:tasks.workunit.client.0.smithi088.stdout:0/170: dwrite d13/d24/d25/f26 [0,4194304] 0 2024-02-19T19:06:16.177 INFO:tasks.workunit.client.0.smithi088.stdout:9/170: dread d1/d21/d2d/d23/f26 [0,4194304] 0 2024-02-19T19:06:16.177 INFO:tasks.workunit.client.0.smithi088.stdout:6/180: read d5/f1e [556589,58340] 0 2024-02-19T19:06:16.179 INFO:tasks.workunit.client.0.smithi088.stdout:9/171: fsync d1/d21/d2d/d23/f25 0 2024-02-19T19:06:16.179 INFO:tasks.workunit.client.0.smithi088.stdout:9/172: chown d1/f6 89 1 2024-02-19T19:06:16.184 INFO:tasks.workunit.client.0.smithi088.stdout:9/173: chown d1/f10 27 1 2024-02-19T19:06:16.184 INFO:tasks.workunit.client.0.smithi088.stdout:6/181: dwrite d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:16.192 INFO:tasks.workunit.client.0.smithi088.stdout:9/174: dwrite d1/d21/d2d/d18/d1f/f2e [0,4194304] 0 2024-02-19T19:06:16.193 INFO:tasks.workunit.client.0.smithi088.stdout:9/175: chown d1/c1e 83416 1 2024-02-19T19:06:16.198 INFO:tasks.workunit.client.0.smithi088.stdout:4/150: rmdir d2/d32 39 2024-02-19T19:06:16.199 INFO:tasks.workunit.client.0.smithi088.stdout:0/171: link d13/d24/c36 d13/d24/d25/c38 0 2024-02-19T19:06:16.200 INFO:tasks.workunit.client.0.smithi088.stdout:9/176: write d1/d21/d2d/d23/f26 [537803,52173] 0 2024-02-19T19:06:16.201 INFO:tasks.workunit.client.0.smithi088.stdout:0/172: write d13/d24/d25/f27 [5696726,4673] 0 2024-02-19T19:06:16.203 INFO:tasks.workunit.client.0.smithi088.stdout:6/182: creat d5/f3b x:0 0 0 2024-02-19T19:06:16.203 INFO:tasks.workunit.client.0.smithi088.stdout:0/173: truncate fb 4863958 0 2024-02-19T19:06:16.204 INFO:tasks.workunit.client.0.smithi088.stdout:6/183: chown d5/f3b 110879 1 2024-02-19T19:06:16.205 INFO:tasks.workunit.client.0.smithi088.stdout:6/184: write d5/d19/f2f [2465308,5270] 0 2024-02-19T19:06:16.206 INFO:tasks.workunit.client.0.smithi088.stdout:6/185: stat d5/ff 0 2024-02-19T19:06:16.206 INFO:tasks.workunit.client.0.smithi088.stdout:9/177: rename d1/d13/f34 to d1/d21/d2d/d18/f3b 0 2024-02-19T19:06:16.209 INFO:tasks.workunit.client.0.smithi088.stdout:6/186: dread d5/d19/d26/f27 [0,4194304] 0 2024-02-19T19:06:16.209 INFO:tasks.workunit.client.0.smithi088.stdout:0/174: creat d13/f39 x:0 0 0 2024-02-19T19:06:16.212 INFO:tasks.workunit.client.0.smithi088.stdout:6/187: fdatasync d5/d7/f14 0 2024-02-19T19:06:16.212 INFO:tasks.workunit.client.0.smithi088.stdout:4/151: dwrite d2/d32/fb [0,4194304] 0 2024-02-19T19:06:16.213 INFO:tasks.workunit.client.0.smithi088.stdout:9/178: rename d1/d21/d2d/d18/c24 to d1/d21/d2f/c3c 0 2024-02-19T19:06:16.214 INFO:tasks.workunit.client.0.smithi088.stdout:1/251: sync 2024-02-19T19:06:16.214 INFO:tasks.workunit.client.0.smithi088.stdout:3/266: sync 2024-02-19T19:06:16.215 INFO:tasks.workunit.client.0.smithi088.stdout:1/252: fsync d6/d10/d26/d4c/f35 0 2024-02-19T19:06:16.215 INFO:tasks.workunit.client.0.smithi088.stdout:3/267: dread - d4/d1a/d1e/d31/d32/f51 zero size 2024-02-19T19:06:16.222 INFO:tasks.workunit.client.0.smithi088.stdout:9/179: symlink d1/d13/l3d 0 2024-02-19T19:06:16.223 INFO:tasks.workunit.client.0.smithi088.stdout:3/268: dwrite d4/d22/f2b [0,4194304] 0 2024-02-19T19:06:16.224 INFO:tasks.workunit.client.0.smithi088.stdout:0/175: creat d13/f3a x:0 0 0 2024-02-19T19:06:16.225 INFO:tasks.workunit.client.0.smithi088.stdout:3/269: write d4/d1a/f24 [270316,54648] 0 2024-02-19T19:06:16.226 INFO:tasks.workunit.client.0.smithi088.stdout:1/253: read d6/d10/d28/d29/f37 [94153,48084] 0 2024-02-19T19:06:16.227 INFO:tasks.workunit.client.0.smithi088.stdout:4/152: symlink d2/l33 0 2024-02-19T19:06:16.229 INFO:tasks.workunit.client.0.smithi088.stdout:4/153: fsync d2/d32/d9/f20 0 2024-02-19T19:06:16.230 INFO:tasks.workunit.client.0.smithi088.stdout:6/188: unlink d5/d19/d26/c30 0 2024-02-19T19:06:16.233 INFO:tasks.workunit.client.0.smithi088.stdout:9/180: mkdir d1/d8/d3e 0 2024-02-19T19:06:16.234 INFO:tasks.workunit.client.0.smithi088.stdout:0/176: symlink d13/d24/l3b 0 2024-02-19T19:06:16.235 INFO:tasks.workunit.client.0.smithi088.stdout:9/181: dread - d1/d13/f36 zero size 2024-02-19T19:06:16.239 INFO:tasks.workunit.client.0.smithi088.stdout:1/254: dwrite d6/d10/f3b [0,4194304] 0 2024-02-19T19:06:16.240 INFO:tasks.workunit.client.0.smithi088.stdout:3/270: unlink d4/d1a/l29 0 2024-02-19T19:06:16.242 INFO:tasks.workunit.client.0.smithi088.stdout:4/154: dwrite d2/f15 [0,4194304] 0 2024-02-19T19:06:16.242 INFO:tasks.workunit.client.0.smithi088.stdout:1/255: truncate d6/fb 811282 0 2024-02-19T19:06:16.246 INFO:tasks.workunit.client.0.smithi088.stdout:6/189: rename d5/de/c38 to d5/d19/d26/c3c 0 2024-02-19T19:06:16.248 INFO:tasks.workunit.client.0.smithi088.stdout:6/190: chown d5/de/f10 26 1 2024-02-19T19:06:16.249 INFO:tasks.workunit.client.0.smithi088.stdout:0/177: mkdir d13/d24/d25/d2a/d3c 0 2024-02-19T19:06:16.250 INFO:tasks.workunit.client.0.smithi088.stdout:3/271: mknod d4/d1a/d1e/d31/d32/c67 0 2024-02-19T19:06:16.250 INFO:tasks.workunit.client.0.smithi088.stdout:0/178: chown d13/f2c 177 1 2024-02-19T19:06:16.251 INFO:tasks.workunit.client.0.smithi088.stdout:9/182: dwrite d1/d21/d2d/d18/d1f/f2e [0,4194304] 0 2024-02-19T19:06:16.255 INFO:tasks.workunit.client.0.smithi088.stdout:9/183: fsync d1/d13/f36 0 2024-02-19T19:06:16.256 INFO:tasks.workunit.client.0.smithi088.stdout:4/155: symlink d2/d32/d25/l34 0 2024-02-19T19:06:16.256 INFO:tasks.workunit.client.0.smithi088.stdout:9/184: chown d1/d13/l1c 5680 1 2024-02-19T19:06:16.256 INFO:tasks.workunit.client.0.smithi088.stdout:1/256: mkdir d6/d10/d26/d53 0 2024-02-19T19:06:16.256 INFO:tasks.workunit.client.0.smithi088.stdout:9/185: chown d1/d21/d2d/d23/f25 91952 1 2024-02-19T19:06:16.259 INFO:tasks.workunit.client.0.smithi088.stdout:6/191: symlink d5/d7/l3d 0 2024-02-19T19:06:16.261 INFO:tasks.workunit.client.0.smithi088.stdout:3/272: symlink d4/d1a/d1e/d31/d32/l68 0 2024-02-19T19:06:16.262 INFO:tasks.workunit.client.0.smithi088.stdout:4/156: dwrite d2/d32/d25/f30 [0,4194304] 0 2024-02-19T19:06:16.265 INFO:tasks.workunit.client.0.smithi088.stdout:9/186: stat d1/fc 0 2024-02-19T19:06:16.266 INFO:tasks.workunit.client.0.smithi088.stdout:9/187: truncate d1/d21/d2d/d18/f3b 300190 0 2024-02-19T19:06:16.272 INFO:tasks.workunit.client.0.smithi088.stdout:1/257: truncate d6/f1e 1056054 0 2024-02-19T19:06:16.273 INFO:tasks.workunit.client.0.smithi088.stdout:3/273: unlink d4/d1a/d23/f61 0 2024-02-19T19:06:16.274 INFO:tasks.workunit.client.0.smithi088.stdout:3/274: chown d4/d12/f27 1324629243 1 2024-02-19T19:06:16.279 INFO:tasks.workunit.client.0.smithi088.stdout:0/179: dread d13/f1c [0,4194304] 0 2024-02-19T19:06:16.279 INFO:tasks.workunit.client.0.smithi088.stdout:9/188: symlink d1/d21/l3f 0 2024-02-19T19:06:16.280 INFO:tasks.workunit.client.0.smithi088.stdout:0/180: chown d13/d24/d25/d2a/d2f 8016803 1 2024-02-19T19:06:16.280 INFO:tasks.workunit.client.0.smithi088.stdout:6/192: symlink d5/d7/d32/l3e 0 2024-02-19T19:06:16.282 INFO:tasks.workunit.client.0.smithi088.stdout:9/189: dread d1/d21/d2d/d18/f3b [0,4194304] 0 2024-02-19T19:06:16.285 INFO:tasks.workunit.client.0.smithi088.stdout:0/181: dread d13/d24/d25/f27 [4194304,4194304] 0 2024-02-19T19:06:16.285 INFO:tasks.workunit.client.0.smithi088.stdout:9/190: dread d1/d21/d2d/d23/f26 [0,4194304] 0 2024-02-19T19:06:16.286 INFO:tasks.workunit.client.0.smithi088.stdout:6/193: dread d5/d19/f2f [0,4194304] 0 2024-02-19T19:06:16.286 INFO:tasks.workunit.client.0.smithi088.stdout:4/157: mkdir d2/d35 0 2024-02-19T19:06:16.287 INFO:tasks.workunit.client.0.smithi088.stdout:6/194: dread - d5/d19/f2e zero size 2024-02-19T19:06:16.289 INFO:tasks.workunit.client.0.smithi088.stdout:7/184: sync 2024-02-19T19:06:16.289 INFO:tasks.workunit.client.0.smithi088.stdout:5/270: sync 2024-02-19T19:06:16.291 INFO:tasks.workunit.client.0.smithi088.stdout:1/258: symlink d6/d10/d28/d29/d4e/l54 0 2024-02-19T19:06:16.294 INFO:tasks.workunit.client.0.smithi088.stdout:3/275: unlink d4/c41 0 2024-02-19T19:06:16.296 INFO:tasks.workunit.client.0.smithi088.stdout:0/182: mkdir d13/d24/d25/d2a/d3d 0 2024-02-19T19:06:16.301 INFO:tasks.workunit.client.0.smithi088.stdout:5/271: creat d4/d1a/f5b x:0 0 0 2024-02-19T19:06:16.305 INFO:tasks.workunit.client.0.smithi088.stdout:9/191: symlink d1/d8/d37/l40 0 2024-02-19T19:06:16.306 INFO:tasks.workunit.client.0.smithi088.stdout:4/158: write f0 [385176,52005] 0 2024-02-19T19:06:16.311 INFO:tasks.workunit.client.0.smithi088.stdout:7/185: mkdir d14/d1d/d3c/d43 0 2024-02-19T19:06:16.311 INFO:tasks.workunit.client.0.smithi088.stdout:5/272: fsync d4/d1a/d22/f28 0 2024-02-19T19:06:16.312 INFO:tasks.workunit.client.0.smithi088.stdout:5/273: readlink d4/l40 0 2024-02-19T19:06:16.313 INFO:tasks.workunit.client.0.smithi088.stdout:3/276: dread d4/f7 [0,4194304] 0 2024-02-19T19:06:16.315 INFO:tasks.workunit.client.0.smithi088.stdout:3/277: dread - d4/d12/d33/d43/f65 zero size 2024-02-19T19:06:16.315 INFO:tasks.workunit.client.0.smithi088.stdout:1/259: mknod d6/d10/d26/d4a/c55 0 2024-02-19T19:06:16.320 INFO:tasks.workunit.client.0.smithi088.stdout:7/186: dwrite d14/f29 [0,4194304] 0 2024-02-19T19:06:16.322 INFO:tasks.workunit.client.0.smithi088.stdout:5/274: dwrite d4/d1a/f21 [0,4194304] 0 2024-02-19T19:06:16.324 INFO:tasks.workunit.client.0.smithi088.stdout:5/275: fdatasync d4/d1a/f37 0 2024-02-19T19:06:16.325 INFO:tasks.workunit.client.0.smithi088.stdout:9/192: getdents d1/d21/d2d/d23/d38 0 2024-02-19T19:06:16.332 INFO:tasks.workunit.client.0.smithi088.stdout:0/183: creat d13/f3e x:0 0 0 2024-02-19T19:06:16.333 INFO:tasks.workunit.client.0.smithi088.stdout:6/195: rename d5/d19/f2e to d5/d8/f3f 0 2024-02-19T19:06:16.336 INFO:tasks.workunit.client.0.smithi088.stdout:9/193: dwrite d1/d21/d2d/d23/f25 [4194304,4194304] 0 2024-02-19T19:06:16.337 INFO:tasks.workunit.client.0.smithi088.stdout:6/196: dread d5/f1e [0,4194304] 0 2024-02-19T19:06:16.337 INFO:tasks.workunit.client.0.smithi088.stdout:5/276: symlink d4/d1a/d41/l5c 0 2024-02-19T19:06:16.338 INFO:tasks.workunit.client.0.smithi088.stdout:3/278: dread d4/d1a/d1e/f2f [0,4194304] 0 2024-02-19T19:06:16.338 INFO:tasks.workunit.client.0.smithi088.stdout:6/197: chown d5/d19/d26 8037 1 2024-02-19T19:06:16.340 INFO:tasks.workunit.client.0.smithi088.stdout:4/159: creat d2/d32/f36 x:0 0 0 2024-02-19T19:06:16.341 INFO:tasks.workunit.client.0.smithi088.stdout:5/277: write d4/d1a/f3c [363824,64077] 0 2024-02-19T19:06:16.341 INFO:tasks.workunit.client.0.smithi088.stdout:8/138: sync 2024-02-19T19:06:16.341 INFO:tasks.workunit.client.0.smithi088.stdout:2/128: sync 2024-02-19T19:06:16.343 INFO:tasks.workunit.client.0.smithi088.stdout:4/160: chown d2/d32/l12 1529608520 1 2024-02-19T19:06:16.344 INFO:tasks.workunit.client.0.smithi088.stdout:1/260: rename d6/d10/f16 to d6/d10/d26/d4a/f56 0 2024-02-19T19:06:16.347 INFO:tasks.workunit.client.0.smithi088.stdout:4/161: dread - d2/d32/d25/f2e zero size 2024-02-19T19:06:16.347 INFO:tasks.workunit.client.0.smithi088.stdout:7/187: write d14/d28/d3b/f40 [301936,4003] 0 2024-02-19T19:06:16.349 INFO:tasks.workunit.client.0.smithi088.stdout:5/278: dread d4/f4d [0,4194304] 0 2024-02-19T19:06:16.352 INFO:tasks.workunit.client.0.smithi088.stdout:8/139: dwrite d9/df/d1c/f25 [0,4194304] 0 2024-02-19T19:06:16.354 INFO:tasks.workunit.client.0.smithi088.stdout:8/140: write d9/df/f27 [688883,11079] 0 2024-02-19T19:06:16.355 INFO:tasks.workunit.client.0.smithi088.stdout:7/188: dread d14/d27/f32 [0,4194304] 0 2024-02-19T19:06:16.359 INFO:tasks.workunit.client.0.smithi088.stdout:4/162: dwrite d2/f15 [0,4194304] 0 2024-02-19T19:06:16.360 INFO:tasks.workunit.client.0.smithi088.stdout:4/163: chown d2/d32/f36 0 1 2024-02-19T19:06:16.363 INFO:tasks.workunit.client.0.smithi088.stdout:8/141: dread d9/df/d1c/f1b [4194304,4194304] 0 2024-02-19T19:06:16.364 INFO:tasks.workunit.client.0.smithi088.stdout:8/142: truncate d9/df/d1c/f13 1506338 0 2024-02-19T19:06:16.373 INFO:tasks.workunit.client.0.smithi088.stdout:0/184: symlink d13/l3f 0 2024-02-19T19:06:16.384 INFO:tasks.workunit.client.0.smithi088.stdout:1/261: mknod d6/d10/c57 0 2024-02-19T19:06:16.388 INFO:tasks.workunit.client.0.smithi088.stdout:5/279: mknod d4/d7/c5d 0 2024-02-19T19:06:16.391 INFO:tasks.workunit.client.0.smithi088.stdout:1/262: dwrite d6/d10/d28/d29/f2e [0,4194304] 0 2024-02-19T19:06:16.392 INFO:tasks.workunit.client.0.smithi088.stdout:1/263: chown d6/d21/f3c 6 1 2024-02-19T19:06:16.394 INFO:tasks.workunit.client.0.smithi088.stdout:4/164: rename d2/fd to d2/d35/f37 0 2024-02-19T19:06:16.395 INFO:tasks.workunit.client.0.smithi088.stdout:8/143: mknod d9/df/c2a 0 2024-02-19T19:06:16.397 INFO:tasks.workunit.client.0.smithi088.stdout:9/194: link d1/d21/d2d/d23/f35 d1/d8/d37/f41 0 2024-02-19T19:06:16.402 INFO:tasks.workunit.client.0.smithi088.stdout:5/280: mkdir d4/d1a/d41/d5e 0 2024-02-19T19:06:16.404 INFO:tasks.workunit.client.0.smithi088.stdout:1/264: mknod d6/d10/d28/d29/d4e/c58 0 2024-02-19T19:06:16.406 INFO:tasks.workunit.client.0.smithi088.stdout:4/165: creat d2/d32/d25/f38 x:0 0 0 2024-02-19T19:06:16.407 INFO:tasks.workunit.client.0.smithi088.stdout:4/166: readlink d2/l33 0 2024-02-19T19:06:16.409 INFO:tasks.workunit.client.0.smithi088.stdout:8/144: rename d9/df/d1c/c26 to d9/df/d1c/c2b 0 2024-02-19T19:06:16.411 INFO:tasks.workunit.client.0.smithi088.stdout:9/195: creat d1/d8/f42 x:0 0 0 2024-02-19T19:06:16.413 INFO:tasks.workunit.client.0.smithi088.stdout:7/189: creat d14/f44 x:0 0 0 2024-02-19T19:06:16.415 INFO:tasks.workunit.client.0.smithi088.stdout:5/281: creat d4/d7/f5f x:0 0 0 2024-02-19T19:06:16.418 INFO:tasks.workunit.client.0.smithi088.stdout:1/265: creat d6/d10/d26/d4a/f59 x:0 0 0 2024-02-19T19:06:16.419 INFO:tasks.workunit.client.0.smithi088.stdout:4/167: mkdir d2/d35/d39 0 2024-02-19T19:06:16.421 INFO:tasks.workunit.client.0.smithi088.stdout:8/145: truncate f8 1263665 0 2024-02-19T19:06:16.422 INFO:tasks.workunit.client.0.smithi088.stdout:0/185: truncate d13/f1c 904041 0 2024-02-19T19:06:16.423 INFO:tasks.workunit.client.0.smithi088.stdout:6/198: truncate d5/de/f18 4103106 0 2024-02-19T19:06:16.425 INFO:tasks.workunit.client.0.smithi088.stdout:5/282: mknod d4/d1a/d41/c60 0 2024-02-19T19:06:16.433 INFO:tasks.workunit.client.0.smithi088.stdout:5/283: mkdir d4/d61 0 2024-02-19T19:06:16.434 INFO:tasks.workunit.client.0.smithi088.stdout:1/266: dread d6/d21/f3a [0,4194304] 0 2024-02-19T19:06:16.434 INFO:tasks.workunit.client.0.smithi088.stdout:1/267: dread - d6/d10/f45 zero size 2024-02-19T19:06:16.435 INFO:tasks.workunit.client.0.smithi088.stdout:4/168: mkdir d2/d35/d39/d3a 0 2024-02-19T19:06:16.435 INFO:tasks.workunit.client.0.smithi088.stdout:1/268: write d6/ff [5187484,23434] 0 2024-02-19T19:06:16.439 INFO:tasks.workunit.client.0.smithi088.stdout:8/146: creat d9/df/d1c/d22/f2c x:0 0 0 2024-02-19T19:06:16.439 INFO:tasks.workunit.client.0.smithi088.stdout:4/169: dread d2/d32/fb [0,4194304] 0 2024-02-19T19:06:16.443 INFO:tasks.workunit.client.0.smithi088.stdout:8/147: dread d9/df/d1c/f23 [0,4194304] 0 2024-02-19T19:06:16.449 INFO:tasks.workunit.client.0.smithi088.stdout:6/199: creat d5/d8/d2c/f40 x:0 0 0 2024-02-19T19:06:16.449 INFO:tasks.workunit.client.0.smithi088.stdout:8/148: dwrite d9/df/f1a [4194304,4194304] 0 2024-02-19T19:06:16.450 INFO:tasks.workunit.client.0.smithi088.stdout:6/200: chown d5/d31/d25/f39 1336885392 1 2024-02-19T19:06:16.451 INFO:tasks.workunit.client.0.smithi088.stdout:7/190: link d14/d28/l3f d14/d1d/d3c/d43/l45 0 2024-02-19T19:06:16.455 INFO:tasks.workunit.client.0.smithi088.stdout:5/284: creat d4/d1a/d41/d5e/f62 x:0 0 0 2024-02-19T19:06:16.461 INFO:tasks.workunit.client.0.smithi088.stdout:4/170: mknod d2/d32/c3b 0 2024-02-19T19:06:16.463 INFO:tasks.workunit.client.0.smithi088.stdout:4/171: write d2/d32/f23 [522483,113349] 0 2024-02-19T19:06:16.466 INFO:tasks.workunit.client.0.smithi088.stdout:5/285: creat d4/d7/f63 x:0 0 0 2024-02-19T19:06:16.471 INFO:tasks.workunit.client.0.smithi088.stdout:7/191: mknod d14/c46 0 2024-02-19T19:06:16.475 INFO:tasks.workunit.client.0.smithi088.stdout:5/286: chown d4/d1a/c56 123897143 1 2024-02-19T19:06:16.475 INFO:tasks.workunit.client.0.smithi088.stdout:4/172: dread d2/d8/f16 [0,4194304] 0 2024-02-19T19:06:16.476 INFO:tasks.workunit.client.0.smithi088.stdout:8/149: dread d9/df/f19 [0,4194304] 0 2024-02-19T19:06:16.478 INFO:tasks.workunit.client.0.smithi088.stdout:4/173: stat d2/d32/f1c 0 2024-02-19T19:06:16.478 INFO:tasks.workunit.client.0.smithi088.stdout:5/287: fsync d4/f12 0 2024-02-19T19:06:16.478 INFO:tasks.workunit.client.0.smithi088.stdout:6/201: dwrite d5/d8/f3f [0,4194304] 0 2024-02-19T19:06:16.478 INFO:tasks.workunit.client.0.smithi088.stdout:8/150: stat d9/df/d1c/f16 0 2024-02-19T19:06:16.479 INFO:tasks.workunit.client.0.smithi088.stdout:4/174: dread - d2/d32/d25/f38 zero size 2024-02-19T19:06:16.480 INFO:tasks.workunit.client.0.smithi088.stdout:6/202: dread - d5/d8/d2c/f40 zero size 2024-02-19T19:06:16.482 INFO:tasks.workunit.client.0.smithi088.stdout:7/192: rename d14/d1d/d3c/d43/l45 to d14/d1d/d3c/d43/l47 0 2024-02-19T19:06:16.484 INFO:tasks.workunit.client.0.smithi088.stdout:8/151: unlink d9/df/d1c/l1d 0 2024-02-19T19:06:16.486 INFO:tasks.workunit.client.0.smithi088.stdout:5/288: creat d4/d7/f64 x:0 0 0 2024-02-19T19:06:16.489 INFO:tasks.workunit.client.0.smithi088.stdout:8/152: write d9/df/d1c/f1b [12940690,81816] 0 2024-02-19T19:06:16.491 INFO:tasks.workunit.client.0.smithi088.stdout:4/175: symlink d2/d35/d39/d3a/l3c 0 2024-02-19T19:06:16.494 INFO:tasks.workunit.client.0.smithi088.stdout:6/203: creat d5/d19/f41 x:0 0 0 2024-02-19T19:06:16.495 INFO:tasks.workunit.client.0.smithi088.stdout:5/289: dwrite d4/d7/f15 [0,4194304] 0 2024-02-19T19:06:16.496 INFO:tasks.workunit.client.0.smithi088.stdout:6/204: dread d5/f1e [0,4194304] 0 2024-02-19T19:06:16.497 INFO:tasks.workunit.client.0.smithi088.stdout:4/176: symlink d2/d35/d39/d3a/l3d 0 2024-02-19T19:06:16.500 INFO:tasks.workunit.client.0.smithi088.stdout:4/177: write d2/d32/d25/f38 [1042550,112128] 0 2024-02-19T19:06:16.502 INFO:tasks.workunit.client.0.smithi088.stdout:4/178: truncate d2/d32/d25/f30 4219294 0 2024-02-19T19:06:16.503 INFO:tasks.workunit.client.0.smithi088.stdout:8/153: fdatasync f8 0 2024-02-19T19:06:16.504 INFO:tasks.workunit.client.0.smithi088.stdout:8/154: chown d9/ld 3273 1 2024-02-19T19:06:16.506 INFO:tasks.workunit.client.0.smithi088.stdout:6/205: rename d5/d7/l3d to d5/d19/d26/l42 0 2024-02-19T19:06:16.508 INFO:tasks.workunit.client.0.smithi088.stdout:7/193: getdents d14/d1d 0 2024-02-19T19:06:16.511 INFO:tasks.workunit.client.0.smithi088.stdout:5/290: mkdir d4/d61/d65 0 2024-02-19T19:06:16.513 INFO:tasks.workunit.client.0.smithi088.stdout:8/155: stat d9/df/d1c/d22/l24 0 2024-02-19T19:06:16.513 INFO:tasks.workunit.client.0.smithi088.stdout:8/156: chown d9/df/d1c/f13 2400 1 2024-02-19T19:06:16.514 INFO:tasks.workunit.client.0.smithi088.stdout:1/269: write d6/f1e [384886,123901] 0 2024-02-19T19:06:16.515 INFO:tasks.workunit.client.0.smithi088.stdout:1/270: stat d6/d10/f15 0 2024-02-19T19:06:16.516 INFO:tasks.workunit.client.0.smithi088.stdout:7/194: rename d14/d28/l2f to d14/d1d/d3c/l48 0 2024-02-19T19:06:16.517 INFO:tasks.workunit.client.0.smithi088.stdout:5/291: mknod d4/d1a/c66 0 2024-02-19T19:06:16.520 INFO:tasks.workunit.client.0.smithi088.stdout:8/157: mkdir d9/df/d1c/d2d 0 2024-02-19T19:06:16.521 INFO:tasks.workunit.client.0.smithi088.stdout:7/195: mkdir d14/d27/d49 0 2024-02-19T19:06:16.525 INFO:tasks.workunit.client.0.smithi088.stdout:5/292: dwrite d4/d7/f63 [0,4194304] 0 2024-02-19T19:06:16.526 INFO:tasks.workunit.client.0.smithi088.stdout:5/293: write d4/f11 [489174,74545] 0 2024-02-19T19:06:16.529 INFO:tasks.workunit.client.0.smithi088.stdout:1/271: dread d6/d21/f3c [0,4194304] 0 2024-02-19T19:06:16.531 INFO:tasks.workunit.client.0.smithi088.stdout:5/294: mknod d4/d1a/d22/d57/c67 0 2024-02-19T19:06:16.535 INFO:tasks.workunit.client.0.smithi088.stdout:8/158: rename d9/df/d1c/f23 to d9/f2e 0 2024-02-19T19:06:16.535 INFO:tasks.workunit.client.0.smithi088.stdout:1/272: dwrite d6/f17 [0,4194304] 0 2024-02-19T19:06:16.540 INFO:tasks.workunit.client.0.smithi088.stdout:1/273: symlink d6/d10/d26/d4c/l5a 0 2024-02-19T19:06:16.541 INFO:tasks.workunit.client.0.smithi088.stdout:1/274: write d6/f1b [221694,94517] 0 2024-02-19T19:06:16.542 INFO:tasks.workunit.client.0.smithi088.stdout:8/159: rename d9/df/l17 to d9/df/d1c/d2d/l2f 0 2024-02-19T19:06:16.544 INFO:tasks.workunit.client.0.smithi088.stdout:1/275: chown d6/c3e 1949460 1 2024-02-19T19:06:16.547 INFO:tasks.workunit.client.0.smithi088.stdout:1/276: symlink d6/d10/l5b 0 2024-02-19T19:06:16.554 INFO:tasks.workunit.client.0.smithi088.stdout:6/206: fsync d5/d19/f41 0 2024-02-19T19:06:16.555 INFO:tasks.workunit.client.0.smithi088.stdout:1/277: dwrite d6/f1e [0,4194304] 0 2024-02-19T19:06:16.557 INFO:tasks.workunit.client.0.smithi088.stdout:1/278: chown d6/d10/d26/d4c 1870 1 2024-02-19T19:06:16.562 INFO:tasks.workunit.client.0.smithi088.stdout:1/279: creat d6/d10/d26/d4c/f5c x:0 0 0 2024-02-19T19:06:16.569 INFO:tasks.workunit.client.0.smithi088.stdout:1/280: dwrite d6/ff [0,4194304] 0 2024-02-19T19:06:16.574 INFO:tasks.workunit.client.0.smithi088.stdout:1/281: rename d6/d10/d28/f4f to d6/d10/f5d 0 2024-02-19T19:06:16.583 INFO:tasks.workunit.client.0.smithi088.stdout:1/282: dwrite d6/d10/d26/d4c/f5c [0,4194304] 0 2024-02-19T19:06:16.590 INFO:tasks.workunit.client.0.smithi088.stdout:5/295: write d4/d1a/f1b [3811703,2129] 0 2024-02-19T19:06:16.593 INFO:tasks.workunit.client.0.smithi088.stdout:6/207: write d5/d31/d25/f39 [1098696,50904] 0 2024-02-19T19:06:16.593 INFO:tasks.workunit.client.0.smithi088.stdout:8/160: chown d9/df/d1c/d2d/l2f 27 1 2024-02-19T19:06:16.597 INFO:tasks.workunit.client.0.smithi088.stdout:6/208: chown d5/d19/d26 417 1 2024-02-19T19:06:16.598 INFO:tasks.workunit.client.0.smithi088.stdout:5/296: rename d4/d1a/c56 to d4/d1a/d22/c68 0 2024-02-19T19:06:16.599 INFO:tasks.workunit.client.0.smithi088.stdout:1/283: dwrite d6/d21/f36 [0,4194304] 0 2024-02-19T19:06:16.602 INFO:tasks.workunit.client.0.smithi088.stdout:5/297: symlink d4/l69 0 2024-02-19T19:06:16.604 INFO:tasks.workunit.client.0.smithi088.stdout:5/298: read - d4/d1a/d22/f4e zero size 2024-02-19T19:06:16.606 INFO:tasks.workunit.client.0.smithi088.stdout:8/161: dwrite d9/df/d1c/f1f [0,4194304] 0 2024-02-19T19:06:16.611 INFO:tasks.workunit.client.0.smithi088.stdout:8/162: dread f8 [0,4194304] 0 2024-02-19T19:06:16.615 INFO:tasks.workunit.client.0.smithi088.stdout:1/284: getdents d6/d10/d26/d4a 0 2024-02-19T19:06:16.617 INFO:tasks.workunit.client.0.smithi088.stdout:5/299: dwrite d4/d7/f15 [4194304,4194304] 0 2024-02-19T19:06:16.617 INFO:tasks.workunit.client.0.smithi088.stdout:1/285: chown d6/l34 3 1 2024-02-19T19:06:16.618 INFO:tasks.workunit.client.0.smithi088.stdout:5/300: truncate d4/d7/f33 625277 0 2024-02-19T19:06:16.622 INFO:tasks.workunit.client.0.smithi088.stdout:8/163: mknod d9/df/d1c/c30 0 2024-02-19T19:06:16.624 INFO:tasks.workunit.client.0.smithi088.stdout:6/209: getdents d5 0 2024-02-19T19:06:16.626 INFO:tasks.workunit.client.0.smithi088.stdout:6/210: fdatasync d5/d19/f2f 0 2024-02-19T19:06:16.627 INFO:tasks.workunit.client.0.smithi088.stdout:1/286: dread d6/ff [0,4194304] 0 2024-02-19T19:06:16.629 INFO:tasks.workunit.client.0.smithi088.stdout:6/211: write d5/d19/f2f [2365291,58849] 0 2024-02-19T19:06:16.630 INFO:tasks.workunit.client.0.smithi088.stdout:1/287: chown d6/d10/d26/c46 447462141 1 2024-02-19T19:06:16.630 INFO:tasks.workunit.client.0.smithi088.stdout:5/301: rename d4/d7/f2f to d4/d1a/d22/d57/f6a 0 2024-02-19T19:06:16.630 INFO:tasks.workunit.client.0.smithi088.stdout:8/164: dread d9/df/d1c/f1f [0,4194304] 0 2024-02-19T19:06:16.630 INFO:tasks.workunit.client.0.smithi088.stdout:5/302: readlink d4/d1a/l3e 0 2024-02-19T19:06:16.631 INFO:tasks.workunit.client.0.smithi088.stdout:6/212: mknod d5/de/c43 0 2024-02-19T19:06:16.632 INFO:tasks.workunit.client.0.smithi088.stdout:3/279: sync 2024-02-19T19:06:16.638 INFO:tasks.workunit.client.0.smithi088.stdout:8/165: mkdir d9/d31 0 2024-02-19T19:06:16.641 INFO:tasks.workunit.client.0.smithi088.stdout:1/288: dwrite d6/f1b [0,4194304] 0 2024-02-19T19:06:16.642 INFO:tasks.workunit.client.0.smithi088.stdout:3/280: rmdir d4/d1a/d1e/d31/d64 0 2024-02-19T19:06:16.643 INFO:tasks.workunit.client.0.smithi088.stdout:6/213: dread d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:16.646 INFO:tasks.workunit.client.0.smithi088.stdout:5/303: dwrite d4/fe [0,4194304] 0 2024-02-19T19:06:16.647 INFO:tasks.workunit.client.0.smithi088.stdout:5/304: chown d4/d7/f36 377612 1 2024-02-19T19:06:16.654 INFO:tasks.workunit.client.0.smithi088.stdout:8/166: rename d9/df/d1c/c2b to d9/df/c32 0 2024-02-19T19:06:16.655 INFO:tasks.workunit.client.0.smithi088.stdout:5/305: creat d4/d1a/d22/d57/f6b x:0 0 0 2024-02-19T19:06:16.660 INFO:tasks.workunit.client.0.smithi088.stdout:1/289: dread d6/d10/d28/d29/f2e [0,4194304] 0 2024-02-19T19:06:16.660 INFO:tasks.workunit.client.0.smithi088.stdout:6/214: unlink d5/d8/l33 0 2024-02-19T19:06:16.660 INFO:tasks.workunit.client.0.smithi088.stdout:3/281: truncate d4/d1a/d1e/d31/d32/f50 654309 0 2024-02-19T19:06:16.663 INFO:tasks.workunit.client.0.smithi088.stdout:1/290: stat d6/d10/d28/d29/d4e/c58 0 2024-02-19T19:06:16.665 INFO:tasks.workunit.client.0.smithi088.stdout:8/167: dwrite d9/df/d1c/f25 [4194304,4194304] 0 2024-02-19T19:06:16.665 INFO:tasks.workunit.client.0.smithi088.stdout:8/168: stat f5 0 2024-02-19T19:06:16.674 INFO:tasks.workunit.client.0.smithi088.stdout:6/215: dwrite d5/d31/d25/f39 [4194304,4194304] 0 2024-02-19T19:06:16.675 INFO:tasks.workunit.client.0.smithi088.stdout:8/169: rename d9/df/d1c/d22/f2c to d9/df/d1c/f33 0 2024-02-19T19:06:16.680 INFO:tasks.workunit.client.0.smithi088.stdout:8/170: dread d9/df/d1c/f25 [0,4194304] 0 2024-02-19T19:06:16.680 INFO:tasks.workunit.client.0.smithi088.stdout:6/216: fsync d5/ff 0 2024-02-19T19:06:16.686 INFO:tasks.workunit.client.0.smithi088.stdout:1/291: link d6/d10/c57 d6/d10/c5e 0 2024-02-19T19:06:16.688 INFO:tasks.workunit.client.0.smithi088.stdout:5/306: dread d4/d7/f1d [0,4194304] 0 2024-02-19T19:06:16.688 INFO:tasks.workunit.client.0.smithi088.stdout:8/171: dwrite d9/df/d1c/f16 [0,4194304] 0 2024-02-19T19:06:16.693 INFO:tasks.workunit.client.0.smithi088.stdout:6/217: readlink d5/d8/l1d 0 2024-02-19T19:06:16.700 INFO:tasks.workunit.client.0.smithi088.stdout:5/307: mknod d4/d1a/c6c 0 2024-02-19T19:06:16.708 INFO:tasks.workunit.client.0.smithi088.stdout:3/282: dread d4/f16 [0,4194304] 0 2024-02-19T19:06:16.710 INFO:tasks.workunit.client.0.smithi088.stdout:1/292: dread d6/f17 [4194304,4194304] 0 2024-02-19T19:06:16.711 INFO:tasks.workunit.client.0.smithi088.stdout:1/293: chown d6/d10/d26/d53 6719 1 2024-02-19T19:06:16.712 INFO:tasks.workunit.client.0.smithi088.stdout:1/294: write d6/d10/d26/d4c/f40 [3018375,68604] 0 2024-02-19T19:06:16.716 INFO:tasks.workunit.client.0.smithi088.stdout:6/218: link d5/ff d5/d19/d26/f44 0 2024-02-19T19:06:16.716 INFO:tasks.workunit.client.0.smithi088.stdout:6/219: chown d5/d19 0 1 2024-02-19T19:06:16.717 INFO:tasks.workunit.client.0.smithi088.stdout:5/308: dwrite d4/d1a/d22/d57/f6a [0,4194304] 0 2024-02-19T19:06:16.725 INFO:tasks.workunit.client.0.smithi088.stdout:1/295: rename d6/d10/f15 to d6/d10/d26/f5f 0 2024-02-19T19:06:16.725 INFO:tasks.workunit.client.0.smithi088.stdout:1/296: dread - d6/f52 zero size 2024-02-19T19:06:16.727 INFO:tasks.workunit.client.0.smithi088.stdout:1/297: truncate d6/d10/d26/f39 332588 0 2024-02-19T19:06:16.732 INFO:tasks.workunit.client.0.smithi088.stdout:6/220: mkdir d5/d45 0 2024-02-19T19:06:16.734 INFO:tasks.workunit.client.0.smithi088.stdout:5/309: unlink d4/d7/f33 0 2024-02-19T19:06:16.739 INFO:tasks.workunit.client.0.smithi088.stdout:8/172: link d9/df/l28 d9/l34 0 2024-02-19T19:06:16.739 INFO:tasks.workunit.client.0.smithi088.stdout:8/173: chown d9/l21 635 1 2024-02-19T19:06:16.740 INFO:tasks.workunit.client.0.smithi088.stdout:9/196: sync 2024-02-19T19:06:16.740 INFO:tasks.workunit.client.0.smithi088.stdout:2/129: sync 2024-02-19T19:06:16.740 INFO:tasks.workunit.client.0.smithi088.stdout:8/174: stat d9/df/d1c/c30 0 2024-02-19T19:06:16.740 INFO:tasks.workunit.client.0.smithi088.stdout:2/130: readlink d3/df/l26 0 2024-02-19T19:06:16.750 INFO:tasks.workunit.client.0.smithi088.stdout:5/310: rmdir d4/d1a/d41 39 2024-02-19T19:06:16.752 INFO:tasks.workunit.client.0.smithi088.stdout:6/221: dwrite d5/d8/f3f [0,4194304] 0 2024-02-19T19:06:16.756 INFO:tasks.workunit.client.0.smithi088.stdout:9/197: creat d1/d21/f43 x:0 0 0 2024-02-19T19:06:16.759 INFO:tasks.workunit.client.0.smithi088.stdout:8/175: rmdir d9/df/d1c 39 2024-02-19T19:06:16.769 INFO:tasks.workunit.client.0.smithi088.stdout:6/222: chown d5/de/f22 242 1 2024-02-19T19:06:16.769 INFO:tasks.workunit.client.0.smithi088.stdout:3/283: getdents d4/d12/d33/d58 0 2024-02-19T19:06:16.773 INFO:tasks.workunit.client.0.smithi088.stdout:2/131: getdents d3/df 0 2024-02-19T19:06:16.777 INFO:tasks.workunit.client.0.smithi088.stdout:1/298: rmdir d6/d10/d26/d53 0 2024-02-19T19:06:16.778 INFO:tasks.workunit.client.0.smithi088.stdout:5/311: mkdir d4/d1a/d6d 0 2024-02-19T19:06:16.778 INFO:tasks.workunit.client.0.smithi088.stdout:8/176: dwrite d9/df/d1c/f1b [4194304,4194304] 0 2024-02-19T19:06:16.782 INFO:tasks.workunit.client.0.smithi088.stdout:3/284: creat d4/d1a/d23/f69 x:0 0 0 2024-02-19T19:06:16.786 INFO:tasks.workunit.client.0.smithi088.stdout:3/285: dread d4/d12/f27 [0,4194304] 0 2024-02-19T19:06:16.788 INFO:tasks.workunit.client.0.smithi088.stdout:2/132: dread d3/f4 [4194304,4194304] 0 2024-02-19T19:06:16.789 INFO:tasks.workunit.client.0.smithi088.stdout:5/312: unlink d4/d1a/l35 0 2024-02-19T19:06:16.791 INFO:tasks.workunit.client.0.smithi088.stdout:6/223: truncate d5/de/f18 2140718 0 2024-02-19T19:06:16.843 INFO:tasks.workunit.client.0.smithi088.stdout:6/224: dread d5/d8/f3f [0,4194304] 0 2024-02-19T19:06:16.843 INFO:tasks.workunit.client.0.smithi088.stdout:9/198: truncate d1/d21/d2d/f2a 3359176 0 2024-02-19T19:06:16.844 INFO:tasks.workunit.client.0.smithi088.stdout:1/299: write f5 [3366938,88077] 0 2024-02-19T19:06:16.845 INFO:tasks.workunit.client.0.smithi088.stdout:2/133: mknod d3/c29 0 2024-02-19T19:06:16.851 INFO:tasks.workunit.client.0.smithi088.stdout:3/286: truncate d4/f38 217021 0 2024-02-19T19:06:16.852 INFO:tasks.workunit.client.0.smithi088.stdout:9/199: rmdir d1/d21/d2d 39 2024-02-19T19:06:16.853 INFO:tasks.workunit.client.0.smithi088.stdout:5/313: creat d4/d1a/f6e x:0 0 0 2024-02-19T19:06:16.860 INFO:tasks.workunit.client.0.smithi088.stdout:9/200: truncate d1/d13/f36 408780 0 2024-02-19T19:06:16.860 INFO:tasks.workunit.client.0.smithi088.stdout:1/300: rename d6/d10/d28 to d6/d10/d60 0 2024-02-19T19:06:16.863 INFO:tasks.workunit.client.0.smithi088.stdout:6/225: getdents d5/d45 0 2024-02-19T19:06:16.863 INFO:tasks.workunit.client.0.smithi088.stdout:7/196: sync 2024-02-19T19:06:16.864 INFO:tasks.workunit.client.0.smithi088.stdout:0/186: sync 2024-02-19T19:06:16.864 INFO:tasks.workunit.client.0.smithi088.stdout:4/179: sync 2024-02-19T19:06:16.864 INFO:tasks.workunit.client.0.smithi088.stdout:9/201: chown d1/d8/d3e 116367 1 2024-02-19T19:06:16.864 INFO:tasks.workunit.client.0.smithi088.stdout:7/197: stat d14/c46 0 2024-02-19T19:06:16.864 INFO:tasks.workunit.client.0.smithi088.stdout:5/314: write d4/d1a/d22/f4c [965036,58048] 0 2024-02-19T19:06:16.864 INFO:tasks.workunit.client.0.smithi088.stdout:0/187: chown d13/d24/d25/d2a/c30 995 1 2024-02-19T19:06:16.864 INFO:tasks.workunit.client.0.smithi088.stdout:2/134: mknod d3/c2a 0 2024-02-19T19:06:16.865 INFO:tasks.workunit.client.0.smithi088.stdout:2/135: readlink d3/l18 0 2024-02-19T19:06:16.866 INFO:tasks.workunit.client.0.smithi088.stdout:7/198: read d14/d1d/d3c/f3d [231235,87591] 0 2024-02-19T19:06:16.868 INFO:tasks.workunit.client.0.smithi088.stdout:1/301: symlink d6/d10/d60/d29/d4e/l61 0 2024-02-19T19:06:16.870 INFO:tasks.workunit.client.0.smithi088.stdout:5/315: dread d4/d1a/f21 [0,4194304] 0 2024-02-19T19:06:16.870 INFO:tasks.workunit.client.0.smithi088.stdout:9/202: creat d1/d8/d37/f44 x:0 0 0 2024-02-19T19:06:16.873 INFO:tasks.workunit.client.0.smithi088.stdout:4/180: dwrite d2/d8/f1a [0,4194304] 0 2024-02-19T19:06:16.876 INFO:tasks.workunit.client.0.smithi088.stdout:3/287: creat d4/f6a x:0 0 0 2024-02-19T19:06:16.877 INFO:tasks.workunit.client.0.smithi088.stdout:3/288: stat d4/c5b 0 2024-02-19T19:06:16.878 INFO:tasks.workunit.client.0.smithi088.stdout:0/188: creat d13/d24/d25/d2a/d34/f40 x:0 0 0 2024-02-19T19:06:16.878 INFO:tasks.workunit.client.0.smithi088.stdout:2/136: creat d3/d14/f2b x:0 0 0 2024-02-19T19:06:16.882 INFO:tasks.workunit.client.0.smithi088.stdout:6/226: unlink d5/d8/c24 0 2024-02-19T19:06:16.883 INFO:tasks.workunit.client.0.smithi088.stdout:1/302: mknod d6/d10/d26/c62 0 2024-02-19T19:06:16.886 INFO:tasks.workunit.client.0.smithi088.stdout:5/316: write d4/d7/f1d [4664555,80440] 0 2024-02-19T19:06:16.891 INFO:tasks.workunit.client.0.smithi088.stdout:2/137: symlink d3/d14/l2c 0 2024-02-19T19:06:16.896 INFO:tasks.workunit.client.0.smithi088.stdout:9/203: creat d1/d8/d3e/f45 x:0 0 0 2024-02-19T19:06:16.901 INFO:tasks.workunit.client.0.smithi088.stdout:3/289: dread d4/d1a/d23/f5d [0,4194304] 0 2024-02-19T19:06:16.901 INFO:tasks.workunit.client.0.smithi088.stdout:2/138: dwrite d3/d14/f1d [0,4194304] 0 2024-02-19T19:06:16.904 INFO:tasks.workunit.client.0.smithi088.stdout:1/303: dwrite d6/d10/f5d [0,4194304] 0 2024-02-19T19:06:16.905 INFO:tasks.workunit.client.0.smithi088.stdout:2/139: write d3/f23 [9959,16317] 0 2024-02-19T19:06:16.905 INFO:tasks.workunit.client.0.smithi088.stdout:7/199: dread d14/f1f [0,4194304] 0 2024-02-19T19:06:16.916 INFO:tasks.workunit.client.0.smithi088.stdout:5/317: write d4/f12 [3589921,25542] 0 2024-02-19T19:06:16.918 INFO:tasks.workunit.client.0.smithi088.stdout:3/290: fdatasync d4/f7 0 2024-02-19T19:06:16.918 INFO:tasks.workunit.client.0.smithi088.stdout:6/227: creat d5/d8/d3a/f46 x:0 0 0 2024-02-19T19:06:16.920 INFO:tasks.workunit.client.0.smithi088.stdout:1/304: unlink d6/lc 0 2024-02-19T19:06:16.923 INFO:tasks.workunit.client.0.smithi088.stdout:9/204: symlink d1/l46 0 2024-02-19T19:06:16.923 INFO:tasks.workunit.client.0.smithi088.stdout:9/205: chown d1 0 1 2024-02-19T19:06:16.926 INFO:tasks.workunit.client.0.smithi088.stdout:1/305: mkdir d6/d10/d60/d29/d63 0 2024-02-19T19:06:16.928 INFO:tasks.workunit.client.0.smithi088.stdout:2/140: fsync d3/f6 0 2024-02-19T19:06:16.930 INFO:tasks.workunit.client.0.smithi088.stdout:7/200: creat d14/d28/d3b/f4a x:0 0 0 2024-02-19T19:06:16.934 INFO:tasks.workunit.client.0.smithi088.stdout:6/228: write d5/d19/d26/f27 [922093,30770] 0 2024-02-19T19:06:16.935 INFO:tasks.workunit.client.0.smithi088.stdout:1/306: mkdir d6/d21/d64 0 2024-02-19T19:06:16.939 INFO:tasks.workunit.client.0.smithi088.stdout:9/206: creat d1/d21/d2d/f47 x:0 0 0 2024-02-19T19:06:16.941 INFO:tasks.workunit.client.0.smithi088.stdout:3/291: creat d4/f6b x:0 0 0 2024-02-19T19:06:16.943 INFO:tasks.workunit.client.0.smithi088.stdout:5/318: dread d4/d7/f31 [0,4194304] 0 2024-02-19T19:06:16.944 INFO:tasks.workunit.client.0.smithi088.stdout:5/319: write d4/f42 [2072158,16944] 0 2024-02-19T19:06:16.950 INFO:tasks.workunit.client.0.smithi088.stdout:7/201: dwrite d14/f1f [0,4194304] 0 2024-02-19T19:06:16.952 INFO:tasks.workunit.client.0.smithi088.stdout:6/229: write d5/d19/d26/f44 [1171728,106543] 0 2024-02-19T19:06:16.952 INFO:tasks.workunit.client.0.smithi088.stdout:1/307: dwrite d6/d21/f3a [0,4194304] 0 2024-02-19T19:06:16.957 INFO:tasks.workunit.client.0.smithi088.stdout:2/141: link d3/l21 d3/d1b/l2d 0 2024-02-19T19:06:16.960 INFO:tasks.workunit.client.0.smithi088.stdout:9/207: mknod d1/d21/d2d/d23/d38/c48 0 2024-02-19T19:06:16.962 INFO:tasks.workunit.client.0.smithi088.stdout:1/308: unlink d6/d10/l13 0 2024-02-19T19:06:16.964 INFO:tasks.workunit.client.0.smithi088.stdout:2/142: dwrite d3/d1b/f1c [0,4194304] 0 2024-02-19T19:06:16.966 INFO:tasks.workunit.client.0.smithi088.stdout:5/320: symlink d4/d1a/l6f 0 2024-02-19T19:06:16.971 INFO:tasks.workunit.client.0.smithi088.stdout:9/208: symlink d1/d8/l49 0 2024-02-19T19:06:16.974 INFO:tasks.workunit.client.0.smithi088.stdout:2/143: creat d3/d1b/f2e x:0 0 0 2024-02-19T19:06:16.981 INFO:tasks.workunit.client.0.smithi088.stdout:2/144: dwrite d3/d14/f1d [0,4194304] 0 2024-02-19T19:06:16.982 INFO:tasks.workunit.client.0.smithi088.stdout:5/321: rename d4/d1a/c66 to d4/d1a/c70 0 2024-02-19T19:06:16.983 INFO:tasks.workunit.client.0.smithi088.stdout:5/322: write d4/d1a/d22/f4c [1628259,77392] 0 2024-02-19T19:06:16.987 INFO:tasks.workunit.client.0.smithi088.stdout:2/145: mknod d3/d1b/c2f 0 2024-02-19T19:06:16.991 INFO:tasks.workunit.client.0.smithi088.stdout:5/323: chown d4/d1a/c52 300 1 2024-02-19T19:06:16.992 INFO:tasks.workunit.client.0.smithi088.stdout:5/324: write d4/d1a/f37 [375525,97900] 0 2024-02-19T19:06:16.994 INFO:tasks.workunit.client.0.smithi088.stdout:9/209: getdents d1/d13 0 2024-02-19T19:06:16.996 INFO:tasks.workunit.client.0.smithi088.stdout:2/146: symlink d3/df/l30 0 2024-02-19T19:06:17.000 INFO:tasks.workunit.client.0.smithi088.stdout:9/210: chown d1/ca 153770 1 2024-02-19T19:06:17.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:17.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:17.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:16 smithi132 ceph-mon[70676]: Reconfiguring osd.3 (monmap changed)... 2024-02-19T19:06:17.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-19T19:06:17.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:17.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:16 smithi132 ceph-mon[70676]: Reconfiguring daemon osd.3 on smithi132 2024-02-19T19:06:17.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:16 smithi132 ceph-mon[70676]: pgmap v42: 65 pgs: 65 active+clean; 526 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 42 MiB/s wr, 502 op/s 2024-02-19T19:06:17.003 INFO:tasks.workunit.client.0.smithi088.stdout:6/230: write d5/d8/f3f [3969555,127486] 0 2024-02-19T19:06:17.006 INFO:tasks.workunit.client.0.smithi088.stdout:9/211: mkdir d1/d8/d4a 0 2024-02-19T19:06:17.011 INFO:tasks.workunit.client.0.smithi088.stdout:3/292: dwrite d4/d1a/d1e/d31/d32/f50 [0,4194304] 0 2024-02-19T19:06:17.017 INFO:tasks.workunit.client.0.smithi088.stdout:7/202: dwrite f2 [0,4194304] 0 2024-02-19T19:06:17.017 INFO:tasks.workunit.client.0.smithi088.stdout:5/325: link d4/f11 d4/d1a/d41/f71 0 2024-02-19T19:06:17.018 INFO:tasks.workunit.client.0.smithi088.stdout:9/212: truncate d1/f10 161852 0 2024-02-19T19:06:17.021 INFO:tasks.workunit.client.0.smithi088.stdout:5/326: write d4/d7/f1d [2042819,69033] 0 2024-02-19T19:06:17.021 INFO:tasks.workunit.client.0.smithi088.stdout:9/213: fdatasync d1/d8/f42 0 2024-02-19T19:06:17.022 INFO:tasks.workunit.client.0.smithi088.stdout:3/293: dwrite d4/f40 [0,4194304] 0 2024-02-19T19:06:17.026 INFO:tasks.workunit.client.0.smithi088.stdout:2/147: creat d3/f31 x:0 0 0 2024-02-19T19:06:17.028 INFO:tasks.workunit.client.0.smithi088.stdout:2/148: readlink d3/le 0 2024-02-19T19:06:17.033 INFO:tasks.workunit.client.0.smithi088.stdout:2/149: write d3/f4 [6788985,55779] 0 2024-02-19T19:06:17.033 INFO:tasks.workunit.client.0.smithi088.stdout:9/214: dwrite d1/f20 [0,4194304] 0 2024-02-19T19:06:17.035 INFO:tasks.workunit.client.0.smithi088.stdout:7/203: creat d14/d28/d3b/f4b x:0 0 0 2024-02-19T19:06:17.037 INFO:tasks.workunit.client.0.smithi088.stdout:6/231: dread d5/de/f18 [0,4194304] 0 2024-02-19T19:06:17.037 INFO:tasks.workunit.client.0.smithi088.stdout:9/215: chown d1/d8/l22 663596372 1 2024-02-19T19:06:17.038 INFO:tasks.workunit.client.0.smithi088.stdout:5/327: mknod d4/d1a/d41/c72 0 2024-02-19T19:06:17.038 INFO:tasks.workunit.client.0.smithi088.stdout:3/294: dwrite d4/f18 [0,4194304] 0 2024-02-19T19:06:17.040 INFO:tasks.workunit.client.0.smithi088.stdout:6/232: write d5/d19/d26/f44 [590670,119441] 0 2024-02-19T19:06:17.049 INFO:tasks.workunit.client.0.smithi088.stdout:9/216: dwrite d1/d21/d2d/f47 [0,4194304] 0 2024-02-19T19:06:17.050 INFO:tasks.workunit.client.0.smithi088.stdout:6/233: dwrite d5/d19/d26/f27 [0,4194304] 0 2024-02-19T19:06:17.051 INFO:tasks.workunit.client.0.smithi088.stdout:9/217: dread - d1/f2c zero size 2024-02-19T19:06:17.055 INFO:tasks.workunit.client.0.smithi088.stdout:7/204: unlink d14/d1d/f42 0 2024-02-19T19:06:17.058 INFO:tasks.workunit.client.0.smithi088.stdout:9/218: dwrite d1/d13/f36 [0,4194304] 0 2024-02-19T19:06:17.060 INFO:tasks.workunit.client.0.smithi088.stdout:5/328: stat d4/d1a/d22/c68 0 2024-02-19T19:06:17.062 INFO:tasks.workunit.client.0.smithi088.stdout:5/329: fdatasync d4/d7/f36 0 2024-02-19T19:06:17.062 INFO:tasks.workunit.client.0.smithi088.stdout:5/330: stat d4/d1a/d41 0 2024-02-19T19:06:17.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:17.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:17.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:16 smithi088 ceph-mon[88643]: Reconfiguring osd.3 (monmap changed)... 2024-02-19T19:06:17.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-19T19:06:17.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:17.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:16 smithi088 ceph-mon[88643]: Reconfiguring daemon osd.3 on smithi132 2024-02-19T19:06:17.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:16 smithi088 ceph-mon[88643]: pgmap v42: 65 pgs: 65 active+clean; 526 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 42 MiB/s wr, 502 op/s 2024-02-19T19:06:17.176 INFO:tasks.workunit.client.0.smithi088.stdout:2/150: creat d3/d1f/f32 x:0 0 0 2024-02-19T19:06:17.177 INFO:tasks.workunit.client.0.smithi088.stdout:2/151: read d3/f23 [16227,22925] 0 2024-02-19T19:06:17.187 INFO:tasks.workunit.client.0.smithi088.stdout:7/205: truncate d14/d28/d3b/f40 1353503 0 2024-02-19T19:06:17.187 INFO:tasks.workunit.client.0.smithi088.stdout:7/206: read f2 [286212,60600] 0 2024-02-19T19:06:17.189 INFO:tasks.workunit.client.0.smithi088.stdout:9/219: symlink d1/d21/d2d/d23/d38/l4b 0 2024-02-19T19:06:17.189 INFO:tasks.workunit.client.0.smithi088.stdout:9/220: write d1/f20 [1860282,45029] 0 2024-02-19T19:06:17.192 INFO:tasks.workunit.client.0.smithi088.stdout:9/221: read d1/d21/d2d/d18/d1f/f2e [2484752,130245] 0 2024-02-19T19:06:17.198 INFO:tasks.workunit.client.0.smithi088.stdout:2/152: rmdir d3/d1f 39 2024-02-19T19:06:17.198 INFO:tasks.workunit.client.0.smithi088.stdout:5/331: dwrite d4/d1a/d22/f3d [0,4194304] 0 2024-02-19T19:06:17.202 INFO:tasks.workunit.client.0.smithi088.stdout:3/295: creat d4/d12/f6c x:0 0 0 2024-02-19T19:06:17.213 INFO:tasks.workunit.client.0.smithi088.stdout:9/222: unlink d1/d13/fd 0 2024-02-19T19:06:17.219 INFO:tasks.workunit.client.0.smithi088.stdout:6/234: creat d5/f47 x:0 0 0 2024-02-19T19:06:17.221 INFO:tasks.workunit.client.0.smithi088.stdout:5/332: creat d4/d61/f73 x:0 0 0 2024-02-19T19:06:17.224 INFO:tasks.workunit.client.0.smithi088.stdout:3/296: rename d4/d1a/d1e/d31/l59 to d4/d1a/d1e/l6d 0 2024-02-19T19:06:17.225 INFO:tasks.workunit.client.0.smithi088.stdout:7/207: symlink d14/d1d/l4c 0 2024-02-19T19:06:17.236 INFO:tasks.workunit.client.0.smithi088.stdout:5/333: mknod d4/d7/c74 0 2024-02-19T19:06:17.236 INFO:tasks.workunit.client.0.smithi088.stdout:2/153: dwrite d3/f6 [0,4194304] 0 2024-02-19T19:06:17.239 INFO:tasks.workunit.client.0.smithi088.stdout:3/297: creat d4/d1a/d23/f6e x:0 0 0 2024-02-19T19:06:17.241 INFO:tasks.workunit.client.0.smithi088.stdout:9/223: rmdir d1/d21 39 2024-02-19T19:06:17.242 INFO:tasks.workunit.client.0.smithi088.stdout:5/334: dwrite d4/d1a/d22/f26 [0,4194304] 0 2024-02-19T19:06:17.243 INFO:tasks.workunit.client.0.smithi088.stdout:8/177: sync 2024-02-19T19:06:17.243 INFO:tasks.workunit.client.0.smithi088.stdout:4/181: sync 2024-02-19T19:06:17.244 INFO:tasks.workunit.client.0.smithi088.stdout:8/178: stat d9/df/d1c/f29 0 2024-02-19T19:06:17.257 INFO:tasks.workunit.client.0.smithi088.stdout:3/298: rename d4/c5f to d4/d22/d66/c6f 0 2024-02-19T19:06:17.259 INFO:tasks.workunit.client.0.smithi088.stdout:9/224: stat d1/d21/d2d/d23 0 2024-02-19T19:06:17.261 INFO:tasks.workunit.client.0.smithi088.stdout:9/225: truncate d1/d21/d2d/d18/f3b 514799 0 2024-02-19T19:06:17.264 INFO:tasks.workunit.client.0.smithi088.stdout:5/335: mkdir d4/d75 0 2024-02-19T19:06:17.264 INFO:tasks.workunit.client.0.smithi088.stdout:3/299: dwrite d4/d1a/d23/f69 [0,4194304] 0 2024-02-19T19:06:17.271 INFO:tasks.workunit.client.0.smithi088.stdout:2/154: mknod d3/d1f/c33 0 2024-02-19T19:06:17.273 INFO:tasks.workunit.client.0.smithi088.stdout:6/235: dwrite d5/d7/f15 [0,4194304] 0 2024-02-19T19:06:17.280 INFO:tasks.workunit.client.0.smithi088.stdout:9/226: unlink d1/d21/d2d/d23/d38/c48 0 2024-02-19T19:06:17.281 INFO:tasks.workunit.client.0.smithi088.stdout:7/208: truncate d14/d27/f32 111174 0 2024-02-19T19:06:17.284 INFO:tasks.workunit.client.0.smithi088.stdout:5/336: mknod d4/d1a/d22/d57/c76 0 2024-02-19T19:06:17.288 INFO:tasks.workunit.client.0.smithi088.stdout:8/179: mkdir d9/d31/d35 0 2024-02-19T19:06:17.290 INFO:tasks.workunit.client.0.smithi088.stdout:6/236: mknod d5/d7/c48 0 2024-02-19T19:06:17.292 INFO:tasks.workunit.client.0.smithi088.stdout:7/209: rmdir d14/d28/d3b 39 2024-02-19T19:06:17.294 INFO:tasks.workunit.client.0.smithi088.stdout:3/300: symlink d4/l70 0 2024-02-19T19:06:17.307 INFO:tasks.workunit.client.0.smithi088.stdout:9/227: unlink d1/d13/ce 0 2024-02-19T19:06:17.307 INFO:tasks.workunit.client.0.smithi088.stdout:9/228: chown d1/d8/d37/f44 61783643 1 2024-02-19T19:06:17.314 INFO:tasks.workunit.client.0.smithi088.stdout:3/301: unlink d4/d12/d33/d58/c5c 0 2024-02-19T19:06:17.315 INFO:tasks.workunit.client.0.smithi088.stdout:1/309: sync 2024-02-19T19:06:17.315 INFO:tasks.workunit.client.0.smithi088.stdout:0/189: sync 2024-02-19T19:06:17.318 INFO:tasks.workunit.client.0.smithi088.stdout:0/190: write d13/d24/d25/d2a/d34/f40 [459140,1279] 0 2024-02-19T19:06:17.319 INFO:tasks.workunit.client.0.smithi088.stdout:2/155: truncate d3/f6 2300196 0 2024-02-19T19:06:17.319 INFO:tasks.workunit.client.0.smithi088.stdout:7/210: dwrite d14/d28/d3b/f4a [0,4194304] 0 2024-02-19T19:06:17.320 INFO:tasks.workunit.client.0.smithi088.stdout:4/182: getdents d2/d35/d39 0 2024-02-19T19:06:17.320 INFO:tasks.workunit.client.0.smithi088.stdout:2/156: chown d3/d14 0 1 2024-02-19T19:06:17.322 INFO:tasks.workunit.client.0.smithi088.stdout:4/183: chown d2/d32/d25/d26/c31 131 1 2024-02-19T19:06:17.322 INFO:tasks.workunit.client.0.smithi088.stdout:2/157: fdatasync d3/d14/f2b 0 2024-02-19T19:06:17.328 INFO:tasks.workunit.client.0.smithi088.stdout:3/302: creat d4/d1a/d1e/d31/f71 x:0 0 0 2024-02-19T19:06:17.329 INFO:tasks.workunit.client.0.smithi088.stdout:3/303: dread - d4/d12/d33/d43/f65 zero size 2024-02-19T19:06:17.330 INFO:tasks.workunit.client.0.smithi088.stdout:5/337: creat d4/d1a/f77 x:0 0 0 2024-02-19T19:06:17.331 INFO:tasks.workunit.client.0.smithi088.stdout:5/338: stat d4/d1a/d41/d5e/f62 0 2024-02-19T19:06:17.332 INFO:tasks.workunit.client.0.smithi088.stdout:5/339: write d4/d1a/d41/d5e/f62 [71051,56920] 0 2024-02-19T19:06:17.333 INFO:tasks.workunit.client.0.smithi088.stdout:4/184: dread d2/d32/f23 [0,4194304] 0 2024-02-19T19:06:17.344 INFO:tasks.workunit.client.0.smithi088.stdout:9/229: link d1/f2c d1/d21/d2d/f4c 0 2024-02-19T19:06:17.348 INFO:tasks.workunit.client.0.smithi088.stdout:2/158: rename d3/df/l20 to d3/d1f/l34 0 2024-02-19T19:06:17.349 INFO:tasks.workunit.client.0.smithi088.stdout:3/304: write d4/f16 [627644,1259] 0 2024-02-19T19:06:17.351 INFO:tasks.workunit.client.0.smithi088.stdout:0/191: write d13/d24/d25/f27 [3471498,8327] 0 2024-02-19T19:06:17.362 INFO:tasks.workunit.client.0.smithi088.stdout:0/192: dwrite d13/d24/d25/f27 [4194304,4194304] 0 2024-02-19T19:06:17.363 INFO:tasks.workunit.client.0.smithi088.stdout:2/159: dread d3/df/f11 [0,4194304] 0 2024-02-19T19:06:17.364 INFO:tasks.workunit.client.0.smithi088.stdout:0/193: write d13/f3a [750328,96467] 0 2024-02-19T19:06:17.365 INFO:tasks.workunit.client.0.smithi088.stdout:0/194: truncate d13/f3e 495834 0 2024-02-19T19:06:17.369 INFO:tasks.workunit.client.0.smithi088.stdout:8/180: getdents d9/df/d1c 0 2024-02-19T19:06:17.370 INFO:tasks.workunit.client.0.smithi088.stdout:6/237: getdents d5/d19/d26 0 2024-02-19T19:06:17.375 INFO:tasks.workunit.client.0.smithi088.stdout:3/305: stat d4/d1a/d1e/d31/d32/c34 0 2024-02-19T19:06:17.392 INFO:tasks.workunit.client.0.smithi088.stdout:6/238: unlink d5/d19/f2f 0 2024-02-19T19:06:17.393 INFO:tasks.workunit.client.0.smithi088.stdout:6/239: write d5/f47 [7000,7555] 0 2024-02-19T19:06:17.394 INFO:tasks.workunit.client.0.smithi088.stdout:6/240: write d5/d7/f15 [346889,123428] 0 2024-02-19T19:06:17.396 INFO:tasks.workunit.client.0.smithi088.stdout:2/160: rmdir d3/df 39 2024-02-19T19:06:17.399 INFO:tasks.workunit.client.0.smithi088.stdout:6/241: dread d5/de/f18 [0,4194304] 0 2024-02-19T19:06:17.399 INFO:tasks.workunit.client.0.smithi088.stdout:6/242: write d5/d8/d3a/f46 [863515,54830] 0 2024-02-19T19:06:17.402 INFO:tasks.workunit.client.0.smithi088.stdout:7/211: rename l11 to d14/d1d/d3c/d43/l4d 0 2024-02-19T19:06:17.402 INFO:tasks.workunit.client.0.smithi088.stdout:0/195: write fe [4315241,53390] 0 2024-02-19T19:06:17.403 INFO:tasks.workunit.client.0.smithi088.stdout:3/306: fdatasync d4/d12/f2e 0 2024-02-19T19:06:17.424 INFO:tasks.workunit.client.0.smithi088.stdout:6/243: write d5/f3b [654616,9406] 0 2024-02-19T19:06:17.424 INFO:tasks.workunit.client.0.smithi088.stdout:7/212: chown d14/d28/l3f 258317634 1 2024-02-19T19:06:17.428 INFO:tasks.workunit.client.0.smithi088.stdout:3/307: mknod d4/d22/d66/c72 0 2024-02-19T19:06:17.429 INFO:tasks.workunit.client.0.smithi088.stdout:3/308: truncate d4/d1a/d23/f6e 296383 0 2024-02-19T19:06:17.430 INFO:tasks.workunit.client.0.smithi088.stdout:3/309: chown d4/d1a/d1e/d31/d32/d55/f57 29424162 1 2024-02-19T19:06:17.432 INFO:tasks.workunit.client.0.smithi088.stdout:5/340: getdents d4/d7 0 2024-02-19T19:06:17.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.411+0000 7fb39cd59700 1 -- 172.21.15.88:0/1261678973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3980ffd60 msgr2=0x7fb398103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:17.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.411+0000 7fb39cd59700 1 --2- 172.21.15.88:0/1261678973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3980ffd60 0x7fb398103d30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb38c009910 tx=0x7fb38c009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:17.434 INFO:tasks.workunit.client.0.smithi088.stdout:4/185: link d2/d32/l27 d2/d32/d9/l3e 0 2024-02-19T19:06:17.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.411+0000 7fb39cd59700 1 -- 172.21.15.88:0/1261678973 shutdown_connections 2024-02-19T19:06:17.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.411+0000 7fb39cd59700 1 --2- 172.21.15.88:0/1261678973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3980ffd60 0x7fb398103d30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.411+0000 7fb39cd59700 1 --2- 172.21.15.88:0/1261678973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3980ff3c0 0x7fb3980ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.411+0000 7fb39cd59700 1 -- 172.21.15.88:0/1261678973 >> 172.21.15.88:0/1261678973 conn(0x7fb3980fac60 msgr2=0x7fb3980fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.411+0000 7fb39cd59700 1 -- 172.21.15.88:0/1261678973 shutdown_connections 2024-02-19T19:06:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.411+0000 7fb39cd59700 1 -- 172.21.15.88:0/1261678973 wait complete. 2024-02-19T19:06:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.412+0000 7fb39cd59700 1 Processor -- start 2024-02-19T19:06:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.412+0000 7fb39cd59700 1 -- start start 2024-02-19T19:06:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.412+0000 7fb39cd59700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3980ff3c0 0x7fb398196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.412+0000 7fb39cd59700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3980ffd60 0x7fb3981972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:17.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.412+0000 7fb39cd59700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3981978c0 con 0x7fb3980ff3c0 2024-02-19T19:06:17.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.412+0000 7fb39cd59700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb398197a00 con 0x7fb3980ffd60 2024-02-19T19:06:17.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.412+0000 7fb39659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3980ff3c0 0x7fb398196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:17.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.412+0000 7fb39659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3980ff3c0 0x7fb398196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48332/0 (socket says 172.21.15.88:48332) 2024-02-19T19:06:17.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.412+0000 7fb39659c700 1 -- 172.21.15.88:0/3459867549 learned_addr learned my addr 172.21.15.88:0/3459867549 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:17.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.413+0000 7fb395d9b700 1 --2- 172.21.15.88:0/3459867549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3980ffd60 0x7fb3981972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:17.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.413+0000 7fb39659c700 1 -- 172.21.15.88:0/3459867549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3980ffd60 msgr2=0x7fb3981972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:17.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.413+0000 7fb39659c700 1 --2- 172.21.15.88:0/3459867549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3980ffd60 0x7fb3981972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:17.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.413+0000 7fb39659c700 1 -- 172.21.15.88:0/3459867549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c0095f0 con 0x7fb3980ff3c0 2024-02-19T19:06:17.436 INFO:tasks.workunit.client.0.smithi088.stdout:4/186: write f0 [3331266,55644] 0 2024-02-19T19:06:17.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.413+0000 7fb39659c700 1 --2- 172.21.15.88:0/3459867549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3980ff3c0 0x7fb398196d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d8d0 tx=0x7fb38800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:17.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.413+0000 7fb3877fe700 1 -- 172.21.15.88:0/3459867549 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb388009940 con 0x7fb3980ff3c0 2024-02-19T19:06:17.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.413+0000 7fb39cd59700 1 -- 172.21.15.88:0/3459867549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb39819fdd0 con 0x7fb3980ff3c0 2024-02-19T19:06:17.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.413+0000 7fb39cd59700 1 -- 172.21.15.88:0/3459867549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3981a0290 con 0x7fb3980ff3c0 2024-02-19T19:06:17.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.414+0000 7fb3877fe700 1 -- 172.21.15.88:0/3459867549 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb388010460 con 0x7fb3980ff3c0 2024-02-19T19:06:17.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.414+0000 7fb3877fe700 1 -- 172.21.15.88:0/3459867549 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb38800f5f0 con 0x7fb3980ff3c0 2024-02-19T19:06:17.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.415+0000 7fb39cd59700 1 -- 172.21.15.88:0/3459867549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb378005320 con 0x7fb3980ff3c0 2024-02-19T19:06:17.438 INFO:tasks.workunit.client.0.smithi088.stdout:8/181: creat d9/df/f36 x:0 0 0 2024-02-19T19:06:17.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.416+0000 7fb3877fe700 1 -- 172.21.15.88:0/3459867549 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fb38800f820 con 0x7fb3980ff3c0 2024-02-19T19:06:17.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.416+0000 7fb3877fe700 1 --2- 172.21.15.88:0/3459867549 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb38006da30 0x7fb38006fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:17.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.417+0000 7fb3877fe700 1 -- 172.21.15.88:0/3459867549 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb38808d710 con 0x7fb3980ff3c0 2024-02-19T19:06:17.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.417+0000 7fb395d9b700 1 --2- 172.21.15.88:0/3459867549 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb38006da30 0x7fb38006fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:17.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.418+0000 7fb395d9b700 1 --2- 172.21.15.88:0/3459867549 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb38006da30 0x7fb38006fee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb38c000c00 tx=0x7fb38c005020 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:17.443 INFO:tasks.workunit.client.0.smithi088.stdout:8/182: dwrite f6 [0,4194304] 0 2024-02-19T19:06:17.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.424+0000 7fb3877fe700 1 -- 172.21.15.88:0/3459867549 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb388056ef0 con 0x7fb3980ff3c0 2024-02-19T19:06:17.446 INFO:tasks.workunit.client.0.smithi088.stdout:9/230: getdents d1/d21/d2d/d23/d38 0 2024-02-19T19:06:17.451 INFO:tasks.workunit.client.0.smithi088.stdout:3/310: mknod d4/d1a/d1e/c73 0 2024-02-19T19:06:17.453 INFO:tasks.workunit.client.0.smithi088.stdout:5/341: creat d4/d61/f78 x:0 0 0 2024-02-19T19:06:17.454 INFO:tasks.workunit.client.0.smithi088.stdout:5/342: chown d4/d7/f2c 8041 1 2024-02-19T19:06:17.455 INFO:tasks.workunit.client.0.smithi088.stdout:7/213: dwrite d14/f17 [4194304,4194304] 0 2024-02-19T19:06:17.456 INFO:tasks.workunit.client.0.smithi088.stdout:5/343: chown d4/d1a/f5b 53262019 1 2024-02-19T19:06:17.457 INFO:tasks.workunit.client.0.smithi088.stdout:4/187: mkdir d2/d32/d25/d3f 0 2024-02-19T19:06:17.458 INFO:tasks.workunit.client.0.smithi088.stdout:4/188: truncate d2/f15 4783766 0 2024-02-19T19:06:17.459 INFO:tasks.workunit.client.0.smithi088.stdout:8/183: mknod d9/df/d1c/d22/c37 0 2024-02-19T19:06:17.465 INFO:tasks.workunit.client.0.smithi088.stdout:7/214: dwrite fb [4194304,4194304] 0 2024-02-19T19:06:17.467 INFO:tasks.workunit.client.0.smithi088.stdout:7/215: readlink d14/d1d/d3c/d43/l47 0 2024-02-19T19:06:17.469 INFO:tasks.workunit.client.0.smithi088.stdout:7/216: fdatasync d14/f29 0 2024-02-19T19:06:17.471 INFO:tasks.workunit.client.0.smithi088.stdout:2/161: getdents d3/d1b 0 2024-02-19T19:06:17.471 INFO:tasks.workunit.client.0.smithi088.stdout:9/231: rmdir d1 39 2024-02-19T19:06:17.472 INFO:tasks.workunit.client.0.smithi088.stdout:2/162: write d3/d1b/f27 [274384,57517] 0 2024-02-19T19:06:17.483 INFO:tasks.workunit.client.0.smithi088.stdout:8/184: dread d9/df/f19 [0,4194304] 0 2024-02-19T19:06:17.484 INFO:tasks.workunit.client.0.smithi088.stdout:7/217: dwrite d14/d1d/f38 [0,4194304] 0 2024-02-19T19:06:17.484 INFO:tasks.workunit.client.0.smithi088.stdout:9/232: mknod d1/d8/d3e/c4d 0 2024-02-19T19:06:17.485 INFO:tasks.workunit.client.0.smithi088.stdout:9/233: chown d1/d21/d2d/d23 223033387 1 2024-02-19T19:06:17.487 INFO:tasks.workunit.client.0.smithi088.stdout:9/234: dread - d1/d21/d2d/f4c zero size 2024-02-19T19:06:17.487 INFO:tasks.workunit.client.0.smithi088.stdout:9/235: fsync d1/f20 0 2024-02-19T19:06:17.488 INFO:tasks.workunit.client.0.smithi088.stdout:4/189: getdents d2/d35 0 2024-02-19T19:06:17.488 INFO:tasks.workunit.client.0.smithi088.stdout:9/236: chown d1/d8 1717367 1 2024-02-19T19:06:17.490 INFO:tasks.workunit.client.0.smithi088.stdout:4/190: chown d2/d35/f37 36 1 2024-02-19T19:06:17.490 INFO:tasks.workunit.client.0.smithi088.stdout:7/218: dread d14/f1f [4194304,4194304] 0 2024-02-19T19:06:17.493 INFO:tasks.workunit.client.0.smithi088.stdout:3/311: getdents d4 0 2024-02-19T19:06:17.496 INFO:tasks.workunit.client.0.smithi088.stdout:6/244: write d5/d19/d26/f27 [4717549,101131] 0 2024-02-19T19:06:17.499 INFO:tasks.workunit.client.0.smithi088.stdout:7/219: dwrite d14/f16 [0,4194304] 0 2024-02-19T19:06:17.500 INFO:tasks.workunit.client.0.smithi088.stdout:7/220: fdatasync d14/d27/f36 0 2024-02-19T19:06:17.501 INFO:tasks.workunit.client.0.smithi088.stdout:7/221: write d14/f44 [598284,48139] 0 2024-02-19T19:06:17.504 INFO:tasks.workunit.client.0.smithi088.stdout:0/196: truncate fe 644430 0 2024-02-19T19:06:17.504 INFO:tasks.workunit.client.0.smithi088.stdout:7/222: stat d14/d1d/d3c/d43/l47 0 2024-02-19T19:06:17.505 INFO:tasks.workunit.client.0.smithi088.stdout:0/197: write d13/d24/d25/d2a/d34/f35 [549066,2008] 0 2024-02-19T19:06:17.509 INFO:tasks.workunit.client.0.smithi088.stdout:9/237: fsync d1/d13/f33 0 2024-02-19T19:06:17.510 INFO:tasks.workunit.client.0.smithi088.stdout:9/238: fsync d1/d21/d2d/f47 0 2024-02-19T19:06:17.513 INFO:tasks.workunit.client.0.smithi088.stdout:3/312: mkdir d4/d1a/d1e/d74 0 2024-02-19T19:06:17.514 INFO:tasks.workunit.client.0.smithi088.stdout:3/313: write d4/f6a [619639,115579] 0 2024-02-19T19:06:17.515 INFO:tasks.workunit.client.0.smithi088.stdout:8/185: symlink d9/df/l38 0 2024-02-19T19:06:17.518 INFO:tasks.workunit.client.0.smithi088.stdout:9/239: dwrite d1/f6 [0,4194304] 0 2024-02-19T19:06:17.520 INFO:tasks.workunit.client.0.smithi088.stdout:8/186: truncate d9/df/d1c/f33 49111 0 2024-02-19T19:06:17.528 INFO:tasks.workunit.client.0.smithi088.stdout:4/191: getdents d2/d32/d25/d3f 0 2024-02-19T19:06:17.528 INFO:tasks.workunit.client.0.smithi088.stdout:4/192: fsync d2/d32/f36 0 2024-02-19T19:06:17.529 INFO:tasks.workunit.client.0.smithi088.stdout:4/193: write d2/f15 [5226330,111713] 0 2024-02-19T19:06:17.530 INFO:tasks.workunit.client.0.smithi088.stdout:4/194: stat d2/d35/d39 0 2024-02-19T19:06:17.532 INFO:tasks.workunit.client.0.smithi088.stdout:4/195: chown d2/d32/d9/c11 12115283 1 2024-02-19T19:06:17.533 INFO:tasks.workunit.client.0.smithi088.stdout:3/314: symlink d4/d1a/d1e/d31/d32/d55/l75 0 2024-02-19T19:06:17.537 INFO:tasks.workunit.client.0.smithi088.stdout:3/315: write d4/d1a/d1e/d31/d32/f4c [34314,93808] 0 2024-02-19T19:06:17.538 INFO:tasks.workunit.client.0.smithi088.stdout:7/223: mknod d14/d1d/c4e 0 2024-02-19T19:06:17.541 INFO:tasks.workunit.client.0.smithi088.stdout:6/245: dwrite d5/d8/d2c/f40 [0,4194304] 0 2024-02-19T19:06:17.541 INFO:tasks.workunit.client.0.smithi088.stdout:0/198: creat d13/d24/d25/d2a/d2f/f41 x:0 0 0 2024-02-19T19:06:17.542 INFO:tasks.workunit.client.0.smithi088.stdout:6/246: fdatasync d5/de/f10 0 2024-02-19T19:06:17.543 INFO:tasks.workunit.client.0.smithi088.stdout:3/316: chown d4/d12/f6c 1098632 1 2024-02-19T19:06:17.548 INFO:tasks.workunit.client.0.smithi088.stdout:4/196: unlink d2/f7 0 2024-02-19T19:06:17.549 INFO:tasks.workunit.client.0.smithi088.stdout:9/240: getdents d1/d21/d2d/d23/d38 0 2024-02-19T19:06:17.561 INFO:tasks.workunit.client.0.smithi088.stdout:6/247: readlink d5/d8/d2c/l35 0 2024-02-19T19:06:17.571 INFO:tasks.workunit.client.0.smithi088.stdout:4/197: write d2/d35/f37 [1849549,66779] 0 2024-02-19T19:06:17.580 INFO:tasks.workunit.client.0.smithi088.stdout:4/198: dread d2/d8/f14 [0,4194304] 0 2024-02-19T19:06:17.597 INFO:tasks.workunit.client.0.smithi088.stdout:9/241: fsync d1/d21/d2d/d18/f3b 0 2024-02-19T19:06:17.602 INFO:tasks.workunit.client.0.smithi088.stdout:4/199: dread d2/d32/f23 [0,4194304] 0 2024-02-19T19:06:17.603 INFO:tasks.workunit.client.0.smithi088.stdout:0/199: dwrite fe [0,4194304] 0 2024-02-19T19:06:17.607 INFO:tasks.workunit.client.0.smithi088.stdout:3/317: rename d4/l26 to d4/d12/l76 0 2024-02-19T19:06:17.608 INFO:tasks.workunit.client.0.smithi088.stdout:7/224: getdents d14/d27 0 2024-02-19T19:06:17.614 INFO:tasks.workunit.client.0.smithi088.stdout:4/200: dread d2/d35/f37 [0,4194304] 0 2024-02-19T19:06:17.625 INFO:tasks.workunit.client.0.smithi088.stdout:0/200: mknod d13/d24/c42 0 2024-02-19T19:06:17.626 INFO:tasks.workunit.client.0.smithi088.stdout:0/201: truncate d13/d24/d25/f2b 5241188 0 2024-02-19T19:06:17.628 INFO:tasks.workunit.client.0.smithi088.stdout:7/225: rmdir d14/d1d/d3c/d43 39 2024-02-19T19:06:17.629 INFO:tasks.workunit.client.0.smithi088.stdout:9/242: fsync d1/d21/d2d/f2a 0 2024-02-19T19:06:17.635 INFO:tasks.workunit.client.0.smithi088.stdout:9/243: dwrite d1/d13/f36 [0,4194304] 0 2024-02-19T19:06:17.644 INFO:tasks.workunit.client.0.smithi088.stdout:6/248: creat d5/d31/d25/f49 x:0 0 0 2024-02-19T19:06:17.646 INFO:tasks.workunit.client.0.smithi088.stdout:6/249: read d5/d19/d26/f44 [199255,85524] 0 2024-02-19T19:06:17.647 INFO:tasks.workunit.client.0.smithi088.stdout:6/250: chown d5/d7 121101418 1 2024-02-19T19:06:17.649 INFO:tasks.workunit.client.0.smithi088.stdout:9/244: rename d1/d21/d2d/d23/d38 to d1/d8/d3e/d4e 0 2024-02-19T19:06:17.653 INFO:tasks.workunit.client.0.smithi088.stdout:3/318: read d4/d12/f1d [881005,30845] 0 2024-02-19T19:06:17.655 INFO:tasks.workunit.client.0.smithi088.stdout:6/251: symlink d5/d7/d32/l4a 0 2024-02-19T19:06:17.659 INFO:tasks.workunit.client.0.smithi088.stdout:0/202: rmdir d13/d24/d25/d2a/d3d 0 2024-02-19T19:06:17.661 INFO:tasks.workunit.client.0.smithi088.stdout:9/245: read d1/d21/d2d/d23/f26 [898610,38345] 0 2024-02-19T19:06:17.663 INFO:tasks.workunit.client.0.smithi088.stdout:9/246: write d1/d13/f36 [953990,99094] 0 2024-02-19T19:06:17.663 INFO:tasks.workunit.client.0.smithi088.stdout:7/226: creat d14/f4f x:0 0 0 2024-02-19T19:06:17.665 INFO:tasks.workunit.client.0.smithi088.stdout:4/201: dwrite d2/d8/f1a [8388608,4194304] 0 2024-02-19T19:06:17.666 INFO:tasks.workunit.client.0.smithi088.stdout:4/202: fsync d2/f4 0 2024-02-19T19:06:17.668 INFO:tasks.workunit.client.0.smithi088.stdout:3/319: readlink d4/d1a/d1e/l37 0 2024-02-19T19:06:17.669 INFO:tasks.workunit.client.0.smithi088.stdout:3/320: readlink d4/d1a/d1e/l62 0 2024-02-19T19:06:17.669 INFO:tasks.workunit.client.0.smithi088.stdout:6/252: creat d5/d7/f4b x:0 0 0 2024-02-19T19:06:17.669 INFO:tasks.workunit.client.0.smithi088.stdout:0/203: creat d13/d24/d25/d2a/d34/f43 x:0 0 0 2024-02-19T19:06:17.672 INFO:tasks.workunit.client.0.smithi088.stdout:7/227: write d14/d1d/f34 [454033,30950] 0 2024-02-19T19:06:17.674 INFO:tasks.workunit.client.0.smithi088.stdout:4/203: truncate d2/d32/f23 1764846 0 2024-02-19T19:06:17.675 INFO:tasks.workunit.client.0.smithi088.stdout:4/204: chown d2/d32/c3b 3 1 2024-02-19T19:06:17.683 INFO:tasks.workunit.client.0.smithi088.stdout:9/247: mkdir d1/d8/d4a/d4f 0 2024-02-19T19:06:17.688 INFO:tasks.workunit.client.0.smithi088.stdout:4/205: rmdir d2/d32/d9 39 2024-02-19T19:06:17.688 INFO:tasks.workunit.client.0.smithi088.stdout:7/228: write d14/d1d/f39 [83341,79558] 0 2024-02-19T19:06:17.691 INFO:tasks.workunit.client.0.smithi088.stdout:0/204: dwrite d13/f29 [0,4194304] 0 2024-02-19T19:06:17.697 INFO:tasks.workunit.client.0.smithi088.stdout:6/253: dwrite d5/d19/d26/f44 [0,4194304] 0 2024-02-19T19:06:17.699 INFO:tasks.workunit.client.0.smithi088.stdout:6/254: write d5/d19/f41 [267860,119283] 0 2024-02-19T19:06:17.699 INFO:tasks.workunit.client.0.smithi088.stdout:7/229: symlink d14/d27/l50 0 2024-02-19T19:06:17.705 INFO:tasks.workunit.client.0.smithi088.stdout:4/206: dread d2/d32/f18 [0,4194304] 0 2024-02-19T19:06:17.708 INFO:tasks.workunit.client.0.smithi088.stdout:3/321: truncate d4/d22/f2c 1592206 0 2024-02-19T19:06:17.713 INFO:tasks.workunit.client.0.smithi088.stdout:3/322: chown d4/f7 9 1 2024-02-19T19:06:17.715 INFO:tasks.workunit.client.0.smithi088.stdout:0/205: creat d13/f44 x:0 0 0 2024-02-19T19:06:17.715 INFO:tasks.workunit.client.0.smithi088.stdout:3/323: dread d4/d12/f2e [0,4194304] 0 2024-02-19T19:06:17.716 INFO:tasks.workunit.client.0.smithi088.stdout:4/207: fdatasync d2/d8/f14 0 2024-02-19T19:06:17.717 INFO:tasks.workunit.client.0.smithi088.stdout:7/230: creat d14/f51 x:0 0 0 2024-02-19T19:06:17.718 INFO:tasks.workunit.client.0.smithi088.stdout:7/231: dread - d14/f19 zero size 2024-02-19T19:06:17.720 INFO:tasks.workunit.client.0.smithi088.stdout:6/255: rename d5/f3b to d5/d8/d2c/f4c 0 2024-02-19T19:06:17.721 INFO:tasks.workunit.client.0.smithi088.stdout:0/206: unlink l12 0 2024-02-19T19:06:17.726 INFO:tasks.workunit.client.0.smithi088.stdout:6/256: dwrite d5/d19/f41 [0,4194304] 0 2024-02-19T19:06:17.728 INFO:tasks.workunit.client.0.smithi088.stdout:4/208: rename d2/d32 to d2/d32/d9/d40 22 2024-02-19T19:06:17.731 INFO:tasks.workunit.client.0.smithi088.stdout:3/324: creat d4/d1a/d1e/d31/d32/f77 x:0 0 0 2024-02-19T19:06:17.733 INFO:tasks.workunit.client.0.smithi088.stdout:9/248: truncate d1/d21/d2d/f16 3948559 0 2024-02-19T19:06:17.733 INFO:tasks.workunit.client.0.smithi088.stdout:9/249: stat d1 0 2024-02-19T19:06:17.742 INFO:tasks.workunit.client.0.smithi088.stdout:0/207: symlink d13/d24/d25/d2a/d34/l45 0 2024-02-19T19:06:17.742 INFO:tasks.workunit.client.0.smithi088.stdout:0/208: fdatasync d13/f3a 0 2024-02-19T19:06:17.743 INFO:tasks.workunit.client.0.smithi088.stdout:1/310: sync 2024-02-19T19:06:17.745 INFO:tasks.workunit.client.0.smithi088.stdout:6/257: mknod d5/d7/d32/c4d 0 2024-02-19T19:06:17.745 INFO:tasks.workunit.client.0.smithi088.stdout:6/258: chown d5/de/f10 0 1 2024-02-19T19:06:17.749 INFO:tasks.workunit.client.0.smithi088.stdout:3/325: dread d4/f7 [0,4194304] 0 2024-02-19T19:06:17.750 INFO:tasks.workunit.client.0.smithi088.stdout:3/326: dread - d4/d1a/d23/f60 zero size 2024-02-19T19:06:17.755 INFO:tasks.workunit.client.0.smithi088.stdout:4/209: rename d2/d8/f1a to d2/d32/d25/f41 0 2024-02-19T19:06:17.757 INFO:tasks.workunit.client.0.smithi088.stdout:7/232: write d14/d27/f32 [795846,105871] 0 2024-02-19T19:06:17.759 INFO:tasks.workunit.client.0.smithi088.stdout:1/311: creat d6/d10/d60/f65 x:0 0 0 2024-02-19T19:06:17.759 INFO:tasks.workunit.client.0.smithi088.stdout:1/312: stat d6/l34 0 2024-02-19T19:06:17.763 INFO:tasks.workunit.client.0.smithi088.stdout:3/327: read d4/f38 [60925,113532] 0 2024-02-19T19:06:17.768 INFO:tasks.workunit.client.0.smithi088.stdout:3/328: dread d4/d1a/d23/f5d [0,4194304] 0 2024-02-19T19:06:17.769 INFO:tasks.workunit.client.0.smithi088.stdout:3/329: write d4/d1a/d1e/d31/d32/f4c [4326774,9084] 0 2024-02-19T19:06:17.771 INFO:tasks.workunit.client.0.smithi088.stdout:0/209: mkdir d13/d46 0 2024-02-19T19:06:17.776 INFO:tasks.workunit.client.0.smithi088.stdout:7/233: dread d14/d28/d3b/f40 [0,4194304] 0 2024-02-19T19:06:17.777 INFO:tasks.workunit.client.0.smithi088.stdout:1/313: dread d6/d10/d60/d29/f30 [0,4194304] 0 2024-02-19T19:06:17.777 INFO:tasks.workunit.client.0.smithi088.stdout:9/250: write d1/f10 [389268,83200] 0 2024-02-19T19:06:17.787 INFO:tasks.workunit.client.0.smithi088.stdout:4/210: truncate d2/d32/d25/f38 1063966 0 2024-02-19T19:06:17.788 INFO:tasks.workunit.client.0.smithi088.stdout:0/210: mknod d13/d24/d25/d2a/d34/c47 0 2024-02-19T19:06:17.788 INFO:tasks.workunit.client.0.smithi088.stdout:0/211: truncate d13/f33 861326 0 2024-02-19T19:06:17.790 INFO:tasks.workunit.client.0.smithi088.stdout:6/259: creat d5/d19/f4e x:0 0 0 2024-02-19T19:06:17.793 INFO:tasks.workunit.client.0.smithi088.stdout:7/234: creat d14/d28/d3b/f52 x:0 0 0 2024-02-19T19:06:17.794 INFO:tasks.workunit.client.0.smithi088.stdout:7/235: write d14/f44 [1656286,9943] 0 2024-02-19T19:06:17.796 INFO:tasks.workunit.client.0.smithi088.stdout:1/314: write d6/d10/d26/d4a/f56 [5438947,25964] 0 2024-02-19T19:06:17.797 INFO:tasks.workunit.client.0.smithi088.stdout:1/315: write d6/f52 [974303,11124] 0 2024-02-19T19:06:17.798 INFO:tasks.workunit.client.0.smithi088.stdout:9/251: symlink d1/d8/d3e/l50 0 2024-02-19T19:06:17.803 INFO:tasks.workunit.client.0.smithi088.stdout:8/187: sync 2024-02-19T19:06:17.803 INFO:tasks.workunit.client.0.smithi088.stdout:5/344: sync 2024-02-19T19:06:17.803 INFO:tasks.workunit.client.0.smithi088.stdout:2/163: sync 2024-02-19T19:06:17.803 INFO:tasks.workunit.client.0.smithi088.stdout:8/188: stat d9/df/d1c/f29 0 2024-02-19T19:06:17.804 INFO:tasks.workunit.client.0.smithi088.stdout:3/330: rename d4/fc to d4/d12/d33/d58/f78 0 2024-02-19T19:06:17.804 INFO:tasks.workunit.client.0.smithi088.stdout:8/189: stat d9/df/c2a 0 2024-02-19T19:06:17.805 INFO:tasks.workunit.client.0.smithi088.stdout:8/190: write d9/df/d1c/f16 [2847609,71772] 0 2024-02-19T19:06:17.806 INFO:tasks.workunit.client.0.smithi088.stdout:0/212: creat d13/d24/d25/d2a/f48 x:0 0 0 2024-02-19T19:06:17.806 INFO:tasks.workunit.client.0.smithi088.stdout:8/191: write d9/df/d1c/f29 [187845,65285] 0 2024-02-19T19:06:17.808 INFO:tasks.workunit.client.0.smithi088.stdout:8/192: write f6 [1654042,47861] 0 2024-02-19T19:06:17.809 INFO:tasks.workunit.client.0.smithi088.stdout:6/260: creat d5/d7/f4f x:0 0 0 2024-02-19T19:06:17.813 INFO:tasks.workunit.client.0.smithi088.stdout:8/193: dread f5 [0,4194304] 0 2024-02-19T19:06:17.813 INFO:tasks.workunit.client.0.smithi088.stdout:7/236: unlink d14/l1b 0 2024-02-19T19:06:17.814 INFO:tasks.workunit.client.0.smithi088.stdout:4/211: dwrite d2/d32/fb [0,4194304] 0 2024-02-19T19:06:17.814 INFO:tasks.workunit.client.0.smithi088.stdout:7/237: stat d14/f2e 0 2024-02-19T19:06:17.814 INFO:tasks.workunit.client.0.smithi088.stdout:8/194: readlink d9/df/d1c/l20 0 2024-02-19T19:06:17.815 INFO:tasks.workunit.client.0.smithi088.stdout:7/238: chown f13 33499913 1 2024-02-19T19:06:17.819 INFO:tasks.workunit.client.0.smithi088.stdout:1/316: fdatasync d6/ff 0 2024-02-19T19:06:17.823 INFO:tasks.workunit.client.0.smithi088.stdout:1/317: write d6/d10/d26/d4c/f40 [1482398,17812] 0 2024-02-19T19:06:17.824 INFO:tasks.workunit.client.0.smithi088.stdout:2/164: truncate d3/df/f17 175969 0 2024-02-19T19:06:17.824 INFO:tasks.workunit.client.0.smithi088.stdout:7/239: dwrite d14/d28/d3b/f52 [0,4194304] 0 2024-02-19T19:06:17.832 INFO:tasks.workunit.client.0.smithi088.stdout:1/318: dread d6/d10/f5d [0,4194304] 0 2024-02-19T19:06:17.837 INFO:tasks.workunit.client.0.smithi088.stdout:7/240: dwrite d14/d1d/f34 [0,4194304] 0 2024-02-19T19:06:17.840 INFO:tasks.workunit.client.0.smithi088.stdout:0/213: dwrite fb [0,4194304] 0 2024-02-19T19:06:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.818+0000 7fb39cd59700 1 -- 172.21.15.88:0/3459867549 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb378000bf0 con 0x7fb38006da30 2024-02-19T19:06:17.841 INFO:tasks.workunit.client.0.smithi088.stdout:0/214: stat c9 0 2024-02-19T19:06:17.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.820+0000 7fb3877fe700 1 -- 172.21.15.88:0/3459867549 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fb378000bf0 con 0x7fb38006da30 2024-02-19T19:06:17.843 INFO:tasks.workunit.client.0.smithi088.stdout:0/215: dread - d13/d24/d25/d2a/d2f/f41 zero size 2024-02-19T19:06:17.845 INFO:tasks.workunit.client.0.smithi088.stdout:0/216: chown d13/l22 547000518 1 2024-02-19T19:06:17.847 INFO:tasks.workunit.client.0.smithi088.stdout:9/252: creat d1/d8/d4a/d4f/f51 x:0 0 0 2024-02-19T19:06:17.847 INFO:tasks.workunit.client.0.smithi088.stdout:1/319: dwrite d6/fb [0,4194304] 0 2024-02-19T19:06:17.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.826+0000 7fb39cd59700 1 -- 172.21.15.88:0/3459867549 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb38006da30 msgr2=0x7fb38006fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:17.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.826+0000 7fb39cd59700 1 --2- 172.21.15.88:0/3459867549 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb38006da30 0x7fb38006fee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb38c000c00 tx=0x7fb38c005020 comp rx=0 tx=0).stop 2024-02-19T19:06:17.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.826+0000 7fb39cd59700 1 -- 172.21.15.88:0/3459867549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3980ff3c0 msgr2=0x7fb398196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:17.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.826+0000 7fb39cd59700 1 --2- 172.21.15.88:0/3459867549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3980ff3c0 0x7fb398196d60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb38800d8d0 tx=0x7fb38800dc90 comp rx=0 tx=0).stop 2024-02-19T19:06:17.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.826+0000 7fb39cd59700 1 -- 172.21.15.88:0/3459867549 shutdown_connections 2024-02-19T19:06:17.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.826+0000 7fb39cd59700 1 --2- 172.21.15.88:0/3459867549 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb38006da30 0x7fb38006fee0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:17.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.826+0000 7fb39cd59700 1 --2- 172.21.15.88:0/3459867549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3980ff3c0 0x7fb398196d60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:17.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.826+0000 7fb39cd59700 1 --2- 172.21.15.88:0/3459867549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3980ffd60 0x7fb3981972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:17.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.826+0000 7fb39cd59700 1 -- 172.21.15.88:0/3459867549 >> 172.21.15.88:0/3459867549 conn(0x7fb3980fac60 msgr2=0x7fb3980fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:17.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.827+0000 7fb39cd59700 1 -- 172.21.15.88:0/3459867549 shutdown_connections 2024-02-19T19:06:17.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:17.827+0000 7fb39cd59700 1 -- 172.21.15.88:0/3459867549 wait complete. 2024-02-19T19:06:17.854 INFO:tasks.workunit.client.0.smithi088.stdout:5/345: mknod d4/d75/c79 0 2024-02-19T19:06:17.855 INFO:tasks.workunit.client.0.smithi088.stdout:2/165: creat d3/d14/f35 x:0 0 0 2024-02-19T19:06:17.856 INFO:tasks.workunit.client.0.smithi088.stdout:2/166: rename d3 to d3/d1b/d36 22 2024-02-19T19:06:17.861 INFO:tasks.workunit.client.0.smithi088.stdout:6/261: mknod d5/d8/c50 0 2024-02-19T19:06:17.863 INFO:tasks.workunit.client.0.smithi088.stdout:8/195: creat d9/d31/d35/f39 x:0 0 0 2024-02-19T19:06:17.866 INFO:tasks.workunit.client.0.smithi088.stdout:1/320: creat d6/d10/d60/d29/d4e/f66 x:0 0 0 2024-02-19T19:06:17.867 INFO:tasks.workunit.client.0.smithi088.stdout:1/321: dread - d6/d10/f45 zero size 2024-02-19T19:06:17.868 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:06:17.870 INFO:tasks.workunit.client.0.smithi088.stdout:5/346: creat d4/d1a/d41/f7a x:0 0 0 2024-02-19T19:06:17.871 INFO:tasks.workunit.client.0.smithi088.stdout:5/347: write d4/d7/f5f [410523,18553] 0 2024-02-19T19:06:17.873 INFO:tasks.workunit.client.0.smithi088.stdout:2/167: fdatasync d3/df/f22 0 2024-02-19T19:06:17.873 INFO:tasks.workunit.client.0.smithi088.stdout:5/348: chown d4/d1a/d41/d5e/f62 16880 1 2024-02-19T19:06:17.874 INFO:tasks.workunit.client.0.smithi088.stdout:7/241: symlink d14/d28/l53 0 2024-02-19T19:06:17.878 INFO:tasks.workunit.client.0.smithi088.stdout:6/262: symlink d5/d7/l51 0 2024-02-19T19:06:17.879 INFO:tasks.workunit.client.0.smithi088.stdout:4/212: truncate d2/f4 715900 0 2024-02-19T19:06:17.880 INFO:tasks.workunit.client.0.smithi088.stdout:5/349: dwrite d4/fe [0,4194304] 0 2024-02-19T19:06:17.888 INFO:tasks.workunit.client.0.smithi088.stdout:5/350: dwrite d4/d1a/d22/f24 [0,4194304] 0 2024-02-19T19:06:17.894 INFO:tasks.workunit.client.0.smithi088.stdout:1/322: rename d6/d10/f2d to d6/d10/d60/f67 0 2024-02-19T19:06:17.900 INFO:tasks.workunit.client.0.smithi088.stdout:3/331: creat d4/f79 x:0 0 0 2024-02-19T19:06:17.903 INFO:tasks.workunit.client.0.smithi088.stdout:2/168: symlink d3/df/l37 0 2024-02-19T19:06:17.909 INFO:tasks.workunit.client.0.smithi088.stdout:0/217: creat d13/d24/d25/f49 x:0 0 0 2024-02-19T19:06:17.913 INFO:tasks.workunit.client.0.smithi088.stdout:6/263: rename d5/d31/c2d to d5/de/c52 0 2024-02-19T19:06:17.917 INFO:tasks.workunit.client.0.smithi088.stdout:2/169: rmdir d3/d14 39 2024-02-19T19:06:17.920 INFO:tasks.workunit.client.0.smithi088.stdout:9/253: write d1/d21/d2d/f16 [3965465,102586] 0 2024-02-19T19:06:17.925 INFO:tasks.workunit.client.0.smithi088.stdout:6/264: rename d5/d31/f37 to d5/d7/d32/f53 0 2024-02-19T19:06:17.925 INFO:tasks.workunit.client.0.smithi088.stdout:1/323: getdents d6/d10/d60/d29/d63 0 2024-02-19T19:06:17.931 INFO:tasks.workunit.client.0.smithi088.stdout:7/242: creat d14/f54 x:0 0 0 2024-02-19T19:06:17.932 INFO:tasks.workunit.client.0.smithi088.stdout:4/213: truncate d2/f15 248947 0 2024-02-19T19:06:17.934 INFO:tasks.workunit.client.0.smithi088.stdout:9/254: mkdir d1/d8/d52 0 2024-02-19T19:06:17.939 INFO:tasks.workunit.client.0.smithi088.stdout:1/324: rename d6/d10/d26/d4c/f40 to d6/d10/d26/f68 0 2024-02-19T19:06:17.941 INFO:tasks.workunit.client.0.smithi088.stdout:8/196: link d9/ld d9/l3a 0 2024-02-19T19:06:17.942 INFO:tasks.workunit.client.0.smithi088.stdout:9/255: dwrite d1/f6 [0,4194304] 0 2024-02-19T19:06:17.944 INFO:tasks.workunit.client.0.smithi088.stdout:2/170: mknod d3/d14/c38 0 2024-02-19T19:06:17.945 INFO:tasks.workunit.client.0.smithi088.stdout:2/171: fsync d3/d14/f2b 0 2024-02-19T19:06:17.946 INFO:tasks.workunit.client.0.smithi088.stdout:6/265: dwrite d5/ff [0,4194304] 0 2024-02-19T19:06:17.948 INFO:tasks.workunit.client.0.smithi088.stdout:7/243: symlink d14/d27/l55 0 2024-02-19T19:06:17.949 INFO:tasks.workunit.client.0.smithi088.stdout:0/218: creat d13/f4a x:0 0 0 2024-02-19T19:06:17.951 INFO:tasks.workunit.client.0.smithi088.stdout:4/214: creat d2/d35/d39/d3a/f42 x:0 0 0 2024-02-19T19:06:17.954 INFO:tasks.workunit.client.0.smithi088.stdout:5/351: link d4/d7/c58 d4/d1a/c7b 0 2024-02-19T19:06:17.959 INFO:tasks.workunit.client.0.smithi088.stdout:9/256: creat d1/d13/f53 x:0 0 0 2024-02-19T19:06:17.961 INFO:tasks.workunit.client.0.smithi088.stdout:7/244: fsync d14/d28/d3b/f40 0 2024-02-19T19:06:17.962 INFO:tasks.workunit.client.0.smithi088.stdout:0/219: creat d13/d24/f4b x:0 0 0 2024-02-19T19:06:17.966 INFO:tasks.workunit.client.0.smithi088.stdout:3/332: getdents d4/d1a 0 2024-02-19T19:06:17.970 INFO:tasks.workunit.client.0.smithi088.stdout:0/220: stat d13/d24/c36 0 2024-02-19T19:06:17.971 INFO:tasks.workunit.client.0.smithi088.stdout:8/197: dread d9/df/d1c/f1f [0,4194304] 0 2024-02-19T19:06:17.971 INFO:tasks.workunit.client.0.smithi088.stdout:8/198: chown d9/l21 1 1 2024-02-19T19:06:17.974 INFO:tasks.workunit.client.0.smithi088.stdout:2/172: creat d3/f39 x:0 0 0 2024-02-19T19:06:17.976 INFO:tasks.workunit.client.0.smithi088.stdout:3/333: fsync d4/d12/d33/f4e 0 2024-02-19T19:06:17.979 INFO:tasks.workunit.client.0.smithi088.stdout:7/245: mkdir d14/d56 0 2024-02-19T19:06:17.980 INFO:tasks.workunit.client.0.smithi088.stdout:9/257: dread d1/f2b [0,4194304] 0 2024-02-19T19:06:17.980 INFO:tasks.workunit.client.0.smithi088.stdout:9/258: dread - d1/d21/f43 zero size 2024-02-19T19:06:17.981 INFO:tasks.workunit.client.0.smithi088.stdout:4/215: fdatasync d2/d8/f29 0 2024-02-19T19:06:17.982 INFO:tasks.workunit.client.0.smithi088.stdout:8/199: mknod d9/df/d1c/d2d/c3b 0 2024-02-19T19:06:17.987 INFO:tasks.workunit.client.0.smithi088.stdout:7/246: dwrite d14/f22 [0,4194304] 0 2024-02-19T19:06:17.988 INFO:tasks.workunit.client.0.smithi088.stdout:7/247: rename d14 to d14/d56/d57 22 2024-02-19T19:06:17.989 INFO:tasks.workunit.client.0.smithi088.stdout:9/259: dwrite d1/d13/f36 [0,4194304] 0 2024-02-19T19:06:17.990 INFO:tasks.workunit.client.0.smithi088.stdout:9/260: chown d1/f20 162623044 1 2024-02-19T19:06:17.992 INFO:tasks.workunit.client.0.smithi088.stdout:4/216: unlink d2/c2c 0 2024-02-19T19:06:17.993 INFO:tasks.workunit.client.0.smithi088.stdout:4/217: write d2/d32/f1c [1176342,37845] 0 2024-02-19T19:06:17.995 INFO:tasks.workunit.client.0.smithi088.stdout:8/200: write d9/df/d1c/f33 [766175,96453] 0 2024-02-19T19:06:17.995 INFO:tasks.workunit.client.0.smithi088.stdout:8/201: chown l4 184 1 2024-02-19T19:06:17.997 INFO:tasks.workunit.client.0.smithi088.stdout:2/173: link d3/f31 d3/d1f/f3a 0 2024-02-19T19:06:18.000 INFO:tasks.workunit.client.0.smithi088.stdout:3/334: mkdir d4/d22/d7a 0 2024-02-19T19:06:18.005 INFO:tasks.workunit.client.0.smithi088.stdout:8/202: rmdir d9/d31/d35 39 2024-02-19T19:06:18.008 INFO:tasks.workunit.client.0.smithi088.stdout:9/261: rmdir d1/d21/d2d/d18 39 2024-02-19T19:06:18.009 INFO:tasks.workunit.client.0.smithi088.stdout:1/325: truncate f5 2417875 0 2024-02-19T19:06:18.010 INFO:tasks.workunit.client.0.smithi088.stdout:7/248: creat d14/d56/f58 x:0 0 0 2024-02-19T19:06:18.011 INFO:tasks.workunit.client.0.smithi088.stdout:1/326: truncate d6/d10/f24 1991266 0 2024-02-19T19:06:18.014 INFO:tasks.workunit.client.0.smithi088.stdout:8/203: symlink d9/df/d1c/d22/l3c 0 2024-02-19T19:06:18.015 INFO:tasks.workunit.client.0.smithi088.stdout:5/352: truncate d4/d1a/f1b 1880909 0 2024-02-19T19:06:18.016 INFO:tasks.workunit.client.0.smithi088.stdout:6/266: dwrite d5/f1e [0,4194304] 0 2024-02-19T19:06:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.000+0000 7f12b2587700 1 -- 172.21.15.88:0/2603422014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12ac102470 msgr2=0x7f12ac1028e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.000+0000 7f12b2587700 1 --2- 172.21.15.88:0/2603422014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12ac102470 0x7f12ac1028e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f12a0009910 tx=0x7f12a0009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.001+0000 7f12b2587700 1 -- 172.21.15.88:0/2603422014 shutdown_connections 2024-02-19T19:06:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.001+0000 7f12b2587700 1 --2- 172.21.15.88:0/2603422014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12ac102470 0x7f12ac1028e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.001+0000 7f12b2587700 1 --2- 172.21.15.88:0/2603422014 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12ac108470 0x7f12ac108840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.001+0000 7f12b2587700 1 -- 172.21.15.88:0/2603422014 >> 172.21.15.88:0/2603422014 conn(0x7f12ac0fdf90 msgr2=0x7f12ac1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.001+0000 7f12b2587700 1 -- 172.21.15.88:0/2603422014 shutdown_connections 2024-02-19T19:06:18.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.001+0000 7f12b2587700 1 -- 172.21.15.88:0/2603422014 wait complete. 2024-02-19T19:06:18.023 INFO:tasks.workunit.client.0.smithi088.stdout:7/249: mknod d14/d27/c59 0 2024-02-19T19:06:18.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.002+0000 7f12b2587700 1 Processor -- start 2024-02-19T19:06:18.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.002+0000 7f12b2587700 1 -- start start 2024-02-19T19:06:18.023 INFO:tasks.workunit.client.0.smithi088.stdout:8/204: dwrite d9/df/f27 [0,4194304] 0 2024-02-19T19:06:18.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.002+0000 7f12b2587700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12ac102470 0x7f12ac198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:18.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.002+0000 7f12b2587700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12ac108470 0x7f12ac1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.002+0000 7f12abfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12ac102470 0x7f12ac198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.002+0000 7f12abfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12ac102470 0x7f12ac198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48354/0 (socket says 172.21.15.88:48354) 2024-02-19T19:06:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.003+0000 7f12abfff700 1 -- 172.21.15.88:0/3025536136 learned_addr learned my addr 172.21.15.88:0/3025536136 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.003+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac198e90 con 0x7f12ac102470 2024-02-19T19:06:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.003+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ac19cc20 con 0x7f12ac108470 2024-02-19T19:06:18.025 INFO:tasks.workunit.client.0.smithi088.stdout:7/250: read d14/d1d/f34 [757753,70773] 0 2024-02-19T19:06:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.003+0000 7f12abfff700 1 -- 172.21.15.88:0/3025536136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12ac108470 msgr2=0x7f12ac1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:18.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.003+0000 7f12abfff700 1 --2- 172.21.15.88:0/3025536136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12ac108470 0x7f12ac1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:18.026 INFO:tasks.workunit.client.0.smithi088.stdout:8/205: write d9/df/f27 [4090104,92611] 0 2024-02-19T19:06:18.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.003+0000 7f12abfff700 1 -- 172.21.15.88:0/3025536136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a00095f0 con 0x7f12ac102470 2024-02-19T19:06:18.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.004+0000 7f12abfff700 1 --2- 172.21.15.88:0/3025536136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12ac102470 0x7f12ac198270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f129c00d8d0 tx=0x7f129c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:18.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.004+0000 7f12a97fa700 1 -- 172.21.15.88:0/3025536136 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f129c009940 con 0x7f12ac102470 2024-02-19T19:06:18.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.004+0000 7f12a97fa700 1 -- 172.21.15.88:0/3025536136 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f129c010460 con 0x7f12ac102470 2024-02-19T19:06:18.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.004+0000 7f12a97fa700 1 -- 172.21.15.88:0/3025536136 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f129c009940 con 0x7f12ac102470 2024-02-19T19:06:18.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.005+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ac19cf00 con 0x7f12ac102470 2024-02-19T19:06:18.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.005+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ac19d400 con 0x7f12ac102470 2024-02-19T19:06:18.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.006+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ac10a970 con 0x7f12ac102470 2024-02-19T19:06:18.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.007+0000 7f12a97fa700 1 -- 172.21.15.88:0/3025536136 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f129c00f3c0 con 0x7f12ac102470 2024-02-19T19:06:18.029 INFO:tasks.workunit.client.0.smithi088.stdout:1/327: symlink d6/d10/d26/d4a/l69 0 2024-02-19T19:06:18.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.008+0000 7f12a97fa700 1 --2- 172.21.15.88:0/3025536136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f129406da80 0x7f129406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:18.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.008+0000 7f12a97fa700 1 -- 172.21.15.88:0/3025536136 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f129c08d2f0 con 0x7f12ac102470 2024-02-19T19:06:18.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.008+0000 7f12ab7fe700 1 --2- 172.21.15.88:0/3025536136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f129406da80 0x7f129406ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:18.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.009+0000 7f12ab7fe700 1 --2- 172.21.15.88:0/3025536136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f129406da80 0x7f129406ff30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f12a000b5c0 tx=0x7f12a0005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:18.035 INFO:tasks.workunit.client.0.smithi088.stdout:2/174: rename d3/d1b/f27 to d3/f3b 0 2024-02-19T19:06:18.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.016+0000 7f12a97fa700 1 -- 172.21.15.88:0/3025536136 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f129c056a80 con 0x7f12ac102470 2024-02-19T19:06:18.043 INFO:tasks.workunit.client.0.smithi088.stdout:2/175: rmdir d3/d1b 39 2024-02-19T19:06:18.049 INFO:tasks.workunit.client.0.smithi088.stdout:3/335: rename d4/d1a/d23/l48 to d4/d1a/l7b 0 2024-02-19T19:06:18.049 INFO:tasks.workunit.client.0.smithi088.stdout:7/251: dread d14/f29 [0,4194304] 0 2024-02-19T19:06:18.049 INFO:tasks.workunit.client.0.smithi088.stdout:7/252: chown d14/d1d/f39 410803844 1 2024-02-19T19:06:18.052 INFO:tasks.workunit.client.0.smithi088.stdout:8/206: mkdir d9/df/d3d 0 2024-02-19T19:06:18.058 INFO:tasks.workunit.client.0.smithi088.stdout:3/336: read - d4/d1a/d1e/d31/d32/f77 zero size 2024-02-19T19:06:18.061 INFO:tasks.workunit.client.0.smithi088.stdout:7/253: truncate d14/f1f 1081022 0 2024-02-19T19:06:18.062 INFO:tasks.workunit.client.0.smithi088.stdout:9/262: write d1/d21/d2d/d18/d1f/f2e [3664494,69025] 0 2024-02-19T19:06:18.068 INFO:tasks.workunit.client.0.smithi088.stdout:7/254: dwrite d14/f3e [0,4194304] 0 2024-02-19T19:06:18.070 INFO:tasks.workunit.client.0.smithi088.stdout:3/337: write d4/f40 [2886681,25743] 0 2024-02-19T19:06:18.072 INFO:tasks.workunit.client.0.smithi088.stdout:9/263: unlink d1/f2c 0 2024-02-19T19:06:18.078 INFO:tasks.workunit.client.0.smithi088.stdout:5/353: rename d4/d7/f5f to d4/d1a/f7c 0 2024-02-19T19:06:18.079 INFO:tasks.workunit.client.0.smithi088.stdout:5/354: stat d4/d1a/d6d 0 2024-02-19T19:06:18.080 INFO:tasks.workunit.client.0.smithi088.stdout:5/355: truncate d4/d7/f15 9181812 0 2024-02-19T19:06:18.086 INFO:tasks.workunit.client.0.smithi088.stdout:3/338: chown d4/d1a/d23/f60 167 1 2024-02-19T19:06:18.092 INFO:tasks.workunit.client.0.smithi088.stdout:8/207: truncate d9/df/f27 2696520 0 2024-02-19T19:06:18.097 INFO:tasks.workunit.client.0.smithi088.stdout:8/208: dread f8 [0,4194304] 0 2024-02-19T19:06:18.097 INFO:tasks.workunit.client.0.smithi088.stdout:5/356: symlink d4/d1a/d22/d57/l7d 0 2024-02-19T19:06:18.099 INFO:tasks.workunit.client.0.smithi088.stdout:7/255: mknod d14/d27/d49/c5a 0 2024-02-19T19:06:18.105 INFO:tasks.workunit.client.0.smithi088.stdout:7/256: dwrite d14/d56/f58 [0,4194304] 0 2024-02-19T19:06:18.120 INFO:tasks.workunit.client.0.smithi088.stdout:1/328: rename d6/d10/d26 to d6/d21/d6a 0 2024-02-19T19:06:18.122 INFO:tasks.workunit.client.0.smithi088.stdout:9/264: truncate d1/d21/d2d/d23/f25 3156067 0 2024-02-19T19:06:18.131 INFO:tasks.workunit.client.0.smithi088.stdout:3/339: mkdir d4/d1a/d1e/d74/d7c 0 2024-02-19T19:06:18.150 INFO:tasks.workunit.client.0.smithi088.stdout:3/340: mkdir d4/d1a/d1e/d74/d7d 0 2024-02-19T19:06:18.153 INFO:tasks.workunit.client.0.smithi088.stdout:1/329: write d6/d21/d6a/d4c/f2c [378817,7299] 0 2024-02-19T19:06:18.161 INFO:tasks.workunit.client.0.smithi088.stdout:8/209: getdents d9/df/d1c 0 2024-02-19T19:06:18.161 INFO:tasks.workunit.client.0.smithi088.stdout:8/210: readlink d9/df/l38 0 2024-02-19T19:06:18.163 INFO:tasks.workunit.client.0.smithi088.stdout:2/176: rename d3/df/c25 to d3/c3c 0 2024-02-19T19:06:18.164 INFO:tasks.workunit.client.0.smithi088.stdout:1/330: mknod d6/d10/c6b 0 2024-02-19T19:06:18.165 INFO:tasks.workunit.client.0.smithi088.stdout:3/341: dwrite d4/d1a/d1e/f2f [4194304,4194304] 0 2024-02-19T19:06:18.167 INFO:tasks.workunit.client.0.smithi088.stdout:5/357: getdents d4/d75 0 2024-02-19T19:06:18.167 INFO:tasks.workunit.client.0.smithi088.stdout:5/358: chown d4/d1a/f3c 7364931 1 2024-02-19T19:06:18.170 INFO:tasks.workunit.client.0.smithi088.stdout:7/257: getdents d14/d1d/d3c 0 2024-02-19T19:06:18.172 INFO:tasks.workunit.client.0.smithi088.stdout:8/211: unlink d9/df/f19 0 2024-02-19T19:06:18.175 INFO:tasks.workunit.client.0.smithi088.stdout:2/177: dread - d3/d1b/f2e zero size 2024-02-19T19:06:18.177 INFO:tasks.workunit.client.0.smithi088.stdout:1/331: creat d6/d21/f6c x:0 0 0 2024-02-19T19:06:18.181 INFO:tasks.workunit.client.0.smithi088.stdout:5/359: mknod d4/d1a/d41/d5e/c7e 0 2024-02-19T19:06:18.185 INFO:tasks.workunit.client.0.smithi088.stdout:7/258: readlink d14/d28/l41 0 2024-02-19T19:06:18.192 INFO:tasks.workunit.client.0.smithi088.stdout:1/332: rename d6/d10/d60/f65 to d6/d21/d6a/f6d 0 2024-02-19T19:06:18.194 INFO:tasks.workunit.client.0.smithi088.stdout:3/342: mknod d4/c7e 0 2024-02-19T19:06:18.196 INFO:tasks.workunit.client.0.smithi088.stdout:8/212: dread d9/df/d1c/f16 [0,4194304] 0 2024-02-19T19:06:18.197 INFO:tasks.workunit.client.0.smithi088.stdout:8/213: write d9/df/f36 [68690,43004] 0 2024-02-19T19:06:18.208 INFO:tasks.workunit.client.0.smithi088.stdout:9/265: dwrite d1/d21/d2d/d23/f25 [0,4194304] 0 2024-02-19T19:06:18.208 INFO:tasks.workunit.client.0.smithi088.stdout:8/214: dread d9/df/f36 [0,4194304] 0 2024-02-19T19:06:18.210 INFO:tasks.workunit.client.0.smithi088.stdout:5/360: rename d4/d7/f4b to d4/d1a/d41/d5e/f7f 0 2024-02-19T19:06:18.215 INFO:tasks.workunit.client.0.smithi088.stdout:2/178: dwrite d3/df/f22 [0,4194304] 0 2024-02-19T19:06:18.215 INFO:tasks.workunit.client.0.smithi088.stdout:2/179: stat d3/c2a 0 2024-02-19T19:06:18.216 INFO:tasks.workunit.client.0.smithi088.stdout:2/180: dread - d3/f39 zero size 2024-02-19T19:06:18.218 INFO:tasks.workunit.client.0.smithi088.stdout:9/266: mkdir d1/d8/d37/d54 0 2024-02-19T19:06:18.223 INFO:tasks.workunit.client.0.smithi088.stdout:3/343: mkdir d4/d1a/d7f 0 2024-02-19T19:06:18.229 INFO:tasks.workunit.client.0.smithi088.stdout:7/259: truncate fd 1643369 0 2024-02-19T19:06:18.230 INFO:tasks.workunit.client.0.smithi088.stdout:8/215: rename d9/d31/d35/f39 to d9/df/d1c/d2d/f3e 0 2024-02-19T19:06:18.232 INFO:tasks.workunit.client.0.smithi088.stdout:8/216: chown d9/df/d1c/d2d/l2f 1352145 1 2024-02-19T19:06:18.232 INFO:tasks.workunit.client.0.smithi088.stdout:2/181: dwrite d3/f3b [0,4194304] 0 2024-02-19T19:06:18.234 INFO:tasks.workunit.client.0.smithi088.stdout:9/267: creat d1/d8/d37/d54/f55 x:0 0 0 2024-02-19T19:06:18.238 INFO:tasks.workunit.client.0.smithi088.stdout:1/333: truncate d6/d21/d6a/d4a/f56 798449 0 2024-02-19T19:06:18.240 INFO:tasks.workunit.client.0.smithi088.stdout:3/344: creat d4/d1a/d1e/d74/d7c/f80 x:0 0 0 2024-02-19T19:06:18.240 INFO:tasks.workunit.client.0.smithi088.stdout:1/334: read - d6/d10/f4d zero size 2024-02-19T19:06:18.242 INFO:tasks.workunit.client.0.smithi088.stdout:2/182: read d3/f23 [10965,123674] 0 2024-02-19T19:06:18.243 INFO:tasks.workunit.client.0.smithi088.stdout:5/361: dwrite d4/f4d [0,4194304] 0 2024-02-19T19:06:18.248 INFO:tasks.workunit.client.0.smithi088.stdout:8/217: write d9/df/d1c/f25 [452767,106430] 0 2024-02-19T19:06:18.252 INFO:tasks.workunit.client.0.smithi088.stdout:1/335: dwrite d6/f1b [0,4194304] 0 2024-02-19T19:06:18.260 INFO:tasks.workunit.client.0.smithi088.stdout:2/183: symlink d3/df/l3d 0 2024-02-19T19:06:18.262 INFO:tasks.workunit.client.0.smithi088.stdout:5/362: creat d4/d75/f80 x:0 0 0 2024-02-19T19:06:18.265 INFO:tasks.workunit.client.0.smithi088.stdout:8/218: mknod d9/df/d1c/c3f 0 2024-02-19T19:06:18.268 INFO:tasks.workunit.client.0.smithi088.stdout:3/345: dwrite d4/d1a/d23/f5d [4194304,4194304] 0 2024-02-19T19:06:18.269 INFO:tasks.workunit.client.0.smithi088.stdout:7/260: write fd [2501136,51795] 0 2024-02-19T19:06:18.270 INFO:tasks.workunit.client.0.smithi088.stdout:1/336: mkdir d6/d10/d60/d29/d4e/d6e 0 2024-02-19T19:06:18.271 INFO:tasks.workunit.client.0.smithi088.stdout:2/184: dwrite d3/d1b/f1c [0,4194304] 0 2024-02-19T19:06:18.273 INFO:tasks.workunit.client.0.smithi088.stdout:2/185: stat d3/f31 0 2024-02-19T19:06:18.276 INFO:tasks.workunit.client.0.smithi088.stdout:9/268: rmdir d1/d21/d2d/d18 39 2024-02-19T19:06:18.280 INFO:tasks.workunit.client.0.smithi088.stdout:1/337: symlink d6/d21/l6f 0 2024-02-19T19:06:18.284 INFO:tasks.workunit.client.0.smithi088.stdout:5/363: creat d4/d61/d65/f81 x:0 0 0 2024-02-19T19:06:18.285 INFO:tasks.workunit.client.0.smithi088.stdout:5/364: dread - d4/d7/f64 zero size 2024-02-19T19:06:18.289 INFO:tasks.workunit.client.0.smithi088.stdout:5/365: dread d4/f4d [0,4194304] 0 2024-02-19T19:06:18.289 INFO:tasks.workunit.client.0.smithi088.stdout:5/366: dread - d4/d1a/f5b zero size 2024-02-19T19:06:18.290 INFO:tasks.workunit.client.0.smithi088.stdout:1/338: fdatasync d6/d10/d60/f67 0 2024-02-19T19:06:18.291 INFO:tasks.workunit.client.0.smithi088.stdout:5/367: readlink d4/d1a/d22/l14 0 2024-02-19T19:06:18.291 INFO:tasks.workunit.client.0.smithi088.stdout:5/368: readlink d4/d1a/l50 0 2024-02-19T19:06:18.302 INFO:tasks.workunit.client.0.smithi088.stdout:8/219: dread d9/df/d1c/f25 [4194304,4194304] 0 2024-02-19T19:06:18.302 INFO:tasks.workunit.client.0.smithi088.stdout:3/346: truncate d4/d1a/d23/f69 3761592 0 2024-02-19T19:06:18.303 INFO:tasks.workunit.client.0.smithi088.stdout:8/220: dread - d9/df/d1c/d2d/f3e zero size 2024-02-19T19:06:18.303 INFO:tasks.workunit.client.0.smithi088.stdout:3/347: stat d4/d1a/d23/f5d 0 2024-02-19T19:06:18.304 INFO:tasks.workunit.client.0.smithi088.stdout:1/339: rename d6/d21/d6a/d4c/f2c to d6/d21/f70 0 2024-02-19T19:06:18.305 INFO:tasks.workunit.client.0.smithi088.stdout:1/340: fdatasync d6/d10/d60/f67 0 2024-02-19T19:06:18.307 INFO:tasks.workunit.client.0.smithi088.stdout:7/261: link d14/f1c d14/d1d/d3c/d43/f5b 0 2024-02-19T19:06:18.310 INFO:tasks.workunit.client.0.smithi088.stdout:9/269: rmdir d1/d8/d52 0 2024-02-19T19:06:18.317 INFO:tasks.workunit.client.0.smithi088.stdout:3/348: dread d4/d1a/d1e/d31/d32/f50 [0,4194304] 0 2024-02-19T19:06:18.321 INFO:tasks.workunit.client.0.smithi088.stdout:7/262: mkdir d14/d1d/d3c/d43/d5c 0 2024-02-19T19:06:18.322 INFO:tasks.workunit.client.0.smithi088.stdout:1/341: dread d6/ff [0,4194304] 0 2024-02-19T19:06:18.324 INFO:tasks.workunit.client.0.smithi088.stdout:9/270: unlink d1/c1e 0 2024-02-19T19:06:18.325 INFO:tasks.workunit.client.0.smithi088.stdout:7/263: dread d14/d28/d3b/f52 [0,4194304] 0 2024-02-19T19:06:18.327 INFO:tasks.workunit.client.0.smithi088.stdout:5/369: rmdir d4/d1a/d6d 0 2024-02-19T19:06:18.330 INFO:tasks.workunit.client.0.smithi088.stdout:3/349: write d4/f14 [4364582,18193] 0 2024-02-19T19:06:18.335 INFO:tasks.workunit.client.0.smithi088.stdout:1/342: rename d6/d21/d6a/d4c to d6/d10/d60/d29/d4e/d71 0 2024-02-19T19:06:18.346 INFO:tasks.workunit.client.0.smithi088.stdout:3/350: unlink d4/d1a/d1e/d31/d32/d55/l75 0 2024-02-19T19:06:18.346 INFO:tasks.workunit.client.0.smithi088.stdout:8/221: dwrite d9/df/f27 [0,4194304] 0 2024-02-19T19:06:18.347 INFO:tasks.workunit.client.0.smithi088.stdout:9/271: readlink d1/d21/d2d/d23/l30 0 2024-02-19T19:06:18.348 INFO:tasks.workunit.client.0.smithi088.stdout:3/351: read - d4/d12/d33/d43/f65 zero size 2024-02-19T19:06:18.350 INFO:tasks.workunit.client.0.smithi088.stdout:7/264: mknod d14/d28/c5d 0 2024-02-19T19:06:18.353 INFO:tasks.workunit.client.0.smithi088.stdout:5/370: dwrite d4/f5 [12582912,4194304] 0 2024-02-19T19:06:18.355 INFO:tasks.workunit.client.0.smithi088.stdout:5/371: dread - d4/d61/f78 zero size 2024-02-19T19:06:18.357 INFO:tasks.workunit.client.0.smithi088.stdout:1/343: dread d6/fd [0,4194304] 0 2024-02-19T19:06:18.358 INFO:tasks.workunit.client.0.smithi088.stdout:3/352: mkdir d4/d22/d81 0 2024-02-19T19:06:18.361 INFO:tasks.workunit.client.0.smithi088.stdout:5/372: creat d4/d1a/d41/d5e/f82 x:0 0 0 2024-02-19T19:06:18.367 INFO:tasks.workunit.client.0.smithi088.stdout:1/344: creat d6/d21/f72 x:0 0 0 2024-02-19T19:06:18.367 INFO:tasks.workunit.client.0.smithi088.stdout:9/272: dwrite d1/d21/d2d/f16 [0,4194304] 0 2024-02-19T19:06:18.367 INFO:tasks.workunit.client.0.smithi088.stdout:8/222: rmdir d9/d31/d35 0 2024-02-19T19:06:18.368 INFO:tasks.workunit.client.0.smithi088.stdout:5/373: chown d4/d7/f64 45071065 1 2024-02-19T19:06:18.376 INFO:tasks.workunit.client.0.smithi088.stdout:3/353: mkdir d4/d22/d7a/d82 0 2024-02-19T19:06:18.383 INFO:tasks.workunit.client.0.smithi088.stdout:7/265: creat d14/d1d/f5e x:0 0 0 2024-02-19T19:06:18.384 INFO:tasks.workunit.client.0.smithi088.stdout:7/266: fdatasync d14/d1d/f5e 0 2024-02-19T19:06:18.384 INFO:tasks.workunit.client.0.smithi088.stdout:8/223: dread d9/f2e [0,4194304] 0 2024-02-19T19:06:18.385 INFO:tasks.workunit.client.0.smithi088.stdout:1/345: dwrite d6/fd [4194304,4194304] 0 2024-02-19T19:06:18.387 INFO:tasks.workunit.client.0.smithi088.stdout:5/374: unlink d4/f4d 0 2024-02-19T19:06:18.388 INFO:tasks.workunit.client.0.smithi088.stdout:3/354: creat d4/d1a/d1e/d74/f83 x:0 0 0 2024-02-19T19:06:18.401 INFO:tasks.workunit.client.0.smithi088.stdout:5/375: symlink d4/d1a/d41/d5e/l83 0 2024-02-19T19:06:18.402 INFO:tasks.workunit.client.0.smithi088.stdout:5/376: chown d4/d61/f73 3802 1 2024-02-19T19:06:18.402 INFO:tasks.workunit.client.0.smithi088.stdout:5/377: dread - d4/d7/f64 zero size 2024-02-19T19:06:18.405 INFO:tasks.workunit.client.0.smithi088.stdout:3/355: rename d4/d1a/d23/f6e to d4/d1a/d1e/d31/f84 0 2024-02-19T19:06:18.408 INFO:tasks.workunit.client.0.smithi088.stdout:1/346: truncate d6/d21/d6a/d4a/f56 1382809 0 2024-02-19T19:06:18.409 INFO:tasks.workunit.client.0.smithi088.stdout:1/347: write d6/d10/f4d [803751,79585] 0 2024-02-19T19:06:18.411 INFO:tasks.workunit.client.0.smithi088.stdout:5/378: unlink d4/d1a/d22/f28 0 2024-02-19T19:06:18.411 INFO:tasks.workunit.client.0.smithi088.stdout:0/221: sync 2024-02-19T19:06:18.412 INFO:tasks.workunit.client.0.smithi088.stdout:0/222: write d13/d24/f4b [237337,72940] 0 2024-02-19T19:06:18.414 INFO:tasks.workunit.client.0.smithi088.stdout:9/273: getdents d1/d8/d37 0 2024-02-19T19:06:18.416 INFO:tasks.workunit.client.0.smithi088.stdout:9/274: read - d1/d13/f53 zero size 2024-02-19T19:06:18.418 INFO:tasks.workunit.client.0.smithi088.stdout:3/356: mkdir d4/d12/d33/d43/d85 0 2024-02-19T19:06:18.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.399+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f12ac199670 con 0x7f129406da80 2024-02-19T19:06:18.421 INFO:tasks.workunit.client.0.smithi088.stdout:0/223: dwrite d13/d24/d25/d2a/f48 [0,4194304] 0 2024-02-19T19:06:18.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.401+0000 7f12a97fa700 1 -- 172.21.15.88:0/3025536136 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f12ac199670 con 0x7f129406da80 2024-02-19T19:06:18.423 INFO:tasks.workunit.client.0.smithi088.stdout:7/267: link d14/d27/c24 d14/d27/d49/c5f 0 2024-02-19T19:06:18.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.404+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f129406da80 msgr2=0x7f129406ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:18.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.404+0000 7f12b2587700 1 --2- 172.21.15.88:0/3025536136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f129406da80 0x7f129406ff30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f12a000b5c0 tx=0x7f12a0005fd0 comp rx=0 tx=0).stop 2024-02-19T19:06:18.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.404+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12ac102470 msgr2=0x7f12ac198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:18.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.404+0000 7f12b2587700 1 --2- 172.21.15.88:0/3025536136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12ac102470 0x7f12ac198270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f129c00d8d0 tx=0x7f129c00dc90 comp rx=0 tx=0).stop 2024-02-19T19:06:18.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.404+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 shutdown_connections 2024-02-19T19:06:18.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.404+0000 7f12b2587700 1 --2- 172.21.15.88:0/3025536136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f129406da80 0x7f129406ff30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:18.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.404+0000 7f12b2587700 1 --2- 172.21.15.88:0/3025536136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12ac102470 0x7f12ac198270 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:18.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.404+0000 7f12b2587700 1 --2- 172.21.15.88:0/3025536136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12ac108470 0x7f12ac1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:18.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.404+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 >> 172.21.15.88:0/3025536136 conn(0x7f12ac0fdf90 msgr2=0x7f12ac0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:18.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.404+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 shutdown_connections 2024-02-19T19:06:18.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.404+0000 7f12b2587700 1 -- 172.21.15.88:0/3025536136 wait complete. 2024-02-19T19:06:18.427 INFO:tasks.workunit.client.0.smithi088.stdout:1/348: creat d6/d10/f73 x:0 0 0 2024-02-19T19:06:18.429 INFO:tasks.workunit.client.0.smithi088.stdout:0/224: dwrite d13/d24/d25/f49 [0,4194304] 0 2024-02-19T19:06:18.429 INFO:tasks.workunit.client.0.smithi088.stdout:1/349: write d6/d21/d6a/d4a/f59 [1013347,6075] 0 2024-02-19T19:06:18.433 INFO:tasks.workunit.client.0.smithi088.stdout:5/379: chown d4/d1a/c7b 4 1 2024-02-19T19:06:18.436 INFO:tasks.workunit.client.0.smithi088.stdout:1/350: dread d6/fb [0,4194304] 0 2024-02-19T19:06:18.437 INFO:tasks.workunit.client.0.smithi088.stdout:1/351: readlink d6/d10/d60/l32 0 2024-02-19T19:06:18.448 INFO:tasks.workunit.client.0.smithi088.stdout:7/268: readlink d14/d1d/d3c/d43/l4d 0 2024-02-19T19:06:18.452 INFO:tasks.workunit.client.0.smithi088.stdout:8/224: getdents d9 0 2024-02-19T19:06:18.454 INFO:tasks.workunit.client.0.smithi088.stdout:4/218: sync 2024-02-19T19:06:18.455 INFO:tasks.workunit.client.0.smithi088.stdout:2/186: sync 2024-02-19T19:06:18.455 INFO:tasks.workunit.client.0.smithi088.stdout:6/267: sync 2024-02-19T19:06:18.456 INFO:tasks.workunit.client.0.smithi088.stdout:9/275: rmdir d1 39 2024-02-19T19:06:18.463 INFO:tasks.workunit.client.0.smithi088.stdout:0/225: creat d13/d24/d25/d2a/d2f/f4c x:0 0 0 2024-02-19T19:06:18.468 INFO:tasks.workunit.client.0.smithi088.stdout:3/357: mkdir d4/d1a/d7f/d86 0 2024-02-19T19:06:18.472 INFO:tasks.workunit.client.0.smithi088.stdout:1/352: symlink d6/d10/l74 0 2024-02-19T19:06:18.473 INFO:tasks.workunit.client.0.smithi088.stdout:7/269: dread - f8 zero size 2024-02-19T19:06:18.479 INFO:tasks.workunit.client.0.smithi088.stdout:2/187: creat d3/d1b/f3e x:0 0 0 2024-02-19T19:06:18.479 INFO:tasks.workunit.client.0.smithi088.stdout:2/188: chown d3/d14/f35 568 1 2024-02-19T19:06:18.487 INFO:tasks.workunit.client.0.smithi088.stdout:0/226: creat d13/d24/d25/d2a/d34/f4d x:0 0 0 2024-02-19T19:06:18.488 INFO:tasks.workunit.client.0.smithi088.stdout:0/227: stat d13/l2d 0 2024-02-19T19:06:18.491 INFO:tasks.workunit.client.0.smithi088.stdout:1/353: creat d6/d10/d60/d29/f75 x:0 0 0 2024-02-19T19:06:18.496 INFO:tasks.workunit.client.0.smithi088.stdout:9/276: mknod d1/d21/d2d/d23/c56 0 2024-02-19T19:06:18.497 INFO:tasks.workunit.client.0.smithi088.stdout:9/277: chown d1/d21/d2d/d23/c56 28852195 1 2024-02-19T19:06:18.499 INFO:tasks.workunit.client.0.smithi088.stdout:3/358: fsync d4/d22/f2c 0 2024-02-19T19:06:18.505 INFO:tasks.workunit.client.0.smithi088.stdout:5/380: getdents d4/d1a/d41 0 2024-02-19T19:06:18.506 INFO:tasks.workunit.client.0.smithi088.stdout:5/381: write d4/f9 [5163385,87534] 0 2024-02-19T19:06:18.509 INFO:tasks.workunit.client.0.smithi088.stdout:7/270: rename d14/d1d/f34 to d14/f60 0 2024-02-19T19:06:18.511 INFO:tasks.workunit.client.0.smithi088.stdout:7/271: read d14/f30 [319220,86433] 0 2024-02-19T19:06:18.512 INFO:tasks.workunit.client.0.smithi088.stdout:3/359: unlink d4/d1a/d1e/d31/d32/l3d 0 2024-02-19T19:06:18.513 INFO:tasks.workunit.client.0.smithi088.stdout:9/278: rmdir d1/d21 39 2024-02-19T19:06:18.515 INFO:tasks.workunit.client.0.smithi088.stdout:4/219: link d2/d32/c2b d2/c43 0 2024-02-19T19:06:18.519 INFO:tasks.workunit.client.0.smithi088.stdout:5/382: mkdir d4/d1a/d22/d57/d84 0 2024-02-19T19:06:18.519 INFO:tasks.workunit.client.0.smithi088.stdout:4/220: dread d2/d32/d9/f20 [0,4194304] 0 2024-02-19T19:06:18.521 INFO:tasks.workunit.client.0.smithi088.stdout:2/189: rename d3/f4 to d3/d1f/f3f 0 2024-02-19T19:06:18.526 INFO:tasks.workunit.client.0.smithi088.stdout:1/354: link d6/d10/d60/d29/d4e/d71/l5a d6/d10/d60/l76 0 2024-02-19T19:06:18.526 INFO:tasks.workunit.client.0.smithi088.stdout:5/383: dwrite d4/d1a/d22/f4c [0,4194304] 0 2024-02-19T19:06:18.527 INFO:tasks.workunit.client.0.smithi088.stdout:5/384: truncate d4/d1a/d22/d57/f6a 4524324 0 2024-02-19T19:06:18.528 INFO:tasks.workunit.client.0.smithi088.stdout:5/385: write d4/d1a/d22/f4c [384898,50079] 0 2024-02-19T19:06:18.533 INFO:tasks.workunit.client.0.smithi088.stdout:7/272: symlink d14/l61 0 2024-02-19T19:06:18.533 INFO:tasks.workunit.client.0.smithi088.stdout:7/273: dread - d14/d28/d3b/f4b zero size 2024-02-19T19:06:18.534 INFO:tasks.workunit.client.0.smithi088.stdout:9/279: chown d1/d21/d2d/f47 48324 1 2024-02-19T19:06:18.543 INFO:tasks.workunit.client.0.smithi088.stdout:9/280: truncate d1/d13/f33 8891 0 2024-02-19T19:06:18.544 INFO:tasks.workunit.client.0.smithi088.stdout:9/281: read - d1/d8/d3e/f45 zero size 2024-02-19T19:06:18.545 INFO:tasks.workunit.client.0.smithi088.stdout:5/386: dwrite d4/d1a/d41/d5e/f7f [0,4194304] 0 2024-02-19T19:06:18.546 INFO:tasks.workunit.client.0.smithi088.stdout:2/190: rename d3/df/f22 to d3/f40 0 2024-02-19T19:06:18.547 INFO:tasks.workunit.client.0.smithi088.stdout:5/387: write d4/d1a/d22/f3d [4323645,103533] 0 2024-02-19T19:06:18.548 INFO:tasks.workunit.client.0.smithi088.stdout:5/388: stat d4/d1a/d22/c68 0 2024-02-19T19:06:18.549 INFO:tasks.workunit.client.0.smithi088.stdout:5/389: write d4/d75/f80 [727733,95504] 0 2024-02-19T19:06:18.550 INFO:tasks.workunit.client.0.smithi088.stdout:5/390: chown d4/d1a/f54 96148 1 2024-02-19T19:06:18.556 INFO:tasks.workunit.client.0.smithi088.stdout:2/191: mknod d3/d1b/c41 0 2024-02-19T19:06:18.556 INFO:tasks.workunit.client.0.smithi088.stdout:2/192: stat d3/d14/c1a 0 2024-02-19T19:06:18.557 INFO:tasks.workunit.client.0.smithi088.stdout:7/274: getdents d14/d1d/d3c 0 2024-02-19T19:06:18.559 INFO:tasks.workunit.client.0.smithi088.stdout:5/391: rename d4/d1a/l25 to d4/d7/l85 0 2024-02-19T19:06:18.562 INFO:tasks.workunit.client.0.smithi088.stdout:5/392: read d4/d1a/d22/f26 [2466260,25118] 0 2024-02-19T19:06:18.570 INFO:tasks.workunit.client.0.smithi088.stdout:5/393: rename d4/d1a/d41/d5e/f62 to d4/d61/d65/f86 0 2024-02-19T19:06:18.573 INFO:tasks.workunit.client.0.smithi088.stdout:8/225: write d9/f2e [984516,4128] 0 2024-02-19T19:06:18.578 INFO:tasks.workunit.client.0.smithi088.stdout:5/394: rename d4/d7/f1d to d4/f87 0 2024-02-19T19:06:18.579 INFO:tasks.workunit.client.0.smithi088.stdout:9/282: dread d1/d21/d2d/d18/d1f/f2e [0,4194304] 0 2024-02-19T19:06:18.580 INFO:tasks.workunit.client.0.smithi088.stdout:5/395: write d4/f8 [6660246,113747] 0 2024-02-19T19:06:18.587 INFO:tasks.workunit.client.0.smithi088.stdout:5/396: symlink d4/d61/d65/l88 0 2024-02-19T19:06:18.588 INFO:tasks.workunit.client.0.smithi088.stdout:2/193: dread d3/d1f/f3f [0,4194304] 0 2024-02-19T19:06:18.590 INFO:tasks.workunit.client.0.smithi088.stdout:8/226: dwrite d9/df/d1c/f16 [4194304,4194304] 0 2024-02-19T19:06:18.592 INFO:tasks.workunit.client.0.smithi088.stdout:7/275: dread f2 [0,4194304] 0 2024-02-19T19:06:18.592 INFO:tasks.workunit.client.0.smithi088.stdout:7/276: stat d14/f17 0 2024-02-19T19:06:18.593 INFO:tasks.workunit.client.0.smithi088.stdout:5/397: rmdir d4/d75 39 2024-02-19T19:06:18.594 INFO:tasks.workunit.client.0.smithi088.stdout:7/277: chown d14/d27/l50 60695 1 2024-02-19T19:06:18.595 INFO:tasks.workunit.client.0.smithi088.stdout:9/283: symlink d1/l57 0 2024-02-19T19:06:18.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.574+0000 7f79bd489700 1 -- 172.21.15.88:0/2966117257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b00a1740 msgr2=0x7f79b00a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:18.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.574+0000 7f79bd489700 1 --2- 172.21.15.88:0/2966117257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b00a1740 0x7f79b00a1b10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009910 tx=0x7f79ac009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:18.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.575+0000 7f79bd489700 1 -- 172.21.15.88:0/2966117257 shutdown_connections 2024-02-19T19:06:18.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.575+0000 7f79bd489700 1 --2- 172.21.15.88:0/2966117257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79b009b740 0x7f79b009bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:18.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.575+0000 7f79bd489700 1 --2- 172.21.15.88:0/2966117257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b00a1740 0x7f79b00a1b10 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f79ac009910 tx=0x7f79ac009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:18.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.575+0000 7f79bd489700 1 -- 172.21.15.88:0/2966117257 >> 172.21.15.88:0/2966117257 conn(0x7f79b000a1d0 msgr2=0x7f79b0099b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:18.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.575+0000 7f79bd489700 1 -- 172.21.15.88:0/2966117257 shutdown_connections 2024-02-19T19:06:18.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.575+0000 7f79bd489700 1 -- 172.21.15.88:0/2966117257 wait complete. 2024-02-19T19:06:18.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.576+0000 7f79bd489700 1 Processor -- start 2024-02-19T19:06:18.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.577+0000 7f79bd489700 1 -- start start 2024-02-19T19:06:18.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.577+0000 7f79bd489700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b009b740 0x7f79b0142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:18.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.577+0000 7f79bd489700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79b0142bd0 0x7f79b0145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:18.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.577+0000 7f79bd489700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01431e0 con 0x7f79b009b740 2024-02-19T19:06:18.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.577+0000 7f79bd489700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0143320 con 0x7f79b0142bd0 2024-02-19T19:06:18.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.577+0000 7f79b7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b009b740 0x7f79b0142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:18.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.577+0000 7f79b7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b009b740 0x7f79b0142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48364/0 (socket says 172.21.15.88:48364) 2024-02-19T19:06:18.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.577+0000 7f79b7fff700 1 -- 172.21.15.88:0/2908757852 learned_addr learned my addr 172.21.15.88:0/2908757852 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:18.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.578+0000 7f79b77fe700 1 --2- 172.21.15.88:0/2908757852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79b0142bd0 0x7f79b0145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:18.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.579+0000 7f79b7fff700 1 -- 172.21.15.88:0/2908757852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79b0142bd0 msgr2=0x7f79b0145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:18.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.579+0000 7f79b7fff700 1 --2- 172.21.15.88:0/2908757852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79b0142bd0 0x7f79b0145f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:18.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.579+0000 7f79b7fff700 1 -- 172.21.15.88:0/2908757852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac0095f0 con 0x7f79b009b740 2024-02-19T19:06:18.600 INFO:tasks.workunit.client.0.smithi088.stdout:2/194: creat d3/d1b/f42 x:0 0 0 2024-02-19T19:06:18.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.579+0000 7f79b7fff700 1 --2- 172.21.15.88:0/2908757852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b009b740 0x7f79b0142690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79ac000c00 tx=0x7f79ac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:18.602 INFO:tasks.workunit.client.0.smithi088.stdout:5/398: write d4/d75/f80 [1373157,69383] 0 2024-02-19T19:06:18.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.580+0000 7f79b57fa700 1 -- 172.21.15.88:0/2908757852 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79ac01d070 con 0x7f79b009b740 2024-02-19T19:06:18.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.581+0000 7f79b57fa700 1 -- 172.21.15.88:0/2908757852 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79ac005e10 con 0x7f79b009b740 2024-02-19T19:06:18.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.581+0000 7f79bd489700 1 -- 172.21.15.88:0/2908757852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b0146530 con 0x7f79b009b740 2024-02-19T19:06:18.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.581+0000 7f79b57fa700 1 -- 172.21.15.88:0/2908757852 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79ac00fe60 con 0x7f79b009b740 2024-02-19T19:06:18.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.581+0000 7f79bd489700 1 -- 172.21.15.88:0/2908757852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b0146a80 con 0x7f79b009b740 2024-02-19T19:06:18.603 INFO:tasks.workunit.client.0.smithi088.stdout:5/399: readlink d4/d1a/l50 0 2024-02-19T19:06:18.605 INFO:tasks.workunit.client.0.smithi088.stdout:5/400: write d4/f8 [6364701,18592] 0 2024-02-19T19:06:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.584+0000 7f79b57fa700 1 -- 172.21.15.88:0/2908757852 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f79ac00f460 con 0x7f79b009b740 2024-02-19T19:06:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.585+0000 7f79b57fa700 1 --2- 172.21.15.88:0/2908757852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f79a806db30 0x7f79a806ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.585+0000 7f79b57fa700 1 -- 172.21.15.88:0/2908757852 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f79ac08e940 con 0x7f79b009b740 2024-02-19T19:06:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.586+0000 7f79bd489700 1 -- 172.21.15.88:0/2908757852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c005320 con 0x7f79b009b740 2024-02-19T19:06:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.586+0000 7f79b77fe700 1 --2- 172.21.15.88:0/2908757852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f79a806db30 0x7f79a806ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.586+0000 7f79b77fe700 1 --2- 172.21.15.88:0/2908757852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f79a806db30 0x7f79a806ffe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009a10 tx=0x7f79a4009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:18.608 INFO:tasks.workunit.client.0.smithi088.stdout:6/268: truncate d5/d8/f3f 3878396 0 2024-02-19T19:06:18.609 INFO:tasks.workunit.client.0.smithi088.stdout:0/228: write d13/f1d [1320678,52109] 0 2024-02-19T19:06:18.611 INFO:tasks.workunit.client.0.smithi088.stdout:0/229: dread - d13/d24/d25/d2a/d2f/f41 zero size 2024-02-19T19:06:18.615 INFO:tasks.workunit.client.0.smithi088.stdout:8/227: mknod d9/c40 0 2024-02-19T19:06:18.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.602+0000 7f79b57fa700 1 -- 172.21.15.88:0/2908757852 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f79ac057fe0 con 0x7f79b009b740 2024-02-19T19:06:18.628 INFO:tasks.workunit.client.0.smithi088.stdout:5/401: creat d4/d1a/d22/d57/d84/f89 x:0 0 0 2024-02-19T19:06:18.633 INFO:tasks.workunit.client.0.smithi088.stdout:8/228: mknod d9/c41 0 2024-02-19T19:06:18.635 INFO:tasks.workunit.client.0.smithi088.stdout:5/402: dwrite d4/fe [0,4194304] 0 2024-02-19T19:06:18.636 INFO:tasks.workunit.client.0.smithi088.stdout:5/403: chown d4/d1a/c52 2784648 1 2024-02-19T19:06:18.637 INFO:tasks.workunit.client.0.smithi088.stdout:5/404: fsync d4/d1a/d22/f4c 0 2024-02-19T19:06:18.638 INFO:tasks.workunit.client.0.smithi088.stdout:6/269: mknod d5/d31/d25/c54 0 2024-02-19T19:06:18.643 INFO:tasks.workunit.client.0.smithi088.stdout:8/229: chown d9/df/d1c/d22/l24 1814 1 2024-02-19T19:06:18.647 INFO:tasks.workunit.client.0.smithi088.stdout:6/270: dwrite d5/d8/d3a/f46 [0,4194304] 0 2024-02-19T19:06:18.648 INFO:tasks.workunit.client.0.smithi088.stdout:8/230: fdatasync f5 0 2024-02-19T19:06:18.649 INFO:tasks.workunit.client.0.smithi088.stdout:6/271: truncate d5/f47 127900 0 2024-02-19T19:06:18.651 INFO:tasks.workunit.client.0.smithi088.stdout:6/272: chown d5/d19/d26/c3c 1636106 1 2024-02-19T19:06:18.654 INFO:tasks.workunit.client.0.smithi088.stdout:5/405: dwrite d4/d61/d65/f86 [0,4194304] 0 2024-02-19T19:06:18.656 INFO:tasks.workunit.client.0.smithi088.stdout:6/273: write d5/d8/d2c/f4c [528980,38261] 0 2024-02-19T19:06:18.657 INFO:tasks.workunit.client.0.smithi088.stdout:6/274: fdatasync d5/d31/d25/f49 0 2024-02-19T19:06:18.664 INFO:tasks.workunit.client.0.smithi088.stdout:8/231: symlink d9/df/d3d/l42 0 2024-02-19T19:06:18.667 INFO:tasks.workunit.client.0.smithi088.stdout:1/355: dwrite d6/d21/d6a/d4a/f56 [0,4194304] 0 2024-02-19T19:06:18.670 INFO:tasks.workunit.client.0.smithi088.stdout:8/232: dread d9/df/d1c/f16 [4194304,4194304] 0 2024-02-19T19:06:18.673 INFO:tasks.workunit.client.0.smithi088.stdout:6/275: mknod d5/d31/d25/c55 0 2024-02-19T19:06:18.674 INFO:tasks.workunit.client.0.smithi088.stdout:1/356: dread d6/d10/f24 [0,4194304] 0 2024-02-19T19:06:18.674 INFO:tasks.workunit.client.0.smithi088.stdout:1/357: fdatasync d6/d10/f4b 0 2024-02-19T19:06:18.675 INFO:tasks.workunit.client.0.smithi088.stdout:1/358: write d6/d21/f3a [4028120,57301] 0 2024-02-19T19:06:18.677 INFO:tasks.workunit.client.0.smithi088.stdout:1/359: truncate d6/d21/f70 421484 0 2024-02-19T19:06:18.678 INFO:tasks.workunit.client.0.smithi088.stdout:8/233: fdatasync f8 0 2024-02-19T19:06:18.686 INFO:tasks.workunit.client.0.smithi088.stdout:1/360: read d6/d10/f1d [636083,96828] 0 2024-02-19T19:06:18.690 INFO:tasks.workunit.client.0.smithi088.stdout:8/234: write d9/df/d1c/f1f [2194087,51381] 0 2024-02-19T19:06:18.696 INFO:tasks.workunit.client.0.smithi088.stdout:1/361: dread - d6/d10/f19 zero size 2024-02-19T19:06:18.698 INFO:tasks.workunit.client.0.smithi088.stdout:8/235: write d9/df/d1c/f16 [1445008,96579] 0 2024-02-19T19:06:18.699 INFO:tasks.workunit.client.0.smithi088.stdout:8/236: chown f5 2 1 2024-02-19T19:06:18.702 INFO:tasks.workunit.client.0.smithi088.stdout:9/284: dwrite d1/d8/d37/f41 [0,4194304] 0 2024-02-19T19:06:18.707 INFO:tasks.workunit.client.0.smithi088.stdout:1/362: rmdir d6/d10/d60/d29/d4e/d71 39 2024-02-19T19:06:18.707 INFO:tasks.workunit.client.0.smithi088.stdout:7/278: truncate d14/d56/f58 2505942 0 2024-02-19T19:06:18.713 INFO:tasks.workunit.client.0.smithi088.stdout:7/279: dwrite d14/d27/f32 [0,4194304] 0 2024-02-19T19:06:18.714 INFO:tasks.workunit.client.0.smithi088.stdout:9/285: creat d1/d8/d4a/f58 x:0 0 0 2024-02-19T19:06:18.718 INFO:tasks.workunit.client.0.smithi088.stdout:1/363: rename d6/d10/d60/l48 to d6/d10/d60/d29/l77 0 2024-02-19T19:06:18.719 INFO:tasks.workunit.client.0.smithi088.stdout:7/280: dread d14/d1d/d3c/d43/f5b [0,4194304] 0 2024-02-19T19:06:18.719 INFO:tasks.workunit.client.0.smithi088.stdout:1/364: write d6/d10/f3b [5090781,50930] 0 2024-02-19T19:06:18.725 INFO:tasks.workunit.client.0.smithi088.stdout:9/286: creat d1/d21/d2d/d18/d1f/f59 x:0 0 0 2024-02-19T19:06:18.729 INFO:tasks.workunit.client.0.smithi088.stdout:5/406: dwrite d4/f11 [4194304,4194304] 0 2024-02-19T19:06:18.730 INFO:tasks.workunit.client.0.smithi088.stdout:5/407: write d4/d7/f63 [4765649,13430] 0 2024-02-19T19:06:18.732 INFO:tasks.workunit.client.0.smithi088.stdout:5/408: chown d4/d75/f80 687381 1 2024-02-19T19:06:18.738 INFO:tasks.workunit.client.0.smithi088.stdout:1/365: creat d6/d10/f78 x:0 0 0 2024-02-19T19:06:18.740 INFO:tasks.workunit.client.0.smithi088.stdout:1/366: write d6/d21/d6a/d4a/f59 [656511,100237] 0 2024-02-19T19:06:18.748 INFO:tasks.workunit.client.0.smithi088.stdout:8/237: creat d9/d31/f43 x:0 0 0 2024-02-19T19:06:18.750 INFO:tasks.workunit.client.0.smithi088.stdout:6/276: truncate d5/d8/f3f 2208651 0 2024-02-19T19:06:18.751 INFO:tasks.workunit.client.0.smithi088.stdout:7/281: mkdir d14/d1d/d62 0 2024-02-19T19:06:18.752 INFO:tasks.workunit.client.0.smithi088.stdout:5/409: dread d4/d1a/d22/d57/f6a [0,4194304] 0 2024-02-19T19:06:18.753 INFO:tasks.workunit.client.0.smithi088.stdout:7/282: write d14/d1d/f38 [3217259,35807] 0 2024-02-19T19:06:18.754 INFO:tasks.workunit.client.0.smithi088.stdout:9/287: mkdir d1/d21/d5a 0 2024-02-19T19:06:18.754 INFO:tasks.workunit.client.0.smithi088.stdout:5/410: write d4/f8 [14854568,13478] 0 2024-02-19T19:06:18.760 INFO:tasks.workunit.client.0.smithi088.stdout:6/277: mkdir d5/d7/d32/d56 0 2024-02-19T19:06:18.760 INFO:tasks.workunit.client.0.smithi088.stdout:6/278: read d5/d19/d26/f44 [58289,15066] 0 2024-02-19T19:06:18.764 INFO:tasks.workunit.client.0.smithi088.stdout:6/279: dread d5/d8/d2c/f4c [0,4194304] 0 2024-02-19T19:06:18.765 INFO:tasks.workunit.client.0.smithi088.stdout:7/283: creat d14/d1d/d3c/f63 x:0 0 0 2024-02-19T19:06:18.769 INFO:tasks.workunit.client.0.smithi088.stdout:5/411: creat d4/d1a/d22/d57/d84/f8a x:0 0 0 2024-02-19T19:06:18.770 INFO:tasks.workunit.client.0.smithi088.stdout:9/288: dread d1/d21/d2d/f2a [0,4194304] 0 2024-02-19T19:06:18.771 INFO:tasks.workunit.client.0.smithi088.stdout:9/289: chown d1/d8/d3e/d4e/l4b 4776016 1 2024-02-19T19:06:18.776 INFO:tasks.workunit.client.0.smithi088.stdout:5/412: symlink d4/d75/l8b 0 2024-02-19T19:06:18.778 INFO:tasks.workunit.client.0.smithi088.stdout:5/413: truncate d4/d1a/d22/d57/d84/f89 450198 0 2024-02-19T19:06:18.779 INFO:tasks.workunit.client.0.smithi088.stdout:8/238: creat d9/d31/f44 x:0 0 0 2024-02-19T19:06:18.786 INFO:tasks.workunit.client.0.smithi088.stdout:8/239: dwrite f6 [4194304,4194304] 0 2024-02-19T19:06:18.789 INFO:tasks.workunit.client.0.smithi088.stdout:6/280: symlink d5/d19/l57 0 2024-02-19T19:06:18.793 INFO:tasks.workunit.client.0.smithi088.stdout:7/284: link d14/d1d/f39 d14/d28/d3b/f64 0 2024-02-19T19:06:18.797 INFO:tasks.workunit.client.0.smithi088.stdout:6/281: dwrite d5/d31/d25/f49 [0,4194304] 0 2024-02-19T19:06:18.798 INFO:tasks.workunit.client.0.smithi088.stdout:1/367: getdents d6 0 2024-02-19T19:06:18.801 INFO:tasks.workunit.client.0.smithi088.stdout:5/414: symlink d4/d1a/d22/d57/l8c 0 2024-02-19T19:06:18.802 INFO:tasks.workunit.client.0.smithi088.stdout:5/415: stat d4/d7/c17 0 2024-02-19T19:06:18.805 INFO:tasks.workunit.client.0.smithi088.stdout:9/290: symlink d1/d21/l5b 0 2024-02-19T19:06:18.806 INFO:tasks.workunit.client.0.smithi088.stdout:9/291: chown d1/d21/d2d/d23/c29 1 1 2024-02-19T19:06:18.807 INFO:tasks.workunit.client.0.smithi088.stdout:9/292: readlink d1/d8/d3e/d4e/l4b 0 2024-02-19T19:06:18.808 INFO:tasks.workunit.client.0.smithi088.stdout:8/240: dread d9/f2e [0,4194304] 0 2024-02-19T19:06:18.815 INFO:tasks.workunit.client.0.smithi088.stdout:9/293: creat d1/d8/f5c x:0 0 0 2024-02-19T19:06:18.817 INFO:tasks.workunit.client.0.smithi088.stdout:7/285: mkdir d14/d28/d65 0 2024-02-19T19:06:18.819 INFO:tasks.workunit.client.0.smithi088.stdout:1/368: mknod d6/d10/d60/d29/d63/c79 0 2024-02-19T19:06:18.819 INFO:tasks.workunit.client.0.smithi088.stdout:1/369: chown d6/d10/f78 0 1 2024-02-19T19:06:18.820 INFO:tasks.workunit.client.0.smithi088.stdout:1/370: dread - d6/d21/d6a/f6d zero size 2024-02-19T19:06:18.823 INFO:tasks.workunit.client.0.smithi088.stdout:8/241: symlink d9/d31/l45 0 2024-02-19T19:06:18.826 INFO:tasks.workunit.client.0.smithi088.stdout:9/294: mknod d1/d8/d4a/d4f/c5d 0 2024-02-19T19:06:18.828 INFO:tasks.workunit.client.0.smithi088.stdout:7/286: truncate d14/f30 4807506 0 2024-02-19T19:06:18.835 INFO:tasks.workunit.client.0.smithi088.stdout:8/242: rmdir d9/df/d1c 39 2024-02-19T19:06:18.839 INFO:tasks.workunit.client.0.smithi088.stdout:5/416: getdents d4/d1a/d22/d57/d84 0 2024-02-19T19:06:18.841 INFO:tasks.workunit.client.0.smithi088.stdout:5/417: read d4/f9 [622731,22853] 0 2024-02-19T19:06:18.843 INFO:tasks.workunit.client.0.smithi088.stdout:1/371: creat d6/d10/d60/d29/d4e/d71/f7a x:0 0 0 2024-02-19T19:06:18.846 INFO:tasks.workunit.client.0.smithi088.stdout:1/372: chown d6/d21/d6a/d4a/c55 67 1 2024-02-19T19:06:18.846 INFO:tasks.workunit.client.0.smithi088.stdout:7/287: dread d14/f1f [0,4194304] 0 2024-02-19T19:06:18.848 INFO:tasks.workunit.client.0.smithi088.stdout:7/288: chown c7 702 1 2024-02-19T19:06:18.849 INFO:tasks.workunit.client.0.smithi088.stdout:9/295: link d1/d8/f5c d1/d8/d4a/f5e 0 2024-02-19T19:06:18.849 INFO:tasks.workunit.client.0.smithi088.stdout:5/418: dwrite d4/d1a/d22/d57/f6b [0,4194304] 0 2024-02-19T19:06:18.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:18.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:18.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:18 smithi088 ceph-mon[88643]: Reconfiguring osd.4 (monmap changed)... 2024-02-19T19:06:18.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-19T19:06:18.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:18.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:18 smithi088 ceph-mon[88643]: Reconfiguring daemon osd.4 on smithi132 2024-02-19T19:06:18.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:18 smithi088 ceph-mon[88643]: from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:18.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:18 smithi088 ceph-mon[88643]: pgmap v43: 65 pgs: 65 active+clean; 526 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 41 MiB/s wr, 307 op/s 2024-02-19T19:06:18.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:18 smithi088 ceph-mon[88643]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:18.851 INFO:tasks.workunit.client.0.smithi088.stdout:5/419: chown d4/d1a/d22/d57/l7d 510954 1 2024-02-19T19:06:18.855 INFO:tasks.workunit.client.0.smithi088.stdout:3/360: sync 2024-02-19T19:06:18.858 INFO:tasks.workunit.client.0.smithi088.stdout:9/296: dwrite d1/d21/d2d/d18/d1f/f59 [0,4194304] 0 2024-02-19T19:06:18.860 INFO:tasks.workunit.client.0.smithi088.stdout:6/282: truncate d5/d8/f3f 1368988 0 2024-02-19T19:06:18.862 INFO:tasks.workunit.client.0.smithi088.stdout:1/373: mkdir d6/d10/d60/d29/d63/d7b 0 2024-02-19T19:06:18.864 INFO:tasks.workunit.client.0.smithi088.stdout:8/243: dwrite d9/df/d1c/f25 [8388608,4194304] 0 2024-02-19T19:06:18.870 INFO:tasks.workunit.client.0.smithi088.stdout:9/297: dwrite d1/f6 [0,4194304] 0 2024-02-19T19:06:18.885 INFO:tasks.workunit.client.0.smithi088.stdout:3/361: dwrite d4/fd [4194304,4194304] 0 2024-02-19T19:06:18.885 INFO:tasks.workunit.client.0.smithi088.stdout:6/283: unlink d5/d19/d26/l42 0 2024-02-19T19:06:18.891 INFO:tasks.workunit.client.0.smithi088.stdout:7/289: creat d14/d28/d65/f66 x:0 0 0 2024-02-19T19:06:18.893 INFO:tasks.workunit.client.0.smithi088.stdout:3/362: dread - d4/f28 zero size 2024-02-19T19:06:18.894 INFO:tasks.workunit.client.0.smithi088.stdout:8/244: unlink d9/ce 0 2024-02-19T19:06:18.895 INFO:tasks.workunit.client.0.smithi088.stdout:1/374: creat d6/d10/d60/d29/d4e/d6e/f7c x:0 0 0 2024-02-19T19:06:18.896 INFO:tasks.workunit.client.0.smithi088.stdout:1/375: truncate d6/d21/f72 889080 0 2024-02-19T19:06:18.897 INFO:tasks.workunit.client.0.smithi088.stdout:5/420: rename d4/d1a/l3e to d4/d1a/l8d 0 2024-02-19T19:06:18.899 INFO:tasks.workunit.client.0.smithi088.stdout:5/421: fsync d4/d1a/d41/f71 0 2024-02-19T19:06:18.900 INFO:tasks.workunit.client.0.smithi088.stdout:7/290: write f2 [4955723,7555] 0 2024-02-19T19:06:18.901 INFO:tasks.workunit.client.0.smithi088.stdout:6/284: mknod d5/d8/c58 0 2024-02-19T19:06:18.902 INFO:tasks.workunit.client.0.smithi088.stdout:9/298: dread d1/f20 [0,4194304] 0 2024-02-19T19:06:18.904 INFO:tasks.workunit.client.0.smithi088.stdout:7/291: read d14/d28/d3b/f64 [53692,42809] 0 2024-02-19T19:06:18.906 INFO:tasks.workunit.client.0.smithi088.stdout:1/376: mkdir d6/d21/d6a/d7d 0 2024-02-19T19:06:18.907 INFO:tasks.workunit.client.0.smithi088.stdout:1/377: write d6/f1b [2422209,2102] 0 2024-02-19T19:06:18.907 INFO:tasks.workunit.client.0.smithi088.stdout:1/378: stat d6/d21/l6f 0 2024-02-19T19:06:18.909 INFO:tasks.workunit.client.0.smithi088.stdout:5/422: mknod d4/d7/c8e 0 2024-02-19T19:06:18.911 INFO:tasks.workunit.client.0.smithi088.stdout:6/285: rmdir d5/d8/d2c 39 2024-02-19T19:06:18.913 INFO:tasks.workunit.client.0.smithi088.stdout:3/363: mkdir d4/d87 0 2024-02-19T19:06:18.914 INFO:tasks.workunit.client.0.smithi088.stdout:3/364: fdatasync d4/d12/f47 0 2024-02-19T19:06:18.915 INFO:tasks.workunit.client.0.smithi088.stdout:3/365: chown d4/d1a/d7f/d86 5040225 1 2024-02-19T19:06:18.916 INFO:tasks.workunit.client.0.smithi088.stdout:9/299: truncate d1/d21/d2d/d23/f26 470503 0 2024-02-19T19:06:18.919 INFO:tasks.workunit.client.0.smithi088.stdout:7/292: rename d14/d27/c2b to d14/d1d/d3c/d43/c67 0 2024-02-19T19:06:18.925 INFO:tasks.workunit.client.0.smithi088.stdout:6/286: creat d5/d31/f59 x:0 0 0 2024-02-19T19:06:18.926 INFO:tasks.workunit.client.0.smithi088.stdout:8/245: creat d9/d31/f46 x:0 0 0 2024-02-19T19:06:18.927 INFO:tasks.workunit.client.0.smithi088.stdout:8/246: dread - d9/d31/f43 zero size 2024-02-19T19:06:18.929 INFO:tasks.workunit.client.0.smithi088.stdout:0/230: sync 2024-02-19T19:06:18.929 INFO:tasks.workunit.client.0.smithi088.stdout:2/195: sync 2024-02-19T19:06:18.929 INFO:tasks.workunit.client.0.smithi088.stdout:4/221: sync 2024-02-19T19:06:18.929 INFO:tasks.workunit.client.0.smithi088.stdout:3/366: write d4/d1a/d1e/d31/f84 [1242830,86993] 0 2024-02-19T19:06:18.929 INFO:tasks.workunit.client.0.smithi088.stdout:0/231: readlink lf 0 2024-02-19T19:06:18.930 INFO:tasks.workunit.client.0.smithi088.stdout:8/247: dread - d9/d31/f44 zero size 2024-02-19T19:06:18.931 INFO:tasks.workunit.client.0.smithi088.stdout:5/423: dwrite d4/d1a/f7c [0,4194304] 0 2024-02-19T19:06:18.932 INFO:tasks.workunit.client.0.smithi088.stdout:2/196: dread - d3/d14/f19 zero size 2024-02-19T19:06:18.935 INFO:tasks.workunit.client.0.smithi088.stdout:1/379: mkdir d6/d21/d6a/d7d/d7e 0 2024-02-19T19:06:18.935 INFO:tasks.workunit.client.0.smithi088.stdout:2/197: chown d3/df/f17 1051 1 2024-02-19T19:06:18.937 INFO:tasks.workunit.client.0.smithi088.stdout:1/380: fdatasync d6/d10/d60/d29/d4e/d71/f7a 0 2024-02-19T19:06:18.938 INFO:tasks.workunit.client.0.smithi088.stdout:5/424: write d4/d7/f63 [1319216,92816] 0 2024-02-19T19:06:18.945 INFO:tasks.workunit.client.0.smithi088.stdout:9/300: mknod d1/d21/d2d/c5f 0 2024-02-19T19:06:18.945 INFO:tasks.workunit.client.0.smithi088.stdout:5/425: dread d4/d1a/d41/d5e/f7f [0,4194304] 0 2024-02-19T19:06:18.947 INFO:tasks.workunit.client.0.smithi088.stdout:5/426: chown d4/d1a/d41/d5e/f82 7186205 1 2024-02-19T19:06:18.948 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:18.948 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:18.948 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:18 smithi132 ceph-mon[70676]: Reconfiguring osd.4 (monmap changed)... 2024-02-19T19:06:18.949 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-19T19:06:18.949 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:18.949 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:18 smithi132 ceph-mon[70676]: Reconfiguring daemon osd.4 on smithi132 2024-02-19T19:06:18.949 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:18 smithi132 ceph-mon[70676]: from='client.34184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:18.949 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:18 smithi132 ceph-mon[70676]: pgmap v43: 65 pgs: 65 active+clean; 526 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 1.0 MiB/s rd, 41 MiB/s wr, 307 op/s 2024-02-19T19:06:18.949 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:18 smithi132 ceph-mon[70676]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:18.952 INFO:tasks.workunit.client.0.smithi088.stdout:3/367: mkdir d4/d1a/d7f/d88 0 2024-02-19T19:06:18.953 INFO:tasks.workunit.client.0.smithi088.stdout:3/368: write d4/d1a/d23/f60 [30187,26134] 0 2024-02-19T19:06:18.955 INFO:tasks.workunit.client.0.smithi088.stdout:3/369: truncate d4/d1a/d1e/d31/f84 2030372 0 2024-02-19T19:06:18.956 INFO:tasks.workunit.client.0.smithi088.stdout:7/293: write f8 [492502,98277] 0 2024-02-19T19:06:18.960 INFO:tasks.workunit.client.0.smithi088.stdout:6/287: symlink d5/d8/l5a 0 2024-02-19T19:06:18.962 INFO:tasks.workunit.client.0.smithi088.stdout:4/222: dread f0 [4194304,4194304] 0 2024-02-19T19:06:18.964 INFO:tasks.workunit.client.0.smithi088.stdout:6/288: write d5/f1e [614542,28708] 0 2024-02-19T19:06:18.964 INFO:tasks.workunit.client.0.smithi088.stdout:0/232: dread d13/f1a [0,4194304] 0 2024-02-19T19:06:18.966 INFO:tasks.workunit.client.0.smithi088.stdout:8/248: dread f6 [0,4194304] 0 2024-02-19T19:06:18.969 INFO:tasks.workunit.client.0.smithi088.stdout:0/233: write d13/d24/d25/f49 [3288185,78473] 0 2024-02-19T19:06:18.970 INFO:tasks.workunit.client.0.smithi088.stdout:8/249: fsync d9/d31/f44 0 2024-02-19T19:06:18.970 INFO:tasks.workunit.client.0.smithi088.stdout:7/294: dwrite d14/d1d/f38 [4194304,4194304] 0 2024-02-19T19:06:18.971 INFO:tasks.workunit.client.0.smithi088.stdout:5/427: creat d4/d7/f8f x:0 0 0 2024-02-19T19:06:18.972 INFO:tasks.workunit.client.0.smithi088.stdout:5/428: write d4/d1a/f77 [941094,33363] 0 2024-02-19T19:06:18.974 INFO:tasks.workunit.client.0.smithi088.stdout:7/295: write d14/d28/d3b/f4a [1072716,74103] 0 2024-02-19T19:06:18.974 INFO:tasks.workunit.client.0.smithi088.stdout:5/429: read - d4/d1a/f54 zero size 2024-02-19T19:06:18.975 INFO:tasks.workunit.client.0.smithi088.stdout:5/430: chown d4/l32 0 1 2024-02-19T19:06:18.978 INFO:tasks.workunit.client.0.smithi088.stdout:2/198: write d3/df/f11 [3826162,110319] 0 2024-02-19T19:06:18.985 INFO:tasks.workunit.client.0.smithi088.stdout:6/289: fdatasync d5/de/f18 0 2024-02-19T19:06:18.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.967+0000 7f79bd489700 1 -- 172.21.15.88:0/2908757852 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f799c000bf0 con 0x7f79a806db30 2024-02-19T19:06:18.991 INFO:tasks.workunit.client.0.smithi088.stdout:6/290: dwrite d5/d7/f4f [0,4194304] 0 2024-02-19T19:06:18.992 INFO:tasks.workunit.client.0.smithi088.stdout:9/301: mkdir d1/d60 0 2024-02-19T19:06:18.992 INFO:tasks.workunit.client.0.smithi088.stdout:6/291: chown d5/de/f10 58595 1 2024-02-19T19:06:18.994 INFO:tasks.workunit.client.0.smithi088.stdout:1/381: truncate d6/d10/f3b 1926301 0 2024-02-19T19:06:18.995 INFO:tasks.workunit.client.0.smithi088.stdout:1/382: readlink d6/d10/d60/d29/l3f 0 2024-02-19T19:06:18.995 INFO:tasks.workunit.client.0.smithi088.stdout:1/383: fsync d6/f1b 0 2024-02-19T19:06:18.999 INFO:tasks.workunit.client.0.smithi088.stdout:8/250: rename d9/d31 to d9/df/d1c/d2d/d47 0 2024-02-19T19:06:19.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.985+0000 7f79b57fa700 1 -- 172.21.15.88:0/2908757852 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f799c000bf0 con 0x7f79a806db30 2024-02-19T19:06:19.006 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:06:19.006 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (3m) 64s ago 13m 25.0M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:06:19.006 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (14m) 64s ago 14m 7960k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:06:19.006 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (12m) 41s ago 12m 7902k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:06:19.006 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (14m) 64s ago 14m 6665k - 18.2.0 dc2bc1663786 c89a22d95e30 2024-02-19T19:06:19.006 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (12m) 41s ago 12m 6660k - 18.2.0 dc2bc1663786 e4340cde7edb 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (2m) 64s ago 13m 83.1M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (7m) 64s ago 7m 15.3M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (7m) 64s ago 7m 107M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (7m) 41s ago 7m 119M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (7m) 41s ago 7m 19.6M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (4m) 64s ago 15m 476M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (4m) 41s ago 12m 448M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (94s) 64s ago 15m 32.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (48s) 41s ago 12m 26.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (4m) 64s ago 14m 15.8M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (4m) 41s ago 12m 15.2M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (11m) 64s ago 11m 176M 1216M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (10m) 64s ago 10m 181M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (10m) 64s ago 10m 147M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:06:19.007 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (10m) 41s ago 10m 164M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:06:19.008 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (9m) 41s ago 9m 149M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:06:19.008 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (9m) 41s ago 9m 144M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:06:19.008 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (3m) 64s ago 12m 64.8M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:06:19.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.988+0000 7f79bd489700 1 -- 172.21.15.88:0/2908757852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f79a806db30 msgr2=0x7f79a806ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:19.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.988+0000 7f79bd489700 1 --2- 172.21.15.88:0/2908757852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f79a806db30 0x7f79a806ffe0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f79a4009a10 tx=0x7f79a4009190 comp rx=0 tx=0).stop 2024-02-19T19:06:19.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.988+0000 7f79bd489700 1 -- 172.21.15.88:0/2908757852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b009b740 msgr2=0x7f79b0142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:19.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.988+0000 7f79bd489700 1 --2- 172.21.15.88:0/2908757852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b009b740 0x7f79b0142690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f79ac000c00 tx=0x7f79ac00bd70 comp rx=0 tx=0).stop 2024-02-19T19:06:19.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.988+0000 7f79bd489700 1 -- 172.21.15.88:0/2908757852 shutdown_connections 2024-02-19T19:06:19.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.988+0000 7f79bd489700 1 --2- 172.21.15.88:0/2908757852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f79a806db30 0x7f79a806ffe0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.988+0000 7f79bd489700 1 --2- 172.21.15.88:0/2908757852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79b009b740 0x7f79b0142690 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.988+0000 7f79bd489700 1 --2- 172.21.15.88:0/2908757852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79b0142bd0 0x7f79b0145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.988+0000 7f79bd489700 1 -- 172.21.15.88:0/2908757852 >> 172.21.15.88:0/2908757852 conn(0x7f79b000a1d0 msgr2=0x7f79b000bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:19.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.988+0000 7f79bd489700 1 -- 172.21.15.88:0/2908757852 shutdown_connections 2024-02-19T19:06:19.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:18.988+0000 7f79bd489700 1 -- 172.21.15.88:0/2908757852 wait complete. 2024-02-19T19:06:19.017 INFO:tasks.workunit.client.0.smithi088.stdout:9/302: chown d1/d21/d2d/d23/l30 31715266 1 2024-02-19T19:06:19.017 INFO:tasks.workunit.client.0.smithi088.stdout:9/303: readlink d1/lb 0 2024-02-19T19:06:19.020 INFO:tasks.workunit.client.0.smithi088.stdout:0/234: mkdir d13/d4e 0 2024-02-19T19:06:19.022 INFO:tasks.workunit.client.0.smithi088.stdout:0/235: readlink d13/d24/d25/d2a/d34/l45 0 2024-02-19T19:06:19.024 INFO:tasks.workunit.client.0.smithi088.stdout:4/223: dwrite d2/d8/f16 [0,4194304] 0 2024-02-19T19:06:19.024 INFO:tasks.workunit.client.0.smithi088.stdout:0/236: write d13/d24/d25/d2a/d34/f4d [356377,7572] 0 2024-02-19T19:06:19.026 INFO:tasks.workunit.client.0.smithi088.stdout:4/224: readlink d2/d35/d39/d3a/l3d 0 2024-02-19T19:06:19.034 INFO:tasks.workunit.client.0.smithi088.stdout:0/237: dwrite d13/d24/d25/d2a/d34/f43 [0,4194304] 0 2024-02-19T19:06:19.040 INFO:tasks.workunit.client.0.smithi088.stdout:4/225: fdatasync f0 0 2024-02-19T19:06:19.041 INFO:tasks.workunit.client.0.smithi088.stdout:4/226: chown d2/d35 1544628 1 2024-02-19T19:06:19.042 INFO:tasks.workunit.client.0.smithi088.stdout:0/238: dwrite d13/d24/d25/f26 [0,4194304] 0 2024-02-19T19:06:19.042 INFO:tasks.workunit.client.0.smithi088.stdout:8/251: mkdir d9/df/d48 0 2024-02-19T19:06:19.045 INFO:tasks.workunit.client.0.smithi088.stdout:7/296: link d14/d1d/d3c/d43/f5b d14/d1d/d3c/d43/d5c/f68 0 2024-02-19T19:06:19.048 INFO:tasks.workunit.client.0.smithi088.stdout:3/370: getdents d4/d1a/d7f 0 2024-02-19T19:06:19.051 INFO:tasks.workunit.client.0.smithi088.stdout:4/227: dwrite d2/d8/f29 [0,4194304] 0 2024-02-19T19:06:19.054 INFO:tasks.workunit.client.0.smithi088.stdout:4/228: dread d2/d32/d9/f20 [0,4194304] 0 2024-02-19T19:06:19.057 INFO:tasks.workunit.client.0.smithi088.stdout:2/199: truncate d3/d1f/f3f 3667322 0 2024-02-19T19:06:19.064 INFO:tasks.workunit.client.0.smithi088.stdout:5/431: getdents d4/d75 0 2024-02-19T19:06:19.075 INFO:tasks.workunit.client.0.smithi088.stdout:6/292: link d5/d8/l1d d5/d19/l5b 0 2024-02-19T19:06:19.077 INFO:tasks.workunit.client.0.smithi088.stdout:1/384: getdents d6/d21/d6a/d4a 0 2024-02-19T19:06:19.078 INFO:tasks.workunit.client.0.smithi088.stdout:1/385: chown d6/d21/f70 72997636 1 2024-02-19T19:06:19.078 INFO:tasks.workunit.client.0.smithi088.stdout:1/386: readlink d6/d21/l42 0 2024-02-19T19:06:19.079 INFO:tasks.workunit.client.0.smithi088.stdout:1/387: fdatasync d6/d21/d6a/d4a/f59 0 2024-02-19T19:06:19.084 INFO:tasks.workunit.client.0.smithi088.stdout:8/252: dwrite d9/df/f36 [0,4194304] 0 2024-02-19T19:06:19.086 INFO:tasks.workunit.client.0.smithi088.stdout:7/297: creat d14/d1d/d3c/d43/d5c/f69 x:0 0 0 2024-02-19T19:06:19.090 INFO:tasks.workunit.client.0.smithi088.stdout:8/253: dread f6 [0,4194304] 0 2024-02-19T19:06:19.091 INFO:tasks.workunit.client.0.smithi088.stdout:9/304: creat d1/d21/d2f/f61 x:0 0 0 2024-02-19T19:06:19.093 INFO:tasks.workunit.client.0.smithi088.stdout:6/293: rmdir d5/d8/d3a 39 2024-02-19T19:06:19.097 INFO:tasks.workunit.client.0.smithi088.stdout:1/388: mknod d6/d10/d60/d29/d63/c7f 0 2024-02-19T19:06:19.098 INFO:tasks.workunit.client.0.smithi088.stdout:0/239: rmdir d13/d4e 0 2024-02-19T19:06:19.099 INFO:tasks.workunit.client.0.smithi088.stdout:8/254: dwrite d9/df/d1c/f16 [8388608,4194304] 0 2024-02-19T19:06:19.100 INFO:tasks.workunit.client.0.smithi088.stdout:0/240: fsync d13/d24/f32 0 2024-02-19T19:06:19.107 INFO:tasks.workunit.client.0.smithi088.stdout:3/371: rename d4/d1a/d1e/c73 to d4/d12/c89 0 2024-02-19T19:06:19.108 INFO:tasks.workunit.client.0.smithi088.stdout:4/229: creat d2/d32/f44 x:0 0 0 2024-02-19T19:06:19.120 INFO:tasks.workunit.client.0.smithi088.stdout:0/241: creat d13/d24/d25/d2a/d2f/f4f x:0 0 0 2024-02-19T19:06:19.120 INFO:tasks.workunit.client.0.smithi088.stdout:0/242: chown d13/f3a 2319 1 2024-02-19T19:06:19.122 INFO:tasks.workunit.client.0.smithi088.stdout:0/243: write d13/d24/d25/d2a/d2f/f4f [131418,73628] 0 2024-02-19T19:06:19.123 INFO:tasks.workunit.client.0.smithi088.stdout:0/244: chown d13/d24/d25/d2a/f48 0 1 2024-02-19T19:06:19.124 INFO:tasks.workunit.client.0.smithi088.stdout:7/298: rename d14/d1d/d3c/l48 to d14/d28/d65/l6a 0 2024-02-19T19:06:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:0/245: write d13/d24/d25/d2a/d34/f35 [1595614,111977] 0 2024-02-19T19:06:19.125 INFO:tasks.workunit.client.0.smithi088.stdout:7/299: chown d14/f54 329 1 2024-02-19T19:06:19.126 INFO:tasks.workunit.client.0.smithi088.stdout:7/300: chown d14/f22 4 1 2024-02-19T19:06:19.127 INFO:tasks.workunit.client.0.smithi088.stdout:7/301: truncate fd 3536813 0 2024-02-19T19:06:19.127 INFO:tasks.workunit.client.0.smithi088.stdout:7/302: chown d14/d1d/f38 4076129 1 2024-02-19T19:06:19.128 INFO:tasks.workunit.client.0.smithi088.stdout:3/372: unlink d4/d1a/d23/c56 0 2024-02-19T19:06:19.128 INFO:tasks.workunit.client.0.smithi088.stdout:9/305: write d1/fc [3172030,122977] 0 2024-02-19T19:06:19.129 INFO:tasks.workunit.client.0.smithi088.stdout:4/230: mkdir d2/d32/d25/d45 0 2024-02-19T19:06:19.130 INFO:tasks.workunit.client.0.smithi088.stdout:8/255: write d9/f2e [383523,73324] 0 2024-02-19T19:06:19.132 INFO:tasks.workunit.client.0.smithi088.stdout:2/200: getdents d3/d14 0 2024-02-19T19:06:19.132 INFO:tasks.workunit.client.0.smithi088.stdout:6/294: getdents d5/d45 0 2024-02-19T19:06:19.134 INFO:tasks.workunit.client.0.smithi088.stdout:6/295: fdatasync d5/de/f10 0 2024-02-19T19:06:19.136 INFO:tasks.workunit.client.0.smithi088.stdout:1/389: rename d6/d10/d60/d29/d4e to d6/d21/d6a/d7d/d7e/d80 0 2024-02-19T19:06:19.137 INFO:tasks.workunit.client.0.smithi088.stdout:3/373: dwrite d4/d1a/d1e/d31/d32/f4c [0,4194304] 0 2024-02-19T19:06:19.138 INFO:tasks.workunit.client.0.smithi088.stdout:3/374: truncate d4/d12/f47 12905 0 2024-02-19T19:06:19.139 INFO:tasks.workunit.client.0.smithi088.stdout:0/246: creat d13/d24/d25/d2a/d2f/f50 x:0 0 0 2024-02-19T19:06:19.143 INFO:tasks.workunit.client.0.smithi088.stdout:7/303: mkdir d14/d27/d6b 0 2024-02-19T19:06:19.144 INFO:tasks.workunit.client.0.smithi088.stdout:7/304: chown d14/c46 0 1 2024-02-19T19:06:19.147 INFO:tasks.workunit.client.0.smithi088.stdout:2/201: write d3/f40 [547663,90415] 0 2024-02-19T19:06:19.152 INFO:tasks.workunit.client.0.smithi088.stdout:3/375: chown d4/ff 10 1 2024-02-19T19:06:19.156 INFO:tasks.workunit.client.0.smithi088.stdout:7/305: rename f8 to d14/d1d/d3c/f6c 0 2024-02-19T19:06:19.159 INFO:tasks.workunit.client.0.smithi088.stdout:8/256: mkdir d9/df/d1c/d2d/d47/d49 0 2024-02-19T19:06:19.161 INFO:tasks.workunit.client.0.smithi088.stdout:8/257: truncate d9/df/d1c/d2d/d47/f43 727061 0 2024-02-19T19:06:19.161 INFO:tasks.workunit.client.0.smithi088.stdout:4/231: link d2/d35/d39/d3a/f42 d2/d32/d25/f46 0 2024-02-19T19:06:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.143+0000 7f95247dc700 1 -- 172.21.15.88:0/3374560663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951c0ff3c0 msgr2=0x7f951c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.143+0000 7f95247dc700 1 --2- 172.21.15.88:0/3374560663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951c0ff3c0 0x7f951c0ff790 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9518009910 tx=0x7f9518009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.143+0000 7f95247dc700 1 -- 172.21.15.88:0/3374560663 shutdown_connections 2024-02-19T19:06:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.143+0000 7f95247dc700 1 --2- 172.21.15.88:0/3374560663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f951c0ffd60 0x7f951c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.143+0000 7f95247dc700 1 --2- 172.21.15.88:0/3374560663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951c0ff3c0 0x7f951c0ff790 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.143+0000 7f95247dc700 1 -- 172.21.15.88:0/3374560663 >> 172.21.15.88:0/3374560663 conn(0x7f951c0fac80 msgr2=0x7f951c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.145+0000 7f95247dc700 1 -- 172.21.15.88:0/3374560663 shutdown_connections 2024-02-19T19:06:19.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.145+0000 7f95247dc700 1 -- 172.21.15.88:0/3374560663 wait complete. 2024-02-19T19:06:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.145+0000 7f95247dc700 1 Processor -- start 2024-02-19T19:06:19.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.145+0000 7f95247dc700 1 -- start start 2024-02-19T19:06:19.166 INFO:tasks.workunit.client.0.smithi088.stdout:2/202: mkdir d3/d1b/d43 0 2024-02-19T19:06:19.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f95247dc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951c0ff3c0 0x7f951c195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:19.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f95247dc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f951c0ffd60 0x7f951c196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f95247dc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c196b20 con 0x7f951c0ff3c0 2024-02-19T19:06:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f95247dc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c19a8b0 con 0x7f951c0ffd60 2024-02-19T19:06:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f9521d77700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f951c0ffd60 0x7f951c196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f9521d77700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f951c0ffd60 0x7f951c196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:60610/0 (socket says 172.21.15.88:60610) 2024-02-19T19:06:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f9521d77700 1 -- 172.21.15.88:0/3792125569 learned_addr learned my addr 172.21.15.88:0/3792125569 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f9522578700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951c0ff3c0 0x7f951c195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f9521d77700 1 -- 172.21.15.88:0/3792125569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951c0ff3c0 msgr2=0x7f951c195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f9521d77700 1 --2- 172.21.15.88:0/3792125569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951c0ff3c0 0x7f951c195f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f9521d77700 1 -- 172.21.15.88:0/3792125569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95180095f0 con 0x7f951c0ffd60 2024-02-19T19:06:19.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.146+0000 7f9522578700 1 --2- 172.21.15.88:0/3792125569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951c0ff3c0 0x7f951c195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:06:19.168 INFO:tasks.workunit.client.0.smithi088.stdout:2/203: write d3/f39 [133317,99251] 0 2024-02-19T19:06:19.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.147+0000 7f9521d77700 1 --2- 172.21.15.88:0/3792125569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f951c0ffd60 0x7f951c196440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f950c00d8d0 tx=0x7f950c00dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:19.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.147+0000 7f95137fe700 1 -- 172.21.15.88:0/3792125569 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f950c0041f0 con 0x7f951c0ffd60 2024-02-19T19:06:19.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.147+0000 7f95247dc700 1 -- 172.21.15.88:0/3792125569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c19ab90 con 0x7f951c0ffd60 2024-02-19T19:06:19.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.147+0000 7f95247dc700 1 -- 172.21.15.88:0/3792125569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c19b0e0 con 0x7f951c0ffd60 2024-02-19T19:06:19.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.147+0000 7f95137fe700 1 -- 172.21.15.88:0/3792125569 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f950c007620 con 0x7f951c0ffd60 2024-02-19T19:06:19.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.147+0000 7f95137fe700 1 -- 172.21.15.88:0/3792125569 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f950c005630 con 0x7f951c0ffd60 2024-02-19T19:06:19.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.148+0000 7f95247dc700 1 -- 172.21.15.88:0/3792125569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c04f2a0 con 0x7f951c0ffd60 2024-02-19T19:06:19.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.149+0000 7f95137fe700 1 -- 172.21.15.88:0/3792125569 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f950c004830 con 0x7f951c0ffd60 2024-02-19T19:06:19.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.149+0000 7f95137fe700 1 --2- 172.21.15.88:0/3792125569 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da30 0x7f950806fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:19.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.149+0000 7f9522578700 1 --2- 172.21.15.88:0/3792125569 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da30 0x7f950806fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:19.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.149+0000 7f95137fe700 1 -- 172.21.15.88:0/3792125569 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f950c08d4f0 con 0x7f951c0ffd60 2024-02-19T19:06:19.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.150+0000 7f9522578700 1 --2- 172.21.15.88:0/3792125569 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da30 0x7f950806fee0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9518006010 tx=0x7f951801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:19.172 INFO:tasks.workunit.client.0.smithi088.stdout:3/376: chown d4/l5 20393 1 2024-02-19T19:06:19.173 INFO:tasks.workunit.client.0.smithi088.stdout:1/390: write d6/fb [1597870,64590] 0 2024-02-19T19:06:19.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.154+0000 7f95137fe700 1 -- 172.21.15.88:0/3792125569 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f950c056b10 con 0x7f951c0ffd60 2024-02-19T19:06:19.181 INFO:tasks.workunit.client.0.smithi088.stdout:7/306: mknod d14/d28/d3b/c6d 0 2024-02-19T19:06:19.187 INFO:tasks.workunit.client.0.smithi088.stdout:2/204: mknod d3/d1b/c44 0 2024-02-19T19:06:19.190 INFO:tasks.workunit.client.0.smithi088.stdout:6/296: link d5/d19/l57 d5/de/l5c 0 2024-02-19T19:06:19.191 INFO:tasks.workunit.client.0.smithi088.stdout:3/377: truncate d4/ff 1765360 0 2024-02-19T19:06:19.193 INFO:tasks.workunit.client.0.smithi088.stdout:0/247: truncate d13/f3a 326862 0 2024-02-19T19:06:19.196 INFO:tasks.workunit.client.0.smithi088.stdout:9/306: getdents d1/d8/d4a 0 2024-02-19T19:06:19.198 INFO:tasks.workunit.client.0.smithi088.stdout:9/307: dread - d1/d8/d4a/d4f/f51 zero size 2024-02-19T19:06:19.201 INFO:tasks.workunit.client.0.smithi088.stdout:8/258: dwrite f8 [0,4194304] 0 2024-02-19T19:06:19.203 INFO:tasks.workunit.client.0.smithi088.stdout:8/259: fsync d9/df/d1c/f29 0 2024-02-19T19:06:19.206 INFO:tasks.workunit.client.0.smithi088.stdout:6/297: dread d5/ff [0,4194304] 0 2024-02-19T19:06:19.214 INFO:tasks.workunit.client.0.smithi088.stdout:4/232: link d2/c3 d2/d35/c47 0 2024-02-19T19:06:19.219 INFO:tasks.workunit.client.0.smithi088.stdout:2/205: write d3/d1f/f3a [698916,32162] 0 2024-02-19T19:06:19.225 INFO:tasks.workunit.client.0.smithi088.stdout:9/308: rmdir d1/d21/d2d/d23 39 2024-02-19T19:06:19.231 INFO:tasks.workunit.client.0.smithi088.stdout:0/248: mknod d13/c51 0 2024-02-19T19:06:19.231 INFO:tasks.workunit.client.0.smithi088.stdout:9/309: dwrite d1/d8/d4a/f58 [0,4194304] 0 2024-02-19T19:06:19.233 INFO:tasks.workunit.client.0.smithi088.stdout:9/310: fsync d1/d13/f12 0 2024-02-19T19:06:19.233 INFO:tasks.workunit.client.0.smithi088.stdout:3/378: write d4/f2d [443385,96500] 0 2024-02-19T19:06:19.234 INFO:tasks.workunit.client.0.smithi088.stdout:3/379: dread - d4/d1a/d1e/d31/f71 zero size 2024-02-19T19:06:19.234 INFO:tasks.workunit.client.0.smithi088.stdout:6/298: rmdir d5/d31 39 2024-02-19T19:06:19.236 INFO:tasks.workunit.client.0.smithi088.stdout:3/380: dread - d4/d1a/d1e/d74/d7c/f80 zero size 2024-02-19T19:06:19.236 INFO:tasks.workunit.client.0.smithi088.stdout:1/391: link d6/d10/d60/d29/f37 d6/d21/d64/f81 0 2024-02-19T19:06:19.241 INFO:tasks.workunit.client.0.smithi088.stdout:7/307: link d14/f2e d14/f6e 0 2024-02-19T19:06:19.245 INFO:tasks.workunit.client.0.smithi088.stdout:1/392: dwrite d6/d21/d6a/d7d/d7e/d80/d6e/f7c [0,4194304] 0 2024-02-19T19:06:19.250 INFO:tasks.workunit.client.0.smithi088.stdout:0/249: rename d13/d24/d25 to d13/d24/d25/d52 22 2024-02-19T19:06:19.260 INFO:tasks.workunit.client.0.smithi088.stdout:4/233: fsync d2/d32/f23 0 2024-02-19T19:06:19.267 INFO:tasks.workunit.client.0.smithi088.stdout:8/260: creat d9/f4a x:0 0 0 2024-02-19T19:06:19.268 INFO:tasks.workunit.client.0.smithi088.stdout:8/261: write d9/f2e [1516462,114484] 0 2024-02-19T19:06:19.274 INFO:tasks.workunit.client.0.smithi088.stdout:1/393: write d6/d10/f19 [131111,13300] 0 2024-02-19T19:06:19.275 INFO:tasks.workunit.client.0.smithi088.stdout:3/381: fsync d4/f38 0 2024-02-19T19:06:19.276 INFO:tasks.workunit.client.0.smithi088.stdout:4/234: dread d2/d32/f1c [0,4194304] 0 2024-02-19T19:06:19.277 INFO:tasks.workunit.client.0.smithi088.stdout:4/235: write d2/d8/f29 [1166839,121608] 0 2024-02-19T19:06:19.310 INFO:tasks.workunit.client.0.smithi088.stdout:6/299: write d5/d8/d2c/f4c [816482,63799] 0 2024-02-19T19:06:19.311 INFO:tasks.workunit.client.0.smithi088.stdout:6/300: chown d5/d7/l51 2479 1 2024-02-19T19:06:19.312 INFO:tasks.workunit.client.0.smithi088.stdout:6/301: dread - d5/d7/f4b zero size 2024-02-19T19:06:19.313 INFO:tasks.workunit.client.0.smithi088.stdout:6/302: truncate d5/d19/f4e 951350 0 2024-02-19T19:06:19.314 INFO:tasks.workunit.client.0.smithi088.stdout:6/303: chown d5/d7/c48 3861201 1 2024-02-19T19:06:19.314 INFO:tasks.workunit.client.0.smithi088.stdout:0/250: symlink d13/d24/d25/l53 0 2024-02-19T19:06:19.315 INFO:tasks.workunit.client.0.smithi088.stdout:8/262: creat d9/df/d1c/d2d/f4b x:0 0 0 2024-02-19T19:06:19.316 INFO:tasks.workunit.client.0.smithi088.stdout:8/263: write d9/df/d1c/f29 [198554,99714] 0 2024-02-19T19:06:19.319 INFO:tasks.workunit.client.0.smithi088.stdout:1/394: rmdir d6/d21/d6a/d7d/d7e/d80/d6e 39 2024-02-19T19:06:19.329 INFO:tasks.workunit.client.0.smithi088.stdout:1/395: dread d6/f1b [0,4194304] 0 2024-02-19T19:06:19.329 INFO:tasks.workunit.client.0.smithi088.stdout:0/251: mknod d13/d24/c54 0 2024-02-19T19:06:19.330 INFO:tasks.workunit.client.0.smithi088.stdout:2/206: getdents d3/df 0 2024-02-19T19:06:19.335 INFO:tasks.workunit.client.0.smithi088.stdout:7/308: link d14/d1d/d3c/d43/c67 d14/d27/d49/c6f 0 2024-02-19T19:06:19.343 INFO:tasks.workunit.client.0.smithi088.stdout:7/309: dwrite d14/f3e [0,4194304] 0 2024-02-19T19:06:19.344 INFO:tasks.workunit.client.0.smithi088.stdout:1/396: dread d6/d10/d60/d29/f37 [0,4194304] 0 2024-02-19T19:06:19.347 INFO:tasks.workunit.client.0.smithi088.stdout:3/382: link d4/f6b d4/d1a/d1e/d74/f8a 0 2024-02-19T19:06:19.348 INFO:tasks.workunit.client.0.smithi088.stdout:6/304: write d5/d8/f3f [1862488,58541] 0 2024-02-19T19:06:19.350 INFO:tasks.workunit.client.0.smithi088.stdout:0/252: creat d13/d24/d25/d2a/d3c/f55 x:0 0 0 2024-02-19T19:06:19.354 INFO:tasks.workunit.client.0.smithi088.stdout:2/207: symlink d3/l45 0 2024-02-19T19:06:19.354 INFO:tasks.workunit.client.0.smithi088.stdout:2/208: fsync d3/f31 0 2024-02-19T19:06:19.355 INFO:tasks.workunit.client.0.smithi088.stdout:2/209: read - d3/d14/f19 zero size 2024-02-19T19:06:19.358 INFO:tasks.workunit.client.0.smithi088.stdout:8/264: rename l4 to d9/df/l4c 0 2024-02-19T19:06:19.359 INFO:tasks.workunit.client.0.smithi088.stdout:7/310: chown d14/f60 150502 1 2024-02-19T19:06:19.364 INFO:tasks.workunit.client.0.smithi088.stdout:2/210: unlink d3/d1f/f3a 0 2024-02-19T19:06:19.365 INFO:tasks.workunit.client.0.smithi088.stdout:7/311: dwrite fd [0,4194304] 0 2024-02-19T19:06:19.366 INFO:tasks.workunit.client.0.smithi088.stdout:1/397: rename d6/d10/d60/l32 to d6/d10/l82 0 2024-02-19T19:06:19.371 INFO:tasks.workunit.client.0.smithi088.stdout:4/236: link d2/d8/f14 d2/d8/f48 0 2024-02-19T19:06:19.373 INFO:tasks.workunit.client.0.smithi088.stdout:4/237: read - d2/d35/d39/d3a/f42 zero size 2024-02-19T19:06:19.374 INFO:tasks.workunit.client.0.smithi088.stdout:7/312: unlink d14/f16 0 2024-02-19T19:06:19.377 INFO:tasks.workunit.client.0.smithi088.stdout:0/253: rename d13/l17 to d13/d24/l56 0 2024-02-19T19:06:19.381 INFO:tasks.workunit.client.0.smithi088.stdout:3/383: truncate d4/d1a/d1e/d31/d32/f50 2546006 0 2024-02-19T19:06:19.382 INFO:tasks.workunit.client.0.smithi088.stdout:3/384: chown d4/f40 32650596 1 2024-02-19T19:06:19.382 INFO:tasks.workunit.client.0.smithi088.stdout:3/385: dread - d4/f28 zero size 2024-02-19T19:06:19.384 INFO:tasks.workunit.client.0.smithi088.stdout:1/398: rmdir d6/d21/d6a/d7d/d7e/d80/d71 39 2024-02-19T19:06:19.386 INFO:tasks.workunit.client.0.smithi088.stdout:7/313: rmdir d14/d1d/d3c/d43 39 2024-02-19T19:06:19.387 INFO:tasks.workunit.client.0.smithi088.stdout:7/314: fdatasync d14/f54 0 2024-02-19T19:06:19.388 INFO:tasks.workunit.client.0.smithi088.stdout:2/211: rename d3/d1b/c2f to d3/d14/c46 0 2024-02-19T19:06:19.396 INFO:tasks.workunit.client.0.smithi088.stdout:6/305: creat d5/d31/d25/f5d x:0 0 0 2024-02-19T19:06:19.404 INFO:tasks.workunit.client.0.smithi088.stdout:4/238: getdents d2/d32/d25/d3f 0 2024-02-19T19:06:19.405 INFO:tasks.workunit.client.0.smithi088.stdout:4/239: write d2/d32/f44 [955306,28194] 0 2024-02-19T19:06:19.406 INFO:tasks.workunit.client.0.smithi088.stdout:4/240: chown d2/d32/d9/l19 6 1 2024-02-19T19:06:19.407 INFO:tasks.workunit.client.0.smithi088.stdout:4/241: write d2/d32/fb [3930107,109579] 0 2024-02-19T19:06:19.414 INFO:tasks.workunit.client.0.smithi088.stdout:3/386: write d4/d22/f2c [554699,109419] 0 2024-02-19T19:06:19.416 INFO:tasks.workunit.client.0.smithi088.stdout:8/265: getdents d9/df 0 2024-02-19T19:06:19.416 INFO:tasks.workunit.client.0.smithi088.stdout:4/242: dwrite d2/d32/f44 [0,4194304] 0 2024-02-19T19:06:19.418 INFO:tasks.workunit.client.0.smithi088.stdout:8/266: write d9/df/f36 [266260,3923] 0 2024-02-19T19:06:19.419 INFO:tasks.workunit.client.0.smithi088.stdout:8/267: dread - d9/df/d1c/d2d/d47/f44 zero size 2024-02-19T19:06:19.426 INFO:tasks.workunit.client.0.smithi088.stdout:7/315: dread d14/d1d/d3c/d43/d5c/f68 [0,4194304] 0 2024-02-19T19:06:19.436 INFO:tasks.workunit.client.0.smithi088.stdout:0/254: rename d13/l2d to d13/l57 0 2024-02-19T19:06:19.441 INFO:tasks.workunit.client.0.smithi088.stdout:4/243: rename f0 to d2/d32/d25/d26/f49 0 2024-02-19T19:06:19.443 INFO:tasks.workunit.client.0.smithi088.stdout:4/244: write d2/d32/d9/f20 [632357,93652] 0 2024-02-19T19:06:19.445 INFO:tasks.workunit.client.0.smithi088.stdout:8/268: unlink c7 0 2024-02-19T19:06:19.447 INFO:tasks.workunit.client.0.smithi088.stdout:1/399: rmdir d6/d10/d60/d29/d63/d7b 0 2024-02-19T19:06:19.450 INFO:tasks.workunit.client.0.smithi088.stdout:6/306: dwrite d5/d19/d26/f44 [4194304,4194304] 0 2024-02-19T19:06:19.451 INFO:tasks.workunit.client.0.smithi088.stdout:7/316: mknod d14/d28/d3b/c70 0 2024-02-19T19:06:19.453 INFO:tasks.workunit.client.0.smithi088.stdout:7/317: read d14/f60 [2134352,14215] 0 2024-02-19T19:06:19.455 INFO:tasks.workunit.client.0.smithi088.stdout:3/387: dread d4/d12/f21 [0,4194304] 0 2024-02-19T19:06:19.455 INFO:tasks.workunit.client.0.smithi088.stdout:4/245: truncate d2/d35/d39/d3a/f42 207390 0 2024-02-19T19:06:19.455 INFO:tasks.workunit.client.0.smithi088.stdout:5/432: sync 2024-02-19T19:06:19.456 INFO:tasks.workunit.client.0.smithi088.stdout:9/311: sync 2024-02-19T19:06:19.457 INFO:tasks.workunit.client.0.smithi088.stdout:8/269: dwrite d9/df/d1c/f33 [0,4194304] 0 2024-02-19T19:06:19.457 INFO:tasks.workunit.client.0.smithi088.stdout:2/212: write d3/f6 [1708083,62151] 0 2024-02-19T19:06:19.458 INFO:tasks.workunit.client.0.smithi088.stdout:4/246: stat d2/d32/l27 0 2024-02-19T19:06:19.467 INFO:tasks.workunit.client.0.smithi088.stdout:7/318: rmdir d14/d28 39 2024-02-19T19:06:19.467 INFO:tasks.workunit.client.0.smithi088.stdout:2/213: dwrite d3/f39 [0,4194304] 0 2024-02-19T19:06:19.471 INFO:tasks.workunit.client.0.smithi088.stdout:3/388: write d4/d1a/f24 [516425,82406] 0 2024-02-19T19:06:19.488 INFO:tasks.workunit.client.0.smithi088.stdout:6/307: dread d5/de/f18 [0,4194304] 0 2024-02-19T19:06:19.488 INFO:tasks.workunit.client.0.smithi088.stdout:3/389: symlink d4/d1a/d1e/l8b 0 2024-02-19T19:06:19.490 INFO:tasks.workunit.client.0.smithi088.stdout:3/390: write d4/d1a/d1e/d74/f83 [895529,90341] 0 2024-02-19T19:06:19.491 INFO:tasks.workunit.client.0.smithi088.stdout:4/247: creat d2/d32/d25/d45/f4a x:0 0 0 2024-02-19T19:06:19.496 INFO:tasks.workunit.client.0.smithi088.stdout:2/214: dread d3/f40 [0,4194304] 0 2024-02-19T19:06:19.497 INFO:tasks.workunit.client.0.smithi088.stdout:2/215: chown d3/c29 2 1 2024-02-19T19:06:19.497 INFO:tasks.workunit.client.0.smithi088.stdout:0/255: getdents d13/d24/d25/d2a/d3c 0 2024-02-19T19:06:19.500 INFO:tasks.workunit.client.0.smithi088.stdout:7/319: mknod d14/d1d/d62/c71 0 2024-02-19T19:06:19.502 INFO:tasks.workunit.client.0.smithi088.stdout:6/308: symlink d5/d31/l5e 0 2024-02-19T19:06:19.503 INFO:tasks.workunit.client.0.smithi088.stdout:6/309: read d5/d7/f15 [365265,75581] 0 2024-02-19T19:06:19.505 INFO:tasks.workunit.client.0.smithi088.stdout:5/433: rename d4/d7/f31 to d4/d1a/f90 0 2024-02-19T19:06:19.506 INFO:tasks.workunit.client.0.smithi088.stdout:3/391: creat d4/d1a/d1e/d31/d32/f8c x:0 0 0 2024-02-19T19:06:19.508 INFO:tasks.workunit.client.0.smithi088.stdout:4/248: mkdir d2/d32/d25/d4b 0 2024-02-19T19:06:19.509 INFO:tasks.workunit.client.0.smithi088.stdout:4/249: fdatasync d2/d32/d25/f2e 0 2024-02-19T19:06:19.510 INFO:tasks.workunit.client.0.smithi088.stdout:8/270: rmdir d9/df/d1c/d2d/d47/d49 0 2024-02-19T19:06:19.516 INFO:tasks.workunit.client.0.smithi088.stdout:8/271: dwrite d9/f2e [0,4194304] 0 2024-02-19T19:06:19.519 INFO:tasks.workunit.client.0.smithi088.stdout:9/312: getdents d1/d21/d2f 0 2024-02-19T19:06:19.522 INFO:tasks.workunit.client.0.smithi088.stdout:5/434: symlink d4/d1a/d22/d57/d84/l91 0 2024-02-19T19:06:19.525 INFO:tasks.workunit.client.0.smithi088.stdout:5/435: chown d4/d1a/f55 627737 1 2024-02-19T19:06:19.525 INFO:tasks.workunit.client.0.smithi088.stdout:8/272: dwrite d9/df/d1c/f1f [0,4194304] 0 2024-02-19T19:06:19.526 INFO:tasks.workunit.client.0.smithi088.stdout:4/250: mknod d2/d35/c4c 0 2024-02-19T19:06:19.527 INFO:tasks.workunit.client.0.smithi088.stdout:1/400: getdents d6/d21/d6a/d7d/d7e/d80/d6e 0 2024-02-19T19:06:19.528 INFO:tasks.workunit.client.0.smithi088.stdout:5/436: truncate d4/d1a/d22/f4e 407778 0 2024-02-19T19:06:19.529 INFO:tasks.workunit.client.0.smithi088.stdout:2/216: dread d3/d1f/f3f [0,4194304] 0 2024-02-19T19:06:19.529 INFO:tasks.workunit.client.0.smithi088.stdout:4/251: readlink d2/d35/d39/d3a/l3c 0 2024-02-19T19:06:19.531 INFO:tasks.workunit.client.0.smithi088.stdout:2/217: dread - d3/d14/f2b zero size 2024-02-19T19:06:19.531 INFO:tasks.workunit.client.0.smithi088.stdout:5/437: truncate d4/d1a/f54 986112 0 2024-02-19T19:06:19.531 INFO:tasks.workunit.client.0.smithi088.stdout:9/313: unlink d1/d13/l1d 0 2024-02-19T19:06:19.532 INFO:tasks.workunit.client.0.smithi088.stdout:9/314: rename d1/d8 to d1/d8/d62 22 2024-02-19T19:06:19.533 INFO:tasks.workunit.client.0.smithi088.stdout:5/438: write d4/d1a/d22/f3d [1663216,89226] 0 2024-02-19T19:06:19.535 INFO:tasks.workunit.client.0.smithi088.stdout:9/315: write d1/d13/f12 [2626847,88188] 0 2024-02-19T19:06:19.538 INFO:tasks.workunit.client.0.smithi088.stdout:4/252: symlink d2/d32/d25/l4d 0 2024-02-19T19:06:19.543 INFO:tasks.workunit.client.0.smithi088.stdout:5/439: creat d4/d7/f92 x:0 0 0 2024-02-19T19:06:19.544 INFO:tasks.workunit.client.0.smithi088.stdout:5/440: truncate d4/d1a/f7c 4894981 0 2024-02-19T19:06:19.546 INFO:tasks.workunit.client.0.smithi088.stdout:6/310: link d5/d7/f14 d5/d19/f5f 0 2024-02-19T19:06:19.547 INFO:tasks.workunit.client.0.smithi088.stdout:9/316: readlink d1/d21/l32 0 2024-02-19T19:06:19.547 INFO:tasks.workunit.client.0.smithi088.stdout:6/311: chown d5/d19/f41 1699601910 1 2024-02-19T19:06:19.548 INFO:tasks.workunit.client.0.smithi088.stdout:6/312: fsync d5/ff 0 2024-02-19T19:06:19.550 INFO:tasks.workunit.client.0.smithi088.stdout:6/313: truncate d5/d31/d25/f5d 418278 0 2024-02-19T19:06:19.556 INFO:tasks.workunit.client.0.smithi088.stdout:4/253: mkdir d2/d32/d25/d26/d4e 0 2024-02-19T19:06:19.567 INFO:tasks.workunit.client.0.smithi088.stdout:7/320: link d14/l37 d14/d28/l72 0 2024-02-19T19:06:19.572 INFO:tasks.workunit.client.0.smithi088.stdout:4/254: creat d2/d35/f4f x:0 0 0 2024-02-19T19:06:19.573 INFO:tasks.workunit.client.0.smithi088.stdout:4/255: chown d2/d35/c4c 6493706 1 2024-02-19T19:06:19.574 INFO:tasks.workunit.client.0.smithi088.stdout:1/401: link d6/ce d6/d10/d60/c83 0 2024-02-19T19:06:19.578 INFO:tasks.workunit.client.0.smithi088.stdout:9/317: unlink d1/d21/d2d/d23/c29 0 2024-02-19T19:06:19.583 INFO:tasks.workunit.client.0.smithi088.stdout:5/441: rename d4/d61/d65/f86 to d4/f93 0 2024-02-19T19:06:19.586 INFO:tasks.workunit.client.0.smithi088.stdout:1/402: creat d6/d21/f84 x:0 0 0 2024-02-19T19:06:19.588 INFO:tasks.workunit.client.0.smithi088.stdout:4/256: dwrite d2/d32/d25/f41 [12582912,4194304] 0 2024-02-19T19:06:19.590 INFO:tasks.workunit.client.0.smithi088.stdout:9/318: dread d1/d13/f33 [0,4194304] 0 2024-02-19T19:06:19.591 INFO:tasks.workunit.client.0.smithi088.stdout:9/319: readlink d1/d8/l22 0 2024-02-19T19:06:19.606 INFO:tasks.workunit.client.0.smithi088.stdout:9/320: dwrite d1/d21/d2d/f16 [0,4194304] 0 2024-02-19T19:06:19.613 INFO:tasks.workunit.client.0.smithi088.stdout:3/392: write d4/d12/f30 [113746,109104] 0 2024-02-19T19:06:19.614 INFO:tasks.workunit.client.0.smithi088.stdout:3/393: stat d4/d12/d33/c3e 0 2024-02-19T19:06:19.619 INFO:tasks.workunit.client.0.smithi088.stdout:7/321: rename d14/d28/d3b/c70 to d14/c73 0 2024-02-19T19:06:19.623 INFO:tasks.workunit.client.0.smithi088.stdout:8/273: truncate d9/df/d1c/f1b 12754025 0 2024-02-19T19:06:19.624 INFO:tasks.workunit.client.0.smithi088.stdout:8/274: stat f5 0 2024-02-19T19:06:19.624 INFO:tasks.workunit.client.0.smithi088.stdout:8/275: chown d9/df/d3d 982 1 2024-02-19T19:06:19.627 INFO:tasks.workunit.client.0.smithi088.stdout:3/394: chown d4/d12/l2a 8538 1 2024-02-19T19:06:19.629 INFO:tasks.workunit.client.0.smithi088.stdout:1/403: link d6/d10/l74 d6/d10/l85 0 2024-02-19T19:06:19.630 INFO:tasks.workunit.client.0.smithi088.stdout:7/322: chown d14/f30 81108 1 2024-02-19T19:06:19.631 INFO:tasks.workunit.client.0.smithi088.stdout:6/314: write d5/d7/f15 [679981,10609] 0 2024-02-19T19:06:19.632 INFO:tasks.workunit.client.0.smithi088.stdout:6/315: readlink d5/d7/d32/l3e 0 2024-02-19T19:06:19.633 INFO:tasks.workunit.client.0.smithi088.stdout:6/316: write d5/d8/f3f [2519824,130526] 0 2024-02-19T19:06:19.633 INFO:tasks.workunit.client.0.smithi088.stdout:9/321: rename d1/d21/d2d/d18/f3b to d1/d21/d5a/f63 0 2024-02-19T19:06:19.634 INFO:tasks.workunit.client.0.smithi088.stdout:4/257: rmdir d2 39 2024-02-19T19:06:19.636 INFO:tasks.workunit.client.0.smithi088.stdout:5/442: getdents d4/d1a/d22/d57/d84 0 2024-02-19T19:06:19.638 INFO:tasks.workunit.client.0.smithi088.stdout:3/395: mkdir d4/d22/d7a/d8d 0 2024-02-19T19:06:19.639 INFO:tasks.workunit.client.0.smithi088.stdout:1/404: mknod d6/d21/d6a/d4a/c86 0 2024-02-19T19:06:19.646 INFO:tasks.workunit.client.0.smithi088.stdout:6/317: mkdir d5/de/d60 0 2024-02-19T19:06:19.647 INFO:tasks.workunit.client.0.smithi088.stdout:6/318: read - d5/d31/f59 zero size 2024-02-19T19:06:19.649 INFO:tasks.workunit.client.0.smithi088.stdout:6/319: dread d5/f47 [0,4194304] 0 2024-02-19T19:06:19.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.630+0000 7f95247dc700 1 -- 172.21.15.88:0/3792125569 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f951c197260 con 0x7f951c0ffd60 2024-02-19T19:06:19.650 INFO:tasks.workunit.client.0.smithi088.stdout:7/323: write d14/f2a [1056116,37399] 0 2024-02-19T19:06:19.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.630+0000 7f95137fe700 1 -- 172.21.15.88:0/3792125569 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f950c05a1d0 con 0x7f951c0ffd60 2024-02-19T19:06:19.651 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:06:19.651 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:06:19.651 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:06:19.651 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:06:19.651 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:06:19.651 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:06:19.651 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:06:19.651 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:06:19.651 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:06:19.651 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:06:19.651 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:06:19.652 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:06:19.652 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:06:19.652 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:06:19.652 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-02-19T19:06:19.652 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:06:19.652 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:06:19.652 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:06:19.653 INFO:tasks.workunit.client.0.smithi088.stdout:7/324: fsync d14/d1d/f38 0 2024-02-19T19:06:19.653 INFO:tasks.workunit.client.0.smithi088.stdout:5/443: rename d4/d1a/d41/c72 to d4/d1a/d22/d57/c94 0 2024-02-19T19:06:19.653 INFO:tasks.workunit.client.0.smithi088.stdout:5/444: stat d4/l69 0 2024-02-19T19:06:19.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.633+0000 7f95247dc700 1 -- 172.21.15.88:0/3792125569 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da30 msgr2=0x7f950806fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:19.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.633+0000 7f95247dc700 1 --2- 172.21.15.88:0/3792125569 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da30 0x7f950806fee0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9518006010 tx=0x7f951801a040 comp rx=0 tx=0).stop 2024-02-19T19:06:19.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.633+0000 7f95247dc700 1 -- 172.21.15.88:0/3792125569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f951c0ffd60 msgr2=0x7f951c196440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:19.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.633+0000 7f95247dc700 1 --2- 172.21.15.88:0/3792125569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f951c0ffd60 0x7f951c196440 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f950c00d8d0 tx=0x7f950c00dbe0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.633+0000 7f95247dc700 1 -- 172.21.15.88:0/3792125569 shutdown_connections 2024-02-19T19:06:19.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.633+0000 7f95247dc700 1 --2- 172.21.15.88:0/3792125569 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f950806da30 0x7f950806fee0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.633+0000 7f95247dc700 1 --2- 172.21.15.88:0/3792125569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f951c0ff3c0 0x7f951c195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.633+0000 7f95247dc700 1 --2- 172.21.15.88:0/3792125569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f951c0ffd60 0x7f951c196440 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.633+0000 7f95247dc700 1 -- 172.21.15.88:0/3792125569 >> 172.21.15.88:0/3792125569 conn(0x7f951c0fac80 msgr2=0x7f951c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:19.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.633+0000 7f95247dc700 1 -- 172.21.15.88:0/3792125569 shutdown_connections 2024-02-19T19:06:19.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.633+0000 7f95247dc700 1 -- 172.21.15.88:0/3792125569 wait complete. 2024-02-19T19:06:19.655 INFO:tasks.workunit.client.0.smithi088.stdout:8/276: creat d9/df/d1c/d2d/d47/f4d x:0 0 0 2024-02-19T19:06:19.656 INFO:tasks.workunit.client.0.smithi088.stdout:1/405: mknod d6/d10/d60/c87 0 2024-02-19T19:06:19.660 INFO:tasks.workunit.client.0.smithi088.stdout:4/258: unlink d2/d32/d25/f2e 0 2024-02-19T19:06:19.663 INFO:tasks.workunit.client.0.smithi088.stdout:7/325: rmdir d14/d56 39 2024-02-19T19:06:19.668 INFO:tasks.workunit.client.0.smithi088.stdout:1/406: creat d6/d21/d6a/d7d/d7e/d80/d6e/f88 x:0 0 0 2024-02-19T19:06:19.670 INFO:tasks.workunit.client.0.smithi088.stdout:9/322: creat d1/d21/f64 x:0 0 0 2024-02-19T19:06:19.670 INFO:tasks.workunit.client.0.smithi088.stdout:9/323: stat d1/l46 0 2024-02-19T19:06:19.671 INFO:tasks.workunit.client.0.smithi088.stdout:9/324: chown d1/d8/d37/f44 8917 1 2024-02-19T19:06:19.672 INFO:tasks.workunit.client.0.smithi088.stdout:9/325: write d1/fc [1121469,102155] 0 2024-02-19T19:06:19.672 INFO:tasks.workunit.client.0.smithi088.stdout:6/320: mknod d5/c61 0 2024-02-19T19:06:19.673 INFO:tasks.workunit.client.0.smithi088.stdout:6/321: stat d5/d31 0 2024-02-19T19:06:19.674 INFO:tasks.workunit.client.0.smithi088.stdout:9/326: dread d1/d21/d5a/f63 [0,4194304] 0 2024-02-19T19:06:19.674 INFO:tasks.workunit.client.0.smithi088.stdout:5/445: write d4/d1a/f21 [603373,93136] 0 2024-02-19T19:06:19.675 INFO:tasks.workunit.client.0.smithi088.stdout:5/446: chown d4/d61 40 1 2024-02-19T19:06:19.676 INFO:tasks.workunit.client.0.smithi088.stdout:9/327: write d1/d21/f64 [880339,25060] 0 2024-02-19T19:06:19.677 INFO:tasks.workunit.client.0.smithi088.stdout:7/326: fdatasync d14/d1d/f39 0 2024-02-19T19:06:19.680 INFO:tasks.workunit.client.0.smithi088.stdout:3/396: creat d4/f8e x:0 0 0 2024-02-19T19:06:19.681 INFO:tasks.workunit.client.0.smithi088.stdout:8/277: rename d9/l11 to d9/df/d3d/l4e 0 2024-02-19T19:06:19.684 INFO:tasks.workunit.client.0.smithi088.stdout:4/259: dread d2/d8/f29 [0,4194304] 0 2024-02-19T19:06:19.684 INFO:tasks.workunit.client.0.smithi088.stdout:6/322: rmdir d5/d7/d32 39 2024-02-19T19:06:19.686 INFO:tasks.workunit.client.0.smithi088.stdout:5/447: creat d4/d1a/d41/f95 x:0 0 0 2024-02-19T19:06:19.686 INFO:tasks.workunit.client.0.smithi088.stdout:7/327: dwrite d14/f4f [0,4194304] 0 2024-02-19T19:06:19.688 INFO:tasks.workunit.client.0.smithi088.stdout:5/448: read d4/d7/f63 [4308404,68994] 0 2024-02-19T19:06:19.691 INFO:tasks.workunit.client.0.smithi088.stdout:1/407: rename d6/d21/d6a/d7d/d7e/d80 to d6/d89 0 2024-02-19T19:06:19.695 INFO:tasks.workunit.client.0.smithi088.stdout:3/397: truncate d4/d12/f21 4149791 0 2024-02-19T19:06:19.700 INFO:tasks.workunit.client.0.smithi088.stdout:8/278: rmdir d9 39 2024-02-19T19:06:19.704 INFO:tasks.workunit.client.0.smithi088.stdout:4/260: dread d2/d35/d39/d3a/f42 [0,4194304] 0 2024-02-19T19:06:19.706 INFO:tasks.workunit.client.0.smithi088.stdout:0/256: sync 2024-02-19T19:06:19.708 INFO:tasks.workunit.client.0.smithi088.stdout:5/449: write d4/d1a/f90 [1724733,29457] 0 2024-02-19T19:06:19.710 INFO:tasks.workunit.client.0.smithi088.stdout:5/450: read d4/d1a/d22/d57/d84/f89 [237821,17865] 0 2024-02-19T19:06:19.711 INFO:tasks.workunit.client.0.smithi088.stdout:5/451: stat d4/d1a/d22/d57/d84 0 2024-02-19T19:06:19.712 INFO:tasks.workunit.client.0.smithi088.stdout:9/328: rename d1/d13/f33 to d1/d8/d37/d54/f65 0 2024-02-19T19:06:19.716 INFO:tasks.workunit.client.0.smithi088.stdout:8/279: fdatasync d9/df/d1c/f33 0 2024-02-19T19:06:19.717 INFO:tasks.workunit.client.0.smithi088.stdout:6/323: symlink d5/d7/l62 0 2024-02-19T19:06:19.718 INFO:tasks.workunit.client.0.smithi088.stdout:7/328: dwrite d14/f1c [0,4194304] 0 2024-02-19T19:06:19.727 INFO:tasks.workunit.client.0.smithi088.stdout:1/408: unlink d6/d10/c6b 0 2024-02-19T19:06:19.733 INFO:tasks.workunit.client.0.smithi088.stdout:4/261: dwrite d2/d32/d25/f46 [0,4194304] 0 2024-02-19T19:06:19.735 INFO:tasks.workunit.client.0.smithi088.stdout:8/280: fsync f5 0 2024-02-19T19:06:19.737 INFO:tasks.workunit.client.0.smithi088.stdout:8/281: write d9/df/d1c/d2d/d47/f43 [1143756,18774] 0 2024-02-19T19:06:19.741 INFO:tasks.workunit.client.0.smithi088.stdout:3/398: dread d4/d1a/d23/f69 [0,4194304] 0 2024-02-19T19:06:19.745 INFO:tasks.workunit.client.0.smithi088.stdout:1/409: chown d6/d21/d6a 0 1 2024-02-19T19:06:19.746 INFO:tasks.workunit.client.0.smithi088.stdout:5/452: symlink d4/d1a/l96 0 2024-02-19T19:06:19.750 INFO:tasks.workunit.client.0.smithi088.stdout:6/324: unlink d5/de/c1c 0 2024-02-19T19:06:19.751 INFO:tasks.workunit.client.0.smithi088.stdout:6/325: write d5/f1e [772513,56675] 0 2024-02-19T19:06:19.755 INFO:tasks.workunit.client.0.smithi088.stdout:9/329: write d1/d21/d2d/d23/f26 [952970,49166] 0 2024-02-19T19:06:19.755 INFO:tasks.workunit.client.0.smithi088.stdout:0/257: creat d13/d24/d25/f58 x:0 0 0 2024-02-19T19:06:19.758 INFO:tasks.workunit.client.0.smithi088.stdout:3/399: creat d4/d1a/d1e/f8f x:0 0 0 2024-02-19T19:06:19.761 INFO:tasks.workunit.client.0.smithi088.stdout:1/410: readlink d6/l41 0 2024-02-19T19:06:19.763 INFO:tasks.workunit.client.0.smithi088.stdout:0/258: dwrite d13/d24/d25/d2a/d2f/f4f [0,4194304] 0 2024-02-19T19:06:19.765 INFO:tasks.workunit.client.0.smithi088.stdout:0/259: write d13/d24/f32 [416629,18945] 0 2024-02-19T19:06:19.768 INFO:tasks.workunit.client.0.smithi088.stdout:6/326: chown d5/d19/l5b 23 1 2024-02-19T19:06:19.769 INFO:tasks.workunit.client.0.smithi088.stdout:7/329: creat d14/f74 x:0 0 0 2024-02-19T19:06:19.772 INFO:tasks.workunit.client.0.smithi088.stdout:9/330: rename d1/d13/l17 to d1/d8/d37/l66 0 2024-02-19T19:06:19.775 INFO:tasks.workunit.client.0.smithi088.stdout:1/411: fsync d6/d89/d71/f7a 0 2024-02-19T19:06:19.776 INFO:tasks.workunit.client.0.smithi088.stdout:1/412: write d6/d89/d71/f35 [444193,16162] 0 2024-02-19T19:06:19.782 INFO:tasks.workunit.client.0.smithi088.stdout:7/330: unlink d14/d1d/d3c/d43/c67 0 2024-02-19T19:06:19.786 INFO:tasks.workunit.client.0.smithi088.stdout:3/400: creat d4/d1a/d7f/d86/f90 x:0 0 0 2024-02-19T19:06:19.789 INFO:tasks.workunit.client.0.smithi088.stdout:5/453: link d4/d61/d65/f81 d4/f97 0 2024-02-19T19:06:19.791 INFO:tasks.workunit.client.0.smithi088.stdout:1/413: mknod d6/d89/d6e/c8a 0 2024-02-19T19:06:19.791 INFO:tasks.workunit.client.0.smithi088.stdout:1/414: write d6/d10/f45 [645098,9548] 0 2024-02-19T19:06:19.794 INFO:tasks.workunit.client.0.smithi088.stdout:6/327: fdatasync d5/d7/d32/f53 0 2024-02-19T19:06:19.798 INFO:tasks.workunit.client.0.smithi088.stdout:9/331: rename d1/d21/d2f/c3c to d1/d8/d3e/d4e/c67 0 2024-02-19T19:06:19.799 INFO:tasks.workunit.client.0.smithi088.stdout:3/401: symlink d4/d1a/d1e/d31/d32/d55/l91 0 2024-02-19T19:06:19.800 INFO:tasks.workunit.client.0.smithi088.stdout:3/402: fsync d4/f14 0 2024-02-19T19:06:19.801 INFO:tasks.workunit.client.0.smithi088.stdout:5/454: creat d4/d75/f98 x:0 0 0 2024-02-19T19:06:19.802 INFO:tasks.workunit.client.0.smithi088.stdout:1/415: creat d6/d21/d6a/d7d/f8b x:0 0 0 2024-02-19T19:06:19.802 INFO:tasks.workunit.client.0.smithi088.stdout:6/328: dwrite d5/d7/f4f [0,4194304] 0 2024-02-19T19:06:19.804 INFO:tasks.workunit.client.0.smithi088.stdout:6/329: fdatasync d5/d19/d26/f27 0 2024-02-19T19:06:19.804 INFO:tasks.workunit.client.0.smithi088.stdout:1/416: dread - d6/d89/d71/f7a zero size 2024-02-19T19:06:19.805 INFO:tasks.workunit.client.0.smithi088.stdout:0/260: link d13/l2e d13/d24/d25/d2a/l59 0 2024-02-19T19:06:19.810 INFO:tasks.workunit.client.0.smithi088.stdout:9/332: truncate d1/d8/d4a/f5e 166846 0 2024-02-19T19:06:19.813 INFO:tasks.workunit.client.0.smithi088.stdout:6/330: dwrite d5/d19/f4e [0,4194304] 0 2024-02-19T19:06:19.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.796+0000 7fdd1eddd700 1 -- 172.21.15.88:0/358146976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd18100e60 msgr2=0x7fdd18104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:19.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.796+0000 7fdd1eddd700 1 --2- 172.21.15.88:0/358146976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd18100e60 0x7fdd18104e30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009910 tx=0x7fdd0c009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:19.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.796+0000 7fdd1eddd700 1 -- 172.21.15.88:0/358146976 shutdown_connections 2024-02-19T19:06:19.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.796+0000 7fdd1eddd700 1 --2- 172.21.15.88:0/358146976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd18100e60 0x7fdd18104e30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.796+0000 7fdd1eddd700 1 --2- 172.21.15.88:0/358146976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd181004c0 0x7fdd18100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.796+0000 7fdd1eddd700 1 -- 172.21.15.88:0/358146976 >> 172.21.15.88:0/358146976 conn(0x7fdd180fbd80 msgr2=0x7fdd180fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:19.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.796+0000 7fdd1eddd700 1 -- 172.21.15.88:0/358146976 shutdown_connections 2024-02-19T19:06:19.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.796+0000 7fdd1eddd700 1 -- 172.21.15.88:0/358146976 wait complete. 2024-02-19T19:06:19.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.797+0000 7fdd1eddd700 1 Processor -- start 2024-02-19T19:06:19.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.797+0000 7fdd1eddd700 1 -- start start 2024-02-19T19:06:19.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.798+0000 7fdd1eddd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd181004c0 0x7fdd18198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:19.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.798+0000 7fdd1eddd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd18198880 0x7fdd1819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:19.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.798+0000 7fdd1eddd700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd18198e90 con 0x7fdd18198880 2024-02-19T19:06:19.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.798+0000 7fdd1eddd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd18199000 con 0x7fdd181004c0 2024-02-19T19:06:19.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.798+0000 7fdd1cb79700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd181004c0 0x7fdd18198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:19.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.798+0000 7fdd1cb79700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd181004c0 0x7fdd18198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:60642/0 (socket says 172.21.15.88:60642) 2024-02-19T19:06:19.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.798+0000 7fdd1cb79700 1 -- 172.21.15.88:0/3040042757 learned_addr learned my addr 172.21.15.88:0/3040042757 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:19.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.799+0000 7fdd17fff700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd18198880 0x7fdd1819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:19.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.799+0000 7fdd1cb79700 1 -- 172.21.15.88:0/3040042757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd18198880 msgr2=0x7fdd1819ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:19.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.799+0000 7fdd1cb79700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd18198880 0x7fdd1819ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:19.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.799+0000 7fdd1cb79700 1 -- 172.21.15.88:0/3040042757 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0c0095f0 con 0x7fdd181004c0 2024-02-19T19:06:19.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.799+0000 7fdd17fff700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd18198880 0x7fdd1819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:06:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.800+0000 7fdd1cb79700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd181004c0 0x7fdd18198340 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400e920 tx=0x7fdd0400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.800+0000 7fdd15ffb700 1 -- 172.21.15.88:0/3040042757 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd0400cba0 con 0x7fdd181004c0 2024-02-19T19:06:19.821 INFO:tasks.workunit.client.0.smithi088.stdout:9/333: dwrite d1/d8/f28 [0,4194304] 0 2024-02-19T19:06:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.800+0000 7fdd15ffb700 1 -- 172.21.15.88:0/3040042757 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd0400cd00 con 0x7fdd181004c0 2024-02-19T19:06:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.800+0000 7fdd15ffb700 1 -- 172.21.15.88:0/3040042757 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd04018770 con 0x7fdd181004c0 2024-02-19T19:06:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.800+0000 7fdd1eddd700 1 -- 172.21.15.88:0/3040042757 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1819d2f0 con 0x7fdd181004c0 2024-02-19T19:06:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.801+0000 7fdd1eddd700 1 -- 172.21.15.88:0/3040042757 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1819d7e0 con 0x7fdd181004c0 2024-02-19T19:06:19.823 INFO:tasks.workunit.client.0.smithi088.stdout:3/403: rename d4/d12/f30 to d4/d1a/d1e/f92 0 2024-02-19T19:06:19.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.803+0000 7fdd1eddd700 1 -- 172.21.15.88:0/3040042757 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd181045b0 con 0x7fdd181004c0 2024-02-19T19:06:19.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.804+0000 7fdd15ffb700 1 -- 172.21.15.88:0/3040042757 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fdd04010430 con 0x7fdd181004c0 2024-02-19T19:06:19.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.804+0000 7fdd15ffb700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd0806da20 0x7fdd0806fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:19.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.804+0000 7fdd15ffb700 1 -- 172.21.15.88:0/3040042757 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fdd04014070 con 0x7fdd181004c0 2024-02-19T19:06:19.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.804+0000 7fdd17fff700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd0806da20 0x7fdd0806fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:19.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.805+0000 7fdd17fff700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd0806da20 0x7fdd0806fed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0098e0 tx=0x7fdd0c00b8a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:19.828 INFO:tasks.workunit.client.0.smithi088.stdout:1/417: dread d6/d10/d60/d29/f30 [0,4194304] 0 2024-02-19T19:06:19.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:19.810+0000 7fdd15ffb700 1 -- 172.21.15.88:0/3040042757 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd04052d20 con 0x7fdd181004c0 2024-02-19T19:06:19.838 INFO:tasks.workunit.client.0.smithi088.stdout:5/455: symlink d4/l99 0 2024-02-19T19:06:19.839 INFO:tasks.workunit.client.0.smithi088.stdout:5/456: fdatasync d4/f5 0 2024-02-19T19:06:19.845 INFO:tasks.workunit.client.0.smithi088.stdout:5/457: dwrite d4/d7/f15 [4194304,4194304] 0 2024-02-19T19:06:19.849 INFO:tasks.workunit.client.0.smithi088.stdout:9/334: rmdir d1/d21/d2d/d18/d1f 39 2024-02-19T19:06:19.851 INFO:tasks.workunit.client.0.smithi088.stdout:9/335: fdatasync d1/d21/d2d/f16 0 2024-02-19T19:06:19.851 INFO:tasks.workunit.client.0.smithi088.stdout:5/458: dwrite d4/f5 [0,4194304] 0 2024-02-19T19:06:19.852 INFO:tasks.workunit.client.0.smithi088.stdout:5/459: chown d4/d7/f63 1043954449 1 2024-02-19T19:06:19.856 INFO:tasks.workunit.client.0.smithi088.stdout:3/404: creat d4/d12/d33/d58/f93 x:0 0 0 2024-02-19T19:06:19.860 INFO:tasks.workunit.client.0.smithi088.stdout:5/460: dwrite d4/d1a/f46 [0,4194304] 0 2024-02-19T19:06:19.861 INFO:tasks.workunit.client.0.smithi088.stdout:5/461: chown d4/d7/c5a 0 1 2024-02-19T19:06:19.862 INFO:tasks.workunit.client.0.smithi088.stdout:7/331: getdents d14/d1d/d62 0 2024-02-19T19:06:19.863 INFO:tasks.workunit.client.0.smithi088.stdout:3/405: dread d4/fd [4194304,4194304] 0 2024-02-19T19:06:19.865 INFO:tasks.workunit.client.0.smithi088.stdout:1/418: mknod d6/d10/d60/c8c 0 2024-02-19T19:06:19.866 INFO:tasks.workunit.client.0.smithi088.stdout:1/419: write d6/d21/d6a/d4a/f56 [2609995,8642] 0 2024-02-19T19:06:19.870 INFO:tasks.workunit.client.0.smithi088.stdout:6/331: creat d5/d8/d3a/f63 x:0 0 0 2024-02-19T19:06:19.875 INFO:tasks.workunit.client.0.smithi088.stdout:0/261: rename d13/d24 to d13/d46/d5a 0 2024-02-19T19:06:19.875 INFO:tasks.workunit.client.0.smithi088.stdout:0/262: readlink lf 0 2024-02-19T19:06:19.876 INFO:tasks.workunit.client.0.smithi088.stdout:0/263: dread - d13/d46/d5a/d25/d2a/d2f/f50 zero size 2024-02-19T19:06:19.877 INFO:tasks.workunit.client.0.smithi088.stdout:0/264: truncate d13/d46/d5a/d25/f49 4480359 0 2024-02-19T19:06:19.882 INFO:tasks.workunit.client.0.smithi088.stdout:5/462: mkdir d4/d61/d65/d9a 0 2024-02-19T19:06:19.889 INFO:tasks.workunit.client.0.smithi088.stdout:7/332: dwrite fc [0,4194304] 0 2024-02-19T19:06:19.890 INFO:tasks.workunit.client.0.smithi088.stdout:1/420: mknod d6/d21/d6a/c8d 0 2024-02-19T19:06:19.891 INFO:tasks.workunit.client.0.smithi088.stdout:7/333: write d14/f19 [242835,42357] 0 2024-02-19T19:06:19.893 INFO:tasks.workunit.client.0.smithi088.stdout:9/336: symlink d1/d21/d2d/d18/l68 0 2024-02-19T19:06:19.894 INFO:tasks.workunit.client.0.smithi088.stdout:9/337: dread - d1/d8/d3e/f45 zero size 2024-02-19T19:06:19.894 INFO:tasks.workunit.client.0.smithi088.stdout:9/338: stat d1/d21/l5b 0 2024-02-19T19:06:19.897 INFO:tasks.workunit.client.0.smithi088.stdout:5/463: creat d4/d7/f9b x:0 0 0 2024-02-19T19:06:19.898 INFO:tasks.workunit.client.0.smithi088.stdout:3/406: creat d4/d87/f94 x:0 0 0 2024-02-19T19:06:19.899 INFO:tasks.workunit.client.0.smithi088.stdout:5/464: truncate d4/d75/f80 1887896 0 2024-02-19T19:06:19.900 INFO:tasks.workunit.client.0.smithi088.stdout:5/465: write d4/d61/f73 [554129,52571] 0 2024-02-19T19:06:19.902 INFO:tasks.workunit.client.0.smithi088.stdout:1/421: rename d6/d21/d6a/f39 to d6/d21/d6a/f8e 0 2024-02-19T19:06:19.903 INFO:tasks.workunit.client.0.smithi088.stdout:7/334: mkdir d14/d1d/d3c/d43/d5c/d75 0 2024-02-19T19:06:19.903 INFO:tasks.workunit.client.0.smithi088.stdout:7/335: dread - d14/d1d/f5e zero size 2024-02-19T19:06:19.905 INFO:tasks.workunit.client.0.smithi088.stdout:9/339: symlink d1/d8/d3e/l69 0 2024-02-19T19:06:19.906 INFO:tasks.workunit.client.0.smithi088.stdout:3/407: read d4/fd [6791854,47560] 0 2024-02-19T19:06:19.908 INFO:tasks.workunit.client.0.smithi088.stdout:3/408: write d4/d1a/d1e/d31/d32/f8c [914419,40345] 0 2024-02-19T19:06:19.909 INFO:tasks.workunit.client.0.smithi088.stdout:3/409: chown d4/d1a/d7f/d86/f90 997 1 2024-02-19T19:06:19.910 INFO:tasks.workunit.client.0.smithi088.stdout:6/332: mkdir d5/de/d64 0 2024-02-19T19:06:19.911 INFO:tasks.workunit.client.0.smithi088.stdout:3/410: write d4/f2d [550389,83583] 0 2024-02-19T19:06:19.912 INFO:tasks.workunit.client.0.smithi088.stdout:3/411: fdatasync d4/f14 0 2024-02-19T19:06:19.913 INFO:tasks.workunit.client.0.smithi088.stdout:3/412: stat d4/d1a/d1e/d31/d32/d55/l91 0 2024-02-19T19:06:19.914 INFO:tasks.workunit.client.0.smithi088.stdout:3/413: write d4/d1a/f24 [479234,20588] 0 2024-02-19T19:06:19.915 INFO:tasks.workunit.client.0.smithi088.stdout:5/466: mkdir d4/d1a/d9c 0 2024-02-19T19:06:19.916 INFO:tasks.workunit.client.0.smithi088.stdout:3/414: chown d4/d1a/d1e/f2f 22826479 1 2024-02-19T19:06:19.919 INFO:tasks.workunit.client.0.smithi088.stdout:2/218: sync 2024-02-19T19:06:19.920 INFO:tasks.workunit.client.0.smithi088.stdout:7/336: mkdir d14/d1d/d3c/d43/d5c/d75/d76 0 2024-02-19T19:06:19.921 INFO:tasks.workunit.client.0.smithi088.stdout:7/337: readlink d14/d27/l50 0 2024-02-19T19:06:19.921 INFO:tasks.workunit.client.0.smithi088.stdout:2/219: write d3/d14/f35 [971916,95111] 0 2024-02-19T19:06:19.925 INFO:tasks.workunit.client.0.smithi088.stdout:7/338: dread fc [0,4194304] 0 2024-02-19T19:06:19.926 INFO:tasks.workunit.client.0.smithi088.stdout:1/422: dread d6/f17 [4194304,4194304] 0 2024-02-19T19:06:19.928 INFO:tasks.workunit.client.0.smithi088.stdout:0/265: getdents d13/d46/d5a 0 2024-02-19T19:06:19.931 INFO:tasks.workunit.client.0.smithi088.stdout:0/266: fdatasync d13/d46/d5a/d25/d2a/d3c/f55 0 2024-02-19T19:06:19.931 INFO:tasks.workunit.client.0.smithi088.stdout:3/415: creat d4/d1a/d23/f95 x:0 0 0 2024-02-19T19:06:19.938 INFO:tasks.workunit.client.0.smithi088.stdout:2/220: unlink d3/d1b/c44 0 2024-02-19T19:06:19.939 INFO:tasks.workunit.client.0.smithi088.stdout:6/333: chown d5/de/d60 47 1 2024-02-19T19:06:19.941 INFO:tasks.workunit.client.0.smithi088.stdout:6/334: truncate d5/d31/d25/f49 4201400 0 2024-02-19T19:06:19.941 INFO:tasks.workunit.client.0.smithi088.stdout:1/423: dread d6/d21/f36 [0,4194304] 0 2024-02-19T19:06:19.942 INFO:tasks.workunit.client.0.smithi088.stdout:6/335: write d5/d8/d2c/f40 [3502638,86143] 0 2024-02-19T19:06:19.945 INFO:tasks.workunit.client.0.smithi088.stdout:0/267: mkdir d13/d46/d5b 0 2024-02-19T19:06:19.954 INFO:tasks.workunit.client.0.smithi088.stdout:2/221: symlink d3/d1b/l47 0 2024-02-19T19:06:19.954 INFO:tasks.workunit.client.0.smithi088.stdout:2/222: fsync d3/d14/f19 0 2024-02-19T19:06:19.961 INFO:tasks.workunit.client.0.smithi088.stdout:2/223: dwrite d3/d1b/f2e [0,4194304] 0 2024-02-19T19:06:19.966 INFO:tasks.workunit.client.0.smithi088.stdout:0/268: unlink d13/d46/d5a/f32 0 2024-02-19T19:06:19.968 INFO:tasks.workunit.client.0.smithi088.stdout:2/224: dwrite d3/f3b [0,4194304] 0 2024-02-19T19:06:19.971 INFO:tasks.workunit.client.0.smithi088.stdout:2/225: dread d3/f23 [0,4194304] 0 2024-02-19T19:06:19.971 INFO:tasks.workunit.client.0.smithi088.stdout:3/416: link d4/f8e d4/d1a/d1e/d74/f96 0 2024-02-19T19:06:19.975 INFO:tasks.workunit.client.0.smithi088.stdout:5/467: rename d4/d1a/f46 to d4/f9d 0 2024-02-19T19:06:19.977 INFO:tasks.workunit.client.0.smithi088.stdout:5/468: stat d4/d1a/l96 0 2024-02-19T19:06:19.978 INFO:tasks.workunit.client.0.smithi088.stdout:1/424: mknod d6/d21/d64/c8f 0 2024-02-19T19:06:19.979 INFO:tasks.workunit.client.0.smithi088.stdout:1/425: chown d6/d10/d60 600136 1 2024-02-19T19:06:19.980 INFO:tasks.workunit.client.0.smithi088.stdout:7/339: creat d14/f77 x:0 0 0 2024-02-19T19:06:19.982 INFO:tasks.workunit.client.0.smithi088.stdout:6/336: dwrite d5/d19/f5f [0,4194304] 0 2024-02-19T19:06:19.985 INFO:tasks.workunit.client.0.smithi088.stdout:2/226: mknod d3/d1f/c48 0 2024-02-19T19:06:19.986 INFO:tasks.workunit.client.0.smithi088.stdout:2/227: write d3/d1b/f42 [6322,19976] 0 2024-02-19T19:06:19.987 INFO:tasks.workunit.client.0.smithi088.stdout:2/228: stat d3/d14/c38 0 2024-02-19T19:06:19.988 INFO:tasks.workunit.client.0.smithi088.stdout:6/337: read d5/d31/d25/f5d [381894,55160] 0 2024-02-19T19:06:19.989 INFO:tasks.workunit.client.0.smithi088.stdout:5/469: creat d4/d1a/d41/d5e/f9e x:0 0 0 2024-02-19T19:06:20.002 INFO:tasks.workunit.client.0.smithi088.stdout:6/338: rename d5/d31/l5e to d5/d8/d3a/l65 0 2024-02-19T19:06:20.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:19 smithi132 ceph-mon[70676]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:20.005 INFO:tasks.workunit.client.0.smithi088.stdout:7/340: mknod d14/c78 0 2024-02-19T19:06:20.008 INFO:tasks.workunit.client.0.smithi088.stdout:3/417: truncate d4/d1a/d23/f5d 4880410 0 2024-02-19T19:06:20.009 INFO:tasks.workunit.client.0.smithi088.stdout:6/339: dwrite d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:20.011 INFO:tasks.workunit.client.0.smithi088.stdout:6/340: write d5/d19/d26/f27 [2783911,45093] 0 2024-02-19T19:06:20.012 INFO:tasks.workunit.client.0.smithi088.stdout:6/341: write d5/d19/d26/f44 [4081074,19646] 0 2024-02-19T19:06:20.012 INFO:tasks.workunit.client.0.smithi088.stdout:2/229: rmdir d3 39 2024-02-19T19:06:20.013 INFO:tasks.workunit.client.0.smithi088.stdout:0/269: creat d13/f5c x:0 0 0 2024-02-19T19:06:20.015 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:19 smithi088 ceph-mon[88643]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:20.016 INFO:tasks.workunit.client.0.smithi088.stdout:7/341: creat d14/d1d/d3c/d43/d5c/f79 x:0 0 0 2024-02-19T19:06:20.018 INFO:tasks.workunit.client.0.smithi088.stdout:2/230: dread - d3/d1b/f3e zero size 2024-02-19T19:06:20.018 INFO:tasks.workunit.client.0.smithi088.stdout:2/231: readlink d3/df/l37 0 2024-02-19T19:06:20.020 INFO:tasks.workunit.client.0.smithi088.stdout:7/342: read d14/f2a [118790,72931] 0 2024-02-19T19:06:20.021 INFO:tasks.workunit.client.0.smithi088.stdout:7/343: stat d14/d27/c59 0 2024-02-19T19:06:20.022 INFO:tasks.workunit.client.0.smithi088.stdout:7/344: write d14/f19 [1116193,118956] 0 2024-02-19T19:06:20.023 INFO:tasks.workunit.client.0.smithi088.stdout:7/345: fsync fd 0 2024-02-19T19:06:20.024 INFO:tasks.workunit.client.0.smithi088.stdout:2/232: symlink d3/df/l49 0 2024-02-19T19:06:20.025 INFO:tasks.workunit.client.0.smithi088.stdout:6/342: creat d5/de/d64/f66 x:0 0 0 2024-02-19T19:06:20.029 INFO:tasks.workunit.client.0.smithi088.stdout:0/270: rename d13/f3a to d13/d46/d5a/d25/d2a/d3c/f5d 0 2024-02-19T19:06:20.048 INFO:tasks.workunit.client.0.smithi088.stdout:2/233: dread d3/f6 [0,4194304] 0 2024-02-19T19:06:20.051 INFO:tasks.workunit.client.0.smithi088.stdout:6/343: dread d5/d19/d26/f27 [0,4194304] 0 2024-02-19T19:06:20.052 INFO:tasks.workunit.client.0.smithi088.stdout:6/344: write d5/d8/f3f [985655,6257] 0 2024-02-19T19:06:20.052 INFO:tasks.workunit.client.0.smithi088.stdout:2/234: rename d3/d1f/f3f to d3/d1f/f4a 0 2024-02-19T19:06:20.053 INFO:tasks.workunit.client.0.smithi088.stdout:6/345: chown d5/d8/l1d 9805 1 2024-02-19T19:06:20.054 INFO:tasks.workunit.client.0.smithi088.stdout:2/235: write d3/d1b/f3e [441091,97800] 0 2024-02-19T19:06:20.061 INFO:tasks.workunit.client.0.smithi088.stdout:0/271: dread d13/d46/d5a/d25/d2a/d3c/f5d [0,4194304] 0 2024-02-19T19:06:20.063 INFO:tasks.workunit.client.0.smithi088.stdout:0/272: write d13/d46/d5a/d25/f2b [3035927,17111] 0 2024-02-19T19:06:20.064 INFO:tasks.workunit.client.0.smithi088.stdout:0/273: fdatasync d13/d46/d5a/d25/d2a/d2f/f50 0 2024-02-19T19:06:20.071 INFO:tasks.workunit.client.0.smithi088.stdout:0/274: dwrite d13/f3e [0,4194304] 0 2024-02-19T19:06:20.072 INFO:tasks.workunit.client.0.smithi088.stdout:0/275: stat d13/c51 0 2024-02-19T19:06:20.077 INFO:tasks.workunit.client.0.smithi088.stdout:6/346: creat d5/d8/d2c/f67 x:0 0 0 2024-02-19T19:06:20.077 INFO:tasks.workunit.client.0.smithi088.stdout:6/347: chown d5/d31/d25 5294 1 2024-02-19T19:06:20.084 INFO:tasks.workunit.client.0.smithi088.stdout:2/236: link d3/d1b/l47 d3/d1b/d43/l4b 0 2024-02-19T19:06:20.096 INFO:tasks.workunit.client.0.smithi088.stdout:2/237: symlink d3/d1f/l4c 0 2024-02-19T19:06:20.097 INFO:tasks.workunit.client.0.smithi088.stdout:2/238: read d3/d14/f1d [2546306,84222] 0 2024-02-19T19:06:20.104 INFO:tasks.workunit.client.0.smithi088.stdout:4/262: sync 2024-02-19T19:06:20.115 INFO:tasks.workunit.client.0.smithi088.stdout:0/276: truncate d13/d46/d5a/d25/d2a/d2f/f4f 2554048 0 2024-02-19T19:06:20.115 INFO:tasks.workunit.client.0.smithi088.stdout:2/239: rmdir d3/d1f 39 2024-02-19T19:06:20.116 INFO:tasks.workunit.client.0.smithi088.stdout:6/348: creat d5/de/f68 x:0 0 0 2024-02-19T19:06:20.121 INFO:tasks.workunit.client.0.smithi088.stdout:0/277: dwrite d13/d46/d5a/d25/d2a/d34/f4d [0,4194304] 0 2024-02-19T19:06:20.121 INFO:tasks.workunit.client.0.smithi088.stdout:6/349: chown d5/de/c43 2017582 1 2024-02-19T19:06:20.122 INFO:tasks.workunit.client.0.smithi088.stdout:4/263: symlink d2/d32/l50 0 2024-02-19T19:06:20.126 INFO:tasks.workunit.client.0.smithi088.stdout:0/278: mkdir d13/d46/d5e 0 2024-02-19T19:06:20.130 INFO:tasks.workunit.client.0.smithi088.stdout:6/350: mknod d5/d8/d2c/c69 0 2024-02-19T19:06:20.130 INFO:tasks.workunit.client.0.smithi088.stdout:0/279: mknod d13/d46/c5f 0 2024-02-19T19:06:20.132 INFO:tasks.workunit.client.0.smithi088.stdout:0/280: write d13/f1c [594559,15450] 0 2024-02-19T19:06:20.135 INFO:tasks.workunit.client.0.smithi088.stdout:4/264: dwrite d2/d8/f16 [0,4194304] 0 2024-02-19T19:06:20.138 INFO:tasks.workunit.client.0.smithi088.stdout:6/351: symlink d5/d8/d2c/l6a 0 2024-02-19T19:06:20.139 INFO:tasks.workunit.client.0.smithi088.stdout:6/352: write d5/d8/d2c/f4c [1297492,46332] 0 2024-02-19T19:06:20.142 INFO:tasks.workunit.client.0.smithi088.stdout:4/265: truncate d2/d8/f29 687056 0 2024-02-19T19:06:20.147 INFO:tasks.workunit.client.0.smithi088.stdout:6/353: write d5/f47 [906565,130958] 0 2024-02-19T19:06:20.149 INFO:tasks.workunit.client.0.smithi088.stdout:4/266: mkdir d2/d35/d39/d51 0 2024-02-19T19:06:20.154 INFO:tasks.workunit.client.0.smithi088.stdout:4/267: creat d2/d35/f52 x:0 0 0 2024-02-19T19:06:20.155 INFO:tasks.workunit.client.0.smithi088.stdout:0/281: getdents d13/d46/d5a/d25/d2a/d2f 0 2024-02-19T19:06:20.162 INFO:tasks.workunit.client.0.smithi088.stdout:0/282: mknod d13/d46/d5a/d25/c60 0 2024-02-19T19:06:20.162 INFO:tasks.workunit.client.0.smithi088.stdout:4/268: dwrite d2/d32/f1c [4194304,4194304] 0 2024-02-19T19:06:20.164 INFO:tasks.workunit.client.0.smithi088.stdout:6/354: link d5/de/c2a d5/d8/d3a/c6b 0 2024-02-19T19:06:20.171 INFO:tasks.workunit.client.0.smithi088.stdout:4/269: dwrite d2/d32/d25/d45/f4a [0,4194304] 0 2024-02-19T19:06:20.178 INFO:tasks.workunit.client.0.smithi088.stdout:6/355: dwrite d5/de/f10 [4194304,4194304] 0 2024-02-19T19:06:20.182 INFO:tasks.workunit.client.0.smithi088.stdout:4/270: dwrite d2/d35/f4f [0,4194304] 0 2024-02-19T19:06:20.192 INFO:tasks.workunit.client.0.smithi088.stdout:4/271: dwrite d2/d32/f44 [0,4194304] 0 2024-02-19T19:06:20.202 INFO:tasks.workunit.client.0.smithi088.stdout:6/356: creat d5/d8/d2c/f6c x:0 0 0 2024-02-19T19:06:20.203 INFO:tasks.workunit.client.0.smithi088.stdout:0/283: write d13/d46/d5a/d25/d2a/d2f/f4f [1616386,44704] 0 2024-02-19T19:06:20.206 INFO:tasks.workunit.client.0.smithi088.stdout:0/284: write d13/f1a [1892637,56398] 0 2024-02-19T19:06:20.210 INFO:tasks.workunit.client.0.smithi088.stdout:6/357: dwrite d5/d7/f4f [0,4194304] 0 2024-02-19T19:06:20.215 INFO:tasks.workunit.client.0.smithi088.stdout:0/285: dwrite d13/f1c [0,4194304] 0 2024-02-19T19:06:20.218 INFO:tasks.workunit.client.0.smithi088.stdout:0/286: symlink d13/d46/d5a/l61 0 2024-02-19T19:06:20.219 INFO:tasks.workunit.client.0.smithi088.stdout:0/287: dread - d13/f37 zero size 2024-02-19T19:06:20.223 INFO:tasks.workunit.client.0.smithi088.stdout:0/288: mkdir d13/d62 0 2024-02-19T19:06:20.225 INFO:tasks.workunit.client.0.smithi088.stdout:0/289: rename d13/d46/d5a/d25 to d13/d46/d5a/d25/d2a/d63 22 2024-02-19T19:06:20.226 INFO:tasks.workunit.client.0.smithi088.stdout:4/272: truncate d2/d32/d25/d26/f49 431990 0 2024-02-19T19:06:20.229 INFO:tasks.workunit.client.0.smithi088.stdout:0/290: rename d13/d46/d5a/d25/d2a/f48 to d13/d46/d5a/d25/d2a/d3c/f64 0 2024-02-19T19:06:20.231 INFO:tasks.workunit.client.0.smithi088.stdout:6/358: truncate d5/d7/f14 3469454 0 2024-02-19T19:06:20.235 INFO:tasks.workunit.client.0.smithi088.stdout:4/273: dread d2/d8/f48 [0,4194304] 0 2024-02-19T19:06:20.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.229+0000 7fdd1eddd700 1 -- 172.21.15.88:0/3040042757 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdd1804f2a0 con 0x7fdd181004c0 2024-02-19T19:06:20.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.229+0000 7fdd15ffb700 1 -- 172.21.15.88:0/3040042757 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1892 (secure 0 0 0) 0x7fdd04057030 con 0x7fdd181004c0 2024-02-19T19:06:20.250 INFO:teuthology.orchestra.run.smithi088.stdout:e20 2024-02-19T19:06:20.250 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:06:20.250 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:06:20.250 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:06:20.250 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:06:20.250 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:06:20.250 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 20 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:05:14.913778+0000 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:06:20.251 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:06:20.252 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T19:06:20.252 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:06:20.252 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:06:20.252 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:06:20.252 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T19:06:20.252 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:06:20.252 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:06:20.252 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:06:20.252 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:06:20.253 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:06:20.254 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:06:20.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.232+0000 7fdd1eddd700 1 -- 172.21.15.88:0/3040042757 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd0806da20 msgr2=0x7fdd0806fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.232+0000 7fdd1eddd700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd0806da20 0x7fdd0806fed0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0098e0 tx=0x7fdd0c00b8a0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.232+0000 7fdd1eddd700 1 -- 172.21.15.88:0/3040042757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd181004c0 msgr2=0x7fdd18198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.232+0000 7fdd1eddd700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd181004c0 0x7fdd18198340 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdd0400e920 tx=0x7fdd0400ece0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.232+0000 7fdd1eddd700 1 -- 172.21.15.88:0/3040042757 shutdown_connections 2024-02-19T19:06:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.232+0000 7fdd1eddd700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd0806da20 0x7fdd0806fed0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.232+0000 7fdd1eddd700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd181004c0 0x7fdd18198340 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.232+0000 7fdd1eddd700 1 --2- 172.21.15.88:0/3040042757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd18198880 0x7fdd1819ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.232+0000 7fdd1eddd700 1 -- 172.21.15.88:0/3040042757 >> 172.21.15.88:0/3040042757 conn(0x7fdd180fbd80 msgr2=0x7fdd180fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.232+0000 7fdd1eddd700 1 -- 172.21.15.88:0/3040042757 shutdown_connections 2024-02-19T19:06:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.232+0000 7fdd1eddd700 1 -- 172.21.15.88:0/3040042757 wait complete. 2024-02-19T19:06:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-02-19T19:06:20.311 INFO:tasks.workunit.client.0.smithi088.stdout:8/282: sync 2024-02-19T19:06:20.313 INFO:tasks.workunit.client.0.smithi088.stdout:6/359: creat d5/d19/f6d x:0 0 0 2024-02-19T19:06:20.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.362+0000 7fc36e3d8700 1 -- 172.21.15.88:0/1421381799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc368104130 msgr2=0x7fc368104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:20.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.362+0000 7fc36e3d8700 1 --2- 172.21.15.88:0/1421381799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc368104130 0x7fc368104500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc35c009910 tx=0x7fc35c009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:20.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.362+0000 7fc36e3d8700 1 -- 172.21.15.88:0/1421381799 shutdown_connections 2024-02-19T19:06:20.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.362+0000 7fc36e3d8700 1 --2- 172.21.15.88:0/1421381799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc3680ffbd0 0x7fc368100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.362+0000 7fc36e3d8700 1 --2- 172.21.15.88:0/1421381799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc368104130 0x7fc368104500 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.362+0000 7fc36e3d8700 1 -- 172.21.15.88:0/1421381799 >> 172.21.15.88:0/1421381799 conn(0x7fc368075130 msgr2=0x7fc368075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:20.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.362+0000 7fc36e3d8700 1 -- 172.21.15.88:0/1421381799 shutdown_connections 2024-02-19T19:06:20.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.362+0000 7fc36e3d8700 1 -- 172.21.15.88:0/1421381799 wait complete. 2024-02-19T19:06:20.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.363+0000 7fc36e3d8700 1 Processor -- start 2024-02-19T19:06:20.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.363+0000 7fc36e3d8700 1 -- start start 2024-02-19T19:06:20.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.363+0000 7fc36e3d8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3680ffbd0 0x7fc368100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:20.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.363+0000 7fc36e3d8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc368104130 0x7fc368101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:20.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.363+0000 7fc36e3d8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681018b0 con 0x7fc3680ffbd0 2024-02-19T19:06:20.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.363+0000 7fc36e3d8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3681019f0 con 0x7fc368104130 2024-02-19T19:06:20.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.363+0000 7fc367fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3680ffbd0 0x7fc368100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.363+0000 7fc367fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3680ffbd0 0x7fc368100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48420/0 (socket says 172.21.15.88:48420) 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.363+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc368104130 0x7fc368101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.363+0000 7fc367fff700 1 -- 172.21.15.88:0/4093777504 learned_addr learned my addr 172.21.15.88:0/4093777504 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.364+0000 7fc367fff700 1 -- 172.21.15.88:0/4093777504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc368104130 msgr2=0x7fc368101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.364+0000 7fc367fff700 1 --2- 172.21.15.88:0/4093777504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc368104130 0x7fc368101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.364+0000 7fc367fff700 1 -- 172.21.15.88:0/4093777504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc35c0095f0 con 0x7fc3680ffbd0 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.364+0000 7fc3677fe700 1 --2- 172.21.15.88:0/4093777504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc368104130 0x7fc368101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.364+0000 7fc367fff700 1 --2- 172.21.15.88:0/4093777504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3680ffbd0 0x7fc368100e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00bd70 tx=0x7fc35c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.364+0000 7fc3657fa700 1 -- 172.21.15.88:0/4093777504 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc35c01d070 con 0x7fc3680ffbd0 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.364+0000 7fc36e3d8700 1 -- 172.21.15.88:0/4093777504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc368101c70 con 0x7fc3680ffbd0 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.364+0000 7fc36e3d8700 1 -- 172.21.15.88:0/4093777504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3681a2350 con 0x7fc3680ffbd0 2024-02-19T19:06:20.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.364+0000 7fc3657fa700 1 -- 172.21.15.88:0/4093777504 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc35c005dc0 con 0x7fc3680ffbd0 2024-02-19T19:06:20.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.364+0000 7fc3657fa700 1 -- 172.21.15.88:0/4093777504 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc35c017400 con 0x7fc3680ffbd0 2024-02-19T19:06:20.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.365+0000 7fc36e3d8700 1 -- 172.21.15.88:0/4093777504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc354005320 con 0x7fc3680ffbd0 2024-02-19T19:06:20.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.366+0000 7fc3657fa700 1 -- 172.21.15.88:0/4093777504 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7fc35c003a60 con 0x7fc3680ffbd0 2024-02-19T19:06:20.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.366+0000 7fc3657fa700 1 --2- 172.21.15.88:0/4093777504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc35006d9e0 0x7fc35006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:20.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.366+0000 7fc3657fa700 1 -- 172.21.15.88:0/4093777504 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc35c08ea00 con 0x7fc3680ffbd0 2024-02-19T19:06:20.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.366+0000 7fc3677fe700 1 --2- 172.21.15.88:0/4093777504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc35006d9e0 0x7fc35006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:20.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.367+0000 7fc3677fe700 1 --2- 172.21.15.88:0/4093777504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc35006d9e0 0x7fc35006fe90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc368102620 tx=0x7fc35800b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:20.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.372+0000 7fc3657fa700 1 -- 172.21.15.88:0/4093777504 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc35c058180 con 0x7fc3680ffbd0 2024-02-19T19:06:20.419 INFO:tasks.workunit.client.0.smithi088.stdout:6/360: dwrite d5/d7/d32/f53 [0,4194304] 0 2024-02-19T19:06:20.420 INFO:tasks.workunit.client.0.smithi088.stdout:0/291: dread d13/f29 [0,4194304] 0 2024-02-19T19:06:20.423 INFO:tasks.workunit.client.0.smithi088.stdout:4/274: getdents d2/d35/d39 0 2024-02-19T19:06:20.428 INFO:tasks.workunit.client.0.smithi088.stdout:0/292: symlink d13/d46/d5a/d25/d2a/d3c/l65 0 2024-02-19T19:06:20.428 INFO:tasks.workunit.client.0.smithi088.stdout:0/293: stat d13/f33 0 2024-02-19T19:06:20.429 INFO:tasks.workunit.client.0.smithi088.stdout:8/283: rename d9/l3a to d9/l4f 0 2024-02-19T19:06:20.432 INFO:tasks.workunit.client.0.smithi088.stdout:4/275: unlink d2/d32/d25/d26/c31 0 2024-02-19T19:06:20.434 INFO:tasks.workunit.client.0.smithi088.stdout:6/361: link d5/d8/d3a/f46 d5/d31/f6e 0 2024-02-19T19:06:20.436 INFO:tasks.workunit.client.0.smithi088.stdout:8/284: mkdir d9/df/d3d/d50 0 2024-02-19T19:06:20.437 INFO:tasks.workunit.client.0.smithi088.stdout:4/276: creat d2/d35/f53 x:0 0 0 2024-02-19T19:06:20.440 INFO:tasks.workunit.client.0.smithi088.stdout:0/294: rename d13/d46/d5a/l61 to d13/d46/d5a/d25/l66 0 2024-02-19T19:06:20.440 INFO:tasks.workunit.client.0.smithi088.stdout:4/277: rename d2/d35/d39 to d2/d35/d39/d51/d54 22 2024-02-19T19:06:20.443 INFO:tasks.workunit.client.0.smithi088.stdout:0/295: write d13/d46/d5a/d25/d2a/d2f/f4c [740388,63427] 0 2024-02-19T19:06:20.443 INFO:tasks.workunit.client.0.smithi088.stdout:8/285: dwrite d9/df/d1c/d2d/d47/f43 [0,4194304] 0 2024-02-19T19:06:20.445 INFO:tasks.workunit.client.0.smithi088.stdout:0/296: write d13/d46/d5a/f4b [1211189,96394] 0 2024-02-19T19:06:20.454 INFO:tasks.workunit.client.0.smithi088.stdout:0/297: dwrite d13/d46/d5a/d25/d2a/d3c/f64 [0,4194304] 0 2024-02-19T19:06:20.462 INFO:tasks.workunit.client.0.smithi088.stdout:0/298: dwrite d13/f44 [0,4194304] 0 2024-02-19T19:06:20.465 INFO:tasks.workunit.client.0.smithi088.stdout:0/299: mknod d13/d46/c67 0 2024-02-19T19:06:20.474 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-02-19T19:06:20.474 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-02-19T19:06:20.481 INFO:tasks.workunit.client.0.smithi088.stdout:8/286: fdatasync d9/df/d1c/d2d/d47/f43 0 2024-02-19T19:06:20.483 INFO:tasks.workunit.client.0.smithi088.stdout:4/278: dwrite d2/d8/f14 [0,4194304] 0 2024-02-19T19:06:20.487 INFO:tasks.workunit.client.0.smithi088.stdout:4/279: symlink d2/l55 0 2024-02-19T19:06:20.488 INFO:tasks.workunit.client.0.smithi088.stdout:4/280: chown d2/d32/d9/c10 268787 1 2024-02-19T19:06:20.496 INFO:tasks.workunit.client.0.smithi088.stdout:4/281: write d2/f4 [779992,5136] 0 2024-02-19T19:06:20.497 INFO:tasks.workunit.client.0.smithi088.stdout:0/300: dread fe [0,4194304] 0 2024-02-19T19:06:20.497 INFO:tasks.workunit.client.0.smithi088.stdout:4/282: stat d2/c3 0 2024-02-19T19:06:20.498 INFO:tasks.workunit.client.0.smithi088.stdout:0/301: truncate d13/f4a 653651 0 2024-02-19T19:06:20.499 INFO:tasks.workunit.client.0.smithi088.stdout:0/302: mknod d13/d46/d5a/c68 0 2024-02-19T19:06:20.500 INFO:tasks.workunit.client.0.smithi088.stdout:0/303: chown d13/d46/d5a/d25/d2a/d2f/f41 52 1 2024-02-19T19:06:20.505 INFO:tasks.workunit.client.0.smithi088.stdout:0/304: dwrite f3 [0,4194304] 0 2024-02-19T19:06:20.537 INFO:tasks.workunit.client.0.smithi088.stdout:9/340: sync 2024-02-19T19:06:20.543 INFO:tasks.workunit.client.0.smithi088.stdout:0/305: chown d13/d46/d5a/d25/d2a/d34/c47 1482213 1 2024-02-19T19:06:20.546 INFO:tasks.workunit.client.0.smithi088.stdout:9/341: dwrite d1/d21/f64 [0,4194304] 0 2024-02-19T19:06:20.546 INFO:tasks.workunit.client.0.smithi088.stdout:0/306: link d13/f2c d13/d46/f69 0 2024-02-19T19:06:20.551 INFO:tasks.workunit.client.0.smithi088.stdout:9/342: creat d1/d21/d2d/d18/f6a x:0 0 0 2024-02-19T19:06:20.552 INFO:tasks.workunit.client.0.smithi088.stdout:9/343: chown d1/d13/f12 8 1 2024-02-19T19:06:20.554 INFO:tasks.workunit.client.0.smithi088.stdout:0/307: rename d13/d46/d5a/d25/d2a/d2f/f4f to d13/f6a 0 2024-02-19T19:06:20.558 INFO:tasks.workunit.client.0.smithi088.stdout:0/308: rename d13/c19 to d13/d46/d5a/d25/d2a/d34/c6b 0 2024-02-19T19:06:20.560 INFO:tasks.workunit.client.0.smithi088.stdout:0/309: creat d13/d46/f6c x:0 0 0 2024-02-19T19:06:20.582 INFO:tasks.workunit.client.0.smithi088.stdout:4/283: dread d2/d32/fb [0,4194304] 0 2024-02-19T19:06:20.588 INFO:tasks.workunit.client.0.smithi088.stdout:4/284: getdents d2/d35/d39 0 2024-02-19T19:06:20.594 INFO:tasks.workunit.client.0.smithi088.stdout:4/285: getdents d2/d32/d25 0 2024-02-19T19:06:20.601 INFO:tasks.workunit.client.0.smithi088.stdout:4/286: dwrite d2/d8/f48 [0,4194304] 0 2024-02-19T19:06:20.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.671+0000 7fc36e3d8700 1 -- 172.21.15.88:0/4093777504 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc354000bf0 con 0x7fc35006d9e0 2024-02-19T19:06:20.691 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:20 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3792125569' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:06:20.691 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:20.691 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:20.691 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:20 smithi088 ceph-mon[88643]: Reconfiguring osd.5 (monmap changed)... 2024-02-19T19:06:20.691 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-19T19:06:20.691 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:20.692 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:20 smithi088 ceph-mon[88643]: Reconfiguring daemon osd.5 on smithi132 2024-02-19T19:06:20.692 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:20 smithi088 ceph-mon[88643]: pgmap v44: 65 pgs: 65 active+clean; 946 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 21 MiB/s rd, 110 MiB/s wr, 476 op/s 2024-02-19T19:06:20.692 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:20 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3040042757' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:06:20.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.672+0000 7fc3657fa700 1 -- 172.21.15.88:0/4093777504 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fc354000bf0 con 0x7fc35006d9e0 2024-02-19T19:06:20.692 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:06:20.692 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:06:20.693 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:06:20.693 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:06:20.693 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:06:20.693 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:06:20.693 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:06:20.693 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:06:20.693 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "4/23 daemons upgraded", 2024-02-19T19:06:20.693 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mon daemons", 2024-02-19T19:06:20.693 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:06:20.693 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:06:20.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.674+0000 7fc36e3d8700 1 -- 172.21.15.88:0/4093777504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc35006d9e0 msgr2=0x7fc35006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:20.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.674+0000 7fc36e3d8700 1 --2- 172.21.15.88:0/4093777504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc35006d9e0 0x7fc35006fe90 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc368102620 tx=0x7fc35800b410 comp rx=0 tx=0).stop 2024-02-19T19:06:20.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.674+0000 7fc36e3d8700 1 -- 172.21.15.88:0/4093777504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3680ffbd0 msgr2=0x7fc368100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:20.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.674+0000 7fc36e3d8700 1 --2- 172.21.15.88:0/4093777504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3680ffbd0 0x7fc368100e30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc35c00bd70 tx=0x7fc35c00bda0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.674+0000 7fc36e3d8700 1 -- 172.21.15.88:0/4093777504 shutdown_connections 2024-02-19T19:06:20.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.674+0000 7fc36e3d8700 1 --2- 172.21.15.88:0/4093777504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc35006d9e0 0x7fc35006fe90 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.674+0000 7fc36e3d8700 1 --2- 172.21.15.88:0/4093777504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3680ffbd0 0x7fc368100e30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.674+0000 7fc36e3d8700 1 --2- 172.21.15.88:0/4093777504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc368104130 0x7fc368101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.674+0000 7fc36e3d8700 1 -- 172.21.15.88:0/4093777504 >> 172.21.15.88:0/4093777504 conn(0x7fc368075130 msgr2=0x7fc3680fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:20.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.674+0000 7fc36e3d8700 1 -- 172.21.15.88:0/4093777504 shutdown_connections 2024-02-19T19:06:20.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.674+0000 7fc36e3d8700 1 -- 172.21.15.88:0/4093777504 wait complete. 2024-02-19T19:06:20.718 INFO:tasks.workunit.client.0.smithi088.stdout:1/426: sync 2024-02-19T19:06:20.720 INFO:tasks.workunit.client.0.smithi088.stdout:1/427: symlink d6/l90 0 2024-02-19T19:06:20.720 INFO:tasks.workunit.client.0.smithi088.stdout:1/428: fsync d6/d21/d6a/d7d/f8b 0 2024-02-19T19:06:20.723 INFO:tasks.workunit.client.0.smithi088.stdout:1/429: getdents d6/d21/d6a/d7d 0 2024-02-19T19:06:20.724 INFO:tasks.workunit.client.0.smithi088.stdout:1/430: creat d6/d10/f91 x:0 0 0 2024-02-19T19:06:20.729 INFO:tasks.workunit.client.0.smithi088.stdout:1/431: dwrite d6/d21/f3c [0,4194304] 0 2024-02-19T19:06:20.732 INFO:tasks.workunit.client.0.smithi088.stdout:1/432: creat d6/f92 x:0 0 0 2024-02-19T19:06:20.735 INFO:tasks.workunit.client.0.smithi088.stdout:1/433: creat d6/d10/f93 x:0 0 0 2024-02-19T19:06:20.738 INFO:tasks.workunit.client.0.smithi088.stdout:1/434: rename d6/d89 to d6/d21/d64/d94 0 2024-02-19T19:06:20.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:20 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3792125569' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:06:20.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:20.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:20.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:20 smithi132 ceph-mon[70676]: Reconfiguring osd.5 (monmap changed)... 2024-02-19T19:06:20.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-19T19:06:20.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:20.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:20 smithi132 ceph-mon[70676]: Reconfiguring daemon osd.5 on smithi132 2024-02-19T19:06:20.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:20 smithi132 ceph-mon[70676]: pgmap v44: 65 pgs: 65 active+clean; 946 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 21 MiB/s rd, 110 MiB/s wr, 476 op/s 2024-02-19T19:06:20.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:20 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3040042757' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:06:20.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.784+0000 7f1ad207b700 1 -- 172.21.15.88:0/957943318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1acc101d30 msgr2=0x7f1acc10a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:20.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.784+0000 7f1ad207b700 1 --2- 172.21.15.88:0/957943318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1acc101d30 0x7f1acc10a230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009910 tx=0x7f1abc009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:20.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.784+0000 7f1ad207b700 1 -- 172.21.15.88:0/957943318 shutdown_connections 2024-02-19T19:06:20.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.784+0000 7f1ad207b700 1 --2- 172.21.15.88:0/957943318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1acc101d30 0x7f1acc10a230 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.784+0000 7f1ad207b700 1 --2- 172.21.15.88:0/957943318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1acc101420 0x7f1acc1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.784+0000 7f1ad207b700 1 -- 172.21.15.88:0/957943318 >> 172.21.15.88:0/957943318 conn(0x7f1acc0fac80 msgr2=0x7f1acc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:20.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.784+0000 7f1ad207b700 1 -- 172.21.15.88:0/957943318 shutdown_connections 2024-02-19T19:06:20.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.784+0000 7f1ad207b700 1 -- 172.21.15.88:0/957943318 wait complete. 2024-02-19T19:06:20.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.785+0000 7f1ad207b700 1 Processor -- start 2024-02-19T19:06:20.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.785+0000 7f1ad207b700 1 -- start start 2024-02-19T19:06:20.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.785+0000 7f1ad207b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1acc101420 0x7f1acc0ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:20.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.785+0000 7f1ad207b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1acc101d30 0x7f1acc0ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:20.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.785+0000 7f1ad207b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0ffb00 con 0x7f1acc101420 2024-02-19T19:06:20.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.785+0000 7f1ad207b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0ffc40 con 0x7f1acc101d30 2024-02-19T19:06:20.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.785+0000 7f1acb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1acc101420 0x7f1acc0ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:20.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.785+0000 7f1acb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1acc101420 0x7f1acc0ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48436/0 (socket says 172.21.15.88:48436) 2024-02-19T19:06:20.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.785+0000 7f1acb7fe700 1 -- 172.21.15.88:0/2379174255 learned_addr learned my addr 172.21.15.88:0/2379174255 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:20.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.785+0000 7f1acaffd700 1 --2- 172.21.15.88:0/2379174255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1acc101d30 0x7f1acc0ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:20.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.786+0000 7f1acb7fe700 1 -- 172.21.15.88:0/2379174255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1acc101d30 msgr2=0x7f1acc0ff5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:20.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.786+0000 7f1acb7fe700 1 --2- 172.21.15.88:0/2379174255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1acc101d30 0x7f1acc0ff5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:20.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.786+0000 7f1acb7fe700 1 -- 172.21.15.88:0/2379174255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc0095f0 con 0x7f1acc101420 2024-02-19T19:06:20.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.786+0000 7f1acb7fe700 1 --2- 172.21.15.88:0/2379174255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1acc101420 0x7f1acc0ff080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000e920 tx=0x7f1ac000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:20.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.786+0000 7f1ac8ff9700 1 -- 172.21.15.88:0/2379174255 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac000c9d0 con 0x7f1acc101420 2024-02-19T19:06:20.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.786+0000 7f1ad207b700 1 -- 172.21.15.88:0/2379174255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1acc1a2030 con 0x7f1acc101420 2024-02-19T19:06:20.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.786+0000 7f1ac8ff9700 1 -- 172.21.15.88:0/2379174255 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ac0010430 con 0x7f1acc101420 2024-02-19T19:06:20.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.786+0000 7f1ac8ff9700 1 -- 172.21.15.88:0/2379174255 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac0018630 con 0x7f1acc101420 2024-02-19T19:06:20.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.786+0000 7f1ad207b700 1 -- 172.21.15.88:0/2379174255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc1a2440 con 0x7f1acc101420 2024-02-19T19:06:20.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.787+0000 7f1ad207b700 1 -- 172.21.15.88:0/2379174255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc1078a0 con 0x7f1acc101420 2024-02-19T19:06:20.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.788+0000 7f1ac8ff9700 1 -- 172.21.15.88:0/2379174255 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 38) v1 ==== 91595+0+0 (secure 0 0 0) 0x7f1ac00105a0 con 0x7f1acc101420 2024-02-19T19:06:20.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.788+0000 7f1ac8ff9700 1 --2- 172.21.15.88:0/2379174255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ab406d9e0 0x7f1ab406fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:20.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.788+0000 7f1ac8ff9700 1 -- 172.21.15.88:0/2379174255 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1ac0014070 con 0x7f1acc101420 2024-02-19T19:06:20.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.788+0000 7f1acaffd700 1 --2- 172.21.15.88:0/2379174255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ab406d9e0 0x7f1ab406fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:20.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.788+0000 7f1acaffd700 1 --2- 172.21.15.88:0/2379174255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ab406d9e0 0x7f1ab406fe90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1acc100870 tx=0x7f1abc005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:20.810 DEBUG:teuthology.parallel:result is None 2024-02-19T19:06:20.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:20.793+0000 7f1ac8ff9700 1 -- 172.21.15.88:0/2379174255 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ac0052d20 con 0x7f1acc101420 2024-02-19T19:06:20.906 INFO:tasks.workunit.client.0.smithi088.stdout:3/418: sync 2024-02-19T19:06:20.906 INFO:tasks.workunit.client.0.smithi088.stdout:5/470: sync 2024-02-19T19:06:20.907 INFO:tasks.workunit.client.0.smithi088.stdout:3/419: chown d4/fd 10647 1 2024-02-19T19:06:20.908 INFO:tasks.workunit.client.0.smithi088.stdout:3/420: write d4/d1a/f24 [216190,96485] 0 2024-02-19T19:06:20.910 INFO:tasks.workunit.client.0.smithi088.stdout:5/471: rename d4/d1a/d22 to d4/d61/d65/d9a/d9f 0 2024-02-19T19:06:20.911 INFO:tasks.workunit.client.0.smithi088.stdout:5/472: fdatasync d4/d61/d65/d9a/d9f/f24 0 2024-02-19T19:06:20.912 INFO:tasks.workunit.client.0.smithi088.stdout:3/421: mkdir d4/d1a/d1e/d74/d7c/d97 0 2024-02-19T19:06:20.912 INFO:tasks.workunit.client.0.smithi088.stdout:5/473: truncate d4/d61/f73 1023464 0 2024-02-19T19:06:20.914 INFO:tasks.workunit.client.0.smithi088.stdout:3/422: chown d4/f6a 1437553223 1 2024-02-19T19:06:20.915 INFO:tasks.workunit.client.0.smithi088.stdout:3/423: fsync d4/d12/d33/d43/f65 0 2024-02-19T19:06:20.915 INFO:tasks.workunit.client.0.smithi088.stdout:3/424: fdatasync d4/f79 0 2024-02-19T19:06:20.920 INFO:tasks.workunit.client.0.smithi088.stdout:3/425: creat d4/d22/d81/f98 x:0 0 0 2024-02-19T19:06:20.921 INFO:tasks.workunit.client.0.smithi088.stdout:5/474: dwrite d4/f87 [0,4194304] 0 2024-02-19T19:06:20.925 INFO:tasks.workunit.client.0.smithi088.stdout:3/426: rename d4/d1a/l52 to d4/d22/l99 0 2024-02-19T19:06:20.929 INFO:tasks.workunit.client.0.smithi088.stdout:5/475: dwrite d4/d1a/f90 [0,4194304] 0 2024-02-19T19:06:20.930 INFO:tasks.workunit.client.0.smithi088.stdout:5/476: chown d4/d61/d65/d9a/d9f/f4c 21218 1 2024-02-19T19:06:20.931 INFO:tasks.workunit.client.0.smithi088.stdout:3/427: rename d4/d1a/d1e/d31/d32/d55/l91 to d4/d12/d33/d43/l9a 0 2024-02-19T19:06:20.932 INFO:tasks.workunit.client.0.smithi088.stdout:3/428: dread - d4/d1a/d1e/f8f zero size 2024-02-19T19:06:20.933 INFO:tasks.workunit.client.0.smithi088.stdout:5/477: write d4/d75/f98 [869110,104407] 0 2024-02-19T19:06:20.934 INFO:tasks.workunit.client.0.smithi088.stdout:5/478: write d4/d1a/d41/d5e/f82 [349283,79692] 0 2024-02-19T19:06:20.936 INFO:tasks.workunit.client.0.smithi088.stdout:3/429: symlink d4/d1a/d1e/d74/d7c/d97/l9b 0 2024-02-19T19:06:20.938 INFO:tasks.workunit.client.0.smithi088.stdout:5/479: mkdir d4/d61/d65/d9a/d9f/da0 0 2024-02-19T19:06:20.939 INFO:tasks.workunit.client.0.smithi088.stdout:5/480: chown d4/d1a 82459 1 2024-02-19T19:06:20.940 INFO:tasks.workunit.client.0.smithi088.stdout:3/430: write d4/f18 [885539,90261] 0 2024-02-19T19:06:20.943 INFO:tasks.workunit.client.0.smithi088.stdout:3/431: mknod d4/d22/d7a/d8d/c9c 0 2024-02-19T19:06:20.946 INFO:tasks.workunit.client.0.smithi088.stdout:3/432: unlink d4/d1a/d1e/d31/f84 0 2024-02-19T19:06:20.948 INFO:tasks.workunit.client.0.smithi088.stdout:3/433: unlink d4/d1a/d1e/f92 0 2024-02-19T19:06:20.954 INFO:tasks.workunit.client.0.smithi088.stdout:5/481: write d4/f9 [6291105,65884] 0 2024-02-19T19:06:20.960 INFO:tasks.workunit.client.0.smithi088.stdout:3/434: dread d4/d22/f2c [0,4194304] 0 2024-02-19T19:06:20.961 INFO:tasks.workunit.client.0.smithi088.stdout:3/435: write d4/d22/f2c [2189286,43580] 0 2024-02-19T19:06:20.965 INFO:tasks.workunit.client.0.smithi088.stdout:3/436: dread d4/d12/f27 [0,4194304] 0 2024-02-19T19:06:20.970 INFO:tasks.workunit.client.0.smithi088.stdout:3/437: dread d4/ff [0,4194304] 0 2024-02-19T19:06:20.984 INFO:tasks.workunit.client.0.smithi088.stdout:3/438: dread d4/f7 [0,4194304] 0 2024-02-19T19:06:20.986 INFO:tasks.workunit.client.0.smithi088.stdout:3/439: readlink d4/l63 0 2024-02-19T19:06:21.028 INFO:tasks.workunit.client.0.smithi088.stdout:3/440: write d4/f38 [486225,63916] 0 2024-02-19T19:06:21.030 INFO:tasks.workunit.client.0.smithi088.stdout:3/441: rename d4/d1a/d1e/f8f to d4/d22/d7a/d8d/f9d 0 2024-02-19T19:06:21.032 INFO:tasks.workunit.client.0.smithi088.stdout:3/442: creat d4/d12/d33/f9e x:0 0 0 2024-02-19T19:06:21.037 INFO:tasks.workunit.client.0.smithi088.stdout:3/443: dwrite d4/f79 [0,4194304] 0 2024-02-19T19:06:21.041 INFO:tasks.workunit.client.0.smithi088.stdout:3/444: rename d4/d12/f27 to d4/d22/d7a/f9f 0 2024-02-19T19:06:21.042 INFO:tasks.workunit.client.0.smithi088.stdout:3/445: dread - d4/d87/f94 zero size 2024-02-19T19:06:21.042 INFO:tasks.workunit.client.0.smithi088.stdout:3/446: chown d4/d1a/d1e/d31/d32/f51 15 1 2024-02-19T19:06:21.046 INFO:tasks.workunit.client.0.smithi088.stdout:3/447: getdents d4/d22 0 2024-02-19T19:06:21.047 INFO:tasks.workunit.client.0.smithi088.stdout:3/448: chown d4/d1a/d1e/d31/d32/f8c 317573 1 2024-02-19T19:06:21.105 INFO:tasks.workunit.client.0.smithi088.stdout:6/362: sync 2024-02-19T19:06:21.105 INFO:tasks.workunit.client.0.smithi088.stdout:2/240: sync 2024-02-19T19:06:21.105 INFO:tasks.workunit.client.0.smithi088.stdout:7/346: sync 2024-02-19T19:06:21.106 INFO:tasks.workunit.client.0.smithi088.stdout:6/363: fdatasync d5/d8/f3f 0 2024-02-19T19:06:21.108 INFO:tasks.workunit.client.0.smithi088.stdout:2/241: creat d3/d14/f4d x:0 0 0 2024-02-19T19:06:21.110 INFO:tasks.workunit.client.0.smithi088.stdout:7/347: rename d14/c1a to d14/d1d/d3c/d43/d5c/c7a 0 2024-02-19T19:06:21.110 INFO:tasks.workunit.client.0.smithi088.stdout:6/364: mknod d5/d8/d3a/c6f 0 2024-02-19T19:06:21.111 INFO:tasks.workunit.client.0.smithi088.stdout:6/365: stat d5/d31 0 2024-02-19T19:06:21.112 INFO:tasks.workunit.client.0.smithi088.stdout:2/242: mknod d3/c4e 0 2024-02-19T19:06:21.113 INFO:tasks.workunit.client.0.smithi088.stdout:7/348: creat d14/d27/d6b/f7b x:0 0 0 2024-02-19T19:06:21.114 INFO:tasks.workunit.client.0.smithi088.stdout:2/243: write d3/f6 [3206740,119630] 0 2024-02-19T19:06:21.115 INFO:tasks.workunit.client.0.smithi088.stdout:2/244: fsync d3/d1f/f32 0 2024-02-19T19:06:21.116 INFO:tasks.workunit.client.0.smithi088.stdout:7/349: mkdir d14/d1d/d3c/d43/d5c/d75/d76/d7c 0 2024-02-19T19:06:21.122 INFO:tasks.workunit.client.0.smithi088.stdout:7/350: dwrite d14/d28/d3b/f4b [0,4194304] 0 2024-02-19T19:06:21.123 INFO:tasks.workunit.client.0.smithi088.stdout:7/351: stat d14/f3e 0 2024-02-19T19:06:21.123 INFO:tasks.workunit.client.0.smithi088.stdout:7/352: chown d14/f29 927044565 1 2024-02-19T19:06:21.129 INFO:tasks.workunit.client.0.smithi088.stdout:2/245: getdents d3 0 2024-02-19T19:06:21.131 INFO:tasks.workunit.client.0.smithi088.stdout:7/353: creat d14/d27/d49/f7d x:0 0 0 2024-02-19T19:06:21.136 INFO:tasks.workunit.client.0.smithi088.stdout:2/246: dwrite d3/d14/f35 [0,4194304] 0 2024-02-19T19:06:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.116+0000 7f1ad207b700 1 -- 172.21.15.88:0/2379174255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1acc04f2a0 con 0x7f1acc101420 2024-02-19T19:06:21.137 INFO:tasks.workunit.client.0.smithi088.stdout:2/247: chown d3/df/l37 195197452 1 2024-02-19T19:06:21.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.116+0000 7f1ac8ff9700 1 -- 172.21.15.88:0/2379174255 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f1ac0057000 con 0x7f1acc101420 2024-02-19T19:06:21.137 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_OK 2024-02-19T19:06:21.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.121+0000 7f1ad207b700 1 -- 172.21.15.88:0/2379174255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ab406d9e0 msgr2=0x7f1ab406fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:21.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.121+0000 7f1ad207b700 1 --2- 172.21.15.88:0/2379174255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ab406d9e0 0x7f1ab406fe90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1acc100870 tx=0x7f1abc005940 comp rx=0 tx=0).stop 2024-02-19T19:06:21.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.121+0000 7f1ad207b700 1 -- 172.21.15.88:0/2379174255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1acc101420 msgr2=0x7f1acc0ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:21.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.121+0000 7f1ad207b700 1 --2- 172.21.15.88:0/2379174255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1acc101420 0x7f1acc0ff080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000e920 tx=0x7f1ac000ec30 comp rx=0 tx=0).stop 2024-02-19T19:06:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.121+0000 7f1ad207b700 1 -- 172.21.15.88:0/2379174255 shutdown_connections 2024-02-19T19:06:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.121+0000 7f1ad207b700 1 --2- 172.21.15.88:0/2379174255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ab406d9e0 0x7f1ab406fe90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.121+0000 7f1ad207b700 1 --2- 172.21.15.88:0/2379174255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1acc101420 0x7f1acc0ff080 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.121+0000 7f1ad207b700 1 --2- 172.21.15.88:0/2379174255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1acc101d30 0x7f1acc0ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.121+0000 7f1ad207b700 1 -- 172.21.15.88:0/2379174255 >> 172.21.15.88:0/2379174255 conn(0x7f1acc0fac80 msgr2=0x7f1acc1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.122+0000 7f1ad207b700 1 -- 172.21.15.88:0/2379174255 shutdown_connections 2024-02-19T19:06:21.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:21.122+0000 7f1ad207b700 1 -- 172.21.15.88:0/2379174255 wait complete. 2024-02-19T19:06:21.145 INFO:tasks.workunit.client.0.smithi088.stdout:7/354: dwrite d14/d1d/d3c/f3d [0,4194304] 0 2024-02-19T19:06:21.152 INFO:tasks.workunit.client.0.smithi088.stdout:7/355: rename d14/d56/f58 to d14/d28/d65/f7e 0 2024-02-19T19:06:21.159 INFO:tasks.workunit.client.0.smithi088.stdout:7/356: truncate d14/d27/f32 1980098 0 2024-02-19T19:06:21.165 INFO:tasks.workunit.client.0.smithi088.stdout:2/248: dread d3/d14/f1d [0,4194304] 0 2024-02-19T19:06:21.165 INFO:tasks.workunit.client.0.smithi088.stdout:7/357: dwrite d14/f3e [4194304,4194304] 0 2024-02-19T19:06:21.166 INFO:tasks.workunit.client.0.smithi088.stdout:2/249: write d3/f6 [4064384,21629] 0 2024-02-19T19:06:21.170 INFO:tasks.workunit.client.0.smithi088.stdout:7/358: link d14/d28/d3b/f40 d14/d1d/d3c/d43/d5c/f7f 0 2024-02-19T19:06:21.175 INFO:tasks.workunit.client.0.smithi088.stdout:2/250: rename d3/df to d3/d1f/d4f 0 2024-02-19T19:06:21.176 INFO:tasks.workunit.client.0.smithi088.stdout:7/359: dwrite d14/d28/d3b/f4b [0,4194304] 0 2024-02-19T19:06:21.184 INFO:tasks.workunit.client.0.smithi088.stdout:7/360: rename d14/d28/d65/f66 to d14/d1d/d3c/f80 0 2024-02-19T19:06:21.184 INFO:tasks.workunit.client.0.smithi088.stdout:7/361: dread - d14/f77 zero size 2024-02-19T19:06:21.187 INFO:tasks.workunit.client.0.smithi088.stdout:7/362: read d14/f30 [3025610,66385] 0 2024-02-19T19:06:21.188 INFO:tasks.workunit.client.0.smithi088.stdout:2/251: mknod d3/d1f/c50 0 2024-02-19T19:06:21.189 INFO:tasks.workunit.client.0.smithi088.stdout:2/252: truncate d3/d14/f4d 24736 0 2024-02-19T19:06:21.195 INFO:tasks.workunit.client.0.smithi088.stdout:2/253: dwrite d3/d14/f2b [0,4194304] 0 2024-02-19T19:06:21.203 INFO:tasks.workunit.client.0.smithi088.stdout:2/254: dwrite d3/f39 [0,4194304] 0 2024-02-19T19:06:21.207 INFO:tasks.workunit.client.0.smithi088.stdout:2/255: mknod d3/d1b/c51 0 2024-02-19T19:06:21.208 INFO:tasks.workunit.client.0.smithi088.stdout:2/256: chown d3/d1f/d4f/l37 10643147 1 2024-02-19T19:06:21.209 INFO:tasks.workunit.client.0.smithi088.stdout:7/363: dread fd [0,4194304] 0 2024-02-19T19:06:21.211 INFO:tasks.workunit.client.0.smithi088.stdout:7/364: creat d14/d28/d3b/f81 x:0 0 0 2024-02-19T19:06:21.212 INFO:tasks.workunit.client.0.smithi088.stdout:7/365: write d14/f74 [939420,18090] 0 2024-02-19T19:06:21.214 INFO:tasks.workunit.client.0.smithi088.stdout:7/366: mknod d14/d28/c82 0 2024-02-19T19:06:21.219 INFO:tasks.workunit.client.0.smithi088.stdout:7/367: unlink d14/d27/d49/c6f 0 2024-02-19T19:06:21.223 INFO:tasks.workunit.client.0.smithi088.stdout:7/368: link d14/d28/l3f d14/d1d/d3c/d43/d5c/d75/d76/d7c/l83 0 2024-02-19T19:06:21.224 INFO:tasks.workunit.client.0.smithi088.stdout:2/257: dwrite d3/d1f/f4a [0,4194304] 0 2024-02-19T19:06:21.227 INFO:tasks.workunit.client.0.smithi088.stdout:2/258: creat d3/d1b/d43/f52 x:0 0 0 2024-02-19T19:06:21.229 INFO:tasks.workunit.client.0.smithi088.stdout:7/369: dread d14/d28/d3b/f64 [0,4194304] 0 2024-02-19T19:06:21.229 INFO:tasks.workunit.client.0.smithi088.stdout:2/259: symlink d3/d14/l53 0 2024-02-19T19:06:21.230 INFO:tasks.workunit.client.0.smithi088.stdout:2/260: write d3/d1f/d4f/f11 [4548559,39168] 0 2024-02-19T19:06:21.232 INFO:tasks.workunit.client.0.smithi088.stdout:2/261: write d3/d1b/f42 [454699,51626] 0 2024-02-19T19:06:21.233 INFO:tasks.workunit.client.0.smithi088.stdout:2/262: chown d3/f31 1356392520 1 2024-02-19T19:06:21.234 INFO:tasks.workunit.client.0.smithi088.stdout:9/344: sync 2024-02-19T19:06:21.234 INFO:tasks.workunit.client.0.smithi088.stdout:1/435: sync 2024-02-19T19:06:21.234 INFO:tasks.workunit.client.0.smithi088.stdout:0/310: sync 2024-02-19T19:06:21.234 INFO:tasks.workunit.client.0.smithi088.stdout:8/287: sync 2024-02-19T19:06:21.234 INFO:tasks.workunit.client.0.smithi088.stdout:4/287: sync 2024-02-19T19:06:21.236 INFO:tasks.workunit.client.0.smithi088.stdout:7/370: rmdir d14/d1d/d3c/d43/d5c 39 2024-02-19T19:06:21.236 INFO:tasks.workunit.client.0.smithi088.stdout:8/288: stat d9/df/d1c/f13 0 2024-02-19T19:06:21.238 INFO:tasks.workunit.client.0.smithi088.stdout:2/263: rmdir d3/d14 39 2024-02-19T19:06:21.241 INFO:tasks.workunit.client.0.smithi088.stdout:8/289: dread d9/df/d1c/d2d/d47/f43 [0,4194304] 0 2024-02-19T19:06:21.241 INFO:tasks.workunit.client.0.smithi088.stdout:0/311: unlink d13/f44 0 2024-02-19T19:06:21.248 INFO:tasks.workunit.client.0.smithi088.stdout:7/371: symlink d14/d1d/d3c/l84 0 2024-02-19T19:06:21.248 INFO:tasks.workunit.client.0.smithi088.stdout:1/436: write d6/f1c [2145940,92573] 0 2024-02-19T19:06:21.252 INFO:tasks.workunit.client.0.smithi088.stdout:2/264: dread d3/d1b/f42 [0,4194304] 0 2024-02-19T19:06:21.252 INFO:tasks.workunit.client.0.smithi088.stdout:0/312: mkdir d13/d46/d5a/d25/d6d 0 2024-02-19T19:06:21.253 INFO:tasks.workunit.client.0.smithi088.stdout:8/290: mknod d9/df/d1c/d2d/d47/c51 0 2024-02-19T19:06:21.255 INFO:tasks.workunit.client.0.smithi088.stdout:8/291: write d9/df/d1c/f1f [2426297,51208] 0 2024-02-19T19:06:21.255 INFO:tasks.workunit.client.0.smithi088.stdout:1/437: truncate d6/d10/f1d 1901163 0 2024-02-19T19:06:21.255 INFO:tasks.workunit.client.0.smithi088.stdout:9/345: link d1/d8/d3e/d4e/c67 d1/d60/c6b 0 2024-02-19T19:06:21.256 INFO:tasks.workunit.client.0.smithi088.stdout:4/288: dread d2/d32/d25/f41 [12582912,4194304] 0 2024-02-19T19:06:21.257 INFO:tasks.workunit.client.0.smithi088.stdout:1/438: stat d6/d21/d64/d94/d71/f5c 0 2024-02-19T19:06:21.258 INFO:tasks.workunit.client.0.smithi088.stdout:9/346: chown d1/f20 205501 1 2024-02-19T19:06:21.258 INFO:tasks.workunit.client.0.smithi088.stdout:4/289: dread - d2/d32/f36 zero size 2024-02-19T19:06:21.259 INFO:tasks.workunit.client.0.smithi088.stdout:7/372: dread fc [0,4194304] 0 2024-02-19T19:06:21.261 INFO:tasks.workunit.client.0.smithi088.stdout:7/373: truncate d14/d28/d3b/f64 1117857 0 2024-02-19T19:06:21.261 INFO:tasks.workunit.client.0.smithi088.stdout:7/374: chown d14/f29 25 1 2024-02-19T19:06:21.263 INFO:tasks.workunit.client.0.smithi088.stdout:4/290: dread d2/d8/f14 [0,4194304] 0 2024-02-19T19:06:21.263 INFO:tasks.workunit.client.0.smithi088.stdout:1/439: chown d6/f1b 2124860 1 2024-02-19T19:06:21.264 INFO:tasks.workunit.client.0.smithi088.stdout:4/291: read - d2/d32/f36 zero size 2024-02-19T19:06:21.265 INFO:tasks.workunit.client.0.smithi088.stdout:0/313: write d13/f1b [1059807,111562] 0 2024-02-19T19:06:21.266 INFO:tasks.workunit.client.0.smithi088.stdout:8/292: creat d9/df/d3d/d50/f52 x:0 0 0 2024-02-19T19:06:21.269 INFO:tasks.workunit.client.0.smithi088.stdout:2/265: rename d3/f39 to d3/d1f/d4f/f54 0 2024-02-19T19:06:21.271 INFO:tasks.workunit.client.0.smithi088.stdout:1/440: mknod d6/d21/d6a/d7d/d7e/c95 0 2024-02-19T19:06:21.272 INFO:tasks.workunit.client.0.smithi088.stdout:4/292: creat d2/d32/d9/f56 x:0 0 0 2024-02-19T19:06:21.275 INFO:tasks.workunit.client.0.smithi088.stdout:7/375: mknod d14/d1d/d3c/d43/d5c/d75/d76/c85 0 2024-02-19T19:06:21.276 INFO:tasks.workunit.client.0.smithi088.stdout:7/376: truncate d14/d28/d3b/f81 45356 0 2024-02-19T19:06:21.276 INFO:tasks.workunit.client.0.smithi088.stdout:9/347: write d1/d8/d4a/f5e [816836,100835] 0 2024-02-19T19:06:21.278 INFO:tasks.workunit.client.0.smithi088.stdout:1/441: mknod d6/d10/c96 0 2024-02-19T19:06:21.280 INFO:tasks.workunit.client.0.smithi088.stdout:0/314: creat d13/d46/d5a/d25/d6d/f6e x:0 0 0 2024-02-19T19:06:21.285 INFO:tasks.workunit.client.0.smithi088.stdout:1/442: creat d6/d21/d64/d94/d6e/f97 x:0 0 0 2024-02-19T19:06:21.286 INFO:tasks.workunit.client.0.smithi088.stdout:2/266: creat d3/d1b/d43/f55 x:0 0 0 2024-02-19T19:06:21.288 INFO:tasks.workunit.client.0.smithi088.stdout:4/293: dwrite d2/d8/f48 [0,4194304] 0 2024-02-19T19:06:21.289 INFO:tasks.workunit.client.0.smithi088.stdout:9/348: creat d1/d60/f6c x:0 0 0 2024-02-19T19:06:21.289 INFO:tasks.workunit.client.0.smithi088.stdout:7/377: dread d14/f1f [0,4194304] 0 2024-02-19T19:06:21.289 INFO:tasks.workunit.client.0.smithi088.stdout:5/482: sync 2024-02-19T19:06:21.290 INFO:tasks.workunit.client.0.smithi088.stdout:3/449: sync 2024-02-19T19:06:21.292 INFO:tasks.workunit.client.0.smithi088.stdout:3/450: stat d4/d12/d33/d58 0 2024-02-19T19:06:21.293 INFO:tasks.workunit.client.0.smithi088.stdout:4/294: truncate d2/d32/d9/f56 343446 0 2024-02-19T19:06:21.293 INFO:tasks.workunit.client.0.smithi088.stdout:7/378: chown d14/d1d/d3c/d43/d5c/d75 0 1 2024-02-19T19:06:21.294 INFO:tasks.workunit.client.0.smithi088.stdout:8/293: rename d9/lb to d9/df/d1c/d2d/d47/l53 0 2024-02-19T19:06:21.295 INFO:tasks.workunit.client.0.smithi088.stdout:0/315: rmdir d13/d46/d5a 39 2024-02-19T19:06:21.299 INFO:tasks.workunit.client.0.smithi088.stdout:8/294: write d9/df/d1c/d2d/d47/f46 [471238,22154] 0 2024-02-19T19:06:21.299 INFO:tasks.workunit.client.0.smithi088.stdout:5/483: write d4/d1a/d41/d5e/f7f [567882,128166] 0 2024-02-19T19:06:21.302 INFO:tasks.workunit.client.0.smithi088.stdout:2/267: dwrite d3/f6 [4194304,4194304] 0 2024-02-19T19:06:21.302 INFO:tasks.workunit.client.0.smithi088.stdout:0/316: dread - d13/d46/d5a/d25/f58 zero size 2024-02-19T19:06:21.303 INFO:tasks.workunit.client.0.smithi088.stdout:2/268: write d3/d1b/f2e [88496,3706] 0 2024-02-19T19:06:21.304 INFO:tasks.workunit.client.0.smithi088.stdout:7/379: rename d14/d28/l41 to d14/d27/d6b/l86 0 2024-02-19T19:06:21.305 INFO:tasks.workunit.client.0.smithi088.stdout:4/295: symlink d2/d8/l57 0 2024-02-19T19:06:21.306 INFO:tasks.workunit.client.0.smithi088.stdout:7/380: write f2 [3779594,23717] 0 2024-02-19T19:06:21.307 INFO:tasks.workunit.client.0.smithi088.stdout:4/296: write d2/d35/f52 [769586,99857] 0 2024-02-19T19:06:21.307 INFO:tasks.workunit.client.0.smithi088.stdout:5/484: symlink d4/d61/d65/d9a/d9f/d57/la1 0 2024-02-19T19:06:21.309 INFO:tasks.workunit.client.0.smithi088.stdout:1/443: getdents d6 0 2024-02-19T19:06:21.310 INFO:tasks.workunit.client.0.smithi088.stdout:5/485: stat d4/d61/d65/d9a/d9f/d57/d84/f8a 0 2024-02-19T19:06:21.311 INFO:tasks.workunit.client.0.smithi088.stdout:8/295: rmdir d9 39 2024-02-19T19:06:21.312 INFO:tasks.workunit.client.0.smithi088.stdout:2/269: chown d3/d14/f4d 3506580 1 2024-02-19T19:06:21.314 INFO:tasks.workunit.client.0.smithi088.stdout:5/486: chown d4/d61/d65/d9a/d9f/d57/f6a 32853206 1 2024-02-19T19:06:21.314 INFO:tasks.workunit.client.0.smithi088.stdout:4/297: mkdir d2/d32/d25/d26/d58 0 2024-02-19T19:06:21.315 INFO:tasks.workunit.client.0.smithi088.stdout:0/317: mknod d13/d46/d5a/d25/c6f 0 2024-02-19T19:06:21.317 INFO:tasks.workunit.client.0.smithi088.stdout:7/381: dwrite d14/f22 [0,4194304] 0 2024-02-19T19:06:21.321 INFO:tasks.workunit.client.0.smithi088.stdout:7/382: creat d14/d1d/d3c/d43/d5c/d75/d76/f87 x:0 0 0 2024-02-19T19:06:21.322 INFO:tasks.workunit.client.0.smithi088.stdout:8/296: dread d9/df/d1c/d2d/d47/f46 [0,4194304] 0 2024-02-19T19:06:21.325 INFO:tasks.workunit.client.0.smithi088.stdout:5/487: truncate d4/d61/d65/d9a/d9f/d57/f6a 2237251 0 2024-02-19T19:06:21.326 INFO:tasks.workunit.client.0.smithi088.stdout:8/297: read d9/df/d1c/f25 [5601300,129546] 0 2024-02-19T19:06:21.328 INFO:tasks.workunit.client.0.smithi088.stdout:7/383: mknod d14/d28/d65/c88 0 2024-02-19T19:06:21.330 INFO:tasks.workunit.client.0.smithi088.stdout:7/384: write d14/f44 [2237293,125138] 0 2024-02-19T19:06:21.332 INFO:tasks.workunit.client.0.smithi088.stdout:2/270: link d3/d1f/f32 d3/d14/f56 0 2024-02-19T19:06:21.332 INFO:tasks.workunit.client.0.smithi088.stdout:5/488: dwrite d4/d7/f15 [0,4194304] 0 2024-02-19T19:06:21.333 INFO:tasks.workunit.client.0.smithi088.stdout:8/298: creat d9/df/d1c/d2d/f54 x:0 0 0 2024-02-19T19:06:21.335 INFO:tasks.workunit.client.0.smithi088.stdout:8/299: chown d9/df/d3d/d50/f52 3227190 1 2024-02-19T19:06:21.338 INFO:tasks.workunit.client.0.smithi088.stdout:2/271: mknod d3/d1f/d4f/c57 0 2024-02-19T19:06:21.339 INFO:tasks.workunit.client.0.smithi088.stdout:2/272: fsync d3/d1b/f2e 0 2024-02-19T19:06:21.339 INFO:tasks.workunit.client.0.smithi088.stdout:5/489: dread d4/f9d [0,4194304] 0 2024-02-19T19:06:21.340 INFO:tasks.workunit.client.0.smithi088.stdout:2/273: stat d3/d1f/d4f/f54 0 2024-02-19T19:06:21.341 INFO:tasks.workunit.client.0.smithi088.stdout:5/490: write d4/f42 [3300063,60283] 0 2024-02-19T19:06:21.344 INFO:tasks.workunit.client.0.smithi088.stdout:7/385: creat d14/d1d/d3c/d43/f89 x:0 0 0 2024-02-19T19:06:21.344 INFO:tasks.workunit.client.0.smithi088.stdout:4/298: truncate d2/d35/d39/d3a/f42 2676067 0 2024-02-19T19:06:21.349 INFO:tasks.workunit.client.0.smithi088.stdout:2/274: rename d3/d1f/d4f/f54 to d3/d1b/d43/f58 0 2024-02-19T19:06:21.349 INFO:tasks.workunit.client.0.smithi088.stdout:2/275: chown d3/d14/f1d 944 1 2024-02-19T19:06:21.350 INFO:tasks.workunit.client.0.smithi088.stdout:7/386: rmdir d14/d1d/d3c/d43/d5c 39 2024-02-19T19:06:21.351 INFO:tasks.workunit.client.0.smithi088.stdout:4/299: creat d2/d35/f59 x:0 0 0 2024-02-19T19:06:21.353 INFO:tasks.workunit.client.0.smithi088.stdout:2/276: mkdir d3/d1f/d4f/d59 0 2024-02-19T19:06:21.355 INFO:tasks.workunit.client.0.smithi088.stdout:4/300: symlink d2/d35/d39/d3a/l5a 0 2024-02-19T19:06:21.355 INFO:tasks.workunit.client.0.smithi088.stdout:7/387: dwrite d14/f4f [0,4194304] 0 2024-02-19T19:06:21.359 INFO:tasks.workunit.client.0.smithi088.stdout:4/301: readlink d2/d8/l57 0 2024-02-19T19:06:21.359 INFO:tasks.workunit.client.0.smithi088.stdout:4/302: chown d2/d32/d9/f56 1734961 1 2024-02-19T19:06:21.360 INFO:tasks.workunit.client.0.smithi088.stdout:2/277: dread d3/d1b/d43/f58 [0,4194304] 0 2024-02-19T19:06:21.361 INFO:tasks.workunit.client.0.smithi088.stdout:8/300: truncate d9/df/f1a 6685703 0 2024-02-19T19:06:21.363 INFO:tasks.workunit.client.0.smithi088.stdout:8/301: read d9/df/d1c/d2d/d47/f46 [15667,31756] 0 2024-02-19T19:06:21.363 INFO:tasks.workunit.client.0.smithi088.stdout:8/302: fdatasync f8 0 2024-02-19T19:06:21.364 INFO:tasks.workunit.client.0.smithi088.stdout:4/303: readlink d2/d32/d9/l1e 0 2024-02-19T19:06:21.365 INFO:tasks.workunit.client.0.smithi088.stdout:4/304: stat d2/d35/f37 0 2024-02-19T19:06:21.367 INFO:tasks.workunit.client.0.smithi088.stdout:2/278: creat d3/d14/f5a x:0 0 0 2024-02-19T19:06:21.367 INFO:tasks.workunit.client.0.smithi088.stdout:2/279: dread - d3/d14/f19 zero size 2024-02-19T19:06:21.369 INFO:tasks.workunit.client.0.smithi088.stdout:8/303: stat d9/df/d3d/l4e 0 2024-02-19T19:06:21.370 INFO:tasks.workunit.client.0.smithi088.stdout:8/304: fsync d9/f2e 0 2024-02-19T19:06:21.372 INFO:tasks.workunit.client.0.smithi088.stdout:7/388: rename fc to d14/d1d/f8a 0 2024-02-19T19:06:21.377 INFO:tasks.workunit.client.0.smithi088.stdout:7/389: dread - d14/d28/f35 zero size 2024-02-19T19:06:21.380 INFO:tasks.workunit.client.0.smithi088.stdout:4/305: rmdir d2/d35/d39/d51 0 2024-02-19T19:06:21.381 INFO:tasks.workunit.client.0.smithi088.stdout:2/280: truncate d3/d1b/f3e 455845 0 2024-02-19T19:06:21.383 INFO:tasks.workunit.client.0.smithi088.stdout:2/281: creat d3/d1b/f5b x:0 0 0 2024-02-19T19:06:21.384 INFO:tasks.workunit.client.0.smithi088.stdout:7/390: write d14/f2a [1544435,94519] 0 2024-02-19T19:06:21.385 INFO:tasks.workunit.client.0.smithi088.stdout:4/306: dread d2/d35/f4f [0,4194304] 0 2024-02-19T19:06:21.389 INFO:tasks.workunit.client.0.smithi088.stdout:4/307: stat d2/d8/c2f 0 2024-02-19T19:06:21.389 INFO:tasks.workunit.client.0.smithi088.stdout:8/305: dread f6 [0,4194304] 0 2024-02-19T19:06:21.391 INFO:tasks.workunit.client.0.smithi088.stdout:2/282: dread d3/f31 [0,4194304] 0 2024-02-19T19:06:21.393 INFO:tasks.workunit.client.0.smithi088.stdout:8/306: creat d9/df/f55 x:0 0 0 2024-02-19T19:06:21.394 INFO:tasks.workunit.client.0.smithi088.stdout:7/391: dwrite d14/f1f [0,4194304] 0 2024-02-19T19:06:21.399 INFO:tasks.workunit.client.0.smithi088.stdout:4/308: dwrite d2/d35/f53 [0,4194304] 0 2024-02-19T19:06:21.399 INFO:tasks.workunit.client.0.smithi088.stdout:7/392: dread d14/f74 [0,4194304] 0 2024-02-19T19:06:21.402 INFO:tasks.workunit.client.0.smithi088.stdout:4/309: creat d2/d35/f5b x:0 0 0 2024-02-19T19:06:21.403 INFO:tasks.workunit.client.0.smithi088.stdout:7/393: symlink d14/l8b 0 2024-02-19T19:06:21.404 INFO:tasks.workunit.client.0.smithi088.stdout:4/310: mknod d2/d8/c5c 0 2024-02-19T19:06:21.406 INFO:tasks.workunit.client.0.smithi088.stdout:7/394: truncate d14/d1d/d3c/d43/d5c/f69 583007 0 2024-02-19T19:06:21.411 INFO:tasks.workunit.client.0.smithi088.stdout:4/311: dwrite d2/d32/d9/f56 [0,4194304] 0 2024-02-19T19:06:21.418 INFO:tasks.workunit.client.0.smithi088.stdout:2/283: truncate d3/d1b/f1c 1960918 0 2024-02-19T19:06:21.422 INFO:tasks.workunit.client.0.smithi088.stdout:7/395: rename d14/c73 to d14/d28/d65/c8c 0 2024-02-19T19:06:21.422 INFO:tasks.workunit.client.0.smithi088.stdout:2/284: chown d3/d1f/d4f/l30 3 1 2024-02-19T19:06:21.423 INFO:tasks.workunit.client.0.smithi088.stdout:2/285: write d3/d1b/d43/f52 [586633,52964] 0 2024-02-19T19:06:21.429 INFO:tasks.workunit.client.0.smithi088.stdout:2/286: creat d3/d14/f5c x:0 0 0 2024-02-19T19:06:21.430 INFO:tasks.workunit.client.0.smithi088.stdout:4/312: link d2/d32/d9/c1b d2/d32/d25/d45/c5d 0 2024-02-19T19:06:21.430 INFO:tasks.workunit.client.0.smithi088.stdout:8/307: truncate d9/df/d1c/f33 2970302 0 2024-02-19T19:06:21.431 INFO:tasks.workunit.client.0.smithi088.stdout:8/308: write d9/f4a [892386,2714] 0 2024-02-19T19:06:21.432 INFO:tasks.workunit.client.0.smithi088.stdout:2/287: mkdir d3/d1f/d5d 0 2024-02-19T19:06:21.432 INFO:tasks.workunit.client.0.smithi088.stdout:8/309: chown d9/df/d1c/f13 408 1 2024-02-19T19:06:21.433 INFO:tasks.workunit.client.0.smithi088.stdout:4/313: creat d2/d32/d25/d26/f5e x:0 0 0 2024-02-19T19:06:21.435 INFO:tasks.workunit.client.0.smithi088.stdout:4/314: readlink d2/d35/d39/d3a/l3c 0 2024-02-19T19:06:21.435 INFO:tasks.workunit.client.0.smithi088.stdout:2/288: chown d3/d1f/l34 134822496 1 2024-02-19T19:06:21.435 INFO:tasks.workunit.client.0.smithi088.stdout:8/310: write d9/df/d1c/d2d/d47/f4d [633760,36688] 0 2024-02-19T19:06:21.436 INFO:tasks.workunit.client.0.smithi088.stdout:4/315: rmdir d2/d35/d39/d3a 39 2024-02-19T19:06:21.438 INFO:tasks.workunit.client.0.smithi088.stdout:2/289: fdatasync d3/d14/f1d 0 2024-02-19T19:06:21.442 INFO:tasks.workunit.client.0.smithi088.stdout:4/316: creat d2/d35/d39/d3a/f5f x:0 0 0 2024-02-19T19:06:21.443 INFO:tasks.workunit.client.0.smithi088.stdout:4/317: stat d2/d32/d9/f56 0 2024-02-19T19:06:21.450 INFO:tasks.workunit.client.0.smithi088.stdout:4/318: rmdir d2/d32/d9 39 2024-02-19T19:06:21.454 INFO:tasks.workunit.client.0.smithi088.stdout:7/396: write d14/d1d/d3c/d43/d5c/f7f [565653,68991] 0 2024-02-19T19:06:21.457 INFO:tasks.workunit.client.0.smithi088.stdout:8/311: dread d9/df/d1c/f1f [0,4194304] 0 2024-02-19T19:06:21.458 INFO:tasks.workunit.client.0.smithi088.stdout:4/319: mknod d2/d32/c60 0 2024-02-19T19:06:21.461 INFO:tasks.workunit.client.0.smithi088.stdout:4/320: write d2/d32/f18 [3853588,54553] 0 2024-02-19T19:06:21.463 INFO:tasks.workunit.client.0.smithi088.stdout:7/397: mknod d14/d1d/d3c/d43/c8d 0 2024-02-19T19:06:21.464 INFO:tasks.workunit.client.0.smithi088.stdout:8/312: symlink d9/l56 0 2024-02-19T19:06:21.471 INFO:tasks.workunit.client.0.smithi088.stdout:4/321: creat d2/d8/f61 x:0 0 0 2024-02-19T19:06:21.472 INFO:tasks.workunit.client.0.smithi088.stdout:7/398: creat d14/d1d/d3c/f8e x:0 0 0 2024-02-19T19:06:21.473 INFO:tasks.workunit.client.0.smithi088.stdout:7/399: chown d14/f29 317202931 1 2024-02-19T19:06:21.476 INFO:tasks.workunit.client.0.smithi088.stdout:4/322: creat d2/d32/d25/d26/f62 x:0 0 0 2024-02-19T19:06:21.481 INFO:tasks.workunit.client.0.smithi088.stdout:4/323: chown d2/d32/d9/c2a 1036139 1 2024-02-19T19:06:21.481 INFO:tasks.workunit.client.0.smithi088.stdout:7/400: unlink d14/d1d/d3c/d43/d5c/d75/d76/f87 0 2024-02-19T19:06:21.486 INFO:tasks.workunit.client.0.smithi088.stdout:4/324: dread d2/d32/d9/f56 [0,4194304] 0 2024-02-19T19:06:21.488 INFO:tasks.workunit.client.0.smithi088.stdout:7/401: creat d14/d1d/d3c/f8f x:0 0 0 2024-02-19T19:06:21.489 INFO:tasks.workunit.client.0.smithi088.stdout:8/313: dread d9/f2e [0,4194304] 0 2024-02-19T19:06:21.492 INFO:tasks.workunit.client.0.smithi088.stdout:7/402: write d14/d1d/d3c/f6c [671683,64952] 0 2024-02-19T19:06:21.494 INFO:tasks.workunit.client.0.smithi088.stdout:4/325: mknod d2/c63 0 2024-02-19T19:06:21.500 INFO:tasks.workunit.client.0.smithi088.stdout:4/326: unlink d2/d32/f23 0 2024-02-19T19:06:21.503 INFO:tasks.workunit.client.0.smithi088.stdout:8/314: creat d9/df/d1c/d2d/d47/f57 x:0 0 0 2024-02-19T19:06:21.503 INFO:tasks.workunit.client.0.smithi088.stdout:7/403: dwrite d14/d1d/d3c/d43/d5c/f79 [0,4194304] 0 2024-02-19T19:06:21.508 INFO:tasks.workunit.client.0.smithi088.stdout:8/315: write d9/df/d3d/d50/f52 [783924,70673] 0 2024-02-19T19:06:21.509 INFO:tasks.workunit.client.0.smithi088.stdout:7/404: read d14/d1d/f8a [920443,91062] 0 2024-02-19T19:06:21.511 INFO:tasks.workunit.client.0.smithi088.stdout:4/327: mknod d2/d32/d25/d4b/c64 0 2024-02-19T19:06:21.512 INFO:tasks.workunit.client.0.smithi088.stdout:4/328: read d2/d8/f48 [3010967,99199] 0 2024-02-19T19:06:21.514 INFO:tasks.workunit.client.0.smithi088.stdout:7/405: creat d14/d1d/d3c/d43/d5c/d75/d76/f90 x:0 0 0 2024-02-19T19:06:21.515 INFO:tasks.workunit.client.0.smithi088.stdout:7/406: read - d14/f6e zero size 2024-02-19T19:06:21.517 INFO:tasks.workunit.client.0.smithi088.stdout:4/329: symlink d2/d32/d25/d45/l65 0 2024-02-19T19:06:21.518 INFO:tasks.workunit.client.0.smithi088.stdout:4/330: dread - d2/d35/f5b zero size 2024-02-19T19:06:21.518 INFO:tasks.workunit.client.0.smithi088.stdout:4/331: fsync d2/d32/f1c 0 2024-02-19T19:06:21.519 INFO:tasks.workunit.client.0.smithi088.stdout:7/407: dread d14/d28/d3b/f40 [0,4194304] 0 2024-02-19T19:06:21.520 INFO:tasks.workunit.client.0.smithi088.stdout:4/332: truncate d2/d35/f59 904657 0 2024-02-19T19:06:21.522 INFO:tasks.workunit.client.0.smithi088.stdout:7/408: mknod d14/d1d/c91 0 2024-02-19T19:06:21.524 INFO:tasks.workunit.client.0.smithi088.stdout:4/333: truncate d2/d32/d25/f46 115835 0 2024-02-19T19:06:21.526 INFO:tasks.workunit.client.0.smithi088.stdout:4/334: chown d2/d8/l22 14 1 2024-02-19T19:06:21.526 INFO:tasks.workunit.client.0.smithi088.stdout:4/335: stat d2/d8/c5c 0 2024-02-19T19:06:21.527 INFO:tasks.workunit.client.0.smithi088.stdout:7/409: mkdir d14/d1d/d3c/d43/d5c/d75/d76/d7c/d92 0 2024-02-19T19:06:21.528 INFO:tasks.workunit.client.0.smithi088.stdout:4/336: symlink d2/d32/d25/d45/l66 0 2024-02-19T19:06:21.529 INFO:tasks.workunit.client.0.smithi088.stdout:7/410: creat d14/d27/d6b/f93 x:0 0 0 2024-02-19T19:06:21.531 INFO:tasks.workunit.client.0.smithi088.stdout:6/366: sync 2024-02-19T19:06:21.535 INFO:tasks.workunit.client.0.smithi088.stdout:4/337: dwrite d2/d32/d9/f20 [0,4194304] 0 2024-02-19T19:06:21.535 INFO:tasks.workunit.client.0.smithi088.stdout:7/411: dread d14/d1d/d3c/d43/d5c/f79 [0,4194304] 0 2024-02-19T19:06:21.537 INFO:tasks.workunit.client.0.smithi088.stdout:7/412: write d14/f1f [3393302,128896] 0 2024-02-19T19:06:21.537 INFO:tasks.workunit.client.0.smithi088.stdout:6/367: unlink d5/d8/d2c/f67 0 2024-02-19T19:06:21.542 INFO:tasks.workunit.client.0.smithi088.stdout:7/413: dwrite d14/f54 [0,4194304] 0 2024-02-19T19:06:21.627 INFO:tasks.workunit.client.0.smithi088.stdout:4/338: chown d2/d32/c2b 0 1 2024-02-19T19:06:21.630 INFO:tasks.workunit.client.0.smithi088.stdout:6/368: mkdir d5/d19/d70 0 2024-02-19T19:06:21.635 INFO:tasks.workunit.client.0.smithi088.stdout:7/414: rename d14/d56 to d14/d28/d94 0 2024-02-19T19:06:21.639 INFO:tasks.workunit.client.0.smithi088.stdout:7/415: dread d14/f22 [0,4194304] 0 2024-02-19T19:06:21.640 INFO:tasks.workunit.client.0.smithi088.stdout:4/339: rename d2/c3 to d2/d35/c67 0 2024-02-19T19:06:21.641 INFO:tasks.workunit.client.0.smithi088.stdout:4/340: chown d2/d32/d9/l1e 26460 1 2024-02-19T19:06:21.643 INFO:tasks.workunit.client.0.smithi088.stdout:4/341: mknod d2/d32/d25/d26/c68 0 2024-02-19T19:06:21.643 INFO:tasks.workunit.client.0.smithi088.stdout:4/342: fsync d2/d32/d25/f30 0 2024-02-19T19:06:21.646 INFO:tasks.workunit.client.0.smithi088.stdout:4/343: creat d2/d32/d25/d26/d58/f69 x:0 0 0 2024-02-19T19:06:21.648 INFO:tasks.workunit.client.0.smithi088.stdout:4/344: fsync d2/d8/f48 0 2024-02-19T19:06:21.648 INFO:tasks.workunit.client.0.smithi088.stdout:6/369: write d5/d31/d25/f39 [1853403,70690] 0 2024-02-19T19:06:21.650 INFO:tasks.workunit.client.0.smithi088.stdout:7/416: truncate d14/f1f 1189670 0 2024-02-19T19:06:21.652 INFO:tasks.workunit.client.0.smithi088.stdout:6/370: mkdir d5/d8/d3a/d71 0 2024-02-19T19:06:21.654 INFO:tasks.workunit.client.0.smithi088.stdout:4/345: truncate d2/d32/d25/d26/f49 1342923 0 2024-02-19T19:06:21.656 INFO:tasks.workunit.client.0.smithi088.stdout:6/371: mkdir d5/d19/d26/d72 0 2024-02-19T19:06:21.661 INFO:tasks.workunit.client.0.smithi088.stdout:4/346: getdents d2/d35 0 2024-02-19T19:06:21.662 INFO:tasks.workunit.client.0.smithi088.stdout:6/372: getdents d5/de/d64 0 2024-02-19T19:06:21.662 INFO:tasks.workunit.client.0.smithi088.stdout:6/373: fsync d5/d8/d2c/f4c 0 2024-02-19T19:06:21.666 INFO:tasks.workunit.client.0.smithi088.stdout:4/347: creat d2/d8/f6a x:0 0 0 2024-02-19T19:06:21.667 INFO:tasks.workunit.client.0.smithi088.stdout:6/374: creat d5/d19/d70/f73 x:0 0 0 2024-02-19T19:06:21.667 INFO:tasks.workunit.client.0.smithi088.stdout:4/348: rmdir d2/d35/d39 39 2024-02-19T19:06:21.668 INFO:tasks.workunit.client.0.smithi088.stdout:4/349: read - d2/d32/d25/d26/d58/f69 zero size 2024-02-19T19:06:21.669 INFO:tasks.workunit.client.0.smithi088.stdout:6/375: creat d5/d8/d3a/d71/f74 x:0 0 0 2024-02-19T19:06:21.671 INFO:tasks.workunit.client.0.smithi088.stdout:7/417: dread fb [4194304,4194304] 0 2024-02-19T19:06:21.672 INFO:tasks.workunit.client.0.smithi088.stdout:6/376: creat d5/d45/f75 x:0 0 0 2024-02-19T19:06:21.673 INFO:tasks.workunit.client.0.smithi088.stdout:7/418: dread d14/d1d/d3c/d43/d5c/f69 [0,4194304] 0 2024-02-19T19:06:21.676 INFO:tasks.workunit.client.0.smithi088.stdout:7/419: mkdir d14/d27/d95 0 2024-02-19T19:06:21.677 INFO:tasks.workunit.client.0.smithi088.stdout:7/420: write d14/d1d/f38 [3456171,12799] 0 2024-02-19T19:06:21.677 INFO:tasks.workunit.client.0.smithi088.stdout:7/421: chown d14/d27/l50 32339 1 2024-02-19T19:06:21.678 INFO:tasks.workunit.client.0.smithi088.stdout:6/377: creat d5/de/d60/f76 x:0 0 0 2024-02-19T19:06:21.679 INFO:tasks.workunit.client.0.smithi088.stdout:4/350: write d2/d35/d39/d3a/f42 [176553,15610] 0 2024-02-19T19:06:21.680 INFO:tasks.workunit.client.0.smithi088.stdout:6/378: write d5/d19/d70/f73 [383930,123297] 0 2024-02-19T19:06:21.684 INFO:tasks.workunit.client.0.smithi088.stdout:6/379: fdatasync d5/d19/f5f 0 2024-02-19T19:06:21.685 INFO:tasks.workunit.client.0.smithi088.stdout:4/351: creat d2/d8/f6b x:0 0 0 2024-02-19T19:06:21.691 INFO:tasks.workunit.client.0.smithi088.stdout:4/352: creat d2/d35/d39/f6c x:0 0 0 2024-02-19T19:06:21.692 INFO:tasks.workunit.client.0.smithi088.stdout:4/353: dread - d2/d32/d25/d26/f62 zero size 2024-02-19T19:06:21.692 INFO:tasks.workunit.client.0.smithi088.stdout:4/354: readlink d2/d32/d25/l34 0 2024-02-19T19:06:21.693 INFO:tasks.workunit.client.0.smithi088.stdout:6/380: truncate d5/d7/d32/f53 4019512 0 2024-02-19T19:06:21.694 INFO:tasks.workunit.client.0.smithi088.stdout:6/381: readlink d5/d8/d3a/l65 0 2024-02-19T19:06:21.696 INFO:tasks.workunit.client.0.smithi088.stdout:6/382: mkdir d5/d8/d77 0 2024-02-19T19:06:21.698 INFO:tasks.workunit.client.0.smithi088.stdout:6/383: mkdir d5/d8/d2c/d78 0 2024-02-19T19:06:21.698 INFO:tasks.workunit.client.0.smithi088.stdout:7/422: dread d14/d1d/d3c/d43/d5c/f68 [0,4194304] 0 2024-02-19T19:06:21.699 INFO:tasks.workunit.client.0.smithi088.stdout:7/423: dread d14/f74 [0,4194304] 0 2024-02-19T19:06:21.701 INFO:tasks.workunit.client.0.smithi088.stdout:7/424: mknod d14/d28/d65/c96 0 2024-02-19T19:06:21.703 INFO:tasks.workunit.client.0.smithi088.stdout:7/425: stat d14/d28/l3f 0 2024-02-19T19:06:21.704 INFO:tasks.workunit.client.0.smithi088.stdout:6/384: dwrite d5/de/f18 [4194304,4194304] 0 2024-02-19T19:06:21.705 INFO:tasks.workunit.client.0.smithi088.stdout:6/385: readlink d5/d19/l57 0 2024-02-19T19:06:21.706 INFO:tasks.workunit.client.0.smithi088.stdout:6/386: write d5/d7/f4b [318470,88028] 0 2024-02-19T19:06:21.707 INFO:tasks.workunit.client.0.smithi088.stdout:7/426: creat d14/d1d/d3c/d43/d5c/d75/d76/d7c/d92/f97 x:0 0 0 2024-02-19T19:06:21.708 INFO:tasks.workunit.client.0.smithi088.stdout:7/427: readlink d14/d27/l55 0 2024-02-19T19:06:21.709 INFO:tasks.workunit.client.0.smithi088.stdout:6/387: read d5/d8/d2c/f4c [797860,36399] 0 2024-02-19T19:06:21.710 INFO:tasks.workunit.client.0.smithi088.stdout:7/428: mkdir d14/d27/d49/d98 0 2024-02-19T19:06:21.711 INFO:tasks.workunit.client.0.smithi088.stdout:7/429: truncate d14/d28/d3b/f40 4813009 0 2024-02-19T19:06:21.712 INFO:tasks.workunit.client.0.smithi088.stdout:7/430: write f6 [3559657,117028] 0 2024-02-19T19:06:21.713 INFO:tasks.workunit.client.0.smithi088.stdout:7/431: fsync d14/f1c 0 2024-02-19T19:06:21.713 INFO:tasks.workunit.client.0.smithi088.stdout:7/432: chown d14/f22 408643 1 2024-02-19T19:06:21.715 INFO:tasks.workunit.client.0.smithi088.stdout:7/433: write d14/d28/d3b/f52 [4789242,113621] 0 2024-02-19T19:06:21.716 INFO:tasks.workunit.client.0.smithi088.stdout:7/434: write d14/d27/d49/f7d [933323,64925] 0 2024-02-19T19:06:21.722 INFO:tasks.workunit.client.0.smithi088.stdout:7/435: dwrite d14/d1d/d3c/d43/d5c/f68 [0,4194304] 0 2024-02-19T19:06:21.723 INFO:tasks.workunit.client.0.smithi088.stdout:7/436: chown d14/d27 19 1 2024-02-19T19:06:21.723 INFO:tasks.workunit.client.0.smithi088.stdout:7/437: readlink d14/d27/l55 0 2024-02-19T19:06:21.730 INFO:tasks.workunit.client.0.smithi088.stdout:7/438: dwrite d14/d1d/d3c/f80 [0,4194304] 0 2024-02-19T19:06:21.732 INFO:tasks.workunit.client.0.smithi088.stdout:7/439: dread d14/f74 [0,4194304] 0 2024-02-19T19:06:21.850 INFO:tasks.workunit.client.0.smithi088.stdout:9/349: sync 2024-02-19T19:06:21.854 INFO:tasks.workunit.client.0.smithi088.stdout:9/350: dwrite d1/fc [0,4194304] 0 2024-02-19T19:06:21.855 INFO:tasks.workunit.client.0.smithi088.stdout:9/351: write d1/d8/d3e/f45 [294066,102600] 0 2024-02-19T19:06:21.858 INFO:tasks.workunit.client.0.smithi088.stdout:9/352: symlink d1/d8/d37/d54/l6d 0 2024-02-19T19:06:21.861 INFO:tasks.workunit.client.0.smithi088.stdout:9/353: mknod d1/d21/d2d/d18/d1f/c6e 0 2024-02-19T19:06:21.923 INFO:tasks.workunit.client.0.smithi088.stdout:9/354: dwrite d1/d8/f42 [0,4194304] 0 2024-02-19T19:06:21.927 INFO:tasks.workunit.client.0.smithi088.stdout:9/355: mkdir d1/d21/d6f 0 2024-02-19T19:06:21.930 INFO:tasks.workunit.client.0.smithi088.stdout:9/356: creat d1/d8/d3e/f70 x:0 0 0 2024-02-19T19:06:21.930 INFO:tasks.workunit.client.0.smithi088.stdout:9/357: readlink d1/d21/d2d/d23/l30 0 2024-02-19T19:06:21.942 INFO:tasks.workunit.client.0.smithi088.stdout:9/358: truncate d1/f6 2513844 0 2024-02-19T19:06:21.947 INFO:tasks.workunit.client.0.smithi088.stdout:9/359: dwrite d1/d21/f43 [0,4194304] 0 2024-02-19T19:06:21.948 INFO:tasks.workunit.client.0.smithi088.stdout:9/360: fdatasync d1/d13/f36 0 2024-02-19T19:06:21.952 INFO:tasks.workunit.client.0.smithi088.stdout:9/361: symlink d1/d60/l71 0 2024-02-19T19:06:21.957 INFO:tasks.workunit.client.0.smithi088.stdout:9/362: link d1/d21/d2d/l39 d1/d8/d3e/d4e/l72 0 2024-02-19T19:06:21.957 INFO:tasks.workunit.client.0.smithi088.stdout:9/363: chown d1/d21/d2d/d23/f26 0 1 2024-02-19T19:06:21.962 INFO:tasks.workunit.client.0.smithi088.stdout:9/364: dwrite d1/d8/d37/f41 [0,4194304] 0 2024-02-19T19:06:21.963 INFO:tasks.workunit.client.0.smithi088.stdout:9/365: chown d1/f10 221310 1 2024-02-19T19:06:21.964 INFO:tasks.workunit.client.0.smithi088.stdout:9/366: mknod d1/d8/d37/c73 0 2024-02-19T19:06:21.966 INFO:tasks.workunit.client.0.smithi088.stdout:9/367: mkdir d1/d8/d3e/d4e/d74 0 2024-02-19T19:06:21.976 INFO:tasks.workunit.client.0.smithi088.stdout:9/368: truncate d1/d21/d2d/d23/f26 103418 0 2024-02-19T19:06:21.979 INFO:tasks.workunit.client.0.smithi088.stdout:9/369: truncate d1/d8/d37/d54/f65 823536 0 2024-02-19T19:06:21.984 INFO:tasks.workunit.client.0.smithi088.stdout:9/370: dwrite d1/d21/f64 [0,4194304] 0 2024-02-19T19:06:21.986 INFO:tasks.workunit.client.0.smithi088.stdout:9/371: readlink d1/d21/d2d/l39 0 2024-02-19T19:06:21.987 INFO:tasks.workunit.client.0.smithi088.stdout:9/372: creat d1/d8/d37/d54/f75 x:0 0 0 2024-02-19T19:06:21.988 INFO:tasks.workunit.client.0.smithi088.stdout:9/373: read d1/d21/f43 [1426526,12628] 0 2024-02-19T19:06:21.989 INFO:tasks.workunit.client.0.smithi088.stdout:9/374: write d1/d13/f36 [960563,107803] 0 2024-02-19T19:06:21.990 INFO:tasks.workunit.client.0.smithi088.stdout:9/375: rename d1/d8/d37/d54 to d1/d8/d37/d54/d76 22 2024-02-19T19:06:21.996 INFO:tasks.workunit.client.0.smithi088.stdout:9/376: dwrite d1/d13/f53 [0,4194304] 0 2024-02-19T19:06:21.996 INFO:tasks.workunit.client.0.smithi088.stdout:3/451: sync 2024-02-19T19:06:22.001 INFO:tasks.workunit.client.0.smithi088.stdout:9/377: rename d1/d21/d2f/f61 to d1/d8/f77 0 2024-02-19T19:06:22.002 INFO:tasks.workunit.client.0.smithi088.stdout:9/378: fsync d1/d8/f28 0 2024-02-19T19:06:22.002 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:21 smithi132 ceph-mon[70676]: from='client.34204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:22.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:21 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2379174255' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:06:22.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:21 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:22.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:21 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:22.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:21 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.afhgdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:06:22.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:21 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:22.003 INFO:tasks.workunit.client.0.smithi088.stdout:9/379: chown d1/d8/d4a/f58 4 1 2024-02-19T19:06:22.004 INFO:tasks.workunit.client.0.smithi088.stdout:9/380: stat d1/d8/d37/d54/f65 0 2024-02-19T19:06:22.006 INFO:tasks.workunit.client.0.smithi088.stdout:3/452: write d4/d12/d33/d58/f78 [330126,33637] 0 2024-02-19T19:06:22.009 INFO:tasks.workunit.client.0.smithi088.stdout:9/381: getdents d1 0 2024-02-19T19:06:22.011 INFO:tasks.workunit.client.0.smithi088.stdout:9/382: read - d1/d8/d37/d54/f75 zero size 2024-02-19T19:06:22.013 INFO:tasks.workunit.client.0.smithi088.stdout:3/453: dwrite d4/d12/f6c [0,4194304] 0 2024-02-19T19:06:22.014 INFO:tasks.workunit.client.0.smithi088.stdout:3/454: stat d4/d12 0 2024-02-19T19:06:22.021 INFO:tasks.workunit.client.0.smithi088.stdout:9/383: dwrite d1/f6 [0,4194304] 0 2024-02-19T19:06:22.023 INFO:tasks.workunit.client.0.smithi088.stdout:3/455: dwrite d4/d22/d7a/f9f [0,4194304] 0 2024-02-19T19:06:22.024 INFO:tasks.workunit.client.0.smithi088.stdout:9/384: mknod d1/c78 0 2024-02-19T19:06:22.032 INFO:tasks.workunit.client.0.smithi088.stdout:9/385: mkdir d1/d21/d2d/d23/d79 0 2024-02-19T19:06:22.033 INFO:tasks.workunit.client.0.smithi088.stdout:3/456: dwrite d4/f18 [0,4194304] 0 2024-02-19T19:06:22.036 INFO:tasks.workunit.client.0.smithi088.stdout:3/457: dread d4/ff [0,4194304] 0 2024-02-19T19:06:22.037 INFO:tasks.workunit.client.0.smithi088.stdout:9/386: creat d1/d8/d4a/d4f/f7a x:0 0 0 2024-02-19T19:06:22.038 INFO:tasks.workunit.client.0.smithi088.stdout:9/387: chown d1/d8/d37/d54/l6d 6 1 2024-02-19T19:06:22.038 INFO:tasks.workunit.client.0.smithi088.stdout:9/388: chown d1/d8/d3e/l69 494682065 1 2024-02-19T19:06:22.040 INFO:tasks.workunit.client.0.smithi088.stdout:3/458: getdents d4/d12/d33/d58 0 2024-02-19T19:06:22.041 INFO:tasks.workunit.client.0.smithi088.stdout:9/389: creat d1/d21/d2f/f7b x:0 0 0 2024-02-19T19:06:22.043 INFO:tasks.workunit.client.0.smithi088.stdout:3/459: rmdir d4/d22 39 2024-02-19T19:06:22.046 INFO:tasks.workunit.client.0.smithi088.stdout:3/460: mknod d4/d1a/d1e/ca0 0 2024-02-19T19:06:22.049 INFO:tasks.workunit.client.0.smithi088.stdout:9/390: dwrite d1/d21/d2d/f16 [4194304,4194304] 0 2024-02-19T19:06:22.049 INFO:tasks.workunit.client.0.smithi088.stdout:3/461: mkdir d4/d22/d7a/da1 0 2024-02-19T19:06:22.050 INFO:tasks.workunit.client.0.smithi088.stdout:3/462: stat d4/d12 0 2024-02-19T19:06:22.052 INFO:tasks.workunit.client.0.smithi088.stdout:9/391: symlink d1/l7c 0 2024-02-19T19:06:22.053 INFO:tasks.workunit.client.0.smithi088.stdout:9/392: fdatasync d1/d8/d4a/f5e 0 2024-02-19T19:06:22.059 INFO:tasks.workunit.client.0.smithi088.stdout:3/463: read d4/d1a/d1e/d31/d32/f50 [2504950,89291] 0 2024-02-19T19:06:22.059 INFO:tasks.workunit.client.0.smithi088.stdout:9/393: dwrite d1/d8/d4a/f58 [0,4194304] 0 2024-02-19T19:06:22.061 INFO:tasks.workunit.client.0.smithi088.stdout:3/464: creat d4/d12/d33/fa2 x:0 0 0 2024-02-19T19:06:22.062 INFO:tasks.workunit.client.0.smithi088.stdout:1/444: sync 2024-02-19T19:06:22.063 INFO:tasks.workunit.client.0.smithi088.stdout:9/394: rename d1/d21/d2f/c3a to d1/d21/d2d/d18/d1f/c7d 0 2024-02-19T19:06:22.065 INFO:tasks.workunit.client.0.smithi088.stdout:9/395: readlink d1/l46 0 2024-02-19T19:06:22.067 INFO:tasks.workunit.client.0.smithi088.stdout:1/445: truncate d6/d21/d6a/d4a/f59 556349 0 2024-02-19T19:06:22.068 INFO:tasks.workunit.client.0.smithi088.stdout:1/446: chown d6/d10 1429 1 2024-02-19T19:06:22.069 INFO:tasks.workunit.client.0.smithi088.stdout:1/447: dread - d6/d21/f6c zero size 2024-02-19T19:06:22.069 INFO:tasks.workunit.client.0.smithi088.stdout:9/396: truncate d1/d8/f28 3788632 0 2024-02-19T19:06:22.069 INFO:tasks.workunit.client.0.smithi088.stdout:1/448: dread - d6/d21/f84 zero size 2024-02-19T19:06:22.071 INFO:tasks.workunit.client.0.smithi088.stdout:3/465: dread d4/d12/d33/d58/f78 [0,4194304] 0 2024-02-19T19:06:22.072 INFO:tasks.workunit.client.0.smithi088.stdout:9/397: symlink d1/d21/l7e 0 2024-02-19T19:06:22.073 INFO:tasks.workunit.client.0.smithi088.stdout:9/398: dread - d1/d8/d4a/d4f/f7a zero size 2024-02-19T19:06:22.074 INFO:tasks.workunit.client.0.smithi088.stdout:3/466: fsync d4/d12/d33/d58/f78 0 2024-02-19T19:06:22.076 INFO:tasks.workunit.client.0.smithi088.stdout:1/449: link d6/d21/d6a/d7d/f8b d6/d21/d64/d94/d71/f98 0 2024-02-19T19:06:22.079 INFO:tasks.workunit.client.0.smithi088.stdout:1/450: dread d6/d21/f72 [0,4194304] 0 2024-02-19T19:06:22.080 INFO:tasks.workunit.client.0.smithi088.stdout:3/467: mknod d4/d1a/d7f/d88/ca3 0 2024-02-19T19:06:22.085 INFO:tasks.workunit.client.0.smithi088.stdout:9/399: link d1/d21/d2d/d23/f26 d1/f7f 0 2024-02-19T19:06:22.085 INFO:tasks.workunit.client.0.smithi088.stdout:1/451: dwrite d6/f1b [0,4194304] 0 2024-02-19T19:06:22.089 INFO:tasks.workunit.client.0.smithi088.stdout:1/452: write d6/d21/f70 [1340006,27617] 0 2024-02-19T19:06:22.092 INFO:tasks.workunit.client.0.smithi088.stdout:3/468: write d4/d1a/d1e/d74/f8a [195746,79786] 0 2024-02-19T19:06:22.095 INFO:tasks.workunit.client.0.smithi088.stdout:1/453: creat d6/f99 x:0 0 0 2024-02-19T19:06:22.095 INFO:tasks.workunit.client.0.smithi088.stdout:9/400: getdents d1/d8/d3e/d4e 0 2024-02-19T19:06:22.097 INFO:tasks.workunit.client.0.smithi088.stdout:1/454: stat d6/d10/l85 0 2024-02-19T19:06:22.097 INFO:tasks.workunit.client.0.smithi088.stdout:9/401: truncate d1/d13/f12 4491403 0 2024-02-19T19:06:22.098 INFO:tasks.workunit.client.0.smithi088.stdout:1/455: fsync d6/d21/f6c 0 2024-02-19T19:06:22.100 INFO:tasks.workunit.client.0.smithi088.stdout:9/402: write d1/f10 [1418047,9996] 0 2024-02-19T19:06:22.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:21 smithi088 ceph-mon[88643]: from='client.34204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:22.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:21 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2379174255' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:06:22.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:21 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:22.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:21 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:22.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:21 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.afhgdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:06:22.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:21 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:22.101 INFO:tasks.workunit.client.0.smithi088.stdout:1/456: mkdir d6/d21/d6a/d7d/d9a 0 2024-02-19T19:06:22.104 INFO:tasks.workunit.client.0.smithi088.stdout:9/403: unlink d1/d8/d3e/d4e/l4b 0 2024-02-19T19:06:22.107 INFO:tasks.workunit.client.0.smithi088.stdout:1/457: link d6/fd d6/d21/d64/d94/d71/f9b 0 2024-02-19T19:06:22.108 INFO:tasks.workunit.client.0.smithi088.stdout:3/469: dread d4/d1a/d23/f5d [0,4194304] 0 2024-02-19T19:06:22.108 INFO:tasks.workunit.client.0.smithi088.stdout:9/404: creat d1/d60/f80 x:0 0 0 2024-02-19T19:06:22.110 INFO:tasks.workunit.client.0.smithi088.stdout:3/470: chown d4/d1a/d1e/d31/d32/c67 264363 1 2024-02-19T19:06:22.112 INFO:tasks.workunit.client.0.smithi088.stdout:3/471: mkdir d4/d22/d66/da4 0 2024-02-19T19:06:22.115 INFO:tasks.workunit.client.0.smithi088.stdout:3/472: mknod d4/d1a/d1e/d74/d7c/d97/ca5 0 2024-02-19T19:06:22.116 INFO:tasks.workunit.client.0.smithi088.stdout:3/473: fsync d4/d1a/d1e/d31/f71 0 2024-02-19T19:06:22.116 INFO:tasks.workunit.client.0.smithi088.stdout:9/405: getdents d1/d21/d2d/d18 0 2024-02-19T19:06:22.118 INFO:tasks.workunit.client.0.smithi088.stdout:3/474: truncate d4/d1a/d1e/d31/f71 323828 0 2024-02-19T19:06:22.119 INFO:tasks.workunit.client.0.smithi088.stdout:9/406: unlink d1/d13/l3d 0 2024-02-19T19:06:22.120 INFO:tasks.workunit.client.0.smithi088.stdout:3/475: readlink d4/d1a/d1e/l6d 0 2024-02-19T19:06:22.120 INFO:tasks.workunit.client.0.smithi088.stdout:9/407: write d1/d8/d4a/f58 [4415079,105362] 0 2024-02-19T19:06:22.120 INFO:tasks.workunit.client.0.smithi088.stdout:1/458: getdents d6/d21/d6a/d7d 0 2024-02-19T19:06:22.125 INFO:tasks.workunit.client.0.smithi088.stdout:3/476: rename d4/d1a/d23/f95 to d4/d1a/d1e/d74/fa6 0 2024-02-19T19:06:22.128 INFO:tasks.workunit.client.0.smithi088.stdout:9/408: link d1/l57 d1/d21/d6f/l81 0 2024-02-19T19:06:22.129 INFO:tasks.workunit.client.0.smithi088.stdout:1/459: rename d6/d21/c50 to d6/c9c 0 2024-02-19T19:06:22.130 INFO:tasks.workunit.client.0.smithi088.stdout:9/409: mknod d1/d8/d3e/c82 0 2024-02-19T19:06:22.131 INFO:tasks.workunit.client.0.smithi088.stdout:1/460: mknod d6/d21/d6a/d7d/d9a/c9d 0 2024-02-19T19:06:22.133 INFO:tasks.workunit.client.0.smithi088.stdout:9/410: getdents d1/d8/d4a 0 2024-02-19T19:06:22.139 INFO:tasks.workunit.client.0.smithi088.stdout:3/477: dread d4/d1a/d23/f69 [0,4194304] 0 2024-02-19T19:06:22.139 INFO:tasks.workunit.client.0.smithi088.stdout:8/316: sync 2024-02-19T19:06:22.139 INFO:tasks.workunit.client.0.smithi088.stdout:0/318: sync 2024-02-19T19:06:22.139 INFO:tasks.workunit.client.0.smithi088.stdout:5/491: sync 2024-02-19T19:06:22.139 INFO:tasks.workunit.client.0.smithi088.stdout:7/440: sync 2024-02-19T19:06:22.139 INFO:tasks.workunit.client.0.smithi088.stdout:6/388: sync 2024-02-19T19:06:22.139 INFO:tasks.workunit.client.0.smithi088.stdout:2/290: sync 2024-02-19T19:06:22.139 INFO:tasks.workunit.client.0.smithi088.stdout:4/355: sync 2024-02-19T19:06:22.141 INFO:tasks.workunit.client.0.smithi088.stdout:6/389: chown d5/d8/d2c/f6c 78 1 2024-02-19T19:06:22.158 INFO:tasks.workunit.client.0.smithi088.stdout:7/441: mknod d14/d1d/d3c/c99 0 2024-02-19T19:06:22.159 INFO:tasks.workunit.client.0.smithi088.stdout:5/492: truncate d4/d1a/f3c 4188019 0 2024-02-19T19:06:22.161 INFO:tasks.workunit.client.0.smithi088.stdout:5/493: stat d4/d7 0 2024-02-19T19:06:22.161 INFO:tasks.workunit.client.0.smithi088.stdout:2/291: rmdir d3/d1f/d4f 39 2024-02-19T19:06:22.162 INFO:tasks.workunit.client.0.smithi088.stdout:7/442: dread - d14/d28/f35 zero size 2024-02-19T19:06:22.162 INFO:tasks.workunit.client.0.smithi088.stdout:8/317: truncate d9/df/d1c/f1b 7790140 0 2024-02-19T19:06:22.163 INFO:tasks.workunit.client.0.smithi088.stdout:6/390: mknod d5/d31/d25/c79 0 2024-02-19T19:06:22.163 INFO:tasks.workunit.client.0.smithi088.stdout:1/461: dwrite d6/d10/f11 [0,4194304] 0 2024-02-19T19:06:22.169 INFO:tasks.workunit.client.0.smithi088.stdout:4/356: dread d2/d35/f53 [0,4194304] 0 2024-02-19T19:06:22.169 INFO:tasks.workunit.client.0.smithi088.stdout:1/462: readlink d6/d21/d64/d94/l54 0 2024-02-19T19:06:22.174 INFO:tasks.workunit.client.0.smithi088.stdout:0/319: dwrite d13/d46/d5a/d25/d2a/d34/f40 [0,4194304] 0 2024-02-19T19:06:22.175 INFO:tasks.workunit.client.0.smithi088.stdout:5/494: symlink d4/d1a/d41/d5e/la2 0 2024-02-19T19:06:22.180 INFO:tasks.workunit.client.0.smithi088.stdout:2/292: write d3/d1b/f1c [794629,33274] 0 2024-02-19T19:06:22.181 INFO:tasks.workunit.client.0.smithi088.stdout:7/443: rename d14/d1d/d3c/d43/d5c/d75/d76/d7c to d14/d1d/d3c/d43/d5c/d9a 0 2024-02-19T19:06:22.184 INFO:tasks.workunit.client.0.smithi088.stdout:3/478: dread d4/d1a/d1e/d31/d32/f50 [0,4194304] 0 2024-02-19T19:06:22.185 INFO:tasks.workunit.client.0.smithi088.stdout:1/463: dwrite d6/d10/f91 [0,4194304] 0 2024-02-19T19:06:22.185 INFO:tasks.workunit.client.0.smithi088.stdout:5/495: symlink d4/d61/la3 0 2024-02-19T19:06:22.186 INFO:tasks.workunit.client.0.smithi088.stdout:6/391: dwrite d5/d19/f6d [0,4194304] 0 2024-02-19T19:06:22.188 INFO:tasks.workunit.client.0.smithi088.stdout:3/479: write d4/d87/f94 [993382,12186] 0 2024-02-19T19:06:22.190 INFO:tasks.workunit.client.0.smithi088.stdout:0/320: symlink d13/d46/d5a/d25/d2a/d2f/l70 0 2024-02-19T19:06:22.193 INFO:tasks.workunit.client.0.smithi088.stdout:1/464: creat d6/d21/d64/d94/d71/f9e x:0 0 0 2024-02-19T19:06:22.193 INFO:tasks.workunit.client.0.smithi088.stdout:1/465: dread - d6/d10/f78 zero size 2024-02-19T19:06:22.195 INFO:tasks.workunit.client.0.smithi088.stdout:5/496: symlink d4/d61/d65/d9a/d9f/la4 0 2024-02-19T19:06:22.197 INFO:tasks.workunit.client.0.smithi088.stdout:3/480: creat d4/d22/d7a/fa7 x:0 0 0 2024-02-19T19:06:22.197 INFO:tasks.workunit.client.0.smithi088.stdout:3/481: stat l0 0 2024-02-19T19:06:22.201 INFO:tasks.workunit.client.0.smithi088.stdout:1/466: chown d6/d10/d60/c83 0 1 2024-02-19T19:06:22.203 INFO:tasks.workunit.client.0.smithi088.stdout:0/321: mknod d13/d46/d5b/c71 0 2024-02-19T19:06:22.204 INFO:tasks.workunit.client.0.smithi088.stdout:5/497: symlink d4/d61/la5 0 2024-02-19T19:06:22.205 INFO:tasks.workunit.client.0.smithi088.stdout:2/293: mkdir d3/d1f/d4f/d59/d5e 0 2024-02-19T19:06:22.209 INFO:tasks.workunit.client.0.smithi088.stdout:4/357: truncate d2/d32/f44 2567134 0 2024-02-19T19:06:22.209 INFO:tasks.workunit.client.0.smithi088.stdout:4/358: fsync d2/d8/f6a 0 2024-02-19T19:06:22.210 INFO:tasks.workunit.client.0.smithi088.stdout:1/467: symlink d6/d21/d64/d94/d71/l9f 0 2024-02-19T19:06:22.210 INFO:tasks.workunit.client.0.smithi088.stdout:1/468: chown d6/d21/d64/d94/d6e 4890 1 2024-02-19T19:06:22.211 INFO:tasks.workunit.client.0.smithi088.stdout:0/322: mknod d13/d46/d5a/d25/d2a/d34/c72 0 2024-02-19T19:06:22.212 INFO:tasks.workunit.client.0.smithi088.stdout:0/323: chown d13/d46/d5a/c54 1 1 2024-02-19T19:06:22.212 INFO:tasks.workunit.client.0.smithi088.stdout:6/392: rmdir d5/d8/d2c/d78 0 2024-02-19T19:06:22.213 INFO:tasks.workunit.client.0.smithi088.stdout:5/498: symlink d4/d61/d65/d9a/d9f/d57/la6 0 2024-02-19T19:06:22.215 INFO:tasks.workunit.client.0.smithi088.stdout:4/359: mkdir d2/d32/d25/d26/d6d 0 2024-02-19T19:06:22.217 INFO:tasks.workunit.client.0.smithi088.stdout:6/393: unlink d5/d7/f15 0 2024-02-19T19:06:22.218 INFO:tasks.workunit.client.0.smithi088.stdout:3/482: truncate d4/d1a/d1e/f2f 9029906 0 2024-02-19T19:06:22.223 INFO:tasks.workunit.client.0.smithi088.stdout:5/499: dwrite d4/d61/d65/d9a/d9f/d57/d84/f8a [0,4194304] 0 2024-02-19T19:06:22.226 INFO:tasks.workunit.client.0.smithi088.stdout:3/483: chown d4/d1a/d1e/d31/d32/d55/f57 54524 1 2024-02-19T19:06:22.227 INFO:tasks.workunit.client.0.smithi088.stdout:1/469: dwrite d6/d21/d6a/f6d [0,4194304] 0 2024-02-19T19:06:22.229 INFO:tasks.workunit.client.0.smithi088.stdout:6/394: dwrite d5/f47 [0,4194304] 0 2024-02-19T19:06:22.235 INFO:tasks.workunit.client.0.smithi088.stdout:6/395: dread d5/d19/f6d [0,4194304] 0 2024-02-19T19:06:22.236 INFO:tasks.workunit.client.0.smithi088.stdout:6/396: chown d5/d19/l5b 32 1 2024-02-19T19:06:22.302 INFO:tasks.workunit.client.0.smithi088.stdout:0/324: rename fb to d13/d46/d5a/d25/d2a/d34/f73 0 2024-02-19T19:06:22.302 INFO:tasks.workunit.client.0.smithi088.stdout:0/325: chown d13/d62 490 1 2024-02-19T19:06:22.303 INFO:tasks.workunit.client.0.smithi088.stdout:5/500: read d4/d61/d65/d9a/d9f/f26 [4112396,72087] 0 2024-02-19T19:06:22.305 INFO:tasks.workunit.client.0.smithi088.stdout:5/501: truncate d4/d1a/f54 1508138 0 2024-02-19T19:06:22.307 INFO:tasks.workunit.client.0.smithi088.stdout:4/360: creat d2/d35/f6e x:0 0 0 2024-02-19T19:06:22.313 INFO:tasks.workunit.client.0.smithi088.stdout:0/326: unlink d13/d46/d5a/d25/f49 0 2024-02-19T19:06:22.313 INFO:tasks.workunit.client.0.smithi088.stdout:1/470: write d6/d10/d60/d29/f37 [5194479,78895] 0 2024-02-19T19:06:22.314 INFO:tasks.workunit.client.0.smithi088.stdout:0/327: fsync d13/f1d 0 2024-02-19T19:06:22.315 INFO:tasks.workunit.client.0.smithi088.stdout:1/471: truncate d6/d21/f84 395950 0 2024-02-19T19:06:22.316 INFO:tasks.workunit.client.0.smithi088.stdout:0/328: write d13/d46/d5a/d25/d2a/d2f/f4c [1527494,42055] 0 2024-02-19T19:06:22.317 INFO:tasks.workunit.client.0.smithi088.stdout:3/484: dread d4/d12/f21 [0,4194304] 0 2024-02-19T19:06:22.318 INFO:tasks.workunit.client.0.smithi088.stdout:3/485: read - d4/d12/d33/d58/f93 zero size 2024-02-19T19:06:22.319 INFO:tasks.workunit.client.0.smithi088.stdout:3/486: truncate d4/d12/f47 193317 0 2024-02-19T19:06:22.322 INFO:tasks.workunit.client.0.smithi088.stdout:6/397: write d5/d7/d32/f53 [3455154,56164] 0 2024-02-19T19:06:22.326 INFO:tasks.workunit.client.0.smithi088.stdout:4/361: mkdir d2/d32/d25/d26/d4e/d6f 0 2024-02-19T19:06:22.326 INFO:tasks.workunit.client.0.smithi088.stdout:4/362: stat d2/d35 0 2024-02-19T19:06:22.327 INFO:tasks.workunit.client.0.smithi088.stdout:0/329: rmdir d13/d46/d5a 39 2024-02-19T19:06:22.328 INFO:tasks.workunit.client.0.smithi088.stdout:1/472: mkdir d6/d21/d64/d94/da0 0 2024-02-19T19:06:22.328 INFO:tasks.workunit.client.0.smithi088.stdout:5/502: creat d4/d1a/fa7 x:0 0 0 2024-02-19T19:06:22.329 INFO:tasks.workunit.client.0.smithi088.stdout:1/473: chown d6/d10/d60/d29 4140 1 2024-02-19T19:06:22.330 INFO:tasks.workunit.client.0.smithi088.stdout:1/474: read - d6/d21/f6c zero size 2024-02-19T19:06:22.330 INFO:tasks.workunit.client.0.smithi088.stdout:9/411: sync 2024-02-19T19:06:22.331 INFO:tasks.workunit.client.0.smithi088.stdout:3/487: creat d4/fa8 x:0 0 0 2024-02-19T19:06:22.333 INFO:tasks.workunit.client.0.smithi088.stdout:5/503: creat d4/d61/fa8 x:0 0 0 2024-02-19T19:06:22.334 INFO:tasks.workunit.client.0.smithi088.stdout:0/330: fdatasync d13/d46/d5a/d25/d2a/d3c/f5d 0 2024-02-19T19:06:22.335 INFO:tasks.workunit.client.0.smithi088.stdout:9/412: mknod d1/d21/d5a/c83 0 2024-02-19T19:06:22.337 INFO:tasks.workunit.client.0.smithi088.stdout:0/331: dread d13/f33 [0,4194304] 0 2024-02-19T19:06:22.337 INFO:tasks.workunit.client.0.smithi088.stdout:5/504: dread d4/f9d [0,4194304] 0 2024-02-19T19:06:22.338 INFO:tasks.workunit.client.0.smithi088.stdout:3/488: rename d4/d12/l2a to d4/d22/d7a/la9 0 2024-02-19T19:06:22.343 INFO:tasks.workunit.client.0.smithi088.stdout:6/398: getdents d5/de 0 2024-02-19T19:06:22.343 INFO:tasks.workunit.client.0.smithi088.stdout:9/413: dwrite d1/d21/d2d/d23/f25 [0,4194304] 0 2024-02-19T19:06:22.345 INFO:tasks.workunit.client.0.smithi088.stdout:3/489: chown d4/ce 53754267 1 2024-02-19T19:06:22.345 INFO:tasks.workunit.client.0.smithi088.stdout:5/505: chown d4/d61/d65/d9a/d9f/d57/c94 4 1 2024-02-19T19:06:22.346 INFO:tasks.workunit.client.0.smithi088.stdout:1/475: creat d6/fa1 x:0 0 0 2024-02-19T19:06:22.347 INFO:tasks.workunit.client.0.smithi088.stdout:4/363: dwrite d2/d8/f29 [0,4194304] 0 2024-02-19T19:06:22.350 INFO:tasks.workunit.client.0.smithi088.stdout:5/506: chown d4/d61/d65/d9a/d9f/d57/d84/f8a 387344898 1 2024-02-19T19:06:22.352 INFO:tasks.workunit.client.0.smithi088.stdout:6/399: symlink d5/d19/d70/l7a 0 2024-02-19T19:06:22.355 INFO:tasks.workunit.client.0.smithi088.stdout:0/332: rename d13/f33 to d13/d46/d5a/f74 0 2024-02-19T19:06:22.357 INFO:tasks.workunit.client.0.smithi088.stdout:9/414: unlink d1/d8/d3e/l50 0 2024-02-19T19:06:22.360 INFO:tasks.workunit.client.0.smithi088.stdout:0/333: read d13/f1d [1476155,103088] 0 2024-02-19T19:06:22.362 INFO:tasks.workunit.client.0.smithi088.stdout:3/490: rename d4/d1a/d7f/d86 to d4/d1a/d1e/d74/d7d/daa 0 2024-02-19T19:06:22.364 INFO:tasks.workunit.client.0.smithi088.stdout:4/364: mkdir d2/d32/d25/d3f/d70 0 2024-02-19T19:06:22.365 INFO:tasks.workunit.client.0.smithi088.stdout:9/415: dwrite d1/d13/f36 [0,4194304] 0 2024-02-19T19:06:22.367 INFO:tasks.workunit.client.0.smithi088.stdout:5/507: write d4/d7/f63 [1123369,75546] 0 2024-02-19T19:06:22.370 INFO:tasks.workunit.client.0.smithi088.stdout:1/476: rename d6/d10/d60/d29/d63/c7f to d6/d21/d6a/d4a/ca2 0 2024-02-19T19:06:22.371 INFO:tasks.workunit.client.0.smithi088.stdout:3/491: mkdir d4/d12/d33/dab 0 2024-02-19T19:06:22.373 INFO:tasks.workunit.client.0.smithi088.stdout:6/400: creat d5/d8/f7b x:0 0 0 2024-02-19T19:06:22.374 INFO:tasks.workunit.client.0.smithi088.stdout:6/401: chown d5/d19/f6d 6233 1 2024-02-19T19:06:22.377 INFO:tasks.workunit.client.0.smithi088.stdout:1/477: rename d6/d10/f91 to d6/d21/d6a/d7d/d9a/fa3 0 2024-02-19T19:06:22.381 INFO:tasks.workunit.client.0.smithi088.stdout:1/478: symlink d6/d21/d64/d94/d6e/la4 0 2024-02-19T19:06:22.381 INFO:tasks.workunit.client.0.smithi088.stdout:6/402: symlink d5/d7/l7c 0 2024-02-19T19:06:22.383 INFO:tasks.workunit.client.0.smithi088.stdout:6/403: mknod d5/d8/c7d 0 2024-02-19T19:06:22.385 INFO:tasks.workunit.client.0.smithi088.stdout:6/404: mkdir d5/de/d64/d7e 0 2024-02-19T19:06:22.386 INFO:tasks.workunit.client.0.smithi088.stdout:1/479: link d6/d21/d6a/c62 d6/d21/ca5 0 2024-02-19T19:06:22.387 INFO:tasks.workunit.client.0.smithi088.stdout:6/405: symlink d5/de/d60/l7f 0 2024-02-19T19:06:22.388 INFO:tasks.workunit.client.0.smithi088.stdout:6/406: chown d5/d7/d32 286 1 2024-02-19T19:06:22.389 INFO:tasks.workunit.client.0.smithi088.stdout:1/480: creat d6/d21/d6a/d7d/fa6 x:0 0 0 2024-02-19T19:06:22.390 INFO:tasks.workunit.client.0.smithi088.stdout:1/481: dread - d6/d10/f73 zero size 2024-02-19T19:06:22.391 INFO:tasks.workunit.client.0.smithi088.stdout:1/482: chown d6/d10/d60/c8c 16616586 1 2024-02-19T19:06:22.392 INFO:tasks.workunit.client.0.smithi088.stdout:1/483: chown d6/d10/d60/d29/d63 15491224 1 2024-02-19T19:06:22.393 INFO:tasks.workunit.client.0.smithi088.stdout:9/416: write d1/d21/d2d/d23/f26 [302210,84036] 0 2024-02-19T19:06:22.397 INFO:tasks.workunit.client.0.smithi088.stdout:1/484: creat d6/d21/d6a/d4a/fa7 x:0 0 0 2024-02-19T19:06:22.399 INFO:tasks.workunit.client.0.smithi088.stdout:4/365: dread d2/d32/f1c [4194304,4194304] 0 2024-02-19T19:06:22.400 INFO:tasks.workunit.client.0.smithi088.stdout:1/485: read - d6/d10/d60/f67 zero size 2024-02-19T19:06:22.402 INFO:tasks.workunit.client.0.smithi088.stdout:0/334: dread f4 [0,4194304] 0 2024-02-19T19:06:22.403 INFO:tasks.workunit.client.0.smithi088.stdout:9/417: write d1/d21/d2d/f4c [500024,80963] 0 2024-02-19T19:06:22.404 INFO:tasks.workunit.client.0.smithi088.stdout:3/492: dwrite d4/f6 [4194304,4194304] 0 2024-02-19T19:06:22.406 INFO:tasks.workunit.client.0.smithi088.stdout:6/407: dread d5/d19/f41 [0,4194304] 0 2024-02-19T19:06:22.406 INFO:tasks.workunit.client.0.smithi088.stdout:9/418: truncate d1/f7f 1295329 0 2024-02-19T19:06:22.407 INFO:tasks.workunit.client.0.smithi088.stdout:4/366: rename d2/d8/f48 to d2/d32/d9/f71 0 2024-02-19T19:06:22.407 INFO:tasks.workunit.client.0.smithi088.stdout:9/419: fsync d1/d8/f42 0 2024-02-19T19:06:22.408 INFO:tasks.workunit.client.0.smithi088.stdout:3/493: write d4/d22/d81/f98 [257338,78165] 0 2024-02-19T19:06:22.410 INFO:tasks.workunit.client.0.smithi088.stdout:3/494: fdatasync d4/d12/d33/d58/f93 0 2024-02-19T19:06:22.411 INFO:tasks.workunit.client.0.smithi088.stdout:6/408: creat d5/d19/d70/f80 x:0 0 0 2024-02-19T19:06:22.412 INFO:tasks.workunit.client.0.smithi088.stdout:0/335: unlink d13/d46/d5a/c36 0 2024-02-19T19:06:22.413 INFO:tasks.workunit.client.0.smithi088.stdout:0/336: readlink d13/d46/d5a/d25/d2a/d3c/l65 0 2024-02-19T19:06:22.415 INFO:tasks.workunit.client.0.smithi088.stdout:9/420: rename d1/d8/d4a/f5e to d1/d8/d37/d54/f84 0 2024-02-19T19:06:22.415 INFO:tasks.workunit.client.0.smithi088.stdout:9/421: chown d1/d8/f42 5 1 2024-02-19T19:06:22.426 INFO:tasks.workunit.client.0.smithi088.stdout:9/422: dwrite d1/d21/d2d/d18/d1f/f59 [0,4194304] 0 2024-02-19T19:06:22.426 INFO:tasks.workunit.client.0.smithi088.stdout:0/337: creat d13/d46/d5a/d25/d2a/d34/f75 x:0 0 0 2024-02-19T19:06:22.427 INFO:tasks.workunit.client.0.smithi088.stdout:6/409: write d5/d31/d25/f5d [941947,58162] 0 2024-02-19T19:06:22.428 INFO:tasks.workunit.client.0.smithi088.stdout:1/486: truncate d6/d21/d64/f81 1139604 0 2024-02-19T19:06:22.429 INFO:tasks.workunit.client.0.smithi088.stdout:0/338: write d13/f4a [929654,31883] 0 2024-02-19T19:06:22.431 INFO:tasks.workunit.client.0.smithi088.stdout:1/487: stat d6/d10/d60/c33 0 2024-02-19T19:06:22.431 INFO:tasks.workunit.client.0.smithi088.stdout:6/410: write d5/d31/d25/f5d [1794622,35398] 0 2024-02-19T19:06:22.432 INFO:tasks.workunit.client.0.smithi088.stdout:0/339: write d13/f4a [1214674,100339] 0 2024-02-19T19:06:22.434 INFO:tasks.workunit.client.0.smithi088.stdout:6/411: stat d5/d31/d25/f39 0 2024-02-19T19:06:22.435 INFO:tasks.workunit.client.0.smithi088.stdout:3/495: dwrite d4/d1a/d1e/d74/fa6 [0,4194304] 0 2024-02-19T19:06:22.440 INFO:tasks.workunit.client.0.smithi088.stdout:9/423: creat d1/d60/f85 x:0 0 0 2024-02-19T19:06:22.449 INFO:tasks.workunit.client.0.smithi088.stdout:1/488: unlink d6/d21/d6a/d4a/l69 0 2024-02-19T19:06:22.450 INFO:tasks.workunit.client.0.smithi088.stdout:6/412: mknod d5/de/d64/c81 0 2024-02-19T19:06:22.452 INFO:tasks.workunit.client.0.smithi088.stdout:6/413: chown d5/de/c43 170411333 1 2024-02-19T19:06:22.453 INFO:tasks.workunit.client.0.smithi088.stdout:3/496: creat d4/d22/d7a/da1/fac x:0 0 0 2024-02-19T19:06:22.456 INFO:tasks.workunit.client.0.smithi088.stdout:6/414: unlink d5/d8/l5a 0 2024-02-19T19:06:22.456 INFO:tasks.workunit.client.0.smithi088.stdout:1/489: dwrite d6/f99 [0,4194304] 0 2024-02-19T19:06:22.458 INFO:tasks.workunit.client.0.smithi088.stdout:3/497: mknod d4/d22/cad 0 2024-02-19T19:06:22.460 INFO:tasks.workunit.client.0.smithi088.stdout:7/444: sync 2024-02-19T19:06:22.461 INFO:tasks.workunit.client.0.smithi088.stdout:8/318: sync 2024-02-19T19:06:22.462 INFO:tasks.workunit.client.0.smithi088.stdout:6/415: mkdir d5/de/d64/d82 0 2024-02-19T19:06:22.463 INFO:tasks.workunit.client.0.smithi088.stdout:1/490: mkdir d6/d21/d6a/d7d/da8 0 2024-02-19T19:06:22.466 INFO:tasks.workunit.client.0.smithi088.stdout:6/416: mkdir d5/d19/d70/d83 0 2024-02-19T19:06:22.467 INFO:tasks.workunit.client.0.smithi088.stdout:3/498: dwrite d4/d1a/f24 [0,4194304] 0 2024-02-19T19:06:22.468 INFO:tasks.workunit.client.0.smithi088.stdout:3/499: readlink d4/l5 0 2024-02-19T19:06:22.469 INFO:tasks.workunit.client.0.smithi088.stdout:7/445: symlink d14/l9b 0 2024-02-19T19:06:22.473 INFO:tasks.workunit.client.0.smithi088.stdout:7/446: chown d14/d1d/c3a 605261036 1 2024-02-19T19:06:22.476 INFO:tasks.workunit.client.0.smithi088.stdout:6/417: mknod d5/d19/c84 0 2024-02-19T19:06:22.477 INFO:tasks.workunit.client.0.smithi088.stdout:3/500: creat d4/d12/d33/d43/d85/fae x:0 0 0 2024-02-19T19:06:22.479 INFO:tasks.workunit.client.0.smithi088.stdout:8/319: getdents d9/df/d1c 0 2024-02-19T19:06:22.480 INFO:tasks.workunit.client.0.smithi088.stdout:8/320: chown d9/df/d1c/d22/l3c 66793119 1 2024-02-19T19:06:22.481 INFO:tasks.workunit.client.0.smithi088.stdout:8/321: chown d9/l56 252 1 2024-02-19T19:06:22.481 INFO:tasks.workunit.client.0.smithi088.stdout:7/447: mkdir d14/d28/d9c 0 2024-02-19T19:06:22.481 INFO:tasks.workunit.client.0.smithi088.stdout:7/448: readlink d14/d28/l53 0 2024-02-19T19:06:22.483 INFO:tasks.workunit.client.0.smithi088.stdout:6/418: getdents d5/de/d64/d82 0 2024-02-19T19:06:22.484 INFO:tasks.workunit.client.0.smithi088.stdout:3/501: creat d4/d12/d33/dab/faf x:0 0 0 2024-02-19T19:06:22.485 INFO:tasks.workunit.client.0.smithi088.stdout:8/322: mkdir d9/df/d1c/d2d/d58 0 2024-02-19T19:06:22.486 INFO:tasks.workunit.client.0.smithi088.stdout:7/449: creat d14/d28/d65/f9d x:0 0 0 2024-02-19T19:06:22.487 INFO:tasks.workunit.client.0.smithi088.stdout:1/491: dread f5 [0,4194304] 0 2024-02-19T19:06:22.488 INFO:tasks.workunit.client.0.smithi088.stdout:7/450: fsync d14/d28/d3b/f52 0 2024-02-19T19:06:22.491 INFO:tasks.workunit.client.0.smithi088.stdout:3/502: fdatasync d4/d1a/d23/f5d 0 2024-02-19T19:06:22.494 INFO:tasks.workunit.client.0.smithi088.stdout:0/340: write d13/d46/d5a/d25/d2a/d3c/f55 [1024656,117171] 0 2024-02-19T19:06:22.495 INFO:tasks.workunit.client.0.smithi088.stdout:0/341: truncate d13/f4a 1729652 0 2024-02-19T19:06:22.496 INFO:tasks.workunit.client.0.smithi088.stdout:1/492: creat d6/d21/fa9 x:0 0 0 2024-02-19T19:06:22.496 INFO:tasks.workunit.client.0.smithi088.stdout:7/451: dread d14/f44 [0,4194304] 0 2024-02-19T19:06:22.498 INFO:tasks.workunit.client.0.smithi088.stdout:6/419: mknod d5/d31/d25/c85 0 2024-02-19T19:06:22.504 INFO:tasks.workunit.client.0.smithi088.stdout:0/342: read - d13/f39 zero size 2024-02-19T19:06:22.506 INFO:tasks.workunit.client.0.smithi088.stdout:1/493: chown d6/d10/f3b 17 1 2024-02-19T19:06:22.509 INFO:tasks.workunit.client.0.smithi088.stdout:1/494: write d6/d21/d6a/d4a/f56 [4563947,96267] 0 2024-02-19T19:06:22.511 INFO:tasks.workunit.client.0.smithi088.stdout:3/503: getdents d4/d22/d7a/d82 0 2024-02-19T19:06:22.513 INFO:tasks.workunit.client.0.smithi088.stdout:0/343: mkdir d13/d46/d5b/d76 0 2024-02-19T19:06:22.517 INFO:tasks.workunit.client.0.smithi088.stdout:6/420: dwrite d5/d19/f6d [0,4194304] 0 2024-02-19T19:06:22.520 INFO:tasks.workunit.client.0.smithi088.stdout:3/504: read - d4/d1a/d1e/d31/d32/f51 zero size 2024-02-19T19:06:22.522 INFO:tasks.workunit.client.0.smithi088.stdout:1/495: mknod d6/d21/d64/d94/da0/caa 0 2024-02-19T19:06:22.524 INFO:tasks.workunit.client.0.smithi088.stdout:1/496: write d6/fa1 [428103,16922] 0 2024-02-19T19:06:22.525 INFO:tasks.workunit.client.0.smithi088.stdout:8/323: link d9/df/l4c d9/df/d48/l59 0 2024-02-19T19:06:22.526 INFO:tasks.workunit.client.0.smithi088.stdout:3/505: rmdir d4/d1a/d1e/d74/d7d/daa 39 2024-02-19T19:06:22.528 INFO:tasks.workunit.client.0.smithi088.stdout:6/421: dwrite d5/de/f10 [4194304,4194304] 0 2024-02-19T19:06:22.530 INFO:tasks.workunit.client.0.smithi088.stdout:0/344: rename d13/f1d to d13/d46/d5a/d25/d2a/d34/f77 0 2024-02-19T19:06:22.532 INFO:tasks.workunit.client.0.smithi088.stdout:0/345: write d13/d46/d5a/f4b [507068,69883] 0 2024-02-19T19:06:22.539 INFO:tasks.workunit.client.0.smithi088.stdout:6/422: creat d5/d7/d32/f86 x:0 0 0 2024-02-19T19:06:22.540 INFO:tasks.workunit.client.0.smithi088.stdout:6/423: fdatasync d5/d19/f4e 0 2024-02-19T19:06:22.544 INFO:tasks.workunit.client.0.smithi088.stdout:8/324: symlink d9/df/d1c/d2d/d58/l5a 0 2024-02-19T19:06:22.545 INFO:tasks.workunit.client.0.smithi088.stdout:1/497: dwrite d6/d21/f36 [0,4194304] 0 2024-02-19T19:06:22.547 INFO:tasks.workunit.client.0.smithi088.stdout:3/506: symlink d4/d1a/d1e/d74/d7d/lb0 0 2024-02-19T19:06:22.547 INFO:tasks.workunit.client.0.smithi088.stdout:8/325: fsync d9/df/d1c/f29 0 2024-02-19T19:06:22.552 INFO:tasks.workunit.client.0.smithi088.stdout:1/498: unlink d6/d10/d60/c83 0 2024-02-19T19:06:22.553 INFO:tasks.workunit.client.0.smithi088.stdout:8/326: dread d9/df/d1c/f29 [0,4194304] 0 2024-02-19T19:06:22.555 INFO:tasks.workunit.client.0.smithi088.stdout:3/507: symlink d4/d1a/d1e/d31/d32/d55/lb1 0 2024-02-19T19:06:22.556 INFO:tasks.workunit.client.0.smithi088.stdout:6/424: link d5/f47 d5/de/d60/f87 0 2024-02-19T19:06:22.559 INFO:tasks.workunit.client.0.smithi088.stdout:8/327: mknod d9/df/d1c/d22/c5b 0 2024-02-19T19:06:22.561 INFO:tasks.workunit.client.0.smithi088.stdout:3/508: creat d4/d1a/d7f/fb2 x:0 0 0 2024-02-19T19:06:22.563 INFO:tasks.workunit.client.0.smithi088.stdout:3/509: write d4/fa8 [768999,39284] 0 2024-02-19T19:06:22.566 INFO:tasks.workunit.client.0.smithi088.stdout:6/425: dwrite d5/d31/d25/f39 [0,4194304] 0 2024-02-19T19:06:22.568 INFO:tasks.workunit.client.0.smithi088.stdout:8/328: dwrite d9/df/d1c/d2d/f4b [0,4194304] 0 2024-02-19T19:06:22.571 INFO:tasks.workunit.client.0.smithi088.stdout:6/426: symlink d5/d19/d70/d83/l88 0 2024-02-19T19:06:22.572 INFO:tasks.workunit.client.0.smithi088.stdout:8/329: write d9/f4a [1712101,40980] 0 2024-02-19T19:06:22.575 INFO:tasks.workunit.client.0.smithi088.stdout:8/330: stat d9/df/f1a 0 2024-02-19T19:06:22.579 INFO:tasks.workunit.client.0.smithi088.stdout:3/510: dwrite d4/d12/d33/d58/f93 [0,4194304] 0 2024-02-19T19:06:22.582 INFO:tasks.workunit.client.0.smithi088.stdout:7/452: write d14/d1d/d3c/d43/d5c/f69 [490653,58612] 0 2024-02-19T19:06:22.588 INFO:tasks.workunit.client.0.smithi088.stdout:6/427: dwrite d5/d8/d2c/f6c [0,4194304] 0 2024-02-19T19:06:22.588 INFO:tasks.workunit.client.0.smithi088.stdout:6/428: truncate d5/de/d60/f76 215530 0 2024-02-19T19:06:22.588 INFO:tasks.workunit.client.0.smithi088.stdout:6/429: truncate d5/de/f68 127465 0 2024-02-19T19:06:22.588 INFO:tasks.workunit.client.0.smithi088.stdout:3/511: read d4/d22/d81/f98 [124461,116228] 0 2024-02-19T19:06:22.589 INFO:tasks.workunit.client.0.smithi088.stdout:7/453: mknod d14/d27/d49/d98/c9e 0 2024-02-19T19:06:22.592 INFO:tasks.workunit.client.0.smithi088.stdout:7/454: write d14/d28/d3b/f4b [3801825,55020] 0 2024-02-19T19:06:22.596 INFO:tasks.workunit.client.0.smithi088.stdout:1/499: rmdir d6 39 2024-02-19T19:06:22.598 INFO:tasks.workunit.client.0.smithi088.stdout:0/346: dwrite d13/f39 [0,4194304] 0 2024-02-19T19:06:22.602 INFO:tasks.workunit.client.0.smithi088.stdout:1/500: dread d6/fa1 [0,4194304] 0 2024-02-19T19:06:22.604 INFO:tasks.workunit.client.0.smithi088.stdout:7/455: link d14/d1d/d3c/d43/d5c/d9a/d92/f97 d14/d1d/d3c/d43/d5c/f9f 0 2024-02-19T19:06:22.607 INFO:tasks.workunit.client.0.smithi088.stdout:6/430: fsync d5/d8/d2c/f6c 0 2024-02-19T19:06:22.608 INFO:tasks.workunit.client.0.smithi088.stdout:7/456: symlink d14/d1d/d3c/d43/d5c/d75/la0 0 2024-02-19T19:06:22.609 INFO:tasks.workunit.client.0.smithi088.stdout:0/347: dwrite d13/f6a [0,4194304] 0 2024-02-19T19:06:22.610 INFO:tasks.workunit.client.0.smithi088.stdout:7/457: chown c5 6557 1 2024-02-19T19:06:22.611 INFO:tasks.workunit.client.0.smithi088.stdout:1/501: truncate d6/d21/d6a/d4a/f59 470039 0 2024-02-19T19:06:22.611 INFO:tasks.workunit.client.0.smithi088.stdout:2/294: sync 2024-02-19T19:06:22.612 INFO:tasks.workunit.client.0.smithi088.stdout:5/508: sync 2024-02-19T19:06:22.612 INFO:tasks.workunit.client.0.smithi088.stdout:7/458: write d14/d1d/d3c/f63 [375249,88777] 0 2024-02-19T19:06:22.612 INFO:tasks.workunit.client.0.smithi088.stdout:6/431: rmdir d5/de/d64 39 2024-02-19T19:06:22.618 INFO:tasks.workunit.client.0.smithi088.stdout:1/502: fdatasync d6/d21/d6a/d4a/fa7 0 2024-02-19T19:06:22.618 INFO:tasks.workunit.client.0.smithi088.stdout:6/432: write d5/d31/f59 [714612,31479] 0 2024-02-19T19:06:22.621 INFO:tasks.workunit.client.0.smithi088.stdout:2/295: creat d3/d1f/d4f/d59/f5f x:0 0 0 2024-02-19T19:06:22.624 INFO:tasks.workunit.client.0.smithi088.stdout:7/459: symlink d14/d1d/d3c/d43/la1 0 2024-02-19T19:06:22.624 INFO:tasks.workunit.client.0.smithi088.stdout:6/433: mknod d5/d8/d3a/d71/c89 0 2024-02-19T19:06:22.625 INFO:tasks.workunit.client.0.smithi088.stdout:8/331: dread d9/f2e [0,4194304] 0 2024-02-19T19:06:22.626 INFO:tasks.workunit.client.0.smithi088.stdout:5/509: dwrite d4/d61/d65/d9a/d9f/f3d [4194304,4194304] 0 2024-02-19T19:06:22.629 INFO:tasks.workunit.client.0.smithi088.stdout:0/348: dread d13/d46/d5a/d25/d2a/d34/f35 [0,4194304] 0 2024-02-19T19:06:22.629 INFO:tasks.workunit.client.0.smithi088.stdout:1/503: dread d6/d10/f11 [0,4194304] 0 2024-02-19T19:06:22.631 INFO:tasks.workunit.client.0.smithi088.stdout:9/424: sync 2024-02-19T19:06:22.631 INFO:tasks.workunit.client.0.smithi088.stdout:4/367: sync 2024-02-19T19:06:22.632 INFO:tasks.workunit.client.0.smithi088.stdout:8/332: mknod d9/df/c5c 0 2024-02-19T19:06:22.636 INFO:tasks.workunit.client.0.smithi088.stdout:1/504: truncate d6/d21/f84 785036 0 2024-02-19T19:06:22.636 INFO:tasks.workunit.client.0.smithi088.stdout:0/349: chown d13/d46/d5a/d25/d2a/d34/f73 246174584 1 2024-02-19T19:06:22.637 INFO:tasks.workunit.client.0.smithi088.stdout:7/460: link d14/d1d/d3c/f80 d14/d27/d49/fa2 0 2024-02-19T19:06:22.638 INFO:tasks.workunit.client.0.smithi088.stdout:2/296: dwrite d3/d14/f19 [0,4194304] 0 2024-02-19T19:06:22.640 INFO:tasks.workunit.client.0.smithi088.stdout:5/510: rename d4/f8 to d4/d61/d65/d9a/d9f/da0/fa9 0 2024-02-19T19:06:22.641 INFO:tasks.workunit.client.0.smithi088.stdout:1/505: chown d6/d21/fa9 791171131 1 2024-02-19T19:06:22.645 INFO:tasks.workunit.client.0.smithi088.stdout:9/425: write d1/d8/f77 [619420,27470] 0 2024-02-19T19:06:22.645 INFO:tasks.workunit.client.0.smithi088.stdout:3/512: dread d4/f7 [0,4194304] 0 2024-02-19T19:06:22.648 INFO:tasks.workunit.client.0.smithi088.stdout:6/434: dwrite d5/de/f68 [0,4194304] 0 2024-02-19T19:06:22.648 INFO:tasks.workunit.client.0.smithi088.stdout:2/297: symlink d3/d14/l60 0 2024-02-19T19:06:22.649 INFO:tasks.workunit.client.0.smithi088.stdout:1/506: write d6/d21/d64/d94/d6e/f88 [183388,79288] 0 2024-02-19T19:06:22.650 INFO:tasks.workunit.client.0.smithi088.stdout:6/435: read - d5/d19/d70/f80 zero size 2024-02-19T19:06:22.651 INFO:tasks.workunit.client.0.smithi088.stdout:2/298: write d3/d1f/d4f/d59/f5f [948211,65248] 0 2024-02-19T19:06:22.654 INFO:tasks.workunit.client.0.smithi088.stdout:7/461: symlink d14/d1d/d3c/d43/d5c/d9a/d92/la3 0 2024-02-19T19:06:22.656 INFO:tasks.workunit.client.0.smithi088.stdout:7/462: chown d14/d1d/d3c/d43/d5c/d9a 75 1 2024-02-19T19:06:22.656 INFO:tasks.workunit.client.0.smithi088.stdout:9/426: rmdir d1/d8/d3e/d4e 39 2024-02-19T19:06:22.656 INFO:tasks.workunit.client.0.smithi088.stdout:7/463: fdatasync f2 0 2024-02-19T19:06:22.657 INFO:tasks.workunit.client.0.smithi088.stdout:9/427: readlink d1/d13/l1c 0 2024-02-19T19:06:22.658 INFO:tasks.workunit.client.0.smithi088.stdout:9/428: write d1/d8/d37/d54/f75 [479529,78557] 0 2024-02-19T19:06:22.659 INFO:tasks.workunit.client.0.smithi088.stdout:3/513: creat d4/d1a/d1e/d31/d32/d55/fb3 x:0 0 0 2024-02-19T19:06:22.660 INFO:tasks.workunit.client.0.smithi088.stdout:9/429: truncate d1/d60/f6c 933128 0 2024-02-19T19:06:22.661 INFO:tasks.workunit.client.0.smithi088.stdout:3/514: chown d4/d22/d7a/d8d 652220 1 2024-02-19T19:06:22.661 INFO:tasks.workunit.client.0.smithi088.stdout:9/430: readlink d1/d13/l1c 0 2024-02-19T19:06:22.662 INFO:tasks.workunit.client.0.smithi088.stdout:9/431: chown d1/d8/d4a/d4f/c5d 2667909 1 2024-02-19T19:06:22.664 INFO:tasks.workunit.client.0.smithi088.stdout:4/368: creat d2/d35/d39/f72 x:0 0 0 2024-02-19T19:06:22.665 INFO:tasks.workunit.client.0.smithi088.stdout:8/333: rename d9/df/d48 to d9/df/d1c/d2d/d58/d5d 0 2024-02-19T19:06:22.667 INFO:tasks.workunit.client.0.smithi088.stdout:2/299: mkdir d3/d1b/d43/d61 0 2024-02-19T19:06:22.668 INFO:tasks.workunit.client.0.smithi088.stdout:4/369: chown d2/d35/d39/d3a/l5a 325903 1 2024-02-19T19:06:22.668 INFO:tasks.workunit.client.0.smithi088.stdout:3/515: dwrite d4/d1a/d1e/d74/fa6 [0,4194304] 0 2024-02-19T19:06:22.671 INFO:tasks.workunit.client.0.smithi088.stdout:9/432: dread d1/d21/d2d/f16 [4194304,4194304] 0 2024-02-19T19:06:22.671 INFO:tasks.workunit.client.0.smithi088.stdout:3/516: write d4/d12/f2e [1348825,119061] 0 2024-02-19T19:06:22.671 INFO:tasks.workunit.client.0.smithi088.stdout:1/507: mkdir d6/d21/d64/d94/dab 0 2024-02-19T19:06:22.674 INFO:tasks.workunit.client.0.smithi088.stdout:7/464: creat d14/d1d/d3c/d43/d5c/d75/fa4 x:0 0 0 2024-02-19T19:06:22.677 INFO:tasks.workunit.client.0.smithi088.stdout:6/436: symlink d5/de/d60/l8a 0 2024-02-19T19:06:22.678 INFO:tasks.workunit.client.0.smithi088.stdout:7/465: dread - d14/d28/d65/f9d zero size 2024-02-19T19:06:22.679 INFO:tasks.workunit.client.0.smithi088.stdout:0/350: write d13/d46/d5a/d25/f27 [7748284,89492] 0 2024-02-19T19:06:22.682 INFO:tasks.workunit.client.0.smithi088.stdout:8/334: chown d9/ld 1201296351 1 2024-02-19T19:06:22.682 INFO:tasks.workunit.client.0.smithi088.stdout:2/300: dwrite d3/d14/f5a [0,4194304] 0 2024-02-19T19:06:22.682 INFO:tasks.workunit.client.0.smithi088.stdout:5/511: rename d4/d7/c5d to d4/d1a/d41/caa 0 2024-02-19T19:06:22.683 INFO:tasks.workunit.client.0.smithi088.stdout:4/370: mknod d2/d32/d9/c73 0 2024-02-19T19:06:22.687 INFO:tasks.workunit.client.0.smithi088.stdout:0/351: dread - d13/d46/f6c zero size 2024-02-19T19:06:22.689 INFO:tasks.workunit.client.0.smithi088.stdout:1/508: dwrite d6/d21/d6a/d4a/f56 [0,4194304] 0 2024-02-19T19:06:22.689 INFO:tasks.workunit.client.0.smithi088.stdout:4/371: chown d2/d32/f1c 0 1 2024-02-19T19:06:22.695 INFO:tasks.workunit.client.0.smithi088.stdout:6/437: chown d5/d8/d3a/c6b 1 1 2024-02-19T19:06:22.695 INFO:tasks.workunit.client.0.smithi088.stdout:7/466: dwrite d14/f19 [0,4194304] 0 2024-02-19T19:06:22.698 INFO:tasks.workunit.client.0.smithi088.stdout:9/433: rename d1/d60/f80 to d1/d21/d2d/d23/d79/f86 0 2024-02-19T19:06:22.700 INFO:tasks.workunit.client.0.smithi088.stdout:5/512: mknod d4/d1a/d41/d5e/cab 0 2024-02-19T19:06:22.700 INFO:tasks.workunit.client.0.smithi088.stdout:1/509: dwrite d6/d10/f45 [0,4194304] 0 2024-02-19T19:06:22.703 INFO:tasks.workunit.client.0.smithi088.stdout:3/517: write d4/d1a/d1e/d31/d32/f51 [241195,5683] 0 2024-02-19T19:06:22.706 INFO:tasks.workunit.client.0.smithi088.stdout:9/434: dread d1/d13/f53 [0,4194304] 0 2024-02-19T19:06:22.707 INFO:tasks.workunit.client.0.smithi088.stdout:9/435: write d1/d60/f85 [614437,96103] 0 2024-02-19T19:06:22.708 INFO:tasks.workunit.client.0.smithi088.stdout:4/372: creat d2/d35/d39/d3a/f74 x:0 0 0 2024-02-19T19:06:22.709 INFO:tasks.workunit.client.0.smithi088.stdout:7/467: chown d14/d28/d65/f7e 1023237 1 2024-02-19T19:06:22.711 INFO:tasks.workunit.client.0.smithi088.stdout:7/468: fdatasync d14/d1d/d3c/f8f 0 2024-02-19T19:06:22.711 INFO:tasks.workunit.client.0.smithi088.stdout:5/513: fdatasync d4/d1a/f6e 0 2024-02-19T19:06:22.716 INFO:tasks.workunit.client.0.smithi088.stdout:1/510: rmdir d6/d21/d6a 39 2024-02-19T19:06:22.717 INFO:tasks.workunit.client.0.smithi088.stdout:8/335: creat d9/df/d1c/d2d/d47/f5e x:0 0 0 2024-02-19T19:06:22.718 INFO:tasks.workunit.client.0.smithi088.stdout:9/436: dread d1/d8/d37/d54/f75 [0,4194304] 0 2024-02-19T19:06:22.719 INFO:tasks.workunit.client.0.smithi088.stdout:0/352: dwrite d13/d46/d5a/f74 [0,4194304] 0 2024-02-19T19:06:22.723 INFO:tasks.workunit.client.0.smithi088.stdout:2/301: getdents d3/d1f 0 2024-02-19T19:06:22.725 INFO:tasks.workunit.client.0.smithi088.stdout:4/373: creat d2/f75 x:0 0 0 2024-02-19T19:06:22.731 INFO:tasks.workunit.client.0.smithi088.stdout:5/514: dwrite d4/d7/f64 [0,4194304] 0 2024-02-19T19:06:22.731 INFO:tasks.workunit.client.0.smithi088.stdout:6/438: truncate d5/d31/d25/f49 2700852 0 2024-02-19T19:06:22.731 INFO:tasks.workunit.client.0.smithi088.stdout:7/469: dwrite d14/d1d/d3c/f8f [0,4194304] 0 2024-02-19T19:06:22.733 INFO:tasks.workunit.client.0.smithi088.stdout:6/439: read - d5/d8/d3a/f63 zero size 2024-02-19T19:06:22.735 INFO:tasks.workunit.client.0.smithi088.stdout:0/353: chown d13/d46/d5a/d25/d2a/d34/c6b 22847 1 2024-02-19T19:06:22.737 INFO:tasks.workunit.client.0.smithi088.stdout:4/374: mknod d2/d32/d25/d26/d58/c76 0 2024-02-19T19:06:22.739 INFO:tasks.workunit.client.0.smithi088.stdout:8/336: write d9/df/d1c/f33 [102328,101771] 0 2024-02-19T19:06:22.740 INFO:tasks.workunit.client.0.smithi088.stdout:1/511: write d6/d21/d64/d94/d71/f9b [6393007,35068] 0 2024-02-19T19:06:22.747 INFO:tasks.workunit.client.0.smithi088.stdout:7/470: truncate d14/f1f 767796 0 2024-02-19T19:06:22.747 INFO:tasks.workunit.client.0.smithi088.stdout:8/337: read d9/df/f27 [185271,7867] 0 2024-02-19T19:06:22.748 INFO:tasks.workunit.client.0.smithi088.stdout:7/471: chown d14/f54 46 1 2024-02-19T19:06:22.750 INFO:tasks.workunit.client.0.smithi088.stdout:0/354: symlink d13/d46/d5b/d76/l78 0 2024-02-19T19:06:22.751 INFO:tasks.workunit.client.0.smithi088.stdout:4/375: fdatasync d2/d32/d25/d26/f49 0 2024-02-19T19:06:22.752 INFO:tasks.workunit.client.0.smithi088.stdout:2/302: dwrite d3/fb [0,4194304] 0 2024-02-19T19:06:22.752 INFO:tasks.workunit.client.0.smithi088.stdout:6/440: creat d5/f8b x:0 0 0 2024-02-19T19:06:22.754 INFO:tasks.workunit.client.0.smithi088.stdout:3/518: dread d4/d22/f2b [0,4194304] 0 2024-02-19T19:06:22.755 INFO:tasks.workunit.client.0.smithi088.stdout:2/303: stat d3/d14/f5c 0 2024-02-19T19:06:22.756 INFO:tasks.workunit.client.0.smithi088.stdout:6/441: chown d5/d8/d2c/l6a 29 1 2024-02-19T19:06:22.756 INFO:tasks.workunit.client.0.smithi088.stdout:4/376: write d2/d8/f6b [687826,123836] 0 2024-02-19T19:06:22.757 INFO:tasks.workunit.client.0.smithi088.stdout:1/512: link d6/d21/d64/d94/f66 d6/d21/d6a/d7d/d7e/fac 0 2024-02-19T19:06:22.761 INFO:tasks.workunit.client.0.smithi088.stdout:3/519: write d4/d22/f2c [1651162,31562] 0 2024-02-19T19:06:22.761 INFO:tasks.workunit.client.0.smithi088.stdout:4/377: write d2/d35/f6e [125016,126717] 0 2024-02-19T19:06:22.761 INFO:tasks.workunit.client.0.smithi088.stdout:7/472: link d14/d28/d65/f9d d14/d28/d94/fa5 0 2024-02-19T19:06:22.763 INFO:tasks.workunit.client.0.smithi088.stdout:0/355: creat d13/d46/d5e/f79 x:0 0 0 2024-02-19T19:06:22.766 INFO:tasks.workunit.client.0.smithi088.stdout:1/513: mkdir d6/d21/d6a/d4a/dad 0 2024-02-19T19:06:22.766 INFO:tasks.workunit.client.0.smithi088.stdout:9/437: dread d1/d8/f28 [0,4194304] 0 2024-02-19T19:06:22.768 INFO:tasks.workunit.client.0.smithi088.stdout:7/473: read d14/f22 [3718950,67147] 0 2024-02-19T19:06:22.769 INFO:tasks.workunit.client.0.smithi088.stdout:4/378: mkdir d2/d35/d39/d3a/d77 0 2024-02-19T19:06:22.773 INFO:tasks.workunit.client.0.smithi088.stdout:1/514: rmdir d6/d21/d64/d94/d71 39 2024-02-19T19:06:22.774 INFO:tasks.workunit.client.0.smithi088.stdout:2/304: dwrite d3/d14/f2b [0,4194304] 0 2024-02-19T19:06:22.775 INFO:tasks.workunit.client.0.smithi088.stdout:6/442: dwrite d5/d8/d2c/f40 [4194304,4194304] 0 2024-02-19T19:06:22.777 INFO:tasks.workunit.client.0.smithi088.stdout:8/338: dread d9/df/d1c/f1f [0,4194304] 0 2024-02-19T19:06:22.779 INFO:tasks.workunit.client.0.smithi088.stdout:4/379: symlink d2/l78 0 2024-02-19T19:06:22.780 INFO:tasks.workunit.client.0.smithi088.stdout:4/380: dread - d2/d8/f6a zero size 2024-02-19T19:06:22.780 INFO:tasks.workunit.client.0.smithi088.stdout:9/438: rmdir d1/d21/d2d/d18/d1f 39 2024-02-19T19:06:22.783 INFO:tasks.workunit.client.0.smithi088.stdout:9/439: truncate d1/f10 1913356 0 2024-02-19T19:06:22.783 INFO:tasks.workunit.client.0.smithi088.stdout:7/474: mkdir d14/d1d/d3c/d43/d5c/da6 0 2024-02-19T19:06:22.784 INFO:tasks.workunit.client.0.smithi088.stdout:9/440: readlink d1/d21/l3f 0 2024-02-19T19:06:22.785 INFO:tasks.workunit.client.0.smithi088.stdout:9/441: fsync d1/d8/f77 0 2024-02-19T19:06:22.787 INFO:tasks.workunit.client.0.smithi088.stdout:0/356: rename d13/d46/d5e to d13/d46/d7a 0 2024-02-19T19:06:22.789 INFO:tasks.workunit.client.0.smithi088.stdout:0/357: write d13/d46/d5a/d25/d2a/d3c/f55 [1879620,109238] 0 2024-02-19T19:06:22.791 INFO:tasks.workunit.client.0.smithi088.stdout:4/381: mkdir d2/d32/d25/d26/d4e/d79 0 2024-02-19T19:06:22.794 INFO:tasks.workunit.client.0.smithi088.stdout:7/475: creat d14/d27/d6b/fa7 x:0 0 0 2024-02-19T19:06:22.795 INFO:tasks.workunit.client.0.smithi088.stdout:2/305: creat d3/d1f/d5d/f62 x:0 0 0 2024-02-19T19:06:22.796 INFO:tasks.workunit.client.0.smithi088.stdout:9/442: symlink d1/d21/d2d/d23/d79/l87 0 2024-02-19T19:06:22.797 INFO:tasks.workunit.client.0.smithi088.stdout:4/382: dread d2/d35/d39/d3a/f42 [0,4194304] 0 2024-02-19T19:06:22.799 INFO:tasks.workunit.client.0.smithi088.stdout:0/358: creat d13/d46/d5b/f7b x:0 0 0 2024-02-19T19:06:22.801 INFO:tasks.workunit.client.0.smithi088.stdout:8/339: dwrite d9/df/d1c/d2d/f3e [0,4194304] 0 2024-02-19T19:06:22.803 INFO:tasks.workunit.client.0.smithi088.stdout:7/476: creat d14/d28/d65/fa8 x:0 0 0 2024-02-19T19:06:22.804 INFO:tasks.workunit.client.0.smithi088.stdout:3/520: dwrite d4/d1a/d23/f69 [0,4194304] 0 2024-02-19T19:06:22.805 INFO:tasks.workunit.client.0.smithi088.stdout:8/340: readlink d9/l56 0 2024-02-19T19:06:22.810 INFO:tasks.workunit.client.0.smithi088.stdout:6/443: getdents d5/d19/d26 0 2024-02-19T19:06:22.813 INFO:tasks.workunit.client.0.smithi088.stdout:0/359: creat d13/d46/d5a/d25/d2a/f7c x:0 0 0 2024-02-19T19:06:22.816 INFO:tasks.workunit.client.0.smithi088.stdout:8/341: readlink d9/df/d1c/d2d/d47/l53 0 2024-02-19T19:06:22.820 INFO:tasks.workunit.client.0.smithi088.stdout:2/306: rename d3/f40 to d3/d1f/d4f/d59/d5e/f63 0 2024-02-19T19:06:22.821 INFO:tasks.workunit.client.0.smithi088.stdout:1/515: dwrite d6/d21/d64/d94/f66 [0,4194304] 0 2024-02-19T19:06:22.821 INFO:tasks.workunit.client.0.smithi088.stdout:9/443: dread d1/d21/d2d/d23/f25 [0,4194304] 0 2024-02-19T19:06:22.825 INFO:tasks.workunit.client.0.smithi088.stdout:4/383: dwrite d2/d8/f61 [0,4194304] 0 2024-02-19T19:06:22.829 INFO:tasks.workunit.client.0.smithi088.stdout:0/360: write d13/d46/d5a/d25/d2a/d34/f35 [1931706,117739] 0 2024-02-19T19:06:22.829 INFO:tasks.workunit.client.0.smithi088.stdout:3/521: symlink d4/d22/d66/da4/lb4 0 2024-02-19T19:06:22.833 INFO:tasks.workunit.client.0.smithi088.stdout:7/477: dwrite d14/f17 [8388608,4194304] 0 2024-02-19T19:06:22.833 INFO:tasks.workunit.client.0.smithi088.stdout:4/384: write d2/d35/d39/f6c [426192,97886] 0 2024-02-19T19:06:22.835 INFO:tasks.workunit.client.0.smithi088.stdout:3/522: rmdir d4/d22/d7a 39 2024-02-19T19:06:22.835 INFO:tasks.workunit.client.0.smithi088.stdout:9/444: fsync d1/d8/d37/d54/f65 0 2024-02-19T19:06:22.836 INFO:tasks.workunit.client.0.smithi088.stdout:1/516: write d6/d10/d60/d29/f30 [2588576,731] 0 2024-02-19T19:06:22.836 INFO:tasks.workunit.client.0.smithi088.stdout:7/478: write d14/f51 [578940,81568] 0 2024-02-19T19:06:22.838 INFO:tasks.workunit.client.0.smithi088.stdout:6/444: dread d5/d8/d2c/f4c [0,4194304] 0 2024-02-19T19:06:22.841 INFO:tasks.workunit.client.0.smithi088.stdout:4/385: chown d2/d32/d25/d4b/c64 35 1 2024-02-19T19:06:22.846 INFO:tasks.workunit.client.0.smithi088.stdout:9/445: symlink d1/d21/d2d/d23/l88 0 2024-02-19T19:06:22.847 INFO:tasks.workunit.client.0.smithi088.stdout:0/361: dread d13/d46/d5a/d25/d2a/d34/f35 [0,4194304] 0 2024-02-19T19:06:22.847 INFO:tasks.workunit.client.0.smithi088.stdout:8/342: dwrite d9/df/d1c/d2d/d47/f44 [0,4194304] 0 2024-02-19T19:06:22.851 INFO:tasks.workunit.client.0.smithi088.stdout:2/307: rename d3/d1f/d4f/d59/d5e to d3/d1b/d43/d61/d64 0 2024-02-19T19:06:22.855 INFO:tasks.workunit.client.0.smithi088.stdout:3/523: dread d4/d22/f2b [0,4194304] 0 2024-02-19T19:06:22.856 INFO:tasks.workunit.client.0.smithi088.stdout:4/386: dwrite d2/f75 [0,4194304] 0 2024-02-19T19:06:22.856 INFO:tasks.workunit.client.0.smithi088.stdout:7/479: getdents d14/d1d/d3c/d43/d5c/da6 0 2024-02-19T19:06:22.857 INFO:tasks.workunit.client.0.smithi088.stdout:1/517: getdents d6/d21/d6a/d7d/d9a 0 2024-02-19T19:06:22.857 INFO:tasks.workunit.client.0.smithi088.stdout:8/343: truncate d9/df/d1c/d2d/f54 674976 0 2024-02-19T19:06:22.860 INFO:tasks.workunit.client.0.smithi088.stdout:3/524: chown d4/d1a/d1e/d31/f71 0 1 2024-02-19T19:06:22.861 INFO:tasks.workunit.client.0.smithi088.stdout:7/480: fsync d14/d1d/d3c/d43/d5c/f68 0 2024-02-19T19:06:22.864 INFO:tasks.workunit.client.0.smithi088.stdout:4/387: stat d2/c13 0 2024-02-19T19:06:22.864 INFO:tasks.workunit.client.0.smithi088.stdout:1/518: chown d6/d21/d6a/c62 47973 1 2024-02-19T19:06:22.864 INFO:tasks.workunit.client.0.smithi088.stdout:9/446: truncate d1/d21/d2d/f47 3579926 0 2024-02-19T19:06:22.865 INFO:tasks.workunit.client.0.smithi088.stdout:4/388: write d2/d32/d25/f30 [3197328,42332] 0 2024-02-19T19:06:22.867 INFO:tasks.workunit.client.0.smithi088.stdout:7/481: creat d14/d28/d3b/fa9 x:0 0 0 2024-02-19T19:06:22.868 INFO:tasks.workunit.client.0.smithi088.stdout:9/447: unlink d1/d8/d4a/d4f/f51 0 2024-02-19T19:06:22.869 INFO:tasks.workunit.client.0.smithi088.stdout:1/519: write d6/d21/d6a/d7d/d9a/fa3 [3448420,76867] 0 2024-02-19T19:06:22.870 INFO:tasks.workunit.client.0.smithi088.stdout:1/520: write d6/d10/f78 [710175,69494] 0 2024-02-19T19:06:22.871 INFO:tasks.workunit.client.0.smithi088.stdout:7/482: symlink d14/d1d/d3c/d43/d5c/d75/laa 0 2024-02-19T19:06:22.872 INFO:tasks.workunit.client.0.smithi088.stdout:7/483: fsync f13 0 2024-02-19T19:06:22.873 INFO:tasks.workunit.client.0.smithi088.stdout:7/484: dread - d14/d27/f36 zero size 2024-02-19T19:06:22.876 INFO:tasks.workunit.client.0.smithi088.stdout:2/308: truncate d3/d14/f35 3377083 0 2024-02-19T19:06:22.880 INFO:tasks.workunit.client.0.smithi088.stdout:7/485: creat d14/d1d/d3c/d43/d5c/d9a/d92/fab x:0 0 0 2024-02-19T19:06:22.881 INFO:tasks.workunit.client.0.smithi088.stdout:2/309: dread d3/d14/f19 [0,4194304] 0 2024-02-19T19:06:22.883 INFO:tasks.workunit.client.0.smithi088.stdout:9/448: write d1/d8/d37/d54/f65 [956340,125386] 0 2024-02-19T19:06:22.885 INFO:tasks.workunit.client.0.smithi088.stdout:0/362: dread d13/d46/d5a/d25/d2a/d34/f77 [0,4194304] 0 2024-02-19T19:06:22.888 INFO:tasks.workunit.client.0.smithi088.stdout:1/521: truncate d6/f99 769122 0 2024-02-19T19:06:22.891 INFO:tasks.workunit.client.0.smithi088.stdout:3/525: dread d4/d1a/d23/f5d [0,4194304] 0 2024-02-19T19:06:22.894 INFO:tasks.workunit.client.0.smithi088.stdout:4/389: dread d2/d32/f18 [0,4194304] 0 2024-02-19T19:06:22.894 INFO:tasks.workunit.client.0.smithi088.stdout:9/449: dwrite d1/d21/d2d/d23/f35 [0,4194304] 0 2024-02-19T19:06:22.899 INFO:tasks.workunit.client.0.smithi088.stdout:7/486: truncate d14/f17 11008681 0 2024-02-19T19:06:22.900 INFO:tasks.workunit.client.0.smithi088.stdout:2/310: creat d3/f65 x:0 0 0 2024-02-19T19:06:22.905 INFO:tasks.workunit.client.0.smithi088.stdout:3/526: mknod d4/d22/cb5 0 2024-02-19T19:06:22.908 INFO:tasks.workunit.client.0.smithi088.stdout:9/450: dwrite d1/f10 [0,4194304] 0 2024-02-19T19:06:22.912 INFO:tasks.workunit.client.0.smithi088.stdout:4/390: dwrite d2/f4 [0,4194304] 0 2024-02-19T19:06:22.912 INFO:tasks.workunit.client.0.smithi088.stdout:2/311: creat d3/d1b/d43/f66 x:0 0 0 2024-02-19T19:06:22.913 INFO:tasks.workunit.client.0.smithi088.stdout:3/527: chown d4/d1a/d1e/d31/d32/l53 6090 1 2024-02-19T19:06:22.914 INFO:tasks.workunit.client.0.smithi088.stdout:0/363: dread d13/d46/d5a/d25/d2a/d34/f4d [0,4194304] 0 2024-02-19T19:06:22.915 INFO:tasks.workunit.client.0.smithi088.stdout:0/364: dread - d13/d46/d7a/f79 zero size 2024-02-19T19:06:22.915 INFO:tasks.workunit.client.0.smithi088.stdout:7/487: getdents d14/d1d/d3c/d43/d5c/d75/d76 0 2024-02-19T19:06:22.918 INFO:tasks.workunit.client.0.smithi088.stdout:7/488: chown d14/d27/d6b/f7b 1 1 2024-02-19T19:06:22.918 INFO:tasks.workunit.client.0.smithi088.stdout:2/312: fdatasync d3/d1f/d5d/f62 0 2024-02-19T19:06:22.919 INFO:tasks.workunit.client.0.smithi088.stdout:3/528: write d4/f7 [4377628,19222] 0 2024-02-19T19:06:22.919 INFO:tasks.workunit.client.0.smithi088.stdout:4/391: chown d2/d32/d9/f71 43520 1 2024-02-19T19:06:22.921 INFO:tasks.workunit.client.0.smithi088.stdout:7/489: chown d14/d28/l53 3 1 2024-02-19T19:06:22.926 INFO:tasks.workunit.client.0.smithi088.stdout:2/313: creat d3/d1b/f67 x:0 0 0 2024-02-19T19:06:22.927 INFO:tasks.workunit.client.0.smithi088.stdout:1/522: dread d6/f1e [0,4194304] 0 2024-02-19T19:06:22.929 INFO:tasks.workunit.client.0.smithi088.stdout:3/529: dread - d4/d12/d33/d43/d85/fae zero size 2024-02-19T19:06:22.929 INFO:tasks.workunit.client.0.smithi088.stdout:2/314: chown d3/c29 0 1 2024-02-19T19:06:22.930 INFO:tasks.workunit.client.0.smithi088.stdout:7/490: mknod d14/d1d/d3c/d43/d5c/d9a/d92/cac 0 2024-02-19T19:06:22.932 INFO:tasks.workunit.client.0.smithi088.stdout:2/315: mknod d3/d1b/d43/c68 0 2024-02-19T19:06:22.934 INFO:tasks.workunit.client.0.smithi088.stdout:9/451: dwrite d1/d60/f85 [0,4194304] 0 2024-02-19T19:06:22.934 INFO:tasks.workunit.client.0.smithi088.stdout:1/523: creat d6/d10/d60/fae x:0 0 0 2024-02-19T19:06:22.936 INFO:tasks.workunit.client.0.smithi088.stdout:3/530: truncate d4/fd 5452768 0 2024-02-19T19:06:22.937 INFO:tasks.workunit.client.0.smithi088.stdout:0/365: dwrite d13/d46/d5a/d25/d2a/d3c/f55 [0,4194304] 0 2024-02-19T19:06:22.938 INFO:tasks.workunit.client.0.smithi088.stdout:2/316: mknod d3/d1f/d5d/c69 0 2024-02-19T19:06:22.945 INFO:tasks.workunit.client.0.smithi088.stdout:9/452: dwrite d1/d60/f6c [0,4194304] 0 2024-02-19T19:06:22.946 INFO:tasks.workunit.client.0.smithi088.stdout:3/531: rename d4/f14 to d4/d87/fb6 0 2024-02-19T19:06:22.947 INFO:tasks.workunit.client.0.smithi088.stdout:2/317: fdatasync d3/d1b/f3e 0 2024-02-19T19:06:22.950 INFO:tasks.workunit.client.0.smithi088.stdout:0/366: rename d13/d46/d5a/d25 to d13/d46/d5a/d25/d7d 22 2024-02-19T19:06:22.950 INFO:tasks.workunit.client.0.smithi088.stdout:9/453: truncate d1/d8/d37/d54/f75 984441 0 2024-02-19T19:06:22.953 INFO:tasks.workunit.client.0.smithi088.stdout:0/367: chown d13/d46/d5a/c54 917147 1 2024-02-19T19:06:22.954 INFO:tasks.workunit.client.0.smithi088.stdout:9/454: truncate d1/d8/d3e/f45 572261 0 2024-02-19T19:06:22.954 INFO:tasks.workunit.client.0.smithi088.stdout:2/318: write d3/d14/f2b [3059313,6614] 0 2024-02-19T19:06:22.956 INFO:tasks.workunit.client.0.smithi088.stdout:3/532: dread d4/d1a/d23/f60 [0,4194304] 0 2024-02-19T19:06:22.959 INFO:tasks.workunit.client.0.smithi088.stdout:0/368: mknod d13/d46/d5a/d25/d2a/c7e 0 2024-02-19T19:06:22.960 INFO:tasks.workunit.client.0.smithi088.stdout:3/533: truncate d4/d12/f21 4015531 0 2024-02-19T19:06:22.966 INFO:tasks.workunit.client.0.smithi088.stdout:2/319: dwrite d3/d1b/f67 [0,4194304] 0 2024-02-19T19:06:22.967 INFO:tasks.workunit.client.0.smithi088.stdout:9/455: dwrite d1/d13/f12 [0,4194304] 0 2024-02-19T19:06:22.970 INFO:tasks.workunit.client.0.smithi088.stdout:2/320: fdatasync d3/d1b/d43/f66 0 2024-02-19T19:06:22.972 INFO:tasks.workunit.client.0.smithi088.stdout:1/524: dread d6/d21/f3a [0,4194304] 0 2024-02-19T19:06:22.972 INFO:tasks.workunit.client.0.smithi088.stdout:3/534: rename d4/d22/d66/da4/lb4 to d4/d12/d33/d58/lb7 0 2024-02-19T19:06:22.976 INFO:tasks.workunit.client.0.smithi088.stdout:0/369: truncate d13/f23 1162075 0 2024-02-19T19:06:22.982 INFO:tasks.workunit.client.0.smithi088.stdout:1/525: creat d6/d21/d6a/d4a/faf x:0 0 0 2024-02-19T19:06:22.985 INFO:tasks.workunit.client.0.smithi088.stdout:2/321: dwrite d3/d1b/f67 [4194304,4194304] 0 2024-02-19T19:06:22.985 INFO:tasks.workunit.client.0.smithi088.stdout:9/456: symlink d1/d21/d2d/d18/d1f/l89 0 2024-02-19T19:06:22.988 INFO:tasks.workunit.client.0.smithi088.stdout:9/457: symlink d1/d13/l8a 0 2024-02-19T19:06:22.993 INFO:tasks.workunit.client.0.smithi088.stdout:2/322: dwrite d3/d1b/d43/f55 [0,4194304] 0 2024-02-19T19:06:22.997 INFO:tasks.workunit.client.0.smithi088.stdout:1/526: dwrite d6/d21/f3a [0,4194304] 0 2024-02-19T19:06:22.999 INFO:tasks.workunit.client.0.smithi088.stdout:9/458: link d1/d21/d2d/d23/c56 d1/d8/d3e/d4e/d74/c8b 0 2024-02-19T19:06:23.004 INFO:tasks.workunit.client.0.smithi088.stdout:2/323: dwrite d3/d1b/f5b [0,4194304] 0 2024-02-19T19:06:23.004 INFO:tasks.workunit.client.0.smithi088.stdout:9/459: mkdir d1/d8/d3e/d4e/d74/d8c 0 2024-02-19T19:06:23.006 INFO:tasks.workunit.client.0.smithi088.stdout:9/460: fsync d1/d21/d2d/d18/d1f/f59 0 2024-02-19T19:06:23.008 INFO:tasks.workunit.client.0.smithi088.stdout:2/324: write d3/f3b [3552257,15246] 0 2024-02-19T19:06:23.009 INFO:tasks.workunit.client.0.smithi088.stdout:2/325: truncate d3/f3b 4871563 0 2024-02-19T19:06:23.009 INFO:tasks.workunit.client.0.smithi088.stdout:9/461: mkdir d1/d8/d8d 0 2024-02-19T19:06:23.015 INFO:tasks.workunit.client.0.smithi088.stdout:9/462: symlink d1/d21/d2d/l8e 0 2024-02-19T19:06:23.021 INFO:tasks.workunit.client.0.smithi088.stdout:9/463: dwrite d1/d8/d3e/f70 [0,4194304] 0 2024-02-19T19:06:23.024 INFO:tasks.workunit.client.0.smithi088.stdout:3/535: write d4/d1a/d1e/f2f [5864841,73842] 0 2024-02-19T19:06:23.025 INFO:tasks.workunit.client.0.smithi088.stdout:9/464: chown d1/d21/d2d/f47 381 1 2024-02-19T19:06:23.026 INFO:tasks.workunit.client.0.smithi088.stdout:9/465: chown d1/d21/d2d/f4c 659171648 1 2024-02-19T19:06:23.026 INFO:tasks.workunit.client.0.smithi088.stdout:3/536: stat d4/c5b 0 2024-02-19T19:06:23.026 INFO:tasks.workunit.client.0.smithi088.stdout:3/537: readlink d4/d22/l49 0 2024-02-19T19:06:23.027 INFO:tasks.workunit.client.0.smithi088.stdout:1/527: truncate d6/d21/d6a/d7d/d7e/fac 312452 0 2024-02-19T19:06:23.031 INFO:tasks.workunit.client.0.smithi088.stdout:1/528: creat d6/d21/d64/fb0 x:0 0 0 2024-02-19T19:06:23.032 INFO:tasks.workunit.client.0.smithi088.stdout:3/538: dread d4/ff [0,4194304] 0 2024-02-19T19:06:23.035 INFO:tasks.workunit.client.0.smithi088.stdout:9/466: link d1/d21/d2d/f16 d1/d21/d2d/d18/f8f 0 2024-02-19T19:06:23.037 INFO:tasks.workunit.client.0.smithi088.stdout:1/529: unlink d6/d10/d60/l76 0 2024-02-19T19:06:23.040 INFO:tasks.workunit.client.0.smithi088.stdout:3/539: creat d4/d1a/fb8 x:0 0 0 2024-02-19T19:06:23.041 INFO:tasks.workunit.client.0.smithi088.stdout:1/530: creat d6/d21/d6a/d4a/fb1 x:0 0 0 2024-02-19T19:06:23.043 INFO:tasks.workunit.client.0.smithi088.stdout:1/531: dread - d6/d10/d60/f67 zero size 2024-02-19T19:06:23.044 INFO:tasks.workunit.client.0.smithi088.stdout:9/467: dread d1/d21/d2d/d23/f26 [0,4194304] 0 2024-02-19T19:06:23.044 INFO:tasks.workunit.client.0.smithi088.stdout:2/326: dread d3/d14/f1d [0,4194304] 0 2024-02-19T19:06:23.047 INFO:tasks.workunit.client.0.smithi088.stdout:9/468: creat d1/d60/f90 x:0 0 0 2024-02-19T19:06:23.048 INFO:tasks.workunit.client.0.smithi088.stdout:3/540: dwrite d4/f2d [0,4194304] 0 2024-02-19T19:06:23.050 INFO:tasks.workunit.client.0.smithi088.stdout:1/532: rename d6/d21/fa9 to d6/d21/d6a/d7d/d9a/fb2 0 2024-02-19T19:06:23.052 INFO:tasks.workunit.client.0.smithi088.stdout:1/533: chown f5 127686987 1 2024-02-19T19:06:23.053 INFO:tasks.workunit.client.0.smithi088.stdout:1/534: chown d6/d10/l44 2102967604 1 2024-02-19T19:06:23.056 INFO:tasks.workunit.client.0.smithi088.stdout:9/469: dwrite d1/d21/d2d/d23/f35 [0,4194304] 0 2024-02-19T19:06:23.068 INFO:tasks.workunit.client.0.smithi088.stdout:3/541: rmdir d4 39 2024-02-19T19:06:23.069 INFO:tasks.workunit.client.0.smithi088.stdout:2/327: dread d3/f6 [0,4194304] 0 2024-02-19T19:06:23.069 INFO:tasks.workunit.client.0.smithi088.stdout:9/470: mknod d1/d21/d6f/c91 0 2024-02-19T19:06:23.072 INFO:tasks.workunit.client.0.smithi088.stdout:2/328: creat d3/d1b/d43/d61/d64/f6a x:0 0 0 2024-02-19T19:06:23.073 INFO:tasks.workunit.client.0.smithi088.stdout:1/535: write d6/d21/f72 [719943,61691] 0 2024-02-19T19:06:23.075 INFO:tasks.workunit.client.0.smithi088.stdout:1/536: write d6/d21/d64/d94/d6e/f7c [2534008,82754] 0 2024-02-19T19:06:23.075 INFO:tasks.workunit.client.0.smithi088.stdout:9/471: mkdir d1/d21/d2d/d18/d92 0 2024-02-19T19:06:23.076 INFO:tasks.workunit.client.0.smithi088.stdout:1/537: dread - d6/d10/d60/fae zero size 2024-02-19T19:06:23.077 INFO:tasks.workunit.client.0.smithi088.stdout:2/329: symlink d3/d1b/l6b 0 2024-02-19T19:06:23.077 INFO:tasks.workunit.client.0.smithi088.stdout:3/542: creat d4/d22/d7a/d8d/fb9 x:0 0 0 2024-02-19T19:06:23.079 INFO:tasks.workunit.client.0.smithi088.stdout:2/330: truncate d3/d1f/d4f/f17 594824 0 2024-02-19T19:06:23.080 INFO:tasks.workunit.client.0.smithi088.stdout:3/543: write d4/d1a/d1e/d74/fa6 [3556886,68970] 0 2024-02-19T19:06:23.082 INFO:tasks.workunit.client.0.smithi088.stdout:9/472: dread d1/d21/d2d/d23/f35 [0,4194304] 0 2024-02-19T19:06:23.082 INFO:tasks.workunit.client.0.smithi088.stdout:3/544: read - d4/d12/d33/d43/d85/fae zero size 2024-02-19T19:06:23.084 INFO:tasks.workunit.client.0.smithi088.stdout:9/473: dread - d1/d21/d2d/d23/d79/f86 zero size 2024-02-19T19:06:23.084 INFO:tasks.workunit.client.0.smithi088.stdout:2/331: mknod d3/d1b/d43/d61/d64/c6c 0 2024-02-19T19:06:23.085 INFO:tasks.workunit.client.0.smithi088.stdout:3/545: chown d4/d1a/d1e/d31/d32/d55/lb1 277927943 1 2024-02-19T19:06:23.086 INFO:tasks.workunit.client.0.smithi088.stdout:2/332: unlink d3/d1b/f3e 0 2024-02-19T19:06:23.087 INFO:tasks.workunit.client.0.smithi088.stdout:2/333: dread - d3/f65 zero size 2024-02-19T19:06:23.087 INFO:tasks.workunit.client.0.smithi088.stdout:9/474: truncate d1/d21/d2d/d23/f26 256690 0 2024-02-19T19:06:23.087 INFO:tasks.workunit.client.0.smithi088.stdout:2/334: chown d3/d1f/d4f/l3d 442 1 2024-02-19T19:06:23.089 INFO:tasks.workunit.client.0.smithi088.stdout:2/335: read d3/d1b/f1c [1212369,83554] 0 2024-02-19T19:06:23.091 INFO:tasks.workunit.client.0.smithi088.stdout:2/336: fsync d3/f6 0 2024-02-19T19:06:23.092 INFO:tasks.workunit.client.0.smithi088.stdout:2/337: creat d3/d1b/d43/d61/d64/f6d x:0 0 0 2024-02-19T19:06:23.097 INFO:tasks.workunit.client.0.smithi088.stdout:2/338: dread d3/d1b/f67 [4194304,4194304] 0 2024-02-19T19:06:23.100 INFO:tasks.workunit.client.0.smithi088.stdout:2/339: unlink d3/d14/f35 0 2024-02-19T19:06:23.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:22 smithi088 ceph-mon[88643]: Reconfiguring mds.cephfs.smithi132.afhgdi (monmap changed)... 2024-02-19T19:06:23.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:22 smithi088 ceph-mon[88643]: Reconfiguring daemon mds.cephfs.smithi132.afhgdi on smithi132 2024-02-19T19:06:23.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:22 smithi088 ceph-mon[88643]: pgmap v45: 65 pgs: 65 active+clean; 946 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 21 MiB/s rd, 108 MiB/s wr, 305 op/s 2024-02-19T19:06:23.101 INFO:tasks.workunit.client.0.smithi088.stdout:9/475: rename d1/d21 to d1/d8/d3e/d93 0 2024-02-19T19:06:23.106 INFO:tasks.workunit.client.0.smithi088.stdout:2/340: dwrite d3/d14/f5a [4194304,4194304] 0 2024-02-19T19:06:23.111 INFO:tasks.workunit.client.0.smithi088.stdout:2/341: link d3/d14/l2c d3/d1b/l6e 0 2024-02-19T19:06:23.112 INFO:tasks.workunit.client.0.smithi088.stdout:9/476: write d1/d8/d3e/d93/d2d/d23/f26 [189283,116228] 0 2024-02-19T19:06:23.112 INFO:tasks.workunit.client.0.smithi088.stdout:2/342: fsync d3/d1b/f67 0 2024-02-19T19:06:23.114 INFO:tasks.workunit.client.0.smithi088.stdout:2/343: write d3/d1b/d43/f66 [853897,57392] 0 2024-02-19T19:06:23.116 INFO:tasks.workunit.client.0.smithi088.stdout:9/477: rename d1/d8/d3e/d4e to d1/d8/d3e/d93/d5a/d94 0 2024-02-19T19:06:23.116 INFO:tasks.workunit.client.0.smithi088.stdout:2/344: mknod d3/d1f/d4f/d59/c6f 0 2024-02-19T19:06:23.119 INFO:tasks.workunit.client.0.smithi088.stdout:9/478: rename d1/d8/d3e/d93/l3f to d1/d8/d3e/d93/d6f/l95 0 2024-02-19T19:06:23.122 INFO:tasks.workunit.client.0.smithi088.stdout:9/479: fsync d1/d8/d3e/d93/d2d/d23/f25 0 2024-02-19T19:06:23.123 INFO:tasks.workunit.client.0.smithi088.stdout:2/345: dwrite d3/d1f/f4a [4194304,4194304] 0 2024-02-19T19:06:23.154 INFO:tasks.workunit.client.0.smithi088.stdout:5/515: sync 2024-02-19T19:06:23.155 INFO:tasks.workunit.client.0.smithi088.stdout:2/346: readlink d3/d1f/d4f/l26 0 2024-02-19T19:06:23.155 INFO:tasks.workunit.client.0.smithi088.stdout:5/516: fdatasync d4/d1a/d41/f71 0 2024-02-19T19:06:23.156 INFO:tasks.workunit.client.0.smithi088.stdout:5/517: write d4/f42 [1225682,84429] 0 2024-02-19T19:06:23.156 INFO:tasks.workunit.client.0.smithi088.stdout:9/480: rmdir d1/d8/d3e/d93/d2d/d18 39 2024-02-19T19:06:23.157 INFO:tasks.workunit.client.0.smithi088.stdout:5/518: dread - d4/d7/f92 zero size 2024-02-19T19:06:23.159 INFO:tasks.workunit.client.0.smithi088.stdout:9/481: unlink d1/d8/d3e/c82 0 2024-02-19T19:06:23.160 INFO:tasks.workunit.client.0.smithi088.stdout:9/482: chown d1/d8/d3e/d93/d2d/d23/f26 14611 1 2024-02-19T19:06:23.160 INFO:tasks.workunit.client.0.smithi088.stdout:9/483: chown d1/d8/d4a 10942304 1 2024-02-19T19:06:23.163 INFO:tasks.workunit.client.0.smithi088.stdout:5/519: creat d4/d61/d65/d9a/fac x:0 0 0 2024-02-19T19:06:23.164 INFO:tasks.workunit.client.0.smithi088.stdout:2/347: rmdir d3 39 2024-02-19T19:06:23.166 INFO:tasks.workunit.client.0.smithi088.stdout:5/520: chown d4/d1a/f3c 9619762 1 2024-02-19T19:06:23.167 INFO:tasks.workunit.client.0.smithi088.stdout:5/521: mknod d4/d61/cad 0 2024-02-19T19:06:23.168 INFO:tasks.workunit.client.0.smithi088.stdout:2/348: mknod d3/d1f/d5d/c70 0 2024-02-19T19:06:23.169 INFO:tasks.workunit.client.0.smithi088.stdout:2/349: fsync d3/d1b/f5b 0 2024-02-19T19:06:23.169 INFO:tasks.workunit.client.0.smithi088.stdout:9/484: rename d1/d8/d3e/d93/d2d/d18/d1f/c6e to d1/d8/d3e/d93/d2d/d18/d1f/c96 0 2024-02-19T19:06:23.170 INFO:tasks.workunit.client.0.smithi088.stdout:9/485: chown d1/d8/d3e/d93/d2d/d18/d1f/f2e 3290 1 2024-02-19T19:06:23.171 INFO:tasks.workunit.client.0.smithi088.stdout:2/350: mkdir d3/d71 0 2024-02-19T19:06:23.173 INFO:tasks.workunit.client.0.smithi088.stdout:2/351: unlink d3/d14/f2b 0 2024-02-19T19:06:23.176 INFO:tasks.workunit.client.0.smithi088.stdout:2/352: creat d3/d1f/d5d/f72 x:0 0 0 2024-02-19T19:06:23.178 INFO:tasks.workunit.client.0.smithi088.stdout:2/353: write d3/d1f/d4f/d59/f5f [228533,80187] 0 2024-02-19T19:06:23.179 INFO:tasks.workunit.client.0.smithi088.stdout:2/354: dread - d3/d14/f56 zero size 2024-02-19T19:06:23.181 INFO:tasks.workunit.client.0.smithi088.stdout:2/355: write d3/d1f/d4f/d59/f5f [661350,63939] 0 2024-02-19T19:06:23.182 INFO:tasks.workunit.client.0.smithi088.stdout:2/356: chown d3/d1f 242 1 2024-02-19T19:06:23.183 INFO:tasks.workunit.client.0.smithi088.stdout:9/486: dwrite d1/f2b [0,4194304] 0 2024-02-19T19:06:23.187 INFO:tasks.workunit.client.0.smithi088.stdout:9/487: rename d1/d13 to d1/d8/d8d/d97 0 2024-02-19T19:06:23.188 INFO:tasks.workunit.client.0.smithi088.stdout:2/357: dread d3/d1b/d43/f52 [0,4194304] 0 2024-02-19T19:06:23.188 INFO:tasks.workunit.client.0.smithi088.stdout:9/488: dread - d1/d60/f90 zero size 2024-02-19T19:06:23.191 INFO:tasks.workunit.client.0.smithi088.stdout:9/489: mknod d1/d8/d3e/d93/d5a/c98 0 2024-02-19T19:06:23.192 INFO:tasks.workunit.client.0.smithi088.stdout:9/490: write d1/d8/d37/d54/f55 [448807,123301] 0 2024-02-19T19:06:23.192 INFO:tasks.workunit.client.0.smithi088.stdout:2/358: getdents d3/d14 0 2024-02-19T19:06:23.195 INFO:tasks.workunit.client.0.smithi088.stdout:9/491: mkdir d1/d8/d3e/d93/d2d/d23/d79/d99 0 2024-02-19T19:06:23.197 INFO:tasks.workunit.client.0.smithi088.stdout:9/492: creat d1/d8/d37/f9a x:0 0 0 2024-02-19T19:06:23.201 INFO:tasks.workunit.client.0.smithi088.stdout:9/493: link d1/d8/d3e/d93/d2d/c5f d1/d8/d3e/d93/d2d/d23/c9b 0 2024-02-19T19:06:23.204 INFO:tasks.workunit.client.0.smithi088.stdout:9/494: creat d1/d8/d37/f9c x:0 0 0 2024-02-19T19:06:23.205 INFO:tasks.workunit.client.0.smithi088.stdout:2/359: dwrite d3/f31 [0,4194304] 0 2024-02-19T19:06:23.208 INFO:tasks.workunit.client.0.smithi088.stdout:9/495: creat d1/d8/d3e/d93/d2d/d23/d79/f9d x:0 0 0 2024-02-19T19:06:23.209 INFO:tasks.workunit.client.0.smithi088.stdout:9/496: fdatasync d1/d8/d3e/f45 0 2024-02-19T19:06:23.212 INFO:tasks.workunit.client.0.smithi088.stdout:2/360: dwrite d3/d1b/f67 [4194304,4194304] 0 2024-02-19T19:06:23.224 INFO:tasks.workunit.client.0.smithi088.stdout:2/361: dwrite d3/d1f/f32 [0,4194304] 0 2024-02-19T19:06:23.225 INFO:tasks.workunit.client.0.smithi088.stdout:2/362: write d3/d1f/f4a [2895594,12938] 0 2024-02-19T19:06:23.235 INFO:tasks.workunit.client.0.smithi088.stdout:2/363: write d3/d14/f1d [4279085,16235] 0 2024-02-19T19:06:23.236 INFO:tasks.workunit.client.0.smithi088.stdout:2/364: chown d3/d14/c1a 420949987 1 2024-02-19T19:06:23.238 INFO:tasks.workunit.client.0.smithi088.stdout:2/365: creat d3/d1f/f73 x:0 0 0 2024-02-19T19:06:23.239 INFO:tasks.workunit.client.0.smithi088.stdout:2/366: read d3/f6 [5410797,117660] 0 2024-02-19T19:06:23.240 INFO:tasks.workunit.client.0.smithi088.stdout:2/367: rmdir d3 39 2024-02-19T19:06:23.245 INFO:tasks.workunit.client.0.smithi088.stdout:2/368: read d3/d1f/d4f/f11 [2095856,34471] 0 2024-02-19T19:06:23.246 INFO:tasks.workunit.client.0.smithi088.stdout:2/369: mkdir d3/d1b/d43/d74 0 2024-02-19T19:06:23.247 INFO:tasks.workunit.client.0.smithi088.stdout:2/370: readlink d3/l45 0 2024-02-19T19:06:23.249 INFO:tasks.workunit.client.0.smithi088.stdout:2/371: symlink d3/d1f/d5d/l75 0 2024-02-19T19:06:23.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:22 smithi132 ceph-mon[70676]: Reconfiguring mds.cephfs.smithi132.afhgdi (monmap changed)... 2024-02-19T19:06:23.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:22 smithi132 ceph-mon[70676]: Reconfiguring daemon mds.cephfs.smithi132.afhgdi on smithi132 2024-02-19T19:06:23.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:22 smithi132 ceph-mon[70676]: pgmap v45: 65 pgs: 65 active+clean; 946 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 21 MiB/s rd, 108 MiB/s wr, 305 op/s 2024-02-19T19:06:23.253 INFO:tasks.workunit.client.0.smithi088.stdout:2/372: link d3/d1f/l34 d3/d1f/l76 0 2024-02-19T19:06:23.254 INFO:tasks.workunit.client.0.smithi088.stdout:2/373: mkdir d3/d1b/d43/d61/d77 0 2024-02-19T19:06:23.254 INFO:tasks.workunit.client.0.smithi088.stdout:2/374: dread - d3/d14/f5c zero size 2024-02-19T19:06:23.256 INFO:tasks.workunit.client.0.smithi088.stdout:2/375: write d3/d1b/d43/f58 [2844334,87357] 0 2024-02-19T19:06:23.319 INFO:tasks.workunit.client.0.smithi088.stdout:6/445: sync 2024-02-19T19:06:23.319 INFO:tasks.workunit.client.0.smithi088.stdout:8/344: sync 2024-02-19T19:06:23.324 INFO:tasks.workunit.client.0.smithi088.stdout:8/345: link d9/df/d1c/d2d/f3e d9/df/d3d/f5f 0 2024-02-19T19:06:23.325 INFO:tasks.workunit.client.0.smithi088.stdout:8/346: write d9/df/d1c/d2d/d47/f43 [2267690,98179] 0 2024-02-19T19:06:23.328 INFO:tasks.workunit.client.0.smithi088.stdout:8/347: mknod d9/df/c60 0 2024-02-19T19:06:23.332 INFO:tasks.workunit.client.0.smithi088.stdout:6/446: dwrite d5/d8/d3a/f46 [4194304,4194304] 0 2024-02-19T19:06:23.332 INFO:tasks.workunit.client.0.smithi088.stdout:6/447: fdatasync d5/f1e 0 2024-02-19T19:06:23.338 INFO:tasks.workunit.client.0.smithi088.stdout:6/448: rename d5/d31 to d5/de/d64/d82/d8c 0 2024-02-19T19:06:23.339 INFO:tasks.workunit.client.0.smithi088.stdout:8/348: dwrite f5 [0,4194304] 0 2024-02-19T19:06:23.340 INFO:tasks.workunit.client.0.smithi088.stdout:6/449: write d5/de/d64/d82/d8c/f59 [64364,63513] 0 2024-02-19T19:06:23.348 INFO:tasks.workunit.client.0.smithi088.stdout:8/349: rename d9/df/d1c/d2d/d58/d5d/l59 to d9/df/d3d/d50/l61 0 2024-02-19T19:06:23.349 INFO:tasks.workunit.client.0.smithi088.stdout:6/450: dread d5/d7/f4b [0,4194304] 0 2024-02-19T19:06:23.351 INFO:tasks.workunit.client.0.smithi088.stdout:6/451: mkdir d5/de/d64/d8d 0 2024-02-19T19:06:23.354 INFO:tasks.workunit.client.0.smithi088.stdout:6/452: creat d5/d7/d32/f8e x:0 0 0 2024-02-19T19:06:23.355 INFO:tasks.workunit.client.0.smithi088.stdout:8/350: dwrite d9/df/d1c/d2d/f54 [0,4194304] 0 2024-02-19T19:06:23.358 INFO:tasks.workunit.client.0.smithi088.stdout:8/351: mkdir d9/df/d3d/d62 0 2024-02-19T19:06:23.360 INFO:tasks.workunit.client.0.smithi088.stdout:8/352: chown d9/df/d1c/d2d/d58/l5a 1306 1 2024-02-19T19:06:23.361 INFO:tasks.workunit.client.0.smithi088.stdout:6/453: dwrite d5/d8/d2c/f6c [0,4194304] 0 2024-02-19T19:06:23.367 INFO:tasks.workunit.client.0.smithi088.stdout:8/353: write d9/df/d1c/f1b [6865537,32405] 0 2024-02-19T19:06:23.367 INFO:tasks.workunit.client.0.smithi088.stdout:6/454: write d5/f47 [893330,23044] 0 2024-02-19T19:06:23.370 INFO:tasks.workunit.client.0.smithi088.stdout:6/455: rename d5/d19/c34 to d5/de/d64/c8f 0 2024-02-19T19:06:23.371 INFO:tasks.workunit.client.0.smithi088.stdout:8/354: creat d9/f63 x:0 0 0 2024-02-19T19:06:23.373 INFO:tasks.workunit.client.0.smithi088.stdout:6/456: symlink d5/d19/l90 0 2024-02-19T19:06:23.375 INFO:tasks.workunit.client.0.smithi088.stdout:6/457: mkdir d5/d8/d3a/d91 0 2024-02-19T19:06:23.377 INFO:tasks.workunit.client.0.smithi088.stdout:6/458: mkdir d5/d7/d92 0 2024-02-19T19:06:23.378 INFO:tasks.workunit.client.0.smithi088.stdout:8/355: dread d9/f4a [0,4194304] 0 2024-02-19T19:06:23.379 INFO:tasks.workunit.client.0.smithi088.stdout:6/459: read d5/ff [646764,103175] 0 2024-02-19T19:06:23.379 INFO:tasks.workunit.client.0.smithi088.stdout:6/460: chown d5/d7/l7c 39 1 2024-02-19T19:06:23.385 INFO:tasks.workunit.client.0.smithi088.stdout:6/461: dwrite d5/de/d60/f87 [0,4194304] 0 2024-02-19T19:06:23.387 INFO:tasks.workunit.client.0.smithi088.stdout:6/462: unlink d5/d8/d2c/l6a 0 2024-02-19T19:06:23.388 INFO:tasks.workunit.client.0.smithi088.stdout:6/463: write d5/d45/f75 [413956,53403] 0 2024-02-19T19:06:23.390 INFO:tasks.workunit.client.0.smithi088.stdout:6/464: rmdir d5/de 39 2024-02-19T19:06:23.391 INFO:tasks.workunit.client.0.smithi088.stdout:8/356: dread f8 [0,4194304] 0 2024-02-19T19:06:23.391 INFO:tasks.workunit.client.0.smithi088.stdout:6/465: write d5/d8/d2c/f4c [1781599,128843] 0 2024-02-19T19:06:23.392 INFO:tasks.workunit.client.0.smithi088.stdout:8/357: stat d9/df/d1c/d2d/f4b 0 2024-02-19T19:06:23.392 INFO:tasks.workunit.client.0.smithi088.stdout:6/466: chown d5/d8/d3a/d71/c89 2036553067 1 2024-02-19T19:06:23.394 INFO:tasks.workunit.client.0.smithi088.stdout:8/358: mknod d9/df/d1c/d22/c64 0 2024-02-19T19:06:23.395 INFO:tasks.workunit.client.0.smithi088.stdout:6/467: symlink d5/d7/d32/l93 0 2024-02-19T19:06:23.396 INFO:tasks.workunit.client.0.smithi088.stdout:8/359: chown d9/df/d1c/d2d/d47/c51 2 1 2024-02-19T19:06:23.396 INFO:tasks.workunit.client.0.smithi088.stdout:8/360: chown d9/df/d1c/d22 431567886 1 2024-02-19T19:06:23.398 INFO:tasks.workunit.client.0.smithi088.stdout:6/468: creat d5/d7/d32/d56/f94 x:0 0 0 2024-02-19T19:06:23.401 INFO:tasks.workunit.client.0.smithi088.stdout:6/469: read d5/d8/d2c/f40 [2106954,83123] 0 2024-02-19T19:06:23.403 INFO:tasks.workunit.client.0.smithi088.stdout:8/361: dwrite d9/df/d1c/d2d/d47/f5e [0,4194304] 0 2024-02-19T19:06:23.407 INFO:tasks.workunit.client.0.smithi088.stdout:6/470: read d5/de/f22 [114346,2783] 0 2024-02-19T19:06:23.408 INFO:tasks.workunit.client.0.smithi088.stdout:6/471: readlink d5/d19/d70/d83/l88 0 2024-02-19T19:06:23.408 INFO:tasks.workunit.client.0.smithi088.stdout:8/362: creat d9/df/d1c/d2d/d58/d5d/f65 x:0 0 0 2024-02-19T19:06:23.408 INFO:tasks.workunit.client.0.smithi088.stdout:6/472: mkdir d5/de/d64/d8d/d95 0 2024-02-19T19:06:23.410 INFO:tasks.workunit.client.0.smithi088.stdout:8/363: rename f8 to d9/df/d1c/d22/f66 0 2024-02-19T19:06:23.418 INFO:tasks.workunit.client.0.smithi088.stdout:6/473: dwrite d5/ff [4194304,4194304] 0 2024-02-19T19:06:23.422 INFO:tasks.workunit.client.0.smithi088.stdout:4/392: sync 2024-02-19T19:06:23.425 INFO:tasks.workunit.client.0.smithi088.stdout:6/474: rename d5/d8/c50 to d5/de/d64/d82/d8c/d25/c96 0 2024-02-19T19:06:23.431 INFO:tasks.workunit.client.0.smithi088.stdout:4/393: creat d2/d32/d25/d26/f7a x:0 0 0 2024-02-19T19:06:23.431 INFO:tasks.workunit.client.0.smithi088.stdout:6/475: fdatasync d5/d8/d3a/f63 0 2024-02-19T19:06:23.432 INFO:tasks.workunit.client.0.smithi088.stdout:4/394: write d2/d32/f36 [40438,72262] 0 2024-02-19T19:06:23.437 INFO:tasks.workunit.client.0.smithi088.stdout:8/364: fsync d9/df/d1c/d2d/d47/f5e 0 2024-02-19T19:06:23.439 INFO:tasks.workunit.client.0.smithi088.stdout:6/476: write d5/d7/f14 [3534336,103783] 0 2024-02-19T19:06:23.440 INFO:tasks.workunit.client.0.smithi088.stdout:8/365: creat d9/df/d1c/d2d/f67 x:0 0 0 2024-02-19T19:06:23.445 INFO:tasks.workunit.client.0.smithi088.stdout:8/366: rename d9/df/d1c/d2d/f67 to d9/df/d1c/d2d/d58/d5d/f68 0 2024-02-19T19:06:23.548 INFO:tasks.workunit.client.0.smithi088.stdout:6/477: truncate d5/de/f68 3406499 0 2024-02-19T19:06:23.549 INFO:tasks.workunit.client.0.smithi088.stdout:8/367: write d9/df/d3d/f5f [4487018,44968] 0 2024-02-19T19:06:23.549 INFO:tasks.workunit.client.0.smithi088.stdout:6/478: chown d5/d7/d32/f86 853 1 2024-02-19T19:06:23.549 INFO:tasks.workunit.client.0.smithi088.stdout:4/395: dread d2/d35/f53 [0,4194304] 0 2024-02-19T19:06:23.550 INFO:tasks.workunit.client.0.smithi088.stdout:8/368: chown d9/df/d1c/d2d/l2f 6 1 2024-02-19T19:06:23.550 INFO:tasks.workunit.client.0.smithi088.stdout:4/396: chown d2/d32/l12 9 1 2024-02-19T19:06:23.550 INFO:tasks.workunit.client.0.smithi088.stdout:6/479: read d5/d19/f41 [3423869,90745] 0 2024-02-19T19:06:23.551 INFO:tasks.workunit.client.0.smithi088.stdout:8/369: write d9/df/d1c/d2d/d47/f4d [1289292,57316] 0 2024-02-19T19:06:23.554 INFO:tasks.workunit.client.0.smithi088.stdout:8/370: getdents d9/df/d1c/d2d/d58 0 2024-02-19T19:06:23.556 INFO:tasks.workunit.client.0.smithi088.stdout:6/480: dwrite d5/d19/d26/f44 [0,4194304] 0 2024-02-19T19:06:23.559 INFO:tasks.workunit.client.0.smithi088.stdout:8/371: rename d9/df/d1c/d2d/d47/f43 to d9/df/d3d/d62/f69 0 2024-02-19T19:06:23.560 INFO:tasks.workunit.client.0.smithi088.stdout:6/481: rmdir d5/d45 39 2024-02-19T19:06:23.562 INFO:tasks.workunit.client.0.smithi088.stdout:8/372: mknod d9/df/d3d/d62/c6a 0 2024-02-19T19:06:23.566 INFO:tasks.workunit.client.0.smithi088.stdout:6/482: rename d5/d7/d92 to d5/de/d97 0 2024-02-19T19:06:23.568 INFO:tasks.workunit.client.0.smithi088.stdout:8/373: dwrite d9/df/d1c/f1b [0,4194304] 0 2024-02-19T19:06:23.569 INFO:tasks.workunit.client.0.smithi088.stdout:6/483: dread d5/de/d60/f76 [0,4194304] 0 2024-02-19T19:06:23.572 INFO:tasks.workunit.client.0.smithi088.stdout:6/484: rmdir d5/d7/d32 39 2024-02-19T19:06:23.573 INFO:tasks.workunit.client.0.smithi088.stdout:8/374: mkdir d9/df/d1c/d2d/d47/d6b 0 2024-02-19T19:06:23.574 INFO:tasks.workunit.client.0.smithi088.stdout:6/485: stat d5/d7/d32/d56/f94 0 2024-02-19T19:06:23.575 INFO:tasks.workunit.client.0.smithi088.stdout:8/375: mkdir d9/df/d1c/d2d/d6c 0 2024-02-19T19:06:23.576 INFO:tasks.workunit.client.0.smithi088.stdout:8/376: chown d9/lc 16417348 1 2024-02-19T19:06:23.578 INFO:tasks.workunit.client.0.smithi088.stdout:8/377: creat d9/df/d1c/d2d/f6d x:0 0 0 2024-02-19T19:06:23.584 INFO:tasks.workunit.client.0.smithi088.stdout:8/378: dwrite d9/f2e [0,4194304] 0 2024-02-19T19:06:23.585 INFO:tasks.workunit.client.0.smithi088.stdout:6/486: fsync d5/de/f68 0 2024-02-19T19:06:23.586 INFO:tasks.workunit.client.0.smithi088.stdout:6/487: chown d5/de/d64/d8d/d95 908 1 2024-02-19T19:06:23.590 INFO:tasks.workunit.client.0.smithi088.stdout:6/488: truncate d5/d8/f3f 117543 0 2024-02-19T19:06:23.592 INFO:tasks.workunit.client.0.smithi088.stdout:6/489: symlink d5/d8/d3a/l98 0 2024-02-19T19:06:23.595 INFO:tasks.workunit.client.0.smithi088.stdout:6/490: creat d5/de/d64/d82/d8c/d25/f99 x:0 0 0 2024-02-19T19:06:23.600 INFO:tasks.workunit.client.0.smithi088.stdout:8/379: dwrite d9/df/f27 [0,4194304] 0 2024-02-19T19:06:23.602 INFO:tasks.workunit.client.0.smithi088.stdout:8/380: symlink d9/df/d3d/d62/l6e 0 2024-02-19T19:06:23.612 INFO:tasks.workunit.client.0.smithi088.stdout:8/381: dwrite d9/df/d1c/d2d/d58/d5d/f65 [0,4194304] 0 2024-02-19T19:06:23.617 INFO:tasks.workunit.client.0.smithi088.stdout:8/382: dwrite d9/df/d1c/d2d/d58/d5d/f68 [0,4194304] 0 2024-02-19T19:06:23.619 INFO:tasks.workunit.client.0.smithi088.stdout:8/383: write d9/df/f27 [4965583,109021] 0 2024-02-19T19:06:23.622 INFO:tasks.workunit.client.0.smithi088.stdout:8/384: rmdir d9/df/d3d/d62 39 2024-02-19T19:06:23.623 INFO:tasks.workunit.client.0.smithi088.stdout:8/385: mkdir d9/d6f 0 2024-02-19T19:06:23.625 INFO:tasks.workunit.client.0.smithi088.stdout:8/386: mkdir d9/df/d1c/d2d/d70 0 2024-02-19T19:06:23.627 INFO:tasks.workunit.client.0.smithi088.stdout:8/387: write d9/df/d1c/f1f [856772,66503] 0 2024-02-19T19:06:23.628 INFO:tasks.workunit.client.0.smithi088.stdout:8/388: write d9/df/d1c/d2d/f3e [2038105,101009] 0 2024-02-19T19:06:23.630 INFO:tasks.workunit.client.0.smithi088.stdout:8/389: write d9/df/f36 [4291324,36773] 0 2024-02-19T19:06:23.631 INFO:tasks.workunit.client.0.smithi088.stdout:8/390: dread - d9/df/d1c/d2d/d47/f57 zero size 2024-02-19T19:06:23.632 INFO:tasks.workunit.client.0.smithi088.stdout:8/391: chown d9/df/d1c/d2d/d58 496815 1 2024-02-19T19:06:23.633 INFO:tasks.workunit.client.0.smithi088.stdout:8/392: chown d9/df/d1c/d22/l24 218520483 1 2024-02-19T19:06:23.635 INFO:tasks.workunit.client.0.smithi088.stdout:8/393: mkdir d9/df/d3d/d50/d71 0 2024-02-19T19:06:23.640 INFO:tasks.workunit.client.0.smithi088.stdout:8/394: rename d9/df/d1c/d2d/d58 to d9/df/d1c/d2d/d47/d6b/d72 0 2024-02-19T19:06:23.642 INFO:tasks.workunit.client.0.smithi088.stdout:8/395: write d9/df/d1c/f16 [8601134,125479] 0 2024-02-19T19:06:23.646 INFO:tasks.workunit.client.0.smithi088.stdout:8/396: getdents d9/df/d3d/d50 0 2024-02-19T19:06:23.648 INFO:tasks.workunit.client.0.smithi088.stdout:8/397: creat d9/df/d1c/f73 x:0 0 0 2024-02-19T19:06:23.648 INFO:tasks.workunit.client.0.smithi088.stdout:8/398: stat d9/df/d1c/d22/l3c 0 2024-02-19T19:06:23.659 INFO:tasks.workunit.client.0.smithi088.stdout:8/399: dread d9/df/d1c/f16 [8388608,4194304] 0 2024-02-19T19:06:23.662 INFO:tasks.workunit.client.0.smithi088.stdout:8/400: dread f5 [0,4194304] 0 2024-02-19T19:06:23.667 INFO:tasks.workunit.client.0.smithi088.stdout:8/401: dwrite d9/df/d1c/d22/f66 [0,4194304] 0 2024-02-19T19:06:23.671 INFO:tasks.workunit.client.0.smithi088.stdout:8/402: dread d9/df/d1c/d2d/f3e [4194304,4194304] 0 2024-02-19T19:06:23.677 INFO:tasks.workunit.client.0.smithi088.stdout:8/403: dread d9/df/d1c/d2d/d47/f4d [0,4194304] 0 2024-02-19T19:06:23.681 INFO:tasks.workunit.client.0.smithi088.stdout:8/404: link d9/df/c32 d9/df/d3d/c74 0 2024-02-19T19:06:23.684 INFO:tasks.workunit.client.0.smithi088.stdout:8/405: link d9/df/d1c/f33 d9/f75 0 2024-02-19T19:06:23.685 INFO:tasks.workunit.client.0.smithi088.stdout:8/406: rmdir d9/df 39 2024-02-19T19:06:23.687 INFO:tasks.workunit.client.0.smithi088.stdout:8/407: creat d9/df/d1c/d2d/d47/d6b/f76 x:0 0 0 2024-02-19T19:06:23.689 INFO:tasks.workunit.client.0.smithi088.stdout:8/408: write d9/df/d1c/d2d/f4b [3185902,30593] 0 2024-02-19T19:06:23.689 INFO:tasks.workunit.client.0.smithi088.stdout:8/409: dread - d9/df/f55 zero size 2024-02-19T19:06:23.691 INFO:tasks.workunit.client.0.smithi088.stdout:8/410: mkdir d9/df/d1c/d2d/d6c/d77 0 2024-02-19T19:06:23.697 INFO:tasks.workunit.client.0.smithi088.stdout:8/411: dwrite d9/df/f27 [0,4194304] 0 2024-02-19T19:06:23.697 INFO:tasks.workunit.client.0.smithi088.stdout:7/491: sync 2024-02-19T19:06:23.697 INFO:tasks.workunit.client.0.smithi088.stdout:1/538: sync 2024-02-19T19:06:23.697 INFO:tasks.workunit.client.0.smithi088.stdout:5/522: sync 2024-02-19T19:06:23.697 INFO:tasks.workunit.client.0.smithi088.stdout:3/546: sync 2024-02-19T19:06:23.697 INFO:tasks.workunit.client.0.smithi088.stdout:0/370: sync 2024-02-19T19:06:23.698 INFO:tasks.workunit.client.0.smithi088.stdout:5/523: fsync d4/d1a/f90 0 2024-02-19T19:06:23.702 INFO:tasks.workunit.client.0.smithi088.stdout:1/539: unlink d6/d21/l6f 0 2024-02-19T19:06:23.703 INFO:tasks.workunit.client.0.smithi088.stdout:8/412: creat d9/df/d3d/d50/f78 x:0 0 0 2024-02-19T19:06:23.704 INFO:tasks.workunit.client.0.smithi088.stdout:7/492: creat d14/d1d/d3c/d43/d5c/d75/fad x:0 0 0 2024-02-19T19:06:23.705 INFO:tasks.workunit.client.0.smithi088.stdout:0/371: truncate d13/d46/f69 821433 0 2024-02-19T19:06:23.707 INFO:tasks.workunit.client.0.smithi088.stdout:5/524: creat d4/d61/d65/d9a/d9f/da0/fae x:0 0 0 2024-02-19T19:06:23.707 INFO:tasks.workunit.client.0.smithi088.stdout:3/547: dwrite d4/d1a/d7f/fb2 [0,4194304] 0 2024-02-19T19:06:23.708 INFO:tasks.workunit.client.0.smithi088.stdout:5/525: write d4/d7/f64 [4457031,10580] 0 2024-02-19T19:06:23.709 INFO:tasks.workunit.client.0.smithi088.stdout:1/540: dread - d6/d21/d64/d94/d71/f9e zero size 2024-02-19T19:06:23.710 INFO:tasks.workunit.client.0.smithi088.stdout:3/548: truncate d4/d1a/d1e/d31/d32/f51 1129408 0 2024-02-19T19:06:23.711 INFO:tasks.workunit.client.0.smithi088.stdout:1/541: chown d6/d10/d60/c33 181 1 2024-02-19T19:06:23.714 INFO:tasks.workunit.client.0.smithi088.stdout:7/493: creat d14/d27/d95/fae x:0 0 0 2024-02-19T19:06:23.715 INFO:tasks.workunit.client.0.smithi088.stdout:5/526: creat d4/d61/d65/d9a/d9f/d57/faf x:0 0 0 2024-02-19T19:06:23.720 INFO:tasks.workunit.client.0.smithi088.stdout:0/372: creat d13/d46/d5a/d25/d6d/f7f x:0 0 0 2024-02-19T19:06:23.721 INFO:tasks.workunit.client.0.smithi088.stdout:0/373: chown d13/d46/c67 0 1 2024-02-19T19:06:23.722 INFO:tasks.workunit.client.0.smithi088.stdout:5/527: dread - d4/d1a/d41/f7a zero size 2024-02-19T19:06:23.723 INFO:tasks.workunit.client.0.smithi088.stdout:3/549: mknod d4/d22/d7a/d82/cba 0 2024-02-19T19:06:23.724 INFO:tasks.workunit.client.0.smithi088.stdout:3/550: write d4/d22/d7a/f9f [3556322,16157] 0 2024-02-19T19:06:23.727 INFO:tasks.workunit.client.0.smithi088.stdout:5/528: rename d4/d7/c58 to d4/d61/d65/d9a/d9f/da0/cb0 0 2024-02-19T19:06:23.730 INFO:tasks.workunit.client.0.smithi088.stdout:7/494: getdents d14/d1d/d3c/d43/d5c/d9a 0 2024-02-19T19:06:23.732 INFO:tasks.workunit.client.0.smithi088.stdout:1/542: dread d6/d10/d60/d29/f37 [0,4194304] 0 2024-02-19T19:06:23.736 INFO:tasks.workunit.client.0.smithi088.stdout:0/374: write d13/f2c [552461,86865] 0 2024-02-19T19:06:23.736 INFO:tasks.workunit.client.0.smithi088.stdout:8/413: dread d9/df/f1a [0,4194304] 0 2024-02-19T19:06:23.737 INFO:tasks.workunit.client.0.smithi088.stdout:3/551: dwrite d4/d1a/d23/f60 [0,4194304] 0 2024-02-19T19:06:23.739 INFO:tasks.workunit.client.0.smithi088.stdout:7/495: link d14/d1d/d3c/d43/d5c/f79 d14/d1d/d3c/d43/d5c/d9a/faf 0 2024-02-19T19:06:23.741 INFO:tasks.workunit.client.0.smithi088.stdout:3/552: fdatasync d4/d1a/f24 0 2024-02-19T19:06:23.743 INFO:tasks.workunit.client.0.smithi088.stdout:0/375: creat d13/d46/d5a/f80 x:0 0 0 2024-02-19T19:06:23.747 INFO:tasks.workunit.client.0.smithi088.stdout:3/553: creat d4/d22/d66/da4/fbb x:0 0 0 2024-02-19T19:06:23.747 INFO:tasks.workunit.client.0.smithi088.stdout:1/543: dwrite d6/d10/f19 [0,4194304] 0 2024-02-19T19:06:23.751 INFO:tasks.workunit.client.0.smithi088.stdout:1/544: truncate d6/d21/d64/d94/d71/f35 496735 0 2024-02-19T19:06:23.752 INFO:tasks.workunit.client.0.smithi088.stdout:7/496: rename d14/d1d/f39 to d14/d27/d49/d98/fb0 0 2024-02-19T19:06:23.753 INFO:tasks.workunit.client.0.smithi088.stdout:1/545: dread d6/d21/d64/f81 [0,4194304] 0 2024-02-19T19:06:23.756 INFO:tasks.workunit.client.0.smithi088.stdout:1/546: write d6/d10/f11 [1099063,102501] 0 2024-02-19T19:06:23.757 INFO:tasks.workunit.client.0.smithi088.stdout:3/554: getdents d4/d1a 0 2024-02-19T19:06:23.761 INFO:tasks.workunit.client.0.smithi088.stdout:0/376: dread d13/f23 [0,4194304] 0 2024-02-19T19:06:23.763 INFO:tasks.workunit.client.0.smithi088.stdout:0/377: dread d13/f23 [0,4194304] 0 2024-02-19T19:06:23.764 INFO:tasks.workunit.client.0.smithi088.stdout:0/378: write d13/d46/d7a/f79 [353601,69359] 0 2024-02-19T19:06:23.765 INFO:tasks.workunit.client.0.smithi088.stdout:0/379: mkdir d13/d81 0 2024-02-19T19:06:23.766 INFO:tasks.workunit.client.0.smithi088.stdout:0/380: readlink d13/d46/d5a/l28 0 2024-02-19T19:06:23.767 INFO:tasks.workunit.client.0.smithi088.stdout:8/414: dread d9/df/d1c/d2d/f3e [0,4194304] 0 2024-02-19T19:06:23.771 INFO:tasks.workunit.client.0.smithi088.stdout:3/555: write d4/d12/f1d [1060271,55888] 0 2024-02-19T19:06:23.773 INFO:tasks.workunit.client.0.smithi088.stdout:3/556: write d4/d22/d66/da4/fbb [722766,34645] 0 2024-02-19T19:06:23.776 INFO:tasks.workunit.client.0.smithi088.stdout:3/557: write d4/f79 [1492483,83999] 0 2024-02-19T19:06:23.779 INFO:tasks.workunit.client.0.smithi088.stdout:0/381: dwrite d13/d46/d5a/d25/d2a/d3c/f5d [0,4194304] 0 2024-02-19T19:06:23.779 INFO:tasks.workunit.client.0.smithi088.stdout:7/497: dread d14/d1d/d3c/d43/f5b [0,4194304] 0 2024-02-19T19:06:23.780 INFO:tasks.workunit.client.0.smithi088.stdout:3/558: creat d4/d22/d7a/fbc x:0 0 0 2024-02-19T19:06:23.783 INFO:tasks.workunit.client.0.smithi088.stdout:1/547: dread d6/d21/d6a/d7d/d9a/fa3 [0,4194304] 0 2024-02-19T19:06:23.784 INFO:tasks.workunit.client.0.smithi088.stdout:3/559: read d4/d12/f2e [339399,124995] 0 2024-02-19T19:06:23.785 INFO:tasks.workunit.client.0.smithi088.stdout:0/382: mknod d13/d46/c82 0 2024-02-19T19:06:23.788 INFO:tasks.workunit.client.0.smithi088.stdout:1/548: getdents d6/d21/d6a/d7d/da8 0 2024-02-19T19:06:23.790 INFO:tasks.workunit.client.0.smithi088.stdout:3/560: getdents d4/d1a/d1e/d74/d7d 0 2024-02-19T19:06:23.792 INFO:tasks.workunit.client.0.smithi088.stdout:8/415: dread d9/df/d1c/d2d/f4b [0,4194304] 0 2024-02-19T19:06:23.793 INFO:tasks.workunit.client.0.smithi088.stdout:7/498: dwrite d14/f19 [4194304,4194304] 0 2024-02-19T19:06:23.793 INFO:tasks.workunit.client.0.smithi088.stdout:0/383: link d13/d46/d5a/d25/d2a/d2f/f4c d13/d46/d5b/d76/f83 0 2024-02-19T19:06:23.794 INFO:tasks.workunit.client.0.smithi088.stdout:0/384: readlink lf 0 2024-02-19T19:06:23.794 INFO:tasks.workunit.client.0.smithi088.stdout:1/549: truncate d6/d10/d60/d29/f75 301221 0 2024-02-19T19:06:23.795 INFO:tasks.workunit.client.0.smithi088.stdout:7/499: readlink d14/d27/l23 0 2024-02-19T19:06:23.795 INFO:tasks.workunit.client.0.smithi088.stdout:8/416: readlink d9/df/d1c/d2d/l2f 0 2024-02-19T19:06:23.797 INFO:tasks.workunit.client.0.smithi088.stdout:7/500: rename d14/d27 to d14/d27/d95/db1 22 2024-02-19T19:06:23.798 INFO:tasks.workunit.client.0.smithi088.stdout:1/550: creat d6/d10/d60/d29/fb3 x:0 0 0 2024-02-19T19:06:23.801 INFO:tasks.workunit.client.0.smithi088.stdout:1/551: dread - d6/d10/f4b zero size 2024-02-19T19:06:23.802 INFO:tasks.workunit.client.0.smithi088.stdout:3/561: dwrite d4/d22/d7a/fbc [0,4194304] 0 2024-02-19T19:06:23.805 INFO:tasks.workunit.client.0.smithi088.stdout:0/385: dwrite d13/f1b [0,4194304] 0 2024-02-19T19:06:23.809 INFO:tasks.workunit.client.0.smithi088.stdout:8/417: write d9/df/d1c/f13 [1403294,62966] 0 2024-02-19T19:06:23.812 INFO:tasks.workunit.client.0.smithi088.stdout:0/386: write d13/d46/d5a/d25/f2b [6263849,53578] 0 2024-02-19T19:06:23.812 INFO:tasks.workunit.client.0.smithi088.stdout:8/418: stat d9/df/d3d/d50/f78 0 2024-02-19T19:06:23.815 INFO:tasks.workunit.client.0.smithi088.stdout:0/387: write d13/d46/d7a/f79 [1363284,85203] 0 2024-02-19T19:06:23.817 INFO:tasks.workunit.client.0.smithi088.stdout:8/419: mkdir d9/df/d1c/d2d/d47/d6b/d72/d79 0 2024-02-19T19:06:23.819 INFO:tasks.workunit.client.0.smithi088.stdout:0/388: mknod d13/d46/d5a/d25/c84 0 2024-02-19T19:06:23.821 INFO:tasks.workunit.client.0.smithi088.stdout:1/552: dwrite d6/d10/d60/d29/f2e [4194304,4194304] 0 2024-02-19T19:06:23.822 INFO:tasks.workunit.client.0.smithi088.stdout:8/420: symlink d9/df/l7a 0 2024-02-19T19:06:23.825 INFO:tasks.workunit.client.0.smithi088.stdout:0/389: mknod d13/d62/c85 0 2024-02-19T19:06:23.827 INFO:tasks.workunit.client.0.smithi088.stdout:8/421: unlink d9/df/d1c/d22/c64 0 2024-02-19T19:06:23.828 INFO:tasks.workunit.client.0.smithi088.stdout:9/497: sync 2024-02-19T19:06:23.828 INFO:tasks.workunit.client.0.smithi088.stdout:2/376: sync 2024-02-19T19:06:23.830 INFO:tasks.workunit.client.0.smithi088.stdout:0/390: mkdir d13/d46/d5a/d25/d2a/d3c/d86 0 2024-02-19T19:06:23.830 INFO:tasks.workunit.client.0.smithi088.stdout:9/498: chown d1/d60/l71 64662 1 2024-02-19T19:06:23.832 INFO:tasks.workunit.client.0.smithi088.stdout:3/562: dread d4/d12/f2e [0,4194304] 0 2024-02-19T19:06:23.833 INFO:tasks.workunit.client.0.smithi088.stdout:2/377: creat d3/d1b/d43/d61/d64/f78 x:0 0 0 2024-02-19T19:06:23.834 INFO:tasks.workunit.client.0.smithi088.stdout:1/553: dwrite d6/d21/d6a/d4a/f56 [0,4194304] 0 2024-02-19T19:06:23.836 INFO:tasks.workunit.client.0.smithi088.stdout:7/501: dread d14/d1d/d3c/d43/d5c/d9a/faf [0,4194304] 0 2024-02-19T19:06:23.837 INFO:tasks.workunit.client.0.smithi088.stdout:8/422: creat d9/df/d1c/d2d/d47/d6b/d72/d5d/f7b x:0 0 0 2024-02-19T19:06:23.837 INFO:tasks.workunit.client.0.smithi088.stdout:1/554: write d6/d21/d6a/d4a/fb1 [459562,10590] 0 2024-02-19T19:06:23.891 INFO:tasks.workunit.client.0.smithi088.stdout:9/499: mknod d1/d8/d3e/d93/d6f/c9e 0 2024-02-19T19:06:23.892 INFO:tasks.workunit.client.0.smithi088.stdout:7/502: read d14/d28/d3b/f81 [18048,96657] 0 2024-02-19T19:06:23.893 INFO:tasks.workunit.client.0.smithi088.stdout:9/500: rename d1 to d1/d8/d8d/d97/d9f 22 2024-02-19T19:06:23.894 INFO:tasks.workunit.client.0.smithi088.stdout:2/378: getdents d3/d1f/d4f/d59 0 2024-02-19T19:06:23.895 INFO:tasks.workunit.client.0.smithi088.stdout:1/555: creat d6/d21/d6a/d7d/d7e/fb4 x:0 0 0 2024-02-19T19:06:23.896 INFO:tasks.workunit.client.0.smithi088.stdout:2/379: write d3/d14/f1d [2646452,80417] 0 2024-02-19T19:06:23.897 INFO:tasks.workunit.client.0.smithi088.stdout:3/563: dread d4/d12/f1d [0,4194304] 0 2024-02-19T19:06:23.898 INFO:tasks.workunit.client.0.smithi088.stdout:7/503: mkdir d14/d1d/d3c/d43/db2 0 2024-02-19T19:06:23.899 INFO:tasks.workunit.client.0.smithi088.stdout:9/501: write d1/d8/d3e/d93/d2d/d18/d1f/f2e [789150,1293] 0 2024-02-19T19:06:23.900 INFO:tasks.workunit.client.0.smithi088.stdout:3/564: unlink d4/d22/d81/f98 0 2024-02-19T19:06:23.901 INFO:tasks.workunit.client.0.smithi088.stdout:1/556: rmdir d6/d21/d6a/d4a 39 2024-02-19T19:06:23.902 INFO:tasks.workunit.client.0.smithi088.stdout:7/504: write d14/d28/d3b/f4b [128554,29665] 0 2024-02-19T19:06:23.904 INFO:tasks.workunit.client.0.smithi088.stdout:3/565: rmdir d4/d1a/d1e/d74/d7d 39 2024-02-19T19:06:23.904 INFO:tasks.workunit.client.0.smithi088.stdout:7/505: readlink d14/d27/l55 0 2024-02-19T19:06:23.905 INFO:tasks.workunit.client.0.smithi088.stdout:9/502: symlink d1/d8/d3e/d93/d2d/d23/la0 0 2024-02-19T19:06:23.907 INFO:tasks.workunit.client.0.smithi088.stdout:2/380: dwrite d3/d1b/f5b [4194304,4194304] 0 2024-02-19T19:06:23.910 INFO:tasks.workunit.client.0.smithi088.stdout:3/566: truncate d4/d22/f2b 2401911 0 2024-02-19T19:06:23.913 INFO:tasks.workunit.client.0.smithi088.stdout:7/506: creat d14/d1d/d3c/d43/d5c/d75/d76/fb3 x:0 0 0 2024-02-19T19:06:23.915 INFO:tasks.workunit.client.0.smithi088.stdout:2/381: creat d3/d1f/d4f/d59/f79 x:0 0 0 2024-02-19T19:06:23.917 INFO:tasks.workunit.client.0.smithi088.stdout:1/557: dwrite d6/d21/d6a/d4a/fb1 [0,4194304] 0 2024-02-19T19:06:23.917 INFO:tasks.workunit.client.0.smithi088.stdout:9/503: dwrite d1/d8/d37/f9c [0,4194304] 0 2024-02-19T19:06:23.919 INFO:tasks.workunit.client.0.smithi088.stdout:7/507: fdatasync d14/f30 0 2024-02-19T19:06:23.924 INFO:tasks.workunit.client.0.smithi088.stdout:7/508: dread - d14/d28/d65/fa8 zero size 2024-02-19T19:06:23.929 INFO:tasks.workunit.client.0.smithi088.stdout:1/558: creat d6/d21/d6a/d7d/da8/fb5 x:0 0 0 2024-02-19T19:06:23.933 INFO:tasks.workunit.client.0.smithi088.stdout:7/509: rename d14/d1d/d3c/d43/la1 to d14/d28/d94/lb4 0 2024-02-19T19:06:23.933 INFO:tasks.workunit.client.0.smithi088.stdout:3/567: link d4/d1a/d1e/d74/f96 d4/d12/d33/d43/fbd 0 2024-02-19T19:06:23.935 INFO:tasks.workunit.client.0.smithi088.stdout:1/559: symlink d6/d21/d6a/d7d/lb6 0 2024-02-19T19:06:23.935 INFO:tasks.workunit.client.0.smithi088.stdout:2/382: dwrite d3/fb [0,4194304] 0 2024-02-19T19:06:23.936 INFO:tasks.workunit.client.0.smithi088.stdout:2/383: chown d3/d1b 2791710 1 2024-02-19T19:06:23.937 INFO:tasks.workunit.client.0.smithi088.stdout:9/504: creat d1/d8/d3e/d93/fa1 x:0 0 0 2024-02-19T19:06:23.937 INFO:tasks.workunit.client.0.smithi088.stdout:2/384: read d3/d14/f1d [738062,126663] 0 2024-02-19T19:06:23.939 INFO:tasks.workunit.client.0.smithi088.stdout:9/505: truncate d1/d8/d37/f9a 846729 0 2024-02-19T19:06:23.940 INFO:tasks.workunit.client.0.smithi088.stdout:3/568: creat d4/d1a/d1e/d74/d7c/fbe x:0 0 0 2024-02-19T19:06:23.945 INFO:tasks.workunit.client.0.smithi088.stdout:3/569: mkdir d4/d87/dbf 0 2024-02-19T19:06:23.946 INFO:tasks.workunit.client.0.smithi088.stdout:9/506: mknod d1/d8/d3e/d93/d2f/ca2 0 2024-02-19T19:06:23.948 INFO:tasks.workunit.client.0.smithi088.stdout:3/570: creat d4/d22/d81/fc0 x:0 0 0 2024-02-19T19:06:23.950 INFO:tasks.workunit.client.0.smithi088.stdout:2/385: getdents d3/d1b/d43/d61 0 2024-02-19T19:06:23.952 INFO:tasks.workunit.client.0.smithi088.stdout:1/560: write d6/d10/f5d [1007909,126762] 0 2024-02-19T19:06:23.953 INFO:tasks.workunit.client.0.smithi088.stdout:6/491: sync 2024-02-19T19:06:23.953 INFO:tasks.workunit.client.0.smithi088.stdout:5/529: sync 2024-02-19T19:06:23.953 INFO:tasks.workunit.client.0.smithi088.stdout:4/397: sync 2024-02-19T19:06:23.953 INFO:tasks.workunit.client.0.smithi088.stdout:9/507: rename d1/d8/d8d/d97/c1b to d1/d8/d3e/d93/d2d/ca3 0 2024-02-19T19:06:23.953 INFO:tasks.workunit.client.0.smithi088.stdout:1/561: read - d6/d21/d6a/d7d/da8/fb5 zero size 2024-02-19T19:06:23.956 INFO:tasks.workunit.client.0.smithi088.stdout:1/562: chown d6/c51 3315333 1 2024-02-19T19:06:23.956 INFO:tasks.workunit.client.0.smithi088.stdout:5/530: mknod d4/d61/d65/d9a/cb1 0 2024-02-19T19:06:23.958 INFO:tasks.workunit.client.0.smithi088.stdout:9/508: truncate d1/d8/d3e/d93/d2d/f4c 1225062 0 2024-02-19T19:06:23.960 INFO:tasks.workunit.client.0.smithi088.stdout:1/563: fdatasync d6/d10/d60/d29/f37 0 2024-02-19T19:06:23.961 INFO:tasks.workunit.client.0.smithi088.stdout:5/531: symlink d4/d61/d65/d9a/d9f/lb2 0 2024-02-19T19:06:23.964 INFO:tasks.workunit.client.0.smithi088.stdout:2/386: dwrite d3/d1f/d4f/f11 [4194304,4194304] 0 2024-02-19T19:06:23.968 INFO:tasks.workunit.client.0.smithi088.stdout:7/510: dwrite d14/f1f [0,4194304] 0 2024-02-19T19:06:23.969 INFO:tasks.workunit.client.0.smithi088.stdout:5/532: mknod d4/d61/d65/d9a/d9f/d57/cb3 0 2024-02-19T19:06:23.971 INFO:tasks.workunit.client.0.smithi088.stdout:1/564: creat d6/d21/d6a/fb7 x:0 0 0 2024-02-19T19:06:23.971 INFO:tasks.workunit.client.0.smithi088.stdout:9/509: dwrite d1/d8/d3e/d93/fa1 [0,4194304] 0 2024-02-19T19:06:23.974 INFO:tasks.workunit.client.0.smithi088.stdout:2/387: rename d3/d14/c1a to d3/d14/c7a 0 2024-02-19T19:06:23.976 INFO:tasks.workunit.client.0.smithi088.stdout:2/388: write d3/d1f/d5d/f72 [349828,110459] 0 2024-02-19T19:06:23.978 INFO:tasks.workunit.client.0.smithi088.stdout:7/511: creat d14/d1d/d3c/d43/d5c/fb5 x:0 0 0 2024-02-19T19:06:23.978 INFO:tasks.workunit.client.0.smithi088.stdout:7/512: fsync d14/f51 0 2024-02-19T19:06:23.980 INFO:tasks.workunit.client.0.smithi088.stdout:9/510: creat d1/d8/d3e/d93/d2f/fa4 x:0 0 0 2024-02-19T19:06:23.981 INFO:tasks.workunit.client.0.smithi088.stdout:1/565: rename d6/d21/d64/d94/da0 to d6/d10/d60/db8 0 2024-02-19T19:06:23.984 INFO:tasks.workunit.client.0.smithi088.stdout:9/511: creat d1/d8/d8d/fa5 x:0 0 0 2024-02-19T19:06:23.987 INFO:tasks.workunit.client.0.smithi088.stdout:2/389: dwrite d3/d1b/d43/d61/d64/f6d [0,4194304] 0 2024-02-19T19:06:23.987 INFO:tasks.workunit.client.0.smithi088.stdout:1/566: rmdir d6/d21/d64/d94/d6e 39 2024-02-19T19:06:23.989 INFO:tasks.workunit.client.0.smithi088.stdout:6/492: dread d5/de/d64/d82/d8c/d25/f39 [0,4194304] 0 2024-02-19T19:06:23.990 INFO:tasks.workunit.client.0.smithi088.stdout:9/512: mkdir d1/d8/d3e/d93/d2d/d23/d79/da6 0 2024-02-19T19:06:23.993 INFO:tasks.workunit.client.0.smithi088.stdout:5/533: getdents d4/d1a 0 2024-02-19T19:06:23.995 INFO:tasks.workunit.client.0.smithi088.stdout:9/513: write d1/d8/d37/d54/f55 [398386,76467] 0 2024-02-19T19:06:23.996 INFO:tasks.workunit.client.0.smithi088.stdout:7/513: rename d14/d27/d49/c5f to d14/d1d/d3c/d43/cb6 0 2024-02-19T19:06:23.999 INFO:tasks.workunit.client.0.smithi088.stdout:1/567: link d6/d21/d6a/d4a/f56 d6/d10/d60/d29/fb9 0 2024-02-19T19:06:24.000 INFO:tasks.workunit.client.0.smithi088.stdout:9/514: mknod d1/d8/ca7 0 2024-02-19T19:06:24.001 INFO:tasks.workunit.client.0.smithi088.stdout:2/390: dwrite d3/d1b/d43/f66 [0,4194304] 0 2024-02-19T19:06:24.004 INFO:tasks.workunit.client.0.smithi088.stdout:6/493: dwrite d5/d8/d2c/f4c [0,4194304] 0 2024-02-19T19:06:24.004 INFO:tasks.workunit.client.0.smithi088.stdout:2/391: chown d3/d1f/f73 43 1 2024-02-19T19:06:24.005 INFO:tasks.workunit.client.0.smithi088.stdout:9/515: readlink d1/d8/d3e/d93/d2d/d18/d1f/l89 0 2024-02-19T19:06:24.006 INFO:tasks.workunit.client.0.smithi088.stdout:7/514: truncate d14/d1d/d3c/d43/d5c/d9a/d92/f97 394399 0 2024-02-19T19:06:24.007 INFO:tasks.workunit.client.0.smithi088.stdout:2/392: write d3/d1b/d43/d61/d64/f6d [1782027,88406] 0 2024-02-19T19:06:24.008 INFO:tasks.workunit.client.0.smithi088.stdout:9/516: write d1/d8/d8d/fa5 [819327,28494] 0 2024-02-19T19:06:24.009 INFO:tasks.workunit.client.0.smithi088.stdout:1/568: getdents d6/d21/d6a/d7d/da8 0 2024-02-19T19:06:24.010 INFO:tasks.workunit.client.0.smithi088.stdout:7/515: write d14/d1d/d3c/f63 [585088,6272] 0 2024-02-19T19:06:24.011 INFO:tasks.workunit.client.0.smithi088.stdout:2/393: write d3/d1b/f2e [2452039,20256] 0 2024-02-19T19:06:24.013 INFO:tasks.workunit.client.0.smithi088.stdout:9/517: fdatasync d1/f20 0 2024-02-19T19:06:24.014 INFO:tasks.workunit.client.0.smithi088.stdout:6/494: rmdir d5/d19 39 2024-02-19T19:06:24.015 INFO:tasks.workunit.client.0.smithi088.stdout:1/569: rename d6/d21/d64/d94 to d6/d21/d64/d94/dba 22 2024-02-19T19:06:24.016 INFO:tasks.workunit.client.0.smithi088.stdout:6/495: fdatasync d5/de/d64/d82/d8c/f59 0 2024-02-19T19:06:24.017 INFO:tasks.workunit.client.0.smithi088.stdout:9/518: truncate d1/d60/f85 5105768 0 2024-02-19T19:06:24.018 INFO:tasks.workunit.client.0.smithi088.stdout:7/516: creat d14/d1d/d3c/fb7 x:0 0 0 2024-02-19T19:06:24.019 INFO:tasks.workunit.client.0.smithi088.stdout:1/570: creat d6/d10/d60/d29/d63/fbb x:0 0 0 2024-02-19T19:06:24.021 INFO:tasks.workunit.client.0.smithi088.stdout:1/571: truncate d6/d21/d6a/d4a/fa7 1042312 0 2024-02-19T19:06:24.024 INFO:tasks.workunit.client.0.smithi088.stdout:7/517: rename d14/d28/l53 to d14/d1d/d3c/d43/d5c/d75/lb8 0 2024-02-19T19:06:24.025 INFO:tasks.workunit.client.0.smithi088.stdout:2/394: dread d3/d1f/d4f/d59/f5f [0,4194304] 0 2024-02-19T19:06:24.027 INFO:tasks.workunit.client.0.smithi088.stdout:9/519: mknod d1/d8/d3e/d93/d2d/d18/ca8 0 2024-02-19T19:06:24.029 INFO:tasks.workunit.client.0.smithi088.stdout:1/572: dwrite d6/d10/d60/d29/f30 [0,4194304] 0 2024-02-19T19:06:24.031 INFO:tasks.workunit.client.0.smithi088.stdout:9/520: rmdir d1/d60 39 2024-02-19T19:06:24.032 INFO:tasks.workunit.client.0.smithi088.stdout:7/518: mknod d14/d28/d9c/cb9 0 2024-02-19T19:06:24.035 INFO:tasks.workunit.client.0.smithi088.stdout:1/573: write d6/d21/d6a/d7d/d9a/fa3 [3176249,92057] 0 2024-02-19T19:06:24.036 INFO:tasks.workunit.client.0.smithi088.stdout:1/574: stat d6/d10/d60/c33 0 2024-02-19T19:06:24.038 INFO:tasks.workunit.client.0.smithi088.stdout:7/519: mkdir d14/d27/d6b/dba 0 2024-02-19T19:06:24.040 INFO:tasks.workunit.client.0.smithi088.stdout:1/575: fsync d6/d21/d6a/d4a/fb1 0 2024-02-19T19:06:24.041 INFO:tasks.workunit.client.0.smithi088.stdout:5/534: dread d4/f11 [0,4194304] 0 2024-02-19T19:06:24.045 INFO:tasks.workunit.client.0.smithi088.stdout:1/576: creat d6/d21/d6a/d4a/fbc x:0 0 0 2024-02-19T19:06:24.047 INFO:tasks.workunit.client.0.smithi088.stdout:5/535: creat d4/d61/d65/d9a/d9f/da0/fb4 x:0 0 0 2024-02-19T19:06:24.047 INFO:tasks.workunit.client.0.smithi088.stdout:6/496: dread d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:24.049 INFO:tasks.workunit.client.0.smithi088.stdout:5/536: chown d4/d61/d65/d9a/d9f/la4 404 1 2024-02-19T19:06:24.052 INFO:tasks.workunit.client.0.smithi088.stdout:1/577: rename d6/f1c to d6/d21/d64/fbd 0 2024-02-19T19:06:24.053 INFO:tasks.workunit.client.0.smithi088.stdout:7/520: rmdir d14/d1d/d3c 39 2024-02-19T19:06:24.056 INFO:tasks.workunit.client.0.smithi088.stdout:6/497: symlink d5/d45/l9a 0 2024-02-19T19:06:24.056 INFO:tasks.workunit.client.0.smithi088.stdout:5/537: dwrite d4/d75/f98 [0,4194304] 0 2024-02-19T19:06:24.059 INFO:tasks.workunit.client.0.smithi088.stdout:7/521: stat d14/d1d/d3c/d43/d5c 0 2024-02-19T19:06:24.060 INFO:tasks.workunit.client.0.smithi088.stdout:5/538: truncate d4/d61/d65/d9a/d9f/f4e 1108759 0 2024-02-19T19:06:24.062 INFO:tasks.workunit.client.0.smithi088.stdout:5/539: write d4/d61/d65/d9a/d9f/da0/fb4 [431068,107415] 0 2024-02-19T19:06:24.062 INFO:tasks.workunit.client.0.smithi088.stdout:1/578: truncate d6/d10/d60/d29/fb9 4241153 0 2024-02-19T19:06:24.063 INFO:tasks.workunit.client.0.smithi088.stdout:7/522: truncate d14/f6e 1033783 0 2024-02-19T19:06:24.066 INFO:tasks.workunit.client.0.smithi088.stdout:9/521: dread d1/d60/f6c [0,4194304] 0 2024-02-19T19:06:24.068 INFO:tasks.workunit.client.0.smithi088.stdout:9/522: stat d1/d8/d3e/d93/d2d/f47 0 2024-02-19T19:06:24.069 INFO:tasks.workunit.client.0.smithi088.stdout:6/498: dwrite d5/d19/f6d [0,4194304] 0 2024-02-19T19:06:24.069 INFO:tasks.workunit.client.0.smithi088.stdout:7/523: mknod d14/d1d/d3c/cbb 0 2024-02-19T19:06:24.074 INFO:tasks.workunit.client.0.smithi088.stdout:1/579: dwrite d6/d21/d64/d94/d71/f35 [0,4194304] 0 2024-02-19T19:06:24.075 INFO:tasks.workunit.client.0.smithi088.stdout:9/523: mknod d1/d8/d37/d54/ca9 0 2024-02-19T19:06:24.076 INFO:tasks.workunit.client.0.smithi088.stdout:1/580: write d6/d21/d64/d94/d71/f35 [2338797,52130] 0 2024-02-19T19:06:24.079 INFO:tasks.workunit.client.0.smithi088.stdout:5/540: rename d4/d1a/f37 to d4/fb5 0 2024-02-19T19:06:24.087 INFO:tasks.workunit.client.0.smithi088.stdout:9/524: rmdir d1/d8/d3e/d93/d6f 39 2024-02-19T19:06:24.087 INFO:tasks.workunit.client.0.smithi088.stdout:5/541: dwrite d4/fe [0,4194304] 0 2024-02-19T19:06:24.088 INFO:tasks.workunit.client.0.smithi088.stdout:1/581: mkdir d6/d21/d64/d94/d71/dbe 0 2024-02-19T19:06:24.092 INFO:tasks.workunit.client.0.smithi088.stdout:5/542: fdatasync d4/d61/d65/d9a/d9f/f26 0 2024-02-19T19:06:24.094 INFO:tasks.workunit.client.0.smithi088.stdout:5/543: chown d4/d61/d65/d9a/d9f/c59 74 1 2024-02-19T19:06:24.096 INFO:tasks.workunit.client.0.smithi088.stdout:1/582: link d6/d10/d60/d29/fb3 d6/d21/d6a/d7d/d9a/fbf 0 2024-02-19T19:06:24.098 INFO:tasks.workunit.client.0.smithi088.stdout:7/524: dwrite d14/f30 [0,4194304] 0 2024-02-19T19:06:24.099 INFO:tasks.workunit.client.0.smithi088.stdout:7/525: dread - d14/d1d/d3c/d43/d5c/d75/fa4 zero size 2024-02-19T19:06:24.104 INFO:tasks.workunit.client.0.smithi088.stdout:7/526: fsync d14/f22 0 2024-02-19T19:06:24.106 INFO:tasks.workunit.client.0.smithi088.stdout:6/499: dwrite d5/de/d64/d82/d8c/d25/f49 [0,4194304] 0 2024-02-19T19:06:24.106 INFO:tasks.workunit.client.0.smithi088.stdout:6/500: readlink d5/d19/l5b 0 2024-02-19T19:06:24.108 INFO:tasks.workunit.client.0.smithi088.stdout:1/583: creat d6/d21/d6a/fc0 x:0 0 0 2024-02-19T19:06:24.110 INFO:tasks.workunit.client.0.smithi088.stdout:5/544: rmdir d4/d1a/d41 39 2024-02-19T19:06:24.178 INFO:tasks.workunit.client.0.smithi088.stdout:5/545: creat d4/d61/fb6 x:0 0 0 2024-02-19T19:06:24.179 INFO:tasks.workunit.client.0.smithi088.stdout:7/527: unlink d14/d28/l72 0 2024-02-19T19:06:24.179 INFO:tasks.workunit.client.0.smithi088.stdout:1/584: dread d6/d10/d60/d29/f30 [0,4194304] 0 2024-02-19T19:06:24.181 INFO:tasks.workunit.client.0.smithi088.stdout:1/585: fdatasync d6/d21/d6a/f6d 0 2024-02-19T19:06:24.182 INFO:tasks.workunit.client.0.smithi088.stdout:7/528: creat d14/d28/d94/fbc x:0 0 0 2024-02-19T19:06:24.183 INFO:tasks.workunit.client.0.smithi088.stdout:5/546: dread d4/fe [0,4194304] 0 2024-02-19T19:06:24.183 INFO:tasks.workunit.client.0.smithi088.stdout:5/547: chown d4 1671615791 1 2024-02-19T19:06:24.183 INFO:tasks.workunit.client.0.smithi088.stdout:1/586: symlink d6/d21/d6a/d7d/d7e/lc1 0 2024-02-19T19:06:24.184 INFO:tasks.workunit.client.0.smithi088.stdout:5/548: write d4/d7/f9b [935129,56995] 0 2024-02-19T19:06:24.185 INFO:tasks.workunit.client.0.smithi088.stdout:1/587: read - d6/d21/d6a/d4a/faf zero size 2024-02-19T19:06:24.187 INFO:tasks.workunit.client.0.smithi088.stdout:5/549: symlink d4/d61/lb7 0 2024-02-19T19:06:24.187 INFO:tasks.workunit.client.0.smithi088.stdout:5/550: write d4/d7/f8f [421463,54760] 0 2024-02-19T19:06:24.188 INFO:tasks.workunit.client.0.smithi088.stdout:1/588: fdatasync d6/d10/d60/d29/f37 0 2024-02-19T19:06:24.191 INFO:tasks.workunit.client.0.smithi088.stdout:1/589: dread d6/d10/d60/d29/f37 [0,4194304] 0 2024-02-19T19:06:24.192 INFO:tasks.workunit.client.0.smithi088.stdout:7/529: truncate d14/d1d/d3c/f3d 1551336 0 2024-02-19T19:06:24.193 INFO:tasks.workunit.client.0.smithi088.stdout:7/530: chown d14/f2e 1148373407 1 2024-02-19T19:06:24.197 INFO:tasks.workunit.client.0.smithi088.stdout:7/531: rename d14/d28/d65/c96 to d14/cbd 0 2024-02-19T19:06:24.199 INFO:tasks.workunit.client.0.smithi088.stdout:7/532: mkdir d14/d27/d49/d98/dbe 0 2024-02-19T19:06:24.200 INFO:tasks.workunit.client.0.smithi088.stdout:5/551: dread d4/f42 [0,4194304] 0 2024-02-19T19:06:24.205 INFO:tasks.workunit.client.0.smithi088.stdout:7/533: rmdir d14/d1d 39 2024-02-19T19:06:24.209 INFO:tasks.workunit.client.0.smithi088.stdout:7/534: mkdir d14/d1d/dbf 0 2024-02-19T19:06:24.210 INFO:tasks.workunit.client.0.smithi088.stdout:7/535: write d14/d27/d6b/f93 [236157,82839] 0 2024-02-19T19:06:24.214 INFO:tasks.workunit.client.0.smithi088.stdout:5/552: dread d4/d61/d65/d9a/d9f/d57/f6a [0,4194304] 0 2024-02-19T19:06:24.216 INFO:tasks.workunit.client.0.smithi088.stdout:5/553: chown d4/d1a/f77 59181635 1 2024-02-19T19:06:24.218 INFO:tasks.workunit.client.0.smithi088.stdout:5/554: write d4/d61/d65/d9a/d9f/da0/fa9 [1514108,42675] 0 2024-02-19T19:06:24.219 INFO:tasks.workunit.client.0.smithi088.stdout:5/555: truncate d4/d7/f63 5537135 0 2024-02-19T19:06:24.223 INFO:tasks.workunit.client.0.smithi088.stdout:5/556: unlink d4/fb5 0 2024-02-19T19:06:24.225 INFO:tasks.workunit.client.0.smithi088.stdout:5/557: mknod d4/d61/d65/d9a/d9f/d57/d84/cb8 0 2024-02-19T19:06:24.226 INFO:tasks.workunit.client.0.smithi088.stdout:5/558: read d4/f93 [3118896,73528] 0 2024-02-19T19:06:24.233 INFO:tasks.workunit.client.0.smithi088.stdout:5/559: truncate d4/d75/f80 134846 0 2024-02-19T19:06:24.235 INFO:tasks.workunit.client.0.smithi088.stdout:5/560: symlink d4/d1a/d41/d5e/lb9 0 2024-02-19T19:06:24.237 INFO:tasks.workunit.client.0.smithi088.stdout:5/561: symlink d4/lba 0 2024-02-19T19:06:24.241 INFO:tasks.workunit.client.0.smithi088.stdout:5/562: rename d4/d61/d65/d9a/cb1 to d4/d61/d65/d9a/d9f/d57/d84/cbb 0 2024-02-19T19:06:24.241 INFO:tasks.workunit.client.0.smithi088.stdout:5/563: read - d4/d61/d65/d9a/d9f/d57/faf zero size 2024-02-19T19:06:24.242 INFO:tasks.workunit.client.0.smithi088.stdout:5/564: chown d4/d7/c8e 32 1 2024-02-19T19:06:24.247 INFO:tasks.workunit.client.0.smithi088.stdout:5/565: dwrite d4/f93 [0,4194304] 0 2024-02-19T19:06:24.250 INFO:tasks.workunit.client.0.smithi088.stdout:5/566: rename d4/d61/lb7 to d4/d1a/d41/d5e/lbc 0 2024-02-19T19:06:24.253 INFO:tasks.workunit.client.0.smithi088.stdout:5/567: link d4/d7/c17 d4/d61/d65/cbd 0 2024-02-19T19:06:24.254 INFO:tasks.workunit.client.0.smithi088.stdout:5/568: creat d4/d7/fbe x:0 0 0 2024-02-19T19:06:24.256 INFO:tasks.workunit.client.0.smithi088.stdout:5/569: creat d4/d75/fbf x:0 0 0 2024-02-19T19:06:24.260 INFO:tasks.workunit.client.0.smithi088.stdout:5/570: link d4/d1a/f1b d4/d61/d65/d9a/d9f/da0/fc0 0 2024-02-19T19:06:24.265 INFO:tasks.workunit.client.0.smithi088.stdout:5/571: dwrite d4/d7/fbe [0,4194304] 0 2024-02-19T19:06:24.267 INFO:tasks.workunit.client.0.smithi088.stdout:5/572: mkdir d4/d1a/d41/dc1 0 2024-02-19T19:06:24.272 INFO:tasks.workunit.client.0.smithi088.stdout:5/573: read d4/d61/d65/d9a/d9f/da0/fc0 [775058,109865] 0 2024-02-19T19:06:24.275 INFO:tasks.workunit.client.0.smithi088.stdout:5/574: mkdir d4/d1a/d41/dc1/dc2 0 2024-02-19T19:06:24.278 INFO:tasks.workunit.client.0.smithi088.stdout:5/575: creat d4/d1a/fc3 x:0 0 0 2024-02-19T19:06:24.281 INFO:tasks.workunit.client.0.smithi088.stdout:5/576: truncate d4/d61/d65/d9a/d9f/f26 4287688 0 2024-02-19T19:06:24.282 INFO:tasks.workunit.client.0.smithi088.stdout:5/577: readlink d4/d1a/d41/l5c 0 2024-02-19T19:06:24.292 INFO:tasks.workunit.client.0.smithi088.stdout:5/578: dwrite d4/d1a/f1b [0,4194304] 0 2024-02-19T19:06:24.298 INFO:tasks.workunit.client.0.smithi088.stdout:5/579: dwrite d4/d7/f15 [4194304,4194304] 0 2024-02-19T19:06:24.301 INFO:tasks.workunit.client.0.smithi088.stdout:5/580: creat d4/d7/fc4 x:0 0 0 2024-02-19T19:06:24.303 INFO:tasks.workunit.client.0.smithi088.stdout:5/581: symlink d4/d61/d65/d9a/lc5 0 2024-02-19T19:06:24.305 INFO:tasks.workunit.client.0.smithi088.stdout:5/582: creat d4/d1a/d41/dc1/dc2/fc6 x:0 0 0 2024-02-19T19:06:24.306 INFO:tasks.workunit.client.0.smithi088.stdout:5/583: creat d4/d1a/d41/dc1/fc7 x:0 0 0 2024-02-19T19:06:24.308 INFO:tasks.workunit.client.0.smithi088.stdout:4/398: sync 2024-02-19T19:06:24.309 INFO:tasks.workunit.client.0.smithi088.stdout:8/423: sync 2024-02-19T19:06:24.309 INFO:tasks.workunit.client.0.smithi088.stdout:3/571: sync 2024-02-19T19:06:24.309 INFO:tasks.workunit.client.0.smithi088.stdout:0/391: sync 2024-02-19T19:06:24.309 INFO:tasks.workunit.client.0.smithi088.stdout:5/584: symlink d4/d61/d65/d9a/d9f/d57/d84/lc8 0 2024-02-19T19:06:24.311 INFO:tasks.workunit.client.0.smithi088.stdout:8/424: dread - d9/df/d1c/d2d/f6d zero size 2024-02-19T19:06:24.313 INFO:tasks.workunit.client.0.smithi088.stdout:0/392: mkdir d13/d46/d7a/d87 0 2024-02-19T19:06:24.315 INFO:tasks.workunit.client.0.smithi088.stdout:5/585: rename d4/d1a/l96 to d4/d7/lc9 0 2024-02-19T19:06:24.315 INFO:tasks.workunit.client.0.smithi088.stdout:5/586: dread - d4/d1a/fc3 zero size 2024-02-19T19:06:24.316 INFO:tasks.workunit.client.0.smithi088.stdout:3/572: unlink d4/d1a/d1e/d74/d7c/d97/l9b 0 2024-02-19T19:06:24.317 INFO:tasks.workunit.client.0.smithi088.stdout:0/393: mknod d13/d46/d5b/d76/c88 0 2024-02-19T19:06:24.318 INFO:tasks.workunit.client.0.smithi088.stdout:3/573: fdatasync d4/d1a/d1e/d31/d32/f51 0 2024-02-19T19:06:24.320 INFO:tasks.workunit.client.0.smithi088.stdout:8/425: link d9/df/d3d/l4e d9/df/d1c/d2d/d6c/d77/l7c 0 2024-02-19T19:06:24.321 INFO:tasks.workunit.client.0.smithi088.stdout:0/394: unlink d13/d46/d5a/c54 0 2024-02-19T19:06:24.322 INFO:tasks.workunit.client.0.smithi088.stdout:3/574: creat d4/d22/d7a/fc1 x:0 0 0 2024-02-19T19:06:24.323 INFO:tasks.workunit.client.0.smithi088.stdout:5/587: dwrite d4/d61/fa8 [0,4194304] 0 2024-02-19T19:06:24.325 INFO:tasks.workunit.client.0.smithi088.stdout:8/426: mknod d9/df/d1c/d2d/d6c/d77/c7d 0 2024-02-19T19:06:24.329 INFO:tasks.workunit.client.0.smithi088.stdout:0/395: dwrite d13/f3e [0,4194304] 0 2024-02-19T19:06:24.329 INFO:tasks.workunit.client.0.smithi088.stdout:5/588: unlink d4/d7/f2c 0 2024-02-19T19:06:24.330 INFO:tasks.workunit.client.0.smithi088.stdout:3/575: creat d4/d87/dbf/fc2 x:0 0 0 2024-02-19T19:06:24.330 INFO:tasks.workunit.client.0.smithi088.stdout:0/396: read d13/f3e [1329619,40689] 0 2024-02-19T19:06:24.335 INFO:tasks.workunit.client.0.smithi088.stdout:5/589: mknod d4/d75/cca 0 2024-02-19T19:06:24.336 INFO:tasks.workunit.client.0.smithi088.stdout:5/590: readlink d4/d1a/d41/d5e/la2 0 2024-02-19T19:06:24.338 INFO:tasks.workunit.client.0.smithi088.stdout:0/397: symlink d13/d46/d7a/l89 0 2024-02-19T19:06:24.339 INFO:tasks.workunit.client.0.smithi088.stdout:8/427: mknod d9/d6f/c7e 0 2024-02-19T19:06:24.340 INFO:tasks.workunit.client.0.smithi088.stdout:0/398: truncate d13/d46/d5a/d25/d2a/d2f/f41 329498 0 2024-02-19T19:06:24.342 INFO:tasks.workunit.client.0.smithi088.stdout:0/399: write d13/d46/d5a/d25/d2a/d34/f43 [570667,346] 0 2024-02-19T19:06:24.344 INFO:tasks.workunit.client.0.smithi088.stdout:3/576: fsync d4/d1a/d23/f5d 0 2024-02-19T19:06:24.345 INFO:tasks.workunit.client.0.smithi088.stdout:8/428: write d9/df/d1c/f16 [10501586,16026] 0 2024-02-19T19:06:24.350 INFO:tasks.workunit.client.0.smithi088.stdout:8/429: mkdir d9/df/d1c/d2d/d47/d6b/d72/d7f 0 2024-02-19T19:06:24.353 INFO:tasks.workunit.client.0.smithi088.stdout:0/400: rename d13/d46/d7a to d13/d62/d8a 0 2024-02-19T19:06:24.353 INFO:tasks.workunit.client.0.smithi088.stdout:3/577: mknod d4/d22/cc3 0 2024-02-19T19:06:24.356 INFO:tasks.workunit.client.0.smithi088.stdout:5/591: rmdir d4/d1a/d9c 0 2024-02-19T19:06:24.357 INFO:tasks.workunit.client.0.smithi088.stdout:0/401: unlink f5 0 2024-02-19T19:06:24.358 INFO:tasks.workunit.client.0.smithi088.stdout:3/578: creat d4/d1a/d1e/d31/d32/d55/fc4 x:0 0 0 2024-02-19T19:06:24.361 INFO:tasks.workunit.client.0.smithi088.stdout:3/579: mkdir d4/d1a/d1e/d31/dc5 0 2024-02-19T19:06:24.362 INFO:tasks.workunit.client.0.smithi088.stdout:5/592: dread d4/d1a/f54 [0,4194304] 0 2024-02-19T19:06:24.362 INFO:tasks.workunit.client.0.smithi088.stdout:5/593: chown d4/l99 1 1 2024-02-19T19:06:24.364 INFO:tasks.workunit.client.0.smithi088.stdout:3/580: creat d4/d12/d33/d43/d85/fc6 x:0 0 0 2024-02-19T19:06:24.365 INFO:tasks.workunit.client.0.smithi088.stdout:3/581: chown d4/f28 498 1 2024-02-19T19:06:24.366 INFO:tasks.workunit.client.0.smithi088.stdout:3/582: chown d4/d1a/d1e/l6d 1820919 1 2024-02-19T19:06:24.367 INFO:tasks.workunit.client.0.smithi088.stdout:0/402: link d13/d46/d5a/d25/d2a/d34/f4d d13/d46/d5a/d25/f8b 0 2024-02-19T19:06:24.370 INFO:tasks.workunit.client.0.smithi088.stdout:8/430: truncate d9/df/f36 2037515 0 2024-02-19T19:06:24.372 INFO:tasks.workunit.client.0.smithi088.stdout:3/583: symlink d4/d1a/d1e/d74/d7d/daa/lc7 0 2024-02-19T19:06:24.374 INFO:tasks.workunit.client.0.smithi088.stdout:0/403: creat d13/f8c x:0 0 0 2024-02-19T19:06:24.375 INFO:tasks.workunit.client.0.smithi088.stdout:3/584: mknod d4/d87/dbf/cc8 0 2024-02-19T19:06:24.376 INFO:tasks.workunit.client.0.smithi088.stdout:5/594: dread d4/d7/f63 [0,4194304] 0 2024-02-19T19:06:24.378 INFO:tasks.workunit.client.0.smithi088.stdout:5/595: write d4/d61/f73 [1700694,107146] 0 2024-02-19T19:06:24.378 INFO:tasks.workunit.client.0.smithi088.stdout:3/585: read d4/d1a/d7f/fb2 [131609,107006] 0 2024-02-19T19:06:24.382 INFO:tasks.workunit.client.0.smithi088.stdout:3/586: truncate d4/d87/dbf/fc2 672024 0 2024-02-19T19:06:24.382 INFO:tasks.workunit.client.0.smithi088.stdout:5/596: mkdir d4/d1a/d41/dcb 0 2024-02-19T19:06:24.386 INFO:tasks.workunit.client.0.smithi088.stdout:0/404: dwrite d13/d46/d5a/d25/d6d/f6e [0,4194304] 0 2024-02-19T19:06:24.386 INFO:tasks.workunit.client.0.smithi088.stdout:8/431: write d9/df/d1c/f33 [32125,76641] 0 2024-02-19T19:06:24.387 INFO:tasks.workunit.client.0.smithi088.stdout:8/432: chown d9/df/d1c/c3f 312 1 2024-02-19T19:06:24.387 INFO:tasks.workunit.client.0.smithi088.stdout:0/405: chown d13/d46/d5a/d25/d2a/c7e 207 1 2024-02-19T19:06:24.388 INFO:tasks.workunit.client.0.smithi088.stdout:3/587: mkdir d4/d1a/d23/dc9 0 2024-02-19T19:06:24.389 INFO:tasks.workunit.client.0.smithi088.stdout:8/433: chown d9/df/d3d 11 1 2024-02-19T19:06:24.391 INFO:tasks.workunit.client.0.smithi088.stdout:3/588: fdatasync d4/d1a/fb8 0 2024-02-19T19:06:24.392 INFO:tasks.workunit.client.0.smithi088.stdout:0/406: fsync d13/d46/d5a/d25/d2a/d34/f77 0 2024-02-19T19:06:24.394 INFO:tasks.workunit.client.0.smithi088.stdout:8/434: symlink d9/df/d1c/d2d/d47/d6b/l80 0 2024-02-19T19:06:24.395 INFO:tasks.workunit.client.0.smithi088.stdout:8/435: chown d9/df/d1c/f16 14140 1 2024-02-19T19:06:24.395 INFO:tasks.workunit.client.0.smithi088.stdout:0/407: mknod d13/d46/d5a/d25/d2a/d2f/c8d 0 2024-02-19T19:06:24.396 INFO:tasks.workunit.client.0.smithi088.stdout:0/408: write d13/f8c [593421,20638] 0 2024-02-19T19:06:24.402 INFO:tasks.workunit.client.0.smithi088.stdout:8/436: dwrite d9/df/d1c/f29 [0,4194304] 0 2024-02-19T19:06:24.402 INFO:tasks.workunit.client.0.smithi088.stdout:0/409: unlink d13/d46/d5a/d25/f58 0 2024-02-19T19:06:24.403 INFO:tasks.workunit.client.0.smithi088.stdout:8/437: write d9/f63 [118569,67968] 0 2024-02-19T19:06:24.404 INFO:tasks.workunit.client.0.smithi088.stdout:8/438: chown d9/df/d1c/d2d 0 1 2024-02-19T19:06:24.407 INFO:tasks.workunit.client.0.smithi088.stdout:0/410: link d13/f1b d13/d62/f8e 0 2024-02-19T19:06:24.408 INFO:tasks.workunit.client.0.smithi088.stdout:5/597: dread d4/d61/d65/d9a/d9f/da0/fa9 [12582912,4194304] 0 2024-02-19T19:06:24.410 INFO:tasks.workunit.client.0.smithi088.stdout:8/439: creat d9/df/d1c/d2d/d47/d6b/d72/f81 x:0 0 0 2024-02-19T19:06:24.412 INFO:tasks.workunit.client.0.smithi088.stdout:8/440: write d9/df/d1c/f73 [1042964,125177] 0 2024-02-19T19:06:24.412 INFO:tasks.workunit.client.0.smithi088.stdout:0/411: fdatasync d13/d46/d5a/d25/f26 0 2024-02-19T19:06:24.412 INFO:tasks.workunit.client.0.smithi088.stdout:0/412: chown d13/f29 434 1 2024-02-19T19:06:24.439 INFO:tasks.workunit.client.0.smithi088.stdout:2/395: sync 2024-02-19T19:06:24.440 INFO:tasks.workunit.client.0.smithi088.stdout:9/525: sync 2024-02-19T19:06:24.445 INFO:tasks.workunit.client.0.smithi088.stdout:2/396: dwrite d3/d1f/d5d/f72 [0,4194304] 0 2024-02-19T19:06:24.446 INFO:tasks.workunit.client.0.smithi088.stdout:2/397: chown d3/d1f/d5d/f62 708913362 1 2024-02-19T19:06:24.447 INFO:tasks.workunit.client.0.smithi088.stdout:2/398: write d3/f3b [463935,40154] 0 2024-02-19T19:06:24.449 INFO:tasks.workunit.client.0.smithi088.stdout:2/399: read - d3/d1b/d43/d61/d64/f78 zero size 2024-02-19T19:06:24.450 INFO:tasks.workunit.client.0.smithi088.stdout:2/400: fdatasync d3/d1b/d43/f55 0 2024-02-19T19:06:24.455 INFO:tasks.workunit.client.0.smithi088.stdout:9/526: mkdir d1/d8/d4a/d4f/daa 0 2024-02-19T19:06:24.460 INFO:tasks.workunit.client.0.smithi088.stdout:2/401: write d3/d1f/f4a [6500934,49938] 0 2024-02-19T19:06:24.461 INFO:tasks.workunit.client.0.smithi088.stdout:9/527: fsync d1/d8/d3e/d93/d5a/f63 0 2024-02-19T19:06:24.462 INFO:tasks.workunit.client.0.smithi088.stdout:9/528: stat d1/d8/d3e/d93/fa1 0 2024-02-19T19:06:24.466 INFO:tasks.workunit.client.0.smithi088.stdout:9/529: creat d1/d8/d3e/d93/d6f/fab x:0 0 0 2024-02-19T19:06:24.471 INFO:tasks.workunit.client.0.smithi088.stdout:9/530: dread d1/d8/d37/d54/f75 [0,4194304] 0 2024-02-19T19:06:24.480 INFO:tasks.workunit.client.0.smithi088.stdout:9/531: dread d1/d8/f28 [0,4194304] 0 2024-02-19T19:06:24.487 INFO:tasks.workunit.client.0.smithi088.stdout:8/441: mkdir d9/df/d1c/d2d/d82 0 2024-02-19T19:06:24.488 INFO:tasks.workunit.client.0.smithi088.stdout:8/442: read f5 [2018861,74365] 0 2024-02-19T19:06:24.489 INFO:tasks.workunit.client.0.smithi088.stdout:9/532: rename d1/d60 to d1/d8/d4a/dac 0 2024-02-19T19:06:24.490 INFO:tasks.workunit.client.0.smithi088.stdout:9/533: fsync d1/f2b 0 2024-02-19T19:06:24.490 INFO:tasks.workunit.client.0.smithi088.stdout:8/443: mknod d9/df/d1c/d2d/d47/d6b/d72/d5d/c83 0 2024-02-19T19:06:24.491 INFO:tasks.workunit.client.0.smithi088.stdout:9/534: chown d1/d8/d3e/d93/d5a/c98 2 1 2024-02-19T19:06:24.493 INFO:tasks.workunit.client.0.smithi088.stdout:9/535: creat d1/d8/d4a/d4f/fad x:0 0 0 2024-02-19T19:06:24.499 INFO:tasks.workunit.client.0.smithi088.stdout:0/413: rmdir d13/d46/d5a/d25 39 2024-02-19T19:06:24.501 INFO:tasks.workunit.client.0.smithi088.stdout:0/414: creat d13/d62/f8f x:0 0 0 2024-02-19T19:06:24.502 INFO:tasks.workunit.client.0.smithi088.stdout:0/415: read d13/f6a [90836,52048] 0 2024-02-19T19:06:24.505 INFO:tasks.workunit.client.0.smithi088.stdout:0/416: mknod d13/d62/d8a/d87/c90 0 2024-02-19T19:06:24.505 INFO:tasks.workunit.client.0.smithi088.stdout:0/417: read - d13/d46/f6c zero size 2024-02-19T19:06:24.506 INFO:tasks.workunit.client.0.smithi088.stdout:0/418: chown d13/f2c 15874 1 2024-02-19T19:06:24.506 INFO:tasks.workunit.client.0.smithi088.stdout:0/419: fdatasync d13/d62/f8f 0 2024-02-19T19:06:24.508 INFO:tasks.workunit.client.0.smithi088.stdout:0/420: fsync d13/d46/d5a/d25/d2a/d3c/f5d 0 2024-02-19T19:06:24.509 INFO:tasks.workunit.client.0.smithi088.stdout:0/421: chown d13/f39 3 1 2024-02-19T19:06:24.510 INFO:tasks.workunit.client.0.smithi088.stdout:0/422: stat d13/d46/d5a/d25/d2a/d34/c47 0 2024-02-19T19:06:24.512 INFO:tasks.workunit.client.0.smithi088.stdout:8/444: write d9/df/f1a [5465639,110766] 0 2024-02-19T19:06:24.515 INFO:tasks.workunit.client.0.smithi088.stdout:8/445: write d9/df/d1c/f73 [1135486,128233] 0 2024-02-19T19:06:24.517 INFO:tasks.workunit.client.0.smithi088.stdout:0/423: dwrite d13/d46/d5a/d25/d2a/f7c [0,4194304] 0 2024-02-19T19:06:24.518 INFO:tasks.workunit.client.0.smithi088.stdout:0/424: chown d13/d46/d5b 2 1 2024-02-19T19:06:24.519 INFO:tasks.workunit.client.0.smithi088.stdout:8/446: mknod d9/c84 0 2024-02-19T19:06:24.520 INFO:tasks.workunit.client.0.smithi088.stdout:0/425: write d13/f6a [4693134,11794] 0 2024-02-19T19:06:24.521 INFO:tasks.workunit.client.0.smithi088.stdout:8/447: read - d9/df/d1c/d2d/d47/d6b/d72/d5d/f7b zero size 2024-02-19T19:06:24.522 INFO:tasks.workunit.client.0.smithi088.stdout:8/448: dread - d9/df/d1c/d2d/f6d zero size 2024-02-19T19:06:24.522 INFO:tasks.workunit.client.0.smithi088.stdout:0/426: creat d13/d46/d5b/d76/f91 x:0 0 0 2024-02-19T19:06:24.524 INFO:tasks.workunit.client.0.smithi088.stdout:0/427: creat d13/d62/d8a/d87/f92 x:0 0 0 2024-02-19T19:06:24.528 INFO:tasks.workunit.client.0.smithi088.stdout:8/449: unlink d9/df/d1c/d2d/d47/d6b/d72/d5d/f68 0 2024-02-19T19:06:24.533 INFO:tasks.workunit.client.0.smithi088.stdout:0/428: write d13/d46/d5a/d25/d2a/d2f/f4c [1764716,124293] 0 2024-02-19T19:06:24.534 INFO:tasks.workunit.client.0.smithi088.stdout:8/450: truncate d9/df/d1c/d2d/f4b 2779481 0 2024-02-19T19:06:24.536 INFO:tasks.workunit.client.0.smithi088.stdout:0/429: dread d13/d46/d5a/d25/d2a/d2f/f41 [0,4194304] 0 2024-02-19T19:06:24.537 INFO:tasks.workunit.client.0.smithi088.stdout:0/430: chown d13/d46/d5a/d25/d2a/d34/l45 57196236 1 2024-02-19T19:06:24.539 INFO:tasks.workunit.client.0.smithi088.stdout:0/431: mknod d13/c93 0 2024-02-19T19:06:24.542 INFO:tasks.workunit.client.0.smithi088.stdout:0/432: fsync d13/d46/d5a/d25/f8b 0 2024-02-19T19:06:24.543 INFO:tasks.workunit.client.0.smithi088.stdout:8/451: dwrite d9/df/d3d/d62/f69 [4194304,4194304] 0 2024-02-19T19:06:24.547 INFO:tasks.workunit.client.0.smithi088.stdout:0/433: mknod d13/d46/d5b/d76/c94 0 2024-02-19T19:06:24.548 INFO:tasks.workunit.client.0.smithi088.stdout:0/434: write d13/d46/d5a/d25/d2a/d34/f75 [103461,74784] 0 2024-02-19T19:06:24.548 INFO:tasks.workunit.client.0.smithi088.stdout:8/452: mkdir d9/df/d1c/d85 0 2024-02-19T19:06:24.552 INFO:tasks.workunit.client.0.smithi088.stdout:8/453: mknod d9/df/d3d/c86 0 2024-02-19T19:06:24.555 INFO:tasks.workunit.client.0.smithi088.stdout:0/435: dwrite d13/d46/d5a/d25/d6d/f7f [0,4194304] 0 2024-02-19T19:06:24.559 INFO:tasks.workunit.client.0.smithi088.stdout:8/454: rename d9/df/d1c/d2d/d47/d6b/f76 to d9/df/d3d/f87 0 2024-02-19T19:06:24.567 INFO:tasks.workunit.client.0.smithi088.stdout:8/455: dwrite d9/df/d1c/d2d/d47/d6b/d72/d5d/f7b [0,4194304] 0 2024-02-19T19:06:24.569 INFO:tasks.workunit.client.0.smithi088.stdout:0/436: rename d13/d46 to d13/d62/d8a/d95 0 2024-02-19T19:06:24.572 INFO:tasks.workunit.client.0.smithi088.stdout:0/437: mkdir d13/d62/d8a/d95/d96 0 2024-02-19T19:06:24.575 INFO:tasks.workunit.client.0.smithi088.stdout:8/456: link d9/l34 d9/df/d3d/l88 0 2024-02-19T19:06:24.578 INFO:tasks.workunit.client.0.smithi088.stdout:8/457: link d9/df/d1c/d2d/d47/f46 d9/d6f/f89 0 2024-02-19T19:06:24.579 INFO:tasks.workunit.client.0.smithi088.stdout:8/458: fsync d9/df/d1c/d2d/d47/f44 0 2024-02-19T19:06:24.582 INFO:tasks.workunit.client.0.smithi088.stdout:0/438: truncate d13/d62/d8a/d95/d5a/d25/d2a/d3c/f64 2724208 0 2024-02-19T19:06:24.585 INFO:tasks.workunit.client.0.smithi088.stdout:0/439: read d13/f39 [3573296,108974] 0 2024-02-19T19:06:24.588 INFO:tasks.workunit.client.0.smithi088.stdout:0/440: creat d13/d62/d8a/d87/f97 x:0 0 0 2024-02-19T19:06:24.590 INFO:tasks.workunit.client.0.smithi088.stdout:0/441: dread d13/d62/d8a/d95/d5a/d25/d2a/d2f/f41 [0,4194304] 0 2024-02-19T19:06:24.591 INFO:tasks.workunit.client.0.smithi088.stdout:6/501: sync 2024-02-19T19:06:24.593 INFO:tasks.workunit.client.0.smithi088.stdout:0/442: creat d13/d62/d8a/d95/d5a/d25/d6d/f98 x:0 0 0 2024-02-19T19:06:24.594 INFO:tasks.workunit.client.0.smithi088.stdout:6/502: creat d5/de/d97/f9b x:0 0 0 2024-02-19T19:06:24.594 INFO:tasks.workunit.client.0.smithi088.stdout:8/459: dread d9/df/f27 [0,4194304] 0 2024-02-19T19:06:24.597 INFO:tasks.workunit.client.0.smithi088.stdout:0/443: link d13/l16 d13/d62/l99 0 2024-02-19T19:06:24.598 INFO:tasks.workunit.client.0.smithi088.stdout:6/503: dread d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:24.599 INFO:tasks.workunit.client.0.smithi088.stdout:0/444: symlink d13/d62/d8a/d95/d96/l9a 0 2024-02-19T19:06:24.602 INFO:tasks.workunit.client.0.smithi088.stdout:0/445: mkdir d13/d62/d8a/d95/d5a/d25/d2a/d34/d9b 0 2024-02-19T19:06:24.604 INFO:tasks.workunit.client.0.smithi088.stdout:0/446: readlink d13/d62/d8a/d95/d5a/d25/d2a/d34/l45 0 2024-02-19T19:06:24.604 INFO:tasks.workunit.client.0.smithi088.stdout:6/504: dwrite d5/d7/d32/f53 [0,4194304] 0 2024-02-19T19:06:24.605 INFO:tasks.workunit.client.0.smithi088.stdout:6/505: chown d5/d8/d3a/l65 4 1 2024-02-19T19:06:24.607 INFO:tasks.workunit.client.0.smithi088.stdout:0/447: mkdir d13/d62/d8a/d9c 0 2024-02-19T19:06:24.609 INFO:tasks.workunit.client.0.smithi088.stdout:6/506: unlink d5/d7/d32/l3e 0 2024-02-19T19:06:24.610 INFO:tasks.workunit.client.0.smithi088.stdout:6/507: mknod d5/d8/d77/c9c 0 2024-02-19T19:06:24.611 INFO:tasks.workunit.client.0.smithi088.stdout:6/508: stat d5/d8/f3f 0 2024-02-19T19:06:24.623 INFO:tasks.workunit.client.0.smithi088.stdout:8/460: dread f6 [0,4194304] 0 2024-02-19T19:06:24.628 INFO:tasks.workunit.client.0.smithi088.stdout:8/461: dwrite d9/df/d1c/f1f [0,4194304] 0 2024-02-19T19:06:24.629 INFO:tasks.workunit.client.0.smithi088.stdout:8/462: write d9/df/d1c/d2d/d47/f57 [786697,61419] 0 2024-02-19T19:06:24.641 INFO:tasks.workunit.client.0.smithi088.stdout:6/509: mkdir d5/d45/d9d 0 2024-02-19T19:06:24.641 INFO:tasks.workunit.client.0.smithi088.stdout:6/510: fsync d5/d8/d3a/d71/f74 0 2024-02-19T19:06:24.642 INFO:tasks.workunit.client.0.smithi088.stdout:6/511: stat d5/d19/d70/l7a 0 2024-02-19T19:06:24.649 INFO:tasks.workunit.client.0.smithi088.stdout:0/448: dwrite d13/f1b [4194304,4194304] 0 2024-02-19T19:06:24.651 INFO:tasks.workunit.client.0.smithi088.stdout:8/463: dread d9/df/d1c/d2d/d47/f5e [0,4194304] 0 2024-02-19T19:06:24.658 INFO:tasks.workunit.client.0.smithi088.stdout:8/464: dwrite d9/df/d1c/d2d/f3e [0,4194304] 0 2024-02-19T19:06:24.664 INFO:tasks.workunit.client.0.smithi088.stdout:8/465: truncate d9/df/d1c/d2d/d47/f44 2835797 0 2024-02-19T19:06:24.666 INFO:tasks.workunit.client.0.smithi088.stdout:8/466: write d9/df/d1c/d2d/d47/f4d [1597401,68643] 0 2024-02-19T19:06:24.669 INFO:tasks.workunit.client.0.smithi088.stdout:0/449: dread d13/d62/d8a/d95/d5a/d25/f8b [0,4194304] 0 2024-02-19T19:06:24.669 INFO:tasks.workunit.client.0.smithi088.stdout:8/467: write d9/df/d3d/f87 [268785,101865] 0 2024-02-19T19:06:24.671 INFO:tasks.workunit.client.0.smithi088.stdout:8/468: creat d9/df/d1c/d2d/d47/d6b/f8a x:0 0 0 2024-02-19T19:06:24.672 INFO:tasks.workunit.client.0.smithi088.stdout:8/469: chown d9/c84 3333786 1 2024-02-19T19:06:24.675 INFO:tasks.workunit.client.0.smithi088.stdout:8/470: rmdir d9/df/d1c/d2d/d6c/d77 39 2024-02-19T19:06:24.676 INFO:tasks.workunit.client.0.smithi088.stdout:0/450: dwrite d13/d62/d8a/d95/d5a/d25/d2a/f7c [0,4194304] 0 2024-02-19T19:06:24.680 INFO:tasks.workunit.client.0.smithi088.stdout:8/471: creat d9/df/d3d/d50/d71/f8b x:0 0 0 2024-02-19T19:06:24.681 INFO:tasks.workunit.client.0.smithi088.stdout:8/472: chown d9/df/d1c/d22/l3c 75585475 1 2024-02-19T19:06:24.684 INFO:tasks.workunit.client.0.smithi088.stdout:0/451: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d3c/f55 [0,4194304] 0 2024-02-19T19:06:24.687 INFO:tasks.workunit.client.0.smithi088.stdout:8/473: truncate d9/f75 2275689 0 2024-02-19T19:06:24.689 INFO:tasks.workunit.client.0.smithi088.stdout:8/474: symlink d9/df/l8c 0 2024-02-19T19:06:24.690 INFO:tasks.workunit.client.0.smithi088.stdout:0/452: rmdir d13/d62/d8a/d95/d5a/d25/d2a/d34/d9b 0 2024-02-19T19:06:24.693 INFO:tasks.workunit.client.0.smithi088.stdout:8/475: link d9/df/d1c/f13 d9/df/d1c/d85/f8d 0 2024-02-19T19:06:24.695 INFO:tasks.workunit.client.0.smithi088.stdout:8/476: unlink d9/df/d1c/d22/f66 0 2024-02-19T19:06:24.698 INFO:tasks.workunit.client.0.smithi088.stdout:8/477: mkdir d9/df/d1c/d2d/d47/d6b/d72/d7f/d8e 0 2024-02-19T19:06:24.700 INFO:tasks.workunit.client.0.smithi088.stdout:8/478: unlink d9/df/d3d/l4e 0 2024-02-19T19:06:24.702 INFO:tasks.workunit.client.0.smithi088.stdout:8/479: mknod d9/df/d1c/d2d/d47/d6b/d72/d79/c8f 0 2024-02-19T19:06:24.704 INFO:tasks.workunit.client.0.smithi088.stdout:8/480: mknod d9/df/d1c/d2d/d47/d6b/d72/d7f/d8e/c90 0 2024-02-19T19:06:24.704 INFO:tasks.workunit.client.0.smithi088.stdout:8/481: truncate d9/f4a 2002988 0 2024-02-19T19:06:24.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:24.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:24.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:24 smithi088 ceph-mon[88643]: Reconfiguring mds.cephfs.smithi132.uyjxaz (monmap changed)... 2024-02-19T19:06:24.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.uyjxaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:06:24.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:24.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:24 smithi088 ceph-mon[88643]: Reconfiguring daemon mds.cephfs.smithi132.uyjxaz on smithi132 2024-02-19T19:06:24.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:24.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:24.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:24 smithi132 ceph-mon[70676]: Reconfiguring mds.cephfs.smithi132.uyjxaz (monmap changed)... 2024-02-19T19:06:24.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.uyjxaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:06:24.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:24.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:24 smithi132 ceph-mon[70676]: Reconfiguring daemon mds.cephfs.smithi132.uyjxaz on smithi132 2024-02-19T19:06:24.918 INFO:tasks.workunit.client.0.smithi088.stdout:1/590: sync 2024-02-19T19:06:24.919 INFO:tasks.workunit.client.0.smithi088.stdout:1/591: write d6/d10/f78 [952506,34624] 0 2024-02-19T19:06:24.930 INFO:tasks.workunit.client.0.smithi088.stdout:1/592: write d6/d21/d6a/d7d/d7e/fac [1294281,26999] 0 2024-02-19T19:06:24.933 INFO:tasks.workunit.client.0.smithi088.stdout:1/593: unlink d6/d21/d64/d94/d71/f5c 0 2024-02-19T19:06:24.935 INFO:tasks.workunit.client.0.smithi088.stdout:1/594: unlink d6/d21/d64/c8f 0 2024-02-19T19:06:24.937 INFO:tasks.workunit.client.0.smithi088.stdout:1/595: symlink d6/d21/d6a/d7d/d7e/lc2 0 2024-02-19T19:06:24.940 INFO:tasks.workunit.client.0.smithi088.stdout:1/596: mknod d6/d21/cc3 0 2024-02-19T19:06:24.943 INFO:tasks.workunit.client.0.smithi088.stdout:1/597: truncate d6/d21/d6a/f6d 14835 0 2024-02-19T19:06:24.948 INFO:tasks.workunit.client.0.smithi088.stdout:1/598: dwrite d6/d21/d6a/f8e [0,4194304] 0 2024-02-19T19:06:24.971 INFO:tasks.workunit.client.0.smithi088.stdout:1/599: dread d6/f1b [0,4194304] 0 2024-02-19T19:06:24.979 INFO:tasks.workunit.client.0.smithi088.stdout:2/402: sync 2024-02-19T19:06:24.979 INFO:tasks.workunit.client.0.smithi088.stdout:3/589: sync 2024-02-19T19:06:24.979 INFO:tasks.workunit.client.0.smithi088.stdout:7/536: sync 2024-02-19T19:06:24.979 INFO:tasks.workunit.client.0.smithi088.stdout:9/536: sync 2024-02-19T19:06:24.979 INFO:tasks.workunit.client.0.smithi088.stdout:8/482: sync 2024-02-19T19:06:24.979 INFO:tasks.workunit.client.0.smithi088.stdout:0/453: sync 2024-02-19T19:06:24.979 INFO:tasks.workunit.client.0.smithi088.stdout:4/399: sync 2024-02-19T19:06:24.980 INFO:tasks.workunit.client.0.smithi088.stdout:5/598: sync 2024-02-19T19:06:24.980 INFO:tasks.workunit.client.0.smithi088.stdout:6/512: sync 2024-02-19T19:06:24.982 INFO:tasks.workunit.client.0.smithi088.stdout:7/537: fdatasync d14/d27/d6b/f7b 0 2024-02-19T19:06:24.984 INFO:tasks.workunit.client.0.smithi088.stdout:5/599: creat d4/d1a/d41/fcc x:0 0 0 2024-02-19T19:06:24.985 INFO:tasks.workunit.client.0.smithi088.stdout:6/513: write d5/de/f22 [777687,52869] 0 2024-02-19T19:06:24.986 INFO:tasks.workunit.client.0.smithi088.stdout:1/600: dread d6/d21/f3c [0,4194304] 0 2024-02-19T19:06:24.987 INFO:tasks.workunit.client.0.smithi088.stdout:9/537: dread d1/d8/f5c [0,4194304] 0 2024-02-19T19:06:24.988 INFO:tasks.workunit.client.0.smithi088.stdout:4/400: rename d2/d32/d25/d26/c68 to d2/d35/d39/d3a/c7b 0 2024-02-19T19:06:24.992 INFO:tasks.workunit.client.0.smithi088.stdout:0/454: creat d13/d62/d8a/d95/d5a/f9d x:0 0 0 2024-02-19T19:06:24.994 INFO:tasks.workunit.client.0.smithi088.stdout:5/600: creat d4/d61/d65/d9a/d9f/fcd x:0 0 0 2024-02-19T19:06:24.995 INFO:tasks.workunit.client.0.smithi088.stdout:8/483: dwrite d9/df/d1c/d2d/d47/f4d [0,4194304] 0 2024-02-19T19:06:24.995 INFO:tasks.workunit.client.0.smithi088.stdout:2/403: dread d3/d1b/d43/d61/d64/f6d [0,4194304] 0 2024-02-19T19:06:24.997 INFO:tasks.workunit.client.0.smithi088.stdout:2/404: chown d3/d1b/d43/d74 55238740 1 2024-02-19T19:06:24.997 INFO:tasks.workunit.client.0.smithi088.stdout:3/590: rename d4/d1a/d1e/d31/d32/d55/f57 to d4/d22/d7a/d82/fca 0 2024-02-19T19:06:24.998 INFO:tasks.workunit.client.0.smithi088.stdout:8/484: read - d9/df/d1c/d2d/d47/d6b/f8a zero size 2024-02-19T19:06:24.999 INFO:tasks.workunit.client.0.smithi088.stdout:2/405: read d3/d14/f4d [17226,88814] 0 2024-02-19T19:06:24.999 INFO:tasks.workunit.client.0.smithi088.stdout:7/538: link d14/d1d/f38 d14/d28/d9c/fc0 0 2024-02-19T19:06:25.000 INFO:tasks.workunit.client.0.smithi088.stdout:9/538: fdatasync d1/d8/d3e/d93/d2d/f2a 0 2024-02-19T19:06:25.002 INFO:tasks.workunit.client.0.smithi088.stdout:1/601: truncate d6/d10/d60/d29/fb9 415862 0 2024-02-19T19:06:25.004 INFO:tasks.workunit.client.0.smithi088.stdout:6/514: dread d5/d8/d2c/f6c [0,4194304] 0 2024-02-19T19:06:25.005 INFO:tasks.workunit.client.0.smithi088.stdout:3/591: rmdir d4/d1a/d1e/d74/d7c/d97 39 2024-02-19T19:06:25.006 INFO:tasks.workunit.client.0.smithi088.stdout:6/515: chown d5/d8/f7b 181 1 2024-02-19T19:06:25.011 INFO:tasks.workunit.client.0.smithi088.stdout:8/485: symlink d9/df/d1c/d2d/d70/l91 0 2024-02-19T19:06:25.012 INFO:tasks.workunit.client.0.smithi088.stdout:1/602: dwrite d6/d21/d64/d94/d71/f7a [0,4194304] 0 2024-02-19T19:06:25.013 INFO:tasks.workunit.client.0.smithi088.stdout:8/486: stat d9/df/f55 0 2024-02-19T19:06:25.015 INFO:tasks.workunit.client.0.smithi088.stdout:3/592: mkdir d4/d1a/d1e/d74/d7d/daa/dcb 0 2024-02-19T19:06:25.015 INFO:tasks.workunit.client.0.smithi088.stdout:8/487: write d9/df/d1c/f16 [10794172,117953] 0 2024-02-19T19:06:25.017 INFO:tasks.workunit.client.0.smithi088.stdout:6/516: mkdir d5/de/d64/d8d/d9e 0 2024-02-19T19:06:25.017 INFO:tasks.workunit.client.0.smithi088.stdout:8/488: dread - d9/df/d3d/d50/d71/f8b zero size 2024-02-19T19:06:25.018 INFO:tasks.workunit.client.0.smithi088.stdout:0/455: link d13/d62/d8a/d95/d5a/d25/f8b d13/d62/f9e 0 2024-02-19T19:06:25.020 INFO:tasks.workunit.client.0.smithi088.stdout:0/456: write d13/f2c [1584850,111441] 0 2024-02-19T19:06:25.021 INFO:tasks.workunit.client.0.smithi088.stdout:4/401: rename d2/d32/d25/d26/f7a to d2/d35/f7c 0 2024-02-19T19:06:25.022 INFO:tasks.workunit.client.0.smithi088.stdout:4/402: stat d2/d32/d25 0 2024-02-19T19:06:25.024 INFO:tasks.workunit.client.0.smithi088.stdout:0/457: chown d13/d62/d8a/d95/d5a/d25/d2a/d34/f43 9 1 2024-02-19T19:06:25.025 INFO:tasks.workunit.client.0.smithi088.stdout:5/601: link d4/d1a/l50 d4/d61/lce 0 2024-02-19T19:06:25.026 INFO:tasks.workunit.client.0.smithi088.stdout:4/403: read d2/d35/f52 [321980,4674] 0 2024-02-19T19:06:25.027 INFO:tasks.workunit.client.0.smithi088.stdout:1/603: creat d6/d10/fc4 x:0 0 0 2024-02-19T19:06:25.028 INFO:tasks.workunit.client.0.smithi088.stdout:3/593: creat d4/d22/d81/fcc x:0 0 0 2024-02-19T19:06:25.029 INFO:tasks.workunit.client.0.smithi088.stdout:6/517: dwrite d5/d8/d2c/f4c [0,4194304] 0 2024-02-19T19:06:25.035 INFO:tasks.workunit.client.0.smithi088.stdout:7/539: dread d14/f3e [0,4194304] 0 2024-02-19T19:06:25.039 INFO:tasks.workunit.client.0.smithi088.stdout:0/458: mknod d13/d62/c9f 0 2024-02-19T19:06:25.042 INFO:tasks.workunit.client.0.smithi088.stdout:1/604: chown d6/d21/d64/d94/d6e/f97 22 1 2024-02-19T19:06:25.044 INFO:tasks.workunit.client.0.smithi088.stdout:1/605: dread - d6/d10/fc4 zero size 2024-02-19T19:06:25.045 INFO:tasks.workunit.client.0.smithi088.stdout:5/602: read d4/d1a/f90 [680777,74125] 0 2024-02-19T19:06:25.045 INFO:tasks.workunit.client.0.smithi088.stdout:2/406: truncate d3/d1b/f5b 3997054 0 2024-02-19T19:06:25.046 INFO:tasks.workunit.client.0.smithi088.stdout:1/606: readlink d6/d21/d64/d94/d71/l31 0 2024-02-19T19:06:25.046 INFO:tasks.workunit.client.0.smithi088.stdout:3/594: creat d4/d1a/d7f/d88/fcd x:0 0 0 2024-02-19T19:06:25.049 INFO:tasks.workunit.client.0.smithi088.stdout:0/459: dwrite d13/d62/d8a/d95/d5a/d25/d6d/f98 [0,4194304] 0 2024-02-19T19:06:25.050 INFO:tasks.workunit.client.0.smithi088.stdout:1/607: truncate d6/d21/f70 1841492 0 2024-02-19T19:06:25.050 INFO:tasks.workunit.client.0.smithi088.stdout:0/460: fdatasync d13/d62/d8a/d87/f92 0 2024-02-19T19:06:25.052 INFO:tasks.workunit.client.0.smithi088.stdout:9/539: truncate d1/d8/d3e/d93/d2d/d23/f35 1999164 0 2024-02-19T19:06:25.052 INFO:tasks.workunit.client.0.smithi088.stdout:0/461: stat d13/d62/d8a/d95/d5a/d25/d2a/d3c/f55 0 2024-02-19T19:06:25.057 INFO:tasks.workunit.client.0.smithi088.stdout:7/540: getdents d14/d27/d49/d98/dbe 0 2024-02-19T19:06:25.060 INFO:tasks.workunit.client.0.smithi088.stdout:1/608: rename d6/d21/d64/d94/d71/l31 to d6/d21/d64/d94/dab/lc5 0 2024-02-19T19:06:25.061 INFO:tasks.workunit.client.0.smithi088.stdout:1/609: write d6/d21/d6a/d4a/fa7 [427365,45993] 0 2024-02-19T19:06:25.063 INFO:tasks.workunit.client.0.smithi088.stdout:8/489: getdents d9/df/d1c/d2d/d47/d6b/d72 0 2024-02-19T19:06:25.065 INFO:tasks.workunit.client.0.smithi088.stdout:0/462: write d13/f29 [3615302,59035] 0 2024-02-19T19:06:25.068 INFO:tasks.workunit.client.0.smithi088.stdout:6/518: dwrite d5/d8/f3f [0,4194304] 0 2024-02-19T19:06:25.070 INFO:tasks.workunit.client.0.smithi088.stdout:2/407: creat d3/d1b/d43/d61/d77/f7b x:0 0 0 2024-02-19T19:06:25.073 INFO:tasks.workunit.client.0.smithi088.stdout:4/404: link d2/d32/d9/l1e d2/d32/l7d 0 2024-02-19T19:06:25.075 INFO:tasks.workunit.client.0.smithi088.stdout:9/540: write d1/d8/d37/f44 [999348,70640] 0 2024-02-19T19:06:25.077 INFO:tasks.workunit.client.0.smithi088.stdout:5/603: truncate d4/f5 4086477 0 2024-02-19T19:06:25.080 INFO:tasks.workunit.client.0.smithi088.stdout:1/610: creat d6/d10/d60/d29/fc6 x:0 0 0 2024-02-19T19:06:25.082 INFO:tasks.workunit.client.0.smithi088.stdout:7/541: mkdir d14/d1d/d3c/d43/db2/dc1 0 2024-02-19T19:06:25.082 INFO:tasks.workunit.client.0.smithi088.stdout:0/463: creat d13/d62/d8a/d95/d5a/d25/d6d/fa0 x:0 0 0 2024-02-19T19:06:25.085 INFO:tasks.workunit.client.0.smithi088.stdout:2/408: mknod d3/d1f/d4f/d59/c7c 0 2024-02-19T19:06:25.087 INFO:tasks.workunit.client.0.smithi088.stdout:4/405: rename d2/d35/d39 to d2/d32/d25/d26/d6d/d7e 0 2024-02-19T19:06:25.090 INFO:tasks.workunit.client.0.smithi088.stdout:8/490: dread d9/df/f36 [0,4194304] 0 2024-02-19T19:06:25.091 INFO:tasks.workunit.client.0.smithi088.stdout:8/491: chown d9/df/d1c/d2d/d47/d6b/d72/d7f/d8e 3 1 2024-02-19T19:06:25.093 INFO:tasks.workunit.client.0.smithi088.stdout:1/611: symlink d6/d10/d60/db8/lc7 0 2024-02-19T19:06:25.096 INFO:tasks.workunit.client.0.smithi088.stdout:3/595: getdents d4/d12/d33/d58 0 2024-02-19T19:06:25.097 INFO:tasks.workunit.client.0.smithi088.stdout:8/492: truncate d9/df/d3d/d50/d71/f8b 769663 0 2024-02-19T19:06:25.098 INFO:tasks.workunit.client.0.smithi088.stdout:6/519: creat d5/de/d64/d8d/d9e/f9f x:0 0 0 2024-02-19T19:06:25.101 INFO:tasks.workunit.client.0.smithi088.stdout:7/542: dread d14/d27/d49/fa2 [0,4194304] 0 2024-02-19T19:06:25.101 INFO:tasks.workunit.client.0.smithi088.stdout:9/541: dwrite d1/d8/d3e/d93/d2d/d23/d79/f86 [0,4194304] 0 2024-02-19T19:06:25.102 INFO:tasks.workunit.client.0.smithi088.stdout:6/520: write d5/de/f22 [672633,30564] 0 2024-02-19T19:06:25.103 INFO:tasks.workunit.client.0.smithi088.stdout:6/521: dread - d5/de/d64/f66 zero size 2024-02-19T19:06:25.105 INFO:tasks.workunit.client.0.smithi088.stdout:2/409: symlink d3/d1b/l7d 0 2024-02-19T19:06:25.106 INFO:tasks.workunit.client.0.smithi088.stdout:4/406: mknod d2/d32/d25/d26/d58/c7f 0 2024-02-19T19:06:25.107 INFO:tasks.workunit.client.0.smithi088.stdout:4/407: write d2/d32/f1c [6378454,106742] 0 2024-02-19T19:06:25.109 INFO:tasks.workunit.client.0.smithi088.stdout:1/612: mkdir d6/d21/d64/dc8 0 2024-02-19T19:06:25.112 INFO:tasks.workunit.client.0.smithi088.stdout:3/596: creat d4/d22/d7a/d8d/fce x:0 0 0 2024-02-19T19:06:25.114 INFO:tasks.workunit.client.0.smithi088.stdout:8/493: creat d9/df/d1c/d2d/d47/d6b/d72/d5d/f92 x:0 0 0 2024-02-19T19:06:25.114 INFO:tasks.workunit.client.0.smithi088.stdout:9/542: fdatasync d1/d8/d37/d54/f75 0 2024-02-19T19:06:25.116 INFO:tasks.workunit.client.0.smithi088.stdout:2/410: mkdir d3/d1b/d43/d61/d7e 0 2024-02-19T19:06:25.117 INFO:tasks.workunit.client.0.smithi088.stdout:2/411: fsync d3/fb 0 2024-02-19T19:06:25.118 INFO:tasks.workunit.client.0.smithi088.stdout:0/464: dwrite d13/f23 [0,4194304] 0 2024-02-19T19:06:25.118 INFO:tasks.workunit.client.0.smithi088.stdout:5/604: creat d4/fcf x:0 0 0 2024-02-19T19:06:25.120 INFO:tasks.workunit.client.0.smithi088.stdout:0/465: readlink d13/d62/d8a/d95/d96/l9a 0 2024-02-19T19:06:25.125 INFO:tasks.workunit.client.0.smithi088.stdout:3/597: truncate d4/f40 2091784 0 2024-02-19T19:06:25.127 INFO:tasks.workunit.client.0.smithi088.stdout:8/494: dread d9/df/d3d/f5f [0,4194304] 0 2024-02-19T19:06:25.128 INFO:tasks.workunit.client.0.smithi088.stdout:4/408: mkdir d2/d32/d25/d26/d6d/d7e/d3a/d77/d80 0 2024-02-19T19:06:25.131 INFO:tasks.workunit.client.0.smithi088.stdout:4/409: fdatasync d2/d35/f6e 0 2024-02-19T19:06:25.134 INFO:tasks.workunit.client.0.smithi088.stdout:2/412: dwrite d3/d14/f56 [4194304,4194304] 0 2024-02-19T19:06:25.134 INFO:tasks.workunit.client.0.smithi088.stdout:1/613: dwrite d6/d21/d6a/f68 [0,4194304] 0 2024-02-19T19:06:25.135 INFO:tasks.workunit.client.0.smithi088.stdout:7/543: getdents d14/d1d/d3c/d43/db2 0 2024-02-19T19:06:25.140 INFO:tasks.workunit.client.0.smithi088.stdout:3/598: truncate d4/f28 95182 0 2024-02-19T19:06:25.143 INFO:tasks.workunit.client.0.smithi088.stdout:0/466: symlink d13/d62/d8a/d95/d5a/d25/la1 0 2024-02-19T19:06:25.143 INFO:tasks.workunit.client.0.smithi088.stdout:6/522: truncate d5/de/d60/f87 1757187 0 2024-02-19T19:06:25.149 INFO:tasks.workunit.client.0.smithi088.stdout:1/614: creat d6/d10/fc9 x:0 0 0 2024-02-19T19:06:25.152 INFO:tasks.workunit.client.0.smithi088.stdout:8/495: rename d9/df/d3d/d62/c6a to d9/df/d1c/d2d/d47/d6b/d72/d7f/d8e/c93 0 2024-02-19T19:06:25.153 INFO:tasks.workunit.client.0.smithi088.stdout:5/605: link d4/d1a/d41/f7a d4/d1a/d41/dc1/fd0 0 2024-02-19T19:06:25.154 INFO:tasks.workunit.client.0.smithi088.stdout:1/615: dread d6/d21/d6a/d4a/fa7 [0,4194304] 0 2024-02-19T19:06:25.159 INFO:tasks.workunit.client.0.smithi088.stdout:9/543: dread d1/fc [0,4194304] 0 2024-02-19T19:06:25.159 INFO:tasks.workunit.client.0.smithi088.stdout:8/496: symlink d9/df/d1c/d2d/l94 0 2024-02-19T19:06:25.164 INFO:tasks.workunit.client.0.smithi088.stdout:0/467: creat d13/d62/d8a/d95/d5a/fa2 x:0 0 0 2024-02-19T19:06:25.165 INFO:tasks.workunit.client.0.smithi088.stdout:7/544: dread fd [0,4194304] 0 2024-02-19T19:06:25.167 INFO:tasks.workunit.client.0.smithi088.stdout:5/606: creat d4/d1a/fd1 x:0 0 0 2024-02-19T19:06:25.169 INFO:tasks.workunit.client.0.smithi088.stdout:1/616: dwrite d6/d21/d6a/d7d/d9a/fb2 [0,4194304] 0 2024-02-19T19:06:25.169 INFO:tasks.workunit.client.0.smithi088.stdout:3/599: rename d4/d22/d66 to d4/d1a/d1e/d31/dc5/dcf 0 2024-02-19T19:06:25.170 INFO:tasks.workunit.client.0.smithi088.stdout:1/617: readlink d6/d10/l44 0 2024-02-19T19:06:25.172 INFO:tasks.workunit.client.0.smithi088.stdout:4/410: dread d2/f4 [0,4194304] 0 2024-02-19T19:06:25.173 INFO:tasks.workunit.client.0.smithi088.stdout:1/618: stat d6/d21/d64/d94/d71/f98 0 2024-02-19T19:06:25.177 INFO:tasks.workunit.client.0.smithi088.stdout:0/468: read d13/d62/d8a/d95/d5a/d25/f27 [3325586,96664] 0 2024-02-19T19:06:25.177 INFO:tasks.workunit.client.0.smithi088.stdout:1/619: chown d6/d21/d64/d94/d71/f35 43021 1 2024-02-19T19:06:25.179 INFO:tasks.workunit.client.0.smithi088.stdout:1/620: fsync d6/d10/d60/fae 0 2024-02-19T19:06:25.180 INFO:tasks.workunit.client.0.smithi088.stdout:5/607: dwrite d4/d61/d65/d9a/d9f/d57/faf [0,4194304] 0 2024-02-19T19:06:25.183 INFO:tasks.workunit.client.0.smithi088.stdout:2/413: truncate d3/f31 3824847 0 2024-02-19T19:06:25.185 INFO:tasks.workunit.client.0.smithi088.stdout:6/523: truncate d5/de/f22 732916 0 2024-02-19T19:06:25.186 INFO:tasks.workunit.client.0.smithi088.stdout:6/524: dread - d5/d7/d32/f86 zero size 2024-02-19T19:06:25.190 INFO:tasks.workunit.client.0.smithi088.stdout:4/411: rmdir d2/d8 39 2024-02-19T19:06:25.192 INFO:tasks.workunit.client.0.smithi088.stdout:0/469: creat d13/d62/d8a/d95/d5a/d25/d2a/d34/fa3 x:0 0 0 2024-02-19T19:06:25.193 INFO:tasks.workunit.client.0.smithi088.stdout:0/470: dread - d13/f37 zero size 2024-02-19T19:06:25.194 INFO:tasks.workunit.client.0.smithi088.stdout:8/497: dwrite d9/df/f36 [0,4194304] 0 2024-02-19T19:06:25.198 INFO:tasks.workunit.client.0.smithi088.stdout:5/608: unlink d4/d75/f98 0 2024-02-19T19:06:25.201 INFO:tasks.workunit.client.0.smithi088.stdout:7/545: write d14/d1d/d3c/d43/d5c/f7f [473638,104255] 0 2024-02-19T19:06:25.203 INFO:tasks.workunit.client.0.smithi088.stdout:6/525: fsync d5/de/d64/d82/d8c/d25/f39 0 2024-02-19T19:06:25.203 INFO:tasks.workunit.client.0.smithi088.stdout:5/609: read d4/d61/d65/d9a/d9f/d57/d84/f8a [3379169,2160] 0 2024-02-19T19:06:25.204 INFO:tasks.workunit.client.0.smithi088.stdout:4/412: unlink d2/d32/d25/d26/d58/c76 0 2024-02-19T19:06:25.206 INFO:tasks.workunit.client.0.smithi088.stdout:3/600: write d4/d12/d33/d43/fbd [273410,70365] 0 2024-02-19T19:06:25.207 INFO:tasks.workunit.client.0.smithi088.stdout:0/471: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d2f/f4c [0,4194304] 0 2024-02-19T19:06:25.208 INFO:tasks.workunit.client.0.smithi088.stdout:3/601: fsync d4/f79 0 2024-02-19T19:06:25.210 INFO:tasks.workunit.client.0.smithi088.stdout:2/414: creat d3/d1b/d43/d61/d7e/f7f x:0 0 0 2024-02-19T19:06:25.210 INFO:tasks.workunit.client.0.smithi088.stdout:3/602: write d4/fa8 [1738485,66108] 0 2024-02-19T19:06:25.212 INFO:tasks.workunit.client.0.smithi088.stdout:9/544: rename d1/d8/d3e/d93/d6f/l81 to d1/d8/d3e/d93/lae 0 2024-02-19T19:06:25.213 INFO:tasks.workunit.client.0.smithi088.stdout:0/472: readlink d13/d62/d8a/d95/d5a/d25/d2a/d3c/l65 0 2024-02-19T19:06:25.214 INFO:tasks.workunit.client.0.smithi088.stdout:9/545: write d1/f20 [3724748,72753] 0 2024-02-19T19:06:25.215 INFO:tasks.workunit.client.0.smithi088.stdout:2/415: write d3/d1b/d43/f55 [1992890,77003] 0 2024-02-19T19:06:25.215 INFO:tasks.workunit.client.0.smithi088.stdout:3/603: dread - d4/d1a/d1e/d31/d32/d55/fc4 zero size 2024-02-19T19:06:25.217 INFO:tasks.workunit.client.0.smithi088.stdout:2/416: stat d3/d1f/d4f/d59/f79 0 2024-02-19T19:06:25.218 INFO:tasks.workunit.client.0.smithi088.stdout:7/546: truncate d14/d28/f35 570657 0 2024-02-19T19:06:25.219 INFO:tasks.workunit.client.0.smithi088.stdout:9/546: truncate d1/d8/d4a/dac/f6c 4654403 0 2024-02-19T19:06:25.224 INFO:tasks.workunit.client.0.smithi088.stdout:1/621: rename d6/d21/d6a/d7d/d9a/c9d to d6/d21/d6a/d7d/d7e/cca 0 2024-02-19T19:06:25.225 INFO:tasks.workunit.client.0.smithi088.stdout:0/473: write d13/f3e [117659,124280] 0 2024-02-19T19:06:25.226 INFO:tasks.workunit.client.0.smithi088.stdout:0/474: fsync d13/f37 0 2024-02-19T19:06:25.227 INFO:tasks.workunit.client.0.smithi088.stdout:2/417: mknod d3/d14/c80 0 2024-02-19T19:06:25.227 INFO:tasks.workunit.client.0.smithi088.stdout:0/475: dread - d13/d62/d8a/d95/d5a/f9d zero size 2024-02-19T19:06:25.228 INFO:tasks.workunit.client.0.smithi088.stdout:5/610: truncate d4/d1a/f5b 13598 0 2024-02-19T19:06:25.232 INFO:tasks.workunit.client.0.smithi088.stdout:9/547: unlink d1/d8/d3e/f70 0 2024-02-19T19:06:25.235 INFO:tasks.workunit.client.0.smithi088.stdout:5/611: dwrite d4/d1a/d41/d5e/f7f [0,4194304] 0 2024-02-19T19:06:25.237 INFO:tasks.workunit.client.0.smithi088.stdout:8/498: write d9/f75 [1021430,83832] 0 2024-02-19T19:06:25.239 INFO:tasks.workunit.client.0.smithi088.stdout:2/418: chown d3/d1f/l34 0 1 2024-02-19T19:06:25.239 INFO:tasks.workunit.client.0.smithi088.stdout:7/547: mkdir d14/d1d/d3c/d43/dc2 0 2024-02-19T19:06:25.247 INFO:tasks.workunit.client.0.smithi088.stdout:6/526: truncate d5/d8/d2c/f4c 364759 0 2024-02-19T19:06:25.251 INFO:tasks.workunit.client.0.smithi088.stdout:5/612: symlink d4/d61/d65/d9a/ld2 0 2024-02-19T19:06:25.254 INFO:tasks.workunit.client.0.smithi088.stdout:7/548: symlink d14/d1d/d3c/d43/d5c/d9a/d92/lc3 0 2024-02-19T19:06:25.257 INFO:tasks.workunit.client.0.smithi088.stdout:9/548: unlink d1/d8/d37/f41 0 2024-02-19T19:06:25.258 INFO:tasks.workunit.client.0.smithi088.stdout:0/476: truncate fe 1775915 0 2024-02-19T19:06:25.260 INFO:tasks.workunit.client.0.smithi088.stdout:3/604: link d4/d12/d33/d43/l9a d4/d87/dbf/ld0 0 2024-02-19T19:06:25.260 INFO:tasks.workunit.client.0.smithi088.stdout:8/499: dwrite d9/df/f27 [4194304,4194304] 0 2024-02-19T19:06:25.266 INFO:tasks.workunit.client.0.smithi088.stdout:1/622: dwrite d6/d21/d6a/d4a/f59 [0,4194304] 0 2024-02-19T19:06:25.266 INFO:tasks.workunit.client.0.smithi088.stdout:9/549: write d1/d8/d3e/d93/d2d/d23/f25 [1934645,5454] 0 2024-02-19T19:06:25.267 INFO:tasks.workunit.client.0.smithi088.stdout:2/419: dwrite d3/d1b/d43/f52 [0,4194304] 0 2024-02-19T19:06:25.273 INFO:tasks.workunit.client.0.smithi088.stdout:9/550: truncate d1/d8/d4a/d4f/f7a 652404 0 2024-02-19T19:06:25.275 INFO:tasks.workunit.client.0.smithi088.stdout:7/549: dwrite f6 [0,4194304] 0 2024-02-19T19:06:25.277 INFO:tasks.workunit.client.0.smithi088.stdout:9/551: truncate d1/d8/d3e/d93/d2d/d18/d1f/f59 4480396 0 2024-02-19T19:06:25.278 INFO:tasks.workunit.client.0.smithi088.stdout:9/552: stat d1/d8/f5c 0 2024-02-19T19:06:25.308 INFO:tasks.workunit.client.0.smithi088.stdout:9/553: dread d1/d8/d4a/dac/f85 [0,4194304] 0 2024-02-19T19:06:25.309 INFO:tasks.workunit.client.0.smithi088.stdout:9/554: truncate d1/f10 4656009 0 2024-02-19T19:06:25.340 INFO:tasks.workunit.client.0.smithi088.stdout:3/605: rmdir d4/d1a/d1e/d74/d7d 39 2024-02-19T19:06:25.340 INFO:tasks.workunit.client.0.smithi088.stdout:3/606: chown d4/d12/d33/c3e 107 1 2024-02-19T19:06:25.341 INFO:tasks.workunit.client.0.smithi088.stdout:3/607: chown d4/c5b 5713 1 2024-02-19T19:06:25.344 INFO:tasks.workunit.client.0.smithi088.stdout:1/623: fsync d6/d10/d60/d29/f30 0 2024-02-19T19:06:25.345 INFO:tasks.workunit.client.0.smithi088.stdout:5/613: truncate d4/d1a/d41/dc1/fd0 59298 0 2024-02-19T19:06:25.347 INFO:tasks.workunit.client.0.smithi088.stdout:5/614: write d4/f87 [1672127,78627] 0 2024-02-19T19:06:25.348 INFO:tasks.workunit.client.0.smithi088.stdout:7/550: mkdir d14/d27/d6b/dc4 0 2024-02-19T19:06:25.351 INFO:tasks.workunit.client.0.smithi088.stdout:9/555: rmdir d1/d8/d4a 39 2024-02-19T19:06:25.351 INFO:tasks.workunit.client.0.smithi088.stdout:8/500: write d9/df/d1c/f13 [1490435,65203] 0 2024-02-19T19:06:25.352 INFO:tasks.workunit.client.0.smithi088.stdout:3/608: symlink d4/d22/d81/ld1 0 2024-02-19T19:06:25.355 INFO:tasks.workunit.client.0.smithi088.stdout:6/527: getdents d5/d8/d77 0 2024-02-19T19:06:25.358 INFO:tasks.workunit.client.0.smithi088.stdout:5/615: dwrite d4/d61/d65/d9a/fac [0,4194304] 0 2024-02-19T19:06:25.361 INFO:tasks.workunit.client.0.smithi088.stdout:5/616: read d4/d61/d65/d9a/d9f/f4e [722738,58195] 0 2024-02-19T19:06:25.363 INFO:tasks.workunit.client.0.smithi088.stdout:9/556: dwrite d1/f7f [0,4194304] 0 2024-02-19T19:06:25.364 INFO:tasks.workunit.client.0.smithi088.stdout:8/501: write d9/df/d3d/f5f [3282917,76681] 0 2024-02-19T19:06:25.365 INFO:tasks.workunit.client.0.smithi088.stdout:2/420: truncate d3/d1f/f4a 308238 0 2024-02-19T19:06:25.369 INFO:tasks.workunit.client.0.smithi088.stdout:3/609: creat d4/d22/d7a/d8d/fd2 x:0 0 0 2024-02-19T19:06:25.373 INFO:tasks.workunit.client.0.smithi088.stdout:2/421: truncate d3/d1b/d43/d61/d64/f6a 264899 0 2024-02-19T19:06:25.373 INFO:tasks.workunit.client.0.smithi088.stdout:6/528: write d5/d19/f41 [4204635,9675] 0 2024-02-19T19:06:25.379 INFO:tasks.workunit.client.0.smithi088.stdout:3/610: write d4/d12/d33/d58/f78 [310104,97730] 0 2024-02-19T19:06:25.379 INFO:tasks.workunit.client.0.smithi088.stdout:1/624: write d6/f99 [374573,79773] 0 2024-02-19T19:06:25.379 INFO:tasks.workunit.client.0.smithi088.stdout:6/529: read d5/de/d64/d82/d8c/d25/f49 [3816297,70099] 0 2024-02-19T19:06:25.380 INFO:tasks.workunit.client.0.smithi088.stdout:3/611: fsync d4/d22/d7a/fbc 0 2024-02-19T19:06:25.383 INFO:tasks.workunit.client.0.smithi088.stdout:6/530: write d5/d8/d3a/f63 [1013070,12537] 0 2024-02-19T19:06:25.384 INFO:tasks.workunit.client.0.smithi088.stdout:9/557: getdents d1/d8/d3e/d93/d2d/d23/d79/da6 0 2024-02-19T19:06:25.386 INFO:tasks.workunit.client.0.smithi088.stdout:7/551: getdents d14/d27/d49/d98 0 2024-02-19T19:06:25.386 INFO:tasks.workunit.client.0.smithi088.stdout:9/558: readlink d1/d8/d3e/d93/l32 0 2024-02-19T19:06:25.387 INFO:tasks.workunit.client.0.smithi088.stdout:6/531: write d5/d7/d32/d56/f94 [749548,102168] 0 2024-02-19T19:06:25.387 INFO:tasks.workunit.client.0.smithi088.stdout:5/617: write d4/d1a/f3c [4865920,79866] 0 2024-02-19T19:06:25.389 INFO:tasks.workunit.client.0.smithi088.stdout:7/552: chown d14/d1d/d3c/d43/c8d 0 1 2024-02-19T19:06:25.391 INFO:tasks.workunit.client.0.smithi088.stdout:1/625: dread d6/d21/d6a/d4a/f59 [0,4194304] 0 2024-02-19T19:06:25.392 INFO:tasks.workunit.client.0.smithi088.stdout:7/553: mknod d14/d27/cc5 0 2024-02-19T19:06:25.394 INFO:tasks.workunit.client.0.smithi088.stdout:3/612: creat d4/d1a/d1e/d74/d7d/daa/fd3 x:0 0 0 2024-02-19T19:06:25.396 INFO:tasks.workunit.client.0.smithi088.stdout:5/618: creat d4/d1a/d41/dcb/fd3 x:0 0 0 2024-02-19T19:06:25.397 INFO:tasks.workunit.client.0.smithi088.stdout:3/613: creat d4/d12/d33/d43/fd4 x:0 0 0 2024-02-19T19:06:25.399 INFO:tasks.workunit.client.0.smithi088.stdout:1/626: mkdir d6/d21/d6a/dcb 0 2024-02-19T19:06:25.399 INFO:tasks.workunit.client.0.smithi088.stdout:1/627: chown d6/d21/d64/f81 0 1 2024-02-19T19:06:25.402 INFO:tasks.workunit.client.0.smithi088.stdout:7/554: symlink d14/d1d/d3c/lc6 0 2024-02-19T19:06:25.405 INFO:tasks.workunit.client.0.smithi088.stdout:9/559: dwrite d1/d8/d3e/d93/d2d/f16 [4194304,4194304] 0 2024-02-19T19:06:25.405 INFO:tasks.workunit.client.0.smithi088.stdout:6/532: getdents d5/d8/d3a/d71 0 2024-02-19T19:06:25.407 INFO:tasks.workunit.client.0.smithi088.stdout:6/533: write d5/d7/d32/f8e [78345,45301] 0 2024-02-19T19:06:25.408 INFO:tasks.workunit.client.0.smithi088.stdout:3/614: mkdir d4/d22/dd5 0 2024-02-19T19:06:25.411 INFO:tasks.workunit.client.0.smithi088.stdout:1/628: mkdir d6/d21/d64/dc8/dcc 0 2024-02-19T19:06:25.411 INFO:tasks.workunit.client.0.smithi088.stdout:3/615: write d4/f18 [1119621,8840] 0 2024-02-19T19:06:25.414 INFO:tasks.workunit.client.0.smithi088.stdout:5/619: link d4/d1a/f77 d4/d61/d65/fd4 0 2024-02-19T19:06:25.414 INFO:tasks.workunit.client.0.smithi088.stdout:3/616: write d4/d1a/d1e/d31/dc5/dcf/da4/fbb [904744,119697] 0 2024-02-19T19:06:25.416 INFO:tasks.workunit.client.0.smithi088.stdout:9/560: creat d1/faf x:0 0 0 2024-02-19T19:06:25.418 INFO:tasks.workunit.client.0.smithi088.stdout:3/617: creat d4/d1a/d1e/d74/d7d/daa/fd6 x:0 0 0 2024-02-19T19:06:25.423 INFO:tasks.workunit.client.0.smithi088.stdout:9/561: rename d1/d8/d3e/d93/d6f to d1/d8/d3e/d93/d2d/d18/d1f/db0 0 2024-02-19T19:06:25.423 INFO:tasks.workunit.client.0.smithi088.stdout:5/620: dread d4/d61/d65/fd4 [0,4194304] 0 2024-02-19T19:06:25.424 INFO:tasks.workunit.client.0.smithi088.stdout:5/621: fdatasync d4/d1a/d41/fcc 0 2024-02-19T19:06:25.425 INFO:tasks.workunit.client.0.smithi088.stdout:3/618: creat d4/d1a/d7f/fd7 x:0 0 0 2024-02-19T19:06:25.425 INFO:tasks.workunit.client.0.smithi088.stdout:9/562: chown d1/d8/d3e/d93/d2f/fa4 0 1 2024-02-19T19:06:25.427 INFO:tasks.workunit.client.0.smithi088.stdout:7/555: dread d14/d27/f32 [0,4194304] 0 2024-02-19T19:06:25.427 INFO:tasks.workunit.client.0.smithi088.stdout:5/622: dread - d4/d61/d65/f81 zero size 2024-02-19T19:06:25.429 INFO:tasks.workunit.client.0.smithi088.stdout:7/556: readlink d14/d1d/d3c/d43/d5c/d9a/d92/lc3 0 2024-02-19T19:06:25.441 INFO:tasks.workunit.client.0.smithi088.stdout:1/629: dread d6/d10/d60/d29/f30 [0,4194304] 0 2024-02-19T19:06:25.448 INFO:tasks.workunit.client.0.smithi088.stdout:9/563: dwrite d1/d8/f5c [0,4194304] 0 2024-02-19T19:06:25.450 INFO:tasks.workunit.client.0.smithi088.stdout:7/557: dwrite d14/d1d/f5e [0,4194304] 0 2024-02-19T19:06:25.452 INFO:tasks.workunit.client.0.smithi088.stdout:3/619: dwrite d4/d1a/d1e/d31/d32/f50 [0,4194304] 0 2024-02-19T19:06:25.452 INFO:tasks.workunit.client.0.smithi088.stdout:1/630: creat d6/d21/d64/d94/d71/fcd x:0 0 0 2024-02-19T19:06:25.463 INFO:tasks.workunit.client.0.smithi088.stdout:9/564: creat d1/d8/d3e/d93/d2d/d18/d1f/fb1 x:0 0 0 2024-02-19T19:06:25.464 INFO:tasks.workunit.client.0.smithi088.stdout:5/623: link d4/d7/l27 d4/d61/ld5 0 2024-02-19T19:06:25.469 INFO:tasks.workunit.client.0.smithi088.stdout:1/631: dwrite d6/d21/f84 [0,4194304] 0 2024-02-19T19:06:25.471 INFO:tasks.workunit.client.0.smithi088.stdout:5/624: chown d4/d61/d65/d9a/d9f/l14 3368 1 2024-02-19T19:06:25.471 INFO:tasks.workunit.client.0.smithi088.stdout:3/620: rename d4/l63 to d4/ld8 0 2024-02-19T19:06:25.472 INFO:tasks.workunit.client.0.smithi088.stdout:1/632: mknod d6/cce 0 2024-02-19T19:06:25.474 INFO:tasks.workunit.client.0.smithi088.stdout:5/625: creat d4/d61/d65/d9a/fd6 x:0 0 0 2024-02-19T19:06:25.478 INFO:tasks.workunit.client.0.smithi088.stdout:3/621: rename d4/d22/d7a/d8d/fce to d4/d12/d33/d43/fd9 0 2024-02-19T19:06:25.480 INFO:tasks.workunit.client.0.smithi088.stdout:5/626: creat d4/d1a/d41/dc1/dc2/fd7 x:0 0 0 2024-02-19T19:06:25.482 INFO:tasks.workunit.client.0.smithi088.stdout:1/633: rename d6/d10/d60/d29/d63 to d6/d10/d60/db8/dcf 0 2024-02-19T19:06:25.483 INFO:tasks.workunit.client.0.smithi088.stdout:3/622: symlink d4/d12/d33/d58/lda 0 2024-02-19T19:06:25.491 INFO:tasks.workunit.client.0.smithi088.stdout:3/623: mknod d4/d1a/d1e/d74/d7d/daa/dcb/cdb 0 2024-02-19T19:06:25.494 INFO:tasks.workunit.client.0.smithi088.stdout:9/565: dread d1/d8/f42 [0,4194304] 0 2024-02-19T19:06:25.499 INFO:tasks.workunit.client.0.smithi088.stdout:3/624: getdents d4 0 2024-02-19T19:06:25.499 INFO:tasks.workunit.client.0.smithi088.stdout:9/566: mkdir d1/d8/d37/db2 0 2024-02-19T19:06:25.503 INFO:tasks.workunit.client.0.smithi088.stdout:1/634: dread d6/d10/f5d [0,4194304] 0 2024-02-19T19:06:25.503 INFO:tasks.workunit.client.0.smithi088.stdout:5/627: dwrite d4/d1a/d41/f7a [0,4194304] 0 2024-02-19T19:06:25.504 INFO:tasks.workunit.client.0.smithi088.stdout:3/625: unlink d4/d12/d33/fa2 0 2024-02-19T19:06:25.504 INFO:tasks.workunit.client.0.smithi088.stdout:9/567: write d1/d8/d3e/d93/d2d/d18/d1f/fb1 [914515,67802] 0 2024-02-19T19:06:25.506 INFO:tasks.workunit.client.0.smithi088.stdout:5/628: chown d4/f12 4147265 1 2024-02-19T19:06:25.508 INFO:tasks.workunit.client.0.smithi088.stdout:1/635: write d6/f1b [326448,50882] 0 2024-02-19T19:06:25.510 INFO:tasks.workunit.client.0.smithi088.stdout:5/629: chown d4/d1a/f21 19683 1 2024-02-19T19:06:25.510 INFO:tasks.workunit.client.0.smithi088.stdout:1/636: chown d6/d10/l74 31661874 1 2024-02-19T19:06:25.513 INFO:tasks.workunit.client.0.smithi088.stdout:1/637: write d6/d21/d64/d94/d71/f7a [5129936,44716] 0 2024-02-19T19:06:25.513 INFO:tasks.workunit.client.0.smithi088.stdout:3/626: dread d4/d12/f2e [0,4194304] 0 2024-02-19T19:06:25.514 INFO:tasks.workunit.client.0.smithi088.stdout:9/568: dread d1/d8/d3e/d93/d2d/d18/f8f [4194304,4194304] 0 2024-02-19T19:06:25.514 INFO:tasks.workunit.client.0.smithi088.stdout:5/630: symlink d4/d61/ld8 0 2024-02-19T19:06:25.515 INFO:tasks.workunit.client.0.smithi088.stdout:1/638: fsync d6/d21/f6c 0 2024-02-19T19:06:25.518 INFO:tasks.workunit.client.0.smithi088.stdout:5/631: unlink d4/d61/d65/d9a/d9f/lb2 0 2024-02-19T19:06:25.520 INFO:tasks.workunit.client.0.smithi088.stdout:3/627: mknod d4/d22/dd5/cdc 0 2024-02-19T19:06:25.522 INFO:tasks.workunit.client.0.smithi088.stdout:9/569: dwrite d1/d8/d37/d54/f65 [0,4194304] 0 2024-02-19T19:06:25.523 INFO:tasks.workunit.client.0.smithi088.stdout:5/632: mknod d4/cd9 0 2024-02-19T19:06:25.524 INFO:tasks.workunit.client.0.smithi088.stdout:3/628: mkdir d4/d12/d33/d43/ddd 0 2024-02-19T19:06:25.527 INFO:tasks.workunit.client.0.smithi088.stdout:5/633: mknod d4/cda 0 2024-02-19T19:06:25.528 INFO:tasks.workunit.client.0.smithi088.stdout:3/629: creat d4/d1a/d1e/fde x:0 0 0 2024-02-19T19:06:25.529 INFO:tasks.workunit.client.0.smithi088.stdout:5/634: read - d4/d61/d65/d9a/d9f/da0/fae zero size 2024-02-19T19:06:25.531 INFO:tasks.workunit.client.0.smithi088.stdout:3/630: creat d4/d12/d33/dab/fdf x:0 0 0 2024-02-19T19:06:25.532 INFO:tasks.workunit.client.0.smithi088.stdout:9/570: getdents d1/d8/d3e/d93/d2d/d23 0 2024-02-19T19:06:25.533 INFO:tasks.workunit.client.0.smithi088.stdout:5/635: mknod d4/d1a/d41/dcb/cdb 0 2024-02-19T19:06:25.533 INFO:tasks.workunit.client.0.smithi088.stdout:3/631: write d4/d87/fb6 [2663900,56974] 0 2024-02-19T19:06:25.535 INFO:tasks.workunit.client.0.smithi088.stdout:5/636: fdatasync d4/f9d 0 2024-02-19T19:06:25.535 INFO:tasks.workunit.client.0.smithi088.stdout:9/571: fdatasync d1/d8/d3e/d93/d2d/d18/d1f/fb1 0 2024-02-19T19:06:25.536 INFO:tasks.workunit.client.0.smithi088.stdout:9/572: fsync d1/d8/f42 0 2024-02-19T19:06:25.539 INFO:tasks.workunit.client.0.smithi088.stdout:9/573: fsync d1/d8/d37/d54/f55 0 2024-02-19T19:06:25.543 INFO:tasks.workunit.client.0.smithi088.stdout:9/574: chown d1/d8/d3e/d93/d2d/d18/d1f/db0/l95 125006 1 2024-02-19T19:06:25.544 INFO:tasks.workunit.client.0.smithi088.stdout:3/632: dwrite d4/d1a/d7f/d88/fcd [0,4194304] 0 2024-02-19T19:06:25.545 INFO:tasks.workunit.client.0.smithi088.stdout:9/575: stat d1/d8/d3e/d93/d2d/d18/f8f 0 2024-02-19T19:06:25.549 INFO:tasks.workunit.client.0.smithi088.stdout:9/576: chown d1/d8/d3e/d93/d2d/d23/d79/f9d 3105408 1 2024-02-19T19:06:25.553 INFO:tasks.workunit.client.0.smithi088.stdout:3/633: dwrite d4/d12/f47 [0,4194304] 0 2024-02-19T19:06:25.558 INFO:tasks.workunit.client.0.smithi088.stdout:3/634: mknod d4/d1a/d23/dc9/ce0 0 2024-02-19T19:06:25.558 INFO:tasks.workunit.client.0.smithi088.stdout:3/635: chown d4/d22/d81 7 1 2024-02-19T19:06:25.559 INFO:tasks.workunit.client.0.smithi088.stdout:9/577: rmdir d1/d8/d37/db2 0 2024-02-19T19:06:25.563 INFO:tasks.workunit.client.0.smithi088.stdout:9/578: rename d1/d8/d3e/d93/l7e to d1/d8/d8d/d97/lb3 0 2024-02-19T19:06:25.566 INFO:tasks.workunit.client.0.smithi088.stdout:5/637: dread d4/d1a/f7c [0,4194304] 0 2024-02-19T19:06:25.569 INFO:tasks.workunit.client.0.smithi088.stdout:5/638: creat d4/d61/d65/d9a/d9f/fdc x:0 0 0 2024-02-19T19:06:25.572 INFO:tasks.workunit.client.0.smithi088.stdout:5/639: write d4/d61/d65/d9a/d9f/d57/faf [96776,129436] 0 2024-02-19T19:06:25.573 INFO:tasks.workunit.client.0.smithi088.stdout:4/413: sync 2024-02-19T19:06:25.574 INFO:tasks.workunit.client.0.smithi088.stdout:9/579: dwrite d1/d8/d4a/d4f/f7a [0,4194304] 0 2024-02-19T19:06:25.577 INFO:tasks.workunit.client.0.smithi088.stdout:5/640: creat d4/d1a/fdd x:0 0 0 2024-02-19T19:06:25.579 INFO:tasks.workunit.client.0.smithi088.stdout:4/414: mknod d2/d32/d25/d26/d4e/c81 0 2024-02-19T19:06:25.581 INFO:tasks.workunit.client.0.smithi088.stdout:4/415: write d2/d32/d25/d26/d6d/d7e/d3a/f5f [307845,44036] 0 2024-02-19T19:06:25.582 INFO:tasks.workunit.client.0.smithi088.stdout:9/580: rename d1/d8/d3e/f45 to d1/d8/d3e/d93/d2d/d18/d1f/fb4 0 2024-02-19T19:06:25.588 INFO:tasks.workunit.client.0.smithi088.stdout:5/641: rename d4/d61/d65/d9a/d9f/d57/l7d to d4/d1a/d41/d5e/lde 0 2024-02-19T19:06:25.588 INFO:tasks.workunit.client.0.smithi088.stdout:3/636: dread d4/f7 [0,4194304] 0 2024-02-19T19:06:25.589 INFO:tasks.workunit.client.0.smithi088.stdout:9/581: creat d1/d8/d3e/d93/d2f/fb5 x:0 0 0 2024-02-19T19:06:25.590 INFO:tasks.workunit.client.0.smithi088.stdout:3/637: write d4/fa8 [2116749,90995] 0 2024-02-19T19:06:25.591 INFO:tasks.workunit.client.0.smithi088.stdout:4/416: dwrite d2/d32/d25/d26/d6d/d7e/d3a/f5f [0,4194304] 0 2024-02-19T19:06:25.593 INFO:tasks.workunit.client.0.smithi088.stdout:9/582: dread - d1/d8/d3e/d93/d2f/f7b zero size 2024-02-19T19:06:25.594 INFO:tasks.workunit.client.0.smithi088.stdout:9/583: chown d1/fc 428844 1 2024-02-19T19:06:25.597 INFO:tasks.workunit.client.0.smithi088.stdout:4/417: rename d2/d32/c2b to d2/d32/d25/d26/d4e/c82 0 2024-02-19T19:06:25.597 INFO:tasks.workunit.client.0.smithi088.stdout:3/638: getdents d4/d1a 0 2024-02-19T19:06:25.599 INFO:tasks.workunit.client.0.smithi088.stdout:3/639: read d4/f6b [201873,76537] 0 2024-02-19T19:06:25.600 INFO:tasks.workunit.client.0.smithi088.stdout:9/584: mkdir d1/d8/d3e/d93/d2d/d18/db6 0 2024-02-19T19:06:25.601 INFO:tasks.workunit.client.0.smithi088.stdout:4/418: symlink d2/d32/d25/d4b/l83 0 2024-02-19T19:06:25.605 INFO:tasks.workunit.client.0.smithi088.stdout:9/585: creat d1/d8/d4a/fb7 x:0 0 0 2024-02-19T19:06:25.606 INFO:tasks.workunit.client.0.smithi088.stdout:5/642: dread d4/f42 [0,4194304] 0 2024-02-19T19:06:25.607 INFO:tasks.workunit.client.0.smithi088.stdout:3/640: rename d4/c5b to d4/d1a/d1e/d31/d32/d55/ce1 0 2024-02-19T19:06:25.609 INFO:tasks.workunit.client.0.smithi088.stdout:4/419: creat d2/d32/d25/d26/d6d/d7e/d3a/d77/f84 x:0 0 0 2024-02-19T19:06:25.609 INFO:tasks.workunit.client.0.smithi088.stdout:5/643: write d4/fe [724285,17496] 0 2024-02-19T19:06:25.610 INFO:tasks.workunit.client.0.smithi088.stdout:9/586: mknod d1/d8/d3e/d93/d2f/cb8 0 2024-02-19T19:06:25.612 INFO:tasks.workunit.client.0.smithi088.stdout:4/420: write d2/d35/f53 [2101926,48848] 0 2024-02-19T19:06:25.614 INFO:tasks.workunit.client.0.smithi088.stdout:9/587: symlink d1/d8/d4a/d4f/daa/lb9 0 2024-02-19T19:06:25.615 INFO:tasks.workunit.client.0.smithi088.stdout:5/644: truncate d4/d61/d65/d9a/d9f/d57/f6b 2937981 0 2024-02-19T19:06:25.616 INFO:tasks.workunit.client.0.smithi088.stdout:5/645: stat d4/d61/d65/d9a/d9f/d57/d84 0 2024-02-19T19:06:25.617 INFO:tasks.workunit.client.0.smithi088.stdout:4/421: unlink d2/d32/d25/d4b/l83 0 2024-02-19T19:06:25.618 INFO:tasks.workunit.client.0.smithi088.stdout:9/588: fsync d1/d8/f28 0 2024-02-19T19:06:25.621 INFO:tasks.workunit.client.0.smithi088.stdout:4/422: dread d2/f4 [0,4194304] 0 2024-02-19T19:06:25.622 INFO:tasks.workunit.client.0.smithi088.stdout:9/589: creat d1/d8/d4a/fba x:0 0 0 2024-02-19T19:06:25.623 INFO:tasks.workunit.client.0.smithi088.stdout:9/590: readlink d1/d8/d3e/d93/d2d/d23/d79/l87 0 2024-02-19T19:06:25.624 INFO:tasks.workunit.client.0.smithi088.stdout:5/646: truncate d4/d75/f80 611154 0 2024-02-19T19:06:25.627 INFO:tasks.workunit.client.0.smithi088.stdout:5/647: unlink d4/d1a/d41/d5e/f7f 0 2024-02-19T19:06:25.628 INFO:tasks.workunit.client.0.smithi088.stdout:5/648: stat d4/d75/c79 0 2024-02-19T19:06:25.632 INFO:tasks.workunit.client.0.smithi088.stdout:5/649: rmdir d4/d7 39 2024-02-19T19:06:25.636 INFO:tasks.workunit.client.0.smithi088.stdout:5/650: chown d4/d61/d65/d9a/d9f/d57/d84/cbb 212975 1 2024-02-19T19:06:25.637 INFO:tasks.workunit.client.0.smithi088.stdout:4/423: dread d2/d32/d25/f41 [12582912,4194304] 0 2024-02-19T19:06:25.638 INFO:tasks.workunit.client.0.smithi088.stdout:5/651: dread - d4/d1a/d41/dc1/dc2/fc6 zero size 2024-02-19T19:06:25.646 INFO:tasks.workunit.client.0.smithi088.stdout:9/591: truncate d1/d8/d3e/d93/d2d/d18/d1f/f59 2986873 0 2024-02-19T19:06:25.651 INFO:tasks.workunit.client.0.smithi088.stdout:3/641: rename d4/d12/c4a to d4/d1a/d7f/ce2 0 2024-02-19T19:06:25.654 INFO:tasks.workunit.client.0.smithi088.stdout:5/652: dread d4/d61/d65/d9a/d9f/f24 [0,4194304] 0 2024-02-19T19:06:25.656 INFO:tasks.workunit.client.0.smithi088.stdout:3/642: chown d4/d1a/d1e/d31/d32/c3a 0 1 2024-02-19T19:06:25.657 INFO:tasks.workunit.client.0.smithi088.stdout:9/592: dwrite d1/fc [0,4194304] 0 2024-02-19T19:06:25.658 INFO:tasks.workunit.client.0.smithi088.stdout:4/424: rmdir d2/d32/d25/d26/d6d/d7e/d3a/d77/d80 0 2024-02-19T19:06:25.661 INFO:tasks.workunit.client.0.smithi088.stdout:5/653: mknod d4/d61/d65/d9a/cdf 0 2024-02-19T19:06:25.662 INFO:tasks.workunit.client.0.smithi088.stdout:3/643: creat d4/d1a/d1e/d74/d7d/fe3 x:0 0 0 2024-02-19T19:06:25.663 INFO:tasks.workunit.client.0.smithi088.stdout:9/593: mkdir d1/d8/d4a/d4f/daa/dbb 0 2024-02-19T19:06:25.664 INFO:tasks.workunit.client.0.smithi088.stdout:9/594: chown d1/d8/d3e/d93/d2d/d23/d79/d99 30774149 1 2024-02-19T19:06:25.666 INFO:tasks.workunit.client.0.smithi088.stdout:5/654: creat d4/d61/d65/d9a/d9f/d57/fe0 x:0 0 0 2024-02-19T19:06:25.667 INFO:tasks.workunit.client.0.smithi088.stdout:4/425: mknod d2/d32/c85 0 2024-02-19T19:06:25.667 INFO:tasks.workunit.client.0.smithi088.stdout:5/655: chown d4/c4a 103 1 2024-02-19T19:06:25.668 INFO:tasks.workunit.client.0.smithi088.stdout:3/644: creat d4/d12/d33/fe4 x:0 0 0 2024-02-19T19:06:25.669 INFO:tasks.workunit.client.0.smithi088.stdout:5/656: creat d4/d1a/d41/dc1/dc2/fe1 x:0 0 0 2024-02-19T19:06:25.670 INFO:tasks.workunit.client.0.smithi088.stdout:4/426: mkdir d2/d32/d25/d26/d6d/d86 0 2024-02-19T19:06:25.671 INFO:tasks.workunit.client.0.smithi088.stdout:9/595: symlink d1/d8/d3e/d93/d2d/d18/d92/lbc 0 2024-02-19T19:06:25.672 INFO:tasks.workunit.client.0.smithi088.stdout:3/645: chown d4/f6a 12913 1 2024-02-19T19:06:25.672 INFO:tasks.workunit.client.0.smithi088.stdout:9/596: write d1/f10 [229358,127483] 0 2024-02-19T19:06:25.673 INFO:tasks.workunit.client.0.smithi088.stdout:5/657: mkdir d4/d1a/d41/de2 0 2024-02-19T19:06:25.674 INFO:tasks.workunit.client.0.smithi088.stdout:9/597: stat d1/d8/d3e/d93/d5a/d94 0 2024-02-19T19:06:25.676 INFO:tasks.workunit.client.0.smithi088.stdout:4/427: symlink d2/d32/d25/d26/d4e/d6f/l87 0 2024-02-19T19:06:25.676 INFO:tasks.workunit.client.0.smithi088.stdout:9/598: chown d1/d8/d3e/d93/d2d/d18/d1f/l89 61195448 1 2024-02-19T19:06:25.677 INFO:tasks.workunit.client.0.smithi088.stdout:5/658: creat d4/d1a/d41/dc1/dc2/fe3 x:0 0 0 2024-02-19T19:06:25.678 INFO:tasks.workunit.client.0.smithi088.stdout:3/646: mknod d4/d22/d7a/d8d/ce5 0 2024-02-19T19:06:25.680 INFO:tasks.workunit.client.0.smithi088.stdout:3/647: dread - d4/d22/d7a/fa7 zero size 2024-02-19T19:06:25.681 INFO:tasks.workunit.client.0.smithi088.stdout:5/659: rmdir d4/d61/d65/d9a/d9f/d57/d84 39 2024-02-19T19:06:25.682 INFO:tasks.workunit.client.0.smithi088.stdout:5/660: readlink d4/lba 0 2024-02-19T19:06:25.683 INFO:tasks.workunit.client.0.smithi088.stdout:3/648: chown d4/d22/d7a/fbc 217374 1 2024-02-19T19:06:25.685 INFO:tasks.workunit.client.0.smithi088.stdout:9/599: dwrite d1/d8/d4a/d4f/fad [0,4194304] 0 2024-02-19T19:06:25.694 INFO:tasks.workunit.client.0.smithi088.stdout:4/428: dread d2/d32/f44 [0,4194304] 0 2024-02-19T19:06:25.696 INFO:tasks.workunit.client.0.smithi088.stdout:5/661: truncate d4/d61/d65/d9a/d9f/f26 2316357 0 2024-02-19T19:06:25.697 INFO:tasks.workunit.client.0.smithi088.stdout:3/649: dwrite d4/d1a/f24 [0,4194304] 0 2024-02-19T19:06:25.697 INFO:tasks.workunit.client.0.smithi088.stdout:9/600: dwrite d1/d8/d3e/d93/fa1 [0,4194304] 0 2024-02-19T19:06:25.699 INFO:tasks.workunit.client.0.smithi088.stdout:4/429: fdatasync d2/f15 0 2024-02-19T19:06:25.700 INFO:tasks.workunit.client.0.smithi088.stdout:5/662: symlink d4/d61/d65/d9a/d9f/da0/le4 0 2024-02-19T19:06:25.704 INFO:tasks.workunit.client.0.smithi088.stdout:3/650: rmdir d4/d1a/d1e/d74/d7c 39 2024-02-19T19:06:25.706 INFO:tasks.workunit.client.0.smithi088.stdout:9/601: symlink d1/d8/d4a/d4f/daa/dbb/lbd 0 2024-02-19T19:06:25.710 INFO:tasks.workunit.client.0.smithi088.stdout:3/651: unlink d4/d1a/d23/f60 0 2024-02-19T19:06:25.715 INFO:tasks.workunit.client.0.smithi088.stdout:4/430: dread d2/d35/f37 [0,4194304] 0 2024-02-19T19:06:25.717 INFO:tasks.workunit.client.0.smithi088.stdout:5/663: getdents d4/d1a/d41/d5e 0 2024-02-19T19:06:25.719 INFO:tasks.workunit.client.0.smithi088.stdout:9/602: link d1/d8/d37/l40 d1/d8/d8d/lbe 0 2024-02-19T19:06:25.721 INFO:tasks.workunit.client.0.smithi088.stdout:4/431: creat d2/d32/d25/d26/f88 x:0 0 0 2024-02-19T19:06:25.724 INFO:tasks.workunit.client.0.smithi088.stdout:5/664: symlink d4/d61/d65/d9a/d9f/d57/le5 0 2024-02-19T19:06:25.726 INFO:tasks.workunit.client.0.smithi088.stdout:4/432: symlink d2/l89 0 2024-02-19T19:06:25.728 INFO:tasks.workunit.client.0.smithi088.stdout:4/433: symlink d2/d32/d25/d26/d6d/d7e/d3a/l8a 0 2024-02-19T19:06:25.733 INFO:tasks.workunit.client.0.smithi088.stdout:5/665: rename d4/d61/d65/d9a/d9f/f26 to d4/d1a/d41/fe6 0 2024-02-19T19:06:25.736 INFO:tasks.workunit.client.0.smithi088.stdout:9/603: dread d1/d8/d3e/d93/f64 [0,4194304] 0 2024-02-19T19:06:25.736 INFO:tasks.workunit.client.0.smithi088.stdout:4/434: rename d2/d8/f61 to d2/d32/d25/d3f/d70/f8b 0 2024-02-19T19:06:25.738 INFO:tasks.workunit.client.0.smithi088.stdout:9/604: dread - d1/d8/d3e/d93/d2f/fb5 zero size 2024-02-19T19:06:25.739 INFO:tasks.workunit.client.0.smithi088.stdout:5/666: mkdir d4/d1a/d41/de2/de7 0 2024-02-19T19:06:25.740 INFO:tasks.workunit.client.0.smithi088.stdout:4/435: creat d2/d32/d25/d3f/d70/f8c x:0 0 0 2024-02-19T19:06:25.740 INFO:tasks.workunit.client.0.smithi088.stdout:5/667: write d4/d61/d65/d9a/fd6 [369779,67067] 0 2024-02-19T19:06:25.746 INFO:tasks.workunit.client.0.smithi088.stdout:5/668: mkdir d4/d75/de8 0 2024-02-19T19:06:25.749 INFO:tasks.workunit.client.0.smithi088.stdout:9/605: rename d1/d8/d3e/d93/d2d/d23/d79/l87 to d1/lbf 0 2024-02-19T19:06:25.752 INFO:tasks.workunit.client.0.smithi088.stdout:5/669: unlink d4/d1a/c70 0 2024-02-19T19:06:25.752 INFO:tasks.workunit.client.0.smithi088.stdout:9/606: read d1/d8/d3e/d93/d2d/f47 [2890059,112261] 0 2024-02-19T19:06:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:25 smithi132 ceph-mon[70676]: pgmap v46: 65 pgs: 65 active+clean; 946 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 21 MiB/s rd, 108 MiB/s wr, 305 op/s 2024-02-19T19:06:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:06:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:06:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:06:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi088"}]: dispatch 2024-02-19T19:06:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi088"}]': finished 2024-02-19T19:06:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi132"}]: dispatch 2024-02-19T19:06:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi132"}]': finished 2024-02-19T19:06:25.754 INFO:tasks.workunit.client.0.smithi088.stdout:4/436: dwrite d2/d32/f44 [0,4194304] 0 2024-02-19T19:06:25.754 INFO:tasks.workunit.client.0.smithi088.stdout:5/670: chown d4/d61/d65/d9a/d9f/d57/le5 4 1 2024-02-19T19:06:25.756 INFO:tasks.workunit.client.0.smithi088.stdout:4/437: truncate d2/d32/d25/d26/f5e 737219 0 2024-02-19T19:06:25.760 INFO:tasks.workunit.client.0.smithi088.stdout:9/607: getdents d1/d8/d3e/d93/d5a 0 2024-02-19T19:06:25.763 INFO:tasks.workunit.client.0.smithi088.stdout:4/438: creat d2/f8d x:0 0 0 2024-02-19T19:06:25.767 INFO:tasks.workunit.client.0.smithi088.stdout:5/671: write d4/d1a/f6e [414302,26158] 0 2024-02-19T19:06:25.769 INFO:tasks.workunit.client.0.smithi088.stdout:4/439: write d2/d8/f14 [1465265,67669] 0 2024-02-19T19:06:25.772 INFO:tasks.workunit.client.0.smithi088.stdout:4/440: symlink d2/d32/d25/d4b/l8e 0 2024-02-19T19:06:25.773 INFO:tasks.workunit.client.0.smithi088.stdout:9/608: dwrite d1/d8/f28 [0,4194304] 0 2024-02-19T19:06:25.775 INFO:tasks.workunit.client.0.smithi088.stdout:4/441: creat d2/d32/d25/f8f x:0 0 0 2024-02-19T19:06:25.775 INFO:tasks.workunit.client.0.smithi088.stdout:9/609: rmdir d1/d8/d4a/dac 39 2024-02-19T19:06:25.778 INFO:tasks.workunit.client.0.smithi088.stdout:9/610: mkdir d1/d8/d3e/d93/d2d/d18/dc0 0 2024-02-19T19:06:25.783 INFO:tasks.workunit.client.0.smithi088.stdout:4/442: dwrite d2/d32/d9/f20 [0,4194304] 0 2024-02-19T19:06:25.785 INFO:tasks.workunit.client.0.smithi088.stdout:9/611: rename d1/d8/d3e/d93/d2f/cb8 to d1/d8/cc1 0 2024-02-19T19:06:25.785 INFO:tasks.workunit.client.0.smithi088.stdout:4/443: rmdir d2/d32/d25/d3f 39 2024-02-19T19:06:25.787 INFO:tasks.workunit.client.0.smithi088.stdout:4/444: write d2/d32/f1c [7842405,83948] 0 2024-02-19T19:06:25.788 INFO:tasks.workunit.client.0.smithi088.stdout:9/612: creat d1/d8/d4a/d4f/fc2 x:0 0 0 2024-02-19T19:06:25.788 INFO:tasks.workunit.client.0.smithi088.stdout:9/613: readlink d1/d8/d3e/l69 0 2024-02-19T19:06:25.789 INFO:tasks.workunit.client.0.smithi088.stdout:9/614: readlink d1/l7c 0 2024-02-19T19:06:25.792 INFO:tasks.workunit.client.0.smithi088.stdout:9/615: stat d1/d8/d3e/d93/d2d/d23/f35 0 2024-02-19T19:06:25.794 INFO:tasks.workunit.client.0.smithi088.stdout:9/616: dread - d1/d8/d4a/fb7 zero size 2024-02-19T19:06:25.795 INFO:tasks.workunit.client.0.smithi088.stdout:4/445: dwrite d2/d32/d9/f20 [0,4194304] 0 2024-02-19T19:06:25.798 INFO:tasks.workunit.client.0.smithi088.stdout:9/617: mkdir d1/d8/d3e/d93/d2d/d23/d79/dc3 0 2024-02-19T19:06:25.799 INFO:tasks.workunit.client.0.smithi088.stdout:4/446: mkdir d2/d32/d25/d26/d6d/d7e/d90 0 2024-02-19T19:06:25.802 INFO:tasks.workunit.client.0.smithi088.stdout:4/447: dread d2/f4 [0,4194304] 0 2024-02-19T19:06:25.807 INFO:tasks.workunit.client.0.smithi088.stdout:9/618: rmdir d1/d8/d3e 39 2024-02-19T19:06:25.820 INFO:tasks.workunit.client.0.smithi088.stdout:4/448: dread d2/d32/d9/f71 [0,4194304] 0 2024-02-19T19:06:25.826 INFO:tasks.workunit.client.0.smithi088.stdout:4/449: dwrite d2/d32/d25/d26/d58/f69 [0,4194304] 0 2024-02-19T19:06:25.828 INFO:tasks.workunit.client.0.smithi088.stdout:4/450: chown d2/d32/d25/d26/d4e/d6f/l87 502785364 1 2024-02-19T19:06:25.829 INFO:tasks.workunit.client.0.smithi088.stdout:4/451: rmdir d2/d32/d25/d26/d6d 39 2024-02-19T19:06:25.831 INFO:tasks.workunit.client.0.smithi088.stdout:4/452: rename d2/d32/d25/d26/d6d to d2/d32/d25/d26/d6d/d7e/d90/d91 22 2024-02-19T19:06:25.832 INFO:tasks.workunit.client.0.smithi088.stdout:4/453: chown d2/d32/d25/l2d 2215885 1 2024-02-19T19:06:25.835 INFO:tasks.workunit.client.0.smithi088.stdout:4/454: getdents d2/d32/d25 0 2024-02-19T19:06:25.836 INFO:tasks.workunit.client.0.smithi088.stdout:4/455: stat d2/d32/d25/d26/d6d/d7e/f72 0 2024-02-19T19:06:25.838 INFO:tasks.workunit.client.0.smithi088.stdout:4/456: creat d2/d32/d25/d26/d6d/d7e/d3a/d77/f92 x:0 0 0 2024-02-19T19:06:25.839 INFO:tasks.workunit.client.0.smithi088.stdout:4/457: stat d2/d32/d25/d3f/d70 0 2024-02-19T19:06:25.840 INFO:tasks.workunit.client.0.smithi088.stdout:4/458: chown d2/d32/d9/c10 880668687 1 2024-02-19T19:06:25.843 INFO:tasks.workunit.client.0.smithi088.stdout:4/459: link d2/d32/d25/d45/l66 d2/d32/d25/d26/d4e/d79/l93 0 2024-02-19T19:06:25.846 INFO:tasks.workunit.client.0.smithi088.stdout:4/460: rename d2/d32/d9/f56 to d2/d32/d25/d26/d6d/d7e/d90/f94 0 2024-02-19T19:06:25.849 INFO:tasks.workunit.client.0.smithi088.stdout:4/461: rename d2/l89 to d2/d8/l95 0 2024-02-19T19:06:25.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:25 smithi088 ceph-mon[88643]: pgmap v46: 65 pgs: 65 active+clean; 946 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 21 MiB/s rd, 108 MiB/s wr, 305 op/s 2024-02-19T19:06:25.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:25.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:25.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:06:25.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:06:25.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:06:25.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:25.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi088"}]: dispatch 2024-02-19T19:06:25.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi088"}]': finished 2024-02-19T19:06:25.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi132"}]: dispatch 2024-02-19T19:06:25.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi132"}]': finished 2024-02-19T19:06:25.851 INFO:tasks.workunit.client.0.smithi088.stdout:4/462: mknod d2/d32/d25/d26/d6d/d7e/d3a/d77/c96 0 2024-02-19T19:06:25.855 INFO:tasks.workunit.client.0.smithi088.stdout:4/463: getdents d2/d32/d25/d45 0 2024-02-19T19:06:25.903 INFO:tasks.workunit.client.0.smithi088.stdout:0/477: sync 2024-02-19T19:06:25.903 INFO:tasks.workunit.client.0.smithi088.stdout:8/502: sync 2024-02-19T19:06:25.903 INFO:tasks.workunit.client.0.smithi088.stdout:6/534: sync 2024-02-19T19:06:25.903 INFO:tasks.workunit.client.0.smithi088.stdout:2/422: sync 2024-02-19T19:06:25.904 INFO:tasks.workunit.client.0.smithi088.stdout:2/423: chown d3/d1b/l6b 1 1 2024-02-19T19:06:25.905 INFO:tasks.workunit.client.0.smithi088.stdout:8/503: mknod d9/df/d1c/c95 0 2024-02-19T19:06:25.905 INFO:tasks.workunit.client.0.smithi088.stdout:2/424: dread - d3/d1b/d43/d61/d64/f78 zero size 2024-02-19T19:06:25.906 INFO:tasks.workunit.client.0.smithi088.stdout:6/535: mknod d5/d19/d70/ca0 0 2024-02-19T19:06:25.906 INFO:tasks.workunit.client.0.smithi088.stdout:8/504: dread - d9/df/d1c/d2d/f6d zero size 2024-02-19T19:06:25.906 INFO:tasks.workunit.client.0.smithi088.stdout:2/425: chown d3/d1b/l7d 661726846 1 2024-02-19T19:06:25.907 INFO:tasks.workunit.client.0.smithi088.stdout:6/536: read - d5/d8/d3a/d71/f74 zero size 2024-02-19T19:06:25.908 INFO:tasks.workunit.client.0.smithi088.stdout:0/478: rmdir d13/d62/d8a/d95/d5a/d25/d2a 39 2024-02-19T19:06:25.982 INFO:tasks.workunit.client.0.smithi088.stdout:2/426: getdents d3/d1f/d5d 0 2024-02-19T19:06:26.043 INFO:tasks.workunit.client.0.smithi088.stdout:7/558: sync 2024-02-19T19:06:26.043 INFO:tasks.workunit.client.0.smithi088.stdout:1/639: sync 2024-02-19T19:06:26.045 INFO:tasks.workunit.client.0.smithi088.stdout:7/559: chown d14/d27/d49 6 1 2024-02-19T19:06:26.045 INFO:tasks.workunit.client.0.smithi088.stdout:0/479: creat d13/d62/d8a/d95/d5a/d25/d2a/d3c/fa4 x:0 0 0 2024-02-19T19:06:26.046 INFO:tasks.workunit.client.0.smithi088.stdout:8/505: dwrite d9/df/d1c/f16 [0,4194304] 0 2024-02-19T19:06:26.046 INFO:tasks.workunit.client.0.smithi088.stdout:6/537: dwrite d5/d8/d3a/f46 [8388608,4194304] 0 2024-02-19T19:06:26.047 INFO:tasks.workunit.client.0.smithi088.stdout:0/480: chown d13/d62/f8f 25697 1 2024-02-19T19:06:26.047 INFO:tasks.workunit.client.0.smithi088.stdout:7/560: chown d14/d1d/c3a 687 1 2024-02-19T19:06:26.053 INFO:tasks.workunit.client.0.smithi088.stdout:1/640: readlink d6/d21/d64/d94/d71/l5a 0 2024-02-19T19:06:26.059 INFO:tasks.workunit.client.0.smithi088.stdout:6/538: rename d5/d8/d2c/f40 to d5/d19/d26/fa1 0 2024-02-19T19:06:26.059 INFO:tasks.workunit.client.0.smithi088.stdout:2/427: dwrite d3/d14/f5c [0,4194304] 0 2024-02-19T19:06:26.061 INFO:tasks.workunit.client.0.smithi088.stdout:7/561: write d14/d28/d65/f9d [184026,11672] 0 2024-02-19T19:06:26.067 INFO:tasks.workunit.client.0.smithi088.stdout:0/481: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d34/f77 [4194304,4194304] 0 2024-02-19T19:06:26.067 INFO:tasks.workunit.client.0.smithi088.stdout:2/428: symlink d3/d1b/l81 0 2024-02-19T19:06:26.079 INFO:tasks.workunit.client.0.smithi088.stdout:2/429: creat d3/d1b/d43/d74/f82 x:0 0 0 2024-02-19T19:06:26.083 INFO:tasks.workunit.client.0.smithi088.stdout:6/539: dwrite d5/d7/d32/f53 [0,4194304] 0 2024-02-19T19:06:26.086 INFO:tasks.workunit.client.0.smithi088.stdout:7/562: dwrite d14/d27/d6b/f7b [0,4194304] 0 2024-02-19T19:06:26.087 INFO:tasks.workunit.client.0.smithi088.stdout:1/641: dwrite d6/d10/f4d [0,4194304] 0 2024-02-19T19:06:26.090 INFO:tasks.workunit.client.0.smithi088.stdout:2/430: mknod d3/d1b/d43/d74/c83 0 2024-02-19T19:06:26.091 INFO:tasks.workunit.client.0.smithi088.stdout:8/506: dread d9/df/d1c/f33 [0,4194304] 0 2024-02-19T19:06:26.095 INFO:tasks.workunit.client.0.smithi088.stdout:0/482: dwrite d13/f4a [0,4194304] 0 2024-02-19T19:06:26.095 INFO:tasks.workunit.client.0.smithi088.stdout:6/540: creat d5/d45/d9d/fa2 x:0 0 0 2024-02-19T19:06:26.095 INFO:tasks.workunit.client.0.smithi088.stdout:1/642: write d6/d21/f36 [1423300,25974] 0 2024-02-19T19:06:26.100 INFO:tasks.workunit.client.0.smithi088.stdout:7/563: mknod d14/d1d/d3c/d43/db2/cc7 0 2024-02-19T19:06:26.101 INFO:tasks.workunit.client.0.smithi088.stdout:1/643: mknod d6/d21/d6a/d4a/cd0 0 2024-02-19T19:06:26.102 INFO:tasks.workunit.client.0.smithi088.stdout:6/541: read d5/de/d64/d82/d8c/d25/f5d [798194,124619] 0 2024-02-19T19:06:26.103 INFO:tasks.workunit.client.0.smithi088.stdout:2/431: mknod d3/d1b/d43/d61/c84 0 2024-02-19T19:06:26.106 INFO:tasks.workunit.client.0.smithi088.stdout:7/564: creat d14/d1d/d3c/d43/d5c/fc8 x:0 0 0 2024-02-19T19:06:26.107 INFO:tasks.workunit.client.0.smithi088.stdout:7/565: fsync d14/d1d/d3c/d43/d5c/fb5 0 2024-02-19T19:06:26.108 INFO:tasks.workunit.client.0.smithi088.stdout:1/644: creat d6/d10/d60/db8/fd1 x:0 0 0 2024-02-19T19:06:26.110 INFO:tasks.workunit.client.0.smithi088.stdout:6/542: creat d5/d8/d3a/d71/fa3 x:0 0 0 2024-02-19T19:06:26.112 INFO:tasks.workunit.client.0.smithi088.stdout:7/566: creat d14/d28/d3b/fc9 x:0 0 0 2024-02-19T19:06:26.113 INFO:tasks.workunit.client.0.smithi088.stdout:1/645: creat d6/d21/d6a/d7d/fd2 x:0 0 0 2024-02-19T19:06:26.117 INFO:tasks.workunit.client.0.smithi088.stdout:6/543: creat d5/d8/d3a/d91/fa4 x:0 0 0 2024-02-19T19:06:26.117 INFO:tasks.workunit.client.0.smithi088.stdout:0/483: write d13/d62/d8a/d95/d5a/d25/d2a/d3c/f64 [36762,53471] 0 2024-02-19T19:06:26.118 INFO:tasks.workunit.client.0.smithi088.stdout:1/646: creat d6/d21/d64/fd3 x:0 0 0 2024-02-19T19:06:26.118 INFO:tasks.workunit.client.0.smithi088.stdout:6/544: fsync d5/d19/f6d 0 2024-02-19T19:06:26.120 INFO:tasks.workunit.client.0.smithi088.stdout:2/432: rename d3/f3b to d3/f85 0 2024-02-19T19:06:26.125 INFO:tasks.workunit.client.0.smithi088.stdout:7/567: write d14/d1d/f38 [1456788,113238] 0 2024-02-19T19:06:26.128 INFO:tasks.workunit.client.0.smithi088.stdout:6/545: dwrite d5/d8/f3f [4194304,4194304] 0 2024-02-19T19:06:26.128 INFO:tasks.workunit.client.0.smithi088.stdout:8/507: dread d9/df/d3d/f5f [0,4194304] 0 2024-02-19T19:06:26.129 INFO:tasks.workunit.client.0.smithi088.stdout:0/484: creat d13/fa5 x:0 0 0 2024-02-19T19:06:26.135 INFO:tasks.workunit.client.0.smithi088.stdout:0/485: creat d13/d62/fa6 x:0 0 0 2024-02-19T19:06:26.138 INFO:tasks.workunit.client.0.smithi088.stdout:7/568: rename d14/d1d/d3c/lc6 to d14/d27/d49/d98/dbe/lca 0 2024-02-19T19:06:26.139 INFO:tasks.workunit.client.0.smithi088.stdout:8/508: dwrite d9/df/d1c/d2d/f54 [0,4194304] 0 2024-02-19T19:06:26.141 INFO:tasks.workunit.client.0.smithi088.stdout:6/546: creat d5/d19/d26/d72/fa5 x:0 0 0 2024-02-19T19:06:26.142 INFO:tasks.workunit.client.0.smithi088.stdout:2/433: getdents d3/d1f 0 2024-02-19T19:06:26.145 INFO:tasks.workunit.client.0.smithi088.stdout:1/647: dwrite d6/d21/d64/f81 [0,4194304] 0 2024-02-19T19:06:26.147 INFO:tasks.workunit.client.0.smithi088.stdout:0/486: rename d13/d62/d8a/d95/f69 to d13/d62/d8a/d95/d5a/d25/d2a/d3c/fa7 0 2024-02-19T19:06:26.148 INFO:tasks.workunit.client.0.smithi088.stdout:8/509: mknod d9/df/d1c/d2d/c96 0 2024-02-19T19:06:26.150 INFO:tasks.workunit.client.0.smithi088.stdout:0/487: write d13/d62/d8a/d95/d5a/d25/d2a/d34/f43 [1964699,114492] 0 2024-02-19T19:06:26.152 INFO:tasks.workunit.client.0.smithi088.stdout:7/569: mkdir d14/d27/d49/d98/dcb 0 2024-02-19T19:06:26.154 INFO:tasks.workunit.client.0.smithi088.stdout:6/547: mknod d5/de/d64/d8d/d9e/ca6 0 2024-02-19T19:06:26.155 INFO:tasks.workunit.client.0.smithi088.stdout:2/434: mkdir d3/d1b/d43/d61/d86 0 2024-02-19T19:06:26.159 INFO:tasks.workunit.client.0.smithi088.stdout:6/548: creat d5/d8/d77/fa7 x:0 0 0 2024-02-19T19:06:26.214 INFO:tasks.workunit.client.0.smithi088.stdout:0/488: mkdir d13/d62/d8a/d9c/da8 0 2024-02-19T19:06:26.214 INFO:tasks.workunit.client.0.smithi088.stdout:7/570: symlink d14/lcc 0 2024-02-19T19:06:26.215 INFO:tasks.workunit.client.0.smithi088.stdout:5/672: sync 2024-02-19T19:06:26.215 INFO:tasks.workunit.client.0.smithi088.stdout:3/652: sync 2024-02-19T19:06:26.215 INFO:tasks.workunit.client.0.smithi088.stdout:8/510: mkdir d9/df/d1c/d2d/d6c/d77/d97 0 2024-02-19T19:06:26.215 INFO:tasks.workunit.client.0.smithi088.stdout:0/489: chown f3 53423 1 2024-02-19T19:06:26.216 INFO:tasks.workunit.client.0.smithi088.stdout:7/571: rmdir d14/d1d/d3c 39 2024-02-19T19:06:26.217 INFO:tasks.workunit.client.0.smithi088.stdout:3/653: creat d4/d22/dd5/fe6 x:0 0 0 2024-02-19T19:06:26.220 INFO:tasks.workunit.client.0.smithi088.stdout:0/490: creat d13/d62/d8a/d87/fa9 x:0 0 0 2024-02-19T19:06:26.221 INFO:tasks.workunit.client.0.smithi088.stdout:5/673: truncate d4/d61/d65/d9a/d9f/d57/fe0 249219 0 2024-02-19T19:06:26.223 INFO:tasks.workunit.client.0.smithi088.stdout:3/654: mkdir d4/d22/d7a/de7 0 2024-02-19T19:06:26.223 INFO:tasks.workunit.client.0.smithi088.stdout:6/549: dwrite d5/d8/f7b [0,4194304] 0 2024-02-19T19:06:26.227 INFO:tasks.workunit.client.0.smithi088.stdout:8/511: dwrite d9/df/d1c/f16 [4194304,4194304] 0 2024-02-19T19:06:26.228 INFO:tasks.workunit.client.0.smithi088.stdout:3/655: write d4/d22/f2c [121220,25350] 0 2024-02-19T19:06:26.230 INFO:tasks.workunit.client.0.smithi088.stdout:5/674: creat d4/d1a/fe9 x:0 0 0 2024-02-19T19:06:26.231 INFO:tasks.workunit.client.0.smithi088.stdout:6/550: mkdir d5/d8/d3a/da8 0 2024-02-19T19:06:26.231 INFO:tasks.workunit.client.0.smithi088.stdout:0/491: dread d13/f2c [0,4194304] 0 2024-02-19T19:06:26.232 INFO:tasks.workunit.client.0.smithi088.stdout:3/656: write d4/d22/d7a/fbc [4581208,72028] 0 2024-02-19T19:06:26.234 INFO:tasks.workunit.client.0.smithi088.stdout:7/572: dwrite d14/d1d/d3c/f8e [0,4194304] 0 2024-02-19T19:06:26.235 INFO:tasks.workunit.client.0.smithi088.stdout:7/573: chown d14/d27/c59 6670396 1 2024-02-19T19:06:26.237 INFO:tasks.workunit.client.0.smithi088.stdout:8/512: unlink d9/l56 0 2024-02-19T19:06:26.240 INFO:tasks.workunit.client.0.smithi088.stdout:3/657: creat d4/d1a/d1e/d31/dc5/dcf/da4/fe8 x:0 0 0 2024-02-19T19:06:26.241 INFO:tasks.workunit.client.0.smithi088.stdout:6/551: dread d5/d7/f4b [0,4194304] 0 2024-02-19T19:06:26.242 INFO:tasks.workunit.client.0.smithi088.stdout:8/513: creat d9/df/d1c/d22/f98 x:0 0 0 2024-02-19T19:06:26.245 INFO:tasks.workunit.client.0.smithi088.stdout:0/492: rename d13/d62/d8a/d95/d5a/d25/d2a/d34/c47 to d13/d62/d8a/d9c/caa 0 2024-02-19T19:06:26.246 INFO:tasks.workunit.client.0.smithi088.stdout:3/658: fsync d4/d1a/d1e/d31/d32/f4c 0 2024-02-19T19:06:26.247 INFO:tasks.workunit.client.0.smithi088.stdout:6/552: truncate d5/d19/d26/d72/fa5 255388 0 2024-02-19T19:06:26.249 INFO:tasks.workunit.client.0.smithi088.stdout:7/574: dwrite d14/d1d/d3c/d43/d5c/fb5 [0,4194304] 0 2024-02-19T19:06:26.249 INFO:tasks.workunit.client.0.smithi088.stdout:8/514: truncate d9/df/d1c/d2d/f3e 4793815 0 2024-02-19T19:06:26.250 INFO:tasks.workunit.client.0.smithi088.stdout:0/493: readlink d13/d62/d8a/d95/d5a/d25/d2a/d34/l45 0 2024-02-19T19:06:26.251 INFO:tasks.workunit.client.0.smithi088.stdout:5/675: rename d4/d7/f9b to d4/d1a/d41/fea 0 2024-02-19T19:06:26.251 INFO:tasks.workunit.client.0.smithi088.stdout:7/575: write d14/d28/d9c/fc0 [406767,79555] 0 2024-02-19T19:06:26.254 INFO:tasks.workunit.client.0.smithi088.stdout:7/576: chown d14/f2a 27042 1 2024-02-19T19:06:26.259 INFO:tasks.workunit.client.0.smithi088.stdout:5/676: dread d4/d1a/f5b [0,4194304] 0 2024-02-19T19:06:26.260 INFO:tasks.workunit.client.0.smithi088.stdout:8/515: stat d9/df/d1c/d2d/d47/d6b/d72/d7f/d8e/c93 0 2024-02-19T19:06:26.263 INFO:tasks.workunit.client.0.smithi088.stdout:0/494: dread - d13/d62/d8a/d95/d5a/d25/d2a/d2f/f50 zero size 2024-02-19T19:06:26.264 INFO:tasks.workunit.client.0.smithi088.stdout:0/495: readlink d13/d62/d8a/d95/d5a/d25/d2a/d2f/l70 0 2024-02-19T19:06:26.265 INFO:tasks.workunit.client.0.smithi088.stdout:5/677: mkdir d4/d75/deb 0 2024-02-19T19:06:26.266 INFO:tasks.workunit.client.0.smithi088.stdout:5/678: chown d4/d61/fa8 5148 1 2024-02-19T19:06:26.269 INFO:tasks.workunit.client.0.smithi088.stdout:3/659: creat d4/d1a/d1e/d74/d7d/daa/fe9 x:0 0 0 2024-02-19T19:06:26.270 INFO:tasks.workunit.client.0.smithi088.stdout:8/516: mknod d9/df/d1c/d2d/d82/c99 0 2024-02-19T19:06:26.272 INFO:tasks.workunit.client.0.smithi088.stdout:0/496: mkdir d13/d62/d8a/d9c/da8/dab 0 2024-02-19T19:06:26.273 INFO:tasks.workunit.client.0.smithi088.stdout:3/660: rename d4/d1a/d1e/d31/d32/l46 to d4/d22/d81/lea 0 2024-02-19T19:06:26.274 INFO:tasks.workunit.client.0.smithi088.stdout:7/577: write d14/d1d/d3c/d43/d5c/f9f [875834,76250] 0 2024-02-19T19:06:26.274 INFO:tasks.workunit.client.0.smithi088.stdout:6/553: write d5/de/f22 [428515,48890] 0 2024-02-19T19:06:26.277 INFO:tasks.workunit.client.0.smithi088.stdout:6/554: chown d5/d7/f4b 0 1 2024-02-19T19:06:26.277 INFO:tasks.workunit.client.0.smithi088.stdout:0/497: creat d13/d62/d8a/d9c/fac x:0 0 0 2024-02-19T19:06:26.277 INFO:tasks.workunit.client.0.smithi088.stdout:8/517: read d9/df/d1c/d2d/d47/f44 [2387221,98098] 0 2024-02-19T19:06:26.279 INFO:tasks.workunit.client.0.smithi088.stdout:7/578: mknod d14/d1d/d3c/d43/d5c/d9a/ccd 0 2024-02-19T19:06:26.280 INFO:tasks.workunit.client.0.smithi088.stdout:5/679: getdents d4/d61/d65/d9a 0 2024-02-19T19:06:26.281 INFO:tasks.workunit.client.0.smithi088.stdout:0/498: write d13/d62/d8a/d95/d5a/d25/d6d/f6e [3625282,113782] 0 2024-02-19T19:06:26.284 INFO:tasks.workunit.client.0.smithi088.stdout:8/518: chown d9/df/c32 1185587489 1 2024-02-19T19:06:26.284 INFO:tasks.workunit.client.0.smithi088.stdout:5/680: read - d4/d1a/d41/dc1/dc2/fc6 zero size 2024-02-19T19:06:26.285 INFO:tasks.workunit.client.0.smithi088.stdout:6/555: dwrite d5/de/f22 [0,4194304] 0 2024-02-19T19:06:26.286 INFO:tasks.workunit.client.0.smithi088.stdout:8/519: chown d9/df/d1c/d22/l3c 3145 1 2024-02-19T19:06:26.287 INFO:tasks.workunit.client.0.smithi088.stdout:6/556: dread - d5/d7/d32/f86 zero size 2024-02-19T19:06:26.287 INFO:tasks.workunit.client.0.smithi088.stdout:8/520: fsync d9/df/d3d/d50/d71/f8b 0 2024-02-19T19:06:26.288 INFO:tasks.workunit.client.0.smithi088.stdout:3/661: rename d4/d12/f17 to d4/d1a/d1e/d31/d32/d55/feb 0 2024-02-19T19:06:26.288 INFO:tasks.workunit.client.0.smithi088.stdout:3/662: chown d4/d1a/l45 754 1 2024-02-19T19:06:26.289 INFO:tasks.workunit.client.0.smithi088.stdout:7/579: creat d14/d27/d95/fce x:0 0 0 2024-02-19T19:06:26.290 INFO:tasks.workunit.client.0.smithi088.stdout:7/580: fdatasync fd 0 2024-02-19T19:06:26.290 INFO:tasks.workunit.client.0.smithi088.stdout:0/499: creat d13/d62/d8a/d95/d5a/d25/d2a/d3c/fad x:0 0 0 2024-02-19T19:06:26.293 INFO:tasks.workunit.client.0.smithi088.stdout:0/500: write d13/d62/d8a/d95/d5a/fa2 [686882,39644] 0 2024-02-19T19:06:26.297 INFO:tasks.workunit.client.0.smithi088.stdout:6/557: rename d5/d19/d70/l7a to d5/de/d64/d82/d8c/la9 0 2024-02-19T19:06:26.299 INFO:tasks.workunit.client.0.smithi088.stdout:5/681: dwrite d4/d1a/fa7 [0,4194304] 0 2024-02-19T19:06:26.300 INFO:tasks.workunit.client.0.smithi088.stdout:7/581: mkdir d14/d1d/d62/dcf 0 2024-02-19T19:06:26.301 INFO:tasks.workunit.client.0.smithi088.stdout:7/582: write d14/f1f [319234,43879] 0 2024-02-19T19:06:26.303 INFO:tasks.workunit.client.0.smithi088.stdout:3/663: dread - d4/d12/d33/d43/d85/fae zero size 2024-02-19T19:06:26.303 INFO:tasks.workunit.client.0.smithi088.stdout:7/583: write d14/d27/d49/f7d [301527,30745] 0 2024-02-19T19:06:26.307 INFO:tasks.workunit.client.0.smithi088.stdout:6/558: write d5/d7/f4b [675268,22274] 0 2024-02-19T19:06:26.310 INFO:tasks.workunit.client.0.smithi088.stdout:5/682: rename d4/d1a/fa7 to d4/d61/d65/fec 0 2024-02-19T19:06:26.312 INFO:tasks.workunit.client.0.smithi088.stdout:8/521: dread d9/f2e [0,4194304] 0 2024-02-19T19:06:26.312 INFO:tasks.workunit.client.0.smithi088.stdout:3/664: stat d4/fd 0 2024-02-19T19:06:26.314 INFO:tasks.workunit.client.0.smithi088.stdout:3/665: dread - d4/d22/d7a/fa7 zero size 2024-02-19T19:06:26.315 INFO:tasks.workunit.client.0.smithi088.stdout:7/584: fsync fb 0 2024-02-19T19:06:26.316 INFO:tasks.workunit.client.0.smithi088.stdout:6/559: dwrite d5/d19/d70/f73 [0,4194304] 0 2024-02-19T19:06:26.319 INFO:tasks.workunit.client.0.smithi088.stdout:5/683: readlink d4/d7/l27 0 2024-02-19T19:06:26.321 INFO:tasks.workunit.client.0.smithi088.stdout:0/501: link d13/f1a d13/d62/fae 0 2024-02-19T19:06:26.321 INFO:tasks.workunit.client.0.smithi088.stdout:5/684: read d4/d7/f8f [332441,66821] 0 2024-02-19T19:06:26.323 INFO:tasks.workunit.client.0.smithi088.stdout:7/585: symlink d14/d27/d95/ld0 0 2024-02-19T19:06:26.325 INFO:tasks.workunit.client.0.smithi088.stdout:6/560: mkdir d5/d8/d3a/d71/daa 0 2024-02-19T19:06:26.326 INFO:tasks.workunit.client.0.smithi088.stdout:3/666: dread d4/d12/f47 [0,4194304] 0 2024-02-19T19:06:26.329 INFO:tasks.workunit.client.0.smithi088.stdout:5/685: rmdir d4/d61/d65 39 2024-02-19T19:06:26.332 INFO:tasks.workunit.client.0.smithi088.stdout:7/586: unlink d14/d1d/d3c/d43/f5b 0 2024-02-19T19:06:26.334 INFO:tasks.workunit.client.0.smithi088.stdout:6/561: creat d5/d45/fab x:0 0 0 2024-02-19T19:06:26.334 INFO:tasks.workunit.client.0.smithi088.stdout:0/502: rmdir d13/d62/d8a/d95 39 2024-02-19T19:06:26.337 INFO:tasks.workunit.client.0.smithi088.stdout:8/522: rename d9/df/d1c/d2d/d47 to d9/df/d1c/d2d/d6c/d9a 0 2024-02-19T19:06:26.337 INFO:tasks.workunit.client.0.smithi088.stdout:8/523: dread - d9/df/f55 zero size 2024-02-19T19:06:26.338 INFO:tasks.workunit.client.0.smithi088.stdout:3/667: symlink d4/d12/d33/d58/lec 0 2024-02-19T19:06:26.341 INFO:tasks.workunit.client.0.smithi088.stdout:7/587: symlink d14/d28/d94/ld1 0 2024-02-19T19:06:26.341 INFO:tasks.workunit.client.0.smithi088.stdout:6/562: creat d5/de/d60/fac x:0 0 0 2024-02-19T19:06:26.342 INFO:tasks.workunit.client.0.smithi088.stdout:6/563: chown d5/ff 226176543 1 2024-02-19T19:06:26.343 INFO:tasks.workunit.client.0.smithi088.stdout:8/524: creat d9/df/d1c/d2d/f9b x:0 0 0 2024-02-19T19:06:26.343 INFO:tasks.workunit.client.0.smithi088.stdout:6/564: truncate d5/d19/d70/f80 377248 0 2024-02-19T19:06:26.344 INFO:tasks.workunit.client.0.smithi088.stdout:5/686: creat d4/d1a/d41/de2/de7/fed x:0 0 0 2024-02-19T19:06:26.344 INFO:tasks.workunit.client.0.smithi088.stdout:7/588: mkdir d14/d27/dd2 0 2024-02-19T19:06:26.346 INFO:tasks.workunit.client.0.smithi088.stdout:8/525: write d9/df/d1c/d2d/d6c/d9a/f5e [2234158,52483] 0 2024-02-19T19:06:26.346 INFO:tasks.workunit.client.0.smithi088.stdout:5/687: mknod d4/d7/cee 0 2024-02-19T19:06:26.351 INFO:tasks.workunit.client.0.smithi088.stdout:7/589: rename d14/d27/d95 to d14/d28/d3b/dd3 0 2024-02-19T19:06:26.354 INFO:tasks.workunit.client.0.smithi088.stdout:3/668: link d4/c1b d4/d1a/d7f/ced 0 2024-02-19T19:06:26.355 INFO:tasks.workunit.client.0.smithi088.stdout:3/669: readlink d4/d1a/d1e/l6d 0 2024-02-19T19:06:26.356 INFO:tasks.workunit.client.0.smithi088.stdout:8/526: mknod d9/df/d1c/d2d/d6c/d77/d97/c9c 0 2024-02-19T19:06:26.358 INFO:tasks.workunit.client.0.smithi088.stdout:3/670: creat d4/d22/d81/fee x:0 0 0 2024-02-19T19:06:26.359 INFO:tasks.workunit.client.0.smithi088.stdout:0/503: truncate f3 641612 0 2024-02-19T19:06:26.361 INFO:tasks.workunit.client.0.smithi088.stdout:6/565: dread d5/de/d64/d82/d8c/f6e [4194304,4194304] 0 2024-02-19T19:06:26.363 INFO:tasks.workunit.client.0.smithi088.stdout:8/527: mknod d9/df/d1c/d2d/d6c/d9a/c9d 0 2024-02-19T19:06:26.363 INFO:tasks.workunit.client.0.smithi088.stdout:6/566: write d5/de/d64/d82/d8c/d25/f39 [4882055,22833] 0 2024-02-19T19:06:26.364 INFO:tasks.workunit.client.0.smithi088.stdout:3/671: dwrite d4/d22/d7a/fbc [0,4194304] 0 2024-02-19T19:06:26.428 INFO:tasks.workunit.client.0.smithi088.stdout:0/504: rmdir d13/d62 39 2024-02-19T19:06:26.429 INFO:tasks.workunit.client.0.smithi088.stdout:9/619: sync 2024-02-19T19:06:26.429 INFO:tasks.workunit.client.0.smithi088.stdout:4/464: sync 2024-02-19T19:06:26.430 INFO:tasks.workunit.client.0.smithi088.stdout:8/528: mkdir d9/df/d1c/d22/d9e 0 2024-02-19T19:06:26.432 INFO:tasks.workunit.client.0.smithi088.stdout:3/672: read d4/f6 [1228002,20758] 0 2024-02-19T19:06:26.433 INFO:tasks.workunit.client.0.smithi088.stdout:8/529: fdatasync d9/df/d1c/d2d/d6c/d9a/d6b/d72/d5d/f65 0 2024-02-19T19:06:26.435 INFO:tasks.workunit.client.0.smithi088.stdout:8/530: write d9/df/f55 [40510,62406] 0 2024-02-19T19:06:26.437 INFO:tasks.workunit.client.0.smithi088.stdout:9/620: chown d1/d8/d4a/dac/f90 470555888 1 2024-02-19T19:06:26.437 INFO:tasks.workunit.client.0.smithi088.stdout:4/465: rmdir d2/d32/d25/d3f/d70 39 2024-02-19T19:06:26.443 INFO:tasks.workunit.client.0.smithi088.stdout:9/621: chown d1/d8/ca7 3609219 1 2024-02-19T19:06:26.443 INFO:tasks.workunit.client.0.smithi088.stdout:6/567: symlink d5/de/d64/d82/d8c/d25/lad 0 2024-02-19T19:06:26.444 INFO:tasks.workunit.client.0.smithi088.stdout:0/505: rename d13/d62/d8a/d87/fa9 to d13/d81/faf 0 2024-02-19T19:06:26.444 INFO:tasks.workunit.client.0.smithi088.stdout:3/673: unlink d4/f6a 0 2024-02-19T19:06:26.451 INFO:tasks.workunit.client.0.smithi088.stdout:8/531: write d9/df/d1c/f25 [3780228,34719] 0 2024-02-19T19:06:26.452 INFO:tasks.workunit.client.0.smithi088.stdout:8/532: write d9/f63 [189120,30284] 0 2024-02-19T19:06:26.460 INFO:tasks.workunit.client.0.smithi088.stdout:4/466: write d2/d32/d25/d3f/d70/f8c [453192,128675] 0 2024-02-19T19:06:26.462 INFO:tasks.workunit.client.0.smithi088.stdout:4/467: chown d2/d32/d25/d26/d4e/d6f 11921700 1 2024-02-19T19:06:26.462 INFO:tasks.workunit.client.0.smithi088.stdout:3/674: chown d4/d22/d7a/f9f 224 1 2024-02-19T19:06:26.467 INFO:tasks.workunit.client.0.smithi088.stdout:9/622: rename d1/d8/d3e/d93/d5a/d94/d74/d8c to d1/d8/d3e/d93/d5a/dc4 0 2024-02-19T19:06:26.469 INFO:tasks.workunit.client.0.smithi088.stdout:9/623: truncate d1/d8/d3e/d93/fa1 5021264 0 2024-02-19T19:06:26.470 INFO:tasks.workunit.client.0.smithi088.stdout:0/506: rename d13/d62/d8a/d95/d5b/f7b to d13/d62/d8a/d95/d5a/d25/d2a/d2f/fb0 0 2024-02-19T19:06:26.471 INFO:tasks.workunit.client.0.smithi088.stdout:8/533: link d9/df/f36 d9/df/d3d/f9f 0 2024-02-19T19:06:26.473 INFO:tasks.workunit.client.0.smithi088.stdout:3/675: creat d4/d22/fef x:0 0 0 2024-02-19T19:06:26.474 INFO:tasks.workunit.client.0.smithi088.stdout:0/507: mknod d13/d81/cb1 0 2024-02-19T19:06:26.476 INFO:tasks.workunit.client.0.smithi088.stdout:9/624: mkdir d1/d8/d3e/d93/d2d/d23/d79/d99/dc5 0 2024-02-19T19:06:26.476 INFO:tasks.workunit.client.0.smithi088.stdout:0/508: rmdir d13/d62/d8a/d95/d5b/d76 39 2024-02-19T19:06:26.477 INFO:tasks.workunit.client.0.smithi088.stdout:0/509: dread - d13/fa5 zero size 2024-02-19T19:06:26.478 INFO:tasks.workunit.client.0.smithi088.stdout:4/468: dread d2/d35/f4f [0,4194304] 0 2024-02-19T19:06:26.479 INFO:tasks.workunit.client.0.smithi088.stdout:0/510: write d13/d62/fa6 [796552,130106] 0 2024-02-19T19:06:26.479 INFO:tasks.workunit.client.0.smithi088.stdout:8/534: creat d9/df/d1c/d2d/d6c/fa0 x:0 0 0 2024-02-19T19:06:26.480 INFO:tasks.workunit.client.0.smithi088.stdout:0/511: fdatasync d13/d62/d8a/d95/d5a/fa2 0 2024-02-19T19:06:26.481 INFO:tasks.workunit.client.0.smithi088.stdout:9/625: creat d1/d8/d3e/d93/d2d/d18/dc0/fc6 x:0 0 0 2024-02-19T19:06:26.493 INFO:tasks.workunit.client.0.smithi088.stdout:8/535: dread d9/df/d1c/f16 [8388608,4194304] 0 2024-02-19T19:06:26.494 INFO:tasks.workunit.client.0.smithi088.stdout:4/469: dread d2/d32/d25/d26/d6d/d7e/d90/f94 [0,4194304] 0 2024-02-19T19:06:26.498 INFO:tasks.workunit.client.0.smithi088.stdout:8/536: dread d9/df/d1c/d2d/f54 [0,4194304] 0 2024-02-19T19:06:26.498 INFO:tasks.workunit.client.0.smithi088.stdout:4/470: unlink d2/d32/d25/d45/l66 0 2024-02-19T19:06:26.499 INFO:tasks.workunit.client.0.smithi088.stdout:4/471: fsync d2/d32/f36 0 2024-02-19T19:06:26.503 INFO:tasks.workunit.client.0.smithi088.stdout:9/626: truncate d1/d8/d4a/dac/f6c 2972613 0 2024-02-19T19:06:26.506 INFO:tasks.workunit.client.0.smithi088.stdout:8/537: dwrite d9/df/d1c/d85/f8d [0,4194304] 0 2024-02-19T19:06:26.506 INFO:tasks.workunit.client.0.smithi088.stdout:9/627: dread - d1/d8/d3e/d93/d2f/f7b zero size 2024-02-19T19:06:26.506 INFO:tasks.workunit.client.0.smithi088.stdout:4/472: dread d2/d32/d25/f38 [0,4194304] 0 2024-02-19T19:06:26.509 INFO:tasks.workunit.client.0.smithi088.stdout:0/512: dwrite d13/d62/d8a/d95/d5b/d76/f83 [4194304,4194304] 0 2024-02-19T19:06:26.515 INFO:tasks.workunit.client.0.smithi088.stdout:8/538: dread d9/df/d1c/f16 [0,4194304] 0 2024-02-19T19:06:26.517 INFO:tasks.workunit.client.0.smithi088.stdout:8/539: chown d9/df/d1c/d2d/d6c/d9a/f4d 453596344 1 2024-02-19T19:06:26.519 INFO:tasks.workunit.client.0.smithi088.stdout:9/628: rename d1/d8/d3e/d93/d2d/d23/l88 to d1/d8/d4a/lc7 0 2024-02-19T19:06:26.521 INFO:tasks.workunit.client.0.smithi088.stdout:8/540: write d9/f75 [553118,101850] 0 2024-02-19T19:06:26.524 INFO:tasks.workunit.client.0.smithi088.stdout:9/629: mkdir d1/d8/d3e/d93/d2d/d23/d79/dc8 0 2024-02-19T19:06:26.525 INFO:tasks.workunit.client.0.smithi088.stdout:8/541: readlink d9/df/d3d/l88 0 2024-02-19T19:06:26.526 INFO:tasks.workunit.client.0.smithi088.stdout:8/542: stat d9/df/d1c/d22/f98 0 2024-02-19T19:06:26.526 INFO:tasks.workunit.client.0.smithi088.stdout:9/630: write d1/d8/d3e/d93/d2d/d18/d1f/db0/fab [470171,56257] 0 2024-02-19T19:06:26.527 INFO:tasks.workunit.client.0.smithi088.stdout:0/513: truncate d13/d62/f8e 62900 0 2024-02-19T19:06:26.532 INFO:tasks.workunit.client.0.smithi088.stdout:0/514: dread d13/d62/d8a/d95/d5a/d25/d2a/d3c/fa7 [0,4194304] 0 2024-02-19T19:06:26.534 INFO:tasks.workunit.client.0.smithi088.stdout:0/515: chown d13/d62/d8a/d95/d5a/d25/d2a/c7e 3516 1 2024-02-19T19:06:26.534 INFO:tasks.workunit.client.0.smithi088.stdout:9/631: dwrite d1/d8/d4a/d4f/f7a [0,4194304] 0 2024-02-19T19:06:26.535 INFO:tasks.workunit.client.0.smithi088.stdout:8/543: rename d9/df/d1c/d2d to d9/df/d3d/d50/d71/da1 0 2024-02-19T19:06:26.537 INFO:tasks.workunit.client.0.smithi088.stdout:8/544: read d9/f4a [1334724,99848] 0 2024-02-19T19:06:26.538 INFO:tasks.workunit.client.0.smithi088.stdout:0/516: symlink d13/d62/d8a/d9c/lb2 0 2024-02-19T19:06:26.541 INFO:tasks.workunit.client.0.smithi088.stdout:8/545: readlink d9/df/d3d/d50/l61 0 2024-02-19T19:06:26.542 INFO:tasks.workunit.client.0.smithi088.stdout:0/517: symlink d13/d62/d8a/d95/d96/lb3 0 2024-02-19T19:06:26.542 INFO:tasks.workunit.client.0.smithi088.stdout:9/632: getdents d1/d8/d3e/d93/d2d/d23/d79/dc3 0 2024-02-19T19:06:26.544 INFO:tasks.workunit.client.0.smithi088.stdout:0/518: stat d13/d62/d8a/d95/d5a/d25/d2a/d34/c6b 0 2024-02-19T19:06:26.544 INFO:tasks.workunit.client.0.smithi088.stdout:0/519: truncate d13/f29 4855976 0 2024-02-19T19:06:26.545 INFO:tasks.workunit.client.0.smithi088.stdout:0/520: readlink d13/d62/d8a/l89 0 2024-02-19T19:06:26.546 INFO:tasks.workunit.client.0.smithi088.stdout:8/546: rename d9/df/d1c/c18 to d9/df/d3d/d50/d71/da1/d6c/ca2 0 2024-02-19T19:06:26.549 INFO:tasks.workunit.client.0.smithi088.stdout:9/633: rmdir d1/d8 39 2024-02-19T19:06:26.550 INFO:tasks.workunit.client.0.smithi088.stdout:0/521: mkdir d13/d62/d8a/d9c/da8/dab/db4 0 2024-02-19T19:06:26.553 INFO:tasks.workunit.client.0.smithi088.stdout:8/547: rename d9/df/d3d/d50/d71/da1/d6c/d77/l7c to d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/la3 0 2024-02-19T19:06:26.555 INFO:tasks.workunit.client.0.smithi088.stdout:9/634: truncate d1/d8/d8d/d97/f53 782314 0 2024-02-19T19:06:26.660 INFO:tasks.workunit.client.0.smithi088.stdout:1/648: sync 2024-02-19T19:06:26.660 INFO:tasks.workunit.client.0.smithi088.stdout:2/435: sync 2024-02-19T19:06:26.661 INFO:tasks.workunit.client.0.smithi088.stdout:8/548: rmdir d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d 39 2024-02-19T19:06:26.661 INFO:tasks.workunit.client.0.smithi088.stdout:2/436: chown d3/d1f/d4f 15 1 2024-02-19T19:06:26.665 INFO:tasks.workunit.client.0.smithi088.stdout:1/649: rename d6/d10/f45 to d6/d21/d64/dc8/dcc/fd4 0 2024-02-19T19:06:26.666 INFO:tasks.workunit.client.0.smithi088.stdout:8/549: symlink d9/df/d3d/d50/d71/da1/la4 0 2024-02-19T19:06:26.667 INFO:tasks.workunit.client.0.smithi088.stdout:0/522: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d34/f4d [0,4194304] 0 2024-02-19T19:06:26.668 INFO:tasks.workunit.client.0.smithi088.stdout:2/437: creat d3/d1b/d43/d74/f87 x:0 0 0 2024-02-19T19:06:26.672 INFO:tasks.workunit.client.0.smithi088.stdout:1/650: write d6/d21/d64/d94/d71/fcd [412485,22871] 0 2024-02-19T19:06:26.672 INFO:tasks.workunit.client.0.smithi088.stdout:8/550: chown d9/df/d3d/d50/d71/f8b 1 1 2024-02-19T19:06:26.672 INFO:tasks.workunit.client.0.smithi088.stdout:9/635: rmdir d1/d8/d3e/d93/d2d/d18/db6 0 2024-02-19T19:06:26.672 INFO:tasks.workunit.client.0.smithi088.stdout:0/523: write d13/f1c [7839568,124087] 0 2024-02-19T19:06:26.672 INFO:tasks.workunit.client.0.smithi088.stdout:1/651: readlink d6/d21/l42 0 2024-02-19T19:06:26.673 INFO:tasks.workunit.client.0.smithi088.stdout:8/551: unlink d9/f75 0 2024-02-19T19:06:26.675 INFO:tasks.workunit.client.0.smithi088.stdout:9/636: read d1/d8/d8d/fa5 [803348,115045] 0 2024-02-19T19:06:26.676 INFO:tasks.workunit.client.0.smithi088.stdout:2/438: rename d3/d1b/c41 to d3/d1b/c88 0 2024-02-19T19:06:26.676 INFO:tasks.workunit.client.0.smithi088.stdout:2/439: stat d3/d1b/d43/d61/d77 0 2024-02-19T19:06:26.677 INFO:tasks.workunit.client.0.smithi088.stdout:8/552: truncate d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/f81 925866 0 2024-02-19T19:06:26.678 INFO:tasks.workunit.client.0.smithi088.stdout:1/652: creat d6/d10/fd5 x:0 0 0 2024-02-19T19:06:26.681 INFO:tasks.workunit.client.0.smithi088.stdout:9/637: rmdir d1/d8/d3e/d93/d2d/d18/d1f/db0 39 2024-02-19T19:06:26.682 INFO:tasks.workunit.client.0.smithi088.stdout:2/440: fsync d3/d1f/d4f/d59/f5f 0 2024-02-19T19:06:26.683 INFO:tasks.workunit.client.0.smithi088.stdout:8/553: mknod d9/df/d3d/d62/ca5 0 2024-02-19T19:06:26.683 INFO:tasks.workunit.client.0.smithi088.stdout:2/441: fsync d3/d1b/f2e 0 2024-02-19T19:06:26.684 INFO:tasks.workunit.client.0.smithi088.stdout:8/554: truncate d9/d6f/f89 1147028 0 2024-02-19T19:06:26.684 INFO:tasks.workunit.client.0.smithi088.stdout:2/442: fsync d3/d1b/d43/d61/d64/f63 0 2024-02-19T19:06:26.685 INFO:tasks.workunit.client.0.smithi088.stdout:9/638: symlink d1/d8/d3e/d93/d5a/d94/d74/lc9 0 2024-02-19T19:06:26.686 INFO:tasks.workunit.client.0.smithi088.stdout:2/443: write d3/d1b/d43/f55 [5162120,48547] 0 2024-02-19T19:06:26.686 INFO:tasks.workunit.client.0.smithi088.stdout:0/524: creat d13/d62/d8a/d95/d5b/fb5 x:0 0 0 2024-02-19T19:06:26.687 INFO:tasks.workunit.client.0.smithi088.stdout:8/555: read d9/df/d1c/f1f [4179904,115335] 0 2024-02-19T19:06:26.688 INFO:tasks.workunit.client.0.smithi088.stdout:0/525: fdatasync d13/d62/d8a/d87/f92 0 2024-02-19T19:06:26.690 INFO:tasks.workunit.client.0.smithi088.stdout:9/639: creat d1/d8/d3e/fca x:0 0 0 2024-02-19T19:06:26.691 INFO:tasks.workunit.client.0.smithi088.stdout:9/640: fsync d1/f7f 0 2024-02-19T19:06:26.691 INFO:tasks.workunit.client.0.smithi088.stdout:0/526: unlink d13/d62/d8a/d95/c5f 0 2024-02-19T19:06:26.696 INFO:tasks.workunit.client.0.smithi088.stdout:9/641: mkdir d1/d8/d3e/d93/d2d/dcb 0 2024-02-19T19:06:26.696 INFO:tasks.workunit.client.0.smithi088.stdout:2/444: dwrite d3/d1b/d43/d74/f82 [0,4194304] 0 2024-02-19T19:06:26.697 INFO:tasks.workunit.client.0.smithi088.stdout:0/527: mkdir d13/d62/d8a/d95/d5a/db6 0 2024-02-19T19:06:26.698 INFO:tasks.workunit.client.0.smithi088.stdout:0/528: readlink d13/l3f 0 2024-02-19T19:06:26.701 INFO:tasks.workunit.client.0.smithi088.stdout:9/642: creat d1/d8/d3e/d93/d2d/d23/fcc x:0 0 0 2024-02-19T19:06:26.702 INFO:tasks.workunit.client.0.smithi088.stdout:9/643: write d1/d8/f42 [3985727,83143] 0 2024-02-19T19:06:26.703 INFO:tasks.workunit.client.0.smithi088.stdout:8/556: dread d9/df/d1c/f29 [0,4194304] 0 2024-02-19T19:06:26.703 INFO:tasks.workunit.client.0.smithi088.stdout:0/529: symlink d13/d62/d8a/d95/d5b/d76/lb7 0 2024-02-19T19:06:26.705 INFO:tasks.workunit.client.0.smithi088.stdout:0/530: read - d13/d62/d8a/d95/d5a/d25/d6d/fa0 zero size 2024-02-19T19:06:26.707 INFO:tasks.workunit.client.0.smithi088.stdout:2/445: getdents d3/d1b/d43/d61 0 2024-02-19T19:06:26.711 INFO:tasks.workunit.client.0.smithi088.stdout:0/531: dread d13/d62/d8a/d95/d5a/d25/d2a/d34/f4d [0,4194304] 0 2024-02-19T19:06:26.714 INFO:tasks.workunit.client.0.smithi088.stdout:9/644: rename d1/d8/d3e/d93/d2d/d23/c9b to d1/d8/d4a/d4f/daa/dbb/ccd 0 2024-02-19T19:06:26.716 INFO:tasks.workunit.client.0.smithi088.stdout:0/532: creat d13/d62/d8a/d95/d5b/d76/fb8 x:0 0 0 2024-02-19T19:06:26.718 INFO:tasks.workunit.client.0.smithi088.stdout:2/446: link d3/d1f/d5d/f72 d3/d1f/f89 0 2024-02-19T19:06:26.720 INFO:tasks.workunit.client.0.smithi088.stdout:9/645: mknod d1/d8/d3e/d93/d2d/d18/d1f/cce 0 2024-02-19T19:06:26.724 INFO:tasks.workunit.client.0.smithi088.stdout:9/646: readlink d1/d8/d37/l40 0 2024-02-19T19:06:26.724 INFO:tasks.workunit.client.0.smithi088.stdout:0/533: dwrite d13/d62/d8a/d95/d5a/d25/d6d/fa0 [0,4194304] 0 2024-02-19T19:06:26.725 INFO:tasks.workunit.client.0.smithi088.stdout:9/647: read - d1/faf zero size 2024-02-19T19:06:26.730 INFO:tasks.workunit.client.0.smithi088.stdout:0/534: write d13/d62/d8a/d95/d5a/d25/d2a/d3c/f5d [4110400,84441] 0 2024-02-19T19:06:26.731 INFO:tasks.workunit.client.0.smithi088.stdout:2/447: dwrite d3/d1b/d43/d61/d64/f6d [0,4194304] 0 2024-02-19T19:06:26.734 INFO:tasks.workunit.client.0.smithi088.stdout:8/557: dwrite d9/df/d3d/d50/d71/da1/d6c/d9a/f44 [0,4194304] 0 2024-02-19T19:06:26.735 INFO:tasks.workunit.client.0.smithi088.stdout:0/535: symlink d13/lb9 0 2024-02-19T19:06:26.740 INFO:tasks.workunit.client.0.smithi088.stdout:9/648: rename d1/d8/d3e/d93/d2d/ca3 to d1/d8/d3e/d93/d2d/d23/d79/d99/ccf 0 2024-02-19T19:06:26.743 INFO:tasks.workunit.client.0.smithi088.stdout:0/536: rename lf to d13/d62/d8a/d87/lba 0 2024-02-19T19:06:26.743 INFO:tasks.workunit.client.0.smithi088.stdout:8/558: rename d9/df/d3d to d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/d8e/da6 22 2024-02-19T19:06:26.744 INFO:tasks.workunit.client.0.smithi088.stdout:9/649: mknod d1/d8/d3e/d93/d2d/d23/d79/cd0 0 2024-02-19T19:06:26.746 INFO:tasks.workunit.client.0.smithi088.stdout:8/559: read d9/df/d1c/f25 [3058462,42493] 0 2024-02-19T19:06:26.747 INFO:tasks.workunit.client.0.smithi088.stdout:9/650: creat d1/d8/d3e/d93/d5a/dc4/fd1 x:0 0 0 2024-02-19T19:06:26.748 INFO:tasks.workunit.client.0.smithi088.stdout:0/537: dread d13/d62/f9e [0,4194304] 0 2024-02-19T19:06:26.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:26 smithi132 ceph-mon[70676]: Upgrade: Setting container_image for all mon 2024-02-19T19:06:26.754 INFO:tasks.workunit.client.0.smithi088.stdout:9/651: dwrite d1/d8/d4a/d4f/fc2 [0,4194304] 0 2024-02-19T19:06:26.757 INFO:tasks.workunit.client.0.smithi088.stdout:9/652: write d1/d8/d3e/d93/d2d/d18/d1f/db0/fab [141390,72356] 0 2024-02-19T19:06:26.758 INFO:tasks.workunit.client.0.smithi088.stdout:9/653: readlink d1/lbf 0 2024-02-19T19:06:26.759 INFO:tasks.workunit.client.0.smithi088.stdout:9/654: dread - d1/d8/d3e/d93/d5a/dc4/fd1 zero size 2024-02-19T19:06:26.759 INFO:tasks.workunit.client.0.smithi088.stdout:9/655: stat d1/d8/d4a/d4f/fad 0 2024-02-19T19:06:26.764 INFO:tasks.workunit.client.0.smithi088.stdout:9/656: rename d1/d8/d3e/d93/d5a/f63 to d1/d8/d3e/d93/d2d/d23/d79/d99/fd2 0 2024-02-19T19:06:26.765 INFO:tasks.workunit.client.0.smithi088.stdout:9/657: readlink d1/d8/d3e/d93/d2d/d18/d1f/l89 0 2024-02-19T19:06:26.766 INFO:tasks.workunit.client.0.smithi088.stdout:9/658: write d1/d8/d3e/fca [358210,21529] 0 2024-02-19T19:06:26.769 INFO:tasks.workunit.client.0.smithi088.stdout:9/659: dread - d1/d8/d3e/d93/d2d/d18/f6a zero size 2024-02-19T19:06:26.771 INFO:tasks.workunit.client.0.smithi088.stdout:9/660: creat d1/d8/d3e/d93/d2d/d18/d92/fd3 x:0 0 0 2024-02-19T19:06:26.777 INFO:tasks.workunit.client.0.smithi088.stdout:2/448: dread d3/d1f/d4f/f11 [4194304,4194304] 0 2024-02-19T19:06:26.778 INFO:tasks.workunit.client.0.smithi088.stdout:2/449: write d3/d1b/d43/d61/d64/f78 [895810,74256] 0 2024-02-19T19:06:26.788 INFO:tasks.workunit.client.0.smithi088.stdout:9/661: dread d1/d8/d8d/d97/f12 [0,4194304] 0 2024-02-19T19:06:26.789 INFO:tasks.workunit.client.0.smithi088.stdout:9/662: read d1/d8/d3e/d93/fa1 [2213386,97871] 0 2024-02-19T19:06:26.791 INFO:tasks.workunit.client.0.smithi088.stdout:9/663: fsync d1/d8/d4a/dac/f6c 0 2024-02-19T19:06:26.792 INFO:tasks.workunit.client.0.smithi088.stdout:9/664: stat d1/d8/d4a/d4f/daa/dbb 0 2024-02-19T19:06:26.792 INFO:tasks.workunit.client.0.smithi088.stdout:9/665: read d1/d8/f28 [3330064,121625] 0 2024-02-19T19:06:26.793 INFO:tasks.workunit.client.0.smithi088.stdout:9/666: chown d1/d8/d37/d54 2885417 1 2024-02-19T19:06:26.794 INFO:tasks.workunit.client.0.smithi088.stdout:9/667: truncate d1/d8/d4a/dac/f90 685592 0 2024-02-19T19:06:26.796 INFO:tasks.workunit.client.0.smithi088.stdout:9/668: chown d1/d8/d3e/d93/c31 3 1 2024-02-19T19:06:26.801 INFO:tasks.workunit.client.0.smithi088.stdout:9/669: dwrite d1/d8/d37/d54/f84 [4194304,4194304] 0 2024-02-19T19:06:26.806 INFO:tasks.workunit.client.0.smithi088.stdout:9/670: link d1/d8/d37/d54/f55 d1/d8/d3e/d93/d2d/d18/dc0/fd4 0 2024-02-19T19:06:26.809 INFO:tasks.workunit.client.0.smithi088.stdout:9/671: fsync d1/d8/d8d/d97/f53 0 2024-02-19T19:06:26.812 INFO:tasks.workunit.client.0.smithi088.stdout:9/672: rename d1/d8/d37/d54/f55 to d1/d8/d3e/d93/d5a/fd5 0 2024-02-19T19:06:26.815 INFO:tasks.workunit.client.0.smithi088.stdout:9/673: creat d1/d8/fd6 x:0 0 0 2024-02-19T19:06:26.817 INFO:tasks.workunit.client.0.smithi088.stdout:9/674: fsync d1/d8/d3e/d93/d2d/d18/d1f/fb4 0 2024-02-19T19:06:26.818 INFO:tasks.workunit.client.0.smithi088.stdout:9/675: mknod d1/d8/d3e/d93/d2d/d23/d79/cd7 0 2024-02-19T19:06:26.819 INFO:tasks.workunit.client.0.smithi088.stdout:9/676: dread - d1/d8/fd6 zero size 2024-02-19T19:06:26.821 INFO:tasks.workunit.client.0.smithi088.stdout:9/677: creat d1/d8/d4a/d4f/daa/fd8 x:0 0 0 2024-02-19T19:06:26.823 INFO:tasks.workunit.client.0.smithi088.stdout:9/678: rename d1/d8/f28 to d1/d8/d3e/d93/d2f/fd9 0 2024-02-19T19:06:26.824 INFO:tasks.workunit.client.0.smithi088.stdout:9/679: mkdir d1/d8/d4a/d4f/daa/dda 0 2024-02-19T19:06:26.826 INFO:tasks.workunit.client.0.smithi088.stdout:9/680: mkdir d1/d8/d8d/d97/ddb 0 2024-02-19T19:06:26.828 INFO:tasks.workunit.client.0.smithi088.stdout:9/681: creat d1/d8/fdc x:0 0 0 2024-02-19T19:06:26.828 INFO:tasks.workunit.client.0.smithi088.stdout:9/682: stat d1/d8/d3e/d93/d2d/d23/d79/f9d 0 2024-02-19T19:06:26.829 INFO:tasks.workunit.client.0.smithi088.stdout:9/683: write d1/d8/d3e/d93/d2d/d23/d79/f9d [83713,4835] 0 2024-02-19T19:06:26.834 INFO:tasks.workunit.client.0.smithi088.stdout:9/684: dwrite d1/f20 [0,4194304] 0 2024-02-19T19:06:26.845 INFO:tasks.workunit.client.0.smithi088.stdout:9/685: truncate d1/d8/d3e/d93/d2d/d23/f25 3582345 0 2024-02-19T19:06:26.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:26 smithi088 ceph-mon[88643]: Upgrade: Setting container_image for all mon 2024-02-19T19:06:26.857 INFO:tasks.workunit.client.0.smithi088.stdout:9/686: dwrite d1/d8/d8d/d97/f53 [0,4194304] 0 2024-02-19T19:06:26.858 INFO:tasks.workunit.client.0.smithi088.stdout:9/687: stat d1/d8/d4a/d4f/daa/dbb/lbd 0 2024-02-19T19:06:26.861 INFO:tasks.workunit.client.0.smithi088.stdout:9/688: mknod d1/d8/d4a/d4f/cdd 0 2024-02-19T19:06:26.863 INFO:tasks.workunit.client.0.smithi088.stdout:9/689: creat d1/d8/d3e/d93/d2d/d23/d79/fde x:0 0 0 2024-02-19T19:06:26.868 INFO:tasks.workunit.client.0.smithi088.stdout:9/690: dwrite d1/d8/d3e/d93/d2d/d23/d79/f86 [0,4194304] 0 2024-02-19T19:06:26.870 INFO:tasks.workunit.client.0.smithi088.stdout:9/691: fdatasync d1/d8/d4a/d4f/fad 0 2024-02-19T19:06:26.870 INFO:tasks.workunit.client.0.smithi088.stdout:9/692: readlink d1/d8/d8d/d97/l8a 0 2024-02-19T19:06:26.890 INFO:tasks.workunit.client.0.smithi088.stdout:9/693: dwrite d1/d8/d4a/dac/f6c [0,4194304] 0 2024-02-19T19:06:26.892 INFO:tasks.workunit.client.0.smithi088.stdout:9/694: readlink d1/d8/d3e/d93/d2d/d18/l68 0 2024-02-19T19:06:26.893 INFO:tasks.workunit.client.0.smithi088.stdout:9/695: write d1/d8/d3e/d93/d2d/d23/d79/f86 [4692519,93267] 0 2024-02-19T19:06:26.899 INFO:tasks.workunit.client.0.smithi088.stdout:9/696: dwrite d1/d8/d37/f9a [0,4194304] 0 2024-02-19T19:06:26.900 INFO:tasks.workunit.client.0.smithi088.stdout:9/697: fsync d1/d8/d4a/d4f/daa/fd8 0 2024-02-19T19:06:26.902 INFO:tasks.workunit.client.0.smithi088.stdout:9/698: mkdir d1/d8/d37/ddf 0 2024-02-19T19:06:26.916 INFO:tasks.workunit.client.0.smithi088.stdout:9/699: dwrite d1/d8/d3e/d93/d2d/d18/dc0/fd4 [0,4194304] 0 2024-02-19T19:06:26.923 INFO:tasks.workunit.client.0.smithi088.stdout:9/700: rename d1/d8/d3e/d93/d2f/fd9 to d1/d8/d3e/d93/d2d/d18/d1f/fe0 0 2024-02-19T19:06:26.924 INFO:tasks.workunit.client.0.smithi088.stdout:9/701: chown d1/d8/d3e/d93/d2d/d23/l30 839 1 2024-02-19T19:06:26.940 INFO:tasks.workunit.client.0.smithi088.stdout:9/702: dread d1/d8/d3e/d93/d2d/f16 [0,4194304] 0 2024-02-19T19:06:26.942 INFO:tasks.workunit.client.0.smithi088.stdout:9/703: mknod d1/d8/d3e/d93/d2d/d23/d79/dc3/ce1 0 2024-02-19T19:06:26.945 INFO:tasks.workunit.client.0.smithi088.stdout:9/704: dread d1/d8/d37/f9a [0,4194304] 0 2024-02-19T19:06:26.946 INFO:tasks.workunit.client.0.smithi088.stdout:9/705: chown d1/f7f 17 1 2024-02-19T19:06:26.948 INFO:tasks.workunit.client.0.smithi088.stdout:9/706: symlink d1/d8/d3e/d93/d2d/d18/d1f/db0/le2 0 2024-02-19T19:06:26.948 INFO:tasks.workunit.client.0.smithi088.stdout:9/707: chown d1/d8/d4a/d4f/daa/dda 6832 1 2024-02-19T19:06:26.950 INFO:tasks.workunit.client.0.smithi088.stdout:9/708: creat d1/d8/d3e/d93/d2d/d23/d79/dc8/fe3 x:0 0 0 2024-02-19T19:06:26.952 INFO:tasks.workunit.client.0.smithi088.stdout:9/709: mknod d1/d8/d3e/d93/d2d/d23/d79/d99/ce4 0 2024-02-19T19:06:26.954 INFO:tasks.workunit.client.0.smithi088.stdout:9/710: symlink d1/d8/d3e/d93/d5a/d94/le5 0 2024-02-19T19:06:26.965 INFO:tasks.workunit.client.0.smithi088.stdout:9/711: dread d1/f6 [0,4194304] 0 2024-02-19T19:06:26.970 INFO:tasks.workunit.client.0.smithi088.stdout:9/712: dwrite d1/d8/d4a/d4f/fc2 [0,4194304] 0 2024-02-19T19:06:26.972 INFO:tasks.workunit.client.0.smithi088.stdout:9/713: truncate d1/d8/d3e/d93/f43 3078088 0 2024-02-19T19:06:26.977 INFO:tasks.workunit.client.0.smithi088.stdout:9/714: dread d1/d8/d8d/fa5 [0,4194304] 0 2024-02-19T19:06:26.979 INFO:tasks.workunit.client.0.smithi088.stdout:9/715: truncate d1/d8/d3e/d93/d2d/d23/d79/f9d 545105 0 2024-02-19T19:06:26.980 INFO:tasks.workunit.client.0.smithi088.stdout:9/716: write d1/d8/f77 [843457,118081] 0 2024-02-19T19:06:26.985 INFO:tasks.workunit.client.0.smithi088.stdout:9/717: getdents d1/d8 0 2024-02-19T19:06:26.988 INFO:tasks.workunit.client.0.smithi088.stdout:9/718: getdents d1 0 2024-02-19T19:06:26.989 INFO:tasks.workunit.client.0.smithi088.stdout:9/719: chown d1/d8/d3e/d93/d2d/d23/d79/da6 0 1 2024-02-19T19:06:26.990 INFO:tasks.workunit.client.0.smithi088.stdout:9/720: symlink d1/d8/d8d/d97/le6 0 2024-02-19T19:06:26.992 INFO:tasks.workunit.client.0.smithi088.stdout:9/721: chown d1/d8/d3e/d93/d5a/d94/c67 422028 1 2024-02-19T19:06:26.994 INFO:tasks.workunit.client.0.smithi088.stdout:9/722: unlink d1/d8/d4a/dac/f6c 0 2024-02-19T19:06:26.999 INFO:tasks.workunit.client.0.smithi088.stdout:9/723: dread d1/d8/d3e/d93/d2d/d18/d1f/fb1 [0,4194304] 0 2024-02-19T19:06:26.999 INFO:tasks.workunit.client.0.smithi088.stdout:9/724: read - d1/d8/d3e/d93/d2f/f7b zero size 2024-02-19T19:06:27.005 INFO:tasks.workunit.client.0.smithi088.stdout:9/725: dwrite d1/d8/d3e/d93/f64 [0,4194304] 0 2024-02-19T19:06:27.011 INFO:tasks.workunit.client.0.smithi088.stdout:9/726: read d1/d8/d3e/d93/f43 [402814,3236] 0 2024-02-19T19:06:27.012 INFO:tasks.workunit.client.0.smithi088.stdout:9/727: write d1/fc [1683627,36036] 0 2024-02-19T19:06:27.018 INFO:tasks.workunit.client.0.smithi088.stdout:9/728: dwrite d1/d8/d3e/d93/d2d/f16 [4194304,4194304] 0 2024-02-19T19:06:27.024 INFO:tasks.workunit.client.0.smithi088.stdout:9/729: dwrite d1/d8/f42 [0,4194304] 0 2024-02-19T19:06:27.031 INFO:tasks.workunit.client.0.smithi088.stdout:5/688: sync 2024-02-19T19:06:27.036 INFO:tasks.workunit.client.0.smithi088.stdout:9/730: write d1/d8/d3e/d93/d2d/d18/d1f/f59 [3234021,7981] 0 2024-02-19T19:06:27.037 INFO:tasks.workunit.client.0.smithi088.stdout:5/689: dwrite d4/d7/fbe [0,4194304] 0 2024-02-19T19:06:27.041 INFO:tasks.workunit.client.0.smithi088.stdout:9/731: rename d1/d8/d3e/d93/d2d/d18/d1f/f2e to d1/d8/d3e/d93/d2d/d18/d92/fe7 0 2024-02-19T19:06:27.042 INFO:tasks.workunit.client.0.smithi088.stdout:9/732: stat d1/d8/d3e/d93/d5a/dc4/fd1 0 2024-02-19T19:06:27.044 INFO:tasks.workunit.client.0.smithi088.stdout:9/733: chown d1/d8/d37/l66 10 1 2024-02-19T19:06:27.044 INFO:tasks.workunit.client.0.smithi088.stdout:9/734: chown d1/d8/d4a/d4f/cdd 305 1 2024-02-19T19:06:27.045 INFO:tasks.workunit.client.0.smithi088.stdout:9/735: stat d1/d8/d3e/d93 0 2024-02-19T19:06:27.048 INFO:tasks.workunit.client.0.smithi088.stdout:9/736: unlink d1/d8/d8d/d97/f36 0 2024-02-19T19:06:27.049 INFO:tasks.workunit.client.0.smithi088.stdout:9/737: symlink d1/d8/d8d/le8 0 2024-02-19T19:06:27.054 INFO:tasks.workunit.client.0.smithi088.stdout:9/738: rename d1/d8/d3e/d93/d5a/d94/c67 to d1/d8/d3e/d93/d2d/d18/d92/ce9 0 2024-02-19T19:06:27.056 INFO:tasks.workunit.client.0.smithi088.stdout:9/739: mkdir d1/d8/d3e/d93/d2d/d23/d79/d99/dc5/dea 0 2024-02-19T19:06:27.060 INFO:tasks.workunit.client.0.smithi088.stdout:5/690: dread d4/d61/fa8 [0,4194304] 0 2024-02-19T19:06:27.062 INFO:tasks.workunit.client.0.smithi088.stdout:9/740: dwrite d1/d8/d3e/d93/d2f/f7b [0,4194304] 0 2024-02-19T19:06:27.067 INFO:tasks.workunit.client.0.smithi088.stdout:9/741: getdents d1/d8/d3e/d93/d5a/d94 0 2024-02-19T19:06:27.073 INFO:tasks.workunit.client.0.smithi088.stdout:9/742: rename d1/d8/d3e/d93/d2d/d18/d92/ce9 to d1/d8/d4a/d4f/daa/ceb 0 2024-02-19T19:06:27.074 INFO:tasks.workunit.client.0.smithi088.stdout:5/691: dwrite d4/d7/f8f [0,4194304] 0 2024-02-19T19:06:27.083 INFO:tasks.workunit.client.0.smithi088.stdout:9/743: write d1/d8/d3e/d93/d2d/d18/d1f/fe0 [3210248,97288] 0 2024-02-19T19:06:27.085 INFO:tasks.workunit.client.0.smithi088.stdout:9/744: rmdir d1/d8/d37/d54 39 2024-02-19T19:06:27.087 INFO:tasks.workunit.client.0.smithi088.stdout:9/745: symlink d1/d8/d4a/d4f/lec 0 2024-02-19T19:06:27.089 INFO:tasks.workunit.client.0.smithi088.stdout:5/692: dread d4/d1a/f7c [0,4194304] 0 2024-02-19T19:06:27.089 INFO:tasks.workunit.client.0.smithi088.stdout:9/746: mknod d1/d8/d3e/d93/d2d/d18/d92/ced 0 2024-02-19T19:06:27.091 INFO:tasks.workunit.client.0.smithi088.stdout:9/747: readlink d1/d8/d3e/d93/lae 0 2024-02-19T19:06:27.094 INFO:tasks.workunit.client.0.smithi088.stdout:9/748: dread d1/d8/d3e/d93/d2d/d23/d79/f9d [0,4194304] 0 2024-02-19T19:06:27.094 INFO:tasks.workunit.client.0.smithi088.stdout:5/693: link d4/d1a/d41/f95 d4/d1a/d41/de2/de7/fef 0 2024-02-19T19:06:27.096 INFO:tasks.workunit.client.0.smithi088.stdout:5/694: unlink d4/f9 0 2024-02-19T19:06:27.097 INFO:tasks.workunit.client.0.smithi088.stdout:9/749: dread d1/d8/d3e/d93/d2d/d18/d1f/fe0 [0,4194304] 0 2024-02-19T19:06:27.099 INFO:tasks.workunit.client.0.smithi088.stdout:9/750: truncate d1/d8/d4a/d4f/f7a 4296384 0 2024-02-19T19:06:27.101 INFO:tasks.workunit.client.0.smithi088.stdout:9/751: mknod d1/cee 0 2024-02-19T19:06:27.103 INFO:tasks.workunit.client.0.smithi088.stdout:5/695: dwrite d4/d61/d65/d9a/d9f/da0/fae [0,4194304] 0 2024-02-19T19:06:27.104 INFO:tasks.workunit.client.0.smithi088.stdout:5/696: chown d4/d61/d65/d9a/d9f/lc 6 1 2024-02-19T19:06:27.105 INFO:tasks.workunit.client.0.smithi088.stdout:9/752: fdatasync d1/d8/d3e/d93/d2d/d18/d1f/fe0 0 2024-02-19T19:06:27.106 INFO:tasks.workunit.client.0.smithi088.stdout:5/697: write d4/d1a/d41/dc1/dc2/fe3 [566876,60821] 0 2024-02-19T19:06:27.108 INFO:tasks.workunit.client.0.smithi088.stdout:9/753: chown d1/d8/d3e/d93/d2d/d23 4795320 1 2024-02-19T19:06:27.110 INFO:tasks.workunit.client.0.smithi088.stdout:5/698: creat d4/d61/d65/ff0 x:0 0 0 2024-02-19T19:06:27.112 INFO:tasks.workunit.client.0.smithi088.stdout:9/754: symlink d1/d8/d3e/d93/d2d/d23/d79/dc8/lef 0 2024-02-19T19:06:27.113 INFO:tasks.workunit.client.0.smithi088.stdout:5/699: chown d4/d61/d65/d9a/d9f/d57/f6b 403 1 2024-02-19T19:06:27.114 INFO:tasks.workunit.client.0.smithi088.stdout:5/700: write d4/d61/d65/d9a/d9f/da0/fb4 [829426,36671] 0 2024-02-19T19:06:27.117 INFO:tasks.workunit.client.0.smithi088.stdout:5/701: unlink d4/d1a/f1b 0 2024-02-19T19:06:27.119 INFO:tasks.workunit.client.0.smithi088.stdout:5/702: unlink d4/d61/d65/d9a/d9f/la4 0 2024-02-19T19:06:27.126 INFO:tasks.workunit.client.0.smithi088.stdout:9/755: dread d1/d8/d3e/d93/d2d/d18/d92/fe7 [0,4194304] 0 2024-02-19T19:06:27.128 INFO:tasks.workunit.client.0.smithi088.stdout:5/703: write d4/d61/d65/d9a/d9f/da0/fa9 [793646,76894] 0 2024-02-19T19:06:27.131 INFO:tasks.workunit.client.0.smithi088.stdout:5/704: fdatasync d4/d61/d65/d9a/d9f/d57/d84/f89 0 2024-02-19T19:06:27.132 INFO:tasks.workunit.client.0.smithi088.stdout:5/705: chown d4/fe 47 1 2024-02-19T19:06:27.132 INFO:tasks.workunit.client.0.smithi088.stdout:9/756: dread d1/f6 [0,4194304] 0 2024-02-19T19:06:27.132 INFO:tasks.workunit.client.0.smithi088.stdout:5/706: stat d4/d1a/d41/f7a 0 2024-02-19T19:06:27.135 INFO:tasks.workunit.client.0.smithi088.stdout:9/757: rename d1/d8/d4a/d4f/c5d to d1/d8/d3e/d93/d2d/dcb/cf0 0 2024-02-19T19:06:27.137 INFO:tasks.workunit.client.0.smithi088.stdout:9/758: write d1/d8/d8d/d97/f12 [3938835,61057] 0 2024-02-19T19:06:27.138 INFO:tasks.workunit.client.0.smithi088.stdout:9/759: fdatasync d1/d8/d3e/d93/d2d/d18/dc0/fd4 0 2024-02-19T19:06:27.139 INFO:tasks.workunit.client.0.smithi088.stdout:9/760: chown d1/cee 6 1 2024-02-19T19:06:27.139 INFO:tasks.workunit.client.0.smithi088.stdout:9/761: chown d1/d8/d4a/d4f 176700 1 2024-02-19T19:06:27.145 INFO:tasks.workunit.client.0.smithi088.stdout:5/707: dread d4/d61/d65/d9a/d9f/f24 [0,4194304] 0 2024-02-19T19:06:27.148 INFO:tasks.workunit.client.0.smithi088.stdout:5/708: rename d4/f42 to d4/d1a/d41/de2/de7/ff1 0 2024-02-19T19:06:27.148 INFO:tasks.workunit.client.0.smithi088.stdout:5/709: fsync d4/d1a/d41/dc1/fc7 0 2024-02-19T19:06:27.149 INFO:tasks.workunit.client.0.smithi088.stdout:5/710: chown d4/d61/d65/d9a/d9f/d57/d84/f89 1352 1 2024-02-19T19:06:27.152 INFO:tasks.workunit.client.0.smithi088.stdout:5/711: truncate d4/f11 3181874 0 2024-02-19T19:06:27.152 INFO:tasks.workunit.client.0.smithi088.stdout:9/762: dread d1/d8/d37/f9c [0,4194304] 0 2024-02-19T19:06:27.155 INFO:tasks.workunit.client.0.smithi088.stdout:5/712: mknod d4/cf2 0 2024-02-19T19:06:27.155 INFO:tasks.workunit.client.0.smithi088.stdout:9/763: symlink d1/d8/d3e/d93/d2d/d18/dc0/lf1 0 2024-02-19T19:06:27.156 INFO:tasks.workunit.client.0.smithi088.stdout:5/713: rename d4/d61/d65/d9a/d9f/d57 to d4/d61/d65/d9a/d9f/d57/df3 22 2024-02-19T19:06:27.160 INFO:tasks.workunit.client.0.smithi088.stdout:5/714: chown d4/d61/d65/d9a/d9f/da0/fae 9263358 1 2024-02-19T19:06:27.163 INFO:tasks.workunit.client.0.smithi088.stdout:5/715: dread d4/d1a/f6e [0,4194304] 0 2024-02-19T19:06:27.163 INFO:tasks.workunit.client.0.smithi088.stdout:9/764: rename d1/d8/d3e/d93/d2d/d23/d79 to d1/d8/d3e/d93/d2d/d18/d1f/df2 0 2024-02-19T19:06:27.168 INFO:tasks.workunit.client.0.smithi088.stdout:9/765: dread d1/d8/d3e/d93/d2d/d18/d1f/df2/f9d [0,4194304] 0 2024-02-19T19:06:27.169 INFO:tasks.workunit.client.0.smithi088.stdout:9/766: chown d1/d8 12716938 1 2024-02-19T19:06:27.169 INFO:tasks.workunit.client.0.smithi088.stdout:5/716: symlink d4/lf4 0 2024-02-19T19:06:27.170 INFO:tasks.workunit.client.0.smithi088.stdout:9/767: mkdir d1/d8/d8d/d97/ddb/df3 0 2024-02-19T19:06:27.172 INFO:tasks.workunit.client.0.smithi088.stdout:5/717: rename d4/d1a/d41/d5e/l83 to d4/d61/d65/lf5 0 2024-02-19T19:06:27.174 INFO:tasks.workunit.client.0.smithi088.stdout:9/768: rename d1/d8/d3e/d93/d2d/d18/f8f to d1/d8/d4a/d4f/daa/dbb/ff4 0 2024-02-19T19:06:27.174 INFO:tasks.workunit.client.0.smithi088.stdout:7/590: sync 2024-02-19T19:06:27.174 INFO:tasks.workunit.client.0.smithi088.stdout:6/568: sync 2024-02-19T19:06:27.176 INFO:tasks.workunit.client.0.smithi088.stdout:5/718: symlink d4/d61/d65/d9a/d9f/da0/lf6 0 2024-02-19T19:06:27.177 INFO:tasks.workunit.client.0.smithi088.stdout:9/769: mknod d1/d8/d3e/d93/d5a/dc4/cf5 0 2024-02-19T19:06:27.178 INFO:tasks.workunit.client.0.smithi088.stdout:9/770: write d1/d8/d8d/d97/f12 [5525005,72093] 0 2024-02-19T19:06:27.178 INFO:tasks.workunit.client.0.smithi088.stdout:6/569: mknod d5/de/d64/d82/d8c/cae 0 2024-02-19T19:06:27.180 INFO:tasks.workunit.client.0.smithi088.stdout:5/719: symlink d4/d75/deb/lf7 0 2024-02-19T19:06:27.182 INFO:tasks.workunit.client.0.smithi088.stdout:7/591: getdents d14/d1d/d62 0 2024-02-19T19:06:27.182 INFO:tasks.workunit.client.0.smithi088.stdout:5/720: readlink d4/d1a/l6f 0 2024-02-19T19:06:27.184 INFO:tasks.workunit.client.0.smithi088.stdout:5/721: truncate d4/d1a/f90 5164829 0 2024-02-19T19:06:27.185 INFO:tasks.workunit.client.0.smithi088.stdout:7/592: creat d14/d1d/d3c/d43/d5c/d9a/d92/fd4 x:0 0 0 2024-02-19T19:06:27.185 INFO:tasks.workunit.client.0.smithi088.stdout:6/570: link d5/d8/d2c/l35 d5/d8/d77/laf 0 2024-02-19T19:06:27.188 INFO:tasks.workunit.client.0.smithi088.stdout:9/771: rename d1/d8/d3e/d93/d2d/d18/d1f/fb1 to d1/d8/d3e/d93/d2d/d18/ff6 0 2024-02-19T19:06:27.190 INFO:tasks.workunit.client.0.smithi088.stdout:5/722: unlink d4/d61/d65/ff0 0 2024-02-19T19:06:27.191 INFO:tasks.workunit.client.0.smithi088.stdout:7/593: symlink d14/d1d/d3c/d43/d5c/ld5 0 2024-02-19T19:06:27.191 INFO:tasks.workunit.client.0.smithi088.stdout:7/594: chown d14/d1d/d3c/f63 13 1 2024-02-19T19:06:27.192 INFO:tasks.workunit.client.0.smithi088.stdout:6/571: fdatasync d5/de/d64/d82/d8c/d25/f5d 0 2024-02-19T19:06:27.195 INFO:tasks.workunit.client.0.smithi088.stdout:5/723: mkdir d4/d61/d65/d9a/d9f/df8 0 2024-02-19T19:06:27.196 INFO:tasks.workunit.client.0.smithi088.stdout:5/724: fsync d4/d1a/d41/dc1/fc7 0 2024-02-19T19:06:27.197 INFO:tasks.workunit.client.0.smithi088.stdout:7/595: dwrite d14/f19 [4194304,4194304] 0 2024-02-19T19:06:27.200 INFO:tasks.workunit.client.0.smithi088.stdout:6/572: creat d5/d8/d3a/da8/fb0 x:0 0 0 2024-02-19T19:06:27.201 INFO:tasks.workunit.client.0.smithi088.stdout:6/573: chown d5/d45/d9d 978112846 1 2024-02-19T19:06:27.201 INFO:tasks.workunit.client.0.smithi088.stdout:7/596: symlink d14/d1d/d3c/d43/d5c/d9a/ld6 0 2024-02-19T19:06:27.202 INFO:tasks.workunit.client.0.smithi088.stdout:7/597: readlink d14/l61 0 2024-02-19T19:06:27.204 INFO:tasks.workunit.client.0.smithi088.stdout:9/772: rmdir d1/d8/d4a/d4f/daa 39 2024-02-19T19:06:27.205 INFO:tasks.workunit.client.0.smithi088.stdout:5/725: dwrite d4/d61/d65/d9a/d9f/fdc [0,4194304] 0 2024-02-19T19:06:27.207 INFO:tasks.workunit.client.0.smithi088.stdout:6/574: mkdir d5/de/d60/db1 0 2024-02-19T19:06:27.209 INFO:tasks.workunit.client.0.smithi088.stdout:9/773: write d1/d8/d37/d54/f75 [869707,39906] 0 2024-02-19T19:06:27.213 INFO:tasks.workunit.client.0.smithi088.stdout:7/598: dwrite d14/d28/d3b/dd3/fce [0,4194304] 0 2024-02-19T19:06:27.215 INFO:tasks.workunit.client.0.smithi088.stdout:6/575: rename d5/d7/f4f to d5/d45/d9d/fb2 0 2024-02-19T19:06:27.217 INFO:tasks.workunit.client.0.smithi088.stdout:6/576: rmdir d5/d19/d70 39 2024-02-19T19:06:27.222 INFO:tasks.workunit.client.0.smithi088.stdout:7/599: dread d14/d28/d3b/f64 [0,4194304] 0 2024-02-19T19:06:27.223 INFO:tasks.workunit.client.0.smithi088.stdout:5/726: getdents d4/d61/d65/d9a/d9f 0 2024-02-19T19:06:27.224 INFO:tasks.workunit.client.0.smithi088.stdout:5/727: fsync d4/d7/fbe 0 2024-02-19T19:06:27.225 INFO:tasks.workunit.client.0.smithi088.stdout:6/577: creat d5/de/d64/d82/d8c/d25/fb3 x:0 0 0 2024-02-19T19:06:27.226 INFO:tasks.workunit.client.0.smithi088.stdout:5/728: symlink d4/d7/lf9 0 2024-02-19T19:06:27.228 INFO:tasks.workunit.client.0.smithi088.stdout:6/578: symlink d5/de/d64/lb4 0 2024-02-19T19:06:27.232 INFO:tasks.workunit.client.0.smithi088.stdout:6/579: symlink d5/de/d64/d7e/lb5 0 2024-02-19T19:06:27.233 INFO:tasks.workunit.client.0.smithi088.stdout:6/580: fsync d5/ff 0 2024-02-19T19:06:27.235 INFO:tasks.workunit.client.0.smithi088.stdout:7/600: truncate d14/d1d/d3c/f8e 3318742 0 2024-02-19T19:06:27.235 INFO:tasks.workunit.client.0.smithi088.stdout:5/729: getdents d4/d1a/d41/de2 0 2024-02-19T19:06:27.236 INFO:tasks.workunit.client.0.smithi088.stdout:6/581: mkdir d5/d8/d3a/d91/db6 0 2024-02-19T19:06:27.237 INFO:tasks.workunit.client.0.smithi088.stdout:5/730: chown d4/d61/d65/d9a/d9f/da0/lf6 23 1 2024-02-19T19:06:27.238 INFO:tasks.workunit.client.0.smithi088.stdout:6/582: write d5/d7/d32/d56/f94 [328673,20327] 0 2024-02-19T19:06:27.239 INFO:tasks.workunit.client.0.smithi088.stdout:5/731: write d4/d61/d65/fec [4893761,77647] 0 2024-02-19T19:06:27.246 INFO:tasks.workunit.client.0.smithi088.stdout:6/583: dwrite d5/d8/f7b [0,4194304] 0 2024-02-19T19:06:27.248 INFO:tasks.workunit.client.0.smithi088.stdout:5/732: dwrite d4/d7/f15 [0,4194304] 0 2024-02-19T19:06:27.248 INFO:tasks.workunit.client.0.smithi088.stdout:7/601: dwrite d14/d1d/d3c/f8e [0,4194304] 0 2024-02-19T19:06:27.257 INFO:tasks.workunit.client.0.smithi088.stdout:7/602: read d14/f30 [1824703,85405] 0 2024-02-19T19:06:27.257 INFO:tasks.workunit.client.0.smithi088.stdout:6/584: link d5/d8/l1d d5/de/d64/d8d/d95/lb7 0 2024-02-19T19:06:27.260 INFO:tasks.workunit.client.0.smithi088.stdout:7/603: write d14/f1f [3352072,90963] 0 2024-02-19T19:06:27.264 INFO:tasks.workunit.client.0.smithi088.stdout:7/604: chown d14/d1d/d3c/d43/f89 5197 1 2024-02-19T19:06:27.264 INFO:tasks.workunit.client.0.smithi088.stdout:5/733: dwrite d4/d61/d65/fd4 [0,4194304] 0 2024-02-19T19:06:27.264 INFO:tasks.workunit.client.0.smithi088.stdout:6/585: truncate d5/de/d60/f76 61604 0 2024-02-19T19:06:27.266 INFO:tasks.workunit.client.0.smithi088.stdout:6/586: mkdir d5/d8/d3a/db8 0 2024-02-19T19:06:27.269 INFO:tasks.workunit.client.0.smithi088.stdout:6/587: readlink d5/de/d64/d8d/d95/lb7 0 2024-02-19T19:06:27.271 INFO:tasks.workunit.client.0.smithi088.stdout:7/605: rename d14/d1d/d3c/d43/d5c/d9a/d92/la3 to d14/d27/d49/ld7 0 2024-02-19T19:06:27.273 INFO:tasks.workunit.client.0.smithi088.stdout:7/606: write d14/d1d/d3c/d43/d5c/d75/fa4 [570894,80530] 0 2024-02-19T19:06:27.275 INFO:tasks.workunit.client.0.smithi088.stdout:7/607: write d14/d1d/d3c/d43/d5c/f9f [1920291,49001] 0 2024-02-19T19:06:27.278 INFO:tasks.workunit.client.0.smithi088.stdout:6/588: dwrite d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:27.279 INFO:tasks.workunit.client.0.smithi088.stdout:7/608: dread d14/d28/d94/fa5 [0,4194304] 0 2024-02-19T19:06:27.283 INFO:tasks.workunit.client.0.smithi088.stdout:5/734: truncate d4/d1a/f90 2974951 0 2024-02-19T19:06:27.288 INFO:tasks.workunit.client.0.smithi088.stdout:6/589: dread d5/d7/f4b [0,4194304] 0 2024-02-19T19:06:27.296 INFO:tasks.workunit.client.0.smithi088.stdout:5/735: dread d4/d7/f63 [0,4194304] 0 2024-02-19T19:06:27.298 INFO:tasks.workunit.client.0.smithi088.stdout:5/736: write d4/d61/d65/d9a/d9f/d57/faf [3265577,71151] 0 2024-02-19T19:06:27.300 INFO:tasks.workunit.client.0.smithi088.stdout:5/737: mkdir d4/d1a/d41/dc1/dc2/dfa 0 2024-02-19T19:06:27.302 INFO:tasks.workunit.client.0.smithi088.stdout:5/738: mkdir d4/d61/d65/d9a/d9f/d57/d84/dfb 0 2024-02-19T19:06:27.303 INFO:tasks.workunit.client.0.smithi088.stdout:6/590: write d5/d8/d2c/f4c [258426,101654] 0 2024-02-19T19:06:27.303 INFO:tasks.workunit.client.0.smithi088.stdout:5/739: dread - d4/fcf zero size 2024-02-19T19:06:27.304 INFO:tasks.workunit.client.0.smithi088.stdout:6/591: chown d5/d7/d32 0 1 2024-02-19T19:06:27.307 INFO:tasks.workunit.client.0.smithi088.stdout:5/740: rename d4/d61/d65/d9a/d9f/c59 to d4/d1a/d41/dcb/cfc 0 2024-02-19T19:06:27.309 INFO:tasks.workunit.client.0.smithi088.stdout:6/592: link d5/de/d97/f9b d5/d8/d3a/d71/daa/fb9 0 2024-02-19T19:06:27.312 INFO:tasks.workunit.client.0.smithi088.stdout:7/609: dread d14/d1d/d3c/d43/d5c/d9a/faf [0,4194304] 0 2024-02-19T19:06:27.313 INFO:tasks.workunit.client.0.smithi088.stdout:6/593: dread d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:27.314 INFO:tasks.workunit.client.0.smithi088.stdout:7/610: creat d14/d1d/d3c/d43/db2/fd8 x:0 0 0 2024-02-19T19:06:27.316 INFO:tasks.workunit.client.0.smithi088.stdout:6/594: mkdir d5/de/d64/d8d/d95/dba 0 2024-02-19T19:06:27.318 INFO:tasks.workunit.client.0.smithi088.stdout:6/595: chown d5/d45/d9d/fb2 0 1 2024-02-19T19:06:27.319 INFO:tasks.workunit.client.0.smithi088.stdout:6/596: stat d5/d19/f5f 0 2024-02-19T19:06:27.324 INFO:tasks.workunit.client.0.smithi088.stdout:5/741: dwrite d4/d61/d65/d9a/d9f/d57/f6b [0,4194304] 0 2024-02-19T19:06:27.326 INFO:tasks.workunit.client.0.smithi088.stdout:5/742: stat d4/c34 0 2024-02-19T19:06:27.326 INFO:tasks.workunit.client.0.smithi088.stdout:5/743: readlink l3 0 2024-02-19T19:06:27.327 INFO:tasks.workunit.client.0.smithi088.stdout:5/744: fsync d4/d1a/d41/dc1/dc2/fd7 0 2024-02-19T19:06:27.330 INFO:tasks.workunit.client.0.smithi088.stdout:7/611: dread d14/d1d/f38 [4194304,4194304] 0 2024-02-19T19:06:27.333 INFO:tasks.workunit.client.0.smithi088.stdout:7/612: symlink d14/d1d/dbf/ld9 0 2024-02-19T19:06:27.333 INFO:tasks.workunit.client.0.smithi088.stdout:7/613: stat d14/d28/d65/f7e 0 2024-02-19T19:06:27.337 INFO:tasks.workunit.client.0.smithi088.stdout:7/614: unlink d14/d27/d49/ld7 0 2024-02-19T19:06:27.343 INFO:tasks.workunit.client.0.smithi088.stdout:7/615: dwrite d14/d1d/f38 [0,4194304] 0 2024-02-19T19:06:27.346 INFO:tasks.workunit.client.0.smithi088.stdout:7/616: creat d14/d28/d94/fda x:0 0 0 2024-02-19T19:06:27.348 INFO:tasks.workunit.client.0.smithi088.stdout:7/617: mkdir d14/d28/d94/ddb 0 2024-02-19T19:06:27.349 INFO:tasks.workunit.client.0.smithi088.stdout:5/745: dread d4/d61/d65/d9a/d9f/d57/faf [0,4194304] 0 2024-02-19T19:06:27.350 INFO:tasks.workunit.client.0.smithi088.stdout:7/618: rmdir d14/d27/d6b 39 2024-02-19T19:06:27.352 INFO:tasks.workunit.client.0.smithi088.stdout:7/619: fsync d14/d1d/f8a 0 2024-02-19T19:06:27.354 INFO:tasks.workunit.client.0.smithi088.stdout:5/746: creat d4/d75/de8/ffd x:0 0 0 2024-02-19T19:06:27.356 INFO:tasks.workunit.client.0.smithi088.stdout:5/747: creat d4/d1a/d41/dcb/ffe x:0 0 0 2024-02-19T19:06:27.359 INFO:tasks.workunit.client.0.smithi088.stdout:5/748: mkdir d4/d1a/d41/de2/de7/dff 0 2024-02-19T19:06:27.361 INFO:tasks.workunit.client.0.smithi088.stdout:7/620: write d14/f1c [5125653,110250] 0 2024-02-19T19:06:27.364 INFO:tasks.workunit.client.0.smithi088.stdout:7/621: readlink d14/d27/d6b/l86 0 2024-02-19T19:06:27.365 INFO:tasks.workunit.client.0.smithi088.stdout:7/622: symlink d14/d27/d49/d98/dcb/ldc 0 2024-02-19T19:06:27.368 INFO:tasks.workunit.client.0.smithi088.stdout:8/560: sync 2024-02-19T19:06:27.368 INFO:tasks.workunit.client.0.smithi088.stdout:3/676: sync 2024-02-19T19:06:27.368 INFO:tasks.workunit.client.0.smithi088.stdout:4/473: sync 2024-02-19T19:06:27.368 INFO:tasks.workunit.client.0.smithi088.stdout:0/538: sync 2024-02-19T19:06:27.368 INFO:tasks.workunit.client.0.smithi088.stdout:1/653: sync 2024-02-19T19:06:27.368 INFO:tasks.workunit.client.0.smithi088.stdout:2/450: sync 2024-02-19T19:06:27.370 INFO:tasks.workunit.client.0.smithi088.stdout:4/474: write d2/d8/f6b [1741137,90025] 0 2024-02-19T19:06:27.417 INFO:tasks.workunit.client.0.smithi088.stdout:0/539: truncate fe 2682449 0 2024-02-19T19:06:27.417 INFO:tasks.workunit.client.0.smithi088.stdout:3/677: dread - d4/d1a/d1e/d74/d7d/fe3 zero size 2024-02-19T19:06:27.418 INFO:tasks.workunit.client.0.smithi088.stdout:8/561: write d9/df/d3d/d50/d71/da1/d6c/d9a/f4d [1027856,30430] 0 2024-02-19T19:06:27.418 INFO:tasks.workunit.client.0.smithi088.stdout:7/623: dwrite d14/d28/d3b/f4b [4194304,4194304] 0 2024-02-19T19:06:27.418 INFO:tasks.workunit.client.0.smithi088.stdout:4/475: fsync d2/d8/f14 0 2024-02-19T19:06:27.422 INFO:tasks.workunit.client.0.smithi088.stdout:8/562: read d9/df/f27 [3450268,101097] 0 2024-02-19T19:06:27.423 INFO:tasks.workunit.client.0.smithi088.stdout:0/540: readlink d13/d62/d8a/d95/d5a/d25/d2a/d34/l45 0 2024-02-19T19:06:27.424 INFO:tasks.workunit.client.0.smithi088.stdout:4/476: chown d2/d32/d25/d26/d6d/d7e/d3a/l3d 0 1 2024-02-19T19:06:27.426 INFO:tasks.workunit.client.0.smithi088.stdout:1/654: getdents d6/d21/d64/d94/dab 0 2024-02-19T19:06:27.427 INFO:tasks.workunit.client.0.smithi088.stdout:7/624: mkdir d14/d27/ddd 0 2024-02-19T19:06:27.429 INFO:tasks.workunit.client.0.smithi088.stdout:2/451: dwrite d3/d14/f4d [0,4194304] 0 2024-02-19T19:06:27.431 INFO:tasks.workunit.client.0.smithi088.stdout:8/563: creat d9/df/d3d/d50/d71/da1/d70/fa7 x:0 0 0 2024-02-19T19:06:27.431 INFO:tasks.workunit.client.0.smithi088.stdout:4/477: rmdir d2/d32/d25/d26/d6d/d7e/d90 39 2024-02-19T19:06:27.435 INFO:tasks.workunit.client.0.smithi088.stdout:0/541: fsync d13/d62/d8a/d95/d5a/d25/d2a/d2f/f50 0 2024-02-19T19:06:27.437 INFO:tasks.workunit.client.0.smithi088.stdout:1/655: creat d6/d10/d60/db8/dcf/fd6 x:0 0 0 2024-02-19T19:06:27.437 INFO:tasks.workunit.client.0.smithi088.stdout:3/678: dread d4/d1a/d1e/d74/fa6 [0,4194304] 0 2024-02-19T19:06:27.439 INFO:tasks.workunit.client.0.smithi088.stdout:2/452: creat d3/d1b/d43/d61/d64/f8a x:0 0 0 2024-02-19T19:06:27.440 INFO:tasks.workunit.client.0.smithi088.stdout:2/453: chown d3/d1b/f1c 585 1 2024-02-19T19:06:27.440 INFO:tasks.workunit.client.0.smithi088.stdout:4/478: dwrite d2/d32/d9/f20 [0,4194304] 0 2024-02-19T19:06:27.441 INFO:tasks.workunit.client.0.smithi088.stdout:2/454: write d3/d1b/d43/f55 [3526037,49649] 0 2024-02-19T19:06:27.442 INFO:tasks.workunit.client.0.smithi088.stdout:8/564: rename d9/d6f to d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8 0 2024-02-19T19:06:27.447 INFO:tasks.workunit.client.0.smithi088.stdout:1/656: mknod d6/d21/d6a/d7d/cd7 0 2024-02-19T19:06:27.447 INFO:tasks.workunit.client.0.smithi088.stdout:3/679: fdatasync d4/d87/dbf/fc2 0 2024-02-19T19:06:27.453 INFO:tasks.workunit.client.0.smithi088.stdout:2/455: dwrite d3/d1f/f73 [0,4194304] 0 2024-02-19T19:06:27.458 INFO:tasks.workunit.client.0.smithi088.stdout:4/479: rename d2/d32/f18 to d2/d32/d25/f97 0 2024-02-19T19:06:27.463 INFO:tasks.workunit.client.0.smithi088.stdout:0/542: creat d13/d62/d8a/fbb x:0 0 0 2024-02-19T19:06:27.464 INFO:tasks.workunit.client.0.smithi088.stdout:8/565: mknod d9/ca9 0 2024-02-19T19:06:27.467 INFO:tasks.workunit.client.0.smithi088.stdout:4/480: rename d2/d32/d9/c2a to d2/d32/d25/d26/d6d/d7e/d3a/d77/c98 0 2024-02-19T19:06:27.467 INFO:tasks.workunit.client.0.smithi088.stdout:1/657: rmdir d6/d10/d60/db8 39 2024-02-19T19:06:27.468 INFO:tasks.workunit.client.0.smithi088.stdout:4/481: read d2/d8/f6b [1325874,37127] 0 2024-02-19T19:06:27.468 INFO:tasks.workunit.client.0.smithi088.stdout:1/658: fdatasync d6/d21/d6a/d7d/d7e/fb4 0 2024-02-19T19:06:27.469 INFO:tasks.workunit.client.0.smithi088.stdout:1/659: stat d6/d21/f36 0 2024-02-19T19:06:27.470 INFO:tasks.workunit.client.0.smithi088.stdout:0/543: mkdir d13/d62/d8a/d95/d5a/d25/d2a/dbc 0 2024-02-19T19:06:27.477 INFO:tasks.workunit.client.0.smithi088.stdout:0/544: creat d13/d62/d8a/d95/d5b/d76/fbd x:0 0 0 2024-02-19T19:06:27.478 INFO:tasks.workunit.client.0.smithi088.stdout:8/566: creat d9/df/d1c/d22/d9e/faa x:0 0 0 2024-02-19T19:06:27.482 INFO:tasks.workunit.client.0.smithi088.stdout:1/660: creat d6/d10/d60/db8/dcf/fd8 x:0 0 0 2024-02-19T19:06:27.484 INFO:tasks.workunit.client.0.smithi088.stdout:3/680: getdents d4/d22 0 2024-02-19T19:06:27.485 INFO:tasks.workunit.client.0.smithi088.stdout:8/567: mknod d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/cab 0 2024-02-19T19:06:27.486 INFO:tasks.workunit.client.0.smithi088.stdout:3/681: dread - d4/d1a/d1e/d31/dc5/dcf/da4/fe8 zero size 2024-02-19T19:06:27.487 INFO:tasks.workunit.client.0.smithi088.stdout:0/545: mkdir d13/d62/d8a/d95/d5b/dbe 0 2024-02-19T19:06:27.490 INFO:tasks.workunit.client.0.smithi088.stdout:1/661: mkdir d6/d10/d60/db8/dd9 0 2024-02-19T19:06:27.491 INFO:tasks.workunit.client.0.smithi088.stdout:7/625: dread d14/d27/d49/fa2 [0,4194304] 0 2024-02-19T19:06:27.494 INFO:tasks.workunit.client.0.smithi088.stdout:2/456: dread d3/d14/f5a [4194304,4194304] 0 2024-02-19T19:06:27.495 INFO:tasks.workunit.client.0.smithi088.stdout:8/568: mknod d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/d8e/cac 0 2024-02-19T19:06:27.497 INFO:tasks.workunit.client.0.smithi088.stdout:1/662: dread - d6/d21/d64/d94/d6e/f97 zero size 2024-02-19T19:06:27.498 INFO:tasks.workunit.client.0.smithi088.stdout:7/626: mknod d14/d1d/d3c/d43/d5c/cde 0 2024-02-19T19:06:27.500 INFO:tasks.workunit.client.0.smithi088.stdout:3/682: truncate d4/fd 1468449 0 2024-02-19T19:06:27.501 INFO:tasks.workunit.client.0.smithi088.stdout:0/546: mkdir d13/d62/d8a/d9c/da8/dab/db4/dbf 0 2024-02-19T19:06:27.502 INFO:tasks.workunit.client.0.smithi088.stdout:0/547: readlink d13/d62/d8a/d95/d5a/l56 0 2024-02-19T19:06:27.502 INFO:tasks.workunit.client.0.smithi088.stdout:8/569: creat d9/df/d3d/d50/d71/da1/d70/fad x:0 0 0 2024-02-19T19:06:27.504 INFO:tasks.workunit.client.0.smithi088.stdout:1/663: symlink d6/d21/d64/dc8/dcc/lda 0 2024-02-19T19:06:27.506 INFO:tasks.workunit.client.0.smithi088.stdout:3/683: creat d4/d22/d7a/ff0 x:0 0 0 2024-02-19T19:06:27.509 INFO:tasks.workunit.client.0.smithi088.stdout:0/548: creat d13/d62/d8a/d95/d96/fc0 x:0 0 0 2024-02-19T19:06:27.510 INFO:tasks.workunit.client.0.smithi088.stdout:0/549: stat d13/d62/d8a/d95/d5a/c68 0 2024-02-19T19:06:27.511 INFO:tasks.workunit.client.0.smithi088.stdout:2/457: truncate d3/f6 8016697 0 2024-02-19T19:06:27.513 INFO:tasks.workunit.client.0.smithi088.stdout:3/684: symlink d4/d1a/d1e/d31/d32/d55/lf1 0 2024-02-19T19:06:27.517 INFO:tasks.workunit.client.0.smithi088.stdout:9/774: sync 2024-02-19T19:06:27.518 INFO:tasks.workunit.client.0.smithi088.stdout:9/775: fsync d1/d8/d4a/dac/f90 0 2024-02-19T19:06:27.522 INFO:tasks.workunit.client.0.smithi088.stdout:2/458: creat d3/d1b/d43/d61/d86/f8b x:0 0 0 2024-02-19T19:06:27.524 INFO:tasks.workunit.client.0.smithi088.stdout:1/664: getdents d6/d21/d64/d94/d71 0 2024-02-19T19:06:27.525 INFO:tasks.workunit.client.0.smithi088.stdout:0/550: dread d13/d62/d8a/d95/d5a/f74 [0,4194304] 0 2024-02-19T19:06:27.525 INFO:tasks.workunit.client.0.smithi088.stdout:1/665: write d6/d21/d64/fb0 [71157,35869] 0 2024-02-19T19:06:27.530 INFO:tasks.workunit.client.0.smithi088.stdout:0/551: fdatasync f4 0 2024-02-19T19:06:27.531 INFO:tasks.workunit.client.0.smithi088.stdout:7/627: write d14/d1d/d3c/d43/f89 [782380,50250] 0 2024-02-19T19:06:27.536 INFO:tasks.workunit.client.0.smithi088.stdout:0/552: rmdir d13/d62/d8a/d95/d5b/d76 39 2024-02-19T19:06:27.539 INFO:tasks.workunit.client.0.smithi088.stdout:9/776: rmdir d1/d8/d37/ddf 0 2024-02-19T19:06:27.540 INFO:tasks.workunit.client.0.smithi088.stdout:7/628: truncate d14/f77 584142 0 2024-02-19T19:06:27.541 INFO:tasks.workunit.client.0.smithi088.stdout:1/666: mknod d6/d21/d64/d94/cdb 0 2024-02-19T19:06:27.542 INFO:tasks.workunit.client.0.smithi088.stdout:7/629: chown d14/d28/d94 0 1 2024-02-19T19:06:27.542 INFO:tasks.workunit.client.0.smithi088.stdout:3/685: link d4/d12/l76 d4/d87/dbf/lf2 0 2024-02-19T19:06:27.543 INFO:tasks.workunit.client.0.smithi088.stdout:8/570: dwrite d9/df/d1c/f33 [0,4194304] 0 2024-02-19T19:06:27.544 INFO:tasks.workunit.client.0.smithi088.stdout:7/630: fdatasync d14/d1d/d3c/f8e 0 2024-02-19T19:06:27.545 INFO:tasks.workunit.client.0.smithi088.stdout:1/667: truncate d6/d21/f6c 909346 0 2024-02-19T19:06:27.550 INFO:tasks.workunit.client.0.smithi088.stdout:2/459: truncate d3/f85 4585895 0 2024-02-19T19:06:27.552 INFO:tasks.workunit.client.0.smithi088.stdout:9/777: dwrite d1/d8/d3e/d93/d2d/d18/d1f/db0/fab [0,4194304] 0 2024-02-19T19:06:27.553 INFO:tasks.workunit.client.0.smithi088.stdout:0/553: dread d13/d62/d8a/d95/d5a/d25/d2a/d3c/f64 [0,4194304] 0 2024-02-19T19:06:27.556 INFO:tasks.workunit.client.0.smithi088.stdout:3/686: fdatasync d4/f16 0 2024-02-19T19:06:27.557 INFO:tasks.workunit.client.0.smithi088.stdout:2/460: creat d3/d1b/f8c x:0 0 0 2024-02-19T19:06:27.558 INFO:tasks.workunit.client.0.smithi088.stdout:3/687: write d4/d87/fb6 [4270823,55458] 0 2024-02-19T19:06:27.559 INFO:tasks.workunit.client.0.smithi088.stdout:0/554: write d13/d62/d8a/d95/d5a/d25/d2a/d2f/fb0 [538269,89806] 0 2024-02-19T19:06:27.562 INFO:tasks.workunit.client.0.smithi088.stdout:9/778: dwrite d1/d8/d4a/d4f/fad [0,4194304] 0 2024-02-19T19:06:27.564 INFO:tasks.workunit.client.0.smithi088.stdout:9/779: dread - d1/d8/d3e/d93/d2d/d18/d92/fd3 zero size 2024-02-19T19:06:27.567 INFO:tasks.workunit.client.0.smithi088.stdout:0/555: creat d13/d62/d8a/d9c/da8/dab/db4/fc1 x:0 0 0 2024-02-19T19:06:27.570 INFO:tasks.workunit.client.0.smithi088.stdout:7/631: write d14/d28/d3b/f4a [2902034,10263] 0 2024-02-19T19:06:27.570 INFO:tasks.workunit.client.0.smithi088.stdout:9/780: chown d1/ca 5476 1 2024-02-19T19:06:27.573 INFO:tasks.workunit.client.0.smithi088.stdout:8/571: write d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8/f89 [960679,55357] 0 2024-02-19T19:06:27.574 INFO:tasks.workunit.client.0.smithi088.stdout:0/556: write d13/d62/d8a/d95/d5a/d25/f8b [1969101,105205] 0 2024-02-19T19:06:27.576 INFO:tasks.workunit.client.0.smithi088.stdout:3/688: creat d4/d1a/d1e/d74/d7c/d97/ff3 x:0 0 0 2024-02-19T19:06:27.578 INFO:tasks.workunit.client.0.smithi088.stdout:2/461: getdents d3 0 2024-02-19T19:06:27.582 INFO:tasks.workunit.client.0.smithi088.stdout:1/668: dwrite d6/d10/f24 [0,4194304] 0 2024-02-19T19:06:27.586 INFO:tasks.workunit.client.0.smithi088.stdout:9/781: dwrite d1/d8/d3e/d93/d5a/dc4/fd1 [0,4194304] 0 2024-02-19T19:06:27.587 INFO:tasks.workunit.client.0.smithi088.stdout:3/689: truncate d4/ff 2705278 0 2024-02-19T19:06:27.588 INFO:tasks.workunit.client.0.smithi088.stdout:2/462: rmdir d3/d1f/d4f 39 2024-02-19T19:06:27.588 INFO:tasks.workunit.client.0.smithi088.stdout:0/557: truncate d13/d62/d8a/d95/d5b/d76/fbd 408286 0 2024-02-19T19:06:27.589 INFO:tasks.workunit.client.0.smithi088.stdout:0/558: chown d13/d62/c9f 39639727 1 2024-02-19T19:06:27.592 INFO:tasks.workunit.client.0.smithi088.stdout:3/690: stat d4/d1a/d1e/d31/d32/d55/fc4 0 2024-02-19T19:06:27.592 INFO:tasks.workunit.client.0.smithi088.stdout:0/559: stat d13/d62/d8a/d95/d5a/c42 0 2024-02-19T19:06:27.594 INFO:tasks.workunit.client.0.smithi088.stdout:8/572: getdents d9/df/d3d/d50/d71/da1 0 2024-02-19T19:06:27.596 INFO:tasks.workunit.client.0.smithi088.stdout:3/691: readlink d4/d1a/d1e/d31/d32/d55/lb1 0 2024-02-19T19:06:27.597 INFO:tasks.workunit.client.0.smithi088.stdout:1/669: creat d6/d10/d60/fdc x:0 0 0 2024-02-19T19:06:27.599 INFO:tasks.workunit.client.0.smithi088.stdout:9/782: dread d1/d8/d37/f9a [0,4194304] 0 2024-02-19T19:06:27.599 INFO:tasks.workunit.client.0.smithi088.stdout:0/560: write d13/d62/d8a/d95/d5a/d25/d2a/d34/f75 [272541,3871] 0 2024-02-19T19:06:27.602 INFO:tasks.workunit.client.0.smithi088.stdout:3/692: creat d4/d12/d33/dab/ff4 x:0 0 0 2024-02-19T19:06:27.603 INFO:tasks.workunit.client.0.smithi088.stdout:2/463: dread d3/d1f/f73 [0,4194304] 0 2024-02-19T19:06:27.604 INFO:tasks.workunit.client.0.smithi088.stdout:8/573: mkdir d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae 0 2024-02-19T19:06:27.606 INFO:tasks.workunit.client.0.smithi088.stdout:0/561: write d13/d62/d8a/d95/d5a/d25/d6d/f98 [2720892,128097] 0 2024-02-19T19:06:27.607 INFO:tasks.workunit.client.0.smithi088.stdout:3/693: chown d4/d22/dd5/cdc 164782 1 2024-02-19T19:06:27.607 INFO:tasks.workunit.client.0.smithi088.stdout:2/464: chown d3/l18 36808127 1 2024-02-19T19:06:27.609 INFO:tasks.workunit.client.0.smithi088.stdout:2/465: chown d3 701699817 1 2024-02-19T19:06:27.611 INFO:tasks.workunit.client.0.smithi088.stdout:7/632: dread f2 [0,4194304] 0 2024-02-19T19:06:27.613 INFO:tasks.workunit.client.0.smithi088.stdout:1/670: dwrite d6/d10/fc4 [0,4194304] 0 2024-02-19T19:06:27.615 INFO:tasks.workunit.client.0.smithi088.stdout:7/633: chown d14/d1d/d3c/d43/d5c/d9a 30 1 2024-02-19T19:06:27.616 INFO:tasks.workunit.client.0.smithi088.stdout:0/562: truncate d13/d62/d8a/d9c/da8/dab/db4/fc1 203818 0 2024-02-19T19:06:27.617 INFO:tasks.workunit.client.0.smithi088.stdout:3/694: rename d4/d1a/d1e/l37 to d4/d22/d81/lf5 0 2024-02-19T19:06:27.618 INFO:tasks.workunit.client.0.smithi088.stdout:0/563: write d13/d62/d8a/f79 [811487,16340] 0 2024-02-19T19:06:27.620 INFO:tasks.workunit.client.0.smithi088.stdout:7/634: write d14/d1d/d3c/d43/d5c/f79 [1901128,122864] 0 2024-02-19T19:06:27.622 INFO:tasks.workunit.client.0.smithi088.stdout:3/695: creat d4/d1a/d1e/d31/ff6 x:0 0 0 2024-02-19T19:06:27.623 INFO:tasks.workunit.client.0.smithi088.stdout:0/564: dread - d13/f5c zero size 2024-02-19T19:06:27.624 INFO:tasks.workunit.client.0.smithi088.stdout:7/635: mknod d14/d1d/d62/dcf/cdf 0 2024-02-19T19:06:27.624 INFO:tasks.workunit.client.0.smithi088.stdout:0/565: chown d13/d62/d8a/d95/d5a/d25 0 1 2024-02-19T19:06:27.626 INFO:tasks.workunit.client.0.smithi088.stdout:0/566: fsync d13/d62/d8a/d95/d5a/d25/d2a/d34/f77 0 2024-02-19T19:06:27.628 INFO:tasks.workunit.client.0.smithi088.stdout:3/696: creat d4/d1a/d1e/d74/ff7 x:0 0 0 2024-02-19T19:06:27.628 INFO:tasks.workunit.client.0.smithi088.stdout:7/636: creat d14/d28/fe0 x:0 0 0 2024-02-19T19:06:27.629 INFO:tasks.workunit.client.0.smithi088.stdout:0/567: write d13/d62/d8a/d95/d5a/d25/d2a/d34/f43 [1187327,117911] 0 2024-02-19T19:06:27.629 INFO:tasks.workunit.client.0.smithi088.stdout:9/783: dread d1/d8/d3e/d93/d2d/d23/f25 [0,4194304] 0 2024-02-19T19:06:27.631 INFO:tasks.workunit.client.0.smithi088.stdout:7/637: read d14/f17 [5377656,75101] 0 2024-02-19T19:06:27.633 INFO:tasks.workunit.client.0.smithi088.stdout:7/638: chown d14/lcc 565532624 1 2024-02-19T19:06:27.634 INFO:tasks.workunit.client.0.smithi088.stdout:3/697: rename d4/d1a/d1e/d31/d32/f50 to d4/d22/d7a/d8d/ff8 0 2024-02-19T19:06:27.638 INFO:tasks.workunit.client.0.smithi088.stdout:7/639: creat d14/d28/d3b/dd3/fe1 x:0 0 0 2024-02-19T19:06:27.640 INFO:tasks.workunit.client.0.smithi088.stdout:3/698: rename d4/d1a/d1e/d31/d32/d55/lb1 to d4/d87/lf9 0 2024-02-19T19:06:27.642 INFO:tasks.workunit.client.0.smithi088.stdout:7/640: mknod d14/d28/d94/ce2 0 2024-02-19T19:06:27.645 INFO:tasks.workunit.client.0.smithi088.stdout:2/466: dread d3/d1f/f32 [4194304,4194304] 0 2024-02-19T19:06:27.648 INFO:tasks.workunit.client.0.smithi088.stdout:9/784: link d1/d8/d8d/d97/l8a d1/d8/d4a/d4f/daa/dda/lf7 0 2024-02-19T19:06:27.649 INFO:tasks.workunit.client.0.smithi088.stdout:9/785: chown d1/d8/d3e/d93/d2d/d18/d92/fe7 1174566 1 2024-02-19T19:06:27.649 INFO:tasks.workunit.client.0.smithi088.stdout:7/641: symlink d14/d1d/d3c/d43/db2/dc1/le3 0 2024-02-19T19:06:27.651 INFO:tasks.workunit.client.0.smithi088.stdout:9/786: write d1/d8/d4a/d4f/fad [4010328,103779] 0 2024-02-19T19:06:27.653 INFO:tasks.workunit.client.0.smithi088.stdout:7/642: creat d14/d1d/d62/dcf/fe4 x:0 0 0 2024-02-19T19:06:27.655 INFO:tasks.workunit.client.0.smithi088.stdout:0/568: dread d13/d62/d8a/d95/d5a/d25/d6d/f98 [0,4194304] 0 2024-02-19T19:06:27.656 INFO:tasks.workunit.client.0.smithi088.stdout:7/643: truncate d14/f2a 4948673 0 2024-02-19T19:06:27.656 INFO:tasks.workunit.client.0.smithi088.stdout:9/787: write d1/d8/d3e/d93/d2d/f2a [3286118,57809] 0 2024-02-19T19:06:27.659 INFO:tasks.workunit.client.0.smithi088.stdout:2/467: dwrite d3/d1b/f1c [0,4194304] 0 2024-02-19T19:06:27.660 INFO:tasks.workunit.client.0.smithi088.stdout:7/644: write d14/d27/f36 [25086,89108] 0 2024-02-19T19:06:27.660 INFO:tasks.workunit.client.0.smithi088.stdout:9/788: truncate d1/d8/fdc 737789 0 2024-02-19T19:06:27.663 INFO:tasks.workunit.client.0.smithi088.stdout:0/569: rename d13/d62/d8a/l89 to d13/d81/lc2 0 2024-02-19T19:06:27.663 INFO:tasks.workunit.client.0.smithi088.stdout:2/468: write d3/d1b/d43/d61/d77/f7b [821744,68051] 0 2024-02-19T19:06:27.663 INFO:tasks.workunit.client.0.smithi088.stdout:3/699: dread d4/d1a/f24 [0,4194304] 0 2024-02-19T19:06:27.664 INFO:tasks.workunit.client.0.smithi088.stdout:7/645: chown d14/d1d/d3c/d43/d5c/d75/fad 65320 1 2024-02-19T19:06:27.667 INFO:tasks.workunit.client.0.smithi088.stdout:2/469: fsync d3/d1b/d43/d61/d77/f7b 0 2024-02-19T19:06:27.667 INFO:tasks.workunit.client.0.smithi088.stdout:0/570: creat d13/d62/d8a/d95/d96/fc3 x:0 0 0 2024-02-19T19:06:27.673 INFO:tasks.workunit.client.0.smithi088.stdout:7/646: dwrite d14/f1f [4194304,4194304] 0 2024-02-19T19:06:27.673 INFO:tasks.workunit.client.0.smithi088.stdout:2/470: symlink d3/d1b/d43/d74/l8d 0 2024-02-19T19:06:27.676 INFO:tasks.workunit.client.0.smithi088.stdout:9/789: getdents d1/d8/d37 0 2024-02-19T19:06:27.678 INFO:tasks.workunit.client.0.smithi088.stdout:9/790: truncate d1/d8/d37/f9c 5225794 0 2024-02-19T19:06:27.678 INFO:tasks.workunit.client.0.smithi088.stdout:0/571: creat d13/d62/d8a/d95/fc4 x:0 0 0 2024-02-19T19:06:27.684 INFO:tasks.workunit.client.0.smithi088.stdout:3/700: truncate d4/d12/f47 378032 0 2024-02-19T19:06:27.685 INFO:tasks.workunit.client.0.smithi088.stdout:7/647: write d14/f6e [916618,39284] 0 2024-02-19T19:06:27.687 INFO:tasks.workunit.client.0.smithi088.stdout:3/701: chown d4/ce 12145 1 2024-02-19T19:06:27.689 INFO:tasks.workunit.client.0.smithi088.stdout:9/791: dwrite d1/fc [0,4194304] 0 2024-02-19T19:06:27.689 INFO:tasks.workunit.client.0.smithi088.stdout:0/572: dwrite d13/d62/d8a/d95/d5b/fb5 [0,4194304] 0 2024-02-19T19:06:27.697 INFO:tasks.workunit.client.0.smithi088.stdout:9/792: read d1/d8/d3e/d93/d2d/d18/d1f/db0/fab [1415091,33102] 0 2024-02-19T19:06:27.697 INFO:tasks.workunit.client.0.smithi088.stdout:0/573: creat d13/d62/d8a/d95/d5a/d25/d2a/d3c/fc5 x:0 0 0 2024-02-19T19:06:27.703 INFO:tasks.workunit.client.0.smithi088.stdout:3/702: dwrite d4/d12/d33/d43/fbd [0,4194304] 0 2024-02-19T19:06:27.707 INFO:tasks.workunit.client.0.smithi088.stdout:0/574: mkdir d13/d62/d8a/d95/d5b/dbe/dc6 0 2024-02-19T19:06:27.707 INFO:tasks.workunit.client.0.smithi088.stdout:9/793: dwrite d1/fc [0,4194304] 0 2024-02-19T19:06:27.709 INFO:tasks.workunit.client.0.smithi088.stdout:3/703: creat d4/d1a/d1e/d31/d32/ffa x:0 0 0 2024-02-19T19:06:27.711 INFO:tasks.workunit.client.0.smithi088.stdout:9/794: rename d1/d8/d3e/d93/d2d to d1/d8/d3e/d93/d2d/d18/df8 22 2024-02-19T19:06:27.712 INFO:tasks.workunit.client.0.smithi088.stdout:2/471: dread d3/fb [0,4194304] 0 2024-02-19T19:06:27.713 INFO:tasks.workunit.client.0.smithi088.stdout:3/704: creat d4/d12/d33/d58/ffb x:0 0 0 2024-02-19T19:06:27.714 INFO:tasks.workunit.client.0.smithi088.stdout:0/575: truncate d13/f1b 554583 0 2024-02-19T19:06:27.716 INFO:tasks.workunit.client.0.smithi088.stdout:9/795: stat d1/d8/d3e/d93/d2d/d18/d1f/df2/d99/ce4 0 2024-02-19T19:06:27.717 INFO:tasks.workunit.client.0.smithi088.stdout:7/648: dread f6 [0,4194304] 0 2024-02-19T19:06:27.720 INFO:tasks.workunit.client.0.smithi088.stdout:3/705: mkdir d4/d12/d33/d43/dfc 0 2024-02-19T19:06:27.720 INFO:tasks.workunit.client.0.smithi088.stdout:2/472: creat d3/d1b/d43/d61/d77/f8e x:0 0 0 2024-02-19T19:06:27.721 INFO:tasks.workunit.client.0.smithi088.stdout:3/706: fsync d4/d12/d33/d58/f93 0 2024-02-19T19:06:27.724 INFO:tasks.workunit.client.0.smithi088.stdout:7/649: symlink d14/d1d/le5 0 2024-02-19T19:06:27.728 INFO:tasks.workunit.client.0.smithi088.stdout:0/576: link d13/l2e d13/d62/d8a/lc7 0 2024-02-19T19:06:27.728 INFO:tasks.workunit.client.0.smithi088.stdout:7/650: read d14/f22 [1197237,105135] 0 2024-02-19T19:06:27.733 INFO:tasks.workunit.client.0.smithi088.stdout:7/651: symlink d14/d28/le6 0 2024-02-19T19:06:27.734 INFO:tasks.workunit.client.0.smithi088.stdout:7/652: stat d14/d1d/d3c/d43/d5c/d9a 0 2024-02-19T19:06:27.734 INFO:tasks.workunit.client.0.smithi088.stdout:7/653: dread - d14/d28/fe0 zero size 2024-02-19T19:06:27.735 INFO:tasks.workunit.client.0.smithi088.stdout:2/473: rename d3/d1b/d43/d61/d77 to d3/d1b/d43/d8f 0 2024-02-19T19:06:27.738 INFO:tasks.workunit.client.0.smithi088.stdout:2/474: symlink d3/d1b/d43/d61/d64/l90 0 2024-02-19T19:06:27.740 INFO:tasks.workunit.client.0.smithi088.stdout:0/577: truncate d13/f6a 41782 0 2024-02-19T19:06:27.744 INFO:tasks.workunit.client.0.smithi088.stdout:2/475: rmdir d3/d1f 39 2024-02-19T19:06:27.744 INFO:tasks.workunit.client.0.smithi088.stdout:3/707: dwrite d4/d12/d33/f9e [0,4194304] 0 2024-02-19T19:06:27.744 INFO:tasks.workunit.client.0.smithi088.stdout:7/654: dwrite d14/d28/d3b/f4a [0,4194304] 0 2024-02-19T19:06:27.747 INFO:tasks.workunit.client.0.smithi088.stdout:0/578: symlink d13/d62/d8a/d95/d5b/lc8 0 2024-02-19T19:06:27.747 INFO:tasks.workunit.client.0.smithi088.stdout:9/796: dread d1/d8/d37/d54/f65 [0,4194304] 0 2024-02-19T19:06:27.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:27 smithi132 ceph-mon[70676]: pgmap v47: 65 pgs: 65 active+clean; 1.4 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 44 MiB/s rd, 173 MiB/s wr, 457 op/s 2024-02-19T19:06:27.757 INFO:tasks.workunit.client.0.smithi088.stdout:0/579: unlink d13/d62/d8a/d95/d5a/d25/d2a/d3c/l65 0 2024-02-19T19:06:27.758 INFO:tasks.workunit.client.0.smithi088.stdout:7/655: write d14/f29 [3626291,93189] 0 2024-02-19T19:06:27.762 INFO:tasks.workunit.client.0.smithi088.stdout:7/656: mkdir d14/d27/d49/d98/dcb/de7 0 2024-02-19T19:06:27.764 INFO:tasks.workunit.client.0.smithi088.stdout:9/797: rename d1/d8/d3e/fca to d1/d8/d3e/ff9 0 2024-02-19T19:06:27.765 INFO:tasks.workunit.client.0.smithi088.stdout:0/580: mknod d13/d62/d8a/d95/d5a/db6/cc9 0 2024-02-19T19:06:27.765 INFO:tasks.workunit.client.0.smithi088.stdout:9/798: fsync d1/d8/d3e/d93/d2d/d18/dc0/fd4 0 2024-02-19T19:06:27.766 INFO:tasks.workunit.client.0.smithi088.stdout:0/581: fsync d13/d62/d8a/d95/d5a/d25/f26 0 2024-02-19T19:06:27.767 INFO:tasks.workunit.client.0.smithi088.stdout:6/597: sync 2024-02-19T19:06:27.768 INFO:tasks.workunit.client.0.smithi088.stdout:6/598: write d5/de/d64/f66 [1019492,106849] 0 2024-02-19T19:06:27.769 INFO:tasks.workunit.client.0.smithi088.stdout:0/582: rmdir d13/d81 39 2024-02-19T19:06:27.770 INFO:tasks.workunit.client.0.smithi088.stdout:7/657: getdents d14/d28/d9c 0 2024-02-19T19:06:27.772 INFO:tasks.workunit.client.0.smithi088.stdout:6/599: write d5/d19/f4e [4264567,94521] 0 2024-02-19T19:06:27.772 INFO:tasks.workunit.client.0.smithi088.stdout:9/799: symlink d1/d8/d4a/d4f/daa/dda/lfa 0 2024-02-19T19:06:27.774 INFO:tasks.workunit.client.0.smithi088.stdout:7/658: mknod d14/d28/d65/ce8 0 2024-02-19T19:06:27.775 INFO:tasks.workunit.client.0.smithi088.stdout:0/583: write d13/d62/d8a/d95/d5b/d76/fb8 [900449,70566] 0 2024-02-19T19:06:27.777 INFO:tasks.workunit.client.0.smithi088.stdout:2/476: dwrite d3/f23 [0,4194304] 0 2024-02-19T19:06:27.777 INFO:tasks.workunit.client.0.smithi088.stdout:6/600: symlink d5/d8/d3a/d91/lbb 0 2024-02-19T19:06:27.779 INFO:tasks.workunit.client.0.smithi088.stdout:9/800: mkdir d1/d8/d4a/d4f/daa/dbb/dfb 0 2024-02-19T19:06:27.783 INFO:tasks.workunit.client.0.smithi088.stdout:7/659: rename d14/d1d/d3c/d43/f89 to d14/d27/dd2/fe9 0 2024-02-19T19:06:27.785 INFO:tasks.workunit.client.0.smithi088.stdout:0/584: rmdir d13/d62/d8a/d95/d5a/d25/d6d 39 2024-02-19T19:06:27.789 INFO:tasks.workunit.client.0.smithi088.stdout:9/801: dwrite d1/d8/fd6 [0,4194304] 0 2024-02-19T19:06:27.792 INFO:tasks.workunit.client.0.smithi088.stdout:0/585: creat d13/d62/d8a/d9c/da8/dab/fca x:0 0 0 2024-02-19T19:06:27.792 INFO:tasks.workunit.client.0.smithi088.stdout:2/477: write d3/d1f/d4f/f11 [3990156,2411] 0 2024-02-19T19:06:27.794 INFO:tasks.workunit.client.0.smithi088.stdout:7/660: dwrite d14/d1d/d3c/f8f [0,4194304] 0 2024-02-19T19:06:27.794 INFO:tasks.workunit.client.0.smithi088.stdout:9/802: mknod d1/d8/d3e/d93/d2d/d18/d1f/df2/dc8/cfc 0 2024-02-19T19:06:27.796 INFO:tasks.workunit.client.0.smithi088.stdout:6/601: dread d5/de/d64/d82/d8c/d25/f39 [0,4194304] 0 2024-02-19T19:06:27.801 INFO:tasks.workunit.client.0.smithi088.stdout:7/661: truncate d14/f54 2650541 0 2024-02-19T19:06:27.804 INFO:tasks.workunit.client.0.smithi088.stdout:0/586: mkdir d13/d62/d8a/d95/d5a/d25/dcb 0 2024-02-19T19:06:27.805 INFO:tasks.workunit.client.0.smithi088.stdout:6/602: dwrite d5/ff [4194304,4194304] 0 2024-02-19T19:06:27.806 INFO:tasks.workunit.client.0.smithi088.stdout:9/803: write d1/d8/d3e/d93/fa1 [2933750,59101] 0 2024-02-19T19:06:27.808 INFO:tasks.workunit.client.0.smithi088.stdout:9/804: chown d1/d8/d8d/d97/f12 492 1 2024-02-19T19:06:27.812 INFO:tasks.workunit.client.0.smithi088.stdout:7/662: symlink d14/d1d/d3c/d43/d5c/d75/lea 0 2024-02-19T19:06:27.814 INFO:tasks.workunit.client.0.smithi088.stdout:2/478: getdents d3/d1f/d4f/d59 0 2024-02-19T19:06:27.815 INFO:tasks.workunit.client.0.smithi088.stdout:7/663: chown d14/d1d/d3c/d43/d5c/d75/d76/c85 36313796 1 2024-02-19T19:06:27.816 INFO:tasks.workunit.client.0.smithi088.stdout:0/587: read d13/f6a [22395,69418] 0 2024-02-19T19:06:27.816 INFO:tasks.workunit.client.0.smithi088.stdout:7/664: write f13 [3423139,48115] 0 2024-02-19T19:06:27.817 INFO:tasks.workunit.client.0.smithi088.stdout:9/805: mknod d1/d8/d8d/cfd 0 2024-02-19T19:06:27.819 INFO:tasks.workunit.client.0.smithi088.stdout:2/479: chown d3/d1f/l76 32043122 1 2024-02-19T19:06:27.821 INFO:tasks.workunit.client.0.smithi088.stdout:6/603: rename d5/d8/d3a/d91 to d5/de/d64/d82/d8c/d25/dbc 0 2024-02-19T19:06:27.822 INFO:tasks.workunit.client.0.smithi088.stdout:0/588: symlink d13/d62/d8a/d9c/da8/lcc 0 2024-02-19T19:06:27.823 INFO:tasks.workunit.client.0.smithi088.stdout:7/665: dread d14/d28/d3b/dd3/fce [0,4194304] 0 2024-02-19T19:06:27.824 INFO:tasks.workunit.client.0.smithi088.stdout:2/480: dread - d3/d1b/d43/d61/d7e/f7f zero size 2024-02-19T19:06:27.826 INFO:tasks.workunit.client.0.smithi088.stdout:6/604: mkdir d5/d8/d3a/da8/dbd 0 2024-02-19T19:06:27.830 INFO:tasks.workunit.client.0.smithi088.stdout:7/666: creat d14/d27/d49/d98/dbe/feb x:0 0 0 2024-02-19T19:06:27.831 INFO:tasks.workunit.client.0.smithi088.stdout:9/806: dwrite d1/d8/d37/f9a [0,4194304] 0 2024-02-19T19:06:27.832 INFO:tasks.workunit.client.0.smithi088.stdout:6/605: creat d5/d8/d3a/fbe x:0 0 0 2024-02-19T19:06:27.832 INFO:tasks.workunit.client.0.smithi088.stdout:7/667: readlink d14/d27/l23 0 2024-02-19T19:06:27.832 INFO:tasks.workunit.client.0.smithi088.stdout:0/589: mkdir d13/d62/d8a/d95/dcd 0 2024-02-19T19:06:27.834 INFO:tasks.workunit.client.0.smithi088.stdout:6/606: write d5/d45/fab [323646,76558] 0 2024-02-19T19:06:27.838 INFO:tasks.workunit.client.0.smithi088.stdout:9/807: creat d1/d8/d3e/d93/d2d/dcb/ffe x:0 0 0 2024-02-19T19:06:27.842 INFO:tasks.workunit.client.0.smithi088.stdout:9/808: creat d1/d8/d4a/dac/fff x:0 0 0 2024-02-19T19:06:27.843 INFO:tasks.workunit.client.0.smithi088.stdout:0/590: creat d13/d62/d8a/d95/d5a/d25/d2a/d3c/d86/fce x:0 0 0 2024-02-19T19:06:27.846 INFO:tasks.workunit.client.0.smithi088.stdout:0/591: symlink d13/d62/lcf 0 2024-02-19T19:06:27.847 INFO:tasks.workunit.client.0.smithi088.stdout:9/809: mkdir d1/d8/d3e/d93/d2d/d18/d1f/df2/d99/dc5/dea/d100 0 2024-02-19T19:06:27.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:27 smithi088 ceph-mon[88643]: pgmap v47: 65 pgs: 65 active+clean; 1.4 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 44 MiB/s rd, 173 MiB/s wr, 457 op/s 2024-02-19T19:06:27.850 INFO:tasks.workunit.client.0.smithi088.stdout:7/668: dwrite d14/f4f [4194304,4194304] 0 2024-02-19T19:06:27.855 INFO:tasks.workunit.client.0.smithi088.stdout:9/810: creat d1/d8/d3e/d93/d5a/d94/d74/f101 x:0 0 0 2024-02-19T19:06:27.856 INFO:tasks.workunit.client.0.smithi088.stdout:2/481: dread d3/d1f/d5d/f72 [0,4194304] 0 2024-02-19T19:06:27.857 INFO:tasks.workunit.client.0.smithi088.stdout:6/607: dread d5/d8/f3f [4194304,4194304] 0 2024-02-19T19:06:27.859 INFO:tasks.workunit.client.0.smithi088.stdout:7/669: creat d14/d28/d65/fec x:0 0 0 2024-02-19T19:06:27.860 INFO:tasks.workunit.client.0.smithi088.stdout:0/592: dwrite d13/d62/d8a/d95/d5a/d25/d6d/fa0 [0,4194304] 0 2024-02-19T19:06:27.864 INFO:tasks.workunit.client.0.smithi088.stdout:0/593: unlink d13/fa5 0 2024-02-19T19:06:27.867 INFO:tasks.workunit.client.0.smithi088.stdout:6/608: unlink d5/d8/d3a/fbe 0 2024-02-19T19:06:27.871 INFO:tasks.workunit.client.0.smithi088.stdout:6/609: chown d5/d8/d2c/l35 11728 1 2024-02-19T19:06:27.872 INFO:tasks.workunit.client.0.smithi088.stdout:7/670: creat d14/d27/d49/fed x:0 0 0 2024-02-19T19:06:27.873 INFO:tasks.workunit.client.0.smithi088.stdout:0/594: getdents d13/d62/d8a/d95/d5b/dbe 0 2024-02-19T19:06:27.874 INFO:tasks.workunit.client.0.smithi088.stdout:7/671: write d14/f2e [507658,92243] 0 2024-02-19T19:06:27.877 INFO:tasks.workunit.client.0.smithi088.stdout:0/595: mkdir d13/d62/d8a/d9c/da8/dab/dd0 0 2024-02-19T19:06:27.878 INFO:tasks.workunit.client.0.smithi088.stdout:7/672: symlink d14/d27/d49/d98/lee 0 2024-02-19T19:06:27.879 INFO:tasks.workunit.client.0.smithi088.stdout:6/610: dwrite d5/d19/f6d [0,4194304] 0 2024-02-19T19:06:27.880 INFO:tasks.workunit.client.0.smithi088.stdout:0/596: fdatasync d13/d62/d8a/d95/d5a/d25/d2a/d34/f4d 0 2024-02-19T19:06:27.883 INFO:tasks.workunit.client.0.smithi088.stdout:2/482: dread d3/d1b/f67 [0,4194304] 0 2024-02-19T19:06:27.886 INFO:tasks.workunit.client.0.smithi088.stdout:9/811: truncate d1/f2b 1720865 0 2024-02-19T19:06:27.890 INFO:tasks.workunit.client.0.smithi088.stdout:6/611: dwrite d5/de/f22 [0,4194304] 0 2024-02-19T19:06:27.892 INFO:tasks.workunit.client.0.smithi088.stdout:0/597: dwrite d13/d62/d8a/d95/d5b/fb5 [4194304,4194304] 0 2024-02-19T19:06:27.893 INFO:tasks.workunit.client.0.smithi088.stdout:2/483: creat d3/d14/f91 x:0 0 0 2024-02-19T19:06:27.895 INFO:tasks.workunit.client.0.smithi088.stdout:0/598: chown d13/d62/d8a/d95/d5a/l56 11863 1 2024-02-19T19:06:27.899 INFO:tasks.workunit.client.0.smithi088.stdout:7/673: getdents d14 0 2024-02-19T19:06:27.899 INFO:tasks.workunit.client.0.smithi088.stdout:6/612: mkdir d5/d7/d32/d56/dbf 0 2024-02-19T19:06:27.904 INFO:tasks.workunit.client.0.smithi088.stdout:7/674: dread - d14/d27/d6b/fa7 zero size 2024-02-19T19:06:27.904 INFO:tasks.workunit.client.0.smithi088.stdout:7/675: dread - d14/d1d/d3c/d43/d5c/d9a/d92/fd4 zero size 2024-02-19T19:06:27.904 INFO:tasks.workunit.client.0.smithi088.stdout:6/613: write d5/d19/d26/f27 [1537051,54706] 0 2024-02-19T19:06:27.905 INFO:tasks.workunit.client.0.smithi088.stdout:2/484: rename d3/d14/l2c to d3/d1f/d4f/d59/l92 0 2024-02-19T19:06:27.909 INFO:tasks.workunit.client.0.smithi088.stdout:0/599: link d13/d62/d8a/d95/d5a/d25/d2a/d3c/d86/fce d13/d62/d8a/d9c/da8/dab/fd1 0 2024-02-19T19:06:27.911 INFO:tasks.workunit.client.0.smithi088.stdout:7/676: symlink d14/d27/d49/lef 0 2024-02-19T19:06:27.912 INFO:tasks.workunit.client.0.smithi088.stdout:6/614: dread d5/d8/d3a/f63 [0,4194304] 0 2024-02-19T19:06:27.914 INFO:tasks.workunit.client.0.smithi088.stdout:7/677: rmdir d14/d1d/d62/dcf 39 2024-02-19T19:06:27.915 INFO:tasks.workunit.client.0.smithi088.stdout:2/485: dwrite d3/d1f/d4f/d59/f79 [0,4194304] 0 2024-02-19T19:06:27.916 INFO:tasks.workunit.client.0.smithi088.stdout:6/615: symlink d5/d19/d26/d72/lc0 0 2024-02-19T19:06:27.918 INFO:tasks.workunit.client.0.smithi088.stdout:7/678: symlink d14/d1d/dbf/lf0 0 2024-02-19T19:06:27.921 INFO:tasks.workunit.client.0.smithi088.stdout:2/486: mkdir d3/d14/d93 0 2024-02-19T19:06:27.922 INFO:tasks.workunit.client.0.smithi088.stdout:0/600: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d34/f77 [0,4194304] 0 2024-02-19T19:06:27.924 INFO:tasks.workunit.client.0.smithi088.stdout:2/487: write d3/d1b/d43/d61/d64/f6a [13244,47290] 0 2024-02-19T19:06:27.926 INFO:tasks.workunit.client.0.smithi088.stdout:6/616: rename d5/d8/d77/fa7 to d5/de/d64/d8d/fc1 0 2024-02-19T19:06:27.927 INFO:tasks.workunit.client.0.smithi088.stdout:0/601: unlink d13/d62/d8a/d95/c67 0 2024-02-19T19:06:27.929 INFO:tasks.workunit.client.0.smithi088.stdout:0/602: write d13/d62/d8a/d95/f6c [846135,105459] 0 2024-02-19T19:06:27.930 INFO:tasks.workunit.client.0.smithi088.stdout:6/617: fsync d5/d19/d70/f73 0 2024-02-19T19:06:27.932 INFO:tasks.workunit.client.0.smithi088.stdout:2/488: mkdir d3/d1b/d43/d8f/d94 0 2024-02-19T19:06:27.933 INFO:tasks.workunit.client.0.smithi088.stdout:9/812: dread d1/d8/d3e/d93/f43 [0,4194304] 0 2024-02-19T19:06:27.935 INFO:tasks.workunit.client.0.smithi088.stdout:6/618: write d5/de/d64/d82/d8c/f6e [6716242,19095] 0 2024-02-19T19:06:27.937 INFO:tasks.workunit.client.0.smithi088.stdout:2/489: mknod d3/d1f/c95 0 2024-02-19T19:06:27.940 INFO:tasks.workunit.client.0.smithi088.stdout:2/490: mknod d3/d1b/d43/d61/d64/c96 0 2024-02-19T19:06:27.944 INFO:tasks.workunit.client.0.smithi088.stdout:0/603: getdents d13 0 2024-02-19T19:06:27.947 INFO:tasks.workunit.client.0.smithi088.stdout:9/813: getdents d1/d8/d4a/d4f/daa/dda 0 2024-02-19T19:06:27.949 INFO:tasks.workunit.client.0.smithi088.stdout:0/604: stat d13/d62/d8a/d95/d5a/d25/c38 0 2024-02-19T19:06:27.949 INFO:tasks.workunit.client.0.smithi088.stdout:2/491: dwrite d3/d1f/d4f/f11 [4194304,4194304] 0 2024-02-19T19:06:27.951 INFO:tasks.workunit.client.0.smithi088.stdout:0/605: chown d13/d62/d8a/d95/d5a/d25/d2a/d2f/f41 730 1 2024-02-19T19:06:27.954 INFO:tasks.workunit.client.0.smithi088.stdout:6/619: dread d5/d7/f14 [0,4194304] 0 2024-02-19T19:06:27.955 INFO:tasks.workunit.client.0.smithi088.stdout:6/620: chown d5/de/d64/d8d/d95 0 1 2024-02-19T19:06:27.956 INFO:tasks.workunit.client.0.smithi088.stdout:2/492: rename d3/l28 to d3/d1b/d43/d8f/d94/l97 0 2024-02-19T19:06:27.959 INFO:tasks.workunit.client.0.smithi088.stdout:0/606: rmdir d13/d62/d8a/d95/d5a/d25/d2a/dbc 0 2024-02-19T19:06:27.960 INFO:tasks.workunit.client.0.smithi088.stdout:9/814: dwrite d1/d8/d3e/d93/d2d/d23/f25 [0,4194304] 0 2024-02-19T19:06:27.961 INFO:tasks.workunit.client.0.smithi088.stdout:0/607: chown d13/d62/d8a/d95/d5a/d25/d2a/d3c/fad 118247 1 2024-02-19T19:06:27.962 INFO:tasks.workunit.client.0.smithi088.stdout:6/621: rename d5/de/d64/d82/d8c/d25 to d5/d8/d3a/d71/daa/dc2 0 2024-02-19T19:06:27.968 INFO:tasks.workunit.client.0.smithi088.stdout:6/622: mkdir d5/d8/d3a/d71/dc3 0 2024-02-19T19:06:27.969 INFO:tasks.workunit.client.0.smithi088.stdout:6/623: write d5/de/d64/d82/d8c/f59 [67401,72886] 0 2024-02-19T19:06:27.970 INFO:tasks.workunit.client.0.smithi088.stdout:6/624: readlink d5/d19/l90 0 2024-02-19T19:06:27.971 INFO:tasks.workunit.client.0.smithi088.stdout:0/608: creat d13/d62/d8a/fd2 x:0 0 0 2024-02-19T19:06:27.971 INFO:tasks.workunit.client.0.smithi088.stdout:2/493: symlink d3/l98 0 2024-02-19T19:06:27.972 INFO:tasks.workunit.client.0.smithi088.stdout:6/625: dread - d5/d8/d3a/d71/daa/dc2/dbc/fa4 zero size 2024-02-19T19:06:27.974 INFO:tasks.workunit.client.0.smithi088.stdout:2/494: write d3/d1b/f2e [302744,77178] 0 2024-02-19T19:06:27.975 INFO:tasks.workunit.client.0.smithi088.stdout:2/495: chown d3/d1b 5 1 2024-02-19T19:06:27.976 INFO:tasks.workunit.client.0.smithi088.stdout:2/496: chown d3/d1b/d43/d61/d86 38 1 2024-02-19T19:06:27.976 INFO:tasks.workunit.client.0.smithi088.stdout:9/815: truncate d1/d8/d37/f9a 2240029 0 2024-02-19T19:06:27.977 INFO:tasks.workunit.client.0.smithi088.stdout:6/626: symlink d5/de/d64/d8d/d9e/lc4 0 2024-02-19T19:06:27.978 INFO:tasks.workunit.client.0.smithi088.stdout:2/497: readlink d3/d14/l53 0 2024-02-19T19:06:27.979 INFO:tasks.workunit.client.0.smithi088.stdout:9/816: mknod d1/d8/d4a/d4f/daa/dbb/c102 0 2024-02-19T19:06:27.981 INFO:tasks.workunit.client.0.smithi088.stdout:6/627: symlink d5/d19/d70/lc5 0 2024-02-19T19:06:27.982 INFO:tasks.workunit.client.0.smithi088.stdout:2/498: symlink d3/d1b/d43/l99 0 2024-02-19T19:06:27.986 INFO:tasks.workunit.client.0.smithi088.stdout:6/628: creat d5/de/fc6 x:0 0 0 2024-02-19T19:06:27.988 INFO:tasks.workunit.client.0.smithi088.stdout:9/817: dwrite d1/d8/fd6 [0,4194304] 0 2024-02-19T19:06:27.989 INFO:tasks.workunit.client.0.smithi088.stdout:0/609: dwrite d13/f1a [0,4194304] 0 2024-02-19T19:06:27.992 INFO:tasks.workunit.client.0.smithi088.stdout:0/610: chown d13/d62/d8a/d95/d5b/fb5 220 1 2024-02-19T19:06:27.993 INFO:tasks.workunit.client.0.smithi088.stdout:2/499: dwrite d3/d1b/d43/d74/f82 [0,4194304] 0 2024-02-19T19:06:27.995 INFO:tasks.workunit.client.0.smithi088.stdout:9/818: write d1/f20 [126602,18392] 0 2024-02-19T19:06:27.998 INFO:tasks.workunit.client.0.smithi088.stdout:6/629: mkdir d5/d8/d3a/da8/dbd/dc7 0 2024-02-19T19:06:28.000 INFO:tasks.workunit.client.0.smithi088.stdout:2/500: dread d3/d14/f5a [4194304,4194304] 0 2024-02-19T19:06:28.000 INFO:tasks.workunit.client.0.smithi088.stdout:0/611: rename d13/d62/d8a/d95/d5a/d25/d2a/d34/f4d to d13/d62/d8a/d95/d5b/dbe/fd3 0 2024-02-19T19:06:28.003 INFO:tasks.workunit.client.0.smithi088.stdout:2/501: chown d3/d1f/l76 1516913004 1 2024-02-19T19:06:28.004 INFO:tasks.workunit.client.0.smithi088.stdout:9/819: link d1/f20 d1/d8/d4a/f103 0 2024-02-19T19:06:28.005 INFO:tasks.workunit.client.0.smithi088.stdout:6/630: mknod d5/d19/cc8 0 2024-02-19T19:06:28.006 INFO:tasks.workunit.client.0.smithi088.stdout:6/631: fdatasync d5/d19/d26/f44 0 2024-02-19T19:06:28.007 INFO:tasks.workunit.client.0.smithi088.stdout:0/612: mknod d13/d62/d8a/d95/d5a/db6/cd4 0 2024-02-19T19:06:28.008 INFO:tasks.workunit.client.0.smithi088.stdout:2/502: read d3/f23 [3801018,13638] 0 2024-02-19T19:06:28.011 INFO:tasks.workunit.client.0.smithi088.stdout:9/820: rename d1/d8/d4a/d4f/daa/dbb/dfb to d1/d8/d3e/d93/d5a/d94/d104 0 2024-02-19T19:06:28.012 INFO:tasks.workunit.client.0.smithi088.stdout:2/503: fsync d3/d14/f56 0 2024-02-19T19:06:28.015 INFO:tasks.workunit.client.0.smithi088.stdout:6/632: mkdir d5/d7/dc9 0 2024-02-19T19:06:28.017 INFO:tasks.workunit.client.0.smithi088.stdout:9/821: mknod d1/d8/d3e/d93/d2d/d18/dc0/c105 0 2024-02-19T19:06:28.017 INFO:tasks.workunit.client.0.smithi088.stdout:0/613: rmdir d13/d62/d8a/d95/d5a/d25/d2a 39 2024-02-19T19:06:28.019 INFO:tasks.workunit.client.0.smithi088.stdout:6/633: chown d5/de/f18 13 1 2024-02-19T19:06:28.019 INFO:tasks.workunit.client.0.smithi088.stdout:9/822: truncate d1/d8/d8d/d97/f53 4218435 0 2024-02-19T19:06:28.021 INFO:tasks.workunit.client.0.smithi088.stdout:0/614: creat d13/d62/d8a/d95/d5a/d25/d6d/fd5 x:0 0 0 2024-02-19T19:06:28.022 INFO:tasks.workunit.client.0.smithi088.stdout:6/634: rmdir d5/de/d97 39 2024-02-19T19:06:28.024 INFO:tasks.workunit.client.0.smithi088.stdout:9/823: unlink d1/d8/d3e/d93/d2d/d18/d1f/df2/dc8/fe3 0 2024-02-19T19:06:28.029 INFO:tasks.workunit.client.0.smithi088.stdout:0/615: rename d13/d62/d8a/d95/d5a/d25/dcb to d13/d62/d8a/d95/d5a/d25/dd6 0 2024-02-19T19:06:28.031 INFO:tasks.workunit.client.0.smithi088.stdout:0/616: rmdir d13/d62/d8a/d9c/da8/dab 39 2024-02-19T19:06:28.037 INFO:tasks.workunit.client.0.smithi088.stdout:0/617: rename d13/f1a to d13/d62/d8a/d95/d5a/db6/fd7 0 2024-02-19T19:06:28.038 INFO:tasks.workunit.client.0.smithi088.stdout:2/504: write d3/f6 [1141809,38314] 0 2024-02-19T19:06:28.039 INFO:tasks.workunit.client.0.smithi088.stdout:2/505: readlink d3/d1f/d4f/l3d 0 2024-02-19T19:06:28.041 INFO:tasks.workunit.client.0.smithi088.stdout:2/506: fdatasync d3/d1b/f67 0 2024-02-19T19:06:28.041 INFO:tasks.workunit.client.0.smithi088.stdout:6/635: dread d5/ff [0,4194304] 0 2024-02-19T19:06:28.043 INFO:tasks.workunit.client.0.smithi088.stdout:2/507: chown d3/l13 177191704 1 2024-02-19T19:06:28.044 INFO:tasks.workunit.client.0.smithi088.stdout:2/508: chown d3/d14/c80 0 1 2024-02-19T19:06:28.045 INFO:tasks.workunit.client.0.smithi088.stdout:2/509: readlink d3/d14/l60 0 2024-02-19T19:06:28.047 INFO:tasks.workunit.client.0.smithi088.stdout:9/824: dread d1/d8/d3e/d93/d2d/f16 [4194304,4194304] 0 2024-02-19T19:06:28.048 INFO:tasks.workunit.client.0.smithi088.stdout:2/510: chown d3/d1b/f5b 13 1 2024-02-19T19:06:28.048 INFO:tasks.workunit.client.0.smithi088.stdout:0/618: fsync d13/d62/d8a/d95/d5a/d25/d6d/f6e 0 2024-02-19T19:06:28.049 INFO:tasks.workunit.client.0.smithi088.stdout:6/636: dwrite d5/d19/d26/f27 [4194304,4194304] 0 2024-02-19T19:06:28.051 INFO:tasks.workunit.client.0.smithi088.stdout:9/825: truncate d1/d8/d3e/d93/d2f/fa4 882897 0 2024-02-19T19:06:28.052 INFO:tasks.workunit.client.0.smithi088.stdout:0/619: mknod d13/d62/cd8 0 2024-02-19T19:06:28.055 INFO:tasks.workunit.client.0.smithi088.stdout:9/826: rmdir d1/d8/d37 39 2024-02-19T19:06:28.057 INFO:tasks.workunit.client.0.smithi088.stdout:6/637: truncate d5/f1e 943117 0 2024-02-19T19:06:28.058 INFO:tasks.workunit.client.0.smithi088.stdout:2/511: dwrite d3/d1b/d43/d8f/f7b [0,4194304] 0 2024-02-19T19:06:28.060 INFO:tasks.workunit.client.0.smithi088.stdout:6/638: read d5/d8/d3a/d71/daa/dc2/f39 [2356575,98280] 0 2024-02-19T19:06:28.062 INFO:tasks.workunit.client.0.smithi088.stdout:2/512: mknod d3/d1b/d43/d74/c9a 0 2024-02-19T19:06:28.065 INFO:tasks.workunit.client.0.smithi088.stdout:9/827: mknod d1/d8/d3e/d93/d2d/d18/d1f/df2/dc3/c106 0 2024-02-19T19:06:28.068 INFO:tasks.workunit.client.0.smithi088.stdout:6/639: mkdir d5/d7/dc9/dca 0 2024-02-19T19:06:28.068 INFO:tasks.workunit.client.0.smithi088.stdout:6/640: stat d5/de/fc6 0 2024-02-19T19:06:28.072 INFO:tasks.workunit.client.0.smithi088.stdout:9/828: creat d1/d8/d3e/d93/d2d/d18/f107 x:0 0 0 2024-02-19T19:06:28.073 INFO:tasks.workunit.client.0.smithi088.stdout:2/513: link d3/c4e d3/d1b/d43/d61/d86/c9b 0 2024-02-19T19:06:28.076 INFO:tasks.workunit.client.0.smithi088.stdout:9/829: getdents d1/d8 0 2024-02-19T19:06:28.077 INFO:tasks.workunit.client.0.smithi088.stdout:6/641: dread d5/de/d64/d82/d8c/f59 [0,4194304] 0 2024-02-19T19:06:28.078 INFO:tasks.workunit.client.0.smithi088.stdout:2/514: link d3/d1b/d43/d61/d64/c96 d3/d1b/d43/d74/c9c 0 2024-02-19T19:06:28.079 INFO:tasks.workunit.client.0.smithi088.stdout:9/830: dread d1/d8/d3e/d93/d2f/fa4 [0,4194304] 0 2024-02-19T19:06:28.080 INFO:tasks.workunit.client.0.smithi088.stdout:9/831: chown d1/d8/d4a/d4f/daa 18775187 1 2024-02-19T19:06:28.080 INFO:tasks.workunit.client.0.smithi088.stdout:6/642: mknod d5/de/d64/d82/d8c/ccb 0 2024-02-19T19:06:28.080 INFO:tasks.workunit.client.0.smithi088.stdout:0/620: dread d13/d62/d8a/d95/d5a/d25/f26 [0,4194304] 0 2024-02-19T19:06:28.082 INFO:tasks.workunit.client.0.smithi088.stdout:0/621: write d13/d62/d8a/d95/d96/fc0 [396651,114674] 0 2024-02-19T19:06:28.082 INFO:tasks.workunit.client.0.smithi088.stdout:9/832: fsync d1/d8/d3e/d93/d2d/d18/d92/fe7 0 2024-02-19T19:06:28.084 INFO:tasks.workunit.client.0.smithi088.stdout:6/643: rename d5/de/d64/d8d/d9e/lc4 to d5/de/d64/d8d/d95/dba/lcc 0 2024-02-19T19:06:28.086 INFO:tasks.workunit.client.0.smithi088.stdout:9/833: mknod d1/d8/d37/d54/c108 0 2024-02-19T19:06:28.087 INFO:tasks.workunit.client.0.smithi088.stdout:0/622: symlink d13/d81/ld9 0 2024-02-19T19:06:28.087 INFO:tasks.workunit.client.0.smithi088.stdout:9/834: chown d1/d8/d3e/d93/d2d/d18/d1f/df2/fde 211649073 1 2024-02-19T19:06:28.088 INFO:tasks.workunit.client.0.smithi088.stdout:6/644: rename d5/d7/d32/f86 to d5/d7/d32/d56/dbf/fcd 0 2024-02-19T19:06:28.090 INFO:tasks.workunit.client.0.smithi088.stdout:9/835: write d1/d8/d3e/d93/d2f/fa4 [1912198,7825] 0 2024-02-19T19:06:28.091 INFO:tasks.workunit.client.0.smithi088.stdout:6/645: rmdir d5/de/d64/d8d 39 2024-02-19T19:06:28.093 INFO:tasks.workunit.client.0.smithi088.stdout:9/836: creat d1/d8/d8d/d97/f109 x:0 0 0 2024-02-19T19:06:28.095 INFO:tasks.workunit.client.0.smithi088.stdout:9/837: write d1/d8/d3e/d93/d2d/f4c [1043437,123711] 0 2024-02-19T19:06:28.095 INFO:tasks.workunit.client.0.smithi088.stdout:0/623: dwrite d13/f3e [0,4194304] 0 2024-02-19T19:06:28.097 INFO:tasks.workunit.client.0.smithi088.stdout:6/646: link d5/d45/d9d/fb2 d5/d8/d3a/da8/fce 0 2024-02-19T19:06:28.098 INFO:tasks.workunit.client.0.smithi088.stdout:5/749: sync 2024-02-19T19:06:28.098 INFO:tasks.workunit.client.0.smithi088.stdout:6/647: chown d5/d8/d3a/da8/fb0 85 1 2024-02-19T19:06:28.101 INFO:tasks.workunit.client.0.smithi088.stdout:5/750: write d4/d1a/d41/dc1/dc2/fd7 [670980,129464] 0 2024-02-19T19:06:28.101 INFO:tasks.workunit.client.0.smithi088.stdout:0/624: chown d13/f1b 22806 1 2024-02-19T19:06:28.102 INFO:tasks.workunit.client.0.smithi088.stdout:6/648: creat d5/d8/d3a/da8/fcf x:0 0 0 2024-02-19T19:06:28.103 INFO:tasks.workunit.client.0.smithi088.stdout:5/751: write d4/fe [758757,23835] 0 2024-02-19T19:06:28.107 INFO:tasks.workunit.client.0.smithi088.stdout:0/625: creat d13/d62/d8a/d95/d5a/db6/fda x:0 0 0 2024-02-19T19:06:28.107 INFO:tasks.workunit.client.0.smithi088.stdout:9/838: truncate d1/d8/d3e/d93/d2d/d18/dc0/fd4 3311617 0 2024-02-19T19:06:28.108 INFO:tasks.workunit.client.0.smithi088.stdout:0/626: readlink d13/d62/d8a/d9c/da8/lcc 0 2024-02-19T19:06:28.110 INFO:tasks.workunit.client.0.smithi088.stdout:6/649: rename d5/d8/d3a/c6f to d5/de/d64/d8d/d9e/cd0 0 2024-02-19T19:06:28.111 INFO:tasks.workunit.client.0.smithi088.stdout:5/752: mkdir d4/d61/d65/d9a/d9f/d57/d84/dfb/d100 0 2024-02-19T19:06:28.114 INFO:tasks.workunit.client.0.smithi088.stdout:6/650: dread d5/d19/f5f [0,4194304] 0 2024-02-19T19:06:28.115 INFO:tasks.workunit.client.0.smithi088.stdout:9/839: creat d1/d8/d4a/d4f/daa/dbb/f10a x:0 0 0 2024-02-19T19:06:28.116 INFO:tasks.workunit.client.0.smithi088.stdout:0/627: mkdir d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb 0 2024-02-19T19:06:28.120 INFO:tasks.workunit.client.0.smithi088.stdout:6/651: dwrite d5/d19/f4e [0,4194304] 0 2024-02-19T19:06:28.123 INFO:tasks.workunit.client.0.smithi088.stdout:5/753: creat d4/f101 x:0 0 0 2024-02-19T19:06:28.126 INFO:tasks.workunit.client.0.smithi088.stdout:6/652: symlink d5/de/d64/d8d/d95/dba/ld1 0 2024-02-19T19:06:28.127 INFO:tasks.workunit.client.0.smithi088.stdout:0/628: truncate f3 270900 0 2024-02-19T19:06:28.130 INFO:tasks.workunit.client.0.smithi088.stdout:6/653: symlink d5/d45/d9d/ld2 0 2024-02-19T19:06:28.131 INFO:tasks.workunit.client.0.smithi088.stdout:6/654: chown d5/d8/d3a/d71/daa/dc2/c79 2 1 2024-02-19T19:06:28.136 INFO:tasks.workunit.client.0.smithi088.stdout:0/629: creat d13/d62/d8a/d9c/da8/dab/dd0/fdc x:0 0 0 2024-02-19T19:06:28.138 INFO:tasks.workunit.client.0.smithi088.stdout:0/630: stat d13/d62/d8a/d9c/da8/dab/db4/fc1 0 2024-02-19T19:06:28.139 INFO:tasks.workunit.client.0.smithi088.stdout:0/631: dread - d13/f37 zero size 2024-02-19T19:06:28.147 INFO:tasks.workunit.client.0.smithi088.stdout:9/840: dwrite d1/d8/d3e/d93/d2d/d18/d1f/df2/f9d [0,4194304] 0 2024-02-19T19:06:28.150 INFO:tasks.workunit.client.0.smithi088.stdout:5/754: dwrite d4/d1a/d41/de2/de7/fef [0,4194304] 0 2024-02-19T19:06:28.151 INFO:tasks.workunit.client.0.smithi088.stdout:0/632: link d13/d62/d8a/d95/d5a/d25/d2a/d34/c72 d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/cdd 0 2024-02-19T19:06:28.152 INFO:tasks.workunit.client.0.smithi088.stdout:0/633: write d13/f3e [1721719,1473] 0 2024-02-19T19:06:28.152 INFO:tasks.workunit.client.0.smithi088.stdout:5/755: symlink d4/d75/deb/l102 0 2024-02-19T19:06:28.153 INFO:tasks.workunit.client.0.smithi088.stdout:0/634: fdatasync d13/d62/d8a/d95/d5a/fa2 0 2024-02-19T19:06:28.156 INFO:tasks.workunit.client.0.smithi088.stdout:6/655: dread d5/d8/d3a/da8/fce [0,4194304] 0 2024-02-19T19:06:28.157 INFO:tasks.workunit.client.0.smithi088.stdout:5/756: rename d4/d1a/f7c to d4/d75/f103 0 2024-02-19T19:06:28.159 INFO:tasks.workunit.client.0.smithi088.stdout:6/656: write d5/d19/f41 [754986,127851] 0 2024-02-19T19:06:28.161 INFO:tasks.workunit.client.0.smithi088.stdout:0/635: rename d13/d62/d8a/d95/dcd to d13/d62/d8a/d95/d5b/d76/dde 0 2024-02-19T19:06:28.163 INFO:tasks.workunit.client.0.smithi088.stdout:5/757: link d4/d61/d65/d9a/d9f/d57/f6b d4/d61/d65/f104 0 2024-02-19T19:06:28.165 INFO:tasks.workunit.client.0.smithi088.stdout:6/657: truncate d5/d8/d3a/d71/daa/dc2/f5d 152499 0 2024-02-19T19:06:28.165 INFO:tasks.workunit.client.0.smithi088.stdout:0/636: symlink d13/d62/d8a/d95/d5a/ldf 0 2024-02-19T19:06:28.167 INFO:tasks.workunit.client.0.smithi088.stdout:5/758: creat d4/d61/d65/f105 x:0 0 0 2024-02-19T19:06:28.171 INFO:tasks.workunit.client.0.smithi088.stdout:0/637: read d13/d62/d8a/d95/d5a/d25/d2a/d34/f73 [1843910,101807] 0 2024-02-19T19:06:28.179 INFO:tasks.workunit.client.0.smithi088.stdout:9/841: dread d1/d8/d4a/d4f/f7a [0,4194304] 0 2024-02-19T19:06:28.188 INFO:tasks.workunit.client.0.smithi088.stdout:0/638: dread d13/d62/d8a/d95/d5a/d25/f27 [0,4194304] 0 2024-02-19T19:06:28.245 INFO:tasks.workunit.client.0.smithi088.stdout:3/708: sync 2024-02-19T19:06:28.245 INFO:tasks.workunit.client.0.smithi088.stdout:8/574: sync 2024-02-19T19:06:28.245 INFO:tasks.workunit.client.0.smithi088.stdout:1/671: sync 2024-02-19T19:06:28.245 INFO:tasks.workunit.client.0.smithi088.stdout:4/482: sync 2024-02-19T19:06:28.248 INFO:tasks.workunit.client.0.smithi088.stdout:1/672: stat d6/d10/f1d 0 2024-02-19T19:06:28.249 INFO:tasks.workunit.client.0.smithi088.stdout:9/842: symlink d1/d8/d3e/d93/d2d/d18/d1f/df2/da6/l10b 0 2024-02-19T19:06:28.252 INFO:tasks.workunit.client.0.smithi088.stdout:4/483: mkdir d2/d32/d25/d26/d6d/d7e/d3a/d77/d99 0 2024-02-19T19:06:28.253 INFO:tasks.workunit.client.0.smithi088.stdout:6/658: write d5/d8/d3a/d71/daa/fb9 [955522,31464] 0 2024-02-19T19:06:28.254 INFO:tasks.workunit.client.0.smithi088.stdout:0/639: mkdir d13/d62/d8a/de0 0 2024-02-19T19:06:28.254 INFO:tasks.workunit.client.0.smithi088.stdout:6/659: readlink d5/d19/d26/d72/lc0 0 2024-02-19T19:06:28.256 INFO:tasks.workunit.client.0.smithi088.stdout:9/843: fdatasync d1/d8/d3e/d93/f43 0 2024-02-19T19:06:28.258 INFO:tasks.workunit.client.0.smithi088.stdout:4/484: rmdir d2/d32/d25 39 2024-02-19T19:06:28.261 INFO:tasks.workunit.client.0.smithi088.stdout:4/485: chown d2/d35/f5b 6851 1 2024-02-19T19:06:28.261 INFO:tasks.workunit.client.0.smithi088.stdout:6/660: chown d5/f47 118868 1 2024-02-19T19:06:28.261 INFO:tasks.workunit.client.0.smithi088.stdout:1/673: creat d6/d21/d6a/dcb/fdd x:0 0 0 2024-02-19T19:06:28.262 INFO:tasks.workunit.client.0.smithi088.stdout:5/759: dwrite d4/f11 [0,4194304] 0 2024-02-19T19:06:28.263 INFO:tasks.workunit.client.0.smithi088.stdout:1/674: chown f5 97 1 2024-02-19T19:06:28.264 INFO:tasks.workunit.client.0.smithi088.stdout:9/844: dwrite d1/d8/d8d/d97/f12 [0,4194304] 0 2024-02-19T19:06:28.270 INFO:tasks.workunit.client.0.smithi088.stdout:1/675: mknod d6/d21/d64/d94/d6e/cde 0 2024-02-19T19:06:28.270 INFO:tasks.workunit.client.0.smithi088.stdout:8/575: getdents d9/df/d1c/d22/d9e 0 2024-02-19T19:06:28.271 INFO:tasks.workunit.client.0.smithi088.stdout:1/676: chown d6/d10/c96 116 1 2024-02-19T19:06:28.271 INFO:tasks.workunit.client.0.smithi088.stdout:0/640: link d13/d62/d8a/d95/d5a/c68 d13/d62/d8a/d95/d5a/d25/d2a/ce1 0 2024-02-19T19:06:28.274 INFO:tasks.workunit.client.0.smithi088.stdout:3/709: getdents d4/d1a/d1e/d31/d32 0 2024-02-19T19:06:28.277 INFO:tasks.workunit.client.0.smithi088.stdout:4/486: mkdir d2/d32/d25/d3f/d9a 0 2024-02-19T19:06:28.277 INFO:tasks.workunit.client.0.smithi088.stdout:6/661: dread d5/de/d97/f9b [0,4194304] 0 2024-02-19T19:06:28.278 INFO:tasks.workunit.client.0.smithi088.stdout:1/677: mknod d6/d21/d64/d94/d6e/cdf 0 2024-02-19T19:06:28.281 INFO:tasks.workunit.client.0.smithi088.stdout:4/487: symlink d2/d32/d25/d26/d4e/d6f/l9b 0 2024-02-19T19:06:28.282 INFO:tasks.workunit.client.0.smithi088.stdout:9/845: dread d1/d8/d3e/d93/d2d/d18/ff6 [0,4194304] 0 2024-02-19T19:06:28.283 INFO:tasks.workunit.client.0.smithi088.stdout:9/846: fsync d1/d8/d4a/fba 0 2024-02-19T19:06:28.284 INFO:tasks.workunit.client.0.smithi088.stdout:3/710: mknod d4/d12/cfd 0 2024-02-19T19:06:28.286 INFO:tasks.workunit.client.0.smithi088.stdout:0/641: rename d13/d62/d8a/d95/d5a/d25/d6d to d13/d62/d8a/d9c/da8/de2 0 2024-02-19T19:06:28.288 INFO:tasks.workunit.client.0.smithi088.stdout:9/847: rmdir d1/d8/d3e/d93/d5a/dc4 39 2024-02-19T19:06:28.290 INFO:tasks.workunit.client.0.smithi088.stdout:0/642: readlink d13/d62/d8a/d95/d5a/d25/d2a/l59 0 2024-02-19T19:06:28.290 INFO:tasks.workunit.client.0.smithi088.stdout:3/711: read d4/f38 [536792,39084] 0 2024-02-19T19:06:28.292 INFO:tasks.workunit.client.0.smithi088.stdout:3/712: write d4/d1a/d1e/d74/d7d/fe3 [702422,130151] 0 2024-02-19T19:06:28.293 INFO:tasks.workunit.client.0.smithi088.stdout:4/488: creat d2/d32/d25/d3f/f9c x:0 0 0 2024-02-19T19:06:28.297 INFO:tasks.workunit.client.0.smithi088.stdout:9/848: mknod d1/d8/d3e/d93/d5a/dc4/c10c 0 2024-02-19T19:06:28.300 INFO:tasks.workunit.client.0.smithi088.stdout:8/576: dwrite d9/df/d3d/d50/f52 [0,4194304] 0 2024-02-19T19:06:28.301 INFO:tasks.workunit.client.0.smithi088.stdout:4/489: creat d2/d32/d25/d26/d4e/d6f/f9d x:0 0 0 2024-02-19T19:06:28.302 INFO:tasks.workunit.client.0.smithi088.stdout:8/577: dread - d9/df/d3d/d50/f78 zero size 2024-02-19T19:06:28.306 INFO:tasks.workunit.client.0.smithi088.stdout:1/678: dwrite d6/f17 [0,4194304] 0 2024-02-19T19:06:28.310 INFO:tasks.workunit.client.0.smithi088.stdout:1/679: read d6/d21/f70 [1138297,10004] 0 2024-02-19T19:06:28.311 INFO:tasks.workunit.client.0.smithi088.stdout:0/643: rename d13/d62/c85 to d13/ce3 0 2024-02-19T19:06:28.314 INFO:tasks.workunit.client.0.smithi088.stdout:9/849: symlink d1/d8/d8d/d97/ddb/df3/l10d 0 2024-02-19T19:06:28.316 INFO:tasks.workunit.client.0.smithi088.stdout:9/850: read d1/d8/d3e/d93/d2f/fa4 [1591875,62182] 0 2024-02-19T19:06:28.317 INFO:tasks.workunit.client.0.smithi088.stdout:8/578: dread d9/df/d1c/f73 [0,4194304] 0 2024-02-19T19:06:28.317 INFO:tasks.workunit.client.0.smithi088.stdout:4/490: dwrite d2/d32/d25/d45/f4a [0,4194304] 0 2024-02-19T19:06:28.319 INFO:tasks.workunit.client.0.smithi088.stdout:3/713: getdents d4/d1a/d1e/d31/dc5/dcf 0 2024-02-19T19:06:28.319 INFO:tasks.workunit.client.0.smithi088.stdout:0/644: chown d13/d62/d8a/d95/d5a/d25/d2a/d34/f73 3820100 1 2024-02-19T19:06:28.321 INFO:tasks.workunit.client.0.smithi088.stdout:9/851: chown d1/d8/d3e/d93/d2d/d18/d92/ced 3186 1 2024-02-19T19:06:28.322 INFO:tasks.workunit.client.0.smithi088.stdout:4/491: mkdir d2/d32/d25/d26/d6d/d9e 0 2024-02-19T19:06:28.323 INFO:tasks.workunit.client.0.smithi088.stdout:9/852: dread - d1/d8/d3e/d93/d2d/dcb/ffe zero size 2024-02-19T19:06:28.324 INFO:tasks.workunit.client.0.smithi088.stdout:8/579: mkdir d9/df/d1c/d85/daf 0 2024-02-19T19:06:28.324 INFO:tasks.workunit.client.0.smithi088.stdout:0/645: read d13/d62/d8a/d95/f6c [517199,127095] 0 2024-02-19T19:06:28.325 INFO:tasks.workunit.client.0.smithi088.stdout:3/714: mkdir d4/d1a/d1e/d74/d7d/daa/dcb/dfe 0 2024-02-19T19:06:28.326 INFO:tasks.workunit.client.0.smithi088.stdout:1/680: dread d6/d10/f3b [0,4194304] 0 2024-02-19T19:06:28.327 INFO:tasks.workunit.client.0.smithi088.stdout:8/580: symlink d9/df/d1c/d85/lb0 0 2024-02-19T19:06:28.329 INFO:tasks.workunit.client.0.smithi088.stdout:9/853: symlink d1/d8/d8d/d97/ddb/l10e 0 2024-02-19T19:06:28.330 INFO:tasks.workunit.client.0.smithi088.stdout:3/715: stat d4/d12/l76 0 2024-02-19T19:06:28.331 INFO:tasks.workunit.client.0.smithi088.stdout:1/681: creat d6/d10/d60/d29/fe0 x:0 0 0 2024-02-19T19:06:28.333 INFO:tasks.workunit.client.0.smithi088.stdout:4/492: mkdir d2/d32/d25/d26/d6d/d86/d9f 0 2024-02-19T19:06:28.334 INFO:tasks.workunit.client.0.smithi088.stdout:1/682: write d6/d21/d6a/d4a/fb1 [415329,106802] 0 2024-02-19T19:06:28.334 INFO:tasks.workunit.client.0.smithi088.stdout:3/716: rmdir d4/d12/d33/d43 39 2024-02-19T19:06:28.337 INFO:tasks.workunit.client.0.smithi088.stdout:0/646: getdents d13/d62/d8a/d87 0 2024-02-19T19:06:28.337 INFO:tasks.workunit.client.0.smithi088.stdout:4/493: stat d2/d32/d25/d26/d6d/d7e/f72 0 2024-02-19T19:06:28.337 INFO:tasks.workunit.client.0.smithi088.stdout:8/581: dwrite d9/df/d3d/d50/f52 [0,4194304] 0 2024-02-19T19:06:28.339 INFO:tasks.workunit.client.0.smithi088.stdout:3/717: readlink d4/d1a/d1e/l62 0 2024-02-19T19:06:28.340 INFO:tasks.workunit.client.0.smithi088.stdout:1/683: creat d6/d10/fe1 x:0 0 0 2024-02-19T19:06:28.345 INFO:tasks.workunit.client.0.smithi088.stdout:0/647: mknod d13/d62/d8a/d95/d5a/d25/d2a/d3c/d86/ce4 0 2024-02-19T19:06:28.347 INFO:tasks.workunit.client.0.smithi088.stdout:8/582: dwrite d9/df/d1c/d22/d9e/faa [0,4194304] 0 2024-02-19T19:06:28.348 INFO:tasks.workunit.client.0.smithi088.stdout:4/494: mkdir d2/d32/d25/d26/da0 0 2024-02-19T19:06:28.351 INFO:tasks.workunit.client.0.smithi088.stdout:3/718: symlink d4/d12/lff 0 2024-02-19T19:06:28.356 INFO:tasks.workunit.client.0.smithi088.stdout:9/854: truncate d1/d8/d3e/d93/d2d/d18/d1f/df2/f9d 4044682 0 2024-02-19T19:06:28.359 INFO:tasks.workunit.client.0.smithi088.stdout:0/648: write d13/d62/d8a/d95/d5a/d25/d2a/d2f/f50 [974188,60866] 0 2024-02-19T19:06:28.361 INFO:tasks.workunit.client.0.smithi088.stdout:3/719: mkdir d4/d100 0 2024-02-19T19:06:28.365 INFO:tasks.workunit.client.0.smithi088.stdout:9/855: rename d1/d8/d3e/d93/d2d/d18/d1f/df2/d99/fd2 to d1/d8/d3e/d93/d2d/d18/d1f/df2/d99/dc5/dea/f10f 0 2024-02-19T19:06:28.366 INFO:tasks.workunit.client.0.smithi088.stdout:1/684: link d6/d10/d60/d29/l3f d6/le2 0 2024-02-19T19:06:28.367 INFO:tasks.workunit.client.0.smithi088.stdout:9/856: write d1/d8/d3e/d93/d2d/d18/d1f/fb4 [362137,46042] 0 2024-02-19T19:06:28.368 INFO:tasks.workunit.client.0.smithi088.stdout:3/720: rmdir d4/d1a/d1e/d74/d7c 39 2024-02-19T19:06:28.370 INFO:tasks.workunit.client.0.smithi088.stdout:3/721: write d4/d22/d7a/d8d/fd2 [868516,126962] 0 2024-02-19T19:06:28.371 INFO:tasks.workunit.client.0.smithi088.stdout:4/495: getdents d2 0 2024-02-19T19:06:28.373 INFO:tasks.workunit.client.0.smithi088.stdout:4/496: dread - d2/d32/d25/d3f/f9c zero size 2024-02-19T19:06:28.373 INFO:tasks.workunit.client.0.smithi088.stdout:9/857: creat d1/d8/d3e/d93/d2f/f110 x:0 0 0 2024-02-19T19:06:28.373 INFO:tasks.workunit.client.0.smithi088.stdout:8/583: dread d9/df/d1c/f13 [0,4194304] 0 2024-02-19T19:06:28.374 INFO:tasks.workunit.client.0.smithi088.stdout:1/685: mknod d6/d21/d6a/ce3 0 2024-02-19T19:06:28.374 INFO:tasks.workunit.client.0.smithi088.stdout:3/722: creat d4/d1a/d23/f101 x:0 0 0 2024-02-19T19:06:28.376 INFO:tasks.workunit.client.0.smithi088.stdout:9/858: symlink d1/d8/d3e/d93/d2d/d18/d1f/df2/l111 0 2024-02-19T19:06:28.377 INFO:tasks.workunit.client.0.smithi088.stdout:4/497: truncate d2/d32/d25/f38 1708483 0 2024-02-19T19:06:28.378 INFO:tasks.workunit.client.0.smithi088.stdout:3/723: creat d4/d1a/d1e/d31/d32/f102 x:0 0 0 2024-02-19T19:06:28.379 INFO:tasks.workunit.client.0.smithi088.stdout:4/498: symlink d2/d32/d25/d4b/la1 0 2024-02-19T19:06:28.381 INFO:tasks.workunit.client.0.smithi088.stdout:9/859: creat d1/d8/d3e/d93/d5a/d94/d74/f112 x:0 0 0 2024-02-19T19:06:28.381 INFO:tasks.workunit.client.0.smithi088.stdout:1/686: dwrite d6/d21/f6c [0,4194304] 0 2024-02-19T19:06:28.386 INFO:tasks.workunit.client.0.smithi088.stdout:4/499: mkdir d2/d32/d25/d3f/da2 0 2024-02-19T19:06:28.390 INFO:tasks.workunit.client.0.smithi088.stdout:9/860: dwrite d1/d8/d3e/d93/d2d/d18/d1f/f59 [0,4194304] 0 2024-02-19T19:06:28.390 INFO:tasks.workunit.client.0.smithi088.stdout:3/724: creat d4/d1a/d1e/d74/d7d/f103 x:0 0 0 2024-02-19T19:06:28.391 INFO:tasks.workunit.client.0.smithi088.stdout:9/861: chown d1/d8/d3e/d93/fa1 2350188 1 2024-02-19T19:06:28.392 INFO:tasks.workunit.client.0.smithi088.stdout:1/687: getdents d6/d21/d64/d94/dab 0 2024-02-19T19:06:28.393 INFO:tasks.workunit.client.0.smithi088.stdout:3/725: truncate d4/d22/d7a/d8d/f9d 307003 0 2024-02-19T19:06:28.396 INFO:tasks.workunit.client.0.smithi088.stdout:9/862: write d1/d8/d3e/d93/d2d/d18/d92/fe7 [4343891,82364] 0 2024-02-19T19:06:28.397 INFO:tasks.workunit.client.0.smithi088.stdout:9/863: fsync d1/d8/d8d/d97/f109 0 2024-02-19T19:06:28.397 INFO:tasks.workunit.client.0.smithi088.stdout:4/500: rename d2/d32/d25/d26/d6d/d86 to d2/d32/d25/d26/d6d/d7e/da3 0 2024-02-19T19:06:28.398 INFO:tasks.workunit.client.0.smithi088.stdout:1/688: read d6/d21/f36 [1226727,80648] 0 2024-02-19T19:06:28.400 INFO:tasks.workunit.client.0.smithi088.stdout:3/726: write d4/d12/d33/d43/fbd [4670037,74039] 0 2024-02-19T19:06:28.400 INFO:tasks.workunit.client.0.smithi088.stdout:1/689: chown d6/d21/d6a/d7d/d7e/fac 848771 1 2024-02-19T19:06:28.401 INFO:tasks.workunit.client.0.smithi088.stdout:4/501: mkdir d2/d32/d25/d26/d4e/da4 0 2024-02-19T19:06:28.401 INFO:tasks.workunit.client.0.smithi088.stdout:9/864: truncate d1/d8/d4a/d4f/daa/dbb/f10a 993365 0 2024-02-19T19:06:28.403 INFO:tasks.workunit.client.0.smithi088.stdout:1/690: truncate d6/d21/d6a/fb7 147527 0 2024-02-19T19:06:28.406 INFO:tasks.workunit.client.0.smithi088.stdout:3/727: rename d4/d1a/d23/dc9 to d4/d22/dd5/d104 0 2024-02-19T19:06:28.408 INFO:tasks.workunit.client.0.smithi088.stdout:1/691: unlink d6/d10/d60/fdc 0 2024-02-19T19:06:28.409 INFO:tasks.workunit.client.0.smithi088.stdout:4/502: dwrite d2/d32/f44 [0,4194304] 0 2024-02-19T19:06:28.410 INFO:tasks.workunit.client.0.smithi088.stdout:4/503: chown d2/d32/d25/d3f/d9a 1945 1 2024-02-19T19:06:28.410 INFO:tasks.workunit.client.0.smithi088.stdout:4/504: readlink d2/l55 0 2024-02-19T19:06:28.417 INFO:tasks.workunit.client.0.smithi088.stdout:9/865: getdents d1/d8/d3e/d93/d2d/d18/d1f/df2/da6 0 2024-02-19T19:06:28.418 INFO:tasks.workunit.client.0.smithi088.stdout:1/692: mkdir d6/d10/d60/db8/dd9/de4 0 2024-02-19T19:06:28.420 INFO:tasks.workunit.client.0.smithi088.stdout:9/866: creat d1/d8/d8d/f113 x:0 0 0 2024-02-19T19:06:28.423 INFO:tasks.workunit.client.0.smithi088.stdout:4/505: dread d2/d8/f6b [0,4194304] 0 2024-02-19T19:06:28.424 INFO:tasks.workunit.client.0.smithi088.stdout:9/867: mkdir d1/d8/d8d/d97/ddb/df3/d114 0 2024-02-19T19:06:28.425 INFO:tasks.workunit.client.0.smithi088.stdout:4/506: chown d2/d8/f16 283072 1 2024-02-19T19:06:28.426 INFO:tasks.workunit.client.0.smithi088.stdout:3/728: dread d4/d87/fb6 [0,4194304] 0 2024-02-19T19:06:28.426 INFO:tasks.workunit.client.0.smithi088.stdout:1/693: dwrite d6/d21/d64/d94/d71/f9b [0,4194304] 0 2024-02-19T19:06:28.426 INFO:tasks.workunit.client.0.smithi088.stdout:9/868: write d1/d8/fdc [1534334,7363] 0 2024-02-19T19:06:28.429 INFO:tasks.workunit.client.0.smithi088.stdout:4/507: chown d2/d32/d9/c1b 7887186 1 2024-02-19T19:06:28.430 INFO:tasks.workunit.client.0.smithi088.stdout:1/694: write d6/d10/f5d [655585,29778] 0 2024-02-19T19:06:28.431 INFO:tasks.workunit.client.0.smithi088.stdout:3/729: mknod d4/d12/d33/d43/ddd/c105 0 2024-02-19T19:06:28.433 INFO:tasks.workunit.client.0.smithi088.stdout:4/508: mkdir d2/d32/d25/d26/da5 0 2024-02-19T19:06:28.437 INFO:tasks.workunit.client.0.smithi088.stdout:9/869: creat d1/d8/d3e/d93/d2d/d18/d1f/f115 x:0 0 0 2024-02-19T19:06:28.440 INFO:tasks.workunit.client.0.smithi088.stdout:3/730: getdents d4/d12/d33 0 2024-02-19T19:06:28.441 INFO:tasks.workunit.client.0.smithi088.stdout:3/731: chown d4/d1a/d1e/d31/c4f 356016 1 2024-02-19T19:06:28.443 INFO:tasks.workunit.client.0.smithi088.stdout:9/870: link d1/d8/d3e/d93/l5b d1/d8/d4a/d4f/daa/dbb/l116 0 2024-02-19T19:06:28.454 INFO:tasks.workunit.client.0.smithi088.stdout:9/871: mkdir d1/d8/d3e/d93/d5a/d117 0 2024-02-19T19:06:28.454 INFO:tasks.workunit.client.0.smithi088.stdout:3/732: dread d4/f6b [0,4194304] 0 2024-02-19T19:06:28.461 INFO:tasks.workunit.client.0.smithi088.stdout:4/509: dread d2/d8/f16 [0,4194304] 0 2024-02-19T19:06:28.470 INFO:tasks.workunit.client.0.smithi088.stdout:9/872: symlink d1/d8/d3e/d93/d2d/d18/d92/l118 0 2024-02-19T19:06:28.471 INFO:tasks.workunit.client.0.smithi088.stdout:3/733: truncate d4/d1a/d1e/d31/d32/f77 381647 0 2024-02-19T19:06:28.472 INFO:tasks.workunit.client.0.smithi088.stdout:4/510: creat d2/d32/d25/d26/d4e/d6f/fa6 x:0 0 0 2024-02-19T19:06:28.473 INFO:tasks.workunit.client.0.smithi088.stdout:3/734: symlink d4/d12/d33/dab/l106 0 2024-02-19T19:06:28.473 INFO:tasks.workunit.client.0.smithi088.stdout:4/511: write d2/d32/d25/d26/d58/f69 [931751,26930] 0 2024-02-19T19:06:28.474 INFO:tasks.workunit.client.0.smithi088.stdout:9/873: creat d1/d8/d3e/d93/d2d/d18/d1f/df2/d99/dc5/dea/f119 x:0 0 0 2024-02-19T19:06:28.476 INFO:tasks.workunit.client.0.smithi088.stdout:3/735: stat d4/d12/f2e 0 2024-02-19T19:06:28.478 INFO:tasks.workunit.client.0.smithi088.stdout:3/736: write d4/d22/d7a/d8d/fb9 [434498,40974] 0 2024-02-19T19:06:28.480 INFO:tasks.workunit.client.0.smithi088.stdout:9/874: mkdir d1/d8/d3e/d93/d2d/d18/d11a 0 2024-02-19T19:06:28.481 INFO:tasks.workunit.client.0.smithi088.stdout:4/512: rmdir d2/d32/d25/d26/d4e/da4 0 2024-02-19T19:06:28.482 INFO:tasks.workunit.client.0.smithi088.stdout:3/737: creat d4/d12/d33/d58/f107 x:0 0 0 2024-02-19T19:06:28.483 INFO:tasks.workunit.client.0.smithi088.stdout:9/875: dread d1/d8/d37/f9c [4194304,4194304] 0 2024-02-19T19:06:28.484 INFO:tasks.workunit.client.0.smithi088.stdout:7/679: sync 2024-02-19T19:06:28.487 INFO:tasks.workunit.client.0.smithi088.stdout:4/513: truncate d2/d32/d9/f71 3155674 0 2024-02-19T19:06:28.488 INFO:tasks.workunit.client.0.smithi088.stdout:4/514: readlink d2/d32/l50 0 2024-02-19T19:06:28.489 INFO:tasks.workunit.client.0.smithi088.stdout:7/680: creat d14/d28/d94/ddb/ff1 x:0 0 0 2024-02-19T19:06:28.490 INFO:tasks.workunit.client.0.smithi088.stdout:7/681: write f2 [3910228,75062] 0 2024-02-19T19:06:28.491 INFO:tasks.workunit.client.0.smithi088.stdout:7/682: stat d14/d28/d94/ddb/ff1 0 2024-02-19T19:06:28.492 INFO:tasks.workunit.client.0.smithi088.stdout:9/876: rename d1/d8/d8d/d97/ddb/df3 to d1/d8/d3e/d93/d11b 0 2024-02-19T19:06:28.493 INFO:tasks.workunit.client.0.smithi088.stdout:4/515: stat d2/d32/d25/d26/d4e/c82 0 2024-02-19T19:06:28.494 INFO:tasks.workunit.client.0.smithi088.stdout:4/516: truncate d2/d32/f36 140466 0 2024-02-19T19:06:28.497 INFO:tasks.workunit.client.0.smithi088.stdout:7/683: rename d14/d28/d3b/dd3/fce to d14/d27/d49/d98/dbe/ff2 0 2024-02-19T19:06:28.498 INFO:tasks.workunit.client.0.smithi088.stdout:3/738: dread d4/d12/d33/d58/f93 [0,4194304] 0 2024-02-19T19:06:28.498 INFO:tasks.workunit.client.0.smithi088.stdout:9/877: mkdir d1/d8/d4a/d11c 0 2024-02-19T19:06:28.500 INFO:tasks.workunit.client.0.smithi088.stdout:7/684: creat d14/d28/d65/ff3 x:0 0 0 2024-02-19T19:06:28.503 INFO:tasks.workunit.client.0.smithi088.stdout:7/685: write d14/d28/d3b/f64 [380139,35193] 0 2024-02-19T19:06:28.506 INFO:tasks.workunit.client.0.smithi088.stdout:7/686: truncate d14/d28/d65/f7e 2737105 0 2024-02-19T19:06:28.509 INFO:tasks.workunit.client.0.smithi088.stdout:7/687: rename d14/d27/ddd to d14/d1d/d3c/d43/d5c/d9a/df4 0 2024-02-19T19:06:28.511 INFO:tasks.workunit.client.0.smithi088.stdout:4/517: dread d2/d32/d25/d26/d6d/d7e/d3a/f5f [0,4194304] 0 2024-02-19T19:06:28.512 INFO:tasks.workunit.client.0.smithi088.stdout:7/688: creat d14/d27/dd2/ff5 x:0 0 0 2024-02-19T19:06:28.517 INFO:tasks.workunit.client.0.smithi088.stdout:9/878: dread d1/d8/d3e/d93/d2d/d18/dc0/fd4 [0,4194304] 0 2024-02-19T19:06:28.519 INFO:tasks.workunit.client.0.smithi088.stdout:7/689: dwrite d14/d27/dd2/ff5 [0,4194304] 0 2024-02-19T19:06:28.520 INFO:tasks.workunit.client.0.smithi088.stdout:9/879: truncate d1/faf 133400 0 2024-02-19T19:06:28.524 INFO:tasks.workunit.client.0.smithi088.stdout:9/880: creat d1/d8/d37/f11d x:0 0 0 2024-02-19T19:06:28.531 INFO:tasks.workunit.client.0.smithi088.stdout:4/518: dwrite d2/f75 [0,4194304] 0 2024-02-19T19:06:28.532 INFO:tasks.workunit.client.0.smithi088.stdout:7/690: dwrite d14/f17 [0,4194304] 0 2024-02-19T19:06:28.535 INFO:tasks.workunit.client.0.smithi088.stdout:9/881: dwrite d1/d8/d3e/d93/d2d/d23/f26 [0,4194304] 0 2024-02-19T19:06:28.538 INFO:tasks.workunit.client.0.smithi088.stdout:7/691: creat d14/d1d/dbf/ff6 x:0 0 0 2024-02-19T19:06:28.544 INFO:tasks.workunit.client.0.smithi088.stdout:9/882: mkdir d1/d8/d4a/d4f/d11e 0 2024-02-19T19:06:28.544 INFO:tasks.workunit.client.0.smithi088.stdout:7/692: mknod d14/d1d/d3c/d43/d5c/cf7 0 2024-02-19T19:06:28.544 INFO:tasks.workunit.client.0.smithi088.stdout:4/519: dwrite d2/d32/d25/f97 [4194304,4194304] 0 2024-02-19T19:06:28.546 INFO:tasks.workunit.client.0.smithi088.stdout:7/693: dread - d14/d1d/d3c/fb7 zero size 2024-02-19T19:06:28.549 INFO:tasks.workunit.client.0.smithi088.stdout:7/694: mkdir d14/d27/d6b/df8 0 2024-02-19T19:06:28.554 INFO:tasks.workunit.client.0.smithi088.stdout:4/520: link d2/d32/d25/d26/d6d/d7e/f72 d2/d32/d25/d26/d6d/d7e/d90/fa7 0 2024-02-19T19:06:28.554 INFO:tasks.workunit.client.0.smithi088.stdout:4/521: readlink d2/d32/d9/l1e 0 2024-02-19T19:06:28.554 INFO:tasks.workunit.client.0.smithi088.stdout:4/522: chown d2/l78 1084310899 1 2024-02-19T19:06:28.555 INFO:tasks.workunit.client.0.smithi088.stdout:4/523: chown d2/d8/f29 228 1 2024-02-19T19:06:28.555 INFO:tasks.workunit.client.0.smithi088.stdout:4/524: dread - d2/d32/d25/f8f zero size 2024-02-19T19:06:28.558 INFO:tasks.workunit.client.0.smithi088.stdout:4/525: rename d2/c13 to d2/d32/d25/d26/d6d/ca8 0 2024-02-19T19:06:28.561 INFO:tasks.workunit.client.0.smithi088.stdout:7/695: write d14/d1d/d3c/d43/d5c/d9a/d92/fab [964026,7561] 0 2024-02-19T19:06:28.563 INFO:tasks.workunit.client.0.smithi088.stdout:7/696: creat d14/ff9 x:0 0 0 2024-02-19T19:06:28.564 INFO:tasks.workunit.client.0.smithi088.stdout:7/697: stat d14/l37 0 2024-02-19T19:06:28.572 INFO:tasks.workunit.client.0.smithi088.stdout:7/698: dwrite d14/d28/d65/fa8 [0,4194304] 0 2024-02-19T19:06:28.574 INFO:tasks.workunit.client.0.smithi088.stdout:9/883: dread d1/d8/d3e/d93/d5a/dc4/fd1 [0,4194304] 0 2024-02-19T19:06:28.576 INFO:tasks.workunit.client.0.smithi088.stdout:9/884: write d1/d8/d3e/d93/d2d/d18/d1f/f59 [78942,69450] 0 2024-02-19T19:06:28.580 INFO:tasks.workunit.client.0.smithi088.stdout:9/885: chown d1/f7f 99158632 1 2024-02-19T19:06:28.580 INFO:tasks.workunit.client.0.smithi088.stdout:9/886: dread - d1/d8/d4a/dac/fff zero size 2024-02-19T19:06:28.583 INFO:tasks.workunit.client.0.smithi088.stdout:9/887: read d1/d8/d3e/d93/d2d/f47 [3175288,17861] 0 2024-02-19T19:06:28.583 INFO:tasks.workunit.client.0.smithi088.stdout:7/699: truncate d14/d27/d49/d98/fb0 510483 0 2024-02-19T19:06:28.584 INFO:tasks.workunit.client.0.smithi088.stdout:9/888: chown d1/d8/d4a/d4f/daa 1263863 1 2024-02-19T19:06:28.585 INFO:tasks.workunit.client.0.smithi088.stdout:7/700: truncate d14/d27/d6b/f93 647561 0 2024-02-19T19:06:28.585 INFO:tasks.workunit.client.0.smithi088.stdout:7/701: read - d14/d27/d6b/fa7 zero size 2024-02-19T19:06:28.589 INFO:tasks.workunit.client.0.smithi088.stdout:7/702: mkdir d14/d1d/d62/dfa 0 2024-02-19T19:06:28.590 INFO:tasks.workunit.client.0.smithi088.stdout:9/889: dwrite d1/d8/d3e/d93/d2d/d18/d1f/df2/f86 [0,4194304] 0 2024-02-19T19:06:28.591 INFO:tasks.workunit.client.0.smithi088.stdout:7/703: symlink d14/d27/d6b/lfb 0 2024-02-19T19:06:28.592 INFO:tasks.workunit.client.0.smithi088.stdout:9/890: fdatasync d1/d8/d3e/d93/d2f/f110 0 2024-02-19T19:06:28.592 INFO:tasks.workunit.client.0.smithi088.stdout:7/704: write d14/d1d/d3c/d43/d5c/d9a/faf [3326600,31106] 0 2024-02-19T19:06:28.593 INFO:tasks.workunit.client.0.smithi088.stdout:9/891: chown d1/d8/d3e/d93/d5a/c98 79949582 1 2024-02-19T19:06:28.596 INFO:tasks.workunit.client.0.smithi088.stdout:9/892: symlink d1/d8/d3e/d93/d5a/d117/l11f 0 2024-02-19T19:06:28.598 INFO:tasks.workunit.client.0.smithi088.stdout:7/705: link d14/f2e d14/d27/d49/d98/dcb/de7/ffc 0 2024-02-19T19:06:28.599 INFO:tasks.workunit.client.0.smithi088.stdout:9/893: symlink d1/d8/d4a/d4f/daa/dbb/l120 0 2024-02-19T19:06:28.600 INFO:tasks.workunit.client.0.smithi088.stdout:7/706: symlink d14/d1d/d62/dfa/lfd 0 2024-02-19T19:06:28.602 INFO:tasks.workunit.client.0.smithi088.stdout:9/894: mkdir d1/d8/d3e/d93/d2d/d121 0 2024-02-19T19:06:28.603 INFO:tasks.workunit.client.0.smithi088.stdout:9/895: read - d1/d8/d4a/fba zero size 2024-02-19T19:06:28.603 INFO:tasks.workunit.client.0.smithi088.stdout:7/707: truncate d14/d1d/d3c/f6c 1587423 0 2024-02-19T19:06:28.606 INFO:tasks.workunit.client.0.smithi088.stdout:7/708: mkdir d14/d27/d49/dfe 0 2024-02-19T19:06:28.608 INFO:tasks.workunit.client.0.smithi088.stdout:9/896: rename d1/d8/d3e/d93/d11b to d1/d8/d3e/d93/d122 0 2024-02-19T19:06:28.609 INFO:tasks.workunit.client.0.smithi088.stdout:9/897: chown d1/d8/d4a/d4f/daa/dbb/f10a 177 1 2024-02-19T19:06:28.610 INFO:tasks.workunit.client.0.smithi088.stdout:7/709: creat d14/d27/d49/d98/dcb/fff x:0 0 0 2024-02-19T19:06:28.610 INFO:tasks.workunit.client.0.smithi088.stdout:9/898: chown d1/d8/l49 30814 1 2024-02-19T19:06:28.613 INFO:tasks.workunit.client.0.smithi088.stdout:9/899: unlink d1/d8/d3e/d93/d2d/d18/d1f/df2/d99/dc5/dea/f10f 0 2024-02-19T19:06:28.615 INFO:tasks.workunit.client.0.smithi088.stdout:9/900: stat d1/d8/d3e/d93/d2d/d18/d1f/df2/d99/ccf 0 2024-02-19T19:06:28.615 INFO:tasks.workunit.client.0.smithi088.stdout:9/901: fsync d1/d8/d3e/d93/f64 0 2024-02-19T19:06:28.617 INFO:tasks.workunit.client.0.smithi088.stdout:9/902: chown d1/d8/d37/d54/c108 208843 1 2024-02-19T19:06:28.618 INFO:tasks.workunit.client.0.smithi088.stdout:9/903: chown d1/d8/d8d/d97 3 1 2024-02-19T19:06:28.620 INFO:tasks.workunit.client.0.smithi088.stdout:9/904: creat d1/d8/d3e/d93/d2d/d18/d1f/df2/d99/dc5/f123 x:0 0 0 2024-02-19T19:06:28.621 INFO:tasks.workunit.client.0.smithi088.stdout:9/905: chown d1/d8/d3e/d93/d2d/d18/d92/lbc 197 1 2024-02-19T19:06:28.622 INFO:tasks.workunit.client.0.smithi088.stdout:9/906: write d1/d8/d8d/f113 [372998,90410] 0 2024-02-19T19:06:28.625 INFO:tasks.workunit.client.0.smithi088.stdout:9/907: dread d1/d8/d3e/ff9 [0,4194304] 0 2024-02-19T19:06:28.630 INFO:tasks.workunit.client.0.smithi088.stdout:9/908: dwrite d1/d8/d3e/d93/d2d/d18/d1f/f59 [0,4194304] 0 2024-02-19T19:06:28.631 INFO:tasks.workunit.client.0.smithi088.stdout:9/909: dread - d1/d8/d3e/d93/d2d/dcb/ffe zero size 2024-02-19T19:06:28.635 INFO:tasks.workunit.client.0.smithi088.stdout:9/910: truncate d1/d8/d3e/d93/d2d/d18/d1f/db0/fab 2530883 0 2024-02-19T19:06:28.635 INFO:tasks.workunit.client.0.smithi088.stdout:9/911: read - d1/d8/d4a/fb7 zero size 2024-02-19T19:06:28.636 INFO:tasks.workunit.client.0.smithi088.stdout:9/912: write d1/d8/d3e/d93/d2d/d18/d92/fd3 [1020536,119976] 0 2024-02-19T19:06:28.637 INFO:tasks.workunit.client.0.smithi088.stdout:9/913: chown d1/d8/d4a/d4f/lec 0 1 2024-02-19T19:06:28.639 INFO:tasks.workunit.client.0.smithi088.stdout:9/914: mknod d1/d8/d4a/d4f/daa/c124 0 2024-02-19T19:06:28.641 INFO:tasks.workunit.client.0.smithi088.stdout:9/915: symlink d1/l125 0 2024-02-19T19:06:28.656 INFO:tasks.workunit.client.0.smithi088.stdout:9/916: write d1/d8/d8d/fa5 [1535809,118986] 0 2024-02-19T19:06:28.657 INFO:tasks.workunit.client.0.smithi088.stdout:9/917: chown d1/d8/d37/d54/f84 6 1 2024-02-19T19:06:28.663 INFO:tasks.workunit.client.0.smithi088.stdout:9/918: write d1/d8/d3e/d93/d2d/d18/d1f/db0/fab [668099,51366] 0 2024-02-19T19:06:28.664 INFO:tasks.workunit.client.0.smithi088.stdout:9/919: read - d1/d8/d37/f11d zero size 2024-02-19T19:06:28.669 INFO:tasks.workunit.client.0.smithi088.stdout:9/920: dwrite d1/d8/d3e/d93/d2d/d23/fcc [0,4194304] 0 2024-02-19T19:06:28.671 INFO:tasks.workunit.client.0.smithi088.stdout:9/921: mkdir d1/d8/d37/d54/d126 0 2024-02-19T19:06:28.674 INFO:tasks.workunit.client.0.smithi088.stdout:9/922: dread d1/d8/d3e/d93/d5a/dc4/fd1 [0,4194304] 0 2024-02-19T19:06:28.674 INFO:tasks.workunit.client.0.smithi088.stdout:9/923: chown d1/d8/d8d 184583 1 2024-02-19T19:06:28.744 INFO:tasks.workunit.client.0.smithi088.stdout:9/924: readlink d1/d8/d3e/d93/d2d/d18/d1f/df2/da6/l10b 0 2024-02-19T19:06:28.748 INFO:tasks.workunit.client.0.smithi088.stdout:9/925: truncate d1/d8/d37/f9c 2922665 0 2024-02-19T19:06:28.750 INFO:tasks.workunit.client.0.smithi088.stdout:9/926: write d1/d8/d3e/d93/d2d/d18/d1f/df2/fde [955030,93794] 0 2024-02-19T19:06:28.751 INFO:tasks.workunit.client.0.smithi088.stdout:9/927: symlink d1/d8/d3e/d93/d2d/l127 0 2024-02-19T19:06:28.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:06:28.757 INFO:tasks.workunit.client.0.smithi088.stdout:9/928: dwrite d1/d8/d3e/d93/d2d/d18/ff6 [0,4194304] 0 2024-02-19T19:06:28.831 INFO:tasks.workunit.client.0.smithi088.stdout:9/929: mkdir d1/d8/d3e/d93/d2d/d128 0 2024-02-19T19:06:28.834 INFO:tasks.workunit.client.0.smithi088.stdout:9/930: dread d1/d8/d3e/d93/d2d/d18/d1f/fb4 [0,4194304] 0 2024-02-19T19:06:28.850 INFO:tasks.workunit.client.0.smithi088.stdout:9/931: dread d1/fc [0,4194304] 0 2024-02-19T19:06:28.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:06:28.879 INFO:tasks.workunit.client.0.smithi088.stdout:9/932: creat d1/d8/d3e/d93/d2d/d18/d11a/f129 x:0 0 0 2024-02-19T19:06:28.880 INFO:tasks.workunit.client.0.smithi088.stdout:9/933: dread - d1/d8/d4a/fb7 zero size 2024-02-19T19:06:28.885 INFO:tasks.workunit.client.0.smithi088.stdout:9/934: write d1/d8/d3e/d93/d5a/fd5 [1642430,79681] 0 2024-02-19T19:06:28.890 INFO:tasks.workunit.client.0.smithi088.stdout:9/935: dwrite d1/d8/d4a/d4f/daa/dbb/ff4 [4194304,4194304] 0 2024-02-19T19:06:28.892 INFO:tasks.workunit.client.0.smithi088.stdout:9/936: creat d1/d8/d4a/dac/f12a x:0 0 0 2024-02-19T19:06:28.894 INFO:tasks.workunit.client.0.smithi088.stdout:9/937: mkdir d1/d8/d8d/d12b 0 2024-02-19T19:06:28.930 INFO:tasks.workunit.client.0.smithi088.stdout:2/515: sync 2024-02-19T19:06:28.932 INFO:tasks.workunit.client.0.smithi088.stdout:2/516: write d3/fb [1237801,12] 0 2024-02-19T19:06:28.934 INFO:tasks.workunit.client.0.smithi088.stdout:2/517: creat d3/d1b/d43/d61/d64/f9d x:0 0 0 2024-02-19T19:06:28.934 INFO:tasks.workunit.client.0.smithi088.stdout:9/938: dwrite d1/d8/d37/d54/f84 [8388608,4194304] 0 2024-02-19T19:06:28.986 INFO:tasks.workunit.client.0.smithi088.stdout:9/939: fdatasync d1/d8/d3e/d93/d2d/d18/dc0/fc6 0 2024-02-19T19:06:28.987 INFO:tasks.workunit.client.0.smithi088.stdout:2/518: unlink d3/d1b/c88 0 2024-02-19T19:06:28.988 INFO:tasks.workunit.client.0.smithi088.stdout:2/519: chown d3/l13 3423882 1 2024-02-19T19:06:28.988 INFO:tasks.workunit.client.0.smithi088.stdout:2/520: read - d3/d1b/d43/d61/d64/f8a zero size 2024-02-19T19:06:28.989 INFO:tasks.workunit.client.0.smithi088.stdout:9/940: rename d1/d8/d8d/d97/ddb to d1/d8/d37/d12c 0 2024-02-19T19:06:29.023 INFO:tasks.workunit.client.0.smithi088.stdout:5/760: sync 2024-02-19T19:06:29.023 INFO:tasks.workunit.client.0.smithi088.stdout:9/941: link d1/faf d1/d8/d8d/f12d 0 2024-02-19T19:06:29.027 INFO:tasks.workunit.client.0.smithi088.stdout:2/521: truncate d3/f6 4991330 0 2024-02-19T19:06:29.030 INFO:tasks.workunit.client.0.smithi088.stdout:2/522: chown d3/d1b/d43/d8f 133355009 1 2024-02-19T19:06:29.033 INFO:tasks.workunit.client.0.smithi088.stdout:5/761: dwrite d4/fcf [0,4194304] 0 2024-02-19T19:06:29.033 INFO:tasks.workunit.client.0.smithi088.stdout:9/942: dwrite d1/d8/d8d/d97/f53 [0,4194304] 0 2024-02-19T19:06:29.036 INFO:tasks.workunit.client.0.smithi088.stdout:5/762: creat d4/d1a/d41/dc1/dc2/f106 x:0 0 0 2024-02-19T19:06:29.044 INFO:tasks.workunit.client.0.smithi088.stdout:5/763: rename d4/d1a/d41/d5e to d4/d1a/d41/de2/de7/dff/d107 0 2024-02-19T19:06:29.045 INFO:tasks.workunit.client.0.smithi088.stdout:9/943: dread d1/d8/d37/d54/f75 [0,4194304] 0 2024-02-19T19:06:29.047 INFO:tasks.workunit.client.0.smithi088.stdout:5/764: chown d4/f12 0 1 2024-02-19T19:06:29.051 INFO:tasks.workunit.client.0.smithi088.stdout:9/944: creat d1/d8/d3e/d93/d2d/d18/d1f/df2/d99/f12e x:0 0 0 2024-02-19T19:06:29.057 INFO:tasks.workunit.client.0.smithi088.stdout:9/945: link d1/d8/d37/f9a d1/d8/d4a/dac/f12f 0 2024-02-19T19:06:29.062 INFO:tasks.workunit.client.0.smithi088.stdout:2/523: dread d3/fb [0,4194304] 0 2024-02-19T19:06:29.063 INFO:tasks.workunit.client.0.smithi088.stdout:9/946: creat d1/d8/d3e/d93/d5a/d94/f130 x:0 0 0 2024-02-19T19:06:29.066 INFO:tasks.workunit.client.0.smithi088.stdout:2/524: creat d3/d1b/d43/d61/f9e x:0 0 0 2024-02-19T19:06:29.073 INFO:tasks.workunit.client.0.smithi088.stdout:9/947: link d1/d8/d3e/d93/d2d/d18/d1f/db0/l95 d1/d8/d4a/dac/l131 0 2024-02-19T19:06:29.077 INFO:tasks.workunit.client.0.smithi088.stdout:9/948: mkdir d1/d8/d3e/d93/d122/d132 0 2024-02-19T19:06:29.078 INFO:tasks.workunit.client.0.smithi088.stdout:9/949: write d1/d8/f77 [31524,96592] 0 2024-02-19T19:06:29.079 INFO:tasks.workunit.client.0.smithi088.stdout:5/765: link d4/d75/c79 d4/d1a/d41/c108 0 2024-02-19T19:06:29.081 INFO:tasks.workunit.client.0.smithi088.stdout:5/766: fdatasync d4/d1a/d41/de2/de7/dff/d107/f82 0 2024-02-19T19:06:29.083 INFO:tasks.workunit.client.0.smithi088.stdout:9/950: mknod d1/d8/d3e/d93/d2d/d121/c133 0 2024-02-19T19:06:29.084 INFO:tasks.workunit.client.0.smithi088.stdout:2/525: dwrite d3/f85 [4194304,4194304] 0 2024-02-19T19:06:29.085 INFO:tasks.workunit.client.0.smithi088.stdout:2/526: chown d3/d1b/d43/d74 583 1 2024-02-19T19:06:29.086 INFO:tasks.workunit.client.0.smithi088.stdout:2/527: chown d3/d14/c38 0 1 2024-02-19T19:06:29.087 INFO:tasks.workunit.client.0.smithi088.stdout:2/528: write d3/d1b/d43/d61/d7e/f7f [643482,2980] 0 2024-02-19T19:06:29.088 INFO:tasks.workunit.client.0.smithi088.stdout:9/951: creat d1/d8/d3e/d93/d2d/d18/d1f/df2/dc8/f134 x:0 0 0 2024-02-19T19:06:29.089 INFO:tasks.workunit.client.0.smithi088.stdout:0/649: sync 2024-02-19T19:06:29.089 INFO:tasks.workunit.client.0.smithi088.stdout:6/662: sync 2024-02-19T19:06:29.090 INFO:tasks.workunit.client.0.smithi088.stdout:5/767: rename d4/f101 to d4/d1a/d41/de2/de7/f109 0 2024-02-19T19:06:29.091 INFO:tasks.workunit.client.0.smithi088.stdout:6/663: symlink d5/d19/d26/d72/ld3 0 2024-02-19T19:06:29.092 INFO:tasks.workunit.client.0.smithi088.stdout:0/650: mkdir d13/d62/d8a/d95/d5a/d25/d2a/d3c/d86/de5 0 2024-02-19T19:06:29.094 INFO:tasks.workunit.client.0.smithi088.stdout:6/664: mknod d5/d7/d32/cd4 0 2024-02-19T19:06:29.103 INFO:tasks.workunit.client.0.smithi088.stdout:0/651: rmdir d13/d62/d8a/d9c/da8/dab/dd0 39 2024-02-19T19:06:29.103 INFO:tasks.workunit.client.0.smithi088.stdout:6/665: rename d5/de/d64/d82/d8c/ccb to d5/d7/d32/cd5 0 2024-02-19T19:06:29.104 INFO:tasks.workunit.client.0.smithi088.stdout:9/952: creat d1/d8/d3e/d93/d2d/f135 x:0 0 0 2024-02-19T19:06:29.107 INFO:tasks.workunit.client.0.smithi088.stdout:9/953: creat d1/f136 x:0 0 0 2024-02-19T19:06:29.107 INFO:tasks.workunit.client.0.smithi088.stdout:5/768: mknod d4/d7/c10a 0 2024-02-19T19:06:29.108 INFO:tasks.workunit.client.0.smithi088.stdout:6/666: truncate d5/d45/d9d/fb2 2860145 0 2024-02-19T19:06:29.110 INFO:tasks.workunit.client.0.smithi088.stdout:6/667: write d5/d19/d70/f73 [577844,77953] 0 2024-02-19T19:06:29.110 INFO:tasks.workunit.client.0.smithi088.stdout:0/652: link d13/d62/d8a/fbb d13/d62/d8a/d87/fe6 0 2024-02-19T19:06:29.112 INFO:tasks.workunit.client.0.smithi088.stdout:5/769: write d4/d1a/f77 [3724438,52674] 0 2024-02-19T19:06:29.112 INFO:tasks.workunit.client.0.smithi088.stdout:6/668: write d5/d8/d3a/da8/fcf [563580,36593] 0 2024-02-19T19:06:29.113 INFO:tasks.workunit.client.0.smithi088.stdout:9/954: creat d1/d8/d3e/d93/d5a/d94/d104/f137 x:0 0 0 2024-02-19T19:06:29.115 INFO:tasks.workunit.client.0.smithi088.stdout:0/653: rename d13/d62/d8a/d95/d5a/l56 to d13/d62/d8a/d9c/le7 0 2024-02-19T19:06:29.117 INFO:tasks.workunit.client.0.smithi088.stdout:0/654: dread - d13/d62/d8a/d95/fc4 zero size 2024-02-19T19:06:29.119 INFO:tasks.workunit.client.0.smithi088.stdout:5/770: dwrite d4/d7/f15 [0,4194304] 0 2024-02-19T19:06:29.120 INFO:tasks.workunit.client.0.smithi088.stdout:9/955: mkdir d1/d8/d4a/d4f/daa/dbb/d138 0 2024-02-19T19:06:29.122 INFO:tasks.workunit.client.0.smithi088.stdout:6/669: mkdir d5/d8/d2c/dd6 0 2024-02-19T19:06:29.125 INFO:tasks.workunit.client.0.smithi088.stdout:6/670: dread - d5/d8/d3a/d71/daa/dc2/f99 zero size 2024-02-19T19:06:29.127 INFO:tasks.workunit.client.0.smithi088.stdout:5/771: creat d4/d1a/d41/de2/de7/dff/f10b x:0 0 0 2024-02-19T19:06:29.128 INFO:tasks.workunit.client.0.smithi088.stdout:0/655: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d2f/fb0 [0,4194304] 0 2024-02-19T19:06:29.130 INFO:tasks.workunit.client.0.smithi088.stdout:9/956: truncate d1/d8/d3e/d93/d2d/d18/f6a 861790 0 2024-02-19T19:06:29.135 INFO:tasks.workunit.client.0.smithi088.stdout:9/957: dread - d1/d8/d3e/d93/d5a/d94/f130 zero size 2024-02-19T19:06:29.138 INFO:tasks.workunit.client.0.smithi088.stdout:6/671: symlink d5/d8/d3a/d71/daa/dc2/dbc/db6/ld7 0 2024-02-19T19:06:29.140 INFO:tasks.workunit.client.0.smithi088.stdout:0/656: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d34/f43 [0,4194304] 0 2024-02-19T19:06:29.141 INFO:tasks.workunit.client.0.smithi088.stdout:6/672: mknod d5/d19/d26/cd8 0 2024-02-19T19:06:29.142 INFO:tasks.workunit.client.0.smithi088.stdout:9/958: dwrite d1/d8/d3e/d93/d2f/f7b [0,4194304] 0 2024-02-19T19:06:29.143 INFO:tasks.workunit.client.0.smithi088.stdout:0/657: stat d13/d62/d8a/d95/d5a/d25/d2a/d2f/f41 0 2024-02-19T19:06:29.145 INFO:tasks.workunit.client.0.smithi088.stdout:6/673: mknod d5/d19/d26/cd9 0 2024-02-19T19:06:29.145 INFO:tasks.workunit.client.0.smithi088.stdout:9/959: write d1/d8/d4a/d4f/daa/dbb/ff4 [5251632,713] 0 2024-02-19T19:06:29.146 INFO:tasks.workunit.client.0.smithi088.stdout:0/658: read d13/d62/d8a/d95/d5a/d25/f26 [391829,109313] 0 2024-02-19T19:06:29.149 INFO:tasks.workunit.client.0.smithi088.stdout:0/659: truncate d13/d62/d8a/d95/d5a/fa2 1533477 0 2024-02-19T19:06:29.149 INFO:tasks.workunit.client.0.smithi088.stdout:6/674: mkdir d5/d8/d3a/da8/dda 0 2024-02-19T19:06:29.150 INFO:tasks.workunit.client.0.smithi088.stdout:6/675: stat d5/d8/d2c/dd6 0 2024-02-19T19:06:29.151 INFO:tasks.workunit.client.0.smithi088.stdout:9/960: creat d1/d8/d37/d12c/f139 x:0 0 0 2024-02-19T19:06:29.152 INFO:tasks.workunit.client.0.smithi088.stdout:6/676: truncate d5/d45/fab 1331473 0 2024-02-19T19:06:29.152 INFO:tasks.workunit.client.0.smithi088.stdout:0/660: getdents d13/d62/d8a/d95/d5a/d25/dd6 0 2024-02-19T19:06:29.154 INFO:tasks.workunit.client.0.smithi088.stdout:6/677: write d5/d19/d26/d72/fa5 [1005885,102503] 0 2024-02-19T19:06:29.155 INFO:tasks.workunit.client.0.smithi088.stdout:6/678: fsync d5/de/d64/d8d/d9e/f9f 0 2024-02-19T19:06:29.156 INFO:tasks.workunit.client.0.smithi088.stdout:6/679: stat d5/d19/d26/c3c 0 2024-02-19T19:06:29.159 INFO:tasks.workunit.client.0.smithi088.stdout:6/680: getdents d5/d8/d2c/dd6 0 2024-02-19T19:06:29.162 INFO:tasks.workunit.client.0.smithi088.stdout:0/661: dwrite d13/f29 [0,4194304] 0 2024-02-19T19:06:29.162 INFO:tasks.workunit.client.0.smithi088.stdout:5/772: dread d4/f93 [0,4194304] 0 2024-02-19T19:06:29.162 INFO:tasks.workunit.client.0.smithi088.stdout:6/681: write d5/d19/f5f [2043483,26441] 0 2024-02-19T19:06:29.166 INFO:tasks.workunit.client.0.smithi088.stdout:6/682: creat d5/d8/d3a/d71/fdb x:0 0 0 2024-02-19T19:06:29.170 INFO:tasks.workunit.client.0.smithi088.stdout:9/961: rmdir d1/d8/d37/d12c 39 2024-02-19T19:06:29.171 INFO:tasks.workunit.client.0.smithi088.stdout:0/662: dwrite d13/d62/d8a/d95/d96/fc0 [0,4194304] 0 2024-02-19T19:06:29.171 INFO:tasks.workunit.client.0.smithi088.stdout:5/773: link d4/d1a/f54 d4/d1a/d41/de2/f10c 0 2024-02-19T19:06:29.176 INFO:tasks.workunit.client.0.smithi088.stdout:5/774: chown d4/d61/d65/d9a/d9f/c2d 43980950 1 2024-02-19T19:06:29.176 INFO:tasks.workunit.client.0.smithi088.stdout:9/962: symlink d1/d8/d3e/d93/d2d/l13a 0 2024-02-19T19:06:29.177 INFO:tasks.workunit.client.0.smithi088.stdout:5/775: chown d4/d61/d65/d9a/d9f/d57/c49 57538382 1 2024-02-19T19:06:29.178 INFO:tasks.workunit.client.0.smithi088.stdout:9/963: write d1/f6 [2053794,17096] 0 2024-02-19T19:06:29.184 INFO:tasks.workunit.client.0.smithi088.stdout:6/683: write d5/d8/d3a/da8/fce [791732,19725] 0 2024-02-19T19:06:29.184 INFO:tasks.workunit.client.0.smithi088.stdout:5/776: write d4/d61/f78 [934328,89974] 0 2024-02-19T19:06:29.185 INFO:tasks.workunit.client.0.smithi088.stdout:6/684: chown d5/d45/f75 1 1 2024-02-19T19:06:29.185 INFO:tasks.workunit.client.0.smithi088.stdout:5/777: write d4/d7/f8f [3351714,91611] 0 2024-02-19T19:06:29.186 INFO:tasks.workunit.client.0.smithi088.stdout:6/685: dread - d5/d8/d3a/d71/fa3 zero size 2024-02-19T19:06:29.189 INFO:tasks.workunit.client.0.smithi088.stdout:5/778: rename d4/d61/d65/d9a/d9f/d57/c76 to d4/d75/deb/c10d 0 2024-02-19T19:06:29.190 INFO:tasks.workunit.client.0.smithi088.stdout:6/686: read d5/d8/d3a/d71/daa/dc2/f39 [3044220,28465] 0 2024-02-19T19:06:29.190 INFO:tasks.workunit.client.0.smithi088.stdout:6/687: fdatasync d5/d7/f14 0 2024-02-19T19:06:29.191 INFO:tasks.workunit.client.0.smithi088.stdout:5/779: mknod d4/d1a/d41/dc1/dc2/dfa/c10e 0 2024-02-19T19:06:29.191 INFO:tasks.workunit.client.0.smithi088.stdout:5/780: readlink d4/d61/ld5 0 2024-02-19T19:06:29.192 INFO:tasks.workunit.client.0.smithi088.stdout:6/688: dread - d5/d8/d3a/d71/daa/dc2/dbc/fa4 zero size 2024-02-19T19:06:29.195 INFO:tasks.workunit.client.0.smithi088.stdout:5/781: rename d4/d1a/c7b to d4/d75/deb/c10f 0 2024-02-19T19:06:29.197 INFO:tasks.workunit.client.0.smithi088.stdout:6/689: link d5/d19/f41 d5/d8/d3a/db8/fdc 0 2024-02-19T19:06:29.198 INFO:tasks.workunit.client.0.smithi088.stdout:6/690: read - d5/d7/d32/d56/dbf/fcd zero size 2024-02-19T19:06:29.199 INFO:tasks.workunit.client.0.smithi088.stdout:5/782: creat d4/d61/f110 x:0 0 0 2024-02-19T19:06:29.200 INFO:tasks.workunit.client.0.smithi088.stdout:6/691: creat d5/d8/d3a/da8/fdd x:0 0 0 2024-02-19T19:06:29.200 INFO:tasks.workunit.client.0.smithi088.stdout:5/783: creat d4/d1a/d41/dc1/f111 x:0 0 0 2024-02-19T19:06:29.208 INFO:tasks.workunit.client.0.smithi088.stdout:6/692: truncate d5/d8/f3f 6590075 0 2024-02-19T19:06:29.209 INFO:tasks.workunit.client.0.smithi088.stdout:5/784: truncate d4/f87 1495016 0 2024-02-19T19:06:29.210 INFO:tasks.workunit.client.0.smithi088.stdout:6/693: write d5/d8/d3a/da8/fdd [747834,83104] 0 2024-02-19T19:06:29.212 INFO:tasks.workunit.client.0.smithi088.stdout:5/785: creat d4/d1a/d41/de2/f112 x:0 0 0 2024-02-19T19:06:29.216 INFO:tasks.workunit.client.0.smithi088.stdout:5/786: link d4/d61/d65/d9a/d9f/f4c d4/d7/f113 0 2024-02-19T19:06:29.218 INFO:tasks.workunit.client.0.smithi088.stdout:6/694: truncate d5/d19/d26/d72/fa5 680230 0 2024-02-19T19:06:29.220 INFO:tasks.workunit.client.0.smithi088.stdout:5/787: unlink d4/d61/d65/d9a/cdf 0 2024-02-19T19:06:29.223 INFO:tasks.workunit.client.0.smithi088.stdout:6/695: symlink d5/d8/d3a/lde 0 2024-02-19T19:06:29.224 INFO:tasks.workunit.client.0.smithi088.stdout:5/788: unlink d4/d61/f110 0 2024-02-19T19:06:29.225 INFO:tasks.workunit.client.0.smithi088.stdout:5/789: dread - d4/d75/fbf zero size 2024-02-19T19:06:29.227 INFO:tasks.workunit.client.0.smithi088.stdout:5/790: mkdir d4/d1a/d41/de2/de7/dff/d114 0 2024-02-19T19:06:29.228 INFO:tasks.workunit.client.0.smithi088.stdout:6/696: link d5/d8/d3a/f63 d5/d7/d32/d56/fdf 0 2024-02-19T19:06:29.232 INFO:tasks.workunit.client.0.smithi088.stdout:5/791: getdents d4/d1a 0 2024-02-19T19:06:29.233 INFO:tasks.workunit.client.0.smithi088.stdout:5/792: fdatasync d4/d1a/fc3 0 2024-02-19T19:06:29.239 INFO:tasks.workunit.client.0.smithi088.stdout:5/793: dwrite d4/d61/fb6 [0,4194304] 0 2024-02-19T19:06:29.240 INFO:tasks.workunit.client.0.smithi088.stdout:5/794: stat d4/d61/d65/d9a/d9f/c18 0 2024-02-19T19:06:29.248 INFO:tasks.workunit.client.0.smithi088.stdout:5/795: dwrite d4/d7/f92 [0,4194304] 0 2024-02-19T19:06:29.250 INFO:tasks.workunit.client.0.smithi088.stdout:5/796: creat d4/d1a/d41/dc1/f115 x:0 0 0 2024-02-19T19:06:29.252 INFO:tasks.workunit.client.0.smithi088.stdout:5/797: creat d4/d1a/d41/dc1/dc2/dfa/f116 x:0 0 0 2024-02-19T19:06:29.255 INFO:tasks.workunit.client.0.smithi088.stdout:6/697: dread d5/d7/d32/f53 [0,4194304] 0 2024-02-19T19:06:29.256 INFO:tasks.workunit.client.0.smithi088.stdout:5/798: creat d4/d61/d65/d9a/d9f/f117 x:0 0 0 2024-02-19T19:06:29.257 INFO:tasks.workunit.client.0.smithi088.stdout:6/698: write d5/de/d64/d82/d8c/f59 [956085,90475] 0 2024-02-19T19:06:29.258 INFO:tasks.workunit.client.0.smithi088.stdout:5/799: creat d4/d1a/d41/dcb/f118 x:0 0 0 2024-02-19T19:06:29.263 INFO:tasks.workunit.client.0.smithi088.stdout:5/800: link d4/d61/d65/d9a/d9f/f4e d4/d61/d65/d9a/d9f/d57/d84/f119 0 2024-02-19T19:06:29.266 INFO:tasks.workunit.client.0.smithi088.stdout:5/801: link d4/d75/cca d4/d1a/d41/dc1/dc2/c11a 0 2024-02-19T19:06:29.268 INFO:tasks.workunit.client.0.smithi088.stdout:5/802: fsync d4/fcf 0 2024-02-19T19:06:29.269 INFO:tasks.workunit.client.0.smithi088.stdout:5/803: dread - d4/d1a/d41/de2/f112 zero size 2024-02-19T19:06:29.270 INFO:tasks.workunit.client.0.smithi088.stdout:5/804: rmdir d4/d61/d65/d9a/d9f/d57/d84 39 2024-02-19T19:06:29.271 INFO:tasks.workunit.client.0.smithi088.stdout:6/699: dread d5/d19/f5f [0,4194304] 0 2024-02-19T19:06:29.274 INFO:tasks.workunit.client.0.smithi088.stdout:5/805: dread d4/d1a/d41/f71 [0,4194304] 0 2024-02-19T19:06:29.275 INFO:tasks.workunit.client.0.smithi088.stdout:6/700: fdatasync d5/de/f10 0 2024-02-19T19:06:29.279 INFO:tasks.workunit.client.0.smithi088.stdout:6/701: dread - d5/d7/d32/d56/dbf/fcd zero size 2024-02-19T19:06:29.279 INFO:tasks.workunit.client.0.smithi088.stdout:5/806: mknod d4/d1a/d41/dcb/c11b 0 2024-02-19T19:06:29.280 INFO:tasks.workunit.client.0.smithi088.stdout:5/807: truncate d4/d1a/d41/fcc 195693 0 2024-02-19T19:06:29.280 INFO:tasks.workunit.client.0.smithi088.stdout:5/808: symlink d4/d1a/d41/de2/de7/dff/d107/l11c 0 2024-02-19T19:06:29.281 INFO:tasks.workunit.client.0.smithi088.stdout:5/809: chown d4/d1a/d41/f7a 3697 1 2024-02-19T19:06:29.283 INFO:tasks.workunit.client.0.smithi088.stdout:6/702: dwrite d5/d8/d3a/d71/daa/dc2/f49 [0,4194304] 0 2024-02-19T19:06:29.283 INFO:tasks.workunit.client.0.smithi088.stdout:5/810: unlink d4/d7/fc4 0 2024-02-19T19:06:29.285 INFO:tasks.workunit.client.0.smithi088.stdout:5/811: write d4/d1a/d41/de2/f112 [19673,1764] 0 2024-02-19T19:06:29.285 INFO:tasks.workunit.client.0.smithi088.stdout:6/703: symlink d5/de/d64/d7e/le0 0 2024-02-19T19:06:29.288 INFO:tasks.workunit.client.0.smithi088.stdout:6/704: chown d5/de/c52 2285802 1 2024-02-19T19:06:29.290 INFO:tasks.workunit.client.0.smithi088.stdout:5/812: link d4/d1a/l50 d4/d1a/d41/dc1/dc2/dfa/l11d 0 2024-02-19T19:06:29.291 INFO:tasks.workunit.client.0.smithi088.stdout:8/584: sync 2024-02-19T19:06:29.291 INFO:tasks.workunit.client.0.smithi088.stdout:4/526: sync 2024-02-19T19:06:29.291 INFO:tasks.workunit.client.0.smithi088.stdout:3/739: sync 2024-02-19T19:06:29.291 INFO:tasks.workunit.client.0.smithi088.stdout:1/695: sync 2024-02-19T19:06:29.292 INFO:tasks.workunit.client.0.smithi088.stdout:7/710: sync 2024-02-19T19:06:29.292 INFO:tasks.workunit.client.0.smithi088.stdout:6/705: creat d5/d19/fe1 x:0 0 0 2024-02-19T19:06:29.294 INFO:tasks.workunit.client.0.smithi088.stdout:8/585: dread - d9/df/d3d/d50/d71/da1/f9b zero size 2024-02-19T19:06:29.294 INFO:tasks.workunit.client.0.smithi088.stdout:3/740: dread - d4/d1a/fb8 zero size 2024-02-19T19:06:29.295 INFO:tasks.workunit.client.0.smithi088.stdout:4/527: stat d2/d35/f7c 0 2024-02-19T19:06:29.296 INFO:tasks.workunit.client.0.smithi088.stdout:1/696: rename d6/d10/f93 to d6/d21/d64/fe5 0 2024-02-19T19:06:29.297 INFO:tasks.workunit.client.0.smithi088.stdout:6/706: mknod d5/d19/ce2 0 2024-02-19T19:06:29.300 INFO:tasks.workunit.client.0.smithi088.stdout:8/586: dread d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8/f89 [0,4194304] 0 2024-02-19T19:06:29.301 INFO:tasks.workunit.client.0.smithi088.stdout:8/587: readlink d9/df/l7a 0 2024-02-19T19:06:29.303 INFO:tasks.workunit.client.0.smithi088.stdout:5/813: rmdir d4/d1a/d41/dc1/dc2/dfa 39 2024-02-19T19:06:29.304 INFO:tasks.workunit.client.0.smithi088.stdout:1/697: readlink d6/la 0 2024-02-19T19:06:29.307 INFO:tasks.workunit.client.0.smithi088.stdout:4/528: mknod d2/d32/d25/d26/d6d/d7e/d90/ca9 0 2024-02-19T19:06:29.307 INFO:tasks.workunit.client.0.smithi088.stdout:8/588: readlink d9/df/l4c 0 2024-02-19T19:06:29.308 INFO:tasks.workunit.client.0.smithi088.stdout:1/698: mknod d6/d21/d6a/d7d/ce6 0 2024-02-19T19:06:29.310 INFO:tasks.workunit.client.0.smithi088.stdout:7/711: link d14/d1d/d3c/fb7 d14/d1d/d62/f100 0 2024-02-19T19:06:29.311 INFO:tasks.workunit.client.0.smithi088.stdout:8/589: write d9/df/d1c/f25 [9842972,86858] 0 2024-02-19T19:06:29.313 INFO:tasks.workunit.client.0.smithi088.stdout:3/741: getdents d4/d22/d7a/d82 0 2024-02-19T19:06:29.316 INFO:tasks.workunit.client.0.smithi088.stdout:1/699: symlink d6/le7 0 2024-02-19T19:06:29.318 INFO:tasks.workunit.client.0.smithi088.stdout:7/712: symlink d14/d1d/d3c/d43/dc2/l101 0 2024-02-19T19:06:29.318 INFO:tasks.workunit.client.0.smithi088.stdout:8/590: write d9/f2e [2487868,35460] 0 2024-02-19T19:06:29.318 INFO:tasks.workunit.client.0.smithi088.stdout:5/814: link d4/d1a/d41/l5c d4/d61/d65/d9a/d9f/d57/l11e 0 2024-02-19T19:06:29.320 INFO:tasks.workunit.client.0.smithi088.stdout:1/700: read d6/d10/f4d [1055863,46162] 0 2024-02-19T19:06:29.321 INFO:tasks.workunit.client.0.smithi088.stdout:8/591: fdatasync d9/df/f1a 0 2024-02-19T19:06:29.323 INFO:tasks.workunit.client.0.smithi088.stdout:4/529: link d2/d32/d25/f38 d2/d32/d25/d26/d6d/d7e/d3a/faa 0 2024-02-19T19:06:29.323 INFO:tasks.workunit.client.0.smithi088.stdout:3/742: dwrite d4/d1a/d1e/d74/f96 [4194304,4194304] 0 2024-02-19T19:06:29.325 INFO:tasks.workunit.client.0.smithi088.stdout:6/707: dread d5/d8/f7b [0,4194304] 0 2024-02-19T19:06:29.325 INFO:tasks.workunit.client.0.smithi088.stdout:1/701: chown d6/d21/d6a/d7d/fa6 2 1 2024-02-19T19:06:29.325 INFO:tasks.workunit.client.0.smithi088.stdout:3/743: fsync d4/d22/d7a/ff0 0 2024-02-19T19:06:29.328 INFO:tasks.workunit.client.0.smithi088.stdout:6/708: write d5/d45/d9d/fb2 [665257,122990] 0 2024-02-19T19:06:29.331 INFO:tasks.workunit.client.0.smithi088.stdout:7/713: rename d14/d1d/dbf to d14/d27/d6b/dba/d102 0 2024-02-19T19:06:29.333 INFO:tasks.workunit.client.0.smithi088.stdout:8/592: unlink d9/df/d3d/c74 0 2024-02-19T19:06:29.333 INFO:tasks.workunit.client.0.smithi088.stdout:7/714: read - d14/d28/d94/ddb/ff1 zero size 2024-02-19T19:06:29.335 INFO:tasks.workunit.client.0.smithi088.stdout:3/744: dwrite d4/d22/d7a/d8d/fd2 [0,4194304] 0 2024-02-19T19:06:29.336 INFO:tasks.workunit.client.0.smithi088.stdout:8/593: truncate d9/df/d1c/f73 1618470 0 2024-02-19T19:06:29.339 INFO:tasks.workunit.client.0.smithi088.stdout:8/594: chown d9/df/d3d/d50/d71/da1/d6c/d9a/c51 0 1 2024-02-19T19:06:29.341 INFO:tasks.workunit.client.0.smithi088.stdout:6/709: rmdir d5/d8/d3a/d71/daa 39 2024-02-19T19:06:29.342 INFO:tasks.workunit.client.0.smithi088.stdout:6/710: stat d5/d8/d2c/l35 0 2024-02-19T19:06:29.342 INFO:tasks.workunit.client.0.smithi088.stdout:6/711: chown d5/d7/d32/f8e 89324 1 2024-02-19T19:06:29.346 INFO:tasks.workunit.client.0.smithi088.stdout:7/715: rename d14/d28/d3b/fc9 to d14/d28/d94/ddb/f103 0 2024-02-19T19:06:29.349 INFO:tasks.workunit.client.0.smithi088.stdout:4/530: dwrite d2/f4 [0,4194304] 0 2024-02-19T19:06:29.350 INFO:tasks.workunit.client.0.smithi088.stdout:4/531: chown d2/d32/d25 1429 1 2024-02-19T19:06:29.351 INFO:tasks.workunit.client.0.smithi088.stdout:5/815: dread d4/d61/d65/d9a/d9f/f4c [0,4194304] 0 2024-02-19T19:06:29.352 INFO:tasks.workunit.client.0.smithi088.stdout:6/712: readlink d5/d8/d3a/d71/daa/dc2/dbc/lbb 0 2024-02-19T19:06:29.353 INFO:tasks.workunit.client.0.smithi088.stdout:3/745: dread d4/ff [0,4194304] 0 2024-02-19T19:06:29.354 INFO:tasks.workunit.client.0.smithi088.stdout:7/716: symlink d14/d27/d6b/l104 0 2024-02-19T19:06:29.357 INFO:tasks.workunit.client.0.smithi088.stdout:4/532: creat d2/d32/d25/d26/d6d/d7e/d90/fab x:0 0 0 2024-02-19T19:06:29.360 INFO:tasks.workunit.client.0.smithi088.stdout:7/717: write d14/d28/d94/fbc [71981,102489] 0 2024-02-19T19:06:29.361 INFO:tasks.workunit.client.0.smithi088.stdout:6/713: truncate d5/de/d64/d82/d8c/f59 1486424 0 2024-02-19T19:06:29.362 INFO:tasks.workunit.client.0.smithi088.stdout:3/746: chown d4/f79 1817 1 2024-02-19T19:06:29.365 INFO:tasks.workunit.client.0.smithi088.stdout:8/595: dwrite d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/f7b [0,4194304] 0 2024-02-19T19:06:29.366 INFO:tasks.workunit.client.0.smithi088.stdout:7/718: stat d14/d27/dd2/ff5 0 2024-02-19T19:06:29.368 INFO:tasks.workunit.client.0.smithi088.stdout:4/533: symlink d2/d32/d25/d26/d4e/d6f/lac 0 2024-02-19T19:06:29.369 INFO:tasks.workunit.client.0.smithi088.stdout:6/714: creat d5/de/d64/d7e/fe3 x:0 0 0 2024-02-19T19:06:29.371 INFO:tasks.workunit.client.0.smithi088.stdout:1/702: dwrite d6/d10/d60/d29/fb3 [0,4194304] 0 2024-02-19T19:06:29.371 INFO:tasks.workunit.client.0.smithi088.stdout:5/816: dwrite d4/d1a/f3c [4194304,4194304] 0 2024-02-19T19:06:29.373 INFO:tasks.workunit.client.0.smithi088.stdout:8/596: creat d9/df/d3d/d62/fb1 x:0 0 0 2024-02-19T19:06:29.373 INFO:tasks.workunit.client.0.smithi088.stdout:3/747: truncate d4/f6 3128991 0 2024-02-19T19:06:29.374 INFO:tasks.workunit.client.0.smithi088.stdout:7/719: write d14/d1d/d3c/d43/d5c/f9f [2564838,44203] 0 2024-02-19T19:06:29.376 INFO:tasks.workunit.client.0.smithi088.stdout:6/715: fdatasync d5/d8/d3a/d71/f74 0 2024-02-19T19:06:29.377 INFO:tasks.workunit.client.0.smithi088.stdout:4/534: symlink d2/d32/d25/d4b/lad 0 2024-02-19T19:06:29.380 INFO:tasks.workunit.client.0.smithi088.stdout:4/535: stat d2/d35/f37 0 2024-02-19T19:06:29.380 INFO:tasks.workunit.client.0.smithi088.stdout:7/720: readlink d14/d1d/d3c/d43/d5c/d9a/d92/lc3 0 2024-02-19T19:06:29.381 INFO:tasks.workunit.client.0.smithi088.stdout:3/748: chown d4/d1a/d23 61 1 2024-02-19T19:06:29.382 INFO:tasks.workunit.client.0.smithi088.stdout:8/597: write d9/df/d3d/d50/d71/da1/f6d [424267,15291] 0 2024-02-19T19:06:29.391 INFO:tasks.workunit.client.0.smithi088.stdout:1/703: dwrite d6/d21/f84 [0,4194304] 0 2024-02-19T19:06:29.393 INFO:tasks.workunit.client.0.smithi088.stdout:4/536: getdents d2/d32/d25/d26/d4e/d6f 0 2024-02-19T19:06:29.393 INFO:tasks.workunit.client.0.smithi088.stdout:6/716: dwrite d5/d45/fab [0,4194304] 0 2024-02-19T19:06:29.394 INFO:tasks.workunit.client.0.smithi088.stdout:7/721: creat d14/d1d/f105 x:0 0 0 2024-02-19T19:06:29.396 INFO:tasks.workunit.client.0.smithi088.stdout:8/598: rename d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/f81 to d9/df/d1c/d85/fb2 0 2024-02-19T19:06:29.402 INFO:tasks.workunit.client.0.smithi088.stdout:1/704: symlink d6/d10/le8 0 2024-02-19T19:06:29.405 INFO:tasks.workunit.client.0.smithi088.stdout:4/537: creat d2/d32/d25/d3f/d70/fae x:0 0 0 2024-02-19T19:06:29.407 INFO:tasks.workunit.client.0.smithi088.stdout:7/722: creat d14/d1d/d3c/d43/db2/f106 x:0 0 0 2024-02-19T19:06:29.408 INFO:tasks.workunit.client.0.smithi088.stdout:8/599: readlink d9/df/d1c/d22/l24 0 2024-02-19T19:06:29.412 INFO:tasks.workunit.client.0.smithi088.stdout:4/538: dread - d2/d32/d25/d26/d6d/d7e/d3a/f74 zero size 2024-02-19T19:06:29.414 INFO:tasks.workunit.client.0.smithi088.stdout:5/817: dwrite d4/d1a/d41/de2/de7/dff/d107/f82 [0,4194304] 0 2024-02-19T19:06:29.422 INFO:tasks.workunit.client.0.smithi088.stdout:2/529: sync 2024-02-19T19:06:29.425 INFO:tasks.workunit.client.0.smithi088.stdout:6/717: rename d5/d8/d3a/d71/daa/dc2/dbc/db6/ld7 to d5/d8/d3a/d71/daa/dc2/le4 0 2024-02-19T19:06:29.435 INFO:tasks.workunit.client.0.smithi088.stdout:1/705: dread d6/d10/d60/d29/f2e [4194304,4194304] 0 2024-02-19T19:06:29.438 INFO:tasks.workunit.client.0.smithi088.stdout:1/706: dread d6/d21/f84 [0,4194304] 0 2024-02-19T19:06:29.440 INFO:tasks.workunit.client.0.smithi088.stdout:3/749: truncate d4/d1a/d1e/f2f 6691568 0 2024-02-19T19:06:29.446 INFO:tasks.workunit.client.0.smithi088.stdout:4/539: unlink d2/d32/l12 0 2024-02-19T19:06:29.451 INFO:tasks.workunit.client.0.smithi088.stdout:7/723: rename d14/d27/dd2 to d14/d27/d49/d98/d107 0 2024-02-19T19:06:29.460 INFO:tasks.workunit.client.0.smithi088.stdout:5/818: symlink d4/d1a/l11f 0 2024-02-19T19:06:29.460 INFO:tasks.workunit.client.0.smithi088.stdout:6/718: read d5/d19/d26/d72/fa5 [373250,76102] 0 2024-02-19T19:06:29.460 INFO:tasks.workunit.client.0.smithi088.stdout:1/707: mknod d6/d10/ce9 0 2024-02-19T19:06:29.461 INFO:tasks.workunit.client.0.smithi088.stdout:6/719: stat d5/d8/d2c/c69 0 2024-02-19T19:06:29.461 INFO:tasks.workunit.client.0.smithi088.stdout:6/720: chown d5 17 1 2024-02-19T19:06:29.462 INFO:tasks.workunit.client.0.smithi088.stdout:8/600: mkdir d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/db3 0 2024-02-19T19:06:29.463 INFO:tasks.workunit.client.0.smithi088.stdout:6/721: write d5/de/d60/fac [8901,76241] 0 2024-02-19T19:06:29.466 INFO:tasks.workunit.client.0.smithi088.stdout:3/750: rmdir d4/d22/dd5 39 2024-02-19T19:06:29.470 INFO:tasks.workunit.client.0.smithi088.stdout:8/601: dwrite d9/df/d3d/d62/fb1 [0,4194304] 0 2024-02-19T19:06:29.471 INFO:tasks.workunit.client.0.smithi088.stdout:7/724: truncate d14/d1d/d62/dcf/fe4 406915 0 2024-02-19T19:06:29.598 INFO:tasks.workunit.client.0.smithi088.stdout:2/530: unlink d3/d1f/c50 0 2024-02-19T19:06:29.607 INFO:tasks.workunit.client.0.smithi088.stdout:3/751: creat d4/d1a/d23/f108 x:0 0 0 2024-02-19T19:06:29.610 INFO:tasks.workunit.client.0.smithi088.stdout:7/725: mkdir d14/d1d/d62/dcf/d108 0 2024-02-19T19:06:29.610 INFO:tasks.workunit.client.0.smithi088.stdout:2/531: symlink d3/d14/l9f 0 2024-02-19T19:06:29.617 INFO:tasks.workunit.client.0.smithi088.stdout:8/602: creat d9/df/fb4 x:0 0 0 2024-02-19T19:06:29.617 INFO:tasks.workunit.client.0.smithi088.stdout:8/603: chown d9/df/d1c/d22 11200545 1 2024-02-19T19:06:29.619 INFO:tasks.workunit.client.0.smithi088.stdout:5/819: rename d4/d75/deb/c10d to d4/d61/d65/d9a/d9f/da0/c120 0 2024-02-19T19:06:29.620 INFO:tasks.workunit.client.0.smithi088.stdout:6/722: getdents d5/d7/d32/d56/dbf 0 2024-02-19T19:06:29.622 INFO:tasks.workunit.client.0.smithi088.stdout:8/604: creat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8/fb5 x:0 0 0 2024-02-19T19:06:29.622 INFO:tasks.workunit.client.0.smithi088.stdout:4/540: dread d2/d8/f29 [0,4194304] 0 2024-02-19T19:06:29.624 INFO:tasks.workunit.client.0.smithi088.stdout:8/605: write d9/df/d3d/f87 [48571,91450] 0 2024-02-19T19:06:29.624 INFO:tasks.workunit.client.0.smithi088.stdout:6/723: mkdir d5/d7/d32/de5 0 2024-02-19T19:06:29.625 INFO:tasks.workunit.client.0.smithi088.stdout:6/724: readlink d5/d19/d26/d72/ld3 0 2024-02-19T19:06:29.626 INFO:tasks.workunit.client.0.smithi088.stdout:6/725: dread - d5/de/fc6 zero size 2024-02-19T19:06:29.626 INFO:tasks.workunit.client.0.smithi088.stdout:5/820: creat d4/d61/d65/d9a/d9f/d57/d84/dfb/f121 x:0 0 0 2024-02-19T19:06:29.631 INFO:tasks.workunit.client.0.smithi088.stdout:8/606: mknod d9/cb6 0 2024-02-19T19:06:29.633 INFO:tasks.workunit.client.0.smithi088.stdout:5/821: creat d4/d1a/d41/de2/de7/dff/d107/f122 x:0 0 0 2024-02-19T19:06:29.634 INFO:tasks.workunit.client.0.smithi088.stdout:6/726: dread d5/f1e [0,4194304] 0 2024-02-19T19:06:29.641 INFO:tasks.workunit.client.0.smithi088.stdout:8/607: dwrite d9/df/d1c/f16 [4194304,4194304] 0 2024-02-19T19:06:29.645 INFO:tasks.workunit.client.0.smithi088.stdout:4/541: dread d2/d32/d25/f30 [0,4194304] 0 2024-02-19T19:06:29.645 INFO:tasks.workunit.client.0.smithi088.stdout:6/727: dwrite d5/de/d60/fac [0,4194304] 0 2024-02-19T19:06:29.646 INFO:tasks.workunit.client.0.smithi088.stdout:6/728: write d5/d8/d3a/f46 [10333878,23969] 0 2024-02-19T19:06:29.647 INFO:tasks.workunit.client.0.smithi088.stdout:6/729: dread - d5/d45/d9d/fa2 zero size 2024-02-19T19:06:29.656 INFO:tasks.workunit.client.0.smithi088.stdout:4/542: rmdir d2/d32/d25/d3f/da2 0 2024-02-19T19:06:29.656 INFO:tasks.workunit.client.0.smithi088.stdout:4/543: chown d2/d35/f37 3 1 2024-02-19T19:06:29.658 INFO:tasks.workunit.client.0.smithi088.stdout:4/544: creat d2/d32/d25/faf x:0 0 0 2024-02-19T19:06:29.658 INFO:tasks.workunit.client.0.smithi088.stdout:2/532: write d3/d14/f1d [3307614,84073] 0 2024-02-19T19:06:29.660 INFO:tasks.workunit.client.0.smithi088.stdout:2/533: read d3/d1b/d43/d61/d64/f78 [929516,25381] 0 2024-02-19T19:06:29.661 INFO:tasks.workunit.client.0.smithi088.stdout:2/534: dread - d3/d1b/d43/d61/f9e zero size 2024-02-19T19:06:29.661 INFO:tasks.workunit.client.0.smithi088.stdout:4/545: dread - d2/d32/d25/d26/d6d/d7e/f72 zero size 2024-02-19T19:06:29.662 INFO:tasks.workunit.client.0.smithi088.stdout:7/726: truncate d14/d27/d6b/f93 413170 0 2024-02-19T19:06:29.664 INFO:tasks.workunit.client.0.smithi088.stdout:2/535: symlink d3/d14/la0 0 2024-02-19T19:06:29.668 INFO:tasks.workunit.client.0.smithi088.stdout:7/727: unlink d14/d1d/d62/dfa/lfd 0 2024-02-19T19:06:29.668 INFO:tasks.workunit.client.0.smithi088.stdout:5/822: getdents d4/d61/d65/d9a/d9f/d57/d84/dfb 0 2024-02-19T19:06:29.672 INFO:tasks.workunit.client.0.smithi088.stdout:2/536: mknod d3/d14/d93/ca1 0 2024-02-19T19:06:29.674 INFO:tasks.workunit.client.0.smithi088.stdout:5/823: mkdir d4/d1a/d41/de2/de7/dff/d114/d123 0 2024-02-19T19:06:29.676 INFO:tasks.workunit.client.0.smithi088.stdout:8/608: truncate d9/df/d3d/d62/fb1 37449 0 2024-02-19T19:06:29.676 INFO:tasks.workunit.client.0.smithi088.stdout:6/730: dread d5/d19/f4e [0,4194304] 0 2024-02-19T19:06:29.676 INFO:tasks.workunit.client.0.smithi088.stdout:7/728: dwrite d14/d28/fe0 [0,4194304] 0 2024-02-19T19:06:29.677 INFO:tasks.workunit.client.0.smithi088.stdout:7/729: chown d14/ff9 129 1 2024-02-19T19:06:29.678 INFO:tasks.workunit.client.0.smithi088.stdout:7/730: stat d14/d1d/f8a 0 2024-02-19T19:06:29.678 INFO:tasks.workunit.client.0.smithi088.stdout:0/663: sync 2024-02-19T19:06:29.678 INFO:tasks.workunit.client.0.smithi088.stdout:9/964: sync 2024-02-19T19:06:29.680 INFO:tasks.workunit.client.0.smithi088.stdout:7/731: chown d14/d1d/d3c/d43/d5c/d75/fad 0 1 2024-02-19T19:06:29.686 INFO:tasks.workunit.client.0.smithi088.stdout:2/537: creat d3/d1f/fa2 x:0 0 0 2024-02-19T19:06:29.687 INFO:tasks.workunit.client.0.smithi088.stdout:2/538: fdatasync d3/d1b/f1c 0 2024-02-19T19:06:29.688 INFO:tasks.workunit.client.0.smithi088.stdout:4/546: write d2/d32/d25/d26/d6d/d7e/f72 [30324,9126] 0 2024-02-19T19:06:29.692 INFO:tasks.workunit.client.0.smithi088.stdout:7/732: mknod d14/d1d/d3c/d43/d5c/d9a/c109 0 2024-02-19T19:06:29.693 INFO:tasks.workunit.client.0.smithi088.stdout:6/731: rmdir d5 39 2024-02-19T19:06:29.695 INFO:tasks.workunit.client.0.smithi088.stdout:5/824: dread d4/d1a/f3c [0,4194304] 0 2024-02-19T19:06:29.695 INFO:tasks.workunit.client.0.smithi088.stdout:5/825: dread - d4/d61/d65/f105 zero size 2024-02-19T19:06:29.701 INFO:tasks.workunit.client.0.smithi088.stdout:0/664: truncate d13/d62/d8a/d95/d5a/d25/d2a/d34/f35 1470571 0 2024-02-19T19:06:29.705 INFO:tasks.workunit.client.0.smithi088.stdout:2/539: mknod d3/d1b/d43/ca3 0 2024-02-19T19:06:29.706 INFO:tasks.workunit.client.0.smithi088.stdout:4/547: rmdir d2/d32/d25/d26/d6d/d7e/d90 39 2024-02-19T19:06:29.708 INFO:tasks.workunit.client.0.smithi088.stdout:7/733: chown d14/f3e 2521 1 2024-02-19T19:06:29.711 INFO:tasks.workunit.client.0.smithi088.stdout:0/665: mknod d13/d62/d8a/d95/d5a/d25/dd6/ce8 0 2024-02-19T19:06:29.714 INFO:tasks.workunit.client.0.smithi088.stdout:2/540: creat d3/d1b/d43/d61/d64/fa4 x:0 0 0 2024-02-19T19:06:29.717 INFO:tasks.workunit.client.0.smithi088.stdout:6/732: creat d5/d19/d26/fe6 x:0 0 0 2024-02-19T19:06:29.718 INFO:tasks.workunit.client.0.smithi088.stdout:9/965: link d1/d8/d3e/d93/l5b d1/d8/d3e/d93/d2d/d18/d1f/df2/l13b 0 2024-02-19T19:06:29.719 INFO:tasks.workunit.client.0.smithi088.stdout:8/609: dwrite d9/df/d3d/d62/fb1 [0,4194304] 0 2024-02-19T19:06:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:6/733: chown d5/d19/d26/d72/ld3 21931099 1 2024-02-19T19:06:29.720 INFO:tasks.workunit.client.0.smithi088.stdout:7/734: mkdir d14/d27/d49/d98/dcb/de7/d10a 0 2024-02-19T19:06:29.721 INFO:tasks.workunit.client.0.smithi088.stdout:9/966: write d1/d8/fd6 [1513774,50255] 0 2024-02-19T19:06:29.723 INFO:tasks.workunit.client.0.smithi088.stdout:0/666: mkdir d13/de9 0 2024-02-19T19:06:29.724 INFO:tasks.workunit.client.0.smithi088.stdout:6/734: chown d5/d19/l57 1 1 2024-02-19T19:06:29.725 INFO:tasks.workunit.client.0.smithi088.stdout:7/735: chown d14/d1d/f105 3211 1 2024-02-19T19:06:29.726 INFO:tasks.workunit.client.0.smithi088.stdout:5/826: dread d4/d61/d65/d9a/d9f/d57/d84/f8a [0,4194304] 0 2024-02-19T19:06:29.727 INFO:tasks.workunit.client.0.smithi088.stdout:6/735: write d5/d7/d32/d56/f94 [579786,127443] 0 2024-02-19T19:06:29.728 INFO:tasks.workunit.client.0.smithi088.stdout:2/541: creat d3/fa5 x:0 0 0 2024-02-19T19:06:29.731 INFO:tasks.workunit.client.0.smithi088.stdout:6/736: write d5/d8/d3a/d71/daa/dc2/f49 [2418364,111921] 0 2024-02-19T19:06:29.732 INFO:tasks.workunit.client.0.smithi088.stdout:2/542: unlink d3/d1b/l7d 0 2024-02-19T19:06:29.732 INFO:tasks.workunit.client.0.smithi088.stdout:8/610: getdents d9/df/d3d/d50/d71/da1/d6c/d77 0 2024-02-19T19:06:29.733 INFO:tasks.workunit.client.0.smithi088.stdout:9/967: dwrite d1/d8/d3e/d93/d2f/f7b [0,4194304] 0 2024-02-19T19:06:29.736 INFO:tasks.workunit.client.0.smithi088.stdout:2/543: fdatasync d3/d1b/d43/d61/d64/f6a 0 2024-02-19T19:06:29.737 INFO:tasks.workunit.client.0.smithi088.stdout:5/827: truncate d4/d1a/d41/de2/de7/dff/d107/f9e 475375 0 2024-02-19T19:06:29.738 INFO:tasks.workunit.client.0.smithi088.stdout:6/737: mkdir d5/de/d64/d8d/de7 0 2024-02-19T19:06:29.740 INFO:tasks.workunit.client.0.smithi088.stdout:6/738: dread d5/d7/d32/f8e [0,4194304] 0 2024-02-19T19:06:29.741 INFO:tasks.workunit.client.0.smithi088.stdout:8/611: symlink d9/df/lb7 0 2024-02-19T19:06:29.743 INFO:tasks.workunit.client.0.smithi088.stdout:2/544: mkdir d3/d1b/d43/d61/da6 0 2024-02-19T19:06:29.743 INFO:tasks.workunit.client.0.smithi088.stdout:8/612: rename d9/df/d3d/d50 to d9/df/d3d/d50/d71/da1/db8 22 2024-02-19T19:06:29.744 INFO:tasks.workunit.client.0.smithi088.stdout:5/828: fdatasync d4/d1a/fe9 0 2024-02-19T19:06:29.747 INFO:tasks.workunit.client.0.smithi088.stdout:0/667: fsync d13/d62/d8a/d95/d5a/d25/d2a/d34/f35 0 2024-02-19T19:06:29.748 INFO:tasks.workunit.client.0.smithi088.stdout:2/545: symlink d3/d1b/d43/la7 0 2024-02-19T19:06:29.748 INFO:tasks.workunit.client.0.smithi088.stdout:5/829: symlink d4/d1a/d41/de2/de7/l124 0 2024-02-19T19:06:29.749 INFO:tasks.workunit.client.0.smithi088.stdout:5/830: chown d4/d1a/d41/c43 16 1 2024-02-19T19:06:29.751 INFO:tasks.workunit.client.0.smithi088.stdout:6/739: mknod d5/d8/d3a/d71/daa/dc2/ce8 0 2024-02-19T19:06:29.752 INFO:tasks.workunit.client.0.smithi088.stdout:2/546: creat d3/d1b/d43/d8f/fa8 x:0 0 0 2024-02-19T19:06:29.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:29 smithi132 ceph-mon[70676]: pgmap v48: 65 pgs: 65 active+clean; 1.4 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 43 MiB/s rd, 135 MiB/s wr, 321 op/s 2024-02-19T19:06:29.756 INFO:tasks.workunit.client.0.smithi088.stdout:5/831: dread d4/d61/d65/d9a/d9f/d57/d84/f119 [0,4194304] 0 2024-02-19T19:06:29.758 INFO:tasks.workunit.client.0.smithi088.stdout:2/547: mknod d3/d1b/d43/d61/ca9 0 2024-02-19T19:06:29.760 INFO:tasks.workunit.client.0.smithi088.stdout:2/548: write d3/d1b/d43/d61/d86/f8b [943680,43522] 0 2024-02-19T19:06:29.762 INFO:tasks.workunit.client.0.smithi088.stdout:0/668: getdents d13/d62/d8a 0 2024-02-19T19:06:29.762 INFO:tasks.workunit.client.0.smithi088.stdout:6/740: dwrite d5/d7/d32/f53 [0,4194304] 0 2024-02-19T19:06:29.762 INFO:tasks.workunit.client.0.smithi088.stdout:9/968: truncate d1/f20 1542419 0 2024-02-19T19:06:29.763 INFO:tasks.workunit.client.0.smithi088.stdout:2/549: chown d3/d1f/d5d/c69 9760 1 2024-02-19T19:06:29.764 INFO:tasks.workunit.client.0.smithi088.stdout:6/741: write d5/ff [5235220,52930] 0 2024-02-19T19:06:29.764 INFO:tasks.workunit.client.0.smithi088.stdout:5/832: symlink d4/d1a/d41/dc1/dc2/l125 0 2024-02-19T19:06:29.766 INFO:tasks.workunit.client.0.smithi088.stdout:2/550: stat d3/d1b/d43/d74/c9c 0 2024-02-19T19:06:29.767 INFO:tasks.workunit.client.0.smithi088.stdout:6/742: chown d5/d8/d3a/l65 3109 1 2024-02-19T19:06:29.769 INFO:tasks.workunit.client.0.smithi088.stdout:9/969: symlink d1/d8/d3e/d93/d5a/d94/l13c 0 2024-02-19T19:06:29.770 INFO:tasks.workunit.client.0.smithi088.stdout:6/743: mknod d5/d45/ce9 0 2024-02-19T19:06:29.776 INFO:tasks.workunit.client.0.smithi088.stdout:2/551: dwrite d3/d1b/d43/d8f/fa8 [0,4194304] 0 2024-02-19T19:06:29.777 INFO:tasks.workunit.client.0.smithi088.stdout:6/744: mknod d5/d8/cea 0 2024-02-19T19:06:29.779 INFO:tasks.workunit.client.0.smithi088.stdout:9/970: dread d1/d8/d8d/d97/f53 [0,4194304] 0 2024-02-19T19:06:29.780 INFO:tasks.workunit.client.0.smithi088.stdout:2/552: creat d3/d1b/d43/d61/d86/faa x:0 0 0 2024-02-19T19:06:29.780 INFO:tasks.workunit.client.0.smithi088.stdout:6/745: fsync d5/d8/d3a/d71/daa/dc2/dbc/fa4 0 2024-02-19T19:06:29.784 INFO:tasks.workunit.client.0.smithi088.stdout:2/553: write d3/d1b/f67 [8651987,938] 0 2024-02-19T19:06:29.784 INFO:tasks.workunit.client.0.smithi088.stdout:9/971: chown d1/d8/d3e/d93/l32 0 1 2024-02-19T19:06:29.788 INFO:tasks.workunit.client.0.smithi088.stdout:9/972: stat d1/f10 0 2024-02-19T19:06:29.789 INFO:tasks.workunit.client.0.smithi088.stdout:2/554: write d3/d1b/d43/f52 [1540435,81501] 0 2024-02-19T19:06:29.789 INFO:tasks.workunit.client.0.smithi088.stdout:6/746: truncate d5/d19/d26/fe6 997198 0 2024-02-19T19:06:29.789 INFO:tasks.workunit.client.0.smithi088.stdout:5/833: dwrite d4/f97 [0,4194304] 0 2024-02-19T19:06:29.789 INFO:tasks.workunit.client.0.smithi088.stdout:0/669: dread d13/d62/d8a/d95/d5a/d25/d2a/d34/f77 [4194304,4194304] 0 2024-02-19T19:06:29.795 INFO:tasks.workunit.client.0.smithi088.stdout:5/834: creat d4/d61/d65/d9a/d9f/d57/d84/f126 x:0 0 0 2024-02-19T19:06:29.796 INFO:tasks.workunit.client.0.smithi088.stdout:2/555: truncate d3/d1f/d4f/f17 1120193 0 2024-02-19T19:06:29.798 INFO:tasks.workunit.client.0.smithi088.stdout:6/747: fsync d5/d8/d3a/d71/daa/dc2/f5d 0 2024-02-19T19:06:29.799 INFO:tasks.workunit.client.0.smithi088.stdout:6/748: stat d5/d8/d3a/c6b 0 2024-02-19T19:06:29.800 INFO:tasks.workunit.client.0.smithi088.stdout:6/749: stat d5/d8/d3a/d71/daa/dc2/c85 0 2024-02-19T19:06:29.801 INFO:tasks.workunit.client.0.smithi088.stdout:6/750: chown d5/de/d64/lb4 1053383464 1 2024-02-19T19:06:29.804 INFO:tasks.workunit.client.0.smithi088.stdout:2/556: mknod d3/cab 0 2024-02-19T19:06:29.805 INFO:tasks.workunit.client.0.smithi088.stdout:5/835: truncate d4/d1a/d41/f95 2344233 0 2024-02-19T19:06:29.807 INFO:tasks.workunit.client.0.smithi088.stdout:0/670: dwrite d13/d62/d8a/d95/d5a/db6/fd7 [4194304,4194304] 0 2024-02-19T19:06:29.808 INFO:tasks.workunit.client.0.smithi088.stdout:0/671: chown d13/d62/d8a/d95/d96 112276550 1 2024-02-19T19:06:29.810 INFO:tasks.workunit.client.0.smithi088.stdout:5/836: mknod d4/d61/d65/d9a/c127 0 2024-02-19T19:06:29.812 INFO:tasks.workunit.client.0.smithi088.stdout:2/557: write d3/d1f/d4f/f17 [836431,106484] 0 2024-02-19T19:06:29.819 INFO:tasks.workunit.client.0.smithi088.stdout:0/672: dwrite d13/d62/d8a/d95/d5b/d76/fbd [0,4194304] 0 2024-02-19T19:06:29.821 INFO:tasks.workunit.client.0.smithi088.stdout:6/751: dread d5/de/d60/f87 [0,4194304] 0 2024-02-19T19:06:29.821 INFO:tasks.workunit.client.0.smithi088.stdout:2/558: rename d3/d1f/d4f/d59/c6f to d3/d14/cac 0 2024-02-19T19:06:29.822 INFO:tasks.workunit.client.0.smithi088.stdout:6/752: chown d5/d7/d32/d56/fdf 3672074 1 2024-02-19T19:06:29.825 INFO:tasks.workunit.client.0.smithi088.stdout:2/559: stat d3/d1f/f4a 0 2024-02-19T19:06:29.827 INFO:tasks.workunit.client.0.smithi088.stdout:0/673: creat d13/d62/d8a/d9c/da8/fea x:0 0 0 2024-02-19T19:06:29.832 INFO:tasks.workunit.client.0.smithi088.stdout:0/674: creat d13/d62/d8a/de0/feb x:0 0 0 2024-02-19T19:06:29.832 INFO:tasks.workunit.client.0.smithi088.stdout:6/753: dwrite d5/d8/d3a/da8/fcf [0,4194304] 0 2024-02-19T19:06:29.835 INFO:tasks.workunit.client.0.smithi088.stdout:5/837: dwrite d4/d1a/d41/de2/de7/dff/d107/f9e [0,4194304] 0 2024-02-19T19:06:29.836 INFO:tasks.workunit.client.0.smithi088.stdout:2/560: truncate d3/fb 3691476 0 2024-02-19T19:06:29.840 INFO:tasks.workunit.client.0.smithi088.stdout:6/754: creat d5/d8/feb x:0 0 0 2024-02-19T19:06:29.845 INFO:tasks.workunit.client.0.smithi088.stdout:5/838: dwrite d4/d61/d65/d9a/d9f/fcd [0,4194304] 0 2024-02-19T19:06:29.845 INFO:tasks.workunit.client.0.smithi088.stdout:6/755: creat d5/de/d64/d8d/fec x:0 0 0 2024-02-19T19:06:29.848 INFO:tasks.workunit.client.0.smithi088.stdout:5/839: write d4/d1a/d41/dc1/dc2/fe3 [401646,15975] 0 2024-02-19T19:06:29.848 INFO:tasks.workunit.client.0.smithi088.stdout:0/675: dread d13/d62/d8a/d95/d5a/d25/d2a/d34/f35 [0,4194304] 0 2024-02-19T19:06:29.849 INFO:tasks.workunit.client.0.smithi088.stdout:6/756: write d5/de/d64/d8d/d9e/f9f [437673,70400] 0 2024-02-19T19:06:29.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:29 smithi088 ceph-mon[88643]: pgmap v48: 65 pgs: 65 active+clean; 1.4 GiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 43 MiB/s rd, 135 MiB/s wr, 321 op/s 2024-02-19T19:06:29.851 INFO:tasks.workunit.client.0.smithi088.stdout:5/840: fsync d4/d7/fbe 0 2024-02-19T19:06:29.851 INFO:tasks.workunit.client.0.smithi088.stdout:6/757: write d5/ff [8528441,130227] 0 2024-02-19T19:06:29.852 INFO:tasks.workunit.client.0.smithi088.stdout:6/758: stat d5/de/d64/f66 0 2024-02-19T19:06:29.855 INFO:tasks.workunit.client.0.smithi088.stdout:5/841: rmdir d4/d61/d65/d9a 39 2024-02-19T19:06:29.856 INFO:tasks.workunit.client.0.smithi088.stdout:5/842: stat d4/d1a/d41/de2/f112 0 2024-02-19T19:06:29.857 INFO:tasks.workunit.client.0.smithi088.stdout:6/759: chown d5/de/d60/f76 405309723 1 2024-02-19T19:06:29.857 INFO:tasks.workunit.client.0.smithi088.stdout:5/843: stat d4/d61/cad 0 2024-02-19T19:06:29.860 INFO:tasks.workunit.client.0.smithi088.stdout:0/676: write fe [397447,116802] 0 2024-02-19T19:06:29.862 INFO:tasks.workunit.client.0.smithi088.stdout:6/760: getdents d5/de/d97 0 2024-02-19T19:06:29.862 INFO:tasks.workunit.client.0.smithi088.stdout:0/677: chown d13/d62/d8a/d95/d5b/d76/fb8 2811896 1 2024-02-19T19:06:29.864 INFO:tasks.workunit.client.0.smithi088.stdout:5/844: rename d4/d61/d65/d9a/d9f/da0/cb0 to d4/d1a/d41/c128 0 2024-02-19T19:06:29.865 INFO:tasks.workunit.client.0.smithi088.stdout:6/761: symlink d5/de/d97/led 0 2024-02-19T19:06:29.867 INFO:tasks.workunit.client.0.smithi088.stdout:5/845: mknod d4/d1a/c129 0 2024-02-19T19:06:29.870 INFO:tasks.workunit.client.0.smithi088.stdout:6/762: creat d5/d7/dc9/dca/fee x:0 0 0 2024-02-19T19:06:29.870 INFO:tasks.workunit.client.0.smithi088.stdout:2/561: dread d3/d1b/d43/f58 [0,4194304] 0 2024-02-19T19:06:29.871 INFO:tasks.workunit.client.0.smithi088.stdout:5/846: symlink d4/d61/d65/d9a/d9f/l12a 0 2024-02-19T19:06:29.873 INFO:tasks.workunit.client.0.smithi088.stdout:6/763: write d5/d7/d32/d56/dbf/fcd [486711,129818] 0 2024-02-19T19:06:29.874 INFO:tasks.workunit.client.0.smithi088.stdout:0/678: write d13/f8c [734859,44959] 0 2024-02-19T19:06:29.874 INFO:tasks.workunit.client.0.smithi088.stdout:6/764: chown d5/d8/d3a/da8 30471 1 2024-02-19T19:06:29.875 INFO:tasks.workunit.client.0.smithi088.stdout:5/847: creat d4/d61/d65/d9a/d9f/d57/d84/f12b x:0 0 0 2024-02-19T19:06:29.878 INFO:tasks.workunit.client.0.smithi088.stdout:5/848: read d4/d61/d65/d9a/d9f/da0/fae [911272,91101] 0 2024-02-19T19:06:29.879 INFO:tasks.workunit.client.0.smithi088.stdout:2/562: getdents d3/d1b/d43/d61/d64 0 2024-02-19T19:06:29.883 INFO:tasks.workunit.client.0.smithi088.stdout:0/679: dwrite d13/d62/d8a/d95/d96/fc0 [0,4194304] 0 2024-02-19T19:06:29.884 INFO:tasks.workunit.client.0.smithi088.stdout:6/765: creat d5/d8/d3a/d71/daa/dc2/fef x:0 0 0 2024-02-19T19:06:29.886 INFO:tasks.workunit.client.0.smithi088.stdout:6/766: write d5/d8/d3a/d71/f74 [1000196,80732] 0 2024-02-19T19:06:29.887 INFO:tasks.workunit.client.0.smithi088.stdout:5/849: symlink d4/d61/l12c 0 2024-02-19T19:06:29.888 INFO:tasks.workunit.client.0.smithi088.stdout:5/850: stat d4/d61/d65/d9a/d9f/l12a 0 2024-02-19T19:06:29.889 INFO:tasks.workunit.client.0.smithi088.stdout:5/851: stat d4/d7/f63 0 2024-02-19T19:06:29.890 INFO:tasks.workunit.client.0.smithi088.stdout:0/680: mknod d13/cec 0 2024-02-19T19:06:29.892 INFO:tasks.workunit.client.0.smithi088.stdout:5/852: symlink d4/d75/l12d 0 2024-02-19T19:06:29.894 INFO:tasks.workunit.client.0.smithi088.stdout:2/563: link d3/d1b/d43/d61/d64/c96 d3/cad 0 2024-02-19T19:06:29.897 INFO:tasks.workunit.client.0.smithi088.stdout:2/564: read d3/d1f/d5d/f72 [1080111,121696] 0 2024-02-19T19:06:29.897 INFO:tasks.workunit.client.0.smithi088.stdout:6/767: write d5/f8b [720427,12873] 0 2024-02-19T19:06:29.901 INFO:tasks.workunit.client.0.smithi088.stdout:2/565: mknod d3/d1b/d43/d61/d64/cae 0 2024-02-19T19:06:29.903 INFO:tasks.workunit.client.0.smithi088.stdout:6/768: dread d5/d8/d2c/f4c [0,4194304] 0 2024-02-19T19:06:29.903 INFO:tasks.workunit.client.0.smithi088.stdout:5/853: link d4/d61/lce d4/d1a/d41/dc1/l12e 0 2024-02-19T19:06:29.904 INFO:tasks.workunit.client.0.smithi088.stdout:5/854: chown d4/d1a/l11f 34940397 1 2024-02-19T19:06:29.904 INFO:tasks.workunit.client.0.smithi088.stdout:6/769: fdatasync d5/d19/d26/fe6 0 2024-02-19T19:06:29.905 INFO:tasks.workunit.client.0.smithi088.stdout:2/566: symlink d3/d1b/d43/d8f/laf 0 2024-02-19T19:06:29.905 INFO:tasks.workunit.client.0.smithi088.stdout:0/681: truncate d13/f4a 2261493 0 2024-02-19T19:06:29.906 INFO:tasks.workunit.client.0.smithi088.stdout:0/682: write d13/d62/d8a/d95/d5a/d25/d2a/d3c/f5d [80384,87184] 0 2024-02-19T19:06:29.908 INFO:tasks.workunit.client.0.smithi088.stdout:6/770: creat d5/de/d64/d8d/d95/ff0 x:0 0 0 2024-02-19T19:06:29.910 INFO:tasks.workunit.client.0.smithi088.stdout:0/683: mkdir d13/d62/d8a/d95/d96/ded 0 2024-02-19T19:06:29.911 INFO:tasks.workunit.client.0.smithi088.stdout:5/855: mknod d4/c12f 0 2024-02-19T19:06:29.911 INFO:tasks.workunit.client.0.smithi088.stdout:6/771: rmdir d5/d8/d3a 39 2024-02-19T19:06:29.912 INFO:tasks.workunit.client.0.smithi088.stdout:2/567: getdents d3/d71 0 2024-02-19T19:06:29.912 INFO:tasks.workunit.client.0.smithi088.stdout:6/772: write d5/d45/d9d/fa2 [318930,65824] 0 2024-02-19T19:06:29.913 INFO:tasks.workunit.client.0.smithi088.stdout:0/684: creat d13/d62/d8a/d95/d5a/d25/d2a/d3c/fee x:0 0 0 2024-02-19T19:06:29.914 INFO:tasks.workunit.client.0.smithi088.stdout:5/856: write d4/d61/d65/d9a/fac [4551198,100799] 0 2024-02-19T19:06:29.917 INFO:tasks.workunit.client.0.smithi088.stdout:6/773: creat d5/d19/d26/d72/ff1 x:0 0 0 2024-02-19T19:06:29.918 INFO:tasks.workunit.client.0.smithi088.stdout:6/774: truncate d5/d8/feb 583207 0 2024-02-19T19:06:29.919 INFO:tasks.workunit.client.0.smithi088.stdout:2/568: creat d3/fb0 x:0 0 0 2024-02-19T19:06:29.920 INFO:tasks.workunit.client.0.smithi088.stdout:0/685: truncate d13/f6a 454764 0 2024-02-19T19:06:29.924 INFO:tasks.workunit.client.0.smithi088.stdout:0/686: fdatasync f4 0 2024-02-19T19:06:29.926 INFO:tasks.workunit.client.0.smithi088.stdout:0/687: fsync d13/d62/d8a/d95/d5a/d25/d2a/d3c/fc5 0 2024-02-19T19:06:29.930 INFO:tasks.workunit.client.0.smithi088.stdout:5/857: link d4/d61/l12c d4/l130 0 2024-02-19T19:06:29.933 INFO:tasks.workunit.client.0.smithi088.stdout:2/569: write d3/d1f/d5d/f62 [970543,79895] 0 2024-02-19T19:06:29.934 INFO:tasks.workunit.client.0.smithi088.stdout:0/688: truncate d13/f1b 949917 0 2024-02-19T19:06:29.936 INFO:tasks.workunit.client.0.smithi088.stdout:2/570: mkdir d3/d1b/d43/d61/d7e/db1 0 2024-02-19T19:06:29.940 INFO:tasks.workunit.client.0.smithi088.stdout:5/858: creat d4/d61/d65/d9a/d9f/da0/f131 x:0 0 0 2024-02-19T19:06:29.941 INFO:tasks.workunit.client.0.smithi088.stdout:2/571: rmdir d3/d1b/d43/d74 39 2024-02-19T19:06:29.941 INFO:tasks.workunit.client.0.smithi088.stdout:6/775: dwrite d5/d8/f3f [0,4194304] 0 2024-02-19T19:06:29.943 INFO:tasks.workunit.client.0.smithi088.stdout:2/572: write d3/d14/f56 [8339780,17024] 0 2024-02-19T19:06:29.951 INFO:tasks.workunit.client.0.smithi088.stdout:5/859: fdatasync d4/d1a/f55 0 2024-02-19T19:06:29.951 INFO:tasks.workunit.client.0.smithi088.stdout:2/573: mkdir d3/d1b/d43/d61/da6/db2 0 2024-02-19T19:06:29.952 INFO:tasks.workunit.client.0.smithi088.stdout:0/689: dwrite d13/d62/d8a/d87/fe6 [0,4194304] 0 2024-02-19T19:06:29.953 INFO:tasks.workunit.client.0.smithi088.stdout:5/860: symlink d4/d75/de8/l132 0 2024-02-19T19:06:29.953 INFO:tasks.workunit.client.0.smithi088.stdout:2/574: truncate d3/d1f/d5d/f72 4376733 0 2024-02-19T19:06:29.955 INFO:tasks.workunit.client.0.smithi088.stdout:5/861: dread - d4/d1a/d41/de2/de7/dff/d107/f122 zero size 2024-02-19T19:06:29.955 INFO:tasks.workunit.client.0.smithi088.stdout:6/776: rename d5/de/c43 to d5/d8/d3a/d71/daa/dc2/dbc/db6/cf2 0 2024-02-19T19:06:29.960 INFO:tasks.workunit.client.0.smithi088.stdout:2/575: getdents d3/d1b/d43/d61/da6/db2 0 2024-02-19T19:06:29.960 INFO:tasks.workunit.client.0.smithi088.stdout:6/777: dread d5/de/d64/d8d/d9e/f9f [0,4194304] 0 2024-02-19T19:06:29.961 INFO:tasks.workunit.client.0.smithi088.stdout:0/690: creat d13/d62/d8a/d95/d5a/d25/fef x:0 0 0 2024-02-19T19:06:29.962 INFO:tasks.workunit.client.0.smithi088.stdout:2/576: creat d3/d1b/fb3 x:0 0 0 2024-02-19T19:06:29.967 INFO:tasks.workunit.client.0.smithi088.stdout:2/577: creat d3/d1f/d5d/fb4 x:0 0 0 2024-02-19T19:06:29.967 INFO:tasks.workunit.client.0.smithi088.stdout:2/578: chown d3/d14/c38 619895 1 2024-02-19T19:06:29.967 INFO:tasks.workunit.client.0.smithi088.stdout:5/862: link d4/d75/f103 d4/d1a/d41/dc1/dc2/f133 0 2024-02-19T19:06:29.969 INFO:tasks.workunit.client.0.smithi088.stdout:0/691: symlink d13/d62/d8a/d9c/da8/dab/db4/dbf/lf0 0 2024-02-19T19:06:29.970 INFO:tasks.workunit.client.0.smithi088.stdout:0/692: stat d13/d62/d8a/d95/d5a/d25/d2a/d34/fa3 0 2024-02-19T19:06:29.972 INFO:tasks.workunit.client.0.smithi088.stdout:5/863: chown d4/d1a/d41/f95 18580 1 2024-02-19T19:06:29.973 INFO:tasks.workunit.client.0.smithi088.stdout:2/579: mknod d3/d1b/d43/d74/cb5 0 2024-02-19T19:06:29.974 INFO:tasks.workunit.client.0.smithi088.stdout:2/580: fdatasync d3/d1b/d43/d61/f9e 0 2024-02-19T19:06:29.974 INFO:tasks.workunit.client.0.smithi088.stdout:0/693: dread d13/d62/d8a/d95/d5a/d25/d2a/d34/f75 [0,4194304] 0 2024-02-19T19:06:29.975 INFO:tasks.workunit.client.0.smithi088.stdout:2/581: chown d3/d1b/f67 815818 1 2024-02-19T19:06:29.976 INFO:tasks.workunit.client.0.smithi088.stdout:6/778: truncate d5/de/d60/fac 779140 0 2024-02-19T19:06:29.977 INFO:tasks.workunit.client.0.smithi088.stdout:0/694: write d13/d62/d8a/d95/d5a/d25/d2a/d2f/f4c [17749,53534] 0 2024-02-19T19:06:29.978 INFO:tasks.workunit.client.0.smithi088.stdout:2/582: mkdir d3/d1b/d43/d61/d7e/db6 0 2024-02-19T19:06:29.980 INFO:tasks.workunit.client.0.smithi088.stdout:5/864: link d4/d1a/d41/dcb/fd3 d4/d61/d65/d9a/d9f/d57/f134 0 2024-02-19T19:06:29.981 INFO:tasks.workunit.client.0.smithi088.stdout:6/779: dread d5/d8/d3a/d71/f74 [0,4194304] 0 2024-02-19T19:06:29.982 INFO:tasks.workunit.client.0.smithi088.stdout:0/695: symlink d13/d62/d8a/d95/d5a/db6/lf1 0 2024-02-19T19:06:29.983 INFO:tasks.workunit.client.0.smithi088.stdout:5/865: write d4/d61/fa8 [673819,97226] 0 2024-02-19T19:06:29.983 INFO:tasks.workunit.client.0.smithi088.stdout:0/696: write d13/d62/d8a/d9c/da8/fea [123902,45708] 0 2024-02-19T19:06:29.983 INFO:tasks.workunit.client.0.smithi088.stdout:6/780: mkdir d5/d19/d70/df3 0 2024-02-19T19:06:29.985 INFO:tasks.workunit.client.0.smithi088.stdout:2/583: creat d3/d71/fb7 x:0 0 0 2024-02-19T19:06:29.985 INFO:tasks.workunit.client.0.smithi088.stdout:6/781: stat d5/d8/l1b 0 2024-02-19T19:06:29.988 INFO:tasks.workunit.client.0.smithi088.stdout:2/584: mknod d3/d71/cb8 0 2024-02-19T19:06:29.990 INFO:tasks.workunit.client.0.smithi088.stdout:6/782: unlink d5/d8/d3a/d71/fa3 0 2024-02-19T19:06:29.993 INFO:tasks.workunit.client.0.smithi088.stdout:2/585: rmdir d3/d1b/d43/d61/d64 39 2024-02-19T19:06:29.995 INFO:tasks.workunit.client.0.smithi088.stdout:0/697: dwrite d13/d62/d8a/d95/d5a/d25/fef [0,4194304] 0 2024-02-19T19:06:29.997 INFO:tasks.workunit.client.0.smithi088.stdout:5/866: dwrite d4/d61/d65/d9a/d9f/da0/f131 [0,4194304] 0 2024-02-19T19:06:29.997 INFO:tasks.workunit.client.0.smithi088.stdout:6/783: dread d5/f47 [0,4194304] 0 2024-02-19T19:06:30.000 INFO:tasks.workunit.client.0.smithi088.stdout:2/586: symlink d3/lb9 0 2024-02-19T19:06:30.000 INFO:tasks.workunit.client.0.smithi088.stdout:0/698: chown d13/d62/d8a/d9c/fac 85400 1 2024-02-19T19:06:30.002 INFO:tasks.workunit.client.0.smithi088.stdout:2/587: write d3/d1b/d43/d61/d86/faa [748236,124080] 0 2024-02-19T19:06:30.004 INFO:tasks.workunit.client.0.smithi088.stdout:6/784: creat d5/de/d97/ff4 x:0 0 0 2024-02-19T19:06:30.007 INFO:tasks.workunit.client.0.smithi088.stdout:5/867: fsync d4/d7/f113 0 2024-02-19T19:06:30.009 INFO:tasks.workunit.client.0.smithi088.stdout:2/588: rmdir d3/d1b/d43/d61/d86 39 2024-02-19T19:06:30.009 INFO:tasks.workunit.client.0.smithi088.stdout:5/868: dread d4/d1a/d41/de2/f112 [0,4194304] 0 2024-02-19T19:06:30.009 INFO:tasks.workunit.client.0.smithi088.stdout:2/589: fdatasync d3/d1f/d5d/f62 0 2024-02-19T19:06:30.010 INFO:tasks.workunit.client.0.smithi088.stdout:5/869: stat d4/d7/f92 0 2024-02-19T19:06:30.015 INFO:tasks.workunit.client.0.smithi088.stdout:0/699: symlink d13/d62/d8a/d95/d96/ded/lf2 0 2024-02-19T19:06:30.019 INFO:tasks.workunit.client.0.smithi088.stdout:0/700: mkdir d13/d62/d8a/d95/d5a/df3 0 2024-02-19T19:06:30.019 INFO:tasks.workunit.client.0.smithi088.stdout:2/590: dread d3/d1b/f42 [0,4194304] 0 2024-02-19T19:06:30.020 INFO:tasks.workunit.client.0.smithi088.stdout:5/870: creat d4/d61/d65/d9a/d9f/d57/f135 x:0 0 0 2024-02-19T19:06:30.023 INFO:tasks.workunit.client.0.smithi088.stdout:0/701: fdatasync d13/f1c 0 2024-02-19T19:06:30.023 INFO:tasks.workunit.client.0.smithi088.stdout:6/785: write d5/d8/d3a/f63 [1227022,27946] 0 2024-02-19T19:06:30.028 INFO:tasks.workunit.client.0.smithi088.stdout:2/591: getdents d3/d1f/d5d 0 2024-02-19T19:06:30.029 INFO:tasks.workunit.client.0.smithi088.stdout:0/702: mknod d13/d62/d8a/d95/d5b/d76/dde/cf4 0 2024-02-19T19:06:30.034 INFO:tasks.workunit.client.0.smithi088.stdout:6/786: getdents d5 0 2024-02-19T19:06:30.035 INFO:tasks.workunit.client.0.smithi088.stdout:5/871: truncate d4/d1a/d41/de2/de7/dff/d107/f9e 3583166 0 2024-02-19T19:06:30.036 INFO:tasks.workunit.client.0.smithi088.stdout:0/703: dwrite d13/d62/fa6 [0,4194304] 0 2024-02-19T19:06:30.037 INFO:tasks.workunit.client.0.smithi088.stdout:2/592: write d3/f23 [9773,78638] 0 2024-02-19T19:06:30.038 INFO:tasks.workunit.client.0.smithi088.stdout:0/704: readlink d13/d62/d8a/d9c/lb2 0 2024-02-19T19:06:30.038 INFO:tasks.workunit.client.0.smithi088.stdout:6/787: mknod d5/de/d64/d82/cf5 0 2024-02-19T19:06:30.040 INFO:tasks.workunit.client.0.smithi088.stdout:2/593: unlink d3/lb9 0 2024-02-19T19:06:30.044 INFO:tasks.workunit.client.0.smithi088.stdout:0/705: read d13/d62/f8e [140286,113217] 0 2024-02-19T19:06:30.044 INFO:tasks.workunit.client.0.smithi088.stdout:5/872: dwrite d4/d75/fbf [0,4194304] 0 2024-02-19T19:06:30.045 INFO:tasks.workunit.client.0.smithi088.stdout:6/788: unlink d5/f8b 0 2024-02-19T19:06:30.046 INFO:tasks.workunit.client.0.smithi088.stdout:2/594: readlink d3/d1b/d43/d8f/d94/l97 0 2024-02-19T19:06:30.048 INFO:tasks.workunit.client.0.smithi088.stdout:5/873: mkdir d4/d75/de8/d136 0 2024-02-19T19:06:30.051 INFO:tasks.workunit.client.0.smithi088.stdout:0/706: dread d13/f15 [0,4194304] 0 2024-02-19T19:06:30.056 INFO:tasks.workunit.client.0.smithi088.stdout:1/708: sync 2024-02-19T19:06:30.056 INFO:tasks.workunit.client.0.smithi088.stdout:2/595: rename d3/d14/la0 to d3/d14/d93/lba 0 2024-02-19T19:06:30.058 INFO:tasks.workunit.client.0.smithi088.stdout:5/874: mknod d4/d1a/d41/de2/c137 0 2024-02-19T19:06:30.058 INFO:tasks.workunit.client.0.smithi088.stdout:6/789: rmdir d5/d45 39 2024-02-19T19:06:30.060 INFO:tasks.workunit.client.0.smithi088.stdout:0/707: creat d13/d62/d8a/d9c/da8/dab/db4/dbf/ff5 x:0 0 0 2024-02-19T19:06:30.061 INFO:tasks.workunit.client.0.smithi088.stdout:1/709: unlink d6/d21/d64/d94/d6e/la4 0 2024-02-19T19:06:30.062 INFO:tasks.workunit.client.0.smithi088.stdout:1/710: chown d6/d21/d64/fb0 0 1 2024-02-19T19:06:30.062 INFO:tasks.workunit.client.0.smithi088.stdout:5/875: chown d4/f12 8732 1 2024-02-19T19:06:30.064 INFO:tasks.workunit.client.0.smithi088.stdout:2/596: creat d3/d1b/d43/d61/d7e/db1/fbb x:0 0 0 2024-02-19T19:06:30.064 INFO:tasks.workunit.client.0.smithi088.stdout:0/708: truncate d13/d62/d8a/d95/d5a/f80 675585 0 2024-02-19T19:06:30.068 INFO:tasks.workunit.client.0.smithi088.stdout:6/790: link d5/de/d64/d8d/d9e/f9f d5/de/d97/ff6 0 2024-02-19T19:06:30.075 INFO:tasks.workunit.client.0.smithi088.stdout:6/791: rename d5/d8/c58 to d5/d8/d3a/d71/daa/cf7 0 2024-02-19T19:06:30.078 INFO:tasks.workunit.client.0.smithi088.stdout:2/597: getdents d3 0 2024-02-19T19:06:30.079 INFO:tasks.workunit.client.0.smithi088.stdout:1/711: dread d6/d21/d64/d94/d71/f35 [0,4194304] 0 2024-02-19T19:06:30.081 INFO:tasks.workunit.client.0.smithi088.stdout:6/792: creat d5/d8/d3a/da8/dbd/ff8 x:0 0 0 2024-02-19T19:06:30.082 INFO:tasks.workunit.client.0.smithi088.stdout:5/876: write d4/d61/d65/d9a/d9f/f4e [487152,91333] 0 2024-02-19T19:06:30.086 INFO:tasks.workunit.client.0.smithi088.stdout:6/793: write d5/de/d64/d82/d8c/f6e [9373628,27294] 0 2024-02-19T19:06:30.087 INFO:tasks.workunit.client.0.smithi088.stdout:5/877: mknod d4/d7/c138 0 2024-02-19T19:06:30.088 INFO:tasks.workunit.client.0.smithi088.stdout:2/598: dwrite d3/d1b/d43/d61/d7e/db1/fbb [0,4194304] 0 2024-02-19T19:06:30.089 INFO:tasks.workunit.client.0.smithi088.stdout:6/794: stat d5/d8/d3a/d71/daa/dc2/dbc/lbb 0 2024-02-19T19:06:30.090 INFO:tasks.workunit.client.0.smithi088.stdout:1/712: write d6/d10/f4b [788106,38002] 0 2024-02-19T19:06:30.091 INFO:tasks.workunit.client.0.smithi088.stdout:5/878: creat d4/d75/deb/f139 x:0 0 0 2024-02-19T19:06:30.096 INFO:tasks.workunit.client.0.smithi088.stdout:2/599: creat d3/d1b/d43/d61/d7e/db6/fbc x:0 0 0 2024-02-19T19:06:30.100 INFO:tasks.workunit.client.0.smithi088.stdout:2/600: rename d3/d71/fb7 to d3/fbd 0 2024-02-19T19:06:30.101 INFO:tasks.workunit.client.0.smithi088.stdout:6/795: dwrite d5/d19/d26/f27 [4194304,4194304] 0 2024-02-19T19:06:30.102 INFO:tasks.workunit.client.0.smithi088.stdout:6/796: fdatasync d5/de/d97/ff4 0 2024-02-19T19:06:30.104 INFO:tasks.workunit.client.0.smithi088.stdout:2/601: stat d3/d1f/d5d/c70 0 2024-02-19T19:06:30.107 INFO:tasks.workunit.client.0.smithi088.stdout:2/602: chown d3/d14/c38 902 1 2024-02-19T19:06:30.108 INFO:tasks.workunit.client.0.smithi088.stdout:2/603: truncate d3/fbd 176059 0 2024-02-19T19:06:30.116 INFO:tasks.workunit.client.0.smithi088.stdout:2/604: link d3/fb0 d3/fbe 0 2024-02-19T19:06:30.122 INFO:tasks.workunit.client.0.smithi088.stdout:1/713: dwrite d6/d21/d6a/d7d/d9a/fa3 [4194304,4194304] 0 2024-02-19T19:06:30.122 INFO:tasks.workunit.client.0.smithi088.stdout:5/879: dwrite d4/d1a/f5b [0,4194304] 0 2024-02-19T19:06:30.122 INFO:tasks.workunit.client.0.smithi088.stdout:6/797: dwrite d5/de/f18 [0,4194304] 0 2024-02-19T19:06:30.125 INFO:tasks.workunit.client.0.smithi088.stdout:2/605: write d3/d1b/d43/d8f/f8e [332758,109918] 0 2024-02-19T19:06:30.125 INFO:tasks.workunit.client.0.smithi088.stdout:5/880: fdatasync d4/d1a/fd1 0 2024-02-19T19:06:30.131 INFO:tasks.workunit.client.0.smithi088.stdout:1/714: dwrite d6/d21/f6c [0,4194304] 0 2024-02-19T19:06:30.132 INFO:tasks.workunit.client.0.smithi088.stdout:5/881: truncate d4/d1a/fdd 763821 0 2024-02-19T19:06:30.137 INFO:tasks.workunit.client.0.smithi088.stdout:2/606: rename d3/d1b/d43/d61/d7e/f7f to d3/d1b/d43/d61/d7e/db1/fbf 0 2024-02-19T19:06:30.138 INFO:tasks.workunit.client.0.smithi088.stdout:2/607: chown d3/d1b/d43/d61/da6/db2 53120 1 2024-02-19T19:06:30.141 INFO:tasks.workunit.client.0.smithi088.stdout:5/882: rmdir d4/d75 39 2024-02-19T19:06:30.142 INFO:tasks.workunit.client.0.smithi088.stdout:1/715: dwrite d6/d10/fd5 [0,4194304] 0 2024-02-19T19:06:30.145 INFO:tasks.workunit.client.0.smithi088.stdout:2/608: creat d3/d1b/d43/d61/d64/fc0 x:0 0 0 2024-02-19T19:06:30.146 INFO:tasks.workunit.client.0.smithi088.stdout:5/883: symlink d4/d61/d65/d9a/d9f/d57/d84/l13a 0 2024-02-19T19:06:30.147 INFO:tasks.workunit.client.0.smithi088.stdout:1/716: mknod d6/d21/cea 0 2024-02-19T19:06:30.148 INFO:tasks.workunit.client.0.smithi088.stdout:5/884: chown d4/d1a/d41/dc1/f111 33142 1 2024-02-19T19:06:30.150 INFO:tasks.workunit.client.0.smithi088.stdout:1/717: chown d6/d21/d6a/d4a/cd0 187793 1 2024-02-19T19:06:30.155 INFO:tasks.workunit.client.0.smithi088.stdout:5/885: truncate d4/d1a/d41/de2/de7/fed 365410 0 2024-02-19T19:06:30.156 INFO:tasks.workunit.client.0.smithi088.stdout:2/609: dwrite d3/d1b/d43/d61/d64/f8a [0,4194304] 0 2024-02-19T19:06:30.156 INFO:tasks.workunit.client.0.smithi088.stdout:6/798: truncate d5/d19/f4e 290678 0 2024-02-19T19:06:30.158 INFO:tasks.workunit.client.0.smithi088.stdout:2/610: write d3/d1b/d43/d8f/fa8 [3082729,54794] 0 2024-02-19T19:06:30.184 INFO:tasks.workunit.client.0.smithi088.stdout:1/718: rename d6/d21/d64/d94/d71/f9b to d6/d21/d6a/d7d/da8/feb 0 2024-02-19T19:06:30.186 INFO:tasks.workunit.client.0.smithi088.stdout:2/611: readlink d3/d1b/d43/d74/l8d 0 2024-02-19T19:06:30.192 INFO:tasks.workunit.client.0.smithi088.stdout:5/886: dwrite d4/d61/d65/d9a/d9f/d57/d84/f89 [0,4194304] 0 2024-02-19T19:06:30.194 INFO:tasks.workunit.client.0.smithi088.stdout:2/612: mknod d3/d1b/d43/d61/d7e/db1/cc1 0 2024-02-19T19:06:30.194 INFO:tasks.workunit.client.0.smithi088.stdout:6/799: dwrite d5/de/d64/d82/d8c/f6e [12582912,4194304] 0 2024-02-19T19:06:30.199 INFO:tasks.workunit.client.0.smithi088.stdout:5/887: mkdir d4/d61/d65/d9a/d9f/d57/d84/dfb/d13b 0 2024-02-19T19:06:30.199 INFO:tasks.workunit.client.0.smithi088.stdout:5/888: stat d4/d61/f78 0 2024-02-19T19:06:30.217 INFO:tasks.workunit.client.0.smithi088.stdout:1/719: dread d6/d21/d64/d94/d6e/f7c [0,4194304] 0 2024-02-19T19:06:30.247 INFO:tasks.workunit.client.0.smithi088.stdout:6/800: creat d5/de/d60/db1/ff9 x:0 0 0 2024-02-19T19:06:30.248 INFO:tasks.workunit.client.0.smithi088.stdout:1/720: fsync d6/d21/f72 0 2024-02-19T19:06:30.252 INFO:tasks.workunit.client.0.smithi088.stdout:1/721: fdatasync d6/d21/d6a/dcb/fdd 0 2024-02-19T19:06:30.253 INFO:tasks.workunit.client.0.smithi088.stdout:6/801: creat d5/d7/d32/ffa x:0 0 0 2024-02-19T19:06:30.258 INFO:tasks.workunit.client.0.smithi088.stdout:5/889: dwrite d4/d1a/d41/f71 [0,4194304] 0 2024-02-19T19:06:30.260 INFO:tasks.workunit.client.0.smithi088.stdout:6/802: write d5/f1e [361505,119408] 0 2024-02-19T19:06:30.262 INFO:tasks.workunit.client.0.smithi088.stdout:5/890: creat d4/d1a/d41/de2/de7/dff/d114/f13c x:0 0 0 2024-02-19T19:06:30.263 INFO:tasks.workunit.client.0.smithi088.stdout:2/613: dwrite d3/fb [0,4194304] 0 2024-02-19T19:06:30.264 INFO:tasks.workunit.client.0.smithi088.stdout:5/891: chown d4/d61/d65/d9a/d9f/c2d 5678582 1 2024-02-19T19:06:30.265 INFO:tasks.workunit.client.0.smithi088.stdout:2/614: write d3/d1b/d43/d61/d64/f9d [6907,115651] 0 2024-02-19T19:06:30.269 INFO:tasks.workunit.client.0.smithi088.stdout:5/892: dread - d4/d1a/d41/de2/de7/f109 zero size 2024-02-19T19:06:30.275 INFO:tasks.workunit.client.0.smithi088.stdout:6/803: dwrite d5/d8/d3a/d71/daa/dc2/fb3 [0,4194304] 0 2024-02-19T19:06:30.278 INFO:tasks.workunit.client.0.smithi088.stdout:1/722: dread d6/d21/f36 [0,4194304] 0 2024-02-19T19:06:30.278 INFO:tasks.workunit.client.0.smithi088.stdout:5/893: creat d4/f13d x:0 0 0 2024-02-19T19:06:30.279 INFO:tasks.workunit.client.0.smithi088.stdout:2/615: dwrite d3/d14/f5a [0,4194304] 0 2024-02-19T19:06:30.280 INFO:tasks.workunit.client.0.smithi088.stdout:1/723: chown d6/c1f 54269410 1 2024-02-19T19:06:30.281 INFO:tasks.workunit.client.0.smithi088.stdout:2/616: readlink d3/d1f/d4f/l37 0 2024-02-19T19:06:30.285 INFO:tasks.workunit.client.0.smithi088.stdout:2/617: creat d3/d1b/d43/d8f/d94/fc2 x:0 0 0 2024-02-19T19:06:30.285 INFO:tasks.workunit.client.0.smithi088.stdout:5/894: write d4/d61/d65/d9a/d9f/d57/d84/f119 [993692,8054] 0 2024-02-19T19:06:30.286 INFO:tasks.workunit.client.0.smithi088.stdout:2/618: fsync d3/d1b/d43/f55 0 2024-02-19T19:06:30.288 INFO:tasks.workunit.client.0.smithi088.stdout:5/895: read d4/d61/f78 [840518,27173] 0 2024-02-19T19:06:30.288 INFO:tasks.workunit.client.0.smithi088.stdout:6/804: rename d5/d8/d3a/d71/daa/dc2/fef to d5/d19/ffb 0 2024-02-19T19:06:30.288 INFO:tasks.workunit.client.0.smithi088.stdout:6/805: stat d5/d19/d26 0 2024-02-19T19:06:30.290 INFO:tasks.workunit.client.0.smithi088.stdout:6/806: write d5/de/f22 [2173112,65704] 0 2024-02-19T19:06:30.290 INFO:tasks.workunit.client.0.smithi088.stdout:1/724: mknod d6/d21/d6a/cec 0 2024-02-19T19:06:30.291 INFO:tasks.workunit.client.0.smithi088.stdout:2/619: chown d3/d1f/c33 479 1 2024-02-19T19:06:30.293 INFO:tasks.workunit.client.0.smithi088.stdout:1/725: write d6/d10/d60/db8/fd1 [899612,54699] 0 2024-02-19T19:06:30.303 INFO:tasks.workunit.client.0.smithi088.stdout:2/620: dwrite d3/d1b/d43/d74/f87 [0,4194304] 0 2024-02-19T19:06:30.303 INFO:tasks.workunit.client.0.smithi088.stdout:5/896: dread d4/d61/d65/d9a/d9f/f4c [0,4194304] 0 2024-02-19T19:06:30.306 INFO:tasks.workunit.client.0.smithi088.stdout:2/621: mknod d3/d1f/d5d/cc3 0 2024-02-19T19:06:30.307 INFO:tasks.workunit.client.0.smithi088.stdout:6/807: write d5/d7/d32/f8e [724995,24892] 0 2024-02-19T19:06:30.308 INFO:tasks.workunit.client.0.smithi088.stdout:6/808: chown d5/d19/d70 4868 1 2024-02-19T19:06:30.309 INFO:tasks.workunit.client.0.smithi088.stdout:2/622: creat d3/d1f/d5d/fc4 x:0 0 0 2024-02-19T19:06:30.316 INFO:tasks.workunit.client.0.smithi088.stdout:1/726: dwrite d6/ff [0,4194304] 0 2024-02-19T19:06:30.321 INFO:tasks.workunit.client.0.smithi088.stdout:1/727: unlink d6/cce 0 2024-02-19T19:06:30.326 INFO:tasks.workunit.client.0.smithi088.stdout:6/809: dread d5/d19/f5f [0,4194304] 0 2024-02-19T19:06:30.327 INFO:tasks.workunit.client.0.smithi088.stdout:1/728: unlink d6/d21/d6a/cec 0 2024-02-19T19:06:30.327 INFO:tasks.workunit.client.0.smithi088.stdout:5/897: dwrite d4/d75/f80 [0,4194304] 0 2024-02-19T19:06:30.329 INFO:tasks.workunit.client.0.smithi088.stdout:2/623: dwrite d3/d1b/d43/f58 [0,4194304] 0 2024-02-19T19:06:30.334 INFO:tasks.workunit.client.0.smithi088.stdout:6/810: rename d5/d8/d3a/lde to d5/de/d64/d8d/d95/dba/lfc 0 2024-02-19T19:06:30.336 INFO:tasks.workunit.client.0.smithi088.stdout:6/811: stat d5/de/d64/d8d/d9e/ca6 0 2024-02-19T19:06:30.337 INFO:tasks.workunit.client.0.smithi088.stdout:1/729: link d6/d10/f3b d6/d21/d64/d94/dab/fed 0 2024-02-19T19:06:30.342 INFO:tasks.workunit.client.0.smithi088.stdout:6/812: mknod d5/d8/d3a/d71/daa/dc2/dbc/db6/cfd 0 2024-02-19T19:06:30.343 INFO:tasks.workunit.client.0.smithi088.stdout:1/730: creat d6/d21/d6a/d7d/da8/fee x:0 0 0 2024-02-19T19:06:30.346 INFO:tasks.workunit.client.0.smithi088.stdout:6/813: dread d5/d19/f4e [0,4194304] 0 2024-02-19T19:06:30.347 INFO:tasks.workunit.client.0.smithi088.stdout:1/731: mknod d6/d10/d60/d29/cef 0 2024-02-19T19:06:30.349 INFO:tasks.workunit.client.0.smithi088.stdout:6/814: mknod d5/d19/d26/cfe 0 2024-02-19T19:06:30.350 INFO:tasks.workunit.client.0.smithi088.stdout:5/898: truncate d4/d61/d65/d9a/fac 3889384 0 2024-02-19T19:06:30.351 INFO:tasks.workunit.client.0.smithi088.stdout:5/899: readlink d4/d1a/d41/dc1/dc2/l125 0 2024-02-19T19:06:30.352 INFO:tasks.workunit.client.0.smithi088.stdout:5/900: chown d4/d1a/d41/fea 2602 1 2024-02-19T19:06:30.353 INFO:tasks.workunit.client.0.smithi088.stdout:6/815: truncate d5/d19/d26/fa1 8437301 0 2024-02-19T19:06:30.355 INFO:tasks.workunit.client.0.smithi088.stdout:1/732: dread d6/d21/f72 [0,4194304] 0 2024-02-19T19:06:30.355 INFO:tasks.workunit.client.0.smithi088.stdout:2/624: dread d3/d1f/f32 [4194304,4194304] 0 2024-02-19T19:06:30.360 INFO:tasks.workunit.client.0.smithi088.stdout:1/733: truncate d6/d21/d64/d94/d71/f98 335031 0 2024-02-19T19:06:30.360 INFO:tasks.workunit.client.0.smithi088.stdout:1/734: chown d6/d10/d60/f67 49389 1 2024-02-19T19:06:30.361 INFO:tasks.workunit.client.0.smithi088.stdout:6/816: link d5/d8/d3a/d71/daa/dc2/f5d d5/d7/d32/d56/dbf/fff 0 2024-02-19T19:06:30.362 INFO:tasks.workunit.client.0.smithi088.stdout:6/817: fsync d5/d8/d3a/d71/daa/dc2/f99 0 2024-02-19T19:06:30.363 INFO:tasks.workunit.client.0.smithi088.stdout:2/625: link d3/f31 d3/d1f/d5d/fc5 0 2024-02-19T19:06:30.364 INFO:tasks.workunit.client.0.smithi088.stdout:5/901: dread d4/f12 [4194304,4194304] 0 2024-02-19T19:06:30.364 INFO:tasks.workunit.client.0.smithi088.stdout:6/818: creat d5/d8/d77/f100 x:0 0 0 2024-02-19T19:06:30.366 INFO:tasks.workunit.client.0.smithi088.stdout:2/626: mkdir d3/d1f/d4f/dc6 0 2024-02-19T19:06:30.368 INFO:tasks.workunit.client.0.smithi088.stdout:2/627: truncate d3/d1b/d43/f55 5670186 0 2024-02-19T19:06:30.370 INFO:tasks.workunit.client.0.smithi088.stdout:6/819: symlink d5/l101 0 2024-02-19T19:06:30.372 INFO:tasks.workunit.client.0.smithi088.stdout:6/820: read d5/d19/d26/f44 [5128909,105159] 0 2024-02-19T19:06:30.376 INFO:tasks.workunit.client.0.smithi088.stdout:6/821: creat d5/d8/d77/f102 x:0 0 0 2024-02-19T19:06:30.377 INFO:tasks.workunit.client.0.smithi088.stdout:5/902: write d4/d61/d65/d9a/d9f/f4c [1636671,1994] 0 2024-02-19T19:06:30.378 INFO:tasks.workunit.client.0.smithi088.stdout:5/903: chown d4/d61/la5 54376 1 2024-02-19T19:06:30.381 INFO:tasks.workunit.client.0.smithi088.stdout:1/735: dwrite d6/d10/f3b [0,4194304] 0 2024-02-19T19:06:30.382 INFO:tasks.workunit.client.0.smithi088.stdout:5/904: creat d4/d1a/d41/de2/de7/f13e x:0 0 0 2024-02-19T19:06:30.383 INFO:tasks.workunit.client.0.smithi088.stdout:5/905: fdatasync d4/d1a/f5b 0 2024-02-19T19:06:30.383 INFO:tasks.workunit.client.0.smithi088.stdout:1/736: mknod d6/d21/d6a/d7d/d7e/cf0 0 2024-02-19T19:06:30.388 INFO:tasks.workunit.client.0.smithi088.stdout:5/906: mknod d4/d7/c13f 0 2024-02-19T19:06:30.389 INFO:tasks.workunit.client.0.smithi088.stdout:1/737: fsync d6/d21/d6a/fc0 0 2024-02-19T19:06:30.389 INFO:tasks.workunit.client.0.smithi088.stdout:1/738: chown d6/d10/l5b 681980 1 2024-02-19T19:06:30.392 INFO:tasks.workunit.client.0.smithi088.stdout:1/739: mknod d6/d21/d64/d94/d6e/cf1 0 2024-02-19T19:06:30.393 INFO:tasks.workunit.client.0.smithi088.stdout:5/907: rename d4/d61/d65/d9a/d9f/l14 to d4/d1a/l140 0 2024-02-19T19:06:30.394 INFO:tasks.workunit.client.0.smithi088.stdout:1/740: symlink d6/d21/d64/d94/dab/lf2 0 2024-02-19T19:06:30.395 INFO:tasks.workunit.client.0.smithi088.stdout:1/741: write d6/f17 [691758,60913] 0 2024-02-19T19:06:30.399 INFO:tasks.workunit.client.0.smithi088.stdout:1/742: creat d6/d21/d6a/d7d/d7e/ff3 x:0 0 0 2024-02-19T19:06:30.400 INFO:tasks.workunit.client.0.smithi088.stdout:1/743: rmdir d6/d21/d64/d94/d6e 39 2024-02-19T19:06:30.406 INFO:tasks.workunit.client.0.smithi088.stdout:1/744: creat d6/d21/d64/d94/d6e/ff4 x:0 0 0 2024-02-19T19:06:30.409 INFO:tasks.workunit.client.0.smithi088.stdout:5/908: dwrite d4/d61/d65/d9a/d9f/d57/f6b [0,4194304] 0 2024-02-19T19:06:30.409 INFO:tasks.workunit.client.0.smithi088.stdout:1/745: rename d6/d21/d64/fd3 to d6/d21/d6a/d7d/d7e/ff5 0 2024-02-19T19:06:30.410 INFO:tasks.workunit.client.0.smithi088.stdout:1/746: fsync d6/d10/fc9 0 2024-02-19T19:06:30.411 INFO:tasks.workunit.client.0.smithi088.stdout:5/909: mkdir d4/d75/deb/d141 0 2024-02-19T19:06:30.413 INFO:tasks.workunit.client.0.smithi088.stdout:1/747: mkdir d6/df6 0 2024-02-19T19:06:30.413 INFO:tasks.workunit.client.0.smithi088.stdout:5/910: mknod d4/d1a/d41/dcb/c142 0 2024-02-19T19:06:30.414 INFO:tasks.workunit.client.0.smithi088.stdout:1/748: symlink d6/d21/d64/dc8/dcc/lf7 0 2024-02-19T19:06:30.415 INFO:tasks.workunit.client.0.smithi088.stdout:5/911: chown d4/d1a/f90 0 1 2024-02-19T19:06:30.418 INFO:tasks.workunit.client.0.smithi088.stdout:1/749: rename d6/d10/le8 to d6/d10/d60/lf8 0 2024-02-19T19:06:30.419 INFO:tasks.workunit.client.0.smithi088.stdout:1/750: chown d6/d10/fc9 52 1 2024-02-19T19:06:30.422 INFO:tasks.workunit.client.0.smithi088.stdout:5/912: rename d4/d61/d65/cbd to d4/d1a/d41/dc1/dc2/dfa/c143 0 2024-02-19T19:06:30.425 INFO:tasks.workunit.client.0.smithi088.stdout:5/913: unlink d4/d1a/d41/dcb/f118 0 2024-02-19T19:06:30.425 INFO:tasks.workunit.client.0.smithi088.stdout:1/751: creat d6/d21/d6a/ff9 x:0 0 0 2024-02-19T19:06:30.427 INFO:tasks.workunit.client.0.smithi088.stdout:5/914: mkdir d4/d1a/d41/dcb/d144 0 2024-02-19T19:06:30.432 INFO:tasks.workunit.client.0.smithi088.stdout:1/752: dwrite d6/d21/d64/fb0 [0,4194304] 0 2024-02-19T19:06:30.433 INFO:tasks.workunit.client.0.smithi088.stdout:5/915: link d4/d61/d65/d9a/d9f/d57/l11e d4/d61/d65/d9a/d9f/d57/d84/dfb/l145 0 2024-02-19T19:06:30.434 INFO:tasks.workunit.client.0.smithi088.stdout:1/753: write d6/f99 [835291,125676] 0 2024-02-19T19:06:30.435 INFO:tasks.workunit.client.0.smithi088.stdout:1/754: stat d6/d10/l85 0 2024-02-19T19:06:30.437 INFO:tasks.workunit.client.0.smithi088.stdout:1/755: chown d6/d21/d6a/d7d/da8/fee 31 1 2024-02-19T19:06:30.439 INFO:tasks.workunit.client.0.smithi088.stdout:5/916: dread d4/d61/d65/d9a/d9f/f4c [0,4194304] 0 2024-02-19T19:06:30.440 INFO:tasks.workunit.client.0.smithi088.stdout:1/756: symlink d6/d21/d6a/d7d/d9a/lfa 0 2024-02-19T19:06:30.441 INFO:tasks.workunit.client.0.smithi088.stdout:5/917: readlink d4/l69 0 2024-02-19T19:06:30.444 INFO:tasks.workunit.client.0.smithi088.stdout:5/918: link d4/d1a/f6e d4/d1a/d41/f146 0 2024-02-19T19:06:30.445 INFO:tasks.workunit.client.0.smithi088.stdout:5/919: fsync d4/d7/f92 0 2024-02-19T19:06:30.447 INFO:tasks.workunit.client.0.smithi088.stdout:5/920: mknod d4/d75/deb/d141/c147 0 2024-02-19T19:06:30.449 INFO:tasks.workunit.client.0.smithi088.stdout:5/921: write d4/d1a/d41/de2/de7/dff/d107/f82 [2487968,1289] 0 2024-02-19T19:06:30.453 INFO:tasks.workunit.client.0.smithi088.stdout:5/922: write d4/fcf [1439330,98441] 0 2024-02-19T19:06:30.457 INFO:tasks.workunit.client.0.smithi088.stdout:1/757: dread d6/d10/f11 [0,4194304] 0 2024-02-19T19:06:30.459 INFO:tasks.workunit.client.0.smithi088.stdout:1/758: symlink d6/d21/d64/dc8/dcc/lfb 0 2024-02-19T19:06:30.481 INFO:tasks.workunit.client.0.smithi088.stdout:1/759: write d6/d21/d64/dc8/dcc/fd4 [95836,104335] 0 2024-02-19T19:06:30.482 INFO:tasks.workunit.client.0.smithi088.stdout:1/760: readlink d6/d21/d6a/d7d/d7e/lc2 0 2024-02-19T19:06:30.484 INFO:tasks.workunit.client.0.smithi088.stdout:1/761: symlink d6/d21/d6a/d4a/dad/lfc 0 2024-02-19T19:06:30.485 INFO:tasks.workunit.client.0.smithi088.stdout:1/762: dread - d6/d10/d60/db8/dcf/fbb zero size 2024-02-19T19:06:30.488 INFO:tasks.workunit.client.0.smithi088.stdout:1/763: unlink d6/d21/d6a/f6d 0 2024-02-19T19:06:30.505 INFO:tasks.workunit.client.0.smithi088.stdout:1/764: dwrite d6/d10/d60/db8/dcf/fbb [0,4194304] 0 2024-02-19T19:06:30.507 INFO:tasks.workunit.client.0.smithi088.stdout:1/765: chown d6/d21/d6a/d7d/da8/fb5 921522 1 2024-02-19T19:06:30.510 INFO:tasks.workunit.client.0.smithi088.stdout:1/766: read d6/d10/d60/d29/fb3 [1306312,7563] 0 2024-02-19T19:06:30.513 INFO:tasks.workunit.client.0.smithi088.stdout:1/767: mkdir d6/d21/d64/d94/dab/dfd 0 2024-02-19T19:06:30.513 INFO:tasks.workunit.client.0.smithi088.stdout:1/768: stat d6/d21/f84 0 2024-02-19T19:06:30.514 INFO:tasks.workunit.client.0.smithi088.stdout:1/769: read - d6/d21/d6a/d7d/fa6 zero size 2024-02-19T19:06:30.515 INFO:tasks.workunit.client.0.smithi088.stdout:1/770: chown d6/d21/d64/d94/d6e/f7c 8962781 1 2024-02-19T19:06:30.516 INFO:tasks.workunit.client.0.smithi088.stdout:1/771: dread - d6/d21/d6a/d7d/d7e/fb4 zero size 2024-02-19T19:06:30.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:30 smithi132 ceph-mon[70676]: mgrmap e39: smithi088.gyviqq(active, since 91s), standbys: smithi132.psllsa 2024-02-19T19:06:30.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:30 smithi132 ceph-mon[70676]: pgmap v49: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 76 MiB/s rd, 197 MiB/s wr, 461 op/s 2024-02-19T19:06:30.787 INFO:tasks.workunit.client.0.smithi088.stdout:3/752: sync 2024-02-19T19:06:30.789 INFO:tasks.workunit.client.0.smithi088.stdout:3/753: chown d4/d22/d7a/d8d/fd2 210493 1 2024-02-19T19:06:30.791 INFO:tasks.workunit.client.0.smithi088.stdout:3/754: creat d4/d12/f109 x:0 0 0 2024-02-19T19:06:30.792 INFO:tasks.workunit.client.0.smithi088.stdout:3/755: fsync d4/f8e 0 2024-02-19T19:06:30.792 INFO:tasks.workunit.client.0.smithi088.stdout:3/756: readlink d4/d12/d33/dab/l106 0 2024-02-19T19:06:30.794 INFO:tasks.workunit.client.0.smithi088.stdout:3/757: mkdir d4/d12/d33/d43/d85/d10a 0 2024-02-19T19:06:30.797 INFO:tasks.workunit.client.0.smithi088.stdout:3/758: creat d4/d1a/d1e/d74/d7d/daa/dcb/f10b x:0 0 0 2024-02-19T19:06:30.798 INFO:tasks.workunit.client.0.smithi088.stdout:3/759: chown d4/d1a/f24 5787 1 2024-02-19T19:06:30.805 INFO:tasks.workunit.client.0.smithi088.stdout:3/760: dwrite d4/d87/fb6 [0,4194304] 0 2024-02-19T19:06:30.807 INFO:tasks.workunit.client.0.smithi088.stdout:3/761: creat d4/d1a/d1e/d74/d7d/daa/dcb/f10c x:0 0 0 2024-02-19T19:06:30.809 INFO:tasks.workunit.client.0.smithi088.stdout:3/762: mkdir d4/d22/d7a/da1/d10d 0 2024-02-19T19:06:30.810 INFO:tasks.workunit.client.0.smithi088.stdout:3/763: creat d4/d22/d7a/f10e x:0 0 0 2024-02-19T19:06:30.813 INFO:tasks.workunit.client.0.smithi088.stdout:3/764: link d4/d1a/d1e/d74/f83 d4/d12/d33/d43/ddd/f10f 0 2024-02-19T19:06:30.815 INFO:tasks.workunit.client.0.smithi088.stdout:3/765: fdatasync d4/d1a/d1e/d74/d7d/daa/fd6 0 2024-02-19T19:06:30.818 INFO:tasks.workunit.client.0.smithi088.stdout:3/766: mknod d4/d1a/d1e/d31/dc5/dcf/c110 0 2024-02-19T19:06:30.820 INFO:tasks.workunit.client.0.smithi088.stdout:3/767: readlink d4/d22/d81/lf5 0 2024-02-19T19:06:30.822 INFO:tasks.workunit.client.0.smithi088.stdout:3/768: creat d4/d87/dbf/f111 x:0 0 0 2024-02-19T19:06:30.824 INFO:tasks.workunit.client.0.smithi088.stdout:3/769: truncate d4/d1a/d23/f101 411227 0 2024-02-19T19:06:30.836 INFO:tasks.workunit.client.0.smithi088.stdout:3/770: dwrite d4/d12/d33/d43/f65 [0,4194304] 0 2024-02-19T19:06:30.838 INFO:tasks.workunit.client.0.smithi088.stdout:3/771: chown d4/d22/d81/lf5 228 1 2024-02-19T19:06:30.847 INFO:tasks.workunit.client.0.smithi088.stdout:3/772: dwrite d4/f6 [0,4194304] 0 2024-02-19T19:06:30.847 INFO:tasks.workunit.client.0.smithi088.stdout:3/773: chown d4/d1a/d1e/d31/d32/d55 7708014 1 2024-02-19T19:06:30.848 INFO:tasks.workunit.client.0.smithi088.stdout:3/774: stat d4/d12/f2e 0 2024-02-19T19:06:30.850 INFO:tasks.workunit.client.0.smithi088.stdout:3/775: mknod d4/d12/d33/d43/d85/c112 0 2024-02-19T19:06:30.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:30 smithi088 ceph-mon[88643]: mgrmap e39: smithi088.gyviqq(active, since 91s), standbys: smithi132.psllsa 2024-02-19T19:06:30.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:30 smithi088 ceph-mon[88643]: pgmap v49: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 76 MiB/s rd, 197 MiB/s wr, 461 op/s 2024-02-19T19:06:30.859 INFO:tasks.workunit.client.0.smithi088.stdout:3/776: dwrite d4/f38 [0,4194304] 0 2024-02-19T19:06:30.861 INFO:tasks.workunit.client.0.smithi088.stdout:3/777: symlink d4/d12/d33/d58/l113 0 2024-02-19T19:06:30.863 INFO:tasks.workunit.client.0.smithi088.stdout:3/778: mkdir d4/d22/dd5/d104/d114 0 2024-02-19T19:06:30.866 INFO:tasks.workunit.client.0.smithi088.stdout:3/779: creat d4/d1a/f115 x:0 0 0 2024-02-19T19:06:30.868 INFO:tasks.workunit.client.0.smithi088.stdout:3/780: dread - d4/d1a/d1e/d74/d7d/daa/fe9 zero size 2024-02-19T19:06:30.870 INFO:tasks.workunit.client.0.smithi088.stdout:1/772: sync 2024-02-19T19:06:30.870 INFO:tasks.workunit.client.0.smithi088.stdout:9/973: sync 2024-02-19T19:06:30.870 INFO:tasks.workunit.client.0.smithi088.stdout:5/923: sync 2024-02-19T19:06:30.870 INFO:tasks.workunit.client.0.smithi088.stdout:4/548: sync 2024-02-19T19:06:30.870 INFO:tasks.workunit.client.0.smithi088.stdout:7/736: sync 2024-02-19T19:06:30.870 INFO:tasks.workunit.client.0.smithi088.stdout:6/822: sync 2024-02-19T19:06:30.870 INFO:tasks.workunit.client.0.smithi088.stdout:0/709: sync 2024-02-19T19:06:30.870 INFO:tasks.workunit.client.0.smithi088.stdout:2/628: sync 2024-02-19T19:06:30.870 INFO:tasks.workunit.client.0.smithi088.stdout:8/613: sync 2024-02-19T19:06:30.871 INFO:tasks.workunit.client.0.smithi088.stdout:5/924: readlink d4/d7/l27 0 2024-02-19T19:06:30.876 INFO:tasks.workunit.client.0.smithi088.stdout:0/710: mknod d13/d62/d8a/d95/d5a/df3/cf6 0 2024-02-19T19:06:30.877 INFO:tasks.workunit.client.0.smithi088.stdout:8/614: write d9/df/d3d/d50/d71/da1/d6c/d9a/f5e [3692316,250] 0 2024-02-19T19:06:30.877 INFO:tasks.workunit.client.0.smithi088.stdout:7/737: unlink d14/d1d/d3c/d43/d5c/d75/la0 0 2024-02-19T19:06:30.880 INFO:tasks.workunit.client.0.smithi088.stdout:0/711: chown d13/d62/d8a/d95/d5a/db6/cd4 51734961 1 2024-02-19T19:06:30.880 INFO:tasks.workunit.client.0.smithi088.stdout:1/773: write d6/f1e [2481576,108153] 0 2024-02-19T19:06:30.881 INFO:tasks.workunit.client.0.smithi088.stdout:6/823: dread d5/d19/d26/f27 [4194304,4194304] 0 2024-02-19T19:06:30.883 INFO:tasks.workunit.client.0.smithi088.stdout:1/774: fdatasync d6/d10/f24 0 2024-02-19T19:06:30.885 INFO:tasks.workunit.client.0.smithi088.stdout:2/629: rename d3/d1b/d43/f58 to d3/d1f/d4f/d59/fc7 0 2024-02-19T19:06:30.885 INFO:tasks.workunit.client.0.smithi088.stdout:0/712: write d13/d62/d8a/d95/fc4 [326366,59153] 0 2024-02-19T19:06:30.887 INFO:tasks.workunit.client.0.smithi088.stdout:5/925: symlink d4/d75/de8/d136/l148 0 2024-02-19T19:06:30.891 INFO:tasks.workunit.client.0.smithi088.stdout:4/549: dwrite d2/d8/f6b [0,4194304] 0 2024-02-19T19:06:30.896 INFO:tasks.workunit.client.0.smithi088.stdout:8/615: mknod d9/df/d1c/d85/daf/cb9 0 2024-02-19T19:06:30.898 INFO:tasks.workunit.client.0.smithi088.stdout:3/781: dread d4/d1a/d1e/d74/f96 [0,4194304] 0 2024-02-19T19:06:30.900 INFO:tasks.workunit.client.0.smithi088.stdout:4/550: chown d2/d32/d25/f30 212694 1 2024-02-19T19:06:30.903 INFO:tasks.workunit.client.0.smithi088.stdout:5/926: write d4/d61/d65/d9a/d9f/d57/d84/f119 [1114310,123540] 0 2024-02-19T19:06:30.903 INFO:tasks.workunit.client.0.smithi088.stdout:7/738: dwrite d14/d27/d49/d98/d107/fe9 [0,4194304] 0 2024-02-19T19:06:30.905 INFO:tasks.workunit.client.0.smithi088.stdout:9/974: write d1/d8/d3e/d93/d2d/d18/d1f/df2/f9d [4230045,14321] 0 2024-02-19T19:06:30.909 INFO:tasks.workunit.client.0.smithi088.stdout:0/713: dwrite d13/d62/d8a/d95/d5b/fb5 [4194304,4194304] 0 2024-02-19T19:06:30.911 INFO:tasks.workunit.client.0.smithi088.stdout:7/739: write d14/d27/d49/d98/dcb/fff [33816,8981] 0 2024-02-19T19:06:30.916 INFO:tasks.workunit.client.0.smithi088.stdout:6/824: rmdir d5/d8/d3a/da8/dda 0 2024-02-19T19:06:30.916 INFO:tasks.workunit.client.0.smithi088.stdout:1/775: dwrite d6/d21/d6a/ff9 [0,4194304] 0 2024-02-19T19:06:30.918 INFO:tasks.workunit.client.0.smithi088.stdout:0/714: stat d13/d62/d8a/d95/d5a/d25/d2a/d34/f40 0 2024-02-19T19:06:30.919 INFO:tasks.workunit.client.0.smithi088.stdout:9/975: rmdir d1/d8/d8d/d97 39 2024-02-19T19:06:30.921 INFO:tasks.workunit.client.0.smithi088.stdout:4/551: dwrite d2/d32/f1c [0,4194304] 0 2024-02-19T19:06:30.923 INFO:tasks.workunit.client.0.smithi088.stdout:7/740: truncate d14/d1d/d3c/d43/db2/f106 730900 0 2024-02-19T19:06:30.924 INFO:tasks.workunit.client.0.smithi088.stdout:0/715: dread d13/d62/d8a/d95/d5a/d25/d2a/d2f/f41 [0,4194304] 0 2024-02-19T19:06:30.924 INFO:tasks.workunit.client.0.smithi088.stdout:2/630: creat d3/d1b/d43/d61/fc8 x:0 0 0 2024-02-19T19:06:30.925 INFO:tasks.workunit.client.0.smithi088.stdout:2/631: chown d3/d1b/d43/d74/c83 324340116 1 2024-02-19T19:06:30.926 INFO:tasks.workunit.client.0.smithi088.stdout:6/825: creat d5/de/d60/f103 x:0 0 0 2024-02-19T19:06:30.932 INFO:tasks.workunit.client.0.smithi088.stdout:2/632: fdatasync d3/d1f/d4f/d59/f5f 0 2024-02-19T19:06:30.934 INFO:tasks.workunit.client.0.smithi088.stdout:4/552: write d2/f75 [923090,13506] 0 2024-02-19T19:06:30.937 INFO:tasks.workunit.client.0.smithi088.stdout:8/616: creat d9/df/d3d/d50/d71/da1/d6c/fba x:0 0 0 2024-02-19T19:06:30.938 INFO:tasks.workunit.client.0.smithi088.stdout:9/976: mkdir d1/d8/d3e/d93/d2d/d18/d13d 0 2024-02-19T19:06:30.939 INFO:tasks.workunit.client.0.smithi088.stdout:7/741: read d14/f19 [6609254,110276] 0 2024-02-19T19:06:30.941 INFO:tasks.workunit.client.0.smithi088.stdout:3/782: dwrite d4/d22/f2b [0,4194304] 0 2024-02-19T19:06:30.941 INFO:tasks.workunit.client.0.smithi088.stdout:2/633: rmdir d3/d1f/d5d 39 2024-02-19T19:06:30.948 INFO:tasks.workunit.client.0.smithi088.stdout:6/826: dwrite d5/d19/f6d [0,4194304] 0 2024-02-19T19:06:30.949 INFO:tasks.workunit.client.0.smithi088.stdout:9/977: fdatasync d1/d8/d3e/d93/d5a/d94/d104/f137 0 2024-02-19T19:06:30.952 INFO:tasks.workunit.client.0.smithi088.stdout:3/783: readlink d4/l70 0 2024-02-19T19:06:30.952 INFO:tasks.workunit.client.0.smithi088.stdout:1/776: dwrite d6/d21/f72 [0,4194304] 0 2024-02-19T19:06:30.954 INFO:tasks.workunit.client.0.smithi088.stdout:9/978: chown d1/c78 0 1 2024-02-19T19:06:30.954 INFO:tasks.workunit.client.0.smithi088.stdout:4/553: unlink d2/d32/d25/d26/d6d/ca8 0 2024-02-19T19:06:30.955 INFO:tasks.workunit.client.0.smithi088.stdout:5/927: dread d4/d1a/d41/fe6 [0,4194304] 0 2024-02-19T19:06:30.958 INFO:tasks.workunit.client.0.smithi088.stdout:5/928: read - d4/d61/d65/d9a/d9f/d57/d84/dfb/f121 zero size 2024-02-19T19:06:30.959 INFO:tasks.workunit.client.0.smithi088.stdout:5/929: dread - d4/f13d zero size 2024-02-19T19:06:30.959 INFO:tasks.workunit.client.0.smithi088.stdout:7/742: mkdir d14/d27/d6b/df8/d10b 0 2024-02-19T19:06:30.959 INFO:tasks.workunit.client.0.smithi088.stdout:2/634: symlink d3/d1b/d43/d61/lc9 0 2024-02-19T19:06:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:2/635: write d3/d1b/d43/d8f/f8e [217449,5134] 0 2024-02-19T19:06:30.962 INFO:tasks.workunit.client.0.smithi088.stdout:4/554: creat d2/d32/d25/d45/fb0 x:0 0 0 2024-02-19T19:06:30.964 INFO:tasks.workunit.client.0.smithi088.stdout:8/617: creat d9/df/d3d/d50/d71/da1/d6c/d9a/fbb x:0 0 0 2024-02-19T19:06:30.965 INFO:tasks.workunit.client.0.smithi088.stdout:6/827: symlink d5/d8/d2c/dd6/l104 0 2024-02-19T19:06:30.965 INFO:tasks.workunit.client.0.smithi088.stdout:8/618: chown d9/df/f36 443500768 1 2024-02-19T19:06:30.967 INFO:tasks.workunit.client.0.smithi088.stdout:0/716: dread d13/d62/d8a/d95/d5a/d25/d2a/d2f/f4c [4194304,4194304] 0 2024-02-19T19:06:30.969 INFO:tasks.workunit.client.0.smithi088.stdout:7/743: dread d14/d27/f36 [0,4194304] 0 2024-02-19T19:06:30.970 INFO:tasks.workunit.client.0.smithi088.stdout:5/930: unlink d4/d61/d65/d9a/d9f/d57/f135 0 2024-02-19T19:06:30.971 INFO:tasks.workunit.client.0.smithi088.stdout:2/636: write d3/d1f/f89 [846323,76044] 0 2024-02-19T19:06:30.976 INFO:tasks.workunit.client.0.smithi088.stdout:6/828: write d5/d45/fab [1908234,123056] 0 2024-02-19T19:06:30.978 INFO:tasks.workunit.client.0.smithi088.stdout:0/717: creat d13/d62/d8a/d95/d96/ded/ff7 x:0 0 0 2024-02-19T19:06:30.980 INFO:tasks.workunit.client.0.smithi088.stdout:4/555: dwrite d2/d32/d25/d26/d6d/d7e/d90/fa7 [0,4194304] 0 2024-02-19T19:06:30.981 INFO:tasks.workunit.client.0.smithi088.stdout:5/931: creat d4/d1a/d41/de2/de7/f149 x:0 0 0 2024-02-19T19:06:30.983 INFO:tasks.workunit.client.0.smithi088.stdout:9/979: truncate d1/fc 832287 0 2024-02-19T19:06:30.983 INFO:tasks.workunit.client.0.smithi088.stdout:4/556: chown d2/d32/d25/d26/d6d/d7e/d3a/l3c 1200 1 2024-02-19T19:06:30.984 INFO:tasks.workunit.client.0.smithi088.stdout:1/777: write d6/f92 [801042,64050] 0 2024-02-19T19:06:30.986 INFO:tasks.workunit.client.0.smithi088.stdout:2/637: write d3/d14/f56 [7487347,30521] 0 2024-02-19T19:06:30.990 INFO:tasks.workunit.client.0.smithi088.stdout:4/557: symlink d2/d32/d25/lb1 0 2024-02-19T19:06:30.993 INFO:tasks.workunit.client.0.smithi088.stdout:0/718: symlink d13/d62/d8a/d95/d5b/dbe/dc6/lf8 0 2024-02-19T19:06:30.994 INFO:tasks.workunit.client.0.smithi088.stdout:8/619: getdents d9/df/d3d/d50/d71 0 2024-02-19T19:06:30.995 INFO:tasks.workunit.client.0.smithi088.stdout:5/932: dwrite d4/d61/d65/d9a/d9f/d57/d84/dfb/f121 [0,4194304] 0 2024-02-19T19:06:30.997 INFO:tasks.workunit.client.0.smithi088.stdout:1/778: truncate d6/d21/f3a 2900611 0 2024-02-19T19:06:30.997 INFO:tasks.workunit.client.0.smithi088.stdout:9/980: creat d1/d8/d3e/d93/d122/d114/f13e x:0 0 0 2024-02-19T19:06:31.000 INFO:tasks.workunit.client.0.smithi088.stdout:6/829: link d5/d19/d26/c3c d5/d7/d32/d56/c105 0 2024-02-19T19:06:31.002 INFO:tasks.workunit.client.0.smithi088.stdout:7/744: write d14/f6e [722669,105178] 0 2024-02-19T19:06:31.003 INFO:tasks.workunit.client.0.smithi088.stdout:7/745: fsync d14/d1d/d3c/f8f 0 2024-02-19T19:06:31.004 INFO:tasks.workunit.client.0.smithi088.stdout:0/719: mkdir d13/d62/d8a/d95/d5a/d25/dd6/df9 0 2024-02-19T19:06:31.005 INFO:tasks.workunit.client.0.smithi088.stdout:0/720: chown d13/d62 24249 1 2024-02-19T19:06:31.006 INFO:tasks.workunit.client.0.smithi088.stdout:8/620: fsync d9/df/d3d/d50/d71/da1/f4b 0 2024-02-19T19:06:31.011 INFO:tasks.workunit.client.0.smithi088.stdout:6/830: creat d5/d19/d70/f106 x:0 0 0 2024-02-19T19:06:31.013 INFO:tasks.workunit.client.0.smithi088.stdout:2/638: link d3/d14/f5c d3/d1b/d43/d61/da6/db2/fca 0 2024-02-19T19:06:31.014 INFO:tasks.workunit.client.0.smithi088.stdout:4/558: mkdir d2/d32/d25/d26/da5/db2 0 2024-02-19T19:06:31.017 INFO:tasks.workunit.client.0.smithi088.stdout:7/746: creat d14/d27/d49/d98/dcb/de7/f10c x:0 0 0 2024-02-19T19:06:31.020 INFO:tasks.workunit.client.0.smithi088.stdout:2/639: dwrite d3/d1b/d43/d61/d64/f6d [4194304,4194304] 0 2024-02-19T19:06:31.021 INFO:tasks.workunit.client.0.smithi088.stdout:9/981: fdatasync d1/d8/d4a/f103 0 2024-02-19T19:06:31.025 INFO:tasks.workunit.client.0.smithi088.stdout:0/721: mknod d13/de9/cfa 0 2024-02-19T19:06:31.031 INFO:tasks.workunit.client.0.smithi088.stdout:5/933: creat d4/d61/d65/d9a/d9f/d57/f14a x:0 0 0 2024-02-19T19:06:31.031 INFO:tasks.workunit.client.0.smithi088.stdout:5/934: readlink d4/d75/l12d 0 2024-02-19T19:06:31.034 INFO:tasks.workunit.client.0.smithi088.stdout:1/779: creat d6/d21/d64/d94/ffe x:0 0 0 2024-02-19T19:06:31.036 INFO:tasks.workunit.client.0.smithi088.stdout:9/982: chown d1/d8/d37/l66 1867 1 2024-02-19T19:06:31.037 INFO:tasks.workunit.client.0.smithi088.stdout:9/983: chown d1/l125 111581 1 2024-02-19T19:06:31.037 INFO:tasks.workunit.client.0.smithi088.stdout:6/831: symlink d5/d19/l107 0 2024-02-19T19:06:31.039 INFO:tasks.workunit.client.0.smithi088.stdout:4/559: mkdir d2/d35/db3 0 2024-02-19T19:06:31.042 INFO:tasks.workunit.client.0.smithi088.stdout:1/780: dwrite d6/d10/f24 [0,4194304] 0 2024-02-19T19:06:31.043 INFO:tasks.workunit.client.0.smithi088.stdout:2/640: creat d3/d1f/d4f/dc6/fcb x:0 0 0 2024-02-19T19:06:31.044 INFO:tasks.workunit.client.0.smithi088.stdout:8/621: truncate d9/df/d3d/d62/fb1 3740254 0 2024-02-19T19:06:31.047 INFO:tasks.workunit.client.0.smithi088.stdout:5/935: mknod d4/d61/d65/c14b 0 2024-02-19T19:06:31.048 INFO:tasks.workunit.client.0.smithi088.stdout:1/781: readlink d6/le2 0 2024-02-19T19:06:31.050 INFO:tasks.workunit.client.0.smithi088.stdout:9/984: symlink d1/d8/d4a/d4f/l13f 0 2024-02-19T19:06:31.050 INFO:tasks.workunit.client.0.smithi088.stdout:7/747: dwrite d14/d1d/d62/f100 [0,4194304] 0 2024-02-19T19:06:31.054 INFO:tasks.workunit.client.0.smithi088.stdout:0/722: getdents d13/d62/d8a/d95/d5b/dbe/dc6 0 2024-02-19T19:06:31.058 INFO:tasks.workunit.client.0.smithi088.stdout:2/641: creat d3/d1b/d43/d61/da6/db2/fcc x:0 0 0 2024-02-19T19:06:31.059 INFO:tasks.workunit.client.0.smithi088.stdout:2/642: chown d3/d1b/d43/d74/c9a 12716 1 2024-02-19T19:06:31.060 INFO:tasks.workunit.client.0.smithi088.stdout:7/748: write d14/f4f [878438,112034] 0 2024-02-19T19:06:31.061 INFO:tasks.workunit.client.0.smithi088.stdout:7/749: chown d14/d1d 48 1 2024-02-19T19:06:31.062 INFO:tasks.workunit.client.0.smithi088.stdout:8/622: symlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/lbc 0 2024-02-19T19:06:31.064 INFO:tasks.workunit.client.0.smithi088.stdout:9/985: write d1/d8/d37/d54/f65 [3884363,95419] 0 2024-02-19T19:06:31.065 INFO:tasks.workunit.client.0.smithi088.stdout:0/723: symlink d13/d62/lfb 0 2024-02-19T19:06:31.067 INFO:tasks.workunit.client.0.smithi088.stdout:9/986: dread - d1/d8/d3e/d93/d122/d114/f13e zero size 2024-02-19T19:06:31.067 INFO:tasks.workunit.client.0.smithi088.stdout:6/832: getdents d5/d19 0 2024-02-19T19:06:31.069 INFO:tasks.workunit.client.0.smithi088.stdout:6/833: fsync d5/de/d60/db1/ff9 0 2024-02-19T19:06:31.070 INFO:tasks.workunit.client.0.smithi088.stdout:7/750: creat d14/d27/d49/d98/dcb/de7/f10d x:0 0 0 2024-02-19T19:06:31.071 INFO:tasks.workunit.client.0.smithi088.stdout:4/560: getdents d2/d35 0 2024-02-19T19:06:31.071 INFO:tasks.workunit.client.0.smithi088.stdout:7/751: write f2 [511077,128366] 0 2024-02-19T19:06:31.073 INFO:tasks.workunit.client.0.smithi088.stdout:5/936: rename d4/d7/l85 to d4/d1a/d41/dc1/dc2/dfa/l14c 0 2024-02-19T19:06:31.076 INFO:tasks.workunit.client.0.smithi088.stdout:0/724: creat d13/d62/d8a/d95/d5a/d25/d2a/d2f/ffc x:0 0 0 2024-02-19T19:06:31.077 INFO:tasks.workunit.client.0.smithi088.stdout:2/643: mknod d3/d1b/d43/d61/ccd 0 2024-02-19T19:06:31.078 INFO:tasks.workunit.client.0.smithi088.stdout:9/987: creat d1/d8/d3e/d93/d2d/d18/d1f/df2/d99/f140 x:0 0 0 2024-02-19T19:06:31.079 INFO:tasks.workunit.client.0.smithi088.stdout:6/834: mkdir d5/d45/d9d/d108 0 2024-02-19T19:06:31.083 INFO:tasks.workunit.client.0.smithi088.stdout:7/752: creat d14/d28/d3b/dd3/f10e x:0 0 0 2024-02-19T19:06:31.086 INFO:tasks.workunit.client.0.smithi088.stdout:9/988: dwrite d1/d8/d3e/d93/d122/d114/f13e [0,4194304] 0 2024-02-19T19:06:31.088 INFO:tasks.workunit.client.0.smithi088.stdout:0/725: fsync d13/d62/d8a/d95/d5b/d76/f83 0 2024-02-19T19:06:31.091 INFO:tasks.workunit.client.0.smithi088.stdout:5/937: dread d4/d61/f78 [0,4194304] 0 2024-02-19T19:06:31.092 INFO:tasks.workunit.client.0.smithi088.stdout:1/782: getdents d6/d10 0 2024-02-19T19:06:31.095 INFO:tasks.workunit.client.0.smithi088.stdout:4/561: write d2/d8/f6a [272899,35481] 0 2024-02-19T19:06:31.097 INFO:tasks.workunit.client.0.smithi088.stdout:2/644: rmdir d3/d1f/d4f 39 2024-02-19T19:06:31.098 INFO:tasks.workunit.client.0.smithi088.stdout:7/753: symlink d14/d1d/d3c/d43/dc2/l10f 0 2024-02-19T19:06:31.099 INFO:tasks.workunit.client.0.smithi088.stdout:7/754: chown d14/d28/d65/fec 9 1 2024-02-19T19:06:31.100 INFO:tasks.workunit.client.0.smithi088.stdout:4/562: dread d2/d32/d25/d26/d6d/d7e/d90/fa7 [0,4194304] 0 2024-02-19T19:06:31.103 INFO:tasks.workunit.client.0.smithi088.stdout:1/783: mkdir d6/d21/d6a/d4a/dff 0 2024-02-19T19:06:31.103 INFO:tasks.workunit.client.0.smithi088.stdout:0/726: truncate d13/d62/d8a/d95/d5a/d25/d2a/d34/f35 2432781 0 2024-02-19T19:06:31.104 INFO:tasks.workunit.client.0.smithi088.stdout:8/623: getdents d9/df/d1c/d22 0 2024-02-19T19:06:31.108 INFO:tasks.workunit.client.0.smithi088.stdout:6/835: dread d5/d8/f3f [0,4194304] 0 2024-02-19T19:06:31.108 INFO:tasks.workunit.client.0.smithi088.stdout:5/938: dwrite d4/f5 [0,4194304] 0 2024-02-19T19:06:31.110 INFO:tasks.workunit.client.0.smithi088.stdout:7/755: mknod d14/d1d/d3c/d43/d5c/d75/c110 0 2024-02-19T19:06:31.111 INFO:tasks.workunit.client.0.smithi088.stdout:4/563: mknod d2/d32/d9/cb4 0 2024-02-19T19:06:31.116 INFO:tasks.workunit.client.0.smithi088.stdout:8/624: mknod d9/df/d3d/d50/d71/da1/d6c/d77/cbd 0 2024-02-19T19:06:31.116 INFO:tasks.workunit.client.0.smithi088.stdout:0/727: rmdir d13 39 2024-02-19T19:06:31.117 INFO:tasks.workunit.client.0.smithi088.stdout:5/939: dwrite d4/d7/f15 [4194304,4194304] 0 2024-02-19T19:06:31.118 INFO:tasks.workunit.client.0.smithi088.stdout:1/784: truncate d6/d10/d60/d29/fb3 153202 0 2024-02-19T19:06:31.119 INFO:tasks.workunit.client.0.smithi088.stdout:1/785: dread - d6/d10/fc9 zero size 2024-02-19T19:06:31.120 INFO:tasks.workunit.client.0.smithi088.stdout:6/836: mkdir d5/d7/dc9/d109 0 2024-02-19T19:06:31.120 INFO:tasks.workunit.client.0.smithi088.stdout:9/989: getdents d1/d8/d3e/d93/d2d/d18/d1f 0 2024-02-19T19:06:31.121 INFO:tasks.workunit.client.0.smithi088.stdout:3/784: sync 2024-02-19T19:06:31.122 INFO:tasks.workunit.client.0.smithi088.stdout:9/990: read - d1/d8/d4a/fb7 zero size 2024-02-19T19:06:31.122 INFO:tasks.workunit.client.0.smithi088.stdout:9/991: chown d1 6 1 2024-02-19T19:06:31.124 INFO:tasks.workunit.client.0.smithi088.stdout:9/992: dread - d1/d8/d3e/d93/d5a/d94/d74/f101 zero size 2024-02-19T19:06:31.125 INFO:tasks.workunit.client.0.smithi088.stdout:8/625: mkdir d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/dbe 0 2024-02-19T19:06:31.126 INFO:tasks.workunit.client.0.smithi088.stdout:5/940: creat d4/d1a/d41/de2/de7/dff/d114/f14d x:0 0 0 2024-02-19T19:06:31.127 INFO:tasks.workunit.client.0.smithi088.stdout:3/785: mknod d4/d12/d33/d43/c116 0 2024-02-19T19:06:31.128 INFO:tasks.workunit.client.0.smithi088.stdout:3/786: chown d4/d1a/d1e/d31/d32/d55/fb3 17 1 2024-02-19T19:06:31.129 INFO:tasks.workunit.client.0.smithi088.stdout:6/837: creat d5/d8/d3a/d71/daa/dc2/dbc/db6/f10a x:0 0 0 2024-02-19T19:06:31.130 INFO:tasks.workunit.client.0.smithi088.stdout:6/838: chown d5/d8/d3a/da8/dbd/ff8 0 1 2024-02-19T19:06:31.131 INFO:tasks.workunit.client.0.smithi088.stdout:7/756: rename d14/d1d/d3c/f6c to d14/d27/d6b/df8/d10b/f111 0 2024-02-19T19:06:31.131 INFO:tasks.workunit.client.0.smithi088.stdout:2/645: link d3/d1b/d43/d61/d64/c96 d3/d1b/d43/d74/cce 0 2024-02-19T19:06:31.132 INFO:tasks.workunit.client.0.smithi088.stdout:2/646: fsync d3/d1b/d43/d74/f82 0 2024-02-19T19:06:31.133 INFO:tasks.workunit.client.0.smithi088.stdout:9/993: creat d1/d8/d3e/d93/d2d/d18/d1f/df2/f141 x:0 0 0 2024-02-19T19:06:31.133 INFO:tasks.workunit.client.0.smithi088.stdout:8/626: fdatasync d9/df/d1c/d85/f8d 0 2024-02-19T19:06:31.136 INFO:tasks.workunit.client.0.smithi088.stdout:9/994: truncate d1/d8/d3e/d93/d2f/f110 69614 0 2024-02-19T19:06:31.138 INFO:tasks.workunit.client.0.smithi088.stdout:4/564: rename d2/d8/f6a to d2/d32/d25/d3f/d9a/fb5 0 2024-02-19T19:06:31.140 INFO:tasks.workunit.client.0.smithi088.stdout:7/757: unlink d14/d1d/d3c/d43/d5c/d9a/d92/fd4 0 2024-02-19T19:06:31.140 INFO:tasks.workunit.client.0.smithi088.stdout:4/565: write d2/d35/f53 [379098,82859] 0 2024-02-19T19:06:31.140 INFO:tasks.workunit.client.0.smithi088.stdout:1/786: dread d6/d21/d6a/d4a/f56 [0,4194304] 0 2024-02-19T19:06:31.144 INFO:tasks.workunit.client.0.smithi088.stdout:3/787: mkdir d4/d117 0 2024-02-19T19:06:31.145 INFO:tasks.workunit.client.0.smithi088.stdout:1/787: chown d6/d21/d6a/fb7 1844015 1 2024-02-19T19:06:31.145 INFO:tasks.workunit.client.0.smithi088.stdout:6/839: fdatasync d5/d19/d26/fa1 0 2024-02-19T19:06:31.148 INFO:tasks.workunit.client.0.smithi088.stdout:8/627: creat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/fbf x:0 0 0 2024-02-19T19:06:31.149 INFO:tasks.workunit.client.0.smithi088.stdout:9/995: dwrite d1/d8/d4a/d4f/daa/fd8 [0,4194304] 0 2024-02-19T19:06:31.153 INFO:tasks.workunit.client.0.smithi088.stdout:0/728: rename d13/d62/fae to d13/de9/ffd 0 2024-02-19T19:06:31.159 INFO:tasks.workunit.client.0.smithi088.stdout:2/647: dread d3/d14/f4d [0,4194304] 0 2024-02-19T19:06:31.160 INFO:tasks.workunit.client.0.smithi088.stdout:5/941: dread d4/d75/f103 [0,4194304] 0 2024-02-19T19:06:31.160 INFO:tasks.workunit.client.0.smithi088.stdout:4/566: symlink d2/d32/d25/d3f/d70/lb6 0 2024-02-19T19:06:31.161 INFO:tasks.workunit.client.0.smithi088.stdout:5/942: chown d4/d61/d65 3807365 1 2024-02-19T19:06:31.161 INFO:tasks.workunit.client.0.smithi088.stdout:4/567: chown d2/f75 61267 1 2024-02-19T19:06:31.162 INFO:tasks.workunit.client.0.smithi088.stdout:3/788: mkdir d4/d87/d118 0 2024-02-19T19:06:31.163 INFO:tasks.workunit.client.0.smithi088.stdout:3/789: chown d4/d1a/d1e/d31/d32/f8c 15 1 2024-02-19T19:06:31.166 INFO:tasks.workunit.client.0.smithi088.stdout:8/628: symlink d9/df/d1c/lc0 0 2024-02-19T19:06:31.168 INFO:tasks.workunit.client.0.smithi088.stdout:8/629: write d9/df/d3d/d50/d71/da1/d6c/d9a/fbb [386509,18839] 0 2024-02-19T19:06:31.170 INFO:tasks.workunit.client.0.smithi088.stdout:9/996: creat d1/d8/d3e/d93/d5a/dc4/f142 x:0 0 0 2024-02-19T19:06:31.172 INFO:tasks.workunit.client.0.smithi088.stdout:7/758: mkdir d14/d27/d6b/dc4/d112 0 2024-02-19T19:06:31.176 INFO:tasks.workunit.client.0.smithi088.stdout:8/630: dwrite d9/df/d3d/d50/d71/da1/d70/fa7 [0,4194304] 0 2024-02-19T19:06:31.185 INFO:tasks.workunit.client.0.smithi088.stdout:5/943: symlink d4/d61/l14e 0 2024-02-19T19:06:31.186 INFO:tasks.workunit.client.0.smithi088.stdout:4/568: mkdir d2/d35/db3/db7 0 2024-02-19T19:06:31.190 INFO:tasks.workunit.client.0.smithi088.stdout:6/840: dwrite d5/d19/d26/f44 [0,4194304] 0 2024-02-19T19:06:31.191 INFO:tasks.workunit.client.0.smithi088.stdout:2/648: write d3/d1b/f42 [734524,100016] 0 2024-02-19T19:06:31.194 INFO:tasks.workunit.client.0.smithi088.stdout:3/790: write d4/f6b [1031176,3959] 0 2024-02-19T19:06:31.194 INFO:tasks.workunit.client.0.smithi088.stdout:6/841: fsync d5/d8/d3a/da8/fcf 0 2024-02-19T19:06:31.196 INFO:tasks.workunit.client.0.smithi088.stdout:0/729: truncate d13/d62/d8a/d95/fc4 88418 0 2024-02-19T19:06:31.197 INFO:tasks.workunit.client.0.smithi088.stdout:1/788: getdents d6/d21/d64/dc8 0 2024-02-19T19:06:31.199 INFO:tasks.workunit.client.0.smithi088.stdout:1/789: truncate d6/f92 1170317 0 2024-02-19T19:06:31.200 INFO:tasks.workunit.client.0.smithi088.stdout:9/997: rmdir d1/d8/d4a/d4f/d11e 0 2024-02-19T19:06:31.204 INFO:tasks.workunit.client.0.smithi088.stdout:6/842: symlink d5/d8/d77/l10b 0 2024-02-19T19:06:31.205 INFO:tasks.workunit.client.0.smithi088.stdout:1/790: dread - d6/d21/d6a/d7d/d7e/ff3 zero size 2024-02-19T19:06:31.205 INFO:tasks.workunit.client.0.smithi088.stdout:2/649: dwrite d3/d1b/d43/f52 [0,4194304] 0 2024-02-19T19:06:31.207 INFO:tasks.workunit.client.0.smithi088.stdout:4/569: mkdir d2/d32/d25/d26/db8 0 2024-02-19T19:06:31.210 INFO:tasks.workunit.client.0.smithi088.stdout:9/998: creat d1/d8/d3e/d93/d2f/f143 x:0 0 0 2024-02-19T19:06:31.211 INFO:tasks.workunit.client.0.smithi088.stdout:4/570: dread - d2/d32/d25/d3f/d70/fae zero size 2024-02-19T19:06:31.212 INFO:tasks.workunit.client.0.smithi088.stdout:7/759: getdents d14/d27/d6b/df8 0 2024-02-19T19:06:31.214 INFO:tasks.workunit.client.0.smithi088.stdout:3/791: creat d4/d100/f119 x:0 0 0 2024-02-19T19:06:31.219 INFO:tasks.workunit.client.0.smithi088.stdout:0/730: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d3c/fad [0,4194304] 0 2024-02-19T19:06:31.219 INFO:tasks.workunit.client.0.smithi088.stdout:9/999: truncate d1/d8/d3e/d93/d2d/f135 462253 0 2024-02-19T19:06:31.219 INFO:tasks.workunit.client.0.smithi088.stdout:7/760: chown d14/d27/d49/d98/dbe/feb 21 1 2024-02-19T19:06:31.221 INFO:tasks.workunit.client.0.smithi088.stdout:7/761: readlink d14/d27/d6b/lfb 0 2024-02-19T19:06:31.224 INFO:tasks.workunit.client.0.smithi088.stdout:2/650: truncate d3/d1f/d5d/fb4 290635 0 2024-02-19T19:06:31.224 INFO:tasks.workunit.client.0.smithi088.stdout:5/944: rename d4/d1a/d41/dc1/dc2/c11a to d4/d61/d65/c14f 0 2024-02-19T19:06:31.224 INFO:tasks.workunit.client.0.smithi088.stdout:8/631: dwrite d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8/f89 [0,4194304] 0 2024-02-19T19:06:31.226 INFO:tasks.workunit.client.0.smithi088.stdout:1/791: symlink d6/d21/d64/d94/d71/dbe/l100 0 2024-02-19T19:06:31.227 INFO:tasks.workunit.client.0.smithi088.stdout:8/632: fdatasync d9/df/d3d/d50/f52 0 2024-02-19T19:06:31.229 INFO:tasks.workunit.client.0.smithi088.stdout:7/762: stat d14/f1f 0 2024-02-19T19:06:31.230 INFO:tasks.workunit.client.0.smithi088.stdout:2/651: stat d3/d1b/d43/d61/d64/c96 0 2024-02-19T19:06:31.232 INFO:tasks.workunit.client.0.smithi088.stdout:0/731: mkdir d13/d62/d8a/dfe 0 2024-02-19T19:06:31.234 INFO:tasks.workunit.client.0.smithi088.stdout:4/571: rename d2/d8/l57 to d2/d32/d25/d4b/lb9 0 2024-02-19T19:06:31.238 INFO:tasks.workunit.client.0.smithi088.stdout:8/633: symlink d9/df/d1c/d22/lc1 0 2024-02-19T19:06:31.239 INFO:tasks.workunit.client.0.smithi088.stdout:2/652: rmdir d3/d1b/d43/d61/d64 39 2024-02-19T19:06:31.242 INFO:tasks.workunit.client.0.smithi088.stdout:1/792: rename d6/d21/d6a/d7d/da8 to d6/d21/d64/dc8/dcc/d101 0 2024-02-19T19:06:31.245 INFO:tasks.workunit.client.0.smithi088.stdout:4/572: unlink d2/d8/c5c 0 2024-02-19T19:06:31.247 INFO:tasks.workunit.client.0.smithi088.stdout:6/843: dread d5/d8/d3a/f46 [4194304,4194304] 0 2024-02-19T19:06:31.249 INFO:tasks.workunit.client.0.smithi088.stdout:6/844: chown d5/d8/d3a/c6b 24 1 2024-02-19T19:06:31.250 INFO:tasks.workunit.client.0.smithi088.stdout:2/653: creat d3/d14/d93/fcf x:0 0 0 2024-02-19T19:06:31.251 INFO:tasks.workunit.client.0.smithi088.stdout:0/732: creat d13/d62/d8a/d95/d5b/d76/dde/fff x:0 0 0 2024-02-19T19:06:31.256 INFO:tasks.workunit.client.0.smithi088.stdout:1/793: unlink f5 0 2024-02-19T19:06:31.258 INFO:tasks.workunit.client.0.smithi088.stdout:3/792: dread d4/d12/f6c [0,4194304] 0 2024-02-19T19:06:31.260 INFO:tasks.workunit.client.0.smithi088.stdout:3/793: fdatasync d4/d22/d7a/ff0 0 2024-02-19T19:06:31.262 INFO:tasks.workunit.client.0.smithi088.stdout:8/634: creat d9/df/d3d/d50/d71/da1/d6c/d77/d97/fc2 x:0 0 0 2024-02-19T19:06:31.262 INFO:tasks.workunit.client.0.smithi088.stdout:0/733: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d2f/f4c [8388608,4194304] 0 2024-02-19T19:06:31.263 INFO:tasks.workunit.client.0.smithi088.stdout:6/845: symlink d5/de/d64/d8d/d9e/l10c 0 2024-02-19T19:06:31.264 INFO:tasks.workunit.client.0.smithi088.stdout:4/573: dread d2/d32/d25/d26/d6d/d7e/d90/f94 [0,4194304] 0 2024-02-19T19:06:31.265 INFO:tasks.workunit.client.0.smithi088.stdout:2/654: creat d3/d1b/d43/d8f/fd0 x:0 0 0 2024-02-19T19:06:31.267 INFO:tasks.workunit.client.0.smithi088.stdout:4/574: stat d2/d8/f29 0 2024-02-19T19:06:31.267 INFO:tasks.workunit.client.0.smithi088.stdout:7/763: truncate d14/f2a 6591380 0 2024-02-19T19:06:31.267 INFO:tasks.workunit.client.0.smithi088.stdout:5/945: getdents d4/d61/d65/d9a/d9f/d57 0 2024-02-19T19:06:31.269 INFO:tasks.workunit.client.0.smithi088.stdout:1/794: rename d6/d21/d64/d94/cdb to d6/d21/d64/dc8/dcc/d101/c102 0 2024-02-19T19:06:31.270 INFO:tasks.workunit.client.0.smithi088.stdout:3/794: mkdir d4/d22/d7a/d11a 0 2024-02-19T19:06:31.273 INFO:tasks.workunit.client.0.smithi088.stdout:8/635: unlink d9/df/d3d/d50/d71/da1/f4b 0 2024-02-19T19:06:31.275 INFO:tasks.workunit.client.0.smithi088.stdout:2/655: symlink d3/d14/ld1 0 2024-02-19T19:06:31.276 INFO:tasks.workunit.client.0.smithi088.stdout:2/656: chown d3/d1b/d43/d8f/f8e 821634516 1 2024-02-19T19:06:31.277 INFO:tasks.workunit.client.0.smithi088.stdout:0/734: mkdir d13/d62/d8a/d95/d5a/d25/d2a/d34/d100 0 2024-02-19T19:06:31.277 INFO:tasks.workunit.client.0.smithi088.stdout:2/657: chown d3/d14/f1d 0 1 2024-02-19T19:06:31.283 INFO:tasks.workunit.client.0.smithi088.stdout:6/846: write d5/de/d60/f87 [2273575,35722] 0 2024-02-19T19:06:31.285 INFO:tasks.workunit.client.0.smithi088.stdout:6/847: read d5/de/f10 [4166477,101951] 0 2024-02-19T19:06:31.286 INFO:tasks.workunit.client.0.smithi088.stdout:8/636: unlink d9/df/d3d/d50/f52 0 2024-02-19T19:06:31.288 INFO:tasks.workunit.client.0.smithi088.stdout:7/764: dread d14/d28/d3b/f64 [0,4194304] 0 2024-02-19T19:06:31.288 INFO:tasks.workunit.client.0.smithi088.stdout:3/795: rename d4/d22/d81 to d4/d22/d7a/d11a/d11b 0 2024-02-19T19:06:31.290 INFO:tasks.workunit.client.0.smithi088.stdout:0/735: chown d13/d62/d8a/d9c/da8/de2/fa0 10378569 1 2024-02-19T19:06:31.295 INFO:tasks.workunit.client.0.smithi088.stdout:1/795: dread d6/d21/d6a/d4a/fb1 [0,4194304] 0 2024-02-19T19:06:31.296 INFO:tasks.workunit.client.0.smithi088.stdout:4/575: creat d2/d32/d25/d26/d6d/d7e/da3/d9f/fba x:0 0 0 2024-02-19T19:06:31.296 INFO:tasks.workunit.client.0.smithi088.stdout:0/736: dread d13/d62/d8a/d95/d5a/d25/d2a/d34/f75 [0,4194304] 0 2024-02-19T19:06:31.296 INFO:tasks.workunit.client.0.smithi088.stdout:0/737: stat d13/d62/d8a/d95/d5a/d25/f8b 0 2024-02-19T19:06:31.296 INFO:tasks.workunit.client.0.smithi088.stdout:5/946: symlink d4/d61/d65/d9a/d9f/d57/d84/dfb/d100/l150 0 2024-02-19T19:06:31.297 INFO:tasks.workunit.client.0.smithi088.stdout:6/848: mknod d5/d45/d9d/c10d 0 2024-02-19T19:06:31.299 INFO:tasks.workunit.client.0.smithi088.stdout:7/765: creat d14/d28/d94/ddb/f113 x:0 0 0 2024-02-19T19:06:31.300 INFO:tasks.workunit.client.0.smithi088.stdout:3/796: rmdir d4/d87/dbf 39 2024-02-19T19:06:31.302 INFO:tasks.workunit.client.0.smithi088.stdout:3/797: write d4/d1a/d1e/d31/d32/f102 [325657,66023] 0 2024-02-19T19:06:31.302 INFO:tasks.workunit.client.0.smithi088.stdout:1/796: mknod d6/d10/d60/db8/dcf/c103 0 2024-02-19T19:06:31.303 INFO:tasks.workunit.client.0.smithi088.stdout:3/798: readlink d4/d12/d33/d43/l9a 0 2024-02-19T19:06:31.304 INFO:tasks.workunit.client.0.smithi088.stdout:0/738: fdatasync d13/d62/d8a/fbb 0 2024-02-19T19:06:31.305 INFO:tasks.workunit.client.0.smithi088.stdout:0/739: fdatasync d13/f1c 0 2024-02-19T19:06:31.306 INFO:tasks.workunit.client.0.smithi088.stdout:5/947: read d4/d7/f64 [3014534,61300] 0 2024-02-19T19:06:31.307 INFO:tasks.workunit.client.0.smithi088.stdout:6/849: rename d5/d8/d3a/d71/daa/dc2/f99 to d5/d8/d3a/d71/daa/f10e 0 2024-02-19T19:06:31.310 INFO:tasks.workunit.client.0.smithi088.stdout:7/766: creat d14/d1d/d62/dcf/f114 x:0 0 0 2024-02-19T19:06:31.311 INFO:tasks.workunit.client.0.smithi088.stdout:3/799: dwrite d4/d1a/d1e/d31/d32/ffa [0,4194304] 0 2024-02-19T19:06:31.318 INFO:tasks.workunit.client.0.smithi088.stdout:5/948: symlink d4/d1a/d41/de2/de7/dff/d107/l151 0 2024-02-19T19:06:31.319 INFO:tasks.workunit.client.0.smithi088.stdout:5/949: stat d4/d1a/fd1 0 2024-02-19T19:06:31.321 INFO:tasks.workunit.client.0.smithi088.stdout:2/658: truncate d3/d1b/d43/d61/d86/faa 143203 0 2024-02-19T19:06:31.321 INFO:tasks.workunit.client.0.smithi088.stdout:1/797: mknod d6/d21/d64/d94/dab/dfd/c104 0 2024-02-19T19:06:31.323 INFO:tasks.workunit.client.0.smithi088.stdout:0/740: mknod d13/d81/c101 0 2024-02-19T19:06:31.325 INFO:tasks.workunit.client.0.smithi088.stdout:8/637: link d9/df/d3d/d50/d71/da1/d6c/ca2 d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/cc3 0 2024-02-19T19:06:31.328 INFO:tasks.workunit.client.0.smithi088.stdout:5/950: unlink d4/d1a/d41/de2/de7/f109 0 2024-02-19T19:06:31.329 INFO:tasks.workunit.client.0.smithi088.stdout:2/659: mknod d3/d1f/d5d/cd2 0 2024-02-19T19:06:31.331 INFO:tasks.workunit.client.0.smithi088.stdout:0/741: rename d13/d62/d8a/d95/d5a/d25/d2a/c30 to d13/d62/d8a/d95/d5b/dbe/dc6/c102 0 2024-02-19T19:06:31.334 INFO:tasks.workunit.client.0.smithi088.stdout:8/638: dwrite d9/df/d3d/d50/d71/da1/d70/fa7 [4194304,4194304] 0 2024-02-19T19:06:31.336 INFO:tasks.workunit.client.0.smithi088.stdout:7/767: dread d14/f3e [4194304,4194304] 0 2024-02-19T19:06:31.337 INFO:tasks.workunit.client.0.smithi088.stdout:3/800: rename d4/d22/d7a/d11a/d11b/lf5 to d4/d1a/d1e/d31/dc5/l11c 0 2024-02-19T19:06:31.339 INFO:tasks.workunit.client.0.smithi088.stdout:8/639: mkdir d9/dc4 0 2024-02-19T19:06:31.342 INFO:tasks.workunit.client.0.smithi088.stdout:5/951: dwrite d4/d1a/d41/fea [0,4194304] 0 2024-02-19T19:06:31.346 INFO:tasks.workunit.client.0.smithi088.stdout:3/801: mkdir d4/d12/d33/d43/d11d 0 2024-02-19T19:06:31.346 INFO:tasks.workunit.client.0.smithi088.stdout:3/802: chown d4/d1a/d1e/d31/dc5 4 1 2024-02-19T19:06:31.350 INFO:tasks.workunit.client.0.smithi088.stdout:8/640: mkdir d9/df/d1c/d85/dc5 0 2024-02-19T19:06:31.351 INFO:tasks.workunit.client.0.smithi088.stdout:1/798: write d6/d21/f3a [187241,2885] 0 2024-02-19T19:06:31.352 INFO:tasks.workunit.client.0.smithi088.stdout:0/742: creat d13/d62/d8a/d95/d5a/d25/d2a/d3c/d86/de5/f103 x:0 0 0 2024-02-19T19:06:31.353 INFO:tasks.workunit.client.0.smithi088.stdout:0/743: readlink d13/d62/d8a/d95/d96/ded/lf2 0 2024-02-19T19:06:31.354 INFO:tasks.workunit.client.0.smithi088.stdout:0/744: chown d13/d81/cb1 4129004 1 2024-02-19T19:06:31.355 INFO:tasks.workunit.client.0.smithi088.stdout:8/641: truncate d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/fbf 726466 0 2024-02-19T19:06:31.358 INFO:tasks.workunit.client.0.smithi088.stdout:1/799: write d6/d10/d60/d29/f2e [4988587,83456] 0 2024-02-19T19:06:31.360 INFO:tasks.workunit.client.0.smithi088.stdout:5/952: creat d4/d61/f152 x:0 0 0 2024-02-19T19:06:31.365 INFO:tasks.workunit.client.0.smithi088.stdout:7/768: dwrite d14/d1d/d3c/d43/d5c/d75/fad [0,4194304] 0 2024-02-19T19:06:31.369 INFO:tasks.workunit.client.0.smithi088.stdout:7/769: dread d14/d1d/d62/dcf/fe4 [0,4194304] 0 2024-02-19T19:06:31.370 INFO:tasks.workunit.client.0.smithi088.stdout:5/953: creat d4/d7/f153 x:0 0 0 2024-02-19T19:06:31.371 INFO:tasks.workunit.client.0.smithi088.stdout:0/745: write d13/d62/d8a/d95/d5a/d25/d2a/d2f/f41 [561130,73041] 0 2024-02-19T19:06:31.373 INFO:tasks.workunit.client.0.smithi088.stdout:1/800: write d6/d21/d6a/d7d/d7e/fac [1342692,114354] 0 2024-02-19T19:06:31.375 INFO:tasks.workunit.client.0.smithi088.stdout:8/642: truncate d9/df/d1c/d22/d9e/faa 3523139 0 2024-02-19T19:06:31.377 INFO:tasks.workunit.client.0.smithi088.stdout:3/803: dread d4/d1a/d7f/d88/fcd [0,4194304] 0 2024-02-19T19:06:31.379 INFO:tasks.workunit.client.0.smithi088.stdout:5/954: rename d4/d61/d65/d9a/d9f/d57/d84/dfb to d4/d1a/d41/d154 0 2024-02-19T19:06:31.381 INFO:tasks.workunit.client.0.smithi088.stdout:0/746: creat d13/d62/d8a/d95/d5a/d25/d2a/d2f/f104 x:0 0 0 2024-02-19T19:06:31.381 INFO:tasks.workunit.client.0.smithi088.stdout:1/801: truncate d6/d10/d60/d29/fb9 1430698 0 2024-02-19T19:06:31.382 INFO:tasks.workunit.client.0.smithi088.stdout:8/643: dread d9/df/d3d/d50/d71/da1/d6c/d9a/fbb [0,4194304] 0 2024-02-19T19:06:31.383 INFO:tasks.workunit.client.0.smithi088.stdout:5/955: write d4/d1a/d41/de2/de7/dff/d107/f122 [270158,71010] 0 2024-02-19T19:06:31.385 INFO:tasks.workunit.client.0.smithi088.stdout:1/802: creat d6/d21/d6a/dcb/f105 x:0 0 0 2024-02-19T19:06:31.385 INFO:tasks.workunit.client.0.smithi088.stdout:0/747: truncate d13/d62/d8a/d9c/fac 109716 0 2024-02-19T19:06:31.387 INFO:tasks.workunit.client.0.smithi088.stdout:1/803: dread - d6/d10/f73 zero size 2024-02-19T19:06:31.387 INFO:tasks.workunit.client.0.smithi088.stdout:5/956: readlink d4/d61/d65/d9a/d9f/d57/d84/lc8 0 2024-02-19T19:06:31.388 INFO:tasks.workunit.client.0.smithi088.stdout:8/644: rename d9 to d9/df/d3d/d50/d71/da1/d6c/d77/dc6 22 2024-02-19T19:06:31.389 INFO:tasks.workunit.client.0.smithi088.stdout:1/804: chown d6/d21/d6a/d7d/cd7 5 1 2024-02-19T19:06:31.390 INFO:tasks.workunit.client.0.smithi088.stdout:8/645: chown d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/db3 9 1 2024-02-19T19:06:31.394 INFO:tasks.workunit.client.0.smithi088.stdout:7/770: dread d14/f29 [0,4194304] 0 2024-02-19T19:06:31.397 INFO:tasks.workunit.client.0.smithi088.stdout:3/804: dread d4/d12/f21 [0,4194304] 0 2024-02-19T19:06:31.397 INFO:tasks.workunit.client.0.smithi088.stdout:7/771: mkdir d14/d28/d94/ddb/d115 0 2024-02-19T19:06:31.397 INFO:tasks.workunit.client.0.smithi088.stdout:1/805: dread d6/d21/d64/d94/d6e/f88 [0,4194304] 0 2024-02-19T19:06:31.399 INFO:tasks.workunit.client.0.smithi088.stdout:3/805: read - d4/d22/d7a/ff0 zero size 2024-02-19T19:06:31.399 INFO:tasks.workunit.client.0.smithi088.stdout:8/646: rename d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/f92 to d9/df/d3d/d50/d71/da1/d6c/d77/d97/fc7 0 2024-02-19T19:06:31.405 INFO:tasks.workunit.client.0.smithi088.stdout:7/772: dwrite d14/d28/d9c/fc0 [0,4194304] 0 2024-02-19T19:06:31.406 INFO:tasks.workunit.client.0.smithi088.stdout:0/748: truncate d13/de9/ffd 7283278 0 2024-02-19T19:06:31.461 INFO:tasks.workunit.client.0.smithi088.stdout:0/749: mkdir d13/d62/d8a/d95/d5a/d25/dd6/d105 0 2024-02-19T19:06:31.462 INFO:tasks.workunit.client.0.smithi088.stdout:3/806: getdents d4/d1a/d1e/d74/d7d/daa/dcb 0 2024-02-19T19:06:31.463 INFO:tasks.workunit.client.0.smithi088.stdout:8/647: dwrite d9/df/d3d/d50/d71/da1/d6c/d9a/f4d [0,4194304] 0 2024-02-19T19:06:31.476 INFO:tasks.workunit.client.0.smithi088.stdout:1/806: dwrite d6/d21/d6a/d7d/d7e/ff5 [0,4194304] 0 2024-02-19T19:06:31.480 INFO:tasks.workunit.client.0.smithi088.stdout:5/957: dwrite d4/d1a/f6e [0,4194304] 0 2024-02-19T19:06:31.480 INFO:tasks.workunit.client.0.smithi088.stdout:7/773: dread d14/d1d/d3c/f63 [0,4194304] 0 2024-02-19T19:06:31.482 INFO:tasks.workunit.client.0.smithi088.stdout:5/958: stat d4/d61/d65/d9a/d9f/f4e 0 2024-02-19T19:06:31.488 INFO:tasks.workunit.client.0.smithi088.stdout:0/750: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d2f/f104 [0,4194304] 0 2024-02-19T19:06:31.489 INFO:tasks.workunit.client.0.smithi088.stdout:0/751: chown f4 193088 1 2024-02-19T19:06:31.514 INFO:tasks.workunit.client.0.smithi088.stdout:8/648: dwrite d9/df/d3d/d50/d71/da1/f54 [4194304,4194304] 0 2024-02-19T19:06:31.515 INFO:tasks.workunit.client.0.smithi088.stdout:0/752: dread d13/f1c [0,4194304] 0 2024-02-19T19:06:31.516 INFO:tasks.workunit.client.0.smithi088.stdout:7/774: rmdir d14/d1d 39 2024-02-19T19:06:31.519 INFO:tasks.workunit.client.0.smithi088.stdout:5/959: getdents d4/d1a/d41/d154/d13b 0 2024-02-19T19:06:31.522 INFO:tasks.workunit.client.0.smithi088.stdout:8/649: mknod d9/df/d3d/d62/cc8 0 2024-02-19T19:06:31.525 INFO:tasks.workunit.client.0.smithi088.stdout:1/807: getdents d6/d21/d6a/d4a/dad 0 2024-02-19T19:06:31.528 INFO:tasks.workunit.client.0.smithi088.stdout:5/960: dwrite d4/d61/d65/d9a/d9f/d57/d84/f12b [0,4194304] 0 2024-02-19T19:06:31.531 INFO:tasks.workunit.client.0.smithi088.stdout:8/650: rmdir d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79 39 2024-02-19T19:06:31.533 INFO:tasks.workunit.client.0.smithi088.stdout:7/775: dwrite d14/d1d/f38 [0,4194304] 0 2024-02-19T19:06:31.534 INFO:tasks.workunit.client.0.smithi088.stdout:7/776: dread - d14/d28/d3b/dd3/f10e zero size 2024-02-19T19:06:31.538 INFO:tasks.workunit.client.0.smithi088.stdout:0/753: getdents d13/d62/d8a/d95/d5a/d25 0 2024-02-19T19:06:31.543 INFO:tasks.workunit.client.0.smithi088.stdout:1/808: link d6/d21/d6a/f68 d6/d21/d6a/d4a/dad/f106 0 2024-02-19T19:06:31.546 INFO:tasks.workunit.client.0.smithi088.stdout:0/754: rename d13/d62/d8a/d95/d5a/d25/d2a/d3c/d86 to d13/d62/d8a/d95/d5a/d25/dd6/df9/d106 0 2024-02-19T19:06:31.554 INFO:tasks.workunit.client.0.smithi088.stdout:8/651: write d9/df/f27 [9227016,4999] 0 2024-02-19T19:06:31.556 INFO:tasks.workunit.client.0.smithi088.stdout:7/777: write d14/f77 [455805,114605] 0 2024-02-19T19:06:31.558 INFO:tasks.workunit.client.0.smithi088.stdout:3/807: dwrite d4/fd [0,4194304] 0 2024-02-19T19:06:31.560 INFO:tasks.workunit.client.0.smithi088.stdout:7/778: write d14/f6e [1505017,52471] 0 2024-02-19T19:06:31.565 INFO:tasks.workunit.client.0.smithi088.stdout:7/779: fdatasync d14/f44 0 2024-02-19T19:06:31.566 INFO:tasks.workunit.client.0.smithi088.stdout:7/780: dread - d14/d28/d3b/dd3/fe1 zero size 2024-02-19T19:06:31.567 INFO:tasks.workunit.client.0.smithi088.stdout:8/652: truncate d9/df/d1c/f33 1698379 0 2024-02-19T19:06:31.569 INFO:tasks.workunit.client.0.smithi088.stdout:3/808: write d4/d12/d33/dab/faf [101015,119693] 0 2024-02-19T19:06:31.572 INFO:tasks.workunit.client.0.smithi088.stdout:3/809: mknod d4/d1a/d1e/d74/d7d/c11e 0 2024-02-19T19:06:31.574 INFO:tasks.workunit.client.0.smithi088.stdout:8/653: write d9/df/f36 [2900422,111017] 0 2024-02-19T19:06:31.575 INFO:tasks.workunit.client.0.smithi088.stdout:8/654: chown d9/c41 76549775 1 2024-02-19T19:06:31.576 INFO:tasks.workunit.client.0.smithi088.stdout:3/810: mkdir d4/d22/d7a/da1/d10d/d11f 0 2024-02-19T19:06:31.579 INFO:tasks.workunit.client.0.smithi088.stdout:3/811: dread - d4/d22/d7a/fc1 zero size 2024-02-19T19:06:31.579 INFO:tasks.workunit.client.0.smithi088.stdout:7/781: dwrite d14/f60 [0,4194304] 0 2024-02-19T19:06:31.582 INFO:tasks.workunit.client.0.smithi088.stdout:3/812: mknod d4/d117/c120 0 2024-02-19T19:06:31.587 INFO:tasks.workunit.client.0.smithi088.stdout:3/813: getdents d4/d12/d33/d43/d85/d10a 0 2024-02-19T19:06:31.588 INFO:tasks.workunit.client.0.smithi088.stdout:3/814: mknod d4/d12/c121 0 2024-02-19T19:06:31.589 INFO:tasks.workunit.client.0.smithi088.stdout:7/782: dwrite d14/f19 [0,4194304] 0 2024-02-19T19:06:31.589 INFO:tasks.workunit.client.0.smithi088.stdout:3/815: chown d4/d1a/d1e/d31/d32/c3a 40087628 1 2024-02-19T19:06:31.593 INFO:tasks.workunit.client.0.smithi088.stdout:8/655: dread d9/df/d3d/f9f [0,4194304] 0 2024-02-19T19:06:31.596 INFO:tasks.workunit.client.0.smithi088.stdout:7/783: link d14/f1c d14/d1d/d3c/d43/d5c/f116 0 2024-02-19T19:06:31.599 INFO:tasks.workunit.client.0.smithi088.stdout:7/784: fsync d14/d28/d65/fa8 0 2024-02-19T19:06:31.603 INFO:tasks.workunit.client.0.smithi088.stdout:8/656: dwrite d9/df/d3d/d50/d71/da1/d6c/fa0 [0,4194304] 0 2024-02-19T19:06:31.604 INFO:tasks.workunit.client.0.smithi088.stdout:3/816: dwrite d4/d22/fef [0,4194304] 0 2024-02-19T19:06:31.604 INFO:tasks.workunit.client.0.smithi088.stdout:7/785: write d14/d28/fe0 [4816174,128098] 0 2024-02-19T19:06:31.610 INFO:tasks.workunit.client.0.smithi088.stdout:3/817: creat d4/d22/d7a/de7/f122 x:0 0 0 2024-02-19T19:06:31.611 INFO:tasks.workunit.client.0.smithi088.stdout:8/657: dread d9/df/f55 [0,4194304] 0 2024-02-19T19:06:31.613 INFO:tasks.workunit.client.0.smithi088.stdout:3/818: dread d4/d22/d7a/d8d/f9d [0,4194304] 0 2024-02-19T19:06:31.614 INFO:tasks.workunit.client.0.smithi088.stdout:8/658: creat d9/fc9 x:0 0 0 2024-02-19T19:06:31.616 INFO:tasks.workunit.client.0.smithi088.stdout:3/819: read d4/d1a/d1e/d31/dc5/dcf/da4/fbb [406801,125527] 0 2024-02-19T19:06:31.619 INFO:tasks.workunit.client.0.smithi088.stdout:3/820: stat d4/c1b 0 2024-02-19T19:06:31.621 INFO:tasks.workunit.client.0.smithi088.stdout:8/659: write d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/f8a [354147,106657] 0 2024-02-19T19:06:31.622 INFO:tasks.workunit.client.0.smithi088.stdout:3/821: truncate d4/d22/d7a/d8d/fb9 951128 0 2024-02-19T19:06:31.623 INFO:tasks.workunit.client.0.smithi088.stdout:8/660: chown d9/df/d3d/d50/d71/da1/d6c/d77/cbd 33909 1 2024-02-19T19:06:31.623 INFO:tasks.workunit.client.0.smithi088.stdout:3/822: chown d4/d12/d33/d43/ddd/c105 2007385 1 2024-02-19T19:06:31.625 INFO:tasks.workunit.client.0.smithi088.stdout:3/823: write d4/d12/d33/dab/ff4 [494907,4348] 0 2024-02-19T19:06:31.627 INFO:tasks.workunit.client.0.smithi088.stdout:8/661: creat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8/fca x:0 0 0 2024-02-19T19:06:31.628 INFO:tasks.workunit.client.0.smithi088.stdout:3/824: symlink d4/l123 0 2024-02-19T19:06:31.630 INFO:tasks.workunit.client.0.smithi088.stdout:8/662: creat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/fcb x:0 0 0 2024-02-19T19:06:31.634 INFO:tasks.workunit.client.0.smithi088.stdout:8/663: unlink d9/df/d3d/d62/fb1 0 2024-02-19T19:06:31.636 INFO:tasks.workunit.client.0.smithi088.stdout:7/786: dread fb [4194304,4194304] 0 2024-02-19T19:06:31.639 INFO:tasks.workunit.client.0.smithi088.stdout:7/787: creat d14/d27/d49/f117 x:0 0 0 2024-02-19T19:06:31.642 INFO:tasks.workunit.client.0.smithi088.stdout:8/664: truncate d9/df/d3d/d50/d71/da1/f54 1463066 0 2024-02-19T19:06:31.643 INFO:tasks.workunit.client.0.smithi088.stdout:8/665: fdatasync d9/df/d3d/d50/d71/da1/d70/fa7 0 2024-02-19T19:06:31.644 INFO:tasks.workunit.client.0.smithi088.stdout:7/788: dwrite d14/d1d/d3c/f8f [0,4194304] 0 2024-02-19T19:06:31.645 INFO:tasks.workunit.client.0.smithi088.stdout:8/666: fdatasync d9/df/f36 0 2024-02-19T19:06:31.652 INFO:tasks.workunit.client.0.smithi088.stdout:7/789: dread d14/d1d/d3c/d43/d5c/d9a/d92/fab [0,4194304] 0 2024-02-19T19:06:31.653 INFO:tasks.workunit.client.0.smithi088.stdout:8/667: dwrite d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/f8a [0,4194304] 0 2024-02-19T19:06:31.654 INFO:tasks.workunit.client.0.smithi088.stdout:7/790: rmdir d14/d1d/d3c/d43/db2 39 2024-02-19T19:06:31.655 INFO:tasks.workunit.client.0.smithi088.stdout:8/668: fsync d9/df/d3d/d50/d71/da1/d70/fa7 0 2024-02-19T19:06:31.657 INFO:tasks.workunit.client.0.smithi088.stdout:7/791: creat d14/d28/d94/ddb/f118 x:0 0 0 2024-02-19T19:06:31.660 INFO:tasks.workunit.client.0.smithi088.stdout:7/792: rmdir d14/d1d/d3c/d43/d5c/d9a/d92 39 2024-02-19T19:06:31.661 INFO:tasks.workunit.client.0.smithi088.stdout:8/669: dread d9/df/d3d/d50/d71/da1/d6c/fa0 [0,4194304] 0 2024-02-19T19:06:31.664 INFO:tasks.workunit.client.0.smithi088.stdout:8/670: mknod d9/df/d3d/d50/ccc 0 2024-02-19T19:06:31.665 INFO:tasks.workunit.client.0.smithi088.stdout:7/793: truncate d14/d28/f35 1576599 0 2024-02-19T19:06:31.667 INFO:tasks.workunit.client.0.smithi088.stdout:7/794: creat d14/d28/d3b/f119 x:0 0 0 2024-02-19T19:06:31.669 INFO:tasks.workunit.client.0.smithi088.stdout:7/795: rename d14/d27/l50 to d14/d1d/d3c/d43/d5c/d75/l11a 0 2024-02-19T19:06:31.671 INFO:tasks.workunit.client.0.smithi088.stdout:7/796: write d14/f30 [2349338,127499] 0 2024-02-19T19:06:31.684 INFO:tasks.workunit.client.0.smithi088.stdout:8/671: dread d9/df/d1c/f1f [0,4194304] 0 2024-02-19T19:06:31.689 INFO:tasks.workunit.client.0.smithi088.stdout:8/672: dwrite d9/df/d1c/d85/f8d [0,4194304] 0 2024-02-19T19:06:31.693 INFO:tasks.workunit.client.0.smithi088.stdout:8/673: creat d9/df/d3d/fcd x:0 0 0 2024-02-19T19:06:31.695 INFO:tasks.workunit.client.0.smithi088.stdout:8/674: rmdir d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79 39 2024-02-19T19:06:31.696 INFO:tasks.workunit.client.0.smithi088.stdout:8/675: stat d9/df/d3d/d50/d71/da1/d6c/d9a/l53 0 2024-02-19T19:06:31.710 INFO:tasks.workunit.client.0.smithi088.stdout:8/676: dread d9/df/d1c/f33 [0,4194304] 0 2024-02-19T19:06:31.712 INFO:tasks.workunit.client.0.smithi088.stdout:8/677: rmdir d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79 39 2024-02-19T19:06:31.715 INFO:tasks.workunit.client.0.smithi088.stdout:8/678: unlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/fbf 0 2024-02-19T19:06:31.766 INFO:tasks.workunit.client.0.smithi088.stdout:8/679: mknod d9/df/d3d/d62/cce 0 2024-02-19T19:06:31.768 INFO:tasks.workunit.client.0.smithi088.stdout:8/680: write d9/df/d3d/d50/d71/da1/d6c/d9a/f44 [2058063,33080] 0 2024-02-19T19:06:31.771 INFO:tasks.workunit.client.0.smithi088.stdout:8/681: write d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/f7b [4730289,88356] 0 2024-02-19T19:06:31.775 INFO:tasks.workunit.client.0.smithi088.stdout:8/682: write d9/df/d3d/f5f [5365746,122941] 0 2024-02-19T19:06:31.777 INFO:tasks.workunit.client.0.smithi088.stdout:8/683: mkdir d9/df/d3d/d50/d71/da1/d6c/dcf 0 2024-02-19T19:06:31.778 INFO:tasks.workunit.client.0.smithi088.stdout:8/684: fdatasync d9/f63 0 2024-02-19T19:06:31.796 INFO:tasks.workunit.client.0.smithi088.stdout:8/685: dread d9/df/d3d/d50/d71/da1/d6c/d9a/f5e [0,4194304] 0 2024-02-19T19:06:31.796 INFO:tasks.workunit.client.0.smithi088.stdout:8/686: symlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/ld0 0 2024-02-19T19:06:31.802 INFO:tasks.workunit.client.0.smithi088.stdout:8/687: dwrite f6 [0,4194304] 0 2024-02-19T19:06:31.803 INFO:tasks.workunit.client.0.smithi088.stdout:8/688: fsync d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8/fb5 0 2024-02-19T19:06:31.806 INFO:tasks.workunit.client.0.smithi088.stdout:8/689: mknod d9/df/cd1 0 2024-02-19T19:06:31.806 INFO:tasks.workunit.client.0.smithi088.stdout:8/690: read - d9/df/d1c/d22/f98 zero size 2024-02-19T19:06:31.807 INFO:tasks.workunit.client.0.smithi088.stdout:8/691: dread - d9/df/d3d/d50/d71/da1/d70/fad zero size 2024-02-19T19:06:31.814 INFO:tasks.workunit.client.0.smithi088.stdout:8/692: dwrite d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8/fb5 [0,4194304] 0 2024-02-19T19:06:31.821 INFO:tasks.workunit.client.0.smithi088.stdout:8/693: dwrite d9/df/d3d/f5f [0,4194304] 0 2024-02-19T19:06:31.822 INFO:tasks.workunit.client.0.smithi088.stdout:8/694: fsync d9/df/d3d/d50/d71/da1/d70/fad 0 2024-02-19T19:06:31.823 INFO:tasks.workunit.client.0.smithi088.stdout:8/695: readlink d9/df/d3d/l88 0 2024-02-19T19:06:31.824 INFO:tasks.workunit.client.0.smithi088.stdout:8/696: write d9/df/d1c/d85/f8d [4060319,35342] 0 2024-02-19T19:06:31.827 INFO:tasks.workunit.client.0.smithi088.stdout:8/697: creat d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/fd2 x:0 0 0 2024-02-19T19:06:31.830 INFO:tasks.workunit.client.0.smithi088.stdout:8/698: symlink d9/df/d3d/d50/d71/da1/d6c/d77/d97/ld3 0 2024-02-19T19:06:31.834 INFO:tasks.workunit.client.0.smithi088.stdout:8/699: getdents d9/df 0 2024-02-19T19:06:31.840 INFO:tasks.workunit.client.0.smithi088.stdout:8/700: dwrite d9/df/f36 [4194304,4194304] 0 2024-02-19T19:06:31.994 INFO:tasks.workunit.client.0.smithi088.stdout:4/576: sync 2024-02-19T19:06:31.995 INFO:tasks.workunit.client.0.smithi088.stdout:4/577: chown d2/d32/d25/d45/l65 32591859 1 2024-02-19T19:06:32.000 INFO:tasks.workunit.client.0.smithi088.stdout:4/578: creat d2/d32/fbb x:0 0 0 2024-02-19T19:06:32.000 INFO:tasks.workunit.client.0.smithi088.stdout:4/579: link d2/d32/l7d d2/d32/d25/d26/d6d/d7e/da3/d9f/lbc 0 2024-02-19T19:06:32.004 INFO:tasks.workunit.client.0.smithi088.stdout:4/580: write d2/d32/d25/d3f/d70/f8c [414952,83162] 0 2024-02-19T19:06:32.008 INFO:tasks.workunit.client.0.smithi088.stdout:4/581: write d2/d32/d25/d26/d6d/d7e/d3a/f42 [836147,6205] 0 2024-02-19T19:06:32.016 INFO:tasks.workunit.client.0.smithi088.stdout:4/582: dwrite d2/d8/f14 [0,4194304] 0 2024-02-19T19:06:32.022 INFO:tasks.workunit.client.0.smithi088.stdout:4/583: dwrite d2/d32/d25/d26/d6d/d7e/d90/fa7 [0,4194304] 0 2024-02-19T19:06:32.024 INFO:tasks.workunit.client.0.smithi088.stdout:4/584: symlink d2/d32/d25/d26/da0/lbd 0 2024-02-19T19:06:32.030 INFO:tasks.workunit.client.0.smithi088.stdout:4/585: rmdir d2/d32/d25 39 2024-02-19T19:06:32.033 INFO:tasks.workunit.client.0.smithi088.stdout:4/586: symlink d2/d32/d25/d26/lbe 0 2024-02-19T19:06:32.034 INFO:tasks.workunit.client.0.smithi088.stdout:4/587: dread - d2/d32/d25/d26/f62 zero size 2024-02-19T19:06:32.034 INFO:tasks.workunit.client.0.smithi088.stdout:4/588: chown d2/d32/f44 14572693 1 2024-02-19T19:06:32.035 INFO:tasks.workunit.client.0.smithi088.stdout:4/589: dread - d2/d32/d25/d26/d4e/d6f/fa6 zero size 2024-02-19T19:06:32.039 INFO:tasks.workunit.client.0.smithi088.stdout:4/590: dwrite d2/f75 [0,4194304] 0 2024-02-19T19:06:32.045 INFO:tasks.workunit.client.0.smithi088.stdout:2/660: sync 2024-02-19T19:06:32.045 INFO:tasks.workunit.client.0.smithi088.stdout:1/809: sync 2024-02-19T19:06:32.045 INFO:tasks.workunit.client.0.smithi088.stdout:8/701: sync 2024-02-19T19:06:32.045 INFO:tasks.workunit.client.0.smithi088.stdout:3/825: sync 2024-02-19T19:06:32.046 INFO:tasks.workunit.client.0.smithi088.stdout:5/961: sync 2024-02-19T19:06:32.046 INFO:tasks.workunit.client.0.smithi088.stdout:7/797: sync 2024-02-19T19:06:32.046 INFO:tasks.workunit.client.0.smithi088.stdout:0/755: sync 2024-02-19T19:06:32.046 INFO:tasks.workunit.client.0.smithi088.stdout:6/850: sync 2024-02-19T19:06:32.048 INFO:tasks.workunit.client.0.smithi088.stdout:2/661: stat d3/c3c 0 2024-02-19T19:06:32.051 INFO:tasks.workunit.client.0.smithi088.stdout:7/798: mkdir d14/d27/d49/d98/dcb/de7/d11b 0 2024-02-19T19:06:32.054 INFO:tasks.workunit.client.0.smithi088.stdout:5/962: rename d4/d75/deb/f139 to d4/d1a/d41/dcb/f155 0 2024-02-19T19:06:32.055 INFO:tasks.workunit.client.0.smithi088.stdout:6/851: mkdir d5/de/d64/d8d/d9e/d10f 0 2024-02-19T19:06:32.055 INFO:tasks.workunit.client.0.smithi088.stdout:4/591: dwrite d2/d32/d25/d26/d6d/d7e/d3a/f74 [0,4194304] 0 2024-02-19T19:06:32.056 INFO:tasks.workunit.client.0.smithi088.stdout:7/799: write d14/d27/d49/f117 [135282,18097] 0 2024-02-19T19:06:32.060 INFO:tasks.workunit.client.0.smithi088.stdout:0/756: unlink d13/d62/d8a/d95/d5a/d25/d2a/d2f/f50 0 2024-02-19T19:06:32.061 INFO:tasks.workunit.client.0.smithi088.stdout:6/852: write d5/de/d64/d8d/fec [894373,91548] 0 2024-02-19T19:06:32.062 INFO:tasks.workunit.client.0.smithi088.stdout:2/662: truncate d3/d14/f19 1381483 0 2024-02-19T19:06:32.064 INFO:tasks.workunit.client.0.smithi088.stdout:2/663: chown d3/d1b/f42 5340 1 2024-02-19T19:06:32.065 INFO:tasks.workunit.client.0.smithi088.stdout:1/810: dwrite d6/d21/d6a/d4a/f59 [4194304,4194304] 0 2024-02-19T19:06:32.065 INFO:tasks.workunit.client.0.smithi088.stdout:3/826: mkdir d4/d22/d7a/da1/d10d/d11f/d124 0 2024-02-19T19:06:32.069 INFO:tasks.workunit.client.0.smithi088.stdout:5/963: mkdir d4/d61/d65/d156 0 2024-02-19T19:06:32.071 INFO:tasks.workunit.client.0.smithi088.stdout:0/757: write d13/d62/d8a/d9c/da8/dab/dd0/fdc [928770,49794] 0 2024-02-19T19:06:32.071 INFO:tasks.workunit.client.0.smithi088.stdout:6/853: mknod d5/d8/d3a/d71/daa/c110 0 2024-02-19T19:06:32.076 INFO:tasks.workunit.client.0.smithi088.stdout:2/664: symlink d3/d1b/d43/d8f/ld3 0 2024-02-19T19:06:32.077 INFO:tasks.workunit.client.0.smithi088.stdout:8/702: truncate d9/df/d1c/f1f 1571509 0 2024-02-19T19:06:32.078 INFO:tasks.workunit.client.0.smithi088.stdout:4/592: symlink d2/d35/db3/db7/lbf 0 2024-02-19T19:06:32.079 INFO:tasks.workunit.client.0.smithi088.stdout:3/827: dread - d4/d1a/d1e/d74/d7d/daa/f90 zero size 2024-02-19T19:06:32.080 INFO:tasks.workunit.client.0.smithi088.stdout:8/703: write d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/fd2 [927268,2269] 0 2024-02-19T19:06:32.081 INFO:tasks.workunit.client.0.smithi088.stdout:1/811: creat d6/d21/d64/d94/dab/f107 x:0 0 0 2024-02-19T19:06:32.084 INFO:tasks.workunit.client.0.smithi088.stdout:6/854: creat d5/d8/d77/f111 x:0 0 0 2024-02-19T19:06:32.085 INFO:tasks.workunit.client.0.smithi088.stdout:6/855: stat d5/de/d64/d82/d8c/f59 0 2024-02-19T19:06:32.085 INFO:tasks.workunit.client.0.smithi088.stdout:2/665: unlink d3/d14/c80 0 2024-02-19T19:06:32.090 INFO:tasks.workunit.client.0.smithi088.stdout:1/812: mkdir d6/d21/d64/d108 0 2024-02-19T19:06:32.092 INFO:tasks.workunit.client.0.smithi088.stdout:0/758: dwrite d13/d62/d8a/de0/feb [0,4194304] 0 2024-02-19T19:06:32.102 INFO:tasks.workunit.client.0.smithi088.stdout:4/593: write d2/d8/f16 [3179869,16085] 0 2024-02-19T19:06:32.106 INFO:tasks.workunit.client.0.smithi088.stdout:0/759: unlink d13/d62/lcf 0 2024-02-19T19:06:32.106 INFO:tasks.workunit.client.0.smithi088.stdout:8/704: write d9/df/d3d/d50/d71/da1/d6c/d9a/f57 [1572983,3683] 0 2024-02-19T19:06:32.107 INFO:tasks.workunit.client.0.smithi088.stdout:3/828: write d4/d12/d33/d43/d85/fc6 [824337,118446] 0 2024-02-19T19:06:32.108 INFO:tasks.workunit.client.0.smithi088.stdout:4/594: dread d2/d35/f52 [0,4194304] 0 2024-02-19T19:06:32.110 INFO:tasks.workunit.client.0.smithi088.stdout:2/666: rename d3/d1f/d4f/c12 to d3/d1b/d43/d8f/cd4 0 2024-02-19T19:06:32.111 INFO:tasks.workunit.client.0.smithi088.stdout:0/760: dread d13/d62/d8a/d9c/da8/dab/db4/fc1 [0,4194304] 0 2024-02-19T19:06:32.115 INFO:tasks.workunit.client.0.smithi088.stdout:6/856: creat d5/d8/d2c/f112 x:0 0 0 2024-02-19T19:06:32.115 INFO:tasks.workunit.client.0.smithi088.stdout:8/705: chown d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/d8e/c93 2159 1 2024-02-19T19:06:32.118 INFO:tasks.workunit.client.0.smithi088.stdout:3/829: creat d4/d1a/d1e/d31/d32/f125 x:0 0 0 2024-02-19T19:06:32.118 INFO:tasks.workunit.client.0.smithi088.stdout:5/964: dwrite d4/d61/d65/d9a/fac [0,4194304] 0 2024-02-19T19:06:32.123 INFO:tasks.workunit.client.0.smithi088.stdout:2/667: rmdir d3/d1b/d43/d61/d7e/db6 39 2024-02-19T19:06:32.124 INFO:tasks.workunit.client.0.smithi088.stdout:0/761: rmdir d13/d62/d8a/d9c/da8/dab 39 2024-02-19T19:06:32.126 INFO:tasks.workunit.client.0.smithi088.stdout:8/706: dwrite d9/df/f36 [4194304,4194304] 0 2024-02-19T19:06:32.129 INFO:tasks.workunit.client.0.smithi088.stdout:8/707: fsync d9/df/fb4 0 2024-02-19T19:06:32.130 INFO:tasks.workunit.client.0.smithi088.stdout:3/830: dread d4/d1a/d1e/d31/d32/ffa [0,4194304] 0 2024-02-19T19:06:32.131 INFO:tasks.workunit.client.0.smithi088.stdout:1/813: dwrite d6/d21/d6a/d7d/d9a/fbf [0,4194304] 0 2024-02-19T19:06:32.135 INFO:tasks.workunit.client.0.smithi088.stdout:5/965: rmdir d4/d1a/d41/de2/de7/dff/d114 39 2024-02-19T19:06:32.138 INFO:tasks.workunit.client.0.smithi088.stdout:2/668: symlink d3/d1f/d4f/d59/ld5 0 2024-02-19T19:06:32.139 INFO:tasks.workunit.client.0.smithi088.stdout:1/814: rmdir d6/d10/d60/db8 39 2024-02-19T19:06:32.141 INFO:tasks.workunit.client.0.smithi088.stdout:4/595: link d2/d32/d9/l3e d2/d32/d25/d26/d6d/d7e/d3a/lc0 0 2024-02-19T19:06:32.142 INFO:tasks.workunit.client.0.smithi088.stdout:8/708: creat d9/df/d3d/d50/d71/fd4 x:0 0 0 2024-02-19T19:06:32.143 INFO:tasks.workunit.client.0.smithi088.stdout:3/831: dwrite d4/d1a/d1e/d74/f96 [4194304,4194304] 0 2024-02-19T19:06:32.145 INFO:tasks.workunit.client.0.smithi088.stdout:6/857: truncate d5/d8/d3a/d71/daa/dc2/f5d 276159 0 2024-02-19T19:06:32.145 INFO:tasks.workunit.client.0.smithi088.stdout:5/966: dread - d4/d1a/d41/de2/de7/dff/d114/f14d zero size 2024-02-19T19:06:32.147 INFO:tasks.workunit.client.0.smithi088.stdout:1/815: creat d6/d21/d6a/d7d/d9a/f109 x:0 0 0 2024-02-19T19:06:32.152 INFO:tasks.workunit.client.0.smithi088.stdout:1/816: dread - d6/d21/d6a/dcb/f105 zero size 2024-02-19T19:06:32.153 INFO:tasks.workunit.client.0.smithi088.stdout:5/967: mknod d4/d1a/d41/d154/c157 0 2024-02-19T19:06:32.157 INFO:tasks.workunit.client.0.smithi088.stdout:3/832: creat d4/d1a/d1e/d74/d7c/f126 x:0 0 0 2024-02-19T19:06:32.157 INFO:tasks.workunit.client.0.smithi088.stdout:8/709: creat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/fd5 x:0 0 0 2024-02-19T19:06:32.159 INFO:tasks.workunit.client.0.smithi088.stdout:5/968: dread - d4/d1a/d41/dc1/f115 zero size 2024-02-19T19:06:32.160 INFO:tasks.workunit.client.0.smithi088.stdout:2/669: dwrite d3/d1b/d43/d8f/fd0 [0,4194304] 0 2024-02-19T19:06:32.160 INFO:tasks.workunit.client.0.smithi088.stdout:4/596: dwrite d2/d32/d25/d45/f4a [4194304,4194304] 0 2024-02-19T19:06:32.160 INFO:tasks.workunit.client.0.smithi088.stdout:1/817: unlink d6/d21/d6a/d7d/fa6 0 2024-02-19T19:06:32.167 INFO:tasks.workunit.client.0.smithi088.stdout:3/833: creat d4/d1a/d1e/f127 x:0 0 0 2024-02-19T19:06:32.167 INFO:tasks.workunit.client.0.smithi088.stdout:8/710: stat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/d8e/cac 0 2024-02-19T19:06:32.168 INFO:tasks.workunit.client.0.smithi088.stdout:6/858: dwrite d5/d8/feb [0,4194304] 0 2024-02-19T19:06:32.170 INFO:tasks.workunit.client.0.smithi088.stdout:1/818: mknod d6/d21/d6a/d4a/c10a 0 2024-02-19T19:06:32.173 INFO:tasks.workunit.client.0.smithi088.stdout:3/834: fdatasync d4/d1a/d1e/d31/d32/f4c 0 2024-02-19T19:06:32.174 INFO:tasks.workunit.client.0.smithi088.stdout:5/969: creat d4/d1a/d41/d154/d13b/f158 x:0 0 0 2024-02-19T19:06:32.175 INFO:tasks.workunit.client.0.smithi088.stdout:1/819: symlink d6/d21/d64/l10b 0 2024-02-19T19:06:32.177 INFO:tasks.workunit.client.0.smithi088.stdout:2/670: mknod d3/d1b/d43/d61/d7e/db6/cd6 0 2024-02-19T19:06:32.182 INFO:tasks.workunit.client.0.smithi088.stdout:8/711: dwrite d9/df/d3d/d50/d71/da1/f9b [0,4194304] 0 2024-02-19T19:06:32.184 INFO:tasks.workunit.client.0.smithi088.stdout:6/859: creat d5/d8/d2c/f113 x:0 0 0 2024-02-19T19:06:32.184 INFO:tasks.workunit.client.0.smithi088.stdout:3/835: dread d4/d12/d33/d43/fbd [4194304,4194304] 0 2024-02-19T19:06:32.186 INFO:tasks.workunit.client.0.smithi088.stdout:6/860: fdatasync d5/d8/d3a/da8/fb0 0 2024-02-19T19:06:32.187 INFO:tasks.workunit.client.0.smithi088.stdout:2/671: creat d3/d1b/d43/d61/da6/db2/fd7 x:0 0 0 2024-02-19T19:06:32.191 INFO:tasks.workunit.client.0.smithi088.stdout:2/672: rename d3/d14/d93 to d3/d14/dd8 0 2024-02-19T19:06:32.195 INFO:tasks.workunit.client.0.smithi088.stdout:6/861: rename d5/d19/f4e to d5/d8/d3a/da8/f114 0 2024-02-19T19:06:32.197 INFO:tasks.workunit.client.0.smithi088.stdout:8/712: creat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/dbe/fd6 x:0 0 0 2024-02-19T19:06:32.198 INFO:tasks.workunit.client.0.smithi088.stdout:2/673: write d3/fbd [968445,88953] 0 2024-02-19T19:06:32.198 INFO:tasks.workunit.client.0.smithi088.stdout:5/970: write d4/d7/f64 [3646171,75327] 0 2024-02-19T19:06:32.199 INFO:tasks.workunit.client.0.smithi088.stdout:5/971: stat d4/d61/cad 0 2024-02-19T19:06:32.199 INFO:tasks.workunit.client.0.smithi088.stdout:1/820: getdents d6/d21/d6a/dcb 0 2024-02-19T19:06:32.200 INFO:tasks.workunit.client.0.smithi088.stdout:5/972: dread - d4/d1a/d41/dc1/f111 zero size 2024-02-19T19:06:32.201 INFO:tasks.workunit.client.0.smithi088.stdout:6/862: unlink d5/de/f18 0 2024-02-19T19:06:32.202 INFO:tasks.workunit.client.0.smithi088.stdout:3/836: write d4/d22/d7a/d11a/d11b/fee [152619,21950] 0 2024-02-19T19:06:32.205 INFO:tasks.workunit.client.0.smithi088.stdout:2/674: creat d3/d1f/d4f/d59/fd9 x:0 0 0 2024-02-19T19:06:32.206 INFO:tasks.workunit.client.0.smithi088.stdout:5/973: mkdir d4/d1a/d41/dc1/d159 0 2024-02-19T19:06:32.207 INFO:tasks.workunit.client.0.smithi088.stdout:8/713: symlink d9/dc4/ld7 0 2024-02-19T19:06:32.208 INFO:tasks.workunit.client.0.smithi088.stdout:6/863: creat d5/d8/d3a/da8/f115 x:0 0 0 2024-02-19T19:06:32.210 INFO:tasks.workunit.client.0.smithi088.stdout:8/714: rmdir d9/df/d1c/d22 39 2024-02-19T19:06:32.211 INFO:tasks.workunit.client.0.smithi088.stdout:1/821: link d6/d21/d6a/f8e d6/d21/d64/d94/d71/f10c 0 2024-02-19T19:06:32.212 INFO:tasks.workunit.client.0.smithi088.stdout:1/822: chown d6/d10/d60/c8c 3378373 1 2024-02-19T19:06:32.214 INFO:tasks.workunit.client.0.smithi088.stdout:6/864: chown d5/d8/d3a/d71/daa/dc2/dbc/db6/cf2 68108854 1 2024-02-19T19:06:32.218 INFO:tasks.workunit.client.0.smithi088.stdout:1/823: creat d6/d10/d60/db8/dcf/f10d x:0 0 0 2024-02-19T19:06:32.220 INFO:tasks.workunit.client.0.smithi088.stdout:6/865: dread d5/d45/f75 [0,4194304] 0 2024-02-19T19:06:32.221 INFO:tasks.workunit.client.0.smithi088.stdout:5/974: dread d4/d75/fbf [0,4194304] 0 2024-02-19T19:06:32.223 INFO:tasks.workunit.client.0.smithi088.stdout:2/675: truncate d3/d14/f1d 2552715 0 2024-02-19T19:06:32.224 INFO:tasks.workunit.client.0.smithi088.stdout:6/866: unlink d5/d8/d3a/d71/f74 0 2024-02-19T19:06:32.225 INFO:tasks.workunit.client.0.smithi088.stdout:1/824: rename d6/l34 to d6/d10/d60/db8/l10e 0 2024-02-19T19:06:32.227 INFO:tasks.workunit.client.0.smithi088.stdout:1/825: write d6/f17 [7027937,83411] 0 2024-02-19T19:06:32.228 INFO:tasks.workunit.client.0.smithi088.stdout:2/676: creat d3/d1b/d43/d8f/fda x:0 0 0 2024-02-19T19:06:32.229 INFO:tasks.workunit.client.0.smithi088.stdout:6/867: creat d5/de/d64/d8d/d95/dba/f116 x:0 0 0 2024-02-19T19:06:32.231 INFO:tasks.workunit.client.0.smithi088.stdout:6/868: write d5/de/d60/f103 [418517,27161] 0 2024-02-19T19:06:32.233 INFO:tasks.workunit.client.0.smithi088.stdout:1/826: fdatasync d6/d10/d60/f67 0 2024-02-19T19:06:32.235 INFO:tasks.workunit.client.0.smithi088.stdout:1/827: dread - d6/d21/d64/dc8/dcc/d101/fee zero size 2024-02-19T19:06:32.236 INFO:tasks.workunit.client.0.smithi088.stdout:5/975: write d4/d1a/d41/fe6 [2252244,1116] 0 2024-02-19T19:06:32.243 INFO:tasks.workunit.client.0.smithi088.stdout:2/677: rename d3/d1b/d43/d8f/d94/l97 to d3/d71/ldb 0 2024-02-19T19:06:32.245 INFO:tasks.workunit.client.0.smithi088.stdout:6/869: truncate d5/d19/d70/f106 8797 0 2024-02-19T19:06:32.247 INFO:tasks.workunit.client.0.smithi088.stdout:5/976: unlink d4/d1a/f54 0 2024-02-19T19:06:32.249 INFO:tasks.workunit.client.0.smithi088.stdout:5/977: chown d4/d61/d65/f105 0 1 2024-02-19T19:06:32.249 INFO:tasks.workunit.client.0.smithi088.stdout:1/828: dwrite d6/d10/f23 [0,4194304] 0 2024-02-19T19:06:32.283 INFO:tasks.workunit.client.0.smithi088.stdout:7/800: sync 2024-02-19T19:06:32.284 INFO:tasks.workunit.client.0.smithi088.stdout:2/678: creat d3/d1f/fdc x:0 0 0 2024-02-19T19:06:32.285 INFO:tasks.workunit.client.0.smithi088.stdout:6/870: write d5/d8/d3a/db8/fdc [2666138,90591] 0 2024-02-19T19:06:32.287 INFO:tasks.workunit.client.0.smithi088.stdout:5/978: creat d4/d75/deb/f15a x:0 0 0 2024-02-19T19:06:32.291 INFO:tasks.workunit.client.0.smithi088.stdout:2/679: creat d3/d14/dd8/fdd x:0 0 0 2024-02-19T19:06:32.293 INFO:tasks.workunit.client.0.smithi088.stdout:7/801: dwrite d14/d27/d49/d98/dcb/de7/f10d [0,4194304] 0 2024-02-19T19:06:32.294 INFO:tasks.workunit.client.0.smithi088.stdout:1/829: creat d6/d21/d64/d94/d71/dbe/f10f x:0 0 0 2024-02-19T19:06:32.294 INFO:tasks.workunit.client.0.smithi088.stdout:5/979: mkdir d4/d7/d15b 0 2024-02-19T19:06:32.295 INFO:tasks.workunit.client.0.smithi088.stdout:6/871: write d5/de/d97/f9b [1117790,67605] 0 2024-02-19T19:06:32.296 INFO:tasks.workunit.client.0.smithi088.stdout:7/802: write d14/f4f [6734931,6355] 0 2024-02-19T19:06:32.297 INFO:tasks.workunit.client.0.smithi088.stdout:1/830: stat d6/d21/d64/d94/d71/dbe/l100 0 2024-02-19T19:06:32.297 INFO:tasks.workunit.client.0.smithi088.stdout:5/980: write d4/d7/fbe [3331991,53393] 0 2024-02-19T19:06:32.301 INFO:tasks.workunit.client.0.smithi088.stdout:7/803: rmdir d14/d1d/d62/dcf 39 2024-02-19T19:06:32.302 INFO:tasks.workunit.client.0.smithi088.stdout:6/872: creat d5/de/d64/d8d/d95/f117 x:0 0 0 2024-02-19T19:06:32.306 INFO:tasks.workunit.client.0.smithi088.stdout:6/873: rmdir d5/de/d64/d82 39 2024-02-19T19:06:32.307 INFO:tasks.workunit.client.0.smithi088.stdout:2/680: creat d3/d1f/fde x:0 0 0 2024-02-19T19:06:32.309 INFO:tasks.workunit.client.0.smithi088.stdout:1/831: creat d6/d10/d60/f110 x:0 0 0 2024-02-19T19:06:32.310 INFO:tasks.workunit.client.0.smithi088.stdout:6/874: creat d5/de/d64/d7e/f118 x:0 0 0 2024-02-19T19:06:32.313 INFO:tasks.workunit.client.0.smithi088.stdout:1/832: unlink d6/d10/d60/d29/l47 0 2024-02-19T19:06:32.315 INFO:tasks.workunit.client.0.smithi088.stdout:1/833: write d6/d21/d64/d94/d71/fcd [955774,17594] 0 2024-02-19T19:06:32.320 INFO:tasks.workunit.client.0.smithi088.stdout:2/681: link d3/f31 d3/d1b/d43/d61/da6/db2/fdf 0 2024-02-19T19:06:32.323 INFO:tasks.workunit.client.0.smithi088.stdout:5/981: dread d4/d61/d65/d9a/d9f/da0/fae [0,4194304] 0 2024-02-19T19:06:32.323 INFO:tasks.workunit.client.0.smithi088.stdout:7/804: dwrite d14/d27/d6b/f93 [0,4194304] 0 2024-02-19T19:06:32.326 INFO:tasks.workunit.client.0.smithi088.stdout:6/875: write d5/d8/d3a/da8/f114 [50662,109130] 0 2024-02-19T19:06:32.329 INFO:tasks.workunit.client.0.smithi088.stdout:5/982: mknod d4/d1a/d41/dcb/c15c 0 2024-02-19T19:06:32.329 INFO:tasks.workunit.client.0.smithi088.stdout:6/876: fsync d5/d8/d3a/da8/fdd 0 2024-02-19T19:06:32.330 INFO:tasks.workunit.client.0.smithi088.stdout:6/877: chown d5/d7/dc9/dca 92528 1 2024-02-19T19:06:32.332 INFO:tasks.workunit.client.0.smithi088.stdout:6/878: write d5/de/d64/d7e/f118 [586497,24529] 0 2024-02-19T19:06:32.334 INFO:tasks.workunit.client.0.smithi088.stdout:5/983: creat d4/d1a/d41/d154/f15d x:0 0 0 2024-02-19T19:06:32.337 INFO:tasks.workunit.client.0.smithi088.stdout:6/879: symlink d5/d19/d26/d72/l119 0 2024-02-19T19:06:32.339 INFO:tasks.workunit.client.0.smithi088.stdout:5/984: creat d4/d1a/d41/d154/d100/f15e x:0 0 0 2024-02-19T19:06:32.339 INFO:tasks.workunit.client.0.smithi088.stdout:1/834: dread d6/d10/f5d [0,4194304] 0 2024-02-19T19:06:32.344 INFO:tasks.workunit.client.0.smithi088.stdout:1/835: unlink d6/d21/d64/d94/d71/l9f 0 2024-02-19T19:06:32.345 INFO:tasks.workunit.client.0.smithi088.stdout:5/985: symlink d4/d1a/d41/dc1/d159/l15f 0 2024-02-19T19:06:32.348 INFO:tasks.workunit.client.0.smithi088.stdout:7/805: dwrite d14/d27/d6b/df8/d10b/f111 [0,4194304] 0 2024-02-19T19:06:32.352 INFO:tasks.workunit.client.0.smithi088.stdout:2/682: dread d3/d1b/d43/d61/d64/f63 [0,4194304] 0 2024-02-19T19:06:32.354 INFO:tasks.workunit.client.0.smithi088.stdout:2/683: chown d3/d1b/d43/d74 26662 1 2024-02-19T19:06:32.355 INFO:tasks.workunit.client.0.smithi088.stdout:5/986: dwrite d4/d75/deb/f15a [0,4194304] 0 2024-02-19T19:06:32.357 INFO:tasks.workunit.client.0.smithi088.stdout:6/880: dread d5/d19/d26/f44 [4194304,4194304] 0 2024-02-19T19:06:32.360 INFO:tasks.workunit.client.0.smithi088.stdout:5/987: dread - d4/d1a/fc3 zero size 2024-02-19T19:06:32.361 INFO:tasks.workunit.client.0.smithi088.stdout:2/684: creat d3/d1f/d4f/dc6/fe0 x:0 0 0 2024-02-19T19:06:32.362 INFO:tasks.workunit.client.0.smithi088.stdout:7/806: dwrite d14/d27/d6b/dba/d102/ff6 [0,4194304] 0 2024-02-19T19:06:32.365 INFO:tasks.workunit.client.0.smithi088.stdout:2/685: write d3/d1b/d43/d61/d64/f8a [819035,51870] 0 2024-02-19T19:06:32.371 INFO:tasks.workunit.client.0.smithi088.stdout:5/988: getdents d4/d1a/d41/d154 0 2024-02-19T19:06:32.375 INFO:tasks.workunit.client.0.smithi088.stdout:6/881: dwrite d5/d19/f5f [4194304,4194304] 0 2024-02-19T19:06:32.375 INFO:tasks.workunit.client.0.smithi088.stdout:5/989: creat d4/d61/d65/d9a/f160 x:0 0 0 2024-02-19T19:06:32.380 INFO:tasks.workunit.client.0.smithi088.stdout:5/990: mknod d4/d61/d65/d9a/d9f/c161 0 2024-02-19T19:06:32.382 INFO:tasks.workunit.client.0.smithi088.stdout:2/686: dwrite d3/d1f/d5d/f72 [0,4194304] 0 2024-02-19T19:06:32.382 INFO:tasks.workunit.client.0.smithi088.stdout:7/807: dwrite d14/f1f [0,4194304] 0 2024-02-19T19:06:32.386 INFO:tasks.workunit.client.0.smithi088.stdout:7/808: fsync d14/f29 0 2024-02-19T19:06:32.388 INFO:tasks.workunit.client.0.smithi088.stdout:2/687: mkdir d3/d1f/d5d/de1 0 2024-02-19T19:06:32.389 INFO:tasks.workunit.client.0.smithi088.stdout:2/688: write d3/d14/dd8/fcf [54244,28027] 0 2024-02-19T19:06:32.390 INFO:tasks.workunit.client.0.smithi088.stdout:2/689: dread - d3/d1b/d43/d61/f9e zero size 2024-02-19T19:06:32.393 INFO:tasks.workunit.client.0.smithi088.stdout:2/690: write d3/d1f/d4f/dc6/fcb [725666,54119] 0 2024-02-19T19:06:32.393 INFO:tasks.workunit.client.0.smithi088.stdout:5/991: rename d4/d1a/d41/de2/de7/f149 to d4/d75/deb/f162 0 2024-02-19T19:06:32.395 INFO:tasks.workunit.client.0.smithi088.stdout:5/992: dread - d4/d1a/d41/dcb/ffe zero size 2024-02-19T19:06:32.424 INFO:tasks.workunit.client.0.smithi088.stdout:2/691: creat d3/d1b/d43/d61/da6/fe2 x:0 0 0 2024-02-19T19:06:32.425 INFO:tasks.workunit.client.0.smithi088.stdout:7/809: dwrite d14/f22 [0,4194304] 0 2024-02-19T19:06:32.428 INFO:tasks.workunit.client.0.smithi088.stdout:2/692: rename d3/d1b/d43/d61/d64/f9d to d3/d1f/d5d/fe3 0 2024-02-19T19:06:32.431 INFO:tasks.workunit.client.0.smithi088.stdout:7/810: rename d14/d27/c59 to d14/d28/d65/c11c 0 2024-02-19T19:06:32.432 INFO:tasks.workunit.client.0.smithi088.stdout:2/693: symlink d3/d1b/d43/d61/d86/le4 0 2024-02-19T19:06:32.432 INFO:tasks.workunit.client.0.smithi088.stdout:5/993: link d4/d1a/d41/dc1/dc2/l125 d4/d75/de8/d136/l163 0 2024-02-19T19:06:32.434 INFO:tasks.workunit.client.0.smithi088.stdout:2/694: write d3/d1b/d43/d8f/fda [279791,123102] 0 2024-02-19T19:06:32.435 INFO:tasks.workunit.client.0.smithi088.stdout:5/994: fsync d4/d61/d65/d9a/d9f/da0/fb4 0 2024-02-19T19:06:32.440 INFO:tasks.workunit.client.0.smithi088.stdout:5/995: rename d4/d7/f64 to d4/d1a/d41/dcb/f164 0 2024-02-19T19:06:32.443 INFO:tasks.workunit.client.0.smithi088.stdout:5/996: mknod d4/d61/d65/d9a/d9f/d57/c165 0 2024-02-19T19:06:32.447 INFO:tasks.workunit.client.0.smithi088.stdout:5/997: mkdir d4/d1a/d41/de2/de7/dff/d166 0 2024-02-19T19:06:32.447 INFO:tasks.workunit.client.0.smithi088.stdout:7/811: write d14/d1d/d62/dcf/fe4 [465611,65491] 0 2024-02-19T19:06:32.448 INFO:tasks.workunit.client.0.smithi088.stdout:5/998: chown d4/cd9 45 1 2024-02-19T19:06:32.450 INFO:tasks.workunit.client.0.smithi088.stdout:7/812: mkdir d14/d27/d49/dfe/d11d 0 2024-02-19T19:06:32.454 INFO:tasks.workunit.client.0.smithi088.stdout:5/999: rename d4/d1a/d41/d154/d100 to d4/d61/d65/d9a/d167 0 2024-02-19T19:06:32.454 INFO:tasks.workunit.client.0.smithi088.stdout:2/695: dwrite d3/d1f/f4a [0,4194304] 0 2024-02-19T19:06:32.496 INFO:tasks.workunit.client.0.smithi088.stdout:2/696: write d3/d1f/d5d/fc5 [4818892,105109] 0 2024-02-19T19:06:32.500 INFO:tasks.workunit.client.0.smithi088.stdout:2/697: creat d3/d1f/d5d/de1/fe5 x:0 0 0 2024-02-19T19:06:32.501 INFO:tasks.workunit.client.0.smithi088.stdout:7/813: dwrite d14/f51 [0,4194304] 0 2024-02-19T19:06:32.504 INFO:tasks.workunit.client.0.smithi088.stdout:2/698: write d3/d1f/f73 [4098512,25036] 0 2024-02-19T19:06:32.506 INFO:tasks.workunit.client.0.smithi088.stdout:7/814: chown d14/cbd 632986809 1 2024-02-19T19:06:32.507 INFO:tasks.workunit.client.0.smithi088.stdout:2/699: creat d3/d1b/d43/d61/da6/fe6 x:0 0 0 2024-02-19T19:06:32.511 INFO:tasks.workunit.client.0.smithi088.stdout:7/815: symlink d14/d1d/d3c/l11e 0 2024-02-19T19:06:32.518 INFO:tasks.workunit.client.0.smithi088.stdout:7/816: getdents d14/d1d/d3c/d43/d5c 0 2024-02-19T19:06:32.520 INFO:tasks.workunit.client.0.smithi088.stdout:7/817: rmdir d14/d27/d6b/df8 39 2024-02-19T19:06:32.532 INFO:tasks.workunit.client.0.smithi088.stdout:2/700: dwrite d3/d1b/d43/d61/d86/faa [0,4194304] 0 2024-02-19T19:06:32.532 INFO:tasks.workunit.client.0.smithi088.stdout:7/818: dwrite d14/d28/d65/f7e [0,4194304] 0 2024-02-19T19:06:32.537 INFO:tasks.workunit.client.0.smithi088.stdout:7/819: symlink d14/d1d/d3c/d43/db2/dc1/l11f 0 2024-02-19T19:06:32.538 INFO:tasks.workunit.client.0.smithi088.stdout:7/820: fsync d14/d1d/f105 0 2024-02-19T19:06:32.540 INFO:tasks.workunit.client.0.smithi088.stdout:7/821: truncate d14/d27/d49/d98/dcb/fff 418934 0 2024-02-19T19:06:32.546 INFO:tasks.workunit.client.0.smithi088.stdout:2/701: dwrite d3/fb0 [0,4194304] 0 2024-02-19T19:06:32.547 INFO:tasks.workunit.client.0.smithi088.stdout:2/702: readlink d3/d1b/d43/la7 0 2024-02-19T19:06:32.548 INFO:tasks.workunit.client.0.smithi088.stdout:7/822: link d14/d1d/d3c/f80 d14/d1d/d3c/d43/d5c/d9a/d92/f120 0 2024-02-19T19:06:32.549 INFO:tasks.workunit.client.0.smithi088.stdout:7/823: fsync fd 0 2024-02-19T19:06:32.552 INFO:tasks.workunit.client.0.smithi088.stdout:0/762: sync 2024-02-19T19:06:32.552 INFO:tasks.workunit.client.0.smithi088.stdout:2/703: creat d3/d1b/d43/d61/fe7 x:0 0 0 2024-02-19T19:06:32.553 INFO:tasks.workunit.client.0.smithi088.stdout:7/824: symlink d14/d1d/l121 0 2024-02-19T19:06:32.554 INFO:tasks.workunit.client.0.smithi088.stdout:0/763: chown d13/l22 361403 1 2024-02-19T19:06:32.555 INFO:tasks.workunit.client.0.smithi088.stdout:0/764: fdatasync d13/d62/fa6 0 2024-02-19T19:06:32.555 INFO:tasks.workunit.client.0.smithi088.stdout:2/704: dread d3/d1f/d5d/fe3 [0,4194304] 0 2024-02-19T19:06:32.560 INFO:tasks.workunit.client.0.smithi088.stdout:7/825: link d14/d28/c82 d14/d27/d49/d98/dbe/c122 0 2024-02-19T19:06:32.562 INFO:tasks.workunit.client.0.smithi088.stdout:2/705: creat d3/fe8 x:0 0 0 2024-02-19T19:06:32.563 INFO:tasks.workunit.client.0.smithi088.stdout:2/706: write d3/d1f/f89 [4213605,53164] 0 2024-02-19T19:06:32.565 INFO:tasks.workunit.client.0.smithi088.stdout:2/707: write d3/d14/dd8/fcf [888965,19261] 0 2024-02-19T19:06:32.568 INFO:tasks.workunit.client.0.smithi088.stdout:7/826: getdents d14/d1d/d3c/d43/d5c/d75 0 2024-02-19T19:06:32.570 INFO:tasks.workunit.client.0.smithi088.stdout:7/827: creat d14/d1d/d62/dcf/f123 x:0 0 0 2024-02-19T19:06:32.570 INFO:tasks.workunit.client.0.smithi088.stdout:2/708: dread d3/d1f/d5d/f62 [0,4194304] 0 2024-02-19T19:06:32.571 INFO:tasks.workunit.client.0.smithi088.stdout:0/765: write d13/de9/ffd [6662073,127331] 0 2024-02-19T19:06:32.578 INFO:tasks.workunit.client.0.smithi088.stdout:7/828: dwrite d14/d1d/d62/dcf/f123 [0,4194304] 0 2024-02-19T19:06:32.581 INFO:tasks.workunit.client.0.smithi088.stdout:7/829: symlink d14/d1d/d3c/l124 0 2024-02-19T19:06:32.583 INFO:tasks.workunit.client.0.smithi088.stdout:0/766: truncate d13/f29 3586416 0 2024-02-19T19:06:32.584 INFO:tasks.workunit.client.0.smithi088.stdout:0/767: creat d13/de9/f107 x:0 0 0 2024-02-19T19:06:32.586 INFO:tasks.workunit.client.0.smithi088.stdout:7/830: link d14/f6e d14/d1d/d3c/d43/d5c/f125 0 2024-02-19T19:06:32.586 INFO:tasks.workunit.client.0.smithi088.stdout:7/831: stat d14/d28/d65/fa8 0 2024-02-19T19:06:32.592 INFO:tasks.workunit.client.0.smithi088.stdout:2/709: dread d3/d1f/f73 [0,4194304] 0 2024-02-19T19:06:32.592 INFO:tasks.workunit.client.0.smithi088.stdout:0/768: link d13/d62/d8a/d95/d5a/d25/d2a/d2f/fb0 d13/d62/d8a/d95/d5b/f108 0 2024-02-19T19:06:32.593 INFO:tasks.workunit.client.0.smithi088.stdout:7/832: dwrite f2 [0,4194304] 0 2024-02-19T19:06:32.593 INFO:tasks.workunit.client.0.smithi088.stdout:0/769: chown d13/d62/d8a 28 1 2024-02-19T19:06:32.594 INFO:tasks.workunit.client.0.smithi088.stdout:2/710: dread - d3/d1b/d43/d61/da6/db2/fd7 zero size 2024-02-19T19:06:32.596 INFO:tasks.workunit.client.0.smithi088.stdout:7/833: fdatasync d14/f44 0 2024-02-19T19:06:32.600 INFO:tasks.workunit.client.0.smithi088.stdout:7/834: getdents d14/d1d/d3c/d43/d5c/da6 0 2024-02-19T19:06:32.604 INFO:tasks.workunit.client.0.smithi088.stdout:7/835: creat d14/d27/d49/d98/dbe/f126 x:0 0 0 2024-02-19T19:06:32.619 INFO:tasks.workunit.client.0.smithi088.stdout:0/770: dread f4 [0,4194304] 0 2024-02-19T19:06:32.620 INFO:tasks.workunit.client.0.smithi088.stdout:2/711: dread d3/d1b/d43/d8f/f7b [0,4194304] 0 2024-02-19T19:06:32.620 INFO:tasks.workunit.client.0.smithi088.stdout:7/836: dwrite d14/d1d/d3c/f80 [0,4194304] 0 2024-02-19T19:06:32.621 INFO:tasks.workunit.client.0.smithi088.stdout:2/712: read - d3/d1f/d4f/d59/fd9 zero size 2024-02-19T19:06:32.623 INFO:tasks.workunit.client.0.smithi088.stdout:2/713: mkdir d3/d14/dd8/de9 0 2024-02-19T19:06:32.631 INFO:tasks.workunit.client.0.smithi088.stdout:7/837: dwrite d14/d27/f36 [0,4194304] 0 2024-02-19T19:06:32.633 INFO:tasks.workunit.client.0.smithi088.stdout:2/714: dwrite d3/d1b/d43/d61/d64/fa4 [0,4194304] 0 2024-02-19T19:06:32.634 INFO:tasks.workunit.client.0.smithi088.stdout:7/838: write d14/d28/d65/ff3 [336777,94451] 0 2024-02-19T19:06:32.638 INFO:tasks.workunit.client.0.smithi088.stdout:0/771: truncate d13/d62/d8a/d9c/da8/dab/db4/fc1 78222 0 2024-02-19T19:06:32.639 INFO:tasks.workunit.client.0.smithi088.stdout:2/715: rename d3/d1b/d43 to d3/d1b/d43/dea 22 2024-02-19T19:06:32.641 INFO:tasks.workunit.client.0.smithi088.stdout:7/839: mknod d14/d27/d6b/c127 0 2024-02-19T19:06:32.642 INFO:tasks.workunit.client.0.smithi088.stdout:0/772: write d13/d62/d8a/d95/d5a/d25/d2a/d3c/f64 [2939502,79636] 0 2024-02-19T19:06:32.646 INFO:tasks.workunit.client.0.smithi088.stdout:0/773: symlink d13/d62/d8a/d95/d5a/d25/dd6/df9/l109 0 2024-02-19T19:06:32.647 INFO:tasks.workunit.client.0.smithi088.stdout:2/716: creat d3/d1b/d43/d61/d7e/feb x:0 0 0 2024-02-19T19:06:32.650 INFO:tasks.workunit.client.0.smithi088.stdout:7/840: write d14/d28/d3b/dd3/fae [1012424,12531] 0 2024-02-19T19:06:32.650 INFO:tasks.workunit.client.0.smithi088.stdout:0/774: write d13/d62/d8a/d95/d5a/f74 [2282774,66048] 0 2024-02-19T19:06:32.651 INFO:tasks.workunit.client.0.smithi088.stdout:7/841: chown d14/d28/d94/ddb/f103 629657 1 2024-02-19T19:06:32.655 INFO:tasks.workunit.client.0.smithi088.stdout:2/717: getdents d3/d1f/d5d/de1 0 2024-02-19T19:06:32.660 INFO:tasks.workunit.client.0.smithi088.stdout:2/718: symlink d3/d14/dd8/de9/lec 0 2024-02-19T19:06:32.660 INFO:tasks.workunit.client.0.smithi088.stdout:7/842: dwrite d14/d1d/d62/dcf/f123 [4194304,4194304] 0 2024-02-19T19:06:32.661 INFO:tasks.workunit.client.0.smithi088.stdout:2/719: stat d3/d1b/d43/la7 0 2024-02-19T19:06:32.661 INFO:tasks.workunit.client.0.smithi088.stdout:7/843: readlink d14/d27/l55 0 2024-02-19T19:06:32.665 INFO:tasks.workunit.client.0.smithi088.stdout:7/844: creat d14/d1d/d3c/d43/d5c/d9a/df4/f128 x:0 0 0 2024-02-19T19:06:32.668 INFO:tasks.workunit.client.0.smithi088.stdout:2/720: rename d3/d1b/f8c to d3/d1b/d43/d61/d64/fed 0 2024-02-19T19:06:32.669 INFO:tasks.workunit.client.0.smithi088.stdout:0/775: dread d13/d62/f9e [0,4194304] 0 2024-02-19T19:06:32.670 INFO:tasks.workunit.client.0.smithi088.stdout:7/845: rename d14/f19 to d14/d1d/d62/dcf/f129 0 2024-02-19T19:06:32.672 INFO:tasks.workunit.client.0.smithi088.stdout:7/846: write d14/d1d/d3c/d43/d5c/d9a/d92/f97 [472729,21063] 0 2024-02-19T19:06:32.672 INFO:tasks.workunit.client.0.smithi088.stdout:2/721: mknod d3/d14/cee 0 2024-02-19T19:06:32.674 INFO:tasks.workunit.client.0.smithi088.stdout:0/776: readlink d13/d81/lc2 0 2024-02-19T19:06:32.676 INFO:tasks.workunit.client.0.smithi088.stdout:7/847: chown d14/d1d/d3c/d43/l4d 1426232 1 2024-02-19T19:06:32.681 INFO:tasks.workunit.client.0.smithi088.stdout:7/848: getdents d14/d27/d49/d98/dbe 0 2024-02-19T19:06:32.683 INFO:tasks.workunit.client.0.smithi088.stdout:0/777: dwrite d13/f37 [0,4194304] 0 2024-02-19T19:06:32.684 INFO:tasks.workunit.client.0.smithi088.stdout:7/849: dread - d14/d1d/d3c/d43/d5c/d75/d76/fb3 zero size 2024-02-19T19:06:32.685 INFO:tasks.workunit.client.0.smithi088.stdout:7/850: chown d14/d1d/d62/c71 46340 1 2024-02-19T19:06:32.686 INFO:tasks.workunit.client.0.smithi088.stdout:7/851: write d14/d27/d49/fa2 [1468791,122277] 0 2024-02-19T19:06:32.687 INFO:tasks.workunit.client.0.smithi088.stdout:0/778: rename d13/d62/d8a/d95/d5a/d25/dd6/df9 to d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/d10a 0 2024-02-19T19:06:32.692 INFO:tasks.workunit.client.0.smithi088.stdout:0/779: getdents d13/d62/d8a/d95/d5b/d76 0 2024-02-19T19:06:32.694 INFO:tasks.workunit.client.0.smithi088.stdout:7/852: write d14/f2e [1024035,117629] 0 2024-02-19T19:06:32.694 INFO:tasks.workunit.client.0.smithi088.stdout:0/780: chown d13/d62/d8a/d95/d5a/d25/la1 5381 1 2024-02-19T19:06:32.696 INFO:tasks.workunit.client.0.smithi088.stdout:7/853: readlink d14/d28/d94/lb4 0 2024-02-19T19:06:32.697 INFO:tasks.workunit.client.0.smithi088.stdout:7/854: write d14/d27/d49/f117 [1040925,107551] 0 2024-02-19T19:06:32.698 INFO:tasks.workunit.client.0.smithi088.stdout:7/855: chown d14/d1d/d3c/d43/d5c/d75 13201664 1 2024-02-19T19:06:32.702 INFO:tasks.workunit.client.0.smithi088.stdout:0/781: write d13/d62/d8a/d95/d5a/f80 [546534,28758] 0 2024-02-19T19:06:32.703 INFO:tasks.workunit.client.0.smithi088.stdout:7/856: dread d14/d27/d6b/dba/d102/ff6 [0,4194304] 0 2024-02-19T19:06:32.704 INFO:tasks.workunit.client.0.smithi088.stdout:7/857: chown d14/d28/d3b/dd3/f10e 435145156 1 2024-02-19T19:06:32.707 INFO:tasks.workunit.client.0.smithi088.stdout:0/782: rename d13/lb9 to d13/l10b 0 2024-02-19T19:06:32.708 INFO:tasks.workunit.client.0.smithi088.stdout:7/858: mkdir d14/d1d/d62/d12a 0 2024-02-19T19:06:32.709 INFO:tasks.workunit.client.0.smithi088.stdout:0/783: symlink d13/de9/l10c 0 2024-02-19T19:06:32.711 INFO:tasks.workunit.client.0.smithi088.stdout:7/859: dread d14/f22 [0,4194304] 0 2024-02-19T19:06:32.712 INFO:tasks.workunit.client.0.smithi088.stdout:7/860: write d14/d27/d6b/dba/d102/ff6 [90542,110966] 0 2024-02-19T19:06:32.717 INFO:tasks.workunit.client.0.smithi088.stdout:7/861: write d14/d28/d3b/f81 [1076421,79185] 0 2024-02-19T19:06:32.724 INFO:tasks.workunit.client.0.smithi088.stdout:7/862: dwrite d14/d28/d3b/dd3/f10e [0,4194304] 0 2024-02-19T19:06:32.734 INFO:tasks.workunit.client.0.smithi088.stdout:7/863: dwrite d14/d28/d3b/f64 [0,4194304] 0 2024-02-19T19:06:32.736 INFO:tasks.workunit.client.0.smithi088.stdout:7/864: chown d14/d27/c24 29 1 2024-02-19T19:06:32.737 INFO:tasks.workunit.client.0.smithi088.stdout:7/865: chown d14/d27/d49/fed 102426 1 2024-02-19T19:06:32.740 INFO:tasks.workunit.client.0.smithi088.stdout:7/866: symlink d14/d27/d49/d98/dcb/de7/l12b 0 2024-02-19T19:06:32.741 INFO:tasks.workunit.client.0.smithi088.stdout:7/867: stat d14/d28/d94/ddb 0 2024-02-19T19:06:32.743 INFO:tasks.workunit.client.0.smithi088.stdout:7/868: rename d14/d27/d6b/dba/d102/lf0 to d14/d27/d49/d98/d107/l12c 0 2024-02-19T19:06:32.745 INFO:tasks.workunit.client.0.smithi088.stdout:7/869: mknod d14/d27/d49/c12d 0 2024-02-19T19:06:32.747 INFO:tasks.workunit.client.0.smithi088.stdout:7/870: creat d14/d1d/d62/dfa/f12e x:0 0 0 2024-02-19T19:06:32.830 INFO:tasks.workunit.client.0.smithi088.stdout:4/597: sync 2024-02-19T19:06:32.830 INFO:tasks.workunit.client.0.smithi088.stdout:3/837: sync 2024-02-19T19:06:32.836 INFO:tasks.workunit.client.0.smithi088.stdout:4/598: write d2/d8/f29 [2695731,87066] 0 2024-02-19T19:06:32.838 INFO:tasks.workunit.client.0.smithi088.stdout:4/599: mknod d2/d32/d25/d26/da0/cc1 0 2024-02-19T19:06:32.839 INFO:tasks.workunit.client.0.smithi088.stdout:3/838: dwrite d4/f2d [0,4194304] 0 2024-02-19T19:06:32.850 INFO:tasks.workunit.client.0.smithi088.stdout:3/839: dwrite d4/d1a/d23/f5d [0,4194304] 0 2024-02-19T19:06:32.851 INFO:tasks.workunit.client.0.smithi088.stdout:4/600: dwrite d2/d32/d25/d26/d6d/d7e/f6c [0,4194304] 0 2024-02-19T19:06:32.858 INFO:tasks.workunit.client.0.smithi088.stdout:4/601: rename d2/d8/f16 to d2/d32/d25/d26/d4e/d6f/fc2 0 2024-02-19T19:06:32.863 INFO:tasks.workunit.client.0.smithi088.stdout:3/840: rmdir d4/d1a/d1e 39 2024-02-19T19:06:32.865 INFO:tasks.workunit.client.0.smithi088.stdout:4/602: link d2/d32/d25/d26/d4e/c81 d2/d32/d25/d26/d6d/cc3 0 2024-02-19T19:06:32.866 INFO:tasks.workunit.client.0.smithi088.stdout:4/603: chown d2/d32/d25/d26/d6d/d7e/d90/f94 11 1 2024-02-19T19:06:32.866 INFO:tasks.workunit.client.0.smithi088.stdout:3/841: dread d4/f28 [0,4194304] 0 2024-02-19T19:06:32.867 INFO:tasks.workunit.client.0.smithi088.stdout:4/604: readlink d2/d32/d25/d26/d4e/d6f/lac 0 2024-02-19T19:06:32.871 INFO:tasks.workunit.client.0.smithi088.stdout:4/605: mkdir d2/dc4 0 2024-02-19T19:06:32.873 INFO:tasks.workunit.client.0.smithi088.stdout:3/842: dwrite d4/d1a/d1e/d31/d32/f102 [0,4194304] 0 2024-02-19T19:06:32.874 INFO:tasks.workunit.client.0.smithi088.stdout:4/606: chown d2/d32/d25/d26/d6d/d7e/d3a/c7b 1693247115 1 2024-02-19T19:06:32.879 INFO:tasks.workunit.client.0.smithi088.stdout:4/607: rename d2/f15 to d2/d35/db3/db7/fc5 0 2024-02-19T19:06:32.879 INFO:tasks.workunit.client.0.smithi088.stdout:3/843: link d4/d12/d33/d43/ddd/c105 d4/d1a/d1e/d31/d32/d55/c128 0 2024-02-19T19:06:32.880 INFO:tasks.workunit.client.0.smithi088.stdout:4/608: stat d2/d32/d25/l34 0 2024-02-19T19:06:32.882 INFO:tasks.workunit.client.0.smithi088.stdout:4/609: write d2/d32/d25/d3f/d70/f8b [211844,96546] 0 2024-02-19T19:06:32.885 INFO:tasks.workunit.client.0.smithi088.stdout:3/844: rename d4/d12/c89 to d4/d1a/d1e/d74/d7d/daa/dcb/dfe/c129 0 2024-02-19T19:06:32.886 INFO:tasks.workunit.client.0.smithi088.stdout:4/610: creat d2/d32/d25/d3f/d9a/fc6 x:0 0 0 2024-02-19T19:06:32.887 INFO:tasks.workunit.client.0.smithi088.stdout:3/845: dread d4/d22/d7a/d11a/d11b/fee [0,4194304] 0 2024-02-19T19:06:32.889 INFO:tasks.workunit.client.0.smithi088.stdout:4/611: creat d2/d32/d25/d26/da5/db2/fc7 x:0 0 0 2024-02-19T19:06:32.891 INFO:tasks.workunit.client.0.smithi088.stdout:4/612: creat d2/d32/d25/d3f/d70/fc8 x:0 0 0 2024-02-19T19:06:32.899 INFO:tasks.workunit.client.0.smithi088.stdout:3/846: write d4/d12/f6c [742821,38752] 0 2024-02-19T19:06:32.903 INFO:tasks.workunit.client.0.smithi088.stdout:3/847: unlink d4/d22/c42 0 2024-02-19T19:06:32.906 INFO:tasks.workunit.client.0.smithi088.stdout:4/613: dread d2/d8/f14 [0,4194304] 0 2024-02-19T19:06:32.907 INFO:tasks.workunit.client.0.smithi088.stdout:4/614: symlink d2/d35/lc9 0 2024-02-19T19:06:32.913 INFO:tasks.workunit.client.0.smithi088.stdout:4/615: dwrite d2/f4 [0,4194304] 0 2024-02-19T19:06:32.915 INFO:tasks.workunit.client.0.smithi088.stdout:3/848: dread d4/d1a/d7f/d88/fcd [0,4194304] 0 2024-02-19T19:06:32.916 INFO:tasks.workunit.client.0.smithi088.stdout:4/616: getdents d2/d32/d25/d4b 0 2024-02-19T19:06:32.923 INFO:tasks.workunit.client.0.smithi088.stdout:8/715: sync 2024-02-19T19:06:32.923 INFO:tasks.workunit.client.0.smithi088.stdout:4/617: dwrite d2/d32/d25/d26/d6d/d7e/d90/fab [0,4194304] 0 2024-02-19T19:06:32.923 INFO:tasks.workunit.client.0.smithi088.stdout:6/882: sync 2024-02-19T19:06:32.923 INFO:tasks.workunit.client.0.smithi088.stdout:1/836: sync 2024-02-19T19:06:32.924 INFO:tasks.workunit.client.0.smithi088.stdout:6/883: readlink d5/de/d97/led 0 2024-02-19T19:06:32.931 INFO:tasks.workunit.client.0.smithi088.stdout:1/837: unlink d6/c9 0 2024-02-19T19:06:32.932 INFO:tasks.workunit.client.0.smithi088.stdout:3/849: dread d4/d12/f1d [0,4194304] 0 2024-02-19T19:06:32.936 INFO:tasks.workunit.client.0.smithi088.stdout:4/618: getdents d2 0 2024-02-19T19:06:32.939 INFO:tasks.workunit.client.0.smithi088.stdout:1/838: dread - d6/d21/d64/fe5 zero size 2024-02-19T19:06:32.939 INFO:tasks.workunit.client.0.smithi088.stdout:8/716: dwrite f5 [0,4194304] 0 2024-02-19T19:06:32.941 INFO:tasks.workunit.client.0.smithi088.stdout:1/839: chown d6/d10/f3b 1018938175 1 2024-02-19T19:06:32.941 INFO:tasks.workunit.client.0.smithi088.stdout:6/884: dwrite d5/d8/d2c/f4c [0,4194304] 0 2024-02-19T19:06:32.942 INFO:tasks.workunit.client.0.smithi088.stdout:4/619: mknod d2/d32/d9/cca 0 2024-02-19T19:06:32.943 INFO:tasks.workunit.client.0.smithi088.stdout:3/850: truncate d4/d22/d7a/f9f 1103237 0 2024-02-19T19:06:32.947 INFO:tasks.workunit.client.0.smithi088.stdout:6/885: mknod d5/d8/d3a/d71/daa/dc2/dbc/db6/c11a 0 2024-02-19T19:06:32.951 INFO:tasks.workunit.client.0.smithi088.stdout:4/620: write d2/d35/f7c [1017227,106016] 0 2024-02-19T19:06:32.953 INFO:tasks.workunit.client.0.smithi088.stdout:8/717: rename d9/df/d1c/d85/daf to d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8 0 2024-02-19T19:06:32.954 INFO:tasks.workunit.client.0.smithi088.stdout:6/886: mkdir d5/de/d64/d8d/d95/dba/d11b 0 2024-02-19T19:06:32.956 INFO:tasks.workunit.client.0.smithi088.stdout:1/840: creat d6/d21/f111 x:0 0 0 2024-02-19T19:06:32.958 INFO:tasks.workunit.client.0.smithi088.stdout:6/887: creat d5/d8/d3a/d71/daa/dc2/dbc/f11c x:0 0 0 2024-02-19T19:06:32.960 INFO:tasks.workunit.client.0.smithi088.stdout:8/718: dwrite d9/df/fb4 [0,4194304] 0 2024-02-19T19:06:32.962 INFO:tasks.workunit.client.0.smithi088.stdout:6/888: creat d5/de/d64/d8d/f11d x:0 0 0 2024-02-19T19:06:32.962 INFO:tasks.workunit.client.0.smithi088.stdout:1/841: getdents d6/d10 0 2024-02-19T19:06:32.967 INFO:tasks.workunit.client.0.smithi088.stdout:8/719: dread d9/df/d3d/d50/d71/f8b [0,4194304] 0 2024-02-19T19:06:32.970 INFO:tasks.workunit.client.0.smithi088.stdout:8/720: stat d9/df/d3d/d50/d71/da1/d70/fad 0 2024-02-19T19:06:32.971 INFO:tasks.workunit.client.0.smithi088.stdout:8/721: truncate d9/fc9 482047 0 2024-02-19T19:06:32.973 INFO:tasks.workunit.client.0.smithi088.stdout:6/889: getdents d5/de/d60/db1 0 2024-02-19T19:06:32.973 INFO:tasks.workunit.client.0.smithi088.stdout:8/722: write d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/dbe/fd6 [520798,83320] 0 2024-02-19T19:06:32.974 INFO:tasks.workunit.client.0.smithi088.stdout:8/723: readlink d9/df/l4c 0 2024-02-19T19:06:32.976 INFO:tasks.workunit.client.0.smithi088.stdout:3/851: write d4/d22/d7a/f9f [968412,386] 0 2024-02-19T19:06:32.977 INFO:tasks.workunit.client.0.smithi088.stdout:3/852: write d4/d87/fb6 [1596528,129850] 0 2024-02-19T19:06:32.978 INFO:tasks.workunit.client.0.smithi088.stdout:6/890: unlink d5/d7/dc9/dca/fee 0 2024-02-19T19:06:32.981 INFO:tasks.workunit.client.0.smithi088.stdout:8/724: mknod d9/df/d3d/d50/d71/da1/d6c/dcf/cd9 0 2024-02-19T19:06:32.982 INFO:tasks.workunit.client.0.smithi088.stdout:6/891: truncate d5/de/f68 2323118 0 2024-02-19T19:06:32.984 INFO:tasks.workunit.client.0.smithi088.stdout:1/842: dwrite d6/d21/d64/d94/d71/f9e [0,4194304] 0 2024-02-19T19:06:32.985 INFO:tasks.workunit.client.0.smithi088.stdout:3/853: creat d4/d1a/f12a x:0 0 0 2024-02-19T19:06:32.985 INFO:tasks.workunit.client.0.smithi088.stdout:1/843: fsync d6/d21/f3a 0 2024-02-19T19:06:32.987 INFO:tasks.workunit.client.0.smithi088.stdout:6/892: creat d5/d7/dc9/f11e x:0 0 0 2024-02-19T19:06:32.989 INFO:tasks.workunit.client.0.smithi088.stdout:6/893: write d5/d8/d3a/d71/daa/dc2/dbc/f11c [843833,27606] 0 2024-02-19T19:06:32.992 INFO:tasks.workunit.client.0.smithi088.stdout:1/844: link d6/d10/f4b d6/d21/d64/d94/d6e/f112 0 2024-02-19T19:06:32.994 INFO:tasks.workunit.client.0.smithi088.stdout:1/845: stat d6/d10/d60/db8/caa 0 2024-02-19T19:06:32.994 INFO:tasks.workunit.client.0.smithi088.stdout:3/854: dwrite d4/d22/d7a/ff0 [0,4194304] 0 2024-02-19T19:06:32.994 INFO:tasks.workunit.client.0.smithi088.stdout:6/894: readlink d5/d8/l1d 0 2024-02-19T19:06:32.998 INFO:tasks.workunit.client.0.smithi088.stdout:3/855: symlink d4/d22/d7a/d11a/l12b 0 2024-02-19T19:06:33.000 INFO:tasks.workunit.client.0.smithi088.stdout:6/895: mknod d5/de/d64/d82/c11f 0 2024-02-19T19:06:33.002 INFO:tasks.workunit.client.0.smithi088.stdout:6/896: write d5/d8/d3a/d71/daa/dc2/dbc/f11c [1554689,80710] 0 2024-02-19T19:06:33.009 INFO:tasks.workunit.client.0.smithi088.stdout:1/846: rmdir d6 39 2024-02-19T19:06:33.012 INFO:tasks.workunit.client.0.smithi088.stdout:6/897: unlink d5/d19/d26/cfe 0 2024-02-19T19:06:33.013 INFO:tasks.workunit.client.0.smithi088.stdout:6/898: write d5/d8/d3a/d71/daa/dc2/fb3 [4122245,67451] 0 2024-02-19T19:06:33.015 INFO:tasks.workunit.client.0.smithi088.stdout:3/856: write d4/d1a/d1e/d74/d7d/daa/fd6 [104,123315] 0 2024-02-19T19:06:33.017 INFO:tasks.workunit.client.0.smithi088.stdout:1/847: truncate d6/d21/d6a/d4a/fa7 1811056 0 2024-02-19T19:06:33.018 INFO:tasks.workunit.client.0.smithi088.stdout:6/899: link d5/de/d64/f66 d5/d45/f120 0 2024-02-19T19:06:33.019 INFO:tasks.workunit.client.0.smithi088.stdout:6/900: stat d5/d8/d2c 0 2024-02-19T19:06:33.021 INFO:tasks.workunit.client.0.smithi088.stdout:3/857: getdents d4/d12/d33/d43/d85/d10a 0 2024-02-19T19:06:33.023 INFO:tasks.workunit.client.0.smithi088.stdout:3/858: write d4/d12/d33/d43/fbd [9267566,111789] 0 2024-02-19T19:06:33.023 INFO:tasks.workunit.client.0.smithi088.stdout:6/901: symlink d5/d7/l121 0 2024-02-19T19:06:33.024 INFO:tasks.workunit.client.0.smithi088.stdout:1/848: dread d6/d21/d6a/d7d/d7e/fac [0,4194304] 0 2024-02-19T19:06:33.025 INFO:tasks.workunit.client.0.smithi088.stdout:3/859: creat d4/d12/d33/d58/f12c x:0 0 0 2024-02-19T19:06:33.027 INFO:tasks.workunit.client.0.smithi088.stdout:6/902: rename d5/d7/dc9/f11e to d5/de/d64/d8d/d95/dba/f122 0 2024-02-19T19:06:33.027 INFO:tasks.workunit.client.0.smithi088.stdout:6/903: dread - d5/d19/d26/d72/ff1 zero size 2024-02-19T19:06:33.028 INFO:tasks.workunit.client.0.smithi088.stdout:3/860: creat d4/d12/d33/d43/d85/f12d x:0 0 0 2024-02-19T19:06:33.028 INFO:tasks.workunit.client.0.smithi088.stdout:6/904: write d5/d8/d77/f102 [364759,91036] 0 2024-02-19T19:06:33.029 INFO:tasks.workunit.client.0.smithi088.stdout:3/861: read - d4/d1a/d1e/d74/d7d/daa/f90 zero size 2024-02-19T19:06:33.029 INFO:tasks.workunit.client.0.smithi088.stdout:6/905: chown d5/de/d64/d82 194598 1 2024-02-19T19:06:33.031 INFO:tasks.workunit.client.0.smithi088.stdout:6/906: mkdir d5/d7/d32/d56/d123 0 2024-02-19T19:06:33.032 INFO:tasks.workunit.client.0.smithi088.stdout:6/907: stat d5/d8/d3a/d71/daa/dc2/dbc/f11c 0 2024-02-19T19:06:33.033 INFO:tasks.workunit.client.0.smithi088.stdout:6/908: write d5/de/fc6 [369476,122307] 0 2024-02-19T19:06:33.033 INFO:tasks.workunit.client.0.smithi088.stdout:3/862: rename d4/d1a/d1e/d74/d7c/d97/ca5 to d4/d22/d7a/da1/d10d/c12e 0 2024-02-19T19:06:33.036 INFO:tasks.workunit.client.0.smithi088.stdout:6/909: read d5/d8/d3a/d71/daa/dc2/fb3 [4042369,106163] 0 2024-02-19T19:06:33.038 INFO:tasks.workunit.client.0.smithi088.stdout:6/910: symlink d5/d7/d32/d56/d123/l124 0 2024-02-19T19:06:33.039 INFO:tasks.workunit.client.0.smithi088.stdout:3/863: getdents d4/d1a/d1e/d31/dc5/dcf 0 2024-02-19T19:06:33.040 INFO:tasks.workunit.client.0.smithi088.stdout:3/864: readlink d4/d1a/d1e/l62 0 2024-02-19T19:06:33.040 INFO:tasks.workunit.client.0.smithi088.stdout:6/911: creat d5/d7/dc9/d109/f125 x:0 0 0 2024-02-19T19:06:33.043 INFO:tasks.workunit.client.0.smithi088.stdout:3/865: getdents d4/d1a/d1e/d74/d7d 0 2024-02-19T19:06:33.046 INFO:tasks.workunit.client.0.smithi088.stdout:3/866: dread d4/d22/d7a/d8d/f9d [0,4194304] 0 2024-02-19T19:06:33.047 INFO:tasks.workunit.client.0.smithi088.stdout:6/912: dwrite d5/d8/d77/f111 [0,4194304] 0 2024-02-19T19:06:33.048 INFO:tasks.workunit.client.0.smithi088.stdout:6/913: chown d5/de/d60/db1/ff9 48 1 2024-02-19T19:06:33.049 INFO:tasks.workunit.client.0.smithi088.stdout:3/867: mkdir d4/d12/d33/d12f 0 2024-02-19T19:06:33.052 INFO:tasks.workunit.client.0.smithi088.stdout:6/914: mknod d5/d8/d3a/d71/daa/dc2/c126 0 2024-02-19T19:06:33.054 INFO:tasks.workunit.client.0.smithi088.stdout:3/868: dread - d4/d12/d33/d43/d85/fae zero size 2024-02-19T19:06:33.054 INFO:tasks.workunit.client.0.smithi088.stdout:6/915: write d5/de/d64/d7e/f118 [213993,60424] 0 2024-02-19T19:06:33.056 INFO:tasks.workunit.client.0.smithi088.stdout:3/869: read d4/f38 [1625783,50789] 0 2024-02-19T19:06:33.060 INFO:tasks.workunit.client.0.smithi088.stdout:3/870: dread - d4/d22/d7a/fa7 zero size 2024-02-19T19:06:33.063 INFO:tasks.workunit.client.0.smithi088.stdout:6/916: dwrite d5/d8/d77/f100 [0,4194304] 0 2024-02-19T19:06:33.063 INFO:tasks.workunit.client.0.smithi088.stdout:3/871: symlink d4/d100/l130 0 2024-02-19T19:06:33.064 INFO:tasks.workunit.client.0.smithi088.stdout:3/872: write d4/d1a/f115 [752867,87277] 0 2024-02-19T19:06:33.065 INFO:tasks.workunit.client.0.smithi088.stdout:6/917: write d5/de/d64/d7e/f118 [714407,103088] 0 2024-02-19T19:06:33.073 INFO:tasks.workunit.client.0.smithi088.stdout:6/918: write d5/d7/f4b [410070,3076] 0 2024-02-19T19:06:33.075 INFO:tasks.workunit.client.0.smithi088.stdout:6/919: creat d5/d7/dc9/f127 x:0 0 0 2024-02-19T19:06:33.078 INFO:tasks.workunit.client.0.smithi088.stdout:6/920: read d5/d8/f3f [999335,32880] 0 2024-02-19T19:06:33.078 INFO:tasks.workunit.client.0.smithi088.stdout:3/873: dwrite d4/d1a/d1e/d31/dc5/dcf/da4/fbb [0,4194304] 0 2024-02-19T19:06:33.079 INFO:tasks.workunit.client.0.smithi088.stdout:2/722: sync 2024-02-19T19:06:33.079 INFO:tasks.workunit.client.0.smithi088.stdout:7/871: sync 2024-02-19T19:06:33.079 INFO:tasks.workunit.client.0.smithi088.stdout:0/784: sync 2024-02-19T19:06:33.081 INFO:tasks.workunit.client.0.smithi088.stdout:3/874: readlink d4/d12/d33/dab/l106 0 2024-02-19T19:06:33.082 INFO:tasks.workunit.client.0.smithi088.stdout:2/723: chown d3/d1b/d43/d61/d64/f78 1061 1 2024-02-19T19:06:33.086 INFO:tasks.workunit.client.0.smithi088.stdout:7/872: rename d14/d28/d3b/f4b to d14/d28/d94/ddb/d115/f12f 0 2024-02-19T19:06:33.087 INFO:tasks.workunit.client.0.smithi088.stdout:7/873: stat d14/d27/f36 0 2024-02-19T19:06:33.089 INFO:tasks.workunit.client.0.smithi088.stdout:3/875: creat d4/d12/d33/d58/f131 x:0 0 0 2024-02-19T19:06:33.090 INFO:tasks.workunit.client.0.smithi088.stdout:7/874: stat d14/c46 0 2024-02-19T19:06:33.091 INFO:tasks.workunit.client.0.smithi088.stdout:6/921: getdents d5/d45 0 2024-02-19T19:06:33.091 INFO:tasks.workunit.client.0.smithi088.stdout:7/875: stat d14/ff9 0 2024-02-19T19:06:33.091 INFO:tasks.workunit.client.0.smithi088.stdout:2/724: dwrite d3/d1f/d4f/f17 [0,4194304] 0 2024-02-19T19:06:33.097 INFO:tasks.workunit.client.0.smithi088.stdout:2/725: mknod d3/d14/dd8/cef 0 2024-02-19T19:06:33.098 INFO:tasks.workunit.client.0.smithi088.stdout:0/785: dwrite d13/d62/d8a/d95/d5b/f108 [0,4194304] 0 2024-02-19T19:06:33.098 INFO:tasks.workunit.client.0.smithi088.stdout:3/876: symlink d4/d1a/d1e/d74/d7d/daa/dcb/l132 0 2024-02-19T19:06:33.100 INFO:tasks.workunit.client.0.smithi088.stdout:2/726: dread - d3/fa5 zero size 2024-02-19T19:06:33.102 INFO:tasks.workunit.client.0.smithi088.stdout:6/922: creat d5/de/d64/d8d/d9e/d10f/f128 x:0 0 0 2024-02-19T19:06:33.107 INFO:tasks.workunit.client.0.smithi088.stdout:7/876: link d14/d1d/d3c/d43/d5c/d9a/d92/cac d14/d28/d94/c130 0 2024-02-19T19:06:33.113 INFO:tasks.workunit.client.0.smithi088.stdout:6/923: dwrite d5/de/d64/d8d/d95/f117 [0,4194304] 0 2024-02-19T19:06:33.114 INFO:tasks.workunit.client.0.smithi088.stdout:2/727: link d3/f6 d3/ff0 0 2024-02-19T19:06:33.114 INFO:tasks.workunit.client.0.smithi088.stdout:3/877: dwrite d4/d22/fef [4194304,4194304] 0 2024-02-19T19:06:33.116 INFO:tasks.workunit.client.0.smithi088.stdout:2/728: chown d3/d14/l60 22 1 2024-02-19T19:06:33.116 INFO:tasks.workunit.client.0.smithi088.stdout:0/786: dread f3 [0,4194304] 0 2024-02-19T19:06:33.123 INFO:tasks.workunit.client.0.smithi088.stdout:7/877: link d14/f44 d14/d1d/d3c/d43/d5c/da6/f131 0 2024-02-19T19:06:33.126 INFO:tasks.workunit.client.0.smithi088.stdout:2/729: link d3/d1f/fa2 d3/d1b/d43/d8f/ff1 0 2024-02-19T19:06:33.128 INFO:tasks.workunit.client.0.smithi088.stdout:0/787: rename d13/d62/d8a/d95/d5b/d76/fb8 to d13/d62/d8a/d95/d5a/df3/f10d 0 2024-02-19T19:06:33.129 INFO:tasks.workunit.client.0.smithi088.stdout:6/924: dwrite d5/d45/d9d/fa2 [0,4194304] 0 2024-02-19T19:06:33.130 INFO:tasks.workunit.client.0.smithi088.stdout:3/878: dwrite d4/d12/d33/d43/f65 [0,4194304] 0 2024-02-19T19:06:33.132 INFO:tasks.workunit.client.0.smithi088.stdout:3/879: dread - d4/d22/d7a/f10e zero size 2024-02-19T19:06:33.134 INFO:tasks.workunit.client.0.smithi088.stdout:6/925: mkdir d5/d7/dc9/dca/d129 0 2024-02-19T19:06:33.136 INFO:tasks.workunit.client.0.smithi088.stdout:3/880: readlink d4/d87/dbf/ld0 0 2024-02-19T19:06:33.138 INFO:tasks.workunit.client.0.smithi088.stdout:2/730: symlink d3/lf2 0 2024-02-19T19:06:33.139 INFO:tasks.workunit.client.0.smithi088.stdout:6/926: mkdir d5/d7/dc9/d12a 0 2024-02-19T19:06:33.143 INFO:tasks.workunit.client.0.smithi088.stdout:2/731: creat d3/d1f/d5d/de1/ff3 x:0 0 0 2024-02-19T19:06:33.147 INFO:tasks.workunit.client.0.smithi088.stdout:7/878: getdents d14/d27/d6b/dba 0 2024-02-19T19:06:33.149 INFO:tasks.workunit.client.0.smithi088.stdout:3/881: getdents d4/d1a/d1e/d74/d7d/daa/dcb/dfe 0 2024-02-19T19:06:33.151 INFO:tasks.workunit.client.0.smithi088.stdout:2/732: rename d3/d1b/d43/d61/d86 to d3/d1b/d43/df4 0 2024-02-19T19:06:33.151 INFO:tasks.workunit.client.0.smithi088.stdout:0/788: dwrite d13/d62/d8a/d95/d5a/d25/f27 [0,4194304] 0 2024-02-19T19:06:33.153 INFO:tasks.workunit.client.0.smithi088.stdout:7/879: mkdir d14/d27/d6b/dba/d102/d132 0 2024-02-19T19:06:33.153 INFO:tasks.workunit.client.0.smithi088.stdout:3/882: symlink d4/d22/d7a/da1/d10d/d11f/l133 0 2024-02-19T19:06:33.154 INFO:tasks.workunit.client.0.smithi088.stdout:2/733: rmdir d3/d1b/d43/d74 39 2024-02-19T19:06:33.155 INFO:tasks.workunit.client.0.smithi088.stdout:7/880: write d14/d28/d3b/f64 [1447239,13119] 0 2024-02-19T19:06:33.156 INFO:tasks.workunit.client.0.smithi088.stdout:2/734: write d3/d1b/f1c [3808776,58660] 0 2024-02-19T19:06:33.159 INFO:tasks.workunit.client.0.smithi088.stdout:3/883: symlink d4/d22/d7a/da1/l134 0 2024-02-19T19:06:33.160 INFO:tasks.workunit.client.0.smithi088.stdout:0/789: mkdir d13/d62/d8a/d95/d5a/d25/dd6/d105/d10e 0 2024-02-19T19:06:33.166 INFO:tasks.workunit.client.0.smithi088.stdout:6/927: dwrite d5/d19/d70/f80 [0,4194304] 0 2024-02-19T19:06:33.167 INFO:tasks.workunit.client.0.smithi088.stdout:0/790: rmdir d13/d62/d8a/d9c/da8/dab 39 2024-02-19T19:06:33.168 INFO:tasks.workunit.client.0.smithi088.stdout:2/735: rename d3/c29 to d3/d14/dd8/cf5 0 2024-02-19T19:06:33.169 INFO:tasks.workunit.client.0.smithi088.stdout:0/791: readlink d13/d62/d8a/lc7 0 2024-02-19T19:06:33.173 INFO:tasks.workunit.client.0.smithi088.stdout:3/884: getdents d4/d22/d7a/d8d 0 2024-02-19T19:06:33.178 INFO:tasks.workunit.client.0.smithi088.stdout:3/885: chown d4/d1a/d1e/d74/d7d/daa/dcb 254 1 2024-02-19T19:06:33.178 INFO:tasks.workunit.client.0.smithi088.stdout:6/928: dwrite d5/de/d60/db1/ff9 [0,4194304] 0 2024-02-19T19:06:33.179 INFO:tasks.workunit.client.0.smithi088.stdout:3/886: dread - d4/d12/d33/d58/f12c zero size 2024-02-19T19:06:33.180 INFO:tasks.workunit.client.0.smithi088.stdout:0/792: rename d13/d62/d8a/d95/d5a/f4b to d13/d62/d8a/d95/d96/ded/f10f 0 2024-02-19T19:06:33.182 INFO:tasks.workunit.client.0.smithi088.stdout:0/793: write d13/d62/d8a/d95/d5a/f80 [1264791,29935] 0 2024-02-19T19:06:33.183 INFO:tasks.workunit.client.0.smithi088.stdout:3/887: symlink d4/d1a/d1e/d31/d32/l135 0 2024-02-19T19:06:33.187 INFO:tasks.workunit.client.0.smithi088.stdout:3/888: creat d4/d1a/d1e/d31/d32/f136 x:0 0 0 2024-02-19T19:06:33.189 INFO:tasks.workunit.client.0.smithi088.stdout:3/889: chown d4/d22/d7a/f9f 227 1 2024-02-19T19:06:33.190 INFO:tasks.workunit.client.0.smithi088.stdout:6/929: truncate d5/de/d60/f87 2056938 0 2024-02-19T19:06:33.191 INFO:tasks.workunit.client.0.smithi088.stdout:0/794: dwrite d13/d62/d8a/d95/d5a/f80 [0,4194304] 0 2024-02-19T19:06:33.195 INFO:tasks.workunit.client.0.smithi088.stdout:0/795: dread - d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/d10a/d106/fce zero size 2024-02-19T19:06:33.196 INFO:tasks.workunit.client.0.smithi088.stdout:7/881: dread d14/d28/d3b/f52 [0,4194304] 0 2024-02-19T19:06:33.201 INFO:tasks.workunit.client.0.smithi088.stdout:7/882: chown fb 8035463 1 2024-02-19T19:06:33.203 INFO:tasks.workunit.client.0.smithi088.stdout:6/930: dwrite d5/d8/d3a/d71/daa/dc2/dbc/db6/f10a [0,4194304] 0 2024-02-19T19:06:33.204 INFO:tasks.workunit.client.0.smithi088.stdout:7/883: fsync d14/d28/d94/ddb/ff1 0 2024-02-19T19:06:33.205 INFO:tasks.workunit.client.0.smithi088.stdout:4/621: sync 2024-02-19T19:06:33.207 INFO:tasks.workunit.client.0.smithi088.stdout:6/931: symlink d5/de/d64/d8d/l12b 0 2024-02-19T19:06:33.209 INFO:tasks.workunit.client.0.smithi088.stdout:0/796: rename d13/d62/d8a/d95/d96/lb3 to d13/d62/d8a/d95/d5a/d25/d2a/d34/d100/l110 0 2024-02-19T19:06:33.211 INFO:tasks.workunit.client.0.smithi088.stdout:0/797: write d13/f37 [3674062,86478] 0 2024-02-19T19:06:33.217 INFO:tasks.workunit.client.0.smithi088.stdout:6/932: symlink d5/d19/d70/df3/l12c 0 2024-02-19T19:06:33.217 INFO:tasks.workunit.client.0.smithi088.stdout:2/736: dread d3/f85 [0,4194304] 0 2024-02-19T19:06:33.218 INFO:tasks.workunit.client.0.smithi088.stdout:4/622: dwrite d2/d32/d25/d26/d4e/d6f/f9d [0,4194304] 0 2024-02-19T19:06:33.225 INFO:tasks.workunit.client.0.smithi088.stdout:3/890: dread d4/d22/f2c [0,4194304] 0 2024-02-19T19:06:33.226 INFO:tasks.workunit.client.0.smithi088.stdout:6/933: write d5/d8/d3a/d71/daa/f10e [778651,62817] 0 2024-02-19T19:06:33.227 INFO:tasks.workunit.client.0.smithi088.stdout:6/934: stat d5/d19/f5f 0 2024-02-19T19:06:33.230 INFO:tasks.workunit.client.0.smithi088.stdout:4/623: creat d2/d32/d25/d26/d6d/d7e/d3a/d77/d99/fcb x:0 0 0 2024-02-19T19:06:33.230 INFO:tasks.workunit.client.0.smithi088.stdout:0/798: dwrite d13/d62/d8a/d9c/da8/de2/fd5 [0,4194304] 0 2024-02-19T19:06:33.231 INFO:tasks.workunit.client.0.smithi088.stdout:2/737: unlink d3/d1b/d43/d61/d7e/db6/fbc 0 2024-02-19T19:06:33.233 INFO:tasks.workunit.client.0.smithi088.stdout:7/884: dread d14/f4f [0,4194304] 0 2024-02-19T19:06:33.236 INFO:tasks.workunit.client.0.smithi088.stdout:3/891: fdatasync d4/d1a/d1e/d31/d32/f77 0 2024-02-19T19:06:33.240 INFO:tasks.workunit.client.0.smithi088.stdout:0/799: symlink d13/d62/d8a/d95/d5a/d25/d2a/d2f/l111 0 2024-02-19T19:06:33.241 INFO:tasks.workunit.client.0.smithi088.stdout:2/738: dwrite d3/d1f/d4f/dc6/fe0 [0,4194304] 0 2024-02-19T19:06:33.241 INFO:tasks.workunit.client.0.smithi088.stdout:7/885: creat d14/d1d/d62/dcf/f133 x:0 0 0 2024-02-19T19:06:33.243 INFO:tasks.workunit.client.0.smithi088.stdout:4/624: mknod d2/d35/ccc 0 2024-02-19T19:06:33.243 INFO:tasks.workunit.client.0.smithi088.stdout:4/625: chown d2/d32/d25/d4b 31537 1 2024-02-19T19:06:33.243 INFO:tasks.workunit.client.0.smithi088.stdout:4/626: stat d2 0 2024-02-19T19:06:33.246 INFO:tasks.workunit.client.0.smithi088.stdout:8/725: sync 2024-02-19T19:06:33.246 INFO:tasks.workunit.client.0.smithi088.stdout:1/849: sync 2024-02-19T19:06:33.249 INFO:tasks.workunit.client.0.smithi088.stdout:6/935: fsync d5/d8/d3a/d71/daa/dc2/dbc/db6/f10a 0 2024-02-19T19:06:33.252 INFO:tasks.workunit.client.0.smithi088.stdout:0/800: creat d13/d62/d8a/d9c/f112 x:0 0 0 2024-02-19T19:06:33.254 INFO:tasks.workunit.client.0.smithi088.stdout:2/739: stat d3/d14/dd8/lba 0 2024-02-19T19:06:33.255 INFO:tasks.workunit.client.0.smithi088.stdout:4/627: truncate d2/d32/d25/d26/f88 166529 0 2024-02-19T19:06:33.261 INFO:tasks.workunit.client.0.smithi088.stdout:6/936: mkdir d5/d45/d9d/d108/d12d 0 2024-02-19T19:06:33.262 INFO:tasks.workunit.client.0.smithi088.stdout:3/892: write d4/f79 [1167184,21006] 0 2024-02-19T19:06:33.269 INFO:tasks.workunit.client.0.smithi088.stdout:6/937: fsync d5/d8/f3f 0 2024-02-19T19:06:33.269 INFO:tasks.workunit.client.0.smithi088.stdout:7/886: getdents d14/d1d/d3c/d43/db2 0 2024-02-19T19:06:33.271 INFO:tasks.workunit.client.0.smithi088.stdout:4/628: truncate d2/d32/d25/d26/d6d/d7e/f72 2424061 0 2024-02-19T19:06:33.275 INFO:tasks.workunit.client.0.smithi088.stdout:8/726: dread d9/df/f27 [4194304,4194304] 0 2024-02-19T19:06:33.277 INFO:tasks.workunit.client.0.smithi088.stdout:7/887: mkdir d14/d1d/d3c/d43/d5c/d9a/df4/d134 0 2024-02-19T19:06:33.278 INFO:tasks.workunit.client.0.smithi088.stdout:0/801: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d3c/fa4 [0,4194304] 0 2024-02-19T19:06:33.279 INFO:tasks.workunit.client.0.smithi088.stdout:4/629: mknod d2/d8/ccd 0 2024-02-19T19:06:33.282 INFO:tasks.workunit.client.0.smithi088.stdout:7/888: symlink d14/d1d/d3c/d43/d5c/l135 0 2024-02-19T19:06:33.284 INFO:tasks.workunit.client.0.smithi088.stdout:4/630: symlink d2/dc4/lce 0 2024-02-19T19:06:33.285 INFO:tasks.workunit.client.0.smithi088.stdout:3/893: dread d4/f40 [0,4194304] 0 2024-02-19T19:06:33.286 INFO:tasks.workunit.client.0.smithi088.stdout:2/740: dwrite d3/d14/f1d [0,4194304] 0 2024-02-19T19:06:33.289 INFO:tasks.workunit.client.0.smithi088.stdout:6/938: dwrite d5/d19/d70/f80 [0,4194304] 0 2024-02-19T19:06:33.289 INFO:tasks.workunit.client.0.smithi088.stdout:8/727: rename d9/df/d1c/f29 to d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/fda 0 2024-02-19T19:06:33.293 INFO:tasks.workunit.client.0.smithi088.stdout:3/894: mkdir d4/d1a/d23/d137 0 2024-02-19T19:06:33.293 INFO:tasks.workunit.client.0.smithi088.stdout:0/802: rmdir d13/d62/d8a/d95/d5a/d25/dd6 39 2024-02-19T19:06:33.295 INFO:tasks.workunit.client.0.smithi088.stdout:2/741: rename d3/d1b/d43/d61/d7e/db1/cc1 to d3/d1b/d43/d61/d7e/db1/cf6 0 2024-02-19T19:06:33.296 INFO:tasks.workunit.client.0.smithi088.stdout:6/939: fdatasync d5/d7/d32/f8e 0 2024-02-19T19:06:33.297 INFO:tasks.workunit.client.0.smithi088.stdout:4/631: creat d2/d32/d25/d26/d6d/d7e/d3a/d77/fcf x:0 0 0 2024-02-19T19:06:33.299 INFO:tasks.workunit.client.0.smithi088.stdout:3/895: chown d4/d1a/d1e/d31/d32/d55/feb 148 1 2024-02-19T19:06:33.307 INFO:tasks.workunit.client.0.smithi088.stdout:8/728: dread d9/f4a [0,4194304] 0 2024-02-19T19:06:33.309 INFO:tasks.workunit.client.0.smithi088.stdout:7/889: write d14/d1d/d3c/d43/d5c/f69 [212416,85718] 0 2024-02-19T19:06:33.311 INFO:tasks.workunit.client.0.smithi088.stdout:6/940: creat d5/d8/d3a/d71/dc3/f12e x:0 0 0 2024-02-19T19:06:33.316 INFO:tasks.workunit.client.0.smithi088.stdout:4/632: dread d2/d32/d25/f38 [0,4194304] 0 2024-02-19T19:06:33.316 INFO:tasks.workunit.client.0.smithi088.stdout:3/896: read d4/d1a/d1e/d74/fa6 [246393,86992] 0 2024-02-19T19:06:33.316 INFO:tasks.workunit.client.0.smithi088.stdout:2/742: rmdir d3/d1b/d43/d61/d7e/db1 39 2024-02-19T19:06:33.316 INFO:tasks.workunit.client.0.smithi088.stdout:0/803: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d2f/f104 [0,4194304] 0 2024-02-19T19:06:33.317 INFO:tasks.workunit.client.0.smithi088.stdout:8/729: symlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/ldb 0 2024-02-19T19:06:33.322 INFO:tasks.workunit.client.0.smithi088.stdout:6/941: fsync d5/d8/d3a/d71/daa/dc2/fb3 0 2024-02-19T19:06:33.324 INFO:tasks.workunit.client.0.smithi088.stdout:6/942: fsync d5/d8/d3a/f63 0 2024-02-19T19:06:33.326 INFO:tasks.workunit.client.0.smithi088.stdout:2/743: unlink d3/d14/dd8/cf5 0 2024-02-19T19:06:33.326 INFO:tasks.workunit.client.0.smithi088.stdout:8/730: dwrite f5 [0,4194304] 0 2024-02-19T19:06:33.330 INFO:tasks.workunit.client.0.smithi088.stdout:6/943: stat d5/d19/fe1 0 2024-02-19T19:06:33.330 INFO:tasks.workunit.client.0.smithi088.stdout:2/744: rmdir d3/d1b/d43/df4 39 2024-02-19T19:06:33.332 INFO:tasks.workunit.client.0.smithi088.stdout:7/890: getdents d14/d1d/d3c/d43/db2/dc1 0 2024-02-19T19:06:33.332 INFO:tasks.workunit.client.0.smithi088.stdout:6/944: readlink d5/d8/d3a/l98 0 2024-02-19T19:06:33.333 INFO:tasks.workunit.client.0.smithi088.stdout:8/731: truncate d9/df/d3d/d50/d71/da1/d6c/d77/d97/fc2 763638 0 2024-02-19T19:06:33.336 INFO:tasks.workunit.client.0.smithi088.stdout:7/891: creat d14/d28/d94/ddb/d115/f136 x:0 0 0 2024-02-19T19:06:33.336 INFO:tasks.workunit.client.0.smithi088.stdout:0/804: dread d13/d62/d8a/d95/d5a/f74 [0,4194304] 0 2024-02-19T19:06:33.337 INFO:tasks.workunit.client.0.smithi088.stdout:6/945: creat d5/d8/d3a/d71/daa/dc2/dbc/f12f x:0 0 0 2024-02-19T19:06:33.339 INFO:tasks.workunit.client.0.smithi088.stdout:6/946: fsync d5/d8/d2c/f4c 0 2024-02-19T19:06:33.339 INFO:tasks.workunit.client.0.smithi088.stdout:2/745: rename d3/d1b/l6b to d3/d1b/d43/d61/d7e/db6/lf7 0 2024-02-19T19:06:33.340 INFO:tasks.workunit.client.0.smithi088.stdout:7/892: chown d14/d1d/d62/dfa/f12e 840270968 1 2024-02-19T19:06:33.342 INFO:tasks.workunit.client.0.smithi088.stdout:0/805: creat d13/d62/d8a/d95/d5a/df3/f113 x:0 0 0 2024-02-19T19:06:33.343 INFO:tasks.workunit.client.0.smithi088.stdout:2/746: read d3/d14/f56 [48709,119665] 0 2024-02-19T19:06:33.346 INFO:tasks.workunit.client.0.smithi088.stdout:8/732: dwrite d9/df/d3d/d50/d71/fd4 [0,4194304] 0 2024-02-19T19:06:33.346 INFO:tasks.workunit.client.0.smithi088.stdout:3/897: truncate d4/d1a/d7f/d88/fcd 2835892 0 2024-02-19T19:06:33.349 INFO:tasks.workunit.client.0.smithi088.stdout:4/633: write d2/d35/db3/db7/fc5 [1065986,42433] 0 2024-02-19T19:06:33.353 INFO:tasks.workunit.client.0.smithi088.stdout:6/947: rename d5/f47 to d5/d8/d3a/d71/dc3/f130 0 2024-02-19T19:06:33.355 INFO:tasks.workunit.client.0.smithi088.stdout:2/747: dwrite d3/fa5 [0,4194304] 0 2024-02-19T19:06:33.358 INFO:tasks.workunit.client.0.smithi088.stdout:8/733: mknod d9/df/d3d/d50/d71/da1/d70/cdc 0 2024-02-19T19:06:33.358 INFO:tasks.workunit.client.0.smithi088.stdout:3/898: write d4/d12/d33/d43/fd9 [314426,86352] 0 2024-02-19T19:06:33.368 INFO:tasks.workunit.client.0.smithi088.stdout:7/893: truncate d14/f2e 1453289 0 2024-02-19T19:06:33.369 INFO:tasks.workunit.client.0.smithi088.stdout:8/734: creat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/fdd x:0 0 0 2024-02-19T19:06:33.373 INFO:tasks.workunit.client.0.smithi088.stdout:6/948: truncate d5/d8/d3a/d71/daa/dc2/fb3 2188507 0 2024-02-19T19:06:33.375 INFO:tasks.workunit.client.0.smithi088.stdout:3/899: rename d4/d12/d33/d12f to d4/d22/dd5/d104/d138 0 2024-02-19T19:06:33.378 INFO:tasks.workunit.client.0.smithi088.stdout:7/894: dwrite d14/d1d/d3c/fb7 [4194304,4194304] 0 2024-02-19T19:06:33.380 INFO:tasks.workunit.client.0.smithi088.stdout:4/634: dread d2/d32/d25/d26/d6d/d7e/d90/f94 [0,4194304] 0 2024-02-19T19:06:33.380 INFO:tasks.workunit.client.0.smithi088.stdout:6/949: mkdir d5/d8/d77/d131 0 2024-02-19T19:06:33.381 INFO:tasks.workunit.client.0.smithi088.stdout:7/895: readlink d14/d27/l31 0 2024-02-19T19:06:33.382 INFO:tasks.workunit.client.0.smithi088.stdout:3/900: symlink d4/d22/d7a/l139 0 2024-02-19T19:06:33.385 INFO:tasks.workunit.client.0.smithi088.stdout:0/806: dwrite d13/d62/d8a/d87/f97 [0,4194304] 0 2024-02-19T19:06:33.385 INFO:tasks.workunit.client.0.smithi088.stdout:4/635: chown d2/d32/d25/d26/da0/cc1 256396 1 2024-02-19T19:06:33.386 INFO:tasks.workunit.client.0.smithi088.stdout:2/748: truncate d3/d1b/f1c 1432891 0 2024-02-19T19:06:33.388 INFO:tasks.workunit.client.0.smithi088.stdout:6/950: rmdir d5/d7/dc9 39 2024-02-19T19:06:33.395 INFO:tasks.workunit.client.0.smithi088.stdout:3/901: rmdir d4/d1a/d1e/d74/d7c 39 2024-02-19T19:06:33.395 INFO:tasks.workunit.client.0.smithi088.stdout:8/735: getdents d9/df/d3d/d50/d71/da1/d6c/d9a/d6b 0 2024-02-19T19:06:33.396 INFO:tasks.workunit.client.0.smithi088.stdout:2/749: dwrite d3/d1f/d5d/de1/fe5 [0,4194304] 0 2024-02-19T19:06:33.397 INFO:tasks.workunit.client.0.smithi088.stdout:0/807: rename f4 to d13/d62/d8a/de0/f114 0 2024-02-19T19:06:33.398 INFO:tasks.workunit.client.0.smithi088.stdout:2/750: dread - d3/d1b/d43/d61/d64/fed zero size 2024-02-19T19:06:33.398 INFO:tasks.workunit.client.0.smithi088.stdout:8/736: dread - d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/fdd zero size 2024-02-19T19:06:33.405 INFO:tasks.workunit.client.0.smithi088.stdout:6/951: read d5/de/d60/f87 [957140,46591] 0 2024-02-19T19:06:33.407 INFO:tasks.workunit.client.0.smithi088.stdout:3/902: fdatasync d4/f28 0 2024-02-19T19:06:33.411 INFO:tasks.workunit.client.0.smithi088.stdout:6/952: mknod d5/de/d97/c132 0 2024-02-19T19:06:33.412 INFO:tasks.workunit.client.0.smithi088.stdout:6/953: write d5/d19/d70/f106 [775780,3134] 0 2024-02-19T19:06:33.414 INFO:tasks.workunit.client.0.smithi088.stdout:3/903: unlink d4/d1a/d1e/d74/ff7 0 2024-02-19T19:06:33.416 INFO:tasks.workunit.client.0.smithi088.stdout:2/751: mknod d3/d1b/d43/d74/cf8 0 2024-02-19T19:06:33.416 INFO:tasks.workunit.client.0.smithi088.stdout:4/636: write d2/d32/d25/f30 [1187127,72992] 0 2024-02-19T19:06:33.416 INFO:tasks.workunit.client.0.smithi088.stdout:0/808: symlink d13/d62/d8a/d95/d5a/d25/l115 0 2024-02-19T19:06:33.417 INFO:tasks.workunit.client.0.smithi088.stdout:4/637: dread - d2/d32/d25/d26/d4e/d6f/fa6 zero size 2024-02-19T19:06:33.418 INFO:tasks.workunit.client.0.smithi088.stdout:8/737: mknod d9/df/d1c/cde 0 2024-02-19T19:06:33.419 INFO:tasks.workunit.client.0.smithi088.stdout:7/896: getdents d14/d1d/d3c/d43/dc2 0 2024-02-19T19:06:33.422 INFO:tasks.workunit.client.0.smithi088.stdout:3/904: unlink d4/ff 0 2024-02-19T19:06:33.423 INFO:tasks.workunit.client.0.smithi088.stdout:2/752: unlink d3/d1f/c33 0 2024-02-19T19:06:33.424 INFO:tasks.workunit.client.0.smithi088.stdout:3/905: chown d4/d1a/d1e/d31/d32/f4c 25623288 1 2024-02-19T19:06:33.425 INFO:tasks.workunit.client.0.smithi088.stdout:4/638: rename d2/d32/c3b to d2/d32/d25/d26/d6d/cd0 0 2024-02-19T19:06:33.426 INFO:tasks.workunit.client.0.smithi088.stdout:0/809: dread d13/d62/d8a/d95/fc4 [0,4194304] 0 2024-02-19T19:06:33.427 INFO:tasks.workunit.client.0.smithi088.stdout:7/897: dwrite d14/d1d/d3c/d43/d5c/f69 [0,4194304] 0 2024-02-19T19:06:33.430 INFO:tasks.workunit.client.0.smithi088.stdout:6/954: getdents d5/d8/d3a/da8 0 2024-02-19T19:06:33.433 INFO:tasks.workunit.client.0.smithi088.stdout:3/906: rename d4/d1a/d7f/d88/ca3 to d4/d87/d118/c13a 0 2024-02-19T19:06:33.434 INFO:tasks.workunit.client.0.smithi088.stdout:4/639: symlink d2/d32/d25/d26/d4e/ld1 0 2024-02-19T19:06:33.434 INFO:tasks.workunit.client.0.smithi088.stdout:8/738: link d9/df/cd1 d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/db3/cdf 0 2024-02-19T19:06:33.436 INFO:tasks.workunit.client.0.smithi088.stdout:8/739: chown d9/df/d3d/d50/d71/da1/d6c/d77 42002022 1 2024-02-19T19:06:33.436 INFO:tasks.workunit.client.0.smithi088.stdout:4/640: chown d2/d32/d25/d26/d4e/d6f/lac 14575 1 2024-02-19T19:06:33.437 INFO:tasks.workunit.client.0.smithi088.stdout:4/641: chown d2/d32/d25/d3f/d70/f8c 1 1 2024-02-19T19:06:33.438 INFO:tasks.workunit.client.0.smithi088.stdout:8/740: chown d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/fd2 14386 1 2024-02-19T19:06:33.440 INFO:tasks.workunit.client.0.smithi088.stdout:7/898: creat d14/d1d/d3c/d43/d5c/d9a/d92/f137 x:0 0 0 2024-02-19T19:06:33.441 INFO:tasks.workunit.client.0.smithi088.stdout:7/899: stat d14/d1d/d3c/d43/d5c/d75/lb8 0 2024-02-19T19:06:33.444 INFO:tasks.workunit.client.0.smithi088.stdout:8/741: mkdir d9/dc4/de0 0 2024-02-19T19:06:33.447 INFO:tasks.workunit.client.0.smithi088.stdout:7/900: mknod d14/d27/d49/d98/dcb/de7/c138 0 2024-02-19T19:06:33.449 INFO:tasks.workunit.client.0.smithi088.stdout:2/753: rename d3/d1b/d43/d61/d7e/db6/cd6 to d3/d1b/d43/d61/d7e/cf9 0 2024-02-19T19:06:33.452 INFO:tasks.workunit.client.0.smithi088.stdout:3/907: creat d4/d1a/f13b x:0 0 0 2024-02-19T19:06:33.453 INFO:tasks.workunit.client.0.smithi088.stdout:6/955: getdents d5/de/d97 0 2024-02-19T19:06:33.454 INFO:tasks.workunit.client.0.smithi088.stdout:0/810: truncate d13/d62/d8a/d95/d5a/db6/fd7 1665267 0 2024-02-19T19:06:33.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:33 smithi132 ceph-mon[70676]: pgmap v50: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 56 MiB/s rd, 128 MiB/s wr, 291 op/s 2024-02-19T19:06:33.541 INFO:tasks.workunit.client.0.smithi088.stdout:4/642: getdents d2/d35 0 2024-02-19T19:06:33.541 INFO:tasks.workunit.client.0.smithi088.stdout:2/754: creat d3/d1b/d43/df4/ffa x:0 0 0 2024-02-19T19:06:33.544 INFO:tasks.workunit.client.0.smithi088.stdout:3/908: dwrite d4/d1a/d23/f5d [0,4194304] 0 2024-02-19T19:06:33.546 INFO:tasks.workunit.client.0.smithi088.stdout:2/755: write d3/d1f/d5d/de1/ff3 [7063,30959] 0 2024-02-19T19:06:33.547 INFO:tasks.workunit.client.0.smithi088.stdout:6/956: dwrite d5/d8/d3a/d71/dc3/f12e [0,4194304] 0 2024-02-19T19:06:33.549 INFO:tasks.workunit.client.0.smithi088.stdout:2/756: truncate d3/fb0 4680142 0 2024-02-19T19:06:33.550 INFO:tasks.workunit.client.0.smithi088.stdout:8/742: dread d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/f65 [0,4194304] 0 2024-02-19T19:06:33.562 INFO:tasks.workunit.client.0.smithi088.stdout:6/957: dread d5/f1e [0,4194304] 0 2024-02-19T19:06:33.562 INFO:tasks.workunit.client.0.smithi088.stdout:6/958: dread - d5/d7/d32/ffa zero size 2024-02-19T19:06:33.563 INFO:tasks.workunit.client.0.smithi088.stdout:6/959: dread - d5/de/d64/d7e/fe3 zero size 2024-02-19T19:06:33.567 INFO:tasks.workunit.client.0.smithi088.stdout:0/811: dread d13/d62/d8a/d95/d5a/d25/d2a/d3c/fad [0,4194304] 0 2024-02-19T19:06:33.567 INFO:tasks.workunit.client.0.smithi088.stdout:6/960: dread d5/d8/d77/f102 [0,4194304] 0 2024-02-19T19:06:33.568 INFO:tasks.workunit.client.0.smithi088.stdout:6/961: write d5/d19/d26/d72/ff1 [753042,25161] 0 2024-02-19T19:06:33.582 INFO:tasks.workunit.client.0.smithi088.stdout:4/643: symlink d2/d32/d25/d26/da5/db2/ld2 0 2024-02-19T19:06:33.583 INFO:tasks.workunit.client.0.smithi088.stdout:4/644: fdatasync d2/d32/d25/d3f/d70/f8c 0 2024-02-19T19:06:33.585 INFO:tasks.workunit.client.0.smithi088.stdout:4/645: dread - d2/d32/d25/d26/f62 zero size 2024-02-19T19:06:33.586 INFO:tasks.workunit.client.0.smithi088.stdout:8/743: truncate d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/f65 2575394 0 2024-02-19T19:06:33.588 INFO:tasks.workunit.client.0.smithi088.stdout:0/812: creat d13/de9/f116 x:0 0 0 2024-02-19T19:06:33.590 INFO:tasks.workunit.client.0.smithi088.stdout:4/646: dread d2/d32/d25/d3f/d9a/fb5 [0,4194304] 0 2024-02-19T19:06:33.593 INFO:tasks.workunit.client.0.smithi088.stdout:2/757: fsync d3/f6 0 2024-02-19T19:06:33.596 INFO:tasks.workunit.client.0.smithi088.stdout:6/962: creat d5/d7/dc9/d109/f133 x:0 0 0 2024-02-19T19:06:33.596 INFO:tasks.workunit.client.0.smithi088.stdout:6/963: readlink d5/d8/d3a/l65 0 2024-02-19T19:06:33.598 INFO:tasks.workunit.client.0.smithi088.stdout:6/964: write d5/d7/f14 [7969578,1309] 0 2024-02-19T19:06:33.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:33 smithi088 ceph-mon[88643]: pgmap v50: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 56 MiB/s rd, 128 MiB/s wr, 291 op/s 2024-02-19T19:06:33.602 INFO:tasks.workunit.client.0.smithi088.stdout:2/758: rmdir d3/d1b/d43/d61/d7e 39 2024-02-19T19:06:33.604 INFO:tasks.workunit.client.0.smithi088.stdout:2/759: read d3/d1f/d4f/dc6/fcb [481358,116522] 0 2024-02-19T19:06:33.606 INFO:tasks.workunit.client.0.smithi088.stdout:3/909: getdents d4/d100 0 2024-02-19T19:06:33.608 INFO:tasks.workunit.client.0.smithi088.stdout:0/813: creat d13/d62/d8a/d9c/da8/f117 x:0 0 0 2024-02-19T19:06:33.609 INFO:tasks.workunit.client.0.smithi088.stdout:8/744: link d9/df/d3d/d50/d71/da1/d6c/d77/d97/c9c d9/df/d3d/d50/d71/da1/ce1 0 2024-02-19T19:06:33.611 INFO:tasks.workunit.client.0.smithi088.stdout:6/965: symlink d5/de/d64/d8d/d95/dba/d11b/l134 0 2024-02-19T19:06:33.611 INFO:tasks.workunit.client.0.smithi088.stdout:3/910: dread - d4/d1a/d1e/d31/d32/d55/fc4 zero size 2024-02-19T19:06:33.613 INFO:tasks.workunit.client.0.smithi088.stdout:4/647: dread d2/d32/d25/d26/d58/f69 [0,4194304] 0 2024-02-19T19:06:33.614 INFO:tasks.workunit.client.0.smithi088.stdout:0/814: rename d13/d62/d8a/d87 to d13/de9/d118 0 2024-02-19T19:06:33.616 INFO:tasks.workunit.client.0.smithi088.stdout:3/911: unlink d4/d12/f1d 0 2024-02-19T19:06:33.618 INFO:tasks.workunit.client.0.smithi088.stdout:4/648: creat d2/d32/d25/fd3 x:0 0 0 2024-02-19T19:06:33.619 INFO:tasks.workunit.client.0.smithi088.stdout:4/649: chown d2/d32/d25/d26/d6d/d7e/d3a/l3d 885 1 2024-02-19T19:06:33.619 INFO:tasks.workunit.client.0.smithi088.stdout:6/966: rename d5/d7/l51 to d5/d7/dc9/l135 0 2024-02-19T19:06:33.621 INFO:tasks.workunit.client.0.smithi088.stdout:4/650: truncate d2/d32/fbb 268850 0 2024-02-19T19:06:33.622 INFO:tasks.workunit.client.0.smithi088.stdout:0/815: dread - d13/d62/d8a/d95/d5a/db6/fda zero size 2024-02-19T19:06:33.622 INFO:tasks.workunit.client.0.smithi088.stdout:6/967: read d5/d19/d70/f73 [2365391,73108] 0 2024-02-19T19:06:33.623 INFO:tasks.workunit.client.0.smithi088.stdout:3/912: mknod d4/d22/dd5/c13c 0 2024-02-19T19:06:33.625 INFO:tasks.workunit.client.0.smithi088.stdout:8/745: rename d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/c8f to d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/ce2 0 2024-02-19T19:06:33.627 INFO:tasks.workunit.client.0.smithi088.stdout:2/760: getdents d3/d1f 0 2024-02-19T19:06:33.631 INFO:tasks.workunit.client.0.smithi088.stdout:1/850: sync 2024-02-19T19:06:33.633 INFO:tasks.workunit.client.0.smithi088.stdout:3/913: mkdir d4/d1a/d1e/d74/d7d/daa/d13d 0 2024-02-19T19:06:33.634 INFO:tasks.workunit.client.0.smithi088.stdout:4/651: dwrite d2/d32/f44 [4194304,4194304] 0 2024-02-19T19:06:33.636 INFO:tasks.workunit.client.0.smithi088.stdout:2/761: symlink d3/d1b/lfb 0 2024-02-19T19:06:33.637 INFO:tasks.workunit.client.0.smithi088.stdout:4/652: write d2/d32/d25/d26/d6d/d7e/d3a/f42 [706880,8093] 0 2024-02-19T19:06:33.638 INFO:tasks.workunit.client.0.smithi088.stdout:6/968: mkdir d5/d45/d9d/d108/d12d/d136 0 2024-02-19T19:06:33.640 INFO:tasks.workunit.client.0.smithi088.stdout:4/653: readlink d2/d32/d25/d26/d6d/d7e/d3a/l3d 0 2024-02-19T19:06:33.642 INFO:tasks.workunit.client.0.smithi088.stdout:3/914: dwrite d4/d1a/d1e/d74/d7d/daa/fe9 [0,4194304] 0 2024-02-19T19:06:33.649 INFO:tasks.workunit.client.0.smithi088.stdout:1/851: rename d6/d21/d64/d94/d71/f7a to d6/d21/d64/dc8/f113 0 2024-02-19T19:06:33.654 INFO:tasks.workunit.client.0.smithi088.stdout:4/654: unlink d2/d32/d25/d3f/d70/f8c 0 2024-02-19T19:06:33.661 INFO:tasks.workunit.client.0.smithi088.stdout:4/655: dread d2/d32/d25/d26/f88 [0,4194304] 0 2024-02-19T19:06:33.663 INFO:tasks.workunit.client.0.smithi088.stdout:2/762: getdents d3/d71 0 2024-02-19T19:06:33.665 INFO:tasks.workunit.client.0.smithi088.stdout:4/656: readlink d2/d32/l27 0 2024-02-19T19:06:33.665 INFO:tasks.workunit.client.0.smithi088.stdout:1/852: dread d6/d10/d60/d29/fb3 [0,4194304] 0 2024-02-19T19:06:33.665 INFO:tasks.workunit.client.0.smithi088.stdout:2/763: read d3/fb [503378,111757] 0 2024-02-19T19:06:33.674 INFO:tasks.workunit.client.0.smithi088.stdout:4/657: dwrite d2/d32/d25/faf [0,4194304] 0 2024-02-19T19:06:33.675 INFO:tasks.workunit.client.0.smithi088.stdout:6/969: rename d5/d7/dc9/d109 to d5/de/d64/d8d/d95/dba/d137 0 2024-02-19T19:06:33.676 INFO:tasks.workunit.client.0.smithi088.stdout:2/764: dwrite d3/f85 [4194304,4194304] 0 2024-02-19T19:06:33.678 INFO:tasks.workunit.client.0.smithi088.stdout:6/970: read - d5/d8/d3a/da8/dbd/ff8 zero size 2024-02-19T19:06:33.679 INFO:tasks.workunit.client.0.smithi088.stdout:2/765: fsync d3/d1f/d5d/de1/ff3 0 2024-02-19T19:06:33.680 INFO:tasks.workunit.client.0.smithi088.stdout:3/915: rename d4/d87/dbf to d4/d1a/d7f/d13e 0 2024-02-19T19:06:33.682 INFO:tasks.workunit.client.0.smithi088.stdout:6/971: write d5/d8/d3a/d71/daa/dc2/dbc/f12f [49119,34757] 0 2024-02-19T19:06:33.683 INFO:tasks.workunit.client.0.smithi088.stdout:2/766: read d3/d14/f1d [2235157,115280] 0 2024-02-19T19:06:33.686 INFO:tasks.workunit.client.0.smithi088.stdout:1/853: dread d6/d21/d6a/f68 [0,4194304] 0 2024-02-19T19:06:33.691 INFO:tasks.workunit.client.0.smithi088.stdout:6/972: fdatasync d5/de/d64/d8d/d95/dba/f122 0 2024-02-19T19:06:33.693 INFO:tasks.workunit.client.0.smithi088.stdout:6/973: write d5/d8/d2c/f113 [166795,105163] 0 2024-02-19T19:06:33.694 INFO:tasks.workunit.client.0.smithi088.stdout:8/746: rmdir d9 39 2024-02-19T19:06:33.694 INFO:tasks.workunit.client.0.smithi088.stdout:6/974: write d5/d8/d3a/d71/daa/dc2/dbc/f11c [96747,3003] 0 2024-02-19T19:06:33.804 INFO:tasks.workunit.client.0.smithi088.stdout:0/816: write d13/d62/d8a/d9c/da8/de2/f6e [326423,116514] 0 2024-02-19T19:06:33.806 INFO:tasks.workunit.client.0.smithi088.stdout:4/658: truncate d2/d32/d25/d26/d6d/d7e/d3a/f74 2027894 0 2024-02-19T19:06:33.806 INFO:tasks.workunit.client.0.smithi088.stdout:6/975: creat d5/d8/d3a/d71/daa/dc2/dbc/db6/f138 x:0 0 0 2024-02-19T19:06:33.807 INFO:tasks.workunit.client.0.smithi088.stdout:1/854: getdents d6/d21/d64/d94/dab 0 2024-02-19T19:06:33.809 INFO:tasks.workunit.client.0.smithi088.stdout:0/817: chown d13/d62/d8a/d95/d5a/d25/f2b 21 1 2024-02-19T19:06:33.811 INFO:tasks.workunit.client.0.smithi088.stdout:6/976: write d5/d7/d32/d56/fdf [1318716,50989] 0 2024-02-19T19:06:33.812 INFO:tasks.workunit.client.0.smithi088.stdout:8/747: getdents d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f 0 2024-02-19T19:06:33.813 INFO:tasks.workunit.client.0.smithi088.stdout:0/818: chown d13/d62/d8a/d95/d96/l9a 3752 1 2024-02-19T19:06:33.814 INFO:tasks.workunit.client.0.smithi088.stdout:3/916: dwrite d4/d1a/d7f/d13e/fc2 [0,4194304] 0 2024-02-19T19:06:33.815 INFO:tasks.workunit.client.0.smithi088.stdout:4/659: creat d2/d32/d25/d26/d6d/d7e/d3a/fd4 x:0 0 0 2024-02-19T19:06:33.815 INFO:tasks.workunit.client.0.smithi088.stdout:0/819: chown d13/d62/d8a/d95/d5a/d25/l115 111345659 1 2024-02-19T19:06:33.816 INFO:tasks.workunit.client.0.smithi088.stdout:1/855: creat d6/d21/d6a/d7d/d9a/f114 x:0 0 0 2024-02-19T19:06:33.821 INFO:tasks.workunit.client.0.smithi088.stdout:8/748: creat d9/df/d3d/d50/d71/fe3 x:0 0 0 2024-02-19T19:06:33.824 INFO:tasks.workunit.client.0.smithi088.stdout:0/820: dwrite d13/d62/d8a/d9c/da8/de2/fd5 [0,4194304] 0 2024-02-19T19:06:33.826 INFO:tasks.workunit.client.0.smithi088.stdout:6/977: symlink d5/l139 0 2024-02-19T19:06:33.828 INFO:tasks.workunit.client.0.smithi088.stdout:1/856: symlink d6/d21/d6a/d4a/dad/l115 0 2024-02-19T19:06:33.831 INFO:tasks.workunit.client.0.smithi088.stdout:3/917: rename d4/d1a/d1e/d74/d7c to d4/d87/d118/d13f 0 2024-02-19T19:06:33.834 INFO:tasks.workunit.client.0.smithi088.stdout:4/660: write d2/d32/d25/f41 [16619136,79846] 0 2024-02-19T19:06:33.834 INFO:tasks.workunit.client.0.smithi088.stdout:6/978: dread d5/d45/f120 [0,4194304] 0 2024-02-19T19:06:33.834 INFO:tasks.workunit.client.0.smithi088.stdout:4/661: chown d2/d35/c4c 661349 1 2024-02-19T19:06:33.836 INFO:tasks.workunit.client.0.smithi088.stdout:4/662: chown d2/d32/d25/d45/l65 1413 1 2024-02-19T19:06:33.836 INFO:tasks.workunit.client.0.smithi088.stdout:1/857: write d6/d21/d64/d94/f66 [1337661,57747] 0 2024-02-19T19:06:33.837 INFO:tasks.workunit.client.0.smithi088.stdout:4/663: dread - d2/f8d zero size 2024-02-19T19:06:33.837 INFO:tasks.workunit.client.0.smithi088.stdout:4/664: stat d2/d35 0 2024-02-19T19:06:33.839 INFO:tasks.workunit.client.0.smithi088.stdout:8/749: rename d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/lbc to d9/df/d3d/d50/d71/da1/le4 0 2024-02-19T19:06:33.839 INFO:tasks.workunit.client.0.smithi088.stdout:4/665: chown d2/d32/d9/cb4 0 1 2024-02-19T19:06:33.844 INFO:tasks.workunit.client.0.smithi088.stdout:3/918: rename d4/d22/fef to d4/d22/dd5/f140 0 2024-02-19T19:06:33.846 INFO:tasks.workunit.client.0.smithi088.stdout:8/750: read - d9/df/d3d/d50/d71/da1/d6c/d77/d97/fc7 zero size 2024-02-19T19:06:33.850 INFO:tasks.workunit.client.0.smithi088.stdout:8/751: read d9/df/d3d/d50/d71/da1/d70/fa7 [6247910,86848] 0 2024-02-19T19:06:33.850 INFO:tasks.workunit.client.0.smithi088.stdout:4/666: creat d2/d32/d25/d26/da0/fd5 x:0 0 0 2024-02-19T19:06:33.851 INFO:tasks.workunit.client.0.smithi088.stdout:3/919: dwrite d4/d1a/f12a [0,4194304] 0 2024-02-19T19:06:33.852 INFO:tasks.workunit.client.0.smithi088.stdout:8/752: write d9/df/d3d/f87 [1097120,79006] 0 2024-02-19T19:06:33.854 INFO:tasks.workunit.client.0.smithi088.stdout:0/821: write d13/de9/ffd [2167350,53961] 0 2024-02-19T19:06:33.854 INFO:tasks.workunit.client.0.smithi088.stdout:3/920: dread d4/d1a/d23/f101 [0,4194304] 0 2024-02-19T19:06:33.858 INFO:tasks.workunit.client.0.smithi088.stdout:4/667: creat d2/d32/d9/fd6 x:0 0 0 2024-02-19T19:06:33.859 INFO:tasks.workunit.client.0.smithi088.stdout:4/668: write d2/d35/db3/db7/fc5 [1787932,49610] 0 2024-02-19T19:06:33.862 INFO:tasks.workunit.client.0.smithi088.stdout:8/753: rename d9/df/d3d/d50/d71/da1/d6c/d9a/fbb to d9/df/d3d/d50/d71/fe5 0 2024-02-19T19:06:33.863 INFO:tasks.workunit.client.0.smithi088.stdout:1/858: truncate d6/d21/d64/d94/dab/fed 3261470 0 2024-02-19T19:06:33.864 INFO:tasks.workunit.client.0.smithi088.stdout:1/859: fdatasync d6/d10/d60/d29/f37 0 2024-02-19T19:06:33.867 INFO:tasks.workunit.client.0.smithi088.stdout:4/669: mkdir d2/d35/db3/db7/dd7 0 2024-02-19T19:06:33.868 INFO:tasks.workunit.client.0.smithi088.stdout:0/822: link d13/de9/d118/fe6 d13/d62/d8a/d95/d5b/d76/f119 0 2024-02-19T19:06:33.869 INFO:tasks.workunit.client.0.smithi088.stdout:4/670: dread d2/d32/d25/f38 [0,4194304] 0 2024-02-19T19:06:33.871 INFO:tasks.workunit.client.0.smithi088.stdout:1/860: creat d6/d10/d60/db8/dd9/de4/f116 x:0 0 0 2024-02-19T19:06:33.872 INFO:tasks.workunit.client.0.smithi088.stdout:4/671: mkdir d2/d32/d9/dd8 0 2024-02-19T19:06:33.876 INFO:tasks.workunit.client.0.smithi088.stdout:8/754: link d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/la3 d9/df/d1c/d22/le6 0 2024-02-19T19:06:33.877 INFO:tasks.workunit.client.0.smithi088.stdout:0/823: dwrite d13/d62/d8a/d95/d5b/d76/fbd [0,4194304] 0 2024-02-19T19:06:33.878 INFO:tasks.workunit.client.0.smithi088.stdout:4/672: creat d2/d32/d25/d26/d6d/d9e/fd9 x:0 0 0 2024-02-19T19:06:33.882 INFO:tasks.workunit.client.0.smithi088.stdout:1/861: dwrite d6/d21/d64/d94/d71/dbe/f10f [0,4194304] 0 2024-02-19T19:06:33.883 INFO:tasks.workunit.client.0.smithi088.stdout:7/901: sync 2024-02-19T19:06:33.892 INFO:tasks.workunit.client.0.smithi088.stdout:7/902: mknod d14/d1d/d62/dfa/c139 0 2024-02-19T19:06:33.895 INFO:tasks.workunit.client.0.smithi088.stdout:0/824: getdents d13/d62/d8a/d95/d96 0 2024-02-19T19:06:33.896 INFO:tasks.workunit.client.0.smithi088.stdout:7/903: mkdir d14/d1d/d3c/d43/d5c/d9a/d13a 0 2024-02-19T19:06:33.898 INFO:tasks.workunit.client.0.smithi088.stdout:0/825: unlink d13/l10b 0 2024-02-19T19:06:33.901 INFO:tasks.workunit.client.0.smithi088.stdout:7/904: symlink d14/d27/d49/l13b 0 2024-02-19T19:06:33.903 INFO:tasks.workunit.client.0.smithi088.stdout:0/826: getdents d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb 0 2024-02-19T19:06:33.904 INFO:tasks.workunit.client.0.smithi088.stdout:7/905: creat d14/f13c x:0 0 0 2024-02-19T19:06:33.907 INFO:tasks.workunit.client.0.smithi088.stdout:7/906: fdatasync d14/f2e 0 2024-02-19T19:06:33.908 INFO:tasks.workunit.client.0.smithi088.stdout:0/827: truncate d13/f23 5165554 0 2024-02-19T19:06:33.909 INFO:tasks.workunit.client.0.smithi088.stdout:4/673: dread d2/d32/d9/f71 [0,4194304] 0 2024-02-19T19:06:33.911 INFO:tasks.workunit.client.0.smithi088.stdout:8/755: truncate d9/df/d1c/f13 1672359 0 2024-02-19T19:06:33.911 INFO:tasks.workunit.client.0.smithi088.stdout:4/674: stat d2/f4 0 2024-02-19T19:06:33.915 INFO:tasks.workunit.client.0.smithi088.stdout:8/756: chown d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/fd5 9 1 2024-02-19T19:06:33.917 INFO:tasks.workunit.client.0.smithi088.stdout:4/675: unlink d2/d32/d25/d3f/f9c 0 2024-02-19T19:06:33.917 INFO:tasks.workunit.client.0.smithi088.stdout:1/862: dwrite d6/d21/d6a/d4a/fbc [0,4194304] 0 2024-02-19T19:06:33.918 INFO:tasks.workunit.client.0.smithi088.stdout:1/863: chown d6/d10/d60/f110 51 1 2024-02-19T19:06:33.919 INFO:tasks.workunit.client.0.smithi088.stdout:7/907: write d14/d1d/d3c/d43/d5c/f125 [1510391,99718] 0 2024-02-19T19:06:33.920 INFO:tasks.workunit.client.0.smithi088.stdout:1/864: fdatasync d6/d21/d6a/d4a/f59 0 2024-02-19T19:06:33.922 INFO:tasks.workunit.client.0.smithi088.stdout:4/676: symlink d2/d32/d25/d26/d6d/d7e/d3a/lda 0 2024-02-19T19:06:33.923 INFO:tasks.workunit.client.0.smithi088.stdout:1/865: readlink d6/d21/d64/d94/dab/lc5 0 2024-02-19T19:06:33.924 INFO:tasks.workunit.client.0.smithi088.stdout:8/757: fsync d9/df/d1c/d22/d9e/faa 0 2024-02-19T19:06:33.929 INFO:tasks.workunit.client.0.smithi088.stdout:4/677: creat d2/fdb x:0 0 0 2024-02-19T19:06:33.931 INFO:tasks.workunit.client.0.smithi088.stdout:1/866: dwrite d6/d21/d64/d94/d71/dbe/f10f [4194304,4194304] 0 2024-02-19T19:06:33.934 INFO:tasks.workunit.client.0.smithi088.stdout:8/758: mknod d9/df/d3d/ce7 0 2024-02-19T19:06:33.934 INFO:tasks.workunit.client.0.smithi088.stdout:1/867: read d6/d10/d60/db8/fd1 [614388,84204] 0 2024-02-19T19:06:33.936 INFO:tasks.workunit.client.0.smithi088.stdout:8/759: readlink d9/df/lb7 0 2024-02-19T19:06:33.937 INFO:tasks.workunit.client.0.smithi088.stdout:8/760: chown d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/f8a 2736 1 2024-02-19T19:06:33.937 INFO:tasks.workunit.client.0.smithi088.stdout:4/678: mknod d2/d32/d25/d45/cdc 0 2024-02-19T19:06:33.938 INFO:tasks.workunit.client.0.smithi088.stdout:4/679: dread - d2/d32/d25/d3f/d9a/fc6 zero size 2024-02-19T19:06:33.940 INFO:tasks.workunit.client.0.smithi088.stdout:1/868: mkdir d6/d10/d60/db8/dd9/de4/d117 0 2024-02-19T19:06:33.941 INFO:tasks.workunit.client.0.smithi088.stdout:0/828: dread d13/d62/d8a/d95/d5a/d25/f8b [0,4194304] 0 2024-02-19T19:06:33.945 INFO:tasks.workunit.client.0.smithi088.stdout:4/680: mknod d2/d32/cdd 0 2024-02-19T19:06:33.946 INFO:tasks.workunit.client.0.smithi088.stdout:7/908: truncate f2 462728 0 2024-02-19T19:06:33.947 INFO:tasks.workunit.client.0.smithi088.stdout:4/681: chown d2/d32/d25/l4d 422366038 1 2024-02-19T19:06:33.948 INFO:tasks.workunit.client.0.smithi088.stdout:0/829: mkdir d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/d11a 0 2024-02-19T19:06:33.948 INFO:tasks.workunit.client.0.smithi088.stdout:8/761: dwrite d9/df/f27 [4194304,4194304] 0 2024-02-19T19:06:33.952 INFO:tasks.workunit.client.0.smithi088.stdout:7/909: rmdir d14/d28 39 2024-02-19T19:06:33.957 INFO:tasks.workunit.client.0.smithi088.stdout:1/869: write d6/d21/f3c [645232,114291] 0 2024-02-19T19:06:33.959 INFO:tasks.workunit.client.0.smithi088.stdout:0/830: fsync d13/f29 0 2024-02-19T19:06:33.966 INFO:tasks.workunit.client.0.smithi088.stdout:1/870: mknod d6/d10/d60/c118 0 2024-02-19T19:06:33.969 INFO:tasks.workunit.client.0.smithi088.stdout:8/762: truncate d9/df/d3d/d50/d71/da1/d6c/d9a/f4d 1442347 0 2024-02-19T19:06:33.970 INFO:tasks.workunit.client.0.smithi088.stdout:8/763: dread - d9/df/d3d/d50/d71/fe3 zero size 2024-02-19T19:06:33.971 INFO:tasks.workunit.client.0.smithi088.stdout:4/682: truncate d2/d8/f6b 3248913 0 2024-02-19T19:06:33.972 INFO:tasks.workunit.client.0.smithi088.stdout:0/831: write d13/f3e [2905886,64191] 0 2024-02-19T19:06:33.974 INFO:tasks.workunit.client.0.smithi088.stdout:8/764: mknod d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/d8e/ce8 0 2024-02-19T19:06:33.974 INFO:tasks.workunit.client.0.smithi088.stdout:0/832: fsync d13/d62/d8a/d95/d5a/fa2 0 2024-02-19T19:06:33.977 INFO:tasks.workunit.client.0.smithi088.stdout:7/910: dread d14/f1f [0,4194304] 0 2024-02-19T19:06:34.014 INFO:tasks.workunit.client.0.smithi088.stdout:4/683: symlink d2/d32/d25/d26/d6d/d7e/d3a/d77/d99/lde 0 2024-02-19T19:06:34.014 INFO:tasks.workunit.client.0.smithi088.stdout:4/684: chown d2/d32/d9/l19 12 1 2024-02-19T19:06:34.016 INFO:tasks.workunit.client.0.smithi088.stdout:7/911: fdatasync d14/d1d/d3c/d43/d5c/d75/d76/f90 0 2024-02-19T19:06:34.017 INFO:tasks.workunit.client.0.smithi088.stdout:7/912: dread - d14/d1d/f105 zero size 2024-02-19T19:06:34.018 INFO:tasks.workunit.client.0.smithi088.stdout:7/913: truncate d14/f77 1453236 0 2024-02-19T19:06:34.021 INFO:tasks.workunit.client.0.smithi088.stdout:0/833: link d13/d62/d8a/d95/fc4 d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/d10a/d106/f11b 0 2024-02-19T19:06:34.027 INFO:tasks.workunit.client.0.smithi088.stdout:4/685: mknod d2/d32/d25/d26/d4e/d79/cdf 0 2024-02-19T19:06:34.029 INFO:tasks.workunit.client.0.smithi088.stdout:8/765: dread d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/fda [0,4194304] 0 2024-02-19T19:06:34.029 INFO:tasks.workunit.client.0.smithi088.stdout:7/914: mknod d14/d28/d9c/c13d 0 2024-02-19T19:06:34.031 INFO:tasks.workunit.client.0.smithi088.stdout:0/834: creat d13/d62/d8a/d95/d5a/d25/dd6/f11c x:0 0 0 2024-02-19T19:06:34.032 INFO:tasks.workunit.client.0.smithi088.stdout:4/686: rename d2/d32/d25/d26/d6d/d9e to d2/d32/d9/dd8/de0 0 2024-02-19T19:06:34.034 INFO:tasks.workunit.client.0.smithi088.stdout:8/766: creat d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/fe9 x:0 0 0 2024-02-19T19:06:34.040 INFO:tasks.workunit.client.0.smithi088.stdout:7/915: rename d14/d1d/d3c/f3d to d14/d28/d9c/f13e 0 2024-02-19T19:06:34.040 INFO:tasks.workunit.client.0.smithi088.stdout:4/687: fdatasync d2/d32/d9/f20 0 2024-02-19T19:06:34.042 INFO:tasks.workunit.client.0.smithi088.stdout:0/835: fsync d13/d62/d8a/d9c/da8/dab/fd1 0 2024-02-19T19:06:34.045 INFO:tasks.workunit.client.0.smithi088.stdout:7/916: symlink d14/d27/d6b/dba/d102/l13f 0 2024-02-19T19:06:34.051 INFO:tasks.workunit.client.0.smithi088.stdout:7/917: symlink d14/d1d/d3c/d43/d5c/d75/l140 0 2024-02-19T19:06:34.053 INFO:tasks.workunit.client.0.smithi088.stdout:4/688: creat d2/d32/d25/d3f/fe1 x:0 0 0 2024-02-19T19:06:34.054 INFO:tasks.workunit.client.0.smithi088.stdout:8/767: getdents d9/dc4 0 2024-02-19T19:06:34.057 INFO:tasks.workunit.client.0.smithi088.stdout:0/836: link d13/d62/d8a/d95/d5a/c68 d13/d62/d8a/d95/d5a/df3/c11d 0 2024-02-19T19:06:34.068 INFO:tasks.workunit.client.0.smithi088.stdout:8/768: mkdir d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/dea 0 2024-02-19T19:06:34.068 INFO:tasks.workunit.client.0.smithi088.stdout:4/689: rename d2/d32/f36 to d2/d35/db3/fe2 0 2024-02-19T19:06:34.068 INFO:tasks.workunit.client.0.smithi088.stdout:7/918: write d14/d1d/d3c/d43/d5c/f68 [969096,84360] 0 2024-02-19T19:06:34.070 INFO:tasks.workunit.client.0.smithi088.stdout:0/837: creat d13/d62/d8a/d9c/f11e x:0 0 0 2024-02-19T19:06:34.074 INFO:tasks.workunit.client.0.smithi088.stdout:8/769: link d9/df/d1c/f73 d9/df/d3d/d50/feb 0 2024-02-19T19:06:34.078 INFO:tasks.workunit.client.0.smithi088.stdout:0/838: mknod d13/d62/d8a/d95/d5a/d25/d2a/d34/d100/c11f 0 2024-02-19T19:06:34.080 INFO:tasks.workunit.client.0.smithi088.stdout:0/839: mkdir d13/de9/d120 0 2024-02-19T19:06:34.081 INFO:tasks.workunit.client.0.smithi088.stdout:4/690: write d2/d32/d25/d26/d58/f69 [3171196,48348] 0 2024-02-19T19:06:34.083 INFO:tasks.workunit.client.0.smithi088.stdout:7/919: truncate d14/d1d/d62/f100 4629320 0 2024-02-19T19:06:34.084 INFO:tasks.workunit.client.0.smithi088.stdout:0/840: readlink d13/d62/d8a/d95/d5a/d25/la1 0 2024-02-19T19:06:34.086 INFO:tasks.workunit.client.0.smithi088.stdout:4/691: rename d2/d32/d25/d26/d6d/d7e/d3a/d77/d99 to d2/d35/db3/db7/de3 0 2024-02-19T19:06:34.088 INFO:tasks.workunit.client.0.smithi088.stdout:7/920: creat d14/d28/d3b/dd3/f141 x:0 0 0 2024-02-19T19:06:34.089 INFO:tasks.workunit.client.0.smithi088.stdout:8/770: dread d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/f7b [0,4194304] 0 2024-02-19T19:06:34.090 INFO:tasks.workunit.client.0.smithi088.stdout:0/841: creat d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/d10a/d106/de5/f121 x:0 0 0 2024-02-19T19:06:34.092 INFO:tasks.workunit.client.0.smithi088.stdout:4/692: mkdir d2/d32/d9/de4 0 2024-02-19T19:06:34.092 INFO:tasks.workunit.client.0.smithi088.stdout:8/771: dread - d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/fdd zero size 2024-02-19T19:06:34.094 INFO:tasks.workunit.client.0.smithi088.stdout:4/693: truncate d2/d32/d25/d3f/d70/fae 779996 0 2024-02-19T19:06:34.094 INFO:tasks.workunit.client.0.smithi088.stdout:8/772: write d9/df/d3d/d50/d71/da1/d6c/d9a/f46 [5155192,101044] 0 2024-02-19T19:06:34.094 INFO:tasks.workunit.client.0.smithi088.stdout:7/921: unlink d14/d1d/d3c/f8f 0 2024-02-19T19:06:34.099 INFO:tasks.workunit.client.0.smithi088.stdout:8/773: rmdir d9/df 39 2024-02-19T19:06:34.100 INFO:tasks.workunit.client.0.smithi088.stdout:4/694: creat d2/d32/d25/d26/d6d/d7e/d3a/fe5 x:0 0 0 2024-02-19T19:06:34.102 INFO:tasks.workunit.client.0.smithi088.stdout:7/922: dwrite d14/d1d/d3c/d43/d5c/d9a/df4/f128 [0,4194304] 0 2024-02-19T19:06:34.102 INFO:tasks.workunit.client.0.smithi088.stdout:4/695: truncate d2/d32/d25/f30 5010174 0 2024-02-19T19:06:34.103 INFO:tasks.workunit.client.0.smithi088.stdout:4/696: stat d2/d32/d25/d26/f88 0 2024-02-19T19:06:34.107 INFO:tasks.workunit.client.0.smithi088.stdout:7/923: mknod d14/d1d/d62/dfa/c142 0 2024-02-19T19:06:34.108 INFO:tasks.workunit.client.0.smithi088.stdout:4/697: mknod d2/d32/d25/d26/d58/ce6 0 2024-02-19T19:06:34.109 INFO:tasks.workunit.client.0.smithi088.stdout:8/774: rmdir d9/df/d3d/d50/d71/da1/d6c/d77 39 2024-02-19T19:06:34.110 INFO:tasks.workunit.client.0.smithi088.stdout:8/775: fdatasync d9/df/f1a 0 2024-02-19T19:06:34.110 INFO:tasks.workunit.client.0.smithi088.stdout:0/842: link d13/l16 d13/de9/l122 0 2024-02-19T19:06:34.112 INFO:tasks.workunit.client.0.smithi088.stdout:0/843: read - d13/d62/d8a/d9c/da8/dab/db4/dbf/ff5 zero size 2024-02-19T19:06:34.113 INFO:tasks.workunit.client.0.smithi088.stdout:4/698: dread - d2/d32/d25/d26/d6d/d7e/d3a/d77/f84 zero size 2024-02-19T19:06:34.116 INFO:tasks.workunit.client.0.smithi088.stdout:4/699: chown d2/c63 2862 1 2024-02-19T19:06:34.118 INFO:tasks.workunit.client.0.smithi088.stdout:0/844: creat d13/d62/d8a/d95/d5a/d25/d2a/f123 x:0 0 0 2024-02-19T19:06:34.118 INFO:tasks.workunit.client.0.smithi088.stdout:7/924: write d14/d28/d94/fda [578365,37563] 0 2024-02-19T19:06:34.120 INFO:tasks.workunit.client.0.smithi088.stdout:8/776: mkdir d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/dea/dec 0 2024-02-19T19:06:34.120 INFO:tasks.workunit.client.0.smithi088.stdout:0/845: fsync d13/d62/d8a/d95/d5a/d25/d2a/d2f/ffc 0 2024-02-19T19:06:34.121 INFO:tasks.workunit.client.0.smithi088.stdout:4/700: mkdir d2/d32/de7 0 2024-02-19T19:06:34.122 INFO:tasks.workunit.client.0.smithi088.stdout:7/925: mkdir d14/d1d/d3c/d43/d5c/d75/d76/d143 0 2024-02-19T19:06:34.122 INFO:tasks.workunit.client.0.smithi088.stdout:4/701: read d2/d32/d25/d26/f88 [58866,75822] 0 2024-02-19T19:06:34.124 INFO:tasks.workunit.client.0.smithi088.stdout:0/846: mkdir d13/d62/d8a/d95/d5b/d124 0 2024-02-19T19:06:34.126 INFO:tasks.workunit.client.0.smithi088.stdout:8/777: rename d9/df/c2a to d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/ced 0 2024-02-19T19:06:34.130 INFO:tasks.workunit.client.0.smithi088.stdout:0/847: rmdir d13/d62/d8a/d95/d5a/d25/d2a/d34 39 2024-02-19T19:06:34.132 INFO:tasks.workunit.client.0.smithi088.stdout:4/702: dread d2/d35/db3/db7/fc5 [0,4194304] 0 2024-02-19T19:06:34.133 INFO:tasks.workunit.client.0.smithi088.stdout:0/848: readlink d13/d62/d8a/d95/d5a/d25/l66 0 2024-02-19T19:06:34.133 INFO:tasks.workunit.client.0.smithi088.stdout:2/767: sync 2024-02-19T19:06:34.137 INFO:tasks.workunit.client.0.smithi088.stdout:2/768: mknod d3/d1f/d5d/de1/cfc 0 2024-02-19T19:06:34.147 INFO:tasks.workunit.client.0.smithi088.stdout:8/778: chown d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8/fca 515544 1 2024-02-19T19:06:34.149 INFO:tasks.workunit.client.0.smithi088.stdout:7/926: dwrite d14/d1d/d3c/d43/d5c/f7f [0,4194304] 0 2024-02-19T19:06:34.151 INFO:tasks.workunit.client.0.smithi088.stdout:2/769: dwrite d3/d1f/f89 [0,4194304] 0 2024-02-19T19:06:34.152 INFO:tasks.workunit.client.0.smithi088.stdout:0/849: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d34/f75 [0,4194304] 0 2024-02-19T19:06:34.153 INFO:tasks.workunit.client.0.smithi088.stdout:4/703: dread d2/d35/f4f [0,4194304] 0 2024-02-19T19:06:34.159 INFO:tasks.workunit.client.0.smithi088.stdout:7/927: symlink d14/d27/d49/d98/dbe/l144 0 2024-02-19T19:06:34.160 INFO:tasks.workunit.client.0.smithi088.stdout:0/850: mkdir d13/d62/d8a/d9c/da8/dab/db4/dbf/d125 0 2024-02-19T19:06:34.164 INFO:tasks.workunit.client.0.smithi088.stdout:2/770: dwrite d3/fe8 [0,4194304] 0 2024-02-19T19:06:34.165 INFO:tasks.workunit.client.0.smithi088.stdout:4/704: link d2/d35/f7c d2/d32/d25/d3f/d70/fe8 0 2024-02-19T19:06:34.167 INFO:tasks.workunit.client.0.smithi088.stdout:0/851: mkdir d13/d62/d8a/d95/d5b/dbe/d126 0 2024-02-19T19:06:34.170 INFO:tasks.workunit.client.0.smithi088.stdout:4/705: truncate d2/d32/d25/d26/d6d/d7e/d3a/f42 1270129 0 2024-02-19T19:06:34.170 INFO:tasks.workunit.client.0.smithi088.stdout:0/852: chown d13/d62/d8a/d95/d5a/db6/cc9 14706 1 2024-02-19T19:06:34.170 INFO:tasks.workunit.client.0.smithi088.stdout:7/928: creat d14/d28/f145 x:0 0 0 2024-02-19T19:06:34.170 INFO:tasks.workunit.client.0.smithi088.stdout:4/706: dread - d2/d35/f5b zero size 2024-02-19T19:06:34.172 INFO:tasks.workunit.client.0.smithi088.stdout:0/853: write d13/d62/d8a/d9c/da8/dab/dd0/fdc [245971,5431] 0 2024-02-19T19:06:34.173 INFO:tasks.workunit.client.0.smithi088.stdout:4/707: mknod d2/d32/d25/d26/d4e/ce9 0 2024-02-19T19:06:34.177 INFO:tasks.workunit.client.0.smithi088.stdout:0/854: rename d13/d62/f8e to d13/d62/d8a/d95/d5a/d25/d2a/d34/d100/f127 0 2024-02-19T19:06:34.180 INFO:tasks.workunit.client.0.smithi088.stdout:0/855: mkdir d13/d81/d128 0 2024-02-19T19:06:34.181 INFO:tasks.workunit.client.0.smithi088.stdout:4/708: dwrite d2/d32/d25/d26/d6d/d7e/d3a/fe5 [0,4194304] 0 2024-02-19T19:06:34.182 INFO:tasks.workunit.client.0.smithi088.stdout:4/709: chown d2/d35/db3/db7/dd7 2107433 1 2024-02-19T19:06:34.183 INFO:tasks.workunit.client.0.smithi088.stdout:0/856: creat d13/de9/f129 x:0 0 0 2024-02-19T19:06:34.185 INFO:tasks.workunit.client.0.smithi088.stdout:8/779: dread d9/df/d3d/d50/d71/da1/d70/fa7 [4194304,4194304] 0 2024-02-19T19:06:34.186 INFO:tasks.workunit.client.0.smithi088.stdout:4/710: unlink d2/d32/d9/cb4 0 2024-02-19T19:06:34.186 INFO:tasks.workunit.client.0.smithi088.stdout:4/711: chown d2/d32/d9/cca 291 1 2024-02-19T19:06:34.187 INFO:tasks.workunit.client.0.smithi088.stdout:2/771: write d3/d1b/d43/f66 [2770529,116252] 0 2024-02-19T19:06:34.194 INFO:tasks.workunit.client.0.smithi088.stdout:4/712: dwrite d2/d32/d25/faf [0,4194304] 0 2024-02-19T19:06:34.195 INFO:tasks.workunit.client.0.smithi088.stdout:0/857: creat d13/d62/d8a/d95/f12a x:0 0 0 2024-02-19T19:06:34.197 INFO:tasks.workunit.client.0.smithi088.stdout:1/871: sync 2024-02-19T19:06:34.197 INFO:tasks.workunit.client.0.smithi088.stdout:3/921: sync 2024-02-19T19:06:34.197 INFO:tasks.workunit.client.0.smithi088.stdout:6/979: sync 2024-02-19T19:06:34.198 INFO:tasks.workunit.client.0.smithi088.stdout:8/780: symlink d9/df/d1c/lee 0 2024-02-19T19:06:34.198 INFO:tasks.workunit.client.0.smithi088.stdout:3/922: dread - d4/d12/d33/d43/d85/f12d zero size 2024-02-19T19:06:34.201 INFO:tasks.workunit.client.0.smithi088.stdout:4/713: rmdir d2/d32/d25/d26/d6d/d7e/da3/d9f 39 2024-02-19T19:06:34.203 INFO:tasks.workunit.client.0.smithi088.stdout:0/858: fsync d13/f23 0 2024-02-19T19:06:34.207 INFO:tasks.workunit.client.0.smithi088.stdout:2/772: getdents d3/d14/dd8 0 2024-02-19T19:06:34.207 INFO:tasks.workunit.client.0.smithi088.stdout:6/980: getdents d5/d7/d32/de5 0 2024-02-19T19:06:34.209 INFO:tasks.workunit.client.0.smithi088.stdout:8/781: dread d9/df/d3d/d50/feb [0,4194304] 0 2024-02-19T19:06:34.210 INFO:tasks.workunit.client.0.smithi088.stdout:6/981: fdatasync d5/d19/d70/f73 0 2024-02-19T19:06:34.211 INFO:tasks.workunit.client.0.smithi088.stdout:4/714: creat d2/d32/fea x:0 0 0 2024-02-19T19:06:34.213 INFO:tasks.workunit.client.0.smithi088.stdout:2/773: dread d3/d1b/d43/d61/d64/f6a [0,4194304] 0 2024-02-19T19:06:34.214 INFO:tasks.workunit.client.0.smithi088.stdout:1/872: write d6/d21/d6a/d4a/faf [8039,35013] 0 2024-02-19T19:06:34.215 INFO:tasks.workunit.client.0.smithi088.stdout:6/982: creat d5/de/d64/d7e/f13a x:0 0 0 2024-02-19T19:06:34.217 INFO:tasks.workunit.client.0.smithi088.stdout:3/923: write d4/d12/d33/fe4 [486607,44784] 0 2024-02-19T19:06:34.219 INFO:tasks.workunit.client.0.smithi088.stdout:1/873: symlink d6/d21/d64/l119 0 2024-02-19T19:06:34.220 INFO:tasks.workunit.client.0.smithi088.stdout:6/983: chown d5/d19/cc8 5811915 1 2024-02-19T19:06:34.221 INFO:tasks.workunit.client.0.smithi088.stdout:8/782: creat d9/df/d3d/d50/d71/da1/d6c/d77/fef x:0 0 0 2024-02-19T19:06:34.223 INFO:tasks.workunit.client.0.smithi088.stdout:8/783: fsync d9/df/d3d/d50/d71/da1/d70/fad 0 2024-02-19T19:06:34.224 INFO:tasks.workunit.client.0.smithi088.stdout:6/984: chown d5/d19/l90 1435563 1 2024-02-19T19:06:34.226 INFO:tasks.workunit.client.0.smithi088.stdout:6/985: readlink d5/de/d64/d8d/d95/lb7 0 2024-02-19T19:06:34.226 INFO:tasks.workunit.client.0.smithi088.stdout:8/784: chown d9/df/d3d/d50/d71/da1/la4 431118 1 2024-02-19T19:06:34.230 INFO:tasks.workunit.client.0.smithi088.stdout:2/774: getdents d3 0 2024-02-19T19:06:34.230 INFO:tasks.workunit.client.0.smithi088.stdout:6/986: write d5/d8/d3a/db8/fdc [1242156,97969] 0 2024-02-19T19:06:34.232 INFO:tasks.workunit.client.0.smithi088.stdout:4/715: dread d2/d32/d25/d45/f4a [4194304,4194304] 0 2024-02-19T19:06:34.235 INFO:tasks.workunit.client.0.smithi088.stdout:4/716: write d2/d32/d25/d45/fb0 [640900,55378] 0 2024-02-19T19:06:34.236 INFO:tasks.workunit.client.0.smithi088.stdout:2/775: rename d3/d1f/d4f/c57 to d3/d1b/d43/df4/cfd 0 2024-02-19T19:06:34.236 INFO:tasks.workunit.client.0.smithi088.stdout:1/874: dwrite d6/d21/d6a/d4a/fa7 [0,4194304] 0 2024-02-19T19:06:34.240 INFO:tasks.workunit.client.0.smithi088.stdout:8/785: getdents d9/df/d3d/d50/d71/da1/d6c/dcf 0 2024-02-19T19:06:34.241 INFO:tasks.workunit.client.0.smithi088.stdout:2/776: creat d3/d14/ffe x:0 0 0 2024-02-19T19:06:34.243 INFO:tasks.workunit.client.0.smithi088.stdout:1/875: symlink d6/d10/d60/db8/dd9/l11a 0 2024-02-19T19:06:34.243 INFO:tasks.workunit.client.0.smithi088.stdout:2/777: dread - d3/d14/f91 zero size 2024-02-19T19:06:34.244 INFO:tasks.workunit.client.0.smithi088.stdout:1/876: chown d6/fd 121240388 1 2024-02-19T19:06:34.246 INFO:tasks.workunit.client.0.smithi088.stdout:8/786: chown d9/df/d3d/d50/d71/da1/d6c/d77 0 1 2024-02-19T19:06:34.247 INFO:tasks.workunit.client.0.smithi088.stdout:2/778: rename d3/d14/c46 to d3/d14/cff 0 2024-02-19T19:06:34.251 INFO:tasks.workunit.client.0.smithi088.stdout:8/787: unlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/l80 0 2024-02-19T19:06:34.252 INFO:tasks.workunit.client.0.smithi088.stdout:4/717: write d2/d35/f59 [13669,126417] 0 2024-02-19T19:06:34.254 INFO:tasks.workunit.client.0.smithi088.stdout:2/779: fsync d3/d1b/d43/d61/d7e/db1/fbf 0 2024-02-19T19:06:34.256 INFO:tasks.workunit.client.0.smithi088.stdout:8/788: unlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8/fb5 0 2024-02-19T19:06:34.257 INFO:tasks.workunit.client.0.smithi088.stdout:4/718: fsync d2/d8/f6b 0 2024-02-19T19:06:34.260 INFO:tasks.workunit.client.0.smithi088.stdout:2/780: creat d3/d1b/d43/d74/f100 x:0 0 0 2024-02-19T19:06:34.262 INFO:tasks.workunit.client.0.smithi088.stdout:2/781: write d3/d1b/d43/d61/da6/fe2 [774945,70685] 0 2024-02-19T19:06:34.263 INFO:tasks.workunit.client.0.smithi088.stdout:8/789: symlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/lf0 0 2024-02-19T19:06:34.265 INFO:tasks.workunit.client.0.smithi088.stdout:1/877: dwrite d6/d10/f11 [0,4194304] 0 2024-02-19T19:06:34.265 INFO:tasks.workunit.client.0.smithi088.stdout:4/719: symlink d2/d32/d25/d26/da5/leb 0 2024-02-19T19:06:34.267 INFO:tasks.workunit.client.0.smithi088.stdout:2/782: write d3/d1f/f32 [6449116,108793] 0 2024-02-19T19:06:34.270 INFO:tasks.workunit.client.0.smithi088.stdout:2/783: stat d3/d1b/d43/d61/d7e/db1 0 2024-02-19T19:06:34.271 INFO:tasks.workunit.client.0.smithi088.stdout:8/790: dread d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/fda [0,4194304] 0 2024-02-19T19:06:34.273 INFO:tasks.workunit.client.0.smithi088.stdout:2/784: symlink d3/d14/dd8/de9/l101 0 2024-02-19T19:06:34.274 INFO:tasks.workunit.client.0.smithi088.stdout:8/791: mkdir d9/df/d3d/d50/df1 0 2024-02-19T19:06:34.278 INFO:tasks.workunit.client.0.smithi088.stdout:8/792: truncate d9/df/d1c/f13 1262383 0 2024-02-19T19:06:34.282 INFO:tasks.workunit.client.0.smithi088.stdout:4/720: dwrite d2/d32/d25/d26/d6d/d7e/d3a/d77/f92 [0,4194304] 0 2024-02-19T19:06:34.283 INFO:tasks.workunit.client.0.smithi088.stdout:8/793: creat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/ff2 x:0 0 0 2024-02-19T19:06:34.287 INFO:tasks.workunit.client.0.smithi088.stdout:8/794: read d9/df/f1a [3797874,29904] 0 2024-02-19T19:06:34.288 INFO:tasks.workunit.client.0.smithi088.stdout:4/721: read d2/d35/db3/fe2 [92337,14401] 0 2024-02-19T19:06:34.293 INFO:tasks.workunit.client.0.smithi088.stdout:8/795: read d9/df/d3d/d50/d71/f8b [63896,105280] 0 2024-02-19T19:06:34.294 INFO:tasks.workunit.client.0.smithi088.stdout:2/785: dread d3/ff0 [0,4194304] 0 2024-02-19T19:06:34.296 INFO:tasks.workunit.client.0.smithi088.stdout:2/786: mknod d3/d14/dd8/c102 0 2024-02-19T19:06:34.298 INFO:tasks.workunit.client.0.smithi088.stdout:8/796: rename d9/df/d3d/d50/d71/da1/d6c/d77/d97/ld3 to d9/df/d1c/d85/lf3 0 2024-02-19T19:06:34.298 INFO:tasks.workunit.client.0.smithi088.stdout:2/787: fsync d3/f65 0 2024-02-19T19:06:34.301 INFO:tasks.workunit.client.0.smithi088.stdout:8/797: mknod d9/df/d3d/d50/d71/da1/cf4 0 2024-02-19T19:06:34.304 INFO:tasks.workunit.client.0.smithi088.stdout:2/788: link d3/d14/dd8/lba d3/d1b/d43/d74/l103 0 2024-02-19T19:06:34.306 INFO:tasks.workunit.client.0.smithi088.stdout:2/789: read - d3/d1b/d43/df4/ffa zero size 2024-02-19T19:06:34.306 INFO:tasks.workunit.client.0.smithi088.stdout:4/722: dread d2/f75 [0,4194304] 0 2024-02-19T19:06:34.310 INFO:tasks.workunit.client.0.smithi088.stdout:4/723: dread d2/d35/db3/fe2 [0,4194304] 0 2024-02-19T19:06:34.313 INFO:tasks.workunit.client.0.smithi088.stdout:4/724: truncate d2/d32/d25/d26/d6d/d7e/d3a/fd4 251522 0 2024-02-19T19:06:34.313 INFO:tasks.workunit.client.0.smithi088.stdout:4/725: write d2/d32/d25/d3f/d70/fae [1242340,73595] 0 2024-02-19T19:06:34.316 INFO:tasks.workunit.client.0.smithi088.stdout:8/798: dread d9/df/d3d/d50/d71/da1/d6c/d9a/f5e [0,4194304] 0 2024-02-19T19:06:34.317 INFO:tasks.workunit.client.0.smithi088.stdout:4/726: unlink d2/l55 0 2024-02-19T19:06:34.324 INFO:tasks.workunit.client.0.smithi088.stdout:4/727: dwrite d2/d32/d25/d3f/d70/f8b [0,4194304] 0 2024-02-19T19:06:34.325 INFO:tasks.workunit.client.0.smithi088.stdout:8/799: truncate d9/df/f27 2433204 0 2024-02-19T19:06:34.326 INFO:tasks.workunit.client.0.smithi088.stdout:8/800: chown d9/df/d1c/f33 9985 1 2024-02-19T19:06:34.326 INFO:tasks.workunit.client.0.smithi088.stdout:4/728: fdatasync d2/d35/f7c 0 2024-02-19T19:06:34.329 INFO:tasks.workunit.client.0.smithi088.stdout:8/801: write d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/ff2 [448133,39880] 0 2024-02-19T19:06:34.329 INFO:tasks.workunit.client.0.smithi088.stdout:2/790: dread d3/d1b/d43/d61/da6/db2/fdf [0,4194304] 0 2024-02-19T19:06:34.332 INFO:tasks.workunit.client.0.smithi088.stdout:4/729: symlink d2/d32/d25/d26/da5/db2/lec 0 2024-02-19T19:06:34.333 INFO:tasks.workunit.client.0.smithi088.stdout:2/791: mkdir d3/d14/d104 0 2024-02-19T19:06:34.335 INFO:tasks.workunit.client.0.smithi088.stdout:4/730: rmdir d2/d32/d25/d4b 39 2024-02-19T19:06:34.336 INFO:tasks.workunit.client.0.smithi088.stdout:8/802: symlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/db3/lf5 0 2024-02-19T19:06:34.338 INFO:tasks.workunit.client.0.smithi088.stdout:4/731: truncate d2/d32/d25/d26/f5e 1056943 0 2024-02-19T19:06:34.340 INFO:tasks.workunit.client.0.smithi088.stdout:2/792: creat d3/d1b/d43/d61/d7e/f105 x:0 0 0 2024-02-19T19:06:34.343 INFO:tasks.workunit.client.0.smithi088.stdout:8/803: dwrite d9/df/d1c/f16 [8388608,4194304] 0 2024-02-19T19:06:34.347 INFO:tasks.workunit.client.0.smithi088.stdout:2/793: rename d3/d1b/d43/d61/d64/fed to d3/d1b/d43/d61/f106 0 2024-02-19T19:06:34.348 INFO:tasks.workunit.client.0.smithi088.stdout:4/732: dwrite d2/f4 [0,4194304] 0 2024-02-19T19:06:34.351 INFO:tasks.workunit.client.0.smithi088.stdout:8/804: fsync d9/df/d3d/d50/d71/da1/d6c/d9a/f4d 0 2024-02-19T19:06:34.351 INFO:tasks.workunit.client.0.smithi088.stdout:4/733: chown d2/d32/d25/d26/d4e/d6f/l87 16210 1 2024-02-19T19:06:34.354 INFO:tasks.workunit.client.0.smithi088.stdout:4/734: write d2/d32/d25/d3f/d70/fae [2172735,22392] 0 2024-02-19T19:06:34.356 INFO:tasks.workunit.client.0.smithi088.stdout:8/805: read - d9/df/d3d/d50/d71/da1/d6c/d77/d97/fc7 zero size 2024-02-19T19:06:34.408 INFO:tasks.workunit.client.0.smithi088.stdout:2/794: dwrite d3/d1f/fdc [0,4194304] 0 2024-02-19T19:06:34.411 INFO:tasks.workunit.client.0.smithi088.stdout:8/806: mkdir d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/df6 0 2024-02-19T19:06:34.419 INFO:tasks.workunit.client.0.smithi088.stdout:8/807: dwrite d9/df/d3d/d50/d71/fd4 [4194304,4194304] 0 2024-02-19T19:06:34.421 INFO:tasks.workunit.client.0.smithi088.stdout:8/808: fdatasync d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/fd5 0 2024-02-19T19:06:34.470 INFO:tasks.workunit.client.0.smithi088.stdout:4/735: write d2/f8d [438615,62499] 0 2024-02-19T19:06:34.471 INFO:tasks.workunit.client.0.smithi088.stdout:4/736: chown d2/d32/d25/d26/d6d/d7e/d3a/l5a 454783 1 2024-02-19T19:06:34.472 INFO:tasks.workunit.client.0.smithi088.stdout:4/737: chown d2/d35/db3/db7/dd7 50 1 2024-02-19T19:06:34.474 INFO:tasks.workunit.client.0.smithi088.stdout:8/809: rename d9/df/d1c/d85/lb0 to d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8/lf7 0 2024-02-19T19:06:34.475 INFO:tasks.workunit.client.0.smithi088.stdout:2/795: write d3/d1b/d43/d8f/ff1 [759072,112712] 0 2024-02-19T19:06:34.477 INFO:tasks.workunit.client.0.smithi088.stdout:8/810: chown d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/fd5 1770592 1 2024-02-19T19:06:34.483 INFO:tasks.workunit.client.0.smithi088.stdout:8/811: symlink d9/df/d3d/d62/lf8 0 2024-02-19T19:06:34.491 INFO:tasks.workunit.client.0.smithi088.stdout:4/738: dread d2/d32/d25/d45/f4a [0,4194304] 0 2024-02-19T19:06:34.492 INFO:tasks.workunit.client.0.smithi088.stdout:4/739: stat d2/d32/d25/d26/d6d/d7e/d3a/d77/c96 0 2024-02-19T19:06:34.497 INFO:tasks.workunit.client.0.smithi088.stdout:8/812: dread d9/df/d1c/f1f [0,4194304] 0 2024-02-19T19:06:34.505 INFO:tasks.workunit.client.0.smithi088.stdout:8/813: mknod d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/dea/dec/cf9 0 2024-02-19T19:06:34.507 INFO:tasks.workunit.client.0.smithi088.stdout:8/814: read d9/df/d3d/f5f [745511,68101] 0 2024-02-19T19:06:34.512 INFO:tasks.workunit.client.0.smithi088.stdout:8/815: creat d9/df/d3d/d62/ffa x:0 0 0 2024-02-19T19:06:34.518 INFO:tasks.workunit.client.0.smithi088.stdout:8/816: rename d9/df/d3d/d50/d71/da1/d6c/ca2 to d9/df/d3d/d50/d71/da1/d6c/dcf/cfb 0 2024-02-19T19:06:34.526 INFO:tasks.workunit.client.0.smithi088.stdout:8/817: dwrite d9/df/d3d/d50/d71/da1/d6c/d9a/f44 [4194304,4194304] 0 2024-02-19T19:06:34.537 INFO:tasks.workunit.client.0.smithi088.stdout:8/818: mkdir d9/df/d3d/dfc 0 2024-02-19T19:06:34.552 INFO:tasks.workunit.client.0.smithi088.stdout:8/819: dwrite d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/fe9 [0,4194304] 0 2024-02-19T19:06:34.557 INFO:tasks.workunit.client.0.smithi088.stdout:8/820: read d9/df/d3d/d50/d71/da1/d6c/d9a/f57 [1243612,117669] 0 2024-02-19T19:06:34.570 INFO:tasks.workunit.client.0.smithi088.stdout:2/796: creat d3/f107 x:0 0 0 2024-02-19T19:06:34.610 INFO:tasks.workunit.client.0.smithi088.stdout:2/797: read d3/d1f/d4f/dc6/fe0 [1319086,90712] 0 2024-02-19T19:06:34.612 INFO:tasks.workunit.client.0.smithi088.stdout:2/798: readlink d3/d1b/d43/d61/lc9 0 2024-02-19T19:06:34.614 INFO:tasks.workunit.client.0.smithi088.stdout:2/799: chown d3/fe8 0 1 2024-02-19T19:06:34.614 INFO:tasks.workunit.client.0.smithi088.stdout:2/800: chown d3/d1f/d4f/dc6 4792271 1 2024-02-19T19:06:34.616 INFO:tasks.workunit.client.0.smithi088.stdout:2/801: read d3/d1b/d43/d74/f82 [2090873,97948] 0 2024-02-19T19:06:34.617 INFO:tasks.workunit.client.0.smithi088.stdout:2/802: fsync d3/d1b/d43/d61/da6/db2/fdf 0 2024-02-19T19:06:34.618 INFO:tasks.workunit.client.0.smithi088.stdout:2/803: chown d3/d71/cb8 2014 1 2024-02-19T19:06:34.618 INFO:tasks.workunit.client.0.smithi088.stdout:2/804: fdatasync d3/d1f/d5d/fc4 0 2024-02-19T19:06:34.620 INFO:tasks.workunit.client.0.smithi088.stdout:2/805: creat d3/d1b/d43/d8f/f108 x:0 0 0 2024-02-19T19:06:34.625 INFO:tasks.workunit.client.0.smithi088.stdout:2/806: dwrite d3/d1f/d5d/fb4 [0,4194304] 0 2024-02-19T19:06:34.627 INFO:tasks.workunit.client.0.smithi088.stdout:2/807: creat d3/d1f/d5d/f109 x:0 0 0 2024-02-19T19:06:34.629 INFO:tasks.workunit.client.0.smithi088.stdout:2/808: truncate d3/d1b/d43/d61/f9e 819471 0 2024-02-19T19:06:34.632 INFO:tasks.workunit.client.0.smithi088.stdout:2/809: creat d3/d14/d104/f10a x:0 0 0 2024-02-19T19:06:34.634 INFO:tasks.workunit.client.0.smithi088.stdout:2/810: creat d3/d14/dd8/de9/f10b x:0 0 0 2024-02-19T19:06:34.636 INFO:tasks.workunit.client.0.smithi088.stdout:2/811: unlink d3/d1b/d43/d61/da6/db2/fcc 0 2024-02-19T19:06:34.637 INFO:tasks.workunit.client.0.smithi088.stdout:2/812: write d3/d1b/d43/d8f/f108 [890546,102676] 0 2024-02-19T19:06:34.639 INFO:tasks.workunit.client.0.smithi088.stdout:2/813: mkdir d3/d1b/d43/d61/d7e/db1/d10c 0 2024-02-19T19:06:34.652 INFO:tasks.workunit.client.0.smithi088.stdout:7/929: sync 2024-02-19T19:06:34.652 INFO:tasks.workunit.client.0.smithi088.stdout:7/930: fdatasync d14/f29 0 2024-02-19T19:06:34.655 INFO:tasks.workunit.client.0.smithi088.stdout:7/931: dread d14/f74 [0,4194304] 0 2024-02-19T19:06:34.657 INFO:tasks.workunit.client.0.smithi088.stdout:7/932: mkdir d14/d27/d49/d98/d107/d146 0 2024-02-19T19:06:34.661 INFO:tasks.workunit.client.0.smithi088.stdout:7/933: dread d14/d27/d49/f7d [0,4194304] 0 2024-02-19T19:06:34.663 INFO:tasks.workunit.client.0.smithi088.stdout:7/934: symlink d14/d1d/d3c/d43/l147 0 2024-02-19T19:06:34.664 INFO:tasks.workunit.client.0.smithi088.stdout:7/935: write d14/d1d/d62/dcf/f129 [8532673,24372] 0 2024-02-19T19:06:34.666 INFO:tasks.workunit.client.0.smithi088.stdout:7/936: mknod d14/d28/d94/ddb/d115/c148 0 2024-02-19T19:06:34.667 INFO:tasks.workunit.client.0.smithi088.stdout:7/937: fsync f13 0 2024-02-19T19:06:34.670 INFO:tasks.workunit.client.0.smithi088.stdout:7/938: creat d14/d27/d6b/df8/f149 x:0 0 0 2024-02-19T19:06:34.671 INFO:tasks.workunit.client.0.smithi088.stdout:7/939: write d14/d28/d3b/dd3/f10e [1764530,15315] 0 2024-02-19T19:06:34.672 INFO:tasks.workunit.client.0.smithi088.stdout:7/940: symlink d14/d27/d6b/dc4/l14a 0 2024-02-19T19:06:34.673 INFO:tasks.workunit.client.0.smithi088.stdout:7/941: write d14/d1d/d62/dcf/f133 [278458,15166] 0 2024-02-19T19:06:34.675 INFO:tasks.workunit.client.0.smithi088.stdout:7/942: mkdir d14/d27/d6b/df8/d14b 0 2024-02-19T19:06:34.686 INFO:tasks.workunit.client.0.smithi088.stdout:7/943: dread d14/d27/d49/d98/dcb/de7/f10d [0,4194304] 0 2024-02-19T19:06:34.806 INFO:tasks.workunit.client.0.smithi088.stdout:7/944: dread d14/d28/d3b/f4a [0,4194304] 0 2024-02-19T19:06:34.818 INFO:tasks.workunit.client.0.smithi088.stdout:8/821: dwrite d9/f63 [0,4194304] 0 2024-02-19T19:06:34.822 INFO:tasks.workunit.client.0.smithi088.stdout:7/945: dread d14/d1d/d62/dcf/f129 [0,4194304] 0 2024-02-19T19:06:34.912 INFO:tasks.workunit.client.0.smithi088.stdout:7/946: rename d14/d1d/d3c/d43/d5c/d9a/d92/f137 to d14/d27/d6b/dc4/f14c 0 2024-02-19T19:06:34.919 INFO:tasks.workunit.client.0.smithi088.stdout:2/814: write d3/d14/f91 [813466,29511] 0 2024-02-19T19:06:34.920 INFO:tasks.workunit.client.0.smithi088.stdout:2/815: read - d3/d1b/d43/d61/da6/db2/fd7 zero size 2024-02-19T19:06:34.925 INFO:tasks.workunit.client.0.smithi088.stdout:2/816: dwrite d3/d14/f5a [4194304,4194304] 0 2024-02-19T19:06:34.936 INFO:tasks.workunit.client.0.smithi088.stdout:0/859: sync 2024-02-19T19:06:34.939 INFO:tasks.workunit.client.0.smithi088.stdout:0/860: read d13/d62/d8a/d95/d5b/f108 [2522228,10006] 0 2024-02-19T19:06:34.940 INFO:tasks.workunit.client.0.smithi088.stdout:0/861: write d13/d62/d8a/d9c/da8/de2/f6e [5058001,102363] 0 2024-02-19T19:06:34.941 INFO:tasks.workunit.client.0.smithi088.stdout:0/862: stat d13/d62/d8a/d95/d5b/d76/fbd 0 2024-02-19T19:06:34.942 INFO:tasks.workunit.client.0.smithi088.stdout:0/863: write d13/d62/d8a/d95/d5a/d25/d2a/d3c/f5d [1502251,65529] 0 2024-02-19T19:06:34.986 INFO:tasks.workunit.client.0.smithi088.stdout:7/947: getdents d14/d27/d6b/dc4 0 2024-02-19T19:06:35.000 INFO:tasks.workunit.client.0.smithi088.stdout:3/924: sync 2024-02-19T19:06:35.000 INFO:tasks.workunit.client.0.smithi088.stdout:6/987: sync 2024-02-19T19:06:35.000 INFO:tasks.workunit.client.0.smithi088.stdout:4/740: sync 2024-02-19T19:06:35.001 INFO:tasks.workunit.client.0.smithi088.stdout:1/878: sync 2024-02-19T19:06:35.002 INFO:tasks.workunit.client.0.smithi088.stdout:1/879: write d6/f99 [1978166,118843] 0 2024-02-19T19:06:35.004 INFO:tasks.workunit.client.0.smithi088.stdout:1/880: truncate d6/d21/d64/d94/dab/f107 884268 0 2024-02-19T19:06:35.012 INFO:tasks.workunit.client.0.smithi088.stdout:8/822: write d9/df/d3d/d50/d71/da1/f54 [2376332,91979] 0 2024-02-19T19:06:35.013 INFO:tasks.workunit.client.0.smithi088.stdout:2/817: creat d3/d1b/d43/d61/d7e/db1/d10c/f10d x:0 0 0 2024-02-19T19:06:35.018 INFO:tasks.workunit.client.0.smithi088.stdout:0/864: mknod d13/d62/d8a/c12b 0 2024-02-19T19:06:35.019 INFO:tasks.workunit.client.0.smithi088.stdout:0/865: chown d13/d81/cb1 1268738171 1 2024-02-19T19:06:35.081 INFO:tasks.workunit.client.0.smithi088.stdout:6/988: creat d5/de/d60/db1/f13b x:0 0 0 2024-02-19T19:06:35.085 INFO:tasks.workunit.client.0.smithi088.stdout:6/989: truncate d5/d7/dc9/f127 948916 0 2024-02-19T19:06:35.089 INFO:tasks.workunit.client.0.smithi088.stdout:0/866: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/d10a/d106/de5/f103 [0,4194304] 0 2024-02-19T19:06:35.114 INFO:tasks.workunit.client.0.smithi088.stdout:6/990: dread d5/d8/d77/f111 [0,4194304] 0 2024-02-19T19:06:35.200 INFO:tasks.workunit.client.0.smithi088.stdout:8/823: symlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/lfd 0 2024-02-19T19:06:35.231 INFO:tasks.workunit.client.0.smithi088.stdout:3/925: sync 2024-02-19T19:06:35.388 INFO:tasks.workunit.client.0.smithi088.stdout:1/881: symlink d6/d21/d64/d108/l11b 0 2024-02-19T19:06:35.397 INFO:tasks.workunit.client.0.smithi088.stdout:0/867: symlink d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/l12c 0 2024-02-19T19:06:35.398 INFO:tasks.workunit.client.0.smithi088.stdout:6/991: mkdir d5/d7/d32/d56/d13c 0 2024-02-19T19:06:35.406 INFO:tasks.workunit.client.0.smithi088.stdout:4/741: truncate d2/f75 1039950 0 2024-02-19T19:06:35.407 INFO:tasks.workunit.client.0.smithi088.stdout:1/882: creat d6/d21/d6a/d4a/dad/f11c x:0 0 0 2024-02-19T19:06:35.411 INFO:tasks.workunit.client.0.smithi088.stdout:6/992: truncate d5/d8/f7b 2595650 0 2024-02-19T19:06:35.421 INFO:tasks.workunit.client.0.smithi088.stdout:4/742: dread d2/d35/f53 [0,4194304] 0 2024-02-19T19:06:35.425 INFO:tasks.workunit.client.0.smithi088.stdout:4/743: write d2/d32/d25/d26/d6d/d7e/d3a/d77/f92 [656047,84189] 0 2024-02-19T19:06:35.430 INFO:tasks.workunit.client.0.smithi088.stdout:6/993: dread d5/de/d60/db1/ff9 [0,4194304] 0 2024-02-19T19:06:35.486 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:35 smithi088 ceph-mon[88643]: pgmap v51: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 56 MiB/s rd, 128 MiB/s wr, 291 op/s 2024-02-19T19:06:35.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:35 smithi132 ceph-mon[70676]: pgmap v51: 65 pgs: 65 active+clean; 1.6 GiB data, 5.2 GiB used, 531 GiB / 536 GiB avail; 56 MiB/s rd, 128 MiB/s wr, 291 op/s 2024-02-19T19:06:35.703 INFO:tasks.workunit.client.0.smithi088.stdout:0/868: dwrite d13/d62/d8a/d95/d5a/f9d [0,4194304] 0 2024-02-19T19:06:35.714 INFO:tasks.workunit.client.0.smithi088.stdout:6/994: mknod d5/d8/d77/c13d 0 2024-02-19T19:06:35.715 INFO:tasks.workunit.client.0.smithi088.stdout:6/995: chown d5/d7/d32/l93 3 1 2024-02-19T19:06:35.722 INFO:tasks.workunit.client.0.smithi088.stdout:6/996: dwrite d5/de/d60/f103 [0,4194304] 0 2024-02-19T19:06:35.735 INFO:tasks.workunit.client.0.smithi088.stdout:6/997: dread d5/d8/d2c/f4c [0,4194304] 0 2024-02-19T19:06:35.736 INFO:tasks.workunit.client.0.smithi088.stdout:6/998: readlink d5/de/d97/led 0 2024-02-19T19:06:35.736 INFO:tasks.workunit.client.0.smithi088.stdout:6/999: stat d5/d7/dc9/dca 0 2024-02-19T19:06:35.772 INFO:tasks.workunit.client.0.smithi088.stdout:8/824: getdents d9/df/d3d/d62 0 2024-02-19T19:06:35.772 INFO:tasks.workunit.client.0.smithi088.stdout:3/926: getdents d4/d22/d7a/de7 0 2024-02-19T19:06:35.774 INFO:tasks.workunit.client.0.smithi088.stdout:4/744: link d2/d32/d25/d3f/d70/lb6 d2/d32/d25/d26/da5/db2/led 0 2024-02-19T19:06:35.777 INFO:tasks.workunit.client.0.smithi088.stdout:8/825: rename d9/fc9 to d9/df/d1c/d22/ffe 0 2024-02-19T19:06:35.778 INFO:tasks.workunit.client.0.smithi088.stdout:8/826: dread - d9/df/d3d/d50/d71/da1/d6c/d77/fef zero size 2024-02-19T19:06:35.778 INFO:tasks.workunit.client.0.smithi088.stdout:3/927: fdatasync d4/d1a/d23/f69 0 2024-02-19T19:06:35.780 INFO:tasks.workunit.client.0.smithi088.stdout:4/745: write d2/d35/db3/db7/fc5 [34744,67667] 0 2024-02-19T19:06:35.783 INFO:tasks.workunit.client.0.smithi088.stdout:3/928: mknod d4/d22/d7a/de7/c141 0 2024-02-19T19:06:35.788 INFO:tasks.workunit.client.0.smithi088.stdout:8/827: mkdir d9/df/d3d/dfc/dff 0 2024-02-19T19:06:35.790 INFO:tasks.workunit.client.0.smithi088.stdout:8/828: stat d9/df/d3d/d50/d71/da1/le4 0 2024-02-19T19:06:35.791 INFO:tasks.workunit.client.0.smithi088.stdout:8/829: chown d9/df/d3d/d50/d71/da1/la4 2753452 1 2024-02-19T19:06:35.792 INFO:tasks.workunit.client.0.smithi088.stdout:1/883: write d6/d10/f3b [1446336,18446] 0 2024-02-19T19:06:35.796 INFO:tasks.workunit.client.0.smithi088.stdout:8/830: rename d9/f2e to d9/df/d3d/dfc/f100 0 2024-02-19T19:06:35.797 INFO:tasks.workunit.client.0.smithi088.stdout:8/831: chown d9/df/d3d/d50/d71/da1/d6c/d77/d97/fc7 259612031 1 2024-02-19T19:06:35.799 INFO:tasks.workunit.client.0.smithi088.stdout:8/832: fdatasync d9/df/d3d/f9f 0 2024-02-19T19:06:35.801 INFO:tasks.workunit.client.0.smithi088.stdout:3/929: write d4/f38 [4488778,112296] 0 2024-02-19T19:06:35.805 INFO:tasks.workunit.client.0.smithi088.stdout:1/884: truncate d6/f92 777835 0 2024-02-19T19:06:35.809 INFO:tasks.workunit.client.0.smithi088.stdout:1/885: write d6/d21/d64/d94/d71/fcd [1438067,26559] 0 2024-02-19T19:06:35.811 INFO:tasks.workunit.client.0.smithi088.stdout:3/930: dwrite d4/f6b [0,4194304] 0 2024-02-19T19:06:35.821 INFO:tasks.workunit.client.0.smithi088.stdout:8/833: dread d9/df/fb4 [0,4194304] 0 2024-02-19T19:06:35.829 INFO:tasks.workunit.client.0.smithi088.stdout:3/931: fdatasync d4/d1a/d1e/d74/f8a 0 2024-02-19T19:06:35.867 INFO:tasks.workunit.client.0.smithi088.stdout:8/834: symlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/l101 0 2024-02-19T19:06:35.912 INFO:tasks.workunit.client.0.smithi088.stdout:8/835: mkdir d9/df/d1c/d22/d9e/d102 0 2024-02-19T19:06:35.914 INFO:tasks.workunit.client.0.smithi088.stdout:3/932: write d4/d1a/d1e/d31/d32/d55/fb3 [204522,6076] 0 2024-02-19T19:06:35.914 INFO:tasks.workunit.client.0.smithi088.stdout:3/933: chown d4/d12/d33/d43 31949962 1 2024-02-19T19:06:35.916 INFO:tasks.workunit.client.0.smithi088.stdout:8/836: creat d9/df/d3d/d50/d71/f103 x:0 0 0 2024-02-19T19:06:35.917 INFO:tasks.workunit.client.0.smithi088.stdout:3/934: creat d4/d22/d7a/d82/f142 x:0 0 0 2024-02-19T19:06:35.920 INFO:tasks.workunit.client.0.smithi088.stdout:3/935: creat d4/d87/d118/f143 x:0 0 0 2024-02-19T19:06:35.936 INFO:tasks.workunit.client.0.smithi088.stdout:7/948: symlink d14/d1d/d62/dcf/d108/l14d 0 2024-02-19T19:06:35.952 INFO:tasks.workunit.client.0.smithi088.stdout:7/949: dread d14/f22 [0,4194304] 0 2024-02-19T19:06:35.956 INFO:tasks.workunit.client.0.smithi088.stdout:8/837: rename d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/da8 to d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/d104 0 2024-02-19T19:06:35.959 INFO:tasks.workunit.client.0.smithi088.stdout:7/950: dwrite d14/f1f [4194304,4194304] 0 2024-02-19T19:06:35.959 INFO:tasks.workunit.client.0.smithi088.stdout:8/838: truncate d9/df/d1c/d85/fb2 1781908 0 2024-02-19T19:06:35.963 INFO:tasks.workunit.client.0.smithi088.stdout:8/839: dread - d9/df/d3d/d50/d71/da1/d6c/fba zero size 2024-02-19T19:06:35.964 INFO:tasks.workunit.client.0.smithi088.stdout:8/840: chown d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/cb9 8688 1 2024-02-19T19:06:35.970 INFO:tasks.workunit.client.0.smithi088.stdout:8/841: read d9/df/d3d/d50/d71/fe5 [72295,128952] 0 2024-02-19T19:06:35.974 INFO:tasks.workunit.client.0.smithi088.stdout:8/842: fdatasync d9/df/d3d/d50/d71/da1/d6c/fa0 0 2024-02-19T19:06:35.979 INFO:tasks.workunit.client.0.smithi088.stdout:2/818: getdents d3/d14/d104 0 2024-02-19T19:06:35.982 INFO:tasks.workunit.client.0.smithi088.stdout:3/936: dwrite d4/d1a/d1e/d74/d7d/daa/f90 [0,4194304] 0 2024-02-19T19:06:35.984 INFO:tasks.workunit.client.0.smithi088.stdout:7/951: dread d14/f60 [0,4194304] 0 2024-02-19T19:06:35.992 INFO:tasks.workunit.client.0.smithi088.stdout:3/937: dwrite d4/fd [0,4194304] 0 2024-02-19T19:06:36.003 INFO:tasks.workunit.client.0.smithi088.stdout:7/952: symlink d14/d28/d3b/dd3/l14e 0 2024-02-19T19:06:36.003 INFO:tasks.workunit.client.0.smithi088.stdout:7/953: chown d14/d1d/c4e 322369 1 2024-02-19T19:06:36.011 INFO:tasks.workunit.client.0.smithi088.stdout:7/954: link d14/d28/d65/ce8 d14/d27/d49/d98/c14f 0 2024-02-19T19:06:36.012 INFO:tasks.workunit.client.0.smithi088.stdout:8/843: rename d9/df/d1c/d85/fb2 to d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/dea/f105 0 2024-02-19T19:06:36.013 INFO:tasks.workunit.client.0.smithi088.stdout:7/955: readlink d14/d27/d6b/dba/d102/l13f 0 2024-02-19T19:06:36.015 INFO:tasks.workunit.client.0.smithi088.stdout:8/844: truncate d9/df/d3d/d50/f78 921413 0 2024-02-19T19:06:36.023 INFO:tasks.workunit.client.0.smithi088.stdout:8/845: creat d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/dea/f106 x:0 0 0 2024-02-19T19:06:36.025 INFO:tasks.workunit.client.0.smithi088.stdout:7/956: dwrite fd [0,4194304] 0 2024-02-19T19:06:36.033 INFO:tasks.workunit.client.0.smithi088.stdout:3/938: dwrite d4/d1a/d1e/d31/d32/f4c [0,4194304] 0 2024-02-19T19:06:36.035 INFO:tasks.workunit.client.0.smithi088.stdout:0/869: dread d13/d62/d8a/d95/d5a/f9d [0,4194304] 0 2024-02-19T19:06:36.039 INFO:tasks.workunit.client.0.smithi088.stdout:3/939: fdatasync d4/d1a/d1e/d74/d7d/daa/dcb/f10c 0 2024-02-19T19:06:36.043 INFO:tasks.workunit.client.0.smithi088.stdout:2/819: dread d3/d1f/d4f/dc6/fe0 [0,4194304] 0 2024-02-19T19:06:36.045 INFO:tasks.workunit.client.0.smithi088.stdout:8/846: dwrite d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/fd2 [0,4194304] 0 2024-02-19T19:06:36.047 INFO:tasks.workunit.client.0.smithi088.stdout:8/847: chown d9/df/d1c/lee 1727425 1 2024-02-19T19:06:36.055 INFO:tasks.workunit.client.0.smithi088.stdout:8/848: dwrite d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/ff2 [0,4194304] 0 2024-02-19T19:06:36.062 INFO:tasks.workunit.client.0.smithi088.stdout:3/940: truncate d4/d1a/d1e/d31/d32/f8c 397932 0 2024-02-19T19:06:36.062 INFO:tasks.workunit.client.0.smithi088.stdout:8/849: creat d9/df/d3d/d50/d71/f107 x:0 0 0 2024-02-19T19:06:36.063 INFO:tasks.workunit.client.0.smithi088.stdout:7/957: mknod d14/d28/d3b/c150 0 2024-02-19T19:06:36.063 INFO:tasks.workunit.client.0.smithi088.stdout:0/870: symlink d13/d62/d8a/d95/d5a/d25/d2a/d34/l12d 0 2024-02-19T19:06:36.064 INFO:tasks.workunit.client.0.smithi088.stdout:3/941: chown d4/d1a/d1e/d74/d7d/daa/dcb 41586 1 2024-02-19T19:06:36.067 INFO:tasks.workunit.client.0.smithi088.stdout:3/942: write d4/d12/d33/fe4 [1018146,76922] 0 2024-02-19T19:06:36.067 INFO:tasks.workunit.client.0.smithi088.stdout:2/820: read d3/fbe [1709972,71844] 0 2024-02-19T19:06:36.068 INFO:tasks.workunit.client.0.smithi088.stdout:8/850: creat d9/df/d3d/d50/d71/da1/d82/f108 x:0 0 0 2024-02-19T19:06:36.069 INFO:tasks.workunit.client.0.smithi088.stdout:7/958: creat d14/d27/d6b/df8/d10b/f151 x:0 0 0 2024-02-19T19:06:36.069 INFO:tasks.workunit.client.0.smithi088.stdout:0/871: creat d13/d62/d8a/d95/d96/f12e x:0 0 0 2024-02-19T19:06:36.071 INFO:tasks.workunit.client.0.smithi088.stdout:7/959: dread - d14/d1d/d3c/d43/d5c/d75/d76/f90 zero size 2024-02-19T19:06:36.072 INFO:tasks.workunit.client.0.smithi088.stdout:3/943: creat d4/d87/f144 x:0 0 0 2024-02-19T19:06:36.073 INFO:tasks.workunit.client.0.smithi088.stdout:8/851: rename d9/df/d1c/c95 to d9/df/d1c/d22/d9e/d102/c109 0 2024-02-19T19:06:36.074 INFO:tasks.workunit.client.0.smithi088.stdout:0/872: dread - d13/d62/d8a/d9c/da8/dab/db4/dbf/ff5 zero size 2024-02-19T19:06:36.074 INFO:tasks.workunit.client.0.smithi088.stdout:2/821: creat d3/d1b/d43/d61/f10e x:0 0 0 2024-02-19T19:06:36.076 INFO:tasks.workunit.client.0.smithi088.stdout:7/960: creat d14/d1d/d3c/d43/dc2/f152 x:0 0 0 2024-02-19T19:06:36.077 INFO:tasks.workunit.client.0.smithi088.stdout:8/852: creat d9/df/d1c/f10a x:0 0 0 2024-02-19T19:06:36.077 INFO:tasks.workunit.client.0.smithi088.stdout:2/822: creat d3/d1b/d43/d61/d64/f10f x:0 0 0 2024-02-19T19:06:36.078 INFO:tasks.workunit.client.0.smithi088.stdout:7/961: rename d14 to d14/d1d/d3c/d43/dc2/d153 22 2024-02-19T19:06:36.078 INFO:tasks.workunit.client.0.smithi088.stdout:3/944: creat d4/d1a/d1e/d74/d7d/daa/dcb/f145 x:0 0 0 2024-02-19T19:06:36.081 INFO:tasks.workunit.client.0.smithi088.stdout:7/962: stat d14/d1d/d3c/d43/d5c/d9a/df4/d134 0 2024-02-19T19:06:36.082 INFO:tasks.workunit.client.0.smithi088.stdout:8/853: stat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/la3 0 2024-02-19T19:06:36.084 INFO:tasks.workunit.client.0.smithi088.stdout:3/945: unlink d4/d12/f2e 0 2024-02-19T19:06:36.088 INFO:tasks.workunit.client.0.smithi088.stdout:7/963: mkdir d14/d1d/d3c/d43/d5c/d75/d76/d154 0 2024-02-19T19:06:36.088 INFO:tasks.workunit.client.0.smithi088.stdout:3/946: symlink d4/d1a/d1e/d31/l146 0 2024-02-19T19:06:36.089 INFO:tasks.workunit.client.0.smithi088.stdout:8/854: read d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/f8a [3964938,111113] 0 2024-02-19T19:06:36.091 INFO:tasks.workunit.client.0.smithi088.stdout:2/823: dwrite d3/d1b/d43/d61/d7e/db1/d10c/f10d [0,4194304] 0 2024-02-19T19:06:36.096 INFO:tasks.workunit.client.0.smithi088.stdout:0/873: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d3c/fc5 [0,4194304] 0 2024-02-19T19:06:36.099 INFO:tasks.workunit.client.0.smithi088.stdout:2/824: mkdir d3/d14/dd8/de9/d110 0 2024-02-19T19:06:36.101 INFO:tasks.workunit.client.0.smithi088.stdout:7/964: write d14/d1d/d3c/d43/d5c/d9a/faf [2787406,78342] 0 2024-02-19T19:06:36.102 INFO:tasks.workunit.client.0.smithi088.stdout:0/874: creat d13/d62/d8a/dfe/f12f x:0 0 0 2024-02-19T19:06:36.104 INFO:tasks.workunit.client.0.smithi088.stdout:7/965: unlink d14/d28/d94/ddb/f103 0 2024-02-19T19:06:36.108 INFO:tasks.workunit.client.0.smithi088.stdout:7/966: symlink d14/d1d/l155 0 2024-02-19T19:06:36.108 INFO:tasks.workunit.client.0.smithi088.stdout:0/875: truncate d13/d62/d8a/d95/d5a/d25/d2a/d3c/f55 2397095 0 2024-02-19T19:06:36.111 INFO:tasks.workunit.client.0.smithi088.stdout:3/947: write d4/d12/d33/d58/ffb [801017,15658] 0 2024-02-19T19:06:36.113 INFO:tasks.workunit.client.0.smithi088.stdout:3/948: write d4/d22/d7a/de7/f122 [1020911,125016] 0 2024-02-19T19:06:36.116 INFO:tasks.workunit.client.0.smithi088.stdout:0/876: dwrite d13/de9/f116 [0,4194304] 0 2024-02-19T19:06:36.120 INFO:tasks.workunit.client.0.smithi088.stdout:0/877: symlink d13/d62/d8a/d95/d5a/d25/d2a/d3c/l130 0 2024-02-19T19:06:36.122 INFO:tasks.workunit.client.0.smithi088.stdout:8/855: truncate d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/d104/f89 4937711 0 2024-02-19T19:06:36.127 INFO:tasks.workunit.client.0.smithi088.stdout:0/878: rmdir d13/de9/d120 0 2024-02-19T19:06:36.127 INFO:tasks.workunit.client.0.smithi088.stdout:7/967: dread d14/d1d/d3c/f8e [0,4194304] 0 2024-02-19T19:06:36.129 INFO:tasks.workunit.client.0.smithi088.stdout:7/968: fsync d14/d28/d3b/dd3/fae 0 2024-02-19T19:06:36.131 INFO:tasks.workunit.client.0.smithi088.stdout:8/856: read d9/df/d3d/d50/d71/da1/d6c/d9a/f46 [3927753,90019] 0 2024-02-19T19:06:36.134 INFO:tasks.workunit.client.0.smithi088.stdout:8/857: creat d9/df/d1c/d85/dc5/f10b x:0 0 0 2024-02-19T19:06:36.137 INFO:tasks.workunit.client.0.smithi088.stdout:0/879: write d13/d62/d8a/d95/d5a/d25/d2a/d34/fa3 [6609,10736] 0 2024-02-19T19:06:36.138 INFO:tasks.workunit.client.0.smithi088.stdout:0/880: fsync d13/d62/d8a/d95/d5a/d25/d2a/d34/f75 0 2024-02-19T19:06:36.148 INFO:tasks.workunit.client.0.smithi088.stdout:7/969: dread d14/d28/fe0 [0,4194304] 0 2024-02-19T19:06:36.150 INFO:tasks.workunit.client.0.smithi088.stdout:7/970: creat d14/d1d/d3c/d43/dc2/f156 x:0 0 0 2024-02-19T19:06:36.151 INFO:tasks.workunit.client.0.smithi088.stdout:8/858: dread d9/df/d3d/d50/d71/fd4 [0,4194304] 0 2024-02-19T19:06:36.154 INFO:tasks.workunit.client.0.smithi088.stdout:7/971: dread d14/d1d/d62/dcf/f133 [0,4194304] 0 2024-02-19T19:06:36.154 INFO:tasks.workunit.client.0.smithi088.stdout:0/881: dread d13/d62/d8a/de0/f114 [0,4194304] 0 2024-02-19T19:06:36.155 INFO:tasks.workunit.client.0.smithi088.stdout:0/882: symlink d13/d81/l131 0 2024-02-19T19:06:36.156 INFO:tasks.workunit.client.0.smithi088.stdout:7/972: truncate d14/d27/d49/f7d 1872109 0 2024-02-19T19:06:36.157 INFO:tasks.workunit.client.0.smithi088.stdout:7/973: chown d14/d1d/d3c/d43/l147 75 1 2024-02-19T19:06:36.160 INFO:tasks.workunit.client.0.smithi088.stdout:0/883: truncate d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/d10a/d106/fce 424328 0 2024-02-19T19:06:36.162 INFO:tasks.workunit.client.0.smithi088.stdout:0/884: dread - d13/d62/d8a/d9c/da8/dab/db4/dbf/ff5 zero size 2024-02-19T19:06:36.163 INFO:tasks.workunit.client.0.smithi088.stdout:7/974: dwrite d14/d28/d3b/dd3/f141 [0,4194304] 0 2024-02-19T19:06:36.164 INFO:tasks.workunit.client.0.smithi088.stdout:7/975: fsync d14/d28/d3b/f40 0 2024-02-19T19:06:36.167 INFO:tasks.workunit.client.0.smithi088.stdout:7/976: read d14/d1d/d3c/d43/d5c/d75/fad [1994951,29411] 0 2024-02-19T19:06:36.172 INFO:tasks.workunit.client.0.smithi088.stdout:0/885: getdents d13/d62/d8a/d95/d5a 0 2024-02-19T19:06:36.177 INFO:tasks.workunit.client.0.smithi088.stdout:7/977: dwrite d14/d27/f32 [0,4194304] 0 2024-02-19T19:06:36.183 INFO:tasks.workunit.client.0.smithi088.stdout:7/978: write d14/d27/d6b/f7b [1408596,101860] 0 2024-02-19T19:06:36.188 INFO:tasks.workunit.client.0.smithi088.stdout:7/979: mkdir d14/d1d/d3c/d43/d5c/d75/d157 0 2024-02-19T19:06:36.188 INFO:tasks.workunit.client.0.smithi088.stdout:0/886: dwrite d13/d62/d8a/fd2 [0,4194304] 0 2024-02-19T19:06:36.189 INFO:tasks.workunit.client.0.smithi088.stdout:7/980: fsync d14/f6e 0 2024-02-19T19:06:36.196 INFO:tasks.workunit.client.0.smithi088.stdout:0/887: creat d13/d62/d8a/d95/d96/ded/f132 x:0 0 0 2024-02-19T19:06:36.197 INFO:tasks.workunit.client.0.smithi088.stdout:0/888: read - d13/d62/d8a/d95/d96/ded/ff7 zero size 2024-02-19T19:06:36.199 INFO:tasks.workunit.client.0.smithi088.stdout:7/981: dwrite d14/d1d/d3c/f80 [0,4194304] 0 2024-02-19T19:06:36.202 INFO:tasks.workunit.client.0.smithi088.stdout:7/982: mkdir d14/d1d/d62/d158 0 2024-02-19T19:06:36.205 INFO:tasks.workunit.client.0.smithi088.stdout:7/983: truncate d14/d28/d3b/f4a 1546008 0 2024-02-19T19:06:36.206 INFO:tasks.workunit.client.0.smithi088.stdout:7/984: write d14/d28/d3b/f119 [499309,32629] 0 2024-02-19T19:06:36.211 INFO:tasks.workunit.client.0.smithi088.stdout:7/985: link d14/d1d/d3c/d43/d5c/d9a/faf d14/d27/d6b/dba/d102/f159 0 2024-02-19T19:06:36.214 INFO:tasks.workunit.client.0.smithi088.stdout:7/986: rename d14/d1d/d3c/d43/d5c/cde to d14/d1d/d3c/d43/d5c/d9a/df4/c15a 0 2024-02-19T19:06:36.226 INFO:tasks.workunit.client.0.smithi088.stdout:7/987: dwrite d14/d27/d49/d98/d107/ff5 [4194304,4194304] 0 2024-02-19T19:06:36.228 INFO:tasks.workunit.client.0.smithi088.stdout:7/988: dread - d14/d1d/d62/dfa/f12e zero size 2024-02-19T19:06:36.232 INFO:tasks.workunit.client.0.smithi088.stdout:7/989: mkdir d14/d15b 0 2024-02-19T19:06:36.234 INFO:tasks.workunit.client.0.smithi088.stdout:0/889: dread d13/d62/d8a/d95/d5a/d25/fef [0,4194304] 0 2024-02-19T19:06:36.237 INFO:tasks.workunit.client.0.smithi088.stdout:0/890: symlink d13/d62/d8a/d95/d5a/d25/d2a/d34/l133 0 2024-02-19T19:06:36.239 INFO:tasks.workunit.client.0.smithi088.stdout:0/891: symlink d13/d62/d8a/d9c/da8/l134 0 2024-02-19T19:06:36.245 INFO:tasks.workunit.client.0.smithi088.stdout:0/892: dread d13/f8c [0,4194304] 0 2024-02-19T19:06:36.245 INFO:tasks.workunit.client.0.smithi088.stdout:7/990: dwrite d14/f44 [0,4194304] 0 2024-02-19T19:06:36.246 INFO:tasks.workunit.client.0.smithi088.stdout:0/893: readlink d13/d62/d8a/d95/d5a/d25/l66 0 2024-02-19T19:06:36.251 INFO:tasks.workunit.client.0.smithi088.stdout:7/991: symlink d14/d27/d6b/dba/l15c 0 2024-02-19T19:06:36.254 INFO:tasks.workunit.client.0.smithi088.stdout:0/894: dwrite d13/d62/d8a/d95/d5a/df3/f10d [0,4194304] 0 2024-02-19T19:06:36.259 INFO:tasks.workunit.client.0.smithi088.stdout:0/895: rename d13/d62/d8a/fbb to d13/d81/f135 0 2024-02-19T19:06:36.262 INFO:tasks.workunit.client.0.smithi088.stdout:7/992: dwrite d14/f60 [4194304,4194304] 0 2024-02-19T19:06:36.263 INFO:tasks.workunit.client.0.smithi088.stdout:0/896: symlink d13/d62/d8a/d95/d5b/dbe/d126/l136 0 2024-02-19T19:06:36.269 INFO:tasks.workunit.client.0.smithi088.stdout:0/897: creat d13/d62/d8a/d95/d5b/f137 x:0 0 0 2024-02-19T19:06:36.270 INFO:tasks.workunit.client.0.smithi088.stdout:7/993: link d14/d1d/d62/dfa/f12e d14/d1d/f15d 0 2024-02-19T19:06:36.271 INFO:tasks.workunit.client.0.smithi088.stdout:0/898: chown d13/d62/d8a/d9c/le7 497 1 2024-02-19T19:06:36.275 INFO:tasks.workunit.client.0.smithi088.stdout:7/994: rename d14/d28/d9c/f13e to d14/d27/d49/d98/d107/d146/f15e 0 2024-02-19T19:06:36.275 INFO:tasks.workunit.client.0.smithi088.stdout:7/995: chown d14/d28/d3b/dd3/f141 263 1 2024-02-19T19:06:36.276 INFO:tasks.workunit.client.0.smithi088.stdout:7/996: chown d14/d28/d94/fbc 0 1 2024-02-19T19:06:36.278 INFO:tasks.workunit.client.0.smithi088.stdout:7/997: write d14/d27/d49/f117 [1046522,9701] 0 2024-02-19T19:06:36.279 INFO:tasks.workunit.client.0.smithi088.stdout:0/899: rename d13/d62/d8a/d9c/caa to d13/d62/d8a/d9c/da8/dab/c138 0 2024-02-19T19:06:36.280 INFO:tasks.workunit.client.0.smithi088.stdout:0/900: stat d13/d62/d8a/d9c/lb2 0 2024-02-19T19:06:36.280 INFO:tasks.workunit.client.0.smithi088.stdout:0/901: stat d13/d62/d8a/de0/f114 0 2024-02-19T19:06:36.280 INFO:tasks.workunit.client.0.smithi088.stdout:7/998: mkdir d14/d27/d15f 0 2024-02-19T19:06:36.281 INFO:tasks.workunit.client.0.smithi088.stdout:0/902: read - d13/d62/d8a/d95/d5a/db6/fda zero size 2024-02-19T19:06:36.283 INFO:tasks.workunit.client.0.smithi088.stdout:7/999: rename d14/d28/d94/ddb/f118 to d14/d1d/d3c/d43/d5c/f160 0 2024-02-19T19:06:36.284 INFO:tasks.workunit.client.0.smithi088.stdout:0/903: creat d13/d62/d8a/d95/d5a/f139 x:0 0 0 2024-02-19T19:06:36.287 INFO:tasks.workunit.client.0.smithi088.stdout:0/904: getdents d13/d62/d8a/d9c/da8/dab/dd0 0 2024-02-19T19:06:36.289 INFO:tasks.workunit.client.0.smithi088.stdout:0/905: creat d13/d62/d8a/d9c/da8/de2/f13a x:0 0 0 2024-02-19T19:06:36.299 INFO:tasks.workunit.client.0.smithi088.stdout:0/906: rename d13/d62/d8a/d9c/da8/dab/fca to d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/d10a/f13b 0 2024-02-19T19:06:36.300 INFO:tasks.workunit.client.0.smithi088.stdout:0/907: truncate d13/d62/d8a/d95/f12a 460861 0 2024-02-19T19:06:36.301 INFO:tasks.workunit.client.0.smithi088.stdout:0/908: creat d13/d81/f13c x:0 0 0 2024-02-19T19:06:36.302 INFO:tasks.workunit.client.0.smithi088.stdout:0/909: mkdir d13/d62/d8a/d95/d5a/d25/dd6/d13d 0 2024-02-19T19:06:36.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:36 smithi088 ceph-mon[88643]: Upgrade: Updating crash.smithi088 (1/2) 2024-02-19T19:06:36.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:36 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:36.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:36 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T19:06:36.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:36 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:36.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:36 smithi088 ceph-mon[88643]: Deploying daemon crash.smithi088 on smithi088 2024-02-19T19:06:36.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:36 smithi088 ceph-mon[88643]: pgmap v52: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 83 MiB/s rd, 197 MiB/s wr, 425 op/s 2024-02-19T19:06:36.703 INFO:tasks.workunit.client.0.smithi088.stdout:1/886: sync 2024-02-19T19:06:36.704 INFO:tasks.workunit.client.0.smithi088.stdout:8/859: sync 2024-02-19T19:06:36.704 INFO:tasks.workunit.client.0.smithi088.stdout:2/825: sync 2024-02-19T19:06:36.704 INFO:tasks.workunit.client.0.smithi088.stdout:3/949: sync 2024-02-19T19:06:36.704 INFO:tasks.workunit.client.0.smithi088.stdout:4/746: sync 2024-02-19T19:06:36.704 INFO:tasks.workunit.client.0.smithi088.stdout:0/910: sync 2024-02-19T19:06:36.708 INFO:tasks.workunit.client.0.smithi088.stdout:2/826: stat d3/d1f/d5d/c69 0 2024-02-19T19:06:36.708 INFO:tasks.workunit.client.0.smithi088.stdout:4/747: dread - d2/d32/d25/d26/da5/db2/fc7 zero size 2024-02-19T19:06:36.708 INFO:tasks.workunit.client.0.smithi088.stdout:3/950: mknod d4/c147 0 2024-02-19T19:06:36.708 INFO:tasks.workunit.client.0.smithi088.stdout:2/827: write d3/d1b/d43/df4/ffa [971501,82126] 0 2024-02-19T19:06:36.708 INFO:tasks.workunit.client.0.smithi088.stdout:0/911: unlink d13/d62/d8a/d95/d5a/d25/c84 0 2024-02-19T19:06:36.710 INFO:tasks.workunit.client.0.smithi088.stdout:3/951: symlink d4/d117/l148 0 2024-02-19T19:06:36.710 INFO:tasks.workunit.client.0.smithi088.stdout:4/748: chown d2/d32/d25/d26/d6d/d7e/d3a 394101640 1 2024-02-19T19:06:36.711 INFO:tasks.workunit.client.0.smithi088.stdout:3/952: stat d4/d12/l76 0 2024-02-19T19:06:36.713 INFO:tasks.workunit.client.0.smithi088.stdout:0/912: read d13/de9/d118/f97 [536647,89762] 0 2024-02-19T19:06:36.714 INFO:tasks.workunit.client.0.smithi088.stdout:2/828: link d3/d14/f4d d3/d1f/d5d/f111 0 2024-02-19T19:06:36.715 INFO:tasks.workunit.client.0.smithi088.stdout:1/887: dwrite d6/d21/d64/d94/dab/f107 [0,4194304] 0 2024-02-19T19:06:36.717 INFO:tasks.workunit.client.0.smithi088.stdout:4/749: fdatasync d2/d32/d25/d26/d6d/d7e/d90/fa7 0 2024-02-19T19:06:36.719 INFO:tasks.workunit.client.0.smithi088.stdout:2/829: unlink d3/d1b/d43/f55 0 2024-02-19T19:06:36.720 INFO:tasks.workunit.client.0.smithi088.stdout:1/888: creat d6/d10/f11d x:0 0 0 2024-02-19T19:06:36.723 INFO:tasks.workunit.client.0.smithi088.stdout:2/830: write d3/d1f/fa2 [763998,85723] 0 2024-02-19T19:06:36.724 INFO:tasks.workunit.client.0.smithi088.stdout:4/750: truncate d2/d32/d25/f38 2207413 0 2024-02-19T19:06:36.725 INFO:tasks.workunit.client.0.smithi088.stdout:0/913: dwrite d13/d62/d8a/d95/d5b/f137 [0,4194304] 0 2024-02-19T19:06:36.727 INFO:tasks.workunit.client.0.smithi088.stdout:4/751: write d2/d32/d25/f41 [6391068,11632] 0 2024-02-19T19:06:36.729 INFO:tasks.workunit.client.0.smithi088.stdout:4/752: stat d2/d32/d25/d26/d4e/ce9 0 2024-02-19T19:06:36.731 INFO:tasks.workunit.client.0.smithi088.stdout:1/889: unlink d6/d10/d60/d29/l77 0 2024-02-19T19:06:36.732 INFO:tasks.workunit.client.0.smithi088.stdout:2/831: rename d3/d1b/d43/df4/faa to d3/d1b/d43/d8f/d94/f112 0 2024-02-19T19:06:36.733 INFO:tasks.workunit.client.0.smithi088.stdout:0/914: mkdir d13/d62/d8a/d9c/d13e 0 2024-02-19T19:06:36.734 INFO:tasks.workunit.client.0.smithi088.stdout:4/753: stat d2/d32/d25/d4b/c64 0 2024-02-19T19:06:36.735 INFO:tasks.workunit.client.0.smithi088.stdout:1/890: creat d6/d21/d64/d94/d71/dbe/f11e x:0 0 0 2024-02-19T19:06:36.736 INFO:tasks.workunit.client.0.smithi088.stdout:2/832: mkdir d3/d1f/d113 0 2024-02-19T19:06:36.736 INFO:tasks.workunit.client.0.smithi088.stdout:4/754: chown d2/d32/d25/d26/d4e/d6f/f9d 4417008 1 2024-02-19T19:06:36.738 INFO:tasks.workunit.client.0.smithi088.stdout:1/891: creat d6/d21/d64/dc8/dcc/f11f x:0 0 0 2024-02-19T19:06:36.744 INFO:tasks.workunit.client.0.smithi088.stdout:8/860: truncate d9/df/d3d/d50/d71/fd4 4646985 0 2024-02-19T19:06:36.752 INFO:tasks.workunit.client.0.smithi088.stdout:2/833: dwrite d3/d1f/d4f/d59/f5f [0,4194304] 0 2024-02-19T19:06:36.753 INFO:tasks.workunit.client.0.smithi088.stdout:1/892: dread d6/d21/f36 [0,4194304] 0 2024-02-19T19:06:36.753 INFO:tasks.workunit.client.0.smithi088.stdout:2/834: chown d3/d1b/d43/df4/f8b 7 1 2024-02-19T19:06:36.756 INFO:tasks.workunit.client.0.smithi088.stdout:8/861: dwrite d9/df/d3d/d50/d71/da1/d6c/d9a/f5e [0,4194304] 0 2024-02-19T19:06:36.758 INFO:tasks.workunit.client.0.smithi088.stdout:2/835: creat d3/d1b/d43/d61/d7e/db6/f114 x:0 0 0 2024-02-19T19:06:36.758 INFO:tasks.workunit.client.0.smithi088.stdout:1/893: creat d6/d10/d60/db8/dd9/de4/f120 x:0 0 0 2024-02-19T19:06:36.759 INFO:tasks.workunit.client.0.smithi088.stdout:8/862: write d9/df/d3d/d50/d71/da1/d70/fad [1001856,51201] 0 2024-02-19T19:06:36.760 INFO:tasks.workunit.client.0.smithi088.stdout:0/915: dread d13/f3e [0,4194304] 0 2024-02-19T19:06:36.762 INFO:tasks.workunit.client.0.smithi088.stdout:1/894: creat d6/d21/d6a/d4a/f121 x:0 0 0 2024-02-19T19:06:36.766 INFO:tasks.workunit.client.0.smithi088.stdout:8/863: symlink d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/l10c 0 2024-02-19T19:06:36.767 INFO:tasks.workunit.client.0.smithi088.stdout:1/895: dread d6/fa1 [0,4194304] 0 2024-02-19T19:06:36.768 INFO:tasks.workunit.client.0.smithi088.stdout:0/916: rename d13/d62/d8a/d95/d5a/d25/d2a/d3c/l130 to d13/d62/d8a/d95/d5a/df3/l13f 0 2024-02-19T19:06:36.770 INFO:tasks.workunit.client.0.smithi088.stdout:0/917: chown d13/d62/d8a/d95/d5a/d25/dd6/f11c 100283770 1 2024-02-19T19:06:36.770 INFO:tasks.workunit.client.0.smithi088.stdout:8/864: creat d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/f10d x:0 0 0 2024-02-19T19:06:36.770 INFO:tasks.workunit.client.0.smithi088.stdout:2/836: dread d3/d1b/d43/d8f/ff1 [0,4194304] 0 2024-02-19T19:06:36.772 INFO:tasks.workunit.client.0.smithi088.stdout:0/918: dread - d13/d62/d8a/d9c/da8/de2/f13a zero size 2024-02-19T19:06:36.776 INFO:tasks.workunit.client.0.smithi088.stdout:0/919: creat d13/d62/d8a/d9c/da8/de2/f140 x:0 0 0 2024-02-19T19:06:36.780 INFO:tasks.workunit.client.0.smithi088.stdout:2/837: write d3/d1f/d4f/dc6/fe0 [2889818,88035] 0 2024-02-19T19:06:36.784 INFO:tasks.workunit.client.0.smithi088.stdout:2/838: creat d3/d1b/d43/d61/da6/f115 x:0 0 0 2024-02-19T19:06:36.786 INFO:tasks.workunit.client.0.smithi088.stdout:2/839: chown d3/d1b/d43/d74/c9a 5667 1 2024-02-19T19:06:36.790 INFO:tasks.workunit.client.0.smithi088.stdout:0/920: dwrite d13/f2c [0,4194304] 0 2024-02-19T19:06:36.790 INFO:tasks.workunit.client.0.smithi088.stdout:8/865: truncate d9/df/d3d/d50/d71/da1/d6c/d77/d97/fc2 318541 0 2024-02-19T19:06:36.791 INFO:tasks.workunit.client.0.smithi088.stdout:1/896: dread d6/d21/d64/d94/d71/f10c [0,4194304] 0 2024-02-19T19:06:36.797 INFO:tasks.workunit.client.0.smithi088.stdout:0/921: rename d13/d62/d8a/d95/d5a/d25/d2a/d34/d100/f127 to d13/d62/d8a/d9c/d13e/f141 0 2024-02-19T19:06:36.798 INFO:tasks.workunit.client.0.smithi088.stdout:0/922: stat d13/f2c 0 2024-02-19T19:06:36.799 INFO:tasks.workunit.client.0.smithi088.stdout:2/840: link d3/d1b/d43/d61/da6/db2/fd7 d3/d1f/d5d/f116 0 2024-02-19T19:06:36.801 INFO:tasks.workunit.client.0.smithi088.stdout:1/897: rename d6/d21/d64/d94/ffe to d6/d10/d60/db8/dcf/f122 0 2024-02-19T19:06:36.801 INFO:tasks.workunit.client.0.smithi088.stdout:0/923: symlink d13/d62/d8a/dfe/l142 0 2024-02-19T19:06:36.802 INFO:tasks.workunit.client.0.smithi088.stdout:2/841: truncate d3/d1b/d43/d61/fc8 747601 0 2024-02-19T19:06:36.803 INFO:tasks.workunit.client.0.smithi088.stdout:0/924: chown d13/c51 3300 1 2024-02-19T19:06:36.804 INFO:tasks.workunit.client.0.smithi088.stdout:2/842: write d3/d1f/d5d/de1/fe5 [2973811,13404] 0 2024-02-19T19:06:36.805 INFO:tasks.workunit.client.0.smithi088.stdout:2/843: chown d3/d1b/d43/d61/da6 183327 1 2024-02-19T19:06:36.806 INFO:tasks.workunit.client.0.smithi088.stdout:0/925: read d13/d62/d8a/d9c/da8/fea [154439,28969] 0 2024-02-19T19:06:36.812 INFO:tasks.workunit.client.0.smithi088.stdout:1/898: dread d6/d21/d64/d94/f66 [0,4194304] 0 2024-02-19T19:06:36.814 INFO:tasks.workunit.client.0.smithi088.stdout:0/926: write d13/d62/d8a/d95/d5a/f80 [4646142,43665] 0 2024-02-19T19:06:36.819 INFO:tasks.workunit.client.0.smithi088.stdout:0/927: mkdir d13/d62/d8a/d9c/da8/d143 0 2024-02-19T19:06:36.820 INFO:tasks.workunit.client.0.smithi088.stdout:8/866: dread d9/df/d3d/d50/d71/da1/d6c/d9a/f44 [0,4194304] 0 2024-02-19T19:06:36.821 INFO:tasks.workunit.client.0.smithi088.stdout:2/844: dwrite d3/d14/f1d [4194304,4194304] 0 2024-02-19T19:06:36.822 INFO:tasks.workunit.client.0.smithi088.stdout:8/867: chown d9/df/d1c/d85/dc5/f10b 9732 1 2024-02-19T19:06:36.823 INFO:tasks.workunit.client.0.smithi088.stdout:0/928: symlink d13/d81/d128/l144 0 2024-02-19T19:06:36.825 INFO:tasks.workunit.client.0.smithi088.stdout:0/929: mkdir d13/d62/d8a/d9c/d145 0 2024-02-19T19:06:36.826 INFO:tasks.workunit.client.0.smithi088.stdout:8/868: read d9/df/d3d/d50/d71/da1/d6c/d9a/f4d [490653,47438] 0 2024-02-19T19:06:36.829 INFO:tasks.workunit.client.0.smithi088.stdout:8/869: read d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/fd2 [1641215,80276] 0 2024-02-19T19:06:36.830 INFO:tasks.workunit.client.0.smithi088.stdout:0/930: unlink d13/d62/d8a/d95/d5a/d25/d2a/d3c/f64 0 2024-02-19T19:06:36.833 INFO:tasks.workunit.client.0.smithi088.stdout:0/931: readlink d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/l12c 0 2024-02-19T19:06:36.833 INFO:tasks.workunit.client.0.smithi088.stdout:1/899: dread d6/d21/d64/d94/d71/f9e [0,4194304] 0 2024-02-19T19:06:36.834 INFO:tasks.workunit.client.0.smithi088.stdout:1/900: dread - d6/d21/d64/dc8/dcc/f11f zero size 2024-02-19T19:06:36.839 INFO:tasks.workunit.client.0.smithi088.stdout:8/870: dread d9/df/d3d/d50/feb [0,4194304] 0 2024-02-19T19:06:36.841 INFO:tasks.workunit.client.0.smithi088.stdout:0/932: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d3c/fc5 [4194304,4194304] 0 2024-02-19T19:06:36.841 INFO:tasks.workunit.client.0.smithi088.stdout:1/901: truncate d6/d21/d6a/d4a/fa7 565077 0 2024-02-19T19:06:36.843 INFO:tasks.workunit.client.0.smithi088.stdout:1/902: readlink d6/d10/l44 0 2024-02-19T19:06:36.844 INFO:tasks.workunit.client.0.smithi088.stdout:0/933: fsync d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/d10a/d106/de5/f121 0 2024-02-19T19:06:36.844 INFO:tasks.workunit.client.0.smithi088.stdout:1/903: chown d6/d21/d6a/d4a/f59 98698 1 2024-02-19T19:06:36.847 INFO:tasks.workunit.client.0.smithi088.stdout:1/904: dread - d6/d21/d6a/d7d/fd2 zero size 2024-02-19T19:06:36.847 INFO:tasks.workunit.client.0.smithi088.stdout:1/905: chown d6/d21/d64/d94/d71/dbe 48961848 1 2024-02-19T19:06:36.849 INFO:tasks.workunit.client.0.smithi088.stdout:1/906: creat d6/d21/d64/d94/d71/f123 x:0 0 0 2024-02-19T19:06:36.851 INFO:tasks.workunit.client.0.smithi088.stdout:8/871: write d9/df/d3d/dfc/f100 [5190849,126103] 0 2024-02-19T19:06:36.854 INFO:tasks.workunit.client.0.smithi088.stdout:8/872: unlink d9/df/d1c/d22/ffe 0 2024-02-19T19:06:36.867 INFO:tasks.workunit.client.0.smithi088.stdout:8/873: dwrite d9/df/d1c/f13 [0,4194304] 0 2024-02-19T19:06:36.868 INFO:tasks.workunit.client.0.smithi088.stdout:8/874: readlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f/l101 0 2024-02-19T19:06:36.869 INFO:tasks.workunit.client.0.smithi088.stdout:8/875: chown d9/df/d3d 11757 1 2024-02-19T19:06:36.872 INFO:tasks.workunit.client.0.smithi088.stdout:1/907: dread d6/d21/d6a/d4a/fb1 [0,4194304] 0 2024-02-19T19:06:36.872 INFO:tasks.workunit.client.0.smithi088.stdout:8/876: unlink d9/df/d3d/f5f 0 2024-02-19T19:06:36.883 INFO:tasks.workunit.client.0.smithi088.stdout:1/908: write d6/d10/d60/db8/dcf/fbb [3869266,117416] 0 2024-02-19T19:06:36.887 INFO:tasks.workunit.client.0.smithi088.stdout:8/877: dwrite d9/df/f27 [0,4194304] 0 2024-02-19T19:06:36.887 INFO:tasks.workunit.client.0.smithi088.stdout:1/909: rename d6/d21/d6a/d4a/f121 to d6/d21/d64/d94/dab/dfd/f124 0 2024-02-19T19:06:36.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:36 smithi132 ceph-mon[70676]: Upgrade: Updating crash.smithi088 (1/2) 2024-02-19T19:06:36.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:36 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:36.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:36 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi088", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T19:06:36.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:36 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:36.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:36 smithi132 ceph-mon[70676]: Deploying daemon crash.smithi088 on smithi088 2024-02-19T19:06:36.891 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:36 smithi132 ceph-mon[70676]: pgmap v52: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 83 MiB/s rd, 197 MiB/s wr, 425 op/s 2024-02-19T19:06:36.893 INFO:tasks.workunit.client.0.smithi088.stdout:1/910: link d6/d21/d6a/d7d/d7e/lc2 d6/d21/d6a/l125 0 2024-02-19T19:06:36.903 INFO:tasks.workunit.client.0.smithi088.stdout:1/911: write d6/d21/f36 [2509223,114956] 0 2024-02-19T19:06:36.905 INFO:tasks.workunit.client.0.smithi088.stdout:1/912: creat d6/d21/d64/d108/f126 x:0 0 0 2024-02-19T19:06:36.907 INFO:tasks.workunit.client.0.smithi088.stdout:1/913: symlink d6/d10/d60/db8/l127 0 2024-02-19T19:06:36.908 INFO:tasks.workunit.client.0.smithi088.stdout:1/914: dread - d6/d21/d6a/d7d/d9a/f114 zero size 2024-02-19T19:06:36.910 INFO:tasks.workunit.client.0.smithi088.stdout:8/878: dread d9/df/d3d/d50/d71/da1/d6c/fa0 [0,4194304] 0 2024-02-19T19:06:36.910 INFO:tasks.workunit.client.0.smithi088.stdout:1/915: read - d6/d10/d60/db8/dcf/fd8 zero size 2024-02-19T19:06:36.912 INFO:tasks.workunit.client.0.smithi088.stdout:8/879: dread d9/df/d3d/d50/d71/f8b [0,4194304] 0 2024-02-19T19:06:36.914 INFO:tasks.workunit.client.0.smithi088.stdout:1/916: write d6/d10/f4b [427535,35901] 0 2024-02-19T19:06:36.916 INFO:tasks.workunit.client.0.smithi088.stdout:1/917: creat d6/d21/d6a/d7d/f128 x:0 0 0 2024-02-19T19:06:36.918 INFO:tasks.workunit.client.0.smithi088.stdout:8/880: rename d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d7f to d9/df/d1c/d10e 0 2024-02-19T19:06:36.919 INFO:tasks.workunit.client.0.smithi088.stdout:1/918: mknod d6/d10/d60/d29/c129 0 2024-02-19T19:06:36.919 INFO:tasks.workunit.client.0.smithi088.stdout:8/881: rmdir d9/df/d1c/d10e/d8e 39 2024-02-19T19:06:36.922 INFO:tasks.workunit.client.0.smithi088.stdout:1/919: getdents d6/d10/d60 0 2024-02-19T19:06:36.923 INFO:tasks.workunit.client.0.smithi088.stdout:8/882: rename d9/df/d1c/d10e/cab to d9/c10f 0 2024-02-19T19:06:36.924 INFO:tasks.workunit.client.0.smithi088.stdout:8/883: chown d9/df/d3d/d62/f69 1837917579 1 2024-02-19T19:06:36.924 INFO:tasks.workunit.client.0.smithi088.stdout:1/920: mkdir d6/d21/d6a/dcb/d12a 0 2024-02-19T19:06:36.925 INFO:tasks.workunit.client.0.smithi088.stdout:8/884: chown d9/df/f1a 11 1 2024-02-19T19:06:36.926 INFO:tasks.workunit.client.0.smithi088.stdout:8/885: rmdir d9/dc4 39 2024-02-19T19:06:36.926 INFO:tasks.workunit.client.0.smithi088.stdout:1/921: read d6/d21/d64/dc8/dcc/d101/feb [3545915,100911] 0 2024-02-19T19:06:36.928 INFO:tasks.workunit.client.0.smithi088.stdout:1/922: mkdir d6/d10/d60/db8/d12b 0 2024-02-19T19:06:36.928 INFO:tasks.workunit.client.0.smithi088.stdout:1/923: fdatasync d6/f17 0 2024-02-19T19:06:36.929 INFO:tasks.workunit.client.0.smithi088.stdout:1/924: stat d6/d10/d60/db8/dcf/fd6 0 2024-02-19T19:06:36.935 INFO:tasks.workunit.client.0.smithi088.stdout:8/886: truncate d9/df/d3d/d50/d71/da1/d6c/d9a/f44 1831999 0 2024-02-19T19:06:36.936 INFO:tasks.workunit.client.0.smithi088.stdout:1/925: write d6/d21/d6a/d7d/d9a/fa3 [1184077,92920] 0 2024-02-19T19:06:36.938 INFO:tasks.workunit.client.0.smithi088.stdout:8/887: creat d9/df/d1c/d10e/d8e/f110 x:0 0 0 2024-02-19T19:06:36.939 INFO:tasks.workunit.client.0.smithi088.stdout:1/926: rename d6/d21/d6a/d7d/cd7 to d6/d21/d6a/dcb/c12c 0 2024-02-19T19:06:37.037 INFO:tasks.workunit.client.0.smithi088.stdout:8/888: write d9/df/d3d/d50/d71/da1/d6c/d9a/f4d [1258111,74911] 0 2024-02-19T19:06:37.037 INFO:tasks.workunit.client.0.smithi088.stdout:1/927: write d6/fb [3813829,5351] 0 2024-02-19T19:06:37.039 INFO:tasks.workunit.client.0.smithi088.stdout:8/889: write d9/df/f27 [76470,58328] 0 2024-02-19T19:06:37.040 INFO:tasks.workunit.client.0.smithi088.stdout:8/890: chown d9/df/d3d/d62/f69 0 1 2024-02-19T19:06:37.041 INFO:tasks.workunit.client.0.smithi088.stdout:1/928: mknod d6/d21/d6a/c12d 0 2024-02-19T19:06:37.042 INFO:tasks.workunit.client.0.smithi088.stdout:1/929: readlink d6/d10/l5b 0 2024-02-19T19:06:37.043 INFO:tasks.workunit.client.0.smithi088.stdout:8/891: unlink d9/df/d1c/d10e/ld0 0 2024-02-19T19:06:37.043 INFO:tasks.workunit.client.0.smithi088.stdout:8/892: chown d9/df/d1c/d85 36 1 2024-02-19T19:06:37.044 INFO:tasks.workunit.client.0.smithi088.stdout:1/930: mkdir d6/d21/d64/d94/d71/d12e 0 2024-02-19T19:06:37.045 INFO:tasks.workunit.client.0.smithi088.stdout:1/931: stat d6/d21/d6a/d4a/c86 0 2024-02-19T19:06:37.046 INFO:tasks.workunit.client.0.smithi088.stdout:8/893: mkdir d9/df/d3d/dfc/d111 0 2024-02-19T19:06:37.052 INFO:tasks.workunit.client.0.smithi088.stdout:1/932: truncate d6/d21/d6a/d7d/d9a/fbf 999753 0 2024-02-19T19:06:37.055 INFO:tasks.workunit.client.0.smithi088.stdout:1/933: symlink d6/d21/d64/dc8/dcc/l12f 0 2024-02-19T19:06:37.056 INFO:tasks.workunit.client.0.smithi088.stdout:8/894: dwrite d9/df/d1c/f25 [4194304,4194304] 0 2024-02-19T19:06:37.061 INFO:tasks.workunit.client.0.smithi088.stdout:1/934: write d6/d10/d60/db8/fd1 [417509,32346] 0 2024-02-19T19:06:37.066 INFO:tasks.workunit.client.0.smithi088.stdout:1/935: write d6/d21/d6a/d4a/fb1 [1817352,29810] 0 2024-02-19T19:06:37.072 INFO:tasks.workunit.client.0.smithi088.stdout:1/936: read d6/d10/d60/d29/f2e [3119021,102999] 0 2024-02-19T19:06:37.073 INFO:tasks.workunit.client.0.smithi088.stdout:1/937: write d6/d21/d6a/d4a/f59 [3507779,23164] 0 2024-02-19T19:06:37.075 INFO:tasks.workunit.client.0.smithi088.stdout:8/895: dread d9/df/f36 [4194304,4194304] 0 2024-02-19T19:06:37.078 INFO:tasks.workunit.client.0.smithi088.stdout:8/896: mknod d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/d104/c112 0 2024-02-19T19:06:37.080 INFO:tasks.workunit.client.0.smithi088.stdout:1/938: dwrite d6/d10/d60/db8/dd9/de4/f120 [0,4194304] 0 2024-02-19T19:06:37.083 INFO:tasks.workunit.client.0.smithi088.stdout:8/897: creat d9/df/d3d/d50/d71/da1/d70/f113 x:0 0 0 2024-02-19T19:06:37.085 INFO:tasks.workunit.client.0.smithi088.stdout:8/898: creat d9/df/d1c/d85/f114 x:0 0 0 2024-02-19T19:06:37.087 INFO:tasks.workunit.client.0.smithi088.stdout:8/899: mknod d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/c115 0 2024-02-19T19:06:37.091 INFO:tasks.workunit.client.0.smithi088.stdout:8/900: link d9/df/d3d/d50/d71/da1/d70/l91 d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/l116 0 2024-02-19T19:06:37.092 INFO:tasks.workunit.client.0.smithi088.stdout:1/939: dread d6/d21/d64/fbd [0,4194304] 0 2024-02-19T19:06:37.100 INFO:tasks.workunit.client.0.smithi088.stdout:8/901: write d9/df/d3d/d50/d71/da1/f6d [558157,101920] 0 2024-02-19T19:06:37.102 INFO:tasks.workunit.client.0.smithi088.stdout:8/902: readlink d9/df/d3d/d50/d71/da1/d70/l91 0 2024-02-19T19:06:37.104 INFO:tasks.workunit.client.0.smithi088.stdout:1/940: dwrite d6/d10/d60/d29/fb3 [0,4194304] 0 2024-02-19T19:06:37.105 INFO:tasks.workunit.client.0.smithi088.stdout:1/941: dread - d6/d10/f11d zero size 2024-02-19T19:06:37.107 INFO:tasks.workunit.client.0.smithi088.stdout:1/942: dread - d6/d21/d6a/d7d/d9a/f109 zero size 2024-02-19T19:06:37.107 INFO:tasks.workunit.client.0.smithi088.stdout:8/903: symlink d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/dea/dec/l117 0 2024-02-19T19:06:37.112 INFO:tasks.workunit.client.0.smithi088.stdout:8/904: symlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/d104/l118 0 2024-02-19T19:06:37.115 INFO:tasks.workunit.client.0.smithi088.stdout:8/905: rename d9/df/d3d/d50/d71/da1/d82 to d9/df/d3d/d50/d71/da1/d70/d119 0 2024-02-19T19:06:37.120 INFO:tasks.workunit.client.0.smithi088.stdout:1/943: truncate d6/fb 1216109 0 2024-02-19T19:06:37.129 INFO:tasks.workunit.client.0.smithi088.stdout:8/906: dwrite d9/df/d3d/d50/d71/da1/d6c/d77/d97/fc7 [0,4194304] 0 2024-02-19T19:06:37.131 INFO:tasks.workunit.client.0.smithi088.stdout:8/907: read - d9/df/d1c/f10a zero size 2024-02-19T19:06:37.131 INFO:tasks.workunit.client.0.smithi088.stdout:1/944: dwrite d6/d10/fc9 [0,4194304] 0 2024-02-19T19:06:37.133 INFO:tasks.workunit.client.0.smithi088.stdout:1/945: dread - d6/d21/d64/d94/d71/f123 zero size 2024-02-19T19:06:37.134 INFO:tasks.workunit.client.0.smithi088.stdout:8/908: chown d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae 344656 1 2024-02-19T19:06:37.139 INFO:tasks.workunit.client.0.smithi088.stdout:1/946: dread d6/d10/d60/d29/f75 [0,4194304] 0 2024-02-19T19:06:37.144 INFO:tasks.workunit.client.0.smithi088.stdout:1/947: dwrite d6/d21/d64/fbd [0,4194304] 0 2024-02-19T19:06:37.148 INFO:tasks.workunit.client.0.smithi088.stdout:1/948: fdatasync d6/d10/d60/db8/dcf/f122 0 2024-02-19T19:06:37.150 INFO:tasks.workunit.client.0.smithi088.stdout:1/949: fsync d6/d10/d60/d29/fb3 0 2024-02-19T19:06:37.152 INFO:tasks.workunit.client.0.smithi088.stdout:8/909: write d9/df/d3d/d50/d71/f8b [109357,20195] 0 2024-02-19T19:06:37.154 INFO:tasks.workunit.client.0.smithi088.stdout:8/910: mknod d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/dea/dec/c11a 0 2024-02-19T19:06:37.170 INFO:tasks.workunit.client.0.smithi088.stdout:1/950: dread d6/d10/f24 [0,4194304] 0 2024-02-19T19:06:37.170 INFO:tasks.workunit.client.0.smithi088.stdout:1/951: readlink d6/d21/d6a/d4a/dad/l115 0 2024-02-19T19:06:37.171 INFO:tasks.workunit.client.0.smithi088.stdout:1/952: chown d6/d21/d6a/d7d/d7e/cf0 8 1 2024-02-19T19:06:37.171 INFO:tasks.workunit.client.0.smithi088.stdout:1/953: stat d6/d10/d60/d29 0 2024-02-19T19:06:37.172 INFO:tasks.workunit.client.0.smithi088.stdout:1/954: chown d6/d10/l74 66117103 1 2024-02-19T19:06:37.173 INFO:tasks.workunit.client.0.smithi088.stdout:1/955: read d6/d21/d6a/fb7 [143696,48701] 0 2024-02-19T19:06:37.174 INFO:tasks.workunit.client.0.smithi088.stdout:1/956: stat d6/d21/d64/dc8/dcc/d101/fee 0 2024-02-19T19:06:37.176 INFO:tasks.workunit.client.0.smithi088.stdout:1/957: creat d6/d21/d64/f130 x:0 0 0 2024-02-19T19:06:37.178 INFO:tasks.workunit.client.0.smithi088.stdout:1/958: getdents d6/d10/d60/db8/dd9/de4 0 2024-02-19T19:06:37.180 INFO:tasks.workunit.client.0.smithi088.stdout:1/959: creat d6/d21/d6a/dcb/f131 x:0 0 0 2024-02-19T19:06:37.182 INFO:tasks.workunit.client.0.smithi088.stdout:1/960: mknod d6/d21/d6a/d7d/d7e/c132 0 2024-02-19T19:06:37.184 INFO:tasks.workunit.client.0.smithi088.stdout:1/961: truncate d6/d21/d6a/d7d/d7e/ff3 639012 0 2024-02-19T19:06:37.185 INFO:tasks.workunit.client.0.smithi088.stdout:1/962: fdatasync d6/d10/fc4 0 2024-02-19T19:06:37.186 INFO:tasks.workunit.client.0.smithi088.stdout:1/963: chown d6/fb 1 1 2024-02-19T19:06:37.188 INFO:tasks.workunit.client.0.smithi088.stdout:1/964: creat d6/d21/d64/d108/f133 x:0 0 0 2024-02-19T19:06:37.190 INFO:tasks.workunit.client.0.smithi088.stdout:1/965: unlink d6/d21/d64/d94/dab/fed 0 2024-02-19T19:06:37.191 INFO:tasks.workunit.client.0.smithi088.stdout:1/966: mkdir d6/d10/d60/d29/d134 0 2024-02-19T19:06:37.193 INFO:tasks.workunit.client.0.smithi088.stdout:1/967: unlink d6/d21/d6a/d7d/ce6 0 2024-02-19T19:06:37.194 INFO:tasks.workunit.client.0.smithi088.stdout:1/968: mknod d6/df6/c135 0 2024-02-19T19:06:37.195 INFO:tasks.workunit.client.0.smithi088.stdout:1/969: dread - d6/d21/d64/d94/dab/dfd/f124 zero size 2024-02-19T19:06:37.196 INFO:tasks.workunit.client.0.smithi088.stdout:1/970: chown d6/d21/d64/d94/d71/dbe 485866 1 2024-02-19T19:06:37.196 INFO:tasks.workunit.client.0.smithi088.stdout:1/971: dread - d6/d21/d64/d94/d71/f123 zero size 2024-02-19T19:06:37.197 INFO:tasks.workunit.client.0.smithi088.stdout:1/972: chown d6/d10/l85 2 1 2024-02-19T19:06:37.206 INFO:tasks.workunit.client.0.smithi088.stdout:1/973: dwrite d6/fa1 [0,4194304] 0 2024-02-19T19:06:37.207 INFO:tasks.workunit.client.0.smithi088.stdout:1/974: fdatasync d6/d21/d6a/dcb/f105 0 2024-02-19T19:06:37.208 INFO:tasks.workunit.client.0.smithi088.stdout:1/975: truncate d6/d21/f111 22527 0 2024-02-19T19:06:37.211 INFO:tasks.workunit.client.0.smithi088.stdout:1/976: truncate d6/d10/fc4 4897467 0 2024-02-19T19:06:37.213 INFO:tasks.workunit.client.0.smithi088.stdout:3/953: sync 2024-02-19T19:06:37.215 INFO:tasks.workunit.client.0.smithi088.stdout:1/977: dread d6/d10/f24 [0,4194304] 0 2024-02-19T19:06:37.217 INFO:tasks.workunit.client.0.smithi088.stdout:1/978: mknod d6/d21/d64/dc8/c136 0 2024-02-19T19:06:37.218 INFO:tasks.workunit.client.0.smithi088.stdout:1/979: read d6/d10/f4b [287910,23299] 0 2024-02-19T19:06:37.219 INFO:tasks.workunit.client.0.smithi088.stdout:1/980: dread - d6/d21/d64/dc8/dcc/d101/fb5 zero size 2024-02-19T19:06:37.221 INFO:tasks.workunit.client.0.smithi088.stdout:1/981: unlink d6/d21/d6a/f68 0 2024-02-19T19:06:37.237 INFO:tasks.workunit.client.0.smithi088.stdout:1/982: fdatasync d6/d21/d64/fb0 0 2024-02-19T19:06:37.241 INFO:tasks.workunit.client.0.smithi088.stdout:1/983: creat d6/d21/d64/d94/d71/dbe/f137 x:0 0 0 2024-02-19T19:06:37.241 INFO:tasks.workunit.client.0.smithi088.stdout:3/954: dwrite d4/d1a/d1e/d31/d32/f77 [0,4194304] 0 2024-02-19T19:06:37.242 INFO:tasks.workunit.client.0.smithi088.stdout:1/984: stat d6/d21/d6a/d7d/d9a/f114 0 2024-02-19T19:06:37.361 INFO:tasks.workunit.client.0.smithi088.stdout:3/955: symlink d4/d22/d7a/d11a/d11b/l149 0 2024-02-19T19:06:37.363 INFO:tasks.workunit.client.0.smithi088.stdout:1/985: rename d6/d10/f73 to d6/d21/d64/d94/dab/dfd/f138 0 2024-02-19T19:06:37.363 INFO:tasks.workunit.client.0.smithi088.stdout:1/986: fsync d6/d10/f11 0 2024-02-19T19:06:37.366 INFO:tasks.workunit.client.0.smithi088.stdout:3/956: mknod d4/d12/d33/d43/c14a 0 2024-02-19T19:06:37.370 INFO:tasks.workunit.client.0.smithi088.stdout:1/987: dwrite d6/d21/d6a/d7d/d9a/fa3 [4194304,4194304] 0 2024-02-19T19:06:37.370 INFO:tasks.workunit.client.0.smithi088.stdout:3/957: fsync d4/d12/d33/f4e 0 2024-02-19T19:06:37.376 INFO:tasks.workunit.client.0.smithi088.stdout:1/988: write d6/d21/f84 [1081417,110915] 0 2024-02-19T19:06:37.379 INFO:tasks.workunit.client.0.smithi088.stdout:1/989: mkdir d6/d10/d60/d29/d134/d139 0 2024-02-19T19:06:37.381 INFO:tasks.workunit.client.0.smithi088.stdout:1/990: truncate d6/d21/d64/dc8/dcc/d101/fee 971553 0 2024-02-19T19:06:37.382 INFO:tasks.workunit.client.0.smithi088.stdout:1/991: mknod d6/d10/d60/c13a 0 2024-02-19T19:06:37.391 INFO:tasks.workunit.client.0.smithi088.stdout:1/992: dwrite d6/d10/d60/d29/f37 [0,4194304] 0 2024-02-19T19:06:37.410 INFO:tasks.workunit.client.0.smithi088.stdout:4/755: sync 2024-02-19T19:06:37.412 INFO:tasks.workunit.client.0.smithi088.stdout:4/756: write d2/d32/d25/d26/d6d/d7e/d3a/fd4 [195352,76506] 0 2024-02-19T19:06:37.416 INFO:tasks.workunit.client.0.smithi088.stdout:4/757: creat d2/d32/d25/d26/d4e/d79/fee x:0 0 0 2024-02-19T19:06:37.420 INFO:tasks.workunit.client.0.smithi088.stdout:4/758: rename d2/d32/d25/d3f/d70/lb6 to d2/d35/db3/db7/dd7/lef 0 2024-02-19T19:06:37.421 INFO:tasks.workunit.client.0.smithi088.stdout:4/759: symlink d2/d32/d25/d26/lf0 0 2024-02-19T19:06:37.426 INFO:tasks.workunit.client.0.smithi088.stdout:4/760: write d2/d32/fb [4840824,27482] 0 2024-02-19T19:06:37.428 INFO:tasks.workunit.client.0.smithi088.stdout:4/761: fsync d2/d32/d25/d3f/d9a/fb5 0 2024-02-19T19:06:37.429 INFO:tasks.workunit.client.0.smithi088.stdout:4/762: truncate d2/d32/d9/fd6 968935 0 2024-02-19T19:06:37.430 INFO:tasks.workunit.client.0.smithi088.stdout:4/763: creat d2/d32/d25/d26/da5/db2/ff1 x:0 0 0 2024-02-19T19:06:37.431 INFO:tasks.workunit.client.0.smithi088.stdout:4/764: chown d2/d32/d25/d3f/d9a/fb5 3185151 1 2024-02-19T19:06:37.433 INFO:tasks.workunit.client.0.smithi088.stdout:4/765: unlink d2/d32/d25/d26/d6d/d7e/d3a/lda 0 2024-02-19T19:06:37.434 INFO:tasks.workunit.client.0.smithi088.stdout:4/766: unlink d2/d32/d9/l19 0 2024-02-19T19:06:37.436 INFO:tasks.workunit.client.0.smithi088.stdout:4/767: fdatasync d2/d35/db3/fe2 0 2024-02-19T19:06:37.437 INFO:tasks.workunit.client.0.smithi088.stdout:4/768: fsync d2/d8/f14 0 2024-02-19T19:06:37.452 INFO:tasks.workunit.client.0.smithi088.stdout:4/769: dread d2/d32/f1c [0,4194304] 0 2024-02-19T19:06:37.460 INFO:tasks.workunit.client.0.smithi088.stdout:4/770: dwrite d2/d32/d25/d26/f5e [0,4194304] 0 2024-02-19T19:06:37.461 INFO:tasks.workunit.client.0.smithi088.stdout:4/771: chown d2/d35/f59 245735 1 2024-02-19T19:06:37.463 INFO:tasks.workunit.client.0.smithi088.stdout:4/772: creat d2/d35/db3/db7/ff2 x:0 0 0 2024-02-19T19:06:37.464 INFO:tasks.workunit.client.0.smithi088.stdout:4/773: write d2/d32/d25/d26/da5/db2/fc7 [550342,43948] 0 2024-02-19T19:06:37.476 INFO:tasks.workunit.client.0.smithi088.stdout:4/774: dread d2/d32/fb [0,4194304] 0 2024-02-19T19:06:37.478 INFO:tasks.workunit.client.0.smithi088.stdout:4/775: mkdir d2/d32/d25/d26/da5/db2/df3 0 2024-02-19T19:06:37.492 INFO:tasks.workunit.client.0.smithi088.stdout:4/776: dread d2/f4 [0,4194304] 0 2024-02-19T19:06:37.502 INFO:tasks.workunit.client.0.smithi088.stdout:4/777: dread d2/d35/f4f [0,4194304] 0 2024-02-19T19:06:37.503 INFO:tasks.workunit.client.0.smithi088.stdout:4/778: chown d2/d32/de7 8866 1 2024-02-19T19:06:37.503 INFO:tasks.workunit.client.0.smithi088.stdout:4/779: chown d2/d32/d25/faf 0 1 2024-02-19T19:06:37.508 INFO:tasks.workunit.client.0.smithi088.stdout:4/780: write d2/d32/d25/f8f [832019,22730] 0 2024-02-19T19:06:37.510 INFO:tasks.workunit.client.0.smithi088.stdout:4/781: symlink d2/d32/d25/d26/d6d/lf4 0 2024-02-19T19:06:37.510 INFO:tasks.workunit.client.0.smithi088.stdout:4/782: chown d2/d32/d25/d26/d6d/d7e/f72 6 1 2024-02-19T19:06:37.512 INFO:tasks.workunit.client.0.smithi088.stdout:4/783: dread d2/d32/d25/d26/f88 [0,4194304] 0 2024-02-19T19:06:37.512 INFO:tasks.workunit.client.0.smithi088.stdout:4/784: fsync d2/d32/d25/d45/fb0 0 2024-02-19T19:06:37.516 INFO:tasks.workunit.client.0.smithi088.stdout:4/785: symlink d2/d32/d25/d26/d4e/d79/lf5 0 2024-02-19T19:06:37.526 INFO:tasks.workunit.client.0.smithi088.stdout:4/786: dread d2/d32/d25/d26/d6d/d7e/f72 [0,4194304] 0 2024-02-19T19:06:37.527 INFO:tasks.workunit.client.0.smithi088.stdout:4/787: creat d2/d32/d25/d3f/d9a/ff6 x:0 0 0 2024-02-19T19:06:37.529 INFO:tasks.workunit.client.0.smithi088.stdout:4/788: creat d2/d32/d25/d26/da5/db2/ff7 x:0 0 0 2024-02-19T19:06:37.531 INFO:tasks.workunit.client.0.smithi088.stdout:4/789: creat d2/d32/d25/d3f/ff8 x:0 0 0 2024-02-19T19:06:37.532 INFO:tasks.workunit.client.0.smithi088.stdout:4/790: chown d2/d32/d25/d26/d58/c7f 1 1 2024-02-19T19:06:37.533 INFO:tasks.workunit.client.0.smithi088.stdout:4/791: mkdir d2/d35/db3/db7/de3/df9 0 2024-02-19T19:06:37.535 INFO:tasks.workunit.client.0.smithi088.stdout:4/792: mkdir d2/d32/d25/d26/dfa 0 2024-02-19T19:06:37.536 INFO:tasks.workunit.client.0.smithi088.stdout:4/793: mkdir d2/d35/db3/db7/dfb 0 2024-02-19T19:06:37.541 INFO:tasks.workunit.client.0.smithi088.stdout:4/794: dwrite d2/d8/f29 [4194304,4194304] 0 2024-02-19T19:06:37.545 INFO:tasks.workunit.client.0.smithi088.stdout:4/795: creat d2/d32/d25/d26/d4e/d79/ffc x:0 0 0 2024-02-19T19:06:37.547 INFO:tasks.workunit.client.0.smithi088.stdout:4/796: write d2/d32/d25/d3f/d70/f8b [1652001,40637] 0 2024-02-19T19:06:37.549 INFO:tasks.workunit.client.0.smithi088.stdout:4/797: rename d2/d32/d25/d26/d4e/d6f/fc2 to d2/d32/d25/d26/d4e/d6f/ffd 0 2024-02-19T19:06:37.557 INFO:tasks.workunit.client.0.smithi088.stdout:2/845: sync 2024-02-19T19:06:37.562 INFO:tasks.workunit.client.0.smithi088.stdout:4/798: truncate d2/d8/f29 6855079 0 2024-02-19T19:06:37.572 INFO:tasks.workunit.client.0.smithi088.stdout:2/846: mkdir d3/d1f/d113/d117 0 2024-02-19T19:06:37.573 INFO:tasks.workunit.client.0.smithi088.stdout:2/847: readlink d3/l45 0 2024-02-19T19:06:37.575 INFO:tasks.workunit.client.0.smithi088.stdout:4/799: rmdir d2/d35/db3/db7/de3/df9 0 2024-02-19T19:06:37.576 INFO:tasks.workunit.client.0.smithi088.stdout:4/800: mknod d2/d32/d9/de4/cfe 0 2024-02-19T19:06:37.589 INFO:tasks.workunit.client.0.smithi088.stdout:4/801: write d2/d35/f52 [821047,58058] 0 2024-02-19T19:06:37.590 INFO:tasks.workunit.client.0.smithi088.stdout:2/848: dwrite d3/fbd [0,4194304] 0 2024-02-19T19:06:37.593 INFO:tasks.workunit.client.0.smithi088.stdout:4/802: mkdir d2/d32/d25/d26/da5/db2/df3/dff 0 2024-02-19T19:06:37.594 INFO:tasks.workunit.client.0.smithi088.stdout:2/849: dread d3/d1f/fa2 [0,4194304] 0 2024-02-19T19:06:37.598 INFO:tasks.workunit.client.0.smithi088.stdout:2/850: creat d3/d1f/d113/d117/f118 x:0 0 0 2024-02-19T19:06:37.600 INFO:tasks.workunit.client.0.smithi088.stdout:2/851: symlink d3/d1b/d43/d74/l119 0 2024-02-19T19:06:37.600 INFO:tasks.workunit.client.0.smithi088.stdout:4/803: dwrite d2/d32/d25/d26/d4e/d6f/f9d [0,4194304] 0 2024-02-19T19:06:37.605 INFO:tasks.workunit.client.0.smithi088.stdout:4/804: getdents d2/d32/d9/dd8/de0 0 2024-02-19T19:06:37.608 INFO:tasks.workunit.client.0.smithi088.stdout:2/852: dwrite d3/d1b/d43/d61/d64/f10f [0,4194304] 0 2024-02-19T19:06:37.613 INFO:tasks.workunit.client.0.smithi088.stdout:4/805: dwrite d2/d32/d25/d26/d6d/d7e/d3a/d77/f92 [0,4194304] 0 2024-02-19T19:06:37.616 INFO:tasks.workunit.client.0.smithi088.stdout:4/806: write d2/d32/d25/d26/d6d/d7e/d90/f94 [2936337,104611] 0 2024-02-19T19:06:37.620 INFO:tasks.workunit.client.0.smithi088.stdout:4/807: creat d2/d32/d25/d45/f100 x:0 0 0 2024-02-19T19:06:37.622 INFO:tasks.workunit.client.0.smithi088.stdout:4/808: mknod d2/d32/d25/d26/d6d/d7e/d3a/c101 0 2024-02-19T19:06:37.628 INFO:tasks.workunit.client.0.smithi088.stdout:4/809: truncate d2/d32/d25/d26/d6d/d7e/d3a/fe5 3657202 0 2024-02-19T19:06:37.629 INFO:tasks.workunit.client.0.smithi088.stdout:4/810: dread - d2/d32/d9/dd8/de0/fd9 zero size 2024-02-19T19:06:37.630 INFO:tasks.workunit.client.0.smithi088.stdout:2/853: dread d3/d1b/f1c [0,4194304] 0 2024-02-19T19:06:37.630 INFO:tasks.workunit.client.0.smithi088.stdout:4/811: write d2/d32/fbb [1133484,75110] 0 2024-02-19T19:06:37.631 INFO:tasks.workunit.client.0.smithi088.stdout:4/812: write d2/d32/fbb [1499561,105544] 0 2024-02-19T19:06:37.634 INFO:tasks.workunit.client.0.smithi088.stdout:4/813: fsync d2/d32/d25/d26/f88 0 2024-02-19T19:06:37.634 INFO:tasks.workunit.client.0.smithi088.stdout:2/854: link d3/d1f/d5d/fb4 d3/d1f/d5d/f11a 0 2024-02-19T19:06:37.635 INFO:tasks.workunit.client.0.smithi088.stdout:2/855: stat d3/d1f/f4a 0 2024-02-19T19:06:37.636 INFO:tasks.workunit.client.0.smithi088.stdout:2/856: chown d3/d1f/d5d/fe3 3 1 2024-02-19T19:06:37.637 INFO:tasks.workunit.client.0.smithi088.stdout:2/857: rmdir d3 39 2024-02-19T19:06:37.641 INFO:tasks.workunit.client.0.smithi088.stdout:4/814: dwrite d2/d32/d25/d26/d6d/d7e/d3a/fd4 [0,4194304] 0 2024-02-19T19:06:37.642 INFO:tasks.workunit.client.0.smithi088.stdout:4/815: chown d2/d32/d25/d26/d58 280159 1 2024-02-19T19:06:37.645 INFO:tasks.workunit.client.0.smithi088.stdout:2/858: dwrite d3/f85 [4194304,4194304] 0 2024-02-19T19:06:37.647 INFO:tasks.workunit.client.0.smithi088.stdout:4/816: dread d2/d32/d25/d26/d6d/d7e/d90/fa7 [0,4194304] 0 2024-02-19T19:06:37.647 INFO:tasks.workunit.client.0.smithi088.stdout:2/859: chown d3/d1b/d43/d61/d64/l90 15353555 1 2024-02-19T19:06:37.652 INFO:tasks.workunit.client.0.smithi088.stdout:4/817: write d2/d35/f4f [3274086,38224] 0 2024-02-19T19:06:37.655 INFO:tasks.workunit.client.0.smithi088.stdout:4/818: creat d2/d32/d25/d26/d6d/d7e/da3/f102 x:0 0 0 2024-02-19T19:06:37.659 INFO:tasks.workunit.client.0.smithi088.stdout:4/819: link d2/d32/d25/d26/d6d/d7e/d3a/d77/c96 d2/dc4/c103 0 2024-02-19T19:06:37.665 INFO:tasks.workunit.client.0.smithi088.stdout:4/820: dwrite d2/d32/d25/d26/d6d/d7e/d3a/f42 [0,4194304] 0 2024-02-19T19:06:37.667 INFO:tasks.workunit.client.0.smithi088.stdout:4/821: read d2/d35/db3/fe2 [122806,42387] 0 2024-02-19T19:06:37.674 INFO:tasks.workunit.client.0.smithi088.stdout:4/822: dwrite d2/d32/d25/fd3 [0,4194304] 0 2024-02-19T19:06:37.680 INFO:tasks.workunit.client.0.smithi088.stdout:4/823: getdents d2/d32/d25/d26/d6d/d7e/d90 0 2024-02-19T19:06:37.699 INFO:tasks.workunit.client.0.smithi088.stdout:4/824: dread d2/d32/d25/d45/f4a [4194304,4194304] 0 2024-02-19T19:06:37.702 INFO:tasks.workunit.client.0.smithi088.stdout:4/825: unlink d2/d32/d25/d26/d6d/d7e/d3a/d77/c96 0 2024-02-19T19:06:37.702 INFO:tasks.workunit.client.0.smithi088.stdout:4/826: fsync d2/d32/d25/d3f/d70/f8b 0 2024-02-19T19:06:37.703 INFO:tasks.workunit.client.0.smithi088.stdout:4/827: read d2/d32/fb [281259,18535] 0 2024-02-19T19:06:37.705 INFO:tasks.workunit.client.0.smithi088.stdout:4/828: unlink d2/c63 0 2024-02-19T19:06:37.707 INFO:tasks.workunit.client.0.smithi088.stdout:4/829: unlink d2/d32/d25/d26/d6d/d7e/d3a/d77/f92 0 2024-02-19T19:06:37.708 INFO:tasks.workunit.client.0.smithi088.stdout:4/830: chown d2/d32/d25/d3f/fe1 134827 1 2024-02-19T19:06:37.708 INFO:tasks.workunit.client.0.smithi088.stdout:4/831: chown d2/d32/d25/d26/d4e/ce9 4258 1 2024-02-19T19:06:37.709 INFO:tasks.workunit.client.0.smithi088.stdout:4/832: fdatasync d2/d35/f4f 0 2024-02-19T19:06:37.711 INFO:tasks.workunit.client.0.smithi088.stdout:4/833: mknod d2/d32/d25/d26/da5/db2/df3/dff/c104 0 2024-02-19T19:06:37.712 INFO:tasks.workunit.client.0.smithi088.stdout:4/834: mknod d2/d32/d25/d26/d4e/d6f/c105 0 2024-02-19T19:06:37.714 INFO:tasks.workunit.client.0.smithi088.stdout:4/835: creat d2/d32/d25/d26/db8/f106 x:0 0 0 2024-02-19T19:06:37.714 INFO:tasks.workunit.client.0.smithi088.stdout:4/836: fsync d2/d32/d25/faf 0 2024-02-19T19:06:37.716 INFO:tasks.workunit.client.0.smithi088.stdout:4/837: write d2/d32/d25/d26/d4e/d6f/ffd [1028419,39536] 0 2024-02-19T19:06:37.717 INFO:tasks.workunit.client.0.smithi088.stdout:4/838: read d2/d32/d25/f46 [3011025,51710] 0 2024-02-19T19:06:37.720 INFO:tasks.workunit.client.0.smithi088.stdout:4/839: unlink d2/d32/d25/f30 0 2024-02-19T19:06:37.720 INFO:tasks.workunit.client.0.smithi088.stdout:4/840: write d2/d8/f14 [2522308,77690] 0 2024-02-19T19:06:37.723 INFO:tasks.workunit.client.0.smithi088.stdout:4/841: truncate d2/d32/d25/d26/d58/f69 4353943 0 2024-02-19T19:06:37.732 INFO:tasks.workunit.client.0.smithi088.stdout:4/842: mknod d2/d32/d25/d3f/c107 0 2024-02-19T19:06:37.732 INFO:tasks.workunit.client.0.smithi088.stdout:4/843: dread - d2/d32/d9/dd8/de0/fd9 zero size 2024-02-19T19:06:37.732 INFO:tasks.workunit.client.0.smithi088.stdout:4/844: chown d2/d32/d25/d26/da5/db2/ld2 30756 1 2024-02-19T19:06:37.735 INFO:tasks.workunit.client.0.smithi088.stdout:4/845: unlink d2/d32/d25/d26/da0/fd5 0 2024-02-19T19:06:37.736 INFO:tasks.workunit.client.0.smithi088.stdout:0/934: sync 2024-02-19T19:06:37.738 INFO:tasks.workunit.client.0.smithi088.stdout:0/935: mknod d13/d62/d8a/d95/c146 0 2024-02-19T19:06:37.748 INFO:tasks.workunit.client.0.smithi088.stdout:0/936: dread d13/d62/d8a/d9c/da8/de2/f6e [0,4194304] 0 2024-02-19T19:06:37.752 INFO:tasks.workunit.client.0.smithi088.stdout:0/937: symlink d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/d10a/d106/de5/l147 0 2024-02-19T19:06:37.754 INFO:tasks.workunit.client.0.smithi088.stdout:0/938: fsync d13/d62/d8a/d9c/da8/dab/db4/fc1 0 2024-02-19T19:06:37.893 INFO:tasks.workunit.client.0.smithi088.stdout:3/958: sync 2024-02-19T19:06:37.894 INFO:tasks.workunit.client.0.smithi088.stdout:8/911: sync 2024-02-19T19:06:37.895 INFO:tasks.workunit.client.0.smithi088.stdout:8/912: dread - d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/fdd zero size 2024-02-19T19:06:37.898 INFO:tasks.workunit.client.0.smithi088.stdout:8/913: rmdir d9/df/d3d/d50/d71/da1/d6c/d9a/d6b 39 2024-02-19T19:06:37.899 INFO:tasks.workunit.client.0.smithi088.stdout:8/914: truncate d9/df/d1c/d10e/d8e/f110 729962 0 2024-02-19T19:06:37.900 INFO:tasks.workunit.client.0.smithi088.stdout:3/959: write d4/d22/d7a/d8d/fb9 [890883,107372] 0 2024-02-19T19:06:37.900 INFO:tasks.workunit.client.0.smithi088.stdout:8/915: read - d9/df/d1c/d85/dc5/f10b zero size 2024-02-19T19:06:37.902 INFO:tasks.workunit.client.0.smithi088.stdout:3/960: getdents d4/d12/d33/d43/d11d 0 2024-02-19T19:06:37.903 INFO:tasks.workunit.client.0.smithi088.stdout:8/916: mkdir d9/df/d3d/d50/d71/da1/d11b 0 2024-02-19T19:06:37.904 INFO:tasks.workunit.client.0.smithi088.stdout:8/917: write d9/df/d1c/f16 [6785424,113938] 0 2024-02-19T19:06:37.904 INFO:tasks.workunit.client.0.smithi088.stdout:3/961: truncate d4/d22/d7a/da1/fac 295785 0 2024-02-19T19:06:37.906 INFO:tasks.workunit.client.0.smithi088.stdout:3/962: chown d4/d22/d7a/da1/fac 30 1 2024-02-19T19:06:37.907 INFO:tasks.workunit.client.0.smithi088.stdout:3/963: fsync d4/d1a/d1e/d31/d32/f4c 0 2024-02-19T19:06:37.908 INFO:tasks.workunit.client.0.smithi088.stdout:8/918: creat d9/df/d1c/f11c x:0 0 0 2024-02-19T19:06:37.908 INFO:tasks.workunit.client.0.smithi088.stdout:8/919: readlink d9/df/d1c/d22/l24 0 2024-02-19T19:06:37.909 INFO:tasks.workunit.client.0.smithi088.stdout:8/920: fsync d9/df/d3d/f87 0 2024-02-19T19:06:37.909 INFO:tasks.workunit.client.0.smithi088.stdout:3/964: creat d4/d100/f14b x:0 0 0 2024-02-19T19:06:37.911 INFO:tasks.workunit.client.0.smithi088.stdout:8/921: chown d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/fe9 1 1 2024-02-19T19:06:37.914 INFO:tasks.workunit.client.0.smithi088.stdout:3/965: dread d4/d12/d33/dab/ff4 [0,4194304] 0 2024-02-19T19:06:37.916 INFO:tasks.workunit.client.0.smithi088.stdout:3/966: mknod d4/d12/d33/d43/d85/c14c 0 2024-02-19T19:06:37.917 INFO:tasks.workunit.client.0.smithi088.stdout:3/967: fsync d4/d12/d33/d43/fd9 0 2024-02-19T19:06:37.920 INFO:tasks.workunit.client.0.smithi088.stdout:3/968: mknod d4/d1a/d1e/d31/dc5/dcf/c14d 0 2024-02-19T19:06:37.922 INFO:tasks.workunit.client.0.smithi088.stdout:3/969: mknod d4/d1a/d1e/d74/d7d/daa/c14e 0 2024-02-19T19:06:37.924 INFO:tasks.workunit.client.0.smithi088.stdout:8/922: dwrite d9/df/d3d/d50/d71/da1/d6c/d9a/f44 [0,4194304] 0 2024-02-19T19:06:37.927 INFO:tasks.workunit.client.0.smithi088.stdout:3/970: rmdir d4/d22/dd5/d104/d114 0 2024-02-19T19:06:37.929 INFO:tasks.workunit.client.0.smithi088.stdout:8/923: mknod d9/df/d3d/d50/d71/c11d 0 2024-02-19T19:06:37.930 INFO:tasks.workunit.client.0.smithi088.stdout:3/971: symlink d4/d1a/l14f 0 2024-02-19T19:06:37.931 INFO:tasks.workunit.client.0.smithi088.stdout:3/972: chown d4/d1a/d1e/d31/ff6 302 1 2024-02-19T19:06:37.936 INFO:tasks.workunit.client.0.smithi088.stdout:3/973: mkdir d4/d1a/d7f/d150 0 2024-02-19T19:06:37.938 INFO:tasks.workunit.client.0.smithi088.stdout:8/924: link d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/f65 d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/f11e 0 2024-02-19T19:06:37.941 INFO:tasks.workunit.client.0.smithi088.stdout:3/974: dread d4/d22/f2c [0,4194304] 0 2024-02-19T19:06:37.942 INFO:tasks.workunit.client.0.smithi088.stdout:3/975: chown d4/d22/d7a/d8d/fb9 134566 1 2024-02-19T19:06:37.943 INFO:tasks.workunit.client.0.smithi088.stdout:8/925: dread d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/fd2 [0,4194304] 0 2024-02-19T19:06:37.945 INFO:tasks.workunit.client.0.smithi088.stdout:8/926: fdatasync d9/df/d1c/d10e/fcb 0 2024-02-19T19:06:37.946 INFO:tasks.workunit.client.0.smithi088.stdout:3/976: getdents d4/d1a/d1e/d74/d7d/daa 0 2024-02-19T19:06:37.951 INFO:tasks.workunit.client.0.smithi088.stdout:8/927: creat d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/dea/dec/f11f x:0 0 0 2024-02-19T19:06:37.952 INFO:tasks.workunit.client.0.smithi088.stdout:8/928: write d9/df/d3d/d50/d71/f103 [1025655,75347] 0 2024-02-19T19:06:37.958 INFO:tasks.workunit.client.0.smithi088.stdout:3/977: dwrite d4/d12/d33/d58/f107 [0,4194304] 0 2024-02-19T19:06:37.958 INFO:tasks.workunit.client.0.smithi088.stdout:8/929: link d9/df/d3d/d50/d71/da1/f54 d9/df/d3d/d50/d71/da1/f120 0 2024-02-19T19:06:37.959 INFO:tasks.workunit.client.0.smithi088.stdout:1/993: sync 2024-02-19T19:06:37.961 INFO:tasks.workunit.client.0.smithi088.stdout:3/978: fdatasync d4/d22/d7a/d11a/d11b/fcc 0 2024-02-19T19:06:37.964 INFO:tasks.workunit.client.0.smithi088.stdout:3/979: dread - d4/d1a/d7f/d13e/f111 zero size 2024-02-19T19:06:37.968 INFO:tasks.workunit.client.0.smithi088.stdout:1/994: dwrite d6/d21/d6a/d7d/d9a/f109 [0,4194304] 0 2024-02-19T19:06:37.971 INFO:tasks.workunit.client.0.smithi088.stdout:1/995: unlink d6/d10/l5b 0 2024-02-19T19:06:37.976 INFO:tasks.workunit.client.0.smithi088.stdout:3/980: dwrite d4/d22/d7a/fbc [0,4194304] 0 2024-02-19T19:06:37.981 INFO:tasks.workunit.client.0.smithi088.stdout:8/930: dread d9/df/d3d/dfc/f100 [0,4194304] 0 2024-02-19T19:06:37.983 INFO:tasks.workunit.client.0.smithi088.stdout:1/996: dwrite d6/d21/d6a/dcb/f105 [0,4194304] 0 2024-02-19T19:06:37.983 INFO:tasks.workunit.client.0.smithi088.stdout:3/981: creat d4/d1a/d1e/d31/d32/f151 x:0 0 0 2024-02-19T19:06:37.986 INFO:tasks.workunit.client.0.smithi088.stdout:3/982: rmdir d4/d87/d118 39 2024-02-19T19:06:37.988 INFO:tasks.workunit.client.0.smithi088.stdout:1/997: mknod d6/d21/d64/d108/c13b 0 2024-02-19T19:06:37.989 INFO:tasks.workunit.client.0.smithi088.stdout:1/998: write d6/d10/f11d [274069,23450] 0 2024-02-19T19:06:37.990 INFO:tasks.workunit.client.0.smithi088.stdout:3/983: chown d4/d87/lf9 49562803 1 2024-02-19T19:06:37.992 INFO:tasks.workunit.client.0.smithi088.stdout:8/931: rename d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/d104 to d9/dc4/de0/d121 0 2024-02-19T19:06:37.993 INFO:tasks.workunit.client.0.smithi088.stdout:1/999: creat d6/d21/d64/d94/d71/f13c x:0 0 0 2024-02-19T19:06:37.995 INFO:tasks.workunit.client.0.smithi088.stdout:8/932: fdatasync d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/fd2 0 2024-02-19T19:06:37.997 INFO:tasks.workunit.client.0.smithi088.stdout:3/984: rename d4/d1a/d1e/d74/d7d/daa/dcb/f10c to d4/d87/d118/f152 0 2024-02-19T19:06:38.002 INFO:tasks.workunit.client.0.smithi088.stdout:3/985: getdents d4/d1a/d1e/d74/d7d/daa 0 2024-02-19T19:06:38.003 INFO:tasks.workunit.client.0.smithi088.stdout:3/986: chown d4/d1a/d1e/d31/d32/c3a 3098115 1 2024-02-19T19:06:38.005 INFO:tasks.workunit.client.0.smithi088.stdout:3/987: mkdir d4/d1a/d1e/d74/d7d/d153 0 2024-02-19T19:06:38.007 INFO:tasks.workunit.client.0.smithi088.stdout:3/988: creat d4/d12/d33/d58/f154 x:0 0 0 2024-02-19T19:06:38.008 INFO:tasks.workunit.client.0.smithi088.stdout:3/989: chown d4/l123 1759 1 2024-02-19T19:06:38.010 INFO:tasks.workunit.client.0.smithi088.stdout:3/990: truncate d4/d87/d118/d13f/fbe 671787 0 2024-02-19T19:06:38.011 INFO:tasks.workunit.client.0.smithi088.stdout:8/933: dwrite d9/df/d3d/d62/f69 [0,4194304] 0 2024-02-19T19:06:38.012 INFO:tasks.workunit.client.0.smithi088.stdout:8/934: stat d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/c115 0 2024-02-19T19:06:38.013 INFO:tasks.workunit.client.0.smithi088.stdout:3/991: creat d4/d1a/d1e/d74/d7d/f155 x:0 0 0 2024-02-19T19:06:38.015 INFO:tasks.workunit.client.0.smithi088.stdout:2/860: sync 2024-02-19T19:06:38.015 INFO:tasks.workunit.client.0.smithi088.stdout:0/939: sync 2024-02-19T19:06:38.015 INFO:tasks.workunit.client.0.smithi088.stdout:4/846: sync 2024-02-19T19:06:38.016 INFO:tasks.workunit.client.0.smithi088.stdout:3/992: creat d4/d1a/d1e/d74/d7d/d153/f156 x:0 0 0 2024-02-19T19:06:38.017 INFO:tasks.workunit.client.0.smithi088.stdout:2/861: read d3/d1f/d5d/f62 [415460,17866] 0 2024-02-19T19:06:38.017 INFO:tasks.workunit.client.0.smithi088.stdout:4/847: creat d2/d32/d25/d4b/f108 x:0 0 0 2024-02-19T19:06:38.018 INFO:tasks.workunit.client.0.smithi088.stdout:2/862: write d3/d1f/d4f/d59/fd9 [583953,19363] 0 2024-02-19T19:06:38.019 INFO:tasks.workunit.client.0.smithi088.stdout:0/940: fsync f3 0 2024-02-19T19:06:38.021 INFO:tasks.workunit.client.0.smithi088.stdout:3/993: creat d4/d1a/d1e/d74/f157 x:0 0 0 2024-02-19T19:06:38.021 INFO:tasks.workunit.client.0.smithi088.stdout:4/848: symlink d2/d32/d25/d26/d4e/l109 0 2024-02-19T19:06:38.022 INFO:tasks.workunit.client.0.smithi088.stdout:2/863: symlink d3/d1b/d43/d8f/d94/l11b 0 2024-02-19T19:06:38.022 INFO:tasks.workunit.client.0.smithi088.stdout:4/849: dread - d2/d32/d25/d3f/d70/fc8 zero size 2024-02-19T19:06:38.023 INFO:tasks.workunit.client.0.smithi088.stdout:3/994: rmdir d4/d22/dd5/d104 39 2024-02-19T19:06:38.024 INFO:tasks.workunit.client.0.smithi088.stdout:0/941: fsync d13/f1b 0 2024-02-19T19:06:38.026 INFO:tasks.workunit.client.0.smithi088.stdout:2/864: mknod d3/d1b/d43/df4/c11c 0 2024-02-19T19:06:38.026 INFO:tasks.workunit.client.0.smithi088.stdout:3/995: creat d4/d12/d33/d43/d85/d10a/f158 x:0 0 0 2024-02-19T19:06:38.028 INFO:tasks.workunit.client.0.smithi088.stdout:3/996: chown d4/d22/d7a/d11a/l12b 884 1 2024-02-19T19:06:38.030 INFO:tasks.workunit.client.0.smithi088.stdout:0/942: read d13/d62/d8a/d95/d5a/d25/f2b [781754,110432] 0 2024-02-19T19:06:38.031 INFO:tasks.workunit.client.0.smithi088.stdout:4/850: dwrite d2/d32/d25/d45/f4a [8388608,4194304] 0 2024-02-19T19:06:38.034 INFO:tasks.workunit.client.0.smithi088.stdout:4/851: chown d2/d32/d25/d26/d6d/d7e/d3a/l3c 16787821 1 2024-02-19T19:06:38.036 INFO:tasks.workunit.client.0.smithi088.stdout:3/997: dread d4/d1a/d1e/d74/d7d/fe3 [0,4194304] 0 2024-02-19T19:06:38.036 INFO:tasks.workunit.client.0.smithi088.stdout:2/865: dwrite d3/d1b/d43/d61/da6/f115 [0,4194304] 0 2024-02-19T19:06:38.037 INFO:tasks.workunit.client.0.smithi088.stdout:4/852: dread d2/d35/db3/fe2 [0,4194304] 0 2024-02-19T19:06:38.039 INFO:tasks.workunit.client.0.smithi088.stdout:3/998: dread d4/d1a/d1e/d74/d7d/fe3 [0,4194304] 0 2024-02-19T19:06:38.040 INFO:tasks.workunit.client.0.smithi088.stdout:2/866: mkdir d3/d14/d11d 0 2024-02-19T19:06:38.041 INFO:tasks.workunit.client.0.smithi088.stdout:3/999: read - d4/d87/d118/f152 zero size 2024-02-19T19:06:38.043 INFO:tasks.workunit.client.0.smithi088.stdout:4/853: creat d2/d32/d25/d26/da5/db2/df3/dff/f10a x:0 0 0 2024-02-19T19:06:38.043 INFO:tasks.workunit.client.0.smithi088.stdout:0/943: link d13/d62/d8a/d95/d5a/d25/d2a/d34/c6b d13/d62/d8a/d95/d5a/d25/dd6/c148 0 2024-02-19T19:06:38.043 INFO:tasks.workunit.client.0.smithi088.stdout:2/867: chown d3/c3c 244 1 2024-02-19T19:06:38.044 INFO:tasks.workunit.client.0.smithi088.stdout:0/944: readlink d13/d62/d8a/d95/d5a/d25/la1 0 2024-02-19T19:06:38.048 INFO:tasks.workunit.client.0.smithi088.stdout:4/854: link d2/d32/fb d2/d32/d25/d26/da5/db2/f10b 0 2024-02-19T19:06:38.048 INFO:tasks.workunit.client.0.smithi088.stdout:0/945: getdents d13/d62/d8a/d95/d5b/d76/dde 0 2024-02-19T19:06:38.049 INFO:tasks.workunit.client.0.smithi088.stdout:0/946: write d13/d62/d8a/d95/d5b/f137 [1803487,92981] 0 2024-02-19T19:06:38.050 INFO:tasks.workunit.client.0.smithi088.stdout:4/855: mknod d2/d32/d25/d26/da5/c10c 0 2024-02-19T19:06:38.052 INFO:tasks.workunit.client.0.smithi088.stdout:4/856: mkdir d2/d35/db3/db7/dd7/d10d 0 2024-02-19T19:06:38.053 INFO:tasks.workunit.client.0.smithi088.stdout:4/857: chown d2/d32/d9/c73 45964204 1 2024-02-19T19:06:38.059 INFO:tasks.workunit.client.0.smithi088.stdout:0/947: dwrite d13/d62/d8a/d9c/da8/dab/db4/dbf/ff5 [0,4194304] 0 2024-02-19T19:06:38.061 INFO:tasks.workunit.client.0.smithi088.stdout:0/948: stat d13/d62/d8a/dfe/f12f 0 2024-02-19T19:06:38.064 INFO:tasks.workunit.client.0.smithi088.stdout:2/868: dwrite d3/d1b/d43/d61/fe7 [0,4194304] 0 2024-02-19T19:06:38.066 INFO:tasks.workunit.client.0.smithi088.stdout:4/858: write d2/f75 [1649463,127527] 0 2024-02-19T19:06:38.070 INFO:tasks.workunit.client.0.smithi088.stdout:2/869: fsync d3/d1b/d43/d61/da6/db2/fca 0 2024-02-19T19:06:38.070 INFO:tasks.workunit.client.0.smithi088.stdout:4/859: symlink d2/d32/d25/d26/da0/l10e 0 2024-02-19T19:06:38.072 INFO:tasks.workunit.client.0.smithi088.stdout:4/860: write d2/d32/fbb [802060,14348] 0 2024-02-19T19:06:38.074 INFO:tasks.workunit.client.0.smithi088.stdout:2/870: symlink d3/d1f/d4f/d59/l11e 0 2024-02-19T19:06:38.079 INFO:tasks.workunit.client.0.smithi088.stdout:2/871: getdents d3/d1b/d43/d61/da6 0 2024-02-19T19:06:38.082 INFO:tasks.workunit.client.0.smithi088.stdout:2/872: dread d3/d1f/d5d/fe3 [0,4194304] 0 2024-02-19T19:06:38.085 INFO:tasks.workunit.client.0.smithi088.stdout:4/861: write d2/d32/d25/d26/d6d/d7e/d3a/fe5 [568666,33963] 0 2024-02-19T19:06:38.087 INFO:tasks.workunit.client.0.smithi088.stdout:0/949: dwrite d13/d62/d8a/d95/d5a/d25/f26 [0,4194304] 0 2024-02-19T19:06:38.128 INFO:tasks.workunit.client.0.smithi088.stdout:4/862: dwrite d2/d8/f14 [4194304,4194304] 0 2024-02-19T19:06:38.130 INFO:tasks.workunit.client.0.smithi088.stdout:4/863: dread d2/d32/d9/fd6 [0,4194304] 0 2024-02-19T19:06:38.132 INFO:tasks.workunit.client.0.smithi088.stdout:4/864: symlink d2/d32/d25/d26/db8/l10f 0 2024-02-19T19:06:38.134 INFO:tasks.workunit.client.0.smithi088.stdout:4/865: rmdir d2/d8 39 2024-02-19T19:06:38.140 INFO:tasks.workunit.client.0.smithi088.stdout:4/866: unlink d2/d8/f6b 0 2024-02-19T19:06:38.140 INFO:tasks.workunit.client.0.smithi088.stdout:4/867: stat d2/d32/fbb 0 2024-02-19T19:06:38.140 INFO:tasks.workunit.client.0.smithi088.stdout:4/868: rename d2/d35/db3/db7/fc5 to d2/d32/d25/d26/da5/db2/f110 0 2024-02-19T19:06:38.141 INFO:tasks.workunit.client.0.smithi088.stdout:4/869: chown d2/d32/d25/d26/d6d/lf4 14045 1 2024-02-19T19:06:38.141 INFO:tasks.workunit.client.0.smithi088.stdout:4/870: chown d2/d32/d25/d3f/d9a/ff6 1641036 1 2024-02-19T19:06:38.152 INFO:tasks.workunit.client.0.smithi088.stdout:4/871: truncate d2/f75 1019122 0 2024-02-19T19:06:38.154 INFO:tasks.workunit.client.0.smithi088.stdout:4/872: write d2/d32/d25/d26/da5/db2/df3/dff/f10a [319906,103446] 0 2024-02-19T19:06:38.160 INFO:tasks.workunit.client.0.smithi088.stdout:4/873: truncate d2/d32/d25/d26/d6d/d7e/d3a/fd4 3986880 0 2024-02-19T19:06:38.162 INFO:tasks.workunit.client.0.smithi088.stdout:4/874: mkdir d2/d32/d25/d26/da5/d111 0 2024-02-19T19:06:38.165 INFO:tasks.workunit.client.0.smithi088.stdout:4/875: dread d2/d35/f4f [0,4194304] 0 2024-02-19T19:06:38.168 INFO:tasks.workunit.client.0.smithi088.stdout:4/876: dread d2/d32/d25/f8f [0,4194304] 0 2024-02-19T19:06:38.170 INFO:tasks.workunit.client.0.smithi088.stdout:4/877: read d2/d35/f53 [3271712,105508] 0 2024-02-19T19:06:38.313 INFO:tasks.workunit.client.0.smithi088.stdout:8/935: sync 2024-02-19T19:06:38.313 INFO:tasks.workunit.client.0.smithi088.stdout:8/936: stat d9/c41 0 2024-02-19T19:06:38.315 INFO:tasks.workunit.client.0.smithi088.stdout:8/937: unlink d9/df/d1c/cde 0 2024-02-19T19:06:38.321 INFO:tasks.workunit.client.0.smithi088.stdout:8/938: write d9/dc4/de0/d121/fca [65889,81026] 0 2024-02-19T19:06:38.324 INFO:tasks.workunit.client.0.smithi088.stdout:8/939: creat d9/df/d1c/d10e/f122 x:0 0 0 2024-02-19T19:06:38.326 INFO:tasks.workunit.client.0.smithi088.stdout:8/940: fsync d9/df/d3d/d50/d71/fd4 0 2024-02-19T19:06:38.330 INFO:tasks.workunit.client.0.smithi088.stdout:8/941: truncate d9/df/d3d/d50/d71/da1/d70/fa7 4847041 0 2024-02-19T19:06:38.331 INFO:tasks.workunit.client.0.smithi088.stdout:8/942: write d9/df/d3d/d50/d71/f8b [1150421,21677] 0 2024-02-19T19:06:38.334 INFO:tasks.workunit.client.0.smithi088.stdout:8/943: unlink d9/df/d1c/d22/d9e/faa 0 2024-02-19T19:06:38.335 INFO:tasks.workunit.client.0.smithi088.stdout:8/944: fsync d9/df/d3d/d50/feb 0 2024-02-19T19:06:38.338 INFO:tasks.workunit.client.0.smithi088.stdout:8/945: dread d9/df/d3d/d50/d71/f8b [0,4194304] 0 2024-02-19T19:06:38.342 INFO:tasks.workunit.client.0.smithi088.stdout:8/946: creat d9/df/d1c/d10e/f123 x:0 0 0 2024-02-19T19:06:38.346 INFO:tasks.workunit.client.0.smithi088.stdout:8/947: link d9/df/d3d/d50/d71/da1/d6c/fba d9/df/d3d/d50/d71/da1/d6c/d77/d97/f124 0 2024-02-19T19:06:38.349 INFO:tasks.workunit.client.0.smithi088.stdout:8/948: creat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/dbe/f125 x:0 0 0 2024-02-19T19:06:38.354 INFO:tasks.workunit.client.0.smithi088.stdout:8/949: dwrite d9/df/d3d/d50/d71/da1/d6c/d77/d97/fc7 [0,4194304] 0 2024-02-19T19:06:38.355 INFO:tasks.workunit.client.0.smithi088.stdout:8/950: chown d9/df 915766424 1 2024-02-19T19:06:38.355 INFO:tasks.workunit.client.0.smithi088.stdout:8/951: readlink d9/df/d1c/l20 0 2024-02-19T19:06:38.356 INFO:tasks.workunit.client.0.smithi088.stdout:8/952: fsync d9/df/d3d/d50/d71/da1/d6c/d9a/f44 0 2024-02-19T19:06:38.357 INFO:tasks.workunit.client.0.smithi088.stdout:8/953: stat d9/df/d3d/f87 0 2024-02-19T19:06:38.358 INFO:tasks.workunit.client.0.smithi088.stdout:4/878: sync 2024-02-19T19:06:38.358 INFO:tasks.workunit.client.0.smithi088.stdout:0/950: sync 2024-02-19T19:06:38.358 INFO:tasks.workunit.client.0.smithi088.stdout:2/873: sync 2024-02-19T19:06:38.359 INFO:tasks.workunit.client.0.smithi088.stdout:8/954: chown d9/df/d3d/d50/d71/da1/f54 11 1 2024-02-19T19:06:38.362 INFO:tasks.workunit.client.0.smithi088.stdout:2/874: chown d3/d1b/d43/d8f/cd4 3606 1 2024-02-19T19:06:38.362 INFO:tasks.workunit.client.0.smithi088.stdout:4/879: fsync d2/d32/d25/d26/da5/db2/f110 0 2024-02-19T19:06:38.364 INFO:tasks.workunit.client.0.smithi088.stdout:8/955: mknod d9/df/d3d/d50/df1/c126 0 2024-02-19T19:06:38.368 INFO:tasks.workunit.client.0.smithi088.stdout:0/951: dwrite d13/d62/d8a/dfe/f12f [0,4194304] 0 2024-02-19T19:06:38.373 INFO:tasks.workunit.client.0.smithi088.stdout:2/875: dwrite d3/d1b/d43/d61/d7e/f105 [0,4194304] 0 2024-02-19T19:06:38.374 INFO:tasks.workunit.client.0.smithi088.stdout:0/952: symlink d13/d62/d8a/d95/d5a/d25/dd6/d105/l149 0 2024-02-19T19:06:38.374 INFO:tasks.workunit.client.0.smithi088.stdout:8/956: dread d9/df/d1c/f25 [4194304,4194304] 0 2024-02-19T19:06:38.380 INFO:tasks.workunit.client.0.smithi088.stdout:0/953: creat d13/d62/d8a/d95/d5a/db6/f14a x:0 0 0 2024-02-19T19:06:38.389 INFO:tasks.workunit.client.0.smithi088.stdout:2/876: dread d3/d1f/f73 [0,4194304] 0 2024-02-19T19:06:38.391 INFO:tasks.workunit.client.0.smithi088.stdout:2/877: fdatasync d3/d1b/f67 0 2024-02-19T19:06:38.393 INFO:tasks.workunit.client.0.smithi088.stdout:2/878: readlink d3/l18 0 2024-02-19T19:06:38.395 INFO:tasks.workunit.client.0.smithi088.stdout:2/879: read d3/d14/f56 [1578094,66632] 0 2024-02-19T19:06:38.398 INFO:tasks.workunit.client.0.smithi088.stdout:2/880: read d3/d14/f91 [538974,6889] 0 2024-02-19T19:06:38.399 INFO:tasks.workunit.client.0.smithi088.stdout:4/880: dread d2/d32/d25/d26/d6d/d7e/d90/f94 [0,4194304] 0 2024-02-19T19:06:38.399 INFO:tasks.workunit.client.0.smithi088.stdout:0/954: dwrite d13/d62/d8a/d95/d5a/d25/d2a/d3c/f55 [0,4194304] 0 2024-02-19T19:06:38.400 INFO:tasks.workunit.client.0.smithi088.stdout:4/881: stat d2/d32/d25/d26/d4e/d6f 0 2024-02-19T19:06:38.400 INFO:tasks.workunit.client.0.smithi088.stdout:0/955: chown d13/d62/d8a/d95/d5a/d25/d2a/d34 17881738 1 2024-02-19T19:06:38.403 INFO:tasks.workunit.client.0.smithi088.stdout:4/882: getdents d2/d32/de7 0 2024-02-19T19:06:38.404 INFO:tasks.workunit.client.0.smithi088.stdout:8/957: dread d9/df/d3d/d50/d71/da1/d70/fa7 [0,4194304] 0 2024-02-19T19:06:38.404 INFO:tasks.workunit.client.0.smithi088.stdout:4/883: stat d2/d35/db3/db7/de3/lde 0 2024-02-19T19:06:38.405 INFO:tasks.workunit.client.0.smithi088.stdout:8/958: chown d9/c84 1667580 1 2024-02-19T19:06:38.407 INFO:tasks.workunit.client.0.smithi088.stdout:0/956: write d13/f8c [5840,65408] 0 2024-02-19T19:06:38.408 INFO:tasks.workunit.client.0.smithi088.stdout:8/959: symlink d9/df/d1c/d10e/l127 0 2024-02-19T19:06:38.408 INFO:tasks.workunit.client.0.smithi088.stdout:2/881: truncate d3/d1b/d43/f66 3900689 0 2024-02-19T19:06:38.409 INFO:tasks.workunit.client.0.smithi088.stdout:0/957: rmdir d13/d62/d8a/d9c/da8 39 2024-02-19T19:06:38.410 INFO:tasks.workunit.client.0.smithi088.stdout:2/882: write d3/d1f/fdc [4804834,126842] 0 2024-02-19T19:06:38.412 INFO:tasks.workunit.client.0.smithi088.stdout:8/960: link d9/df/d3d/d62/ffa d9/dc4/de0/f128 0 2024-02-19T19:06:38.412 INFO:tasks.workunit.client.0.smithi088.stdout:2/883: mkdir d3/d71/d11f 0 2024-02-19T19:06:38.413 INFO:tasks.workunit.client.0.smithi088.stdout:0/958: symlink d13/d62/d8a/d95/d5a/db6/l14b 0 2024-02-19T19:06:38.415 INFO:tasks.workunit.client.0.smithi088.stdout:8/961: creat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/dbe/f129 x:0 0 0 2024-02-19T19:06:38.416 INFO:tasks.workunit.client.0.smithi088.stdout:2/884: rename d3/d1b/d43/d61/da6 to d3/d1b/d43/d61/d7e/db6/d120 0 2024-02-19T19:06:38.416 INFO:tasks.workunit.client.0.smithi088.stdout:8/962: chown d9/df/d3d/d50/d71/da1/la4 239 1 2024-02-19T19:06:38.417 INFO:tasks.workunit.client.0.smithi088.stdout:0/959: mknod d13/d62/d8a/d9c/da8/d143/c14c 0 2024-02-19T19:06:38.418 INFO:tasks.workunit.client.0.smithi088.stdout:8/963: write d9/df/d1c/f25 [8699079,66571] 0 2024-02-19T19:06:38.419 INFO:tasks.workunit.client.0.smithi088.stdout:2/885: write d3/fa5 [4565824,30472] 0 2024-02-19T19:06:38.421 INFO:tasks.workunit.client.0.smithi088.stdout:0/960: dread d13/f15 [0,4194304] 0 2024-02-19T19:06:38.422 INFO:tasks.workunit.client.0.smithi088.stdout:2/886: mkdir d3/d1f/d113/d117/d121 0 2024-02-19T19:06:38.427 INFO:tasks.workunit.client.0.smithi088.stdout:0/961: link d13/d62/d8a/d95/d5b/d76/c88 d13/d62/d8a/d95/d5a/d25/d2a/d34/d100/c14d 0 2024-02-19T19:06:38.430 INFO:tasks.workunit.client.0.smithi088.stdout:8/964: dread d9/df/d3d/d50/d71/da1/f120 [0,4194304] 0 2024-02-19T19:06:38.430 INFO:tasks.workunit.client.0.smithi088.stdout:2/887: dwrite d3/d1b/d43/d61/f9e [0,4194304] 0 2024-02-19T19:06:38.430 INFO:tasks.workunit.client.0.smithi088.stdout:0/962: mknod d13/d81/c14e 0 2024-02-19T19:06:38.432 INFO:tasks.workunit.client.0.smithi088.stdout:8/965: chown d9/df/d1c/f11c 872104 1 2024-02-19T19:06:38.433 INFO:tasks.workunit.client.0.smithi088.stdout:0/963: write d13/d62/d8a/d95/d5a/d25/dd6/f11c [586923,98168] 0 2024-02-19T19:06:38.433 INFO:tasks.workunit.client.0.smithi088.stdout:2/888: truncate d3/d1b/d43/d61/d7e/db1/d10c/f10d 4600130 0 2024-02-19T19:06:38.446 INFO:tasks.workunit.client.0.smithi088.stdout:0/964: dread d13/d62/d8a/d95/d5b/d76/f119 [0,4194304] 0 2024-02-19T19:06:38.447 INFO:tasks.workunit.client.0.smithi088.stdout:0/965: dread - d13/d62/d8a/d95/d96/f12e zero size 2024-02-19T19:06:38.455 INFO:tasks.workunit.client.0.smithi088.stdout:2/889: dread d3/d1f/fdc [0,4194304] 0 2024-02-19T19:06:38.463 INFO:tasks.workunit.client.0.smithi088.stdout:2/890: dwrite d3/d1b/d43/f66 [0,4194304] 0 2024-02-19T19:06:38.470 INFO:tasks.workunit.client.0.smithi088.stdout:2/891: dwrite d3/fe8 [4194304,4194304] 0 2024-02-19T19:06:38.473 INFO:tasks.workunit.client.0.smithi088.stdout:2/892: creat d3/d1f/d113/d117/d121/f122 x:0 0 0 2024-02-19T19:06:38.487 INFO:tasks.workunit.client.0.smithi088.stdout:0/966: unlink d13/f6a 0 2024-02-19T19:06:38.490 INFO:tasks.workunit.client.0.smithi088.stdout:2/893: write d3/d1f/d4f/f11 [9008062,87233] 0 2024-02-19T19:06:38.491 INFO:tasks.workunit.client.0.smithi088.stdout:0/967: rmdir d13/d62/d8a/d9c/da8/dab 39 2024-02-19T19:06:38.495 INFO:tasks.workunit.client.0.smithi088.stdout:0/968: chown d13/d62/d8a/d95/d5a/d25/c60 541475592 1 2024-02-19T19:06:38.495 INFO:tasks.workunit.client.0.smithi088.stdout:8/966: dwrite d9/df/d1c/d10e/fcb [0,4194304] 0 2024-02-19T19:06:38.503 INFO:tasks.workunit.client.0.smithi088.stdout:0/969: dwrite d13/d62/d8a/d95/d96/ded/f132 [0,4194304] 0 2024-02-19T19:06:38.507 INFO:tasks.workunit.client.0.smithi088.stdout:2/894: dread d3/d1f/d5d/de1/fe5 [0,4194304] 0 2024-02-19T19:06:38.508 INFO:tasks.workunit.client.0.smithi088.stdout:2/895: readlink d3/d1f/d4f/l16 0 2024-02-19T19:06:38.508 INFO:tasks.workunit.client.0.smithi088.stdout:0/970: mkdir d13/d62/d8a/d95/d5b/d14f 0 2024-02-19T19:06:38.511 INFO:tasks.workunit.client.0.smithi088.stdout:2/896: mknod d3/d1b/d43/d8f/c123 0 2024-02-19T19:06:38.512 INFO:tasks.workunit.client.0.smithi088.stdout:0/971: rmdir d13/d62/d8a/d95/d5a 39 2024-02-19T19:06:38.513 INFO:tasks.workunit.client.0.smithi088.stdout:2/897: mkdir d3/d14/dd8/de9/d110/d124 0 2024-02-19T19:06:38.516 INFO:tasks.workunit.client.0.smithi088.stdout:2/898: getdents d3/d1f/d113 0 2024-02-19T19:06:38.517 INFO:tasks.workunit.client.0.smithi088.stdout:0/972: link d13/d62/d8a/d95/d5a/d25/d2a/d2f/ffc d13/d62/d8a/d95/f150 0 2024-02-19T19:06:38.520 INFO:tasks.workunit.client.0.smithi088.stdout:0/973: write d13/d62/d8a/d9c/da8/dab/db4/dbf/ff5 [715441,20556] 0 2024-02-19T19:06:38.523 INFO:tasks.workunit.client.0.smithi088.stdout:0/974: mknod d13/d62/d8a/d95/d5b/d76/dde/c151 0 2024-02-19T19:06:38.526 INFO:tasks.workunit.client.0.smithi088.stdout:0/975: fdatasync d13/d62/d8a/f79 0 2024-02-19T19:06:38.530 INFO:tasks.workunit.client.0.smithi088.stdout:2/899: dwrite d3/d1b/d43/d61/fc8 [0,4194304] 0 2024-02-19T19:06:38.531 INFO:tasks.workunit.client.0.smithi088.stdout:2/900: chown d3/d1b/d43/d61/d7e/feb 68 1 2024-02-19T19:06:38.532 INFO:tasks.workunit.client.0.smithi088.stdout:2/901: write d3/d14/f1d [7920501,73101] 0 2024-02-19T19:06:38.534 INFO:tasks.workunit.client.0.smithi088.stdout:2/902: write d3/d1b/d43/d8f/f108 [339811,2814] 0 2024-02-19T19:06:38.540 INFO:tasks.workunit.client.0.smithi088.stdout:2/903: link d3/d1f/d5d/de1/ff3 d3/d1b/d43/f125 0 2024-02-19T19:06:38.544 INFO:tasks.workunit.client.0.smithi088.stdout:2/904: dread d3/d1b/d43/d61/d7e/db1/fbf [0,4194304] 0 2024-02-19T19:06:38.545 INFO:tasks.workunit.client.0.smithi088.stdout:2/905: fsync d3/d1b/d43/d61/d64/f10f 0 2024-02-19T19:06:38.545 INFO:tasks.workunit.client.0.smithi088.stdout:2/906: stat d3/d1b/d43/d61/ca9 0 2024-02-19T19:06:38.546 INFO:tasks.workunit.client.0.smithi088.stdout:2/907: fdatasync d3/fa5 0 2024-02-19T19:06:38.548 INFO:tasks.workunit.client.0.smithi088.stdout:2/908: creat d3/d1b/f126 x:0 0 0 2024-02-19T19:06:38.568 INFO:tasks.workunit.client.0.smithi088.stdout:2/909: rename d3/d1b/d43/l99 to d3/d1b/d43/d61/d7e/db6/d120/db2/l127 0 2024-02-19T19:06:38.568 INFO:tasks.workunit.client.0.smithi088.stdout:0/976: dread d13/d62/d8a/d95/d5b/d76/f83 [0,4194304] 0 2024-02-19T19:06:38.580 INFO:tasks.workunit.client.0.smithi088.stdout:0/977: dread d13/d62/d8a/d95/d5a/d25/fef [0,4194304] 0 2024-02-19T19:06:38.590 INFO:tasks.workunit.client.0.smithi088.stdout:2/910: chown d3/fb 5081462 1 2024-02-19T19:06:38.593 INFO:tasks.workunit.client.0.smithi088.stdout:0/978: fdatasync d13/d62/d8a/d95/d5a/d25/d2a/d2f/ddb/d10a/d106/fce 0 2024-02-19T19:06:38.596 INFO:tasks.workunit.client.0.smithi088.stdout:2/911: dwrite d3/d1b/d43/d8f/d94/fc2 [0,4194304] 0 2024-02-19T19:06:38.601 INFO:tasks.workunit.client.0.smithi088.stdout:0/979: rename d13/d62/d8a/d95/d5a/d25/dd6/d105 to d13/d81/d128/d152 0 2024-02-19T19:06:38.603 INFO:tasks.workunit.client.0.smithi088.stdout:2/912: dwrite d3/d1b/d43/d61/d7e/db6/d120/f115 [4194304,4194304] 0 2024-02-19T19:06:38.604 INFO:tasks.workunit.client.0.smithi088.stdout:0/980: creat d13/d81/d128/f153 x:0 0 0 2024-02-19T19:06:38.614 INFO:tasks.workunit.client.0.smithi088.stdout:0/981: dwrite d13/de9/f129 [0,4194304] 0 2024-02-19T19:06:38.618 INFO:tasks.workunit.client.0.smithi088.stdout:0/982: mkdir d13/d62/d8a/d9c/da8/dab/dd0/d154 0 2024-02-19T19:06:38.618 INFO:tasks.workunit.client.0.smithi088.stdout:2/913: dwrite d3/d1b/f2e [0,4194304] 0 2024-02-19T19:06:38.621 INFO:tasks.workunit.client.0.smithi088.stdout:0/983: dread - d13/f5c zero size 2024-02-19T19:06:38.623 INFO:tasks.workunit.client.0.smithi088.stdout:2/914: mknod d3/d1f/d5d/de1/c128 0 2024-02-19T19:06:38.625 INFO:tasks.workunit.client.0.smithi088.stdout:0/984: truncate d13/d62/d8a/d95/d5a/db6/fda 202639 0 2024-02-19T19:06:38.628 INFO:tasks.workunit.client.0.smithi088.stdout:2/915: link d3/d1b/d43/d61/d64/f63 d3/d1b/d43/d61/d64/f129 0 2024-02-19T19:06:38.628 INFO:tasks.workunit.client.0.smithi088.stdout:0/985: write d13/d62/d8a/d95/d5b/d76/f119 [4922584,18115] 0 2024-02-19T19:06:38.629 INFO:tasks.workunit.client.0.smithi088.stdout:2/916: read d3/f85 [6828923,18085] 0 2024-02-19T19:06:38.632 INFO:tasks.workunit.client.0.smithi088.stdout:2/917: write d3/d1f/d5d/fe3 [577008,124275] 0 2024-02-19T19:06:38.632 INFO:tasks.workunit.client.0.smithi088.stdout:0/986: truncate d13/f39 1750632 0 2024-02-19T19:06:38.635 INFO:tasks.workunit.client.0.smithi088.stdout:2/918: unlink d3/l45 0 2024-02-19T19:06:38.636 INFO:tasks.workunit.client.0.smithi088.stdout:0/987: creat d13/de9/f155 x:0 0 0 2024-02-19T19:06:38.638 INFO:tasks.workunit.client.0.smithi088.stdout:2/919: mknod d3/d14/dd8/de9/d110/d124/c12a 0 2024-02-19T19:06:38.641 INFO:tasks.workunit.client.0.smithi088.stdout:0/988: dwrite d13/de9/d118/fe6 [0,4194304] 0 2024-02-19T19:06:38.644 INFO:tasks.workunit.client.0.smithi088.stdout:0/989: creat d13/d62/d8a/d95/d5a/d25/dd6/f156 x:0 0 0 2024-02-19T19:06:38.649 INFO:tasks.workunit.client.0.smithi088.stdout:0/990: symlink d13/d62/d8a/d95/d5a/d25/l157 0 2024-02-19T19:06:38.651 INFO:tasks.workunit.client.0.smithi088.stdout:0/991: mkdir d13/d62/d8a/d95/d5b/dbe/d158 0 2024-02-19T19:06:38.652 INFO:tasks.workunit.client.0.smithi088.stdout:2/920: write d3/d1f/d5d/fb4 [4221728,47403] 0 2024-02-19T19:06:38.653 INFO:tasks.workunit.client.0.smithi088.stdout:0/992: mknod d13/d81/d128/c159 0 2024-02-19T19:06:38.654 INFO:tasks.workunit.client.0.smithi088.stdout:0/993: stat d13/de9/d118/lba 0 2024-02-19T19:06:38.654 INFO:tasks.workunit.client.0.smithi088.stdout:2/921: mknod d3/d1b/d43/d8f/c12b 0 2024-02-19T19:06:38.656 INFO:tasks.workunit.client.0.smithi088.stdout:0/994: creat d13/d62/d8a/de0/f15a x:0 0 0 2024-02-19T19:06:38.657 INFO:tasks.workunit.client.0.smithi088.stdout:2/922: creat d3/d1f/d113/d117/f12c x:0 0 0 2024-02-19T19:06:38.660 INFO:tasks.workunit.client.0.smithi088.stdout:2/923: truncate d3/d1b/d43/df4/f8b 171332 0 2024-02-19T19:06:38.665 INFO:tasks.workunit.client.0.smithi088.stdout:2/924: dwrite d3/d14/f56 [4194304,4194304] 0 2024-02-19T19:06:38.672 INFO:tasks.workunit.client.0.smithi088.stdout:0/995: dread d13/d62/d8a/d95/d5a/df3/f10d [0,4194304] 0 2024-02-19T19:06:38.673 INFO:tasks.workunit.client.0.smithi088.stdout:2/925: dwrite d3/d1b/d43/d61/d64/f10f [0,4194304] 0 2024-02-19T19:06:38.675 INFO:tasks.workunit.client.0.smithi088.stdout:0/996: chown d13/d62/d8a/d95/d5b/d76/f119 34 1 2024-02-19T19:06:38.675 INFO:tasks.workunit.client.0.smithi088.stdout:0/997: truncate d13/d81/faf 1017319 0 2024-02-19T19:06:38.678 INFO:tasks.workunit.client.0.smithi088.stdout:0/998: creat d13/d62/d8a/d95/d5b/dbe/f15b x:0 0 0 2024-02-19T19:06:38.680 INFO:tasks.workunit.client.0.smithi088.stdout:0/999: creat d13/d62/d8a/d95/d5a/df3/f15c x:0 0 0 2024-02-19T19:06:38.816 INFO:tasks.workunit.client.0.smithi088.stdout:8/967: sync 2024-02-19T19:06:38.816 INFO:tasks.workunit.client.0.smithi088.stdout:4/884: sync 2024-02-19T19:06:38.817 INFO:tasks.workunit.client.0.smithi088.stdout:4/885: chown d2/d32/d25/d3f/d9a/ff6 1 1 2024-02-19T19:06:38.819 INFO:tasks.workunit.client.0.smithi088.stdout:8/968: mknod d9/df/d3d/dfc/dff/c12a 0 2024-02-19T19:06:38.825 INFO:tasks.workunit.client.0.smithi088.stdout:4/886: dread d2/d32/d25/d26/d6d/d7e/d90/fa7 [0,4194304] 0 2024-02-19T19:06:38.828 INFO:tasks.workunit.client.0.smithi088.stdout:4/887: truncate d2/d32/d25/d26/d6d/d7e/d3a/f5f 3193324 0 2024-02-19T19:06:38.831 INFO:tasks.workunit.client.0.smithi088.stdout:8/969: dread d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/fda [0,4194304] 0 2024-02-19T19:06:38.832 INFO:tasks.workunit.client.0.smithi088.stdout:8/970: write d9/df/d3d/d50/d71/da1/d6c/d9a/f44 [1294122,67723] 0 2024-02-19T19:06:38.837 INFO:tasks.workunit.client.0.smithi088.stdout:8/971: dread d9/df/d3d/d50/d71/da1/d6c/d77/d97/fc7 [0,4194304] 0 2024-02-19T19:06:38.840 INFO:tasks.workunit.client.0.smithi088.stdout:8/972: mkdir d9/df/d1c/d10e/d12b 0 2024-02-19T19:06:38.840 INFO:tasks.workunit.client.0.smithi088.stdout:8/973: readlink d9/df/d1c/d10e/l127 0 2024-02-19T19:06:38.842 INFO:tasks.workunit.client.0.smithi088.stdout:8/974: symlink d9/df/d1c/d22/d9e/d102/l12c 0 2024-02-19T19:06:38.843 INFO:tasks.workunit.client.0.smithi088.stdout:4/888: write d2/d32/d25/d26/f62 [441611,70647] 0 2024-02-19T19:06:38.846 INFO:tasks.workunit.client.0.smithi088.stdout:4/889: creat d2/d32/d25/d26/da5/db2/df3/dff/f112 x:0 0 0 2024-02-19T19:06:38.848 INFO:tasks.workunit.client.0.smithi088.stdout:8/975: creat d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/df6/f12d x:0 0 0 2024-02-19T19:06:38.850 INFO:tasks.workunit.client.0.smithi088.stdout:4/890: getdents d2/d35 0 2024-02-19T19:06:38.852 INFO:tasks.workunit.client.0.smithi088.stdout:8/976: creat d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/f12e x:0 0 0 2024-02-19T19:06:38.854 INFO:tasks.workunit.client.0.smithi088.stdout:8/977: mknod d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/c12f 0 2024-02-19T19:06:38.857 INFO:tasks.workunit.client.0.smithi088.stdout:8/978: creat d9/df/d1c/d22/d9e/d102/f130 x:0 0 0 2024-02-19T19:06:38.858 INFO:tasks.workunit.client.0.smithi088.stdout:4/891: write d2/d35/f5b [727914,41883] 0 2024-02-19T19:06:38.860 INFO:tasks.workunit.client.0.smithi088.stdout:4/892: mkdir d2/d32/d25/d26/d58/d113 0 2024-02-19T19:06:38.861 INFO:tasks.workunit.client.0.smithi088.stdout:8/979: unlink d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/fdd 0 2024-02-19T19:06:38.865 INFO:tasks.workunit.client.0.smithi088.stdout:4/893: getdents d2/d32/d25/d3f/d70 0 2024-02-19T19:06:38.867 INFO:tasks.workunit.client.0.smithi088.stdout:4/894: symlink d2/d35/l114 0 2024-02-19T19:06:38.867 INFO:tasks.workunit.client.0.smithi088.stdout:2/926: sync 2024-02-19T19:06:38.870 INFO:tasks.workunit.client.0.smithi088.stdout:4/895: dread - d2/d32/d25/d26/da5/db2/ff7 zero size 2024-02-19T19:06:38.870 INFO:tasks.workunit.client.0.smithi088.stdout:8/980: dwrite d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/dea/f106 [0,4194304] 0 2024-02-19T19:06:38.871 INFO:tasks.workunit.client.0.smithi088.stdout:4/896: chown d2/d32/d9/de4 779 1 2024-02-19T19:06:38.872 INFO:tasks.workunit.client.0.smithi088.stdout:2/927: creat d3/d1b/d43/f12d x:0 0 0 2024-02-19T19:06:38.872 INFO:tasks.workunit.client.0.smithi088.stdout:8/981: readlink d9/df/lb7 0 2024-02-19T19:06:38.874 INFO:tasks.workunit.client.0.smithi088.stdout:4/897: symlink d2/d32/d25/d26/d58/l115 0 2024-02-19T19:06:38.875 INFO:tasks.workunit.client.0.smithi088.stdout:2/928: mknod d3/d1b/c12e 0 2024-02-19T19:06:38.876 INFO:tasks.workunit.client.0.smithi088.stdout:2/929: stat d3/d1b 0 2024-02-19T19:06:38.879 INFO:tasks.workunit.client.0.smithi088.stdout:2/930: mknod d3/d1f/d113/d117/d121/c12f 0 2024-02-19T19:06:38.880 INFO:tasks.workunit.client.0.smithi088.stdout:8/982: dwrite d9/df/d1c/d10e/f122 [0,4194304] 0 2024-02-19T19:06:38.884 INFO:tasks.workunit.client.0.smithi088.stdout:8/983: unlink d9/df/f27 0 2024-02-19T19:06:38.886 INFO:tasks.workunit.client.0.smithi088.stdout:8/984: fdatasync d9/df/d3d/dfc/f100 0 2024-02-19T19:06:38.887 INFO:tasks.workunit.client.0.smithi088.stdout:8/985: rmdir d9/df/d3d/d50 39 2024-02-19T19:06:38.891 INFO:tasks.workunit.client.0.smithi088.stdout:8/986: mkdir d9/df/d3d/d50/d71/da1/d11b/d131 0 2024-02-19T19:06:38.892 INFO:tasks.workunit.client.0.smithi088.stdout:2/931: read d3/d1b/d43/d8f/fd0 [737041,106967] 0 2024-02-19T19:06:38.892 INFO:tasks.workunit.client.0.smithi088.stdout:8/987: fdatasync d9/df/d1c/f25 0 2024-02-19T19:06:38.895 INFO:tasks.workunit.client.0.smithi088.stdout:8/988: creat d9/df/d3d/d50/f132 x:0 0 0 2024-02-19T19:06:38.896 INFO:tasks.workunit.client.0.smithi088.stdout:8/989: write d9/df/d3d/d50/d71/f107 [274158,59925] 0 2024-02-19T19:06:38.896 INFO:tasks.workunit.client.0.smithi088.stdout:2/932: link d3/d1b/d43/df4/le4 d3/d1f/d5d/de1/l130 0 2024-02-19T19:06:38.900 INFO:tasks.workunit.client.0.smithi088.stdout:8/990: write d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/f7b [3098738,68786] 0 2024-02-19T19:06:38.902 INFO:tasks.workunit.client.0.smithi088.stdout:4/898: dread d2/d35/f53 [0,4194304] 0 2024-02-19T19:06:38.903 INFO:tasks.workunit.client.0.smithi088.stdout:2/933: dwrite d3/d14/f5a [0,4194304] 0 2024-02-19T19:06:38.904 INFO:tasks.workunit.client.0.smithi088.stdout:2/934: dread - d3/d14/ffe zero size 2024-02-19T19:06:38.905 INFO:tasks.workunit.client.0.smithi088.stdout:8/991: symlink d9/df/d3d/d50/d71/da1/d6c/d9a/l133 0 2024-02-19T19:06:38.905 INFO:tasks.workunit.client.0.smithi088.stdout:4/899: symlink d2/d32/d25/d3f/d70/l116 0 2024-02-19T19:06:38.908 INFO:tasks.workunit.client.0.smithi088.stdout:2/935: symlink d3/d1b/d43/df4/l131 0 2024-02-19T19:06:38.910 INFO:tasks.workunit.client.0.smithi088.stdout:2/936: write d3/d1b/d43/d61/d7e/db6/d120/f115 [2771027,94399] 0 2024-02-19T19:06:38.912 INFO:tasks.workunit.client.0.smithi088.stdout:2/937: symlink d3/d1b/l132 0 2024-02-19T19:06:38.913 INFO:tasks.workunit.client.0.smithi088.stdout:2/938: readlink d3/d1b/d43/d61/d64/l90 0 2024-02-19T19:06:38.916 INFO:tasks.workunit.client.0.smithi088.stdout:2/939: rename d3/d1f/d4f/f17 to d3/d1b/d43/d61/d64/f133 0 2024-02-19T19:06:38.921 INFO:tasks.workunit.client.0.smithi088.stdout:8/992: dwrite d9/dc4/de0/f128 [0,4194304] 0 2024-02-19T19:06:38.922 INFO:tasks.workunit.client.0.smithi088.stdout:4/900: write d2/d32/d25/d26/d6d/d7e/d3a/faa [2197049,122938] 0 2024-02-19T19:06:38.925 INFO:tasks.workunit.client.0.smithi088.stdout:2/940: write d3/f6 [4182650,86402] 0 2024-02-19T19:06:38.927 INFO:tasks.workunit.client.0.smithi088.stdout:2/941: chown d3/d1b/d43/d8f/laf 87362426 1 2024-02-19T19:06:38.929 INFO:tasks.workunit.client.0.smithi088.stdout:2/942: read d3/d1f/d5d/f62 [348466,111678] 0 2024-02-19T19:06:38.929 INFO:tasks.workunit.client.0.smithi088.stdout:4/901: creat d2/d32/d25/d26/d6d/d7e/da3/d9f/f117 x:0 0 0 2024-02-19T19:06:38.931 INFO:tasks.workunit.client.0.smithi088.stdout:2/943: creat d3/d14/f134 x:0 0 0 2024-02-19T19:06:38.934 INFO:tasks.workunit.client.0.smithi088.stdout:4/902: getdents d2/d32/d25/d3f 0 2024-02-19T19:06:38.934 INFO:tasks.workunit.client.0.smithi088.stdout:2/944: truncate d3/d14/f56 8670926 0 2024-02-19T19:06:38.935 INFO:tasks.workunit.client.0.smithi088.stdout:8/993: dwrite d9/df/f55 [0,4194304] 0 2024-02-19T19:06:38.939 INFO:tasks.workunit.client.0.smithi088.stdout:4/903: getdents d2/d35/db3/db7/dfb 0 2024-02-19T19:06:38.941 INFO:tasks.workunit.client.0.smithi088.stdout:8/994: mkdir d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d79/d134 0 2024-02-19T19:06:38.942 INFO:tasks.workunit.client.0.smithi088.stdout:2/945: rename d3/d1f/d113/d117 to d3/d1b/d43/d61/d7e/db6/d135 0 2024-02-19T19:06:38.944 INFO:tasks.workunit.client.0.smithi088.stdout:4/904: rename d2/d32/d25/d26/d4e/d6f/lac to d2/d35/db3/db7/dd7/l118 0 2024-02-19T19:06:38.946 INFO:tasks.workunit.client.0.smithi088.stdout:2/946: getdents d3/d14/dd8/de9 0 2024-02-19T19:06:38.946 INFO:tasks.workunit.client.0.smithi088.stdout:8/995: rename d9/df/l28 to d9/df/d3d/d50/d71/da1/d6c/d9a/d6b/d72/d5d/dbe/l135 0 2024-02-19T19:06:38.947 INFO:tasks.workunit.client.0.smithi088.stdout:2/947: dread - d3/d1b/d43/d61/d7e/db6/d135/f118 zero size 2024-02-19T19:06:38.949 INFO:tasks.workunit.client.0.smithi088.stdout:4/905: rename d2/d32/d25/d26/d6d/d7e/d90 to d2/d32/d25/d26/db8/d119 0 2024-02-19T19:06:38.951 INFO:tasks.workunit.client.0.smithi088.stdout:4/906: truncate d2/d32/d25/d3f/d9a/fc6 519081 0 2024-02-19T19:06:38.951 INFO:tasks.workunit.client.0.smithi088.stdout:2/948: getdents d3/d1b/d43/d61/d7e/db1 0 2024-02-19T19:06:38.952 INFO:tasks.workunit.client.0.smithi088.stdout:4/907: chown d2/d8/c2f 28999315 1 2024-02-19T19:06:38.955 INFO:tasks.workunit.client.0.smithi088.stdout:2/949: rename d3/d1f/d4f/dc6/fe0 to d3/d1f/d4f/f136 0 2024-02-19T19:06:38.959 INFO:tasks.workunit.client.0.smithi088.stdout:2/950: write d3/d1b/d43/d61/d64/f133 [1817195,130284] 0 2024-02-19T19:06:38.960 INFO:tasks.workunit.client.0.smithi088.stdout:4/908: getdents d2/d32/d25/d26/d6d/d7e 0 2024-02-19T19:06:38.961 INFO:tasks.workunit.client.0.smithi088.stdout:8/996: write d9/df/d3d/d50/d71/fd4 [1555069,109833] 0 2024-02-19T19:06:38.962 INFO:tasks.workunit.client.0.smithi088.stdout:2/951: write d3/ff0 [1888722,69762] 0 2024-02-19T19:06:38.962 INFO:tasks.workunit.client.0.smithi088.stdout:4/909: write d2/d32/d9/dd8/de0/fd9 [251213,76022] 0 2024-02-19T19:06:38.966 INFO:tasks.workunit.client.0.smithi088.stdout:2/952: unlink d3/d1f/d5d/f109 0 2024-02-19T19:06:38.968 INFO:tasks.workunit.client.0.smithi088.stdout:2/953: write d3/d1b/f126 [855424,121492] 0 2024-02-19T19:06:38.970 INFO:tasks.workunit.client.0.smithi088.stdout:2/954: write d3/d1f/d5d/f62 [1851772,72667] 0 2024-02-19T19:06:38.972 INFO:tasks.workunit.client.0.smithi088.stdout:4/910: dwrite d2/d32/d25/d26/da5/db2/df3/dff/f10a [0,4194304] 0 2024-02-19T19:06:38.973 INFO:tasks.workunit.client.0.smithi088.stdout:2/955: mkdir d3/d14/d137 0 2024-02-19T19:06:38.976 INFO:tasks.workunit.client.0.smithi088.stdout:2/956: truncate d3/d1b/d43/d61/d7e/db1/d10c/f10d 5543405 0 2024-02-19T19:06:38.978 INFO:tasks.workunit.client.0.smithi088.stdout:2/957: write d3/d1b/d43/d61/d7e/db6/d135/f12c [529893,66523] 0 2024-02-19T19:06:38.979 INFO:tasks.workunit.client.0.smithi088.stdout:4/911: rename d2/d35/db3/db7/dd7/l118 to d2/d32/d25/d26/d58/l11a 0 2024-02-19T19:06:38.979 INFO:tasks.workunit.client.0.smithi088.stdout:8/997: dread d9/f63 [0,4194304] 0 2024-02-19T19:06:38.982 INFO:tasks.workunit.client.0.smithi088.stdout:8/998: creat d9/df/d3d/d50/d71/da1/d6c/d77/d97/dd8/df6/f136 x:0 0 0 2024-02-19T19:06:38.982 INFO:tasks.workunit.client.0.smithi088.stdout:2/958: rename d3/d1f/d5d/de1/fe5 to d3/d1b/d43/d61/d7e/db6/d135/f138 0 2024-02-19T19:06:38.986 INFO:tasks.workunit.client.0.smithi088.stdout:2/959: rename d3/d1b/d43/d61/d7e/db6/d135/d121 to d3/d1b/d43/d61/d7e/db1/d139 0 2024-02-19T19:06:38.987 INFO:tasks.workunit.client.0.smithi088.stdout:8/999: dread d9/df/d3d/d50/d71/da1/d6c/d77/d97/dae/dea/f106 [0,4194304] 0 2024-02-19T19:06:38.988 INFO:tasks.workunit.client.0.smithi088.stdout:2/960: write d3/d1f/f73 [2679375,27510] 0 2024-02-19T19:06:38.990 INFO:tasks.workunit.client.0.smithi088.stdout:2/961: write d3/d1b/fb3 [509371,106864] 0 2024-02-19T19:06:38.992 INFO:tasks.workunit.client.0.smithi088.stdout:2/962: fsync d3/d1f/d5d/de1/ff3 0 2024-02-19T19:06:38.992 INFO:tasks.workunit.client.0.smithi088.stdout:2/963: readlink d3/d1b/d43/d8f/ld3 0 2024-02-19T19:06:38.996 INFO:tasks.workunit.client.0.smithi088.stdout:2/964: creat d3/d1b/d43/d61/d7e/db6/d120/db2/f13a x:0 0 0 2024-02-19T19:06:38.998 INFO:tasks.workunit.client.0.smithi088.stdout:2/965: write d3/d1b/d43/d61/d7e/db1/fbb [1725711,80991] 0 2024-02-19T19:06:39.003 INFO:tasks.workunit.client.0.smithi088.stdout:2/966: truncate d3/fbd 1529205 0 2024-02-19T19:06:39.004 INFO:tasks.workunit.client.0.smithi088.stdout:2/967: chown d3/d1b/d43/df4/le4 11 1 2024-02-19T19:06:39.007 INFO:tasks.workunit.client.0.smithi088.stdout:2/968: getdents d3/d1b/d43/d61/d7e 0 2024-02-19T19:06:39.011 INFO:tasks.workunit.client.0.smithi088.stdout:2/969: dwrite d3/fa5 [0,4194304] 0 2024-02-19T19:06:39.013 INFO:tasks.workunit.client.0.smithi088.stdout:2/970: readlink d3/d1f/d4f/d59/l92 0 2024-02-19T19:06:39.015 INFO:tasks.workunit.client.0.smithi088.stdout:2/971: readlink d3/d1f/l76 0 2024-02-19T19:06:39.015 INFO:tasks.workunit.client.0.smithi088.stdout:2/972: chown d3/d1b/d43/d61/d64/cae 5949 1 2024-02-19T19:06:39.019 INFO:tasks.workunit.client.0.smithi088.stdout:2/973: truncate d3/d1b/d43/d61/d64/f129 887679 0 2024-02-19T19:06:39.019 INFO:tasks.workunit.client.0.smithi088.stdout:2/974: readlink d3/d14/l60 0 2024-02-19T19:06:39.019 INFO:tasks.workunit.client.0.smithi088.stdout:2/975: stat d3/d1b/d43/d74/cf8 0 2024-02-19T19:06:39.022 INFO:tasks.workunit.client.0.smithi088.stdout:2/976: creat d3/d1b/d43/d61/f13b x:0 0 0 2024-02-19T19:06:39.024 INFO:tasks.workunit.client.0.smithi088.stdout:2/977: truncate d3/f85 5354363 0 2024-02-19T19:06:39.044 INFO:tasks.workunit.client.0.smithi088.stdout:2/978: write d3/d1b/d43/d8f/f7b [4419272,32770] 0 2024-02-19T19:06:39.045 INFO:tasks.workunit.client.0.smithi088.stdout:2/979: readlink d3/l13 0 2024-02-19T19:06:39.048 INFO:tasks.workunit.client.0.smithi088.stdout:2/980: dread d3/d1b/d43/d61/d64/f6a [0,4194304] 0 2024-02-19T19:06:39.048 INFO:tasks.workunit.client.0.smithi088.stdout:2/981: chown d3/d1b/fb3 127952 1 2024-02-19T19:06:39.049 INFO:tasks.workunit.client.0.smithi088.stdout:2/982: stat d3/d1b/d43/d61/d64/fa4 0 2024-02-19T19:06:39.050 INFO:tasks.workunit.client.0.smithi088.stdout:2/983: creat d3/d1b/d43/df4/f13c x:0 0 0 2024-02-19T19:06:39.051 INFO:tasks.workunit.client.0.smithi088.stdout:2/984: chown d3/d14/l9f 727 1 2024-02-19T19:06:39.052 INFO:tasks.workunit.client.0.smithi088.stdout:2/985: unlink d3/d1f/d5d/fc5 0 2024-02-19T19:06:39.061 INFO:tasks.workunit.client.0.smithi088.stdout:2/986: dwrite d3/d1f/d5d/f116 [0,4194304] 0 2024-02-19T19:06:39.063 INFO:tasks.workunit.client.0.smithi088.stdout:2/987: chown d3/d1b/l2d 92295 1 2024-02-19T19:06:39.063 INFO:tasks.workunit.client.0.smithi088.stdout:2/988: dread - d3/d1b/d43/d61/d64/fc0 zero size 2024-02-19T19:06:39.064 INFO:tasks.workunit.client.0.smithi088.stdout:2/989: stat d3/d1b/d43/d61/d7e/db1/d10c/f10d 0 2024-02-19T19:06:39.065 INFO:tasks.workunit.client.0.smithi088.stdout:2/990: read - d3/d1b/d43/d61/d7e/db6/f114 zero size 2024-02-19T19:06:39.065 INFO:tasks.workunit.client.0.smithi088.stdout:2/991: stat d3/d1b/lfb 0 2024-02-19T19:06:39.068 INFO:tasks.workunit.client.0.smithi088.stdout:2/992: unlink d3/d1f/l76 0 2024-02-19T19:06:39.071 INFO:tasks.workunit.client.0.smithi088.stdout:2/993: creat d3/f13d x:0 0 0 2024-02-19T19:06:39.075 INFO:tasks.workunit.client.0.smithi088.stdout:2/994: dread d3/d1b/d43/df4/f8b [0,4194304] 0 2024-02-19T19:06:39.076 INFO:tasks.workunit.client.0.smithi088.stdout:2/995: mkdir d3/d1f/d5d/d13e 0 2024-02-19T19:06:39.078 INFO:tasks.workunit.client.0.smithi088.stdout:2/996: symlink d3/d1b/d43/d61/d64/l13f 0 2024-02-19T19:06:39.078 INFO:tasks.workunit.client.0.smithi088.stdout:2/997: chown d3/d1f/d4f/l3d 3858 1 2024-02-19T19:06:39.079 INFO:tasks.workunit.client.0.smithi088.stdout:2/998: fsync d3/d1b/d43/d61/d7e/db1/d139/f122 0 2024-02-19T19:06:39.080 INFO:tasks.workunit.client.0.smithi088.stdout:2/999: creat d3/d1b/d43/d61/d64/f140 x:0 0 0 2024-02-19T19:06:39.193 INFO:tasks.workunit.client.0.smithi088.stdout:4/912: sync 2024-02-19T19:06:39.204 INFO:tasks.workunit.client.0.smithi088.stdout:4/913: dread d2/d32/d25/d26/da5/db2/f10b [0,4194304] 0 2024-02-19T19:06:39.205 INFO:tasks.workunit.client.0.smithi088.stdout:4/914: fdatasync d2/d32/d25/d26/f5e 0 2024-02-19T19:06:39.208 INFO:tasks.workunit.client.0.smithi088.stdout:4/915: mknod d2/d35/db3/db7/dd7/d10d/c11b 0 2024-02-19T19:06:39.208 INFO:tasks.workunit.client.0.smithi088.stdout:4/916: chown d2/d32/d25/d4b/l8e 13 1 2024-02-19T19:06:39.216 INFO:tasks.workunit.client.0.smithi088.stdout:4/917: dwrite d2/f75 [0,4194304] 0 2024-02-19T19:06:39.217 INFO:tasks.workunit.client.0.smithi088.stdout:4/918: chown d2/d32/d25 1879 1 2024-02-19T19:06:39.218 INFO:tasks.workunit.client.0.smithi088.stdout:4/919: write d2/d32/d25/d26/da5/db2/ff7 [396322,87139] 0 2024-02-19T19:06:39.221 INFO:tasks.workunit.client.0.smithi088.stdout:4/920: symlink d2/d32/d25/d3f/d70/l11c 0 2024-02-19T19:06:39.235 INFO:tasks.workunit.client.0.smithi088.stdout:4/921: dwrite d2/d32/d25/d26/d6d/d7e/d3a/d77/fcf [0,4194304] 0 2024-02-19T19:06:39.236 INFO:tasks.workunit.client.0.smithi088.stdout:4/922: write d2/d32/d25/f38 [2990625,83272] 0 2024-02-19T19:06:39.243 INFO:tasks.workunit.client.0.smithi088.stdout:4/923: dwrite d2/d32/d25/d26/da5/db2/ff1 [0,4194304] 0 2024-02-19T19:06:39.244 INFO:tasks.workunit.client.0.smithi088.stdout:4/924: chown d2/d32/d25/d3f/c107 5543 1 2024-02-19T19:06:39.247 INFO:tasks.workunit.client.0.smithi088.stdout:4/925: mkdir d2/d35/db3/db7/de3/d11d 0 2024-02-19T19:06:39.250 INFO:tasks.workunit.client.0.smithi088.stdout:4/926: creat d2/d35/db3/f11e x:0 0 0 2024-02-19T19:06:39.251 INFO:tasks.workunit.client.0.smithi088.stdout:4/927: chown d2/d32/d25/d26/db8/l10f 31514 1 2024-02-19T19:06:39.251 INFO:tasks.workunit.client.0.smithi088.stdout:4/928: chown d2/d35 589059208 1 2024-02-19T19:06:39.253 INFO:tasks.workunit.client.0.smithi088.stdout:4/929: mkdir d2/d35/db3/db7/dd7/d11f 0 2024-02-19T19:06:39.261 INFO:tasks.workunit.client.0.smithi088.stdout:4/930: dwrite d2/d35/f53 [0,4194304] 0 2024-02-19T19:06:39.263 INFO:tasks.workunit.client.0.smithi088.stdout:4/931: dread - d2/d32/d25/d26/d4e/d6f/fa6 zero size 2024-02-19T19:06:39.265 INFO:tasks.workunit.client.0.smithi088.stdout:4/932: rmdir d2/d32/d25/d45 39 2024-02-19T19:06:39.266 INFO:tasks.workunit.client.0.smithi088.stdout:4/933: mkdir d2/d32/d9/de4/d120 0 2024-02-19T19:06:39.267 INFO:tasks.workunit.client.0.smithi088.stdout:4/934: dread - d2/d32/d25/d3f/d70/fc8 zero size 2024-02-19T19:06:39.292 INFO:tasks.workunit.client.0.smithi088.stdout:4/935: dread d2/d32/d25/d26/d6d/d7e/d3a/f5f [0,4194304] 0 2024-02-19T19:06:39.294 INFO:tasks.workunit.client.0.smithi088.stdout:4/936: symlink d2/d32/d25/l121 0 2024-02-19T19:06:39.296 INFO:tasks.workunit.client.0.smithi088.stdout:4/937: symlink d2/d32/d25/d26/d6d/d7e/l122 0 2024-02-19T19:06:39.297 INFO:tasks.workunit.client.0.smithi088.stdout:4/938: truncate d2/d32/d25/d26/da5/db2/ff1 4827634 0 2024-02-19T19:06:39.299 INFO:tasks.workunit.client.0.smithi088.stdout:4/939: symlink d2/d32/d25/d45/l123 0 2024-02-19T19:06:39.300 INFO:tasks.workunit.client.0.smithi088.stdout:4/940: truncate d2/d35/db3/fe2 861505 0 2024-02-19T19:06:39.303 INFO:tasks.workunit.client.0.smithi088.stdout:4/941: truncate d2/d32/d25/d26/d6d/d7e/f6c 4521335 0 2024-02-19T19:06:39.304 INFO:tasks.workunit.client.0.smithi088.stdout:4/942: fsync d2/d32/d25/f46 0 2024-02-19T19:06:39.308 INFO:tasks.workunit.client.0.smithi088.stdout:4/943: rename d2/d32/d25/lb1 to d2/d8/l124 0 2024-02-19T19:06:39.311 INFO:tasks.workunit.client.0.smithi088.stdout:4/944: unlink d2/d32/d25/f97 0 2024-02-19T19:06:39.315 INFO:tasks.workunit.client.0.smithi088.stdout:4/945: dread d2/d32/d25/d3f/d70/fe8 [0,4194304] 0 2024-02-19T19:06:39.316 INFO:tasks.workunit.client.0.smithi088.stdout:4/946: mknod d2/d32/d9/de4/c125 0 2024-02-19T19:06:39.317 INFO:tasks.workunit.client.0.smithi088.stdout:4/947: chown d2/d32/d25/d26/db8 106406116 1 2024-02-19T19:06:39.323 INFO:tasks.workunit.client.0.smithi088.stdout:4/948: write d2/d32/d25/d26/d6d/d7e/da3/d9f/fba [1018529,49582] 0 2024-02-19T19:06:39.324 INFO:tasks.workunit.client.0.smithi088.stdout:4/949: creat d2/d35/db3/f126 x:0 0 0 2024-02-19T19:06:39.327 INFO:tasks.workunit.client.0.smithi088.stdout:4/950: dread d2/d35/f4f [0,4194304] 0 2024-02-19T19:06:39.329 INFO:tasks.workunit.client.0.smithi088.stdout:4/951: rmdir d2/d32/d25/d26/d6d/d7e/da3 39 2024-02-19T19:06:39.332 INFO:tasks.workunit.client.0.smithi088.stdout:4/952: write d2/d35/f6e [291554,107529] 0 2024-02-19T19:06:39.333 INFO:tasks.workunit.client.0.smithi088.stdout:4/953: dread - d2/d35/db3/f126 zero size 2024-02-19T19:06:39.335 INFO:tasks.workunit.client.0.smithi088.stdout:4/954: creat d2/d35/db3/db7/de3/d11d/f127 x:0 0 0 2024-02-19T19:06:39.336 INFO:tasks.workunit.client.0.smithi088.stdout:4/955: write d2/d32/d25/d26/db8/f106 [1015369,30735] 0 2024-02-19T19:06:39.338 INFO:tasks.workunit.client.0.smithi088.stdout:4/956: mknod d2/d32/d25/d26/d4e/d6f/c128 0 2024-02-19T19:06:39.344 INFO:tasks.workunit.client.0.smithi088.stdout:4/957: dwrite d2/d32/d25/d45/fb0 [0,4194304] 0 2024-02-19T19:06:39.502 INFO:tasks.workunit.client.0.smithi088.stdout:4/958: sync 2024-02-19T19:06:39.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:39 smithi132 ceph-mon[70676]: pgmap v53: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 60 MiB/s rd, 131 MiB/s wr, 273 op/s 2024-02-19T19:06:39.507 INFO:tasks.workunit.client.0.smithi088.stdout:4/959: dwrite d2/d8/f14 [0,4194304] 0 2024-02-19T19:06:39.510 INFO:tasks.workunit.client.0.smithi088.stdout:4/960: unlink d2/d32/d25/d26/db8/d119/f94 0 2024-02-19T19:06:39.513 INFO:tasks.workunit.client.0.smithi088.stdout:4/961: creat d2/d32/d25/d26/d58/d113/f129 x:0 0 0 2024-02-19T19:06:39.518 INFO:tasks.workunit.client.0.smithi088.stdout:4/962: dwrite d2/d32/d25/d26/db8/f106 [0,4194304] 0 2024-02-19T19:06:39.522 INFO:tasks.workunit.client.0.smithi088.stdout:4/963: rename d2/d32/d25/d26/d58/l11a to d2/d32/d25/d26/d6d/l12a 0 2024-02-19T19:06:39.532 INFO:tasks.workunit.client.0.smithi088.stdout:4/964: dwrite d2/d32/d9/f20 [0,4194304] 0 2024-02-19T19:06:39.535 INFO:tasks.workunit.client.0.smithi088.stdout:4/965: rename d2/d32/d25/d4b/l8e to d2/d35/db3/db7/dd7/d11f/l12b 0 2024-02-19T19:06:39.536 INFO:tasks.workunit.client.0.smithi088.stdout:4/966: truncate d2/d32/d9/fd6 452473 0 2024-02-19T19:06:39.540 INFO:tasks.workunit.client.0.smithi088.stdout:4/967: getdents d2/d32/d25/d26/d58/d113 0 2024-02-19T19:06:39.549 INFO:tasks.workunit.client.0.smithi088.stdout:4/968: write d2/d32/d25/d3f/d70/fe8 [1843564,19003] 0 2024-02-19T19:06:39.549 INFO:tasks.workunit.client.0.smithi088.stdout:4/969: fsync d2/d35/f53 0 2024-02-19T19:06:39.554 INFO:tasks.workunit.client.0.smithi088.stdout:4/970: truncate d2/d32/d25/d3f/d70/f8b 3407190 0 2024-02-19T19:06:39.555 INFO:tasks.workunit.client.0.smithi088.stdout:4/971: readlink d2/d32/d25/d26/d6d/d7e/d3a/l3c 0 2024-02-19T19:06:39.556 INFO:tasks.workunit.client.0.smithi088.stdout:4/972: stat d2/d32/d25/d26/d4e/d6f/l87 0 2024-02-19T19:06:39.558 INFO:tasks.workunit.client.0.smithi088.stdout:4/973: creat d2/d32/d25/d26/dfa/f12c x:0 0 0 2024-02-19T19:06:39.566 INFO:tasks.workunit.client.0.smithi088.stdout:4/974: dwrite d2/d32/d25/d26/f88 [0,4194304] 0 2024-02-19T19:06:39.571 INFO:tasks.workunit.client.0.smithi088.stdout:4/975: mknod d2/d32/d25/d3f/d70/c12d 0 2024-02-19T19:06:39.588 INFO:tasks.workunit.client.0.smithi088.stdout:4/976: write d2/d32/d25/d26/db8/d119/fa7 [1654357,26302] 0 2024-02-19T19:06:39.589 INFO:tasks.workunit.client.0.smithi088.stdout:4/977: creat d2/d35/db3/db7/de3/d11d/f12e x:0 0 0 2024-02-19T19:06:39.591 INFO:tasks.workunit.client.0.smithi088.stdout:4/978: rmdir d2/d32/d25/d26/da5 39 2024-02-19T19:06:39.591 INFO:tasks.workunit.client.0.smithi088.stdout:4/979: chown d2/d32/d25/d26/d4e/d79 120582 1 2024-02-19T19:06:39.596 INFO:tasks.workunit.client.0.smithi088.stdout:4/980: dwrite d2/d8/f29 [0,4194304] 0 2024-02-19T19:06:39.597 INFO:tasks.workunit.client.0.smithi088.stdout:4/981: chown d2/d32/d9/de4/c125 254 1 2024-02-19T19:06:39.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:39 smithi088 ceph-mon[88643]: pgmap v53: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 60 MiB/s rd, 131 MiB/s wr, 273 op/s 2024-02-19T19:06:39.600 INFO:tasks.workunit.client.0.smithi088.stdout:4/982: rename d2/d8/l124 to d2/d8/l12f 0 2024-02-19T19:06:39.601 INFO:tasks.workunit.client.0.smithi088.stdout:4/983: fsync d2/d32/d25/d26/d6d/d7e/f72 0 2024-02-19T19:06:39.603 INFO:tasks.workunit.client.0.smithi088.stdout:4/984: truncate d2/d32/d25/d3f/d9a/ff6 254052 0 2024-02-19T19:06:39.603 INFO:tasks.workunit.client.0.smithi088.stdout:4/985: stat d2/d32/cdd 0 2024-02-19T19:06:39.605 INFO:tasks.workunit.client.0.smithi088.stdout:4/986: unlink d2/d32/d25/d26/d6d/d7e/d3a/c7b 0 2024-02-19T19:06:39.610 INFO:tasks.workunit.client.0.smithi088.stdout:4/987: dwrite d2/d32/d25/f41 [0,4194304] 0 2024-02-19T19:06:39.612 INFO:tasks.workunit.client.0.smithi088.stdout:4/988: dread - d2/d32/d25/d3f/fe1 zero size 2024-02-19T19:06:39.625 INFO:tasks.workunit.client.0.smithi088.stdout:4/989: write d2/d32/f1c [8455747,125871] 0 2024-02-19T19:06:39.625 INFO:tasks.workunit.client.0.smithi088.stdout:4/990: stat d2/d32/d25/d26/db8/d119 0 2024-02-19T19:06:39.627 INFO:tasks.workunit.client.0.smithi088.stdout:4/991: truncate d2/d32/d25/d26/d4e/d6f/f9d 4265052 0 2024-02-19T19:06:39.630 INFO:tasks.workunit.client.0.smithi088.stdout:4/992: rename d2/d32/d25/d45 to d2/d35/db3/db7/dd7/d130 0 2024-02-19T19:06:39.630 INFO:tasks.workunit.client.0.smithi088.stdout:4/993: chown d2/d32/d25/d26/d4e/d79/cdf 8191 1 2024-02-19T19:06:39.632 INFO:tasks.workunit.client.0.smithi088.stdout:4/994: fdatasync d2/d32/d25/f46 0 2024-02-19T19:06:39.634 INFO:tasks.workunit.client.0.smithi088.stdout:4/995: mknod d2/d32/d25/d26/d6d/c131 0 2024-02-19T19:06:39.641 INFO:tasks.workunit.client.0.smithi088.stdout:4/996: dwrite d2/d32/d25/d26/da5/db2/ff1 [4194304,4194304] 0 2024-02-19T19:06:39.644 INFO:tasks.workunit.client.0.smithi088.stdout:4/997: chown d2/d32/d25/d26/d4e/d79/l93 3 1 2024-02-19T19:06:39.645 INFO:tasks.workunit.client.0.smithi088.stdout:4/998: symlink d2/d32/d25/d4b/l132 0 2024-02-19T19:06:39.650 INFO:tasks.workunit.client.0.smithi088.stdout:4/999: dwrite d2/d32/d25/d3f/d9a/fb5 [0,4194304] 0 2024-02-19T19:06:39.661 INFO:tasks.workunit.client.0.smithi088.stderr:+ rm -rf -- ./tmp.eME568FDig 2024-02-19T19:06:41.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:41 smithi088 ceph-mon[88643]: pgmap v54: 65 pgs: 65 active+clean; 2.3 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 81 MiB/s rd, 168 MiB/s wr, 377 op/s 2024-02-19T19:06:41.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:41 smithi132 ceph-mon[70676]: pgmap v54: 65 pgs: 65 active+clean; 2.3 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 81 MiB/s rd, 168 MiB/s wr, 377 op/s 2024-02-19T19:06:43.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:43 smithi132 ceph-mon[70676]: pgmap v55: 65 pgs: 65 active+clean; 2.3 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 49 MiB/s rd, 106 MiB/s wr, 238 op/s 2024-02-19T19:06:43.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:43.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:43.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:43 smithi088 ceph-mon[88643]: pgmap v55: 65 pgs: 65 active+clean; 2.3 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 49 MiB/s rd, 106 MiB/s wr, 238 op/s 2024-02-19T19:06:43.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:43.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:44.502 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:06:44.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:06:45.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:45 smithi132 ceph-mon[70676]: pgmap v56: 65 pgs: 65 active+clean; 2.3 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 49 MiB/s rd, 106 MiB/s wr, 238 op/s 2024-02-19T19:06:45.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:45 smithi088 ceph-mon[88643]: pgmap v56: 65 pgs: 65 active+clean; 2.3 GiB data, 7.1 GiB used, 529 GiB / 536 GiB avail; 49 MiB/s rd, 106 MiB/s wr, 238 op/s 2024-02-19T19:06:47.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:47 smithi132 ceph-mon[70676]: pgmap v57: 65 pgs: 65 active+clean; 1.9 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 50 MiB/s rd, 110 MiB/s wr, 286 op/s 2024-02-19T19:06:47.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:47.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi132", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T19:06:47.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:47.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:47 smithi088 ceph-mon[88643]: pgmap v57: 65 pgs: 65 active+clean; 1.9 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 50 MiB/s rd, 110 MiB/s wr, 286 op/s 2024-02-19T19:06:47.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:47.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi132", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-02-19T19:06:47.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:06:48.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:48 smithi132 ceph-mon[70676]: Upgrade: Updating crash.smithi132 (2/2) 2024-02-19T19:06:48.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:48 smithi132 ceph-mon[70676]: Deploying daemon crash.smithi132 on smithi132 2024-02-19T19:06:48.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:48 smithi088 ceph-mon[88643]: Upgrade: Updating crash.smithi132 (2/2) 2024-02-19T19:06:48.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:48 smithi088 ceph-mon[88643]: Deploying daemon crash.smithi132 on smithi132 2024-02-19T19:06:49.409 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:49 smithi132 ceph-mon[70676]: pgmap v58: 65 pgs: 65 active+clean; 1.9 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 22 MiB/s rd, 42 MiB/s wr, 152 op/s 2024-02-19T19:06:49.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:49 smithi088 ceph-mon[88643]: pgmap v58: 65 pgs: 65 active+clean; 1.9 GiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 22 MiB/s rd, 42 MiB/s wr, 152 op/s 2024-02-19T19:06:51.173 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-19T19:06:51.173 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-19T19:06:51.252 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:51 smithi132 ceph-mon[70676]: pgmap v59: 65 pgs: 65 active+clean; 935 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 22 MiB/s rd, 42 MiB/s wr, 226 op/s 2024-02-19T19:06:51.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:51 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:51.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:51 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:51.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:51 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:06:51.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.261+0000 7f01567ad700 1 -- 172.21.15.88:0/2624531619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0150104130 msgr2=0x7f0150104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:51.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.261+0000 7f01567ad700 1 --2- 172.21.15.88:0/2624531619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0150104130 0x7f0150104500 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0140009940 tx=0x7f0140009c50 comp rx=0 tx=0).stop 2024-02-19T19:06:51.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.261+0000 7f01567ad700 1 -- 172.21.15.88:0/2624531619 shutdown_connections 2024-02-19T19:06:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.261+0000 7f01567ad700 1 --2- 172.21.15.88:0/2624531619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01500ffbd0 0x7f0150100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.261+0000 7f01567ad700 1 --2- 172.21.15.88:0/2624531619 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0150104130 0x7f0150104500 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.261+0000 7f01567ad700 1 -- 172.21.15.88:0/2624531619 >> 172.21.15.88:0/2624531619 conn(0x7f0150075130 msgr2=0x7f0150075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.261+0000 7f01567ad700 1 -- 172.21.15.88:0/2624531619 shutdown_connections 2024-02-19T19:06:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.261+0000 7f01567ad700 1 -- 172.21.15.88:0/2624531619 wait complete. 2024-02-19T19:06:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.262+0000 7f01567ad700 1 Processor -- start 2024-02-19T19:06:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.262+0000 7f01567ad700 1 -- start start 2024-02-19T19:06:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.262+0000 7f01567ad700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01500ffbd0 0x7f0150196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.262+0000 7f01567ad700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0150104130 0x7f01501971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:51.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.262+0000 7f01567ad700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0150197810 con 0x7f01500ffbd0 2024-02-19T19:06:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.262+0000 7f01567ad700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0150197950 con 0x7f0150104130 2024-02-19T19:06:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.262+0000 7f014ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01500ffbd0 0x7f0150196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.262+0000 7f014ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01500ffbd0 0x7f0150196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48676/0 (socket says 172.21.15.88:48676) 2024-02-19T19:06:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.262+0000 7f014ffff700 1 -- 172.21.15.88:0/588171392 learned_addr learned my addr 172.21.15.88:0/588171392 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.262+0000 7f014f7fe700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0150104130 0x7f01501971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.263+0000 7f014ffff700 1 -- 172.21.15.88:0/588171392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0150104130 msgr2=0x7f01501971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.263+0000 7f014ffff700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0150104130 0x7f01501971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.263+0000 7f014ffff700 1 -- 172.21.15.88:0/588171392 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01400095f0 con 0x7f01500ffbd0 2024-02-19T19:06:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.263+0000 7f014f7fe700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0150104130 0x7f01501971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:06:51.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.263+0000 7f014ffff700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01500ffbd0 0x7f0150196cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f01400055f0 tx=0x7f0140005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:51.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.263+0000 7f014d7fa700 1 -- 172.21.15.88:0/588171392 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f014001d070 con 0x7f01500ffbd0 2024-02-19T19:06:51.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.263+0000 7f01567ad700 1 -- 172.21.15.88:0/588171392 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015019fd20 con 0x7f01500ffbd0 2024-02-19T19:06:51.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.263+0000 7f014d7fa700 1 -- 172.21.15.88:0/588171392 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0140005940 con 0x7f01500ffbd0 2024-02-19T19:06:51.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.263+0000 7f014d7fa700 1 -- 172.21.15.88:0/588171392 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f014001d070 con 0x7f01500ffbd0 2024-02-19T19:06:51.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.263+0000 7f01567ad700 1 -- 172.21.15.88:0/588171392 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01501a0130 con 0x7f01500ffbd0 2024-02-19T19:06:51.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.264+0000 7f01567ad700 1 -- 172.21.15.88:0/588171392 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01501076a0 con 0x7f01500ffbd0 2024-02-19T19:06:51.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.265+0000 7f014d7fa700 1 -- 172.21.15.88:0/588171392 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f014000f680 con 0x7f01500ffbd0 2024-02-19T19:06:51.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.266+0000 7f014d7fa700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f013806d960 0x7f013806fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:51.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.266+0000 7f014d7fa700 1 -- 172.21.15.88:0/588171392 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f014008e3d0 con 0x7f01500ffbd0 2024-02-19T19:06:51.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.266+0000 7f014f7fe700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f013806d960 0x7f013806fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:51.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.266+0000 7f014f7fe700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f013806d960 0x7f013806fe10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0150068df0 tx=0x7f014400b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:51.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.271+0000 7f014d7fa700 1 -- 172.21.15.88:0/588171392 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0140057cf0 con 0x7f01500ffbd0 2024-02-19T19:06:51.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.520+0000 7f01567ad700 1 -- 172.21.15.88:0/588171392 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0150061190 con 0x7f013806d960 2024-02-19T19:06:51.541 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:51 smithi088 ceph-mon[88643]: pgmap v59: 65 pgs: 65 active+clean; 935 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 22 MiB/s rd, 42 MiB/s wr, 226 op/s 2024-02-19T19:06:51.541 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:51 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:51.541 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:51 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:51.541 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:51 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:06:51.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.521+0000 7f014d7fa700 1 -- 172.21.15.88:0/588171392 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f0150061190 con 0x7f013806d960 2024-02-19T19:06:51.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.524+0000 7f01567ad700 1 -- 172.21.15.88:0/588171392 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f013806d960 msgr2=0x7f013806fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:51.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.524+0000 7f01567ad700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f013806d960 0x7f013806fe10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0150068df0 tx=0x7f014400b410 comp rx=0 tx=0).stop 2024-02-19T19:06:51.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.524+0000 7f01567ad700 1 -- 172.21.15.88:0/588171392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01500ffbd0 msgr2=0x7f0150196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:51.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.524+0000 7f01567ad700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01500ffbd0 0x7f0150196cb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f01400055f0 tx=0x7f0140005620 comp rx=0 tx=0).stop 2024-02-19T19:06:51.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.525+0000 7f01567ad700 1 -- 172.21.15.88:0/588171392 shutdown_connections 2024-02-19T19:06:51.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.525+0000 7f01567ad700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f013806d960 0x7f013806fe10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.525+0000 7f01567ad700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01500ffbd0 0x7f0150196cb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.525+0000 7f01567ad700 1 --2- 172.21.15.88:0/588171392 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0150104130 0x7f01501971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.525+0000 7f01567ad700 1 -- 172.21.15.88:0/588171392 >> 172.21.15.88:0/588171392 conn(0x7f0150075130 msgr2=0x7f01500fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:51.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.525+0000 7f01567ad700 1 -- 172.21.15.88:0/588171392 shutdown_connections 2024-02-19T19:06:51.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.525+0000 7f01567ad700 1 -- 172.21.15.88:0/588171392 wait complete. 2024-02-19T19:06:51.557 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:06:51.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.645+0000 7f5170309700 1 -- 172.21.15.88:0/2047520846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516806dc80 msgr2=0x7f5168107d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:51.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.645+0000 7f5170309700 1 --2- 172.21.15.88:0/2047520846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516806dc80 0x7f5168107d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5160009940 tx=0x7f5160009c50 comp rx=0 tx=0).stop 2024-02-19T19:06:51.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.645+0000 7f5170309700 1 -- 172.21.15.88:0/2047520846 shutdown_connections 2024-02-19T19:06:51.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.645+0000 7f5170309700 1 --2- 172.21.15.88:0/2047520846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5168108290 0x7f5168108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.645+0000 7f5170309700 1 --2- 172.21.15.88:0/2047520846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f516806dc80 0x7f5168107d50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.645+0000 7f5170309700 1 -- 172.21.15.88:0/2047520846 >> 172.21.15.88:0/2047520846 conn(0x7f516806ca90 msgr2=0x7f516806ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:51.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.645+0000 7f5170309700 1 -- 172.21.15.88:0/2047520846 shutdown_connections 2024-02-19T19:06:51.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.645+0000 7f5170309700 1 -- 172.21.15.88:0/2047520846 wait complete. 2024-02-19T19:06:51.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.646+0000 7f5170309700 1 Processor -- start 2024-02-19T19:06:51.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.646+0000 7f5170309700 1 -- start start 2024-02-19T19:06:51.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.646+0000 7f5170309700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f516806dc80 0x7f51681a6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:51.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.646+0000 7f5170309700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168108290 0x7f51681a6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:51.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.646+0000 7f5170309700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51681a7150 con 0x7f5168108290 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.646+0000 7f5170309700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51681a0600 con 0x7f516806dc80 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.646+0000 7f516d8a4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168108290 0x7f51681a6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.646+0000 7f516d8a4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168108290 0x7f51681a6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48708/0 (socket says 172.21.15.88:48708) 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.646+0000 7f516d8a4700 1 -- 172.21.15.88:0/2167337313 learned_addr learned my addr 172.21.15.88:0/2167337313 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.646+0000 7f516e0a5700 1 --2- 172.21.15.88:0/2167337313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f516806dc80 0x7f51681a6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.647+0000 7f516d8a4700 1 -- 172.21.15.88:0/2167337313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f516806dc80 msgr2=0x7f51681a6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.647+0000 7f516d8a4700 1 --2- 172.21.15.88:0/2167337313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f516806dc80 0x7f51681a6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.647+0000 7f516d8a4700 1 -- 172.21.15.88:0/2167337313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51600095f0 con 0x7f5168108290 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.647+0000 7f516d8a4700 1 --2- 172.21.15.88:0/2167337313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168108290 0x7f51681a6ac0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f515c00ef10 tx=0x7f515c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.647+0000 7f51677fe700 1 -- 172.21.15.88:0/2167337313 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f515c00c9f0 con 0x7f5168108290 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.647+0000 7f51677fe700 1 -- 172.21.15.88:0/2167337313 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f515c00f040 con 0x7f5168108290 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.647+0000 7f51677fe700 1 -- 172.21.15.88:0/2167337313 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f515c0146b0 con 0x7f5168108290 2024-02-19T19:06:51.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.647+0000 7f5170309700 1 -- 172.21.15.88:0/2167337313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51681a0880 con 0x7f5168108290 2024-02-19T19:06:51.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.647+0000 7f5170309700 1 -- 172.21.15.88:0/2167337313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51681a0dd0 con 0x7f5168108290 2024-02-19T19:06:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.649+0000 7f5170309700 1 -- 172.21.15.88:0/2167337313 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516804f2a0 con 0x7f5168108290 2024-02-19T19:06:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.649+0000 7f51677fe700 1 -- 172.21.15.88:0/2167337313 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f515c014920 con 0x7f5168108290 2024-02-19T19:06:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.650+0000 7f51677fe700 1 --2- 172.21.15.88:0/2167337313 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f515006d8a0 0x7f515006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.650+0000 7f51677fe700 1 -- 172.21.15.88:0/2167337313 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f515c08d720 con 0x7f5168108290 2024-02-19T19:06:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.650+0000 7f516e0a5700 1 --2- 172.21.15.88:0/2167337313 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f515006d8a0 0x7f515006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:51.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.650+0000 7f516e0a5700 1 --2- 172.21.15.88:0/2167337313 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f515006d8a0 0x7f515006fd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5160006010 tx=0x7f5160005ad0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:51.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.655+0000 7f51677fe700 1 -- 172.21.15.88:0/2167337313 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f515c05ac80 con 0x7f5168108290 2024-02-19T19:06:51.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.935+0000 7f5170309700 1 -- 172.21.15.88:0/2167337313 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f51681a1120 con 0x7f515006d8a0 2024-02-19T19:06:51.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.937+0000 7f51677fe700 1 -- 172.21.15.88:0/2167337313 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f51681a1120 con 0x7f515006d8a0 2024-02-19T19:06:51.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.939+0000 7f5170309700 1 -- 172.21.15.88:0/2167337313 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f515006d8a0 msgr2=0x7f515006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:51.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.939+0000 7f5170309700 1 --2- 172.21.15.88:0/2167337313 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f515006d8a0 0x7f515006fd50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5160006010 tx=0x7f5160005ad0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.939+0000 7f5170309700 1 -- 172.21.15.88:0/2167337313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168108290 msgr2=0x7f51681a6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:51.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.939+0000 7f5170309700 1 --2- 172.21.15.88:0/2167337313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168108290 0x7f51681a6ac0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f515c00ef10 tx=0x7f515c00c3a0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.939+0000 7f5170309700 1 -- 172.21.15.88:0/2167337313 shutdown_connections 2024-02-19T19:06:51.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.939+0000 7f5170309700 1 --2- 172.21.15.88:0/2167337313 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f515006d8a0 0x7f515006fd50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.939+0000 7f5170309700 1 --2- 172.21.15.88:0/2167337313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f516806dc80 0x7f51681a6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.939+0000 7f5170309700 1 --2- 172.21.15.88:0/2167337313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168108290 0x7f51681a6ac0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:51.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.939+0000 7f5170309700 1 -- 172.21.15.88:0/2167337313 >> 172.21.15.88:0/2167337313 conn(0x7f516806ca90 msgr2=0x7f5168070820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:51.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.939+0000 7f5170309700 1 -- 172.21.15.88:0/2167337313 shutdown_connections 2024-02-19T19:06:51.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:51.940+0000 7f5170309700 1 -- 172.21.15.88:0/2167337313 wait complete. 2024-02-19T19:06:52.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.052+0000 7f2570674700 1 -- 172.21.15.88:0/2794934525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2568104130 msgr2=0x7f2568104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:52.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.052+0000 7f2570674700 1 --2- 172.21.15.88:0/2794934525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2568104130 0x7f2568104500 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2564009910 tx=0x7f2564009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:52.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.052+0000 7f2570674700 1 -- 172.21.15.88:0/2794934525 shutdown_connections 2024-02-19T19:06:52.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.052+0000 7f2570674700 1 --2- 172.21.15.88:0/2794934525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25680ffbd0 0x7f2568100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.052+0000 7f2570674700 1 --2- 172.21.15.88:0/2794934525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2568104130 0x7f2568104500 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.052+0000 7f2570674700 1 -- 172.21.15.88:0/2794934525 >> 172.21.15.88:0/2794934525 conn(0x7f2568075130 msgr2=0x7f2568075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.052+0000 7f2570674700 1 -- 172.21.15.88:0/2794934525 shutdown_connections 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.052+0000 7f2570674700 1 -- 172.21.15.88:0/2794934525 wait complete. 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.052+0000 7f2570674700 1 Processor -- start 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.052+0000 7f2570674700 1 -- start start 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f2570674700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25680ffbd0 0x7f2568104fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f2570674700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2568104130 0x7f2568105500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f2570674700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2568105a40 con 0x7f25680ffbd0 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f2570674700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2568105b80 con 0x7f2568104130 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f256e410700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25680ffbd0 0x7f2568104fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f256e410700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25680ffbd0 0x7f2568104fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48720/0 (socket says 172.21.15.88:48720) 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f256e410700 1 -- 172.21.15.88:0/4219780906 learned_addr learned my addr 172.21.15.88:0/4219780906 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f256dc0f700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2568104130 0x7f2568105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f256e410700 1 -- 172.21.15.88:0/4219780906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2568104130 msgr2=0x7f2568105500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:52.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f256e410700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2568104130 0x7f2568105500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f256e410700 1 -- 172.21.15.88:0/4219780906 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25640095f0 con 0x7f25680ffbd0 2024-02-19T19:06:52.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f256dc0f700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2568104130 0x7f2568105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:06:52.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f256e410700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25680ffbd0 0x7f2568104fc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f256400b7d0 tx=0x7f256400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:52.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.053+0000 7f255f7fe700 1 -- 172.21.15.88:0/4219780906 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256401d070 con 0x7f25680ffbd0 2024-02-19T19:06:52.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.054+0000 7f255f7fe700 1 -- 172.21.15.88:0/4219780906 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25640052a0 con 0x7f25680ffbd0 2024-02-19T19:06:52.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.054+0000 7f2570674700 1 -- 172.21.15.88:0/4219780906 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2568105e00 con 0x7f25680ffbd0 2024-02-19T19:06:52.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.054+0000 7f255f7fe700 1 -- 172.21.15.88:0/4219780906 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2564022470 con 0x7f25680ffbd0 2024-02-19T19:06:52.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.054+0000 7f2570674700 1 -- 172.21.15.88:0/4219780906 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25681a2330 con 0x7f25680ffbd0 2024-02-19T19:06:52.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.055+0000 7f2570674700 1 -- 172.21.15.88:0/4219780906 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256804f2a0 con 0x7f25680ffbd0 2024-02-19T19:06:52.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.055+0000 7f255f7fe700 1 -- 172.21.15.88:0/4219780906 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f256400f850 con 0x7f25680ffbd0 2024-02-19T19:06:52.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.056+0000 7f255f7fe700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255406d8a0 0x7f255406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:52.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.056+0000 7f255f7fe700 1 -- 172.21.15.88:0/4219780906 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f256408e940 con 0x7f25680ffbd0 2024-02-19T19:06:52.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.056+0000 7f256dc0f700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255406d8a0 0x7f255406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:52.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.056+0000 7f256dc0f700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255406d8a0 0x7f255406fd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f25681067b0 tx=0x7f2558009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:52.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.061+0000 7f255f7fe700 1 -- 172.21.15.88:0/4219780906 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f25640581a0 con 0x7f25680ffbd0 2024-02-19T19:06:52.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.339+0000 7f2570674700 1 -- 172.21.15.88:0/4219780906 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2568068cf0 con 0x7f255406d8a0 2024-02-19T19:06:52.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.358+0000 7f255f7fe700 1 -- 172.21.15.88:0/4219780906 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2568068cf0 con 0x7f255406d8a0 2024-02-19T19:06:52.379 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:06:52.379 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (4m) 97s ago 14m 25.0M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:06:52.379 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (14m) 97s ago 14m 7960k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:06:52.379 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (13m) 74s ago 13m 7902k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:06:52.379 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 starting - - - - 2024-02-19T19:06:52.379 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 starting - - - - 2024-02-19T19:06:52.379 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (3m) 97s ago 13m 83.1M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:06:52.379 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (8m) 97s ago 8m 15.3M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:06:52.379 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (8m) 97s ago 8m 107M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (8m) 74s ago 8m 119M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (8m) 74s ago 8m 19.6M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (5m) 97s ago 16m 476M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (4m) 74s ago 12m 448M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (2m) 97s ago 16m 32.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (81s) 74s ago 12m 26.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (4m) 97s ago 14m 15.8M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (4m) 74s ago 12m 15.2M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (11m) 97s ago 11m 176M 1216M 18.2.0 dc2bc1663786 0cd963a2a48a 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (11m) 97s ago 11m 181M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (11m) 97s ago 11m 147M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (10m) 74s ago 10m 164M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (10m) 74s ago 10m 149M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:06:52.380 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (9m) 74s ago 9m 144M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:06:52.381 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (4m) 97s ago 13m 64.8M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:06:52.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.362+0000 7f2570674700 1 -- 172.21.15.88:0/4219780906 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255406d8a0 msgr2=0x7f255406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:52.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.362+0000 7f2570674700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255406d8a0 0x7f255406fd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f25681067b0 tx=0x7f2558009260 comp rx=0 tx=0).stop 2024-02-19T19:06:52.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.362+0000 7f2570674700 1 -- 172.21.15.88:0/4219780906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25680ffbd0 msgr2=0x7f2568104fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:52.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.362+0000 7f2570674700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25680ffbd0 0x7f2568104fc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f256400b7d0 tx=0x7f256400bf90 comp rx=0 tx=0).stop 2024-02-19T19:06:52.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.362+0000 7f2570674700 1 -- 172.21.15.88:0/4219780906 shutdown_connections 2024-02-19T19:06:52.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.362+0000 7f2570674700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255406d8a0 0x7f255406fd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.362+0000 7f2570674700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25680ffbd0 0x7f2568104fc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.362+0000 7f2570674700 1 --2- 172.21.15.88:0/4219780906 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2568104130 0x7f2568105500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.362+0000 7f2570674700 1 -- 172.21.15.88:0/4219780906 >> 172.21.15.88:0/4219780906 conn(0x7f2568075130 msgr2=0x7f25680685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:52.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.363+0000 7f2570674700 1 -- 172.21.15.88:0/4219780906 shutdown_connections 2024-02-19T19:06:52.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.363+0000 7f2570674700 1 -- 172.21.15.88:0/4219780906 wait complete. 2024-02-19T19:06:52.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.490+0000 7f430eb74700 1 -- 172.21.15.88:0/932127665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4308104130 msgr2=0x7f4308104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:52.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.490+0000 7f430eb74700 1 --2- 172.21.15.88:0/932127665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4308104130 0x7f4308104500 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42fc009910 tx=0x7f42fc009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:52.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.490+0000 7f430eb74700 1 -- 172.21.15.88:0/932127665 shutdown_connections 2024-02-19T19:06:52.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.490+0000 7f430eb74700 1 --2- 172.21.15.88:0/932127665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080ffbd0 0x7f4308100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.490+0000 7f430eb74700 1 --2- 172.21.15.88:0/932127665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4308104130 0x7f4308104500 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.490+0000 7f430eb74700 1 -- 172.21.15.88:0/932127665 >> 172.21.15.88:0/932127665 conn(0x7f4308075130 msgr2=0x7f4308075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:52.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.490+0000 7f430eb74700 1 -- 172.21.15.88:0/932127665 shutdown_connections 2024-02-19T19:06:52.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.490+0000 7f430eb74700 1 -- 172.21.15.88:0/932127665 wait complete. 2024-02-19T19:06:52.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f430eb74700 1 Processor -- start 2024-02-19T19:06:52.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f430eb74700 1 -- start start 2024-02-19T19:06:52.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f430eb74700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080ffbd0 0x7f4308195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:52.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f430eb74700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4308104130 0x7f4308196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:52.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f430eb74700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4308196a50 con 0x7f4308104130 2024-02-19T19:06:52.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f430eb74700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430819a7e0 con 0x7f43080ffbd0 2024-02-19T19:06:52.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f4307fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4308104130 0x7f4308196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:52.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f4307fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4308104130 0x7f4308196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48732/0 (socket says 172.21.15.88:48732) 2024-02-19T19:06:52.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f4307fff700 1 -- 172.21.15.88:0/2160362922 learned_addr learned my addr 172.21.15.88:0/2160362922 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:52.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f430c910700 1 --2- 172.21.15.88:0/2160362922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080ffbd0 0x7f4308195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:52.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f4307fff700 1 -- 172.21.15.88:0/2160362922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080ffbd0 msgr2=0x7f4308195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:52.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f4307fff700 1 --2- 172.21.15.88:0/2160362922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080ffbd0 0x7f4308195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.491+0000 7f4307fff700 1 -- 172.21.15.88:0/2160362922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc0095f0 con 0x7f4308104130 2024-02-19T19:06:52.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.492+0000 7f4307fff700 1 --2- 172.21.15.88:0/2160362922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4308104130 0x7f4308196370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f42f400e920 tx=0x7f42f400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:52.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.492+0000 7f4305ffb700 1 -- 172.21.15.88:0/2160362922 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f400c9d0 con 0x7f4308104130 2024-02-19T19:06:52.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.492+0000 7f4305ffb700 1 -- 172.21.15.88:0/2160362922 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42f4010430 con 0x7f4308104130 2024-02-19T19:06:52.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.492+0000 7f4305ffb700 1 -- 172.21.15.88:0/2160362922 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f4018630 con 0x7f4308104130 2024-02-19T19:06:52.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.492+0000 7f430eb74700 1 -- 172.21.15.88:0/2160362922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430819aac0 con 0x7f4308104130 2024-02-19T19:06:52.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.492+0000 7f430eb74700 1 -- 172.21.15.88:0/2160362922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430819b010 con 0x7f4308104130 2024-02-19T19:06:52.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.493+0000 7f430eb74700 1 -- 172.21.15.88:0/2160362922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430804f2a0 con 0x7f4308104130 2024-02-19T19:06:52.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.494+0000 7f4305ffb700 1 -- 172.21.15.88:0/2160362922 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f42f40105a0 con 0x7f4308104130 2024-02-19T19:06:52.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.494+0000 7f4305ffb700 1 --2- 172.21.15.88:0/2160362922 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42f806d8a0 0x7f42f806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:52.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.494+0000 7f4305ffb700 1 -- 172.21.15.88:0/2160362922 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f42f4014070 con 0x7f4308104130 2024-02-19T19:06:52.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.494+0000 7f430c910700 1 --2- 172.21.15.88:0/2160362922 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42f806d8a0 0x7f42f806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:52.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.495+0000 7f430c910700 1 --2- 172.21.15.88:0/2160362922 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42f806d8a0 0x7f42f806fd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f42fc000c00 tx=0x7f42fc009f70 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:52.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.499+0000 7f4305ffb700 1 -- 172.21.15.88:0/2160362922 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42f4052d20 con 0x7f4308104130 2024-02-19T19:06:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.830+0000 7f430eb74700 1 -- 172.21.15.88:0/2160362922 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f430819b230 con 0x7f4308104130 2024-02-19T19:06:52.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.831+0000 7f4305ffb700 1 -- 172.21.15.88:0/2160362922 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f42f4056fa0 con 0x7f4308104130 2024-02-19T19:06:52.851 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:06:52.851 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:06:52.851 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:06:52.852 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:06:52.853 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:06:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.833+0000 7f430eb74700 1 -- 172.21.15.88:0/2160362922 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42f806d8a0 msgr2=0x7f42f806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.833+0000 7f430eb74700 1 --2- 172.21.15.88:0/2160362922 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42f806d8a0 0x7f42f806fd50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f42fc000c00 tx=0x7f42fc009f70 comp rx=0 tx=0).stop 2024-02-19T19:06:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.833+0000 7f430eb74700 1 -- 172.21.15.88:0/2160362922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4308104130 msgr2=0x7f4308196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.833+0000 7f430eb74700 1 --2- 172.21.15.88:0/2160362922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4308104130 0x7f4308196370 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f42f400e920 tx=0x7f42f400ec30 comp rx=0 tx=0).stop 2024-02-19T19:06:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.833+0000 7f430eb74700 1 -- 172.21.15.88:0/2160362922 shutdown_connections 2024-02-19T19:06:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.833+0000 7f430eb74700 1 --2- 172.21.15.88:0/2160362922 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42f806d8a0 0x7f42f806fd50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.833+0000 7f430eb74700 1 --2- 172.21.15.88:0/2160362922 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43080ffbd0 0x7f4308195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.833+0000 7f430eb74700 1 --2- 172.21.15.88:0/2160362922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4308104130 0x7f4308196370 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.833+0000 7f430eb74700 1 -- 172.21.15.88:0/2160362922 >> 172.21.15.88:0/2160362922 conn(0x7f4308075130 msgr2=0x7f43080685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.833+0000 7f430eb74700 1 -- 172.21.15.88:0/2160362922 shutdown_connections 2024-02-19T19:06:52.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.833+0000 7f430eb74700 1 -- 172.21.15.88:0/2160362922 wait complete. 2024-02-19T19:06:52.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.949+0000 7f7f463f2700 1 -- 172.21.15.88:0/2478237513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f400ffc60 msgr2=0x7f7f4010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:52.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.949+0000 7f7f463f2700 1 --2- 172.21.15.88:0/2478237513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f400ffc60 0x7f7f4010a6d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009910 tx=0x7f7f34009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:52.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.949+0000 7f7f463f2700 1 -- 172.21.15.88:0/2478237513 shutdown_connections 2024-02-19T19:06:52.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.949+0000 7f7f463f2700 1 --2- 172.21.15.88:0/2478237513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f400ffc60 0x7f7f4010a6d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.949+0000 7f7f463f2700 1 --2- 172.21.15.88:0/2478237513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f400ff350 0x7f7f400ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.949+0000 7f7f463f2700 1 -- 172.21.15.88:0/2478237513 >> 172.21.15.88:0/2478237513 conn(0x7f7f400748b0 msgr2=0x7f7f40074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:52.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.949+0000 7f7f463f2700 1 -- 172.21.15.88:0/2478237513 shutdown_connections 2024-02-19T19:06:52.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.949+0000 7f7f463f2700 1 -- 172.21.15.88:0/2478237513 wait complete. 2024-02-19T19:06:52.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.949+0000 7f7f463f2700 1 Processor -- start 2024-02-19T19:06:52.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.950+0000 7f7f463f2700 1 -- start start 2024-02-19T19:06:52.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.950+0000 7f7f463f2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f400ff350 0x7f7f40195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:52.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.950+0000 7f7f463f2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f400ffc60 0x7f7f401963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:52.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.950+0000 7f7f463f2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f40196ac0 con 0x7f7f400ff350 2024-02-19T19:06:52.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.950+0000 7f7f463f2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4019a850 con 0x7f7f400ffc60 2024-02-19T19:06:52.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.950+0000 7f7f3ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f400ff350 0x7f7f40195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:52.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.950+0000 7f7f3f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f400ffc60 0x7f7f401963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:52.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.950+0000 7f7f3f7fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f400ffc60 0x7f7f401963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:60630/0 (socket says 172.21.15.88:60630) 2024-02-19T19:06:52.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.950+0000 7f7f3f7fe700 1 -- 172.21.15.88:0/4241220518 learned_addr learned my addr 172.21.15.88:0/4241220518 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:52.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.950+0000 7f7f3ffff700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f400ff350 0x7f7f40195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48754/0 (socket says 172.21.15.88:48754) 2024-02-19T19:06:52.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.951+0000 7f7f3f7fe700 1 -- 172.21.15.88:0/4241220518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f400ff350 msgr2=0x7f7f40195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:52.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.951+0000 7f7f3f7fe700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f400ff350 0x7f7f40195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:52.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.951+0000 7f7f3f7fe700 1 -- 172.21.15.88:0/4241220518 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f340095f0 con 0x7f7f400ffc60 2024-02-19T19:06:52.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.951+0000 7f7f3ffff700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f400ff350 0x7f7f40195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:06:52.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.951+0000 7f7f3f7fe700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f400ffc60 0x7f7f401963e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f34005b40 tx=0x7f7f3400b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:52.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.951+0000 7f7f3d7fa700 1 -- 172.21.15.88:0/4241220518 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f3401d070 con 0x7f7f400ffc60 2024-02-19T19:06:52.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.951+0000 7f7f3d7fa700 1 -- 172.21.15.88:0/4241220518 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f340052a0 con 0x7f7f400ffc60 2024-02-19T19:06:52.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.951+0000 7f7f463f2700 1 -- 172.21.15.88:0/4241220518 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4019aad0 con 0x7f7f400ffc60 2024-02-19T19:06:52.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.951+0000 7f7f3d7fa700 1 -- 172.21.15.88:0/4241220518 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f34022470 con 0x7f7f400ffc60 2024-02-19T19:06:52.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.951+0000 7f7f463f2700 1 -- 172.21.15.88:0/4241220518 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4019b020 con 0x7f7f400ffc60 2024-02-19T19:06:52.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.952+0000 7f7f463f2700 1 -- 172.21.15.88:0/4241220518 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f40107dd0 con 0x7f7f400ffc60 2024-02-19T19:06:52.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.953+0000 7f7f3d7fa700 1 -- 172.21.15.88:0/4241220518 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7f3400f770 con 0x7f7f400ffc60 2024-02-19T19:06:52.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.953+0000 7f7f3d7fa700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f2806d8a0 0x7f7f2806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:52.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.953+0000 7f7f3d7fa700 1 -- 172.21.15.88:0/4241220518 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7f3408e800 con 0x7f7f400ffc60 2024-02-19T19:06:52.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.953+0000 7f7f3ffff700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f2806d8a0 0x7f7f2806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:52.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.954+0000 7f7f3ffff700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f2806d8a0 0x7f7f2806fd50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009730 tx=0x7f7f30012040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:52.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:52.958+0000 7f7f3d7fa700 1 -- 172.21.15.88:0/4241220518 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f34058040 con 0x7f7f400ffc60 2024-02-19T19:06:53.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.228+0000 7f7f463f2700 1 -- 172.21.15.88:0/4241220518 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7f4019b290 con 0x7f7f400ffc60 2024-02-19T19:06:53.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.228+0000 7f7f3d7fa700 1 -- 172.21.15.88:0/4241220518 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1892 (secure 0 0 0) 0x7f7f3405b700 con 0x7f7f400ffc60 2024-02-19T19:06:53.249 INFO:teuthology.orchestra.run.smithi088.stdout:e20 2024-02-19T19:06:53.249 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:06:53.249 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:06:53.250 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:06:53.250 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:06:53.250 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:06:53.250 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:06:53.250 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 20 2024-02-19T19:06:53.250 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:06:53.250 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:06:53.250 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:05:14.913778+0000 2024-02-19T19:06:53.250 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:06:53.251 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:06:53.252 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:06:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.230+0000 7f7f463f2700 1 -- 172.21.15.88:0/4241220518 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f2806d8a0 msgr2=0x7f7f2806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.230+0000 7f7f463f2700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f2806d8a0 0x7f7f2806fd50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009730 tx=0x7f7f30012040 comp rx=0 tx=0).stop 2024-02-19T19:06:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.230+0000 7f7f463f2700 1 -- 172.21.15.88:0/4241220518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f400ffc60 msgr2=0x7f7f401963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.230+0000 7f7f463f2700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f400ffc60 0x7f7f401963e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f7f34005b40 tx=0x7f7f3400b5b0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.231+0000 7f7f463f2700 1 -- 172.21.15.88:0/4241220518 shutdown_connections 2024-02-19T19:06:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.231+0000 7f7f463f2700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f2806d8a0 0x7f7f2806fd50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.231+0000 7f7f463f2700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f400ff350 0x7f7f40195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.231+0000 7f7f463f2700 1 --2- 172.21.15.88:0/4241220518 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f400ffc60 0x7f7f401963e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.231+0000 7f7f463f2700 1 -- 172.21.15.88:0/4241220518 >> 172.21.15.88:0/4241220518 conn(0x7f7f400748b0 msgr2=0x7f7f40104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.231+0000 7f7f463f2700 1 -- 172.21.15.88:0/4241220518 shutdown_connections 2024-02-19T19:06:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.231+0000 7f7f463f2700 1 -- 172.21.15.88:0/4241220518 wait complete. 2024-02-19T19:06:53.254 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-02-19T19:06:53.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.342+0000 7f4f77c5d700 1 -- 172.21.15.88:0/874698001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f701004e0 msgr2=0x7f4f701008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:53.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.342+0000 7f4f77c5d700 1 --2- 172.21.15.88:0/874698001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f701004e0 0x7f4f701008b0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009910 tx=0x7f4f6c009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:53.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.342+0000 7f4f77c5d700 1 -- 172.21.15.88:0/874698001 shutdown_connections 2024-02-19T19:06:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.342+0000 7f4f77c5d700 1 --2- 172.21.15.88:0/874698001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f70100e80 0x7f4f70104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.342+0000 7f4f77c5d700 1 --2- 172.21.15.88:0/874698001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f701004e0 0x7f4f701008b0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.342+0000 7f4f77c5d700 1 -- 172.21.15.88:0/874698001 >> 172.21.15.88:0/874698001 conn(0x7f4f700fbda0 msgr2=0x7f4f700fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.343+0000 7f4f77c5d700 1 -- 172.21.15.88:0/874698001 shutdown_connections 2024-02-19T19:06:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.343+0000 7f4f77c5d700 1 -- 172.21.15.88:0/874698001 wait complete. 2024-02-19T19:06:53.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.343+0000 7f4f77c5d700 1 Processor -- start 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.343+0000 7f4f77c5d700 1 -- start start 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.343+0000 7f4f77c5d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f701004e0 0x7f4f70195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.343+0000 7f4f77c5d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f70100e80 0x7f4f70196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f77c5d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f70196b10 con 0x7f4f70100e80 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f77c5d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7019a8a0 con 0x7f4f701004e0 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f751f8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f70100e80 0x7f4f70196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f751f8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f70100e80 0x7f4f70196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48768/0 (socket says 172.21.15.88:48768) 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f751f8700 1 -- 172.21.15.88:0/671954677 learned_addr learned my addr 172.21.15.88:0/671954677 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f759f9700 1 --2- 172.21.15.88:0/671954677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f701004e0 0x7f4f70195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f751f8700 1 -- 172.21.15.88:0/671954677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f701004e0 msgr2=0x7f4f70195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f751f8700 1 --2- 172.21.15.88:0/671954677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f701004e0 0x7f4f70195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f751f8700 1 -- 172.21.15.88:0/671954677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c0095f0 con 0x7f4f70100e80 2024-02-19T19:06:53.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f751f8700 1 --2- 172.21.15.88:0/671954677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f70100e80 0x7f4f70196430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000d900 tx=0x7f4f6000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:53.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f66ffd700 1 -- 172.21.15.88:0/671954677 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f60009900 con 0x7f4f70100e80 2024-02-19T19:06:53.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f77c5d700 1 -- 172.21.15.88:0/671954677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7019ab80 con 0x7f4f70100e80 2024-02-19T19:06:53.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f66ffd700 1 -- 172.21.15.88:0/671954677 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f60010460 con 0x7f4f70100e80 2024-02-19T19:06:53.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f66ffd700 1 -- 172.21.15.88:0/671954677 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f6000f5f0 con 0x7f4f70100e80 2024-02-19T19:06:53.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.344+0000 7f4f77c5d700 1 -- 172.21.15.88:0/671954677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7019b0d0 con 0x7f4f70100e80 2024-02-19T19:06:53.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.345+0000 7f4f77c5d700 1 -- 172.21.15.88:0/671954677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f701087f0 con 0x7f4f70100e80 2024-02-19T19:06:53.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.346+0000 7f4f66ffd700 1 -- 172.21.15.88:0/671954677 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4f600105d0 con 0x7f4f70100e80 2024-02-19T19:06:53.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.346+0000 7f4f66ffd700 1 --2- 172.21.15.88:0/671954677 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5c06d8a0 0x7f4f5c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:53.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.346+0000 7f4f66ffd700 1 -- 172.21.15.88:0/671954677 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4f6008cf20 con 0x7f4f70100e80 2024-02-19T19:06:53.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.346+0000 7f4f759f9700 1 --2- 172.21.15.88:0/671954677 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5c06d8a0 0x7f4f5c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:53.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.347+0000 7f4f759f9700 1 --2- 172.21.15.88:0/671954677 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5c06d8a0 0x7f4f5c06fd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c006010 tx=0x7f4f6c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:53.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.351+0000 7f4f66ffd700 1 -- 172.21.15.88:0/671954677 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f600c0d20 con 0x7f4f70100e80 2024-02-19T19:06:53.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:53 smithi132 ceph-mon[70676]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:53.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:53 smithi132 ceph-mon[70676]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:53.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:53 smithi132 ceph-mon[70676]: pgmap v60: 65 pgs: 65 active+clean; 935 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 763 KiB/s rd, 5.3 MiB/s wr, 122 op/s 2024-02-19T19:06:53.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:53 smithi132 ceph-mon[70676]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:53.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:53 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2160362922' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:06:53.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:53 smithi088 ceph-mon[88643]: from='client.34212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:53.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:53 smithi088 ceph-mon[88643]: from='client.34216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:53.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:53 smithi088 ceph-mon[88643]: pgmap v60: 65 pgs: 65 active+clean; 935 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 763 KiB/s rd, 5.3 MiB/s wr, 122 op/s 2024-02-19T19:06:53.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:53 smithi088 ceph-mon[88643]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:53.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:53 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2160362922' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:06:53.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.643+0000 7f4f77c5d700 1 -- 172.21.15.88:0/671954677 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f7019b340 con 0x7f4f5c06d8a0 2024-02-19T19:06:53.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.645+0000 7f4f66ffd700 1 -- 172.21.15.88:0/671954677 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f4f7019b340 con 0x7f4f5c06d8a0 2024-02-19T19:06:53.665 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:06:53.665 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:06:53.665 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:06:53.665 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:06:53.665 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:06:53.665 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:06:53.665 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:06:53.666 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:06:53.666 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "4/23 daemons upgraded", 2024-02-19T19:06:53.666 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading crash daemons", 2024-02-19T19:06:53.666 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:06:53.666 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:06:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.647+0000 7f4f77c5d700 1 -- 172.21.15.88:0/671954677 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5c06d8a0 msgr2=0x7f4f5c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.647+0000 7f4f77c5d700 1 --2- 172.21.15.88:0/671954677 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5c06d8a0 0x7f4f5c06fd50 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c006010 tx=0x7f4f6c01a040 comp rx=0 tx=0).stop 2024-02-19T19:06:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.647+0000 7f4f77c5d700 1 -- 172.21.15.88:0/671954677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f70100e80 msgr2=0x7f4f70196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.647+0000 7f4f77c5d700 1 --2- 172.21.15.88:0/671954677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f70100e80 0x7f4f70196430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000d900 tx=0x7f4f6000dc10 comp rx=0 tx=0).stop 2024-02-19T19:06:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.647+0000 7f4f77c5d700 1 -- 172.21.15.88:0/671954677 shutdown_connections 2024-02-19T19:06:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.647+0000 7f4f77c5d700 1 --2- 172.21.15.88:0/671954677 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5c06d8a0 0x7f4f5c06fd50 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.647+0000 7f4f77c5d700 1 --2- 172.21.15.88:0/671954677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f701004e0 0x7f4f70195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.647+0000 7f4f77c5d700 1 --2- 172.21.15.88:0/671954677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f70100e80 0x7f4f70196430 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.647+0000 7f4f77c5d700 1 -- 172.21.15.88:0/671954677 >> 172.21.15.88:0/671954677 conn(0x7f4f700fbda0 msgr2=0x7f4f700fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.647+0000 7f4f77c5d700 1 -- 172.21.15.88:0/671954677 shutdown_connections 2024-02-19T19:06:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.647+0000 7f4f77c5d700 1 -- 172.21.15.88:0/671954677 wait complete. 2024-02-19T19:06:53.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.760+0000 7f4fcc32f700 1 -- 172.21.15.88:0/1590118368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc40ff350 msgr2=0x7f4fc40ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:53.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.760+0000 7f4fcc32f700 1 --2- 172.21.15.88:0/1590118368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc40ff350 0x7f4fc40ff720 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0009910 tx=0x7f4fc0009c20 comp rx=0 tx=0).stop 2024-02-19T19:06:53.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.760+0000 7f4fcc32f700 1 -- 172.21.15.88:0/1590118368 shutdown_connections 2024-02-19T19:06:53.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.760+0000 7f4fcc32f700 1 --2- 172.21.15.88:0/1590118368 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc40ffc60 0x7f4fc410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.760+0000 7f4fcc32f700 1 --2- 172.21.15.88:0/1590118368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc40ff350 0x7f4fc40ff720 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.760+0000 7f4fcc32f700 1 -- 172.21.15.88:0/1590118368 >> 172.21.15.88:0/1590118368 conn(0x7f4fc40748b0 msgr2=0x7f4fc4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:53.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.760+0000 7f4fcc32f700 1 -- 172.21.15.88:0/1590118368 shutdown_connections 2024-02-19T19:06:53.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.760+0000 7f4fcc32f700 1 -- 172.21.15.88:0/1590118368 wait complete. 2024-02-19T19:06:53.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.760+0000 7f4fcc32f700 1 Processor -- start 2024-02-19T19:06:53.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fcc32f700 1 -- start start 2024-02-19T19:06:53.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fcc32f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc40ff350 0x7f4fc4100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:53.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fcc32f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc40ffc60 0x7f4fc4100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:53.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fcc32f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc4100fb0 con 0x7f4fc40ff350 2024-02-19T19:06:53.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fcc32f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc41010f0 con 0x7f4fc40ffc60 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fca0cb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc40ff350 0x7f4fc4100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fca0cb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc40ff350 0x7f4fc4100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48792/0 (socket says 172.21.15.88:48792) 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fca0cb700 1 -- 172.21.15.88:0/434663366 learned_addr learned my addr 172.21.15.88:0/434663366 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fc98ca700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc40ffc60 0x7f4fc4100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fca0cb700 1 -- 172.21.15.88:0/434663366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc40ffc60 msgr2=0x7f4fc4100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fca0cb700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc40ffc60 0x7f4fc4100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fca0cb700 1 -- 172.21.15.88:0/434663366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc00095f0 con 0x7f4fc40ff350 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.761+0000 7f4fc98ca700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc40ffc60 0x7f4fc4100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.762+0000 7f4fca0cb700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc40ff350 0x7f4fc4100530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006010 tx=0x7f4fc000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.762+0000 7f4fbb7fe700 1 -- 172.21.15.88:0/434663366 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc001d070 con 0x7f4fc40ff350 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.762+0000 7f4fbb7fe700 1 -- 172.21.15.88:0/434663366 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fc0003d10 con 0x7f4fc40ff350 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.762+0000 7f4fbb7fe700 1 -- 172.21.15.88:0/434663366 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc000fdb0 con 0x7f4fc40ff350 2024-02-19T19:06:53.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.762+0000 7f4fcc32f700 1 -- 172.21.15.88:0/434663366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc41a2020 con 0x7f4fc40ff350 2024-02-19T19:06:53.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.762+0000 7f4fcc32f700 1 -- 172.21.15.88:0/434663366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc41a2430 con 0x7f4fc40ff350 2024-02-19T19:06:53.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.763+0000 7f4fcc32f700 1 -- 172.21.15.88:0/434663366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc4107dd0 con 0x7f4fc40ff350 2024-02-19T19:06:53.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.764+0000 7f4fbb7fe700 1 -- 172.21.15.88:0/434663366 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4fc000f5f0 con 0x7f4fc40ff350 2024-02-19T19:06:53.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.764+0000 7f4fbb7fe700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fb006d8a0 0x7f4fb006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:06:53.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.764+0000 7f4fc98ca700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fb006d8a0 0x7f4fb006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:06:53.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.764+0000 7f4fbb7fe700 1 -- 172.21.15.88:0/434663366 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4fc008e5d0 con 0x7f4fc40ff350 2024-02-19T19:06:53.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.765+0000 7f4fc98ca700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fb006d8a0 0x7f4fb006fd50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4101d20 tx=0x7f4fb4009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:06:53.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:53.769+0000 7f4fbb7fe700 1 -- 172.21.15.88:0/434663366 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4fc0057eb0 con 0x7f4fc40ff350 2024-02-19T19:06:54.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.081+0000 7f4fcc32f700 1 -- 172.21.15.88:0/434663366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4fc404f2a0 con 0x7f4fc40ff350 2024-02-19T19:06:54.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.082+0000 7f4fbb7fe700 1 -- 172.21.15.88:0/434663366 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f4fc005b570 con 0x7f4fc40ff350 2024-02-19T19:06:54.103 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_OK 2024-02-19T19:06:54.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.084+0000 7f4fcc32f700 1 -- 172.21.15.88:0/434663366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fb006d8a0 msgr2=0x7f4fb006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:54.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.084+0000 7f4fcc32f700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fb006d8a0 0x7f4fb006fd50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4101d20 tx=0x7f4fb4009260 comp rx=0 tx=0).stop 2024-02-19T19:06:54.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.084+0000 7f4fcc32f700 1 -- 172.21.15.88:0/434663366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc40ff350 msgr2=0x7f4fc4100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:06:54.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.084+0000 7f4fcc32f700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc40ff350 0x7f4fc4100530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006010 tx=0x7f4fc000ba40 comp rx=0 tx=0).stop 2024-02-19T19:06:54.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.084+0000 7f4fcc32f700 1 -- 172.21.15.88:0/434663366 shutdown_connections 2024-02-19T19:06:54.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.084+0000 7f4fcc32f700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fb006d8a0 0x7f4fb006fd50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:54.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.084+0000 7f4fcc32f700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fc40ff350 0x7f4fc4100530 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:54.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.084+0000 7f4fcc32f700 1 --2- 172.21.15.88:0/434663366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4fc40ffc60 0x7f4fc4100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:06:54.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.084+0000 7f4fcc32f700 1 -- 172.21.15.88:0/434663366 >> 172.21.15.88:0/434663366 conn(0x7f4fc40748b0 msgr2=0x7f4fc4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:06:54.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.084+0000 7f4fcc32f700 1 -- 172.21.15.88:0/434663366 shutdown_connections 2024-02-19T19:06:54.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:06:54.084+0000 7f4fcc32f700 1 -- 172.21.15.88:0/434663366 wait complete. 2024-02-19T19:06:54.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:54 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4241220518' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:06:54.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:54 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/434663366' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:06:54.532 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:54 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4241220518' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:06:54.532 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:54 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/434663366' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:06:55.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:55 smithi132 ceph-mon[70676]: from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:55.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:55 smithi132 ceph-mon[70676]: pgmap v61: 65 pgs: 65 active+clean; 935 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 763 KiB/s rd, 5.3 MiB/s wr, 122 op/s 2024-02-19T19:06:55.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:55.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:55.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:55 smithi088 ceph-mon[88643]: from='client.34232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:06:55.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:55 smithi088 ceph-mon[88643]: pgmap v61: 65 pgs: 65 active+clean; 935 MiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 763 KiB/s rd, 5.3 MiB/s wr, 122 op/s 2024-02-19T19:06:55.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:55.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:57.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:56 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:57.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:56 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:57.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:56 smithi132 ceph-mon[70676]: pgmap v62: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 771 KiB/s rd, 5.7 MiB/s wr, 174 op/s 2024-02-19T19:06:57.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:56 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:57.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:56 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:57.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:56 smithi088 ceph-mon[88643]: pgmap v62: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 771 KiB/s rd, 5.7 MiB/s wr, 174 op/s 2024-02-19T19:06:58.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:57 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:58.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:57 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:58.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:57 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:58.350 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:57 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:59.446 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:59 smithi132 ceph-mon[70676]: pgmap v63: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 900 KiB/s wr, 126 op/s 2024-02-19T19:06:59.447 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:59.447 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:59.447 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:06:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:06:59.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:59 smithi088 ceph-mon[88643]: pgmap v63: 65 pgs: 65 active+clean; 283 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 900 KiB/s wr, 126 op/s 2024-02-19T19:06:59.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:59.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:06:59.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:06:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:07:01.100 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:01.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:01.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:00 smithi088 ceph-mon[88643]: pgmap v64: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-02-19T19:07:01.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:01.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:01.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:00 smithi132 ceph-mon[70676]: pgmap v64: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 1.3 MiB/s wr, 246 op/s 2024-02-19T19:07:03.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:03 smithi132 ceph-mon[70676]: pgmap v65: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 898 KiB/s wr, 172 op/s 2024-02-19T19:07:03.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:03 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:03.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:03 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:03.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:03 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:07:03.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:03 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:07:03.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:03 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:03.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:03 smithi088 ceph-mon[88643]: pgmap v65: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 898 KiB/s wr, 172 op/s 2024-02-19T19:07:03.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:03 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:03.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:03 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:03.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:03 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:07:03.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:03 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:07:03.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:03 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:04.226 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:07:04.226 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:04.226 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:04.226 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:04.226 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: Upgrade: Setting container_image for all crash 2024-02-19T19:07:04.227 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:04.227 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi088"}]: dispatch 2024-02-19T19:07:04.227 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi088"}]': finished 2024-02-19T19:07:04.227 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi132"}]: dispatch 2024-02-19T19:07:04.227 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi132"}]': finished 2024-02-19T19:07:04.227 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-19T19:07:04.227 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-19T19:07:04.227 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:04 smithi088 ceph-mon[88643]: Upgrade: osd.0 is safe to restart 2024-02-19T19:07:04.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:07:04.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:04.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:04.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:04.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: Upgrade: Setting container_image for all crash 2024-02-19T19:07:04.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:04.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi088"}]: dispatch 2024-02-19T19:07:04.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi088"}]': finished 2024-02-19T19:07:04.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi132"}]: dispatch 2024-02-19T19:07:04.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi132"}]': finished 2024-02-19T19:07:04.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-19T19:07:04.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-02-19T19:07:04.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:04 smithi132 ceph-mon[70676]: Upgrade: osd.0 is safe to restart 2024-02-19T19:07:05.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:05 smithi088 ceph-mon[88643]: pgmap v66: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 898 KiB/s wr, 172 op/s 2024-02-19T19:07:05.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:05.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-19T19:07:05.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:07:05.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:05 smithi132 ceph-mon[70676]: pgmap v66: 65 pgs: 65 active+clean; 285 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 898 KiB/s wr, 172 op/s 2024-02-19T19:07:05.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:05.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-19T19:07:05.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:07:06.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:06 smithi132 ceph-mon[70676]: Upgrade: Updating osd.0 2024-02-19T19:07:06.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:06 smithi132 ceph-mon[70676]: Deploying daemon osd.0 on smithi088 2024-02-19T19:07:06.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:06 smithi088 ceph-mon[88643]: Upgrade: Updating osd.0 2024-02-19T19:07:06.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:06 smithi088 ceph-mon[88643]: Deploying daemon osd.0 on smithi088 2024-02-19T19:07:07.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:07 smithi132 ceph-mon[70676]: pgmap v67: 65 pgs: 65 active+clean; 289 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 273 op/s 2024-02-19T19:07:07.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:07 smithi088 ceph-mon[88643]: pgmap v67: 65 pgs: 65 active+clean; 289 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.4 MiB/s wr, 273 op/s 2024-02-19T19:07:09.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:09 smithi088 ceph-mon[88643]: pgmap v68: 65 pgs: 65 active+clean; 289 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 952 KiB/s wr, 221 op/s 2024-02-19T19:07:09.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:09 smithi132 ceph-mon[70676]: pgmap v68: 65 pgs: 65 active+clean; 289 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 22 KiB/s rd, 952 KiB/s wr, 221 op/s 2024-02-19T19:07:11.673 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:11 smithi088 ceph-mon[88643]: pgmap v69: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.4 MiB/s wr, 372 op/s 2024-02-19T19:07:11.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:11 smithi132 ceph-mon[70676]: pgmap v69: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 38 KiB/s rd, 1.4 MiB/s wr, 372 op/s 2024-02-19T19:07:13.497 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:13 smithi088 ceph-mon[88643]: pgmap v70: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1015 KiB/s wr, 252 op/s 2024-02-19T19:07:13.497 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:07:13.497 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:13 smithi088 systemd[1]: Stopping Ceph osd.0 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:07:13.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:13 smithi132 ceph-mon[70676]: pgmap v70: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1015 KiB/s wr, 252 op/s 2024-02-19T19:07:13.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:07:13.850 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:13 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[44165]: 2024-02-19T19:07:13.474+0000 7f11bd5f8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T19:07:13.851 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:13 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[44165]: 2024-02-19T19:07:13.474+0000 7f11bd5f8700 -1 osd.0 49 *** Got signal Terminated *** 2024-02-19T19:07:13.851 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:13 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[44165]: 2024-02-19T19:07:13.474+0000 7f11bd5f8700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T19:07:14.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:14 smithi088 ceph-mon[88643]: osd.0 marked itself down and dead 2024-02-19T19:07:14.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:14 smithi132 ceph-mon[70676]: osd.0 marked itself down and dead 2024-02-19T19:07:15.101 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:14 smithi088 podman[100299]: 2024-02-19 19:07:14.90769531 +0000 UTC m=+1.553388053 container died 0cd963a2a48a1befe40cffa90399f032a1b7113ed0c9442afa3440a77977b092 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, RELEASE=HEAD, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-02-19T19:07:15.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:15 smithi088 ceph-mon[88643]: pgmap v71: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1015 KiB/s wr, 252 op/s 2024-02-19T19:07:15.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:15 smithi088 ceph-mon[88643]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:07:15.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:15 smithi088 ceph-mon[88643]: osdmap e50: 6 total, 5 up, 6 in 2024-02-19T19:07:15.601 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:15 smithi088 podman[100299]: 2024-02-19 19:07:15.314268162 +0000 UTC m=+1.959960905 container cleanup 0cd963a2a48a1befe40cffa90399f032a1b7113ed0c9442afa3440a77977b092 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.build-date=20231212) 2024-02-19T19:07:15.601 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:15 smithi088 bash[100299]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0 2024-02-19T19:07:15.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:15 smithi132 ceph-mon[70676]: pgmap v71: 65 pgs: 65 active+clean; 286 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 26 KiB/s rd, 1015 KiB/s wr, 252 op/s 2024-02-19T19:07:15.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:15 smithi132 ceph-mon[70676]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:07:15.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:15 smithi132 ceph-mon[70676]: osdmap e50: 6 total, 5 up, 6 in 2024-02-19T19:07:16.100 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:15 smithi088 podman[100317]: 2024-02-19 19:07:15.724529211 +0000 UTC m=+0.816033658 container remove 0cd963a2a48a1befe40cffa90399f032a1b7113ed0c9442afa3440a77977b092 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_CLEAN=True, RELEASE=HEAD, ceph=True, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-02-19T19:07:16.549 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:16 smithi088 podman[100407]: 2024-02-19 19:07:16.143813468 +0000 UTC m=+0.021249861 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:07:16.549 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:16 smithi088 podman[100407]: 2024-02-19 19:07:16.275836784 +0000 UTC m=+0.153273165 container create e13ef2ccf2652dfa60103246e0f7c5eddcdb3e3a6143c2e27c821586de93a822 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:07:16.549 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:16 smithi088 ceph-mon[88643]: osdmap e51: 6 total, 5 up, 6 in 2024-02-19T19:07:16.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:16 smithi132 ceph-mon[70676]: osdmap e51: 6 total, 5 up, 6 in 2024-02-19T19:07:16.850 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:16 smithi088 podman[100407]: 2024-02-19 19:07:16.529188281 +0000 UTC m=+0.406624681 container init e13ef2ccf2652dfa60103246e0f7c5eddcdb3e3a6143c2e27c821586de93a822 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-19T19:07:16.851 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:16 smithi088 podman[100407]: 2024-02-19 19:07:16.533980353 +0000 UTC m=+0.411416743 container start e13ef2ccf2652dfa60103246e0f7c5eddcdb3e3a6143c2e27c821586de93a822 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:07:16.851 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:16 smithi088 podman[100407]: 2024-02-19 19:07:16.729488893 +0000 UTC m=+0.606925284 container attach e13ef2ccf2652dfa60103246e0f7c5eddcdb3e3a6143c2e27c821586de93a822 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:07:17.299 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:16 smithi088 podman[100407]: 2024-02-19 19:07:16.906900507 +0000 UTC m=+0.784336882 container died e13ef2ccf2652dfa60103246e0f7c5eddcdb3e3a6143c2e27c821586de93a822 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240214, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T19:07:17.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:17 smithi088 ceph-mon[88643]: pgmap v74: 65 pgs: 9 stale+active+clean, 56 active+clean; 278 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.6 MiB/s wr, 467 op/s 2024-02-19T19:07:17.564 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:17 smithi088 podman[100433]: 2024-02-19 19:07:17.526574612 +0000 UTC m=+0.609913886 container remove e13ef2ccf2652dfa60103246e0f7c5eddcdb3e3a6143c2e27c821586de93a822 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-19T19:07:17.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:17 smithi132 ceph-mon[70676]: pgmap v74: 65 pgs: 9 stale+active+clean, 56 active+clean; 278 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 48 KiB/s rd, 1.6 MiB/s wr, 467 op/s 2024-02-19T19:07:17.851 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:17 smithi088 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.0.service: Deactivated successfully. 2024-02-19T19:07:17.851 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:17 smithi088 systemd[1]: Stopped Ceph osd.0 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:07:17.851 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:17 smithi088 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.0.service: Consumed 57.851s CPU time. 2024-02-19T19:07:17.851 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:17 smithi088 systemd[1]: Starting Ceph osd.0 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:07:18.326 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 podman[100538]: 2024-02-19 19:07:17.91335621 +0000 UTC m=+0.019392624 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:07:18.326 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 podman[100538]: 2024-02-19 19:07:18.044677307 +0000 UTC m=+0.150713746 container create b36e3b902723109a0dd013dff2e0424d1aee5f2a7e250d8b0a4ce734f9b7ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-19T19:07:18.326 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 podman[100538]: 2024-02-19 19:07:18.305997777 +0000 UTC m=+0.412034175 container init b36e3b902723109a0dd013dff2e0424d1aee5f2a7e250d8b0a4ce734f9b7ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.schema-version=1.0) 2024-02-19T19:07:18.600 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 podman[100538]: 2024-02-19 19:07:18.309770604 +0000 UTC m=+0.415807008 container start b36e3b902723109a0dd013dff2e0424d1aee5f2a7e250d8b0a4ce734f9b7ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-02-19T19:07:18.601 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 podman[100538]: 2024-02-19 19:07:18.397802247 +0000 UTC m=+0.503838656 container attach b36e3b902723109a0dd013dff2e0424d1aee5f2a7e250d8b0a4ce734f9b7ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-19T19:07:19.101 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate[100553]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-19T19:07:19.101 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate[100553]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-19T19:07:19.101 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 bash[100538]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-19T19:07:19.101 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 bash[100538]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-19T19:07:19.101 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 bash[100538]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-19T19:07:19.101 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 bash[100538]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-19T19:07:19.101 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 bash[100538]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-19T19:07:19.102 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 bash[100538]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-19T19:07:19.102 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 bash[100538]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-19T19:07:19.102 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate[100553]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-19T19:07:19.102 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate[100553]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-19T19:07:19.102 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate[100553]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-02-19T19:07:19.102 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate[100553]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-02-19T19:07:19.102 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate[100553]: --> ceph-volume raw activate successful for osd ID: 0 2024-02-19T19:07:19.102 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:18 smithi088 podman[100677]: 2024-02-19 19:07:18.850642049 +0000 UTC m=+0.029253918 container died b36e3b902723109a0dd013dff2e0424d1aee5f2a7e250d8b0a4ce734f9b7ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T19:07:19.489 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:19 smithi088 ceph-mon[88643]: pgmap v75: 65 pgs: 9 stale+active+clean, 56 active+clean; 278 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 826 KiB/s wr, 240 op/s 2024-02-19T19:07:19.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:19 smithi132 ceph-mon[70676]: pgmap v75: 65 pgs: 9 stale+active+clean, 56 active+clean; 278 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 826 KiB/s wr, 240 op/s 2024-02-19T19:07:19.757 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:19 smithi088 podman[100677]: 2024-02-19 19:07:19.469798984 +0000 UTC m=+0.648410833 container remove b36e3b902723109a0dd013dff2e0424d1aee5f2a7e250d8b0a4ce734f9b7ec7e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-activate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214) 2024-02-19T19:07:19.757 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:19 smithi088 podman[100741]: 2024-02-19 19:07:19.598229437 +0000 UTC m=+0.020451086 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:07:19.757 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:19 smithi088 podman[100741]: 2024-02-19 19:07:19.737676249 +0000 UTC m=+0.159897889 container create b544778e0f5de9443e1c981b32b5ce3ac406ebfcca41a925a363928442b18ee7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-19T19:07:20.018 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:19 smithi088 podman[100741]: 2024-02-19 19:07:19.97376481 +0000 UTC m=+0.395986459 container init b544778e0f5de9443e1c981b32b5ce3ac406ebfcca41a925a363928442b18ee7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-19T19:07:20.018 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:19 smithi088 podman[100741]: 2024-02-19 19:07:19.976943755 +0000 UTC m=+0.399165398 container start b544778e0f5de9443e1c981b32b5ce3ac406ebfcca41a925a363928442b18ee7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-02-19T19:07:20.351 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:19 smithi088 ceph-osd[100760]: -- 172.21.15.88:0/1802459216 <== mon.0 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561bb595f1e0 con 0x561bb5931000 2024-02-19T19:07:20.351 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:20 smithi088 bash[100741]: b544778e0f5de9443e1c981b32b5ce3ac406ebfcca41a925a363928442b18ee7 2024-02-19T19:07:20.351 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:20 smithi088 systemd[1]: Started Ceph osd.0 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:07:20.851 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:20 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[100756]: 2024-02-19T19:07:20.517+0000 7efd9a66e7c0 -1 Falling back to public interface 2024-02-19T19:07:21.273 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:21 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:21.273 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:21 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:21.273 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:21 smithi088 ceph-mon[88643]: pgmap v76: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.4 MiB/s wr, 396 op/s; 1793/11877 objects degraded (15.096%) 2024-02-19T19:07:21.273 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:21 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:07:21.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:21 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:21.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:21 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:21.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:21 smithi132 ceph-mon[70676]: pgmap v76: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.4 MiB/s wr, 396 op/s; 1793/11877 objects degraded (15.096%) 2024-02-19T19:07:21.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:21 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:07:22.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:22 smithi132 ceph-mon[70676]: Health check failed: Degraded data redundancy: 1793/11877 objects degraded (15.096%), 33 pgs degraded (PG_DEGRADED) 2024-02-19T19:07:22.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:22 smithi088 ceph-mon[88643]: Health check failed: Degraded data redundancy: 1793/11877 objects degraded (15.096%), 33 pgs degraded (PG_DEGRADED) 2024-02-19T19:07:23.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:23 smithi088 ceph-mon[88643]: pgmap v77: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.4 MiB/s wr, 396 op/s; 1793/11877 objects degraded (15.096%) 2024-02-19T19:07:23.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:23 smithi132 ceph-mon[70676]: pgmap v77: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.4 MiB/s wr, 396 op/s; 1793/11877 objects degraded (15.096%) 2024-02-19T19:07:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.211+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1329641602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b04104130 msgr2=0x7f9b04104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.211+0000 7f9b0c53c700 1 --2- 172.21.15.88:0/1329641602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b04104130 0x7f9b04104500 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9b00009940 tx=0x7f9b00009c50 comp rx=0 tx=0).stop 2024-02-19T19:07:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.211+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1329641602 shutdown_connections 2024-02-19T19:07:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.211+0000 7f9b0c53c700 1 --2- 172.21.15.88:0/1329641602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b040ffbd0 0x7f9b04100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.211+0000 7f9b0c53c700 1 --2- 172.21.15.88:0/1329641602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b04104130 0x7f9b04104500 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.211+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1329641602 >> 172.21.15.88:0/1329641602 conn(0x7f9b04075130 msgr2=0x7f9b04075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.211+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1329641602 shutdown_connections 2024-02-19T19:07:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.211+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1329641602 wait complete. 2024-02-19T19:07:24.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.212+0000 7f9b0c53c700 1 Processor -- start 2024-02-19T19:07:24.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.212+0000 7f9b0c53c700 1 -- start start 2024-02-19T19:07:24.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.212+0000 7f9b0c53c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b040ffbd0 0x7f9b04193a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:24.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.212+0000 7f9b0c53c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b04104130 0x7f9b04193f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:24.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.212+0000 7f9b0c53c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b04197bd0 con 0x7f9b040ffbd0 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.212+0000 7f9b0c53c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b041944c0 con 0x7f9b04104130 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.213+0000 7f9b09ad7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b04104130 0x7f9b04193f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.213+0000 7f9b09ad7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b04104130 0x7f9b04193f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:51824/0 (socket says 172.21.15.88:51824) 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.213+0000 7f9b09ad7700 1 -- 172.21.15.88:0/1131889061 learned_addr learned my addr 172.21.15.88:0/1131889061 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.213+0000 7f9b0a2d8700 1 --2- 172.21.15.88:0/1131889061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b040ffbd0 0x7f9b04193a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.213+0000 7f9b09ad7700 1 -- 172.21.15.88:0/1131889061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b040ffbd0 msgr2=0x7f9b04193a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.213+0000 7f9b09ad7700 1 --2- 172.21.15.88:0/1131889061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b040ffbd0 0x7f9b04193a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.213+0000 7f9b09ad7700 1 -- 172.21.15.88:0/1131889061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b000095f0 con 0x7f9b04104130 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.213+0000 7f9b09ad7700 1 --2- 172.21.15.88:0/1131889061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b04104130 0x7f9b04193f80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9af400e920 tx=0x7f9af400ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.213+0000 7f9afb7fe700 1 -- 172.21.15.88:0/1131889061 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9af400c9d0 con 0x7f9b04104130 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.214+0000 7f9afb7fe700 1 -- 172.21.15.88:0/1131889061 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9af4010430 con 0x7f9b04104130 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.214+0000 7f9afb7fe700 1 -- 172.21.15.88:0/1131889061 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9af4018630 con 0x7f9b04104130 2024-02-19T19:07:24.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.214+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1131889061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b041947a0 con 0x7f9b04104130 2024-02-19T19:07:24.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.214+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1131889061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b041a6850 con 0x7f9b04104130 2024-02-19T19:07:24.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.215+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1131889061 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b041076a0 con 0x7f9b04104130 2024-02-19T19:07:24.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.216+0000 7f9afb7fe700 1 -- 172.21.15.88:0/1131889061 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9af40105a0 con 0x7f9b04104130 2024-02-19T19:07:24.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.216+0000 7f9afb7fe700 1 --2- 172.21.15.88:0/1131889061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9af006d8a0 0x7f9af006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:24.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.216+0000 7f9afb7fe700 1 -- 172.21.15.88:0/1131889061 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9af4014070 con 0x7f9b04104130 2024-02-19T19:07:24.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.216+0000 7f9b0a2d8700 1 --2- 172.21.15.88:0/1131889061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9af006d8a0 0x7f9af006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:24.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.217+0000 7f9b0a2d8700 1 --2- 172.21.15.88:0/1131889061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9af006d8a0 0x7f9af006fd50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9b00006010 tx=0x7f9b00005ad0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:24.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.223+0000 7f9afb7fe700 1 -- 172.21.15.88:0/1131889061 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9af4052d20 con 0x7f9b04104130 2024-02-19T19:07:24.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.553+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1131889061 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b04195080 con 0x7f9af006d8a0 2024-02-19T19:07:24.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.555+0000 7f9afb7fe700 1 -- 172.21.15.88:0/1131889061 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9b04195080 con 0x7f9af006d8a0 2024-02-19T19:07:24.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.559+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1131889061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9af006d8a0 msgr2=0x7f9af006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:24.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.559+0000 7f9b0c53c700 1 --2- 172.21.15.88:0/1131889061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9af006d8a0 0x7f9af006fd50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9b00006010 tx=0x7f9b00005ad0 comp rx=0 tx=0).stop 2024-02-19T19:07:24.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.559+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1131889061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b04104130 msgr2=0x7f9b04193f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:24.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.559+0000 7f9b0c53c700 1 --2- 172.21.15.88:0/1131889061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b04104130 0x7f9b04193f80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9af400e920 tx=0x7f9af400ec30 comp rx=0 tx=0).stop 2024-02-19T19:07:24.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.559+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1131889061 shutdown_connections 2024-02-19T19:07:24.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.559+0000 7f9b0c53c700 1 --2- 172.21.15.88:0/1131889061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9af006d8a0 0x7f9af006fd50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:24.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.559+0000 7f9b0c53c700 1 --2- 172.21.15.88:0/1131889061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b040ffbd0 0x7f9b04193a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:24.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.559+0000 7f9b0c53c700 1 --2- 172.21.15.88:0/1131889061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b04104130 0x7f9b04193f80 secure :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9af400e920 tx=0x7f9af400ec30 comp rx=0 tx=0).stop 2024-02-19T19:07:24.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.559+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1131889061 >> 172.21.15.88:0/1131889061 conn(0x7f9b04075130 msgr2=0x7f9b040feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:24.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.561+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1131889061 shutdown_connections 2024-02-19T19:07:24.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.561+0000 7f9b0c53c700 1 -- 172.21.15.88:0/1131889061 wait complete. 2024-02-19T19:07:24.601 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:07:24.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.737+0000 7f49ee277700 1 -- 172.21.15.88:0/3595005416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e80687a0 msgr2=0x7f49e8068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:24.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.738+0000 7f49ee277700 1 --2- 172.21.15.88:0/3595005416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e80687a0 0x7f49e8068b70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f49e000b0a0 tx=0x7f49e000b3b0 comp rx=0 tx=0).stop 2024-02-19T19:07:24.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.738+0000 7f49ee277700 1 -- 172.21.15.88:0/3595005416 shutdown_connections 2024-02-19T19:07:24.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.738+0000 7f49ee277700 1 --2- 172.21.15.88:0/3595005416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e8069140 0x7f49e8104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:24.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.738+0000 7f49ee277700 1 --2- 172.21.15.88:0/3595005416 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e80687a0 0x7f49e8068b70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:24.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.738+0000 7f49ee277700 1 -- 172.21.15.88:0/3595005416 >> 172.21.15.88:0/3595005416 conn(0x7f49e8075130 msgr2=0x7f49e8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:24.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.739+0000 7f49ee277700 1 -- 172.21.15.88:0/3595005416 shutdown_connections 2024-02-19T19:07:24.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.740+0000 7f49ee277700 1 -- 172.21.15.88:0/3595005416 wait complete. 2024-02-19T19:07:24.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.741+0000 7f49ee277700 1 Processor -- start 2024-02-19T19:07:24.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.743+0000 7f49ee277700 1 -- start start 2024-02-19T19:07:24.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.743+0000 7f49ee277700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e80687a0 0x7f49e819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:24.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.744+0000 7f49ee277700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e8069140 0x7f49e819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:24.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.744+0000 7f49ee277700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e819e010 con 0x7f49e8069140 2024-02-19T19:07:24.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.744+0000 7f49ee277700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e81a6290 con 0x7f49e80687a0 2024-02-19T19:07:24.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.744+0000 7f49e7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e80687a0 0x7f49e819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:24.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.744+0000 7f49e7fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e80687a0 0x7f49e819d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:51832/0 (socket says 172.21.15.88:51832) 2024-02-19T19:07:24.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.744+0000 7f49e7fff700 1 -- 172.21.15.88:0/944528546 learned_addr learned my addr 172.21.15.88:0/944528546 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:24.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.745+0000 7f49e7fff700 1 -- 172.21.15.88:0/944528546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e8069140 msgr2=0x7f49e819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:24.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.745+0000 7f49e7fff700 1 --2- 172.21.15.88:0/944528546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e8069140 0x7f49e819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:24.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.745+0000 7f49e7fff700 1 -- 172.21.15.88:0/944528546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e0009d00 con 0x7f49e80687a0 2024-02-19T19:07:24.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.745+0000 7f49e7fff700 1 --2- 172.21.15.88:0/944528546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e80687a0 0x7f49e819d3f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009750 tx=0x7f49e000bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:24.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.746+0000 7f49e57fa700 1 -- 172.21.15.88:0/944528546 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e000e040 con 0x7f49e80687a0 2024-02-19T19:07:24.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.746+0000 7f49e57fa700 1 -- 172.21.15.88:0/944528546 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49e0003de0 con 0x7f49e80687a0 2024-02-19T19:07:24.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.746+0000 7f49e57fa700 1 -- 172.21.15.88:0/944528546 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e0004440 con 0x7f49e80687a0 2024-02-19T19:07:24.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.746+0000 7f49ee277700 1 -- 172.21.15.88:0/944528546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e81a6430 con 0x7f49e80687a0 2024-02-19T19:07:24.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.747+0000 7f49ee277700 1 -- 172.21.15.88:0/944528546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e81a68d0 con 0x7f49e80687a0 2024-02-19T19:07:24.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.749+0000 7f49e57fa700 1 -- 172.21.15.88:0/944528546 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f49e0019070 con 0x7f49e80687a0 2024-02-19T19:07:24.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.750+0000 7f49e57fa700 1 --2- 172.21.15.88:0/944528546 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49d006d940 0x7f49d006fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:24.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.750+0000 7f49e57fa700 1 -- 172.21.15.88:0/944528546 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f49e008dfb0 con 0x7f49e80687a0 2024-02-19T19:07:24.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.752+0000 7f49e77fe700 1 --2- 172.21.15.88:0/944528546 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49d006d940 0x7f49d006fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:24.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.753+0000 7f49ee277700 1 -- 172.21.15.88:0/944528546 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e810d430 con 0x7f49e80687a0 2024-02-19T19:07:24.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.755+0000 7f49e77fe700 1 --2- 172.21.15.88:0/944528546 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49d006d940 0x7f49d006fdf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49e8069620 tx=0x7f49dc006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:24.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:24.762+0000 7f49e57fa700 1 -- 172.21.15.88:0/944528546 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49e0057930 con 0x7f49e80687a0 2024-02-19T19:07:25.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.091+0000 7f49ee277700 1 -- 172.21.15.88:0/944528546 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49e810a940 con 0x7f49d006d940 2024-02-19T19:07:25.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.093+0000 7f49e57fa700 1 -- 172.21.15.88:0/944528546 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f49e810a940 con 0x7f49d006d940 2024-02-19T19:07:25.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.095+0000 7f49ee277700 1 -- 172.21.15.88:0/944528546 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49d006d940 msgr2=0x7f49d006fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:25.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.095+0000 7f49ee277700 1 --2- 172.21.15.88:0/944528546 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49d006d940 0x7f49d006fdf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49e8069620 tx=0x7f49dc006d20 comp rx=0 tx=0).stop 2024-02-19T19:07:25.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.095+0000 7f49ee277700 1 -- 172.21.15.88:0/944528546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e80687a0 msgr2=0x7f49e819d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:25.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.095+0000 7f49ee277700 1 --2- 172.21.15.88:0/944528546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e80687a0 0x7f49e819d3f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009750 tx=0x7f49e000bda0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.095+0000 7f49ee277700 1 -- 172.21.15.88:0/944528546 shutdown_connections 2024-02-19T19:07:25.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.095+0000 7f49ee277700 1 --2- 172.21.15.88:0/944528546 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49d006d940 0x7f49d006fdf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.095+0000 7f49ee277700 1 --2- 172.21.15.88:0/944528546 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49e80687a0 0x7f49e819d3f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.095+0000 7f49ee277700 1 --2- 172.21.15.88:0/944528546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49e8069140 0x7f49e819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.095+0000 7f49ee277700 1 -- 172.21.15.88:0/944528546 >> 172.21.15.88:0/944528546 conn(0x7f49e8075130 msgr2=0x7f49e80fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:25.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.096+0000 7f49ee277700 1 -- 172.21.15.88:0/944528546 shutdown_connections 2024-02-19T19:07:25.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.096+0000 7f49ee277700 1 -- 172.21.15.88:0/944528546 wait complete. 2024-02-19T19:07:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.234+0000 7f6fd6eac700 1 -- 172.21.15.88:0/415734904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd0100e60 msgr2=0x7f6fd0104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.234+0000 7f6fd6eac700 1 --2- 172.21.15.88:0/415734904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd0100e60 0x7f6fd0104e30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc009910 tx=0x7f6fbc009c20 comp rx=0 tx=0).stop 2024-02-19T19:07:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.234+0000 7f6fd6eac700 1 -- 172.21.15.88:0/415734904 shutdown_connections 2024-02-19T19:07:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.234+0000 7f6fd6eac700 1 --2- 172.21.15.88:0/415734904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd0100e60 0x7f6fd0104e30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.234+0000 7f6fd6eac700 1 --2- 172.21.15.88:0/415734904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fd01004c0 0x7f6fd0100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.234+0000 7f6fd6eac700 1 -- 172.21.15.88:0/415734904 >> 172.21.15.88:0/415734904 conn(0x7f6fd00fbd80 msgr2=0x7f6fd00fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.234+0000 7f6fd6eac700 1 -- 172.21.15.88:0/415734904 shutdown_connections 2024-02-19T19:07:25.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.234+0000 7f6fd6eac700 1 -- 172.21.15.88:0/415734904 wait complete. 2024-02-19T19:07:25.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.235+0000 7f6fd6eac700 1 Processor -- start 2024-02-19T19:07:25.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.235+0000 7f6fd6eac700 1 -- start start 2024-02-19T19:07:25.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.235+0000 7f6fd6eac700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd01004c0 0x7f6fd01960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:25.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.235+0000 7f6fd6eac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fd0196620 0x7f6fd019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:25.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.235+0000 7f6fd6eac700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0196c30 con 0x7f6fd01004c0 2024-02-19T19:07:25.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.235+0000 7f6fd6eac700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd0196da0 con 0x7f6fd0196620 2024-02-19T19:07:25.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.235+0000 7f6fd4c48700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd01004c0 0x7f6fd01960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:25.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.235+0000 7f6fd4c48700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd01004c0 0x7f6fd01960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54772/0 (socket says 172.21.15.88:54772) 2024-02-19T19:07:25.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.235+0000 7f6fd4c48700 1 -- 172.21.15.88:0/3282617714 learned_addr learned my addr 172.21.15.88:0/3282617714 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.236+0000 7f6fcffff700 1 --2- 172.21.15.88:0/3282617714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fd0196620 0x7f6fd019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.236+0000 7f6fd4c48700 1 -- 172.21.15.88:0/3282617714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fd0196620 msgr2=0x7f6fd019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.236+0000 7f6fd4c48700 1 --2- 172.21.15.88:0/3282617714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fd0196620 0x7f6fd019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.236+0000 7f6fd4c48700 1 -- 172.21.15.88:0/3282617714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0095f0 con 0x7f6fd01004c0 2024-02-19T19:07:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.236+0000 7f6fd4c48700 1 --2- 172.21.15.88:0/3282617714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd01004c0 0x7f6fd01960e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400e950 tx=0x7f6fc400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.236+0000 7f6fcdffb700 1 -- 172.21.15.88:0/3282617714 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc4009790 con 0x7f6fd01004c0 2024-02-19T19:07:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.236+0000 7f6fd6eac700 1 -- 172.21.15.88:0/3282617714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd019b090 con 0x7f6fd01004c0 2024-02-19T19:07:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.237+0000 7f6fd6eac700 1 -- 172.21.15.88:0/3282617714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd019b5e0 con 0x7f6fd01004c0 2024-02-19T19:07:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.237+0000 7f6fcdffb700 1 -- 172.21.15.88:0/3282617714 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fc400cc80 con 0x7f6fd01004c0 2024-02-19T19:07:25.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.237+0000 7f6fcdffb700 1 -- 172.21.15.88:0/3282617714 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fc401fe20 con 0x7f6fd01004c0 2024-02-19T19:07:25.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.238+0000 7f6fd6eac700 1 -- 172.21.15.88:0/3282617714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb4005320 con 0x7f6fd01004c0 2024-02-19T19:07:25.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.238+0000 7f6fcdffb700 1 -- 172.21.15.88:0/3282617714 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6fc40283b0 con 0x7f6fd01004c0 2024-02-19T19:07:25.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.239+0000 7f6fcdffb700 1 --2- 172.21.15.88:0/3282617714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fc006d8a0 0x7f6fc006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:25.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.239+0000 7f6fcffff700 1 --2- 172.21.15.88:0/3282617714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fc006d8a0 0x7f6fc006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:25.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.239+0000 7f6fcdffb700 1 -- 172.21.15.88:0/3282617714 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6fc4095bd0 con 0x7f6fd01004c0 2024-02-19T19:07:25.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.240+0000 7f6fcffff700 1 --2- 172.21.15.88:0/3282617714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fc006d8a0 0x7f6fc006fd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc000c00 tx=0x7f6fbc005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:25.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.244+0000 7f6fcdffb700 1 -- 172.21.15.88:0/3282617714 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6fc4064140 con 0x7f6fd01004c0 2024-02-19T19:07:25.475 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:25 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[100756]: 2024-02-19T19:07:25.289+0000 7efd9a66e7c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-02-19T19:07:25.475 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:25 smithi088 ceph-mon[88643]: pgmap v78: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 8.6 KiB/s rd, 465 KiB/s wr, 128 op/s; 1793/11877 objects degraded (15.096%) 2024-02-19T19:07:25.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:25 smithi132 ceph-mon[70676]: pgmap v78: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 279 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 8.6 KiB/s rd, 465 KiB/s wr, 128 op/s; 1793/11877 objects degraded (15.096%) 2024-02-19T19:07:25.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.534+0000 7f6fd6eac700 1 -- 172.21.15.88:0/3282617714 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fb4000bf0 con 0x7f6fc006d8a0 2024-02-19T19:07:25.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.550+0000 7f6fcdffb700 1 -- 172.21.15.88:0/3282617714 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f6fb4000bf0 con 0x7f6fc006d8a0 2024-02-19T19:07:25.571 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:07:25.571 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (4m) 28s ago 15m 24.2M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:07:25.571 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (15m) 28s ago 15m 8140k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:07:25.571 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (13m) 31s ago 13m 8010k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:07:25.571 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (43s) 28s ago 15m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:07:25.571 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (34s) 31s ago 13m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:07:25.571 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (3m) 28s ago 14m 87.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:07:25.571 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (8m) 28s ago 8m 16.3M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:07:25.571 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (8m) 28s ago 8m 121M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:07:25.571 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (8m) 31s ago 8m 139M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (8m) 31s ago 8m 20.2M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (6m) 28s ago 16m 496M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (5m) 31s ago 13m 450M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (2m) 28s ago 16m 47.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (115s) 31s ago 13m 37.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (5m) 28s ago 15m 15.9M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (5m) 31s ago 13m 15.8M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 starting - - - 1216M 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (12m) 28s ago 12m 223M 1216M 18.2.0 dc2bc1663786 8b42e7ed1c67 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (11m) 28s ago 11m 180M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (11m) 31s ago 11m 236M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:07:25.572 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (10m) 31s ago 10m 215M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:07:25.573 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (10m) 31s ago 10m 201M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:07:25.573 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (4m) 28s ago 13m 69.2M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:07:25.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.552+0000 7f6fd6eac700 1 -- 172.21.15.88:0/3282617714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fc006d8a0 msgr2=0x7f6fc006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:25.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.552+0000 7f6fd6eac700 1 --2- 172.21.15.88:0/3282617714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fc006d8a0 0x7f6fc006fd50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc000c00 tx=0x7f6fbc005940 comp rx=0 tx=0).stop 2024-02-19T19:07:25.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.552+0000 7f6fd6eac700 1 -- 172.21.15.88:0/3282617714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd01004c0 msgr2=0x7f6fd01960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:25.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.552+0000 7f6fd6eac700 1 --2- 172.21.15.88:0/3282617714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd01004c0 0x7f6fd01960e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400e950 tx=0x7f6fc400ed10 comp rx=0 tx=0).stop 2024-02-19T19:07:25.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.552+0000 7f6fd6eac700 1 -- 172.21.15.88:0/3282617714 shutdown_connections 2024-02-19T19:07:25.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.552+0000 7f6fd6eac700 1 --2- 172.21.15.88:0/3282617714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fc006d8a0 0x7f6fc006fd50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.552+0000 7f6fd6eac700 1 --2- 172.21.15.88:0/3282617714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd01004c0 0x7f6fd01960e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.552+0000 7f6fd6eac700 1 --2- 172.21.15.88:0/3282617714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6fd0196620 0x7f6fd019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.552+0000 7f6fd6eac700 1 -- 172.21.15.88:0/3282617714 >> 172.21.15.88:0/3282617714 conn(0x7f6fd00fbd80 msgr2=0x7f6fd00fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:25.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.553+0000 7f6fd6eac700 1 -- 172.21.15.88:0/3282617714 shutdown_connections 2024-02-19T19:07:25.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.553+0000 7f6fd6eac700 1 -- 172.21.15.88:0/3282617714 wait complete. 2024-02-19T19:07:25.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.687+0000 7f037f9fb700 1 -- 172.21.15.88:0/3605811930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03780ffd60 msgr2=0x7f0378103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:25.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.687+0000 7f037f9fb700 1 --2- 172.21.15.88:0/3605811930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03780ffd60 0x7f0378103d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0368009910 tx=0x7f0368009c20 comp rx=0 tx=0).stop 2024-02-19T19:07:25.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.687+0000 7f037f9fb700 1 -- 172.21.15.88:0/3605811930 shutdown_connections 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.687+0000 7f037f9fb700 1 --2- 172.21.15.88:0/3605811930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03780ffd60 0x7f0378103d30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.687+0000 7f037f9fb700 1 --2- 172.21.15.88:0/3605811930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03780ff3c0 0x7f03780ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.687+0000 7f037f9fb700 1 -- 172.21.15.88:0/3605811930 >> 172.21.15.88:0/3605811930 conn(0x7f03780fac60 msgr2=0x7f03780fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.688+0000 7f037f9fb700 1 -- 172.21.15.88:0/3605811930 shutdown_connections 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.688+0000 7f037f9fb700 1 -- 172.21.15.88:0/3605811930 wait complete. 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.688+0000 7f037f9fb700 1 Processor -- start 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.688+0000 7f037f9fb700 1 -- start start 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.688+0000 7f037f9fb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03780ff3c0 0x7f0378195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.688+0000 7f037f9fb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03780ffd60 0x7f03781963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f037f9fb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0378196ac0 con 0x7f03780ffd60 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f037f9fb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037819a850 con 0x7f03780ff3c0 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f037cf96700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03780ffd60 0x7f03781963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f037cf96700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03780ffd60 0x7f03781963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60718/0 (socket says 172.21.15.88:60718) 2024-02-19T19:07:25.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f037cf96700 1 -- 172.21.15.88:0/2095171786 learned_addr learned my addr 172.21.15.88:0/2095171786 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:25.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f037cf96700 1 -- 172.21.15.88:0/2095171786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03780ff3c0 msgr2=0x7f0378195ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-19T19:07:25.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f037cf96700 1 --2- 172.21.15.88:0/2095171786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03780ff3c0 0x7f0378195ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:25.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f037cf96700 1 -- 172.21.15.88:0/2095171786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03680095f0 con 0x7f03780ffd60 2024-02-19T19:07:25.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f037cf96700 1 --2- 172.21.15.88:0/2095171786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03780ffd60 0x7f03781963e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f03680092e0 tx=0x7f036800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:25.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f036e7fc700 1 -- 172.21.15.88:0/2095171786 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f036801d070 con 0x7f03780ffd60 2024-02-19T19:07:25.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f037f9fb700 1 -- 172.21.15.88:0/2095171786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037819aad0 con 0x7f03780ffd60 2024-02-19T19:07:25.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f036e7fc700 1 -- 172.21.15.88:0/2095171786 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0368005dc0 con 0x7f03780ffd60 2024-02-19T19:07:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.689+0000 7f036e7fc700 1 -- 172.21.15.88:0/2095171786 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0368017400 con 0x7f03780ffd60 2024-02-19T19:07:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.690+0000 7f037f9fb700 1 -- 172.21.15.88:0/2095171786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037819afc0 con 0x7f03780ffd60 2024-02-19T19:07:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.691+0000 7f037f9fb700 1 -- 172.21.15.88:0/2095171786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f037804f2a0 con 0x7f03780ffd60 2024-02-19T19:07:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.691+0000 7f036e7fc700 1 -- 172.21.15.88:0/2095171786 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f036800f460 con 0x7f03780ffd60 2024-02-19T19:07:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.692+0000 7f036e7fc700 1 --2- 172.21.15.88:0/2095171786 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f036406d8a0 0x7f036406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.692+0000 7f036e7fc700 1 -- 172.21.15.88:0/2095171786 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f036808e960 con 0x7f03780ffd60 2024-02-19T19:07:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.692+0000 7f037d797700 1 --2- 172.21.15.88:0/2095171786 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f036406d8a0 0x7f036406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.692+0000 7f037d797700 1 --2- 172.21.15.88:0/2095171786 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f036406d8a0 0x7f036406fd50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f037400ba30 tx=0x7f037400b3f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:25.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:25.698+0000 7f036e7fc700 1 -- 172.21.15.88:0/2095171786 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f03680581c0 con 0x7f03780ffd60 2024-02-19T19:07:26.101 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:25 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[100756]: 2024-02-19T19:07:25.893+0000 7efd9a66e7c0 -1 osd.0 49 log_to_monitors true 2024-02-19T19:07:26.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.119+0000 7f037f9fb700 1 -- 172.21.15.88:0/2095171786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f037819b2a0 con 0x7f03780ffd60 2024-02-19T19:07:26.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.120+0000 7f036e7fc700 1 -- 172.21.15.88:0/2095171786 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+687 (secure 0 0 0) 0x7f036805b880 con 0x7f03780ffd60 2024-02-19T19:07:26.140 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:07:26.141 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 9, 2024-02-19T19:07:26.142 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:07:26.142 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:07:26.142 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:07:26.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.122+0000 7f037f9fb700 1 -- 172.21.15.88:0/2095171786 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f036406d8a0 msgr2=0x7f036406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:26.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.122+0000 7f037f9fb700 1 --2- 172.21.15.88:0/2095171786 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f036406d8a0 0x7f036406fd50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f037400ba30 tx=0x7f037400b3f0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.122+0000 7f037f9fb700 1 -- 172.21.15.88:0/2095171786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03780ffd60 msgr2=0x7f03781963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:26.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.122+0000 7f037f9fb700 1 --2- 172.21.15.88:0/2095171786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03780ffd60 0x7f03781963e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f03680092e0 tx=0x7f036800be20 comp rx=0 tx=0).stop 2024-02-19T19:07:26.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.122+0000 7f037f9fb700 1 -- 172.21.15.88:0/2095171786 shutdown_connections 2024-02-19T19:07:26.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.122+0000 7f037f9fb700 1 --2- 172.21.15.88:0/2095171786 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f036406d8a0 0x7f036406fd50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.122+0000 7f037f9fb700 1 --2- 172.21.15.88:0/2095171786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03780ff3c0 0x7f0378195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.122+0000 7f037f9fb700 1 --2- 172.21.15.88:0/2095171786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03780ffd60 0x7f03781963e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.122+0000 7f037f9fb700 1 -- 172.21.15.88:0/2095171786 >> 172.21.15.88:0/2095171786 conn(0x7f03780fac60 msgr2=0x7f03781034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:26.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.122+0000 7f037f9fb700 1 -- 172.21.15.88:0/2095171786 shutdown_connections 2024-02-19T19:07:26.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.122+0000 7f037f9fb700 1 -- 172.21.15.88:0/2095171786 wait complete. 2024-02-19T19:07:26.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.250+0000 7fcc985d8700 1 -- 172.21.15.88:0/2074663978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc900ffc60 msgr2=0x7fcc9010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.250+0000 7fcc985d8700 1 --2- 172.21.15.88:0/2074663978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc900ffc60 0x7fcc9010a6d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009910 tx=0x7fcc80009c20 comp rx=0 tx=0).stop 2024-02-19T19:07:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.251+0000 7fcc985d8700 1 -- 172.21.15.88:0/2074663978 shutdown_connections 2024-02-19T19:07:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.251+0000 7fcc985d8700 1 --2- 172.21.15.88:0/2074663978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc900ffc60 0x7fcc9010a6d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.251+0000 7fcc985d8700 1 --2- 172.21.15.88:0/2074663978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc900ff350 0x7fcc900ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.251+0000 7fcc985d8700 1 -- 172.21.15.88:0/2074663978 >> 172.21.15.88:0/2074663978 conn(0x7fcc900748b0 msgr2=0x7fcc90074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.251+0000 7fcc985d8700 1 -- 172.21.15.88:0/2074663978 shutdown_connections 2024-02-19T19:07:26.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.251+0000 7fcc985d8700 1 -- 172.21.15.88:0/2074663978 wait complete. 2024-02-19T19:07:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.251+0000 7fcc985d8700 1 Processor -- start 2024-02-19T19:07:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.251+0000 7fcc985d8700 1 -- start start 2024-02-19T19:07:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc985d8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc900ff350 0x7fcc90198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc985d8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc900ffc60 0x7fcc901985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc985d8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc90198c90 con 0x7fcc900ffc60 2024-02-19T19:07:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc985d8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9019ca20 con 0x7fcc900ff350 2024-02-19T19:07:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc95b73700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc900ffc60 0x7fcc901985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc95b73700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc900ffc60 0x7fcc901985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60740/0 (socket says 172.21.15.88:60740) 2024-02-19T19:07:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc95b73700 1 -- 172.21.15.88:0/2216007036 learned_addr learned my addr 172.21.15.88:0/2216007036 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:26.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc96374700 1 --2- 172.21.15.88:0/2216007036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc900ff350 0x7fcc90198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc95b73700 1 -- 172.21.15.88:0/2216007036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc900ff350 msgr2=0x7fcc90198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc95b73700 1 --2- 172.21.15.88:0/2216007036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc900ff350 0x7fcc90198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc95b73700 1 -- 172.21.15.88:0/2216007036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c009520 con 0x7fcc900ffc60 2024-02-19T19:07:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc95b73700 1 --2- 172.21.15.88:0/2216007036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc900ffc60 0x7fcc901985b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009fd0 tx=0x7fcc8000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc877fe700 1 -- 172.21.15.88:0/2216007036 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc8001d070 con 0x7fcc900ffc60 2024-02-19T19:07:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc985d8700 1 -- 172.21.15.88:0/2216007036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc800095f0 con 0x7fcc900ffc60 2024-02-19T19:07:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc877fe700 1 -- 172.21.15.88:0/2216007036 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc800050d0 con 0x7fcc900ffc60 2024-02-19T19:07:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc877fe700 1 -- 172.21.15.88:0/2216007036 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc80017400 con 0x7fcc900ffc60 2024-02-19T19:07:26.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.252+0000 7fcc985d8700 1 -- 172.21.15.88:0/2216007036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9019d000 con 0x7fcc900ffc60 2024-02-19T19:07:26.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.254+0000 7fcc985d8700 1 -- 172.21.15.88:0/2216007036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc9010a230 con 0x7fcc900ffc60 2024-02-19T19:07:26.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.254+0000 7fcc877fe700 1 -- 172.21.15.88:0/2216007036 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcc800176c0 con 0x7fcc900ffc60 2024-02-19T19:07:26.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.255+0000 7fcc877fe700 1 --2- 172.21.15.88:0/2216007036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc7c06d8a0 0x7fcc7c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:26.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.255+0000 7fcc877fe700 1 -- 172.21.15.88:0/2216007036 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fcc8008e7d0 con 0x7fcc900ffc60 2024-02-19T19:07:26.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.255+0000 7fcc96374700 1 --2- 172.21.15.88:0/2216007036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc7c06d8a0 0x7fcc7c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:26.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.255+0000 7fcc96374700 1 --2- 172.21.15.88:0/2216007036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc7c06d8a0 0x7fcc7c06fd50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009df0 tx=0x7fcc8c009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:26.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.261+0000 7fcc877fe700 1 -- 172.21.15.88:0/2216007036 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc800580a0 con 0x7fcc900ffc60 2024-02-19T19:07:26.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:26 smithi132 ceph-mon[70676]: from='client.44179 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:26.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:26 smithi132 ceph-mon[70676]: from='client.44183 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:26.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:26 smithi132 ceph-mon[70676]: from='osd.0 [v2:172.21.15.88:6802/4275403566,v1:172.21.15.88:6803/4275403566]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-19T19:07:26.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:26 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2095171786' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:26.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:26.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:26.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.549+0000 7fcc985d8700 1 -- 172.21.15.88:0/2216007036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcc9004f2a0 con 0x7fcc900ffc60 2024-02-19T19:07:26.570 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 19:07:26 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[100756]: 2024-02-19T19:07:26.209+0000 7efd8bac6700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T19:07:26.571 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:26 smithi088 ceph-mon[88643]: from='client.44179 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:26.571 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:26 smithi088 ceph-mon[88643]: from='client.44183 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:26.571 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:26 smithi088 ceph-mon[88643]: from='osd.0 [v2:172.21.15.88:6802/4275403566,v1:172.21.15.88:6803/4275403566]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-19T19:07:26.571 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:26 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2095171786' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:26.571 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:26.571 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:26.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.550+0000 7fcc877fe700 1 -- 172.21.15.88:0/2216007036 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1892 (secure 0 0 0) 0x7fcc8008ce60 con 0x7fcc900ffc60 2024-02-19T19:07:26.572 INFO:teuthology.orchestra.run.smithi088.stdout:e20 2024-02-19T19:07:26.572 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:07:26.572 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:07:26.572 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:07:26.572 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:07:26.572 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:07:26.572 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:07:26.572 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 20 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:05:14.913778+0000 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:07:26.573 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:07:26.574 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:07:26.575 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:07:26.575 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:07:26.575 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:07:26.575 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:07:26.575 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:07:26.575 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:07:26.575 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.553+0000 7fcc985d8700 1 -- 172.21.15.88:0/2216007036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc7c06d8a0 msgr2=0x7fcc7c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.553+0000 7fcc985d8700 1 --2- 172.21.15.88:0/2216007036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc7c06d8a0 0x7fcc7c06fd50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009df0 tx=0x7fcc8c009260 comp rx=0 tx=0).stop 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.553+0000 7fcc985d8700 1 -- 172.21.15.88:0/2216007036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc900ffc60 msgr2=0x7fcc901985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.553+0000 7fcc985d8700 1 --2- 172.21.15.88:0/2216007036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc900ffc60 0x7fcc901985b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcc80009fd0 tx=0x7fcc8000bfd0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.553+0000 7fcc985d8700 1 -- 172.21.15.88:0/2216007036 shutdown_connections 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.553+0000 7fcc985d8700 1 --2- 172.21.15.88:0/2216007036 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc7c06d8a0 0x7fcc7c06fd50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.553+0000 7fcc985d8700 1 --2- 172.21.15.88:0/2216007036 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc900ff350 0x7fcc90198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.553+0000 7fcc985d8700 1 --2- 172.21.15.88:0/2216007036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc900ffc60 0x7fcc901985b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.553+0000 7fcc985d8700 1 -- 172.21.15.88:0/2216007036 >> 172.21.15.88:0/2216007036 conn(0x7fcc900748b0 msgr2=0x7fcc90100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.553+0000 7fcc985d8700 1 -- 172.21.15.88:0/2216007036 shutdown_connections 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.553+0000 7fcc985d8700 1 -- 172.21.15.88:0/2216007036 wait complete. 2024-02-19T19:07:26.576 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-02-19T19:07:26.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.701+0000 7f62bf76c700 1 -- 172.21.15.88:0/441470523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b80ffd60 msgr2=0x7f62b8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:26.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.701+0000 7f62bf76c700 1 --2- 172.21.15.88:0/441470523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b80ffd60 0x7f62b8103d30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62a8009910 tx=0x7f62a8009c20 comp rx=0 tx=0).stop 2024-02-19T19:07:26.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.701+0000 7f62bf76c700 1 -- 172.21.15.88:0/441470523 shutdown_connections 2024-02-19T19:07:26.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.701+0000 7f62bf76c700 1 --2- 172.21.15.88:0/441470523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b80ffd60 0x7f62b8103d30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.701+0000 7f62bf76c700 1 --2- 172.21.15.88:0/441470523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b80ff3c0 0x7f62b80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.701+0000 7f62bf76c700 1 -- 172.21.15.88:0/441470523 >> 172.21.15.88:0/441470523 conn(0x7f62b80fac60 msgr2=0x7f62b80fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:26.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.701+0000 7f62bf76c700 1 -- 172.21.15.88:0/441470523 shutdown_connections 2024-02-19T19:07:26.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.701+0000 7f62bf76c700 1 -- 172.21.15.88:0/441470523 wait complete. 2024-02-19T19:07:26.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.702+0000 7f62bf76c700 1 Processor -- start 2024-02-19T19:07:26.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.702+0000 7f62bf76c700 1 -- start start 2024-02-19T19:07:26.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.702+0000 7f62bf76c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b80ff3c0 0x7f62b8194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:26.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.702+0000 7f62bf76c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b80ffd60 0x7f62b81952f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:26.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.702+0000 7f62bf76c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b81959d0 con 0x7f62b80ffd60 2024-02-19T19:07:26.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.702+0000 7f62bf76c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b81996f0 con 0x7f62b80ff3c0 2024-02-19T19:07:26.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62bcd07700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b80ffd60 0x7f62b81952f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:26.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62bcd07700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b80ffd60 0x7f62b81952f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60750/0 (socket says 172.21.15.88:60750) 2024-02-19T19:07:26.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62bcd07700 1 -- 172.21.15.88:0/557946722 learned_addr learned my addr 172.21.15.88:0/557946722 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:26.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62bd508700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b80ff3c0 0x7f62b8194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:26.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62bd508700 1 -- 172.21.15.88:0/557946722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b80ffd60 msgr2=0x7f62b81952f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:26.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62bd508700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b80ffd60 0x7f62b81952f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:26.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62bd508700 1 -- 172.21.15.88:0/557946722 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a80095f0 con 0x7f62b80ff3c0 2024-02-19T19:07:26.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62bcd07700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b80ffd60 0x7f62b81952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:07:26.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62bd508700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b80ff3c0 0x7f62b8194db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f62b400e970 tx=0x7f62b400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:26.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62ae7fc700 1 -- 172.21.15.88:0/557946722 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b400c9d0 con 0x7f62b80ff3c0 2024-02-19T19:07:26.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62bf76c700 1 -- 172.21.15.88:0/557946722 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b81999d0 con 0x7f62b80ff3c0 2024-02-19T19:07:26.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.703+0000 7f62ae7fc700 1 -- 172.21.15.88:0/557946722 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62b4010430 con 0x7f62b80ff3c0 2024-02-19T19:07:26.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.704+0000 7f62ae7fc700 1 -- 172.21.15.88:0/557946722 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62b40185f0 con 0x7f62b80ff3c0 2024-02-19T19:07:26.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.704+0000 7f62bf76c700 1 -- 172.21.15.88:0/557946722 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b8199f20 con 0x7f62b80ff3c0 2024-02-19T19:07:26.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.705+0000 7f62bf76c700 1 -- 172.21.15.88:0/557946722 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b8107730 con 0x7f62b80ff3c0 2024-02-19T19:07:26.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.705+0000 7f62ae7fc700 1 -- 172.21.15.88:0/557946722 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f62b40105a0 con 0x7f62b80ff3c0 2024-02-19T19:07:26.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.705+0000 7f62ae7fc700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62a406d8a0 0x7f62a406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:26.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.705+0000 7f62ae7fc700 1 -- 172.21.15.88:0/557946722 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f62b4014070 con 0x7f62b80ff3c0 2024-02-19T19:07:26.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.705+0000 7f62bcd07700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62a406d8a0 0x7f62a406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:26.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.706+0000 7f62bcd07700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62a406d8a0 0x7f62a406fd50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62a800b760 tx=0x7f62a8005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:26.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.710+0000 7f62ae7fc700 1 -- 172.21.15.88:0/557946722 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f62b4052d20 con 0x7f62b80ff3c0 2024-02-19T19:07:27.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.987+0000 7f62bf76c700 1 -- 172.21.15.88:0/557946722 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62b819a190 con 0x7f62a406d8a0 2024-02-19T19:07:27.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.988+0000 7f62ae7fc700 1 -- 172.21.15.88:0/557946722 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f62b819a190 con 0x7f62a406d8a0 2024-02-19T19:07:27.010 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:07:27.010 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:07:27.010 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:07:27.010 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:07:27.010 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:07:27.010 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:07:27.010 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:07:27.010 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:07:27.010 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:07:27.010 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "7/23 daemons upgraded", 2024-02-19T19:07:27.010 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading osd daemons", 2024-02-19T19:07:27.011 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:07:27.011 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:07:27.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.991+0000 7f62bf76c700 1 -- 172.21.15.88:0/557946722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62a406d8a0 msgr2=0x7f62a406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.991+0000 7f62bf76c700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62a406d8a0 0x7f62a406fd50 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f62a800b760 tx=0x7f62a8005940 comp rx=0 tx=0).stop 2024-02-19T19:07:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.991+0000 7f62bf76c700 1 -- 172.21.15.88:0/557946722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b80ff3c0 msgr2=0x7f62b8194db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.991+0000 7f62bf76c700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b80ff3c0 0x7f62b8194db0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f62b400e970 tx=0x7f62b400ec80 comp rx=0 tx=0).stop 2024-02-19T19:07:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.991+0000 7f62bf76c700 1 -- 172.21.15.88:0/557946722 shutdown_connections 2024-02-19T19:07:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.991+0000 7f62bf76c700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62a406d8a0 0x7f62a406fd50 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.991+0000 7f62bf76c700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62b80ff3c0 0x7f62b8194db0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.991+0000 7f62bf76c700 1 --2- 172.21.15.88:0/557946722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b80ffd60 0x7f62b81952f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.991+0000 7f62bf76c700 1 -- 172.21.15.88:0/557946722 >> 172.21.15.88:0/557946722 conn(0x7f62b80fac60 msgr2=0x7f62b80fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.991+0000 7f62bf76c700 1 -- 172.21.15.88:0/557946722 shutdown_connections 2024-02-19T19:07:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:26.991+0000 7f62bf76c700 1 -- 172.21.15.88:0/557946722 wait complete. 2024-02-19T19:07:27.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.111+0000 7f31d484f700 1 -- 172.21.15.88:0/1040939264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31d00ffbd0 msgr2=0x7f31d0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:27.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.111+0000 7f31d484f700 1 --2- 172.21.15.88:0/1040939264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31d00ffbd0 0x7f31d0100040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f31c4009910 tx=0x7f31c4009c20 comp rx=0 tx=0).stop 2024-02-19T19:07:27.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.111+0000 7f31d484f700 1 -- 172.21.15.88:0/1040939264 shutdown_connections 2024-02-19T19:07:27.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.111+0000 7f31d484f700 1 --2- 172.21.15.88:0/1040939264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31d00ffbd0 0x7f31d0100040 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:27.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.111+0000 7f31d484f700 1 --2- 172.21.15.88:0/1040939264 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31d0104130 0x7f31d0104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:27.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.111+0000 7f31d484f700 1 -- 172.21.15.88:0/1040939264 >> 172.21.15.88:0/1040939264 conn(0x7f31d0075130 msgr2=0x7f31d0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:27.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.111+0000 7f31d484f700 1 -- 172.21.15.88:0/1040939264 shutdown_connections 2024-02-19T19:07:27.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.111+0000 7f31d484f700 1 -- 172.21.15.88:0/1040939264 wait complete. 2024-02-19T19:07:27.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.112+0000 7f31d484f700 1 Processor -- start 2024-02-19T19:07:27.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.112+0000 7f31d484f700 1 -- start start 2024-02-19T19:07:27.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.112+0000 7f31d484f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31d00ffbd0 0x7f31d010a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:27.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.112+0000 7f31d484f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31d0104130 0x7f31d010a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:27.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.112+0000 7f31cdd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31d0104130 0x7f31d010a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:27.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.112+0000 7f31cdd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31d0104130 0x7f31d010a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60776/0 (socket says 172.21.15.88:60776) 2024-02-19T19:07:27.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.112+0000 7f31d484f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d010bf90 con 0x7f31d0104130 2024-02-19T19:07:27.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.112+0000 7f31cdd9b700 1 -- 172.21.15.88:0/2538656700 learned_addr learned my addr 172.21.15.88:0/2538656700 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:27.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.112+0000 7f31ce59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31d00ffbd0 0x7f31d010a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:27.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.112+0000 7f31d484f700 1 -- 172.21.15.88:0/2538656700 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d010aab0 con 0x7f31d00ffbd0 2024-02-19T19:07:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.113+0000 7f31ce59c700 1 -- 172.21.15.88:0/2538656700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31d0104130 msgr2=0x7f31d010a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.113+0000 7f31ce59c700 1 --2- 172.21.15.88:0/2538656700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31d0104130 0x7f31d010a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.113+0000 7f31ce59c700 1 -- 172.21.15.88:0/2538656700 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31c40095f0 con 0x7f31d00ffbd0 2024-02-19T19:07:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.113+0000 7f31ce59c700 1 --2- 172.21.15.88:0/2538656700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31d00ffbd0 0x7f31d010a030 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f31c000e970 tx=0x7f31c000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.113+0000 7f31bf7fe700 1 -- 172.21.15.88:0/2538656700 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31c000c9d0 con 0x7f31d00ffbd0 2024-02-19T19:07:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.113+0000 7f31bf7fe700 1 -- 172.21.15.88:0/2538656700 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31c0010430 con 0x7f31d00ffbd0 2024-02-19T19:07:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.113+0000 7f31bf7fe700 1 -- 172.21.15.88:0/2538656700 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31c0018630 con 0x7f31d00ffbd0 2024-02-19T19:07:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.113+0000 7f31d484f700 1 -- 172.21.15.88:0/2538656700 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31d010ad90 con 0x7f31d00ffbd0 2024-02-19T19:07:27.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.113+0000 7f31d484f700 1 -- 172.21.15.88:0/2538656700 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31d01a21d0 con 0x7f31d00ffbd0 2024-02-19T19:07:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.114+0000 7f31d484f700 1 -- 172.21.15.88:0/2538656700 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31d01076a0 con 0x7f31d00ffbd0 2024-02-19T19:07:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.115+0000 7f31bf7fe700 1 -- 172.21.15.88:0/2538656700 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f31c000cb30 con 0x7f31d00ffbd0 2024-02-19T19:07:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.115+0000 7f31bf7fe700 1 --2- 172.21.15.88:0/2538656700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31b806d8a0 0x7f31b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.115+0000 7f31bf7fe700 1 -- 172.21.15.88:0/2538656700 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f31c0014070 con 0x7f31d00ffbd0 2024-02-19T19:07:27.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.115+0000 7f31cdd9b700 1 --2- 172.21.15.88:0/2538656700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31b806d8a0 0x7f31b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:27.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.116+0000 7f31cdd9b700 1 --2- 172.21.15.88:0/2538656700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31b806d8a0 0x7f31b806fd50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31c4005850 tx=0x7f31c4005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:27.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.120+0000 7f31bf7fe700 1 -- 172.21.15.88:0/2538656700 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31c0052d20 con 0x7f31d00ffbd0 2024-02-19T19:07:27.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:27 smithi088 ceph-mon[88643]: from='client.34246 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:27.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:27 smithi088 ceph-mon[88643]: pgmap v79: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 280 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 939 KiB/s wr, 247 op/s; 1291/8658 objects degraded (14.911%) 2024-02-19T19:07:27.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:27 smithi088 ceph-mon[88643]: from='osd.0 [v2:172.21.15.88:6802/4275403566,v1:172.21.15.88:6803/4275403566]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-19T19:07:27.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:27 smithi088 ceph-mon[88643]: osdmap e52: 6 total, 5 up, 6 in 2024-02-19T19:07:27.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:27 smithi088 ceph-mon[88643]: from='osd.0 [v2:172.21.15.88:6802/4275403566,v1:172.21.15.88:6803/4275403566]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T19:07:27.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:27 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2216007036' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:07:27.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:27 smithi088 ceph-mon[88643]: Health check update: Degraded data redundancy: 1291/8658 objects degraded (14.911%), 33 pgs degraded (PG_DEGRADED) 2024-02-19T19:07:27.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.471+0000 7f31d484f700 1 -- 172.21.15.88:0/2538656700 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f31d004f2a0 con 0x7f31d00ffbd0 2024-02-19T19:07:27.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.471+0000 7f31bf7fe700 1 -- 172.21.15.88:0/2538656700 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+2023 (secure 0 0 0) 0x7f31c0056f60 con 0x7f31d00ffbd0 2024-02-19T19:07:27.492 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_WARN Degraded data redundancy: 1291/8658 objects degraded (14.911%), 33 pgs degraded 2024-02-19T19:07:27.492 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 1291/8658 objects degraded (14.911%), 33 pgs degraded 2024-02-19T19:07:27.492 INFO:teuthology.orchestra.run.smithi088.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-02-19T19:07:27.492 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-02-19T19:07:27.492 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-02-19T19:07:27.492 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-02-19T19:07:27.492 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-02-19T19:07:27.493 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.1 is active+undersized+degraded, acting [4,2] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.3 is active+undersized+degraded, acting [4,3] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.6 is active+undersized+degraded, acting [1,4] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.b is active+undersized+degraded, acting [1,4] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.c is active+undersized+degraded, acting [5,3] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.f is active+undersized+degraded, acting [5,3] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.10 is active+undersized+degraded, acting [5,1] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.11 is active+undersized+degraded, acting [3,4] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.12 is active+undersized+degraded, acting [3,1] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.15 is active+undersized+degraded, acting [3,4] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.17 is active+undersized+degraded, acting [5,2] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.18 is active+undersized+degraded, acting [2,1] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.1b is active+undersized+degraded, acting [4,3] 2024-02-19T19:07:27.494 INFO:teuthology.orchestra.run.smithi088.stdout: pg 3.1f is active+undersized+degraded, acting [3,2] 2024-02-19T19:07:27.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.474+0000 7f31d484f700 1 -- 172.21.15.88:0/2538656700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31b806d8a0 msgr2=0x7f31b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:27.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.474+0000 7f31d484f700 1 --2- 172.21.15.88:0/2538656700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31b806d8a0 0x7f31b806fd50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f31c4005850 tx=0x7f31c4005dc0 comp rx=0 tx=0).stop 2024-02-19T19:07:27.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.474+0000 7f31d484f700 1 -- 172.21.15.88:0/2538656700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31d00ffbd0 msgr2=0x7f31d010a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:27.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.474+0000 7f31d484f700 1 --2- 172.21.15.88:0/2538656700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31d00ffbd0 0x7f31d010a030 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f31c000e970 tx=0x7f31c000ec80 comp rx=0 tx=0).stop 2024-02-19T19:07:27.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.474+0000 7f31d484f700 1 -- 172.21.15.88:0/2538656700 shutdown_connections 2024-02-19T19:07:27.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.474+0000 7f31d484f700 1 --2- 172.21.15.88:0/2538656700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31b806d8a0 0x7f31b806fd50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:27.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.474+0000 7f31d484f700 1 --2- 172.21.15.88:0/2538656700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31d00ffbd0 0x7f31d010a030 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:27.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.474+0000 7f31d484f700 1 --2- 172.21.15.88:0/2538656700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31d0104130 0x7f31d010a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:27.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.474+0000 7f31d484f700 1 -- 172.21.15.88:0/2538656700 >> 172.21.15.88:0/2538656700 conn(0x7f31d0075130 msgr2=0x7f31d00fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:27.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.474+0000 7f31d484f700 1 -- 172.21.15.88:0/2538656700 shutdown_connections 2024-02-19T19:07:27.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:27.474+0000 7f31d484f700 1 -- 172.21.15.88:0/2538656700 wait complete. 2024-02-19T19:07:27.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:27 smithi132 ceph-mon[70676]: from='client.34246 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:27.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:27 smithi132 ceph-mon[70676]: pgmap v79: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 280 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 939 KiB/s wr, 247 op/s; 1291/8658 objects degraded (14.911%) 2024-02-19T19:07:27.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:27 smithi132 ceph-mon[70676]: from='osd.0 [v2:172.21.15.88:6802/4275403566,v1:172.21.15.88:6803/4275403566]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-19T19:07:27.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:27 smithi132 ceph-mon[70676]: osdmap e52: 6 total, 5 up, 6 in 2024-02-19T19:07:27.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:27 smithi132 ceph-mon[70676]: from='osd.0 [v2:172.21.15.88:6802/4275403566,v1:172.21.15.88:6803/4275403566]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T19:07:27.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:27 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2216007036' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:07:27.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:27 smithi132 ceph-mon[70676]: Health check update: Degraded data redundancy: 1291/8658 objects degraded (14.911%), 33 pgs degraded (PG_DEGRADED) 2024-02-19T19:07:28.337 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:28 smithi088 ceph-mon[88643]: from='client.44199 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:28.337 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:28 smithi088 ceph-mon[88643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:07:28.337 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:28 smithi088 ceph-mon[88643]: osd.0 [v2:172.21.15.88:6802/4275403566,v1:172.21.15.88:6803/4275403566] boot 2024-02-19T19:07:28.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:28 smithi088 ceph-mon[88643]: osdmap e53: 6 total, 6 up, 6 in 2024-02-19T19:07:28.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T19:07:28.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:28.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:28.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:28 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2538656700' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:07:28.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:28 smithi088 ceph-mon[88643]: osdmap e54: 6 total, 6 up, 6 in 2024-02-19T19:07:28.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:28 smithi132 ceph-mon[70676]: from='client.44199 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:28.541 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:28 smithi132 ceph-mon[70676]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:07:28.542 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:28 smithi132 ceph-mon[70676]: osd.0 [v2:172.21.15.88:6802/4275403566,v1:172.21.15.88:6803/4275403566] boot 2024-02-19T19:07:28.542 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:28 smithi132 ceph-mon[70676]: osdmap e53: 6 total, 6 up, 6 in 2024-02-19T19:07:28.542 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-19T19:07:28.542 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:28.542 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:28.542 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:28 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2538656700' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:07:28.542 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:28 smithi132 ceph-mon[70676]: osdmap e54: 6 total, 6 up, 6 in 2024-02-19T19:07:29.396 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:29 smithi088 ceph-mon[88643]: pgmap v82: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 698 KiB/s wr, 176 op/s; 1291/8658 objects degraded (14.911%) 2024-02-19T19:07:29.396 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:29.396 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:07:29.396 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:29 smithi088 ceph-mon[88643]: osdmap e55: 6 total, 6 up, 6 in 2024-02-19T19:07:29.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:29 smithi132 ceph-mon[70676]: pgmap v82: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 698 KiB/s wr, 176 op/s; 1291/8658 objects degraded (14.911%) 2024-02-19T19:07:29.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:29.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:07:29.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:29 smithi132 ceph-mon[70676]: osdmap e55: 6 total, 6 up, 6 in 2024-02-19T19:07:31.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:30 smithi132 ceph-mon[70676]: osdmap e56: 6 total, 6 up, 6 in 2024-02-19T19:07:31.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:30 smithi132 ceph-mon[70676]: pgmap v86: 65 pgs: 3 remapped+peering, 9 active+undersized+degraded, 53 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.6 KiB/s rd, 897 KiB/s wr, 245 op/s; 519/6240 objects degraded (8.317%); 268 KiB/s, 155 objects/s recovering 2024-02-19T19:07:31.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:30 smithi088 ceph-mon[88643]: osdmap e56: 6 total, 6 up, 6 in 2024-02-19T19:07:31.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:30 smithi088 ceph-mon[88643]: pgmap v86: 65 pgs: 3 remapped+peering, 9 active+undersized+degraded, 53 active+clean; 279 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.6 KiB/s rd, 897 KiB/s wr, 245 op/s; 519/6240 objects degraded (8.317%); 268 KiB/s, 155 objects/s recovering 2024-02-19T19:07:32.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:31 smithi132 ceph-mon[70676]: osdmap e57: 6 total, 6 up, 6 in 2024-02-19T19:07:32.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:32.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:32.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:07:32.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:07:32.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:32.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:07:32.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:32.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:32.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:32.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-19T19:07:32.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:31 smithi088 ceph-mon[88643]: osdmap e57: 6 total, 6 up, 6 in 2024-02-19T19:07:32.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:32.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:32.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:07:32.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:07:32.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:32.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:07:32.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:32.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:32.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:32.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-19T19:07:33.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:32 smithi132 ceph-mon[70676]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-19T19:07:33.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:32 smithi132 ceph-mon[70676]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-02-19T19:07:33.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:32 smithi132 ceph-mon[70676]: pgmap v88: 65 pgs: 3 remapped+peering, 62 active+clean; 278 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 912 KiB/s wr, 298 op/s; 330 KiB/s, 273 objects/s recovering 2024-02-19T19:07:33.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:32 smithi088 ceph-mon[88643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-19T19:07:33.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:32 smithi088 ceph-mon[88643]: Upgrade: unsafe to stop osd(s) at this time (3 PGs are or would become offline) 2024-02-19T19:07:33.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:32 smithi088 ceph-mon[88643]: pgmap v88: 65 pgs: 3 remapped+peering, 62 active+clean; 278 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 28 KiB/s rd, 912 KiB/s wr, 298 op/s; 330 KiB/s, 273 objects/s recovering 2024-02-19T19:07:34.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:33 smithi132 ceph-mon[70676]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 519/6240 objects degraded (8.317%), 9 pgs degraded) 2024-02-19T19:07:34.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:33 smithi132 ceph-mon[70676]: Cluster is now healthy 2024-02-19T19:07:34.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:33 smithi088 ceph-mon[88643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 519/6240 objects degraded (8.317%), 9 pgs degraded) 2024-02-19T19:07:34.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:33 smithi088 ceph-mon[88643]: Cluster is now healthy 2024-02-19T19:07:35.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:34 smithi088 ceph-mon[88643]: pgmap v89: 65 pgs: 2 remapped+peering, 63 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 1.1 MiB/s wr, 263 op/s; 222 KiB/s, 202 objects/s recovering 2024-02-19T19:07:35.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:34 smithi132 ceph-mon[70676]: pgmap v89: 65 pgs: 2 remapped+peering, 63 active+clean; 280 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 1.1 MiB/s wr, 263 op/s; 222 KiB/s, 202 objects/s recovering 2024-02-19T19:07:37.499 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:37 smithi132 ceph-mon[70676]: pgmap v90: 65 pgs: 65 active+clean; 278 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 561 KiB/s wr, 135 op/s; 1.2 MiB/s, 85 objects/s recovering 2024-02-19T19:07:37.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:37 smithi088 ceph-mon[88643]: pgmap v90: 65 pgs: 65 active+clean; 278 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 561 KiB/s wr, 135 op/s; 1.2 MiB/s, 85 objects/s recovering 2024-02-19T19:07:39.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:39 smithi132 ceph-mon[70676]: pgmap v91: 65 pgs: 65 active+clean; 277 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 474 KiB/s wr, 131 op/s; 1008 KiB/s, 71 objects/s recovering 2024-02-19T19:07:39.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:39 smithi088 ceph-mon[88643]: pgmap v91: 65 pgs: 65 active+clean; 277 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 474 KiB/s wr, 131 op/s; 1008 KiB/s, 71 objects/s recovering 2024-02-19T19:07:41.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:41 smithi088 ceph-mon[88643]: pgmap v92: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 416 KiB/s wr, 146 op/s; 829 KiB/s, 59 objects/s recovering 2024-02-19T19:07:41.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:41 smithi132 ceph-mon[70676]: pgmap v92: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 416 KiB/s wr, 146 op/s; 829 KiB/s, 59 objects/s recovering 2024-02-19T19:07:43.148 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-02-19T19:07:43.149 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-19T19:07:43.480 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:43 smithi088 ceph-mon[88643]: pgmap v93: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 371 KiB/s wr, 131 op/s; 739 KiB/s, 52 objects/s recovering 2024-02-19T19:07:43.481 DEBUG:teuthology.parallel:result is None 2024-02-19T19:07:43.482 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-19T19:07:43.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:43 smithi132 ceph-mon[70676]: pgmap v93: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 371 KiB/s wr, 131 op/s; 739 KiB/s, 52 objects/s recovering 2024-02-19T19:07:43.560 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-19T19:07:43.561 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-02-19T19:07:43.599 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-02-19T19:07:43.599 DEBUG:teuthology.parallel:result is None 2024-02-19T19:07:44.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:44.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:44.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:07:44.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:44.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:44.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:07:45.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:45 smithi132 ceph-mon[70676]: pgmap v94: 65 pgs: 65 active+clean; 277 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 9.6 KiB/s rd, 544 KiB/s wr, 123 op/s; 668 KiB/s, 8 objects/s recovering 2024-02-19T19:07:45.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:45 smithi088 ceph-mon[88643]: pgmap v94: 65 pgs: 65 active+clean; 277 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 9.6 KiB/s rd, 544 KiB/s wr, 123 op/s; 668 KiB/s, 8 objects/s recovering 2024-02-19T19:07:47.498 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:47 smithi088 ceph-mon[88643]: pgmap v95: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 418 KiB/s wr, 129 op/s; 668 KiB/s, 0 objects/s recovering 2024-02-19T19:07:47.498 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-19T19:07:47.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:47 smithi132 ceph-mon[70676]: pgmap v95: 65 pgs: 65 active+clean; 276 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 418 KiB/s wr, 129 op/s; 668 KiB/s, 0 objects/s recovering 2024-02-19T19:07:47.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-19T19:07:48.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:48 smithi132 ceph-mon[70676]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-19T19:07:48.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:48 smithi132 ceph-mon[70676]: Upgrade: osd.1 is safe to restart 2024-02-19T19:07:48.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:48 smithi088 ceph-mon[88643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-02-19T19:07:48.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:48 smithi088 ceph-mon[88643]: Upgrade: osd.1 is safe to restart 2024-02-19T19:07:49.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:49 smithi132 ceph-mon[70676]: pgmap v96: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 374 KiB/s wr, 115 op/s 2024-02-19T19:07:49.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:49 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:49.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:49 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-19T19:07:49.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:49 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:07:49.539 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:49 smithi088 ceph-mon[88643]: pgmap v96: 65 pgs: 65 active+clean; 275 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 374 KiB/s wr, 115 op/s 2024-02-19T19:07:49.540 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:49 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:49.540 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:49 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-19T19:07:49.540 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:49 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:07:50.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:50 smithi132 ceph-mon[70676]: Upgrade: Updating osd.1 2024-02-19T19:07:50.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:50 smithi132 ceph-mon[70676]: Deploying daemon osd.1 on smithi088 2024-02-19T19:07:50.511 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:50 smithi088 ceph-mon[88643]: Upgrade: Updating osd.1 2024-02-19T19:07:50.511 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:50 smithi088 ceph-mon[88643]: Deploying daemon osd.1 on smithi088 2024-02-19T19:07:51.413 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:51 smithi088 ceph-mon[88643]: pgmap v97: 65 pgs: 65 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 392 KiB/s wr, 131 op/s 2024-02-19T19:07:51.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:51 smithi132 ceph-mon[70676]: pgmap v97: 65 pgs: 65 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 15 KiB/s rd, 392 KiB/s wr, 131 op/s 2024-02-19T19:07:52.101 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:51 smithi088 systemd[1]: Stopping Ceph osd.1 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:07:52.601 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:52 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[49795]: 2024-02-19T19:07:52.226+0000 7f9d47ed4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T19:07:52.601 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:52 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[49795]: 2024-02-19T19:07:52.226+0000 7f9d47ed4700 -1 osd.1 57 *** Got signal Terminated *** 2024-02-19T19:07:52.601 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:52 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[49795]: 2024-02-19T19:07:52.226+0000 7f9d47ed4700 -1 osd.1 57 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T19:07:53.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:53 smithi132 ceph-mon[70676]: pgmap v98: 65 pgs: 65 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 9.9 KiB/s rd, 370 KiB/s wr, 100 op/s 2024-02-19T19:07:53.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:53 smithi132 ceph-mon[70676]: osd.1 marked itself down and dead 2024-02-19T19:07:53.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:53 smithi088 ceph-mon[88643]: pgmap v98: 65 pgs: 65 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 9.9 KiB/s rd, 370 KiB/s wr, 100 op/s 2024-02-19T19:07:53.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:53 smithi088 ceph-mon[88643]: osd.1 marked itself down and dead 2024-02-19T19:07:53.950 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:53 smithi088 podman[106180]: 2024-02-19 19:07:53.636145959 +0000 UTC m=+1.516866441 container died 8b42e7ed1c67bb0ecc24690c2ff2363d1928a915c607708aaa9d2295b270d806 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1, io.buildah.version=1.29.1, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212) 2024-02-19T19:07:54.250 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:54 smithi088 podman[106180]: 2024-02-19 19:07:54.026473784 +0000 UTC m=+1.907194272 container cleanup 8b42e7ed1c67bb0ecc24690c2ff2363d1928a915c607708aaa9d2295b270d806 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_CLEAN=True) 2024-02-19T19:07:54.250 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:54 smithi088 bash[106180]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1 2024-02-19T19:07:54.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:54 smithi132 ceph-mon[70676]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:07:54.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:54 smithi132 ceph-mon[70676]: osdmap e58: 6 total, 5 up, 6 in 2024-02-19T19:07:54.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:54 smithi088 ceph-mon[88643]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:07:54.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:54 smithi088 ceph-mon[88643]: osdmap e58: 6 total, 5 up, 6 in 2024-02-19T19:07:54.601 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:54 smithi088 podman[106198]: 2024-02-19 19:07:54.289113581 +0000 UTC m=+0.653056413 container remove 8b42e7ed1c67bb0ecc24690c2ff2363d1928a915c607708aaa9d2295b270d806 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1) 2024-02-19T19:07:54.935 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:54 smithi088 podman[106290]: 2024-02-19 19:07:54.546810144 +0000 UTC m=+0.017719237 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:07:54.935 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:54 smithi088 podman[106290]: 2024-02-19 19:07:54.695929901 +0000 UTC m=+0.166838983 container create b1ac3381a671420da28924c308adf5d114cff181b6979163bdd78383bd15365d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T19:07:54.935 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:54 smithi088 podman[106290]: 2024-02-19 19:07:54.915003045 +0000 UTC m=+0.385912130 container init b1ac3381a671420da28924c308adf5d114cff181b6979163bdd78383bd15365d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:07:55.223 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:54 smithi088 podman[106290]: 2024-02-19 19:07:54.918339552 +0000 UTC m=+0.389248636 container start b1ac3381a671420da28924c308adf5d114cff181b6979163bdd78383bd15365d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-19T19:07:55.223 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:55 smithi088 podman[106290]: 2024-02-19 19:07:55.01514053 +0000 UTC m=+0.486049621 container attach b1ac3381a671420da28924c308adf5d114cff181b6979163bdd78383bd15365d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:07:55.506 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:55 smithi088 ceph-mon[88643]: pgmap v100: 65 pgs: 15 peering, 7 stale+active+clean, 43 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 202 KiB/s wr, 95 op/s 2024-02-19T19:07:55.506 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:55 smithi088 ceph-mon[88643]: osdmap e59: 6 total, 5 up, 6 in 2024-02-19T19:07:55.506 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:55 smithi088 podman[106290]: 2024-02-19 19:07:55.203521816 +0000 UTC m=+0.674430908 container died b1ac3381a671420da28924c308adf5d114cff181b6979163bdd78383bd15365d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True) 2024-02-19T19:07:55.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:55 smithi132 ceph-mon[70676]: pgmap v100: 65 pgs: 15 peering, 7 stale+active+clean, 43 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 12 KiB/s rd, 202 KiB/s wr, 95 op/s 2024-02-19T19:07:55.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:55 smithi132 ceph-mon[70676]: osdmap e59: 6 total, 5 up, 6 in 2024-02-19T19:07:56.101 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:55 smithi088 podman[106316]: 2024-02-19 19:07:55.760322734 +0000 UTC m=+0.551828857 container remove b1ac3381a671420da28924c308adf5d114cff181b6979163bdd78383bd15365d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-19T19:07:56.101 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:55 smithi088 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.1.service: Deactivated successfully. 2024-02-19T19:07:56.101 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:55 smithi088 systemd[1]: Stopped Ceph osd.1 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:07:56.101 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:55 smithi088 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.1.service: Consumed 1min 8.814s CPU time. 2024-02-19T19:07:56.101 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:55 smithi088 systemd[1]: Starting Ceph osd.1 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:07:56.478 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 podman[106421]: 2024-02-19 19:07:56.09987783 +0000 UTC m=+0.017995400 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:07:56.478 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 podman[106421]: 2024-02-19 19:07:56.228587181 +0000 UTC m=+0.146704795 container create 089199948f5539fad6758fa380a15c023dbb436a0bf4daceb0ab6641edc50d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:07:56.478 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 podman[106421]: 2024-02-19 19:07:56.457717687 +0000 UTC m=+0.375835267 container init 089199948f5539fad6758fa380a15c023dbb436a0bf4daceb0ab6641edc50d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T19:07:56.851 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 podman[106421]: 2024-02-19 19:07:56.461367391 +0000 UTC m=+0.379484968 container start 089199948f5539fad6758fa380a15c023dbb436a0bf4daceb0ab6641edc50d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214) 2024-02-19T19:07:56.851 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 podman[106421]: 2024-02-19 19:07:56.564687972 +0000 UTC m=+0.482805541 container attach 089199948f5539fad6758fa380a15c023dbb436a0bf4daceb0ab6641edc50d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, ceph=True) 2024-02-19T19:07:57.253 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate[106436]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-19T19:07:57.253 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate[106436]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-19T19:07:57.253 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 bash[106421]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-19T19:07:57.253 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 bash[106421]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-19T19:07:57.253 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 bash[106421]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-19T19:07:57.253 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 bash[106421]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-19T19:07:57.254 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 bash[106421]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-19T19:07:57.254 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 bash[106421]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-19T19:07:57.254 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 bash[106421]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-19T19:07:57.254 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate[106436]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-19T19:07:57.254 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate[106436]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-19T19:07:57.254 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate[106436]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-02-19T19:07:57.254 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate[106436]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-02-19T19:07:57.254 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate[106436]: --> ceph-volume raw activate successful for osd ID: 1 2024-02-19T19:07:57.254 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:56 smithi088 podman[106421]: 2024-02-19 19:07:56.931950199 +0000 UTC m=+0.850067761 container died 089199948f5539fad6758fa380a15c023dbb436a0bf4daceb0ab6641edc50d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-19T19:07:57.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:57 smithi088 ceph-mon[88643]: pgmap v102: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 34 KiB/s wr, 43 op/s 2024-02-19T19:07:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.585+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1432742254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefc0100460 msgr2=0x7fefc0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.585+0000 7fefc6bdc700 1 --2- 172.21.15.88:0/1432742254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefc0100460 0x7fefc0100830 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fefbc00b0a0 tx=0x7fefbc00b3b0 comp rx=0 tx=0).stop 2024-02-19T19:07:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.585+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1432742254 shutdown_connections 2024-02-19T19:07:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.585+0000 7fefc6bdc700 1 --2- 172.21.15.88:0/1432742254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefc0100d70 0x7fefc01094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.585+0000 7fefc6bdc700 1 --2- 172.21.15.88:0/1432742254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefc0100460 0x7fefc0100830 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.585+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1432742254 >> 172.21.15.88:0/1432742254 conn(0x7fefc00748b0 msgr2=0x7fefc0074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.585+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1432742254 shutdown_connections 2024-02-19T19:07:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.585+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1432742254 wait complete. 2024-02-19T19:07:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc6bdc700 1 Processor -- start 2024-02-19T19:07:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc6bdc700 1 -- start start 2024-02-19T19:07:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc6bdc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefc0100460 0x7fefc0101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc6bdc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefc0100d70 0x7fefc0101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc6bdc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc0101fc0 con 0x7fefc0100460 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc6bdc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc0102100 con 0x7fefc0100d70 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc4978700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefc0100460 0x7fefc0101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc4978700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefc0100460 0x7fefc0101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54060/0 (socket says 172.21.15.88:54060) 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc4978700 1 -- 172.21.15.88:0/1884703860 learned_addr learned my addr 172.21.15.88:0/1884703860 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefb7fff700 1 --2- 172.21.15.88:0/1884703860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefc0100d70 0x7fefc0101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc4978700 1 -- 172.21.15.88:0/1884703860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefc0100d70 msgr2=0x7fefc0101a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc4978700 1 --2- 172.21.15.88:0/1884703860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefc0100d70 0x7fefc0101a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc4978700 1 -- 172.21.15.88:0/1884703860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc009d00 con 0x7fefc0100460 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefc4978700 1 --2- 172.21.15.88:0/1884703860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefc0100460 0x7fefc0101540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fefbc000f80 tx=0x7fefbc00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefb5ffb700 1 -- 172.21.15.88:0/1884703860 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefbc00e040 con 0x7fefc0100460 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.586+0000 7fefb5ffb700 1 -- 172.21.15.88:0/1884703860 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefbc003e60 con 0x7fefc0100460 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.587+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1884703860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc0102380 con 0x7fefc0100460 2024-02-19T19:07:57.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.587+0000 7fefb5ffb700 1 -- 172.21.15.88:0/1884703860 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefbc01d910 con 0x7fefc0100460 2024-02-19T19:07:57.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.587+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1884703860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc01a22e0 con 0x7fefc0100460 2024-02-19T19:07:57.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.588+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1884703860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc0107e40 con 0x7fefc0100460 2024-02-19T19:07:57.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.588+0000 7fefb5ffb700 1 -- 172.21.15.88:0/1884703860 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fefbc004820 con 0x7fefc0100460 2024-02-19T19:07:57.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.589+0000 7fefb5ffb700 1 --2- 172.21.15.88:0/1884703860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fefa806d8a0 0x7fefa806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:57.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.589+0000 7fefb5ffb700 1 -- 172.21.15.88:0/1884703860 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fefbc08ed90 con 0x7fefc0100460 2024-02-19T19:07:57.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.589+0000 7fefb7fff700 1 --2- 172.21.15.88:0/1884703860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fefa806d8a0 0x7fefa806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:57.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.589+0000 7fefb7fff700 1 --2- 172.21.15.88:0/1884703860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fefa806d8a0 0x7fefa806fd50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fefb0005fd0 tx=0x7fefb000a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:57.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.594+0000 7fefb5ffb700 1 -- 172.21.15.88:0/1884703860 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fefbc0585f0 con 0x7fefc0100460 2024-02-19T19:07:57.702 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:57 smithi132 ceph-mon[70676]: pgmap v102: 65 pgs: 29 peering, 3 stale+active+clean, 33 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 255 B/s rd, 34 KiB/s wr, 43 op/s 2024-02-19T19:07:57.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.831+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1884703860 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fefc0061190 con 0x7fefa806d8a0 2024-02-19T19:07:57.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.832+0000 7fefb5ffb700 1 -- 172.21.15.88:0/1884703860 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fefc0061190 con 0x7fefa806d8a0 2024-02-19T19:07:57.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.834+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1884703860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fefa806d8a0 msgr2=0x7fefa806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:57.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.834+0000 7fefc6bdc700 1 --2- 172.21.15.88:0/1884703860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fefa806d8a0 0x7fefa806fd50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fefb0005fd0 tx=0x7fefb000a560 comp rx=0 tx=0).stop 2024-02-19T19:07:57.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.834+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1884703860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefc0100460 msgr2=0x7fefc0101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:57.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.834+0000 7fefc6bdc700 1 --2- 172.21.15.88:0/1884703860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefc0100460 0x7fefc0101540 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fefbc000f80 tx=0x7fefbc00cdd0 comp rx=0 tx=0).stop 2024-02-19T19:07:57.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.834+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1884703860 shutdown_connections 2024-02-19T19:07:57.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.834+0000 7fefc6bdc700 1 --2- 172.21.15.88:0/1884703860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fefa806d8a0 0x7fefa806fd50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:57.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.834+0000 7fefc6bdc700 1 --2- 172.21.15.88:0/1884703860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefc0100460 0x7fefc0101540 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:57.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.834+0000 7fefc6bdc700 1 --2- 172.21.15.88:0/1884703860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fefc0100d70 0x7fefc0101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:57.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.834+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1884703860 >> 172.21.15.88:0/1884703860 conn(0x7fefc00748b0 msgr2=0x7fefc00feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:57.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.835+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1884703860 shutdown_connections 2024-02-19T19:07:57.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.835+0000 7fefc6bdc700 1 -- 172.21.15.88:0/1884703860 wait complete. 2024-02-19T19:07:57.866 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:07:57.866 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:57 smithi088 podman[106576]: 2024-02-19 19:07:57.593793613 +0000 UTC m=+0.656529235 container remove 089199948f5539fad6758fa380a15c023dbb436a0bf4daceb0ab6641edc50d38 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-activate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-19T19:07:57.866 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:57 smithi088 podman[106660]: 2024-02-19 19:07:57.811293489 +0000 UTC m=+0.117777810 container create 5b940966d8833fc65603ca204d3e18f06791796a684c0645ab9b2435e94042c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux ) 2024-02-19T19:07:57.867 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:57 smithi088 podman[106660]: 2024-02-19 19:07:57.715775184 +0000 UTC m=+0.022259512 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:07:57.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.940+0000 7fbc4bcde700 1 -- 172.21.15.88:0/2791901988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc440ffd50 msgr2=0x7fbc44103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:57.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.940+0000 7fbc4bcde700 1 --2- 172.21.15.88:0/2791901988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc440ffd50 0x7fbc44103d20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fbc40009910 tx=0x7fbc40009c20 comp rx=0 tx=0).stop 2024-02-19T19:07:57.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.940+0000 7fbc4bcde700 1 -- 172.21.15.88:0/2791901988 shutdown_connections 2024-02-19T19:07:57.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.940+0000 7fbc4bcde700 1 --2- 172.21.15.88:0/2791901988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc440ffd50 0x7fbc44103d20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:57.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.940+0000 7fbc4bcde700 1 --2- 172.21.15.88:0/2791901988 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc440ff3b0 0x7fbc440ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:57.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.940+0000 7fbc4bcde700 1 -- 172.21.15.88:0/2791901988 >> 172.21.15.88:0/2791901988 conn(0x7fbc440fac90 msgr2=0x7fbc440fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.940+0000 7fbc4bcde700 1 -- 172.21.15.88:0/2791901988 shutdown_connections 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.940+0000 7fbc4bcde700 1 -- 172.21.15.88:0/2791901988 wait complete. 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc4bcde700 1 Processor -- start 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc4bcde700 1 -- start start 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc4bcde700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc440ff3b0 0x7fbc44196d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc4bcde700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc440ffd50 0x7fbc44197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc4bcde700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc441978b0 con 0x7fbc440ff3b0 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc4bcde700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc44197a20 con 0x7fbc440ffd50 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc49a7a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc440ff3b0 0x7fbc44196d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc49a7a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc440ff3b0 0x7fbc44196d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54080/0 (socket says 172.21.15.88:54080) 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc49a7a700 1 -- 172.21.15.88:0/289590776 learned_addr learned my addr 172.21.15.88:0/289590776 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc49279700 1 --2- 172.21.15.88:0/289590776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc440ffd50 0x7fbc44197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc49a7a700 1 -- 172.21.15.88:0/289590776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc440ffd50 msgr2=0x7fbc44197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:57.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc49a7a700 1 --2- 172.21.15.88:0/289590776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc440ffd50 0x7fbc44197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:57.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.941+0000 7fbc49a7a700 1 -- 172.21.15.88:0/289590776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc400095f0 con 0x7fbc440ff3b0 2024-02-19T19:07:57.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.942+0000 7fbc49a7a700 1 --2- 172.21.15.88:0/289590776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc440ff3b0 0x7fbc44196d50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400d8d0 tx=0x7fbc3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:57.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.942+0000 7fbc3affd700 1 -- 172.21.15.88:0/289590776 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc34009940 con 0x7fbc440ff3b0 2024-02-19T19:07:57.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.942+0000 7fbc3affd700 1 -- 172.21.15.88:0/289590776 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc34010460 con 0x7fbc440ff3b0 2024-02-19T19:07:57.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.942+0000 7fbc4bcde700 1 -- 172.21.15.88:0/289590776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4419fdc0 con 0x7fbc440ff3b0 2024-02-19T19:07:57.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.942+0000 7fbc3affd700 1 -- 172.21.15.88:0/289590776 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3400f5f0 con 0x7fbc440ff3b0 2024-02-19T19:07:57.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.942+0000 7fbc4bcde700 1 -- 172.21.15.88:0/289590776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc441a0280 con 0x7fbc440ff3b0 2024-02-19T19:07:57.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.943+0000 7fbc4bcde700 1 -- 172.21.15.88:0/289590776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc44107720 con 0x7fbc440ff3b0 2024-02-19T19:07:57.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.943+0000 7fbc3affd700 1 -- 172.21.15.88:0/289590776 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbc340105d0 con 0x7fbc440ff3b0 2024-02-19T19:07:57.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.944+0000 7fbc3affd700 1 --2- 172.21.15.88:0/289590776 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbc3006d8a0 0x7fbc3006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:57.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.944+0000 7fbc3affd700 1 -- 172.21.15.88:0/289590776 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fbc3408d380 con 0x7fbc440ff3b0 2024-02-19T19:07:57.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.944+0000 7fbc49279700 1 --2- 172.21.15.88:0/289590776 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbc3006d8a0 0x7fbc3006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:57.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.944+0000 7fbc49279700 1 --2- 172.21.15.88:0/289590776 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbc3006d8a0 0x7fbc3006fd50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbc40006010 tx=0x7fbc4001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:57.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:57.948+0000 7fbc3affd700 1 -- 172.21.15.88:0/289590776 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbc34056c60 con 0x7fbc440ff3b0 2024-02-19T19:07:58.193 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:58 smithi088 podman[106660]: 2024-02-19 19:07:58.081041095 +0000 UTC m=+0.387525420 container init 5b940966d8833fc65603ca204d3e18f06791796a684c0645ab9b2435e94042c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:07:58.193 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:58 smithi088 podman[106660]: 2024-02-19 19:07:58.084288727 +0000 UTC m=+0.390773049 container start 5b940966d8833fc65603ca204d3e18f06791796a684c0645ab9b2435e94042c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-19T19:07:58.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.201+0000 7fbc4bcde700 1 -- 172.21.15.88:0/289590776 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc44061190 con 0x7fbc3006d8a0 2024-02-19T19:07:58.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.203+0000 7fbc3affd700 1 -- 172.21.15.88:0/289590776 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbc44061190 con 0x7fbc3006d8a0 2024-02-19T19:07:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.207+0000 7fbc4bcde700 1 -- 172.21.15.88:0/289590776 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbc3006d8a0 msgr2=0x7fbc3006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.207+0000 7fbc4bcde700 1 --2- 172.21.15.88:0/289590776 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbc3006d8a0 0x7fbc3006fd50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbc40006010 tx=0x7fbc4001a040 comp rx=0 tx=0).stop 2024-02-19T19:07:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.207+0000 7fbc4bcde700 1 -- 172.21.15.88:0/289590776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc440ff3b0 msgr2=0x7fbc44196d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.207+0000 7fbc4bcde700 1 --2- 172.21.15.88:0/289590776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc440ff3b0 0x7fbc44196d50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbc3400d8d0 tx=0x7fbc3400dc90 comp rx=0 tx=0).stop 2024-02-19T19:07:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.207+0000 7fbc4bcde700 1 -- 172.21.15.88:0/289590776 shutdown_connections 2024-02-19T19:07:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.207+0000 7fbc4bcde700 1 --2- 172.21.15.88:0/289590776 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbc3006d8a0 0x7fbc3006fd50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.207+0000 7fbc4bcde700 1 --2- 172.21.15.88:0/289590776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbc440ff3b0 0x7fbc44196d50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.207+0000 7fbc4bcde700 1 --2- 172.21.15.88:0/289590776 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbc440ffd50 0x7fbc44197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.207+0000 7fbc4bcde700 1 -- 172.21.15.88:0/289590776 >> 172.21.15.88:0/289590776 conn(0x7fbc440fac90 msgr2=0x7fbc440fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.207+0000 7fbc4bcde700 1 -- 172.21.15.88:0/289590776 shutdown_connections 2024-02-19T19:07:58.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.207+0000 7fbc4bcde700 1 -- 172.21.15.88:0/289590776 wait complete. 2024-02-19T19:07:58.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.320+0000 7f0876a5a700 1 -- 172.21.15.88:0/703962759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0870100d70 msgr2=0x7f08701094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:58.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.320+0000 7f0876a5a700 1 --2- 172.21.15.88:0/703962759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0870100d70 0x7f08701094d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0864009940 tx=0x7f0864009c50 comp rx=0 tx=0).stop 2024-02-19T19:07:58.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.321+0000 7f0876a5a700 1 -- 172.21.15.88:0/703962759 shutdown_connections 2024-02-19T19:07:58.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.321+0000 7f0876a5a700 1 --2- 172.21.15.88:0/703962759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0870100d70 0x7f08701094d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.321+0000 7f0876a5a700 1 --2- 172.21.15.88:0/703962759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0870100460 0x7f0870100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.321+0000 7f0876a5a700 1 -- 172.21.15.88:0/703962759 >> 172.21.15.88:0/703962759 conn(0x7f08700748b0 msgr2=0x7f0870074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:58.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.321+0000 7f0876a5a700 1 -- 172.21.15.88:0/703962759 shutdown_connections 2024-02-19T19:07:58.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.321+0000 7f0876a5a700 1 -- 172.21.15.88:0/703962759 wait complete. 2024-02-19T19:07:58.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.322+0000 7f0876a5a700 1 Processor -- start 2024-02-19T19:07:58.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.322+0000 7f0876a5a700 1 -- start start 2024-02-19T19:07:58.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.322+0000 7f0876a5a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0870100460 0x7f0870101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:58.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.322+0000 7f0876a5a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0870100d70 0x7f0870101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:58.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.322+0000 7f0876a5a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0870101fc0 con 0x7f0870100460 2024-02-19T19:07:58.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.322+0000 7f0876a5a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0870102100 con 0x7f0870100d70 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.322+0000 7f086ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0870100460 0x7f0870101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.322+0000 7f086ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0870100460 0x7f0870101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54108/0 (socket says 172.21.15.88:54108) 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.322+0000 7f086ffff700 1 -- 172.21.15.88:0/2202902240 learned_addr learned my addr 172.21.15.88:0/2202902240 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.322+0000 7f086f7fe700 1 --2- 172.21.15.88:0/2202902240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0870100d70 0x7f0870101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.323+0000 7f086ffff700 1 -- 172.21.15.88:0/2202902240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0870100d70 msgr2=0x7f0870101a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.323+0000 7f086ffff700 1 --2- 172.21.15.88:0/2202902240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0870100d70 0x7f0870101a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.323+0000 7f086ffff700 1 -- 172.21.15.88:0/2202902240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08640095f0 con 0x7f0870100460 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.323+0000 7f086ffff700 1 --2- 172.21.15.88:0/2202902240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0870100460 0x7f0870101540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f086000d8d0 tx=0x7f086000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.323+0000 7f086d7fa700 1 -- 172.21.15.88:0/2202902240 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0860009940 con 0x7f0870100460 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.323+0000 7f086d7fa700 1 -- 172.21.15.88:0/2202902240 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0860010460 con 0x7f0870100460 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.323+0000 7f086d7fa700 1 -- 172.21.15.88:0/2202902240 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f086000f5f0 con 0x7f0870100460 2024-02-19T19:07:58.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.323+0000 7f0876a5a700 1 -- 172.21.15.88:0/2202902240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08701a20e0 con 0x7f0870100460 2024-02-19T19:07:58.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.323+0000 7f0876a5a700 1 -- 172.21.15.88:0/2202902240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08701a24f0 con 0x7f0870100460 2024-02-19T19:07:58.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.324+0000 7f0876a5a700 1 -- 172.21.15.88:0/2202902240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0870107e40 con 0x7f0870100460 2024-02-19T19:07:58.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.325+0000 7f086d7fa700 1 -- 172.21.15.88:0/2202902240 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f08600105d0 con 0x7f0870100460 2024-02-19T19:07:58.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.325+0000 7f086d7fa700 1 --2- 172.21.15.88:0/2202902240 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f085806d8a0 0x7f085806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:58.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.325+0000 7f086d7fa700 1 -- 172.21.15.88:0/2202902240 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f086008d490 con 0x7f0870100460 2024-02-19T19:07:58.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.325+0000 7f086f7fe700 1 --2- 172.21.15.88:0/2202902240 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f085806d8a0 0x7f085806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:58.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.326+0000 7f086f7fe700 1 --2- 172.21.15.88:0/2202902240 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f085806d8a0 0x7f085806fd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0864005b40 tx=0x7f0864005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:58.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.330+0000 7f086d7fa700 1 -- 172.21.15.88:0/2202902240 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0860056cf0 con 0x7f0870100460 2024-02-19T19:07:58.573 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:58 smithi088 bash[106660]: 5b940966d8833fc65603ca204d3e18f06791796a684c0645ab9b2435e94042c7 2024-02-19T19:07:58.573 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:58 smithi088 systemd[1]: Started Ceph osd.1 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:07:58.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.551+0000 7f0876a5a700 1 -- 172.21.15.88:0/2202902240 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0870061190 con 0x7f085806d8a0 2024-02-19T19:07:58.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.562+0000 7f086d7fa700 1 -- 172.21.15.88:0/2202902240 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0870061190 con 0x7f085806d8a0 2024-02-19T19:07:58.583 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:07:58.583 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (5m) 32s ago 15m 24.2M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:07:58.583 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (16m) 32s ago 16m 8170k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:07:58.583 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (14m) 64s ago 14m 8010k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:07:58.583 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (76s) 32s ago 16m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (67s) 64s ago 14m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (4m) 32s ago 14m 85.1M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (9m) 32s ago 9m 16.5M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (9m) 32s ago 9m 118M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (9m) 64s ago 9m 139M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (9m) 64s ago 9m 20.2M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (6m) 32s ago 17m 498M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (6m) 64s ago 13m 450M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (3m) 32s ago 17m 48.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (2m) 64s ago 13m 37.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (5m) 32s ago 15m 16.0M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (5m) 64s ago 13m 15.8M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (38s) 32s ago 13m 12.6M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:07:58.584 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 starting - - - 1216M 2024-02-19T19:07:58.585 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (12m) 32s ago 12m 187M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:07:58.585 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (11m) 64s ago 11m 236M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:07:58.585 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (11m) 64s ago 11m 215M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:07:58.585 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (10m) 64s ago 10m 201M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:07:58.585 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (5m) 32s ago 14m 69.2M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:07:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.565+0000 7f0876a5a700 1 -- 172.21.15.88:0/2202902240 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f085806d8a0 msgr2=0x7f085806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.565+0000 7f0876a5a700 1 --2- 172.21.15.88:0/2202902240 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f085806d8a0 0x7f085806fd50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0864005b40 tx=0x7f0864005a90 comp rx=0 tx=0).stop 2024-02-19T19:07:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.565+0000 7f0876a5a700 1 -- 172.21.15.88:0/2202902240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0870100460 msgr2=0x7f0870101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.565+0000 7f0876a5a700 1 --2- 172.21.15.88:0/2202902240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0870100460 0x7f0870101540 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f086000d8d0 tx=0x7f086000dc90 comp rx=0 tx=0).stop 2024-02-19T19:07:58.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.565+0000 7f0876a5a700 1 -- 172.21.15.88:0/2202902240 shutdown_connections 2024-02-19T19:07:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.565+0000 7f0876a5a700 1 --2- 172.21.15.88:0/2202902240 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f085806d8a0 0x7f085806fd50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.565+0000 7f0876a5a700 1 --2- 172.21.15.88:0/2202902240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0870100460 0x7f0870101540 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.565+0000 7f0876a5a700 1 --2- 172.21.15.88:0/2202902240 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0870100d70 0x7f0870101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.565+0000 7f0876a5a700 1 -- 172.21.15.88:0/2202902240 >> 172.21.15.88:0/2202902240 conn(0x7f08700748b0 msgr2=0x7f08700fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.565+0000 7f0876a5a700 1 -- 172.21.15.88:0/2202902240 shutdown_connections 2024-02-19T19:07:58.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.565+0000 7f0876a5a700 1 -- 172.21.15.88:0/2202902240 wait complete. 2024-02-19T19:07:58.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.674+0000 7f582c0fb700 1 -- 172.21.15.88:0/1332989107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58241038b0 msgr2=0x7f5824103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:58.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.674+0000 7f582c0fb700 1 --2- 172.21.15.88:0/1332989107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58241038b0 0x7f5824103c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5814009910 tx=0x7f5814009c20 comp rx=0 tx=0).stop 2024-02-19T19:07:58.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.674+0000 7f582c0fb700 1 -- 172.21.15.88:0/1332989107 shutdown_connections 2024-02-19T19:07:58.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.674+0000 7f582c0fb700 1 --2- 172.21.15.88:0/1332989107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240ff350 0x7f58240ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.674+0000 7f582c0fb700 1 --2- 172.21.15.88:0/1332989107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58241038b0 0x7f5824103c80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.674+0000 7f582c0fb700 1 -- 172.21.15.88:0/1332989107 >> 172.21.15.88:0/1332989107 conn(0x7f58240748b0 msgr2=0x7f5824074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:58.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.674+0000 7f582c0fb700 1 -- 172.21.15.88:0/1332989107 shutdown_connections 2024-02-19T19:07:58.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.674+0000 7f582c0fb700 1 -- 172.21.15.88:0/1332989107 wait complete. 2024-02-19T19:07:58.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.675+0000 7f582c0fb700 1 Processor -- start 2024-02-19T19:07:58.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.675+0000 7f582c0fb700 1 -- start start 2024-02-19T19:07:58.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.675+0000 7f582c0fb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240ff350 0x7f5824194d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.675+0000 7f582c0fb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58241038b0 0x7f5824195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.675+0000 7f582c0fb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5824195940 con 0x7f58241038b0 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.675+0000 7f582c0fb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58241996d0 con 0x7f58240ff350 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.675+0000 7f5829696700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58241038b0 0x7f5824195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.675+0000 7f5829696700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58241038b0 0x7f5824195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54120/0 (socket says 172.21.15.88:54120) 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.675+0000 7f5829696700 1 -- 172.21.15.88:0/833153580 learned_addr learned my addr 172.21.15.88:0/833153580 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.675+0000 7f5829e97700 1 --2- 172.21.15.88:0/833153580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240ff350 0x7f5824194d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.676+0000 7f5829696700 1 -- 172.21.15.88:0/833153580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240ff350 msgr2=0x7f5824194d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.676+0000 7f5829696700 1 --2- 172.21.15.88:0/833153580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240ff350 0x7f5824194d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.676+0000 7f5829696700 1 -- 172.21.15.88:0/833153580 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140095f0 con 0x7f58241038b0 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.676+0000 7f5829696700 1 --2- 172.21.15.88:0/833153580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58241038b0 0x7f5824195260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f582000e920 tx=0x7f582000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.676+0000 7f581affd700 1 -- 172.21.15.88:0/833153580 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f582000c9d0 con 0x7f58241038b0 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.676+0000 7f581affd700 1 -- 172.21.15.88:0/833153580 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5820010430 con 0x7f58241038b0 2024-02-19T19:07:58.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.676+0000 7f582c0fb700 1 -- 172.21.15.88:0/833153580 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58241999b0 con 0x7f58241038b0 2024-02-19T19:07:58.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.676+0000 7f582c0fb700 1 -- 172.21.15.88:0/833153580 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5824199f00 con 0x7f58241038b0 2024-02-19T19:07:58.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.676+0000 7f581affd700 1 -- 172.21.15.88:0/833153580 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5820018630 con 0x7f58241038b0 2024-02-19T19:07:58.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.677+0000 7f582c0fb700 1 -- 172.21.15.88:0/833153580 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5824106e20 con 0x7f58241038b0 2024-02-19T19:07:58.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.677+0000 7f581affd700 1 -- 172.21.15.88:0/833153580 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f58200188a0 con 0x7f58241038b0 2024-02-19T19:07:58.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.678+0000 7f581affd700 1 --2- 172.21.15.88:0/833153580 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 0x7f581006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:58.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.678+0000 7f581affd700 1 -- 172.21.15.88:0/833153580 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5820014070 con 0x7f58241038b0 2024-02-19T19:07:58.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.678+0000 7f5829e97700 1 --2- 172.21.15.88:0/833153580 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 0x7f581006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:58.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.678+0000 7f5829e97700 1 --2- 172.21.15.88:0/833153580 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 0x7f581006fd50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5814000c00 tx=0x7f5814009f70 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:58.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.683+0000 7f581affd700 1 -- 172.21.15.88:0/833153580 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5820052d20 con 0x7f58241038b0 2024-02-19T19:07:58.851 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:07:58 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[106700]: 2024-02-19T19:07:58.632+0000 7f24139dd7c0 -1 Falling back to public interface 2024-02-19T19:07:59.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.995+0000 7f582c0fb700 1 -- 172.21.15.88:0/833153580 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f582419a170 con 0x7f58241038b0 2024-02-19T19:07:59.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.996+0000 7f581affd700 1 -- 172.21.15.88:0/833153580 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f5820056fa0 con 0x7f58241038b0 2024-02-19T19:07:59.017 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:07:59.017 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:07:59.017 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:07:59.017 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:07:59.017 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:07:59.017 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:07:59.017 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:07:59.017 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:07:59.017 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-02-19T19:07:59.018 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-19T19:07:59.018 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:07:59.018 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:07:59.018 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:07:59.018 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:07:59.018 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:07:59.018 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 8, 2024-02-19T19:07:59.018 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 5 2024-02-19T19:07:59.018 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:07:59.018 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:07:59.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.998+0000 7f582c0fb700 1 -- 172.21.15.88:0/833153580 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 msgr2=0x7f581006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.998+0000 7f582c0fb700 1 --2- 172.21.15.88:0/833153580 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 0x7f581006fd50 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5814000c00 tx=0x7f5814009f70 comp rx=0 tx=0).stop 2024-02-19T19:07:59.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.998+0000 7f582c0fb700 1 -- 172.21.15.88:0/833153580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58241038b0 msgr2=0x7f5824195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.998+0000 7f582c0fb700 1 --2- 172.21.15.88:0/833153580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58241038b0 0x7f5824195260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f582000e920 tx=0x7f582000ec30 comp rx=0 tx=0).stop 2024-02-19T19:07:59.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.998+0000 7f582c0fb700 1 -- 172.21.15.88:0/833153580 shutdown_connections 2024-02-19T19:07:59.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.998+0000 7f582c0fb700 1 --2- 172.21.15.88:0/833153580 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 0x7f581006fd50 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.998+0000 7f582c0fb700 1 --2- 172.21.15.88:0/833153580 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58240ff350 0x7f5824194d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.998+0000 7f582c0fb700 1 --2- 172.21.15.88:0/833153580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58241038b0 0x7f5824195260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.998+0000 7f582c0fb700 1 -- 172.21.15.88:0/833153580 >> 172.21.15.88:0/833153580 conn(0x7f58240748b0 msgr2=0x7f58240fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:59.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.998+0000 7f582c0fb700 1 -- 172.21.15.88:0/833153580 shutdown_connections 2024-02-19T19:07:59.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:58.998+0000 7f582c0fb700 1 -- 172.21.15.88:0/833153580 wait complete. 2024-02-19T19:07:59.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.102+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/87193981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd40ff3c0 msgr2=0x7f8bd40ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.102+0000 7f8bdb5b5700 1 --2- 172.21.15.88:0/87193981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd40ff3c0 0x7f8bd40ff790 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009910 tx=0x7f8bd0009c20 comp rx=0 tx=0).stop 2024-02-19T19:07:59.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.102+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/87193981 shutdown_connections 2024-02-19T19:07:59.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.102+0000 7f8bdb5b5700 1 --2- 172.21.15.88:0/87193981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd40ffd60 0x7f8bd4103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.102+0000 7f8bdb5b5700 1 --2- 172.21.15.88:0/87193981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd40ff3c0 0x7f8bd40ff790 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.102+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/87193981 >> 172.21.15.88:0/87193981 conn(0x7f8bd40fac80 msgr2=0x7f8bd40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:59.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.103+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/87193981 shutdown_connections 2024-02-19T19:07:59.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.103+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/87193981 wait complete. 2024-02-19T19:07:59.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.103+0000 7f8bdb5b5700 1 Processor -- start 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.103+0000 7f8bdb5b5700 1 -- start start 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.103+0000 7f8bdb5b5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd40ff3c0 0x7f8bd4195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.103+0000 7f8bdb5b5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd40ffd60 0x7f8bd41964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.103+0000 7f8bdb5b5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd4196bb0 con 0x7f8bd40ffd60 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.103+0000 7f8bdb5b5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd419a940 con 0x7f8bd40ff3c0 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bd8b50700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd40ffd60 0x7f8bd41964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bd8b50700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd40ffd60 0x7f8bd41964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54138/0 (socket says 172.21.15.88:54138) 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bd8b50700 1 -- 172.21.15.88:0/706406149 learned_addr learned my addr 172.21.15.88:0/706406149 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bd9351700 1 --2- 172.21.15.88:0/706406149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd40ff3c0 0x7f8bd4195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bd8b50700 1 -- 172.21.15.88:0/706406149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd40ff3c0 msgr2=0x7f8bd4195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bd8b50700 1 --2- 172.21.15.88:0/706406149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd40ff3c0 0x7f8bd4195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bd8b50700 1 -- 172.21.15.88:0/706406149 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd00095f0 con 0x7f8bd40ffd60 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bd8b50700 1 --2- 172.21.15.88:0/706406149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd40ffd60 0x7f8bd41964d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400c8f0 tx=0x7f8bc400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:59.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bca7fc700 1 -- 172.21.15.88:0/706406149 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc40078a0 con 0x7f8bd40ffd60 2024-02-19T19:07:59.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/706406149 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd419ac20 con 0x7f8bd40ffd60 2024-02-19T19:07:59.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/706406149 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd419b170 con 0x7f8bd40ffd60 2024-02-19T19:07:59.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bca7fc700 1 -- 172.21.15.88:0/706406149 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bc400f450 con 0x7f8bd40ffd60 2024-02-19T19:07:59.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.104+0000 7f8bca7fc700 1 -- 172.21.15.88:0/706406149 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bc4018610 con 0x7f8bd40ffd60 2024-02-19T19:07:59.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.105+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/706406149 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb8005320 con 0x7f8bd40ffd60 2024-02-19T19:07:59.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.106+0000 7f8bca7fc700 1 -- 172.21.15.88:0/706406149 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8bc4007a00 con 0x7f8bd40ffd60 2024-02-19T19:07:59.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.106+0000 7f8bca7fc700 1 --2- 172.21.15.88:0/706406149 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8bc006d8a0 0x7f8bc006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:59.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.107+0000 7f8bca7fc700 1 -- 172.21.15.88:0/706406149 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8bc408d310 con 0x7f8bd40ffd60 2024-02-19T19:07:59.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.107+0000 7f8bd9351700 1 --2- 172.21.15.88:0/706406149 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8bc006d8a0 0x7f8bc006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:59.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.107+0000 7f8bd9351700 1 --2- 172.21.15.88:0/706406149 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8bc006d8a0 0x7f8bc006fd50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0006010 tx=0x7f8bd001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:59.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.111+0000 7f8bca7fc700 1 -- 172.21.15.88:0/706406149 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8bc4056b70 con 0x7f8bd40ffd60 2024-02-19T19:07:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:59 smithi088 ceph-mon[88643]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:59 smithi088 ceph-mon[88643]: pgmap v103: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.9 KiB/s wr, 0 op/s; 6/333 objects degraded (1.802%) 2024-02-19T19:07:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:59 smithi088 ceph-mon[88643]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:07:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:59 smithi088 ceph-mon[88643]: Health check failed: Degraded data redundancy: 6/333 objects degraded (1.802%), 2 pgs degraded (PG_DEGRADED) 2024-02-19T19:07:59.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:07:59.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:07:59 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/833153580' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:59.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.361+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/706406149 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8bb8006200 con 0x7f8bd40ffd60 2024-02-19T19:07:59.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.361+0000 7f8bca7fc700 1 -- 172.21.15.88:0/706406149 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1892 (secure 0 0 0) 0x7f8bc405a230 con 0x7f8bd40ffd60 2024-02-19T19:07:59.382 INFO:teuthology.orchestra.run.smithi088.stdout:e20 2024-02-19T19:07:59.382 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:07:59.382 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:07:59.382 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:07:59.382 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:07:59.382 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:07:59.382 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:07:59.382 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 20 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:05:14.913778+0000 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:07:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:07:59.384 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:07:59.385 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:07:59.385 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:07:59.385 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:07:59.385 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:07:59.385 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:07:59.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.363+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/706406149 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8bc006d8a0 msgr2=0x7f8bc006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.363+0000 7f8bdb5b5700 1 --2- 172.21.15.88:0/706406149 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8bc006d8a0 0x7f8bc006fd50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0006010 tx=0x7f8bd001a040 comp rx=0 tx=0).stop 2024-02-19T19:07:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.363+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/706406149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd40ffd60 msgr2=0x7f8bd41964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.363+0000 7f8bdb5b5700 1 --2- 172.21.15.88:0/706406149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd40ffd60 0x7f8bd41964d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400c8f0 tx=0x7f8bc400cc00 comp rx=0 tx=0).stop 2024-02-19T19:07:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.363+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/706406149 shutdown_connections 2024-02-19T19:07:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.363+0000 7f8bdb5b5700 1 --2- 172.21.15.88:0/706406149 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8bc006d8a0 0x7f8bc006fd50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.363+0000 7f8bdb5b5700 1 --2- 172.21.15.88:0/706406149 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8bd40ff3c0 0x7f8bd4195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.363+0000 7f8bdb5b5700 1 --2- 172.21.15.88:0/706406149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd40ffd60 0x7f8bd41964d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.363+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/706406149 >> 172.21.15.88:0/706406149 conn(0x7f8bd40fac80 msgr2=0x7f8bd4068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.363+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/706406149 shutdown_connections 2024-02-19T19:07:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.363+0000 7f8bdb5b5700 1 -- 172.21.15.88:0/706406149 wait complete. 2024-02-19T19:07:59.386 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-02-19T19:07:59.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.471+0000 7f215bd16700 1 -- 172.21.15.88:0/2191637996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2154069140 msgr2=0x7f2154104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.471+0000 7f215bd16700 1 --2- 172.21.15.88:0/2191637996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2154069140 0x7f2154104eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2148009910 tx=0x7f2148009c20 comp rx=0 tx=0).stop 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.471+0000 7f215bd16700 1 -- 172.21.15.88:0/2191637996 shutdown_connections 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.471+0000 7f215bd16700 1 --2- 172.21.15.88:0/2191637996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2154069140 0x7f2154104eb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.471+0000 7f215bd16700 1 --2- 172.21.15.88:0/2191637996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21540687a0 0x7f2154068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.471+0000 7f215bd16700 1 -- 172.21.15.88:0/2191637996 >> 172.21.15.88:0/2191637996 conn(0x7f2154075130 msgr2=0x7f2154075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.471+0000 7f215bd16700 1 -- 172.21.15.88:0/2191637996 shutdown_connections 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.471+0000 7f215bd16700 1 -- 172.21.15.88:0/2191637996 wait complete. 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f215bd16700 1 Processor -- start 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f215bd16700 1 -- start start 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f215bd16700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21540687a0 0x7f2154195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f215bd16700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2154069140 0x7f21541963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:59.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f215bd16700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2154196a80 con 0x7f21540687a0 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f215bd16700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215419a810 con 0x7f2154069140 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f2159ab2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21540687a0 0x7f2154195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f2159ab2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21540687a0 0x7f2154195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54160/0 (socket says 172.21.15.88:54160) 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f2159ab2700 1 -- 172.21.15.88:0/2314389202 learned_addr learned my addr 172.21.15.88:0/2314389202 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f21592b1700 1 --2- 172.21.15.88:0/2314389202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2154069140 0x7f21541963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f2159ab2700 1 -- 172.21.15.88:0/2314389202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2154069140 msgr2=0x7f21541963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f2159ab2700 1 --2- 172.21.15.88:0/2314389202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2154069140 0x7f21541963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.472+0000 7f2159ab2700 1 -- 172.21.15.88:0/2314389202 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21480095f0 con 0x7f21540687a0 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.473+0000 7f2159ab2700 1 --2- 172.21.15.88:0/2314389202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21540687a0 0x7f2154195e60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f214400efc0 tx=0x7f214400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.473+0000 7f2152ffd700 1 -- 172.21.15.88:0/2314389202 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2144018070 con 0x7f21540687a0 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.473+0000 7f2152ffd700 1 -- 172.21.15.88:0/2314389202 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f214400f040 con 0x7f21540687a0 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.473+0000 7f215bd16700 1 -- 172.21.15.88:0/2314389202 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215419aa90 con 0x7f21540687a0 2024-02-19T19:07:59.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.473+0000 7f2152ffd700 1 -- 172.21.15.88:0/2314389202 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21440044b0 con 0x7f21540687a0 2024-02-19T19:07:59.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.473+0000 7f215bd16700 1 -- 172.21.15.88:0/2314389202 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215419afb0 con 0x7f21540687a0 2024-02-19T19:07:59.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.474+0000 7f215bd16700 1 -- 172.21.15.88:0/2314389202 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2154108860 con 0x7f21540687a0 2024-02-19T19:07:59.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.474+0000 7f2152ffd700 1 -- 172.21.15.88:0/2314389202 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2144004730 con 0x7f21540687a0 2024-02-19T19:07:59.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.475+0000 7f2152ffd700 1 --2- 172.21.15.88:0/2314389202 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f213c06d8a0 0x7f213c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:59.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.475+0000 7f2152ffd700 1 -- 172.21.15.88:0/2314389202 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f214408d6a0 con 0x7f21540687a0 2024-02-19T19:07:59.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.475+0000 7f21592b1700 1 --2- 172.21.15.88:0/2314389202 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f213c06d8a0 0x7f213c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:59.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.475+0000 7f21592b1700 1 --2- 172.21.15.88:0/2314389202 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f213c06d8a0 0x7f213c06fd50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2148005b40 tx=0x7f2148005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:59.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.480+0000 7f2152ffd700 1 -- 172.21.15.88:0/2314389202 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2144056f00 con 0x7f21540687a0 2024-02-19T19:07:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:59 smithi132 ceph-mon[70676]: from='client.34268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:59 smithi132 ceph-mon[70676]: pgmap v103: 65 pgs: 2 active+undersized+degraded, 29 peering, 3 active+undersized, 31 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.9 KiB/s wr, 0 op/s; 6/333 objects degraded (1.802%) 2024-02-19T19:07:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:59 smithi132 ceph-mon[70676]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:07:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:07:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:07:59.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:59 smithi132 ceph-mon[70676]: Health check failed: Degraded data redundancy: 6/333 objects degraded (1.802%), 2 pgs degraded (PG_DEGRADED) 2024-02-19T19:07:59.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:07:59.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:07:59 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/833153580' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:07:59.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.720+0000 7f215bd16700 1 -- 172.21.15.88:0/2314389202 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f215419b1d0 con 0x7f213c06d8a0 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.721+0000 7f2152ffd700 1 -- 172.21.15.88:0/2314389202 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f215419b1d0 con 0x7f213c06d8a0 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "7/23 daemons upgraded", 2024-02-19T19:07:59.742 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading osd daemons", 2024-02-19T19:07:59.743 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:07:59.743 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:07:59.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.723+0000 7f215bd16700 1 -- 172.21.15.88:0/2314389202 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f213c06d8a0 msgr2=0x7f213c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.723+0000 7f215bd16700 1 --2- 172.21.15.88:0/2314389202 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f213c06d8a0 0x7f213c06fd50 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2148005b40 tx=0x7f2148005a90 comp rx=0 tx=0).stop 2024-02-19T19:07:59.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.723+0000 7f215bd16700 1 -- 172.21.15.88:0/2314389202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21540687a0 msgr2=0x7f2154195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.723+0000 7f215bd16700 1 --2- 172.21.15.88:0/2314389202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21540687a0 0x7f2154195e60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f214400efc0 tx=0x7f214400c3a0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.723+0000 7f215bd16700 1 -- 172.21.15.88:0/2314389202 shutdown_connections 2024-02-19T19:07:59.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.723+0000 7f215bd16700 1 --2- 172.21.15.88:0/2314389202 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f213c06d8a0 0x7f213c06fd50 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.723+0000 7f215bd16700 1 --2- 172.21.15.88:0/2314389202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21540687a0 0x7f2154195e60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.723+0000 7f215bd16700 1 --2- 172.21.15.88:0/2314389202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2154069140 0x7f21541963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.723+0000 7f215bd16700 1 -- 172.21.15.88:0/2314389202 >> 172.21.15.88:0/2314389202 conn(0x7f2154075130 msgr2=0x7f21540ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:59.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.723+0000 7f215bd16700 1 -- 172.21.15.88:0/2314389202 shutdown_connections 2024-02-19T19:07:59.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.723+0000 7f215bd16700 1 -- 172.21.15.88:0/2314389202 wait complete. 2024-02-19T19:07:59.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.829+0000 7f2ddcccf700 1 -- 172.21.15.88:0/3051117592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dd8100e80 msgr2=0x7f2dd8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.829+0000 7f2ddcccf700 1 --2- 172.21.15.88:0/3051117592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dd8100e80 0x7f2dd8104e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2dc8009910 tx=0x7f2dc8009c20 comp rx=0 tx=0).stop 2024-02-19T19:07:59.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.829+0000 7f2ddcccf700 1 -- 172.21.15.88:0/3051117592 shutdown_connections 2024-02-19T19:07:59.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.829+0000 7f2ddcccf700 1 --2- 172.21.15.88:0/3051117592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dd8100e80 0x7f2dd8104e50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.829+0000 7f2ddcccf700 1 --2- 172.21.15.88:0/3051117592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dd81004e0 0x7f2dd81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.829+0000 7f2ddcccf700 1 -- 172.21.15.88:0/3051117592 >> 172.21.15.88:0/3051117592 conn(0x7f2dd80fbda0 msgr2=0x7f2dd80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:07:59.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.829+0000 7f2ddcccf700 1 -- 172.21.15.88:0/3051117592 shutdown_connections 2024-02-19T19:07:59.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.829+0000 7f2ddcccf700 1 -- 172.21.15.88:0/3051117592 wait complete. 2024-02-19T19:07:59.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2ddcccf700 1 Processor -- start 2024-02-19T19:07:59.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2ddcccf700 1 -- start start 2024-02-19T19:07:59.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2ddcccf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dd81004e0 0x7f2dd8195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:59.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2ddcccf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dd8100e80 0x7f2dd8196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:59.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2ddcccf700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd8196b50 con 0x7f2dd8100e80 2024-02-19T19:07:59.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2ddcccf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd819a8e0 con 0x7f2dd81004e0 2024-02-19T19:07:59.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2dd5d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dd8100e80 0x7f2dd8196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:59.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2dd5d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dd8100e80 0x7f2dd8196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54174/0 (socket says 172.21.15.88:54174) 2024-02-19T19:07:59.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2dd5d9b700 1 -- 172.21.15.88:0/4061488516 learned_addr learned my addr 172.21.15.88:0/4061488516 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:07:59.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2dd659c700 1 --2- 172.21.15.88:0/4061488516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dd81004e0 0x7f2dd8195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:59.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2dd5d9b700 1 -- 172.21.15.88:0/4061488516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dd81004e0 msgr2=0x7f2dd8195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:07:59.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2dd5d9b700 1 --2- 172.21.15.88:0/4061488516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dd81004e0 0x7f2dd8195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:07:59.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2dd5d9b700 1 -- 172.21.15.88:0/4061488516 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc80095f0 con 0x7f2dd8100e80 2024-02-19T19:07:59.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.830+0000 7f2dd5d9b700 1 --2- 172.21.15.88:0/4061488516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dd8100e80 0x7f2dd8196470 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800bd70 tx=0x7f2dc800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:59.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.831+0000 7f2dc77fe700 1 -- 172.21.15.88:0/4061488516 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dc801d070 con 0x7f2dd8100e80 2024-02-19T19:07:59.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.831+0000 7f2ddcccf700 1 -- 172.21.15.88:0/4061488516 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd819ab60 con 0x7f2dd8100e80 2024-02-19T19:07:59.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.831+0000 7f2dc77fe700 1 -- 172.21.15.88:0/4061488516 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2dc8005dc0 con 0x7f2dd8100e80 2024-02-19T19:07:59.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.831+0000 7f2dc77fe700 1 -- 172.21.15.88:0/4061488516 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2dc8017400 con 0x7f2dd8100e80 2024-02-19T19:07:59.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.831+0000 7f2ddcccf700 1 -- 172.21.15.88:0/4061488516 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd819b050 con 0x7f2dd8100e80 2024-02-19T19:07:59.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.832+0000 7f2ddcccf700 1 -- 172.21.15.88:0/4061488516 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd8108850 con 0x7f2dd8100e80 2024-02-19T19:07:59.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.832+0000 7f2dc77fe700 1 -- 172.21.15.88:0/4061488516 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2dc800f7f0 con 0x7f2dd8100e80 2024-02-19T19:07:59.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.832+0000 7f2dc77fe700 1 --2- 172.21.15.88:0/4061488516 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2dc006d8a0 0x7f2dc006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:07:59.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.833+0000 7f2dc77fe700 1 -- 172.21.15.88:0/4061488516 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2dc808e9a0 con 0x7f2dd8100e80 2024-02-19T19:07:59.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.833+0000 7f2dd659c700 1 --2- 172.21.15.88:0/4061488516 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2dc006d8a0 0x7f2dc006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:07:59.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.833+0000 7f2dd659c700 1 --2- 172.21.15.88:0/4061488516 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2dc006d8a0 0x7f2dc006fd50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc005950 tx=0x7f2dcc0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:07:59.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:07:59.837+0000 7f2dc77fe700 1 -- 172.21.15.88:0/4061488516 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2dc8058200 con 0x7f2dd8100e80 2024-02-19T19:08:00.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.123+0000 7f2ddcccf700 1 -- 172.21.15.88:0/4061488516 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2dd8197290 con 0x7f2dd8100e80 2024-02-19T19:08:00.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.123+0000 7f2dc77fe700 1 -- 172.21.15.88:0/4061488516 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+379 (secure 0 0 0) 0x7f2dc805b8c0 con 0x7f2dd8100e80 2024-02-19T19:08:00.144 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 6/333 objects degraded (1.802%), 2 pgs degraded 2024-02-19T19:08:00.144 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-19T19:08:00.144 INFO:teuthology.orchestra.run.smithi088.stdout: osd.1 (root=default,host=smithi088) is down 2024-02-19T19:08:00.144 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 6/333 objects degraded (1.802%), 2 pgs degraded 2024-02-19T19:08:00.144 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-02-19T19:08:00.145 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-02-19T19:08:00.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.125+0000 7f2ddcccf700 1 -- 172.21.15.88:0/4061488516 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2dc006d8a0 msgr2=0x7f2dc006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:00.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.125+0000 7f2ddcccf700 1 --2- 172.21.15.88:0/4061488516 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2dc006d8a0 0x7f2dc006fd50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc005950 tx=0x7f2dcc0058e0 comp rx=0 tx=0).stop 2024-02-19T19:08:00.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.125+0000 7f2ddcccf700 1 -- 172.21.15.88:0/4061488516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dd8100e80 msgr2=0x7f2dd8196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.125+0000 7f2ddcccf700 1 --2- 172.21.15.88:0/4061488516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dd8100e80 0x7f2dd8196470 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800bd70 tx=0x7f2dc800be50 comp rx=0 tx=0).stop 2024-02-19T19:08:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.126+0000 7f2ddcccf700 1 -- 172.21.15.88:0/4061488516 shutdown_connections 2024-02-19T19:08:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.126+0000 7f2ddcccf700 1 --2- 172.21.15.88:0/4061488516 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2dc006d8a0 0x7f2dc006fd50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.126+0000 7f2ddcccf700 1 --2- 172.21.15.88:0/4061488516 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2dd81004e0 0x7f2dd8195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.126+0000 7f2ddcccf700 1 --2- 172.21.15.88:0/4061488516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2dd8100e80 0x7f2dd8196470 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.126+0000 7f2ddcccf700 1 -- 172.21.15.88:0/4061488516 >> 172.21.15.88:0/4061488516 conn(0x7f2dd80fbda0 msgr2=0x7f2dd80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.126+0000 7f2ddcccf700 1 -- 172.21.15.88:0/4061488516 shutdown_connections 2024-02-19T19:08:00.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:00.126+0000 7f2ddcccf700 1 -- 172.21.15.88:0/4061488516 wait complete. 2024-02-19T19:08:00.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:00 smithi088 ceph-mon[88643]: from='client.34278 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:00.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:00 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/706406149' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:08:00.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:00 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4061488516' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:08:00.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:00 smithi132 ceph-mon[70676]: from='client.34278 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:00.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:00 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/706406149' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:08:00.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:00 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4061488516' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:08:01.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:01 smithi132 ceph-mon[70676]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:01.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:01 smithi132 ceph-mon[70676]: pgmap v104: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.9 KiB/s wr, 0 op/s; 52/333 objects degraded (15.616%) 2024-02-19T19:08:01.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:01 smithi088 ceph-mon[88643]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:01.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:01 smithi088 ceph-mon[88643]: pgmap v104: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.9 KiB/s wr, 0 op/s; 52/333 objects degraded (15.616%) 2024-02-19T19:08:03.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:03 smithi088 ceph-mon[88643]: pgmap v105: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s wr, 0 op/s; 52/333 objects degraded (15.616%) 2024-02-19T19:08:03.351 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:08:03 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[106700]: 2024-02-19T19:08:03.152+0000 7f24139dd7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-02-19T19:08:03.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:03 smithi132 ceph-mon[70676]: pgmap v105: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 6.2 KiB/s wr, 0 op/s; 52/333 objects degraded (15.616%) 2024-02-19T19:08:04.101 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:08:03 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[106700]: 2024-02-19T19:08:03.711+0000 7f24139dd7c0 -1 osd.1 57 log_to_monitors true 2024-02-19T19:08:05.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:05.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:04 smithi132 ceph-mon[70676]: from='osd.1 [v2:172.21.15.88:6810/1931660446,v1:172.21.15.88:6811/1931660446]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-19T19:08:05.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:05.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:04 smithi132 ceph-mon[70676]: pgmap v106: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 102 B/s wr, 0 op/s; 52/333 objects degraded (15.616%) 2024-02-19T19:08:05.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:05.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:04 smithi088 ceph-mon[88643]: from='osd.1 [v2:172.21.15.88:6810/1931660446,v1:172.21.15.88:6811/1931660446]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-19T19:08:05.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:05.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:04 smithi088 ceph-mon[88643]: pgmap v106: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 102 B/s wr, 0 op/s; 52/333 objects degraded (15.616%) 2024-02-19T19:08:05.945 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 19:08:05 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[106700]: 2024-02-19T19:08:05.598+0000 7f2400e2d700 -1 osd.1 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T19:08:05.946 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:05 smithi088 ceph-mon[88643]: from='osd.1 [v2:172.21.15.88:6810/1931660446,v1:172.21.15.88:6811/1931660446]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-19T19:08:05.946 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:05 smithi088 ceph-mon[88643]: osdmap e60: 6 total, 5 up, 6 in 2024-02-19T19:08:05.946 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:05 smithi088 ceph-mon[88643]: from='osd.1 [v2:172.21.15.88:6810/1931660446,v1:172.21.15.88:6811/1931660446]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T19:08:05.946 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:05.946 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:05.946 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:05 smithi088 ceph-mon[88643]: Health check update: Degraded data redundancy: 52/333 objects degraded (15.616%), 16 pgs degraded (PG_DEGRADED) 2024-02-19T19:08:05.946 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:05 smithi088 ceph-mon[88643]: from='osd.1 [v2:172.21.15.88:6810/1931660446,v1:172.21.15.88:6811/1931660446]' entity='osd.1' 2024-02-19T19:08:06.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:05 smithi132 ceph-mon[70676]: from='osd.1 [v2:172.21.15.88:6810/1931660446,v1:172.21.15.88:6811/1931660446]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-19T19:08:06.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:05 smithi132 ceph-mon[70676]: osdmap e60: 6 total, 5 up, 6 in 2024-02-19T19:08:06.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:05 smithi132 ceph-mon[70676]: from='osd.1 [v2:172.21.15.88:6810/1931660446,v1:172.21.15.88:6811/1931660446]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T19:08:06.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:06.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:06.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:05 smithi132 ceph-mon[70676]: Health check update: Degraded data redundancy: 52/333 objects degraded (15.616%), 16 pgs degraded (PG_DEGRADED) 2024-02-19T19:08:06.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:05 smithi132 ceph-mon[70676]: from='osd.1 [v2:172.21.15.88:6810/1931660446,v1:172.21.15.88:6811/1931660446]' entity='osd.1' 2024-02-19T19:08:06.965 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:06 smithi088 ceph-mon[88643]: pgmap v108: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-02-19T19:08:06.965 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:06 smithi088 ceph-mon[88643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:08:06.966 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:06 smithi088 ceph-mon[88643]: osd.1 [v2:172.21.15.88:6810/1931660446,v1:172.21.15.88:6811/1931660446] boot 2024-02-19T19:08:06.966 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:06 smithi088 ceph-mon[88643]: osdmap e61: 6 total, 6 up, 6 in 2024-02-19T19:08:06.966 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T19:08:07.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:06 smithi132 ceph-mon[70676]: pgmap v108: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-02-19T19:08:07.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:06 smithi132 ceph-mon[70676]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:08:07.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:06 smithi132 ceph-mon[70676]: osd.1 [v2:172.21.15.88:6810/1931660446,v1:172.21.15.88:6811/1931660446] boot 2024-02-19T19:08:07.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:06 smithi132 ceph-mon[70676]: osdmap e61: 6 total, 6 up, 6 in 2024-02-19T19:08:07.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-19T19:08:08.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:08 smithi088 ceph-mon[88643]: osdmap e62: 6 total, 6 up, 6 in 2024-02-19T19:08:08.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:08 smithi088 ceph-mon[88643]: pgmap v111: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-02-19T19:08:08.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:08.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:08.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:08:08.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:08:08.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:09.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:08 smithi132 ceph-mon[70676]: osdmap e62: 6 total, 6 up, 6 in 2024-02-19T19:08:09.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:08 smithi132 ceph-mon[70676]: pgmap v111: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 52/333 objects degraded (15.616%) 2024-02-19T19:08:09.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:09.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:09.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:08:09.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:08:09.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:10.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:08:10.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:10.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:10.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:10.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-19T19:08:10.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:09 smithi132 ceph-mon[70676]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-19T19:08:10.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:09 smithi132 ceph-mon[70676]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-19T19:08:10.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:08:10.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:10.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:10.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:10.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-19T19:08:10.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:09 smithi088 ceph-mon[88643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-19T19:08:10.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:09 smithi088 ceph-mon[88643]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-02-19T19:08:11.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:10 smithi132 ceph-mon[70676]: pgmap v112: 65 pgs: 2 active+undersized, 63 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:11.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:10 smithi088 ceph-mon[88643]: pgmap v112: 65 pgs: 2 active+undersized, 63 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:11.885 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:11 smithi088 ceph-mon[88643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 52/333 objects degraded (15.616%), 16 pgs degraded) 2024-02-19T19:08:11.885 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:11 smithi088 ceph-mon[88643]: Cluster is now healthy 2024-02-19T19:08:12.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:11 smithi132 ceph-mon[70676]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 52/333 objects degraded (15.616%), 16 pgs degraded) 2024-02-19T19:08:12.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:11 smithi132 ceph-mon[70676]: Cluster is now healthy 2024-02-19T19:08:13.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:12 smithi132 ceph-mon[70676]: pgmap v113: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:13.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:12 smithi088 ceph-mon[88643]: pgmap v113: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:14.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:14.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:08:14.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:14.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:08:15.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:15 smithi132 ceph-mon[70676]: pgmap v114: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:15.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:15 smithi088 ceph-mon[88643]: pgmap v114: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:17.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:17 smithi132 ceph-mon[70676]: pgmap v115: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:17.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:17 smithi088 ceph-mon[88643]: pgmap v115: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:19.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:19 smithi132 ceph-mon[70676]: pgmap v116: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:19.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:19.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:19 smithi088 ceph-mon[88643]: pgmap v116: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:19.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:21.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:21 smithi132 ceph-mon[70676]: pgmap v117: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:21.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:21 smithi088 ceph-mon[88643]: pgmap v117: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:23.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:23 smithi132 ceph-mon[70676]: pgmap v118: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:23.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:23 smithi088 ceph-mon[88643]: pgmap v118: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:24.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-19T19:08:24.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-19T19:08:25.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:25 smithi132 ceph-mon[70676]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-19T19:08:25.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:25 smithi132 ceph-mon[70676]: Upgrade: osd.2 is safe to restart 2024-02-19T19:08:25.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:25 smithi132 ceph-mon[70676]: pgmap v119: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:25.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:25 smithi088 ceph-mon[88643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-02-19T19:08:25.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:25 smithi088 ceph-mon[88643]: Upgrade: osd.2 is safe to restart 2024-02-19T19:08:25.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:25 smithi088 ceph-mon[88643]: pgmap v119: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:26.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:26 smithi088 ceph-mon[88643]: Upgrade: Updating osd.2 2024-02-19T19:08:26.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:26.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-19T19:08:26.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:08:26.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:26 smithi088 ceph-mon[88643]: Deploying daemon osd.2 on smithi088 2024-02-19T19:08:26.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:26 smithi132 ceph-mon[70676]: Upgrade: Updating osd.2 2024-02-19T19:08:26.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:26.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-19T19:08:26.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:08:26.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:26 smithi132 ceph-mon[70676]: Deploying daemon osd.2 on smithi088 2024-02-19T19:08:27.542 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:27 smithi088 ceph-mon[88643]: pgmap v120: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:27.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:27 smithi132 ceph-mon[70676]: pgmap v120: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:28.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:08:28.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:08:28.956 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:28 smithi088 systemd[1]: Stopping Ceph osd.2 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:08:28.957 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:28 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[55263]: 2024-02-19T19:08:28.922+0000 7f98e07bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T19:08:28.957 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:28 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[55263]: 2024-02-19T19:08:28.922+0000 7f98e07bd700 -1 osd.2 62 *** Got signal Terminated *** 2024-02-19T19:08:28.957 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:28 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[55263]: 2024-02-19T19:08:28.922+0000 7f98e07bd700 -1 osd.2 62 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T19:08:29.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:29 smithi132 ceph-mon[70676]: pgmap v121: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:29.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:29 smithi132 ceph-mon[70676]: osd.2 marked itself down and dead 2024-02-19T19:08:29.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:29 smithi088 ceph-mon[88643]: pgmap v121: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:29.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:29 smithi088 ceph-mon[88643]: osd.2 marked itself down and dead 2024-02-19T19:08:30.101 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:29 smithi088 podman[112058]: 2024-02-19 19:08:29.817419995 +0000 UTC m=+0.993942219 container died b08d2d56f94c04a42e8a4a31dea1cf7dece7fa3732af150e02d9ae194767a1aa (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-02-19T19:08:30.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.237+0000 7fb99e953700 1 -- 172.21.15.88:0/3033950965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9980ffc60 msgr2=0x7fb99810a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:30.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.237+0000 7fb99e953700 1 --2- 172.21.15.88:0/3033950965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9980ffc60 0x7fb99810a6d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009910 tx=0x7fb98c009c20 comp rx=0 tx=0).stop 2024-02-19T19:08:30.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.237+0000 7fb99e953700 1 -- 172.21.15.88:0/3033950965 shutdown_connections 2024-02-19T19:08:30.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.237+0000 7fb99e953700 1 --2- 172.21.15.88:0/3033950965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9980ffc60 0x7fb99810a6d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.237+0000 7fb99e953700 1 --2- 172.21.15.88:0/3033950965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9980ff350 0x7fb9980ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.237+0000 7fb99e953700 1 -- 172.21.15.88:0/3033950965 >> 172.21.15.88:0/3033950965 conn(0x7fb9980748b0 msgr2=0x7fb998074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.237+0000 7fb99e953700 1 -- 172.21.15.88:0/3033950965 shutdown_connections 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.237+0000 7fb99e953700 1 -- 172.21.15.88:0/3033950965 wait complete. 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb99e953700 1 Processor -- start 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb99e953700 1 -- start start 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb99e953700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9980ff350 0x7fb998195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb99e953700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9980ffc60 0x7fb998196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb99e953700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb998196b50 con 0x7fb9980ff350 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb99e953700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99819a8e0 con 0x7fb9980ffc60 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb997fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9980ff350 0x7fb998195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb997fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9980ff350 0x7fb998195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36402/0 (socket says 172.21.15.88:36402) 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb997fff700 1 -- 172.21.15.88:0/2407540556 learned_addr learned my addr 172.21.15.88:0/2407540556 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:08:30.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb9977fe700 1 --2- 172.21.15.88:0/2407540556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9980ffc60 0x7fb998196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:30.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb997fff700 1 -- 172.21.15.88:0/2407540556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9980ffc60 msgr2=0x7fb998196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:30.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb997fff700 1 --2- 172.21.15.88:0/2407540556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9980ffc60 0x7fb998196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.238+0000 7fb997fff700 1 -- 172.21.15.88:0/2407540556 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb98c0095f0 con 0x7fb9980ff350 2024-02-19T19:08:30.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.239+0000 7fb997fff700 1 --2- 172.21.15.88:0/2407540556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9980ff350 0x7fb998195f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb98800b480 tx=0x7fb98800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:30.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.239+0000 7fb9957fa700 1 -- 172.21.15.88:0/2407540556 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9880118c0 con 0x7fb9980ff350 2024-02-19T19:08:30.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.239+0000 7fb99e953700 1 -- 172.21.15.88:0/2407540556 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99819abc0 con 0x7fb9980ff350 2024-02-19T19:08:30.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.239+0000 7fb9957fa700 1 -- 172.21.15.88:0/2407540556 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb988010380 con 0x7fb9980ff350 2024-02-19T19:08:30.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.239+0000 7fb9957fa700 1 -- 172.21.15.88:0/2407540556 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb98800f4f0 con 0x7fb9980ff350 2024-02-19T19:08:30.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.239+0000 7fb99e953700 1 -- 172.21.15.88:0/2407540556 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99819b110 con 0x7fb9980ff350 2024-02-19T19:08:30.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.240+0000 7fb99e953700 1 -- 172.21.15.88:0/2407540556 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb998107dd0 con 0x7fb9980ff350 2024-02-19T19:08:30.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.240+0000 7fb9957fa700 1 -- 172.21.15.88:0/2407540556 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb9880104f0 con 0x7fb9980ff350 2024-02-19T19:08:30.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.241+0000 7fb9957fa700 1 --2- 172.21.15.88:0/2407540556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb98006d8a0 0x7fb98006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:30.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.241+0000 7fb9957fa700 1 -- 172.21.15.88:0/2407540556 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb98808d0f0 con 0x7fb9980ff350 2024-02-19T19:08:30.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.241+0000 7fb9977fe700 1 --2- 172.21.15.88:0/2407540556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb98006d8a0 0x7fb98006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:30.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.241+0000 7fb9977fe700 1 --2- 172.21.15.88:0/2407540556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb98006d8a0 0x7fb98006fd50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb998197550 tx=0x7fb98c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:30.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.246+0000 7fb9957fa700 1 -- 172.21.15.88:0/2407540556 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb988056950 con 0x7fb9980ff350 2024-02-19T19:08:30.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.488+0000 7fb99e953700 1 -- 172.21.15.88:0/2407540556 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb99819b390 con 0x7fb98006d8a0 2024-02-19T19:08:30.509 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:30 smithi088 ceph-mon[88643]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:08:30.509 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:30 smithi088 ceph-mon[88643]: osdmap e63: 6 total, 5 up, 6 in 2024-02-19T19:08:30.510 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:30 smithi088 podman[112058]: 2024-02-19 19:08:30.355786678 +0000 UTC m=+1.532308901 container cleanup b08d2d56f94c04a42e8a4a31dea1cf7dece7fa3732af150e02d9ae194767a1aa (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, io.buildah.version=1.29.1, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-19T19:08:30.510 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:30 smithi088 bash[112058]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2 2024-02-19T19:08:30.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.490+0000 7fb9957fa700 1 -- 172.21.15.88:0/2407540556 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb99819b390 con 0x7fb98006d8a0 2024-02-19T19:08:30.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.492+0000 7fb99e953700 1 -- 172.21.15.88:0/2407540556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb98006d8a0 msgr2=0x7fb98006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:30.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.492+0000 7fb99e953700 1 --2- 172.21.15.88:0/2407540556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb98006d8a0 0x7fb98006fd50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb998197550 tx=0x7fb98c01a040 comp rx=0 tx=0).stop 2024-02-19T19:08:30.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.492+0000 7fb99e953700 1 -- 172.21.15.88:0/2407540556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9980ff350 msgr2=0x7fb998195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:30.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.492+0000 7fb99e953700 1 --2- 172.21.15.88:0/2407540556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9980ff350 0x7fb998195f30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb98800b480 tx=0x7fb98800b840 comp rx=0 tx=0).stop 2024-02-19T19:08:30.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.492+0000 7fb99e953700 1 -- 172.21.15.88:0/2407540556 shutdown_connections 2024-02-19T19:08:30.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.492+0000 7fb99e953700 1 --2- 172.21.15.88:0/2407540556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb98006d8a0 0x7fb98006fd50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.492+0000 7fb99e953700 1 --2- 172.21.15.88:0/2407540556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9980ff350 0x7fb998195f30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.492+0000 7fb99e953700 1 --2- 172.21.15.88:0/2407540556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9980ffc60 0x7fb998196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.492+0000 7fb99e953700 1 -- 172.21.15.88:0/2407540556 >> 172.21.15.88:0/2407540556 conn(0x7fb9980748b0 msgr2=0x7fb998104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:30.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.492+0000 7fb99e953700 1 -- 172.21.15.88:0/2407540556 shutdown_connections 2024-02-19T19:08:30.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.492+0000 7fb99e953700 1 -- 172.21.15.88:0/2407540556 wait complete. 2024-02-19T19:08:30.523 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:08:30.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.598+0000 7f4c63606700 1 -- 172.21.15.88:0/2023073912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5c104130 msgr2=0x7f4c5c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:30.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.598+0000 7f4c63606700 1 --2- 172.21.15.88:0/2023073912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5c104130 0x7f4c5c104500 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c009910 tx=0x7f4c4c009c20 comp rx=0 tx=0).stop 2024-02-19T19:08:30.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.598+0000 7f4c63606700 1 -- 172.21.15.88:0/2023073912 shutdown_connections 2024-02-19T19:08:30.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.598+0000 7f4c63606700 1 --2- 172.21.15.88:0/2023073912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0ffbd0 0x7f4c5c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.598+0000 7f4c63606700 1 --2- 172.21.15.88:0/2023073912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5c104130 0x7f4c5c104500 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.598+0000 7f4c63606700 1 -- 172.21.15.88:0/2023073912 >> 172.21.15.88:0/2023073912 conn(0x7f4c5c075130 msgr2=0x7f4c5c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:30.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.598+0000 7f4c63606700 1 -- 172.21.15.88:0/2023073912 shutdown_connections 2024-02-19T19:08:30.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.598+0000 7f4c63606700 1 -- 172.21.15.88:0/2023073912 wait complete. 2024-02-19T19:08:30.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c63606700 1 Processor -- start 2024-02-19T19:08:30.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c63606700 1 -- start start 2024-02-19T19:08:30.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c63606700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0ffbd0 0x7f4c5c194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:30.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c63606700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5c104130 0x7f4c5c195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:30.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c63606700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c1958d0 con 0x7f4c5c104130 2024-02-19T19:08:30.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c63606700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5c195a10 con 0x7f4c5c0ffbd0 2024-02-19T19:08:30.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c60ba1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5c104130 0x7f4c5c195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:30.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c60ba1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5c104130 0x7f4c5c195280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36420/0 (socket says 172.21.15.88:36420) 2024-02-19T19:08:30.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c60ba1700 1 -- 172.21.15.88:0/2105106321 learned_addr learned my addr 172.21.15.88:0/2105106321 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:08:30.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c613a2700 1 --2- 172.21.15.88:0/2105106321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0ffbd0 0x7f4c5c194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:30.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c60ba1700 1 -- 172.21.15.88:0/2105106321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0ffbd0 msgr2=0x7f4c5c194d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:30.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c60ba1700 1 --2- 172.21.15.88:0/2105106321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0ffbd0 0x7f4c5c194d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c60ba1700 1 -- 172.21.15.88:0/2105106321 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c0095f0 con 0x7f4c5c104130 2024-02-19T19:08:30.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.599+0000 7f4c60ba1700 1 --2- 172.21.15.88:0/2105106321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5c104130 0x7f4c5c195280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800c930 tx=0x7f4c5800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:30.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.600+0000 7f4c527fc700 1 -- 172.21.15.88:0/2105106321 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c58007ad0 con 0x7f4c5c104130 2024-02-19T19:08:30.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.600+0000 7f4c63606700 1 -- 172.21.15.88:0/2105106321 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c5c199860 con 0x7f4c5c104130 2024-02-19T19:08:30.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.600+0000 7f4c527fc700 1 -- 172.21.15.88:0/2105106321 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4c58007c30 con 0x7f4c5c104130 2024-02-19T19:08:30.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.600+0000 7f4c527fc700 1 -- 172.21.15.88:0/2105106321 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c58018610 con 0x7f4c5c104130 2024-02-19T19:08:30.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.600+0000 7f4c63606700 1 -- 172.21.15.88:0/2105106321 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5c199db0 con 0x7f4c5c104130 2024-02-19T19:08:30.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.601+0000 7f4c63606700 1 -- 172.21.15.88:0/2105106321 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5c1076a0 con 0x7f4c5c104130 2024-02-19T19:08:30.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.601+0000 7f4c527fc700 1 -- 172.21.15.88:0/2105106321 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4c5800f450 con 0x7f4c5c104130 2024-02-19T19:08:30.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.601+0000 7f4c527fc700 1 --2- 172.21.15.88:0/2105106321 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4c4806d8a0 0x7f4c4806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:30.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.602+0000 7f4c527fc700 1 -- 172.21.15.88:0/2105106321 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4c5808d6d0 con 0x7f4c5c104130 2024-02-19T19:08:30.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.602+0000 7f4c613a2700 1 --2- 172.21.15.88:0/2105106321 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4c4806d8a0 0x7f4c4806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:30.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.602+0000 7f4c613a2700 1 --2- 172.21.15.88:0/2105106321 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4c4806d8a0 0x7f4c4806fd50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c005f50 tx=0x7f4c4c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:30.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.606+0000 7f4c527fc700 1 -- 172.21.15.88:0/2105106321 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c58056f30 con 0x7f4c5c104130 2024-02-19T19:08:30.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:30 smithi132 ceph-mon[70676]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:08:30.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:30 smithi132 ceph-mon[70676]: osdmap e63: 6 total, 5 up, 6 in 2024-02-19T19:08:30.851 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:30 smithi088 podman[112086]: 2024-02-19 19:08:30.616451849 +0000 UTC m=+0.797645807 container remove b08d2d56f94c04a42e8a4a31dea1cf7dece7fa3732af150e02d9ae194767a1aa (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20231212, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-02-19T19:08:30.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.850+0000 7f4c63606700 1 -- 172.21.15.88:0/2105106321 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c5c02cca0 con 0x7f4c4806d8a0 2024-02-19T19:08:30.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.852+0000 7f4c527fc700 1 -- 172.21.15.88:0/2105106321 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4c5c02cca0 con 0x7f4c4806d8a0 2024-02-19T19:08:30.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.855+0000 7f4c63606700 1 -- 172.21.15.88:0/2105106321 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4c4806d8a0 msgr2=0x7f4c4806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:30.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.855+0000 7f4c63606700 1 --2- 172.21.15.88:0/2105106321 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4c4806d8a0 0x7f4c4806fd50 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c005f50 tx=0x7f4c4c01a040 comp rx=0 tx=0).stop 2024-02-19T19:08:30.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.855+0000 7f4c63606700 1 -- 172.21.15.88:0/2105106321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5c104130 msgr2=0x7f4c5c195280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:30.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.855+0000 7f4c63606700 1 --2- 172.21.15.88:0/2105106321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5c104130 0x7f4c5c195280 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4c5800c930 tx=0x7f4c5800ccf0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.855+0000 7f4c63606700 1 -- 172.21.15.88:0/2105106321 shutdown_connections 2024-02-19T19:08:30.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.855+0000 7f4c63606700 1 --2- 172.21.15.88:0/2105106321 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4c4806d8a0 0x7f4c4806fd50 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.855+0000 7f4c63606700 1 --2- 172.21.15.88:0/2105106321 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4c5c0ffbd0 0x7f4c5c194d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.855+0000 7f4c63606700 1 --2- 172.21.15.88:0/2105106321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c5c104130 0x7f4c5c195280 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:30.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.855+0000 7f4c63606700 1 -- 172.21.15.88:0/2105106321 >> 172.21.15.88:0/2105106321 conn(0x7f4c5c075130 msgr2=0x7f4c5c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:30.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.855+0000 7f4c63606700 1 -- 172.21.15.88:0/2105106321 shutdown_connections 2024-02-19T19:08:30.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.855+0000 7f4c63606700 1 -- 172.21.15.88:0/2105106321 wait complete. 2024-02-19T19:08:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.981+0000 7efef8ffa700 1 -- 172.21.15.88:0/3777904107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef40fefb0 msgr2=0x7efef40ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.981+0000 7efef8ffa700 1 --2- 172.21.15.88:0/3777904107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef40fefb0 0x7efef40ff380 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efee4009910 tx=0x7efee4009c20 comp rx=0 tx=0).stop 2024-02-19T19:08:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.981+0000 7efef8ffa700 1 -- 172.21.15.88:0/3777904107 shutdown_connections 2024-02-19T19:08:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.981+0000 7efef8ffa700 1 --2- 172.21.15.88:0/3777904107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efef40ff8c0 0x7efef410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.981+0000 7efef8ffa700 1 --2- 172.21.15.88:0/3777904107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef40fefb0 0x7efef40ff380 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.981+0000 7efef8ffa700 1 -- 172.21.15.88:0/3777904107 >> 172.21.15.88:0/3777904107 conn(0x7efef40fac60 msgr2=0x7efef40fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.981+0000 7efef8ffa700 1 -- 172.21.15.88:0/3777904107 shutdown_connections 2024-02-19T19:08:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.981+0000 7efef8ffa700 1 -- 172.21.15.88:0/3777904107 wait complete. 2024-02-19T19:08:31.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.982+0000 7efef8ffa700 1 Processor -- start 2024-02-19T19:08:31.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.982+0000 7efef8ffa700 1 -- start start 2024-02-19T19:08:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.982+0000 7efef8ffa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efef40fefb0 0x7efef4198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.982+0000 7efef8ffa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef40ff8c0 0x7efef4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.982+0000 7efef8ffa700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef4198c60 con 0x7efef40ff8c0 2024-02-19T19:08:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.982+0000 7efef8ffa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef419c9f0 con 0x7efef40fefb0 2024-02-19T19:08:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efef1d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef40ff8c0 0x7efef4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efef1d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef40ff8c0 0x7efef4198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36442/0 (socket says 172.21.15.88:36442) 2024-02-19T19:08:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efef1d9b700 1 -- 172.21.15.88:0/1495027519 learned_addr learned my addr 172.21.15.88:0/1495027519 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:08:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efef259c700 1 --2- 172.21.15.88:0/1495027519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efef40fefb0 0x7efef4198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efef1d9b700 1 -- 172.21.15.88:0/1495027519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efef40fefb0 msgr2=0x7efef4198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efef1d9b700 1 --2- 172.21.15.88:0/1495027519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efef40fefb0 0x7efef4198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efef1d9b700 1 -- 172.21.15.88:0/1495027519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee40095f0 con 0x7efef40ff8c0 2024-02-19T19:08:31.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efef1d9b700 1 --2- 172.21.15.88:0/1495027519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef40ff8c0 0x7efef4198580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efee800c8f0 tx=0x7efee800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:31.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efee37fe700 1 -- 172.21.15.88:0/1495027519 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efee8004a00 con 0x7efef40ff8c0 2024-02-19T19:08:31.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efee37fe700 1 -- 172.21.15.88:0/1495027519 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efee80054e0 con 0x7efef40ff8c0 2024-02-19T19:08:31.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efee37fe700 1 -- 172.21.15.88:0/1495027519 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efee8007cf0 con 0x7efef40ff8c0 2024-02-19T19:08:31.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efef8ffa700 1 -- 172.21.15.88:0/1495027519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef419ccd0 con 0x7efef40ff8c0 2024-02-19T19:08:31.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.983+0000 7efef8ffa700 1 -- 172.21.15.88:0/1495027519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef419d1d0 con 0x7efef40ff8c0 2024-02-19T19:08:31.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.984+0000 7efef8ffa700 1 -- 172.21.15.88:0/1495027519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef4109d90 con 0x7efef40ff8c0 2024-02-19T19:08:31.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.985+0000 7efee37fe700 1 -- 172.21.15.88:0/1495027519 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7efee8005020 con 0x7efef40ff8c0 2024-02-19T19:08:31.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.985+0000 7efee37fe700 1 --2- 172.21.15.88:0/1495027519 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efedc06d8a0 0x7efedc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:31.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.985+0000 7efee37fe700 1 -- 172.21.15.88:0/1495027519 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efee8013070 con 0x7efef40ff8c0 2024-02-19T19:08:31.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.985+0000 7efef259c700 1 --2- 172.21.15.88:0/1495027519 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efedc06d8a0 0x7efedc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:31.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.986+0000 7efef259c700 1 --2- 172.21.15.88:0/1495027519 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efedc06d8a0 0x7efedc06fd50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efee4006010 tx=0x7efee401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:31.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:30.990+0000 7efee37fe700 1 -- 172.21.15.88:0/1495027519 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efee8056b10 con 0x7efef40ff8c0 2024-02-19T19:08:31.211 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:31 smithi088 podman[112232]: 2024-02-19 19:08:30.95053199 +0000 UTC m=+0.018029496 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:08:31.211 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:31 smithi088 podman[112232]: 2024-02-19 19:08:31.06739165 +0000 UTC m=+0.134889144 container create 643ee47003cfea8fc0bb0ef6671b10abd24f35abb525e4390585a42a4faa7c0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-19T19:08:31.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.207+0000 7efef8ffa700 1 -- 172.21.15.88:0/1495027519 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efef419d500 con 0x7efedc06d8a0 2024-02-19T19:08:31.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.218+0000 7efee37fe700 1 -- 172.21.15.88:0/1495027519 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7efef419d500 con 0x7efedc06d8a0 2024-02-19T19:08:31.239 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:08:31.239 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (5m) 27s ago 16m 24.2M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (16m) 27s ago 16m 8422k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (14m) 96s ago 14m 8010k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (109s) 27s ago 16m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (100s) 96s ago 14m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (5m) 27s ago 15m 85.1M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (9m) 27s ago 9m 16.8M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (9m) 27s ago 9m 114M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (9m) 96s ago 9m 139M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (9m) 96s ago 9m 20.2M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (7m) 27s ago 17m 500M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (6m) 96s ago 14m 450M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (3m) 27s ago 18m 50.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (3m) 96s ago 14m 37.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (6m) 27s ago 16m 16.1M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:08:31.240 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (6m) 96s ago 14m 15.8M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:08:31.241 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (71s) 27s ago 13m 126M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:08:31.241 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (33s) 27s ago 13m 11.8M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:08:31.241 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (12m) 27s ago 12m 200M 1216M 18.2.0 dc2bc1663786 b08d2d56f94c 2024-02-19T19:08:31.241 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (12m) 96s ago 12m 236M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:08:31.241 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (11m) 96s ago 11m 215M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:08:31.241 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (11m) 96s ago 11m 201M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:08:31.241 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (6m) 27s ago 14m 69.2M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:08:31.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.220+0000 7efef8ffa700 1 -- 172.21.15.88:0/1495027519 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efedc06d8a0 msgr2=0x7efedc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:31.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.220+0000 7efef8ffa700 1 --2- 172.21.15.88:0/1495027519 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efedc06d8a0 0x7efedc06fd50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efee4006010 tx=0x7efee401a040 comp rx=0 tx=0).stop 2024-02-19T19:08:31.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.220+0000 7efef8ffa700 1 -- 172.21.15.88:0/1495027519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef40ff8c0 msgr2=0x7efef4198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:31.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.220+0000 7efef8ffa700 1 --2- 172.21.15.88:0/1495027519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef40ff8c0 0x7efef4198580 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efee800c8f0 tx=0x7efee800cc00 comp rx=0 tx=0).stop 2024-02-19T19:08:31.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.220+0000 7efef8ffa700 1 -- 172.21.15.88:0/1495027519 shutdown_connections 2024-02-19T19:08:31.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.220+0000 7efef8ffa700 1 --2- 172.21.15.88:0/1495027519 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efedc06d8a0 0x7efedc06fd50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.220+0000 7efef8ffa700 1 --2- 172.21.15.88:0/1495027519 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efef40fefb0 0x7efef4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.220+0000 7efef8ffa700 1 --2- 172.21.15.88:0/1495027519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef40ff8c0 0x7efef4198580 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.220+0000 7efef8ffa700 1 -- 172.21.15.88:0/1495027519 >> 172.21.15.88:0/1495027519 conn(0x7efef40fac60 msgr2=0x7efef40fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:31.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.221+0000 7efef8ffa700 1 -- 172.21.15.88:0/1495027519 shutdown_connections 2024-02-19T19:08:31.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.221+0000 7efef8ffa700 1 -- 172.21.15.88:0/1495027519 wait complete. 2024-02-19T19:08:31.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.326+0000 7f32cf8c2700 1 -- 172.21.15.88:0/3444682183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c8100460 msgr2=0x7f32c8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:31.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.326+0000 7f32cf8c2700 1 --2- 172.21.15.88:0/3444682183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c8100460 0x7f32c8100830 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009940 tx=0x7f32c4009c50 comp rx=0 tx=0).stop 2024-02-19T19:08:31.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.326+0000 7f32cf8c2700 1 -- 172.21.15.88:0/3444682183 shutdown_connections 2024-02-19T19:08:31.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.326+0000 7f32cf8c2700 1 --2- 172.21.15.88:0/3444682183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c8100d70 0x7f32c81094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.326+0000 7f32cf8c2700 1 --2- 172.21.15.88:0/3444682183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c8100460 0x7f32c8100830 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.326+0000 7f32cf8c2700 1 -- 172.21.15.88:0/3444682183 >> 172.21.15.88:0/3444682183 conn(0x7f32c80748b0 msgr2=0x7f32c8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:31.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.326+0000 7f32cf8c2700 1 -- 172.21.15.88:0/3444682183 shutdown_connections 2024-02-19T19:08:31.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.326+0000 7f32cf8c2700 1 -- 172.21.15.88:0/3444682183 wait complete. 2024-02-19T19:08:31.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.326+0000 7f32cf8c2700 1 Processor -- start 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.326+0000 7f32cf8c2700 1 -- start start 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cf8c2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c8100460 0x7f32c8196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cf8c2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c8100d70 0x7f32c81972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cf8c2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81978e0 con 0x7f32c8100d70 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cf8c2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c8197a20 con 0x7f32c8100460 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cce5d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c8100d70 0x7f32c81972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cce5d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c8100d70 0x7f32c81972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36450/0 (socket says 172.21.15.88:36450) 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cce5d700 1 -- 172.21.15.88:0/358051879 learned_addr learned my addr 172.21.15.88:0/358051879 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cd65e700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c8100460 0x7f32c8196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cce5d700 1 -- 172.21.15.88:0/358051879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c8100460 msgr2=0x7f32c8196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cce5d700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c8100460 0x7f32c8196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cce5d700 1 -- 172.21.15.88:0/358051879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c40095f0 con 0x7f32c8100d70 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cd65e700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c8100460 0x7f32c8196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:08:31.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cce5d700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c8100d70 0x7f32c81972c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32b800d900 tx=0x7f32b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:31.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32be7fc700 1 -- 172.21.15.88:0/358051879 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b80041f0 con 0x7f32c8100d70 2024-02-19T19:08:31.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32be7fc700 1 -- 172.21.15.88:0/358051879 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32b8004cf0 con 0x7f32c8100d70 2024-02-19T19:08:31.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32cf8c2700 1 -- 172.21.15.88:0/358051879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c819fdf0 con 0x7f32c8100d70 2024-02-19T19:08:31.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.327+0000 7f32be7fc700 1 -- 172.21.15.88:0/358051879 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32b800b750 con 0x7f32c8100d70 2024-02-19T19:08:31.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.328+0000 7f32cf8c2700 1 -- 172.21.15.88:0/358051879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c81a02b0 con 0x7f32c8100d70 2024-02-19T19:08:31.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.329+0000 7f32cf8c2700 1 -- 172.21.15.88:0/358051879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c8106bd0 con 0x7f32c8100d70 2024-02-19T19:08:31.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.329+0000 7f32be7fc700 1 -- 172.21.15.88:0/358051879 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f32b8004350 con 0x7f32c8100d70 2024-02-19T19:08:31.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.330+0000 7f32be7fc700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f32b406d8a0 0x7f32b406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:31.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.330+0000 7f32be7fc700 1 -- 172.21.15.88:0/358051879 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f32b808d5b0 con 0x7f32c8100d70 2024-02-19T19:08:31.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.330+0000 7f32cd65e700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f32b406d8a0 0x7f32b406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:31.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.330+0000 7f32cd65e700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f32b406d8a0 0x7f32b406fd50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32c4005b40 tx=0x7f32c4005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:31.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.334+0000 7f32be7fc700 1 -- 172.21.15.88:0/358051879 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32b8056e10 con 0x7f32c8100d70 2024-02-19T19:08:31.517 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:31 smithi088 ceph-mon[88643]: pgmap v123: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:31.518 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:31 smithi088 ceph-mon[88643]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-19T19:08:31.518 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:31 smithi088 ceph-mon[88643]: osdmap e64: 6 total, 5 up, 6 in 2024-02-19T19:08:31.518 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:31 smithi088 podman[112232]: 2024-02-19 19:08:31.30347194 +0000 UTC m=+0.370969427 container init 643ee47003cfea8fc0bb0ef6671b10abd24f35abb525e4390585a42a4faa7c0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-02-19T19:08:31.518 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:31 smithi088 podman[112232]: 2024-02-19 19:08:31.306950133 +0000 UTC m=+0.374447638 container start 643ee47003cfea8fc0bb0ef6671b10abd24f35abb525e4390585a42a4faa7c0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2) 2024-02-19T19:08:31.518 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:31 smithi088 podman[112232]: 2024-02-19 19:08:31.395130713 +0000 UTC m=+0.462628201 container attach 643ee47003cfea8fc0bb0ef6671b10abd24f35abb525e4390585a42a4faa7c0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-19T19:08:31.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.634+0000 7f32cf8c2700 1 -- 172.21.15.88:0/358051879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f32c81065b0 con 0x7f32c8100d70 2024-02-19T19:08:31.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.634+0000 7f32be7fc700 1 -- 172.21.15.88:0/358051879 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f32b805a4d0 con 0x7f32c8100d70 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:08:31.656 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:08:31.657 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:08:31.657 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:08:31.657 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 7, 2024-02-19T19:08:31.657 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:08:31.657 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:08:31.657 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:08:31.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.637+0000 7f32cf8c2700 1 -- 172.21.15.88:0/358051879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f32b406d8a0 msgr2=0x7f32b406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:31.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.637+0000 7f32cf8c2700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f32b406d8a0 0x7f32b406fd50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f32c4005b40 tx=0x7f32c4005a90 comp rx=0 tx=0).stop 2024-02-19T19:08:31.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.637+0000 7f32cf8c2700 1 -- 172.21.15.88:0/358051879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c8100d70 msgr2=0x7f32c81972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:31.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.637+0000 7f32cf8c2700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c8100d70 0x7f32c81972c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f32b800d900 tx=0x7f32b800dcc0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.637+0000 7f32cf8c2700 1 -- 172.21.15.88:0/358051879 shutdown_connections 2024-02-19T19:08:31.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.637+0000 7f32cf8c2700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f32b406d8a0 0x7f32b406fd50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.637+0000 7f32cf8c2700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32c8100460 0x7f32c8196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.637+0000 7f32cf8c2700 1 --2- 172.21.15.88:0/358051879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32c8100d70 0x7f32c81972c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.637+0000 7f32cf8c2700 1 -- 172.21.15.88:0/358051879 >> 172.21.15.88:0/358051879 conn(0x7f32c80748b0 msgr2=0x7f32c8103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:31.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.637+0000 7f32cf8c2700 1 -- 172.21.15.88:0/358051879 shutdown_connections 2024-02-19T19:08:31.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.637+0000 7f32cf8c2700 1 -- 172.21.15.88:0/358051879 wait complete. 2024-02-19T19:08:31.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:31 smithi132 ceph-mon[70676]: pgmap v123: 65 pgs: 7 peering, 5 stale+active+clean, 53 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:31.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:31 smithi132 ceph-mon[70676]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-19T19:08:31.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:31 smithi132 ceph-mon[70676]: osdmap e64: 6 total, 5 up, 6 in 2024-02-19T19:08:31.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.742+0000 7fa1fbd25700 1 -- 172.21.15.88:0/111817183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4101d20 msgr2=0x7fa1f410a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:31.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.742+0000 7fa1fbd25700 1 --2- 172.21.15.88:0/111817183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4101d20 0x7fa1f410a220 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009910 tx=0x7fa1e4009c20 comp rx=0 tx=0).stop 2024-02-19T19:08:31.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.742+0000 7fa1fbd25700 1 -- 172.21.15.88:0/111817183 shutdown_connections 2024-02-19T19:08:31.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.742+0000 7fa1fbd25700 1 --2- 172.21.15.88:0/111817183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4101d20 0x7fa1f410a220 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.742+0000 7fa1fbd25700 1 --2- 172.21.15.88:0/111817183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f4101410 0x7fa1f41017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.742+0000 7fa1fbd25700 1 -- 172.21.15.88:0/111817183 >> 172.21.15.88:0/111817183 conn(0x7fa1f40fac90 msgr2=0x7fa1f40fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:31.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.742+0000 7fa1fbd25700 1 -- 172.21.15.88:0/111817183 shutdown_connections 2024-02-19T19:08:31.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.742+0000 7fa1fbd25700 1 -- 172.21.15.88:0/111817183 wait complete. 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.742+0000 7fa1fbd25700 1 Processor -- start 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.742+0000 7fa1fbd25700 1 -- start start 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.742+0000 7fa1fbd25700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f4101410 0x7fa1f40ff070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1fbd25700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4101d20 0x7fa1f40ff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1fbd25700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40ffaf0 con 0x7fa1f4101d20 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1fbd25700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f40ffc30 con 0x7fa1f4101410 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1f92c0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4101d20 0x7fa1f40ff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1f92c0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4101d20 0x7fa1f40ff5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36470/0 (socket says 172.21.15.88:36470) 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1f92c0700 1 -- 172.21.15.88:0/3886149514 learned_addr learned my addr 172.21.15.88:0/3886149514 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1f9ac1700 1 --2- 172.21.15.88:0/3886149514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f4101410 0x7fa1f40ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1f92c0700 1 -- 172.21.15.88:0/3886149514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f4101410 msgr2=0x7fa1f40ff070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:31.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1f92c0700 1 --2- 172.21.15.88:0/3886149514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f4101410 0x7fa1f40ff070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:31.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1f92c0700 1 -- 172.21.15.88:0/3886149514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e40095f0 con 0x7fa1f4101d20 2024-02-19T19:08:31.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1f92c0700 1 --2- 172.21.15.88:0/3886149514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4101d20 0x7fa1f40ff5b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4006010 tx=0x7fa1e400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:31.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1eaffd700 1 -- 172.21.15.88:0/3886149514 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1e401d070 con 0x7fa1f4101d20 2024-02-19T19:08:31.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1eaffd700 1 -- 172.21.15.88:0/3886149514 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1e4004fb0 con 0x7fa1f4101d20 2024-02-19T19:08:31.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1fbd25700 1 -- 172.21.15.88:0/3886149514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f40ffeb0 con 0x7fa1f4101d20 2024-02-19T19:08:31.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1eaffd700 1 -- 172.21.15.88:0/3886149514 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1e4022ea0 con 0x7fa1f4101d20 2024-02-19T19:08:31.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.743+0000 7fa1fbd25700 1 -- 172.21.15.88:0/3886149514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f41a2340 con 0x7fa1f4101d20 2024-02-19T19:08:31.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.744+0000 7fa1fbd25700 1 -- 172.21.15.88:0/3886149514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f4107890 con 0x7fa1f4101d20 2024-02-19T19:08:31.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.745+0000 7fa1eaffd700 1 -- 172.21.15.88:0/3886149514 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa1e4022470 con 0x7fa1f4101d20 2024-02-19T19:08:31.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.745+0000 7fa1eaffd700 1 --2- 172.21.15.88:0/3886149514 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1e006d8a0 0x7fa1e006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:31.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.745+0000 7fa1eaffd700 1 -- 172.21.15.88:0/3886149514 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa1e408ec20 con 0x7fa1f4101d20 2024-02-19T19:08:31.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.745+0000 7fa1f9ac1700 1 --2- 172.21.15.88:0/3886149514 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1e006d8a0 0x7fa1e006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:31.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.746+0000 7fa1f9ac1700 1 --2- 172.21.15.88:0/3886149514 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1e006d8a0 0x7fa1e006fd50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0005fd0 tx=0x7fa1f000a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:31.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.750+0000 7fa1eaffd700 1 -- 172.21.15.88:0/3886149514 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa1e4058650 con 0x7fa1f4101d20 2024-02-19T19:08:31.771 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:31 smithi088 podman[112232]: 2024-02-19 19:08:31.601036317 +0000 UTC m=+0.668533804 container died 643ee47003cfea8fc0bb0ef6671b10abd24f35abb525e4390585a42a4faa7c0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-19T19:08:32.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.995+0000 7fa1fbd25700 1 -- 172.21.15.88:0/3886149514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa1f402cc30 con 0x7fa1f4101d20 2024-02-19T19:08:32.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.995+0000 7fa1eaffd700 1 -- 172.21.15.88:0/3886149514 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1892 (secure 0 0 0) 0x7fa1e4027780 con 0x7fa1f4101d20 2024-02-19T19:08:32.016 INFO:teuthology.orchestra.run.smithi088.stdout:e20 2024-02-19T19:08:32.016 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:08:32.016 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 20 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:05:14.913778+0000 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:08:32.017 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:08:32.018 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:08:32.019 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:08:32.019 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:08:32.019 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:08:32.019 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:08:32.019 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:08:32.019 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:08:32.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.997+0000 7fa1fbd25700 1 -- 172.21.15.88:0/3886149514 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1e006d8a0 msgr2=0x7fa1e006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:32.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.997+0000 7fa1fbd25700 1 --2- 172.21.15.88:0/3886149514 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1e006d8a0 0x7fa1e006fd50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0005fd0 tx=0x7fa1f000a560 comp rx=0 tx=0).stop 2024-02-19T19:08:32.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.997+0000 7fa1fbd25700 1 -- 172.21.15.88:0/3886149514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4101d20 msgr2=0x7fa1f40ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:32.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.997+0000 7fa1fbd25700 1 --2- 172.21.15.88:0/3886149514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4101d20 0x7fa1f40ff5b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4006010 tx=0x7fa1e400be20 comp rx=0 tx=0).stop 2024-02-19T19:08:32.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.998+0000 7fa1fbd25700 1 -- 172.21.15.88:0/3886149514 shutdown_connections 2024-02-19T19:08:32.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.998+0000 7fa1fbd25700 1 --2- 172.21.15.88:0/3886149514 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1e006d8a0 0x7fa1e006fd50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.998+0000 7fa1fbd25700 1 --2- 172.21.15.88:0/3886149514 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1f4101410 0x7fa1f40ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.998+0000 7fa1fbd25700 1 --2- 172.21.15.88:0/3886149514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1f4101d20 0x7fa1f40ff5b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.998+0000 7fa1fbd25700 1 -- 172.21.15.88:0/3886149514 >> 172.21.15.88:0/3886149514 conn(0x7fa1f40fac90 msgr2=0x7fa1f41049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:32.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.998+0000 7fa1fbd25700 1 -- 172.21.15.88:0/3886149514 shutdown_connections 2024-02-19T19:08:32.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:31.998+0000 7fa1fbd25700 1 -- 172.21.15.88:0/3886149514 wait complete. 2024-02-19T19:08:32.021 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-02-19T19:08:32.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.105+0000 7f923278b700 1 -- 172.21.15.88:0/1511362553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f922c0ff3c0 msgr2=0x7f922c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:32.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.105+0000 7f923278b700 1 --2- 172.21.15.88:0/1511362553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f922c0ff3c0 0x7f922c0ff790 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f921c009910 tx=0x7f921c009c20 comp rx=0 tx=0).stop 2024-02-19T19:08:32.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.105+0000 7f923278b700 1 -- 172.21.15.88:0/1511362553 shutdown_connections 2024-02-19T19:08:32.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.105+0000 7f923278b700 1 --2- 172.21.15.88:0/1511362553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f922c0ffd60 0x7f922c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.105+0000 7f923278b700 1 --2- 172.21.15.88:0/1511362553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f922c0ff3c0 0x7f922c0ff790 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.105+0000 7f923278b700 1 -- 172.21.15.88:0/1511362553 >> 172.21.15.88:0/1511362553 conn(0x7f922c0fac60 msgr2=0x7f922c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:32.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.105+0000 7f923278b700 1 -- 172.21.15.88:0/1511362553 shutdown_connections 2024-02-19T19:08:32.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.105+0000 7f923278b700 1 -- 172.21.15.88:0/1511362553 wait complete. 2024-02-19T19:08:32.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f923278b700 1 Processor -- start 2024-02-19T19:08:32.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f923278b700 1 -- start start 2024-02-19T19:08:32.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f923278b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f922c0ff3c0 0x7f922c195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:32.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f923278b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f922c0ffd60 0x7f922c1964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:32.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f923278b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c196ba0 con 0x7f922c0ffd60 2024-02-19T19:08:32.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f923278b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c19a930 con 0x7f922c0ff3c0 2024-02-19T19:08:32.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f922b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f922c0ffd60 0x7f922c1964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:32.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f922b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f922c0ffd60 0x7f922c1964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36486/0 (socket says 172.21.15.88:36486) 2024-02-19T19:08:32.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f922b7fe700 1 -- 172.21.15.88:0/1173078022 learned_addr learned my addr 172.21.15.88:0/1173078022 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:08:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f922bfff700 1 --2- 172.21.15.88:0/1173078022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f922c0ff3c0 0x7f922c195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f922b7fe700 1 -- 172.21.15.88:0/1173078022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f922c0ff3c0 msgr2=0x7f922c195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.106+0000 7f922b7fe700 1 --2- 172.21.15.88:0/1173078022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f922c0ff3c0 0x7f922c195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.107+0000 7f922b7fe700 1 -- 172.21.15.88:0/1173078022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921c0095f0 con 0x7f922c0ffd60 2024-02-19T19:08:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.107+0000 7f922b7fe700 1 --2- 172.21.15.88:0/1173078022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f922c0ffd60 0x7f922c1964c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f922000b480 tx=0x7f922000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.107+0000 7f92297fa700 1 -- 172.21.15.88:0/1173078022 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9220011800 con 0x7f922c0ffd60 2024-02-19T19:08:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.107+0000 7f92297fa700 1 -- 172.21.15.88:0/1173078022 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9220011e20 con 0x7f922c0ffd60 2024-02-19T19:08:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.107+0000 7f923278b700 1 -- 172.21.15.88:0/1173078022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c19ac10 con 0x7f922c0ffd60 2024-02-19T19:08:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.107+0000 7f92297fa700 1 -- 172.21.15.88:0/1173078022 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9220010ea0 con 0x7f922c0ffd60 2024-02-19T19:08:32.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.107+0000 7f923278b700 1 -- 172.21.15.88:0/1173078022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c19b160 con 0x7f922c0ffd60 2024-02-19T19:08:32.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.108+0000 7f923278b700 1 -- 172.21.15.88:0/1173078022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c04f2a0 con 0x7f922c0ffd60 2024-02-19T19:08:32.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.108+0000 7f92297fa700 1 -- 172.21.15.88:0/1173078022 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9220010380 con 0x7f922c0ffd60 2024-02-19T19:08:32.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.109+0000 7f92297fa700 1 --2- 172.21.15.88:0/1173078022 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921406d8a0 0x7f921406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:32.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.109+0000 7f92297fa700 1 -- 172.21.15.88:0/1173078022 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f922008cef0 con 0x7f922c0ffd60 2024-02-19T19:08:32.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.109+0000 7f922bfff700 1 --2- 172.21.15.88:0/1173078022 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921406d8a0 0x7f921406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:32.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.109+0000 7f922bfff700 1 --2- 172.21.15.88:0/1173078022 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921406d8a0 0x7f921406fd50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f921c006010 tx=0x7f921c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:32.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.114+0000 7f92297fa700 1 -- 172.21.15.88:0/1173078022 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92200567d0 con 0x7f922c0ffd60 2024-02-19T19:08:32.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.346+0000 7f923278b700 1 -- 172.21.15.88:0/1173078022 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f922c19b4f0 con 0x7f921406d8a0 2024-02-19T19:08:32.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.347+0000 7f92297fa700 1 -- 172.21.15.88:0/1173078022 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f922c19b4f0 con 0x7f921406d8a0 2024-02-19T19:08:32.368 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:08:32.368 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:08:32.368 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:08:32.368 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:08:32.369 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:08:32.369 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:08:32.369 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:08:32.369 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:08:32.369 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:08:32.369 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "8/23 daemons upgraded", 2024-02-19T19:08:32.369 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading osd daemons", 2024-02-19T19:08:32.369 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:08:32.369 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:08:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.350+0000 7f923278b700 1 -- 172.21.15.88:0/1173078022 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921406d8a0 msgr2=0x7f921406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.350+0000 7f923278b700 1 --2- 172.21.15.88:0/1173078022 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921406d8a0 0x7f921406fd50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f921c006010 tx=0x7f921c01a040 comp rx=0 tx=0).stop 2024-02-19T19:08:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.350+0000 7f923278b700 1 -- 172.21.15.88:0/1173078022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f922c0ffd60 msgr2=0x7f922c1964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.350+0000 7f923278b700 1 --2- 172.21.15.88:0/1173078022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f922c0ffd60 0x7f922c1964c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f922000b480 tx=0x7f922000b790 comp rx=0 tx=0).stop 2024-02-19T19:08:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.350+0000 7f923278b700 1 -- 172.21.15.88:0/1173078022 shutdown_connections 2024-02-19T19:08:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.350+0000 7f923278b700 1 --2- 172.21.15.88:0/1173078022 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921406d8a0 0x7f921406fd50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.350+0000 7f923278b700 1 --2- 172.21.15.88:0/1173078022 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f922c0ff3c0 0x7f922c195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.350+0000 7f923278b700 1 --2- 172.21.15.88:0/1173078022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f922c0ffd60 0x7f922c1964c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.350+0000 7f923278b700 1 -- 172.21.15.88:0/1173078022 >> 172.21.15.88:0/1173078022 conn(0x7f922c0fac60 msgr2=0x7f922c0fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.350+0000 7f923278b700 1 -- 172.21.15.88:0/1173078022 shutdown_connections 2024-02-19T19:08:32.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.350+0000 7f923278b700 1 -- 172.21.15.88:0/1173078022 wait complete. 2024-02-19T19:08:32.452 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:32 smithi088 podman[112301]: 2024-02-19 19:08:32.208064779 +0000 UTC m=+0.601882558 container remove 643ee47003cfea8fc0bb0ef6671b10abd24f35abb525e4390585a42a4faa7c0d (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-02-19T19:08:32.452 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:32 smithi088 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.2.service: Deactivated successfully. 2024-02-19T19:08:32.452 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:32 smithi088 systemd[1]: Stopped Ceph osd.2 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:08:32.452 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:32 smithi088 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.2.service: Consumed 56.676s CPU time. 2024-02-19T19:08:32.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.461+0000 7f2a22877700 1 -- 172.21.15.88:0/1133208337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c1004c0 msgr2=0x7f2a1c100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:32.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.461+0000 7f2a22877700 1 --2- 172.21.15.88:0/1133208337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c1004c0 0x7f2a1c100890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009910 tx=0x7f2a10009c20 comp rx=0 tx=0).stop 2024-02-19T19:08:32.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.461+0000 7f2a22877700 1 -- 172.21.15.88:0/1133208337 shutdown_connections 2024-02-19T19:08:32.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.461+0000 7f2a22877700 1 --2- 172.21.15.88:0/1133208337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c100e60 0x7f2a1c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.461+0000 7f2a22877700 1 --2- 172.21.15.88:0/1133208337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c1004c0 0x7f2a1c100890 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.461+0000 7f2a22877700 1 -- 172.21.15.88:0/1133208337 >> 172.21.15.88:0/1133208337 conn(0x7f2a1c0fbd80 msgr2=0x7f2a1c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:32.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.461+0000 7f2a22877700 1 -- 172.21.15.88:0/1133208337 shutdown_connections 2024-02-19T19:08:32.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.461+0000 7f2a22877700 1 -- 172.21.15.88:0/1133208337 wait complete. 2024-02-19T19:08:32.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.462+0000 7f2a22877700 1 Processor -- start 2024-02-19T19:08:32.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.462+0000 7f2a22877700 1 -- start start 2024-02-19T19:08:32.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a22877700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c1004c0 0x7f2a1c193b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a22877700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c100e60 0x7f2a1c194060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a22877700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c196ca0 con 0x7f2a1c100e60 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a22877700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c1945a0 con 0x7f2a1c1004c0 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a1b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c100e60 0x7f2a1c194060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a1b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c100e60 0x7f2a1c194060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36502/0 (socket says 172.21.15.88:36502) 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a1b7fe700 1 -- 172.21.15.88:0/2178639952 learned_addr learned my addr 172.21.15.88:0/2178639952 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a1bfff700 1 --2- 172.21.15.88:0/2178639952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c1004c0 0x7f2a1c193b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a1b7fe700 1 -- 172.21.15.88:0/2178639952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c1004c0 msgr2=0x7f2a1c193b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a1b7fe700 1 --2- 172.21.15.88:0/2178639952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c1004c0 0x7f2a1c193b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a1b7fe700 1 -- 172.21.15.88:0/2178639952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a100095f0 con 0x7f2a1c100e60 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a1b7fe700 1 --2- 172.21.15.88:0/2178639952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c100e60 0x7f2a1c194060 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00c8f0 tx=0x7f2a0c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.463+0000 7f2a197fa700 1 -- 172.21.15.88:0/2178639952 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a0c007a50 con 0x7f2a1c100e60 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.464+0000 7f2a197fa700 1 -- 172.21.15.88:0/2178639952 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a0c007bb0 con 0x7f2a1c100e60 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.464+0000 7f2a197fa700 1 -- 172.21.15.88:0/2178639952 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a0c018630 con 0x7f2a1c100e60 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.464+0000 7f2a22877700 1 -- 172.21.15.88:0/2178639952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c194880 con 0x7f2a1c100e60 2024-02-19T19:08:32.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.464+0000 7f2a22877700 1 -- 172.21.15.88:0/2178639952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c1a6950 con 0x7f2a1c100e60 2024-02-19T19:08:32.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.465+0000 7f2a22877700 1 -- 172.21.15.88:0/2178639952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1c1045b0 con 0x7f2a1c100e60 2024-02-19T19:08:32.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.465+0000 7f2a197fa700 1 -- 172.21.15.88:0/2178639952 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2a0c007d20 con 0x7f2a1c100e60 2024-02-19T19:08:32.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.466+0000 7f2a197fa700 1 --2- 172.21.15.88:0/2178639952 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a0406d8a0 0x7f2a0406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:08:32.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.466+0000 7f2a197fa700 1 -- 172.21.15.88:0/2178639952 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2a0c08d500 con 0x7f2a1c100e60 2024-02-19T19:08:32.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.466+0000 7f2a1bfff700 1 --2- 172.21.15.88:0/2178639952 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a0406d8a0 0x7f2a0406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:08:32.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.467+0000 7f2a1bfff700 1 --2- 172.21.15.88:0/2178639952 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a0406d8a0 0x7f2a0406fd50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a10006010 tx=0x7f2a1001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:08:32.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.471+0000 7f2a197fa700 1 -- 172.21.15.88:0/2178639952 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a0c056cc0 con 0x7f2a1c100e60 2024-02-19T19:08:32.713 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:32 smithi088 systemd[1]: Starting Ceph osd.2 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:08:32.713 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:32 smithi088 podman[112469]: 2024-02-19 19:08:32.562565269 +0000 UTC m=+0.021115878 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:08:32.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:32 smithi088 ceph-mon[88643]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:32.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:32 smithi088 ceph-mon[88643]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:32.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:32 smithi088 ceph-mon[88643]: from='client.34308 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:32.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:32 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/358051879' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:32.713 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:32 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3886149514' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:08:32.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:32 smithi132 ceph-mon[70676]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:32.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:32 smithi132 ceph-mon[70676]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:32.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:32 smithi132 ceph-mon[70676]: from='client.34308 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:32.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:32 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/358051879' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:32.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:32 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3886149514' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:08:32.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.765+0000 7f2a22877700 1 -- 172.21.15.88:0/2178639952 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2a1c04f2a0 con 0x7f2a1c100e60 2024-02-19T19:08:32.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.765+0000 7f2a197fa700 1 -- 172.21.15.88:0/2178639952 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+676 (secure 0 0 0) 0x7f2a0c05a380 con 0x7f2a1c100e60 2024-02-19T19:08:32.787 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 1 pg peering; Degraded data redundancy: 13/333 objects degraded (3.904%), 4 pgs degraded 2024-02-19T19:08:32.787 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-19T19:08:32.787 INFO:teuthology.orchestra.run.smithi088.stdout: osd.2 (root=default,host=smithi088) is down 2024-02-19T19:08:32.787 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 1 pg peering 2024-02-19T19:08:32.787 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.10 is stuck peering for 76s, current state peering, last acting [1,0] 2024-02-19T19:08:32.787 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 13/333 objects degraded (3.904%), 4 pgs degraded 2024-02-19T19:08:32.787 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-02-19T19:08:32.787 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-02-19T19:08:32.787 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-02-19T19:08:32.787 INFO:teuthology.orchestra.run.smithi088.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-02-19T19:08:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.767+0000 7f2a22877700 1 -- 172.21.15.88:0/2178639952 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a0406d8a0 msgr2=0x7f2a0406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.767+0000 7f2a22877700 1 --2- 172.21.15.88:0/2178639952 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a0406d8a0 0x7f2a0406fd50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a10006010 tx=0x7f2a1001a040 comp rx=0 tx=0).stop 2024-02-19T19:08:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.768+0000 7f2a22877700 1 -- 172.21.15.88:0/2178639952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c100e60 msgr2=0x7f2a1c194060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:08:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.768+0000 7f2a22877700 1 --2- 172.21.15.88:0/2178639952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c100e60 0x7f2a1c194060 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c00c8f0 tx=0x7f2a0c00ccb0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.768+0000 7f2a22877700 1 -- 172.21.15.88:0/2178639952 shutdown_connections 2024-02-19T19:08:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.768+0000 7f2a22877700 1 --2- 172.21.15.88:0/2178639952 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a0406d8a0 0x7f2a0406fd50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.768+0000 7f2a22877700 1 --2- 172.21.15.88:0/2178639952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a1c1004c0 0x7f2a1c193b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.768+0000 7f2a22877700 1 --2- 172.21.15.88:0/2178639952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a1c100e60 0x7f2a1c194060 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:08:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.768+0000 7f2a22877700 1 -- 172.21.15.88:0/2178639952 >> 172.21.15.88:0/2178639952 conn(0x7f2a1c0fbd80 msgr2=0x7f2a1c0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:08:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.768+0000 7f2a22877700 1 -- 172.21.15.88:0/2178639952 shutdown_connections 2024-02-19T19:08:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:08:32.768+0000 7f2a22877700 1 -- 172.21.15.88:0/2178639952 wait complete. 2024-02-19T19:08:33.007 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:32 smithi088 podman[112469]: 2024-02-19 19:08:32.69272421 +0000 UTC m=+0.151274827 container create 9fec5ce54ef36cdbd108e2ea543cda09a6f91cf5de145488249266c7c4116ecd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T19:08:33.007 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:32 smithi088 podman[112469]: 2024-02-19 19:08:32.905758393 +0000 UTC m=+0.364308980 container init 9fec5ce54ef36cdbd108e2ea543cda09a6f91cf5de145488249266c7c4116ecd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-19T19:08:33.007 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:32 smithi088 podman[112469]: 2024-02-19 19:08:32.909324228 +0000 UTC m=+0.367874821 container start 9fec5ce54ef36cdbd108e2ea543cda09a6f91cf5de145488249266c7c4116ecd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5) 2024-02-19T19:08:33.007 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:32 smithi088 podman[112469]: 2024-02-19 19:08:32.987018557 +0000 UTC m=+0.445569151 container attach 9fec5ce54ef36cdbd108e2ea543cda09a6f91cf5de145488249266c7c4116ecd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-19T19:08:33.719 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:33 smithi088 ceph-mon[88643]: pgmap v125: 65 pgs: 4 active+undersized+degraded, 7 peering, 2 stale+active+clean, 4 active+undersized, 48 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 13/333 objects degraded (3.904%) 2024-02-19T19:08:33.720 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:33 smithi088 ceph-mon[88643]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:33.720 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:33 smithi088 ceph-mon[88643]: Health check failed: Degraded data redundancy: 13/333 objects degraded (3.904%), 4 pgs degraded (PG_DEGRADED) 2024-02-19T19:08:33.720 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:33 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2178639952' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:08:33.720 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate[112486]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-19T19:08:33.720 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate[112486]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-19T19:08:33.720 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate[112486]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-19T19:08:33.720 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate[112486]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-19T19:08:33.720 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate[112486]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-19T19:08:33.720 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate[112486]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-19T19:08:33.721 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 bash[112469]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-19T19:08:33.721 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 bash[112469]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-19T19:08:33.721 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 bash[112469]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-19T19:08:33.721 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 bash[112469]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-19T19:08:33.721 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 bash[112469]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-02-19T19:08:33.721 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 bash[112469]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-02-19T19:08:33.721 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 bash[112469]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-19T19:08:33.721 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate[112486]: --> ceph-volume raw activate successful for osd ID: 2 2024-02-19T19:08:33.721 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 podman[112469]: 2024-02-19 19:08:33.37258571 +0000 UTC m=+0.831136296 container died 9fec5ce54ef36cdbd108e2ea543cda09a6f91cf5de145488249266c7c4116ecd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-02-19T19:08:33.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:33 smithi132 ceph-mon[70676]: pgmap v125: 65 pgs: 4 active+undersized+degraded, 7 peering, 2 stale+active+clean, 4 active+undersized, 48 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 13/333 objects degraded (3.904%) 2024-02-19T19:08:33.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:33 smithi132 ceph-mon[70676]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:08:33.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:33 smithi132 ceph-mon[70676]: Health check failed: Degraded data redundancy: 13/333 objects degraded (3.904%), 4 pgs degraded (PG_DEGRADED) 2024-02-19T19:08:33.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:33 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2178639952' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:08:34.351 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:33 smithi088 podman[112611]: 2024-02-19 19:08:33.999730784 +0000 UTC m=+0.622203511 container remove 9fec5ce54ef36cdbd108e2ea543cda09a6f91cf5de145488249266c7c4116ecd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-activate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-19T19:08:34.351 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:34 smithi088 podman[112674]: 2024-02-19 19:08:34.115798436 +0000 UTC m=+0.018822569 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:08:34.351 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:34 smithi088 podman[112674]: 2024-02-19 19:08:34.225635801 +0000 UTC m=+0.128659958 container create f72e97fd38a53f34b5798540949b4aa015de64fab8ebb4585ae26ae17a9e5cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0) 2024-02-19T19:08:34.714 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:34 smithi088 podman[112674]: 2024-02-19 19:08:34.41119145 +0000 UTC m=+0.314215565 container init f72e97fd38a53f34b5798540949b4aa015de64fab8ebb4585ae26ae17a9e5cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:08:34.714 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:34 smithi088 podman[112674]: 2024-02-19 19:08:34.414505803 +0000 UTC m=+0.317529934 container start f72e97fd38a53f34b5798540949b4aa015de64fab8ebb4585ae26ae17a9e5cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T19:08:34.714 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:34 smithi088 ceph-osd[112692]: -- 172.21.15.88:0/319605283 <== mon.0 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563ee70471e0 con 0x563ee7018c00 2024-02-19T19:08:34.715 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:34 smithi088 bash[112674]: f72e97fd38a53f34b5798540949b4aa015de64fab8ebb4585ae26ae17a9e5cc6 2024-02-19T19:08:34.715 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:34 smithi088 systemd[1]: Started Ceph osd.2 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:08:35.351 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:34 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[112688]: 2024-02-19T19:08:34.958+0000 7f8d7e4e77c0 -1 Falling back to public interface 2024-02-19T19:08:35.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:35 smithi132 ceph-mon[70676]: pgmap v126: 65 pgs: 5 active+undersized+degraded, 7 peering, 8 active+undersized, 45 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17/333 objects degraded (5.105%) 2024-02-19T19:08:35.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:35.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:35.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:08:35.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:35 smithi088 ceph-mon[88643]: pgmap v126: 65 pgs: 5 active+undersized+degraded, 7 peering, 8 active+undersized, 45 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17/333 objects degraded (5.105%) 2024-02-19T19:08:35.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:35.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:35.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:08:36.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:36 smithi088 ceph-mon[88643]: pgmap v127: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-02-19T19:08:36.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:36 smithi088 ceph-mon[88643]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-19T19:08:37.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:36 smithi132 ceph-mon[70676]: pgmap v127: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-02-19T19:08:37.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:36 smithi132 ceph-mon[70676]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-19T19:08:39.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:39 smithi088 ceph-mon[88643]: pgmap v128: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-02-19T19:08:39.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:39 smithi132 ceph-mon[70676]: pgmap v128: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-02-19T19:08:39.846 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:39 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[112688]: 2024-02-19T19:08:39.400+0000 7f8d7e4e77c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-02-19T19:08:40.101 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:39 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[112688]: 2024-02-19T19:08:39.824+0000 7f8d7e4e77c0 -1 osd.2 62 log_to_monitors true 2024-02-19T19:08:41.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:40 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:41.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:40 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:41.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:40 smithi088 ceph-mon[88643]: from='osd.2 [v2:172.21.15.88:6818/2845929374,v1:172.21.15.88:6819/2845929374]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-19T19:08:41.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:40 smithi088 ceph-mon[88643]: Health check update: Degraded data redundancy: 41/333 objects degraded (12.312%), 12 pgs degraded (PG_DEGRADED) 2024-02-19T19:08:41.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:40 smithi088 ceph-mon[88643]: pgmap v129: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-02-19T19:08:41.102 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 19:08:40 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[112688]: 2024-02-19T19:08:40.794+0000 7f8d6f93f700 -1 osd.2 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T19:08:41.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:40 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:41.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:40 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:41.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:40 smithi132 ceph-mon[70676]: from='osd.2 [v2:172.21.15.88:6818/2845929374,v1:172.21.15.88:6819/2845929374]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-19T19:08:41.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:40 smithi132 ceph-mon[70676]: Health check update: Degraded data redundancy: 41/333 objects degraded (12.312%), 12 pgs degraded (PG_DEGRADED) 2024-02-19T19:08:41.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:40 smithi132 ceph-mon[70676]: pgmap v129: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-02-19T19:08:42.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:41 smithi088 ceph-mon[88643]: from='osd.2 [v2:172.21.15.88:6818/2845929374,v1:172.21.15.88:6819/2845929374]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-19T19:08:42.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:41 smithi088 ceph-mon[88643]: osdmap e65: 6 total, 5 up, 6 in 2024-02-19T19:08:42.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:41 smithi088 ceph-mon[88643]: from='osd.2 [v2:172.21.15.88:6818/2845929374,v1:172.21.15.88:6819/2845929374]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T19:08:42.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:41 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:42.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:41 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:42.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:41 smithi132 ceph-mon[70676]: from='osd.2 [v2:172.21.15.88:6818/2845929374,v1:172.21.15.88:6819/2845929374]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-19T19:08:42.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:41 smithi132 ceph-mon[70676]: osdmap e65: 6 total, 5 up, 6 in 2024-02-19T19:08:42.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:41 smithi132 ceph-mon[70676]: from='osd.2 [v2:172.21.15.88:6818/2845929374,v1:172.21.15.88:6819/2845929374]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-02-19T19:08:42.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:41 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:42.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:41 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:43.068 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:42 smithi088 ceph-mon[88643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:08:43.068 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:42 smithi088 ceph-mon[88643]: osd.2 [v2:172.21.15.88:6818/2845929374,v1:172.21.15.88:6819/2845929374] boot 2024-02-19T19:08:43.068 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:42 smithi088 ceph-mon[88643]: osdmap e66: 6 total, 6 up, 6 in 2024-02-19T19:08:43.068 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T19:08:43.068 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:42 smithi088 ceph-mon[88643]: pgmap v132: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-02-19T19:08:43.141 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:42 smithi132 ceph-mon[70676]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:08:43.141 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:42 smithi132 ceph-mon[70676]: osd.2 [v2:172.21.15.88:6818/2845929374,v1:172.21.15.88:6819/2845929374] boot 2024-02-19T19:08:43.142 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:42 smithi132 ceph-mon[70676]: osdmap e66: 6 total, 6 up, 6 in 2024-02-19T19:08:43.142 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-19T19:08:43.142 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:42 smithi132 ceph-mon[70676]: pgmap v132: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/333 objects degraded (12.312%) 2024-02-19T19:08:44.065 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:43 smithi088 ceph-mon[88643]: osdmap e67: 6 total, 6 up, 6 in 2024-02-19T19:08:44.065 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:44.065 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:08:44.253 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:43 smithi132 ceph-mon[70676]: osdmap e67: 6 total, 6 up, 6 in 2024-02-19T19:08:44.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:44.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:08:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: pgmap v134: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 37/333 objects degraded (11.111%) 2024-02-19T19:08:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:08:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:08:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:08:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:45.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:45.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:45.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-19T19:08:45.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:45 smithi132 ceph-mon[70676]: Health check update: Degraded data redundancy: 37/333 objects degraded (11.111%), 11 pgs degraded (PG_DEGRADED) 2024-02-19T19:08:45.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: pgmap v134: 65 pgs: 11 active+undersized+degraded, 13 active+undersized, 41 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 37/333 objects degraded (11.111%) 2024-02-19T19:08:45.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:45.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:45.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:08:45.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:08:45.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:45.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:08:45.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:45.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:45.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:08:45.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-19T19:08:45.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:45 smithi088 ceph-mon[88643]: Health check update: Degraded data redundancy: 37/333 objects degraded (11.111%), 11 pgs degraded (PG_DEGRADED) 2024-02-19T19:08:46.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:46 smithi132 ceph-mon[70676]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-19T19:08:46.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:46 smithi132 ceph-mon[70676]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-02-19T19:08:46.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:46 smithi088 ceph-mon[88643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-19T19:08:46.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:46 smithi088 ceph-mon[88643]: Upgrade: unsafe to stop osd(s) at this time (8 PGs are or would become offline) 2024-02-19T19:08:47.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:47 smithi132 ceph-mon[70676]: pgmap v135: 65 pgs: 2 active+undersized+degraded, 3 active+undersized, 60 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8/333 objects degraded (2.402%) 2024-02-19T19:08:47.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:47 smithi088 ceph-mon[88643]: pgmap v135: 65 pgs: 2 active+undersized+degraded, 3 active+undersized, 60 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 8/333 objects degraded (2.402%) 2024-02-19T19:08:48.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:48 smithi088 ceph-mon[88643]: pgmap v136: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:48.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:48 smithi088 ceph-mon[88643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/333 objects degraded (2.402%), 2 pgs degraded) 2024-02-19T19:08:48.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:48 smithi088 ceph-mon[88643]: Cluster is now healthy 2024-02-19T19:08:49.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:48 smithi132 ceph-mon[70676]: pgmap v136: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:49.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:48 smithi132 ceph-mon[70676]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 8/333 objects degraded (2.402%), 2 pgs degraded) 2024-02-19T19:08:49.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:48 smithi132 ceph-mon[70676]: Cluster is now healthy 2024-02-19T19:08:51.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:51 smithi132 ceph-mon[70676]: pgmap v137: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:51.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:51 smithi088 ceph-mon[88643]: pgmap v137: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:53.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:53 smithi132 ceph-mon[70676]: pgmap v138: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:53.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:53 smithi088 ceph-mon[88643]: pgmap v138: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:54.503 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:54.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:55.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:55 smithi088 ceph-mon[88643]: pgmap v139: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:55.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:55 smithi132 ceph-mon[70676]: pgmap v139: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:57.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:57 smithi088 ceph-mon[88643]: pgmap v140: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:57.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:57 smithi132 ceph-mon[70676]: pgmap v140: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:59.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:59 smithi132 ceph-mon[70676]: pgmap v141: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:59.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:59.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:08:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:08:59.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:59 smithi088 ceph-mon[88643]: pgmap v141: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:08:59.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:08:59.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:08:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:09:00.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-19T19:09:00.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-19T19:09:01.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:01 smithi132 ceph-mon[70676]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-19T19:09:01.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:01 smithi132 ceph-mon[70676]: Upgrade: osd.3 is safe to restart 2024-02-19T19:09:01.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:01 smithi132 ceph-mon[70676]: pgmap v142: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:01.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:01 smithi088 ceph-mon[88643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-02-19T19:09:01.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:01 smithi088 ceph-mon[88643]: Upgrade: osd.3 is safe to restart 2024-02-19T19:09:01.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:01 smithi088 ceph-mon[88643]: pgmap v142: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:02.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:02 smithi132 ceph-mon[70676]: Upgrade: Updating osd.3 2024-02-19T19:09:02.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:02.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-19T19:09:02.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:09:02.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:02 smithi132 ceph-mon[70676]: Deploying daemon osd.3 on smithi132 2024-02-19T19:09:02.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:02 smithi088 ceph-mon[88643]: Upgrade: Updating osd.3 2024-02-19T19:09:02.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:02.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-19T19:09:02.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:09:02.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:02 smithi088 ceph-mon[88643]: Deploying daemon osd.3 on smithi132 2024-02-19T19:09:02.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.886+0000 7fb2451fa700 1 -- 172.21.15.88:0/3152795222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2400ff3c0 msgr2=0x7fb2400ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:02.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.886+0000 7fb2451fa700 1 --2- 172.21.15.88:0/3152795222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2400ff3c0 0x7fb2400ff790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb230009910 tx=0x7fb230009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:02.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.886+0000 7fb2451fa700 1 -- 172.21.15.88:0/3152795222 shutdown_connections 2024-02-19T19:09:02.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.886+0000 7fb2451fa700 1 --2- 172.21.15.88:0/3152795222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2400ffd60 0x7fb240103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:02.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.886+0000 7fb2451fa700 1 --2- 172.21.15.88:0/3152795222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2400ff3c0 0x7fb2400ff790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:02.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.886+0000 7fb2451fa700 1 -- 172.21.15.88:0/3152795222 >> 172.21.15.88:0/3152795222 conn(0x7fb2400fac80 msgr2=0x7fb2400fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.887+0000 7fb2451fa700 1 -- 172.21.15.88:0/3152795222 shutdown_connections 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.887+0000 7fb2451fa700 1 -- 172.21.15.88:0/3152795222 wait complete. 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.887+0000 7fb2451fa700 1 Processor -- start 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb2451fa700 1 -- start start 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb2451fa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2400ff3c0 0x7fb240194e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb2451fa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2400ffd60 0x7fb240195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb2451fa700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb240195a60 con 0x7fb2400ff3c0 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb2451fa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2401997f0 con 0x7fb2400ffd60 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb23ed9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2400ff3c0 0x7fb240194e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb23ed9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2400ff3c0 0x7fb240194e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38340/0 (socket says 172.21.15.88:38340) 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb23ed9d700 1 -- 172.21.15.88:0/2197006054 learned_addr learned my addr 172.21.15.88:0/2197006054 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb23e59c700 1 --2- 172.21.15.88:0/2197006054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2400ffd60 0x7fb240195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:02.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb23ed9d700 1 -- 172.21.15.88:0/2197006054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2400ffd60 msgr2=0x7fb240195380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:02.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb23ed9d700 1 --2- 172.21.15.88:0/2197006054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2400ffd60 0x7fb240195380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:02.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb23ed9d700 1 -- 172.21.15.88:0/2197006054 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2300095f0 con 0x7fb2400ff3c0 2024-02-19T19:09:02.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.888+0000 7fb23ed9d700 1 --2- 172.21.15.88:0/2197006054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2400ff3c0 0x7fb240194e40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb2300092e0 tx=0x7fb23000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:02.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.889+0000 7fb227fff700 1 -- 172.21.15.88:0/2197006054 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb23001d070 con 0x7fb2400ff3c0 2024-02-19T19:09:02.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.889+0000 7fb2451fa700 1 -- 172.21.15.88:0/2197006054 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb240199990 con 0x7fb2400ff3c0 2024-02-19T19:09:02.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.889+0000 7fb227fff700 1 -- 172.21.15.88:0/2197006054 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb230005e00 con 0x7fb2400ff3c0 2024-02-19T19:09:02.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.889+0000 7fb227fff700 1 -- 172.21.15.88:0/2197006054 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb230017400 con 0x7fb2400ff3c0 2024-02-19T19:09:02.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.889+0000 7fb2451fa700 1 -- 172.21.15.88:0/2197006054 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb240199e80 con 0x7fb2400ff3c0 2024-02-19T19:09:02.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.890+0000 7fb2451fa700 1 -- 172.21.15.88:0/2197006054 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb24004f2a0 con 0x7fb2400ff3c0 2024-02-19T19:09:02.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.890+0000 7fb227fff700 1 -- 172.21.15.88:0/2197006054 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb23000f6f0 con 0x7fb2400ff3c0 2024-02-19T19:09:02.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.891+0000 7fb227fff700 1 --2- 172.21.15.88:0/2197006054 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb22806d8a0 0x7fb22806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:02.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.891+0000 7fb23e59c700 1 --2- 172.21.15.88:0/2197006054 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb22806d8a0 0x7fb22806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:02.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.891+0000 7fb227fff700 1 -- 172.21.15.88:0/2197006054 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb23008e900 con 0x7fb2400ff3c0 2024-02-19T19:09:02.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.891+0000 7fb23e59c700 1 --2- 172.21.15.88:0/2197006054 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb22806d8a0 0x7fb22806fd50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb240196460 tx=0x7fb23400b3c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:02.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:02.896+0000 7fb227fff700 1 -- 172.21.15.88:0/2197006054 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb230058160 con 0x7fb2400ff3c0 2024-02-19T19:09:03.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.111+0000 7fb2451fa700 1 -- 172.21.15.88:0/2197006054 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb24019a110 con 0x7fb22806d8a0 2024-02-19T19:09:03.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.113+0000 7fb227fff700 1 -- 172.21.15.88:0/2197006054 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb24019a110 con 0x7fb22806d8a0 2024-02-19T19:09:03.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.115+0000 7fb2451fa700 1 -- 172.21.15.88:0/2197006054 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb22806d8a0 msgr2=0x7fb22806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.115+0000 7fb2451fa700 1 --2- 172.21.15.88:0/2197006054 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb22806d8a0 0x7fb22806fd50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb240196460 tx=0x7fb23400b3c0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.115+0000 7fb2451fa700 1 -- 172.21.15.88:0/2197006054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2400ff3c0 msgr2=0x7fb240194e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.115+0000 7fb2451fa700 1 --2- 172.21.15.88:0/2197006054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2400ff3c0 0x7fb240194e40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb2300092e0 tx=0x7fb23000bd70 comp rx=0 tx=0).stop 2024-02-19T19:09:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.115+0000 7fb2451fa700 1 -- 172.21.15.88:0/2197006054 shutdown_connections 2024-02-19T19:09:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.115+0000 7fb2451fa700 1 --2- 172.21.15.88:0/2197006054 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb22806d8a0 0x7fb22806fd50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.115+0000 7fb2451fa700 1 --2- 172.21.15.88:0/2197006054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2400ff3c0 0x7fb240194e40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.115+0000 7fb2451fa700 1 --2- 172.21.15.88:0/2197006054 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2400ffd60 0x7fb240195380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.115+0000 7fb2451fa700 1 -- 172.21.15.88:0/2197006054 >> 172.21.15.88:0/2197006054 conn(0x7fb2400fac80 msgr2=0x7fb2400fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.115+0000 7fb2451fa700 1 -- 172.21.15.88:0/2197006054 shutdown_connections 2024-02-19T19:09:03.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.115+0000 7fb2451fa700 1 -- 172.21.15.88:0/2197006054 wait complete. 2024-02-19T19:09:03.147 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:09:03.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.220+0000 7fcb364fa700 1 -- 172.21.15.88:0/263501476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb301004e0 msgr2=0x7fcb301008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.220+0000 7fcb364fa700 1 --2- 172.21.15.88:0/263501476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb301004e0 0x7fcb301008b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009910 tx=0x7fcb20009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.220+0000 7fcb364fa700 1 -- 172.21.15.88:0/263501476 shutdown_connections 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.220+0000 7fcb364fa700 1 --2- 172.21.15.88:0/263501476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb30100e80 0x7fcb30104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.220+0000 7fcb364fa700 1 --2- 172.21.15.88:0/263501476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb301004e0 0x7fcb301008b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.220+0000 7fcb364fa700 1 -- 172.21.15.88:0/263501476 >> 172.21.15.88:0/263501476 conn(0x7fcb300fbda0 msgr2=0x7fcb300fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.220+0000 7fcb364fa700 1 -- 172.21.15.88:0/263501476 shutdown_connections 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.220+0000 7fcb364fa700 1 -- 172.21.15.88:0/263501476 wait complete. 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.220+0000 7fcb364fa700 1 Processor -- start 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.220+0000 7fcb364fa700 1 -- start start 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb364fa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb301004e0 0x7fcb30073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb364fa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb30100e80 0x7fcb30073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb364fa700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30073ad0 con 0x7fcb30100e80 2024-02-19T19:09:03.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb364fa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30073c10 con 0x7fcb301004e0 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb2f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb30100e80 0x7fcb30073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb2f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb30100e80 0x7fcb30073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38360/0 (socket says 172.21.15.88:38360) 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb2f7fe700 1 -- 172.21.15.88:0/344963564 learned_addr learned my addr 172.21.15.88:0/344963564 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb2ffff700 1 --2- 172.21.15.88:0/344963564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb301004e0 0x7fcb30073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb2f7fe700 1 -- 172.21.15.88:0/344963564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb301004e0 msgr2=0x7fcb30073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb2f7fe700 1 --2- 172.21.15.88:0/344963564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb301004e0 0x7fcb30073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb2f7fe700 1 -- 172.21.15.88:0/344963564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb200095f0 con 0x7fcb30100e80 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.221+0000 7fcb2f7fe700 1 --2- 172.21.15.88:0/344963564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb30100e80 0x7fcb30073590 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b480 tx=0x7fcb2400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.222+0000 7fcb2d7fa700 1 -- 172.21.15.88:0/344963564 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb24011800 con 0x7fcb30100e80 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.222+0000 7fcb2d7fa700 1 -- 172.21.15.88:0/344963564 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb24011e20 con 0x7fcb30100e80 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.222+0000 7fcb364fa700 1 -- 172.21.15.88:0/344963564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb301a2160 con 0x7fcb30100e80 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.222+0000 7fcb2d7fa700 1 -- 172.21.15.88:0/344963564 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb24010ea0 con 0x7fcb30100e80 2024-02-19T19:09:03.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.222+0000 7fcb364fa700 1 -- 172.21.15.88:0/344963564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb301a2570 con 0x7fcb30100e80 2024-02-19T19:09:03.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.223+0000 7fcb364fa700 1 -- 172.21.15.88:0/344963564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb301087f0 con 0x7fcb30100e80 2024-02-19T19:09:03.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.223+0000 7fcb2d7fa700 1 -- 172.21.15.88:0/344963564 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcb2401a480 con 0x7fcb30100e80 2024-02-19T19:09:03.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.224+0000 7fcb2d7fa700 1 --2- 172.21.15.88:0/344963564 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb1806d8a0 0x7fcb1806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:03.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.224+0000 7fcb2d7fa700 1 -- 172.21.15.88:0/344963564 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fcb2408cf20 con 0x7fcb30100e80 2024-02-19T19:09:03.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.224+0000 7fcb2ffff700 1 --2- 172.21.15.88:0/344963564 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb1806d8a0 0x7fcb1806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:03.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.224+0000 7fcb2ffff700 1 --2- 172.21.15.88:0/344963564 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb1806d8a0 0x7fcb1806fd50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcb20006010 tx=0x7fcb2001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.228+0000 7fcb2d7fa700 1 -- 172.21.15.88:0/344963564 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcb240c0d20 con 0x7fcb30100e80 2024-02-19T19:09:03.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.443+0000 7fcb364fa700 1 -- 172.21.15.88:0/344963564 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb30061190 con 0x7fcb1806d8a0 2024-02-19T19:09:03.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.444+0000 7fcb2d7fa700 1 -- 172.21.15.88:0/344963564 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fcb30061190 con 0x7fcb1806d8a0 2024-02-19T19:09:03.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.446+0000 7fcb364fa700 1 -- 172.21.15.88:0/344963564 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb1806d8a0 msgr2=0x7fcb1806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.446+0000 7fcb364fa700 1 --2- 172.21.15.88:0/344963564 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb1806d8a0 0x7fcb1806fd50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcb20006010 tx=0x7fcb2001a040 comp rx=0 tx=0).stop 2024-02-19T19:09:03.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.446+0000 7fcb364fa700 1 -- 172.21.15.88:0/344963564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb30100e80 msgr2=0x7fcb30073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.446+0000 7fcb364fa700 1 --2- 172.21.15.88:0/344963564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb30100e80 0x7fcb30073590 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b480 tx=0x7fcb2400b790 comp rx=0 tx=0).stop 2024-02-19T19:09:03.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.447+0000 7fcb364fa700 1 -- 172.21.15.88:0/344963564 shutdown_connections 2024-02-19T19:09:03.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.447+0000 7fcb364fa700 1 --2- 172.21.15.88:0/344963564 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb1806d8a0 0x7fcb1806fd50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.447+0000 7fcb364fa700 1 --2- 172.21.15.88:0/344963564 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb301004e0 0x7fcb30073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.447+0000 7fcb364fa700 1 --2- 172.21.15.88:0/344963564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb30100e80 0x7fcb30073590 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.447+0000 7fcb364fa700 1 -- 172.21.15.88:0/344963564 >> 172.21.15.88:0/344963564 conn(0x7fcb300fbda0 msgr2=0x7fcb300fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:03.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.447+0000 7fcb364fa700 1 -- 172.21.15.88:0/344963564 shutdown_connections 2024-02-19T19:09:03.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.447+0000 7fcb364fa700 1 -- 172.21.15.88:0/344963564 wait complete. 2024-02-19T19:09:03.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:03 smithi132 ceph-mon[70676]: pgmap v143: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:03.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.553+0000 7f6809bd1700 1 -- 172.21.15.88:0/3130759246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6804101420 msgr2=0x7f68041017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.553+0000 7f6809bd1700 1 --2- 172.21.15.88:0/3130759246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6804101420 0x7f68041017f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67fc009910 tx=0x7f67fc009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:03.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.553+0000 7f6809bd1700 1 -- 172.21.15.88:0/3130759246 shutdown_connections 2024-02-19T19:09:03.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.553+0000 7f6809bd1700 1 --2- 172.21.15.88:0/3130759246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6804101d30 0x7f680410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.553+0000 7f6809bd1700 1 --2- 172.21.15.88:0/3130759246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6804101420 0x7f68041017f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.553+0000 7f6809bd1700 1 -- 172.21.15.88:0/3130759246 >> 172.21.15.88:0/3130759246 conn(0x7f68040fac80 msgr2=0x7f68040fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:03.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:03 smithi088 ceph-mon[88643]: pgmap v143: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:03.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.553+0000 7f6809bd1700 1 -- 172.21.15.88:0/3130759246 shutdown_connections 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.553+0000 7f6809bd1700 1 -- 172.21.15.88:0/3130759246 wait complete. 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.554+0000 7f6809bd1700 1 Processor -- start 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.554+0000 7f6809bd1700 1 -- start start 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.554+0000 7f6809bd1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6804101420 0x7f6804195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.554+0000 7f6809bd1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6804101d30 0x7f68041963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.554+0000 7f6809bd1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6804196a90 con 0x7f6804101d30 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.554+0000 7f6809bd1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680419a820 con 0x7f6804101420 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.554+0000 7f6802ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6804101d30 0x7f68041963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.554+0000 7f68037fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6804101420 0x7f6804195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.554+0000 7f6802ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6804101d30 0x7f68041963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38372/0 (socket says 172.21.15.88:38372) 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.554+0000 7f6802ffd700 1 -- 172.21.15.88:0/976738834 learned_addr learned my addr 172.21.15.88:0/976738834 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.555+0000 7f6802ffd700 1 -- 172.21.15.88:0/976738834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6804101420 msgr2=0x7f6804195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.555+0000 7f6802ffd700 1 --2- 172.21.15.88:0/976738834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6804101420 0x7f6804195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.555+0000 7f6802ffd700 1 -- 172.21.15.88:0/976738834 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc0095f0 con 0x7f6804101d30 2024-02-19T19:09:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.555+0000 7f6802ffd700 1 --2- 172.21.15.88:0/976738834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6804101d30 0x7f68041963b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00d8d0 tx=0x7f67ec00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.555+0000 7f6800ff9700 1 -- 172.21.15.88:0/976738834 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67ec0098a0 con 0x7f6804101d30 2024-02-19T19:09:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.555+0000 7f6800ff9700 1 -- 172.21.15.88:0/976738834 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67ec010460 con 0x7f6804101d30 2024-02-19T19:09:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.555+0000 7f6800ff9700 1 -- 172.21.15.88:0/976738834 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67ec00f5f0 con 0x7f6804101d30 2024-02-19T19:09:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.555+0000 7f6809bd1700 1 -- 172.21.15.88:0/976738834 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680419ab00 con 0x7f6804101d30 2024-02-19T19:09:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.555+0000 7f6809bd1700 1 -- 172.21.15.88:0/976738834 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680419b050 con 0x7f6804101d30 2024-02-19T19:09:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.556+0000 7f6809bd1700 1 -- 172.21.15.88:0/976738834 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6804107900 con 0x7f6804101d30 2024-02-19T19:09:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.557+0000 7f6800ff9700 1 -- 172.21.15.88:0/976738834 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f67ec009a00 con 0x7f6804101d30 2024-02-19T19:09:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.557+0000 7f6800ff9700 1 --2- 172.21.15.88:0/976738834 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67f006d8a0 0x7f67f006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.557+0000 7f6800ff9700 1 -- 172.21.15.88:0/976738834 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f67ec08d2b0 con 0x7f6804101d30 2024-02-19T19:09:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.557+0000 7f68037fe700 1 --2- 172.21.15.88:0/976738834 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67f006d8a0 0x7f67f006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.558+0000 7f68037fe700 1 --2- 172.21.15.88:0/976738834 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67f006d8a0 0x7f67f006fd50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006010 tx=0x7f67fc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.562+0000 7f6800ff9700 1 -- 172.21.15.88:0/976738834 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f67ec056b90 con 0x7f6804101d30 2024-02-19T19:09:03.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.770+0000 7f6809bd1700 1 -- 172.21.15.88:0/976738834 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f680419b330 con 0x7f67f006d8a0 2024-02-19T19:09:03.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.780+0000 7f6800ff9700 1 -- 172.21.15.88:0/976738834 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f680419b330 con 0x7f67f006d8a0 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (6m) 24s ago 16m 24.2M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (17m) 24s ago 17m 8674k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (15m) 2m ago 15m 8010k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (2m) 24s ago 17m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (2m) 2m ago 15m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (5m) 24s ago 16m 85.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (10m) 24s ago 10m 17.0M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (10m) 24s ago 10m 114M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (10m) 2m ago 10m 139M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (10m) 2m ago 10m 20.2M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (7m) 24s ago 18m 501M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (7m) 2m ago 14m 450M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:09:03.802 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (4m) 24s ago 18m 50.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:09:03.803 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (3m) 2m ago 14m 37.6M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:09:03.803 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (6m) 24s ago 16m 15.4M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:09:03.803 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (6m) 2m ago 15m 15.8M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:09:03.803 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (104s) 24s ago 14m 127M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:09:03.803 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (66s) 24s ago 13m 99.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:09:03.803 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (29s) 24s ago 13m 12.5M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:09:03.803 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (12m) 2m ago 12m 236M 2240M 18.2.0 dc2bc1663786 3a3ade48fb36 2024-02-19T19:09:03.803 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (12m) 2m ago 12m 215M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:09:03.803 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (11m) 2m ago 11m 201M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:09:03.803 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (6m) 24s ago 15m 69.2M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:09:03.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.782+0000 7f6809bd1700 1 -- 172.21.15.88:0/976738834 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67f006d8a0 msgr2=0x7f67f006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.782+0000 7f6809bd1700 1 --2- 172.21.15.88:0/976738834 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67f006d8a0 0x7f67f006fd50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006010 tx=0x7f67fc01a040 comp rx=0 tx=0).stop 2024-02-19T19:09:03.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.782+0000 7f6809bd1700 1 -- 172.21.15.88:0/976738834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6804101d30 msgr2=0x7f68041963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.782+0000 7f6809bd1700 1 --2- 172.21.15.88:0/976738834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6804101d30 0x7f68041963b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00d8d0 tx=0x7f67ec00dbe0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.782+0000 7f6809bd1700 1 -- 172.21.15.88:0/976738834 shutdown_connections 2024-02-19T19:09:03.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.782+0000 7f6809bd1700 1 --2- 172.21.15.88:0/976738834 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67f006d8a0 0x7f67f006fd50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.782+0000 7f6809bd1700 1 --2- 172.21.15.88:0/976738834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6804101420 0x7f6804195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.782+0000 7f6809bd1700 1 --2- 172.21.15.88:0/976738834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6804101d30 0x7f68041963b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.782+0000 7f6809bd1700 1 -- 172.21.15.88:0/976738834 >> 172.21.15.88:0/976738834 conn(0x7f68040fac80 msgr2=0x7f6804104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:03.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.783+0000 7f6809bd1700 1 -- 172.21.15.88:0/976738834 shutdown_connections 2024-02-19T19:09:03.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.783+0000 7f6809bd1700 1 -- 172.21.15.88:0/976738834 wait complete. 2024-02-19T19:09:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.886+0000 7ff636aed700 1 -- 172.21.15.88:0/3447780815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6300ffbd0 msgr2=0x7ff630100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.886+0000 7ff636aed700 1 --2- 172.21.15.88:0/3447780815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6300ffbd0 0x7ff630100040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff620009910 tx=0x7ff620009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.886+0000 7ff636aed700 1 -- 172.21.15.88:0/3447780815 shutdown_connections 2024-02-19T19:09:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.886+0000 7ff636aed700 1 --2- 172.21.15.88:0/3447780815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6300ffbd0 0x7ff630100040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.886+0000 7ff636aed700 1 --2- 172.21.15.88:0/3447780815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff630104130 0x7ff630104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.886+0000 7ff636aed700 1 -- 172.21.15.88:0/3447780815 >> 172.21.15.88:0/3447780815 conn(0x7ff630075130 msgr2=0x7ff630075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.886+0000 7ff636aed700 1 -- 172.21.15.88:0/3447780815 shutdown_connections 2024-02-19T19:09:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.886+0000 7ff636aed700 1 -- 172.21.15.88:0/3447780815 wait complete. 2024-02-19T19:09:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.886+0000 7ff636aed700 1 Processor -- start 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff636aed700 1 -- start start 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff636aed700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6300ffbd0 0x7ff630196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff636aed700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff630104130 0x7ff6301971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff636aed700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff630197800 con 0x7ff6300ffbd0 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff636aed700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff630197940 con 0x7ff630104130 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff634889700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6300ffbd0 0x7ff630196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff634889700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6300ffbd0 0x7ff630196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38384/0 (socket says 172.21.15.88:38384) 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff634889700 1 -- 172.21.15.88:0/4035461813 learned_addr learned my addr 172.21.15.88:0/4035461813 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff62ffff700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff630104130 0x7ff6301971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff634889700 1 -- 172.21.15.88:0/4035461813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff630104130 msgr2=0x7ff6301971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff634889700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff630104130 0x7ff6301971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff634889700 1 -- 172.21.15.88:0/4035461813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6200095f0 con 0x7ff6300ffbd0 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff62ffff700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff630104130 0x7ff6301971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:09:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff634889700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6300ffbd0 0x7ff630196ca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff62800ef10 tx=0x7ff62800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:03.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff62dffb700 1 -- 172.21.15.88:0/4035461813 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff62800ca50 con 0x7ff6300ffbd0 2024-02-19T19:09:03.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff636aed700 1 -- 172.21.15.88:0/4035461813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63019fd10 con 0x7ff6300ffbd0 2024-02-19T19:09:03.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff62dffb700 1 -- 172.21.15.88:0/4035461813 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff62800f040 con 0x7ff6300ffbd0 2024-02-19T19:09:03.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.887+0000 7ff62dffb700 1 -- 172.21.15.88:0/4035461813 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6280146b0 con 0x7ff6300ffbd0 2024-02-19T19:09:03.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.888+0000 7ff636aed700 1 -- 172.21.15.88:0/4035461813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6301a0180 con 0x7ff6300ffbd0 2024-02-19T19:09:03.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.889+0000 7ff636aed700 1 -- 172.21.15.88:0/4035461813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63004f2a0 con 0x7ff6300ffbd0 2024-02-19T19:09:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.889+0000 7ff62dffb700 1 -- 172.21.15.88:0/4035461813 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff62800cbb0 con 0x7ff6300ffbd0 2024-02-19T19:09:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.889+0000 7ff62dffb700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff61c06d8a0 0x7ff61c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.890+0000 7ff62ffff700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff61c06d8a0 0x7ff61c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.890+0000 7ff62dffb700 1 -- 172.21.15.88:0/4035461813 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff62808d860 con 0x7ff6300ffbd0 2024-02-19T19:09:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.890+0000 7ff62ffff700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff61c06d8a0 0x7ff61c06fd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff630068df0 tx=0x7ff620009f70 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:03.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:03.894+0000 7ff62dffb700 1 -- 172.21.15.88:0/4035461813 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff6280570c0 con 0x7ff6300ffbd0 2024-02-19T19:09:04.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.175+0000 7ff636aed700 1 -- 172.21.15.88:0/4035461813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff6301a0470 con 0x7ff6300ffbd0 2024-02-19T19:09:04.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.176+0000 7ff62dffb700 1 -- 172.21.15.88:0/4035461813 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7ff62805a780 con 0x7ff6300ffbd0 2024-02-19T19:09:04.197 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:09:04.197 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:09:04.197 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:09:04.197 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:09:04.197 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:09:04.197 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:09:04.197 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:09:04.197 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:09:04.198 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-02-19T19:09:04.198 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:09:04.198 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:09:04.198 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:09:04.198 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:09:04.198 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:09:04.198 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:09:04.198 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 7, 2024-02-19T19:09:04.198 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 7 2024-02-19T19:09:04.198 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:09:04.198 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:09:04.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.178+0000 7ff636aed700 1 -- 172.21.15.88:0/4035461813 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff61c06d8a0 msgr2=0x7ff61c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:04.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.178+0000 7ff636aed700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff61c06d8a0 0x7ff61c06fd50 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff630068df0 tx=0x7ff620009f70 comp rx=0 tx=0).stop 2024-02-19T19:09:04.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.178+0000 7ff636aed700 1 -- 172.21.15.88:0/4035461813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6300ffbd0 msgr2=0x7ff630196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:04.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.178+0000 7ff636aed700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6300ffbd0 0x7ff630196ca0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff62800ef10 tx=0x7ff62800c3a0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.178+0000 7ff636aed700 1 -- 172.21.15.88:0/4035461813 shutdown_connections 2024-02-19T19:09:04.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.178+0000 7ff636aed700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff61c06d8a0 0x7ff61c06fd50 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.178+0000 7ff636aed700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6300ffbd0 0x7ff630196ca0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.178+0000 7ff636aed700 1 --2- 172.21.15.88:0/4035461813 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff630104130 0x7ff6301971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.178+0000 7ff636aed700 1 -- 172.21.15.88:0/4035461813 >> 172.21.15.88:0/4035461813 conn(0x7ff630075130 msgr2=0x7ff6300feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:04.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.178+0000 7ff636aed700 1 -- 172.21.15.88:0/4035461813 shutdown_connections 2024-02-19T19:09:04.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.178+0000 7ff636aed700 1 -- 172.21.15.88:0/4035461813 wait complete. 2024-02-19T19:09:04.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.286+0000 7fc844a77700 1 -- 172.21.15.88:0/3924454621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc840100530 msgr2=0x7fc840100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.286+0000 7fc844a77700 1 --2- 172.21.15.88:0/3924454621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc840100530 0x7fc840100900 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc830009910 tx=0x7fc830009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.286+0000 7fc844a77700 1 -- 172.21.15.88:0/3924454621 shutdown_connections 2024-02-19T19:09:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.286+0000 7fc844a77700 1 --2- 172.21.15.88:0/3924454621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc840100ed0 0x7fc840104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.286+0000 7fc844a77700 1 --2- 172.21.15.88:0/3924454621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc840100530 0x7fc840100900 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.286+0000 7fc844a77700 1 -- 172.21.15.88:0/3924454621 >> 172.21.15.88:0/3924454621 conn(0x7fc8400fbe10 msgr2=0x7fc8400fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.287+0000 7fc844a77700 1 -- 172.21.15.88:0/3924454621 shutdown_connections 2024-02-19T19:09:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.287+0000 7fc844a77700 1 -- 172.21.15.88:0/3924454621 wait complete. 2024-02-19T19:09:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.287+0000 7fc844a77700 1 Processor -- start 2024-02-19T19:09:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.287+0000 7fc844a77700 1 -- start start 2024-02-19T19:09:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.287+0000 7fc844a77700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc840100530 0x7fc840195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.287+0000 7fc844a77700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc840100ed0 0x7fc840196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.287+0000 7fc844a77700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc840196a50 con 0x7fc840100ed0 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.287+0000 7fc844a77700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84019a7e0 con 0x7fc840100530 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc83dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc840100ed0 0x7fc840196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc83dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc840100ed0 0x7fc840196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38410/0 (socket says 172.21.15.88:38410) 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc83dd9b700 1 -- 172.21.15.88:0/634902330 learned_addr learned my addr 172.21.15.88:0/634902330 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc83e59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc840100530 0x7fc840195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc83dd9b700 1 -- 172.21.15.88:0/634902330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc840100530 msgr2=0x7fc840195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc83dd9b700 1 --2- 172.21.15.88:0/634902330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc840100530 0x7fc840195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc83dd9b700 1 -- 172.21.15.88:0/634902330 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8300095f0 con 0x7fc840100ed0 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc83dd9b700 1 --2- 172.21.15.88:0/634902330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc840100ed0 0x7fc840196370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc83400d900 tx=0x7fc83400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc82f7fe700 1 -- 172.21.15.88:0/634902330 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc834009900 con 0x7fc840100ed0 2024-02-19T19:09:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc844a77700 1 -- 172.21.15.88:0/634902330 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84019aac0 con 0x7fc840100ed0 2024-02-19T19:09:04.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc82f7fe700 1 -- 172.21.15.88:0/634902330 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc834010460 con 0x7fc840100ed0 2024-02-19T19:09:04.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc82f7fe700 1 -- 172.21.15.88:0/634902330 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc83400f5f0 con 0x7fc840100ed0 2024-02-19T19:09:04.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.288+0000 7fc844a77700 1 -- 172.21.15.88:0/634902330 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84019b010 con 0x7fc840100ed0 2024-02-19T19:09:04.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.289+0000 7fc844a77700 1 -- 172.21.15.88:0/634902330 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc840108840 con 0x7fc840100ed0 2024-02-19T19:09:04.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.290+0000 7fc82f7fe700 1 -- 172.21.15.88:0/634902330 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc8340105d0 con 0x7fc840100ed0 2024-02-19T19:09:04.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.290+0000 7fc82f7fe700 1 --2- 172.21.15.88:0/634902330 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc82806d8a0 0x7fc82806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:04.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.290+0000 7fc82f7fe700 1 -- 172.21.15.88:0/634902330 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc83408d030 con 0x7fc840100ed0 2024-02-19T19:09:04.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.290+0000 7fc83e59c700 1 --2- 172.21.15.88:0/634902330 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc82806d8a0 0x7fc82806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:04.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.291+0000 7fc83e59c700 1 --2- 172.21.15.88:0/634902330 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc82806d8a0 0x7fc82806fd50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc830006010 tx=0x7fc83001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:04.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.295+0000 7fc82f7fe700 1 -- 172.21.15.88:0/634902330 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc834056890 con 0x7fc840100ed0 2024-02-19T19:09:04.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:04 smithi132 ceph-mon[70676]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:04.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:04 smithi132 ceph-mon[70676]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:04.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:04 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4035461813' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:04.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.538+0000 7fc844a77700 1 -- 172.21.15.88:0/634902330 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc84019b230 con 0x7fc840100ed0 2024-02-19T19:09:04.560 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:04 smithi088 ceph-mon[88643]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:04.560 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:04 smithi088 ceph-mon[88643]: from='client.34336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:04.560 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:04 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4035461813' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:04.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.539+0000 7fc82f7fe700 1 -- 172.21.15.88:0/634902330 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1892 (secure 0 0 0) 0x7fc834059f50 con 0x7fc840100ed0 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:e20 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 20 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:05:14.913778+0000 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:09:04.561 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T19:09:04.562 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:09:04.563 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:09:04.564 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:09:04.564 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.542+0000 7fc844a77700 1 -- 172.21.15.88:0/634902330 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc82806d8a0 msgr2=0x7fc82806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.542+0000 7fc844a77700 1 --2- 172.21.15.88:0/634902330 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc82806d8a0 0x7fc82806fd50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc830006010 tx=0x7fc83001a040 comp rx=0 tx=0).stop 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.542+0000 7fc844a77700 1 -- 172.21.15.88:0/634902330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc840100ed0 msgr2=0x7fc840196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.542+0000 7fc844a77700 1 --2- 172.21.15.88:0/634902330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc840100ed0 0x7fc840196370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc83400d900 tx=0x7fc83400dc10 comp rx=0 tx=0).stop 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.542+0000 7fc844a77700 1 -- 172.21.15.88:0/634902330 shutdown_connections 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.542+0000 7fc844a77700 1 --2- 172.21.15.88:0/634902330 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc82806d8a0 0x7fc82806fd50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.542+0000 7fc844a77700 1 --2- 172.21.15.88:0/634902330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc840100530 0x7fc840195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.542+0000 7fc844a77700 1 --2- 172.21.15.88:0/634902330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc840100ed0 0x7fc840196370 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.542+0000 7fc844a77700 1 -- 172.21.15.88:0/634902330 >> 172.21.15.88:0/634902330 conn(0x7fc8400fbe10 msgr2=0x7fc8400fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.542+0000 7fc844a77700 1 -- 172.21.15.88:0/634902330 shutdown_connections 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.542+0000 7fc844a77700 1 -- 172.21.15.88:0/634902330 wait complete. 2024-02-19T19:09:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-02-19T19:09:04.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.647+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2552058715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63cc0ff350 msgr2=0x7f63cc0ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:04.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.647+0000 7f63d3f3e700 1 --2- 172.21.15.88:0/2552058715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63cc0ff350 0x7f63cc0ff720 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009910 tx=0x7f63c8009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:04.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2552058715 shutdown_connections 2024-02-19T19:09:04.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 --2- 172.21.15.88:0/2552058715 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63cc0ffc60 0x7f63cc10a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 --2- 172.21.15.88:0/2552058715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63cc0ff350 0x7f63cc0ff720 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2552058715 >> 172.21.15.88:0/2552058715 conn(0x7f63cc0748b0 msgr2=0x7f63cc074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:04.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2552058715 shutdown_connections 2024-02-19T19:09:04.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2552058715 wait complete. 2024-02-19T19:09:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 Processor -- start 2024-02-19T19:09:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 -- start start 2024-02-19T19:09:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63cc0ff350 0x7f63cc1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63cc0ffc60 0x7f63cc198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc198cf0 con 0x7f63cc0ff350 2024-02-19T19:09:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.648+0000 7f63d3f3e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc19ca80 con 0x7f63cc0ffc60 2024-02-19T19:09:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63d1cda700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63cc0ff350 0x7f63cc1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63d1cda700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63cc0ff350 0x7f63cc1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38424/0 (socket says 172.21.15.88:38424) 2024-02-19T19:09:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63d1cda700 1 -- 172.21.15.88:0/2211356183 learned_addr learned my addr 172.21.15.88:0/2211356183 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63d14d9700 1 --2- 172.21.15.88:0/2211356183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63cc0ffc60 0x7f63cc198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63d1cda700 1 -- 172.21.15.88:0/2211356183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63cc0ffc60 msgr2=0x7f63cc198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63d1cda700 1 --2- 172.21.15.88:0/2211356183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63cc0ffc60 0x7f63cc198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63d1cda700 1 -- 172.21.15.88:0/2211356183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c80095f0 con 0x7f63cc0ff350 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63d1cda700 1 --2- 172.21.15.88:0/2211356183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63cc0ff350 0x7f63cc1980d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63c80092e0 tx=0x7f63c800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63c2ffd700 1 -- 172.21.15.88:0/2211356183 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63c801d070 con 0x7f63cc0ff350 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63c2ffd700 1 -- 172.21.15.88:0/2211356183 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f63c8005dc0 con 0x7f63cc0ff350 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2211356183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc19cd00 con 0x7f63cc0ff350 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63c2ffd700 1 -- 172.21.15.88:0/2211356183 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63c8017400 con 0x7f63cc0ff350 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.649+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2211356183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc19d1f0 con 0x7f63cc0ff350 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.650+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2211356183 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63cc10a1d0 con 0x7f63cc0ff350 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.651+0000 7f63c2ffd700 1 -- 172.21.15.88:0/2211356183 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f63c800f6f0 con 0x7f63cc0ff350 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.651+0000 7f63c2ffd700 1 --2- 172.21.15.88:0/2211356183 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f63b806d8a0 0x7f63b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.651+0000 7f63c2ffd700 1 -- 172.21.15.88:0/2211356183 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f63c808e790 con 0x7f63cc0ff350 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.651+0000 7f63d14d9700 1 --2- 172.21.15.88:0/2211356183 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f63b806d8a0 0x7f63b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:04.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.651+0000 7f63d14d9700 1 --2- 172.21.15.88:0/2211356183 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f63b806d8a0 0x7f63b806fd50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f63cc1996f0 tx=0x7f63bc00b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:04.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.656+0000 7f63c2ffd700 1 -- 172.21.15.88:0/2211356183 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f63c8058070 con 0x7f63cc0ff350 2024-02-19T19:09:04.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.874+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2211356183 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63cc02cca0 con 0x7f63b806d8a0 2024-02-19T19:09:04.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.875+0000 7f63c2ffd700 1 -- 172.21.15.88:0/2211356183 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f63cc02cca0 con 0x7f63b806d8a0 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "9/23 daemons upgraded", 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading osd daemons", 2024-02-19T19:09:04.897 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:09:04.898 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:09:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.878+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2211356183 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f63b806d8a0 msgr2=0x7f63b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.878+0000 7f63d3f3e700 1 --2- 172.21.15.88:0/2211356183 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f63b806d8a0 0x7f63b806fd50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f63cc1996f0 tx=0x7f63bc00b410 comp rx=0 tx=0).stop 2024-02-19T19:09:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.878+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2211356183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63cc0ff350 msgr2=0x7f63cc1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.878+0000 7f63d3f3e700 1 --2- 172.21.15.88:0/2211356183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63cc0ff350 0x7f63cc1980d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f63c80092e0 tx=0x7f63c800bd70 comp rx=0 tx=0).stop 2024-02-19T19:09:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.878+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2211356183 shutdown_connections 2024-02-19T19:09:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.878+0000 7f63d3f3e700 1 --2- 172.21.15.88:0/2211356183 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f63b806d8a0 0x7f63b806fd50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.878+0000 7f63d3f3e700 1 --2- 172.21.15.88:0/2211356183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63cc0ff350 0x7f63cc1980d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.878+0000 7f63d3f3e700 1 --2- 172.21.15.88:0/2211356183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f63cc0ffc60 0x7f63cc198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:04.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.878+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2211356183 >> 172.21.15.88:0/2211356183 conn(0x7f63cc0748b0 msgr2=0x7f63cc107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:04.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.878+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2211356183 shutdown_connections 2024-02-19T19:09:04.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.878+0000 7f63d3f3e700 1 -- 172.21.15.88:0/2211356183 wait complete. 2024-02-19T19:09:05.004 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:04 smithi132 systemd[1]: Stopping Ceph osd.3 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:09:05.004 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:04 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[43878]: 2024-02-19T19:09:04.835+0000 7f1083f42700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T19:09:05.004 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:04 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[43878]: 2024-02-19T19:09:04.835+0000 7f1083f42700 -1 osd.3 67 *** Got signal Terminated *** 2024-02-19T19:09:05.004 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:04 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[43878]: 2024-02-19T19:09:04.835+0000 7f1083f42700 -1 osd.3 67 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T19:09:05.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.982+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3504591899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c540ff350 msgr2=0x7f0c540ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.982+0000 7f0c5bca7700 1 --2- 172.21.15.88:0/3504591899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c540ff350 0x7f0c540ff720 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009910 tx=0x7f0c50009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.982+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3504591899 shutdown_connections 2024-02-19T19:09:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.982+0000 7f0c5bca7700 1 --2- 172.21.15.88:0/3504591899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c540ffc60 0x7f0c5410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.982+0000 7f0c5bca7700 1 --2- 172.21.15.88:0/3504591899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c540ff350 0x7f0c540ff720 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.982+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3504591899 >> 172.21.15.88:0/3504591899 conn(0x7f0c540748b0 msgr2=0x7f0c54074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3504591899 shutdown_connections 2024-02-19T19:09:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3504591899 wait complete. 2024-02-19T19:09:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c5bca7700 1 Processor -- start 2024-02-19T19:09:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c5bca7700 1 -- start start 2024-02-19T19:09:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c5bca7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c540ff350 0x7f0c54198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:05.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c5bca7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c540ffc60 0x7f0c541985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c5bca7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c54198cb0 con 0x7f0c540ff350 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c5bca7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5419ca40 con 0x7f0c540ffc60 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c59a43700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c540ff350 0x7f0c54198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c59a43700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c540ff350 0x7f0c54198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38444/0 (socket says 172.21.15.88:38444) 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c59a43700 1 -- 172.21.15.88:0/3590417673 learned_addr learned my addr 172.21.15.88:0/3590417673 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.983+0000 7f0c59242700 1 --2- 172.21.15.88:0/3590417673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c540ffc60 0x7f0c541985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.984+0000 7f0c59a43700 1 -- 172.21.15.88:0/3590417673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c540ffc60 msgr2=0x7f0c541985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.984+0000 7f0c59a43700 1 --2- 172.21.15.88:0/3590417673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c540ffc60 0x7f0c541985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.984+0000 7f0c59a43700 1 -- 172.21.15.88:0/3590417673 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c500095f0 con 0x7f0c540ff350 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.984+0000 7f0c59a43700 1 --2- 172.21.15.88:0/3590417673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c540ff350 0x7f0c54198090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c500092e0 tx=0x7f0c5000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.984+0000 7f0c4affd700 1 -- 172.21.15.88:0/3590417673 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c5001d070 con 0x7f0c540ff350 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.984+0000 7f0c4affd700 1 -- 172.21.15.88:0/3590417673 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c50005dc0 con 0x7f0c540ff350 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.984+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3590417673 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c5419ccc0 con 0x7f0c540ff350 2024-02-19T19:09:05.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.984+0000 7f0c4affd700 1 -- 172.21.15.88:0/3590417673 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c50017400 con 0x7f0c540ff350 2024-02-19T19:09:05.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.984+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3590417673 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c5419d1b0 con 0x7f0c540ff350 2024-02-19T19:09:05.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.985+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3590417673 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c5410a230 con 0x7f0c540ff350 2024-02-19T19:09:05.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.986+0000 7f0c4affd700 1 -- 172.21.15.88:0/3590417673 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0c5000f6f0 con 0x7f0c540ff350 2024-02-19T19:09:05.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.986+0000 7f0c4affd700 1 --2- 172.21.15.88:0/3590417673 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0c4006d8a0 0x7f0c4006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:05.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.986+0000 7f0c4affd700 1 -- 172.21.15.88:0/3590417673 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0c5008e860 con 0x7f0c540ff350 2024-02-19T19:09:05.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.986+0000 7f0c59242700 1 --2- 172.21.15.88:0/3590417673 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0c4006d8a0 0x7f0c4006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:05.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.986+0000 7f0c59242700 1 --2- 172.21.15.88:0/3590417673 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0c4006d8a0 0x7f0c4006fd50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0c541996b0 tx=0x7f0c4400b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:05.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:04.991+0000 7f0c4affd700 1 -- 172.21.15.88:0/3590417673 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0c500580c0 con 0x7f0c540ff350 2024-02-19T19:09:05.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.267+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3590417673 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0c5404f2a0 con 0x7f0c540ff350 2024-02-19T19:09:05.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.272+0000 7f0c4affd700 1 -- 172.21.15.88:0/3590417673 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f0c5005b780 con 0x7f0c540ff350 2024-02-19T19:09:05.294 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_WARN 1 osds down 2024-02-19T19:09:05.294 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] OSD_DOWN: 1 osds down 2024-02-19T19:09:05.294 INFO:teuthology.orchestra.run.smithi088.stdout: osd.3 (root=default,host=smithi132) is down 2024-02-19T19:09:05.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.276+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3590417673 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0c4006d8a0 msgr2=0x7f0c4006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:05.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.276+0000 7f0c5bca7700 1 --2- 172.21.15.88:0/3590417673 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0c4006d8a0 0x7f0c4006fd50 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0c541996b0 tx=0x7f0c4400b410 comp rx=0 tx=0).stop 2024-02-19T19:09:05.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.276+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3590417673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c540ff350 msgr2=0x7f0c54198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:05.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.276+0000 7f0c5bca7700 1 --2- 172.21.15.88:0/3590417673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c540ff350 0x7f0c54198090 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c500092e0 tx=0x7f0c5000bd70 comp rx=0 tx=0).stop 2024-02-19T19:09:05.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.276+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3590417673 shutdown_connections 2024-02-19T19:09:05.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.276+0000 7f0c5bca7700 1 --2- 172.21.15.88:0/3590417673 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0c4006d8a0 0x7f0c4006fd50 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:05.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.276+0000 7f0c5bca7700 1 --2- 172.21.15.88:0/3590417673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c540ff350 0x7f0c54198090 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:05.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.276+0000 7f0c5bca7700 1 --2- 172.21.15.88:0/3590417673 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0c540ffc60 0x7f0c541985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:05.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.276+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3590417673 >> 172.21.15.88:0/3590417673 conn(0x7f0c540748b0 msgr2=0x7f0c54100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:05.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.276+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3590417673 shutdown_connections 2024-02-19T19:09:05.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:05.276+0000 7f0c5bca7700 1 -- 172.21.15.88:0/3590417673 wait complete. 2024-02-19T19:09:05.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:05 smithi088 ceph-mon[88643]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:05.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:05 smithi088 ceph-mon[88643]: pgmap v144: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:05.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:05 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/634902330' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:09:05.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:05 smithi088 ceph-mon[88643]: osd.3 marked itself down and dead 2024-02-19T19:09:05.693 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:05 smithi132 ceph-mon[70676]: from='client.34340 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:05.693 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:05 smithi132 ceph-mon[70676]: pgmap v144: 65 pgs: 65 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:05.693 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:05 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/634902330' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:09:05.693 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:05 smithi132 ceph-mon[70676]: osd.3 marked itself down and dead 2024-02-19T19:09:05.966 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:05 smithi132 podman[80224]: 2024-02-19 19:09:05.672819089 +0000 UTC m=+0.930264868 container died 3a3ade48fb36ec78c60564545d69c39ab26cb68819260f2aba7b4cc8b1c7915e (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=HEAD, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , ceph=True) 2024-02-19T19:09:06.254 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:06 smithi132 podman[80224]: 2024-02-19 19:09:06.044508146 +0000 UTC m=+1.301953924 container cleanup 3a3ade48fb36ec78c60564545d69c39ab26cb68819260f2aba7b4cc8b1c7915e (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-02-19T19:09:06.254 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:06 smithi132 bash[80224]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3 2024-02-19T19:09:06.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:06 smithi088 ceph-mon[88643]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:06.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:06 smithi088 ceph-mon[88643]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:09:06.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:06 smithi088 ceph-mon[88643]: osdmap e68: 6 total, 5 up, 6 in 2024-02-19T19:09:06.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:06 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3590417673' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:09:06.681 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:06 smithi132 podman[80242]: 2024-02-19 19:09:06.337741291 +0000 UTC m=+0.660409943 container remove 3a3ade48fb36ec78c60564545d69c39ab26cb68819260f2aba7b4cc8b1c7915e (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:09:06.682 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:06 smithi132 ceph-mon[70676]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:06.682 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:06 smithi132 ceph-mon[70676]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:09:06.682 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:06 smithi132 ceph-mon[70676]: osdmap e68: 6 total, 5 up, 6 in 2024-02-19T19:09:06.682 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:06 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3590417673' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:09:06.945 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:06 smithi132 podman[80333]: 2024-02-19 19:09:06.564069238 +0000 UTC m=+0.018278367 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:09:06.946 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:06 smithi132 podman[80333]: 2024-02-19 19:09:06.68905629 +0000 UTC m=+0.143265457 container create 58823781718c815090e5b6dbc58836804371bb9bdb90e8bafcbe7fefb8fffabd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-02-19T19:09:06.946 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:06 smithi132 podman[80333]: 2024-02-19 19:09:06.925044462 +0000 UTC m=+0.379253586 container init 58823781718c815090e5b6dbc58836804371bb9bdb90e8bafcbe7fefb8fffabd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-19T19:09:07.235 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:06 smithi132 podman[80333]: 2024-02-19 19:09:06.928666753 +0000 UTC m=+0.382875885 container start 58823781718c815090e5b6dbc58836804371bb9bdb90e8bafcbe7fefb8fffabd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-19T19:09:07.235 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:07 smithi132 podman[80333]: 2024-02-19 19:09:07.016508103 +0000 UTC m=+0.470717228 container attach 58823781718c815090e5b6dbc58836804371bb9bdb90e8bafcbe7fefb8fffabd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-19T19:09:07.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:07 smithi132 ceph-mon[70676]: pgmap v146: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:07.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:07 smithi132 ceph-mon[70676]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-19T19:09:07.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:07 smithi132 ceph-mon[70676]: osdmap e69: 6 total, 5 up, 6 in 2024-02-19T19:09:07.504 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:07 smithi132 podman[80333]: 2024-02-19 19:09:07.214382035 +0000 UTC m=+0.668591161 container died 58823781718c815090e5b6dbc58836804371bb9bdb90e8bafcbe7fefb8fffabd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214) 2024-02-19T19:09:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:07 smithi088 ceph-mon[88643]: pgmap v146: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:07 smithi088 ceph-mon[88643]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-19T19:09:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:07 smithi088 ceph-mon[88643]: osdmap e69: 6 total, 5 up, 6 in 2024-02-19T19:09:08.254 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:07 smithi132 podman[80359]: 2024-02-19 19:09:07.903824809 +0000 UTC m=+0.683988102 container remove 58823781718c815090e5b6dbc58836804371bb9bdb90e8bafcbe7fefb8fffabd (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0) 2024-02-19T19:09:08.254 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:07 smithi132 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.3.service: Deactivated successfully. 2024-02-19T19:09:08.254 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:07 smithi132 systemd[1]: Stopped Ceph osd.3 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:09:08.254 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:07 smithi132 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.3.service: Consumed 1min 15.098s CPU time. 2024-02-19T19:09:08.254 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:08 smithi132 systemd[1]: Starting Ceph osd.3 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:09:08.654 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:08 smithi132 podman[80461]: 2024-02-19 19:09:08.249469551 +0000 UTC m=+0.017643129 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:09:08.654 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:08 smithi132 podman[80461]: 2024-02-19 19:09:08.388469051 +0000 UTC m=+0.156642624 container create 4f724c4e5d242a983427bd36cc4bb860d7673cc7c35ff5cf92e487dbde07355f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T19:09:08.654 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:08 smithi132 podman[80461]: 2024-02-19 19:09:08.633124245 +0000 UTC m=+0.401297826 container init 4f724c4e5d242a983427bd36cc4bb860d7673cc7c35ff5cf92e487dbde07355f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-19T19:09:09.003 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:08 smithi132 podman[80461]: 2024-02-19 19:09:08.636611635 +0000 UTC m=+0.404785211 container start 4f724c4e5d242a983427bd36cc4bb860d7673cc7c35ff5cf92e487dbde07355f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:09:09.004 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:08 smithi132 podman[80461]: 2024-02-19 19:09:08.724762237 +0000 UTC m=+0.492935826 container attach 4f724c4e5d242a983427bd36cc4bb860d7673cc7c35ff5cf92e487dbde07355f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2) 2024-02-19T19:09:09.412 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:09 smithi132 ceph-mon[70676]: pgmap v148: 65 pgs: 11 peering, 12 stale+active+clean, 42 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:09.412 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate[80477]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-19T19:09:09.413 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate[80477]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-19T19:09:09.413 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 bash[80461]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-19T19:09:09.413 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 bash[80461]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-02-19T19:09:09.413 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 bash[80461]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-19T19:09:09.413 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 bash[80461]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-19T19:09:09.413 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 bash[80461]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-19T19:09:09.413 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 bash[80461]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-19T19:09:09.413 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 bash[80461]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-19T19:09:09.413 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate[80477]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-02-19T19:09:09.413 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate[80477]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-02-19T19:09:09.414 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate[80477]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-02-19T19:09:09.414 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate[80477]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-02-19T19:09:09.414 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate[80477]: --> ceph-volume raw activate successful for osd ID: 3 2024-02-19T19:09:09.414 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 podman[80461]: 2024-02-19 19:09:09.096825764 +0000 UTC m=+0.864999339 container died 4f724c4e5d242a983427bd36cc4bb860d7673cc7c35ff5cf92e487dbde07355f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-19T19:09:09.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:09 smithi088 ceph-mon[88643]: pgmap v148: 65 pgs: 11 peering, 12 stale+active+clean, 42 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:10.004 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 podman[80613]: 2024-02-19 19:09:09.703445415 +0000 UTC m=+0.602979092 container remove 4f724c4e5d242a983427bd36cc4bb860d7673cc7c35ff5cf92e487dbde07355f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-activate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214) 2024-02-19T19:09:10.004 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 podman[80680]: 2024-02-19 19:09:09.823181609 +0000 UTC m=+0.018077072 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:09:10.004 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:09 smithi132 podman[80680]: 2024-02-19 19:09:09.937833678 +0000 UTC m=+0.132729176 container create 6dc5d99df1e5edec7d0005ac665903d35217f827693fb35041e421cf8d42c50f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T19:09:10.340 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:10 smithi132 podman[80680]: 2024-02-19 19:09:10.174552695 +0000 UTC m=+0.369448148 container init 6dc5d99df1e5edec7d0005ac665903d35217f827693fb35041e421cf8d42c50f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-19T19:09:10.341 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:10 smithi132 podman[80680]: 2024-02-19 19:09:10.177551268 +0000 UTC m=+0.372446723 container start 6dc5d99df1e5edec7d0005ac665903d35217f827693fb35041e421cf8d42c50f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-19T19:09:10.341 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:10 smithi132 bash[80680]: 6dc5d99df1e5edec7d0005ac665903d35217f827693fb35041e421cf8d42c50f 2024-02-19T19:09:10.341 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:10 smithi132 systemd[1]: Started Ceph osd.3 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:09:10.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:10 smithi088 ceph-mon[88643]: Health check failed: Degraded data redundancy: 59/333 objects degraded (17.718%), 15 pgs degraded (PG_DEGRADED) 2024-02-19T19:09:10.739 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:10 smithi132 ceph-mon[70676]: Health check failed: Degraded data redundancy: 59/333 objects degraded (17.718%), 15 pgs degraded (PG_DEGRADED) 2024-02-19T19:09:11.004 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:10 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[80695]: 2024-02-19T19:09:10.717+0000 7fa2f27bc7c0 -1 Falling back to public interface 2024-02-19T19:09:11.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:11 smithi132 ceph-mon[70676]: pgmap v149: 65 pgs: 15 active+undersized+degraded, 11 peering, 12 active+undersized, 27 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/333 objects degraded (17.718%) 2024-02-19T19:09:11.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:11.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:11.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:09:11.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:11 smithi088 ceph-mon[88643]: pgmap v149: 65 pgs: 15 active+undersized+degraded, 11 peering, 12 active+undersized, 27 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/333 objects degraded (17.718%) 2024-02-19T19:09:11.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:11.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:11.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:09:12.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:12 smithi132 ceph-mon[70676]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 4 pgs peering) 2024-02-19T19:09:12.715 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:12 smithi088 ceph-mon[88643]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 4 pgs peering) 2024-02-19T19:09:13.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:13 smithi132 ceph-mon[70676]: pgmap v150: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-02-19T19:09:13.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:13.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:09:13.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:13 smithi088 ceph-mon[88643]: pgmap v150: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-02-19T19:09:13.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:13.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:09:14.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:14.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:14.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:14.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:15.517 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:15 smithi132 ceph-mon[70676]: pgmap v151: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-02-19T19:09:15.518 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:15 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:15.518 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:15 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:15.518 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:15 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[80695]: 2024-02-19T19:09:15.302+0000 7fa2f27bc7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-02-19T19:09:15.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:15 smithi088 ceph-mon[88643]: pgmap v151: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-02-19T19:09:15.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:15.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:16.254 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:15 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[80695]: 2024-02-19T19:09:15.823+0000 7fa2f27bc7c0 -1 osd.3 67 log_to_monitors true 2024-02-19T19:09:16.583 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:16 smithi132 ceph-mon[70676]: from='osd.3 [v2:172.21.15.132:6800/2930324400,v1:172.21.15.132:6801/2930324400]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-19T19:09:16.583 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:16 smithi132 ceph-mon[70676]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-19T19:09:16.584 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 19:09:16 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[80695]: 2024-02-19T19:09:16.489+0000 7fa2e3c14700 -1 osd.3 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T19:09:16.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:16 smithi088 ceph-mon[88643]: from='osd.3 [v2:172.21.15.132:6800/2930324400,v1:172.21.15.132:6801/2930324400]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-19T19:09:16.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:16 smithi088 ceph-mon[88643]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-19T19:09:17.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:17 smithi132 ceph-mon[70676]: pgmap v152: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-02-19T19:09:17.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:17 smithi132 ceph-mon[70676]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-19T19:09:17.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:17 smithi132 ceph-mon[70676]: osdmap e70: 6 total, 5 up, 6 in 2024-02-19T19:09:17.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:17 smithi132 ceph-mon[70676]: from='osd.3 [v2:172.21.15.132:6800/2930324400,v1:172.21.15.132:6801/2930324400]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:09:17.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:17 smithi132 ceph-mon[70676]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:09:17.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:17 smithi088 ceph-mon[88643]: pgmap v152: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 272 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-02-19T19:09:17.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:17 smithi088 ceph-mon[88643]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-19T19:09:17.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:17 smithi088 ceph-mon[88643]: osdmap e70: 6 total, 5 up, 6 in 2024-02-19T19:09:17.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:17 smithi088 ceph-mon[88643]: from='osd.3 [v2:172.21.15.132:6800/2930324400,v1:172.21.15.132:6801/2930324400]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:09:17.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:17 smithi088 ceph-mon[88643]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:09:18.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:09:18.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: osd.3 [v2:172.21.15.132:6800/2930324400,v1:172.21.15.132:6801/2930324400] boot 2024-02-19T19:09:18.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: osdmap e71: 6 total, 6 up, 6 in 2024-02-19T19:09:18.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T19:09:18.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:18.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:18.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:09:18.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:09:18.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:18.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:09:18.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:18.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:18.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:18.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-19T19:09:18.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:18 smithi088 ceph-mon[88643]: osdmap e72: 6 total, 6 up, 6 in 2024-02-19T19:09:18.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:09:18.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: osd.3 [v2:172.21.15.132:6800/2930324400,v1:172.21.15.132:6801/2930324400] boot 2024-02-19T19:09:18.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: osdmap e71: 6 total, 6 up, 6 in 2024-02-19T19:09:18.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-19T19:09:18.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:18.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:18.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:09:18.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:09:18.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:18.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:09:18.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:18.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:18.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:18.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-19T19:09:18.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:18 smithi132 ceph-mon[70676]: osdmap e72: 6 total, 6 up, 6 in 2024-02-19T19:09:19.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:19 smithi088 ceph-mon[88643]: pgmap v155: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-02-19T19:09:19.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:19 smithi088 ceph-mon[88643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-19T19:09:19.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:19 smithi088 ceph-mon[88643]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-19T19:09:20.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:19 smithi132 ceph-mon[70676]: pgmap v155: 65 pgs: 19 active+undersized+degraded, 19 active+undersized, 27 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 69/333 objects degraded (20.721%) 2024-02-19T19:09:20.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:19 smithi132 ceph-mon[70676]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-19T19:09:20.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:19 smithi132 ceph-mon[70676]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-02-19T19:09:20.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:20 smithi088 ceph-mon[88643]: Health check update: Degraded data redundancy: 69/333 objects degraded (20.721%), 19 pgs degraded (PG_DEGRADED) 2024-02-19T19:09:21.003 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:20 smithi132 ceph-mon[70676]: Health check update: Degraded data redundancy: 69/333 objects degraded (20.721%), 19 pgs degraded (PG_DEGRADED) 2024-02-19T19:09:21.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:21 smithi088 ceph-mon[88643]: pgmap v157: 65 pgs: 12 active+undersized+degraded, 11 active+undersized, 42 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 48/333 objects degraded (14.414%) 2024-02-19T19:09:21.905 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:21 smithi132 ceph-mon[70676]: pgmap v157: 65 pgs: 12 active+undersized+degraded, 11 active+undersized, 42 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 48/333 objects degraded (14.414%) 2024-02-19T19:09:22.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:22 smithi088 ceph-mon[88643]: pgmap v158: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:22.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:22 smithi088 ceph-mon[88643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 48/333 objects degraded (14.414%), 12 pgs degraded) 2024-02-19T19:09:22.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:22 smithi088 ceph-mon[88643]: Cluster is now healthy 2024-02-19T19:09:23.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:22 smithi132 ceph-mon[70676]: pgmap v158: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:23.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:22 smithi132 ceph-mon[70676]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 48/333 objects degraded (14.414%), 12 pgs degraded) 2024-02-19T19:09:23.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:22 smithi132 ceph-mon[70676]: Cluster is now healthy 2024-02-19T19:09:25.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:25 smithi132 ceph-mon[70676]: pgmap v159: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:25.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:25 smithi088 ceph-mon[88643]: pgmap v159: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:27.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:27 smithi132 ceph-mon[70676]: pgmap v160: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:27.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:27 smithi088 ceph-mon[88643]: pgmap v160: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:29.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:29 smithi132 ceph-mon[70676]: pgmap v161: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:29.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:29.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:09:29.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:29.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:29 smithi088 ceph-mon[88643]: pgmap v161: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:29.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:29.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:09:29.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:31.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:30 smithi132 ceph-mon[70676]: pgmap v162: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:31.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:30 smithi088 ceph-mon[88643]: pgmap v162: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:33.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:33 smithi132 ceph-mon[70676]: pgmap v163: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:33.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:33 smithi088 ceph-mon[88643]: pgmap v163: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:34.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:34 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-19T19:09:34.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:34 smithi132 ceph-mon[70676]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-19T19:09:34.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:34 smithi132 ceph-mon[70676]: Upgrade: osd.4 is safe to restart 2024-02-19T19:09:34.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:34 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-19T19:09:34.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:34 smithi088 ceph-mon[88643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-02-19T19:09:34.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:34 smithi088 ceph-mon[88643]: Upgrade: osd.4 is safe to restart 2024-02-19T19:09:35.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.393+0000 7f9e643f1700 1 -- 172.21.15.88:0/2828869266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e5c100e80 msgr2=0x7f9e5c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:35.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.393+0000 7f9e643f1700 1 --2- 172.21.15.88:0/2828869266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e5c100e80 0x7f9e5c104e50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c009910 tx=0x7f9e4c009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:35.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.393+0000 7f9e643f1700 1 -- 172.21.15.88:0/2828869266 shutdown_connections 2024-02-19T19:09:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.393+0000 7f9e643f1700 1 --2- 172.21.15.88:0/2828869266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e5c100e80 0x7f9e5c104e50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.393+0000 7f9e643f1700 1 --2- 172.21.15.88:0/2828869266 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5c1004e0 0x7f9e5c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.393+0000 7f9e643f1700 1 -- 172.21.15.88:0/2828869266 >> 172.21.15.88:0/2828869266 conn(0x7f9e5c0fbda0 msgr2=0x7f9e5c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.393+0000 7f9e643f1700 1 -- 172.21.15.88:0/2828869266 shutdown_connections 2024-02-19T19:09:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.393+0000 7f9e643f1700 1 -- 172.21.15.88:0/2828869266 wait complete. 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.393+0000 7f9e643f1700 1 Processor -- start 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.393+0000 7f9e643f1700 1 -- start start 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e643f1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5c1004e0 0x7f9e5c195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e643f1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e5c100e80 0x7f9e5c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e643f1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c196b30 con 0x7f9e5c100e80 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e643f1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c19a8c0 con 0x7f9e5c1004e0 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e6198c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e5c100e80 0x7f9e5c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e6198c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e5c100e80 0x7f9e5c196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50088/0 (socket says 172.21.15.88:50088) 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e6198c700 1 -- 172.21.15.88:0/1395703791 learned_addr learned my addr 172.21.15.88:0/1395703791 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e6218d700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5c1004e0 0x7f9e5c195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e6198c700 1 -- 172.21.15.88:0/1395703791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5c1004e0 msgr2=0x7f9e5c195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:35.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e6198c700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5c1004e0 0x7f9e5c195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e6198c700 1 -- 172.21.15.88:0/1395703791 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c0095f0 con 0x7f9e5c100e80 2024-02-19T19:09:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e6218d700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5c1004e0 0x7f9e5c195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:09:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e6198c700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e5c100e80 0x7f9e5c196450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0059d0 tx=0x7f9e4c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e537fe700 1 -- 172.21.15.88:0/1395703791 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e4c01d070 con 0x7f9e5c100e80 2024-02-19T19:09:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e537fe700 1 -- 172.21.15.88:0/1395703791 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e4c00b610 con 0x7f9e5c100e80 2024-02-19T19:09:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e537fe700 1 -- 172.21.15.88:0/1395703791 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e4c022470 con 0x7f9e5c100e80 2024-02-19T19:09:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.394+0000 7f9e643f1700 1 -- 172.21.15.88:0/1395703791 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5c19ab40 con 0x7f9e5c100e80 2024-02-19T19:09:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.395+0000 7f9e643f1700 1 -- 172.21.15.88:0/1395703791 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c19b030 con 0x7f9e5c100e80 2024-02-19T19:09:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.396+0000 7f9e643f1700 1 -- 172.21.15.88:0/1395703791 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5c108850 con 0x7f9e5c100e80 2024-02-19T19:09:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.396+0000 7f9e537fe700 1 -- 172.21.15.88:0/1395703791 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9e4c00fc70 con 0x7f9e5c100e80 2024-02-19T19:09:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.397+0000 7f9e537fe700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4806d8a0 0x7f9e4806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.397+0000 7f9e6218d700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4806d8a0 0x7f9e4806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.397+0000 7f9e537fe700 1 -- 172.21.15.88:0/1395703791 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9e4c08e810 con 0x7f9e5c100e80 2024-02-19T19:09:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.397+0000 7f9e6218d700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4806d8a0 0x7f9e4806fd50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009cb0 tx=0x7f9e58009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.402+0000 7f9e537fe700 1 -- 172.21.15.88:0/1395703791 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9e4c0580f0 con 0x7f9e5c100e80 2024-02-19T19:09:35.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:35 smithi132 ceph-mon[70676]: pgmap v164: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:35.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:35.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-19T19:09:35.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:09:35.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:35 smithi088 ceph-mon[88643]: pgmap v164: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:35.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:35.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-19T19:09:35.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:09:35.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.617+0000 7f9e643f1700 1 -- 172.21.15.88:0/1395703791 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e5c19b2a0 con 0x7f9e4806d8a0 2024-02-19T19:09:35.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.618+0000 7f9e537fe700 1 -- 172.21.15.88:0/1395703791 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9e5c19b2a0 con 0x7f9e4806d8a0 2024-02-19T19:09:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.620+0000 7f9e643f1700 1 -- 172.21.15.88:0/1395703791 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4806d8a0 msgr2=0x7f9e4806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.620+0000 7f9e643f1700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4806d8a0 0x7f9e4806fd50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009cb0 tx=0x7f9e58009260 comp rx=0 tx=0).stop 2024-02-19T19:09:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.620+0000 7f9e643f1700 1 -- 172.21.15.88:0/1395703791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e5c100e80 msgr2=0x7f9e5c196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.620+0000 7f9e643f1700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e5c100e80 0x7f9e5c196450 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0059d0 tx=0x7f9e4c00f740 comp rx=0 tx=0).stop 2024-02-19T19:09:35.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.620+0000 7f9e643f1700 1 -- 172.21.15.88:0/1395703791 shutdown_connections 2024-02-19T19:09:35.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.620+0000 7f9e643f1700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4806d8a0 0x7f9e4806fd50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.620+0000 7f9e643f1700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e5c1004e0 0x7f9e5c195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.620+0000 7f9e643f1700 1 --2- 172.21.15.88:0/1395703791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e5c100e80 0x7f9e5c196450 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.620+0000 7f9e643f1700 1 -- 172.21.15.88:0/1395703791 >> 172.21.15.88:0/1395703791 conn(0x7f9e5c0fbda0 msgr2=0x7f9e5c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:35.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.620+0000 7f9e643f1700 1 -- 172.21.15.88:0/1395703791 shutdown_connections 2024-02-19T19:09:35.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.620+0000 7f9e643f1700 1 -- 172.21.15.88:0/1395703791 wait complete. 2024-02-19T19:09:35.655 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:09:35.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.725+0000 7f52ac7b9700 1 -- 172.21.15.88:0/1299071525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52a4100e80 msgr2=0x7f52a4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:35.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.725+0000 7f52ac7b9700 1 --2- 172.21.15.88:0/1299071525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52a4100e80 0x7f52a4104e50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5294009910 tx=0x7f5294009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:35.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.725+0000 7f52ac7b9700 1 -- 172.21.15.88:0/1299071525 shutdown_connections 2024-02-19T19:09:35.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.725+0000 7f52ac7b9700 1 --2- 172.21.15.88:0/1299071525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52a4100e80 0x7f52a4104e50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.725+0000 7f52ac7b9700 1 --2- 172.21.15.88:0/1299071525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52a41004e0 0x7f52a41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.725+0000 7f52ac7b9700 1 -- 172.21.15.88:0/1299071525 >> 172.21.15.88:0/1299071525 conn(0x7f52a40fbda0 msgr2=0x7f52a40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:35.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.725+0000 7f52ac7b9700 1 -- 172.21.15.88:0/1299071525 shutdown_connections 2024-02-19T19:09:35.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.725+0000 7f52ac7b9700 1 -- 172.21.15.88:0/1299071525 wait complete. 2024-02-19T19:09:35.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52ac7b9700 1 Processor -- start 2024-02-19T19:09:35.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52ac7b9700 1 -- start start 2024-02-19T19:09:35.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52ac7b9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52a41004e0 0x7f52a4195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:35.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52ac7b9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52a4100e80 0x7f52a4196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:35.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52ac7b9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a4196b30 con 0x7f52a41004e0 2024-02-19T19:09:35.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52ac7b9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a419a8c0 con 0x7f52a4100e80 2024-02-19T19:09:35.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52aa555700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52a41004e0 0x7f52a4195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:35.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52aa555700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52a41004e0 0x7f52a4195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50024/0 (socket says 172.21.15.88:50024) 2024-02-19T19:09:35.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52aa555700 1 -- 172.21.15.88:0/4155815072 learned_addr learned my addr 172.21.15.88:0/4155815072 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52a9d54700 1 --2- 172.21.15.88:0/4155815072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52a4100e80 0x7f52a4196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52aa555700 1 -- 172.21.15.88:0/4155815072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52a4100e80 msgr2=0x7f52a4196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52aa555700 1 --2- 172.21.15.88:0/4155815072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52a4100e80 0x7f52a4196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.726+0000 7f52aa555700 1 -- 172.21.15.88:0/4155815072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52940095f0 con 0x7f52a41004e0 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.727+0000 7f52aa555700 1 --2- 172.21.15.88:0/4155815072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52a41004e0 0x7f52a4195f10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f52a000e920 tx=0x7f52a000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.727+0000 7f529b7fe700 1 -- 172.21.15.88:0/4155815072 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52a000c9d0 con 0x7f52a41004e0 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.727+0000 7f52ac7b9700 1 -- 172.21.15.88:0/4155815072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a419aba0 con 0x7f52a41004e0 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.727+0000 7f529b7fe700 1 -- 172.21.15.88:0/4155815072 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52a0010430 con 0x7f52a41004e0 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.727+0000 7f529b7fe700 1 -- 172.21.15.88:0/4155815072 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52a0018630 con 0x7f52a41004e0 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.727+0000 7f52ac7b9700 1 -- 172.21.15.88:0/4155815072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a419b0c0 con 0x7f52a41004e0 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.728+0000 7f52ac7b9700 1 -- 172.21.15.88:0/4155815072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a4108850 con 0x7f52a41004e0 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.728+0000 7f529b7fe700 1 -- 172.21.15.88:0/4155815072 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f52a00105a0 con 0x7f52a41004e0 2024-02-19T19:09:35.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.729+0000 7f529b7fe700 1 --2- 172.21.15.88:0/4155815072 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f529006d8a0 0x7f529006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:35.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.729+0000 7f52a9d54700 1 --2- 172.21.15.88:0/4155815072 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f529006d8a0 0x7f529006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:35.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.729+0000 7f529b7fe700 1 -- 172.21.15.88:0/4155815072 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f52a0014070 con 0x7f52a41004e0 2024-02-19T19:09:35.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.729+0000 7f52a9d54700 1 --2- 172.21.15.88:0/4155815072 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f529006d8a0 0x7f529006fd50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f52940059d0 tx=0x7f5294005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:35.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.733+0000 7f529b7fe700 1 -- 172.21.15.88:0/4155815072 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f52a0052d20 con 0x7f52a41004e0 2024-02-19T19:09:35.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.945+0000 7f52ac7b9700 1 -- 172.21.15.88:0/4155815072 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f52a4197380 con 0x7f529006d8a0 2024-02-19T19:09:35.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.946+0000 7f529b7fe700 1 -- 172.21.15.88:0/4155815072 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f52a4197380 con 0x7f529006d8a0 2024-02-19T19:09:35.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.948+0000 7f52ac7b9700 1 -- 172.21.15.88:0/4155815072 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f529006d8a0 msgr2=0x7f529006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:35.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.948+0000 7f52ac7b9700 1 --2- 172.21.15.88:0/4155815072 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f529006d8a0 0x7f529006fd50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f52940059d0 tx=0x7f5294005940 comp rx=0 tx=0).stop 2024-02-19T19:09:35.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.948+0000 7f52ac7b9700 1 -- 172.21.15.88:0/4155815072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52a41004e0 msgr2=0x7f52a4195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:35.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.948+0000 7f52ac7b9700 1 --2- 172.21.15.88:0/4155815072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52a41004e0 0x7f52a4195f10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f52a000e920 tx=0x7f52a000ec30 comp rx=0 tx=0).stop 2024-02-19T19:09:35.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.948+0000 7f52ac7b9700 1 -- 172.21.15.88:0/4155815072 shutdown_connections 2024-02-19T19:09:35.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.948+0000 7f52ac7b9700 1 --2- 172.21.15.88:0/4155815072 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f529006d8a0 0x7f529006fd50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.948+0000 7f52ac7b9700 1 --2- 172.21.15.88:0/4155815072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52a41004e0 0x7f52a4195f10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.948+0000 7f52ac7b9700 1 --2- 172.21.15.88:0/4155815072 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52a4100e80 0x7f52a4196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:35.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.948+0000 7f52ac7b9700 1 -- 172.21.15.88:0/4155815072 >> 172.21.15.88:0/4155815072 conn(0x7f52a40fbda0 msgr2=0x7f52a40fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:35.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.948+0000 7f52ac7b9700 1 -- 172.21.15.88:0/4155815072 shutdown_connections 2024-02-19T19:09:35.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:35.949+0000 7f52ac7b9700 1 -- 172.21.15.88:0/4155815072 wait complete. 2024-02-19T19:09:36.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.052+0000 7f33f15e2700 1 -- 172.21.15.88:0/2957335143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ec104130 msgr2=0x7f33ec104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:36.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.052+0000 7f33f15e2700 1 --2- 172.21.15.88:0/2957335143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ec104130 0x7f33ec104500 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009910 tx=0x7f33e0009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:36.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.052+0000 7f33f15e2700 1 -- 172.21.15.88:0/2957335143 shutdown_connections 2024-02-19T19:09:36.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.052+0000 7f33f15e2700 1 --2- 172.21.15.88:0/2957335143 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33ec0ffbd0 0x7f33ec100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.052+0000 7f33f15e2700 1 --2- 172.21.15.88:0/2957335143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ec104130 0x7f33ec104500 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.052+0000 7f33f15e2700 1 -- 172.21.15.88:0/2957335143 >> 172.21.15.88:0/2957335143 conn(0x7f33ec075130 msgr2=0x7f33ec075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:36.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.052+0000 7f33f15e2700 1 -- 172.21.15.88:0/2957335143 shutdown_connections 2024-02-19T19:09:36.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.052+0000 7f33f15e2700 1 -- 172.21.15.88:0/2957335143 wait complete. 2024-02-19T19:09:36.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.053+0000 7f33f15e2700 1 Processor -- start 2024-02-19T19:09:36.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.053+0000 7f33f15e2700 1 -- start start 2024-02-19T19:09:36.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.053+0000 7f33f15e2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33ec0ffbd0 0x7f33ec196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:36.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.053+0000 7f33f15e2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ec104130 0x7f33ec1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:36.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.053+0000 7f33f15e2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec1977f0 con 0x7f33ec104130 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.053+0000 7f33f15e2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec197960 con 0x7f33ec0ffbd0 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.053+0000 7f33ea7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ec104130 0x7f33ec1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.053+0000 7f33ea7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ec104130 0x7f33ec1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50044/0 (socket says 172.21.15.88:50044) 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.053+0000 7f33ea7fc700 1 -- 172.21.15.88:0/2740129893 learned_addr learned my addr 172.21.15.88:0/2740129893 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.053+0000 7f33eaffd700 1 --2- 172.21.15.88:0/2740129893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33ec0ffbd0 0x7f33ec196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.054+0000 7f33ea7fc700 1 -- 172.21.15.88:0/2740129893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33ec0ffbd0 msgr2=0x7f33ec196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.054+0000 7f33ea7fc700 1 --2- 172.21.15.88:0/2740129893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33ec0ffbd0 0x7f33ec196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.054+0000 7f33ea7fc700 1 -- 172.21.15.88:0/2740129893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e00095f0 con 0x7f33ec104130 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.054+0000 7f33ea7fc700 1 --2- 172.21.15.88:0/2740129893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ec104130 0x7f33ec1971d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00d8d0 tx=0x7f33dc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.054+0000 7f33d3fff700 1 -- 172.21.15.88:0/2740129893 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33dc0098a0 con 0x7f33ec104130 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.054+0000 7f33d3fff700 1 -- 172.21.15.88:0/2740129893 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33dc010460 con 0x7f33ec104130 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.054+0000 7f33f15e2700 1 -- 172.21.15.88:0/2740129893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ec19fd00 con 0x7f33ec104130 2024-02-19T19:09:36.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.054+0000 7f33d3fff700 1 -- 172.21.15.88:0/2740129893 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33dc00f5f0 con 0x7f33ec104130 2024-02-19T19:09:36.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.054+0000 7f33f15e2700 1 -- 172.21.15.88:0/2740129893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ec1a01f0 con 0x7f33ec104130 2024-02-19T19:09:36.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.055+0000 7f33f15e2700 1 -- 172.21.15.88:0/2740129893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ec1076a0 con 0x7f33ec104130 2024-02-19T19:09:36.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.055+0000 7f33d3fff700 1 -- 172.21.15.88:0/2740129893 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f33dc009a00 con 0x7f33ec104130 2024-02-19T19:09:36.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.056+0000 7f33d3fff700 1 --2- 172.21.15.88:0/2740129893 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f33d406d8a0 0x7f33d406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:36.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.056+0000 7f33d3fff700 1 -- 172.21.15.88:0/2740129893 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f33dc08d260 con 0x7f33ec104130 2024-02-19T19:09:36.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.056+0000 7f33eaffd700 1 --2- 172.21.15.88:0/2740129893 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f33d406d8a0 0x7f33d406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:36.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.056+0000 7f33eaffd700 1 --2- 172.21.15.88:0/2740129893 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f33d406d8a0 0x7f33d406fd50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:36.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.060+0000 7f33d3fff700 1 -- 172.21.15.88:0/2740129893 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33dc056b40 con 0x7f33ec104130 2024-02-19T19:09:36.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.267+0000 7f33f15e2700 1 -- 172.21.15.88:0/2740129893 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f33ec061190 con 0x7f33d406d8a0 2024-02-19T19:09:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.278+0000 7f33d3fff700 1 -- 172.21.15.88:0/2740129893 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f33ec061190 con 0x7f33d406d8a0 2024-02-19T19:09:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:09:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (6m) 56s ago 17m 24.2M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:09:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (17m) 56s ago 17m 8674k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:09:36.299 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (15m) 22s ago 15m 8200k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (2m) 56s ago 17m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (2m) 22s ago 15m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (6m) 56s ago 16m 85.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (11m) 56s ago 10m 17.0M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (10m) 56s ago 10m 114M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (10m) 22s ago 10m 112M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (10m) 22s ago 10m 21.1M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (8m) 56s ago 18m 501M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (7m) 22s ago 15m 451M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (4m) 56s ago 19m 50.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (4m) 22s ago 15m 39.8M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (7m) 56s ago 17m 15.4M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (7m) 22s ago 15m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (2m) 56s ago 14m 127M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (98s) 56s ago 14m 99.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:09:36.300 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (62s) 56s ago 13m 12.5M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:09:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (26s) 22s ago 13m 14.9M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:09:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (12m) 22s ago 12m 229M 2240M 18.2.0 dc2bc1663786 f4bfc92e5c93 2024-02-19T19:09:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (12m) 22s ago 12m 209M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:09:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (7m) 56s ago 16m 69.2M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:09:36.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.280+0000 7f33f15e2700 1 -- 172.21.15.88:0/2740129893 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f33d406d8a0 msgr2=0x7f33d406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:36.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.280+0000 7f33f15e2700 1 --2- 172.21.15.88:0/2740129893 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f33d406d8a0 0x7f33d406fd50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f33e0000c00 tx=0x7f33e001a040 comp rx=0 tx=0).stop 2024-02-19T19:09:36.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.280+0000 7f33f15e2700 1 -- 172.21.15.88:0/2740129893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ec104130 msgr2=0x7f33ec1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:36.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.280+0000 7f33f15e2700 1 --2- 172.21.15.88:0/2740129893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ec104130 0x7f33ec1971d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00d8d0 tx=0x7f33dc00dbe0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.280+0000 7f33f15e2700 1 -- 172.21.15.88:0/2740129893 shutdown_connections 2024-02-19T19:09:36.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.280+0000 7f33f15e2700 1 --2- 172.21.15.88:0/2740129893 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f33d406d8a0 0x7f33d406fd50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.280+0000 7f33f15e2700 1 --2- 172.21.15.88:0/2740129893 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33ec0ffbd0 0x7f33ec196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.280+0000 7f33f15e2700 1 --2- 172.21.15.88:0/2740129893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33ec104130 0x7f33ec1971d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.280+0000 7f33f15e2700 1 -- 172.21.15.88:0/2740129893 >> 172.21.15.88:0/2740129893 conn(0x7f33ec075130 msgr2=0x7f33ec0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:36.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.280+0000 7f33f15e2700 1 -- 172.21.15.88:0/2740129893 shutdown_connections 2024-02-19T19:09:36.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.280+0000 7f33f15e2700 1 -- 172.21.15.88:0/2740129893 wait complete. 2024-02-19T19:09:36.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.384+0000 7fb2f2d24700 1 -- 172.21.15.88:0/1573775782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2ec0ff720 msgr2=0x7fb2ec0ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:36.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.384+0000 7fb2f2d24700 1 --2- 172.21.15.88:0/1573775782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2ec0ff720 0x7fb2ec0ffb90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009910 tx=0x7fb2dc009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:36.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.384+0000 7fb2f2d24700 1 -- 172.21.15.88:0/1573775782 shutdown_connections 2024-02-19T19:09:36.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.384+0000 7fb2f2d24700 1 --2- 172.21.15.88:0/1573775782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2ec0ff720 0x7fb2ec0ffb90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.384+0000 7fb2f2d24700 1 --2- 172.21.15.88:0/1573775782 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2ec0ff350 0x7fb2ec1063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.384+0000 7fb2f2d24700 1 -- 172.21.15.88:0/1573775782 >> 172.21.15.88:0/1573775782 conn(0x7fb2ec0748b0 msgr2=0x7fb2ec074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:36.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.384+0000 7fb2f2d24700 1 -- 172.21.15.88:0/1573775782 shutdown_connections 2024-02-19T19:09:36.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.384+0000 7fb2f2d24700 1 -- 172.21.15.88:0/1573775782 wait complete. 2024-02-19T19:09:36.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.384+0000 7fb2f2d24700 1 Processor -- start 2024-02-19T19:09:36.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f2d24700 1 -- start start 2024-02-19T19:09:36.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f2d24700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2ec0ff350 0x7fb2ec19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:36.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f2d24700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2ec0ff720 0x7fb2ec19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:36.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f2d24700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec19cfa0 con 0x7fb2ec0ff350 2024-02-19T19:09:36.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f2d24700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec19d0e0 con 0x7fb2ec0ff720 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f0ac0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2ec0ff350 0x7fb2ec19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f0ac0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2ec0ff350 0x7fb2ec19c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50076/0 (socket says 172.21.15.88:50076) 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f0ac0700 1 -- 172.21.15.88:0/3240964781 learned_addr learned my addr 172.21.15.88:0/3240964781 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2e3fff700 1 --2- 172.21.15.88:0/3240964781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2ec0ff720 0x7fb2ec19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f0ac0700 1 -- 172.21.15.88:0/3240964781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2ec0ff720 msgr2=0x7fb2ec19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f0ac0700 1 --2- 172.21.15.88:0/3240964781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2ec0ff720 0x7fb2ec19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f0ac0700 1 -- 172.21.15.88:0/3240964781 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc0095f0 con 0x7fb2ec0ff350 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f0ac0700 1 --2- 172.21.15.88:0/3240964781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2ec0ff350 0x7fb2ec19c410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800c930 tx=0x7fb2e800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2e1ffb700 1 -- 172.21.15.88:0/3240964781 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2e8007ad0 con 0x7fb2ec0ff350 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2f2d24700 1 -- 172.21.15.88:0/3240964781 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ec1a0f30 con 0x7fb2ec0ff350 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2e1ffb700 1 -- 172.21.15.88:0/3240964781 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2e8007c30 con 0x7fb2ec0ff350 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.385+0000 7fb2e1ffb700 1 -- 172.21.15.88:0/3240964781 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2e8018610 con 0x7fb2ec0ff350 2024-02-19T19:09:36.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.386+0000 7fb2f2d24700 1 -- 172.21.15.88:0/3240964781 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ec1a1480 con 0x7fb2ec0ff350 2024-02-19T19:09:36.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.387+0000 7fb2f2d24700 1 -- 172.21.15.88:0/3240964781 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec103ae0 con 0x7fb2ec0ff350 2024-02-19T19:09:36.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.387+0000 7fb2e1ffb700 1 -- 172.21.15.88:0/3240964781 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb2e800f450 con 0x7fb2ec0ff350 2024-02-19T19:09:36.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.387+0000 7fb2e1ffb700 1 --2- 172.21.15.88:0/3240964781 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb2d406d8a0 0x7fb2d406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:36.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.387+0000 7fb2e1ffb700 1 -- 172.21.15.88:0/3240964781 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb2e808d6b0 con 0x7fb2ec0ff350 2024-02-19T19:09:36.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.388+0000 7fb2e3fff700 1 --2- 172.21.15.88:0/3240964781 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb2d406d8a0 0x7fb2d406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:36.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.388+0000 7fb2e3fff700 1 --2- 172.21.15.88:0/3240964781 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb2d406d8a0 0x7fb2d406fd50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc005f50 tx=0x7fb2dc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:36.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.392+0000 7fb2e1ffb700 1 -- 172.21.15.88:0/3240964781 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb2e8056f10 con 0x7fb2ec0ff350 2024-02-19T19:09:36.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:36 smithi132 ceph-mon[70676]: Upgrade: Updating osd.4 2024-02-19T19:09:36.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:36 smithi132 ceph-mon[70676]: Deploying daemon osd.4 on smithi132 2024-02-19T19:09:36.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:36 smithi088 ceph-mon[88643]: Upgrade: Updating osd.4 2024-02-19T19:09:36.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:36 smithi088 ceph-mon[88643]: Deploying daemon osd.4 on smithi132 2024-02-19T19:09:36.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.675+0000 7fb2f2d24700 1 -- 172.21.15.88:0/3240964781 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb2ec19d840 con 0x7fb2ec0ff350 2024-02-19T19:09:36.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.676+0000 7fb2e1ffb700 1 -- 172.21.15.88:0/3240964781 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fb2e805a5d0 con 0x7fb2ec0ff350 2024-02-19T19:09:36.697 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:09:36.697 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:09:36.697 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:09:36.697 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:09:36.697 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:09:36.697 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6, 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 8 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:09:36.698 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:09:36.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.678+0000 7fb2f2d24700 1 -- 172.21.15.88:0/3240964781 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb2d406d8a0 msgr2=0x7fb2d406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:36.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.678+0000 7fb2f2d24700 1 --2- 172.21.15.88:0/3240964781 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb2d406d8a0 0x7fb2d406fd50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc005f50 tx=0x7fb2dc01a040 comp rx=0 tx=0).stop 2024-02-19T19:09:36.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.678+0000 7fb2f2d24700 1 -- 172.21.15.88:0/3240964781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2ec0ff350 msgr2=0x7fb2ec19c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:36.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.678+0000 7fb2f2d24700 1 --2- 172.21.15.88:0/3240964781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2ec0ff350 0x7fb2ec19c410 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb2e800c930 tx=0x7fb2e800ccf0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.678+0000 7fb2f2d24700 1 -- 172.21.15.88:0/3240964781 shutdown_connections 2024-02-19T19:09:36.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.678+0000 7fb2f2d24700 1 --2- 172.21.15.88:0/3240964781 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb2d406d8a0 0x7fb2d406fd50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.678+0000 7fb2f2d24700 1 --2- 172.21.15.88:0/3240964781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2ec0ff350 0x7fb2ec19c410 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.678+0000 7fb2f2d24700 1 --2- 172.21.15.88:0/3240964781 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb2ec0ff720 0x7fb2ec19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.678+0000 7fb2f2d24700 1 -- 172.21.15.88:0/3240964781 >> 172.21.15.88:0/3240964781 conn(0x7fb2ec0748b0 msgr2=0x7fb2ec0fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:36.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.678+0000 7fb2f2d24700 1 -- 172.21.15.88:0/3240964781 shutdown_connections 2024-02-19T19:09:36.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.678+0000 7fb2f2d24700 1 -- 172.21.15.88:0/3240964781 wait complete. 2024-02-19T19:09:36.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.783+0000 7f057398c700 1 -- 172.21.15.88:0/3319979017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f056c068540 msgr2=0x7f056c0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:36.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.783+0000 7f057398c700 1 --2- 172.21.15.88:0/3319979017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f056c068540 0x7f056c0689b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f056000b0a0 tx=0x7f056000b3b0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.783+0000 7f057398c700 1 -- 172.21.15.88:0/3319979017 shutdown_connections 2024-02-19T19:09:36.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.783+0000 7f057398c700 1 --2- 172.21.15.88:0/3319979017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f056c068540 0x7f056c0689b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.783+0000 7f057398c700 1 --2- 172.21.15.88:0/3319979017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f056c106320 0x7f056c1066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.783+0000 7f057398c700 1 -- 172.21.15.88:0/3319979017 >> 172.21.15.88:0/3319979017 conn(0x7f056c075130 msgr2=0x7f056c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:36.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.783+0000 7f057398c700 1 -- 172.21.15.88:0/3319979017 shutdown_connections 2024-02-19T19:09:36.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.783+0000 7f057398c700 1 -- 172.21.15.88:0/3319979017 wait complete. 2024-02-19T19:09:36.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f057398c700 1 Processor -- start 2024-02-19T19:09:36.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f057398c700 1 -- start start 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f057398c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f056c068540 0x7f056c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f057398c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f056c106320 0x7f056c1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f057398c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c196a80 con 0x7f056c106320 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f057398c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c19a810 con 0x7f056c068540 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f0570f27700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f056c106320 0x7f056c1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f0570f27700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f056c106320 0x7f056c1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50094/0 (socket says 172.21.15.88:50094) 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f0570f27700 1 -- 172.21.15.88:0/1142026310 learned_addr learned my addr 172.21.15.88:0/1142026310 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f0571728700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f056c068540 0x7f056c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f0570f27700 1 -- 172.21.15.88:0/1142026310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f056c068540 msgr2=0x7f056c195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f0570f27700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f056c068540 0x7f056c195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f0570f27700 1 -- 172.21.15.88:0/1142026310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560009d00 con 0x7f056c106320 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.784+0000 7f0571728700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f056c068540 0x7f056c195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.785+0000 7f0570f27700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f056c106320 0x7f056c1963a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0560000f80 tx=0x7f0560003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.785+0000 7f056a7fc700 1 -- 172.21.15.88:0/1142026310 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f056000e040 con 0x7f056c106320 2024-02-19T19:09:36.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.785+0000 7f057398c700 1 -- 172.21.15.88:0/1142026310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c19aa90 con 0x7f056c106320 2024-02-19T19:09:36.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.785+0000 7f056a7fc700 1 -- 172.21.15.88:0/1142026310 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0560007c00 con 0x7f056c106320 2024-02-19T19:09:36.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.785+0000 7f056a7fc700 1 -- 172.21.15.88:0/1142026310 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f056001d6b0 con 0x7f056c106320 2024-02-19T19:09:36.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.785+0000 7f057398c700 1 -- 172.21.15.88:0/1142026310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c19af80 con 0x7f056c106320 2024-02-19T19:09:36.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.786+0000 7f057398c700 1 -- 172.21.15.88:0/1142026310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c108770 con 0x7f056c106320 2024-02-19T19:09:36.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.786+0000 7f056a7fc700 1 -- 172.21.15.88:0/1142026310 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0560004040 con 0x7f056c106320 2024-02-19T19:09:36.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.787+0000 7f056a7fc700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f055406d8a0 0x7f055406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:36.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.787+0000 7f056a7fc700 1 -- 172.21.15.88:0/1142026310 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f056008ec00 con 0x7f056c106320 2024-02-19T19:09:36.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.787+0000 7f0571728700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f055406d8a0 0x7f055406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:36.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.787+0000 7f0571728700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f055406d8a0 0x7f055406fd50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f055c005fd0 tx=0x7f055c005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:36.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:36.791+0000 7f056a7fc700 1 -- 172.21.15.88:0/1142026310 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0560094020 con 0x7f056c106320 2024-02-19T19:09:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.033+0000 7f057398c700 1 -- 172.21.15.88:0/1142026310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f056c19b260 con 0x7f056c106320 2024-02-19T19:09:37.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.033+0000 7f056a7fc700 1 -- 172.21.15.88:0/1142026310 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1892 (secure 0 0 0) 0x7f0560017780 con 0x7f056c106320 2024-02-19T19:09:37.055 INFO:teuthology.orchestra.run.smithi088.stdout:e20 2024-02-19T19:09:37.055 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:09:37.055 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:09:37.055 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:09:37.055 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:09:37.055 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:09:37.055 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:09:37.055 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 20 2024-02-19T19:09:37.055 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:05:14.913778+0000 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:09:37.056 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 export targets 1 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:09:37.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:09:37.058 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:09:37.058 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:09:37.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.036+0000 7f057398c700 1 -- 172.21.15.88:0/1142026310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f055406d8a0 msgr2=0x7f055406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:37.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.036+0000 7f057398c700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f055406d8a0 0x7f055406fd50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f055c005fd0 tx=0x7f055c005ee0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.036+0000 7f057398c700 1 -- 172.21.15.88:0/1142026310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f056c106320 msgr2=0x7f056c1963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:37.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.036+0000 7f057398c700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f056c106320 0x7f056c1963a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0560000f80 tx=0x7f0560003ce0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.036+0000 7f057398c700 1 -- 172.21.15.88:0/1142026310 shutdown_connections 2024-02-19T19:09:37.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.036+0000 7f057398c700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f055406d8a0 0x7f055406fd50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.036+0000 7f057398c700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f056c068540 0x7f056c195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.036+0000 7f057398c700 1 --2- 172.21.15.88:0/1142026310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f056c106320 0x7f056c1963a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.036+0000 7f057398c700 1 -- 172.21.15.88:0/1142026310 >> 172.21.15.88:0/1142026310 conn(0x7f056c075130 msgr2=0x7f056c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:37.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.036+0000 7f057398c700 1 -- 172.21.15.88:0/1142026310 shutdown_connections 2024-02-19T19:09:37.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.036+0000 7f057398c700 1 -- 172.21.15.88:0/1142026310 wait complete. 2024-02-19T19:09:37.059 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-02-19T19:09:37.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.141+0000 7efd5df1e700 1 -- 172.21.15.88:0/1493758612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd581004e0 msgr2=0x7efd581008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:37.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.141+0000 7efd5df1e700 1 --2- 172.21.15.88:0/1493758612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd581004e0 0x7efd581008b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7efd48009910 tx=0x7efd48009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:37.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.142+0000 7efd5df1e700 1 -- 172.21.15.88:0/1493758612 shutdown_connections 2024-02-19T19:09:37.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.142+0000 7efd5df1e700 1 --2- 172.21.15.88:0/1493758612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd58100e80 0x7efd58104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.142+0000 7efd5df1e700 1 --2- 172.21.15.88:0/1493758612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd581004e0 0x7efd581008b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.142+0000 7efd5df1e700 1 -- 172.21.15.88:0/1493758612 >> 172.21.15.88:0/1493758612 conn(0x7efd580fbda0 msgr2=0x7efd580fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:37.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.142+0000 7efd5df1e700 1 -- 172.21.15.88:0/1493758612 shutdown_connections 2024-02-19T19:09:37.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.142+0000 7efd5df1e700 1 -- 172.21.15.88:0/1493758612 wait complete. 2024-02-19T19:09:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.142+0000 7efd5df1e700 1 Processor -- start 2024-02-19T19:09:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.142+0000 7efd5df1e700 1 -- start start 2024-02-19T19:09:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd5df1e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd581004e0 0x7efd58195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd5df1e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd58100e80 0x7efd58196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:37.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd5df1e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd58196b10 con 0x7efd581004e0 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd5df1e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd5819a8a0 con 0x7efd58100e80 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd577fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd581004e0 0x7efd58195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd577fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd581004e0 0x7efd58195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50120/0 (socket says 172.21.15.88:50120) 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd577fe700 1 -- 172.21.15.88:0/1601619958 learned_addr learned my addr 172.21.15.88:0/1601619958 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd56ffd700 1 --2- 172.21.15.88:0/1601619958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd58100e80 0x7efd58196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd577fe700 1 -- 172.21.15.88:0/1601619958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd58100e80 msgr2=0x7efd58196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd577fe700 1 --2- 172.21.15.88:0/1601619958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd58100e80 0x7efd58196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd577fe700 1 -- 172.21.15.88:0/1601619958 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480095f0 con 0x7efd581004e0 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd577fe700 1 --2- 172.21.15.88:0/1601619958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd581004e0 0x7efd58195ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efd48006010 tx=0x7efd4800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.143+0000 7efd54ff9700 1 -- 172.21.15.88:0/1601619958 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd4801d070 con 0x7efd581004e0 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.144+0000 7efd54ff9700 1 -- 172.21.15.88:0/1601619958 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd48005dc0 con 0x7efd581004e0 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.144+0000 7efd5df1e700 1 -- 172.21.15.88:0/1601619958 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd5819ab20 con 0x7efd581004e0 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.144+0000 7efd54ff9700 1 -- 172.21.15.88:0/1601619958 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd48017400 con 0x7efd581004e0 2024-02-19T19:09:37.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.144+0000 7efd5df1e700 1 -- 172.21.15.88:0/1601619958 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5819b010 con 0x7efd581004e0 2024-02-19T19:09:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.144+0000 7efd5df1e700 1 -- 172.21.15.88:0/1601619958 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd581087f0 con 0x7efd581004e0 2024-02-19T19:09:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.145+0000 7efd54ff9700 1 -- 172.21.15.88:0/1601619958 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7efd48003a60 con 0x7efd581004e0 2024-02-19T19:09:37.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.145+0000 7efd54ff9700 1 --2- 172.21.15.88:0/1601619958 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd4006d8a0 0x7efd4006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:37.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.145+0000 7efd54ff9700 1 -- 172.21.15.88:0/1601619958 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efd4808e850 con 0x7efd581004e0 2024-02-19T19:09:37.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.146+0000 7efd56ffd700 1 --2- 172.21.15.88:0/1601619958 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd4006d8a0 0x7efd4006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:37.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.146+0000 7efd56ffd700 1 --2- 172.21.15.88:0/1601619958 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd4006d8a0 0x7efd4006fd50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd58197510 tx=0x7efd4c00b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:37.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.150+0000 7efd54ff9700 1 -- 172.21.15.88:0/1601619958 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd48058130 con 0x7efd581004e0 2024-02-19T19:09:37.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:37 smithi088 ceph-mon[88643]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:37.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:37 smithi088 ceph-mon[88643]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:37.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:37 smithi088 ceph-mon[88643]: pgmap v165: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:37.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:37 smithi088 ceph-mon[88643]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:37.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:37 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3240964781' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:37.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:37 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1142026310' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:09:37.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.382+0000 7efd5df1e700 1 -- 172.21.15.88:0/1601619958 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd5819b340 con 0x7efd4006d8a0 2024-02-19T19:09:37.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.383+0000 7efd54ff9700 1 -- 172.21.15.88:0/1601619958 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7efd5819b340 con 0x7efd4006d8a0 2024-02-19T19:09:37.405 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:09:37.405 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:09:37.405 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:09:37.405 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:09:37.405 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:09:37.405 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:09:37.405 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:09:37.405 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:09:37.405 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:09:37.405 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "10/23 daemons upgraded", 2024-02-19T19:09:37.405 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading osd daemons", 2024-02-19T19:09:37.406 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:09:37.406 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:09:37.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.385+0000 7efd5df1e700 1 -- 172.21.15.88:0/1601619958 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd4006d8a0 msgr2=0x7efd4006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:37.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.385+0000 7efd5df1e700 1 --2- 172.21.15.88:0/1601619958 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd4006d8a0 0x7efd4006fd50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd58197510 tx=0x7efd4c00b410 comp rx=0 tx=0).stop 2024-02-19T19:09:37.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.385+0000 7efd5df1e700 1 -- 172.21.15.88:0/1601619958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd581004e0 msgr2=0x7efd58195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:37.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.385+0000 7efd5df1e700 1 --2- 172.21.15.88:0/1601619958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd581004e0 0x7efd58195ef0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efd48006010 tx=0x7efd4800bd70 comp rx=0 tx=0).stop 2024-02-19T19:09:37.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.386+0000 7efd5df1e700 1 -- 172.21.15.88:0/1601619958 shutdown_connections 2024-02-19T19:09:37.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.386+0000 7efd5df1e700 1 --2- 172.21.15.88:0/1601619958 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd4006d8a0 0x7efd4006fd50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.386+0000 7efd5df1e700 1 --2- 172.21.15.88:0/1601619958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd581004e0 0x7efd58195ef0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.386+0000 7efd5df1e700 1 --2- 172.21.15.88:0/1601619958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd58100e80 0x7efd58196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.386+0000 7efd5df1e700 1 -- 172.21.15.88:0/1601619958 >> 172.21.15.88:0/1601619958 conn(0x7efd580fbda0 msgr2=0x7efd580fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:37.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.386+0000 7efd5df1e700 1 -- 172.21.15.88:0/1601619958 shutdown_connections 2024-02-19T19:09:37.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.386+0000 7efd5df1e700 1 -- 172.21.15.88:0/1601619958 wait complete. 2024-02-19T19:09:37.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:37 smithi132 ceph-mon[70676]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:37.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:37 smithi132 ceph-mon[70676]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:37.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:37 smithi132 ceph-mon[70676]: pgmap v165: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:37.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:37 smithi132 ceph-mon[70676]: from='client.34368 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:37.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:37 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3240964781' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:37.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:37 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1142026310' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:09:37.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.489+0000 7f260a940700 1 -- 172.21.15.88:0/1692613626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26040ffd60 msgr2=0x7f2604103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:37.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.489+0000 7f260a940700 1 --2- 172.21.15.88:0/1692613626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26040ffd60 0x7f2604103d30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009910 tx=0x7f25f8009c20 comp rx=0 tx=0).stop 2024-02-19T19:09:37.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.489+0000 7f260a940700 1 -- 172.21.15.88:0/1692613626 shutdown_connections 2024-02-19T19:09:37.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.489+0000 7f260a940700 1 --2- 172.21.15.88:0/1692613626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26040ffd60 0x7f2604103d30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.489+0000 7f260a940700 1 --2- 172.21.15.88:0/1692613626 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040ff3c0 0x7f26040ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.489+0000 7f260a940700 1 -- 172.21.15.88:0/1692613626 >> 172.21.15.88:0/1692613626 conn(0x7f26040fac80 msgr2=0x7f26040fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:37.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.489+0000 7f260a940700 1 -- 172.21.15.88:0/1692613626 shutdown_connections 2024-02-19T19:09:37.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.489+0000 7f260a940700 1 -- 172.21.15.88:0/1692613626 wait complete. 2024-02-19T19:09:37.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.490+0000 7f260a940700 1 Processor -- start 2024-02-19T19:09:37.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.490+0000 7f260a940700 1 -- start start 2024-02-19T19:09:37.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.490+0000 7f260a940700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26040ff3c0 0x7f2604195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:37.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.490+0000 7f260a940700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040ffd60 0x7f26041963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:37.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.490+0000 7f260a940700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2604196ab0 con 0x7f26040ff3c0 2024-02-19T19:09:37.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.490+0000 7f260a940700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f260419a840 con 0x7f26040ffd60 2024-02-19T19:09:37.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.490+0000 7f2603fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26040ff3c0 0x7f2604195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:37.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.490+0000 7f2603fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26040ff3c0 0x7f2604195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50142/0 (socket says 172.21.15.88:50142) 2024-02-19T19:09:37.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.490+0000 7f2603fff700 1 -- 172.21.15.88:0/1389333796 learned_addr learned my addr 172.21.15.88:0/1389333796 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:09:37.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.491+0000 7f26037fe700 1 --2- 172.21.15.88:0/1389333796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040ffd60 0x7f26041963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:37.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.491+0000 7f2603fff700 1 -- 172.21.15.88:0/1389333796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040ffd60 msgr2=0x7f26041963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:37.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.491+0000 7f2603fff700 1 --2- 172.21.15.88:0/1389333796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040ffd60 0x7f26041963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.491+0000 7f2603fff700 1 -- 172.21.15.88:0/1389333796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25f80095f0 con 0x7f26040ff3c0 2024-02-19T19:09:37.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.491+0000 7f2603fff700 1 --2- 172.21.15.88:0/1389333796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26040ff3c0 0x7f2604195e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f25f400d900 tx=0x7f25f400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:37.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.491+0000 7f26017fa700 1 -- 172.21.15.88:0/1389333796 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25f40041f0 con 0x7f26040ff3c0 2024-02-19T19:09:37.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.491+0000 7f260a940700 1 -- 172.21.15.88:0/1389333796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f260419ab20 con 0x7f26040ff3c0 2024-02-19T19:09:37.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.491+0000 7f26017fa700 1 -- 172.21.15.88:0/1389333796 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25f4004cf0 con 0x7f26040ff3c0 2024-02-19T19:09:37.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.491+0000 7f26017fa700 1 -- 172.21.15.88:0/1389333796 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25f400b750 con 0x7f26040ff3c0 2024-02-19T19:09:37.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.491+0000 7f260a940700 1 -- 172.21.15.88:0/1389333796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f260419b070 con 0x7f26040ff3c0 2024-02-19T19:09:37.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.492+0000 7f260a940700 1 -- 172.21.15.88:0/1389333796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2604108740 con 0x7f26040ff3c0 2024-02-19T19:09:37.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.493+0000 7f26017fa700 1 -- 172.21.15.88:0/1389333796 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f25f4004750 con 0x7f26040ff3c0 2024-02-19T19:09:37.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.493+0000 7f26017fa700 1 --2- 172.21.15.88:0/1389333796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25ec06d8a0 0x7f25ec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:09:37.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.493+0000 7f26037fe700 1 --2- 172.21.15.88:0/1389333796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25ec06d8a0 0x7f25ec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:09:37.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.493+0000 7f26017fa700 1 -- 172.21.15.88:0/1389333796 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f25f408d330 con 0x7f26040ff3c0 2024-02-19T19:09:37.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.494+0000 7f26037fe700 1 --2- 172.21.15.88:0/1389333796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25ec06d8a0 0x7f25ec06fd50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f26041974b0 tx=0x7f25f801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:09:37.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.498+0000 7f26017fa700 1 -- 172.21.15.88:0/1389333796 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f25f4056b90 con 0x7f26040ff3c0 2024-02-19T19:09:37.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.780+0000 7f260a940700 1 -- 172.21.15.88:0/1389333796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f260404f2a0 con 0x7f26040ff3c0 2024-02-19T19:09:37.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.781+0000 7f26017fa700 1 -- 172.21.15.88:0/1389333796 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f25f405a250 con 0x7f26040ff3c0 2024-02-19T19:09:37.802 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_OK 2024-02-19T19:09:37.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.783+0000 7f260a940700 1 -- 172.21.15.88:0/1389333796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25ec06d8a0 msgr2=0x7f25ec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:37.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.783+0000 7f260a940700 1 --2- 172.21.15.88:0/1389333796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25ec06d8a0 0x7f25ec06fd50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f26041974b0 tx=0x7f25f801a040 comp rx=0 tx=0).stop 2024-02-19T19:09:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.783+0000 7f260a940700 1 -- 172.21.15.88:0/1389333796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26040ff3c0 msgr2=0x7f2604195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:09:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.783+0000 7f260a940700 1 --2- 172.21.15.88:0/1389333796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26040ff3c0 0x7f2604195e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f25f400d900 tx=0x7f25f400dc10 comp rx=0 tx=0).stop 2024-02-19T19:09:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.783+0000 7f260a940700 1 -- 172.21.15.88:0/1389333796 shutdown_connections 2024-02-19T19:09:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.783+0000 7f260a940700 1 --2- 172.21.15.88:0/1389333796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25ec06d8a0 0x7f25ec06fd50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.783+0000 7f260a940700 1 --2- 172.21.15.88:0/1389333796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26040ff3c0 0x7f2604195e90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.783+0000 7f260a940700 1 --2- 172.21.15.88:0/1389333796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f26040ffd60 0x7f26041963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:09:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.783+0000 7f260a940700 1 -- 172.21.15.88:0/1389333796 >> 172.21.15.88:0/1389333796 conn(0x7f26040fac80 msgr2=0x7f26040690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:09:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.783+0000 7f260a940700 1 -- 172.21.15.88:0/1389333796 shutdown_connections 2024-02-19T19:09:37.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:09:37.783+0000 7f260a940700 1 -- 172.21.15.88:0/1389333796 wait complete. 2024-02-19T19:09:38.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:38 smithi132 ceph-mon[70676]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:38.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:38 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1389333796' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:09:38.484 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:38 smithi132 systemd[1]: Stopping Ceph osd.4 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:09:38.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:38 smithi088 ceph-mon[88643]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:09:38.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:38 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1389333796' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:09:38.754 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:38 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[48560]: 2024-02-19T19:09:38.460+0000 7f20c6ed1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T19:09:38.754 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:38 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[48560]: 2024-02-19T19:09:38.460+0000 7f20c6ed1700 -1 osd.4 72 *** Got signal Terminated *** 2024-02-19T19:09:38.754 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:38 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[48560]: 2024-02-19T19:09:38.460+0000 7f20c6ed1700 -1 osd.4 72 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T19:09:39.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:39 smithi088 ceph-mon[88643]: pgmap v166: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:39.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:39 smithi088 ceph-mon[88643]: osd.4 marked itself down and dead 2024-02-19T19:09:39.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:39 smithi132 ceph-mon[70676]: pgmap v166: 65 pgs: 65 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:39.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:39 smithi132 ceph-mon[70676]: osd.4 marked itself down and dead 2024-02-19T19:09:40.004 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:39 smithi132 podman[85619]: 2024-02-19 19:09:39.728120012 +0000 UTC m=+1.369876120 container died f4bfc92e5c9368378986f1bf5b17af6395c5eeac0b79a628fd431a2dd6e3c67c (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, RELEASE=HEAD) 2024-02-19T19:09:40.384 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:40 smithi132 ceph-mon[70676]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:09:40.384 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:40 smithi132 ceph-mon[70676]: osdmap e73: 6 total, 5 up, 6 in 2024-02-19T19:09:40.384 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:40 smithi132 podman[85619]: 2024-02-19 19:09:40.095311797 +0000 UTC m=+1.737067941 container cleanup f4bfc92e5c9368378986f1bf5b17af6395c5eeac0b79a628fd431a2dd6e3c67c (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:09:40.384 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:40 smithi132 bash[85619]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4 2024-02-19T19:09:40.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:40 smithi088 ceph-mon[88643]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:09:40.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:40 smithi088 ceph-mon[88643]: osdmap e73: 6 total, 5 up, 6 in 2024-02-19T19:09:40.734 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:40 smithi132 podman[85637]: 2024-02-19 19:09:40.363205635 +0000 UTC m=+0.629867931 container remove f4bfc92e5c9368378986f1bf5b17af6395c5eeac0b79a628fd431a2dd6e3c67c (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, ceph=True, org.label-schema.vendor=CentOS) 2024-02-19T19:09:41.004 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:40 smithi132 podman[85728]: 2024-02-19 19:09:40.616960355 +0000 UTC m=+0.017856272 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:09:41.004 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:40 smithi132 podman[85728]: 2024-02-19 19:09:40.739295851 +0000 UTC m=+0.140191759 container create d9ee4d94d4ac9e3dc8cb6d0de08f736fe1c53cc990e1394a48f3e7b80f6535a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-02-19T19:09:41.004 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:40 smithi132 podman[85728]: 2024-02-19 19:09:40.941744071 +0000 UTC m=+0.342639983 container init d9ee4d94d4ac9e3dc8cb6d0de08f736fe1c53cc990e1394a48f3e7b80f6535a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240214, ceph=True) 2024-02-19T19:09:41.005 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:40 smithi132 podman[85728]: 2024-02-19 19:09:40.944746825 +0000 UTC m=+0.345642734 container start d9ee4d94d4ac9e3dc8cb6d0de08f736fe1c53cc990e1394a48f3e7b80f6535a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-19T19:09:41.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:41 smithi132 ceph-mon[70676]: pgmap v168: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:41.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:41 smithi132 ceph-mon[70676]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-19T19:09:41.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:41 smithi132 ceph-mon[70676]: osdmap e74: 6 total, 5 up, 6 in 2024-02-19T19:09:41.504 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:41 smithi132 podman[85728]: 2024-02-19 19:09:41.041858208 +0000 UTC m=+0.442754128 container attach d9ee4d94d4ac9e3dc8cb6d0de08f736fe1c53cc990e1394a48f3e7b80f6535a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T19:09:41.504 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:41 smithi132 podman[85728]: 2024-02-19 19:09:41.231622763 +0000 UTC m=+0.632518666 container died d9ee4d94d4ac9e3dc8cb6d0de08f736fe1c53cc990e1394a48f3e7b80f6535a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-19T19:09:41.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:41 smithi088 ceph-mon[88643]: pgmap v168: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:41.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:41 smithi088 ceph-mon[88643]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-19T19:09:41.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:41 smithi088 ceph-mon[88643]: osdmap e74: 6 total, 5 up, 6 in 2024-02-19T19:09:42.254 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:41 smithi132 podman[85754]: 2024-02-19 19:09:41.779067389 +0000 UTC m=+0.542184202 container remove d9ee4d94d4ac9e3dc8cb6d0de08f736fe1c53cc990e1394a48f3e7b80f6535a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-19T19:09:42.254 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:41 smithi132 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.4.service: Deactivated successfully. 2024-02-19T19:09:42.254 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:41 smithi132 systemd[1]: Stopped Ceph osd.4 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:09:42.254 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:41 smithi132 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.4.service: Consumed 1min 3.658s CPU time. 2024-02-19T19:09:42.254 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 systemd[1]: Starting Ceph osd.4 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:09:42.578 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:42 smithi132 ceph-mon[70676]: Health check failed: Degraded data redundancy: 38/333 objects degraded (11.411%), 10 pgs degraded (PG_DEGRADED) 2024-02-19T19:09:42.579 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 podman[85858]: 2024-02-19 19:09:42.139815072 +0000 UTC m=+0.018569462 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:09:42.579 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 podman[85858]: 2024-02-19 19:09:42.259774168 +0000 UTC m=+0.138528543 container create a8fd12538ebed239f9ef1f29e04acca48e7fc019d5ecdb48317a165ec098358a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T19:09:42.579 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 podman[85858]: 2024-02-19 19:09:42.466450516 +0000 UTC m=+0.345204902 container init a8fd12538ebed239f9ef1f29e04acca48e7fc019d5ecdb48317a165ec098358a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-02-19T19:09:42.579 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 podman[85858]: 2024-02-19 19:09:42.469915516 +0000 UTC m=+0.348669894 container start a8fd12538ebed239f9ef1f29e04acca48e7fc019d5ecdb48317a165ec098358a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-19T19:09:42.579 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 podman[85858]: 2024-02-19 19:09:42.557791167 +0000 UTC m=+0.436545556 container attach a8fd12538ebed239f9ef1f29e04acca48e7fc019d5ecdb48317a165ec098358a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:09:42.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:42 smithi088 ceph-mon[88643]: Health check failed: Degraded data redundancy: 38/333 objects degraded (11.411%), 10 pgs degraded (PG_DEGRADED) 2024-02-19T19:09:43.254 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate[85873]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-19T19:09:43.254 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate[85873]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-19T19:09:43.254 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 bash[85858]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 bash[85858]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 bash[85858]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 bash[85858]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 bash[85858]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 bash[85858]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 bash[85858]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate[85873]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate[85873]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate[85873]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate[85873]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate[85873]: --> ceph-volume raw activate successful for osd ID: 4 2024-02-19T19:09:43.255 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:42 smithi132 podman[85858]: 2024-02-19 19:09:42.935581662 +0000 UTC m=+0.814336035 container died a8fd12538ebed239f9ef1f29e04acca48e7fc019d5ecdb48317a165ec098358a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-02-19T19:09:43.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:43 smithi088 ceph-mon[88643]: pgmap v170: 65 pgs: 10 active+undersized+degraded, 8 peering, 1 stale+active+clean, 11 active+undersized, 35 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 38/333 objects degraded (11.411%) 2024-02-19T19:09:43.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:43.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:09:43.607 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:43 smithi132 ceph-mon[70676]: pgmap v170: 65 pgs: 10 active+undersized+degraded, 8 peering, 1 stale+active+clean, 11 active+undersized, 35 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 38/333 objects degraded (11.411%) 2024-02-19T19:09:43.608 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:43.608 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:09:43.957 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:43 smithi132 podman[85998]: 2024-02-19 19:09:43.586945303 +0000 UTC m=+0.646498215 container remove a8fd12538ebed239f9ef1f29e04acca48e7fc019d5ecdb48317a165ec098358a (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=) 2024-02-19T19:09:43.957 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:43 smithi132 podman[86055]: 2024-02-19 19:09:43.702419418 +0000 UTC m=+0.017899104 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:09:43.957 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:43 smithi132 podman[86055]: 2024-02-19 19:09:43.829928137 +0000 UTC m=+0.145407825 container create c7a967e24a5519df7b41a855412d80c4cb0f47d21fca7d98ee3ca05bc43f1915 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214) 2024-02-19T19:09:44.504 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:44 smithi132 podman[86055]: 2024-02-19 19:09:44.040823727 +0000 UTC m=+0.356303416 container init c7a967e24a5519df7b41a855412d80c4cb0f47d21fca7d98ee3ca05bc43f1915 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-19T19:09:44.504 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:44 smithi132 podman[86055]: 2024-02-19 19:09:44.044197509 +0000 UTC m=+0.359677196 container start c7a967e24a5519df7b41a855412d80c4cb0f47d21fca7d98ee3ca05bc43f1915 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-19T19:09:44.504 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:44 smithi132 ceph-osd[86074]: -- 172.21.15.132:0/4144371239 <== mon.1 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5646f248b1e0 con 0x5646f245cc00 2024-02-19T19:09:44.505 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:44 smithi132 bash[86055]: c7a967e24a5519df7b41a855412d80c4cb0f47d21fca7d98ee3ca05bc43f1915 2024-02-19T19:09:44.505 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:44 smithi132 systemd[1]: Started Ceph osd.4 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:09:45.004 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:44 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T19:09:44.592+0000 7fb5647757c0 -1 Falling back to public interface 2024-02-19T19:09:45.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:45 smithi132 ceph-mon[70676]: pgmap v171: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-02-19T19:09:45.290 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:45.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:45.291 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:09:45.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:45 smithi088 ceph-mon[88643]: pgmap v171: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 43/333 objects degraded (12.913%) 2024-02-19T19:09:45.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:45.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:45.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:09:46.592 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:46 smithi132 ceph-mon[70676]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-19T19:09:46.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:46 smithi088 ceph-mon[88643]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-19T19:09:47.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:47 smithi088 ceph-mon[88643]: pgmap v172: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/333 objects degraded (16.216%) 2024-02-19T19:09:47.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:47.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:47.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:47 smithi132 ceph-mon[70676]: pgmap v172: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/333 objects degraded (16.216%) 2024-02-19T19:09:47.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:47.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:49.453 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:49 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T19:09:49.125+0000 7fb5647757c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-02-19T19:09:49.453 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:49 smithi132 ceph-mon[70676]: pgmap v173: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/333 objects degraded (16.216%) 2024-02-19T19:09:49.453 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:49 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:49.453 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:49 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:49.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:49 smithi088 ceph-mon[88643]: pgmap v173: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/333 objects degraded (16.216%) 2024-02-19T19:09:49.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:49 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:49.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:49 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:50.004 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:49 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T19:09:49.700+0000 7fb5647757c0 -1 osd.4 72 log_to_monitors true 2024-02-19T19:09:50.727 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:50 smithi132 ceph-mon[70676]: from='osd.4 [v2:172.21.15.132:6808/682905746,v1:172.21.15.132:6809/682905746]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T19:09:50.728 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:50 smithi132 ceph-mon[70676]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T19:09:50.728 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:50 smithi132 ceph-mon[70676]: Health check update: Degraded data redundancy: 54/333 objects degraded (16.216%), 16 pgs degraded (PG_DEGRADED) 2024-02-19T19:09:50.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:50 smithi088 ceph-mon[88643]: from='osd.4 [v2:172.21.15.132:6808/682905746,v1:172.21.15.132:6809/682905746]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T19:09:50.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:50 smithi088 ceph-mon[88643]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-19T19:09:50.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:50 smithi088 ceph-mon[88643]: Health check update: Degraded data redundancy: 54/333 objects degraded (16.216%), 16 pgs degraded (PG_DEGRADED) 2024-02-19T19:09:51.332 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 19:09:51 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T19:09:51.204+0000 7fb551bc5700 -1 osd.4 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T19:09:51.607 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:51 smithi132 ceph-mon[70676]: pgmap v174: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/333 objects degraded (16.216%) 2024-02-19T19:09:51.607 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:51 smithi132 ceph-mon[70676]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-19T19:09:51.607 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:51 smithi132 ceph-mon[70676]: osdmap e75: 6 total, 5 up, 6 in 2024-02-19T19:09:51.608 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:51 smithi132 ceph-mon[70676]: from='osd.4 [v2:172.21.15.132:6808/682905746,v1:172.21.15.132:6809/682905746]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:09:51.608 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:51 smithi132 ceph-mon[70676]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:09:51.608 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:51 smithi132 ceph-mon[70676]: from='osd.4 ' entity='osd.4' 2024-02-19T19:09:51.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:51 smithi088 ceph-mon[88643]: pgmap v174: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 54/333 objects degraded (16.216%) 2024-02-19T19:09:51.869 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:51 smithi088 ceph-mon[88643]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-19T19:09:51.869 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:51 smithi088 ceph-mon[88643]: osdmap e75: 6 total, 5 up, 6 in 2024-02-19T19:09:51.869 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:51 smithi088 ceph-mon[88643]: from='osd.4 [v2:172.21.15.132:6808/682905746,v1:172.21.15.132:6809/682905746]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:09:51.869 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:51 smithi088 ceph-mon[88643]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:09:51.869 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:51 smithi088 ceph-mon[88643]: from='osd.4 ' entity='osd.4' 2024-02-19T19:09:52.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:52 smithi132 ceph-mon[70676]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:09:52.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:52 smithi132 ceph-mon[70676]: osd.4 [v2:172.21.15.132:6808/682905746,v1:172.21.15.132:6809/682905746] boot 2024-02-19T19:09:52.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:52 smithi132 ceph-mon[70676]: osdmap e76: 6 total, 6 up, 6 in 2024-02-19T19:09:52.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T19:09:52.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:52 smithi088 ceph-mon[88643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:09:52.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:52 smithi088 ceph-mon[88643]: osd.4 [v2:172.21.15.132:6808/682905746,v1:172.21.15.132:6809/682905746] boot 2024-02-19T19:09:52.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:52 smithi088 ceph-mon[88643]: osdmap e76: 6 total, 6 up, 6 in 2024-02-19T19:09:52.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-19T19:09:53.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: pgmap v176: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 54/333 objects degraded (16.216%) 2024-02-19T19:09:53.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:53.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:53.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:09:53.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:09:53.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:53.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:09:53.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:53.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:53.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:53.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-19T19:09:53.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:53 smithi132 ceph-mon[70676]: osdmap e77: 6 total, 6 up, 6 in 2024-02-19T19:09:53.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: pgmap v176: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 54/333 objects degraded (16.216%) 2024-02-19T19:09:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:09:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:09:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:09:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:09:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-19T19:09:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:53 smithi088 ceph-mon[88643]: osdmap e77: 6 total, 6 up, 6 in 2024-02-19T19:09:54.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:54 smithi088 ceph-mon[88643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-19T19:09:54.714 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:54 smithi088 ceph-mon[88643]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-19T19:09:54.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:54 smithi132 ceph-mon[70676]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-19T19:09:54.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:54 smithi132 ceph-mon[70676]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-02-19T19:09:55.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:55 smithi132 ceph-mon[70676]: pgmap v179: 65 pgs: 15 active+undersized+degraded, 15 active+undersized, 35 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-02-19T19:09:55.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:55 smithi132 ceph-mon[70676]: Health check update: Degraded data redundancy: 51/333 objects degraded (15.315%), 15 pgs degraded (PG_DEGRADED) 2024-02-19T19:09:55.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:55 smithi088 ceph-mon[88643]: pgmap v179: 65 pgs: 15 active+undersized+degraded, 15 active+undersized, 35 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-02-19T19:09:55.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:55 smithi088 ceph-mon[88643]: Health check update: Degraded data redundancy: 51/333 objects degraded (15.315%), 15 pgs degraded (PG_DEGRADED) 2024-02-19T19:09:57.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:57 smithi132 ceph-mon[70676]: pgmap v180: 65 pgs: 8 active+undersized+degraded, 8 active+undersized, 49 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 34/333 objects degraded (10.210%) 2024-02-19T19:09:57.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:57 smithi088 ceph-mon[88643]: pgmap v180: 65 pgs: 8 active+undersized+degraded, 8 active+undersized, 49 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 34/333 objects degraded (10.210%) 2024-02-19T19:09:59.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:59 smithi132 ceph-mon[70676]: pgmap v181: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:59.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:59.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:09:59.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:59 smithi132 ceph-mon[70676]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/333 objects degraded (10.210%), 8 pgs degraded) 2024-02-19T19:09:59.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:09:59 smithi132 ceph-mon[70676]: Cluster is now healthy 2024-02-19T19:09:59.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:59 smithi088 ceph-mon[88643]: pgmap v181: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:09:59.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:09:59.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:09:59.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:59 smithi088 ceph-mon[88643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/333 objects degraded (10.210%), 8 pgs degraded) 2024-02-19T19:09:59.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:09:59 smithi088 ceph-mon[88643]: Cluster is now healthy 2024-02-19T19:10:00.498 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:00 smithi132 ceph-mon[70676]: overall HEALTH_OK 2024-02-19T19:10:00.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:00 smithi088 ceph-mon[88643]: overall HEALTH_OK 2024-02-19T19:10:01.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:01 smithi132 ceph-mon[70676]: pgmap v182: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:01.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:01 smithi088 ceph-mon[88643]: pgmap v182: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:03.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:03 smithi132 ceph-mon[70676]: pgmap v183: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:03.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:03 smithi088 ceph-mon[88643]: pgmap v183: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:04.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:04.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:05.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:05 smithi088 ceph-mon[88643]: pgmap v184: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:05.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:05 smithi132 ceph-mon[70676]: pgmap v184: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:06.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:06 smithi088 ceph-mon[88643]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:10:06.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:06 smithi132 ceph-mon[70676]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:10:07.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:07 smithi088 ceph-mon[88643]: pgmap v185: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:07.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:07 smithi132 ceph-mon[70676]: pgmap v185: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:07.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.899+0000 7f9b21e9a700 1 -- 172.21.15.88:0/2143628658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b1c100e80 msgr2=0x7f9b1c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:07.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.899+0000 7f9b21e9a700 1 --2- 172.21.15.88:0/2143628658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b1c100e80 0x7f9b1c104e50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c009910 tx=0x7f9b0c009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:07.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.899+0000 7f9b21e9a700 1 -- 172.21.15.88:0/2143628658 shutdown_connections 2024-02-19T19:10:07.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.899+0000 7f9b21e9a700 1 --2- 172.21.15.88:0/2143628658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b1c100e80 0x7f9b1c104e50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:07.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.899+0000 7f9b21e9a700 1 --2- 172.21.15.88:0/2143628658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b1c1004e0 0x7f9b1c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:07.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.899+0000 7f9b21e9a700 1 -- 172.21.15.88:0/2143628658 >> 172.21.15.88:0/2143628658 conn(0x7f9b1c0fbda0 msgr2=0x7f9b1c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:07.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.899+0000 7f9b21e9a700 1 -- 172.21.15.88:0/2143628658 shutdown_connections 2024-02-19T19:10:07.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.899+0000 7f9b21e9a700 1 -- 172.21.15.88:0/2143628658 wait complete. 2024-02-19T19:10:07.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.900+0000 7f9b21e9a700 1 Processor -- start 2024-02-19T19:10:07.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.900+0000 7f9b21e9a700 1 -- start start 2024-02-19T19:10:07.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.900+0000 7f9b21e9a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b1c1004e0 0x7f9b1c195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:07.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.900+0000 7f9b21e9a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b1c100e80 0x7f9b1c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:07.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.900+0000 7f9b21e9a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c196aa0 con 0x7f9b1c100e80 2024-02-19T19:10:07.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.900+0000 7f9b21e9a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b1c196be0 con 0x7f9b1c1004e0 2024-02-19T19:10:07.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.900+0000 7f9b1affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b1c100e80 0x7f9b1c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:07.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.900+0000 7f9b1affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b1c100e80 0x7f9b1c196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38796/0 (socket says 172.21.15.88:38796) 2024-02-19T19:10:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.900+0000 7f9b1affd700 1 -- 172.21.15.88:0/512268204 learned_addr learned my addr 172.21.15.88:0/512268204 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.901+0000 7f9b1b7fe700 1 --2- 172.21.15.88:0/512268204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b1c1004e0 0x7f9b1c195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.901+0000 7f9b1affd700 1 -- 172.21.15.88:0/512268204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b1c1004e0 msgr2=0x7f9b1c195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.901+0000 7f9b1affd700 1 --2- 172.21.15.88:0/512268204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b1c1004e0 0x7f9b1c195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.901+0000 7f9b1affd700 1 -- 172.21.15.88:0/512268204 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b0c0095f0 con 0x7f9b1c100e80 2024-02-19T19:10:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.901+0000 7f9b1affd700 1 --2- 172.21.15.88:0/512268204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b1c100e80 0x7f9b1c196450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00bda0 tx=0x7f9b0c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.901+0000 7f9b18ff9700 1 -- 172.21.15.88:0/512268204 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b0c01d070 con 0x7f9b1c100e80 2024-02-19T19:10:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.901+0000 7f9b21e9a700 1 -- 172.21.15.88:0/512268204 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1c19a9d0 con 0x7f9b1c100e80 2024-02-19T19:10:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.901+0000 7f9b18ff9700 1 -- 172.21.15.88:0/512268204 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b0c00f460 con 0x7f9b1c100e80 2024-02-19T19:10:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.901+0000 7f9b18ff9700 1 -- 172.21.15.88:0/512268204 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b0c017630 con 0x7f9b1c100e80 2024-02-19T19:10:07.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.901+0000 7f9b21e9a700 1 -- 172.21.15.88:0/512268204 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b1c19aec0 con 0x7f9b1c100e80 2024-02-19T19:10:07.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.902+0000 7f9b21e9a700 1 -- 172.21.15.88:0/512268204 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b1c108850 con 0x7f9b1c100e80 2024-02-19T19:10:07.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.903+0000 7f9b18ff9700 1 -- 172.21.15.88:0/512268204 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9b0c00fab0 con 0x7f9b1c100e80 2024-02-19T19:10:07.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.903+0000 7f9b18ff9700 1 --2- 172.21.15.88:0/512268204 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b0406d8a0 0x7f9b0406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:07.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.903+0000 7f9b1b7fe700 1 --2- 172.21.15.88:0/512268204 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b0406d8a0 0x7f9b0406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:07.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.903+0000 7f9b18ff9700 1 -- 172.21.15.88:0/512268204 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f9b0c08e9a0 con 0x7f9b1c100e80 2024-02-19T19:10:07.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.904+0000 7f9b1b7fe700 1 --2- 172.21.15.88:0/512268204 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b0406d8a0 0x7f9b0406fd50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000ac50 tx=0x7f9b1000a380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:07.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:07.908+0000 7f9b18ff9700 1 -- 172.21.15.88:0/512268204 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b0c058200 con 0x7f9b1c100e80 2024-02-19T19:10:08.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.125+0000 7f9b21e9a700 1 -- 172.21.15.88:0/512268204 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b1c19b1a0 con 0x7f9b0406d8a0 2024-02-19T19:10:08.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.126+0000 7f9b18ff9700 1 -- 172.21.15.88:0/512268204 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9b1c19b1a0 con 0x7f9b0406d8a0 2024-02-19T19:10:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.128+0000 7f9b21e9a700 1 -- 172.21.15.88:0/512268204 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b0406d8a0 msgr2=0x7f9b0406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.128+0000 7f9b21e9a700 1 --2- 172.21.15.88:0/512268204 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b0406d8a0 0x7f9b0406fd50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9b1000ac50 tx=0x7f9b1000a380 comp rx=0 tx=0).stop 2024-02-19T19:10:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.128+0000 7f9b21e9a700 1 -- 172.21.15.88:0/512268204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b1c100e80 msgr2=0x7f9b1c196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.128+0000 7f9b21e9a700 1 --2- 172.21.15.88:0/512268204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b1c100e80 0x7f9b1c196450 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c00bda0 tx=0x7f9b0c00bdd0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.128+0000 7f9b21e9a700 1 -- 172.21.15.88:0/512268204 shutdown_connections 2024-02-19T19:10:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.128+0000 7f9b21e9a700 1 --2- 172.21.15.88:0/512268204 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b0406d8a0 0x7f9b0406fd50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.128+0000 7f9b21e9a700 1 --2- 172.21.15.88:0/512268204 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b1c1004e0 0x7f9b1c195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.128+0000 7f9b21e9a700 1 --2- 172.21.15.88:0/512268204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b1c100e80 0x7f9b1c196450 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.128+0000 7f9b21e9a700 1 -- 172.21.15.88:0/512268204 >> 172.21.15.88:0/512268204 conn(0x7f9b1c0fbda0 msgr2=0x7f9b1c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.128+0000 7f9b21e9a700 1 -- 172.21.15.88:0/512268204 shutdown_connections 2024-02-19T19:10:08.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.128+0000 7f9b21e9a700 1 -- 172.21.15.88:0/512268204 wait complete. 2024-02-19T19:10:08.160 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:10:08.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.234+0000 7f847a99b700 1 -- 172.21.15.88:0/1114986439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84740ff350 msgr2=0x7f84740ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.234+0000 7f847a99b700 1 --2- 172.21.15.88:0/1114986439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84740ff350 0x7f84740ff720 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8464009910 tx=0x7f8464009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:08.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.234+0000 7f847a99b700 1 -- 172.21.15.88:0/1114986439 shutdown_connections 2024-02-19T19:10:08.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.234+0000 7f847a99b700 1 --2- 172.21.15.88:0/1114986439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84740ffc60 0x7f847410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.234+0000 7f847a99b700 1 --2- 172.21.15.88:0/1114986439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84740ff350 0x7f84740ff720 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.234+0000 7f847a99b700 1 -- 172.21.15.88:0/1114986439 >> 172.21.15.88:0/1114986439 conn(0x7f84740748b0 msgr2=0x7f8474074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:08.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.234+0000 7f847a99b700 1 -- 172.21.15.88:0/1114986439 shutdown_connections 2024-02-19T19:10:08.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.234+0000 7f847a99b700 1 -- 172.21.15.88:0/1114986439 wait complete. 2024-02-19T19:10:08.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.235+0000 7f847a99b700 1 Processor -- start 2024-02-19T19:10:08.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.235+0000 7f847a99b700 1 -- start start 2024-02-19T19:10:08.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.235+0000 7f847a99b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84740ff350 0x7f8474195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:08.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.235+0000 7f847a99b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84740ffc60 0x7f8474196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:08.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.235+0000 7f847a99b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474196b60 con 0x7f84740ffc60 2024-02-19T19:10:08.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.235+0000 7f847a99b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847419a8f0 con 0x7f84740ff350 2024-02-19T19:10:08.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f84737fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84740ffc60 0x7f8474196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:08.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f8473fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84740ff350 0x7f8474195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:08.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f8473fff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84740ff350 0x7f8474195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:49636/0 (socket says 172.21.15.88:49636) 2024-02-19T19:10:08.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f8473fff700 1 -- 172.21.15.88:0/2613212286 learned_addr learned my addr 172.21.15.88:0/2613212286 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:08.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f84737fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84740ffc60 0x7f8474196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38824/0 (socket says 172.21.15.88:38824) 2024-02-19T19:10:08.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f84737fe700 1 -- 172.21.15.88:0/2613212286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84740ff350 msgr2=0x7f8474195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f84737fe700 1 --2- 172.21.15.88:0/2613212286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84740ff350 0x7f8474195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f84737fe700 1 -- 172.21.15.88:0/2613212286 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84640095f0 con 0x7f84740ffc60 2024-02-19T19:10:08.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f8473fff700 1 --2- 172.21.15.88:0/2613212286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84740ff350 0x7f8474195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:10:08.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f84737fe700 1 --2- 172.21.15.88:0/2613212286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84740ffc60 0x7f8474196480 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f846800b3d0 tx=0x7f846800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:08.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f84717fa700 1 -- 172.21.15.88:0/2613212286 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8468011800 con 0x7f84740ffc60 2024-02-19T19:10:08.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f847a99b700 1 -- 172.21.15.88:0/2613212286 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847419abd0 con 0x7f84740ffc60 2024-02-19T19:10:08.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f84717fa700 1 -- 172.21.15.88:0/2613212286 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f846800be90 con 0x7f84740ffc60 2024-02-19T19:10:08.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.236+0000 7f847a99b700 1 -- 172.21.15.88:0/2613212286 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847419b120 con 0x7f84740ffc60 2024-02-19T19:10:08.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.237+0000 7f84717fa700 1 -- 172.21.15.88:0/2613212286 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8468010e70 con 0x7f84740ffc60 2024-02-19T19:10:08.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.238+0000 7f847a99b700 1 -- 172.21.15.88:0/2613212286 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8460005320 con 0x7f84740ffc60 2024-02-19T19:10:08.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.238+0000 7f84717fa700 1 -- 172.21.15.88:0/2613212286 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8468011960 con 0x7f84740ffc60 2024-02-19T19:10:08.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.238+0000 7f84717fa700 1 --2- 172.21.15.88:0/2613212286 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f845c06d8a0 0x7f845c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:08.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.239+0000 7f84717fa700 1 -- 172.21.15.88:0/2613212286 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f846808ce10 con 0x7f84740ffc60 2024-02-19T19:10:08.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.239+0000 7f8473fff700 1 --2- 172.21.15.88:0/2613212286 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f845c06d8a0 0x7f845c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:08.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.239+0000 7f8473fff700 1 --2- 172.21.15.88:0/2613212286 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f845c06d8a0 0x7f845c06fd50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8464009fd0 tx=0x7f846400b770 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:08.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.243+0000 7f84717fa700 1 -- 172.21.15.88:0/2613212286 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8468056670 con 0x7f84740ffc60 2024-02-19T19:10:08.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.459+0000 7f847a99b700 1 -- 172.21.15.88:0/2613212286 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8460000bf0 con 0x7f845c06d8a0 2024-02-19T19:10:08.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.460+0000 7f84717fa700 1 -- 172.21.15.88:0/2613212286 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f8460000bf0 con 0x7f845c06d8a0 2024-02-19T19:10:08.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.463+0000 7f847a99b700 1 -- 172.21.15.88:0/2613212286 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f845c06d8a0 msgr2=0x7f845c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.463+0000 7f847a99b700 1 --2- 172.21.15.88:0/2613212286 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f845c06d8a0 0x7f845c06fd50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8464009fd0 tx=0x7f846400b770 comp rx=0 tx=0).stop 2024-02-19T19:10:08.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.463+0000 7f847a99b700 1 -- 172.21.15.88:0/2613212286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84740ffc60 msgr2=0x7f8474196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.463+0000 7f847a99b700 1 --2- 172.21.15.88:0/2613212286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84740ffc60 0x7f8474196480 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f846800b3d0 tx=0x7f846800b6e0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.463+0000 7f847a99b700 1 -- 172.21.15.88:0/2613212286 shutdown_connections 2024-02-19T19:10:08.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.463+0000 7f847a99b700 1 --2- 172.21.15.88:0/2613212286 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f845c06d8a0 0x7f845c06fd50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.463+0000 7f847a99b700 1 --2- 172.21.15.88:0/2613212286 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84740ff350 0x7f8474195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.463+0000 7f847a99b700 1 --2- 172.21.15.88:0/2613212286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84740ffc60 0x7f8474196480 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.463+0000 7f847a99b700 1 -- 172.21.15.88:0/2613212286 >> 172.21.15.88:0/2613212286 conn(0x7f84740748b0 msgr2=0x7f8474104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:08.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.463+0000 7f847a99b700 1 -- 172.21.15.88:0/2613212286 shutdown_connections 2024-02-19T19:10:08.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.463+0000 7f847a99b700 1 -- 172.21.15.88:0/2613212286 wait complete. 2024-02-19T19:10:08.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.567+0000 7f1f6e297700 1 -- 172.21.15.88:0/3799243844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f68100e80 msgr2=0x7f1f68104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.567+0000 7f1f6e297700 1 --2- 172.21.15.88:0/3799243844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f68100e80 0x7f1f68104e50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009910 tx=0x7f1f5c009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:08.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.567+0000 7f1f6e297700 1 -- 172.21.15.88:0/3799243844 shutdown_connections 2024-02-19T19:10:08.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.567+0000 7f1f6e297700 1 --2- 172.21.15.88:0/3799243844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f68100e80 0x7f1f68104e50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.567+0000 7f1f6e297700 1 --2- 172.21.15.88:0/3799243844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f681004e0 0x7f1f681008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.567+0000 7f1f6e297700 1 -- 172.21.15.88:0/3799243844 >> 172.21.15.88:0/3799243844 conn(0x7f1f680fbda0 msgr2=0x7f1f680fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:08.589 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-19T19:10:08.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.567+0000 7f1f6e297700 1 -- 172.21.15.88:0/3799243844 shutdown_connections 2024-02-19T19:10:08.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.567+0000 7f1f6e297700 1 -- 172.21.15.88:0/3799243844 wait complete. 2024-02-19T19:10:08.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.568+0000 7f1f6e297700 1 Processor -- start 2024-02-19T19:10:08.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.568+0000 7f1f6e297700 1 -- start start 2024-02-19T19:10:08.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.568+0000 7f1f6e297700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f681004e0 0x7f1f68195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.568+0000 7f1f6e297700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f68100e80 0x7f1f68196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.568+0000 7f1f6e297700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f68196b50 con 0x7f1f68100e80 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.568+0000 7f1f6e297700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6819a8e0 con 0x7f1f681004e0 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.568+0000 7f1f677fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f68100e80 0x7f1f68196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.568+0000 7f1f677fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f68100e80 0x7f1f68196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38834/0 (socket says 172.21.15.88:38834) 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.568+0000 7f1f677fe700 1 -- 172.21.15.88:0/1762429675 learned_addr learned my addr 172.21.15.88:0/1762429675 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.568+0000 7f1f67fff700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f681004e0 0x7f1f68195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.569+0000 7f1f677fe700 1 -- 172.21.15.88:0/1762429675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f681004e0 msgr2=0x7f1f68195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.569+0000 7f1f677fe700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f681004e0 0x7f1f68195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.569+0000 7f1f677fe700 1 -- 172.21.15.88:0/1762429675 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c0095f0 con 0x7f1f68100e80 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.569+0000 7f1f67fff700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f681004e0 0x7f1f68195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.569+0000 7f1f677fe700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f68100e80 0x7f1f68196470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0059d0 tx=0x7f1f5c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.569+0000 7f1f657fa700 1 -- 172.21.15.88:0/1762429675 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f5c01d070 con 0x7f1f68100e80 2024-02-19T19:10:08.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.569+0000 7f1f657fa700 1 -- 172.21.15.88:0/1762429675 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f5c00b610 con 0x7f1f68100e80 2024-02-19T19:10:08.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.569+0000 7f1f657fa700 1 -- 172.21.15.88:0/1762429675 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f5c022470 con 0x7f1f68100e80 2024-02-19T19:10:08.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.569+0000 7f1f6e297700 1 -- 172.21.15.88:0/1762429675 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6819ab60 con 0x7f1f68100e80 2024-02-19T19:10:08.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.569+0000 7f1f6e297700 1 -- 172.21.15.88:0/1762429675 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6819b050 con 0x7f1f68100e80 2024-02-19T19:10:08.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.570+0000 7f1f6e297700 1 -- 172.21.15.88:0/1762429675 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f68108850 con 0x7f1f68100e80 2024-02-19T19:10:08.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.571+0000 7f1f657fa700 1 -- 172.21.15.88:0/1762429675 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1f5c003710 con 0x7f1f68100e80 2024-02-19T19:10:08.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.571+0000 7f1f657fa700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1f5006d8a0 0x7f1f5006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:08.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.571+0000 7f1f657fa700 1 -- 172.21.15.88:0/1762429675 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1f5c08e950 con 0x7f1f68100e80 2024-02-19T19:10:08.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.571+0000 7f1f67fff700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1f5006d8a0 0x7f1f5006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:08.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.571+0000 7f1f67fff700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1f5006d8a0 0x7f1f5006fd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f58011490 tx=0x7f1f58009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.576+0000 7f1f657fa700 1 -- 172.21.15.88:0/1762429675 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1f5c0581b0 con 0x7f1f68100e80 2024-02-19T19:10:08.614 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-19T19:10:08.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.783+0000 7f1f6e297700 1 -- 172.21.15.88:0/1762429675 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f6819b270 con 0x7f1f5006d8a0 2024-02-19T19:10:08.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.793+0000 7f1f657fa700 1 -- 172.21.15.88:0/1762429675 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1f6819b270 con 0x7f1f5006d8a0 2024-02-19T19:10:08.815 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:10:08.815 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (7m) 89s ago 17m 24.2M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:10:08.815 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (18m) 89s ago 18m 8674k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:10:08.815 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (16m) 21s ago 16m 8526k - 18.2.0 dc2bc1663786 76f0e73bfa82 2024-02-19T19:10:08.815 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (3m) 89s ago 18m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:10:08.816 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (3m) 21s ago 16m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:10:08.816 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (6m) 89s ago 17m 85.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:10:08.816 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (11m) 89s ago 11m 17.0M - 18.2.0 dc2bc1663786 033db1f9fe36 2024-02-19T19:10:08.816 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (11m) 89s ago 11m 114M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:10:08.816 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (11m) 21s ago 11m 112M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:10:08.816 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (11m) 21s ago 11m 21.3M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:10:08.816 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (8m) 89s ago 19m 501M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:10:08.816 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (8m) 21s ago 16m 451M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:10:08.816 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (5m) 89s ago 19m 50.5M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:10:08.816 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (4m) 21s ago 15m 41.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:10:08.816 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (8m) 89s ago 18m 15.4M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:10:08.817 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (7m) 21s ago 16m 15.9M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:10:08.817 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (2m) 89s ago 15m 127M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:10:08.817 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (2m) 89s ago 14m 99.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:10:08.817 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (94s) 89s ago 14m 12.5M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:10:08.817 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (58s) 21s ago 13m 100M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:10:08.817 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (25s) 21s ago 13m 16.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:10:08.817 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (13m) 21s ago 13m 211M 2240M 18.2.0 dc2bc1663786 d93e850af992 2024-02-19T19:10:08.817 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (7m) 89s ago 16m 69.2M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:10:08.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.795+0000 7f1f6e297700 1 -- 172.21.15.88:0/1762429675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1f5006d8a0 msgr2=0x7f1f5006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.795+0000 7f1f6e297700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1f5006d8a0 0x7f1f5006fd50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1f58011490 tx=0x7f1f58009260 comp rx=0 tx=0).stop 2024-02-19T19:10:08.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.795+0000 7f1f6e297700 1 -- 172.21.15.88:0/1762429675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f68100e80 msgr2=0x7f1f68196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.795+0000 7f1f6e297700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f68100e80 0x7f1f68196470 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c0059d0 tx=0x7f1f5c00f690 comp rx=0 tx=0).stop 2024-02-19T19:10:08.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.795+0000 7f1f6e297700 1 -- 172.21.15.88:0/1762429675 shutdown_connections 2024-02-19T19:10:08.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.795+0000 7f1f6e297700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1f5006d8a0 0x7f1f5006fd50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.795+0000 7f1f6e297700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f681004e0 0x7f1f68195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.795+0000 7f1f6e297700 1 --2- 172.21.15.88:0/1762429675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f68100e80 0x7f1f68196470 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.795+0000 7f1f6e297700 1 -- 172.21.15.88:0/1762429675 >> 172.21.15.88:0/1762429675 conn(0x7f1f680fbda0 msgr2=0x7f1f680fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:08.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.795+0000 7f1f6e297700 1 -- 172.21.15.88:0/1762429675 shutdown_connections 2024-02-19T19:10:08.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.795+0000 7f1f6e297700 1 -- 172.21.15.88:0/1762429675 wait complete. 2024-02-19T19:10:08.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.900+0000 7f2ab08a0700 1 -- 172.21.15.88:0/768500986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aac0ff3c0 msgr2=0x7f2aac0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.900+0000 7f2ab08a0700 1 --2- 172.21.15.88:0/768500986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aac0ff3c0 0x7f2aac0ff790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c009910 tx=0x7f2a9c009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:08.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.900+0000 7f2ab08a0700 1 -- 172.21.15.88:0/768500986 shutdown_connections 2024-02-19T19:10:08.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.900+0000 7f2ab08a0700 1 --2- 172.21.15.88:0/768500986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2aac0ffd60 0x7f2aac103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.900+0000 7f2ab08a0700 1 --2- 172.21.15.88:0/768500986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aac0ff3c0 0x7f2aac0ff790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.901+0000 7f2ab08a0700 1 -- 172.21.15.88:0/768500986 >> 172.21.15.88:0/768500986 conn(0x7f2aac0fac80 msgr2=0x7f2aac0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:08.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.901+0000 7f2ab08a0700 1 -- 172.21.15.88:0/768500986 shutdown_connections 2024-02-19T19:10:08.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.901+0000 7f2ab08a0700 1 -- 172.21.15.88:0/768500986 wait complete. 2024-02-19T19:10:08.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.901+0000 7f2ab08a0700 1 Processor -- start 2024-02-19T19:10:08.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.901+0000 7f2ab08a0700 1 -- start start 2024-02-19T19:10:08.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.901+0000 7f2ab08a0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2aac0ff3c0 0x7f2aac195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:08.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2ab08a0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aac0ffd60 0x7f2aac196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:08.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2ab08a0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac196b20 con 0x7f2aac0ffd60 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2ab08a0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac19a8b0 con 0x7f2aac0ff3c0 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2aaa59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2aac0ff3c0 0x7f2aac195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2aa9d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aac0ffd60 0x7f2aac196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2aaa59c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2aac0ff3c0 0x7f2aac195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:49662/0 (socket says 172.21.15.88:49662) 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2aaa59c700 1 -- 172.21.15.88:0/1240552994 learned_addr learned my addr 172.21.15.88:0/1240552994 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2aaa59c700 1 -- 172.21.15.88:0/1240552994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aac0ffd60 msgr2=0x7f2aac196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2aaa59c700 1 --2- 172.21.15.88:0/1240552994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aac0ffd60 0x7f2aac196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2aaa59c700 1 -- 172.21.15.88:0/1240552994 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c0095f0 con 0x7f2aac0ff3c0 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2aa9d9b700 1 --2- 172.21.15.88:0/1240552994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aac0ffd60 0x7f2aac196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2aaa59c700 1 --2- 172.21.15.88:0/1240552994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2aac0ff3c0 0x7f2aac195f00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c006010 tx=0x7f2a9c00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2a9b7fe700 1 -- 172.21.15.88:0/1240552994 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a9c01d070 con 0x7f2aac0ff3c0 2024-02-19T19:10:08.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2ab08a0700 1 -- 172.21.15.88:0/1240552994 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aac19ab30 con 0x7f2aac0ff3c0 2024-02-19T19:10:08.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2a9b7fe700 1 -- 172.21.15.88:0/1240552994 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a9c005dc0 con 0x7f2aac0ff3c0 2024-02-19T19:10:08.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2a9b7fe700 1 -- 172.21.15.88:0/1240552994 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a9c017400 con 0x7f2aac0ff3c0 2024-02-19T19:10:08.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.902+0000 7f2ab08a0700 1 -- 172.21.15.88:0/1240552994 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac19b020 con 0x7f2aac0ff3c0 2024-02-19T19:10:08.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.903+0000 7f2ab08a0700 1 -- 172.21.15.88:0/1240552994 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aac04f2a0 con 0x7f2aac0ff3c0 2024-02-19T19:10:08.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.904+0000 7f2a9b7fe700 1 -- 172.21.15.88:0/1240552994 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2a9c003a60 con 0x7f2aac0ff3c0 2024-02-19T19:10:08.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.904+0000 7f2a9b7fe700 1 --2- 172.21.15.88:0/1240552994 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a9406d8a0 0x7f2a9406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:08.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.904+0000 7f2a9b7fe700 1 -- 172.21.15.88:0/1240552994 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2a9c08e9c0 con 0x7f2aac0ff3c0 2024-02-19T19:10:08.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.904+0000 7f2aa9d9b700 1 --2- 172.21.15.88:0/1240552994 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a9406d8a0 0x7f2a9406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:08.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.905+0000 7f2aa9d9b700 1 --2- 172.21.15.88:0/1240552994 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a9406d8a0 0x7f2a9406fd50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2aac197520 tx=0x7f2aa000b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:08.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:08.909+0000 7f2a9b7fe700 1 -- 172.21.15.88:0/1240552994 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a9c058180 con 0x7f2aac0ff3c0 2024-02-19T19:10:09.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.188+0000 7f2ab08a0700 1 -- 172.21.15.88:0/1240552994 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2aac19b240 con 0x7f2aac0ff3c0 2024-02-19T19:10:09.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.189+0000 7f2a9b7fe700 1 -- 172.21.15.88:0/1240552994 <== mon.1 v2:172.21.15.132:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f2a9c05b840 con 0x7f2aac0ff3c0 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 5 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:10:09.211 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:10:09.212 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-02-19T19:10:09.212 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:10:09.212 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:10:09.212 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-02-19T19:10:09.212 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 9 2024-02-19T19:10:09.212 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:10:09.212 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:10:09.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.192+0000 7f2ab08a0700 1 -- 172.21.15.88:0/1240552994 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a9406d8a0 msgr2=0x7f2a9406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:09.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.192+0000 7f2ab08a0700 1 --2- 172.21.15.88:0/1240552994 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a9406d8a0 0x7f2a9406fd50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2aac197520 tx=0x7f2aa000b410 comp rx=0 tx=0).stop 2024-02-19T19:10:09.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.192+0000 7f2ab08a0700 1 -- 172.21.15.88:0/1240552994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2aac0ff3c0 msgr2=0x7f2aac195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:09.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.192+0000 7f2ab08a0700 1 --2- 172.21.15.88:0/1240552994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2aac0ff3c0 0x7f2aac195f00 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c006010 tx=0x7f2a9c00bd70 comp rx=0 tx=0).stop 2024-02-19T19:10:09.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.192+0000 7f2ab08a0700 1 -- 172.21.15.88:0/1240552994 shutdown_connections 2024-02-19T19:10:09.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.192+0000 7f2ab08a0700 1 --2- 172.21.15.88:0/1240552994 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a9406d8a0 0x7f2a9406fd50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.192+0000 7f2ab08a0700 1 --2- 172.21.15.88:0/1240552994 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2aac0ff3c0 0x7f2aac195f00 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.192+0000 7f2ab08a0700 1 --2- 172.21.15.88:0/1240552994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aac0ffd60 0x7f2aac196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.192+0000 7f2ab08a0700 1 -- 172.21.15.88:0/1240552994 >> 172.21.15.88:0/1240552994 conn(0x7f2aac0fac80 msgr2=0x7f2aac1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:09.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.192+0000 7f2ab08a0700 1 -- 172.21.15.88:0/1240552994 shutdown_connections 2024-02-19T19:10:09.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.192+0000 7f2ab08a0700 1 -- 172.21.15.88:0/1240552994 wait complete. 2024-02-19T19:10:09.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.299+0000 7f2905948700 1 -- 172.21.15.88:0/63185017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29000ff3c0 msgr2=0x7f29000ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:09.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.299+0000 7f2905948700 1 --2- 172.21.15.88:0/63185017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29000ff3c0 0x7f29000ff790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f28f0009910 tx=0x7f28f0009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:09.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.299+0000 7f2905948700 1 -- 172.21.15.88:0/63185017 shutdown_connections 2024-02-19T19:10:09.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.299+0000 7f2905948700 1 --2- 172.21.15.88:0/63185017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29000ffd60 0x7f2900103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.299+0000 7f2905948700 1 --2- 172.21.15.88:0/63185017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29000ff3c0 0x7f29000ff790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.299+0000 7f2905948700 1 -- 172.21.15.88:0/63185017 >> 172.21.15.88:0/63185017 conn(0x7f29000fac80 msgr2=0x7f29000fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:09.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.299+0000 7f2905948700 1 -- 172.21.15.88:0/63185017 shutdown_connections 2024-02-19T19:10:09.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.299+0000 7f2905948700 1 -- 172.21.15.88:0/63185017 wait complete. 2024-02-19T19:10:09.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.300+0000 7f2905948700 1 Processor -- start 2024-02-19T19:10:09.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.300+0000 7f2905948700 1 -- start start 2024-02-19T19:10:09.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.300+0000 7f2905948700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29000ff3c0 0x7f2900195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:09.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.300+0000 7f2905948700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29000ffd60 0x7f2900196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:09.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.300+0000 7f2905948700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2900196b20 con 0x7f29000ffd60 2024-02-19T19:10:09.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.300+0000 7f2905948700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290019a8b0 con 0x7f29000ff3c0 2024-02-19T19:10:09.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.300+0000 7f28fe7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29000ffd60 0x7f2900196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:09.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.300+0000 7f28fe7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29000ffd60 0x7f2900196440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38878/0 (socket says 172.21.15.88:38878) 2024-02-19T19:10:09.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.300+0000 7f28fe7fc700 1 -- 172.21.15.88:0/1324939450 learned_addr learned my addr 172.21.15.88:0/1324939450 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:09.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.301+0000 7f28feffd700 1 --2- 172.21.15.88:0/1324939450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29000ff3c0 0x7f2900195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:09.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.301+0000 7f28fe7fc700 1 -- 172.21.15.88:0/1324939450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29000ff3c0 msgr2=0x7f2900195f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:09.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.301+0000 7f28fe7fc700 1 --2- 172.21.15.88:0/1324939450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29000ff3c0 0x7f2900195f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.301+0000 7f28fe7fc700 1 -- 172.21.15.88:0/1324939450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f00095f0 con 0x7f29000ffd60 2024-02-19T19:10:09.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.301+0000 7f28fe7fc700 1 --2- 172.21.15.88:0/1324939450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29000ffd60 0x7f2900196440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f28f400d900 tx=0x7f28f400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:09.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.301+0000 7f2904946700 1 -- 172.21.15.88:0/1324939450 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28f4009900 con 0x7f29000ffd60 2024-02-19T19:10:09.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.301+0000 7f2904946700 1 -- 172.21.15.88:0/1324939450 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28f4010460 con 0x7f29000ffd60 2024-02-19T19:10:09.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.301+0000 7f2904946700 1 -- 172.21.15.88:0/1324939450 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28f400f5f0 con 0x7f29000ffd60 2024-02-19T19:10:09.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.301+0000 7f2905948700 1 -- 172.21.15.88:0/1324939450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290019ab90 con 0x7f29000ffd60 2024-02-19T19:10:09.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.301+0000 7f2905948700 1 -- 172.21.15.88:0/1324939450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290019b090 con 0x7f29000ffd60 2024-02-19T19:10:09.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.302+0000 7f2905948700 1 -- 172.21.15.88:0/1324939450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290004f2a0 con 0x7f29000ffd60 2024-02-19T19:10:09.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.303+0000 7f2904946700 1 -- 172.21.15.88:0/1324939450 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f28f40105d0 con 0x7f29000ffd60 2024-02-19T19:10:09.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.303+0000 7f2904946700 1 --2- 172.21.15.88:0/1324939450 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f28e806d8a0 0x7f28e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:09.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.303+0000 7f2904946700 1 -- 172.21.15.88:0/1324939450 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f28f408d030 con 0x7f29000ffd60 2024-02-19T19:10:09.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.303+0000 7f28feffd700 1 --2- 172.21.15.88:0/1324939450 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f28e806d8a0 0x7f28e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:09.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.304+0000 7f28feffd700 1 --2- 172.21.15.88:0/1324939450 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f28e806d8a0 0x7f28e806fd50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f28f0006010 tx=0x7f28f001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:09.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.308+0000 7f2904946700 1 -- 172.21.15.88:0/1324939450 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28f4056890 con 0x7f29000ffd60 2024-02-19T19:10:09.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.548+0000 7f2905948700 1 -- 172.21.15.88:0/1324939450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f290019b300 con 0x7f29000ffd60 2024-02-19T19:10:09.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:09 smithi088 ceph-mon[88643]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-19T19:10:09.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:09 smithi088 ceph-mon[88643]: Upgrade: osd.5 is safe to restart 2024-02-19T19:10:09.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:09 smithi088 ceph-mon[88643]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:09.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:09 smithi088 ceph-mon[88643]: pgmap v186: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:09.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:09 smithi088 ceph-mon[88643]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:09.571 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:09 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1240552994' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:09.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.548+0000 7f2904946700 1 -- 172.21.15.88:0/1324939450 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1875 (secure 0 0 0) 0x7f28f4059f50 con 0x7f29000ffd60 2024-02-19T19:10:09.571 INFO:teuthology.orchestra.run.smithi088.stdout:e21 2024-02-19T19:10:09.571 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:10:09.571 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:10:09.571 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:10:09.571 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 21 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:04.398801+0000 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:10:09.572 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 46 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=14510,1=24263} 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:10:09.573 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:10:09.574 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:10:09.574 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:10:09.574 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:10:09.574 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{0:14510} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.88:6828/2325623973,v1:172.21.15.88:6829/2325623973] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:10:09.574 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{1:24263} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.132:6824/2841430379,v1:172.21.15.132:6825/2841430379] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:10:09.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:10:09.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:10:09.574 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:10:09.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:10:09.574 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:14522} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/2124144962,v1:172.21.15.88:6827/2124144962] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:10:09.574 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:10:09.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.550+0000 7f2905948700 1 -- 172.21.15.88:0/1324939450 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f28e806d8a0 msgr2=0x7f28e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:09.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.550+0000 7f2905948700 1 --2- 172.21.15.88:0/1324939450 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f28e806d8a0 0x7f28e806fd50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f28f0006010 tx=0x7f28f001a040 comp rx=0 tx=0).stop 2024-02-19T19:10:09.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.550+0000 7f2905948700 1 -- 172.21.15.88:0/1324939450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29000ffd60 msgr2=0x7f2900196440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.550+0000 7f2905948700 1 --2- 172.21.15.88:0/1324939450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29000ffd60 0x7f2900196440 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f28f400d900 tx=0x7f28f400dc10 comp rx=0 tx=0).stop 2024-02-19T19:10:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.551+0000 7f2905948700 1 -- 172.21.15.88:0/1324939450 shutdown_connections 2024-02-19T19:10:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.551+0000 7f2905948700 1 --2- 172.21.15.88:0/1324939450 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f28e806d8a0 0x7f28e806fd50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.551+0000 7f2905948700 1 --2- 172.21.15.88:0/1324939450 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f29000ff3c0 0x7f2900195f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.551+0000 7f2905948700 1 --2- 172.21.15.88:0/1324939450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29000ffd60 0x7f2900196440 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.551+0000 7f2905948700 1 -- 172.21.15.88:0/1324939450 >> 172.21.15.88:0/1324939450 conn(0x7f29000fac80 msgr2=0x7f29001034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.551+0000 7f2905948700 1 -- 172.21.15.88:0/1324939450 shutdown_connections 2024-02-19T19:10:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.551+0000 7f2905948700 1 -- 172.21.15.88:0/1324939450 wait complete. 2024-02-19T19:10:09.576 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 21 2024-02-19T19:10:09.630 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:09 smithi132 ceph-mon[70676]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-02-19T19:10:09.630 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:09 smithi132 ceph-mon[70676]: Upgrade: osd.5 is safe to restart 2024-02-19T19:10:09.630 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:09 smithi132 ceph-mon[70676]: from='client.34390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:09.631 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:09 smithi132 ceph-mon[70676]: pgmap v186: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:09.631 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:09 smithi132 ceph-mon[70676]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:09.631 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:09 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1240552994' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:09.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.656+0000 7fb7f01a1700 1 -- 172.21.15.88:0/1219031759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7e8104130 msgr2=0x7fb7e8104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:09.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.656+0000 7fb7f01a1700 1 --2- 172.21.15.88:0/1219031759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7e8104130 0x7fb7e8104500 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009910 tx=0x7fb7d8009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:09.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.656+0000 7fb7f01a1700 1 -- 172.21.15.88:0/1219031759 shutdown_connections 2024-02-19T19:10:09.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.656+0000 7fb7f01a1700 1 --2- 172.21.15.88:0/1219031759 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7e80ffbd0 0x7fb7e8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.656+0000 7fb7f01a1700 1 --2- 172.21.15.88:0/1219031759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7e8104130 0x7fb7e8104500 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.656+0000 7fb7f01a1700 1 -- 172.21.15.88:0/1219031759 >> 172.21.15.88:0/1219031759 conn(0x7fb7e8075130 msgr2=0x7fb7e8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:09.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.656+0000 7fb7f01a1700 1 -- 172.21.15.88:0/1219031759 shutdown_connections 2024-02-19T19:10:09.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.656+0000 7fb7f01a1700 1 -- 172.21.15.88:0/1219031759 wait complete. 2024-02-19T19:10:09.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7f01a1700 1 Processor -- start 2024-02-19T19:10:09.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7f01a1700 1 -- start start 2024-02-19T19:10:09.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7f01a1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7e80ffbd0 0x7fb7e8195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:09.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7f01a1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7e8104130 0x7fb7e8196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:09.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7f01a1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e8196a70 con 0x7fb7e8104130 2024-02-19T19:10:09.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7f01a1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e819a800 con 0x7fb7e80ffbd0 2024-02-19T19:10:09.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7ed73c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7e8104130 0x7fb7e8196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:09.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7ed73c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7e8104130 0x7fb7e8196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38900/0 (socket says 172.21.15.88:38900) 2024-02-19T19:10:09.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7ed73c700 1 -- 172.21.15.88:0/3328075030 learned_addr learned my addr 172.21.15.88:0/3328075030 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:09.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7edf3d700 1 --2- 172.21.15.88:0/3328075030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7e80ffbd0 0x7fb7e8195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:09.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7ed73c700 1 -- 172.21.15.88:0/3328075030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7e80ffbd0 msgr2=0x7fb7e8195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:09.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7ed73c700 1 --2- 172.21.15.88:0/3328075030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7e80ffbd0 0x7fb7e8195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.657+0000 7fb7ed73c700 1 -- 172.21.15.88:0/3328075030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d80095f0 con 0x7fb7e8104130 2024-02-19T19:10:09.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.658+0000 7fb7ed73c700 1 --2- 172.21.15.88:0/3328075030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7e8104130 0x7fb7e8196390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400d8d0 tx=0x7fb7e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:09.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.658+0000 7fb7deffd700 1 -- 172.21.15.88:0/3328075030 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7e4009940 con 0x7fb7e8104130 2024-02-19T19:10:09.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.658+0000 7fb7deffd700 1 -- 172.21.15.88:0/3328075030 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7e4010460 con 0x7fb7e8104130 2024-02-19T19:10:09.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.658+0000 7fb7f01a1700 1 -- 172.21.15.88:0/3328075030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e819aae0 con 0x7fb7e8104130 2024-02-19T19:10:09.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.658+0000 7fb7deffd700 1 -- 172.21.15.88:0/3328075030 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7e400f5f0 con 0x7fb7e8104130 2024-02-19T19:10:09.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.658+0000 7fb7f01a1700 1 -- 172.21.15.88:0/3328075030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e819b030 con 0x7fb7e8104130 2024-02-19T19:10:09.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.659+0000 7fb7f01a1700 1 -- 172.21.15.88:0/3328075030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e81086a0 con 0x7fb7e8104130 2024-02-19T19:10:09.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.659+0000 7fb7deffd700 1 -- 172.21.15.88:0/3328075030 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb7e40105d0 con 0x7fb7e8104130 2024-02-19T19:10:09.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.660+0000 7fb7deffd700 1 --2- 172.21.15.88:0/3328075030 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7d406d8a0 0x7fb7d406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:09.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.660+0000 7fb7deffd700 1 -- 172.21.15.88:0/3328075030 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb7e408d380 con 0x7fb7e8104130 2024-02-19T19:10:09.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.660+0000 7fb7edf3d700 1 --2- 172.21.15.88:0/3328075030 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7d406d8a0 0x7fb7d406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:09.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.660+0000 7fb7edf3d700 1 --2- 172.21.15.88:0/3328075030 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7d406d8a0 0x7fb7d406fd50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800b5c0 tx=0x7fb7d801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:09.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.664+0000 7fb7deffd700 1 -- 172.21.15.88:0/3328075030 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb7e4056c60 con 0x7fb7e8104130 2024-02-19T19:10:09.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.897+0000 7fb7f01a1700 1 -- 172.21.15.88:0/3328075030 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7e819b310 con 0x7fb7d406d8a0 2024-02-19T19:10:09.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.898+0000 7fb7deffd700 1 -- 172.21.15.88:0/3328075030 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fb7e819b310 con 0x7fb7d406d8a0 2024-02-19T19:10:09.919 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:10:09.920 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:10:09.920 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:10:09.920 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:10:09.920 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:10:09.920 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:10:09.920 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:10:09.920 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:10:09.920 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:10:09.920 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "11/23 daemons upgraded", 2024-02-19T19:10:09.920 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading osd daemons", 2024-02-19T19:10:09.920 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:10:09.921 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:10:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.900+0000 7fb7f01a1700 1 -- 172.21.15.88:0/3328075030 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7d406d8a0 msgr2=0x7fb7d406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:09.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.900+0000 7fb7f01a1700 1 --2- 172.21.15.88:0/3328075030 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7d406d8a0 0x7fb7d406fd50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb7d800b5c0 tx=0x7fb7d801a040 comp rx=0 tx=0).stop 2024-02-19T19:10:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.900+0000 7fb7f01a1700 1 -- 172.21.15.88:0/3328075030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7e8104130 msgr2=0x7fb7e8196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.900+0000 7fb7f01a1700 1 --2- 172.21.15.88:0/3328075030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7e8104130 0x7fb7e8196390 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400d8d0 tx=0x7fb7e400dc90 comp rx=0 tx=0).stop 2024-02-19T19:10:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.900+0000 7fb7f01a1700 1 -- 172.21.15.88:0/3328075030 shutdown_connections 2024-02-19T19:10:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.900+0000 7fb7f01a1700 1 --2- 172.21.15.88:0/3328075030 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7d406d8a0 0x7fb7d406fd50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.900+0000 7fb7f01a1700 1 --2- 172.21.15.88:0/3328075030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7e80ffbd0 0x7fb7e8195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.900+0000 7fb7f01a1700 1 --2- 172.21.15.88:0/3328075030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7e8104130 0x7fb7e8196390 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.900+0000 7fb7f01a1700 1 -- 172.21.15.88:0/3328075030 >> 172.21.15.88:0/3328075030 conn(0x7fb7e8075130 msgr2=0x7fb7e80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.900+0000 7fb7f01a1700 1 -- 172.21.15.88:0/3328075030 shutdown_connections 2024-02-19T19:10:09.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:09.900+0000 7fb7f01a1700 1 -- 172.21.15.88:0/3328075030 wait complete. 2024-02-19T19:10:10.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.004+0000 7f610de1c700 1 -- 172.21.15.88:0/99357535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61080ff350 msgr2=0x7f61080ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:10.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.004+0000 7f610de1c700 1 --2- 172.21.15.88:0/99357535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61080ff350 0x7f61080ff720 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f60f800b0a0 tx=0x7f60f800b3b0 comp rx=0 tx=0).stop 2024-02-19T19:10:10.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.004+0000 7f610de1c700 1 -- 172.21.15.88:0/99357535 shutdown_connections 2024-02-19T19:10:10.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.004+0000 7f610de1c700 1 --2- 172.21.15.88:0/99357535 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61080ffc60 0x7f610810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:10.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.004+0000 7f610de1c700 1 --2- 172.21.15.88:0/99357535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61080ff350 0x7f61080ff720 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:10.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.004+0000 7f610de1c700 1 -- 172.21.15.88:0/99357535 >> 172.21.15.88:0/99357535 conn(0x7f61080748b0 msgr2=0x7f6108074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:10.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.004+0000 7f610de1c700 1 -- 172.21.15.88:0/99357535 shutdown_connections 2024-02-19T19:10:10.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.004+0000 7f610de1c700 1 -- 172.21.15.88:0/99357535 wait complete. 2024-02-19T19:10:10.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.004+0000 7f610de1c700 1 Processor -- start 2024-02-19T19:10:10.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f610de1c700 1 -- start start 2024-02-19T19:10:10.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f610de1c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61080ff350 0x7f6108100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:10.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f610de1c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61080ffc60 0x7f6108100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:10.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f610de1c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6108100fb0 con 0x7f61080ff350 2024-02-19T19:10:10.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f610de1c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61081010f0 con 0x7f61080ffc60 2024-02-19T19:10:10.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f61077fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61080ff350 0x7f6108100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f61077fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61080ff350 0x7f6108100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38916/0 (socket says 172.21.15.88:38916) 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f61077fe700 1 -- 172.21.15.88:0/3104684523 learned_addr learned my addr 172.21.15.88:0/3104684523 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f6106ffd700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61080ffc60 0x7f6108100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f61077fe700 1 -- 172.21.15.88:0/3104684523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61080ffc60 msgr2=0x7f6108100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f61077fe700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61080ffc60 0x7f6108100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f61077fe700 1 -- 172.21.15.88:0/3104684523 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f8009d00 con 0x7f61080ff350 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f6106ffd700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61080ffc60 0x7f6108100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.005+0000 7f61077fe700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61080ff350 0x7f6108100530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60f8015040 tx=0x7f60f800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.006+0000 7f6104ff9700 1 -- 172.21.15.88:0/3104684523 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f800e040 con 0x7f61080ff350 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.006+0000 7f6104ff9700 1 -- 172.21.15.88:0/3104684523 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f60f8003de0 con 0x7f61080ff350 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.006+0000 7f6104ff9700 1 -- 172.21.15.88:0/3104684523 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60f801d910 con 0x7f61080ff350 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.006+0000 7f610de1c700 1 -- 172.21.15.88:0/3104684523 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6108101370 con 0x7f61080ff350 2024-02-19T19:10:10.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.006+0000 7f610de1c700 1 -- 172.21.15.88:0/3104684523 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61081a23f0 con 0x7f61080ff350 2024-02-19T19:10:10.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.007+0000 7f610de1c700 1 -- 172.21.15.88:0/3104684523 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6108107dd0 con 0x7f61080ff350 2024-02-19T19:10:10.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.007+0000 7f6104ff9700 1 -- 172.21.15.88:0/3104684523 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f60f8004040 con 0x7f61080ff350 2024-02-19T19:10:10.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.007+0000 7f6104ff9700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f60f006d8a0 0x7f60f006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:10.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.008+0000 7f6104ff9700 1 -- 172.21.15.88:0/3104684523 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f60f808ee50 con 0x7f61080ff350 2024-02-19T19:10:10.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.008+0000 7f6106ffd700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f60f006d8a0 0x7f60f006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:10.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.008+0000 7f6106ffd700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f60f006d8a0 0x7f60f006fd50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6108101d20 tx=0x7f60fc008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:10.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.012+0000 7f6104ff9700 1 -- 172.21.15.88:0/3104684523 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f60f80586b0 con 0x7f61080ff350 2024-02-19T19:10:10.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.293+0000 7f610de1c700 1 -- 172.21.15.88:0/3104684523 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f610804f2a0 con 0x7f61080ff350 2024-02-19T19:10:10.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.293+0000 7f6104ff9700 1 -- 172.21.15.88:0/3104684523 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f60f805bd70 con 0x7f61080ff350 2024-02-19T19:10:10.315 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_OK 2024-02-19T19:10:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.295+0000 7f610de1c700 1 -- 172.21.15.88:0/3104684523 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f60f006d8a0 msgr2=0x7f60f006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.295+0000 7f610de1c700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f60f006d8a0 0x7f60f006fd50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6108101d20 tx=0x7f60fc008040 comp rx=0 tx=0).stop 2024-02-19T19:10:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.295+0000 7f610de1c700 1 -- 172.21.15.88:0/3104684523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61080ff350 msgr2=0x7f6108100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.295+0000 7f610de1c700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61080ff350 0x7f6108100530 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f60f8015040 tx=0x7f60f800cdd0 comp rx=0 tx=0).stop 2024-02-19T19:10:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.295+0000 7f610de1c700 1 -- 172.21.15.88:0/3104684523 shutdown_connections 2024-02-19T19:10:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.295+0000 7f610de1c700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f60f006d8a0 0x7f60f006fd50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.295+0000 7f610de1c700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61080ff350 0x7f6108100530 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:10.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.295+0000 7f610de1c700 1 --2- 172.21.15.88:0/3104684523 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61080ffc60 0x7f6108100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:10.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.295+0000 7f610de1c700 1 -- 172.21.15.88:0/3104684523 >> 172.21.15.88:0/3104684523 conn(0x7f61080748b0 msgr2=0x7f6108104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:10.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.295+0000 7f610de1c700 1 -- 172.21.15.88:0/3104684523 shutdown_connections 2024-02-19T19:10:10.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:10.295+0000 7f610de1c700 1 -- 172.21.15.88:0/3104684523 wait complete. 2024-02-19T19:10:10.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:10 smithi088 ceph-mon[88643]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:10.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:10 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1324939450' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:10:10.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:10 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:10.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:10 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-19T19:10:10.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:10 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:10:10.603 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:10 smithi132 ceph-mon[70676]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:10.603 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:10 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1324939450' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:10:10.603 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:10 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:10.603 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:10 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-19T19:10:10.603 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:10 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:10:11.581 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:11 smithi132 ceph-mon[70676]: Upgrade: Updating osd.5 2024-02-19T19:10:11.581 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:11 smithi132 ceph-mon[70676]: Deploying daemon osd.5 on smithi132 2024-02-19T19:10:11.581 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:11 smithi132 ceph-mon[70676]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:11.581 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:11 smithi132 ceph-mon[70676]: pgmap v187: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:11.582 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:11 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3104684523' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:10:11.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:11 smithi088 ceph-mon[88643]: Upgrade: Updating osd.5 2024-02-19T19:10:11.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:11 smithi088 ceph-mon[88643]: Deploying daemon osd.5 on smithi132 2024-02-19T19:10:11.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:11 smithi088 ceph-mon[88643]: from='client.34410 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:11.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:11 smithi088 ceph-mon[88643]: pgmap v187: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:11.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:11 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3104684523' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:10:13.430 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:13 smithi132 systemd[1]: Stopping Ceph osd.5 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:10:13.430 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:13 smithi132 ceph-mon[70676]: pgmap v188: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:13.430 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:13.431 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:10:13.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:13 smithi088 ceph-mon[88643]: pgmap v188: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:13.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:13.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:10:13.754 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:13 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[53201]: 2024-02-19T19:10:13.407+0000 7fec3d0d3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T19:10:13.754 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:13 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[53201]: 2024-02-19T19:10:13.407+0000 7fec3d0d3700 -1 osd.5 77 *** Got signal Terminated *** 2024-02-19T19:10:13.754 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:13 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[53201]: 2024-02-19T19:10:13.407+0000 7fec3d0d3700 -1 osd.5 77 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T19:10:14.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:14 smithi088 ceph-mon[88643]: osd.5 marked itself down and dead 2024-02-19T19:10:14.668 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:14 smithi132 ceph-mon[70676]: osd.5 marked itself down and dead 2024-02-19T19:10:15.004 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:14 smithi132 podman[91097]: 2024-02-19 19:10:14.646831748 +0000 UTC m=+1.361868288 container died d93e850af9920bdee6e7d6740c69c06aa64a9ad2ab3929c28b194ed7ddb23696 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-19T19:10:15.300 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:15 smithi132 podman[91097]: 2024-02-19 19:10:15.066681574 +0000 UTC m=+1.781718101 container cleanup d93e850af9920bdee6e7d6740c69c06aa64a9ad2ab3929c28b194ed7ddb23696 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, GIT_CLEAN=True, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-19T19:10:15.300 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:15 smithi132 bash[91097]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5 2024-02-19T19:10:15.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:15 smithi088 ceph-mon[88643]: pgmap v189: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:15.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:15 smithi088 ceph-mon[88643]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:10:15.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:15 smithi088 ceph-mon[88643]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-19T19:10:15.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:15 smithi088 ceph-mon[88643]: osdmap e78: 6 total, 5 up, 6 in 2024-02-19T19:10:15.754 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:15 smithi132 podman[91116]: 2024-02-19 19:10:15.301712562 +0000 UTC m=+0.649161352 container remove d93e850af9920bdee6e7d6740c69c06aa64a9ad2ab3929c28b194ed7ddb23696 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-02-19T19:10:15.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:15 smithi132 ceph-mon[70676]: pgmap v189: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:15.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:15 smithi132 ceph-mon[70676]: Health check failed: 1 osds down (OSD_DOWN) 2024-02-19T19:10:15.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:15 smithi132 ceph-mon[70676]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-02-19T19:10:15.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:15 smithi132 ceph-mon[70676]: osdmap e78: 6 total, 5 up, 6 in 2024-02-19T19:10:16.051 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:15 smithi132 podman[91212]: 2024-02-19 19:10:15.636317985 +0000 UTC m=+0.018189769 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:10:16.052 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:15 smithi132 podman[91212]: 2024-02-19 19:10:15.777687196 +0000 UTC m=+0.159558965 container create 0c9e1c6fe79162c64afe887087e5b5ca257a53455592023b9ef13de4fdda7855 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:10:16.052 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:16 smithi132 podman[91212]: 2024-02-19 19:10:16.030807788 +0000 UTC m=+0.412679567 container init 0c9e1c6fe79162c64afe887087e5b5ca257a53455592023b9ef13de4fdda7855 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T19:10:16.308 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:16 smithi132 podman[91212]: 2024-02-19 19:10:16.034188069 +0000 UTC m=+0.416059838 container start 0c9e1c6fe79162c64afe887087e5b5ca257a53455592023b9ef13de4fdda7855 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T19:10:16.309 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:16 smithi132 podman[91212]: 2024-02-19 19:10:16.113979994 +0000 UTC m=+0.495851781 container attach 0c9e1c6fe79162c64afe887087e5b5ca257a53455592023b9ef13de4fdda7855 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T19:10:16.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:16 smithi088 ceph-mon[88643]: osdmap e79: 6 total, 5 up, 6 in 2024-02-19T19:10:16.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:16 smithi132 ceph-mon[70676]: osdmap e79: 6 total, 5 up, 6 in 2024-02-19T19:10:16.754 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:16 smithi132 podman[91212]: 2024-02-19 19:10:16.319589905 +0000 UTC m=+0.701461683 container died 0c9e1c6fe79162c64afe887087e5b5ca257a53455592023b9ef13de4fdda7855 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:10:17.178 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:16 smithi132 podman[91239]: 2024-02-19 19:10:16.926427131 +0000 UTC m=+0.598201693 container remove 0c9e1c6fe79162c64afe887087e5b5ca257a53455592023b9ef13de4fdda7855 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-02-19T19:10:17.178 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:16 smithi132 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.5.service: Deactivated successfully. 2024-02-19T19:10:17.178 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:16 smithi132 systemd[1]: Stopped Ceph osd.5 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:10:17.178 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:16 smithi132 systemd[1]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.5.service: Consumed 1min 3.154s CPU time. 2024-02-19T19:10:17.178 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:17 smithi132 systemd[1]: Starting Ceph osd.5 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T19:10:17.432 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:17 smithi132 ceph-mon[70676]: pgmap v192: 65 pgs: 6 active+undersized+degraded, 3 peering, 5 stale+active+clean, 6 active+undersized, 45 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/333 objects degraded (4.204%) 2024-02-19T19:10:17.432 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:17 smithi132 ceph-mon[70676]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-19T19:10:17.432 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:17 smithi132 ceph-mon[70676]: Health check failed: Degraded data redundancy: 14/333 objects degraded (4.204%), 6 pgs degraded (PG_DEGRADED) 2024-02-19T19:10:17.432 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:17 smithi132 podman[91344]: 2024-02-19 19:10:17.276190675 +0000 UTC m=+0.017848980 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:10:17.433 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:17 smithi132 podman[91344]: 2024-02-19 19:10:17.411396923 +0000 UTC m=+0.153055259 container create 8d12d6a152d29e63e9c8d9dbb30dcdef8fe63943ef8d37bcdea4cb217058ea2b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T19:10:17.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:17 smithi088 ceph-mon[88643]: pgmap v192: 65 pgs: 6 active+undersized+degraded, 3 peering, 5 stale+active+clean, 6 active+undersized, 45 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 14/333 objects degraded (4.204%) 2024-02-19T19:10:17.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:17 smithi088 ceph-mon[88643]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-02-19T19:10:17.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:17 smithi088 ceph-mon[88643]: Health check failed: Degraded data redundancy: 14/333 objects degraded (4.204%), 6 pgs degraded (PG_DEGRADED) 2024-02-19T19:10:17.718 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:17 smithi132 podman[91344]: 2024-02-19 19:10:17.631440573 +0000 UTC m=+0.373098870 container init 8d12d6a152d29e63e9c8d9dbb30dcdef8fe63943ef8d37bcdea4cb217058ea2b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-02-19T19:10:17.718 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:17 smithi132 podman[91344]: 2024-02-19 19:10:17.634683281 +0000 UTC m=+0.376341586 container start 8d12d6a152d29e63e9c8d9dbb30dcdef8fe63943ef8d37bcdea4cb217058ea2b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-19T19:10:17.718 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:17 smithi132 podman[91344]: 2024-02-19 19:10:17.69715317 +0000 UTC m=+0.438811480 container attach 8d12d6a152d29e63e9c8d9dbb30dcdef8fe63943ef8d37bcdea4cb217058ea2b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate[91359]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate[91359]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 bash[91344]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 bash[91344]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 bash[91344]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 bash[91344]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 bash[91344]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 bash[91344]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 bash[91344]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate[91359]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate[91359]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate[91359]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-02-19T19:10:18.447 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate[91359]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-02-19T19:10:18.448 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate[91359]: --> ceph-volume raw activate successful for osd ID: 5 2024-02-19T19:10:18.448 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 podman[91344]: 2024-02-19 19:10:18.104403898 +0000 UTC m=+0.846062187 container died 8d12d6a152d29e63e9c8d9dbb30dcdef8fe63943ef8d37bcdea4cb217058ea2b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-02-19T19:10:19.087 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 podman[91495]: 2024-02-19 19:10:18.768580485 +0000 UTC m=+0.659227395 container remove 8d12d6a152d29e63e9c8d9dbb30dcdef8fe63943ef8d37bcdea4cb217058ea2b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-activate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-19T19:10:19.088 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 podman[91561]: 2024-02-19 19:10:18.969436853 +0000 UTC m=+0.102482695 container create be60ff50bffe95fb74c096cf466cd3c87397142c04c4f14fddb540728794666e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=) 2024-02-19T19:10:19.088 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:18 smithi132 podman[91561]: 2024-02-19 19:10:18.884433681 +0000 UTC m=+0.017479537 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T19:10:19.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:19 smithi132 ceph-mon[70676]: pgmap v193: 65 pgs: 12 active+undersized+degraded, 3 peering, 13 active+undersized, 37 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/333 objects degraded (13.213%) 2024-02-19T19:10:19.504 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:19 smithi132 podman[91561]: 2024-02-19 19:10:19.164426507 +0000 UTC m=+0.297472350 container init be60ff50bffe95fb74c096cf466cd3c87397142c04c4f14fddb540728794666e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, org.label-schema.license=GPLv2) 2024-02-19T19:10:19.504 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:19 smithi132 podman[91561]: 2024-02-19 19:10:19.167355691 +0000 UTC m=+0.300401535 container start be60ff50bffe95fb74c096cf466cd3c87397142c04c4f14fddb540728794666e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214) 2024-02-19T19:10:19.504 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:19 smithi132 ceph-osd[91580]: -- 172.21.15.132:0/186909956 <== mon.1 v2:172.21.15.132:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56118485f1e0 con 0x561184831000 2024-02-19T19:10:19.504 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:19 smithi132 bash[91561]: be60ff50bffe95fb74c096cf466cd3c87397142c04c4f14fddb540728794666e 2024-02-19T19:10:19.504 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:19 smithi132 systemd[1]: Started Ceph osd.5 for a60b4d3a-cf57-11ee-95bb-87774f69a715. 2024-02-19T19:10:19.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:19 smithi088 ceph-mon[88643]: pgmap v193: 65 pgs: 12 active+undersized+degraded, 3 peering, 13 active+undersized, 37 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/333 objects degraded (13.213%) 2024-02-19T19:10:20.004 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:19 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T19:10:19.720+0000 7f9d0e9c27c0 -1 Falling back to public interface 2024-02-19T19:10:20.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:20.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:20.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:10:20.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:20.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:20.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:10:21.511 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:21 smithi132 ceph-mon[70676]: pgmap v194: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-02-19T19:10:21.511 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:21 smithi132 ceph-mon[70676]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-19T19:10:21.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:21 smithi088 ceph-mon[88643]: pgmap v194: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-02-19T19:10:21.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:21 smithi088 ceph-mon[88643]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-02-19T19:10:23.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:23 smithi088 ceph-mon[88643]: pgmap v195: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-02-19T19:10:23.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:23.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:23.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:23 smithi132 ceph-mon[70676]: pgmap v195: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-02-19T19:10:23.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:23 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:23.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:23 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:24.196 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:23 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T19:10:23.908+0000 7f9d0e9c27c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-02-19T19:10:24.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:24.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:24.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:24 smithi132 ceph-mon[70676]: from='osd.5 [v2:172.21.15.132:6816/1555619083,v1:172.21.15.132:6817/1555619083]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T19:10:24.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:24 smithi132 ceph-mon[70676]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T19:10:24.505 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:24 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T19:10:24.351+0000 7f9d0e9c27c0 -1 osd.5 77 log_to_monitors true 2024-02-19T19:10:24.505 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 19:10:24 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T19:10:24.417+0000 7f9cffe1a700 -1 osd.5 77 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-19T19:10:24.716 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:24.716 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:24.716 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:24 smithi088 ceph-mon[88643]: from='osd.5 [v2:172.21.15.132:6816/1555619083,v1:172.21.15.132:6817/1555619083]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T19:10:24.716 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:24 smithi088 ceph-mon[88643]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-19T19:10:25.677 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:25 smithi132 ceph-mon[70676]: pgmap v196: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-02-19T19:10:25.677 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:25 smithi132 ceph-mon[70676]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-19T19:10:25.677 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:25 smithi132 ceph-mon[70676]: osdmap e80: 6 total, 5 up, 6 in 2024-02-19T19:10:25.677 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:25 smithi132 ceph-mon[70676]: from='osd.5 [v2:172.21.15.132:6816/1555619083,v1:172.21.15.132:6817/1555619083]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:10:25.677 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:25 smithi132 ceph-mon[70676]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:10:25.678 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:25 smithi132 ceph-mon[70676]: Health check update: Degraded data redundancy: 51/333 objects degraded (15.315%), 14 pgs degraded (PG_DEGRADED) 2024-02-19T19:10:25.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:25 smithi088 ceph-mon[88643]: pgmap v196: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 51/333 objects degraded (15.315%) 2024-02-19T19:10:25.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:25 smithi088 ceph-mon[88643]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-19T19:10:25.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:25 smithi088 ceph-mon[88643]: osdmap e80: 6 total, 5 up, 6 in 2024-02-19T19:10:25.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:25 smithi088 ceph-mon[88643]: from='osd.5 [v2:172.21.15.132:6816/1555619083,v1:172.21.15.132:6817/1555619083]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:10:25.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:25 smithi088 ceph-mon[88643]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi132", "root=default"]}]: dispatch 2024-02-19T19:10:25.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:25 smithi088 ceph-mon[88643]: Health check update: Degraded data redundancy: 51/333 objects degraded (15.315%), 14 pgs degraded (PG_DEGRADED) 2024-02-19T19:10:26.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:26 smithi132 ceph-mon[70676]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:10:26.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:26 smithi132 ceph-mon[70676]: osd.5 [v2:172.21.15.132:6816/1555619083,v1:172.21.15.132:6817/1555619083] boot 2024-02-19T19:10:26.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:26 smithi132 ceph-mon[70676]: osdmap e81: 6 total, 6 up, 6 in 2024-02-19T19:10:26.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T19:10:26.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:26 smithi132 ceph-mon[70676]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:10:26.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:26 smithi088 ceph-mon[88643]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-02-19T19:10:26.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:26 smithi088 ceph-mon[88643]: osd.5 [v2:172.21.15.132:6816/1555619083,v1:172.21.15.132:6817/1555619083] boot 2024-02-19T19:10:26.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:26 smithi088 ceph-mon[88643]: osdmap e81: 6 total, 6 up, 6 in 2024-02-19T19:10:26.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-19T19:10:26.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:26 smithi088 ceph-mon[88643]: fsmap cephfs:2 {0=cephfs.smithi088.psjiah=up:active,1=cephfs.smithi132.afhgdi=up:active} 2 up:standby 2024-02-19T19:10:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: pgmap v199: 65 pgs: 2 peering, 13 active+undersized+degraded, 13 active+undersized, 37 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-02-19T19:10:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: osdmap e82: 6 total, 6 up, 6 in 2024-02-19T19:10:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:10:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:10:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:10:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:27.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:27.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:27.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:27.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:27.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-19T19:10:27.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-19T19:10:27.759 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-19T19:10:27.759 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-19T19:10:27.759 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-19T19:10:27.759 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-19T19:10:27.759 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-19T19:10:27.759 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-19T19:10:27.759 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-19T19:10:27.759 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-19T19:10:27.759 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-19T19:10:27.760 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-19T19:10:27.760 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-19T19:10:27.828 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: pgmap v199: 65 pgs: 2 peering, 13 active+undersized+degraded, 13 active+undersized, 37 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 49/333 objects degraded (14.715%) 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: osdmap e82: 6 total, 6 up, 6 in 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:27.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-02-19T19:10:27.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-02-19T19:10:28.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:10:28.264+0000 7fb6027b9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-19T19:10:28.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: Upgrade: Setting container_image for all osd 2024-02-19T19:10:28.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: Upgrade: Setting require_osd_release to 19 squid 2024-02-19T19:10:28.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:28.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:10:28.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-19T19:10:28.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-19T19:10:28.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: osdmap e83: 6 total, 6 up, 6 in 2024-02-19T19:10:28.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-19T19:10:28.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-19T19:10:28.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-19T19:10:28.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-19T19:10:28.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: osdmap e84: 6 total, 6 up, 6 in 2024-02-19T19:10:28.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:28 smithi088 ceph-mon[88643]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-02-19T19:10:28.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: Upgrade: Setting container_image for all osd 2024-02-19T19:10:28.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: Upgrade: Setting require_osd_release to 19 squid 2024-02-19T19:10:28.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:28.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:10:28.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-02-19T19:10:28.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-02-19T19:10:28.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: osdmap e83: 6 total, 6 up, 6 in 2024-02-19T19:10:28.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-19T19:10:28.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-19T19:10:28.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-19T19:10:28.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-19T19:10:28.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: osdmap e84: 6 total, 6 up, 6 in 2024-02-19T19:10:28.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:28 smithi132 ceph-mon[70676]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-02-19T19:10:29.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:29 smithi132 ceph-mon[70676]: pgmap v201: 65 pgs: 3 peering, 8 active+undersized+degraded, 10 active+undersized, 44 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23/333 objects degraded (6.907%) 2024-02-19T19:10:29.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:29 smithi132 ceph-mon[70676]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-02-19T19:10:29.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:29 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.132:6824/1971026183,v1:172.21.15.132:6825/1971026183] up:boot 2024-02-19T19:10:29.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:29 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.88:6828/3440020652,v1:172.21.15.88:6829/3440020652] up:boot 2024-02-19T19:10:29.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:29 smithi132 ceph-mon[70676]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:10:29.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:10:29.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:10:29.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:29 smithi088 ceph-mon[88643]: pgmap v201: 65 pgs: 3 peering, 8 active+undersized+degraded, 10 active+undersized, 44 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23/333 objects degraded (6.907%) 2024-02-19T19:10:29.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:29 smithi088 ceph-mon[88643]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-02-19T19:10:29.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:29 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.132:6824/1971026183,v1:172.21.15.132:6825/1971026183] up:boot 2024-02-19T19:10:29.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:29 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.88:6828/3440020652,v1:172.21.15.88:6829/3440020652] up:boot 2024-02-19T19:10:29.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:29 smithi088 ceph-mon[88643]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:10:29.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:10:29.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:10:31.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:30 smithi088 ceph-mon[88643]: Upgrade: Updating mds.cephfs.smithi088.ajgvpz 2024-02-19T19:10:31.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:30 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:31.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:30 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.ajgvpz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:10:31.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:30 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:10:31.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:30 smithi088 ceph-mon[88643]: Deploying daemon mds.cephfs.smithi088.ajgvpz on smithi088 2024-02-19T19:10:31.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:30 smithi088 ceph-mon[88643]: Health check update: Degraded data redundancy: 23/333 objects degraded (6.907%), 8 pgs degraded (PG_DEGRADED) 2024-02-19T19:10:31.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:30 smithi088 ceph-mon[88643]: pgmap v204: 65 pgs: 3 peering, 2 active+undersized+degraded, 1 active+undersized, 59 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4/333 objects degraded (1.201%) 2024-02-19T19:10:31.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:30 smithi132 ceph-mon[70676]: Upgrade: Updating mds.cephfs.smithi088.ajgvpz 2024-02-19T19:10:31.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:30 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:31.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:30 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.ajgvpz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:10:31.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:30 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:10:31.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:30 smithi132 ceph-mon[70676]: Deploying daemon mds.cephfs.smithi088.ajgvpz on smithi088 2024-02-19T19:10:31.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:30 smithi132 ceph-mon[70676]: Health check update: Degraded data redundancy: 23/333 objects degraded (6.907%), 8 pgs degraded (PG_DEGRADED) 2024-02-19T19:10:31.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:30 smithi132 ceph-mon[70676]: pgmap v204: 65 pgs: 3 peering, 2 active+undersized+degraded, 1 active+undersized, 59 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4/333 objects degraded (1.201%) 2024-02-19T19:10:33.281 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:33 smithi088 ceph-mon[88643]: osdmap e85: 6 total, 6 up, 6 in 2024-02-19T19:10:33.281 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:33 smithi088 ceph-mon[88643]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-19T19:10:33.281 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:33 smithi088 ceph-mon[88643]: pgmap v206: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:33.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:33 smithi132 ceph-mon[70676]: osdmap e85: 6 total, 6 up, 6 in 2024-02-19T19:10:33.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:33 smithi132 ceph-mon[70676]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-19T19:10:33.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:33 smithi132 ceph-mon[70676]: pgmap v206: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:34.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:34 smithi088 ceph-mon[88643]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/333 objects degraded (1.201%), 2 pgs degraded) 2024-02-19T19:10:34.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:34 smithi132 ceph-mon[70676]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/333 objects degraded (1.201%), 2 pgs degraded) 2024-02-19T19:10:35.351 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:35 smithi088 ceph-mon[88643]: pgmap v207: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:35.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:35.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:35.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:10:35.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:35 smithi132 ceph-mon[70676]: pgmap v207: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:35.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:35.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:35.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:10:36.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:36 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] up:boot 2024-02-19T19:10:36.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:36 smithi088 ceph-mon[88643]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:10:36.430 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:36 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T19:10:36.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:36 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] up:boot 2024-02-19T19:10:36.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:36 smithi132 ceph-mon[70676]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:10:36.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:36 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T19:10:37.389 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:37 smithi088 ceph-mon[88643]: pgmap v208: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:37.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:37 smithi132 ceph-mon[70676]: pgmap v208: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:39.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:39 smithi088 ceph-mon[88643]: pgmap v209: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:39.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:39 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:39 smithi132 ceph-mon[70676]: pgmap v209: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:39 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:40.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.411+0000 7f72db5bc700 1 -- 172.21.15.88:0/1933654446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72d40ffc60 msgr2=0x7f72d410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:40.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.411+0000 7f72db5bc700 1 --2- 172.21.15.88:0/1933654446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72d40ffc60 0x7f72d410a6d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72c4009910 tx=0x7f72c4009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:40.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.411+0000 7f72db5bc700 1 -- 172.21.15.88:0/1933654446 shutdown_connections 2024-02-19T19:10:40.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.411+0000 7f72db5bc700 1 --2- 172.21.15.88:0/1933654446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72d40ffc60 0x7f72d410a6d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:40.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.411+0000 7f72db5bc700 1 --2- 172.21.15.88:0/1933654446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d40ff350 0x7f72d40ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:40.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.411+0000 7f72db5bc700 1 -- 172.21.15.88:0/1933654446 >> 172.21.15.88:0/1933654446 conn(0x7f72d40748b0 msgr2=0x7f72d4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:40.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.411+0000 7f72db5bc700 1 -- 172.21.15.88:0/1933654446 shutdown_connections 2024-02-19T19:10:40.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.411+0000 7f72db5bc700 1 -- 172.21.15.88:0/1933654446 wait complete. 2024-02-19T19:10:40.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72db5bc700 1 Processor -- start 2024-02-19T19:10:40.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72db5bc700 1 -- start start 2024-02-19T19:10:40.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72db5bc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d40ff350 0x7f72d4195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:40.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72db5bc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72d40ffc60 0x7f72d4196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:40.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72db5bc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d4196b60 con 0x7f72d40ffc60 2024-02-19T19:10:40.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72db5bc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d419a8a0 con 0x7f72d40ff350 2024-02-19T19:10:40.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72d8b57700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72d40ffc60 0x7f72d4196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:40.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72d8b57700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72d40ffc60 0x7f72d4196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42884/0 (socket says 172.21.15.88:42884) 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72d8b57700 1 -- 172.21.15.88:0/2373428930 learned_addr learned my addr 172.21.15.88:0/2373428930 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72d9358700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d40ff350 0x7f72d4195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72d8b57700 1 -- 172.21.15.88:0/2373428930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d40ff350 msgr2=0x7f72d4195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72d8b57700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d40ff350 0x7f72d4195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72d8b57700 1 -- 172.21.15.88:0/2373428930 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c40095f0 con 0x7f72d40ffc60 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.412+0000 7f72d9358700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d40ff350 0x7f72d4195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.413+0000 7f72d8b57700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72d40ffc60 0x7f72d4196480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f72c4000c00 tx=0x7f72c40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.413+0000 7f72ca7fc700 1 -- 172.21.15.88:0/2373428930 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c401d070 con 0x7f72d40ffc60 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.413+0000 7f72db5bc700 1 -- 172.21.15.88:0/2373428930 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d419ab80 con 0x7f72d40ffc60 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.413+0000 7f72ca7fc700 1 -- 172.21.15.88:0/2373428930 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72c400f460 con 0x7f72d40ffc60 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.413+0000 7f72ca7fc700 1 -- 172.21.15.88:0/2373428930 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c4021640 con 0x7f72d40ffc60 2024-02-19T19:10:40.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.413+0000 7f72db5bc700 1 -- 172.21.15.88:0/2373428930 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d419b0d0 con 0x7f72d40ffc60 2024-02-19T19:10:40.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.415+0000 7f72db5bc700 1 -- 172.21.15.88:0/2373428930 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d4107dd0 con 0x7f72d40ffc60 2024-02-19T19:10:40.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.415+0000 7f72ca7fc700 1 -- 172.21.15.88:0/2373428930 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f72c400f5d0 con 0x7f72d40ffc60 2024-02-19T19:10:40.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.415+0000 7f72ca7fc700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72c006d8a0 0x7f72c006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:40.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.415+0000 7f72ca7fc700 1 -- 172.21.15.88:0/2373428930 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f72c408e800 con 0x7f72d40ffc60 2024-02-19T19:10:40.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.415+0000 7f72d9358700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72c006d8a0 0x7f72c006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:40.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.416+0000 7f72d9358700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72c006d8a0 0x7f72c006fd50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72d0007900 tx=0x7f72d0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:40.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.421+0000 7f72ca7fc700 1 -- 172.21.15.88:0/2373428930 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f72c4058030 con 0x7f72d40ffc60 2024-02-19T19:10:40.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.649+0000 7f72db5bc700 1 -- 172.21.15.88:0/2373428930 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72d4197360 con 0x7f72c006d8a0 2024-02-19T19:10:40.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.650+0000 7f72ca7fc700 1 -- 172.21.15.88:0/2373428930 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f72d4197360 con 0x7f72c006d8a0 2024-02-19T19:10:40.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.652+0000 7f72db5bc700 1 -- 172.21.15.88:0/2373428930 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72c006d8a0 msgr2=0x7f72c006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:40.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.652+0000 7f72db5bc700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72c006d8a0 0x7f72c006fd50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f72d0007900 tx=0x7f72d0008040 comp rx=0 tx=0).stop 2024-02-19T19:10:40.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.652+0000 7f72db5bc700 1 -- 172.21.15.88:0/2373428930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72d40ffc60 msgr2=0x7f72d4196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:40.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.652+0000 7f72db5bc700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72d40ffc60 0x7f72d4196480 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f72c4000c00 tx=0x7f72c40056a0 comp rx=0 tx=0).stop 2024-02-19T19:10:40.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.652+0000 7f72db5bc700 1 -- 172.21.15.88:0/2373428930 shutdown_connections 2024-02-19T19:10:40.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.652+0000 7f72db5bc700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72c006d8a0 0x7f72c006fd50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:40.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.652+0000 7f72db5bc700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72d40ff350 0x7f72d4195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:40.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.652+0000 7f72db5bc700 1 --2- 172.21.15.88:0/2373428930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72d40ffc60 0x7f72d4196480 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:40.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.652+0000 7f72db5bc700 1 -- 172.21.15.88:0/2373428930 >> 172.21.15.88:0/2373428930 conn(0x7f72d40748b0 msgr2=0x7f72d4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:40.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.652+0000 7f72db5bc700 1 -- 172.21.15.88:0/2373428930 shutdown_connections 2024-02-19T19:10:40.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.652+0000 7f72db5bc700 1 -- 172.21.15.88:0/2373428930 wait complete. 2024-02-19T19:10:40.685 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:10:40.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.762+0000 7f3ba3694700 1 -- 172.21.15.88:0/2911674074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b9c0ff350 msgr2=0x7f3b9c0ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:40.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.762+0000 7f3ba3694700 1 --2- 172.21.15.88:0/2911674074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b9c0ff350 0x7f3b9c0ff720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009910 tx=0x7f3b98009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:40.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.762+0000 7f3ba3694700 1 -- 172.21.15.88:0/2911674074 shutdown_connections 2024-02-19T19:10:40.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.762+0000 7f3ba3694700 1 --2- 172.21.15.88:0/2911674074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b9c0ffc60 0x7f3b9c10a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:40.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.762+0000 7f3ba3694700 1 --2- 172.21.15.88:0/2911674074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b9c0ff350 0x7f3b9c0ff720 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.762+0000 7f3ba3694700 1 -- 172.21.15.88:0/2911674074 >> 172.21.15.88:0/2911674074 conn(0x7f3b9c0748b0 msgr2=0x7f3b9c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.762+0000 7f3ba3694700 1 -- 172.21.15.88:0/2911674074 shutdown_connections 2024-02-19T19:10:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.762+0000 7f3ba3694700 1 -- 172.21.15.88:0/2911674074 wait complete. 2024-02-19T19:10:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.762+0000 7f3ba3694700 1 Processor -- start 2024-02-19T19:10:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba3694700 1 -- start start 2024-02-19T19:10:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba3694700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b9c0ff350 0x7f3b9c195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba3694700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b9c0ffc60 0x7f3b9c196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba3694700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c196a70 con 0x7f3b9c0ffc60 2024-02-19T19:10:40.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba3694700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c19a800 con 0x7f3b9c0ff350 2024-02-19T19:10:40.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba0c2f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b9c0ffc60 0x7f3b9c196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:40.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba0c2f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b9c0ffc60 0x7f3b9c196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42910/0 (socket says 172.21.15.88:42910) 2024-02-19T19:10:40.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba0c2f700 1 -- 172.21.15.88:0/548986585 learned_addr learned my addr 172.21.15.88:0/548986585 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:40.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba1430700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b9c0ff350 0x7f3b9c195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:40.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba0c2f700 1 -- 172.21.15.88:0/548986585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b9c0ff350 msgr2=0x7f3b9c195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba0c2f700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b9c0ff350 0x7f3b9c195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba0c2f700 1 -- 172.21.15.88:0/548986585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b980095f0 con 0x7f3b9c0ffc60 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.763+0000 7f3ba1430700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b9c0ff350 0x7f3b9c195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.764+0000 7f3ba0c2f700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b9c0ffc60 0x7f3b9c196390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00e920 tx=0x7f3b8c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.764+0000 7f3b927fc700 1 -- 172.21.15.88:0/548986585 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b8c009790 con 0x7f3b9c0ffc60 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.764+0000 7f3b927fc700 1 -- 172.21.15.88:0/548986585 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b8c00cba0 con 0x7f3b9c0ffc60 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.764+0000 7f3ba3694700 1 -- 172.21.15.88:0/548986585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9c19aae0 con 0x7f3b9c0ffc60 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.764+0000 7f3b927fc700 1 -- 172.21.15.88:0/548986585 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b8c018650 con 0x7f3b9c0ffc60 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.764+0000 7f3ba3694700 1 -- 172.21.15.88:0/548986585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9c19b030 con 0x7f3b9c0ffc60 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.765+0000 7f3ba3694700 1 -- 172.21.15.88:0/548986585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9c107dd0 con 0x7f3b9c0ffc60 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.765+0000 7f3b927fc700 1 -- 172.21.15.88:0/548986585 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3b8c00cd10 con 0x7f3b9c0ffc60 2024-02-19T19:10:40.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.766+0000 7f3b927fc700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3b8806d8a0 0x7f3b8806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:40.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.766+0000 7f3b927fc700 1 -- 172.21.15.88:0/548986585 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3b8c014070 con 0x7f3b9c0ffc60 2024-02-19T19:10:40.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.766+0000 7f3ba1430700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3b8806d8a0 0x7f3b8806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:40.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.766+0000 7f3ba1430700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3b8806d8a0 0x7f3b8806fd50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800b700 tx=0x7f3b98005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:40.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:40.770+0000 7f3b927fc700 1 -- 172.21.15.88:0/548986585 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3b8c052d20 con 0x7f3b9c0ffc60 2024-02-19T19:10:41.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.010+0000 7f3ba3694700 1 -- 172.21.15.88:0/548986585 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b9c19b310 con 0x7f3b8806d8a0 2024-02-19T19:10:41.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.012+0000 7f3b927fc700 1 -- 172.21.15.88:0/548986585 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3b9c19b310 con 0x7f3b8806d8a0 2024-02-19T19:10:41.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.016+0000 7f3ba3694700 1 -- 172.21.15.88:0/548986585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3b8806d8a0 msgr2=0x7f3b8806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.016+0000 7f3ba3694700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3b8806d8a0 0x7f3b8806fd50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b9800b700 tx=0x7f3b98005a90 comp rx=0 tx=0).stop 2024-02-19T19:10:41.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.016+0000 7f3ba3694700 1 -- 172.21.15.88:0/548986585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b9c0ffc60 msgr2=0x7f3b9c196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.016+0000 7f3ba3694700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b9c0ffc60 0x7f3b9c196390 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c00e920 tx=0x7f3b8c00ece0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.016+0000 7f3ba3694700 1 -- 172.21.15.88:0/548986585 shutdown_connections 2024-02-19T19:10:41.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.016+0000 7f3ba3694700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3b8806d8a0 0x7f3b8806fd50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.016+0000 7f3ba3694700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3b9c0ff350 0x7f3b9c195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.016+0000 7f3ba3694700 1 --2- 172.21.15.88:0/548986585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b9c0ffc60 0x7f3b9c196390 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.016+0000 7f3ba3694700 1 -- 172.21.15.88:0/548986585 >> 172.21.15.88:0/548986585 conn(0x7f3b9c0748b0 msgr2=0x7f3b9c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:41.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.016+0000 7f3ba3694700 1 -- 172.21.15.88:0/548986585 shutdown_connections 2024-02-19T19:10:41.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.016+0000 7f3ba3694700 1 -- 172.21.15.88:0/548986585 wait complete. 2024-02-19T19:10:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.129+0000 7ff7f81fe700 1 -- 172.21.15.88:0/3372876152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f00ffc60 msgr2=0x7ff7f010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.129+0000 7ff7f81fe700 1 --2- 172.21.15.88:0/3372876152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f00ffc60 0x7ff7f010a6d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0009910 tx=0x7ff7e0009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.129+0000 7ff7f81fe700 1 -- 172.21.15.88:0/3372876152 shutdown_connections 2024-02-19T19:10:41.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.129+0000 7ff7f81fe700 1 --2- 172.21.15.88:0/3372876152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f00ffc60 0x7ff7f010a6d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.129+0000 7ff7f81fe700 1 --2- 172.21.15.88:0/3372876152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f00ff350 0x7ff7f00ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.129+0000 7ff7f81fe700 1 -- 172.21.15.88:0/3372876152 >> 172.21.15.88:0/3372876152 conn(0x7ff7f00748b0 msgr2=0x7ff7f0074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:41.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.129+0000 7ff7f81fe700 1 -- 172.21.15.88:0/3372876152 shutdown_connections 2024-02-19T19:10:41.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.129+0000 7ff7f81fe700 1 -- 172.21.15.88:0/3372876152 wait complete. 2024-02-19T19:10:41.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.130+0000 7ff7f81fe700 1 Processor -- start 2024-02-19T19:10:41.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.130+0000 7ff7f81fe700 1 -- start start 2024-02-19T19:10:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.130+0000 7ff7f81fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f00ff350 0x7ff7f0195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.130+0000 7ff7f81fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f00ffc60 0x7ff7f01963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.130+0000 7ff7f81fe700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f01969f0 con 0x7ff7f00ffc60 2024-02-19T19:10:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.130+0000 7ff7f81fe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f0196b30 con 0x7ff7f00ff350 2024-02-19T19:10:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7f5799700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f00ffc60 0x7ff7f01963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7f5799700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f00ffc60 0x7ff7f01963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42924/0 (socket says 172.21.15.88:42924) 2024-02-19T19:10:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7f5799700 1 -- 172.21.15.88:0/658752182 learned_addr learned my addr 172.21.15.88:0/658752182 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7f5f9a700 1 --2- 172.21.15.88:0/658752182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f00ff350 0x7ff7f0195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7f5799700 1 -- 172.21.15.88:0/658752182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f00ff350 msgr2=0x7ff7f0195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7f5799700 1 --2- 172.21.15.88:0/658752182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f00ff350 0x7ff7f0195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7f5799700 1 -- 172.21.15.88:0/658752182 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e00095f0 con 0x7ff7f00ffc60 2024-02-19T19:10:41.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7f5799700 1 --2- 172.21.15.88:0/658752182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f00ffc60 0x7ff7f01963a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0006010 tx=0x7ff7e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:41.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7e6ffd700 1 -- 172.21.15.88:0/658752182 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e001d070 con 0x7ff7f00ffc60 2024-02-19T19:10:41.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7e6ffd700 1 -- 172.21.15.88:0/658752182 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7e000f460 con 0x7ff7f00ffc60 2024-02-19T19:10:41.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7f81fe700 1 -- 172.21.15.88:0/658752182 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f019a920 con 0x7ff7f00ffc60 2024-02-19T19:10:41.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7e6ffd700 1 -- 172.21.15.88:0/658752182 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7e0017630 con 0x7ff7f00ffc60 2024-02-19T19:10:41.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.131+0000 7ff7f81fe700 1 -- 172.21.15.88:0/658752182 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f019ae10 con 0x7ff7f00ffc60 2024-02-19T19:10:41.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.132+0000 7ff7f81fe700 1 -- 172.21.15.88:0/658752182 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f0107dd0 con 0x7ff7f00ffc60 2024-02-19T19:10:41.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.133+0000 7ff7e6ffd700 1 -- 172.21.15.88:0/658752182 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff7e000f5d0 con 0x7ff7f00ffc60 2024-02-19T19:10:41.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.133+0000 7ff7e6ffd700 1 --2- 172.21.15.88:0/658752182 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff7dc06d8a0 0x7ff7dc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:41.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.133+0000 7ff7e6ffd700 1 -- 172.21.15.88:0/658752182 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff7e008e9a0 con 0x7ff7f00ffc60 2024-02-19T19:10:41.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.133+0000 7ff7f5f9a700 1 --2- 172.21.15.88:0/658752182 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff7dc06d8a0 0x7ff7dc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:41.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.134+0000 7ff7f5f9a700 1 --2- 172.21.15.88:0/658752182 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff7dc06d8a0 0x7ff7dc06fd50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00ac50 tx=0x7ff7ec00a380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:41.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.138+0000 7ff7e6ffd700 1 -- 172.21.15.88:0/658752182 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7e00581d0 con 0x7ff7f00ffc60 2024-02-19T19:10:41.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:41 smithi088 ceph-mon[88643]: pgmap v210: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:41.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:41 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:41.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:41 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:41.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.358+0000 7ff7f81fe700 1 -- 172.21.15.88:0/658752182 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7f019b080 con 0x7ff7dc06d8a0 2024-02-19T19:10:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.369+0000 7ff7e6ffd700 1 -- 172.21.15.88:0/658752182 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff7f019b080 con 0x7ff7dc06d8a0 2024-02-19T19:10:41.390 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (7m) 0s ago 18m 24.2M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (18m) 0s ago 18m 9143k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (40s) 19s ago 16m 7675k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (3m) 0s ago 18m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (3m) 19s ago 16m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (7m) 0s ago 17m 85.4M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (6s) 0s ago 12m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (11m) 0s ago 11m 11.3M - 18.2.0 dc2bc1663786 7fdd0e0d43a6 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (12m) 19s ago 12m 112M - 18.2.0 dc2bc1663786 30627b9788f2 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (11m) 19s ago 11m 21.6M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (9m) 0s ago 19m 508M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (8m) 19s ago 16m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (5m) 0s ago 20m 56.7M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (5m) 19s ago 16m 42.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:10:41.391 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (8m) 0s ago 18m 16.3M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:10:41.392 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (8m) 19s ago 16m 15.2M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:10:41.392 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (3m) 0s ago 15m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:10:41.392 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (2m) 0s ago 15m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:10:41.392 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (2m) 0s ago 14m 85.2M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:10:41.392 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (91s) 19s ago 14m 102M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:10:41.392 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (57s) 19s ago 13m 102M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:10:41.392 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (22s) 19s ago 13m 12.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:10:41.392 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (8m) 0s ago 17m 69.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:10:41.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.371+0000 7ff7f81fe700 1 -- 172.21.15.88:0/658752182 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff7dc06d8a0 msgr2=0x7ff7dc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.371+0000 7ff7f81fe700 1 --2- 172.21.15.88:0/658752182 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff7dc06d8a0 0x7ff7dc06fd50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec00ac50 tx=0x7ff7ec00a380 comp rx=0 tx=0).stop 2024-02-19T19:10:41.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.371+0000 7ff7f81fe700 1 -- 172.21.15.88:0/658752182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f00ffc60 msgr2=0x7ff7f01963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.371+0000 7ff7f81fe700 1 --2- 172.21.15.88:0/658752182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f00ffc60 0x7ff7f01963a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0006010 tx=0x7ff7e000bd70 comp rx=0 tx=0).stop 2024-02-19T19:10:41.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.371+0000 7ff7f81fe700 1 -- 172.21.15.88:0/658752182 shutdown_connections 2024-02-19T19:10:41.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.371+0000 7ff7f81fe700 1 --2- 172.21.15.88:0/658752182 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff7dc06d8a0 0x7ff7dc06fd50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.371+0000 7ff7f81fe700 1 --2- 172.21.15.88:0/658752182 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7f00ff350 0x7ff7f0195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.371+0000 7ff7f81fe700 1 --2- 172.21.15.88:0/658752182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7f00ffc60 0x7ff7f01963a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.371+0000 7ff7f81fe700 1 -- 172.21.15.88:0/658752182 >> 172.21.15.88:0/658752182 conn(0x7ff7f00748b0 msgr2=0x7ff7f0104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:41.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.371+0000 7ff7f81fe700 1 -- 172.21.15.88:0/658752182 shutdown_connections 2024-02-19T19:10:41.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.371+0000 7ff7f81fe700 1 -- 172.21.15.88:0/658752182 wait complete. 2024-02-19T19:10:41.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.478+0000 7fc27edef700 1 -- 172.21.15.88:0/4186617948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc278104130 msgr2=0x7fc278104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.478+0000 7fc27edef700 1 --2- 172.21.15.88:0/4186617948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc278104130 0x7fc278104500 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc26c009910 tx=0x7fc26c009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.479+0000 7fc27edef700 1 -- 172.21.15.88:0/4186617948 shutdown_connections 2024-02-19T19:10:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.479+0000 7fc27edef700 1 --2- 172.21.15.88:0/4186617948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2780ffbd0 0x7fc278100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.479+0000 7fc27edef700 1 --2- 172.21.15.88:0/4186617948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc278104130 0x7fc278104500 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.479+0000 7fc27edef700 1 -- 172.21.15.88:0/4186617948 >> 172.21.15.88:0/4186617948 conn(0x7fc278075130 msgr2=0x7fc278075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.479+0000 7fc27edef700 1 -- 172.21.15.88:0/4186617948 shutdown_connections 2024-02-19T19:10:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.479+0000 7fc27edef700 1 -- 172.21.15.88:0/4186617948 wait complete. 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.479+0000 7fc27edef700 1 Processor -- start 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.479+0000 7fc27edef700 1 -- start start 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc27edef700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2780ffbd0 0x7fc278194e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc27edef700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc278104130 0x7fc278195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc27edef700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc278195a60 con 0x7fc278104130 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc27edef700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2781997f0 con 0x7fc2780ffbd0 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc277fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc278104130 0x7fc278195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc277fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc278104130 0x7fc278195380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42946/0 (socket says 172.21.15.88:42946) 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc277fff700 1 -- 172.21.15.88:0/369300698 learned_addr learned my addr 172.21.15.88:0/369300698 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc27cb8b700 1 --2- 172.21.15.88:0/369300698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2780ffbd0 0x7fc278194e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc277fff700 1 -- 172.21.15.88:0/369300698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2780ffbd0 msgr2=0x7fc278194e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc277fff700 1 --2- 172.21.15.88:0/369300698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2780ffbd0 0x7fc278194e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc277fff700 1 -- 172.21.15.88:0/369300698 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c0095f0 con 0x7fc278104130 2024-02-19T19:10:41.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc277fff700 1 --2- 172.21.15.88:0/369300698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc278104130 0x7fc278195380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc26400b4b0 tx=0x7fc26400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:41.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc275ffb700 1 -- 172.21.15.88:0/369300698 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc264011800 con 0x7fc278104130 2024-02-19T19:10:41.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.480+0000 7fc275ffb700 1 -- 172.21.15.88:0/369300698 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc264011e20 con 0x7fc278104130 2024-02-19T19:10:41.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.481+0000 7fc27edef700 1 -- 172.21.15.88:0/369300698 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc278199ad0 con 0x7fc278104130 2024-02-19T19:10:41.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.481+0000 7fc275ffb700 1 -- 172.21.15.88:0/369300698 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc26400f2e0 con 0x7fc278104130 2024-02-19T19:10:41.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.481+0000 7fc27edef700 1 -- 172.21.15.88:0/369300698 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27819a020 con 0x7fc278104130 2024-02-19T19:10:41.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:41 smithi132 ceph-mon[70676]: pgmap v210: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:41.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:41 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:41.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:41 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:41.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.482+0000 7fc27edef700 1 -- 172.21.15.88:0/369300698 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27804f2a0 con 0x7fc278104130 2024-02-19T19:10:41.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.482+0000 7fc275ffb700 1 -- 172.21.15.88:0/369300698 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc264010380 con 0x7fc278104130 2024-02-19T19:10:41.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.483+0000 7fc275ffb700 1 --2- 172.21.15.88:0/369300698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc26806fab0 0x7fc268071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:41.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.483+0000 7fc275ffb700 1 -- 172.21.15.88:0/369300698 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc26408d060 con 0x7fc278104130 2024-02-19T19:10:41.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.483+0000 7fc27cb8b700 1 --2- 172.21.15.88:0/369300698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc26806fab0 0x7fc268071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:41.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.483+0000 7fc27cb8b700 1 --2- 172.21.15.88:0/369300698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc26806fab0 0x7fc268071f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00b5c0 tx=0x7fc26c005460 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:41.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.487+0000 7fc275ffb700 1 -- 172.21.15.88:0/369300698 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc264056940 con 0x7fc278104130 2024-02-19T19:10:41.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.783+0000 7fc27edef700 1 -- 172.21.15.88:0/369300698 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc27819a390 con 0x7fc278104130 2024-02-19T19:10:41.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.784+0000 7fc275ffb700 1 -- 172.21.15.88:0/369300698 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fc26405a000 con 0x7fc278104130 2024-02-19T19:10:41.805 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-02-19T19:10:41.806 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 1 2024-02-19T19:10:41.807 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:10:41.807 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:10:41.807 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-02-19T19:10:41.807 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 11 2024-02-19T19:10:41.807 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:10:41.807 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:10:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.787+0000 7fc27edef700 1 -- 172.21.15.88:0/369300698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc26806fab0 msgr2=0x7fc268071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.787+0000 7fc27edef700 1 --2- 172.21.15.88:0/369300698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc26806fab0 0x7fc268071f60 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc26c00b5c0 tx=0x7fc26c005460 comp rx=0 tx=0).stop 2024-02-19T19:10:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.787+0000 7fc27edef700 1 -- 172.21.15.88:0/369300698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc278104130 msgr2=0x7fc278195380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.787+0000 7fc27edef700 1 --2- 172.21.15.88:0/369300698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc278104130 0x7fc278195380 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc26400b4b0 tx=0x7fc26400b870 comp rx=0 tx=0).stop 2024-02-19T19:10:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.787+0000 7fc27edef700 1 -- 172.21.15.88:0/369300698 shutdown_connections 2024-02-19T19:10:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.787+0000 7fc27edef700 1 --2- 172.21.15.88:0/369300698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc26806fab0 0x7fc268071f60 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.787+0000 7fc27edef700 1 --2- 172.21.15.88:0/369300698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc2780ffbd0 0x7fc278194e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.787+0000 7fc27edef700 1 --2- 172.21.15.88:0/369300698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc278104130 0x7fc278195380 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.787+0000 7fc27edef700 1 -- 172.21.15.88:0/369300698 >> 172.21.15.88:0/369300698 conn(0x7fc278075130 msgr2=0x7fc2780fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:41.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.787+0000 7fc27edef700 1 -- 172.21.15.88:0/369300698 shutdown_connections 2024-02-19T19:10:41.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.787+0000 7fc27edef700 1 -- 172.21.15.88:0/369300698 wait complete. 2024-02-19T19:10:41.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.899+0000 7f9f7a184700 1 -- 172.21.15.88:0/1447240396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f741004c0 msgr2=0x7f9f74100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.899+0000 7f9f7a184700 1 --2- 172.21.15.88:0/1447240396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f741004c0 0x7f9f74100890 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9f64009910 tx=0x7f9f64009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:41.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.899+0000 7f9f7a184700 1 -- 172.21.15.88:0/1447240396 shutdown_connections 2024-02-19T19:10:41.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.899+0000 7f9f7a184700 1 --2- 172.21.15.88:0/1447240396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f74100e60 0x7f9f74104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.899+0000 7f9f7a184700 1 --2- 172.21.15.88:0/1447240396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f741004c0 0x7f9f74100890 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.899+0000 7f9f7a184700 1 -- 172.21.15.88:0/1447240396 >> 172.21.15.88:0/1447240396 conn(0x7f9f740fbd80 msgr2=0x7f9f740fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:41.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.899+0000 7f9f7a184700 1 -- 172.21.15.88:0/1447240396 shutdown_connections 2024-02-19T19:10:41.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.899+0000 7f9f7a184700 1 -- 172.21.15.88:0/1447240396 wait complete. 2024-02-19T19:10:41.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.900+0000 7f9f7a184700 1 Processor -- start 2024-02-19T19:10:41.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.900+0000 7f9f7a184700 1 -- start start 2024-02-19T19:10:41.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.900+0000 7f9f7a184700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f741004c0 0x7f9f74195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:41.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.900+0000 7f9f7a184700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f74100e60 0x7f9f74196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:41.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.900+0000 7f9f7a184700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f74196b50 con 0x7f9f741004c0 2024-02-19T19:10:41.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.900+0000 7f9f7a184700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f7419a8e0 con 0x7f9f74100e60 2024-02-19T19:10:41.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.900+0000 7f9f737fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f741004c0 0x7f9f74195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:41.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.900+0000 7f9f737fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f741004c0 0x7f9f74195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42970/0 (socket says 172.21.15.88:42970) 2024-02-19T19:10:41.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.900+0000 7f9f737fe700 1 -- 172.21.15.88:0/3058038671 learned_addr learned my addr 172.21.15.88:0/3058038671 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:41.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.900+0000 7f9f72ffd700 1 --2- 172.21.15.88:0/3058038671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f74100e60 0x7f9f74196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:41.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.901+0000 7f9f737fe700 1 -- 172.21.15.88:0/3058038671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f74100e60 msgr2=0x7f9f74196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:41.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.901+0000 7f9f737fe700 1 --2- 172.21.15.88:0/3058038671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f74100e60 0x7f9f74196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:41.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.901+0000 7f9f737fe700 1 -- 172.21.15.88:0/3058038671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f640095f0 con 0x7f9f741004c0 2024-02-19T19:10:41.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.901+0000 7f9f737fe700 1 --2- 172.21.15.88:0/3058038671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f741004c0 0x7f9f74195f30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9f64006010 tx=0x7f9f6400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:41.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.901+0000 7f9f70ff9700 1 -- 172.21.15.88:0/3058038671 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f6401d070 con 0x7f9f741004c0 2024-02-19T19:10:41.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.901+0000 7f9f70ff9700 1 -- 172.21.15.88:0/3058038671 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f64005dc0 con 0x7f9f741004c0 2024-02-19T19:10:41.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.901+0000 7f9f7a184700 1 -- 172.21.15.88:0/3058038671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f7419ab60 con 0x7f9f741004c0 2024-02-19T19:10:41.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.901+0000 7f9f70ff9700 1 -- 172.21.15.88:0/3058038671 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f64017400 con 0x7f9f741004c0 2024-02-19T19:10:41.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.901+0000 7f9f7a184700 1 -- 172.21.15.88:0/3058038671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f7419b050 con 0x7f9f741004c0 2024-02-19T19:10:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.902+0000 7f9f7a184700 1 -- 172.21.15.88:0/3058038671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f741087d0 con 0x7f9f741004c0 2024-02-19T19:10:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.903+0000 7f9f70ff9700 1 -- 172.21.15.88:0/3058038671 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9f64003a60 con 0x7f9f741004c0 2024-02-19T19:10:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.903+0000 7f9f70ff9700 1 --2- 172.21.15.88:0/3058038671 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9f5c06d8a0 0x7f9f5c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.903+0000 7f9f70ff9700 1 -- 172.21.15.88:0/3058038671 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9f6408e9e0 con 0x7f9f741004c0 2024-02-19T19:10:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.903+0000 7f9f72ffd700 1 --2- 172.21.15.88:0/3058038671 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9f5c06d8a0 0x7f9f5c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.904+0000 7f9f72ffd700 1 --2- 172.21.15.88:0/3058038671 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9f5c06d8a0 0x7f9f5c06fd50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f74197550 tx=0x7f9f6800b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:41.908+0000 7f9f70ff9700 1 -- 172.21.15.88:0/3058038671 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9f64058190 con 0x7f9f741004c0 2024-02-19T19:10:42.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.173+0000 7f9f7a184700 1 -- 172.21.15.88:0/3058038671 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9f7404f2a0 con 0x7f9f741004c0 2024-02-19T19:10:42.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.174+0000 7f9f70ff9700 1 -- 172.21.15.88:0/3058038671 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1839 (secure 0 0 0) 0x7f9f6405b850 con 0x7f9f741004c0 2024-02-19T19:10:42.195 INFO:teuthology.orchestra.run.smithi088.stdout:e26 2024-02-19T19:10:42.195 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:10:42.196 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:10:42.197 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:10:42.198 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:10:42.198 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:10:42.198 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:10:42.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:10:42.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:10:42.198 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:10:42.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:10:42.198 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:10:42.198 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34426} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/3440020652,v1:172.21.15.88:6829/3440020652] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:10:42.198 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:10:42.198 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44339} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/1971026183,v1:172.21.15.132:6825/1971026183] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:10:42.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.176+0000 7f9f7a184700 1 -- 172.21.15.88:0/3058038671 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9f5c06d8a0 msgr2=0x7f9f5c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:42.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.176+0000 7f9f7a184700 1 --2- 172.21.15.88:0/3058038671 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9f5c06d8a0 0x7f9f5c06fd50 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9f74197550 tx=0x7f9f6800b410 comp rx=0 tx=0).stop 2024-02-19T19:10:42.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.176+0000 7f9f7a184700 1 -- 172.21.15.88:0/3058038671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f741004c0 msgr2=0x7f9f74195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:42.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.176+0000 7f9f7a184700 1 --2- 172.21.15.88:0/3058038671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f741004c0 0x7f9f74195f30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9f64006010 tx=0x7f9f6400bd70 comp rx=0 tx=0).stop 2024-02-19T19:10:42.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.176+0000 7f9f7a184700 1 -- 172.21.15.88:0/3058038671 shutdown_connections 2024-02-19T19:10:42.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.176+0000 7f9f7a184700 1 --2- 172.21.15.88:0/3058038671 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9f5c06d8a0 0x7f9f5c06fd50 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.176+0000 7f9f7a184700 1 --2- 172.21.15.88:0/3058038671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f741004c0 0x7f9f74195f30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.176+0000 7f9f7a184700 1 --2- 172.21.15.88:0/3058038671 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9f74100e60 0x7f9f74196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.176+0000 7f9f7a184700 1 -- 172.21.15.88:0/3058038671 >> 172.21.15.88:0/3058038671 conn(0x7f9f740fbd80 msgr2=0x7f9f740fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:42.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.176+0000 7f9f7a184700 1 -- 172.21.15.88:0/3058038671 shutdown_connections 2024-02-19T19:10:42.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.176+0000 7f9f7a184700 1 -- 172.21.15.88:0/3058038671 wait complete. 2024-02-19T19:10:42.200 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 26 2024-02-19T19:10:42.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.284+0000 7f78f9138700 1 -- 172.21.15.88:0/261482278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78f4101460 msgr2=0x7f78f4101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:42.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.284+0000 7f78f9138700 1 --2- 172.21.15.88:0/261482278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78f4101460 0x7f78f4101830 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f78e8009910 tx=0x7f78e8009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:42.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.284+0000 7f78f9138700 1 -- 172.21.15.88:0/261482278 shutdown_connections 2024-02-19T19:10:42.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.284+0000 7f78f9138700 1 --2- 172.21.15.88:0/261482278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78f4101d70 0x7f78f410a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.284+0000 7f78f9138700 1 --2- 172.21.15.88:0/261482278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78f4101460 0x7f78f4101830 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.284+0000 7f78f9138700 1 -- 172.21.15.88:0/261482278 >> 172.21.15.88:0/261482278 conn(0x7f78f40fac80 msgr2=0x7f78f40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:42.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.284+0000 7f78f9138700 1 -- 172.21.15.88:0/261482278 shutdown_connections 2024-02-19T19:10:42.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.284+0000 7f78f9138700 1 -- 172.21.15.88:0/261482278 wait complete. 2024-02-19T19:10:42.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.285+0000 7f78f9138700 1 Processor -- start 2024-02-19T19:10:42.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.285+0000 7f78f9138700 1 -- start start 2024-02-19T19:10:42.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f9138700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78f4101460 0x7f78f40ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:42.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f9138700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78f4101d70 0x7f78f40ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:42.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f9138700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40ffb40 con 0x7f78f4101460 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f9138700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f40ffc80 con 0x7f78f4101d70 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f2d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78f4101460 0x7f78f40ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f2d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78f4101460 0x7f78f40ff0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42986/0 (socket says 172.21.15.88:42986) 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f2d9d700 1 -- 172.21.15.88:0/774013304 learned_addr learned my addr 172.21.15.88:0/774013304 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f2d9d700 1 -- 172.21.15.88:0/774013304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78f4101d70 msgr2=0x7f78f40ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f2d9d700 1 --2- 172.21.15.88:0/774013304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78f4101d70 0x7f78f40ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f2d9d700 1 -- 172.21.15.88:0/774013304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e80095f0 con 0x7f78f4101460 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f2d9d700 1 --2- 172.21.15.88:0/774013304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78f4101460 0x7f78f40ff0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f78e8006010 tx=0x7f78e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78dbfff700 1 -- 172.21.15.88:0/774013304 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e801d070 con 0x7f78f4101460 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f9138700 1 -- 172.21.15.88:0/774013304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f41a2130 con 0x7f78f4101460 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78dbfff700 1 -- 172.21.15.88:0/774013304 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78e8004fb0 con 0x7f78f4101460 2024-02-19T19:10:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78dbfff700 1 -- 172.21.15.88:0/774013304 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e8022ea0 con 0x7f78f4101460 2024-02-19T19:10:42.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.286+0000 7f78f9138700 1 -- 172.21.15.88:0/774013304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f41a2540 con 0x7f78f4101460 2024-02-19T19:10:42.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.287+0000 7f78f9138700 1 -- 172.21.15.88:0/774013304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f4107920 con 0x7f78f4101460 2024-02-19T19:10:42.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.288+0000 7f78dbfff700 1 -- 172.21.15.88:0/774013304 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f78e8022470 con 0x7f78f4101460 2024-02-19T19:10:42.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.288+0000 7f78dbfff700 1 --2- 172.21.15.88:0/774013304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f78dc06d8a0 0x7f78dc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:42.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.288+0000 7f78dbfff700 1 -- 172.21.15.88:0/774013304 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f78e808ea40 con 0x7f78f4101460 2024-02-19T19:10:42.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.288+0000 7f78f259c700 1 --2- 172.21.15.88:0/774013304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f78dc06d8a0 0x7f78dc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:42.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.289+0000 7f78f259c700 1 --2- 172.21.15.88:0/774013304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f78dc06d8a0 0x7f78dc06fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f78f41008b0 tx=0x7f78e4006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:42.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.293+0000 7f78dbfff700 1 -- 172.21.15.88:0/774013304 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f78e80582a0 con 0x7f78f4101460 2024-02-19T19:10:42.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:42 smithi088 ceph-mon[88643]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:42.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:42 smithi088 ceph-mon[88643]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:42.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:42 smithi088 ceph-mon[88643]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:42.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:42.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:42.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:42 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/369300698' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:42.446 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:42 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3058038671' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:10:42.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.520+0000 7f78f9138700 1 -- 172.21.15.88:0/774013304 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f78f4061190 con 0x7f78dc06d8a0 2024-02-19T19:10:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.522+0000 7f78dbfff700 1 -- 172.21.15.88:0/774013304 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f78f4061190 con 0x7f78dc06d8a0 2024-02-19T19:10:42.543 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:10:42.543 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:10:42.543 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "13/23 daemons upgraded", 2024-02-19T19:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:10:42.544 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:10:42.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.524+0000 7f78f9138700 1 -- 172.21.15.88:0/774013304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f78dc06d8a0 msgr2=0x7f78dc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:42.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.524+0000 7f78f9138700 1 --2- 172.21.15.88:0/774013304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f78dc06d8a0 0x7f78dc06fd50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f78f41008b0 tx=0x7f78e4006d20 comp rx=0 tx=0).stop 2024-02-19T19:10:42.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.524+0000 7f78f9138700 1 -- 172.21.15.88:0/774013304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78f4101460 msgr2=0x7f78f40ff0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:42.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.524+0000 7f78f9138700 1 --2- 172.21.15.88:0/774013304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78f4101460 0x7f78f40ff0c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f78e8006010 tx=0x7f78e800be20 comp rx=0 tx=0).stop 2024-02-19T19:10:42.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.524+0000 7f78f9138700 1 -- 172.21.15.88:0/774013304 shutdown_connections 2024-02-19T19:10:42.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.524+0000 7f78f9138700 1 --2- 172.21.15.88:0/774013304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f78dc06d8a0 0x7f78dc06fd50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.524+0000 7f78f9138700 1 --2- 172.21.15.88:0/774013304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78f4101460 0x7f78f40ff0c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.524+0000 7f78f9138700 1 --2- 172.21.15.88:0/774013304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f78f4101d70 0x7f78f40ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.524+0000 7f78f9138700 1 -- 172.21.15.88:0/774013304 >> 172.21.15.88:0/774013304 conn(0x7f78f40fac80 msgr2=0x7f78f4104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:42.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.524+0000 7f78f9138700 1 -- 172.21.15.88:0/774013304 shutdown_connections 2024-02-19T19:10:42.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.524+0000 7f78f9138700 1 -- 172.21.15.88:0/774013304 wait complete. 2024-02-19T19:10:42.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.631+0000 7f59c05e2700 1 -- 172.21.15.88:0/2498197380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59b80ff350 msgr2=0x7f59b80ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:42.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.631+0000 7f59c05e2700 1 --2- 172.21.15.88:0/2498197380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59b80ff350 0x7f59b80ff720 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f59b4009910 tx=0x7f59b4009c20 comp rx=0 tx=0).stop 2024-02-19T19:10:42.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.631+0000 7f59c05e2700 1 -- 172.21.15.88:0/2498197380 shutdown_connections 2024-02-19T19:10:42.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.631+0000 7f59c05e2700 1 --2- 172.21.15.88:0/2498197380 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59b80ffc60 0x7f59b810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.631+0000 7f59c05e2700 1 --2- 172.21.15.88:0/2498197380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59b80ff350 0x7f59b80ff720 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.631+0000 7f59c05e2700 1 -- 172.21.15.88:0/2498197380 >> 172.21.15.88:0/2498197380 conn(0x7f59b80748b0 msgr2=0x7f59b8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.631+0000 7f59c05e2700 1 -- 172.21.15.88:0/2498197380 shutdown_connections 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.632+0000 7f59c05e2700 1 -- 172.21.15.88:0/2498197380 wait complete. 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.632+0000 7f59c05e2700 1 Processor -- start 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.632+0000 7f59c05e2700 1 -- start start 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.632+0000 7f59c05e2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59b80ff350 0x7f59b8100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.632+0000 7f59c05e2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59b80ffc60 0x7f59b8100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.632+0000 7f59c05e2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8100fb0 con 0x7f59b80ff350 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.632+0000 7f59c05e2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b81010f0 con 0x7f59b80ffc60 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.632+0000 7f59be37e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59b80ff350 0x7f59b8100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.632+0000 7f59be37e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59b80ff350 0x7f59b8100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43002/0 (socket says 172.21.15.88:43002) 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.632+0000 7f59be37e700 1 -- 172.21.15.88:0/3528773792 learned_addr learned my addr 172.21.15.88:0/3528773792 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.633+0000 7f59bdb7d700 1 --2- 172.21.15.88:0/3528773792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59b80ffc60 0x7f59b8100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.633+0000 7f59be37e700 1 -- 172.21.15.88:0/3528773792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59b80ffc60 msgr2=0x7f59b8100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:42.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.633+0000 7f59be37e700 1 --2- 172.21.15.88:0/3528773792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59b80ffc60 0x7f59b8100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.633+0000 7f59be37e700 1 -- 172.21.15.88:0/3528773792 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b40095f0 con 0x7f59b80ff350 2024-02-19T19:10:42.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.633+0000 7f59be37e700 1 --2- 172.21.15.88:0/3528773792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59b80ff350 0x7f59b8100530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f59b400bda0 tx=0x7f59b400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:42.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.633+0000 7f59af7fe700 1 -- 172.21.15.88:0/3528773792 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59b401d070 con 0x7f59b80ff350 2024-02-19T19:10:42.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.633+0000 7f59af7fe700 1 -- 172.21.15.88:0/3528773792 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59b4004fb0 con 0x7f59b80ff350 2024-02-19T19:10:42.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.633+0000 7f59c05e2700 1 -- 172.21.15.88:0/3528773792 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b8101370 con 0x7f59b80ff350 2024-02-19T19:10:42.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.633+0000 7f59af7fe700 1 -- 172.21.15.88:0/3528773792 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59b400f460 con 0x7f59b80ff350 2024-02-19T19:10:42.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.633+0000 7f59c05e2700 1 -- 172.21.15.88:0/3528773792 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b81a2350 con 0x7f59b80ff350 2024-02-19T19:10:42.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.634+0000 7f59c05e2700 1 -- 172.21.15.88:0/3528773792 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b8107dd0 con 0x7f59b80ff350 2024-02-19T19:10:42.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.634+0000 7f59af7fe700 1 -- 172.21.15.88:0/3528773792 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f59b4022800 con 0x7f59b80ff350 2024-02-19T19:10:42.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.635+0000 7f59af7fe700 1 --2- 172.21.15.88:0/3528773792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f59a406d8a0 0x7f59a406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:10:42.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.635+0000 7f59af7fe700 1 -- 172.21.15.88:0/3528773792 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f59b408ea70 con 0x7f59b80ff350 2024-02-19T19:10:42.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.635+0000 7f59bdb7d700 1 --2- 172.21.15.88:0/3528773792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f59a406d8a0 0x7f59a406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:10:42.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.635+0000 7f59bdb7d700 1 --2- 172.21.15.88:0/3528773792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f59a406d8a0 0x7f59a406fd50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005ea0 tx=0x7f59a8005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:10:42.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.640+0000 7f59af7fe700 1 -- 172.21.15.88:0/3528773792 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f59b40583f0 con 0x7f59b80ff350 2024-02-19T19:10:42.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:42 smithi132 ceph-mon[70676]: from='client.34434 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:42.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:42 smithi132 ceph-mon[70676]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:42.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:42 smithi132 ceph-mon[70676]: from='client.34442 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:42.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:42.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:42.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:42 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/369300698' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:42.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:42 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3058038671' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:10:42.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.923+0000 7f59c05e2700 1 -- 172.21.15.88:0/3528773792 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f59b81077b0 con 0x7f59b80ff350 2024-02-19T19:10:42.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.924+0000 7f59af7fe700 1 -- 172.21.15.88:0/3528773792 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f59b408ee60 con 0x7f59b80ff350 2024-02-19T19:10:42.946 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:10:42.946 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:10:42.946 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:10:42.946 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:10:42.946 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:10:42.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.926+0000 7f59c05e2700 1 -- 172.21.15.88:0/3528773792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f59a406d8a0 msgr2=0x7f59a406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:42.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.926+0000 7f59c05e2700 1 --2- 172.21.15.88:0/3528773792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f59a406d8a0 0x7f59a406fd50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005ea0 tx=0x7f59a8005e30 comp rx=0 tx=0).stop 2024-02-19T19:10:42.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.926+0000 7f59c05e2700 1 -- 172.21.15.88:0/3528773792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59b80ff350 msgr2=0x7f59b8100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:10:42.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.926+0000 7f59c05e2700 1 --2- 172.21.15.88:0/3528773792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59b80ff350 0x7f59b8100530 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f59b400bda0 tx=0x7f59b400be80 comp rx=0 tx=0).stop 2024-02-19T19:10:42.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.926+0000 7f59c05e2700 1 -- 172.21.15.88:0/3528773792 shutdown_connections 2024-02-19T19:10:42.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.926+0000 7f59c05e2700 1 --2- 172.21.15.88:0/3528773792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f59a406d8a0 0x7f59a406fd50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.926+0000 7f59c05e2700 1 --2- 172.21.15.88:0/3528773792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59b80ff350 0x7f59b8100530 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.926+0000 7f59c05e2700 1 --2- 172.21.15.88:0/3528773792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59b80ffc60 0x7f59b8100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:10:42.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.926+0000 7f59c05e2700 1 -- 172.21.15.88:0/3528773792 >> 172.21.15.88:0/3528773792 conn(0x7f59b80748b0 msgr2=0x7f59b8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:10:42.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.926+0000 7f59c05e2700 1 -- 172.21.15.88:0/3528773792 shutdown_connections 2024-02-19T19:10:42.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:10:42.926+0000 7f59c05e2700 1 -- 172.21.15.88:0/3528773792 wait complete. 2024-02-19T19:10:43.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:43 smithi088 ceph-mon[88643]: pgmap v211: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:43.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:43 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3528773792' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:10:43.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:43.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:10:43.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:43 smithi132 ceph-mon[70676]: pgmap v211: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:43.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:43 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3528773792' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:10:43.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:43.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:10:44.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:44 smithi088 ceph-mon[88643]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:44.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:44 smithi132 ceph-mon[70676]: from='client.34454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:10:45.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:45 smithi088 ceph-mon[88643]: pgmap v212: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:45.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:45.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:45.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:10:45.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:10:45.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:45 smithi132 ceph-mon[70676]: pgmap v212: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:10:45.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:10:45.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:46.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:10:46.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:46.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:46.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:46.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:46.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:10:46.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:46.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:46.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:46.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:10:47.560 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:47 smithi088 ceph-mon[88643]: pgmap v213: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:47.560 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:47.560 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.psjiah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:10:47.560 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:10:47.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:47 smithi132 ceph-mon[70676]: pgmap v213: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:47.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:47.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi088.psjiah", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:10:47.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:10:48.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:48 smithi088 ceph-mon[88643]: Upgrade: Updating mds.cephfs.smithi088.psjiah 2024-02-19T19:10:48.604 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:48 smithi088 ceph-mon[88643]: Deploying daemon mds.cephfs.smithi088.psjiah on smithi088 2024-02-19T19:10:48.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:48 smithi132 ceph-mon[70676]: Upgrade: Updating mds.cephfs.smithi088.psjiah 2024-02-19T19:10:48.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:48 smithi132 ceph-mon[70676]: Deploying daemon mds.cephfs.smithi088.psjiah on smithi088 2024-02-19T19:10:49.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:49 smithi088 ceph-mon[88643]: pgmap v214: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:49.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:49 smithi132 ceph-mon[70676]: pgmap v214: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:50.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:50 smithi088 ceph-mon[88643]: osdmap e86: 6 total, 6 up, 6 in 2024-02-19T19:10:50.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:50 smithi088 ceph-mon[88643]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-19T19:10:50.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:50 smithi132 ceph-mon[70676]: osdmap e86: 6 total, 6 up, 6 in 2024-02-19T19:10:50.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:50 smithi132 ceph-mon[70676]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-19T19:10:51.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:51 smithi088 ceph-mon[88643]: pgmap v216: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:51.655 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:51 smithi132 ceph-mon[70676]: pgmap v216: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:53.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:53 smithi088 ceph-mon[88643]: pgmap v217: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:53.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:53 smithi132 ceph-mon[70676]: pgmap v217: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:55.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:55.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:55.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:10:55.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:55 smithi088 ceph-mon[88643]: pgmap v218: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:55.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:55 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] up:boot 2024-02-19T19:10:55.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:55 smithi088 ceph-mon[88643]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:10:55.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:10:55.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:55.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:55.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:10:55.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:55 smithi132 ceph-mon[70676]: pgmap v218: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:55.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:55 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] up:boot 2024-02-19T19:10:55.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:55 smithi132 ceph-mon[70676]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:10:55.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:10:57.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:57 smithi088 ceph-mon[88643]: pgmap v219: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:57.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:57 smithi132 ceph-mon[70676]: pgmap v219: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:58.520 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:10:58.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:10:59.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:59 smithi088 ceph-mon[88643]: pgmap v220: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:59.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:59.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:10:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:59.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:59 smithi132 ceph-mon[70676]: pgmap v220: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:10:59.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:10:59.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:10:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:01.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:01 smithi088 ceph-mon[88643]: pgmap v221: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:01.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:01 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:01.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:01 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:01.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:01 smithi132 ceph-mon[70676]: pgmap v221: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:01.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:01 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:01.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:01 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:03.539 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:03 smithi088 ceph-mon[88643]: pgmap v222: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:03.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:03 smithi132 ceph-mon[70676]: pgmap v222: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:05.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:05.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:05.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:11:05.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:11:05.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:05.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:11:05.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:05.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:05.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:05.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:05.103 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:04 smithi088 ceph-mon[88643]: pgmap v223: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:05.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:05.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:05.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:11:05.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:11:05.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:05.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:11:05.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:05.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:05.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:05.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:05.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:04 smithi132 ceph-mon[70676]: pgmap v223: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:07.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:06 smithi132 ceph-mon[70676]: Upgrade: Updating mds.cephfs.smithi132.afhgdi 2024-02-19T19:11:07.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:07.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.afhgdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:11:07.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:11:07.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:06 smithi132 ceph-mon[70676]: Deploying daemon mds.cephfs.smithi132.afhgdi on smithi132 2024-02-19T19:11:07.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:06 smithi132 ceph-mon[70676]: pgmap v224: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:07.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:06 smithi088 ceph-mon[88643]: Upgrade: Updating mds.cephfs.smithi132.afhgdi 2024-02-19T19:11:07.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:07.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.afhgdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:11:07.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:11:07.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:06 smithi088 ceph-mon[88643]: Deploying daemon mds.cephfs.smithi132.afhgdi on smithi132 2024-02-19T19:11:07.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:06 smithi088 ceph-mon[88643]: pgmap v224: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:09.255 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:08 smithi132 ceph-mon[70676]: osdmap e87: 6 total, 6 up, 6 in 2024-02-19T19:11:09.256 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:08 smithi132 ceph-mon[70676]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-19T19:11:09.256 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:08 smithi132 ceph-mon[70676]: pgmap v226: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:09.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:08 smithi088 ceph-mon[88643]: osdmap e87: 6 total, 6 up, 6 in 2024-02-19T19:11:09.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:08 smithi088 ceph-mon[88643]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-19T19:11:09.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:08 smithi088 ceph-mon[88643]: pgmap v226: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:10.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:11:10.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:11:11.253 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:10 smithi088 ceph-mon[88643]: pgmap v227: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:11.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:10 smithi132 ceph-mon[70676]: pgmap v227: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:13.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.041+0000 7f8044b11700 1 -- 172.21.15.88:0/107778814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8040101420 msgr2=0x7f80401017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.041+0000 7f8044b11700 1 --2- 172.21.15.88:0/107778814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8040101420 0x7f80401017f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8030009910 tx=0x7f8030009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:13.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.041+0000 7f8044b11700 1 -- 172.21.15.88:0/107778814 shutdown_connections 2024-02-19T19:11:13.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.041+0000 7f8044b11700 1 --2- 172.21.15.88:0/107778814 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8040101d30 0x7f804010a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.041+0000 7f8044b11700 1 --2- 172.21.15.88:0/107778814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8040101420 0x7f80401017f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.041+0000 7f8044b11700 1 -- 172.21.15.88:0/107778814 >> 172.21.15.88:0/107778814 conn(0x7f80400fac80 msgr2=0x7f80400fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.041+0000 7f8044b11700 1 -- 172.21.15.88:0/107778814 shutdown_connections 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.041+0000 7f8044b11700 1 -- 172.21.15.88:0/107778814 wait complete. 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.042+0000 7f8044b11700 1 Processor -- start 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.042+0000 7f8044b11700 1 -- start start 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.042+0000 7f8044b11700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8040101420 0x7f8040195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.042+0000 7f8044b11700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8040101d30 0x7f80401963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.042+0000 7f8044b11700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8040196a90 con 0x7f8040101d30 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.042+0000 7f8044b11700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804019a820 con 0x7f8040101420 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.042+0000 7f803dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8040101d30 0x7f80401963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.042+0000 7f803dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8040101d30 0x7f80401963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37942/0 (socket says 172.21.15.88:37942) 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.042+0000 7f803dd9b700 1 -- 172.21.15.88:0/1980442356 learned_addr learned my addr 172.21.15.88:0/1980442356 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:13.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.042+0000 7f803e59c700 1 --2- 172.21.15.88:0/1980442356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8040101420 0x7f8040195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:13.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.043+0000 7f803dd9b700 1 -- 172.21.15.88:0/1980442356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8040101420 msgr2=0x7f8040195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.043+0000 7f803dd9b700 1 --2- 172.21.15.88:0/1980442356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8040101420 0x7f8040195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.043+0000 7f803dd9b700 1 -- 172.21.15.88:0/1980442356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80300095f0 con 0x7f8040101d30 2024-02-19T19:11:13.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.043+0000 7f803dd9b700 1 --2- 172.21.15.88:0/1980442356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8040101d30 0x7f80401963b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f803400d8d0 tx=0x7f803400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:13.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.043+0000 7f802f7fe700 1 -- 172.21.15.88:0/1980442356 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80340098a0 con 0x7f8040101d30 2024-02-19T19:11:13.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.043+0000 7f8044b11700 1 -- 172.21.15.88:0/1980442356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804019ab00 con 0x7f8040101d30 2024-02-19T19:11:13.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.043+0000 7f802f7fe700 1 -- 172.21.15.88:0/1980442356 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8034010460 con 0x7f8040101d30 2024-02-19T19:11:13.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.043+0000 7f802f7fe700 1 -- 172.21.15.88:0/1980442356 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f803400f5f0 con 0x7f8040101d30 2024-02-19T19:11:13.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.043+0000 7f8044b11700 1 -- 172.21.15.88:0/1980442356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804019b050 con 0x7f8040101d30 2024-02-19T19:11:13.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.044+0000 7f8044b11700 1 -- 172.21.15.88:0/1980442356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8040107900 con 0x7f8040101d30 2024-02-19T19:11:13.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.044+0000 7f802f7fe700 1 -- 172.21.15.88:0/1980442356 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8034009a00 con 0x7f8040101d30 2024-02-19T19:11:13.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.045+0000 7f802f7fe700 1 --2- 172.21.15.88:0/1980442356 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802806d8a0 0x7f802806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:13.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.045+0000 7f802f7fe700 1 -- 172.21.15.88:0/1980442356 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f803408d400 con 0x7f8040101d30 2024-02-19T19:11:13.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.045+0000 7f803e59c700 1 --2- 172.21.15.88:0/1980442356 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802806d8a0 0x7f802806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:13.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.045+0000 7f803e59c700 1 --2- 172.21.15.88:0/1980442356 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802806d8a0 0x7f802806fd50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8030006010 tx=0x7f803001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:13.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.050+0000 7f802f7fe700 1 -- 172.21.15.88:0/1980442356 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8034056bb0 con 0x7f8040101d30 2024-02-19T19:11:13.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.267+0000 7f8044b11700 1 -- 172.21.15.88:0/1980442356 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f804019b2e0 con 0x7f802806d8a0 2024-02-19T19:11:13.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.268+0000 7f802f7fe700 1 -- 172.21.15.88:0/1980442356 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f804019b2e0 con 0x7f802806d8a0 2024-02-19T19:11:13.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.271+0000 7f8044b11700 1 -- 172.21.15.88:0/1980442356 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802806d8a0 msgr2=0x7f802806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.271+0000 7f8044b11700 1 --2- 172.21.15.88:0/1980442356 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802806d8a0 0x7f802806fd50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8030006010 tx=0x7f803001a040 comp rx=0 tx=0).stop 2024-02-19T19:11:13.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.271+0000 7f8044b11700 1 -- 172.21.15.88:0/1980442356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8040101d30 msgr2=0x7f80401963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.271+0000 7f8044b11700 1 --2- 172.21.15.88:0/1980442356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8040101d30 0x7f80401963b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f803400d8d0 tx=0x7f803400dbe0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.271+0000 7f8044b11700 1 -- 172.21.15.88:0/1980442356 shutdown_connections 2024-02-19T19:11:13.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.271+0000 7f8044b11700 1 --2- 172.21.15.88:0/1980442356 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802806d8a0 0x7f802806fd50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.271+0000 7f8044b11700 1 --2- 172.21.15.88:0/1980442356 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8040101420 0x7f8040195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.271+0000 7f8044b11700 1 --2- 172.21.15.88:0/1980442356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8040101d30 0x7f80401963b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.271+0000 7f8044b11700 1 -- 172.21.15.88:0/1980442356 >> 172.21.15.88:0/1980442356 conn(0x7f80400fac80 msgr2=0x7f8040104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:13.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.271+0000 7f8044b11700 1 -- 172.21.15.88:0/1980442356 shutdown_connections 2024-02-19T19:11:13.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.271+0000 7f8044b11700 1 -- 172.21.15.88:0/1980442356 wait complete. 2024-02-19T19:11:13.303 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:11:13.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:13 smithi088 ceph-mon[88643]: pgmap v228: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:13.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:11:13.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.382+0000 7f1f12617700 1 -- 172.21.15.88:0/2830554644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f0c0ffd60 msgr2=0x7f1f0c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.382+0000 7f1f12617700 1 --2- 172.21.15.88:0/2830554644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f0c0ffd60 0x7f1f0c103d30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1f00009910 tx=0x7f1f00009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:13.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.383+0000 7f1f12617700 1 -- 172.21.15.88:0/2830554644 shutdown_connections 2024-02-19T19:11:13.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.383+0000 7f1f12617700 1 --2- 172.21.15.88:0/2830554644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f0c0ffd60 0x7f1f0c103d30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.383+0000 7f1f12617700 1 --2- 172.21.15.88:0/2830554644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0ff3c0 0x7f1f0c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.383+0000 7f1f12617700 1 -- 172.21.15.88:0/2830554644 >> 172.21.15.88:0/2830554644 conn(0x7f1f0c0fac80 msgr2=0x7f1f0c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:13.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.383+0000 7f1f12617700 1 -- 172.21.15.88:0/2830554644 shutdown_connections 2024-02-19T19:11:13.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.383+0000 7f1f12617700 1 -- 172.21.15.88:0/2830554644 wait complete. 2024-02-19T19:11:13.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.383+0000 7f1f12617700 1 Processor -- start 2024-02-19T19:11:13.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.383+0000 7f1f12617700 1 -- start start 2024-02-19T19:11:13.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.383+0000 7f1f12617700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f0c0ff3c0 0x7f1f0c195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:13.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.383+0000 7f1f12617700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0ffd60 0x7f1f0c1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:13.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.383+0000 7f1f12617700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c196aa0 con 0x7f1f0c0ff3c0 2024-02-19T19:11:13.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f12617700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c19a830 con 0x7f1f0c0ffd60 2024-02-19T19:11:13.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f0bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f0c0ff3c0 0x7f1f0c195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:13.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f0bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f0c0ff3c0 0x7f1f0c195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37964/0 (socket says 172.21.15.88:37964) 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f0bfff700 1 -- 172.21.15.88:0/3132339055 learned_addr learned my addr 172.21.15.88:0/3132339055 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f0b7fe700 1 --2- 172.21.15.88:0/3132339055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0ffd60 0x7f1f0c1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f0bfff700 1 -- 172.21.15.88:0/3132339055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0ffd60 msgr2=0x7f1f0c1963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f0bfff700 1 --2- 172.21.15.88:0/3132339055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0ffd60 0x7f1f0c1963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f0bfff700 1 -- 172.21.15.88:0/3132339055 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f000095f0 con 0x7f1f0c0ff3c0 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f0bfff700 1 --2- 172.21.15.88:0/3132339055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f0c0ff3c0 0x7f1f0c195e80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e920 tx=0x7f1efc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f097fa700 1 -- 172.21.15.88:0/3132339055 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1efc00c9d0 con 0x7f1f0c0ff3c0 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f12617700 1 -- 172.21.15.88:0/3132339055 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c19ab10 con 0x7f1f0c0ff3c0 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f097fa700 1 -- 172.21.15.88:0/3132339055 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1efc010430 con 0x7f1f0c0ff3c0 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f097fa700 1 -- 172.21.15.88:0/3132339055 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1efc018630 con 0x7f1f0c0ff3c0 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.384+0000 7f1f12617700 1 -- 172.21.15.88:0/3132339055 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c19b030 con 0x7f1f0c0ff3c0 2024-02-19T19:11:13.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.385+0000 7f1f12617700 1 -- 172.21.15.88:0/3132339055 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c04f2a0 con 0x7f1f0c0ff3c0 2024-02-19T19:11:13.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.386+0000 7f1f097fa700 1 -- 172.21.15.88:0/3132339055 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1efc0105a0 con 0x7f1f0c0ff3c0 2024-02-19T19:11:13.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.386+0000 7f1f097fa700 1 --2- 172.21.15.88:0/3132339055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ef406d8a0 0x7f1ef406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:13.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.386+0000 7f1f097fa700 1 -- 172.21.15.88:0/3132339055 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1efc014070 con 0x7f1f0c0ff3c0 2024-02-19T19:11:13.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.386+0000 7f1f0b7fe700 1 --2- 172.21.15.88:0/3132339055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ef406d8a0 0x7f1ef406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:13.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.387+0000 7f1f0b7fe700 1 --2- 172.21.15.88:0/3132339055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ef406d8a0 0x7f1ef406fd50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1f000059d0 tx=0x7f1f00005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:13.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.391+0000 7f1f097fa700 1 -- 172.21.15.88:0/3132339055 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1efc052d20 con 0x7f1f0c0ff3c0 2024-02-19T19:11:13.531 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:13 smithi132 ceph-mon[70676]: pgmap v228: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:13.532 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:11:13.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.609+0000 7f1f12617700 1 -- 172.21.15.88:0/3132339055 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f0c19b400 con 0x7f1ef406d8a0 2024-02-19T19:11:13.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.610+0000 7f1f097fa700 1 -- 172.21.15.88:0/3132339055 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1f0c19b400 con 0x7f1ef406d8a0 2024-02-19T19:11:13.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.612+0000 7f1f12617700 1 -- 172.21.15.88:0/3132339055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ef406d8a0 msgr2=0x7f1ef406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.612+0000 7f1f12617700 1 --2- 172.21.15.88:0/3132339055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ef406d8a0 0x7f1ef406fd50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1f000059d0 tx=0x7f1f00005940 comp rx=0 tx=0).stop 2024-02-19T19:11:13.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.612+0000 7f1f12617700 1 -- 172.21.15.88:0/3132339055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f0c0ff3c0 msgr2=0x7f1f0c195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.612+0000 7f1f12617700 1 --2- 172.21.15.88:0/3132339055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f0c0ff3c0 0x7f1f0c195e80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e920 tx=0x7f1efc00ec30 comp rx=0 tx=0).stop 2024-02-19T19:11:13.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.612+0000 7f1f12617700 1 -- 172.21.15.88:0/3132339055 shutdown_connections 2024-02-19T19:11:13.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.612+0000 7f1f12617700 1 --2- 172.21.15.88:0/3132339055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ef406d8a0 0x7f1ef406fd50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.612+0000 7f1f12617700 1 --2- 172.21.15.88:0/3132339055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f0c0ff3c0 0x7f1f0c195e80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.612+0000 7f1f12617700 1 --2- 172.21.15.88:0/3132339055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1f0c0ffd60 0x7f1f0c1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.612+0000 7f1f12617700 1 -- 172.21.15.88:0/3132339055 >> 172.21.15.88:0/3132339055 conn(0x7f1f0c0fac80 msgr2=0x7f1f0c103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:13.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.612+0000 7f1f12617700 1 -- 172.21.15.88:0/3132339055 shutdown_connections 2024-02-19T19:11:13.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.612+0000 7f1f12617700 1 -- 172.21.15.88:0/3132339055 wait complete. 2024-02-19T19:11:13.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.717+0000 7f5c8341b700 1 -- 172.21.15.88:0/2802834569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c7c0ff3c0 msgr2=0x7f5c7c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.717+0000 7f5c8341b700 1 --2- 172.21.15.88:0/2802834569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c7c0ff3c0 0x7f5c7c0ff790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009910 tx=0x7f5c78009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:13.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.717+0000 7f5c8341b700 1 -- 172.21.15.88:0/2802834569 shutdown_connections 2024-02-19T19:11:13.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.717+0000 7f5c8341b700 1 --2- 172.21.15.88:0/2802834569 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c7c0ffd60 0x7f5c7c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.717+0000 7f5c8341b700 1 --2- 172.21.15.88:0/2802834569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c7c0ff3c0 0x7f5c7c0ff790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.717+0000 7f5c8341b700 1 -- 172.21.15.88:0/2802834569 >> 172.21.15.88:0/2802834569 conn(0x7f5c7c0fac80 msgr2=0x7f5c7c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:13.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.718+0000 7f5c8341b700 1 -- 172.21.15.88:0/2802834569 shutdown_connections 2024-02-19T19:11:13.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.718+0000 7f5c8341b700 1 -- 172.21.15.88:0/2802834569 wait complete. 2024-02-19T19:11:13.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.718+0000 7f5c8341b700 1 Processor -- start 2024-02-19T19:11:13.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.718+0000 7f5c8341b700 1 -- start start 2024-02-19T19:11:13.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.718+0000 7f5c8341b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c7c0ff3c0 0x7f5c7c100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:13.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.718+0000 7f5c8341b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c7c0ffd60 0x7f5c7c101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:13.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.718+0000 7f5c8341b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c101850 con 0x7f5c7c0ffd60 2024-02-19T19:11:13.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.718+0000 7f5c8341b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7c101990 con 0x7f5c7c0ff3c0 2024-02-19T19:11:13.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c809b6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c7c0ffd60 0x7f5c7c101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:13.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c809b6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c7c0ffd60 0x7f5c7c101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37972/0 (socket says 172.21.15.88:37972) 2024-02-19T19:11:13.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c809b6700 1 -- 172.21.15.88:0/1028509236 learned_addr learned my addr 172.21.15.88:0/1028509236 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:13.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c811b7700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c7c0ff3c0 0x7f5c7c100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:13.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c811b7700 1 -- 172.21.15.88:0/1028509236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c7c0ffd60 msgr2=0x7f5c7c101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c811b7700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c7c0ffd60 0x7f5c7c101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c811b7700 1 -- 172.21.15.88:0/1028509236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780095f0 con 0x7f5c7c0ff3c0 2024-02-19T19:11:13.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c809b6700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c7c0ffd60 0x7f5c7c101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:11:13.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c811b7700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c7c0ff3c0 0x7f5c7c100dd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009910 tx=0x7f5c7800bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:13.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c727fc700 1 -- 172.21.15.88:0/1028509236 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c7801d070 con 0x7f5c7c0ff3c0 2024-02-19T19:11:13.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c727fc700 1 -- 172.21.15.88:0/1028509236 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c78005dc0 con 0x7f5c7c0ff3c0 2024-02-19T19:11:13.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.719+0000 7f5c8341b700 1 -- 172.21.15.88:0/1028509236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7c101c10 con 0x7f5c7c0ff3c0 2024-02-19T19:11:13.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.720+0000 7f5c727fc700 1 -- 172.21.15.88:0/1028509236 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c7800fe80 con 0x7f5c7c0ff3c0 2024-02-19T19:11:13.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.720+0000 7f5c8341b700 1 -- 172.21.15.88:0/1028509236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c7c1a2400 con 0x7f5c7c0ff3c0 2024-02-19T19:11:13.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.721+0000 7f5c8341b700 1 -- 172.21.15.88:0/1028509236 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7c04f2a0 con 0x7f5c7c0ff3c0 2024-02-19T19:11:13.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.721+0000 7f5c727fc700 1 -- 172.21.15.88:0/1028509236 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5c780174e0 con 0x7f5c7c0ff3c0 2024-02-19T19:11:13.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.722+0000 7f5c727fc700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5c6806d8a0 0x7f5c6806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:13.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.722+0000 7f5c809b6700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5c6806d8a0 0x7f5c6806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:13.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.722+0000 7f5c727fc700 1 -- 172.21.15.88:0/1028509236 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5c7808e8e0 con 0x7f5c7c0ff3c0 2024-02-19T19:11:13.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.722+0000 7f5c809b6700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5c6806d8a0 0x7f5c6806fd50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c1025c0 tx=0x7f5c6c007480 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:13.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.726+0000 7f5c727fc700 1 -- 172.21.15.88:0/1028509236 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5c780580a0 con 0x7f5c7c0ff3c0 2024-02-19T19:11:13.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.934+0000 7f5c8341b700 1 -- 172.21.15.88:0/1028509236 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c7c0fb9f0 con 0x7f5c6806d8a0 2024-02-19T19:11:13.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.945+0000 7f5c727fc700 1 -- 172.21.15.88:0/1028509236 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5c7c0fb9f0 con 0x7f5c6806d8a0 2024-02-19T19:11:13.967 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:11:13.967 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (8m) 14s ago 18m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:11:13.967 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (19m) 14s ago 19m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:11:13.967 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (73s) 51s ago 17m 7675k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:11:13.967 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (4m) 14s ago 19m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:11:13.967 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (4m) 51s ago 17m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:11:13.967 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (7m) 14s ago 18m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:11:13.967 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (39s) 14s ago 12m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:11:13.967 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (20s) 14s ago 12m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:11:13.967 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 starting - - - - 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (12m) 51s ago 12m 21.6M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (9m) 14s ago 20m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (9m) 51s ago 17m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (6m) 14s ago 20m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (5m) 51s ago 17m 42.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (9m) 14s ago 19m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (9m) 51s ago 17m 15.2M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (3m) 14s ago 16m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (3m) 14s ago 15m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (2m) 14s ago 15m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (2m) 51s ago 14m 102M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (90s) 51s ago 14m 102M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (55s) 51s ago 14m 12.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:11:13.968 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (8m) 14s ago 17m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:11:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.947+0000 7f5c8341b700 1 -- 172.21.15.88:0/1028509236 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5c6806d8a0 msgr2=0x7f5c6806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.947+0000 7f5c8341b700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5c6806d8a0 0x7f5c6806fd50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c1025c0 tx=0x7f5c6c007480 comp rx=0 tx=0).stop 2024-02-19T19:11:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.947+0000 7f5c8341b700 1 -- 172.21.15.88:0/1028509236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c7c0ff3c0 msgr2=0x7f5c7c100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.947+0000 7f5c8341b700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c7c0ff3c0 0x7f5c7c100dd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009910 tx=0x7f5c7800bd70 comp rx=0 tx=0).stop 2024-02-19T19:11:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.947+0000 7f5c8341b700 1 -- 172.21.15.88:0/1028509236 shutdown_connections 2024-02-19T19:11:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.947+0000 7f5c8341b700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5c6806d8a0 0x7f5c6806fd50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.947+0000 7f5c8341b700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5c7c0ff3c0 0x7f5c7c100dd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.947+0000 7f5c8341b700 1 --2- 172.21.15.88:0/1028509236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c7c0ffd60 0x7f5c7c101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.947+0000 7f5c8341b700 1 -- 172.21.15.88:0/1028509236 >> 172.21.15.88:0/1028509236 conn(0x7f5c7c0fac80 msgr2=0x7f5c7c1034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.947+0000 7f5c8341b700 1 -- 172.21.15.88:0/1028509236 shutdown_connections 2024-02-19T19:11:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:13.947+0000 7f5c8341b700 1 -- 172.21.15.88:0/1028509236 wait complete. 2024-02-19T19:11:14.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.050+0000 7f43c23c0700 1 -- 172.21.15.88:0/2336468398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc0ffbd0 msgr2=0x7f43bc100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:14.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.050+0000 7f43c23c0700 1 --2- 172.21.15.88:0/2336468398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc0ffbd0 0x7f43bc100040 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009910 tx=0x7f43b0009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:14.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.050+0000 7f43c23c0700 1 -- 172.21.15.88:0/2336468398 shutdown_connections 2024-02-19T19:11:14.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.050+0000 7f43c23c0700 1 --2- 172.21.15.88:0/2336468398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc0ffbd0 0x7f43bc100040 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.050+0000 7f43c23c0700 1 --2- 172.21.15.88:0/2336468398 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc104130 0x7f43bc104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.050+0000 7f43c23c0700 1 -- 172.21.15.88:0/2336468398 >> 172.21.15.88:0/2336468398 conn(0x7f43bc075130 msgr2=0x7f43bc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:14.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.050+0000 7f43c23c0700 1 -- 172.21.15.88:0/2336468398 shutdown_connections 2024-02-19T19:11:14.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.050+0000 7f43c23c0700 1 -- 172.21.15.88:0/2336468398 wait complete. 2024-02-19T19:11:14.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.051+0000 7f43c23c0700 1 Processor -- start 2024-02-19T19:11:14.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.051+0000 7f43c23c0700 1 -- start start 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.051+0000 7f43c23c0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc0ffbd0 0x7f43bc100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.051+0000 7f43c23c0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc104130 0x7f43bc101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.051+0000 7f43c23c0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc1018b0 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.051+0000 7f43c23c0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc1019f0 con 0x7f43bc104130 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.051+0000 7f43bbfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc0ffbd0 0x7f43bc100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.051+0000 7f43bbfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc0ffbd0 0x7f43bc100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37990/0 (socket says 172.21.15.88:37990) 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.051+0000 7f43bbfff700 1 -- 172.21.15.88:0/2963668896 learned_addr learned my addr 172.21.15.88:0/2963668896 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.051+0000 7f43bb7fe700 1 --2- 172.21.15.88:0/2963668896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc104130 0x7f43bc101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.052+0000 7f43bbfff700 1 -- 172.21.15.88:0/2963668896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc104130 msgr2=0x7f43bc101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.052+0000 7f43bbfff700 1 --2- 172.21.15.88:0/2963668896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc104130 0x7f43bc101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.052+0000 7f43bbfff700 1 -- 172.21.15.88:0/2963668896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b00095f0 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.052+0000 7f43bbfff700 1 --2- 172.21.15.88:0/2963668896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc0ffbd0 0x7f43bc100e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00d8d0 tx=0x7f43ac00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:14.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.052+0000 7f43b97fa700 1 -- 172.21.15.88:0/2963668896 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43ac0098a0 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.052+0000 7f43b97fa700 1 -- 172.21.15.88:0/2963668896 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43ac010460 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.052+0000 7f43c23c0700 1 -- 172.21.15.88:0/2963668896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc1a2020 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.052+0000 7f43b97fa700 1 -- 172.21.15.88:0/2963668896 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43ac00f5f0 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.052+0000 7f43c23c0700 1 -- 172.21.15.88:0/2963668896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc1a2430 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.053+0000 7f43c23c0700 1 -- 172.21.15.88:0/2963668896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc1086a0 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.054+0000 7f43b97fa700 1 -- 172.21.15.88:0/2963668896 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f43ac009a00 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.054+0000 7f43b97fa700 1 --2- 172.21.15.88:0/2963668896 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43a406d8a0 0x7f43a406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:14.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.054+0000 7f43bb7fe700 1 --2- 172.21.15.88:0/2963668896 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43a406d8a0 0x7f43a406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:14.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.054+0000 7f43b97fa700 1 -- 172.21.15.88:0/2963668896 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f43ac08d310 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.055+0000 7f43bb7fe700 1 --2- 172.21.15.88:0/2963668896 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43a406d8a0 0x7f43a406fd50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f43bc102620 tx=0x7f43b001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:14.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.059+0000 7f43b97fa700 1 -- 172.21.15.88:0/2963668896 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43ac056b40 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:14 smithi088 ceph-mon[88643]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:14.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:14.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:14.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:11:14.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.340+0000 7f43c23c0700 1 -- 172.21.15.88:0/2963668896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f43bc108080 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.340+0000 7f43b97fa700 1 -- 172.21.15.88:0/2963668896 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f43ac05a200 con 0x7f43bc0ffbd0 2024-02-19T19:11:14.362 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:11:14.362 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:11:14.362 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:11:14.362 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:11:14.362 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:11:14.362 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:11:14.363 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:11:14.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.342+0000 7f43c23c0700 1 -- 172.21.15.88:0/2963668896 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43a406d8a0 msgr2=0x7f43a406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:14.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.342+0000 7f43c23c0700 1 --2- 172.21.15.88:0/2963668896 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43a406d8a0 0x7f43a406fd50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f43bc102620 tx=0x7f43b001a040 comp rx=0 tx=0).stop 2024-02-19T19:11:14.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.342+0000 7f43c23c0700 1 -- 172.21.15.88:0/2963668896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc0ffbd0 msgr2=0x7f43bc100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:14.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.342+0000 7f43c23c0700 1 --2- 172.21.15.88:0/2963668896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc0ffbd0 0x7f43bc100e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f43ac00d8d0 tx=0x7f43ac00dbe0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.343+0000 7f43c23c0700 1 -- 172.21.15.88:0/2963668896 shutdown_connections 2024-02-19T19:11:14.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.343+0000 7f43c23c0700 1 --2- 172.21.15.88:0/2963668896 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43a406d8a0 0x7f43a406fd50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.343+0000 7f43c23c0700 1 --2- 172.21.15.88:0/2963668896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43bc0ffbd0 0x7f43bc100e30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.343+0000 7f43c23c0700 1 --2- 172.21.15.88:0/2963668896 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43bc104130 0x7f43bc101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.343+0000 7f43c23c0700 1 -- 172.21.15.88:0/2963668896 >> 172.21.15.88:0/2963668896 conn(0x7f43bc075130 msgr2=0x7f43bc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:14.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.343+0000 7f43c23c0700 1 -- 172.21.15.88:0/2963668896 shutdown_connections 2024-02-19T19:11:14.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.343+0000 7f43c23c0700 1 -- 172.21.15.88:0/2963668896 wait complete. 2024-02-19T19:11:14.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.447+0000 7ff8f3584700 1 -- 172.21.15.88:0/3429283761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8ec0ffbd0 msgr2=0x7ff8ec100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.447+0000 7ff8f3584700 1 --2- 172.21.15.88:0/3429283761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8ec0ffbd0 0x7ff8ec100040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009910 tx=0x7ff8dc009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.447+0000 7ff8f3584700 1 -- 172.21.15.88:0/3429283761 shutdown_connections 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.447+0000 7ff8f3584700 1 --2- 172.21.15.88:0/3429283761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8ec0ffbd0 0x7ff8ec100040 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.447+0000 7ff8f3584700 1 --2- 172.21.15.88:0/3429283761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8ec104130 0x7ff8ec104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.447+0000 7ff8f3584700 1 -- 172.21.15.88:0/3429283761 >> 172.21.15.88:0/3429283761 conn(0x7ff8ec075130 msgr2=0x7ff8ec075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.447+0000 7ff8f3584700 1 -- 172.21.15.88:0/3429283761 shutdown_connections 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.447+0000 7ff8f3584700 1 -- 172.21.15.88:0/3429283761 wait complete. 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f3584700 1 Processor -- start 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f3584700 1 -- start start 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f3584700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8ec0ffbd0 0x7ff8ec194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f3584700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8ec104130 0x7ff8ec195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f3584700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec195950 con 0x7ff8ec104130 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f3584700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec1996e0 con 0x7ff8ec0ffbd0 2024-02-19T19:11:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f0b1f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8ec104130 0x7ff8ec195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f0b1f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8ec104130 0x7ff8ec195270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38012/0 (socket says 172.21.15.88:38012) 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f0b1f700 1 -- 172.21.15.88:0/1952757971 learned_addr learned my addr 172.21.15.88:0/1952757971 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f1320700 1 --2- 172.21.15.88:0/1952757971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8ec0ffbd0 0x7ff8ec194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f0b1f700 1 -- 172.21.15.88:0/1952757971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8ec0ffbd0 msgr2=0x7ff8ec194d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f0b1f700 1 --2- 172.21.15.88:0/1952757971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8ec0ffbd0 0x7ff8ec194d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.448+0000 7ff8f0b1f700 1 -- 172.21.15.88:0/1952757971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc0095f0 con 0x7ff8ec104130 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.449+0000 7ff8f0b1f700 1 --2- 172.21.15.88:0/1952757971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8ec104130 0x7ff8ec195270 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00b7d0 tx=0x7ff8dc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.449+0000 7ff8e27fc700 1 -- 172.21.15.88:0/1952757971 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8dc01f070 con 0x7ff8ec104130 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.449+0000 7ff8e27fc700 1 -- 172.21.15.88:0/1952757971 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8dc0052a0 con 0x7ff8ec104130 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.449+0000 7ff8f3584700 1 -- 172.21.15.88:0/1952757971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ec199960 con 0x7ff8ec104130 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.449+0000 7ff8e27fc700 1 -- 172.21.15.88:0/1952757971 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8dc00f460 con 0x7ff8ec104130 2024-02-19T19:11:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.449+0000 7ff8f3584700 1 -- 172.21.15.88:0/1952757971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec199e50 con 0x7ff8ec104130 2024-02-19T19:11:14.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.450+0000 7ff8f3584700 1 -- 172.21.15.88:0/1952757971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ec04f2a0 con 0x7ff8ec104130 2024-02-19T19:11:14.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.450+0000 7ff8e27fc700 1 -- 172.21.15.88:0/1952757971 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff8dc023070 con 0x7ff8ec104130 2024-02-19T19:11:14.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.450+0000 7ff8e27fc700 1 --2- 172.21.15.88:0/1952757971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8d806d8a0 0x7ff8d806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:14.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.451+0000 7ff8e27fc700 1 -- 172.21.15.88:0/1952757971 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff8dc090690 con 0x7ff8ec104130 2024-02-19T19:11:14.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.451+0000 7ff8f1320700 1 --2- 172.21.15.88:0/1952757971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8d806d8a0 0x7ff8d806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:14.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.451+0000 7ff8f1320700 1 --2- 172.21.15.88:0/1952757971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8d806d8a0 0x7ff8d806fd50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009c60 tx=0x7ff8e8009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:14.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.455+0000 7ff8e27fc700 1 -- 172.21.15.88:0/1952757971 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff8dc059ec0 con 0x7ff8ec104130 2024-02-19T19:11:14.571 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:14 smithi132 ceph-mon[70676]: from='client.34466 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:14.571 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:14.571 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:14.571 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:11:14.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.705+0000 7ff8f3584700 1 -- 172.21.15.88:0/1952757971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff8ec19a0c0 con 0x7ff8ec104130 2024-02-19T19:11:14.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.705+0000 7ff8e27fc700 1 -- 172.21.15.88:0/1952757971 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7ff8dc05d580 con 0x7ff8ec104130 2024-02-19T19:11:14.727 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:11:14.727 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:11:14.727 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:11:14.727 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:11:14.727 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:11:14.727 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:11:14.728 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:11:14.729 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:11:14.730 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:11:14.730 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:11:14.730 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:11:14.730 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:11:14.730 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:11:14.730 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:11:14.730 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:11:14.730 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:11:14.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.707+0000 7ff8f3584700 1 -- 172.21.15.88:0/1952757971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8d806d8a0 msgr2=0x7ff8d806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:14.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.707+0000 7ff8f3584700 1 --2- 172.21.15.88:0/1952757971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8d806d8a0 0x7ff8d806fd50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8009c60 tx=0x7ff8e8009260 comp rx=0 tx=0).stop 2024-02-19T19:11:14.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.707+0000 7ff8f3584700 1 -- 172.21.15.88:0/1952757971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8ec104130 msgr2=0x7ff8ec195270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:14.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.707+0000 7ff8f3584700 1 --2- 172.21.15.88:0/1952757971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8ec104130 0x7ff8ec195270 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc00b7d0 tx=0x7ff8dc00bf90 comp rx=0 tx=0).stop 2024-02-19T19:11:14.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.707+0000 7ff8f3584700 1 -- 172.21.15.88:0/1952757971 shutdown_connections 2024-02-19T19:11:14.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.707+0000 7ff8f3584700 1 --2- 172.21.15.88:0/1952757971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8d806d8a0 0x7ff8d806fd50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.707+0000 7ff8f3584700 1 --2- 172.21.15.88:0/1952757971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8ec0ffbd0 0x7ff8ec194d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.707+0000 7ff8f3584700 1 --2- 172.21.15.88:0/1952757971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8ec104130 0x7ff8ec195270 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.707+0000 7ff8f3584700 1 -- 172.21.15.88:0/1952757971 >> 172.21.15.88:0/1952757971 conn(0x7ff8ec075130 msgr2=0x7ff8ec0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:14.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.708+0000 7ff8f3584700 1 -- 172.21.15.88:0/1952757971 shutdown_connections 2024-02-19T19:11:14.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.708+0000 7ff8f3584700 1 -- 172.21.15.88:0/1952757971 wait complete. 2024-02-19T19:11:14.732 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:11:14.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.813+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3727989939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb00100460 msgr2=0x7fcb00100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:14.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.813+0000 7fcb07bf1700 1 --2- 172.21.15.88:0/3727989939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb00100460 0x7fcb00100830 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcaf800b0a0 tx=0x7fcaf800b3b0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.813+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3727989939 shutdown_connections 2024-02-19T19:11:14.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.813+0000 7fcb07bf1700 1 --2- 172.21.15.88:0/3727989939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb00100d70 0x7fcb001094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.813+0000 7fcb07bf1700 1 --2- 172.21.15.88:0/3727989939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb00100460 0x7fcb00100830 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.813+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3727989939 >> 172.21.15.88:0/3727989939 conn(0x7fcb000748b0 msgr2=0x7fcb00074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:14.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.813+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3727989939 shutdown_connections 2024-02-19T19:11:14.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.813+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3727989939 wait complete. 2024-02-19T19:11:14.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.813+0000 7fcb07bf1700 1 Processor -- start 2024-02-19T19:11:14.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.813+0000 7fcb07bf1700 1 -- start start 2024-02-19T19:11:14.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb07bf1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb00100460 0x7fcb00195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:14.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb07bf1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb00100d70 0x7fcb00196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:14.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb07bf1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb00196b70 con 0x7fcb00100d70 2024-02-19T19:11:14.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb07bf1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0019a900 con 0x7fcb00100460 2024-02-19T19:11:14.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb0518c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb00100d70 0x7fcb00196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:14.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb0518c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb00100d70 0x7fcb00196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38042/0 (socket says 172.21.15.88:38042) 2024-02-19T19:11:14.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb0518c700 1 -- 172.21.15.88:0/3096422418 learned_addr learned my addr 172.21.15.88:0/3096422418 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:14.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb0598d700 1 --2- 172.21.15.88:0/3096422418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb00100460 0x7fcb00195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:14.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb0518c700 1 -- 172.21.15.88:0/3096422418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb00100460 msgr2=0x7fcb00195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:14.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb0518c700 1 --2- 172.21.15.88:0/3096422418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb00100460 0x7fcb00195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:14.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb0518c700 1 -- 172.21.15.88:0/3096422418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf8009d00 con 0x7fcb00100d70 2024-02-19T19:11:14.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb0518c700 1 --2- 172.21.15.88:0/3096422418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb00100d70 0x7fcb00196490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400b4b0 tx=0x7fcaf400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:14.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcafeffd700 1 -- 172.21.15.88:0/3096422418 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf4011800 con 0x7fcb00100d70 2024-02-19T19:11:14.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3096422418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0019abe0 con 0x7fcb00100d70 2024-02-19T19:11:14.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcafeffd700 1 -- 172.21.15.88:0/3096422418 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcaf4011e20 con 0x7fcb00100d70 2024-02-19T19:11:14.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcafeffd700 1 -- 172.21.15.88:0/3096422418 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf400f2e0 con 0x7fcb00100d70 2024-02-19T19:11:14.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.814+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3096422418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0019b130 con 0x7fcb00100d70 2024-02-19T19:11:14.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.815+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3096422418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb00107e40 con 0x7fcb00100d70 2024-02-19T19:11:14.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.816+0000 7fcafeffd700 1 -- 172.21.15.88:0/3096422418 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcaf4010380 con 0x7fcb00100d70 2024-02-19T19:11:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.816+0000 7fcafeffd700 1 --2- 172.21.15.88:0/3096422418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcae806d8a0 0x7fcae806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.816+0000 7fcafeffd700 1 -- 172.21.15.88:0/3096422418 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcaf408d070 con 0x7fcb00100d70 2024-02-19T19:11:14.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.816+0000 7fcb0598d700 1 --2- 172.21.15.88:0/3096422418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcae806d8a0 0x7fcae806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:14.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.817+0000 7fcb0598d700 1 --2- 172.21.15.88:0/3096422418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcae806d8a0 0x7fcae806fd50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8000f80 tx=0x7fcaf80097e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:14.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:14.821+0000 7fcafeffd700 1 -- 172.21.15.88:0/3096422418 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcaf4056820 con 0x7fcb00100d70 2024-02-19T19:11:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.038+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3096422418 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb0019b420 con 0x7fcae806d8a0 2024-02-19T19:11:15.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.040+0000 7fcafeffd700 1 -- 172.21.15.88:0/3096422418 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fcb0019b420 con 0x7fcae806d8a0 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "14/23 daemons upgraded", 2024-02-19T19:11:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:11:15.063 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:11:15.063 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:11:15.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.042+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3096422418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcae806d8a0 msgr2=0x7fcae806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:15.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.042+0000 7fcb07bf1700 1 --2- 172.21.15.88:0/3096422418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcae806d8a0 0x7fcae806fd50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8000f80 tx=0x7fcaf80097e0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.042+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3096422418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb00100d70 msgr2=0x7fcb00196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:15.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.042+0000 7fcb07bf1700 1 --2- 172.21.15.88:0/3096422418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb00100d70 0x7fcb00196490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400b4b0 tx=0x7fcaf400b7c0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.042+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3096422418 shutdown_connections 2024-02-19T19:11:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.042+0000 7fcb07bf1700 1 --2- 172.21.15.88:0/3096422418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcae806d8a0 0x7fcae806fd50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.042+0000 7fcb07bf1700 1 --2- 172.21.15.88:0/3096422418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcb00100460 0x7fcb00195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.042+0000 7fcb07bf1700 1 --2- 172.21.15.88:0/3096422418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb00100d70 0x7fcb00196490 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.042+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3096422418 >> 172.21.15.88:0/3096422418 conn(0x7fcb000748b0 msgr2=0x7fcb000fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.042+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3096422418 shutdown_connections 2024-02-19T19:11:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.042+0000 7fcb07bf1700 1 -- 172.21.15.88:0/3096422418 wait complete. 2024-02-19T19:11:15.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.146+0000 7f21423c3700 1 -- 172.21.15.88:0/408512311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0ffc60 msgr2=0x7f213c10a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:15.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.146+0000 7f21423c3700 1 --2- 172.21.15.88:0/408512311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0ffc60 0x7f213c10a6d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2130009910 tx=0x7f2130009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:15.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.147+0000 7f21423c3700 1 -- 172.21.15.88:0/408512311 shutdown_connections 2024-02-19T19:11:15.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.147+0000 7f21423c3700 1 --2- 172.21.15.88:0/408512311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0ffc60 0x7f213c10a6d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.147+0000 7f21423c3700 1 --2- 172.21.15.88:0/408512311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0ff350 0x7f213c0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.147+0000 7f21423c3700 1 -- 172.21.15.88:0/408512311 >> 172.21.15.88:0/408512311 conn(0x7f213c0748b0 msgr2=0x7f213c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:15.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.147+0000 7f21423c3700 1 -- 172.21.15.88:0/408512311 shutdown_connections 2024-02-19T19:11:15.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.147+0000 7f21423c3700 1 -- 172.21.15.88:0/408512311 wait complete. 2024-02-19T19:11:15.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.147+0000 7f21423c3700 1 Processor -- start 2024-02-19T19:11:15.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.147+0000 7f21423c3700 1 -- start start 2024-02-19T19:11:15.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f21423c3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0ff350 0x7f213c100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:15.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f21423c3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0ffc60 0x7f213c100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:15.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f21423c3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c100fb0 con 0x7f213c0ffc60 2024-02-19T19:11:15.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f21423c3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f213c1010f0 con 0x7f213c0ff350 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f213bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0ff350 0x7f213c100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f213b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0ffc60 0x7f213c100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f213bfff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0ff350 0x7f213c100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:34738/0 (socket says 172.21.15.88:34738) 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f213bfff700 1 -- 172.21.15.88:0/1852380684 learned_addr learned my addr 172.21.15.88:0/1852380684 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f213b7fe700 1 -- 172.21.15.88:0/1852380684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0ff350 msgr2=0x7f213c100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f213b7fe700 1 --2- 172.21.15.88:0/1852380684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0ff350 0x7f213c100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f213b7fe700 1 -- 172.21.15.88:0/1852380684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21300095f0 con 0x7f213c0ffc60 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f213b7fe700 1 --2- 172.21.15.88:0/1852380684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0ffc60 0x7f213c100a70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2130009910 tx=0x7f213000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f21397fa700 1 -- 172.21.15.88:0/1852380684 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f213001d070 con 0x7f213c0ffc60 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f21397fa700 1 -- 172.21.15.88:0/1852380684 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2130005dc0 con 0x7f213c0ffc60 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f21423c3700 1 -- 172.21.15.88:0/1852380684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213c101370 con 0x7f213c0ffc60 2024-02-19T19:11:15.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.148+0000 7f21397fa700 1 -- 172.21.15.88:0/1852380684 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f213000fe80 con 0x7f213c0ffc60 2024-02-19T19:11:15.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.149+0000 7f21423c3700 1 -- 172.21.15.88:0/1852380684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f213c1a23e0 con 0x7f213c0ffc60 2024-02-19T19:11:15.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.149+0000 7f21423c3700 1 -- 172.21.15.88:0/1852380684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213c107dd0 con 0x7f213c0ffc60 2024-02-19T19:11:15.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.150+0000 7f21397fa700 1 -- 172.21.15.88:0/1852380684 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f213000f6f0 con 0x7f213c0ffc60 2024-02-19T19:11:15.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.150+0000 7f21397fa700 1 --2- 172.21.15.88:0/1852380684 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f212406d8a0 0x7f212406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:15.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.150+0000 7f21397fa700 1 -- 172.21.15.88:0/1852380684 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f213008e7d0 con 0x7f213c0ffc60 2024-02-19T19:11:15.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.150+0000 7f213bfff700 1 --2- 172.21.15.88:0/1852380684 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f212406d8a0 0x7f212406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:15.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.151+0000 7f213bfff700 1 --2- 172.21.15.88:0/1852380684 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f212406d8a0 0x7f212406fd50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f212c00bd70 tx=0x7f212c00b4a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:15.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.155+0000 7f21397fa700 1 -- 172.21.15.88:0/1852380684 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2130058000 con 0x7f213c0ffc60 2024-02-19T19:11:15.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:15 smithi088 ceph-mon[88643]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:15.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:15 smithi088 ceph-mon[88643]: from='client.44369 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:15.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:15 smithi088 ceph-mon[88643]: pgmap v229: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:15.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:15 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] up:boot 2024-02-19T19:11:15.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:15 smithi088 ceph-mon[88643]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:11:15.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:11:15.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:15 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2963668896' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:15.353 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:15 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1952757971' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:11:15.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.433+0000 7f21423c3700 1 -- 172.21.15.88:0/1852380684 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f213c04f2a0 con 0x7f213c0ffc60 2024-02-19T19:11:15.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.433+0000 7f21397fa700 1 -- 172.21.15.88:0/1852380684 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f213005b6c0 con 0x7f213c0ffc60 2024-02-19T19:11:15.455 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:11:15.455 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:11:15.455 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:11:15.455 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:11:15.456 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:11:15.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.435+0000 7f21423c3700 1 -- 172.21.15.88:0/1852380684 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f212406d8a0 msgr2=0x7f212406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:15.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.435+0000 7f21423c3700 1 --2- 172.21.15.88:0/1852380684 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f212406d8a0 0x7f212406fd50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f212c00bd70 tx=0x7f212c00b4a0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.435+0000 7f21423c3700 1 -- 172.21.15.88:0/1852380684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0ffc60 msgr2=0x7f213c100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:15.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.435+0000 7f21423c3700 1 --2- 172.21.15.88:0/1852380684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0ffc60 0x7f213c100a70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2130009910 tx=0x7f213000bd70 comp rx=0 tx=0).stop 2024-02-19T19:11:15.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.436+0000 7f21423c3700 1 -- 172.21.15.88:0/1852380684 shutdown_connections 2024-02-19T19:11:15.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.436+0000 7f21423c3700 1 --2- 172.21.15.88:0/1852380684 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f212406d8a0 0x7f212406fd50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.436+0000 7f21423c3700 1 --2- 172.21.15.88:0/1852380684 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f213c0ff350 0x7f213c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.436+0000 7f21423c3700 1 --2- 172.21.15.88:0/1852380684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f213c0ffc60 0x7f213c100a70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:15.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.436+0000 7f21423c3700 1 -- 172.21.15.88:0/1852380684 >> 172.21.15.88:0/1852380684 conn(0x7f213c0748b0 msgr2=0x7f213c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:15.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.436+0000 7f21423c3700 1 -- 172.21.15.88:0/1852380684 shutdown_connections 2024-02-19T19:11:15.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:15.436+0000 7f21423c3700 1 -- 172.21.15.88:0/1852380684 wait complete. 2024-02-19T19:11:15.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:15 smithi132 ceph-mon[70676]: from='client.34470 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:15.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:15 smithi132 ceph-mon[70676]: from='client.44369 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:15.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:15 smithi132 ceph-mon[70676]: pgmap v229: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:15.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:15 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] up:boot 2024-02-19T19:11:15.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:15 smithi132 ceph-mon[70676]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:11:15.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:15 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:11:15.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:15 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2963668896' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:15.505 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:15 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1952757971' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:11:16.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:16 smithi088 ceph-mon[88643]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:16.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:16 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1852380684' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:11:16.616 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:16 smithi132 ceph-mon[70676]: from='client.34490 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:16.616 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:16 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1852380684' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:11:17.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:17 smithi088 ceph-mon[88643]: pgmap v230: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:17.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:17.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:17.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:17 smithi132 ceph-mon[70676]: pgmap v230: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:17.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:17.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:19.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:19.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:19.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:18 smithi132 ceph-mon[70676]: pgmap v231: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:19.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:19.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:11:19.102 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:18 smithi088 ceph-mon[88643]: pgmap v231: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:21.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:21 smithi088 ceph-mon[88643]: pgmap v232: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:21.668 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:21 smithi132 ceph-mon[70676]: pgmap v232: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:23.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:23 smithi088 ceph-mon[88643]: pgmap v233: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:23.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:23 smithi132 ceph-mon[70676]: pgmap v233: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:25.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:25 smithi088 ceph-mon[88643]: pgmap v234: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:25.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:25 smithi132 ceph-mon[70676]: pgmap v234: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:27.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:27 smithi088 ceph-mon[88643]: pgmap v235: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:27.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:27 smithi132 ceph-mon[70676]: pgmap v235: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:28.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:11:28.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:11:29.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:29 smithi088 ceph-mon[88643]: pgmap v236: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:29.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:29 smithi132 ceph-mon[70676]: pgmap v236: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:31.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:31 smithi088 ceph-mon[88643]: pgmap v237: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:31.720 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:31 smithi132 ceph-mon[70676]: pgmap v237: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:33.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:33 smithi088 ceph-mon[88643]: pgmap v238: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:33.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:33 smithi132 ceph-mon[70676]: pgmap v238: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:35.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:35 smithi088 ceph-mon[88643]: pgmap v239: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:35.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:35 smithi132 ceph-mon[70676]: pgmap v239: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:37.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:37 smithi088 ceph-mon[88643]: pgmap v240: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:37.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:37 smithi132 ceph-mon[70676]: pgmap v240: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:39.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:39 smithi088 ceph-mon[88643]: pgmap v241: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:39 smithi132 ceph-mon[70676]: pgmap v241: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:41.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:41 smithi088 ceph-mon[88643]: pgmap v242: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:41.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:41 smithi132 ceph-mon[70676]: pgmap v242: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:43.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:43 smithi088 ceph-mon[88643]: pgmap v243: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:43.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:11:43.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:43 smithi132 ceph-mon[70676]: pgmap v243: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:43.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:11:45.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.553+0000 7f2b33da6700 1 -- 172.21.15.88:0/2079692904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b2c1004e0 msgr2=0x7f2b2c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:45.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.553+0000 7f2b33da6700 1 --2- 172.21.15.88:0/2079692904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b2c1004e0 0x7f2b2c1008b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2b2400b0a0 tx=0x7f2b2400b3b0 comp rx=0 tx=0).stop 2024-02-19T19:11:45.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.553+0000 7f2b33da6700 1 -- 172.21.15.88:0/2079692904 shutdown_connections 2024-02-19T19:11:45.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.553+0000 7f2b33da6700 1 --2- 172.21.15.88:0/2079692904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b2c100e80 0x7f2b2c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:45.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.553+0000 7f2b33da6700 1 --2- 172.21.15.88:0/2079692904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b2c1004e0 0x7f2b2c1008b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:45.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.553+0000 7f2b33da6700 1 -- 172.21.15.88:0/2079692904 >> 172.21.15.88:0/2079692904 conn(0x7f2b2c0fbda0 msgr2=0x7f2b2c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:45.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.554+0000 7f2b33da6700 1 -- 172.21.15.88:0/2079692904 shutdown_connections 2024-02-19T19:11:45.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.554+0000 7f2b33da6700 1 -- 172.21.15.88:0/2079692904 wait complete. 2024-02-19T19:11:45.577 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:45 smithi088 ceph-mon[88643]: pgmap v244: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.554+0000 7f2b33da6700 1 Processor -- start 2024-02-19T19:11:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.554+0000 7f2b33da6700 1 -- start start 2024-02-19T19:11:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.554+0000 7f2b33da6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b2c1004e0 0x7f2b2c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.554+0000 7f2b33da6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b2c100e80 0x7f2b2c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.554+0000 7f2b33da6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c196bb0 con 0x7f2b2c1004e0 2024-02-19T19:11:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.554+0000 7f2b33da6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c19a940 con 0x7f2b2c100e80 2024-02-19T19:11:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b31b42700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b2c1004e0 0x7f2b2c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b31b42700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b2c1004e0 0x7f2b2c195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42606/0 (socket says 172.21.15.88:42606) 2024-02-19T19:11:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b31b42700 1 -- 172.21.15.88:0/1722495083 learned_addr learned my addr 172.21.15.88:0/1722495083 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b31341700 1 --2- 172.21.15.88:0/1722495083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b2c100e80 0x7f2b2c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:45.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b31b42700 1 -- 172.21.15.88:0/1722495083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b2c100e80 msgr2=0x7f2b2c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b31b42700 1 --2- 172.21.15.88:0/1722495083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b2c100e80 0x7f2b2c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b31b42700 1 -- 172.21.15.88:0/1722495083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b24009d00 con 0x7f2b2c1004e0 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b31b42700 1 --2- 172.21.15.88:0/1722495083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b2c1004e0 0x7f2b2c195f90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2b24000f80 tx=0x7f2b2400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b2affd700 1 -- 172.21.15.88:0/1722495083 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b2400e040 con 0x7f2b2c1004e0 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b2affd700 1 -- 172.21.15.88:0/1722495083 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b24003de0 con 0x7f2b2c1004e0 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b2affd700 1 -- 172.21.15.88:0/1722495083 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b2401d7d0 con 0x7f2b2c1004e0 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b33da6700 1 -- 172.21.15.88:0/1722495083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c19abc0 con 0x7f2b2c1004e0 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.555+0000 7f2b33da6700 1 -- 172.21.15.88:0/1722495083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c19b0b0 con 0x7f2b2c1004e0 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.556+0000 7f2b33da6700 1 -- 172.21.15.88:0/1722495083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c1087f0 con 0x7f2b2c1004e0 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.557+0000 7f2b2affd700 1 -- 172.21.15.88:0/1722495083 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2b24004040 con 0x7f2b2c1004e0 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.557+0000 7f2b2affd700 1 --2- 172.21.15.88:0/1722495083 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2b1406d8a0 0x7f2b1406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:45.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.557+0000 7f2b2affd700 1 -- 172.21.15.88:0/1722495083 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2b2408ee60 con 0x7f2b2c1004e0 2024-02-19T19:11:45.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.557+0000 7f2b31341700 1 --2- 172.21.15.88:0/1722495083 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2b1406d8a0 0x7f2b1406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:45.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.558+0000 7f2b31341700 1 --2- 172.21.15.88:0/1722495083 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2b1406d8a0 0x7f2b1406fd50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c1975b0 tx=0x7f2b20009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:45.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.562+0000 7f2b2affd700 1 -- 172.21.15.88:0/1722495083 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2b24058610 con 0x7f2b2c1004e0 2024-02-19T19:11:45.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:45 smithi132 ceph-mon[70676]: pgmap v244: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:45.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.779+0000 7f2b33da6700 1 -- 172.21.15.88:0/1722495083 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b2c19b2d0 con 0x7f2b1406d8a0 2024-02-19T19:11:45.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.780+0000 7f2b2affd700 1 -- 172.21.15.88:0/1722495083 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2b2c19b2d0 con 0x7f2b1406d8a0 2024-02-19T19:11:45.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.782+0000 7f2b33da6700 1 -- 172.21.15.88:0/1722495083 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2b1406d8a0 msgr2=0x7f2b1406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:45.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.782+0000 7f2b33da6700 1 --2- 172.21.15.88:0/1722495083 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2b1406d8a0 0x7f2b1406fd50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c1975b0 tx=0x7f2b20009190 comp rx=0 tx=0).stop 2024-02-19T19:11:45.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.782+0000 7f2b33da6700 1 -- 172.21.15.88:0/1722495083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b2c1004e0 msgr2=0x7f2b2c195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:45.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.782+0000 7f2b33da6700 1 --2- 172.21.15.88:0/1722495083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b2c1004e0 0x7f2b2c195f90 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2b24000f80 tx=0x7f2b2400cd20 comp rx=0 tx=0).stop 2024-02-19T19:11:45.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.782+0000 7f2b33da6700 1 -- 172.21.15.88:0/1722495083 shutdown_connections 2024-02-19T19:11:45.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.782+0000 7f2b33da6700 1 --2- 172.21.15.88:0/1722495083 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2b1406d8a0 0x7f2b1406fd50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:45.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.782+0000 7f2b33da6700 1 --2- 172.21.15.88:0/1722495083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b2c1004e0 0x7f2b2c195f90 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:45.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.782+0000 7f2b33da6700 1 --2- 172.21.15.88:0/1722495083 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2b2c100e80 0x7f2b2c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:45.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.782+0000 7f2b33da6700 1 -- 172.21.15.88:0/1722495083 >> 172.21.15.88:0/1722495083 conn(0x7f2b2c0fbda0 msgr2=0x7f2b2c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:45.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.783+0000 7f2b33da6700 1 -- 172.21.15.88:0/1722495083 shutdown_connections 2024-02-19T19:11:45.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.783+0000 7f2b33da6700 1 -- 172.21.15.88:0/1722495083 wait complete. 2024-02-19T19:11:45.815 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:11:45.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.888+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/3268483666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb480ffbd0 msgr2=0x7fbb48100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:45.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.888+0000 7fbb4ffc5700 1 --2- 172.21.15.88:0/3268483666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb480ffbd0 0x7fbb48100040 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009910 tx=0x7fbb38009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:45.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.888+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/3268483666 shutdown_connections 2024-02-19T19:11:45.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.888+0000 7fbb4ffc5700 1 --2- 172.21.15.88:0/3268483666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb480ffbd0 0x7fbb48100040 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:45.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.888+0000 7fbb4ffc5700 1 --2- 172.21.15.88:0/3268483666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb48104130 0x7fbb48104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:45.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.888+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/3268483666 >> 172.21.15.88:0/3268483666 conn(0x7fbb48075130 msgr2=0x7fbb48075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:45.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.888+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/3268483666 shutdown_connections 2024-02-19T19:11:45.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.888+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/3268483666 wait complete. 2024-02-19T19:11:45.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4ffc5700 1 Processor -- start 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4ffc5700 1 -- start start 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4ffc5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb480ffbd0 0x7fbb48196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4ffc5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb48104130 0x7fbb48197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4ffc5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb48197920 con 0x7fbb48104130 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4ffc5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb48197a60 con 0x7fbb480ffbd0 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4d560700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb48104130 0x7fbb48197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4d560700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb48104130 0x7fbb48197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42614/0 (socket says 172.21.15.88:42614) 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4d560700 1 -- 172.21.15.88:0/1057186439 learned_addr learned my addr 172.21.15.88:0/1057186439 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4dd61700 1 --2- 172.21.15.88:0/1057186439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb480ffbd0 0x7fbb48196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4d560700 1 -- 172.21.15.88:0/1057186439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb480ffbd0 msgr2=0x7fbb48196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4d560700 1 --2- 172.21.15.88:0/1057186439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb480ffbd0 0x7fbb48196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4d560700 1 -- 172.21.15.88:0/1057186439 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb380095f0 con 0x7fbb48104130 2024-02-19T19:11:45.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.889+0000 7fbb4d560700 1 --2- 172.21.15.88:0/1057186439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb48104130 0x7fbb48197300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800bd70 tx=0x7fbb3800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:45.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.890+0000 7fbb3effd700 1 -- 172.21.15.88:0/1057186439 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb3801d070 con 0x7fbb48104130 2024-02-19T19:11:45.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.890+0000 7fbb3effd700 1 -- 172.21.15.88:0/1057186439 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb3800f460 con 0x7fbb48104130 2024-02-19T19:11:45.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.890+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/1057186439 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb4819fe30 con 0x7fbb48104130 2024-02-19T19:11:45.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.890+0000 7fbb3effd700 1 -- 172.21.15.88:0/1057186439 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb38017630 con 0x7fbb48104130 2024-02-19T19:11:45.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.890+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/1057186439 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb481a0240 con 0x7fbb48104130 2024-02-19T19:11:45.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.890+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/1057186439 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4804f2a0 con 0x7fbb48104130 2024-02-19T19:11:45.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.891+0000 7fbb3effd700 1 -- 172.21.15.88:0/1057186439 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbb3800f5d0 con 0x7fbb48104130 2024-02-19T19:11:45.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.891+0000 7fbb3effd700 1 --2- 172.21.15.88:0/1057186439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb3406d8a0 0x7fbb3406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:45.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.891+0000 7fbb3effd700 1 -- 172.21.15.88:0/1057186439 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbb3808eb20 con 0x7fbb48104130 2024-02-19T19:11:45.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.891+0000 7fbb4dd61700 1 --2- 172.21.15.88:0/1057186439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb3406d8a0 0x7fbb3406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:45.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.892+0000 7fbb4dd61700 1 --2- 172.21.15.88:0/1057186439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb3406d8a0 0x7fbb3406fd50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009f20 tx=0x7fbb400094f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:45.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:45.896+0000 7fbb3effd700 1 -- 172.21.15.88:0/1057186439 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb380582d0 con 0x7fbb48104130 2024-02-19T19:11:46.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.113+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/1057186439 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb480ff3a0 con 0x7fbb3406d8a0 2024-02-19T19:11:46.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.115+0000 7fbb3effd700 1 -- 172.21.15.88:0/1057186439 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbb480ff3a0 con 0x7fbb3406d8a0 2024-02-19T19:11:46.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.117+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/1057186439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb3406d8a0 msgr2=0x7fbb3406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.117+0000 7fbb4ffc5700 1 --2- 172.21.15.88:0/1057186439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb3406d8a0 0x7fbb3406fd50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009f20 tx=0x7fbb400094f0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.117+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/1057186439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb48104130 msgr2=0x7fbb48197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.117+0000 7fbb4ffc5700 1 --2- 172.21.15.88:0/1057186439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb48104130 0x7fbb48197300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800bd70 tx=0x7fbb3800be50 comp rx=0 tx=0).stop 2024-02-19T19:11:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.117+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/1057186439 shutdown_connections 2024-02-19T19:11:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.117+0000 7fbb4ffc5700 1 --2- 172.21.15.88:0/1057186439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb3406d8a0 0x7fbb3406fd50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.117+0000 7fbb4ffc5700 1 --2- 172.21.15.88:0/1057186439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb480ffbd0 0x7fbb48196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.117+0000 7fbb4ffc5700 1 --2- 172.21.15.88:0/1057186439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb48104130 0x7fbb48197300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.117+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/1057186439 >> 172.21.15.88:0/1057186439 conn(0x7fbb48075130 msgr2=0x7fbb480fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.117+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/1057186439 shutdown_connections 2024-02-19T19:11:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.117+0000 7fbb4ffc5700 1 -- 172.21.15.88:0/1057186439 wait complete. 2024-02-19T19:11:46.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.220+0000 7eff6884a700 1 -- 172.21.15.88:0/845349800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff640ff350 msgr2=0x7eff640ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.220+0000 7eff6884a700 1 --2- 172.21.15.88:0/845349800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff640ff350 0x7eff640ff720 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7eff54009910 tx=0x7eff54009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:46.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.221+0000 7eff6884a700 1 -- 172.21.15.88:0/845349800 shutdown_connections 2024-02-19T19:11:46.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.221+0000 7eff6884a700 1 --2- 172.21.15.88:0/845349800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff640ffc60 0x7eff6410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.221+0000 7eff6884a700 1 --2- 172.21.15.88:0/845349800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff640ff350 0x7eff640ff720 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.221+0000 7eff6884a700 1 -- 172.21.15.88:0/845349800 >> 172.21.15.88:0/845349800 conn(0x7eff640748b0 msgr2=0x7eff64074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:46.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.221+0000 7eff6884a700 1 -- 172.21.15.88:0/845349800 shutdown_connections 2024-02-19T19:11:46.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.221+0000 7eff6884a700 1 -- 172.21.15.88:0/845349800 wait complete. 2024-02-19T19:11:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.221+0000 7eff6884a700 1 Processor -- start 2024-02-19T19:11:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.221+0000 7eff6884a700 1 -- start start 2024-02-19T19:11:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff6884a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff640ff350 0x7eff64100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff6884a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff640ffc60 0x7eff64100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff6884a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff64100fb0 con 0x7eff640ff350 2024-02-19T19:11:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff6884a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff641010f0 con 0x7eff640ffc60 2024-02-19T19:11:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff6259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff640ff350 0x7eff64100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff6259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff640ff350 0x7eff64100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42632/0 (socket says 172.21.15.88:42632) 2024-02-19T19:11:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff6259c700 1 -- 172.21.15.88:0/945867073 learned_addr learned my addr 172.21.15.88:0/945867073 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:46.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff61d9b700 1 --2- 172.21.15.88:0/945867073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff640ffc60 0x7eff64100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:46.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff6259c700 1 -- 172.21.15.88:0/945867073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff640ffc60 msgr2=0x7eff64100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff6259c700 1 --2- 172.21.15.88:0/945867073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff640ffc60 0x7eff64100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff6259c700 1 -- 172.21.15.88:0/945867073 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff540095f0 con 0x7eff640ff350 2024-02-19T19:11:46.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.222+0000 7eff6259c700 1 --2- 172.21.15.88:0/945867073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff640ff350 0x7eff64100530 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7eff54005f50 tx=0x7eff5400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:46.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.223+0000 7eff537fe700 1 -- 172.21.15.88:0/945867073 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff5401d070 con 0x7eff640ff350 2024-02-19T19:11:46.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.223+0000 7eff537fe700 1 -- 172.21.15.88:0/945867073 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff5400f460 con 0x7eff640ff350 2024-02-19T19:11:46.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.223+0000 7eff6884a700 1 -- 172.21.15.88:0/945867073 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff64101370 con 0x7eff640ff350 2024-02-19T19:11:46.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.223+0000 7eff537fe700 1 -- 172.21.15.88:0/945867073 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff54017630 con 0x7eff640ff350 2024-02-19T19:11:46.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.223+0000 7eff6884a700 1 -- 172.21.15.88:0/945867073 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff641a24c0 con 0x7eff640ff350 2024-02-19T19:11:46.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.223+0000 7eff6884a700 1 -- 172.21.15.88:0/945867073 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff64107dd0 con 0x7eff640ff350 2024-02-19T19:11:46.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.224+0000 7eff537fe700 1 -- 172.21.15.88:0/945867073 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7eff5400f5d0 con 0x7eff640ff350 2024-02-19T19:11:46.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.225+0000 7eff537fe700 1 --2- 172.21.15.88:0/945867073 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7eff4c06d8a0 0x7eff4c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:46.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.225+0000 7eff537fe700 1 -- 172.21.15.88:0/945867073 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7eff5408ead0 con 0x7eff640ff350 2024-02-19T19:11:46.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.225+0000 7eff61d9b700 1 --2- 172.21.15.88:0/945867073 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7eff4c06d8a0 0x7eff4c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:46.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.226+0000 7eff61d9b700 1 --2- 172.21.15.88:0/945867073 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7eff4c06d8a0 0x7eff4c06fd50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff58006fd0 tx=0x7eff58008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.230+0000 7eff537fe700 1 -- 172.21.15.88:0/945867073 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7eff54058280 con 0x7eff640ff350 2024-02-19T19:11:46.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.436+0000 7eff6884a700 1 -- 172.21.15.88:0/945867073 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7eff64101b70 con 0x7eff4c06d8a0 2024-02-19T19:11:46.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.447+0000 7eff537fe700 1 -- 172.21.15.88:0/945867073 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7eff64101b70 con 0x7eff4c06d8a0 2024-02-19T19:11:46.469 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:11:46.469 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (8m) 47s ago 19m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:11:46.470 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (19m) 47s ago 19m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:11:46.470 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (106s) 29s ago 17m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:11:46.470 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (5m) 47s ago 19m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:11:46.470 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (4m) 29s ago 17m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:11:46.470 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (8m) 47s ago 18m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:11:46.470 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (71s) 47s ago 13m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:11:46.470 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (52s) 47s ago 13m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:11:46.470 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (33s) 29s ago 13m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:11:46.470 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (12m) 29s ago 12m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:11:46.470 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (10m) 47s ago 21m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:11:46.471 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (9m) 29s ago 17m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:11:46.471 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (7m) 47s ago 21m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:11:46.471 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (6m) 29s ago 17m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:11:46.471 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (9m) 47s ago 19m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:11:46.471 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (9m) 29s ago 17m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:11:46.471 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (4m) 47s ago 16m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:11:46.471 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (3m) 47s ago 16m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:11:46.471 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (3m) 47s ago 15m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:11:46.471 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (2m) 29s ago 15m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:11:46.471 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (2m) 29s ago 15m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:11:46.471 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (87s) 29s ago 14m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:11:46.472 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (9m) 47s ago 18m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:11:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.449+0000 7eff6884a700 1 -- 172.21.15.88:0/945867073 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7eff4c06d8a0 msgr2=0x7eff4c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.449+0000 7eff6884a700 1 --2- 172.21.15.88:0/945867073 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7eff4c06d8a0 0x7eff4c06fd50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7eff58006fd0 tx=0x7eff58008040 comp rx=0 tx=0).stop 2024-02-19T19:11:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.449+0000 7eff6884a700 1 -- 172.21.15.88:0/945867073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff640ff350 msgr2=0x7eff64100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.449+0000 7eff6884a700 1 --2- 172.21.15.88:0/945867073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff640ff350 0x7eff64100530 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7eff54005f50 tx=0x7eff5400be20 comp rx=0 tx=0).stop 2024-02-19T19:11:46.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.449+0000 7eff6884a700 1 -- 172.21.15.88:0/945867073 shutdown_connections 2024-02-19T19:11:46.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.449+0000 7eff6884a700 1 --2- 172.21.15.88:0/945867073 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7eff4c06d8a0 0x7eff4c06fd50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.449+0000 7eff6884a700 1 --2- 172.21.15.88:0/945867073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff640ff350 0x7eff64100530 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.449+0000 7eff6884a700 1 --2- 172.21.15.88:0/945867073 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7eff640ffc60 0x7eff64100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.449+0000 7eff6884a700 1 -- 172.21.15.88:0/945867073 >> 172.21.15.88:0/945867073 conn(0x7eff640748b0 msgr2=0x7eff64104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:46.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.449+0000 7eff6884a700 1 -- 172.21.15.88:0/945867073 shutdown_connections 2024-02-19T19:11:46.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.449+0000 7eff6884a700 1 -- 172.21.15.88:0/945867073 wait complete. 2024-02-19T19:11:46.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.553+0000 7fa03354f700 1 -- 172.21.15.88:0/1838385315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02c0ffd60 msgr2=0x7fa02c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.553+0000 7fa03354f700 1 --2- 172.21.15.88:0/1838385315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02c0ffd60 0x7fa02c103d30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009910 tx=0x7fa01c009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:46.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.553+0000 7fa03354f700 1 -- 172.21.15.88:0/1838385315 shutdown_connections 2024-02-19T19:11:46.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.553+0000 7fa03354f700 1 --2- 172.21.15.88:0/1838385315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02c0ffd60 0x7fa02c103d30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.553+0000 7fa03354f700 1 --2- 172.21.15.88:0/1838385315 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa02c0ff3c0 0x7fa02c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.553+0000 7fa03354f700 1 -- 172.21.15.88:0/1838385315 >> 172.21.15.88:0/1838385315 conn(0x7fa02c0fac60 msgr2=0x7fa02c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:46.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.553+0000 7fa03354f700 1 -- 172.21.15.88:0/1838385315 shutdown_connections 2024-02-19T19:11:46.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.553+0000 7fa03354f700 1 -- 172.21.15.88:0/1838385315 wait complete. 2024-02-19T19:11:46.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.554+0000 7fa03354f700 1 Processor -- start 2024-02-19T19:11:46.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.554+0000 7fa03354f700 1 -- start start 2024-02-19T19:11:46.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.554+0000 7fa03354f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa02c0ff3c0 0x7fa02c194b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:46.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.554+0000 7fa03354f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02c0ffd60 0x7fa02c195070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:46.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.554+0000 7fa03354f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c1955b0 con 0x7fa02c0ffd60 2024-02-19T19:11:46.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.554+0000 7fa03354f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa02c195720 con 0x7fa02c0ff3c0 2024-02-19T19:11:46.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.554+0000 7fa030aea700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02c0ffd60 0x7fa02c195070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:46.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.554+0000 7fa030aea700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02c0ffd60 0x7fa02c195070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42646/0 (socket says 172.21.15.88:42646) 2024-02-19T19:11:46.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.554+0000 7fa030aea700 1 -- 172.21.15.88:0/2082963360 learned_addr learned my addr 172.21.15.88:0/2082963360 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:46.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.554+0000 7fa0312eb700 1 --2- 172.21.15.88:0/2082963360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa02c0ff3c0 0x7fa02c194b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:46.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.555+0000 7fa030aea700 1 -- 172.21.15.88:0/2082963360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa02c0ff3c0 msgr2=0x7fa02c194b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.555+0000 7fa030aea700 1 --2- 172.21.15.88:0/2082963360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa02c0ff3c0 0x7fa02c194b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.555+0000 7fa030aea700 1 -- 172.21.15.88:0/2082963360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c0095f0 con 0x7fa02c0ffd60 2024-02-19T19:11:46.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.555+0000 7fa030aea700 1 --2- 172.21.15.88:0/2082963360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02c0ffd60 0x7fa02c195070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00bd70 tx=0x7fa01c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:46.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.555+0000 7fa0227fc700 1 -- 172.21.15.88:0/2082963360 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa01c01d070 con 0x7fa02c0ffd60 2024-02-19T19:11:46.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.555+0000 7fa0227fc700 1 -- 172.21.15.88:0/2082963360 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa01c005dc0 con 0x7fa02c0ffd60 2024-02-19T19:11:46.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.555+0000 7fa03354f700 1 -- 172.21.15.88:0/2082963360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02c1959a0 con 0x7fa02c0ffd60 2024-02-19T19:11:46.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.555+0000 7fa0227fc700 1 -- 172.21.15.88:0/2082963360 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa01c017400 con 0x7fa02c0ffd60 2024-02-19T19:11:46.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.555+0000 7fa03354f700 1 -- 172.21.15.88:0/2082963360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02c1a2420 con 0x7fa02c0ffd60 2024-02-19T19:11:46.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.556+0000 7fa03354f700 1 -- 172.21.15.88:0/2082963360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02c107730 con 0x7fa02c0ffd60 2024-02-19T19:11:46.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.557+0000 7fa0227fc700 1 -- 172.21.15.88:0/2082963360 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa01c00f460 con 0x7fa02c0ffd60 2024-02-19T19:11:46.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.558+0000 7fa0227fc700 1 --2- 172.21.15.88:0/2082963360 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa01806d8a0 0x7fa01806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:46.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.558+0000 7fa0227fc700 1 -- 172.21.15.88:0/2082963360 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa01c08ea00 con 0x7fa02c0ffd60 2024-02-19T19:11:46.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.558+0000 7fa0312eb700 1 --2- 172.21.15.88:0/2082963360 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa01806d8a0 0x7fa01806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:46.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.558+0000 7fa0312eb700 1 --2- 172.21.15.88:0/2082963360 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa01806d8a0 0x7fa01806fd50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa024009550 tx=0x7fa0240091b0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.562+0000 7fa0227fc700 1 -- 172.21.15.88:0/2082963360 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa01c05bd60 con 0x7fa02c0ffd60 2024-02-19T19:11:46.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.840+0000 7fa03354f700 1 -- 172.21.15.88:0/2082963360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa02c1959a0 con 0x7fa02c0ffd60 2024-02-19T19:11:46.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.841+0000 7fa0227fc700 1 -- 172.21.15.88:0/2082963360 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fa02c1959a0 con 0x7fa02c0ffd60 2024-02-19T19:11:46.863 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:11:46.863 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:11:46.863 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:11:46.863 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:11:46.863 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:11:46.863 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:11:46.863 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:11:46.864 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:11:46.864 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:11:46.864 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:11:46.864 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:11:46.864 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:11:46.864 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:11:46.864 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:11:46.864 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:11:46.864 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:11:46.864 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:11:46.864 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:11:46.865 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:11:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.843+0000 7fa03354f700 1 -- 172.21.15.88:0/2082963360 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa01806d8a0 msgr2=0x7fa01806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.843+0000 7fa03354f700 1 --2- 172.21.15.88:0/2082963360 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa01806d8a0 0x7fa01806fd50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa024009550 tx=0x7fa0240091b0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.843+0000 7fa03354f700 1 -- 172.21.15.88:0/2082963360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02c0ffd60 msgr2=0x7fa02c195070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.843+0000 7fa03354f700 1 --2- 172.21.15.88:0/2082963360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02c0ffd60 0x7fa02c195070 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa01c00bd70 tx=0x7fa01c00bda0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.843+0000 7fa03354f700 1 -- 172.21.15.88:0/2082963360 shutdown_connections 2024-02-19T19:11:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.843+0000 7fa03354f700 1 --2- 172.21.15.88:0/2082963360 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa01806d8a0 0x7fa01806fd50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.843+0000 7fa03354f700 1 --2- 172.21.15.88:0/2082963360 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa02c0ff3c0 0x7fa02c194b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.843+0000 7fa03354f700 1 --2- 172.21.15.88:0/2082963360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa02c0ffd60 0x7fa02c195070 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.843+0000 7fa03354f700 1 -- 172.21.15.88:0/2082963360 >> 172.21.15.88:0/2082963360 conn(0x7fa02c0fac60 msgr2=0x7fa02c0fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.843+0000 7fa03354f700 1 -- 172.21.15.88:0/2082963360 shutdown_connections 2024-02-19T19:11:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.843+0000 7fa03354f700 1 -- 172.21.15.88:0/2082963360 wait complete. 2024-02-19T19:11:46.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.947+0000 7fbd7223a700 1 -- 172.21.15.88:0/21881579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6c100e80 msgr2=0x7fbd6c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.947+0000 7fbd7223a700 1 --2- 172.21.15.88:0/21881579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6c100e80 0x7fbd6c104e50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009910 tx=0x7fbd60009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:46.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.947+0000 7fbd7223a700 1 -- 172.21.15.88:0/21881579 shutdown_connections 2024-02-19T19:11:46.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.947+0000 7fbd7223a700 1 --2- 172.21.15.88:0/21881579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6c100e80 0x7fbd6c104e50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.947+0000 7fbd7223a700 1 --2- 172.21.15.88:0/21881579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd6c1004e0 0x7fbd6c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.947+0000 7fbd7223a700 1 -- 172.21.15.88:0/21881579 >> 172.21.15.88:0/21881579 conn(0x7fbd6c0fbda0 msgr2=0x7fbd6c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.948+0000 7fbd7223a700 1 -- 172.21.15.88:0/21881579 shutdown_connections 2024-02-19T19:11:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.948+0000 7fbd7223a700 1 -- 172.21.15.88:0/21881579 wait complete. 2024-02-19T19:11:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.948+0000 7fbd7223a700 1 Processor -- start 2024-02-19T19:11:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.948+0000 7fbd7223a700 1 -- start start 2024-02-19T19:11:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.948+0000 7fbd7223a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd6c1004e0 0x7fbd6c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.948+0000 7fbd7223a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6c100e80 0x7fbd6c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.948+0000 7fbd7223a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c198ce0 con 0x7fbd6c100e80 2024-02-19T19:11:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.948+0000 7fbd7223a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c19ca70 con 0x7fbd6c1004e0 2024-02-19T19:11:46.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd6affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6c100e80 0x7fbd6c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:46.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd6affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6c100e80 0x7fbd6c198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42662/0 (socket says 172.21.15.88:42662) 2024-02-19T19:11:46.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd6affd700 1 -- 172.21.15.88:0/84225230 learned_addr learned my addr 172.21.15.88:0/84225230 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:46.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd6b7fe700 1 --2- 172.21.15.88:0/84225230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd6c1004e0 0x7fbd6c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:46.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd6affd700 1 -- 172.21.15.88:0/84225230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd6c1004e0 msgr2=0x7fbd6c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:46.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd6affd700 1 --2- 172.21.15.88:0/84225230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd6c1004e0 0x7fbd6c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:46.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd6affd700 1 -- 172.21.15.88:0/84225230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd600095f0 con 0x7fbd6c100e80 2024-02-19T19:11:46.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd6affd700 1 --2- 172.21.15.88:0/84225230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6c100e80 0x7fbd6c198600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbd600092e0 tx=0x7fbd6000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:46.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd68ff9700 1 -- 172.21.15.88:0/84225230 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd6001d070 con 0x7fbd6c100e80 2024-02-19T19:11:46.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd68ff9700 1 -- 172.21.15.88:0/84225230 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd60005dc0 con 0x7fbd6c100e80 2024-02-19T19:11:46.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd7223a700 1 -- 172.21.15.88:0/84225230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c19ccf0 con 0x7fbd6c100e80 2024-02-19T19:11:46.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd68ff9700 1 -- 172.21.15.88:0/84225230 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd60017400 con 0x7fbd6c100e80 2024-02-19T19:11:46.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.949+0000 7fbd7223a700 1 -- 172.21.15.88:0/84225230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c19d1e0 con 0x7fbd6c100e80 2024-02-19T19:11:46.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.950+0000 7fbd7223a700 1 -- 172.21.15.88:0/84225230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c104ce0 con 0x7fbd6c100e80 2024-02-19T19:11:46.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.951+0000 7fbd68ff9700 1 -- 172.21.15.88:0/84225230 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbd6000f6f0 con 0x7fbd6c100e80 2024-02-19T19:11:46.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.951+0000 7fbd68ff9700 1 --2- 172.21.15.88:0/84225230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd5406d8a0 0x7fbd5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:46.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.951+0000 7fbd6b7fe700 1 --2- 172.21.15.88:0/84225230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd5406d8a0 0x7fbd5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:46.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.952+0000 7fbd68ff9700 1 -- 172.21.15.88:0/84225230 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbd6008e910 con 0x7fbd6c100e80 2024-02-19T19:11:46.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.952+0000 7fbd6b7fe700 1 --2- 172.21.15.88:0/84225230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd5406d8a0 0x7fbd5406fd50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c004630 tx=0x7fbd5c00b3c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:46.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:46.956+0000 7fbd68ff9700 1 -- 172.21.15.88:0/84225230 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd600580c0 con 0x7fbd6c100e80 2024-02-19T19:11:47.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.197+0000 7fbd7223a700 1 -- 172.21.15.88:0/84225230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbd6c1994c0 con 0x7fbd6c100e80 2024-02-19T19:11:47.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.197+0000 7fbd68ff9700 1 -- 172.21.15.88:0/84225230 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fbd6005b780 con 0x7fbd6c100e80 2024-02-19T19:11:47.220 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:11:47.220 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:11:47.220 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:11:47.220 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:11:47.220 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:11:47.220 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:11:47.220 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:11:47.220 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:11:47.220 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:11:47.220 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:11:47.220 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:11:47.221 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:11:47.222 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:11:47.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:11:47.223 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:11:47.223 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:11:47.223 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:11:47.223 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:11:47.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.199+0000 7fbd7223a700 1 -- 172.21.15.88:0/84225230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd5406d8a0 msgr2=0x7fbd5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:47.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.199+0000 7fbd7223a700 1 --2- 172.21.15.88:0/84225230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd5406d8a0 0x7fbd5406fd50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c004630 tx=0x7fbd5c00b3c0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.199+0000 7fbd7223a700 1 -- 172.21.15.88:0/84225230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6c100e80 msgr2=0x7fbd6c198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.199+0000 7fbd7223a700 1 --2- 172.21.15.88:0/84225230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6c100e80 0x7fbd6c198600 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbd600092e0 tx=0x7fbd6000bd70 comp rx=0 tx=0).stop 2024-02-19T19:11:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.199+0000 7fbd7223a700 1 -- 172.21.15.88:0/84225230 shutdown_connections 2024-02-19T19:11:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.199+0000 7fbd7223a700 1 --2- 172.21.15.88:0/84225230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd5406d8a0 0x7fbd5406fd50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.199+0000 7fbd7223a700 1 --2- 172.21.15.88:0/84225230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd6c1004e0 0x7fbd6c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.199+0000 7fbd7223a700 1 --2- 172.21.15.88:0/84225230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd6c100e80 0x7fbd6c198600 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.199+0000 7fbd7223a700 1 -- 172.21.15.88:0/84225230 >> 172.21.15.88:0/84225230 conn(0x7fbd6c0fbda0 msgr2=0x7fbd6c101e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.200+0000 7fbd7223a700 1 -- 172.21.15.88:0/84225230 shutdown_connections 2024-02-19T19:11:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.200+0000 7fbd7223a700 1 -- 172.21.15.88:0/84225230 wait complete. 2024-02-19T19:11:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:11:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.305+0000 7fd98956f700 1 -- 172.21.15.88:0/3370278007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9840ffd60 msgr2=0x7fd984103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.305+0000 7fd98956f700 1 --2- 172.21.15.88:0/3370278007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9840ffd60 0x7fd984103d30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd974009910 tx=0x7fd974009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.305+0000 7fd98956f700 1 -- 172.21.15.88:0/3370278007 shutdown_connections 2024-02-19T19:11:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.305+0000 7fd98956f700 1 --2- 172.21.15.88:0/3370278007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9840ffd60 0x7fd984103d30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.305+0000 7fd98956f700 1 --2- 172.21.15.88:0/3370278007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9840ff3c0 0x7fd9840ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.305+0000 7fd98956f700 1 -- 172.21.15.88:0/3370278007 >> 172.21.15.88:0/3370278007 conn(0x7fd9840fac80 msgr2=0x7fd9840fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.305+0000 7fd98956f700 1 -- 172.21.15.88:0/3370278007 shutdown_connections 2024-02-19T19:11:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.305+0000 7fd98956f700 1 -- 172.21.15.88:0/3370278007 wait complete. 2024-02-19T19:11:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd98956f700 1 Processor -- start 2024-02-19T19:11:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd98956f700 1 -- start start 2024-02-19T19:11:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd98956f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9840ff3c0 0x7fd984194b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd98956f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9840ffd60 0x7fd984195090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd98956f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9841955d0 con 0x7fd9840ffd60 2024-02-19T19:11:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd98956f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd984195740 con 0x7fd9840ff3c0 2024-02-19T19:11:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd9827fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9840ffd60 0x7fd984195090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd9827fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9840ffd60 0x7fd984195090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42678/0 (socket says 172.21.15.88:42678) 2024-02-19T19:11:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd9827fc700 1 -- 172.21.15.88:0/1691347259 learned_addr learned my addr 172.21.15.88:0/1691347259 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd9827fc700 1 -- 172.21.15.88:0/1691347259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9840ff3c0 msgr2=0x7fd984194b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd9827fc700 1 --2- 172.21.15.88:0/1691347259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9840ff3c0 0x7fd984194b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd9827fc700 1 -- 172.21.15.88:0/1691347259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9740095f0 con 0x7fd9840ffd60 2024-02-19T19:11:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.306+0000 7fd9827fc700 1 --2- 172.21.15.88:0/1691347259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9840ffd60 0x7fd984195090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd97400bda0 tx=0x7fd97400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:47.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.307+0000 7fd96bfff700 1 -- 172.21.15.88:0/1691347259 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd97401d070 con 0x7fd9840ffd60 2024-02-19T19:11:47.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.307+0000 7fd98956f700 1 -- 172.21.15.88:0/1691347259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9841959c0 con 0x7fd9840ffd60 2024-02-19T19:11:47.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.307+0000 7fd96bfff700 1 -- 172.21.15.88:0/1691347259 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd974004fb0 con 0x7fd9840ffd60 2024-02-19T19:11:47.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.307+0000 7fd96bfff700 1 -- 172.21.15.88:0/1691347259 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd97400f460 con 0x7fd9840ffd60 2024-02-19T19:11:47.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.307+0000 7fd98956f700 1 -- 172.21.15.88:0/1691347259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9841a23e0 con 0x7fd9840ffd60 2024-02-19T19:11:47.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.308+0000 7fd98956f700 1 -- 172.21.15.88:0/1691347259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd984107730 con 0x7fd9840ffd60 2024-02-19T19:11:47.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.308+0000 7fd96bfff700 1 -- 172.21.15.88:0/1691347259 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd974004af0 con 0x7fd9840ffd60 2024-02-19T19:11:47.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.308+0000 7fd96bfff700 1 --2- 172.21.15.88:0/1691347259 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd96c06d8a0 0x7fd96c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:47.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.309+0000 7fd96bfff700 1 -- 172.21.15.88:0/1691347259 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd97408e910 con 0x7fd9840ffd60 2024-02-19T19:11:47.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.309+0000 7fd982ffd700 1 --2- 172.21.15.88:0/1691347259 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd96c06d8a0 0x7fd96c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:47.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.309+0000 7fd982ffd700 1 --2- 172.21.15.88:0/1691347259 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd96c06d8a0 0x7fd96c06fd50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd978005fd0 tx=0x7fd978005d00 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:47.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.313+0000 7fd96bfff700 1 -- 172.21.15.88:0/1691347259 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd97405be40 con 0x7fd9840ffd60 2024-02-19T19:11:47.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.527+0000 7fd98956f700 1 -- 172.21.15.88:0/1691347259 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9841959c0 con 0x7fd96c06d8a0 2024-02-19T19:11:47.550 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:47 smithi088 ceph-mon[88643]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:47.550 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:47 smithi088 ceph-mon[88643]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:47.550 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:47 smithi088 ceph-mon[88643]: pgmap v245: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:47.550 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:47 smithi088 ceph-mon[88643]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:47.550 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:47 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2082963360' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:47.550 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:47 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/84225230' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:11:47.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.529+0000 7fd96bfff700 1 -- 172.21.15.88:0/1691347259 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd9841959c0 con 0x7fd96c06d8a0 2024-02-19T19:11:47.551 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:11:47.551 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:11:47.551 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:11:47.551 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:11:47.551 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:11:47.551 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:11:47.552 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:11:47.552 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:11:47.552 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:11:47.552 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:11:47.552 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:11:47.552 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:11:47.552 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:11:47.552 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:11:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.531+0000 7fd98956f700 1 -- 172.21.15.88:0/1691347259 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd96c06d8a0 msgr2=0x7fd96c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.531+0000 7fd98956f700 1 --2- 172.21.15.88:0/1691347259 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd96c06d8a0 0x7fd96c06fd50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd978005fd0 tx=0x7fd978005d00 comp rx=0 tx=0).stop 2024-02-19T19:11:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.531+0000 7fd98956f700 1 -- 172.21.15.88:0/1691347259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9840ffd60 msgr2=0x7fd984195090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.531+0000 7fd98956f700 1 --2- 172.21.15.88:0/1691347259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9840ffd60 0x7fd984195090 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd97400bda0 tx=0x7fd97400be80 comp rx=0 tx=0).stop 2024-02-19T19:11:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.531+0000 7fd98956f700 1 -- 172.21.15.88:0/1691347259 shutdown_connections 2024-02-19T19:11:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.531+0000 7fd98956f700 1 --2- 172.21.15.88:0/1691347259 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd96c06d8a0 0x7fd96c06fd50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.531+0000 7fd98956f700 1 --2- 172.21.15.88:0/1691347259 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9840ff3c0 0x7fd984194b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.531+0000 7fd98956f700 1 --2- 172.21.15.88:0/1691347259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9840ffd60 0x7fd984195090 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.531+0000 7fd98956f700 1 -- 172.21.15.88:0/1691347259 >> 172.21.15.88:0/1691347259 conn(0x7fd9840fac80 msgr2=0x7fd9840fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:47.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.531+0000 7fd98956f700 1 -- 172.21.15.88:0/1691347259 shutdown_connections 2024-02-19T19:11:47.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.531+0000 7fd98956f700 1 -- 172.21.15.88:0/1691347259 wait complete. 2024-02-19T19:11:47.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.636+0000 7f41dc322700 1 -- 172.21.15.88:0/2794565294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d40ff720 msgr2=0x7f41d40ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.636+0000 7f41dc322700 1 --2- 172.21.15.88:0/2794565294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d40ff720 0x7f41d40ffb90 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f41c4009910 tx=0x7f41c4009c20 comp rx=0 tx=0).stop 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.636+0000 7f41dc322700 1 -- 172.21.15.88:0/2794565294 shutdown_connections 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.636+0000 7f41dc322700 1 --2- 172.21.15.88:0/2794565294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d40ff720 0x7f41d40ffb90 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.636+0000 7f41dc322700 1 --2- 172.21.15.88:0/2794565294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40ff350 0x7f41d41063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.636+0000 7f41dc322700 1 -- 172.21.15.88:0/2794565294 >> 172.21.15.88:0/2794565294 conn(0x7f41d40748b0 msgr2=0x7f41d4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.636+0000 7f41dc322700 1 -- 172.21.15.88:0/2794565294 shutdown_connections 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.636+0000 7f41dc322700 1 -- 172.21.15.88:0/2794565294 wait complete. 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.636+0000 7f41dc322700 1 Processor -- start 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.636+0000 7f41dc322700 1 -- start start 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41dc322700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d40ff350 0x7f41d419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41dc322700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40ff720 0x7f41d419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41dc322700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d419d030 con 0x7f41d40ff350 2024-02-19T19:11:47.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41dc322700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d41a0dc0 con 0x7f41d40ff720 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41da0be700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d40ff350 0x7f41d419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41da0be700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d40ff350 0x7f41d419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42708/0 (socket says 172.21.15.88:42708) 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41da0be700 1 -- 172.21.15.88:0/72687866 learned_addr learned my addr 172.21.15.88:0/72687866 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41d98bd700 1 --2- 172.21.15.88:0/72687866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40ff720 0x7f41d419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41da0be700 1 -- 172.21.15.88:0/72687866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40ff720 msgr2=0x7f41d419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41da0be700 1 --2- 172.21.15.88:0/72687866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40ff720 0x7f41d419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41da0be700 1 -- 172.21.15.88:0/72687866 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c40095f0 con 0x7f41d40ff350 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41da0be700 1 --2- 172.21.15.88:0/72687866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d40ff350 0x7f41d419c410 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c8a0 tx=0x7f41d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41cb7fe700 1 -- 172.21.15.88:0/72687866 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41d000cea0 con 0x7f41d40ff350 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41dc322700 1 -- 172.21.15.88:0/72687866 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d41a10a0 con 0x7f41d40ff350 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41cb7fe700 1 -- 172.21.15.88:0/72687866 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41d0004830 con 0x7f41d40ff350 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41cb7fe700 1 -- 172.21.15.88:0/72687866 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41d00055b0 con 0x7f41d40ff350 2024-02-19T19:11:47.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.637+0000 7f41dc322700 1 -- 172.21.15.88:0/72687866 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d41a15f0 con 0x7f41d40ff350 2024-02-19T19:11:47.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.638+0000 7f41dc322700 1 -- 172.21.15.88:0/72687866 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d4103ae0 con 0x7f41d40ff350 2024-02-19T19:11:47.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.639+0000 7f41cb7fe700 1 -- 172.21.15.88:0/72687866 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f41d00049a0 con 0x7f41d40ff350 2024-02-19T19:11:47.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.639+0000 7f41cb7fe700 1 --2- 172.21.15.88:0/72687866 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f41c006d8a0 0x7f41c006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:11:47.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.639+0000 7f41cb7fe700 1 -- 172.21.15.88:0/72687866 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f41d008d1f0 con 0x7f41d40ff350 2024-02-19T19:11:47.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.639+0000 7f41d98bd700 1 --2- 172.21.15.88:0/72687866 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f41c006d8a0 0x7f41c006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:11:47.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.640+0000 7f41d98bd700 1 --2- 172.21.15.88:0/72687866 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f41c006d8a0 0x7f41c006fd50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f41c40059d0 tx=0x7f41c401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:11:47.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.644+0000 7f41cb7fe700 1 -- 172.21.15.88:0/72687866 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f41d00569a0 con 0x7f41d40ff350 2024-02-19T19:11:47.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:47 smithi132 ceph-mon[70676]: from='client.34498 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:47.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:47 smithi132 ceph-mon[70676]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:47.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:47 smithi132 ceph-mon[70676]: pgmap v245: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:47.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:47 smithi132 ceph-mon[70676]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:47.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:47 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2082963360' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:11:47.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:47 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/84225230' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:11:47.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.929+0000 7f41dc322700 1 -- 172.21.15.88:0/72687866 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f41d419d840 con 0x7f41d40ff350 2024-02-19T19:11:47.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.930+0000 7f41cb7fe700 1 -- 172.21.15.88:0/72687866 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f41d0004c50 con 0x7f41d40ff350 2024-02-19T19:11:47.952 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:11:47.952 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:11:47.952 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:11:47.952 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:11:47.952 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:11:47.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.932+0000 7f41dc322700 1 -- 172.21.15.88:0/72687866 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f41c006d8a0 msgr2=0x7f41c006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:47.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.932+0000 7f41dc322700 1 --2- 172.21.15.88:0/72687866 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f41c006d8a0 0x7f41c006fd50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f41c40059d0 tx=0x7f41c401a040 comp rx=0 tx=0).stop 2024-02-19T19:11:47.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.932+0000 7f41dc322700 1 -- 172.21.15.88:0/72687866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d40ff350 msgr2=0x7f41d419c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:11:47.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.932+0000 7f41dc322700 1 --2- 172.21.15.88:0/72687866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d40ff350 0x7f41d419c410 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f41d000c8a0 tx=0x7f41d000cc60 comp rx=0 tx=0).stop 2024-02-19T19:11:47.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.932+0000 7f41dc322700 1 -- 172.21.15.88:0/72687866 shutdown_connections 2024-02-19T19:11:47.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.932+0000 7f41dc322700 1 --2- 172.21.15.88:0/72687866 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f41c006d8a0 0x7f41c006fd50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.932+0000 7f41dc322700 1 --2- 172.21.15.88:0/72687866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41d40ff350 0x7f41d419c410 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.932+0000 7f41dc322700 1 --2- 172.21.15.88:0/72687866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41d40ff720 0x7f41d419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:11:47.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.932+0000 7f41dc322700 1 -- 172.21.15.88:0/72687866 >> 172.21.15.88:0/72687866 conn(0x7f41d40748b0 msgr2=0x7f41d40fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:11:47.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.932+0000 7f41dc322700 1 -- 172.21.15.88:0/72687866 shutdown_connections 2024-02-19T19:11:47.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:11:47.932+0000 7f41dc322700 1 -- 172.21.15.88:0/72687866 wait complete. 2024-02-19T19:11:48.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:48 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/72687866' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:11:48.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:48 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/72687866' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:11:49.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:49 smithi088 ceph-mon[88643]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:49.618 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:49 smithi088 ceph-mon[88643]: pgmap v246: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:49.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:49 smithi132 ceph-mon[70676]: from='client.34518 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:11:49.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:49 smithi132 ceph-mon[70676]: pgmap v246: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:51.571 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:51 smithi132 ceph-mon[70676]: pgmap v247: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:51.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:51 smithi088 ceph-mon[88643]: pgmap v247: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:53.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:53 smithi088 ceph-mon[88643]: pgmap v248: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:53.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:53 smithi132 ceph-mon[70676]: pgmap v248: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:55.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:55 smithi088 ceph-mon[88643]: pgmap v249: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:55.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:55 smithi132 ceph-mon[70676]: pgmap v249: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:57.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:57 smithi088 ceph-mon[88643]: pgmap v250: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:57.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:57 smithi132 ceph-mon[70676]: pgmap v250: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:58.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:11:58.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:11:59.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:11:59 smithi088 ceph-mon[88643]: pgmap v251: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:11:59.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:11:59 smithi132 ceph-mon[70676]: pgmap v251: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:01.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:01 smithi088 ceph-mon[88643]: pgmap v252: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:01.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:01 smithi132 ceph-mon[70676]: pgmap v252: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:03.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:03 smithi088 ceph-mon[88643]: pgmap v253: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:03.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:03 smithi132 ceph-mon[70676]: pgmap v253: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:05.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:05 smithi088 ceph-mon[88643]: pgmap v254: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:05.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:05 smithi132 ceph-mon[70676]: pgmap v254: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:07.504 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:07 smithi132 ceph-mon[70676]: pgmap v255: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:07.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:07 smithi088 ceph-mon[88643]: pgmap v255: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:09.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:09 smithi088 ceph-mon[88643]: pgmap v256: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:09.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:09 smithi132 ceph-mon[70676]: pgmap v256: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:11.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:11 smithi132 ceph-mon[70676]: pgmap v257: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:11.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:11 smithi088 ceph-mon[88643]: pgmap v257: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:13.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:13 smithi132 ceph-mon[70676]: pgmap v258: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:13.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:12:13.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:13 smithi088 ceph-mon[88643]: pgmap v258: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:13.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:12:15.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:15 smithi132 ceph-mon[70676]: pgmap v259: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:15.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:15 smithi088 ceph-mon[88643]: pgmap v259: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:17.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:17 smithi132 ceph-mon[70676]: pgmap v260: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:17.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:17 smithi088 ceph-mon[88643]: pgmap v260: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:18.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.050+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/3484408521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb4100e80 msgr2=0x7f2fb4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.050+0000 7f2fb9e7a700 1 --2- 172.21.15.88:0/3484408521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb4100e80 0x7f2fb4104e50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009910 tx=0x7f2f9c009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:18.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.050+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/3484408521 shutdown_connections 2024-02-19T19:12:18.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.050+0000 7f2fb9e7a700 1 --2- 172.21.15.88:0/3484408521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb4100e80 0x7f2fb4104e50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.050+0000 7f2fb9e7a700 1 --2- 172.21.15.88:0/3484408521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb41004e0 0x7f2fb41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.050+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/3484408521 >> 172.21.15.88:0/3484408521 conn(0x7f2fb40fbda0 msgr2=0x7f2fb40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:18.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.050+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/3484408521 shutdown_connections 2024-02-19T19:12:18.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.050+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/3484408521 wait complete. 2024-02-19T19:12:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb9e7a700 1 Processor -- start 2024-02-19T19:12:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb9e7a700 1 -- start start 2024-02-19T19:12:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb9e7a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb41004e0 0x7f2fb4195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb9e7a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb4100e80 0x7f2fb4196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb9e7a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb4196b50 con 0x7f2fb41004e0 2024-02-19T19:12:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb9e7a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb419a8e0 con 0x7f2fb4100e80 2024-02-19T19:12:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb41004e0 0x7f2fb4195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb41004e0 0x7f2fb4195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35456/0 (socket says 172.21.15.88:35456) 2024-02-19T19:12:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb37fe700 1 -- 172.21.15.88:0/1889972334 learned_addr learned my addr 172.21.15.88:0/1889972334 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb37fe700 1 -- 172.21.15.88:0/1889972334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb4100e80 msgr2=0x7f2fb4196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2faaffd700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb4100e80 0x7f2fb4196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:18.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb37fe700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb4100e80 0x7f2fb4196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2fb37fe700 1 -- 172.21.15.88:0/1889972334 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f9c0095f0 con 0x7f2fb41004e0 2024-02-19T19:12:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.051+0000 7f2faaffd700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb4100e80 0x7f2fb4196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:12:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.052+0000 7f2fb37fe700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb41004e0 0x7f2fb4195f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400ada0 tx=0x7f2fa400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.052+0000 7f2fb17fa700 1 -- 172.21.15.88:0/1889972334 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fa400f070 con 0x7f2fb41004e0 2024-02-19T19:12:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.052+0000 7f2fb17fa700 1 -- 172.21.15.88:0/1889972334 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fa4007bf0 con 0x7f2fb41004e0 2024-02-19T19:12:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.052+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/1889972334 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb419abc0 con 0x7f2fb41004e0 2024-02-19T19:12:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.052+0000 7f2fb17fa700 1 -- 172.21.15.88:0/1889972334 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fa4005690 con 0x7f2fb41004e0 2024-02-19T19:12:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.052+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/1889972334 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb419b110 con 0x7f2fb41004e0 2024-02-19T19:12:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.053+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/1889972334 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb4108850 con 0x7f2fb41004e0 2024-02-19T19:12:18.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.053+0000 7f2fb17fa700 1 -- 172.21.15.88:0/1889972334 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2fa4005900 con 0x7f2fb41004e0 2024-02-19T19:12:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.054+0000 7f2fb17fa700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fa006d8a0 0x7f2fa006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.054+0000 7f2fb17fa700 1 -- 172.21.15.88:0/1889972334 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2fa408cfa0 con 0x7f2fb41004e0 2024-02-19T19:12:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.054+0000 7f2faaffd700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fa006d8a0 0x7f2fa006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:18.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.054+0000 7f2faaffd700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fa006d8a0 0x7f2fa006fd50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00b5c0 tx=0x7f2f9c005480 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:18.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.059+0000 7f2fb17fa700 1 -- 172.21.15.88:0/1889972334 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2fa4056750 con 0x7f2fb41004e0 2024-02-19T19:12:18.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.273+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/1889972334 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2fb419b330 con 0x7f2fa006d8a0 2024-02-19T19:12:18.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.275+0000 7f2fb17fa700 1 -- 172.21.15.88:0/1889972334 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2fb419b330 con 0x7f2fa006d8a0 2024-02-19T19:12:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.277+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/1889972334 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fa006d8a0 msgr2=0x7f2fa006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.277+0000 7f2fb9e7a700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fa006d8a0 0x7f2fa006fd50 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00b5c0 tx=0x7f2f9c005480 comp rx=0 tx=0).stop 2024-02-19T19:12:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.277+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/1889972334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb41004e0 msgr2=0x7f2fb4195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.277+0000 7f2fb9e7a700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb41004e0 0x7f2fb4195f30 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400ada0 tx=0x7f2fa400c330 comp rx=0 tx=0).stop 2024-02-19T19:12:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.277+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/1889972334 shutdown_connections 2024-02-19T19:12:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.277+0000 7f2fb9e7a700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fa006d8a0 0x7f2fa006fd50 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.277+0000 7f2fb9e7a700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb41004e0 0x7f2fb4195f30 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.277+0000 7f2fb9e7a700 1 --2- 172.21.15.88:0/1889972334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fb4100e80 0x7f2fb4196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.277+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/1889972334 >> 172.21.15.88:0/1889972334 conn(0x7f2fb40fbda0 msgr2=0x7f2fb40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:18.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.277+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/1889972334 shutdown_connections 2024-02-19T19:12:18.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.277+0000 7f2fb9e7a700 1 -- 172.21.15.88:0/1889972334 wait complete. 2024-02-19T19:12:18.310 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:12:18.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.383+0000 7f5f31707700 1 -- 172.21.15.88:0/1033462845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f2c101420 msgr2=0x7f5f2c1017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.383+0000 7f5f31707700 1 --2- 172.21.15.88:0/1033462845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f2c101420 0x7f5f2c1017f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009910 tx=0x7f5f1c009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:18.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.383+0000 7f5f31707700 1 -- 172.21.15.88:0/1033462845 shutdown_connections 2024-02-19T19:12:18.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.383+0000 7f5f31707700 1 --2- 172.21.15.88:0/1033462845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f2c101d30 0x7f5f2c10a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.383+0000 7f5f31707700 1 --2- 172.21.15.88:0/1033462845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f2c101420 0x7f5f2c1017f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.383+0000 7f5f31707700 1 -- 172.21.15.88:0/1033462845 >> 172.21.15.88:0/1033462845 conn(0x7f5f2c0fac80 msgr2=0x7f5f2c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:18.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.383+0000 7f5f31707700 1 -- 172.21.15.88:0/1033462845 shutdown_connections 2024-02-19T19:12:18.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.383+0000 7f5f31707700 1 -- 172.21.15.88:0/1033462845 wait complete. 2024-02-19T19:12:18.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.383+0000 7f5f31707700 1 Processor -- start 2024-02-19T19:12:18.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f31707700 1 -- start start 2024-02-19T19:12:18.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f31707700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f2c101420 0x7f5f2c195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:18.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f31707700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f2c101d30 0x7f5f2c196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:18.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f31707700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c196c10 con 0x7f5f2c101420 2024-02-19T19:12:18.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f31707700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c19a9a0 con 0x7f5f2c101d30 2024-02-19T19:12:18.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f2affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f2c101420 0x7f5f2c195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:18.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f2affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f2c101420 0x7f5f2c195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35484/0 (socket says 172.21.15.88:35484) 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f2affd700 1 -- 172.21.15.88:0/846851968 learned_addr learned my addr 172.21.15.88:0/846851968 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f2affd700 1 -- 172.21.15.88:0/846851968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f2c101d30 msgr2=0x7f5f2c196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f227fc700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f2c101d30 0x7f5f2c196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f2affd700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f2c101d30 0x7f5f2c196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f2affd700 1 -- 172.21.15.88:0/846851968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c0095f0 con 0x7f5f2c101420 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f227fc700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f2c101d30 0x7f5f2c196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f2affd700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f2c101420 0x7f5f2c195ff0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c006010 tx=0x7f5f1c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.384+0000 7f5f28ff9700 1 -- 172.21.15.88:0/846851968 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f1c01d070 con 0x7f5f2c101420 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.385+0000 7f5f31707700 1 -- 172.21.15.88:0/846851968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c19ac20 con 0x7f5f2c101420 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.385+0000 7f5f28ff9700 1 -- 172.21.15.88:0/846851968 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f1c004fb0 con 0x7f5f2c101420 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.385+0000 7f5f28ff9700 1 -- 172.21.15.88:0/846851968 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f1c022ea0 con 0x7f5f2c101420 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.385+0000 7f5f31707700 1 -- 172.21.15.88:0/846851968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c19b110 con 0x7f5f2c101420 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.385+0000 7f5f31707700 1 -- 172.21.15.88:0/846851968 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c1078e0 con 0x7f5f2c101420 2024-02-19T19:12:18.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.386+0000 7f5f28ff9700 1 -- 172.21.15.88:0/846851968 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5f1c022470 con 0x7f5f2c101420 2024-02-19T19:12:18.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.386+0000 7f5f28ff9700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f1806d8a0 0x7f5f1806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:18.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.387+0000 7f5f28ff9700 1 -- 172.21.15.88:0/846851968 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5f1c08ebb0 con 0x7f5f2c101420 2024-02-19T19:12:18.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.387+0000 7f5f227fc700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f1806d8a0 0x7f5f1806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:18.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.387+0000 7f5f227fc700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f1806d8a0 0x7f5f1806fd50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c197610 tx=0x7f5f14008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:18.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.391+0000 7f5f28ff9700 1 -- 172.21.15.88:0/846851968 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f1c095080 con 0x7f5f2c101420 2024-02-19T19:12:18.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.607+0000 7f5f31707700 1 -- 172.21.15.88:0/846851968 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f2c1973f0 con 0x7f5f1806d8a0 2024-02-19T19:12:18.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.608+0000 7f5f28ff9700 1 -- 172.21.15.88:0/846851968 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5f2c1973f0 con 0x7f5f1806d8a0 2024-02-19T19:12:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.610+0000 7f5f31707700 1 -- 172.21.15.88:0/846851968 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f1806d8a0 msgr2=0x7f5f1806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.610+0000 7f5f31707700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f1806d8a0 0x7f5f1806fd50 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c197610 tx=0x7f5f14008040 comp rx=0 tx=0).stop 2024-02-19T19:12:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.610+0000 7f5f31707700 1 -- 172.21.15.88:0/846851968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f2c101420 msgr2=0x7f5f2c195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.610+0000 7f5f31707700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f2c101420 0x7f5f2c195ff0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c006010 tx=0x7f5f1c00be20 comp rx=0 tx=0).stop 2024-02-19T19:12:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.610+0000 7f5f31707700 1 -- 172.21.15.88:0/846851968 shutdown_connections 2024-02-19T19:12:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.610+0000 7f5f31707700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f1806d8a0 0x7f5f1806fd50 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.610+0000 7f5f31707700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f2c101420 0x7f5f2c195ff0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.610+0000 7f5f31707700 1 --2- 172.21.15.88:0/846851968 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f2c101d30 0x7f5f2c196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.610+0000 7f5f31707700 1 -- 172.21.15.88:0/846851968 >> 172.21.15.88:0/846851968 conn(0x7f5f2c0fac80 msgr2=0x7f5f2c104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.610+0000 7f5f31707700 1 -- 172.21.15.88:0/846851968 shutdown_connections 2024-02-19T19:12:18.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.610+0000 7f5f31707700 1 -- 172.21.15.88:0/846851968 wait complete. 2024-02-19T19:12:18.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.716+0000 7f66309ae700 1 -- 172.21.15.88:0/768023500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f662c100e80 msgr2=0x7f662c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.716+0000 7f66309ae700 1 --2- 172.21.15.88:0/768023500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f662c100e80 0x7f662c104e50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f661c009910 tx=0x7f661c009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:18.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.716+0000 7f66309ae700 1 -- 172.21.15.88:0/768023500 shutdown_connections 2024-02-19T19:12:18.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.716+0000 7f66309ae700 1 --2- 172.21.15.88:0/768023500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f662c100e80 0x7f662c104e50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.716+0000 7f66309ae700 1 --2- 172.21.15.88:0/768023500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f662c1004e0 0x7f662c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.716+0000 7f66309ae700 1 -- 172.21.15.88:0/768023500 >> 172.21.15.88:0/768023500 conn(0x7f662c0fbda0 msgr2=0x7f662c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:18.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.716+0000 7f66309ae700 1 -- 172.21.15.88:0/768023500 shutdown_connections 2024-02-19T19:12:18.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.716+0000 7f66309ae700 1 -- 172.21.15.88:0/768023500 wait complete. 2024-02-19T19:12:18.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.717+0000 7f66309ae700 1 Processor -- start 2024-02-19T19:12:18.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.717+0000 7f66309ae700 1 -- start start 2024-02-19T19:12:18.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.717+0000 7f66309ae700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f662c1004e0 0x7f662c195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:18.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.717+0000 7f66309ae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f662c100e80 0x7f662c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:18.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.717+0000 7f66309ae700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c196b30 con 0x7f662c1004e0 2024-02-19T19:12:18.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.717+0000 7f66309ae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f662c19a8c0 con 0x7f662c100e80 2024-02-19T19:12:18.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.717+0000 7f662a59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f662c1004e0 0x7f662c195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:18.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.717+0000 7f662a59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f662c1004e0 0x7f662c195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35508/0 (socket says 172.21.15.88:35508) 2024-02-19T19:12:18.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.717+0000 7f662a59c700 1 -- 172.21.15.88:0/1368430632 learned_addr learned my addr 172.21.15.88:0/1368430632 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:18.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.718+0000 7f6629d9b700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f662c100e80 0x7f662c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:18.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.718+0000 7f662a59c700 1 -- 172.21.15.88:0/1368430632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f662c100e80 msgr2=0x7f662c196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.718+0000 7f662a59c700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f662c100e80 0x7f662c196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.718+0000 7f662a59c700 1 -- 172.21.15.88:0/1368430632 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f661c0095f0 con 0x7f662c1004e0 2024-02-19T19:12:18.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.718+0000 7f6629d9b700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f662c100e80 0x7f662c196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:12:18.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.718+0000 7f662a59c700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f662c1004e0 0x7f662c195f10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f662000d8d0 tx=0x7f662000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:18.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.718+0000 7f661b7fe700 1 -- 172.21.15.88:0/1368430632 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6620009940 con 0x7f662c1004e0 2024-02-19T19:12:18.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.718+0000 7f66309ae700 1 -- 172.21.15.88:0/1368430632 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f662c19aba0 con 0x7f662c1004e0 2024-02-19T19:12:18.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.718+0000 7f661b7fe700 1 -- 172.21.15.88:0/1368430632 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6620010460 con 0x7f662c1004e0 2024-02-19T19:12:18.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.718+0000 7f66309ae700 1 -- 172.21.15.88:0/1368430632 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662c19b0f0 con 0x7f662c1004e0 2024-02-19T19:12:18.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.718+0000 7f661b7fe700 1 -- 172.21.15.88:0/1368430632 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f662000f5f0 con 0x7f662c1004e0 2024-02-19T19:12:18.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.719+0000 7f66309ae700 1 -- 172.21.15.88:0/1368430632 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660c005320 con 0x7f662c1004e0 2024-02-19T19:12:18.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.720+0000 7f661b7fe700 1 -- 172.21.15.88:0/1368430632 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6620010ad0 con 0x7f662c1004e0 2024-02-19T19:12:18.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.720+0000 7f661b7fe700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f661406d8a0 0x7f661406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:18.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.721+0000 7f661b7fe700 1 -- 172.21.15.88:0/1368430632 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f662008d480 con 0x7f662c1004e0 2024-02-19T19:12:18.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.721+0000 7f6629d9b700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f661406d8a0 0x7f661406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:18.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.721+0000 7f6629d9b700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f661406d8a0 0x7f661406fd50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f662c197530 tx=0x7f661c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:18.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.725+0000 7f661b7fe700 1 -- 172.21.15.88:0/1368430632 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6620056c30 con 0x7f662c1004e0 2024-02-19T19:12:18.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.932+0000 7f66309ae700 1 -- 172.21.15.88:0/1368430632 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f660c000bf0 con 0x7f661406d8a0 2024-02-19T19:12:18.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.942+0000 7f661b7fe700 1 -- 172.21.15.88:0/1368430632 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f660c000bf0 con 0x7f661406d8a0 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (9m) 79s ago 19m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (20m) 79s ago 20m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (2m) 62s ago 18m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (5m) 79s ago 20m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (5m) 62s ago 18m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (8m) 79s ago 19m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (104s) 79s ago 13m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (85s) 79s ago 13m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (65s) 62s ago 13m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (13m) 62s ago 13m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (10m) 79s ago 21m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (10m) 62s ago 18m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (7m) 79s ago 21m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:12:18.965 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (6m) 62s ago 18m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:12:18.966 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (10m) 79s ago 20m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:12:18.966 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (10m) 62s ago 18m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:12:18.966 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (4m) 79s ago 17m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:12:18.966 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (4m) 79s ago 16m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:12:18.966 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (3m) 79s ago 16m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:12:18.966 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (3m) 62s ago 16m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:12:18.966 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (2m) 62s ago 15m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:12:18.966 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (2m) 62s ago 15m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:12:18.966 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (9m) 79s ago 18m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:12:18.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.944+0000 7f66309ae700 1 -- 172.21.15.88:0/1368430632 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f661406d8a0 msgr2=0x7f661406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.944+0000 7f66309ae700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f661406d8a0 0x7f661406fd50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f662c197530 tx=0x7f661c01a040 comp rx=0 tx=0).stop 2024-02-19T19:12:18.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.944+0000 7f66309ae700 1 -- 172.21.15.88:0/1368430632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f662c1004e0 msgr2=0x7f662c195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:18.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.944+0000 7f66309ae700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f662c1004e0 0x7f662c195f10 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f662000d8d0 tx=0x7f662000dc90 comp rx=0 tx=0).stop 2024-02-19T19:12:18.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.944+0000 7f66309ae700 1 -- 172.21.15.88:0/1368430632 shutdown_connections 2024-02-19T19:12:18.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.944+0000 7f66309ae700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f661406d8a0 0x7f661406fd50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.944+0000 7f66309ae700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f662c1004e0 0x7f662c195f10 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.944+0000 7f66309ae700 1 --2- 172.21.15.88:0/1368430632 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f662c100e80 0x7f662c196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:18.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.944+0000 7f66309ae700 1 -- 172.21.15.88:0/1368430632 >> 172.21.15.88:0/1368430632 conn(0x7f662c0fbda0 msgr2=0x7f662c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:18.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.945+0000 7f66309ae700 1 -- 172.21.15.88:0/1368430632 shutdown_connections 2024-02-19T19:12:18.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:18.945+0000 7f66309ae700 1 -- 172.21.15.88:0/1368430632 wait complete. 2024-02-19T19:12:19.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.051+0000 7f04d41c4700 1 -- 172.21.15.88:0/621331025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04cc106320 msgr2=0x7f04cc1066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:19.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.051+0000 7f04d41c4700 1 --2- 172.21.15.88:0/621331025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04cc106320 0x7f04cc1066f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f04bc009910 tx=0x7f04bc009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:19.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.051+0000 7f04d41c4700 1 -- 172.21.15.88:0/621331025 shutdown_connections 2024-02-19T19:12:19.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.051+0000 7f04d41c4700 1 --2- 172.21.15.88:0/621331025 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04cc068540 0x7f04cc0689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.051+0000 7f04d41c4700 1 --2- 172.21.15.88:0/621331025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04cc106320 0x7f04cc1066f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.051+0000 7f04d41c4700 1 -- 172.21.15.88:0/621331025 >> 172.21.15.88:0/621331025 conn(0x7f04cc075130 msgr2=0x7f04cc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:19.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.051+0000 7f04d41c4700 1 -- 172.21.15.88:0/621331025 shutdown_connections 2024-02-19T19:12:19.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.051+0000 7f04d41c4700 1 -- 172.21.15.88:0/621331025 wait complete. 2024-02-19T19:12:19.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.051+0000 7f04d41c4700 1 Processor -- start 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d41c4700 1 -- start start 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d41c4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04cc068540 0x7f04cc195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d41c4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04cc106320 0x7f04cc1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d41c4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc196a00 con 0x7f04cc106320 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d41c4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04cc196b40 con 0x7f04cc068540 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d175f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04cc106320 0x7f04cc1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d175f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04cc106320 0x7f04cc1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35528/0 (socket says 172.21.15.88:35528) 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d175f700 1 -- 172.21.15.88:0/3295200597 learned_addr learned my addr 172.21.15.88:0/3295200597 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d1f60700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04cc068540 0x7f04cc195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d175f700 1 -- 172.21.15.88:0/3295200597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04cc068540 msgr2=0x7f04cc195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d175f700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04cc068540 0x7f04cc195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d175f700 1 -- 172.21.15.88:0/3295200597 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc0095f0 con 0x7f04cc106320 2024-02-19T19:12:19.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d1f60700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04cc068540 0x7f04cc195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:12:19.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d175f700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04cc106320 0x7f04cc1963b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f04c800c930 tx=0x7f04c800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:19.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04c2ffd700 1 -- 172.21.15.88:0/3295200597 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04c8007ad0 con 0x7f04cc106320 2024-02-19T19:12:19.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04c2ffd700 1 -- 172.21.15.88:0/3295200597 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04c8007c30 con 0x7f04cc106320 2024-02-19T19:12:19.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d41c4700 1 -- 172.21.15.88:0/3295200597 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04cc19a990 con 0x7f04cc106320 2024-02-19T19:12:19.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04c2ffd700 1 -- 172.21.15.88:0/3295200597 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04c8018610 con 0x7f04cc106320 2024-02-19T19:12:19.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.052+0000 7f04d41c4700 1 -- 172.21.15.88:0/3295200597 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04cc19aee0 con 0x7f04cc106320 2024-02-19T19:12:19.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.053+0000 7f04d41c4700 1 -- 172.21.15.88:0/3295200597 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04cc108770 con 0x7f04cc106320 2024-02-19T19:12:19.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.054+0000 7f04c2ffd700 1 -- 172.21.15.88:0/3295200597 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f04c800f450 con 0x7f04cc106320 2024-02-19T19:12:19.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.054+0000 7f04c2ffd700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f04b806d8a0 0x7f04b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:19.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.055+0000 7f04d1f60700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f04b806d8a0 0x7f04b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:19.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.055+0000 7f04c2ffd700 1 -- 172.21.15.88:0/3295200597 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f04c808d6a0 con 0x7f04cc106320 2024-02-19T19:12:19.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.055+0000 7f04d1f60700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f04b806d8a0 0x7f04b806fd50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00b5c0 tx=0x7f04bc005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:19.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.059+0000 7f04c2ffd700 1 -- 172.21.15.88:0/3295200597 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f04c8056ed0 con 0x7f04cc106320 2024-02-19T19:12:19.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.340+0000 7f04d41c4700 1 -- 172.21.15.88:0/3295200597 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f04cc04f2a0 con 0x7f04cc106320 2024-02-19T19:12:19.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.341+0000 7f04c2ffd700 1 -- 172.21.15.88:0/3295200597 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f04c805a590 con 0x7f04cc106320 2024-02-19T19:12:19.363 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:12:19.363 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:12:19.363 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:12:19.363 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:12:19.363 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:12:19.364 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:12:19.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.343+0000 7f04d41c4700 1 -- 172.21.15.88:0/3295200597 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f04b806d8a0 msgr2=0x7f04b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.343+0000 7f04d41c4700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f04b806d8a0 0x7f04b806fd50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00b5c0 tx=0x7f04bc005960 comp rx=0 tx=0).stop 2024-02-19T19:12:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.343+0000 7f04d41c4700 1 -- 172.21.15.88:0/3295200597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04cc106320 msgr2=0x7f04cc1963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.343+0000 7f04d41c4700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04cc106320 0x7f04cc1963b0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f04c800c930 tx=0x7f04c800ccf0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.343+0000 7f04d41c4700 1 -- 172.21.15.88:0/3295200597 shutdown_connections 2024-02-19T19:12:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.343+0000 7f04d41c4700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f04b806d8a0 0x7f04b806fd50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.343+0000 7f04d41c4700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f04cc068540 0x7f04cc195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.343+0000 7f04d41c4700 1 --2- 172.21.15.88:0/3295200597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04cc106320 0x7f04cc1963b0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.343+0000 7f04d41c4700 1 -- 172.21.15.88:0/3295200597 >> 172.21.15.88:0/3295200597 conn(0x7f04cc075130 msgr2=0x7f04cc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.343+0000 7f04d41c4700 1 -- 172.21.15.88:0/3295200597 shutdown_connections 2024-02-19T19:12:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.343+0000 7f04d41c4700 1 -- 172.21.15.88:0/3295200597 wait complete. 2024-02-19T19:12:19.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.445+0000 7fc7a9efb700 1 -- 172.21.15.88:0/222623649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4101d30 msgr2=0x7fc7a410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:19.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.445+0000 7fc7a9efb700 1 --2- 172.21.15.88:0/222623649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4101d30 0x7fc7a410a230 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc798009910 tx=0x7fc798009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:19.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.446+0000 7fc7a9efb700 1 -- 172.21.15.88:0/222623649 shutdown_connections 2024-02-19T19:12:19.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.446+0000 7fc7a9efb700 1 --2- 172.21.15.88:0/222623649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4101d30 0x7fc7a410a230 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.446+0000 7fc7a9efb700 1 --2- 172.21.15.88:0/222623649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a4101420 0x7fc7a41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.446+0000 7fc7a9efb700 1 -- 172.21.15.88:0/222623649 >> 172.21.15.88:0/222623649 conn(0x7fc7a40fac80 msgr2=0x7fc7a40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:19.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.446+0000 7fc7a9efb700 1 -- 172.21.15.88:0/222623649 shutdown_connections 2024-02-19T19:12:19.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.446+0000 7fc7a9efb700 1 -- 172.21.15.88:0/222623649 wait complete. 2024-02-19T19:12:19.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.446+0000 7fc7a9efb700 1 Processor -- start 2024-02-19T19:12:19.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.446+0000 7fc7a9efb700 1 -- start start 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a9efb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a4101420 0x7fc7a40ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a9efb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4101d30 0x7fc7a40ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a9efb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40ffb00 con 0x7fc7a4101d30 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a9efb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40ffc40 con 0x7fc7a4101420 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4101d30 0x7fc7a40ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4101d30 0x7fc7a40ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35548/0 (socket says 172.21.15.88:35548) 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a2ffd700 1 -- 172.21.15.88:0/1019680574 learned_addr learned my addr 172.21.15.88:0/1019680574 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a37fe700 1 --2- 172.21.15.88:0/1019680574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a4101420 0x7fc7a40ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a2ffd700 1 -- 172.21.15.88:0/1019680574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a4101420 msgr2=0x7fc7a40ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a2ffd700 1 --2- 172.21.15.88:0/1019680574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a4101420 0x7fc7a40ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a2ffd700 1 -- 172.21.15.88:0/1019680574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980095f0 con 0x7fc7a4101d30 2024-02-19T19:12:19.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a2ffd700 1 --2- 172.21.15.88:0/1019680574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4101d30 0x7fc7a40ff5c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc798006010 tx=0x7fc79800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:19.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a0ff9700 1 -- 172.21.15.88:0/1019680574 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc79801d070 con 0x7fc7a4101d30 2024-02-19T19:12:19.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a0ff9700 1 -- 172.21.15.88:0/1019680574 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc79800f460 con 0x7fc7a4101d30 2024-02-19T19:12:19.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a9efb700 1 -- 172.21.15.88:0/1019680574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a40ffec0 con 0x7fc7a4101d30 2024-02-19T19:12:19.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.447+0000 7fc7a0ff9700 1 -- 172.21.15.88:0/1019680574 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc798017610 con 0x7fc7a4101d30 2024-02-19T19:12:19.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.448+0000 7fc7a9efb700 1 -- 172.21.15.88:0/1019680574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a41a24f0 con 0x7fc7a4101d30 2024-02-19T19:12:19.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.448+0000 7fc7a9efb700 1 -- 172.21.15.88:0/1019680574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a41078a0 con 0x7fc7a4101d30 2024-02-19T19:12:19.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.449+0000 7fc7a0ff9700 1 -- 172.21.15.88:0/1019680574 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc79800f5d0 con 0x7fc7a4101d30 2024-02-19T19:12:19.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.449+0000 7fc7a0ff9700 1 --2- 172.21.15.88:0/1019680574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc78c06d8a0 0x7fc78c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:19.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.449+0000 7fc7a0ff9700 1 -- 172.21.15.88:0/1019680574 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc79808e9f0 con 0x7fc7a4101d30 2024-02-19T19:12:19.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.449+0000 7fc7a37fe700 1 --2- 172.21.15.88:0/1019680574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc78c06d8a0 0x7fc78c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:19.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.450+0000 7fc7a37fe700 1 --2- 172.21.15.88:0/1019680574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc78c06d8a0 0x7fc78c06fd50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc794009a10 tx=0x7fc794009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:19.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.454+0000 7fc7a0ff9700 1 -- 172.21.15.88:0/1019680574 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc798090e60 con 0x7fc7a4101d30 2024-02-19T19:12:19.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.692+0000 7fc7a9efb700 1 -- 172.21.15.88:0/1019680574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc7a4100650 con 0x7fc7a4101d30 2024-02-19T19:12:19.715 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:19 smithi088 ceph-mon[88643]: pgmap v261: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:19.715 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:19 smithi088 ceph-mon[88643]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:19.715 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:19 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3295200597' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:12:19.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.693+0000 7fc7a0ff9700 1 -- 172.21.15.88:0/1019680574 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fc798058220 con 0x7fc7a4101d30 2024-02-19T19:12:19.716 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:12:19.716 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:12:19.716 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:12:19.716 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:12:19.716 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:12:19.716 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:12:19.716 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:12:19.716 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:12:19.717 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:12:19.717 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:12:19.717 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:12:19.717 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:12:19.717 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:12:19.717 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:12:19.717 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:12:19.717 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:12:19.717 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:12:19.717 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:12:19.718 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:12:19.719 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:12:19.719 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:12:19.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:12:19.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:12:19.719 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:12:19.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:12:19.719 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:12:19.719 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:12:19.719 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:12:19.719 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.695+0000 7fc7a9efb700 1 -- 172.21.15.88:0/1019680574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc78c06d8a0 msgr2=0x7fc78c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.695+0000 7fc7a9efb700 1 --2- 172.21.15.88:0/1019680574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc78c06d8a0 0x7fc78c06fd50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc794009a10 tx=0x7fc794009190 comp rx=0 tx=0).stop 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.695+0000 7fc7a9efb700 1 -- 172.21.15.88:0/1019680574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4101d30 msgr2=0x7fc7a40ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.695+0000 7fc7a9efb700 1 --2- 172.21.15.88:0/1019680574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4101d30 0x7fc7a40ff5c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc798006010 tx=0x7fc79800bd70 comp rx=0 tx=0).stop 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.695+0000 7fc7a9efb700 1 -- 172.21.15.88:0/1019680574 shutdown_connections 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.695+0000 7fc7a9efb700 1 --2- 172.21.15.88:0/1019680574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc78c06d8a0 0x7fc78c06fd50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.695+0000 7fc7a9efb700 1 --2- 172.21.15.88:0/1019680574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a4101420 0x7fc7a40ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.695+0000 7fc7a9efb700 1 --2- 172.21.15.88:0/1019680574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4101d30 0x7fc7a40ff5c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.695+0000 7fc7a9efb700 1 -- 172.21.15.88:0/1019680574 >> 172.21.15.88:0/1019680574 conn(0x7fc7a40fac80 msgr2=0x7fc7a41049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.695+0000 7fc7a9efb700 1 -- 172.21.15.88:0/1019680574 shutdown_connections 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.695+0000 7fc7a9efb700 1 -- 172.21.15.88:0/1019680574 wait complete. 2024-02-19T19:12:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:12:19.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:19 smithi132 ceph-mon[70676]: pgmap v261: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:19.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:19 smithi132 ceph-mon[70676]: from='client.34526 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:19.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:19 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3295200597' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:12:19.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.800+0000 7ffad89c5700 1 -- 172.21.15.88:0/286633172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40ffd60 msgr2=0x7ffad4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:19.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.800+0000 7ffad89c5700 1 --2- 172.21.15.88:0/286633172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40ffd60 0x7ffad4103d30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ffac4009910 tx=0x7ffac4009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:19.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.800+0000 7ffad89c5700 1 -- 172.21.15.88:0/286633172 shutdown_connections 2024-02-19T19:12:19.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.800+0000 7ffad89c5700 1 --2- 172.21.15.88:0/286633172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40ffd60 0x7ffad4103d30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.800+0000 7ffad89c5700 1 --2- 172.21.15.88:0/286633172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad40ff3c0 0x7ffad40ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.800+0000 7ffad89c5700 1 -- 172.21.15.88:0/286633172 >> 172.21.15.88:0/286633172 conn(0x7ffad40fac80 msgr2=0x7ffad40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:19.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.800+0000 7ffad89c5700 1 -- 172.21.15.88:0/286633172 shutdown_connections 2024-02-19T19:12:19.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.800+0000 7ffad89c5700 1 -- 172.21.15.88:0/286633172 wait complete. 2024-02-19T19:12:19.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.801+0000 7ffad89c5700 1 Processor -- start 2024-02-19T19:12:19.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.801+0000 7ffad89c5700 1 -- start start 2024-02-19T19:12:19.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.801+0000 7ffad89c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad40ff3c0 0x7ffad4194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:19.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.801+0000 7ffad89c5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40ffd60 0x7ffad4195420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:19.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.801+0000 7ffad89c5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4195b00 con 0x7ffad40ffd60 2024-02-19T19:12:19.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.801+0000 7ffad89c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4199890 con 0x7ffad40ff3c0 2024-02-19T19:12:19.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.801+0000 7ffad1d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40ffd60 0x7ffad4195420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:19.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.801+0000 7ffad1d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40ffd60 0x7ffad4195420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35568/0 (socket says 172.21.15.88:35568) 2024-02-19T19:12:19.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.801+0000 7ffad1d9b700 1 -- 172.21.15.88:0/4180645029 learned_addr learned my addr 172.21.15.88:0/4180645029 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:19.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.802+0000 7ffad259c700 1 --2- 172.21.15.88:0/4180645029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad40ff3c0 0x7ffad4194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:19.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.802+0000 7ffad1d9b700 1 -- 172.21.15.88:0/4180645029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad40ff3c0 msgr2=0x7ffad4194ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:19.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.802+0000 7ffad1d9b700 1 --2- 172.21.15.88:0/4180645029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad40ff3c0 0x7ffad4194ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:19.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.802+0000 7ffad1d9b700 1 -- 172.21.15.88:0/4180645029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac40095f0 con 0x7ffad40ffd60 2024-02-19T19:12:19.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.802+0000 7ffad1d9b700 1 --2- 172.21.15.88:0/4180645029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40ffd60 0x7ffad4195420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffac40092e0 tx=0x7ffac400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:19.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.802+0000 7ffac37fe700 1 -- 172.21.15.88:0/4180645029 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac401d070 con 0x7ffad40ffd60 2024-02-19T19:12:19.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.802+0000 7ffac37fe700 1 -- 172.21.15.88:0/4180645029 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffac400f460 con 0x7ffad40ffd60 2024-02-19T19:12:19.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.802+0000 7ffad89c5700 1 -- 172.21.15.88:0/4180645029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad4199b10 con 0x7ffad40ffd60 2024-02-19T19:12:19.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.802+0000 7ffac37fe700 1 -- 172.21.15.88:0/4180645029 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac40175f0 con 0x7ffad40ffd60 2024-02-19T19:12:19.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.802+0000 7ffad89c5700 1 -- 172.21.15.88:0/4180645029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad419a000 con 0x7ffad40ffd60 2024-02-19T19:12:19.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.803+0000 7ffad89c5700 1 -- 172.21.15.88:0/4180645029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad404f2a0 con 0x7ffad40ffd60 2024-02-19T19:12:19.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.804+0000 7ffac37fe700 1 -- 172.21.15.88:0/4180645029 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ffac400f5d0 con 0x7ffad40ffd60 2024-02-19T19:12:19.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.804+0000 7ffac37fe700 1 --2- 172.21.15.88:0/4180645029 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffabc06d8a0 0x7ffabc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:19.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.804+0000 7ffac37fe700 1 -- 172.21.15.88:0/4180645029 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ffac408eaa0 con 0x7ffad40ffd60 2024-02-19T19:12:19.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.804+0000 7ffad259c700 1 --2- 172.21.15.88:0/4180645029 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffabc06d8a0 0x7ffabc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:19.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.805+0000 7ffad259c700 1 --2- 172.21.15.88:0/4180645029 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffabc06d8a0 0x7ffabc06fd50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffac8006fd0 tx=0x7ffac8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:19.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:19.809+0000 7ffac37fe700 1 -- 172.21.15.88:0/4180645029 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffac4058250 con 0x7ffad40ffd60 2024-02-19T19:12:20.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.025+0000 7ffad89c5700 1 -- 172.21.15.88:0/4180645029 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffad419a2a0 con 0x7ffabc06d8a0 2024-02-19T19:12:20.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.027+0000 7ffac37fe700 1 -- 172.21.15.88:0/4180645029 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ffad419a2a0 con 0x7ffabc06d8a0 2024-02-19T19:12:20.049 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:12:20.049 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:12:20.049 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:12:20.049 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:12:20.049 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:12:20.050 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:12:20.050 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:12:20.050 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:12:20.050 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:12:20.050 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:12:20.050 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:12:20.050 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:12:20.050 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:12:20.050 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:12:20.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.029+0000 7ffad89c5700 1 -- 172.21.15.88:0/4180645029 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffabc06d8a0 msgr2=0x7ffabc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:20.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.029+0000 7ffad89c5700 1 --2- 172.21.15.88:0/4180645029 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffabc06d8a0 0x7ffabc06fd50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ffac8006fd0 tx=0x7ffac8008040 comp rx=0 tx=0).stop 2024-02-19T19:12:20.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.029+0000 7ffad89c5700 1 -- 172.21.15.88:0/4180645029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40ffd60 msgr2=0x7ffad4195420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:20.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.029+0000 7ffad89c5700 1 --2- 172.21.15.88:0/4180645029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40ffd60 0x7ffad4195420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ffac40092e0 tx=0x7ffac400be20 comp rx=0 tx=0).stop 2024-02-19T19:12:20.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.029+0000 7ffad89c5700 1 -- 172.21.15.88:0/4180645029 shutdown_connections 2024-02-19T19:12:20.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.029+0000 7ffad89c5700 1 --2- 172.21.15.88:0/4180645029 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffabc06d8a0 0x7ffabc06fd50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:20.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.029+0000 7ffad89c5700 1 --2- 172.21.15.88:0/4180645029 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffad40ff3c0 0x7ffad4194ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:20.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.029+0000 7ffad89c5700 1 --2- 172.21.15.88:0/4180645029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffad40ffd60 0x7ffad4195420 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:20.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.029+0000 7ffad89c5700 1 -- 172.21.15.88:0/4180645029 >> 172.21.15.88:0/4180645029 conn(0x7ffad40fac80 msgr2=0x7ffad40fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:20.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.029+0000 7ffad89c5700 1 -- 172.21.15.88:0/4180645029 shutdown_connections 2024-02-19T19:12:20.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.029+0000 7ffad89c5700 1 -- 172.21.15.88:0/4180645029 wait complete. 2024-02-19T19:12:20.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.132+0000 7f94591cb700 1 -- 172.21.15.88:0/1109532151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9454100e80 msgr2=0x7f9454104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:20.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.132+0000 7f94591cb700 1 --2- 172.21.15.88:0/1109532151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9454100e80 0x7f9454104e50 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9448009910 tx=0x7f9448009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:20.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.132+0000 7f94591cb700 1 -- 172.21.15.88:0/1109532151 shutdown_connections 2024-02-19T19:12:20.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.132+0000 7f94591cb700 1 --2- 172.21.15.88:0/1109532151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9454100e80 0x7f9454104e50 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:20.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.132+0000 7f94591cb700 1 --2- 172.21.15.88:0/1109532151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94541004e0 0x7f94541008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:20.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.132+0000 7f94591cb700 1 -- 172.21.15.88:0/1109532151 >> 172.21.15.88:0/1109532151 conn(0x7f94540fbda0 msgr2=0x7f94540fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:20.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.132+0000 7f94591cb700 1 -- 172.21.15.88:0/1109532151 shutdown_connections 2024-02-19T19:12:20.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.132+0000 7f94591cb700 1 -- 172.21.15.88:0/1109532151 wait complete. 2024-02-19T19:12:20.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.133+0000 7f94591cb700 1 Processor -- start 2024-02-19T19:12:20.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.133+0000 7f94591cb700 1 -- start start 2024-02-19T19:12:20.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f94591cb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94541004e0 0x7f9454195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:20.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f94591cb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9454100e80 0x7f9454196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:20.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f94591cb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9454196b30 con 0x7f94541004e0 2024-02-19T19:12:20.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f94591cb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945419a8c0 con 0x7f9454100e80 2024-02-19T19:12:20.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f9452d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94541004e0 0x7f9454195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:20.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f9452d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94541004e0 0x7f9454195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35590/0 (socket says 172.21.15.88:35590) 2024-02-19T19:12:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f9452d9d700 1 -- 172.21.15.88:0/1854628614 learned_addr learned my addr 172.21.15.88:0/1854628614 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f945259c700 1 --2- 172.21.15.88:0/1854628614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9454100e80 0x7f9454196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f9452d9d700 1 -- 172.21.15.88:0/1854628614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9454100e80 msgr2=0x7f9454196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f9452d9d700 1 --2- 172.21.15.88:0/1854628614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9454100e80 0x7f9454196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f9452d9d700 1 -- 172.21.15.88:0/1854628614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94480095f0 con 0x7f94541004e0 2024-02-19T19:12:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f9452d9d700 1 --2- 172.21.15.88:0/1854628614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94541004e0 0x7f9454195f10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f944400d8d0 tx=0x7f944400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f943bfff700 1 -- 172.21.15.88:0/1854628614 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94440098a0 con 0x7f94541004e0 2024-02-19T19:12:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f94591cb700 1 -- 172.21.15.88:0/1854628614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945419aba0 con 0x7f94541004e0 2024-02-19T19:12:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f943bfff700 1 -- 172.21.15.88:0/1854628614 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9444010460 con 0x7f94541004e0 2024-02-19T19:12:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f943bfff700 1 -- 172.21.15.88:0/1854628614 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f944400f5f0 con 0x7f94541004e0 2024-02-19T19:12:20.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.134+0000 7f94591cb700 1 -- 172.21.15.88:0/1854628614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945419b0f0 con 0x7f94541004e0 2024-02-19T19:12:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.135+0000 7f94591cb700 1 -- 172.21.15.88:0/1854628614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9454108850 con 0x7f94541004e0 2024-02-19T19:12:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.136+0000 7f943bfff700 1 -- 172.21.15.88:0/1854628614 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9444009a00 con 0x7f94541004e0 2024-02-19T19:12:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.136+0000 7f943bfff700 1 --2- 172.21.15.88:0/1854628614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f943c06d8a0 0x7f943c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.136+0000 7f943bfff700 1 -- 172.21.15.88:0/1854628614 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f944408d310 con 0x7f94541004e0 2024-02-19T19:12:20.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.136+0000 7f945259c700 1 --2- 172.21.15.88:0/1854628614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f943c06d8a0 0x7f943c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:20.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.137+0000 7f945259c700 1 --2- 172.21.15.88:0/1854628614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f943c06d8a0 0x7f943c06fd50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9454197530 tx=0x7f944801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:20.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.141+0000 7f943bfff700 1 -- 172.21.15.88:0/1854628614 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9444056b40 con 0x7f94541004e0 2024-02-19T19:12:20.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.426+0000 7f94591cb700 1 -- 172.21.15.88:0/1854628614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f945404f2a0 con 0x7f94541004e0 2024-02-19T19:12:20.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.427+0000 7f943bfff700 1 -- 172.21.15.88:0/1854628614 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f944405a200 con 0x7f94541004e0 2024-02-19T19:12:20.449 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:12:20.449 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:12:20.449 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:12:20.449 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:12:20.449 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:12:20.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.429+0000 7f94591cb700 1 -- 172.21.15.88:0/1854628614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f943c06d8a0 msgr2=0x7f943c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:20.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.429+0000 7f94591cb700 1 --2- 172.21.15.88:0/1854628614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f943c06d8a0 0x7f943c06fd50 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9454197530 tx=0x7f944801a040 comp rx=0 tx=0).stop 2024-02-19T19:12:20.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.429+0000 7f94591cb700 1 -- 172.21.15.88:0/1854628614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94541004e0 msgr2=0x7f9454195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:20.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.429+0000 7f94591cb700 1 --2- 172.21.15.88:0/1854628614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94541004e0 0x7f9454195f10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f944400d8d0 tx=0x7f944400dbe0 comp rx=0 tx=0).stop 2024-02-19T19:12:20.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.429+0000 7f94591cb700 1 -- 172.21.15.88:0/1854628614 shutdown_connections 2024-02-19T19:12:20.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.429+0000 7f94591cb700 1 --2- 172.21.15.88:0/1854628614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f943c06d8a0 0x7f943c06fd50 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:20.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.429+0000 7f94591cb700 1 --2- 172.21.15.88:0/1854628614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94541004e0 0x7f9454195f10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:20.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.429+0000 7f94591cb700 1 --2- 172.21.15.88:0/1854628614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9454100e80 0x7f9454196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:20.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.429+0000 7f94591cb700 1 -- 172.21.15.88:0/1854628614 >> 172.21.15.88:0/1854628614 conn(0x7f94540fbda0 msgr2=0x7f94540fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:20.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.429+0000 7f94591cb700 1 -- 172.21.15.88:0/1854628614 shutdown_connections 2024-02-19T19:12:20.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:20.429+0000 7f94591cb700 1 -- 172.21.15.88:0/1854628614 wait complete. 2024-02-19T19:12:20.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:20 smithi132 ceph-mon[70676]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:20.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:20 smithi132 ceph-mon[70676]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:20.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:20 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1019680574' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:12:20.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:20 smithi088 ceph-mon[88643]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:20.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:20 smithi088 ceph-mon[88643]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:20.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:20 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1019680574' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:12:21.656 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:21 smithi132 ceph-mon[70676]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:21.656 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:21 smithi132 ceph-mon[70676]: pgmap v262: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:21.656 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:21 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1854628614' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:12:21.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:21 smithi088 ceph-mon[88643]: from='client.34546 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:21.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:21 smithi088 ceph-mon[88643]: pgmap v262: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:21.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:21 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1854628614' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:12:23.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:23 smithi132 ceph-mon[70676]: pgmap v263: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:23.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:23 smithi088 ceph-mon[88643]: pgmap v263: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:25.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:25 smithi132 ceph-mon[70676]: pgmap v264: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:25.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:25 smithi088 ceph-mon[88643]: pgmap v264: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:27 smithi132 ceph-mon[70676]: pgmap v265: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:27.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:27 smithi088 ceph-mon[88643]: pgmap v265: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:28.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:12:28.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:12:29.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:29 smithi132 ceph-mon[70676]: pgmap v266: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:29.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:29 smithi088 ceph-mon[88643]: pgmap v266: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:31.254 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:30 smithi132 ceph-mon[70676]: pgmap v267: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:31.352 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:30 smithi088 ceph-mon[88643]: pgmap v267: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:33.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:33 smithi088 ceph-mon[88643]: pgmap v268: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:33.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:33 smithi132 ceph-mon[70676]: pgmap v268: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:35.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:35 smithi088 ceph-mon[88643]: pgmap v269: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:35.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:35 smithi132 ceph-mon[70676]: pgmap v269: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:37.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:37 smithi088 ceph-mon[88643]: pgmap v270: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:37.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:37 smithi132 ceph-mon[70676]: pgmap v270: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:39.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:39 smithi088 ceph-mon[88643]: pgmap v271: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:39.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:39 smithi132 ceph-mon[70676]: pgmap v271: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:41.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:41 smithi088 ceph-mon[88643]: pgmap v272: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:41.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:41 smithi132 ceph-mon[70676]: pgmap v272: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:43.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:43 smithi088 ceph-mon[88643]: pgmap v273: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:43.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:12:43.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:43 smithi132 ceph-mon[70676]: pgmap v273: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:43.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:12:45.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:45 smithi088 ceph-mon[88643]: pgmap v274: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:45.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:45 smithi132 ceph-mon[70676]: pgmap v274: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:47.581 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:47 smithi132 ceph-mon[70676]: pgmap v275: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:47.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:47 smithi088 ceph-mon[88643]: pgmap v275: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:49.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:49 smithi088 ceph-mon[88643]: pgmap v276: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:49.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:49 smithi132 ceph-mon[70676]: pgmap v276: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:50.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.546+0000 7f60889f4700 1 -- 172.21.15.88:0/3341236311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60841004e0 msgr2=0x7f60841008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:50.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.546+0000 7f60889f4700 1 --2- 172.21.15.88:0/3341236311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60841004e0 0x7f60841008b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6074009910 tx=0x7f6074009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:50.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.546+0000 7f60889f4700 1 -- 172.21.15.88:0/3341236311 shutdown_connections 2024-02-19T19:12:50.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.546+0000 7f60889f4700 1 --2- 172.21.15.88:0/3341236311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6084100e80 0x7f6084104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:50.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.546+0000 7f60889f4700 1 --2- 172.21.15.88:0/3341236311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60841004e0 0x7f60841008b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:50.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.546+0000 7f60889f4700 1 -- 172.21.15.88:0/3341236311 >> 172.21.15.88:0/3341236311 conn(0x7f60840fbda0 msgr2=0x7f60840fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:50.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.546+0000 7f60889f4700 1 -- 172.21.15.88:0/3341236311 shutdown_connections 2024-02-19T19:12:50.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.546+0000 7f60889f4700 1 -- 172.21.15.88:0/3341236311 wait complete. 2024-02-19T19:12:50.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f60889f4700 1 Processor -- start 2024-02-19T19:12:50.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f60889f4700 1 -- start start 2024-02-19T19:12:50.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f60889f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60841004e0 0x7f6084195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:50.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f60889f4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6084100e80 0x7f60841964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:50.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f60889f4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6084196bb0 con 0x7f6084100e80 2024-02-19T19:12:50.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f60889f4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f608419a940 con 0x7f60841004e0 2024-02-19T19:12:50.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f6079d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6084100e80 0x7f60841964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:50.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f6079d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6084100e80 0x7f60841964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36716/0 (socket says 172.21.15.88:36716) 2024-02-19T19:12:50.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f6079d9b700 1 -- 172.21.15.88:0/2080523841 learned_addr learned my addr 172.21.15.88:0/2080523841 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:50.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f608259c700 1 --2- 172.21.15.88:0/2080523841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60841004e0 0x7f6084195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:50.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f6079d9b700 1 -- 172.21.15.88:0/2080523841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60841004e0 msgr2=0x7f6084195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:50.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.547+0000 7f6079d9b700 1 --2- 172.21.15.88:0/2080523841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60841004e0 0x7f6084195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:50.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.548+0000 7f6079d9b700 1 -- 172.21.15.88:0/2080523841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60740095f0 con 0x7f6084100e80 2024-02-19T19:12:50.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.548+0000 7f6079d9b700 1 --2- 172.21.15.88:0/2080523841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6084100e80 0x7f60841964d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f606c00b480 tx=0x7f606c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:50.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.548+0000 7f607bfff700 1 -- 172.21.15.88:0/2080523841 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f606c011800 con 0x7f6084100e80 2024-02-19T19:12:50.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.548+0000 7f60889f4700 1 -- 172.21.15.88:0/2080523841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f608419ac20 con 0x7f6084100e80 2024-02-19T19:12:50.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.548+0000 7f607bfff700 1 -- 172.21.15.88:0/2080523841 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f606c011e20 con 0x7f6084100e80 2024-02-19T19:12:50.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.548+0000 7f607bfff700 1 -- 172.21.15.88:0/2080523841 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f606c010ea0 con 0x7f6084100e80 2024-02-19T19:12:50.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.548+0000 7f60889f4700 1 -- 172.21.15.88:0/2080523841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f608419b170 con 0x7f6084100e80 2024-02-19T19:12:50.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.549+0000 7f60889f4700 1 -- 172.21.15.88:0/2080523841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60841087f0 con 0x7f6084100e80 2024-02-19T19:12:50.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.550+0000 7f607bfff700 1 -- 172.21.15.88:0/2080523841 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f606c010380 con 0x7f6084100e80 2024-02-19T19:12:50.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.550+0000 7f607bfff700 1 --2- 172.21.15.88:0/2080523841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f607006d8a0 0x7f607006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:50.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.550+0000 7f608259c700 1 --2- 172.21.15.88:0/2080523841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f607006d8a0 0x7f607006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:50.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.550+0000 7f607bfff700 1 -- 172.21.15.88:0/2080523841 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f606c08d000 con 0x7f6084100e80 2024-02-19T19:12:50.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.551+0000 7f608259c700 1 --2- 172.21.15.88:0/2080523841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f607006d8a0 0x7f607006fd50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60740059d0 tx=0x7f607401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:50.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.556+0000 7f607bfff700 1 -- 172.21.15.88:0/2080523841 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f606c0567b0 con 0x7f6084100e80 2024-02-19T19:12:50.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.778+0000 7f60889f4700 1 -- 172.21.15.88:0/2080523841 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f608419b460 con 0x7f607006d8a0 2024-02-19T19:12:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.779+0000 7f607bfff700 1 -- 172.21.15.88:0/2080523841 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f608419b460 con 0x7f607006d8a0 2024-02-19T19:12:50.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.781+0000 7f60889f4700 1 -- 172.21.15.88:0/2080523841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f607006d8a0 msgr2=0x7f607006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.781+0000 7f60889f4700 1 --2- 172.21.15.88:0/2080523841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f607006d8a0 0x7f607006fd50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f60740059d0 tx=0x7f607401a040 comp rx=0 tx=0).stop 2024-02-19T19:12:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.781+0000 7f60889f4700 1 -- 172.21.15.88:0/2080523841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6084100e80 msgr2=0x7f60841964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.781+0000 7f60889f4700 1 --2- 172.21.15.88:0/2080523841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6084100e80 0x7f60841964d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f606c00b480 tx=0x7f606c00b790 comp rx=0 tx=0).stop 2024-02-19T19:12:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.781+0000 7f60889f4700 1 -- 172.21.15.88:0/2080523841 shutdown_connections 2024-02-19T19:12:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.781+0000 7f60889f4700 1 --2- 172.21.15.88:0/2080523841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f607006d8a0 0x7f607006fd50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.781+0000 7f60889f4700 1 --2- 172.21.15.88:0/2080523841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f60841004e0 0x7f6084195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.781+0000 7f60889f4700 1 --2- 172.21.15.88:0/2080523841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6084100e80 0x7f60841964d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.781+0000 7f60889f4700 1 -- 172.21.15.88:0/2080523841 >> 172.21.15.88:0/2080523841 conn(0x7f60840fbda0 msgr2=0x7f60840fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.781+0000 7f60889f4700 1 -- 172.21.15.88:0/2080523841 shutdown_connections 2024-02-19T19:12:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.781+0000 7f60889f4700 1 -- 172.21.15.88:0/2080523841 wait complete. 2024-02-19T19:12:50.815 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:12:50.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.888+0000 7ff4297e5700 1 -- 172.21.15.88:0/3163647602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4240ffbd0 msgr2=0x7ff424100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:50.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.888+0000 7ff4297e5700 1 --2- 172.21.15.88:0/3163647602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4240ffbd0 0x7ff424100040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff40c009910 tx=0x7ff40c009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:50.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.888+0000 7ff4297e5700 1 -- 172.21.15.88:0/3163647602 shutdown_connections 2024-02-19T19:12:50.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.888+0000 7ff4297e5700 1 --2- 172.21.15.88:0/3163647602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4240ffbd0 0x7ff424100040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:50.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.888+0000 7ff4297e5700 1 --2- 172.21.15.88:0/3163647602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff424104130 0x7ff424104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:50.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.888+0000 7ff4297e5700 1 -- 172.21.15.88:0/3163647602 >> 172.21.15.88:0/3163647602 conn(0x7ff424075130 msgr2=0x7ff424075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:50.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.888+0000 7ff4297e5700 1 -- 172.21.15.88:0/3163647602 shutdown_connections 2024-02-19T19:12:50.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.888+0000 7ff4297e5700 1 -- 172.21.15.88:0/3163647602 wait complete. 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.888+0000 7ff4297e5700 1 Processor -- start 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff4297e5700 1 -- start start 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff4297e5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4240ffbd0 0x7ff424196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff4297e5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff424104130 0x7ff4241972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff4297e5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff424197900 con 0x7ff4240ffbd0 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff4297e5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff424197a70 con 0x7ff424104130 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff422ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4240ffbd0 0x7ff424196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff422ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4240ffbd0 0x7ff424196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36728/0 (socket says 172.21.15.88:36728) 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff422ffd700 1 -- 172.21.15.88:0/2859410977 learned_addr learned my addr 172.21.15.88:0/2859410977 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff4127fc700 1 --2- 172.21.15.88:0/2859410977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff424104130 0x7ff4241972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff422ffd700 1 -- 172.21.15.88:0/2859410977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff424104130 msgr2=0x7ff4241972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff422ffd700 1 --2- 172.21.15.88:0/2859410977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff424104130 0x7ff4241972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:50.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff422ffd700 1 -- 172.21.15.88:0/2859410977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff40c0095f0 con 0x7ff4240ffbd0 2024-02-19T19:12:50.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff422ffd700 1 --2- 172.21.15.88:0/2859410977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4240ffbd0 0x7ff424196da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff41400ba50 tx=0x7ff41400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:50.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.889+0000 7ff420ff9700 1 -- 172.21.15.88:0/2859410977 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff41400c900 con 0x7ff4240ffbd0 2024-02-19T19:12:50.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.890+0000 7ff420ff9700 1 -- 172.21.15.88:0/2859410977 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff414014440 con 0x7ff4240ffbd0 2024-02-19T19:12:50.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.890+0000 7ff420ff9700 1 -- 172.21.15.88:0/2859410977 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff4140075a0 con 0x7ff4240ffbd0 2024-02-19T19:12:50.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.890+0000 7ff4297e5700 1 -- 172.21.15.88:0/2859410977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42419fe10 con 0x7ff4240ffbd0 2024-02-19T19:12:50.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.890+0000 7ff4297e5700 1 -- 172.21.15.88:0/2859410977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4241a02d0 con 0x7ff4240ffbd0 2024-02-19T19:12:50.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.890+0000 7ff4297e5700 1 -- 172.21.15.88:0/2859410977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42404f2a0 con 0x7ff4240ffbd0 2024-02-19T19:12:50.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.891+0000 7ff420ff9700 1 -- 172.21.15.88:0/2859410977 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff4140145b0 con 0x7ff4240ffbd0 2024-02-19T19:12:50.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.891+0000 7ff420ff9700 1 --2- 172.21.15.88:0/2859410977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff40406d8a0 0x7ff40406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:50.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.892+0000 7ff4127fc700 1 --2- 172.21.15.88:0/2859410977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff40406d8a0 0x7ff40406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:50.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.892+0000 7ff420ff9700 1 -- 172.21.15.88:0/2859410977 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff414006e40 con 0x7ff4240ffbd0 2024-02-19T19:12:50.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.892+0000 7ff4127fc700 1 --2- 172.21.15.88:0/2859410977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff40406d8a0 0x7ff40406fd50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff424068e10 tx=0x7ff40c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:50.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:50.897+0000 7ff420ff9700 1 -- 172.21.15.88:0/2859410977 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4140563d0 con 0x7ff4240ffbd0 2024-02-19T19:12:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.116+0000 7ff4297e5700 1 -- 172.21.15.88:0/2859410977 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff4240ff3a0 con 0x7ff40406d8a0 2024-02-19T19:12:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.117+0000 7ff420ff9700 1 -- 172.21.15.88:0/2859410977 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff4240ff3a0 con 0x7ff40406d8a0 2024-02-19T19:12:51.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.119+0000 7ff4297e5700 1 -- 172.21.15.88:0/2859410977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff40406d8a0 msgr2=0x7ff40406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.119+0000 7ff4297e5700 1 --2- 172.21.15.88:0/2859410977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff40406d8a0 0x7ff40406fd50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff424068e10 tx=0x7ff40c01a040 comp rx=0 tx=0).stop 2024-02-19T19:12:51.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.119+0000 7ff4297e5700 1 -- 172.21.15.88:0/2859410977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4240ffbd0 msgr2=0x7ff424196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.119+0000 7ff4297e5700 1 --2- 172.21.15.88:0/2859410977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4240ffbd0 0x7ff424196da0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff41400ba50 tx=0x7ff41400bd60 comp rx=0 tx=0).stop 2024-02-19T19:12:51.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.119+0000 7ff4297e5700 1 -- 172.21.15.88:0/2859410977 shutdown_connections 2024-02-19T19:12:51.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.119+0000 7ff4297e5700 1 --2- 172.21.15.88:0/2859410977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff40406d8a0 0x7ff40406fd50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.119+0000 7ff4297e5700 1 --2- 172.21.15.88:0/2859410977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4240ffbd0 0x7ff424196da0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.119+0000 7ff4297e5700 1 --2- 172.21.15.88:0/2859410977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff424104130 0x7ff4241972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.119+0000 7ff4297e5700 1 -- 172.21.15.88:0/2859410977 >> 172.21.15.88:0/2859410977 conn(0x7ff424075130 msgr2=0x7ff4240fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:51.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.120+0000 7ff4297e5700 1 -- 172.21.15.88:0/2859410977 shutdown_connections 2024-02-19T19:12:51.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.120+0000 7ff4297e5700 1 -- 172.21.15.88:0/2859410977 wait complete. 2024-02-19T19:12:51.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.224+0000 7f1a4ade0700 1 -- 172.21.15.88:0/2015968711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a440ffc60 msgr2=0x7f1a4410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.224+0000 7f1a4ade0700 1 --2- 172.21.15.88:0/2015968711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a440ffc60 0x7f1a4410a6d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009910 tx=0x7f1a34009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:51.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.224+0000 7f1a4ade0700 1 -- 172.21.15.88:0/2015968711 shutdown_connections 2024-02-19T19:12:51.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.224+0000 7f1a4ade0700 1 --2- 172.21.15.88:0/2015968711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a440ffc60 0x7f1a4410a6d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.224+0000 7f1a4ade0700 1 --2- 172.21.15.88:0/2015968711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440ff350 0x7f1a440ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.224+0000 7f1a4ade0700 1 -- 172.21.15.88:0/2015968711 >> 172.21.15.88:0/2015968711 conn(0x7f1a440748b0 msgr2=0x7f1a44074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:51.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.224+0000 7f1a4ade0700 1 -- 172.21.15.88:0/2015968711 shutdown_connections 2024-02-19T19:12:51.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.224+0000 7f1a4ade0700 1 -- 172.21.15.88:0/2015968711 wait complete. 2024-02-19T19:12:51.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.224+0000 7f1a4ade0700 1 Processor -- start 2024-02-19T19:12:51.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.224+0000 7f1a4ade0700 1 -- start start 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a4ade0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440ff350 0x7f1a44100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a4ade0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a440ffc60 0x7f1a44100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a4ade0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a44100fb0 con 0x7f1a440ffc60 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a4ade0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a441010f0 con 0x7f1a440ff350 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a3bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a440ffc60 0x7f1a44100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a3bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a440ffc60 0x7f1a44100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36734/0 (socket says 172.21.15.88:36734) 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a3bfff700 1 -- 172.21.15.88:0/259896144 learned_addr learned my addr 172.21.15.88:0/259896144 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a48b7c700 1 --2- 172.21.15.88:0/259896144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440ff350 0x7f1a44100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a3bfff700 1 -- 172.21.15.88:0/259896144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440ff350 msgr2=0x7f1a44100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a3bfff700 1 --2- 172.21.15.88:0/259896144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440ff350 0x7f1a44100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a3bfff700 1 -- 172.21.15.88:0/259896144 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a340095f0 con 0x7f1a440ffc60 2024-02-19T19:12:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.225+0000 7f1a3bfff700 1 --2- 172.21.15.88:0/259896144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a440ffc60 0x7f1a44100a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1a34006010 tx=0x7f1a3400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:51.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.226+0000 7f1a39ffb700 1 -- 172.21.15.88:0/259896144 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a3401d070 con 0x7f1a440ffc60 2024-02-19T19:12:51.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.226+0000 7f1a39ffb700 1 -- 172.21.15.88:0/259896144 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a3400f460 con 0x7f1a440ffc60 2024-02-19T19:12:51.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.226+0000 7f1a39ffb700 1 -- 172.21.15.88:0/259896144 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a34017610 con 0x7f1a440ffc60 2024-02-19T19:12:51.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.226+0000 7f1a4ade0700 1 -- 172.21.15.88:0/259896144 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a44101370 con 0x7f1a440ffc60 2024-02-19T19:12:51.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.226+0000 7f1a4ade0700 1 -- 172.21.15.88:0/259896144 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a441a23e0 con 0x7f1a440ffc60 2024-02-19T19:12:51.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.227+0000 7f1a4ade0700 1 -- 172.21.15.88:0/259896144 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a44107dd0 con 0x7f1a440ffc60 2024-02-19T19:12:51.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.228+0000 7f1a39ffb700 1 -- 172.21.15.88:0/259896144 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1a3400f5d0 con 0x7f1a440ffc60 2024-02-19T19:12:51.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.228+0000 7f1a39ffb700 1 --2- 172.21.15.88:0/259896144 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a2c06d8a0 0x7f1a2c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:51.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.228+0000 7f1a39ffb700 1 -- 172.21.15.88:0/259896144 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1a3408ea70 con 0x7f1a440ffc60 2024-02-19T19:12:51.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.228+0000 7f1a48b7c700 1 --2- 172.21.15.88:0/259896144 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a2c06d8a0 0x7f1a2c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:51.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.229+0000 7f1a48b7c700 1 --2- 172.21.15.88:0/259896144 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a2c06d8a0 0x7f1a2c06fd50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a4406a640 tx=0x7f1a40009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:51.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.233+0000 7f1a39ffb700 1 -- 172.21.15.88:0/259896144 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a34058220 con 0x7f1a440ffc60 2024-02-19T19:12:51.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.441+0000 7f1a4ade0700 1 -- 172.21.15.88:0/259896144 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a44061190 con 0x7f1a2c06d8a0 2024-02-19T19:12:51.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.451+0000 7f1a39ffb700 1 -- 172.21.15.88:0/259896144 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f1a44061190 con 0x7f1a2c06d8a0 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (10m) 112s ago 20m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (20m) 112s ago 20m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (2m) 94s ago 19m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (6m) 112s ago 20m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (6m) 94s ago 18m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (9m) 112s ago 19m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (2m) 112s ago 14m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (117s) 112s ago 14m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (98s) 94s ago 14m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (14m) 94s ago 14m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (11m) 112s ago 22m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:12:51.474 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (10m) 94s ago 18m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:12:51.475 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (8m) 112s ago 22m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:12:51.475 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (7m) 94s ago 18m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:12:51.475 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (10m) 112s ago 20m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:12:51.475 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (10m) 94s ago 18m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:12:51.475 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (5m) 112s ago 17m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:12:51.475 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (4m) 112s ago 17m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:12:51.475 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (4m) 112s ago 17m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:12:51.475 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (3m) 94s ago 16m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:12:51.475 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (3m) 94s ago 16m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:12:51.475 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (2m) 94s ago 15m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:12:51.475 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (10m) 112s ago 19m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:12:51.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.453+0000 7f1a4ade0700 1 -- 172.21.15.88:0/259896144 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a2c06d8a0 msgr2=0x7f1a2c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.453+0000 7f1a4ade0700 1 --2- 172.21.15.88:0/259896144 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a2c06d8a0 0x7f1a2c06fd50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1a4406a640 tx=0x7f1a40009190 comp rx=0 tx=0).stop 2024-02-19T19:12:51.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.453+0000 7f1a4ade0700 1 -- 172.21.15.88:0/259896144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a440ffc60 msgr2=0x7f1a44100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.453+0000 7f1a4ade0700 1 --2- 172.21.15.88:0/259896144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a440ffc60 0x7f1a44100a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1a34006010 tx=0x7f1a3400bd70 comp rx=0 tx=0).stop 2024-02-19T19:12:51.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.453+0000 7f1a4ade0700 1 -- 172.21.15.88:0/259896144 shutdown_connections 2024-02-19T19:12:51.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.453+0000 7f1a4ade0700 1 --2- 172.21.15.88:0/259896144 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a2c06d8a0 0x7f1a2c06fd50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.453+0000 7f1a4ade0700 1 --2- 172.21.15.88:0/259896144 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a440ff350 0x7f1a44100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.453+0000 7f1a4ade0700 1 --2- 172.21.15.88:0/259896144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a440ffc60 0x7f1a44100a70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.453+0000 7f1a4ade0700 1 -- 172.21.15.88:0/259896144 >> 172.21.15.88:0/259896144 conn(0x7f1a440748b0 msgr2=0x7f1a44104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:51.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.453+0000 7f1a4ade0700 1 -- 172.21.15.88:0/259896144 shutdown_connections 2024-02-19T19:12:51.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.454+0000 7f1a4ade0700 1 -- 172.21.15.88:0/259896144 wait complete. 2024-02-19T19:12:51.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.556+0000 7fd58b59e700 1 -- 172.21.15.88:0/212366976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5841004e0 msgr2=0x7fd5841008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.556+0000 7fd58b59e700 1 --2- 172.21.15.88:0/212366976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5841004e0 0x7fd5841008b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd57c00b0a0 tx=0x7fd57c00b3b0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.556+0000 7fd58b59e700 1 -- 172.21.15.88:0/212366976 shutdown_connections 2024-02-19T19:12:51.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.556+0000 7fd58b59e700 1 --2- 172.21.15.88:0/212366976 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd584100e80 0x7fd584104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.556+0000 7fd58b59e700 1 --2- 172.21.15.88:0/212366976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5841004e0 0x7fd5841008b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.556+0000 7fd58b59e700 1 -- 172.21.15.88:0/212366976 >> 172.21.15.88:0/212366976 conn(0x7fd5840fbda0 msgr2=0x7fd5840fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:51.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:51 smithi088 ceph-mon[88643]: pgmap v277: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:51.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.556+0000 7fd58b59e700 1 -- 172.21.15.88:0/212366976 shutdown_connections 2024-02-19T19:12:51.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.556+0000 7fd58b59e700 1 -- 172.21.15.88:0/212366976 wait complete. 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.557+0000 7fd58b59e700 1 Processor -- start 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.557+0000 7fd58b59e700 1 -- start start 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.557+0000 7fd58b59e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5841004e0 0x7fd584073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.557+0000 7fd58b59e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd584100e80 0x7fd584073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.557+0000 7fd58b59e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd584073ad0 con 0x7fd5841004e0 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.557+0000 7fd58b59e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd584073c10 con 0x7fd584100e80 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.557+0000 7fd58933a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5841004e0 0x7fd584073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.557+0000 7fd58933a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5841004e0 0x7fd584073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36748/0 (socket says 172.21.15.88:36748) 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.557+0000 7fd58933a700 1 -- 172.21.15.88:0/3271942576 learned_addr learned my addr 172.21.15.88:0/3271942576 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.557+0000 7fd588b39700 1 --2- 172.21.15.88:0/3271942576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd584100e80 0x7fd584073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.558+0000 7fd58933a700 1 -- 172.21.15.88:0/3271942576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd584100e80 msgr2=0x7fd584073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.558+0000 7fd58933a700 1 --2- 172.21.15.88:0/3271942576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd584100e80 0x7fd584073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.558+0000 7fd58933a700 1 -- 172.21.15.88:0/3271942576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57c009d00 con 0x7fd5841004e0 2024-02-19T19:12:51.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.558+0000 7fd58933a700 1 --2- 172.21.15.88:0/3271942576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5841004e0 0x7fd584073050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd57c000f80 tx=0x7fd57c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:51.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.558+0000 7fd5827fc700 1 -- 172.21.15.88:0/3271942576 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd57c00e040 con 0x7fd5841004e0 2024-02-19T19:12:51.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.558+0000 7fd58b59e700 1 -- 172.21.15.88:0/3271942576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd584073e90 con 0x7fd5841004e0 2024-02-19T19:12:51.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.558+0000 7fd58b59e700 1 -- 172.21.15.88:0/3271942576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5841a2490 con 0x7fd5841004e0 2024-02-19T19:12:51.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.558+0000 7fd5827fc700 1 -- 172.21.15.88:0/3271942576 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd57c003e60 con 0x7fd5841004e0 2024-02-19T19:12:51.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.558+0000 7fd5827fc700 1 -- 172.21.15.88:0/3271942576 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd57c01d910 con 0x7fd5841004e0 2024-02-19T19:12:51.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.559+0000 7fd58b59e700 1 -- 172.21.15.88:0/3271942576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd570005320 con 0x7fd5841004e0 2024-02-19T19:12:51.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.560+0000 7fd5827fc700 1 -- 172.21.15.88:0/3271942576 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd57c004040 con 0x7fd5841004e0 2024-02-19T19:12:51.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.560+0000 7fd5827fc700 1 --2- 172.21.15.88:0/3271942576 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56c06d8a0 0x7fd56c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:51.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.560+0000 7fd5827fc700 1 -- 172.21.15.88:0/3271942576 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd57c08eca0 con 0x7fd5841004e0 2024-02-19T19:12:51.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.560+0000 7fd588b39700 1 --2- 172.21.15.88:0/3271942576 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56c06d8a0 0x7fd56c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:51.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.561+0000 7fd588b39700 1 --2- 172.21.15.88:0/3271942576 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56c06d8a0 0x7fd56c06fd50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd578006fd0 tx=0x7fd578008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:51.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.565+0000 7fd5827fc700 1 -- 172.21.15.88:0/3271942576 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd57c093070 con 0x7fd5841004e0 2024-02-19T19:12:51.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:51 smithi132 ceph-mon[70676]: pgmap v277: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:51.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.845+0000 7fd58b59e700 1 -- 172.21.15.88:0/3271942576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd570006200 con 0x7fd5841004e0 2024-02-19T19:12:51.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.845+0000 7fd5827fc700 1 -- 172.21.15.88:0/3271942576 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fd57c0584d0 con 0x7fd5841004e0 2024-02-19T19:12:51.868 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:12:51.868 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:12:51.868 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:12:51.868 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:12:51.868 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:12:51.868 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:12:51.868 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:12:51.868 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:12:51.869 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:12:51.869 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:12:51.869 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:12:51.869 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:12:51.869 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:12:51.869 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:12:51.869 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:12:51.869 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:12:51.869 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:12:51.869 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:12:51.869 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:12:51.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.848+0000 7fd58b59e700 1 -- 172.21.15.88:0/3271942576 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56c06d8a0 msgr2=0x7fd56c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.848+0000 7fd58b59e700 1 --2- 172.21.15.88:0/3271942576 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56c06d8a0 0x7fd56c06fd50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd578006fd0 tx=0x7fd578008040 comp rx=0 tx=0).stop 2024-02-19T19:12:51.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.848+0000 7fd58b59e700 1 -- 172.21.15.88:0/3271942576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5841004e0 msgr2=0x7fd584073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.848+0000 7fd58b59e700 1 --2- 172.21.15.88:0/3271942576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5841004e0 0x7fd584073050 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd57c000f80 tx=0x7fd57c00cdd0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.848+0000 7fd58b59e700 1 -- 172.21.15.88:0/3271942576 shutdown_connections 2024-02-19T19:12:51.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.848+0000 7fd58b59e700 1 --2- 172.21.15.88:0/3271942576 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56c06d8a0 0x7fd56c06fd50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.848+0000 7fd58b59e700 1 --2- 172.21.15.88:0/3271942576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5841004e0 0x7fd584073050 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.848+0000 7fd58b59e700 1 --2- 172.21.15.88:0/3271942576 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd584100e80 0x7fd584073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.848+0000 7fd58b59e700 1 -- 172.21.15.88:0/3271942576 >> 172.21.15.88:0/3271942576 conn(0x7fd5840fbda0 msgr2=0x7fd5840fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:51.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.848+0000 7fd58b59e700 1 -- 172.21.15.88:0/3271942576 shutdown_connections 2024-02-19T19:12:51.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.848+0000 7fd58b59e700 1 -- 172.21.15.88:0/3271942576 wait complete. 2024-02-19T19:12:51.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.952+0000 7f46b300f700 1 -- 172.21.15.88:0/3210739498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46ac0ff3c0 msgr2=0x7f46ac0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.952+0000 7f46b300f700 1 --2- 172.21.15.88:0/3210739498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46ac0ff3c0 0x7f46ac0ff790 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f46a0009910 tx=0x7f46a0009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:51.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.952+0000 7f46b300f700 1 -- 172.21.15.88:0/3210739498 shutdown_connections 2024-02-19T19:12:51.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.952+0000 7f46b300f700 1 --2- 172.21.15.88:0/3210739498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac0ffd60 0x7f46ac103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.952+0000 7f46b300f700 1 --2- 172.21.15.88:0/3210739498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46ac0ff3c0 0x7f46ac0ff790 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.952+0000 7f46b300f700 1 -- 172.21.15.88:0/3210739498 >> 172.21.15.88:0/3210739498 conn(0x7f46ac0fac80 msgr2=0x7f46ac0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:51.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.952+0000 7f46b300f700 1 -- 172.21.15.88:0/3210739498 shutdown_connections 2024-02-19T19:12:51.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.952+0000 7f46b300f700 1 -- 172.21.15.88:0/3210739498 wait complete. 2024-02-19T19:12:51.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.953+0000 7f46b300f700 1 Processor -- start 2024-02-19T19:12:51.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.953+0000 7f46b300f700 1 -- start start 2024-02-19T19:12:51.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.953+0000 7f46b300f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46ac0ff3c0 0x7f46ac198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:51.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.953+0000 7f46b300f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac0ffd60 0x7f46ac1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:51.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.953+0000 7f46b300f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac198da0 con 0x7f46ac0ff3c0 2024-02-19T19:12:51.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.953+0000 7f46b300f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac19cb30 con 0x7f46ac0ffd60 2024-02-19T19:12:51.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.953+0000 7f46b0dab700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46ac0ff3c0 0x7f46ac198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:51.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.953+0000 7f46b0dab700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46ac0ff3c0 0x7f46ac198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36766/0 (socket says 172.21.15.88:36766) 2024-02-19T19:12:51.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.953+0000 7f46b0dab700 1 -- 172.21.15.88:0/2774838220 learned_addr learned my addr 172.21.15.88:0/2774838220 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:51.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.954+0000 7f46abfff700 1 --2- 172.21.15.88:0/2774838220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac0ffd60 0x7f46ac1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:51.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.954+0000 7f46b0dab700 1 -- 172.21.15.88:0/2774838220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac0ffd60 msgr2=0x7f46ac1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:51.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.954+0000 7f46b0dab700 1 --2- 172.21.15.88:0/2774838220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac0ffd60 0x7f46ac1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.954+0000 7f46b0dab700 1 -- 172.21.15.88:0/2774838220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00095f0 con 0x7f46ac0ff3c0 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.954+0000 7f46b0dab700 1 --2- 172.21.15.88:0/2774838220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46ac0ff3c0 0x7f46ac198180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46a0006010 tx=0x7f46a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.954+0000 7f46a9ffb700 1 -- 172.21.15.88:0/2774838220 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46a001d070 con 0x7f46ac0ff3c0 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.954+0000 7f46a9ffb700 1 -- 172.21.15.88:0/2774838220 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f46a0004fb0 con 0x7f46ac0ff3c0 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.954+0000 7f46b300f700 1 -- 172.21.15.88:0/2774838220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46ac19cdb0 con 0x7f46ac0ff3c0 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.954+0000 7f46a9ffb700 1 -- 172.21.15.88:0/2774838220 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46a0022ea0 con 0x7f46ac0ff3c0 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.954+0000 7f46b300f700 1 -- 172.21.15.88:0/2774838220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac19d2a0 con 0x7f46ac0ff3c0 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.955+0000 7f46b300f700 1 -- 172.21.15.88:0/2774838220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46ac04f2a0 con 0x7f46ac0ff3c0 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.956+0000 7f46a9ffb700 1 -- 172.21.15.88:0/2774838220 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f46a0022470 con 0x7f46ac0ff3c0 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.956+0000 7f46a9ffb700 1 --2- 172.21.15.88:0/2774838220 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f469c06d8a0 0x7f469c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.956+0000 7f46a9ffb700 1 -- 172.21.15.88:0/2774838220 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f46a008ec30 con 0x7f46ac0ff3c0 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.956+0000 7f46abfff700 1 --2- 172.21.15.88:0/2774838220 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f469c06d8a0 0x7f469c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:51.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.956+0000 7f46abfff700 1 --2- 172.21.15.88:0/2774838220 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f469c06d8a0 0x7f469c06fd50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4698006fd0 tx=0x7f4698008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:51.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:51.961+0000 7f46a9ffb700 1 -- 172.21.15.88:0/2774838220 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f46a0058630 con 0x7f46ac0ff3c0 2024-02-19T19:12:52.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.199+0000 7f46b300f700 1 -- 172.21.15.88:0/2774838220 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f46ac19d5b0 con 0x7f46ac0ff3c0 2024-02-19T19:12:52.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.200+0000 7f46a9ffb700 1 -- 172.21.15.88:0/2774838220 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f46a0027070 con 0x7f46ac0ff3c0 2024-02-19T19:12:52.222 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:12:52.222 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:12:52.222 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:12:52.223 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:12:52.224 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:12:52.225 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:12:52.225 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:12:52.225 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:12:52.225 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:12:52.225 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:12:52.225 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:12:52.225 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:12:52.225 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:12:52.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.202+0000 7f46b300f700 1 -- 172.21.15.88:0/2774838220 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f469c06d8a0 msgr2=0x7f469c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:52.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.202+0000 7f46b300f700 1 --2- 172.21.15.88:0/2774838220 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f469c06d8a0 0x7f469c06fd50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4698006fd0 tx=0x7f4698008040 comp rx=0 tx=0).stop 2024-02-19T19:12:52.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.202+0000 7f46b300f700 1 -- 172.21.15.88:0/2774838220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46ac0ff3c0 msgr2=0x7f46ac198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:52.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.202+0000 7f46b300f700 1 --2- 172.21.15.88:0/2774838220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46ac0ff3c0 0x7f46ac198180 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f46a0006010 tx=0x7f46a000be20 comp rx=0 tx=0).stop 2024-02-19T19:12:52.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.202+0000 7f46b300f700 1 -- 172.21.15.88:0/2774838220 shutdown_connections 2024-02-19T19:12:52.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.202+0000 7f46b300f700 1 --2- 172.21.15.88:0/2774838220 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f469c06d8a0 0x7f469c06fd50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.202+0000 7f46b300f700 1 --2- 172.21.15.88:0/2774838220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46ac0ff3c0 0x7f46ac198180 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.202+0000 7f46b300f700 1 --2- 172.21.15.88:0/2774838220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46ac0ffd60 0x7f46ac1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.202+0000 7f46b300f700 1 -- 172.21.15.88:0/2774838220 >> 172.21.15.88:0/2774838220 conn(0x7f46ac0fac80 msgr2=0x7f46ac102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:52.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.202+0000 7f46b300f700 1 -- 172.21.15.88:0/2774838220 shutdown_connections 2024-02-19T19:12:52.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.202+0000 7f46b300f700 1 -- 172.21.15.88:0/2774838220 wait complete. 2024-02-19T19:12:52.227 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:12:52.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.308+0000 7f8d8cacc700 1 -- 172.21.15.88:0/303533236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d880ffbd0 msgr2=0x7f8d88100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:52.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.308+0000 7f8d8cacc700 1 --2- 172.21.15.88:0/303533236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d880ffbd0 0x7f8d88100040 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8d70009910 tx=0x7f8d70009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:52.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.308+0000 7f8d8cacc700 1 -- 172.21.15.88:0/303533236 shutdown_connections 2024-02-19T19:12:52.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.308+0000 7f8d8cacc700 1 --2- 172.21.15.88:0/303533236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d880ffbd0 0x7f8d88100040 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.308+0000 7f8d8cacc700 1 --2- 172.21.15.88:0/303533236 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d88104130 0x7f8d88104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.308+0000 7f8d8cacc700 1 -- 172.21.15.88:0/303533236 >> 172.21.15.88:0/303533236 conn(0x7f8d88075130 msgr2=0x7f8d88075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:52.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.308+0000 7f8d8cacc700 1 -- 172.21.15.88:0/303533236 shutdown_connections 2024-02-19T19:12:52.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.308+0000 7f8d8cacc700 1 -- 172.21.15.88:0/303533236 wait complete. 2024-02-19T19:12:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d8cacc700 1 Processor -- start 2024-02-19T19:12:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d8cacc700 1 -- start start 2024-02-19T19:12:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d8cacc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d880ffbd0 0x7f8d88196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d8cacc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d88104130 0x7f8d88197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d8cacc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88197860 con 0x7f8d88104130 2024-02-19T19:12:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d8cacc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d881979a0 con 0x7f8d880ffbd0 2024-02-19T19:12:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d85d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d88104130 0x7f8d88197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:52.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d8659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d880ffbd0 0x7f8d88196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d85d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d88104130 0x7f8d88197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36778/0 (socket says 172.21.15.88:36778) 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d8659c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d880ffbd0 0x7f8d88196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:46840/0 (socket says 172.21.15.88:46840) 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d85d9b700 1 -- 172.21.15.88:0/109627175 learned_addr learned my addr 172.21.15.88:0/109627175 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d85d9b700 1 -- 172.21.15.88:0/109627175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d880ffbd0 msgr2=0x7f8d88196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d85d9b700 1 --2- 172.21.15.88:0/109627175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d880ffbd0 0x7f8d88196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.309+0000 7f8d85d9b700 1 -- 172.21.15.88:0/109627175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d700095f0 con 0x7f8d88104130 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.310+0000 7f8d85d9b700 1 --2- 172.21.15.88:0/109627175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d88104130 0x7f8d88197180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000bd70 tx=0x7f8d7000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.310+0000 7f8d7f7fe700 1 -- 172.21.15.88:0/109627175 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d7001d070 con 0x7f8d88104130 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.310+0000 7f8d7f7fe700 1 -- 172.21.15.88:0/109627175 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d7000f460 con 0x7f8d88104130 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.310+0000 7f8d8cacc700 1 -- 172.21.15.88:0/109627175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8819fcb0 con 0x7f8d88104130 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.310+0000 7f8d7f7fe700 1 -- 172.21.15.88:0/109627175 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d70017630 con 0x7f8d88104130 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.310+0000 7f8d8cacc700 1 -- 172.21.15.88:0/109627175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d881a0120 con 0x7f8d88104130 2024-02-19T19:12:52.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.311+0000 7f8d8cacc700 1 -- 172.21.15.88:0/109627175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8804f2a0 con 0x7f8d88104130 2024-02-19T19:12:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.311+0000 7f8d7f7fe700 1 -- 172.21.15.88:0/109627175 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8d7000f5d0 con 0x7f8d88104130 2024-02-19T19:12:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.312+0000 7f8d7f7fe700 1 --2- 172.21.15.88:0/109627175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d7406d8a0 0x7f8d7406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.312+0000 7f8d7f7fe700 1 -- 172.21.15.88:0/109627175 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8d7008eac0 con 0x7f8d88104130 2024-02-19T19:12:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.312+0000 7f8d8659c700 1 --2- 172.21.15.88:0/109627175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d7406d8a0 0x7f8d7406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:52.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.312+0000 7f8d8659c700 1 --2- 172.21.15.88:0/109627175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d7406d8a0 0x7f8d7406fd50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d80005950 tx=0x7f8d8000a300 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:52.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.316+0000 7f8d7f7fe700 1 -- 172.21.15.88:0/109627175 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d70058270 con 0x7f8d88104130 2024-02-19T19:12:52.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.533+0000 7f8d8cacc700 1 -- 172.21.15.88:0/109627175 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d880ff240 con 0x7f8d7406d8a0 2024-02-19T19:12:52.556 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:52 smithi088 ceph-mon[88643]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:52.556 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:52 smithi088 ceph-mon[88643]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:52.556 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:52 smithi088 ceph-mon[88643]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:52.556 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:52 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3271942576' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:12:52.556 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:52 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2774838220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:12:52.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.534+0000 7f8d7f7fe700 1 -- 172.21.15.88:0/109627175 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8d880ff240 con 0x7f8d7406d8a0 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:12:52.557 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:12:52.558 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:12:52.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.536+0000 7f8d8cacc700 1 -- 172.21.15.88:0/109627175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d7406d8a0 msgr2=0x7f8d7406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:52.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.536+0000 7f8d8cacc700 1 --2- 172.21.15.88:0/109627175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d7406d8a0 0x7f8d7406fd50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8d80005950 tx=0x7f8d8000a300 comp rx=0 tx=0).stop 2024-02-19T19:12:52.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.536+0000 7f8d8cacc700 1 -- 172.21.15.88:0/109627175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d88104130 msgr2=0x7f8d88197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:52.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.536+0000 7f8d8cacc700 1 --2- 172.21.15.88:0/109627175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d88104130 0x7f8d88197180 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000bd70 tx=0x7f8d7000bda0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.536+0000 7f8d8cacc700 1 -- 172.21.15.88:0/109627175 shutdown_connections 2024-02-19T19:12:52.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.536+0000 7f8d8cacc700 1 --2- 172.21.15.88:0/109627175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d7406d8a0 0x7f8d7406fd50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.536+0000 7f8d8cacc700 1 --2- 172.21.15.88:0/109627175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d880ffbd0 0x7f8d88196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.536+0000 7f8d8cacc700 1 --2- 172.21.15.88:0/109627175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d88104130 0x7f8d88197180 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.536+0000 7f8d8cacc700 1 -- 172.21.15.88:0/109627175 >> 172.21.15.88:0/109627175 conn(0x7f8d88075130 msgr2=0x7f8d880feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:52.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.536+0000 7f8d8cacc700 1 -- 172.21.15.88:0/109627175 shutdown_connections 2024-02-19T19:12:52.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.536+0000 7f8d8cacc700 1 -- 172.21.15.88:0/109627175 wait complete. 2024-02-19T19:12:52.635 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:52 smithi132 ceph-mon[70676]: from='client.34554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:52.635 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:52 smithi132 ceph-mon[70676]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:52.635 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:52 smithi132 ceph-mon[70676]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:52.635 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:52 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3271942576' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:12:52.636 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:52 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2774838220' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:12:52.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.640+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/210970904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd04104130 msgr2=0x7fcd04104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:52.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.640+0000 7fcd0c3ca700 1 --2- 172.21.15.88:0/210970904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd04104130 0x7fcd04104500 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009910 tx=0x7fccf4009c20 comp rx=0 tx=0).stop 2024-02-19T19:12:52.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.640+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/210970904 shutdown_connections 2024-02-19T19:12:52.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.640+0000 7fcd0c3ca700 1 --2- 172.21.15.88:0/210970904 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd040ffbd0 0x7fcd04100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.640+0000 7fcd0c3ca700 1 --2- 172.21.15.88:0/210970904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd04104130 0x7fcd04104500 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.640+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/210970904 >> 172.21.15.88:0/210970904 conn(0x7fcd04075130 msgr2=0x7fcd04075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:52.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.640+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/210970904 shutdown_connections 2024-02-19T19:12:52.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.640+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/210970904 wait complete. 2024-02-19T19:12:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd0c3ca700 1 Processor -- start 2024-02-19T19:12:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd0c3ca700 1 -- start start 2024-02-19T19:12:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd0c3ca700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd040ffbd0 0x7fcd04194aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd0c3ca700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd04104130 0x7fcd04194fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd0c3ca700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04195520 con 0x7fcd04104130 2024-02-19T19:12:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd0c3ca700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04195690 con 0x7fcd040ffbd0 2024-02-19T19:12:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd09965700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd04104130 0x7fcd04194fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd09965700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd04104130 0x7fcd04194fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36786/0 (socket says 172.21.15.88:36786) 2024-02-19T19:12:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd09965700 1 -- 172.21.15.88:0/814941667 learned_addr learned my addr 172.21.15.88:0/814941667 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd0a166700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd040ffbd0 0x7fcd04194aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd09965700 1 -- 172.21.15.88:0/814941667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd040ffbd0 msgr2=0x7fcd04194aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd09965700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd040ffbd0 0x7fcd04194aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd09965700 1 -- 172.21.15.88:0/814941667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf40095f0 con 0x7fcd04104130 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd0a166700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd040ffbd0 0x7fcd04194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.641+0000 7fcd09965700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd04104130 0x7fcd04194fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000d900 tx=0x7fcd0000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.642+0000 7fccfb7fe700 1 -- 172.21.15.88:0/814941667 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd00004a00 con 0x7fcd04104130 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.642+0000 7fccfb7fe700 1 -- 172.21.15.88:0/814941667 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd000054e0 con 0x7fcd04104130 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.642+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/814941667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd041a2110 con 0x7fcd04104130 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.642+0000 7fccfb7fe700 1 -- 172.21.15.88:0/814941667 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd00009d60 con 0x7fcd04104130 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.642+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/814941667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd041a2470 con 0x7fcd04104130 2024-02-19T19:12:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.643+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/814941667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd041076a0 con 0x7fcd04104130 2024-02-19T19:12:52.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.643+0000 7fccfb7fe700 1 -- 172.21.15.88:0/814941667 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcd00005020 con 0x7fcd04104130 2024-02-19T19:12:52.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.644+0000 7fccfb7fe700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fccf006d8a0 0x7fccf006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:12:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.644+0000 7fcd0a166700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fccf006d8a0 0x7fccf006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:12:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.644+0000 7fccfb7fe700 1 -- 172.21.15.88:0/814941667 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcd0008d630 con 0x7fcd04104130 2024-02-19T19:12:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.644+0000 7fcd0a166700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fccf006d8a0 0x7fccf006fd50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fccf40059d0 tx=0x7fccf401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:12:52.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.649+0000 7fccfb7fe700 1 -- 172.21.15.88:0/814941667 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd00056e60 con 0x7fcd04104130 2024-02-19T19:12:52.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.927+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/814941667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fcd0402cc30 con 0x7fcd04104130 2024-02-19T19:12:52.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.927+0000 7fccfb7fe700 1 -- 172.21.15.88:0/814941667 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fcd0005a520 con 0x7fcd04104130 2024-02-19T19:12:52.950 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:12:52.950 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:12:52.950 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:12:52.950 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:12:52.951 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:12:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.930+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/814941667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fccf006d8a0 msgr2=0x7fccf006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.930+0000 7fcd0c3ca700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fccf006d8a0 0x7fccf006fd50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fccf40059d0 tx=0x7fccf401a040 comp rx=0 tx=0).stop 2024-02-19T19:12:52.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.930+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/814941667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd04104130 msgr2=0x7fcd04194fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:12:52.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.930+0000 7fcd0c3ca700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd04104130 0x7fcd04194fe0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcd0000d900 tx=0x7fcd0000dcc0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.930+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/814941667 shutdown_connections 2024-02-19T19:12:52.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.930+0000 7fcd0c3ca700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fccf006d8a0 0x7fccf006fd50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.930+0000 7fcd0c3ca700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd040ffbd0 0x7fcd04194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.930+0000 7fcd0c3ca700 1 --2- 172.21.15.88:0/814941667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd04104130 0x7fcd04194fe0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:12:52.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.930+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/814941667 >> 172.21.15.88:0/814941667 conn(0x7fcd04075130 msgr2=0x7fcd040feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:12:52.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.930+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/814941667 shutdown_connections 2024-02-19T19:12:52.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:12:52.930+0000 7fcd0c3ca700 1 -- 172.21.15.88:0/814941667 wait complete. 2024-02-19T19:12:53.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:53 smithi088 ceph-mon[88643]: pgmap v278: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:53.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:53 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/814941667' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:12:53.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:53 smithi132 ceph-mon[70676]: pgmap v278: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:53.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:53 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/814941667' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:12:54.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:54 smithi088 ceph-mon[88643]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:54.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:54 smithi132 ceph-mon[70676]: from='client.34574 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:12:55.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:55 smithi088 ceph-mon[88643]: pgmap v279: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:55.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:55 smithi132 ceph-mon[70676]: pgmap v279: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:57.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:57 smithi088 ceph-mon[88643]: pgmap v280: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:57.688 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:57 smithi132 ceph-mon[70676]: pgmap v280: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:58.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:12:58.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:12:59.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:12:59 smithi088 ceph-mon[88643]: pgmap v281: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:12:59.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:12:59 smithi132 ceph-mon[70676]: pgmap v281: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:01.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:01 smithi088 ceph-mon[88643]: pgmap v282: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:01.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:01 smithi132 ceph-mon[70676]: pgmap v282: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:03.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:03 smithi088 ceph-mon[88643]: pgmap v283: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:03.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:03 smithi132 ceph-mon[70676]: pgmap v283: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:05.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:05 smithi088 ceph-mon[88643]: pgmap v284: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:05.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:05 smithi132 ceph-mon[70676]: pgmap v284: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:07.602 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:07 smithi088 ceph-mon[88643]: pgmap v285: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:07.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:07 smithi132 ceph-mon[70676]: pgmap v285: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:09.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:09 smithi132 ceph-mon[70676]: pgmap v286: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:09.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:09 smithi088 ceph-mon[88643]: pgmap v286: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:11.655 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:11 smithi088 ceph-mon[88643]: pgmap v287: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:11.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:11 smithi132 ceph-mon[70676]: pgmap v287: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:13.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:13 smithi132 ceph-mon[70676]: pgmap v288: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:13.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:13:13.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:13 smithi088 ceph-mon[88643]: pgmap v288: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:13.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:13:15.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:15 smithi132 ceph-mon[70676]: pgmap v289: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:15.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:15 smithi088 ceph-mon[88643]: pgmap v289: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:17.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:17 smithi132 ceph-mon[70676]: pgmap v290: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:17.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:17 smithi088 ceph-mon[88643]: pgmap v290: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:19.634 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:19 smithi088 ceph-mon[88643]: pgmap v291: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:19.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:19 smithi132 ceph-mon[70676]: pgmap v291: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:21.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:21 smithi132 ceph-mon[70676]: pgmap v292: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:21.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:21 smithi088 ceph-mon[88643]: pgmap v292: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:23.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.048+0000 7f2e25f97700 1 -- 172.21.15.88:0/1156924633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e200ff940 msgr2=0x7f2e2010c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.048+0000 7f2e25f97700 1 --2- 172.21.15.88:0/1156924633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e200ff940 0x7f2e2010c6a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2e14009910 tx=0x7f2e14009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:23.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.048+0000 7f2e25f97700 1 -- 172.21.15.88:0/1156924633 shutdown_connections 2024-02-19T19:13:23.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.048+0000 7f2e25f97700 1 --2- 172.21.15.88:0/1156924633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e200ff940 0x7f2e2010c6a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.048+0000 7f2e25f97700 1 --2- 172.21.15.88:0/1156924633 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e200ff030 0x7f2e200ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.048+0000 7f2e25f97700 1 -- 172.21.15.88:0/1156924633 >> 172.21.15.88:0/1156924633 conn(0x7f2e200fac60 msgr2=0x7f2e200fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:23.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.048+0000 7f2e25f97700 1 -- 172.21.15.88:0/1156924633 shutdown_connections 2024-02-19T19:13:23.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.048+0000 7f2e25f97700 1 -- 172.21.15.88:0/1156924633 wait complete. 2024-02-19T19:13:23.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.049+0000 7f2e25f97700 1 Processor -- start 2024-02-19T19:13:23.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.049+0000 7f2e25f97700 1 -- start start 2024-02-19T19:13:23.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.049+0000 7f2e25f97700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e200ff030 0x7f2e20198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.049+0000 7f2e25f97700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e200ff940 0x7f2e20198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.049+0000 7f2e25f97700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e20198c50 con 0x7f2e200ff940 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.049+0000 7f2e25f97700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2019c9e0 con 0x7f2e200ff030 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.049+0000 7f2e1effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e200ff940 0x7f2e20198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.049+0000 7f2e1effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e200ff940 0x7f2e20198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35938/0 (socket says 172.21.15.88:35938) 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.049+0000 7f2e1effd700 1 -- 172.21.15.88:0/2488410473 learned_addr learned my addr 172.21.15.88:0/2488410473 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.050+0000 7f2e1f7fe700 1 --2- 172.21.15.88:0/2488410473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e200ff030 0x7f2e20198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.050+0000 7f2e1effd700 1 -- 172.21.15.88:0/2488410473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e200ff030 msgr2=0x7f2e20198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.050+0000 7f2e1effd700 1 --2- 172.21.15.88:0/2488410473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e200ff030 0x7f2e20198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.050+0000 7f2e1effd700 1 -- 172.21.15.88:0/2488410473 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e140095f0 con 0x7f2e200ff940 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.050+0000 7f2e1effd700 1 --2- 172.21.15.88:0/2488410473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e200ff940 0x7f2e20198570 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2e14006010 tx=0x7f2e1400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:23.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.050+0000 7f2e1cff9700 1 -- 172.21.15.88:0/2488410473 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e1401d070 con 0x7f2e200ff940 2024-02-19T19:13:23.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.050+0000 7f2e25f97700 1 -- 172.21.15.88:0/2488410473 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e2019ccc0 con 0x7f2e200ff940 2024-02-19T19:13:23.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.050+0000 7f2e1cff9700 1 -- 172.21.15.88:0/2488410473 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e14004fb0 con 0x7f2e200ff940 2024-02-19T19:13:23.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.050+0000 7f2e1cff9700 1 -- 172.21.15.88:0/2488410473 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e1400f690 con 0x7f2e200ff940 2024-02-19T19:13:23.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.050+0000 7f2e25f97700 1 -- 172.21.15.88:0/2488410473 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e2019d210 con 0x7f2e200ff940 2024-02-19T19:13:23.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.051+0000 7f2e25f97700 1 -- 172.21.15.88:0/2488410473 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e20109e10 con 0x7f2e200ff940 2024-02-19T19:13:23.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.052+0000 7f2e1cff9700 1 -- 172.21.15.88:0/2488410473 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2e140228a0 con 0x7f2e200ff940 2024-02-19T19:13:23.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.052+0000 7f2e1cff9700 1 --2- 172.21.15.88:0/2488410473 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2e0806d8a0 0x7f2e0806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:23.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.052+0000 7f2e1cff9700 1 -- 172.21.15.88:0/2488410473 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2e1408edf0 con 0x7f2e200ff940 2024-02-19T19:13:23.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.052+0000 7f2e1f7fe700 1 --2- 172.21.15.88:0/2488410473 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2e0806d8a0 0x7f2e0806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:23.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.052+0000 7f2e1f7fe700 1 --2- 172.21.15.88:0/2488410473 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2e0806d8a0 0x7f2e0806fd50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2e10005fd0 tx=0x7f2e1000a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:23.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.057+0000 7f2e1cff9700 1 -- 172.21.15.88:0/2488410473 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2e140586c0 con 0x7f2e200ff940 2024-02-19T19:13:23.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.273+0000 7f2e25f97700 1 -- 172.21.15.88:0/2488410473 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e201994a0 con 0x7f2e0806d8a0 2024-02-19T19:13:23.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.274+0000 7f2e1cff9700 1 -- 172.21.15.88:0/2488410473 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2e201994a0 con 0x7f2e0806d8a0 2024-02-19T19:13:23.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.276+0000 7f2e25f97700 1 -- 172.21.15.88:0/2488410473 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2e0806d8a0 msgr2=0x7f2e0806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.276+0000 7f2e25f97700 1 --2- 172.21.15.88:0/2488410473 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2e0806d8a0 0x7f2e0806fd50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2e10005fd0 tx=0x7f2e1000a560 comp rx=0 tx=0).stop 2024-02-19T19:13:23.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.276+0000 7f2e25f97700 1 -- 172.21.15.88:0/2488410473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e200ff940 msgr2=0x7f2e20198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.276+0000 7f2e25f97700 1 --2- 172.21.15.88:0/2488410473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e200ff940 0x7f2e20198570 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2e14006010 tx=0x7f2e1400be20 comp rx=0 tx=0).stop 2024-02-19T19:13:23.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.276+0000 7f2e25f97700 1 -- 172.21.15.88:0/2488410473 shutdown_connections 2024-02-19T19:13:23.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.276+0000 7f2e25f97700 1 --2- 172.21.15.88:0/2488410473 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2e0806d8a0 0x7f2e0806fd50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.276+0000 7f2e25f97700 1 --2- 172.21.15.88:0/2488410473 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e200ff030 0x7f2e20198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.276+0000 7f2e25f97700 1 --2- 172.21.15.88:0/2488410473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e200ff940 0x7f2e20198570 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.277+0000 7f2e25f97700 1 -- 172.21.15.88:0/2488410473 >> 172.21.15.88:0/2488410473 conn(0x7f2e200fac60 msgr2=0x7f2e200fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:23.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.277+0000 7f2e25f97700 1 -- 172.21.15.88:0/2488410473 shutdown_connections 2024-02-19T19:13:23.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.277+0000 7f2e25f97700 1 -- 172.21.15.88:0/2488410473 wait complete. 2024-02-19T19:13:23.310 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:13:23.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.383+0000 7f9384d9f700 1 -- 172.21.15.88:0/2866409105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9380100280 msgr2=0x7f9380100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.383+0000 7f9384d9f700 1 --2- 172.21.15.88:0/2866409105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9380100280 0x7f9380100650 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9370009910 tx=0x7f9370009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:23.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.383+0000 7f9384d9f700 1 -- 172.21.15.88:0/2866409105 shutdown_connections 2024-02-19T19:13:23.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.383+0000 7f9384d9f700 1 --2- 172.21.15.88:0/2866409105 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9380100b90 0x7f93801092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.383+0000 7f9384d9f700 1 --2- 172.21.15.88:0/2866409105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9380100280 0x7f9380100650 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.383+0000 7f9384d9f700 1 -- 172.21.15.88:0/2866409105 >> 172.21.15.88:0/2866409105 conn(0x7f9380074770 msgr2=0x7f9380074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:23.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.383+0000 7f9384d9f700 1 -- 172.21.15.88:0/2866409105 shutdown_connections 2024-02-19T19:13:23.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.383+0000 7f9384d9f700 1 -- 172.21.15.88:0/2866409105 wait complete. 2024-02-19T19:13:23.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.383+0000 7f9384d9f700 1 Processor -- start 2024-02-19T19:13:23.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f9384d9f700 1 -- start start 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f9384d9f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9380100280 0x7f9380196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f9384d9f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9380100b90 0x7f9380196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f9384d9f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93801975e0 con 0x7f9380100280 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f9384d9f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9380197750 con 0x7f9380100b90 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f937f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9380100280 0x7f9380196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f937f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9380100280 0x7f9380196a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35956/0 (socket says 172.21.15.88:35956) 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f937f7fe700 1 -- 172.21.15.88:0/4278264172 learned_addr learned my addr 172.21.15.88:0/4278264172 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f937f7fe700 1 -- 172.21.15.88:0/4278264172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9380100b90 msgr2=0x7f9380196fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f937f7fe700 1 --2- 172.21.15.88:0/4278264172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9380100b90 0x7f9380196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f937f7fe700 1 -- 172.21.15.88:0/4278264172 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93700095f0 con 0x7f9380100280 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f937f7fe700 1 --2- 172.21.15.88:0/4278264172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9380100280 0x7f9380196a80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9370000c00 tx=0x7f93700056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.384+0000 7f937cff9700 1 -- 172.21.15.88:0/4278264172 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f937001d070 con 0x7f9380100280 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.385+0000 7f9384d9f700 1 -- 172.21.15.88:0/4278264172 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938019faf0 con 0x7f9380100280 2024-02-19T19:13:23.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.385+0000 7f937cff9700 1 -- 172.21.15.88:0/4278264172 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f937000f460 con 0x7f9380100280 2024-02-19T19:13:23.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.385+0000 7f937cff9700 1 -- 172.21.15.88:0/4278264172 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9370021640 con 0x7f9380100280 2024-02-19T19:13:23.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.385+0000 7f9384d9f700 1 -- 172.21.15.88:0/4278264172 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938019ff90 con 0x7f9380100280 2024-02-19T19:13:23.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.385+0000 7f9384d9f700 1 -- 172.21.15.88:0/4278264172 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93801069c0 con 0x7f9380100280 2024-02-19T19:13:23.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.386+0000 7f937cff9700 1 -- 172.21.15.88:0/4278264172 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f937000fa60 con 0x7f9380100280 2024-02-19T19:13:23.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.386+0000 7f937cff9700 1 --2- 172.21.15.88:0/4278264172 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f936806d8a0 0x7f936806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:23.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.387+0000 7f937effd700 1 --2- 172.21.15.88:0/4278264172 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f936806d8a0 0x7f936806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:23.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.387+0000 7f937cff9700 1 -- 172.21.15.88:0/4278264172 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f937008e750 con 0x7f9380100280 2024-02-19T19:13:23.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.387+0000 7f937effd700 1 --2- 172.21.15.88:0/4278264172 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f936806d8a0 0x7f936806fd50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9374005fd0 tx=0x7f9374005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:23.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.391+0000 7f937cff9700 1 -- 172.21.15.88:0/4278264172 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9370057f00 con 0x7f9380100280 2024-02-19T19:13:23.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.607+0000 7f9384d9f700 1 -- 172.21.15.88:0/4278264172 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9380061190 con 0x7f936806d8a0 2024-02-19T19:13:23.630 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:23 smithi088 ceph-mon[88643]: pgmap v293: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:23.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.608+0000 7f937cff9700 1 -- 172.21.15.88:0/4278264172 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9380061190 con 0x7f936806d8a0 2024-02-19T19:13:23.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.610+0000 7f9384d9f700 1 -- 172.21.15.88:0/4278264172 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f936806d8a0 msgr2=0x7f936806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.610+0000 7f9384d9f700 1 --2- 172.21.15.88:0/4278264172 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f936806d8a0 0x7f936806fd50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9374005fd0 tx=0x7f9374005ee0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.610+0000 7f9384d9f700 1 -- 172.21.15.88:0/4278264172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9380100280 msgr2=0x7f9380196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.610+0000 7f9384d9f700 1 --2- 172.21.15.88:0/4278264172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9380100280 0x7f9380196a80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9370000c00 tx=0x7f93700056a0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.610+0000 7f9384d9f700 1 -- 172.21.15.88:0/4278264172 shutdown_connections 2024-02-19T19:13:23.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.610+0000 7f9384d9f700 1 --2- 172.21.15.88:0/4278264172 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f936806d8a0 0x7f936806fd50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.610+0000 7f9384d9f700 1 --2- 172.21.15.88:0/4278264172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9380100280 0x7f9380196a80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.610+0000 7f9384d9f700 1 --2- 172.21.15.88:0/4278264172 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9380100b90 0x7f9380196fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.610+0000 7f9384d9f700 1 -- 172.21.15.88:0/4278264172 >> 172.21.15.88:0/4278264172 conn(0x7f9380074770 msgr2=0x7f9380103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:23.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.611+0000 7f9384d9f700 1 -- 172.21.15.88:0/4278264172 shutdown_connections 2024-02-19T19:13:23.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.611+0000 7f9384d9f700 1 -- 172.21.15.88:0/4278264172 wait complete. 2024-02-19T19:13:23.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.715+0000 7fbe211fd700 1 -- 172.21.15.88:0/3040002480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe1c0ffd60 msgr2=0x7fbe1c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.715+0000 7fbe211fd700 1 --2- 172.21.15.88:0/3040002480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe1c0ffd60 0x7fbe1c103d30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c009910 tx=0x7fbe0c009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:23.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.716+0000 7fbe211fd700 1 -- 172.21.15.88:0/3040002480 shutdown_connections 2024-02-19T19:13:23.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.716+0000 7fbe211fd700 1 --2- 172.21.15.88:0/3040002480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe1c0ffd60 0x7fbe1c103d30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.716+0000 7fbe211fd700 1 --2- 172.21.15.88:0/3040002480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0ff3c0 0x7fbe1c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.716+0000 7fbe211fd700 1 -- 172.21.15.88:0/3040002480 >> 172.21.15.88:0/3040002480 conn(0x7fbe1c0fac80 msgr2=0x7fbe1c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:23.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.716+0000 7fbe211fd700 1 -- 172.21.15.88:0/3040002480 shutdown_connections 2024-02-19T19:13:23.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.716+0000 7fbe211fd700 1 -- 172.21.15.88:0/3040002480 wait complete. 2024-02-19T19:13:23.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.716+0000 7fbe211fd700 1 Processor -- start 2024-02-19T19:13:23.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe211fd700 1 -- start start 2024-02-19T19:13:23.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe211fd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe1c0ff3c0 0x7fbe1c194b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:23.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe211fd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0ffd60 0x7fbe1c1950b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:23.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe211fd700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c1955f0 con 0x7fbe1c0ff3c0 2024-02-19T19:13:23.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe211fd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c195760 con 0x7fbe1c0ffd60 2024-02-19T19:13:23.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe1ad9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe1c0ff3c0 0x7fbe1c194b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:23.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe1ad9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe1c0ff3c0 0x7fbe1c194b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35982/0 (socket says 172.21.15.88:35982) 2024-02-19T19:13:23.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe1ad9d700 1 -- 172.21.15.88:0/134092136 learned_addr learned my addr 172.21.15.88:0/134092136 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:23.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe1a59c700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0ffd60 0x7fbe1c1950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:23.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe1ad9d700 1 -- 172.21.15.88:0/134092136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0ffd60 msgr2=0x7fbe1c1950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe1ad9d700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0ffd60 0x7fbe1c1950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe1ad9d700 1 -- 172.21.15.88:0/134092136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0c0095f0 con 0x7fbe1c0ff3c0 2024-02-19T19:13:23.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe1a59c700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0ffd60 0x7fbe1c1950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:13:23.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.717+0000 7fbe1ad9d700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe1c0ff3c0 0x7fbe1c194b70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c8a0 tx=0x7fbe1000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:23.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.718+0000 7fbe03fff700 1 -- 172.21.15.88:0/134092136 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe1000cea0 con 0x7fbe1c0ff3c0 2024-02-19T19:13:23.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.718+0000 7fbe211fd700 1 -- 172.21.15.88:0/134092136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c1a21e0 con 0x7fbe1c0ff3c0 2024-02-19T19:13:23.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.718+0000 7fbe03fff700 1 -- 172.21.15.88:0/134092136 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe10004830 con 0x7fbe1c0ff3c0 2024-02-19T19:13:23.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.718+0000 7fbe03fff700 1 -- 172.21.15.88:0/134092136 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe100055b0 con 0x7fbe1c0ff3c0 2024-02-19T19:13:23.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.718+0000 7fbe211fd700 1 -- 172.21.15.88:0/134092136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c1a2540 con 0x7fbe1c0ff3c0 2024-02-19T19:13:23.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.719+0000 7fbe211fd700 1 -- 172.21.15.88:0/134092136 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe1c107730 con 0x7fbe1c0ff3c0 2024-02-19T19:13:23.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.719+0000 7fbe03fff700 1 -- 172.21.15.88:0/134092136 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbe100049a0 con 0x7fbe1c0ff3c0 2024-02-19T19:13:23.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.720+0000 7fbe03fff700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe0406d8a0 0x7fbe0406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:23.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.720+0000 7fbe1a59c700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe0406d8a0 0x7fbe0406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:23.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.720+0000 7fbe03fff700 1 -- 172.21.15.88:0/134092136 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbe1008d170 con 0x7fbe1c0ff3c0 2024-02-19T19:13:23.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.720+0000 7fbe1a59c700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe0406d8a0 0x7fbe0406fd50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006010 tx=0x7fbe0c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:23.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.724+0000 7fbe03fff700 1 -- 172.21.15.88:0/134092136 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe1005a450 con 0x7fbe1c0ff3c0 2024-02-19T19:13:23.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:23 smithi132 ceph-mon[70676]: pgmap v293: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.931+0000 7fbe211fd700 1 -- 172.21.15.88:0/134092136 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe1c0694d0 con 0x7fbe0406d8a0 2024-02-19T19:13:23.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.942+0000 7fbe03fff700 1 -- 172.21.15.88:0/134092136 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbe1c0694d0 con 0x7fbe0406d8a0 2024-02-19T19:13:23.965 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:13:23.965 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (10m) 2m ago 20m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:13:23.965 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (21m) 2m ago 21m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:13:23.965 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (3m) 2m ago 19m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:13:23.965 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (6m) 2m ago 21m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:13:23.965 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (6m) 2m ago 19m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:13:23.966 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (9m) 2m ago 20m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:13:23.966 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (2m) 2m ago 14m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:13:23.966 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (2m) 2m ago 14m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:13:23.966 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (2m) 2m ago 14m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:13:23.966 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (14m) 2m ago 14m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:13:23.966 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (12m) 2m ago 22m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:13:23.966 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (11m) 2m ago 19m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:13:23.966 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (8m) 2m ago 22m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:13:23.966 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (7m) 2m ago 19m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:13:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (11m) 2m ago 21m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:13:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (11m) 2m ago 19m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:13:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (6m) 2m ago 18m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:13:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (5m) 2m ago 18m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:13:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (4m) 2m ago 17m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:13:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (4m) 2m ago 17m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:13:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (3m) 2m ago 16m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:13:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (3m) 2m ago 16m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:13:23.967 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (10m) 2m ago 19m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:13:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.944+0000 7fbe211fd700 1 -- 172.21.15.88:0/134092136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe0406d8a0 msgr2=0x7fbe0406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.944+0000 7fbe211fd700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe0406d8a0 0x7fbe0406fd50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c006010 tx=0x7fbe0c00b540 comp rx=0 tx=0).stop 2024-02-19T19:13:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.944+0000 7fbe211fd700 1 -- 172.21.15.88:0/134092136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe1c0ff3c0 msgr2=0x7fbe1c194b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.944+0000 7fbe211fd700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe1c0ff3c0 0x7fbe1c194b70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000c8a0 tx=0x7fbe1000cc60 comp rx=0 tx=0).stop 2024-02-19T19:13:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.944+0000 7fbe211fd700 1 -- 172.21.15.88:0/134092136 shutdown_connections 2024-02-19T19:13:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.944+0000 7fbe211fd700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe0406d8a0 0x7fbe0406fd50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.944+0000 7fbe211fd700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe1c0ff3c0 0x7fbe1c194b70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.944+0000 7fbe211fd700 1 --2- 172.21.15.88:0/134092136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe1c0ffd60 0x7fbe1c1950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:23.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.944+0000 7fbe211fd700 1 -- 172.21.15.88:0/134092136 >> 172.21.15.88:0/134092136 conn(0x7fbe1c0fac80 msgr2=0x7fbe1c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:23.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.945+0000 7fbe211fd700 1 -- 172.21.15.88:0/134092136 shutdown_connections 2024-02-19T19:13:23.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:23.945+0000 7fbe211fd700 1 -- 172.21.15.88:0/134092136 wait complete. 2024-02-19T19:13:24.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.046+0000 7f4568324700 1 -- 172.21.15.88:0/1701826581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560073a00 msgr2=0x7f4560110d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:24.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 --2- 172.21.15.88:0/1701826581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560073a00 0x7f4560110d50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4550009910 tx=0x7f4550009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:24.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 -- 172.21.15.88:0/1701826581 shutdown_connections 2024-02-19T19:13:24.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 --2- 172.21.15.88:0/1701826581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560073a00 0x7f4560110d50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 --2- 172.21.15.88:0/1701826581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45600730f0 0x7f45600734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 -- 172.21.15.88:0/1701826581 >> 172.21.15.88:0/1701826581 conn(0x7f45600fbda0 msgr2=0x7f45600fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 -- 172.21.15.88:0/1701826581 shutdown_connections 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 -- 172.21.15.88:0/1701826581 wait complete. 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 Processor -- start 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 -- start start 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45600730f0 0x7f456019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560073a00 0x7f456019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456019cfc0 con 0x7f4560073a00 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.047+0000 7f4568324700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456019d100 con 0x7f45600730f0 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f45658bf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560073a00 0x7f456019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f45658bf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560073a00 0x7f456019c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35998/0 (socket says 172.21.15.88:35998) 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f45658bf700 1 -- 172.21.15.88:0/3836039000 learned_addr learned my addr 172.21.15.88:0/3836039000 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f45660c0700 1 --2- 172.21.15.88:0/3836039000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45600730f0 0x7f456019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f45658bf700 1 -- 172.21.15.88:0/3836039000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45600730f0 msgr2=0x7f456019c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f45658bf700 1 --2- 172.21.15.88:0/3836039000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45600730f0 0x7f456019c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f45658bf700 1 -- 172.21.15.88:0/3836039000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45500095f0 con 0x7f4560073a00 2024-02-19T19:13:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f45658bf700 1 --2- 172.21.15.88:0/3836039000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560073a00 0x7f456019c970 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f455000b5c0 tx=0x7f455000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f45577fe700 1 -- 172.21.15.88:0/3836039000 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f455001d070 con 0x7f4560073a00 2024-02-19T19:13:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f45577fe700 1 -- 172.21.15.88:0/3836039000 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f455000f460 con 0x7f4560073a00 2024-02-19T19:13:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f4568324700 1 -- 172.21.15.88:0/3836039000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45601a0f50 con 0x7f4560073a00 2024-02-19T19:13:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f45577fe700 1 -- 172.21.15.88:0/3836039000 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4550017630 con 0x7f4560073a00 2024-02-19T19:13:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.048+0000 7f4568324700 1 -- 172.21.15.88:0/3836039000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45601a14a0 con 0x7f4560073a00 2024-02-19T19:13:24.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.049+0000 7f4568324700 1 -- 172.21.15.88:0/3836039000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456010e4d0 con 0x7f4560073a00 2024-02-19T19:13:24.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.050+0000 7f45577fe700 1 -- 172.21.15.88:0/3836039000 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f455000f5d0 con 0x7f4560073a00 2024-02-19T19:13:24.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.050+0000 7f45577fe700 1 --2- 172.21.15.88:0/3836039000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f454c06d8a0 0x7f454c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:24.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.050+0000 7f45660c0700 1 --2- 172.21.15.88:0/3836039000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f454c06d8a0 0x7f454c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:24.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.050+0000 7f45577fe700 1 -- 172.21.15.88:0/3836039000 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f455008ea20 con 0x7f4560073a00 2024-02-19T19:13:24.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.051+0000 7f45660c0700 1 --2- 172.21.15.88:0/3836039000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f454c06d8a0 0x7f454c06fd50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f455c005ea0 tx=0x7f455c005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:24.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.055+0000 7f45577fe700 1 -- 172.21.15.88:0/3836039000 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4550058250 con 0x7f4560073a00 2024-02-19T19:13:24.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.337+0000 7f4568324700 1 -- 172.21.15.88:0/3836039000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f456002cc30 con 0x7f4560073a00 2024-02-19T19:13:24.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.337+0000 7f45577fe700 1 -- 172.21.15.88:0/3836039000 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f455005b910 con 0x7f4560073a00 2024-02-19T19:13:24.360 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:13:24.360 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:13:24.360 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:13:24.360 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:13:24.360 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:13:24.360 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:13:24.360 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:13:24.360 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:13:24.361 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:13:24.361 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:13:24.361 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:13:24.361 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:13:24.361 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:13:24.361 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:13:24.361 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:13:24.361 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:13:24.361 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:13:24.361 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:13:24.361 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:13:24.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.340+0000 7f4568324700 1 -- 172.21.15.88:0/3836039000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f454c06d8a0 msgr2=0x7f454c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:24.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.340+0000 7f4568324700 1 --2- 172.21.15.88:0/3836039000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f454c06d8a0 0x7f454c06fd50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f455c005ea0 tx=0x7f455c005e30 comp rx=0 tx=0).stop 2024-02-19T19:13:24.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.340+0000 7f4568324700 1 -- 172.21.15.88:0/3836039000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560073a00 msgr2=0x7f456019c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:24.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.340+0000 7f4568324700 1 --2- 172.21.15.88:0/3836039000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560073a00 0x7f456019c970 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f455000b5c0 tx=0x7f455000be00 comp rx=0 tx=0).stop 2024-02-19T19:13:24.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.340+0000 7f4568324700 1 -- 172.21.15.88:0/3836039000 shutdown_connections 2024-02-19T19:13:24.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.340+0000 7f4568324700 1 --2- 172.21.15.88:0/3836039000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f454c06d8a0 0x7f454c06fd50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.340+0000 7f4568324700 1 --2- 172.21.15.88:0/3836039000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f45600730f0 0x7f456019c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.340+0000 7f4568324700 1 --2- 172.21.15.88:0/3836039000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4560073a00 0x7f456019c970 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.340+0000 7f4568324700 1 -- 172.21.15.88:0/3836039000 >> 172.21.15.88:0/3836039000 conn(0x7f45600fbda0 msgr2=0x7f4560102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:24.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.340+0000 7f4568324700 1 -- 172.21.15.88:0/3836039000 shutdown_connections 2024-02-19T19:13:24.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.340+0000 7f4568324700 1 -- 172.21.15.88:0/3836039000 wait complete. 2024-02-19T19:13:24.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.444+0000 7f5fadcd1700 1 -- 172.21.15.88:0/4200830611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa80ffbd0 msgr2=0x7f5fa8100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:24.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.444+0000 7f5fadcd1700 1 --2- 172.21.15.88:0/4200830611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa80ffbd0 0x7f5fa8100040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009910 tx=0x7f5f9c009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:24.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.444+0000 7f5fadcd1700 1 -- 172.21.15.88:0/4200830611 shutdown_connections 2024-02-19T19:13:24.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.444+0000 7f5fadcd1700 1 --2- 172.21.15.88:0/4200830611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa80ffbd0 0x7f5fa8100040 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.444+0000 7f5fadcd1700 1 --2- 172.21.15.88:0/4200830611 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5fa8104130 0x7f5fa8104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.444+0000 7f5fadcd1700 1 -- 172.21.15.88:0/4200830611 >> 172.21.15.88:0/4200830611 conn(0x7f5fa8075130 msgr2=0x7f5fa8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:24.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.444+0000 7f5fadcd1700 1 -- 172.21.15.88:0/4200830611 shutdown_connections 2024-02-19T19:13:24.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.444+0000 7f5fadcd1700 1 -- 172.21.15.88:0/4200830611 wait complete. 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fadcd1700 1 Processor -- start 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fadcd1700 1 -- start start 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fadcd1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa80ffbd0 0x7f5fa8194bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fadcd1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5fa8104130 0x7f5fa81950f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fadcd1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa8195630 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fadcd1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fa81957a0 con 0x7f5fa8104130 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa80ffbd0 0x7f5fa8194bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fa77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa80ffbd0 0x7f5fa8194bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36010/0 (socket says 172.21.15.88:36010) 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fa77fe700 1 -- 172.21.15.88:0/476157563 learned_addr learned my addr 172.21.15.88:0/476157563 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fa6ffd700 1 --2- 172.21.15.88:0/476157563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5fa8104130 0x7f5fa81950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fa77fe700 1 -- 172.21.15.88:0/476157563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5fa8104130 msgr2=0x7f5fa81950f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fa77fe700 1 --2- 172.21.15.88:0/476157563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5fa8104130 0x7f5fa81950f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.445+0000 7f5fa77fe700 1 -- 172.21.15.88:0/476157563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c0095f0 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.446+0000 7f5fa77fe700 1 --2- 172.21.15.88:0/476157563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa80ffbd0 0x7f5fa8194bb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800b4b0 tx=0x7f5f9800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:24.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.446+0000 7f5fa4ff9700 1 -- 172.21.15.88:0/476157563 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f98011800 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.446+0000 7f5fa4ff9700 1 -- 172.21.15.88:0/476157563 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f98011e20 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.446+0000 7f5fadcd1700 1 -- 172.21.15.88:0/476157563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fa81a2220 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.446+0000 7f5fa4ff9700 1 -- 172.21.15.88:0/476157563 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f9800f2e0 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.446+0000 7f5fadcd1700 1 -- 172.21.15.88:0/476157563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fa81a2580 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.447+0000 7f5fadcd1700 1 -- 172.21.15.88:0/476157563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fa804f2a0 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.447+0000 7f5fa4ff9700 1 -- 172.21.15.88:0/476157563 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5f98010380 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.448+0000 7f5fa4ff9700 1 --2- 172.21.15.88:0/476157563 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f9006d8a0 0x7f5f9006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:24.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.448+0000 7f5fa4ff9700 1 -- 172.21.15.88:0/476157563 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5f9808d090 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.448+0000 7f5fa6ffd700 1 --2- 172.21.15.88:0/476157563 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f9006d8a0 0x7f5f9006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:24.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.448+0000 7f5fa6ffd700 1 --2- 172.21.15.88:0/476157563 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f9006d8a0 0x7f5f9006fd50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8068ea0 tx=0x7f5f9c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.453+0000 7f5fa4ff9700 1 -- 172.21.15.88:0/476157563 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f9805a370 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.687 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:24 smithi088 ceph-mon[88643]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:24.687 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:24 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3836039000' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:13:24.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.693+0000 7f5fadcd1700 1 -- 172.21.15.88:0/476157563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5fa81a2860 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.694+0000 7f5fa4ff9700 1 -- 172.21.15.88:0/476157563 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f5f98059f00 con 0x7f5fa80ffbd0 2024-02-19T19:13:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:13:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:13:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:13:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:13:24.717 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:13:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:13:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:13:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:13:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:13:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:13:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:13:24.718 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:13:24.719 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:13:24.720 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:13:24.720 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:13:24.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.697+0000 7f5fadcd1700 1 -- 172.21.15.88:0/476157563 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f9006d8a0 msgr2=0x7f5f9006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:24.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.697+0000 7f5fadcd1700 1 --2- 172.21.15.88:0/476157563 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f9006d8a0 0x7f5f9006fd50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8068ea0 tx=0x7f5f9c01a040 comp rx=0 tx=0).stop 2024-02-19T19:13:24.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.697+0000 7f5fadcd1700 1 -- 172.21.15.88:0/476157563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa80ffbd0 msgr2=0x7f5fa8194bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:24.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.697+0000 7f5fadcd1700 1 --2- 172.21.15.88:0/476157563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa80ffbd0 0x7f5fa8194bb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f9800b4b0 tx=0x7f5f9800b870 comp rx=0 tx=0).stop 2024-02-19T19:13:24.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.697+0000 7f5fadcd1700 1 -- 172.21.15.88:0/476157563 shutdown_connections 2024-02-19T19:13:24.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.697+0000 7f5fadcd1700 1 --2- 172.21.15.88:0/476157563 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f9006d8a0 0x7f5f9006fd50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.697+0000 7f5fadcd1700 1 --2- 172.21.15.88:0/476157563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5fa80ffbd0 0x7f5fa8194bb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.697+0000 7f5fadcd1700 1 --2- 172.21.15.88:0/476157563 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5fa8104130 0x7f5fa81950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.697+0000 7f5fadcd1700 1 -- 172.21.15.88:0/476157563 >> 172.21.15.88:0/476157563 conn(0x7f5fa8075130 msgr2=0x7f5fa80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:24.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.697+0000 7f5fadcd1700 1 -- 172.21.15.88:0/476157563 shutdown_connections 2024-02-19T19:13:24.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.697+0000 7f5fadcd1700 1 -- 172.21.15.88:0/476157563 wait complete. 2024-02-19T19:13:24.722 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:13:24.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:24 smithi132 ceph-mon[70676]: from='client.34582 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:24.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:24 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3836039000' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:13:24.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.804+0000 7f9e601c7700 1 -- 172.21.15.88:0/2979070513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e581004c0 msgr2=0x7f9e58100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.804+0000 7f9e601c7700 1 --2- 172.21.15.88:0/2979070513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e581004c0 0x7f9e58100890 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009910 tx=0x7f9e48009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.804+0000 7f9e601c7700 1 -- 172.21.15.88:0/2979070513 shutdown_connections 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.804+0000 7f9e601c7700 1 --2- 172.21.15.88:0/2979070513 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e58100e60 0x7f9e58104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.804+0000 7f9e601c7700 1 --2- 172.21.15.88:0/2979070513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e581004c0 0x7f9e58100890 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.804+0000 7f9e601c7700 1 -- 172.21.15.88:0/2979070513 >> 172.21.15.88:0/2979070513 conn(0x7f9e580fbd80 msgr2=0x7f9e580fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.804+0000 7f9e601c7700 1 -- 172.21.15.88:0/2979070513 shutdown_connections 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.804+0000 7f9e601c7700 1 -- 172.21.15.88:0/2979070513 wait complete. 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e601c7700 1 Processor -- start 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e601c7700 1 -- start start 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e601c7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e581004c0 0x7f9e58195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e601c7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e58100e60 0x7f9e58196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e601c7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e58196b50 con 0x7f9e58100e60 2024-02-19T19:13:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e601c7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5819a8e0 con 0x7f9e581004c0 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e5d762700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e58100e60 0x7f9e58196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e5d762700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e58100e60 0x7f9e58196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36026/0 (socket says 172.21.15.88:36026) 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e5d762700 1 -- 172.21.15.88:0/3281034329 learned_addr learned my addr 172.21.15.88:0/3281034329 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e5df63700 1 --2- 172.21.15.88:0/3281034329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e581004c0 0x7f9e58195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e5d762700 1 -- 172.21.15.88:0/3281034329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e581004c0 msgr2=0x7f9e58195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e5d762700 1 --2- 172.21.15.88:0/3281034329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e581004c0 0x7f9e58195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e5d762700 1 -- 172.21.15.88:0/3281034329 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e480095f0 con 0x7f9e58100e60 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.805+0000 7f9e5d762700 1 --2- 172.21.15.88:0/3281034329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e58100e60 0x7f9e58196470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00d900 tx=0x7f9e4c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.806+0000 7f9e56ffd700 1 -- 172.21.15.88:0/3281034329 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e4c009900 con 0x7f9e58100e60 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.806+0000 7f9e56ffd700 1 -- 172.21.15.88:0/3281034329 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e4c010460 con 0x7f9e58100e60 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.806+0000 7f9e601c7700 1 -- 172.21.15.88:0/3281034329 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5819abc0 con 0x7f9e58100e60 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.806+0000 7f9e56ffd700 1 -- 172.21.15.88:0/3281034329 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e4c00f5f0 con 0x7f9e58100e60 2024-02-19T19:13:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.806+0000 7f9e601c7700 1 -- 172.21.15.88:0/3281034329 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5819b110 con 0x7f9e58100e60 2024-02-19T19:13:24.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.806+0000 7f9e601c7700 1 -- 172.21.15.88:0/3281034329 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e581087d0 con 0x7f9e58100e60 2024-02-19T19:13:24.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.807+0000 7f9e56ffd700 1 -- 172.21.15.88:0/3281034329 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9e4c0105d0 con 0x7f9e58100e60 2024-02-19T19:13:24.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.807+0000 7f9e56ffd700 1 --2- 172.21.15.88:0/3281034329 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4006d8a0 0x7f9e4006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:24.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.807+0000 7f9e56ffd700 1 -- 172.21.15.88:0/3281034329 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9e4c08d290 con 0x7f9e58100e60 2024-02-19T19:13:24.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.807+0000 7f9e5df63700 1 --2- 172.21.15.88:0/3281034329 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4006d8a0 0x7f9e4006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:24.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.808+0000 7f9e5df63700 1 --2- 172.21.15.88:0/3281034329 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4006d8a0 0x7f9e4006fd50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800b5c0 tx=0x7f9e4801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:24.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:24.812+0000 7f9e56ffd700 1 -- 172.21.15.88:0/3281034329 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9e4c056ac0 con 0x7f9e58100e60 2024-02-19T19:13:25.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.026+0000 7f9e601c7700 1 -- 172.21.15.88:0/3281034329 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e5819b380 con 0x7f9e4006d8a0 2024-02-19T19:13:25.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.027+0000 7f9e56ffd700 1 -- 172.21.15.88:0/3281034329 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9e5819b380 con 0x7f9e4006d8a0 2024-02-19T19:13:25.050 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:13:25.050 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:13:25.050 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:13:25.050 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:13:25.050 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:13:25.050 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:13:25.050 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:13:25.050 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:13:25.050 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:13:25.050 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:13:25.050 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:13:25.051 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:13:25.051 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:13:25.051 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:13:25.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.029+0000 7f9e601c7700 1 -- 172.21.15.88:0/3281034329 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4006d8a0 msgr2=0x7f9e4006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:25.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.029+0000 7f9e601c7700 1 --2- 172.21.15.88:0/3281034329 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4006d8a0 0x7f9e4006fd50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800b5c0 tx=0x7f9e4801a040 comp rx=0 tx=0).stop 2024-02-19T19:13:25.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.029+0000 7f9e601c7700 1 -- 172.21.15.88:0/3281034329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e58100e60 msgr2=0x7f9e58196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:25.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.029+0000 7f9e601c7700 1 --2- 172.21.15.88:0/3281034329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e58100e60 0x7f9e58196470 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00d900 tx=0x7f9e4c00dcc0 comp rx=0 tx=0).stop 2024-02-19T19:13:25.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.029+0000 7f9e601c7700 1 -- 172.21.15.88:0/3281034329 shutdown_connections 2024-02-19T19:13:25.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.029+0000 7f9e601c7700 1 --2- 172.21.15.88:0/3281034329 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e4006d8a0 0x7f9e4006fd50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:25.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.029+0000 7f9e601c7700 1 --2- 172.21.15.88:0/3281034329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e581004c0 0x7f9e58195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:25.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.029+0000 7f9e601c7700 1 --2- 172.21.15.88:0/3281034329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e58100e60 0x7f9e58196470 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:25.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.029+0000 7f9e601c7700 1 -- 172.21.15.88:0/3281034329 >> 172.21.15.88:0/3281034329 conn(0x7f9e580fbd80 msgr2=0x7f9e580fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:25.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.029+0000 7f9e601c7700 1 -- 172.21.15.88:0/3281034329 shutdown_connections 2024-02-19T19:13:25.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.029+0000 7f9e601c7700 1 -- 172.21.15.88:0/3281034329 wait complete. 2024-02-19T19:13:25.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.133+0000 7ff781b76700 1 -- 172.21.15.88:0/2244082484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff77c1004e0 msgr2=0x7ff77c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.133+0000 7ff781b76700 1 --2- 172.21.15.88:0/2244082484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff77c1004e0 0x7ff77c1008b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff76c009910 tx=0x7ff76c009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.134+0000 7ff781b76700 1 -- 172.21.15.88:0/2244082484 shutdown_connections 2024-02-19T19:13:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.134+0000 7ff781b76700 1 --2- 172.21.15.88:0/2244082484 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff77c100e80 0x7ff77c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.134+0000 7ff781b76700 1 --2- 172.21.15.88:0/2244082484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff77c1004e0 0x7ff77c1008b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.134+0000 7ff781b76700 1 -- 172.21.15.88:0/2244082484 >> 172.21.15.88:0/2244082484 conn(0x7ff77c0fbda0 msgr2=0x7ff77c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.134+0000 7ff781b76700 1 -- 172.21.15.88:0/2244082484 shutdown_connections 2024-02-19T19:13:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.134+0000 7ff781b76700 1 -- 172.21.15.88:0/2244082484 wait complete. 2024-02-19T19:13:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.134+0000 7ff781b76700 1 Processor -- start 2024-02-19T19:13:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.134+0000 7ff781b76700 1 -- start start 2024-02-19T19:13:25.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff781b76700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff77c1004e0 0x7ff77c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:25.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff781b76700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff77c100e80 0x7ff77c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:25.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff781b76700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c196bb0 con 0x7ff77c100e80 2024-02-19T19:13:25.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff781b76700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c19a940 con 0x7ff77c1004e0 2024-02-19T19:13:25.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff77affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff77c100e80 0x7ff77c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:25.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff77affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff77c100e80 0x7ff77c1964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36040/0 (socket says 172.21.15.88:36040) 2024-02-19T19:13:25.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff77affd700 1 -- 172.21.15.88:0/3857627485 learned_addr learned my addr 172.21.15.88:0/3857627485 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff77b7fe700 1 --2- 172.21.15.88:0/3857627485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff77c1004e0 0x7ff77c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff77affd700 1 -- 172.21.15.88:0/3857627485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff77c1004e0 msgr2=0x7ff77c195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff77affd700 1 --2- 172.21.15.88:0/3857627485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff77c1004e0 0x7ff77c195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff77affd700 1 -- 172.21.15.88:0/3857627485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff76c0095f0 con 0x7ff77c100e80 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff77affd700 1 --2- 172.21.15.88:0/3857627485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff77c100e80 0x7ff77c1964d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff77000b480 tx=0x7ff77000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff778ff9700 1 -- 172.21.15.88:0/3857627485 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff770011800 con 0x7ff77c100e80 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff778ff9700 1 -- 172.21.15.88:0/3857627485 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff770011e20 con 0x7ff77c100e80 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff781b76700 1 -- 172.21.15.88:0/3857627485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c19ac20 con 0x7ff77c100e80 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff778ff9700 1 -- 172.21.15.88:0/3857627485 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff770010ea0 con 0x7ff77c100e80 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.135+0000 7ff781b76700 1 -- 172.21.15.88:0/3857627485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c19b170 con 0x7ff77c100e80 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.136+0000 7ff781b76700 1 -- 172.21.15.88:0/3857627485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77c1087f0 con 0x7ff77c100e80 2024-02-19T19:13:25.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.137+0000 7ff778ff9700 1 -- 172.21.15.88:0/3857627485 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff770010380 con 0x7ff77c100e80 2024-02-19T19:13:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.137+0000 7ff778ff9700 1 --2- 172.21.15.88:0/3857627485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff76406d8a0 0x7ff76406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.137+0000 7ff778ff9700 1 -- 172.21.15.88:0/3857627485 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff77008cfa0 con 0x7ff77c100e80 2024-02-19T19:13:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.137+0000 7ff77b7fe700 1 --2- 172.21.15.88:0/3857627485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff76406d8a0 0x7ff76406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.138+0000 7ff77b7fe700 1 --2- 172.21.15.88:0/3857627485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff76406d8a0 0x7ff76406fd50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff76c006010 tx=0x7ff76c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:25.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.142+0000 7ff778ff9700 1 -- 172.21.15.88:0/3857627485 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff7700567d0 con 0x7ff77c100e80 2024-02-19T19:13:25.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.421+0000 7ff781b76700 1 -- 172.21.15.88:0/3857627485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff77c19b450 con 0x7ff77c100e80 2024-02-19T19:13:25.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.421+0000 7ff778ff9700 1 -- 172.21.15.88:0/3857627485 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff770059e90 con 0x7ff77c100e80 2024-02-19T19:13:25.444 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:13:25.444 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:13:25.444 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:13:25.444 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:13:25.445 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:13:25.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.424+0000 7ff781b76700 1 -- 172.21.15.88:0/3857627485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff76406d8a0 msgr2=0x7ff76406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:25.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.424+0000 7ff781b76700 1 --2- 172.21.15.88:0/3857627485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff76406d8a0 0x7ff76406fd50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff76c006010 tx=0x7ff76c01a040 comp rx=0 tx=0).stop 2024-02-19T19:13:25.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.424+0000 7ff781b76700 1 -- 172.21.15.88:0/3857627485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff77c100e80 msgr2=0x7ff77c1964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:25.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.424+0000 7ff781b76700 1 --2- 172.21.15.88:0/3857627485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff77c100e80 0x7ff77c1964d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff77000b480 tx=0x7ff77000b790 comp rx=0 tx=0).stop 2024-02-19T19:13:25.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.424+0000 7ff781b76700 1 -- 172.21.15.88:0/3857627485 shutdown_connections 2024-02-19T19:13:25.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.424+0000 7ff781b76700 1 --2- 172.21.15.88:0/3857627485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff76406d8a0 0x7ff76406fd50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:25.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.424+0000 7ff781b76700 1 --2- 172.21.15.88:0/3857627485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff77c1004e0 0x7ff77c195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:25.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.424+0000 7ff781b76700 1 --2- 172.21.15.88:0/3857627485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff77c100e80 0x7ff77c1964d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:25.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.424+0000 7ff781b76700 1 -- 172.21.15.88:0/3857627485 >> 172.21.15.88:0/3857627485 conn(0x7ff77c0fbda0 msgr2=0x7ff77c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:25.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.424+0000 7ff781b76700 1 -- 172.21.15.88:0/3857627485 shutdown_connections 2024-02-19T19:13:25.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:25.424+0000 7ff781b76700 1 -- 172.21.15.88:0/3857627485 wait complete. 2024-02-19T19:13:25.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:25 smithi132 ceph-mon[70676]: from='client.34586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:25.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:25 smithi132 ceph-mon[70676]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:25.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:25 smithi132 ceph-mon[70676]: pgmap v294: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:25.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:25 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/476157563' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:13:25.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:25 smithi088 ceph-mon[88643]: from='client.34586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:25.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:25 smithi088 ceph-mon[88643]: from='client.34590 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:25.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:25 smithi088 ceph-mon[88643]: pgmap v294: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:25.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:25 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/476157563' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:13:26.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:26 smithi132 ceph-mon[70676]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:26.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:26 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3857627485' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:13:26.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:26 smithi088 ceph-mon[88643]: from='client.34602 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:26.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:26 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3857627485' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:13:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:27 smithi132 ceph-mon[70676]: pgmap v295: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:27.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:27 smithi088 ceph-mon[88643]: pgmap v295: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:28.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:13:28.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:13:29.742 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:29 smithi088 ceph-mon[88643]: pgmap v296: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:29.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:29 smithi132 ceph-mon[70676]: pgmap v296: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:31.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:31 smithi132 ceph-mon[70676]: pgmap v297: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:31.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:31 smithi088 ceph-mon[88643]: pgmap v297: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:33.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:33 smithi132 ceph-mon[70676]: pgmap v298: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:33.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:33 smithi088 ceph-mon[88643]: pgmap v298: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:35.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:35 smithi132 ceph-mon[70676]: pgmap v299: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:35.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:35 smithi088 ceph-mon[88643]: pgmap v299: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:37.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:37 smithi132 ceph-mon[70676]: pgmap v300: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:37.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:37 smithi088 ceph-mon[88643]: pgmap v300: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:39.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:39 smithi132 ceph-mon[70676]: pgmap v301: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:39.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:39 smithi088 ceph-mon[88643]: pgmap v301: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:41.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:41 smithi132 ceph-mon[70676]: pgmap v302: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:41.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:41 smithi088 ceph-mon[88643]: pgmap v302: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:43.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:43 smithi132 ceph-mon[70676]: pgmap v303: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:43.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:13:43.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:43 smithi088 ceph-mon[88643]: pgmap v303: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:43.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:13:45.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:45 smithi132 ceph-mon[70676]: pgmap v304: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:45.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:45 smithi088 ceph-mon[88643]: pgmap v304: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:47.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:47 smithi132 ceph-mon[70676]: pgmap v305: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:47.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:47 smithi088 ceph-mon[88643]: pgmap v305: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:49.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:49 smithi132 ceph-mon[70676]: pgmap v306: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:49.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:49 smithi088 ceph-mon[88643]: pgmap v306: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:51.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:51 smithi132 ceph-mon[70676]: pgmap v307: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:51.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:51 smithi088 ceph-mon[88643]: pgmap v307: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:53.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:53 smithi132 ceph-mon[70676]: pgmap v308: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:53.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:53 smithi088 ceph-mon[88643]: pgmap v308: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.540+0000 7f93161e6700 1 -- 172.21.15.88:0/3550965901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9310100e40 msgr2=0x7f9310104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.540+0000 7f93161e6700 1 --2- 172.21.15.88:0/3550965901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9310100e40 0x7f9310104e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009910 tx=0x7f92f8009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.540+0000 7f93161e6700 1 -- 172.21.15.88:0/3550965901 shutdown_connections 2024-02-19T19:13:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.540+0000 7f93161e6700 1 --2- 172.21.15.88:0/3550965901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9310100e40 0x7f9310104e10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.540+0000 7f93161e6700 1 --2- 172.21.15.88:0/3550965901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93101004a0 0x7f9310100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.540+0000 7f93161e6700 1 -- 172.21.15.88:0/3550965901 >> 172.21.15.88:0/3550965901 conn(0x7f93100fbd40 msgr2=0x7f93100fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.540+0000 7f93161e6700 1 -- 172.21.15.88:0/3550965901 shutdown_connections 2024-02-19T19:13:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.540+0000 7f93161e6700 1 -- 172.21.15.88:0/3550965901 wait complete. 2024-02-19T19:13:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.540+0000 7f93161e6700 1 Processor -- start 2024-02-19T19:13:55.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f93161e6700 1 -- start start 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f93161e6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93101004a0 0x7f9310195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f93161e6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9310100e40 0x7f93101963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f93161e6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9310196ad0 con 0x7f9310100e40 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f93161e6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931019a860 con 0x7f93101004a0 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f9306ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9310100e40 0x7f93101963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f9306ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9310100e40 0x7f93101963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38180/0 (socket says 172.21.15.88:38180) 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f9306ffd700 1 -- 172.21.15.88:0/1468532825 learned_addr learned my addr 172.21.15.88:0/1468532825 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f9306ffd700 1 -- 172.21.15.88:0/1468532825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93101004a0 msgr2=0x7f9310195eb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f9306ffd700 1 --2- 172.21.15.88:0/1468532825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93101004a0 0x7f9310195eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f9306ffd700 1 -- 172.21.15.88:0/1468532825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f80095f0 con 0x7f9310100e40 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.541+0000 7f9306ffd700 1 --2- 172.21.15.88:0/1468532825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9310100e40 0x7f93101963f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f92f80092e0 tx=0x7f92f800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.542+0000 7f930d7fa700 1 -- 172.21.15.88:0/1468532825 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f801d070 con 0x7f9310100e40 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.542+0000 7f930d7fa700 1 -- 172.21.15.88:0/1468532825 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92f8005dc0 con 0x7f9310100e40 2024-02-19T19:13:55.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.542+0000 7f93161e6700 1 -- 172.21.15.88:0/1468532825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931019aae0 con 0x7f9310100e40 2024-02-19T19:13:55.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.542+0000 7f930d7fa700 1 -- 172.21.15.88:0/1468532825 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92f8017400 con 0x7f9310100e40 2024-02-19T19:13:55.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.542+0000 7f93161e6700 1 -- 172.21.15.88:0/1468532825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931019afd0 con 0x7f9310100e40 2024-02-19T19:13:55.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.543+0000 7f93161e6700 1 -- 172.21.15.88:0/1468532825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9310108810 con 0x7f9310100e40 2024-02-19T19:13:55.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.543+0000 7f930d7fa700 1 -- 172.21.15.88:0/1468532825 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f92f800f460 con 0x7f9310100e40 2024-02-19T19:13:55.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.544+0000 7f930d7fa700 1 --2- 172.21.15.88:0/1468532825 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f92fc06d8a0 0x7f92fc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:55.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.544+0000 7f930d7fa700 1 -- 172.21.15.88:0/1468532825 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f92f808ea30 con 0x7f9310100e40 2024-02-19T19:13:55.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.544+0000 7f930f7fe700 1 --2- 172.21.15.88:0/1468532825 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f92fc06d8a0 0x7f92fc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:55.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.544+0000 7f930f7fe700 1 --2- 172.21.15.88:0/1468532825 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f92fc06d8a0 0x7f92fc06fd50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f930000ba10 tx=0x7f930000b3f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:55.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.549+0000 7f930d7fa700 1 -- 172.21.15.88:0/1468532825 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92f80581e0 con 0x7f9310100e40 2024-02-19T19:13:55.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:55 smithi132 ceph-mon[70676]: pgmap v309: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:55.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.764+0000 7f93161e6700 1 -- 172.21.15.88:0/1468532825 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f931019b1f0 con 0x7f92fc06d8a0 2024-02-19T19:13:55.787 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:55 smithi088 ceph-mon[88643]: pgmap v309: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:55.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.765+0000 7f930d7fa700 1 -- 172.21.15.88:0/1468532825 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f931019b1f0 con 0x7f92fc06d8a0 2024-02-19T19:13:55.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.767+0000 7f93161e6700 1 -- 172.21.15.88:0/1468532825 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f92fc06d8a0 msgr2=0x7f92fc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:55.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.767+0000 7f93161e6700 1 --2- 172.21.15.88:0/1468532825 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f92fc06d8a0 0x7f92fc06fd50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f930000ba10 tx=0x7f930000b3f0 comp rx=0 tx=0).stop 2024-02-19T19:13:55.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.767+0000 7f93161e6700 1 -- 172.21.15.88:0/1468532825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9310100e40 msgr2=0x7f93101963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:55.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.767+0000 7f93161e6700 1 --2- 172.21.15.88:0/1468532825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9310100e40 0x7f93101963f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f92f80092e0 tx=0x7f92f800be20 comp rx=0 tx=0).stop 2024-02-19T19:13:55.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.768+0000 7f93161e6700 1 -- 172.21.15.88:0/1468532825 shutdown_connections 2024-02-19T19:13:55.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.768+0000 7f93161e6700 1 --2- 172.21.15.88:0/1468532825 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f92fc06d8a0 0x7f92fc06fd50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:55.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.768+0000 7f93161e6700 1 --2- 172.21.15.88:0/1468532825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93101004a0 0x7f9310195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:55.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.768+0000 7f93161e6700 1 --2- 172.21.15.88:0/1468532825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9310100e40 0x7f93101963f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:55.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.768+0000 7f93161e6700 1 -- 172.21.15.88:0/1468532825 >> 172.21.15.88:0/1468532825 conn(0x7f93100fbd40 msgr2=0x7f93100fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:55.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.768+0000 7f93161e6700 1 -- 172.21.15.88:0/1468532825 shutdown_connections 2024-02-19T19:13:55.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.768+0000 7f93161e6700 1 -- 172.21.15.88:0/1468532825 wait complete. 2024-02-19T19:13:55.801 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:13:55.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.871+0000 7f735bd61700 1 -- 172.21.15.88:0/2508188951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7354100e80 msgr2=0x7f7354104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:55.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.871+0000 7f735bd61700 1 --2- 172.21.15.88:0/2508188951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7354100e80 0x7f7354104e50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7344009910 tx=0x7f7344009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:55.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.871+0000 7f735bd61700 1 -- 172.21.15.88:0/2508188951 shutdown_connections 2024-02-19T19:13:55.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.871+0000 7f735bd61700 1 --2- 172.21.15.88:0/2508188951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7354100e80 0x7f7354104e50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:55.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.871+0000 7f735bd61700 1 --2- 172.21.15.88:0/2508188951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73541004e0 0x7f73541008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:55.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.871+0000 7f735bd61700 1 -- 172.21.15.88:0/2508188951 >> 172.21.15.88:0/2508188951 conn(0x7f73540fbda0 msgr2=0x7f73540fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:55.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.871+0000 7f735bd61700 1 -- 172.21.15.88:0/2508188951 shutdown_connections 2024-02-19T19:13:55.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.871+0000 7f735bd61700 1 -- 172.21.15.88:0/2508188951 wait complete. 2024-02-19T19:13:55.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f735bd61700 1 Processor -- start 2024-02-19T19:13:55.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f735bd61700 1 -- start start 2024-02-19T19:13:55.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f735bd61700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73541004e0 0x7f7354195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f735bd61700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7354100e80 0x7f7354196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f735bd61700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7354196aa0 con 0x7f7354100e80 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f735bd61700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7354196be0 con 0x7f73541004e0 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f73592fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7354100e80 0x7f7354196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f73592fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7354100e80 0x7f7354196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38200/0 (socket says 172.21.15.88:38200) 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f73592fc700 1 -- 172.21.15.88:0/4107589643 learned_addr learned my addr 172.21.15.88:0/4107589643 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f7359afd700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73541004e0 0x7f7354195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f73592fc700 1 -- 172.21.15.88:0/4107589643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73541004e0 msgr2=0x7f7354195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.872+0000 7f73592fc700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73541004e0 0x7f7354195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.873+0000 7f73592fc700 1 -- 172.21.15.88:0/4107589643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73440095f0 con 0x7f7354100e80 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.873+0000 7f7359afd700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73541004e0 0x7f7354195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.873+0000 7f73592fc700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7354100e80 0x7f7354196450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f734400bda0 tx=0x7f734400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.873+0000 7f734affd700 1 -- 172.21.15.88:0/4107589643 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f734401d070 con 0x7f7354100e80 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.873+0000 7f734affd700 1 -- 172.21.15.88:0/4107589643 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7344004fb0 con 0x7f7354100e80 2024-02-19T19:13:55.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.873+0000 7f734affd700 1 -- 172.21.15.88:0/4107589643 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f734400f480 con 0x7f7354100e80 2024-02-19T19:13:55.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.873+0000 7f735bd61700 1 -- 172.21.15.88:0/4107589643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735419a9d0 con 0x7f7354100e80 2024-02-19T19:13:55.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.873+0000 7f735bd61700 1 -- 172.21.15.88:0/4107589643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735419aec0 con 0x7f7354100e80 2024-02-19T19:13:55.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.874+0000 7f735bd61700 1 -- 172.21.15.88:0/4107589643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7354108850 con 0x7f7354100e80 2024-02-19T19:13:55.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.874+0000 7f734affd700 1 -- 172.21.15.88:0/4107589643 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7344022870 con 0x7f7354100e80 2024-02-19T19:13:55.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.875+0000 7f734affd700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f734006d8a0 0x7f734006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:55.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.875+0000 7f734affd700 1 -- 172.21.15.88:0/4107589643 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f734408e9a0 con 0x7f7354100e80 2024-02-19T19:13:55.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.875+0000 7f7359afd700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f734006d8a0 0x7f734006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:55.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.875+0000 7f7359afd700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f734006d8a0 0x7f734006fd50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7350005fd0 tx=0x7f7350005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:55.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:55.879+0000 7f734affd700 1 -- 172.21.15.88:0/4107589643 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f73440583a0 con 0x7f7354100e80 2024-02-19T19:13:56.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.093+0000 7f735bd61700 1 -- 172.21.15.88:0/4107589643 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f735419b130 con 0x7f734006d8a0 2024-02-19T19:13:56.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.094+0000 7f734affd700 1 -- 172.21.15.88:0/4107589643 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f735419b130 con 0x7f734006d8a0 2024-02-19T19:13:56.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.096+0000 7f735bd61700 1 -- 172.21.15.88:0/4107589643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f734006d8a0 msgr2=0x7f734006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:56.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.096+0000 7f735bd61700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f734006d8a0 0x7f734006fd50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7350005fd0 tx=0x7f7350005e20 comp rx=0 tx=0).stop 2024-02-19T19:13:56.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.096+0000 7f735bd61700 1 -- 172.21.15.88:0/4107589643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7354100e80 msgr2=0x7f7354196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:56.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.096+0000 7f735bd61700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7354100e80 0x7f7354196450 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f734400bda0 tx=0x7f734400be80 comp rx=0 tx=0).stop 2024-02-19T19:13:56.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.096+0000 7f735bd61700 1 -- 172.21.15.88:0/4107589643 shutdown_connections 2024-02-19T19:13:56.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.096+0000 7f735bd61700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f734006d8a0 0x7f734006fd50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.096+0000 7f735bd61700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73541004e0 0x7f7354195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.096+0000 7f735bd61700 1 --2- 172.21.15.88:0/4107589643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7354100e80 0x7f7354196450 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.096+0000 7f735bd61700 1 -- 172.21.15.88:0/4107589643 >> 172.21.15.88:0/4107589643 conn(0x7f73540fbda0 msgr2=0x7f73540fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:56.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.097+0000 7f735bd61700 1 -- 172.21.15.88:0/4107589643 shutdown_connections 2024-02-19T19:13:56.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.097+0000 7f735bd61700 1 -- 172.21.15.88:0/4107589643 wait complete. 2024-02-19T19:13:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.199+0000 7f4ef306a700 1 -- 172.21.15.88:0/980990476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec1004e0 msgr2=0x7f4eec1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.199+0000 7f4ef306a700 1 --2- 172.21.15.88:0/980990476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec1004e0 0x7f4eec1008b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0009910 tx=0x7f4ee0009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.199+0000 7f4ef306a700 1 -- 172.21.15.88:0/980990476 shutdown_connections 2024-02-19T19:13:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.199+0000 7f4ef306a700 1 --2- 172.21.15.88:0/980990476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec100e80 0x7f4eec104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.199+0000 7f4ef306a700 1 --2- 172.21.15.88:0/980990476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec1004e0 0x7f4eec1008b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.199+0000 7f4ef306a700 1 -- 172.21.15.88:0/980990476 >> 172.21.15.88:0/980990476 conn(0x7f4eec0fbda0 msgr2=0x7f4eec0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.199+0000 7f4ef306a700 1 -- 172.21.15.88:0/980990476 shutdown_connections 2024-02-19T19:13:56.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.199+0000 7f4ef306a700 1 -- 172.21.15.88:0/980990476 wait complete. 2024-02-19T19:13:56.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.200+0000 7f4ef306a700 1 Processor -- start 2024-02-19T19:13:56.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.200+0000 7f4ef306a700 1 -- start start 2024-02-19T19:13:56.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.200+0000 7f4ef306a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec1004e0 0x7f4eec195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:56.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.200+0000 7f4ef306a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec100e80 0x7f4eec1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:56.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.200+0000 7f4ef306a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec196bb0 con 0x7f4eec100e80 2024-02-19T19:13:56.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.200+0000 7f4ef306a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19a940 con 0x7f4eec1004e0 2024-02-19T19:13:56.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.200+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec100e80 0x7f4eec1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:56.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.200+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec100e80 0x7f4eec1964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38228/0 (socket says 172.21.15.88:38228) 2024-02-19T19:13:56.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.200+0000 7f4eebfff700 1 -- 172.21.15.88:0/2111062126 learned_addr learned my addr 172.21.15.88:0/2111062126 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:56.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.201+0000 7f4eebfff700 1 -- 172.21.15.88:0/2111062126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec1004e0 msgr2=0x7f4eec195f90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-19T19:13:56.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.201+0000 7f4eebfff700 1 --2- 172.21.15.88:0/2111062126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec1004e0 0x7f4eec195f90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.201+0000 7f4eebfff700 1 -- 172.21.15.88:0/2111062126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00095f0 con 0x7f4eec100e80 2024-02-19T19:13:56.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.201+0000 7f4eebfff700 1 --2- 172.21.15.88:0/2111062126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec100e80 0x7f4eec1964d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b4d0 tx=0x7f4ed800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:56.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.201+0000 7f4ee9ffb700 1 -- 172.21.15.88:0/2111062126 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed8011800 con 0x7f4eec100e80 2024-02-19T19:13:56.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.201+0000 7f4ef306a700 1 -- 172.21.15.88:0/2111062126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec19ac20 con 0x7f4eec100e80 2024-02-19T19:13:56.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.201+0000 7f4ee9ffb700 1 -- 172.21.15.88:0/2111062126 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ed8011e20 con 0x7f4eec100e80 2024-02-19T19:13:56.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.201+0000 7f4ee9ffb700 1 -- 172.21.15.88:0/2111062126 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed800f2e0 con 0x7f4eec100e80 2024-02-19T19:13:56.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.201+0000 7f4ef306a700 1 -- 172.21.15.88:0/2111062126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec19b170 con 0x7f4eec100e80 2024-02-19T19:13:56.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.202+0000 7f4ef306a700 1 -- 172.21.15.88:0/2111062126 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec1087f0 con 0x7f4eec100e80 2024-02-19T19:13:56.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.202+0000 7f4ee9ffb700 1 -- 172.21.15.88:0/2111062126 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4ed8010380 con 0x7f4eec100e80 2024-02-19T19:13:56.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.203+0000 7f4ee9ffb700 1 --2- 172.21.15.88:0/2111062126 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4edc06d8a0 0x7f4edc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:56.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.203+0000 7f4ef0e06700 1 --2- 172.21.15.88:0/2111062126 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4edc06d8a0 0x7f4edc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:56.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.203+0000 7f4ee9ffb700 1 -- 172.21.15.88:0/2111062126 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4ed808cfb0 con 0x7f4eec100e80 2024-02-19T19:13:56.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.203+0000 7f4ef0e06700 1 --2- 172.21.15.88:0/2111062126 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4edc06d8a0 0x7f4edc06fd50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00059d0 tx=0x7f4ee0005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:56.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.208+0000 7f4ee9ffb700 1 -- 172.21.15.88:0/2111062126 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ed80567e0 con 0x7f4eec100e80 2024-02-19T19:13:56.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.413+0000 7f4ef306a700 1 -- 172.21.15.88:0/2111062126 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4eec19b440 con 0x7f4edc06d8a0 2024-02-19T19:13:56.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.424+0000 7f4ee9ffb700 1 -- 172.21.15.88:0/2111062126 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4eec19b440 con 0x7f4edc06d8a0 2024-02-19T19:13:56.447 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:13:56.447 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (11m) 2m ago 21m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:13:56.447 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (22m) 2m ago 22m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (3m) 2m ago 20m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (7m) 2m ago 21m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (7m) 2m ago 20m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (10m) 2m ago 20m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (3m) 2m ago 15m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (3m) 2m ago 15m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (2m) 2m ago 15m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (15m) 2m ago 15m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (12m) 2m ago 23m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (12m) 2m ago 19m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (9m) 2m ago 23m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (8m) 2m ago 19m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (11m) 2m ago 21m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (11m) 2m ago 19m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:13:56.448 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (6m) 2m ago 19m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:13:56.449 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (5m) 2m ago 18m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:13:56.449 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (5m) 2m ago 18m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:13:56.449 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (4m) 2m ago 17m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:13:56.449 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (4m) 2m ago 17m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:13:56.449 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (3m) 2m ago 16m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:13:56.449 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (11m) 2m ago 20m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:13:56.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.427+0000 7f4ef306a700 1 -- 172.21.15.88:0/2111062126 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4edc06d8a0 msgr2=0x7f4edc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:56.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.427+0000 7f4ef306a700 1 --2- 172.21.15.88:0/2111062126 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4edc06d8a0 0x7f4edc06fd50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4ee00059d0 tx=0x7f4ee0005940 comp rx=0 tx=0).stop 2024-02-19T19:13:56.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.427+0000 7f4ef306a700 1 -- 172.21.15.88:0/2111062126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec100e80 msgr2=0x7f4eec1964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:56.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.427+0000 7f4ef306a700 1 --2- 172.21.15.88:0/2111062126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec100e80 0x7f4eec1964d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b4d0 tx=0x7f4ed800b890 comp rx=0 tx=0).stop 2024-02-19T19:13:56.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.427+0000 7f4ef306a700 1 -- 172.21.15.88:0/2111062126 shutdown_connections 2024-02-19T19:13:56.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.427+0000 7f4ef306a700 1 --2- 172.21.15.88:0/2111062126 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4edc06d8a0 0x7f4edc06fd50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.427+0000 7f4ef306a700 1 --2- 172.21.15.88:0/2111062126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4eec1004e0 0x7f4eec195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.427+0000 7f4ef306a700 1 --2- 172.21.15.88:0/2111062126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eec100e80 0x7f4eec1964d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.427+0000 7f4ef306a700 1 -- 172.21.15.88:0/2111062126 >> 172.21.15.88:0/2111062126 conn(0x7f4eec0fbda0 msgr2=0x7f4eec0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:56.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.427+0000 7f4ef306a700 1 -- 172.21.15.88:0/2111062126 shutdown_connections 2024-02-19T19:13:56.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.427+0000 7f4ef306a700 1 -- 172.21.15.88:0/2111062126 wait complete. 2024-02-19T19:13:56.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.529+0000 7f2fcf967700 1 -- 172.21.15.88:0/4279867251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc80ff3c0 msgr2=0x7f2fc80ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:56.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.529+0000 7f2fcf967700 1 --2- 172.21.15.88:0/4279867251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc80ff3c0 0x7f2fc80ff790 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4009910 tx=0x7f2fc4009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:56.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.529+0000 7f2fcf967700 1 -- 172.21.15.88:0/4279867251 shutdown_connections 2024-02-19T19:13:56.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.529+0000 7f2fcf967700 1 --2- 172.21.15.88:0/4279867251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc80ffd60 0x7f2fc8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.529+0000 7f2fcf967700 1 --2- 172.21.15.88:0/4279867251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc80ff3c0 0x7f2fc80ff790 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.529+0000 7f2fcf967700 1 -- 172.21.15.88:0/4279867251 >> 172.21.15.88:0/4279867251 conn(0x7f2fc80fac80 msgr2=0x7f2fc80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:56.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.529+0000 7f2fcf967700 1 -- 172.21.15.88:0/4279867251 shutdown_connections 2024-02-19T19:13:56.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.529+0000 7f2fcf967700 1 -- 172.21.15.88:0/4279867251 wait complete. 2024-02-19T19:13:56.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.529+0000 7f2fcf967700 1 Processor -- start 2024-02-19T19:13:56.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fcf967700 1 -- start start 2024-02-19T19:13:56.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fcf967700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc80ff3c0 0x7f2fc8195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:56.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fcf967700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc80ffd60 0x7f2fc81964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fcf967700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc8196b90 con 0x7f2fc80ffd60 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fcf967700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc819a920 con 0x7f2fc80ff3c0 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fccf02700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc80ffd60 0x7f2fc81964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fccf02700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc80ffd60 0x7f2fc81964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38256/0 (socket says 172.21.15.88:38256) 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fccf02700 1 -- 172.21.15.88:0/3886400610 learned_addr learned my addr 172.21.15.88:0/3886400610 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fcd703700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc80ff3c0 0x7f2fc8195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fccf02700 1 -- 172.21.15.88:0/3886400610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc80ff3c0 msgr2=0x7f2fc8195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fccf02700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc80ff3c0 0x7f2fc8195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fccf02700 1 -- 172.21.15.88:0/3886400610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc40095f0 con 0x7f2fc80ffd60 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fcd703700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc80ff3c0 0x7f2fc8195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fccf02700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc80ffd60 0x7f2fc81964b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b480 tx=0x7f2fb800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.530+0000 7f2fbe7fc700 1 -- 172.21.15.88:0/3886400610 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fb80118c0 con 0x7f2fc80ffd60 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.531+0000 7f2fbe7fc700 1 -- 172.21.15.88:0/3886400610 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2fb8010380 con 0x7f2fc80ffd60 2024-02-19T19:13:56.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.531+0000 7f2fbe7fc700 1 -- 172.21.15.88:0/3886400610 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2fb800f4f0 con 0x7f2fc80ffd60 2024-02-19T19:13:56.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.531+0000 7f2fcf967700 1 -- 172.21.15.88:0/3886400610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc819ac00 con 0x7f2fc80ffd60 2024-02-19T19:13:56.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.531+0000 7f2fcf967700 1 -- 172.21.15.88:0/3886400610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc819b150 con 0x7f2fc80ffd60 2024-02-19T19:13:56.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.532+0000 7f2fcf967700 1 -- 172.21.15.88:0/3886400610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc804f2a0 con 0x7f2fc80ffd60 2024-02-19T19:13:56.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.532+0000 7f2fbe7fc700 1 -- 172.21.15.88:0/3886400610 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2fb80104f0 con 0x7f2fc80ffd60 2024-02-19T19:13:56.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.533+0000 7f2fbe7fc700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fb406d8a0 0x7f2fb406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:56.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.533+0000 7f2fcd703700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fb406d8a0 0x7f2fb406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:56.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.533+0000 7f2fbe7fc700 1 -- 172.21.15.88:0/3886400610 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2fb808d180 con 0x7f2fc80ffd60 2024-02-19T19:13:56.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.533+0000 7f2fcd703700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fb406d8a0 0x7f2fb406fd50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4005100 tx=0x7f2fc401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:56.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.538+0000 7f2fbe7fc700 1 -- 172.21.15.88:0/3886400610 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2fb8056930 con 0x7f2fc80ffd60 2024-02-19T19:13:56.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.816+0000 7f2fcf967700 1 -- 172.21.15.88:0/3886400610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2fc819ad90 con 0x7f2fc80ffd60 2024-02-19T19:13:56.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.816+0000 7f2fbe7fc700 1 -- 172.21.15.88:0/3886400610 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f2fc819ad90 con 0x7f2fc80ffd60 2024-02-19T19:13:56.839 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:13:56.839 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:13:56.839 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:13:56.839 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:13:56.840 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:13:56.841 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:13:56.841 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:13:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.819+0000 7f2fcf967700 1 -- 172.21.15.88:0/3886400610 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fb406d8a0 msgr2=0x7f2fb406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.819+0000 7f2fcf967700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fb406d8a0 0x7f2fb406fd50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2fc4005100 tx=0x7f2fc401a040 comp rx=0 tx=0).stop 2024-02-19T19:13:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.819+0000 7f2fcf967700 1 -- 172.21.15.88:0/3886400610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc80ffd60 msgr2=0x7f2fc81964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.819+0000 7f2fcf967700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc80ffd60 0x7f2fc81964b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b480 tx=0x7f2fb800b840 comp rx=0 tx=0).stop 2024-02-19T19:13:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.819+0000 7f2fcf967700 1 -- 172.21.15.88:0/3886400610 shutdown_connections 2024-02-19T19:13:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.819+0000 7f2fcf967700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2fb406d8a0 0x7f2fb406fd50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.819+0000 7f2fcf967700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2fc80ff3c0 0x7f2fc8195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.819+0000 7f2fcf967700 1 --2- 172.21.15.88:0/3886400610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc80ffd60 0x7f2fc81964b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.819+0000 7f2fcf967700 1 -- 172.21.15.88:0/3886400610 >> 172.21.15.88:0/3886400610 conn(0x7f2fc80fac80 msgr2=0x7f2fc8103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:56.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.819+0000 7f2fcf967700 1 -- 172.21.15.88:0/3886400610 shutdown_connections 2024-02-19T19:13:56.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.819+0000 7f2fcf967700 1 -- 172.21.15.88:0/3886400610 wait complete. 2024-02-19T19:13:56.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.923+0000 7f4a4da9f700 1 -- 172.21.15.88:0/2724224284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a480687a0 msgr2=0x7f4a48068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:56.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.923+0000 7f4a4da9f700 1 --2- 172.21.15.88:0/2724224284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a480687a0 0x7f4a48068b70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f4a4000b0a0 tx=0x7f4a4000b3b0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.923+0000 7f4a4da9f700 1 -- 172.21.15.88:0/2724224284 shutdown_connections 2024-02-19T19:13:56.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.923+0000 7f4a4da9f700 1 --2- 172.21.15.88:0/2724224284 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a48069140 0x7f4a48104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.923+0000 7f4a4da9f700 1 --2- 172.21.15.88:0/2724224284 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a480687a0 0x7f4a48068b70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.923+0000 7f4a4da9f700 1 -- 172.21.15.88:0/2724224284 >> 172.21.15.88:0/2724224284 conn(0x7f4a48075130 msgr2=0x7f4a48075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:56.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.923+0000 7f4a4da9f700 1 -- 172.21.15.88:0/2724224284 shutdown_connections 2024-02-19T19:13:56.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.923+0000 7f4a4da9f700 1 -- 172.21.15.88:0/2724224284 wait complete. 2024-02-19T19:13:56.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a4da9f700 1 Processor -- start 2024-02-19T19:13:56.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a4da9f700 1 -- start start 2024-02-19T19:13:56.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a4da9f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a480687a0 0x7f4a48195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:56.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a4da9f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a48069140 0x7f4a481964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:56.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a4da9f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48196b80 con 0x7f4a480687a0 2024-02-19T19:13:56.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a4da9f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4819a910 con 0x7f4a48069140 2024-02-19T19:13:56.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a477fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a480687a0 0x7f4a48195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:56.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a477fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a480687a0 0x7f4a48195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38260/0 (socket says 172.21.15.88:38260) 2024-02-19T19:13:56.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a477fe700 1 -- 172.21.15.88:0/1093657318 learned_addr learned my addr 172.21.15.88:0/1093657318 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:56.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a46ffd700 1 --2- 172.21.15.88:0/1093657318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a48069140 0x7f4a481964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:56.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a477fe700 1 -- 172.21.15.88:0/1093657318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a48069140 msgr2=0x7f4a481964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:56.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a477fe700 1 --2- 172.21.15.88:0/1093657318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a48069140 0x7f4a481964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:56.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.924+0000 7f4a477fe700 1 -- 172.21.15.88:0/1093657318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40009d00 con 0x7f4a480687a0 2024-02-19T19:13:56.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.925+0000 7f4a477fe700 1 --2- 172.21.15.88:0/1093657318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a480687a0 0x7f4a48195f60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4a40000f80 tx=0x7f4a4000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:56.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.925+0000 7f4a44ff9700 1 -- 172.21.15.88:0/1093657318 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4000e040 con 0x7f4a480687a0 2024-02-19T19:13:56.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.925+0000 7f4a44ff9700 1 -- 172.21.15.88:0/1093657318 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a40003e60 con 0x7f4a480687a0 2024-02-19T19:13:56.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.925+0000 7f4a44ff9700 1 -- 172.21.15.88:0/1093657318 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a4001d910 con 0x7f4a480687a0 2024-02-19T19:13:56.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.925+0000 7f4a4da9f700 1 -- 172.21.15.88:0/1093657318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4819ab90 con 0x7f4a480687a0 2024-02-19T19:13:56.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.925+0000 7f4a4da9f700 1 -- 172.21.15.88:0/1093657318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4819b080 con 0x7f4a480687a0 2024-02-19T19:13:56.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.926+0000 7f4a4da9f700 1 -- 172.21.15.88:0/1093657318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a48108790 con 0x7f4a480687a0 2024-02-19T19:13:56.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.926+0000 7f4a44ff9700 1 -- 172.21.15.88:0/1093657318 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4a40004820 con 0x7f4a480687a0 2024-02-19T19:13:56.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.927+0000 7f4a44ff9700 1 --2- 172.21.15.88:0/1093657318 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a3006d8a0 0x7f4a3006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:56.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.927+0000 7f4a44ff9700 1 -- 172.21.15.88:0/1093657318 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4a4008ed70 con 0x7f4a480687a0 2024-02-19T19:13:56.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.927+0000 7f4a46ffd700 1 --2- 172.21.15.88:0/1093657318 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a3006d8a0 0x7f4a3006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:56.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.927+0000 7f4a46ffd700 1 --2- 172.21.15.88:0/1093657318 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a3006d8a0 0x7f4a3006fd50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c006fd0 tx=0x7f4a3c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:56.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:56.931+0000 7f4a44ff9700 1 -- 172.21.15.88:0/1093657318 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a400585a0 con 0x7f4a480687a0 2024-02-19T19:13:57.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.173+0000 7f4a4da9f700 1 -- 172.21.15.88:0/1093657318 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4a4804f2a0 con 0x7f4a480687a0 2024-02-19T19:13:57.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.173+0000 7f4a44ff9700 1 -- 172.21.15.88:0/1093657318 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f4a40017070 con 0x7f4a480687a0 2024-02-19T19:13:57.196 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:13:57.196 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:13:57.196 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:13:57.196 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:13:57.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:13:57.197 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:13:57.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:13:57.199 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:13:57.199 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:13:57.199 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:13:57.199 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:13:57.199 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:13:57.199 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:13:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.175+0000 7f4a4da9f700 1 -- 172.21.15.88:0/1093657318 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a3006d8a0 msgr2=0x7f4a3006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.175+0000 7f4a4da9f700 1 --2- 172.21.15.88:0/1093657318 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a3006d8a0 0x7f4a3006fd50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c006fd0 tx=0x7f4a3c008040 comp rx=0 tx=0).stop 2024-02-19T19:13:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.175+0000 7f4a4da9f700 1 -- 172.21.15.88:0/1093657318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a480687a0 msgr2=0x7f4a48195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.175+0000 7f4a4da9f700 1 --2- 172.21.15.88:0/1093657318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a480687a0 0x7f4a48195f60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4a40000f80 tx=0x7f4a4000cdd0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.175+0000 7f4a4da9f700 1 -- 172.21.15.88:0/1093657318 shutdown_connections 2024-02-19T19:13:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.175+0000 7f4a4da9f700 1 --2- 172.21.15.88:0/1093657318 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a3006d8a0 0x7f4a3006fd50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.175+0000 7f4a4da9f700 1 --2- 172.21.15.88:0/1093657318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a480687a0 0x7f4a48195f60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.175+0000 7f4a4da9f700 1 --2- 172.21.15.88:0/1093657318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a48069140 0x7f4a481964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.175+0000 7f4a4da9f700 1 -- 172.21.15.88:0/1093657318 >> 172.21.15.88:0/1093657318 conn(0x7f4a48075130 msgr2=0x7f4a480fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:57.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.175+0000 7f4a4da9f700 1 -- 172.21.15.88:0/1093657318 shutdown_connections 2024-02-19T19:13:57.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.175+0000 7f4a4da9f700 1 -- 172.21.15.88:0/1093657318 wait complete. 2024-02-19T19:13:57.201 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:13:57.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.280+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/391228714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c180ff350 msgr2=0x7f9c180ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:57.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.280+0000 7f9c1cdaf700 1 --2- 172.21.15.88:0/391228714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c180ff350 0x7f9c180ff720 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009910 tx=0x7f9c0c009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:57.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.280+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/391228714 shutdown_connections 2024-02-19T19:13:57.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.280+0000 7f9c1cdaf700 1 --2- 172.21.15.88:0/391228714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c180ffc60 0x7f9c1810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.280+0000 7f9c1cdaf700 1 --2- 172.21.15.88:0/391228714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c180ff350 0x7f9c180ff720 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.280+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/391228714 >> 172.21.15.88:0/391228714 conn(0x7f9c180748b0 msgr2=0x7f9c18074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:57.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.280+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/391228714 shutdown_connections 2024-02-19T19:13:57.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.280+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/391228714 wait complete. 2024-02-19T19:13:57.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.281+0000 7f9c1cdaf700 1 Processor -- start 2024-02-19T19:13:57.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.281+0000 7f9c1cdaf700 1 -- start start 2024-02-19T19:13:57.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.281+0000 7f9c1cdaf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c180ff350 0x7f9c18195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:57.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.281+0000 7f9c1cdaf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c180ffc60 0x7f9c181963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.281+0000 7f9c1cdaf700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c18196ac0 con 0x7f9c180ff350 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.281+0000 7f9c1cdaf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1819a850 con 0x7f9c180ffc60 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c1659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c180ff350 0x7f9c18195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c1659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c180ff350 0x7f9c18195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38280/0 (socket says 172.21.15.88:38280) 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c1659c700 1 -- 172.21.15.88:0/4209683367 learned_addr learned my addr 172.21.15.88:0/4209683367 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c15d9b700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c180ffc60 0x7f9c181963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c1659c700 1 -- 172.21.15.88:0/4209683367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c180ffc60 msgr2=0x7f9c181963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c1659c700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c180ffc60 0x7f9c181963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c1659c700 1 -- 172.21.15.88:0/4209683367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c0095f0 con 0x7f9c180ff350 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c15d9b700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c180ffc60 0x7f9c181963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c1659c700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c180ff350 0x7f9c18195ea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0092e0 tx=0x7f9c0c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c077fe700 1 -- 172.21.15.88:0/4209683367 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c0c01d070 con 0x7f9c180ff350 2024-02-19T19:13:57.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c077fe700 1 -- 172.21.15.88:0/4209683367 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c0c005dc0 con 0x7f9c180ff350 2024-02-19T19:13:57.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/4209683367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1819aad0 con 0x7f9c180ff350 2024-02-19T19:13:57.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c077fe700 1 -- 172.21.15.88:0/4209683367 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c0c017400 con 0x7f9c180ff350 2024-02-19T19:13:57.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.282+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/4209683367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1819afc0 con 0x7f9c180ff350 2024-02-19T19:13:57.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.283+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/4209683367 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c18107dd0 con 0x7f9c180ff350 2024-02-19T19:13:57.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.284+0000 7f9c077fe700 1 -- 172.21.15.88:0/4209683367 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9c0c00f460 con 0x7f9c180ff350 2024-02-19T19:13:57.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.284+0000 7f9c077fe700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c0006d8a0 0x7f9c0006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:57.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.284+0000 7f9c077fe700 1 -- 172.21.15.88:0/4209683367 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9c0c08ea30 con 0x7f9c180ff350 2024-02-19T19:13:57.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.284+0000 7f9c15d9b700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c0006d8a0 0x7f9c0006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:57.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.285+0000 7f9c15d9b700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c0006d8a0 0x7f9c0006fd50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9c08005fd0 tx=0x7f9c08005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:57.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.289+0000 7f9c077fe700 1 -- 172.21.15.88:0/4209683367 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c0c0581e0 con 0x7f9c180ff350 2024-02-19T19:13:57.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.502+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/4209683367 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c1819b1e0 con 0x7f9c0006d8a0 2024-02-19T19:13:57.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.503+0000 7f9c077fe700 1 -- 172.21.15.88:0/4209683367 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9c1819b1e0 con 0x7f9c0006d8a0 2024-02-19T19:13:57.526 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:13:57.526 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:13:57.526 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:13:57.526 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:13:57.526 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:13:57.526 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:13:57.526 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:13:57.526 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:13:57.527 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:13:57.527 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:13:57.527 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:13:57.527 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:13:57.527 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:13:57.527 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:13:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.505+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/4209683367 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c0006d8a0 msgr2=0x7f9c0006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.505+0000 7f9c1cdaf700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c0006d8a0 0x7f9c0006fd50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9c08005fd0 tx=0x7f9c08005ee0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.505+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/4209683367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c180ff350 msgr2=0x7f9c18195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.505+0000 7f9c1cdaf700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c180ff350 0x7f9c18195ea0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c0092e0 tx=0x7f9c0c00be20 comp rx=0 tx=0).stop 2024-02-19T19:13:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.505+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/4209683367 shutdown_connections 2024-02-19T19:13:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.505+0000 7f9c1cdaf700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c0006d8a0 0x7f9c0006fd50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.505+0000 7f9c1cdaf700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c180ff350 0x7f9c18195ea0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.505+0000 7f9c1cdaf700 1 --2- 172.21.15.88:0/4209683367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c180ffc60 0x7f9c181963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.505+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/4209683367 >> 172.21.15.88:0/4209683367 conn(0x7f9c180748b0 msgr2=0x7f9c18104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:57.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.505+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/4209683367 shutdown_connections 2024-02-19T19:13:57.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.505+0000 7f9c1cdaf700 1 -- 172.21.15.88:0/4209683367 wait complete. 2024-02-19T19:13:57.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.609+0000 7f7c2e091700 1 -- 172.21.15.88:0/4252822345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c280ffc60 msgr2=0x7f7c2810a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.609+0000 7f7c2e091700 1 --2- 172.21.15.88:0/4252822345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c280ffc60 0x7f7c2810a6d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009910 tx=0x7f7c10009c20 comp rx=0 tx=0).stop 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.609+0000 7f7c2e091700 1 -- 172.21.15.88:0/4252822345 shutdown_connections 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.609+0000 7f7c2e091700 1 --2- 172.21.15.88:0/4252822345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c280ffc60 0x7f7c2810a6d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.609+0000 7f7c2e091700 1 --2- 172.21.15.88:0/4252822345 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c280ff350 0x7f7c280ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.609+0000 7f7c2e091700 1 -- 172.21.15.88:0/4252822345 >> 172.21.15.88:0/4252822345 conn(0x7f7c280748b0 msgr2=0x7f7c28074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.609+0000 7f7c2e091700 1 -- 172.21.15.88:0/4252822345 shutdown_connections 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.609+0000 7f7c2e091700 1 -- 172.21.15.88:0/4252822345 wait complete. 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c2e091700 1 Processor -- start 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c2e091700 1 -- start start 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c2e091700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c280ff350 0x7f7c281980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c2e091700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c280ffc60 0x7f7c28198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:57.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c2e091700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c28198cf0 con 0x7f7c280ff350 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c2e091700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2819ca80 con 0x7f7c280ffc60 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c277fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c280ff350 0x7f7c281980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c277fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c280ff350 0x7f7c281980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38298/0 (socket says 172.21.15.88:38298) 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c277fe700 1 -- 172.21.15.88:0/449984742 learned_addr learned my addr 172.21.15.88:0/449984742 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c1ffff700 1 --2- 172.21.15.88:0/449984742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c280ffc60 0x7f7c28198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c277fe700 1 -- 172.21.15.88:0/449984742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c280ffc60 msgr2=0x7f7c28198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c277fe700 1 --2- 172.21.15.88:0/449984742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c280ffc60 0x7f7c28198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.610+0000 7f7c277fe700 1 -- 172.21.15.88:0/449984742 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c100095f0 con 0x7f7c280ff350 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.611+0000 7f7c277fe700 1 --2- 172.21.15.88:0/449984742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c280ff350 0x7f7c281980d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800b480 tx=0x7f7c1800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.611+0000 7f7c257fa700 1 -- 172.21.15.88:0/449984742 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c180118c0 con 0x7f7c280ff350 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.611+0000 7f7c257fa700 1 -- 172.21.15.88:0/449984742 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c18010380 con 0x7f7c280ff350 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.611+0000 7f7c257fa700 1 -- 172.21.15.88:0/449984742 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c1800f4f0 con 0x7f7c280ff350 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.611+0000 7f7c2e091700 1 -- 172.21.15.88:0/449984742 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2819cd60 con 0x7f7c280ff350 2024-02-19T19:13:57.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.611+0000 7f7c2e091700 1 -- 172.21.15.88:0/449984742 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2819d2b0 con 0x7f7c280ff350 2024-02-19T19:13:57.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.612+0000 7f7c2e091700 1 -- 172.21.15.88:0/449984742 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c2810a230 con 0x7f7c280ff350 2024-02-19T19:13:57.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.612+0000 7f7c257fa700 1 -- 172.21.15.88:0/449984742 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7c180104f0 con 0x7f7c280ff350 2024-02-19T19:13:57.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.613+0000 7f7c257fa700 1 --2- 172.21.15.88:0/449984742 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7c1406d8a0 0x7f7c1406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:13:57.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.613+0000 7f7c257fa700 1 -- 172.21.15.88:0/449984742 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7c1808d1a0 con 0x7f7c280ff350 2024-02-19T19:13:57.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.613+0000 7f7c1ffff700 1 --2- 172.21.15.88:0/449984742 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7c1406d8a0 0x7f7c1406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:13:57.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.613+0000 7f7c1ffff700 1 --2- 172.21.15.88:0/449984742 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7c1406d8a0 0x7f7c1406fd50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c100059d0 tx=0x7f7c1001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:13:57.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.618+0000 7f7c257fa700 1 -- 172.21.15.88:0/449984742 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7c18056950 con 0x7f7c280ff350 2024-02-19T19:13:57.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:57 smithi132 ceph-mon[70676]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:57.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:57 smithi132 ceph-mon[70676]: from='client.34614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:57.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:57 smithi132 ceph-mon[70676]: pgmap v310: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:57.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:57 smithi132 ceph-mon[70676]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:57.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:57 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3886400610' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:13:57.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:57 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1093657318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:13:57.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:57 smithi088 ceph-mon[88643]: from='client.34610 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:57.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:57 smithi088 ceph-mon[88643]: from='client.34614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:57.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:57 smithi088 ceph-mon[88643]: pgmap v310: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:57.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:57 smithi088 ceph-mon[88643]: from='client.34618 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:57.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:57 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3886400610' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:13:57.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:57 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1093657318' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:13:57.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.901+0000 7f7c2e091700 1 -- 172.21.15.88:0/449984742 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7c2819d530 con 0x7f7c280ff350 2024-02-19T19:13:57.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.902+0000 7f7c257fa700 1 -- 172.21.15.88:0/449984742 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f7c1805a010 con 0x7f7c280ff350 2024-02-19T19:13:57.925 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:13:57.925 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:13:57.925 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:13:57.925 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:13:57.925 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:13:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.904+0000 7f7c2e091700 1 -- 172.21.15.88:0/449984742 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7c1406d8a0 msgr2=0x7f7c1406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.904+0000 7f7c2e091700 1 --2- 172.21.15.88:0/449984742 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7c1406d8a0 0x7f7c1406fd50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c100059d0 tx=0x7f7c1001a040 comp rx=0 tx=0).stop 2024-02-19T19:13:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.904+0000 7f7c2e091700 1 -- 172.21.15.88:0/449984742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c280ff350 msgr2=0x7f7c281980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:13:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.904+0000 7f7c2e091700 1 --2- 172.21.15.88:0/449984742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c280ff350 0x7f7c281980d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800b480 tx=0x7f7c1800b840 comp rx=0 tx=0).stop 2024-02-19T19:13:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.904+0000 7f7c2e091700 1 -- 172.21.15.88:0/449984742 shutdown_connections 2024-02-19T19:13:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.904+0000 7f7c2e091700 1 --2- 172.21.15.88:0/449984742 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7c1406d8a0 0x7f7c1406fd50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.904+0000 7f7c2e091700 1 --2- 172.21.15.88:0/449984742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c280ff350 0x7f7c281980d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.904+0000 7f7c2e091700 1 --2- 172.21.15.88:0/449984742 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7c280ffc60 0x7f7c28198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:13:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.904+0000 7f7c2e091700 1 -- 172.21.15.88:0/449984742 >> 172.21.15.88:0/449984742 conn(0x7f7c280748b0 msgr2=0x7f7c28100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:13:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.904+0000 7f7c2e091700 1 -- 172.21.15.88:0/449984742 shutdown_connections 2024-02-19T19:13:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:13:57.904+0000 7f7c2e091700 1 -- 172.21.15.88:0/449984742 wait complete. 2024-02-19T19:13:58.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:58 smithi132 ceph-mon[70676]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:58.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:58 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/449984742' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:13:58.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:13:58.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:58 smithi088 ceph-mon[88643]: from='client.34630 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:13:58.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:58 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/449984742' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:13:58.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:13:59.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:13:59 smithi132 ceph-mon[70676]: pgmap v311: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:13:59.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:13:59 smithi088 ceph-mon[88643]: pgmap v311: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:01.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:01 smithi132 ceph-mon[70676]: pgmap v312: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:01.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:01 smithi088 ceph-mon[88643]: pgmap v312: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:03.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:03 smithi132 ceph-mon[70676]: pgmap v313: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:03.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:03 smithi088 ceph-mon[88643]: pgmap v313: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:05.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:05 smithi132 ceph-mon[70676]: pgmap v314: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:05.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:05 smithi088 ceph-mon[88643]: pgmap v314: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:07.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:07 smithi132 ceph-mon[70676]: pgmap v315: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:07.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:07 smithi088 ceph-mon[88643]: pgmap v315: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:09.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:09 smithi132 ceph-mon[70676]: pgmap v316: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:09.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:09 smithi088 ceph-mon[88643]: pgmap v316: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:11.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:11 smithi132 ceph-mon[70676]: pgmap v317: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:11.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:11 smithi088 ceph-mon[88643]: pgmap v317: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:13.757 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:13 smithi132 ceph-mon[70676]: pgmap v318: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:13.757 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:14:13.855 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:13 smithi088 ceph-mon[88643]: pgmap v318: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:13.855 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:14:15.756 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:15 smithi132 ceph-mon[70676]: pgmap v319: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:15.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:15 smithi088 ceph-mon[88643]: pgmap v319: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:17.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:17 smithi132 ceph-mon[70676]: pgmap v320: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:17.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:17 smithi088 ceph-mon[88643]: pgmap v320: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:19.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:19 smithi132 ceph-mon[70676]: pgmap v321: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:19.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:19 smithi088 ceph-mon[88643]: pgmap v321: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:21.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:21 smithi132 ceph-mon[70676]: pgmap v322: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:21.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:21 smithi088 ceph-mon[88643]: pgmap v322: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:23.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:23 smithi132 ceph-mon[70676]: pgmap v323: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:23.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:23 smithi088 ceph-mon[88643]: pgmap v323: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:25.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:25 smithi088 ceph-mon[88643]: pgmap v324: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:25.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:25 smithi132 ceph-mon[70676]: pgmap v324: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:27.755 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:27 smithi132 ceph-mon[70676]: pgmap v325: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:27.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:27 smithi088 ceph-mon[88643]: pgmap v325: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:28.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.020+0000 7f4e69418700 1 -- 172.21.15.88:0/1429637980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e640ffd60 msgr2=0x7f4e64103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.020+0000 7f4e69418700 1 --2- 172.21.15.88:0/1429637980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e640ffd60 0x7f4e64103d30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4e58009910 tx=0x7f4e58009c20 comp rx=0 tx=0).stop 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.020+0000 7f4e69418700 1 -- 172.21.15.88:0/1429637980 shutdown_connections 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.020+0000 7f4e69418700 1 --2- 172.21.15.88:0/1429637980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e640ffd60 0x7f4e64103d30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.020+0000 7f4e69418700 1 --2- 172.21.15.88:0/1429637980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e640ff3c0 0x7f4e640ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.020+0000 7f4e69418700 1 -- 172.21.15.88:0/1429637980 >> 172.21.15.88:0/1429637980 conn(0x7f4e640fac80 msgr2=0x7f4e640fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.021+0000 7f4e69418700 1 -- 172.21.15.88:0/1429637980 shutdown_connections 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.021+0000 7f4e69418700 1 -- 172.21.15.88:0/1429637980 wait complete. 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.021+0000 7f4e69418700 1 Processor -- start 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.021+0000 7f4e69418700 1 -- start start 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e69418700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e640ff3c0 0x7f4e64100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e69418700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e640ffd60 0x7f4e64101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e69418700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e64101850 con 0x7f4e640ffd60 2024-02-19T19:14:28.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e69418700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e64101990 con 0x7f4e640ff3c0 2024-02-19T19:14:28.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e627fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e640ffd60 0x7f4e64101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:28.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e627fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e640ffd60 0x7f4e64101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44956/0 (socket says 172.21.15.88:44956) 2024-02-19T19:14:28.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e627fc700 1 -- 172.21.15.88:0/1979296939 learned_addr learned my addr 172.21.15.88:0/1979296939 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:14:28.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e62ffd700 1 --2- 172.21.15.88:0/1979296939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e640ff3c0 0x7f4e64100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:28.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e627fc700 1 -- 172.21.15.88:0/1979296939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e640ff3c0 msgr2=0x7f4e64100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e627fc700 1 --2- 172.21.15.88:0/1979296939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e640ff3c0 0x7f4e64100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e627fc700 1 -- 172.21.15.88:0/1979296939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580095f0 con 0x7f4e640ffd60 2024-02-19T19:14:28.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e627fc700 1 --2- 172.21.15.88:0/1979296939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e640ffd60 0x7f4e64101310 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4e580092e0 tx=0x7f4e5800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:28.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e4bfff700 1 -- 172.21.15.88:0/1979296939 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e5801d070 con 0x7f4e640ffd60 2024-02-19T19:14:28.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e69418700 1 -- 172.21.15.88:0/1979296939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e64101c10 con 0x7f4e640ffd60 2024-02-19T19:14:28.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e4bfff700 1 -- 172.21.15.88:0/1979296939 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e58005dc0 con 0x7f4e640ffd60 2024-02-19T19:14:28.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.022+0000 7f4e4bfff700 1 -- 172.21.15.88:0/1979296939 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e58017400 con 0x7f4e640ffd60 2024-02-19T19:14:28.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.023+0000 7f4e69418700 1 -- 172.21.15.88:0/1979296939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e641a2450 con 0x7f4e640ffd60 2024-02-19T19:14:28.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.023+0000 7f4e69418700 1 -- 172.21.15.88:0/1979296939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e64108740 con 0x7f4e640ffd60 2024-02-19T19:14:28.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.024+0000 7f4e4bfff700 1 -- 172.21.15.88:0/1979296939 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4e5800f6f0 con 0x7f4e640ffd60 2024-02-19T19:14:28.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.024+0000 7f4e4bfff700 1 --2- 172.21.15.88:0/1979296939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e4c06d8a0 0x7f4e4c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:28.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.024+0000 7f4e4bfff700 1 -- 172.21.15.88:0/1979296939 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4e5808e910 con 0x7f4e640ffd60 2024-02-19T19:14:28.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.024+0000 7f4e62ffd700 1 --2- 172.21.15.88:0/1979296939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e4c06d8a0 0x7f4e4c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:28.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.025+0000 7f4e62ffd700 1 --2- 172.21.15.88:0/1979296939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e4c06d8a0 0x7f4e4c06fd50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e54005950 tx=0x7f4e5400b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:28.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.030+0000 7f4e4bfff700 1 -- 172.21.15.88:0/1979296939 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e580580c0 con 0x7f4e640ffd60 2024-02-19T19:14:28.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.247+0000 7f4e69418700 1 -- 172.21.15.88:0/1979296939 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e64102410 con 0x7f4e4c06d8a0 2024-02-19T19:14:28.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.248+0000 7f4e4bfff700 1 -- 172.21.15.88:0/1979296939 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4e64102410 con 0x7f4e4c06d8a0 2024-02-19T19:14:28.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.250+0000 7f4e69418700 1 -- 172.21.15.88:0/1979296939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e4c06d8a0 msgr2=0x7f4e4c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.250+0000 7f4e69418700 1 --2- 172.21.15.88:0/1979296939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e4c06d8a0 0x7f4e4c06fd50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4e54005950 tx=0x7f4e5400b410 comp rx=0 tx=0).stop 2024-02-19T19:14:28.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.250+0000 7f4e69418700 1 -- 172.21.15.88:0/1979296939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e640ffd60 msgr2=0x7f4e64101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.250+0000 7f4e69418700 1 --2- 172.21.15.88:0/1979296939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e640ffd60 0x7f4e64101310 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4e580092e0 tx=0x7f4e5800bd70 comp rx=0 tx=0).stop 2024-02-19T19:14:28.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.250+0000 7f4e69418700 1 -- 172.21.15.88:0/1979296939 shutdown_connections 2024-02-19T19:14:28.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.250+0000 7f4e69418700 1 --2- 172.21.15.88:0/1979296939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e4c06d8a0 0x7f4e4c06fd50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.250+0000 7f4e69418700 1 --2- 172.21.15.88:0/1979296939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e640ff3c0 0x7f4e64100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.250+0000 7f4e69418700 1 --2- 172.21.15.88:0/1979296939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e640ffd60 0x7f4e64101310 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.250+0000 7f4e69418700 1 -- 172.21.15.88:0/1979296939 >> 172.21.15.88:0/1979296939 conn(0x7f4e640fac80 msgr2=0x7f4e64069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.250+0000 7f4e69418700 1 -- 172.21.15.88:0/1979296939 shutdown_connections 2024-02-19T19:14:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.250+0000 7f4e69418700 1 -- 172.21.15.88:0/1979296939 wait complete. 2024-02-19T19:14:28.285 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:14:28.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.356+0000 7f4e6e08e700 1 -- 172.21.15.88:0/492010691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e68101420 msgr2=0x7f4e681017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.356+0000 7f4e6e08e700 1 --2- 172.21.15.88:0/492010691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e68101420 0x7f4e681017f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800b0a0 tx=0x7f4e5800b3b0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.356+0000 7f4e6e08e700 1 -- 172.21.15.88:0/492010691 shutdown_connections 2024-02-19T19:14:28.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.356+0000 7f4e6e08e700 1 --2- 172.21.15.88:0/492010691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e68101d30 0x7f4e6810a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.356+0000 7f4e6e08e700 1 --2- 172.21.15.88:0/492010691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e68101420 0x7f4e681017f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.356+0000 7f4e6e08e700 1 -- 172.21.15.88:0/492010691 >> 172.21.15.88:0/492010691 conn(0x7f4e680fac80 msgr2=0x7f4e680fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:28.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.356+0000 7f4e6e08e700 1 -- 172.21.15.88:0/492010691 shutdown_connections 2024-02-19T19:14:28.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.356+0000 7f4e6e08e700 1 -- 172.21.15.88:0/492010691 wait complete. 2024-02-19T19:14:28.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.357+0000 7f4e6e08e700 1 Processor -- start 2024-02-19T19:14:28.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.357+0000 7f4e6e08e700 1 -- start start 2024-02-19T19:14:28.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.357+0000 7f4e6e08e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e68101420 0x7f4e68195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:28.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.357+0000 7f4e6e08e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e68101d30 0x7f4e68196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:28.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.357+0000 7f4e6e08e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e68196c10 con 0x7f4e68101420 2024-02-19T19:14:28.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.357+0000 7f4e6e08e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6819a9a0 con 0x7f4e68101d30 2024-02-19T19:14:28.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.357+0000 7f4e677fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e68101420 0x7f4e68195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.357+0000 7f4e677fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e68101420 0x7f4e68195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44980/0 (socket says 172.21.15.88:44980) 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.357+0000 7f4e677fe700 1 -- 172.21.15.88:0/2400846836 learned_addr learned my addr 172.21.15.88:0/2400846836 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.357+0000 7f4e66ffd700 1 --2- 172.21.15.88:0/2400846836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e68101d30 0x7f4e68196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.358+0000 7f4e677fe700 1 -- 172.21.15.88:0/2400846836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e68101d30 msgr2=0x7f4e68196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.358+0000 7f4e677fe700 1 --2- 172.21.15.88:0/2400846836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e68101d30 0x7f4e68196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.358+0000 7f4e677fe700 1 -- 172.21.15.88:0/2400846836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e58009d00 con 0x7f4e68101420 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.358+0000 7f4e677fe700 1 --2- 172.21.15.88:0/2400846836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e68101420 0x7f4e68195ff0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000f80 tx=0x7f4e5800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.358+0000 7f4e64ff9700 1 -- 172.21.15.88:0/2400846836 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e5800e040 con 0x7f4e68101420 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.358+0000 7f4e6e08e700 1 -- 172.21.15.88:0/2400846836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6819ac20 con 0x7f4e68101420 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.358+0000 7f4e64ff9700 1 -- 172.21.15.88:0/2400846836 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e58003e60 con 0x7f4e68101420 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.358+0000 7f4e64ff9700 1 -- 172.21.15.88:0/2400846836 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e5801d910 con 0x7f4e68101420 2024-02-19T19:14:28.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.358+0000 7f4e6e08e700 1 -- 172.21.15.88:0/2400846836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6819b110 con 0x7f4e68101420 2024-02-19T19:14:28.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.359+0000 7f4e6e08e700 1 -- 172.21.15.88:0/2400846836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e681078e0 con 0x7f4e68101420 2024-02-19T19:14:28.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.360+0000 7f4e64ff9700 1 -- 172.21.15.88:0/2400846836 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4e58004820 con 0x7f4e68101420 2024-02-19T19:14:28.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.360+0000 7f4e64ff9700 1 --2- 172.21.15.88:0/2400846836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e5006d8a0 0x7f4e5006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:28.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.360+0000 7f4e64ff9700 1 -- 172.21.15.88:0/2400846836 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4e5808ee10 con 0x7f4e68101420 2024-02-19T19:14:28.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.360+0000 7f4e66ffd700 1 --2- 172.21.15.88:0/2400846836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e5006d8a0 0x7f4e5006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:28.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.360+0000 7f4e66ffd700 1 --2- 172.21.15.88:0/2400846836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e5006d8a0 0x7f4e5006fd50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c006fd0 tx=0x7f4e5c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:28.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.365+0000 7f4e64ff9700 1 -- 172.21.15.88:0/2400846836 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4e580585c0 con 0x7f4e68101420 2024-02-19T19:14:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.585+0000 7f4e6e08e700 1 -- 172.21.15.88:0/2400846836 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e6819b330 con 0x7f4e5006d8a0 2024-02-19T19:14:28.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.586+0000 7f4e64ff9700 1 -- 172.21.15.88:0/2400846836 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4e6819b330 con 0x7f4e5006d8a0 2024-02-19T19:14:28.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.589+0000 7f4e6e08e700 1 -- 172.21.15.88:0/2400846836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e5006d8a0 msgr2=0x7f4e5006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.589+0000 7f4e6e08e700 1 --2- 172.21.15.88:0/2400846836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e5006d8a0 0x7f4e5006fd50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f4e5c006fd0 tx=0x7f4e5c008040 comp rx=0 tx=0).stop 2024-02-19T19:14:28.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.589+0000 7f4e6e08e700 1 -- 172.21.15.88:0/2400846836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e68101420 msgr2=0x7f4e68195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.589+0000 7f4e6e08e700 1 --2- 172.21.15.88:0/2400846836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e68101420 0x7f4e68195ff0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000f80 tx=0x7f4e5800cdd0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.589+0000 7f4e6e08e700 1 -- 172.21.15.88:0/2400846836 shutdown_connections 2024-02-19T19:14:28.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.589+0000 7f4e6e08e700 1 --2- 172.21.15.88:0/2400846836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4e5006d8a0 0x7f4e5006fd50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.589+0000 7f4e6e08e700 1 --2- 172.21.15.88:0/2400846836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e68101420 0x7f4e68195ff0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.589+0000 7f4e6e08e700 1 --2- 172.21.15.88:0/2400846836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4e68101d30 0x7f4e68196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.589+0000 7f4e6e08e700 1 -- 172.21.15.88:0/2400846836 >> 172.21.15.88:0/2400846836 conn(0x7f4e680fac80 msgr2=0x7f4e68104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:28.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.589+0000 7f4e6e08e700 1 -- 172.21.15.88:0/2400846836 shutdown_connections 2024-02-19T19:14:28.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.589+0000 7f4e6e08e700 1 -- 172.21.15.88:0/2400846836 wait complete. 2024-02-19T19:14:28.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.693+0000 7f66e6fd2700 1 -- 172.21.15.88:0/1992042955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0101c90 msgr2=0x7f66e010a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.693+0000 7f66e6fd2700 1 --2- 172.21.15.88:0/1992042955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0101c90 0x7f66e010a2d0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f66cc009910 tx=0x7f66cc009c20 comp rx=0 tx=0).stop 2024-02-19T19:14:28.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.693+0000 7f66e6fd2700 1 -- 172.21.15.88:0/1992042955 shutdown_connections 2024-02-19T19:14:28.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.693+0000 7f66e6fd2700 1 --2- 172.21.15.88:0/1992042955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0101c90 0x7f66e010a2d0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.693+0000 7f66e6fd2700 1 --2- 172.21.15.88:0/1992042955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e01012f0 0x7f66e01016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.693+0000 7f66e6fd2700 1 -- 172.21.15.88:0/1992042955 >> 172.21.15.88:0/1992042955 conn(0x7f66e00fac80 msgr2=0x7f66e00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:28.718 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:14:28.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.693+0000 7f66e6fd2700 1 -- 172.21.15.88:0/1992042955 shutdown_connections 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.693+0000 7f66e6fd2700 1 -- 172.21.15.88:0/1992042955 wait complete. 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.694+0000 7f66e6fd2700 1 Processor -- start 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.694+0000 7f66e6fd2700 1 -- start start 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.694+0000 7f66e6fd2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e01012f0 0x7f66e0196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.694+0000 7f66e6fd2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0101c90 0x7f66e0196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.694+0000 7f66e6fd2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0196c40 con 0x7f66e0101c90 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.694+0000 7f66e6fd2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e019a9d0 con 0x7f66e01012f0 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66dffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0101c90 0x7f66e0196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66dffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0101c90 0x7f66e0196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45000/0 (socket says 172.21.15.88:45000) 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66dffff700 1 -- 172.21.15.88:0/4233621310 learned_addr learned my addr 172.21.15.88:0/4233621310 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66e4d6e700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e01012f0 0x7f66e0196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66dffff700 1 -- 172.21.15.88:0/4233621310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e01012f0 msgr2=0x7f66e0196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66dffff700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e01012f0 0x7f66e0196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66dffff700 1 -- 172.21.15.88:0/4233621310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc0095f0 con 0x7f66e0101c90 2024-02-19T19:14:28.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66e4d6e700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e01012f0 0x7f66e0196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:14:28.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66dffff700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0101c90 0x7f66e0196560 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f66cc000c00 tx=0x7f66cc0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:28.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66ddffb700 1 -- 172.21.15.88:0/4233621310 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66cc01d070 con 0x7f66e0101c90 2024-02-19T19:14:28.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66ddffb700 1 -- 172.21.15.88:0/4233621310 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66cc00f460 con 0x7f66e0101c90 2024-02-19T19:14:28.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66e6fd2700 1 -- 172.21.15.88:0/4233621310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e019ac50 con 0x7f66e0101c90 2024-02-19T19:14:28.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66ddffb700 1 -- 172.21.15.88:0/4233621310 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66cc021640 con 0x7f66e0101c90 2024-02-19T19:14:28.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.695+0000 7f66e6fd2700 1 -- 172.21.15.88:0/4233621310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e019b1a0 con 0x7f66e0101c90 2024-02-19T19:14:28.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.697+0000 7f66e6fd2700 1 -- 172.21.15.88:0/4233621310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e01079d0 con 0x7f66e0101c90 2024-02-19T19:14:28.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.697+0000 7f66ddffb700 1 -- 172.21.15.88:0/4233621310 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f66cc00f5d0 con 0x7f66e0101c90 2024-02-19T19:14:28.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.697+0000 7f66ddffb700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66d006d8a0 0x7f66d006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:28.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.698+0000 7f66ddffb700 1 -- 172.21.15.88:0/4233621310 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f66cc08e820 con 0x7f66e0101c90 2024-02-19T19:14:28.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.698+0000 7f66e4d6e700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66d006d8a0 0x7f66d006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:28.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.698+0000 7f66e4d6e700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66d006d8a0 0x7f66d006fd50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f66d40099d0 tx=0x7f66d4008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:28.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.702+0000 7f66ddffb700 1 -- 172.21.15.88:0/4233621310 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f66cc058050 con 0x7f66e0101c90 2024-02-19T19:14:28.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.909+0000 7f66e6fd2700 1 -- 172.21.15.88:0/4233621310 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f66e0197440 con 0x7f66d006d8a0 2024-02-19T19:14:28.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.919+0000 7f66ddffb700 1 -- 172.21.15.88:0/4233621310 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f66e0197440 con 0x7f66d006d8a0 2024-02-19T19:14:28.942 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:14:28.942 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (11m) 3m ago 22m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (22m) 3m ago 22m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (4m) 3m ago 20m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (7m) 3m ago 22m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (7m) 3m ago 20m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (11m) 3m ago 21m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (3m) 3m ago 15m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (3m) 3m ago 15m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (3m) 3m ago 15m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (15m) 3m ago 15m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (13m) 3m ago 23m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (12m) 3m ago 20m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (9m) 3m ago 24m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (8m) 3m ago 20m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (12m) 3m ago 22m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (12m) 3m ago 20m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:14:28.943 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (7m) 3m ago 19m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:14:28.944 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (6m) 3m ago 19m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:14:28.944 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (5m) 3m ago 18m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:14:28.944 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (5m) 3m ago 18m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:14:28.944 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (4m) 3m ago 17m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:14:28.944 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (4m) 3m ago 17m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:14:28.944 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (11m) 3m ago 20m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:14:28.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.922+0000 7f66e6fd2700 1 -- 172.21.15.88:0/4233621310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66d006d8a0 msgr2=0x7f66d006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.922+0000 7f66e6fd2700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66d006d8a0 0x7f66d006fd50 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f66d40099d0 tx=0x7f66d4008040 comp rx=0 tx=0).stop 2024-02-19T19:14:28.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.922+0000 7f66e6fd2700 1 -- 172.21.15.88:0/4233621310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0101c90 msgr2=0x7f66e0196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:28.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.922+0000 7f66e6fd2700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0101c90 0x7f66e0196560 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f66cc000c00 tx=0x7f66cc0056a0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.922+0000 7f66e6fd2700 1 -- 172.21.15.88:0/4233621310 shutdown_connections 2024-02-19T19:14:28.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.922+0000 7f66e6fd2700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66d006d8a0 0x7f66d006fd50 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.922+0000 7f66e6fd2700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e01012f0 0x7f66e0196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.922+0000 7f66e6fd2700 1 --2- 172.21.15.88:0/4233621310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0101c90 0x7f66e0196560 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:28.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.922+0000 7f66e6fd2700 1 -- 172.21.15.88:0/4233621310 >> 172.21.15.88:0/4233621310 conn(0x7f66e00fac80 msgr2=0x7f66e0104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:28.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.922+0000 7f66e6fd2700 1 -- 172.21.15.88:0/4233621310 shutdown_connections 2024-02-19T19:14:28.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:28.922+0000 7f66e6fd2700 1 -- 172.21.15.88:0/4233621310 wait complete. 2024-02-19T19:14:29.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:14:29.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.024+0000 7f1a60509700 1 -- 172.21.15.88:0/844803005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a580ff3c0 msgr2=0x7f1a580ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:29.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.024+0000 7f1a60509700 1 --2- 172.21.15.88:0/844803005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a580ff3c0 0x7f1a580ff790 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1a50009910 tx=0x7f1a50009c20 comp rx=0 tx=0).stop 2024-02-19T19:14:29.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.024+0000 7f1a60509700 1 -- 172.21.15.88:0/844803005 shutdown_connections 2024-02-19T19:14:29.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.024+0000 7f1a60509700 1 --2- 172.21.15.88:0/844803005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a580ffd60 0x7f1a58103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.024+0000 7f1a60509700 1 --2- 172.21.15.88:0/844803005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a580ff3c0 0x7f1a580ff790 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.024+0000 7f1a60509700 1 -- 172.21.15.88:0/844803005 >> 172.21.15.88:0/844803005 conn(0x7f1a580fac80 msgr2=0x7f1a580fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:29.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.024+0000 7f1a60509700 1 -- 172.21.15.88:0/844803005 shutdown_connections 2024-02-19T19:14:29.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.024+0000 7f1a60509700 1 -- 172.21.15.88:0/844803005 wait complete. 2024-02-19T19:14:29.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.025+0000 7f1a60509700 1 Processor -- start 2024-02-19T19:14:29.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.025+0000 7f1a60509700 1 -- start start 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.025+0000 7f1a60509700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a580ff3c0 0x7f1a58196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.025+0000 7f1a60509700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a580ffd60 0x7f1a58196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.025+0000 7f1a60509700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a58196c40 con 0x7f1a580ff3c0 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.025+0000 7f1a60509700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5819a9d0 con 0x7f1a580ffd60 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.025+0000 7f1a5e2a5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a580ff3c0 0x7f1a58196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.025+0000 7f1a5e2a5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a580ff3c0 0x7f1a58196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45024/0 (socket says 172.21.15.88:45024) 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.025+0000 7f1a5e2a5700 1 -- 172.21.15.88:0/1003976090 learned_addr learned my addr 172.21.15.88:0/1003976090 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.026+0000 7f1a5daa4700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a580ffd60 0x7f1a58196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.026+0000 7f1a5e2a5700 1 -- 172.21.15.88:0/1003976090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a580ffd60 msgr2=0x7f1a58196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.026+0000 7f1a5e2a5700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a580ffd60 0x7f1a58196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.026+0000 7f1a5e2a5700 1 -- 172.21.15.88:0/1003976090 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a500095f0 con 0x7f1a580ff3c0 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.026+0000 7f1a5daa4700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a580ffd60 0x7f1a58196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:14:29.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.026+0000 7f1a5e2a5700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a580ff3c0 0x7f1a58196020 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a50006010 tx=0x7f1a5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:29.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.026+0000 7f1a4f7fe700 1 -- 172.21.15.88:0/1003976090 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a5001d070 con 0x7f1a580ff3c0 2024-02-19T19:14:29.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.026+0000 7f1a4f7fe700 1 -- 172.21.15.88:0/1003976090 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a50004fb0 con 0x7f1a580ff3c0 2024-02-19T19:14:29.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.026+0000 7f1a60509700 1 -- 172.21.15.88:0/1003976090 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5819ac50 con 0x7f1a580ff3c0 2024-02-19T19:14:29.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.026+0000 7f1a4f7fe700 1 -- 172.21.15.88:0/1003976090 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a5000f690 con 0x7f1a580ff3c0 2024-02-19T19:14:29.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.026+0000 7f1a60509700 1 -- 172.21.15.88:0/1003976090 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5819b140 con 0x7f1a580ff3c0 2024-02-19T19:14:29.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.027+0000 7f1a60509700 1 -- 172.21.15.88:0/1003976090 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a5804f2a0 con 0x7f1a580ff3c0 2024-02-19T19:14:29.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.028+0000 7f1a4f7fe700 1 -- 172.21.15.88:0/1003976090 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1a500228a0 con 0x7f1a580ff3c0 2024-02-19T19:14:29.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.028+0000 7f1a4f7fe700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a4406d8a0 0x7f1a4406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:29.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.029+0000 7f1a4f7fe700 1 -- 172.21.15.88:0/1003976090 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1a5008f4b0 con 0x7f1a580ff3c0 2024-02-19T19:14:29.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.029+0000 7f1a5daa4700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a4406d8a0 0x7f1a4406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:29.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.029+0000 7f1a5daa4700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a4406d8a0 0x7f1a4406fd50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a58197640 tx=0x7f1a48008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:29.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.033+0000 7f1a4f7fe700 1 -- 172.21.15.88:0/1003976090 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1a50058ce0 con 0x7f1a580ff3c0 2024-02-19T19:14:29.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.309+0000 7f1a60509700 1 -- 172.21.15.88:0/1003976090 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1a5819b460 con 0x7f1a580ff3c0 2024-02-19T19:14:29.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.309+0000 7f1a4f7fe700 1 -- 172.21.15.88:0/1003976090 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f1a50027340 con 0x7f1a580ff3c0 2024-02-19T19:14:29.332 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:14:29.332 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:14:29.332 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:14:29.332 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:14:29.332 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:14:29.333 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:14:29.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.311+0000 7f1a60509700 1 -- 172.21.15.88:0/1003976090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a4406d8a0 msgr2=0x7f1a4406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:29.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.311+0000 7f1a60509700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a4406d8a0 0x7f1a4406fd50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a58197640 tx=0x7f1a48008040 comp rx=0 tx=0).stop 2024-02-19T19:14:29.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.311+0000 7f1a60509700 1 -- 172.21.15.88:0/1003976090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a580ff3c0 msgr2=0x7f1a58196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:29.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.311+0000 7f1a60509700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a580ff3c0 0x7f1a58196020 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1a50006010 tx=0x7f1a5000be20 comp rx=0 tx=0).stop 2024-02-19T19:14:29.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.311+0000 7f1a60509700 1 -- 172.21.15.88:0/1003976090 shutdown_connections 2024-02-19T19:14:29.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.311+0000 7f1a60509700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1a4406d8a0 0x7f1a4406fd50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.311+0000 7f1a60509700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a580ff3c0 0x7f1a58196020 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.311+0000 7f1a60509700 1 --2- 172.21.15.88:0/1003976090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1a580ffd60 0x7f1a58196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.311+0000 7f1a60509700 1 -- 172.21.15.88:0/1003976090 >> 172.21.15.88:0/1003976090 conn(0x7f1a580fac80 msgr2=0x7f1a58103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:29.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.311+0000 7f1a60509700 1 -- 172.21.15.88:0/1003976090 shutdown_connections 2024-02-19T19:14:29.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.311+0000 7f1a60509700 1 -- 172.21.15.88:0/1003976090 wait complete. 2024-02-19T19:14:29.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.415+0000 7fad903be700 1 -- 172.21.15.88:0/915522853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad880ffd60 msgr2=0x7fad88103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:29.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.415+0000 7fad903be700 1 --2- 172.21.15.88:0/915522853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad880ffd60 0x7fad88103d30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fad78009910 tx=0x7fad78009c20 comp rx=0 tx=0).stop 2024-02-19T19:14:29.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.415+0000 7fad903be700 1 -- 172.21.15.88:0/915522853 shutdown_connections 2024-02-19T19:14:29.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.415+0000 7fad903be700 1 --2- 172.21.15.88:0/915522853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad880ffd60 0x7fad88103d30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.415+0000 7fad903be700 1 --2- 172.21.15.88:0/915522853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad880ff3c0 0x7fad880ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.415+0000 7fad903be700 1 -- 172.21.15.88:0/915522853 >> 172.21.15.88:0/915522853 conn(0x7fad880fac60 msgr2=0x7fad880fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:29.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.415+0000 7fad903be700 1 -- 172.21.15.88:0/915522853 shutdown_connections 2024-02-19T19:14:29.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.415+0000 7fad903be700 1 -- 172.21.15.88:0/915522853 wait complete. 2024-02-19T19:14:29.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.415+0000 7fad903be700 1 Processor -- start 2024-02-19T19:14:29.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad903be700 1 -- start start 2024-02-19T19:14:29.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad903be700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad880ff3c0 0x7fad88196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:29.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad903be700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad880ffd60 0x7fad88197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:29.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad903be700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad881978e0 con 0x7fad880ff3c0 2024-02-19T19:14:29.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad903be700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad88197a20 con 0x7fad880ffd60 2024-02-19T19:14:29.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad8e15a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad880ff3c0 0x7fad88196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad8e15a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad880ff3c0 0x7fad88196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45044/0 (socket says 172.21.15.88:45044) 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad8e15a700 1 -- 172.21.15.88:0/1588078186 learned_addr learned my addr 172.21.15.88:0/1588078186 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad8d959700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad880ffd60 0x7fad88197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad8e15a700 1 -- 172.21.15.88:0/1588078186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad880ffd60 msgr2=0x7fad88197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad8e15a700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad880ffd60 0x7fad88197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad8e15a700 1 -- 172.21.15.88:0/1588078186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad780095f0 con 0x7fad880ff3c0 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad8d959700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad880ffd60 0x7fad88197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad8e15a700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad880ff3c0 0x7fad88196cc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fad840060b0 tx=0x7fad8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad7f7fe700 1 -- 172.21.15.88:0/1588078186 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad8400dda0 con 0x7fad880ff3c0 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad7f7fe700 1 -- 172.21.15.88:0/1588078186 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad8400f040 con 0x7fad880ff3c0 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad903be700 1 -- 172.21.15.88:0/1588078186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8819fd30 con 0x7fad880ff3c0 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.416+0000 7fad7f7fe700 1 -- 172.21.15.88:0/1588078186 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad84014790 con 0x7fad880ff3c0 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.417+0000 7fad903be700 1 -- 172.21.15.88:0/1588078186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad881a0250 con 0x7fad880ff3c0 2024-02-19T19:14:29.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.417+0000 7fad903be700 1 -- 172.21.15.88:0/1588078186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad88107730 con 0x7fad880ff3c0 2024-02-19T19:14:29.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.418+0000 7fad7f7fe700 1 -- 172.21.15.88:0/1588078186 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fad840148f0 con 0x7fad880ff3c0 2024-02-19T19:14:29.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.418+0000 7fad7f7fe700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad7406d8a0 0x7fad7406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:29.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.418+0000 7fad7f7fe700 1 -- 172.21.15.88:0/1588078186 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fad8408ced0 con 0x7fad880ff3c0 2024-02-19T19:14:29.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.418+0000 7fad8d959700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad7406d8a0 0x7fad7406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:29.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.419+0000 7fad8d959700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad7406d8a0 0x7fad7406fd50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad88069eb0 tx=0x7fad7801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:29.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.423+0000 7fad7f7fe700 1 -- 172.21.15.88:0/1588078186 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad840567b0 con 0x7fad880ff3c0 2024-02-19T19:14:29.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:29 smithi088 ceph-mon[88643]: pgmap v326: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:29.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:29 smithi088 ceph-mon[88643]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:14:29.603 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:29 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1003976090' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:14:29.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.665+0000 7fad903be700 1 -- 172.21.15.88:0/1588078186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fad88107110 con 0x7fad880ff3c0 2024-02-19T19:14:29.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.665+0000 7fad7f7fe700 1 -- 172.21.15.88:0/1588078186 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fad8401a070 con 0x7fad880ff3c0 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:14:29.689 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:14:29.690 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:14:29.691 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:14:29.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.668+0000 7fad903be700 1 -- 172.21.15.88:0/1588078186 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad7406d8a0 msgr2=0x7fad7406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:29.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.668+0000 7fad903be700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad7406d8a0 0x7fad7406fd50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fad88069eb0 tx=0x7fad7801a040 comp rx=0 tx=0).stop 2024-02-19T19:14:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.668+0000 7fad903be700 1 -- 172.21.15.88:0/1588078186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad880ff3c0 msgr2=0x7fad88196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.668+0000 7fad903be700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad880ff3c0 0x7fad88196cc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fad840060b0 tx=0x7fad8400d3b0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.668+0000 7fad903be700 1 -- 172.21.15.88:0/1588078186 shutdown_connections 2024-02-19T19:14:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.668+0000 7fad903be700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad7406d8a0 0x7fad7406fd50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.668+0000 7fad903be700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad880ff3c0 0x7fad88196cc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.668+0000 7fad903be700 1 --2- 172.21.15.88:0/1588078186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad880ffd60 0x7fad88197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.668+0000 7fad903be700 1 -- 172.21.15.88:0/1588078186 >> 172.21.15.88:0/1588078186 conn(0x7fad880fac60 msgr2=0x7fad880fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.668+0000 7fad903be700 1 -- 172.21.15.88:0/1588078186 shutdown_connections 2024-02-19T19:14:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.668+0000 7fad903be700 1 -- 172.21.15.88:0/1588078186 wait complete. 2024-02-19T19:14:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:14:29.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.773+0000 7f53680af700 1 -- 172.21.15.88:0/912732319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53600ff3c0 msgr2=0x7f53600ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:29.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.773+0000 7f53680af700 1 --2- 172.21.15.88:0/912732319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53600ff3c0 0x7f53600ff790 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f535c009910 tx=0x7f535c009c20 comp rx=0 tx=0).stop 2024-02-19T19:14:29.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.773+0000 7f53680af700 1 -- 172.21.15.88:0/912732319 shutdown_connections 2024-02-19T19:14:29.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.773+0000 7f53680af700 1 --2- 172.21.15.88:0/912732319 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53600ffd60 0x7f5360103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.773+0000 7f53680af700 1 --2- 172.21.15.88:0/912732319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53600ff3c0 0x7f53600ff790 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.773+0000 7f53680af700 1 -- 172.21.15.88:0/912732319 >> 172.21.15.88:0/912732319 conn(0x7f53600fac80 msgr2=0x7f53600fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:29.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.773+0000 7f53680af700 1 -- 172.21.15.88:0/912732319 shutdown_connections 2024-02-19T19:14:29.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.773+0000 7f53680af700 1 -- 172.21.15.88:0/912732319 wait complete. 2024-02-19T19:14:29.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.774+0000 7f53680af700 1 Processor -- start 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.774+0000 7f53680af700 1 -- start start 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.774+0000 7f53680af700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53600ff3c0 0x7f5360194e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.774+0000 7f53680af700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53600ffd60 0x7f53601953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.774+0000 7f53680af700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5360195ab0 con 0x7f53600ff3c0 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.774+0000 7f53680af700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5360199840 con 0x7f53600ffd60 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.774+0000 7f5365e4b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53600ff3c0 0x7f5360194e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.774+0000 7f5365e4b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53600ff3c0 0x7f5360194e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45070/0 (socket says 172.21.15.88:45070) 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.774+0000 7f5365e4b700 1 -- 172.21.15.88:0/1522563512 learned_addr learned my addr 172.21.15.88:0/1522563512 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.775+0000 7f5365e4b700 1 -- 172.21.15.88:0/1522563512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53600ffd60 msgr2=0x7f53601953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.775+0000 7f5365e4b700 1 --2- 172.21.15.88:0/1522563512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53600ffd60 0x7f53601953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.775+0000 7f5365e4b700 1 -- 172.21.15.88:0/1522563512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535c0095f0 con 0x7f53600ff3c0 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.775+0000 7f5365e4b700 1 --2- 172.21.15.88:0/1522563512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53600ff3c0 0x7f5360194e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f535c006010 tx=0x7f535c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.775+0000 7f5356ffd700 1 -- 172.21.15.88:0/1522563512 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f535c01d070 con 0x7f53600ff3c0 2024-02-19T19:14:29.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.775+0000 7f5356ffd700 1 -- 172.21.15.88:0/1522563512 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f535c004fb0 con 0x7f53600ff3c0 2024-02-19T19:14:29.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.775+0000 7f53680af700 1 -- 172.21.15.88:0/1522563512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5360199ac0 con 0x7f53600ff3c0 2024-02-19T19:14:29.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.775+0000 7f5356ffd700 1 -- 172.21.15.88:0/1522563512 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f535c022ea0 con 0x7f53600ff3c0 2024-02-19T19:14:29.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.775+0000 7f53680af700 1 -- 172.21.15.88:0/1522563512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5360199fb0 con 0x7f53600ff3c0 2024-02-19T19:14:29.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.776+0000 7f53680af700 1 -- 172.21.15.88:0/1522563512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53601076d0 con 0x7f53600ff3c0 2024-02-19T19:14:29.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.776+0000 7f5356ffd700 1 -- 172.21.15.88:0/1522563512 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f535c022470 con 0x7f53600ff3c0 2024-02-19T19:14:29.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.777+0000 7f5356ffd700 1 --2- 172.21.15.88:0/1522563512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f534c06d8a0 0x7f534c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:29.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.777+0000 7f5356ffd700 1 -- 172.21.15.88:0/1522563512 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f535c08ebb0 con 0x7f53600ff3c0 2024-02-19T19:14:29.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.777+0000 7f536564a700 1 --2- 172.21.15.88:0/1522563512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f534c06d8a0 0x7f534c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:29.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.777+0000 7f536564a700 1 --2- 172.21.15.88:0/1522563512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f534c06d8a0 0x7f534c06fd50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53601964b0 tx=0x7f5350006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:29.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.782+0000 7f5356ffd700 1 -- 172.21.15.88:0/1522563512 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f535c095080 con 0x7f53600ff3c0 2024-02-19T19:14:30.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:29 smithi132 ceph-mon[70676]: pgmap v326: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:30.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:29 smithi132 ceph-mon[70676]: from='client.34638 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:14:30.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:29 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1003976090' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:14:30.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.996+0000 7f53680af700 1 -- 172.21.15.88:0/1522563512 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f536002cca0 con 0x7f534c06d8a0 2024-02-19T19:14:30.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:29.997+0000 7f5356ffd700 1 -- 172.21.15.88:0/1522563512 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f536002cca0 con 0x7f534c06d8a0 2024-02-19T19:14:30.020 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:14:30.020 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:14:30.021 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:14:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.000+0000 7f53680af700 1 -- 172.21.15.88:0/1522563512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f534c06d8a0 msgr2=0x7f534c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.000+0000 7f53680af700 1 --2- 172.21.15.88:0/1522563512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f534c06d8a0 0x7f534c06fd50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f53601964b0 tx=0x7f5350006d20 comp rx=0 tx=0).stop 2024-02-19T19:14:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.000+0000 7f53680af700 1 -- 172.21.15.88:0/1522563512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53600ff3c0 msgr2=0x7f5360194e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.000+0000 7f53680af700 1 --2- 172.21.15.88:0/1522563512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53600ff3c0 0x7f5360194e90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f535c006010 tx=0x7f535c00be20 comp rx=0 tx=0).stop 2024-02-19T19:14:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.000+0000 7f53680af700 1 -- 172.21.15.88:0/1522563512 shutdown_connections 2024-02-19T19:14:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.000+0000 7f53680af700 1 --2- 172.21.15.88:0/1522563512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f534c06d8a0 0x7f534c06fd50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.000+0000 7f53680af700 1 --2- 172.21.15.88:0/1522563512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53600ff3c0 0x7f5360194e90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.000+0000 7f53680af700 1 --2- 172.21.15.88:0/1522563512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53600ffd60 0x7f53601953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.000+0000 7f53680af700 1 -- 172.21.15.88:0/1522563512 >> 172.21.15.88:0/1522563512 conn(0x7f53600fac80 msgr2=0x7f53600fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.000+0000 7f53680af700 1 -- 172.21.15.88:0/1522563512 shutdown_connections 2024-02-19T19:14:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.000+0000 7f53680af700 1 -- 172.21.15.88:0/1522563512 wait complete. 2024-02-19T19:14:30.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.103+0000 7f21ccd4c700 1 -- 172.21.15.88:0/2310407699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c80ffd60 msgr2=0x7f21c8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:30.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.103+0000 7f21ccd4c700 1 --2- 172.21.15.88:0/2310407699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c80ffd60 0x7f21c8103d30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009910 tx=0x7f21bc009c20 comp rx=0 tx=0).stop 2024-02-19T19:14:30.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.103+0000 7f21ccd4c700 1 -- 172.21.15.88:0/2310407699 shutdown_connections 2024-02-19T19:14:30.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.103+0000 7f21ccd4c700 1 --2- 172.21.15.88:0/2310407699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c80ffd60 0x7f21c8103d30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:30.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.103+0000 7f21ccd4c700 1 --2- 172.21.15.88:0/2310407699 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c80ff3c0 0x7f21c80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:30.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.103+0000 7f21ccd4c700 1 -- 172.21.15.88:0/2310407699 >> 172.21.15.88:0/2310407699 conn(0x7f21c80fac60 msgr2=0x7f21c80fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:30.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.103+0000 7f21ccd4c700 1 -- 172.21.15.88:0/2310407699 shutdown_connections 2024-02-19T19:14:30.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.103+0000 7f21ccd4c700 1 -- 172.21.15.88:0/2310407699 wait complete. 2024-02-19T19:14:30.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21ccd4c700 1 Processor -- start 2024-02-19T19:14:30.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21ccd4c700 1 -- start start 2024-02-19T19:14:30.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21ccd4c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c80ff3c0 0x7f21c8100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:30.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21ccd4c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c80ffd60 0x7f21c8101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:30.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21ccd4c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8101850 con 0x7f21c80ff3c0 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21ccd4c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c8101990 con 0x7f21c80ffd60 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21c659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c80ff3c0 0x7f21c8100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21c659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c80ff3c0 0x7f21c8100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45084/0 (socket says 172.21.15.88:45084) 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21c659c700 1 -- 172.21.15.88:0/1058971020 learned_addr learned my addr 172.21.15.88:0/1058971020 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21c5d9b700 1 --2- 172.21.15.88:0/1058971020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c80ffd60 0x7f21c8101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21c659c700 1 -- 172.21.15.88:0/1058971020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c80ffd60 msgr2=0x7f21c8101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21c659c700 1 --2- 172.21.15.88:0/1058971020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c80ffd60 0x7f21c8101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21c659c700 1 -- 172.21.15.88:0/1058971020 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc0095f0 con 0x7f21c80ff3c0 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.104+0000 7f21c659c700 1 --2- 172.21.15.88:0/1058971020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c80ff3c0 0x7f21c8100dd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f21b80060b0 tx=0x7f21b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.105+0000 7f21b77fe700 1 -- 172.21.15.88:0/1058971020 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b800dda0 con 0x7f21c80ff3c0 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.105+0000 7f21ccd4c700 1 -- 172.21.15.88:0/1058971020 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c81a2090 con 0x7f21c80ff3c0 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.105+0000 7f21b77fe700 1 -- 172.21.15.88:0/1058971020 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21b800f040 con 0x7f21c80ff3c0 2024-02-19T19:14:30.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.105+0000 7f21b77fe700 1 -- 172.21.15.88:0/1058971020 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b80146f0 con 0x7f21c80ff3c0 2024-02-19T19:14:30.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.105+0000 7f21ccd4c700 1 -- 172.21.15.88:0/1058971020 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c81a24a0 con 0x7f21c80ff3c0 2024-02-19T19:14:30.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.106+0000 7f21ccd4c700 1 -- 172.21.15.88:0/1058971020 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c804f2a0 con 0x7f21c80ff3c0 2024-02-19T19:14:30.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.106+0000 7f21b77fe700 1 -- 172.21.15.88:0/1058971020 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f21b8015400 con 0x7f21c80ff3c0 2024-02-19T19:14:30.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.106+0000 7f21b77fe700 1 --2- 172.21.15.88:0/1058971020 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21b006d8a0 0x7f21b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:14:30.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.107+0000 7f21b77fe700 1 -- 172.21.15.88:0/1058971020 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f21b808ce40 con 0x7f21c80ff3c0 2024-02-19T19:14:30.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.107+0000 7f21c5d9b700 1 --2- 172.21.15.88:0/1058971020 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21b006d8a0 0x7f21b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:14:30.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.107+0000 7f21c5d9b700 1 --2- 172.21.15.88:0/1058971020 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21b006d8a0 0x7f21b006fd50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f21c81025c0 tx=0x7f21bc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:14:30.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.111+0000 7f21b77fe700 1 -- 172.21.15.88:0/1058971020 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f21b8056720 con 0x7f21c80ff3c0 2024-02-19T19:14:30.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.395+0000 7f21ccd4c700 1 -- 172.21.15.88:0/1058971020 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f21c81a2800 con 0x7f21c80ff3c0 2024-02-19T19:14:30.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.395+0000 7f21b77fe700 1 -- 172.21.15.88:0/1058971020 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f21b801a100 con 0x7f21c80ff3c0 2024-02-19T19:14:30.418 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:14:30.419 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:14:30.419 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:14:30.419 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:14:30.419 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:14:30.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.398+0000 7f21ccd4c700 1 -- 172.21.15.88:0/1058971020 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21b006d8a0 msgr2=0x7f21b006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:30.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.398+0000 7f21ccd4c700 1 --2- 172.21.15.88:0/1058971020 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21b006d8a0 0x7f21b006fd50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f21c81025c0 tx=0x7f21bc01a040 comp rx=0 tx=0).stop 2024-02-19T19:14:30.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.398+0000 7f21ccd4c700 1 -- 172.21.15.88:0/1058971020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c80ff3c0 msgr2=0x7f21c8100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:14:30.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.398+0000 7f21ccd4c700 1 --2- 172.21.15.88:0/1058971020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c80ff3c0 0x7f21c8100dd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f21b80060b0 tx=0x7f21b800d3b0 comp rx=0 tx=0).stop 2024-02-19T19:14:30.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.398+0000 7f21ccd4c700 1 -- 172.21.15.88:0/1058971020 shutdown_connections 2024-02-19T19:14:30.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.398+0000 7f21ccd4c700 1 --2- 172.21.15.88:0/1058971020 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21b006d8a0 0x7f21b006fd50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:30.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.398+0000 7f21ccd4c700 1 --2- 172.21.15.88:0/1058971020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c80ff3c0 0x7f21c8100dd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:30.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.398+0000 7f21ccd4c700 1 --2- 172.21.15.88:0/1058971020 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21c80ffd60 0x7f21c8101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:14:30.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.398+0000 7f21ccd4c700 1 -- 172.21.15.88:0/1058971020 >> 172.21.15.88:0/1058971020 conn(0x7f21c80fac60 msgr2=0x7f21c80fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:14:30.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.398+0000 7f21ccd4c700 1 -- 172.21.15.88:0/1058971020 shutdown_connections 2024-02-19T19:14:30.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:14:30.398+0000 7f21ccd4c700 1 -- 172.21.15.88:0/1058971020 wait complete. 2024-02-19T19:14:30.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:30 smithi088 ceph-mon[88643]: from='client.34642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:14:30.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:30 smithi088 ceph-mon[88643]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:14:30.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:30 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1588078186' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:14:30.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:30 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1058971020' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:14:31.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:30 smithi132 ceph-mon[70676]: from='client.34642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:14:31.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:30 smithi132 ceph-mon[70676]: from='client.34646 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:14:31.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:30 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1588078186' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:14:31.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:30 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1058971020' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:14:31.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:31 smithi088 ceph-mon[88643]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:14:31.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:31 smithi088 ceph-mon[88643]: pgmap v327: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:32.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:31 smithi132 ceph-mon[70676]: from='client.34658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:14:32.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:31 smithi132 ceph-mon[70676]: pgmap v327: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:33.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:33 smithi088 ceph-mon[88643]: pgmap v328: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:34.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:33 smithi132 ceph-mon[70676]: pgmap v328: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:35.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:35 smithi088 ceph-mon[88643]: pgmap v329: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:36.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:35 smithi132 ceph-mon[70676]: pgmap v329: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:37.853 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:37 smithi088 ceph-mon[88643]: pgmap v330: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:38.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:37 smithi132 ceph-mon[70676]: pgmap v330: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:39.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:39 smithi088 ceph-mon[88643]: pgmap v331: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:40.005 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:39 smithi132 ceph-mon[70676]: pgmap v331: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:41.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:41 smithi088 ceph-mon[88643]: pgmap v332: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:42.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:41 smithi132 ceph-mon[70676]: pgmap v332: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:43.814 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:43 smithi132 ceph-mon[70676]: pgmap v333: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:43.814 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:14:43.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:43 smithi088 ceph-mon[88643]: pgmap v333: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:43.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:14:45.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:45 smithi088 ceph-mon[88643]: pgmap v334: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:46.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:45 smithi132 ceph-mon[70676]: pgmap v334: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:47.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:47 smithi088 ceph-mon[88643]: pgmap v335: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:48.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:47 smithi132 ceph-mon[70676]: pgmap v335: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:49.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:49 smithi088 ceph-mon[88643]: pgmap v336: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:50.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:49 smithi132 ceph-mon[70676]: pgmap v336: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:51.807 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:51 smithi132 ceph-mon[70676]: pgmap v337: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:51.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:51 smithi088 ceph-mon[88643]: pgmap v337: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:53.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:53 smithi088 ceph-mon[88643]: pgmap v338: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:53.920 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:53 smithi132 ceph-mon[70676]: pgmap v338: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:55.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:55 smithi088 ceph-mon[88643]: pgmap v339: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:56.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:55 smithi132 ceph-mon[70676]: pgmap v339: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:57.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:57 smithi088 ceph-mon[88643]: pgmap v340: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:58.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:57 smithi132 ceph-mon[70676]: pgmap v340: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:14:58.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:14:58.974 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:14:59.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:14:59 smithi088 ceph-mon[88643]: pgmap v341: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:00.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:14:59 smithi132 ceph-mon[70676]: pgmap v341: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:00.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.516+0000 7f21f3cd1700 1 -- 172.21.15.88:0/455388293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec0ff350 msgr2=0x7f21ec0ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.516+0000 7f21f3cd1700 1 --2- 172.21.15.88:0/455388293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec0ff350 0x7f21ec0ff720 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f21e8009910 tx=0x7f21e8009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.516+0000 7f21f3cd1700 1 -- 172.21.15.88:0/455388293 shutdown_connections 2024-02-19T19:15:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.516+0000 7f21f3cd1700 1 --2- 172.21.15.88:0/455388293 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21ec0ffc60 0x7f21ec10a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.516+0000 7f21f3cd1700 1 --2- 172.21.15.88:0/455388293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec0ff350 0x7f21ec0ff720 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.516+0000 7f21f3cd1700 1 -- 172.21.15.88:0/455388293 >> 172.21.15.88:0/455388293 conn(0x7f21ec0748b0 msgr2=0x7f21ec074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.516+0000 7f21f3cd1700 1 -- 172.21.15.88:0/455388293 shutdown_connections 2024-02-19T19:15:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.516+0000 7f21f3cd1700 1 -- 172.21.15.88:0/455388293 wait complete. 2024-02-19T19:15:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f3cd1700 1 Processor -- start 2024-02-19T19:15:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f3cd1700 1 -- start start 2024-02-19T19:15:00.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f3cd1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec0ff350 0x7f21ec100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f3cd1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21ec0ffc60 0x7f21ec100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f3cd1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec100fb0 con 0x7f21ec0ff350 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f3cd1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ec1010f0 con 0x7f21ec0ffc60 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f1a6d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec0ff350 0x7f21ec100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f1a6d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec0ff350 0x7f21ec100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46474/0 (socket says 172.21.15.88:46474) 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f1a6d700 1 -- 172.21.15.88:0/2888151944 learned_addr learned my addr 172.21.15.88:0/2888151944 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f126c700 1 --2- 172.21.15.88:0/2888151944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21ec0ffc60 0x7f21ec100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f1a6d700 1 -- 172.21.15.88:0/2888151944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21ec0ffc60 msgr2=0x7f21ec100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f1a6d700 1 --2- 172.21.15.88:0/2888151944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21ec0ffc60 0x7f21ec100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.517+0000 7f21f1a6d700 1 -- 172.21.15.88:0/2888151944 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80095f0 con 0x7f21ec0ff350 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.518+0000 7f21f1a6d700 1 --2- 172.21.15.88:0/2888151944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec0ff350 0x7f21ec100530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f21e80092e0 tx=0x7f21e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.518+0000 7f21e2ffd700 1 -- 172.21.15.88:0/2888151944 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21e801d070 con 0x7f21ec0ff350 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.518+0000 7f21e2ffd700 1 -- 172.21.15.88:0/2888151944 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21e8005dc0 con 0x7f21ec0ff350 2024-02-19T19:15:00.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.518+0000 7f21e2ffd700 1 -- 172.21.15.88:0/2888151944 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21e8017400 con 0x7f21ec0ff350 2024-02-19T19:15:00.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.518+0000 7f21f3cd1700 1 -- 172.21.15.88:0/2888151944 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ec101370 con 0x7f21ec0ff350 2024-02-19T19:15:00.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.518+0000 7f21f3cd1700 1 -- 172.21.15.88:0/2888151944 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ec1a23a0 con 0x7f21ec0ff350 2024-02-19T19:15:00.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.519+0000 7f21f3cd1700 1 -- 172.21.15.88:0/2888151944 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ec107dd0 con 0x7f21ec0ff350 2024-02-19T19:15:00.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.519+0000 7f21e2ffd700 1 -- 172.21.15.88:0/2888151944 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f21e800f460 con 0x7f21ec0ff350 2024-02-19T19:15:00.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.520+0000 7f21e2ffd700 1 --2- 172.21.15.88:0/2888151944 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21d806d8a0 0x7f21d806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:00.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.520+0000 7f21e2ffd700 1 -- 172.21.15.88:0/2888151944 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f21e808e950 con 0x7f21ec0ff350 2024-02-19T19:15:00.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.520+0000 7f21f126c700 1 --2- 172.21.15.88:0/2888151944 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21d806d8a0 0x7f21d806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:00.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.520+0000 7f21f126c700 1 --2- 172.21.15.88:0/2888151944 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21d806d8a0 0x7f21d806fd50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005950 tx=0x7f21dc0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:00.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.525+0000 7f21e2ffd700 1 -- 172.21.15.88:0/2888151944 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f21e8058180 con 0x7f21ec0ff350 2024-02-19T19:15:00.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.752+0000 7f21f3cd1700 1 -- 172.21.15.88:0/2888151944 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21ec061190 con 0x7f21d806d8a0 2024-02-19T19:15:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.753+0000 7f21e2ffd700 1 -- 172.21.15.88:0/2888151944 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f21ec061190 con 0x7f21d806d8a0 2024-02-19T19:15:00.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.755+0000 7f21f3cd1700 1 -- 172.21.15.88:0/2888151944 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21d806d8a0 msgr2=0x7f21d806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:00.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.756+0000 7f21f3cd1700 1 --2- 172.21.15.88:0/2888151944 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21d806d8a0 0x7f21d806fd50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21dc005950 tx=0x7f21dc0058e0 comp rx=0 tx=0).stop 2024-02-19T19:15:00.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.756+0000 7f21f3cd1700 1 -- 172.21.15.88:0/2888151944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec0ff350 msgr2=0x7f21ec100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:00.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.756+0000 7f21f3cd1700 1 --2- 172.21.15.88:0/2888151944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec0ff350 0x7f21ec100530 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f21e80092e0 tx=0x7f21e800be20 comp rx=0 tx=0).stop 2024-02-19T19:15:00.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.756+0000 7f21f3cd1700 1 -- 172.21.15.88:0/2888151944 shutdown_connections 2024-02-19T19:15:00.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.756+0000 7f21f3cd1700 1 --2- 172.21.15.88:0/2888151944 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f21d806d8a0 0x7f21d806fd50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:00.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.756+0000 7f21f3cd1700 1 --2- 172.21.15.88:0/2888151944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21ec0ff350 0x7f21ec100530 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:00.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.756+0000 7f21f3cd1700 1 --2- 172.21.15.88:0/2888151944 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f21ec0ffc60 0x7f21ec100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:00.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.756+0000 7f21f3cd1700 1 -- 172.21.15.88:0/2888151944 >> 172.21.15.88:0/2888151944 conn(0x7f21ec0748b0 msgr2=0x7f21ec104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:00.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.756+0000 7f21f3cd1700 1 -- 172.21.15.88:0/2888151944 shutdown_connections 2024-02-19T19:15:00.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.756+0000 7f21f3cd1700 1 -- 172.21.15.88:0/2888151944 wait complete. 2024-02-19T19:15:00.788 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:15:00.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.861+0000 7f0a48280700 1 -- 172.21.15.88:0/591064174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a401004e0 msgr2=0x7f0a401008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:00.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.861+0000 7f0a48280700 1 --2- 172.21.15.88:0/591064174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a401004e0 0x7f0a401008b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009910 tx=0x7f0a38009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:00.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.861+0000 7f0a48280700 1 -- 172.21.15.88:0/591064174 shutdown_connections 2024-02-19T19:15:00.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.861+0000 7f0a48280700 1 --2- 172.21.15.88:0/591064174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a40100e80 0x7f0a40104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.861+0000 7f0a48280700 1 --2- 172.21.15.88:0/591064174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a401004e0 0x7f0a401008b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.861+0000 7f0a48280700 1 -- 172.21.15.88:0/591064174 >> 172.21.15.88:0/591064174 conn(0x7f0a400fbda0 msgr2=0x7f0a400fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.861+0000 7f0a48280700 1 -- 172.21.15.88:0/591064174 shutdown_connections 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.861+0000 7f0a48280700 1 -- 172.21.15.88:0/591064174 wait complete. 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a48280700 1 Processor -- start 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a48280700 1 -- start start 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a48280700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a401004e0 0x7f0a401981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a48280700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a40100e80 0x7f0a401986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a48280700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a40198dc0 con 0x7f0a40100e80 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a48280700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4019cb50 con 0x7f0a401004e0 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a4581b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a40100e80 0x7f0a401986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a4581b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a40100e80 0x7f0a401986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46488/0 (socket says 172.21.15.88:46488) 2024-02-19T19:15:00.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a4581b700 1 -- 172.21.15.88:0/3887679841 learned_addr learned my addr 172.21.15.88:0/3887679841 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a4601c700 1 --2- 172.21.15.88:0/3887679841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a401004e0 0x7f0a401981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a4581b700 1 -- 172.21.15.88:0/3887679841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a401004e0 msgr2=0x7f0a401981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a4581b700 1 --2- 172.21.15.88:0/3887679841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a401004e0 0x7f0a401981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a4581b700 1 -- 172.21.15.88:0/3887679841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a380095f0 con 0x7f0a40100e80 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.862+0000 7f0a4581b700 1 --2- 172.21.15.88:0/3887679841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a40100e80 0x7f0a401986e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000b480 tx=0x7f0a3000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.863+0000 7f0a377fe700 1 -- 172.21.15.88:0/3887679841 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a300118c0 con 0x7f0a40100e80 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.863+0000 7f0a48280700 1 -- 172.21.15.88:0/3887679841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4019ce30 con 0x7f0a40100e80 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.863+0000 7f0a377fe700 1 -- 172.21.15.88:0/3887679841 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a30010380 con 0x7f0a40100e80 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.863+0000 7f0a377fe700 1 -- 172.21.15.88:0/3887679841 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a3000f4f0 con 0x7f0a40100e80 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.863+0000 7f0a48280700 1 -- 172.21.15.88:0/3887679841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4019d380 con 0x7f0a40100e80 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.864+0000 7f0a48280700 1 -- 172.21.15.88:0/3887679841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a401045d0 con 0x7f0a40100e80 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.864+0000 7f0a377fe700 1 -- 172.21.15.88:0/3887679841 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0a300104f0 con 0x7f0a40100e80 2024-02-19T19:15:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.864+0000 7f0a377fe700 1 --2- 172.21.15.88:0/3887679841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a2c06d8a0 0x7f0a2c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:00.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.865+0000 7f0a377fe700 1 -- 172.21.15.88:0/3887679841 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0a3008d180 con 0x7f0a40100e80 2024-02-19T19:15:00.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.865+0000 7f0a4601c700 1 --2- 172.21.15.88:0/3887679841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a2c06d8a0 0x7f0a2c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:00.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.865+0000 7f0a4601c700 1 --2- 172.21.15.88:0/3887679841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a2c06d8a0 0x7f0a2c06fd50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a38006010 tx=0x7f0a3801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:00.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:00.869+0000 7f0a377fe700 1 -- 172.21.15.88:0/3887679841 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0a30056930 con 0x7f0a40100e80 2024-02-19T19:15:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.086+0000 7f0a48280700 1 -- 172.21.15.88:0/3887679841 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a4019d670 con 0x7f0a2c06d8a0 2024-02-19T19:15:01.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.088+0000 7f0a377fe700 1 -- 172.21.15.88:0/3887679841 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0a4019d670 con 0x7f0a2c06d8a0 2024-02-19T19:15:01.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.090+0000 7f0a48280700 1 -- 172.21.15.88:0/3887679841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a2c06d8a0 msgr2=0x7f0a2c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.090+0000 7f0a48280700 1 --2- 172.21.15.88:0/3887679841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a2c06d8a0 0x7f0a2c06fd50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0a38006010 tx=0x7f0a3801a040 comp rx=0 tx=0).stop 2024-02-19T19:15:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.090+0000 7f0a48280700 1 -- 172.21.15.88:0/3887679841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a40100e80 msgr2=0x7f0a401986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.090+0000 7f0a48280700 1 --2- 172.21.15.88:0/3887679841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a40100e80 0x7f0a401986e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0a3000b480 tx=0x7f0a3000b840 comp rx=0 tx=0).stop 2024-02-19T19:15:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.090+0000 7f0a48280700 1 -- 172.21.15.88:0/3887679841 shutdown_connections 2024-02-19T19:15:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.090+0000 7f0a48280700 1 --2- 172.21.15.88:0/3887679841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a2c06d8a0 0x7f0a2c06fd50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.090+0000 7f0a48280700 1 --2- 172.21.15.88:0/3887679841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a401004e0 0x7f0a401981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.090+0000 7f0a48280700 1 --2- 172.21.15.88:0/3887679841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a40100e80 0x7f0a401986e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.090+0000 7f0a48280700 1 -- 172.21.15.88:0/3887679841 >> 172.21.15.88:0/3887679841 conn(0x7f0a400fbda0 msgr2=0x7f0a400fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.090+0000 7f0a48280700 1 -- 172.21.15.88:0/3887679841 shutdown_connections 2024-02-19T19:15:01.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.090+0000 7f0a48280700 1 -- 172.21.15.88:0/3887679841 wait complete. 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.194+0000 7f69e2c17700 1 -- 172.21.15.88:0/4232755952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc1004e0 msgr2=0x7f69dc1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.194+0000 7f69e2c17700 1 --2- 172.21.15.88:0/4232755952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc1004e0 0x7f69dc1008b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f69d0009910 tx=0x7f69d0009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.194+0000 7f69e2c17700 1 -- 172.21.15.88:0/4232755952 shutdown_connections 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.194+0000 7f69e2c17700 1 --2- 172.21.15.88:0/4232755952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69dc100e80 0x7f69dc104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.194+0000 7f69e2c17700 1 --2- 172.21.15.88:0/4232755952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc1004e0 0x7f69dc1008b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.194+0000 7f69e2c17700 1 -- 172.21.15.88:0/4232755952 >> 172.21.15.88:0/4232755952 conn(0x7f69dc0fbda0 msgr2=0x7f69dc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.194+0000 7f69e2c17700 1 -- 172.21.15.88:0/4232755952 shutdown_connections 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.194+0000 7f69e2c17700 1 -- 172.21.15.88:0/4232755952 wait complete. 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.195+0000 7f69e2c17700 1 Processor -- start 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.195+0000 7f69e2c17700 1 -- start start 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.195+0000 7f69e2c17700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69dc1004e0 0x7f69dc195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:01.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.195+0000 7f69e2c17700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc100e80 0x7f69dc196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.195+0000 7f69e2c17700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc196a80 con 0x7f69dc100e80 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.195+0000 7f69e2c17700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69dc196bc0 con 0x7f69dc1004e0 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.195+0000 7f69dbfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc100e80 0x7f69dc196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.195+0000 7f69dbfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc100e80 0x7f69dc196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46504/0 (socket says 172.21.15.88:46504) 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.195+0000 7f69dbfff700 1 -- 172.21.15.88:0/3472671405 learned_addr learned my addr 172.21.15.88:0/3472671405 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.195+0000 7f69e09b3700 1 --2- 172.21.15.88:0/3472671405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69dc1004e0 0x7f69dc195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.196+0000 7f69dbfff700 1 -- 172.21.15.88:0/3472671405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69dc1004e0 msgr2=0x7f69dc195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.196+0000 7f69dbfff700 1 --2- 172.21.15.88:0/3472671405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69dc1004e0 0x7f69dc195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.196+0000 7f69dbfff700 1 -- 172.21.15.88:0/3472671405 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d00095f0 con 0x7f69dc100e80 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.196+0000 7f69dbfff700 1 --2- 172.21.15.88:0/3472671405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc100e80 0x7f69dc196430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f69c800c960 tx=0x7f69c800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.196+0000 7f69d9ffb700 1 -- 172.21.15.88:0/3472671405 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69c8007a70 con 0x7f69dc100e80 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.196+0000 7f69d9ffb700 1 -- 172.21.15.88:0/3472671405 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69c8007bd0 con 0x7f69dc100e80 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.196+0000 7f69e2c17700 1 -- 172.21.15.88:0/3472671405 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69dc19aa10 con 0x7f69dc100e80 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.196+0000 7f69d9ffb700 1 -- 172.21.15.88:0/3472671405 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69c8018630 con 0x7f69dc100e80 2024-02-19T19:15:01.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.196+0000 7f69e2c17700 1 -- 172.21.15.88:0/3472671405 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69dc19af60 con 0x7f69dc100e80 2024-02-19T19:15:01.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.197+0000 7f69e2c17700 1 -- 172.21.15.88:0/3472671405 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69dc1087f0 con 0x7f69dc100e80 2024-02-19T19:15:01.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.197+0000 7f69d9ffb700 1 -- 172.21.15.88:0/3472671405 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f69c800f450 con 0x7f69dc100e80 2024-02-19T19:15:01.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.198+0000 7f69d9ffb700 1 --2- 172.21.15.88:0/3472671405 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f69cc06d8a0 0x7f69cc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:01.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.198+0000 7f69d9ffb700 1 -- 172.21.15.88:0/3472671405 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f69c808d4e0 con 0x7f69dc100e80 2024-02-19T19:15:01.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.198+0000 7f69e09b3700 1 --2- 172.21.15.88:0/3472671405 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f69cc06d8a0 0x7f69cc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:01.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.198+0000 7f69e09b3700 1 --2- 172.21.15.88:0/3472671405 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f69cc06d8a0 0x7f69cc06fd50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f69d0006010 tx=0x7f69d001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:01.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.203+0000 7f69d9ffb700 1 -- 172.21.15.88:0/3472671405 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f69c8056c90 con 0x7f69dc100e80 2024-02-19T19:15:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.408+0000 7f69e2c17700 1 -- 172.21.15.88:0/3472671405 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f69dc19b1e0 con 0x7f69cc06d8a0 2024-02-19T19:15:01.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.419+0000 7f69d9ffb700 1 -- 172.21.15.88:0/3472671405 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f69dc19b1e0 con 0x7f69cc06d8a0 2024-02-19T19:15:01.441 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:15:01.441 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (12m) 4m ago 22m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:15:01.441 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (23m) 4m ago 23m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:15:01.441 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (5m) 3m ago 21m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:15:01.441 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (8m) 4m ago 23m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:15:01.441 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (8m) 3m ago 21m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (11m) 4m ago 22m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (4m) 4m ago 16m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (4m) 4m ago 16m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (3m) 3m ago 16m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (16m) 3m ago 16m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (13m) 4m ago 24m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (13m) 3m ago 20m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (10m) 4m ago 24m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (9m) 3m ago 20m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (12m) 4m ago 22m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (12m) 3m ago 20m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (7m) 4m ago 20m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (7m) 4m ago 19m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (6m) 4m ago 19m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:15:01.442 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (5m) 3m ago 18m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:15:01.443 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (5m) 3m ago 18m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:15:01.443 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (4m) 3m ago 17m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:15:01.443 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (12m) 4m ago 21m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:15:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.422+0000 7f69e2c17700 1 -- 172.21.15.88:0/3472671405 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f69cc06d8a0 msgr2=0x7f69cc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.422+0000 7f69e2c17700 1 --2- 172.21.15.88:0/3472671405 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f69cc06d8a0 0x7f69cc06fd50 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f69d0006010 tx=0x7f69d001a040 comp rx=0 tx=0).stop 2024-02-19T19:15:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.422+0000 7f69e2c17700 1 -- 172.21.15.88:0/3472671405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc100e80 msgr2=0x7f69dc196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.422+0000 7f69e2c17700 1 --2- 172.21.15.88:0/3472671405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc100e80 0x7f69dc196430 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f69c800c960 tx=0x7f69c800cc70 comp rx=0 tx=0).stop 2024-02-19T19:15:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.422+0000 7f69e2c17700 1 -- 172.21.15.88:0/3472671405 shutdown_connections 2024-02-19T19:15:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.422+0000 7f69e2c17700 1 --2- 172.21.15.88:0/3472671405 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f69cc06d8a0 0x7f69cc06fd50 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.422+0000 7f69e2c17700 1 --2- 172.21.15.88:0/3472671405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69dc1004e0 0x7f69dc195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.422+0000 7f69e2c17700 1 --2- 172.21.15.88:0/3472671405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69dc100e80 0x7f69dc196430 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.422+0000 7f69e2c17700 1 -- 172.21.15.88:0/3472671405 >> 172.21.15.88:0/3472671405 conn(0x7f69dc0fbda0 msgr2=0x7f69dc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.422+0000 7f69e2c17700 1 -- 172.21.15.88:0/3472671405 shutdown_connections 2024-02-19T19:15:01.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.422+0000 7f69e2c17700 1 -- 172.21.15.88:0/3472671405 wait complete. 2024-02-19T19:15:01.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.524+0000 7f440b17f700 1 -- 172.21.15.88:0/1381748403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44040ff210 msgr2=0x7f44040ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.524+0000 7f440b17f700 1 --2- 172.21.15.88:0/1381748403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44040ff210 0x7f44040ff5e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4400009910 tx=0x7f4400009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:01.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.524+0000 7f440b17f700 1 -- 172.21.15.88:0/1381748403 shutdown_connections 2024-02-19T19:15:01.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.524+0000 7f440b17f700 1 --2- 172.21.15.88:0/1381748403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44040ffb20 0x7f4404104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.524+0000 7f440b17f700 1 --2- 172.21.15.88:0/1381748403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44040ff210 0x7f44040ff5e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.524+0000 7f440b17f700 1 -- 172.21.15.88:0/1381748403 >> 172.21.15.88:0/1381748403 conn(0x7f44040fac80 msgr2=0x7f44040fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:01.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.524+0000 7f440b17f700 1 -- 172.21.15.88:0/1381748403 shutdown_connections 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.524+0000 7f440b17f700 1 -- 172.21.15.88:0/1381748403 wait complete. 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.524+0000 7f440b17f700 1 Processor -- start 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.524+0000 7f440b17f700 1 -- start start 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f440b17f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44040ff210 0x7f440419c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f440b17f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44040ffb20 0x7f440419c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f440b17f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440419d050 con 0x7f44040ff210 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f440b17f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44041a0de0 con 0x7f44040ffb20 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f4408f1b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44040ff210 0x7f440419c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f4408f1b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44040ff210 0x7f440419c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46526/0 (socket says 172.21.15.88:46526) 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f4408f1b700 1 -- 172.21.15.88:0/2092661947 learned_addr learned my addr 172.21.15.88:0/2092661947 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f43fbfff700 1 --2- 172.21.15.88:0/2092661947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44040ffb20 0x7f440419c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f4408f1b700 1 -- 172.21.15.88:0/2092661947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44040ffb20 msgr2=0x7f440419c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f4408f1b700 1 --2- 172.21.15.88:0/2092661947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44040ffb20 0x7f440419c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f4408f1b700 1 -- 172.21.15.88:0/2092661947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44000095f0 con 0x7f44040ff210 2024-02-19T19:15:01.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f4408f1b700 1 --2- 172.21.15.88:0/2092661947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44040ff210 0x7f440419c430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4400005f50 tx=0x7f440000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:01.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f43f9ffb700 1 -- 172.21.15.88:0/2092661947 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f440001d070 con 0x7f44040ff210 2024-02-19T19:15:01.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f43f9ffb700 1 -- 172.21.15.88:0/2092661947 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f440000b610 con 0x7f44040ff210 2024-02-19T19:15:01.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f440b17f700 1 -- 172.21.15.88:0/2092661947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44041a1060 con 0x7f44040ff210 2024-02-19T19:15:01.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.525+0000 7f43f9ffb700 1 -- 172.21.15.88:0/2092661947 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4400022470 con 0x7f44040ff210 2024-02-19T19:15:01.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.526+0000 7f440b17f700 1 -- 172.21.15.88:0/2092661947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44041a1550 con 0x7f44040ff210 2024-02-19T19:15:01.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.527+0000 7f440b17f700 1 -- 172.21.15.88:0/2092661947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4404101730 con 0x7f44040ff210 2024-02-19T19:15:01.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.527+0000 7f43f9ffb700 1 -- 172.21.15.88:0/2092661947 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4400003780 con 0x7f44040ff210 2024-02-19T19:15:01.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.527+0000 7f43f9ffb700 1 --2- 172.21.15.88:0/2092661947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43ec06d8a0 0x7f43ec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:01.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.528+0000 7f43f9ffb700 1 -- 172.21.15.88:0/2092661947 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f440008e830 con 0x7f44040ff210 2024-02-19T19:15:01.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.528+0000 7f43fbfff700 1 --2- 172.21.15.88:0/2092661947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43ec06d8a0 0x7f43ec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:01.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.528+0000 7f43fbfff700 1 --2- 172.21.15.88:0/2092661947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43ec06d8a0 0x7f43ec06fd50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f440419da50 tx=0x7f43f4009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:01.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.532+0000 7f43f9ffb700 1 -- 172.21.15.88:0/2092661947 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4400058060 con 0x7f44040ff210 2024-02-19T19:15:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.812+0000 7f440b17f700 1 -- 172.21.15.88:0/2092661947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f440404f2a0 con 0x7f44040ff210 2024-02-19T19:15:01.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:01 smithi088 ceph-mon[88643]: pgmap v342: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.812+0000 7f43f9ffb700 1 -- 172.21.15.88:0/2092661947 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f440005b720 con 0x7f44040ff210 2024-02-19T19:15:01.834 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:15:01.834 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:15:01.834 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:15:01.834 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:15:01.834 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:15:01.835 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:15:01.836 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:15:01.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.814+0000 7f440b17f700 1 -- 172.21.15.88:0/2092661947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43ec06d8a0 msgr2=0x7f43ec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.814+0000 7f440b17f700 1 --2- 172.21.15.88:0/2092661947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43ec06d8a0 0x7f43ec06fd50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f440419da50 tx=0x7f43f4009260 comp rx=0 tx=0).stop 2024-02-19T19:15:01.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.814+0000 7f440b17f700 1 -- 172.21.15.88:0/2092661947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44040ff210 msgr2=0x7f440419c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.814+0000 7f440b17f700 1 --2- 172.21.15.88:0/2092661947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44040ff210 0x7f440419c430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4400005f50 tx=0x7f440000f690 comp rx=0 tx=0).stop 2024-02-19T19:15:01.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.814+0000 7f440b17f700 1 -- 172.21.15.88:0/2092661947 shutdown_connections 2024-02-19T19:15:01.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.814+0000 7f440b17f700 1 --2- 172.21.15.88:0/2092661947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43ec06d8a0 0x7f43ec06fd50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.814+0000 7f440b17f700 1 --2- 172.21.15.88:0/2092661947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44040ff210 0x7f440419c430 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.814+0000 7f440b17f700 1 --2- 172.21.15.88:0/2092661947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f44040ffb20 0x7f440419c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.814+0000 7f440b17f700 1 -- 172.21.15.88:0/2092661947 >> 172.21.15.88:0/2092661947 conn(0x7f44040fac80 msgr2=0x7f4404110680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:01.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.815+0000 7f440b17f700 1 -- 172.21.15.88:0/2092661947 shutdown_connections 2024-02-19T19:15:01.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.815+0000 7f440b17f700 1 -- 172.21.15.88:0/2092661947 wait complete. 2024-02-19T19:15:01.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.918+0000 7f4f174b3700 1 -- 172.21.15.88:0/2692336472 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f100ff3c0 msgr2=0x7f4f100ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.918+0000 7f4f174b3700 1 --2- 172.21.15.88:0/2692336472 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f100ff3c0 0x7f4f100ff790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009910 tx=0x7f4f0c009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:01.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.918+0000 7f4f174b3700 1 -- 172.21.15.88:0/2692336472 shutdown_connections 2024-02-19T19:15:01.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.918+0000 7f4f174b3700 1 --2- 172.21.15.88:0/2692336472 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f100ffd60 0x7f4f10103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.918+0000 7f4f174b3700 1 --2- 172.21.15.88:0/2692336472 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f100ff3c0 0x7f4f100ff790 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.918+0000 7f4f174b3700 1 -- 172.21.15.88:0/2692336472 >> 172.21.15.88:0/2692336472 conn(0x7f4f100fac80 msgr2=0x7f4f100fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:01.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.919+0000 7f4f174b3700 1 -- 172.21.15.88:0/2692336472 shutdown_connections 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.919+0000 7f4f174b3700 1 -- 172.21.15.88:0/2692336472 wait complete. 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.919+0000 7f4f174b3700 1 Processor -- start 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.919+0000 7f4f174b3700 1 -- start start 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.919+0000 7f4f174b3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f100ff3c0 0x7f4f10100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.919+0000 7f4f174b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f100ffd60 0x7f4f10101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.919+0000 7f4f174b3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f10101850 con 0x7f4f100ff3c0 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.919+0000 7f4f174b3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f10101990 con 0x7f4f100ffd60 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.919+0000 7f4f1524f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f100ff3c0 0x7f4f10100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.919+0000 7f4f1524f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f100ff3c0 0x7f4f10100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46548/0 (socket says 172.21.15.88:46548) 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.919+0000 7f4f1524f700 1 -- 172.21.15.88:0/1872151357 learned_addr learned my addr 172.21.15.88:0/1872151357 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.920+0000 7f4f14a4e700 1 --2- 172.21.15.88:0/1872151357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f100ffd60 0x7f4f10101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:01.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.920+0000 7f4f1524f700 1 -- 172.21.15.88:0/1872151357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f100ffd60 msgr2=0x7f4f10101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.920+0000 7f4f1524f700 1 --2- 172.21.15.88:0/1872151357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f100ffd60 0x7f4f10101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.920+0000 7f4f1524f700 1 -- 172.21.15.88:0/1872151357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c0095f0 con 0x7f4f100ff3c0 2024-02-19T19:15:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.920+0000 7f4f1524f700 1 --2- 172.21.15.88:0/1872151357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f100ff3c0 0x7f4f10100dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c006010 tx=0x7f4f0c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.920+0000 7f4f067fc700 1 -- 172.21.15.88:0/1872151357 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f0c01d070 con 0x7f4f100ff3c0 2024-02-19T19:15:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.920+0000 7f4f067fc700 1 -- 172.21.15.88:0/1872151357 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f0c005dc0 con 0x7f4f100ff3c0 2024-02-19T19:15:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.920+0000 7f4f174b3700 1 -- 172.21.15.88:0/1872151357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f10101c10 con 0x7f4f100ff3c0 2024-02-19T19:15:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.920+0000 7f4f067fc700 1 -- 172.21.15.88:0/1872151357 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f0c017400 con 0x7f4f100ff3c0 2024-02-19T19:15:01.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.920+0000 7f4f174b3700 1 -- 172.21.15.88:0/1872151357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f101a2360 con 0x7f4f100ff3c0 2024-02-19T19:15:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.921+0000 7f4f174b3700 1 -- 172.21.15.88:0/1872151357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1004f2a0 con 0x7f4f100ff3c0 2024-02-19T19:15:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.921+0000 7f4f067fc700 1 -- 172.21.15.88:0/1872151357 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4f0c003a60 con 0x7f4f100ff3c0 2024-02-19T19:15:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.922+0000 7f4f067fc700 1 --2- 172.21.15.88:0/1872151357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4efc06d8a0 0x7f4efc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.922+0000 7f4f067fc700 1 -- 172.21.15.88:0/1872151357 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4f0c08e8e0 con 0x7f4f100ff3c0 2024-02-19T19:15:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.922+0000 7f4f14a4e700 1 --2- 172.21.15.88:0/1872151357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4efc06d8a0 0x7f4efc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:01.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.922+0000 7f4f14a4e700 1 --2- 172.21.15.88:0/1872151357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4efc06d8a0 0x7f4efc06fd50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f101025c0 tx=0x7f4f000073d0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:01.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:01.926+0000 7f4f067fc700 1 -- 172.21.15.88:0/1872151357 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f0c058110 con 0x7f4f100ff3c0 2024-02-19T19:15:02.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:01 smithi132 ceph-mon[70676]: pgmap v342: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:02.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.166+0000 7f4f174b3700 1 -- 172.21.15.88:0/1872151357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4f101a25d0 con 0x7f4f100ff3c0 2024-02-19T19:15:02.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.167+0000 7f4f067fc700 1 -- 172.21.15.88:0/1872151357 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f4f0c05b7d0 con 0x7f4f100ff3c0 2024-02-19T19:15:02.188 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:15:02.189 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:15:02.190 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:15:02.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:15:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.169+0000 7f4f174b3700 1 -- 172.21.15.88:0/1872151357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4efc06d8a0 msgr2=0x7f4efc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.169+0000 7f4f174b3700 1 --2- 172.21.15.88:0/1872151357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4efc06d8a0 0x7f4efc06fd50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f101025c0 tx=0x7f4f000073d0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.169+0000 7f4f174b3700 1 -- 172.21.15.88:0/1872151357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f100ff3c0 msgr2=0x7f4f10100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.169+0000 7f4f174b3700 1 --2- 172.21.15.88:0/1872151357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f100ff3c0 0x7f4f10100dd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c006010 tx=0x7f4f0c00bd70 comp rx=0 tx=0).stop 2024-02-19T19:15:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.169+0000 7f4f174b3700 1 -- 172.21.15.88:0/1872151357 shutdown_connections 2024-02-19T19:15:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.169+0000 7f4f174b3700 1 --2- 172.21.15.88:0/1872151357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4efc06d8a0 0x7f4efc06fd50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.169+0000 7f4f174b3700 1 --2- 172.21.15.88:0/1872151357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f100ff3c0 0x7f4f10100dd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.169+0000 7f4f174b3700 1 --2- 172.21.15.88:0/1872151357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f100ffd60 0x7f4f10101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.169+0000 7f4f174b3700 1 -- 172.21.15.88:0/1872151357 >> 172.21.15.88:0/1872151357 conn(0x7f4f100fac80 msgr2=0x7f4f100fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.169+0000 7f4f174b3700 1 -- 172.21.15.88:0/1872151357 shutdown_connections 2024-02-19T19:15:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.169+0000 7f4f174b3700 1 -- 172.21.15.88:0/1872151357 wait complete. 2024-02-19T19:15:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:15:02.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.274+0000 7fb78a1c1700 1 -- 172.21.15.88:0/3228980843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb784101d30 msgr2=0x7fb78410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:02.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.274+0000 7fb78a1c1700 1 --2- 172.21.15.88:0/3228980843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb784101d30 0x7fb78410a230 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb778009910 tx=0x7fb778009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:02.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.274+0000 7fb78a1c1700 1 -- 172.21.15.88:0/3228980843 shutdown_connections 2024-02-19T19:15:02.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.274+0000 7fb78a1c1700 1 --2- 172.21.15.88:0/3228980843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb784101d30 0x7fb78410a230 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.274+0000 7fb78a1c1700 1 --2- 172.21.15.88:0/3228980843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb784101420 0x7fb7841017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.274+0000 7fb78a1c1700 1 -- 172.21.15.88:0/3228980843 >> 172.21.15.88:0/3228980843 conn(0x7fb7840fac80 msgr2=0x7fb7840fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:02.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.274+0000 7fb78a1c1700 1 -- 172.21.15.88:0/3228980843 shutdown_connections 2024-02-19T19:15:02.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.274+0000 7fb78a1c1700 1 -- 172.21.15.88:0/3228980843 wait complete. 2024-02-19T19:15:02.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.274+0000 7fb78a1c1700 1 Processor -- start 2024-02-19T19:15:02.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb78a1c1700 1 -- start start 2024-02-19T19:15:02.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb78a1c1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb784101420 0x7fb784195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:02.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb78a1c1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb784101d30 0x7fb7841963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:02.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb78a1c1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb784196a80 con 0x7fb784101420 2024-02-19T19:15:02.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb78a1c1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb78419a810 con 0x7fb784101d30 2024-02-19T19:15:02.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb7837fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb784101420 0x7fb784195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:02.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb7837fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb784101420 0x7fb784195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46558/0 (socket says 172.21.15.88:46558) 2024-02-19T19:15:02.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb7837fe700 1 -- 172.21.15.88:0/1986307404 learned_addr learned my addr 172.21.15.88:0/1986307404 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:02.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb782ffd700 1 --2- 172.21.15.88:0/1986307404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb784101d30 0x7fb7841963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:02.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb7837fe700 1 -- 172.21.15.88:0/1986307404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb784101d30 msgr2=0x7fb7841963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:02.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb7837fe700 1 --2- 172.21.15.88:0/1986307404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb784101d30 0x7fb7841963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb7837fe700 1 -- 172.21.15.88:0/1986307404 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780095f0 con 0x7fb784101420 2024-02-19T19:15:02.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb7837fe700 1 --2- 172.21.15.88:0/1986307404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb784101420 0x7fb784195e60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb77400d8d0 tx=0x7fb77400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:02.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb780ff9700 1 -- 172.21.15.88:0/1986307404 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7740098a0 con 0x7fb784101420 2024-02-19T19:15:02.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb780ff9700 1 -- 172.21.15.88:0/1986307404 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb774010460 con 0x7fb784101420 2024-02-19T19:15:02.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.275+0000 7fb78a1c1700 1 -- 172.21.15.88:0/1986307404 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb78419aaf0 con 0x7fb784101420 2024-02-19T19:15:02.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.276+0000 7fb780ff9700 1 -- 172.21.15.88:0/1986307404 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb77400f5f0 con 0x7fb784101420 2024-02-19T19:15:02.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.276+0000 7fb78a1c1700 1 -- 172.21.15.88:0/1986307404 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb78419b010 con 0x7fb784101420 2024-02-19T19:15:02.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.276+0000 7fb78a1c1700 1 -- 172.21.15.88:0/1986307404 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7841078a0 con 0x7fb784101420 2024-02-19T19:15:02.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.277+0000 7fb780ff9700 1 -- 172.21.15.88:0/1986307404 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb774009a00 con 0x7fb784101420 2024-02-19T19:15:02.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.277+0000 7fb780ff9700 1 --2- 172.21.15.88:0/1986307404 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb76c06d8a0 0x7fb76c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:02.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.277+0000 7fb780ff9700 1 -- 172.21.15.88:0/1986307404 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb77408d340 con 0x7fb784101420 2024-02-19T19:15:02.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.277+0000 7fb782ffd700 1 --2- 172.21.15.88:0/1986307404 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb76c06d8a0 0x7fb76c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:02.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.278+0000 7fb782ffd700 1 --2- 172.21.15.88:0/1986307404 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb76c06d8a0 0x7fb76c06fd50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb77800b5c0 tx=0x7fb77801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:02.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.282+0000 7fb780ff9700 1 -- 172.21.15.88:0/1986307404 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb774056b70 con 0x7fb784101420 2024-02-19T19:15:02.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.498+0000 7fb78a1c1700 1 -- 172.21.15.88:0/1986307404 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb784197330 con 0x7fb76c06d8a0 2024-02-19T19:15:02.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.500+0000 7fb780ff9700 1 -- 172.21.15.88:0/1986307404 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb77801e040 con 0x7fb76c06d8a0 2024-02-19T19:15:02.521 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:15:02.521 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:15:02.521 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:15:02.521 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:15:02.521 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:15:02.521 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:15:02.521 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:15:02.522 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:15:02.522 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:15:02.522 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:15:02.522 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:15:02.522 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:15:02.522 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:15:02.522 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:15:02.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.502+0000 7fb78a1c1700 1 -- 172.21.15.88:0/1986307404 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb76c06d8a0 msgr2=0x7fb76c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.502+0000 7fb78a1c1700 1 --2- 172.21.15.88:0/1986307404 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb76c06d8a0 0x7fb76c06fd50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb77800b5c0 tx=0x7fb77801a040 comp rx=0 tx=0).stop 2024-02-19T19:15:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.502+0000 7fb78a1c1700 1 -- 172.21.15.88:0/1986307404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb784101420 msgr2=0x7fb784195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.502+0000 7fb78a1c1700 1 --2- 172.21.15.88:0/1986307404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb784101420 0x7fb784195e60 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb77400d8d0 tx=0x7fb77400dbe0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.503+0000 7fb78a1c1700 1 -- 172.21.15.88:0/1986307404 shutdown_connections 2024-02-19T19:15:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.503+0000 7fb78a1c1700 1 --2- 172.21.15.88:0/1986307404 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb76c06d8a0 0x7fb76c06fd50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.503+0000 7fb78a1c1700 1 --2- 172.21.15.88:0/1986307404 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb784101420 0x7fb784195e60 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.503+0000 7fb78a1c1700 1 --2- 172.21.15.88:0/1986307404 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb784101d30 0x7fb7841963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.503+0000 7fb78a1c1700 1 -- 172.21.15.88:0/1986307404 >> 172.21.15.88:0/1986307404 conn(0x7fb7840fac80 msgr2=0x7fb7841049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.503+0000 7fb78a1c1700 1 -- 172.21.15.88:0/1986307404 shutdown_connections 2024-02-19T19:15:02.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.503+0000 7fb78a1c1700 1 -- 172.21.15.88:0/1986307404 wait complete. 2024-02-19T19:15:02.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.609+0000 7efc0fb44700 1 -- 172.21.15.88:0/2145964384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc080ff3c0 msgr2=0x7efc080ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:02.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.609+0000 7efc0fb44700 1 --2- 172.21.15.88:0/2145964384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc080ff3c0 0x7efc080ff790 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efc00009910 tx=0x7efc00009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:02.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.609+0000 7efc0fb44700 1 -- 172.21.15.88:0/2145964384 shutdown_connections 2024-02-19T19:15:02.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.609+0000 7efc0fb44700 1 --2- 172.21.15.88:0/2145964384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc080ffd60 0x7efc08103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.609+0000 7efc0fb44700 1 --2- 172.21.15.88:0/2145964384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc080ff3c0 0x7efc080ff790 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.609+0000 7efc0fb44700 1 -- 172.21.15.88:0/2145964384 >> 172.21.15.88:0/2145964384 conn(0x7efc080fac60 msgr2=0x7efc080fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:02.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.609+0000 7efc0fb44700 1 -- 172.21.15.88:0/2145964384 shutdown_connections 2024-02-19T19:15:02.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.609+0000 7efc0fb44700 1 -- 172.21.15.88:0/2145964384 wait complete. 2024-02-19T19:15:02.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0fb44700 1 Processor -- start 2024-02-19T19:15:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0fb44700 1 -- start start 2024-02-19T19:15:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0fb44700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc080ff3c0 0x7efc08196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0fb44700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc080ffd60 0x7efc08197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0fb44700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc08197940 con 0x7efc080ff3c0 2024-02-19T19:15:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0fb44700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc08197ab0 con 0x7efc080ffd60 2024-02-19T19:15:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0d8e0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc080ff3c0 0x7efc08196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0d8e0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc080ff3c0 0x7efc08196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46564/0 (socket says 172.21.15.88:46564) 2024-02-19T19:15:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0d8e0700 1 -- 172.21.15.88:0/457223962 learned_addr learned my addr 172.21.15.88:0/457223962 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:02.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0d0df700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc080ffd60 0x7efc08197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0d8e0700 1 -- 172.21.15.88:0/457223962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc080ffd60 msgr2=0x7efc08197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0d8e0700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc080ffd60 0x7efc08197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0d8e0700 1 -- 172.21.15.88:0/457223962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc000095f0 con 0x7efc080ff3c0 2024-02-19T19:15:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0d0df700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc080ffd60 0x7efc08197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:15:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.610+0000 7efc0d8e0700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc080ff3c0 0x7efc08196de0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efc00006010 tx=0x7efc0000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.611+0000 7efbfeffd700 1 -- 172.21.15.88:0/457223962 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc0001d070 con 0x7efc080ff3c0 2024-02-19T19:15:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.611+0000 7efbfeffd700 1 -- 172.21.15.88:0/457223962 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc00004fb0 con 0x7efc080ff3c0 2024-02-19T19:15:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.611+0000 7efc0fb44700 1 -- 172.21.15.88:0/457223962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0819fe50 con 0x7efc080ff3c0 2024-02-19T19:15:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.611+0000 7efbfeffd700 1 -- 172.21.15.88:0/457223962 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc00022ea0 con 0x7efc080ff3c0 2024-02-19T19:15:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.611+0000 7efc0fb44700 1 -- 172.21.15.88:0/457223962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc081a02c0 con 0x7efc080ff3c0 2024-02-19T19:15:02.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.611+0000 7efc0fb44700 1 -- 172.21.15.88:0/457223962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc081076d0 con 0x7efc080ff3c0 2024-02-19T19:15:02.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.612+0000 7efbfeffd700 1 -- 172.21.15.88:0/457223962 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7efc00022470 con 0x7efc080ff3c0 2024-02-19T19:15:02.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.612+0000 7efbfeffd700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efbf406d8a0 0x7efbf406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:02.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.613+0000 7efc0d0df700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efbf406d8a0 0x7efbf406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:02.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.613+0000 7efbfeffd700 1 -- 172.21.15.88:0/457223962 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efc0008ea10 con 0x7efc080ff3c0 2024-02-19T19:15:02.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.613+0000 7efc0d0df700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efbf406d8a0 0x7efbf406fd50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efc080696a0 tx=0x7efbf8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:02.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.617+0000 7efbfeffd700 1 -- 172.21.15.88:0/457223962 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efc000582f0 con 0x7efc080ff3c0 2024-02-19T19:15:02.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:02 smithi088 ceph-mon[88643]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:02.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:02 smithi088 ceph-mon[88643]: from='client.34670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:02.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:02 smithi088 ceph-mon[88643]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:02.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:02 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2092661947' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:15:02.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:02 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1872151357' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:15:02.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.894+0000 7efc0fb44700 1 -- 172.21.15.88:0/457223962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7efc081070b0 con 0x7efc080ff3c0 2024-02-19T19:15:02.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.895+0000 7efbfeffd700 1 -- 172.21.15.88:0/457223962 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7efc00027020 con 0x7efc080ff3c0 2024-02-19T19:15:02.916 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:15:02.916 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:15:02.917 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:15:02.917 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:15:02.917 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:15:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.897+0000 7efc0fb44700 1 -- 172.21.15.88:0/457223962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efbf406d8a0 msgr2=0x7efbf406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.897+0000 7efc0fb44700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efbf406d8a0 0x7efbf406fd50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efc080696a0 tx=0x7efbf8008040 comp rx=0 tx=0).stop 2024-02-19T19:15:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.897+0000 7efc0fb44700 1 -- 172.21.15.88:0/457223962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc080ff3c0 msgr2=0x7efc08196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.897+0000 7efc0fb44700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc080ff3c0 0x7efc08196de0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7efc00006010 tx=0x7efc0000be20 comp rx=0 tx=0).stop 2024-02-19T19:15:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.897+0000 7efc0fb44700 1 -- 172.21.15.88:0/457223962 shutdown_connections 2024-02-19T19:15:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.897+0000 7efc0fb44700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efbf406d8a0 0x7efbf406fd50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.897+0000 7efc0fb44700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc080ff3c0 0x7efc08196de0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.897+0000 7efc0fb44700 1 --2- 172.21.15.88:0/457223962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efc080ffd60 0x7efc08197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.897+0000 7efc0fb44700 1 -- 172.21.15.88:0/457223962 >> 172.21.15.88:0/457223962 conn(0x7efc080fac60 msgr2=0x7efc080fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.897+0000 7efc0fb44700 1 -- 172.21.15.88:0/457223962 shutdown_connections 2024-02-19T19:15:02.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:02.897+0000 7efc0fb44700 1 -- 172.21.15.88:0/457223962 wait complete. 2024-02-19T19:15:03.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:02 smithi132 ceph-mon[70676]: from='client.34666 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:03.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:02 smithi132 ceph-mon[70676]: from='client.34670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:03.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:02 smithi132 ceph-mon[70676]: from='client.34674 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:03.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:02 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2092661947' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:15:03.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:02 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1872151357' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:15:03.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:03 smithi088 ceph-mon[88643]: pgmap v343: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:03.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:03 smithi088 ceph-mon[88643]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:03.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:03 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/457223962' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:15:04.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:03 smithi132 ceph-mon[70676]: pgmap v343: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:04.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:03 smithi132 ceph-mon[70676]: from='client.34686 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:04.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:03 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/457223962' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:15:05.004 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:04 smithi132 ceph-mon[70676]: pgmap v344: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:05.101 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:04 smithi088 ceph-mon[88643]: pgmap v344: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:07.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:07 smithi088 ceph-mon[88643]: pgmap v345: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:07.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:07 smithi132 ceph-mon[70676]: pgmap v345: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:09.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:09 smithi088 ceph-mon[88643]: pgmap v346: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:09.754 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:09 smithi132 ceph-mon[70676]: pgmap v346: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:11.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:11 smithi088 ceph-mon[88643]: pgmap v347: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:11.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:11 smithi132 ceph-mon[70676]: pgmap v347: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:13.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:13 smithi088 ceph-mon[88643]: pgmap v348: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:13.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:15:13.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:13 smithi132 ceph-mon[70676]: pgmap v348: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:13.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:15:15.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:15 smithi088 ceph-mon[88643]: pgmap v349: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:15.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:15 smithi132 ceph-mon[70676]: pgmap v349: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:17.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:17 smithi088 ceph-mon[88643]: pgmap v350: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:17.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:17 smithi132 ceph-mon[70676]: pgmap v350: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:19.503 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:19 smithi132 ceph-mon[70676]: pgmap v351: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:19.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:19 smithi088 ceph-mon[88643]: pgmap v351: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:21.601 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:21 smithi088 ceph-mon[88643]: pgmap v352: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:21.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:21 smithi132 ceph-mon[70676]: pgmap v352: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:23.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:23 smithi132 ceph-mon[70676]: pgmap v353: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:23.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:23 smithi088 ceph-mon[88643]: pgmap v353: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:25.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:25 smithi132 ceph-mon[70676]: pgmap v354: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:25.851 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:25 smithi088 ceph-mon[88643]: pgmap v354: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:27.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:27 smithi132 ceph-mon[70676]: pgmap v355: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:27.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:27 smithi088 ceph-mon[88643]: pgmap v355: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:28.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:15:28.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:15:29.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:29 smithi132 ceph-mon[70676]: pgmap v356: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:29.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:29 smithi088 ceph-mon[88643]: pgmap v356: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:31.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:31 smithi132 ceph-mon[70676]: pgmap v357: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:31.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:31 smithi088 ceph-mon[88643]: pgmap v357: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:33.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.015+0000 7f68ea9f1700 1 -- 172.21.15.88:0/7047505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e4100e80 msgr2=0x7f68e4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.015+0000 7f68ea9f1700 1 --2- 172.21.15.88:0/7047505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e4100e80 0x7f68e4104e50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f68cc009910 tx=0x7f68cc009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.015+0000 7f68ea9f1700 1 -- 172.21.15.88:0/7047505 shutdown_connections 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.015+0000 7f68ea9f1700 1 --2- 172.21.15.88:0/7047505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e4100e80 0x7f68e4104e50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.015+0000 7f68ea9f1700 1 --2- 172.21.15.88:0/7047505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68e41004e0 0x7f68e41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.015+0000 7f68ea9f1700 1 -- 172.21.15.88:0/7047505 >> 172.21.15.88:0/7047505 conn(0x7f68e40fbda0 msgr2=0x7f68e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.015+0000 7f68ea9f1700 1 -- 172.21.15.88:0/7047505 shutdown_connections 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.015+0000 7f68ea9f1700 1 -- 172.21.15.88:0/7047505 wait complete. 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68ea9f1700 1 Processor -- start 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68ea9f1700 1 -- start start 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68ea9f1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e41004e0 0x7f68e4195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68ea9f1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68e4100e80 0x7f68e4196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68ea9f1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4196b30 con 0x7f68e41004e0 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68ea9f1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e419a8c0 con 0x7f68e4100e80 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68e3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e41004e0 0x7f68e4195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:33.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68e3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e41004e0 0x7f68e4195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41116/0 (socket says 172.21.15.88:41116) 2024-02-19T19:15:33.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68e3fff700 1 -- 172.21.15.88:0/1567969908 learned_addr learned my addr 172.21.15.88:0/1567969908 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:33.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68e3fff700 1 -- 172.21.15.88:0/1567969908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68e4100e80 msgr2=0x7f68e4196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68e3fff700 1 --2- 172.21.15.88:0/1567969908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68e4100e80 0x7f68e4196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.016+0000 7f68e3fff700 1 -- 172.21.15.88:0/1567969908 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc0095f0 con 0x7f68e41004e0 2024-02-19T19:15:33.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.017+0000 7f68e3fff700 1 --2- 172.21.15.88:0/1567969908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e41004e0 0x7f68e4195f10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f68d400d8d0 tx=0x7f68d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:33.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.017+0000 7f68e1ffb700 1 -- 172.21.15.88:0/1567969908 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68d4009940 con 0x7f68e41004e0 2024-02-19T19:15:33.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.017+0000 7f68ea9f1700 1 -- 172.21.15.88:0/1567969908 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e419aba0 con 0x7f68e41004e0 2024-02-19T19:15:33.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.017+0000 7f68e1ffb700 1 -- 172.21.15.88:0/1567969908 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68d4010460 con 0x7f68e41004e0 2024-02-19T19:15:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.017+0000 7f68e1ffb700 1 -- 172.21.15.88:0/1567969908 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68d400f5f0 con 0x7f68e41004e0 2024-02-19T19:15:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.017+0000 7f68ea9f1700 1 -- 172.21.15.88:0/1567969908 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e419b0f0 con 0x7f68e41004e0 2024-02-19T19:15:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.018+0000 7f68ea9f1700 1 -- 172.21.15.88:0/1567969908 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68e4108850 con 0x7f68e41004e0 2024-02-19T19:15:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.018+0000 7f68e1ffb700 1 -- 172.21.15.88:0/1567969908 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f68d40105d0 con 0x7f68e41004e0 2024-02-19T19:15:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.019+0000 7f68e1ffb700 1 --2- 172.21.15.88:0/1567969908 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68d006d8a0 0x7f68d006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.019+0000 7f68dbfff700 1 --2- 172.21.15.88:0/1567969908 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68d006d8a0 0x7f68d006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.019+0000 7f68e1ffb700 1 -- 172.21.15.88:0/1567969908 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f68d408d540 con 0x7f68e41004e0 2024-02-19T19:15:33.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.019+0000 7f68dbfff700 1 --2- 172.21.15.88:0/1567969908 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68d006d8a0 0x7f68d006fd50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00b5c0 tx=0x7f68cc005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:33.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.024+0000 7f68e1ffb700 1 -- 172.21.15.88:0/1567969908 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f68d4056cf0 con 0x7f68e41004e0 2024-02-19T19:15:33.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.237+0000 7f68ea9f1700 1 -- 172.21.15.88:0/1567969908 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f68e419b310 con 0x7f68d006d8a0 2024-02-19T19:15:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.239+0000 7f68e1ffb700 1 -- 172.21.15.88:0/1567969908 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f68e419b310 con 0x7f68d006d8a0 2024-02-19T19:15:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.241+0000 7f68ea9f1700 1 -- 172.21.15.88:0/1567969908 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68d006d8a0 msgr2=0x7f68d006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.241+0000 7f68ea9f1700 1 --2- 172.21.15.88:0/1567969908 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68d006d8a0 0x7f68d006fd50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f68cc00b5c0 tx=0x7f68cc005960 comp rx=0 tx=0).stop 2024-02-19T19:15:33.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.241+0000 7f68ea9f1700 1 -- 172.21.15.88:0/1567969908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e41004e0 msgr2=0x7f68e4195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.241+0000 7f68ea9f1700 1 --2- 172.21.15.88:0/1567969908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e41004e0 0x7f68e4195f10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f68d400d8d0 tx=0x7f68d400dc90 comp rx=0 tx=0).stop 2024-02-19T19:15:33.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.241+0000 7f68ea9f1700 1 -- 172.21.15.88:0/1567969908 shutdown_connections 2024-02-19T19:15:33.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.241+0000 7f68ea9f1700 1 --2- 172.21.15.88:0/1567969908 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68d006d8a0 0x7f68d006fd50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.241+0000 7f68ea9f1700 1 --2- 172.21.15.88:0/1567969908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e41004e0 0x7f68e4195f10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.241+0000 7f68ea9f1700 1 --2- 172.21.15.88:0/1567969908 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68e4100e80 0x7f68e4196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.241+0000 7f68ea9f1700 1 -- 172.21.15.88:0/1567969908 >> 172.21.15.88:0/1567969908 conn(0x7f68e40fbda0 msgr2=0x7f68e40fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:33.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.241+0000 7f68ea9f1700 1 -- 172.21.15.88:0/1567969908 shutdown_connections 2024-02-19T19:15:33.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.241+0000 7f68ea9f1700 1 -- 172.21.15.88:0/1567969908 wait complete. 2024-02-19T19:15:33.272 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:15:33.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.346+0000 7f23c34dd700 1 -- 172.21.15.88:0/2528785634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23bc0ffd60 msgr2=0x7f23bc103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.346+0000 7f23c34dd700 1 --2- 172.21.15.88:0/2528785634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23bc0ffd60 0x7f23bc103d30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f23ac009910 tx=0x7f23ac009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:33.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.346+0000 7f23c34dd700 1 -- 172.21.15.88:0/2528785634 shutdown_connections 2024-02-19T19:15:33.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.346+0000 7f23c34dd700 1 --2- 172.21.15.88:0/2528785634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23bc0ffd60 0x7f23bc103d30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.346+0000 7f23c34dd700 1 --2- 172.21.15.88:0/2528785634 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23bc0ff3c0 0x7f23bc0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.346+0000 7f23c34dd700 1 -- 172.21.15.88:0/2528785634 >> 172.21.15.88:0/2528785634 conn(0x7f23bc0fac80 msgr2=0x7f23bc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:33.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.346+0000 7f23c34dd700 1 -- 172.21.15.88:0/2528785634 shutdown_connections 2024-02-19T19:15:33.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.346+0000 7f23c34dd700 1 -- 172.21.15.88:0/2528785634 wait complete. 2024-02-19T19:15:33.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.346+0000 7f23c34dd700 1 Processor -- start 2024-02-19T19:15:33.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c34dd700 1 -- start start 2024-02-19T19:15:33.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c34dd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23bc0ff3c0 0x7f23bc194f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:33.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c34dd700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23bc0ffd60 0x7f23bc195490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:33.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c34dd700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc195b70 con 0x7f23bc0ff3c0 2024-02-19T19:15:33.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c34dd700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc199900 con 0x7f23bc0ffd60 2024-02-19T19:15:33.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c1279700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23bc0ff3c0 0x7f23bc194f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:33.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c1279700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23bc0ff3c0 0x7f23bc194f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41126/0 (socket says 172.21.15.88:41126) 2024-02-19T19:15:33.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c1279700 1 -- 172.21.15.88:0/969597712 learned_addr learned my addr 172.21.15.88:0/969597712 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:33.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c0a78700 1 --2- 172.21.15.88:0/969597712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23bc0ffd60 0x7f23bc195490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:33.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c1279700 1 -- 172.21.15.88:0/969597712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23bc0ffd60 msgr2=0x7f23bc195490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c1279700 1 --2- 172.21.15.88:0/969597712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23bc0ffd60 0x7f23bc195490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c1279700 1 -- 172.21.15.88:0/969597712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac0095f0 con 0x7f23bc0ff3c0 2024-02-19T19:15:33.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23c1279700 1 --2- 172.21.15.88:0/969597712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23bc0ff3c0 0x7f23bc194f50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f23b800d960 tx=0x7f23b800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:33.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.347+0000 7f23b27fc700 1 -- 172.21.15.88:0/969597712 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23b800ec20 con 0x7f23bc0ff3c0 2024-02-19T19:15:33.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.348+0000 7f23c34dd700 1 -- 172.21.15.88:0/969597712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc199b80 con 0x7f23bc0ff3c0 2024-02-19T19:15:33.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.348+0000 7f23b27fc700 1 -- 172.21.15.88:0/969597712 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23b800ed80 con 0x7f23bc0ff3c0 2024-02-19T19:15:33.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.348+0000 7f23b27fc700 1 -- 172.21.15.88:0/969597712 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23b80147d0 con 0x7f23bc0ff3c0 2024-02-19T19:15:33.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.348+0000 7f23c34dd700 1 -- 172.21.15.88:0/969597712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc19a0d0 con 0x7f23bc0ff3c0 2024-02-19T19:15:33.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.349+0000 7f23c34dd700 1 -- 172.21.15.88:0/969597712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23bc04f2a0 con 0x7f23bc0ff3c0 2024-02-19T19:15:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.349+0000 7f23b27fc700 1 -- 172.21.15.88:0/969597712 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f23b80149b0 con 0x7f23bc0ff3c0 2024-02-19T19:15:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.349+0000 7f23b27fc700 1 --2- 172.21.15.88:0/969597712 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23a806d8a0 0x7f23a806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.350+0000 7f23c0a78700 1 --2- 172.21.15.88:0/969597712 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23a806d8a0 0x7f23a806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.350+0000 7f23b27fc700 1 -- 172.21.15.88:0/969597712 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f23b808ef50 con 0x7f23bc0ff3c0 2024-02-19T19:15:33.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.350+0000 7f23c0a78700 1 --2- 172.21.15.88:0/969597712 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23a806d8a0 0x7f23a806fd50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f23bc196570 tx=0x7f23ac01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:33.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.356+0000 7f23b27fc700 1 -- 172.21.15.88:0/969597712 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f23b80c2d20 con 0x7f23bc0ff3c0 2024-02-19T19:15:33.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.572+0000 7f23c34dd700 1 -- 172.21.15.88:0/969597712 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f23bc19a460 con 0x7f23a806d8a0 2024-02-19T19:15:33.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.574+0000 7f23b27fc700 1 -- 172.21.15.88:0/969597712 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f23bc19a460 con 0x7f23a806d8a0 2024-02-19T19:15:33.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.576+0000 7f23c34dd700 1 -- 172.21.15.88:0/969597712 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23a806d8a0 msgr2=0x7f23a806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.576+0000 7f23c34dd700 1 --2- 172.21.15.88:0/969597712 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23a806d8a0 0x7f23a806fd50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f23bc196570 tx=0x7f23ac01a040 comp rx=0 tx=0).stop 2024-02-19T19:15:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.576+0000 7f23c34dd700 1 -- 172.21.15.88:0/969597712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23bc0ff3c0 msgr2=0x7f23bc194f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.576+0000 7f23c34dd700 1 --2- 172.21.15.88:0/969597712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23bc0ff3c0 0x7f23bc194f50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f23b800d960 tx=0x7f23b800dd20 comp rx=0 tx=0).stop 2024-02-19T19:15:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.576+0000 7f23c34dd700 1 -- 172.21.15.88:0/969597712 shutdown_connections 2024-02-19T19:15:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.576+0000 7f23c34dd700 1 --2- 172.21.15.88:0/969597712 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23a806d8a0 0x7f23a806fd50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.576+0000 7f23c34dd700 1 --2- 172.21.15.88:0/969597712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f23bc0ff3c0 0x7f23bc194f50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.576+0000 7f23c34dd700 1 --2- 172.21.15.88:0/969597712 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f23bc0ffd60 0x7f23bc195490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.576+0000 7f23c34dd700 1 -- 172.21.15.88:0/969597712 >> 172.21.15.88:0/969597712 conn(0x7f23bc0fac80 msgr2=0x7f23bc0fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.576+0000 7f23c34dd700 1 -- 172.21.15.88:0/969597712 shutdown_connections 2024-02-19T19:15:33.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.576+0000 7f23c34dd700 1 -- 172.21.15.88:0/969597712 wait complete. 2024-02-19T19:15:33.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.680+0000 7f22207d9700 1 -- 172.21.15.88:0/745479320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22181004e0 msgr2=0x7f22181008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.680+0000 7f22207d9700 1 --2- 172.21.15.88:0/745479320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22181004e0 0x7f22181008b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2214009910 tx=0x7f2214009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:33.701 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:33 smithi088 ceph-mon[88643]: pgmap v358: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:33.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.680+0000 7f22207d9700 1 -- 172.21.15.88:0/745479320 shutdown_connections 2024-02-19T19:15:33.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.680+0000 7f22207d9700 1 --2- 172.21.15.88:0/745479320 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2218100e80 0x7f2218104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.681+0000 7f22207d9700 1 --2- 172.21.15.88:0/745479320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22181004e0 0x7f22181008b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.681+0000 7f22207d9700 1 -- 172.21.15.88:0/745479320 >> 172.21.15.88:0/745479320 conn(0x7f22180fbda0 msgr2=0x7f22180fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:33.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.681+0000 7f22207d9700 1 -- 172.21.15.88:0/745479320 shutdown_connections 2024-02-19T19:15:33.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.681+0000 7f22207d9700 1 -- 172.21.15.88:0/745479320 wait complete. 2024-02-19T19:15:33.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.681+0000 7f22207d9700 1 Processor -- start 2024-02-19T19:15:33.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.681+0000 7f22207d9700 1 -- start start 2024-02-19T19:15:33.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f22207d9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22181004e0 0x7f2218195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f22207d9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2218100e80 0x7f2218196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f22207d9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2218196b10 con 0x7f22181004e0 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f22207d9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221819a8a0 con 0x7f2218100e80 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f221e575700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22181004e0 0x7f2218195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f221e575700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22181004e0 0x7f2218195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41154/0 (socket says 172.21.15.88:41154) 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f221e575700 1 -- 172.21.15.88:0/2051845663 learned_addr learned my addr 172.21.15.88:0/2051845663 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f221dd74700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2218100e80 0x7f2218196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f221e575700 1 -- 172.21.15.88:0/2051845663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2218100e80 msgr2=0x7f2218196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f221e575700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2218100e80 0x7f2218196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f221e575700 1 -- 172.21.15.88:0/2051845663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22140095f0 con 0x7f22181004e0 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f221dd74700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2218100e80 0x7f2218196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f221e575700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22181004e0 0x7f2218195ef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2214006010 tx=0x7f221400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:33.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f220f7fe700 1 -- 172.21.15.88:0/2051845663 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f221401d070 con 0x7f22181004e0 2024-02-19T19:15:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f220f7fe700 1 -- 172.21.15.88:0/2051845663 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2214004fb0 con 0x7f22181004e0 2024-02-19T19:15:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f22207d9700 1 -- 172.21.15.88:0/2051845663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f221819ab20 con 0x7f22181004e0 2024-02-19T19:15:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f220f7fe700 1 -- 172.21.15.88:0/2051845663 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2214022ea0 con 0x7f22181004e0 2024-02-19T19:15:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.682+0000 7f22207d9700 1 -- 172.21.15.88:0/2051845663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221819b010 con 0x7f22181004e0 2024-02-19T19:15:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.683+0000 7f22207d9700 1 -- 172.21.15.88:0/2051845663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22181087f0 con 0x7f22181004e0 2024-02-19T19:15:33.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.684+0000 7f220f7fe700 1 -- 172.21.15.88:0/2051845663 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2214022470 con 0x7f22181004e0 2024-02-19T19:15:33.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.684+0000 7f220f7fe700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f220406d8a0 0x7f220406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:33.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.684+0000 7f220f7fe700 1 -- 172.21.15.88:0/2051845663 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f221408ebb0 con 0x7f22181004e0 2024-02-19T19:15:33.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.685+0000 7f221dd74700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f220406d8a0 0x7f220406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:33.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.685+0000 7f221dd74700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f220406d8a0 0x7f220406fd50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2218197510 tx=0x7f2208005f50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:33.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.689+0000 7f220f7fe700 1 -- 172.21.15.88:0/2051845663 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2214095080 con 0x7f22181004e0 2024-02-19T19:15:33.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:33 smithi132 ceph-mon[70676]: pgmap v358: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:33.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.898+0000 7f22207d9700 1 -- 172.21.15.88:0/2051845663 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f221819b230 con 0x7f220406d8a0 2024-02-19T19:15:33.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.908+0000 7f220f7fe700 1 -- 172.21.15.88:0/2051845663 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f221819b230 con 0x7f220406d8a0 2024-02-19T19:15:33.928 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:15:33.928 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (12m) 4m ago 23m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:15:33.928 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (23m) 4m ago 23m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (5m) 4m ago 21m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (8m) 4m ago 23m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (8m) 4m ago 21m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (12m) 4m ago 22m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (4m) 4m ago 16m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (4m) 4m ago 16m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (4m) 4m ago 16m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (16m) 4m ago 16m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (14m) 4m ago 24m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (13m) 4m ago 21m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (10m) 4m ago 25m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (10m) 4m ago 21m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (13m) 4m ago 23m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:15:33.929 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (13m) 4m ago 21m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:15:33.930 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (8m) 4m ago 20m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:15:33.930 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (7m) 4m ago 20m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:15:33.930 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (6m) 4m ago 19m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:15:33.930 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (6m) 4m ago 19m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:15:33.930 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (5m) 4m ago 18m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:15:33.930 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (5m) 4m ago 18m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:15:33.930 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (13m) 4m ago 22m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:15:33.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.910+0000 7f22207d9700 1 -- 172.21.15.88:0/2051845663 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f220406d8a0 msgr2=0x7f220406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.910+0000 7f22207d9700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f220406d8a0 0x7f220406fd50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2218197510 tx=0x7f2208005f50 comp rx=0 tx=0).stop 2024-02-19T19:15:33.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.910+0000 7f22207d9700 1 -- 172.21.15.88:0/2051845663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22181004e0 msgr2=0x7f2218195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:33.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.910+0000 7f22207d9700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22181004e0 0x7f2218195ef0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2214006010 tx=0x7f221400be20 comp rx=0 tx=0).stop 2024-02-19T19:15:33.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.910+0000 7f22207d9700 1 -- 172.21.15.88:0/2051845663 shutdown_connections 2024-02-19T19:15:33.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.910+0000 7f22207d9700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f220406d8a0 0x7f220406fd50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.910+0000 7f22207d9700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22181004e0 0x7f2218195ef0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.910+0000 7f22207d9700 1 --2- 172.21.15.88:0/2051845663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2218100e80 0x7f2218196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:33.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.910+0000 7f22207d9700 1 -- 172.21.15.88:0/2051845663 >> 172.21.15.88:0/2051845663 conn(0x7f22180fbda0 msgr2=0x7f22180fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:33.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.910+0000 7f22207d9700 1 -- 172.21.15.88:0/2051845663 shutdown_connections 2024-02-19T19:15:33.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:33.910+0000 7f22207d9700 1 -- 172.21.15.88:0/2051845663 wait complete. 2024-02-19T19:15:34.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.013+0000 7faa97ad4700 1 -- 172.21.15.88:0/3640262873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa901004e0 msgr2=0x7faa901008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:34.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.013+0000 7faa97ad4700 1 --2- 172.21.15.88:0/3640262873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa901004e0 0x7faa901008b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faa8c009910 tx=0x7faa8c009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:34.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.013+0000 7faa97ad4700 1 -- 172.21.15.88:0/3640262873 shutdown_connections 2024-02-19T19:15:34.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.013+0000 7faa97ad4700 1 --2- 172.21.15.88:0/3640262873 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa90100e80 0x7faa90104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.013+0000 7faa97ad4700 1 --2- 172.21.15.88:0/3640262873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa901004e0 0x7faa901008b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.013+0000 7faa97ad4700 1 -- 172.21.15.88:0/3640262873 >> 172.21.15.88:0/3640262873 conn(0x7faa900fbda0 msgr2=0x7faa900fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:34.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.014+0000 7faa97ad4700 1 -- 172.21.15.88:0/3640262873 shutdown_connections 2024-02-19T19:15:34.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.014+0000 7faa97ad4700 1 -- 172.21.15.88:0/3640262873 wait complete. 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.014+0000 7faa97ad4700 1 Processor -- start 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.014+0000 7faa97ad4700 1 -- start start 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa97ad4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa901004e0 0x7faa90073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa97ad4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa90100e80 0x7faa90073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa97ad4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa90073ad0 con 0x7faa90100e80 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa97ad4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa90073c10 con 0x7faa901004e0 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa9506f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa90100e80 0x7faa90073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa9506f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa90100e80 0x7faa90073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41184/0 (socket says 172.21.15.88:41184) 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa9506f700 1 -- 172.21.15.88:0/1489714766 learned_addr learned my addr 172.21.15.88:0/1489714766 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa95870700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa901004e0 0x7faa90073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa9506f700 1 -- 172.21.15.88:0/1489714766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa901004e0 msgr2=0x7faa90073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:34.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa9506f700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa901004e0 0x7faa90073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa9506f700 1 -- 172.21.15.88:0/1489714766 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c0095f0 con 0x7faa90100e80 2024-02-19T19:15:34.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa95870700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa901004e0 0x7faa90073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:15:34.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.015+0000 7faa9506f700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa90100e80 0x7faa90073590 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faa8000b480 tx=0x7faa8000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:34.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.016+0000 7faa86ffd700 1 -- 172.21.15.88:0/1489714766 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa800118c0 con 0x7faa90100e80 2024-02-19T19:15:34.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.016+0000 7faa86ffd700 1 -- 172.21.15.88:0/1489714766 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa80010380 con 0x7faa90100e80 2024-02-19T19:15:34.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.016+0000 7faa97ad4700 1 -- 172.21.15.88:0/1489714766 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa901a2160 con 0x7faa90100e80 2024-02-19T19:15:34.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.016+0000 7faa86ffd700 1 -- 172.21.15.88:0/1489714766 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa8000f4f0 con 0x7faa90100e80 2024-02-19T19:15:34.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.016+0000 7faa97ad4700 1 -- 172.21.15.88:0/1489714766 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa901a2570 con 0x7faa90100e80 2024-02-19T19:15:34.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.017+0000 7faa97ad4700 1 -- 172.21.15.88:0/1489714766 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa901087f0 con 0x7faa90100e80 2024-02-19T19:15:34.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.018+0000 7faa86ffd700 1 -- 172.21.15.88:0/1489714766 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7faa800104f0 con 0x7faa90100e80 2024-02-19T19:15:34.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.018+0000 7faa86ffd700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa7c06d8a0 0x7faa7c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:34.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.018+0000 7faa86ffd700 1 -- 172.21.15.88:0/1489714766 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faa8008d180 con 0x7faa90100e80 2024-02-19T19:15:34.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.018+0000 7faa95870700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa7c06d8a0 0x7faa7c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:34.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.019+0000 7faa95870700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa7c06d8a0 0x7faa7c06fd50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faa8c005100 tx=0x7faa8c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:34.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.024+0000 7faa86ffd700 1 -- 172.21.15.88:0/1489714766 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa80056930 con 0x7faa90100e80 2024-02-19T19:15:34.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.302+0000 7faa97ad4700 1 -- 172.21.15.88:0/1489714766 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faa9004f2a0 con 0x7faa90100e80 2024-02-19T19:15:34.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.303+0000 7faa86ffd700 1 -- 172.21.15.88:0/1489714766 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7faa80059ff0 con 0x7faa90100e80 2024-02-19T19:15:34.323 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:15:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:15:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:15:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:15:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:15:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:15:34.324 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:15:34.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.305+0000 7faa97ad4700 1 -- 172.21.15.88:0/1489714766 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa7c06d8a0 msgr2=0x7faa7c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.305+0000 7faa97ad4700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa7c06d8a0 0x7faa7c06fd50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faa8c005100 tx=0x7faa8c01a040 comp rx=0 tx=0).stop 2024-02-19T19:15:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.305+0000 7faa97ad4700 1 -- 172.21.15.88:0/1489714766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa90100e80 msgr2=0x7faa90073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.305+0000 7faa97ad4700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa90100e80 0x7faa90073590 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7faa8000b480 tx=0x7faa8000b840 comp rx=0 tx=0).stop 2024-02-19T19:15:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.305+0000 7faa97ad4700 1 -- 172.21.15.88:0/1489714766 shutdown_connections 2024-02-19T19:15:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.305+0000 7faa97ad4700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa7c06d8a0 0x7faa7c06fd50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.305+0000 7faa97ad4700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa901004e0 0x7faa90073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.305+0000 7faa97ad4700 1 --2- 172.21.15.88:0/1489714766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa90100e80 0x7faa90073590 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.305+0000 7faa97ad4700 1 -- 172.21.15.88:0/1489714766 >> 172.21.15.88:0/1489714766 conn(0x7faa900fbda0 msgr2=0x7faa900fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.305+0000 7faa97ad4700 1 -- 172.21.15.88:0/1489714766 shutdown_connections 2024-02-19T19:15:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.305+0000 7faa97ad4700 1 -- 172.21.15.88:0/1489714766 wait complete. 2024-02-19T19:15:34.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.410+0000 7f430607a700 1 -- 172.21.15.88:0/2841019300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43000ff350 msgr2=0x7f43000ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:34.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.410+0000 7f430607a700 1 --2- 172.21.15.88:0/2841019300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43000ff350 0x7f43000ff7c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009910 tx=0x7f42f0009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:34.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.410+0000 7f430607a700 1 -- 172.21.15.88:0/2841019300 shutdown_connections 2024-02-19T19:15:34.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.410+0000 7f430607a700 1 --2- 172.21.15.88:0/2841019300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43000ff350 0x7f43000ff7c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.410+0000 7f430607a700 1 --2- 172.21.15.88:0/2841019300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43001038b0 0x7f4300103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.410+0000 7f430607a700 1 -- 172.21.15.88:0/2841019300 >> 172.21.15.88:0/2841019300 conn(0x7f43000748b0 msgr2=0x7f4300074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:34.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.410+0000 7f430607a700 1 -- 172.21.15.88:0/2841019300 shutdown_connections 2024-02-19T19:15:34.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.410+0000 7f430607a700 1 -- 172.21.15.88:0/2841019300 wait complete. 2024-02-19T19:15:34.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.411+0000 7f430607a700 1 Processor -- start 2024-02-19T19:15:34.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.411+0000 7f430607a700 1 -- start start 2024-02-19T19:15:34.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.411+0000 7f430607a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43000ff350 0x7f4300196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.411+0000 7f430607a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43001038b0 0x7f43001974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.411+0000 7f430607a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4300197b10 con 0x7f43001038b0 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.411+0000 7f430607a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4300197c50 con 0x7f43000ff350 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.411+0000 7f42feffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43001038b0 0x7f43001974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f42feffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43001038b0 0x7f43001974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41196/0 (socket says 172.21.15.88:41196) 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f42feffd700 1 -- 172.21.15.88:0/3930153508 learned_addr learned my addr 172.21.15.88:0/3930153508 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f42ff7fe700 1 --2- 172.21.15.88:0/3930153508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43000ff350 0x7f4300196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f42feffd700 1 -- 172.21.15.88:0/3930153508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43000ff350 msgr2=0x7f4300196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f42feffd700 1 --2- 172.21.15.88:0/3930153508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43000ff350 0x7f4300196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f42feffd700 1 -- 172.21.15.88:0/3930153508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f00095f0 con 0x7f43001038b0 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f42feffd700 1 --2- 172.21.15.88:0/3930153508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43001038b0 0x7f43001974f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f42f000bd70 tx=0x7f42f000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f42fcff9700 1 -- 172.21.15.88:0/3930153508 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f001d070 con 0x7f43001038b0 2024-02-19T19:15:34.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f430607a700 1 -- 172.21.15.88:0/3930153508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43001a0020 con 0x7f43001038b0 2024-02-19T19:15:34.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f42fcff9700 1 -- 172.21.15.88:0/3930153508 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42f000f460 con 0x7f43001038b0 2024-02-19T19:15:34.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f42fcff9700 1 -- 172.21.15.88:0/3930153508 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f0017630 con 0x7f43001038b0 2024-02-19T19:15:34.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.412+0000 7f430607a700 1 -- 172.21.15.88:0/3930153508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43001a0430 con 0x7f43001038b0 2024-02-19T19:15:34.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.413+0000 7f430607a700 1 -- 172.21.15.88:0/3930153508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430004f2a0 con 0x7f43001038b0 2024-02-19T19:15:34.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.414+0000 7f42fcff9700 1 -- 172.21.15.88:0/3930153508 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f42f000f5d0 con 0x7f43001038b0 2024-02-19T19:15:34.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.414+0000 7f42fcff9700 1 --2- 172.21.15.88:0/3930153508 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42e806d8a0 0x7f42e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:34.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.414+0000 7f42fcff9700 1 -- 172.21.15.88:0/3930153508 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f42f008e9d0 con 0x7f43001038b0 2024-02-19T19:15:34.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.414+0000 7f42ff7fe700 1 --2- 172.21.15.88:0/3930153508 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42e806d8a0 0x7f42e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:34.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.415+0000 7f42ff7fe700 1 --2- 172.21.15.88:0/3930153508 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42e806d8a0 0x7f42e806fd50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009a10 tx=0x7f42f4009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:34.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.419+0000 7f42fcff9700 1 -- 172.21.15.88:0/3930153508 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42f0058200 con 0x7f43001038b0 2024-02-19T19:15:34.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:34 smithi088 ceph-mon[88643]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:34.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:34 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1489714766' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:15:34.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.665+0000 7f430607a700 1 -- 172.21.15.88:0/3930153508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4300105cb0 con 0x7f43001038b0 2024-02-19T19:15:34.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.665+0000 7f42fcff9700 1 -- 172.21.15.88:0/3930153508 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f42f005b8c0 con 0x7f43001038b0 2024-02-19T19:15:34.686 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:15:34.686 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:15:34.686 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:15:34.686 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:15:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:15:34.686 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:15:34.686 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:15:34.686 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:15:34.686 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:15:34.687 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:15:34.688 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:15:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:15:34.689 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:15:34.689 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:15:34.689 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:15:34.689 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.667+0000 7f430607a700 1 -- 172.21.15.88:0/3930153508 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42e806d8a0 msgr2=0x7f42e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.667+0000 7f430607a700 1 --2- 172.21.15.88:0/3930153508 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42e806d8a0 0x7f42e806fd50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009a10 tx=0x7f42f4009190 comp rx=0 tx=0).stop 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.667+0000 7f430607a700 1 -- 172.21.15.88:0/3930153508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43001038b0 msgr2=0x7f43001974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.667+0000 7f430607a700 1 --2- 172.21.15.88:0/3930153508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43001038b0 0x7f43001974f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f42f000bd70 tx=0x7f42f000bda0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.668+0000 7f430607a700 1 -- 172.21.15.88:0/3930153508 shutdown_connections 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.668+0000 7f430607a700 1 --2- 172.21.15.88:0/3930153508 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42e806d8a0 0x7f42e806fd50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.668+0000 7f430607a700 1 --2- 172.21.15.88:0/3930153508 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43000ff350 0x7f4300196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.668+0000 7f430607a700 1 --2- 172.21.15.88:0/3930153508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43001038b0 0x7f43001974f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.668+0000 7f430607a700 1 -- 172.21.15.88:0/3930153508 >> 172.21.15.88:0/3930153508 conn(0x7f43000748b0 msgr2=0x7f4300108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.668+0000 7f430607a700 1 -- 172.21.15.88:0/3930153508 shutdown_connections 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.668+0000 7f430607a700 1 -- 172.21.15.88:0/3930153508 wait complete. 2024-02-19T19:15:34.690 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:15:34.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:34 smithi132 ceph-mon[70676]: from='client.34694 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:34.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:34 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1489714766' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:15:34.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.773+0000 7fb01bdf4700 1 -- 172.21.15.88:0/2805865861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0140ff3c0 msgr2=0x7fb0140ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:34.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.773+0000 7fb01bdf4700 1 --2- 172.21.15.88:0/2805865861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0140ff3c0 0x7fb0140ff790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb004009910 tx=0x7fb004009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:34.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.773+0000 7fb01bdf4700 1 -- 172.21.15.88:0/2805865861 shutdown_connections 2024-02-19T19:15:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.773+0000 7fb01bdf4700 1 --2- 172.21.15.88:0/2805865861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0140ffd60 0x7fb014103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.773+0000 7fb01bdf4700 1 --2- 172.21.15.88:0/2805865861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0140ff3c0 0x7fb0140ff790 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.773+0000 7fb01bdf4700 1 -- 172.21.15.88:0/2805865861 >> 172.21.15.88:0/2805865861 conn(0x7fb0140fac80 msgr2=0x7fb0140fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.773+0000 7fb01bdf4700 1 -- 172.21.15.88:0/2805865861 shutdown_connections 2024-02-19T19:15:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.773+0000 7fb01bdf4700 1 -- 172.21.15.88:0/2805865861 wait complete. 2024-02-19T19:15:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01bdf4700 1 Processor -- start 2024-02-19T19:15:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01bdf4700 1 -- start start 2024-02-19T19:15:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01bdf4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0140ff3c0 0x7fb014195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01bdf4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0140ffd60 0x7fb0141964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01bdf4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb014196bb0 con 0x7fb0140ffd60 2024-02-19T19:15:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01bdf4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01419a940 con 0x7fb0140ff3c0 2024-02-19T19:15:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01938f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0140ffd60 0x7fb0141964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01938f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0140ffd60 0x7fb0141964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41214/0 (socket says 172.21.15.88:41214) 2024-02-19T19:15:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01938f700 1 -- 172.21.15.88:0/3538434720 learned_addr learned my addr 172.21.15.88:0/3538434720 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01938f700 1 -- 172.21.15.88:0/3538434720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0140ff3c0 msgr2=0x7fb014195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01938f700 1 --2- 172.21.15.88:0/3538434720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0140ff3c0 0x7fb014195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01938f700 1 -- 172.21.15.88:0/3538434720 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0040095f0 con 0x7fb0140ffd60 2024-02-19T19:15:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.774+0000 7fb01938f700 1 --2- 172.21.15.88:0/3538434720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0140ffd60 0x7fb0141964d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb01000b480 tx=0x7fb01000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.775+0000 7fb00affd700 1 -- 172.21.15.88:0/3538434720 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0100118c0 con 0x7fb0140ffd60 2024-02-19T19:15:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.775+0000 7fb00affd700 1 -- 172.21.15.88:0/3538434720 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb010010380 con 0x7fb0140ffd60 2024-02-19T19:15:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.775+0000 7fb01bdf4700 1 -- 172.21.15.88:0/3538434720 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01419ac20 con 0x7fb0140ffd60 2024-02-19T19:15:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.775+0000 7fb00affd700 1 -- 172.21.15.88:0/3538434720 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01000f4f0 con 0x7fb0140ffd60 2024-02-19T19:15:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.775+0000 7fb01bdf4700 1 -- 172.21.15.88:0/3538434720 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01419b170 con 0x7fb0140ffd60 2024-02-19T19:15:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.776+0000 7fb01bdf4700 1 -- 172.21.15.88:0/3538434720 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0141076d0 con 0x7fb0140ffd60 2024-02-19T19:15:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.776+0000 7fb00affd700 1 -- 172.21.15.88:0/3538434720 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb0100104f0 con 0x7fb0140ffd60 2024-02-19T19:15:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.777+0000 7fb00affd700 1 --2- 172.21.15.88:0/3538434720 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb00006d8a0 0x7fb00006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.777+0000 7fb00affd700 1 -- 172.21.15.88:0/3538434720 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb01008d180 con 0x7fb0140ffd60 2024-02-19T19:15:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.777+0000 7fb019b90700 1 --2- 172.21.15.88:0/3538434720 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb00006d8a0 0x7fb00006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.777+0000 7fb019b90700 1 --2- 172.21.15.88:0/3538434720 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb00006d8a0 0x7fb00006fd50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb00400b5c0 tx=0x7fb004005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:34.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.781+0000 7fb00affd700 1 -- 172.21.15.88:0/3538434720 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb010056930 con 0x7fb0140ffd60 2024-02-19T19:15:35.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.998+0000 7fb01bdf4700 1 -- 172.21.15.88:0/3538434720 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb01419b460 con 0x7fb00006d8a0 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:34.999+0000 7fb00affd700 1 -- 172.21.15.88:0/3538434720 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb01419b460 con 0x7fb00006d8a0 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:15:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:15:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:15:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:15:35.021 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:15:35.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.001+0000 7fb01bdf4700 1 -- 172.21.15.88:0/3538434720 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb00006d8a0 msgr2=0x7fb00006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:35.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.002+0000 7fb01bdf4700 1 --2- 172.21.15.88:0/3538434720 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb00006d8a0 0x7fb00006fd50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb00400b5c0 tx=0x7fb004005960 comp rx=0 tx=0).stop 2024-02-19T19:15:35.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.002+0000 7fb01bdf4700 1 -- 172.21.15.88:0/3538434720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0140ffd60 msgr2=0x7fb0141964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:35.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.002+0000 7fb01bdf4700 1 --2- 172.21.15.88:0/3538434720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0140ffd60 0x7fb0141964d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb01000b480 tx=0x7fb01000b840 comp rx=0 tx=0).stop 2024-02-19T19:15:35.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.002+0000 7fb01bdf4700 1 -- 172.21.15.88:0/3538434720 shutdown_connections 2024-02-19T19:15:35.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.002+0000 7fb01bdf4700 1 --2- 172.21.15.88:0/3538434720 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb00006d8a0 0x7fb00006fd50 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:35.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.002+0000 7fb01bdf4700 1 --2- 172.21.15.88:0/3538434720 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb0140ff3c0 0x7fb014195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:35.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.002+0000 7fb01bdf4700 1 --2- 172.21.15.88:0/3538434720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0140ffd60 0x7fb0141964d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:35.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.002+0000 7fb01bdf4700 1 -- 172.21.15.88:0/3538434720 >> 172.21.15.88:0/3538434720 conn(0x7fb0140fac80 msgr2=0x7fb0140fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:35.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.002+0000 7fb01bdf4700 1 -- 172.21.15.88:0/3538434720 shutdown_connections 2024-02-19T19:15:35.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.002+0000 7fb01bdf4700 1 -- 172.21.15.88:0/3538434720 wait complete. 2024-02-19T19:15:35.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.105+0000 7ff79c746700 1 -- 172.21.15.88:0/1532248222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7940ffd50 msgr2=0x7ff794103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:35.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.105+0000 7ff79c746700 1 --2- 172.21.15.88:0/1532248222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7940ffd50 0x7ff794103d20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff790009910 tx=0x7ff790009c20 comp rx=0 tx=0).stop 2024-02-19T19:15:35.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.105+0000 7ff79c746700 1 -- 172.21.15.88:0/1532248222 shutdown_connections 2024-02-19T19:15:35.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.105+0000 7ff79c746700 1 --2- 172.21.15.88:0/1532248222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7940ffd50 0x7ff794103d20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:35.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.105+0000 7ff79c746700 1 --2- 172.21.15.88:0/1532248222 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7940ff3b0 0x7ff7940ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:35.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.105+0000 7ff79c746700 1 -- 172.21.15.88:0/1532248222 >> 172.21.15.88:0/1532248222 conn(0x7ff7940fac90 msgr2=0x7ff7940fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:35.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.105+0000 7ff79c746700 1 -- 172.21.15.88:0/1532248222 shutdown_connections 2024-02-19T19:15:35.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.105+0000 7ff79c746700 1 -- 172.21.15.88:0/1532248222 wait complete. 2024-02-19T19:15:35.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79c746700 1 Processor -- start 2024-02-19T19:15:35.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79c746700 1 -- start start 2024-02-19T19:15:35.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79c746700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7940ff3b0 0x7ff794194e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79c746700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7940ffd50 0x7ff794195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79c746700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff794195a20 con 0x7ff7940ff3b0 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79c746700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7941997b0 con 0x7ff7940ffd50 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79a4e2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7940ff3b0 0x7ff794194e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79a4e2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7940ff3b0 0x7ff794194e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41228/0 (socket says 172.21.15.88:41228) 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79a4e2700 1 -- 172.21.15.88:0/2529391175 learned_addr learned my addr 172.21.15.88:0/2529391175 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff799ce1700 1 --2- 172.21.15.88:0/2529391175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7940ffd50 0x7ff794195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79a4e2700 1 -- 172.21.15.88:0/2529391175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7940ffd50 msgr2=0x7ff794195340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79a4e2700 1 --2- 172.21.15.88:0/2529391175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7940ffd50 0x7ff794195340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.106+0000 7ff79a4e2700 1 -- 172.21.15.88:0/2529391175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900095f0 con 0x7ff7940ff3b0 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.107+0000 7ff79a4e2700 1 --2- 172.21.15.88:0/2529391175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7940ff3b0 0x7ff794194e00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff784009970 tx=0x7ff784009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.107+0000 7ff78b7fe700 1 -- 172.21.15.88:0/2529391175 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff784012070 con 0x7ff7940ff3b0 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.107+0000 7ff78b7fe700 1 -- 172.21.15.88:0/2529391175 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff784007af0 con 0x7ff7940ff3b0 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.107+0000 7ff79c746700 1 -- 172.21.15.88:0/2529391175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff794199a30 con 0x7ff7940ff3b0 2024-02-19T19:15:35.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.107+0000 7ff78b7fe700 1 -- 172.21.15.88:0/2529391175 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff784017c60 con 0x7ff7940ff3b0 2024-02-19T19:15:35.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.107+0000 7ff79c746700 1 -- 172.21.15.88:0/2529391175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff794199f80 con 0x7ff7940ff3b0 2024-02-19T19:15:35.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.108+0000 7ff79c746700 1 -- 172.21.15.88:0/2529391175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79404f2a0 con 0x7ff7940ff3b0 2024-02-19T19:15:35.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.108+0000 7ff78b7fe700 1 -- 172.21.15.88:0/2529391175 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff784007c60 con 0x7ff7940ff3b0 2024-02-19T19:15:35.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.109+0000 7ff78b7fe700 1 --2- 172.21.15.88:0/2529391175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff78006d8a0 0x7ff78006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:15:35.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.109+0000 7ff78b7fe700 1 -- 172.21.15.88:0/2529391175 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff78408ead0 con 0x7ff7940ff3b0 2024-02-19T19:15:35.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.109+0000 7ff799ce1700 1 --2- 172.21.15.88:0/2529391175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff78006d8a0 0x7ff78006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:15:35.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.109+0000 7ff799ce1700 1 --2- 172.21.15.88:0/2529391175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff78006d8a0 0x7ff78006fd50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff794196420 tx=0x7ff79001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:15:35.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.113+0000 7ff78b7fe700 1 -- 172.21.15.88:0/2529391175 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff784058330 con 0x7ff7940ff3b0 2024-02-19T19:15:35.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.392+0000 7ff79c746700 1 -- 172.21.15.88:0/2529391175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff79419a2a0 con 0x7ff7940ff3b0 2024-02-19T19:15:35.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.393+0000 7ff78b7fe700 1 -- 172.21.15.88:0/2529391175 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff78405b9f0 con 0x7ff7940ff3b0 2024-02-19T19:15:35.413 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:15:35.413 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:15:35.413 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:15:35.413 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:15:35.413 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:15:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.395+0000 7ff79c746700 1 -- 172.21.15.88:0/2529391175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff78006d8a0 msgr2=0x7ff78006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.395+0000 7ff79c746700 1 --2- 172.21.15.88:0/2529391175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff78006d8a0 0x7ff78006fd50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff794196420 tx=0x7ff79001a040 comp rx=0 tx=0).stop 2024-02-19T19:15:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.395+0000 7ff79c746700 1 -- 172.21.15.88:0/2529391175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7940ff3b0 msgr2=0x7ff794194e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:15:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.395+0000 7ff79c746700 1 --2- 172.21.15.88:0/2529391175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7940ff3b0 0x7ff794194e00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ff784009970 tx=0x7ff784009d30 comp rx=0 tx=0).stop 2024-02-19T19:15:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.395+0000 7ff79c746700 1 -- 172.21.15.88:0/2529391175 shutdown_connections 2024-02-19T19:15:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.395+0000 7ff79c746700 1 --2- 172.21.15.88:0/2529391175 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff78006d8a0 0x7ff78006fd50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:35.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.395+0000 7ff79c746700 1 --2- 172.21.15.88:0/2529391175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7940ff3b0 0x7ff794194e00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:35.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.395+0000 7ff79c746700 1 --2- 172.21.15.88:0/2529391175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff7940ffd50 0x7ff794195340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:15:35.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.395+0000 7ff79c746700 1 -- 172.21.15.88:0/2529391175 >> 172.21.15.88:0/2529391175 conn(0x7ff7940fac90 msgr2=0x7ff7940fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:15:35.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.395+0000 7ff79c746700 1 -- 172.21.15.88:0/2529391175 shutdown_connections 2024-02-19T19:15:35.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:15:35.395+0000 7ff79c746700 1 -- 172.21.15.88:0/2529391175 wait complete. 2024-02-19T19:15:35.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:35 smithi132 ceph-mon[70676]: from='client.34698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:35.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:35 smithi132 ceph-mon[70676]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:35.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:35 smithi132 ceph-mon[70676]: pgmap v359: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:35.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:35 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3930153508' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:15:35.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:35 smithi088 ceph-mon[88643]: from='client.34698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:35.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:35 smithi088 ceph-mon[88643]: from='client.34702 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:35.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:35 smithi088 ceph-mon[88643]: pgmap v359: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:35.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:35 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3930153508' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:15:36.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:36 smithi132 ceph-mon[70676]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:36.753 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:36 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2529391175' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:15:36.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:36 smithi088 ceph-mon[88643]: from='client.34714 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:15:36.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:36 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2529391175' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:15:37.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:37 smithi132 ceph-mon[70676]: pgmap v360: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:37.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:37 smithi088 ceph-mon[88643]: pgmap v360: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:39.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:39 smithi132 ceph-mon[70676]: pgmap v361: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:39.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:39 smithi088 ceph-mon[88643]: pgmap v361: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:41.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:41 smithi132 ceph-mon[70676]: pgmap v362: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:41.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:41 smithi088 ceph-mon[88643]: pgmap v362: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:43.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:43 smithi132 ceph-mon[70676]: pgmap v363: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:43.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:15:43.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:43 smithi088 ceph-mon[88643]: pgmap v363: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:43.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:15:45.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:45 smithi132 ceph-mon[70676]: pgmap v364: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:45.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:45 smithi088 ceph-mon[88643]: pgmap v364: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:47.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:47 smithi132 ceph-mon[70676]: pgmap v365: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:47.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:47 smithi088 ceph-mon[88643]: pgmap v365: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:49.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:49 smithi132 ceph-mon[70676]: pgmap v366: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:49.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:49 smithi088 ceph-mon[88643]: pgmap v366: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:51.600 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:51 smithi088 ceph-mon[88643]: pgmap v367: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:51.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:51 smithi132 ceph-mon[70676]: pgmap v367: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:53.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:53 smithi132 ceph-mon[70676]: pgmap v368: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:53.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:53 smithi088 ceph-mon[88643]: pgmap v368: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:55.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:55 smithi132 ceph-mon[70676]: pgmap v369: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:55.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:55 smithi088 ceph-mon[88643]: pgmap v369: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:57.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:57 smithi132 ceph-mon[70676]: pgmap v370: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:57.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:57 smithi088 ceph-mon[88643]: pgmap v370: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:58.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:15:58.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:15:59.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:15:59 smithi132 ceph-mon[70676]: pgmap v371: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:15:59.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:15:59 smithi088 ceph-mon[88643]: pgmap v371: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:01.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:01 smithi088 ceph-mon[88643]: pgmap v372: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:01.501 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:01 smithi132 ceph-mon[70676]: pgmap v372: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:03.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:03 smithi088 ceph-mon[88643]: pgmap v373: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:03.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:03 smithi132 ceph-mon[70676]: pgmap v373: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:05.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.511+0000 7fa227bfa700 1 -- 172.21.15.88:0/1002804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2200ffd60 msgr2=0x7fa220103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:05.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.511+0000 7fa227bfa700 1 --2- 172.21.15.88:0/1002804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2200ffd60 0x7fa220103d30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa210009910 tx=0x7fa210009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:05.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.511+0000 7fa227bfa700 1 -- 172.21.15.88:0/1002804927 shutdown_connections 2024-02-19T19:16:05.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.511+0000 7fa227bfa700 1 --2- 172.21.15.88:0/1002804927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2200ffd60 0x7fa220103d30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:05.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.511+0000 7fa227bfa700 1 --2- 172.21.15.88:0/1002804927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2200ff3c0 0x7fa2200ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:05.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.511+0000 7fa227bfa700 1 -- 172.21.15.88:0/1002804927 >> 172.21.15.88:0/1002804927 conn(0x7fa2200fac80 msgr2=0x7fa2200fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:05.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.511+0000 7fa227bfa700 1 -- 172.21.15.88:0/1002804927 shutdown_connections 2024-02-19T19:16:05.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.511+0000 7fa227bfa700 1 -- 172.21.15.88:0/1002804927 wait complete. 2024-02-19T19:16:05.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa227bfa700 1 Processor -- start 2024-02-19T19:16:05.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa227bfa700 1 -- start start 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa227bfa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2200ff3c0 0x7fa220196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa227bfa700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2200ffd60 0x7fa220197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa227bfa700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2201979b0 con 0x7fa2200ff3c0 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa227bfa700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa220197af0 con 0x7fa2200ffd60 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa225996700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2200ff3c0 0x7fa220196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa225996700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2200ff3c0 0x7fa220196e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34584/0 (socket says 172.21.15.88:34584) 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa225996700 1 -- 172.21.15.88:0/3597817912 learned_addr learned my addr 172.21.15.88:0/3597817912 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa225996700 1 -- 172.21.15.88:0/3597817912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2200ffd60 msgr2=0x7fa220197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa225996700 1 --2- 172.21.15.88:0/3597817912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2200ffd60 0x7fa220197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa225996700 1 -- 172.21.15.88:0/3597817912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2100095f0 con 0x7fa2200ff3c0 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.512+0000 7fa225996700 1 --2- 172.21.15.88:0/3597817912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2200ff3c0 0x7fa220196e50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00b4b0 tx=0x7fa21c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:05.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.513+0000 7fa216ffd700 1 -- 172.21.15.88:0/3597817912 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa21c011800 con 0x7fa2200ff3c0 2024-02-19T19:16:05.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.513+0000 7fa227bfa700 1 -- 172.21.15.88:0/3597817912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22019fec0 con 0x7fa2200ff3c0 2024-02-19T19:16:05.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.513+0000 7fa216ffd700 1 -- 172.21.15.88:0/3597817912 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa21c011e20 con 0x7fa2200ff3c0 2024-02-19T19:16:05.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.513+0000 7fa216ffd700 1 -- 172.21.15.88:0/3597817912 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa21c00f2e0 con 0x7fa2200ff3c0 2024-02-19T19:16:05.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.513+0000 7fa227bfa700 1 -- 172.21.15.88:0/3597817912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2201a0380 con 0x7fa2200ff3c0 2024-02-19T19:16:05.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.514+0000 7fa227bfa700 1 -- 172.21.15.88:0/3597817912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa220107660 con 0x7fa2200ff3c0 2024-02-19T19:16:05.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.514+0000 7fa216ffd700 1 -- 172.21.15.88:0/3597817912 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa21c010380 con 0x7fa2200ff3c0 2024-02-19T19:16:05.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.515+0000 7fa216ffd700 1 --2- 172.21.15.88:0/3597817912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa20c06d8a0 0x7fa20c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:05.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.515+0000 7fa216ffd700 1 -- 172.21.15.88:0/3597817912 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa21c08d110 con 0x7fa2200ff3c0 2024-02-19T19:16:05.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.515+0000 7fa225195700 1 --2- 172.21.15.88:0/3597817912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa20c06d8a0 0x7fa20c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:05.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.515+0000 7fa225195700 1 --2- 172.21.15.88:0/3597817912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa20c06d8a0 0x7fa20c06fd50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa210006010 tx=0x7fa21000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:05.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.520+0000 7fa216ffd700 1 -- 172.21.15.88:0/3597817912 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa21c0568c0 con 0x7fa2200ff3c0 2024-02-19T19:16:05.599 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:05 smithi088 ceph-mon[88643]: pgmap v374: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:05.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:05 smithi132 ceph-mon[70676]: pgmap v374: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:05.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.742+0000 7fa227bfa700 1 -- 172.21.15.88:0/3597817912 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa220061190 con 0x7fa20c06d8a0 2024-02-19T19:16:05.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.743+0000 7fa216ffd700 1 -- 172.21.15.88:0/3597817912 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa220061190 con 0x7fa20c06d8a0 2024-02-19T19:16:05.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.745+0000 7fa227bfa700 1 -- 172.21.15.88:0/3597817912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa20c06d8a0 msgr2=0x7fa20c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:05.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.745+0000 7fa227bfa700 1 --2- 172.21.15.88:0/3597817912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa20c06d8a0 0x7fa20c06fd50 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa210006010 tx=0x7fa21000b540 comp rx=0 tx=0).stop 2024-02-19T19:16:05.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.745+0000 7fa227bfa700 1 -- 172.21.15.88:0/3597817912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2200ff3c0 msgr2=0x7fa220196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:05.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.745+0000 7fa227bfa700 1 --2- 172.21.15.88:0/3597817912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2200ff3c0 0x7fa220196e50 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa21c00b4b0 tx=0x7fa21c00b870 comp rx=0 tx=0).stop 2024-02-19T19:16:05.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.745+0000 7fa227bfa700 1 -- 172.21.15.88:0/3597817912 shutdown_connections 2024-02-19T19:16:05.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.745+0000 7fa227bfa700 1 --2- 172.21.15.88:0/3597817912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa20c06d8a0 0x7fa20c06fd50 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:05.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.745+0000 7fa227bfa700 1 --2- 172.21.15.88:0/3597817912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2200ff3c0 0x7fa220196e50 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:05.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.745+0000 7fa227bfa700 1 --2- 172.21.15.88:0/3597817912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2200ffd60 0x7fa220197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:05.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.745+0000 7fa227bfa700 1 -- 172.21.15.88:0/3597817912 >> 172.21.15.88:0/3597817912 conn(0x7fa2200fac80 msgr2=0x7fa2200fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:05.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.745+0000 7fa227bfa700 1 -- 172.21.15.88:0/3597817912 shutdown_connections 2024-02-19T19:16:05.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.745+0000 7fa227bfa700 1 -- 172.21.15.88:0/3597817912 wait complete. 2024-02-19T19:16:05.776 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:16:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.849+0000 7fec8c913700 1 -- 172.21.15.88:0/1024297347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec880ffd60 msgr2=0x7fec88103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.849+0000 7fec8c913700 1 --2- 172.21.15.88:0/1024297347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec880ffd60 0x7fec88103d30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fec7c009910 tx=0x7fec7c009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.849+0000 7fec8c913700 1 -- 172.21.15.88:0/1024297347 shutdown_connections 2024-02-19T19:16:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.849+0000 7fec8c913700 1 --2- 172.21.15.88:0/1024297347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec880ffd60 0x7fec88103d30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.849+0000 7fec8c913700 1 --2- 172.21.15.88:0/1024297347 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec880ff3c0 0x7fec880ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.849+0000 7fec8c913700 1 -- 172.21.15.88:0/1024297347 >> 172.21.15.88:0/1024297347 conn(0x7fec880fac80 msgr2=0x7fec880fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.849+0000 7fec8c913700 1 -- 172.21.15.88:0/1024297347 shutdown_connections 2024-02-19T19:16:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.849+0000 7fec8c913700 1 -- 172.21.15.88:0/1024297347 wait complete. 2024-02-19T19:16:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.849+0000 7fec8c913700 1 Processor -- start 2024-02-19T19:16:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.849+0000 7fec8c913700 1 -- start start 2024-02-19T19:16:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8c913700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec880ff3c0 0x7fec88100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8c913700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec880ffd60 0x7fec88101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8c913700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec88101850 con 0x7fec880ff3c0 2024-02-19T19:16:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8c913700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec88101990 con 0x7fec880ffd60 2024-02-19T19:16:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec880ff3c0 0x7fec88100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec880ff3c0 0x7fec88100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34600/0 (socket says 172.21.15.88:34600) 2024-02-19T19:16:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8659c700 1 -- 172.21.15.88:0/2464983590 learned_addr learned my addr 172.21.15.88:0/2464983590 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec85d9b700 1 --2- 172.21.15.88:0/2464983590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec880ffd60 0x7fec88101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8659c700 1 -- 172.21.15.88:0/2464983590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec880ffd60 msgr2=0x7fec88101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8659c700 1 --2- 172.21.15.88:0/2464983590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec880ffd60 0x7fec88101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8659c700 1 -- 172.21.15.88:0/2464983590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c0095f0 con 0x7fec880ff3c0 2024-02-19T19:16:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8659c700 1 --2- 172.21.15.88:0/2464983590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec880ff3c0 0x7fec88100dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fec7800d8d0 tx=0x7fec7800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec777fe700 1 -- 172.21.15.88:0/2464983590 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec780098a0 con 0x7fec880ff3c0 2024-02-19T19:16:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec777fe700 1 -- 172.21.15.88:0/2464983590 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec78010460 con 0x7fec880ff3c0 2024-02-19T19:16:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8c913700 1 -- 172.21.15.88:0/2464983590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec881a2090 con 0x7fec880ff3c0 2024-02-19T19:16:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec777fe700 1 -- 172.21.15.88:0/2464983590 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec7800f5f0 con 0x7fec880ff3c0 2024-02-19T19:16:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.850+0000 7fec8c913700 1 -- 172.21.15.88:0/2464983590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec881a24a0 con 0x7fec880ff3c0 2024-02-19T19:16:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.851+0000 7fec8c913700 1 -- 172.21.15.88:0/2464983590 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8804f2a0 con 0x7fec880ff3c0 2024-02-19T19:16:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.852+0000 7fec777fe700 1 -- 172.21.15.88:0/2464983590 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fec78009a00 con 0x7fec880ff3c0 2024-02-19T19:16:05.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.852+0000 7fec777fe700 1 --2- 172.21.15.88:0/2464983590 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fec7006d8a0 0x7fec7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:05.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.852+0000 7fec777fe700 1 -- 172.21.15.88:0/2464983590 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fec7808d400 con 0x7fec880ff3c0 2024-02-19T19:16:05.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.852+0000 7fec85d9b700 1 --2- 172.21.15.88:0/2464983590 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fec7006d8a0 0x7fec7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:05.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.853+0000 7fec85d9b700 1 --2- 172.21.15.88:0/2464983590 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fec7006d8a0 0x7fec7006fd50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fec881025c0 tx=0x7fec7c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:05.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:05.858+0000 7fec777fe700 1 -- 172.21.15.88:0/2464983590 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fec78056bb0 con 0x7fec880ff3c0 2024-02-19T19:16:06.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.071+0000 7fec8c913700 1 -- 172.21.15.88:0/2464983590 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec88102ef0 con 0x7fec7006d8a0 2024-02-19T19:16:06.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.072+0000 7fec777fe700 1 -- 172.21.15.88:0/2464983590 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fec88102ef0 con 0x7fec7006d8a0 2024-02-19T19:16:06.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.075+0000 7fec8c913700 1 -- 172.21.15.88:0/2464983590 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fec7006d8a0 msgr2=0x7fec7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.075+0000 7fec8c913700 1 --2- 172.21.15.88:0/2464983590 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fec7006d8a0 0x7fec7006fd50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fec881025c0 tx=0x7fec7c01a040 comp rx=0 tx=0).stop 2024-02-19T19:16:06.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.075+0000 7fec8c913700 1 -- 172.21.15.88:0/2464983590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec880ff3c0 msgr2=0x7fec88100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.075+0000 7fec8c913700 1 --2- 172.21.15.88:0/2464983590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec880ff3c0 0x7fec88100dd0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fec7800d8d0 tx=0x7fec7800dbe0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.076+0000 7fec8c913700 1 -- 172.21.15.88:0/2464983590 shutdown_connections 2024-02-19T19:16:06.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.076+0000 7fec8c913700 1 --2- 172.21.15.88:0/2464983590 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fec7006d8a0 0x7fec7006fd50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.076+0000 7fec8c913700 1 --2- 172.21.15.88:0/2464983590 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec880ff3c0 0x7fec88100dd0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.076+0000 7fec8c913700 1 --2- 172.21.15.88:0/2464983590 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fec880ffd60 0x7fec88101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.076+0000 7fec8c913700 1 -- 172.21.15.88:0/2464983590 >> 172.21.15.88:0/2464983590 conn(0x7fec880fac80 msgr2=0x7fec880fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:06.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.076+0000 7fec8c913700 1 -- 172.21.15.88:0/2464983590 shutdown_connections 2024-02-19T19:16:06.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.076+0000 7fec8c913700 1 -- 172.21.15.88:0/2464983590 wait complete. 2024-02-19T19:16:06.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.179+0000 7fb7d85df700 1 -- 172.21.15.88:0/25380325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d0100e40 msgr2=0x7fb7d0104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.179+0000 7fb7d85df700 1 --2- 172.21.15.88:0/25380325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d0100e40 0x7fb7d0104e10 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009910 tx=0x7fb7c0009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:06.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 -- 172.21.15.88:0/25380325 shutdown_connections 2024-02-19T19:16:06.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 --2- 172.21.15.88:0/25380325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d0100e40 0x7fb7d0104e10 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 --2- 172.21.15.88:0/25380325 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7d01004a0 0x7fb7d0100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 -- 172.21.15.88:0/25380325 >> 172.21.15.88:0/25380325 conn(0x7fb7d00fbd40 msgr2=0x7fb7d00fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:06.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 -- 172.21.15.88:0/25380325 shutdown_connections 2024-02-19T19:16:06.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 -- 172.21.15.88:0/25380325 wait complete. 2024-02-19T19:16:06.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 Processor -- start 2024-02-19T19:16:06.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 -- start start 2024-02-19T19:16:06.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d01004a0 0x7fb7d0198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:06.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7d0100e40 0x7fb7d0198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:06.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d0198d50 con 0x7fb7d01004a0 2024-02-19T19:16:06.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.180+0000 7fb7d85df700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d019cae0 con 0x7fb7d0100e40 2024-02-19T19:16:06.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7d637b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d01004a0 0x7fb7d0198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:06.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7d637b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d01004a0 0x7fb7d0198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34616/0 (socket says 172.21.15.88:34616) 2024-02-19T19:16:06.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7d637b700 1 -- 172.21.15.88:0/199686862 learned_addr learned my addr 172.21.15.88:0/199686862 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:06.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7d5b7a700 1 --2- 172.21.15.88:0/199686862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7d0100e40 0x7fb7d0198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:06.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7d637b700 1 -- 172.21.15.88:0/199686862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7d0100e40 msgr2=0x7fb7d0198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7d637b700 1 --2- 172.21.15.88:0/199686862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7d0100e40 0x7fb7d0198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7d637b700 1 -- 172.21.15.88:0/199686862 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c00095f0 con 0x7fb7d01004a0 2024-02-19T19:16:06.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7d637b700 1 --2- 172.21.15.88:0/199686862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d01004a0 0x7fb7d0198130 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00b4b0 tx=0x7fb7cc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:06.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7c77fe700 1 -- 172.21.15.88:0/199686862 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7cc011800 con 0x7fb7d01004a0 2024-02-19T19:16:06.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7d85df700 1 -- 172.21.15.88:0/199686862 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d019cdc0 con 0x7fb7d01004a0 2024-02-19T19:16:06.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7c77fe700 1 -- 172.21.15.88:0/199686862 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7cc011e20 con 0x7fb7d01004a0 2024-02-19T19:16:06.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7c77fe700 1 -- 172.21.15.88:0/199686862 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7cc00f2e0 con 0x7fb7d01004a0 2024-02-19T19:16:06.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.181+0000 7fb7d85df700 1 -- 172.21.15.88:0/199686862 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d019d2e0 con 0x7fb7d01004a0 2024-02-19T19:16:06.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.182+0000 7fb7d85df700 1 -- 172.21.15.88:0/199686862 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d0104590 con 0x7fb7d01004a0 2024-02-19T19:16:06.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.183+0000 7fb7c77fe700 1 -- 172.21.15.88:0/199686862 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb7cc010380 con 0x7fb7d01004a0 2024-02-19T19:16:06.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.183+0000 7fb7c77fe700 1 --2- 172.21.15.88:0/199686862 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7bc06d8a0 0x7fb7bc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:06.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.183+0000 7fb7c77fe700 1 -- 172.21.15.88:0/199686862 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb7cc08d070 con 0x7fb7d01004a0 2024-02-19T19:16:06.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.183+0000 7fb7d5b7a700 1 --2- 172.21.15.88:0/199686862 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7bc06d8a0 0x7fb7bc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:06.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.184+0000 7fb7d5b7a700 1 --2- 172.21.15.88:0/199686862 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7bc06d8a0 0x7fb7bc06fd50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0006010 tx=0x7fb7c001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:06.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.188+0000 7fb7c77fe700 1 -- 172.21.15.88:0/199686862 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb7cc056820 con 0x7fb7d01004a0 2024-02-19T19:16:06.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.396+0000 7fb7d85df700 1 -- 172.21.15.88:0/199686862 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7d019d5c0 con 0x7fb7bc06d8a0 2024-02-19T19:16:06.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.407+0000 7fb7c77fe700 1 -- 172.21.15.88:0/199686862 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb7d019d5c0 con 0x7fb7bc06d8a0 2024-02-19T19:16:06.426 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (13m) 5m ago 23m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (24m) 5m ago 24m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (6m) 4m ago 22m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (9m) 5m ago 24m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (9m) 4m ago 22m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (12m) 5m ago 23m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (5m) 5m ago 17m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (5m) 5m ago 17m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (4m) 4m ago 17m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (17m) 4m ago 17m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (14m) 5m ago 25m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (14m) 4m ago 22m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:16:06.427 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (11m) 5m ago 25m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:16:06.428 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (10m) 4m ago 21m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:16:06.428 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (13m) 5m ago 24m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:16:06.428 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (13m) 4m ago 22m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:16:06.428 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (8m) 5m ago 21m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:16:06.428 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (8m) 5m ago 20m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:16:06.428 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (7m) 5m ago 20m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:16:06.428 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (6m) 4m ago 19m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:16:06.428 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (6m) 4m ago 19m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:16:06.428 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (5m) 4m ago 18m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:16:06.428 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (13m) 5m ago 22m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:16:06.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.410+0000 7fb7d85df700 1 -- 172.21.15.88:0/199686862 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7bc06d8a0 msgr2=0x7fb7bc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.410+0000 7fb7d85df700 1 --2- 172.21.15.88:0/199686862 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7bc06d8a0 0x7fb7bc06fd50 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0006010 tx=0x7fb7c001a040 comp rx=0 tx=0).stop 2024-02-19T19:16:06.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.410+0000 7fb7d85df700 1 -- 172.21.15.88:0/199686862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d01004a0 msgr2=0x7fb7d0198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.410+0000 7fb7d85df700 1 --2- 172.21.15.88:0/199686862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d01004a0 0x7fb7d0198130 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc00b4b0 tx=0x7fb7cc00b7c0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.410+0000 7fb7d85df700 1 -- 172.21.15.88:0/199686862 shutdown_connections 2024-02-19T19:16:06.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.410+0000 7fb7d85df700 1 --2- 172.21.15.88:0/199686862 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb7bc06d8a0 0x7fb7bc06fd50 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.410+0000 7fb7d85df700 1 --2- 172.21.15.88:0/199686862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d01004a0 0x7fb7d0198130 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.410+0000 7fb7d85df700 1 --2- 172.21.15.88:0/199686862 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7d0100e40 0x7fb7d0198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.410+0000 7fb7d85df700 1 -- 172.21.15.88:0/199686862 >> 172.21.15.88:0/199686862 conn(0x7fb7d00fbd40 msgr2=0x7fb7d00fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:06.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.410+0000 7fb7d85df700 1 -- 172.21.15.88:0/199686862 shutdown_connections 2024-02-19T19:16:06.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.410+0000 7fb7d85df700 1 -- 172.21.15.88:0/199686862 wait complete. 2024-02-19T19:16:06.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.514+0000 7fb29a48c700 1 -- 172.21.15.88:0/3856922615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb294068540 msgr2=0x7fb2940689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.514+0000 7fb29a48c700 1 --2- 172.21.15.88:0/3856922615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb294068540 0x7fb2940689b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb288009910 tx=0x7fb288009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:06.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.514+0000 7fb29a48c700 1 -- 172.21.15.88:0/3856922615 shutdown_connections 2024-02-19T19:16:06.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.514+0000 7fb29a48c700 1 --2- 172.21.15.88:0/3856922615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb294068540 0x7fb2940689b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.514+0000 7fb29a48c700 1 --2- 172.21.15.88:0/3856922615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb294106320 0x7fb2941066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.514+0000 7fb29a48c700 1 -- 172.21.15.88:0/3856922615 >> 172.21.15.88:0/3856922615 conn(0x7fb294075130 msgr2=0x7fb294075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:06.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.514+0000 7fb29a48c700 1 -- 172.21.15.88:0/3856922615 shutdown_connections 2024-02-19T19:16:06.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.514+0000 7fb29a48c700 1 -- 172.21.15.88:0/3856922615 wait complete. 2024-02-19T19:16:06.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.515+0000 7fb29a48c700 1 Processor -- start 2024-02-19T19:16:06.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.515+0000 7fb29a48c700 1 -- start start 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.515+0000 7fb29a48c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb294068540 0x7fb294195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.515+0000 7fb29a48c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb294106320 0x7fb294196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.515+0000 7fb29a48c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2941969b0 con 0x7fb294068540 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.515+0000 7fb29a48c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb294196af0 con 0x7fb294106320 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.515+0000 7fb293fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb294068540 0x7fb294195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.515+0000 7fb293fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb294068540 0x7fb294195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34634/0 (socket says 172.21.15.88:34634) 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.515+0000 7fb293fff700 1 -- 172.21.15.88:0/2455738857 learned_addr learned my addr 172.21.15.88:0/2455738857 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.516+0000 7fb2937fe700 1 --2- 172.21.15.88:0/2455738857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb294106320 0x7fb294196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.516+0000 7fb293fff700 1 -- 172.21.15.88:0/2455738857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb294106320 msgr2=0x7fb294196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.516+0000 7fb293fff700 1 --2- 172.21.15.88:0/2455738857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb294106320 0x7fb294196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.516+0000 7fb293fff700 1 -- 172.21.15.88:0/2455738857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880095f0 con 0x7fb294068540 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.516+0000 7fb293fff700 1 --2- 172.21.15.88:0/2455738857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb294068540 0x7fb294195e20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c930 tx=0x7fb28400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:06.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.516+0000 7fb2917fa700 1 -- 172.21.15.88:0/2455738857 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb284007ad0 con 0x7fb294068540 2024-02-19T19:16:06.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.516+0000 7fb29a48c700 1 -- 172.21.15.88:0/2455738857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29419a940 con 0x7fb294068540 2024-02-19T19:16:06.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.516+0000 7fb2917fa700 1 -- 172.21.15.88:0/2455738857 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb284007c30 con 0x7fb294068540 2024-02-19T19:16:06.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.516+0000 7fb2917fa700 1 -- 172.21.15.88:0/2455738857 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb284018610 con 0x7fb294068540 2024-02-19T19:16:06.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.516+0000 7fb29a48c700 1 -- 172.21.15.88:0/2455738857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29419ae90 con 0x7fb294068540 2024-02-19T19:16:06.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.517+0000 7fb29a48c700 1 -- 172.21.15.88:0/2455738857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb294108770 con 0x7fb294068540 2024-02-19T19:16:06.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.517+0000 7fb2917fa700 1 -- 172.21.15.88:0/2455738857 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb28400f450 con 0x7fb294068540 2024-02-19T19:16:06.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.518+0000 7fb2917fa700 1 --2- 172.21.15.88:0/2455738857 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb27c06d8a0 0x7fb27c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:06.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.518+0000 7fb2937fe700 1 --2- 172.21.15.88:0/2455738857 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb27c06d8a0 0x7fb27c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:06.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.518+0000 7fb2917fa700 1 -- 172.21.15.88:0/2455738857 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb28408d760 con 0x7fb294068540 2024-02-19T19:16:06.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.519+0000 7fb2937fe700 1 --2- 172.21.15.88:0/2455738857 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb27c06d8a0 0x7fb27c06fd50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb288005f50 tx=0x7fb28801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:06.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.523+0000 7fb2917fa700 1 -- 172.21.15.88:0/2455738857 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb284056f10 con 0x7fb294068540 2024-02-19T19:16:06.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.802+0000 7fb29a48c700 1 -- 172.21.15.88:0/2455738857 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb294108150 con 0x7fb294068540 2024-02-19T19:16:06.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.802+0000 7fb2917fa700 1 -- 172.21.15.88:0/2455738857 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fb28405a5d0 con 0x7fb294068540 2024-02-19T19:16:06.821 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:16:06.822 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:16:06.823 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:16:06.823 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:16:06.823 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:16:06.823 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:16:06.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.804+0000 7fb29a48c700 1 -- 172.21.15.88:0/2455738857 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb27c06d8a0 msgr2=0x7fb27c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.804+0000 7fb29a48c700 1 --2- 172.21.15.88:0/2455738857 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb27c06d8a0 0x7fb27c06fd50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb288005f50 tx=0x7fb28801a040 comp rx=0 tx=0).stop 2024-02-19T19:16:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.804+0000 7fb29a48c700 1 -- 172.21.15.88:0/2455738857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb294068540 msgr2=0x7fb294195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.804+0000 7fb29a48c700 1 --2- 172.21.15.88:0/2455738857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb294068540 0x7fb294195e20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb28400c930 tx=0x7fb28400ccf0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.805+0000 7fb29a48c700 1 -- 172.21.15.88:0/2455738857 shutdown_connections 2024-02-19T19:16:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.805+0000 7fb29a48c700 1 --2- 172.21.15.88:0/2455738857 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb27c06d8a0 0x7fb27c06fd50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.805+0000 7fb29a48c700 1 --2- 172.21.15.88:0/2455738857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb294068540 0x7fb294195e20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.805+0000 7fb29a48c700 1 --2- 172.21.15.88:0/2455738857 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb294106320 0x7fb294196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.805+0000 7fb29a48c700 1 -- 172.21.15.88:0/2455738857 >> 172.21.15.88:0/2455738857 conn(0x7fb294075130 msgr2=0x7fb2940feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.805+0000 7fb29a48c700 1 -- 172.21.15.88:0/2455738857 shutdown_connections 2024-02-19T19:16:06.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.805+0000 7fb29a48c700 1 -- 172.21.15.88:0/2455738857 wait complete. 2024-02-19T19:16:06.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.909+0000 7fc977920700 1 -- 172.21.15.88:0/1203514567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9700ffbd0 msgr2=0x7fc970100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.909+0000 7fc977920700 1 --2- 172.21.15.88:0/1203514567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9700ffbd0 0x7fc970100040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc96c009910 tx=0x7fc96c009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:06.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.909+0000 7fc977920700 1 -- 172.21.15.88:0/1203514567 shutdown_connections 2024-02-19T19:16:06.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.909+0000 7fc977920700 1 --2- 172.21.15.88:0/1203514567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9700ffbd0 0x7fc970100040 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.909+0000 7fc977920700 1 --2- 172.21.15.88:0/1203514567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc970104130 0x7fc970104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.909+0000 7fc977920700 1 -- 172.21.15.88:0/1203514567 >> 172.21.15.88:0/1203514567 conn(0x7fc970075130 msgr2=0x7fc970075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:06.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.909+0000 7fc977920700 1 -- 172.21.15.88:0/1203514567 shutdown_connections 2024-02-19T19:16:06.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.909+0000 7fc977920700 1 -- 172.21.15.88:0/1203514567 wait complete. 2024-02-19T19:16:06.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.909+0000 7fc977920700 1 Processor -- start 2024-02-19T19:16:06.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc977920700 1 -- start start 2024-02-19T19:16:06.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc977920700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9700ffbd0 0x7fc970194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:06.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc977920700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc970104130 0x7fc970194fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:06.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc977920700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc970195510 con 0x7fc970104130 2024-02-19T19:16:06.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc977920700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc970195680 con 0x7fc9700ffbd0 2024-02-19T19:16:06.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc974ebb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc970104130 0x7fc970194fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc974ebb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc970104130 0x7fc970194fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34654/0 (socket says 172.21.15.88:34654) 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc974ebb700 1 -- 172.21.15.88:0/2929174156 learned_addr learned my addr 172.21.15.88:0/2929174156 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc9756bc700 1 --2- 172.21.15.88:0/2929174156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9700ffbd0 0x7fc970194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc974ebb700 1 -- 172.21.15.88:0/2929174156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9700ffbd0 msgr2=0x7fc970194a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc974ebb700 1 --2- 172.21.15.88:0/2929174156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9700ffbd0 0x7fc970194a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc974ebb700 1 -- 172.21.15.88:0/2929174156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c0095f0 con 0x7fc970104130 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc974ebb700 1 --2- 172.21.15.88:0/2929174156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc970104130 0x7fc970194fd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00bd70 tx=0x7fc96c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc9667fc700 1 -- 172.21.15.88:0/2929174156 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc96c01d070 con 0x7fc970104130 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc9667fc700 1 -- 172.21.15.88:0/2929174156 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc96c00f460 con 0x7fc970104130 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc9667fc700 1 -- 172.21.15.88:0/2929174156 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc96c017630 con 0x7fc970104130 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.910+0000 7fc977920700 1 -- 172.21.15.88:0/2929174156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc970195900 con 0x7fc970104130 2024-02-19T19:16:06.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.911+0000 7fc977920700 1 -- 172.21.15.88:0/2929174156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9701a2380 con 0x7fc970104130 2024-02-19T19:16:06.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.911+0000 7fc977920700 1 -- 172.21.15.88:0/2929174156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97004f2a0 con 0x7fc970104130 2024-02-19T19:16:06.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.912+0000 7fc9667fc700 1 -- 172.21.15.88:0/2929174156 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc96c00f5d0 con 0x7fc970104130 2024-02-19T19:16:06.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.912+0000 7fc9667fc700 1 --2- 172.21.15.88:0/2929174156 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc95c06d8a0 0x7fc95c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:06.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.912+0000 7fc9667fc700 1 -- 172.21.15.88:0/2929174156 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc96c08e810 con 0x7fc970104130 2024-02-19T19:16:06.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.912+0000 7fc9756bc700 1 --2- 172.21.15.88:0/2929174156 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc95c06d8a0 0x7fc95c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:06.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.913+0000 7fc9756bc700 1 --2- 172.21.15.88:0/2929174156 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc95c06d8a0 0x7fc95c06fd50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc960005950 tx=0x7fc96000a300 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:06.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:06.917+0000 7fc9667fc700 1 -- 172.21.15.88:0/2929174156 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc96c05bb70 con 0x7fc970104130 2024-02-19T19:16:07.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.160+0000 7fc977920700 1 -- 172.21.15.88:0/2929174156 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc970068b30 con 0x7fc970104130 2024-02-19T19:16:07.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.160+0000 7fc9667fc700 1 -- 172.21.15.88:0/2929174156 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fc96c05b700 con 0x7fc970104130 2024-02-19T19:16:07.180 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:16:07.180 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:16:07.180 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:16:07.180 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:16:07.180 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:16:07.180 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:16:07.180 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:16:07.180 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:16:07.181 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:16:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:16:07.183 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:16:07.183 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:16:07.183 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:16:07.183 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:16:07.183 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:16:07.183 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:16:07.183 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:16:07.183 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:16:07.183 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:16:07.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.162+0000 7fc977920700 1 -- 172.21.15.88:0/2929174156 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc95c06d8a0 msgr2=0x7fc95c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:07.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.162+0000 7fc977920700 1 --2- 172.21.15.88:0/2929174156 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc95c06d8a0 0x7fc95c06fd50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc960005950 tx=0x7fc96000a300 comp rx=0 tx=0).stop 2024-02-19T19:16:07.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.162+0000 7fc977920700 1 -- 172.21.15.88:0/2929174156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc970104130 msgr2=0x7fc970194fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:07.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.162+0000 7fc977920700 1 --2- 172.21.15.88:0/2929174156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc970104130 0x7fc970194fd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00bd70 tx=0x7fc96c00bda0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.163+0000 7fc977920700 1 -- 172.21.15.88:0/2929174156 shutdown_connections 2024-02-19T19:16:07.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.163+0000 7fc977920700 1 --2- 172.21.15.88:0/2929174156 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc95c06d8a0 0x7fc95c06fd50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.163+0000 7fc977920700 1 --2- 172.21.15.88:0/2929174156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc9700ffbd0 0x7fc970194a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.163+0000 7fc977920700 1 --2- 172.21.15.88:0/2929174156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc970104130 0x7fc970194fd0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.163+0000 7fc977920700 1 -- 172.21.15.88:0/2929174156 >> 172.21.15.88:0/2929174156 conn(0x7fc970075130 msgr2=0x7fc9700feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:07.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.163+0000 7fc977920700 1 -- 172.21.15.88:0/2929174156 shutdown_connections 2024-02-19T19:16:07.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.163+0000 7fc977920700 1 -- 172.21.15.88:0/2929174156 wait complete. 2024-02-19T19:16:07.185 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:16:07.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.266+0000 7f7766170700 1 -- 172.21.15.88:0/2622304567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77600ff3c0 msgr2=0x7f77600ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:07.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.266+0000 7f7766170700 1 --2- 172.21.15.88:0/2622304567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77600ff3c0 0x7f77600ff790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f7754009910 tx=0x7f7754009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:07.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.266+0000 7f7766170700 1 -- 172.21.15.88:0/2622304567 shutdown_connections 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.266+0000 7f7766170700 1 --2- 172.21.15.88:0/2622304567 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77600ffd60 0x7f7760103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.266+0000 7f7766170700 1 --2- 172.21.15.88:0/2622304567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77600ff3c0 0x7f77600ff790 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.266+0000 7f7766170700 1 -- 172.21.15.88:0/2622304567 >> 172.21.15.88:0/2622304567 conn(0x7f77600fac80 msgr2=0x7f77600fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.266+0000 7f7766170700 1 -- 172.21.15.88:0/2622304567 shutdown_connections 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.266+0000 7f7766170700 1 -- 172.21.15.88:0/2622304567 wait complete. 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f7766170700 1 Processor -- start 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f7766170700 1 -- start start 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f7766170700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77600ff3c0 0x7f7760196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f7766170700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77600ffd60 0x7f7760197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f7766170700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7760197850 con 0x7f77600ffd60 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f7766170700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7760197990 con 0x7f77600ff3c0 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f775effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77600ffd60 0x7f7760197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:07.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f775effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77600ffd60 0x7f7760197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34670/0 (socket says 172.21.15.88:34670) 2024-02-19T19:16:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f775effd700 1 -- 172.21.15.88:0/3571410446 learned_addr learned my addr 172.21.15.88:0/3571410446 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f775f7fe700 1 --2- 172.21.15.88:0/3571410446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77600ff3c0 0x7f7760196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f775effd700 1 -- 172.21.15.88:0/3571410446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77600ff3c0 msgr2=0x7f7760196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.267+0000 7f775effd700 1 --2- 172.21.15.88:0/3571410446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77600ff3c0 0x7f7760196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.268+0000 7f775effd700 1 -- 172.21.15.88:0/3571410446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77540095f0 con 0x7f77600ffd60 2024-02-19T19:16:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.268+0000 7f775effd700 1 --2- 172.21.15.88:0/3571410446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77600ffd60 0x7f7760197230 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f775000d900 tx=0x7f775000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.268+0000 7f775cff9700 1 -- 172.21.15.88:0/3571410446 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7750009900 con 0x7f77600ffd60 2024-02-19T19:16:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.268+0000 7f7766170700 1 -- 172.21.15.88:0/3571410446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f776019fd60 con 0x7f77600ffd60 2024-02-19T19:16:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.268+0000 7f775cff9700 1 -- 172.21.15.88:0/3571410446 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7750010460 con 0x7f77600ffd60 2024-02-19T19:16:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.268+0000 7f775cff9700 1 -- 172.21.15.88:0/3571410446 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f775000f5f0 con 0x7f77600ffd60 2024-02-19T19:16:07.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.268+0000 7f7766170700 1 -- 172.21.15.88:0/3571410446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77601a0220 con 0x7f77600ffd60 2024-02-19T19:16:07.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.269+0000 7f7766170700 1 -- 172.21.15.88:0/3571410446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77601076d0 con 0x7f77600ffd60 2024-02-19T19:16:07.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.269+0000 7f775cff9700 1 -- 172.21.15.88:0/3571410446 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f77500105d0 con 0x7f77600ffd60 2024-02-19T19:16:07.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.270+0000 7f775cff9700 1 --2- 172.21.15.88:0/3571410446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f774806d8a0 0x7f774806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:07.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.270+0000 7f775cff9700 1 -- 172.21.15.88:0/3571410446 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f775008d380 con 0x7f77600ffd60 2024-02-19T19:16:07.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.270+0000 7f775f7fe700 1 --2- 172.21.15.88:0/3571410446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f774806d8a0 0x7f774806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:07.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.270+0000 7f775f7fe700 1 --2- 172.21.15.88:0/3571410446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f774806d8a0 0x7f774806fd50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f775400b5c0 tx=0x7f775401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:07.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.274+0000 7f775cff9700 1 -- 172.21.15.88:0/3571410446 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7750056b30 con 0x7f77600ffd60 2024-02-19T19:16:07.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.489+0000 7f7766170700 1 -- 172.21.15.88:0/3571410446 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7760061190 con 0x7f774806d8a0 2024-02-19T19:16:07.509 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:07 smithi088 ceph-mon[88643]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:07.509 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:07 smithi088 ceph-mon[88643]: from='client.34726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:07.509 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:07 smithi088 ceph-mon[88643]: pgmap v375: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:07.509 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:07 smithi088 ceph-mon[88643]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:07.509 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:07 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2455738857' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:16:07.509 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:07 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2929174156' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:16:07.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.491+0000 7f775cff9700 1 -- 172.21.15.88:0/3571410446 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7760061190 con 0x7f774806d8a0 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:16:07.510 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:16:07.511 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:16:07.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.493+0000 7f7766170700 1 -- 172.21.15.88:0/3571410446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f774806d8a0 msgr2=0x7f774806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:07.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.493+0000 7f7766170700 1 --2- 172.21.15.88:0/3571410446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f774806d8a0 0x7f774806fd50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f775400b5c0 tx=0x7f775401a040 comp rx=0 tx=0).stop 2024-02-19T19:16:07.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.493+0000 7f7766170700 1 -- 172.21.15.88:0/3571410446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77600ffd60 msgr2=0x7f7760197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:07.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.493+0000 7f7766170700 1 --2- 172.21.15.88:0/3571410446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77600ffd60 0x7f7760197230 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f775000d900 tx=0x7f775000dcc0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.493+0000 7f7766170700 1 -- 172.21.15.88:0/3571410446 shutdown_connections 2024-02-19T19:16:07.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.493+0000 7f7766170700 1 --2- 172.21.15.88:0/3571410446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f774806d8a0 0x7f774806fd50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.493+0000 7f7766170700 1 --2- 172.21.15.88:0/3571410446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77600ff3c0 0x7f7760196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.493+0000 7f7766170700 1 --2- 172.21.15.88:0/3571410446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77600ffd60 0x7f7760197230 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.493+0000 7f7766170700 1 -- 172.21.15.88:0/3571410446 >> 172.21.15.88:0/3571410446 conn(0x7f77600fac80 msgr2=0x7f77600fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:07.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.493+0000 7f7766170700 1 -- 172.21.15.88:0/3571410446 shutdown_connections 2024-02-19T19:16:07.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.493+0000 7f7766170700 1 -- 172.21.15.88:0/3571410446 wait complete. 2024-02-19T19:16:07.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.597+0000 7f276bdf9700 1 -- 172.21.15.88:0/3302148690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27641004c0 msgr2=0x7f2764100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:07.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.597+0000 7f276bdf9700 1 --2- 172.21.15.88:0/3302148690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27641004c0 0x7f2764100890 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f276000b0a0 tx=0x7f276000b3b0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.597+0000 7f276bdf9700 1 -- 172.21.15.88:0/3302148690 shutdown_connections 2024-02-19T19:16:07.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.597+0000 7f276bdf9700 1 --2- 172.21.15.88:0/3302148690 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2764100e60 0x7f2764104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.597+0000 7f276bdf9700 1 --2- 172.21.15.88:0/3302148690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27641004c0 0x7f2764100890 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.597+0000 7f276bdf9700 1 -- 172.21.15.88:0/3302148690 >> 172.21.15.88:0/3302148690 conn(0x7f27640fbd80 msgr2=0x7f27640fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:07.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.597+0000 7f276bdf9700 1 -- 172.21.15.88:0/3302148690 shutdown_connections 2024-02-19T19:16:07.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.597+0000 7f276bdf9700 1 -- 172.21.15.88:0/3302148690 wait complete. 2024-02-19T19:16:07.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f276bdf9700 1 Processor -- start 2024-02-19T19:16:07.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f276bdf9700 1 -- start start 2024-02-19T19:16:07.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f276bdf9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27641004c0 0x7f2764195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:07.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f276bdf9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2764100e60 0x7f2764196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:07.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f276bdf9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2764196b50 con 0x7f2764100e60 2024-02-19T19:16:07.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f276bdf9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276419a8e0 con 0x7f27641004c0 2024-02-19T19:16:07.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f2769394700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2764100e60 0x7f2764196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:07.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f2769394700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2764100e60 0x7f2764196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34680/0 (socket says 172.21.15.88:34680) 2024-02-19T19:16:07.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f2769394700 1 -- 172.21.15.88:0/3241147823 learned_addr learned my addr 172.21.15.88:0/3241147823 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:07.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f2769b95700 1 --2- 172.21.15.88:0/3241147823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27641004c0 0x7f2764195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:07.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f2769394700 1 -- 172.21.15.88:0/3241147823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27641004c0 msgr2=0x7f2764195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:07.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f2769394700 1 --2- 172.21.15.88:0/3241147823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27641004c0 0x7f2764195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f2769394700 1 -- 172.21.15.88:0/3241147823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2760009d00 con 0x7f2764100e60 2024-02-19T19:16:07.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f2769394700 1 --2- 172.21.15.88:0/3241147823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2764100e60 0x7f2764196470 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f275400d900 tx=0x7f275400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:07.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.598+0000 7f275affd700 1 -- 172.21.15.88:0/3241147823 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2754009900 con 0x7f2764100e60 2024-02-19T19:16:07.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.599+0000 7f276bdf9700 1 -- 172.21.15.88:0/3241147823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276419abc0 con 0x7f2764100e60 2024-02-19T19:16:07.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.599+0000 7f275affd700 1 -- 172.21.15.88:0/3241147823 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2754010460 con 0x7f2764100e60 2024-02-19T19:16:07.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.599+0000 7f275affd700 1 -- 172.21.15.88:0/3241147823 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f275400f5f0 con 0x7f2764100e60 2024-02-19T19:16:07.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.599+0000 7f276bdf9700 1 -- 172.21.15.88:0/3241147823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276419b110 con 0x7f2764100e60 2024-02-19T19:16:07.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.599+0000 7f276bdf9700 1 -- 172.21.15.88:0/3241147823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27641087d0 con 0x7f2764100e60 2024-02-19T19:16:07.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.600+0000 7f275affd700 1 -- 172.21.15.88:0/3241147823 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f27540105d0 con 0x7f2764100e60 2024-02-19T19:16:07.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.600+0000 7f275affd700 1 --2- 172.21.15.88:0/3241147823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f275006d8a0 0x7f275006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:07.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.601+0000 7f275affd700 1 -- 172.21.15.88:0/3241147823 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f275408d0e0 con 0x7f2764100e60 2024-02-19T19:16:07.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.601+0000 7f2769b95700 1 --2- 172.21.15.88:0/3241147823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f275006d8a0 0x7f275006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:07.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.601+0000 7f2769b95700 1 --2- 172.21.15.88:0/3241147823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f275006d8a0 0x7f275006fd50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2760000f80 tx=0x7f27600060c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:07.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.605+0000 7f275affd700 1 -- 172.21.15.88:0/3241147823 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2754056890 con 0x7f2764100e60 2024-02-19T19:16:07.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:07 smithi132 ceph-mon[70676]: from='client.34722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:07.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:07 smithi132 ceph-mon[70676]: from='client.34726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:07.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:07 smithi132 ceph-mon[70676]: pgmap v375: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:07.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:07 smithi132 ceph-mon[70676]: from='client.34730 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:07.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:07 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2455738857' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:16:07.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:07 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2929174156' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:16:07.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.888+0000 7f276bdf9700 1 -- 172.21.15.88:0/3241147823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f276404f2a0 con 0x7f2764100e60 2024-02-19T19:16:07.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.888+0000 7f275affd700 1 -- 172.21.15.88:0/3241147823 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2754059f50 con 0x7f2764100e60 2024-02-19T19:16:07.907 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:16:07.908 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:16:07.908 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:16:07.908 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:16:07.908 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:16:07.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.890+0000 7f276bdf9700 1 -- 172.21.15.88:0/3241147823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f275006d8a0 msgr2=0x7f275006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:07.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.890+0000 7f276bdf9700 1 --2- 172.21.15.88:0/3241147823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f275006d8a0 0x7f275006fd50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2760000f80 tx=0x7f27600060c0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.890+0000 7f276bdf9700 1 -- 172.21.15.88:0/3241147823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2764100e60 msgr2=0x7f2764196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:07.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.890+0000 7f276bdf9700 1 --2- 172.21.15.88:0/3241147823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2764100e60 0x7f2764196470 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f275400d900 tx=0x7f275400dc10 comp rx=0 tx=0).stop 2024-02-19T19:16:07.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.890+0000 7f276bdf9700 1 -- 172.21.15.88:0/3241147823 shutdown_connections 2024-02-19T19:16:07.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.890+0000 7f276bdf9700 1 --2- 172.21.15.88:0/3241147823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f275006d8a0 0x7f275006fd50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.890+0000 7f276bdf9700 1 --2- 172.21.15.88:0/3241147823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27641004c0 0x7f2764195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.890+0000 7f276bdf9700 1 --2- 172.21.15.88:0/3241147823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2764100e60 0x7f2764196470 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:07.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.890+0000 7f276bdf9700 1 -- 172.21.15.88:0/3241147823 >> 172.21.15.88:0/3241147823 conn(0x7f27640fbd80 msgr2=0x7f27640fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:07.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.891+0000 7f276bdf9700 1 -- 172.21.15.88:0/3241147823 shutdown_connections 2024-02-19T19:16:07.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:07.891+0000 7f276bdf9700 1 -- 172.21.15.88:0/3241147823 wait complete. 2024-02-19T19:16:08.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:08 smithi132 ceph-mon[70676]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:08.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:08 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3241147823' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:16:08.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:08 smithi088 ceph-mon[88643]: from='client.34742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:08.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:08 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3241147823' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:16:09.723 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:09 smithi132 ceph-mon[70676]: pgmap v376: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:09.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:09 smithi088 ceph-mon[88643]: pgmap v376: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:11.651 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:11 smithi088 ceph-mon[88643]: pgmap v377: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:11.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:11 smithi132 ceph-mon[70676]: pgmap v377: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:13.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:13 smithi132 ceph-mon[70676]: pgmap v378: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:13.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:16:13.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:13 smithi088 ceph-mon[88643]: pgmap v378: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:13.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:16:15.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:15 smithi132 ceph-mon[70676]: pgmap v379: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:15.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:15 smithi088 ceph-mon[88643]: pgmap v379: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:17.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:17 smithi132 ceph-mon[70676]: pgmap v380: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:17.849 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:17 smithi088 ceph-mon[88643]: pgmap v380: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:19.752 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:19 smithi132 ceph-mon[70676]: pgmap v381: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:19.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:19 smithi088 ceph-mon[88643]: pgmap v381: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:21.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:21 smithi132 ceph-mon[70676]: pgmap v382: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:21.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:21 smithi088 ceph-mon[88643]: pgmap v382: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:23.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:23 smithi132 ceph-mon[70676]: pgmap v383: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:23.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:23 smithi088 ceph-mon[88643]: pgmap v383: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:25.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:25 smithi132 ceph-mon[70676]: pgmap v384: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:25.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:25 smithi088 ceph-mon[88643]: pgmap v384: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:27.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:27 smithi132 ceph-mon[70676]: pgmap v385: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:27.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:27 smithi088 ceph-mon[88643]: pgmap v385: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:28.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:16:28.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:16:29.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:29 smithi132 ceph-mon[70676]: pgmap v386: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:29.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:29 smithi088 ceph-mon[88643]: pgmap v386: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:31.655 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:31 smithi088 ceph-mon[88643]: pgmap v387: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:31.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:31 smithi132 ceph-mon[70676]: pgmap v387: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:33.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:33 smithi132 ceph-mon[70676]: pgmap v388: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:33.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:33 smithi088 ceph-mon[88643]: pgmap v388: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:35.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:35 smithi132 ceph-mon[70676]: pgmap v389: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:35.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:35 smithi088 ceph-mon[88643]: pgmap v389: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:37.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:37 smithi132 ceph-mon[70676]: pgmap v390: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:37.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:37 smithi088 ceph-mon[88643]: pgmap v390: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.007+0000 7f1dc8e36700 1 -- 172.21.15.88:0/2644798335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dc40ffbd0 msgr2=0x7f1dc4100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.007+0000 7f1dc8e36700 1 --2- 172.21.15.88:0/2644798335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dc40ffbd0 0x7f1dc4100040 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f1db4009910 tx=0x7f1db4009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.007+0000 7f1dc8e36700 1 -- 172.21.15.88:0/2644798335 shutdown_connections 2024-02-19T19:16:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.007+0000 7f1dc8e36700 1 --2- 172.21.15.88:0/2644798335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dc40ffbd0 0x7f1dc4100040 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.007+0000 7f1dc8e36700 1 --2- 172.21.15.88:0/2644798335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dc4104130 0x7f1dc4104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.007+0000 7f1dc8e36700 1 -- 172.21.15.88:0/2644798335 >> 172.21.15.88:0/2644798335 conn(0x7f1dc4075130 msgr2=0x7f1dc4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.007+0000 7f1dc8e36700 1 -- 172.21.15.88:0/2644798335 shutdown_connections 2024-02-19T19:16:38.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.007+0000 7f1dc8e36700 1 -- 172.21.15.88:0/2644798335 wait complete. 2024-02-19T19:16:38.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.008+0000 7f1dc8e36700 1 Processor -- start 2024-02-19T19:16:38.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.008+0000 7f1dc8e36700 1 -- start start 2024-02-19T19:16:38.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.008+0000 7f1dc8e36700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dc40ffbd0 0x7f1dc4195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:38.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.008+0000 7f1dc8e36700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dc4104130 0x7f1dc4196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:38.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.008+0000 7f1dc8e36700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc4196a40 con 0x7f1dc40ffbd0 2024-02-19T19:16:38.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.008+0000 7f1dc8e36700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc419a7d0 con 0x7f1dc4104130 2024-02-19T19:16:38.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.008+0000 7f1dc259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dc40ffbd0 0x7f1dc4195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:38.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.008+0000 7f1dc259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dc40ffbd0 0x7f1dc4195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46454/0 (socket says 172.21.15.88:46454) 2024-02-19T19:16:38.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.008+0000 7f1dc259c700 1 -- 172.21.15.88:0/1603963872 learned_addr learned my addr 172.21.15.88:0/1603963872 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:38.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.009+0000 7f1dc1d9b700 1 --2- 172.21.15.88:0/1603963872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dc4104130 0x7f1dc4196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:38.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.009+0000 7f1dc259c700 1 -- 172.21.15.88:0/1603963872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dc4104130 msgr2=0x7f1dc4196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.009+0000 7f1dc259c700 1 --2- 172.21.15.88:0/1603963872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dc4104130 0x7f1dc4196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.009+0000 7f1dc259c700 1 -- 172.21.15.88:0/1603963872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db40095f0 con 0x7f1dc40ffbd0 2024-02-19T19:16:38.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.009+0000 7f1dc259c700 1 --2- 172.21.15.88:0/1603963872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dc40ffbd0 0x7f1dc4195e20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1db800d8d0 tx=0x7f1db800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:38.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.009+0000 7f1db37fe700 1 -- 172.21.15.88:0/1603963872 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1db8009940 con 0x7f1dc40ffbd0 2024-02-19T19:16:38.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.009+0000 7f1db37fe700 1 -- 172.21.15.88:0/1603963872 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1db8010460 con 0x7f1dc40ffbd0 2024-02-19T19:16:38.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.009+0000 7f1dc8e36700 1 -- 172.21.15.88:0/1603963872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc419aab0 con 0x7f1dc40ffbd0 2024-02-19T19:16:38.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.009+0000 7f1db37fe700 1 -- 172.21.15.88:0/1603963872 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1db800f5f0 con 0x7f1dc40ffbd0 2024-02-19T19:16:38.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.009+0000 7f1dc8e36700 1 -- 172.21.15.88:0/1603963872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc419b000 con 0x7f1dc40ffbd0 2024-02-19T19:16:38.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.010+0000 7f1dc8e36700 1 -- 172.21.15.88:0/1603963872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc41086a0 con 0x7f1dc40ffbd0 2024-02-19T19:16:38.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.011+0000 7f1db37fe700 1 -- 172.21.15.88:0/1603963872 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1db80105d0 con 0x7f1dc40ffbd0 2024-02-19T19:16:38.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.011+0000 7f1db37fe700 1 --2- 172.21.15.88:0/1603963872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1dac06d8a0 0x7f1dac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:38.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.011+0000 7f1db37fe700 1 -- 172.21.15.88:0/1603963872 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1db808d540 con 0x7f1dc40ffbd0 2024-02-19T19:16:38.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.011+0000 7f1dc1d9b700 1 --2- 172.21.15.88:0/1603963872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1dac06d8a0 0x7f1dac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:38.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.012+0000 7f1dc1d9b700 1 --2- 172.21.15.88:0/1603963872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1dac06d8a0 0x7f1dac06fd50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1db40059d0 tx=0x7f1db401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:38.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.016+0000 7f1db37fe700 1 -- 172.21.15.88:0/1603963872 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1db8056cf0 con 0x7f1dc40ffbd0 2024-02-19T19:16:38.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.232+0000 7f1dc8e36700 1 -- 172.21.15.88:0/1603963872 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dc419b220 con 0x7f1dac06d8a0 2024-02-19T19:16:38.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.234+0000 7f1db37fe700 1 -- 172.21.15.88:0/1603963872 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1dc419b220 con 0x7f1dac06d8a0 2024-02-19T19:16:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.236+0000 7f1dc8e36700 1 -- 172.21.15.88:0/1603963872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1dac06d8a0 msgr2=0x7f1dac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.236+0000 7f1dc8e36700 1 --2- 172.21.15.88:0/1603963872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1dac06d8a0 0x7f1dac06fd50 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1db40059d0 tx=0x7f1db401a040 comp rx=0 tx=0).stop 2024-02-19T19:16:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.236+0000 7f1dc8e36700 1 -- 172.21.15.88:0/1603963872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dc40ffbd0 msgr2=0x7f1dc4195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.236+0000 7f1dc8e36700 1 --2- 172.21.15.88:0/1603963872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dc40ffbd0 0x7f1dc4195e20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1db800d8d0 tx=0x7f1db800dc90 comp rx=0 tx=0).stop 2024-02-19T19:16:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.236+0000 7f1dc8e36700 1 -- 172.21.15.88:0/1603963872 shutdown_connections 2024-02-19T19:16:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.236+0000 7f1dc8e36700 1 --2- 172.21.15.88:0/1603963872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1dac06d8a0 0x7f1dac06fd50 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.236+0000 7f1dc8e36700 1 --2- 172.21.15.88:0/1603963872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1dc40ffbd0 0x7f1dc4195e20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.236+0000 7f1dc8e36700 1 --2- 172.21.15.88:0/1603963872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1dc4104130 0x7f1dc4196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.236+0000 7f1dc8e36700 1 -- 172.21.15.88:0/1603963872 >> 172.21.15.88:0/1603963872 conn(0x7f1dc4075130 msgr2=0x7f1dc40fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:38.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.236+0000 7f1dc8e36700 1 -- 172.21.15.88:0/1603963872 shutdown_connections 2024-02-19T19:16:38.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.236+0000 7f1dc8e36700 1 -- 172.21.15.88:0/1603963872 wait complete. 2024-02-19T19:16:38.265 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:16:38.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.342+0000 7f4924360700 1 -- 172.21.15.88:0/1542220096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491c104130 msgr2=0x7f491c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.342+0000 7f4924360700 1 --2- 172.21.15.88:0/1542220096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491c104130 0x7f491c104500 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4914009910 tx=0x7f4914009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:38.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.342+0000 7f4924360700 1 -- 172.21.15.88:0/1542220096 shutdown_connections 2024-02-19T19:16:38.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.342+0000 7f4924360700 1 --2- 172.21.15.88:0/1542220096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f491c0ffbd0 0x7f491c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.342+0000 7f4924360700 1 --2- 172.21.15.88:0/1542220096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491c104130 0x7f491c104500 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.342+0000 7f4924360700 1 -- 172.21.15.88:0/1542220096 >> 172.21.15.88:0/1542220096 conn(0x7f491c075130 msgr2=0x7f491c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:38.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.342+0000 7f4924360700 1 -- 172.21.15.88:0/1542220096 shutdown_connections 2024-02-19T19:16:38.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.342+0000 7f4924360700 1 -- 172.21.15.88:0/1542220096 wait complete. 2024-02-19T19:16:38.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f4924360700 1 Processor -- start 2024-02-19T19:16:38.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f4924360700 1 -- start start 2024-02-19T19:16:38.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f4924360700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f491c0ffbd0 0x7f491c195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:38.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f4924360700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491c104130 0x7f491c196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:38.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f4924360700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c196bf0 con 0x7f491c104130 2024-02-19T19:16:38.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f4924360700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491c19a980 con 0x7f491c0ffbd0 2024-02-19T19:16:38.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f49218fb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491c104130 0x7f491c196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:38.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f49218fb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491c104130 0x7f491c196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46460/0 (socket says 172.21.15.88:46460) 2024-02-19T19:16:38.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f49218fb700 1 -- 172.21.15.88:0/2624389366 learned_addr learned my addr 172.21.15.88:0/2624389366 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:38.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f49220fc700 1 --2- 172.21.15.88:0/2624389366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f491c0ffbd0 0x7f491c195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:38.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f49218fb700 1 -- 172.21.15.88:0/2624389366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f491c0ffbd0 msgr2=0x7f491c195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f49218fb700 1 --2- 172.21.15.88:0/2624389366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f491c0ffbd0 0x7f491c195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f49218fb700 1 -- 172.21.15.88:0/2624389366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49140095f0 con 0x7f491c104130 2024-02-19T19:16:38.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.343+0000 7f49218fb700 1 --2- 172.21.15.88:0/2624389366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491c104130 0x7f491c196510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f491000b4b0 tx=0x7f491000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:38.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.344+0000 7f491b7fe700 1 -- 172.21.15.88:0/2624389366 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4910011800 con 0x7f491c104130 2024-02-19T19:16:38.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.344+0000 7f4924360700 1 -- 172.21.15.88:0/2624389366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491c19ac60 con 0x7f491c104130 2024-02-19T19:16:38.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.344+0000 7f491b7fe700 1 -- 172.21.15.88:0/2624389366 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4910011e20 con 0x7f491c104130 2024-02-19T19:16:38.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.344+0000 7f491b7fe700 1 -- 172.21.15.88:0/2624389366 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f491000f2e0 con 0x7f491c104130 2024-02-19T19:16:38.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.344+0000 7f4924360700 1 -- 172.21.15.88:0/2624389366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491c19b1b0 con 0x7f491c104130 2024-02-19T19:16:38.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.345+0000 7f4924360700 1 -- 172.21.15.88:0/2624389366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491c04f2a0 con 0x7f491c104130 2024-02-19T19:16:38.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.345+0000 7f491b7fe700 1 -- 172.21.15.88:0/2624389366 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4910010380 con 0x7f491c104130 2024-02-19T19:16:38.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.346+0000 7f491b7fe700 1 --2- 172.21.15.88:0/2624389366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490406d8a0 0x7f490406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:38.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.346+0000 7f49220fc700 1 --2- 172.21.15.88:0/2624389366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490406d8a0 0x7f490406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:38.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.346+0000 7f491b7fe700 1 -- 172.21.15.88:0/2624389366 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f491008d090 con 0x7f491c104130 2024-02-19T19:16:38.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.346+0000 7f49220fc700 1 --2- 172.21.15.88:0/2624389366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490406d8a0 0x7f490406fd50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4914000c00 tx=0x7f491401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:38.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.350+0000 7f491b7fe700 1 -- 172.21.15.88:0/2624389366 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4910056840 con 0x7f491c104130 2024-02-19T19:16:38.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.575+0000 7f4924360700 1 -- 172.21.15.88:0/2624389366 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f491c19b4a0 con 0x7f490406d8a0 2024-02-19T19:16:38.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.577+0000 7f491b7fe700 1 -- 172.21.15.88:0/2624389366 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f491c19b4a0 con 0x7f490406d8a0 2024-02-19T19:16:38.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.579+0000 7f4924360700 1 -- 172.21.15.88:0/2624389366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490406d8a0 msgr2=0x7f490406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.579+0000 7f4924360700 1 --2- 172.21.15.88:0/2624389366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490406d8a0 0x7f490406fd50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f4914000c00 tx=0x7f491401a040 comp rx=0 tx=0).stop 2024-02-19T19:16:38.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.579+0000 7f4924360700 1 -- 172.21.15.88:0/2624389366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491c104130 msgr2=0x7f491c196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.579+0000 7f4924360700 1 --2- 172.21.15.88:0/2624389366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491c104130 0x7f491c196510 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f491000b4b0 tx=0x7f491000b7c0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.579+0000 7f4924360700 1 -- 172.21.15.88:0/2624389366 shutdown_connections 2024-02-19T19:16:38.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.579+0000 7f4924360700 1 --2- 172.21.15.88:0/2624389366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490406d8a0 0x7f490406fd50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.579+0000 7f4924360700 1 --2- 172.21.15.88:0/2624389366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f491c0ffbd0 0x7f491c195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.579+0000 7f4924360700 1 --2- 172.21.15.88:0/2624389366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f491c104130 0x7f491c196510 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.579+0000 7f4924360700 1 -- 172.21.15.88:0/2624389366 >> 172.21.15.88:0/2624389366 conn(0x7f491c075130 msgr2=0x7f491c109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:38.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.579+0000 7f4924360700 1 -- 172.21.15.88:0/2624389366 shutdown_connections 2024-02-19T19:16:38.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.579+0000 7f4924360700 1 -- 172.21.15.88:0/2624389366 wait complete. 2024-02-19T19:16:38.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.683+0000 7fa4866f1700 1 -- 172.21.15.88:0/3527519128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa480101db0 msgr2=0x7fa48010a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.683+0000 7fa4866f1700 1 --2- 172.21.15.88:0/3527519128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa480101db0 0x7fa48010a2b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa474009910 tx=0x7fa474009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:38.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.683+0000 7fa4866f1700 1 -- 172.21.15.88:0/3527519128 shutdown_connections 2024-02-19T19:16:38.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.683+0000 7fa4866f1700 1 --2- 172.21.15.88:0/3527519128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa480101db0 0x7fa48010a2b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.683+0000 7fa4866f1700 1 --2- 172.21.15.88:0/3527519128 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801014a0 0x7fa480101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.683+0000 7fa4866f1700 1 -- 172.21.15.88:0/3527519128 >> 172.21.15.88:0/3527519128 conn(0x7fa4800fac80 msgr2=0x7fa4800fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:38.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.683+0000 7fa4866f1700 1 -- 172.21.15.88:0/3527519128 shutdown_connections 2024-02-19T19:16:38.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.683+0000 7fa4866f1700 1 -- 172.21.15.88:0/3527519128 wait complete. 2024-02-19T19:16:38.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.683+0000 7fa4866f1700 1 Processor -- start 2024-02-19T19:16:38.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.683+0000 7fa4866f1700 1 -- start start 2024-02-19T19:16:38.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa4866f1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801014a0 0x7fa4801980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:38.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa4866f1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa480101db0 0x7fa4801985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:38.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa4866f1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa480198cd0 con 0x7fa480101db0 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa4866f1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48019ca60 con 0x7fa4801014a0 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa47f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa480101db0 0x7fa4801985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa47f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa480101db0 0x7fa4801985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46472/0 (socket says 172.21.15.88:46472) 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa47f7fe700 1 -- 172.21.15.88:0/3761608780 learned_addr learned my addr 172.21.15.88:0/3761608780 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa47ffff700 1 --2- 172.21.15.88:0/3761608780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801014a0 0x7fa4801980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa47f7fe700 1 -- 172.21.15.88:0/3761608780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801014a0 msgr2=0x7fa4801980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa47f7fe700 1 --2- 172.21.15.88:0/3761608780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801014a0 0x7fa4801980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa47f7fe700 1 -- 172.21.15.88:0/3761608780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4740095f0 con 0x7fa480101db0 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.684+0000 7fa47f7fe700 1 --2- 172.21.15.88:0/3761608780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa480101db0 0x7fa4801985f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa474006010 tx=0x7fa47400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.685+0000 7fa47d7fa700 1 -- 172.21.15.88:0/3761608780 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47401d070 con 0x7fa480101db0 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.685+0000 7fa47d7fa700 1 -- 172.21.15.88:0/3761608780 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa474004fb0 con 0x7fa480101db0 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.685+0000 7fa47d7fa700 1 -- 172.21.15.88:0/3761608780 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa474022ea0 con 0x7fa480101db0 2024-02-19T19:16:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.685+0000 7fa4866f1700 1 -- 172.21.15.88:0/3761608780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48019cce0 con 0x7fa480101db0 2024-02-19T19:16:38.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.685+0000 7fa4866f1700 1 -- 172.21.15.88:0/3761608780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48019d1d0 con 0x7fa480101db0 2024-02-19T19:16:38.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.686+0000 7fa4866f1700 1 -- 172.21.15.88:0/3761608780 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa480109d80 con 0x7fa480101db0 2024-02-19T19:16:38.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.686+0000 7fa47d7fa700 1 -- 172.21.15.88:0/3761608780 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa474022470 con 0x7fa480101db0 2024-02-19T19:16:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.687+0000 7fa47d7fa700 1 --2- 172.21.15.88:0/3761608780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa46806d8a0 0x7fa46806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.687+0000 7fa47d7fa700 1 -- 172.21.15.88:0/3761608780 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa47408ec30 con 0x7fa480101db0 2024-02-19T19:16:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.687+0000 7fa47ffff700 1 --2- 172.21.15.88:0/3761608780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa46806d8a0 0x7fa46806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.687+0000 7fa47ffff700 1 --2- 172.21.15.88:0/3761608780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa46806d8a0 0x7fa46806fd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa470005fd0 tx=0x7fa47000a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:38.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.692+0000 7fa47d7fa700 1 -- 172.21.15.88:0/3761608780 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa474058630 con 0x7fa480101db0 2024-02-19T19:16:38.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.897+0000 7fa4866f1700 1 -- 172.21.15.88:0/3761608780 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa48019d440 con 0x7fa46806d8a0 2024-02-19T19:16:38.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.908+0000 7fa47d7fa700 1 -- 172.21.15.88:0/3761608780 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa48019d440 con 0x7fa46806d8a0 2024-02-19T19:16:38.927 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:16:38.927 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (13m) 5m ago 24m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:16:38.927 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (24m) 5m ago 24m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:16:38.927 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (6m) 5m ago 22m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:16:38.927 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (9m) 5m ago 24m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:16:38.927 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (9m) 5m ago 22m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:16:38.927 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (13m) 5m ago 23m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:16:38.927 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (6m) 5m ago 18m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (5m) 5m ago 17m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (5m) 5m ago 17m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (17m) 5m ago 17m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (15m) 5m ago 25m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (14m) 5m ago 22m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (11m) 5m ago 26m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (11m) 5m ago 22m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (14m) 5m ago 24m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (14m) 5m ago 22m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (9m) 5m ago 21m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (8m) 5m ago 21m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:16:38.928 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (8m) 5m ago 20m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:16:38.929 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (7m) 5m ago 20m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:16:38.929 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (6m) 5m ago 19m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:16:38.929 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (6m) 5m ago 19m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:16:38.929 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (14m) 5m ago 23m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:16:38.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.910+0000 7fa4866f1700 1 -- 172.21.15.88:0/3761608780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa46806d8a0 msgr2=0x7fa46806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.910+0000 7fa4866f1700 1 --2- 172.21.15.88:0/3761608780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa46806d8a0 0x7fa46806fd50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa470005fd0 tx=0x7fa47000a560 comp rx=0 tx=0).stop 2024-02-19T19:16:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.910+0000 7fa4866f1700 1 -- 172.21.15.88:0/3761608780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa480101db0 msgr2=0x7fa4801985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.910+0000 7fa4866f1700 1 --2- 172.21.15.88:0/3761608780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa480101db0 0x7fa4801985f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa474006010 tx=0x7fa47400be20 comp rx=0 tx=0).stop 2024-02-19T19:16:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.910+0000 7fa4866f1700 1 -- 172.21.15.88:0/3761608780 shutdown_connections 2024-02-19T19:16:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.910+0000 7fa4866f1700 1 --2- 172.21.15.88:0/3761608780 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa46806d8a0 0x7fa46806fd50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.910+0000 7fa4866f1700 1 --2- 172.21.15.88:0/3761608780 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4801014a0 0x7fa4801980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.910+0000 7fa4866f1700 1 --2- 172.21.15.88:0/3761608780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa480101db0 0x7fa4801985f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.910+0000 7fa4866f1700 1 -- 172.21.15.88:0/3761608780 >> 172.21.15.88:0/3761608780 conn(0x7fa4800fac80 msgr2=0x7fa4800ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.910+0000 7fa4866f1700 1 -- 172.21.15.88:0/3761608780 shutdown_connections 2024-02-19T19:16:38.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:38.910+0000 7fa4866f1700 1 -- 172.21.15.88:0/3761608780 wait complete. 2024-02-19T19:16:39.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.013+0000 7f8519e9d700 1 -- 172.21.15.88:0/2075422880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8514100e60 msgr2=0x7f8514104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:39.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.013+0000 7f8519e9d700 1 --2- 172.21.15.88:0/2075422880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8514100e60 0x7f8514104e30 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8508009910 tx=0x7f8508009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:39.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.013+0000 7f8519e9d700 1 -- 172.21.15.88:0/2075422880 shutdown_connections 2024-02-19T19:16:39.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.013+0000 7f8519e9d700 1 --2- 172.21.15.88:0/2075422880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8514100e60 0x7f8514104e30 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.013+0000 7f8519e9d700 1 --2- 172.21.15.88:0/2075422880 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85141004c0 0x7f8514100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.013+0000 7f8519e9d700 1 -- 172.21.15.88:0/2075422880 >> 172.21.15.88:0/2075422880 conn(0x7f85140fbd80 msgr2=0x7f85140fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:39.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.013+0000 7f8519e9d700 1 -- 172.21.15.88:0/2075422880 shutdown_connections 2024-02-19T19:16:39.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.013+0000 7f8519e9d700 1 -- 172.21.15.88:0/2075422880 wait complete. 2024-02-19T19:16:39.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8519e9d700 1 Processor -- start 2024-02-19T19:16:39.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8519e9d700 1 -- start start 2024-02-19T19:16:39.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8519e9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85141004c0 0x7f8514195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:39.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8519e9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8514100e60 0x7f8514196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:39.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8519e9d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8514196b10 con 0x7f8514100e60 2024-02-19T19:16:39.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8519e9d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851419a8a0 con 0x7f85141004c0 2024-02-19T19:16:39.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8512ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8514100e60 0x7f8514196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:39.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8512ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8514100e60 0x7f8514196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46502/0 (socket says 172.21.15.88:46502) 2024-02-19T19:16:39.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8512ffd700 1 -- 172.21.15.88:0/1335058887 learned_addr learned my addr 172.21.15.88:0/1335058887 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:39.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f85137fe700 1 --2- 172.21.15.88:0/1335058887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85141004c0 0x7f8514195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:39.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8512ffd700 1 -- 172.21.15.88:0/1335058887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85141004c0 msgr2=0x7f8514195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:39.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8512ffd700 1 --2- 172.21.15.88:0/1335058887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85141004c0 0x7f8514195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.014+0000 7f8512ffd700 1 -- 172.21.15.88:0/1335058887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85080095f0 con 0x7f8514100e60 2024-02-19T19:16:39.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.015+0000 7f8512ffd700 1 --2- 172.21.15.88:0/1335058887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8514100e60 0x7f8514196430 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f85080092e0 tx=0x7f850800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:39.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.015+0000 7f8510ff9700 1 -- 172.21.15.88:0/1335058887 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f850801d070 con 0x7f8514100e60 2024-02-19T19:16:39.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.015+0000 7f8519e9d700 1 -- 172.21.15.88:0/1335058887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851419ab20 con 0x7f8514100e60 2024-02-19T19:16:39.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.015+0000 7f8510ff9700 1 -- 172.21.15.88:0/1335058887 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8508005dc0 con 0x7f8514100e60 2024-02-19T19:16:39.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.015+0000 7f8510ff9700 1 -- 172.21.15.88:0/1335058887 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8508017400 con 0x7f8514100e60 2024-02-19T19:16:39.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.015+0000 7f8519e9d700 1 -- 172.21.15.88:0/1335058887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851419b010 con 0x7f8514100e60 2024-02-19T19:16:39.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.016+0000 7f8519e9d700 1 -- 172.21.15.88:0/1335058887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8514108830 con 0x7f8514100e60 2024-02-19T19:16:39.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.016+0000 7f8510ff9700 1 -- 172.21.15.88:0/1335058887 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f850800f6f0 con 0x7f8514100e60 2024-02-19T19:16:39.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.017+0000 7f8510ff9700 1 --2- 172.21.15.88:0/1335058887 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84fc06d8a0 0x7f84fc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:39.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.017+0000 7f8510ff9700 1 -- 172.21.15.88:0/1335058887 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f850808e910 con 0x7f8514100e60 2024-02-19T19:16:39.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.017+0000 7f85137fe700 1 --2- 172.21.15.88:0/1335058887 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84fc06d8a0 0x7f84fc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:39.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.017+0000 7f85137fe700 1 --2- 172.21.15.88:0/1335058887 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84fc06d8a0 0x7f84fc06fd50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8504005950 tx=0x7f850400b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:39.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.022+0000 7f8510ff9700 1 -- 172.21.15.88:0/1335058887 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f85080580c0 con 0x7f8514100e60 2024-02-19T19:16:39.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.299+0000 7f8519e9d700 1 -- 172.21.15.88:0/1335058887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8514108210 con 0x7f8514100e60 2024-02-19T19:16:39.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.300+0000 7f8510ff9700 1 -- 172.21.15.88:0/1335058887 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f850805b780 con 0x7f8514100e60 2024-02-19T19:16:39.318 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:16:39.319 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:16:39.319 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:16:39.319 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:16:39.319 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:16:39.319 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:16:39.319 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:16:39.319 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:16:39.319 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:16:39.320 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:16:39.320 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:16:39.320 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:16:39.320 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:16:39.320 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:16:39.320 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:16:39.320 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:16:39.320 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:16:39.320 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:16:39.321 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:16:39.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.302+0000 7f8519e9d700 1 -- 172.21.15.88:0/1335058887 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84fc06d8a0 msgr2=0x7f84fc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.302+0000 7f8519e9d700 1 --2- 172.21.15.88:0/1335058887 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84fc06d8a0 0x7f84fc06fd50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f8504005950 tx=0x7f850400b410 comp rx=0 tx=0).stop 2024-02-19T19:16:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.302+0000 7f8519e9d700 1 -- 172.21.15.88:0/1335058887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8514100e60 msgr2=0x7f8514196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.302+0000 7f8519e9d700 1 --2- 172.21.15.88:0/1335058887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8514100e60 0x7f8514196430 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f85080092e0 tx=0x7f850800bd70 comp rx=0 tx=0).stop 2024-02-19T19:16:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.302+0000 7f8519e9d700 1 -- 172.21.15.88:0/1335058887 shutdown_connections 2024-02-19T19:16:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.302+0000 7f8519e9d700 1 --2- 172.21.15.88:0/1335058887 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84fc06d8a0 0x7f84fc06fd50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.302+0000 7f8519e9d700 1 --2- 172.21.15.88:0/1335058887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85141004c0 0x7f8514195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.302+0000 7f8519e9d700 1 --2- 172.21.15.88:0/1335058887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8514100e60 0x7f8514196430 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.302+0000 7f8519e9d700 1 -- 172.21.15.88:0/1335058887 >> 172.21.15.88:0/1335058887 conn(0x7f85140fbd80 msgr2=0x7f85140fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.302+0000 7f8519e9d700 1 -- 172.21.15.88:0/1335058887 shutdown_connections 2024-02-19T19:16:39.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.302+0000 7f8519e9d700 1 -- 172.21.15.88:0/1335058887 wait complete. 2024-02-19T19:16:39.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.406+0000 7f24af6dc700 1 -- 172.21.15.88:0/893256367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a81004e0 msgr2=0x7f24a81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:39.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.406+0000 7f24af6dc700 1 --2- 172.21.15.88:0/893256367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a81004e0 0x7f24a81008b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009910 tx=0x7f24a4009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.406+0000 7f24af6dc700 1 -- 172.21.15.88:0/893256367 shutdown_connections 2024-02-19T19:16:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.406+0000 7f24af6dc700 1 --2- 172.21.15.88:0/893256367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24a8100e80 0x7f24a8104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.406+0000 7f24af6dc700 1 --2- 172.21.15.88:0/893256367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a81004e0 0x7f24a81008b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.406+0000 7f24af6dc700 1 -- 172.21.15.88:0/893256367 >> 172.21.15.88:0/893256367 conn(0x7f24a80fbda0 msgr2=0x7f24a80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.406+0000 7f24af6dc700 1 -- 172.21.15.88:0/893256367 shutdown_connections 2024-02-19T19:16:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.407+0000 7f24af6dc700 1 -- 172.21.15.88:0/893256367 wait complete. 2024-02-19T19:16:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.407+0000 7f24af6dc700 1 Processor -- start 2024-02-19T19:16:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.407+0000 7f24af6dc700 1 -- start start 2024-02-19T19:16:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.407+0000 7f24af6dc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a81004e0 0x7f24a8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.407+0000 7f24af6dc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24a8100e80 0x7f24a8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.407+0000 7f24af6dc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a8073ad0 con 0x7f24a81004e0 2024-02-19T19:16:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.407+0000 7f24af6dc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24a8073c10 con 0x7f24a8100e80 2024-02-19T19:16:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.407+0000 7f24ad478700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a81004e0 0x7f24a8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.407+0000 7f24ad478700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a81004e0 0x7f24a8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46518/0 (socket says 172.21.15.88:46518) 2024-02-19T19:16:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.407+0000 7f24ad478700 1 -- 172.21.15.88:0/3700079716 learned_addr learned my addr 172.21.15.88:0/3700079716 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.407+0000 7f24acc77700 1 --2- 172.21.15.88:0/3700079716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24a8100e80 0x7f24a8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.408+0000 7f24ad478700 1 -- 172.21.15.88:0/3700079716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24a8100e80 msgr2=0x7f24a8073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.408+0000 7f24ad478700 1 --2- 172.21.15.88:0/3700079716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24a8100e80 0x7f24a8073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.408+0000 7f24ad478700 1 -- 172.21.15.88:0/3700079716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a40095f0 con 0x7f24a81004e0 2024-02-19T19:16:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.408+0000 7f24ad478700 1 --2- 172.21.15.88:0/3700079716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a81004e0 0x7f24a8073050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f24a4006010 tx=0x7f24a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:39.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.408+0000 7f249e7fc700 1 -- 172.21.15.88:0/3700079716 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24a401d070 con 0x7f24a81004e0 2024-02-19T19:16:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.408+0000 7f249e7fc700 1 -- 172.21.15.88:0/3700079716 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24a4004fb0 con 0x7f24a81004e0 2024-02-19T19:16:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.408+0000 7f24af6dc700 1 -- 172.21.15.88:0/3700079716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24a8073e90 con 0x7f24a81004e0 2024-02-19T19:16:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.408+0000 7f249e7fc700 1 -- 172.21.15.88:0/3700079716 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24a4022ea0 con 0x7f24a81004e0 2024-02-19T19:16:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.408+0000 7f24af6dc700 1 -- 172.21.15.88:0/3700079716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a81a23a0 con 0x7f24a81004e0 2024-02-19T19:16:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.409+0000 7f24af6dc700 1 -- 172.21.15.88:0/3700079716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24a81087f0 con 0x7f24a81004e0 2024-02-19T19:16:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.410+0000 7f249e7fc700 1 -- 172.21.15.88:0/3700079716 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f24a4022470 con 0x7f24a81004e0 2024-02-19T19:16:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.410+0000 7f249e7fc700 1 --2- 172.21.15.88:0/3700079716 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f249406d8a0 0x7f249406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.410+0000 7f249e7fc700 1 -- 172.21.15.88:0/3700079716 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f24a408ea20 con 0x7f24a81004e0 2024-02-19T19:16:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.410+0000 7f24acc77700 1 --2- 172.21.15.88:0/3700079716 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f249406d8a0 0x7f249406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.410+0000 7f24acc77700 1 --2- 172.21.15.88:0/3700079716 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f249406d8a0 0x7f249406fd50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2498005ea0 tx=0x7f2498005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.415+0000 7f249e7fc700 1 -- 172.21.15.88:0/3700079716 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f24a4058420 con 0x7f24a81004e0 2024-02-19T19:16:39.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:39 smithi088 ceph-mon[88643]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:39.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:39 smithi088 ceph-mon[88643]: pgmap v391: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:39.598 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:39 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1335058887' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:16:39.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.659+0000 7f24af6dc700 1 -- 172.21.15.88:0/3700079716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f24a804f2a0 con 0x7f24a81004e0 2024-02-19T19:16:39.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.660+0000 7f249e7fc700 1 -- 172.21.15.88:0/3700079716 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f24a4027070 con 0x7f24a81004e0 2024-02-19T19:16:39.678 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:16:39.678 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:16:39.678 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:16:39.678 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:16:39.678 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:16:39.678 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:16:39.678 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:16:39.678 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:16:39.678 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:16:39.678 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:16:39.679 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:16:39.679 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:16:39.679 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:16:39.679 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:16:39.679 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:16:39.679 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:16:39.679 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:16:39.679 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:16:39.679 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:16:39.679 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:16:39.679 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:16:39.680 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:16:39.680 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:16:39.680 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:16:39.680 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:16:39.680 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:16:39.680 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:16:39.680 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:16:39.680 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:16:39.680 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:16:39.680 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:16:39.680 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:16:39.681 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:16:39.681 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:16:39.681 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:16:39.681 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:16:39.681 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:16:39.681 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:16:39.681 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:16:39.681 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:16:39.681 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:16:39.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.662+0000 7f24af6dc700 1 -- 172.21.15.88:0/3700079716 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f249406d8a0 msgr2=0x7f249406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:39.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.662+0000 7f24af6dc700 1 --2- 172.21.15.88:0/3700079716 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f249406d8a0 0x7f249406fd50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2498005ea0 tx=0x7f2498005e30 comp rx=0 tx=0).stop 2024-02-19T19:16:39.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.662+0000 7f24af6dc700 1 -- 172.21.15.88:0/3700079716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a81004e0 msgr2=0x7f24a8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:39.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.662+0000 7f24af6dc700 1 --2- 172.21.15.88:0/3700079716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a81004e0 0x7f24a8073050 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f24a4006010 tx=0x7f24a400be20 comp rx=0 tx=0).stop 2024-02-19T19:16:39.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.662+0000 7f24af6dc700 1 -- 172.21.15.88:0/3700079716 shutdown_connections 2024-02-19T19:16:39.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.662+0000 7f24af6dc700 1 --2- 172.21.15.88:0/3700079716 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f249406d8a0 0x7f249406fd50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.662+0000 7f24af6dc700 1 --2- 172.21.15.88:0/3700079716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24a81004e0 0x7f24a8073050 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.662+0000 7f24af6dc700 1 --2- 172.21.15.88:0/3700079716 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24a8100e80 0x7f24a8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.662+0000 7f24af6dc700 1 -- 172.21.15.88:0/3700079716 >> 172.21.15.88:0/3700079716 conn(0x7f24a80fbda0 msgr2=0x7f24a80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:39.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.662+0000 7f24af6dc700 1 -- 172.21.15.88:0/3700079716 shutdown_connections 2024-02-19T19:16:39.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.662+0000 7f24af6dc700 1 -- 172.21.15.88:0/3700079716 wait complete. 2024-02-19T19:16:39.684 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:16:39.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:39 smithi132 ceph-mon[70676]: from='client.34750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:39.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:39 smithi132 ceph-mon[70676]: pgmap v391: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:39.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:39 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1335058887' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:16:39.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.767+0000 7faa78e01700 1 -- 172.21.15.88:0/763125974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa741004a0 msgr2=0x7faa74100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:39.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.767+0000 7faa78e01700 1 --2- 172.21.15.88:0/763125974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa741004a0 0x7faa74100870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faa68009910 tx=0x7faa68009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:39.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.767+0000 7faa78e01700 1 -- 172.21.15.88:0/763125974 shutdown_connections 2024-02-19T19:16:39.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.767+0000 7faa78e01700 1 --2- 172.21.15.88:0/763125974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa74100e40 0x7faa74104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.767+0000 7faa78e01700 1 --2- 172.21.15.88:0/763125974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa741004a0 0x7faa74100870 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.767+0000 7faa78e01700 1 -- 172.21.15.88:0/763125974 >> 172.21.15.88:0/763125974 conn(0x7faa740fbd40 msgr2=0x7faa740fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:39.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.767+0000 7faa78e01700 1 -- 172.21.15.88:0/763125974 shutdown_connections 2024-02-19T19:16:39.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.767+0000 7faa78e01700 1 -- 172.21.15.88:0/763125974 wait complete. 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.768+0000 7faa78e01700 1 Processor -- start 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.768+0000 7faa78e01700 1 -- start start 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.768+0000 7faa78e01700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa741004a0 0x7faa74195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.768+0000 7faa78e01700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa74100e40 0x7faa74196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.768+0000 7faa78e01700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa74196af0 con 0x7faa741004a0 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.768+0000 7faa78e01700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7419a880 con 0x7faa74100e40 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.768+0000 7faa7259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa741004a0 0x7faa74195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.768+0000 7faa7259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa741004a0 0x7faa74195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46540/0 (socket says 172.21.15.88:46540) 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.768+0000 7faa7259c700 1 -- 172.21.15.88:0/4136132152 learned_addr learned my addr 172.21.15.88:0/4136132152 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.769+0000 7faa71d9b700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa74100e40 0x7faa74196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.769+0000 7faa7259c700 1 -- 172.21.15.88:0/4136132152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa74100e40 msgr2=0x7faa74196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:39.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.769+0000 7faa7259c700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa74100e40 0x7faa74196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.769+0000 7faa7259c700 1 -- 172.21.15.88:0/4136132152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa680095f0 con 0x7faa741004a0 2024-02-19T19:16:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.769+0000 7faa71d9b700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa74100e40 0x7faa74196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:16:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.769+0000 7faa7259c700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa741004a0 0x7faa74195ed0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faa68006010 tx=0x7faa6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.769+0000 7faa637fe700 1 -- 172.21.15.88:0/4136132152 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa6801d070 con 0x7faa741004a0 2024-02-19T19:16:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.769+0000 7faa78e01700 1 -- 172.21.15.88:0/4136132152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7419ab00 con 0x7faa741004a0 2024-02-19T19:16:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.769+0000 7faa637fe700 1 -- 172.21.15.88:0/4136132152 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa6800f460 con 0x7faa741004a0 2024-02-19T19:16:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.769+0000 7faa637fe700 1 -- 172.21.15.88:0/4136132152 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa68017630 con 0x7faa741004a0 2024-02-19T19:16:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.769+0000 7faa78e01700 1 -- 172.21.15.88:0/4136132152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7419aff0 con 0x7faa741004a0 2024-02-19T19:16:39.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.770+0000 7faa78e01700 1 -- 172.21.15.88:0/4136132152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa741087b0 con 0x7faa741004a0 2024-02-19T19:16:39.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.771+0000 7faa637fe700 1 -- 172.21.15.88:0/4136132152 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7faa6800f5d0 con 0x7faa741004a0 2024-02-19T19:16:39.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.771+0000 7faa637fe700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa5c06d8a0 0x7faa5c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:39.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.771+0000 7faa71d9b700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa5c06d8a0 0x7faa5c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:39.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.771+0000 7faa637fe700 1 -- 172.21.15.88:0/4136132152 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faa6808e9f0 con 0x7faa741004a0 2024-02-19T19:16:39.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.771+0000 7faa71d9b700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa5c06d8a0 0x7faa5c06fd50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faa64005950 tx=0x7faa640058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:39.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.776+0000 7faa637fe700 1 -- 172.21.15.88:0/4136132152 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa68090e60 con 0x7faa741004a0 2024-02-19T19:16:40.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.990+0000 7faa78e01700 1 -- 172.21.15.88:0/4136132152 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa7419b210 con 0x7faa5c06d8a0 2024-02-19T19:16:40.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.991+0000 7faa637fe700 1 -- 172.21.15.88:0/4136132152 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faa7419b210 con 0x7faa5c06d8a0 2024-02-19T19:16:40.009 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:16:40.009 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:16:40.009 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:16:40.009 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:16:40.009 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:16:40.009 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:16:40.009 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:16:40.010 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:16:40.010 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:16:40.010 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:16:40.010 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:16:40.010 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:16:40.010 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:16:40.010 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:16:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.994+0000 7faa78e01700 1 -- 172.21.15.88:0/4136132152 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa5c06d8a0 msgr2=0x7faa5c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.994+0000 7faa78e01700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa5c06d8a0 0x7faa5c06fd50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faa64005950 tx=0x7faa640058e0 comp rx=0 tx=0).stop 2024-02-19T19:16:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.994+0000 7faa78e01700 1 -- 172.21.15.88:0/4136132152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa741004a0 msgr2=0x7faa74195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.994+0000 7faa78e01700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa741004a0 0x7faa74195ed0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7faa68006010 tx=0x7faa6800be20 comp rx=0 tx=0).stop 2024-02-19T19:16:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.994+0000 7faa78e01700 1 -- 172.21.15.88:0/4136132152 shutdown_connections 2024-02-19T19:16:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.994+0000 7faa78e01700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa5c06d8a0 0x7faa5c06fd50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.994+0000 7faa78e01700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa741004a0 0x7faa74195ed0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.994+0000 7faa78e01700 1 --2- 172.21.15.88:0/4136132152 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa74100e40 0x7faa74196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.994+0000 7faa78e01700 1 -- 172.21.15.88:0/4136132152 >> 172.21.15.88:0/4136132152 conn(0x7faa740fbd40 msgr2=0x7faa740fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.994+0000 7faa78e01700 1 -- 172.21.15.88:0/4136132152 shutdown_connections 2024-02-19T19:16:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:39.994+0000 7faa78e01700 1 -- 172.21.15.88:0/4136132152 wait complete. 2024-02-19T19:16:40.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.099+0000 7f1d6b661700 1 -- 172.21.15.88:0/4289870352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d640ffc60 msgr2=0x7f1d6410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:40.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.099+0000 7f1d6b661700 1 --2- 172.21.15.88:0/4289870352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d640ffc60 0x7f1d6410a6d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009910 tx=0x7f1d54009c20 comp rx=0 tx=0).stop 2024-02-19T19:16:40.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.099+0000 7f1d6b661700 1 -- 172.21.15.88:0/4289870352 shutdown_connections 2024-02-19T19:16:40.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.099+0000 7f1d6b661700 1 --2- 172.21.15.88:0/4289870352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d640ffc60 0x7f1d6410a6d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:40.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.099+0000 7f1d6b661700 1 --2- 172.21.15.88:0/4289870352 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d640ff350 0x7f1d640ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:40.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.099+0000 7f1d6b661700 1 -- 172.21.15.88:0/4289870352 >> 172.21.15.88:0/4289870352 conn(0x7f1d640748b0 msgr2=0x7f1d64074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:40.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.099+0000 7f1d6b661700 1 -- 172.21.15.88:0/4289870352 shutdown_connections 2024-02-19T19:16:40.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.099+0000 7f1d6b661700 1 -- 172.21.15.88:0/4289870352 wait complete. 2024-02-19T19:16:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.099+0000 7f1d6b661700 1 Processor -- start 2024-02-19T19:16:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.099+0000 7f1d6b661700 1 -- start start 2024-02-19T19:16:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d6b661700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d640ff350 0x7f1d64195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d6b661700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d640ffc60 0x7f1d64196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d6b661700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64196a60 con 0x7f1d640ffc60 2024-02-19T19:16:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d6b661700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64196ba0 con 0x7f1d640ff350 2024-02-19T19:16:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d68bfc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d640ffc60 0x7f1d64196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d68bfc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d640ffc60 0x7f1d64196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46562/0 (socket says 172.21.15.88:46562) 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d68bfc700 1 -- 172.21.15.88:0/972891756 learned_addr learned my addr 172.21.15.88:0/972891756 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d693fd700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d640ff350 0x7f1d64195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d68bfc700 1 -- 172.21.15.88:0/972891756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d640ff350 msgr2=0x7f1d64195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d68bfc700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d640ff350 0x7f1d64195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d68bfc700 1 -- 172.21.15.88:0/972891756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d540095f0 con 0x7f1d640ffc60 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d693fd700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d640ff350 0x7f1d64195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d68bfc700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d640ffc60 0x7f1d64196410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006010 tx=0x7f1d5400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d5a7fc700 1 -- 172.21.15.88:0/972891756 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d5401d070 con 0x7f1d640ffc60 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d5a7fc700 1 -- 172.21.15.88:0/972891756 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d54004fb0 con 0x7f1d640ffc60 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d6b661700 1 -- 172.21.15.88:0/972891756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6419a990 con 0x7f1d640ffc60 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.100+0000 7f1d5a7fc700 1 -- 172.21.15.88:0/972891756 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d54022ea0 con 0x7f1d640ffc60 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.101+0000 7f1d6b661700 1 -- 172.21.15.88:0/972891756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6419ae80 con 0x7f1d640ffc60 2024-02-19T19:16:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.101+0000 7f1d6b661700 1 -- 172.21.15.88:0/972891756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d6404f2a0 con 0x7f1d640ffc60 2024-02-19T19:16:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.102+0000 7f1d5a7fc700 1 -- 172.21.15.88:0/972891756 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1d54022470 con 0x7f1d640ffc60 2024-02-19T19:16:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.102+0000 7f1d5a7fc700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d5006d8a0 0x7f1d5006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:16:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.102+0000 7f1d5a7fc700 1 -- 172.21.15.88:0/972891756 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1d5408ec10 con 0x7f1d640ffc60 2024-02-19T19:16:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.102+0000 7f1d693fd700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d5006d8a0 0x7f1d5006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:16:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.103+0000 7f1d693fd700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d5006d8a0 0x7f1d5006fd50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d6406a660 tx=0x7f1d6000a680 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:16:40.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.107+0000 7f1d5a7fc700 1 -- 172.21.15.88:0/972891756 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d54058590 con 0x7f1d640ffc60 2024-02-19T19:16:40.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.384+0000 7f1d6b661700 1 -- 172.21.15.88:0/972891756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1d64107850 con 0x7f1d640ffc60 2024-02-19T19:16:40.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.384+0000 7f1d5a7fc700 1 -- 172.21.15.88:0/972891756 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f1d54027090 con 0x7f1d640ffc60 2024-02-19T19:16:40.403 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:16:40.403 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:16:40.403 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:16:40.403 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:16:40.403 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:16:40.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.387+0000 7f1d6b661700 1 -- 172.21.15.88:0/972891756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d5006d8a0 msgr2=0x7f1d5006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.387+0000 7f1d6b661700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d5006d8a0 0x7f1d5006fd50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1d6406a660 tx=0x7f1d6000a680 comp rx=0 tx=0).stop 2024-02-19T19:16:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.387+0000 7f1d6b661700 1 -- 172.21.15.88:0/972891756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d640ffc60 msgr2=0x7f1d64196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:16:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.387+0000 7f1d6b661700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d640ffc60 0x7f1d64196410 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1d54006010 tx=0x7f1d5400be20 comp rx=0 tx=0).stop 2024-02-19T19:16:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.387+0000 7f1d6b661700 1 -- 172.21.15.88:0/972891756 shutdown_connections 2024-02-19T19:16:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.387+0000 7f1d6b661700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d5006d8a0 0x7f1d5006fd50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.387+0000 7f1d6b661700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d640ff350 0x7f1d64195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.387+0000 7f1d6b661700 1 --2- 172.21.15.88:0/972891756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d640ffc60 0x7f1d64196410 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:16:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.387+0000 7f1d6b661700 1 -- 172.21.15.88:0/972891756 >> 172.21.15.88:0/972891756 conn(0x7f1d640748b0 msgr2=0x7f1d64104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:16:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.387+0000 7f1d6b661700 1 -- 172.21.15.88:0/972891756 shutdown_connections 2024-02-19T19:16:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:16:40.387+0000 7f1d6b661700 1 -- 172.21.15.88:0/972891756 wait complete. 2024-02-19T19:16:40.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:40 smithi132 ceph-mon[70676]: from='client.34754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:40.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:40 smithi132 ceph-mon[70676]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:40.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:40 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3700079716' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:16:40.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:40 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/972891756' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:16:40.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:40 smithi088 ceph-mon[88643]: from='client.34754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:40.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:40 smithi088 ceph-mon[88643]: from='client.34758 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:40.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:40 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3700079716' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:16:40.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:40 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/972891756' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:16:41.750 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:41 smithi088 ceph-mon[88643]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:41.750 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:41 smithi088 ceph-mon[88643]: pgmap v392: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:41.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:41 smithi132 ceph-mon[70676]: from='client.34770 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:16:41.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:41 smithi132 ceph-mon[70676]: pgmap v392: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:43.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:43 smithi132 ceph-mon[70676]: pgmap v393: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:43.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:16:43.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:43 smithi088 ceph-mon[88643]: pgmap v393: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:43.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:16:45.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:45 smithi132 ceph-mon[70676]: pgmap v394: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:45.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:45 smithi088 ceph-mon[88643]: pgmap v394: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:47.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:47 smithi132 ceph-mon[70676]: pgmap v395: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:47.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:47 smithi088 ceph-mon[88643]: pgmap v395: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:49.750 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:49 smithi132 ceph-mon[70676]: pgmap v396: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:49.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:49 smithi088 ceph-mon[88643]: pgmap v396: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:51.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:51 smithi132 ceph-mon[70676]: pgmap v397: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:51.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:51 smithi088 ceph-mon[88643]: pgmap v397: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:53.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:53 smithi132 ceph-mon[70676]: pgmap v398: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:53.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:53 smithi088 ceph-mon[88643]: pgmap v398: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:55.500 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:55 smithi132 ceph-mon[70676]: pgmap v399: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:55.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:55 smithi088 ceph-mon[88643]: pgmap v399: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:57.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:57 smithi132 ceph-mon[70676]: pgmap v400: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:57.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:57 smithi088 ceph-mon[88643]: pgmap v400: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:58.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:16:58.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:16:59.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:16:59 smithi132 ceph-mon[70676]: pgmap v401: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:16:59.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:16:59 smithi088 ceph-mon[88643]: pgmap v401: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:01.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:01 smithi132 ceph-mon[70676]: pgmap v402: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:01.848 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:01 smithi088 ceph-mon[88643]: pgmap v402: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:03.751 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:03 smithi132 ceph-mon[70676]: pgmap v403: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:03.850 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:03 smithi088 ceph-mon[88643]: pgmap v403: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:05.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:05 smithi132 ceph-mon[70676]: pgmap v404: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:05.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:05 smithi088 ceph-mon[88643]: pgmap v404: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:07.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:07 smithi132 ceph-mon[70676]: pgmap v405: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:07.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:07 smithi088 ceph-mon[88643]: pgmap v405: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:09.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:09 smithi132 ceph-mon[70676]: pgmap v406: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:09.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:09 smithi088 ceph-mon[88643]: pgmap v406: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:10.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.506+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3826755788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8c8100530 msgr2=0x7fd8c8100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:10.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.506+0000 7fd8cdcdc700 1 --2- 172.21.15.88:0/3826755788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8c8100530 0x7fd8c8100900 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800b0a0 tx=0x7fd8b800b3b0 comp rx=0 tx=0).stop 2024-02-19T19:17:10.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.506+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3826755788 shutdown_connections 2024-02-19T19:17:10.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.506+0000 7fd8cdcdc700 1 --2- 172.21.15.88:0/3826755788 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c8100ed0 0x7fd8c8104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:10.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.506+0000 7fd8cdcdc700 1 --2- 172.21.15.88:0/3826755788 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8c8100530 0x7fd8c8100900 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:10.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.506+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3826755788 >> 172.21.15.88:0/3826755788 conn(0x7fd8c80fbe10 msgr2=0x7fd8c80fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:10.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.506+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3826755788 shutdown_connections 2024-02-19T19:17:10.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.506+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3826755788 wait complete. 2024-02-19T19:17:10.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.507+0000 7fd8cdcdc700 1 Processor -- start 2024-02-19T19:17:10.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.507+0000 7fd8cdcdc700 1 -- start start 2024-02-19T19:17:10.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8cdcdc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8c8100530 0x7fd8c8195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:10.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8cdcdc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c8100ed0 0x7fd8c8196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:10.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8cdcdc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c8196a50 con 0x7fd8c8100530 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8cdcdc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c819a7e0 con 0x7fd8c8100ed0 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8c77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8c8100530 0x7fd8c8195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8c77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8c8100530 0x7fd8c8195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56080/0 (socket says 172.21.15.88:56080) 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8c77fe700 1 -- 172.21.15.88:0/3955213326 learned_addr learned my addr 172.21.15.88:0/3955213326 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8c6ffd700 1 --2- 172.21.15.88:0/3955213326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c8100ed0 0x7fd8c8196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8c77fe700 1 -- 172.21.15.88:0/3955213326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c8100ed0 msgr2=0x7fd8c8196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8c77fe700 1 --2- 172.21.15.88:0/3955213326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c8100ed0 0x7fd8c8196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8c77fe700 1 -- 172.21.15.88:0/3955213326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b8009d00 con 0x7fd8c8100530 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8c77fe700 1 --2- 172.21.15.88:0/3955213326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8c8100530 0x7fd8c8195e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8000f80 tx=0x7fd8b800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.508+0000 7fd8c4ff9700 1 -- 172.21.15.88:0/3955213326 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8b800e040 con 0x7fd8c8100530 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.509+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3955213326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c819aa60 con 0x7fd8c8100530 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.509+0000 7fd8c4ff9700 1 -- 172.21.15.88:0/3955213326 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8b8003c10 con 0x7fd8c8100530 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.509+0000 7fd8c4ff9700 1 -- 172.21.15.88:0/3955213326 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8b801d6b0 con 0x7fd8c8100530 2024-02-19T19:17:10.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.509+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3955213326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c819af50 con 0x7fd8c8100530 2024-02-19T19:17:10.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.510+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3955213326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c8108840 con 0x7fd8c8100530 2024-02-19T19:17:10.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.510+0000 7fd8c4ff9700 1 -- 172.21.15.88:0/3955213326 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd8b8004040 con 0x7fd8c8100530 2024-02-19T19:17:10.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.511+0000 7fd8c4ff9700 1 --2- 172.21.15.88:0/3955213326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd8b406d8a0 0x7fd8b406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:10.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.511+0000 7fd8c6ffd700 1 --2- 172.21.15.88:0/3955213326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd8b406d8a0 0x7fd8b406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:10.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.511+0000 7fd8c4ff9700 1 -- 172.21.15.88:0/3955213326 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd8b808ec60 con 0x7fd8c8100530 2024-02-19T19:17:10.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.511+0000 7fd8c6ffd700 1 --2- 172.21.15.88:0/3955213326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd8b406d8a0 0x7fd8b406fd50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8197450 tx=0x7fd8b000b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:10.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.516+0000 7fd8c4ff9700 1 -- 172.21.15.88:0/3955213326 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd8b8058410 con 0x7fd8c8100530 2024-02-19T19:17:10.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.732+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3955213326 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8c819b170 con 0x7fd8b406d8a0 2024-02-19T19:17:10.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.734+0000 7fd8c4ff9700 1 -- 172.21.15.88:0/3955213326 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd8c819b170 con 0x7fd8b406d8a0 2024-02-19T19:17:10.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.736+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3955213326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd8b406d8a0 msgr2=0x7fd8b406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:10.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.736+0000 7fd8cdcdc700 1 --2- 172.21.15.88:0/3955213326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd8b406d8a0 0x7fd8b406fd50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8197450 tx=0x7fd8b000b410 comp rx=0 tx=0).stop 2024-02-19T19:17:10.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.736+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3955213326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8c8100530 msgr2=0x7fd8c8195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:10.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.736+0000 7fd8cdcdc700 1 --2- 172.21.15.88:0/3955213326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8c8100530 0x7fd8c8195e30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd8b8000f80 tx=0x7fd8b800cd20 comp rx=0 tx=0).stop 2024-02-19T19:17:10.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.736+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3955213326 shutdown_connections 2024-02-19T19:17:10.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.736+0000 7fd8cdcdc700 1 --2- 172.21.15.88:0/3955213326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd8b406d8a0 0x7fd8b406fd50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:10.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.736+0000 7fd8cdcdc700 1 --2- 172.21.15.88:0/3955213326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8c8100530 0x7fd8c8195e30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:10.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.736+0000 7fd8cdcdc700 1 --2- 172.21.15.88:0/3955213326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd8c8100ed0 0x7fd8c8196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:10.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.736+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3955213326 >> 172.21.15.88:0/3955213326 conn(0x7fd8c80fbe10 msgr2=0x7fd8c80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:10.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.736+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3955213326 shutdown_connections 2024-02-19T19:17:10.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.736+0000 7fd8cdcdc700 1 -- 172.21.15.88:0/3955213326 wait complete. 2024-02-19T19:17:10.763 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:17:10.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.841+0000 7f19b5120700 1 -- 172.21.15.88:0/111434941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19b0100460 msgr2=0x7f19b0100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:10.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.841+0000 7f19b5120700 1 --2- 172.21.15.88:0/111434941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19b0100460 0x7f19b0100830 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009940 tx=0x7f19a0009c50 comp rx=0 tx=0).stop 2024-02-19T19:17:10.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.841+0000 7f19b5120700 1 -- 172.21.15.88:0/111434941 shutdown_connections 2024-02-19T19:17:10.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.841+0000 7f19b5120700 1 --2- 172.21.15.88:0/111434941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19b0100d70 0x7f19b01094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:10.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.841+0000 7f19b5120700 1 --2- 172.21.15.88:0/111434941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19b0100460 0x7f19b0100830 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:10.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.841+0000 7f19b5120700 1 -- 172.21.15.88:0/111434941 >> 172.21.15.88:0/111434941 conn(0x7f19b00748b0 msgr2=0x7f19b0074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:10.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.841+0000 7f19b5120700 1 -- 172.21.15.88:0/111434941 shutdown_connections 2024-02-19T19:17:10.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.841+0000 7f19b5120700 1 -- 172.21.15.88:0/111434941 wait complete. 2024-02-19T19:17:10.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.841+0000 7f19b5120700 1 Processor -- start 2024-02-19T19:17:10.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19b5120700 1 -- start start 2024-02-19T19:17:10.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19b5120700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19b0100460 0x7f19b019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:10.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19b5120700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19b0100d70 0x7f19b019af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:10.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19b5120700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b019b600 con 0x7f19b0100460 2024-02-19T19:17:10.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19b5120700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b0194ab0 con 0x7f19b0100d70 2024-02-19T19:17:10.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19aed9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19b0100460 0x7f19b019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:10.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19aed9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19b0100460 0x7f19b019aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56092/0 (socket says 172.21.15.88:56092) 2024-02-19T19:17:10.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19aed9d700 1 -- 172.21.15.88:0/2023007548 learned_addr learned my addr 172.21.15.88:0/2023007548 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:10.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19aed9d700 1 -- 172.21.15.88:0/2023007548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19b0100d70 msgr2=0x7f19b019af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:10.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19aed9d700 1 --2- 172.21.15.88:0/2023007548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19b0100d70 0x7f19b019af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:10.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19aed9d700 1 -- 172.21.15.88:0/2023007548 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a00095f0 con 0x7f19b0100460 2024-02-19T19:17:10.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19aed9d700 1 --2- 172.21.15.88:0/2023007548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19b0100460 0x7f19b019aa30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f19a000b580 tx=0x7f19a00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:10.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.842+0000 7f19acd99700 1 -- 172.21.15.88:0/2023007548 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19a001d070 con 0x7f19b0100460 2024-02-19T19:17:10.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.843+0000 7f19acd99700 1 -- 172.21.15.88:0/2023007548 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19a0005dc0 con 0x7f19b0100460 2024-02-19T19:17:10.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.843+0000 7f19b5120700 1 -- 172.21.15.88:0/2023007548 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b0194d90 con 0x7f19b0100460 2024-02-19T19:17:10.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.843+0000 7f19acd99700 1 -- 172.21.15.88:0/2023007548 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19a0021410 con 0x7f19b0100460 2024-02-19T19:17:10.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.843+0000 7f19b5120700 1 -- 172.21.15.88:0/2023007548 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b01952e0 con 0x7f19b0100460 2024-02-19T19:17:10.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.844+0000 7f19b5120700 1 -- 172.21.15.88:0/2023007548 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b0106bd0 con 0x7f19b0100460 2024-02-19T19:17:10.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.844+0000 7f19acd99700 1 -- 172.21.15.88:0/2023007548 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f19a000f7b0 con 0x7f19b0100460 2024-02-19T19:17:10.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.845+0000 7f19acd99700 1 --2- 172.21.15.88:0/2023007548 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f199c06d8a0 0x7f199c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:10.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.845+0000 7f19a659c700 1 --2- 172.21.15.88:0/2023007548 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f199c06d8a0 0x7f199c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:10.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.845+0000 7f19acd99700 1 -- 172.21.15.88:0/2023007548 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f19a008e780 con 0x7f19b0100460 2024-02-19T19:17:10.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.845+0000 7f19a659c700 1 --2- 172.21.15.88:0/2023007548 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f199c06d8a0 0x7f199c06fd50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1998005950 tx=0x7f19980058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:10.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:10.849+0000 7f19acd99700 1 -- 172.21.15.88:0/2023007548 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f19a005ba60 con 0x7f19b0100460 2024-02-19T19:17:11.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.064+0000 7f19b5120700 1 -- 172.21.15.88:0/2023007548 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19b0195500 con 0x7f199c06d8a0 2024-02-19T19:17:11.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.065+0000 7f19acd99700 1 -- 172.21.15.88:0/2023007548 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f19b0195500 con 0x7f199c06d8a0 2024-02-19T19:17:11.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.067+0000 7f19b5120700 1 -- 172.21.15.88:0/2023007548 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f199c06d8a0 msgr2=0x7f199c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.067+0000 7f19b5120700 1 --2- 172.21.15.88:0/2023007548 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f199c06d8a0 0x7f199c06fd50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1998005950 tx=0x7f19980058e0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.067+0000 7f19b5120700 1 -- 172.21.15.88:0/2023007548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19b0100460 msgr2=0x7f19b019aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.067+0000 7f19b5120700 1 --2- 172.21.15.88:0/2023007548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19b0100460 0x7f19b019aa30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f19a000b580 tx=0x7f19a00056a0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.067+0000 7f19b5120700 1 -- 172.21.15.88:0/2023007548 shutdown_connections 2024-02-19T19:17:11.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.067+0000 7f19b5120700 1 --2- 172.21.15.88:0/2023007548 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f199c06d8a0 0x7f199c06fd50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.067+0000 7f19b5120700 1 --2- 172.21.15.88:0/2023007548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19b0100460 0x7f19b019aa30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.067+0000 7f19b5120700 1 --2- 172.21.15.88:0/2023007548 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19b0100d70 0x7f19b019af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.067+0000 7f19b5120700 1 -- 172.21.15.88:0/2023007548 >> 172.21.15.88:0/2023007548 conn(0x7f19b00748b0 msgr2=0x7f19b0103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:11.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.067+0000 7f19b5120700 1 -- 172.21.15.88:0/2023007548 shutdown_connections 2024-02-19T19:17:11.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.067+0000 7f19b5120700 1 -- 172.21.15.88:0/2023007548 wait complete. 2024-02-19T19:17:11.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.170+0000 7feaeaef6700 1 -- 172.21.15.88:0/3631061387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feae40ff350 msgr2=0x7feae40ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.170+0000 7feaeaef6700 1 --2- 172.21.15.88:0/3631061387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feae40ff350 0x7feae40ff720 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fead8009910 tx=0x7fead8009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:11.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.170+0000 7feaeaef6700 1 -- 172.21.15.88:0/3631061387 shutdown_connections 2024-02-19T19:17:11.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.170+0000 7feaeaef6700 1 --2- 172.21.15.88:0/3631061387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40ffc60 0x7feae410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.170+0000 7feaeaef6700 1 --2- 172.21.15.88:0/3631061387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feae40ff350 0x7feae40ff720 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.170+0000 7feaeaef6700 1 -- 172.21.15.88:0/3631061387 >> 172.21.15.88:0/3631061387 conn(0x7feae40748b0 msgr2=0x7feae4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:11.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.171+0000 7feaeaef6700 1 -- 172.21.15.88:0/3631061387 shutdown_connections 2024-02-19T19:17:11.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.171+0000 7feaeaef6700 1 -- 172.21.15.88:0/3631061387 wait complete. 2024-02-19T19:17:11.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.171+0000 7feaeaef6700 1 Processor -- start 2024-02-19T19:17:11.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.171+0000 7feaeaef6700 1 -- start start 2024-02-19T19:17:11.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feaeaef6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feae40ff350 0x7feae4195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:11.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feaeaef6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40ffc60 0x7feae41964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:11.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feaeaef6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae4196bc0 con 0x7feae40ff350 2024-02-19T19:17:11.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feaeaef6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae419a950 con 0x7feae40ffc60 2024-02-19T19:17:11.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feae8c92700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feae40ff350 0x7feae4195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:11.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feae8c92700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feae40ff350 0x7feae4195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56116/0 (socket says 172.21.15.88:56116) 2024-02-19T19:17:11.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feae8c92700 1 -- 172.21.15.88:0/1932734000 learned_addr learned my addr 172.21.15.88:0/1932734000 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feae3fff700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40ffc60 0x7feae41964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feae8c92700 1 -- 172.21.15.88:0/1932734000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40ffc60 msgr2=0x7feae41964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feae8c92700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40ffc60 0x7feae41964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feae8c92700 1 -- 172.21.15.88:0/1932734000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead80095f0 con 0x7feae40ff350 2024-02-19T19:17:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feae3fff700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40ffc60 0x7feae41964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:17:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.172+0000 7feae8c92700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feae40ff350 0x7feae4195fa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fead8000c00 tx=0x7fead80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.173+0000 7feae1ffb700 1 -- 172.21.15.88:0/1932734000 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead801d070 con 0x7feae40ff350 2024-02-19T19:17:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.173+0000 7feaeaef6700 1 -- 172.21.15.88:0/1932734000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae419ac30 con 0x7feae40ff350 2024-02-19T19:17:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.173+0000 7feae1ffb700 1 -- 172.21.15.88:0/1932734000 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fead800f460 con 0x7feae40ff350 2024-02-19T19:17:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.173+0000 7feae1ffb700 1 -- 172.21.15.88:0/1932734000 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead8021640 con 0x7feae40ff350 2024-02-19T19:17:11.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.173+0000 7feaeaef6700 1 -- 172.21.15.88:0/1932734000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae419b180 con 0x7feae40ff350 2024-02-19T19:17:11.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.174+0000 7feaeaef6700 1 -- 172.21.15.88:0/1932734000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae4107dd0 con 0x7feae40ff350 2024-02-19T19:17:11.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.174+0000 7feae1ffb700 1 -- 172.21.15.88:0/1932734000 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fead800f5d0 con 0x7feae40ff350 2024-02-19T19:17:11.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.175+0000 7feae1ffb700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fead406d8a0 0x7fead406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:11.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.175+0000 7feae1ffb700 1 -- 172.21.15.88:0/1932734000 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fead808e800 con 0x7feae40ff350 2024-02-19T19:17:11.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.175+0000 7feae3fff700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fead406d8a0 0x7fead406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:11.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.175+0000 7feae3fff700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fead406d8a0 0x7fead406fd50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feae41975c0 tx=0x7fead0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:11.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.180+0000 7feae1ffb700 1 -- 172.21.15.88:0/1932734000 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fead8058030 con 0x7feae40ff350 2024-02-19T19:17:11.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.384+0000 7feaeaef6700 1 -- 172.21.15.88:0/1932734000 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feae41973a0 con 0x7fead406d8a0 2024-02-19T19:17:11.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.394+0000 7feae1ffb700 1 -- 172.21.15.88:0/1932734000 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7feae41973a0 con 0x7fead406d8a0 2024-02-19T19:17:11.411 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:17:11.411 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (14m) 6m ago 24m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:17:11.411 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (25m) 6m ago 25m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:17:11.411 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (7m) 5m ago 23m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:17:11.411 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (10m) 6m ago 25m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:17:11.411 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (10m) 5m ago 23m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:17:11.411 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (13m) 6m ago 24m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (6m) 6m ago 18m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (6m) 6m ago 18m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (5m) 5m ago 18m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (18m) 5m ago 18m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (15m) 6m ago 26m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (15m) 5m ago 23m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (12m) 6m ago 26m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (11m) 5m ago 23m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (15m) 6m ago 25m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (14m) 5m ago 23m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (9m) 6m ago 22m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (9m) 6m ago 21m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (8m) 6m ago 21m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:17:11.412 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (8m) 5m ago 20m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:17:11.413 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (7m) 5m ago 20m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:17:11.413 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (6m) 5m ago 20m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:17:11.413 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (14m) 6m ago 23m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:17:11.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.396+0000 7feaeaef6700 1 -- 172.21.15.88:0/1932734000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fead406d8a0 msgr2=0x7fead406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.396+0000 7feaeaef6700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fead406d8a0 0x7fead406fd50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7feae41975c0 tx=0x7fead0008040 comp rx=0 tx=0).stop 2024-02-19T19:17:11.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.396+0000 7feaeaef6700 1 -- 172.21.15.88:0/1932734000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feae40ff350 msgr2=0x7feae4195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.396+0000 7feaeaef6700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feae40ff350 0x7feae4195fa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fead8000c00 tx=0x7fead80056a0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.396+0000 7feaeaef6700 1 -- 172.21.15.88:0/1932734000 shutdown_connections 2024-02-19T19:17:11.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.396+0000 7feaeaef6700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fead406d8a0 0x7fead406fd50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.396+0000 7feaeaef6700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feae40ff350 0x7feae4195fa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.396+0000 7feaeaef6700 1 --2- 172.21.15.88:0/1932734000 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feae40ffc60 0x7feae41964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.396+0000 7feaeaef6700 1 -- 172.21.15.88:0/1932734000 >> 172.21.15.88:0/1932734000 conn(0x7feae40748b0 msgr2=0x7feae4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:11.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.396+0000 7feaeaef6700 1 -- 172.21.15.88:0/1932734000 shutdown_connections 2024-02-19T19:17:11.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.396+0000 7feaeaef6700 1 -- 172.21.15.88:0/1932734000 wait complete. 2024-02-19T19:17:11.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.501+0000 7fa99f57a700 1 -- 172.21.15.88:0/3147724512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa998101d30 msgr2=0x7fa99810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.501+0000 7fa99f57a700 1 --2- 172.21.15.88:0/3147724512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa998101d30 0x7fa99810a230 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fa988009910 tx=0x7fa988009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:11.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.501+0000 7fa99f57a700 1 -- 172.21.15.88:0/3147724512 shutdown_connections 2024-02-19T19:17:11.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.501+0000 7fa99f57a700 1 --2- 172.21.15.88:0/3147724512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa998101d30 0x7fa99810a230 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.501+0000 7fa99f57a700 1 --2- 172.21.15.88:0/3147724512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa998101420 0x7fa9981017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.501+0000 7fa99f57a700 1 -- 172.21.15.88:0/3147724512 >> 172.21.15.88:0/3147724512 conn(0x7fa9980fac80 msgr2=0x7fa9980fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:11.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.501+0000 7fa99f57a700 1 -- 172.21.15.88:0/3147724512 shutdown_connections 2024-02-19T19:17:11.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.501+0000 7fa99f57a700 1 -- 172.21.15.88:0/3147724512 wait complete. 2024-02-19T19:17:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.502+0000 7fa99f57a700 1 Processor -- start 2024-02-19T19:17:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.502+0000 7fa99f57a700 1 -- start start 2024-02-19T19:17:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.502+0000 7fa99f57a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa998101420 0x7fa9980ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.502+0000 7fa99f57a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa998101d30 0x7fa9980ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.502+0000 7fa99f57a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9980ffb00 con 0x7fa998101d30 2024-02-19T19:17:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.502+0000 7fa99f57a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9980ffc40 con 0x7fa998101420 2024-02-19T19:17:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.502+0000 7fa99cb15700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa998101d30 0x7fa9980ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.502+0000 7fa99cb15700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa998101d30 0x7fa9980ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56138/0 (socket says 172.21.15.88:56138) 2024-02-19T19:17:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.502+0000 7fa99cb15700 1 -- 172.21.15.88:0/1678744831 learned_addr learned my addr 172.21.15.88:0/1678744831 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.502+0000 7fa99cb15700 1 -- 172.21.15.88:0/1678744831 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa998101420 msgr2=0x7fa9980ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.502+0000 7fa99d316700 1 --2- 172.21.15.88:0/1678744831 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa998101420 0x7fa9980ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:11.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.503+0000 7fa99cb15700 1 --2- 172.21.15.88:0/1678744831 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa998101420 0x7fa9980ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.503+0000 7fa99cb15700 1 -- 172.21.15.88:0/1678744831 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9880095f0 con 0x7fa998101d30 2024-02-19T19:17:11.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.503+0000 7fa99cb15700 1 --2- 172.21.15.88:0/1678744831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa998101d30 0x7fa9980ff5c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa988006010 tx=0x7fa98800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:11.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.503+0000 7fa98e7fc700 1 -- 172.21.15.88:0/1678744831 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa98801d070 con 0x7fa998101d30 2024-02-19T19:17:11.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.503+0000 7fa98e7fc700 1 -- 172.21.15.88:0/1678744831 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa988004fb0 con 0x7fa998101d30 2024-02-19T19:17:11.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.503+0000 7fa99f57a700 1 -- 172.21.15.88:0/1678744831 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9980ffec0 con 0x7fa998101d30 2024-02-19T19:17:11.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.503+0000 7fa98e7fc700 1 -- 172.21.15.88:0/1678744831 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa988022ea0 con 0x7fa998101d30 2024-02-19T19:17:11.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.503+0000 7fa99f57a700 1 -- 172.21.15.88:0/1678744831 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9981a2310 con 0x7fa998101d30 2024-02-19T19:17:11.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.504+0000 7fa99f57a700 1 -- 172.21.15.88:0/1678744831 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9981078a0 con 0x7fa998101d30 2024-02-19T19:17:11.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.504+0000 7fa98e7fc700 1 -- 172.21.15.88:0/1678744831 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa988022470 con 0x7fa998101d30 2024-02-19T19:17:11.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.505+0000 7fa98e7fc700 1 --2- 172.21.15.88:0/1678744831 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa98406d8a0 0x7fa98406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:11.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.505+0000 7fa98e7fc700 1 -- 172.21.15.88:0/1678744831 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa98808ea20 con 0x7fa998101d30 2024-02-19T19:17:11.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.505+0000 7fa99d316700 1 --2- 172.21.15.88:0/1678744831 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa98406d8a0 0x7fa98406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:11.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.505+0000 7fa99d316700 1 --2- 172.21.15.88:0/1678744831 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa98406d8a0 0x7fa98406fd50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa99400ba60 tx=0x7fa994005d50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:11.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.509+0000 7fa98e7fc700 1 -- 172.21.15.88:0/1678744831 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa988058420 con 0x7fa998101d30 2024-02-19T19:17:11.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:11 smithi132 ceph-mon[70676]: pgmap v407: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:11.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.789+0000 7fa99f57a700 1 -- 172.21.15.88:0/1678744831 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa998107280 con 0x7fa998101d30 2024-02-19T19:17:11.806 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:11 smithi088 ceph-mon[88643]: pgmap v407: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:11.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.790+0000 7fa98e7fc700 1 -- 172.21.15.88:0/1678744831 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fa988027340 con 0x7fa998101d30 2024-02-19T19:17:11.807 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:17:11.807 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:17:11.807 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:17:11.807 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:17:11.807 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:17:11.807 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:17:11.807 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:17:11.807 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:17:11.807 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:17:11.808 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:17:11.808 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:17:11.808 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:17:11.808 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:17:11.808 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:17:11.808 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:17:11.808 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:17:11.808 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:17:11.808 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:17:11.808 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:17:11.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.793+0000 7fa99f57a700 1 -- 172.21.15.88:0/1678744831 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa98406d8a0 msgr2=0x7fa98406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.793+0000 7fa99f57a700 1 --2- 172.21.15.88:0/1678744831 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa98406d8a0 0x7fa98406fd50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa99400ba60 tx=0x7fa994005d50 comp rx=0 tx=0).stop 2024-02-19T19:17:11.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.793+0000 7fa99f57a700 1 -- 172.21.15.88:0/1678744831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa998101d30 msgr2=0x7fa9980ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.793+0000 7fa99f57a700 1 --2- 172.21.15.88:0/1678744831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa998101d30 0x7fa9980ff5c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa988006010 tx=0x7fa98800be20 comp rx=0 tx=0).stop 2024-02-19T19:17:11.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.793+0000 7fa99f57a700 1 -- 172.21.15.88:0/1678744831 shutdown_connections 2024-02-19T19:17:11.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.793+0000 7fa99f57a700 1 --2- 172.21.15.88:0/1678744831 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa98406d8a0 0x7fa98406fd50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.793+0000 7fa99f57a700 1 --2- 172.21.15.88:0/1678744831 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa998101420 0x7fa9980ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.793+0000 7fa99f57a700 1 --2- 172.21.15.88:0/1678744831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa998101d30 0x7fa9980ff5c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.793+0000 7fa99f57a700 1 -- 172.21.15.88:0/1678744831 >> 172.21.15.88:0/1678744831 conn(0x7fa9980fac80 msgr2=0x7fa9981049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:11.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.793+0000 7fa99f57a700 1 -- 172.21.15.88:0/1678744831 shutdown_connections 2024-02-19T19:17:11.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.793+0000 7fa99f57a700 1 -- 172.21.15.88:0/1678744831 wait complete. 2024-02-19T19:17:11.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.897+0000 7fad6f84a700 1 -- 172.21.15.88:0/2607468551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680ff350 msgr2=0x7fad680ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.897+0000 7fad6f84a700 1 --2- 172.21.15.88:0/2607468551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680ff350 0x7fad680ff720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fad64009910 tx=0x7fad64009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:11.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.897+0000 7fad6f84a700 1 -- 172.21.15.88:0/2607468551 shutdown_connections 2024-02-19T19:17:11.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.897+0000 7fad6f84a700 1 --2- 172.21.15.88:0/2607468551 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad680ffc60 0x7fad6810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.897+0000 7fad6f84a700 1 --2- 172.21.15.88:0/2607468551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680ff350 0x7fad680ff720 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.897+0000 7fad6f84a700 1 -- 172.21.15.88:0/2607468551 >> 172.21.15.88:0/2607468551 conn(0x7fad680748b0 msgr2=0x7fad68074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.897+0000 7fad6f84a700 1 -- 172.21.15.88:0/2607468551 shutdown_connections 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.897+0000 7fad6f84a700 1 -- 172.21.15.88:0/2607468551 wait complete. 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.897+0000 7fad6f84a700 1 Processor -- start 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.897+0000 7fad6f84a700 1 -- start start 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6f84a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad680ff350 0x7fad68195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6f84a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680ffc60 0x7fad681964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6f84a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad68196bd0 con 0x7fad680ffc60 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6f84a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6819a960 con 0x7fad680ff350 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6cde5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680ffc60 0x7fad681964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6cde5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680ffc60 0x7fad681964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56148/0 (socket says 172.21.15.88:56148) 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6cde5700 1 -- 172.21.15.88:0/2320486624 learned_addr learned my addr 172.21.15.88:0/2320486624 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:11.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6d5e6700 1 --2- 172.21.15.88:0/2320486624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad680ff350 0x7fad68195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:11.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6cde5700 1 -- 172.21.15.88:0/2320486624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad680ff350 msgr2=0x7fad68195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:11.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6cde5700 1 --2- 172.21.15.88:0/2320486624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad680ff350 0x7fad68195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:11.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6cde5700 1 -- 172.21.15.88:0/2320486624 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad640095f0 con 0x7fad680ffc60 2024-02-19T19:17:11.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6cde5700 1 --2- 172.21.15.88:0/2320486624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680ffc60 0x7fad681964f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fad5800b4b0 tx=0x7fad5800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:11.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad5e7fc700 1 -- 172.21.15.88:0/2320486624 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad58011800 con 0x7fad680ffc60 2024-02-19T19:17:11.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad6f84a700 1 -- 172.21.15.88:0/2320486624 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6819ac40 con 0x7fad680ffc60 2024-02-19T19:17:11.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad5e7fc700 1 -- 172.21.15.88:0/2320486624 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad58011e20 con 0x7fad680ffc60 2024-02-19T19:17:11.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.898+0000 7fad5e7fc700 1 -- 172.21.15.88:0/2320486624 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad5800f2e0 con 0x7fad680ffc60 2024-02-19T19:17:11.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.899+0000 7fad6f84a700 1 -- 172.21.15.88:0/2320486624 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6819b190 con 0x7fad680ffc60 2024-02-19T19:17:11.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.900+0000 7fad6f84a700 1 -- 172.21.15.88:0/2320486624 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad68107dd0 con 0x7fad680ffc60 2024-02-19T19:17:11.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.900+0000 7fad5e7fc700 1 -- 172.21.15.88:0/2320486624 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fad58010380 con 0x7fad680ffc60 2024-02-19T19:17:11.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.900+0000 7fad5e7fc700 1 --2- 172.21.15.88:0/2320486624 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad5406d8a0 0x7fad5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:11.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.901+0000 7fad5e7fc700 1 -- 172.21.15.88:0/2320486624 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fad5808d040 con 0x7fad680ffc60 2024-02-19T19:17:11.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.901+0000 7fad6d5e6700 1 --2- 172.21.15.88:0/2320486624 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad5406d8a0 0x7fad5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:11.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.901+0000 7fad6d5e6700 1 --2- 172.21.15.88:0/2320486624 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad5406d8a0 0x7fad5406fd50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad6400b5c0 tx=0x7fad6401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:11.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:11.905+0000 7fad5e7fc700 1 -- 172.21.15.88:0/2320486624 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fad58056870 con 0x7fad680ffc60 2024-02-19T19:17:12.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.171+0000 7fad6f84a700 1 -- 172.21.15.88:0/2320486624 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fad6804f2a0 con 0x7fad680ffc60 2024-02-19T19:17:12.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.171+0000 7fad5e7fc700 1 -- 172.21.15.88:0/2320486624 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fad58059f30 con 0x7fad680ffc60 2024-02-19T19:17:12.188 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:17:12.188 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:17:12.188 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:17:12.188 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:17:12.188 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:17:12.189 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:17:12.190 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:17:12.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:17:12.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:17:12.191 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:17:12.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:17:12.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:17:12.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:17:12.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:17:12.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:17:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.173+0000 7fad6f84a700 1 -- 172.21.15.88:0/2320486624 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad5406d8a0 msgr2=0x7fad5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.173+0000 7fad6f84a700 1 --2- 172.21.15.88:0/2320486624 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad5406d8a0 0x7fad5406fd50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad6400b5c0 tx=0x7fad6401a040 comp rx=0 tx=0).stop 2024-02-19T19:17:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.173+0000 7fad6f84a700 1 -- 172.21.15.88:0/2320486624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680ffc60 msgr2=0x7fad681964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.173+0000 7fad6f84a700 1 --2- 172.21.15.88:0/2320486624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680ffc60 0x7fad681964f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fad5800b4b0 tx=0x7fad5800b870 comp rx=0 tx=0).stop 2024-02-19T19:17:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.173+0000 7fad6f84a700 1 -- 172.21.15.88:0/2320486624 shutdown_connections 2024-02-19T19:17:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.173+0000 7fad6f84a700 1 --2- 172.21.15.88:0/2320486624 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fad5406d8a0 0x7fad5406fd50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.173+0000 7fad6f84a700 1 --2- 172.21.15.88:0/2320486624 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fad680ff350 0x7fad68195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.173+0000 7fad6f84a700 1 --2- 172.21.15.88:0/2320486624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad680ffc60 0x7fad681964f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.173+0000 7fad6f84a700 1 -- 172.21.15.88:0/2320486624 >> 172.21.15.88:0/2320486624 conn(0x7fad680748b0 msgr2=0x7fad68104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.173+0000 7fad6f84a700 1 -- 172.21.15.88:0/2320486624 shutdown_connections 2024-02-19T19:17:12.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.173+0000 7fad6f84a700 1 -- 172.21.15.88:0/2320486624 wait complete. 2024-02-19T19:17:12.193 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:17:12.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.277+0000 7fb923a07700 1 -- 172.21.15.88:0/2463924635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb91c1014a0 msgr2=0x7fb91c101870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:12.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.277+0000 7fb923a07700 1 --2- 172.21.15.88:0/2463924635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb91c1014a0 0x7fb91c101870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009910 tx=0x7fb90c009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.277+0000 7fb923a07700 1 -- 172.21.15.88:0/2463924635 shutdown_connections 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.277+0000 7fb923a07700 1 --2- 172.21.15.88:0/2463924635 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb91c101db0 0x7fb91c10a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.277+0000 7fb923a07700 1 --2- 172.21.15.88:0/2463924635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb91c1014a0 0x7fb91c101870 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.277+0000 7fb923a07700 1 -- 172.21.15.88:0/2463924635 >> 172.21.15.88:0/2463924635 conn(0x7fb91c0fac80 msgr2=0x7fb91c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.277+0000 7fb923a07700 1 -- 172.21.15.88:0/2463924635 shutdown_connections 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.277+0000 7fb923a07700 1 -- 172.21.15.88:0/2463924635 wait complete. 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb923a07700 1 Processor -- start 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb923a07700 1 -- start start 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb923a07700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb91c1014a0 0x7fb91c0ff100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb923a07700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb91c101db0 0x7fb91c0ff640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb923a07700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0ffb80 con 0x7fb91c101db0 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb923a07700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0ffcc0 con 0x7fb91c1014a0 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb920fa2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb91c101db0 0x7fb91c0ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:12.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb920fa2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb91c101db0 0x7fb91c0ff640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56166/0 (socket says 172.21.15.88:56166) 2024-02-19T19:17:12.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb920fa2700 1 -- 172.21.15.88:0/295067314 learned_addr learned my addr 172.21.15.88:0/295067314 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:12.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb920fa2700 1 -- 172.21.15.88:0/295067314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb91c1014a0 msgr2=0x7fb91c0ff100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:12.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb920fa2700 1 --2- 172.21.15.88:0/295067314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb91c1014a0 0x7fb91c0ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb920fa2700 1 -- 172.21.15.88:0/295067314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c0095f0 con 0x7fb91c101db0 2024-02-19T19:17:12.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb920fa2700 1 --2- 172.21.15.88:0/295067314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb91c101db0 0x7fb91c0ff640 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb91800c8a0 tx=0x7fb91800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:12.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.278+0000 7fb9127fc700 1 -- 172.21.15.88:0/295067314 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb91800cea0 con 0x7fb91c101db0 2024-02-19T19:17:12.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.279+0000 7fb923a07700 1 -- 172.21.15.88:0/295067314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91c1a2020 con 0x7fb91c101db0 2024-02-19T19:17:12.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.279+0000 7fb9127fc700 1 -- 172.21.15.88:0/295067314 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb918004830 con 0x7fb91c101db0 2024-02-19T19:17:12.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.279+0000 7fb9127fc700 1 -- 172.21.15.88:0/295067314 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9180055b0 con 0x7fb91c101db0 2024-02-19T19:17:12.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.279+0000 7fb923a07700 1 -- 172.21.15.88:0/295067314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91c1a2430 con 0x7fb91c101db0 2024-02-19T19:17:12.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.280+0000 7fb923a07700 1 -- 172.21.15.88:0/295067314 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91c107980 con 0x7fb91c101db0 2024-02-19T19:17:12.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.280+0000 7fb9127fc700 1 -- 172.21.15.88:0/295067314 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb9180049a0 con 0x7fb91c101db0 2024-02-19T19:17:12.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.280+0000 7fb9127fc700 1 --2- 172.21.15.88:0/295067314 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90806d8a0 0x7fb90806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:12.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.280+0000 7fb9127fc700 1 -- 172.21.15.88:0/295067314 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb91808d1d0 con 0x7fb91c101db0 2024-02-19T19:17:12.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.280+0000 7fb9217a3700 1 --2- 172.21.15.88:0/295067314 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90806d8a0 0x7fb90806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:12.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.281+0000 7fb9217a3700 1 --2- 172.21.15.88:0/295067314 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90806d8a0 0x7fb90806fd50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00b5c0 tx=0x7fb90c005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:12.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.285+0000 7fb9127fc700 1 -- 172.21.15.88:0/295067314 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb918056980 con 0x7fb91c101db0 2024-02-19T19:17:12.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.503+0000 7fb923a07700 1 -- 172.21.15.88:0/295067314 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb91c061190 con 0x7fb90806d8a0 2024-02-19T19:17:12.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.504+0000 7fb9127fc700 1 -- 172.21.15.88:0/295067314 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb91c061190 con 0x7fb90806d8a0 2024-02-19T19:17:12.521 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:17:12.521 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:17:12.521 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:17:12.521 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:17:12.522 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:17:12.522 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:17:12.522 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:17:12.522 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:17:12.522 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:17:12.522 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:17:12.522 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:17:12.522 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:17:12.522 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:17:12.522 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:17:12.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.506+0000 7fb923a07700 1 -- 172.21.15.88:0/295067314 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90806d8a0 msgr2=0x7fb90806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:12.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.506+0000 7fb923a07700 1 --2- 172.21.15.88:0/295067314 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90806d8a0 0x7fb90806fd50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb90c00b5c0 tx=0x7fb90c005960 comp rx=0 tx=0).stop 2024-02-19T19:17:12.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.506+0000 7fb923a07700 1 -- 172.21.15.88:0/295067314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb91c101db0 msgr2=0x7fb91c0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:12.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.506+0000 7fb923a07700 1 --2- 172.21.15.88:0/295067314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb91c101db0 0x7fb91c0ff640 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb91800c8a0 tx=0x7fb91800cc60 comp rx=0 tx=0).stop 2024-02-19T19:17:12.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.507+0000 7fb923a07700 1 -- 172.21.15.88:0/295067314 shutdown_connections 2024-02-19T19:17:12.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.507+0000 7fb923a07700 1 --2- 172.21.15.88:0/295067314 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90806d8a0 0x7fb90806fd50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.507+0000 7fb923a07700 1 --2- 172.21.15.88:0/295067314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb91c1014a0 0x7fb91c0ff100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.507+0000 7fb923a07700 1 --2- 172.21.15.88:0/295067314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb91c101db0 0x7fb91c0ff640 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.507+0000 7fb923a07700 1 -- 172.21.15.88:0/295067314 >> 172.21.15.88:0/295067314 conn(0x7fb91c0fac80 msgr2=0x7fb91c104ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:12.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.507+0000 7fb923a07700 1 -- 172.21.15.88:0/295067314 shutdown_connections 2024-02-19T19:17:12.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.507+0000 7fb923a07700 1 -- 172.21.15.88:0/295067314 wait complete. 2024-02-19T19:17:12.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.610+0000 7fdb41859700 1 -- 172.21.15.88:0/2057280803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb3c1004e0 msgr2=0x7fdb3c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:12.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.610+0000 7fdb41859700 1 --2- 172.21.15.88:0/2057280803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb3c1004e0 0x7fdb3c1008b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009910 tx=0x7fdb2c009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:12.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.610+0000 7fdb41859700 1 -- 172.21.15.88:0/2057280803 shutdown_connections 2024-02-19T19:17:12.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.610+0000 7fdb41859700 1 --2- 172.21.15.88:0/2057280803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb3c100e80 0x7fdb3c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.610+0000 7fdb41859700 1 --2- 172.21.15.88:0/2057280803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb3c1004e0 0x7fdb3c1008b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.610+0000 7fdb41859700 1 -- 172.21.15.88:0/2057280803 >> 172.21.15.88:0/2057280803 conn(0x7fdb3c0fbda0 msgr2=0x7fdb3c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:12.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.610+0000 7fdb41859700 1 -- 172.21.15.88:0/2057280803 shutdown_connections 2024-02-19T19:17:12.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.610+0000 7fdb41859700 1 -- 172.21.15.88:0/2057280803 wait complete. 2024-02-19T19:17:12.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.610+0000 7fdb41859700 1 Processor -- start 2024-02-19T19:17:12.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.610+0000 7fdb41859700 1 -- start start 2024-02-19T19:17:12.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb41859700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb3c1004e0 0x7fdb3c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb41859700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb3c100e80 0x7fdb3c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb41859700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c196bb0 con 0x7fdb3c100e80 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb41859700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c19a940 con 0x7fdb3c1004e0 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb327fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb3c100e80 0x7fdb3c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb327fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb3c100e80 0x7fdb3c1964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56184/0 (socket says 172.21.15.88:56184) 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb327fc700 1 -- 172.21.15.88:0/3045125739 learned_addr learned my addr 172.21.15.88:0/3045125739 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb3affd700 1 --2- 172.21.15.88:0/3045125739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb3c1004e0 0x7fdb3c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb327fc700 1 -- 172.21.15.88:0/3045125739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb3c1004e0 msgr2=0x7fdb3c195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb327fc700 1 --2- 172.21.15.88:0/3045125739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb3c1004e0 0x7fdb3c195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb327fc700 1 -- 172.21.15.88:0/3045125739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c0095f0 con 0x7fdb3c100e80 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb327fc700 1 --2- 172.21.15.88:0/3045125739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb3c100e80 0x7fdb3c1964d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400b480 tx=0x7fdb2400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb40857700 1 -- 172.21.15.88:0/3045125739 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb240118c0 con 0x7fdb3c100e80 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb40857700 1 -- 172.21.15.88:0/3045125739 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb24010380 con 0x7fdb3c100e80 2024-02-19T19:17:12.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb40857700 1 -- 172.21.15.88:0/3045125739 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb2400f4f0 con 0x7fdb3c100e80 2024-02-19T19:17:12.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.611+0000 7fdb41859700 1 -- 172.21.15.88:0/3045125739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c19ac20 con 0x7fdb3c100e80 2024-02-19T19:17:12.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.612+0000 7fdb41859700 1 -- 172.21.15.88:0/3045125739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c19b170 con 0x7fdb3c100e80 2024-02-19T19:17:12.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.612+0000 7fdb41859700 1 -- 172.21.15.88:0/3045125739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3c1087f0 con 0x7fdb3c100e80 2024-02-19T19:17:12.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.613+0000 7fdb40857700 1 -- 172.21.15.88:0/3045125739 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdb240104f0 con 0x7fdb3c100e80 2024-02-19T19:17:12.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.613+0000 7fdb40857700 1 --2- 172.21.15.88:0/3045125739 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb2806d8a0 0x7fdb2806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:12.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.614+0000 7fdb40857700 1 -- 172.21.15.88:0/3045125739 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdb2408d0a0 con 0x7fdb3c100e80 2024-02-19T19:17:12.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.614+0000 7fdb3affd700 1 --2- 172.21.15.88:0/3045125739 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb2806d8a0 0x7fdb2806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:12.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.614+0000 7fdb3affd700 1 --2- 172.21.15.88:0/3045125739 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb2806d8a0 0x7fdb2806fd50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c006010 tx=0x7fdb2c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:12.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.618+0000 7fdb40857700 1 -- 172.21.15.88:0/3045125739 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb240568d0 con 0x7fdb3c100e80 2024-02-19T19:17:12.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:12 smithi132 ceph-mon[70676]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:12.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:12 smithi132 ceph-mon[70676]: from='client.34782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:12.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:12 smithi132 ceph-mon[70676]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:12.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:12 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1678744831' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:17:12.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:12 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2320486624' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:17:12.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:12 smithi088 ceph-mon[88643]: from='client.34778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:12.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:12 smithi088 ceph-mon[88643]: from='client.34782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:12.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:12 smithi088 ceph-mon[88643]: from='client.34786 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:12.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:12 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1678744831' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:17:12.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:12 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2320486624' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:17:12.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.896+0000 7fdb41859700 1 -- 172.21.15.88:0/3045125739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdb3c19b440 con 0x7fdb3c100e80 2024-02-19T19:17:12.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.896+0000 7fdb40857700 1 -- 172.21.15.88:0/3045125739 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fdb24059f90 con 0x7fdb3c100e80 2024-02-19T19:17:12.913 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:17:12.913 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:17:12.913 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:17:12.913 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:17:12.913 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:17:12.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.898+0000 7fdb41859700 1 -- 172.21.15.88:0/3045125739 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb2806d8a0 msgr2=0x7fdb2806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:12.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.898+0000 7fdb41859700 1 --2- 172.21.15.88:0/3045125739 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb2806d8a0 0x7fdb2806fd50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c006010 tx=0x7fdb2c01a040 comp rx=0 tx=0).stop 2024-02-19T19:17:12.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.898+0000 7fdb41859700 1 -- 172.21.15.88:0/3045125739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb3c100e80 msgr2=0x7fdb3c1964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:12.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.898+0000 7fdb41859700 1 --2- 172.21.15.88:0/3045125739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb3c100e80 0x7fdb3c1964d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fdb2400b480 tx=0x7fdb2400b840 comp rx=0 tx=0).stop 2024-02-19T19:17:12.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.898+0000 7fdb41859700 1 -- 172.21.15.88:0/3045125739 shutdown_connections 2024-02-19T19:17:12.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.898+0000 7fdb41859700 1 --2- 172.21.15.88:0/3045125739 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb2806d8a0 0x7fdb2806fd50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.898+0000 7fdb41859700 1 --2- 172.21.15.88:0/3045125739 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb3c1004e0 0x7fdb3c195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.898+0000 7fdb41859700 1 --2- 172.21.15.88:0/3045125739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb3c100e80 0x7fdb3c1964d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:12.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.898+0000 7fdb41859700 1 -- 172.21.15.88:0/3045125739 >> 172.21.15.88:0/3045125739 conn(0x7fdb3c0fbda0 msgr2=0x7fdb3c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:12.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.898+0000 7fdb41859700 1 -- 172.21.15.88:0/3045125739 shutdown_connections 2024-02-19T19:17:12.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:12.898+0000 7fdb41859700 1 -- 172.21.15.88:0/3045125739 wait complete. 2024-02-19T19:17:13.730 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:13 smithi132 ceph-mon[70676]: pgmap v408: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:13.730 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:13 smithi132 ceph-mon[70676]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:13.730 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:13 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3045125739' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:17:13.730 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:17:13.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:13 smithi088 ceph-mon[88643]: pgmap v408: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:13.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:13 smithi088 ceph-mon[88643]: from='client.34798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:13.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:13 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3045125739' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:17:13.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:17:15.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:15 smithi132 ceph-mon[70676]: pgmap v409: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:15.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:15 smithi088 ceph-mon[88643]: pgmap v409: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:17.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:17 smithi132 ceph-mon[70676]: pgmap v410: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:17.847 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:17 smithi088 ceph-mon[88643]: pgmap v410: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:19.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:19 smithi132 ceph-mon[70676]: pgmap v411: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:19.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:19 smithi088 ceph-mon[88643]: pgmap v411: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:21.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:21 smithi132 ceph-mon[70676]: pgmap v412: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:21.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:21 smithi088 ceph-mon[88643]: pgmap v412: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:23.748 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:23 smithi132 ceph-mon[70676]: pgmap v413: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:23.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:23 smithi088 ceph-mon[88643]: pgmap v413: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:25.749 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:25 smithi132 ceph-mon[70676]: pgmap v414: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:25.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:25 smithi088 ceph-mon[88643]: pgmap v414: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:27.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:27 smithi088 ceph-mon[88643]: pgmap v415: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:27.748 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:27 smithi132 ceph-mon[70676]: pgmap v415: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:28.748 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:17:28.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:17:29.748 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:29 smithi132 ceph-mon[70676]: pgmap v416: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:29.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:29 smithi088 ceph-mon[88643]: pgmap v416: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:31.748 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:31 smithi132 ceph-mon[70676]: pgmap v417: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:31.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:31 smithi088 ceph-mon[88643]: pgmap v417: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:33.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:33 smithi088 ceph-mon[88643]: pgmap v418: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:33.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:33 smithi132 ceph-mon[70676]: pgmap v418: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:35.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:35 smithi088 ceph-mon[88643]: pgmap v419: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:35.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:35 smithi132 ceph-mon[70676]: pgmap v419: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:37.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:37 smithi088 ceph-mon[88643]: pgmap v420: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:37.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:37 smithi132 ceph-mon[70676]: pgmap v420: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:39.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:39 smithi088 ceph-mon[88643]: pgmap v421: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:39.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:39 smithi132 ceph-mon[70676]: pgmap v421: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:41.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:41 smithi088 ceph-mon[88643]: pgmap v422: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:41.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:41 smithi132 ceph-mon[70676]: pgmap v422: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:43.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.015+0000 7f64311d3700 1 -- 172.21.15.88:0/2554142860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f642c0ffbd0 msgr2=0x7f642c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.015+0000 7f64311d3700 1 --2- 172.21.15.88:0/2554142860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f642c0ffbd0 0x7f642c100040 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6414009910 tx=0x7f6414009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:43.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.015+0000 7f64311d3700 1 -- 172.21.15.88:0/2554142860 shutdown_connections 2024-02-19T19:17:43.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.015+0000 7f64311d3700 1 --2- 172.21.15.88:0/2554142860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f642c0ffbd0 0x7f642c100040 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.015+0000 7f64311d3700 1 --2- 172.21.15.88:0/2554142860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f642c104130 0x7f642c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.015+0000 7f64311d3700 1 -- 172.21.15.88:0/2554142860 >> 172.21.15.88:0/2554142860 conn(0x7f642c075130 msgr2=0x7f642c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:43.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.015+0000 7f64311d3700 1 -- 172.21.15.88:0/2554142860 shutdown_connections 2024-02-19T19:17:43.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.015+0000 7f64311d3700 1 -- 172.21.15.88:0/2554142860 wait complete. 2024-02-19T19:17:43.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f64311d3700 1 Processor -- start 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f64311d3700 1 -- start start 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f64311d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f642c0ffbd0 0x7f642c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f64311d3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f642c104130 0x7f642c197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f64311d3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c1977d0 con 0x7f642c104130 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f64311d3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f642c197910 con 0x7f642c0ffbd0 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f642259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f642c104130 0x7f642c197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f642259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f642c104130 0x7f642c197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39328/0 (socket says 172.21.15.88:39328) 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f642259c700 1 -- 172.21.15.88:0/705155926 learned_addr learned my addr 172.21.15.88:0/705155926 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f642ad9d700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f642c0ffbd0 0x7f642c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f642259c700 1 -- 172.21.15.88:0/705155926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f642c0ffbd0 msgr2=0x7f642c196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f642259c700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f642c0ffbd0 0x7f642c196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.016+0000 7f642259c700 1 -- 172.21.15.88:0/705155926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64140095f0 con 0x7f642c104130 2024-02-19T19:17:43.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.017+0000 7f642ad9d700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f642c0ffbd0 0x7f642c196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:17:43.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.017+0000 7f642259c700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f642c104130 0x7f642c197180 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6414006010 tx=0x7f641400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:43.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.017+0000 7f6428d99700 1 -- 172.21.15.88:0/705155926 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f641401d070 con 0x7f642c104130 2024-02-19T19:17:43.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.017+0000 7f6428d99700 1 -- 172.21.15.88:0/705155926 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6414004fb0 con 0x7f642c104130 2024-02-19T19:17:43.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.017+0000 7f64311d3700 1 -- 172.21.15.88:0/705155926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f642c19fcb0 con 0x7f642c104130 2024-02-19T19:17:43.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.017+0000 7f6428d99700 1 -- 172.21.15.88:0/705155926 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f641400f690 con 0x7f642c104130 2024-02-19T19:17:43.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.017+0000 7f64311d3700 1 -- 172.21.15.88:0/705155926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f642c1a00c0 con 0x7f642c104130 2024-02-19T19:17:43.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.018+0000 7f64311d3700 1 -- 172.21.15.88:0/705155926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f642c04f2a0 con 0x7f642c104130 2024-02-19T19:17:43.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.018+0000 7f6428d99700 1 -- 172.21.15.88:0/705155926 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6414022470 con 0x7f642c104130 2024-02-19T19:17:43.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.019+0000 7f6428d99700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f641806d8a0 0x7f641806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:43.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.019+0000 7f6428d99700 1 -- 172.21.15.88:0/705155926 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f641408edd0 con 0x7f642c104130 2024-02-19T19:17:43.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.019+0000 7f642ad9d700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f641806d8a0 0x7f641806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:43.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.019+0000 7f642ad9d700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f641806d8a0 0x7f641806fd50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f641c005fd0 tx=0x7f641c005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:43.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.024+0000 7f6428d99700 1 -- 172.21.15.88:0/705155926 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64140586a0 con 0x7f642c104130 2024-02-19T19:17:43.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.239+0000 7f64311d3700 1 -- 172.21.15.88:0/705155926 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f642c0ff240 con 0x7f641806d8a0 2024-02-19T19:17:43.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.240+0000 7f6428d99700 1 -- 172.21.15.88:0/705155926 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f642c0ff240 con 0x7f641806d8a0 2024-02-19T19:17:43.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.242+0000 7f64311d3700 1 -- 172.21.15.88:0/705155926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f641806d8a0 msgr2=0x7f641806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.242+0000 7f64311d3700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f641806d8a0 0x7f641806fd50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f641c005fd0 tx=0x7f641c005e20 comp rx=0 tx=0).stop 2024-02-19T19:17:43.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.242+0000 7f64311d3700 1 -- 172.21.15.88:0/705155926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f642c104130 msgr2=0x7f642c197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.242+0000 7f64311d3700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f642c104130 0x7f642c197180 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6414006010 tx=0x7f641400be20 comp rx=0 tx=0).stop 2024-02-19T19:17:43.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.242+0000 7f64311d3700 1 -- 172.21.15.88:0/705155926 shutdown_connections 2024-02-19T19:17:43.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.242+0000 7f64311d3700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f641806d8a0 0x7f641806fd50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.242+0000 7f64311d3700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f642c0ffbd0 0x7f642c196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.242+0000 7f64311d3700 1 --2- 172.21.15.88:0/705155926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f642c104130 0x7f642c197180 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.242+0000 7f64311d3700 1 -- 172.21.15.88:0/705155926 >> 172.21.15.88:0/705155926 conn(0x7f642c075130 msgr2=0x7f642c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:43.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.242+0000 7f64311d3700 1 -- 172.21.15.88:0/705155926 shutdown_connections 2024-02-19T19:17:43.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.242+0000 7f64311d3700 1 -- 172.21.15.88:0/705155926 wait complete. 2024-02-19T19:17:43.270 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:17:43.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.347+0000 7f28615de700 1 -- 172.21.15.88:0/2961660469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0ffd60 msgr2=0x7f285c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.347+0000 7f28615de700 1 --2- 172.21.15.88:0/2961660469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0ffd60 0x7f285c103d30 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2850009910 tx=0x7f2850009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:43.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.347+0000 7f28615de700 1 -- 172.21.15.88:0/2961660469 shutdown_connections 2024-02-19T19:17:43.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.347+0000 7f28615de700 1 --2- 172.21.15.88:0/2961660469 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0ffd60 0x7f285c103d30 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.347+0000 7f28615de700 1 --2- 172.21.15.88:0/2961660469 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0ff3c0 0x7f285c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.347+0000 7f28615de700 1 -- 172.21.15.88:0/2961660469 >> 172.21.15.88:0/2961660469 conn(0x7f285c0fac60 msgr2=0x7f285c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:43.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.347+0000 7f28615de700 1 -- 172.21.15.88:0/2961660469 shutdown_connections 2024-02-19T19:17:43.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.347+0000 7f28615de700 1 -- 172.21.15.88:0/2961660469 wait complete. 2024-02-19T19:17:43.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.347+0000 7f28615de700 1 Processor -- start 2024-02-19T19:17:43.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f28615de700 1 -- start start 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f28615de700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0ff3c0 0x7f285c195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f28615de700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0ffd60 0x7f285c1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f28615de700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c196ab0 con 0x7f285c0ffd60 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f28615de700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c19a840 con 0x7f285c0ff3c0 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f285a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0ffd60 0x7f285c1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f285a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0ffd60 0x7f285c1963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39342/0 (socket says 172.21.15.88:39342) 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f285a7fc700 1 -- 172.21.15.88:0/1763842271 learned_addr learned my addr 172.21.15.88:0/1763842271 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f285affd700 1 --2- 172.21.15.88:0/1763842271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0ff3c0 0x7f285c195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f285a7fc700 1 -- 172.21.15.88:0/1763842271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0ff3c0 msgr2=0x7f285c195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f285a7fc700 1 --2- 172.21.15.88:0/1763842271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0ff3c0 0x7f285c195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f285a7fc700 1 -- 172.21.15.88:0/1763842271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28500095f0 con 0x7f285c0ffd60 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f285a7fc700 1 --2- 172.21.15.88:0/1763842271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0ffd60 0x7f285c1963d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f28500092e0 tx=0x7f285000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:43.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.348+0000 7f2843fff700 1 -- 172.21.15.88:0/1763842271 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f285001d070 con 0x7f285c0ffd60 2024-02-19T19:17:43.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.349+0000 7f28615de700 1 -- 172.21.15.88:0/1763842271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285c19aac0 con 0x7f285c0ffd60 2024-02-19T19:17:43.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.349+0000 7f2843fff700 1 -- 172.21.15.88:0/1763842271 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2850005dc0 con 0x7f285c0ffd60 2024-02-19T19:17:43.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.349+0000 7f2843fff700 1 -- 172.21.15.88:0/1763842271 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2850017400 con 0x7f285c0ffd60 2024-02-19T19:17:43.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.349+0000 7f28615de700 1 -- 172.21.15.88:0/1763842271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285c19afb0 con 0x7f285c0ffd60 2024-02-19T19:17:43.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.350+0000 7f28615de700 1 -- 172.21.15.88:0/1763842271 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285c108740 con 0x7f285c0ffd60 2024-02-19T19:17:43.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.350+0000 7f2843fff700 1 -- 172.21.15.88:0/1763842271 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f285000f6f0 con 0x7f285c0ffd60 2024-02-19T19:17:43.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.350+0000 7f2843fff700 1 --2- 172.21.15.88:0/1763842271 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f284406d8a0 0x7f284406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:43.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.351+0000 7f2843fff700 1 -- 172.21.15.88:0/1763842271 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f285008e910 con 0x7f285c0ffd60 2024-02-19T19:17:43.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.351+0000 7f285affd700 1 --2- 172.21.15.88:0/1763842271 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f284406d8a0 0x7f284406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:43.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.351+0000 7f285affd700 1 --2- 172.21.15.88:0/1763842271 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f284406d8a0 0x7f284406fd50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2854009550 tx=0x7f28540091b0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:43.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.355+0000 7f2843fff700 1 -- 172.21.15.88:0/1763842271 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f28500580c0 con 0x7f285c0ffd60 2024-02-19T19:17:43.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.571+0000 7f28615de700 1 -- 172.21.15.88:0/1763842271 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f285c19b1d0 con 0x7f284406d8a0 2024-02-19T19:17:43.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.573+0000 7f2843fff700 1 -- 172.21.15.88:0/1763842271 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f285c19b1d0 con 0x7f284406d8a0 2024-02-19T19:17:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.576+0000 7f28615de700 1 -- 172.21.15.88:0/1763842271 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f284406d8a0 msgr2=0x7f284406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.576+0000 7f28615de700 1 --2- 172.21.15.88:0/1763842271 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f284406d8a0 0x7f284406fd50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2854009550 tx=0x7f28540091b0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.576+0000 7f28615de700 1 -- 172.21.15.88:0/1763842271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0ffd60 msgr2=0x7f285c1963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.576+0000 7f28615de700 1 --2- 172.21.15.88:0/1763842271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0ffd60 0x7f285c1963d0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f28500092e0 tx=0x7f285000bd70 comp rx=0 tx=0).stop 2024-02-19T19:17:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.576+0000 7f28615de700 1 -- 172.21.15.88:0/1763842271 shutdown_connections 2024-02-19T19:17:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.576+0000 7f28615de700 1 --2- 172.21.15.88:0/1763842271 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f284406d8a0 0x7f284406fd50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.576+0000 7f28615de700 1 --2- 172.21.15.88:0/1763842271 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f285c0ff3c0 0x7f285c195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.576+0000 7f28615de700 1 --2- 172.21.15.88:0/1763842271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285c0ffd60 0x7f285c1963d0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.576+0000 7f28615de700 1 -- 172.21.15.88:0/1763842271 >> 172.21.15.88:0/1763842271 conn(0x7f285c0fac60 msgr2=0x7f285c069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.576+0000 7f28615de700 1 -- 172.21.15.88:0/1763842271 shutdown_connections 2024-02-19T19:17:43.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.576+0000 7f28615de700 1 -- 172.21.15.88:0/1763842271 wait complete. 2024-02-19T19:17:43.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.679+0000 7f432f89f700 1 -- 172.21.15.88:0/835742184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43280ffbd0 msgr2=0x7f4328100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.679+0000 7f432f89f700 1 --2- 172.21.15.88:0/835742184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43280ffbd0 0x7f4328100040 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f4318009910 tx=0x7f4318009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:43.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.679+0000 7f432f89f700 1 -- 172.21.15.88:0/835742184 shutdown_connections 2024-02-19T19:17:43.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.679+0000 7f432f89f700 1 --2- 172.21.15.88:0/835742184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43280ffbd0 0x7f4328100040 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.679+0000 7f432f89f700 1 --2- 172.21.15.88:0/835742184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4328104130 0x7f4328104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.679+0000 7f432f89f700 1 -- 172.21.15.88:0/835742184 >> 172.21.15.88:0/835742184 conn(0x7f4328075130 msgr2=0x7f4328075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:43.696 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:43 smithi088 ceph-mon[88643]: pgmap v423: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:43.696 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:17:43.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.679+0000 7f432f89f700 1 -- 172.21.15.88:0/835742184 shutdown_connections 2024-02-19T19:17:43.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.679+0000 7f432f89f700 1 -- 172.21.15.88:0/835742184 wait complete. 2024-02-19T19:17:43.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432f89f700 1 Processor -- start 2024-02-19T19:17:43.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432f89f700 1 -- start start 2024-02-19T19:17:43.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432f89f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43280ffbd0 0x7f4328100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:43.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432f89f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4328104130 0x7f4328101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:43.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432f89f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43281018b0 con 0x7f4328104130 2024-02-19T19:17:43.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432f89f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43281019f0 con 0x7f43280ffbd0 2024-02-19T19:17:43.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432ce3a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4328104130 0x7f4328101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:43.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432ce3a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4328104130 0x7f4328101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39358/0 (socket says 172.21.15.88:39358) 2024-02-19T19:17:43.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432ce3a700 1 -- 172.21.15.88:0/1507727008 learned_addr learned my addr 172.21.15.88:0/1507727008 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:43.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432d63b700 1 --2- 172.21.15.88:0/1507727008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43280ffbd0 0x7f4328100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:43.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432ce3a700 1 -- 172.21.15.88:0/1507727008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43280ffbd0 msgr2=0x7f4328100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432ce3a700 1 --2- 172.21.15.88:0/1507727008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43280ffbd0 0x7f4328100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432ce3a700 1 -- 172.21.15.88:0/1507727008 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43180095f0 con 0x7f4328104130 2024-02-19T19:17:43.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432ce3a700 1 --2- 172.21.15.88:0/1507727008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4328104130 0x7f4328101370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f431800bd70 tx=0x7f431800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:43.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f431e7fc700 1 -- 172.21.15.88:0/1507727008 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f431801d070 con 0x7f4328104130 2024-02-19T19:17:43.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f431e7fc700 1 -- 172.21.15.88:0/1507727008 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f431800f460 con 0x7f4328104130 2024-02-19T19:17:43.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432f89f700 1 -- 172.21.15.88:0/1507727008 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4328101c70 con 0x7f4328104130 2024-02-19T19:17:43.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f431e7fc700 1 -- 172.21.15.88:0/1507727008 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4318017630 con 0x7f4328104130 2024-02-19T19:17:43.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.681+0000 7f432f89f700 1 -- 172.21.15.88:0/1507727008 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43281a2370 con 0x7f4328104130 2024-02-19T19:17:43.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.682+0000 7f432f89f700 1 -- 172.21.15.88:0/1507727008 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43281086a0 con 0x7f4328104130 2024-02-19T19:17:43.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.682+0000 7f431e7fc700 1 -- 172.21.15.88:0/1507727008 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4318003a60 con 0x7f4328104130 2024-02-19T19:17:43.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.683+0000 7f431e7fc700 1 --2- 172.21.15.88:0/1507727008 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f431406d8a0 0x7f431406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:43.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.683+0000 7f431e7fc700 1 -- 172.21.15.88:0/1507727008 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f431808ea70 con 0x7f4328104130 2024-02-19T19:17:43.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.683+0000 7f432d63b700 1 --2- 172.21.15.88:0/1507727008 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f431406d8a0 0x7f431406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:43.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.685+0000 7f432d63b700 1 --2- 172.21.15.88:0/1507727008 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f431406d8a0 0x7f431406fd50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4324005950 tx=0x7f43240058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:43.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.689+0000 7f431e7fc700 1 -- 172.21.15.88:0/1507727008 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43180582e0 con 0x7f4328104130 2024-02-19T19:17:43.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.897+0000 7f432f89f700 1 -- 172.21.15.88:0/1507727008 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4328102470 con 0x7f431406d8a0 2024-02-19T19:17:43.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.908+0000 7f431e7fc700 1 -- 172.21.15.88:0/1507727008 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4328102470 con 0x7f431406d8a0 2024-02-19T19:17:43.924 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:17:43.924 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (14m) 6m ago 25m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:17:43.924 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (25m) 6m ago 25m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:17:43.924 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (7m) 6m ago 23m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:17:43.924 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (11m) 6m ago 25m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:17:43.924 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (10m) 6m ago 23m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (14m) 6m ago 24m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (7m) 6m ago 19m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (6m) 6m ago 18m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (6m) 6m ago 19m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (18m) 6m ago 18m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (16m) 6m ago 26m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (15m) 6m ago 23m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (12m) 6m ago 27m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (12m) 6m ago 23m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (15m) 6m ago 25m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (15m) 6m ago 23m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (10m) 6m ago 22m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:17:43.925 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (9m) 6m ago 22m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:17:43.926 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (9m) 6m ago 21m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:17:43.926 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (8m) 6m ago 21m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:17:43.926 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (8m) 6m ago 21m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:17:43.926 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (7m) 6m ago 20m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:17:43.926 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (15m) 6m ago 24m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:17:43.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.911+0000 7f432f89f700 1 -- 172.21.15.88:0/1507727008 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f431406d8a0 msgr2=0x7f431406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.911+0000 7f432f89f700 1 --2- 172.21.15.88:0/1507727008 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f431406d8a0 0x7f431406fd50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4324005950 tx=0x7f43240058e0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.911+0000 7f432f89f700 1 -- 172.21.15.88:0/1507727008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4328104130 msgr2=0x7f4328101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:43.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.911+0000 7f432f89f700 1 --2- 172.21.15.88:0/1507727008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4328104130 0x7f4328101370 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f431800bd70 tx=0x7f431800be50 comp rx=0 tx=0).stop 2024-02-19T19:17:43.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.911+0000 7f432f89f700 1 -- 172.21.15.88:0/1507727008 shutdown_connections 2024-02-19T19:17:43.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.911+0000 7f432f89f700 1 --2- 172.21.15.88:0/1507727008 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f431406d8a0 0x7f431406fd50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.911+0000 7f432f89f700 1 --2- 172.21.15.88:0/1507727008 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43280ffbd0 0x7f4328100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.911+0000 7f432f89f700 1 --2- 172.21.15.88:0/1507727008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4328104130 0x7f4328101370 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:43.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.911+0000 7f432f89f700 1 -- 172.21.15.88:0/1507727008 >> 172.21.15.88:0/1507727008 conn(0x7f4328075130 msgr2=0x7f43280fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:43.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.911+0000 7f432f89f700 1 -- 172.21.15.88:0/1507727008 shutdown_connections 2024-02-19T19:17:43.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:43.911+0000 7f432f89f700 1 -- 172.21.15.88:0/1507727008 wait complete. 2024-02-19T19:17:43.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:43 smithi132 ceph-mon[70676]: pgmap v423: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:43.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:17:44.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.013+0000 7f7283173700 1 -- 172.21.15.88:0/213296223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f727c0ffd60 msgr2=0x7f727c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.013+0000 7f7283173700 1 --2- 172.21.15.88:0/213296223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f727c0ffd60 0x7f727c103d30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f726c009910 tx=0x7f726c009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.013+0000 7f7283173700 1 -- 172.21.15.88:0/213296223 shutdown_connections 2024-02-19T19:17:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.013+0000 7f7283173700 1 --2- 172.21.15.88:0/213296223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f727c0ffd60 0x7f727c103d30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.013+0000 7f7283173700 1 --2- 172.21.15.88:0/213296223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f727c0ff3c0 0x7f727c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.013+0000 7f7283173700 1 -- 172.21.15.88:0/213296223 >> 172.21.15.88:0/213296223 conn(0x7f727c0fac80 msgr2=0x7f727c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.013+0000 7f7283173700 1 -- 172.21.15.88:0/213296223 shutdown_connections 2024-02-19T19:17:44.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.013+0000 7f7283173700 1 -- 172.21.15.88:0/213296223 wait complete. 2024-02-19T19:17:44.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.013+0000 7f7283173700 1 Processor -- start 2024-02-19T19:17:44.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.013+0000 7f7283173700 1 -- start start 2024-02-19T19:17:44.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7283173700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f727c0ff3c0 0x7f727c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:44.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7283173700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f727c0ffd60 0x7f727c1985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:44.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7283173700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c198ca0 con 0x7f727c0ffd60 2024-02-19T19:17:44.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7283173700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c19ca30 con 0x7f727c0ff3c0 2024-02-19T19:17:44.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7273fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f727c0ffd60 0x7f727c1985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:44.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7273fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f727c0ffd60 0x7f727c1985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39384/0 (socket says 172.21.15.88:39384) 2024-02-19T19:17:44.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7273fff700 1 -- 172.21.15.88:0/604638124 learned_addr learned my addr 172.21.15.88:0/604638124 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:44.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7280f0f700 1 --2- 172.21.15.88:0/604638124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f727c0ff3c0 0x7f727c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:44.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7273fff700 1 -- 172.21.15.88:0/604638124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f727c0ff3c0 msgr2=0x7f727c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:44.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7273fff700 1 --2- 172.21.15.88:0/604638124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f727c0ff3c0 0x7f727c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7273fff700 1 -- 172.21.15.88:0/604638124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c0095f0 con 0x7f727c0ffd60 2024-02-19T19:17:44.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7273fff700 1 --2- 172.21.15.88:0/604638124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f727c0ffd60 0x7f727c1985c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f726c0092e0 tx=0x7f726c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:44.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7271ffb700 1 -- 172.21.15.88:0/604638124 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f726c01d070 con 0x7f727c0ffd60 2024-02-19T19:17:44.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7271ffb700 1 -- 172.21.15.88:0/604638124 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f726c00f460 con 0x7f727c0ffd60 2024-02-19T19:17:44.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.014+0000 7f7271ffb700 1 -- 172.21.15.88:0/604638124 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f726c0175d0 con 0x7f727c0ffd60 2024-02-19T19:17:44.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.015+0000 7f7283173700 1 -- 172.21.15.88:0/604638124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727c19ccb0 con 0x7f727c0ffd60 2024-02-19T19:17:44.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.015+0000 7f7283173700 1 -- 172.21.15.88:0/604638124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727c19d150 con 0x7f727c0ffd60 2024-02-19T19:17:44.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.016+0000 7f7283173700 1 -- 172.21.15.88:0/604638124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727c107730 con 0x7f727c0ffd60 2024-02-19T19:17:44.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.017+0000 7f7271ffb700 1 -- 172.21.15.88:0/604638124 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f726c00f5d0 con 0x7f727c0ffd60 2024-02-19T19:17:44.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.017+0000 7f7271ffb700 1 --2- 172.21.15.88:0/604638124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f726406d8a0 0x7f726406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:44.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.017+0000 7f7271ffb700 1 -- 172.21.15.88:0/604638124 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f726c08e9e0 con 0x7f727c0ffd60 2024-02-19T19:17:44.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.017+0000 7f7280f0f700 1 --2- 172.21.15.88:0/604638124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f726406d8a0 0x7f726406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:44.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.018+0000 7f7280f0f700 1 --2- 172.21.15.88:0/604638124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f726406d8a0 0x7f726406fd50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7278005950 tx=0x7f727800a300 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:44.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.022+0000 7f7271ffb700 1 -- 172.21.15.88:0/604638124 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f726c058190 con 0x7f727c0ffd60 2024-02-19T19:17:44.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.299+0000 7f7283173700 1 -- 172.21.15.88:0/604638124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f727c04f2a0 con 0x7f727c0ffd60 2024-02-19T19:17:44.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.300+0000 7f7271ffb700 1 -- 172.21.15.88:0/604638124 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f726c05b850 con 0x7f727c0ffd60 2024-02-19T19:17:44.316 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:17:44.316 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:17:44.316 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:17:44.316 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:17:44.316 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:17:44.316 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:17:44.316 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:17:44.316 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:17:44.316 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:17:44.316 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:17:44.317 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:17:44.317 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:17:44.317 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:17:44.317 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:17:44.317 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:17:44.317 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:17:44.317 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:17:44.317 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:17:44.317 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:17:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.302+0000 7f7283173700 1 -- 172.21.15.88:0/604638124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f726406d8a0 msgr2=0x7f726406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.302+0000 7f7283173700 1 --2- 172.21.15.88:0/604638124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f726406d8a0 0x7f726406fd50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7278005950 tx=0x7f727800a300 comp rx=0 tx=0).stop 2024-02-19T19:17:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.302+0000 7f7283173700 1 -- 172.21.15.88:0/604638124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f727c0ffd60 msgr2=0x7f727c1985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.302+0000 7f7283173700 1 --2- 172.21.15.88:0/604638124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f727c0ffd60 0x7f727c1985c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f726c0092e0 tx=0x7f726c00bd70 comp rx=0 tx=0).stop 2024-02-19T19:17:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.302+0000 7f7283173700 1 -- 172.21.15.88:0/604638124 shutdown_connections 2024-02-19T19:17:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.302+0000 7f7283173700 1 --2- 172.21.15.88:0/604638124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f726406d8a0 0x7f726406fd50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.302+0000 7f7283173700 1 --2- 172.21.15.88:0/604638124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f727c0ff3c0 0x7f727c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.302+0000 7f7283173700 1 --2- 172.21.15.88:0/604638124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f727c0ffd60 0x7f727c1985c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.302+0000 7f7283173700 1 -- 172.21.15.88:0/604638124 >> 172.21.15.88:0/604638124 conn(0x7f727c0fac80 msgr2=0x7f727c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.302+0000 7f7283173700 1 -- 172.21.15.88:0/604638124 shutdown_connections 2024-02-19T19:17:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.302+0000 7f7283173700 1 -- 172.21.15.88:0/604638124 wait complete. 2024-02-19T19:17:44.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.405+0000 7f11c1206700 1 -- 172.21.15.88:0/1845845253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11bc0ff3c0 msgr2=0x7f11bc0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.405+0000 7f11c1206700 1 --2- 172.21.15.88:0/1845845253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11bc0ff3c0 0x7f11bc0ff790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f11ac009910 tx=0x7f11ac009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.405+0000 7f11c1206700 1 -- 172.21.15.88:0/1845845253 shutdown_connections 2024-02-19T19:17:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.405+0000 7f11c1206700 1 --2- 172.21.15.88:0/1845845253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11bc0ffd60 0x7f11bc103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.405+0000 7f11c1206700 1 --2- 172.21.15.88:0/1845845253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11bc0ff3c0 0x7f11bc0ff790 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.405+0000 7f11c1206700 1 -- 172.21.15.88:0/1845845253 >> 172.21.15.88:0/1845845253 conn(0x7f11bc0fac80 msgr2=0x7f11bc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.405+0000 7f11c1206700 1 -- 172.21.15.88:0/1845845253 shutdown_connections 2024-02-19T19:17:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.405+0000 7f11c1206700 1 -- 172.21.15.88:0/1845845253 wait complete. 2024-02-19T19:17:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.406+0000 7f11c1206700 1 Processor -- start 2024-02-19T19:17:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.406+0000 7f11c1206700 1 -- start start 2024-02-19T19:17:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11c1206700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11bc0ff3c0 0x7f11bc196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11c1206700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11bc0ffd60 0x7f11bc197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11c1206700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc197830 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11c1206700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11bc1979a0 con 0x7f11bc0ffd60 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11bad9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11bc0ff3c0 0x7f11bc196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11bad9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11bc0ff3c0 0x7f11bc196cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39396/0 (socket says 172.21.15.88:39396) 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11bad9d700 1 -- 172.21.15.88:0/3825007238 learned_addr learned my addr 172.21.15.88:0/3825007238 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11ba59c700 1 --2- 172.21.15.88:0/3825007238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11bc0ffd60 0x7f11bc197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11bad9d700 1 -- 172.21.15.88:0/3825007238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11bc0ffd60 msgr2=0x7f11bc197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11bad9d700 1 --2- 172.21.15.88:0/3825007238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11bc0ffd60 0x7f11bc197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11bad9d700 1 -- 172.21.15.88:0/3825007238 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac0095f0 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11bad9d700 1 --2- 172.21.15.88:0/3825007238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11bc0ff3c0 0x7f11bc196cd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f11ac006010 tx=0x7f11ac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.407+0000 7f11a3fff700 1 -- 172.21.15.88:0/3825007238 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11ac01d070 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.408+0000 7f11a3fff700 1 -- 172.21.15.88:0/3825007238 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11ac004fb0 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.408+0000 7f11c1206700 1 -- 172.21.15.88:0/3825007238 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11bc19fd40 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.408+0000 7f11a3fff700 1 -- 172.21.15.88:0/3825007238 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11ac022ea0 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.408+0000 7f11c1206700 1 -- 172.21.15.88:0/3825007238 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11bc1a01b0 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.409+0000 7f11c1206700 1 -- 172.21.15.88:0/3825007238 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11bc1076d0 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.409+0000 7f11a3fff700 1 -- 172.21.15.88:0/3825007238 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f11ac022470 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.410+0000 7f11a3fff700 1 --2- 172.21.15.88:0/3825007238 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11a406d8a0 0x7f11a406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:44.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.410+0000 7f11a3fff700 1 -- 172.21.15.88:0/3825007238 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f11ac08e9d0 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.410+0000 7f11ba59c700 1 --2- 172.21.15.88:0/3825007238 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11a406d8a0 0x7f11a406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:44.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.410+0000 7f11ba59c700 1 --2- 172.21.15.88:0/3825007238 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11a406d8a0 0x7f11a406fd50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0696a0 tx=0x7f11b0005f10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:44.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.415+0000 7f11a3fff700 1 -- 172.21.15.88:0/3825007238 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f11ac0583d0 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:44 smithi088 ceph-mon[88643]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:44.596 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:44 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/604638124' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:17:44.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.665+0000 7f11c1206700 1 -- 172.21.15.88:0/3825007238 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f11bc1070b0 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.665+0000 7f11a3fff700 1 -- 172.21.15.88:0/3825007238 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f11ac027070 con 0x7f11bc0ff3c0 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:17:44.682 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:17:44.683 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:17:44.684 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:17:44.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.668+0000 7f11c1206700 1 -- 172.21.15.88:0/3825007238 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11a406d8a0 msgr2=0x7f11a406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:44.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.668+0000 7f11c1206700 1 --2- 172.21.15.88:0/3825007238 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11a406d8a0 0x7f11a406fd50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0696a0 tx=0x7f11b0005f10 comp rx=0 tx=0).stop 2024-02-19T19:17:44.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.668+0000 7f11c1206700 1 -- 172.21.15.88:0/3825007238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11bc0ff3c0 msgr2=0x7f11bc196cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:44.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.668+0000 7f11c1206700 1 --2- 172.21.15.88:0/3825007238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11bc0ff3c0 0x7f11bc196cd0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f11ac006010 tx=0x7f11ac00be20 comp rx=0 tx=0).stop 2024-02-19T19:17:44.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.668+0000 7f11c1206700 1 -- 172.21.15.88:0/3825007238 shutdown_connections 2024-02-19T19:17:44.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.668+0000 7f11c1206700 1 --2- 172.21.15.88:0/3825007238 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11a406d8a0 0x7f11a406fd50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.668+0000 7f11c1206700 1 --2- 172.21.15.88:0/3825007238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11bc0ff3c0 0x7f11bc196cd0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.668+0000 7f11c1206700 1 --2- 172.21.15.88:0/3825007238 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f11bc0ffd60 0x7f11bc197210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.668+0000 7f11c1206700 1 -- 172.21.15.88:0/3825007238 >> 172.21.15.88:0/3825007238 conn(0x7f11bc0fac80 msgr2=0x7f11bc0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:44.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.669+0000 7f11c1206700 1 -- 172.21.15.88:0/3825007238 shutdown_connections 2024-02-19T19:17:44.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.669+0000 7f11c1206700 1 -- 172.21.15.88:0/3825007238 wait complete. 2024-02-19T19:17:44.686 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:17:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.774+0000 7f7691b85700 1 -- 172.21.15.88:0/1621947875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f768c0ffd60 msgr2=0x7f768c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.774+0000 7f7691b85700 1 --2- 172.21.15.88:0/1621947875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f768c0ffd60 0x7f768c103d30 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f767c009910 tx=0x7f767c009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.774+0000 7f7691b85700 1 -- 172.21.15.88:0/1621947875 shutdown_connections 2024-02-19T19:17:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.774+0000 7f7691b85700 1 --2- 172.21.15.88:0/1621947875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f768c0ffd60 0x7f768c103d30 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.774+0000 7f7691b85700 1 --2- 172.21.15.88:0/1621947875 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f768c0ff3c0 0x7f768c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.774+0000 7f7691b85700 1 -- 172.21.15.88:0/1621947875 >> 172.21.15.88:0/1621947875 conn(0x7f768c0fac80 msgr2=0x7f768c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.774+0000 7f7691b85700 1 -- 172.21.15.88:0/1621947875 shutdown_connections 2024-02-19T19:17:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.774+0000 7f7691b85700 1 -- 172.21.15.88:0/1621947875 wait complete. 2024-02-19T19:17:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f7691b85700 1 Processor -- start 2024-02-19T19:17:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f7691b85700 1 -- start start 2024-02-19T19:17:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f7691b85700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f768c0ff3c0 0x7f768c196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f7691b85700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f768c0ffd60 0x7f768c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f7691b85700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c1978c0 con 0x7f768c0ffd60 2024-02-19T19:17:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f7691b85700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f768c197a30 con 0x7f768c0ff3c0 2024-02-19T19:17:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f768affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f768c0ffd60 0x7f768c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f768affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f768c0ffd60 0x7f768c1972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39410/0 (socket says 172.21.15.88:39410) 2024-02-19T19:17:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f768affd700 1 -- 172.21.15.88:0/2398462765 learned_addr learned my addr 172.21.15.88:0/2398462765 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:44.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f768b7fe700 1 --2- 172.21.15.88:0/2398462765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f768c0ff3c0 0x7f768c196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:44.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f768affd700 1 -- 172.21.15.88:0/2398462765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f768c0ff3c0 msgr2=0x7f768c196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:44.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.775+0000 7f768affd700 1 --2- 172.21.15.88:0/2398462765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f768c0ff3c0 0x7f768c196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:44.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.776+0000 7f768affd700 1 -- 172.21.15.88:0/2398462765 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767c0095f0 con 0x7f768c0ffd60 2024-02-19T19:17:44.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.776+0000 7f768affd700 1 --2- 172.21.15.88:0/2398462765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f768c0ffd60 0x7f768c1972a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f767c0092e0 tx=0x7f767c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:44.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.776+0000 7f7688ff9700 1 -- 172.21.15.88:0/2398462765 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f767c01d070 con 0x7f768c0ffd60 2024-02-19T19:17:44.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.776+0000 7f7691b85700 1 -- 172.21.15.88:0/2398462765 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f768c19fdd0 con 0x7f768c0ffd60 2024-02-19T19:17:44.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.776+0000 7f7688ff9700 1 -- 172.21.15.88:0/2398462765 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f767c00f460 con 0x7f768c0ffd60 2024-02-19T19:17:44.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.776+0000 7f7688ff9700 1 -- 172.21.15.88:0/2398462765 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f767c0175f0 con 0x7f768c0ffd60 2024-02-19T19:17:44.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.776+0000 7f7691b85700 1 -- 172.21.15.88:0/2398462765 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f768c1a0240 con 0x7f768c0ffd60 2024-02-19T19:17:44.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.777+0000 7f7691b85700 1 -- 172.21.15.88:0/2398462765 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768c107730 con 0x7f768c0ffd60 2024-02-19T19:17:44.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.777+0000 7f7688ff9700 1 -- 172.21.15.88:0/2398462765 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f767c00f5d0 con 0x7f768c0ffd60 2024-02-19T19:17:44.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.778+0000 7f7688ff9700 1 --2- 172.21.15.88:0/2398462765 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f767406d8a0 0x7f767406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:44.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.778+0000 7f7688ff9700 1 -- 172.21.15.88:0/2398462765 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f767c08eab0 con 0x7f768c0ffd60 2024-02-19T19:17:44.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.778+0000 7f768b7fe700 1 --2- 172.21.15.88:0/2398462765 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f767406d8a0 0x7f767406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:44.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.778+0000 7f768b7fe700 1 --2- 172.21.15.88:0/2398462765 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f767406d8a0 0x7f767406fd50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7680005fd0 tx=0x7f768000a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:44.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.782+0000 7f7688ff9700 1 -- 172.21.15.88:0/2398462765 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f767c058260 con 0x7f768c0ffd60 2024-02-19T19:17:44.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:44 smithi132 ceph-mon[70676]: from='client.34806 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:44.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:44 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/604638124' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:17:45.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.995+0000 7f7691b85700 1 -- 172.21.15.88:0/2398462765 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f768c061190 con 0x7f767406d8a0 2024-02-19T19:17:45.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.997+0000 7f7688ff9700 1 -- 172.21.15.88:0/2398462765 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f768c061190 con 0x7f767406d8a0 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:17:45.013 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:17:45.014 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:17:45.014 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:17:45.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.999+0000 7f7691b85700 1 -- 172.21.15.88:0/2398462765 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f767406d8a0 msgr2=0x7f767406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:45.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.999+0000 7f7691b85700 1 --2- 172.21.15.88:0/2398462765 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f767406d8a0 0x7f767406fd50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7680005fd0 tx=0x7f768000a560 comp rx=0 tx=0).stop 2024-02-19T19:17:45.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.999+0000 7f7691b85700 1 -- 172.21.15.88:0/2398462765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f768c0ffd60 msgr2=0x7f768c1972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:45.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.999+0000 7f7691b85700 1 --2- 172.21.15.88:0/2398462765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f768c0ffd60 0x7f768c1972a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f767c0092e0 tx=0x7f767c00be20 comp rx=0 tx=0).stop 2024-02-19T19:17:45.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.999+0000 7f7691b85700 1 -- 172.21.15.88:0/2398462765 shutdown_connections 2024-02-19T19:17:45.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.999+0000 7f7691b85700 1 --2- 172.21.15.88:0/2398462765 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f767406d8a0 0x7f767406fd50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:45.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.999+0000 7f7691b85700 1 --2- 172.21.15.88:0/2398462765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f768c0ff3c0 0x7f768c196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:45.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.999+0000 7f7691b85700 1 --2- 172.21.15.88:0/2398462765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f768c0ffd60 0x7f768c1972a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:45.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:44.999+0000 7f7691b85700 1 -- 172.21.15.88:0/2398462765 >> 172.21.15.88:0/2398462765 conn(0x7f768c0fac80 msgr2=0x7f768c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:45.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.000+0000 7f7691b85700 1 -- 172.21.15.88:0/2398462765 shutdown_connections 2024-02-19T19:17:45.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.000+0000 7f7691b85700 1 -- 172.21.15.88:0/2398462765 wait complete. 2024-02-19T19:17:45.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.104+0000 7f3e621f9700 1 -- 172.21.15.88:0/2894018282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e5c0ff3c0 msgr2=0x7f3e5c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:45.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.104+0000 7f3e621f9700 1 --2- 172.21.15.88:0/2894018282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e5c0ff3c0 0x7f3e5c0ff790 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3e54009910 tx=0x7f3e54009c20 comp rx=0 tx=0).stop 2024-02-19T19:17:45.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.104+0000 7f3e621f9700 1 -- 172.21.15.88:0/2894018282 shutdown_connections 2024-02-19T19:17:45.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.104+0000 7f3e621f9700 1 --2- 172.21.15.88:0/2894018282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:45.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.104+0000 7f3e621f9700 1 --2- 172.21.15.88:0/2894018282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e5c0ff3c0 0x7f3e5c0ff790 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:45.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.104+0000 7f3e621f9700 1 -- 172.21.15.88:0/2894018282 >> 172.21.15.88:0/2894018282 conn(0x7f3e5c0fac60 msgr2=0x7f3e5c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:45.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.104+0000 7f3e621f9700 1 -- 172.21.15.88:0/2894018282 shutdown_connections 2024-02-19T19:17:45.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.104+0000 7f3e621f9700 1 -- 172.21.15.88:0/2894018282 wait complete. 2024-02-19T19:17:45.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.104+0000 7f3e621f9700 1 Processor -- start 2024-02-19T19:17:45.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e621f9700 1 -- start start 2024-02-19T19:17:45.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e621f9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e5c0ff3c0 0x7f3e5c194e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:45.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e621f9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c1953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:45.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e621f9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c195ab0 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e621f9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5c199840 con 0x7f3e5c0ffd60 2024-02-19T19:17:45.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e5b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e5c0ff3c0 0x7f3e5c194e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:45.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e5b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e5c0ff3c0 0x7f3e5c194e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39428/0 (socket says 172.21.15.88:39428) 2024-02-19T19:17:45.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e5b7fe700 1 -- 172.21.15.88:0/160732058 learned_addr learned my addr 172.21.15.88:0/160732058 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:17:45.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e5b7fe700 1 -- 172.21.15.88:0/160732058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e5c0ffd60 msgr2=0x7f3e5c1953d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:45.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e5affd700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c1953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:45.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e5b7fe700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c1953d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:45.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e5b7fe700 1 -- 172.21.15.88:0/160732058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540095f0 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e5affd700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c1953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:17:45.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.105+0000 7f3e5b7fe700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e5c0ff3c0 0x7f3e5c194e90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3e54006010 tx=0x7f3e5400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:45.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.106+0000 7f3e58ff9700 1 -- 172.21.15.88:0/160732058 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e5401d070 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.106+0000 7f3e58ff9700 1 -- 172.21.15.88:0/160732058 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e54004fb0 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.106+0000 7f3e621f9700 1 -- 172.21.15.88:0/160732058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e5c199ac0 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.106+0000 7f3e58ff9700 1 -- 172.21.15.88:0/160732058 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e54022ea0 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.106+0000 7f3e621f9700 1 -- 172.21.15.88:0/160732058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5c199fb0 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.107+0000 7f3e621f9700 1 -- 172.21.15.88:0/160732058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5c1076d0 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.107+0000 7f3e58ff9700 1 -- 172.21.15.88:0/160732058 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3e54022470 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.107+0000 7f3e58ff9700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e4406d8a0 0x7f3e4406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:17:45.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.108+0000 7f3e58ff9700 1 -- 172.21.15.88:0/160732058 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3e5408ebb0 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.108+0000 7f3e5affd700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e4406d8a0 0x7f3e4406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:17:45.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.108+0000 7f3e5affd700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e4406d8a0 0x7f3e4406fd50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c1964b0 tx=0x7f3e500073c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:17:45.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.112+0000 7f3e58ff9700 1 -- 172.21.15.88:0/160732058 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e54095080 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.398+0000 7f3e621f9700 1 -- 172.21.15.88:0/160732058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3e5c04f2a0 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.399+0000 7f3e58ff9700 1 -- 172.21.15.88:0/160732058 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f3e54027090 con 0x7f3e5c0ff3c0 2024-02-19T19:17:45.415 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:17:45.415 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:17:45.415 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:17:45.415 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:17:45.415 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:17:45.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.401+0000 7f3e621f9700 1 -- 172.21.15.88:0/160732058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e4406d8a0 msgr2=0x7f3e4406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:45.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.401+0000 7f3e621f9700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e4406d8a0 0x7f3e4406fd50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c1964b0 tx=0x7f3e500073c0 comp rx=0 tx=0).stop 2024-02-19T19:17:45.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.401+0000 7f3e621f9700 1 -- 172.21.15.88:0/160732058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e5c0ff3c0 msgr2=0x7f3e5c194e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:17:45.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.401+0000 7f3e621f9700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e5c0ff3c0 0x7f3e5c194e90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f3e54006010 tx=0x7f3e5400be20 comp rx=0 tx=0).stop 2024-02-19T19:17:45.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.401+0000 7f3e621f9700 1 -- 172.21.15.88:0/160732058 shutdown_connections 2024-02-19T19:17:45.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.401+0000 7f3e621f9700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e4406d8a0 0x7f3e4406fd50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:45.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.401+0000 7f3e621f9700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e5c0ff3c0 0x7f3e5c194e90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:45.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.401+0000 7f3e621f9700 1 --2- 172.21.15.88:0/160732058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e5c0ffd60 0x7f3e5c1953d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:17:45.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.401+0000 7f3e621f9700 1 -- 172.21.15.88:0/160732058 >> 172.21.15.88:0/160732058 conn(0x7f3e5c0fac60 msgr2=0x7f3e5c0fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:17:45.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.401+0000 7f3e621f9700 1 -- 172.21.15.88:0/160732058 shutdown_connections 2024-02-19T19:17:45.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:17:45.401+0000 7f3e621f9700 1 -- 172.21.15.88:0/160732058 wait complete. 2024-02-19T19:17:45.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:45 smithi088 ceph-mon[88643]: from='client.34810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:45.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:45 smithi088 ceph-mon[88643]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:45.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:45 smithi088 ceph-mon[88643]: pgmap v424: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:45.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:45 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3825007238' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:17:45.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:45 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/160732058' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:17:45.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:45 smithi132 ceph-mon[70676]: from='client.34810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:45.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:45 smithi132 ceph-mon[70676]: from='client.34814 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:45.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:45 smithi132 ceph-mon[70676]: pgmap v424: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:45.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:45 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3825007238' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:17:45.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:45 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/160732058' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:17:46.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:46 smithi088 ceph-mon[88643]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:46.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:46 smithi132 ceph-mon[70676]: from='client.34826 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:17:47.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:47 smithi088 ceph-mon[88643]: pgmap v425: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:47.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:47 smithi132 ceph-mon[70676]: pgmap v425: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:49.846 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:49 smithi088 ceph-mon[88643]: pgmap v426: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:49.998 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:49 smithi132 ceph-mon[70676]: pgmap v426: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:51.345 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:51 smithi088 ceph-mon[88643]: pgmap v427: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:51.498 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:51 smithi132 ceph-mon[70676]: pgmap v427: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:53.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:53 smithi132 ceph-mon[70676]: pgmap v428: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:53.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:53 smithi088 ceph-mon[88643]: pgmap v428: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:55.748 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:55 smithi132 ceph-mon[70676]: pgmap v429: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:55.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:55 smithi088 ceph-mon[88643]: pgmap v429: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:57.748 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:57 smithi132 ceph-mon[70676]: pgmap v430: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:57.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:57 smithi088 ceph-mon[88643]: pgmap v430: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:58.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:17:58.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:17:59.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:17:59 smithi132 ceph-mon[70676]: pgmap v431: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:17:59.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:17:59 smithi088 ceph-mon[88643]: pgmap v431: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:01.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:01 smithi132 ceph-mon[70676]: pgmap v432: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:01.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:01 smithi088 ceph-mon[88643]: pgmap v432: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:03.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:03 smithi132 ceph-mon[70676]: pgmap v433: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:03.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:03 smithi088 ceph-mon[88643]: pgmap v433: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:05.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:05 smithi132 ceph-mon[70676]: pgmap v434: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:05.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:05 smithi088 ceph-mon[88643]: pgmap v434: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:07.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:07 smithi088 ceph-mon[88643]: pgmap v435: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:07.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:07 smithi132 ceph-mon[70676]: pgmap v435: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:09.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:09 smithi132 ceph-mon[70676]: pgmap v436: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:09.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:09 smithi088 ceph-mon[88643]: pgmap v436: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:11.647 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:11 smithi088 ceph-mon[88643]: pgmap v437: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:11.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:11 smithi132 ceph-mon[70676]: pgmap v437: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:13.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:13 smithi132 ceph-mon[70676]: pgmap v438: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:13.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:18:13.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:13 smithi088 ceph-mon[88643]: pgmap v438: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:13.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:18:15.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.514+0000 7f3543322700 1 -- 172.21.15.88:0/735133853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f353c0ffb20 msgr2=0x7f353c104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.514+0000 7f3543322700 1 --2- 172.21.15.88:0/735133853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f353c0ffb20 0x7f353c104000 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f352c009910 tx=0x7f352c009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.514+0000 7f3543322700 1 -- 172.21.15.88:0/735133853 shutdown_connections 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.514+0000 7f3543322700 1 --2- 172.21.15.88:0/735133853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f353c0ffb20 0x7f353c104000 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.514+0000 7f3543322700 1 --2- 172.21.15.88:0/735133853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f353c0ff210 0x7f353c0ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.514+0000 7f3543322700 1 -- 172.21.15.88:0/735133853 >> 172.21.15.88:0/735133853 conn(0x7f353c0fac60 msgr2=0x7f353c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.514+0000 7f3543322700 1 -- 172.21.15.88:0/735133853 shutdown_connections 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.514+0000 7f3543322700 1 -- 172.21.15.88:0/735133853 wait complete. 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.514+0000 7f3543322700 1 Processor -- start 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f3543322700 1 -- start start 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f3543322700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f353c0ff210 0x7f353c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f3543322700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f353c0ffb20 0x7f353c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:15.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f3543322700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353c19d070 con 0x7f353c0ff210 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f3543322700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f353c1a0e00 con 0x7f353c0ffb20 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f35410be700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f353c0ff210 0x7f353c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f35410be700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f353c0ff210 0x7f353c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49362/0 (socket says 172.21.15.88:49362) 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f35410be700 1 -- 172.21.15.88:0/4134849282 learned_addr learned my addr 172.21.15.88:0/4134849282 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f35408bd700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f353c0ffb20 0x7f353c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f35410be700 1 -- 172.21.15.88:0/4134849282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f353c0ffb20 msgr2=0x7f353c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f35410be700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f353c0ffb20 0x7f353c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f35410be700 1 -- 172.21.15.88:0/4134849282 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0095f0 con 0x7f353c0ff210 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f35408bd700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f353c0ffb20 0x7f353c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f35410be700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f353c0ff210 0x7f353c19c450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f353800d8d0 tx=0x7f353800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.515+0000 7f35327fc700 1 -- 172.21.15.88:0/4134849282 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3538009940 con 0x7f353c0ff210 2024-02-19T19:18:15.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.516+0000 7f35327fc700 1 -- 172.21.15.88:0/4134849282 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3538010460 con 0x7f353c0ff210 2024-02-19T19:18:15.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.516+0000 7f3543322700 1 -- 172.21.15.88:0/4134849282 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f353c1a10e0 con 0x7f353c0ff210 2024-02-19T19:18:15.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.516+0000 7f35327fc700 1 -- 172.21.15.88:0/4134849282 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f353800f5f0 con 0x7f353c0ff210 2024-02-19T19:18:15.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.516+0000 7f3543322700 1 -- 172.21.15.88:0/4134849282 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f353c1a1630 con 0x7f353c0ff210 2024-02-19T19:18:15.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.517+0000 7f3543322700 1 -- 172.21.15.88:0/4134849282 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f353c1016b0 con 0x7f353c0ff210 2024-02-19T19:18:15.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.517+0000 7f35327fc700 1 -- 172.21.15.88:0/4134849282 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f35380105d0 con 0x7f353c0ff210 2024-02-19T19:18:15.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.517+0000 7f35327fc700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f352806d8a0 0x7f352806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:15.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.518+0000 7f35327fc700 1 -- 172.21.15.88:0/4134849282 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f353808d430 con 0x7f353c0ff210 2024-02-19T19:18:15.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.518+0000 7f35408bd700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f352806d8a0 0x7f352806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:15.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.518+0000 7f35408bd700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f352806d8a0 0x7f352806fd50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f352c00b5c0 tx=0x7f352c0053a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:15.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.522+0000 7f35327fc700 1 -- 172.21.15.88:0/4134849282 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3538056c60 con 0x7f353c0ff210 2024-02-19T19:18:15.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:15 smithi132 ceph-mon[70676]: pgmap v439: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:15.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.736+0000 7f3543322700 1 -- 172.21.15.88:0/4134849282 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f353c1a18a0 con 0x7f352806d8a0 2024-02-19T19:18:15.751 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:15 smithi088 ceph-mon[88643]: pgmap v439: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:15.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.738+0000 7f35327fc700 1 -- 172.21.15.88:0/4134849282 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f353c1a18a0 con 0x7f352806d8a0 2024-02-19T19:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.740+0000 7f3543322700 1 -- 172.21.15.88:0/4134849282 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f352806d8a0 msgr2=0x7f352806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.740+0000 7f3543322700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f352806d8a0 0x7f352806fd50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f352c00b5c0 tx=0x7f352c0053a0 comp rx=0 tx=0).stop 2024-02-19T19:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.740+0000 7f3543322700 1 -- 172.21.15.88:0/4134849282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f353c0ff210 msgr2=0x7f353c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.740+0000 7f3543322700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f353c0ff210 0x7f353c19c450 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f353800d8d0 tx=0x7f353800dc90 comp rx=0 tx=0).stop 2024-02-19T19:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.740+0000 7f3543322700 1 -- 172.21.15.88:0/4134849282 shutdown_connections 2024-02-19T19:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.740+0000 7f3543322700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f352806d8a0 0x7f352806fd50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.740+0000 7f3543322700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f353c0ff210 0x7f353c19c450 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.740+0000 7f3543322700 1 --2- 172.21.15.88:0/4134849282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f353c0ffb20 0x7f353c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.740+0000 7f3543322700 1 -- 172.21.15.88:0/4134849282 >> 172.21.15.88:0/4134849282 conn(0x7f353c0fac60 msgr2=0x7f353c110680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:15.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.740+0000 7f3543322700 1 -- 172.21.15.88:0/4134849282 shutdown_connections 2024-02-19T19:18:15.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.740+0000 7f3543322700 1 -- 172.21.15.88:0/4134849282 wait complete. 2024-02-19T19:18:15.765 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:18:15.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.844+0000 7f834c30f700 1 -- 172.21.15.88:0/1219340951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83440ffd60 msgr2=0x7f8344103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:15.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.844+0000 7f834c30f700 1 --2- 172.21.15.88:0/1219340951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83440ffd60 0x7f8344103d30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8340009910 tx=0x7f8340009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:15.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.844+0000 7f834c30f700 1 -- 172.21.15.88:0/1219340951 shutdown_connections 2024-02-19T19:18:15.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.844+0000 7f834c30f700 1 --2- 172.21.15.88:0/1219340951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83440ffd60 0x7f8344103d30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:15.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.844+0000 7f834c30f700 1 --2- 172.21.15.88:0/1219340951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83440ff3c0 0x7f83440ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:15.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.844+0000 7f834c30f700 1 -- 172.21.15.88:0/1219340951 >> 172.21.15.88:0/1219340951 conn(0x7f83440fac80 msgr2=0x7f83440fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:15.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.844+0000 7f834c30f700 1 -- 172.21.15.88:0/1219340951 shutdown_connections 2024-02-19T19:18:15.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.844+0000 7f834c30f700 1 -- 172.21.15.88:0/1219340951 wait complete. 2024-02-19T19:18:15.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f834c30f700 1 Processor -- start 2024-02-19T19:18:15.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f834c30f700 1 -- start start 2024-02-19T19:18:15.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f834c30f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83440ff3c0 0x7f8344196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:15.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f834c30f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83440ffd60 0x7f83441972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:15.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f834c30f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83441978e0 con 0x7f83440ffd60 2024-02-19T19:18:15.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f834c30f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8344197a20 con 0x7f83440ff3c0 2024-02-19T19:18:15.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f83498aa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83440ffd60 0x7f83441972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:15.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f83498aa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83440ffd60 0x7f83441972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49382/0 (socket says 172.21.15.88:49382) 2024-02-19T19:18:15.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f83498aa700 1 -- 172.21.15.88:0/1839783541 learned_addr learned my addr 172.21.15.88:0/1839783541 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:15.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f83498aa700 1 -- 172.21.15.88:0/1839783541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83440ff3c0 msgr2=0x7f8344196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:15.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f834a0ab700 1 --2- 172.21.15.88:0/1839783541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83440ff3c0 0x7f8344196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:15.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f83498aa700 1 --2- 172.21.15.88:0/1839783541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83440ff3c0 0x7f8344196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:15.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f83498aa700 1 -- 172.21.15.88:0/1839783541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83400095f0 con 0x7f83440ffd60 2024-02-19T19:18:15.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.845+0000 7f83498aa700 1 --2- 172.21.15.88:0/1839783541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83440ffd60 0x7f83441972c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f83400092e0 tx=0x7f834000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:15.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.846+0000 7f833b7fe700 1 -- 172.21.15.88:0/1839783541 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f834001d070 con 0x7f83440ffd60 2024-02-19T19:18:15.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.846+0000 7f833b7fe700 1 -- 172.21.15.88:0/1839783541 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8340005dc0 con 0x7f83440ffd60 2024-02-19T19:18:15.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.846+0000 7f834c30f700 1 -- 172.21.15.88:0/1839783541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f834419fdf0 con 0x7f83440ffd60 2024-02-19T19:18:15.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.846+0000 7f833b7fe700 1 -- 172.21.15.88:0/1839783541 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8340017400 con 0x7f83440ffd60 2024-02-19T19:18:15.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.846+0000 7f834c30f700 1 -- 172.21.15.88:0/1839783541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83441a0200 con 0x7f83440ffd60 2024-02-19T19:18:15.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.847+0000 7f834c30f700 1 -- 172.21.15.88:0/1839783541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8344107730 con 0x7f83440ffd60 2024-02-19T19:18:15.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.847+0000 7f833b7fe700 1 -- 172.21.15.88:0/1839783541 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f834000f460 con 0x7f83440ffd60 2024-02-19T19:18:15.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.848+0000 7f833b7fe700 1 --2- 172.21.15.88:0/1839783541 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f833006d8a0 0x7f833006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:15.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.848+0000 7f834a0ab700 1 --2- 172.21.15.88:0/1839783541 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f833006d8a0 0x7f833006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:15.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.848+0000 7f833b7fe700 1 -- 172.21.15.88:0/1839783541 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f834008e950 con 0x7f83440ffd60 2024-02-19T19:18:15.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.848+0000 7f834a0ab700 1 --2- 172.21.15.88:0/1839783541 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f833006d8a0 0x7f833006fd50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8334005950 tx=0x7f83340058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:15.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:15.853+0000 7f833b7fe700 1 -- 172.21.15.88:0/1839783541 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8340058180 con 0x7f83440ffd60 2024-02-19T19:18:16.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.067+0000 7f834c30f700 1 -- 172.21.15.88:0/1839783541 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8344061190 con 0x7f833006d8a0 2024-02-19T19:18:16.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.068+0000 7f833b7fe700 1 -- 172.21.15.88:0/1839783541 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8344061190 con 0x7f833006d8a0 2024-02-19T19:18:16.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.071+0000 7f834c30f700 1 -- 172.21.15.88:0/1839783541 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f833006d8a0 msgr2=0x7f833006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.071+0000 7f834c30f700 1 --2- 172.21.15.88:0/1839783541 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f833006d8a0 0x7f833006fd50 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8334005950 tx=0x7f83340058e0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.071+0000 7f834c30f700 1 -- 172.21.15.88:0/1839783541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83440ffd60 msgr2=0x7f83441972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.071+0000 7f834c30f700 1 --2- 172.21.15.88:0/1839783541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83440ffd60 0x7f83441972c0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f83400092e0 tx=0x7f834000be20 comp rx=0 tx=0).stop 2024-02-19T19:18:16.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.071+0000 7f834c30f700 1 -- 172.21.15.88:0/1839783541 shutdown_connections 2024-02-19T19:18:16.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.071+0000 7f834c30f700 1 --2- 172.21.15.88:0/1839783541 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f833006d8a0 0x7f833006fd50 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.071+0000 7f834c30f700 1 --2- 172.21.15.88:0/1839783541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83440ff3c0 0x7f8344196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.071+0000 7f834c30f700 1 --2- 172.21.15.88:0/1839783541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83440ffd60 0x7f83441972c0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.071+0000 7f834c30f700 1 -- 172.21.15.88:0/1839783541 >> 172.21.15.88:0/1839783541 conn(0x7f83440fac80 msgr2=0x7f83440fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:16.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.071+0000 7f834c30f700 1 -- 172.21.15.88:0/1839783541 shutdown_connections 2024-02-19T19:18:16.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.071+0000 7f834c30f700 1 -- 172.21.15.88:0/1839783541 wait complete. 2024-02-19T19:18:16.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.175+0000 7fbeed7a9700 1 -- 172.21.15.88:0/607806035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee80ffc70 msgr2=0x7fbee810a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.175+0000 7fbeed7a9700 1 --2- 172.21.15.88:0/607806035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee80ffc70 0x7fbee810a6e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009910 tx=0x7fbed0009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:16.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.175+0000 7fbeed7a9700 1 -- 172.21.15.88:0/607806035 shutdown_connections 2024-02-19T19:18:16.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.175+0000 7fbeed7a9700 1 --2- 172.21.15.88:0/607806035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee80ffc70 0x7fbee810a6e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.175+0000 7fbeed7a9700 1 --2- 172.21.15.88:0/607806035 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbee80ff360 0x7fbee80ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.175+0000 7fbeed7a9700 1 -- 172.21.15.88:0/607806035 >> 172.21.15.88:0/607806035 conn(0x7fbee80748c0 msgr2=0x7fbee8074cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:16.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.175+0000 7fbeed7a9700 1 -- 172.21.15.88:0/607806035 shutdown_connections 2024-02-19T19:18:16.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.175+0000 7fbeed7a9700 1 -- 172.21.15.88:0/607806035 wait complete. 2024-02-19T19:18:16.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.175+0000 7fbeed7a9700 1 Processor -- start 2024-02-19T19:18:16.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbeed7a9700 1 -- start start 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbeed7a9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbee80ff360 0x7fbee8195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbeed7a9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee80ffc70 0x7fbee8196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbeed7a9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee8196a60 con 0x7fbee80ffc70 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbeed7a9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbee819a7f0 con 0x7fbee80ff360 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbede7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee80ffc70 0x7fbee8196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbede7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee80ffc70 0x7fbee8196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49402/0 (socket says 172.21.15.88:49402) 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbede7fc700 1 -- 172.21.15.88:0/1324196396 learned_addr learned my addr 172.21.15.88:0/1324196396 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbee6ffd700 1 --2- 172.21.15.88:0/1324196396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbee80ff360 0x7fbee8195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbede7fc700 1 -- 172.21.15.88:0/1324196396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbee80ff360 msgr2=0x7fbee8195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbede7fc700 1 --2- 172.21.15.88:0/1324196396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbee80ff360 0x7fbee8195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbede7fc700 1 -- 172.21.15.88:0/1324196396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed00095f0 con 0x7fbee80ffc70 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbede7fc700 1 --2- 172.21.15.88:0/1324196396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee80ffc70 0x7fbee8196380 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbed0006010 tx=0x7fbed000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:16.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.176+0000 7fbee4ff9700 1 -- 172.21.15.88:0/1324196396 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbed001d070 con 0x7fbee80ffc70 2024-02-19T19:18:16.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.177+0000 7fbee4ff9700 1 -- 172.21.15.88:0/1324196396 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbed0005dc0 con 0x7fbee80ffc70 2024-02-19T19:18:16.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.177+0000 7fbee4ff9700 1 -- 172.21.15.88:0/1324196396 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbed0017400 con 0x7fbee80ffc70 2024-02-19T19:18:16.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.177+0000 7fbeed7a9700 1 -- 172.21.15.88:0/1324196396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbee819aad0 con 0x7fbee80ffc70 2024-02-19T19:18:16.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.177+0000 7fbeed7a9700 1 -- 172.21.15.88:0/1324196396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbee819b020 con 0x7fbee80ffc70 2024-02-19T19:18:16.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.177+0000 7fbeed7a9700 1 -- 172.21.15.88:0/1324196396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbee8107de0 con 0x7fbee80ffc70 2024-02-19T19:18:16.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.178+0000 7fbee4ff9700 1 -- 172.21.15.88:0/1324196396 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbed0003a60 con 0x7fbee80ffc70 2024-02-19T19:18:16.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.178+0000 7fbee4ff9700 1 --2- 172.21.15.88:0/1324196396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbed406d8a0 0x7fbed406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:16.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.179+0000 7fbee4ff9700 1 -- 172.21.15.88:0/1324196396 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbed008e960 con 0x7fbee80ffc70 2024-02-19T19:18:16.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.179+0000 7fbee6ffd700 1 --2- 172.21.15.88:0/1324196396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbed406d8a0 0x7fbed406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:16.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.179+0000 7fbee6ffd700 1 --2- 172.21.15.88:0/1324196396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbed406d8a0 0x7fbed406fd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbed8005950 tx=0x7fbed800b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:16.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.183+0000 7fbee4ff9700 1 -- 172.21.15.88:0/1324196396 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbed0058110 con 0x7fbee80ffc70 2024-02-19T19:18:16.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.394+0000 7fbeed7a9700 1 -- 172.21.15.88:0/1324196396 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbee819b240 con 0x7fbed406d8a0 2024-02-19T19:18:16.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.404+0000 7fbee4ff9700 1 -- 172.21.15.88:0/1324196396 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbee819b240 con 0x7fbed406d8a0 2024-02-19T19:18:16.419 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (15m) 7m ago 25m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (26m) 7m ago 26m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (8m) 6m ago 24m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (11m) 7m ago 26m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (11m) 6m ago 24m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (14m) 7m ago 25m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (7m) 7m ago 19m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (7m) 7m ago 19m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (7m) 6m ago 19m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (19m) 6m ago 19m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (16m) 7m ago 27m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:18:16.420 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (16m) 6m ago 24m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:18:16.421 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (13m) 7m ago 27m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:18:16.421 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (12m) 6m ago 24m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:18:16.421 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (16m) 7m ago 26m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:18:16.421 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (16m) 6m ago 24m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:18:16.421 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (10m) 7m ago 23m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:18:16.421 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (10m) 7m ago 22m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:18:16.421 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (9m) 7m ago 22m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:18:16.421 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (9m) 6m ago 22m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:18:16.421 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (8m) 6m ago 21m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:18:16.421 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (7m) 6m ago 21m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:18:16.421 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (15m) 7m ago 24m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:18:16.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.407+0000 7fbeed7a9700 1 -- 172.21.15.88:0/1324196396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbed406d8a0 msgr2=0x7fbed406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.407+0000 7fbeed7a9700 1 --2- 172.21.15.88:0/1324196396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbed406d8a0 0x7fbed406fd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fbed8005950 tx=0x7fbed800b410 comp rx=0 tx=0).stop 2024-02-19T19:18:16.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.407+0000 7fbeed7a9700 1 -- 172.21.15.88:0/1324196396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee80ffc70 msgr2=0x7fbee8196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.407+0000 7fbeed7a9700 1 --2- 172.21.15.88:0/1324196396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee80ffc70 0x7fbee8196380 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fbed0006010 tx=0x7fbed000bd70 comp rx=0 tx=0).stop 2024-02-19T19:18:16.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.407+0000 7fbeed7a9700 1 -- 172.21.15.88:0/1324196396 shutdown_connections 2024-02-19T19:18:16.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.407+0000 7fbeed7a9700 1 --2- 172.21.15.88:0/1324196396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbed406d8a0 0x7fbed406fd50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.407+0000 7fbeed7a9700 1 --2- 172.21.15.88:0/1324196396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbee80ff360 0x7fbee8195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.407+0000 7fbeed7a9700 1 --2- 172.21.15.88:0/1324196396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbee80ffc70 0x7fbee8196380 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.407+0000 7fbeed7a9700 1 -- 172.21.15.88:0/1324196396 >> 172.21.15.88:0/1324196396 conn(0x7fbee80748c0 msgr2=0x7fbee8104f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:16.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.407+0000 7fbeed7a9700 1 -- 172.21.15.88:0/1324196396 shutdown_connections 2024-02-19T19:18:16.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.407+0000 7fbeed7a9700 1 -- 172.21.15.88:0/1324196396 wait complete. 2024-02-19T19:18:16.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.509+0000 7fe39d6f5700 1 -- 172.21.15.88:0/3324378729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3980ffc60 msgr2=0x7fe39810a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.509+0000 7fe39d6f5700 1 --2- 172.21.15.88:0/3324378729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3980ffc60 0x7fe39810a6d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009910 tx=0x7fe38c009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:16.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.509+0000 7fe39d6f5700 1 -- 172.21.15.88:0/3324378729 shutdown_connections 2024-02-19T19:18:16.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.509+0000 7fe39d6f5700 1 --2- 172.21.15.88:0/3324378729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3980ffc60 0x7fe39810a6d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.509+0000 7fe39d6f5700 1 --2- 172.21.15.88:0/3324378729 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3980ff350 0x7fe3980ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.509+0000 7fe39d6f5700 1 -- 172.21.15.88:0/3324378729 >> 172.21.15.88:0/3324378729 conn(0x7fe3980748b0 msgr2=0x7fe398074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:16.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.510+0000 7fe39d6f5700 1 -- 172.21.15.88:0/3324378729 shutdown_connections 2024-02-19T19:18:16.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.510+0000 7fe39d6f5700 1 -- 172.21.15.88:0/3324378729 wait complete. 2024-02-19T19:18:16.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.510+0000 7fe39d6f5700 1 Processor -- start 2024-02-19T19:18:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.510+0000 7fe39d6f5700 1 -- start start 2024-02-19T19:18:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe39d6f5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3980ff350 0x7fe398100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe39d6f5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3980ffc60 0x7fe398100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe39d6f5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe398100fb0 con 0x7fe3980ff350 2024-02-19T19:18:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe39d6f5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3981010f0 con 0x7fe3980ffc60 2024-02-19T19:18:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe396ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3980ff350 0x7fe398100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe396ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3980ff350 0x7fe398100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49430/0 (socket says 172.21.15.88:49430) 2024-02-19T19:18:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe396ffd700 1 -- 172.21.15.88:0/2867154383 learned_addr learned my addr 172.21.15.88:0/2867154383 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe3967fc700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3980ffc60 0x7fe398100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe396ffd700 1 -- 172.21.15.88:0/2867154383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3980ffc60 msgr2=0x7fe398100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe396ffd700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3980ffc60 0x7fe398100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe396ffd700 1 -- 172.21.15.88:0/2867154383 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe38c0095f0 con 0x7fe3980ff350 2024-02-19T19:18:16.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe3967fc700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3980ffc60 0x7fe398100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:18:16.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe396ffd700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3980ff350 0x7fe398100530 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe38800b480 tx=0x7fe38800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:16.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe37ffff700 1 -- 172.21.15.88:0/2867154383 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3880118c0 con 0x7fe3980ff350 2024-02-19T19:18:16.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe37ffff700 1 -- 172.21.15.88:0/2867154383 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe388010380 con 0x7fe3980ff350 2024-02-19T19:18:16.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe39d6f5700 1 -- 172.21.15.88:0/2867154383 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3981a2130 con 0x7fe3980ff350 2024-02-19T19:18:16.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe37ffff700 1 -- 172.21.15.88:0/2867154383 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe38800f4f0 con 0x7fe3980ff350 2024-02-19T19:18:16.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.511+0000 7fe39d6f5700 1 -- 172.21.15.88:0/2867154383 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3981a2540 con 0x7fe3980ff350 2024-02-19T19:18:16.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.512+0000 7fe39d6f5700 1 -- 172.21.15.88:0/2867154383 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe398107dd0 con 0x7fe3980ff350 2024-02-19T19:18:16.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.513+0000 7fe37ffff700 1 -- 172.21.15.88:0/2867154383 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe3880104f0 con 0x7fe3980ff350 2024-02-19T19:18:16.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.513+0000 7fe37ffff700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe38006d8a0 0x7fe38006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:16.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.513+0000 7fe37ffff700 1 -- 172.21.15.88:0/2867154383 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe38808d1a0 con 0x7fe3980ff350 2024-02-19T19:18:16.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.513+0000 7fe3967fc700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe38006d8a0 0x7fe38006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:16.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.514+0000 7fe3967fc700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe38006d8a0 0x7fe38006fd50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00b5c0 tx=0x7fe38c005480 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:16.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.518+0000 7fe37ffff700 1 -- 172.21.15.88:0/2867154383 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe388056950 con 0x7fe3980ff350 2024-02-19T19:18:16.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.795+0000 7fe39d6f5700 1 -- 172.21.15.88:0/2867154383 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe39804f2a0 con 0x7fe3980ff350 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.796+0000 7fe37ffff700 1 -- 172.21.15.88:0/2867154383 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fe38805a010 con 0x7fe3980ff350 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:18:16.811 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:18:16.812 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:18:16.812 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:18:16.812 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:18:16.812 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:18:16.812 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:18:16.812 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:18:16.812 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:18:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.798+0000 7fe39d6f5700 1 -- 172.21.15.88:0/2867154383 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe38006d8a0 msgr2=0x7fe38006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.798+0000 7fe39d6f5700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe38006d8a0 0x7fe38006fd50 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe38c00b5c0 tx=0x7fe38c005480 comp rx=0 tx=0).stop 2024-02-19T19:18:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.798+0000 7fe39d6f5700 1 -- 172.21.15.88:0/2867154383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3980ff350 msgr2=0x7fe398100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.798+0000 7fe39d6f5700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3980ff350 0x7fe398100530 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fe38800b480 tx=0x7fe38800b840 comp rx=0 tx=0).stop 2024-02-19T19:18:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.798+0000 7fe39d6f5700 1 -- 172.21.15.88:0/2867154383 shutdown_connections 2024-02-19T19:18:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.798+0000 7fe39d6f5700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe38006d8a0 0x7fe38006fd50 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.798+0000 7fe39d6f5700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3980ff350 0x7fe398100530 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.798+0000 7fe39d6f5700 1 --2- 172.21.15.88:0/2867154383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe3980ffc60 0x7fe398100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.798+0000 7fe39d6f5700 1 -- 172.21.15.88:0/2867154383 >> 172.21.15.88:0/2867154383 conn(0x7fe3980748b0 msgr2=0x7fe398104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:16.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.798+0000 7fe39d6f5700 1 -- 172.21.15.88:0/2867154383 shutdown_connections 2024-02-19T19:18:16.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.798+0000 7fe39d6f5700 1 -- 172.21.15.88:0/2867154383 wait complete. 2024-02-19T19:18:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.903+0000 7fa14f874700 1 -- 172.21.15.88:0/1854110086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa148100e80 msgr2=0x7fa148104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.903+0000 7fa14f874700 1 --2- 172.21.15.88:0/1854110086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa148100e80 0x7fa148104e50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa138009910 tx=0x7fa138009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.903+0000 7fa14f874700 1 -- 172.21.15.88:0/1854110086 shutdown_connections 2024-02-19T19:18:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.903+0000 7fa14f874700 1 --2- 172.21.15.88:0/1854110086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa148100e80 0x7fa148104e50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.903+0000 7fa14f874700 1 --2- 172.21.15.88:0/1854110086 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1481004e0 0x7fa1481008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.903+0000 7fa14f874700 1 -- 172.21.15.88:0/1854110086 >> 172.21.15.88:0/1854110086 conn(0x7fa1480fbda0 msgr2=0x7fa1480fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.903+0000 7fa14f874700 1 -- 172.21.15.88:0/1854110086 shutdown_connections 2024-02-19T19:18:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.903+0000 7fa14f874700 1 -- 172.21.15.88:0/1854110086 wait complete. 2024-02-19T19:18:16.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14f874700 1 Processor -- start 2024-02-19T19:18:16.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14f874700 1 -- start start 2024-02-19T19:18:16.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14f874700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1481004e0 0x7fa148195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:16.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14f874700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa148100e80 0x7fa148196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:16.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14f874700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa148196b30 con 0x7fa1481004e0 2024-02-19T19:18:16.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14f874700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14819a8c0 con 0x7fa148100e80 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14d610700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1481004e0 0x7fa148195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14d610700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1481004e0 0x7fa148195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49450/0 (socket says 172.21.15.88:49450) 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14d610700 1 -- 172.21.15.88:0/2874656529 learned_addr learned my addr 172.21.15.88:0/2874656529 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14ce0f700 1 --2- 172.21.15.88:0/2874656529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa148100e80 0x7fa148196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14d610700 1 -- 172.21.15.88:0/2874656529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa148100e80 msgr2=0x7fa148196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14d610700 1 --2- 172.21.15.88:0/2874656529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa148100e80 0x7fa148196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14d610700 1 -- 172.21.15.88:0/2874656529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1380095f0 con 0x7fa1481004e0 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.904+0000 7fa14d610700 1 --2- 172.21.15.88:0/2874656529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1481004e0 0x7fa148195f10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa14000ada0 tx=0x7fa14000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.905+0000 7fa13e7fc700 1 -- 172.21.15.88:0/2874656529 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa14000f070 con 0x7fa1481004e0 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.905+0000 7fa13e7fc700 1 -- 172.21.15.88:0/2874656529 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa140007bf0 con 0x7fa1481004e0 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.905+0000 7fa14f874700 1 -- 172.21.15.88:0/2874656529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14819aba0 con 0x7fa1481004e0 2024-02-19T19:18:16.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.905+0000 7fa13e7fc700 1 -- 172.21.15.88:0/2874656529 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa140005690 con 0x7fa1481004e0 2024-02-19T19:18:16.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.905+0000 7fa14f874700 1 -- 172.21.15.88:0/2874656529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14819b0f0 con 0x7fa1481004e0 2024-02-19T19:18:16.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.906+0000 7fa14f874700 1 -- 172.21.15.88:0/2874656529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa148108850 con 0x7fa1481004e0 2024-02-19T19:18:16.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.906+0000 7fa13e7fc700 1 -- 172.21.15.88:0/2874656529 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa140005910 con 0x7fa1481004e0 2024-02-19T19:18:16.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.907+0000 7fa13e7fc700 1 --2- 172.21.15.88:0/2874656529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa13406d8a0 0x7fa13406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:16.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.907+0000 7fa13e7fc700 1 -- 172.21.15.88:0/2874656529 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa14008ced0 con 0x7fa1481004e0 2024-02-19T19:18:16.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.907+0000 7fa14ce0f700 1 --2- 172.21.15.88:0/2874656529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa13406d8a0 0x7fa13406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:16.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.907+0000 7fa14ce0f700 1 --2- 172.21.15.88:0/2874656529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa13406d8a0 0x7fa13406fd50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa148197530 tx=0x7fa13801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:16.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:16.912+0000 7fa13e7fc700 1 -- 172.21.15.88:0/2874656529 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa140056700 con 0x7fa1481004e0 2024-02-19T19:18:17.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.151+0000 7fa14f874700 1 -- 172.21.15.88:0/2874656529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa14819b360 con 0x7fa1481004e0 2024-02-19T19:18:17.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.152+0000 7fa13e7fc700 1 -- 172.21.15.88:0/2874656529 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fa140059dc0 con 0x7fa1481004e0 2024-02-19T19:18:17.167 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:18:17.167 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:18:17.167 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:18:17.167 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:18:17.167 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:18:17.167 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:18:17.167 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:18:17.167 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:18:17.167 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:18:17.167 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:18:17.167 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:18:17.168 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:18:17.169 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:18:17.170 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:18:17.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.154+0000 7fa14f874700 1 -- 172.21.15.88:0/2874656529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa13406d8a0 msgr2=0x7fa13406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:17.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.154+0000 7fa14f874700 1 --2- 172.21.15.88:0/2874656529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa13406d8a0 0x7fa13406fd50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa148197530 tx=0x7fa13801a040 comp rx=0 tx=0).stop 2024-02-19T19:18:17.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.154+0000 7fa14f874700 1 -- 172.21.15.88:0/2874656529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1481004e0 msgr2=0x7fa148195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:17.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.154+0000 7fa14f874700 1 --2- 172.21.15.88:0/2874656529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1481004e0 0x7fa148195f10 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa14000ada0 tx=0x7fa14000c330 comp rx=0 tx=0).stop 2024-02-19T19:18:17.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.154+0000 7fa14f874700 1 -- 172.21.15.88:0/2874656529 shutdown_connections 2024-02-19T19:18:17.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.154+0000 7fa14f874700 1 --2- 172.21.15.88:0/2874656529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa13406d8a0 0x7fa13406fd50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.154+0000 7fa14f874700 1 --2- 172.21.15.88:0/2874656529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1481004e0 0x7fa148195f10 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.154+0000 7fa14f874700 1 --2- 172.21.15.88:0/2874656529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa148100e80 0x7fa148196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.154+0000 7fa14f874700 1 -- 172.21.15.88:0/2874656529 >> 172.21.15.88:0/2874656529 conn(0x7fa1480fbda0 msgr2=0x7fa1480fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.154+0000 7fa14f874700 1 -- 172.21.15.88:0/2874656529 shutdown_connections 2024-02-19T19:18:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.154+0000 7fa14f874700 1 -- 172.21.15.88:0/2874656529 wait complete. 2024-02-19T19:18:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:18:17.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.260+0000 7f5443916700 1 -- 172.21.15.88:0/1648732336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f543c100460 msgr2=0x7f543c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:17.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.260+0000 7f5443916700 1 --2- 172.21.15.88:0/1648732336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f543c100460 0x7f543c100830 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5438009910 tx=0x7f5438009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:17.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.260+0000 7f5443916700 1 -- 172.21.15.88:0/1648732336 shutdown_connections 2024-02-19T19:18:17.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.260+0000 7f5443916700 1 --2- 172.21.15.88:0/1648732336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f543c100d70 0x7f543c1094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.260+0000 7f5443916700 1 --2- 172.21.15.88:0/1648732336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f543c100460 0x7f543c100830 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.260+0000 7f5443916700 1 -- 172.21.15.88:0/1648732336 >> 172.21.15.88:0/1648732336 conn(0x7f543c0748b0 msgr2=0x7f543c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.261+0000 7f5443916700 1 -- 172.21.15.88:0/1648732336 shutdown_connections 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.261+0000 7f5443916700 1 -- 172.21.15.88:0/1648732336 wait complete. 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.261+0000 7f5443916700 1 Processor -- start 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.261+0000 7f5443916700 1 -- start start 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.261+0000 7f5443916700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f543c100460 0x7f543c195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.261+0000 7f5443916700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f543c100d70 0x7f543c1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.261+0000 7f5443916700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c196bc0 con 0x7f543c100d70 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.261+0000 7f5443916700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c19a950 con 0x7f543c100460 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f5440eb1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f543c100d70 0x7f543c1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f5440eb1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f543c100d70 0x7f543c1964e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49468/0 (socket says 172.21.15.88:49468) 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f5440eb1700 1 -- 172.21.15.88:0/1385108134 learned_addr learned my addr 172.21.15.88:0/1385108134 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f54416b2700 1 --2- 172.21.15.88:0/1385108134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f543c100460 0x7f543c195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:17.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f5440eb1700 1 -- 172.21.15.88:0/1385108134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f543c100460 msgr2=0x7f543c195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:17.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f5440eb1700 1 --2- 172.21.15.88:0/1385108134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f543c100460 0x7f543c195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f5440eb1700 1 -- 172.21.15.88:0/1385108134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54380095f0 con 0x7f543c100d70 2024-02-19T19:18:17.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f5440eb1700 1 --2- 172.21.15.88:0/1385108134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f543c100d70 0x7f543c1964e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f542c00b4b0 tx=0x7f542c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:17.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f54327fc700 1 -- 172.21.15.88:0/1385108134 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f542c011800 con 0x7f543c100d70 2024-02-19T19:18:17.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f5443916700 1 -- 172.21.15.88:0/1385108134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543c19ac30 con 0x7f543c100d70 2024-02-19T19:18:17.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f54327fc700 1 -- 172.21.15.88:0/1385108134 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f542c011e20 con 0x7f543c100d70 2024-02-19T19:18:17.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f54327fc700 1 -- 172.21.15.88:0/1385108134 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f542c00f2e0 con 0x7f543c100d70 2024-02-19T19:18:17.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.262+0000 7f5443916700 1 -- 172.21.15.88:0/1385108134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c19b180 con 0x7f543c100d70 2024-02-19T19:18:17.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.263+0000 7f5443916700 1 -- 172.21.15.88:0/1385108134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543c107e40 con 0x7f543c100d70 2024-02-19T19:18:17.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.264+0000 7f54327fc700 1 -- 172.21.15.88:0/1385108134 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f542c010380 con 0x7f543c100d70 2024-02-19T19:18:17.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.264+0000 7f54327fc700 1 --2- 172.21.15.88:0/1385108134 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542806d8a0 0x7f542806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:17.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.264+0000 7f54327fc700 1 -- 172.21.15.88:0/1385108134 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f542c08d070 con 0x7f543c100d70 2024-02-19T19:18:17.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.264+0000 7f54416b2700 1 --2- 172.21.15.88:0/1385108134 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542806d8a0 0x7f542806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:17.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.264+0000 7f54416b2700 1 --2- 172.21.15.88:0/1385108134 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542806d8a0 0x7f542806fd50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5438005f50 tx=0x7f543801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:17.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.269+0000 7f54327fc700 1 -- 172.21.15.88:0/1385108134 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f542c056820 con 0x7f543c100d70 2024-02-19T19:18:17.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.482+0000 7f5443916700 1 -- 172.21.15.88:0/1385108134 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f543c19b400 con 0x7f542806d8a0 2024-02-19T19:18:17.497 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:17 smithi088 ceph-mon[88643]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:17.497 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:17 smithi088 ceph-mon[88643]: from='client.34838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:17.498 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:17 smithi088 ceph-mon[88643]: pgmap v440: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:17.498 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:17 smithi088 ceph-mon[88643]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:17.498 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:17 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2867154383' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:18:17.498 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:17 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2874656529' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:18:17.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.484+0000 7f54327fc700 1 -- 172.21.15.88:0/1385108134 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f543c19b400 con 0x7f542806d8a0 2024-02-19T19:18:17.498 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:18:17.499 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:18:17.500 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:18:17.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.486+0000 7f5443916700 1 -- 172.21.15.88:0/1385108134 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542806d8a0 msgr2=0x7f542806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:17.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.486+0000 7f5443916700 1 --2- 172.21.15.88:0/1385108134 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542806d8a0 0x7f542806fd50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5438005f50 tx=0x7f543801a040 comp rx=0 tx=0).stop 2024-02-19T19:18:17.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.486+0000 7f5443916700 1 -- 172.21.15.88:0/1385108134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f543c100d70 msgr2=0x7f543c1964e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:17.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.486+0000 7f5443916700 1 --2- 172.21.15.88:0/1385108134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f543c100d70 0x7f543c1964e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f542c00b4b0 tx=0x7f542c00b7c0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.486+0000 7f5443916700 1 -- 172.21.15.88:0/1385108134 shutdown_connections 2024-02-19T19:18:17.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.486+0000 7f5443916700 1 --2- 172.21.15.88:0/1385108134 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542806d8a0 0x7f542806fd50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.486+0000 7f5443916700 1 --2- 172.21.15.88:0/1385108134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f543c100460 0x7f543c195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.486+0000 7f5443916700 1 --2- 172.21.15.88:0/1385108134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f543c100d70 0x7f543c1964e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.486+0000 7f5443916700 1 -- 172.21.15.88:0/1385108134 >> 172.21.15.88:0/1385108134 conn(0x7f543c0748b0 msgr2=0x7f543c0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:17.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.486+0000 7f5443916700 1 -- 172.21.15.88:0/1385108134 shutdown_connections 2024-02-19T19:18:17.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.486+0000 7f5443916700 1 -- 172.21.15.88:0/1385108134 wait complete. 2024-02-19T19:18:17.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.592+0000 7fa217959700 1 -- 172.21.15.88:0/1159747643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2101004c0 msgr2=0x7fa210100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:17.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.592+0000 7fa217959700 1 --2- 172.21.15.88:0/1159747643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2101004c0 0x7fa210100890 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa20000b0a0 tx=0x7fa20000b3b0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.592+0000 7fa217959700 1 -- 172.21.15.88:0/1159747643 shutdown_connections 2024-02-19T19:18:17.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.592+0000 7fa217959700 1 --2- 172.21.15.88:0/1159747643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa210100e60 0x7fa210104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.592+0000 7fa217959700 1 --2- 172.21.15.88:0/1159747643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2101004c0 0x7fa210100890 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.592+0000 7fa217959700 1 -- 172.21.15.88:0/1159747643 >> 172.21.15.88:0/1159747643 conn(0x7fa2100fbd80 msgr2=0x7fa2100fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:17.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.592+0000 7fa217959700 1 -- 172.21.15.88:0/1159747643 shutdown_connections 2024-02-19T19:18:17.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.592+0000 7fa217959700 1 -- 172.21.15.88:0/1159747643 wait complete. 2024-02-19T19:18:17.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa217959700 1 Processor -- start 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa217959700 1 -- start start 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa217959700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2101004c0 0x7fa210195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa217959700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa210100e60 0x7fa210196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa217959700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa210196b50 con 0x7fa210100e60 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa217959700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21019a8e0 con 0x7fa2101004c0 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa214ef4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa210100e60 0x7fa210196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa214ef4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa210100e60 0x7fa210196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49484/0 (socket says 172.21.15.88:49484) 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa214ef4700 1 -- 172.21.15.88:0/401857351 learned_addr learned my addr 172.21.15.88:0/401857351 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa214ef4700 1 -- 172.21.15.88:0/401857351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2101004c0 msgr2=0x7fa210195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa214ef4700 1 --2- 172.21.15.88:0/401857351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2101004c0 0x7fa210195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.593+0000 7fa214ef4700 1 -- 172.21.15.88:0/401857351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa200009d00 con 0x7fa210100e60 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.594+0000 7fa214ef4700 1 --2- 172.21.15.88:0/401857351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa210100e60 0x7fa210196470 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa20400c8f0 tx=0x7fa20400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:17.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.594+0000 7fa20e7fc700 1 -- 172.21.15.88:0/401857351 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa204007a50 con 0x7fa210100e60 2024-02-19T19:18:17.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.594+0000 7fa217959700 1 -- 172.21.15.88:0/401857351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa21019abc0 con 0x7fa210100e60 2024-02-19T19:18:17.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.594+0000 7fa20e7fc700 1 -- 172.21.15.88:0/401857351 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa204007bb0 con 0x7fa210100e60 2024-02-19T19:18:17.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.594+0000 7fa20e7fc700 1 -- 172.21.15.88:0/401857351 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa204018630 con 0x7fa210100e60 2024-02-19T19:18:17.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.594+0000 7fa217959700 1 -- 172.21.15.88:0/401857351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa21019b110 con 0x7fa210100e60 2024-02-19T19:18:17.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.595+0000 7fa217959700 1 -- 172.21.15.88:0/401857351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2101087d0 con 0x7fa210100e60 2024-02-19T19:18:17.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.595+0000 7fa20e7fc700 1 -- 172.21.15.88:0/401857351 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa204007d20 con 0x7fa210100e60 2024-02-19T19:18:17.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.596+0000 7fa20e7fc700 1 --2- 172.21.15.88:0/401857351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1f806d8a0 0x7fa1f806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:17.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.596+0000 7fa20e7fc700 1 -- 172.21.15.88:0/401857351 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa20408d510 con 0x7fa210100e60 2024-02-19T19:18:17.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.596+0000 7fa2156f5700 1 --2- 172.21.15.88:0/401857351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1f806d8a0 0x7fa1f806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:17.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.596+0000 7fa2156f5700 1 --2- 172.21.15.88:0/401857351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1f806d8a0 0x7fa1f806fd50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa200009480 tx=0x7fa200006000 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:17.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.600+0000 7fa20e7fc700 1 -- 172.21.15.88:0/401857351 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa204056cc0 con 0x7fa210100e60 2024-02-19T19:18:17.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:17 smithi132 ceph-mon[70676]: from='client.34834 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:17.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:17 smithi132 ceph-mon[70676]: from='client.34838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:17.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:17 smithi132 ceph-mon[70676]: pgmap v440: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:17.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:17 smithi132 ceph-mon[70676]: from='client.34842 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:17.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:17 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2867154383' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:18:17.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:17 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2874656529' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:18:17.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.887+0000 7fa217959700 1 -- 172.21.15.88:0/401857351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa21019b390 con 0x7fa210100e60 2024-02-19T19:18:17.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.887+0000 7fa20e7fc700 1 -- 172.21.15.88:0/401857351 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa20405a380 con 0x7fa210100e60 2024-02-19T19:18:17.902 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:18:17.902 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:18:17.902 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:18:17.902 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:18:17.902 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:18:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.889+0000 7fa217959700 1 -- 172.21.15.88:0/401857351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1f806d8a0 msgr2=0x7fa1f806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.889+0000 7fa217959700 1 --2- 172.21.15.88:0/401857351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1f806d8a0 0x7fa1f806fd50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa200009480 tx=0x7fa200006000 comp rx=0 tx=0).stop 2024-02-19T19:18:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.889+0000 7fa217959700 1 -- 172.21.15.88:0/401857351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa210100e60 msgr2=0x7fa210196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.889+0000 7fa217959700 1 --2- 172.21.15.88:0/401857351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa210100e60 0x7fa210196470 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa20400c8f0 tx=0x7fa20400ccb0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.889+0000 7fa217959700 1 -- 172.21.15.88:0/401857351 shutdown_connections 2024-02-19T19:18:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.889+0000 7fa217959700 1 --2- 172.21.15.88:0/401857351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1f806d8a0 0x7fa1f806fd50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.889+0000 7fa217959700 1 --2- 172.21.15.88:0/401857351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa2101004c0 0x7fa210195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.889+0000 7fa217959700 1 --2- 172.21.15.88:0/401857351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa210100e60 0x7fa210196470 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.889+0000 7fa217959700 1 -- 172.21.15.88:0/401857351 >> 172.21.15.88:0/401857351 conn(0x7fa2100fbd80 msgr2=0x7fa2100fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.889+0000 7fa217959700 1 -- 172.21.15.88:0/401857351 shutdown_connections 2024-02-19T19:18:17.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:17.889+0000 7fa217959700 1 -- 172.21.15.88:0/401857351 wait complete. 2024-02-19T19:18:18.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:18 smithi132 ceph-mon[70676]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:18.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:18 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/401857351' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:18:18.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:18 smithi088 ceph-mon[88643]: from='client.34854 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:18.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:18 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/401857351' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:18:19.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:19 smithi132 ceph-mon[70676]: pgmap v441: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:19.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:19 smithi088 ceph-mon[88643]: pgmap v441: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:21.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:21 smithi132 ceph-mon[70676]: pgmap v442: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:21.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:21 smithi088 ceph-mon[88643]: pgmap v442: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:23.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:23 smithi132 ceph-mon[70676]: pgmap v443: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:23.845 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:23 smithi088 ceph-mon[88643]: pgmap v443: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:25.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:25 smithi132 ceph-mon[70676]: pgmap v444: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:25.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:25 smithi088 ceph-mon[88643]: pgmap v444: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:27.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:27 smithi132 ceph-mon[70676]: pgmap v445: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:27.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:27 smithi088 ceph-mon[88643]: pgmap v445: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:28.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:18:28.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:18:29.747 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:29 smithi132 ceph-mon[70676]: pgmap v446: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:29.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:29 smithi088 ceph-mon[88643]: pgmap v446: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:31.496 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:31 smithi132 ceph-mon[70676]: pgmap v447: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:31.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:31 smithi088 ceph-mon[88643]: pgmap v447: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:33.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:33 smithi132 ceph-mon[70676]: pgmap v448: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:33.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:33 smithi088 ceph-mon[88643]: pgmap v448: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:35.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:35 smithi132 ceph-mon[70676]: pgmap v449: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:35.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:35 smithi088 ceph-mon[88643]: pgmap v449: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:37.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:37 smithi132 ceph-mon[70676]: pgmap v450: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:37.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:37 smithi088 ceph-mon[88643]: pgmap v450: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:39.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:39 smithi132 ceph-mon[70676]: pgmap v451: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:39.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:39 smithi088 ceph-mon[88643]: pgmap v451: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:41.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:41 smithi132 ceph-mon[70676]: pgmap v452: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:41.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:41 smithi088 ceph-mon[88643]: pgmap v452: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:43.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:43 smithi132 ceph-mon[70676]: pgmap v453: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:43.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:18:43.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:43 smithi088 ceph-mon[88643]: pgmap v453: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:43.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:18:45.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:45 smithi132 ceph-mon[70676]: pgmap v454: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:45.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:45 smithi088 ceph-mon[88643]: pgmap v454: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:47.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:47 smithi132 ceph-mon[70676]: pgmap v455: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:47.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:47 smithi088 ceph-mon[88643]: pgmap v455: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:48.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.006+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/2990469494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b980ff3b0 msgr2=0x7f1b980ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.006+0000 7f1b9dc9a700 1 --2- 172.21.15.88:0/2990469494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b980ff3b0 0x7f1b980ff780 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1b88009910 tx=0x7f1b88009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:48.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.006+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/2990469494 shutdown_connections 2024-02-19T19:18:48.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.006+0000 7f1b9dc9a700 1 --2- 172.21.15.88:0/2990469494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b980ffd50 0x7f1b98103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.006+0000 7f1b9dc9a700 1 --2- 172.21.15.88:0/2990469494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b980ff3b0 0x7f1b980ff780 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.006+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/2990469494 >> 172.21.15.88:0/2990469494 conn(0x7f1b980fac90 msgr2=0x7f1b980fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:48.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.007+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/2990469494 shutdown_connections 2024-02-19T19:18:48.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.007+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/2990469494 wait complete. 2024-02-19T19:18:48.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.007+0000 7f1b9dc9a700 1 Processor -- start 2024-02-19T19:18:48.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.007+0000 7f1b9dc9a700 1 -- start start 2024-02-19T19:18:48.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b9dc9a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b980ff3b0 0x7f1b98195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b9dc9a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b980ffd50 0x7f1b98196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b9dc9a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b98196b70 con 0x7f1b980ffd50 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b9dc9a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9819a900 con 0x7f1b980ff3b0 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b96ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b980ffd50 0x7f1b98196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b96ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b980ffd50 0x7f1b98196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39462/0 (socket says 172.21.15.88:39462) 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b96ffd700 1 -- 172.21.15.88:0/4235341912 learned_addr learned my addr 172.21.15.88:0/4235341912 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b977fe700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b980ff3b0 0x7f1b98195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b96ffd700 1 -- 172.21.15.88:0/4235341912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b980ff3b0 msgr2=0x7f1b98195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b96ffd700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b980ff3b0 0x7f1b98195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b96ffd700 1 -- 172.21.15.88:0/4235341912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b880095f0 con 0x7f1b980ffd50 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b977fe700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b980ff3b0 0x7f1b98195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b96ffd700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b980ffd50 0x7f1b98196490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b480 tx=0x7f1b8c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:48.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b94ff9700 1 -- 172.21.15.88:0/4235341912 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b8c0118c0 con 0x7f1b980ffd50 2024-02-19T19:18:48.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/4235341912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9819abe0 con 0x7f1b980ffd50 2024-02-19T19:18:48.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b94ff9700 1 -- 172.21.15.88:0/4235341912 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b8c010380 con 0x7f1b980ffd50 2024-02-19T19:18:48.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b94ff9700 1 -- 172.21.15.88:0/4235341912 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b8c00f4f0 con 0x7f1b980ffd50 2024-02-19T19:18:48.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.008+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/4235341912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9819b130 con 0x7f1b980ffd50 2024-02-19T19:18:48.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.010+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/4235341912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9804f2a0 con 0x7f1b980ffd50 2024-02-19T19:18:48.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.010+0000 7f1b94ff9700 1 -- 172.21.15.88:0/4235341912 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1b8c0104f0 con 0x7f1b980ffd50 2024-02-19T19:18:48.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.010+0000 7f1b94ff9700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b8006d8a0 0x7f1b8006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:48.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.011+0000 7f1b94ff9700 1 -- 172.21.15.88:0/4235341912 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1b8c08d180 con 0x7f1b980ffd50 2024-02-19T19:18:48.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.011+0000 7f1b977fe700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b8006d8a0 0x7f1b8006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:48.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.011+0000 7f1b977fe700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b8006d8a0 0x7f1b8006fd50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800b5c0 tx=0x7f1b88005480 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:48.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.015+0000 7f1b94ff9700 1 -- 172.21.15.88:0/4235341912 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b8c056930 con 0x7f1b980ffd50 2024-02-19T19:18:48.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.241+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/4235341912 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b9819b490 con 0x7f1b8006d8a0 2024-02-19T19:18:48.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.242+0000 7f1b94ff9700 1 -- 172.21.15.88:0/4235341912 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1b9819b490 con 0x7f1b8006d8a0 2024-02-19T19:18:48.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.245+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/4235341912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b8006d8a0 msgr2=0x7f1b8006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.245+0000 7f1b9dc9a700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b8006d8a0 0x7f1b8006fd50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1b8800b5c0 tx=0x7f1b88005480 comp rx=0 tx=0).stop 2024-02-19T19:18:48.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.245+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/4235341912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b980ffd50 msgr2=0x7f1b98196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.245+0000 7f1b9dc9a700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b980ffd50 0x7f1b98196490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b480 tx=0x7f1b8c00b840 comp rx=0 tx=0).stop 2024-02-19T19:18:48.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.245+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/4235341912 shutdown_connections 2024-02-19T19:18:48.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.245+0000 7f1b9dc9a700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b8006d8a0 0x7f1b8006fd50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.245+0000 7f1b9dc9a700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b980ff3b0 0x7f1b98195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.245+0000 7f1b9dc9a700 1 --2- 172.21.15.88:0/4235341912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b980ffd50 0x7f1b98196490 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.245+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/4235341912 >> 172.21.15.88:0/4235341912 conn(0x7f1b980fac90 msgr2=0x7f1b98103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:48.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.245+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/4235341912 shutdown_connections 2024-02-19T19:18:48.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.245+0000 7f1b9dc9a700 1 -- 172.21.15.88:0/4235341912 wait complete. 2024-02-19T19:18:48.269 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:18:48.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.351+0000 7f9086b2d700 1 -- 172.21.15.88:0/2607418453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9080100e80 msgr2=0x7f9080104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.351+0000 7f9086b2d700 1 --2- 172.21.15.88:0/2607418453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9080100e80 0x7f9080104e50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9070009910 tx=0x7f9070009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:48.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.351+0000 7f9086b2d700 1 -- 172.21.15.88:0/2607418453 shutdown_connections 2024-02-19T19:18:48.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.351+0000 7f9086b2d700 1 --2- 172.21.15.88:0/2607418453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9080100e80 0x7f9080104e50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.351+0000 7f9086b2d700 1 --2- 172.21.15.88:0/2607418453 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90801004e0 0x7f90801008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.351+0000 7f9086b2d700 1 -- 172.21.15.88:0/2607418453 >> 172.21.15.88:0/2607418453 conn(0x7f90800fbda0 msgr2=0x7f90800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:48.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.351+0000 7f9086b2d700 1 -- 172.21.15.88:0/2607418453 shutdown_connections 2024-02-19T19:18:48.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.351+0000 7f9086b2d700 1 -- 172.21.15.88:0/2607418453 wait complete. 2024-02-19T19:18:48.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.351+0000 7f9086b2d700 1 Processor -- start 2024-02-19T19:18:48.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.351+0000 7f9086b2d700 1 -- start start 2024-02-19T19:18:48.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f9086b2d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90801004e0 0x7f9080195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:48.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f9086b2d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9080100e80 0x7f9080196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:48.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f9086b2d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9080196aa0 con 0x7f90801004e0 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f9086b2d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9080196be0 con 0x7f9080100e80 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f90848c9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90801004e0 0x7f9080195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f90848c9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90801004e0 0x7f9080195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39492/0 (socket says 172.21.15.88:39492) 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f90848c9700 1 -- 172.21.15.88:0/1498654954 learned_addr learned my addr 172.21.15.88:0/1498654954 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f90848c9700 1 -- 172.21.15.88:0/1498654954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9080100e80 msgr2=0x7f9080196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f90848c9700 1 --2- 172.21.15.88:0/1498654954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9080100e80 0x7f9080196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f90848c9700 1 -- 172.21.15.88:0/1498654954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700095f0 con 0x7f90801004e0 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f90848c9700 1 --2- 172.21.15.88:0/1498654954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90801004e0 0x7f9080195f10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f907c00c930 tx=0x7f907c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f9075ffb700 1 -- 172.21.15.88:0/1498654954 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f907c007ad0 con 0x7f90801004e0 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f9086b2d700 1 -- 172.21.15.88:0/1498654954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f908019aa30 con 0x7f90801004e0 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f9075ffb700 1 -- 172.21.15.88:0/1498654954 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f907c007c30 con 0x7f90801004e0 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.352+0000 7f9075ffb700 1 -- 172.21.15.88:0/1498654954 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f907c018610 con 0x7f90801004e0 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.353+0000 7f9086b2d700 1 -- 172.21.15.88:0/1498654954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f908019af80 con 0x7f90801004e0 2024-02-19T19:18:48.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.354+0000 7f9086b2d700 1 -- 172.21.15.88:0/1498654954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9080108850 con 0x7f90801004e0 2024-02-19T19:18:48.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.354+0000 7f9075ffb700 1 -- 172.21.15.88:0/1498654954 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f907c00f450 con 0x7f90801004e0 2024-02-19T19:18:48.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.354+0000 7f9075ffb700 1 --2- 172.21.15.88:0/1498654954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f906806d8a0 0x7f906806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:48.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.354+0000 7f9075ffb700 1 -- 172.21.15.88:0/1498654954 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f907c08d780 con 0x7f90801004e0 2024-02-19T19:18:48.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.354+0000 7f9077fff700 1 --2- 172.21.15.88:0/1498654954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f906806d8a0 0x7f906806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:48.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.355+0000 7f9077fff700 1 --2- 172.21.15.88:0/1498654954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f906806d8a0 0x7f906806fd50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f907000b5c0 tx=0x7f9070005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:48.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.359+0000 7f9075ffb700 1 -- 172.21.15.88:0/1498654954 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f907c056f30 con 0x7f90801004e0 2024-02-19T19:18:48.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.576+0000 7f9086b2d700 1 -- 172.21.15.88:0/1498654954 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f908019b200 con 0x7f906806d8a0 2024-02-19T19:18:48.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.577+0000 7f9075ffb700 1 -- 172.21.15.88:0/1498654954 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f908019b200 con 0x7f906806d8a0 2024-02-19T19:18:48.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.579+0000 7f9086b2d700 1 -- 172.21.15.88:0/1498654954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f906806d8a0 msgr2=0x7f906806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.579+0000 7f9086b2d700 1 --2- 172.21.15.88:0/1498654954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f906806d8a0 0x7f906806fd50 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f907000b5c0 tx=0x7f9070005960 comp rx=0 tx=0).stop 2024-02-19T19:18:48.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.579+0000 7f9086b2d700 1 -- 172.21.15.88:0/1498654954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90801004e0 msgr2=0x7f9080195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.579+0000 7f9086b2d700 1 --2- 172.21.15.88:0/1498654954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90801004e0 0x7f9080195f10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f907c00c930 tx=0x7f907c00ccf0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.579+0000 7f9086b2d700 1 -- 172.21.15.88:0/1498654954 shutdown_connections 2024-02-19T19:18:48.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.579+0000 7f9086b2d700 1 --2- 172.21.15.88:0/1498654954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f906806d8a0 0x7f906806fd50 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.579+0000 7f9086b2d700 1 --2- 172.21.15.88:0/1498654954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90801004e0 0x7f9080195f10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.579+0000 7f9086b2d700 1 --2- 172.21.15.88:0/1498654954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9080100e80 0x7f9080196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.579+0000 7f9086b2d700 1 -- 172.21.15.88:0/1498654954 >> 172.21.15.88:0/1498654954 conn(0x7f90800fbda0 msgr2=0x7f90800fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:48.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.579+0000 7f9086b2d700 1 -- 172.21.15.88:0/1498654954 shutdown_connections 2024-02-19T19:18:48.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.579+0000 7f9086b2d700 1 -- 172.21.15.88:0/1498654954 wait complete. 2024-02-19T19:18:48.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.684+0000 7fc423b0f700 1 -- 172.21.15.88:0/3255453526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc41c1004e0 msgr2=0x7fc41c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.684+0000 7fc423b0f700 1 --2- 172.21.15.88:0/3255453526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc41c1004e0 0x7fc41c1008b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009910 tx=0x7fc40c009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:48.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.684+0000 7fc423b0f700 1 -- 172.21.15.88:0/3255453526 shutdown_connections 2024-02-19T19:18:48.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.684+0000 7fc423b0f700 1 --2- 172.21.15.88:0/3255453526 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc41c100e80 0x7fc41c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.684+0000 7fc423b0f700 1 --2- 172.21.15.88:0/3255453526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc41c1004e0 0x7fc41c1008b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.684+0000 7fc423b0f700 1 -- 172.21.15.88:0/3255453526 >> 172.21.15.88:0/3255453526 conn(0x7fc41c0fbda0 msgr2=0x7fc41c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:48.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.684+0000 7fc423b0f700 1 -- 172.21.15.88:0/3255453526 shutdown_connections 2024-02-19T19:18:48.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.684+0000 7fc423b0f700 1 -- 172.21.15.88:0/3255453526 wait complete. 2024-02-19T19:18:48.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc423b0f700 1 Processor -- start 2024-02-19T19:18:48.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc423b0f700 1 -- start start 2024-02-19T19:18:48.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc423b0f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc41c1004e0 0x7fc41c1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:48.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc423b0f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc41c100e80 0x7fc41c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc423b0f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c198e10 con 0x7fc41c100e80 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc423b0f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c19cba0 con 0x7fc41c1004e0 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc4210aa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc41c100e80 0x7fc41c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc4210aa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc41c100e80 0x7fc41c198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39510/0 (socket says 172.21.15.88:39510) 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc4210aa700 1 -- 172.21.15.88:0/3920687681 learned_addr learned my addr 172.21.15.88:0/3920687681 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc4218ab700 1 --2- 172.21.15.88:0/3920687681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc41c1004e0 0x7fc41c1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc4210aa700 1 -- 172.21.15.88:0/3920687681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc41c1004e0 msgr2=0x7fc41c1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc4210aa700 1 --2- 172.21.15.88:0/3920687681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc41c1004e0 0x7fc41c1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.685+0000 7fc4210aa700 1 -- 172.21.15.88:0/3920687681 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c0095f0 con 0x7fc41c100e80 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.686+0000 7fc4210aa700 1 --2- 172.21.15.88:0/3920687681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc41c100e80 0x7fc41c198730 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc41800b480 tx=0x7fc41800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.686+0000 7fc412ffd700 1 -- 172.21.15.88:0/3920687681 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4180118c0 con 0x7fc41c100e80 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.686+0000 7fc423b0f700 1 -- 172.21.15.88:0/3920687681 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41c19ce80 con 0x7fc41c100e80 2024-02-19T19:18:48.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.686+0000 7fc412ffd700 1 -- 172.21.15.88:0/3920687681 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc418010380 con 0x7fc41c100e80 2024-02-19T19:18:48.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.686+0000 7fc412ffd700 1 -- 172.21.15.88:0/3920687681 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc41800f4f0 con 0x7fc41c100e80 2024-02-19T19:18:48.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.686+0000 7fc423b0f700 1 -- 172.21.15.88:0/3920687681 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41c19d3d0 con 0x7fc41c100e80 2024-02-19T19:18:48.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.687+0000 7fc423b0f700 1 -- 172.21.15.88:0/3920687681 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41c1045d0 con 0x7fc41c100e80 2024-02-19T19:18:48.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.688+0000 7fc412ffd700 1 -- 172.21.15.88:0/3920687681 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc4180104f0 con 0x7fc41c100e80 2024-02-19T19:18:48.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.688+0000 7fc412ffd700 1 --2- 172.21.15.88:0/3920687681 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc40806d8a0 0x7fc40806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:48.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.688+0000 7fc412ffd700 1 -- 172.21.15.88:0/3920687681 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc41808d0a0 con 0x7fc41c100e80 2024-02-19T19:18:48.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.688+0000 7fc4218ab700 1 --2- 172.21.15.88:0/3920687681 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc40806d8a0 0x7fc40806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:48.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.689+0000 7fc4218ab700 1 --2- 172.21.15.88:0/3920687681 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc40806d8a0 0x7fc40806fd50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00b5c0 tx=0x7fc40c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:48.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.693+0000 7fc412ffd700 1 -- 172.21.15.88:0/3920687681 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc4180568d0 con 0x7fc41c100e80 2024-02-19T19:18:48.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.900+0000 7fc423b0f700 1 -- 172.21.15.88:0/3920687681 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc41c19d700 con 0x7fc40806d8a0 2024-02-19T19:18:48.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.910+0000 7fc412ffd700 1 -- 172.21.15.88:0/3920687681 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc41c19d700 con 0x7fc40806d8a0 2024-02-19T19:18:48.924 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (15m) 7m ago 26m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (26m) 7m ago 26m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (8m) 7m ago 24m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (12m) 7m ago 26m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (11m) 7m ago 24m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (15m) 7m ago 25m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (8m) 7m ago 20m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (7m) 7m ago 20m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (7m) 7m ago 20m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (20m) 7m ago 20m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (17m) 7m ago 28m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (16m) 7m ago 24m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (14m) 7m ago 28m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:18:48.925 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (13m) 7m ago 24m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:18:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (16m) 7m ago 26m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:18:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (16m) 7m ago 24m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:18:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (11m) 7m ago 23m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:18:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (10m) 7m ago 23m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:18:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (10m) 7m ago 22m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:18:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (9m) 7m ago 22m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:18:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (9m) 7m ago 22m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:18:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (8m) 7m ago 21m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:18:48.926 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (16m) 7m ago 25m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:18:48.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.913+0000 7fc423b0f700 1 -- 172.21.15.88:0/3920687681 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc40806d8a0 msgr2=0x7fc40806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.913+0000 7fc423b0f700 1 --2- 172.21.15.88:0/3920687681 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc40806d8a0 0x7fc40806fd50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00b5c0 tx=0x7fc40c01a040 comp rx=0 tx=0).stop 2024-02-19T19:18:48.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.913+0000 7fc423b0f700 1 -- 172.21.15.88:0/3920687681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc41c100e80 msgr2=0x7fc41c198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:48.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.913+0000 7fc423b0f700 1 --2- 172.21.15.88:0/3920687681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc41c100e80 0x7fc41c198730 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc41800b480 tx=0x7fc41800b840 comp rx=0 tx=0).stop 2024-02-19T19:18:48.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.913+0000 7fc423b0f700 1 -- 172.21.15.88:0/3920687681 shutdown_connections 2024-02-19T19:18:48.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.913+0000 7fc423b0f700 1 --2- 172.21.15.88:0/3920687681 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc40806d8a0 0x7fc40806fd50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.913+0000 7fc423b0f700 1 --2- 172.21.15.88:0/3920687681 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc41c1004e0 0x7fc41c1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.913+0000 7fc423b0f700 1 --2- 172.21.15.88:0/3920687681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc41c100e80 0x7fc41c198730 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:48.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.913+0000 7fc423b0f700 1 -- 172.21.15.88:0/3920687681 >> 172.21.15.88:0/3920687681 conn(0x7fc41c0fbda0 msgr2=0x7fc41c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:48.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.913+0000 7fc423b0f700 1 -- 172.21.15.88:0/3920687681 shutdown_connections 2024-02-19T19:18:48.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:48.913+0000 7fc423b0f700 1 -- 172.21.15.88:0/3920687681 wait complete. 2024-02-19T19:18:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.015+0000 7f25a217a700 1 -- 172.21.15.88:0/474755212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c100e80 msgr2=0x7f259c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.015+0000 7f25a217a700 1 --2- 172.21.15.88:0/474755212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c100e80 0x7f259c104e50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2590009910 tx=0x7f2590009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.015+0000 7f25a217a700 1 -- 172.21.15.88:0/474755212 shutdown_connections 2024-02-19T19:18:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.015+0000 7f25a217a700 1 --2- 172.21.15.88:0/474755212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c100e80 0x7f259c104e50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.015+0000 7f25a217a700 1 --2- 172.21.15.88:0/474755212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c1004e0 0x7f259c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.015+0000 7f25a217a700 1 -- 172.21.15.88:0/474755212 >> 172.21.15.88:0/474755212 conn(0x7f259c0fbda0 msgr2=0x7f259c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.015+0000 7f25a217a700 1 -- 172.21.15.88:0/474755212 shutdown_connections 2024-02-19T19:18:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.015+0000 7f25a217a700 1 -- 172.21.15.88:0/474755212 wait complete. 2024-02-19T19:18:49.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.016+0000 7f25a217a700 1 Processor -- start 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.016+0000 7f25a217a700 1 -- start start 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.016+0000 7f25a217a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c1004e0 0x7f259c195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.016+0000 7f25a217a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c100e80 0x7f259c1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.016+0000 7f25a217a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c196ab0 con 0x7f259c1004e0 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.016+0000 7f25a217a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c19a840 con 0x7f259c100e80 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.016+0000 7f259b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c1004e0 0x7f259c195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.016+0000 7f259b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c1004e0 0x7f259c195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39534/0 (socket says 172.21.15.88:39534) 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.016+0000 7f259b7fe700 1 -- 172.21.15.88:0/844818724 learned_addr learned my addr 172.21.15.88:0/844818724 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.016+0000 7f259affd700 1 --2- 172.21.15.88:0/844818724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c100e80 0x7f259c1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.017+0000 7f259b7fe700 1 -- 172.21.15.88:0/844818724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c100e80 msgr2=0x7f259c1963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.017+0000 7f259b7fe700 1 --2- 172.21.15.88:0/844818724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c100e80 0x7f259c1963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.017+0000 7f259b7fe700 1 -- 172.21.15.88:0/844818724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25900095f0 con 0x7f259c1004e0 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.017+0000 7f259b7fe700 1 --2- 172.21.15.88:0/844818724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c1004e0 0x7f259c195e90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f258c007e90 tx=0x7f258c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.017+0000 7f2598ff9700 1 -- 172.21.15.88:0/844818724 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f258c017070 con 0x7f259c1004e0 2024-02-19T19:18:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.017+0000 7f25a217a700 1 -- 172.21.15.88:0/844818724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c19ab20 con 0x7f259c1004e0 2024-02-19T19:18:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.017+0000 7f2598ff9700 1 -- 172.21.15.88:0/844818724 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f258c00f040 con 0x7f259c1004e0 2024-02-19T19:18:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.017+0000 7f2598ff9700 1 -- 172.21.15.88:0/844818724 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f258c005890 con 0x7f259c1004e0 2024-02-19T19:18:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.017+0000 7f25a217a700 1 -- 172.21.15.88:0/844818724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c19b070 con 0x7f259c1004e0 2024-02-19T19:18:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.018+0000 7f25a217a700 1 -- 172.21.15.88:0/844818724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c108760 con 0x7f259c1004e0 2024-02-19T19:18:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.018+0000 7f2598ff9700 1 -- 172.21.15.88:0/844818724 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f258c004c40 con 0x7f259c1004e0 2024-02-19T19:18:49.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.019+0000 7f2598ff9700 1 --2- 172.21.15.88:0/844818724 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f258406d8a0 0x7f258406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:49.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.019+0000 7f2598ff9700 1 -- 172.21.15.88:0/844818724 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f258c08ce90 con 0x7f259c1004e0 2024-02-19T19:18:49.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.019+0000 7f259affd700 1 --2- 172.21.15.88:0/844818724 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f258406d8a0 0x7f258406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:49.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.019+0000 7f259affd700 1 --2- 172.21.15.88:0/844818724 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f258406d8a0 0x7f258406fd50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f259c1974b0 tx=0x7f259001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:49.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.024+0000 7f2598ff9700 1 -- 172.21.15.88:0/844818724 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f258c0566c0 con 0x7f259c1004e0 2024-02-19T19:18:49.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.307+0000 7f25a217a700 1 -- 172.21.15.88:0/844818724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f259c19b2e0 con 0x7f259c1004e0 2024-02-19T19:18:49.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.307+0000 7f2598ff9700 1 -- 172.21.15.88:0/844818724 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f258c059d80 con 0x7f259c1004e0 2024-02-19T19:18:49.322 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:18:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:18:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:18:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:18:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:18:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:18:49.322 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:18:49.323 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:18:49.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.309+0000 7f25a217a700 1 -- 172.21.15.88:0/844818724 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f258406d8a0 msgr2=0x7f258406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:49.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.309+0000 7f25a217a700 1 --2- 172.21.15.88:0/844818724 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f258406d8a0 0x7f258406fd50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f259c1974b0 tx=0x7f259001a040 comp rx=0 tx=0).stop 2024-02-19T19:18:49.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.309+0000 7f25a217a700 1 -- 172.21.15.88:0/844818724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c1004e0 msgr2=0x7f259c195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:49.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.309+0000 7f25a217a700 1 --2- 172.21.15.88:0/844818724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c1004e0 0x7f259c195e90 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f258c007e90 tx=0x7f258c00d3b0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.309+0000 7f25a217a700 1 -- 172.21.15.88:0/844818724 shutdown_connections 2024-02-19T19:18:49.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.309+0000 7f25a217a700 1 --2- 172.21.15.88:0/844818724 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f258406d8a0 0x7f258406fd50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.309+0000 7f25a217a700 1 --2- 172.21.15.88:0/844818724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f259c1004e0 0x7f259c195e90 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.309+0000 7f25a217a700 1 --2- 172.21.15.88:0/844818724 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f259c100e80 0x7f259c1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.309+0000 7f25a217a700 1 -- 172.21.15.88:0/844818724 >> 172.21.15.88:0/844818724 conn(0x7f259c0fbda0 msgr2=0x7f259c1046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:49.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.310+0000 7f25a217a700 1 -- 172.21.15.88:0/844818724 shutdown_connections 2024-02-19T19:18:49.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.310+0000 7f25a217a700 1 -- 172.21.15.88:0/844818724 wait complete. 2024-02-19T19:18:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.414+0000 7f7dcea33700 1 -- 172.21.15.88:0/2800558202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dc8069140 msgr2=0x7f7dc8104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.414+0000 7f7dcea33700 1 --2- 172.21.15.88:0/2800558202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dc8069140 0x7f7dc8104eb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7dbc009910 tx=0x7f7dbc009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.414+0000 7f7dcea33700 1 -- 172.21.15.88:0/2800558202 shutdown_connections 2024-02-19T19:18:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.414+0000 7f7dcea33700 1 --2- 172.21.15.88:0/2800558202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dc8069140 0x7f7dc8104eb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.414+0000 7f7dcea33700 1 --2- 172.21.15.88:0/2800558202 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dc80687a0 0x7f7dc8068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.414+0000 7f7dcea33700 1 -- 172.21.15.88:0/2800558202 >> 172.21.15.88:0/2800558202 conn(0x7f7dc8075130 msgr2=0x7f7dc8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.414+0000 7f7dcea33700 1 -- 172.21.15.88:0/2800558202 shutdown_connections 2024-02-19T19:18:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.414+0000 7f7dcea33700 1 -- 172.21.15.88:0/2800558202 wait complete. 2024-02-19T19:18:49.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.414+0000 7f7dcea33700 1 Processor -- start 2024-02-19T19:18:49.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.414+0000 7f7dcea33700 1 -- start start 2024-02-19T19:18:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dcea33700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dc80687a0 0x7f7dc8195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dcea33700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dc8069140 0x7f7dc81964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dcea33700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8196ba0 con 0x7f7dc80687a0 2024-02-19T19:18:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dcea33700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc819a930 con 0x7f7dc8069140 2024-02-19T19:18:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dc7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dc80687a0 0x7f7dc8195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dc7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dc80687a0 0x7f7dc8195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39538/0 (socket says 172.21.15.88:39538) 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dc7fff700 1 -- 172.21.15.88:0/70416939 learned_addr learned my addr 172.21.15.88:0/70416939 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dc77fe700 1 --2- 172.21.15.88:0/70416939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dc8069140 0x7f7dc81964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dc7fff700 1 -- 172.21.15.88:0/70416939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dc8069140 msgr2=0x7f7dc81964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dc7fff700 1 --2- 172.21.15.88:0/70416939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dc8069140 0x7f7dc81964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dc7fff700 1 -- 172.21.15.88:0/70416939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dbc0095f0 con 0x7f7dc80687a0 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dc7fff700 1 --2- 172.21.15.88:0/70416939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dc80687a0 0x7f7dc8195f80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7db800b4b0 tx=0x7f7db800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dc57fa700 1 -- 172.21.15.88:0/70416939 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db8011800 con 0x7f7dc80687a0 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dc57fa700 1 -- 172.21.15.88:0/70416939 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7db8011e20 con 0x7f7dc80687a0 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dc57fa700 1 -- 172.21.15.88:0/70416939 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db800f2e0 con 0x7f7dc80687a0 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.415+0000 7f7dcea33700 1 -- 172.21.15.88:0/70416939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc819ac10 con 0x7f7dc80687a0 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.416+0000 7f7dcea33700 1 -- 172.21.15.88:0/70416939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc819b160 con 0x7f7dc80687a0 2024-02-19T19:18:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.417+0000 7f7dcea33700 1 -- 172.21.15.88:0/70416939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc8108800 con 0x7f7dc80687a0 2024-02-19T19:18:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.417+0000 7f7dc57fa700 1 -- 172.21.15.88:0/70416939 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7db8010380 con 0x7f7dc80687a0 2024-02-19T19:18:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.417+0000 7f7dc57fa700 1 --2- 172.21.15.88:0/70416939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7db006d8a0 0x7f7db006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.418+0000 7f7dc57fa700 1 -- 172.21.15.88:0/70416939 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7db808d060 con 0x7f7dc80687a0 2024-02-19T19:18:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.418+0000 7f7dc77fe700 1 --2- 172.21.15.88:0/70416939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7db006d8a0 0x7f7db006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.418+0000 7f7dc77fe700 1 --2- 172.21.15.88:0/70416939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7db006d8a0 0x7f7db006fd50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7dc81975a0 tx=0x7f7dbc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:49.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.422+0000 7f7dc57fa700 1 -- 172.21.15.88:0/70416939 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7db8056940 con 0x7f7dc80687a0 2024-02-19T19:18:49.594 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:49 smithi088 ceph-mon[88643]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:49.594 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:49 smithi088 ceph-mon[88643]: pgmap v456: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:49.594 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:49 smithi088 ceph-mon[88643]: from='client.34866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:49.594 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:49 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/844818724' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:18:49.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.670+0000 7f7dcea33700 1 -- 172.21.15.88:0/70416939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7dc804f2a0 con 0x7f7dc80687a0 2024-02-19T19:18:49.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.670+0000 7f7dc57fa700 1 -- 172.21.15.88:0/70416939 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f7db805a000 con 0x7f7dc80687a0 2024-02-19T19:18:49.684 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:18:49.684 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:18:49.684 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:18:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:18:49.686 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:18:49.687 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:18:49.687 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:18:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:18:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:18:49.687 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:18:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:18:49.687 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:18:49.687 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:18:49.687 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:18:49.687 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:18:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.672+0000 7f7dcea33700 1 -- 172.21.15.88:0/70416939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7db006d8a0 msgr2=0x7f7db006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.672+0000 7f7dcea33700 1 --2- 172.21.15.88:0/70416939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7db006d8a0 0x7f7db006fd50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7dc81975a0 tx=0x7f7dbc01a040 comp rx=0 tx=0).stop 2024-02-19T19:18:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.672+0000 7f7dcea33700 1 -- 172.21.15.88:0/70416939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dc80687a0 msgr2=0x7f7dc8195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.672+0000 7f7dcea33700 1 --2- 172.21.15.88:0/70416939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dc80687a0 0x7f7dc8195f80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f7db800b4b0 tx=0x7f7db800b870 comp rx=0 tx=0).stop 2024-02-19T19:18:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.672+0000 7f7dcea33700 1 -- 172.21.15.88:0/70416939 shutdown_connections 2024-02-19T19:18:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.672+0000 7f7dcea33700 1 --2- 172.21.15.88:0/70416939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7db006d8a0 0x7f7db006fd50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.672+0000 7f7dcea33700 1 --2- 172.21.15.88:0/70416939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dc80687a0 0x7f7dc8195f80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.672+0000 7f7dcea33700 1 --2- 172.21.15.88:0/70416939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7dc8069140 0x7f7dc81964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.672+0000 7f7dcea33700 1 -- 172.21.15.88:0/70416939 >> 172.21.15.88:0/70416939 conn(0x7f7dc8075130 msgr2=0x7f7dc80fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:49.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.673+0000 7f7dcea33700 1 -- 172.21.15.88:0/70416939 shutdown_connections 2024-02-19T19:18:49.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.673+0000 7f7dcea33700 1 -- 172.21.15.88:0/70416939 wait complete. 2024-02-19T19:18:49.689 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:18:49.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:49 smithi132 ceph-mon[70676]: from='client.34862 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:49.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:49 smithi132 ceph-mon[70676]: pgmap v456: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:49.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:49 smithi132 ceph-mon[70676]: from='client.34866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:49.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:49 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/844818724' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:18:49.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.778+0000 7f61435c9700 1 -- 172.21.15.88:0/1651932825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f613c100d70 msgr2=0x7f613c1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:49.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.778+0000 7f61435c9700 1 --2- 172.21.15.88:0/1651932825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f613c100d70 0x7f613c1094d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f612c009940 tx=0x7f612c009c50 comp rx=0 tx=0).stop 2024-02-19T19:18:49.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.778+0000 7f61435c9700 1 -- 172.21.15.88:0/1651932825 shutdown_connections 2024-02-19T19:18:49.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.778+0000 7f61435c9700 1 --2- 172.21.15.88:0/1651932825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f613c100d70 0x7f613c1094d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.778+0000 7f61435c9700 1 --2- 172.21.15.88:0/1651932825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c100460 0x7f613c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.778+0000 7f61435c9700 1 -- 172.21.15.88:0/1651932825 >> 172.21.15.88:0/1651932825 conn(0x7f613c0748b0 msgr2=0x7f613c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:49.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.778+0000 7f61435c9700 1 -- 172.21.15.88:0/1651932825 shutdown_connections 2024-02-19T19:18:49.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.778+0000 7f61435c9700 1 -- 172.21.15.88:0/1651932825 wait complete. 2024-02-19T19:18:49.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.778+0000 7f61435c9700 1 Processor -- start 2024-02-19T19:18:49.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f61435c9700 1 -- start start 2024-02-19T19:18:49.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f61435c9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f613c100460 0x7f613c196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:49.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f61435c9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c100d70 0x7f613c1970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:49.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f61435c9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c1976d0 con 0x7f613c100460 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f61435c9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613c197840 con 0x7f613c100d70 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f6141365700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f613c100460 0x7f613c196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f6141365700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f613c100460 0x7f613c196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39560/0 (socket says 172.21.15.88:39560) 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f6141365700 1 -- 172.21.15.88:0/2077616989 learned_addr learned my addr 172.21.15.88:0/2077616989 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f6140b64700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c100d70 0x7f613c1970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f6141365700 1 -- 172.21.15.88:0/2077616989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c100d70 msgr2=0x7f613c1970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f6141365700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c100d70 0x7f613c1970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f6141365700 1 -- 172.21.15.88:0/2077616989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c0095f0 con 0x7f613c100460 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f6140b64700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c100d70 0x7f613c1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f6141365700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f613c100460 0x7f613c196b70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f613800d8d0 tx=0x7f613800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f61327fc700 1 -- 172.21.15.88:0/2077616989 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6138009940 con 0x7f613c100460 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f61435c9700 1 -- 172.21.15.88:0/2077616989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f613c19fbe0 con 0x7f613c100460 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f61327fc700 1 -- 172.21.15.88:0/2077616989 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6138010460 con 0x7f613c100460 2024-02-19T19:18:49.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.779+0000 7f61327fc700 1 -- 172.21.15.88:0/2077616989 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f613800f5f0 con 0x7f613c100460 2024-02-19T19:18:49.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.780+0000 7f61435c9700 1 -- 172.21.15.88:0/2077616989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f613c1a00d0 con 0x7f613c100460 2024-02-19T19:18:49.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.780+0000 7f61435c9700 1 -- 172.21.15.88:0/2077616989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f613c106bd0 con 0x7f613c100460 2024-02-19T19:18:49.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.781+0000 7f61327fc700 1 -- 172.21.15.88:0/2077616989 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f61380105d0 con 0x7f613c100460 2024-02-19T19:18:49.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.781+0000 7f61327fc700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f612806d8a0 0x7f612806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:49.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.782+0000 7f6140b64700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f612806d8a0 0x7f612806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:49.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.782+0000 7f61327fc700 1 -- 172.21.15.88:0/2077616989 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f613808d540 con 0x7f613c100460 2024-02-19T19:18:49.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.782+0000 7f6140b64700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f612806d8a0 0x7f612806fd50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f613c0ffc20 tx=0x7f612c005fb0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:49.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:49.786+0000 7f61327fc700 1 -- 172.21.15.88:0/2077616989 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6138056cf0 con 0x7f613c100460 2024-02-19T19:18:50.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.002+0000 7f61435c9700 1 -- 172.21.15.88:0/2077616989 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f613c02cca0 con 0x7f612806d8a0 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.003+0000 7f61327fc700 1 -- 172.21.15.88:0/2077616989 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f613c02cca0 con 0x7f612806d8a0 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:18:50.017 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:18:50.018 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:18:50.018 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:18:50.018 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:18:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.005+0000 7f61435c9700 1 -- 172.21.15.88:0/2077616989 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f612806d8a0 msgr2=0x7f612806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.005+0000 7f61435c9700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f612806d8a0 0x7f612806fd50 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f613c0ffc20 tx=0x7f612c005fb0 comp rx=0 tx=0).stop 2024-02-19T19:18:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.005+0000 7f61435c9700 1 -- 172.21.15.88:0/2077616989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f613c100460 msgr2=0x7f613c196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.005+0000 7f61435c9700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f613c100460 0x7f613c196b70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f613800d8d0 tx=0x7f613800dc90 comp rx=0 tx=0).stop 2024-02-19T19:18:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.005+0000 7f61435c9700 1 -- 172.21.15.88:0/2077616989 shutdown_connections 2024-02-19T19:18:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.005+0000 7f61435c9700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f612806d8a0 0x7f612806fd50 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.005+0000 7f61435c9700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f613c100460 0x7f613c196b70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:50.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.005+0000 7f61435c9700 1 --2- 172.21.15.88:0/2077616989 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f613c100d70 0x7f613c1970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:50.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.005+0000 7f61435c9700 1 -- 172.21.15.88:0/2077616989 >> 172.21.15.88:0/2077616989 conn(0x7f613c0748b0 msgr2=0x7f613c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:50.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.005+0000 7f61435c9700 1 -- 172.21.15.88:0/2077616989 shutdown_connections 2024-02-19T19:18:50.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.005+0000 7f61435c9700 1 -- 172.21.15.88:0/2077616989 wait complete. 2024-02-19T19:18:50.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.110+0000 7f0a8ba66700 1 -- 172.21.15.88:0/1223368065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a84101420 msgr2=0x7f0a841017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:50.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.110+0000 7f0a8ba66700 1 --2- 172.21.15.88:0/1223368065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a84101420 0x7f0a841017f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009910 tx=0x7f0a80009c20 comp rx=0 tx=0).stop 2024-02-19T19:18:50.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.110+0000 7f0a8ba66700 1 -- 172.21.15.88:0/1223368065 shutdown_connections 2024-02-19T19:18:50.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.110+0000 7f0a8ba66700 1 --2- 172.21.15.88:0/1223368065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a84101d30 0x7f0a8410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:50.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.110+0000 7f0a8ba66700 1 --2- 172.21.15.88:0/1223368065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a84101420 0x7f0a841017f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:50.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.110+0000 7f0a8ba66700 1 -- 172.21.15.88:0/1223368065 >> 172.21.15.88:0/1223368065 conn(0x7f0a840fac80 msgr2=0x7f0a840fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:50.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.110+0000 7f0a8ba66700 1 -- 172.21.15.88:0/1223368065 shutdown_connections 2024-02-19T19:18:50.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.110+0000 7f0a8ba66700 1 -- 172.21.15.88:0/1223368065 wait complete. 2024-02-19T19:18:50.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a8ba66700 1 Processor -- start 2024-02-19T19:18:50.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a8ba66700 1 -- start start 2024-02-19T19:18:50.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a8ba66700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a84101420 0x7f0a84195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a8ba66700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a84101d30 0x7f0a841963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a8ba66700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a84196a90 con 0x7f0a84101420 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a8ba66700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8419a820 con 0x7f0a84101d30 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a89802700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a84101420 0x7f0a84195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a89802700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a84101420 0x7f0a84195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39572/0 (socket says 172.21.15.88:39572) 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a89802700 1 -- 172.21.15.88:0/872369512 learned_addr learned my addr 172.21.15.88:0/872369512 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a89802700 1 -- 172.21.15.88:0/872369512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a84101d30 msgr2=0x7f0a841963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a89802700 1 --2- 172.21.15.88:0/872369512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a84101d30 0x7f0a841963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a89802700 1 -- 172.21.15.88:0/872369512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a800095f0 con 0x7f0a84101420 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.111+0000 7f0a89802700 1 --2- 172.21.15.88:0/872369512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a84101420 0x7f0a84195e70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0a80006010 tx=0x7f0a8000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.112+0000 7f0a7affd700 1 -- 172.21.15.88:0/872369512 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a8001d070 con 0x7f0a84101420 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.112+0000 7f0a7affd700 1 -- 172.21.15.88:0/872369512 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a8000f460 con 0x7f0a84101420 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.112+0000 7f0a7affd700 1 -- 172.21.15.88:0/872369512 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a80017630 con 0x7f0a84101420 2024-02-19T19:18:50.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.112+0000 7f0a8ba66700 1 -- 172.21.15.88:0/872369512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8419aaa0 con 0x7f0a84101420 2024-02-19T19:18:50.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.112+0000 7f0a8ba66700 1 -- 172.21.15.88:0/872369512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8419af90 con 0x7f0a84101420 2024-02-19T19:18:50.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.113+0000 7f0a8ba66700 1 -- 172.21.15.88:0/872369512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a84107900 con 0x7f0a84101420 2024-02-19T19:18:50.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.113+0000 7f0a7affd700 1 -- 172.21.15.88:0/872369512 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0a8000f5d0 con 0x7f0a84101420 2024-02-19T19:18:50.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.113+0000 7f0a7affd700 1 --2- 172.21.15.88:0/872369512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a7006d8a0 0x7f0a7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:18:50.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.114+0000 7f0a7affd700 1 -- 172.21.15.88:0/872369512 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0a8008e980 con 0x7f0a84101420 2024-02-19T19:18:50.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.114+0000 7f0a89001700 1 --2- 172.21.15.88:0/872369512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a7006d8a0 0x7f0a7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:18:50.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.114+0000 7f0a89001700 1 --2- 172.21.15.88:0/872369512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a7006d8a0 0x7f0a7006fd50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a74005950 tx=0x7f0a740058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:18:50.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.118+0000 7f0a7affd700 1 -- 172.21.15.88:0/872369512 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0a800581b0 con 0x7f0a84101420 2024-02-19T19:18:50.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.395+0000 7f0a8ba66700 1 -- 172.21.15.88:0/872369512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0a8404f2a0 con 0x7f0a84101420 2024-02-19T19:18:50.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.395+0000 7f0a7affd700 1 -- 172.21.15.88:0/872369512 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f0a8005b870 con 0x7f0a84101420 2024-02-19T19:18:50.410 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:18:50.410 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:18:50.410 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:18:50.410 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:18:50.410 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:18:50.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.398+0000 7f0a8ba66700 1 -- 172.21.15.88:0/872369512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a7006d8a0 msgr2=0x7f0a7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:50.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.398+0000 7f0a8ba66700 1 --2- 172.21.15.88:0/872369512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a7006d8a0 0x7f0a7006fd50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a74005950 tx=0x7f0a740058e0 comp rx=0 tx=0).stop 2024-02-19T19:18:50.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.398+0000 7f0a8ba66700 1 -- 172.21.15.88:0/872369512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a84101420 msgr2=0x7f0a84195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:18:50.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.398+0000 7f0a8ba66700 1 --2- 172.21.15.88:0/872369512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a84101420 0x7f0a84195e70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0a80006010 tx=0x7f0a8000be20 comp rx=0 tx=0).stop 2024-02-19T19:18:50.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.398+0000 7f0a8ba66700 1 -- 172.21.15.88:0/872369512 shutdown_connections 2024-02-19T19:18:50.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.398+0000 7f0a8ba66700 1 --2- 172.21.15.88:0/872369512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0a7006d8a0 0x7f0a7006fd50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:50.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.398+0000 7f0a8ba66700 1 --2- 172.21.15.88:0/872369512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a84101420 0x7f0a84195e70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:50.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.398+0000 7f0a8ba66700 1 --2- 172.21.15.88:0/872369512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0a84101d30 0x7f0a841963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:18:50.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.398+0000 7f0a8ba66700 1 -- 172.21.15.88:0/872369512 >> 172.21.15.88:0/872369512 conn(0x7f0a840fac80 msgr2=0x7f0a84104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:18:50.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.398+0000 7f0a8ba66700 1 -- 172.21.15.88:0/872369512 shutdown_connections 2024-02-19T19:18:50.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:18:50.398+0000 7f0a8ba66700 1 -- 172.21.15.88:0/872369512 wait complete. 2024-02-19T19:18:50.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:50 smithi132 ceph-mon[70676]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:50.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:50 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/70416939' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:18:50.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:50 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/872369512' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:18:50.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:50 smithi088 ceph-mon[88643]: from='client.34870 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:50.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:50 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/70416939' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:18:50.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:50 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/872369512' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:18:51.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:51 smithi132 ceph-mon[70676]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:51.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:51 smithi132 ceph-mon[70676]: pgmap v457: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:51.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:51 smithi088 ceph-mon[88643]: from='client.34882 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:18:51.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:51 smithi088 ceph-mon[88643]: pgmap v457: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:53.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:53 smithi132 ceph-mon[70676]: pgmap v458: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:53.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:53 smithi088 ceph-mon[88643]: pgmap v458: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:55.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:55 smithi132 ceph-mon[70676]: pgmap v459: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:55.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:55 smithi088 ceph-mon[88643]: pgmap v459: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:57.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:57 smithi132 ceph-mon[70676]: pgmap v460: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:57.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:57 smithi088 ceph-mon[88643]: pgmap v460: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:58.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:18:58.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:18:59.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:18:59 smithi132 ceph-mon[70676]: pgmap v461: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:18:59.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:18:59 smithi088 ceph-mon[88643]: pgmap v461: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:01.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:01 smithi132 ceph-mon[70676]: pgmap v462: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:01.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:01 smithi088 ceph-mon[88643]: pgmap v462: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:03.594 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:03 smithi088 ceph-mon[88643]: pgmap v463: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:03.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:03 smithi132 ceph-mon[70676]: pgmap v463: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:05.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:05 smithi132 ceph-mon[70676]: pgmap v464: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:05.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:05 smithi088 ceph-mon[88643]: pgmap v464: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:07.746 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:07 smithi132 ceph-mon[70676]: pgmap v465: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:07.844 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:07 smithi088 ceph-mon[88643]: pgmap v465: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:09.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:09 smithi088 ceph-mon[88643]: pgmap v466: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:09.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:09 smithi132 ceph-mon[70676]: pgmap v466: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:11.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:11 smithi088 ceph-mon[88643]: pgmap v467: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:11.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:11 smithi132 ceph-mon[70676]: pgmap v467: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:13.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:13 smithi088 ceph-mon[88643]: pgmap v468: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:13.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:19:13.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:13 smithi132 ceph-mon[70676]: pgmap v468: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:13.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:19:15.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:15 smithi088 ceph-mon[88643]: pgmap v469: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:15.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:15 smithi132 ceph-mon[70676]: pgmap v469: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:17.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:17 smithi088 ceph-mon[88643]: pgmap v470: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:17.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:17 smithi132 ceph-mon[70676]: pgmap v470: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:19.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:19 smithi088 ceph-mon[88643]: pgmap v471: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:19.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:19 smithi132 ceph-mon[70676]: pgmap v471: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:20.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.515+0000 7f1ea213c700 1 -- 172.21.15.88:0/166223637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c100e80 msgr2=0x7f1e9c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:20.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.515+0000 7f1ea213c700 1 --2- 172.21.15.88:0/166223637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c100e80 0x7f1e9c104e50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009910 tx=0x7f1e90009c20 comp rx=0 tx=0).stop 2024-02-19T19:19:20.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.515+0000 7f1ea213c700 1 -- 172.21.15.88:0/166223637 shutdown_connections 2024-02-19T19:19:20.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.515+0000 7f1ea213c700 1 --2- 172.21.15.88:0/166223637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c100e80 0x7f1e9c104e50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:20.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.515+0000 7f1ea213c700 1 --2- 172.21.15.88:0/166223637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e9c1004e0 0x7f1e9c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:20.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.515+0000 7f1ea213c700 1 -- 172.21.15.88:0/166223637 >> 172.21.15.88:0/166223637 conn(0x7f1e9c0fbda0 msgr2=0x7f1e9c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:20.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.515+0000 7f1ea213c700 1 -- 172.21.15.88:0/166223637 shutdown_connections 2024-02-19T19:19:20.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.515+0000 7f1ea213c700 1 -- 172.21.15.88:0/166223637 wait complete. 2024-02-19T19:19:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.516+0000 7f1ea213c700 1 Processor -- start 2024-02-19T19:19:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.516+0000 7f1ea213c700 1 -- start start 2024-02-19T19:19:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.516+0000 7f1ea213c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c1004e0 0x7f1e9c193df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.516+0000 7f1ea213c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e9c100e80 0x7f1e9c194330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.516+0000 7f1ea213c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c194a10 con 0x7f1e9c1004e0 2024-02-19T19:19:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.516+0000 7f1ea213c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9c1987a0 con 0x7f1e9c100e80 2024-02-19T19:19:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1e9b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c1004e0 0x7f1e9c193df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1e9b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c1004e0 0x7f1e9c193df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51276/0 (socket says 172.21.15.88:51276) 2024-02-19T19:19:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1e9b7fe700 1 -- 172.21.15.88:0/29610874 learned_addr learned my addr 172.21.15.88:0/29610874 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1e9affd700 1 --2- 172.21.15.88:0/29610874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e9c100e80 0x7f1e9c194330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1e9b7fe700 1 -- 172.21.15.88:0/29610874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e9c100e80 msgr2=0x7f1e9c194330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1e9b7fe700 1 --2- 172.21.15.88:0/29610874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e9c100e80 0x7f1e9c194330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1e9b7fe700 1 -- 172.21.15.88:0/29610874 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e900095f0 con 0x7f1e9c1004e0 2024-02-19T19:19:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1e9b7fe700 1 --2- 172.21.15.88:0/29610874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c1004e0 0x7f1e9c193df0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00b4b0 tx=0x7f1e8c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1e98ff9700 1 -- 172.21.15.88:0/29610874 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e8c011800 con 0x7f1e9c1004e0 2024-02-19T19:19:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1ea213c700 1 -- 172.21.15.88:0/29610874 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9c198a80 con 0x7f1e9c1004e0 2024-02-19T19:19:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1e98ff9700 1 -- 172.21.15.88:0/29610874 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e8c011e20 con 0x7f1e9c1004e0 2024-02-19T19:19:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1e98ff9700 1 -- 172.21.15.88:0/29610874 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e8c00f2e0 con 0x7f1e9c1004e0 2024-02-19T19:19:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.517+0000 7f1ea213c700 1 -- 172.21.15.88:0/29610874 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9c198fd0 con 0x7f1e9c1004e0 2024-02-19T19:19:20.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.518+0000 7f1ea213c700 1 -- 172.21.15.88:0/29610874 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e9c108780 con 0x7f1e9c1004e0 2024-02-19T19:19:20.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.519+0000 7f1e98ff9700 1 -- 172.21.15.88:0/29610874 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1e8c010380 con 0x7f1e9c1004e0 2024-02-19T19:19:20.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.519+0000 7f1e98ff9700 1 --2- 172.21.15.88:0/29610874 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e84071cb0 0x7f1e84074160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:20.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.520+0000 7f1e98ff9700 1 -- 172.21.15.88:0/29610874 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1e8c08d0f0 con 0x7f1e9c1004e0 2024-02-19T19:19:20.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.520+0000 7f1e9affd700 1 --2- 172.21.15.88:0/29610874 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e84071cb0 0x7f1e84074160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:20.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.520+0000 7f1e9affd700 1 --2- 172.21.15.88:0/29610874 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e84071cb0 0x7f1e84074160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1e900059d0 tx=0x7f1e90005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:20.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.525+0000 7f1e98ff9700 1 -- 172.21.15.88:0/29610874 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e8c0568a0 con 0x7f1e9c1004e0 2024-02-19T19:19:20.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.743+0000 7f1ea213c700 1 -- 172.21.15.88:0/29610874 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e9c1992c0 con 0x7f1e84071cb0 2024-02-19T19:19:20.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.744+0000 7f1e98ff9700 1 -- 172.21.15.88:0/29610874 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1e9c1992c0 con 0x7f1e84071cb0 2024-02-19T19:19:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.746+0000 7f1ea213c700 1 -- 172.21.15.88:0/29610874 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e84071cb0 msgr2=0x7f1e84074160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.746+0000 7f1ea213c700 1 --2- 172.21.15.88:0/29610874 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e84071cb0 0x7f1e84074160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1e900059d0 tx=0x7f1e90005960 comp rx=0 tx=0).stop 2024-02-19T19:19:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.746+0000 7f1ea213c700 1 -- 172.21.15.88:0/29610874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c1004e0 msgr2=0x7f1e9c193df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.746+0000 7f1ea213c700 1 --2- 172.21.15.88:0/29610874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c1004e0 0x7f1e9c193df0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00b4b0 tx=0x7f1e8c00b870 comp rx=0 tx=0).stop 2024-02-19T19:19:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.746+0000 7f1ea213c700 1 -- 172.21.15.88:0/29610874 shutdown_connections 2024-02-19T19:19:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.746+0000 7f1ea213c700 1 --2- 172.21.15.88:0/29610874 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e84071cb0 0x7f1e84074160 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.746+0000 7f1ea213c700 1 --2- 172.21.15.88:0/29610874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c1004e0 0x7f1e9c193df0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.746+0000 7f1ea213c700 1 --2- 172.21.15.88:0/29610874 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e9c100e80 0x7f1e9c194330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.746+0000 7f1ea213c700 1 -- 172.21.15.88:0/29610874 >> 172.21.15.88:0/29610874 conn(0x7f1e9c0fbda0 msgr2=0x7f1e9c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.747+0000 7f1ea213c700 1 -- 172.21.15.88:0/29610874 shutdown_connections 2024-02-19T19:19:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.747+0000 7f1ea213c700 1 -- 172.21.15.88:0/29610874 wait complete. 2024-02-19T19:19:20.771 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:19:20.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.852+0000 7f3f20b84700 1 -- 172.21.15.88:0/1043825851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f1c104130 msgr2=0x7f3f1c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:20.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.852+0000 7f3f20b84700 1 --2- 172.21.15.88:0/1043825851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f1c104130 0x7f3f1c104500 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009990 tx=0x7f3f0c009ca0 comp rx=0 tx=0).stop 2024-02-19T19:19:20.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.852+0000 7f3f20b84700 1 -- 172.21.15.88:0/1043825851 shutdown_connections 2024-02-19T19:19:20.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.852+0000 7f3f20b84700 1 --2- 172.21.15.88:0/1043825851 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f1c0ffbd0 0x7f3f1c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:20.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.852+0000 7f3f20b84700 1 --2- 172.21.15.88:0/1043825851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f1c104130 0x7f3f1c104500 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:20.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.852+0000 7f3f20b84700 1 -- 172.21.15.88:0/1043825851 >> 172.21.15.88:0/1043825851 conn(0x7f3f1c075130 msgr2=0x7f3f1c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:20.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.852+0000 7f3f20b84700 1 -- 172.21.15.88:0/1043825851 shutdown_connections 2024-02-19T19:19:20.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.852+0000 7f3f20b84700 1 -- 172.21.15.88:0/1043825851 wait complete. 2024-02-19T19:19:20.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.853+0000 7f3f20b84700 1 Processor -- start 2024-02-19T19:19:20.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.853+0000 7f3f20b84700 1 -- start start 2024-02-19T19:19:20.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.853+0000 7f3f20b84700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f1c0ffbd0 0x7f3f1c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:20.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.853+0000 7f3f20b84700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f1c104130 0x7f3f1c199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:20.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.853+0000 7f3f20b84700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c193a40 con 0x7f3f1c104130 2024-02-19T19:19:20.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.853+0000 7f3f20b84700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c193bb0 con 0x7f3f1c0ffbd0 2024-02-19T19:19:20.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.853+0000 7f3f19d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f1c104130 0x7f3f1c199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.853+0000 7f3f19d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f1c104130 0x7f3f1c199f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51286/0 (socket says 172.21.15.88:51286) 2024-02-19T19:19:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.853+0000 7f3f19d9b700 1 -- 172.21.15.88:0/2038282190 learned_addr learned my addr 172.21.15.88:0/2038282190 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.853+0000 7f3f1a59c700 1 --2- 172.21.15.88:0/2038282190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f1c0ffbd0 0x7f3f1c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.854+0000 7f3f19d9b700 1 -- 172.21.15.88:0/2038282190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f1c0ffbd0 msgr2=0x7f3f1c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.854+0000 7f3f19d9b700 1 --2- 172.21.15.88:0/2038282190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f1c0ffbd0 0x7f3f1c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.854+0000 7f3f19d9b700 1 -- 172.21.15.88:0/2038282190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0095f0 con 0x7f3f1c104130 2024-02-19T19:19:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.854+0000 7f3f19d9b700 1 --2- 172.21.15.88:0/2038282190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f1c104130 0x7f3f1c199f90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000d8d0 tx=0x7f3f1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.854+0000 7f3f0b7fe700 1 -- 172.21.15.88:0/2038282190 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f10009940 con 0x7f3f1c104130 2024-02-19T19:19:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.854+0000 7f3f0b7fe700 1 -- 172.21.15.88:0/2038282190 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f10010460 con 0x7f3f1c104130 2024-02-19T19:19:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.854+0000 7f3f0b7fe700 1 -- 172.21.15.88:0/2038282190 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f1000f5f0 con 0x7f3f1c104130 2024-02-19T19:19:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.854+0000 7f3f20b84700 1 -- 172.21.15.88:0/2038282190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c193e90 con 0x7f3f1c104130 2024-02-19T19:19:20.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.854+0000 7f3f20b84700 1 -- 172.21.15.88:0/2038282190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c1943e0 con 0x7f3f1c104130 2024-02-19T19:19:20.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.855+0000 7f3f20b84700 1 -- 172.21.15.88:0/2038282190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c1076a0 con 0x7f3f1c104130 2024-02-19T19:19:20.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.856+0000 7f3f0b7fe700 1 -- 172.21.15.88:0/2038282190 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3f100105d0 con 0x7f3f1c104130 2024-02-19T19:19:20.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.856+0000 7f3f0b7fe700 1 --2- 172.21.15.88:0/2038282190 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3f0406d8a0 0x7f3f0406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:20.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.856+0000 7f3f0b7fe700 1 -- 172.21.15.88:0/2038282190 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3f1008d430 con 0x7f3f1c104130 2024-02-19T19:19:20.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.856+0000 7f3f1a59c700 1 --2- 172.21.15.88:0/2038282190 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3f0406d8a0 0x7f3f0406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:20.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.856+0000 7f3f1a59c700 1 --2- 172.21.15.88:0/2038282190 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3f0406d8a0 0x7f3f0406fd50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c006010 tx=0x7f3f0c010040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:20.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:20.861+0000 7f3f0b7fe700 1 -- 172.21.15.88:0/2038282190 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3f10056c60 con 0x7f3f1c104130 2024-02-19T19:19:21.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.075+0000 7f3f20b84700 1 -- 172.21.15.88:0/2038282190 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f1c195080 con 0x7f3f0406d8a0 2024-02-19T19:19:21.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.077+0000 7f3f0b7fe700 1 -- 172.21.15.88:0/2038282190 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3f1c195080 con 0x7f3f0406d8a0 2024-02-19T19:19:21.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.079+0000 7f3f20b84700 1 -- 172.21.15.88:0/2038282190 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3f0406d8a0 msgr2=0x7f3f0406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.079+0000 7f3f20b84700 1 --2- 172.21.15.88:0/2038282190 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3f0406d8a0 0x7f3f0406fd50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c006010 tx=0x7f3f0c010040 comp rx=0 tx=0).stop 2024-02-19T19:19:21.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.079+0000 7f3f20b84700 1 -- 172.21.15.88:0/2038282190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f1c104130 msgr2=0x7f3f1c199f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.079+0000 7f3f20b84700 1 --2- 172.21.15.88:0/2038282190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f1c104130 0x7f3f1c199f90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3f1000d8d0 tx=0x7f3f1000dc90 comp rx=0 tx=0).stop 2024-02-19T19:19:21.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.079+0000 7f3f20b84700 1 -- 172.21.15.88:0/2038282190 shutdown_connections 2024-02-19T19:19:21.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.079+0000 7f3f20b84700 1 --2- 172.21.15.88:0/2038282190 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3f0406d8a0 0x7f3f0406fd50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.079+0000 7f3f20b84700 1 --2- 172.21.15.88:0/2038282190 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3f1c0ffbd0 0x7f3f1c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.079+0000 7f3f20b84700 1 --2- 172.21.15.88:0/2038282190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f1c104130 0x7f3f1c199f90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.079+0000 7f3f20b84700 1 -- 172.21.15.88:0/2038282190 >> 172.21.15.88:0/2038282190 conn(0x7f3f1c075130 msgr2=0x7f3f1c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:21.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.079+0000 7f3f20b84700 1 -- 172.21.15.88:0/2038282190 shutdown_connections 2024-02-19T19:19:21.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.079+0000 7f3f20b84700 1 -- 172.21.15.88:0/2038282190 wait complete. 2024-02-19T19:19:21.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.182+0000 7f5f723f5700 1 -- 172.21.15.88:0/901447007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f6c0ffbd0 msgr2=0x7f5f6c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.182+0000 7f5f723f5700 1 --2- 172.21.15.88:0/901447007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f6c0ffbd0 0x7f5f6c100040 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c009910 tx=0x7f5f5c009c20 comp rx=0 tx=0).stop 2024-02-19T19:19:21.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.182+0000 7f5f723f5700 1 -- 172.21.15.88:0/901447007 shutdown_connections 2024-02-19T19:19:21.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.182+0000 7f5f723f5700 1 --2- 172.21.15.88:0/901447007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f6c0ffbd0 0x7f5f6c100040 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.182+0000 7f5f723f5700 1 --2- 172.21.15.88:0/901447007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f6c104130 0x7f5f6c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.182+0000 7f5f723f5700 1 -- 172.21.15.88:0/901447007 >> 172.21.15.88:0/901447007 conn(0x7f5f6c075130 msgr2=0x7f5f6c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:21.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.182+0000 7f5f723f5700 1 -- 172.21.15.88:0/901447007 shutdown_connections 2024-02-19T19:19:21.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.182+0000 7f5f723f5700 1 -- 172.21.15.88:0/901447007 wait complete. 2024-02-19T19:19:21.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.182+0000 7f5f723f5700 1 Processor -- start 2024-02-19T19:19:21.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f723f5700 1 -- start start 2024-02-19T19:19:21.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f723f5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f6c0ffbd0 0x7f5f6c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:21.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f723f5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f6c104130 0x7f5f6c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:21.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f723f5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c1018b0 con 0x7f5f6c104130 2024-02-19T19:19:21.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f723f5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c1019f0 con 0x7f5f6c0ffbd0 2024-02-19T19:19:21.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f6b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f6c104130 0x7f5f6c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f6b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f6c104130 0x7f5f6c101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51312/0 (socket says 172.21.15.88:51312) 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f6b7fe700 1 -- 172.21.15.88:0/4030281605 learned_addr learned my addr 172.21.15.88:0/4030281605 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f6bfff700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f6c0ffbd0 0x7f5f6c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f6b7fe700 1 -- 172.21.15.88:0/4030281605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f6c0ffbd0 msgr2=0x7f5f6c100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f6b7fe700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f6c0ffbd0 0x7f5f6c100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f6b7fe700 1 -- 172.21.15.88:0/4030281605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c0095f0 con 0x7f5f6c104130 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.183+0000 7f5f6bfff700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f6c0ffbd0 0x7f5f6c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.184+0000 7f5f6b7fe700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f6c104130 0x7f5f6c101370 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00bd70 tx=0x7f5f5c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.184+0000 7f5f697fa700 1 -- 172.21.15.88:0/4030281605 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5c01d070 con 0x7f5f6c104130 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.184+0000 7f5f697fa700 1 -- 172.21.15.88:0/4030281605 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f5c004fb0 con 0x7f5f6c104130 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.184+0000 7f5f723f5700 1 -- 172.21.15.88:0/4030281605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c101c70 con 0x7f5f6c104130 2024-02-19T19:19:21.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.184+0000 7f5f697fa700 1 -- 172.21.15.88:0/4030281605 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f5c00f690 con 0x7f5f6c104130 2024-02-19T19:19:21.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.184+0000 7f5f723f5700 1 -- 172.21.15.88:0/4030281605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c1a2450 con 0x7f5f6c104130 2024-02-19T19:19:21.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.185+0000 7f5f723f5700 1 -- 172.21.15.88:0/4030281605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f58005320 con 0x7f5f6c104130 2024-02-19T19:19:21.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.185+0000 7f5f697fa700 1 -- 172.21.15.88:0/4030281605 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5f5c004af0 con 0x7f5f6c104130 2024-02-19T19:19:21.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.186+0000 7f5f697fa700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f5406d8a0 0x7f5f5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:21.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.186+0000 7f5f697fa700 1 -- 172.21.15.88:0/4030281605 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5f5c08ebd0 con 0x7f5f6c104130 2024-02-19T19:19:21.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.186+0000 7f5f6bfff700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f5406d8a0 0x7f5f5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:21.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.186+0000 7f5f6bfff700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f5406d8a0 0x7f5f5406fd50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5f60007900 tx=0x7f5f60008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:21.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.191+0000 7f5f697fa700 1 -- 172.21.15.88:0/4030281605 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f5c058380 con 0x7f5f6c104130 2024-02-19T19:19:21.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.399+0000 7f5f723f5700 1 -- 172.21.15.88:0/4030281605 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f58000bf0 con 0x7f5f5406d8a0 2024-02-19T19:19:21.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.409+0000 7f5f697fa700 1 -- 172.21.15.88:0/4030281605 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5f58000bf0 con 0x7f5f5406d8a0 2024-02-19T19:19:21.424 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:19:21.424 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (16m) 8m ago 26m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:19:21.424 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (27m) 8m ago 27m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:19:21.424 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (9m) 8m ago 25m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:19:21.424 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (12m) 8m ago 27m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:19:21.424 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (12m) 8m ago 25m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:19:21.424 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (15m) 8m ago 26m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:19:21.424 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (8m) 8m ago 20m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:19:21.424 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (8m) 8m ago 20m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (8m) 8m ago 20m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (20m) 8m ago 20m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (18m) 8m ago 28m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (17m) 8m ago 25m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (14m) 8m ago 28m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (13m) 8m ago 25m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (17m) 8m ago 27m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (17m) 8m ago 25m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (12m) 8m ago 24m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (11m) 8m ago 24m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (10m) 8m ago 23m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:19:21.425 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (10m) 8m ago 23m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:19:21.426 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (9m) 8m ago 22m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:19:21.426 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (9m) 8m ago 22m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:19:21.426 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (16m) 8m ago 25m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:19:21.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.411+0000 7f5f723f5700 1 -- 172.21.15.88:0/4030281605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f5406d8a0 msgr2=0x7f5f5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.411+0000 7f5f723f5700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f5406d8a0 0x7f5f5406fd50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5f60007900 tx=0x7f5f60008040 comp rx=0 tx=0).stop 2024-02-19T19:19:21.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.411+0000 7f5f723f5700 1 -- 172.21.15.88:0/4030281605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f6c104130 msgr2=0x7f5f6c101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.411+0000 7f5f723f5700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f6c104130 0x7f5f6c101370 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00bd70 tx=0x7f5f5c00be50 comp rx=0 tx=0).stop 2024-02-19T19:19:21.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.411+0000 7f5f723f5700 1 -- 172.21.15.88:0/4030281605 shutdown_connections 2024-02-19T19:19:21.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.411+0000 7f5f723f5700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f5406d8a0 0x7f5f5406fd50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.411+0000 7f5f723f5700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f6c0ffbd0 0x7f5f6c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.411+0000 7f5f723f5700 1 --2- 172.21.15.88:0/4030281605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f6c104130 0x7f5f6c101370 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.411+0000 7f5f723f5700 1 -- 172.21.15.88:0/4030281605 >> 172.21.15.88:0/4030281605 conn(0x7f5f6c075130 msgr2=0x7f5f6c068860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:21.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.412+0000 7f5f723f5700 1 -- 172.21.15.88:0/4030281605 shutdown_connections 2024-02-19T19:19:21.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.412+0000 7f5f723f5700 1 -- 172.21.15.88:0/4030281605 wait complete. 2024-02-19T19:19:21.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 -- 172.21.15.88:0/3854416498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2f4100e60 msgr2=0x7ff2f4104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 --2- 172.21.15.88:0/3854416498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2f4100e60 0x7ff2f4104e30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0009910 tx=0x7ff2e0009c20 comp rx=0 tx=0).stop 2024-02-19T19:19:21.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 -- 172.21.15.88:0/3854416498 shutdown_connections 2024-02-19T19:19:21.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 --2- 172.21.15.88:0/3854416498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2f4100e60 0x7ff2f4104e30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 --2- 172.21.15.88:0/3854416498 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2f41004c0 0x7ff2f4100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 -- 172.21.15.88:0/3854416498 >> 172.21.15.88:0/3854416498 conn(0x7ff2f40fbd80 msgr2=0x7ff2f40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 -- 172.21.15.88:0/3854416498 shutdown_connections 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 -- 172.21.15.88:0/3854416498 wait complete. 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 Processor -- start 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 -- start start 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2f41004c0 0x7ff2f4195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2f4100e60 0x7ff2f4196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f4196be0 con 0x7ff2f41004c0 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.517+0000 7ff2fb258700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f419a970 con 0x7ff2f4100e60 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f8ff4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2f41004c0 0x7ff2f4195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f8ff4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2f41004c0 0x7ff2f4195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51322/0 (socket says 172.21.15.88:51322) 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f8ff4700 1 -- 172.21.15.88:0/271613245 learned_addr learned my addr 172.21.15.88:0/271613245 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:21.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f3fff700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2f4100e60 0x7ff2f4196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:21.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f8ff4700 1 -- 172.21.15.88:0/271613245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2f4100e60 msgr2=0x7ff2f4196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f8ff4700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2f4100e60 0x7ff2f4196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f8ff4700 1 -- 172.21.15.88:0/271613245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e00095f0 con 0x7ff2f41004c0 2024-02-19T19:19:21.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f3fff700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2f4100e60 0x7ff2f4196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:19:21.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f8ff4700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2f41004c0 0x7ff2f4195fc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800b4b0 tx=0x7ff2e800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:21.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f1ffb700 1 -- 172.21.15.88:0/271613245 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e8011800 con 0x7ff2f41004c0 2024-02-19T19:19:21.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2fb258700 1 -- 172.21.15.88:0/271613245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f419ac50 con 0x7ff2f41004c0 2024-02-19T19:19:21.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f1ffb700 1 -- 172.21.15.88:0/271613245 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff2e8011e20 con 0x7ff2f41004c0 2024-02-19T19:19:21.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2f1ffb700 1 -- 172.21.15.88:0/271613245 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2e800f2e0 con 0x7ff2f41004c0 2024-02-19T19:19:21.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.518+0000 7ff2fb258700 1 -- 172.21.15.88:0/271613245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f419b1a0 con 0x7ff2f41004c0 2024-02-19T19:19:21.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.519+0000 7ff2fb258700 1 -- 172.21.15.88:0/271613245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f4108760 con 0x7ff2f41004c0 2024-02-19T19:19:21.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.521+0000 7ff2f1ffb700 1 -- 172.21.15.88:0/271613245 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff2e8011960 con 0x7ff2f41004c0 2024-02-19T19:19:21.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.521+0000 7ff2f1ffb700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff2e406d8a0 0x7ff2e406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:21.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.521+0000 7ff2f1ffb700 1 -- 172.21.15.88:0/271613245 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff2e808d060 con 0x7ff2f41004c0 2024-02-19T19:19:21.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.521+0000 7ff2f3fff700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff2e406d8a0 0x7ff2e406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:21.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.521+0000 7ff2f3fff700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff2e406d8a0 0x7ff2e406fd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff2f41975e0 tx=0x7ff2e001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:21.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.526+0000 7ff2f1ffb700 1 -- 172.21.15.88:0/271613245 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff2e8056890 con 0x7ff2f41004c0 2024-02-19T19:19:21.797 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:21 smithi132 ceph-mon[70676]: pgmap v472: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:21.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.805+0000 7ff2fb258700 1 -- 172.21.15.88:0/271613245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff2f419b470 con 0x7ff2f41004c0 2024-02-19T19:19:21.818 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:21 smithi088 ceph-mon[88643]: pgmap v472: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:21.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.805+0000 7ff2f1ffb700 1 -- 172.21.15.88:0/271613245 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7ff2e8059f50 con 0x7ff2f41004c0 2024-02-19T19:19:21.818 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:19:21.819 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:19:21.820 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:19:21.820 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:19:21.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.807+0000 7ff2fb258700 1 -- 172.21.15.88:0/271613245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff2e406d8a0 msgr2=0x7ff2e406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.807+0000 7ff2fb258700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff2e406d8a0 0x7ff2e406fd50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff2f41975e0 tx=0x7ff2e001a040 comp rx=0 tx=0).stop 2024-02-19T19:19:21.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.807+0000 7ff2fb258700 1 -- 172.21.15.88:0/271613245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2f41004c0 msgr2=0x7ff2f4195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.807+0000 7ff2fb258700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2f41004c0 0x7ff2f4195fc0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800b4b0 tx=0x7ff2e800b870 comp rx=0 tx=0).stop 2024-02-19T19:19:21.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.808+0000 7ff2fb258700 1 -- 172.21.15.88:0/271613245 shutdown_connections 2024-02-19T19:19:21.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.808+0000 7ff2fb258700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff2e406d8a0 0x7ff2e406fd50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.808+0000 7ff2fb258700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2f41004c0 0x7ff2f4195fc0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.808+0000 7ff2fb258700 1 --2- 172.21.15.88:0/271613245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2f4100e60 0x7ff2f4196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.808+0000 7ff2fb258700 1 -- 172.21.15.88:0/271613245 >> 172.21.15.88:0/271613245 conn(0x7ff2f40fbd80 msgr2=0x7ff2f40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:21.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.808+0000 7ff2fb258700 1 -- 172.21.15.88:0/271613245 shutdown_connections 2024-02-19T19:19:21.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.808+0000 7ff2fb258700 1 -- 172.21.15.88:0/271613245 wait complete. 2024-02-19T19:19:21.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.912+0000 7fa630927700 1 -- 172.21.15.88:0/1893249600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa62c0ffbd0 msgr2=0x7fa62c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.912+0000 7fa630927700 1 --2- 172.21.15.88:0/1893249600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa62c0ffbd0 0x7fa62c100040 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa61c009910 tx=0x7fa61c009c20 comp rx=0 tx=0).stop 2024-02-19T19:19:21.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.912+0000 7fa630927700 1 -- 172.21.15.88:0/1893249600 shutdown_connections 2024-02-19T19:19:21.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.912+0000 7fa630927700 1 --2- 172.21.15.88:0/1893249600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa62c0ffbd0 0x7fa62c100040 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.912+0000 7fa630927700 1 --2- 172.21.15.88:0/1893249600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c104130 0x7fa62c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.912+0000 7fa630927700 1 -- 172.21.15.88:0/1893249600 >> 172.21.15.88:0/1893249600 conn(0x7fa62c075130 msgr2=0x7fa62c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:21.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.912+0000 7fa630927700 1 -- 172.21.15.88:0/1893249600 shutdown_connections 2024-02-19T19:19:21.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.912+0000 7fa630927700 1 -- 172.21.15.88:0/1893249600 wait complete. 2024-02-19T19:19:21.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa630927700 1 Processor -- start 2024-02-19T19:19:21.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa630927700 1 -- start start 2024-02-19T19:19:21.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa630927700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c0ffbd0 0x7fa62c194bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:21.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa630927700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa62c104130 0x7fa62c1950f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:21.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa630927700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c195630 con 0x7fa62c104130 2024-02-19T19:19:21.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa630927700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa62c1957a0 con 0x7fa62c0ffbd0 2024-02-19T19:19:21.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa629d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa62c104130 0x7fa62c1950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa629d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa62c104130 0x7fa62c1950f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51336/0 (socket says 172.21.15.88:51336) 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa629d9b700 1 -- 172.21.15.88:0/283866962 learned_addr learned my addr 172.21.15.88:0/283866962 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa62a59c700 1 --2- 172.21.15.88:0/283866962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c0ffbd0 0x7fa62c194bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa629d9b700 1 -- 172.21.15.88:0/283866962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c0ffbd0 msgr2=0x7fa62c194bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa629d9b700 1 --2- 172.21.15.88:0/283866962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c0ffbd0 0x7fa62c194bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.913+0000 7fa629d9b700 1 -- 172.21.15.88:0/283866962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c0095f0 con 0x7fa62c104130 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.914+0000 7fa629d9b700 1 --2- 172.21.15.88:0/283866962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa62c104130 0x7fa62c1950f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00bd70 tx=0x7fa61c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.914+0000 7fa6237fe700 1 -- 172.21.15.88:0/283866962 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa61c01d070 con 0x7fa62c104130 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.914+0000 7fa6237fe700 1 -- 172.21.15.88:0/283866962 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa61c00f460 con 0x7fa62c104130 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.914+0000 7fa630927700 1 -- 172.21.15.88:0/283866962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa62c195a20 con 0x7fa62c104130 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.914+0000 7fa6237fe700 1 -- 172.21.15.88:0/283866962 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa61c017630 con 0x7fa62c104130 2024-02-19T19:19:21.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.914+0000 7fa630927700 1 -- 172.21.15.88:0/283866962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa62c1a24a0 con 0x7fa62c104130 2024-02-19T19:19:21.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.915+0000 7fa630927700 1 -- 172.21.15.88:0/283866962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa62c04f2a0 con 0x7fa62c104130 2024-02-19T19:19:21.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.915+0000 7fa6237fe700 1 -- 172.21.15.88:0/283866962 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa61c00f5d0 con 0x7fa62c104130 2024-02-19T19:19:21.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.916+0000 7fa6237fe700 1 --2- 172.21.15.88:0/283866962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa61806d8a0 0x7fa61806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:21.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.916+0000 7fa62a59c700 1 --2- 172.21.15.88:0/283866962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa61806d8a0 0x7fa61806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:21.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.916+0000 7fa6237fe700 1 -- 172.21.15.88:0/283866962 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa61c08e9e0 con 0x7fa62c104130 2024-02-19T19:19:21.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.916+0000 7fa62a59c700 1 --2- 172.21.15.88:0/283866962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa61806d8a0 0x7fa61806fd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa614006fd0 tx=0x7fa614008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:21.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:21.921+0000 7fa6237fe700 1 -- 172.21.15.88:0/283866962 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa61c05bcc0 con 0x7fa62c104130 2024-02-19T19:19:22.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.162+0000 7fa630927700 1 -- 172.21.15.88:0/283866962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa62c195a20 con 0x7fa62c104130 2024-02-19T19:19:22.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.163+0000 7fa6237fe700 1 -- 172.21.15.88:0/283866962 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fa62c195a20 con 0x7fa62c104130 2024-02-19T19:19:22.176 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:19:22.347 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:19:22.347 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:19:22.348 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:19:22.348 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:19:22.348 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:19:22.348 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:19:22.348 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:19:22.348 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:19:22.348 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:19:22.348 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:19:22.348 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:19:22.348 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:19:22.348 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:19:22.349 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:19:22.350 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:19:22.351 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:19:22.351 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:19:22.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.165+0000 7fa630927700 1 -- 172.21.15.88:0/283866962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa61806d8a0 msgr2=0x7fa61806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:22.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.165+0000 7fa630927700 1 --2- 172.21.15.88:0/283866962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa61806d8a0 0x7fa61806fd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa614006fd0 tx=0x7fa614008040 comp rx=0 tx=0).stop 2024-02-19T19:19:22.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.165+0000 7fa630927700 1 -- 172.21.15.88:0/283866962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa62c104130 msgr2=0x7fa62c1950f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.165+0000 7fa630927700 1 --2- 172.21.15.88:0/283866962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa62c104130 0x7fa62c1950f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00bd70 tx=0x7fa61c00be50 comp rx=0 tx=0).stop 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.165+0000 7fa630927700 1 -- 172.21.15.88:0/283866962 shutdown_connections 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.165+0000 7fa630927700 1 --2- 172.21.15.88:0/283866962 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa61806d8a0 0x7fa61806fd50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.165+0000 7fa630927700 1 --2- 172.21.15.88:0/283866962 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa62c0ffbd0 0x7fa62c194bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.165+0000 7fa630927700 1 --2- 172.21.15.88:0/283866962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa62c104130 0x7fa62c1950f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.165+0000 7fa630927700 1 -- 172.21.15.88:0/283866962 >> 172.21.15.88:0/283866962 conn(0x7fa62c075130 msgr2=0x7fa62c0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.165+0000 7fa630927700 1 -- 172.21.15.88:0/283866962 shutdown_connections 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.165+0000 7fa630927700 1 -- 172.21.15.88:0/283866962 wait complete. 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.270+0000 7f5bee12b700 1 -- 172.21.15.88:0/4119766841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5be8100ed0 msgr2=0x7f5be8104ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.270+0000 7f5bee12b700 1 --2- 172.21.15.88:0/4119766841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5be8100ed0 0x7f5be8104ea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc009910 tx=0x7f5bdc009c20 comp rx=0 tx=0).stop 2024-02-19T19:19:22.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.270+0000 7f5bee12b700 1 -- 172.21.15.88:0/4119766841 shutdown_connections 2024-02-19T19:19:22.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.270+0000 7f5bee12b700 1 --2- 172.21.15.88:0/4119766841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5be8100ed0 0x7f5be8104ea0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.270+0000 7f5bee12b700 1 --2- 172.21.15.88:0/4119766841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5be8100530 0x7f5be8100900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.270+0000 7f5bee12b700 1 -- 172.21.15.88:0/4119766841 >> 172.21.15.88:0/4119766841 conn(0x7f5be80fbe10 msgr2=0x7f5be80fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:22.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.270+0000 7f5bee12b700 1 -- 172.21.15.88:0/4119766841 shutdown_connections 2024-02-19T19:19:22.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.270+0000 7f5bee12b700 1 -- 172.21.15.88:0/4119766841 wait complete. 2024-02-19T19:19:22.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.270+0000 7f5bee12b700 1 Processor -- start 2024-02-19T19:19:22.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5bee12b700 1 -- start start 2024-02-19T19:19:22.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5bee12b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5be8100530 0x7f5be8195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:22.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5bee12b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5be8100ed0 0x7f5be8196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:22.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5bee12b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be8196b10 con 0x7f5be8100ed0 2024-02-19T19:19:22.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5bee12b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be819a8a0 con 0x7f5be8100530 2024-02-19T19:19:22.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5be6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5be8100ed0 0x7f5be8196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:22.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5be6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5be8100ed0 0x7f5be8196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51350/0 (socket says 172.21.15.88:51350) 2024-02-19T19:19:22.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5be6ffd700 1 -- 172.21.15.88:0/4236946517 learned_addr learned my addr 172.21.15.88:0/4236946517 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:22.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5be6ffd700 1 -- 172.21.15.88:0/4236946517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5be8100530 msgr2=0x7f5be8195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:22.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5be6ffd700 1 --2- 172.21.15.88:0/4236946517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5be8100530 0x7f5be8195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5be6ffd700 1 -- 172.21.15.88:0/4236946517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0095f0 con 0x7f5be8100ed0 2024-02-19T19:19:22.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.271+0000 7f5be6ffd700 1 --2- 172.21.15.88:0/4236946517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5be8100ed0 0x7f5be8196430 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005f50 tx=0x7f5bdc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:22.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.272+0000 7f5be4ff9700 1 -- 172.21.15.88:0/4236946517 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bdc01d070 con 0x7f5be8100ed0 2024-02-19T19:19:22.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.272+0000 7f5be4ff9700 1 -- 172.21.15.88:0/4236946517 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bdc00f460 con 0x7f5be8100ed0 2024-02-19T19:19:22.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.272+0000 7f5bee12b700 1 -- 172.21.15.88:0/4236946517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be819ab20 con 0x7f5be8100ed0 2024-02-19T19:19:22.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.272+0000 7f5be4ff9700 1 -- 172.21.15.88:0/4236946517 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bdc017630 con 0x7f5be8100ed0 2024-02-19T19:19:22.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.272+0000 7f5bee12b700 1 -- 172.21.15.88:0/4236946517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be819b010 con 0x7f5be8100ed0 2024-02-19T19:19:22.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.273+0000 7f5bee12b700 1 -- 172.21.15.88:0/4236946517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be81088a0 con 0x7f5be8100ed0 2024-02-19T19:19:22.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.273+0000 7f5be4ff9700 1 -- 172.21.15.88:0/4236946517 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5bdc00f5d0 con 0x7f5be8100ed0 2024-02-19T19:19:22.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.273+0000 7f5be4ff9700 1 --2- 172.21.15.88:0/4236946517 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5bd006d8a0 0x7f5bd006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:22.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.274+0000 7f5be4ff9700 1 -- 172.21.15.88:0/4236946517 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5bdc08ea40 con 0x7f5be8100ed0 2024-02-19T19:19:22.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.274+0000 7f5be77fe700 1 --2- 172.21.15.88:0/4236946517 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5bd006d8a0 0x7f5bd006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:22.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.274+0000 7f5be77fe700 1 --2- 172.21.15.88:0/4236946517 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5bd006d8a0 0x7f5bd006fd50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800a990 tx=0x7f5bd8005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:22.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.278+0000 7f5be4ff9700 1 -- 172.21.15.88:0/4236946517 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5bdc058270 con 0x7f5be8100ed0 2024-02-19T19:19:22.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.497+0000 7f5bee12b700 1 -- 172.21.15.88:0/4236946517 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5be819b280 con 0x7f5bd006d8a0 2024-02-19T19:19:22.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.499+0000 7f5be4ff9700 1 -- 172.21.15.88:0/4236946517 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5be819b280 con 0x7f5bd006d8a0 2024-02-19T19:19:22.512 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:19:22.512 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:19:22.512 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:19:22.512 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:19:22.512 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:19:22.512 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:19:22.512 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:19:22.512 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:19:22.513 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:19:22.513 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:19:22.513 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:19:22.513 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:19:22.513 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:19:22.513 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:19:22.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.501+0000 7f5bee12b700 1 -- 172.21.15.88:0/4236946517 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5bd006d8a0 msgr2=0x7f5bd006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:22.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.501+0000 7f5bee12b700 1 --2- 172.21.15.88:0/4236946517 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5bd006d8a0 0x7f5bd006fd50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5bd800a990 tx=0x7f5bd8005c80 comp rx=0 tx=0).stop 2024-02-19T19:19:22.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.501+0000 7f5bee12b700 1 -- 172.21.15.88:0/4236946517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5be8100ed0 msgr2=0x7f5be8196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:22.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.501+0000 7f5bee12b700 1 --2- 172.21.15.88:0/4236946517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5be8100ed0 0x7f5be8196430 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005f50 tx=0x7f5bdc00be20 comp rx=0 tx=0).stop 2024-02-19T19:19:22.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.501+0000 7f5bee12b700 1 -- 172.21.15.88:0/4236946517 shutdown_connections 2024-02-19T19:19:22.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.501+0000 7f5bee12b700 1 --2- 172.21.15.88:0/4236946517 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5bd006d8a0 0x7f5bd006fd50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.501+0000 7f5bee12b700 1 --2- 172.21.15.88:0/4236946517 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5be8100530 0x7f5be8195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.501+0000 7f5bee12b700 1 --2- 172.21.15.88:0/4236946517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5be8100ed0 0x7f5be8196430 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.501+0000 7f5bee12b700 1 -- 172.21.15.88:0/4236946517 >> 172.21.15.88:0/4236946517 conn(0x7f5be80fbe10 msgr2=0x7f5be80fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:22.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.501+0000 7f5bee12b700 1 -- 172.21.15.88:0/4236946517 shutdown_connections 2024-02-19T19:19:22.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.501+0000 7f5bee12b700 1 -- 172.21.15.88:0/4236946517 wait complete. 2024-02-19T19:19:22.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.607+0000 7f7e37190700 1 -- 172.21.15.88:0/4025330657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e30104130 msgr2=0x7f7e30104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:22.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.607+0000 7f7e37190700 1 --2- 172.21.15.88:0/4025330657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e30104130 0x7f7e30104500 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800b0a0 tx=0x7f7e2800b3b0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.607+0000 7f7e37190700 1 -- 172.21.15.88:0/4025330657 shutdown_connections 2024-02-19T19:19:22.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.607+0000 7f7e37190700 1 --2- 172.21.15.88:0/4025330657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e300ffbd0 0x7f7e30100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.607+0000 7f7e37190700 1 --2- 172.21.15.88:0/4025330657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e30104130 0x7f7e30104500 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.607+0000 7f7e37190700 1 -- 172.21.15.88:0/4025330657 >> 172.21.15.88:0/4025330657 conn(0x7f7e30075130 msgr2=0x7f7e30075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:22.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.607+0000 7f7e37190700 1 -- 172.21.15.88:0/4025330657 shutdown_connections 2024-02-19T19:19:22.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.607+0000 7f7e37190700 1 -- 172.21.15.88:0/4025330657 wait complete. 2024-02-19T19:19:22.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e37190700 1 Processor -- start 2024-02-19T19:19:22.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e37190700 1 -- start start 2024-02-19T19:19:22.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e37190700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e300ffbd0 0x7f7e30196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e37190700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e30104130 0x7f7e301972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e37190700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e301979d0 con 0x7f7e300ffbd0 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e37190700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30197b40 con 0x7f7e30104130 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e34f2c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e300ffbd0 0x7f7e30196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e34f2c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e300ffbd0 0x7f7e30196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51366/0 (socket says 172.21.15.88:51366) 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e34f2c700 1 -- 172.21.15.88:0/2006489810 learned_addr learned my addr 172.21.15.88:0/2006489810 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e2ffff700 1 --2- 172.21.15.88:0/2006489810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e30104130 0x7f7e301972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e34f2c700 1 -- 172.21.15.88:0/2006489810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e30104130 msgr2=0x7f7e301972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e34f2c700 1 --2- 172.21.15.88:0/2006489810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e30104130 0x7f7e301972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e34f2c700 1 -- 172.21.15.88:0/2006489810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e28009d00 con 0x7f7e300ffbd0 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.608+0000 7f7e34f2c700 1 --2- 172.21.15.88:0/2006489810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e300ffbd0 0x7f7e30196db0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7e28000f80 tx=0x7f7e2800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:22.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.609+0000 7f7e2dffb700 1 -- 172.21.15.88:0/2006489810 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e2800e040 con 0x7f7e300ffbd0 2024-02-19T19:19:22.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.609+0000 7f7e37190700 1 -- 172.21.15.88:0/2006489810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3019fe20 con 0x7f7e300ffbd0 2024-02-19T19:19:22.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.609+0000 7f7e2dffb700 1 -- 172.21.15.88:0/2006489810 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e28003e60 con 0x7f7e300ffbd0 2024-02-19T19:19:22.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.609+0000 7f7e2dffb700 1 -- 172.21.15.88:0/2006489810 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e2801d910 con 0x7f7e300ffbd0 2024-02-19T19:19:22.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.609+0000 7f7e37190700 1 -- 172.21.15.88:0/2006489810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e301a0290 con 0x7f7e300ffbd0 2024-02-19T19:19:22.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.610+0000 7f7e37190700 1 -- 172.21.15.88:0/2006489810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e301076a0 con 0x7f7e300ffbd0 2024-02-19T19:19:22.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.610+0000 7f7e2dffb700 1 -- 172.21.15.88:0/2006489810 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7e28004820 con 0x7f7e300ffbd0 2024-02-19T19:19:22.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.611+0000 7f7e2dffb700 1 --2- 172.21.15.88:0/2006489810 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e1806d8a0 0x7f7e1806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:22.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.611+0000 7f7e2dffb700 1 -- 172.21.15.88:0/2006489810 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7e2808ee40 con 0x7f7e300ffbd0 2024-02-19T19:19:22.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.611+0000 7f7e2ffff700 1 --2- 172.21.15.88:0/2006489810 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e1806d8a0 0x7f7e1806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:22.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.611+0000 7f7e2ffff700 1 --2- 172.21.15.88:0/2006489810 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e1806d8a0 0x7f7e1806fd50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e24006fd0 tx=0x7f7e24008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:22.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.615+0000 7f7e2dffb700 1 -- 172.21.15.88:0/2006489810 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e280585f0 con 0x7f7e300ffbd0 2024-02-19T19:19:22.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:22 smithi088 ceph-mon[88643]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:22.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:22 smithi088 ceph-mon[88643]: from='client.34894 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:22.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:22 smithi088 ceph-mon[88643]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:22.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:22 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/271613245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:19:22.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:22 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/283866962' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:19:22.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.895+0000 7f7e37190700 1 -- 172.21.15.88:0/2006489810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7e30068b30 con 0x7f7e300ffbd0 2024-02-19T19:19:22.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.896+0000 7f7e2dffb700 1 -- 172.21.15.88:0/2006489810 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f7e28017100 con 0x7f7e300ffbd0 2024-02-19T19:19:22.909 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:19:22.909 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:19:22.909 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:19:22.909 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:19:22.909 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:19:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.898+0000 7f7e37190700 1 -- 172.21.15.88:0/2006489810 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e1806d8a0 msgr2=0x7f7e1806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.898+0000 7f7e37190700 1 --2- 172.21.15.88:0/2006489810 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e1806d8a0 0x7f7e1806fd50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e24006fd0 tx=0x7f7e24008040 comp rx=0 tx=0).stop 2024-02-19T19:19:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.898+0000 7f7e37190700 1 -- 172.21.15.88:0/2006489810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e300ffbd0 msgr2=0x7f7e30196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.898+0000 7f7e37190700 1 --2- 172.21.15.88:0/2006489810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e300ffbd0 0x7f7e30196db0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7e28000f80 tx=0x7f7e2800cdd0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.898+0000 7f7e37190700 1 -- 172.21.15.88:0/2006489810 shutdown_connections 2024-02-19T19:19:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.898+0000 7f7e37190700 1 --2- 172.21.15.88:0/2006489810 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e1806d8a0 0x7f7e1806fd50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.898+0000 7f7e37190700 1 --2- 172.21.15.88:0/2006489810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e300ffbd0 0x7f7e30196db0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.898+0000 7f7e37190700 1 --2- 172.21.15.88:0/2006489810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e30104130 0x7f7e301972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:22.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.898+0000 7f7e37190700 1 -- 172.21.15.88:0/2006489810 >> 172.21.15.88:0/2006489810 conn(0x7f7e30075130 msgr2=0x7f7e300fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:22.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.898+0000 7f7e37190700 1 -- 172.21.15.88:0/2006489810 shutdown_connections 2024-02-19T19:19:22.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:22.898+0000 7f7e37190700 1 -- 172.21.15.88:0/2006489810 wait complete. 2024-02-19T19:19:22.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:22 smithi132 ceph-mon[70676]: from='client.34890 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:22.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:22 smithi132 ceph-mon[70676]: from='client.34894 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:22.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:22 smithi132 ceph-mon[70676]: from='client.34898 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:22.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:22 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/271613245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:19:22.996 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:22 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/283866962' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:19:23.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:23 smithi088 ceph-mon[88643]: pgmap v473: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:23.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:23 smithi088 ceph-mon[88643]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:23.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:23 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2006489810' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:19:23.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:23 smithi132 ceph-mon[70676]: pgmap v473: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:23.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:23 smithi132 ceph-mon[70676]: from='client.34910 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:23.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:23 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2006489810' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:19:25.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:25 smithi088 ceph-mon[88643]: pgmap v474: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:25.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:25 smithi132 ceph-mon[70676]: pgmap v474: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:27.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:27 smithi088 ceph-mon[88643]: pgmap v475: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:27.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:27 smithi132 ceph-mon[70676]: pgmap v475: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:28.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:19:28.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:19:29.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:29 smithi088 ceph-mon[88643]: pgmap v476: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:29.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:29 smithi132 ceph-mon[70676]: pgmap v476: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:31.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:31 smithi088 ceph-mon[88643]: pgmap v477: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:31.858 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:31 smithi132 ceph-mon[70676]: pgmap v477: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:33.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:33 smithi088 ceph-mon[88643]: pgmap v478: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:33.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:33 smithi132 ceph-mon[70676]: pgmap v478: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:35.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:35 smithi088 ceph-mon[88643]: pgmap v479: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:35.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:35 smithi132 ceph-mon[70676]: pgmap v479: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:37.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:37 smithi088 ceph-mon[88643]: pgmap v480: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:37.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:37 smithi132 ceph-mon[70676]: pgmap v480: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:39.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:39 smithi088 ceph-mon[88643]: pgmap v481: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:39.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:39 smithi132 ceph-mon[70676]: pgmap v481: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:41.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:41 smithi088 ceph-mon[88643]: pgmap v482: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:41.965 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:41 smithi132 ceph-mon[70676]: pgmap v482: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:43.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:43 smithi088 ceph-mon[88643]: pgmap v483: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:43.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:19:43.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:43 smithi132 ceph-mon[70676]: pgmap v483: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:43.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:19:45.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:45 smithi088 ceph-mon[88643]: pgmap v484: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:45.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:45 smithi132 ceph-mon[70676]: pgmap v484: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:47.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:47 smithi088 ceph-mon[88643]: pgmap v485: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:47.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:47 smithi132 ceph-mon[70676]: pgmap v485: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:49.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:49 smithi088 ceph-mon[88643]: pgmap v486: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:49.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:49 smithi132 ceph-mon[70676]: pgmap v486: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:51.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:51 smithi088 ceph-mon[88643]: pgmap v487: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:51.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:51 smithi132 ceph-mon[70676]: pgmap v487: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:53.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.016+0000 7f0f1a647700 1 -- 172.21.15.88:0/192474403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f141004a0 msgr2=0x7f0f14100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.016+0000 7f0f1a647700 1 --2- 172.21.15.88:0/192474403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f141004a0 0x7f0f14100870 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f0f04009910 tx=0x7f0f04009c20 comp rx=0 tx=0).stop 2024-02-19T19:19:53.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.016+0000 7f0f1a647700 1 -- 172.21.15.88:0/192474403 shutdown_connections 2024-02-19T19:19:53.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.016+0000 7f0f1a647700 1 --2- 172.21.15.88:0/192474403 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f14100e40 0x7f0f14104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.016+0000 7f0f1a647700 1 --2- 172.21.15.88:0/192474403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f141004a0 0x7f0f14100870 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.016+0000 7f0f1a647700 1 -- 172.21.15.88:0/192474403 >> 172.21.15.88:0/192474403 conn(0x7f0f140fbd40 msgr2=0x7f0f140fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:53.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.016+0000 7f0f1a647700 1 -- 172.21.15.88:0/192474403 shutdown_connections 2024-02-19T19:19:53.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.016+0000 7f0f1a647700 1 -- 172.21.15.88:0/192474403 wait complete. 2024-02-19T19:19:53.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.016+0000 7f0f1a647700 1 Processor -- start 2024-02-19T19:19:53.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.016+0000 7f0f1a647700 1 -- start start 2024-02-19T19:19:53.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f1a647700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f141004a0 0x7f0f14198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:53.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f1a647700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f14100e40 0x7f0f14198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:53.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f1a647700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f14198d70 con 0x7f0f14100e40 2024-02-19T19:19:53.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f1a647700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f1419cb00 con 0x7f0f141004a0 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f137fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f14100e40 0x7f0f14198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f137fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f14100e40 0x7f0f14198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43922/0 (socket says 172.21.15.88:43922) 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f137fe700 1 -- 172.21.15.88:0/3682155268 learned_addr learned my addr 172.21.15.88:0/3682155268 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f13fff700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f141004a0 0x7f0f14198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f137fe700 1 -- 172.21.15.88:0/3682155268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f141004a0 msgr2=0x7f0f14198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f137fe700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f141004a0 0x7f0f14198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f137fe700 1 -- 172.21.15.88:0/3682155268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f040095f0 con 0x7f0f14100e40 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.017+0000 7f0f13fff700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f141004a0 0x7f0f14198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.018+0000 7f0f137fe700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f14100e40 0x7f0f14198690 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800b480 tx=0x7f0f0800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.018+0000 7f0f117fa700 1 -- 172.21.15.88:0/3682155268 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f080118c0 con 0x7f0f14100e40 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.018+0000 7f0f1a647700 1 -- 172.21.15.88:0/3682155268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f1419cde0 con 0x7f0f14100e40 2024-02-19T19:19:53.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.018+0000 7f0f117fa700 1 -- 172.21.15.88:0/3682155268 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f08010380 con 0x7f0f14100e40 2024-02-19T19:19:53.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.018+0000 7f0f117fa700 1 -- 172.21.15.88:0/3682155268 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f0800f4f0 con 0x7f0f14100e40 2024-02-19T19:19:53.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.018+0000 7f0f1a647700 1 -- 172.21.15.88:0/3682155268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f1419d330 con 0x7f0f14100e40 2024-02-19T19:19:53.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.019+0000 7f0f1a647700 1 -- 172.21.15.88:0/3682155268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f1404f2a0 con 0x7f0f14100e40 2024-02-19T19:19:53.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.019+0000 7f0f117fa700 1 -- 172.21.15.88:0/3682155268 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0f080104f0 con 0x7f0f14100e40 2024-02-19T19:19:53.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.020+0000 7f0f117fa700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0efc06d8a0 0x7f0efc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:53.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.020+0000 7f0f117fa700 1 -- 172.21.15.88:0/3682155268 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0f0808d0a0 con 0x7f0f14100e40 2024-02-19T19:19:53.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.020+0000 7f0f13fff700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0efc06d8a0 0x7f0efc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:53.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.020+0000 7f0f13fff700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0efc06d8a0 0x7f0efc06fd50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005100 tx=0x7f0f0401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:53.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.025+0000 7f0f117fa700 1 -- 172.21.15.88:0/3682155268 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0f080568d0 con 0x7f0f14100e40 2024-02-19T19:19:53.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.239+0000 7f0f1a647700 1 -- 172.21.15.88:0/3682155268 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f1419d6d0 con 0x7f0efc06d8a0 2024-02-19T19:19:53.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.241+0000 7f0f117fa700 1 -- 172.21.15.88:0/3682155268 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0f1419d6d0 con 0x7f0efc06d8a0 2024-02-19T19:19:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.243+0000 7f0f1a647700 1 -- 172.21.15.88:0/3682155268 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0efc06d8a0 msgr2=0x7f0efc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.243+0000 7f0f1a647700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0efc06d8a0 0x7f0efc06fd50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0f04005100 tx=0x7f0f0401a040 comp rx=0 tx=0).stop 2024-02-19T19:19:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.243+0000 7f0f1a647700 1 -- 172.21.15.88:0/3682155268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f14100e40 msgr2=0x7f0f14198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.243+0000 7f0f1a647700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f14100e40 0x7f0f14198690 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0f0800b480 tx=0x7f0f0800b840 comp rx=0 tx=0).stop 2024-02-19T19:19:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.243+0000 7f0f1a647700 1 -- 172.21.15.88:0/3682155268 shutdown_connections 2024-02-19T19:19:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.243+0000 7f0f1a647700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0efc06d8a0 0x7f0efc06fd50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.243+0000 7f0f1a647700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0f141004a0 0x7f0f14198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.243+0000 7f0f1a647700 1 --2- 172.21.15.88:0/3682155268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f14100e40 0x7f0f14198690 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.243+0000 7f0f1a647700 1 -- 172.21.15.88:0/3682155268 >> 172.21.15.88:0/3682155268 conn(0x7f0f140fbd40 msgr2=0x7f0f140fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.243+0000 7f0f1a647700 1 -- 172.21.15.88:0/3682155268 shutdown_connections 2024-02-19T19:19:53.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.243+0000 7f0f1a647700 1 -- 172.21.15.88:0/3682155268 wait complete. 2024-02-19T19:19:53.267 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:19:53.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.349+0000 7fef7f777700 1 -- 172.21.15.88:0/2045314313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef78100460 msgr2=0x7fef78100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.349+0000 7fef7f777700 1 --2- 172.21.15.88:0/2045314313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef78100460 0x7fef78100830 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fef68009910 tx=0x7fef68009c20 comp rx=0 tx=0).stop 2024-02-19T19:19:53.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.349+0000 7fef7f777700 1 -- 172.21.15.88:0/2045314313 shutdown_connections 2024-02-19T19:19:53.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.349+0000 7fef7f777700 1 --2- 172.21.15.88:0/2045314313 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef78100d70 0x7fef781094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.349+0000 7fef7f777700 1 --2- 172.21.15.88:0/2045314313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef78100460 0x7fef78100830 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.349+0000 7fef7f777700 1 -- 172.21.15.88:0/2045314313 >> 172.21.15.88:0/2045314313 conn(0x7fef780748b0 msgr2=0x7fef78074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:53.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.349+0000 7fef7f777700 1 -- 172.21.15.88:0/2045314313 shutdown_connections 2024-02-19T19:19:53.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.349+0000 7fef7f777700 1 -- 172.21.15.88:0/2045314313 wait complete. 2024-02-19T19:19:53.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7f777700 1 Processor -- start 2024-02-19T19:19:53.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7f777700 1 -- start start 2024-02-19T19:19:53.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7f777700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef78100460 0x7fef78195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:53.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7f777700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef78100d70 0x7fef78196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7f777700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef78196b40 con 0x7fef78100460 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7f777700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7819a8d0 con 0x7fef78100d70 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7d513700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef78100460 0x7fef78195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7d513700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef78100460 0x7fef78195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43938/0 (socket says 172.21.15.88:43938) 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7d513700 1 -- 172.21.15.88:0/4098311090 learned_addr learned my addr 172.21.15.88:0/4098311090 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7d513700 1 -- 172.21.15.88:0/4098311090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef78100d70 msgr2=0x7fef78196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7d513700 1 --2- 172.21.15.88:0/4098311090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef78100d70 0x7fef78196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7d513700 1 -- 172.21.15.88:0/4098311090 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef680095f0 con 0x7fef78100460 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef7d513700 1 --2- 172.21.15.88:0/4098311090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef78100460 0x7fef78195f20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fef68006010 tx=0x7fef6800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.350+0000 7fef767fc700 1 -- 172.21.15.88:0/4098311090 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef6801d070 con 0x7fef78100460 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.351+0000 7fef7f777700 1 -- 172.21.15.88:0/4098311090 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef7819ab50 con 0x7fef78100460 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.351+0000 7fef767fc700 1 -- 172.21.15.88:0/4098311090 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef68004fb0 con 0x7fef78100460 2024-02-19T19:19:53.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.351+0000 7fef767fc700 1 -- 172.21.15.88:0/4098311090 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef68022ea0 con 0x7fef78100460 2024-02-19T19:19:53.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.351+0000 7fef7f777700 1 -- 172.21.15.88:0/4098311090 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef7819b040 con 0x7fef78100460 2024-02-19T19:19:53.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.352+0000 7fef7f777700 1 -- 172.21.15.88:0/4098311090 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef78107e40 con 0x7fef78100460 2024-02-19T19:19:53.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.352+0000 7fef767fc700 1 -- 172.21.15.88:0/4098311090 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fef68022470 con 0x7fef78100460 2024-02-19T19:19:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.352+0000 7fef767fc700 1 --2- 172.21.15.88:0/4098311090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fef6006d8a0 0x7fef6006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.353+0000 7fef767fc700 1 -- 172.21.15.88:0/4098311090 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fef6808ebb0 con 0x7fef78100460 2024-02-19T19:19:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.353+0000 7fef7cd12700 1 --2- 172.21.15.88:0/4098311090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fef6006d8a0 0x7fef6006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:53.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.353+0000 7fef7cd12700 1 --2- 172.21.15.88:0/4098311090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fef6006d8a0 0x7fef6006fd50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fef78197540 tx=0x7fef6c008540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:53.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.357+0000 7fef767fc700 1 -- 172.21.15.88:0/4098311090 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fef68095080 con 0x7fef78100460 2024-02-19T19:19:53.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.574+0000 7fef7f777700 1 -- 172.21.15.88:0/4098311090 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fef7819b260 con 0x7fef6006d8a0 2024-02-19T19:19:53.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.576+0000 7fef767fc700 1 -- 172.21.15.88:0/4098311090 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fef7819b260 con 0x7fef6006d8a0 2024-02-19T19:19:53.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.578+0000 7fef7f777700 1 -- 172.21.15.88:0/4098311090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fef6006d8a0 msgr2=0x7fef6006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.578+0000 7fef7f777700 1 --2- 172.21.15.88:0/4098311090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fef6006d8a0 0x7fef6006fd50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fef78197540 tx=0x7fef6c008540 comp rx=0 tx=0).stop 2024-02-19T19:19:53.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.578+0000 7fef7f777700 1 -- 172.21.15.88:0/4098311090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef78100460 msgr2=0x7fef78195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.578+0000 7fef7f777700 1 --2- 172.21.15.88:0/4098311090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef78100460 0x7fef78195f20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fef68006010 tx=0x7fef6800be20 comp rx=0 tx=0).stop 2024-02-19T19:19:53.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.578+0000 7fef7f777700 1 -- 172.21.15.88:0/4098311090 shutdown_connections 2024-02-19T19:19:53.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.578+0000 7fef7f777700 1 --2- 172.21.15.88:0/4098311090 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fef6006d8a0 0x7fef6006fd50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.578+0000 7fef7f777700 1 --2- 172.21.15.88:0/4098311090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef78100460 0x7fef78195f20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.578+0000 7fef7f777700 1 --2- 172.21.15.88:0/4098311090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fef78100d70 0x7fef78196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.578+0000 7fef7f777700 1 -- 172.21.15.88:0/4098311090 >> 172.21.15.88:0/4098311090 conn(0x7fef780748b0 msgr2=0x7fef780feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:53.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.578+0000 7fef7f777700 1 -- 172.21.15.88:0/4098311090 shutdown_connections 2024-02-19T19:19:53.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.578+0000 7fef7f777700 1 -- 172.21.15.88:0/4098311090 wait complete. 2024-02-19T19:19:53.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.681+0000 7f64dcad4700 1 -- 172.21.15.88:0/3028607706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d8101c90 msgr2=0x7f64d810a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.681+0000 7f64dcad4700 1 --2- 172.21.15.88:0/3028607706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d8101c90 0x7f64d810a2d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009910 tx=0x7f64cc009c20 comp rx=0 tx=0).stop 2024-02-19T19:19:53.694 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:53 smithi088 ceph-mon[88643]: pgmap v488: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.682+0000 7f64dcad4700 1 -- 172.21.15.88:0/3028607706 shutdown_connections 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.682+0000 7f64dcad4700 1 --2- 172.21.15.88:0/3028607706 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d8101c90 0x7f64d810a2d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.682+0000 7f64dcad4700 1 --2- 172.21.15.88:0/3028607706 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81012f0 0x7f64d81016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.682+0000 7f64dcad4700 1 -- 172.21.15.88:0/3028607706 >> 172.21.15.88:0/3028607706 conn(0x7f64d80fac80 msgr2=0x7f64d80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.682+0000 7f64dcad4700 1 -- 172.21.15.88:0/3028607706 shutdown_connections 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.682+0000 7f64dcad4700 1 -- 172.21.15.88:0/3028607706 wait complete. 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.682+0000 7f64dcad4700 1 Processor -- start 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.682+0000 7f64dcad4700 1 -- start start 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64dcad4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81012f0 0x7f64d80ff120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64dcad4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d8101c90 0x7f64d80ff660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64dcad4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80ffba0 con 0x7f64d8101c90 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64dcad4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d80ffce0 con 0x7f64d81012f0 2024-02-19T19:19:53.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64d5d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d8101c90 0x7f64d80ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64d5d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d8101c90 0x7f64d80ff660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43970/0 (socket says 172.21.15.88:43970) 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64d5d9b700 1 -- 172.21.15.88:0/3064945792 learned_addr learned my addr 172.21.15.88:0/3064945792 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64d659c700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81012f0 0x7f64d80ff120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64d5d9b700 1 -- 172.21.15.88:0/3064945792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81012f0 msgr2=0x7f64d80ff120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64d5d9b700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81012f0 0x7f64d80ff120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64d5d9b700 1 -- 172.21.15.88:0/3064945792 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc0095f0 con 0x7f64d8101c90 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64d659c700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81012f0 0x7f64d80ff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64d5d9b700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d8101c90 0x7f64d80ff660 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f64cc000c00 tx=0x7f64cc0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64c77fe700 1 -- 172.21.15.88:0/3064945792 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64cc01d070 con 0x7f64d8101c90 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64dcad4700 1 -- 172.21.15.88:0/3064945792 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d80fff60 con 0x7f64d8101c90 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.683+0000 7f64c77fe700 1 -- 172.21.15.88:0/3064945792 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f64cc00f460 con 0x7f64d8101c90 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.684+0000 7f64c77fe700 1 -- 172.21.15.88:0/3064945792 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64cc021640 con 0x7f64d8101c90 2024-02-19T19:19:53.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.684+0000 7f64dcad4700 1 -- 172.21.15.88:0/3064945792 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d81a2580 con 0x7f64d8101c90 2024-02-19T19:19:53.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.685+0000 7f64dcad4700 1 -- 172.21.15.88:0/3064945792 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d81079d0 con 0x7f64d8101c90 2024-02-19T19:19:53.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.685+0000 7f64c77fe700 1 -- 172.21.15.88:0/3064945792 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f64cc00f5d0 con 0x7f64d8101c90 2024-02-19T19:19:53.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.685+0000 7f64c77fe700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f64c006d8a0 0x7f64c006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:53.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.686+0000 7f64c77fe700 1 -- 172.21.15.88:0/3064945792 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f64cc08e7a0 con 0x7f64d8101c90 2024-02-19T19:19:53.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.686+0000 7f64d659c700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f64c006d8a0 0x7f64c006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:53.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.686+0000 7f64d659c700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f64c006d8a0 0x7f64c006fd50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f64c8007900 tx=0x7f64c8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:53.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.691+0000 7f64c77fe700 1 -- 172.21.15.88:0/3064945792 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f64cc058000 con 0x7f64d8101c90 2024-02-19T19:19:53.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.926+0000 7f64dcad4700 1 -- 172.21.15.88:0/3064945792 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f64d81006c0 con 0x7f64c006d8a0 2024-02-19T19:19:53.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.936+0000 7f64c77fe700 1 -- 172.21.15.88:0/3064945792 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f64d81006c0 con 0x7f64c006d8a0 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (17m) 8m ago 27m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (28m) 8m ago 28m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (9m) 8m ago 26m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (13m) 8m ago 27m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (13m) 8m ago 25m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (16m) 8m ago 26m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (9m) 8m ago 21m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (9m) 8m ago 21m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (8m) 8m ago 21m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (21m) 8m ago 21m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (18m) 8m ago 29m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (17m) 8m ago 25m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:19:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (15m) 8m ago 29m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:19:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (14m) 8m ago 25m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:19:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (17m) 8m ago 27m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:19:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (17m) 8m ago 25m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:19:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (12m) 8m ago 25m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:19:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (11m) 8m ago 24m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:19:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (11m) 8m ago 24m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:19:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (10m) 8m ago 23m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:19:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (10m) 8m ago 23m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:19:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (9m) 8m ago 22m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:19:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (17m) 8m ago 26m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:19:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.938+0000 7f64dcad4700 1 -- 172.21.15.88:0/3064945792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f64c006d8a0 msgr2=0x7f64c006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.938+0000 7f64dcad4700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f64c006d8a0 0x7f64c006fd50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f64c8007900 tx=0x7f64c8008040 comp rx=0 tx=0).stop 2024-02-19T19:19:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.938+0000 7f64dcad4700 1 -- 172.21.15.88:0/3064945792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d8101c90 msgr2=0x7f64d80ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.938+0000 7f64dcad4700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d8101c90 0x7f64d80ff660 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f64cc000c00 tx=0x7f64cc0056a0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.938+0000 7f64dcad4700 1 -- 172.21.15.88:0/3064945792 shutdown_connections 2024-02-19T19:19:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.938+0000 7f64dcad4700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f64c006d8a0 0x7f64c006fd50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.938+0000 7f64dcad4700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f64d81012f0 0x7f64d80ff120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.938+0000 7f64dcad4700 1 --2- 172.21.15.88:0/3064945792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64d8101c90 0x7f64d80ff660 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:53.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.938+0000 7f64dcad4700 1 -- 172.21.15.88:0/3064945792 >> 172.21.15.88:0/3064945792 conn(0x7f64d80fac80 msgr2=0x7f64d8104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:53.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.938+0000 7f64dcad4700 1 -- 172.21.15.88:0/3064945792 shutdown_connections 2024-02-19T19:19:53.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:53.939+0000 7f64dcad4700 1 -- 172.21.15.88:0/3064945792 wait complete. 2024-02-19T19:19:53.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:53 smithi132 ceph-mon[70676]: pgmap v488: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.040+0000 7fde54cd7700 1 -- 172.21.15.88:0/393177777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50101d30 msgr2=0x7fde5010a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.040+0000 7fde54cd7700 1 --2- 172.21.15.88:0/393177777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50101d30 0x7fde5010a230 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fde44009910 tx=0x7fde44009c20 comp rx=0 tx=0).stop 2024-02-19T19:19:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.041+0000 7fde54cd7700 1 -- 172.21.15.88:0/393177777 shutdown_connections 2024-02-19T19:19:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.041+0000 7fde54cd7700 1 --2- 172.21.15.88:0/393177777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50101d30 0x7fde5010a230 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.041+0000 7fde54cd7700 1 --2- 172.21.15.88:0/393177777 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde50101420 0x7fde501017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.041+0000 7fde54cd7700 1 -- 172.21.15.88:0/393177777 >> 172.21.15.88:0/393177777 conn(0x7fde500fac80 msgr2=0x7fde500fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.041+0000 7fde54cd7700 1 -- 172.21.15.88:0/393177777 shutdown_connections 2024-02-19T19:19:54.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.041+0000 7fde54cd7700 1 -- 172.21.15.88:0/393177777 wait complete. 2024-02-19T19:19:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.041+0000 7fde54cd7700 1 Processor -- start 2024-02-19T19:19:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.041+0000 7fde54cd7700 1 -- start start 2024-02-19T19:19:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde54cd7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde50101420 0x7fde50195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde54cd7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50101d30 0x7fde50196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde54cd7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde50196c10 con 0x7fde50101d30 2024-02-19T19:19:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde54cd7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5019a9a0 con 0x7fde50101420 2024-02-19T19:19:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde4dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50101d30 0x7fde50196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde4dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50101d30 0x7fde50196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43984/0 (socket says 172.21.15.88:43984) 2024-02-19T19:19:54.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde4dd9b700 1 -- 172.21.15.88:0/2869854181 learned_addr learned my addr 172.21.15.88:0/2869854181 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:54.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde4e59c700 1 --2- 172.21.15.88:0/2869854181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde50101420 0x7fde50195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:54.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde4dd9b700 1 -- 172.21.15.88:0/2869854181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde50101420 msgr2=0x7fde50195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:54.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde4dd9b700 1 --2- 172.21.15.88:0/2869854181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde50101420 0x7fde50195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde4dd9b700 1 -- 172.21.15.88:0/2869854181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde440095f0 con 0x7fde50101d30 2024-02-19T19:19:54.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde4dd9b700 1 --2- 172.21.15.88:0/2869854181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50101d30 0x7fde50196530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fde44006010 tx=0x7fde4400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:54.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde3f7fe700 1 -- 172.21.15.88:0/2869854181 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde4401d070 con 0x7fde50101d30 2024-02-19T19:19:54.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde3f7fe700 1 -- 172.21.15.88:0/2869854181 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde4400f460 con 0x7fde50101d30 2024-02-19T19:19:54.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde54cd7700 1 -- 172.21.15.88:0/2869854181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5019ac20 con 0x7fde50101d30 2024-02-19T19:19:54.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.042+0000 7fde3f7fe700 1 -- 172.21.15.88:0/2869854181 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde44017610 con 0x7fde50101d30 2024-02-19T19:19:54.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.043+0000 7fde54cd7700 1 -- 172.21.15.88:0/2869854181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5019b110 con 0x7fde50101d30 2024-02-19T19:19:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.044+0000 7fde54cd7700 1 -- 172.21.15.88:0/2869854181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde501078a0 con 0x7fde50101d30 2024-02-19T19:19:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.044+0000 7fde3f7fe700 1 -- 172.21.15.88:0/2869854181 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fde4400f5d0 con 0x7fde50101d30 2024-02-19T19:19:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.044+0000 7fde3f7fe700 1 --2- 172.21.15.88:0/2869854181 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fde3806d8a0 0x7fde3806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.044+0000 7fde3f7fe700 1 -- 172.21.15.88:0/2869854181 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fde4408eab0 con 0x7fde50101d30 2024-02-19T19:19:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.044+0000 7fde4e59c700 1 --2- 172.21.15.88:0/2869854181 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fde3806d8a0 0x7fde3806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.045+0000 7fde4e59c700 1 --2- 172.21.15.88:0/2869854181 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fde3806d8a0 0x7fde3806fd50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fde40009a10 tx=0x7fde40009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:54.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.049+0000 7fde3f7fe700 1 -- 172.21.15.88:0/2869854181 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fde44058260 con 0x7fde50101d30 2024-02-19T19:19:54.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.326+0000 7fde54cd7700 1 -- 172.21.15.88:0/2869854181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fde50107280 con 0x7fde50101d30 2024-02-19T19:19:54.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.327+0000 7fde3f7fe700 1 -- 172.21.15.88:0/2869854181 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fde4405b920 con 0x7fde50101d30 2024-02-19T19:19:54.339 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:19:54.339 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:19:54.339 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:19:54.340 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:19:54.341 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:19:54.341 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:19:54.341 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:19:54.341 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:19:54.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.329+0000 7fde54cd7700 1 -- 172.21.15.88:0/2869854181 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fde3806d8a0 msgr2=0x7fde3806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:54.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.329+0000 7fde54cd7700 1 --2- 172.21.15.88:0/2869854181 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fde3806d8a0 0x7fde3806fd50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fde40009a10 tx=0x7fde40009190 comp rx=0 tx=0).stop 2024-02-19T19:19:54.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.329+0000 7fde54cd7700 1 -- 172.21.15.88:0/2869854181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50101d30 msgr2=0x7fde50196530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:54.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.329+0000 7fde54cd7700 1 --2- 172.21.15.88:0/2869854181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50101d30 0x7fde50196530 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fde44006010 tx=0x7fde4400bd70 comp rx=0 tx=0).stop 2024-02-19T19:19:54.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.329+0000 7fde54cd7700 1 -- 172.21.15.88:0/2869854181 shutdown_connections 2024-02-19T19:19:54.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.329+0000 7fde54cd7700 1 --2- 172.21.15.88:0/2869854181 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fde3806d8a0 0x7fde3806fd50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.329+0000 7fde54cd7700 1 --2- 172.21.15.88:0/2869854181 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fde50101420 0x7fde50195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.329+0000 7fde54cd7700 1 --2- 172.21.15.88:0/2869854181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50101d30 0x7fde50196530 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.329+0000 7fde54cd7700 1 -- 172.21.15.88:0/2869854181 >> 172.21.15.88:0/2869854181 conn(0x7fde500fac80 msgr2=0x7fde501049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:54.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.329+0000 7fde54cd7700 1 -- 172.21.15.88:0/2869854181 shutdown_connections 2024-02-19T19:19:54.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.329+0000 7fde54cd7700 1 -- 172.21.15.88:0/2869854181 wait complete. 2024-02-19T19:19:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.433+0000 7f2a4ad57700 1 -- 172.21.15.88:0/3964797691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a440ffd60 msgr2=0x7f2a44103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.433+0000 7f2a4ad57700 1 --2- 172.21.15.88:0/3964797691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a440ffd60 0x7f2a44103d30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009990 tx=0x7f2a34009ca0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.433+0000 7f2a4ad57700 1 -- 172.21.15.88:0/3964797691 shutdown_connections 2024-02-19T19:19:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.433+0000 7f2a4ad57700 1 --2- 172.21.15.88:0/3964797691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a440ffd60 0x7f2a44103d30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.433+0000 7f2a4ad57700 1 --2- 172.21.15.88:0/3964797691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a440ff3c0 0x7f2a440ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.433+0000 7f2a4ad57700 1 -- 172.21.15.88:0/3964797691 >> 172.21.15.88:0/3964797691 conn(0x7f2a440fac80 msgr2=0x7f2a440fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:54.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.433+0000 7f2a4ad57700 1 -- 172.21.15.88:0/3964797691 shutdown_connections 2024-02-19T19:19:54.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.433+0000 7f2a4ad57700 1 -- 172.21.15.88:0/3964797691 wait complete. 2024-02-19T19:19:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.433+0000 7f2a4ad57700 1 Processor -- start 2024-02-19T19:19:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a4ad57700 1 -- start start 2024-02-19T19:19:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a4ad57700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a440ff3c0 0x7f2a44199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a4ad57700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a440ffd60 0x7f2a4419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a4ad57700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4419a6a0 con 0x7f2a440ffd60 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a4ad57700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a44193b50 con 0x7f2a440ff3c0 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a3bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a440ffd60 0x7f2a4419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a3bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a440ffd60 0x7f2a4419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44008/0 (socket says 172.21.15.88:44008) 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a3bfff700 1 -- 172.21.15.88:0/1029423841 learned_addr learned my addr 172.21.15.88:0/1029423841 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a3bfff700 1 -- 172.21.15.88:0/1029423841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a440ff3c0 msgr2=0x7f2a44199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a3bfff700 1 --2- 172.21.15.88:0/1029423841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a440ff3c0 0x7f2a44199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a3bfff700 1 -- 172.21.15.88:0/1029423841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340095f0 con 0x7f2a440ffd60 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a3bfff700 1 --2- 172.21.15.88:0/1029423841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a440ffd60 0x7f2a4419a010 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2a34000c00 tx=0x7f2a3400e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a39ffb700 1 -- 172.21.15.88:0/1029423841 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a340052e0 con 0x7f2a440ffd60 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a39ffb700 1 -- 172.21.15.88:0/1029423841 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a34003680 con 0x7f2a440ffd60 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a4ad57700 1 -- 172.21.15.88:0/1029423841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a44193dd0 con 0x7f2a440ffd60 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.434+0000 7f2a39ffb700 1 -- 172.21.15.88:0/1029423841 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a34020790 con 0x7f2a440ffd60 2024-02-19T19:19:54.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.435+0000 7f2a4ad57700 1 -- 172.21.15.88:0/1029423841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a441942c0 con 0x7f2a440ffd60 2024-02-19T19:19:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.436+0000 7f2a4ad57700 1 -- 172.21.15.88:0/1029423841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a44107730 con 0x7f2a440ffd60 2024-02-19T19:19:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.437+0000 7f2a39ffb700 1 -- 172.21.15.88:0/1029423841 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2a340037f0 con 0x7f2a440ffd60 2024-02-19T19:19:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.437+0000 7f2a39ffb700 1 --2- 172.21.15.88:0/1029423841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a2c06d8a0 0x7f2a2c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.437+0000 7f2a39ffb700 1 -- 172.21.15.88:0/1029423841 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2a34090e80 con 0x7f2a440ffd60 2024-02-19T19:19:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.437+0000 7f2a48af3700 1 --2- 172.21.15.88:0/1029423841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a2c06d8a0 0x7f2a2c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.438+0000 7f2a48af3700 1 --2- 172.21.15.88:0/1029423841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a2c06d8a0 0x7f2a2c06fd50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2a40005950 tx=0x7f2a400058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:54.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.442+0000 7f2a39ffb700 1 -- 172.21.15.88:0/1029423841 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2a3405e160 con 0x7f2a440ffd60 2024-02-19T19:19:54.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:54 smithi088 ceph-mon[88643]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:54.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:54 smithi088 ceph-mon[88643]: from='client.34922 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:54.592 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:54 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2869854181' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:19:54.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.681+0000 7f2a4ad57700 1 -- 172.21.15.88:0/1029423841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2a4404f2a0 con 0x7f2a440ffd60 2024-02-19T19:19:54.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.682+0000 7f2a39ffb700 1 -- 172.21.15.88:0/1029423841 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f2a34005c00 con 0x7f2a440ffd60 2024-02-19T19:19:54.694 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:19:54.694 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:19:54.694 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:19:54.694 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:19:54.694 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:19:54.694 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:19:54.694 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:19:54.695 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:19:54.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:19:54.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:19:54.697 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:19:54.697 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:19:54.697 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:19:54.697 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:19:54.697 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:19:54.697 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.684+0000 7f2a4ad57700 1 -- 172.21.15.88:0/1029423841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a2c06d8a0 msgr2=0x7f2a2c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.684+0000 7f2a4ad57700 1 --2- 172.21.15.88:0/1029423841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a2c06d8a0 0x7f2a2c06fd50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2a40005950 tx=0x7f2a400058e0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.684+0000 7f2a4ad57700 1 -- 172.21.15.88:0/1029423841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a440ffd60 msgr2=0x7f2a4419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.684+0000 7f2a4ad57700 1 --2- 172.21.15.88:0/1029423841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a440ffd60 0x7f2a4419a010 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2a34000c00 tx=0x7f2a3400e7c0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.684+0000 7f2a4ad57700 1 -- 172.21.15.88:0/1029423841 shutdown_connections 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.684+0000 7f2a4ad57700 1 --2- 172.21.15.88:0/1029423841 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2a2c06d8a0 0x7f2a2c06fd50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.684+0000 7f2a4ad57700 1 --2- 172.21.15.88:0/1029423841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2a440ff3c0 0x7f2a44199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.684+0000 7f2a4ad57700 1 --2- 172.21.15.88:0/1029423841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a440ffd60 0x7f2a4419a010 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.684+0000 7f2a4ad57700 1 -- 172.21.15.88:0/1029423841 >> 172.21.15.88:0/1029423841 conn(0x7f2a440fac80 msgr2=0x7f2a440fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.684+0000 7f2a4ad57700 1 -- 172.21.15.88:0/1029423841 shutdown_connections 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.684+0000 7f2a4ad57700 1 -- 172.21.15.88:0/1029423841 wait complete. 2024-02-19T19:19:54.698 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:19:54.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.789+0000 7f97cd838700 1 -- 172.21.15.88:0/3912486231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97c8100e80 msgr2=0x7f97c8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:54.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.789+0000 7f97cd838700 1 --2- 172.21.15.88:0/3912486231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97c8100e80 0x7f97c8104e50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009910 tx=0x7f97bc009c20 comp rx=0 tx=0).stop 2024-02-19T19:19:54.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.789+0000 7f97cd838700 1 -- 172.21.15.88:0/3912486231 shutdown_connections 2024-02-19T19:19:54.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.789+0000 7f97cd838700 1 --2- 172.21.15.88:0/3912486231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97c8100e80 0x7f97c8104e50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.789+0000 7f97cd838700 1 --2- 172.21.15.88:0/3912486231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c81004e0 0x7f97c81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.789+0000 7f97cd838700 1 -- 172.21.15.88:0/3912486231 >> 172.21.15.88:0/3912486231 conn(0x7f97c80fbda0 msgr2=0x7f97c80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:54.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.789+0000 7f97cd838700 1 -- 172.21.15.88:0/3912486231 shutdown_connections 2024-02-19T19:19:54.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.789+0000 7f97cd838700 1 -- 172.21.15.88:0/3912486231 wait complete. 2024-02-19T19:19:54.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.789+0000 7f97cd838700 1 Processor -- start 2024-02-19T19:19:54.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97cd838700 1 -- start start 2024-02-19T19:19:54.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97cd838700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97c81004e0 0x7f97c8195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:54.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97cd838700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c8100e80 0x7f97c8196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:54.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97cd838700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8196b30 con 0x7f97c81004e0 2024-02-19T19:19:54.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97cd838700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c819a8c0 con 0x7f97c8100e80 2024-02-19T19:19:54.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97c6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97c81004e0 0x7f97c8195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:54.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97c6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97c81004e0 0x7f97c8195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44024/0 (socket says 172.21.15.88:44024) 2024-02-19T19:19:54.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97c6ffd700 1 -- 172.21.15.88:0/2343167366 learned_addr learned my addr 172.21.15.88:0/2343167366 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:54.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97c6ffd700 1 -- 172.21.15.88:0/2343167366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c8100e80 msgr2=0x7f97c8196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:54.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97c6ffd700 1 --2- 172.21.15.88:0/2343167366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c8100e80 0x7f97c8196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:54.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97c6ffd700 1 -- 172.21.15.88:0/2343167366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97bc0095f0 con 0x7f97c81004e0 2024-02-19T19:19:54.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97c6ffd700 1 --2- 172.21.15.88:0/2343167366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97c81004e0 0x7f97c8195f10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f97b800d8d0 tx=0x7f97b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:54.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.790+0000 7f97cc836700 1 -- 172.21.15.88:0/2343167366 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97b8009940 con 0x7f97c81004e0 2024-02-19T19:19:54.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.791+0000 7f97cd838700 1 -- 172.21.15.88:0/2343167366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c819aba0 con 0x7f97c81004e0 2024-02-19T19:19:54.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.791+0000 7f97cc836700 1 -- 172.21.15.88:0/2343167366 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97b8010460 con 0x7f97c81004e0 2024-02-19T19:19:54.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.791+0000 7f97cc836700 1 -- 172.21.15.88:0/2343167366 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97b800f5f0 con 0x7f97c81004e0 2024-02-19T19:19:54.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.791+0000 7f97cd838700 1 -- 172.21.15.88:0/2343167366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c819b0f0 con 0x7f97c81004e0 2024-02-19T19:19:54.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.792+0000 7f97cd838700 1 -- 172.21.15.88:0/2343167366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c8108850 con 0x7f97c81004e0 2024-02-19T19:19:54.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.792+0000 7f97cc836700 1 -- 172.21.15.88:0/2343167366 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f97b80105d0 con 0x7f97c81004e0 2024-02-19T19:19:54.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.793+0000 7f97cc836700 1 --2- 172.21.15.88:0/2343167366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f97b006d8a0 0x7f97b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:54.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.793+0000 7f97cc836700 1 -- 172.21.15.88:0/2343167366 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f97b808d540 con 0x7f97c81004e0 2024-02-19T19:19:54.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.793+0000 7f97c67fc700 1 --2- 172.21.15.88:0/2343167366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f97b006d8a0 0x7f97b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:54.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.793+0000 7f97c67fc700 1 --2- 172.21.15.88:0/2343167366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f97b006d8a0 0x7f97b006fd50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f97bc006010 tx=0x7f97bc00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:54.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:54.797+0000 7f97cc836700 1 -- 172.21.15.88:0/2343167366 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f97b8056cf0 con 0x7f97c81004e0 2024-02-19T19:19:54.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:54 smithi132 ceph-mon[70676]: from='client.34918 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:54.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:54 smithi132 ceph-mon[70676]: from='client.34922 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:54.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:54 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2869854181' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:19:55.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.015+0000 7f97cd838700 1 -- 172.21.15.88:0/2343167366 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f97c8197380 con 0x7f97b006d8a0 2024-02-19T19:19:55.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.017+0000 7f97cc836700 1 -- 172.21.15.88:0/2343167366 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f97c8197380 con 0x7f97b006d8a0 2024-02-19T19:19:55.029 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:19:55.029 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:19:55.029 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:19:55.029 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:19:55.029 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:19:55.030 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:19:55.030 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:19:55.030 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:19:55.030 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:19:55.030 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:19:55.030 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:19:55.030 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:19:55.030 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:19:55.030 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:19:55.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.020+0000 7f97cd838700 1 -- 172.21.15.88:0/2343167366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f97b006d8a0 msgr2=0x7f97b006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:55.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.020+0000 7f97cd838700 1 --2- 172.21.15.88:0/2343167366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f97b006d8a0 0x7f97b006fd50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f97bc006010 tx=0x7f97bc00b540 comp rx=0 tx=0).stop 2024-02-19T19:19:55.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.020+0000 7f97cd838700 1 -- 172.21.15.88:0/2343167366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97c81004e0 msgr2=0x7f97c8195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:55.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.020+0000 7f97cd838700 1 --2- 172.21.15.88:0/2343167366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97c81004e0 0x7f97c8195f10 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f97b800d8d0 tx=0x7f97b800dc90 comp rx=0 tx=0).stop 2024-02-19T19:19:55.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.020+0000 7f97cd838700 1 -- 172.21.15.88:0/2343167366 shutdown_connections 2024-02-19T19:19:55.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.020+0000 7f97cd838700 1 --2- 172.21.15.88:0/2343167366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f97b006d8a0 0x7f97b006fd50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:55.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.020+0000 7f97cd838700 1 --2- 172.21.15.88:0/2343167366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97c81004e0 0x7f97c8195f10 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:55.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.020+0000 7f97cd838700 1 --2- 172.21.15.88:0/2343167366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f97c8100e80 0x7f97c8196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:55.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.020+0000 7f97cd838700 1 -- 172.21.15.88:0/2343167366 >> 172.21.15.88:0/2343167366 conn(0x7f97c80fbda0 msgr2=0x7f97c80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:55.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.020+0000 7f97cd838700 1 -- 172.21.15.88:0/2343167366 shutdown_connections 2024-02-19T19:19:55.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.020+0000 7f97cd838700 1 -- 172.21.15.88:0/2343167366 wait complete. 2024-02-19T19:19:55.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.125+0000 7fb727782700 1 -- 172.21.15.88:0/1098896174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7200ff030 msgr2=0x7fb7200ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:55.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.125+0000 7fb727782700 1 --2- 172.21.15.88:0/1098896174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7200ff030 0x7fb7200ff400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009910 tx=0x7fb71c009c20 comp rx=0 tx=0).stop 2024-02-19T19:19:55.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.125+0000 7fb727782700 1 -- 172.21.15.88:0/1098896174 shutdown_connections 2024-02-19T19:19:55.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.125+0000 7fb727782700 1 --2- 172.21.15.88:0/1098896174 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7200ff940 0x7fb72010c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:55.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.125+0000 7fb727782700 1 --2- 172.21.15.88:0/1098896174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7200ff030 0x7fb7200ff400 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:55.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.125+0000 7fb727782700 1 -- 172.21.15.88:0/1098896174 >> 172.21.15.88:0/1098896174 conn(0x7fb7200fac80 msgr2=0x7fb7200fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:55.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.126+0000 7fb727782700 1 -- 172.21.15.88:0/1098896174 shutdown_connections 2024-02-19T19:19:55.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.126+0000 7fb727782700 1 -- 172.21.15.88:0/1098896174 wait complete. 2024-02-19T19:19:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.126+0000 7fb727782700 1 Processor -- start 2024-02-19T19:19:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.126+0000 7fb727782700 1 -- start start 2024-02-19T19:19:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb727782700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7200ff030 0x7fb720198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb727782700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7200ff940 0x7fb7201985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb727782700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb720198cb0 con 0x7fb7200ff030 2024-02-19T19:19:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb727782700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72019ca40 con 0x7fb7200ff940 2024-02-19T19:19:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb72551e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7200ff030 0x7fb720198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb72551e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7200ff030 0x7fb720198090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44040/0 (socket says 172.21.15.88:44040) 2024-02-19T19:19:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb72551e700 1 -- 172.21.15.88:0/2320414843 learned_addr learned my addr 172.21.15.88:0/2320414843 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:19:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb72551e700 1 -- 172.21.15.88:0/2320414843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7200ff940 msgr2=0x7fb7201985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb72551e700 1 --2- 172.21.15.88:0/2320414843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7200ff940 0x7fb7201985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb72551e700 1 -- 172.21.15.88:0/2320414843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c0095f0 con 0x7fb7200ff030 2024-02-19T19:19:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb72551e700 1 --2- 172.21.15.88:0/2320414843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7200ff030 0x7fb720198090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb71c006010 tx=0x7fb71c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb7167fc700 1 -- 172.21.15.88:0/2320414843 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb71c01d070 con 0x7fb7200ff030 2024-02-19T19:19:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb727782700 1 -- 172.21.15.88:0/2320414843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb72019ccc0 con 0x7fb7200ff030 2024-02-19T19:19:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb7167fc700 1 -- 172.21.15.88:0/2320414843 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb71c00f460 con 0x7fb7200ff030 2024-02-19T19:19:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb7167fc700 1 -- 172.21.15.88:0/2320414843 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb71c017630 con 0x7fb7200ff030 2024-02-19T19:19:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.127+0000 7fb727782700 1 -- 172.21.15.88:0/2320414843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72019d1b0 con 0x7fb7200ff030 2024-02-19T19:19:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.128+0000 7fb727782700 1 -- 172.21.15.88:0/2320414843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb720109e10 con 0x7fb7200ff030 2024-02-19T19:19:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.129+0000 7fb7167fc700 1 -- 172.21.15.88:0/2320414843 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb71c00f5d0 con 0x7fb7200ff030 2024-02-19T19:19:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.129+0000 7fb7167fc700 1 --2- 172.21.15.88:0/2320414843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb70c06d8a0 0x7fb70c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:19:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.129+0000 7fb724d1d700 1 --2- 172.21.15.88:0/2320414843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb70c06d8a0 0x7fb70c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:19:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.129+0000 7fb7167fc700 1 -- 172.21.15.88:0/2320414843 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb71c08e9f0 con 0x7fb7200ff030 2024-02-19T19:19:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.130+0000 7fb724d1d700 1 --2- 172.21.15.88:0/2320414843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb70c06d8a0 0x7fb70c06fd50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb710005950 tx=0x7fb7100058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:19:55.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.134+0000 7fb7167fc700 1 -- 172.21.15.88:0/2320414843 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb71c090e60 con 0x7fb7200ff030 2024-02-19T19:19:55.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.416+0000 7fb727782700 1 -- 172.21.15.88:0/2320414843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb72004f2a0 con 0x7fb7200ff030 2024-02-19T19:19:55.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.416+0000 7fb7167fc700 1 -- 172.21.15.88:0/2320414843 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb71c0581a0 con 0x7fb7200ff030 2024-02-19T19:19:55.428 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:19:55.428 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:19:55.428 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:19:55.429 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:19:55.429 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:19:55.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.418+0000 7fb727782700 1 -- 172.21.15.88:0/2320414843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb70c06d8a0 msgr2=0x7fb70c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:55.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.418+0000 7fb727782700 1 --2- 172.21.15.88:0/2320414843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb70c06d8a0 0x7fb70c06fd50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb710005950 tx=0x7fb7100058e0 comp rx=0 tx=0).stop 2024-02-19T19:19:55.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.418+0000 7fb727782700 1 -- 172.21.15.88:0/2320414843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7200ff030 msgr2=0x7fb720198090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:19:55.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.418+0000 7fb727782700 1 --2- 172.21.15.88:0/2320414843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7200ff030 0x7fb720198090 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fb71c006010 tx=0x7fb71c00be20 comp rx=0 tx=0).stop 2024-02-19T19:19:55.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.418+0000 7fb727782700 1 -- 172.21.15.88:0/2320414843 shutdown_connections 2024-02-19T19:19:55.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.418+0000 7fb727782700 1 --2- 172.21.15.88:0/2320414843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb70c06d8a0 0x7fb70c06fd50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:55.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.418+0000 7fb727782700 1 --2- 172.21.15.88:0/2320414843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7200ff030 0x7fb720198090 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:55.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.418+0000 7fb727782700 1 --2- 172.21.15.88:0/2320414843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb7200ff940 0x7fb7201985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:19:55.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.418+0000 7fb727782700 1 -- 172.21.15.88:0/2320414843 >> 172.21.15.88:0/2320414843 conn(0x7fb7200fac80 msgr2=0x7fb7200fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:19:55.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.418+0000 7fb727782700 1 -- 172.21.15.88:0/2320414843 shutdown_connections 2024-02-19T19:19:55.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:19:55.418+0000 7fb727782700 1 -- 172.21.15.88:0/2320414843 wait complete. 2024-02-19T19:19:55.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:55 smithi088 ceph-mon[88643]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:55.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:55 smithi088 ceph-mon[88643]: pgmap v489: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:55.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:55 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1029423841' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:19:55.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:55 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2320414843' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:19:55.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:55 smithi132 ceph-mon[70676]: from='client.34926 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:55.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:55 smithi132 ceph-mon[70676]: pgmap v489: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:55.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:55 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1029423841' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:19:55.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:55 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2320414843' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:19:56.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:56 smithi088 ceph-mon[88643]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:56.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:56 smithi132 ceph-mon[70676]: from='client.34938 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:19:57.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:57 smithi088 ceph-mon[88643]: pgmap v490: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:57.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:57 smithi132 ceph-mon[70676]: pgmap v490: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:58.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:19:58.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:19:59.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:19:59 smithi088 ceph-mon[88643]: pgmap v491: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:19:59.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:19:59 smithi132 ceph-mon[70676]: pgmap v491: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:00.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:19:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:20:00.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:19:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:20:00.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:19:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-02-19T19:20:00.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:19:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:20:00.342 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:19:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-02-19T19:20:00.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:00 smithi132 ceph-mon[70676]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:20:00.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:00 smithi132 ceph-mon[70676]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:20:00.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:00 smithi132 ceph-mon[70676]: fs cephfs is degraded 2024-02-19T19:20:00.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:00 smithi132 ceph-mon[70676]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:20:00.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:00 smithi132 ceph-mon[70676]: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:20:01.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:00 smithi088 ceph-mon[88643]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:20:01.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:00 smithi088 ceph-mon[88643]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:20:01.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:00 smithi088 ceph-mon[88643]: fs cephfs is degraded 2024-02-19T19:20:01.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:00 smithi088 ceph-mon[88643]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:20:01.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:00 smithi088 ceph-mon[88643]: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:20:01.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:01 smithi132 ceph-mon[70676]: pgmap v492: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:02.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:01 smithi088 ceph-mon[88643]: pgmap v492: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:03.922 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:03 smithi088 ceph-mon[88643]: pgmap v493: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:03.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:03 smithi132 ceph-mon[70676]: pgmap v493: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:05.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:05 smithi132 ceph-mon[70676]: pgmap v494: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:06.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:05 smithi088 ceph-mon[88643]: pgmap v494: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:07.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:07 smithi132 ceph-mon[70676]: pgmap v495: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:08.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:07 smithi088 ceph-mon[88643]: pgmap v495: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:09.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:09 smithi132 ceph-mon[70676]: pgmap v496: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:10.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:09 smithi088 ceph-mon[88643]: pgmap v496: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:10.994 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:10 smithi132 ceph-mon[70676]: pgmap v497: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:11.092 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:10 smithi088 ceph-mon[88643]: pgmap v497: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:13.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:13 smithi132 ceph-mon[70676]: pgmap v498: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:13.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:20:13.795 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:13 smithi088 ceph-mon[88643]: pgmap v498: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:13.795 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:20:15.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:15 smithi132 ceph-mon[70676]: pgmap v499: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:15.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:15 smithi088 ceph-mon[88643]: pgmap v499: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:17.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:17 smithi132 ceph-mon[70676]: pgmap v500: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:17.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:17 smithi088 ceph-mon[88643]: pgmap v500: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:19.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:19 smithi132 ceph-mon[70676]: pgmap v501: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:19.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:19 smithi088 ceph-mon[88643]: pgmap v501: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:21.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:21 smithi132 ceph-mon[70676]: pgmap v502: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:21.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:21 smithi088 ceph-mon[88643]: pgmap v502: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:23.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:23 smithi132 ceph-mon[70676]: pgmap v503: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:23.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:23 smithi088 ceph-mon[88643]: pgmap v503: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:25.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.535+0000 7f1c3ca13700 1 -- 172.21.15.88:0/1814127235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c38101d30 msgr2=0x7f1c3810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:25.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.535+0000 7f1c3ca13700 1 --2- 172.21.15.88:0/1814127235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c38101d30 0x7f1c3810a230 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c009910 tx=0x7f1c2c009c20 comp rx=0 tx=0).stop 2024-02-19T19:20:25.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.536+0000 7f1c3ca13700 1 -- 172.21.15.88:0/1814127235 shutdown_connections 2024-02-19T19:20:25.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.536+0000 7f1c3ca13700 1 --2- 172.21.15.88:0/1814127235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c38101d30 0x7f1c3810a230 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:25.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.536+0000 7f1c3ca13700 1 --2- 172.21.15.88:0/1814127235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c38101420 0x7f1c381017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:25.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.536+0000 7f1c3ca13700 1 -- 172.21.15.88:0/1814127235 >> 172.21.15.88:0/1814127235 conn(0x7f1c380fac80 msgr2=0x7f1c380fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:25.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.536+0000 7f1c3ca13700 1 -- 172.21.15.88:0/1814127235 shutdown_connections 2024-02-19T19:20:25.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.536+0000 7f1c3ca13700 1 -- 172.21.15.88:0/1814127235 wait complete. 2024-02-19T19:20:25.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.536+0000 7f1c3ca13700 1 Processor -- start 2024-02-19T19:20:25.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.536+0000 7f1c3ca13700 1 -- start start 2024-02-19T19:20:25.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c3ca13700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c38101420 0x7f1c380ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:25.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c3ca13700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c38101d30 0x7f1c380ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:25.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c3ca13700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c380ffb00 con 0x7f1c38101d30 2024-02-19T19:20:25.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c3ca13700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c380ffc40 con 0x7f1c38101420 2024-02-19T19:20:25.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c35d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c38101d30 0x7f1c380ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:25.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c35d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c38101d30 0x7f1c380ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53534/0 (socket says 172.21.15.88:53534) 2024-02-19T19:20:25.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c35d9b700 1 -- 172.21.15.88:0/701121746 learned_addr learned my addr 172.21.15.88:0/701121746 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:25.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c3659c700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c38101420 0x7f1c380ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c35d9b700 1 -- 172.21.15.88:0/701121746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c38101420 msgr2=0x7f1c380ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c35d9b700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c38101420 0x7f1c380ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c35d9b700 1 -- 172.21.15.88:0/701121746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c0095f0 con 0x7f1c38101d30 2024-02-19T19:20:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c3659c700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c38101420 0x7f1c380ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:20:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c35d9b700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c38101d30 0x7f1c380ff5c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c006010 tx=0x7f1c2c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c277fe700 1 -- 172.21.15.88:0/701121746 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c2c01d070 con 0x7f1c38101d30 2024-02-19T19:20:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c3ca13700 1 -- 172.21.15.88:0/701121746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c380ffec0 con 0x7f1c38101d30 2024-02-19T19:20:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c277fe700 1 -- 172.21.15.88:0/701121746 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c2c004fb0 con 0x7f1c38101d30 2024-02-19T19:20:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.537+0000 7f1c277fe700 1 -- 172.21.15.88:0/701121746 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c2c022ea0 con 0x7f1c38101d30 2024-02-19T19:20:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.538+0000 7f1c3ca13700 1 -- 172.21.15.88:0/701121746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c381a24f0 con 0x7f1c38101d30 2024-02-19T19:20:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.539+0000 7f1c3ca13700 1 -- 172.21.15.88:0/701121746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c381078a0 con 0x7f1c38101d30 2024-02-19T19:20:25.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.539+0000 7f1c277fe700 1 -- 172.21.15.88:0/701121746 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1c2c022470 con 0x7f1c38101d30 2024-02-19T19:20:25.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.539+0000 7f1c277fe700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c2006d8a0 0x7f1c2006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:25.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.540+0000 7f1c277fe700 1 -- 172.21.15.88:0/701121746 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1c2c08ecd0 con 0x7f1c38101d30 2024-02-19T19:20:25.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.540+0000 7f1c3659c700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c2006d8a0 0x7f1c2006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:25.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.540+0000 7f1c3659c700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c2006d8a0 0x7f1c2006fd50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c280099d0 tx=0x7f1c28008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:25.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.544+0000 7f1c277fe700 1 -- 172.21.15.88:0/701121746 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c2c058650 con 0x7f1c38101d30 2024-02-19T19:20:25.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:25 smithi132 ceph-mon[70676]: pgmap v504: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:25.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.762+0000 7f1c3ca13700 1 -- 172.21.15.88:0/701121746 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c381006c0 con 0x7f1c2006d8a0 2024-02-19T19:20:25.774 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:25 smithi088 ceph-mon[88643]: pgmap v504: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:25.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.764+0000 7f1c277fe700 1 -- 172.21.15.88:0/701121746 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1c381006c0 con 0x7f1c2006d8a0 2024-02-19T19:20:25.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.766+0000 7f1c3ca13700 1 -- 172.21.15.88:0/701121746 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c2006d8a0 msgr2=0x7f1c2006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:25.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.766+0000 7f1c3ca13700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c2006d8a0 0x7f1c2006fd50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f1c280099d0 tx=0x7f1c28008040 comp rx=0 tx=0).stop 2024-02-19T19:20:25.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.766+0000 7f1c3ca13700 1 -- 172.21.15.88:0/701121746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c38101d30 msgr2=0x7f1c380ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:25.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.766+0000 7f1c3ca13700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c38101d30 0x7f1c380ff5c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c006010 tx=0x7f1c2c00be20 comp rx=0 tx=0).stop 2024-02-19T19:20:25.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.766+0000 7f1c3ca13700 1 -- 172.21.15.88:0/701121746 shutdown_connections 2024-02-19T19:20:25.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.766+0000 7f1c3ca13700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c2006d8a0 0x7f1c2006fd50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:25.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.766+0000 7f1c3ca13700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c38101420 0x7f1c380ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:25.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.766+0000 7f1c3ca13700 1 --2- 172.21.15.88:0/701121746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c38101d30 0x7f1c380ff5c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:25.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.766+0000 7f1c3ca13700 1 -- 172.21.15.88:0/701121746 >> 172.21.15.88:0/701121746 conn(0x7f1c380fac80 msgr2=0x7f1c381049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:25.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.766+0000 7f1c3ca13700 1 -- 172.21.15.88:0/701121746 shutdown_connections 2024-02-19T19:20:25.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.766+0000 7f1c3ca13700 1 -- 172.21.15.88:0/701121746 wait complete. 2024-02-19T19:20:25.788 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:20:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.869+0000 7f61eae24700 1 -- 172.21.15.88:0/2400902116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61e4104130 msgr2=0x7f61e4104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.869+0000 7f61eae24700 1 --2- 172.21.15.88:0/2400902116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61e4104130 0x7f61e4104500 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009910 tx=0x7f61d8009c20 comp rx=0 tx=0).stop 2024-02-19T19:20:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.869+0000 7f61eae24700 1 -- 172.21.15.88:0/2400902116 shutdown_connections 2024-02-19T19:20:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.869+0000 7f61eae24700 1 --2- 172.21.15.88:0/2400902116 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61e40ffbd0 0x7f61e4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.869+0000 7f61eae24700 1 --2- 172.21.15.88:0/2400902116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61e4104130 0x7f61e4104500 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.869+0000 7f61eae24700 1 -- 172.21.15.88:0/2400902116 >> 172.21.15.88:0/2400902116 conn(0x7f61e4075130 msgr2=0x7f61e4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.870+0000 7f61eae24700 1 -- 172.21.15.88:0/2400902116 shutdown_connections 2024-02-19T19:20:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.870+0000 7f61eae24700 1 -- 172.21.15.88:0/2400902116 wait complete. 2024-02-19T19:20:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.870+0000 7f61eae24700 1 Processor -- start 2024-02-19T19:20:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.870+0000 7f61eae24700 1 -- start start 2024-02-19T19:20:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.870+0000 7f61eae24700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61e40ffbd0 0x7f61e4100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.870+0000 7f61eae24700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61e4104130 0x7f61e4101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.870+0000 7f61eae24700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e41018b0 con 0x7f61e4104130 2024-02-19T19:20:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.870+0000 7f61eae24700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e41019f0 con 0x7f61e40ffbd0 2024-02-19T19:20:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.870+0000 7f61e3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61e4104130 0x7f61e4101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.870+0000 7f61e3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61e4104130 0x7f61e4101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53552/0 (socket says 172.21.15.88:53552) 2024-02-19T19:20:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.870+0000 7f61e3fff700 1 -- 172.21.15.88:0/779244117 learned_addr learned my addr 172.21.15.88:0/779244117 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.871+0000 7f61e8bc0700 1 --2- 172.21.15.88:0/779244117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61e40ffbd0 0x7f61e4100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.871+0000 7f61e3fff700 1 -- 172.21.15.88:0/779244117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61e40ffbd0 msgr2=0x7f61e4100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.871+0000 7f61e3fff700 1 --2- 172.21.15.88:0/779244117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61e40ffbd0 0x7f61e4100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.871+0000 7f61e3fff700 1 -- 172.21.15.88:0/779244117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d80095f0 con 0x7f61e4104130 2024-02-19T19:20:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.871+0000 7f61e3fff700 1 --2- 172.21.15.88:0/779244117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61e4104130 0x7f61e4101370 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f61d000b4b0 tx=0x7f61d000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.871+0000 7f61e1ffb700 1 -- 172.21.15.88:0/779244117 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61d0011800 con 0x7f61e4104130 2024-02-19T19:20:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.871+0000 7f61eae24700 1 -- 172.21.15.88:0/779244117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e41a2130 con 0x7f61e4104130 2024-02-19T19:20:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.871+0000 7f61e1ffb700 1 -- 172.21.15.88:0/779244117 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61d0011e20 con 0x7f61e4104130 2024-02-19T19:20:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.871+0000 7f61e1ffb700 1 -- 172.21.15.88:0/779244117 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61d000f2e0 con 0x7f61e4104130 2024-02-19T19:20:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.871+0000 7f61eae24700 1 -- 172.21.15.88:0/779244117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e41a2540 con 0x7f61e4104130 2024-02-19T19:20:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.872+0000 7f61eae24700 1 -- 172.21.15.88:0/779244117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e41086a0 con 0x7f61e4104130 2024-02-19T19:20:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.873+0000 7f61e1ffb700 1 -- 172.21.15.88:0/779244117 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f61d0010380 con 0x7f61e4104130 2024-02-19T19:20:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.873+0000 7f61e1ffb700 1 --2- 172.21.15.88:0/779244117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61d406d8a0 0x7f61d406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.873+0000 7f61e1ffb700 1 -- 172.21.15.88:0/779244117 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f61d008d110 con 0x7f61e4104130 2024-02-19T19:20:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.873+0000 7f61e8bc0700 1 --2- 172.21.15.88:0/779244117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61d406d8a0 0x7f61d406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.874+0000 7f61e8bc0700 1 --2- 172.21.15.88:0/779244117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61d406d8a0 0x7f61d406fd50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f61d800b5c0 tx=0x7f61d801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:25.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:25.878+0000 7f61e1ffb700 1 -- 172.21.15.88:0/779244117 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f61d00568c0 con 0x7f61e4104130 2024-02-19T19:20:26.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.093+0000 7f61eae24700 1 -- 172.21.15.88:0/779244117 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f61e402cca0 con 0x7f61d406d8a0 2024-02-19T19:20:26.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.094+0000 7f61e1ffb700 1 -- 172.21.15.88:0/779244117 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f61e402cca0 con 0x7f61d406d8a0 2024-02-19T19:20:26.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.096+0000 7f61eae24700 1 -- 172.21.15.88:0/779244117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61d406d8a0 msgr2=0x7f61d406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.096+0000 7f61eae24700 1 --2- 172.21.15.88:0/779244117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61d406d8a0 0x7f61d406fd50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f61d800b5c0 tx=0x7f61d801a040 comp rx=0 tx=0).stop 2024-02-19T19:20:26.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.096+0000 7f61eae24700 1 -- 172.21.15.88:0/779244117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61e4104130 msgr2=0x7f61e4101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.096+0000 7f61eae24700 1 --2- 172.21.15.88:0/779244117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61e4104130 0x7f61e4101370 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f61d000b4b0 tx=0x7f61d000b870 comp rx=0 tx=0).stop 2024-02-19T19:20:26.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.096+0000 7f61eae24700 1 -- 172.21.15.88:0/779244117 shutdown_connections 2024-02-19T19:20:26.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.096+0000 7f61eae24700 1 --2- 172.21.15.88:0/779244117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61d406d8a0 0x7f61d406fd50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.096+0000 7f61eae24700 1 --2- 172.21.15.88:0/779244117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61e40ffbd0 0x7f61e4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.096+0000 7f61eae24700 1 --2- 172.21.15.88:0/779244117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61e4104130 0x7f61e4101370 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.096+0000 7f61eae24700 1 -- 172.21.15.88:0/779244117 >> 172.21.15.88:0/779244117 conn(0x7f61e4075130 msgr2=0x7f61e40fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:26.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.096+0000 7f61eae24700 1 -- 172.21.15.88:0/779244117 shutdown_connections 2024-02-19T19:20:26.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.096+0000 7f61eae24700 1 -- 172.21.15.88:0/779244117 wait complete. 2024-02-19T19:20:26.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.200+0000 7f3419654700 1 -- 172.21.15.88:0/3530301075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414073090 msgr2=0x7f3414073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.200+0000 7f3419654700 1 --2- 172.21.15.88:0/3530301075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414073090 0x7f3414073460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3404009940 tx=0x7f3404009c50 comp rx=0 tx=0).stop 2024-02-19T19:20:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.200+0000 7f3419654700 1 -- 172.21.15.88:0/3530301075 shutdown_connections 2024-02-19T19:20:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.200+0000 7f3419654700 1 --2- 172.21.15.88:0/3530301075 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34140739a0 0x7f341410c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.200+0000 7f3419654700 1 --2- 172.21.15.88:0/3530301075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414073090 0x7f3414073460 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.200+0000 7f3419654700 1 -- 172.21.15.88:0/3530301075 >> 172.21.15.88:0/3530301075 conn(0x7f34140fbda0 msgr2=0x7f34140fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.200+0000 7f3419654700 1 -- 172.21.15.88:0/3530301075 shutdown_connections 2024-02-19T19:20:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.200+0000 7f3419654700 1 -- 172.21.15.88:0/3530301075 wait complete. 2024-02-19T19:20:26.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.200+0000 7f3419654700 1 Processor -- start 2024-02-19T19:20:26.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f3419654700 1 -- start start 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f3419654700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3414073090 0x7f3414198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f3419654700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34140739a0 0x7f34141986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f3419654700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3414198da0 con 0x7f34140739a0 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f3419654700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341419cb30 con 0x7f3414073090 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f340a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34140739a0 0x7f34141986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f340a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34140739a0 0x7f34141986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53576/0 (socket says 172.21.15.88:53576) 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f340a7fc700 1 -- 172.21.15.88:0/4051801991 learned_addr learned my addr 172.21.15.88:0/4051801991 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f340a7fc700 1 -- 172.21.15.88:0/4051801991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3414073090 msgr2=0x7f3414198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f340a7fc700 1 --2- 172.21.15.88:0/4051801991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3414073090 0x7f3414198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f340a7fc700 1 -- 172.21.15.88:0/4051801991 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34040095f0 con 0x7f34140739a0 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.201+0000 7f340a7fc700 1 --2- 172.21.15.88:0/4051801991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34140739a0 0x7f34141986c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00d8d0 tx=0x7f33fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.202+0000 7f3410ff9700 1 -- 172.21.15.88:0/4051801991 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33fc009940 con 0x7f34140739a0 2024-02-19T19:20:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.202+0000 7f3410ff9700 1 -- 172.21.15.88:0/4051801991 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f33fc010460 con 0x7f34140739a0 2024-02-19T19:20:26.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.202+0000 7f3410ff9700 1 -- 172.21.15.88:0/4051801991 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33fc00f5f0 con 0x7f34140739a0 2024-02-19T19:20:26.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.202+0000 7f3419654700 1 -- 172.21.15.88:0/4051801991 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341419ce10 con 0x7f34140739a0 2024-02-19T19:20:26.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.202+0000 7f3419654700 1 -- 172.21.15.88:0/4051801991 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341419d360 con 0x7f34140739a0 2024-02-19T19:20:26.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.203+0000 7f3419654700 1 -- 172.21.15.88:0/4051801991 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3414109ce0 con 0x7f34140739a0 2024-02-19T19:20:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.203+0000 7f3410ff9700 1 -- 172.21.15.88:0/4051801991 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f33fc0105d0 con 0x7f34140739a0 2024-02-19T19:20:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.203+0000 7f3410ff9700 1 --2- 172.21.15.88:0/4051801991 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340006d8a0 0x7f340006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.204+0000 7f3410ff9700 1 -- 172.21.15.88:0/4051801991 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f33fc08d5a0 con 0x7f34140739a0 2024-02-19T19:20:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.204+0000 7f3412ffd700 1 --2- 172.21.15.88:0/4051801991 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340006d8a0 0x7f340006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.204+0000 7f3412ffd700 1 --2- 172.21.15.88:0/4051801991 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340006d8a0 0x7f340006fd50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3404006010 tx=0x7f3404000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:26.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.208+0000 7f3410ff9700 1 -- 172.21.15.88:0/4051801991 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33fc056cf0 con 0x7f34140739a0 2024-02-19T19:20:26.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.415+0000 7f3419654700 1 -- 172.21.15.88:0/4051801991 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f34141995f0 con 0x7f340006d8a0 2024-02-19T19:20:26.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.428+0000 7f3410ff9700 1 -- 172.21.15.88:0/4051801991 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f34141995f0 con 0x7f340006d8a0 2024-02-19T19:20:26.439 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (17m) 9m ago 28m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (28m) 9m ago 28m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (10m) 9m ago 26m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (13m) 9m ago 28m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (13m) 9m ago 26m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (16m) 9m ago 27m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (9m) 9m ago 21m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (9m) 9m ago 21m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (9m) 9m ago 21m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (21m) 9m ago 21m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (19m) 9m ago 29m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:20:26.440 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (18m) 9m ago 26m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:20:26.441 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (15m) 9m ago 29m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:20:26.441 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (14m) 9m ago 26m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:20:26.441 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (18m) 9m ago 28m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:20:26.441 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (18m) 9m ago 26m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:20:26.441 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (13m) 9m ago 25m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:20:26.441 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (12m) 9m ago 25m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:20:26.441 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (11m) 9m ago 24m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:20:26.441 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (11m) 9m ago 24m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:20:26.441 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (10m) 9m ago 23m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:20:26.441 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (10m) 9m ago 23m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:20:26.441 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (17m) 9m ago 26m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:20:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.430+0000 7f3419654700 1 -- 172.21.15.88:0/4051801991 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340006d8a0 msgr2=0x7f340006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.430+0000 7f3419654700 1 --2- 172.21.15.88:0/4051801991 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340006d8a0 0x7f340006fd50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3404006010 tx=0x7f3404000bc0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.430+0000 7f3419654700 1 -- 172.21.15.88:0/4051801991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34140739a0 msgr2=0x7f34141986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.430+0000 7f3419654700 1 --2- 172.21.15.88:0/4051801991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34140739a0 0x7f34141986c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f33fc00d8d0 tx=0x7f33fc00dc90 comp rx=0 tx=0).stop 2024-02-19T19:20:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.430+0000 7f3419654700 1 -- 172.21.15.88:0/4051801991 shutdown_connections 2024-02-19T19:20:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.430+0000 7f3419654700 1 --2- 172.21.15.88:0/4051801991 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340006d8a0 0x7f340006fd50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.430+0000 7f3419654700 1 --2- 172.21.15.88:0/4051801991 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3414073090 0x7f3414198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.430+0000 7f3419654700 1 --2- 172.21.15.88:0/4051801991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34140739a0 0x7f34141986c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.430+0000 7f3419654700 1 -- 172.21.15.88:0/4051801991 >> 172.21.15.88:0/4051801991 conn(0x7f34140fbda0 msgr2=0x7f3414106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:26.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.430+0000 7f3419654700 1 -- 172.21.15.88:0/4051801991 shutdown_connections 2024-02-19T19:20:26.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.430+0000 7f3419654700 1 -- 172.21.15.88:0/4051801991 wait complete. 2024-02-19T19:20:26.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.534+0000 7fe2635a3700 1 -- 172.21.15.88:0/3632217318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c1004a0 msgr2=0x7fe25c100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.534+0000 7fe2635a3700 1 --2- 172.21.15.88:0/3632217318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c1004a0 0x7fe25c100870 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe258009910 tx=0x7fe258009c20 comp rx=0 tx=0).stop 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.534+0000 7fe2635a3700 1 -- 172.21.15.88:0/3632217318 shutdown_connections 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.534+0000 7fe2635a3700 1 --2- 172.21.15.88:0/3632217318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25c100e40 0x7fe25c104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.534+0000 7fe2635a3700 1 --2- 172.21.15.88:0/3632217318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c1004a0 0x7fe25c100870 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.534+0000 7fe2635a3700 1 -- 172.21.15.88:0/3632217318 >> 172.21.15.88:0/3632217318 conn(0x7fe25c0fbd40 msgr2=0x7fe25c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.534+0000 7fe2635a3700 1 -- 172.21.15.88:0/3632217318 shutdown_connections 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.534+0000 7fe2635a3700 1 -- 172.21.15.88:0/3632217318 wait complete. 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.535+0000 7fe2635a3700 1 Processor -- start 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.535+0000 7fe2635a3700 1 -- start start 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.535+0000 7fe2635a3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25c1004a0 0x7fe25c195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.535+0000 7fe2635a3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c100e40 0x7fe25c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:26.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.535+0000 7fe2635a3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c196b90 con 0x7fe25c100e40 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.535+0000 7fe2635a3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c19a920 con 0x7fe25c1004a0 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.535+0000 7fe260b3e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c100e40 0x7fe25c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.535+0000 7fe260b3e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c100e40 0x7fe25c1964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53584/0 (socket says 172.21.15.88:53584) 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.535+0000 7fe260b3e700 1 -- 172.21.15.88:0/3399366758 learned_addr learned my addr 172.21.15.88:0/3399366758 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.536+0000 7fe26133f700 1 --2- 172.21.15.88:0/3399366758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25c1004a0 0x7fe25c195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.536+0000 7fe260b3e700 1 -- 172.21.15.88:0/3399366758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25c1004a0 msgr2=0x7fe25c195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.536+0000 7fe260b3e700 1 --2- 172.21.15.88:0/3399366758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25c1004a0 0x7fe25c195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.536+0000 7fe260b3e700 1 -- 172.21.15.88:0/3399366758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2580095f0 con 0x7fe25c100e40 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.536+0000 7fe260b3e700 1 --2- 172.21.15.88:0/3399366758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c100e40 0x7fe25c1964b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00b480 tx=0x7fe24c00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.536+0000 7fe2527fc700 1 -- 172.21.15.88:0/3399366758 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe24c011800 con 0x7fe25c100e40 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.536+0000 7fe2635a3700 1 -- 172.21.15.88:0/3399366758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25c19ac00 con 0x7fe25c100e40 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.536+0000 7fe2527fc700 1 -- 172.21.15.88:0/3399366758 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe24c011e20 con 0x7fe25c100e40 2024-02-19T19:20:26.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.536+0000 7fe2527fc700 1 -- 172.21.15.88:0/3399366758 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe24c010ea0 con 0x7fe25c100e40 2024-02-19T19:20:26.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.536+0000 7fe2635a3700 1 -- 172.21.15.88:0/3399366758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25c19b150 con 0x7fe25c100e40 2024-02-19T19:20:26.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.537+0000 7fe2635a3700 1 -- 172.21.15.88:0/3399366758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25c1087b0 con 0x7fe25c100e40 2024-02-19T19:20:26.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.538+0000 7fe2527fc700 1 -- 172.21.15.88:0/3399366758 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe24c010380 con 0x7fe25c100e40 2024-02-19T19:20:26.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.538+0000 7fe2527fc700 1 --2- 172.21.15.88:0/3399366758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe24806d8a0 0x7fe24806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:26.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.538+0000 7fe2527fc700 1 -- 172.21.15.88:0/3399366758 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe24c08d000 con 0x7fe25c100e40 2024-02-19T19:20:26.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.538+0000 7fe26133f700 1 --2- 172.21.15.88:0/3399366758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe24806d8a0 0x7fe24806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:26.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.539+0000 7fe26133f700 1 --2- 172.21.15.88:0/3399366758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe24806d8a0 0x7fe24806fd50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe258006010 tx=0x7fe25801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:26.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.543+0000 7fe2527fc700 1 -- 172.21.15.88:0/3399366758 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe24c0567b0 con 0x7fe25c100e40 2024-02-19T19:20:26.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.821+0000 7fe2635a3700 1 -- 172.21.15.88:0/3399366758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe25c19b420 con 0x7fe25c100e40 2024-02-19T19:20:26.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.821+0000 7fe2527fc700 1 -- 172.21.15.88:0/3399366758 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fe24c059e70 con 0x7fe25c100e40 2024-02-19T19:20:26.833 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:20:26.833 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:20:26.833 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:20:26.833 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:20:26.833 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:20:26.833 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:20:26.833 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:20:26.833 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:20:26.833 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:20:26.834 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:20:26.834 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:20:26.834 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:20:26.834 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:20:26.834 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:20:26.834 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:20:26.834 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:20:26.834 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:20:26.834 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:20:26.834 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:20:26.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.823+0000 7fe2635a3700 1 -- 172.21.15.88:0/3399366758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe24806d8a0 msgr2=0x7fe24806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.823+0000 7fe2635a3700 1 --2- 172.21.15.88:0/3399366758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe24806d8a0 0x7fe24806fd50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe258006010 tx=0x7fe25801a040 comp rx=0 tx=0).stop 2024-02-19T19:20:26.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.823+0000 7fe2635a3700 1 -- 172.21.15.88:0/3399366758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c100e40 msgr2=0x7fe25c1964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.823+0000 7fe2635a3700 1 --2- 172.21.15.88:0/3399366758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c100e40 0x7fe25c1964b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00b480 tx=0x7fe24c00b790 comp rx=0 tx=0).stop 2024-02-19T19:20:26.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.823+0000 7fe2635a3700 1 -- 172.21.15.88:0/3399366758 shutdown_connections 2024-02-19T19:20:26.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.823+0000 7fe2635a3700 1 --2- 172.21.15.88:0/3399366758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe24806d8a0 0x7fe24806fd50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.823+0000 7fe2635a3700 1 --2- 172.21.15.88:0/3399366758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe25c1004a0 0x7fe25c195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.823+0000 7fe2635a3700 1 --2- 172.21.15.88:0/3399366758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe25c100e40 0x7fe25c1964b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.823+0000 7fe2635a3700 1 -- 172.21.15.88:0/3399366758 >> 172.21.15.88:0/3399366758 conn(0x7fe25c0fbd40 msgr2=0x7fe25c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:26.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.824+0000 7fe2635a3700 1 -- 172.21.15.88:0/3399366758 shutdown_connections 2024-02-19T19:20:26.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.824+0000 7fe2635a3700 1 -- 172.21.15.88:0/3399366758 wait complete. 2024-02-19T19:20:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.928+0000 7fae6e1d6700 1 -- 172.21.15.88:0/1028640275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae68100d70 msgr2=0x7fae681094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.928+0000 7fae6e1d6700 1 --2- 172.21.15.88:0/1028640275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae68100d70 0x7fae681094d0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fae5c009940 tx=0x7fae5c009c50 comp rx=0 tx=0).stop 2024-02-19T19:20:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.928+0000 7fae6e1d6700 1 -- 172.21.15.88:0/1028640275 shutdown_connections 2024-02-19T19:20:26.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.928+0000 7fae6e1d6700 1 --2- 172.21.15.88:0/1028640275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae68100d70 0x7fae681094d0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.928+0000 7fae6e1d6700 1 --2- 172.21.15.88:0/1028640275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae68100460 0x7fae68100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.928+0000 7fae6e1d6700 1 -- 172.21.15.88:0/1028640275 >> 172.21.15.88:0/1028640275 conn(0x7fae680748b0 msgr2=0x7fae68074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.928+0000 7fae6e1d6700 1 -- 172.21.15.88:0/1028640275 shutdown_connections 2024-02-19T19:20:26.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.928+0000 7fae6e1d6700 1 -- 172.21.15.88:0/1028640275 wait complete. 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.928+0000 7fae6e1d6700 1 Processor -- start 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae6e1d6700 1 -- start start 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae6e1d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae68100460 0x7fae68196d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae6e1d6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae68100d70 0x7fae68197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae6e1d6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae68197860 con 0x7fae68100d70 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae6e1d6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae681979d0 con 0x7fae68100460 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae66ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae68100d70 0x7fae68197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae66ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae68100d70 0x7fae68197240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53596/0 (socket says 172.21.15.88:53596) 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae66ffd700 1 -- 172.21.15.88:0/220125760 learned_addr learned my addr 172.21.15.88:0/220125760 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae677fe700 1 --2- 172.21.15.88:0/220125760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae68100460 0x7fae68196d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae66ffd700 1 -- 172.21.15.88:0/220125760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae68100460 msgr2=0x7fae68196d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae66ffd700 1 --2- 172.21.15.88:0/220125760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae68100460 0x7fae68196d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae66ffd700 1 -- 172.21.15.88:0/220125760 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae5c0095f0 con 0x7fae68100d70 2024-02-19T19:20:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.929+0000 7fae66ffd700 1 --2- 172.21.15.88:0/220125760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae68100d70 0x7fae68197240 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0055f0 tx=0x7fae5c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.930+0000 7fae64ff9700 1 -- 172.21.15.88:0/220125760 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae5c01d070 con 0x7fae68100d70 2024-02-19T19:20:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.930+0000 7fae6e1d6700 1 -- 172.21.15.88:0/220125760 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae6819fd70 con 0x7fae68100d70 2024-02-19T19:20:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.930+0000 7fae64ff9700 1 -- 172.21.15.88:0/220125760 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae5c00f460 con 0x7fae68100d70 2024-02-19T19:20:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.930+0000 7fae64ff9700 1 -- 172.21.15.88:0/220125760 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae5c021640 con 0x7fae68100d70 2024-02-19T19:20:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.930+0000 7fae6e1d6700 1 -- 172.21.15.88:0/220125760 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae681a0260 con 0x7fae68100d70 2024-02-19T19:20:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.931+0000 7fae6e1d6700 1 -- 172.21.15.88:0/220125760 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae68106bd0 con 0x7fae68100d70 2024-02-19T19:20:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.931+0000 7fae64ff9700 1 -- 172.21.15.88:0/220125760 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fae5c00f5d0 con 0x7fae68100d70 2024-02-19T19:20:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.932+0000 7fae64ff9700 1 --2- 172.21.15.88:0/220125760 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fae5006d8a0 0x7fae5006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.932+0000 7fae677fe700 1 --2- 172.21.15.88:0/220125760 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fae5006d8a0 0x7fae5006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.932+0000 7fae64ff9700 1 -- 172.21.15.88:0/220125760 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fae5c08e850 con 0x7fae68100d70 2024-02-19T19:20:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.932+0000 7fae677fe700 1 --2- 172.21.15.88:0/220125760 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fae5006d8a0 0x7fae5006fd50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fae58006fd0 tx=0x7fae58008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:26.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:26.936+0000 7fae64ff9700 1 -- 172.21.15.88:0/220125760 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fae5c058000 con 0x7fae68100d70 2024-02-19T19:20:27.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.180+0000 7fae6e1d6700 1 -- 172.21.15.88:0/220125760 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fae680ffa30 con 0x7fae68100d70 2024-02-19T19:20:27.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.181+0000 7fae64ff9700 1 -- 172.21.15.88:0/220125760 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fae5c05b6c0 con 0x7fae68100d70 2024-02-19T19:20:27.192 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:20:27.192 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:20:27.193 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:20:27.194 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:20:27.195 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:20:27.195 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:20:27.195 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:20:27.195 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:20:27.195 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:20:27.195 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:20:27.195 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:20:27.195 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:20:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.183+0000 7fae6e1d6700 1 -- 172.21.15.88:0/220125760 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fae5006d8a0 msgr2=0x7fae5006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.183+0000 7fae6e1d6700 1 --2- 172.21.15.88:0/220125760 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fae5006d8a0 0x7fae5006fd50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fae58006fd0 tx=0x7fae58008040 comp rx=0 tx=0).stop 2024-02-19T19:20:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.183+0000 7fae6e1d6700 1 -- 172.21.15.88:0/220125760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae68100d70 msgr2=0x7fae68197240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.183+0000 7fae6e1d6700 1 --2- 172.21.15.88:0/220125760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae68100d70 0x7fae68197240 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fae5c0055f0 tx=0x7fae5c0056d0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.183+0000 7fae6e1d6700 1 -- 172.21.15.88:0/220125760 shutdown_connections 2024-02-19T19:20:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.183+0000 7fae6e1d6700 1 --2- 172.21.15.88:0/220125760 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fae5006d8a0 0x7fae5006fd50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.183+0000 7fae6e1d6700 1 --2- 172.21.15.88:0/220125760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fae68100460 0x7fae68196d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.183+0000 7fae6e1d6700 1 --2- 172.21.15.88:0/220125760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae68100d70 0x7fae68197240 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.183+0000 7fae6e1d6700 1 -- 172.21.15.88:0/220125760 >> 172.21.15.88:0/220125760 conn(0x7fae680748b0 msgr2=0x7fae68103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:27.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.183+0000 7fae6e1d6700 1 -- 172.21.15.88:0/220125760 shutdown_connections 2024-02-19T19:20:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.183+0000 7fae6e1d6700 1 -- 172.21.15.88:0/220125760 wait complete. 2024-02-19T19:20:27.197 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:20:27.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.287+0000 7fecf2583700 1 -- 172.21.15.88:0/1862540655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecec100530 msgr2=0x7fecec100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:27.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.287+0000 7fecf2583700 1 --2- 172.21.15.88:0/1862540655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecec100530 0x7fecec100900 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009910 tx=0x7fecdc009c20 comp rx=0 tx=0).stop 2024-02-19T19:20:27.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.287+0000 7fecf2583700 1 -- 172.21.15.88:0/1862540655 shutdown_connections 2024-02-19T19:20:27.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.287+0000 7fecf2583700 1 --2- 172.21.15.88:0/1862540655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecec100ed0 0x7fecec104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.287+0000 7fecf2583700 1 --2- 172.21.15.88:0/1862540655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecec100530 0x7fecec100900 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.287+0000 7fecf2583700 1 -- 172.21.15.88:0/1862540655 >> 172.21.15.88:0/1862540655 conn(0x7fecec0fbe10 msgr2=0x7fecec0fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:27.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.287+0000 7fecf2583700 1 -- 172.21.15.88:0/1862540655 shutdown_connections 2024-02-19T19:20:27.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.287+0000 7fecf2583700 1 -- 172.21.15.88:0/1862540655 wait complete. 2024-02-19T19:20:27.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.288+0000 7fecf2583700 1 Processor -- start 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.288+0000 7fecf2583700 1 -- start start 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.288+0000 7fecf2583700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecec100530 0x7fecec073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.288+0000 7fecf2583700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecec100ed0 0x7fecec073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.288+0000 7fecf2583700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec073ad0 con 0x7fecec100530 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.288+0000 7fecf2583700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecec073c10 con 0x7fecec100ed0 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.288+0000 7fecebfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecec100530 0x7fecec073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.288+0000 7fecebfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecec100530 0x7fecec073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53612/0 (socket says 172.21.15.88:53612) 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.288+0000 7fecebfff700 1 -- 172.21.15.88:0/405855714 learned_addr learned my addr 172.21.15.88:0/405855714 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.288+0000 7feceb7fe700 1 --2- 172.21.15.88:0/405855714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecec100ed0 0x7fecec073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.289+0000 7fecebfff700 1 -- 172.21.15.88:0/405855714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecec100ed0 msgr2=0x7fecec073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.289+0000 7fecebfff700 1 --2- 172.21.15.88:0/405855714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecec100ed0 0x7fecec073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.289+0000 7fecebfff700 1 -- 172.21.15.88:0/405855714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc0095f0 con 0x7fecec100530 2024-02-19T19:20:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.289+0000 7fecebfff700 1 --2- 172.21.15.88:0/405855714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecec100530 0x7fecec073050 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fecdc000c00 tx=0x7fecdc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:27.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.289+0000 7fece97fa700 1 -- 172.21.15.88:0/405855714 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecdc01d070 con 0x7fecec100530 2024-02-19T19:20:27.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.289+0000 7fecf2583700 1 -- 172.21.15.88:0/405855714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecec073e90 con 0x7fecec100530 2024-02-19T19:20:27.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.289+0000 7fece97fa700 1 -- 172.21.15.88:0/405855714 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecdc00f460 con 0x7fecec100530 2024-02-19T19:20:27.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.289+0000 7fece97fa700 1 -- 172.21.15.88:0/405855714 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecdc017630 con 0x7fecec100530 2024-02-19T19:20:27.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.289+0000 7fecf2583700 1 -- 172.21.15.88:0/405855714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecec1a2450 con 0x7fecec100530 2024-02-19T19:20:27.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.290+0000 7fecf2583700 1 -- 172.21.15.88:0/405855714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecec108840 con 0x7fecec100530 2024-02-19T19:20:27.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.290+0000 7fece97fa700 1 -- 172.21.15.88:0/405855714 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fecdc00f5d0 con 0x7fecec100530 2024-02-19T19:20:27.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.291+0000 7fece97fa700 1 --2- 172.21.15.88:0/405855714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fecd406d8a0 0x7fecd406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:27.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.291+0000 7fece97fa700 1 -- 172.21.15.88:0/405855714 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fecdc08ebb0 con 0x7fecec100530 2024-02-19T19:20:27.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.291+0000 7feceb7fe700 1 --2- 172.21.15.88:0/405855714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fecd406d8a0 0x7fecd406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:27.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.291+0000 7feceb7fe700 1 --2- 172.21.15.88:0/405855714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fecd406d8a0 0x7fecd406fd50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fece0006fd0 tx=0x7fece0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:27.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.295+0000 7fece97fa700 1 -- 172.21.15.88:0/405855714 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fecdc095080 con 0x7fecec100530 2024-02-19T19:20:27.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.509+0000 7fecf2583700 1 -- 172.21.15.88:0/405855714 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fecec061190 con 0x7fecd406d8a0 2024-02-19T19:20:27.520 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:27 smithi088 ceph-mon[88643]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:27.520 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:27 smithi088 ceph-mon[88643]: from='client.34950 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:27.520 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:27 smithi088 ceph-mon[88643]: pgmap v505: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:27.521 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:27 smithi088 ceph-mon[88643]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:27.521 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:27 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3399366758' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:20:27.521 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:27 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/220125760' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:20:27.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.510+0000 7fece97fa700 1 -- 172.21.15.88:0/405855714 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fecec061190 con 0x7fecd406d8a0 2024-02-19T19:20:27.521 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:20:27.521 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:20:27.522 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:20:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.512+0000 7fecf2583700 1 -- 172.21.15.88:0/405855714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fecd406d8a0 msgr2=0x7fecd406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.512+0000 7fecf2583700 1 --2- 172.21.15.88:0/405855714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fecd406d8a0 0x7fecd406fd50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fece0006fd0 tx=0x7fece0008040 comp rx=0 tx=0).stop 2024-02-19T19:20:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.512+0000 7fecf2583700 1 -- 172.21.15.88:0/405855714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecec100530 msgr2=0x7fecec073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.512+0000 7fecf2583700 1 --2- 172.21.15.88:0/405855714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecec100530 0x7fecec073050 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fecdc000c00 tx=0x7fecdc00be20 comp rx=0 tx=0).stop 2024-02-19T19:20:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.512+0000 7fecf2583700 1 -- 172.21.15.88:0/405855714 shutdown_connections 2024-02-19T19:20:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.512+0000 7fecf2583700 1 --2- 172.21.15.88:0/405855714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fecd406d8a0 0x7fecd406fd50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.512+0000 7fecf2583700 1 --2- 172.21.15.88:0/405855714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecec100530 0x7fecec073050 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.512+0000 7fecf2583700 1 --2- 172.21.15.88:0/405855714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fecec100ed0 0x7fecec073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.512+0000 7fecf2583700 1 -- 172.21.15.88:0/405855714 >> 172.21.15.88:0/405855714 conn(0x7fecec0fbe10 msgr2=0x7fecec0fd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:27.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.513+0000 7fecf2583700 1 -- 172.21.15.88:0/405855714 shutdown_connections 2024-02-19T19:20:27.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.513+0000 7fecf2583700 1 -- 172.21.15.88:0/405855714 wait complete. 2024-02-19T19:20:27.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.616+0000 7ff5a4325700 1 -- 172.21.15.88:0/3584773829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff59c104130 msgr2=0x7ff59c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:27.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.616+0000 7ff5a4325700 1 --2- 172.21.15.88:0/3584773829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff59c104130 0x7ff59c104500 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff598009910 tx=0x7ff598009c20 comp rx=0 tx=0).stop 2024-02-19T19:20:27.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.616+0000 7ff5a4325700 1 -- 172.21.15.88:0/3584773829 shutdown_connections 2024-02-19T19:20:27.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.616+0000 7ff5a4325700 1 --2- 172.21.15.88:0/3584773829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff59c0ffbd0 0x7ff59c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.616+0000 7ff5a4325700 1 --2- 172.21.15.88:0/3584773829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff59c104130 0x7ff59c104500 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.616+0000 7ff5a4325700 1 -- 172.21.15.88:0/3584773829 >> 172.21.15.88:0/3584773829 conn(0x7ff59c075130 msgr2=0x7ff59c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:27.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.616+0000 7ff5a4325700 1 -- 172.21.15.88:0/3584773829 shutdown_connections 2024-02-19T19:20:27.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.616+0000 7ff5a4325700 1 -- 172.21.15.88:0/3584773829 wait complete. 2024-02-19T19:20:27.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.616+0000 7ff5a4325700 1 Processor -- start 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a4325700 1 -- start start 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a4325700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff59c0ffbd0 0x7ff59c194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a4325700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff59c104130 0x7ff59c195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a4325700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c1958d0 con 0x7ff59c104130 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a4325700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59c195a10 con 0x7ff59c0ffbd0 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a18c0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff59c104130 0x7ff59c195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a18c0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff59c104130 0x7ff59c195280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53624/0 (socket says 172.21.15.88:53624) 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a18c0700 1 -- 172.21.15.88:0/1029489574 learned_addr learned my addr 172.21.15.88:0/1029489574 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a20c1700 1 --2- 172.21.15.88:0/1029489574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff59c0ffbd0 0x7ff59c194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a18c0700 1 -- 172.21.15.88:0/1029489574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff59c0ffbd0 msgr2=0x7ff59c194d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a18c0700 1 --2- 172.21.15.88:0/1029489574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff59c0ffbd0 0x7ff59c194d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a18c0700 1 -- 172.21.15.88:0/1029489574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5980095f0 con 0x7ff59c104130 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a18c0700 1 --2- 172.21.15.88:0/1029489574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff59c104130 0x7ff59c195280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff58c007b50 tx=0x7ff58c007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:27.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5937fe700 1 -- 172.21.15.88:0/1029489574 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff58c011070 con 0x7ff59c104130 2024-02-19T19:20:27.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5937fe700 1 -- 172.21.15.88:0/1029489574 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff58c00c2e0 con 0x7ff59c104130 2024-02-19T19:20:27.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.617+0000 7ff5a4325700 1 -- 172.21.15.88:0/1029489574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff59c199860 con 0x7ff59c104130 2024-02-19T19:20:27.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.618+0000 7ff5937fe700 1 -- 172.21.15.88:0/1029489574 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff58c00ac60 con 0x7ff59c104130 2024-02-19T19:20:27.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.618+0000 7ff5a4325700 1 -- 172.21.15.88:0/1029489574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff59c199db0 con 0x7ff59c104130 2024-02-19T19:20:27.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.619+0000 7ff5a4325700 1 -- 172.21.15.88:0/1029489574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff59c04f2a0 con 0x7ff59c104130 2024-02-19T19:20:27.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.619+0000 7ff5937fe700 1 -- 172.21.15.88:0/1029489574 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff58c00e650 con 0x7ff59c104130 2024-02-19T19:20:27.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.619+0000 7ff5937fe700 1 --2- 172.21.15.88:0/1029489574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff58806d8a0 0x7ff58806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:27.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.619+0000 7ff5937fe700 1 -- 172.21.15.88:0/1029489574 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff58c08eb60 con 0x7ff59c104130 2024-02-19T19:20:27.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.620+0000 7ff5a20c1700 1 --2- 172.21.15.88:0/1029489574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff58806d8a0 0x7ff58806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:27.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.620+0000 7ff5a20c1700 1 --2- 172.21.15.88:0/1029489574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff58806d8a0 0x7ff58806fd50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff598006010 tx=0x7ff59801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.624+0000 7ff5937fe700 1 -- 172.21.15.88:0/1029489574 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff58c058310 con 0x7ff59c104130 2024-02-19T19:20:27.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:27 smithi132 ceph-mon[70676]: from='client.34946 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:27.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:27 smithi132 ceph-mon[70676]: from='client.34950 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:27.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:27 smithi132 ceph-mon[70676]: pgmap v505: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:27.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:27 smithi132 ceph-mon[70676]: from='client.34954 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:27.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:27 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3399366758' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:20:27.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:27 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/220125760' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:20:27.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.902+0000 7ff5a4325700 1 -- 172.21.15.88:0/1029489574 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff59c19a050 con 0x7ff59c104130 2024-02-19T19:20:27.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.902+0000 7ff5937fe700 1 -- 172.21.15.88:0/1029489574 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7ff58c05b9d0 con 0x7ff59c104130 2024-02-19T19:20:27.914 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:20:27.914 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:20:27.914 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:20:27.914 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:20:27.914 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:20:27.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.904+0000 7ff5a4325700 1 -- 172.21.15.88:0/1029489574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff58806d8a0 msgr2=0x7ff58806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:27.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.904+0000 7ff5a4325700 1 --2- 172.21.15.88:0/1029489574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff58806d8a0 0x7ff58806fd50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff598006010 tx=0x7ff59801a040 comp rx=0 tx=0).stop 2024-02-19T19:20:27.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.904+0000 7ff5a4325700 1 -- 172.21.15.88:0/1029489574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff59c104130 msgr2=0x7ff59c195280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:27.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.904+0000 7ff5a4325700 1 --2- 172.21.15.88:0/1029489574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff59c104130 0x7ff59c195280 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff58c007b50 tx=0x7ff58c007e60 comp rx=0 tx=0).stop 2024-02-19T19:20:27.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.905+0000 7ff5a4325700 1 -- 172.21.15.88:0/1029489574 shutdown_connections 2024-02-19T19:20:27.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.905+0000 7ff5a4325700 1 --2- 172.21.15.88:0/1029489574 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff58806d8a0 0x7ff58806fd50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.905+0000 7ff5a4325700 1 --2- 172.21.15.88:0/1029489574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff59c0ffbd0 0x7ff59c194d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.905+0000 7ff5a4325700 1 --2- 172.21.15.88:0/1029489574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff59c104130 0x7ff59c195280 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:27.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.905+0000 7ff5a4325700 1 -- 172.21.15.88:0/1029489574 >> 172.21.15.88:0/1029489574 conn(0x7ff59c075130 msgr2=0x7ff59c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:27.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.905+0000 7ff5a4325700 1 -- 172.21.15.88:0/1029489574 shutdown_connections 2024-02-19T19:20:27.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:27.905+0000 7ff5a4325700 1 -- 172.21.15.88:0/1029489574 wait complete. 2024-02-19T19:20:28.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:28 smithi132 ceph-mon[70676]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:28.810 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:28 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1029489574' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:20:28.810 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:20:28.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:28 smithi088 ceph-mon[88643]: from='client.34966 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:28.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:28 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1029489574' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:20:28.842 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:20:29.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:29 smithi088 ceph-mon[88643]: pgmap v506: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:29.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:29 smithi132 ceph-mon[70676]: pgmap v506: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:31.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:31 smithi132 ceph-mon[70676]: pgmap v507: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:31.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:31 smithi088 ceph-mon[88643]: pgmap v507: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:33.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:33 smithi132 ceph-mon[70676]: pgmap v508: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:33.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:33 smithi088 ceph-mon[88643]: pgmap v508: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:35.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:35 smithi132 ceph-mon[70676]: pgmap v509: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:35.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:35 smithi088 ceph-mon[88643]: pgmap v509: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:37.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:37 smithi132 ceph-mon[70676]: pgmap v510: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:37.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:37 smithi088 ceph-mon[88643]: pgmap v510: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:39.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:39 smithi088 ceph-mon[88643]: pgmap v511: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:39.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:39 smithi132 ceph-mon[70676]: pgmap v511: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:41.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:41 smithi132 ceph-mon[70676]: pgmap v512: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:41.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:41 smithi088 ceph-mon[88643]: pgmap v512: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:43.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:43 smithi132 ceph-mon[70676]: pgmap v513: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:43.744 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:20:43.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:43 smithi088 ceph-mon[88643]: pgmap v513: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:43.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:20:45.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:45 smithi132 ceph-mon[70676]: pgmap v514: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:45.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:45 smithi088 ceph-mon[88643]: pgmap v514: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:47.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:47 smithi132 ceph-mon[70676]: pgmap v515: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:47.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:47 smithi088 ceph-mon[88643]: pgmap v515: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:49.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:49 smithi132 ceph-mon[70676]: pgmap v516: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:49.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:49 smithi088 ceph-mon[88643]: pgmap v516: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:51.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:51 smithi132 ceph-mon[70676]: pgmap v517: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:51.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:51 smithi088 ceph-mon[88643]: pgmap v517: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:53.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:53 smithi132 ceph-mon[70676]: pgmap v518: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:53.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:53 smithi088 ceph-mon[88643]: pgmap v518: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:55.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:55 smithi132 ceph-mon[70676]: pgmap v519: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:55.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:55 smithi088 ceph-mon[88643]: pgmap v519: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:57.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:57 smithi132 ceph-mon[70676]: pgmap v520: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:57.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:57 smithi088 ceph-mon[88643]: pgmap v520: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:58.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.021+0000 7f51946d5700 1 -- 172.21.15.88:0/899386088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f518c1038c0 msgr2=0x7f518c103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.021+0000 7f51946d5700 1 --2- 172.21.15.88:0/899386088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f518c1038c0 0x7f518c103c90 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f518800b0a0 tx=0x7f518800b3b0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.021+0000 7f51946d5700 1 -- 172.21.15.88:0/899386088 shutdown_connections 2024-02-19T19:20:58.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.021+0000 7f51946d5700 1 --2- 172.21.15.88:0/899386088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f518c0ff3d0 0x7f518c0ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.021+0000 7f51946d5700 1 --2- 172.21.15.88:0/899386088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f518c1038c0 0x7f518c103c90 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.021+0000 7f51946d5700 1 -- 172.21.15.88:0/899386088 >> 172.21.15.88:0/899386088 conn(0x7f518c074930 msgr2=0x7f518c074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:58.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.021+0000 7f51946d5700 1 -- 172.21.15.88:0/899386088 shutdown_connections 2024-02-19T19:20:58.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.021+0000 7f51946d5700 1 -- 172.21.15.88:0/899386088 wait complete. 2024-02-19T19:20:58.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f51946d5700 1 Processor -- start 2024-02-19T19:20:58.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f51946d5700 1 -- start start 2024-02-19T19:20:58.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f51946d5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f518c0ff3d0 0x7f518c1005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:58.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f51946d5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f518c1038c0 0x7f518c100b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:58.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f51946d5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c101040 con 0x7f518c1038c0 2024-02-19T19:20:58.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f51946d5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c101180 con 0x7f518c0ff3d0 2024-02-19T19:20:58.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f5191c70700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f518c1038c0 0x7f518c100b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:58.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f5191c70700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f518c1038c0 0x7f518c100b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38422/0 (socket says 172.21.15.88:38422) 2024-02-19T19:20:58.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f5191c70700 1 -- 172.21.15.88:0/2100456589 learned_addr learned my addr 172.21.15.88:0/2100456589 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:58.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f5192471700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f518c0ff3d0 0x7f518c1005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:58.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f5191c70700 1 -- 172.21.15.88:0/2100456589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f518c0ff3d0 msgr2=0x7f518c1005c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f5191c70700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f518c0ff3d0 0x7f518c1005c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f5191c70700 1 -- 172.21.15.88:0/2100456589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5188009d00 con 0x7f518c1038c0 2024-02-19T19:20:58.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f5192471700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f518c0ff3d0 0x7f518c1005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:20:58.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.022+0000 7f5191c70700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f518c1038c0 0x7f518c100b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f517c00c8a0 tx=0x7f517c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:58.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.023+0000 7f51837fe700 1 -- 172.21.15.88:0/2100456589 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f517c00cea0 con 0x7f518c1038c0 2024-02-19T19:20:58.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.023+0000 7f51837fe700 1 -- 172.21.15.88:0/2100456589 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f517c004830 con 0x7f518c1038c0 2024-02-19T19:20:58.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.023+0000 7f51946d5700 1 -- 172.21.15.88:0/2100456589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c1a1fd0 con 0x7f518c1038c0 2024-02-19T19:20:58.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.023+0000 7f51837fe700 1 -- 172.21.15.88:0/2100456589 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f517c0055b0 con 0x7f518c1038c0 2024-02-19T19:20:58.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.023+0000 7f51946d5700 1 -- 172.21.15.88:0/2100456589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c1a23e0 con 0x7f518c1038c0 2024-02-19T19:20:58.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.024+0000 7f51946d5700 1 -- 172.21.15.88:0/2100456589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518c1085b0 con 0x7f518c1038c0 2024-02-19T19:20:58.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.024+0000 7f51837fe700 1 -- 172.21.15.88:0/2100456589 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f517c0049a0 con 0x7f518c1038c0 2024-02-19T19:20:58.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.025+0000 7f51837fe700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f517806d8a0 0x7f517806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:58.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.025+0000 7f51837fe700 1 -- 172.21.15.88:0/2100456589 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f517c08d1d0 con 0x7f518c1038c0 2024-02-19T19:20:58.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.025+0000 7f5192471700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f517806d8a0 0x7f517806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:58.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.025+0000 7f5192471700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f517806d8a0 0x7f517806fd50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f518800b830 tx=0x7f5188006210 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.030+0000 7f51837fe700 1 -- 172.21.15.88:0/2100456589 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f517c056980 con 0x7f518c1038c0 2024-02-19T19:20:58.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.245+0000 7f51946d5700 1 -- 172.21.15.88:0/2100456589 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f518c02cca0 con 0x7f517806d8a0 2024-02-19T19:20:58.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.247+0000 7f51837fe700 1 -- 172.21.15.88:0/2100456589 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f518c02cca0 con 0x7f517806d8a0 2024-02-19T19:20:58.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.249+0000 7f51946d5700 1 -- 172.21.15.88:0/2100456589 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f517806d8a0 msgr2=0x7f517806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.249+0000 7f51946d5700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f517806d8a0 0x7f517806fd50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f518800b830 tx=0x7f5188006210 comp rx=0 tx=0).stop 2024-02-19T19:20:58.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.249+0000 7f51946d5700 1 -- 172.21.15.88:0/2100456589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f518c1038c0 msgr2=0x7f518c100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.249+0000 7f51946d5700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f518c1038c0 0x7f518c100b00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f517c00c8a0 tx=0x7f517c00cc60 comp rx=0 tx=0).stop 2024-02-19T19:20:58.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.249+0000 7f51946d5700 1 -- 172.21.15.88:0/2100456589 shutdown_connections 2024-02-19T19:20:58.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.249+0000 7f51946d5700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f517806d8a0 0x7f517806fd50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.249+0000 7f51946d5700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f518c0ff3d0 0x7f518c1005c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.249+0000 7f51946d5700 1 --2- 172.21.15.88:0/2100456589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f518c1038c0 0x7f518c100b00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.249+0000 7f51946d5700 1 -- 172.21.15.88:0/2100456589 >> 172.21.15.88:0/2100456589 conn(0x7f518c074930 msgr2=0x7f518c1056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:58.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.249+0000 7f51946d5700 1 -- 172.21.15.88:0/2100456589 shutdown_connections 2024-02-19T19:20:58.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.249+0000 7f51946d5700 1 -- 172.21.15.88:0/2100456589 wait complete. 2024-02-19T19:20:58.271 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:20:58.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.359+0000 7f3e1fa60700 1 -- 172.21.15.88:0/2066118386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e18100e80 msgr2=0x7f3e18104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.359+0000 7f3e1fa60700 1 --2- 172.21.15.88:0/2066118386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e18100e80 0x7f3e18104e50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009910 tx=0x7f3e08009c20 comp rx=0 tx=0).stop 2024-02-19T19:20:58.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.359+0000 7f3e1fa60700 1 -- 172.21.15.88:0/2066118386 shutdown_connections 2024-02-19T19:20:58.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.359+0000 7f3e1fa60700 1 --2- 172.21.15.88:0/2066118386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e18100e80 0x7f3e18104e50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.359+0000 7f3e1fa60700 1 --2- 172.21.15.88:0/2066118386 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e181004e0 0x7f3e181008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.359+0000 7f3e1fa60700 1 -- 172.21.15.88:0/2066118386 >> 172.21.15.88:0/2066118386 conn(0x7f3e180fbda0 msgr2=0x7f3e180fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:58.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.359+0000 7f3e1fa60700 1 -- 172.21.15.88:0/2066118386 shutdown_connections 2024-02-19T19:20:58.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.359+0000 7f3e1fa60700 1 -- 172.21.15.88:0/2066118386 wait complete. 2024-02-19T19:20:58.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1fa60700 1 Processor -- start 2024-02-19T19:20:58.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1fa60700 1 -- start start 2024-02-19T19:20:58.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1fa60700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e181004e0 0x7f3e18073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:58.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1fa60700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e18100e80 0x7f3e18073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:58.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1fa60700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18073ad0 con 0x7f3e18100e80 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1fa60700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18073c10 con 0x7f3e181004e0 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1cffb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e18100e80 0x7f3e18073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1cffb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e18100e80 0x7f3e18073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38440/0 (socket says 172.21.15.88:38440) 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1cffb700 1 -- 172.21.15.88:0/3129688507 learned_addr learned my addr 172.21.15.88:0/3129688507 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1d7fc700 1 --2- 172.21.15.88:0/3129688507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e181004e0 0x7f3e18073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1cffb700 1 -- 172.21.15.88:0/3129688507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e181004e0 msgr2=0x7f3e18073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1cffb700 1 --2- 172.21.15.88:0/3129688507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e181004e0 0x7f3e18073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1cffb700 1 -- 172.21.15.88:0/3129688507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e080095f0 con 0x7f3e18100e80 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.360+0000 7f3e1cffb700 1 --2- 172.21.15.88:0/3129688507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e18100e80 0x7f3e18073590 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3e080092e0 tx=0x7f3e0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.361+0000 7f3e0e7fc700 1 -- 172.21.15.88:0/3129688507 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e0801d070 con 0x7f3e18100e80 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.361+0000 7f3e0e7fc700 1 -- 172.21.15.88:0/3129688507 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e08005dc0 con 0x7f3e18100e80 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.361+0000 7f3e1fa60700 1 -- 172.21.15.88:0/3129688507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e18073e90 con 0x7f3e18100e80 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.361+0000 7f3e0e7fc700 1 -- 172.21.15.88:0/3129688507 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e08017400 con 0x7f3e18100e80 2024-02-19T19:20:58.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.361+0000 7f3e1fa60700 1 -- 172.21.15.88:0/3129688507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e181a2410 con 0x7f3e18100e80 2024-02-19T19:20:58.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.362+0000 7f3e1fa60700 1 -- 172.21.15.88:0/3129688507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e18108850 con 0x7f3e18100e80 2024-02-19T19:20:58.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.362+0000 7f3e0e7fc700 1 -- 172.21.15.88:0/3129688507 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3e0800f460 con 0x7f3e18100e80 2024-02-19T19:20:58.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.362+0000 7f3e0e7fc700 1 --2- 172.21.15.88:0/3129688507 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e0406d8a0 0x7f3e0406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:58.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.363+0000 7f3e0e7fc700 1 -- 172.21.15.88:0/3129688507 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3e0808ea20 con 0x7f3e18100e80 2024-02-19T19:20:58.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.363+0000 7f3e1d7fc700 1 --2- 172.21.15.88:0/3129688507 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e0406d8a0 0x7f3e0406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:58.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.363+0000 7f3e1d7fc700 1 --2- 172.21.15.88:0/3129688507 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e0406d8a0 0x7f3e0406fd50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e14005950 tx=0x7f3e140058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.367+0000 7f3e0e7fc700 1 -- 172.21.15.88:0/3129688507 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e080581d0 con 0x7f3e18100e80 2024-02-19T19:20:58.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.585+0000 7f3e1fa60700 1 -- 172.21.15.88:0/3129688507 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e18074690 con 0x7f3e0406d8a0 2024-02-19T19:20:58.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.586+0000 7f3e0e7fc700 1 -- 172.21.15.88:0/3129688507 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3e18074690 con 0x7f3e0406d8a0 2024-02-19T19:20:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.589+0000 7f3e1fa60700 1 -- 172.21.15.88:0/3129688507 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e0406d8a0 msgr2=0x7f3e0406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.589+0000 7f3e1fa60700 1 --2- 172.21.15.88:0/3129688507 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e0406d8a0 0x7f3e0406fd50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e14005950 tx=0x7f3e140058e0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.589+0000 7f3e1fa60700 1 -- 172.21.15.88:0/3129688507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e18100e80 msgr2=0x7f3e18073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.589+0000 7f3e1fa60700 1 --2- 172.21.15.88:0/3129688507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e18100e80 0x7f3e18073590 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3e080092e0 tx=0x7f3e0800be20 comp rx=0 tx=0).stop 2024-02-19T19:20:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.589+0000 7f3e1fa60700 1 -- 172.21.15.88:0/3129688507 shutdown_connections 2024-02-19T19:20:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.589+0000 7f3e1fa60700 1 --2- 172.21.15.88:0/3129688507 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e0406d8a0 0x7f3e0406fd50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.589+0000 7f3e1fa60700 1 --2- 172.21.15.88:0/3129688507 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e181004e0 0x7f3e18073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.589+0000 7f3e1fa60700 1 --2- 172.21.15.88:0/3129688507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e18100e80 0x7f3e18073590 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.589+0000 7f3e1fa60700 1 -- 172.21.15.88:0/3129688507 >> 172.21.15.88:0/3129688507 conn(0x7f3e180fbda0 msgr2=0x7f3e180fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.589+0000 7f3e1fa60700 1 -- 172.21.15.88:0/3129688507 shutdown_connections 2024-02-19T19:20:58.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.589+0000 7f3e1fa60700 1 -- 172.21.15.88:0/3129688507 wait complete. 2024-02-19T19:20:58.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.693+0000 7fc1226e0700 1 -- 172.21.15.88:0/1997120941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11c100e80 msgr2=0x7fc11c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.693+0000 7fc1226e0700 1 --2- 172.21.15.88:0/1997120941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11c100e80 0x7fc11c104e50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc110009910 tx=0x7fc110009c20 comp rx=0 tx=0).stop 2024-02-19T19:20:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.693+0000 7fc1226e0700 1 -- 172.21.15.88:0/1997120941 shutdown_connections 2024-02-19T19:20:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.693+0000 7fc1226e0700 1 --2- 172.21.15.88:0/1997120941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11c100e80 0x7fc11c104e50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.693+0000 7fc1226e0700 1 --2- 172.21.15.88:0/1997120941 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c1004e0 0x7fc11c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.693+0000 7fc1226e0700 1 -- 172.21.15.88:0/1997120941 >> 172.21.15.88:0/1997120941 conn(0x7fc11c0fbda0 msgr2=0x7fc11c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:58.706 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.694+0000 7fc1226e0700 1 -- 172.21.15.88:0/1997120941 shutdown_connections 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.694+0000 7fc1226e0700 1 -- 172.21.15.88:0/1997120941 wait complete. 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.694+0000 7fc1226e0700 1 Processor -- start 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.694+0000 7fc1226e0700 1 -- start start 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.694+0000 7fc1226e0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11c1004e0 0x7fc11c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.694+0000 7fc1226e0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c100e80 0x7fc11c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.694+0000 7fc1226e0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c073ad0 con 0x7fc11c1004e0 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.694+0000 7fc1226e0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11c073c10 con 0x7fc11c100e80 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc11bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11c1004e0 0x7fc11c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc11bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11c1004e0 0x7fc11c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38458/0 (socket says 172.21.15.88:38458) 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc11bfff700 1 -- 172.21.15.88:0/1235967683 learned_addr learned my addr 172.21.15.88:0/1235967683 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc11b7fe700 1 --2- 172.21.15.88:0/1235967683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c100e80 0x7fc11c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:58.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc11bfff700 1 -- 172.21.15.88:0/1235967683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c100e80 msgr2=0x7fc11c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc11bfff700 1 --2- 172.21.15.88:0/1235967683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c100e80 0x7fc11c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc11bfff700 1 -- 172.21.15.88:0/1235967683 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1100095f0 con 0x7fc11c1004e0 2024-02-19T19:20:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc11bfff700 1 --2- 172.21.15.88:0/1235967683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11c1004e0 0x7fc11c073050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00b4b0 tx=0x7fc10c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc1197fa700 1 -- 172.21.15.88:0/1235967683 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc10c011800 con 0x7fc11c1004e0 2024-02-19T19:20:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc1226e0700 1 -- 172.21.15.88:0/1235967683 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11c1a21d0 con 0x7fc11c1004e0 2024-02-19T19:20:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc1197fa700 1 -- 172.21.15.88:0/1235967683 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc10c011e20 con 0x7fc11c1004e0 2024-02-19T19:20:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc1197fa700 1 -- 172.21.15.88:0/1235967683 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc10c00f2e0 con 0x7fc11c1004e0 2024-02-19T19:20:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.695+0000 7fc1226e0700 1 -- 172.21.15.88:0/1235967683 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11c1a25e0 con 0x7fc11c1004e0 2024-02-19T19:20:58.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.697+0000 7fc1226e0700 1 -- 172.21.15.88:0/1235967683 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11c108780 con 0x7fc11c1004e0 2024-02-19T19:20:58.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.697+0000 7fc1197fa700 1 -- 172.21.15.88:0/1235967683 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc10c010380 con 0x7fc11c1004e0 2024-02-19T19:20:58.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.697+0000 7fc1197fa700 1 --2- 172.21.15.88:0/1235967683 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10406d8a0 0x7fc10406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:58.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.697+0000 7fc1197fa700 1 -- 172.21.15.88:0/1235967683 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc10c08d0f0 con 0x7fc11c1004e0 2024-02-19T19:20:58.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.697+0000 7fc11b7fe700 1 --2- 172.21.15.88:0/1235967683 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10406d8a0 0x7fc10406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:58.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.698+0000 7fc11b7fe700 1 --2- 172.21.15.88:0/1235967683 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10406d8a0 0x7fc10406fd50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc11c074840 tx=0x7fc11001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:58.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.702+0000 7fc1197fa700 1 -- 172.21.15.88:0/1235967683 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc10c0568a0 con 0x7fc11c1004e0 2024-02-19T19:20:58.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:20:58.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.909+0000 7fc1226e0700 1 -- 172.21.15.88:0/1235967683 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc11c061190 con 0x7fc10406d8a0 2024-02-19T19:20:58.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.919+0000 7fc1197fa700 1 -- 172.21.15.88:0/1235967683 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc11c061190 con 0x7fc10406d8a0 2024-02-19T19:20:58.931 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (18m) 9m ago 28m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (29m) 9m ago 29m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (10m) 9m ago 27m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (14m) 9m ago 29m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (14m) 9m ago 27m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (17m) 9m ago 27m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (10m) 9m ago 22m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (10m) 9m ago 22m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (9m) 9m ago 22m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (22m) 9m ago 22m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (19m) 9m ago 30m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (19m) 9m ago 26m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (16m) 9m ago 30m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:20:58.932 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (15m) 9m ago 26m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:20:58.933 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (18m) 9m ago 28m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:20:58.933 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (18m) 9m ago 26m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:20:58.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (13m) 9m ago 26m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:20:58.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (13m) 9m ago 25m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:20:58.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (12m) 9m ago 25m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:20:58.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (11m) 9m ago 24m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:20:58.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (11m) 9m ago 24m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:20:58.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (10m) 9m ago 23m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:20:58.933 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (18m) 9m ago 27m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:20:58.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.921+0000 7fc1226e0700 1 -- 172.21.15.88:0/1235967683 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10406d8a0 msgr2=0x7fc10406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.921+0000 7fc1226e0700 1 --2- 172.21.15.88:0/1235967683 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10406d8a0 0x7fc10406fd50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc11c074840 tx=0x7fc11001a040 comp rx=0 tx=0).stop 2024-02-19T19:20:58.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.921+0000 7fc1226e0700 1 -- 172.21.15.88:0/1235967683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11c1004e0 msgr2=0x7fc11c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:58.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.921+0000 7fc1226e0700 1 --2- 172.21.15.88:0/1235967683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11c1004e0 0x7fc11c073050 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fc10c00b4b0 tx=0x7fc10c00b870 comp rx=0 tx=0).stop 2024-02-19T19:20:58.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.922+0000 7fc1226e0700 1 -- 172.21.15.88:0/1235967683 shutdown_connections 2024-02-19T19:20:58.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.922+0000 7fc1226e0700 1 --2- 172.21.15.88:0/1235967683 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10406d8a0 0x7fc10406fd50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.922+0000 7fc1226e0700 1 --2- 172.21.15.88:0/1235967683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc11c1004e0 0x7fc11c073050 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.922+0000 7fc1226e0700 1 --2- 172.21.15.88:0/1235967683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc11c100e80 0x7fc11c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:58.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.922+0000 7fc1226e0700 1 -- 172.21.15.88:0/1235967683 >> 172.21.15.88:0/1235967683 conn(0x7fc11c0fbda0 msgr2=0x7fc11c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:58.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.922+0000 7fc1226e0700 1 -- 172.21.15.88:0/1235967683 shutdown_connections 2024-02-19T19:20:58.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:58.922+0000 7fc1226e0700 1 -- 172.21.15.88:0/1235967683 wait complete. 2024-02-19T19:20:59.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.024+0000 7f6c32124700 1 -- 172.21.15.88:0/2646520859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c2c101d30 msgr2=0x7f6c2c10a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:59.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.024+0000 7f6c32124700 1 --2- 172.21.15.88:0/2646520859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c2c101d30 0x7f6c2c10a230 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009910 tx=0x7f6c14009c20 comp rx=0 tx=0).stop 2024-02-19T19:20:59.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.024+0000 7f6c32124700 1 -- 172.21.15.88:0/2646520859 shutdown_connections 2024-02-19T19:20:59.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.025+0000 7f6c32124700 1 --2- 172.21.15.88:0/2646520859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c2c101d30 0x7f6c2c10a230 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.025+0000 7f6c32124700 1 --2- 172.21.15.88:0/2646520859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c2c101420 0x7f6c2c1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.025+0000 7f6c32124700 1 -- 172.21.15.88:0/2646520859 >> 172.21.15.88:0/2646520859 conn(0x7f6c2c0fac80 msgr2=0x7f6c2c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:59.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.025+0000 7f6c32124700 1 -- 172.21.15.88:0/2646520859 shutdown_connections 2024-02-19T19:20:59.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.025+0000 7f6c32124700 1 -- 172.21.15.88:0/2646520859 wait complete. 2024-02-19T19:20:59.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.025+0000 7f6c32124700 1 Processor -- start 2024-02-19T19:20:59.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.025+0000 7f6c32124700 1 -- start start 2024-02-19T19:20:59.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c32124700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c2c101420 0x7f6c2c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:59.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c32124700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c2c101d30 0x7f6c2c1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:59.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c32124700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c196a80 con 0x7f6c2c101420 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c32124700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c19a810 con 0x7f6c2c101d30 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c2b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c2c101420 0x7f6c2c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c2b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c2c101420 0x7f6c2c195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38470/0 (socket says 172.21.15.88:38470) 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c2b7fe700 1 -- 172.21.15.88:0/105038927 learned_addr learned my addr 172.21.15.88:0/105038927 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c2affd700 1 --2- 172.21.15.88:0/105038927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c2c101d30 0x7f6c2c1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c2b7fe700 1 -- 172.21.15.88:0/105038927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c2c101d30 msgr2=0x7f6c2c1963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c2b7fe700 1 --2- 172.21.15.88:0/105038927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c2c101d30 0x7f6c2c1963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c2b7fe700 1 -- 172.21.15.88:0/105038927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c140095f0 con 0x7f6c2c101420 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c2b7fe700 1 --2- 172.21.15.88:0/105038927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c2c101420 0x7f6c2c195e60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0060b0 tx=0x7f6c1c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.026+0000 7f6c28ff9700 1 -- 172.21.15.88:0/105038927 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c1c00dda0 con 0x7f6c2c101420 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.027+0000 7f6c32124700 1 -- 172.21.15.88:0/105038927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c19aaf0 con 0x7f6c2c101420 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.027+0000 7f6c28ff9700 1 -- 172.21.15.88:0/105038927 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c1c00f040 con 0x7f6c2c101420 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.027+0000 7f6c28ff9700 1 -- 172.21.15.88:0/105038927 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c1c014790 con 0x7f6c2c101420 2024-02-19T19:20:59.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.027+0000 7f6c32124700 1 -- 172.21.15.88:0/105038927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c19b040 con 0x7f6c2c101420 2024-02-19T19:20:59.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.028+0000 7f6c32124700 1 -- 172.21.15.88:0/105038927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c1078a0 con 0x7f6c2c101420 2024-02-19T19:20:59.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.029+0000 7f6c28ff9700 1 -- 172.21.15.88:0/105038927 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6c1c014980 con 0x7f6c2c101420 2024-02-19T19:20:59.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.029+0000 7f6c28ff9700 1 --2- 172.21.15.88:0/105038927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 0x7f6c1806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:59.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.029+0000 7f6c28ff9700 1 -- 172.21.15.88:0/105038927 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6c1c08d010 con 0x7f6c2c101420 2024-02-19T19:20:59.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.029+0000 7f6c2affd700 1 --2- 172.21.15.88:0/105038927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 0x7f6c1806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:59.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.030+0000 7f6c2affd700 1 --2- 172.21.15.88:0/105038927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 0x7f6c1806fd50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6c14005b40 tx=0x7f6c1401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:59.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.034+0000 7f6c28ff9700 1 -- 172.21.15.88:0/105038927 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c1c056870 con 0x7f6c2c101420 2024-02-19T19:20:59.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.312+0000 7f6c32124700 1 -- 172.21.15.88:0/105038927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6c2c04f2a0 con 0x7f6c2c101420 2024-02-19T19:20:59.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.312+0000 7f6c28ff9700 1 -- 172.21.15.88:0/105038927 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f6c1c059f30 con 0x7f6c2c101420 2024-02-19T19:20:59.324 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:20:59.324 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:20:59.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:20:59.324 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:20:59.325 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:20:59.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.315+0000 7f6c32124700 1 -- 172.21.15.88:0/105038927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 msgr2=0x7f6c1806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:59.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.315+0000 7f6c32124700 1 --2- 172.21.15.88:0/105038927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 0x7f6c1806fd50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6c14005b40 tx=0x7f6c1401a040 comp rx=0 tx=0).stop 2024-02-19T19:20:59.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.315+0000 7f6c32124700 1 -- 172.21.15.88:0/105038927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c2c101420 msgr2=0x7f6c2c195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:59.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.315+0000 7f6c32124700 1 --2- 172.21.15.88:0/105038927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c2c101420 0x7f6c2c195e60 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0060b0 tx=0x7f6c1c00d3b0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.315+0000 7f6c32124700 1 -- 172.21.15.88:0/105038927 shutdown_connections 2024-02-19T19:20:59.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.315+0000 7f6c32124700 1 --2- 172.21.15.88:0/105038927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 0x7f6c1806fd50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.315+0000 7f6c32124700 1 --2- 172.21.15.88:0/105038927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c2c101420 0x7f6c2c195e60 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.315+0000 7f6c32124700 1 --2- 172.21.15.88:0/105038927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c2c101d30 0x7f6c2c1963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.315+0000 7f6c32124700 1 -- 172.21.15.88:0/105038927 >> 172.21.15.88:0/105038927 conn(0x7f6c2c0fac80 msgr2=0x7f6c2c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:59.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.315+0000 7f6c32124700 1 -- 172.21.15.88:0/105038927 shutdown_connections 2024-02-19T19:20:59.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.315+0000 7f6c32124700 1 -- 172.21.15.88:0/105038927 wait complete. 2024-02-19T19:20:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.417+0000 7f70af0ab700 1 -- 172.21.15.88:0/2215544314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70a8100e80 msgr2=0x7f70a8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.417+0000 7f70af0ab700 1 --2- 172.21.15.88:0/2215544314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70a8100e80 0x7f70a8104e50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f7094009910 tx=0x7f7094009c20 comp rx=0 tx=0).stop 2024-02-19T19:20:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 -- 172.21.15.88:0/2215544314 shutdown_connections 2024-02-19T19:20:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 --2- 172.21.15.88:0/2215544314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70a8100e80 0x7f70a8104e50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 --2- 172.21.15.88:0/2215544314 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a81004e0 0x7f70a81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 -- 172.21.15.88:0/2215544314 >> 172.21.15.88:0/2215544314 conn(0x7f70a80fbda0 msgr2=0x7f70a80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 -- 172.21.15.88:0/2215544314 shutdown_connections 2024-02-19T19:20:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 -- 172.21.15.88:0/2215544314 wait complete. 2024-02-19T19:20:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 Processor -- start 2024-02-19T19:20:59.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 -- start start 2024-02-19T19:20:59.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70a81004e0 0x7f70a8195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:59.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a8100e80 0x7f70a8196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:59.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a8196b30 con 0x7f70a81004e0 2024-02-19T19:20:59.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.418+0000 7f70af0ab700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a819a850 con 0x7f70a8100e80 2024-02-19T19:20:59.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70ace47700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70a81004e0 0x7f70a8195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:59.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70ace47700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70a81004e0 0x7f70a8195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38484/0 (socket says 172.21.15.88:38484) 2024-02-19T19:20:59.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70ace47700 1 -- 172.21.15.88:0/1944944432 learned_addr learned my addr 172.21.15.88:0/1944944432 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:59.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70a7fff700 1 --2- 172.21.15.88:0/1944944432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a8100e80 0x7f70a8196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:59.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70ace47700 1 -- 172.21.15.88:0/1944944432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a8100e80 msgr2=0x7f70a8196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:59.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70ace47700 1 --2- 172.21.15.88:0/1944944432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a8100e80 0x7f70a8196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70ace47700 1 -- 172.21.15.88:0/1944944432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70940095f0 con 0x7f70a81004e0 2024-02-19T19:20:59.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70ace47700 1 --2- 172.21.15.88:0/1944944432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70a81004e0 0x7f70a8195f10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f709c00d8d0 tx=0x7f709c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:59.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70a5ffb700 1 -- 172.21.15.88:0/1944944432 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f709c0098a0 con 0x7f70a81004e0 2024-02-19T19:20:59.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70a5ffb700 1 -- 172.21.15.88:0/1944944432 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f709c010460 con 0x7f70a81004e0 2024-02-19T19:20:59.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70af0ab700 1 -- 172.21.15.88:0/1944944432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a819ab30 con 0x7f70a81004e0 2024-02-19T19:20:59.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70a5ffb700 1 -- 172.21.15.88:0/1944944432 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f709c00f5f0 con 0x7f70a81004e0 2024-02-19T19:20:59.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.419+0000 7f70af0ab700 1 -- 172.21.15.88:0/1944944432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a819b080 con 0x7f70a81004e0 2024-02-19T19:20:59.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.420+0000 7f70af0ab700 1 -- 172.21.15.88:0/1944944432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a8108850 con 0x7f70a81004e0 2024-02-19T19:20:59.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.421+0000 7f70a5ffb700 1 -- 172.21.15.88:0/1944944432 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f709c009a00 con 0x7f70a81004e0 2024-02-19T19:20:59.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.421+0000 7f70a5ffb700 1 --2- 172.21.15.88:0/1944944432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f709806d8a0 0x7f709806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:59.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.421+0000 7f70a5ffb700 1 -- 172.21.15.88:0/1944944432 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f709c08d310 con 0x7f70a81004e0 2024-02-19T19:20:59.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.421+0000 7f70a7fff700 1 --2- 172.21.15.88:0/1944944432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f709806d8a0 0x7f709806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:59.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.422+0000 7f70a7fff700 1 --2- 172.21.15.88:0/1944944432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f709806d8a0 0x7f709806fd50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f70a81974c0 tx=0x7f709401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:59.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.427+0000 7f70a5ffb700 1 -- 172.21.15.88:0/1944944432 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f709c056b40 con 0x7f70a81004e0 2024-02-19T19:20:59.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:59 smithi088 ceph-mon[88643]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:59.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:59 smithi088 ceph-mon[88643]: pgmap v521: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:59.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:59 smithi088 ceph-mon[88643]: from='client.34978 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:59.591 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:20:59 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/105038927' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:20:59.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.675+0000 7f70af0ab700 1 -- 172.21.15.88:0/1944944432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f70a819b2f0 con 0x7f70a81004e0 2024-02-19T19:20:59.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.675+0000 7f70a5ffb700 1 -- 172.21.15.88:0/1944944432 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f709c016740 con 0x7f70a81004e0 2024-02-19T19:20:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:20:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:20:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:20:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:20:59.688 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:20:59.689 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:20:59.689 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:20:59.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:20:59.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:20:59.689 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:20:59.689 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:20:59.689 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:20:59.689 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:20:59.689 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:20:59.689 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:20:59.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.678+0000 7f70af0ab700 1 -- 172.21.15.88:0/1944944432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f709806d8a0 msgr2=0x7f709806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:59.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.678+0000 7f70af0ab700 1 --2- 172.21.15.88:0/1944944432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f709806d8a0 0x7f709806fd50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f70a81974c0 tx=0x7f709401a040 comp rx=0 tx=0).stop 2024-02-19T19:20:59.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.678+0000 7f70af0ab700 1 -- 172.21.15.88:0/1944944432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70a81004e0 msgr2=0x7f70a8195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:59.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.678+0000 7f70af0ab700 1 --2- 172.21.15.88:0/1944944432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70a81004e0 0x7f70a8195f10 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f709c00d8d0 tx=0x7f709c00dbe0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.678+0000 7f70af0ab700 1 -- 172.21.15.88:0/1944944432 shutdown_connections 2024-02-19T19:20:59.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.678+0000 7f70af0ab700 1 --2- 172.21.15.88:0/1944944432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f709806d8a0 0x7f709806fd50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.678+0000 7f70af0ab700 1 --2- 172.21.15.88:0/1944944432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70a81004e0 0x7f70a8195f10 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.678+0000 7f70af0ab700 1 --2- 172.21.15.88:0/1944944432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70a8100e80 0x7f70a8196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.678+0000 7f70af0ab700 1 -- 172.21.15.88:0/1944944432 >> 172.21.15.88:0/1944944432 conn(0x7f70a80fbda0 msgr2=0x7f70a80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:59.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.678+0000 7f70af0ab700 1 -- 172.21.15.88:0/1944944432 shutdown_connections 2024-02-19T19:20:59.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.678+0000 7f70af0ab700 1 -- 172.21.15.88:0/1944944432 wait complete. 2024-02-19T19:20:59.691 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:20:59.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:59 smithi132 ceph-mon[70676]: from='client.34974 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:59.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:59 smithi132 ceph-mon[70676]: pgmap v521: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:20:59.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:59 smithi132 ceph-mon[70676]: from='client.34978 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:20:59.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:20:59 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/105038927' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:20:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.782+0000 7fda64c97700 1 -- 172.21.15.88:0/2164484117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda600ffbd0 msgr2=0x7fda60100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.782+0000 7fda64c97700 1 --2- 172.21.15.88:0/2164484117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda600ffbd0 0x7fda60100040 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fda50009910 tx=0x7fda50009c20 comp rx=0 tx=0).stop 2024-02-19T19:20:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.782+0000 7fda64c97700 1 -- 172.21.15.88:0/2164484117 shutdown_connections 2024-02-19T19:20:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.782+0000 7fda64c97700 1 --2- 172.21.15.88:0/2164484117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda600ffbd0 0x7fda60100040 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.782+0000 7fda64c97700 1 --2- 172.21.15.88:0/2164484117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60104130 0x7fda60104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.782+0000 7fda64c97700 1 -- 172.21.15.88:0/2164484117 >> 172.21.15.88:0/2164484117 conn(0x7fda60075130 msgr2=0x7fda60075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:20:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.782+0000 7fda64c97700 1 -- 172.21.15.88:0/2164484117 shutdown_connections 2024-02-19T19:20:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.782+0000 7fda64c97700 1 -- 172.21.15.88:0/2164484117 wait complete. 2024-02-19T19:20:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda64c97700 1 Processor -- start 2024-02-19T19:20:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda64c97700 1 -- start start 2024-02-19T19:20:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda64c97700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda600ffbd0 0x7fda60194e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda64c97700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60104130 0x7fda60195390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda64c97700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda60195a70 con 0x7fda600ffbd0 2024-02-19T19:20:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda64c97700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda60199800 con 0x7fda60104130 2024-02-19T19:20:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda5e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda600ffbd0 0x7fda60194e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda5e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda600ffbd0 0x7fda60194e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38512/0 (socket says 172.21.15.88:38512) 2024-02-19T19:20:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda5e59c700 1 -- 172.21.15.88:0/2315384795 learned_addr learned my addr 172.21.15.88:0/2315384795 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:20:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda5dd9b700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60104130 0x7fda60195390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:59.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda5e59c700 1 -- 172.21.15.88:0/2315384795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60104130 msgr2=0x7fda60195390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:20:59.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda5e59c700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60104130 0x7fda60195390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:20:59.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda5e59c700 1 -- 172.21.15.88:0/2315384795 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda500095f0 con 0x7fda600ffbd0 2024-02-19T19:20:59.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.783+0000 7fda5dd9b700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60104130 0x7fda60195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:20:59.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.784+0000 7fda5e59c700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda600ffbd0 0x7fda60194e50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fda5400b4b0 tx=0x7fda5400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:59.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.784+0000 7fda4f7fe700 1 -- 172.21.15.88:0/2315384795 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda54011800 con 0x7fda600ffbd0 2024-02-19T19:20:59.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.784+0000 7fda4f7fe700 1 -- 172.21.15.88:0/2315384795 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda54011e20 con 0x7fda600ffbd0 2024-02-19T19:20:59.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.784+0000 7fda4f7fe700 1 -- 172.21.15.88:0/2315384795 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda5400f2e0 con 0x7fda600ffbd0 2024-02-19T19:20:59.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.784+0000 7fda64c97700 1 -- 172.21.15.88:0/2315384795 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda60199ae0 con 0x7fda600ffbd0 2024-02-19T19:20:59.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.784+0000 7fda64c97700 1 -- 172.21.15.88:0/2315384795 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda60199fe0 con 0x7fda600ffbd0 2024-02-19T19:20:59.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.785+0000 7fda64c97700 1 -- 172.21.15.88:0/2315384795 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6004f2a0 con 0x7fda600ffbd0 2024-02-19T19:20:59.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.785+0000 7fda4f7fe700 1 -- 172.21.15.88:0/2315384795 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fda54010380 con 0x7fda600ffbd0 2024-02-19T19:20:59.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.786+0000 7fda4f7fe700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fda4806d8a0 0x7fda4806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:20:59.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.786+0000 7fda4f7fe700 1 -- 172.21.15.88:0/2315384795 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fda5408d0f0 con 0x7fda600ffbd0 2024-02-19T19:20:59.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.786+0000 7fda5dd9b700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fda4806d8a0 0x7fda4806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:20:59.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.786+0000 7fda5dd9b700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fda4806d8a0 0x7fda4806fd50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fda50006010 tx=0x7fda5000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:20:59.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:20:59.791+0000 7fda4f7fe700 1 -- 172.21.15.88:0/2315384795 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fda540568a0 con 0x7fda600ffbd0 2024-02-19T19:21:00.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.003+0000 7fda64c97700 1 -- 172.21.15.88:0/2315384795 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fda6019a3c0 con 0x7fda4806d8a0 2024-02-19T19:21:00.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.005+0000 7fda4f7fe700 1 -- 172.21.15.88:0/2315384795 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fda6019a3c0 con 0x7fda4806d8a0 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:21:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:21:00.017 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:21:00.017 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:21:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.007+0000 7fda64c97700 1 -- 172.21.15.88:0/2315384795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fda4806d8a0 msgr2=0x7fda4806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.007+0000 7fda64c97700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fda4806d8a0 0x7fda4806fd50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fda50006010 tx=0x7fda5000b540 comp rx=0 tx=0).stop 2024-02-19T19:21:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.007+0000 7fda64c97700 1 -- 172.21.15.88:0/2315384795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda600ffbd0 msgr2=0x7fda60194e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.007+0000 7fda64c97700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda600ffbd0 0x7fda60194e50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fda5400b4b0 tx=0x7fda5400b870 comp rx=0 tx=0).stop 2024-02-19T19:21:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.007+0000 7fda64c97700 1 -- 172.21.15.88:0/2315384795 shutdown_connections 2024-02-19T19:21:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.007+0000 7fda64c97700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fda4806d8a0 0x7fda4806fd50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.007+0000 7fda64c97700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda600ffbd0 0x7fda60194e50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.007+0000 7fda64c97700 1 --2- 172.21.15.88:0/2315384795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fda60104130 0x7fda60195390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.007+0000 7fda64c97700 1 -- 172.21.15.88:0/2315384795 >> 172.21.15.88:0/2315384795 conn(0x7fda60075130 msgr2=0x7fda600fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.007+0000 7fda64c97700 1 -- 172.21.15.88:0/2315384795 shutdown_connections 2024-02-19T19:21:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.007+0000 7fda64c97700 1 -- 172.21.15.88:0/2315384795 wait complete. 2024-02-19T19:21:00.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.113+0000 7fd02b7f4700 1 -- 172.21.15.88:0/1560356080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd024069140 msgr2=0x7fd024104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:00.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.113+0000 7fd02b7f4700 1 --2- 172.21.15.88:0/1560356080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd024069140 0x7fd024104eb0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd018009910 tx=0x7fd018009c20 comp rx=0 tx=0).stop 2024-02-19T19:21:00.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.113+0000 7fd02b7f4700 1 -- 172.21.15.88:0/1560356080 shutdown_connections 2024-02-19T19:21:00.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.113+0000 7fd02b7f4700 1 --2- 172.21.15.88:0/1560356080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd024069140 0x7fd024104eb0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:00.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.113+0000 7fd02b7f4700 1 --2- 172.21.15.88:0/1560356080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd0240687a0 0x7fd024068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:00.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.113+0000 7fd02b7f4700 1 -- 172.21.15.88:0/1560356080 >> 172.21.15.88:0/1560356080 conn(0x7fd024075130 msgr2=0x7fd024075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:00.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.113+0000 7fd02b7f4700 1 -- 172.21.15.88:0/1560356080 shutdown_connections 2024-02-19T19:21:00.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.113+0000 7fd02b7f4700 1 -- 172.21.15.88:0/1560356080 wait complete. 2024-02-19T19:21:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.113+0000 7fd02b7f4700 1 Processor -- start 2024-02-19T19:21:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.113+0000 7fd02b7f4700 1 -- start start 2024-02-19T19:21:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd02b7f4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0240687a0 0x7fd024195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd02b7f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd024069140 0x7fd0241963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd02b7f4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd024196a80 con 0x7fd0240687a0 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd02b7f4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02419a810 con 0x7fd024069140 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd029590700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0240687a0 0x7fd024195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd029590700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0240687a0 0x7fd024195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38540/0 (socket says 172.21.15.88:38540) 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd029590700 1 -- 172.21.15.88:0/4271139971 learned_addr learned my addr 172.21.15.88:0/4271139971 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd029590700 1 -- 172.21.15.88:0/4271139971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd024069140 msgr2=0x7fd0241963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd029590700 1 --2- 172.21.15.88:0/4271139971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd024069140 0x7fd0241963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd029590700 1 -- 172.21.15.88:0/4271139971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0180095f0 con 0x7fd0240687a0 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd029590700 1 --2- 172.21.15.88:0/4271139971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0240687a0 0x7fd024195e60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0060b0 tx=0x7fd01c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd0227fc700 1 -- 172.21.15.88:0/4271139971 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd01c00dda0 con 0x7fd0240687a0 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd02b7f4700 1 -- 172.21.15.88:0/4271139971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02419aaf0 con 0x7fd0240687a0 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd0227fc700 1 -- 172.21.15.88:0/4271139971 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd01c00f040 con 0x7fd0240687a0 2024-02-19T19:21:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd0227fc700 1 -- 172.21.15.88:0/4271139971 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd01c014790 con 0x7fd0240687a0 2024-02-19T19:21:00.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.114+0000 7fd02b7f4700 1 -- 172.21.15.88:0/4271139971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02419b040 con 0x7fd0240687a0 2024-02-19T19:21:00.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.115+0000 7fd02b7f4700 1 -- 172.21.15.88:0/4271139971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd024108750 con 0x7fd0240687a0 2024-02-19T19:21:00.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.116+0000 7fd0227fc700 1 -- 172.21.15.88:0/4271139971 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd01c014980 con 0x7fd0240687a0 2024-02-19T19:21:00.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.116+0000 7fd0227fc700 1 --2- 172.21.15.88:0/4271139971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00c06d8a0 0x7fd00c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:00.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.116+0000 7fd0227fc700 1 -- 172.21.15.88:0/4271139971 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd01c08d010 con 0x7fd0240687a0 2024-02-19T19:21:00.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.116+0000 7fd028d8f700 1 --2- 172.21.15.88:0/4271139971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00c06d8a0 0x7fd00c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:00.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.117+0000 7fd028d8f700 1 --2- 172.21.15.88:0/4271139971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00c06d8a0 0x7fd00c06fd50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd01800b5c0 tx=0x7fd018005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:00.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.121+0000 7fd0227fc700 1 -- 172.21.15.88:0/4271139971 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd01c056870 con 0x7fd0240687a0 2024-02-19T19:21:00.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.401+0000 7fd02b7f4700 1 -- 172.21.15.88:0/4271139971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd02404f2a0 con 0x7fd0240687a0 2024-02-19T19:21:00.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.401+0000 7fd0227fc700 1 -- 172.21.15.88:0/4271139971 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fd01c059f30 con 0x7fd0240687a0 2024-02-19T19:21:00.412 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:21:00.412 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:21:00.413 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:21:00.413 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:21:00.413 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:21:00.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.404+0000 7fd02b7f4700 1 -- 172.21.15.88:0/4271139971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00c06d8a0 msgr2=0x7fd00c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:00.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.404+0000 7fd02b7f4700 1 --2- 172.21.15.88:0/4271139971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00c06d8a0 0x7fd00c06fd50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd01800b5c0 tx=0x7fd018005960 comp rx=0 tx=0).stop 2024-02-19T19:21:00.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.404+0000 7fd02b7f4700 1 -- 172.21.15.88:0/4271139971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0240687a0 msgr2=0x7fd024195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:00.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.404+0000 7fd02b7f4700 1 --2- 172.21.15.88:0/4271139971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0240687a0 0x7fd024195e60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd01c0060b0 tx=0x7fd01c00d3b0 comp rx=0 tx=0).stop 2024-02-19T19:21:00.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.404+0000 7fd02b7f4700 1 -- 172.21.15.88:0/4271139971 shutdown_connections 2024-02-19T19:21:00.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.404+0000 7fd02b7f4700 1 --2- 172.21.15.88:0/4271139971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00c06d8a0 0x7fd00c06fd50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:00.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.404+0000 7fd02b7f4700 1 --2- 172.21.15.88:0/4271139971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0240687a0 0x7fd024195e60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:00.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.404+0000 7fd02b7f4700 1 --2- 172.21.15.88:0/4271139971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd024069140 0x7fd0241963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:00.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.404+0000 7fd02b7f4700 1 -- 172.21.15.88:0/4271139971 >> 172.21.15.88:0/4271139971 conn(0x7fd024075130 msgr2=0x7fd024104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:00.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.404+0000 7fd02b7f4700 1 -- 172.21.15.88:0/4271139971 shutdown_connections 2024-02-19T19:21:00.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:00.404+0000 7fd02b7f4700 1 -- 172.21.15.88:0/4271139971 wait complete. 2024-02-19T19:21:00.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:00 smithi132 ceph-mon[70676]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:00.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:00 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1944944432' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:21:00.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:00 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4271139971' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:21:00.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:00 smithi088 ceph-mon[88643]: from='client.34982 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:00.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:00 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1944944432' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:21:00.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:00 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4271139971' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:21:01.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:01 smithi132 ceph-mon[70676]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:01.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:01 smithi132 ceph-mon[70676]: pgmap v522: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:01.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:01 smithi088 ceph-mon[88643]: from='client.34994 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:01.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:01 smithi088 ceph-mon[88643]: pgmap v522: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:03.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:03 smithi132 ceph-mon[70676]: pgmap v523: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:03.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:03 smithi088 ceph-mon[88643]: pgmap v523: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:05.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:05 smithi132 ceph-mon[70676]: pgmap v524: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:05.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:05 smithi088 ceph-mon[88643]: pgmap v524: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:07.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:07 smithi132 ceph-mon[70676]: pgmap v525: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:07.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:07 smithi088 ceph-mon[88643]: pgmap v525: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:09.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:09 smithi132 ceph-mon[70676]: pgmap v526: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:09.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:09 smithi088 ceph-mon[88643]: pgmap v526: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:11.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:11 smithi088 ceph-mon[88643]: pgmap v527: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:11.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:11 smithi132 ceph-mon[70676]: pgmap v527: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:13.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:13 smithi088 ceph-mon[88643]: pgmap v528: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:13.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:21:13.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:13 smithi132 ceph-mon[70676]: pgmap v528: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:13.993 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:21:15.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:15 smithi088 ceph-mon[88643]: pgmap v529: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:15.993 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:15 smithi132 ceph-mon[70676]: pgmap v529: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:17.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:17 smithi088 ceph-mon[88643]: pgmap v530: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:17.975 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:17 smithi132 ceph-mon[70676]: pgmap v530: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:19.743 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:19 smithi132 ceph-mon[70676]: pgmap v531: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:19.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:19 smithi088 ceph-mon[88643]: pgmap v531: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:21.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:21 smithi088 ceph-mon[88643]: pgmap v532: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:21.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:21 smithi132 ceph-mon[70676]: pgmap v532: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:23.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:23 smithi088 ceph-mon[88643]: pgmap v533: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:23.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:23 smithi132 ceph-mon[70676]: pgmap v533: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:25.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:25 smithi088 ceph-mon[88643]: pgmap v534: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:25.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:25 smithi132 ceph-mon[70676]: pgmap v534: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:27.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:27 smithi088 ceph-mon[88643]: pgmap v535: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:27.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:27 smithi132 ceph-mon[70676]: pgmap v535: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:28.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:21:28.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:21:29.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:29 smithi088 ceph-mon[88643]: pgmap v536: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:29.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:29 smithi132 ceph-mon[70676]: pgmap v536: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:30.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.523+0000 7f25746fe700 1 -- 172.21.15.88:0/657242820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f256c100e80 msgr2=0x7f256c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:30.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.523+0000 7f25746fe700 1 --2- 172.21.15.88:0/657242820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f256c100e80 0x7f256c104e50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2568009910 tx=0x7f2568009c20 comp rx=0 tx=0).stop 2024-02-19T19:21:30.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.524+0000 7f25746fe700 1 -- 172.21.15.88:0/657242820 shutdown_connections 2024-02-19T19:21:30.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.524+0000 7f25746fe700 1 --2- 172.21.15.88:0/657242820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f256c100e80 0x7f256c104e50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.524+0000 7f25746fe700 1 --2- 172.21.15.88:0/657242820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f256c1004e0 0x7f256c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.524+0000 7f25746fe700 1 -- 172.21.15.88:0/657242820 >> 172.21.15.88:0/657242820 conn(0x7f256c0fbda0 msgr2=0x7f256c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.524+0000 7f25746fe700 1 -- 172.21.15.88:0/657242820 shutdown_connections 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.524+0000 7f25746fe700 1 -- 172.21.15.88:0/657242820 wait complete. 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.524+0000 7f25746fe700 1 Processor -- start 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.524+0000 7f25746fe700 1 -- start start 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f25746fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f256c1004e0 0x7f256c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f25746fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f256c100e80 0x7f256c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f25746fe700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c073ad0 con 0x7f256c100e80 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f25746fe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256c073c10 con 0x7f256c1004e0 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f2571c99700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f256c100e80 0x7f256c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f2571c99700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f256c100e80 0x7f256c073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33344/0 (socket says 172.21.15.88:33344) 2024-02-19T19:21:30.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f2571c99700 1 -- 172.21.15.88:0/2551205675 learned_addr learned my addr 172.21.15.88:0/2551205675 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:21:30.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f257249a700 1 --2- 172.21.15.88:0/2551205675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f256c1004e0 0x7f256c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:30.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f2571c99700 1 -- 172.21.15.88:0/2551205675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f256c1004e0 msgr2=0x7f256c073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:30.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f2571c99700 1 --2- 172.21.15.88:0/2551205675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f256c1004e0 0x7f256c073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:30.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f2571c99700 1 -- 172.21.15.88:0/2551205675 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25680095f0 con 0x7f256c100e80 2024-02-19T19:21:30.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.525+0000 7f2571c99700 1 --2- 172.21.15.88:0/2551205675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f256c100e80 0x7f256c073590 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f256800bda0 tx=0x7f256800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:30.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.526+0000 7f25637fe700 1 -- 172.21.15.88:0/2551205675 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256801d070 con 0x7f256c100e80 2024-02-19T19:21:30.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.526+0000 7f25637fe700 1 -- 172.21.15.88:0/2551205675 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2568004fb0 con 0x7f256c100e80 2024-02-19T19:21:30.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.526+0000 7f25746fe700 1 -- 172.21.15.88:0/2551205675 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256c073e90 con 0x7f256c100e80 2024-02-19T19:21:30.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.526+0000 7f25637fe700 1 -- 172.21.15.88:0/2551205675 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f256800f480 con 0x7f256c100e80 2024-02-19T19:21:30.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.526+0000 7f25746fe700 1 -- 172.21.15.88:0/2551205675 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256c1a23c0 con 0x7f256c100e80 2024-02-19T19:21:30.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.527+0000 7f25746fe700 1 -- 172.21.15.88:0/2551205675 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f256c108850 con 0x7f256c100e80 2024-02-19T19:21:30.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.527+0000 7f25637fe700 1 -- 172.21.15.88:0/2551205675 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2568022870 con 0x7f256c100e80 2024-02-19T19:21:30.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.528+0000 7f25637fe700 1 --2- 172.21.15.88:0/2551205675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255806d8a0 0x7f255806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:30.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.528+0000 7f25637fe700 1 -- 172.21.15.88:0/2551205675 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f256808e9a0 con 0x7f256c100e80 2024-02-19T19:21:30.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.528+0000 7f257249a700 1 --2- 172.21.15.88:0/2551205675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255806d8a0 0x7f255806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:30.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.528+0000 7f257249a700 1 --2- 172.21.15.88:0/2551205675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255806d8a0 0x7f255806fd50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f255c005ea0 tx=0x7f255c005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:30.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.533+0000 7f25637fe700 1 -- 172.21.15.88:0/2551205675 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2568058280 con 0x7f256c100e80 2024-02-19T19:21:30.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.749+0000 7f25746fe700 1 -- 172.21.15.88:0/2551205675 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f256c074660 con 0x7f255806d8a0 2024-02-19T19:21:30.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.750+0000 7f25637fe700 1 -- 172.21.15.88:0/2551205675 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f256c074660 con 0x7f255806d8a0 2024-02-19T19:21:30.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.752+0000 7f25746fe700 1 -- 172.21.15.88:0/2551205675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255806d8a0 msgr2=0x7f255806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:30.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.752+0000 7f25746fe700 1 --2- 172.21.15.88:0/2551205675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255806d8a0 0x7f255806fd50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f255c005ea0 tx=0x7f255c005e30 comp rx=0 tx=0).stop 2024-02-19T19:21:30.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.752+0000 7f25746fe700 1 -- 172.21.15.88:0/2551205675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f256c100e80 msgr2=0x7f256c073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:30.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.752+0000 7f25746fe700 1 --2- 172.21.15.88:0/2551205675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f256c100e80 0x7f256c073590 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f256800bda0 tx=0x7f256800be80 comp rx=0 tx=0).stop 2024-02-19T19:21:30.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.752+0000 7f25746fe700 1 -- 172.21.15.88:0/2551205675 shutdown_connections 2024-02-19T19:21:30.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.752+0000 7f25746fe700 1 --2- 172.21.15.88:0/2551205675 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f255806d8a0 0x7f255806fd50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:30.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.752+0000 7f25746fe700 1 --2- 172.21.15.88:0/2551205675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f256c1004e0 0x7f256c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:30.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.752+0000 7f25746fe700 1 --2- 172.21.15.88:0/2551205675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f256c100e80 0x7f256c073590 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:30.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.752+0000 7f25746fe700 1 -- 172.21.15.88:0/2551205675 >> 172.21.15.88:0/2551205675 conn(0x7f256c0fbda0 msgr2=0x7f256c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:30.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.752+0000 7f25746fe700 1 -- 172.21.15.88:0/2551205675 shutdown_connections 2024-02-19T19:21:30.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.752+0000 7f25746fe700 1 -- 172.21.15.88:0/2551205675 wait complete. 2024-02-19T19:21:30.773 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:21:30.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.857+0000 7f1cac295700 1 -- 172.21.15.88:0/3791532023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ca4100d70 msgr2=0x7f1ca41094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:30.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.857+0000 7f1cac295700 1 --2- 172.21.15.88:0/3791532023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ca4100d70 0x7f1ca41094d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0009940 tx=0x7f1ca0009c50 comp rx=0 tx=0).stop 2024-02-19T19:21:30.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.857+0000 7f1cac295700 1 -- 172.21.15.88:0/3791532023 shutdown_connections 2024-02-19T19:21:30.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.857+0000 7f1cac295700 1 --2- 172.21.15.88:0/3791532023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ca4100d70 0x7f1ca41094d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:30.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.857+0000 7f1cac295700 1 --2- 172.21.15.88:0/3791532023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ca4100460 0x7f1ca4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:30.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.857+0000 7f1cac295700 1 -- 172.21.15.88:0/3791532023 >> 172.21.15.88:0/3791532023 conn(0x7f1ca40748b0 msgr2=0x7f1ca4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:30.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.857+0000 7f1cac295700 1 -- 172.21.15.88:0/3791532023 shutdown_connections 2024-02-19T19:21:30.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.857+0000 7f1cac295700 1 -- 172.21.15.88:0/3791532023 wait complete. 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.858+0000 7f1cac295700 1 Processor -- start 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.858+0000 7f1cac295700 1 -- start start 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.858+0000 7f1cac295700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ca4100460 0x7f1ca419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.858+0000 7f1cac295700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ca4100d70 0x7f1ca419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.858+0000 7f1cac295700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca419b5c0 con 0x7f1ca4100460 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.858+0000 7f1cac295700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4194a70 con 0x7f1ca4100d70 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.858+0000 7f1caa031700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ca4100460 0x7f1ca419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.858+0000 7f1caa031700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ca4100460 0x7f1ca419a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33362/0 (socket says 172.21.15.88:33362) 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.858+0000 7f1caa031700 1 -- 172.21.15.88:0/3880199211 learned_addr learned my addr 172.21.15.88:0/3880199211 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.858+0000 7f1ca9830700 1 --2- 172.21.15.88:0/3880199211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ca4100d70 0x7f1ca419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.859+0000 7f1caa031700 1 -- 172.21.15.88:0/3880199211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ca4100d70 msgr2=0x7f1ca419af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:30.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.859+0000 7f1caa031700 1 --2- 172.21.15.88:0/3880199211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ca4100d70 0x7f1ca419af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:30.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.859+0000 7f1caa031700 1 -- 172.21.15.88:0/3880199211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca00095f0 con 0x7f1ca4100460 2024-02-19T19:21:30.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.859+0000 7f1caa031700 1 --2- 172.21.15.88:0/3880199211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ca4100460 0x7f1ca419a9f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400d8d0 tx=0x7f1c9400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:30.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.859+0000 7f1c9b7fe700 1 -- 172.21.15.88:0/3880199211 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c940098a0 con 0x7f1ca4100460 2024-02-19T19:21:30.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.859+0000 7f1cac295700 1 -- 172.21.15.88:0/3880199211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca4194d50 con 0x7f1ca4100460 2024-02-19T19:21:30.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.859+0000 7f1c9b7fe700 1 -- 172.21.15.88:0/3880199211 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c94010460 con 0x7f1ca4100460 2024-02-19T19:21:30.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.859+0000 7f1c9b7fe700 1 -- 172.21.15.88:0/3880199211 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c9400f5f0 con 0x7f1ca4100460 2024-02-19T19:21:30.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.859+0000 7f1cac295700 1 -- 172.21.15.88:0/3880199211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca4195270 con 0x7f1ca4100460 2024-02-19T19:21:30.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.860+0000 7f1cac295700 1 -- 172.21.15.88:0/3880199211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca4106bd0 con 0x7f1ca4100460 2024-02-19T19:21:30.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.860+0000 7f1c9b7fe700 1 -- 172.21.15.88:0/3880199211 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1c94009a00 con 0x7f1ca4100460 2024-02-19T19:21:30.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.861+0000 7f1c9b7fe700 1 --2- 172.21.15.88:0/3880199211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c9006d8a0 0x7f1c9006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:30.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.861+0000 7f1c9b7fe700 1 -- 172.21.15.88:0/3880199211 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1c9408d400 con 0x7f1ca4100460 2024-02-19T19:21:30.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.861+0000 7f1ca9830700 1 --2- 172.21.15.88:0/3880199211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c9006d8a0 0x7f1c9006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:30.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.861+0000 7f1ca9830700 1 --2- 172.21.15.88:0/3880199211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c9006d8a0 0x7f1c9006fd50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0006010 tx=0x7f1ca0005ab0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:30.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:30.865+0000 7f1c9b7fe700 1 -- 172.21.15.88:0/3880199211 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c9405a6e0 con 0x7f1ca4100460 2024-02-19T19:21:31.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.080+0000 7f1cac295700 1 -- 172.21.15.88:0/3880199211 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ca4195490 con 0x7f1c9006d8a0 2024-02-19T19:21:31.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.082+0000 7f1c9b7fe700 1 -- 172.21.15.88:0/3880199211 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1ca4195490 con 0x7f1c9006d8a0 2024-02-19T19:21:31.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.084+0000 7f1cac295700 1 -- 172.21.15.88:0/3880199211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c9006d8a0 msgr2=0x7f1c9006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.084+0000 7f1cac295700 1 --2- 172.21.15.88:0/3880199211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c9006d8a0 0x7f1c9006fd50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1ca0006010 tx=0x7f1ca0005ab0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.084+0000 7f1cac295700 1 -- 172.21.15.88:0/3880199211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ca4100460 msgr2=0x7f1ca419a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.084+0000 7f1cac295700 1 --2- 172.21.15.88:0/3880199211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ca4100460 0x7f1ca419a9f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f1c9400d8d0 tx=0x7f1c9400dbe0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.084+0000 7f1cac295700 1 -- 172.21.15.88:0/3880199211 shutdown_connections 2024-02-19T19:21:31.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.084+0000 7f1cac295700 1 --2- 172.21.15.88:0/3880199211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c9006d8a0 0x7f1c9006fd50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.084+0000 7f1cac295700 1 --2- 172.21.15.88:0/3880199211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ca4100460 0x7f1ca419a9f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.084+0000 7f1cac295700 1 --2- 172.21.15.88:0/3880199211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ca4100d70 0x7f1ca419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.084+0000 7f1cac295700 1 -- 172.21.15.88:0/3880199211 >> 172.21.15.88:0/3880199211 conn(0x7f1ca40748b0 msgr2=0x7f1ca4103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:31.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.084+0000 7f1cac295700 1 -- 172.21.15.88:0/3880199211 shutdown_connections 2024-02-19T19:21:31.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.084+0000 7f1cac295700 1 -- 172.21.15.88:0/3880199211 wait complete. 2024-02-19T19:21:31.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.187+0000 7f9bfcbda700 1 -- 172.21.15.88:0/486915800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf80fe6f0 msgr2=0x7f9bf80feac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.187+0000 7f9bfcbda700 1 --2- 172.21.15.88:0/486915800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf80fe6f0 0x7f9bf80feac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9be8009910 tx=0x7f9be8009c20 comp rx=0 tx=0).stop 2024-02-19T19:21:31.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.188+0000 7f9bfcbda700 1 -- 172.21.15.88:0/486915800 shutdown_connections 2024-02-19T19:21:31.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.188+0000 7f9bfcbda700 1 --2- 172.21.15.88:0/486915800 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf80ff090 0x7f9bf8106080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.188+0000 7f9bfcbda700 1 --2- 172.21.15.88:0/486915800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf80fe6f0 0x7f9bf80feac0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.188+0000 7f9bfcbda700 1 -- 172.21.15.88:0/486915800 >> 172.21.15.88:0/486915800 conn(0x7f9bf8075440 msgr2=0x7f9bf8075840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:31.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.188+0000 7f9bfcbda700 1 -- 172.21.15.88:0/486915800 shutdown_connections 2024-02-19T19:21:31.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.188+0000 7f9bfcbda700 1 -- 172.21.15.88:0/486915800 wait complete. 2024-02-19T19:21:31.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bfcbda700 1 Processor -- start 2024-02-19T19:21:31.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bfcbda700 1 -- start start 2024-02-19T19:21:31.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bfcbda700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf80fe6f0 0x7f9bf8196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:31.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bfcbda700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf80ff090 0x7f9bf8196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:31.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bfcbda700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf8196c40 con 0x7f9bf80fe6f0 2024-02-19T19:21:31.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bfcbda700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf819a9d0 con 0x7f9bf80ff090 2024-02-19T19:21:31.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bf659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf80fe6f0 0x7f9bf8196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:31.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bf659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf80fe6f0 0x7f9bf8196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33380/0 (socket says 172.21.15.88:33380) 2024-02-19T19:21:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bf659c700 1 -- 172.21.15.88:0/3021982868 learned_addr learned my addr 172.21.15.88:0/3021982868 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:21:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bf5d9b700 1 --2- 172.21.15.88:0/3021982868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf80ff090 0x7f9bf8196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bf659c700 1 -- 172.21.15.88:0/3021982868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf80ff090 msgr2=0x7f9bf8196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bf659c700 1 --2- 172.21.15.88:0/3021982868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf80ff090 0x7f9bf8196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.189+0000 7f9bf659c700 1 -- 172.21.15.88:0/3021982868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be80095f0 con 0x7f9bf80fe6f0 2024-02-19T19:21:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.190+0000 7f9bf659c700 1 --2- 172.21.15.88:0/3021982868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf80fe6f0 0x7f9bf8196020 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006010 tx=0x7f9be800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.190+0000 7f9be77fe700 1 -- 172.21.15.88:0/3021982868 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be801d070 con 0x7f9bf80fe6f0 2024-02-19T19:21:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.190+0000 7f9be77fe700 1 -- 172.21.15.88:0/3021982868 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9be800f460 con 0x7f9bf80fe6f0 2024-02-19T19:21:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.190+0000 7f9bfcbda700 1 -- 172.21.15.88:0/3021982868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf819ac50 con 0x7f9bf80fe6f0 2024-02-19T19:21:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.190+0000 7f9be77fe700 1 -- 172.21.15.88:0/3021982868 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be8017610 con 0x7f9bf80fe6f0 2024-02-19T19:21:31.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.190+0000 7f9bfcbda700 1 -- 172.21.15.88:0/3021982868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf819b140 con 0x7f9bf80fe6f0 2024-02-19T19:21:31.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.191+0000 7f9bfcbda700 1 -- 172.21.15.88:0/3021982868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf81037e0 con 0x7f9bf80fe6f0 2024-02-19T19:21:31.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.192+0000 7f9be77fe700 1 -- 172.21.15.88:0/3021982868 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9be800f5d0 con 0x7f9bf80fe6f0 2024-02-19T19:21:31.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.192+0000 7f9be77fe700 1 --2- 172.21.15.88:0/3021982868 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9be006d8a0 0x7f9be006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:31.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.192+0000 7f9be77fe700 1 -- 172.21.15.88:0/3021982868 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9be808eb90 con 0x7f9bf80fe6f0 2024-02-19T19:21:31.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.192+0000 7f9bf5d9b700 1 --2- 172.21.15.88:0/3021982868 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9be006d8a0 0x7f9be006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:31.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.193+0000 7f9bf5d9b700 1 --2- 172.21.15.88:0/3021982868 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9be006d8a0 0x7f9be006fd50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8197640 tx=0x7f9bec009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:31.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.196+0000 7f9be77fe700 1 -- 172.21.15.88:0/3021982868 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9be8058460 con 0x7f9bf80fe6f0 2024-02-19T19:21:31.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.401+0000 7f9bfcbda700 1 -- 172.21.15.88:0/3021982868 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bf819b470 con 0x7f9be006d8a0 2024-02-19T19:21:31.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.412+0000 7f9be77fe700 1 -- 172.21.15.88:0/3021982868 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9bf819b470 con 0x7f9be006d8a0 2024-02-19T19:21:31.422 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:21:31.422 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (18m) 10m ago 29m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (29m) 10m ago 29m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (11m) 10m ago 27m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (14m) 10m ago 29m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (14m) 10m ago 27m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (18m) 10m ago 28m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (10m) 10m ago 22m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (10m) 10m ago 22m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (10m) 10m ago 22m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (22m) 10m ago 22m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (20m) 10m ago 30m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (19m) 10m ago 27m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (16m) 10m ago 31m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (16m) 10m ago 27m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (19m) 10m ago 29m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:21:31.423 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (19m) 10m ago 27m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:21:31.424 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (14m) 10m ago 26m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:21:31.424 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (13m) 10m ago 26m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:21:31.424 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (12m) 10m ago 25m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:21:31.424 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (12m) 10m ago 25m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:21:31.424 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (11m) 10m ago 24m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:21:31.424 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (11m) 10m ago 24m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:21:31.424 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (19m) 10m ago 27m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:21:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.414+0000 7f9bfcbda700 1 -- 172.21.15.88:0/3021982868 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9be006d8a0 msgr2=0x7f9be006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.414+0000 7f9bfcbda700 1 --2- 172.21.15.88:0/3021982868 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9be006d8a0 0x7f9be006fd50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9bf8197640 tx=0x7f9bec009190 comp rx=0 tx=0).stop 2024-02-19T19:21:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.414+0000 7f9bfcbda700 1 -- 172.21.15.88:0/3021982868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf80fe6f0 msgr2=0x7f9bf8196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.414+0000 7f9bfcbda700 1 --2- 172.21.15.88:0/3021982868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf80fe6f0 0x7f9bf8196020 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9be8006010 tx=0x7f9be800bd70 comp rx=0 tx=0).stop 2024-02-19T19:21:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.415+0000 7f9bfcbda700 1 -- 172.21.15.88:0/3021982868 shutdown_connections 2024-02-19T19:21:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.415+0000 7f9bfcbda700 1 --2- 172.21.15.88:0/3021982868 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9be006d8a0 0x7f9be006fd50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.415+0000 7f9bfcbda700 1 --2- 172.21.15.88:0/3021982868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9bf80fe6f0 0x7f9bf8196020 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.415+0000 7f9bfcbda700 1 --2- 172.21.15.88:0/3021982868 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9bf80ff090 0x7f9bf8196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.415+0000 7f9bfcbda700 1 -- 172.21.15.88:0/3021982868 >> 172.21.15.88:0/3021982868 conn(0x7f9bf8075440 msgr2=0x7f9bf8109930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.415+0000 7f9bfcbda700 1 -- 172.21.15.88:0/3021982868 shutdown_connections 2024-02-19T19:21:31.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.415+0000 7f9bfcbda700 1 -- 172.21.15.88:0/3021982868 wait complete. 2024-02-19T19:21:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.518+0000 7f492c75e700 1 -- 172.21.15.88:0/2385302730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49240ffd60 msgr2=0x7f4924103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.518+0000 7f492c75e700 1 --2- 172.21.15.88:0/2385302730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49240ffd60 0x7f4924103d30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4918009910 tx=0x7f4918009c20 comp rx=0 tx=0).stop 2024-02-19T19:21:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.518+0000 7f492c75e700 1 -- 172.21.15.88:0/2385302730 shutdown_connections 2024-02-19T19:21:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.518+0000 7f492c75e700 1 --2- 172.21.15.88:0/2385302730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49240ffd60 0x7f4924103d30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.518+0000 7f492c75e700 1 --2- 172.21.15.88:0/2385302730 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49240ff3c0 0x7f49240ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.518+0000 7f492c75e700 1 -- 172.21.15.88:0/2385302730 >> 172.21.15.88:0/2385302730 conn(0x7f49240fac80 msgr2=0x7f49240fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.518+0000 7f492c75e700 1 -- 172.21.15.88:0/2385302730 shutdown_connections 2024-02-19T19:21:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.518+0000 7f492c75e700 1 -- 172.21.15.88:0/2385302730 wait complete. 2024-02-19T19:21:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492c75e700 1 Processor -- start 2024-02-19T19:21:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492c75e700 1 -- start start 2024-02-19T19:21:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492c75e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49240ff3c0 0x7f49241981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492c75e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49240ffd60 0x7f4924198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492c75e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4924198e10 con 0x7f49240ff3c0 2024-02-19T19:21:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492c75e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492419cba0 con 0x7f49240ffd60 2024-02-19T19:21:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492a4fa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49240ff3c0 0x7f49241981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492a4fa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49240ff3c0 0x7f49241981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33398/0 (socket says 172.21.15.88:33398) 2024-02-19T19:21:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492a4fa700 1 -- 172.21.15.88:0/2745105003 learned_addr learned my addr 172.21.15.88:0/2745105003 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:21:31.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f4929cf9700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49240ffd60 0x7f4924198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:31.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492a4fa700 1 -- 172.21.15.88:0/2745105003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49240ffd60 msgr2=0x7f4924198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492a4fa700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49240ffd60 0x7f4924198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f492a4fa700 1 -- 172.21.15.88:0/2745105003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49180095f0 con 0x7f49240ff3c0 2024-02-19T19:21:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.519+0000 7f4929cf9700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49240ffd60 0x7f4924198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:21:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.520+0000 7f492a4fa700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49240ff3c0 0x7f49241981f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f491c00b4b0 tx=0x7f491c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.520+0000 7f49237fe700 1 -- 172.21.15.88:0/2745105003 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f491c011800 con 0x7f49240ff3c0 2024-02-19T19:21:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.520+0000 7f49237fe700 1 -- 172.21.15.88:0/2745105003 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f491c011e20 con 0x7f49240ff3c0 2024-02-19T19:21:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.520+0000 7f492c75e700 1 -- 172.21.15.88:0/2745105003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492419ce80 con 0x7f49240ff3c0 2024-02-19T19:21:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.520+0000 7f49237fe700 1 -- 172.21.15.88:0/2745105003 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f491c00f2e0 con 0x7f49240ff3c0 2024-02-19T19:21:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.520+0000 7f492c75e700 1 -- 172.21.15.88:0/2745105003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492419d3d0 con 0x7f49240ff3c0 2024-02-19T19:21:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.521+0000 7f492c75e700 1 -- 172.21.15.88:0/2745105003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492404f2a0 con 0x7f49240ff3c0 2024-02-19T19:21:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.522+0000 7f49237fe700 1 -- 172.21.15.88:0/2745105003 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f491c010380 con 0x7f49240ff3c0 2024-02-19T19:21:31.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.522+0000 7f49237fe700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490c06d8a0 0x7f490c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:31.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.522+0000 7f49237fe700 1 -- 172.21.15.88:0/2745105003 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f491c08d0f0 con 0x7f49240ff3c0 2024-02-19T19:21:31.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.522+0000 7f4929cf9700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490c06d8a0 0x7f490c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:31.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.523+0000 7f4929cf9700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490c06d8a0 0x7f490c06fd50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f491800b5c0 tx=0x7f4918005480 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.527+0000 7f49237fe700 1 -- 172.21.15.88:0/2745105003 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f491c0568a0 con 0x7f49240ff3c0 2024-02-19T19:21:31.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.806+0000 7f492c75e700 1 -- 172.21.15.88:0/2745105003 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f492419d650 con 0x7f49240ff3c0 2024-02-19T19:21:31.817 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:31 smithi088 ceph-mon[88643]: pgmap v537: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:31.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.806+0000 7f49237fe700 1 -- 172.21.15.88:0/2745105003 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f491c056020 con 0x7f49240ff3c0 2024-02-19T19:21:31.817 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:21:31.817 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:21:31.817 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:21:31.817 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:21:31.817 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:21:31.817 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:21:31.818 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:21:31.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.809+0000 7f492c75e700 1 -- 172.21.15.88:0/2745105003 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490c06d8a0 msgr2=0x7f490c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.809+0000 7f492c75e700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490c06d8a0 0x7f490c06fd50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f491800b5c0 tx=0x7f4918005480 comp rx=0 tx=0).stop 2024-02-19T19:21:31.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.809+0000 7f492c75e700 1 -- 172.21.15.88:0/2745105003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49240ff3c0 msgr2=0x7f49241981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.809+0000 7f492c75e700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49240ff3c0 0x7f49241981f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f491c00b4b0 tx=0x7f491c00b870 comp rx=0 tx=0).stop 2024-02-19T19:21:31.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.809+0000 7f492c75e700 1 -- 172.21.15.88:0/2745105003 shutdown_connections 2024-02-19T19:21:31.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.809+0000 7f492c75e700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490c06d8a0 0x7f490c06fd50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.809+0000 7f492c75e700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49240ff3c0 0x7f49241981f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.809+0000 7f492c75e700 1 --2- 172.21.15.88:0/2745105003 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49240ffd60 0x7f4924198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.809+0000 7f492c75e700 1 -- 172.21.15.88:0/2745105003 >> 172.21.15.88:0/2745105003 conn(0x7f49240fac80 msgr2=0x7f4924069660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:31.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.809+0000 7f492c75e700 1 -- 172.21.15.88:0/2745105003 shutdown_connections 2024-02-19T19:21:31.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.809+0000 7f492c75e700 1 -- 172.21.15.88:0/2745105003 wait complete. 2024-02-19T19:21:31.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.912+0000 7ff2778f2700 1 -- 172.21.15.88:0/1809499151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2700ff350 msgr2=0x7ff2700ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.912+0000 7ff2778f2700 1 --2- 172.21.15.88:0/1809499151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2700ff350 0x7ff2700ff720 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff260009910 tx=0x7ff260009c20 comp rx=0 tx=0).stop 2024-02-19T19:21:31.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.912+0000 7ff2778f2700 1 -- 172.21.15.88:0/1809499151 shutdown_connections 2024-02-19T19:21:31.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.912+0000 7ff2778f2700 1 --2- 172.21.15.88:0/1809499151 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2700ffc60 0x7ff27010a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.912+0000 7ff2778f2700 1 --2- 172.21.15.88:0/1809499151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2700ff350 0x7ff2700ff720 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.912+0000 7ff2778f2700 1 -- 172.21.15.88:0/1809499151 >> 172.21.15.88:0/1809499151 conn(0x7ff2700748b0 msgr2=0x7ff270074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:31.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.912+0000 7ff2778f2700 1 -- 172.21.15.88:0/1809499151 shutdown_connections 2024-02-19T19:21:31.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.912+0000 7ff2778f2700 1 -- 172.21.15.88:0/1809499151 wait complete. 2024-02-19T19:21:31.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.913+0000 7ff2778f2700 1 Processor -- start 2024-02-19T19:21:31.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.913+0000 7ff2778f2700 1 -- start start 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.913+0000 7ff2778f2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2700ff350 0x7ff270195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.913+0000 7ff2778f2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2700ffc60 0x7ff270196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.913+0000 7ff2778f2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff270196af0 con 0x7ff2700ffc60 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.913+0000 7ff2778f2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27019a880 con 0x7ff2700ff350 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.913+0000 7ff274e8d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2700ffc60 0x7ff270196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.913+0000 7ff274e8d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2700ffc60 0x7ff270196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33426/0 (socket says 172.21.15.88:33426) 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.913+0000 7ff274e8d700 1 -- 172.21.15.88:0/3155431192 learned_addr learned my addr 172.21.15.88:0/3155431192 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.913+0000 7ff27568e700 1 --2- 172.21.15.88:0/3155431192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2700ff350 0x7ff270195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.914+0000 7ff274e8d700 1 -- 172.21.15.88:0/3155431192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2700ff350 msgr2=0x7ff270195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.914+0000 7ff274e8d700 1 --2- 172.21.15.88:0/3155431192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2700ff350 0x7ff270195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.914+0000 7ff274e8d700 1 -- 172.21.15.88:0/3155431192 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2600095f0 con 0x7ff2700ffc60 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.914+0000 7ff274e8d700 1 --2- 172.21.15.88:0/3155431192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2700ffc60 0x7ff270196410 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00d900 tx=0x7ff26c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.914+0000 7ff2667fc700 1 -- 172.21.15.88:0/3155431192 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff26c004a00 con 0x7ff2700ffc60 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.914+0000 7ff2667fc700 1 -- 172.21.15.88:0/3155431192 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff26c0054e0 con 0x7ff2700ffc60 2024-02-19T19:21:31.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.914+0000 7ff2778f2700 1 -- 172.21.15.88:0/3155431192 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27019ab60 con 0x7ff2700ffc60 2024-02-19T19:21:31.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.914+0000 7ff2667fc700 1 -- 172.21.15.88:0/3155431192 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff26c009d00 con 0x7ff2700ffc60 2024-02-19T19:21:31.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.914+0000 7ff2778f2700 1 -- 172.21.15.88:0/3155431192 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27019b0b0 con 0x7ff2700ffc60 2024-02-19T19:21:31.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.915+0000 7ff2778f2700 1 -- 172.21.15.88:0/3155431192 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff270107dd0 con 0x7ff2700ffc60 2024-02-19T19:21:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.916+0000 7ff2667fc700 1 -- 172.21.15.88:0/3155431192 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff26c005020 con 0x7ff2700ffc60 2024-02-19T19:21:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.916+0000 7ff2667fc700 1 --2- 172.21.15.88:0/3155431192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff25c06d8a0 0x7ff25c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.916+0000 7ff2667fc700 1 -- 172.21.15.88:0/3155431192 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff26c08d3f0 con 0x7ff2700ffc60 2024-02-19T19:21:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.916+0000 7ff27568e700 1 --2- 172.21.15.88:0/3155431192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff25c06d8a0 0x7ff25c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.917+0000 7ff27568e700 1 --2- 172.21.15.88:0/3155431192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff25c06d8a0 0x7ff25c06fd50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff260006010 tx=0x7ff26000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:31.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:31.921+0000 7ff2667fc700 1 -- 172.21.15.88:0/3155431192 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff26c056c20 con 0x7ff2700ffc60 2024-02-19T19:21:31.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:31 smithi132 ceph-mon[70676]: pgmap v537: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:32.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.161+0000 7ff2778f2700 1 -- 172.21.15.88:0/3155431192 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff27004f2a0 con 0x7ff2700ffc60 2024-02-19T19:21:32.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.162+0000 7ff2667fc700 1 -- 172.21.15.88:0/3155431192 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7ff26c017740 con 0x7ff2700ffc60 2024-02-19T19:21:32.172 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:21:32.173 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:21:32.173 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:21:32.173 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:21:32.173 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:21:32.173 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:21:32.174 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:21:32.174 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:21:32.174 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:21:32.174 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:21:32.174 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:21:32.174 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:21:32.175 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:21:32.175 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:21:32.175 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:21:32.175 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:21:32.175 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:21:32.176 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:21:32.176 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:21:32.176 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:21:32.176 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:21:32.176 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:21:32.176 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:21:32.177 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:21:32.177 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:21:32.177 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:21:32.177 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:21:32.177 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:21:32.177 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:21:32.178 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:21:32.178 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:21:32.178 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:21:32.178 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:21:32.178 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:21:32.178 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:21:32.179 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:21:32.179 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:21:32.179 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:21:32.179 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:21:32.179 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:21:32.179 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:21:32.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.164+0000 7ff2778f2700 1 -- 172.21.15.88:0/3155431192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff25c06d8a0 msgr2=0x7ff25c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:32.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.164+0000 7ff2778f2700 1 --2- 172.21.15.88:0/3155431192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff25c06d8a0 0x7ff25c06fd50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff260006010 tx=0x7ff26000b540 comp rx=0 tx=0).stop 2024-02-19T19:21:32.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.164+0000 7ff2778f2700 1 -- 172.21.15.88:0/3155431192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2700ffc60 msgr2=0x7ff270196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:32.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.164+0000 7ff2778f2700 1 --2- 172.21.15.88:0/3155431192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2700ffc60 0x7ff270196410 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff26c00d900 tx=0x7ff26c00dc10 comp rx=0 tx=0).stop 2024-02-19T19:21:32.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.164+0000 7ff2778f2700 1 -- 172.21.15.88:0/3155431192 shutdown_connections 2024-02-19T19:21:32.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.164+0000 7ff2778f2700 1 --2- 172.21.15.88:0/3155431192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff25c06d8a0 0x7ff25c06fd50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.164+0000 7ff2778f2700 1 --2- 172.21.15.88:0/3155431192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2700ff350 0x7ff270195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.164+0000 7ff2778f2700 1 --2- 172.21.15.88:0/3155431192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2700ffc60 0x7ff270196410 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.164+0000 7ff2778f2700 1 -- 172.21.15.88:0/3155431192 >> 172.21.15.88:0/3155431192 conn(0x7ff2700748b0 msgr2=0x7ff270104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:32.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.164+0000 7ff2778f2700 1 -- 172.21.15.88:0/3155431192 shutdown_connections 2024-02-19T19:21:32.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.164+0000 7ff2778f2700 1 -- 172.21.15.88:0/3155431192 wait complete. 2024-02-19T19:21:32.183 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:21:32.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.270+0000 7f43dc7b4700 1 -- 172.21.15.88:0/4183059606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d40ff210 msgr2=0x7f43d40ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:32.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.270+0000 7f43dc7b4700 1 --2- 172.21.15.88:0/4183059606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d40ff210 0x7f43d40ff5e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009910 tx=0x7f43d0009c20 comp rx=0 tx=0).stop 2024-02-19T19:21:32.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.270+0000 7f43dc7b4700 1 -- 172.21.15.88:0/4183059606 shutdown_connections 2024-02-19T19:21:32.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.270+0000 7f43dc7b4700 1 --2- 172.21.15.88:0/4183059606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d40ffb20 0x7f43d4104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.270+0000 7f43dc7b4700 1 --2- 172.21.15.88:0/4183059606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d40ff210 0x7f43d40ff5e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.270+0000 7f43dc7b4700 1 -- 172.21.15.88:0/4183059606 >> 172.21.15.88:0/4183059606 conn(0x7f43d40fac80 msgr2=0x7f43d40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:32.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.270+0000 7f43dc7b4700 1 -- 172.21.15.88:0/4183059606 shutdown_connections 2024-02-19T19:21:32.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.270+0000 7f43dc7b4700 1 -- 172.21.15.88:0/4183059606 wait complete. 2024-02-19T19:21:32.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.270+0000 7f43dc7b4700 1 Processor -- start 2024-02-19T19:21:32.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.270+0000 7f43dc7b4700 1 -- start start 2024-02-19T19:21:32.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43dc7b4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d40ff210 0x7f43d419c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:32.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43dc7b4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d40ffb20 0x7f43d419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:32.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43dc7b4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d419d190 con 0x7f43d40ff210 2024-02-19T19:21:32.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43dc7b4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d41a0f20 con 0x7f43d40ffb20 2024-02-19T19:21:32.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43da550700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d40ff210 0x7f43d419c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43da550700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d40ff210 0x7f43d419c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33446/0 (socket says 172.21.15.88:33446) 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43da550700 1 -- 172.21.15.88:0/39897927 learned_addr learned my addr 172.21.15.88:0/39897927 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43d9d4f700 1 --2- 172.21.15.88:0/39897927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d40ffb20 0x7f43d419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43da550700 1 -- 172.21.15.88:0/39897927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d40ffb20 msgr2=0x7f43d419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43da550700 1 --2- 172.21.15.88:0/39897927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d40ffb20 0x7f43d419cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43da550700 1 -- 172.21.15.88:0/39897927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d00095f0 con 0x7f43d40ff210 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43da550700 1 --2- 172.21.15.88:0/39897927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d40ff210 0x7f43d419c570 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f43d0006010 tx=0x7f43d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43cb7fe700 1 -- 172.21.15.88:0/39897927 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43d001d070 con 0x7f43d40ff210 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43cb7fe700 1 -- 172.21.15.88:0/39897927 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43d0004fb0 con 0x7f43d40ff210 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43dc7b4700 1 -- 172.21.15.88:0/39897927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d41a11a0 con 0x7f43d40ff210 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43cb7fe700 1 -- 172.21.15.88:0/39897927 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43d0022ea0 con 0x7f43d40ff210 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.271+0000 7f43dc7b4700 1 -- 172.21.15.88:0/39897927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d41a1690 con 0x7f43d40ff210 2024-02-19T19:21:32.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.272+0000 7f43dc7b4700 1 -- 172.21.15.88:0/39897927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d41016f0 con 0x7f43d40ff210 2024-02-19T19:21:32.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.273+0000 7f43cb7fe700 1 -- 172.21.15.88:0/39897927 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f43d0022470 con 0x7f43d40ff210 2024-02-19T19:21:32.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.273+0000 7f43cb7fe700 1 --2- 172.21.15.88:0/39897927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43c006d8a0 0x7f43c006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:32.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.273+0000 7f43cb7fe700 1 -- 172.21.15.88:0/39897927 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f43d008ec30 con 0x7f43d40ff210 2024-02-19T19:21:32.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.273+0000 7f43d9d4f700 1 --2- 172.21.15.88:0/39897927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43c006d8a0 0x7f43c006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:32.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.274+0000 7f43d9d4f700 1 --2- 172.21.15.88:0/39897927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43c006d8a0 0x7f43c006fd50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f43c4006fd0 tx=0x7f43c4008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:32.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.278+0000 7f43cb7fe700 1 -- 172.21.15.88:0/39897927 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f43d0058630 con 0x7f43d40ff210 2024-02-19T19:21:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.495+0000 7f43dc7b4700 1 -- 172.21.15.88:0/39897927 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f43d402cca0 con 0x7f43c006d8a0 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.496+0000 7f43cb7fe700 1 -- 172.21.15.88:0/39897927 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f43d402cca0 con 0x7f43c006d8a0 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:21:32.506 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:21:32.507 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:21:32.507 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:21:32.507 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:21:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.498+0000 7f43dc7b4700 1 -- 172.21.15.88:0/39897927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43c006d8a0 msgr2=0x7f43c006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.498+0000 7f43dc7b4700 1 --2- 172.21.15.88:0/39897927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43c006d8a0 0x7f43c006fd50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f43c4006fd0 tx=0x7f43c4008040 comp rx=0 tx=0).stop 2024-02-19T19:21:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.498+0000 7f43dc7b4700 1 -- 172.21.15.88:0/39897927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d40ff210 msgr2=0x7f43d419c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.498+0000 7f43dc7b4700 1 --2- 172.21.15.88:0/39897927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d40ff210 0x7f43d419c570 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f43d0006010 tx=0x7f43d000be20 comp rx=0 tx=0).stop 2024-02-19T19:21:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.498+0000 7f43dc7b4700 1 -- 172.21.15.88:0/39897927 shutdown_connections 2024-02-19T19:21:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.498+0000 7f43dc7b4700 1 --2- 172.21.15.88:0/39897927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f43c006d8a0 0x7f43c006fd50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.498+0000 7f43dc7b4700 1 --2- 172.21.15.88:0/39897927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43d40ff210 0x7f43d419c570 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.498+0000 7f43dc7b4700 1 --2- 172.21.15.88:0/39897927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43d40ffb20 0x7f43d419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.498+0000 7f43dc7b4700 1 -- 172.21.15.88:0/39897927 >> 172.21.15.88:0/39897927 conn(0x7f43d40fac80 msgr2=0x7f43d4110680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:32.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.498+0000 7f43dc7b4700 1 -- 172.21.15.88:0/39897927 shutdown_connections 2024-02-19T19:21:32.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.498+0000 7f43dc7b4700 1 -- 172.21.15.88:0/39897927 wait complete. 2024-02-19T19:21:32.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.608+0000 7f32b045d700 1 -- 172.21.15.88:0/3165792460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32a8100e80 msgr2=0x7f32a8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:32.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.608+0000 7f32b045d700 1 --2- 172.21.15.88:0/3165792460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32a8100e80 0x7f32a8104e50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3298009910 tx=0x7f3298009c20 comp rx=0 tx=0).stop 2024-02-19T19:21:32.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.608+0000 7f32b045d700 1 -- 172.21.15.88:0/3165792460 shutdown_connections 2024-02-19T19:21:32.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32b045d700 1 --2- 172.21.15.88:0/3165792460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32a8100e80 0x7f32a8104e50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32b045d700 1 --2- 172.21.15.88:0/3165792460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a81004e0 0x7f32a81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32b045d700 1 -- 172.21.15.88:0/3165792460 >> 172.21.15.88:0/3165792460 conn(0x7f32a80fbda0 msgr2=0x7f32a80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:32.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32b045d700 1 -- 172.21.15.88:0/3165792460 shutdown_connections 2024-02-19T19:21:32.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32b045d700 1 -- 172.21.15.88:0/3165792460 wait complete. 2024-02-19T19:21:32.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32b045d700 1 Processor -- start 2024-02-19T19:21:32.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32b045d700 1 -- start start 2024-02-19T19:21:32.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32b045d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a81004e0 0x7f32a8196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:32.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32b045d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32a8100e80 0x7f32a8196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:32.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32b045d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a8196c20 con 0x7f32a8100e80 2024-02-19T19:21:32.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32b045d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a819a9b0 con 0x7f32a81004e0 2024-02-19T19:21:32.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.609+0000 7f32ad9f8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32a8100e80 0x7f32a8196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:32.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f32ad9f8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32a8100e80 0x7f32a8196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33474/0 (socket says 172.21.15.88:33474) 2024-02-19T19:21:32.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f32ad9f8700 1 -- 172.21.15.88:0/396970127 learned_addr learned my addr 172.21.15.88:0/396970127 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:21:32.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f32ae1f9700 1 --2- 172.21.15.88:0/396970127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a81004e0 0x7f32a8196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:32.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f32ad9f8700 1 -- 172.21.15.88:0/396970127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a81004e0 msgr2=0x7f32a8196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:32.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f32ad9f8700 1 --2- 172.21.15.88:0/396970127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a81004e0 0x7f32a8196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f32ad9f8700 1 -- 172.21.15.88:0/396970127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32980095f0 con 0x7f32a8100e80 2024-02-19T19:21:32.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f32ad9f8700 1 --2- 172.21.15.88:0/396970127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32a8100e80 0x7f32a8196540 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3298005f50 tx=0x7f329800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:32.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f329f7fe700 1 -- 172.21.15.88:0/396970127 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f329801d070 con 0x7f32a8100e80 2024-02-19T19:21:32.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f32b045d700 1 -- 172.21.15.88:0/396970127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a819ac30 con 0x7f32a8100e80 2024-02-19T19:21:32.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f329f7fe700 1 -- 172.21.15.88:0/396970127 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f329800f460 con 0x7f32a8100e80 2024-02-19T19:21:32.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f329f7fe700 1 -- 172.21.15.88:0/396970127 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3298017630 con 0x7f32a8100e80 2024-02-19T19:21:32.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.610+0000 7f32b045d700 1 -- 172.21.15.88:0/396970127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a819b120 con 0x7f32a8100e80 2024-02-19T19:21:32.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.611+0000 7f32b045d700 1 -- 172.21.15.88:0/396970127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a81087a0 con 0x7f32a8100e80 2024-02-19T19:21:32.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.612+0000 7f329f7fe700 1 -- 172.21.15.88:0/396970127 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f329800f5d0 con 0x7f32a8100e80 2024-02-19T19:21:32.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.612+0000 7f329f7fe700 1 --2- 172.21.15.88:0/396970127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f329406d8a0 0x7f329406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:21:32.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.612+0000 7f329f7fe700 1 -- 172.21.15.88:0/396970127 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f329808eb20 con 0x7f32a8100e80 2024-02-19T19:21:32.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.612+0000 7f32ae1f9700 1 --2- 172.21.15.88:0/396970127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f329406d8a0 0x7f329406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:21:32.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.612+0000 7f32ae1f9700 1 --2- 172.21.15.88:0/396970127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f329406d8a0 0x7f329406fd50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32a4006fd0 tx=0x7f32a4008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:21:32.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.617+0000 7f329f7fe700 1 -- 172.21.15.88:0/396970127 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f32980582d0 con 0x7f32a8100e80 2024-02-19T19:21:32.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:32 smithi088 ceph-mon[88643]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:32.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:32 smithi088 ceph-mon[88643]: from='client.35006 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:32.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:32 smithi088 ceph-mon[88643]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:32.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:32 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2745105003' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:21:32.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:32 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3155431192' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:21:32.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.892+0000 7f32b045d700 1 -- 172.21.15.88:0/396970127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f32a802cc30 con 0x7f32a8100e80 2024-02-19T19:21:32.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.893+0000 7f329f7fe700 1 -- 172.21.15.88:0/396970127 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f329805b990 con 0x7f32a8100e80 2024-02-19T19:21:32.903 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:21:32.903 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:21:32.903 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:21:32.903 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:21:32.903 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:21:32.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.895+0000 7f32b045d700 1 -- 172.21.15.88:0/396970127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f329406d8a0 msgr2=0x7f329406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:32.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.895+0000 7f32b045d700 1 --2- 172.21.15.88:0/396970127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f329406d8a0 0x7f329406fd50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f32a4006fd0 tx=0x7f32a4008040 comp rx=0 tx=0).stop 2024-02-19T19:21:32.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.895+0000 7f32b045d700 1 -- 172.21.15.88:0/396970127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32a8100e80 msgr2=0x7f32a8196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:21:32.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.895+0000 7f32b045d700 1 --2- 172.21.15.88:0/396970127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32a8100e80 0x7f32a8196540 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3298005f50 tx=0x7f329800be20 comp rx=0 tx=0).stop 2024-02-19T19:21:32.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.895+0000 7f32b045d700 1 -- 172.21.15.88:0/396970127 shutdown_connections 2024-02-19T19:21:32.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.895+0000 7f32b045d700 1 --2- 172.21.15.88:0/396970127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f329406d8a0 0x7f329406fd50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.895+0000 7f32b045d700 1 --2- 172.21.15.88:0/396970127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f32a81004e0 0x7f32a8196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.895+0000 7f32b045d700 1 --2- 172.21.15.88:0/396970127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32a8100e80 0x7f32a8196540 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:21:32.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.895+0000 7f32b045d700 1 -- 172.21.15.88:0/396970127 >> 172.21.15.88:0/396970127 conn(0x7f32a80fbda0 msgr2=0x7f32a80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:21:32.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.895+0000 7f32b045d700 1 -- 172.21.15.88:0/396970127 shutdown_connections 2024-02-19T19:21:32.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:21:32.895+0000 7f32b045d700 1 -- 172.21.15.88:0/396970127 wait complete. 2024-02-19T19:21:32.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:32 smithi132 ceph-mon[70676]: from='client.35002 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:32.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:32 smithi132 ceph-mon[70676]: from='client.35006 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:32.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:32 smithi132 ceph-mon[70676]: from='client.35010 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:32.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:32 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2745105003' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:21:32.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:32 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3155431192' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:21:33.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:33 smithi088 ceph-mon[88643]: pgmap v538: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:33.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:33 smithi088 ceph-mon[88643]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:33.841 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:33 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/396970127' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:21:33.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:33 smithi132 ceph-mon[70676]: pgmap v538: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:33.993 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:33 smithi132 ceph-mon[70676]: from='client.35022 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:21:33.993 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:33 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/396970127' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:21:35.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:35 smithi088 ceph-mon[88643]: pgmap v539: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:35.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:35 smithi132 ceph-mon[70676]: pgmap v539: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:37.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:37 smithi088 ceph-mon[88643]: pgmap v540: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:37.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:37 smithi132 ceph-mon[70676]: pgmap v540: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:39.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:39 smithi088 ceph-mon[88643]: pgmap v541: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:39.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:39 smithi132 ceph-mon[70676]: pgmap v541: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:41.843 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:41 smithi088 ceph-mon[88643]: pgmap v542: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:41.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:41 smithi132 ceph-mon[70676]: pgmap v542: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:43.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:43 smithi088 ceph-mon[88643]: pgmap v543: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:43.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:21:43.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:43 smithi132 ceph-mon[70676]: pgmap v543: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:43.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:21:45.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:45 smithi088 ceph-mon[88643]: pgmap v544: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:45.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:45 smithi132 ceph-mon[70676]: pgmap v544: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:47.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:47 smithi088 ceph-mon[88643]: pgmap v545: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:47.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:47 smithi132 ceph-mon[70676]: pgmap v545: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:49.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:49 smithi088 ceph-mon[88643]: pgmap v546: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:49.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:49 smithi132 ceph-mon[70676]: pgmap v546: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:51.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:51 smithi088 ceph-mon[88643]: pgmap v547: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:51.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:51 smithi132 ceph-mon[70676]: pgmap v547: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:53.742 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:53 smithi132 ceph-mon[70676]: pgmap v548: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:53.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:53 smithi088 ceph-mon[88643]: pgmap v548: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:55.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:55 smithi088 ceph-mon[88643]: pgmap v549: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:55.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:55 smithi132 ceph-mon[70676]: pgmap v549: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:57.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:57 smithi088 ceph-mon[88643]: pgmap v550: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:57.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:57 smithi132 ceph-mon[70676]: pgmap v550: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:58.742 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:21:58.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:21:59.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:21:59 smithi088 ceph-mon[88643]: pgmap v551: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:21:59.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:21:59 smithi132 ceph-mon[70676]: pgmap v551: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:01.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:01 smithi088 ceph-mon[88643]: pgmap v552: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:01.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:01 smithi132 ceph-mon[70676]: pgmap v552: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:03.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.012+0000 7f72b5086700 1 -- 172.21.15.88:0/1327282074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72b01004e0 msgr2=0x7f72b01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.012+0000 7f72b5086700 1 --2- 172.21.15.88:0/1327282074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72b01004e0 0x7f72b01008b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f72a0009910 tx=0x7f72a0009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.013+0000 7f72b5086700 1 -- 172.21.15.88:0/1327282074 shutdown_connections 2024-02-19T19:22:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.013+0000 7f72b5086700 1 --2- 172.21.15.88:0/1327282074 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b0100e80 0x7f72b0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.013+0000 7f72b5086700 1 --2- 172.21.15.88:0/1327282074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72b01004e0 0x7f72b01008b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.013+0000 7f72b5086700 1 -- 172.21.15.88:0/1327282074 >> 172.21.15.88:0/1327282074 conn(0x7f72b00fbda0 msgr2=0x7f72b00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.013+0000 7f72b5086700 1 -- 172.21.15.88:0/1327282074 shutdown_connections 2024-02-19T19:22:03.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.013+0000 7f72b5086700 1 -- 172.21.15.88:0/1327282074 wait complete. 2024-02-19T19:22:03.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.013+0000 7f72b5086700 1 Processor -- start 2024-02-19T19:22:03.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72b5086700 1 -- start start 2024-02-19T19:22:03.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72b5086700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b01004e0 0x7f72b0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:03.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72b5086700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72b0100e80 0x7f72b0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:03.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72b5086700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b0073ad0 con 0x7f72b0100e80 2024-02-19T19:22:03.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72b5086700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b0073c10 con 0x7f72b01004e0 2024-02-19T19:22:03.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72ae59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72b0100e80 0x7f72b0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72ae59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72b0100e80 0x7f72b0073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48436/0 (socket says 172.21.15.88:48436) 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72ae59c700 1 -- 172.21.15.88:0/3510770709 learned_addr learned my addr 172.21.15.88:0/3510770709 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72aed9d700 1 --2- 172.21.15.88:0/3510770709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b01004e0 0x7f72b0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72ae59c700 1 -- 172.21.15.88:0/3510770709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b01004e0 msgr2=0x7f72b0073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72ae59c700 1 --2- 172.21.15.88:0/3510770709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b01004e0 0x7f72b0073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72ae59c700 1 -- 172.21.15.88:0/3510770709 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a00095f0 con 0x7f72b0100e80 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.014+0000 7f72ae59c700 1 --2- 172.21.15.88:0/3510770709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72b0100e80 0x7f72b0073590 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f72a400b480 tx=0x7f72a400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.015+0000 7f7297fff700 1 -- 172.21.15.88:0/3510770709 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72a40118c0 con 0x7f72b0100e80 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.015+0000 7f72b5086700 1 -- 172.21.15.88:0/3510770709 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b01a2160 con 0x7f72b0100e80 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.015+0000 7f7297fff700 1 -- 172.21.15.88:0/3510770709 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72a4010380 con 0x7f72b0100e80 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.015+0000 7f7297fff700 1 -- 172.21.15.88:0/3510770709 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72a400f4f0 con 0x7f72b0100e80 2024-02-19T19:22:03.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.015+0000 7f72b5086700 1 -- 172.21.15.88:0/3510770709 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b01a2570 con 0x7f72b0100e80 2024-02-19T19:22:03.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.016+0000 7f72b5086700 1 -- 172.21.15.88:0/3510770709 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b01087f0 con 0x7f72b0100e80 2024-02-19T19:22:03.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.016+0000 7f7297fff700 1 -- 172.21.15.88:0/3510770709 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f72a40104f0 con 0x7f72b0100e80 2024-02-19T19:22:03.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.017+0000 7f7297fff700 1 --2- 172.21.15.88:0/3510770709 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f729806d8a0 0x7f729806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:03.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.017+0000 7f7297fff700 1 -- 172.21.15.88:0/3510770709 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f72a408d180 con 0x7f72b0100e80 2024-02-19T19:22:03.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.017+0000 7f72aed9d700 1 --2- 172.21.15.88:0/3510770709 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f729806d8a0 0x7f729806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:03.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.017+0000 7f72aed9d700 1 --2- 172.21.15.88:0/3510770709 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f729806d8a0 0x7f729806fd50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f72a000b5c0 tx=0x7f72a001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:03.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.021+0000 7f7297fff700 1 -- 172.21.15.88:0/3510770709 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f72a4056930 con 0x7f72b0100e80 2024-02-19T19:22:03.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.236+0000 7f72b5086700 1 -- 172.21.15.88:0/3510770709 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72b0061190 con 0x7f729806d8a0 2024-02-19T19:22:03.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.238+0000 7f7297fff700 1 -- 172.21.15.88:0/3510770709 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f72b0061190 con 0x7f729806d8a0 2024-02-19T19:22:03.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.240+0000 7f72b5086700 1 -- 172.21.15.88:0/3510770709 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f729806d8a0 msgr2=0x7f729806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.240+0000 7f72b5086700 1 --2- 172.21.15.88:0/3510770709 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f729806d8a0 0x7f729806fd50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f72a000b5c0 tx=0x7f72a001a040 comp rx=0 tx=0).stop 2024-02-19T19:22:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.240+0000 7f72b5086700 1 -- 172.21.15.88:0/3510770709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72b0100e80 msgr2=0x7f72b0073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.240+0000 7f72b5086700 1 --2- 172.21.15.88:0/3510770709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72b0100e80 0x7f72b0073590 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f72a400b480 tx=0x7f72a400b840 comp rx=0 tx=0).stop 2024-02-19T19:22:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.240+0000 7f72b5086700 1 -- 172.21.15.88:0/3510770709 shutdown_connections 2024-02-19T19:22:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.240+0000 7f72b5086700 1 --2- 172.21.15.88:0/3510770709 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f729806d8a0 0x7f729806fd50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.240+0000 7f72b5086700 1 --2- 172.21.15.88:0/3510770709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72b01004e0 0x7f72b0073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.240+0000 7f72b5086700 1 --2- 172.21.15.88:0/3510770709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72b0100e80 0x7f72b0073590 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.240+0000 7f72b5086700 1 -- 172.21.15.88:0/3510770709 >> 172.21.15.88:0/3510770709 conn(0x7f72b00fbda0 msgr2=0x7f72b00fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.240+0000 7f72b5086700 1 -- 172.21.15.88:0/3510770709 shutdown_connections 2024-02-19T19:22:03.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.240+0000 7f72b5086700 1 -- 172.21.15.88:0/3510770709 wait complete. 2024-02-19T19:22:03.260 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:22:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.345+0000 7f9a43c77700 1 -- 172.21.15.88:0/2416414783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a3c0ffd60 msgr2=0x7f9a3c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.345+0000 7f9a43c77700 1 --2- 172.21.15.88:0/2416414783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a3c0ffd60 0x7f9a3c103d30 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009910 tx=0x7f9a2c009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.345+0000 7f9a43c77700 1 -- 172.21.15.88:0/2416414783 shutdown_connections 2024-02-19T19:22:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.345+0000 7f9a43c77700 1 --2- 172.21.15.88:0/2416414783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a3c0ffd60 0x7f9a3c103d30 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.345+0000 7f9a43c77700 1 --2- 172.21.15.88:0/2416414783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a3c0ff3c0 0x7f9a3c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.345+0000 7f9a43c77700 1 -- 172.21.15.88:0/2416414783 >> 172.21.15.88:0/2416414783 conn(0x7f9a3c0fac80 msgr2=0x7f9a3c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:03.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.345+0000 7f9a43c77700 1 -- 172.21.15.88:0/2416414783 shutdown_connections 2024-02-19T19:22:03.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.345+0000 7f9a43c77700 1 -- 172.21.15.88:0/2416414783 wait complete. 2024-02-19T19:22:03.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.346+0000 7f9a43c77700 1 Processor -- start 2024-02-19T19:22:03.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.346+0000 7f9a43c77700 1 -- start start 2024-02-19T19:22:03.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.346+0000 7f9a43c77700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a3c0ff3c0 0x7f9a3c195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:03.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.346+0000 7f9a43c77700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a3c0ffd60 0x7f9a3c1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:03.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.346+0000 7f9a43c77700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c196a20 con 0x7f9a3c0ff3c0 2024-02-19T19:22:03.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.346+0000 7f9a43c77700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3c196b60 con 0x7f9a3c0ffd60 2024-02-19T19:22:03.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.346+0000 7f9a41a13700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a3c0ff3c0 0x7f9a3c195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:03.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.346+0000 7f9a41a13700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a3c0ff3c0 0x7f9a3c195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48456/0 (socket says 172.21.15.88:48456) 2024-02-19T19:22:03.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.346+0000 7f9a41a13700 1 -- 172.21.15.88:0/254903032 learned_addr learned my addr 172.21.15.88:0/254903032 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.347+0000 7f9a41212700 1 --2- 172.21.15.88:0/254903032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a3c0ffd60 0x7f9a3c1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.347+0000 7f9a41a13700 1 -- 172.21.15.88:0/254903032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a3c0ffd60 msgr2=0x7f9a3c1963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.347+0000 7f9a41a13700 1 --2- 172.21.15.88:0/254903032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a3c0ffd60 0x7f9a3c1963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.347+0000 7f9a41a13700 1 -- 172.21.15.88:0/254903032 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2c0095f0 con 0x7f9a3c0ff3c0 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.347+0000 7f9a41a13700 1 --2- 172.21.15.88:0/254903032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a3c0ff3c0 0x7f9a3c195e90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800c930 tx=0x7f9a3800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.347+0000 7f9a32ffd700 1 -- 172.21.15.88:0/254903032 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a38007ad0 con 0x7f9a3c0ff3c0 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.347+0000 7f9a32ffd700 1 -- 172.21.15.88:0/254903032 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a3800ce80 con 0x7f9a3c0ff3c0 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.347+0000 7f9a43c77700 1 -- 172.21.15.88:0/254903032 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a3c19a9b0 con 0x7f9a3c0ff3c0 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.347+0000 7f9a32ffd700 1 -- 172.21.15.88:0/254903032 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a38018630 con 0x7f9a3c0ff3c0 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.347+0000 7f9a43c77700 1 -- 172.21.15.88:0/254903032 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a3c19af00 con 0x7f9a3c0ff3c0 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.348+0000 7f9a43c77700 1 -- 172.21.15.88:0/254903032 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3c108740 con 0x7f9a3c0ff3c0 2024-02-19T19:22:03.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.349+0000 7f9a32ffd700 1 -- 172.21.15.88:0/254903032 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9a38007c30 con 0x7f9a3c0ff3c0 2024-02-19T19:22:03.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.349+0000 7f9a32ffd700 1 --2- 172.21.15.88:0/254903032 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9a2806d8a0 0x7f9a2806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:03.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.349+0000 7f9a41212700 1 --2- 172.21.15.88:0/254903032 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9a2806d8a0 0x7f9a2806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:03.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.349+0000 7f9a32ffd700 1 -- 172.21.15.88:0/254903032 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9a3808d4d0 con 0x7f9a3c0ff3c0 2024-02-19T19:22:03.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.350+0000 7f9a41212700 1 --2- 172.21.15.88:0/254903032 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9a2806d8a0 0x7f9a2806fd50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c1974b0 tx=0x7f9a2c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:03.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.354+0000 7f9a32ffd700 1 -- 172.21.15.88:0/254903032 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9a38056c80 con 0x7f9a3c0ff3c0 2024-02-19T19:22:03.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.570+0000 7f9a43c77700 1 -- 172.21.15.88:0/254903032 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a3c19b1e0 con 0x7f9a2806d8a0 2024-02-19T19:22:03.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.571+0000 7f9a32ffd700 1 -- 172.21.15.88:0/254903032 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9a3c19b1e0 con 0x7f9a2806d8a0 2024-02-19T19:22:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.573+0000 7f9a43c77700 1 -- 172.21.15.88:0/254903032 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9a2806d8a0 msgr2=0x7f9a2806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.573+0000 7f9a43c77700 1 --2- 172.21.15.88:0/254903032 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9a2806d8a0 0x7f9a2806fd50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c1974b0 tx=0x7f9a2c01a040 comp rx=0 tx=0).stop 2024-02-19T19:22:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.573+0000 7f9a43c77700 1 -- 172.21.15.88:0/254903032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a3c0ff3c0 msgr2=0x7f9a3c195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.573+0000 7f9a43c77700 1 --2- 172.21.15.88:0/254903032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a3c0ff3c0 0x7f9a3c195e90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800c930 tx=0x7f9a3800cc40 comp rx=0 tx=0).stop 2024-02-19T19:22:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.573+0000 7f9a43c77700 1 -- 172.21.15.88:0/254903032 shutdown_connections 2024-02-19T19:22:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.573+0000 7f9a43c77700 1 --2- 172.21.15.88:0/254903032 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9a2806d8a0 0x7f9a2806fd50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.573+0000 7f9a43c77700 1 --2- 172.21.15.88:0/254903032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a3c0ff3c0 0x7f9a3c195e90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.573+0000 7f9a43c77700 1 --2- 172.21.15.88:0/254903032 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a3c0ffd60 0x7f9a3c1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.573+0000 7f9a43c77700 1 -- 172.21.15.88:0/254903032 >> 172.21.15.88:0/254903032 conn(0x7f9a3c0fac80 msgr2=0x7f9a3c0690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.574+0000 7f9a43c77700 1 -- 172.21.15.88:0/254903032 shutdown_connections 2024-02-19T19:22:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.574+0000 7f9a43c77700 1 -- 172.21.15.88:0/254903032 wait complete. 2024-02-19T19:22:03.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.679+0000 7f5961274700 1 -- 172.21.15.88:0/33255385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f595c105230 msgr2=0x7f595c105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.679+0000 7f5961274700 1 --2- 172.21.15.88:0/33255385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f595c105230 0x7f595c105600 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f594c009910 tx=0x7f594c009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.679+0000 7f5961274700 1 -- 172.21.15.88:0/33255385 shutdown_connections 2024-02-19T19:22:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.679+0000 7f5961274700 1 --2- 172.21.15.88:0/33255385 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c068540 0x7f595c0689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.679+0000 7f5961274700 1 --2- 172.21.15.88:0/33255385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f595c105230 0x7f595c105600 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.679+0000 7f5961274700 1 -- 172.21.15.88:0/33255385 >> 172.21.15.88:0/33255385 conn(0x7f595c075130 msgr2=0x7f595c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.679+0000 7f5961274700 1 -- 172.21.15.88:0/33255385 shutdown_connections 2024-02-19T19:22:03.689 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:03 smithi088 ceph-mon[88643]: pgmap v553: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.679+0000 7f5961274700 1 -- 172.21.15.88:0/33255385 wait complete. 2024-02-19T19:22:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f5961274700 1 Processor -- start 2024-02-19T19:22:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f5961274700 1 -- start start 2024-02-19T19:22:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f5961274700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f595c068540 0x7f595c195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f5961274700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c105230 0x7f595c196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f5961274700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c196ae0 con 0x7f595c068540 2024-02-19T19:22:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f5961274700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c19a870 con 0x7f595c105230 2024-02-19T19:22:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f595affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f595c068540 0x7f595c195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f595affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f595c068540 0x7f595c195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48480/0 (socket says 172.21.15.88:48480) 2024-02-19T19:22:03.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f595affd700 1 -- 172.21.15.88:0/2452511267 learned_addr learned my addr 172.21.15.88:0/2452511267 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f595a7fc700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c105230 0x7f595c196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f595affd700 1 -- 172.21.15.88:0/2452511267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c105230 msgr2=0x7f595c196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f595affd700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c105230 0x7f595c196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f595affd700 1 -- 172.21.15.88:0/2452511267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c0095f0 con 0x7f595c068540 2024-02-19T19:22:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.680+0000 7f595a7fc700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c105230 0x7f595c196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:22:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.681+0000 7f595affd700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f595c068540 0x7f595c195ec0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f594c000c00 tx=0x7f594c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.681+0000 7f5943fff700 1 -- 172.21.15.88:0/2452511267 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594c01d070 con 0x7f595c068540 2024-02-19T19:22:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.681+0000 7f5961274700 1 -- 172.21.15.88:0/2452511267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c19aaf0 con 0x7f595c068540 2024-02-19T19:22:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.681+0000 7f5943fff700 1 -- 172.21.15.88:0/2452511267 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f594c00f460 con 0x7f595c068540 2024-02-19T19:22:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.681+0000 7f5943fff700 1 -- 172.21.15.88:0/2452511267 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594c017630 con 0x7f595c068540 2024-02-19T19:22:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.681+0000 7f5961274700 1 -- 172.21.15.88:0/2452511267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c19afe0 con 0x7f595c068540 2024-02-19T19:22:03.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.682+0000 7f5961274700 1 -- 172.21.15.88:0/2452511267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595c1087c0 con 0x7f595c068540 2024-02-19T19:22:03.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.682+0000 7f5943fff700 1 -- 172.21.15.88:0/2452511267 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f594c00f5d0 con 0x7f595c068540 2024-02-19T19:22:03.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.683+0000 7f5943fff700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f594406d8a0 0x7f594406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:03.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.683+0000 7f5943fff700 1 -- 172.21.15.88:0/2452511267 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f594c08eb90 con 0x7f595c068540 2024-02-19T19:22:03.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.683+0000 7f595a7fc700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f594406d8a0 0x7f594406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:03.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.683+0000 7f595a7fc700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f594406d8a0 0x7f594406fd50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f595c1974e0 tx=0x7f5950008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:03.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.687+0000 7f5943fff700 1 -- 172.21.15.88:0/2452511267 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f594c058340 con 0x7f595c068540 2024-02-19T19:22:03.742 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:03 smithi132 ceph-mon[70676]: pgmap v553: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:03.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.893+0000 7f5961274700 1 -- 172.21.15.88:0/2452511267 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f595c19b200 con 0x7f594406d8a0 2024-02-19T19:22:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.904+0000 7f5943fff700 1 -- 172.21.15.88:0/2452511267 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f595c19b200 con 0x7f594406d8a0 2024-02-19T19:22:03.913 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (19m) 11m ago 29m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (30m) 11m ago 30m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (12m) 10m ago 28m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (15m) 11m ago 30m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (15m) 10m ago 28m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (18m) 11m ago 29m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (11m) 11m ago 23m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (11m) 11m ago 23m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (10m) 10m ago 23m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (23m) 10m ago 23m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (20m) 11m ago 31m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (20m) 10m ago 27m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (17m) 11m ago 31m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:22:03.914 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (16m) 10m ago 27m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:22:03.915 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (19m) 11m ago 29m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:22:03.915 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (19m) 10m ago 28m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:22:03.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (14m) 11m ago 27m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:22:03.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (14m) 11m ago 26m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:22:03.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (13m) 11m ago 26m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:22:03.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (12m) 10m ago 25m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:22:03.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (12m) 10m ago 25m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:22:03.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (11m) 10m ago 24m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:22:03.915 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (19m) 11m ago 28m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:22:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.906+0000 7f5961274700 1 -- 172.21.15.88:0/2452511267 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f594406d8a0 msgr2=0x7f594406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.906+0000 7f5961274700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f594406d8a0 0x7f594406fd50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f595c1974e0 tx=0x7f5950008040 comp rx=0 tx=0).stop 2024-02-19T19:22:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.906+0000 7f5961274700 1 -- 172.21.15.88:0/2452511267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f595c068540 msgr2=0x7f595c195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.906+0000 7f5961274700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f595c068540 0x7f595c195ec0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f594c000c00 tx=0x7f594c00be20 comp rx=0 tx=0).stop 2024-02-19T19:22:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.906+0000 7f5961274700 1 -- 172.21.15.88:0/2452511267 shutdown_connections 2024-02-19T19:22:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.906+0000 7f5961274700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f594406d8a0 0x7f594406fd50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.906+0000 7f5961274700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f595c068540 0x7f595c195ec0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.906+0000 7f5961274700 1 --2- 172.21.15.88:0/2452511267 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f595c105230 0x7f595c196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.906+0000 7f5961274700 1 -- 172.21.15.88:0/2452511267 >> 172.21.15.88:0/2452511267 conn(0x7f595c075130 msgr2=0x7f595c0fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.906+0000 7f5961274700 1 -- 172.21.15.88:0/2452511267 shutdown_connections 2024-02-19T19:22:03.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:03.906+0000 7f5961274700 1 -- 172.21.15.88:0/2452511267 wait complete. 2024-02-19T19:22:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.009+0000 7f6bfc022700 1 -- 172.21.15.88:0/3582082582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf40ffbd0 msgr2=0x7f6bf4100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.009+0000 7f6bfc022700 1 --2- 172.21.15.88:0/3582082582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf40ffbd0 0x7f6bf4100040 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6be4009910 tx=0x7f6be4009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.009+0000 7f6bfc022700 1 -- 172.21.15.88:0/3582082582 shutdown_connections 2024-02-19T19:22:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.009+0000 7f6bfc022700 1 --2- 172.21.15.88:0/3582082582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf40ffbd0 0x7f6bf4100040 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.009+0000 7f6bfc022700 1 --2- 172.21.15.88:0/3582082582 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6bf4104130 0x7f6bf4104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.009+0000 7f6bfc022700 1 -- 172.21.15.88:0/3582082582 >> 172.21.15.88:0/3582082582 conn(0x7f6bf4075130 msgr2=0x7f6bf4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.009+0000 7f6bfc022700 1 -- 172.21.15.88:0/3582082582 shutdown_connections 2024-02-19T19:22:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.009+0000 7f6bfc022700 1 -- 172.21.15.88:0/3582082582 wait complete. 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.010+0000 7f6bfc022700 1 Processor -- start 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.010+0000 7f6bfc022700 1 -- start start 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.010+0000 7f6bfc022700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6bf40ffbd0 0x7f6bf4100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.010+0000 7f6bfc022700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf4104130 0x7f6bf4101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.010+0000 7f6bfc022700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf41018b0 con 0x7f6bf4104130 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.010+0000 7f6bfc022700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf41019f0 con 0x7f6bf40ffbd0 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.010+0000 7f6bf95bd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf4104130 0x7f6bf4101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.010+0000 7f6bf95bd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf4104130 0x7f6bf4101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48498/0 (socket says 172.21.15.88:48498) 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.010+0000 7f6bf95bd700 1 -- 172.21.15.88:0/455458255 learned_addr learned my addr 172.21.15.88:0/455458255 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.010+0000 7f6bf9dbe700 1 --2- 172.21.15.88:0/455458255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6bf40ffbd0 0x7f6bf4100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.010+0000 7f6bf95bd700 1 -- 172.21.15.88:0/455458255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6bf40ffbd0 msgr2=0x7f6bf4100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.011+0000 7f6bf95bd700 1 --2- 172.21.15.88:0/455458255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6bf40ffbd0 0x7f6bf4100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.011+0000 7f6bf95bd700 1 -- 172.21.15.88:0/455458255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be40095f0 con 0x7f6bf4104130 2024-02-19T19:22:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.011+0000 7f6bf95bd700 1 --2- 172.21.15.88:0/455458255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf4104130 0x7f6bf4101370 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6be400bd70 tx=0x7f6be400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.011+0000 7f6beaffd700 1 -- 172.21.15.88:0/455458255 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6be401d070 con 0x7f6bf4104130 2024-02-19T19:22:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.011+0000 7f6beaffd700 1 -- 172.21.15.88:0/455458255 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6be400f460 con 0x7f6bf4104130 2024-02-19T19:22:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.011+0000 7f6bfc022700 1 -- 172.21.15.88:0/455458255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf4101c70 con 0x7f6bf4104130 2024-02-19T19:22:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.011+0000 7f6beaffd700 1 -- 172.21.15.88:0/455458255 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6be4017630 con 0x7f6bf4104130 2024-02-19T19:22:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.011+0000 7f6bfc022700 1 -- 172.21.15.88:0/455458255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf41a2430 con 0x7f6bf4104130 2024-02-19T19:22:04.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.012+0000 7f6bfc022700 1 -- 172.21.15.88:0/455458255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf41086a0 con 0x7f6bf4104130 2024-02-19T19:22:04.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.013+0000 7f6beaffd700 1 -- 172.21.15.88:0/455458255 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6be40178f0 con 0x7f6bf4104130 2024-02-19T19:22:04.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.013+0000 7f6beaffd700 1 --2- 172.21.15.88:0/455458255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6be006d8a0 0x7f6be006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:04.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.013+0000 7f6beaffd700 1 -- 172.21.15.88:0/455458255 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6be408eb00 con 0x7f6bf4104130 2024-02-19T19:22:04.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.013+0000 7f6bf9dbe700 1 --2- 172.21.15.88:0/455458255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6be006d8a0 0x7f6be006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:04.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.013+0000 7f6bf9dbe700 1 --2- 172.21.15.88:0/455458255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6be006d8a0 0x7f6be006fd50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6bec006fd0 tx=0x7f6bec008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:04.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.019+0000 7f6beaffd700 1 -- 172.21.15.88:0/455458255 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6be40592e0 con 0x7f6bf4104130 2024-02-19T19:22:04.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.296+0000 7f6bfc022700 1 -- 172.21.15.88:0/455458255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6bf404f2a0 con 0x7f6bf4104130 2024-02-19T19:22:04.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.297+0000 7f6beaffd700 1 -- 172.21.15.88:0/455458255 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f6be400f5d0 con 0x7f6bf4104130 2024-02-19T19:22:04.306 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:22:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:22:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:22:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:22:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:22:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:22:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:22:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:22:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:22:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:22:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:22:04.307 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:22:04.308 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:22:04.308 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:22:04.308 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:22:04.308 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:22:04.308 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:22:04.308 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:22:04.308 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:22:04.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.299+0000 7f6bfc022700 1 -- 172.21.15.88:0/455458255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6be006d8a0 msgr2=0x7f6be006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.299+0000 7f6bfc022700 1 --2- 172.21.15.88:0/455458255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6be006d8a0 0x7f6be006fd50 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6bec006fd0 tx=0x7f6bec008040 comp rx=0 tx=0).stop 2024-02-19T19:22:04.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.299+0000 7f6bfc022700 1 -- 172.21.15.88:0/455458255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf4104130 msgr2=0x7f6bf4101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.299+0000 7f6bfc022700 1 --2- 172.21.15.88:0/455458255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf4104130 0x7f6bf4101370 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6be400bd70 tx=0x7f6be400be50 comp rx=0 tx=0).stop 2024-02-19T19:22:04.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.299+0000 7f6bfc022700 1 -- 172.21.15.88:0/455458255 shutdown_connections 2024-02-19T19:22:04.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.299+0000 7f6bfc022700 1 --2- 172.21.15.88:0/455458255 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6be006d8a0 0x7f6be006fd50 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.299+0000 7f6bfc022700 1 --2- 172.21.15.88:0/455458255 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6bf40ffbd0 0x7f6bf4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.299+0000 7f6bfc022700 1 --2- 172.21.15.88:0/455458255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bf4104130 0x7f6bf4101370 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.299+0000 7f6bfc022700 1 -- 172.21.15.88:0/455458255 >> 172.21.15.88:0/455458255 conn(0x7f6bf4075130 msgr2=0x7f6bf4068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.299+0000 7f6bfc022700 1 -- 172.21.15.88:0/455458255 shutdown_connections 2024-02-19T19:22:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.299+0000 7f6bfc022700 1 -- 172.21.15.88:0/455458255 wait complete. 2024-02-19T19:22:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.402+0000 7fbe79027700 1 -- 172.21.15.88:0/1706596412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe740ff350 msgr2=0x7fbe740ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.402+0000 7fbe79027700 1 --2- 172.21.15.88:0/1706596412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe740ff350 0x7fbe740ff7c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009910 tx=0x7fbe68009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.402+0000 7fbe79027700 1 -- 172.21.15.88:0/1706596412 shutdown_connections 2024-02-19T19:22:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.402+0000 7fbe79027700 1 --2- 172.21.15.88:0/1706596412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe740ff350 0x7fbe740ff7c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.402+0000 7fbe79027700 1 --2- 172.21.15.88:0/1706596412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe741038b0 0x7fbe74103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.402+0000 7fbe79027700 1 -- 172.21.15.88:0/1706596412 >> 172.21.15.88:0/1706596412 conn(0x7fbe740748b0 msgr2=0x7fbe74074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.402+0000 7fbe79027700 1 -- 172.21.15.88:0/1706596412 shutdown_connections 2024-02-19T19:22:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.402+0000 7fbe79027700 1 -- 172.21.15.88:0/1706596412 wait complete. 2024-02-19T19:22:04.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.403+0000 7fbe79027700 1 Processor -- start 2024-02-19T19:22:04.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.403+0000 7fbe79027700 1 -- start start 2024-02-19T19:22:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.403+0000 7fbe79027700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe740ff350 0x7fbe74194e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.403+0000 7fbe79027700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe741038b0 0x7fbe741953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.403+0000 7fbe79027700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe74195a90 con 0x7fbe740ff350 2024-02-19T19:22:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.403+0000 7fbe79027700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe74199820 con 0x7fbe741038b0 2024-02-19T19:22:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.403+0000 7fbe72d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe740ff350 0x7fbe74194e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.403+0000 7fbe72d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe740ff350 0x7fbe74194e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48506/0 (socket says 172.21.15.88:48506) 2024-02-19T19:22:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.403+0000 7fbe72d9d700 1 -- 172.21.15.88:0/504523421 learned_addr learned my addr 172.21.15.88:0/504523421 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.404+0000 7fbe7259c700 1 --2- 172.21.15.88:0/504523421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe741038b0 0x7fbe741953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.404+0000 7fbe72d9d700 1 -- 172.21.15.88:0/504523421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe741038b0 msgr2=0x7fbe741953b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.404+0000 7fbe72d9d700 1 --2- 172.21.15.88:0/504523421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe741038b0 0x7fbe741953b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.404+0000 7fbe72d9d700 1 -- 172.21.15.88:0/504523421 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe680095f0 con 0x7fbe740ff350 2024-02-19T19:22:04.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.404+0000 7fbe72d9d700 1 --2- 172.21.15.88:0/504523421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe740ff350 0x7fbe74194e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400b4b0 tx=0x7fbe6400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:04.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.404+0000 7fbe5bfff700 1 -- 172.21.15.88:0/504523421 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe64011800 con 0x7fbe740ff350 2024-02-19T19:22:04.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.404+0000 7fbe5bfff700 1 -- 172.21.15.88:0/504523421 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe64011e20 con 0x7fbe740ff350 2024-02-19T19:22:04.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.404+0000 7fbe79027700 1 -- 172.21.15.88:0/504523421 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe74199b00 con 0x7fbe740ff350 2024-02-19T19:22:04.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.404+0000 7fbe5bfff700 1 -- 172.21.15.88:0/504523421 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe6400f2e0 con 0x7fbe740ff350 2024-02-19T19:22:04.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.404+0000 7fbe79027700 1 -- 172.21.15.88:0/504523421 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe7419a050 con 0x7fbe740ff350 2024-02-19T19:22:04.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.405+0000 7fbe79027700 1 -- 172.21.15.88:0/504523421 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe74107530 con 0x7fbe740ff350 2024-02-19T19:22:04.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.406+0000 7fbe5bfff700 1 -- 172.21.15.88:0/504523421 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbe64010380 con 0x7fbe740ff350 2024-02-19T19:22:04.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.406+0000 7fbe5bfff700 1 --2- 172.21.15.88:0/504523421 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe5c06d8a0 0x7fbe5c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:04.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.406+0000 7fbe5bfff700 1 -- 172.21.15.88:0/504523421 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbe6408d0f0 con 0x7fbe740ff350 2024-02-19T19:22:04.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.406+0000 7fbe7259c700 1 --2- 172.21.15.88:0/504523421 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe5c06d8a0 0x7fbe5c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:04.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.406+0000 7fbe7259c700 1 --2- 172.21.15.88:0/504523421 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe5c06d8a0 0x7fbe5c06fd50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe680059d0 tx=0x7fbe6801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:04.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.411+0000 7fbe5bfff700 1 -- 172.21.15.88:0/504523421 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbe640568a0 con 0x7fbe740ff350 2024-02-19T19:22:04.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.650+0000 7fbe79027700 1 -- 172.21.15.88:0/504523421 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbe7419a2d0 con 0x7fbe740ff350 2024-02-19T19:22:04.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.651+0000 7fbe5bfff700 1 -- 172.21.15.88:0/504523421 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fbe64056070 con 0x7fbe740ff350 2024-02-19T19:22:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:22:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:22:04.662 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:22:04.663 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:22:04.664 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:22:04.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.653+0000 7fbe79027700 1 -- 172.21.15.88:0/504523421 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe5c06d8a0 msgr2=0x7fbe5c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.653+0000 7fbe79027700 1 --2- 172.21.15.88:0/504523421 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe5c06d8a0 0x7fbe5c06fd50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fbe680059d0 tx=0x7fbe6801a040 comp rx=0 tx=0).stop 2024-02-19T19:22:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.653+0000 7fbe79027700 1 -- 172.21.15.88:0/504523421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe740ff350 msgr2=0x7fbe74194e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.653+0000 7fbe79027700 1 --2- 172.21.15.88:0/504523421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe740ff350 0x7fbe74194e70 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fbe6400b4b0 tx=0x7fbe6400b870 comp rx=0 tx=0).stop 2024-02-19T19:22:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.653+0000 7fbe79027700 1 -- 172.21.15.88:0/504523421 shutdown_connections 2024-02-19T19:22:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.653+0000 7fbe79027700 1 --2- 172.21.15.88:0/504523421 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbe5c06d8a0 0x7fbe5c06fd50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.653+0000 7fbe79027700 1 --2- 172.21.15.88:0/504523421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbe740ff350 0x7fbe74194e70 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.653+0000 7fbe79027700 1 --2- 172.21.15.88:0/504523421 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbe741038b0 0x7fbe741953b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.653+0000 7fbe79027700 1 -- 172.21.15.88:0/504523421 >> 172.21.15.88:0/504523421 conn(0x7fbe740748b0 msgr2=0x7fbe74104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.653+0000 7fbe79027700 1 -- 172.21.15.88:0/504523421 shutdown_connections 2024-02-19T19:22:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.653+0000 7fbe79027700 1 -- 172.21.15.88:0/504523421 wait complete. 2024-02-19T19:22:04.665 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:22:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.759+0000 7f5e88d95700 1 -- 172.21.15.88:0/121320541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84100e80 msgr2=0x7f5e84104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.759+0000 7f5e88d95700 1 --2- 172.21.15.88:0/121320541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84100e80 0x7f5e84104e50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009910 tx=0x7f5e74009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.759+0000 7f5e88d95700 1 -- 172.21.15.88:0/121320541 shutdown_connections 2024-02-19T19:22:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.759+0000 7f5e88d95700 1 --2- 172.21.15.88:0/121320541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84100e80 0x7f5e84104e50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.759+0000 7f5e88d95700 1 --2- 172.21.15.88:0/121320541 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e841004e0 0x7f5e841008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.759+0000 7f5e88d95700 1 -- 172.21.15.88:0/121320541 >> 172.21.15.88:0/121320541 conn(0x7f5e840fbda0 msgr2=0x7f5e840fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.759+0000 7f5e88d95700 1 -- 172.21.15.88:0/121320541 shutdown_connections 2024-02-19T19:22:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.759+0000 7f5e88d95700 1 -- 172.21.15.88:0/121320541 wait complete. 2024-02-19T19:22:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.760+0000 7f5e88d95700 1 Processor -- start 2024-02-19T19:22:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.760+0000 7f5e88d95700 1 -- start start 2024-02-19T19:22:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.760+0000 7f5e88d95700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e841004e0 0x7f5e84195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.760+0000 7f5e88d95700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84100e80 0x7f5e84196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.760+0000 7f5e88d95700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e84196b30 con 0x7f5e84100e80 2024-02-19T19:22:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.760+0000 7f5e88d95700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8419a8c0 con 0x7f5e841004e0 2024-02-19T19:22:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.760+0000 7f5e81d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84100e80 0x7f5e84196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.760+0000 7f5e81d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84100e80 0x7f5e84196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48526/0 (socket says 172.21.15.88:48526) 2024-02-19T19:22:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.760+0000 7f5e81d9b700 1 -- 172.21.15.88:0/4209003733 learned_addr learned my addr 172.21.15.88:0/4209003733 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.761+0000 7f5e8259c700 1 --2- 172.21.15.88:0/4209003733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e841004e0 0x7f5e84195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.761+0000 7f5e81d9b700 1 -- 172.21.15.88:0/4209003733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e841004e0 msgr2=0x7f5e84195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.761+0000 7f5e81d9b700 1 --2- 172.21.15.88:0/4209003733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e841004e0 0x7f5e84195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.761+0000 7f5e81d9b700 1 -- 172.21.15.88:0/4209003733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e740095f0 con 0x7f5e84100e80 2024-02-19T19:22:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.761+0000 7f5e81d9b700 1 --2- 172.21.15.88:0/4209003733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84100e80 0x7f5e84196450 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5e740092e0 tx=0x7f5e7400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.761+0000 7f5e737fe700 1 -- 172.21.15.88:0/4209003733 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e7401d070 con 0x7f5e84100e80 2024-02-19T19:22:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.761+0000 7f5e88d95700 1 -- 172.21.15.88:0/4209003733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8419ab40 con 0x7f5e84100e80 2024-02-19T19:22:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.761+0000 7f5e737fe700 1 -- 172.21.15.88:0/4209003733 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e7400f460 con 0x7f5e84100e80 2024-02-19T19:22:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.761+0000 7f5e88d95700 1 -- 172.21.15.88:0/4209003733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8419b030 con 0x7f5e84100e80 2024-02-19T19:22:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.761+0000 7f5e737fe700 1 -- 172.21.15.88:0/4209003733 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e740175f0 con 0x7f5e84100e80 2024-02-19T19:22:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.762+0000 7f5e88d95700 1 -- 172.21.15.88:0/4209003733 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e84108850 con 0x7f5e84100e80 2024-02-19T19:22:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.762+0000 7f5e737fe700 1 -- 172.21.15.88:0/4209003733 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5e7400f5d0 con 0x7f5e84100e80 2024-02-19T19:22:04.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.763+0000 7f5e737fe700 1 --2- 172.21.15.88:0/4209003733 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6c06d8a0 0x7f5e6c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:04.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.763+0000 7f5e8259c700 1 --2- 172.21.15.88:0/4209003733 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6c06d8a0 0x7f5e6c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:04.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.763+0000 7f5e737fe700 1 -- 172.21.15.88:0/4209003733 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5e7408eaa0 con 0x7f5e84100e80 2024-02-19T19:22:04.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.763+0000 7f5e8259c700 1 --2- 172.21.15.88:0/4209003733 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6c06d8a0 0x7f5e6c06fd50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005fd0 tx=0x7f5e7800a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:04.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.768+0000 7f5e737fe700 1 -- 172.21.15.88:0/4209003733 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e74058250 con 0x7f5e84100e80 2024-02-19T19:22:04.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:04 smithi132 ceph-mon[70676]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:04.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:04 smithi132 ceph-mon[70676]: from='client.35034 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:04.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:04 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/455458255' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:22:04.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.983+0000 7f5e88d95700 1 -- 172.21.15.88:0/4209003733 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e84197380 con 0x7f5e6c06d8a0 2024-02-19T19:22:04.993 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:04 smithi088 ceph-mon[88643]: from='client.35030 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:04.993 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:04 smithi088 ceph-mon[88643]: from='client.35034 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:04.993 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:04 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/455458255' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:22:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.985+0000 7f5e737fe700 1 -- 172.21.15.88:0/4209003733 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f5e84197380 con 0x7f5e6c06d8a0 2024-02-19T19:22:04.994 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:22:04.994 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:22:04.994 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:22:04.994 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:22:04.994 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:22:04.995 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:22:04.995 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:22:04.995 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:22:04.995 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:22:04.995 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:22:04.995 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:22:04.995 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:22:04.995 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:22:04.995 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:22:04.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.987+0000 7f5e88d95700 1 -- 172.21.15.88:0/4209003733 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6c06d8a0 msgr2=0x7f5e6c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.987+0000 7f5e88d95700 1 --2- 172.21.15.88:0/4209003733 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6c06d8a0 0x7f5e6c06fd50 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005fd0 tx=0x7f5e7800a560 comp rx=0 tx=0).stop 2024-02-19T19:22:04.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.987+0000 7f5e88d95700 1 -- 172.21.15.88:0/4209003733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84100e80 msgr2=0x7f5e84196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:04.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.987+0000 7f5e88d95700 1 --2- 172.21.15.88:0/4209003733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84100e80 0x7f5e84196450 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f5e740092e0 tx=0x7f5e7400be20 comp rx=0 tx=0).stop 2024-02-19T19:22:04.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.987+0000 7f5e88d95700 1 -- 172.21.15.88:0/4209003733 shutdown_connections 2024-02-19T19:22:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.987+0000 7f5e88d95700 1 --2- 172.21.15.88:0/4209003733 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6c06d8a0 0x7f5e6c06fd50 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.987+0000 7f5e88d95700 1 --2- 172.21.15.88:0/4209003733 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e841004e0 0x7f5e84195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.987+0000 7f5e88d95700 1 --2- 172.21.15.88:0/4209003733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84100e80 0x7f5e84196450 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.987+0000 7f5e88d95700 1 -- 172.21.15.88:0/4209003733 >> 172.21.15.88:0/4209003733 conn(0x7f5e840fbda0 msgr2=0x7f5e840fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.987+0000 7f5e88d95700 1 -- 172.21.15.88:0/4209003733 shutdown_connections 2024-02-19T19:22:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:04.987+0000 7f5e88d95700 1 -- 172.21.15.88:0/4209003733 wait complete. 2024-02-19T19:22:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.091+0000 7f6ce78eb700 1 -- 172.21.15.88:0/1414206932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ce00ff350 msgr2=0x7f6ce01063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.091+0000 7f6ce78eb700 1 --2- 172.21.15.88:0/1414206932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ce00ff350 0x7f6ce01063b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009910 tx=0x7f6cdc009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 -- 172.21.15.88:0/1414206932 shutdown_connections 2024-02-19T19:22:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 --2- 172.21.15.88:0/1414206932 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce00ff720 0x7f6ce00ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 --2- 172.21.15.88:0/1414206932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ce00ff350 0x7f6ce01063b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 -- 172.21.15.88:0/1414206932 >> 172.21.15.88:0/1414206932 conn(0x7f6ce00748b0 msgr2=0x7f6ce0074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 -- 172.21.15.88:0/1414206932 shutdown_connections 2024-02-19T19:22:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 -- 172.21.15.88:0/1414206932 wait complete. 2024-02-19T19:22:05.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 Processor -- start 2024-02-19T19:22:05.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 -- start start 2024-02-19T19:22:05.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ce00ff350 0x7f6ce019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:05.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce00ff720 0x7f6ce019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:05.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce019d170 con 0x7f6ce00ff350 2024-02-19T19:22:05.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.092+0000 7f6ce78eb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce01a0f00 con 0x7f6ce00ff720 2024-02-19T19:22:05.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6ce5687700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ce00ff350 0x7f6ce019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:05.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6ce5687700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ce00ff350 0x7f6ce019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48550/0 (socket says 172.21.15.88:48550) 2024-02-19T19:22:05.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6ce5687700 1 -- 172.21.15.88:0/2349725135 learned_addr learned my addr 172.21.15.88:0/2349725135 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:05.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6ce4e86700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce00ff720 0x7f6ce019ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:05.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6ce5687700 1 -- 172.21.15.88:0/2349725135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce00ff720 msgr2=0x7f6ce019ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:05.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6ce5687700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce00ff720 0x7f6ce019ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6ce5687700 1 -- 172.21.15.88:0/2349725135 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cdc0095f0 con 0x7f6ce00ff350 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6ce4e86700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce00ff720 0x7f6ce019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6ce5687700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ce00ff350 0x7f6ce019c550 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc006010 tx=0x7f6cdc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6cd67fc700 1 -- 172.21.15.88:0/2349725135 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cdc01d070 con 0x7f6ce00ff350 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6cd67fc700 1 -- 172.21.15.88:0/2349725135 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6cdc004fb0 con 0x7f6ce00ff350 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6ce78eb700 1 -- 172.21.15.88:0/2349725135 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce01a11e0 con 0x7f6ce00ff350 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6cd67fc700 1 -- 172.21.15.88:0/2349725135 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cdc022ea0 con 0x7f6ce00ff350 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.093+0000 7f6ce78eb700 1 -- 172.21.15.88:0/2349725135 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce01a1730 con 0x7f6ce00ff350 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.094+0000 7f6ce78eb700 1 -- 172.21.15.88:0/2349725135 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce0103ae0 con 0x7f6ce00ff350 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.095+0000 7f6cd67fc700 1 -- 172.21.15.88:0/2349725135 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6cdc022470 con 0x7f6ce00ff350 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.095+0000 7f6cd67fc700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ccc06d8a0 0x7f6ccc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.095+0000 7f6cd67fc700 1 -- 172.21.15.88:0/2349725135 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6cdc08ea40 con 0x7f6ce00ff350 2024-02-19T19:22:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.095+0000 7f6ce4e86700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ccc06d8a0 0x7f6ccc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:05.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.095+0000 7f6ce4e86700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ccc06d8a0 0x7f6ccc06fd50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ce019db70 tx=0x7f6cd0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:05.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.100+0000 7f6cd67fc700 1 -- 172.21.15.88:0/2349725135 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6cdc0582a0 con 0x7f6ce00ff350 2024-02-19T19:22:05.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.389+0000 7f6ce78eb700 1 -- 172.21.15.88:0/2349725135 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6ce019d980 con 0x7f6ce00ff350 2024-02-19T19:22:05.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.390+0000 7f6cd67fc700 1 -- 172.21.15.88:0/2349725135 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f6cdc022720 con 0x7f6ce00ff350 2024-02-19T19:22:05.399 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:22:05.399 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:22:05.399 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:22:05.399 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:22:05.400 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:22:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.392+0000 7f6ce78eb700 1 -- 172.21.15.88:0/2349725135 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ccc06d8a0 msgr2=0x7f6ccc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.392+0000 7f6ce78eb700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ccc06d8a0 0x7f6ccc06fd50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ce019db70 tx=0x7f6cd0008040 comp rx=0 tx=0).stop 2024-02-19T19:22:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.392+0000 7f6ce78eb700 1 -- 172.21.15.88:0/2349725135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ce00ff350 msgr2=0x7f6ce019c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.392+0000 7f6ce78eb700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ce00ff350 0x7f6ce019c550 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc006010 tx=0x7f6cdc00be20 comp rx=0 tx=0).stop 2024-02-19T19:22:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.392+0000 7f6ce78eb700 1 -- 172.21.15.88:0/2349725135 shutdown_connections 2024-02-19T19:22:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.392+0000 7f6ce78eb700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ccc06d8a0 0x7f6ccc06fd50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.392+0000 7f6ce78eb700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ce00ff350 0x7f6ce019c550 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.392+0000 7f6ce78eb700 1 --2- 172.21.15.88:0/2349725135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ce00ff720 0x7f6ce019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.392+0000 7f6ce78eb700 1 -- 172.21.15.88:0/2349725135 >> 172.21.15.88:0/2349725135 conn(0x7f6ce00748b0 msgr2=0x7f6ce00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.392+0000 7f6ce78eb700 1 -- 172.21.15.88:0/2349725135 shutdown_connections 2024-02-19T19:22:05.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:05.392+0000 7f6ce78eb700 1 -- 172.21.15.88:0/2349725135 wait complete. 2024-02-19T19:22:05.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:05 smithi088 ceph-mon[88643]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:05.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:05 smithi088 ceph-mon[88643]: pgmap v554: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:05.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:05 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/504523421' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:22:05.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:05 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2349725135' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:22:05.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:05 smithi132 ceph-mon[70676]: from='client.35038 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:05.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:05 smithi132 ceph-mon[70676]: pgmap v554: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:05.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:05 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/504523421' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:22:05.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:05 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2349725135' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:22:06.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:06 smithi132 ceph-mon[70676]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:07.089 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:06 smithi088 ceph-mon[88643]: from='client.35050 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:07.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:07 smithi132 ceph-mon[70676]: pgmap v555: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:08.089 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:07 smithi088 ceph-mon[88643]: pgmap v555: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:09.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:09 smithi132 ceph-mon[70676]: pgmap v556: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:10.089 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:09 smithi088 ceph-mon[88643]: pgmap v556: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:11.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:11 smithi132 ceph-mon[70676]: pgmap v557: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:12.089 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:11 smithi088 ceph-mon[88643]: pgmap v557: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:13.892 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:13 smithi088 ceph-mon[88643]: pgmap v558: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:13.892 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:22:13.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:13 smithi132 ceph-mon[70676]: pgmap v558: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:13.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:22:15.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:15 smithi132 ceph-mon[70676]: pgmap v559: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:16.089 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:15 smithi088 ceph-mon[88643]: pgmap v559: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:17.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:17 smithi132 ceph-mon[70676]: pgmap v560: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:18.089 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:17 smithi088 ceph-mon[88643]: pgmap v560: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:19.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:19 smithi132 ceph-mon[70676]: pgmap v561: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:20.089 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:19 smithi088 ceph-mon[88643]: pgmap v561: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:21.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:21 smithi132 ceph-mon[70676]: pgmap v562: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:22.089 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:21 smithi088 ceph-mon[88643]: pgmap v562: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:22.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:22 smithi132 ceph-mon[70676]: pgmap v563: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:23.089 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:22 smithi088 ceph-mon[88643]: pgmap v563: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:25.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:25 smithi132 ceph-mon[70676]: pgmap v564: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:25.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:25 smithi088 ceph-mon[88643]: pgmap v564: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:27.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:27 smithi132 ceph-mon[70676]: pgmap v565: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:27.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:27 smithi088 ceph-mon[88643]: pgmap v565: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:28.720 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:22:28.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:22:29.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:29 smithi132 ceph-mon[70676]: pgmap v566: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:29.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:29 smithi088 ceph-mon[88643]: pgmap v566: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:31.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:31 smithi132 ceph-mon[70676]: pgmap v567: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:31.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:31 smithi088 ceph-mon[88643]: pgmap v567: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:33.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:33 smithi132 ceph-mon[70676]: pgmap v568: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:33.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:33 smithi088 ceph-mon[88643]: pgmap v568: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.509+0000 7f086057f700 1 -- 172.21.15.88:0/2140450461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0858105230 msgr2=0x7f0858105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.509+0000 7f086057f700 1 --2- 172.21.15.88:0/2140450461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0858105230 0x7f0858105600 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f085000b0a0 tx=0x7f085000b3b0 comp rx=0 tx=0).stop 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.509+0000 7f086057f700 1 -- 172.21.15.88:0/2140450461 shutdown_connections 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.509+0000 7f086057f700 1 --2- 172.21.15.88:0/2140450461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0858068540 0x7f08580689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.509+0000 7f086057f700 1 --2- 172.21.15.88:0/2140450461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0858105230 0x7f0858105600 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.509+0000 7f086057f700 1 -- 172.21.15.88:0/2140450461 >> 172.21.15.88:0/2140450461 conn(0x7f0858075130 msgr2=0x7f0858075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.509+0000 7f086057f700 1 -- 172.21.15.88:0/2140450461 shutdown_connections 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.509+0000 7f086057f700 1 -- 172.21.15.88:0/2140450461 wait complete. 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f086057f700 1 Processor -- start 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f086057f700 1 -- start start 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f086057f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0858068540 0x7f0858197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:35.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f086057f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0858105230 0x7f0858198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f086057f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0858198b80 con 0x7f0858105230 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f086057f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0858198cc0 con 0x7f0858068540 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f085db1a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0858105230 0x7f0858198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f085db1a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0858105230 0x7f0858198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50040/0 (socket says 172.21.15.88:50040) 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f085db1a700 1 -- 172.21.15.88:0/1563305031 learned_addr learned my addr 172.21.15.88:0/1563305031 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f085db1a700 1 -- 172.21.15.88:0/1563305031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0858068540 msgr2=0x7f0858197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f085e31b700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0858068540 0x7f0858197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f085db1a700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0858068540 0x7f0858197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f085db1a700 1 -- 172.21.15.88:0/1563305031 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0850009d00 con 0x7f0858105230 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.510+0000 7f085e31b700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0858068540 0x7f0858197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.511+0000 7f085db1a700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0858105230 0x7f0858198530 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f084800c930 tx=0x7f084800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.511+0000 7f084f7fe700 1 -- 172.21.15.88:0/1563305031 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0848007ad0 con 0x7f0858105230 2024-02-19T19:22:35.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.511+0000 7f084f7fe700 1 -- 172.21.15.88:0/1563305031 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0848007c30 con 0x7f0858105230 2024-02-19T19:22:35.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.511+0000 7f086057f700 1 -- 172.21.15.88:0/1563305031 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085819cb10 con 0x7f0858105230 2024-02-19T19:22:35.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.511+0000 7f084f7fe700 1 -- 172.21.15.88:0/1563305031 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0848018610 con 0x7f0858105230 2024-02-19T19:22:35.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.511+0000 7f086057f700 1 -- 172.21.15.88:0/1563305031 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085819d060 con 0x7f0858105230 2024-02-19T19:22:35.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.512+0000 7f086057f700 1 -- 172.21.15.88:0/1563305031 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085810a7d0 con 0x7f0858105230 2024-02-19T19:22:35.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.512+0000 7f084f7fe700 1 -- 172.21.15.88:0/1563305031 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f084800f450 con 0x7f0858105230 2024-02-19T19:22:35.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.513+0000 7f084f7fe700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f084406d8a0 0x7f084406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:35.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.513+0000 7f084f7fe700 1 -- 172.21.15.88:0/1563305031 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f084808d780 con 0x7f0858105230 2024-02-19T19:22:35.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.513+0000 7f085e31b700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f084406d8a0 0x7f084406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:35.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.513+0000 7f085e31b700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f084406d8a0 0x7f084406fd50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0850009480 tx=0x7f0850000f40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.518+0000 7f084f7fe700 1 -- 172.21.15.88:0/1563305031 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0848056f30 con 0x7f0858105230 2024-02-19T19:22:35.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:35 smithi132 ceph-mon[70676]: pgmap v569: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:35.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.744+0000 7f086057f700 1 -- 172.21.15.88:0/1563305031 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f085802cca0 con 0x7f084406d8a0 2024-02-19T19:22:35.755 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:35 smithi088 ceph-mon[88643]: pgmap v569: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:35.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.746+0000 7f084f7fe700 1 -- 172.21.15.88:0/1563305031 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f085802cca0 con 0x7f084406d8a0 2024-02-19T19:22:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.748+0000 7f086057f700 1 -- 172.21.15.88:0/1563305031 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f084406d8a0 msgr2=0x7f084406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.748+0000 7f086057f700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f084406d8a0 0x7f084406fd50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0850009480 tx=0x7f0850000f40 comp rx=0 tx=0).stop 2024-02-19T19:22:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.748+0000 7f086057f700 1 -- 172.21.15.88:0/1563305031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0858105230 msgr2=0x7f0858198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.748+0000 7f086057f700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0858105230 0x7f0858198530 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f084800c930 tx=0x7f084800ccf0 comp rx=0 tx=0).stop 2024-02-19T19:22:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.748+0000 7f086057f700 1 -- 172.21.15.88:0/1563305031 shutdown_connections 2024-02-19T19:22:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.748+0000 7f086057f700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f084406d8a0 0x7f084406fd50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.748+0000 7f086057f700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0858068540 0x7f0858197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:35.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.748+0000 7f086057f700 1 --2- 172.21.15.88:0/1563305031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0858105230 0x7f0858198530 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:35.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.748+0000 7f086057f700 1 -- 172.21.15.88:0/1563305031 >> 172.21.15.88:0/1563305031 conn(0x7f0858075130 msgr2=0x7f0858102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:35.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.748+0000 7f086057f700 1 -- 172.21.15.88:0/1563305031 shutdown_connections 2024-02-19T19:22:35.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.748+0000 7f086057f700 1 -- 172.21.15.88:0/1563305031 wait complete. 2024-02-19T19:22:35.768 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:22:35.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.852+0000 7fb44916c700 1 -- 172.21.15.88:0/3961812120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4440ff3c0 msgr2=0x7fb4440ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:35.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.852+0000 7fb44916c700 1 --2- 172.21.15.88:0/3961812120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4440ff3c0 0x7fb4440ff790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb434009910 tx=0x7fb434009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:35.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.852+0000 7fb44916c700 1 -- 172.21.15.88:0/3961812120 shutdown_connections 2024-02-19T19:22:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.852+0000 7fb44916c700 1 --2- 172.21.15.88:0/3961812120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4440ffd60 0x7fb444103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.852+0000 7fb44916c700 1 --2- 172.21.15.88:0/3961812120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4440ff3c0 0x7fb4440ff790 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.852+0000 7fb44916c700 1 -- 172.21.15.88:0/3961812120 >> 172.21.15.88:0/3961812120 conn(0x7fb4440fac80 msgr2=0x7fb4440fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.852+0000 7fb44916c700 1 -- 172.21.15.88:0/3961812120 shutdown_connections 2024-02-19T19:22:35.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.852+0000 7fb44916c700 1 -- 172.21.15.88:0/3961812120 wait complete. 2024-02-19T19:22:35.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.853+0000 7fb44916c700 1 Processor -- start 2024-02-19T19:22:35.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.853+0000 7fb44916c700 1 -- start start 2024-02-19T19:22:35.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.853+0000 7fb44916c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4440ff3c0 0x7fb444196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:35.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44916c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4440ffd60 0x7fb444197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:35.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44916c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb444197960 con 0x7fb4440ffd60 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44916c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb444197aa0 con 0x7fb4440ff3c0 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4440ffd60 0x7fb444197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4440ffd60 0x7fb444197340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50058/0 (socket says 172.21.15.88:50058) 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44259c700 1 -- 172.21.15.88:0/2404216153 learned_addr learned my addr 172.21.15.88:0/2404216153 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb442d9d700 1 --2- 172.21.15.88:0/2404216153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4440ff3c0 0x7fb444196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44259c700 1 -- 172.21.15.88:0/2404216153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4440ff3c0 msgr2=0x7fb444196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44259c700 1 --2- 172.21.15.88:0/2404216153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4440ff3c0 0x7fb444196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44259c700 1 -- 172.21.15.88:0/2404216153 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4340095f0 con 0x7fb4440ffd60 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44259c700 1 --2- 172.21.15.88:0/2404216153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4440ffd60 0x7fb444197340 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb43800b480 tx=0x7fb43800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb42bfff700 1 -- 172.21.15.88:0/2404216153 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb438011800 con 0x7fb4440ffd60 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb42bfff700 1 -- 172.21.15.88:0/2404216153 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb438011e20 con 0x7fb4440ffd60 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb42bfff700 1 -- 172.21.15.88:0/2404216153 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb438010ea0 con 0x7fb4440ffd60 2024-02-19T19:22:35.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44916c700 1 -- 172.21.15.88:0/2404216153 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44419fe70 con 0x7fb4440ffd60 2024-02-19T19:22:35.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.854+0000 7fb44916c700 1 -- 172.21.15.88:0/2404216153 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4441a0330 con 0x7fb4440ffd60 2024-02-19T19:22:35.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.855+0000 7fb44916c700 1 -- 172.21.15.88:0/2404216153 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4441076d0 con 0x7fb4440ffd60 2024-02-19T19:22:35.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.856+0000 7fb42bfff700 1 -- 172.21.15.88:0/2404216153 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb438010380 con 0x7fb4440ffd60 2024-02-19T19:22:35.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.856+0000 7fb42bfff700 1 --2- 172.21.15.88:0/2404216153 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb42c06d8a0 0x7fb42c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:35.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.856+0000 7fb42bfff700 1 -- 172.21.15.88:0/2404216153 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb43808cf80 con 0x7fb4440ffd60 2024-02-19T19:22:35.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.856+0000 7fb442d9d700 1 --2- 172.21.15.88:0/2404216153 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb42c06d8a0 0x7fb42c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:35.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.857+0000 7fb442d9d700 1 --2- 172.21.15.88:0/2404216153 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb42c06d8a0 0x7fb42c06fd50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb434005f50 tx=0x7fb43401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:35.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:35.861+0000 7fb42bfff700 1 -- 172.21.15.88:0/2404216153 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb4380567b0 con 0x7fb4440ffd60 2024-02-19T19:22:36.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.075+0000 7fb44916c700 1 -- 172.21.15.88:0/2404216153 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb444061190 con 0x7fb42c06d8a0 2024-02-19T19:22:36.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.077+0000 7fb42bfff700 1 -- 172.21.15.88:0/2404216153 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb444061190 con 0x7fb42c06d8a0 2024-02-19T19:22:36.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.079+0000 7fb44916c700 1 -- 172.21.15.88:0/2404216153 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb42c06d8a0 msgr2=0x7fb42c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.079+0000 7fb44916c700 1 --2- 172.21.15.88:0/2404216153 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb42c06d8a0 0x7fb42c06fd50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb434005f50 tx=0x7fb43401a040 comp rx=0 tx=0).stop 2024-02-19T19:22:36.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.079+0000 7fb44916c700 1 -- 172.21.15.88:0/2404216153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4440ffd60 msgr2=0x7fb444197340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.079+0000 7fb44916c700 1 --2- 172.21.15.88:0/2404216153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4440ffd60 0x7fb444197340 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb43800b480 tx=0x7fb43800b790 comp rx=0 tx=0).stop 2024-02-19T19:22:36.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.079+0000 7fb44916c700 1 -- 172.21.15.88:0/2404216153 shutdown_connections 2024-02-19T19:22:36.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.079+0000 7fb44916c700 1 --2- 172.21.15.88:0/2404216153 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb42c06d8a0 0x7fb42c06fd50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.079+0000 7fb44916c700 1 --2- 172.21.15.88:0/2404216153 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb4440ff3c0 0x7fb444196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.079+0000 7fb44916c700 1 --2- 172.21.15.88:0/2404216153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4440ffd60 0x7fb444197340 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.079+0000 7fb44916c700 1 -- 172.21.15.88:0/2404216153 >> 172.21.15.88:0/2404216153 conn(0x7fb4440fac80 msgr2=0x7fb4440fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:36.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.079+0000 7fb44916c700 1 -- 172.21.15.88:0/2404216153 shutdown_connections 2024-02-19T19:22:36.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.079+0000 7fb44916c700 1 -- 172.21.15.88:0/2404216153 wait complete. 2024-02-19T19:22:36.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.182+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/455321951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8dc0ffc50 msgr2=0x7ff8dc103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.182+0000 7ff8e2c8e700 1 --2- 172.21.15.88:0/455321951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8dc0ffc50 0x7ff8dc103c20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8009910 tx=0x7ff8c8009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:36.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.182+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/455321951 shutdown_connections 2024-02-19T19:22:36.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.182+0000 7ff8e2c8e700 1 --2- 172.21.15.88:0/455321951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8dc0ffc50 0x7ff8dc103c20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.182+0000 7ff8e2c8e700 1 --2- 172.21.15.88:0/455321951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8dc0ff2b0 0x7ff8dc0ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.182+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/455321951 >> 172.21.15.88:0/455321951 conn(0x7ff8dc0fab50 msgr2=0x7ff8dc0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:36.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.182+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/455321951 shutdown_connections 2024-02-19T19:22:36.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.182+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/455321951 wait complete. 2024-02-19T19:22:36.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e2c8e700 1 Processor -- start 2024-02-19T19:22:36.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e2c8e700 1 -- start start 2024-02-19T19:22:36.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e2c8e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8dc0ff2b0 0x7ff8dc195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:36.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e2c8e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8dc0ffc50 0x7ff8dc1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:36.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e2c8e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc196ab0 con 0x7ff8dc0ff2b0 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e2c8e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc19a840 con 0x7ff8dc0ffc50 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e0a2a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8dc0ff2b0 0x7ff8dc195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e0a2a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8dc0ff2b0 0x7ff8dc195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50084/0 (socket says 172.21.15.88:50084) 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e0a2a700 1 -- 172.21.15.88:0/2000967088 learned_addr learned my addr 172.21.15.88:0/2000967088 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e0a2a700 1 -- 172.21.15.88:0/2000967088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8dc0ffc50 msgr2=0x7ff8dc1963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e0a2a700 1 --2- 172.21.15.88:0/2000967088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8dc0ffc50 0x7ff8dc1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.183+0000 7ff8e0a2a700 1 -- 172.21.15.88:0/2000967088 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c80095f0 con 0x7ff8dc0ff2b0 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.184+0000 7ff8e0a2a700 1 --2- 172.21.15.88:0/2000967088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8dc0ff2b0 0x7ff8dc195e90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000b4b0 tx=0x7ff8d000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.184+0000 7ff8d9ffb700 1 -- 172.21.15.88:0/2000967088 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d0011800 con 0x7ff8dc0ff2b0 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.184+0000 7ff8d9ffb700 1 -- 172.21.15.88:0/2000967088 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8d0011e20 con 0x7ff8dc0ff2b0 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.184+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/2000967088 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8dc19ab20 con 0x7ff8dc0ff2b0 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.184+0000 7ff8d9ffb700 1 -- 172.21.15.88:0/2000967088 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8d000f2e0 con 0x7ff8dc0ff2b0 2024-02-19T19:22:36.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.184+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/2000967088 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8dc19b070 con 0x7ff8dc0ff2b0 2024-02-19T19:22:36.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.185+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/2000967088 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8dc04f2a0 con 0x7ff8dc0ff2b0 2024-02-19T19:22:36.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.185+0000 7ff8d9ffb700 1 -- 172.21.15.88:0/2000967088 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff8d0010380 con 0x7ff8dc0ff2b0 2024-02-19T19:22:36.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.186+0000 7ff8d9ffb700 1 --2- 172.21.15.88:0/2000967088 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8cc06d8a0 0x7ff8cc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:36.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.186+0000 7ff8d9ffb700 1 -- 172.21.15.88:0/2000967088 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff8d008d110 con 0x7ff8dc0ff2b0 2024-02-19T19:22:36.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.186+0000 7ff8dbfff700 1 --2- 172.21.15.88:0/2000967088 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8cc06d8a0 0x7ff8cc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:36.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.186+0000 7ff8dbfff700 1 --2- 172.21.15.88:0/2000967088 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8cc06d8a0 0x7ff8cc06fd50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800b5c0 tx=0x7ff8c8005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:36.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.191+0000 7ff8d9ffb700 1 -- 172.21.15.88:0/2000967088 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff8d00568c0 con 0x7ff8dc0ff2b0 2024-02-19T19:22:36.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.400+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/2000967088 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8dc19b590 con 0x7ff8cc06d8a0 2024-02-19T19:22:36.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.410+0000 7ff8d9ffb700 1 -- 172.21.15.88:0/2000967088 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff8dc19b590 con 0x7ff8cc06d8a0 2024-02-19T19:22:36.420 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:22:36.420 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (19m) 11m ago 30m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (30m) 11m ago 30m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (12m) 11m ago 28m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (15m) 11m ago 30m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (15m) 11m ago 28m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (19m) 11m ago 29m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (12m) 11m ago 23m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (11m) 11m ago 23m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (11m) 11m ago 23m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (23m) 11m ago 23m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (21m) 11m ago 31m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (20m) 11m ago 28m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (17m) 11m ago 32m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (17m) 11m ago 28m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (20m) 11m ago 30m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (20m) 11m ago 28m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:22:36.421 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (15m) 11m ago 27m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:22:36.422 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (14m) 11m ago 27m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:22:36.422 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (14m) 11m ago 26m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:22:36.422 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (13m) 11m ago 26m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:22:36.422 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (12m) 11m ago 25m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:22:36.422 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (12m) 11m ago 25m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:22:36.422 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (20m) 11m ago 29m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:22:36.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.412+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/2000967088 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8cc06d8a0 msgr2=0x7ff8cc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.412+0000 7ff8e2c8e700 1 --2- 172.21.15.88:0/2000967088 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8cc06d8a0 0x7ff8cc06fd50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800b5c0 tx=0x7ff8c8005960 comp rx=0 tx=0).stop 2024-02-19T19:22:36.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.413+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/2000967088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8dc0ff2b0 msgr2=0x7ff8dc195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.413+0000 7ff8e2c8e700 1 --2- 172.21.15.88:0/2000967088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8dc0ff2b0 0x7ff8dc195e90 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000b4b0 tx=0x7ff8d000b870 comp rx=0 tx=0).stop 2024-02-19T19:22:36.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.413+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/2000967088 shutdown_connections 2024-02-19T19:22:36.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.413+0000 7ff8e2c8e700 1 --2- 172.21.15.88:0/2000967088 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8cc06d8a0 0x7ff8cc06fd50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.413+0000 7ff8e2c8e700 1 --2- 172.21.15.88:0/2000967088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8dc0ff2b0 0x7ff8dc195e90 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.413+0000 7ff8e2c8e700 1 --2- 172.21.15.88:0/2000967088 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8dc0ffc50 0x7ff8dc1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.413+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/2000967088 >> 172.21.15.88:0/2000967088 conn(0x7ff8dc0fab50 msgr2=0x7ff8dc103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:36.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.413+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/2000967088 shutdown_connections 2024-02-19T19:22:36.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.413+0000 7ff8e2c8e700 1 -- 172.21.15.88:0/2000967088 wait complete. 2024-02-19T19:22:36.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.515+0000 7fdba1847700 1 -- 172.21.15.88:0/2376033843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0ffc60 msgr2=0x7fdb9c10a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.515+0000 7fdba1847700 1 --2- 172.21.15.88:0/2376033843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0ffc60 0x7fdb9c10a6d0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdb90009910 tx=0x7fdb90009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:36.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.515+0000 7fdba1847700 1 -- 172.21.15.88:0/2376033843 shutdown_connections 2024-02-19T19:22:36.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.515+0000 7fdba1847700 1 --2- 172.21.15.88:0/2376033843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0ffc60 0x7fdb9c10a6d0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.515+0000 7fdba1847700 1 --2- 172.21.15.88:0/2376033843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb9c0ff350 0x7fdb9c0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.515+0000 7fdba1847700 1 -- 172.21.15.88:0/2376033843 >> 172.21.15.88:0/2376033843 conn(0x7fdb9c0748b0 msgr2=0x7fdb9c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:36.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.515+0000 7fdba1847700 1 -- 172.21.15.88:0/2376033843 shutdown_connections 2024-02-19T19:22:36.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.516+0000 7fdba1847700 1 -- 172.21.15.88:0/2376033843 wait complete. 2024-02-19T19:22:36.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.516+0000 7fdba1847700 1 Processor -- start 2024-02-19T19:22:36.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.516+0000 7fdba1847700 1 -- start start 2024-02-19T19:22:36.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.516+0000 7fdba1847700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb9c0ff350 0x7fdb9c195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:36.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.516+0000 7fdba1847700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0ffc60 0x7fdb9c196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:36.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.516+0000 7fdba1847700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c196b40 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.516+0000 7fdba1847700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c19a8d0 con 0x7fdb9c0ff350 2024-02-19T19:22:36.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdb9a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0ffc60 0x7fdb9c196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:36.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdb9a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0ffc60 0x7fdb9c196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50092/0 (socket says 172.21.15.88:50092) 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdb9a7fc700 1 -- 172.21.15.88:0/4097062685 learned_addr learned my addr 172.21.15.88:0/4097062685 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdb9affd700 1 --2- 172.21.15.88:0/4097062685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb9c0ff350 0x7fdb9c195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdb9a7fc700 1 -- 172.21.15.88:0/4097062685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb9c0ff350 msgr2=0x7fdb9c195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdb9a7fc700 1 --2- 172.21.15.88:0/4097062685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb9c0ff350 0x7fdb9c195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdb9a7fc700 1 -- 172.21.15.88:0/4097062685 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb900095f0 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdb9a7fc700 1 --2- 172.21.15.88:0/4097062685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0ffc60 0x7fdb9c196460 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdb90000c00 tx=0x7fdb900056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdba0845700 1 -- 172.21.15.88:0/4097062685 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb9001d070 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdba1847700 1 -- 172.21.15.88:0/4097062685 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9c19abb0 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdba0845700 1 -- 172.21.15.88:0/4097062685 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb9000f460 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdba0845700 1 -- 172.21.15.88:0/4097062685 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb90021640 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.517+0000 7fdba1847700 1 -- 172.21.15.88:0/4097062685 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c19b100 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.518+0000 7fdba1847700 1 -- 172.21.15.88:0/4097062685 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9c107dd0 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.519+0000 7fdba0845700 1 -- 172.21.15.88:0/4097062685 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdb9000f5d0 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.519+0000 7fdba0845700 1 --2- 172.21.15.88:0/4097062685 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb8406d8a0 0x7fdb8406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:36.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.519+0000 7fdba0845700 1 -- 172.21.15.88:0/4097062685 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdb9008e8b0 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.519+0000 7fdb9affd700 1 --2- 172.21.15.88:0/4097062685 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb8406d8a0 0x7fdb8406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:36.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.520+0000 7fdb9affd700 1 --2- 172.21.15.88:0/4097062685 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb8406d8a0 0x7fdb8406fd50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c006fd0 tx=0x7fdb8c008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:36.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.525+0000 7fdba0845700 1 -- 172.21.15.88:0/4097062685 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb90058060 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.806+0000 7fdba1847700 1 -- 172.21.15.88:0/4097062685 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdb9c04f2a0 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.806+0000 7fdba0845700 1 -- 172.21.15.88:0/4097062685 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fdb9005b720 con 0x7fdb9c0ffc60 2024-02-19T19:22:36.816 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:22:36.817 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:22:36.818 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:22:36.818 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:22:36.818 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:22:36.818 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:22:36.818 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:22:36.818 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:22:36.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.809+0000 7fdba1847700 1 -- 172.21.15.88:0/4097062685 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb8406d8a0 msgr2=0x7fdb8406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.809+0000 7fdba1847700 1 --2- 172.21.15.88:0/4097062685 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb8406d8a0 0x7fdb8406fd50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c006fd0 tx=0x7fdb8c008040 comp rx=0 tx=0).stop 2024-02-19T19:22:36.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.809+0000 7fdba1847700 1 -- 172.21.15.88:0/4097062685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0ffc60 msgr2=0x7fdb9c196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.809+0000 7fdba1847700 1 --2- 172.21.15.88:0/4097062685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0ffc60 0x7fdb9c196460 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fdb90000c00 tx=0x7fdb900056a0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.809+0000 7fdba1847700 1 -- 172.21.15.88:0/4097062685 shutdown_connections 2024-02-19T19:22:36.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.809+0000 7fdba1847700 1 --2- 172.21.15.88:0/4097062685 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb8406d8a0 0x7fdb8406fd50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.809+0000 7fdba1847700 1 --2- 172.21.15.88:0/4097062685 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb9c0ff350 0x7fdb9c195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.809+0000 7fdba1847700 1 --2- 172.21.15.88:0/4097062685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0ffc60 0x7fdb9c196460 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.809+0000 7fdba1847700 1 -- 172.21.15.88:0/4097062685 >> 172.21.15.88:0/4097062685 conn(0x7fdb9c0748b0 msgr2=0x7fdb9c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:36.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.809+0000 7fdba1847700 1 -- 172.21.15.88:0/4097062685 shutdown_connections 2024-02-19T19:22:36.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.809+0000 7fdba1847700 1 -- 172.21.15.88:0/4097062685 wait complete. 2024-02-19T19:22:36.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.912+0000 7f5b59522700 1 -- 172.21.15.88:0/2740722765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b541004c0 msgr2=0x7f5b54100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.912+0000 7f5b59522700 1 --2- 172.21.15.88:0/2740722765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b541004c0 0x7f5b54100890 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009910 tx=0x7f5b44009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:36.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.912+0000 7f5b59522700 1 -- 172.21.15.88:0/2740722765 shutdown_connections 2024-02-19T19:22:36.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.912+0000 7f5b59522700 1 --2- 172.21.15.88:0/2740722765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b54100e60 0x7f5b54104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.912+0000 7f5b59522700 1 --2- 172.21.15.88:0/2740722765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b541004c0 0x7f5b54100890 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.912+0000 7f5b59522700 1 -- 172.21.15.88:0/2740722765 >> 172.21.15.88:0/2740722765 conn(0x7f5b540fbd80 msgr2=0x7f5b540fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:36.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.912+0000 7f5b59522700 1 -- 172.21.15.88:0/2740722765 shutdown_connections 2024-02-19T19:22:36.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.912+0000 7f5b59522700 1 -- 172.21.15.88:0/2740722765 wait complete. 2024-02-19T19:22:36.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.912+0000 7f5b59522700 1 Processor -- start 2024-02-19T19:22:36.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b59522700 1 -- start start 2024-02-19T19:22:36.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b59522700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b541004c0 0x7f5b54073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:36.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b59522700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b54100e60 0x7f5b54073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b59522700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54073ad0 con 0x7f5b54100e60 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b59522700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b54073c10 con 0x7f5b541004c0 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b4a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b54100e60 0x7f5b54073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b4a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b54100e60 0x7f5b54073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50098/0 (socket says 172.21.15.88:50098) 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b4a7fc700 1 -- 172.21.15.88:0/3400387443 learned_addr learned my addr 172.21.15.88:0/3400387443 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b52ffd700 1 --2- 172.21.15.88:0/3400387443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b541004c0 0x7f5b54073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b4a7fc700 1 -- 172.21.15.88:0/3400387443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b541004c0 msgr2=0x7f5b54073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b4a7fc700 1 --2- 172.21.15.88:0/3400387443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b541004c0 0x7f5b54073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b4a7fc700 1 -- 172.21.15.88:0/3400387443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b440095f0 con 0x7f5b54100e60 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b4a7fc700 1 --2- 172.21.15.88:0/3400387443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b54100e60 0x7f5b54073590 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00b480 tx=0x7f5b3c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.913+0000 7f5b50ff9700 1 -- 172.21.15.88:0/3400387443 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b3c0118c0 con 0x7f5b54100e60 2024-02-19T19:22:36.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.914+0000 7f5b50ff9700 1 -- 172.21.15.88:0/3400387443 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b3c010380 con 0x7f5b54100e60 2024-02-19T19:22:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.914+0000 7f5b50ff9700 1 -- 172.21.15.88:0/3400387443 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b3c00f4f0 con 0x7f5b54100e60 2024-02-19T19:22:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.914+0000 7f5b59522700 1 -- 172.21.15.88:0/3400387443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b541a2160 con 0x7f5b54100e60 2024-02-19T19:22:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.914+0000 7f5b59522700 1 -- 172.21.15.88:0/3400387443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b541a2570 con 0x7f5b54100e60 2024-02-19T19:22:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.915+0000 7f5b59522700 1 -- 172.21.15.88:0/3400387443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b541087d0 con 0x7f5b54100e60 2024-02-19T19:22:36.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.915+0000 7f5b50ff9700 1 -- 172.21.15.88:0/3400387443 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5b3c0104f0 con 0x7f5b54100e60 2024-02-19T19:22:36.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.915+0000 7f5b50ff9700 1 --2- 172.21.15.88:0/3400387443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b4006d8a0 0x7f5b4006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:36.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.916+0000 7f5b52ffd700 1 --2- 172.21.15.88:0/3400387443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b4006d8a0 0x7f5b4006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:36.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.916+0000 7f5b50ff9700 1 -- 172.21.15.88:0/3400387443 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5b3c08d0a0 con 0x7f5b54100e60 2024-02-19T19:22:36.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.916+0000 7f5b52ffd700 1 --2- 172.21.15.88:0/3400387443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b4006d8a0 0x7f5b4006fd50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006010 tx=0x7f5b4401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:36.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:36.921+0000 7f5b50ff9700 1 -- 172.21.15.88:0/3400387443 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b3c0568d0 con 0x7f5b54100e60 2024-02-19T19:22:37.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.164+0000 7f5b59522700 1 -- 172.21.15.88:0/3400387443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5b5404f2a0 con 0x7f5b54100e60 2024-02-19T19:22:37.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.164+0000 7f5b50ff9700 1 -- 172.21.15.88:0/3400387443 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f5b3c059f90 con 0x7f5b54100e60 2024-02-19T19:22:37.174 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:22:37.174 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:22:37.174 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:22:37.174 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:22:37.174 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:22:37.175 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:22:37.176 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:22:37.177 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:22:37.177 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:22:37.177 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:22:37.177 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:22:37.177 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:22:37.177 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:22:37.177 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:22:37.177 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:22:37.177 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:22:37.177 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:22:37.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.166+0000 7f5b59522700 1 -- 172.21.15.88:0/3400387443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b4006d8a0 msgr2=0x7f5b4006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:37.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.166+0000 7f5b59522700 1 --2- 172.21.15.88:0/3400387443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b4006d8a0 0x7f5b4006fd50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5b44006010 tx=0x7f5b4401a040 comp rx=0 tx=0).stop 2024-02-19T19:22:37.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.166+0000 7f5b59522700 1 -- 172.21.15.88:0/3400387443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b54100e60 msgr2=0x7f5b54073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:37.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.166+0000 7f5b59522700 1 --2- 172.21.15.88:0/3400387443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b54100e60 0x7f5b54073590 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00b480 tx=0x7f5b3c00b840 comp rx=0 tx=0).stop 2024-02-19T19:22:37.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.166+0000 7f5b59522700 1 -- 172.21.15.88:0/3400387443 shutdown_connections 2024-02-19T19:22:37.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.166+0000 7f5b59522700 1 --2- 172.21.15.88:0/3400387443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b4006d8a0 0x7f5b4006fd50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.166+0000 7f5b59522700 1 --2- 172.21.15.88:0/3400387443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b541004c0 0x7f5b54073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.166+0000 7f5b59522700 1 --2- 172.21.15.88:0/3400387443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b54100e60 0x7f5b54073590 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.166+0000 7f5b59522700 1 -- 172.21.15.88:0/3400387443 >> 172.21.15.88:0/3400387443 conn(0x7f5b540fbd80 msgr2=0x7f5b540fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:37.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.166+0000 7f5b59522700 1 -- 172.21.15.88:0/3400387443 shutdown_connections 2024-02-19T19:22:37.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.166+0000 7f5b59522700 1 -- 172.21.15.88:0/3400387443 wait complete. 2024-02-19T19:22:37.179 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:22:37.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.271+0000 7f004bdac700 1 -- 172.21.15.88:0/3640067098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0044100d70 msgr2=0x7f00441094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:37.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.271+0000 7f004bdac700 1 --2- 172.21.15.88:0/3640067098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0044100d70 0x7f00441094d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0034009910 tx=0x7f0034009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:37.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.271+0000 7f004bdac700 1 -- 172.21.15.88:0/3640067098 shutdown_connections 2024-02-19T19:22:37.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.271+0000 7f004bdac700 1 --2- 172.21.15.88:0/3640067098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0044100d70 0x7f00441094d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.271+0000 7f004bdac700 1 --2- 172.21.15.88:0/3640067098 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0044100460 0x7f0044100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.271+0000 7f004bdac700 1 -- 172.21.15.88:0/3640067098 >> 172.21.15.88:0/3640067098 conn(0x7f00440748b0 msgr2=0x7f0044074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:37.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.271+0000 7f004bdac700 1 -- 172.21.15.88:0/3640067098 shutdown_connections 2024-02-19T19:22:37.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.271+0000 7f004bdac700 1 -- 172.21.15.88:0/3640067098 wait complete. 2024-02-19T19:22:37.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f004bdac700 1 Processor -- start 2024-02-19T19:22:37.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f004bdac700 1 -- start start 2024-02-19T19:22:37.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f004bdac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0044100460 0x7f004419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:37.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f004bdac700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0044100d70 0x7f004419b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:37.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f004bdac700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004419b690 con 0x7f0044100d70 2024-02-19T19:22:37.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f004bdac700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0044194b40 con 0x7f0044100460 2024-02-19T19:22:37.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f0049347700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0044100d70 0x7f004419b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:37.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f0049347700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0044100d70 0x7f004419b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50112/0 (socket says 172.21.15.88:50112) 2024-02-19T19:22:37.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f0049347700 1 -- 172.21.15.88:0/2424538295 learned_addr learned my addr 172.21.15.88:0/2424538295 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:37.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f0049347700 1 -- 172.21.15.88:0/2424538295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0044100460 msgr2=0x7f004419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:37.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f0049347700 1 --2- 172.21.15.88:0/2424538295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0044100460 0x7f004419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f0049347700 1 -- 172.21.15.88:0/2424538295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00340095f0 con 0x7f0044100d70 2024-02-19T19:22:37.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f0049347700 1 --2- 172.21.15.88:0/2424538295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0044100d70 0x7f004419b000 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0034006010 tx=0x7f003400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:37.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f003affd700 1 -- 172.21.15.88:0/2424538295 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f003401d070 con 0x7f0044100d70 2024-02-19T19:22:37.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.272+0000 7f003affd700 1 -- 172.21.15.88:0/2424538295 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0034004fb0 con 0x7f0044100d70 2024-02-19T19:22:37.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.273+0000 7f003affd700 1 -- 172.21.15.88:0/2424538295 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0034022ea0 con 0x7f0044100d70 2024-02-19T19:22:37.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.273+0000 7f004bdac700 1 -- 172.21.15.88:0/2424538295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0044194dc0 con 0x7f0044100d70 2024-02-19T19:22:37.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.273+0000 7f004bdac700 1 -- 172.21.15.88:0/2424538295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00441952b0 con 0x7f0044100d70 2024-02-19T19:22:37.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.273+0000 7f004bdac700 1 -- 172.21.15.88:0/2424538295 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0044106bd0 con 0x7f0044100d70 2024-02-19T19:22:37.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.274+0000 7f003affd700 1 -- 172.21.15.88:0/2424538295 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0034022470 con 0x7f0044100d70 2024-02-19T19:22:37.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.274+0000 7f003affd700 1 --2- 172.21.15.88:0/2424538295 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f003006d8a0 0x7f003006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:37.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.274+0000 7f003affd700 1 -- 172.21.15.88:0/2424538295 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f003408ec30 con 0x7f0044100d70 2024-02-19T19:22:37.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.274+0000 7f0049b48700 1 --2- 172.21.15.88:0/2424538295 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f003006d8a0 0x7f003006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:37.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.275+0000 7f0049b48700 1 --2- 172.21.15.88:0/2424538295 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f003006d8a0 0x7f003006fd50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f004000a990 tx=0x7f0040005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:37.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.279+0000 7f003affd700 1 -- 172.21.15.88:0/2424538295 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f003405c160 con 0x7f0044100d70 2024-02-19T19:22:37.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.495+0000 7f004bdac700 1 -- 172.21.15.88:0/2424538295 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0044195520 con 0x7f003006d8a0 2024-02-19T19:22:37.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.496+0000 7f003affd700 1 -- 172.21.15.88:0/2424538295 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0044195520 con 0x7f003006d8a0 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:22:37.506 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:22:37.507 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:22:37.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.498+0000 7f004bdac700 1 -- 172.21.15.88:0/2424538295 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f003006d8a0 msgr2=0x7f003006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:37.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.498+0000 7f004bdac700 1 --2- 172.21.15.88:0/2424538295 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f003006d8a0 0x7f003006fd50 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f004000a990 tx=0x7f0040005c80 comp rx=0 tx=0).stop 2024-02-19T19:22:37.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.498+0000 7f004bdac700 1 -- 172.21.15.88:0/2424538295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0044100d70 msgr2=0x7f004419b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:37.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.498+0000 7f004bdac700 1 --2- 172.21.15.88:0/2424538295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0044100d70 0x7f004419b000 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0034006010 tx=0x7f003400be20 comp rx=0 tx=0).stop 2024-02-19T19:22:37.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.498+0000 7f004bdac700 1 -- 172.21.15.88:0/2424538295 shutdown_connections 2024-02-19T19:22:37.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.498+0000 7f004bdac700 1 --2- 172.21.15.88:0/2424538295 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f003006d8a0 0x7f003006fd50 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.498+0000 7f004bdac700 1 --2- 172.21.15.88:0/2424538295 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0044100460 0x7f004419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.498+0000 7f004bdac700 1 --2- 172.21.15.88:0/2424538295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0044100d70 0x7f004419b000 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.498+0000 7f004bdac700 1 -- 172.21.15.88:0/2424538295 >> 172.21.15.88:0/2424538295 conn(0x7f00440748b0 msgr2=0x7f0044103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:37.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.498+0000 7f004bdac700 1 -- 172.21.15.88:0/2424538295 shutdown_connections 2024-02-19T19:22:37.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.498+0000 7f004bdac700 1 -- 172.21.15.88:0/2424538295 wait complete. 2024-02-19T19:22:37.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.601+0000 7f4f72943700 1 -- 172.21.15.88:0/380156503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f6c104130 msgr2=0x7f4f6c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.601+0000 7f4f72943700 1 --2- 172.21.15.88:0/380156503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f6c104130 0x7f4f6c104500 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009910 tx=0x7f4f5c009c20 comp rx=0 tx=0).stop 2024-02-19T19:22:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.601+0000 7f4f72943700 1 -- 172.21.15.88:0/380156503 shutdown_connections 2024-02-19T19:22:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.601+0000 7f4f72943700 1 --2- 172.21.15.88:0/380156503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f6c0ffbd0 0x7f4f6c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.601+0000 7f4f72943700 1 --2- 172.21.15.88:0/380156503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f6c104130 0x7f4f6c104500 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.601+0000 7f4f72943700 1 -- 172.21.15.88:0/380156503 >> 172.21.15.88:0/380156503 conn(0x7f4f6c075130 msgr2=0x7f4f6c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.601+0000 7f4f72943700 1 -- 172.21.15.88:0/380156503 shutdown_connections 2024-02-19T19:22:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.601+0000 7f4f72943700 1 -- 172.21.15.88:0/380156503 wait complete. 2024-02-19T19:22:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.602+0000 7f4f72943700 1 Processor -- start 2024-02-19T19:22:37.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.602+0000 7f4f72943700 1 -- start start 2024-02-19T19:22:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.602+0000 7f4f72943700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f6c0ffbd0 0x7f4f6c196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.602+0000 7f4f72943700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f6c104130 0x7f4f6c1971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.602+0000 7f4f72943700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c197780 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.602+0000 7f4f72943700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c1978c0 con 0x7f4f6c104130 2024-02-19T19:22:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.602+0000 7f4f6bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f6c0ffbd0 0x7f4f6c196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.602+0000 7f4f6bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f6c0ffbd0 0x7f4f6c196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50138/0 (socket says 172.21.15.88:50138) 2024-02-19T19:22:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.602+0000 7f4f6bfff700 1 -- 172.21.15.88:0/286483387 learned_addr learned my addr 172.21.15.88:0/286483387 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:22:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.602+0000 7f4f6b7fe700 1 --2- 172.21.15.88:0/286483387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f6c104130 0x7f4f6c1971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.603+0000 7f4f6bfff700 1 -- 172.21.15.88:0/286483387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f6c104130 msgr2=0x7f4f6c1971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.603+0000 7f4f6bfff700 1 --2- 172.21.15.88:0/286483387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f6c104130 0x7f4f6c1971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.603+0000 7f4f6bfff700 1 -- 172.21.15.88:0/286483387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5c0095f0 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.603+0000 7f4f6bfff700 1 --2- 172.21.15.88:0/286483387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f6c0ffbd0 0x7f4f6c196cb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c006010 tx=0x7f4f5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.603+0000 7f4f697fa700 1 -- 172.21.15.88:0/286483387 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f5c01d070 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.603+0000 7f4f72943700 1 -- 172.21.15.88:0/286483387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c19fd20 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.603+0000 7f4f697fa700 1 -- 172.21.15.88:0/286483387 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f5c00f460 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.603+0000 7f4f697fa700 1 -- 172.21.15.88:0/286483387 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f5c0175f0 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.603+0000 7f4f72943700 1 -- 172.21.15.88:0/286483387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c1a00d0 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.604+0000 7f4f72943700 1 -- 172.21.15.88:0/286483387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c1076a0 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.605+0000 7f4f697fa700 1 -- 172.21.15.88:0/286483387 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4f5c00f5d0 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.605+0000 7f4f697fa700 1 --2- 172.21.15.88:0/286483387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5406d8a0 0x7f4f5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:22:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.605+0000 7f4f697fa700 1 -- 172.21.15.88:0/286483387 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4f5c08ea70 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.605+0000 7f4f6b7fe700 1 --2- 172.21.15.88:0/286483387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5406d8a0 0x7f4f5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:22:37.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.605+0000 7f4f6b7fe700 1 --2- 172.21.15.88:0/286483387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5406d8a0 0x7f4f5406fd50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c068df0 tx=0x7f4f6000a380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:22:37.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.611+0000 7f4f697fa700 1 -- 172.21.15.88:0/286483387 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4f5c058220 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:37 smithi132 ceph-mon[70676]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:37.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:37 smithi132 ceph-mon[70676]: from='client.35062 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:37.742 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:37 smithi132 ceph-mon[70676]: pgmap v570: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:37.742 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:37 smithi132 ceph-mon[70676]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:37.742 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:37 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4097062685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:22:37.742 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:37 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3400387443' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:22:37.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:37 smithi088 ceph-mon[88643]: from='client.35058 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:37.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:37 smithi088 ceph-mon[88643]: from='client.35062 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:37.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:37 smithi088 ceph-mon[88643]: pgmap v570: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:37.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:37 smithi088 ceph-mon[88643]: from='client.35066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:37.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:37 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4097062685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:22:37.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:37 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3400387443' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:22:37.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.893+0000 7f4f72943700 1 -- 172.21.15.88:0/286483387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4f6c04f2a0 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.894+0000 7f4f697fa700 1 -- 172.21.15.88:0/286483387 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f4f5c05b8e0 con 0x7f4f6c0ffbd0 2024-02-19T19:22:37.903 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:22:37.903 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:22:37.903 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:22:37.903 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:22:37.903 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:22:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.896+0000 7f4f72943700 1 -- 172.21.15.88:0/286483387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5406d8a0 msgr2=0x7f4f5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.896+0000 7f4f72943700 1 --2- 172.21.15.88:0/286483387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5406d8a0 0x7f4f5406fd50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c068df0 tx=0x7f4f6000a380 comp rx=0 tx=0).stop 2024-02-19T19:22:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.896+0000 7f4f72943700 1 -- 172.21.15.88:0/286483387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f6c0ffbd0 msgr2=0x7f4f6c196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:22:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.896+0000 7f4f72943700 1 --2- 172.21.15.88:0/286483387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f6c0ffbd0 0x7f4f6c196cb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c006010 tx=0x7f4f5c00bd70 comp rx=0 tx=0).stop 2024-02-19T19:22:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.896+0000 7f4f72943700 1 -- 172.21.15.88:0/286483387 shutdown_connections 2024-02-19T19:22:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.896+0000 7f4f72943700 1 --2- 172.21.15.88:0/286483387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4f5406d8a0 0x7f4f5406fd50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.896+0000 7f4f72943700 1 --2- 172.21.15.88:0/286483387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f6c0ffbd0 0x7f4f6c196cb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.896+0000 7f4f72943700 1 --2- 172.21.15.88:0/286483387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4f6c104130 0x7f4f6c1971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:22:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.896+0000 7f4f72943700 1 -- 172.21.15.88:0/286483387 >> 172.21.15.88:0/286483387 conn(0x7f4f6c075130 msgr2=0x7f4f6c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:22:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.896+0000 7f4f72943700 1 -- 172.21.15.88:0/286483387 shutdown_connections 2024-02-19T19:22:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:22:37.896+0000 7f4f72943700 1 -- 172.21.15.88:0/286483387 wait complete. 2024-02-19T19:22:38.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:38 smithi132 ceph-mon[70676]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:38.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:38 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/286483387' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:22:38.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:38 smithi088 ceph-mon[88643]: from='client.35078 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:22:38.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:38 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/286483387' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:22:39.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:39 smithi132 ceph-mon[70676]: pgmap v571: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:39.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:39 smithi088 ceph-mon[88643]: pgmap v571: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:41.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:41 smithi132 ceph-mon[70676]: pgmap v572: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:41.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:41 smithi088 ceph-mon[88643]: pgmap v572: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:43.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:43 smithi132 ceph-mon[70676]: pgmap v573: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:43.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:22:43.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:43 smithi088 ceph-mon[88643]: pgmap v573: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:43.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:22:45.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:45 smithi132 ceph-mon[70676]: pgmap v574: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:45.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:45 smithi088 ceph-mon[88643]: pgmap v574: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:47.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:47 smithi132 ceph-mon[70676]: pgmap v575: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:47.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:47 smithi088 ceph-mon[88643]: pgmap v575: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:49.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:49 smithi132 ceph-mon[70676]: pgmap v576: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:49.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:49 smithi088 ceph-mon[88643]: pgmap v576: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:51.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:51 smithi132 ceph-mon[70676]: pgmap v577: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:51.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:51 smithi088 ceph-mon[88643]: pgmap v577: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:53.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:53 smithi132 ceph-mon[70676]: pgmap v578: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:53.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:53 smithi088 ceph-mon[88643]: pgmap v578: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:55.735 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:55 smithi088 ceph-mon[88643]: pgmap v579: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:55.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:55 smithi132 ceph-mon[70676]: pgmap v579: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:57.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:57 smithi132 ceph-mon[70676]: pgmap v580: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:57.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:57 smithi088 ceph-mon[88643]: pgmap v580: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:58.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:22:58.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:22:59.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:22:59 smithi088 ceph-mon[88643]: pgmap v581: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:22:59.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:22:59 smithi132 ceph-mon[70676]: pgmap v581: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:01.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:01 smithi088 ceph-mon[88643]: pgmap v582: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:01.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:01 smithi132 ceph-mon[70676]: pgmap v582: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:03.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:03 smithi088 ceph-mon[88643]: pgmap v583: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:03.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:03 smithi132 ceph-mon[70676]: pgmap v583: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:05.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:05 smithi088 ceph-mon[88643]: pgmap v584: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:05.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:05 smithi132 ceph-mon[70676]: pgmap v584: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:07.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:07 smithi088 ceph-mon[88643]: pgmap v585: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:07.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:07 smithi132 ceph-mon[70676]: pgmap v585: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:08.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.009+0000 7fdf72723700 1 -- 172.21.15.88:0/161146859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c0ffc60 msgr2=0x7fdf6c10a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.009+0000 7fdf72723700 1 --2- 172.21.15.88:0/161146859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c0ffc60 0x7fdf6c10a6d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009910 tx=0x7fdf60009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:08.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.009+0000 7fdf72723700 1 -- 172.21.15.88:0/161146859 shutdown_connections 2024-02-19T19:23:08.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.009+0000 7fdf72723700 1 --2- 172.21.15.88:0/161146859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c0ffc60 0x7fdf6c10a6d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.009+0000 7fdf72723700 1 --2- 172.21.15.88:0/161146859 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf6c0ff350 0x7fdf6c0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.009+0000 7fdf72723700 1 -- 172.21.15.88:0/161146859 >> 172.21.15.88:0/161146859 conn(0x7fdf6c0748b0 msgr2=0x7fdf6c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:08.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.009+0000 7fdf72723700 1 -- 172.21.15.88:0/161146859 shutdown_connections 2024-02-19T19:23:08.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.009+0000 7fdf72723700 1 -- 172.21.15.88:0/161146859 wait complete. 2024-02-19T19:23:08.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.009+0000 7fdf72723700 1 Processor -- start 2024-02-19T19:23:08.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.010+0000 7fdf72723700 1 -- start start 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.010+0000 7fdf72723700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf6c0ff350 0x7fdf6c100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.010+0000 7fdf72723700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c100a70 0x7fdf6c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.010+0000 7fdf72723700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c1a23e0 con 0x7fdf6c100a70 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.010+0000 7fdf72723700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf6c1a2520 con 0x7fdf6c0ff350 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.010+0000 7fdf6b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c100a70 0x7fdf6c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.010+0000 7fdf6b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c100a70 0x7fdf6c100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38080/0 (socket says 172.21.15.88:38080) 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.010+0000 7fdf6b7fe700 1 -- 172.21.15.88:0/197940492 learned_addr learned my addr 172.21.15.88:0/197940492 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.010+0000 7fdf6b7fe700 1 -- 172.21.15.88:0/197940492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf6c0ff350 msgr2=0x7fdf6c100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.010+0000 7fdf6b7fe700 1 --2- 172.21.15.88:0/197940492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf6c0ff350 0x7fdf6c100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.010+0000 7fdf6b7fe700 1 -- 172.21.15.88:0/197940492 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600095f0 con 0x7fdf6c100a70 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.011+0000 7fdf6b7fe700 1 --2- 172.21.15.88:0/197940492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c100a70 0x7fdf6c100ee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdf60015000 tx=0x7fdf6000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.011+0000 7fdf697fa700 1 -- 172.21.15.88:0/197940492 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf6001d070 con 0x7fdf6c100a70 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.011+0000 7fdf72723700 1 -- 172.21.15.88:0/197940492 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf6c1a2660 con 0x7fdf6c100a70 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.011+0000 7fdf697fa700 1 -- 172.21.15.88:0/197940492 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf6000f460 con 0x7fdf6c100a70 2024-02-19T19:23:08.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.011+0000 7fdf697fa700 1 -- 172.21.15.88:0/197940492 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf60017610 con 0x7fdf6c100a70 2024-02-19T19:23:08.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.011+0000 7fdf72723700 1 -- 172.21.15.88:0/197940492 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf6c1a2b50 con 0x7fdf6c100a70 2024-02-19T19:23:08.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.012+0000 7fdf72723700 1 -- 172.21.15.88:0/197940492 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf6c107dd0 con 0x7fdf6c100a70 2024-02-19T19:23:08.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.012+0000 7fdf697fa700 1 -- 172.21.15.88:0/197940492 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdf6000fa50 con 0x7fdf6c100a70 2024-02-19T19:23:08.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.013+0000 7fdf697fa700 1 --2- 172.21.15.88:0/197940492 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5406d8a0 0x7fdf5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:08.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.013+0000 7fdf6bfff700 1 --2- 172.21.15.88:0/197940492 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5406d8a0 0x7fdf5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:08.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.013+0000 7fdf697fa700 1 -- 172.21.15.88:0/197940492 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdf6008e930 con 0x7fdf6c100a70 2024-02-19T19:23:08.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.013+0000 7fdf6bfff700 1 --2- 172.21.15.88:0/197940492 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5406d8a0 0x7fdf5406fd50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0773d0 tx=0x7fdf5c006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:08.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.018+0000 7fdf697fa700 1 -- 172.21.15.88:0/197940492 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf600580e0 con 0x7fdf6c100a70 2024-02-19T19:23:08.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.232+0000 7fdf72723700 1 -- 172.21.15.88:0/197940492 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf6c061190 con 0x7fdf5406d8a0 2024-02-19T19:23:08.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.234+0000 7fdf697fa700 1 -- 172.21.15.88:0/197940492 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdf6c061190 con 0x7fdf5406d8a0 2024-02-19T19:23:08.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.236+0000 7fdf72723700 1 -- 172.21.15.88:0/197940492 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5406d8a0 msgr2=0x7fdf5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.236+0000 7fdf72723700 1 --2- 172.21.15.88:0/197940492 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5406d8a0 0x7fdf5406fd50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c0773d0 tx=0x7fdf5c006d20 comp rx=0 tx=0).stop 2024-02-19T19:23:08.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.236+0000 7fdf72723700 1 -- 172.21.15.88:0/197940492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c100a70 msgr2=0x7fdf6c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.236+0000 7fdf72723700 1 --2- 172.21.15.88:0/197940492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c100a70 0x7fdf6c100ee0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdf60015000 tx=0x7fdf6000be20 comp rx=0 tx=0).stop 2024-02-19T19:23:08.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.236+0000 7fdf72723700 1 -- 172.21.15.88:0/197940492 shutdown_connections 2024-02-19T19:23:08.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.236+0000 7fdf72723700 1 --2- 172.21.15.88:0/197940492 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5406d8a0 0x7fdf5406fd50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.236+0000 7fdf72723700 1 --2- 172.21.15.88:0/197940492 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf6c0ff350 0x7fdf6c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.236+0000 7fdf72723700 1 --2- 172.21.15.88:0/197940492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf6c100a70 0x7fdf6c100ee0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.236+0000 7fdf72723700 1 -- 172.21.15.88:0/197940492 >> 172.21.15.88:0/197940492 conn(0x7fdf6c0748b0 msgr2=0x7fdf6c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:08.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.236+0000 7fdf72723700 1 -- 172.21.15.88:0/197940492 shutdown_connections 2024-02-19T19:23:08.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.236+0000 7fdf72723700 1 -- 172.21.15.88:0/197940492 wait complete. 2024-02-19T19:23:08.254 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:23:08.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.341+0000 7f33279b4700 1 -- 172.21.15.88:0/2880052783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33201004e0 msgr2=0x7f33201008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.341+0000 7f33279b4700 1 --2- 172.21.15.88:0/2880052783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33201004e0 0x7f33201008b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f331c009910 tx=0x7f331c009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:08.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.341+0000 7f33279b4700 1 -- 172.21.15.88:0/2880052783 shutdown_connections 2024-02-19T19:23:08.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.341+0000 7f33279b4700 1 --2- 172.21.15.88:0/2880052783 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3320100e80 0x7f3320104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.341+0000 7f33279b4700 1 --2- 172.21.15.88:0/2880052783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33201004e0 0x7f33201008b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.341+0000 7f33279b4700 1 -- 172.21.15.88:0/2880052783 >> 172.21.15.88:0/2880052783 conn(0x7f33200fbda0 msgr2=0x7f33200fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:08.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.341+0000 7f33279b4700 1 -- 172.21.15.88:0/2880052783 shutdown_connections 2024-02-19T19:23:08.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.341+0000 7f33279b4700 1 -- 172.21.15.88:0/2880052783 wait complete. 2024-02-19T19:23:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.341+0000 7f33279b4700 1 Processor -- start 2024-02-19T19:23:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.341+0000 7f33279b4700 1 -- start start 2024-02-19T19:23:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f33279b4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33201004e0 0x7f3320195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f33279b4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3320100e80 0x7f33201964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f33279b4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3320196bd0 con 0x7f3320100e80 2024-02-19T19:23:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f33279b4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332019a960 con 0x7f33201004e0 2024-02-19T19:23:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f3324f4f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3320100e80 0x7f33201964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f3324f4f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3320100e80 0x7f33201964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38112/0 (socket says 172.21.15.88:38112) 2024-02-19T19:23:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f3324f4f700 1 -- 172.21.15.88:0/3706038312 learned_addr learned my addr 172.21.15.88:0/3706038312 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:08.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f3324f4f700 1 -- 172.21.15.88:0/3706038312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33201004e0 msgr2=0x7f3320195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f3324f4f700 1 --2- 172.21.15.88:0/3706038312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33201004e0 0x7f3320195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f3324f4f700 1 -- 172.21.15.88:0/3706038312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c0095f0 con 0x7f3320100e80 2024-02-19T19:23:08.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f3324f4f700 1 --2- 172.21.15.88:0/3706038312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3320100e80 0x7f33201964f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f331000b480 tx=0x7f331000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:08.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f33167fc700 1 -- 172.21.15.88:0/3706038312 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33100118c0 con 0x7f3320100e80 2024-02-19T19:23:08.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f33167fc700 1 -- 172.21.15.88:0/3706038312 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3310010380 con 0x7f3320100e80 2024-02-19T19:23:08.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f33279b4700 1 -- 172.21.15.88:0/3706038312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332019ac40 con 0x7f3320100e80 2024-02-19T19:23:08.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.342+0000 7f33167fc700 1 -- 172.21.15.88:0/3706038312 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f331000f4f0 con 0x7f3320100e80 2024-02-19T19:23:08.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.343+0000 7f33279b4700 1 -- 172.21.15.88:0/3706038312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f332019b190 con 0x7f3320100e80 2024-02-19T19:23:08.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.344+0000 7f33279b4700 1 -- 172.21.15.88:0/3706038312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33201087f0 con 0x7f3320100e80 2024-02-19T19:23:08.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.344+0000 7f33167fc700 1 -- 172.21.15.88:0/3706038312 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f33100104f0 con 0x7f3320100e80 2024-02-19T19:23:08.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.344+0000 7f33167fc700 1 --2- 172.21.15.88:0/3706038312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f330c06d8a0 0x7f330c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:08.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.344+0000 7f33167fc700 1 -- 172.21.15.88:0/3706038312 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f331008d0a0 con 0x7f3320100e80 2024-02-19T19:23:08.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.345+0000 7f3325750700 1 --2- 172.21.15.88:0/3706038312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f330c06d8a0 0x7f330c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:08.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.345+0000 7f3325750700 1 --2- 172.21.15.88:0/3706038312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f330c06d8a0 0x7f330c06fd50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f331c00b5c0 tx=0x7f331c005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:08.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.349+0000 7f33167fc700 1 -- 172.21.15.88:0/3706038312 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33100568d0 con 0x7f3320100e80 2024-02-19T19:23:08.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.566+0000 7f33279b4700 1 -- 172.21.15.88:0/3706038312 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f332019b4d0 con 0x7f330c06d8a0 2024-02-19T19:23:08.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.568+0000 7f33167fc700 1 -- 172.21.15.88:0/3706038312 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f332019b4d0 con 0x7f330c06d8a0 2024-02-19T19:23:08.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.570+0000 7f33279b4700 1 -- 172.21.15.88:0/3706038312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f330c06d8a0 msgr2=0x7f330c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.570+0000 7f33279b4700 1 --2- 172.21.15.88:0/3706038312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f330c06d8a0 0x7f330c06fd50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f331c00b5c0 tx=0x7f331c005960 comp rx=0 tx=0).stop 2024-02-19T19:23:08.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.570+0000 7f33279b4700 1 -- 172.21.15.88:0/3706038312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3320100e80 msgr2=0x7f33201964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.570+0000 7f33279b4700 1 --2- 172.21.15.88:0/3706038312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3320100e80 0x7f33201964f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f331000b480 tx=0x7f331000b840 comp rx=0 tx=0).stop 2024-02-19T19:23:08.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.570+0000 7f33279b4700 1 -- 172.21.15.88:0/3706038312 shutdown_connections 2024-02-19T19:23:08.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.570+0000 7f33279b4700 1 --2- 172.21.15.88:0/3706038312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f330c06d8a0 0x7f330c06fd50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.570+0000 7f33279b4700 1 --2- 172.21.15.88:0/3706038312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33201004e0 0x7f3320195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.570+0000 7f33279b4700 1 --2- 172.21.15.88:0/3706038312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3320100e80 0x7f33201964f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.570+0000 7f33279b4700 1 -- 172.21.15.88:0/3706038312 >> 172.21.15.88:0/3706038312 conn(0x7f33200fbda0 msgr2=0x7f33200fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:08.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.570+0000 7f33279b4700 1 -- 172.21.15.88:0/3706038312 shutdown_connections 2024-02-19T19:23:08.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.570+0000 7f33279b4700 1 -- 172.21.15.88:0/3706038312 wait complete. 2024-02-19T19:23:08.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.674+0000 7fcbba313700 1 -- 172.21.15.88:0/201074597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbb40730f0 msgr2=0x7fcbb40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.674+0000 7fcbba313700 1 --2- 172.21.15.88:0/201074597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbb40730f0 0x7fcbb40734c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009910 tx=0x7fcba4009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:08.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.674+0000 7fcbba313700 1 -- 172.21.15.88:0/201074597 shutdown_connections 2024-02-19T19:23:08.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.674+0000 7fcbba313700 1 --2- 172.21.15.88:0/201074597 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbb4073a00 0x7fcbb4110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.674+0000 7fcbba313700 1 --2- 172.21.15.88:0/201074597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbb40730f0 0x7fcbb40734c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.674+0000 7fcbba313700 1 -- 172.21.15.88:0/201074597 >> 172.21.15.88:0/201074597 conn(0x7fcbb40fbda0 msgr2=0x7fcbb40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:08.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.674+0000 7fcbba313700 1 -- 172.21.15.88:0/201074597 shutdown_connections 2024-02-19T19:23:08.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.674+0000 7fcbba313700 1 -- 172.21.15.88:0/201074597 wait complete. 2024-02-19T19:23:08.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.674+0000 7fcbba313700 1 Processor -- start 2024-02-19T19:23:08.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.674+0000 7fcbba313700 1 -- start start 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbba313700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbb40730f0 0x7fcbb419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbba313700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbb4073a00 0x7fcbb419cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbba313700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb419d230 con 0x7fcbb4073a00 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbba313700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb41a0fc0 con 0x7fcbb40730f0 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbb37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbb4073a00 0x7fcbb419cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbb37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbb4073a00 0x7fcbb419cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38130/0 (socket says 172.21.15.88:38130) 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbb37fe700 1 -- 172.21.15.88:0/301100442 learned_addr learned my addr 172.21.15.88:0/301100442 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbb3fff700 1 --2- 172.21.15.88:0/301100442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbb40730f0 0x7fcbb419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbb37fe700 1 -- 172.21.15.88:0/301100442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbb40730f0 msgr2=0x7fcbb419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbb37fe700 1 --2- 172.21.15.88:0/301100442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbb40730f0 0x7fcbb419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbb37fe700 1 -- 172.21.15.88:0/301100442 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba40095f0 con 0x7fcbb4073a00 2024-02-19T19:23:08.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.675+0000 7fcbb37fe700 1 --2- 172.21.15.88:0/301100442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbb4073a00 0x7fcbb419cb50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcba800b4b0 tx=0x7fcba800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:08.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.676+0000 7fcbb17fa700 1 -- 172.21.15.88:0/301100442 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcba8011800 con 0x7fcbb4073a00 2024-02-19T19:23:08.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.676+0000 7fcbb17fa700 1 -- 172.21.15.88:0/301100442 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcba8011e20 con 0x7fcbb4073a00 2024-02-19T19:23:08.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.676+0000 7fcbb17fa700 1 -- 172.21.15.88:0/301100442 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcba800f2e0 con 0x7fcbb4073a00 2024-02-19T19:23:08.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.676+0000 7fcbba313700 1 -- 172.21.15.88:0/301100442 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb41a12a0 con 0x7fcbb4073a00 2024-02-19T19:23:08.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.676+0000 7fcbba313700 1 -- 172.21.15.88:0/301100442 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb41a17f0 con 0x7fcbb4073a00 2024-02-19T19:23:08.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.677+0000 7fcbba313700 1 -- 172.21.15.88:0/301100442 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb410e4d0 con 0x7fcbb4073a00 2024-02-19T19:23:08.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.677+0000 7fcbb17fa700 1 -- 172.21.15.88:0/301100442 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcba8010380 con 0x7fcbb4073a00 2024-02-19T19:23:08.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.678+0000 7fcbb17fa700 1 --2- 172.21.15.88:0/301100442 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb9c06d8a0 0x7fcb9c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:08.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.678+0000 7fcbb17fa700 1 -- 172.21.15.88:0/301100442 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcba808d060 con 0x7fcbb4073a00 2024-02-19T19:23:08.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.678+0000 7fcbb3fff700 1 --2- 172.21.15.88:0/301100442 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb9c06d8a0 0x7fcb9c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:08.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.678+0000 7fcbb3fff700 1 --2- 172.21.15.88:0/301100442 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb9c06d8a0 0x7fcb9c06fd50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcba4006010 tx=0x7fcba401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:08.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.682+0000 7fcbb17fa700 1 -- 172.21.15.88:0/301100442 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcba8056940 con 0x7fcbb4073a00 2024-02-19T19:23:08.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.894+0000 7fcbba313700 1 -- 172.21.15.88:0/301100442 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbb41a1ac0 con 0x7fcb9c06d8a0 2024-02-19T19:23:08.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.905+0000 7fcbb17fa700 1 -- 172.21.15.88:0/301100442 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fcbb41a1ac0 con 0x7fcb9c06d8a0 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (20m) 12m ago 30m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (31m) 12m ago 31m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (13m) 11m ago 29m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (16m) 12m ago 31m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (16m) 11m ago 29m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (19m) 12m ago 30m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (12m) 12m ago 24m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (12m) 12m ago 24m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (11m) 11m ago 24m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (24m) 11m ago 24m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (21m) 12m ago 32m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (21m) 11m ago 29m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:23:08.914 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (18m) 12m ago 32m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:23:08.915 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (17m) 11m ago 28m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:23:08.915 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (21m) 12m ago 31m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:23:08.915 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (20m) 11m ago 29m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:23:08.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (15m) 12m ago 28m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:23:08.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (15m) 12m ago 27m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:23:08.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (14m) 12m ago 27m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:23:08.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (13m) 11m ago 26m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:23:08.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (13m) 11m ago 26m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:23:08.915 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (12m) 11m ago 26m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:23:08.915 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (20m) 12m ago 29m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:23:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.907+0000 7fcbba313700 1 -- 172.21.15.88:0/301100442 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb9c06d8a0 msgr2=0x7fcb9c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.907+0000 7fcbba313700 1 --2- 172.21.15.88:0/301100442 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb9c06d8a0 0x7fcb9c06fd50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcba4006010 tx=0x7fcba401a040 comp rx=0 tx=0).stop 2024-02-19T19:23:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.907+0000 7fcbba313700 1 -- 172.21.15.88:0/301100442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbb4073a00 msgr2=0x7fcbb419cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.907+0000 7fcbba313700 1 --2- 172.21.15.88:0/301100442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbb4073a00 0x7fcbb419cb50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fcba800b4b0 tx=0x7fcba800b870 comp rx=0 tx=0).stop 2024-02-19T19:23:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.908+0000 7fcbba313700 1 -- 172.21.15.88:0/301100442 shutdown_connections 2024-02-19T19:23:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.908+0000 7fcbba313700 1 --2- 172.21.15.88:0/301100442 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcb9c06d8a0 0x7fcb9c06fd50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.908+0000 7fcbba313700 1 --2- 172.21.15.88:0/301100442 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcbb40730f0 0x7fcbb419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.908+0000 7fcbba313700 1 --2- 172.21.15.88:0/301100442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcbb4073a00 0x7fcbb419cb50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.908+0000 7fcbba313700 1 -- 172.21.15.88:0/301100442 >> 172.21.15.88:0/301100442 conn(0x7fcbb40fbda0 msgr2=0x7fcbb4102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.908+0000 7fcbba313700 1 -- 172.21.15.88:0/301100442 shutdown_connections 2024-02-19T19:23:08.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:08.908+0000 7fcbba313700 1 -- 172.21.15.88:0/301100442 wait complete. 2024-02-19T19:23:09.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.011+0000 7fa88cb74700 1 -- 172.21.15.88:0/537512687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880ffbd0 msgr2=0x7fa888100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.011+0000 7fa88cb74700 1 --2- 172.21.15.88:0/537512687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880ffbd0 0x7fa888100040 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa870009910 tx=0x7fa870009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:09.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.011+0000 7fa88cb74700 1 -- 172.21.15.88:0/537512687 shutdown_connections 2024-02-19T19:23:09.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.011+0000 7fa88cb74700 1 --2- 172.21.15.88:0/537512687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880ffbd0 0x7fa888100040 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.011+0000 7fa88cb74700 1 --2- 172.21.15.88:0/537512687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa888104130 0x7fa888104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.011+0000 7fa88cb74700 1 -- 172.21.15.88:0/537512687 >> 172.21.15.88:0/537512687 conn(0x7fa888075130 msgr2=0x7fa888075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:09.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.011+0000 7fa88cb74700 1 -- 172.21.15.88:0/537512687 shutdown_connections 2024-02-19T19:23:09.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.011+0000 7fa88cb74700 1 -- 172.21.15.88:0/537512687 wait complete. 2024-02-19T19:23:09.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.011+0000 7fa88cb74700 1 Processor -- start 2024-02-19T19:23:09.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa88cb74700 1 -- start start 2024-02-19T19:23:09.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa88cb74700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880ffbd0 0x7fa888195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:09.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa88cb74700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa888104130 0x7fa888196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa88cb74700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa888196a40 con 0x7fa8880ffbd0 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa88cb74700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88819a7d0 con 0x7fa888104130 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa88659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880ffbd0 0x7fa888195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa88659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880ffbd0 0x7fa888195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38144/0 (socket says 172.21.15.88:38144) 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa88659c700 1 -- 172.21.15.88:0/1049045391 learned_addr learned my addr 172.21.15.88:0/1049045391 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa87dd9b700 1 --2- 172.21.15.88:0/1049045391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa888104130 0x7fa888196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa88659c700 1 -- 172.21.15.88:0/1049045391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa888104130 msgr2=0x7fa888196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa88659c700 1 --2- 172.21.15.88:0/1049045391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa888104130 0x7fa888196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.012+0000 7fa88659c700 1 -- 172.21.15.88:0/1049045391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8700095f0 con 0x7fa8880ffbd0 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.013+0000 7fa88659c700 1 --2- 172.21.15.88:0/1049045391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880ffbd0 0x7fa888195e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa87800c8a0 tx=0x7fa87800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.013+0000 7fa87ffff700 1 -- 172.21.15.88:0/1049045391 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa87800cea0 con 0x7fa8880ffbd0 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.013+0000 7fa88cb74700 1 -- 172.21.15.88:0/1049045391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88819aab0 con 0x7fa8880ffbd0 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.013+0000 7fa87ffff700 1 -- 172.21.15.88:0/1049045391 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa878004830 con 0x7fa8880ffbd0 2024-02-19T19:23:09.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.013+0000 7fa87ffff700 1 -- 172.21.15.88:0/1049045391 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8780055b0 con 0x7fa8880ffbd0 2024-02-19T19:23:09.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.013+0000 7fa88cb74700 1 -- 172.21.15.88:0/1049045391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88819b000 con 0x7fa8880ffbd0 2024-02-19T19:23:09.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.014+0000 7fa88cb74700 1 -- 172.21.15.88:0/1049045391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8881086a0 con 0x7fa8880ffbd0 2024-02-19T19:23:09.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.014+0000 7fa87ffff700 1 -- 172.21.15.88:0/1049045391 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa878007ad0 con 0x7fa8880ffbd0 2024-02-19T19:23:09.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.015+0000 7fa87ffff700 1 --2- 172.21.15.88:0/1049045391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa87406d8a0 0x7fa87406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:09.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.015+0000 7fa87ffff700 1 -- 172.21.15.88:0/1049045391 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa87808d260 con 0x7fa8880ffbd0 2024-02-19T19:23:09.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.015+0000 7fa87dd9b700 1 --2- 172.21.15.88:0/1049045391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa87406d8a0 0x7fa87406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:09.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.015+0000 7fa87dd9b700 1 --2- 172.21.15.88:0/1049045391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa87406d8a0 0x7fa87406fd50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa87000b5c0 tx=0x7fa870005460 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:09.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.020+0000 7fa87ffff700 1 -- 172.21.15.88:0/1049045391 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa878056a10 con 0x7fa8880ffbd0 2024-02-19T19:23:09.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.295+0000 7fa88cb74700 1 -- 172.21.15.88:0/1049045391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa88819b280 con 0x7fa8880ffbd0 2024-02-19T19:23:09.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.296+0000 7fa87ffff700 1 -- 172.21.15.88:0/1049045391 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fa87805a0d0 con 0x7fa8880ffbd0 2024-02-19T19:23:09.304 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:23:09.304 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:23:09.304 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:23:09.305 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:23:09.306 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:23:09.306 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:23:09.306 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:23:09.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.298+0000 7fa88cb74700 1 -- 172.21.15.88:0/1049045391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa87406d8a0 msgr2=0x7fa87406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.298+0000 7fa88cb74700 1 --2- 172.21.15.88:0/1049045391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa87406d8a0 0x7fa87406fd50 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa87000b5c0 tx=0x7fa870005460 comp rx=0 tx=0).stop 2024-02-19T19:23:09.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.298+0000 7fa88cb74700 1 -- 172.21.15.88:0/1049045391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880ffbd0 msgr2=0x7fa888195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.298+0000 7fa88cb74700 1 --2- 172.21.15.88:0/1049045391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880ffbd0 0x7fa888195e20 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa87800c8a0 tx=0x7fa87800cc60 comp rx=0 tx=0).stop 2024-02-19T19:23:09.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.298+0000 7fa88cb74700 1 -- 172.21.15.88:0/1049045391 shutdown_connections 2024-02-19T19:23:09.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.298+0000 7fa88cb74700 1 --2- 172.21.15.88:0/1049045391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa87406d8a0 0x7fa87406fd50 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.298+0000 7fa88cb74700 1 --2- 172.21.15.88:0/1049045391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8880ffbd0 0x7fa888195e20 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.298+0000 7fa88cb74700 1 --2- 172.21.15.88:0/1049045391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa888104130 0x7fa888196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.298+0000 7fa88cb74700 1 -- 172.21.15.88:0/1049045391 >> 172.21.15.88:0/1049045391 conn(0x7fa888075130 msgr2=0x7fa8880fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:09.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.298+0000 7fa88cb74700 1 -- 172.21.15.88:0/1049045391 shutdown_connections 2024-02-19T19:23:09.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.298+0000 7fa88cb74700 1 -- 172.21.15.88:0/1049045391 wait complete. 2024-02-19T19:23:09.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.401+0000 7f94a6896700 1 -- 172.21.15.88:0/183667173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00ff3c0 msgr2=0x7f94a00ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.401+0000 7f94a6896700 1 --2- 172.21.15.88:0/183667173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00ff3c0 0x7f94a00ff790 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9490009910 tx=0x7f9490009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:09.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.402+0000 7f94a6896700 1 -- 172.21.15.88:0/183667173 shutdown_connections 2024-02-19T19:23:09.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.402+0000 7f94a6896700 1 --2- 172.21.15.88:0/183667173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a00ffd60 0x7f94a0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.402+0000 7f94a6896700 1 --2- 172.21.15.88:0/183667173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00ff3c0 0x7f94a00ff790 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.402+0000 7f94a6896700 1 -- 172.21.15.88:0/183667173 >> 172.21.15.88:0/183667173 conn(0x7f94a00fac80 msgr2=0x7f94a00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:09.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.402+0000 7f94a6896700 1 -- 172.21.15.88:0/183667173 shutdown_connections 2024-02-19T19:23:09.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.402+0000 7f94a6896700 1 -- 172.21.15.88:0/183667173 wait complete. 2024-02-19T19:23:09.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.402+0000 7f94a6896700 1 Processor -- start 2024-02-19T19:23:09.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.402+0000 7f94a6896700 1 -- start start 2024-02-19T19:23:09.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f94a6896700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a00ff3c0 0x7f94a0195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:09.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f94a6896700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00ffd60 0x7f94a01964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:09.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f94a6896700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0196b90 con 0x7f94a00ffd60 2024-02-19T19:23:09.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f94a6896700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a019a920 con 0x7f94a00ff3c0 2024-02-19T19:23:09.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f949f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00ffd60 0x7f94a01964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:09.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f949f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00ffd60 0x7f94a01964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38164/0 (socket says 172.21.15.88:38164) 2024-02-19T19:23:09.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f949f7fe700 1 -- 172.21.15.88:0/3598561095 learned_addr learned my addr 172.21.15.88:0/3598561095 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:09.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f949ffff700 1 --2- 172.21.15.88:0/3598561095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a00ff3c0 0x7f94a0195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:09.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f949f7fe700 1 -- 172.21.15.88:0/3598561095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a00ff3c0 msgr2=0x7f94a0195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f949f7fe700 1 --2- 172.21.15.88:0/3598561095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a00ff3c0 0x7f94a0195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f949f7fe700 1 -- 172.21.15.88:0/3598561095 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94900095f0 con 0x7f94a00ffd60 2024-02-19T19:23:09.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f949f7fe700 1 --2- 172.21.15.88:0/3598561095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00ffd60 0x7f94a01964b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f949400b480 tx=0x7f949400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:09.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f949d7fa700 1 -- 172.21.15.88:0/3598561095 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9494011800 con 0x7f94a00ffd60 2024-02-19T19:23:09.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f949d7fa700 1 -- 172.21.15.88:0/3598561095 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9494011e20 con 0x7f94a00ffd60 2024-02-19T19:23:09.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f94a6896700 1 -- 172.21.15.88:0/3598561095 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a019ac00 con 0x7f94a00ffd60 2024-02-19T19:23:09.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f949d7fa700 1 -- 172.21.15.88:0/3598561095 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9494010ea0 con 0x7f94a00ffd60 2024-02-19T19:23:09.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.403+0000 7f94a6896700 1 -- 172.21.15.88:0/3598561095 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a019b150 con 0x7f94a00ffd60 2024-02-19T19:23:09.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.404+0000 7f94a6896700 1 -- 172.21.15.88:0/3598561095 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a01086e0 con 0x7f94a00ffd60 2024-02-19T19:23:09.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.405+0000 7f949d7fa700 1 -- 172.21.15.88:0/3598561095 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9494010380 con 0x7f94a00ffd60 2024-02-19T19:23:09.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.405+0000 7f949d7fa700 1 --2- 172.21.15.88:0/3598561095 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 0x7f948806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:09.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.405+0000 7f949d7fa700 1 -- 172.21.15.88:0/3598561095 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f949408d000 con 0x7f94a00ffd60 2024-02-19T19:23:09.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.405+0000 7f949ffff700 1 --2- 172.21.15.88:0/3598561095 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 0x7f948806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:09.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.406+0000 7f949ffff700 1 --2- 172.21.15.88:0/3598561095 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 0x7f948806fd50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9490006010 tx=0x7f949001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:09.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.410+0000 7f949d7fa700 1 -- 172.21.15.88:0/3598561095 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f94940567b0 con 0x7f94a00ffd60 2024-02-19T19:23:09.588 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:09 smithi088 ceph-mon[88643]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:09.588 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:09 smithi088 ceph-mon[88643]: pgmap v586: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:09.588 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:09 smithi088 ceph-mon[88643]: from='client.35090 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:09.588 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:09 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1049045391' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:23:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.652+0000 7f94a6896700 1 -- 172.21.15.88:0/3598561095 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f94a019b3d0 con 0x7f94a00ffd60 2024-02-19T19:23:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.652+0000 7f949d7fa700 1 -- 172.21.15.88:0/3598561095 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f9494059e70 con 0x7f94a00ffd60 2024-02-19T19:23:09.661 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:23:09.661 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:23:09.661 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:23:09.661 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:23:09.661 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:23:09.661 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:23:09.661 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:23:09.661 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:23:09.661 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:23:09.661 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:23:09.662 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:23:09.663 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:23:09.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.654+0000 7f94a6896700 1 -- 172.21.15.88:0/3598561095 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 msgr2=0x7f948806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.654+0000 7f94a6896700 1 --2- 172.21.15.88:0/3598561095 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 0x7f948806fd50 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9490006010 tx=0x7f949001a040 comp rx=0 tx=0).stop 2024-02-19T19:23:09.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.654+0000 7f94a6896700 1 -- 172.21.15.88:0/3598561095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00ffd60 msgr2=0x7f94a01964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.654+0000 7f94a6896700 1 --2- 172.21.15.88:0/3598561095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00ffd60 0x7f94a01964b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f949400b480 tx=0x7f949400b790 comp rx=0 tx=0).stop 2024-02-19T19:23:09.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.655+0000 7f94a6896700 1 -- 172.21.15.88:0/3598561095 shutdown_connections 2024-02-19T19:23:09.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.655+0000 7f94a6896700 1 --2- 172.21.15.88:0/3598561095 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 0x7f948806fd50 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.655+0000 7f94a6896700 1 --2- 172.21.15.88:0/3598561095 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a00ff3c0 0x7f94a0195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.655+0000 7f94a6896700 1 --2- 172.21.15.88:0/3598561095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00ffd60 0x7f94a01964b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.655+0000 7f94a6896700 1 -- 172.21.15.88:0/3598561095 >> 172.21.15.88:0/3598561095 conn(0x7f94a00fac80 msgr2=0x7f94a0069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:09.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.655+0000 7f94a6896700 1 -- 172.21.15.88:0/3598561095 shutdown_connections 2024-02-19T19:23:09.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.655+0000 7f94a6896700 1 -- 172.21.15.88:0/3598561095 wait complete. 2024-02-19T19:23:09.665 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:23:09.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.759+0000 7f3099170700 1 -- 172.21.15.88:0/302945887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30940ffbd0 msgr2=0x7f3094100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.759+0000 7f3099170700 1 --2- 172.21.15.88:0/302945887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30940ffbd0 0x7f3094100040 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f307c009910 tx=0x7f307c009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:09.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.759+0000 7f3099170700 1 -- 172.21.15.88:0/302945887 shutdown_connections 2024-02-19T19:23:09.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.759+0000 7f3099170700 1 --2- 172.21.15.88:0/302945887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30940ffbd0 0x7f3094100040 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.759+0000 7f3099170700 1 --2- 172.21.15.88:0/302945887 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3094104130 0x7f3094104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.759+0000 7f3099170700 1 -- 172.21.15.88:0/302945887 >> 172.21.15.88:0/302945887 conn(0x7f3094075130 msgr2=0x7f3094075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:09.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.759+0000 7f3099170700 1 -- 172.21.15.88:0/302945887 shutdown_connections 2024-02-19T19:23:09.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.759+0000 7f3099170700 1 -- 172.21.15.88:0/302945887 wait complete. 2024-02-19T19:23:09.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f3099170700 1 Processor -- start 2024-02-19T19:23:09.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f3099170700 1 -- start start 2024-02-19T19:23:09.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f3099170700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30940ffbd0 0x7f3094194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f3099170700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3094104130 0x7f3094194fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f3099170700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3094069500 con 0x7f3094104130 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f3099170700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3094195510 con 0x7f30940ffbd0 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f308bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3094104130 0x7f3094194fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f308bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3094104130 0x7f3094194fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38182/0 (socket says 172.21.15.88:38182) 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f308bfff700 1 -- 172.21.15.88:0/44373882 learned_addr learned my addr 172.21.15.88:0/44373882 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f308bfff700 1 -- 172.21.15.88:0/44373882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30940ffbd0 msgr2=0x7f3094194a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f308bfff700 1 --2- 172.21.15.88:0/44373882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30940ffbd0 0x7f3094194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f308bfff700 1 -- 172.21.15.88:0/44373882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f307c0095f0 con 0x7f3094104130 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.760+0000 7f308bfff700 1 --2- 172.21.15.88:0/44373882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3094104130 0x7f3094194fd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f307c0092e0 tx=0x7f307c00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.761+0000 7f3090d99700 1 -- 172.21.15.88:0/44373882 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f307c01d070 con 0x7f3094104130 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.761+0000 7f3090d99700 1 -- 172.21.15.88:0/44373882 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f307c00f460 con 0x7f3094104130 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.761+0000 7f3099170700 1 -- 172.21.15.88:0/44373882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3094195790 con 0x7f3094104130 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.761+0000 7f3090d99700 1 -- 172.21.15.88:0/44373882 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f307c017630 con 0x7f3094104130 2024-02-19T19:23:09.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.761+0000 7f3099170700 1 -- 172.21.15.88:0/44373882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30941a2320 con 0x7f3094104130 2024-02-19T19:23:09.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.762+0000 7f3099170700 1 -- 172.21.15.88:0/44373882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f309404f2a0 con 0x7f3094104130 2024-02-19T19:23:09.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.762+0000 7f3090d99700 1 -- 172.21.15.88:0/44373882 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f307c00f5d0 con 0x7f3094104130 2024-02-19T19:23:09.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.763+0000 7f3090d99700 1 --2- 172.21.15.88:0/44373882 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f308006d8a0 0x7f308006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:09.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.763+0000 7f3090d99700 1 -- 172.21.15.88:0/44373882 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f307c08eb20 con 0x7f3094104130 2024-02-19T19:23:09.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.763+0000 7f3092d9d700 1 --2- 172.21.15.88:0/44373882 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f308006d8a0 0x7f308006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:09.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.763+0000 7f3092d9d700 1 --2- 172.21.15.88:0/44373882 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f308006d8a0 0x7f308006fd50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3084005fd0 tx=0x7f3084005d00 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:09.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.767+0000 7f3090d99700 1 -- 172.21.15.88:0/44373882 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f307c05be00 con 0x7f3094104130 2024-02-19T19:23:09.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.982+0000 7f3099170700 1 -- 172.21.15.88:0/44373882 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f30940ff240 con 0x7f308006d8a0 2024-02-19T19:23:09.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:09 smithi132 ceph-mon[70676]: from='client.35086 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:09.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:09 smithi132 ceph-mon[70676]: pgmap v586: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:09.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:09 smithi132 ceph-mon[70676]: from='client.35090 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:09.992 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:09 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1049045391' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:23:09.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.983+0000 7f3090d99700 1 -- 172.21.15.88:0/44373882 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f30940ff240 con 0x7f308006d8a0 2024-02-19T19:23:09.993 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:23:09.993 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:23:09.993 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:23:09.993 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:23:09.993 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:23:09.993 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:23:09.994 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:23:09.994 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:23:09.994 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:23:09.994 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:23:09.994 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:23:09.994 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:23:09.995 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:23:09.995 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:23:09.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.985+0000 7f3099170700 1 -- 172.21.15.88:0/44373882 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f308006d8a0 msgr2=0x7f308006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.986+0000 7f3099170700 1 --2- 172.21.15.88:0/44373882 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f308006d8a0 0x7f308006fd50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3084005fd0 tx=0x7f3084005d00 comp rx=0 tx=0).stop 2024-02-19T19:23:09.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.986+0000 7f3099170700 1 -- 172.21.15.88:0/44373882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3094104130 msgr2=0x7f3094194fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:09.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.986+0000 7f3099170700 1 --2- 172.21.15.88:0/44373882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3094104130 0x7f3094194fd0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f307c0092e0 tx=0x7f307c00be00 comp rx=0 tx=0).stop 2024-02-19T19:23:09.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.986+0000 7f3099170700 1 -- 172.21.15.88:0/44373882 shutdown_connections 2024-02-19T19:23:09.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.986+0000 7f3099170700 1 --2- 172.21.15.88:0/44373882 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f308006d8a0 0x7f308006fd50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.986+0000 7f3099170700 1 --2- 172.21.15.88:0/44373882 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30940ffbd0 0x7f3094194a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.986+0000 7f3099170700 1 --2- 172.21.15.88:0/44373882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3094104130 0x7f3094194fd0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:09.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.986+0000 7f3099170700 1 -- 172.21.15.88:0/44373882 >> 172.21.15.88:0/44373882 conn(0x7f3094075130 msgr2=0x7f30940feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:09.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.986+0000 7f3099170700 1 -- 172.21.15.88:0/44373882 shutdown_connections 2024-02-19T19:23:09.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:09.986+0000 7f3099170700 1 -- 172.21.15.88:0/44373882 wait complete. 2024-02-19T19:23:10.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.089+0000 7fa1a100e700 1 -- 172.21.15.88:0/2838485915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19c0ff350 msgr2=0x7fa19c1063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:10.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.089+0000 7fa1a100e700 1 --2- 172.21.15.88:0/2838485915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19c0ff350 0x7fa19c1063b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009910 tx=0x7fa18c009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:10.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.089+0000 7fa1a100e700 1 -- 172.21.15.88:0/2838485915 shutdown_connections 2024-02-19T19:23:10.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.089+0000 7fa1a100e700 1 --2- 172.21.15.88:0/2838485915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19c0ff720 0x7fa19c0ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:10.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.089+0000 7fa1a100e700 1 --2- 172.21.15.88:0/2838485915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19c0ff350 0x7fa19c1063b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:10.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.089+0000 7fa1a100e700 1 -- 172.21.15.88:0/2838485915 >> 172.21.15.88:0/2838485915 conn(0x7fa19c0748b0 msgr2=0x7fa19c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:10.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.089+0000 7fa1a100e700 1 -- 172.21.15.88:0/2838485915 shutdown_connections 2024-02-19T19:23:10.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.089+0000 7fa1a100e700 1 -- 172.21.15.88:0/2838485915 wait complete. 2024-02-19T19:23:10.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.090+0000 7fa1a100e700 1 Processor -- start 2024-02-19T19:23:10.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.090+0000 7fa1a100e700 1 -- start start 2024-02-19T19:23:10.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa1a100e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19c0ff350 0x7fa19c19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:10.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa1a100e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19c0ff720 0x7fa19c19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:10.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa1a100e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c19d030 con 0x7fa19c0ff720 2024-02-19T19:23:10.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa1a100e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19c1a0dc0 con 0x7fa19c0ff350 2024-02-19T19:23:10.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa19a59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19c0ff720 0x7fa19c19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:10.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa19a59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19c0ff720 0x7fa19c19c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38204/0 (socket says 172.21.15.88:38204) 2024-02-19T19:23:10.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa19a59c700 1 -- 172.21.15.88:0/177860477 learned_addr learned my addr 172.21.15.88:0/177860477 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:10.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa19ad9d700 1 --2- 172.21.15.88:0/177860477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19c0ff350 0x7fa19c19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:10.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa19a59c700 1 -- 172.21.15.88:0/177860477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19c0ff350 msgr2=0x7fa19c19c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:10.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa19a59c700 1 --2- 172.21.15.88:0/177860477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19c0ff350 0x7fa19c19c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:10.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa19a59c700 1 -- 172.21.15.88:0/177860477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa18c0095f0 con 0x7fa19c0ff720 2024-02-19T19:23:10.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa19a59c700 1 --2- 172.21.15.88:0/177860477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19c0ff720 0x7fa19c19c950 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa19000d8d0 tx=0x7fa19000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:10.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa183fff700 1 -- 172.21.15.88:0/177860477 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa190009940 con 0x7fa19c0ff720 2024-02-19T19:23:10.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa183fff700 1 -- 172.21.15.88:0/177860477 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa190010460 con 0x7fa19c0ff720 2024-02-19T19:23:10.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa1a100e700 1 -- 172.21.15.88:0/177860477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19c1a10a0 con 0x7fa19c0ff720 2024-02-19T19:23:10.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa183fff700 1 -- 172.21.15.88:0/177860477 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa19000f5f0 con 0x7fa19c0ff720 2024-02-19T19:23:10.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.091+0000 7fa1a100e700 1 -- 172.21.15.88:0/177860477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19c1a15f0 con 0x7fa19c0ff720 2024-02-19T19:23:10.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.092+0000 7fa1a100e700 1 -- 172.21.15.88:0/177860477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19c103ae0 con 0x7fa19c0ff720 2024-02-19T19:23:10.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.093+0000 7fa183fff700 1 -- 172.21.15.88:0/177860477 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa1900105d0 con 0x7fa19c0ff720 2024-02-19T19:23:10.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.093+0000 7fa183fff700 1 --2- 172.21.15.88:0/177860477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa18406d8a0 0x7fa18406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:10.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.093+0000 7fa183fff700 1 -- 172.21.15.88:0/177860477 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa19008d540 con 0x7fa19c0ff720 2024-02-19T19:23:10.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.093+0000 7fa19ad9d700 1 --2- 172.21.15.88:0/177860477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa18406d8a0 0x7fa18406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:10.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.094+0000 7fa19ad9d700 1 --2- 172.21.15.88:0/177860477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa18406d8a0 0x7fa18406fd50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00b5c0 tx=0x7fa18c005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:10.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.098+0000 7fa183fff700 1 -- 172.21.15.88:0/177860477 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa190056cf0 con 0x7fa19c0ff720 2024-02-19T19:23:10.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.377+0000 7fa1a100e700 1 -- 172.21.15.88:0/177860477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa19c04f2a0 con 0x7fa19c0ff720 2024-02-19T19:23:10.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.378+0000 7fa183fff700 1 -- 172.21.15.88:0/177860477 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fa19005a3b0 con 0x7fa19c0ff720 2024-02-19T19:23:10.386 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:23:10.386 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:23:10.386 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:23:10.386 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:23:10.386 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:23:10.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.380+0000 7fa1a100e700 1 -- 172.21.15.88:0/177860477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa18406d8a0 msgr2=0x7fa18406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:10.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.380+0000 7fa1a100e700 1 --2- 172.21.15.88:0/177860477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa18406d8a0 0x7fa18406fd50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa18c00b5c0 tx=0x7fa18c005960 comp rx=0 tx=0).stop 2024-02-19T19:23:10.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.380+0000 7fa1a100e700 1 -- 172.21.15.88:0/177860477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19c0ff720 msgr2=0x7fa19c19c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:10.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.380+0000 7fa1a100e700 1 --2- 172.21.15.88:0/177860477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19c0ff720 0x7fa19c19c950 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fa19000d8d0 tx=0x7fa19000dc90 comp rx=0 tx=0).stop 2024-02-19T19:23:10.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.380+0000 7fa1a100e700 1 -- 172.21.15.88:0/177860477 shutdown_connections 2024-02-19T19:23:10.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.380+0000 7fa1a100e700 1 --2- 172.21.15.88:0/177860477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa18406d8a0 0x7fa18406fd50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:10.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.380+0000 7fa1a100e700 1 --2- 172.21.15.88:0/177860477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa19c0ff350 0x7fa19c19c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:10.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.380+0000 7fa1a100e700 1 --2- 172.21.15.88:0/177860477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19c0ff720 0x7fa19c19c950 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:10.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.380+0000 7fa1a100e700 1 -- 172.21.15.88:0/177860477 >> 172.21.15.88:0/177860477 conn(0x7fa19c0748b0 msgr2=0x7fa19c0fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:10.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.380+0000 7fa1a100e700 1 -- 172.21.15.88:0/177860477 shutdown_connections 2024-02-19T19:23:10.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:10.380+0000 7fa1a100e700 1 -- 172.21.15.88:0/177860477 wait complete. 2024-02-19T19:23:10.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:10 smithi088 ceph-mon[88643]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:10.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:10 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3598561095' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:23:10.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:10 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/177860477' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:23:10.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:10 smithi132 ceph-mon[70676]: from='client.35094 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:10.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:10 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3598561095' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:23:10.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:10 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/177860477' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:23:11.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:11 smithi088 ceph-mon[88643]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:11.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:11 smithi088 ceph-mon[88643]: pgmap v587: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:11.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:11 smithi132 ceph-mon[70676]: from='client.35106 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:11.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:11 smithi132 ceph-mon[70676]: pgmap v587: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:13.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:13 smithi088 ceph-mon[88643]: pgmap v588: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:13.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:23:13.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:13 smithi132 ceph-mon[70676]: pgmap v588: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:13.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:23:15.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:15 smithi088 ceph-mon[88643]: pgmap v589: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:15.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:15 smithi132 ceph-mon[70676]: pgmap v589: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:17.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:17 smithi088 ceph-mon[88643]: pgmap v590: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:17.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:17 smithi132 ceph-mon[70676]: pgmap v590: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:19.840 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:19 smithi088 ceph-mon[88643]: pgmap v591: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:19.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:19 smithi132 ceph-mon[70676]: pgmap v591: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:21.338 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:21 smithi088 ceph-mon[88643]: pgmap v592: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:21.490 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:21 smithi132 ceph-mon[70676]: pgmap v592: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:23.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:23 smithi132 ceph-mon[70676]: pgmap v593: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:23.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:23 smithi088 ceph-mon[88643]: pgmap v593: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:25.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:25 smithi132 ceph-mon[70676]: pgmap v594: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:25.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:25 smithi088 ceph-mon[88643]: pgmap v594: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:27.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:27 smithi132 ceph-mon[70676]: pgmap v595: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:27.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:27 smithi088 ceph-mon[88643]: pgmap v595: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:28.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:23:28.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:23:29.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:29 smithi132 ceph-mon[70676]: pgmap v596: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:29.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:29 smithi088 ceph-mon[88643]: pgmap v596: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:31.490 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:31 smithi132 ceph-mon[70676]: pgmap v597: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:31.588 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:31 smithi088 ceph-mon[88643]: pgmap v597: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:33.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:33 smithi132 ceph-mon[70676]: pgmap v598: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:33.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:33 smithi088 ceph-mon[88643]: pgmap v598: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:35.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:35 smithi132 ceph-mon[70676]: pgmap v599: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:35.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:35 smithi088 ceph-mon[88643]: pgmap v599: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:37.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:37 smithi132 ceph-mon[70676]: pgmap v600: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:37.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:37 smithi088 ceph-mon[88643]: pgmap v600: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:39.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:39 smithi132 ceph-mon[70676]: pgmap v601: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:39.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:39 smithi088 ceph-mon[88643]: pgmap v601: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:40.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.498+0000 7f70e77ba700 1 -- 172.21.15.88:0/3761341773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e00ffd60 msgr2=0x7f70e0103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:40.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.498+0000 7f70e77ba700 1 --2- 172.21.15.88:0/3761341773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e00ffd60 0x7f70e0103d30 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f70dc009910 tx=0x7f70dc009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:40.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.498+0000 7f70e77ba700 1 -- 172.21.15.88:0/3761341773 shutdown_connections 2024-02-19T19:23:40.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.498+0000 7f70e77ba700 1 --2- 172.21.15.88:0/3761341773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e00ffd60 0x7f70e0103d30 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:40.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.498+0000 7f70e77ba700 1 --2- 172.21.15.88:0/3761341773 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e00ff3c0 0x7f70e00ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:40.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.498+0000 7f70e77ba700 1 -- 172.21.15.88:0/3761341773 >> 172.21.15.88:0/3761341773 conn(0x7f70e00fac60 msgr2=0x7f70e00fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:40.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.498+0000 7f70e77ba700 1 -- 172.21.15.88:0/3761341773 shutdown_connections 2024-02-19T19:23:40.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.498+0000 7f70e77ba700 1 -- 172.21.15.88:0/3761341773 wait complete. 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e77ba700 1 Processor -- start 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e77ba700 1 -- start start 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e77ba700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e00ff3c0 0x7f70e0195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e77ba700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e00ffd60 0x7f70e0196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e77ba700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e0196b10 con 0x7f70e00ff3c0 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e77ba700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e019a8a0 con 0x7f70e00ffd60 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e5556700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e00ff3c0 0x7f70e0195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e5556700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e00ff3c0 0x7f70e0195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49866/0 (socket says 172.21.15.88:49866) 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e5556700 1 -- 172.21.15.88:0/3639615465 learned_addr learned my addr 172.21.15.88:0/3639615465 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e4d55700 1 --2- 172.21.15.88:0/3639615465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e00ffd60 0x7f70e0196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e5556700 1 -- 172.21.15.88:0/3639615465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e00ffd60 msgr2=0x7f70e0196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:40.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e5556700 1 --2- 172.21.15.88:0/3639615465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e00ffd60 0x7f70e0196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:40.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.499+0000 7f70e5556700 1 -- 172.21.15.88:0/3639615465 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0095f0 con 0x7f70e00ff3c0 2024-02-19T19:23:40.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.500+0000 7f70e5556700 1 --2- 172.21.15.88:0/3639615465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e00ff3c0 0x7f70e0195ef0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f70d000c8f0 tx=0x7f70d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:40.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.500+0000 7f70d67fc700 1 -- 172.21.15.88:0/3639615465 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70d0007a70 con 0x7f70e00ff3c0 2024-02-19T19:23:40.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.500+0000 7f70d67fc700 1 -- 172.21.15.88:0/3639615465 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70d0004cf0 con 0x7f70e00ff3c0 2024-02-19T19:23:40.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.500+0000 7f70e77ba700 1 -- 172.21.15.88:0/3639615465 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e019ab80 con 0x7f70e00ff3c0 2024-02-19T19:23:40.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.500+0000 7f70d67fc700 1 -- 172.21.15.88:0/3639615465 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70d0005610 con 0x7f70e00ff3c0 2024-02-19T19:23:40.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.500+0000 7f70e77ba700 1 -- 172.21.15.88:0/3639615465 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e019b0d0 con 0x7f70e00ff3c0 2024-02-19T19:23:40.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.501+0000 7f70e77ba700 1 -- 172.21.15.88:0/3639615465 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e0108640 con 0x7f70e00ff3c0 2024-02-19T19:23:40.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.501+0000 7f70d67fc700 1 -- 172.21.15.88:0/3639615465 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f70d0004750 con 0x7f70e00ff3c0 2024-02-19T19:23:40.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.502+0000 7f70d67fc700 1 --2- 172.21.15.88:0/3639615465 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70cc06d8a0 0x7f70cc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:40.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.502+0000 7f70d67fc700 1 -- 172.21.15.88:0/3639615465 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f70d0013070 con 0x7f70e00ff3c0 2024-02-19T19:23:40.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.502+0000 7f70e4d55700 1 --2- 172.21.15.88:0/3639615465 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70cc06d8a0 0x7f70cc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:40.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.502+0000 7f70e4d55700 1 --2- 172.21.15.88:0/3639615465 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70cc06d8a0 0x7f70cc06fd50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f70e0197510 tx=0x7f70dc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:40.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.507+0000 7f70d67fc700 1 -- 172.21.15.88:0/3639615465 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f70d0056b00 con 0x7f70e00ff3c0 2024-02-19T19:23:40.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.721+0000 7f70e77ba700 1 -- 172.21.15.88:0/3639615465 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70e019b3a0 con 0x7f70cc06d8a0 2024-02-19T19:23:40.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.722+0000 7f70d67fc700 1 -- 172.21.15.88:0/3639615465 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f70e019b3a0 con 0x7f70cc06d8a0 2024-02-19T19:23:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.724+0000 7f70e77ba700 1 -- 172.21.15.88:0/3639615465 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70cc06d8a0 msgr2=0x7f70cc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.724+0000 7f70e77ba700 1 --2- 172.21.15.88:0/3639615465 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70cc06d8a0 0x7f70cc06fd50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f70e0197510 tx=0x7f70dc01a040 comp rx=0 tx=0).stop 2024-02-19T19:23:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.724+0000 7f70e77ba700 1 -- 172.21.15.88:0/3639615465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e00ff3c0 msgr2=0x7f70e0195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.724+0000 7f70e77ba700 1 --2- 172.21.15.88:0/3639615465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e00ff3c0 0x7f70e0195ef0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f70d000c8f0 tx=0x7f70d000ccb0 comp rx=0 tx=0).stop 2024-02-19T19:23:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.724+0000 7f70e77ba700 1 -- 172.21.15.88:0/3639615465 shutdown_connections 2024-02-19T19:23:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.724+0000 7f70e77ba700 1 --2- 172.21.15.88:0/3639615465 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70cc06d8a0 0x7f70cc06fd50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.724+0000 7f70e77ba700 1 --2- 172.21.15.88:0/3639615465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70e00ff3c0 0x7f70e0195ef0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.724+0000 7f70e77ba700 1 --2- 172.21.15.88:0/3639615465 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70e00ffd60 0x7f70e0196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.724+0000 7f70e77ba700 1 -- 172.21.15.88:0/3639615465 >> 172.21.15.88:0/3639615465 conn(0x7f70e00fac60 msgr2=0x7f70e0069660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.724+0000 7f70e77ba700 1 -- 172.21.15.88:0/3639615465 shutdown_connections 2024-02-19T19:23:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.724+0000 7f70e77ba700 1 -- 172.21.15.88:0/3639615465 wait complete. 2024-02-19T19:23:40.743 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:23:40.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.829+0000 7f3c9626a700 1 -- 172.21.15.88:0/1228121183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c901004e0 msgr2=0x7f3c901008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:40.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.829+0000 7f3c9626a700 1 --2- 172.21.15.88:0/1228121183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c901004e0 0x7f3c901008b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3c80009910 tx=0x7f3c80009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:40.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.829+0000 7f3c9626a700 1 -- 172.21.15.88:0/1228121183 shutdown_connections 2024-02-19T19:23:40.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.829+0000 7f3c9626a700 1 --2- 172.21.15.88:0/1228121183 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c90100e80 0x7f3c90104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:40.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.829+0000 7f3c9626a700 1 --2- 172.21.15.88:0/1228121183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c901004e0 0x7f3c901008b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:40.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.829+0000 7f3c9626a700 1 -- 172.21.15.88:0/1228121183 >> 172.21.15.88:0/1228121183 conn(0x7f3c900fbda0 msgr2=0x7f3c900fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:40.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.829+0000 7f3c9626a700 1 -- 172.21.15.88:0/1228121183 shutdown_connections 2024-02-19T19:23:40.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.829+0000 7f3c9626a700 1 -- 172.21.15.88:0/1228121183 wait complete. 2024-02-19T19:23:40.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.830+0000 7f3c9626a700 1 Processor -- start 2024-02-19T19:23:40.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.830+0000 7f3c9626a700 1 -- start start 2024-02-19T19:23:40.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c9626a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c901004e0 0x7f3c90195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:40.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c9626a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c90100e80 0x7f3c901964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:40.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c9626a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c90196bb0 con 0x7f3c90100e80 2024-02-19T19:23:40.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c9626a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9019a940 con 0x7f3c901004e0 2024-02-19T19:23:40.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c8f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c90100e80 0x7f3c901964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:40.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c8f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c90100e80 0x7f3c901964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49872/0 (socket says 172.21.15.88:49872) 2024-02-19T19:23:40.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c8f7fe700 1 -- 172.21.15.88:0/3000536638 learned_addr learned my addr 172.21.15.88:0/3000536638 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:40.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c8f7fe700 1 -- 172.21.15.88:0/3000536638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c901004e0 msgr2=0x7f3c90195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:40.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c8f7fe700 1 --2- 172.21.15.88:0/3000536638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c901004e0 0x7f3c90195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:40.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c8f7fe700 1 -- 172.21.15.88:0/3000536638 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c800095f0 con 0x7f3c90100e80 2024-02-19T19:23:40.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c8f7fe700 1 --2- 172.21.15.88:0/3000536638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c90100e80 0x7f3c901964d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400b480 tx=0x7f3c8400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:40.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c8d7fa700 1 -- 172.21.15.88:0/3000536638 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c840118c0 con 0x7f3c90100e80 2024-02-19T19:23:40.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c8d7fa700 1 -- 172.21.15.88:0/3000536638 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c84010380 con 0x7f3c90100e80 2024-02-19T19:23:40.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c9626a700 1 -- 172.21.15.88:0/3000536638 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9019ac20 con 0x7f3c90100e80 2024-02-19T19:23:40.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c8d7fa700 1 -- 172.21.15.88:0/3000536638 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c8400f4f0 con 0x7f3c90100e80 2024-02-19T19:23:40.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.831+0000 7f3c9626a700 1 -- 172.21.15.88:0/3000536638 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9019b170 con 0x7f3c90100e80 2024-02-19T19:23:40.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.832+0000 7f3c9626a700 1 -- 172.21.15.88:0/3000536638 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c901087f0 con 0x7f3c90100e80 2024-02-19T19:23:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.833+0000 7f3c8d7fa700 1 -- 172.21.15.88:0/3000536638 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3c840104f0 con 0x7f3c90100e80 2024-02-19T19:23:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.833+0000 7f3c8d7fa700 1 --2- 172.21.15.88:0/3000536638 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c7806d8a0 0x7f3c7806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.833+0000 7f3c8d7fa700 1 -- 172.21.15.88:0/3000536638 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3c8408d0a0 con 0x7f3c90100e80 2024-02-19T19:23:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.833+0000 7f3c8ffff700 1 --2- 172.21.15.88:0/3000536638 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c7806d8a0 0x7f3c7806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.834+0000 7f3c8ffff700 1 --2- 172.21.15.88:0/3000536638 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c7806d8a0 0x7f3c7806fd50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000b5c0 tx=0x7f3c80005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:40.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:40.838+0000 7f3c8d7fa700 1 -- 172.21.15.88:0/3000536638 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3c840568d0 con 0x7f3c90100e80 2024-02-19T19:23:41.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.052+0000 7f3c9626a700 1 -- 172.21.15.88:0/3000536638 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c9019b510 con 0x7f3c7806d8a0 2024-02-19T19:23:41.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.053+0000 7f3c8d7fa700 1 -- 172.21.15.88:0/3000536638 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3c9019b510 con 0x7f3c7806d8a0 2024-02-19T19:23:41.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.055+0000 7f3c9626a700 1 -- 172.21.15.88:0/3000536638 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c7806d8a0 msgr2=0x7f3c7806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.055+0000 7f3c9626a700 1 --2- 172.21.15.88:0/3000536638 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c7806d8a0 0x7f3c7806fd50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000b5c0 tx=0x7f3c80005960 comp rx=0 tx=0).stop 2024-02-19T19:23:41.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.055+0000 7f3c9626a700 1 -- 172.21.15.88:0/3000536638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c90100e80 msgr2=0x7f3c901964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.055+0000 7f3c9626a700 1 --2- 172.21.15.88:0/3000536638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c90100e80 0x7f3c901964d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400b480 tx=0x7f3c8400b840 comp rx=0 tx=0).stop 2024-02-19T19:23:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.056+0000 7f3c9626a700 1 -- 172.21.15.88:0/3000536638 shutdown_connections 2024-02-19T19:23:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.056+0000 7f3c9626a700 1 --2- 172.21.15.88:0/3000536638 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c7806d8a0 0x7f3c7806fd50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.056+0000 7f3c9626a700 1 --2- 172.21.15.88:0/3000536638 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3c901004e0 0x7f3c90195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.056+0000 7f3c9626a700 1 --2- 172.21.15.88:0/3000536638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c90100e80 0x7f3c901964d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.056+0000 7f3c9626a700 1 -- 172.21.15.88:0/3000536638 >> 172.21.15.88:0/3000536638 conn(0x7f3c900fbda0 msgr2=0x7f3c900fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.056+0000 7f3c9626a700 1 -- 172.21.15.88:0/3000536638 shutdown_connections 2024-02-19T19:23:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.056+0000 7f3c9626a700 1 -- 172.21.15.88:0/3000536638 wait complete. 2024-02-19T19:23:41.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.159+0000 7f6264b62700 1 -- 172.21.15.88:0/2781231656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6260100e80 msgr2=0x7f6260104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.159+0000 7f6264b62700 1 --2- 172.21.15.88:0/2781231656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6260100e80 0x7f6260104e50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6254009910 tx=0x7f6254009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:41.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.160+0000 7f6264b62700 1 -- 172.21.15.88:0/2781231656 shutdown_connections 2024-02-19T19:23:41.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.160+0000 7f6264b62700 1 --2- 172.21.15.88:0/2781231656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6260100e80 0x7f6260104e50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.160+0000 7f6264b62700 1 --2- 172.21.15.88:0/2781231656 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62601004e0 0x7f62601008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.160+0000 7f6264b62700 1 -- 172.21.15.88:0/2781231656 >> 172.21.15.88:0/2781231656 conn(0x7f62600fbda0 msgr2=0x7f62600fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:41.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.160+0000 7f6264b62700 1 -- 172.21.15.88:0/2781231656 shutdown_connections 2024-02-19T19:23:41.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.160+0000 7f6264b62700 1 -- 172.21.15.88:0/2781231656 wait complete. 2024-02-19T19:23:41.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.160+0000 7f6264b62700 1 Processor -- start 2024-02-19T19:23:41.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.160+0000 7f6264b62700 1 -- start start 2024-02-19T19:23:41.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f6264b62700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62601004e0 0x7f6260196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:41.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f6264b62700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6260100e80 0x7f6260196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:41.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f6264b62700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6260196c20 con 0x7f62601004e0 2024-02-19T19:23:41.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f6264b62700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626019a9b0 con 0x7f6260100e80 2024-02-19T19:23:41.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f625e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62601004e0 0x7f6260196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:41.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f625e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62601004e0 0x7f6260196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49880/0 (socket says 172.21.15.88:49880) 2024-02-19T19:23:41.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f625e59c700 1 -- 172.21.15.88:0/3191296509 learned_addr learned my addr 172.21.15.88:0/3191296509 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:41.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f625dd9b700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6260100e80 0x7f6260196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:41.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f625e59c700 1 -- 172.21.15.88:0/3191296509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6260100e80 msgr2=0x7f6260196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f625e59c700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6260100e80 0x7f6260196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f625e59c700 1 -- 172.21.15.88:0/3191296509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62540095f0 con 0x7f62601004e0 2024-02-19T19:23:41.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f625dd9b700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6260100e80 0x7f6260196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:23:41.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f625e59c700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62601004e0 0x7f6260196000 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f625000b4b0 tx=0x7f625000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:41.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f624f7fe700 1 -- 172.21.15.88:0/3191296509 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6250011800 con 0x7f62601004e0 2024-02-19T19:23:41.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f624f7fe700 1 -- 172.21.15.88:0/3191296509 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6250011e20 con 0x7f62601004e0 2024-02-19T19:23:41.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.161+0000 7f6264b62700 1 -- 172.21.15.88:0/3191296509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626019ac90 con 0x7f62601004e0 2024-02-19T19:23:41.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.162+0000 7f624f7fe700 1 -- 172.21.15.88:0/3191296509 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f625000f2e0 con 0x7f62601004e0 2024-02-19T19:23:41.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.162+0000 7f6264b62700 1 -- 172.21.15.88:0/3191296509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626019b1e0 con 0x7f62601004e0 2024-02-19T19:23:41.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.163+0000 7f6264b62700 1 -- 172.21.15.88:0/3191296509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6260108780 con 0x7f62601004e0 2024-02-19T19:23:41.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.163+0000 7f624f7fe700 1 -- 172.21.15.88:0/3191296509 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6250010380 con 0x7f62601004e0 2024-02-19T19:23:41.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.163+0000 7f624f7fe700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f624806d8a0 0x7f624806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:41.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.164+0000 7f625dd9b700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f624806d8a0 0x7f624806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:41.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.164+0000 7f624f7fe700 1 -- 172.21.15.88:0/3191296509 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f625008d0f0 con 0x7f62601004e0 2024-02-19T19:23:41.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.164+0000 7f625dd9b700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f624806d8a0 0x7f624806fd50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6260197620 tx=0x7f625401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:41.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.168+0000 7f624f7fe700 1 -- 172.21.15.88:0/3191296509 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f62500568a0 con 0x7f62601004e0 2024-02-19T19:23:41.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.402+0000 7f6264b62700 1 -- 172.21.15.88:0/3191296509 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f626019b460 con 0x7f624806d8a0 2024-02-19T19:23:41.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.413+0000 7f624f7fe700 1 -- 172.21.15.88:0/3191296509 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f626019b460 con 0x7f624806d8a0 2024-02-19T19:23:41.421 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:23:41.421 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (20m) 12m ago 31m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:23:41.421 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (31m) 12m ago 31m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:23:41.421 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (13m) 12m ago 29m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:23:41.421 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (16m) 12m ago 31m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (16m) 12m ago 29m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (20m) 12m ago 30m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (13m) 12m ago 25m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (12m) 12m ago 24m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (12m) 12m ago 25m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (24m) 12m ago 24m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (22m) 12m ago 32m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (21m) 12m ago 29m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (18m) 12m ago 33m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (18m) 12m ago 29m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (21m) 12m ago 31m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (21m) 12m ago 29m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (16m) 12m ago 28m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (15m) 12m ago 28m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (15m) 12m ago 27m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:23:41.422 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (14m) 12m ago 27m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:23:41.423 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (13m) 12m ago 26m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:23:41.423 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (13m) 12m ago 26m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:23:41.423 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (21m) 12m ago 30m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:23:41.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.415+0000 7f6264b62700 1 -- 172.21.15.88:0/3191296509 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f624806d8a0 msgr2=0x7f624806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.415+0000 7f6264b62700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f624806d8a0 0x7f624806fd50 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6260197620 tx=0x7f625401a040 comp rx=0 tx=0).stop 2024-02-19T19:23:41.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.415+0000 7f6264b62700 1 -- 172.21.15.88:0/3191296509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62601004e0 msgr2=0x7f6260196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.415+0000 7f6264b62700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62601004e0 0x7f6260196000 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f625000b4b0 tx=0x7f625000b870 comp rx=0 tx=0).stop 2024-02-19T19:23:41.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.415+0000 7f6264b62700 1 -- 172.21.15.88:0/3191296509 shutdown_connections 2024-02-19T19:23:41.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.415+0000 7f6264b62700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f624806d8a0 0x7f624806fd50 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.415+0000 7f6264b62700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62601004e0 0x7f6260196000 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.415+0000 7f6264b62700 1 --2- 172.21.15.88:0/3191296509 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6260100e80 0x7f6260196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.415+0000 7f6264b62700 1 -- 172.21.15.88:0/3191296509 >> 172.21.15.88:0/3191296509 conn(0x7f62600fbda0 msgr2=0x7f62600fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:41.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.415+0000 7f6264b62700 1 -- 172.21.15.88:0/3191296509 shutdown_connections 2024-02-19T19:23:41.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.415+0000 7f6264b62700 1 -- 172.21.15.88:0/3191296509 wait complete. 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.519+0000 7f05b3a64700 1 -- 172.21.15.88:0/2656045926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0ff3c0 msgr2=0x7f05ac0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.519+0000 7f05b3a64700 1 --2- 172.21.15.88:0/2656045926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0ff3c0 0x7f05ac0ff790 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009910 tx=0x7f05a8009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.519+0000 7f05b3a64700 1 -- 172.21.15.88:0/2656045926 shutdown_connections 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.519+0000 7f05b3a64700 1 --2- 172.21.15.88:0/2656045926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0ffd60 0x7f05ac103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.519+0000 7f05b3a64700 1 --2- 172.21.15.88:0/2656045926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0ff3c0 0x7f05ac0ff790 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.519+0000 7f05b3a64700 1 -- 172.21.15.88:0/2656045926 >> 172.21.15.88:0/2656045926 conn(0x7f05ac0fac80 msgr2=0x7f05ac0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.519+0000 7f05b3a64700 1 -- 172.21.15.88:0/2656045926 shutdown_connections 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.519+0000 7f05b3a64700 1 -- 172.21.15.88:0/2656045926 wait complete. 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b3a64700 1 Processor -- start 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b3a64700 1 -- start start 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b3a64700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0ff3c0 0x7f05ac196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b3a64700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0ffd60 0x7f05ac1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b3a64700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1978b0 con 0x7f05ac0ffd60 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b3a64700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1979f0 con 0x7f05ac0ff3c0 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b0fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0ffd60 0x7f05ac1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b0fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0ffd60 0x7f05ac1971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49900/0 (socket says 172.21.15.88:49900) 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b0fff700 1 -- 172.21.15.88:0/2986716245 learned_addr learned my addr 172.21.15.88:0/2986716245 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b1800700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0ff3c0 0x7f05ac196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b0fff700 1 -- 172.21.15.88:0/2986716245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0ff3c0 msgr2=0x7f05ac196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b0fff700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0ff3c0 0x7f05ac196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b0fff700 1 -- 172.21.15.88:0/2986716245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80095f0 con 0x7f05ac0ffd60 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.520+0000 7f05b1800700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0ff3c0 0x7f05ac196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.521+0000 7f05b0fff700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0ffd60 0x7f05ac1971d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f059c00d900 tx=0x7f059c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.521+0000 7f05a27fc700 1 -- 172.21.15.88:0/2986716245 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f059c009900 con 0x7f05ac0ffd60 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.521+0000 7f05a27fc700 1 -- 172.21.15.88:0/2986716245 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f059c010460 con 0x7f05ac0ffd60 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.521+0000 7f05a27fc700 1 -- 172.21.15.88:0/2986716245 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f059c00f5f0 con 0x7f05ac0ffd60 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.521+0000 7f05b3a64700 1 -- 172.21.15.88:0/2986716245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05ac19fd00 con 0x7f05ac0ffd60 2024-02-19T19:23:41.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.521+0000 7f05b3a64700 1 -- 172.21.15.88:0/2986716245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac1a0220 con 0x7f05ac0ffd60 2024-02-19T19:23:41.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.522+0000 7f05b3a64700 1 -- 172.21.15.88:0/2986716245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac1076d0 con 0x7f05ac0ffd60 2024-02-19T19:23:41.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.522+0000 7f05a27fc700 1 -- 172.21.15.88:0/2986716245 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f059c0105d0 con 0x7f05ac0ffd60 2024-02-19T19:23:41.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.523+0000 7f05a27fc700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f059806d8a0 0x7f059806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:41.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.523+0000 7f05b1800700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f059806d8a0 0x7f059806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:41.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.523+0000 7f05a27fc700 1 -- 172.21.15.88:0/2986716245 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f059c08d070 con 0x7f05ac0ffd60 2024-02-19T19:23:41.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.523+0000 7f05b1800700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f059806d8a0 0x7f059806fd50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f05a8005b40 tx=0x7f05a801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:41.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.528+0000 7f05a27fc700 1 -- 172.21.15.88:0/2986716245 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f059c0568a0 con 0x7f05ac0ffd60 2024-02-19T19:23:41.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:41 smithi132 ceph-mon[70676]: pgmap v602: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:41.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.811+0000 7f05b3a64700 1 -- 172.21.15.88:0/2986716245 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f05ac04f2a0 con 0x7f05ac0ffd60 2024-02-19T19:23:41.819 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:41 smithi088 ceph-mon[88643]: pgmap v602: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:41.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.812+0000 7f05a27fc700 1 -- 172.21.15.88:0/2986716245 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f059c056070 con 0x7f05ac0ffd60 2024-02-19T19:23:41.820 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:23:41.820 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:23:41.820 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:23:41.820 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:23:41.820 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:23:41.820 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:23:41.820 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:23:41.820 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:23:41.821 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:23:41.821 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:23:41.821 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:23:41.821 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:23:41.821 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:23:41.821 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:23:41.821 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:23:41.821 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:23:41.821 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:23:41.821 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:23:41.821 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:23:41.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.814+0000 7f05b3a64700 1 -- 172.21.15.88:0/2986716245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f059806d8a0 msgr2=0x7f059806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.814+0000 7f05b3a64700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f059806d8a0 0x7f059806fd50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f05a8005b40 tx=0x7f05a801a040 comp rx=0 tx=0).stop 2024-02-19T19:23:41.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.814+0000 7f05b3a64700 1 -- 172.21.15.88:0/2986716245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0ffd60 msgr2=0x7f05ac1971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.814+0000 7f05b3a64700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0ffd60 0x7f05ac1971d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f059c00d900 tx=0x7f059c00dcc0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.814+0000 7f05b3a64700 1 -- 172.21.15.88:0/2986716245 shutdown_connections 2024-02-19T19:23:41.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.814+0000 7f05b3a64700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f059806d8a0 0x7f059806fd50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.814+0000 7f05b3a64700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05ac0ff3c0 0x7f05ac196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.814+0000 7f05b3a64700 1 --2- 172.21.15.88:0/2986716245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05ac0ffd60 0x7f05ac1971d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.814+0000 7f05b3a64700 1 -- 172.21.15.88:0/2986716245 >> 172.21.15.88:0/2986716245 conn(0x7f05ac0fac80 msgr2=0x7f05ac0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:41.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.814+0000 7f05b3a64700 1 -- 172.21.15.88:0/2986716245 shutdown_connections 2024-02-19T19:23:41.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.814+0000 7f05b3a64700 1 -- 172.21.15.88:0/2986716245 wait complete. 2024-02-19T19:23:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.918+0000 7f49bb408700 1 -- 172.21.15.88:0/1575578786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b40ffd60 msgr2=0x7f49b4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.918+0000 7f49bb408700 1 --2- 172.21.15.88:0/1575578786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b40ffd60 0x7f49b4103d30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009910 tx=0x7f49b0009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.918+0000 7f49bb408700 1 -- 172.21.15.88:0/1575578786 shutdown_connections 2024-02-19T19:23:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.918+0000 7f49bb408700 1 --2- 172.21.15.88:0/1575578786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b40ffd60 0x7f49b4103d30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.918+0000 7f49bb408700 1 --2- 172.21.15.88:0/1575578786 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b40ff3c0 0x7f49b40ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.918+0000 7f49bb408700 1 -- 172.21.15.88:0/1575578786 >> 172.21.15.88:0/1575578786 conn(0x7f49b40fac80 msgr2=0x7f49b40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:41.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.918+0000 7f49bb408700 1 -- 172.21.15.88:0/1575578786 shutdown_connections 2024-02-19T19:23:41.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.918+0000 7f49bb408700 1 -- 172.21.15.88:0/1575578786 wait complete. 2024-02-19T19:23:41.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49bb408700 1 Processor -- start 2024-02-19T19:23:41.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49bb408700 1 -- start start 2024-02-19T19:23:41.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49bb408700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b40ff3c0 0x7f49b4196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:41.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49bb408700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b40ffd60 0x7f49b4196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:41.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49bb408700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b4196c20 con 0x7f49b40ff3c0 2024-02-19T19:23:41.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49bb408700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b419a9b0 con 0x7f49b40ffd60 2024-02-19T19:23:41.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49b91a4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b40ff3c0 0x7f49b4196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:41.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49b91a4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b40ff3c0 0x7f49b4196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49918/0 (socket says 172.21.15.88:49918) 2024-02-19T19:23:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49b91a4700 1 -- 172.21.15.88:0/3566357872 learned_addr learned my addr 172.21.15.88:0/3566357872 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49b89a3700 1 --2- 172.21.15.88:0/3566357872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b40ffd60 0x7f49b4196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49b91a4700 1 -- 172.21.15.88:0/3566357872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b40ffd60 msgr2=0x7f49b4196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49b91a4700 1 --2- 172.21.15.88:0/3566357872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b40ffd60 0x7f49b4196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49b91a4700 1 -- 172.21.15.88:0/3566357872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b00095f0 con 0x7f49b40ff3c0 2024-02-19T19:23:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.919+0000 7f49b91a4700 1 --2- 172.21.15.88:0/3566357872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b40ff3c0 0x7f49b4196000 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f49a400b4b0 tx=0x7f49a400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.920+0000 7f49aa7fc700 1 -- 172.21.15.88:0/3566357872 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49a4011800 con 0x7f49b40ff3c0 2024-02-19T19:23:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.920+0000 7f49aa7fc700 1 -- 172.21.15.88:0/3566357872 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49a4011e20 con 0x7f49b40ff3c0 2024-02-19T19:23:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.920+0000 7f49bb408700 1 -- 172.21.15.88:0/3566357872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b419ac90 con 0x7f49b40ff3c0 2024-02-19T19:23:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.920+0000 7f49aa7fc700 1 -- 172.21.15.88:0/3566357872 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49a400f2e0 con 0x7f49b40ff3c0 2024-02-19T19:23:41.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.920+0000 7f49bb408700 1 -- 172.21.15.88:0/3566357872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49b419b1e0 con 0x7f49b40ff3c0 2024-02-19T19:23:41.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.921+0000 7f49bb408700 1 -- 172.21.15.88:0/3566357872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49b404f2a0 con 0x7f49b40ff3c0 2024-02-19T19:23:41.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.921+0000 7f49aa7fc700 1 -- 172.21.15.88:0/3566357872 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f49a4010380 con 0x7f49b40ff3c0 2024-02-19T19:23:41.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.922+0000 7f49aa7fc700 1 --2- 172.21.15.88:0/3566357872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49a006d8a0 0x7f49a006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:41.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.922+0000 7f49aa7fc700 1 -- 172.21.15.88:0/3566357872 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f49a408d040 con 0x7f49b40ff3c0 2024-02-19T19:23:41.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.922+0000 7f49b89a3700 1 --2- 172.21.15.88:0/3566357872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49a006d8a0 0x7f49a006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:41.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.922+0000 7f49b89a3700 1 --2- 172.21.15.88:0/3566357872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49a006d8a0 0x7f49a006fd50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49b00059d0 tx=0x7f49b0005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:41.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:41.926+0000 7f49aa7fc700 1 -- 172.21.15.88:0/3566357872 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49a4056870 con 0x7f49b40ff3c0 2024-02-19T19:23:42.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.167+0000 7f49bb408700 1 -- 172.21.15.88:0/3566357872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f49b419b510 con 0x7f49b40ff3c0 2024-02-19T19:23:42.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.167+0000 7f49aa7fc700 1 -- 172.21.15.88:0/3566357872 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f49a4059f30 con 0x7f49b40ff3c0 2024-02-19T19:23:42.175 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:23:42.176 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:23:42.177 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:23:42.178 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:23:42.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.170+0000 7f49bb408700 1 -- 172.21.15.88:0/3566357872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49a006d8a0 msgr2=0x7f49a006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:42.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.170+0000 7f49bb408700 1 --2- 172.21.15.88:0/3566357872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49a006d8a0 0x7f49a006fd50 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f49b00059d0 tx=0x7f49b0005960 comp rx=0 tx=0).stop 2024-02-19T19:23:42.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.170+0000 7f49bb408700 1 -- 172.21.15.88:0/3566357872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b40ff3c0 msgr2=0x7f49b4196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:42.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.170+0000 7f49bb408700 1 --2- 172.21.15.88:0/3566357872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b40ff3c0 0x7f49b4196000 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f49a400b4b0 tx=0x7f49a400b870 comp rx=0 tx=0).stop 2024-02-19T19:23:42.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.170+0000 7f49bb408700 1 -- 172.21.15.88:0/3566357872 shutdown_connections 2024-02-19T19:23:42.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.170+0000 7f49bb408700 1 --2- 172.21.15.88:0/3566357872 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f49a006d8a0 0x7f49a006fd50 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.170+0000 7f49bb408700 1 --2- 172.21.15.88:0/3566357872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49b40ff3c0 0x7f49b4196000 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.170+0000 7f49bb408700 1 --2- 172.21.15.88:0/3566357872 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49b40ffd60 0x7f49b4196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.170+0000 7f49bb408700 1 -- 172.21.15.88:0/3566357872 >> 172.21.15.88:0/3566357872 conn(0x7f49b40fac80 msgr2=0x7f49b41034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:42.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.170+0000 7f49bb408700 1 -- 172.21.15.88:0/3566357872 shutdown_connections 2024-02-19T19:23:42.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.170+0000 7f49bb408700 1 -- 172.21.15.88:0/3566357872 wait complete. 2024-02-19T19:23:42.180 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:23:42.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.274+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/2255808534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8dc1004e0 msgr2=0x7fb8dc1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:42.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.274+0000 7fb8e3e5c700 1 --2- 172.21.15.88:0/2255808534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8dc1004e0 0x7fb8dc1008b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009910 tx=0x7fb8d8009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:42.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/2255808534 shutdown_connections 2024-02-19T19:23:42.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 --2- 172.21.15.88:0/2255808534 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8dc100e80 0x7fb8dc104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 --2- 172.21.15.88:0/2255808534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8dc1004e0 0x7fb8dc1008b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/2255808534 >> 172.21.15.88:0/2255808534 conn(0x7fb8dc0fbda0 msgr2=0x7fb8dc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:42.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/2255808534 shutdown_connections 2024-02-19T19:23:42.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/2255808534 wait complete. 2024-02-19T19:23:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 Processor -- start 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 -- start start 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8dc1004e0 0x7fb8dc195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8dc100e80 0x7fb8dc196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc196b10 con 0x7fb8dc100e80 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.275+0000 7fb8e3e5c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc19a8a0 con 0x7fb8dc1004e0 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8e13f7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8dc100e80 0x7fb8dc196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8e13f7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8dc100e80 0x7fb8dc196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49948/0 (socket says 172.21.15.88:49948) 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8e13f7700 1 -- 172.21.15.88:0/3408748789 learned_addr learned my addr 172.21.15.88:0/3408748789 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8e1bf8700 1 --2- 172.21.15.88:0/3408748789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8dc1004e0 0x7fb8dc195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8e13f7700 1 -- 172.21.15.88:0/3408748789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8dc1004e0 msgr2=0x7fb8dc195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8e13f7700 1 --2- 172.21.15.88:0/3408748789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8dc1004e0 0x7fb8dc195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8e13f7700 1 -- 172.21.15.88:0/3408748789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d80095f0 con 0x7fb8dc100e80 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8e13f7700 1 --2- 172.21.15.88:0/3408748789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8dc100e80 0x7fb8dc196430 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00d900 tx=0x7fb8cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8d2ffd700 1 -- 172.21.15.88:0/3408748789 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8cc009900 con 0x7fb8dc100e80 2024-02-19T19:23:42.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8d2ffd700 1 -- 172.21.15.88:0/3408748789 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8cc010460 con 0x7fb8dc100e80 2024-02-19T19:23:42.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/3408748789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc19ab80 con 0x7fb8dc100e80 2024-02-19T19:23:42.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8d2ffd700 1 -- 172.21.15.88:0/3408748789 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8cc00f5f0 con 0x7fb8dc100e80 2024-02-19T19:23:42.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.276+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/3408748789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc19b0d0 con 0x7fb8dc100e80 2024-02-19T19:23:42.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.277+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/3408748789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8dc1087f0 con 0x7fb8dc100e80 2024-02-19T19:23:42.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.278+0000 7fb8d2ffd700 1 -- 172.21.15.88:0/3408748789 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb8cc0105d0 con 0x7fb8dc100e80 2024-02-19T19:23:42.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.278+0000 7fb8d2ffd700 1 --2- 172.21.15.88:0/3408748789 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8c806d8a0 0x7fb8c806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:42.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.278+0000 7fb8e1bf8700 1 --2- 172.21.15.88:0/3408748789 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8c806d8a0 0x7fb8c806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:42.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.278+0000 7fb8d2ffd700 1 -- 172.21.15.88:0/3408748789 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb8cc08d0e0 con 0x7fb8dc100e80 2024-02-19T19:23:42.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.279+0000 7fb8e1bf8700 1 --2- 172.21.15.88:0/3408748789 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8c806d8a0 0x7fb8c806fd50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8006010 tx=0x7fb8d801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:42.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.283+0000 7fb8d2ffd700 1 -- 172.21.15.88:0/3408748789 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb8cc056890 con 0x7fb8dc100e80 2024-02-19T19:23:42.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.499+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/3408748789 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8dc19b3b0 con 0x7fb8c806d8a0 2024-02-19T19:23:42.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.501+0000 7fb8d2ffd700 1 -- 172.21.15.88:0/3408748789 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb8dc19b3b0 con 0x7fb8c806d8a0 2024-02-19T19:23:42.508 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:23:42.508 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:23:42.508 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:23:42.509 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:23:42.509 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:23:42.509 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:23:42.509 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:23:42.509 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:23:42.509 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:23:42.509 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:23:42.509 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:23:42.509 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:23:42.509 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:23:42.509 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:23:42.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.503+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/3408748789 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8c806d8a0 msgr2=0x7fb8c806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:42.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.503+0000 7fb8e3e5c700 1 --2- 172.21.15.88:0/3408748789 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8c806d8a0 0x7fb8c806fd50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8006010 tx=0x7fb8d801a040 comp rx=0 tx=0).stop 2024-02-19T19:23:42.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.503+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/3408748789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8dc100e80 msgr2=0x7fb8dc196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:42.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.503+0000 7fb8e3e5c700 1 --2- 172.21.15.88:0/3408748789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8dc100e80 0x7fb8dc196430 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc00d900 tx=0x7fb8cc00dc10 comp rx=0 tx=0).stop 2024-02-19T19:23:42.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.503+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/3408748789 shutdown_connections 2024-02-19T19:23:42.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.503+0000 7fb8e3e5c700 1 --2- 172.21.15.88:0/3408748789 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8c806d8a0 0x7fb8c806fd50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.503+0000 7fb8e3e5c700 1 --2- 172.21.15.88:0/3408748789 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8dc1004e0 0x7fb8dc195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.503+0000 7fb8e3e5c700 1 --2- 172.21.15.88:0/3408748789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8dc100e80 0x7fb8dc196430 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.503+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/3408748789 >> 172.21.15.88:0/3408748789 conn(0x7fb8dc0fbda0 msgr2=0x7fb8dc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:42.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.503+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/3408748789 shutdown_connections 2024-02-19T19:23:42.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.503+0000 7fb8e3e5c700 1 -- 172.21.15.88:0/3408748789 wait complete. 2024-02-19T19:23:42.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.607+0000 7feb3e9f5700 1 -- 172.21.15.88:0/2465156939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb380ffd50 msgr2=0x7feb38103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:42.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.607+0000 7feb3e9f5700 1 --2- 172.21.15.88:0/2465156939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb380ffd50 0x7feb38103d20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7feb2c009910 tx=0x7feb2c009c20 comp rx=0 tx=0).stop 2024-02-19T19:23:42.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.607+0000 7feb3e9f5700 1 -- 172.21.15.88:0/2465156939 shutdown_connections 2024-02-19T19:23:42.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.607+0000 7feb3e9f5700 1 --2- 172.21.15.88:0/2465156939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb380ffd50 0x7feb38103d20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.607+0000 7feb3e9f5700 1 --2- 172.21.15.88:0/2465156939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb380ff3b0 0x7feb380ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.607+0000 7feb3e9f5700 1 -- 172.21.15.88:0/2465156939 >> 172.21.15.88:0/2465156939 conn(0x7feb380fac90 msgr2=0x7feb380fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:42.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.607+0000 7feb3e9f5700 1 -- 172.21.15.88:0/2465156939 shutdown_connections 2024-02-19T19:23:42.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.607+0000 7feb3e9f5700 1 -- 172.21.15.88:0/2465156939 wait complete. 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.608+0000 7feb3e9f5700 1 Processor -- start 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.608+0000 7feb3e9f5700 1 -- start start 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.608+0000 7feb3e9f5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb380ff3b0 0x7feb38195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.608+0000 7feb3e9f5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb380ffd50 0x7feb381963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.608+0000 7feb3e9f5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb38196a10 con 0x7feb380ff3b0 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.608+0000 7feb3e9f5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb38196b50 con 0x7feb380ffd50 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.608+0000 7feb37fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb380ff3b0 0x7feb38195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.608+0000 7feb37fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb380ff3b0 0x7feb38195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49966/0 (socket says 172.21.15.88:49966) 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.608+0000 7feb37fff700 1 -- 172.21.15.88:0/955861648 learned_addr learned my addr 172.21.15.88:0/955861648 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.609+0000 7feb37fff700 1 -- 172.21.15.88:0/955861648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb380ffd50 msgr2=0x7feb381963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.609+0000 7feb37fff700 1 --2- 172.21.15.88:0/955861648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb380ffd50 0x7feb381963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.609+0000 7feb37fff700 1 -- 172.21.15.88:0/955861648 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2c0095f0 con 0x7feb380ff3b0 2024-02-19T19:23:42.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.609+0000 7feb37fff700 1 --2- 172.21.15.88:0/955861648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb380ff3b0 0x7feb38195e80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c930 tx=0x7feb2800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:42.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.609+0000 7feb357fa700 1 -- 172.21.15.88:0/955861648 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb28007ad0 con 0x7feb380ff3b0 2024-02-19T19:23:42.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.609+0000 7feb357fa700 1 -- 172.21.15.88:0/955861648 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb28007c30 con 0x7feb380ff3b0 2024-02-19T19:23:42.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.609+0000 7feb357fa700 1 -- 172.21.15.88:0/955861648 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb28018610 con 0x7feb380ff3b0 2024-02-19T19:23:42.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.609+0000 7feb3e9f5700 1 -- 172.21.15.88:0/955861648 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3819a9a0 con 0x7feb380ff3b0 2024-02-19T19:23:42.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.609+0000 7feb3e9f5700 1 -- 172.21.15.88:0/955861648 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3819aea0 con 0x7feb380ff3b0 2024-02-19T19:23:42.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.610+0000 7feb3e9f5700 1 -- 172.21.15.88:0/955861648 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb38108730 con 0x7feb380ff3b0 2024-02-19T19:23:42.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.610+0000 7feb357fa700 1 -- 172.21.15.88:0/955861648 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7feb2800f450 con 0x7feb380ff3b0 2024-02-19T19:23:42.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.611+0000 7feb357fa700 1 --2- 172.21.15.88:0/955861648 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feb2006d8a0 0x7feb2006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:23:42.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.611+0000 7feb357fa700 1 -- 172.21.15.88:0/955861648 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7feb2808d780 con 0x7feb380ff3b0 2024-02-19T19:23:42.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.611+0000 7feb377fe700 1 --2- 172.21.15.88:0/955861648 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feb2006d8a0 0x7feb2006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:23:42.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.611+0000 7feb377fe700 1 --2- 172.21.15.88:0/955861648 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feb2006d8a0 0x7feb2006fd50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00b5c0 tx=0x7feb2c005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:23:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.616+0000 7feb357fa700 1 -- 172.21.15.88:0/955861648 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feb28056f30 con 0x7feb380ff3b0 2024-02-19T19:23:42.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:42 smithi132 ceph-mon[70676]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:42.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:42 smithi132 ceph-mon[70676]: from='client.35118 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:42.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:42 smithi132 ceph-mon[70676]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:42.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:42 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2986716245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:23:42.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:42 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3566357872' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:23:42.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:42 smithi088 ceph-mon[88643]: from='client.35114 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:42.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:42 smithi088 ceph-mon[88643]: from='client.35118 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:42.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:42 smithi088 ceph-mon[88643]: from='client.35122 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:42.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:42 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2986716245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:23:42.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:42 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3566357872' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:23:42.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.892+0000 7feb3e9f5700 1 -- 172.21.15.88:0/955861648 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feb3804f2a0 con 0x7feb380ff3b0 2024-02-19T19:23:42.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.892+0000 7feb357fa700 1 -- 172.21.15.88:0/955861648 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7feb3804f2a0 con 0x7feb380ff3b0 2024-02-19T19:23:42.900 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:23:42.900 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:23:42.900 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:23:42.900 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:23:42.900 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:23:42.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.894+0000 7feb3e9f5700 1 -- 172.21.15.88:0/955861648 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feb2006d8a0 msgr2=0x7feb2006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:42.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.894+0000 7feb3e9f5700 1 --2- 172.21.15.88:0/955861648 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feb2006d8a0 0x7feb2006fd50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feb2c00b5c0 tx=0x7feb2c005960 comp rx=0 tx=0).stop 2024-02-19T19:23:42.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.894+0000 7feb3e9f5700 1 -- 172.21.15.88:0/955861648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb380ff3b0 msgr2=0x7feb38195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:23:42.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.894+0000 7feb3e9f5700 1 --2- 172.21.15.88:0/955861648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb380ff3b0 0x7feb38195e80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feb2800c930 tx=0x7feb2800ccf0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.894+0000 7feb3e9f5700 1 -- 172.21.15.88:0/955861648 shutdown_connections 2024-02-19T19:23:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.894+0000 7feb3e9f5700 1 --2- 172.21.15.88:0/955861648 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feb2006d8a0 0x7feb2006fd50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.894+0000 7feb3e9f5700 1 --2- 172.21.15.88:0/955861648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb380ff3b0 0x7feb38195e80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.894+0000 7feb3e9f5700 1 --2- 172.21.15.88:0/955861648 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feb380ffd50 0x7feb381963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:23:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.894+0000 7feb3e9f5700 1 -- 172.21.15.88:0/955861648 >> 172.21.15.88:0/955861648 conn(0x7feb380fac90 msgr2=0x7feb38069110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:23:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.894+0000 7feb3e9f5700 1 -- 172.21.15.88:0/955861648 shutdown_connections 2024-02-19T19:23:42.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:23:42.894+0000 7feb3e9f5700 1 -- 172.21.15.88:0/955861648 wait complete. 2024-02-19T19:23:43.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:43 smithi132 ceph-mon[70676]: pgmap v603: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:43.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:43 smithi132 ceph-mon[70676]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:43.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:43 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/955861648' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:23:43.741 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:23:43.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:43 smithi088 ceph-mon[88643]: pgmap v603: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:43.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:43 smithi088 ceph-mon[88643]: from='client.35134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:23:43.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:43 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/955861648' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:23:43.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:23:45.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:45 smithi132 ceph-mon[70676]: pgmap v604: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:45.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:45 smithi088 ceph-mon[88643]: pgmap v604: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:47.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:47 smithi132 ceph-mon[70676]: pgmap v605: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:47.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:47 smithi088 ceph-mon[88643]: pgmap v605: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:49.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:49 smithi132 ceph-mon[70676]: pgmap v606: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:49.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:49 smithi088 ceph-mon[88643]: pgmap v606: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:51.588 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:51 smithi088 ceph-mon[88643]: pgmap v607: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:51.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:51 smithi132 ceph-mon[70676]: pgmap v607: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:53.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:53 smithi132 ceph-mon[70676]: pgmap v608: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:53.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:53 smithi088 ceph-mon[88643]: pgmap v608: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:55.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:55 smithi132 ceph-mon[70676]: pgmap v609: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:55.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:55 smithi088 ceph-mon[88643]: pgmap v609: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:57.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:57 smithi132 ceph-mon[70676]: pgmap v610: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:57.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:57 smithi088 ceph-mon[88643]: pgmap v610: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:58.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:23:58.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:23:59.740 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:23:59 smithi132 ceph-mon[70676]: pgmap v611: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:23:59.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:23:59 smithi088 ceph-mon[88643]: pgmap v611: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:01.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:01 smithi088 ceph-mon[88643]: pgmap v612: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:01.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:01 smithi132 ceph-mon[70676]: pgmap v612: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:03.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:03 smithi088 ceph-mon[88643]: pgmap v613: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:03.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:03 smithi132 ceph-mon[70676]: pgmap v613: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:05.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:05 smithi088 ceph-mon[88643]: pgmap v614: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:05.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:05 smithi132 ceph-mon[70676]: pgmap v614: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:07.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:07 smithi088 ceph-mon[88643]: pgmap v615: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:07.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:07 smithi132 ceph-mon[70676]: pgmap v615: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:09.789 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:09 smithi132 ceph-mon[70676]: pgmap v616: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:09.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:09 smithi088 ceph-mon[88643]: pgmap v616: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:11.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:11 smithi088 ceph-mon[88643]: pgmap v617: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:11.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:11 smithi132 ceph-mon[70676]: pgmap v617: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.013+0000 7fed80f8f700 1 -- 172.21.15.88:0/1849065652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed7c1004e0 msgr2=0x7fed7c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.013+0000 7fed80f8f700 1 --2- 172.21.15.88:0/1849065652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed7c1004e0 0x7fed7c1008b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fed6c009910 tx=0x7fed6c009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.013+0000 7fed80f8f700 1 -- 172.21.15.88:0/1849065652 shutdown_connections 2024-02-19T19:24:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.013+0000 7fed80f8f700 1 --2- 172.21.15.88:0/1849065652 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed7c100e80 0x7fed7c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.013+0000 7fed80f8f700 1 --2- 172.21.15.88:0/1849065652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed7c1004e0 0x7fed7c1008b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.013+0000 7fed80f8f700 1 -- 172.21.15.88:0/1849065652 >> 172.21.15.88:0/1849065652 conn(0x7fed7c0fbda0 msgr2=0x7fed7c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.013+0000 7fed80f8f700 1 -- 172.21.15.88:0/1849065652 shutdown_connections 2024-02-19T19:24:13.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.013+0000 7fed80f8f700 1 -- 172.21.15.88:0/1849065652 wait complete. 2024-02-19T19:24:13.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.014+0000 7fed80f8f700 1 Processor -- start 2024-02-19T19:24:13.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.014+0000 7fed80f8f700 1 -- start start 2024-02-19T19:24:13.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.014+0000 7fed80f8f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed7c1004e0 0x7fed7c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.014+0000 7fed80f8f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed7c100e80 0x7fed7c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.014+0000 7fed80f8f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c196bb0 con 0x7fed7c100e80 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.014+0000 7fed80f8f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed7c19a8d0 con 0x7fed7c1004e0 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.014+0000 7fed79d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed7c100e80 0x7fed7c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.014+0000 7fed79d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed7c100e80 0x7fed7c1964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36268/0 (socket says 172.21.15.88:36268) 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.014+0000 7fed79d9b700 1 -- 172.21.15.88:0/3017535263 learned_addr learned my addr 172.21.15.88:0/3017535263 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.014+0000 7fed7a59c700 1 --2- 172.21.15.88:0/3017535263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed7c1004e0 0x7fed7c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.015+0000 7fed79d9b700 1 -- 172.21.15.88:0/3017535263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed7c1004e0 msgr2=0x7fed7c195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.015+0000 7fed79d9b700 1 --2- 172.21.15.88:0/3017535263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed7c1004e0 0x7fed7c195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.015+0000 7fed79d9b700 1 -- 172.21.15.88:0/3017535263 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c0095f0 con 0x7fed7c100e80 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.015+0000 7fed79d9b700 1 --2- 172.21.15.88:0/3017535263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed7c100e80 0x7fed7c1964d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fed7000b480 tx=0x7fed7000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.015+0000 7fed6b7fe700 1 -- 172.21.15.88:0/3017535263 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed700118c0 con 0x7fed7c100e80 2024-02-19T19:24:13.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.015+0000 7fed6b7fe700 1 -- 172.21.15.88:0/3017535263 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed70010380 con 0x7fed7c100e80 2024-02-19T19:24:13.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.015+0000 7fed80f8f700 1 -- 172.21.15.88:0/3017535263 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c19abb0 con 0x7fed7c100e80 2024-02-19T19:24:13.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.015+0000 7fed6b7fe700 1 -- 172.21.15.88:0/3017535263 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed7000f4f0 con 0x7fed7c100e80 2024-02-19T19:24:13.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.015+0000 7fed80f8f700 1 -- 172.21.15.88:0/3017535263 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c19b100 con 0x7fed7c100e80 2024-02-19T19:24:13.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.016+0000 7fed80f8f700 1 -- 172.21.15.88:0/3017535263 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed7c1087f0 con 0x7fed7c100e80 2024-02-19T19:24:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.016+0000 7fed6b7fe700 1 -- 172.21.15.88:0/3017535263 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fed700104f0 con 0x7fed7c100e80 2024-02-19T19:24:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.017+0000 7fed6b7fe700 1 --2- 172.21.15.88:0/3017535263 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed6406d8a0 0x7fed6406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.017+0000 7fed6b7fe700 1 -- 172.21.15.88:0/3017535263 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fed7008d0a0 con 0x7fed7c100e80 2024-02-19T19:24:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.017+0000 7fed7a59c700 1 --2- 172.21.15.88:0/3017535263 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed6406d8a0 0x7fed6406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:13.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.017+0000 7fed7a59c700 1 --2- 172.21.15.88:0/3017535263 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed6406d8a0 0x7fed6406fd50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00b5c0 tx=0x7fed6c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:13.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.022+0000 7fed6b7fe700 1 -- 172.21.15.88:0/3017535263 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed700568d0 con 0x7fed7c100e80 2024-02-19T19:24:13.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.237+0000 7fed80f8f700 1 -- 172.21.15.88:0/3017535263 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fed7c19b3d0 con 0x7fed6406d8a0 2024-02-19T19:24:13.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.240+0000 7fed6b7fe700 1 -- 172.21.15.88:0/3017535263 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fed7c19b3d0 con 0x7fed6406d8a0 2024-02-19T19:24:13.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.243+0000 7fed80f8f700 1 -- 172.21.15.88:0/3017535263 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed6406d8a0 msgr2=0x7fed6406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.243+0000 7fed80f8f700 1 --2- 172.21.15.88:0/3017535263 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed6406d8a0 0x7fed6406fd50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed6c00b5c0 tx=0x7fed6c01a040 comp rx=0 tx=0).stop 2024-02-19T19:24:13.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.243+0000 7fed80f8f700 1 -- 172.21.15.88:0/3017535263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed7c100e80 msgr2=0x7fed7c1964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.243+0000 7fed80f8f700 1 --2- 172.21.15.88:0/3017535263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed7c100e80 0x7fed7c1964d0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fed7000b480 tx=0x7fed7000b840 comp rx=0 tx=0).stop 2024-02-19T19:24:13.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.243+0000 7fed80f8f700 1 -- 172.21.15.88:0/3017535263 shutdown_connections 2024-02-19T19:24:13.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.243+0000 7fed80f8f700 1 --2- 172.21.15.88:0/3017535263 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed6406d8a0 0x7fed6406fd50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.243+0000 7fed80f8f700 1 --2- 172.21.15.88:0/3017535263 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed7c1004e0 0x7fed7c195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.243+0000 7fed80f8f700 1 --2- 172.21.15.88:0/3017535263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed7c100e80 0x7fed7c1964d0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.243+0000 7fed80f8f700 1 -- 172.21.15.88:0/3017535263 >> 172.21.15.88:0/3017535263 conn(0x7fed7c0fbda0 msgr2=0x7fed7c0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:13.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.243+0000 7fed80f8f700 1 -- 172.21.15.88:0/3017535263 shutdown_connections 2024-02-19T19:24:13.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.243+0000 7fed80f8f700 1 -- 172.21.15.88:0/3017535263 wait complete. 2024-02-19T19:24:13.261 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:24:13.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.349+0000 7f4a5bb74700 1 -- 172.21.15.88:0/1066042134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a54101420 msgr2=0x7f4a541017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.349+0000 7f4a5bb74700 1 --2- 172.21.15.88:0/1066042134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a54101420 0x7f4a541017f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009910 tx=0x7f4a50009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:13.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.349+0000 7f4a5bb74700 1 -- 172.21.15.88:0/1066042134 shutdown_connections 2024-02-19T19:24:13.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.349+0000 7f4a5bb74700 1 --2- 172.21.15.88:0/1066042134 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a54101d30 0x7f4a5410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.349+0000 7f4a5bb74700 1 --2- 172.21.15.88:0/1066042134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a54101420 0x7f4a541017f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.349+0000 7f4a5bb74700 1 -- 172.21.15.88:0/1066042134 >> 172.21.15.88:0/1066042134 conn(0x7f4a540fac80 msgr2=0x7f4a540fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:13.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.349+0000 7f4a5bb74700 1 -- 172.21.15.88:0/1066042134 shutdown_connections 2024-02-19T19:24:13.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.349+0000 7f4a5bb74700 1 -- 172.21.15.88:0/1066042134 wait complete. 2024-02-19T19:24:13.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.350+0000 7f4a5bb74700 1 Processor -- start 2024-02-19T19:24:13.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.350+0000 7f4a5bb74700 1 -- start start 2024-02-19T19:24:13.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.350+0000 7f4a5bb74700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a54101420 0x7f4a54195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:13.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.350+0000 7f4a5bb74700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a54101d30 0x7f4a541963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:13.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.350+0000 7f4a5bb74700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a54196a00 con 0x7f4a54101420 2024-02-19T19:24:13.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.350+0000 7f4a5bb74700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a54196b40 con 0x7f4a54101d30 2024-02-19T19:24:13.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.350+0000 7f4a59910700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a54101420 0x7f4a54195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:13.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a59910700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a54101420 0x7f4a54195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36288/0 (socket says 172.21.15.88:36288) 2024-02-19T19:24:13.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a59910700 1 -- 172.21.15.88:0/2816814796 learned_addr learned my addr 172.21.15.88:0/2816814796 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:13.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a5910f700 1 --2- 172.21.15.88:0/2816814796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a54101d30 0x7f4a541963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:13.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a59910700 1 -- 172.21.15.88:0/2816814796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a54101d30 msgr2=0x7f4a541963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a59910700 1 --2- 172.21.15.88:0/2816814796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a54101d30 0x7f4a541963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a59910700 1 -- 172.21.15.88:0/2816814796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500095f0 con 0x7f4a54101420 2024-02-19T19:24:13.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a59910700 1 --2- 172.21.15.88:0/2816814796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a54101420 0x7f4a54195e70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4a50006010 tx=0x7f4a5000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:13.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a4affd700 1 -- 172.21.15.88:0/2816814796 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a5001d070 con 0x7f4a54101420 2024-02-19T19:24:13.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a4affd700 1 -- 172.21.15.88:0/2816814796 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a5000f460 con 0x7f4a54101420 2024-02-19T19:24:13.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a5bb74700 1 -- 172.21.15.88:0/2816814796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5419a930 con 0x7f4a54101420 2024-02-19T19:24:13.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a4affd700 1 -- 172.21.15.88:0/2816814796 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a50017630 con 0x7f4a54101420 2024-02-19T19:24:13.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.351+0000 7f4a5bb74700 1 -- 172.21.15.88:0/2816814796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5419ae20 con 0x7f4a54101420 2024-02-19T19:24:13.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.352+0000 7f4a5bb74700 1 -- 172.21.15.88:0/2816814796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a54107900 con 0x7f4a54101420 2024-02-19T19:24:13.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.353+0000 7f4a4affd700 1 -- 172.21.15.88:0/2816814796 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4a5000f5d0 con 0x7f4a54101420 2024-02-19T19:24:13.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.353+0000 7f4a4affd700 1 --2- 172.21.15.88:0/2816814796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a4006d8a0 0x7f4a4006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:13.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.353+0000 7f4a4affd700 1 -- 172.21.15.88:0/2816814796 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4a5008e9a0 con 0x7f4a54101420 2024-02-19T19:24:13.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.353+0000 7f4a5910f700 1 --2- 172.21.15.88:0/2816814796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a4006d8a0 0x7f4a4006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:13.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.354+0000 7f4a5910f700 1 --2- 172.21.15.88:0/2816814796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a4006d8a0 0x7f4a4006fd50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4a54197490 tx=0x7f4a4400a380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:13.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.358+0000 7f4a4affd700 1 -- 172.21.15.88:0/2816814796 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a500581d0 con 0x7f4a54101420 2024-02-19T19:24:13.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.577+0000 7f4a5bb74700 1 -- 172.21.15.88:0/2816814796 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a5402cca0 con 0x7f4a4006d8a0 2024-02-19T19:24:13.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.578+0000 7f4a4affd700 1 -- 172.21.15.88:0/2816814796 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4a5402cca0 con 0x7f4a4006d8a0 2024-02-19T19:24:13.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.580+0000 7f4a5bb74700 1 -- 172.21.15.88:0/2816814796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a4006d8a0 msgr2=0x7f4a4006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.580+0000 7f4a5bb74700 1 --2- 172.21.15.88:0/2816814796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a4006d8a0 0x7f4a4006fd50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4a54197490 tx=0x7f4a4400a380 comp rx=0 tx=0).stop 2024-02-19T19:24:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.580+0000 7f4a5bb74700 1 -- 172.21.15.88:0/2816814796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a54101420 msgr2=0x7f4a54195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.580+0000 7f4a5bb74700 1 --2- 172.21.15.88:0/2816814796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a54101420 0x7f4a54195e70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4a50006010 tx=0x7f4a5000bd70 comp rx=0 tx=0).stop 2024-02-19T19:24:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.580+0000 7f4a5bb74700 1 -- 172.21.15.88:0/2816814796 shutdown_connections 2024-02-19T19:24:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.580+0000 7f4a5bb74700 1 --2- 172.21.15.88:0/2816814796 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4a4006d8a0 0x7f4a4006fd50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.580+0000 7f4a5bb74700 1 --2- 172.21.15.88:0/2816814796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a54101420 0x7f4a54195e70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.580+0000 7f4a5bb74700 1 --2- 172.21.15.88:0/2816814796 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4a54101d30 0x7f4a541963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.580+0000 7f4a5bb74700 1 -- 172.21.15.88:0/2816814796 >> 172.21.15.88:0/2816814796 conn(0x7f4a540fac80 msgr2=0x7f4a54104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.580+0000 7f4a5bb74700 1 -- 172.21.15.88:0/2816814796 shutdown_connections 2024-02-19T19:24:13.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.580+0000 7f4a5bb74700 1 -- 172.21.15.88:0/2816814796 wait complete. 2024-02-19T19:24:13.588 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:13 smithi088 ceph-mon[88643]: pgmap v618: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:13.588 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:24:13.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.685+0000 7f143126f700 1 -- 172.21.15.88:0/3649800757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f142c104130 msgr2=0x7f142c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.685+0000 7f143126f700 1 --2- 172.21.15.88:0/3649800757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f142c104130 0x7f142c104500 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f141c009910 tx=0x7f141c009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:13.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.685+0000 7f143126f700 1 -- 172.21.15.88:0/3649800757 shutdown_connections 2024-02-19T19:24:13.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.685+0000 7f143126f700 1 --2- 172.21.15.88:0/3649800757 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f142c0ffbd0 0x7f142c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.685+0000 7f143126f700 1 --2- 172.21.15.88:0/3649800757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f142c104130 0x7f142c104500 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.685+0000 7f143126f700 1 -- 172.21.15.88:0/3649800757 >> 172.21.15.88:0/3649800757 conn(0x7f142c075130 msgr2=0x7f142c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:13.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.685+0000 7f143126f700 1 -- 172.21.15.88:0/3649800757 shutdown_connections 2024-02-19T19:24:13.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.685+0000 7f143126f700 1 -- 172.21.15.88:0/3649800757 wait complete. 2024-02-19T19:24:13.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.685+0000 7f143126f700 1 Processor -- start 2024-02-19T19:24:13.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.685+0000 7f143126f700 1 -- start start 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f143126f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f142c0ffbd0 0x7f142c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f143126f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f142c104130 0x7f142c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f143126f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c1018b0 con 0x7f142c0ffbd0 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f143126f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c1019f0 con 0x7f142c104130 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f142affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f142c0ffbd0 0x7f142c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f142affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f142c0ffbd0 0x7f142c100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36312/0 (socket says 172.21.15.88:36312) 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f142affd700 1 -- 172.21.15.88:0/1398922223 learned_addr learned my addr 172.21.15.88:0/1398922223 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f142a7fc700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f142c104130 0x7f142c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f142affd700 1 -- 172.21.15.88:0/1398922223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f142c104130 msgr2=0x7f142c101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f142affd700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f142c104130 0x7f142c101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f142affd700 1 -- 172.21.15.88:0/1398922223 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0095f0 con 0x7f142c0ffbd0 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f142a7fc700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f142c104130 0x7f142c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:24:13.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f142affd700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f142c0ffbd0 0x7f142c100e30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f141c006010 tx=0x7f141c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:13.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f1413fff700 1 -- 172.21.15.88:0/1398922223 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141c01d070 con 0x7f142c0ffbd0 2024-02-19T19:24:13.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f1413fff700 1 -- 172.21.15.88:0/1398922223 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f141c004fb0 con 0x7f142c0ffbd0 2024-02-19T19:24:13.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f143126f700 1 -- 172.21.15.88:0/1398922223 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c101c70 con 0x7f142c0ffbd0 2024-02-19T19:24:13.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f1413fff700 1 -- 172.21.15.88:0/1398922223 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f141c00f690 con 0x7f142c0ffbd0 2024-02-19T19:24:13.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.686+0000 7f143126f700 1 -- 172.21.15.88:0/1398922223 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c1a2300 con 0x7f142c0ffbd0 2024-02-19T19:24:13.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.687+0000 7f143126f700 1 -- 172.21.15.88:0/1398922223 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c1086a0 con 0x7f142c0ffbd0 2024-02-19T19:24:13.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.688+0000 7f1413fff700 1 -- 172.21.15.88:0/1398922223 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f141c022470 con 0x7f142c0ffbd0 2024-02-19T19:24:13.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.688+0000 7f1413fff700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f141406d8a0 0x7f141406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:13.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.688+0000 7f1413fff700 1 -- 172.21.15.88:0/1398922223 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f141c08edd0 con 0x7f142c0ffbd0 2024-02-19T19:24:13.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.688+0000 7f142a7fc700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f141406d8a0 0x7f141406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:13.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.689+0000 7f142a7fc700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f141406d8a0 0x7f141406fd50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1420005fd0 tx=0x7f1420005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:13.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.693+0000 7f1413fff700 1 -- 172.21.15.88:0/1398922223 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f141c0586a0 con 0x7f142c0ffbd0 2024-02-19T19:24:13.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.899+0000 7f143126f700 1 -- 172.21.15.88:0/1398922223 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f142c061190 con 0x7f141406d8a0 2024-02-19T19:24:13.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.910+0000 7f1413fff700 1 -- 172.21.15.88:0/1398922223 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f142c061190 con 0x7f141406d8a0 2024-02-19T19:24:13.918 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:24:13.918 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (21m) 13m ago 31m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:24:13.918 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (32m) 13m ago 32m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:24:13.918 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (14m) 12m ago 30m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:24:13.918 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (17m) 13m ago 32m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:24:13.918 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (17m) 12m ago 30m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:24:13.918 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (20m) 13m ago 31m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:24:13.918 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (13m) 13m ago 25m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:24:13.918 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (13m) 13m ago 25m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:24:13.918 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (13m) 12m ago 25m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (25m) 12m ago 25m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (22m) 13m ago 33m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (22m) 12m ago 30m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (19m) 13m ago 33m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (18m) 12m ago 30m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (22m) 13m ago 32m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (22m) 12m ago 30m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (16m) 13m ago 29m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (16m) 13m ago 28m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (15m) 13m ago 28m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (15m) 12m ago 27m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (14m) 12m ago 27m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (13m) 12m ago 27m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:24:13.919 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (21m) 13m ago 30m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:24:13.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.912+0000 7f143126f700 1 -- 172.21.15.88:0/1398922223 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f141406d8a0 msgr2=0x7f141406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.912+0000 7f143126f700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f141406d8a0 0x7f141406fd50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1420005fd0 tx=0x7f1420005e20 comp rx=0 tx=0).stop 2024-02-19T19:24:13.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.912+0000 7f143126f700 1 -- 172.21.15.88:0/1398922223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f142c0ffbd0 msgr2=0x7f142c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:13.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.912+0000 7f143126f700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f142c0ffbd0 0x7f142c100e30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f141c006010 tx=0x7f141c00be20 comp rx=0 tx=0).stop 2024-02-19T19:24:13.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.912+0000 7f143126f700 1 -- 172.21.15.88:0/1398922223 shutdown_connections 2024-02-19T19:24:13.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.912+0000 7f143126f700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f141406d8a0 0x7f141406fd50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.912+0000 7f143126f700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f142c0ffbd0 0x7f142c100e30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.912+0000 7f143126f700 1 --2- 172.21.15.88:0/1398922223 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f142c104130 0x7f142c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:13.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.912+0000 7f143126f700 1 -- 172.21.15.88:0/1398922223 >> 172.21.15.88:0/1398922223 conn(0x7f142c075130 msgr2=0x7f142c0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:13.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.913+0000 7f143126f700 1 -- 172.21.15.88:0/1398922223 shutdown_connections 2024-02-19T19:24:13.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:13.913+0000 7f143126f700 1 -- 172.21.15.88:0/1398922223 wait complete. 2024-02-19T19:24:13.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:13 smithi132 ceph-mon[70676]: pgmap v618: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:13.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:24:14.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.015+0000 7f81cec2f700 1 -- 172.21.15.88:0/3517215529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c8104130 msgr2=0x7f81c8104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:14.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.015+0000 7f81cec2f700 1 --2- 172.21.15.88:0/3517215529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c8104130 0x7f81c8104500 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f81c400b0a0 tx=0x7f81c400b3b0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.015+0000 7f81cec2f700 1 -- 172.21.15.88:0/3517215529 shutdown_connections 2024-02-19T19:24:14.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.015+0000 7f81cec2f700 1 --2- 172.21.15.88:0/3517215529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c80ffbd0 0x7f81c8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.015+0000 7f81cec2f700 1 --2- 172.21.15.88:0/3517215529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c8104130 0x7f81c8104500 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.015+0000 7f81cec2f700 1 -- 172.21.15.88:0/3517215529 >> 172.21.15.88:0/3517215529 conn(0x7f81c8075130 msgr2=0x7f81c8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:14.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.015+0000 7f81cec2f700 1 -- 172.21.15.88:0/3517215529 shutdown_connections 2024-02-19T19:24:14.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.015+0000 7f81cec2f700 1 -- 172.21.15.88:0/3517215529 wait complete. 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81cec2f700 1 Processor -- start 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81cec2f700 1 -- start start 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81cec2f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c80ffbd0 0x7f81c8196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81cec2f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c8104130 0x7f81c81972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81cec2f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c81979d0 con 0x7f81c8104130 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81cec2f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81c8197b40 con 0x7f81c80ffbd0 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81bffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c8104130 0x7f81c81972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81bffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c8104130 0x7f81c81972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36330/0 (socket says 172.21.15.88:36330) 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81bffff700 1 -- 172.21.15.88:0/3816084459 learned_addr learned my addr 172.21.15.88:0/3816084459 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81cc9cb700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c80ffbd0 0x7f81c8196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81bffff700 1 -- 172.21.15.88:0/3816084459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c80ffbd0 msgr2=0x7f81c8196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81bffff700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c80ffbd0 0x7f81c8196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81bffff700 1 -- 172.21.15.88:0/3816084459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c4009d00 con 0x7f81c8104130 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81cc9cb700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c80ffbd0 0x7f81c8196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.016+0000 7f81bffff700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c8104130 0x7f81c81972f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f81b800b4b0 tx=0x7f81b800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.017+0000 7f81bdffb700 1 -- 172.21.15.88:0/3816084459 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b8011800 con 0x7f81c8104130 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.017+0000 7f81bdffb700 1 -- 172.21.15.88:0/3816084459 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81b8011e20 con 0x7f81c8104130 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.017+0000 7f81bdffb700 1 -- 172.21.15.88:0/3816084459 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81b800f2e0 con 0x7f81c8104130 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.017+0000 7f81cec2f700 1 -- 172.21.15.88:0/3816084459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81c819fe20 con 0x7f81c8104130 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.017+0000 7f81cec2f700 1 -- 172.21.15.88:0/3816084459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81c81a02f0 con 0x7f81c8104130 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.018+0000 7f81cec2f700 1 -- 172.21.15.88:0/3816084459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81c81076a0 con 0x7f81c8104130 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.018+0000 7f81bdffb700 1 -- 172.21.15.88:0/3816084459 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f81b8010380 con 0x7f81c8104130 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.018+0000 7f81bdffb700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f81b006d8a0 0x7f81b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.019+0000 7f81bdffb700 1 -- 172.21.15.88:0/3816084459 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f81b808d130 con 0x7f81c8104130 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.019+0000 7f81cc9cb700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f81b006d8a0 0x7f81b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:14.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.019+0000 7f81cc9cb700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f81b006d8a0 0x7f81b006fd50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f81c40062a0 tx=0x7f81c4006210 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:14.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.023+0000 7f81bdffb700 1 -- 172.21.15.88:0/3816084459 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f81b80568e0 con 0x7f81c8104130 2024-02-19T19:24:14.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.303+0000 7f81cec2f700 1 -- 172.21.15.88:0/3816084459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f81c804f2a0 con 0x7f81c8104130 2024-02-19T19:24:14.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.303+0000 7f81bdffb700 1 -- 172.21.15.88:0/3816084459 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f81b8059fa0 con 0x7f81c8104130 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:24:14.311 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:24:14.312 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:24:14.312 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:24:14.312 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:24:14.312 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:24:14.312 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:24:14.312 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:24:14.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.305+0000 7f81cec2f700 1 -- 172.21.15.88:0/3816084459 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f81b006d8a0 msgr2=0x7f81b006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:14.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.305+0000 7f81cec2f700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f81b006d8a0 0x7f81b006fd50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f81c40062a0 tx=0x7f81c4006210 comp rx=0 tx=0).stop 2024-02-19T19:24:14.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.305+0000 7f81cec2f700 1 -- 172.21.15.88:0/3816084459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c8104130 msgr2=0x7f81c81972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:14.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.305+0000 7f81cec2f700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c8104130 0x7f81c81972f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f81b800b4b0 tx=0x7f81b800b870 comp rx=0 tx=0).stop 2024-02-19T19:24:14.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.305+0000 7f81cec2f700 1 -- 172.21.15.88:0/3816084459 shutdown_connections 2024-02-19T19:24:14.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.305+0000 7f81cec2f700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f81b006d8a0 0x7f81b006fd50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.305+0000 7f81cec2f700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f81c80ffbd0 0x7f81c8196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.305+0000 7f81cec2f700 1 --2- 172.21.15.88:0/3816084459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c8104130 0x7f81c81972f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.305+0000 7f81cec2f700 1 -- 172.21.15.88:0/3816084459 >> 172.21.15.88:0/3816084459 conn(0x7f81c8075130 msgr2=0x7f81c80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:14.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.305+0000 7f81cec2f700 1 -- 172.21.15.88:0/3816084459 shutdown_connections 2024-02-19T19:24:14.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.305+0000 7f81cec2f700 1 -- 172.21.15.88:0/3816084459 wait complete. 2024-02-19T19:24:14.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.410+0000 7fa43803c700 1 -- 172.21.15.88:0/186740415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4300ff350 msgr2=0x7fa4300ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:14.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.410+0000 7fa43803c700 1 --2- 172.21.15.88:0/186740415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4300ff350 0x7fa4300ff7c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009910 tx=0x7fa42c009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:14.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.410+0000 7fa43803c700 1 -- 172.21.15.88:0/186740415 shutdown_connections 2024-02-19T19:24:14.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.410+0000 7fa43803c700 1 --2- 172.21.15.88:0/186740415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4300ff350 0x7fa4300ff7c0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.410+0000 7fa43803c700 1 --2- 172.21.15.88:0/186740415 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4301038b0 0x7fa430103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.410+0000 7fa43803c700 1 -- 172.21.15.88:0/186740415 >> 172.21.15.88:0/186740415 conn(0x7fa4300748b0 msgr2=0x7fa430074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:14.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.410+0000 7fa43803c700 1 -- 172.21.15.88:0/186740415 shutdown_connections 2024-02-19T19:24:14.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.410+0000 7fa43803c700 1 -- 172.21.15.88:0/186740415 wait complete. 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.410+0000 7fa43803c700 1 Processor -- start 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa43803c700 1 -- start start 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa43803c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4300ff350 0x7fa430196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa43803c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4301038b0 0x7fa430196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa43803c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa430196c20 con 0x7fa4300ff350 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa43803c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43019a9b0 con 0x7fa4301038b0 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa435dd8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4300ff350 0x7fa430196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa435dd8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4300ff350 0x7fa430196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36342/0 (socket says 172.21.15.88:36342) 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa435dd8700 1 -- 172.21.15.88:0/2034134170 learned_addr learned my addr 172.21.15.88:0/2034134170 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa4355d7700 1 --2- 172.21.15.88:0/2034134170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4301038b0 0x7fa430196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa435dd8700 1 -- 172.21.15.88:0/2034134170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4301038b0 msgr2=0x7fa430196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa435dd8700 1 --2- 172.21.15.88:0/2034134170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4301038b0 0x7fa430196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa435dd8700 1 -- 172.21.15.88:0/2034134170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c0095f0 con 0x7fa4300ff350 2024-02-19T19:24:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa435dd8700 1 --2- 172.21.15.88:0/2034134170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4300ff350 0x7fa430196000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa42000b4b0 tx=0x7fa42000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.411+0000 7fa426ffd700 1 -- 172.21.15.88:0/2034134170 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa420011800 con 0x7fa4300ff350 2024-02-19T19:24:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.412+0000 7fa43803c700 1 -- 172.21.15.88:0/2034134170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43019ac90 con 0x7fa4300ff350 2024-02-19T19:24:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.412+0000 7fa426ffd700 1 -- 172.21.15.88:0/2034134170 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa420011e20 con 0x7fa4300ff350 2024-02-19T19:24:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.412+0000 7fa43803c700 1 -- 172.21.15.88:0/2034134170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43019b1e0 con 0x7fa4300ff350 2024-02-19T19:24:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.412+0000 7fa426ffd700 1 -- 172.21.15.88:0/2034134170 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa42001a400 con 0x7fa4300ff350 2024-02-19T19:24:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.412+0000 7fa43803c700 1 -- 172.21.15.88:0/2034134170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa430108e30 con 0x7fa4300ff350 2024-02-19T19:24:14.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.413+0000 7fa426ffd700 1 -- 172.21.15.88:0/2034134170 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa420010380 con 0x7fa4300ff350 2024-02-19T19:24:14.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.413+0000 7fa426ffd700 1 --2- 172.21.15.88:0/2034134170 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa41c06d8a0 0x7fa41c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:14.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.413+0000 7fa426ffd700 1 -- 172.21.15.88:0/2034134170 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa42008cff0 con 0x7fa4300ff350 2024-02-19T19:24:14.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.413+0000 7fa4355d7700 1 --2- 172.21.15.88:0/2034134170 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa41c06d8a0 0x7fa41c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:14.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.414+0000 7fa4355d7700 1 --2- 172.21.15.88:0/2034134170 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa41c06d8a0 0x7fa41c06fd50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa430197620 tx=0x7fa42c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:14.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.418+0000 7fa426ffd700 1 -- 172.21.15.88:0/2034134170 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa4200567a0 con 0x7fa4300ff350 2024-02-19T19:24:14.587 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:14 smithi088 ceph-mon[88643]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:14.587 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:14 smithi088 ceph-mon[88643]: from='client.35146 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:14.587 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:14 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3816084459' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:24:14.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.662+0000 7fa43803c700 1 -- 172.21.15.88:0/2034134170 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa43004f2a0 con 0x7fa4300ff350 2024-02-19T19:24:14.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.662+0000 7fa426ffd700 1 -- 172.21.15.88:0/2034134170 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fa420059e60 con 0x7fa4300ff350 2024-02-19T19:24:14.670 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:24:14.670 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:24:14.670 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:24:14.670 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:24:14.670 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:24:14.670 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:24:14.670 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:24:14.670 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:24:14.670 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:24:14.670 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:24:14.671 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:24:14.672 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:24:14.672 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:24:14.672 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:24:14.672 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:24:14.672 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:24:14.672 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:24:14.672 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:24:14.672 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:24:14.672 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:24:14.672 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:24:14.673 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:24:14.673 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:24:14.673 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:24:14.673 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:24:14.673 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:24:14.673 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:24:14.673 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:24:14.673 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:24:14.673 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:24:14.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.664+0000 7fa43803c700 1 -- 172.21.15.88:0/2034134170 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa41c06d8a0 msgr2=0x7fa41c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:14.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.664+0000 7fa43803c700 1 --2- 172.21.15.88:0/2034134170 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa41c06d8a0 0x7fa41c06fd50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa430197620 tx=0x7fa42c01a040 comp rx=0 tx=0).stop 2024-02-19T19:24:14.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.664+0000 7fa43803c700 1 -- 172.21.15.88:0/2034134170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4300ff350 msgr2=0x7fa430196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:14.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.664+0000 7fa43803c700 1 --2- 172.21.15.88:0/2034134170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4300ff350 0x7fa430196000 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fa42000b4b0 tx=0x7fa42000b7c0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.664+0000 7fa43803c700 1 -- 172.21.15.88:0/2034134170 shutdown_connections 2024-02-19T19:24:14.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.664+0000 7fa43803c700 1 --2- 172.21.15.88:0/2034134170 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa41c06d8a0 0x7fa41c06fd50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.664+0000 7fa43803c700 1 --2- 172.21.15.88:0/2034134170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4300ff350 0x7fa430196000 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.664+0000 7fa43803c700 1 --2- 172.21.15.88:0/2034134170 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa4301038b0 0x7fa430196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.664+0000 7fa43803c700 1 -- 172.21.15.88:0/2034134170 >> 172.21.15.88:0/2034134170 conn(0x7fa4300748b0 msgr2=0x7fa430102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:14.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.665+0000 7fa43803c700 1 -- 172.21.15.88:0/2034134170 shutdown_connections 2024-02-19T19:24:14.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.665+0000 7fa43803c700 1 -- 172.21.15.88:0/2034134170 wait complete. 2024-02-19T19:24:14.675 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:24:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.770+0000 7ff22b1cf700 1 -- 172.21.15.88:0/1643207218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2240ffbd0 msgr2=0x7ff224100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:14.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.770+0000 7ff22b1cf700 1 --2- 172.21.15.88:0/1643207218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2240ffbd0 0x7ff224100040 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff210009910 tx=0x7ff210009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.770+0000 7ff22b1cf700 1 -- 172.21.15.88:0/1643207218 shutdown_connections 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.770+0000 7ff22b1cf700 1 --2- 172.21.15.88:0/1643207218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2240ffbd0 0x7ff224100040 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.770+0000 7ff22b1cf700 1 --2- 172.21.15.88:0/1643207218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff224104130 0x7ff224104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.770+0000 7ff22b1cf700 1 -- 172.21.15.88:0/1643207218 >> 172.21.15.88:0/1643207218 conn(0x7ff224075130 msgr2=0x7ff224075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.770+0000 7ff22b1cf700 1 -- 172.21.15.88:0/1643207218 shutdown_connections 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.770+0000 7ff22b1cf700 1 -- 172.21.15.88:0/1643207218 wait complete. 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff22b1cf700 1 Processor -- start 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff22b1cf700 1 -- start start 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff22b1cf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2240ffbd0 0x7ff224195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff22b1cf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff224104130 0x7ff224196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff22b1cf700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff224196a40 con 0x7ff2240ffbd0 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff22b1cf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22419a7d0 con 0x7ff224104130 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff228f6b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2240ffbd0 0x7ff224195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:14.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff228f6b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2240ffbd0 0x7ff224195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36348/0 (socket says 172.21.15.88:36348) 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff228f6b700 1 -- 172.21.15.88:0/3137983171 learned_addr learned my addr 172.21.15.88:0/3137983171 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff228f6b700 1 -- 172.21.15.88:0/3137983171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff224104130 msgr2=0x7ff224196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff228f6b700 1 --2- 172.21.15.88:0/3137983171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff224104130 0x7ff224196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff228f6b700 1 -- 172.21.15.88:0/3137983171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2100095f0 con 0x7ff2240ffbd0 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff228f6b700 1 --2- 172.21.15.88:0/3137983171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2240ffbd0 0x7ff224195e20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff21800d8d0 tx=0x7ff21800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.771+0000 7ff221ffb700 1 -- 172.21.15.88:0/3137983171 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff218009940 con 0x7ff2240ffbd0 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.772+0000 7ff22b1cf700 1 -- 172.21.15.88:0/3137983171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22419aab0 con 0x7ff2240ffbd0 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.772+0000 7ff221ffb700 1 -- 172.21.15.88:0/3137983171 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff218010460 con 0x7ff2240ffbd0 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.772+0000 7ff221ffb700 1 -- 172.21.15.88:0/3137983171 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff21800f5f0 con 0x7ff2240ffbd0 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.772+0000 7ff22b1cf700 1 -- 172.21.15.88:0/3137983171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22419b000 con 0x7ff2240ffbd0 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.773+0000 7ff22b1cf700 1 -- 172.21.15.88:0/3137983171 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2241086a0 con 0x7ff2240ffbd0 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.773+0000 7ff221ffb700 1 -- 172.21.15.88:0/3137983171 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff2180105d0 con 0x7ff2240ffbd0 2024-02-19T19:24:14.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.773+0000 7ff221ffb700 1 --2- 172.21.15.88:0/3137983171 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff21406d8a0 0x7ff21406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:14.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.773+0000 7ff221ffb700 1 -- 172.21.15.88:0/3137983171 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff21808d540 con 0x7ff2240ffbd0 2024-02-19T19:24:14.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.773+0000 7ff223fff700 1 --2- 172.21.15.88:0/3137983171 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff21406d8a0 0x7ff21406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:14.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.774+0000 7ff223fff700 1 --2- 172.21.15.88:0/3137983171 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff21406d8a0 0x7ff21406fd50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff21000b5c0 tx=0x7ff210005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:14.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.778+0000 7ff221ffb700 1 -- 172.21.15.88:0/3137983171 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff218056cf0 con 0x7ff2240ffbd0 2024-02-19T19:24:14.842 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:14 smithi132 ceph-mon[70676]: from='client.35142 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:14.842 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:14 smithi132 ceph-mon[70676]: from='client.35146 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:14.842 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:14 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3816084459' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:24:14.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.992+0000 7ff22b1cf700 1 -- 172.21.15.88:0/3137983171 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff22419b220 con 0x7ff21406d8a0 2024-02-19T19:24:15.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.993+0000 7ff221ffb700 1 -- 172.21.15.88:0/3137983171 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff22419b220 con 0x7ff21406d8a0 2024-02-19T19:24:15.000 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:24:15.001 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:24:15.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.995+0000 7ff22b1cf700 1 -- 172.21.15.88:0/3137983171 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff21406d8a0 msgr2=0x7ff21406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:15.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.995+0000 7ff22b1cf700 1 --2- 172.21.15.88:0/3137983171 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff21406d8a0 0x7ff21406fd50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff21000b5c0 tx=0x7ff210005e20 comp rx=0 tx=0).stop 2024-02-19T19:24:15.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.995+0000 7ff22b1cf700 1 -- 172.21.15.88:0/3137983171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2240ffbd0 msgr2=0x7ff224195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:15.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.995+0000 7ff22b1cf700 1 --2- 172.21.15.88:0/3137983171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2240ffbd0 0x7ff224195e20 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff21800d8d0 tx=0x7ff21800dc90 comp rx=0 tx=0).stop 2024-02-19T19:24:15.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.995+0000 7ff22b1cf700 1 -- 172.21.15.88:0/3137983171 shutdown_connections 2024-02-19T19:24:15.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.995+0000 7ff22b1cf700 1 --2- 172.21.15.88:0/3137983171 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff21406d8a0 0x7ff21406fd50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:15.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.995+0000 7ff22b1cf700 1 --2- 172.21.15.88:0/3137983171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2240ffbd0 0x7ff224195e20 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:15.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.995+0000 7ff22b1cf700 1 --2- 172.21.15.88:0/3137983171 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff224104130 0x7ff224196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:15.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.995+0000 7ff22b1cf700 1 -- 172.21.15.88:0/3137983171 >> 172.21.15.88:0/3137983171 conn(0x7ff224075130 msgr2=0x7ff2240fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:15.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.995+0000 7ff22b1cf700 1 -- 172.21.15.88:0/3137983171 shutdown_connections 2024-02-19T19:24:15.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:14.995+0000 7ff22b1cf700 1 -- 172.21.15.88:0/3137983171 wait complete. 2024-02-19T19:24:15.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.099+0000 7f2f70f2f700 1 -- 172.21.15.88:0/961512089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f6c0ff3a0 msgr2=0x7f2f6c0ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:15.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.099+0000 7f2f70f2f700 1 --2- 172.21.15.88:0/961512089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f6c0ff3a0 0x7f2f6c0ff770 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c009910 tx=0x7f2f5c009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:15.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.100+0000 7f2f70f2f700 1 -- 172.21.15.88:0/961512089 shutdown_connections 2024-02-19T19:24:15.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.100+0000 7f2f70f2f700 1 --2- 172.21.15.88:0/961512089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0ffd40 0x7f2f6c103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:15.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.100+0000 7f2f70f2f700 1 --2- 172.21.15.88:0/961512089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f6c0ff3a0 0x7f2f6c0ff770 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:15.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.100+0000 7f2f70f2f700 1 -- 172.21.15.88:0/961512089 >> 172.21.15.88:0/961512089 conn(0x7f2f6c0fac40 msgr2=0x7f2f6c0fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:15.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.100+0000 7f2f70f2f700 1 -- 172.21.15.88:0/961512089 shutdown_connections 2024-02-19T19:24:15.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.100+0000 7f2f70f2f700 1 -- 172.21.15.88:0/961512089 wait complete. 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.100+0000 7f2f70f2f700 1 Processor -- start 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.100+0000 7f2f70f2f700 1 -- start start 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f70f2f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0ff3a0 0x7f2f6c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f70f2f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f6c0ffd40 0x7f2f6c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f70f2f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c196bb0 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f70f2f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c19a940 con 0x7f2f6c0ff3a0 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f69d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f6c0ffd40 0x7f2f6c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f69d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f6c0ffd40 0x7f2f6c1964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36362/0 (socket says 172.21.15.88:36362) 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f69d9b700 1 -- 172.21.15.88:0/1007566335 learned_addr learned my addr 172.21.15.88:0/1007566335 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f6a59c700 1 --2- 172.21.15.88:0/1007566335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0ff3a0 0x7f2f6c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f69d9b700 1 -- 172.21.15.88:0/1007566335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0ff3a0 msgr2=0x7f2f6c195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:15.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f69d9b700 1 --2- 172.21.15.88:0/1007566335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0ff3a0 0x7f2f6c195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:15.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f69d9b700 1 -- 172.21.15.88:0/1007566335 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5c0095f0 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f69d9b700 1 --2- 172.21.15.88:0/1007566335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f6c0ffd40 0x7f2f6c1964d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000b480 tx=0x7f2f6000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:15.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f5b7fe700 1 -- 172.21.15.88:0/1007566335 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f60011800 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f5b7fe700 1 -- 172.21.15.88:0/1007566335 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f60011e20 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f70f2f700 1 -- 172.21.15.88:0/1007566335 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6c19ac20 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.101+0000 7f2f5b7fe700 1 -- 172.21.15.88:0/1007566335 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f60010ea0 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.102+0000 7f2f70f2f700 1 -- 172.21.15.88:0/1007566335 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c19b170 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.102+0000 7f2f70f2f700 1 -- 172.21.15.88:0/1007566335 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6c04f2a0 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.103+0000 7f2f5b7fe700 1 -- 172.21.15.88:0/1007566335 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2f60010380 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.103+0000 7f2f5b7fe700 1 --2- 172.21.15.88:0/1007566335 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f5406d8a0 0x7f2f5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:15.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.103+0000 7f2f5b7fe700 1 -- 172.21.15.88:0/1007566335 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2f6008d000 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.103+0000 7f2f6a59c700 1 --2- 172.21.15.88:0/1007566335 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f5406d8a0 0x7f2f5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:15.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.104+0000 7f2f6a59c700 1 --2- 172.21.15.88:0/1007566335 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f5406d8a0 0x7f2f5406fd50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c006010 tx=0x7f2f5c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:15.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.108+0000 7f2f5b7fe700 1 -- 172.21.15.88:0/1007566335 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f600567b0 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.386+0000 7f2f70f2f700 1 -- 172.21.15.88:0/1007566335 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2f6c1972f0 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.386+0000 7f2f5b7fe700 1 -- 172.21.15.88:0/1007566335 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f2f60059e70 con 0x7f2f6c0ffd40 2024-02-19T19:24:15.394 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:24:15.394 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:24:15.394 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:24:15.394 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:24:15.394 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:24:15.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.389+0000 7f2f70f2f700 1 -- 172.21.15.88:0/1007566335 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f5406d8a0 msgr2=0x7f2f5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:15.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.389+0000 7f2f70f2f700 1 --2- 172.21.15.88:0/1007566335 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f5406d8a0 0x7f2f5406fd50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c006010 tx=0x7f2f5c01a040 comp rx=0 tx=0).stop 2024-02-19T19:24:15.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.389+0000 7f2f70f2f700 1 -- 172.21.15.88:0/1007566335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f6c0ffd40 msgr2=0x7f2f6c1964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:15.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.389+0000 7f2f70f2f700 1 --2- 172.21.15.88:0/1007566335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f6c0ffd40 0x7f2f6c1964d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2f6000b480 tx=0x7f2f6000b790 comp rx=0 tx=0).stop 2024-02-19T19:24:15.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.389+0000 7f2f70f2f700 1 -- 172.21.15.88:0/1007566335 shutdown_connections 2024-02-19T19:24:15.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.389+0000 7f2f70f2f700 1 --2- 172.21.15.88:0/1007566335 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f5406d8a0 0x7f2f5406fd50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:15.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.389+0000 7f2f70f2f700 1 --2- 172.21.15.88:0/1007566335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f6c0ff3a0 0x7f2f6c195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:15.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.389+0000 7f2f70f2f700 1 --2- 172.21.15.88:0/1007566335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f6c0ffd40 0x7f2f6c1964d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:15.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.389+0000 7f2f70f2f700 1 -- 172.21.15.88:0/1007566335 >> 172.21.15.88:0/1007566335 conn(0x7f2f6c0fac40 msgr2=0x7f2f6c103520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:15.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.389+0000 7f2f70f2f700 1 -- 172.21.15.88:0/1007566335 shutdown_connections 2024-02-19T19:24:15.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:15.389+0000 7f2f70f2f700 1 -- 172.21.15.88:0/1007566335 wait complete. 2024-02-19T19:24:15.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:15 smithi088 ceph-mon[88643]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:15.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:15 smithi088 ceph-mon[88643]: pgmap v619: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:15.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:15 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2034134170' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:24:15.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:15 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1007566335' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:24:15.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:15 smithi132 ceph-mon[70676]: from='client.35150 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:15.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:15 smithi132 ceph-mon[70676]: pgmap v619: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:15.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:15 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2034134170' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:24:15.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:15 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1007566335' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:24:16.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:16 smithi088 ceph-mon[88643]: from='client.35162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:16.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:16 smithi132 ceph-mon[70676]: from='client.35162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:17.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:17 smithi088 ceph-mon[88643]: pgmap v620: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:17.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:17 smithi132 ceph-mon[70676]: pgmap v620: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:19.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:19 smithi088 ceph-mon[88643]: pgmap v621: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:19.895 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:19 smithi132 ceph-mon[70676]: pgmap v621: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:21.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:21 smithi088 ceph-mon[88643]: pgmap v622: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:21.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:21 smithi132 ceph-mon[70676]: pgmap v622: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:23.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:23 smithi088 ceph-mon[88643]: pgmap v623: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:23.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:23 smithi132 ceph-mon[70676]: pgmap v623: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:25.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:25 smithi088 ceph-mon[88643]: pgmap v624: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:25.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:25 smithi132 ceph-mon[70676]: pgmap v624: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:27.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:27 smithi088 ceph-mon[88643]: pgmap v625: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:27.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:27 smithi132 ceph-mon[70676]: pgmap v625: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:28.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:24:28.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:24:29.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:29 smithi088 ceph-mon[88643]: pgmap v626: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:29.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:29 smithi132 ceph-mon[70676]: pgmap v626: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:31.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:31 smithi088 ceph-mon[88643]: pgmap v627: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:31.990 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:31 smithi132 ceph-mon[70676]: pgmap v627: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:33.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:33 smithi088 ceph-mon[88643]: pgmap v628: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:33.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:33 smithi132 ceph-mon[70676]: pgmap v628: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:35.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:35 smithi088 ceph-mon[88643]: pgmap v629: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:35.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:35 smithi132 ceph-mon[70676]: pgmap v629: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:37.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:37 smithi088 ceph-mon[88643]: pgmap v630: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:37.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:37 smithi132 ceph-mon[70676]: pgmap v630: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:39.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:39 smithi088 ceph-mon[88643]: pgmap v631: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:39.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:39 smithi132 ceph-mon[70676]: pgmap v631: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:41.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:41 smithi088 ceph-mon[88643]: pgmap v632: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:41.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:41 smithi132 ceph-mon[70676]: pgmap v632: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:43.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:43 smithi088 ceph-mon[88643]: pgmap v633: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:43.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:24:43.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:43 smithi132 ceph-mon[70676]: pgmap v633: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:43.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:24:45.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.506+0000 7fb8672bc700 1 -- 172.21.15.88:0/337251703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8600730f0 msgr2=0x7fb8600734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:45.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.506+0000 7fb8672bc700 1 --2- 172.21.15.88:0/337251703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8600730f0 0x7fb8600734c0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009910 tx=0x7fb85c009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:45.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.506+0000 7fb8672bc700 1 -- 172.21.15.88:0/337251703 shutdown_connections 2024-02-19T19:24:45.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.506+0000 7fb8672bc700 1 --2- 172.21.15.88:0/337251703 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb860073a00 0x7fb86010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:45.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.506+0000 7fb8672bc700 1 --2- 172.21.15.88:0/337251703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8600730f0 0x7fb8600734c0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:45.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.506+0000 7fb8672bc700 1 -- 172.21.15.88:0/337251703 >> 172.21.15.88:0/337251703 conn(0x7fb8600fbda0 msgr2=0x7fb8600fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:45.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.506+0000 7fb8672bc700 1 -- 172.21.15.88:0/337251703 shutdown_connections 2024-02-19T19:24:45.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.506+0000 7fb8672bc700 1 -- 172.21.15.88:0/337251703 wait complete. 2024-02-19T19:24:45.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb8672bc700 1 Processor -- start 2024-02-19T19:24:45.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb8672bc700 1 -- start start 2024-02-19T19:24:45.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb8672bc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8600730f0 0x7fb860197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:45.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb8672bc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860073a00 0x7fb860198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:45.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb8672bc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86019c970 con 0x7fb860073a00 2024-02-19T19:24:45.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb8672bc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb86019cae0 con 0x7fb8600730f0 2024-02-19T19:24:45.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb864857700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860073a00 0x7fb860198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:45.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb864857700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860073a00 0x7fb860198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41146/0 (socket says 172.21.15.88:41146) 2024-02-19T19:24:45.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb864857700 1 -- 172.21.15.88:0/2264145512 learned_addr learned my addr 172.21.15.88:0/2264145512 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:45.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb864857700 1 -- 172.21.15.88:0/2264145512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8600730f0 msgr2=0x7fb860197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:45.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb865058700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8600730f0 0x7fb860197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:45.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.507+0000 7fb864857700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8600730f0 0x7fb860197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:45.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.508+0000 7fb864857700 1 -- 172.21.15.88:0/2264145512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c0095f0 con 0x7fb860073a00 2024-02-19T19:24:45.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.508+0000 7fb865058700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8600730f0 0x7fb860197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:24:45.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.508+0000 7fb864857700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860073a00 0x7fb860198630 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb85000d8d0 tx=0x7fb85000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:45.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.508+0000 7fb8567fc700 1 -- 172.21.15.88:0/2264145512 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb850009940 con 0x7fb860073a00 2024-02-19T19:24:45.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.508+0000 7fb8567fc700 1 -- 172.21.15.88:0/2264145512 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb850010460 con 0x7fb860073a00 2024-02-19T19:24:45.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.508+0000 7fb8567fc700 1 -- 172.21.15.88:0/2264145512 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb85000f5f0 con 0x7fb860073a00 2024-02-19T19:24:45.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.508+0000 7fb8672bc700 1 -- 172.21.15.88:0/2264145512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb86019cdc0 con 0x7fb860073a00 2024-02-19T19:24:45.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.508+0000 7fb8672bc700 1 -- 172.21.15.88:0/2264145512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86019d2c0 con 0x7fb860073a00 2024-02-19T19:24:45.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.509+0000 7fb8672bc700 1 -- 172.21.15.88:0/2264145512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb860109c80 con 0x7fb860073a00 2024-02-19T19:24:45.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.509+0000 7fb8567fc700 1 -- 172.21.15.88:0/2264145512 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb8500105d0 con 0x7fb860073a00 2024-02-19T19:24:45.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.510+0000 7fb8567fc700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb84c06d8a0 0x7fb84c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:45.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.510+0000 7fb8567fc700 1 -- 172.21.15.88:0/2264145512 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb85008d5a0 con 0x7fb860073a00 2024-02-19T19:24:45.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.510+0000 7fb865058700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb84c06d8a0 0x7fb84c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:45.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.510+0000 7fb865058700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb84c06d8a0 0x7fb84c06fd50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00b5c0 tx=0x7fb85c0053a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:45.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.515+0000 7fb8567fc700 1 -- 172.21.15.88:0/2264145512 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb850056cf0 con 0x7fb860073a00 2024-02-19T19:24:45.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.732+0000 7fb8672bc700 1 -- 172.21.15.88:0/2264145512 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb860199390 con 0x7fb84c06d8a0 2024-02-19T19:24:45.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.734+0000 7fb8567fc700 1 -- 172.21.15.88:0/2264145512 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb860199390 con 0x7fb84c06d8a0 2024-02-19T19:24:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.736+0000 7fb8672bc700 1 -- 172.21.15.88:0/2264145512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb84c06d8a0 msgr2=0x7fb84c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.736+0000 7fb8672bc700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb84c06d8a0 0x7fb84c06fd50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00b5c0 tx=0x7fb85c0053a0 comp rx=0 tx=0).stop 2024-02-19T19:24:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.736+0000 7fb8672bc700 1 -- 172.21.15.88:0/2264145512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860073a00 msgr2=0x7fb860198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.736+0000 7fb8672bc700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860073a00 0x7fb860198630 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb85000d8d0 tx=0x7fb85000dc90 comp rx=0 tx=0).stop 2024-02-19T19:24:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.736+0000 7fb8672bc700 1 -- 172.21.15.88:0/2264145512 shutdown_connections 2024-02-19T19:24:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.736+0000 7fb8672bc700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb84c06d8a0 0x7fb84c06fd50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.736+0000 7fb8672bc700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8600730f0 0x7fb860197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.736+0000 7fb8672bc700 1 --2- 172.21.15.88:0/2264145512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860073a00 0x7fb860198630 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.736+0000 7fb8672bc700 1 -- 172.21.15.88:0/2264145512 >> 172.21.15.88:0/2264145512 conn(0x7fb8600fbda0 msgr2=0x7fb860106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.736+0000 7fb8672bc700 1 -- 172.21.15.88:0/2264145512 shutdown_connections 2024-02-19T19:24:45.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.736+0000 7fb8672bc700 1 -- 172.21.15.88:0/2264145512 wait complete. 2024-02-19T19:24:45.753 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:24:45.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:45 smithi088 ceph-mon[88643]: pgmap v634: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:45.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.841+0000 7f2c8b440700 1 -- 172.21.15.88:0/2358811005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c840ffd50 msgr2=0x7f2c84103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:45.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.841+0000 7f2c8b440700 1 --2- 172.21.15.88:0/2358811005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c840ffd50 0x7f2c84103d20 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009910 tx=0x7f2c74009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:45.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.841+0000 7f2c8b440700 1 -- 172.21.15.88:0/2358811005 shutdown_connections 2024-02-19T19:24:45.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.841+0000 7f2c8b440700 1 --2- 172.21.15.88:0/2358811005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c840ffd50 0x7f2c84103d20 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:45.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.841+0000 7f2c8b440700 1 --2- 172.21.15.88:0/2358811005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c840ff3b0 0x7f2c840ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:45.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.841+0000 7f2c8b440700 1 -- 172.21.15.88:0/2358811005 >> 172.21.15.88:0/2358811005 conn(0x7f2c840fac90 msgr2=0x7f2c840fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:45.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.841+0000 7f2c8b440700 1 -- 172.21.15.88:0/2358811005 shutdown_connections 2024-02-19T19:24:45.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.841+0000 7f2c8b440700 1 -- 172.21.15.88:0/2358811005 wait complete. 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c8b440700 1 Processor -- start 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c8b440700 1 -- start start 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c8b440700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c840ff3b0 0x7f2c84196e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c8b440700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c840ffd50 0x7f2c84197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c8b440700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c841979a0 con 0x7f2c840ffd50 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c8b440700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c84197ae0 con 0x7f2c840ff3b0 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c889db700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c840ffd50 0x7f2c84197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c889db700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c840ffd50 0x7f2c84197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41160/0 (socket says 172.21.15.88:41160) 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c889db700 1 -- 172.21.15.88:0/302747700 learned_addr learned my addr 172.21.15.88:0/302747700 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c889db700 1 -- 172.21.15.88:0/302747700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c840ff3b0 msgr2=0x7f2c84196e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c889db700 1 --2- 172.21.15.88:0/302747700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c840ff3b0 0x7f2c84196e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c889db700 1 -- 172.21.15.88:0/302747700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740095f0 con 0x7f2c840ffd50 2024-02-19T19:24:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.842+0000 7f2c889db700 1 --2- 172.21.15.88:0/302747700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c840ffd50 0x7f2c84197380 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2c74005f50 tx=0x7f2c7400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.843+0000 7f2c7a7fc700 1 -- 172.21.15.88:0/302747700 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c7401d070 con 0x7f2c840ffd50 2024-02-19T19:24:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.843+0000 7f2c7a7fc700 1 -- 172.21.15.88:0/302747700 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c7400f460 con 0x7f2c840ffd50 2024-02-19T19:24:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.843+0000 7f2c8b440700 1 -- 172.21.15.88:0/302747700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8419feb0 con 0x7f2c840ffd50 2024-02-19T19:24:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.843+0000 7f2c7a7fc700 1 -- 172.21.15.88:0/302747700 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c74017630 con 0x7f2c840ffd50 2024-02-19T19:24:45.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.843+0000 7f2c8b440700 1 -- 172.21.15.88:0/302747700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c841a02c0 con 0x7f2c840ffd50 2024-02-19T19:24:45.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.844+0000 7f2c8b440700 1 -- 172.21.15.88:0/302747700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c84107670 con 0x7f2c840ffd50 2024-02-19T19:24:45.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.844+0000 7f2c7a7fc700 1 -- 172.21.15.88:0/302747700 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2c7400f5d0 con 0x7f2c840ffd50 2024-02-19T19:24:45.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.845+0000 7f2c7a7fc700 1 --2- 172.21.15.88:0/302747700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2c7006d8a0 0x7f2c7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:45.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.845+0000 7f2c891dc700 1 --2- 172.21.15.88:0/302747700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2c7006d8a0 0x7f2c7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:45.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.845+0000 7f2c7a7fc700 1 -- 172.21.15.88:0/302747700 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2c7408ead0 con 0x7f2c840ffd50 2024-02-19T19:24:45.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.845+0000 7f2c891dc700 1 --2- 172.21.15.88:0/302747700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2c7006d8a0 0x7f2c7006fd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2c800097b0 tx=0x7f2c80006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:45.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:45.850+0000 7f2c7a7fc700 1 -- 172.21.15.88:0/302747700 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2c74058280 con 0x7f2c840ffd50 2024-02-19T19:24:45.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:45 smithi132 ceph-mon[70676]: pgmap v634: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:46.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.066+0000 7f2c8b440700 1 -- 172.21.15.88:0/302747700 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c84061190 con 0x7f2c7006d8a0 2024-02-19T19:24:46.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.067+0000 7f2c7a7fc700 1 -- 172.21.15.88:0/302747700 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2c84061190 con 0x7f2c7006d8a0 2024-02-19T19:24:46.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.069+0000 7f2c8b440700 1 -- 172.21.15.88:0/302747700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2c7006d8a0 msgr2=0x7f2c7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.069+0000 7f2c8b440700 1 --2- 172.21.15.88:0/302747700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2c7006d8a0 0x7f2c7006fd50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2c800097b0 tx=0x7f2c80006d20 comp rx=0 tx=0).stop 2024-02-19T19:24:46.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.069+0000 7f2c8b440700 1 -- 172.21.15.88:0/302747700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c840ffd50 msgr2=0x7f2c84197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.069+0000 7f2c8b440700 1 --2- 172.21.15.88:0/302747700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c840ffd50 0x7f2c84197380 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f2c74005f50 tx=0x7f2c7400be20 comp rx=0 tx=0).stop 2024-02-19T19:24:46.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.069+0000 7f2c8b440700 1 -- 172.21.15.88:0/302747700 shutdown_connections 2024-02-19T19:24:46.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.069+0000 7f2c8b440700 1 --2- 172.21.15.88:0/302747700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2c7006d8a0 0x7f2c7006fd50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.069+0000 7f2c8b440700 1 --2- 172.21.15.88:0/302747700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2c840ff3b0 0x7f2c84196e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.069+0000 7f2c8b440700 1 --2- 172.21.15.88:0/302747700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c840ffd50 0x7f2c84197380 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.069+0000 7f2c8b440700 1 -- 172.21.15.88:0/302747700 >> 172.21.15.88:0/302747700 conn(0x7f2c840fac90 msgr2=0x7f2c840fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:46.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.070+0000 7f2c8b440700 1 -- 172.21.15.88:0/302747700 shutdown_connections 2024-02-19T19:24:46.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.070+0000 7f2c8b440700 1 -- 172.21.15.88:0/302747700 wait complete. 2024-02-19T19:24:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.173+0000 7f431b799700 1 -- 172.21.15.88:0/710639191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43141003b0 msgr2=0x7f4314100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.173+0000 7f431b799700 1 --2- 172.21.15.88:0/710639191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43141003b0 0x7f4314100780 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f4304009910 tx=0x7f4304009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.173+0000 7f431b799700 1 -- 172.21.15.88:0/710639191 shutdown_connections 2024-02-19T19:24:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.173+0000 7f431b799700 1 --2- 172.21.15.88:0/710639191 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4314100d50 0x7f4314104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.173+0000 7f431b799700 1 --2- 172.21.15.88:0/710639191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43141003b0 0x7f4314100780 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.173+0000 7f431b799700 1 -- 172.21.15.88:0/710639191 >> 172.21.15.88:0/710639191 conn(0x7f43140fbc90 msgr2=0x7f43140fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.173+0000 7f431b799700 1 -- 172.21.15.88:0/710639191 shutdown_connections 2024-02-19T19:24:46.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.173+0000 7f431b799700 1 -- 172.21.15.88:0/710639191 wait complete. 2024-02-19T19:24:46.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f431b799700 1 Processor -- start 2024-02-19T19:24:46.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f431b799700 1 -- start start 2024-02-19T19:24:46.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f431b799700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43141003b0 0x7f431419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:46.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f431b799700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4314100d50 0x7f431419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:46.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f431b799700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431419c690 con 0x7f4314100d50 2024-02-19T19:24:46.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f431b799700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4314195b40 con 0x7f43141003b0 2024-02-19T19:24:46.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f4318d34700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4314100d50 0x7f431419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:46.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f4318d34700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4314100d50 0x7f431419c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41180/0 (socket says 172.21.15.88:41180) 2024-02-19T19:24:46.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f4318d34700 1 -- 172.21.15.88:0/170778336 learned_addr learned my addr 172.21.15.88:0/170778336 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:46.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f4319535700 1 --2- 172.21.15.88:0/170778336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43141003b0 0x7f431419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:46.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f4318d34700 1 -- 172.21.15.88:0/170778336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43141003b0 msgr2=0x7f431419bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f4318d34700 1 --2- 172.21.15.88:0/170778336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43141003b0 0x7f431419bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.174+0000 7f4318d34700 1 -- 172.21.15.88:0/170778336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43040095f0 con 0x7f4314100d50 2024-02-19T19:24:46.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.175+0000 7f4318d34700 1 --2- 172.21.15.88:0/170778336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4314100d50 0x7f431419c000 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f431000d900 tx=0x7f431000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:46.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.175+0000 7f430a7fc700 1 -- 172.21.15.88:0/170778336 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4310009900 con 0x7f4314100d50 2024-02-19T19:24:46.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.175+0000 7f430a7fc700 1 -- 172.21.15.88:0/170778336 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4310010460 con 0x7f4314100d50 2024-02-19T19:24:46.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.175+0000 7f430a7fc700 1 -- 172.21.15.88:0/170778336 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f431000f5f0 con 0x7f4314100d50 2024-02-19T19:24:46.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.175+0000 7f431b799700 1 -- 172.21.15.88:0/170778336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4314195e20 con 0x7f4314100d50 2024-02-19T19:24:46.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.175+0000 7f431b799700 1 -- 172.21.15.88:0/170778336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4314196370 con 0x7f4314100d50 2024-02-19T19:24:46.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.176+0000 7f431b799700 1 -- 172.21.15.88:0/170778336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43141086c0 con 0x7f4314100d50 2024-02-19T19:24:46.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.177+0000 7f430a7fc700 1 -- 172.21.15.88:0/170778336 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f43100105d0 con 0x7f4314100d50 2024-02-19T19:24:46.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.177+0000 7f430a7fc700 1 --2- 172.21.15.88:0/170778336 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f430006d8a0 0x7f430006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:46.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.178+0000 7f430a7fc700 1 -- 172.21.15.88:0/170778336 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f431008cfd0 con 0x7f4314100d50 2024-02-19T19:24:46.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.178+0000 7f4319535700 1 --2- 172.21.15.88:0/170778336 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f430006d8a0 0x7f430006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:46.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.178+0000 7f4319535700 1 --2- 172.21.15.88:0/170778336 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f430006d8a0 0x7f430006fd50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4304006010 tx=0x7f430401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:46.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.182+0000 7f430a7fc700 1 -- 172.21.15.88:0/170778336 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f431005a330 con 0x7f4314100d50 2024-02-19T19:24:46.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.390+0000 7f431b799700 1 -- 172.21.15.88:0/170778336 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f43141966a0 con 0x7f430006d8a0 2024-02-19T19:24:46.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.401+0000 7f430a7fc700 1 -- 172.21.15.88:0/170778336 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f43141966a0 con 0x7f430006d8a0 2024-02-19T19:24:46.408 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:24:46.408 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (21m) 13m ago 32m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:24:46.408 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (32m) 13m ago 32m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:24:46.408 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (14m) 13m ago 30m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (18m) 13m ago 32m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (17m) 13m ago 30m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (21m) 13m ago 31m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (14m) 13m ago 26m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (13m) 13m ago 26m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (13m) 13m ago 26m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (25m) 13m ago 25m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (23m) 13m ago 34m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (22m) 13m ago 30m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (20m) 13m ago 34m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (19m) 13m ago 30m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (22m) 13m ago 32m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (22m) 13m ago 30m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (17m) 13m ago 29m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:24:46.409 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (16m) 13m ago 29m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:24:46.410 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (16m) 13m ago 28m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:24:46.410 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (15m) 13m ago 28m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:24:46.410 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (15m) 13m ago 28m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:24:46.410 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (14m) 13m ago 27m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:24:46.410 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (22m) 13m ago 31m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:24:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.403+0000 7f431b799700 1 -- 172.21.15.88:0/170778336 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f430006d8a0 msgr2=0x7f430006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.403+0000 7f431b799700 1 --2- 172.21.15.88:0/170778336 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f430006d8a0 0x7f430006fd50 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4304006010 tx=0x7f430401a040 comp rx=0 tx=0).stop 2024-02-19T19:24:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.403+0000 7f431b799700 1 -- 172.21.15.88:0/170778336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4314100d50 msgr2=0x7f431419c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.403+0000 7f431b799700 1 --2- 172.21.15.88:0/170778336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4314100d50 0x7f431419c000 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f431000d900 tx=0x7f431000dc10 comp rx=0 tx=0).stop 2024-02-19T19:24:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.403+0000 7f431b799700 1 -- 172.21.15.88:0/170778336 shutdown_connections 2024-02-19T19:24:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.403+0000 7f431b799700 1 --2- 172.21.15.88:0/170778336 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f430006d8a0 0x7f430006fd50 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.403+0000 7f431b799700 1 --2- 172.21.15.88:0/170778336 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43141003b0 0x7f431419bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.403+0000 7f431b799700 1 --2- 172.21.15.88:0/170778336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4314100d50 0x7f431419c000 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.403+0000 7f431b799700 1 -- 172.21.15.88:0/170778336 >> 172.21.15.88:0/170778336 conn(0x7f43140fbc90 msgr2=0x7f43140fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.403+0000 7f431b799700 1 -- 172.21.15.88:0/170778336 shutdown_connections 2024-02-19T19:24:46.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.403+0000 7f431b799700 1 -- 172.21.15.88:0/170778336 wait complete. 2024-02-19T19:24:46.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.507+0000 7fb8fc462700 1 -- 172.21.15.88:0/3939066938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f40ff3c0 msgr2=0x7fb8f40ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.507+0000 7fb8fc462700 1 --2- 172.21.15.88:0/3939066938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f40ff3c0 0x7fb8f40ff790 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb8f0009910 tx=0x7fb8f0009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:46.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.507+0000 7fb8fc462700 1 -- 172.21.15.88:0/3939066938 shutdown_connections 2024-02-19T19:24:46.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.507+0000 7fb8fc462700 1 --2- 172.21.15.88:0/3939066938 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8f40ffd60 0x7fb8f4103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.507+0000 7fb8fc462700 1 --2- 172.21.15.88:0/3939066938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f40ff3c0 0x7fb8f40ff790 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.507+0000 7fb8fc462700 1 -- 172.21.15.88:0/3939066938 >> 172.21.15.88:0/3939066938 conn(0x7fb8f40fac80 msgr2=0x7fb8f40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:46.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.507+0000 7fb8fc462700 1 -- 172.21.15.88:0/3939066938 shutdown_connections 2024-02-19T19:24:46.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.507+0000 7fb8fc462700 1 -- 172.21.15.88:0/3939066938 wait complete. 2024-02-19T19:24:46.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fc462700 1 Processor -- start 2024-02-19T19:24:46.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fc462700 1 -- start start 2024-02-19T19:24:46.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fc462700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f40ff3c0 0x7fb8f4194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:46.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fc462700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8f40ffd60 0x7fb8f41952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:46.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fc462700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f4195910 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fc462700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f4195a50 con 0x7fb8f40ffd60 2024-02-19T19:24:46.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fa1fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f40ff3c0 0x7fb8f4194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:46.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fa1fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f40ff3c0 0x7fb8f4194d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41198/0 (socket says 172.21.15.88:41198) 2024-02-19T19:24:46.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fa1fe700 1 -- 172.21.15.88:0/3844962961 learned_addr learned my addr 172.21.15.88:0/3844962961 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:46.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fa1fe700 1 -- 172.21.15.88:0/3844962961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8f40ffd60 msgr2=0x7fb8f41952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fa1fe700 1 --2- 172.21.15.88:0/3844962961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8f40ffd60 0x7fb8f41952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.508+0000 7fb8fa1fe700 1 -- 172.21.15.88:0/3844962961 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f00095f0 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.509+0000 7fb8fa1fe700 1 --2- 172.21.15.88:0/3844962961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f40ff3c0 0x7fb8f4194d80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000b5c0 tx=0x7fb8f000be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:46.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.509+0000 7fb8eb7fe700 1 -- 172.21.15.88:0/3844962961 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8f001d070 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.509+0000 7fb8eb7fe700 1 -- 172.21.15.88:0/3844962961 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8f000f460 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.509+0000 7fb8eb7fe700 1 -- 172.21.15.88:0/3844962961 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8f0017610 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.509+0000 7fb8fc462700 1 -- 172.21.15.88:0/3844962961 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f4199840 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.509+0000 7fb8fc462700 1 -- 172.21.15.88:0/3844962961 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f4199d30 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.510+0000 7fb8fc462700 1 -- 172.21.15.88:0/3844962961 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f404f2a0 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.510+0000 7fb8eb7fe700 1 -- 172.21.15.88:0/3844962961 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb8f000f5d0 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.511+0000 7fb8eb7fe700 1 --2- 172.21.15.88:0/3844962961 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8e006fab0 0x7fb8e0071f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:46.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.511+0000 7fb8eb7fe700 1 -- 172.21.15.88:0/3844962961 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb8f008eaa0 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.511+0000 7fb8f99fd700 1 --2- 172.21.15.88:0/3844962961 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8e006fab0 0x7fb8e0071f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:46.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.511+0000 7fb8f99fd700 1 --2- 172.21.15.88:0/3844962961 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8e006fab0 0x7fb8e0071f60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4005fd0 tx=0x7fb8e4005d00 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:46.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.516+0000 7fb8eb7fe700 1 -- 172.21.15.88:0/3844962961 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb8f00582d0 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.797+0000 7fb8fc462700 1 -- 172.21.15.88:0/3844962961 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb8f419a0a0 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.797+0000 7fb8eb7fe700 1 -- 172.21.15.88:0/3844962961 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fb8f005b990 con 0x7fb8f40ff3c0 2024-02-19T19:24:46.804 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:24:46.804 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:24:46.805 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:24:46.806 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:24:46.806 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:24:46.806 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:24:46.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.799+0000 7fb8fc462700 1 -- 172.21.15.88:0/3844962961 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8e006fab0 msgr2=0x7fb8e0071f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.799+0000 7fb8fc462700 1 --2- 172.21.15.88:0/3844962961 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8e006fab0 0x7fb8e0071f60 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4005fd0 tx=0x7fb8e4005d00 comp rx=0 tx=0).stop 2024-02-19T19:24:46.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.799+0000 7fb8fc462700 1 -- 172.21.15.88:0/3844962961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f40ff3c0 msgr2=0x7fb8f4194d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.799+0000 7fb8fc462700 1 --2- 172.21.15.88:0/3844962961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f40ff3c0 0x7fb8f4194d80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fb8f000b5c0 tx=0x7fb8f000be00 comp rx=0 tx=0).stop 2024-02-19T19:24:46.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.799+0000 7fb8fc462700 1 -- 172.21.15.88:0/3844962961 shutdown_connections 2024-02-19T19:24:46.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.800+0000 7fb8fc462700 1 --2- 172.21.15.88:0/3844962961 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8e006fab0 0x7fb8e0071f60 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.800+0000 7fb8fc462700 1 --2- 172.21.15.88:0/3844962961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8f40ff3c0 0x7fb8f4194d80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.800+0000 7fb8fc462700 1 --2- 172.21.15.88:0/3844962961 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8f40ffd60 0x7fb8f41952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.800+0000 7fb8fc462700 1 -- 172.21.15.88:0/3844962961 >> 172.21.15.88:0/3844962961 conn(0x7fb8f40fac80 msgr2=0x7fb8f40fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:46.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.800+0000 7fb8fc462700 1 -- 172.21.15.88:0/3844962961 shutdown_connections 2024-02-19T19:24:46.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.800+0000 7fb8fc462700 1 -- 172.21.15.88:0/3844962961 wait complete. 2024-02-19T19:24:46.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.904+0000 7f7d22e20700 1 -- 172.21.15.88:0/3177565583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d1c0ff3c0 msgr2=0x7f7d1c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.904+0000 7f7d22e20700 1 --2- 172.21.15.88:0/3177565583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d1c0ff3c0 0x7f7d1c0ff790 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009910 tx=0x7f7d10009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:46.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.904+0000 7f7d22e20700 1 -- 172.21.15.88:0/3177565583 shutdown_connections 2024-02-19T19:24:46.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.904+0000 7f7d22e20700 1 --2- 172.21.15.88:0/3177565583 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d1c0ffd60 0x7f7d1c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.904+0000 7f7d22e20700 1 --2- 172.21.15.88:0/3177565583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d1c0ff3c0 0x7f7d1c0ff790 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.904+0000 7f7d22e20700 1 -- 172.21.15.88:0/3177565583 >> 172.21.15.88:0/3177565583 conn(0x7f7d1c0fac80 msgr2=0x7f7d1c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:46.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.904+0000 7f7d22e20700 1 -- 172.21.15.88:0/3177565583 shutdown_connections 2024-02-19T19:24:46.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.904+0000 7f7d22e20700 1 -- 172.21.15.88:0/3177565583 wait complete. 2024-02-19T19:24:46.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d22e20700 1 Processor -- start 2024-02-19T19:24:46.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d22e20700 1 -- start start 2024-02-19T19:24:46.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d22e20700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d1c0ff3c0 0x7f7d1c195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d22e20700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d1c0ffd60 0x7f7d1c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d22e20700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c196b90 con 0x7f7d1c0ffd60 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d22e20700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1c19a920 con 0x7f7d1c0ff3c0 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d1bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d1c0ffd60 0x7f7d1c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d1bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d1c0ffd60 0x7f7d1c1964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41226/0 (socket says 172.21.15.88:41226) 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d1bfff700 1 -- 172.21.15.88:0/2252449310 learned_addr learned my addr 172.21.15.88:0/2252449310 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d20bbc700 1 --2- 172.21.15.88:0/2252449310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d1c0ff3c0 0x7f7d1c195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d1bfff700 1 -- 172.21.15.88:0/2252449310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d1c0ff3c0 msgr2=0x7f7d1c195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d1bfff700 1 --2- 172.21.15.88:0/2252449310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d1c0ff3c0 0x7f7d1c195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.905+0000 7f7d1bfff700 1 -- 172.21.15.88:0/2252449310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d100095f0 con 0x7f7d1c0ffd60 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.906+0000 7f7d1bfff700 1 --2- 172.21.15.88:0/2252449310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d1c0ffd60 0x7f7d1c1964b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800b480 tx=0x7f7d0800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.906+0000 7f7d19ffb700 1 -- 172.21.15.88:0/2252449310 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d080118c0 con 0x7f7d1c0ffd60 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.906+0000 7f7d19ffb700 1 -- 172.21.15.88:0/2252449310 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d08010380 con 0x7f7d1c0ffd60 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.906+0000 7f7d22e20700 1 -- 172.21.15.88:0/2252449310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1c19ac00 con 0x7f7d1c0ffd60 2024-02-19T19:24:46.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.906+0000 7f7d19ffb700 1 -- 172.21.15.88:0/2252449310 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d0800f4f0 con 0x7f7d1c0ffd60 2024-02-19T19:24:46.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.906+0000 7f7d22e20700 1 -- 172.21.15.88:0/2252449310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1c19b150 con 0x7f7d1c0ffd60 2024-02-19T19:24:46.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.907+0000 7f7d22e20700 1 -- 172.21.15.88:0/2252449310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1c1086e0 con 0x7f7d1c0ffd60 2024-02-19T19:24:46.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.907+0000 7f7d19ffb700 1 -- 172.21.15.88:0/2252449310 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7d08011a20 con 0x7f7d1c0ffd60 2024-02-19T19:24:46.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.907+0000 7f7d19ffb700 1 --2- 172.21.15.88:0/2252449310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7d0c06d8a0 0x7f7d0c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:46.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.908+0000 7f7d19ffb700 1 -- 172.21.15.88:0/2252449310 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7d0808d120 con 0x7f7d1c0ffd60 2024-02-19T19:24:46.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.908+0000 7f7d20bbc700 1 --2- 172.21.15.88:0/2252449310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7d0c06d8a0 0x7f7d0c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:46.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.908+0000 7f7d20bbc700 1 --2- 172.21.15.88:0/2252449310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7d0c06d8a0 0x7f7d0c06fd50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7d10006010 tx=0x7f7d1000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:46.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:46.912+0000 7f7d19ffb700 1 -- 172.21.15.88:0/2252449310 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7d08056950 con 0x7f7d1c0ffd60 2024-02-19T19:24:47.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.181+0000 7f7d22e20700 1 -- 172.21.15.88:0/2252449310 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7d1c19b420 con 0x7f7d1c0ffd60 2024-02-19T19:24:47.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.181+0000 7f7d19ffb700 1 -- 172.21.15.88:0/2252449310 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7f7d0805a010 con 0x7f7d1c0ffd60 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:24:47.189 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:24:47.190 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:24:47.191 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:24:47.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.184+0000 7f7d22e20700 1 -- 172.21.15.88:0/2252449310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7d0c06d8a0 msgr2=0x7f7d0c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:47.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.184+0000 7f7d22e20700 1 --2- 172.21.15.88:0/2252449310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7d0c06d8a0 0x7f7d0c06fd50 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f7d10006010 tx=0x7f7d1000b540 comp rx=0 tx=0).stop 2024-02-19T19:24:47.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.184+0000 7f7d22e20700 1 -- 172.21.15.88:0/2252449310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d1c0ffd60 msgr2=0x7f7d1c1964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:47.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.184+0000 7f7d22e20700 1 --2- 172.21.15.88:0/2252449310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d1c0ffd60 0x7f7d1c1964b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7d0800b480 tx=0x7f7d0800b840 comp rx=0 tx=0).stop 2024-02-19T19:24:47.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.184+0000 7f7d22e20700 1 -- 172.21.15.88:0/2252449310 shutdown_connections 2024-02-19T19:24:47.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.184+0000 7f7d22e20700 1 --2- 172.21.15.88:0/2252449310 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7d0c06d8a0 0x7f7d0c06fd50 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.184+0000 7f7d22e20700 1 --2- 172.21.15.88:0/2252449310 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7d1c0ff3c0 0x7f7d1c195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.184+0000 7f7d22e20700 1 --2- 172.21.15.88:0/2252449310 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d1c0ffd60 0x7f7d1c1964b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.184+0000 7f7d22e20700 1 -- 172.21.15.88:0/2252449310 >> 172.21.15.88:0/2252449310 conn(0x7f7d1c0fac80 msgr2=0x7f7d1c069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:47.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.184+0000 7f7d22e20700 1 -- 172.21.15.88:0/2252449310 shutdown_connections 2024-02-19T19:24:47.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.184+0000 7f7d22e20700 1 -- 172.21.15.88:0/2252449310 wait complete. 2024-02-19T19:24:47.193 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.289+0000 7f563214f700 1 -- 172.21.15.88:0/792086538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f562c101d20 msgr2=0x7f562c10a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.289+0000 7f563214f700 1 --2- 172.21.15.88:0/792086538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f562c101d20 0x7f562c10a220 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5620009910 tx=0x7f5620009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.289+0000 7f563214f700 1 -- 172.21.15.88:0/792086538 shutdown_connections 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.289+0000 7f563214f700 1 --2- 172.21.15.88:0/792086538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f562c101d20 0x7f562c10a220 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.289+0000 7f563214f700 1 --2- 172.21.15.88:0/792086538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f562c101410 0x7f562c1017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.289+0000 7f563214f700 1 -- 172.21.15.88:0/792086538 >> 172.21.15.88:0/792086538 conn(0x7f562c0fac90 msgr2=0x7f562c0fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.289+0000 7f563214f700 1 -- 172.21.15.88:0/792086538 shutdown_connections 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.289+0000 7f563214f700 1 -- 172.21.15.88:0/792086538 wait complete. 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.289+0000 7f563214f700 1 Processor -- start 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f563214f700 1 -- start start 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f563214f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f562c101410 0x7f562c195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:47.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f563214f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f562c101d20 0x7f562c196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f563214f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c196a60 con 0x7f562c101410 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f563214f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c19a7f0 con 0x7f562c101d20 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f562b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f562c101410 0x7f562c195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f562b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f562c101410 0x7f562c195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41230/0 (socket says 172.21.15.88:41230) 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f562b7fe700 1 -- 172.21.15.88:0/2664227892 learned_addr learned my addr 172.21.15.88:0/2664227892 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f562affd700 1 --2- 172.21.15.88:0/2664227892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f562c101d20 0x7f562c196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f562b7fe700 1 -- 172.21.15.88:0/2664227892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f562c101d20 msgr2=0x7f562c196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f562b7fe700 1 --2- 172.21.15.88:0/2664227892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f562c101d20 0x7f562c196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f562b7fe700 1 -- 172.21.15.88:0/2664227892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200095f0 con 0x7f562c101410 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f562b7fe700 1 --2- 172.21.15.88:0/2664227892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f562c101410 0x7f562c195e40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f561c0060b0 tx=0x7f561c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.290+0000 7f5628ff9700 1 -- 172.21.15.88:0/2664227892 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f561c00dda0 con 0x7f562c101410 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.291+0000 7f5628ff9700 1 -- 172.21.15.88:0/2664227892 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f561c00f040 con 0x7f562c101410 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.291+0000 7f563214f700 1 -- 172.21.15.88:0/2664227892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c19aad0 con 0x7f562c101410 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.291+0000 7f5628ff9700 1 -- 172.21.15.88:0/2664227892 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f561c014790 con 0x7f562c101410 2024-02-19T19:24:47.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.291+0000 7f563214f700 1 -- 172.21.15.88:0/2664227892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c19b020 con 0x7f562c101410 2024-02-19T19:24:47.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.292+0000 7f563214f700 1 -- 172.21.15.88:0/2664227892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c107890 con 0x7f562c101410 2024-02-19T19:24:47.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.292+0000 7f5628ff9700 1 -- 172.21.15.88:0/2664227892 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f561c014980 con 0x7f562c101410 2024-02-19T19:24:47.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.292+0000 7f5628ff9700 1 --2- 172.21.15.88:0/2664227892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f561406d8a0 0x7f561406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:47.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.292+0000 7f5628ff9700 1 -- 172.21.15.88:0/2664227892 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f561c08cfa0 con 0x7f562c101410 2024-02-19T19:24:47.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.292+0000 7f562affd700 1 --2- 172.21.15.88:0/2664227892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f561406d8a0 0x7f561406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:47.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.293+0000 7f562affd700 1 --2- 172.21.15.88:0/2664227892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f561406d8a0 0x7f561406fd50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5620005b40 tx=0x7f562001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:47.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.297+0000 7f5628ff9700 1 -- 172.21.15.88:0/2664227892 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f561c0c0d20 con 0x7f562c101410 2024-02-19T19:24:47.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.513+0000 7f563214f700 1 -- 172.21.15.88:0/2664227892 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f562c19b300 con 0x7f561406d8a0 2024-02-19T19:24:47.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.514+0000 7f5628ff9700 1 -- 172.21.15.88:0/2664227892 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f562c19b300 con 0x7f561406d8a0 2024-02-19T19:24:47.521 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:24:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:24:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:24:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:24:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:24:47.521 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:24:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:24:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:24:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:24:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:24:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:24:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:24:47.522 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:24:47.522 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:24:47.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.517+0000 7f563214f700 1 -- 172.21.15.88:0/2664227892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f561406d8a0 msgr2=0x7f561406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:47.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.517+0000 7f563214f700 1 --2- 172.21.15.88:0/2664227892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f561406d8a0 0x7f561406fd50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f5620005b40 tx=0x7f562001a040 comp rx=0 tx=0).stop 2024-02-19T19:24:47.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.517+0000 7f563214f700 1 -- 172.21.15.88:0/2664227892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f562c101410 msgr2=0x7f562c195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:47.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.517+0000 7f563214f700 1 --2- 172.21.15.88:0/2664227892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f562c101410 0x7f562c195e40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f561c0060b0 tx=0x7f561c00d3b0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.517+0000 7f563214f700 1 -- 172.21.15.88:0/2664227892 shutdown_connections 2024-02-19T19:24:47.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.517+0000 7f563214f700 1 --2- 172.21.15.88:0/2664227892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f561406d8a0 0x7f561406fd50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.517+0000 7f563214f700 1 --2- 172.21.15.88:0/2664227892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f562c101410 0x7f562c195e40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.517+0000 7f563214f700 1 --2- 172.21.15.88:0/2664227892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f562c101d20 0x7f562c196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.517+0000 7f563214f700 1 -- 172.21.15.88:0/2664227892 >> 172.21.15.88:0/2664227892 conn(0x7f562c0fac90 msgr2=0x7f562c1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:47.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.518+0000 7f563214f700 1 -- 172.21.15.88:0/2664227892 shutdown_connections 2024-02-19T19:24:47.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.518+0000 7f563214f700 1 -- 172.21.15.88:0/2664227892 wait complete. 2024-02-19T19:24:47.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.625+0000 7fb8e95c0700 1 -- 172.21.15.88:0/4212954329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8e40ffc60 msgr2=0x7fb8e410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:47.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.625+0000 7fb8e95c0700 1 --2- 172.21.15.88:0/4212954329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8e40ffc60 0x7fb8e410a6d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8009910 tx=0x7fb8d8009c20 comp rx=0 tx=0).stop 2024-02-19T19:24:47.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.625+0000 7fb8e95c0700 1 -- 172.21.15.88:0/4212954329 shutdown_connections 2024-02-19T19:24:47.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.625+0000 7fb8e95c0700 1 --2- 172.21.15.88:0/4212954329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8e40ffc60 0x7fb8e410a6d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.625+0000 7fb8e95c0700 1 --2- 172.21.15.88:0/4212954329 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e40ff350 0x7fb8e40ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.625+0000 7fb8e95c0700 1 -- 172.21.15.88:0/4212954329 >> 172.21.15.88:0/4212954329 conn(0x7fb8e40748b0 msgr2=0x7fb8e4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:47.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.625+0000 7fb8e95c0700 1 -- 172.21.15.88:0/4212954329 shutdown_connections 2024-02-19T19:24:47.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.625+0000 7fb8e95c0700 1 -- 172.21.15.88:0/4212954329 wait complete. 2024-02-19T19:24:47.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.626+0000 7fb8e95c0700 1 Processor -- start 2024-02-19T19:24:47.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.626+0000 7fb8e95c0700 1 -- start start 2024-02-19T19:24:47.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.626+0000 7fb8e95c0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e40ff350 0x7fb8e4195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.626+0000 7fb8e95c0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8e40ffc60 0x7fb8e4196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.626+0000 7fb8e95c0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e4196b50 con 0x7fb8e40ffc60 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.626+0000 7fb8e95c0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e419a8e0 con 0x7fb8e40ff350 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.626+0000 7fb8e27fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8e40ffc60 0x7fb8e4196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.626+0000 7fb8e27fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8e40ffc60 0x7fb8e4196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41240/0 (socket says 172.21.15.88:41240) 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.626+0000 7fb8e27fc700 1 -- 172.21.15.88:0/3962851377 learned_addr learned my addr 172.21.15.88:0/3962851377 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.627+0000 7fb8e2ffd700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e40ff350 0x7fb8e4195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.627+0000 7fb8e27fc700 1 -- 172.21.15.88:0/3962851377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e40ff350 msgr2=0x7fb8e4195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.627+0000 7fb8e27fc700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e40ff350 0x7fb8e4195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.627+0000 7fb8e27fc700 1 -- 172.21.15.88:0/3962851377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d80095f0 con 0x7fb8e40ffc60 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.627+0000 7fb8e2ffd700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e40ff350 0x7fb8e4195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.627+0000 7fb8e27fc700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8e40ffc60 0x7fb8e4196470 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8006010 tx=0x7fb8d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.627+0000 7fb8cbfff700 1 -- 172.21.15.88:0/3962851377 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8d801d070 con 0x7fb8e40ffc60 2024-02-19T19:24:47.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.627+0000 7fb8cbfff700 1 -- 172.21.15.88:0/3962851377 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8d8004fb0 con 0x7fb8e40ffc60 2024-02-19T19:24:47.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.627+0000 7fb8e95c0700 1 -- 172.21.15.88:0/3962851377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8e419ab60 con 0x7fb8e40ffc60 2024-02-19T19:24:47.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.627+0000 7fb8cbfff700 1 -- 172.21.15.88:0/3962851377 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8d8022ea0 con 0x7fb8e40ffc60 2024-02-19T19:24:47.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.627+0000 7fb8e95c0700 1 -- 172.21.15.88:0/3962851377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e419b050 con 0x7fb8e40ffc60 2024-02-19T19:24:47.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.628+0000 7fb8e95c0700 1 -- 172.21.15.88:0/3962851377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e4107dd0 con 0x7fb8e40ffc60 2024-02-19T19:24:47.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.629+0000 7fb8cbfff700 1 -- 172.21.15.88:0/3962851377 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb8d8022470 con 0x7fb8e40ffc60 2024-02-19T19:24:47.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.629+0000 7fb8cbfff700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8cc06d8a0 0x7fb8cc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:24:47.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.629+0000 7fb8cbfff700 1 -- 172.21.15.88:0/3962851377 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb8d808ebb0 con 0x7fb8e40ffc60 2024-02-19T19:24:47.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.629+0000 7fb8e2ffd700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8cc06d8a0 0x7fb8cc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:24:47.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.630+0000 7fb8e2ffd700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8cc06d8a0 0x7fb8cc06fd50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb8e406a640 tx=0x7fb8d4008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:24:47.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.634+0000 7fb8cbfff700 1 -- 172.21.15.88:0/3962851377 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb8d8095080 con 0x7fb8e40ffc60 2024-02-19T19:24:47.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:47 smithi088 ceph-mon[88643]: from='client.35170 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:47.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:47 smithi088 ceph-mon[88643]: from='client.35174 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:47.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:47 smithi088 ceph-mon[88643]: from='client.35178 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:47.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:47 smithi088 ceph-mon[88643]: pgmap v635: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:47.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:47 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3844962961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:24:47.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:47 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2252449310' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:24:47.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.916+0000 7fb8e95c0700 1 -- 172.21.15.88:0/3962851377 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb8e402ccf0 con 0x7fb8e40ffc60 2024-02-19T19:24:47.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.916+0000 7fb8cbfff700 1 -- 172.21.15.88:0/3962851377 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fb8d8027020 con 0x7fb8e40ffc60 2024-02-19T19:24:47.923 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:24:47.923 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:24:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:24:47.923 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:24:47.923 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:24:47.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.918+0000 7fb8e95c0700 1 -- 172.21.15.88:0/3962851377 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8cc06d8a0 msgr2=0x7fb8cc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:47.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.918+0000 7fb8e95c0700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8cc06d8a0 0x7fb8cc06fd50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb8e406a640 tx=0x7fb8d4008040 comp rx=0 tx=0).stop 2024-02-19T19:24:47.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.918+0000 7fb8e95c0700 1 -- 172.21.15.88:0/3962851377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8e40ffc60 msgr2=0x7fb8e4196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:24:47.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.918+0000 7fb8e95c0700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8e40ffc60 0x7fb8e4196470 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fb8d8006010 tx=0x7fb8d800be20 comp rx=0 tx=0).stop 2024-02-19T19:24:47.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.918+0000 7fb8e95c0700 1 -- 172.21.15.88:0/3962851377 shutdown_connections 2024-02-19T19:24:47.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.918+0000 7fb8e95c0700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb8cc06d8a0 0x7fb8cc06fd50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.918+0000 7fb8e95c0700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb8e40ff350 0x7fb8e4195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.918+0000 7fb8e95c0700 1 --2- 172.21.15.88:0/3962851377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8e40ffc60 0x7fb8e4196470 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:24:47.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.918+0000 7fb8e95c0700 1 -- 172.21.15.88:0/3962851377 >> 172.21.15.88:0/3962851377 conn(0x7fb8e40748b0 msgr2=0x7fb8e4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:24:47.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.918+0000 7fb8e95c0700 1 -- 172.21.15.88:0/3962851377 shutdown_connections 2024-02-19T19:24:47.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:24:47.918+0000 7fb8e95c0700 1 -- 172.21.15.88:0/3962851377 wait complete. 2024-02-19T19:24:47.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:47 smithi132 ceph-mon[70676]: from='client.35170 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:47.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:47 smithi132 ceph-mon[70676]: from='client.35174 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:47.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:47 smithi132 ceph-mon[70676]: from='client.35178 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:47.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:47 smithi132 ceph-mon[70676]: pgmap v635: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:47.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:47 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3844962961' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:24:47.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:47 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2252449310' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:24:48.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:48 smithi088 ceph-mon[88643]: from='client.35190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:48.838 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:48 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3962851377' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:24:48.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:48 smithi132 ceph-mon[70676]: from='client.35190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:24:48.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:48 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3962851377' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:24:49.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:49 smithi088 ceph-mon[88643]: pgmap v636: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:49.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:49 smithi132 ceph-mon[70676]: pgmap v636: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:51.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:51 smithi088 ceph-mon[88643]: pgmap v637: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:51.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:51 smithi132 ceph-mon[70676]: pgmap v637: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:53.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:53 smithi088 ceph-mon[88643]: pgmap v638: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:53.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:53 smithi132 ceph-mon[70676]: pgmap v638: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:55.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:55 smithi088 ceph-mon[88643]: pgmap v639: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:55.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:55 smithi132 ceph-mon[70676]: pgmap v639: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:57.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:57 smithi132 ceph-mon[70676]: pgmap v640: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:58.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:57 smithi088 ceph-mon[88643]: pgmap v640: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:24:58.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:24:59.087 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:24:59.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:24:59 smithi132 ceph-mon[70676]: pgmap v641: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:00.087 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:24:59 smithi088 ceph-mon[88643]: pgmap v641: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:01.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:01 smithi132 ceph-mon[70676]: pgmap v642: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:02.087 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:01 smithi088 ceph-mon[88643]: pgmap v642: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:03.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:03 smithi132 ceph-mon[70676]: pgmap v643: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:04.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:03 smithi088 ceph-mon[88643]: pgmap v643: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:05.739 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:05 smithi132 ceph-mon[70676]: pgmap v644: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:06.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:05 smithi088 ceph-mon[88643]: pgmap v644: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:07.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:07 smithi132 ceph-mon[70676]: pgmap v645: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:08.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:07 smithi088 ceph-mon[88643]: pgmap v645: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:09.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:09 smithi132 ceph-mon[70676]: pgmap v646: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:10.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:09 smithi088 ceph-mon[88643]: pgmap v646: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:11.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:11 smithi132 ceph-mon[70676]: pgmap v647: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:12.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:11 smithi088 ceph-mon[88643]: pgmap v647: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:13.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:13 smithi132 ceph-mon[70676]: pgmap v648: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:13.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:25:14.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:13 smithi088 ceph-mon[88643]: pgmap v648: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:14.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:25:15.739 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:15 smithi132 ceph-mon[70676]: pgmap v649: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:16.087 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:15 smithi088 ceph-mon[88643]: pgmap v649: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:17.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:17 smithi132 ceph-mon[70676]: pgmap v650: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:18.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.035+0000 7f1153aca700 1 -- 172.21.15.88:0/42987986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1012f0 msgr2=0x7f114c1016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.035+0000 7f1153aca700 1 --2- 172.21.15.88:0/42987986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1012f0 0x7f114c1016c0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1148009910 tx=0x7f1148009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:18.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.036+0000 7f1153aca700 1 -- 172.21.15.88:0/42987986 shutdown_connections 2024-02-19T19:25:18.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.036+0000 7f1153aca700 1 --2- 172.21.15.88:0/42987986 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f114c101c90 0x7f114c10a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.036+0000 7f1153aca700 1 --2- 172.21.15.88:0/42987986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1012f0 0x7f114c1016c0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.036+0000 7f1153aca700 1 -- 172.21.15.88:0/42987986 >> 172.21.15.88:0/42987986 conn(0x7f114c0fac80 msgr2=0x7f114c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:18.043 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:17 smithi088 ceph-mon[88643]: pgmap v650: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:18.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.036+0000 7f1153aca700 1 -- 172.21.15.88:0/42987986 shutdown_connections 2024-02-19T19:25:18.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.036+0000 7f1153aca700 1 -- 172.21.15.88:0/42987986 wait complete. 2024-02-19T19:25:18.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.036+0000 7f1153aca700 1 Processor -- start 2024-02-19T19:25:18.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.036+0000 7f1153aca700 1 -- start start 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1153aca700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1012f0 0x7f114c195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1153aca700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f114c101c90 0x7f114c1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1153aca700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c196ad0 con 0x7f114c1012f0 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1153aca700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114c19a860 con 0x7f114c101c90 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1151866700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1012f0 0x7f114c195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1151866700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1012f0 0x7f114c195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55618/0 (socket says 172.21.15.88:55618) 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1151866700 1 -- 172.21.15.88:0/454402558 learned_addr learned my addr 172.21.15.88:0/454402558 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1151065700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f114c101c90 0x7f114c1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1151866700 1 -- 172.21.15.88:0/454402558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f114c101c90 msgr2=0x7f114c1963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1151866700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f114c101c90 0x7f114c1963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1151866700 1 -- 172.21.15.88:0/454402558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11480095f0 con 0x7f114c1012f0 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1151065700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f114c101c90 0x7f114c1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:25:18.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1151866700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1012f0 0x7f114c195eb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1148000c00 tx=0x7f11480056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1142ffd700 1 -- 172.21.15.88:0/454402558 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f114801d070 con 0x7f114c1012f0 2024-02-19T19:25:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1142ffd700 1 -- 172.21.15.88:0/454402558 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f114800f460 con 0x7f114c1012f0 2024-02-19T19:25:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1153aca700 1 -- 172.21.15.88:0/454402558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f114c19aae0 con 0x7f114c1012f0 2024-02-19T19:25:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1142ffd700 1 -- 172.21.15.88:0/454402558 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1148021640 con 0x7f114c1012f0 2024-02-19T19:25:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.037+0000 7f1153aca700 1 -- 172.21.15.88:0/454402558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f114c19b030 con 0x7f114c1012f0 2024-02-19T19:25:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.038+0000 7f1153aca700 1 -- 172.21.15.88:0/454402558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f114c1079d0 con 0x7f114c1012f0 2024-02-19T19:25:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.039+0000 7f1142ffd700 1 -- 172.21.15.88:0/454402558 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f114800fa60 con 0x7f114c1012f0 2024-02-19T19:25:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.039+0000 7f1142ffd700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f113806d8a0 0x7f113806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.039+0000 7f1142ffd700 1 -- 172.21.15.88:0/454402558 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f114808e770 con 0x7f114c1012f0 2024-02-19T19:25:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.039+0000 7f1151065700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f113806d8a0 0x7f113806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.040+0000 7f1151065700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f113806d8a0 0x7f113806fd50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f114c1974d0 tx=0x7f113c00b2c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:18.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.044+0000 7f1142ffd700 1 -- 172.21.15.88:0/454402558 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1148057f20 con 0x7f114c1012f0 2024-02-19T19:25:18.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.259+0000 7f1153aca700 1 -- 172.21.15.88:0/454402558 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f114c19b250 con 0x7f113806d8a0 2024-02-19T19:25:18.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.260+0000 7f1142ffd700 1 -- 172.21.15.88:0/454402558 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f114c19b250 con 0x7f113806d8a0 2024-02-19T19:25:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.262+0000 7f1153aca700 1 -- 172.21.15.88:0/454402558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f113806d8a0 msgr2=0x7f113806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.262+0000 7f1153aca700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f113806d8a0 0x7f113806fd50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f114c1974d0 tx=0x7f113c00b2c0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.262+0000 7f1153aca700 1 -- 172.21.15.88:0/454402558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1012f0 msgr2=0x7f114c195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.262+0000 7f1153aca700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1012f0 0x7f114c195eb0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1148000c00 tx=0x7f11480056a0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.262+0000 7f1153aca700 1 -- 172.21.15.88:0/454402558 shutdown_connections 2024-02-19T19:25:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.262+0000 7f1153aca700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f113806d8a0 0x7f113806fd50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.262+0000 7f1153aca700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1012f0 0x7f114c195eb0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.262+0000 7f1153aca700 1 --2- 172.21.15.88:0/454402558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f114c101c90 0x7f114c1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.262+0000 7f1153aca700 1 -- 172.21.15.88:0/454402558 >> 172.21.15.88:0/454402558 conn(0x7f114c0fac80 msgr2=0x7f114c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:18.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.262+0000 7f1153aca700 1 -- 172.21.15.88:0/454402558 shutdown_connections 2024-02-19T19:25:18.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.262+0000 7f1153aca700 1 -- 172.21.15.88:0/454402558 wait complete. 2024-02-19T19:25:18.279 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:25:18.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.367+0000 7f47cead8700 1 -- 172.21.15.88:0/4140627231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c8100e80 msgr2=0x7f47c8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.367+0000 7f47cead8700 1 --2- 172.21.15.88:0/4140627231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c8100e80 0x7f47c8104e50 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009910 tx=0x7f47b4009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:18.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.367+0000 7f47cead8700 1 -- 172.21.15.88:0/4140627231 shutdown_connections 2024-02-19T19:25:18.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.367+0000 7f47cead8700 1 --2- 172.21.15.88:0/4140627231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c8100e80 0x7f47c8104e50 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.367+0000 7f47cead8700 1 --2- 172.21.15.88:0/4140627231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47c81004e0 0x7f47c81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.367+0000 7f47cead8700 1 -- 172.21.15.88:0/4140627231 >> 172.21.15.88:0/4140627231 conn(0x7f47c80fbda0 msgr2=0x7f47c80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:18.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.367+0000 7f47cead8700 1 -- 172.21.15.88:0/4140627231 shutdown_connections 2024-02-19T19:25:18.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.368+0000 7f47cead8700 1 -- 172.21.15.88:0/4140627231 wait complete. 2024-02-19T19:25:18.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.368+0000 7f47cead8700 1 Processor -- start 2024-02-19T19:25:18.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.368+0000 7f47cead8700 1 -- start start 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.368+0000 7f47cead8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c81004e0 0x7f47c8195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.368+0000 7f47cead8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47c8100e80 0x7f47c8196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.368+0000 7f47cead8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c8196b30 con 0x7f47c81004e0 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.368+0000 7f47cead8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47c819a8c0 con 0x7f47c8100e80 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47cc874700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c81004e0 0x7f47c8195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47cc874700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c81004e0 0x7f47c8195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55640/0 (socket says 172.21.15.88:55640) 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47cc874700 1 -- 172.21.15.88:0/1460387714 learned_addr learned my addr 172.21.15.88:0/1460387714 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47c7fff700 1 --2- 172.21.15.88:0/1460387714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47c8100e80 0x7f47c8196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47cc874700 1 -- 172.21.15.88:0/1460387714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47c8100e80 msgr2=0x7f47c8196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47cc874700 1 --2- 172.21.15.88:0/1460387714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47c8100e80 0x7f47c8196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47cc874700 1 -- 172.21.15.88:0/1460387714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47b40095f0 con 0x7f47c81004e0 2024-02-19T19:25:18.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47cc874700 1 --2- 172.21.15.88:0/1460387714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c81004e0 0x7f47c8195f10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c8a0 tx=0x7f47bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47c5ffb700 1 -- 172.21.15.88:0/1460387714 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47bc00cea0 con 0x7f47c81004e0 2024-02-19T19:25:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47c5ffb700 1 -- 172.21.15.88:0/1460387714 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47bc004830 con 0x7f47c81004e0 2024-02-19T19:25:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47cead8700 1 -- 172.21.15.88:0/1460387714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47c819aba0 con 0x7f47c81004e0 2024-02-19T19:25:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47c5ffb700 1 -- 172.21.15.88:0/1460387714 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47bc0055b0 con 0x7f47c81004e0 2024-02-19T19:25:18.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.369+0000 7f47cead8700 1 -- 172.21.15.88:0/1460387714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47c819b0f0 con 0x7f47c81004e0 2024-02-19T19:25:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.370+0000 7f47cead8700 1 -- 172.21.15.88:0/1460387714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47c8108850 con 0x7f47c81004e0 2024-02-19T19:25:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.371+0000 7f47c5ffb700 1 -- 172.21.15.88:0/1460387714 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f47bc0049a0 con 0x7f47c81004e0 2024-02-19T19:25:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.371+0000 7f47c5ffb700 1 --2- 172.21.15.88:0/1460387714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f47b806d8a0 0x7f47b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.371+0000 7f47c5ffb700 1 -- 172.21.15.88:0/1460387714 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f47bc08d1d0 con 0x7f47c81004e0 2024-02-19T19:25:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.371+0000 7f47c7fff700 1 --2- 172.21.15.88:0/1460387714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f47b806d8a0 0x7f47b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:18.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.372+0000 7f47c7fff700 1 --2- 172.21.15.88:0/1460387714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f47b806d8a0 0x7f47b806fd50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f47c8197530 tx=0x7f47b401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:18.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.376+0000 7f47c5ffb700 1 -- 172.21.15.88:0/1460387714 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f47bc056980 con 0x7f47c81004e0 2024-02-19T19:25:18.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.593+0000 7f47cead8700 1 -- 172.21.15.88:0/1460387714 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47c819b370 con 0x7f47b806d8a0 2024-02-19T19:25:18.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.594+0000 7f47c5ffb700 1 -- 172.21.15.88:0/1460387714 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f47c819b370 con 0x7f47b806d8a0 2024-02-19T19:25:18.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.596+0000 7f47cead8700 1 -- 172.21.15.88:0/1460387714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f47b806d8a0 msgr2=0x7f47b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.596+0000 7f47cead8700 1 --2- 172.21.15.88:0/1460387714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f47b806d8a0 0x7f47b806fd50 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f47c8197530 tx=0x7f47b401a040 comp rx=0 tx=0).stop 2024-02-19T19:25:18.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.596+0000 7f47cead8700 1 -- 172.21.15.88:0/1460387714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c81004e0 msgr2=0x7f47c8195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.596+0000 7f47cead8700 1 --2- 172.21.15.88:0/1460387714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c81004e0 0x7f47c8195f10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f47bc00c8a0 tx=0x7f47bc00cc60 comp rx=0 tx=0).stop 2024-02-19T19:25:18.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.596+0000 7f47cead8700 1 -- 172.21.15.88:0/1460387714 shutdown_connections 2024-02-19T19:25:18.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.596+0000 7f47cead8700 1 --2- 172.21.15.88:0/1460387714 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f47b806d8a0 0x7f47b806fd50 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.596+0000 7f47cead8700 1 --2- 172.21.15.88:0/1460387714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c81004e0 0x7f47c8195f10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.596+0000 7f47cead8700 1 --2- 172.21.15.88:0/1460387714 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f47c8100e80 0x7f47c8196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.596+0000 7f47cead8700 1 -- 172.21.15.88:0/1460387714 >> 172.21.15.88:0/1460387714 conn(0x7f47c80fbda0 msgr2=0x7f47c80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:18.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.597+0000 7f47cead8700 1 -- 172.21.15.88:0/1460387714 shutdown_connections 2024-02-19T19:25:18.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.597+0000 7f47cead8700 1 -- 172.21.15.88:0/1460387714 wait complete. 2024-02-19T19:25:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.701+0000 7fe82305d700 1 -- 172.21.15.88:0/4028870552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe81c1004e0 msgr2=0x7fe81c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.701+0000 7fe82305d700 1 --2- 172.21.15.88:0/4028870552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe81c1004e0 0x7fe81c1008b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe810009910 tx=0x7fe810009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.701+0000 7fe82305d700 1 -- 172.21.15.88:0/4028870552 shutdown_connections 2024-02-19T19:25:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.701+0000 7fe82305d700 1 --2- 172.21.15.88:0/4028870552 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe81c100e80 0x7fe81c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.701+0000 7fe82305d700 1 --2- 172.21.15.88:0/4028870552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe81c1004e0 0x7fe81c1008b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.701+0000 7fe82305d700 1 -- 172.21.15.88:0/4028870552 >> 172.21.15.88:0/4028870552 conn(0x7fe81c0fbda0 msgr2=0x7fe81c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.701+0000 7fe82305d700 1 -- 172.21.15.88:0/4028870552 shutdown_connections 2024-02-19T19:25:18.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.701+0000 7fe82305d700 1 -- 172.21.15.88:0/4028870552 wait complete. 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.701+0000 7fe82305d700 1 Processor -- start 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.701+0000 7fe82305d700 1 -- start start 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe82305d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe81c1004e0 0x7fe81c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe82305d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe81c100e80 0x7fe81c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe82305d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c073ad0 con 0x7fe81c100e80 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe82305d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c073c10 con 0x7fe81c1004e0 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe81bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe81c100e80 0x7fe81c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe81bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe81c100e80 0x7fe81c073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55650/0 (socket says 172.21.15.88:55650) 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe81bfff700 1 -- 172.21.15.88:0/2559717923 learned_addr learned my addr 172.21.15.88:0/2559717923 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe820df9700 1 --2- 172.21.15.88:0/2559717923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe81c1004e0 0x7fe81c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe81bfff700 1 -- 172.21.15.88:0/2559717923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe81c1004e0 msgr2=0x7fe81c073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe81bfff700 1 --2- 172.21.15.88:0/2559717923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe81c1004e0 0x7fe81c073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe81bfff700 1 -- 172.21.15.88:0/2559717923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8100095f0 con 0x7fe81c100e80 2024-02-19T19:25:18.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.702+0000 7fe81bfff700 1 --2- 172.21.15.88:0/2559717923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe81c100e80 0x7fe81c073590 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe80800b480 tx=0x7fe80800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:18.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.703+0000 7fe819ffb700 1 -- 172.21.15.88:0/2559717923 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8080118c0 con 0x7fe81c100e80 2024-02-19T19:25:18.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.703+0000 7fe819ffb700 1 -- 172.21.15.88:0/2559717923 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe808010380 con 0x7fe81c100e80 2024-02-19T19:25:18.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.703+0000 7fe82305d700 1 -- 172.21.15.88:0/2559717923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c1a2180 con 0x7fe81c100e80 2024-02-19T19:25:18.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.703+0000 7fe819ffb700 1 -- 172.21.15.88:0/2559717923 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe80800f4f0 con 0x7fe81c100e80 2024-02-19T19:25:18.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.703+0000 7fe82305d700 1 -- 172.21.15.88:0/2559717923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c1a2590 con 0x7fe81c100e80 2024-02-19T19:25:18.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.704+0000 7fe819ffb700 1 -- 172.21.15.88:0/2559717923 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe8080104f0 con 0x7fe81c100e80 2024-02-19T19:25:18.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.704+0000 7fe82305d700 1 -- 172.21.15.88:0/2559717923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c1087f0 con 0x7fe81c100e80 2024-02-19T19:25:18.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.705+0000 7fe819ffb700 1 --2- 172.21.15.88:0/2559717923 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe80c06d8a0 0x7fe80c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:18.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.705+0000 7fe819ffb700 1 -- 172.21.15.88:0/2559717923 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe80808d150 con 0x7fe81c100e80 2024-02-19T19:25:18.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.705+0000 7fe820df9700 1 --2- 172.21.15.88:0/2559717923 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe80c06d8a0 0x7fe80c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:18.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.705+0000 7fe820df9700 1 --2- 172.21.15.88:0/2559717923 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe80c06d8a0 0x7fe80c06fd50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe810006010 tx=0x7fe81001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:18.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.710+0000 7fe819ffb700 1 -- 172.21.15.88:0/2559717923 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe808056900 con 0x7fe81c100e80 2024-02-19T19:25:18.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.914+0000 7fe82305d700 1 -- 172.21.15.88:0/2559717923 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe81c061190 con 0x7fe80c06d8a0 2024-02-19T19:25:18.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.925+0000 7fe819ffb700 1 -- 172.21.15.88:0/2559717923 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fe81c061190 con 0x7fe80c06d8a0 2024-02-19T19:25:18.931 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:25:18.931 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (22m) 14m ago 32m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (33m) 14m ago 33m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (15m) 14m ago 31m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (18m) 14m ago 33m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (18m) 14m ago 31m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (21m) 14m ago 32m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (14m) 14m ago 26m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (14m) 14m ago 26m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (14m) 14m ago 26m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (26m) 14m ago 26m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (23m) 14m ago 34m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (23m) 14m ago 31m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (20m) 14m ago 34m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (19m) 14m ago 31m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (23m) 14m ago 33m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (23m) 14m ago 31m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:25:18.932 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (17m) 14m ago 30m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:25:18.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (17m) 14m ago 29m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:25:18.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (16m) 14m ago 29m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:25:18.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (16m) 14m ago 29m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:25:18.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (15m) 14m ago 28m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:25:18.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (15m) 14m ago 28m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:25:18.933 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (22m) 14m ago 31m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:25:18.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.927+0000 7fe82305d700 1 -- 172.21.15.88:0/2559717923 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe80c06d8a0 msgr2=0x7fe80c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.927+0000 7fe82305d700 1 --2- 172.21.15.88:0/2559717923 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe80c06d8a0 0x7fe80c06fd50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe810006010 tx=0x7fe81001a040 comp rx=0 tx=0).stop 2024-02-19T19:25:18.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.927+0000 7fe82305d700 1 -- 172.21.15.88:0/2559717923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe81c100e80 msgr2=0x7fe81c073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:18.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.927+0000 7fe82305d700 1 --2- 172.21.15.88:0/2559717923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe81c100e80 0x7fe81c073590 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe80800b480 tx=0x7fe80800b840 comp rx=0 tx=0).stop 2024-02-19T19:25:18.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.927+0000 7fe82305d700 1 -- 172.21.15.88:0/2559717923 shutdown_connections 2024-02-19T19:25:18.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.927+0000 7fe82305d700 1 --2- 172.21.15.88:0/2559717923 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe80c06d8a0 0x7fe80c06fd50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.927+0000 7fe82305d700 1 --2- 172.21.15.88:0/2559717923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe81c1004e0 0x7fe81c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.927+0000 7fe82305d700 1 --2- 172.21.15.88:0/2559717923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe81c100e80 0x7fe81c073590 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:18.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.927+0000 7fe82305d700 1 -- 172.21.15.88:0/2559717923 >> 172.21.15.88:0/2559717923 conn(0x7fe81c0fbda0 msgr2=0x7fe81c0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:18.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.927+0000 7fe82305d700 1 -- 172.21.15.88:0/2559717923 shutdown_connections 2024-02-19T19:25:18.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:18.928+0000 7fe82305d700 1 -- 172.21.15.88:0/2559717923 wait complete. 2024-02-19T19:25:19.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.031+0000 7f5f55d64700 1 -- 172.21.15.88:0/246703126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f501004c0 msgr2=0x7f5f50100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:19.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.031+0000 7f5f55d64700 1 --2- 172.21.15.88:0/246703126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f501004c0 0x7f5f50100890 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800b0a0 tx=0x7f5f4800b3b0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.031+0000 7f5f55d64700 1 -- 172.21.15.88:0/246703126 shutdown_connections 2024-02-19T19:25:19.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.031+0000 7f5f55d64700 1 --2- 172.21.15.88:0/246703126 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f50100e60 0x7f5f50104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.031+0000 7f5f55d64700 1 --2- 172.21.15.88:0/246703126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f501004c0 0x7f5f50100890 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.031+0000 7f5f55d64700 1 -- 172.21.15.88:0/246703126 >> 172.21.15.88:0/246703126 conn(0x7f5f500fbd80 msgr2=0x7f5f500fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:19.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.031+0000 7f5f55d64700 1 -- 172.21.15.88:0/246703126 shutdown_connections 2024-02-19T19:25:19.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.031+0000 7f5f55d64700 1 -- 172.21.15.88:0/246703126 wait complete. 2024-02-19T19:25:19.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f55d64700 1 Processor -- start 2024-02-19T19:25:19.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f55d64700 1 -- start start 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f55d64700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f501004c0 0x7f5f50195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f55d64700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f50100e60 0x7f5f50196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f55d64700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f50196b50 con 0x7f5f50100e60 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f55d64700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5019a8e0 con 0x7f5f501004c0 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f4effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f50100e60 0x7f5f50196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f4effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f50100e60 0x7f5f50196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55682/0 (socket says 172.21.15.88:55682) 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f4effd700 1 -- 172.21.15.88:0/2640814905 learned_addr learned my addr 172.21.15.88:0/2640814905 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f4f7fe700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f501004c0 0x7f5f50195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f4effd700 1 -- 172.21.15.88:0/2640814905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f501004c0 msgr2=0x7f5f50195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f4effd700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f501004c0 0x7f5f50195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f4effd700 1 -- 172.21.15.88:0/2640814905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f48009d00 con 0x7f5f50100e60 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.032+0000 7f5f4f7fe700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f501004c0 0x7f5f50195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.033+0000 7f5f4effd700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f50100e60 0x7f5f50196470 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800c8f0 tx=0x7f5f3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:19.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.033+0000 7f5f4cff9700 1 -- 172.21.15.88:0/2640814905 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f38007a50 con 0x7f5f50100e60 2024-02-19T19:25:19.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.033+0000 7f5f4cff9700 1 -- 172.21.15.88:0/2640814905 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f38007bb0 con 0x7f5f50100e60 2024-02-19T19:25:19.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.033+0000 7f5f4cff9700 1 -- 172.21.15.88:0/2640814905 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f38018630 con 0x7f5f50100e60 2024-02-19T19:25:19.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.033+0000 7f5f55d64700 1 -- 172.21.15.88:0/2640814905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f5019abc0 con 0x7f5f50100e60 2024-02-19T19:25:19.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.033+0000 7f5f55d64700 1 -- 172.21.15.88:0/2640814905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f5019b110 con 0x7f5f50100e60 2024-02-19T19:25:19.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.034+0000 7f5f55d64700 1 -- 172.21.15.88:0/2640814905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f501087d0 con 0x7f5f50100e60 2024-02-19T19:25:19.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.034+0000 7f5f4cff9700 1 -- 172.21.15.88:0/2640814905 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5f38007d20 con 0x7f5f50100e60 2024-02-19T19:25:19.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.035+0000 7f5f4cff9700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f3c06d8a0 0x7f5f3c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:19.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.035+0000 7f5f4cff9700 1 -- 172.21.15.88:0/2640814905 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5f3808d500 con 0x7f5f50100e60 2024-02-19T19:25:19.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.035+0000 7f5f4f7fe700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f3c06d8a0 0x7f5f3c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:19.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.035+0000 7f5f4f7fe700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f3c06d8a0 0x7f5f3c06fd50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f480062a0 tx=0x7f5f48006210 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:19.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.039+0000 7f5f4cff9700 1 -- 172.21.15.88:0/2640814905 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f38056cb0 con 0x7f5f50100e60 2024-02-19T19:25:19.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.322+0000 7f5f55d64700 1 -- 172.21.15.88:0/2640814905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5f5004f2a0 con 0x7f5f50100e60 2024-02-19T19:25:19.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.322+0000 7f5f4cff9700 1 -- 172.21.15.88:0/2640814905 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f5f3805a370 con 0x7f5f50100e60 2024-02-19T19:25:19.329 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:25:19.329 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:25:19.329 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:25:19.329 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:25:19.329 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:25:19.329 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:25:19.330 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:25:19.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.325+0000 7f5f55d64700 1 -- 172.21.15.88:0/2640814905 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f3c06d8a0 msgr2=0x7f5f3c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:19.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.325+0000 7f5f55d64700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f3c06d8a0 0x7f5f3c06fd50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5f480062a0 tx=0x7f5f48006210 comp rx=0 tx=0).stop 2024-02-19T19:25:19.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.325+0000 7f5f55d64700 1 -- 172.21.15.88:0/2640814905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f50100e60 msgr2=0x7f5f50196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:19.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.325+0000 7f5f55d64700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f50100e60 0x7f5f50196470 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5f3800c8f0 tx=0x7f5f3800ccb0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.325+0000 7f5f55d64700 1 -- 172.21.15.88:0/2640814905 shutdown_connections 2024-02-19T19:25:19.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.325+0000 7f5f55d64700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f3c06d8a0 0x7f5f3c06fd50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.325+0000 7f5f55d64700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f501004c0 0x7f5f50195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.325+0000 7f5f55d64700 1 --2- 172.21.15.88:0/2640814905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f50100e60 0x7f5f50196470 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.325+0000 7f5f55d64700 1 -- 172.21.15.88:0/2640814905 >> 172.21.15.88:0/2640814905 conn(0x7f5f500fbd80 msgr2=0x7f5f500fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:19.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.325+0000 7f5f55d64700 1 -- 172.21.15.88:0/2640814905 shutdown_connections 2024-02-19T19:25:19.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.325+0000 7f5f55d64700 1 -- 172.21.15.88:0/2640814905 wait complete. 2024-02-19T19:25:19.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.429+0000 7fc7abc16700 1 -- 172.21.15.88:0/1628106965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4104130 msgr2=0x7fc7a4104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:19.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.429+0000 7fc7abc16700 1 --2- 172.21.15.88:0/1628106965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4104130 0x7fc7a4104500 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc794009910 tx=0x7fc794009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:19.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.429+0000 7fc7abc16700 1 -- 172.21.15.88:0/1628106965 shutdown_connections 2024-02-19T19:25:19.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.429+0000 7fc7abc16700 1 --2- 172.21.15.88:0/1628106965 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a40ffbd0 0x7fc7a4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.429+0000 7fc7abc16700 1 --2- 172.21.15.88:0/1628106965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4104130 0x7fc7a4104500 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.429+0000 7fc7abc16700 1 -- 172.21.15.88:0/1628106965 >> 172.21.15.88:0/1628106965 conn(0x7fc7a4075130 msgr2=0x7fc7a4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:19.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.429+0000 7fc7abc16700 1 -- 172.21.15.88:0/1628106965 shutdown_connections 2024-02-19T19:25:19.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.429+0000 7fc7abc16700 1 -- 172.21.15.88:0/1628106965 wait complete. 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7abc16700 1 Processor -- start 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7abc16700 1 -- start start 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7abc16700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a40ffbd0 0x7fc7a4100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7abc16700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4104130 0x7fc7a4101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7abc16700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a41018b0 con 0x7fc7a4104130 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7abc16700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a41019f0 con 0x7fc7a40ffbd0 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7a91b1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4104130 0x7fc7a4101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7a91b1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4104130 0x7fc7a4101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55696/0 (socket says 172.21.15.88:55696) 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7a91b1700 1 -- 172.21.15.88:0/1266986836 learned_addr learned my addr 172.21.15.88:0/1266986836 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7a91b1700 1 -- 172.21.15.88:0/1266986836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a40ffbd0 msgr2=0x7fc7a4100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7a99b2700 1 --2- 172.21.15.88:0/1266986836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a40ffbd0 0x7fc7a4100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7a91b1700 1 --2- 172.21.15.88:0/1266986836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a40ffbd0 0x7fc7a4100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7a91b1700 1 -- 172.21.15.88:0/1266986836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7940095f0 con 0x7fc7a4104130 2024-02-19T19:25:19.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.430+0000 7fc7a91b1700 1 --2- 172.21.15.88:0/1266986836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4104130 0x7fc7a4101370 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000d900 tx=0x7fc7a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:19.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.431+0000 7fc79affd700 1 -- 172.21.15.88:0/1266986836 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7a00041f0 con 0x7fc7a4104130 2024-02-19T19:25:19.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.431+0000 7fc7abc16700 1 -- 172.21.15.88:0/1266986836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a41a20d0 con 0x7fc7a4104130 2024-02-19T19:25:19.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.431+0000 7fc79affd700 1 -- 172.21.15.88:0/1266986836 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7a0004cf0 con 0x7fc7a4104130 2024-02-19T19:25:19.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.431+0000 7fc79affd700 1 -- 172.21.15.88:0/1266986836 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7a000b750 con 0x7fc7a4104130 2024-02-19T19:25:19.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.431+0000 7fc7abc16700 1 -- 172.21.15.88:0/1266986836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a41a24e0 con 0x7fc7a4104130 2024-02-19T19:25:19.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.432+0000 7fc7abc16700 1 -- 172.21.15.88:0/1266986836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a41086a0 con 0x7fc7a4104130 2024-02-19T19:25:19.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.432+0000 7fc79affd700 1 -- 172.21.15.88:0/1266986836 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc7a0004350 con 0x7fc7a4104130 2024-02-19T19:25:19.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.433+0000 7fc79affd700 1 --2- 172.21.15.88:0/1266986836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc79006d8a0 0x7fc79006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:19.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.433+0000 7fc7a99b2700 1 --2- 172.21.15.88:0/1266986836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc79006d8a0 0x7fc79006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:19.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.433+0000 7fc79affd700 1 -- 172.21.15.88:0/1266986836 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc7a008d660 con 0x7fc7a4104130 2024-02-19T19:25:19.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.433+0000 7fc7a99b2700 1 --2- 172.21.15.88:0/1266986836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc79006d8a0 0x7fc79006fd50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc79400b5c0 tx=0x7fc794005380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:19.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.438+0000 7fc79affd700 1 -- 172.21.15.88:0/1266986836 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc7a0056e10 con 0x7fc7a4104130 2024-02-19T19:25:19.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.678+0000 7fc7abc16700 1 -- 172.21.15.88:0/1266986836 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc7a404f2a0 con 0x7fc7a4104130 2024-02-19T19:25:19.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.678+0000 7fc79affd700 1 -- 172.21.15.88:0/1266986836 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fc7a005a4d0 con 0x7fc7a4104130 2024-02-19T19:25:19.685 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:25:19.685 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:25:19.686 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:25:19.687 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:25:19.688 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:25:19.688 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:25:19.688 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:25:19.688 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:25:19.688 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:25:19.688 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:25:19.688 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:25:19.688 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:25:19.688 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:25:19.688 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:25:19.688 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:25:19.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.681+0000 7fc7abc16700 1 -- 172.21.15.88:0/1266986836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc79006d8a0 msgr2=0x7fc79006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:19.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.681+0000 7fc7abc16700 1 --2- 172.21.15.88:0/1266986836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc79006d8a0 0x7fc79006fd50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc79400b5c0 tx=0x7fc794005380 comp rx=0 tx=0).stop 2024-02-19T19:25:19.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.681+0000 7fc7abc16700 1 -- 172.21.15.88:0/1266986836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4104130 msgr2=0x7fc7a4101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:19.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.681+0000 7fc7abc16700 1 --2- 172.21.15.88:0/1266986836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4104130 0x7fc7a4101370 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc7a000d900 tx=0x7fc7a000dcc0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.681+0000 7fc7abc16700 1 -- 172.21.15.88:0/1266986836 shutdown_connections 2024-02-19T19:25:19.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.681+0000 7fc7abc16700 1 --2- 172.21.15.88:0/1266986836 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc79006d8a0 0x7fc79006fd50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.681+0000 7fc7abc16700 1 --2- 172.21.15.88:0/1266986836 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc7a40ffbd0 0x7fc7a4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.681+0000 7fc7abc16700 1 --2- 172.21.15.88:0/1266986836 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7a4104130 0x7fc7a4101370 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.681+0000 7fc7abc16700 1 -- 172.21.15.88:0/1266986836 >> 172.21.15.88:0/1266986836 conn(0x7fc7a4075130 msgr2=0x7fc7a40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:19.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.681+0000 7fc7abc16700 1 -- 172.21.15.88:0/1266986836 shutdown_connections 2024-02-19T19:25:19.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.681+0000 7fc7abc16700 1 -- 172.21.15.88:0/1266986836 wait complete. 2024-02-19T19:25:19.690 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:25:19.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.786+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2923687779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a00ffc60 msgr2=0x7f43a010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:19.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.786+0000 7f43a4ab9700 1 --2- 172.21.15.88:0/2923687779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a00ffc60 0x7f43a010a6d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4394009910 tx=0x7f4394009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:19.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.786+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2923687779 shutdown_connections 2024-02-19T19:25:19.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.786+0000 7f43a4ab9700 1 --2- 172.21.15.88:0/2923687779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a00ffc60 0x7f43a010a6d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.786+0000 7f43a4ab9700 1 --2- 172.21.15.88:0/2923687779 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43a00ff350 0x7f43a00ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.786+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2923687779 >> 172.21.15.88:0/2923687779 conn(0x7f43a00748b0 msgr2=0x7f43a0074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:19.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.786+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2923687779 shutdown_connections 2024-02-19T19:25:19.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.786+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2923687779 wait complete. 2024-02-19T19:25:19.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.786+0000 7f43a4ab9700 1 Processor -- start 2024-02-19T19:25:19.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.786+0000 7f43a4ab9700 1 -- start start 2024-02-19T19:25:19.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f43a4ab9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43a00ff350 0x7f43a0195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:19.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f43a4ab9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a00ffc60 0x7f43a0196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:19.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f43a4ab9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a0196b00 con 0x7f43a00ffc60 2024-02-19T19:25:19.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f43a4ab9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a019a890 con 0x7f43a00ff350 2024-02-19T19:25:19.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f439dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a00ffc60 0x7f43a0196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:19.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f439dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a00ffc60 0x7f43a0196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55718/0 (socket says 172.21.15.88:55718) 2024-02-19T19:25:19.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f439dd9b700 1 -- 172.21.15.88:0/2012874127 learned_addr learned my addr 172.21.15.88:0/2012874127 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:19.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f439e59c700 1 --2- 172.21.15.88:0/2012874127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43a00ff350 0x7f43a0195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:19.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f439dd9b700 1 -- 172.21.15.88:0/2012874127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43a00ff350 msgr2=0x7f43a0195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:19.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f439dd9b700 1 --2- 172.21.15.88:0/2012874127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43a00ff350 0x7f43a0195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:19.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f439dd9b700 1 -- 172.21.15.88:0/2012874127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43940095f0 con 0x7f43a00ffc60 2024-02-19T19:25:19.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f439dd9b700 1 --2- 172.21.15.88:0/2012874127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a00ffc60 0x7f43a0196420 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4394006010 tx=0x7f439400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:19.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.787+0000 7f438f7fe700 1 -- 172.21.15.88:0/2012874127 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f439401d070 con 0x7f43a00ffc60 2024-02-19T19:25:19.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.788+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2012874127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a019ab10 con 0x7f43a00ffc60 2024-02-19T19:25:19.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.788+0000 7f438f7fe700 1 -- 172.21.15.88:0/2012874127 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4394005dc0 con 0x7f43a00ffc60 2024-02-19T19:25:19.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.788+0000 7f438f7fe700 1 -- 172.21.15.88:0/2012874127 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4394017400 con 0x7f43a00ffc60 2024-02-19T19:25:19.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.788+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2012874127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a019b000 con 0x7f43a00ffc60 2024-02-19T19:25:19.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.788+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2012874127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a0107dd0 con 0x7f43a00ffc60 2024-02-19T19:25:19.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.789+0000 7f438f7fe700 1 -- 172.21.15.88:0/2012874127 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4394003a60 con 0x7f43a00ffc60 2024-02-19T19:25:19.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.789+0000 7f438f7fe700 1 --2- 172.21.15.88:0/2012874127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806d8a0 0x7f438806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:19.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.789+0000 7f438f7fe700 1 -- 172.21.15.88:0/2012874127 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f439408e900 con 0x7f43a00ffc60 2024-02-19T19:25:19.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.789+0000 7f439e59c700 1 --2- 172.21.15.88:0/2012874127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806d8a0 0x7f438806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:19.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.790+0000 7f439e59c700 1 --2- 172.21.15.88:0/2012874127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806d8a0 0x7f438806fd50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f43a006a640 tx=0x7f439000b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:19.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:19.794+0000 7f438f7fe700 1 -- 172.21.15.88:0/2012874127 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4394058130 con 0x7f43a00ffc60 2024-02-19T19:25:19.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:19 smithi132 ceph-mon[70676]: from='client.35198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:19.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:19 smithi132 ceph-mon[70676]: pgmap v651: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:19.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:19 smithi132 ceph-mon[70676]: from='client.35202 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:19.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:19 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2640814905' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:25:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.009+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2012874127 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f43a019b270 con 0x7f438806d8a0 2024-02-19T19:25:20.016 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:19 smithi088 ceph-mon[88643]: from='client.35198 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:20.016 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:19 smithi088 ceph-mon[88643]: pgmap v651: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:20.016 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:19 smithi088 ceph-mon[88643]: from='client.35202 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:20.017 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:19 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2640814905' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:25:20.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.011+0000 7f438f7fe700 1 -- 172.21.15.88:0/2012874127 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f43a019b270 con 0x7f438806d8a0 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:25:20.022 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:25:20.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.013+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2012874127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806d8a0 msgr2=0x7f438806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:20.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.013+0000 7f43a4ab9700 1 --2- 172.21.15.88:0/2012874127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806d8a0 0x7f438806fd50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f43a006a640 tx=0x7f439000b410 comp rx=0 tx=0).stop 2024-02-19T19:25:20.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.013+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2012874127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a00ffc60 msgr2=0x7f43a0196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:20.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.013+0000 7f43a4ab9700 1 --2- 172.21.15.88:0/2012874127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a00ffc60 0x7f43a0196420 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4394006010 tx=0x7f439400bd70 comp rx=0 tx=0).stop 2024-02-19T19:25:20.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.013+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2012874127 shutdown_connections 2024-02-19T19:25:20.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.013+0000 7f43a4ab9700 1 --2- 172.21.15.88:0/2012874127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f438806d8a0 0x7f438806fd50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:20.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.013+0000 7f43a4ab9700 1 --2- 172.21.15.88:0/2012874127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f43a00ff350 0x7f43a0195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:20.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.013+0000 7f43a4ab9700 1 --2- 172.21.15.88:0/2012874127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43a00ffc60 0x7f43a0196420 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:20.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.013+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2012874127 >> 172.21.15.88:0/2012874127 conn(0x7f43a00748b0 msgr2=0x7f43a0104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:20.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.014+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2012874127 shutdown_connections 2024-02-19T19:25:20.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.014+0000 7f43a4ab9700 1 -- 172.21.15.88:0/2012874127 wait complete. 2024-02-19T19:25:20.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.118+0000 7f6d7b324700 1 -- 172.21.15.88:0/359280756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d74100e80 msgr2=0x7f6d74104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:20.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.118+0000 7f6d7b324700 1 --2- 172.21.15.88:0/359280756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d74100e80 0x7f6d74104e50 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009910 tx=0x7f6d64009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:20.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.118+0000 7f6d7b324700 1 -- 172.21.15.88:0/359280756 shutdown_connections 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.118+0000 7f6d7b324700 1 --2- 172.21.15.88:0/359280756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d74100e80 0x7f6d74104e50 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.118+0000 7f6d7b324700 1 --2- 172.21.15.88:0/359280756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d741004e0 0x7f6d741008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.118+0000 7f6d7b324700 1 -- 172.21.15.88:0/359280756 >> 172.21.15.88:0/359280756 conn(0x7f6d740fbda0 msgr2=0x7f6d740fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.118+0000 7f6d7b324700 1 -- 172.21.15.88:0/359280756 shutdown_connections 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.118+0000 7f6d7b324700 1 -- 172.21.15.88:0/359280756 wait complete. 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d7b324700 1 Processor -- start 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d7b324700 1 -- start start 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d7b324700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d741004e0 0x7f6d74073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d7b324700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d74100e80 0x7f6d74073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d7b324700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74073ad0 con 0x7f6d741004e0 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d7b324700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d74073c10 con 0x7f6d74100e80 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d790c0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d741004e0 0x7f6d74073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:20.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d790c0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d741004e0 0x7f6d74073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55726/0 (socket says 172.21.15.88:55726) 2024-02-19T19:25:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d790c0700 1 -- 172.21.15.88:0/1899759542 learned_addr learned my addr 172.21.15.88:0/1899759542 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d788bf700 1 --2- 172.21.15.88:0/1899759542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d74100e80 0x7f6d74073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d790c0700 1 -- 172.21.15.88:0/1899759542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d74100e80 msgr2=0x7f6d74073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d790c0700 1 --2- 172.21.15.88:0/1899759542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d74100e80 0x7f6d74073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.119+0000 7f6d790c0700 1 -- 172.21.15.88:0/1899759542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640095f0 con 0x7f6d741004e0 2024-02-19T19:25:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.120+0000 7f6d790c0700 1 --2- 172.21.15.88:0/1899759542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d741004e0 0x7f6d74073050 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c930 tx=0x7f6d6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.120+0000 7f6d6a7fc700 1 -- 172.21.15.88:0/1899759542 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d6c007ad0 con 0x7f6d741004e0 2024-02-19T19:25:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.120+0000 7f6d6a7fc700 1 -- 172.21.15.88:0/1899759542 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d6c00ce80 con 0x7f6d741004e0 2024-02-19T19:25:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.120+0000 7f6d6a7fc700 1 -- 172.21.15.88:0/1899759542 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d6c018630 con 0x7f6d741004e0 2024-02-19T19:25:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.120+0000 7f6d7b324700 1 -- 172.21.15.88:0/1899759542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d741a20e0 con 0x7f6d741004e0 2024-02-19T19:25:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.120+0000 7f6d7b324700 1 -- 172.21.15.88:0/1899759542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d741a2440 con 0x7f6d741004e0 2024-02-19T19:25:20.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.121+0000 7f6d7b324700 1 -- 172.21.15.88:0/1899759542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d74108850 con 0x7f6d741004e0 2024-02-19T19:25:20.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.121+0000 7f6d6a7fc700 1 -- 172.21.15.88:0/1899759542 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6d6c007c30 con 0x7f6d741004e0 2024-02-19T19:25:20.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.122+0000 7f6d6a7fc700 1 --2- 172.21.15.88:0/1899759542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d6006d8a0 0x7f6d6006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:20.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.122+0000 7f6d6a7fc700 1 -- 172.21.15.88:0/1899759542 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6d6c08d420 con 0x7f6d741004e0 2024-02-19T19:25:20.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.122+0000 7f6d788bf700 1 --2- 172.21.15.88:0/1899759542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d6006d8a0 0x7f6d6006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:20.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.122+0000 7f6d788bf700 1 --2- 172.21.15.88:0/1899759542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d6006d8a0 0x7f6d6006fd50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d74074840 tx=0x7f6d6401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:20.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.126+0000 7f6d6a7fc700 1 -- 172.21.15.88:0/1899759542 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6d6c056c50 con 0x7f6d741004e0 2024-02-19T19:25:20.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.404+0000 7f6d7b324700 1 -- 172.21.15.88:0/1899759542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6d7404f2a0 con 0x7f6d741004e0 2024-02-19T19:25:20.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.404+0000 7f6d6a7fc700 1 -- 172.21.15.88:0/1899759542 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f6d6c05a310 con 0x7f6d741004e0 2024-02-19T19:25:20.411 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:25:20.411 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:25:20.411 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:25:20.411 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:25:20.411 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:25:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.406+0000 7f6d7b324700 1 -- 172.21.15.88:0/1899759542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d6006d8a0 msgr2=0x7f6d6006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.406+0000 7f6d7b324700 1 --2- 172.21.15.88:0/1899759542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d6006d8a0 0x7f6d6006fd50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d74074840 tx=0x7f6d6401a040 comp rx=0 tx=0).stop 2024-02-19T19:25:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.406+0000 7f6d7b324700 1 -- 172.21.15.88:0/1899759542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d741004e0 msgr2=0x7f6d74073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.406+0000 7f6d7b324700 1 --2- 172.21.15.88:0/1899759542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d741004e0 0x7f6d74073050 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00c930 tx=0x7f6d6c00cc40 comp rx=0 tx=0).stop 2024-02-19T19:25:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.406+0000 7f6d7b324700 1 -- 172.21.15.88:0/1899759542 shutdown_connections 2024-02-19T19:25:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.406+0000 7f6d7b324700 1 --2- 172.21.15.88:0/1899759542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d6006d8a0 0x7f6d6006fd50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.406+0000 7f6d7b324700 1 --2- 172.21.15.88:0/1899759542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d741004e0 0x7f6d74073050 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.406+0000 7f6d7b324700 1 --2- 172.21.15.88:0/1899759542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d74100e80 0x7f6d74073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.406+0000 7f6d7b324700 1 -- 172.21.15.88:0/1899759542 >> 172.21.15.88:0/1899759542 conn(0x7f6d740fbda0 msgr2=0x7f6d740fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.406+0000 7f6d7b324700 1 -- 172.21.15.88:0/1899759542 shutdown_connections 2024-02-19T19:25:20.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:20.406+0000 7f6d7b324700 1 -- 172.21.15.88:0/1899759542 wait complete. 2024-02-19T19:25:20.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:20 smithi088 ceph-mon[88643]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:20.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:20 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1266986836' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:25:20.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:20 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1899759542' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:25:20.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:20 smithi132 ceph-mon[70676]: from='client.35206 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:20.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:20 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1266986836' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:25:20.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:20 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1899759542' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:25:21.889 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:21 smithi132 ceph-mon[70676]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:21.890 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:21 smithi132 ceph-mon[70676]: pgmap v652: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:22.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:21 smithi088 ceph-mon[88643]: from='client.35218 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:22.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:21 smithi088 ceph-mon[88643]: pgmap v652: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:23.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:23 smithi132 ceph-mon[70676]: pgmap v653: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:24.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:23 smithi088 ceph-mon[88643]: pgmap v653: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:25.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:25 smithi132 ceph-mon[70676]: pgmap v654: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:26.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:25 smithi088 ceph-mon[88643]: pgmap v654: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:27.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:27 smithi132 ceph-mon[70676]: pgmap v655: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:28.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:27 smithi088 ceph-mon[88643]: pgmap v655: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:28.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:25:29.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:25:29.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:29 smithi132 ceph-mon[70676]: pgmap v656: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:30.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:29 smithi088 ceph-mon[88643]: pgmap v656: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:31.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:31 smithi132 ceph-mon[70676]: pgmap v657: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:32.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:31 smithi088 ceph-mon[88643]: pgmap v657: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:33.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:33 smithi132 ceph-mon[70676]: pgmap v658: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:34.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:33 smithi088 ceph-mon[88643]: pgmap v658: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:34.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:34 smithi132 ceph-mon[70676]: pgmap v659: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:35.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:34 smithi088 ceph-mon[88643]: pgmap v659: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:37.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:37 smithi132 ceph-mon[70676]: pgmap v660: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:37.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:37 smithi088 ceph-mon[88643]: pgmap v660: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:39.739 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:39 smithi132 ceph-mon[70676]: pgmap v661: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:39.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:39 smithi088 ceph-mon[88643]: pgmap v661: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:41.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:41 smithi132 ceph-mon[70676]: pgmap v662: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:41.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:41 smithi088 ceph-mon[88643]: pgmap v662: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:43.739 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:43 smithi132 ceph-mon[70676]: pgmap v663: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:43.739 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:25:43.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:43 smithi088 ceph-mon[88643]: pgmap v663: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:43.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:25:45.739 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:45 smithi132 ceph-mon[70676]: pgmap v664: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:45.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:45 smithi088 ceph-mon[88643]: pgmap v664: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:47.739 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:47 smithi132 ceph-mon[70676]: pgmap v665: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:47.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:47 smithi088 ceph-mon[88643]: pgmap v665: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:49.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:49 smithi088 ceph-mon[88643]: pgmap v666: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:49.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:49 smithi132 ceph-mon[70676]: pgmap v666: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:50.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.523+0000 7f026066f700 1 -- 172.21.15.88:0/482805587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258100e80 msgr2=0x7f0258104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:50.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.523+0000 7f026066f700 1 --2- 172.21.15.88:0/482805587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258100e80 0x7f0258104e50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0254009910 tx=0x7f0254009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:50.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.523+0000 7f026066f700 1 -- 172.21.15.88:0/482805587 shutdown_connections 2024-02-19T19:25:50.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.523+0000 7f026066f700 1 --2- 172.21.15.88:0/482805587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258100e80 0x7f0258104e50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:50.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.523+0000 7f026066f700 1 --2- 172.21.15.88:0/482805587 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02581004e0 0x7f02581008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:50.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.523+0000 7f026066f700 1 -- 172.21.15.88:0/482805587 >> 172.21.15.88:0/482805587 conn(0x7f02580fbda0 msgr2=0x7f02580fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:50.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.524+0000 7f026066f700 1 -- 172.21.15.88:0/482805587 shutdown_connections 2024-02-19T19:25:50.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.524+0000 7f026066f700 1 -- 172.21.15.88:0/482805587 wait complete. 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f026066f700 1 Processor -- start 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f026066f700 1 -- start start 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f026066f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02581004e0 0x7f0258073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f026066f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258100e80 0x7f0258073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f026066f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0258073ad0 con 0x7f0258100e80 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f026066f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0258073c10 con 0x7f02581004e0 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f025dc0a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258100e80 0x7f0258073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f025dc0a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258100e80 0x7f0258073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57200/0 (socket says 172.21.15.88:57200) 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f025dc0a700 1 -- 172.21.15.88:0/3316192867 learned_addr learned my addr 172.21.15.88:0/3316192867 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f025e40b700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02581004e0 0x7f0258073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f025dc0a700 1 -- 172.21.15.88:0/3316192867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02581004e0 msgr2=0x7f0258073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f025dc0a700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02581004e0 0x7f0258073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f025dc0a700 1 -- 172.21.15.88:0/3316192867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02540095f0 con 0x7f0258100e80 2024-02-19T19:25:50.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.525+0000 7f025e40b700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02581004e0 0x7f0258073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.526+0000 7f025dc0a700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258100e80 0x7f0258073590 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f025400bda0 tx=0x7f025400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.526+0000 7f024f7fe700 1 -- 172.21.15.88:0/3316192867 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f025401d070 con 0x7f0258100e80 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.526+0000 7f024f7fe700 1 -- 172.21.15.88:0/3316192867 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0254004fb0 con 0x7f0258100e80 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.526+0000 7f026066f700 1 -- 172.21.15.88:0/3316192867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0258073e90 con 0x7f0258100e80 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.526+0000 7f024f7fe700 1 -- 172.21.15.88:0/3316192867 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f025400f480 con 0x7f0258100e80 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.526+0000 7f026066f700 1 -- 172.21.15.88:0/3316192867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02581a24f0 con 0x7f0258100e80 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.527+0000 7f026066f700 1 -- 172.21.15.88:0/3316192867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02581087a0 con 0x7f0258100e80 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.527+0000 7f024f7fe700 1 -- 172.21.15.88:0/3316192867 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0254022870 con 0x7f0258100e80 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.528+0000 7f024f7fe700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f024406d8a0 0x7f024406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.528+0000 7f024f7fe700 1 -- 172.21.15.88:0/3316192867 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f025408e9a0 con 0x7f0258100e80 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.528+0000 7f025e40b700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f024406d8a0 0x7f024406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:50.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.528+0000 7f025e40b700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f024406d8a0 0x7f024406fd50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f02480099d0 tx=0x7f0248008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:50.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.533+0000 7f024f7fe700 1 -- 172.21.15.88:0/3316192867 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0254058280 con 0x7f0258100e80 2024-02-19T19:25:50.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.757+0000 7f026066f700 1 -- 172.21.15.88:0/3316192867 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0258074660 con 0x7f024406d8a0 2024-02-19T19:25:50.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.758+0000 7f024f7fe700 1 -- 172.21.15.88:0/3316192867 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0258074660 con 0x7f024406d8a0 2024-02-19T19:25:50.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.760+0000 7f026066f700 1 -- 172.21.15.88:0/3316192867 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f024406d8a0 msgr2=0x7f024406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:50.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.760+0000 7f026066f700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f024406d8a0 0x7f024406fd50 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f02480099d0 tx=0x7f0248008040 comp rx=0 tx=0).stop 2024-02-19T19:25:50.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.760+0000 7f026066f700 1 -- 172.21.15.88:0/3316192867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258100e80 msgr2=0x7f0258073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:50.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.760+0000 7f026066f700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258100e80 0x7f0258073590 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f025400bda0 tx=0x7f025400be80 comp rx=0 tx=0).stop 2024-02-19T19:25:50.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.760+0000 7f026066f700 1 -- 172.21.15.88:0/3316192867 shutdown_connections 2024-02-19T19:25:50.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.760+0000 7f026066f700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f024406d8a0 0x7f024406fd50 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:50.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.760+0000 7f026066f700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02581004e0 0x7f0258073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:50.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.760+0000 7f026066f700 1 --2- 172.21.15.88:0/3316192867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0258100e80 0x7f0258073590 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:50.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.760+0000 7f026066f700 1 -- 172.21.15.88:0/3316192867 >> 172.21.15.88:0/3316192867 conn(0x7f02580fbda0 msgr2=0x7f02580fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:50.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.760+0000 7f026066f700 1 -- 172.21.15.88:0/3316192867 shutdown_connections 2024-02-19T19:25:50.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.760+0000 7f026066f700 1 -- 172.21.15.88:0/3316192867 wait complete. 2024-02-19T19:25:50.777 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:25:50.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.865+0000 7f136acaf700 1 -- 172.21.15.88:0/3811289366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13641004e0 msgr2=0x7f13641008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.865+0000 7f136acaf700 1 --2- 172.21.15.88:0/3811289366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13641004e0 0x7f13641008b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1358009910 tx=0x7f1358009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.865+0000 7f136acaf700 1 -- 172.21.15.88:0/3811289366 shutdown_connections 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.865+0000 7f136acaf700 1 --2- 172.21.15.88:0/3811289366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1364100e80 0x7f1364104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.865+0000 7f136acaf700 1 --2- 172.21.15.88:0/3811289366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13641004e0 0x7f13641008b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f136acaf700 1 -- 172.21.15.88:0/3811289366 >> 172.21.15.88:0/3811289366 conn(0x7f13640fbda0 msgr2=0x7f13640fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f136acaf700 1 -- 172.21.15.88:0/3811289366 shutdown_connections 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f136acaf700 1 -- 172.21.15.88:0/3811289366 wait complete. 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f136acaf700 1 Processor -- start 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f136acaf700 1 -- start start 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f136acaf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13641004e0 0x7f1364195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f136acaf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1364100e80 0x7f1364196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:50.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f136acaf700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1364196a80 con 0x7f13641004e0 2024-02-19T19:25:50.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f136acaf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1364196bc0 con 0x7f1364100e80 2024-02-19T19:25:50.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f1368a4b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13641004e0 0x7f1364195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:50.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f1368a4b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13641004e0 0x7f1364195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57222/0 (socket says 172.21.15.88:57222) 2024-02-19T19:25:50.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.866+0000 7f1368a4b700 1 -- 172.21.15.88:0/3954165903 learned_addr learned my addr 172.21.15.88:0/3954165903 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:50.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.867+0000 7f1363fff700 1 --2- 172.21.15.88:0/3954165903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1364100e80 0x7f1364196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:50.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.867+0000 7f1368a4b700 1 -- 172.21.15.88:0/3954165903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1364100e80 msgr2=0x7f1364196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:50.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.867+0000 7f1368a4b700 1 --2- 172.21.15.88:0/3954165903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1364100e80 0x7f1364196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:50.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.867+0000 7f1368a4b700 1 -- 172.21.15.88:0/3954165903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13580095f0 con 0x7f13641004e0 2024-02-19T19:25:50.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.867+0000 7f1368a4b700 1 --2- 172.21.15.88:0/3954165903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13641004e0 0x7f1364195ef0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1358006010 tx=0x7f135800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:50.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.867+0000 7f1361ffb700 1 -- 172.21.15.88:0/3954165903 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f135801d070 con 0x7f13641004e0 2024-02-19T19:25:50.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.867+0000 7f1361ffb700 1 -- 172.21.15.88:0/3954165903 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1358004fb0 con 0x7f13641004e0 2024-02-19T19:25:50.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.867+0000 7f136acaf700 1 -- 172.21.15.88:0/3954165903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136419a9b0 con 0x7f13641004e0 2024-02-19T19:25:50.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.867+0000 7f1361ffb700 1 -- 172.21.15.88:0/3954165903 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1358022ea0 con 0x7f13641004e0 2024-02-19T19:25:50.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.867+0000 7f136acaf700 1 -- 172.21.15.88:0/3954165903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136419aea0 con 0x7f13641004e0 2024-02-19T19:25:50.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.868+0000 7f136acaf700 1 -- 172.21.15.88:0/3954165903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136404f2a0 con 0x7f13641004e0 2024-02-19T19:25:50.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.869+0000 7f1361ffb700 1 -- 172.21.15.88:0/3954165903 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1358022470 con 0x7f13641004e0 2024-02-19T19:25:50.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.869+0000 7f1361ffb700 1 --2- 172.21.15.88:0/3954165903 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f135406d8a0 0x7f135406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:50.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.869+0000 7f1361ffb700 1 -- 172.21.15.88:0/3954165903 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f135808ec10 con 0x7f13641004e0 2024-02-19T19:25:50.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.869+0000 7f1363fff700 1 --2- 172.21.15.88:0/3954165903 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f135406d8a0 0x7f135406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:50.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.870+0000 7f1363fff700 1 --2- 172.21.15.88:0/3954165903 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f135406d8a0 0x7f135406fd50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1350005ea0 tx=0x7f1350005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:50.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:50.874+0000 7f1361ffb700 1 -- 172.21.15.88:0/3954165903 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1358058590 con 0x7f13641004e0 2024-02-19T19:25:51.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.089+0000 7f136acaf700 1 -- 172.21.15.88:0/3954165903 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1364108240 con 0x7f135406d8a0 2024-02-19T19:25:51.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.090+0000 7f1361ffb700 1 -- 172.21.15.88:0/3954165903 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1364108240 con 0x7f135406d8a0 2024-02-19T19:25:51.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.092+0000 7f136acaf700 1 -- 172.21.15.88:0/3954165903 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f135406d8a0 msgr2=0x7f135406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.092+0000 7f136acaf700 1 --2- 172.21.15.88:0/3954165903 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f135406d8a0 0x7f135406fd50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1350005ea0 tx=0x7f1350005e30 comp rx=0 tx=0).stop 2024-02-19T19:25:51.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.092+0000 7f136acaf700 1 -- 172.21.15.88:0/3954165903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13641004e0 msgr2=0x7f1364195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.092+0000 7f136acaf700 1 --2- 172.21.15.88:0/3954165903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13641004e0 0x7f1364195ef0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1358006010 tx=0x7f135800be20 comp rx=0 tx=0).stop 2024-02-19T19:25:51.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.092+0000 7f136acaf700 1 -- 172.21.15.88:0/3954165903 shutdown_connections 2024-02-19T19:25:51.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.092+0000 7f136acaf700 1 --2- 172.21.15.88:0/3954165903 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f135406d8a0 0x7f135406fd50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.092+0000 7f136acaf700 1 --2- 172.21.15.88:0/3954165903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13641004e0 0x7f1364195ef0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.092+0000 7f136acaf700 1 --2- 172.21.15.88:0/3954165903 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1364100e80 0x7f1364196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.092+0000 7f136acaf700 1 -- 172.21.15.88:0/3954165903 >> 172.21.15.88:0/3954165903 conn(0x7f13640fbda0 msgr2=0x7f13640fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:51.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.093+0000 7f136acaf700 1 -- 172.21.15.88:0/3954165903 shutdown_connections 2024-02-19T19:25:51.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.093+0000 7f136acaf700 1 -- 172.21.15.88:0/3954165903 wait complete. 2024-02-19T19:25:51.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.197+0000 7f2f43037700 1 -- 172.21.15.88:0/2212521068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f3c1038b0 msgr2=0x7f2f3c103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.197+0000 7f2f43037700 1 --2- 172.21.15.88:0/2212521068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f3c1038b0 0x7f2f3c103c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800b0a0 tx=0x7f2f3800b3b0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.197+0000 7f2f43037700 1 -- 172.21.15.88:0/2212521068 shutdown_connections 2024-02-19T19:25:51.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.197+0000 7f2f43037700 1 --2- 172.21.15.88:0/2212521068 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f3c0ff350 0x7f2f3c0ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.197+0000 7f2f43037700 1 --2- 172.21.15.88:0/2212521068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f3c1038b0 0x7f2f3c103c80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.197+0000 7f2f43037700 1 -- 172.21.15.88:0/2212521068 >> 172.21.15.88:0/2212521068 conn(0x7f2f3c0748b0 msgr2=0x7f2f3c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:51.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.197+0000 7f2f43037700 1 -- 172.21.15.88:0/2212521068 shutdown_connections 2024-02-19T19:25:51.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.197+0000 7f2f43037700 1 -- 172.21.15.88:0/2212521068 wait complete. 2024-02-19T19:25:51.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.197+0000 7f2f43037700 1 Processor -- start 2024-02-19T19:25:51.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.197+0000 7f2f43037700 1 -- start start 2024-02-19T19:25:51.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f43037700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f3c0ff350 0x7f2f3c194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:51.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f43037700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f3c1038b0 0x7f2f3c195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:51.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f43037700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c195960 con 0x7f2f3c1038b0 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f43037700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c1996f0 con 0x7f2f3c0ff350 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f33fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f3c1038b0 0x7f2f3c195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f33fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f3c1038b0 0x7f2f3c195280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57248/0 (socket says 172.21.15.88:57248) 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f33fff700 1 -- 172.21.15.88:0/2443364977 learned_addr learned my addr 172.21.15.88:0/2443364977 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f40dd3700 1 --2- 172.21.15.88:0/2443364977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f3c0ff350 0x7f2f3c194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f33fff700 1 -- 172.21.15.88:0/2443364977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f3c0ff350 msgr2=0x7f2f3c194d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f33fff700 1 --2- 172.21.15.88:0/2443364977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f3c0ff350 0x7f2f3c194d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f33fff700 1 -- 172.21.15.88:0/2443364977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38009d00 con 0x7f2f3c1038b0 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f33fff700 1 --2- 172.21.15.88:0/2443364977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f3c1038b0 0x7f2f3c195280 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00d8d0 tx=0x7f2f2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f31ffb700 1 -- 172.21.15.88:0/2443364977 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f2c009940 con 0x7f2f3c1038b0 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f31ffb700 1 -- 172.21.15.88:0/2443364977 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f2c010460 con 0x7f2f3c1038b0 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f31ffb700 1 -- 172.21.15.88:0/2443364977 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f2c00f5f0 con 0x7f2f3c1038b0 2024-02-19T19:25:51.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.198+0000 7f2f43037700 1 -- 172.21.15.88:0/2443364977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c1999d0 con 0x7f2f3c1038b0 2024-02-19T19:25:51.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.199+0000 7f2f43037700 1 -- 172.21.15.88:0/2443364977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c199f20 con 0x7f2f3c1038b0 2024-02-19T19:25:51.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.200+0000 7f2f43037700 1 -- 172.21.15.88:0/2443364977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c04f2a0 con 0x7f2f3c1038b0 2024-02-19T19:25:51.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.200+0000 7f2f31ffb700 1 -- 172.21.15.88:0/2443364977 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2f2c0105d0 con 0x7f2f3c1038b0 2024-02-19T19:25:51.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.200+0000 7f2f31ffb700 1 --2- 172.21.15.88:0/2443364977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f2406d8a0 0x7f2f2406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:51.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.200+0000 7f2f31ffb700 1 -- 172.21.15.88:0/2443364977 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2f2c08d430 con 0x7f2f3c1038b0 2024-02-19T19:25:51.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.200+0000 7f2f40dd3700 1 --2- 172.21.15.88:0/2443364977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f2406d8a0 0x7f2f2406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:51.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.201+0000 7f2f40dd3700 1 --2- 172.21.15.88:0/2443364977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f2406d8a0 0x7f2f2406fd50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800b830 tx=0x7f2f380061f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:51.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.205+0000 7f2f31ffb700 1 -- 172.21.15.88:0/2443364977 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f2c056c60 con 0x7f2f3c1038b0 2024-02-19T19:25:51.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.410+0000 7f2f43037700 1 -- 172.21.15.88:0/2443364977 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f3c19a270 con 0x7f2f2406d8a0 2024-02-19T19:25:51.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.421+0000 7f2f31ffb700 1 -- 172.21.15.88:0/2443364977 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f2f3c19a270 con 0x7f2f2406d8a0 2024-02-19T19:25:51.428 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:25:51.428 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (23m) 14m ago 33m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:25:51.428 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (33m) 14m ago 33m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:25:51.428 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (15m) 14m ago 32m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:25:51.428 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (19m) 14m ago 33m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:25:51.428 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (19m) 14m ago 31m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:25:51.429 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (22m) 14m ago 32m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:25:51.429 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (15m) 14m ago 27m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:25:51.429 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (14m) 14m ago 27m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:25:51.429 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (14m) 14m ago 27m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:25:51.429 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (27m) 14m ago 27m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:25:51.429 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (24m) 14m ago 35m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:25:51.429 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (23m) 14m ago 31m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:25:51.429 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (21m) 14m ago 35m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:25:51.429 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (20m) 14m ago 31m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:25:51.430 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (23m) 14m ago 33m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:25:51.430 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (23m) 14m ago 31m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:25:51.430 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (18m) 14m ago 30m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:25:51.430 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (17m) 14m ago 30m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:25:51.430 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (17m) 14m ago 30m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:25:51.430 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (16m) 14m ago 29m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:25:51.430 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (16m) 14m ago 29m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:25:51.431 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (15m) 14m ago 28m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:25:51.431 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (23m) 14m ago 32m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:25:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.423+0000 7f2f43037700 1 -- 172.21.15.88:0/2443364977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f2406d8a0 msgr2=0x7f2f2406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.423+0000 7f2f43037700 1 --2- 172.21.15.88:0/2443364977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f2406d8a0 0x7f2f2406fd50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800b830 tx=0x7f2f380061f0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.423+0000 7f2f43037700 1 -- 172.21.15.88:0/2443364977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f3c1038b0 msgr2=0x7f2f3c195280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.423+0000 7f2f43037700 1 --2- 172.21.15.88:0/2443364977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f3c1038b0 0x7f2f3c195280 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00d8d0 tx=0x7f2f2c00dc90 comp rx=0 tx=0).stop 2024-02-19T19:25:51.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.423+0000 7f2f43037700 1 -- 172.21.15.88:0/2443364977 shutdown_connections 2024-02-19T19:25:51.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.423+0000 7f2f43037700 1 --2- 172.21.15.88:0/2443364977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f2406d8a0 0x7f2f2406fd50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.423+0000 7f2f43037700 1 --2- 172.21.15.88:0/2443364977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f3c0ff350 0x7f2f3c194d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.423+0000 7f2f43037700 1 --2- 172.21.15.88:0/2443364977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f3c1038b0 0x7f2f3c195280 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.423+0000 7f2f43037700 1 -- 172.21.15.88:0/2443364977 >> 172.21.15.88:0/2443364977 conn(0x7f2f3c0748b0 msgr2=0x7f2f3c0fe340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:51.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.423+0000 7f2f43037700 1 -- 172.21.15.88:0/2443364977 shutdown_connections 2024-02-19T19:25:51.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.423+0000 7f2f43037700 1 -- 172.21.15.88:0/2443364977 wait complete. 2024-02-19T19:25:51.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.528+0000 7f39b459b700 1 -- 172.21.15.88:0/138027197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac1004e0 msgr2=0x7f39ac1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.528+0000 7f39b459b700 1 --2- 172.21.15.88:0/138027197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac1004e0 0x7f39ac1008b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009910 tx=0x7f39a8009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:51.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 -- 172.21.15.88:0/138027197 shutdown_connections 2024-02-19T19:25:51.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 --2- 172.21.15.88:0/138027197 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39ac100e80 0x7f39ac104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 --2- 172.21.15.88:0/138027197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac1004e0 0x7f39ac1008b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 -- 172.21.15.88:0/138027197 >> 172.21.15.88:0/138027197 conn(0x7f39ac0fbda0 msgr2=0x7f39ac0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:51.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 -- 172.21.15.88:0/138027197 shutdown_connections 2024-02-19T19:25:51.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 -- 172.21.15.88:0/138027197 wait complete. 2024-02-19T19:25:51.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 Processor -- start 2024-02-19T19:25:51.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 -- start start 2024-02-19T19:25:51.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac1004e0 0x7f39ac195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:51.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39ac100e80 0x7f39ac1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:51.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac196bb0 con 0x7f39ac1004e0 2024-02-19T19:25:51.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.529+0000 7f39b459b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac19a940 con 0x7f39ac100e80 2024-02-19T19:25:51.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39b2337700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac1004e0 0x7f39ac195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39b2337700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac1004e0 0x7f39ac195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57264/0 (socket says 172.21.15.88:57264) 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39b2337700 1 -- 172.21.15.88:0/1847725919 learned_addr learned my addr 172.21.15.88:0/1847725919 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39b2337700 1 -- 172.21.15.88:0/1847725919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39ac100e80 msgr2=0x7f39ac1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39b2337700 1 --2- 172.21.15.88:0/1847725919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39ac100e80 0x7f39ac1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39b2337700 1 -- 172.21.15.88:0/1847725919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80095f0 con 0x7f39ac1004e0 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39b2337700 1 --2- 172.21.15.88:0/1847725919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac1004e0 0x7f39ac195f90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f39a8006010 tx=0x7f39a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39a37fe700 1 -- 172.21.15.88:0/1847725919 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39a801d070 con 0x7f39ac1004e0 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39a37fe700 1 -- 172.21.15.88:0/1847725919 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39a8004fb0 con 0x7f39ac1004e0 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39b459b700 1 -- 172.21.15.88:0/1847725919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac19abc0 con 0x7f39ac1004e0 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39a37fe700 1 -- 172.21.15.88:0/1847725919 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39a8022ea0 con 0x7f39ac1004e0 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.530+0000 7f39b459b700 1 -- 172.21.15.88:0/1847725919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac19b0b0 con 0x7f39ac1004e0 2024-02-19T19:25:51.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.531+0000 7f39b459b700 1 -- 172.21.15.88:0/1847725919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ac1087f0 con 0x7f39ac1004e0 2024-02-19T19:25:51.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.532+0000 7f39a37fe700 1 -- 172.21.15.88:0/1847725919 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f39a8022470 con 0x7f39ac1004e0 2024-02-19T19:25:51.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.532+0000 7f39a37fe700 1 --2- 172.21.15.88:0/1847725919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 0x7f399806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:51.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.532+0000 7f39b1b36700 1 --2- 172.21.15.88:0/1847725919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 0x7f399806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:51.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.532+0000 7f39a37fe700 1 -- 172.21.15.88:0/1847725919 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f39a808ebb0 con 0x7f39ac1004e0 2024-02-19T19:25:51.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.533+0000 7f39b1b36700 1 --2- 172.21.15.88:0/1847725919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 0x7f399806fd50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f39ac1975b0 tx=0x7f399c006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:51.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.537+0000 7f39a37fe700 1 -- 172.21.15.88:0/1847725919 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f39a8095080 con 0x7f39ac1004e0 2024-02-19T19:25:51.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.817+0000 7f39b459b700 1 -- 172.21.15.88:0/1847725919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f39ac02ccf0 con 0x7f39ac1004e0 2024-02-19T19:25:51.824 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:51 smithi088 ceph-mon[88643]: pgmap v667: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:51.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.817+0000 7f39a37fe700 1 -- 172.21.15.88:0/1847725919 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7f39a8027070 con 0x7f39ac1004e0 2024-02-19T19:25:51.824 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:25:51.824 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:25:51.824 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:25:51.824 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 3 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 13 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:25:51.825 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:25:51.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.820+0000 7f39b459b700 1 -- 172.21.15.88:0/1847725919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 msgr2=0x7f399806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.820+0000 7f39b459b700 1 --2- 172.21.15.88:0/1847725919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 0x7f399806fd50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f39ac1975b0 tx=0x7f399c006d20 comp rx=0 tx=0).stop 2024-02-19T19:25:51.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.820+0000 7f39b459b700 1 -- 172.21.15.88:0/1847725919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac1004e0 msgr2=0x7f39ac195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.820+0000 7f39b459b700 1 --2- 172.21.15.88:0/1847725919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac1004e0 0x7f39ac195f90 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f39a8006010 tx=0x7f39a800be20 comp rx=0 tx=0).stop 2024-02-19T19:25:51.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.820+0000 7f39b459b700 1 -- 172.21.15.88:0/1847725919 shutdown_connections 2024-02-19T19:25:51.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.820+0000 7f39b459b700 1 --2- 172.21.15.88:0/1847725919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 0x7f399806fd50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.820+0000 7f39b459b700 1 --2- 172.21.15.88:0/1847725919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac1004e0 0x7f39ac195f90 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.820+0000 7f39b459b700 1 --2- 172.21.15.88:0/1847725919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39ac100e80 0x7f39ac1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.820+0000 7f39b459b700 1 -- 172.21.15.88:0/1847725919 >> 172.21.15.88:0/1847725919 conn(0x7f39ac0fbda0 msgr2=0x7f39ac0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:51.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.820+0000 7f39b459b700 1 -- 172.21.15.88:0/1847725919 shutdown_connections 2024-02-19T19:25:51.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.820+0000 7f39b459b700 1 -- 172.21.15.88:0/1847725919 wait complete. 2024-02-19T19:25:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.924+0000 7fba82e70700 1 -- 172.21.15.88:0/1460610866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c0ffbd0 msgr2=0x7fba7c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.924+0000 7fba82e70700 1 --2- 172.21.15.88:0/1460610866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c0ffbd0 0x7fba7c100040 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fba6c009910 tx=0x7fba6c009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.924+0000 7fba82e70700 1 -- 172.21.15.88:0/1460610866 shutdown_connections 2024-02-19T19:25:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.924+0000 7fba82e70700 1 --2- 172.21.15.88:0/1460610866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c0ffbd0 0x7fba7c100040 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.924+0000 7fba82e70700 1 --2- 172.21.15.88:0/1460610866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c104130 0x7fba7c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.924+0000 7fba82e70700 1 -- 172.21.15.88:0/1460610866 >> 172.21.15.88:0/1460610866 conn(0x7fba7c075130 msgr2=0x7fba7c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.924+0000 7fba82e70700 1 -- 172.21.15.88:0/1460610866 shutdown_connections 2024-02-19T19:25:51.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.924+0000 7fba82e70700 1 -- 172.21.15.88:0/1460610866 wait complete. 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.924+0000 7fba82e70700 1 Processor -- start 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.924+0000 7fba82e70700 1 -- start start 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba82e70700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c0ffbd0 0x7fba7c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba82e70700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c104130 0x7fba7c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba82e70700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1018b0 con 0x7fba7c0ffbd0 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba82e70700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7c1019f0 con 0x7fba7c104130 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba80c0c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c0ffbd0 0x7fba7c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba80c0c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c0ffbd0 0x7fba7c100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57268/0 (socket says 172.21.15.88:57268) 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba80c0c700 1 -- 172.21.15.88:0/468389702 learned_addr learned my addr 172.21.15.88:0/468389702 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba7bfff700 1 --2- 172.21.15.88:0/468389702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c104130 0x7fba7c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba80c0c700 1 -- 172.21.15.88:0/468389702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c104130 msgr2=0x7fba7c101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba80c0c700 1 --2- 172.21.15.88:0/468389702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c104130 0x7fba7c101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba80c0c700 1 -- 172.21.15.88:0/468389702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c0095f0 con 0x7fba7c0ffbd0 2024-02-19T19:25:51.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba80c0c700 1 --2- 172.21.15.88:0/468389702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c0ffbd0 0x7fba7c100e30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fba7400ba50 tx=0x7fba7400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:51.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba79ffb700 1 -- 172.21.15.88:0/468389702 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba7400ca00 con 0x7fba7c0ffbd0 2024-02-19T19:25:51.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba79ffb700 1 -- 172.21.15.88:0/468389702 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba7400cb60 con 0x7fba7c0ffbd0 2024-02-19T19:25:51.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba82e70700 1 -- 172.21.15.88:0/468389702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba7c1a2130 con 0x7fba7c0ffbd0 2024-02-19T19:25:51.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.925+0000 7fba79ffb700 1 -- 172.21.15.88:0/468389702 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba740075a0 con 0x7fba7c0ffbd0 2024-02-19T19:25:51.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.926+0000 7fba82e70700 1 -- 172.21.15.88:0/468389702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba7c1a2540 con 0x7fba7c0ffbd0 2024-02-19T19:25:51.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.927+0000 7fba82e70700 1 -- 172.21.15.88:0/468389702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba7c1086a0 con 0x7fba7c0ffbd0 2024-02-19T19:25:51.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.927+0000 7fba79ffb700 1 -- 172.21.15.88:0/468389702 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fba7400ccd0 con 0x7fba7c0ffbd0 2024-02-19T19:25:51.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.927+0000 7fba79ffb700 1 --2- 172.21.15.88:0/468389702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba6806d8a0 0x7fba6806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:51.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.927+0000 7fba79ffb700 1 -- 172.21.15.88:0/468389702 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fba74006e40 con 0x7fba7c0ffbd0 2024-02-19T19:25:51.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.927+0000 7fba7bfff700 1 --2- 172.21.15.88:0/468389702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba6806d8a0 0x7fba6806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:51.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.928+0000 7fba7bfff700 1 --2- 172.21.15.88:0/468389702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba6806d8a0 0x7fba6806fd50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0059d0 tx=0x7fba6c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:51.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:51.932+0000 7fba79ffb700 1 -- 172.21.15.88:0/468389702 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba74056490 con 0x7fba7c0ffbd0 2024-02-19T19:25:51.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:51 smithi132 ceph-mon[70676]: pgmap v667: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:52.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.176+0000 7fba82e70700 1 -- 172.21.15.88:0/468389702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fba7c04f2a0 con 0x7fba7c0ffbd0 2024-02-19T19:25:52.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.176+0000 7fba79ffb700 1 -- 172.21.15.88:0/468389702 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1835 (secure 0 0 0) 0x7fba74019070 con 0x7fba7c0ffbd0 2024-02-19T19:25:52.182 INFO:teuthology.orchestra.run.smithi088.stdout:e30 2024-02-19T19:25:52.182 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:25:52.182 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:25:52.182 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:25:52.182 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:25:52.183 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:25:52.184 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:25:52.185 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:25:52.185 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:25:52.185 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:25:52.185 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:25:52.185 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:24275} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.132:6826/3724797063,v1:172.21.15.132:6827/3724797063] compat {c=[1],r=[1],i=[7ff]}] 2024-02-19T19:25:52.185 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:25:52.185 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:25:52.185 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:25:52.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.178+0000 7fba82e70700 1 -- 172.21.15.88:0/468389702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba6806d8a0 msgr2=0x7fba6806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:52.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.178+0000 7fba82e70700 1 --2- 172.21.15.88:0/468389702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba6806d8a0 0x7fba6806fd50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fba6c0059d0 tx=0x7fba6c01a040 comp rx=0 tx=0).stop 2024-02-19T19:25:52.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.178+0000 7fba82e70700 1 -- 172.21.15.88:0/468389702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c0ffbd0 msgr2=0x7fba7c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:52.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.178+0000 7fba82e70700 1 --2- 172.21.15.88:0/468389702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c0ffbd0 0x7fba7c100e30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fba7400ba50 tx=0x7fba7400be10 comp rx=0 tx=0).stop 2024-02-19T19:25:52.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.178+0000 7fba82e70700 1 -- 172.21.15.88:0/468389702 shutdown_connections 2024-02-19T19:25:52.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.178+0000 7fba82e70700 1 --2- 172.21.15.88:0/468389702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba6806d8a0 0x7fba6806fd50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.179+0000 7fba82e70700 1 --2- 172.21.15.88:0/468389702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba7c0ffbd0 0x7fba7c100e30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.179+0000 7fba82e70700 1 --2- 172.21.15.88:0/468389702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba7c104130 0x7fba7c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.179+0000 7fba82e70700 1 -- 172.21.15.88:0/468389702 >> 172.21.15.88:0/468389702 conn(0x7fba7c075130 msgr2=0x7fba7c0fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:52.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.179+0000 7fba82e70700 1 -- 172.21.15.88:0/468389702 shutdown_connections 2024-02-19T19:25:52.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.179+0000 7fba82e70700 1 -- 172.21.15.88:0/468389702 wait complete. 2024-02-19T19:25:52.187 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:25:52.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.284+0000 7ff8d5777700 1 -- 172.21.15.88:0/1702716709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d0104130 msgr2=0x7ff8d0104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:52.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.284+0000 7ff8d5777700 1 --2- 172.21.15.88:0/1702716709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d0104130 0x7ff8d0104500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009910 tx=0x7ff8c0009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:52.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.284+0000 7ff8d5777700 1 -- 172.21.15.88:0/1702716709 shutdown_connections 2024-02-19T19:25:52.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.284+0000 7ff8d5777700 1 --2- 172.21.15.88:0/1702716709 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d00ffbd0 0x7ff8d0100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.284+0000 7ff8d5777700 1 --2- 172.21.15.88:0/1702716709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d0104130 0x7ff8d0104500 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.284+0000 7ff8d5777700 1 -- 172.21.15.88:0/1702716709 >> 172.21.15.88:0/1702716709 conn(0x7ff8d0075130 msgr2=0x7ff8d0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:52.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.284+0000 7ff8d5777700 1 -- 172.21.15.88:0/1702716709 shutdown_connections 2024-02-19T19:25:52.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.284+0000 7ff8d5777700 1 -- 172.21.15.88:0/1702716709 wait complete. 2024-02-19T19:25:52.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.284+0000 7ff8d5777700 1 Processor -- start 2024-02-19T19:25:52.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8d5777700 1 -- start start 2024-02-19T19:25:52.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8d5777700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d00ffbd0 0x7ff8d0100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8d5777700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d0104130 0x7ff8d0101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8d5777700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0101940 con 0x7ff8d00ffbd0 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8d5777700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d0101ab0 con 0x7ff8d0104130 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8ceffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d00ffbd0 0x7ff8d0100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8ceffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d00ffbd0 0x7ff8d0100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57284/0 (socket says 172.21.15.88:57284) 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8ceffd700 1 -- 172.21.15.88:0/4193750017 learned_addr learned my addr 172.21.15.88:0/4193750017 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8ceffd700 1 -- 172.21.15.88:0/4193750017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d0104130 msgr2=0x7ff8d0101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8ceffd700 1 --2- 172.21.15.88:0/4193750017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d0104130 0x7ff8d0101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8ceffd700 1 -- 172.21.15.88:0/4193750017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c00095f0 con 0x7ff8d00ffbd0 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8ceffd700 1 --2- 172.21.15.88:0/4193750017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d00ffbd0 0x7ff8d0100e30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000bd70 tx=0x7ff8c000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8ccff9700 1 -- 172.21.15.88:0/4193750017 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8c001d070 con 0x7ff8d00ffbd0 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.285+0000 7ff8d5777700 1 -- 172.21.15.88:0/4193750017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d01a21a0 con 0x7ff8d00ffbd0 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.286+0000 7ff8ccff9700 1 -- 172.21.15.88:0/4193750017 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8c000f460 con 0x7ff8d00ffbd0 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.286+0000 7ff8ccff9700 1 -- 172.21.15.88:0/4193750017 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8c0017630 con 0x7ff8d00ffbd0 2024-02-19T19:25:52.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.286+0000 7ff8d5777700 1 -- 172.21.15.88:0/4193750017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d01a2550 con 0x7ff8d00ffbd0 2024-02-19T19:25:52.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.286+0000 7ff8d5777700 1 -- 172.21.15.88:0/4193750017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d004f2a0 con 0x7ff8d00ffbd0 2024-02-19T19:25:52.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.287+0000 7ff8ccff9700 1 -- 172.21.15.88:0/4193750017 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff8c000f5d0 con 0x7ff8d00ffbd0 2024-02-19T19:25:52.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.287+0000 7ff8ccff9700 1 --2- 172.21.15.88:0/4193750017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8bc06d8a0 0x7ff8bc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:52.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.287+0000 7ff8ccff9700 1 -- 172.21.15.88:0/4193750017 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff8c008eb60 con 0x7ff8d00ffbd0 2024-02-19T19:25:52.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.287+0000 7ff8c67fc700 1 --2- 172.21.15.88:0/4193750017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8bc06d8a0 0x7ff8bc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:52.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.288+0000 7ff8c67fc700 1 --2- 172.21.15.88:0/4193750017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8bc06d8a0 0x7ff8bc06fd50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0102620 tx=0x7ff8b8006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:52.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.292+0000 7ff8ccff9700 1 -- 172.21.15.88:0/4193750017 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff8c0058310 con 0x7ff8d00ffbd0 2024-02-19T19:25:52.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.509+0000 7ff8d5777700 1 -- 172.21.15.88:0/4193750017 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8d0068540 con 0x7ff8bc06d8a0 2024-02-19T19:25:52.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.510+0000 7ff8ccff9700 1 -- 172.21.15.88:0/4193750017 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff8d0068540 con 0x7ff8bc06d8a0 2024-02-19T19:25:52.516 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:25:52.516 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:25:52.516 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:25:52.517 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:25:52.517 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:25:52.517 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:25:52.517 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:25:52.517 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:25:52.517 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:25:52.517 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:25:52.517 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:25:52.517 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:25:52.517 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:25:52.517 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:25:52.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.512+0000 7ff8d5777700 1 -- 172.21.15.88:0/4193750017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8bc06d8a0 msgr2=0x7ff8bc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:52.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.512+0000 7ff8d5777700 1 --2- 172.21.15.88:0/4193750017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8bc06d8a0 0x7ff8bc06fd50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0102620 tx=0x7ff8b8006d20 comp rx=0 tx=0).stop 2024-02-19T19:25:52.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.512+0000 7ff8d5777700 1 -- 172.21.15.88:0/4193750017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d00ffbd0 msgr2=0x7ff8d0100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:52.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.512+0000 7ff8d5777700 1 --2- 172.21.15.88:0/4193750017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d00ffbd0 0x7ff8d0100e30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff8c000bd70 tx=0x7ff8c000be50 comp rx=0 tx=0).stop 2024-02-19T19:25:52.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.512+0000 7ff8d5777700 1 -- 172.21.15.88:0/4193750017 shutdown_connections 2024-02-19T19:25:52.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.512+0000 7ff8d5777700 1 --2- 172.21.15.88:0/4193750017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff8bc06d8a0 0x7ff8bc06fd50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.512+0000 7ff8d5777700 1 --2- 172.21.15.88:0/4193750017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8d00ffbd0 0x7ff8d0100e30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.512+0000 7ff8d5777700 1 --2- 172.21.15.88:0/4193750017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff8d0104130 0x7ff8d0101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.512+0000 7ff8d5777700 1 -- 172.21.15.88:0/4193750017 >> 172.21.15.88:0/4193750017 conn(0x7ff8d0075130 msgr2=0x7ff8d0109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:52.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.513+0000 7ff8d5777700 1 -- 172.21.15.88:0/4193750017 shutdown_connections 2024-02-19T19:25:52.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.513+0000 7ff8d5777700 1 -- 172.21.15.88:0/4193750017 wait complete. 2024-02-19T19:25:52.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.616+0000 7fdadecbf700 1 -- 172.21.15.88:0/1546755837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdad8104130 msgr2=0x7fdad8104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:52.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.616+0000 7fdadecbf700 1 --2- 172.21.15.88:0/1546755837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdad8104130 0x7fdad8104500 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009910 tx=0x7fdacc009c20 comp rx=0 tx=0).stop 2024-02-19T19:25:52.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.616+0000 7fdadecbf700 1 -- 172.21.15.88:0/1546755837 shutdown_connections 2024-02-19T19:25:52.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.616+0000 7fdadecbf700 1 --2- 172.21.15.88:0/1546755837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdad80ffbd0 0x7fdad8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.616+0000 7fdadecbf700 1 --2- 172.21.15.88:0/1546755837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdad8104130 0x7fdad8104500 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.616+0000 7fdadecbf700 1 -- 172.21.15.88:0/1546755837 >> 172.21.15.88:0/1546755837 conn(0x7fdad8075130 msgr2=0x7fdad8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:52.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.616+0000 7fdadecbf700 1 -- 172.21.15.88:0/1546755837 shutdown_connections 2024-02-19T19:25:52.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.616+0000 7fdadecbf700 1 -- 172.21.15.88:0/1546755837 wait complete. 2024-02-19T19:25:52.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadecbf700 1 Processor -- start 2024-02-19T19:25:52.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadecbf700 1 -- start start 2024-02-19T19:25:52.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadecbf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdad80ffbd0 0x7fdad8195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadecbf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdad8104130 0x7fdad8196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadecbf700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad8196a70 con 0x7fdad80ffbd0 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadecbf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad819a800 con 0x7fdad8104130 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadca5b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdad80ffbd0 0x7fdad8195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadca5b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdad80ffbd0 0x7fdad8195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57304/0 (socket says 172.21.15.88:57304) 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadca5b700 1 -- 172.21.15.88:0/133622137 learned_addr learned my addr 172.21.15.88:0/133622137 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadca5b700 1 -- 172.21.15.88:0/133622137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdad8104130 msgr2=0x7fdad8196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadca5b700 1 --2- 172.21.15.88:0/133622137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdad8104130 0x7fdad8196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadca5b700 1 -- 172.21.15.88:0/133622137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc0095f0 con 0x7fdad80ffbd0 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.617+0000 7fdadca5b700 1 --2- 172.21.15.88:0/133622137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdad80ffbd0 0x7fdad8195e50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00bd70 tx=0x7fdacc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.618+0000 7fdad5ffb700 1 -- 172.21.15.88:0/133622137 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdacc01d070 con 0x7fdad80ffbd0 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.618+0000 7fdad5ffb700 1 -- 172.21.15.88:0/133622137 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdacc00f460 con 0x7fdad80ffbd0 2024-02-19T19:25:52.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.618+0000 7fdadecbf700 1 -- 172.21.15.88:0/133622137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad819aa80 con 0x7fdad80ffbd0 2024-02-19T19:25:52.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.618+0000 7fdad5ffb700 1 -- 172.21.15.88:0/133622137 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdacc017630 con 0x7fdad80ffbd0 2024-02-19T19:25:52.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.618+0000 7fdadecbf700 1 -- 172.21.15.88:0/133622137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad819af70 con 0x7fdad80ffbd0 2024-02-19T19:25:52.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.619+0000 7fdadecbf700 1 -- 172.21.15.88:0/133622137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad81086a0 con 0x7fdad80ffbd0 2024-02-19T19:25:52.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.619+0000 7fdad5ffb700 1 -- 172.21.15.88:0/133622137 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdacc00f5d0 con 0x7fdad80ffbd0 2024-02-19T19:25:52.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.620+0000 7fdad5ffb700 1 --2- 172.21.15.88:0/133622137 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdac806d8a0 0x7fdac806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:25:52.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.620+0000 7fdad5ffb700 1 -- 172.21.15.88:0/133622137 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdacc08ea90 con 0x7fdad80ffbd0 2024-02-19T19:25:52.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.620+0000 7fdad7fff700 1 --2- 172.21.15.88:0/133622137 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdac806d8a0 0x7fdac806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:25:52.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.620+0000 7fdad7fff700 1 --2- 172.21.15.88:0/133622137 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdac806d8a0 0x7fdac806fd50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdad8197470 tx=0x7fdac4005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:25:52.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.624+0000 7fdad5ffb700 1 -- 172.21.15.88:0/133622137 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdacc0582c0 con 0x7fdad80ffbd0 2024-02-19T19:25:52.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:52 smithi088 ceph-mon[88643]: from='client.35226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:52.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:52 smithi088 ceph-mon[88643]: from='client.35230 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:52.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:52 smithi088 ceph-mon[88643]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:52.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:52 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1847725919' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:25:52.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:52 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/468389702' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:25:52.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.907+0000 7fdadecbf700 1 -- 172.21.15.88:0/133622137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fdad804f2a0 con 0x7fdad80ffbd0 2024-02-19T19:25:52.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.907+0000 7fdad5ffb700 1 -- 172.21.15.88:0/133622137 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fdacc05b980 con 0x7fdad80ffbd0 2024-02-19T19:25:52.913 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:25:52.914 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:25:52.914 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:25:52.914 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:25:52.914 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:25:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.909+0000 7fdadecbf700 1 -- 172.21.15.88:0/133622137 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdac806d8a0 msgr2=0x7fdac806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.910+0000 7fdadecbf700 1 --2- 172.21.15.88:0/133622137 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdac806d8a0 0x7fdac806fd50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fdad8197470 tx=0x7fdac4005c80 comp rx=0 tx=0).stop 2024-02-19T19:25:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.910+0000 7fdadecbf700 1 -- 172.21.15.88:0/133622137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdad80ffbd0 msgr2=0x7fdad8195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:25:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.910+0000 7fdadecbf700 1 --2- 172.21.15.88:0/133622137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdad80ffbd0 0x7fdad8195e50 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fdacc00bd70 tx=0x7fdacc00be50 comp rx=0 tx=0).stop 2024-02-19T19:25:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.910+0000 7fdadecbf700 1 -- 172.21.15.88:0/133622137 shutdown_connections 2024-02-19T19:25:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.910+0000 7fdadecbf700 1 --2- 172.21.15.88:0/133622137 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdac806d8a0 0x7fdac806fd50 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.910+0000 7fdadecbf700 1 --2- 172.21.15.88:0/133622137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdad80ffbd0 0x7fdad8195e50 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.910+0000 7fdadecbf700 1 --2- 172.21.15.88:0/133622137 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdad8104130 0x7fdad8196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:25:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.910+0000 7fdadecbf700 1 -- 172.21.15.88:0/133622137 >> 172.21.15.88:0/133622137 conn(0x7fdad8075130 msgr2=0x7fdad80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:25:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.910+0000 7fdadecbf700 1 -- 172.21.15.88:0/133622137 shutdown_connections 2024-02-19T19:25:52.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:25:52.910+0000 7fdadecbf700 1 -- 172.21.15.88:0/133622137 wait complete. 2024-02-19T19:25:52.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:52 smithi132 ceph-mon[70676]: from='client.35226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:52.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:52 smithi132 ceph-mon[70676]: from='client.35230 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:52.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:52 smithi132 ceph-mon[70676]: from='client.35234 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:52.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:52 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1847725919' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:25:52.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:52 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/468389702' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:25:53.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:53 smithi088 ceph-mon[88643]: pgmap v668: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:53.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:53 smithi088 ceph-mon[88643]: from='client.35246 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:53.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:53 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/133622137' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:25:53.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:53 smithi132 ceph-mon[70676]: pgmap v668: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:53.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:53 smithi132 ceph-mon[70676]: from='client.35246 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:25:53.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:53 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/133622137' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:25:55.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:55 smithi088 ceph-mon[88643]: pgmap v669: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:55.905 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:55 smithi132 ceph-mon[70676]: pgmap v669: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:57.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:57 smithi088 ceph-mon[88643]: pgmap v670: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:57.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:57 smithi132 ceph-mon[70676]: pgmap v670: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:58.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:25:58.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:25:59.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:25:59 smithi088 ceph-mon[88643]: pgmap v671: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:25:59.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:25:59 smithi132 ceph-mon[70676]: pgmap v671: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:01.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:01 smithi088 ceph-mon[88643]: pgmap v672: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:01.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:01 smithi132 ceph-mon[70676]: pgmap v672: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:03.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:03 smithi088 ceph-mon[88643]: pgmap v673: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:03.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:03 smithi132 ceph-mon[70676]: pgmap v673: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:05.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:05 smithi088 ceph-mon[88643]: pgmap v674: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:05.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:05 smithi132 ceph-mon[70676]: pgmap v674: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:07.797 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:07 smithi088 ceph-mon[88643]: pgmap v675: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:07.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:07 smithi132 ceph-mon[70676]: pgmap v675: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:09.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:09 smithi088 ceph-mon[88643]: pgmap v676: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:09.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:09 smithi132 ceph-mon[70676]: pgmap v676: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:11.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:11 smithi132 ceph-mon[70676]: pgmap v677: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:11.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:11 smithi088 ceph-mon[88643]: pgmap v677: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:13.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:13 smithi132 ceph-mon[70676]: pgmap v678: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:13.739 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:26:13.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:13 smithi088 ceph-mon[88643]: pgmap v678: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:13.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:26:15.530 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:15 smithi132 ceph-mon[70676]: pgmap v679: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:15.530 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:15 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:26:15.530 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:15 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:26:15.531 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:15 smithi132 ceph-mon[70676]: pgmap v680: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:15.531 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:15 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:15.531 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:15 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:26:15.531 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:15 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:15.531 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:15 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:15.531 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:15 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:15.531 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:15 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:15.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:15 smithi088 ceph-mon[88643]: pgmap v679: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:15.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:26:15.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:26:15.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:15 smithi088 ceph-mon[88643]: pgmap v680: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:15.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:15.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:26:15.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:15.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:15.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:15.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:15 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:16.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:16 smithi132 ceph-mon[70676]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-02-19T19:26:16.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:16.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.uyjxaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:26:16.739 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:26:16.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:16 smithi088 ceph-mon[88643]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-02-19T19:26:16.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:16.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi132.uyjxaz", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-19T19:26:16.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:26:17.698 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:17 smithi132 ceph-mon[70676]: Upgrade: Updating mds.cephfs.smithi132.uyjxaz 2024-02-19T19:26:17.698 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:17 smithi132 ceph-mon[70676]: Deploying daemon mds.cephfs.smithi132.uyjxaz on smithi132 2024-02-19T19:26:17.698 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:17 smithi132 ceph-mon[70676]: pgmap v681: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:17.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:17 smithi088 ceph-mon[88643]: Upgrade: Updating mds.cephfs.smithi132.uyjxaz 2024-02-19T19:26:17.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:17 smithi088 ceph-mon[88643]: Deploying daemon mds.cephfs.smithi132.uyjxaz on smithi132 2024-02-19T19:26:17.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:17 smithi088 ceph-mon[88643]: pgmap v681: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:19.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:19 smithi132 ceph-mon[70676]: pgmap v682: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:19.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:19 smithi132 ceph-mon[70676]: osdmap e88: 6 total, 6 up, 6 in 2024-02-19T19:26:19.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:19 smithi132 ceph-mon[70676]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-19T19:26:19.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:26:20.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:19 smithi088 ceph-mon[88643]: pgmap v682: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:20.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:19 smithi088 ceph-mon[88643]: osdmap e88: 6 total, 6 up, 6 in 2024-02-19T19:26:20.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:19 smithi088 ceph-mon[88643]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-02-19T19:26:20.087 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:26:21.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:21 smithi132 ceph-mon[70676]: pgmap v684: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:22.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:21 smithi088 ceph-mon[88643]: pgmap v684: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.031+0000 7f773cb9d700 1 -- 172.21.15.88:0/3218776804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77380ff9e0 msgr2=0x7f77380ffe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.031+0000 7f773cb9d700 1 --2- 172.21.15.88:0/3218776804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77380ff9e0 0x7f77380ffe50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7720009910 tx=0x7f7720009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.031+0000 7f773cb9d700 1 -- 172.21.15.88:0/3218776804 shutdown_connections 2024-02-19T19:26:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.031+0000 7f773cb9d700 1 --2- 172.21.15.88:0/3218776804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77380ff9e0 0x7f77380ffe50 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.031+0000 7f773cb9d700 1 --2- 172.21.15.88:0/3218776804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7738102e30 0x7f7738103200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.031+0000 7f773cb9d700 1 -- 172.21.15.88:0/3218776804 >> 172.21.15.88:0/3218776804 conn(0x7f7738074fa0 msgr2=0x7f77380753a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.031+0000 7f773cb9d700 1 -- 172.21.15.88:0/3218776804 shutdown_connections 2024-02-19T19:26:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.031+0000 7f773cb9d700 1 -- 172.21.15.88:0/3218776804 wait complete. 2024-02-19T19:26:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.032+0000 7f773cb9d700 1 Processor -- start 2024-02-19T19:26:23.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.032+0000 7f773cb9d700 1 -- start start 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.032+0000 7f773cb9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77380ff9e0 0x7f7738196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.032+0000 7f773cb9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7738102e30 0x7f7738196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.032+0000 7f773cb9d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77381975b0 con 0x7f7738102e30 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.032+0000 7f773cb9d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7738197720 con 0x7f77380ff9e0 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.032+0000 7f772effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7738102e30 0x7f7738196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.032+0000 7f772effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7738102e30 0x7f7738196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35084/0 (socket says 172.21.15.88:35084) 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.032+0000 7f772effd700 1 -- 172.21.15.88:0/33236047 learned_addr learned my addr 172.21.15.88:0/33236047 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.032+0000 7f77377fe700 1 --2- 172.21.15.88:0/33236047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77380ff9e0 0x7f7738196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.033+0000 7f772effd700 1 -- 172.21.15.88:0/33236047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77380ff9e0 msgr2=0x7f7738196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.033+0000 7f772effd700 1 --2- 172.21.15.88:0/33236047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77380ff9e0 0x7f7738196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.033+0000 7f772effd700 1 -- 172.21.15.88:0/33236047 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77200095f0 con 0x7f7738102e30 2024-02-19T19:26:23.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.033+0000 7f772effd700 1 --2- 172.21.15.88:0/33236047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7738102e30 0x7f7738196ed0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7720006010 tx=0x7f772000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.033+0000 7f77357fa700 1 -- 172.21.15.88:0/33236047 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f772001d070 con 0x7f7738102e30 2024-02-19T19:26:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.033+0000 7f77357fa700 1 -- 172.21.15.88:0/33236047 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7720005dc0 con 0x7f7738102e30 2024-02-19T19:26:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.033+0000 7f773cb9d700 1 -- 172.21.15.88:0/33236047 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f773819fa00 con 0x7f7738102e30 2024-02-19T19:26:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.033+0000 7f77357fa700 1 -- 172.21.15.88:0/33236047 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7720017400 con 0x7f7738102e30 2024-02-19T19:26:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.033+0000 7f773cb9d700 1 -- 172.21.15.88:0/33236047 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f773819fea0 con 0x7f7738102e30 2024-02-19T19:26:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.034+0000 7f773cb9d700 1 -- 172.21.15.88:0/33236047 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77381074a0 con 0x7f7738102e30 2024-02-19T19:26:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.034+0000 7f77357fa700 1 -- 172.21.15.88:0/33236047 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7720003a60 con 0x7f7738102e30 2024-02-19T19:26:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.035+0000 7f77357fa700 1 --2- 172.21.15.88:0/33236047 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f772406d8a0 0x7f772406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.035+0000 7f77357fa700 1 -- 172.21.15.88:0/33236047 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f772008e9d0 con 0x7f7738102e30 2024-02-19T19:26:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.035+0000 7f77377fe700 1 --2- 172.21.15.88:0/33236047 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f772406d8a0 0x7f772406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:23.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.035+0000 7f77377fe700 1 --2- 172.21.15.88:0/33236047 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f772406d8a0 0x7f772406fd50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7728007be0 tx=0x7f77280073d0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:23.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.040+0000 7f77357fa700 1 -- 172.21.15.88:0/33236047 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7720058180 con 0x7f7738102e30 2024-02-19T19:26:23.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.254+0000 7f773cb9d700 1 -- 172.21.15.88:0/33236047 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7738061190 con 0x7f772406d8a0 2024-02-19T19:26:23.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.256+0000 7f77357fa700 1 -- 172.21.15.88:0/33236047 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7738061190 con 0x7f772406d8a0 2024-02-19T19:26:23.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.258+0000 7f773cb9d700 1 -- 172.21.15.88:0/33236047 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f772406d8a0 msgr2=0x7f772406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.258+0000 7f773cb9d700 1 --2- 172.21.15.88:0/33236047 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f772406d8a0 0x7f772406fd50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7728007be0 tx=0x7f77280073d0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.258+0000 7f773cb9d700 1 -- 172.21.15.88:0/33236047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7738102e30 msgr2=0x7f7738196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.258+0000 7f773cb9d700 1 --2- 172.21.15.88:0/33236047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7738102e30 0x7f7738196ed0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7720006010 tx=0x7f772000bd70 comp rx=0 tx=0).stop 2024-02-19T19:26:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.258+0000 7f773cb9d700 1 -- 172.21.15.88:0/33236047 shutdown_connections 2024-02-19T19:26:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.258+0000 7f773cb9d700 1 --2- 172.21.15.88:0/33236047 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f772406d8a0 0x7f772406fd50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.258+0000 7f773cb9d700 1 --2- 172.21.15.88:0/33236047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77380ff9e0 0x7f7738196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.258+0000 7f773cb9d700 1 --2- 172.21.15.88:0/33236047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7738102e30 0x7f7738196ed0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.258+0000 7f773cb9d700 1 -- 172.21.15.88:0/33236047 >> 172.21.15.88:0/33236047 conn(0x7f7738074fa0 msgr2=0x7f7738101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.258+0000 7f773cb9d700 1 -- 172.21.15.88:0/33236047 shutdown_connections 2024-02-19T19:26:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.258+0000 7f773cb9d700 1 -- 172.21.15.88:0/33236047 wait complete. 2024-02-19T19:26:23.274 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:26:23.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.363+0000 7ff72b946700 1 -- 172.21.15.88:0/578874303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240687a0 msgr2=0x7ff724068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.363+0000 7ff72b946700 1 --2- 172.21.15.88:0/578874303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240687a0 0x7ff724068b70 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7ff71400b0a0 tx=0x7ff71400b3b0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.363+0000 7ff72b946700 1 -- 172.21.15.88:0/578874303 shutdown_connections 2024-02-19T19:26:23.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.363+0000 7ff72b946700 1 --2- 172.21.15.88:0/578874303 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff724069140 0x7ff724104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.363+0000 7ff72b946700 1 --2- 172.21.15.88:0/578874303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240687a0 0x7ff724068b70 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.363+0000 7ff72b946700 1 -- 172.21.15.88:0/578874303 >> 172.21.15.88:0/578874303 conn(0x7ff724075130 msgr2=0x7ff724075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:23.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.363+0000 7ff72b946700 1 -- 172.21.15.88:0/578874303 shutdown_connections 2024-02-19T19:26:23.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.363+0000 7ff72b946700 1 -- 172.21.15.88:0/578874303 wait complete. 2024-02-19T19:26:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.363+0000 7ff72b946700 1 Processor -- start 2024-02-19T19:26:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff72b946700 1 -- start start 2024-02-19T19:26:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff72b946700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240687a0 0x7ff7240ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff72b946700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff724069140 0x7ff7241001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:23.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff72b946700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff724100720 con 0x7ff7240687a0 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff72b946700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff724100860 con 0x7ff724069140 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff7296e2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240687a0 0x7ff7240ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff7296e2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240687a0 0x7ff7240ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35100/0 (socket says 172.21.15.88:35100) 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff7296e2700 1 -- 172.21.15.88:0/2017567413 learned_addr learned my addr 172.21.15.88:0/2017567413 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff728ee1700 1 --2- 172.21.15.88:0/2017567413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff724069140 0x7ff7241001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff7296e2700 1 -- 172.21.15.88:0/2017567413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff724069140 msgr2=0x7ff7241001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff7296e2700 1 --2- 172.21.15.88:0/2017567413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff724069140 0x7ff7241001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff7296e2700 1 -- 172.21.15.88:0/2017567413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff714009d00 con 0x7ff7240687a0 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff7296e2700 1 --2- 172.21.15.88:0/2017567413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240687a0 0x7ff7240ffca0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff71400b070 tx=0x7ff71400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff7227fc700 1 -- 172.21.15.88:0/2017567413 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff71400e040 con 0x7ff7240687a0 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff7227fc700 1 -- 172.21.15.88:0/2017567413 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff714003de0 con 0x7ff7240687a0 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff72b946700 1 -- 172.21.15.88:0/2017567413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff724100ae0 con 0x7ff7240687a0 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.364+0000 7ff7227fc700 1 -- 172.21.15.88:0/2017567413 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff71401d8e0 con 0x7ff7240687a0 2024-02-19T19:26:23.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.365+0000 7ff72b946700 1 -- 172.21.15.88:0/2017567413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7241a2420 con 0x7ff7240687a0 2024-02-19T19:26:23.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.365+0000 7ff72b946700 1 -- 172.21.15.88:0/2017567413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff724108790 con 0x7ff7240687a0 2024-02-19T19:26:23.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.366+0000 7ff7227fc700 1 -- 172.21.15.88:0/2017567413 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff714004040 con 0x7ff7240687a0 2024-02-19T19:26:23.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.366+0000 7ff7227fc700 1 --2- 172.21.15.88:0/2017567413 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff70c06d8a0 0x7ff70c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:23.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.366+0000 7ff728ee1700 1 --2- 172.21.15.88:0/2017567413 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff70c06d8a0 0x7ff70c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:23.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.366+0000 7ff7227fc700 1 -- 172.21.15.88:0/2017567413 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff71408ed50 con 0x7ff7240687a0 2024-02-19T19:26:23.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.367+0000 7ff728ee1700 1 --2- 172.21.15.88:0/2017567413 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff70c06d8a0 0x7ff70c06fd50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff718005950 tx=0x7ff7180058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:23.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.371+0000 7ff7227fc700 1 -- 172.21.15.88:0/2017567413 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff714058500 con 0x7ff7240687a0 2024-02-19T19:26:23.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.589+0000 7ff72b946700 1 -- 172.21.15.88:0/2017567413 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff724061190 con 0x7ff70c06d8a0 2024-02-19T19:26:23.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.590+0000 7ff7227fc700 1 -- 172.21.15.88:0/2017567413 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff724061190 con 0x7ff70c06d8a0 2024-02-19T19:26:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.593+0000 7ff72b946700 1 -- 172.21.15.88:0/2017567413 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff70c06d8a0 msgr2=0x7ff70c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.593+0000 7ff72b946700 1 --2- 172.21.15.88:0/2017567413 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff70c06d8a0 0x7ff70c06fd50 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff718005950 tx=0x7ff7180058e0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.593+0000 7ff72b946700 1 -- 172.21.15.88:0/2017567413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240687a0 msgr2=0x7ff7240ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.593+0000 7ff72b946700 1 --2- 172.21.15.88:0/2017567413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240687a0 0x7ff7240ffca0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7ff71400b070 tx=0x7ff71400cdd0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.593+0000 7ff72b946700 1 -- 172.21.15.88:0/2017567413 shutdown_connections 2024-02-19T19:26:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.593+0000 7ff72b946700 1 --2- 172.21.15.88:0/2017567413 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff70c06d8a0 0x7ff70c06fd50 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.593+0000 7ff72b946700 1 --2- 172.21.15.88:0/2017567413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7240687a0 0x7ff7240ffca0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.593+0000 7ff72b946700 1 --2- 172.21.15.88:0/2017567413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff724069140 0x7ff7241001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.593+0000 7ff72b946700 1 -- 172.21.15.88:0/2017567413 >> 172.21.15.88:0/2017567413 conn(0x7ff724075130 msgr2=0x7ff7240fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.593+0000 7ff72b946700 1 -- 172.21.15.88:0/2017567413 shutdown_connections 2024-02-19T19:26:23.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.593+0000 7ff72b946700 1 -- 172.21.15.88:0/2017567413 wait complete. 2024-02-19T19:26:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.698+0000 7f100ca1f700 1 -- 172.21.15.88:0/2166403053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10081004e0 msgr2=0x7f10081008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.698+0000 7f100ca1f700 1 --2- 172.21.15.88:0/2166403053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10081004e0 0x7f10081008b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc009910 tx=0x7f0ffc009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.698+0000 7f100ca1f700 1 -- 172.21.15.88:0/2166403053 shutdown_connections 2024-02-19T19:26:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.698+0000 7f100ca1f700 1 --2- 172.21.15.88:0/2166403053 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1008100e80 0x7f1008104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.698+0000 7f100ca1f700 1 --2- 172.21.15.88:0/2166403053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10081004e0 0x7f10081008b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.698+0000 7f100ca1f700 1 -- 172.21.15.88:0/2166403053 >> 172.21.15.88:0/2166403053 conn(0x7f10080fbda0 msgr2=0x7f10080fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.698+0000 7f100ca1f700 1 -- 172.21.15.88:0/2166403053 shutdown_connections 2024-02-19T19:26:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.698+0000 7f100ca1f700 1 -- 172.21.15.88:0/2166403053 wait complete. 2024-02-19T19:26:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.699+0000 7f100ca1f700 1 Processor -- start 2024-02-19T19:26:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.699+0000 7f100ca1f700 1 -- start start 2024-02-19T19:26:23.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.699+0000 7f100ca1f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10081004e0 0x7f1008195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.699+0000 7f100ca1f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1008100e80 0x7f1008196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.699+0000 7f100ca1f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1008196b10 con 0x7f1008100e80 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.699+0000 7f100ca1f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100819a8a0 con 0x7f10081004e0 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.699+0000 7f1005d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1008100e80 0x7f1008196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.699+0000 7f1005d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1008100e80 0x7f1008196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35120/0 (socket says 172.21.15.88:35120) 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.699+0000 7f1005d9b700 1 -- 172.21.15.88:0/4163387879 learned_addr learned my addr 172.21.15.88:0/4163387879 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.699+0000 7f100659c700 1 --2- 172.21.15.88:0/4163387879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10081004e0 0x7f1008195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.700+0000 7f1005d9b700 1 -- 172.21.15.88:0/4163387879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10081004e0 msgr2=0x7f1008195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.700+0000 7f1005d9b700 1 --2- 172.21.15.88:0/4163387879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10081004e0 0x7f1008195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.700+0000 7f1005d9b700 1 -- 172.21.15.88:0/4163387879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc0095f0 con 0x7f1008100e80 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.700+0000 7f1005d9b700 1 --2- 172.21.15.88:0/4163387879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1008100e80 0x7f1008196430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800e8f0 tx=0x7f0ff800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:23.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.700+0000 7f0ff77fe700 1 -- 172.21.15.88:0/4163387879 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ff8009790 con 0x7f1008100e80 2024-02-19T19:26:23.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.700+0000 7f0ff77fe700 1 -- 172.21.15.88:0/4163387879 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ff800c810 con 0x7f1008100e80 2024-02-19T19:26:23.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.700+0000 7f100ca1f700 1 -- 172.21.15.88:0/4163387879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100819ab80 con 0x7f1008100e80 2024-02-19T19:26:23.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.700+0000 7f0ff77fe700 1 -- 172.21.15.88:0/4163387879 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ff8005580 con 0x7f1008100e80 2024-02-19T19:26:23.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.700+0000 7f100ca1f700 1 -- 172.21.15.88:0/4163387879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100819b0d0 con 0x7f1008100e80 2024-02-19T19:26:23.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.701+0000 7f100ca1f700 1 -- 172.21.15.88:0/4163387879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10081087f0 con 0x7f1008100e80 2024-02-19T19:26:23.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.702+0000 7f0ff77fe700 1 -- 172.21.15.88:0/4163387879 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0ff800c980 con 0x7f1008100e80 2024-02-19T19:26:23.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.702+0000 7f0ff77fe700 1 --2- 172.21.15.88:0/4163387879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0ff006d8a0 0x7f0ff006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:23.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.702+0000 7f0ff77fe700 1 -- 172.21.15.88:0/4163387879 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0ff8014070 con 0x7f1008100e80 2024-02-19T19:26:23.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.702+0000 7f100659c700 1 --2- 172.21.15.88:0/4163387879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0ff006d8a0 0x7f0ff006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:23.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.702+0000 7f100659c700 1 --2- 172.21.15.88:0/4163387879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0ff006d8a0 0x7f0ff006fd50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00b760 tx=0x7f0ffc005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:23.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.707+0000 7f0ff77fe700 1 -- 172.21.15.88:0/4163387879 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0ff8056f80 con 0x7f1008100e80 2024-02-19T19:26:23.898 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:23 smithi132 ceph-mon[70676]: pgmap v685: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:23.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.914+0000 7f100ca1f700 1 -- 172.21.15.88:0/4163387879 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f100819b350 con 0x7f0ff006d8a0 2024-02-19T19:26:23.920 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:23 smithi088 ceph-mon[88643]: pgmap v685: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:23.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.926+0000 7f0ff77fe700 1 -- 172.21.15.88:0/4163387879 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f100819b350 con 0x7f0ff006d8a0 2024-02-19T19:26:23.931 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (23m) 15m ago 33m 24.3M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (34m) 15m ago 34m 9168k - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (16m) 15m ago 32m 7910k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (19m) 15m ago 34m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (19m) 15m ago 32m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (22m) 15m ago 33m 86.5M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (15m) 15m ago 27m 13.5M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (15m) 15m ago 27m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (15m) 15m ago 27m 12.6M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (27m) 15m ago 27m 21.9M - 18.2.0 dc2bc1663786 2a54627323bf 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (25m) 15m ago 35m 509M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:26:23.932 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (24m) 15m ago 32m 452M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:26:23.933 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (21m) 15m ago 35m 57.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:26:23.933 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (20m) 15m ago 32m 46.3M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:26:23.933 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (24m) 15m ago 34m 15.7M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:26:23.933 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (24m) 15m ago 32m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:26:23.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (19m) 15m ago 31m 128M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:26:23.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (18m) 15m ago 31m 103M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:26:23.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (17m) 15m ago 30m 85.4M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:26:23.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (17m) 15m ago 30m 104M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:26:23.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (16m) 15m ago 29m 103M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:26:23.933 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (16m) 15m ago 29m 84.4M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:26:23.933 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (23m) 15m ago 32m 68.1M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:26:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.928+0000 7f100ca1f700 1 -- 172.21.15.88:0/4163387879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0ff006d8a0 msgr2=0x7f0ff006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.928+0000 7f100ca1f700 1 --2- 172.21.15.88:0/4163387879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0ff006d8a0 0x7f0ff006fd50 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00b760 tx=0x7f0ffc005a90 comp rx=0 tx=0).stop 2024-02-19T19:26:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.928+0000 7f100ca1f700 1 -- 172.21.15.88:0/4163387879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1008100e80 msgr2=0x7f1008196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.928+0000 7f100ca1f700 1 --2- 172.21.15.88:0/4163387879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1008100e80 0x7f1008196430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0ff800e8f0 tx=0x7f0ff800ec00 comp rx=0 tx=0).stop 2024-02-19T19:26:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.928+0000 7f100ca1f700 1 -- 172.21.15.88:0/4163387879 shutdown_connections 2024-02-19T19:26:23.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.928+0000 7f100ca1f700 1 --2- 172.21.15.88:0/4163387879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0ff006d8a0 0x7f0ff006fd50 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.928+0000 7f100ca1f700 1 --2- 172.21.15.88:0/4163387879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10081004e0 0x7f1008195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.928+0000 7f100ca1f700 1 --2- 172.21.15.88:0/4163387879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1008100e80 0x7f1008196430 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:23.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.928+0000 7f100ca1f700 1 -- 172.21.15.88:0/4163387879 >> 172.21.15.88:0/4163387879 conn(0x7f10080fbda0 msgr2=0x7f10080fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:23.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.928+0000 7f100ca1f700 1 -- 172.21.15.88:0/4163387879 shutdown_connections 2024-02-19T19:26:23.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:23.928+0000 7f100ca1f700 1 -- 172.21.15.88:0/4163387879 wait complete. 2024-02-19T19:26:24.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.030+0000 7fcc99cbc700 1 -- 172.21.15.88:0/452297982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc940ffc60 msgr2=0x7fcc9410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:24.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.030+0000 7fcc99cbc700 1 --2- 172.21.15.88:0/452297982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc940ffc60 0x7fcc9410a6d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c009910 tx=0x7fcc7c009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:24.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.030+0000 7fcc99cbc700 1 -- 172.21.15.88:0/452297982 shutdown_connections 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.030+0000 7fcc99cbc700 1 --2- 172.21.15.88:0/452297982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc940ffc60 0x7fcc9410a6d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.030+0000 7fcc99cbc700 1 --2- 172.21.15.88:0/452297982 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc940ff350 0x7fcc940ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.030+0000 7fcc99cbc700 1 -- 172.21.15.88:0/452297982 >> 172.21.15.88:0/452297982 conn(0x7fcc940748b0 msgr2=0x7fcc94074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.030+0000 7fcc99cbc700 1 -- 172.21.15.88:0/452297982 shutdown_connections 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.030+0000 7fcc99cbc700 1 -- 172.21.15.88:0/452297982 wait complete. 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.030+0000 7fcc99cbc700 1 Processor -- start 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc99cbc700 1 -- start start 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc99cbc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc940ff350 0x7fcc94195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc99cbc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc940ffc60 0x7fcc941963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc99cbc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc94196ad0 con 0x7fcc940ffc60 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc99cbc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9419a860 con 0x7fcc940ff350 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc8affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc940ffc60 0x7fcc941963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc8affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc940ffc60 0x7fcc941963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35144/0 (socket says 172.21.15.88:35144) 2024-02-19T19:26:24.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc8affd700 1 -- 172.21.15.88:0/3948790425 learned_addr learned my addr 172.21.15.88:0/3948790425 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc937fe700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc940ff350 0x7fcc94195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc8affd700 1 -- 172.21.15.88:0/3948790425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc940ff350 msgr2=0x7fcc94195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc8affd700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc940ff350 0x7fcc94195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc8affd700 1 -- 172.21.15.88:0/3948790425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc7c0095f0 con 0x7fcc940ffc60 2024-02-19T19:26:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc937fe700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc940ff350 0x7fcc94195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:26:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.031+0000 7fcc8affd700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc940ffc60 0x7fcc941963f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c006010 tx=0x7fcc7c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.032+0000 7fcc917fa700 1 -- 172.21.15.88:0/3948790425 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc7c01d070 con 0x7fcc940ffc60 2024-02-19T19:26:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.032+0000 7fcc917fa700 1 -- 172.21.15.88:0/3948790425 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc7c00f460 con 0x7fcc940ffc60 2024-02-19T19:26:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.032+0000 7fcc99cbc700 1 -- 172.21.15.88:0/3948790425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9419aae0 con 0x7fcc940ffc60 2024-02-19T19:26:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.032+0000 7fcc917fa700 1 -- 172.21.15.88:0/3948790425 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc7c017630 con 0x7fcc940ffc60 2024-02-19T19:26:24.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.032+0000 7fcc99cbc700 1 -- 172.21.15.88:0/3948790425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9419afd0 con 0x7fcc940ffc60 2024-02-19T19:26:24.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.033+0000 7fcc99cbc700 1 -- 172.21.15.88:0/3948790425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc94107dd0 con 0x7fcc940ffc60 2024-02-19T19:26:24.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.033+0000 7fcc917fa700 1 -- 172.21.15.88:0/3948790425 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcc7c00f5d0 con 0x7fcc940ffc60 2024-02-19T19:26:24.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.033+0000 7fcc917fa700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc8006d8a0 0x7fcc8006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:24.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.034+0000 7fcc917fa700 1 -- 172.21.15.88:0/3948790425 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcc7c08e9f0 con 0x7fcc940ffc60 2024-02-19T19:26:24.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.034+0000 7fcc937fe700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc8006d8a0 0x7fcc8006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:24.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.034+0000 7fcc937fe700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc8006d8a0 0x7fcc8006fd50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcc84005950 tx=0x7fcc840058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:24.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.038+0000 7fcc917fa700 1 -- 172.21.15.88:0/3948790425 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcc7c090e60 con 0x7fcc940ffc60 2024-02-19T19:26:24.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.325+0000 7fcc99cbc700 1 -- 172.21.15.88:0/3948790425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcc9402cc30 con 0x7fcc940ffc60 2024-02-19T19:26:24.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.326+0000 7fcc917fa700 1 -- 172.21.15.88:0/3948790425 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fcc7c0581a0 con 0x7fcc940ffc60 2024-02-19T19:26:24.332 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:26:24.332 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:26:24.332 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:26:24.332 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:26:24.332 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:26:24.332 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:26:24.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:26:24.333 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:26:24.333 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:26:24.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:26:24.333 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:26:24.333 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:26:24.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:26:24.333 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:26:24.333 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:26:24.333 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:26:24.333 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:26:24.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.328+0000 7fcc99cbc700 1 -- 172.21.15.88:0/3948790425 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc8006d8a0 msgr2=0x7fcc8006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:24.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.328+0000 7fcc99cbc700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc8006d8a0 0x7fcc8006fd50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcc84005950 tx=0x7fcc840058e0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.328+0000 7fcc99cbc700 1 -- 172.21.15.88:0/3948790425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc940ffc60 msgr2=0x7fcc941963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:24.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.328+0000 7fcc99cbc700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc940ffc60 0x7fcc941963f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c006010 tx=0x7fcc7c00be20 comp rx=0 tx=0).stop 2024-02-19T19:26:24.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.329+0000 7fcc99cbc700 1 -- 172.21.15.88:0/3948790425 shutdown_connections 2024-02-19T19:26:24.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.329+0000 7fcc99cbc700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcc8006d8a0 0x7fcc8006fd50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.329+0000 7fcc99cbc700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc940ff350 0x7fcc94195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.329+0000 7fcc99cbc700 1 --2- 172.21.15.88:0/3948790425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc940ffc60 0x7fcc941963f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.329+0000 7fcc99cbc700 1 -- 172.21.15.88:0/3948790425 >> 172.21.15.88:0/3948790425 conn(0x7fcc940748b0 msgr2=0x7fcc94104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:24.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.329+0000 7fcc99cbc700 1 -- 172.21.15.88:0/3948790425 shutdown_connections 2024-02-19T19:26:24.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.329+0000 7fcc99cbc700 1 -- 172.21.15.88:0/3948790425 wait complete. 2024-02-19T19:26:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.437+0000 7f1ebb81d700 1 -- 172.21.15.88:0/1881108389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb4068540 msgr2=0x7f1eb40689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.437+0000 7f1ebb81d700 1 --2- 172.21.15.88:0/1881108389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb4068540 0x7f1eb40689b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1eb0009910 tx=0x7f1eb0009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.437+0000 7f1ebb81d700 1 -- 172.21.15.88:0/1881108389 shutdown_connections 2024-02-19T19:26:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.437+0000 7f1ebb81d700 1 --2- 172.21.15.88:0/1881108389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb4068540 0x7f1eb40689b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.437+0000 7f1ebb81d700 1 --2- 172.21.15.88:0/1881108389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb4106320 0x7f1eb41066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.437+0000 7f1ebb81d700 1 -- 172.21.15.88:0/1881108389 >> 172.21.15.88:0/1881108389 conn(0x7f1eb4075130 msgr2=0x7f1eb4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.437+0000 7f1ebb81d700 1 -- 172.21.15.88:0/1881108389 shutdown_connections 2024-02-19T19:26:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.437+0000 7f1ebb81d700 1 -- 172.21.15.88:0/1881108389 wait complete. 2024-02-19T19:26:24.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1ebb81d700 1 Processor -- start 2024-02-19T19:26:24.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1ebb81d700 1 -- start start 2024-02-19T19:26:24.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1ebb81d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb4068540 0x7f1eb4195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:24.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1ebb81d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb4106320 0x7f1eb41963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:24.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1ebb81d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb4196a80 con 0x7f1eb4068540 2024-02-19T19:26:24.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1ebb81d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb419a810 con 0x7f1eb4106320 2024-02-19T19:26:24.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1eb95b9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb4068540 0x7f1eb4195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:24.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1eb95b9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb4068540 0x7f1eb4195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35160/0 (socket says 172.21.15.88:35160) 2024-02-19T19:26:24.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1eb95b9700 1 -- 172.21.15.88:0/2190530110 learned_addr learned my addr 172.21.15.88:0/2190530110 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:24.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1eb8db8700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb4106320 0x7f1eb41963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:24.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1eb95b9700 1 -- 172.21.15.88:0/2190530110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb4106320 msgr2=0x7f1eb41963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:24.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1eb95b9700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb4106320 0x7f1eb41963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1eb95b9700 1 -- 172.21.15.88:0/2190530110 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb00095f0 con 0x7f1eb4068540 2024-02-19T19:26:24.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.438+0000 7f1eb8db8700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb4106320 0x7f1eb41963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:26:24.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.439+0000 7f1eb95b9700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb4068540 0x7f1eb4195e60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400e920 tx=0x7f1ea400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:24.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.439+0000 7f1eaa7fc700 1 -- 172.21.15.88:0/2190530110 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ea400c9d0 con 0x7f1eb4068540 2024-02-19T19:26:24.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.439+0000 7f1eaa7fc700 1 -- 172.21.15.88:0/2190530110 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ea4010430 con 0x7f1eb4068540 2024-02-19T19:26:24.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.439+0000 7f1eaa7fc700 1 -- 172.21.15.88:0/2190530110 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ea4018630 con 0x7f1eb4068540 2024-02-19T19:26:24.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.439+0000 7f1ebb81d700 1 -- 172.21.15.88:0/2190530110 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb419aaf0 con 0x7f1eb4068540 2024-02-19T19:26:24.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.439+0000 7f1ebb81d700 1 -- 172.21.15.88:0/2190530110 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb419b040 con 0x7f1eb4068540 2024-02-19T19:26:24.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.440+0000 7f1ebb81d700 1 -- 172.21.15.88:0/2190530110 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb4108770 con 0x7f1eb4068540 2024-02-19T19:26:24.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.440+0000 7f1eaa7fc700 1 -- 172.21.15.88:0/2190530110 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1ea40105a0 con 0x7f1eb4068540 2024-02-19T19:26:24.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.441+0000 7f1eaa7fc700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ea006d8a0 0x7f1ea006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:24.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.441+0000 7f1eaa7fc700 1 -- 172.21.15.88:0/2190530110 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1ea4014070 con 0x7f1eb4068540 2024-02-19T19:26:24.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.441+0000 7f1eb8db8700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ea006d8a0 0x7f1ea006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:24.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.441+0000 7f1eb8db8700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ea006d8a0 0x7f1ea006fd50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4197480 tx=0x7f1eb0005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:24.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.446+0000 7f1eaa7fc700 1 -- 172.21.15.88:0/2190530110 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ea4052d20 con 0x7f1eb4068540 2024-02-19T19:26:24.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.707+0000 7f1ebb81d700 1 -- 172.21.15.88:0/2190530110 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1eb41972c0 con 0x7f1eb4068540 2024-02-19T19:26:24.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.708+0000 7f1eaa7fc700 1 -- 172.21.15.88:0/2190530110 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f1ea4056f50 con 0x7f1eb4068540 2024-02-19T19:26:24.714 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:26:24.714 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:26:24.714 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:26:24.714 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:26:24.714 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:26:24.715 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:26:24.716 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:26:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:26:24.717 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:26:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:26:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:26:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:26:24.717 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:26:24.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.710+0000 7f1ebb81d700 1 -- 172.21.15.88:0/2190530110 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ea006d8a0 msgr2=0x7f1ea006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:24.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.710+0000 7f1ebb81d700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ea006d8a0 0x7f1ea006fd50 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4197480 tx=0x7f1eb0005940 comp rx=0 tx=0).stop 2024-02-19T19:26:24.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.710+0000 7f1ebb81d700 1 -- 172.21.15.88:0/2190530110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb4068540 msgr2=0x7f1eb4195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:24.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.710+0000 7f1ebb81d700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb4068540 0x7f1eb4195e60 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400e920 tx=0x7f1ea400ec30 comp rx=0 tx=0).stop 2024-02-19T19:26:24.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.710+0000 7f1ebb81d700 1 -- 172.21.15.88:0/2190530110 shutdown_connections 2024-02-19T19:26:24.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.710+0000 7f1ebb81d700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ea006d8a0 0x7f1ea006fd50 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.710+0000 7f1ebb81d700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1eb4068540 0x7f1eb4195e60 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.710+0000 7f1ebb81d700 1 --2- 172.21.15.88:0/2190530110 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1eb4106320 0x7f1eb41963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.710+0000 7f1ebb81d700 1 -- 172.21.15.88:0/2190530110 >> 172.21.15.88:0/2190530110 conn(0x7f1eb4075130 msgr2=0x7f1eb40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:24.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.710+0000 7f1ebb81d700 1 -- 172.21.15.88:0/2190530110 shutdown_connections 2024-02-19T19:26:24.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.710+0000 7f1ebb81d700 1 -- 172.21.15.88:0/2190530110 wait complete. 2024-02-19T19:26:24.718 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:26:24.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.821+0000 7f2ed5663700 1 -- 172.21.15.88:0/716588039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ed00ff3c0 msgr2=0x7f2ed00ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:24.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.821+0000 7f2ed5663700 1 --2- 172.21.15.88:0/716588039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ed00ff3c0 0x7f2ed00ff790 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4009910 tx=0x7f2ec4009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.821+0000 7f2ed5663700 1 -- 172.21.15.88:0/716588039 shutdown_connections 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.821+0000 7f2ed5663700 1 --2- 172.21.15.88:0/716588039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ed00ffd60 0x7f2ed0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.821+0000 7f2ed5663700 1 --2- 172.21.15.88:0/716588039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ed00ff3c0 0x7f2ed00ff790 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.821+0000 7f2ed5663700 1 -- 172.21.15.88:0/716588039 >> 172.21.15.88:0/716588039 conn(0x7f2ed00fac80 msgr2=0x7f2ed00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.821+0000 7f2ed5663700 1 -- 172.21.15.88:0/716588039 shutdown_connections 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.821+0000 7f2ed5663700 1 -- 172.21.15.88:0/716588039 wait complete. 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.821+0000 7f2ed5663700 1 Processor -- start 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ed5663700 1 -- start start 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ed5663700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ed00ff3c0 0x7f2ed0196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ed5663700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ed00ffd60 0x7f2ed0197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ed5663700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed0197850 con 0x7f2ed00ffd60 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ed5663700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ed0197990 con 0x7f2ed00ff3c0 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ece7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ed00ffd60 0x7f2ed0197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ece7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ed00ffd60 0x7f2ed0197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35178/0 (socket says 172.21.15.88:35178) 2024-02-19T19:26:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ece7fc700 1 -- 172.21.15.88:0/3866468687 learned_addr learned my addr 172.21.15.88:0/3866468687 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2eceffd700 1 --2- 172.21.15.88:0/3866468687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ed00ff3c0 0x7f2ed0196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ece7fc700 1 -- 172.21.15.88:0/3866468687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ed00ff3c0 msgr2=0x7f2ed0196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ece7fc700 1 --2- 172.21.15.88:0/3866468687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ed00ff3c0 0x7f2ed0196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ece7fc700 1 -- 172.21.15.88:0/3866468687 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec40095f0 con 0x7f2ed00ffd60 2024-02-19T19:26:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.822+0000 7f2ece7fc700 1 --2- 172.21.15.88:0/3866468687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ed00ffd60 0x7f2ed0197230 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000d900 tx=0x7f2ec000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.823+0000 7f2eb7fff700 1 -- 172.21.15.88:0/3866468687 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ec0009900 con 0x7f2ed00ffd60 2024-02-19T19:26:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.823+0000 7f2eb7fff700 1 -- 172.21.15.88:0/3866468687 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ec0010460 con 0x7f2ed00ffd60 2024-02-19T19:26:24.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.823+0000 7f2ed5663700 1 -- 172.21.15.88:0/3866468687 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ed019fd60 con 0x7f2ed00ffd60 2024-02-19T19:26:24.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.823+0000 7f2eb7fff700 1 -- 172.21.15.88:0/3866468687 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ec000f5f0 con 0x7f2ed00ffd60 2024-02-19T19:26:24.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.823+0000 7f2ed5663700 1 -- 172.21.15.88:0/3866468687 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed01a0220 con 0x7f2ed00ffd60 2024-02-19T19:26:24.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.824+0000 7f2ed5663700 1 -- 172.21.15.88:0/3866468687 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ed01076d0 con 0x7f2ed00ffd60 2024-02-19T19:26:24.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.824+0000 7f2eb7fff700 1 -- 172.21.15.88:0/3866468687 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2ec00105d0 con 0x7f2ed00ffd60 2024-02-19T19:26:24.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.825+0000 7f2eb7fff700 1 --2- 172.21.15.88:0/3866468687 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2eb806d8a0 0x7f2eb806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:24.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.825+0000 7f2eb7fff700 1 -- 172.21.15.88:0/3866468687 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2ec008d0b0 con 0x7f2ed00ffd60 2024-02-19T19:26:24.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.825+0000 7f2eceffd700 1 --2- 172.21.15.88:0/3866468687 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2eb806d8a0 0x7f2eb806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:24.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.825+0000 7f2eceffd700 1 --2- 172.21.15.88:0/3866468687 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2eb806d8a0 0x7f2eb806fd50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400b5c0 tx=0x7f2ec401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:24.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:24.834+0000 7f2eb7fff700 1 -- 172.21.15.88:0/3866468687 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2ec0056860 con 0x7f2ed00ffd60 2024-02-19T19:26:24.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:24 smithi132 ceph-mon[70676]: from='client.35254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:24.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:24 smithi132 ceph-mon[70676]: from='client.35258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:24.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:24.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:24.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:26:24.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:24 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3948790425' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:25.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.060+0000 7f2ed5663700 1 -- 172.21.15.88:0/3866468687 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ed0061190 con 0x7f2eb806d8a0 2024-02-19T19:26:25.066 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:24 smithi088 ceph-mon[88643]: from='client.35254 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:25.066 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:24 smithi088 ceph-mon[88643]: from='client.35258 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:25.066 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:25.066 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:25.067 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:26:25.067 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:24 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3948790425' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:25.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.062+0000 7f2eb7fff700 1 -- 172.21.15.88:0/3866468687 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2ed0061190 con 0x7f2eb806d8a0 2024-02-19T19:26:25.067 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:26:25.067 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:26:25.067 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:26:25.068 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:26:25.069 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:26:25.069 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr", 2024-02-19T19:26:25.069 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:26:25.069 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:26:25.069 INFO:teuthology.orchestra.run.smithi088.stdout: "mon" 2024-02-19T19:26:25.069 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:26:25.069 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "15/23 daemons upgraded", 2024-02-19T19:26:25.069 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:26:25.069 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:26:25.069 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:26:25.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.064+0000 7f2ed5663700 1 -- 172.21.15.88:0/3866468687 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2eb806d8a0 msgr2=0x7f2eb806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:25.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.064+0000 7f2ed5663700 1 --2- 172.21.15.88:0/3866468687 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2eb806d8a0 0x7f2eb806fd50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2ec400b5c0 tx=0x7f2ec401a040 comp rx=0 tx=0).stop 2024-02-19T19:26:25.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.064+0000 7f2ed5663700 1 -- 172.21.15.88:0/3866468687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ed00ffd60 msgr2=0x7f2ed0197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:25.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.064+0000 7f2ed5663700 1 --2- 172.21.15.88:0/3866468687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ed00ffd60 0x7f2ed0197230 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2ec000d900 tx=0x7f2ec000dc10 comp rx=0 tx=0).stop 2024-02-19T19:26:25.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.064+0000 7f2ed5663700 1 -- 172.21.15.88:0/3866468687 shutdown_connections 2024-02-19T19:26:25.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.064+0000 7f2ed5663700 1 --2- 172.21.15.88:0/3866468687 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2eb806d8a0 0x7f2eb806fd50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:25.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.064+0000 7f2ed5663700 1 --2- 172.21.15.88:0/3866468687 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2ed00ff3c0 0x7f2ed0196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:25.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.064+0000 7f2ed5663700 1 --2- 172.21.15.88:0/3866468687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ed00ffd60 0x7f2ed0197230 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:25.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.064+0000 7f2ed5663700 1 -- 172.21.15.88:0/3866468687 >> 172.21.15.88:0/3866468687 conn(0x7f2ed00fac80 msgr2=0x7f2ed00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:25.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.064+0000 7f2ed5663700 1 -- 172.21.15.88:0/3866468687 shutdown_connections 2024-02-19T19:26:25.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.064+0000 7f2ed5663700 1 -- 172.21.15.88:0/3866468687 wait complete. 2024-02-19T19:26:25.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.175+0000 7fbb99458700 1 -- 172.21.15.88:0/2054595853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb940ff3c0 msgr2=0x7fbb940ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:25.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.175+0000 7fbb99458700 1 --2- 172.21.15.88:0/2054595853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb940ff3c0 0x7fbb940ff790 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fbb84009910 tx=0x7fbb84009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:25.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.176+0000 7fbb99458700 1 -- 172.21.15.88:0/2054595853 shutdown_connections 2024-02-19T19:26:25.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.176+0000 7fbb99458700 1 --2- 172.21.15.88:0/2054595853 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb940ffd60 0x7fbb94103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:25.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.176+0000 7fbb99458700 1 --2- 172.21.15.88:0/2054595853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb940ff3c0 0x7fbb940ff790 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:25.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.176+0000 7fbb99458700 1 -- 172.21.15.88:0/2054595853 >> 172.21.15.88:0/2054595853 conn(0x7fbb940fac80 msgr2=0x7fbb940fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:25.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.176+0000 7fbb99458700 1 -- 172.21.15.88:0/2054595853 shutdown_connections 2024-02-19T19:26:25.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.176+0000 7fbb99458700 1 -- 172.21.15.88:0/2054595853 wait complete. 2024-02-19T19:26:25.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.176+0000 7fbb99458700 1 Processor -- start 2024-02-19T19:26:25.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb99458700 1 -- start start 2024-02-19T19:26:25.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb99458700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb940ff3c0 0x7fbb94196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb99458700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb940ffd60 0x7fbb94197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb99458700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb94197830 con 0x7fbb940ffd60 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb99458700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb941979a0 con 0x7fbb940ff3c0 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb927fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb940ffd60 0x7fbb94197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb927fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb940ffd60 0x7fbb94197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35194/0 (socket says 172.21.15.88:35194) 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb927fc700 1 -- 172.21.15.88:0/2046578101 learned_addr learned my addr 172.21.15.88:0/2046578101 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb92ffd700 1 --2- 172.21.15.88:0/2046578101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb940ff3c0 0x7fbb94196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb927fc700 1 -- 172.21.15.88:0/2046578101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb940ff3c0 msgr2=0x7fbb94196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb927fc700 1 --2- 172.21.15.88:0/2046578101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb940ff3c0 0x7fbb94196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.177+0000 7fbb927fc700 1 -- 172.21.15.88:0/2046578101 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840095f0 con 0x7fbb940ffd60 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.178+0000 7fbb927fc700 1 --2- 172.21.15.88:0/2046578101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb940ffd60 0x7fbb94197210 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800e8f0 tx=0x7fbb8800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.178+0000 7fbb7bfff700 1 -- 172.21.15.88:0/2046578101 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb88009790 con 0x7fbb940ffd60 2024-02-19T19:26:25.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.178+0000 7fbb7bfff700 1 -- 172.21.15.88:0/2046578101 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb8800c810 con 0x7fbb940ffd60 2024-02-19T19:26:25.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.178+0000 7fbb7bfff700 1 -- 172.21.15.88:0/2046578101 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb88005580 con 0x7fbb940ffd60 2024-02-19T19:26:25.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.178+0000 7fbb99458700 1 -- 172.21.15.88:0/2046578101 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb9419fd40 con 0x7fbb940ffd60 2024-02-19T19:26:25.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.178+0000 7fbb99458700 1 -- 172.21.15.88:0/2046578101 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb941a0230 con 0x7fbb940ffd60 2024-02-19T19:26:25.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.179+0000 7fbb99458700 1 -- 172.21.15.88:0/2046578101 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb941076d0 con 0x7fbb940ffd60 2024-02-19T19:26:25.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.180+0000 7fbb7bfff700 1 -- 172.21.15.88:0/2046578101 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbb8800c980 con 0x7fbb940ffd60 2024-02-19T19:26:25.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.180+0000 7fbb7bfff700 1 --2- 172.21.15.88:0/2046578101 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb7c06d8a0 0x7fbb7c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:25.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.181+0000 7fbb7bfff700 1 -- 172.21.15.88:0/2046578101 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbb88014070 con 0x7fbb940ffd60 2024-02-19T19:26:25.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.181+0000 7fbb92ffd700 1 --2- 172.21.15.88:0/2046578101 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb7c06d8a0 0x7fbb7c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:25.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.181+0000 7fbb92ffd700 1 --2- 172.21.15.88:0/2046578101 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb7c06d8a0 0x7fbb7c06fd50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb840059d0 tx=0x7fbb84005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:25.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.186+0000 7fbb7bfff700 1 -- 172.21.15.88:0/2046578101 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbb88056f80 con 0x7fbb940ffd60 2024-02-19T19:26:25.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.498+0000 7fbb99458700 1 -- 172.21.15.88:0/2046578101 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fbb9404f2a0 con 0x7fbb940ffd60 2024-02-19T19:26:25.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.499+0000 7fbb7bfff700 1 -- 172.21.15.88:0/2046578101 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fbb8805a640 con 0x7fbb940ffd60 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:26:25.505 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:26:25.506 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:26:25.506 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:26:25.506 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:26:25.506 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:26:25.506 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:26:25.506 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:26:25.506 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:26:25.506 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:26:25.506 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:26:25.506 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:26:25.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.501+0000 7fbb99458700 1 -- 172.21.15.88:0/2046578101 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb7c06d8a0 msgr2=0x7fbb7c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:25.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.501+0000 7fbb99458700 1 --2- 172.21.15.88:0/2046578101 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb7c06d8a0 0x7fbb7c06fd50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb840059d0 tx=0x7fbb84005940 comp rx=0 tx=0).stop 2024-02-19T19:26:25.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.501+0000 7fbb99458700 1 -- 172.21.15.88:0/2046578101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb940ffd60 msgr2=0x7fbb94197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:25.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.501+0000 7fbb99458700 1 --2- 172.21.15.88:0/2046578101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb940ffd60 0x7fbb94197210 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800e8f0 tx=0x7fbb8800ec00 comp rx=0 tx=0).stop 2024-02-19T19:26:25.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.501+0000 7fbb99458700 1 -- 172.21.15.88:0/2046578101 shutdown_connections 2024-02-19T19:26:25.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.501+0000 7fbb99458700 1 --2- 172.21.15.88:0/2046578101 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbb7c06d8a0 0x7fbb7c06fd50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:25.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.501+0000 7fbb99458700 1 --2- 172.21.15.88:0/2046578101 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbb940ff3c0 0x7fbb94196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:25.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.501+0000 7fbb99458700 1 --2- 172.21.15.88:0/2046578101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb940ffd60 0x7fbb94197210 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:25.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.501+0000 7fbb99458700 1 -- 172.21.15.88:0/2046578101 >> 172.21.15.88:0/2046578101 conn(0x7fbb940fac80 msgr2=0x7fbb940fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:25.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.501+0000 7fbb99458700 1 -- 172.21.15.88:0/2046578101 shutdown_connections 2024-02-19T19:26:25.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:25.501+0000 7fbb99458700 1 -- 172.21.15.88:0/2046578101 wait complete. 2024-02-19T19:26:25.801 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:25 smithi132 ceph-mon[70676]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:25.801 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:25 smithi132 ceph-mon[70676]: pgmap v686: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:25.801 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:25 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] up:boot 2024-02-19T19:26:25.801 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:25 smithi132 ceph-mon[70676]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:26:25.801 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:26:25.801 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:25 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2190530110' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:26:25.801 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:25.801 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:25.801 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:25.801 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:25.802 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:25 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2046578101' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:26:25.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:25 smithi088 ceph-mon[88643]: from='client.35262 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:25.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:25 smithi088 ceph-mon[88643]: pgmap v686: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:25.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:25 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] up:boot 2024-02-19T19:26:25.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:25 smithi088 ceph-mon[88643]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:26:25.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:26:25.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:25 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2190530110' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:26:25.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:25.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:25.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:25.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:25.837 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:25 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2046578101' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:26:26.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:26 smithi132 ceph-mon[70676]: from='client.35278 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:26.911 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:26 smithi088 ceph-mon[88643]: from='client.35278 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:27.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:27 smithi088 ceph-mon[88643]: pgmap v687: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:27.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:27 smithi132 ceph-mon[70676]: pgmap v687: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:29.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:29.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:29.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:29.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:26:29.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:28 smithi088 ceph-mon[88643]: pgmap v688: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:29.238 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:29.238 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:29.238 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:29.238 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:26:29.238 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:28 smithi132 ceph-mon[70676]: pgmap v688: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:30.238 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:30.238 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:30.238 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:30.238 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:30.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:30.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:30.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:30.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:26:30.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:30 smithi132 ceph-mon[70676]: pgmap v689: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:31.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:30 smithi088 ceph-mon[88643]: pgmap v689: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:33.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:33 smithi132 ceph-mon[70676]: pgmap v690: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:34.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:33 smithi088 ceph-mon[88643]: pgmap v690: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:35.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:35 smithi132 ceph-mon[70676]: pgmap v691: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:36.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:35 smithi088 ceph-mon[88643]: pgmap v691: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:37.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:37 smithi132 ceph-mon[70676]: pgmap v692: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:38.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:37 smithi088 ceph-mon[88643]: pgmap v692: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:39.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:39 smithi132 ceph-mon[70676]: pgmap v693: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:40.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:39 smithi088 ceph-mon[88643]: pgmap v693: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:41.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:41 smithi132 ceph-mon[70676]: pgmap v694: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:42.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:41 smithi088 ceph-mon[88643]: pgmap v694: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:43.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:43 smithi132 ceph-mon[70676]: pgmap v695: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:43.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:26:44.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:43 smithi088 ceph-mon[88643]: pgmap v695: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:44.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:26:45.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:45 smithi132 ceph-mon[70676]: pgmap v696: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:46.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:45 smithi088 ceph-mon[88643]: pgmap v696: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:47.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:47 smithi132 ceph-mon[70676]: pgmap v697: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:48.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:47 smithi088 ceph-mon[88643]: pgmap v697: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:49.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:49 smithi132 ceph-mon[70676]: pgmap v698: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:50.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:49 smithi088 ceph-mon[88643]: pgmap v698: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:51.790 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:51 smithi132 ceph-mon[70676]: pgmap v699: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:52.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:51 smithi088 ceph-mon[88643]: pgmap v699: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:53.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:53 smithi132 ceph-mon[70676]: pgmap v700: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:54.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:53 smithi088 ceph-mon[88643]: pgmap v700: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:55.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.616+0000 7f40065b5700 1 -- 172.21.15.88:0/2176143018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40000ffd60 msgr2=0x7f4000103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.616+0000 7f40065b5700 1 --2- 172.21.15.88:0/2176143018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40000ffd60 0x7f4000103d30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009910 tx=0x7f3ff4009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.616+0000 7f40065b5700 1 -- 172.21.15.88:0/2176143018 shutdown_connections 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.616+0000 7f40065b5700 1 --2- 172.21.15.88:0/2176143018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40000ffd60 0x7f4000103d30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.616+0000 7f40065b5700 1 --2- 172.21.15.88:0/2176143018 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40000ff3c0 0x7f40000ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.616+0000 7f40065b5700 1 -- 172.21.15.88:0/2176143018 >> 172.21.15.88:0/2176143018 conn(0x7f40000fac80 msgr2=0x7f40000fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.616+0000 7f40065b5700 1 -- 172.21.15.88:0/2176143018 shutdown_connections 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.616+0000 7f40065b5700 1 -- 172.21.15.88:0/2176143018 wait complete. 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.616+0000 7f40065b5700 1 Processor -- start 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f40065b5700 1 -- start start 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f40065b5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40000ff3c0 0x7f4000198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f40065b5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40000ffd60 0x7f40001985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f40065b5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4000198c80 con 0x7f40000ffd60 2024-02-19T19:26:55.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f40065b5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400019ca10 con 0x7f40000ff3c0 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f3fff7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40000ffd60 0x7f40001985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f3ffffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40000ff3c0 0x7f4000198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f3fff7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40000ffd60 0x7f40001985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51322/0 (socket says 172.21.15.88:51322) 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f3fff7fe700 1 -- 172.21.15.88:0/3751978954 learned_addr learned my addr 172.21.15.88:0/3751978954 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f3fff7fe700 1 -- 172.21.15.88:0/3751978954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40000ff3c0 msgr2=0x7f4000198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f3fff7fe700 1 --2- 172.21.15.88:0/3751978954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40000ff3c0 0x7f4000198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.617+0000 7f3fff7fe700 1 -- 172.21.15.88:0/3751978954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff40095f0 con 0x7f40000ffd60 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.618+0000 7f3fff7fe700 1 --2- 172.21.15.88:0/3751978954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40000ffd60 0x7f40001985a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40092e0 tx=0x7f3ff400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.618+0000 7f3ffd7fa700 1 -- 172.21.15.88:0/3751978954 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ff401d070 con 0x7f40000ffd60 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.618+0000 7f40065b5700 1 -- 172.21.15.88:0/3751978954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f400019cc90 con 0x7f40000ffd60 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.618+0000 7f3ffd7fa700 1 -- 172.21.15.88:0/3751978954 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ff4005dc0 con 0x7f40000ffd60 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.618+0000 7f3ffd7fa700 1 -- 172.21.15.88:0/3751978954 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ff4017400 con 0x7f40000ffd60 2024-02-19T19:26:55.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.618+0000 7f40065b5700 1 -- 172.21.15.88:0/3751978954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f400019d180 con 0x7f40000ffd60 2024-02-19T19:26:55.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.619+0000 7f40065b5700 1 -- 172.21.15.88:0/3751978954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f400004f2a0 con 0x7f40000ffd60 2024-02-19T19:26:55.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.619+0000 7f3ffd7fa700 1 -- 172.21.15.88:0/3751978954 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3ff400f6f0 con 0x7f40000ffd60 2024-02-19T19:26:55.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.620+0000 7f3ffd7fa700 1 --2- 172.21.15.88:0/3751978954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3fe806d8a0 0x7f3fe806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:55.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.620+0000 7f3ffd7fa700 1 -- 172.21.15.88:0/3751978954 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3ff408e910 con 0x7f40000ffd60 2024-02-19T19:26:55.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.620+0000 7f3ffffff700 1 --2- 172.21.15.88:0/3751978954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3fe806d8a0 0x7f3fe806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:55.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.620+0000 7f3ffffff700 1 --2- 172.21.15.88:0/3751978954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3fe806d8a0 0x7f3fe806fd50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0005950 tx=0x7f3ff000b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:55.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.625+0000 7f3ffd7fa700 1 -- 172.21.15.88:0/3751978954 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3ff40580c0 con 0x7f40000ffd60 2024-02-19T19:26:55.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.840+0000 7f40065b5700 1 -- 172.21.15.88:0/3751978954 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f400019d4b0 con 0x7f3fe806d8a0 2024-02-19T19:26:55.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.841+0000 7f3ffd7fa700 1 -- 172.21.15.88:0/3751978954 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f400019d4b0 con 0x7f3fe806d8a0 2024-02-19T19:26:55.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.843+0000 7f40065b5700 1 -- 172.21.15.88:0/3751978954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3fe806d8a0 msgr2=0x7f3fe806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:55.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.843+0000 7f40065b5700 1 --2- 172.21.15.88:0/3751978954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3fe806d8a0 0x7f3fe806fd50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0005950 tx=0x7f3ff000b410 comp rx=0 tx=0).stop 2024-02-19T19:26:55.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.843+0000 7f40065b5700 1 -- 172.21.15.88:0/3751978954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40000ffd60 msgr2=0x7f40001985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:55.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.843+0000 7f40065b5700 1 --2- 172.21.15.88:0/3751978954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40000ffd60 0x7f40001985a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40092e0 tx=0x7f3ff400bd70 comp rx=0 tx=0).stop 2024-02-19T19:26:55.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.843+0000 7f40065b5700 1 -- 172.21.15.88:0/3751978954 shutdown_connections 2024-02-19T19:26:55.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.843+0000 7f40065b5700 1 --2- 172.21.15.88:0/3751978954 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3fe806d8a0 0x7f3fe806fd50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:55.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.843+0000 7f40065b5700 1 --2- 172.21.15.88:0/3751978954 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40000ff3c0 0x7f4000198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:55.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.843+0000 7f40065b5700 1 --2- 172.21.15.88:0/3751978954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40000ffd60 0x7f40001985a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:55.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.843+0000 7f40065b5700 1 -- 172.21.15.88:0/3751978954 >> 172.21.15.88:0/3751978954 conn(0x7f40000fac80 msgr2=0x7f40000fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:55.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.843+0000 7f40065b5700 1 -- 172.21.15.88:0/3751978954 shutdown_connections 2024-02-19T19:26:55.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.843+0000 7f40065b5700 1 -- 172.21.15.88:0/3751978954 wait complete. 2024-02-19T19:26:55.859 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:26:55.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.949+0000 7f7ebe047700 1 -- 172.21.15.88:0/3738986665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7eb8068540 msgr2=0x7f7eb80689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:55.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.949+0000 7f7ebe047700 1 --2- 172.21.15.88:0/3738986665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7eb8068540 0x7f7eb80689b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009910 tx=0x7f7ea0009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:55.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.949+0000 7f7ebe047700 1 -- 172.21.15.88:0/3738986665 shutdown_connections 2024-02-19T19:26:55.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.949+0000 7f7ebe047700 1 --2- 172.21.15.88:0/3738986665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7eb8068540 0x7f7eb80689b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:55.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.949+0000 7f7ebe047700 1 --2- 172.21.15.88:0/3738986665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7eb8106320 0x7f7eb81066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:55.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.949+0000 7f7ebe047700 1 -- 172.21.15.88:0/3738986665 >> 172.21.15.88:0/3738986665 conn(0x7f7eb8075130 msgr2=0x7f7eb8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:55.956 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:55 smithi088 ceph-mon[88643]: pgmap v701: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:55.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.949+0000 7f7ebe047700 1 -- 172.21.15.88:0/3738986665 shutdown_connections 2024-02-19T19:26:55.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.949+0000 7f7ebe047700 1 -- 172.21.15.88:0/3738986665 wait complete. 2024-02-19T19:26:55.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.950+0000 7f7ebe047700 1 Processor -- start 2024-02-19T19:26:55.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.950+0000 7f7ebe047700 1 -- start start 2024-02-19T19:26:55.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.950+0000 7f7ebe047700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7eb8068540 0x7f7eb8195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:55.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.950+0000 7f7ebe047700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7eb8106320 0x7f7eb81963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:55.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.950+0000 7f7ebe047700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb8196a80 con 0x7f7eb8106320 2024-02-19T19:26:55.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.950+0000 7f7ebe047700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb819a810 con 0x7f7eb8068540 2024-02-19T19:26:55.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.950+0000 7f7eaeffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7eb8106320 0x7f7eb81963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:55.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.950+0000 7f7eaeffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7eb8106320 0x7f7eb81963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51352/0 (socket says 172.21.15.88:51352) 2024-02-19T19:26:55.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.950+0000 7f7eaeffd700 1 -- 172.21.15.88:0/1081513925 learned_addr learned my addr 172.21.15.88:0/1081513925 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:55.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.951+0000 7f7eb77fe700 1 --2- 172.21.15.88:0/1081513925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7eb8068540 0x7f7eb8195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.951+0000 7f7eaeffd700 1 -- 172.21.15.88:0/1081513925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7eb8068540 msgr2=0x7f7eb8195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.951+0000 7f7eaeffd700 1 --2- 172.21.15.88:0/1081513925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7eb8068540 0x7f7eb8195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.951+0000 7f7eaeffd700 1 -- 172.21.15.88:0/1081513925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea00095f0 con 0x7f7eb8106320 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.951+0000 7f7eaeffd700 1 --2- 172.21.15.88:0/1081513925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7eb8106320 0x7f7eb81963a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00092e0 tx=0x7f7ea000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.951+0000 7f7eb57fa700 1 -- 172.21.15.88:0/1081513925 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ea001d070 con 0x7f7eb8106320 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.951+0000 7f7eb57fa700 1 -- 172.21.15.88:0/1081513925 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ea0005e00 con 0x7f7eb8106320 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.951+0000 7f7ebe047700 1 -- 172.21.15.88:0/1081513925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb819aa90 con 0x7f7eb8106320 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.951+0000 7f7eb57fa700 1 -- 172.21.15.88:0/1081513925 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ea0017400 con 0x7f7eb8106320 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.951+0000 7f7ebe047700 1 -- 172.21.15.88:0/1081513925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb819af80 con 0x7f7eb8106320 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.952+0000 7f7ebe047700 1 -- 172.21.15.88:0/1081513925 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb8108770 con 0x7f7eb8106320 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.953+0000 7f7eb57fa700 1 -- 172.21.15.88:0/1081513925 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7ea0003a60 con 0x7f7eb8106320 2024-02-19T19:26:55.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.953+0000 7f7eb57fa700 1 --2- 172.21.15.88:0/1081513925 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7ea406d8a0 0x7f7ea406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:55.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.953+0000 7f7eb57fa700 1 -- 172.21.15.88:0/1081513925 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7ea008e9d0 con 0x7f7eb8106320 2024-02-19T19:26:55.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.953+0000 7f7eb77fe700 1 --2- 172.21.15.88:0/1081513925 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7ea406d8a0 0x7f7ea406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:55.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.953+0000 7f7eb77fe700 1 --2- 172.21.15.88:0/1081513925 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7ea406d8a0 0x7f7ea406fd50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009d10 tx=0x7f7ea8009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:55.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:55.958+0000 7f7eb57fa700 1 -- 172.21.15.88:0/1081513925 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7ea0058200 con 0x7f7eb8106320 2024-02-19T19:26:55.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:55 smithi132 ceph-mon[70676]: pgmap v701: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:56.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.173+0000 7f7ebe047700 1 -- 172.21.15.88:0/1081513925 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7eb819b1f0 con 0x7f7ea406d8a0 2024-02-19T19:26:56.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.175+0000 7f7eb57fa700 1 -- 172.21.15.88:0/1081513925 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7eb819b1f0 con 0x7f7ea406d8a0 2024-02-19T19:26:56.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.177+0000 7f7ebe047700 1 -- 172.21.15.88:0/1081513925 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7ea406d8a0 msgr2=0x7f7ea406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:56.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.177+0000 7f7ebe047700 1 --2- 172.21.15.88:0/1081513925 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7ea406d8a0 0x7f7ea406fd50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009d10 tx=0x7f7ea8009040 comp rx=0 tx=0).stop 2024-02-19T19:26:56.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.177+0000 7f7ebe047700 1 -- 172.21.15.88:0/1081513925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7eb8106320 msgr2=0x7f7eb81963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:56.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.177+0000 7f7ebe047700 1 --2- 172.21.15.88:0/1081513925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7eb8106320 0x7f7eb81963a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00092e0 tx=0x7f7ea000be20 comp rx=0 tx=0).stop 2024-02-19T19:26:56.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.177+0000 7f7ebe047700 1 -- 172.21.15.88:0/1081513925 shutdown_connections 2024-02-19T19:26:56.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.177+0000 7f7ebe047700 1 --2- 172.21.15.88:0/1081513925 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7ea406d8a0 0x7f7ea406fd50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.177+0000 7f7ebe047700 1 --2- 172.21.15.88:0/1081513925 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7eb8068540 0x7f7eb8195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.177+0000 7f7ebe047700 1 --2- 172.21.15.88:0/1081513925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7eb8106320 0x7f7eb81963a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.177+0000 7f7ebe047700 1 -- 172.21.15.88:0/1081513925 >> 172.21.15.88:0/1081513925 conn(0x7f7eb8075130 msgr2=0x7f7eb80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:56.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.177+0000 7f7ebe047700 1 -- 172.21.15.88:0/1081513925 shutdown_connections 2024-02-19T19:26:56.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.177+0000 7f7ebe047700 1 -- 172.21.15.88:0/1081513925 wait complete. 2024-02-19T19:26:56.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.282+0000 7f85102ee700 1 -- 172.21.15.88:0/1873401520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85080ff350 msgr2=0x7f85080ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:56.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.282+0000 7f85102ee700 1 --2- 172.21.15.88:0/1873401520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85080ff350 0x7f85080ff720 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8504009910 tx=0x7f8504009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:56.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.282+0000 7f85102ee700 1 -- 172.21.15.88:0/1873401520 shutdown_connections 2024-02-19T19:26:56.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.282+0000 7f85102ee700 1 --2- 172.21.15.88:0/1873401520 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85080ffc60 0x7f850810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.282+0000 7f85102ee700 1 --2- 172.21.15.88:0/1873401520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85080ff350 0x7f85080ff720 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.282+0000 7f85102ee700 1 -- 172.21.15.88:0/1873401520 >> 172.21.15.88:0/1873401520 conn(0x7f85080748b0 msgr2=0x7f8508074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:56.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.282+0000 7f85102ee700 1 -- 172.21.15.88:0/1873401520 shutdown_connections 2024-02-19T19:26:56.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.282+0000 7f85102ee700 1 -- 172.21.15.88:0/1873401520 wait complete. 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f85102ee700 1 Processor -- start 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f85102ee700 1 -- start start 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f85102ee700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85080ff350 0x7f8508100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f85102ee700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85080ffc60 0x7f8508100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f85102ee700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8508100fb0 con 0x7f85080ffc60 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f85102ee700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85081010f0 con 0x7f85080ff350 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f850d889700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85080ffc60 0x7f8508100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f850d889700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85080ffc60 0x7f8508100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51376/0 (socket says 172.21.15.88:51376) 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f850d889700 1 -- 172.21.15.88:0/3883952756 learned_addr learned my addr 172.21.15.88:0/3883952756 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f850e08a700 1 --2- 172.21.15.88:0/3883952756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85080ff350 0x7f8508100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f850d889700 1 -- 172.21.15.88:0/3883952756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85080ff350 msgr2=0x7f8508100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f850d889700 1 --2- 172.21.15.88:0/3883952756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85080ff350 0x7f8508100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.283+0000 7f850d889700 1 -- 172.21.15.88:0/3883952756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85040095f0 con 0x7f85080ffc60 2024-02-19T19:26:56.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.284+0000 7f850d889700 1 --2- 172.21.15.88:0/3883952756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85080ffc60 0x7f8508100a70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f84f800e920 tx=0x7f84f800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:56.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.284+0000 7f84ff7fe700 1 -- 172.21.15.88:0/3883952756 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84f8009790 con 0x7f85080ffc60 2024-02-19T19:26:56.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.284+0000 7f84ff7fe700 1 -- 172.21.15.88:0/3883952756 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84f800cba0 con 0x7f85080ffc60 2024-02-19T19:26:56.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.284+0000 7f84ff7fe700 1 -- 172.21.15.88:0/3883952756 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84f8018650 con 0x7f85080ffc60 2024-02-19T19:26:56.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.284+0000 7f85102ee700 1 -- 172.21.15.88:0/3883952756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85081a20b0 con 0x7f85080ffc60 2024-02-19T19:26:56.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.284+0000 7f85102ee700 1 -- 172.21.15.88:0/3883952756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85081a2470 con 0x7f85080ffc60 2024-02-19T19:26:56.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.285+0000 7f85102ee700 1 -- 172.21.15.88:0/3883952756 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8508107dd0 con 0x7f85080ffc60 2024-02-19T19:26:56.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.285+0000 7f84ff7fe700 1 -- 172.21.15.88:0/3883952756 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f84f800cd10 con 0x7f85080ffc60 2024-02-19T19:26:56.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.286+0000 7f84ff7fe700 1 --2- 172.21.15.88:0/3883952756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84f406d8a0 0x7f84f406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:56.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.286+0000 7f84ff7fe700 1 -- 172.21.15.88:0/3883952756 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f84f8014070 con 0x7f85080ffc60 2024-02-19T19:26:56.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.286+0000 7f850e08a700 1 --2- 172.21.15.88:0/3883952756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84f406d8a0 0x7f84f406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:56.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.286+0000 7f850e08a700 1 --2- 172.21.15.88:0/3883952756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84f406d8a0 0x7f84f406fd50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f850400b760 tx=0x7f8504005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:56.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.290+0000 7f84ff7fe700 1 -- 172.21.15.88:0/3883952756 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f84f8052d20 con 0x7f85080ffc60 2024-02-19T19:26:56.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.496+0000 7f85102ee700 1 -- 172.21.15.88:0/3883952756 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8508061190 con 0x7f84f406d8a0 2024-02-19T19:26:56.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.507+0000 7f84ff7fe700 1 -- 172.21.15.88:0/3883952756 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8508061190 con 0x7f84f406d8a0 2024-02-19T19:26:56.512 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:26:56.512 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (24m) 26s ago 34m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:26:56.512 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (35m) 26s ago 35m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (16m) 28s ago 33m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (20m) 26s ago 34m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (20m) 28s ago 33m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (23m) 26s ago 33m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (16m) 26s ago 28m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (16m) 26s ago 28m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (15m) 28s ago 28m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (32s) 28s ago 28m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (25m) 26s ago 36m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (25m) 28s ago 32m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (22m) 26s ago 36m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (21m) 28s ago 32m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:26:56.513 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (24m) 26s ago 34m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:26:56.514 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (24m) 28s ago 32m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:26:56.514 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (19m) 26s ago 32m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:26:56.514 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (18m) 26s ago 31m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:26:56.514 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (18m) 26s ago 31m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:26:56.514 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (17m) 28s ago 30m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:26:56.514 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (17m) 28s ago 30m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:26:56.514 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (16m) 28s ago 29m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:26:56.514 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (24m) 26s ago 33m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:26:56.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.509+0000 7f85102ee700 1 -- 172.21.15.88:0/3883952756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84f406d8a0 msgr2=0x7f84f406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:56.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.509+0000 7f85102ee700 1 --2- 172.21.15.88:0/3883952756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84f406d8a0 0x7f84f406fd50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f850400b760 tx=0x7f8504005940 comp rx=0 tx=0).stop 2024-02-19T19:26:56.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.509+0000 7f85102ee700 1 -- 172.21.15.88:0/3883952756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85080ffc60 msgr2=0x7f8508100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:56.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.509+0000 7f85102ee700 1 --2- 172.21.15.88:0/3883952756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85080ffc60 0x7f8508100a70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f84f800e920 tx=0x7f84f800ece0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.509+0000 7f85102ee700 1 -- 172.21.15.88:0/3883952756 shutdown_connections 2024-02-19T19:26:56.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.509+0000 7f85102ee700 1 --2- 172.21.15.88:0/3883952756 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f84f406d8a0 0x7f84f406fd50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.509+0000 7f85102ee700 1 --2- 172.21.15.88:0/3883952756 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85080ff350 0x7f8508100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.509+0000 7f85102ee700 1 --2- 172.21.15.88:0/3883952756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85080ffc60 0x7f8508100a70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.509+0000 7f85102ee700 1 -- 172.21.15.88:0/3883952756 >> 172.21.15.88:0/3883952756 conn(0x7f85080748b0 msgr2=0x7f8508104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:56.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.509+0000 7f85102ee700 1 -- 172.21.15.88:0/3883952756 shutdown_connections 2024-02-19T19:26:56.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.509+0000 7f85102ee700 1 -- 172.21.15.88:0/3883952756 wait complete. 2024-02-19T19:26:56.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.616+0000 7f6e99dea700 1 -- 172.21.15.88:0/831732080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e940ffd60 msgr2=0x7f6e94103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:56.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.616+0000 7f6e99dea700 1 --2- 172.21.15.88:0/831732080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e940ffd60 0x7f6e94103d30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c009910 tx=0x7f6e7c009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:56.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.616+0000 7f6e99dea700 1 -- 172.21.15.88:0/831732080 shutdown_connections 2024-02-19T19:26:56.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.616+0000 7f6e99dea700 1 --2- 172.21.15.88:0/831732080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e940ffd60 0x7f6e94103d30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.616+0000 7f6e99dea700 1 --2- 172.21.15.88:0/831732080 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e940ff3c0 0x7f6e940ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.616+0000 7f6e99dea700 1 -- 172.21.15.88:0/831732080 >> 172.21.15.88:0/831732080 conn(0x7f6e940fac80 msgr2=0x7f6e940fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:56.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.616+0000 7f6e99dea700 1 -- 172.21.15.88:0/831732080 shutdown_connections 2024-02-19T19:26:56.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.616+0000 7f6e99dea700 1 -- 172.21.15.88:0/831732080 wait complete. 2024-02-19T19:26:56.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.617+0000 7f6e99dea700 1 Processor -- start 2024-02-19T19:26:56.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.617+0000 7f6e99dea700 1 -- start start 2024-02-19T19:26:56.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.617+0000 7f6e99dea700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e940ff3c0 0x7f6e94196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.617+0000 7f6e99dea700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e940ffd60 0x7f6e94196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.617+0000 7f6e99dea700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e94196c20 con 0x7f6e940ff3c0 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.617+0000 7f6e99dea700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9419a9b0 con 0x7f6e940ffd60 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.617+0000 7f6e937fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e940ff3c0 0x7f6e94196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.617+0000 7f6e937fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e940ff3c0 0x7f6e94196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51398/0 (socket says 172.21.15.88:51398) 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.617+0000 7f6e937fe700 1 -- 172.21.15.88:0/864978581 learned_addr learned my addr 172.21.15.88:0/864978581 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.618+0000 7f6e8bfff700 1 --2- 172.21.15.88:0/864978581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e940ffd60 0x7f6e94196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.618+0000 7f6e937fe700 1 -- 172.21.15.88:0/864978581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e940ffd60 msgr2=0x7f6e94196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.618+0000 7f6e937fe700 1 --2- 172.21.15.88:0/864978581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e940ffd60 0x7f6e94196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.618+0000 7f6e937fe700 1 -- 172.21.15.88:0/864978581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c0095f0 con 0x7f6e940ff3c0 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.618+0000 7f6e937fe700 1 --2- 172.21.15.88:0/864978581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e940ff3c0 0x7f6e94196000 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400b4b0 tx=0x7f6e8400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:56.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.618+0000 7f6e917fa700 1 -- 172.21.15.88:0/864978581 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e84011800 con 0x7f6e940ff3c0 2024-02-19T19:26:56.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.618+0000 7f6e99dea700 1 -- 172.21.15.88:0/864978581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9419ac90 con 0x7f6e940ff3c0 2024-02-19T19:26:56.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.618+0000 7f6e917fa700 1 -- 172.21.15.88:0/864978581 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e84011e20 con 0x7f6e940ff3c0 2024-02-19T19:26:56.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.618+0000 7f6e917fa700 1 -- 172.21.15.88:0/864978581 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e8400f2e0 con 0x7f6e940ff3c0 2024-02-19T19:26:56.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.618+0000 7f6e99dea700 1 -- 172.21.15.88:0/864978581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9419b1e0 con 0x7f6e940ff3c0 2024-02-19T19:26:56.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.619+0000 7f6e99dea700 1 -- 172.21.15.88:0/864978581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9404f2a0 con 0x7f6e940ff3c0 2024-02-19T19:26:56.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.619+0000 7f6e917fa700 1 -- 172.21.15.88:0/864978581 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6e84010380 con 0x7f6e940ff3c0 2024-02-19T19:26:56.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.620+0000 7f6e917fa700 1 --2- 172.21.15.88:0/864978581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8006d8a0 0x7f6e8006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:56.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.620+0000 7f6e917fa700 1 -- 172.21.15.88:0/864978581 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6e8408d0f0 con 0x7f6e940ff3c0 2024-02-19T19:26:56.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.620+0000 7f6e8bfff700 1 --2- 172.21.15.88:0/864978581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8006d8a0 0x7f6e8006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:56.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.620+0000 7f6e8bfff700 1 --2- 172.21.15.88:0/864978581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8006d8a0 0x7f6e8006fd50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6e94197620 tx=0x7f6e7c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:56.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.625+0000 7f6e917fa700 1 -- 172.21.15.88:0/864978581 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e840568a0 con 0x7f6e940ff3c0 2024-02-19T19:26:56.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.903+0000 7f6e99dea700 1 -- 172.21.15.88:0/864978581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6e9419b460 con 0x7f6e940ff3c0 2024-02-19T19:26:56.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.904+0000 7f6e917fa700 1 -- 172.21.15.88:0/864978581 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6e84056020 con 0x7f6e940ff3c0 2024-02-19T19:26:56.909 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:26:56.909 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:26:56.909 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:26:56.909 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:26:56.909 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:26:56.910 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:26:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.906+0000 7f6e99dea700 1 -- 172.21.15.88:0/864978581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8006d8a0 msgr2=0x7f6e8006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.906+0000 7f6e99dea700 1 --2- 172.21.15.88:0/864978581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8006d8a0 0x7f6e8006fd50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6e94197620 tx=0x7f6e7c01a040 comp rx=0 tx=0).stop 2024-02-19T19:26:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.906+0000 7f6e99dea700 1 -- 172.21.15.88:0/864978581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e940ff3c0 msgr2=0x7f6e94196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.906+0000 7f6e99dea700 1 --2- 172.21.15.88:0/864978581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e940ff3c0 0x7f6e94196000 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400b4b0 tx=0x7f6e8400b870 comp rx=0 tx=0).stop 2024-02-19T19:26:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.906+0000 7f6e99dea700 1 -- 172.21.15.88:0/864978581 shutdown_connections 2024-02-19T19:26:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.906+0000 7f6e99dea700 1 --2- 172.21.15.88:0/864978581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8006d8a0 0x7f6e8006fd50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.906+0000 7f6e99dea700 1 --2- 172.21.15.88:0/864978581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e940ff3c0 0x7f6e94196000 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.906+0000 7f6e99dea700 1 --2- 172.21.15.88:0/864978581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e940ffd60 0x7f6e94196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:56.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.906+0000 7f6e99dea700 1 -- 172.21.15.88:0/864978581 >> 172.21.15.88:0/864978581 conn(0x7f6e940fac80 msgr2=0x7f6e941034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:56.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.906+0000 7f6e99dea700 1 -- 172.21.15.88:0/864978581 shutdown_connections 2024-02-19T19:26:56.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:56.906+0000 7f6e99dea700 1 -- 172.21.15.88:0/864978581 wait complete. 2024-02-19T19:26:57.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.009+0000 7ff758d61700 1 -- 172.21.15.88:0/550392490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7541004a0 msgr2=0x7ff754100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.009+0000 7ff758d61700 1 --2- 172.21.15.88:0/550392490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7541004a0 0x7ff754100870 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff744009910 tx=0x7ff744009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:57.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.009+0000 7ff758d61700 1 -- 172.21.15.88:0/550392490 shutdown_connections 2024-02-19T19:26:57.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.009+0000 7ff758d61700 1 --2- 172.21.15.88:0/550392490 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff754100e40 0x7ff754104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.009+0000 7ff758d61700 1 --2- 172.21.15.88:0/550392490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7541004a0 0x7ff754100870 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.009+0000 7ff758d61700 1 -- 172.21.15.88:0/550392490 >> 172.21.15.88:0/550392490 conn(0x7ff7540fbd40 msgr2=0x7ff7540fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.009+0000 7ff758d61700 1 -- 172.21.15.88:0/550392490 shutdown_connections 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.009+0000 7ff758d61700 1 -- 172.21.15.88:0/550392490 wait complete. 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff758d61700 1 Processor -- start 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff758d61700 1 -- start start 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff758d61700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7541004a0 0x7ff754195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff758d61700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff754100e40 0x7ff754196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff758d61700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff754196af0 con 0x7ff7541004a0 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff758d61700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75419a880 con 0x7ff754100e40 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff75259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7541004a0 0x7ff754195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff75259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7541004a0 0x7ff754195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51424/0 (socket says 172.21.15.88:51424) 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff75259c700 1 -- 172.21.15.88:0/2066201382 learned_addr learned my addr 172.21.15.88:0/2066201382 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff751d9b700 1 --2- 172.21.15.88:0/2066201382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff754100e40 0x7ff754196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:57.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff75259c700 1 -- 172.21.15.88:0/2066201382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff754100e40 msgr2=0x7ff754196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff75259c700 1 --2- 172.21.15.88:0/2066201382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff754100e40 0x7ff754196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff75259c700 1 -- 172.21.15.88:0/2066201382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440095f0 con 0x7ff7541004a0 2024-02-19T19:26:57.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff75259c700 1 --2- 172.21.15.88:0/2066201382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7541004a0 0x7ff754195ed0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff744006010 tx=0x7ff74400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:57.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.010+0000 7ff7437fe700 1 -- 172.21.15.88:0/2066201382 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74401d070 con 0x7ff7541004a0 2024-02-19T19:26:57.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.011+0000 7ff7437fe700 1 -- 172.21.15.88:0/2066201382 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff74400f460 con 0x7ff7541004a0 2024-02-19T19:26:57.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.011+0000 7ff758d61700 1 -- 172.21.15.88:0/2066201382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75419ab00 con 0x7ff7541004a0 2024-02-19T19:26:57.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.011+0000 7ff7437fe700 1 -- 172.21.15.88:0/2066201382 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff744017630 con 0x7ff7541004a0 2024-02-19T19:26:57.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.011+0000 7ff758d61700 1 -- 172.21.15.88:0/2066201382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75419aff0 con 0x7ff7541004a0 2024-02-19T19:26:57.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.011+0000 7ff758d61700 1 -- 172.21.15.88:0/2066201382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7541087b0 con 0x7ff7541004a0 2024-02-19T19:26:57.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.012+0000 7ff7437fe700 1 -- 172.21.15.88:0/2066201382 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff74400f5d0 con 0x7ff7541004a0 2024-02-19T19:26:57.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.012+0000 7ff7437fe700 1 --2- 172.21.15.88:0/2066201382 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff73c06d8a0 0x7ff73c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:57.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.013+0000 7ff7437fe700 1 -- 172.21.15.88:0/2066201382 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff74408e9f0 con 0x7ff7541004a0 2024-02-19T19:26:57.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.013+0000 7ff751d9b700 1 --2- 172.21.15.88:0/2066201382 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff73c06d8a0 0x7ff73c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:57.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.013+0000 7ff751d9b700 1 --2- 172.21.15.88:0/2066201382 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff73c06d8a0 0x7ff73c06fd50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff748005950 tx=0x7ff7480058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:57.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.017+0000 7ff7437fe700 1 -- 172.21.15.88:0/2066201382 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff744090e60 con 0x7ff7541004a0 2024-02-19T19:26:57.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.255+0000 7ff758d61700 1 -- 172.21.15.88:0/2066201382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff75404f2a0 con 0x7ff7541004a0 2024-02-19T19:26:57.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.255+0000 7ff7437fe700 1 -- 172.21.15.88:0/2066201382 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7ff7440581a0 con 0x7ff7541004a0 2024-02-19T19:26:57.261 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:26:57.261 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:26:57.261 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:26:57.262 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:26:57.263 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:26:57.264 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:26:57.264 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:26:57.264 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:26:57.264 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:26:57.264 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:26:57.264 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:26:57.264 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:26:57.264 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:26:57.264 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:26:57.264 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:26:57.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.258+0000 7ff758d61700 1 -- 172.21.15.88:0/2066201382 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff73c06d8a0 msgr2=0x7ff73c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.258+0000 7ff758d61700 1 --2- 172.21.15.88:0/2066201382 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff73c06d8a0 0x7ff73c06fd50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff748005950 tx=0x7ff7480058e0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.258+0000 7ff758d61700 1 -- 172.21.15.88:0/2066201382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7541004a0 msgr2=0x7ff754195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.258+0000 7ff758d61700 1 --2- 172.21.15.88:0/2066201382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7541004a0 0x7ff754195ed0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7ff744006010 tx=0x7ff74400be20 comp rx=0 tx=0).stop 2024-02-19T19:26:57.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.259+0000 7ff758d61700 1 -- 172.21.15.88:0/2066201382 shutdown_connections 2024-02-19T19:26:57.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.259+0000 7ff758d61700 1 --2- 172.21.15.88:0/2066201382 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff73c06d8a0 0x7ff73c06fd50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.259+0000 7ff758d61700 1 --2- 172.21.15.88:0/2066201382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7541004a0 0x7ff754195ed0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.259+0000 7ff758d61700 1 --2- 172.21.15.88:0/2066201382 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff754100e40 0x7ff754196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.259+0000 7ff758d61700 1 -- 172.21.15.88:0/2066201382 >> 172.21.15.88:0/2066201382 conn(0x7ff7540fbd40 msgr2=0x7ff7540fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:57.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.259+0000 7ff758d61700 1 -- 172.21.15.88:0/2066201382 shutdown_connections 2024-02-19T19:26:57.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.259+0000 7ff758d61700 1 -- 172.21.15.88:0/2066201382 wait complete. 2024-02-19T19:26:57.266 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:26:57.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.363+0000 7f1322545700 1 -- 172.21.15.88:0/3190292556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131c0ff3c0 msgr2=0x7f131c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.363+0000 7f1322545700 1 --2- 172.21.15.88:0/3190292556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131c0ff3c0 0x7f131c0ff790 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f130c009910 tx=0x7f130c009c20 comp rx=0 tx=0).stop 2024-02-19T19:26:57.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.363+0000 7f1322545700 1 -- 172.21.15.88:0/3190292556 shutdown_connections 2024-02-19T19:26:57.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.363+0000 7f1322545700 1 --2- 172.21.15.88:0/3190292556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f131c0ffd60 0x7f131c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.363+0000 7f1322545700 1 --2- 172.21.15.88:0/3190292556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131c0ff3c0 0x7f131c0ff790 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.363+0000 7f1322545700 1 -- 172.21.15.88:0/3190292556 >> 172.21.15.88:0/3190292556 conn(0x7f131c0fac80 msgr2=0x7f131c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:57.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.363+0000 7f1322545700 1 -- 172.21.15.88:0/3190292556 shutdown_connections 2024-02-19T19:26:57.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.363+0000 7f1322545700 1 -- 172.21.15.88:0/3190292556 wait complete. 2024-02-19T19:26:57.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.364+0000 7f1322545700 1 Processor -- start 2024-02-19T19:26:57.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.364+0000 7f1322545700 1 -- start start 2024-02-19T19:26:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f1322545700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131c0ff3c0 0x7f131c195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f1322545700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f131c0ffd60 0x7f131c1964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f1322545700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c196bb0 con 0x7f131c0ff3c0 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f1322545700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c19a940 con 0x7f131c0ffd60 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f131bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131c0ff3c0 0x7f131c195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f131bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131c0ff3c0 0x7f131c195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51440/0 (socket says 172.21.15.88:51440) 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f131bfff700 1 -- 172.21.15.88:0/2446336393 learned_addr learned my addr 172.21.15.88:0/2446336393 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f131b7fe700 1 --2- 172.21.15.88:0/2446336393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f131c0ffd60 0x7f131c1964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f131bfff700 1 -- 172.21.15.88:0/2446336393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f131c0ffd60 msgr2=0x7f131c1964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f131bfff700 1 --2- 172.21.15.88:0/2446336393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f131c0ffd60 0x7f131c1964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f131bfff700 1 -- 172.21.15.88:0/2446336393 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f130c0095f0 con 0x7f131c0ff3c0 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f131bfff700 1 --2- 172.21.15.88:0/2446336393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131c0ff3c0 0x7f131c195f90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f130c006010 tx=0x7f130c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f13197fa700 1 -- 172.21.15.88:0/2446336393 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c01d070 con 0x7f131c0ff3c0 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f13197fa700 1 -- 172.21.15.88:0/2446336393 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f130c004fb0 con 0x7f131c0ff3c0 2024-02-19T19:26:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f1322545700 1 -- 172.21.15.88:0/2446336393 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c19abc0 con 0x7f131c0ff3c0 2024-02-19T19:26:57.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f13197fa700 1 -- 172.21.15.88:0/2446336393 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c022ea0 con 0x7f131c0ff3c0 2024-02-19T19:26:57.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.365+0000 7f1322545700 1 -- 172.21.15.88:0/2446336393 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131c19b0b0 con 0x7f131c0ff3c0 2024-02-19T19:26:57.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.366+0000 7f1322545700 1 -- 172.21.15.88:0/2446336393 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131c1076d0 con 0x7f131c0ff3c0 2024-02-19T19:26:57.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.367+0000 7f13197fa700 1 -- 172.21.15.88:0/2446336393 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f130c004af0 con 0x7f131c0ff3c0 2024-02-19T19:26:57.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.367+0000 7f13197fa700 1 --2- 172.21.15.88:0/2446336393 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f130406d8a0 0x7f130406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:57.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.367+0000 7f13197fa700 1 -- 172.21.15.88:0/2446336393 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f130c08ece0 con 0x7f131c0ff3c0 2024-02-19T19:26:57.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.367+0000 7f131b7fe700 1 --2- 172.21.15.88:0/2446336393 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f130406d8a0 0x7f130406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:57.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.368+0000 7f131b7fe700 1 --2- 172.21.15.88:0/2446336393 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f130406d8a0 0x7f130406fd50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1310006fd0 tx=0x7f1310008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:57.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.372+0000 7f13197fa700 1 -- 172.21.15.88:0/2446336393 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f130c058660 con 0x7f131c0ff3c0 2024-02-19T19:26:57.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.586+0000 7f1322545700 1 -- 172.21.15.88:0/2446336393 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f131c19b2d0 con 0x7f130406d8a0 2024-02-19T19:26:57.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.588+0000 7f13197fa700 1 -- 172.21.15.88:0/2446336393 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f131c19b2d0 con 0x7f130406d8a0 2024-02-19T19:26:57.593 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:26:57.593 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:26:57.593 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:26:57.593 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:26:57.593 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:26:57.593 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:26:57.594 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:26:57.594 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:26:57.594 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:26:57.594 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:26:57.594 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:26:57.594 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:26:57.594 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:26:57.594 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:26:57.594 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:26:57.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.590+0000 7f1322545700 1 -- 172.21.15.88:0/2446336393 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f130406d8a0 msgr2=0x7f130406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.590+0000 7f1322545700 1 --2- 172.21.15.88:0/2446336393 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f130406d8a0 0x7f130406fd50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1310006fd0 tx=0x7f1310008040 comp rx=0 tx=0).stop 2024-02-19T19:26:57.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.590+0000 7f1322545700 1 -- 172.21.15.88:0/2446336393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131c0ff3c0 msgr2=0x7f131c195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.590+0000 7f1322545700 1 --2- 172.21.15.88:0/2446336393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131c0ff3c0 0x7f131c195f90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f130c006010 tx=0x7f130c00be20 comp rx=0 tx=0).stop 2024-02-19T19:26:57.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.590+0000 7f1322545700 1 -- 172.21.15.88:0/2446336393 shutdown_connections 2024-02-19T19:26:57.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.590+0000 7f1322545700 1 --2- 172.21.15.88:0/2446336393 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f130406d8a0 0x7f130406fd50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.590+0000 7f1322545700 1 --2- 172.21.15.88:0/2446336393 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131c0ff3c0 0x7f131c195f90 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.590+0000 7f1322545700 1 --2- 172.21.15.88:0/2446336393 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f131c0ffd60 0x7f131c1964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.590+0000 7f1322545700 1 -- 172.21.15.88:0/2446336393 >> 172.21.15.88:0/2446336393 conn(0x7f131c0fac80 msgr2=0x7f131c0fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:57.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.590+0000 7f1322545700 1 -- 172.21.15.88:0/2446336393 shutdown_connections 2024-02-19T19:26:57.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.590+0000 7f1322545700 1 -- 172.21.15.88:0/2446336393 wait complete. 2024-02-19T19:26:57.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.695+0000 7f6deb9fe700 1 -- 172.21.15.88:0/168201494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6de41004e0 msgr2=0x7f6de41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.695+0000 7f6deb9fe700 1 --2- 172.21.15.88:0/168201494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6de41004e0 0x7f6de41008b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00b0a0 tx=0x7f6ddc00b3b0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.695+0000 7f6deb9fe700 1 -- 172.21.15.88:0/168201494 shutdown_connections 2024-02-19T19:26:57.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.695+0000 7f6deb9fe700 1 --2- 172.21.15.88:0/168201494 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6de4100e80 0x7f6de4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.695+0000 7f6deb9fe700 1 --2- 172.21.15.88:0/168201494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6de41004e0 0x7f6de41008b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.695+0000 7f6deb9fe700 1 -- 172.21.15.88:0/168201494 >> 172.21.15.88:0/168201494 conn(0x7f6de40fbda0 msgr2=0x7f6de40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:57.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.695+0000 7f6deb9fe700 1 -- 172.21.15.88:0/168201494 shutdown_connections 2024-02-19T19:26:57.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.695+0000 7f6deb9fe700 1 -- 172.21.15.88:0/168201494 wait complete. 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.695+0000 7f6deb9fe700 1 Processor -- start 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.695+0000 7f6deb9fe700 1 -- start start 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6deb9fe700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6de41004e0 0x7f6de4195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6deb9fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6de4100e80 0x7f6de4196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6deb9fe700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4196b10 con 0x7f6de4100e80 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6deb9fe700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de419a8a0 con 0x7f6de41004e0 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6de8f99700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6de4100e80 0x7f6de4196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6de8f99700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6de4100e80 0x7f6de4196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51468/0 (socket says 172.21.15.88:51468) 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6de8f99700 1 -- 172.21.15.88:0/2672944464 learned_addr learned my addr 172.21.15.88:0/2672944464 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6de979a700 1 --2- 172.21.15.88:0/2672944464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6de41004e0 0x7f6de4195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6de8f99700 1 -- 172.21.15.88:0/2672944464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6de41004e0 msgr2=0x7f6de4195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6de8f99700 1 --2- 172.21.15.88:0/2672944464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6de41004e0 0x7f6de4195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6de8f99700 1 -- 172.21.15.88:0/2672944464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc009d00 con 0x7f6de4100e80 2024-02-19T19:26:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6de8f99700 1 --2- 172.21.15.88:0/2672944464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6de4100e80 0x7f6de4196430 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800d8d0 tx=0x7f6dd800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6de27fc700 1 -- 172.21.15.88:0/2672944464 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd80041f0 con 0x7f6de4100e80 2024-02-19T19:26:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6de27fc700 1 -- 172.21.15.88:0/2672944464 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dd8007620 con 0x7f6de4100e80 2024-02-19T19:26:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6deb9fe700 1 -- 172.21.15.88:0/2672944464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de419ab80 con 0x7f6de4100e80 2024-02-19T19:26:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6de27fc700 1 -- 172.21.15.88:0/2672944464 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dd8003c50 con 0x7f6de4100e80 2024-02-19T19:26:57.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.696+0000 7f6deb9fe700 1 -- 172.21.15.88:0/2672944464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de419b0d0 con 0x7f6de4100e80 2024-02-19T19:26:57.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.697+0000 7f6deb9fe700 1 -- 172.21.15.88:0/2672944464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de41087f0 con 0x7f6de4100e80 2024-02-19T19:26:57.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.698+0000 7f6de27fc700 1 -- 172.21.15.88:0/2672944464 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6dd8004350 con 0x7f6de4100e80 2024-02-19T19:26:57.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.698+0000 7f6de27fc700 1 --2- 172.21.15.88:0/2672944464 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6dcc06d8a0 0x7f6dcc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:26:57.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.698+0000 7f6de27fc700 1 -- 172.21.15.88:0/2672944464 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6dd808d040 con 0x7f6de4100e80 2024-02-19T19:26:57.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.698+0000 7f6de979a700 1 --2- 172.21.15.88:0/2672944464 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6dcc06d8a0 0x7f6dcc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:26:57.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.699+0000 7f6de979a700 1 --2- 172.21.15.88:0/2672944464 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6dcc06d8a0 0x7f6dcc06fd50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00b830 tx=0x7f6ddc0061f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:26:57.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.703+0000 7f6de27fc700 1 -- 172.21.15.88:0/2672944464 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6dd8056870 con 0x7f6de4100e80 2024-02-19T19:26:57.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.979+0000 7f6deb9fe700 1 -- 172.21.15.88:0/2672944464 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6de419b340 con 0x7f6de4100e80 2024-02-19T19:26:57.985 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:57 smithi088 ceph-mon[88643]: from='client.35286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:57.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:57 smithi088 ceph-mon[88643]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:57.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:57 smithi088 ceph-mon[88643]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:57.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:57 smithi088 ceph-mon[88643]: pgmap v702: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:57.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:57 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/864978581' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:57.986 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:57 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2066201382' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:26:57.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.980+0000 7f6de27fc700 1 -- 172.21.15.88:0/2672944464 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f6dd8059f30 con 0x7f6de4100e80 2024-02-19T19:26:57.986 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:26:57.987 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:26:57.987 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:26:57.987 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:26:57.987 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:26:57.987 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:26:57.987 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:26:57.987 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:26:57.987 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:26:57.987 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:26:57.987 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:26:57.988 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:26:57.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:57 smithi132 ceph-mon[70676]: from='client.35286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:57.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:57 smithi132 ceph-mon[70676]: from='client.35290 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:57.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:57 smithi132 ceph-mon[70676]: from='client.35294 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:57.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:57 smithi132 ceph-mon[70676]: pgmap v702: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:26:57.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:57 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/864978581' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:26:57.989 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:57 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2066201382' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:26:57.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.982+0000 7f6deb9fe700 1 -- 172.21.15.88:0/2672944464 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6dcc06d8a0 msgr2=0x7f6dcc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.982+0000 7f6deb9fe700 1 --2- 172.21.15.88:0/2672944464 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6dcc06d8a0 0x7f6dcc06fd50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00b830 tx=0x7f6ddc0061f0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.982+0000 7f6deb9fe700 1 -- 172.21.15.88:0/2672944464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6de4100e80 msgr2=0x7f6de4196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:26:57.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.982+0000 7f6deb9fe700 1 --2- 172.21.15.88:0/2672944464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6de4100e80 0x7f6de4196430 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800d8d0 tx=0x7f6dd800dc90 comp rx=0 tx=0).stop 2024-02-19T19:26:57.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.982+0000 7f6deb9fe700 1 -- 172.21.15.88:0/2672944464 shutdown_connections 2024-02-19T19:26:57.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.982+0000 7f6deb9fe700 1 --2- 172.21.15.88:0/2672944464 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6dcc06d8a0 0x7f6dcc06fd50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.982+0000 7f6deb9fe700 1 --2- 172.21.15.88:0/2672944464 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6de41004e0 0x7f6de4195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.982+0000 7f6deb9fe700 1 --2- 172.21.15.88:0/2672944464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6de4100e80 0x7f6de4196430 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:26:57.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.982+0000 7f6deb9fe700 1 -- 172.21.15.88:0/2672944464 >> 172.21.15.88:0/2672944464 conn(0x7f6de40fbda0 msgr2=0x7f6de40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:26:57.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.982+0000 7f6deb9fe700 1 -- 172.21.15.88:0/2672944464 shutdown_connections 2024-02-19T19:26:57.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:26:57.982+0000 7f6deb9fe700 1 -- 172.21.15.88:0/2672944464 wait complete. 2024-02-19T19:26:58.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:58 smithi132 ceph-mon[70676]: from='client.35306 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:58.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:58 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2672944464' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:26:58.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:26:59.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:58 smithi088 ceph-mon[88643]: from='client.35306 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:26:59.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:58 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2672944464' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:26:59.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:26:59.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:26:59 smithi132 ceph-mon[70676]: pgmap v703: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:00.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:26:59 smithi088 ceph-mon[88643]: pgmap v703: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:01.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:01 smithi132 ceph-mon[70676]: pgmap v704: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:02.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:01 smithi088 ceph-mon[88643]: pgmap v704: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:03.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:03 smithi132 ceph-mon[70676]: pgmap v705: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:04.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:03 smithi088 ceph-mon[88643]: pgmap v705: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:05.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:05 smithi132 ceph-mon[70676]: pgmap v706: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:06.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:05 smithi088 ceph-mon[88643]: pgmap v706: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:07.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:07 smithi132 ceph-mon[70676]: pgmap v707: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:08.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:07 smithi088 ceph-mon[88643]: pgmap v707: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:09.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:09 smithi132 ceph-mon[70676]: pgmap v708: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:10.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:09 smithi088 ceph-mon[88643]: pgmap v708: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:11.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:11 smithi132 ceph-mon[70676]: pgmap v709: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:12.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:11 smithi088 ceph-mon[88643]: pgmap v709: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:13.835 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:13 smithi088 ceph-mon[88643]: pgmap v710: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:13.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:27:13.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:13 smithi132 ceph-mon[70676]: pgmap v710: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:13.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:27:14.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:14 smithi132 ceph-mon[70676]: pgmap v711: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:15.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:14 smithi088 ceph-mon[88643]: pgmap v711: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:17.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:17 smithi132 ceph-mon[70676]: pgmap v712: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:18.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:17 smithi088 ceph-mon[88643]: pgmap v712: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:19.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:19 smithi132 ceph-mon[70676]: pgmap v713: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:20.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:19 smithi088 ceph-mon[88643]: pgmap v713: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:21.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:21 smithi132 ceph-mon[70676]: pgmap v714: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:22.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:21 smithi088 ceph-mon[88643]: pgmap v714: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:23.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:23 smithi132 ceph-mon[70676]: pgmap v715: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:24.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:23 smithi088 ceph-mon[88643]: pgmap v715: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:25.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:25 smithi132 ceph-mon[70676]: pgmap v716: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:26.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:25 smithi088 ceph-mon[88643]: pgmap v716: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:27.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:27 smithi132 ceph-mon[70676]: pgmap v717: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:28.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:27 smithi088 ceph-mon[88643]: pgmap v717: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:28.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.100+0000 7f3553b14700 1 -- 172.21.15.88:0/1195190307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f354c1004e0 msgr2=0x7f354c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:28.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.100+0000 7f3553b14700 1 --2- 172.21.15.88:0/1195190307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f354c1004e0 0x7f354c1008b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f354400b0a0 tx=0x7f354400b3b0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.100+0000 7f3553b14700 1 -- 172.21.15.88:0/1195190307 shutdown_connections 2024-02-19T19:27:28.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.100+0000 7f3553b14700 1 --2- 172.21.15.88:0/1195190307 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f354c100e80 0x7f354c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.100+0000 7f3553b14700 1 --2- 172.21.15.88:0/1195190307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f354c1004e0 0x7f354c1008b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.100+0000 7f3553b14700 1 -- 172.21.15.88:0/1195190307 >> 172.21.15.88:0/1195190307 conn(0x7f354c0fbda0 msgr2=0x7f354c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:28.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.100+0000 7f3553b14700 1 -- 172.21.15.88:0/1195190307 shutdown_connections 2024-02-19T19:27:28.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.101+0000 7f3553b14700 1 -- 172.21.15.88:0/1195190307 wait complete. 2024-02-19T19:27:28.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.101+0000 7f3553b14700 1 Processor -- start 2024-02-19T19:27:28.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.101+0000 7f3553b14700 1 -- start start 2024-02-19T19:27:28.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.101+0000 7f3553b14700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f354c1004e0 0x7f354c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:28.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.101+0000 7f3553b14700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f354c100e80 0x7f354c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:28.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.101+0000 7f3553b14700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c073ad0 con 0x7f354c1004e0 2024-02-19T19:27:28.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.101+0000 7f3553b14700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f354c073c10 con 0x7f354c100e80 2024-02-19T19:27:28.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.101+0000 7f35518b0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f354c1004e0 0x7f354c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:28.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.101+0000 7f35518b0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f354c1004e0 0x7f354c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58612/0 (socket says 172.21.15.88:58612) 2024-02-19T19:27:28.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.101+0000 7f35518b0700 1 -- 172.21.15.88:0/1489187858 learned_addr learned my addr 172.21.15.88:0/1489187858 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.102+0000 7f35510af700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f354c100e80 0x7f354c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.102+0000 7f35518b0700 1 -- 172.21.15.88:0/1489187858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f354c100e80 msgr2=0x7f354c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.102+0000 7f35518b0700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f354c100e80 0x7f354c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.102+0000 7f35518b0700 1 -- 172.21.15.88:0/1489187858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3544009d00 con 0x7f354c1004e0 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.102+0000 7f35510af700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f354c100e80 0x7f354c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.102+0000 7f35518b0700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f354c1004e0 0x7f354c073050 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3544000f80 tx=0x7f354400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.102+0000 7f354affd700 1 -- 172.21.15.88:0/1489187858 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f354400e040 con 0x7f354c1004e0 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.102+0000 7f354affd700 1 -- 172.21.15.88:0/1489187858 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3544003de0 con 0x7f354c1004e0 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.102+0000 7f354affd700 1 -- 172.21.15.88:0/1489187858 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f354401d910 con 0x7f354c1004e0 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.102+0000 7f3553b14700 1 -- 172.21.15.88:0/1489187858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f354c073e90 con 0x7f354c1004e0 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.102+0000 7f3553b14700 1 -- 172.21.15.88:0/1489187858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354c1a2520 con 0x7f354c1004e0 2024-02-19T19:27:28.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.103+0000 7f3553b14700 1 -- 172.21.15.88:0/1489187858 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f354c1087f0 con 0x7f354c1004e0 2024-02-19T19:27:28.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.104+0000 7f354affd700 1 -- 172.21.15.88:0/1489187858 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3544004820 con 0x7f354c1004e0 2024-02-19T19:27:28.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.104+0000 7f354affd700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f353406d8a0 0x7f353406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:28.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.104+0000 7f354affd700 1 -- 172.21.15.88:0/1489187858 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f354408ef30 con 0x7f354c1004e0 2024-02-19T19:27:28.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.104+0000 7f35510af700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f353406d8a0 0x7f353406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:28.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.105+0000 7f35510af700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f353406d8a0 0x7f353406fd50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f354c074840 tx=0x7f3540008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:28.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.109+0000 7f354affd700 1 -- 172.21.15.88:0/1489187858 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f35440c2d20 con 0x7f354c1004e0 2024-02-19T19:27:28.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.327+0000 7f3553b14700 1 -- 172.21.15.88:0/1489187858 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f354c061190 con 0x7f353406d8a0 2024-02-19T19:27:28.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.329+0000 7f354affd700 1 -- 172.21.15.88:0/1489187858 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f354c061190 con 0x7f353406d8a0 2024-02-19T19:27:28.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.331+0000 7f3553b14700 1 -- 172.21.15.88:0/1489187858 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f353406d8a0 msgr2=0x7f353406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:28.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.331+0000 7f3553b14700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f353406d8a0 0x7f353406fd50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f354c074840 tx=0x7f3540008040 comp rx=0 tx=0).stop 2024-02-19T19:27:28.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.331+0000 7f3553b14700 1 -- 172.21.15.88:0/1489187858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f354c1004e0 msgr2=0x7f354c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:28.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.331+0000 7f3553b14700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f354c1004e0 0x7f354c073050 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f3544000f80 tx=0x7f354400cdd0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.331+0000 7f3553b14700 1 -- 172.21.15.88:0/1489187858 shutdown_connections 2024-02-19T19:27:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.331+0000 7f3553b14700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f353406d8a0 0x7f353406fd50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.331+0000 7f3553b14700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f354c1004e0 0x7f354c073050 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.331+0000 7f3553b14700 1 --2- 172.21.15.88:0/1489187858 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f354c100e80 0x7f354c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.331+0000 7f3553b14700 1 -- 172.21.15.88:0/1489187858 >> 172.21.15.88:0/1489187858 conn(0x7f354c0fbda0 msgr2=0x7f354c0fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.331+0000 7f3553b14700 1 -- 172.21.15.88:0/1489187858 shutdown_connections 2024-02-19T19:27:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.331+0000 7f3553b14700 1 -- 172.21.15.88:0/1489187858 wait complete. 2024-02-19T19:27:28.346 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:27:28.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.436+0000 7f795be28700 1 -- 172.21.15.88:0/3154728374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7954104130 msgr2=0x7f7954104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:28.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.436+0000 7f795be28700 1 --2- 172.21.15.88:0/3154728374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7954104130 0x7f7954104500 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f794c00b0a0 tx=0x7f794c00b3b0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.436+0000 7f795be28700 1 -- 172.21.15.88:0/3154728374 shutdown_connections 2024-02-19T19:27:28.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.436+0000 7f795be28700 1 --2- 172.21.15.88:0/3154728374 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79540ffbd0 0x7f7954100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.436+0000 7f795be28700 1 --2- 172.21.15.88:0/3154728374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7954104130 0x7f7954104500 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.436+0000 7f795be28700 1 -- 172.21.15.88:0/3154728374 >> 172.21.15.88:0/3154728374 conn(0x7f7954075130 msgr2=0x7f7954075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:28.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.436+0000 7f795be28700 1 -- 172.21.15.88:0/3154728374 shutdown_connections 2024-02-19T19:27:28.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.436+0000 7f795be28700 1 -- 172.21.15.88:0/3154728374 wait complete. 2024-02-19T19:27:28.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f795be28700 1 Processor -- start 2024-02-19T19:27:28.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f795be28700 1 -- start start 2024-02-19T19:27:28.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f795be28700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79540ffbd0 0x7f7954100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:28.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f795be28700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7954104130 0x7f7954101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:28.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f795be28700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79541018b0 con 0x7f7954104130 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f795be28700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79541019f0 con 0x7f79540ffbd0 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f79593c3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7954104130 0x7f7954101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f79593c3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7954104130 0x7f7954101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58632/0 (socket says 172.21.15.88:58632) 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f79593c3700 1 -- 172.21.15.88:0/2267776189 learned_addr learned my addr 172.21.15.88:0/2267776189 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f7959bc4700 1 --2- 172.21.15.88:0/2267776189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79540ffbd0 0x7f7954100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f79593c3700 1 -- 172.21.15.88:0/2267776189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79540ffbd0 msgr2=0x7f7954100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f79593c3700 1 --2- 172.21.15.88:0/2267776189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79540ffbd0 0x7f7954100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f79593c3700 1 -- 172.21.15.88:0/2267776189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f794c009d00 con 0x7f7954104130 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.437+0000 7f79593c3700 1 --2- 172.21.15.88:0/2267776189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7954104130 0x7f7954101370 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f794800c930 tx=0x7f794800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.438+0000 7f7952ffd700 1 -- 172.21.15.88:0/2267776189 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7948007ad0 con 0x7f7954104130 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.438+0000 7f795be28700 1 -- 172.21.15.88:0/2267776189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79541a20d0 con 0x7f7954104130 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.438+0000 7f7952ffd700 1 -- 172.21.15.88:0/2267776189 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f794800ce80 con 0x7f7954104130 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.438+0000 7f7952ffd700 1 -- 172.21.15.88:0/2267776189 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7948018630 con 0x7f7954104130 2024-02-19T19:27:28.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.438+0000 7f795be28700 1 -- 172.21.15.88:0/2267776189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79541a2430 con 0x7f7954104130 2024-02-19T19:27:28.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.439+0000 7f795be28700 1 -- 172.21.15.88:0/2267776189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79541086a0 con 0x7f7954104130 2024-02-19T19:27:28.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.439+0000 7f7952ffd700 1 -- 172.21.15.88:0/2267776189 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f794800f450 con 0x7f7954104130 2024-02-19T19:27:28.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.440+0000 7f7952ffd700 1 --2- 172.21.15.88:0/2267776189 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f793c06d8a0 0x7f793c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:28.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.440+0000 7f7952ffd700 1 -- 172.21.15.88:0/2267776189 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f794808d500 con 0x7f7954104130 2024-02-19T19:27:28.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.440+0000 7f7959bc4700 1 --2- 172.21.15.88:0/2267776189 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f793c06d8a0 0x7f793c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:28.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.440+0000 7f7959bc4700 1 --2- 172.21.15.88:0/2267776189 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f793c06d8a0 0x7f793c06fd50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f794c000f80 tx=0x7f794c0060c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:28.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.445+0000 7f7952ffd700 1 -- 172.21.15.88:0/2267776189 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7948056cb0 con 0x7f7954104130 2024-02-19T19:27:28.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.661+0000 7f795be28700 1 -- 172.21.15.88:0/2267776189 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7954061190 con 0x7f793c06d8a0 2024-02-19T19:27:28.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.662+0000 7f7952ffd700 1 -- 172.21.15.88:0/2267776189 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7954061190 con 0x7f793c06d8a0 2024-02-19T19:27:28.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.664+0000 7f795be28700 1 -- 172.21.15.88:0/2267776189 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f793c06d8a0 msgr2=0x7f793c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:28.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.664+0000 7f795be28700 1 --2- 172.21.15.88:0/2267776189 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f793c06d8a0 0x7f793c06fd50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f794c000f80 tx=0x7f794c0060c0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.664+0000 7f795be28700 1 -- 172.21.15.88:0/2267776189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7954104130 msgr2=0x7f7954101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:28.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.664+0000 7f795be28700 1 --2- 172.21.15.88:0/2267776189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7954104130 0x7f7954101370 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f794800c930 tx=0x7f794800cc40 comp rx=0 tx=0).stop 2024-02-19T19:27:28.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.664+0000 7f795be28700 1 -- 172.21.15.88:0/2267776189 shutdown_connections 2024-02-19T19:27:28.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.664+0000 7f795be28700 1 --2- 172.21.15.88:0/2267776189 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f793c06d8a0 0x7f793c06fd50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.664+0000 7f795be28700 1 --2- 172.21.15.88:0/2267776189 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f79540ffbd0 0x7f7954100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.664+0000 7f795be28700 1 --2- 172.21.15.88:0/2267776189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7954104130 0x7f7954101370 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.664+0000 7f795be28700 1 -- 172.21.15.88:0/2267776189 >> 172.21.15.88:0/2267776189 conn(0x7f7954075130 msgr2=0x7f79540fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:28.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.664+0000 7f795be28700 1 -- 172.21.15.88:0/2267776189 shutdown_connections 2024-02-19T19:27:28.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.664+0000 7f795be28700 1 -- 172.21.15.88:0/2267776189 wait complete. 2024-02-19T19:27:28.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.774+0000 7fdcc99b1700 1 -- 172.21.15.88:0/3551547090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc40ff350 msgr2=0x7fdcc40ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:28.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.774+0000 7fdcc99b1700 1 --2- 172.21.15.88:0/3551547090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc40ff350 0x7fdcc40ff720 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009910 tx=0x7fdcb4009c20 comp rx=0 tx=0).stop 2024-02-19T19:27:28.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.774+0000 7fdcc99b1700 1 -- 172.21.15.88:0/3551547090 shutdown_connections 2024-02-19T19:27:28.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.774+0000 7fdcc99b1700 1 --2- 172.21.15.88:0/3551547090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcc40ffc60 0x7fdcc410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.774+0000 7fdcc99b1700 1 --2- 172.21.15.88:0/3551547090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc40ff350 0x7fdcc40ff720 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.774+0000 7fdcc99b1700 1 -- 172.21.15.88:0/3551547090 >> 172.21.15.88:0/3551547090 conn(0x7fdcc40748b0 msgr2=0x7fdcc4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:28.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.774+0000 7fdcc99b1700 1 -- 172.21.15.88:0/3551547090 shutdown_connections 2024-02-19T19:27:28.781 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:27:28.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.775+0000 7fdcc99b1700 1 -- 172.21.15.88:0/3551547090 wait complete. 2024-02-19T19:27:28.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.775+0000 7fdcc99b1700 1 Processor -- start 2024-02-19T19:27:28.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.775+0000 7fdcc99b1700 1 -- start start 2024-02-19T19:27:28.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc99b1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcc40ff350 0x7fdcc4195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:28.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc99b1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc40ffc60 0x7fdcc41963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:28.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc99b1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4196ac0 con 0x7fdcc40ffc60 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc99b1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc419a850 con 0x7fdcc40ff350 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc27fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc40ffc60 0x7fdcc41963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc27fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc40ffc60 0x7fdcc41963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58650/0 (socket says 172.21.15.88:58650) 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc27fc700 1 -- 172.21.15.88:0/4120213951 learned_addr learned my addr 172.21.15.88:0/4120213951 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc2ffd700 1 --2- 172.21.15.88:0/4120213951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcc40ff350 0x7fdcc4195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc27fc700 1 -- 172.21.15.88:0/4120213951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcc40ff350 msgr2=0x7fdcc4195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc27fc700 1 --2- 172.21.15.88:0/4120213951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcc40ff350 0x7fdcc4195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc27fc700 1 -- 172.21.15.88:0/4120213951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb40095f0 con 0x7fdcc40ffc60 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.776+0000 7fdcc27fc700 1 --2- 172.21.15.88:0/4120213951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc40ffc60 0x7fdcc41963e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800d8d0 tx=0x7fdcb800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.777+0000 7fdcc89af700 1 -- 172.21.15.88:0/4120213951 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcb8009940 con 0x7fdcc40ffc60 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.777+0000 7fdcc99b1700 1 -- 172.21.15.88:0/4120213951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc419ab30 con 0x7fdcc40ffc60 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.777+0000 7fdcc89af700 1 -- 172.21.15.88:0/4120213951 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcb8010460 con 0x7fdcc40ffc60 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.777+0000 7fdcc89af700 1 -- 172.21.15.88:0/4120213951 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcb800f5f0 con 0x7fdcc40ffc60 2024-02-19T19:27:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.777+0000 7fdcc99b1700 1 -- 172.21.15.88:0/4120213951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc419b080 con 0x7fdcc40ffc60 2024-02-19T19:27:28.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.778+0000 7fdcc99b1700 1 -- 172.21.15.88:0/4120213951 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc4107dd0 con 0x7fdcc40ffc60 2024-02-19T19:27:28.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.778+0000 7fdcc89af700 1 -- 172.21.15.88:0/4120213951 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdcb800f820 con 0x7fdcc40ffc60 2024-02-19T19:27:28.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.778+0000 7fdcc89af700 1 --2- 172.21.15.88:0/4120213951 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdcac06d8a0 0x7fdcac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:28.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.779+0000 7fdcc89af700 1 -- 172.21.15.88:0/4120213951 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdcb808d480 con 0x7fdcc40ffc60 2024-02-19T19:27:28.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.779+0000 7fdcc2ffd700 1 --2- 172.21.15.88:0/4120213951 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdcac06d8a0 0x7fdcac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:28.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.779+0000 7fdcc2ffd700 1 --2- 172.21.15.88:0/4120213951 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdcac06d8a0 0x7fdcac06fd50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4005b40 tx=0x7fdcb401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:28.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.783+0000 7fdcc89af700 1 -- 172.21.15.88:0/4120213951 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdcb8056cb0 con 0x7fdcc40ffc60 2024-02-19T19:27:28.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:27:28.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.988+0000 7fdcc99b1700 1 -- 172.21.15.88:0/4120213951 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdcc419b3b0 con 0x7fdcac06d8a0 2024-02-19T19:27:29.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:28.998+0000 7fdcc89af700 1 -- 172.21.15.88:0/4120213951 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fdcc419b3b0 con 0x7fdcac06d8a0 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (24m) 59s ago 35m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (35m) 59s ago 35m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (17m) 61s ago 33m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (20m) 59s ago 35m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (20m) 61s ago 33m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (24m) 59s ago 34m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (16m) 59s ago 28m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (16m) 59s ago 28m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (16m) 61s ago 28m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (65s) 61s ago 28m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (26m) 59s ago 36m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (25m) 61s ago 33m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:27:29.004 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (22m) 59s ago 37m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:27:29.005 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (21m) 61s ago 33m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:27:29.005 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (25m) 59s ago 35m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:27:29.005 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (25m) 61s ago 33m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:27:29.005 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (20m) 59s ago 32m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:27:29.005 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (19m) 59s ago 32m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:27:29.005 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (18m) 59s ago 31m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:27:29.005 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (18m) 61s ago 31m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:27:29.005 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (17m) 61s ago 30m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:27:29.005 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (17m) 61s ago 30m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:27:29.005 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (24m) 59s ago 33m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:27:29.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.001+0000 7fdcc99b1700 1 -- 172.21.15.88:0/4120213951 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdcac06d8a0 msgr2=0x7fdcac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.001+0000 7fdcc99b1700 1 --2- 172.21.15.88:0/4120213951 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdcac06d8a0 0x7fdcac06fd50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4005b40 tx=0x7fdcb401a040 comp rx=0 tx=0).stop 2024-02-19T19:27:29.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.001+0000 7fdcc99b1700 1 -- 172.21.15.88:0/4120213951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc40ffc60 msgr2=0x7fdcc41963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.001+0000 7fdcc99b1700 1 --2- 172.21.15.88:0/4120213951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc40ffc60 0x7fdcc41963e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800d8d0 tx=0x7fdcb800dc90 comp rx=0 tx=0).stop 2024-02-19T19:27:29.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.001+0000 7fdcc99b1700 1 -- 172.21.15.88:0/4120213951 shutdown_connections 2024-02-19T19:27:29.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.001+0000 7fdcc99b1700 1 --2- 172.21.15.88:0/4120213951 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdcac06d8a0 0x7fdcac06fd50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.001+0000 7fdcc99b1700 1 --2- 172.21.15.88:0/4120213951 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdcc40ff350 0x7fdcc4195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.001+0000 7fdcc99b1700 1 --2- 172.21.15.88:0/4120213951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc40ffc60 0x7fdcc41963e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.001+0000 7fdcc99b1700 1 -- 172.21.15.88:0/4120213951 >> 172.21.15.88:0/4120213951 conn(0x7fdcc40748b0 msgr2=0x7fdcc4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:29.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.001+0000 7fdcc99b1700 1 -- 172.21.15.88:0/4120213951 shutdown_connections 2024-02-19T19:27:29.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.001+0000 7fdcc99b1700 1 -- 172.21.15.88:0/4120213951 wait complete. 2024-02-19T19:27:29.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.104+0000 7f38ce407700 1 -- 172.21.15.88:0/3332088560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c81004e0 msgr2=0x7f38c81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.104+0000 7f38ce407700 1 --2- 172.21.15.88:0/3332088560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c81004e0 0x7f38c81008b0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009910 tx=0x7f38b8009c20 comp rx=0 tx=0).stop 2024-02-19T19:27:29.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.104+0000 7f38ce407700 1 -- 172.21.15.88:0/3332088560 shutdown_connections 2024-02-19T19:27:29.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.104+0000 7f38ce407700 1 --2- 172.21.15.88:0/3332088560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c8100e80 0x7f38c8104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.104+0000 7f38ce407700 1 --2- 172.21.15.88:0/3332088560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c81004e0 0x7f38c81008b0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.104+0000 7f38ce407700 1 -- 172.21.15.88:0/3332088560 >> 172.21.15.88:0/3332088560 conn(0x7f38c80fbda0 msgr2=0x7f38c80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:29.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.104+0000 7f38ce407700 1 -- 172.21.15.88:0/3332088560 shutdown_connections 2024-02-19T19:27:29.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.104+0000 7f38ce407700 1 -- 172.21.15.88:0/3332088560 wait complete. 2024-02-19T19:27:29.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38ce407700 1 Processor -- start 2024-02-19T19:27:29.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38ce407700 1 -- start start 2024-02-19T19:27:29.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38ce407700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c81004e0 0x7f38c8195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:29.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38ce407700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c8100e80 0x7f38c8196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:29.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38ce407700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c8196b10 con 0x7f38c81004e0 2024-02-19T19:27:29.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38ce407700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c819a8a0 con 0x7f38c8100e80 2024-02-19T19:27:29.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38c7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c81004e0 0x7f38c8195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:29.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38c7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c81004e0 0x7f38c8195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58660/0 (socket says 172.21.15.88:58660) 2024-02-19T19:27:29.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38c7fff700 1 -- 172.21.15.88:0/3638628311 learned_addr learned my addr 172.21.15.88:0/3638628311 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38c7fff700 1 -- 172.21.15.88:0/3638628311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c8100e80 msgr2=0x7f38c8196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38c77fe700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c8100e80 0x7f38c8196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38c7fff700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c8100e80 0x7f38c8196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38c7fff700 1 -- 172.21.15.88:0/3638628311 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b80095f0 con 0x7f38c81004e0 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.105+0000 7f38c77fe700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c8100e80 0x7f38c8196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.106+0000 7f38c7fff700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c81004e0 0x7f38c8195ef0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f38b8006010 tx=0x7f38b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.106+0000 7f38c57fa700 1 -- 172.21.15.88:0/3638628311 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38b801d070 con 0x7f38c81004e0 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.106+0000 7f38c57fa700 1 -- 172.21.15.88:0/3638628311 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38b800f460 con 0x7f38c81004e0 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.106+0000 7f38ce407700 1 -- 172.21.15.88:0/3638628311 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c819ab20 con 0x7f38c81004e0 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.106+0000 7f38c57fa700 1 -- 172.21.15.88:0/3638628311 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38b8017630 con 0x7f38c81004e0 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.106+0000 7f38ce407700 1 -- 172.21.15.88:0/3638628311 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c819b010 con 0x7f38c81004e0 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.107+0000 7f38ce407700 1 -- 172.21.15.88:0/3638628311 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c81087f0 con 0x7f38c81004e0 2024-02-19T19:27:29.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.107+0000 7f38c57fa700 1 -- 172.21.15.88:0/3638628311 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f38b800f5d0 con 0x7f38c81004e0 2024-02-19T19:27:29.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.108+0000 7f38c57fa700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f38b006d8a0 0x7f38b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:29.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.108+0000 7f38c57fa700 1 -- 172.21.15.88:0/3638628311 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f38b808e9f0 con 0x7f38c81004e0 2024-02-19T19:27:29.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.108+0000 7f38c77fe700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f38b006d8a0 0x7f38b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:29.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.108+0000 7f38c77fe700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f38b006d8a0 0x7f38b006fd50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f38bc005fd0 tx=0x7f38bc005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:29.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.112+0000 7f38c57fa700 1 -- 172.21.15.88:0/3638628311 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f38b8090e60 con 0x7f38c81004e0 2024-02-19T19:27:29.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.397+0000 7f38ce407700 1 -- 172.21.15.88:0/3638628311 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f38c81081d0 con 0x7f38c81004e0 2024-02-19T19:27:29.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.398+0000 7f38c57fa700 1 -- 172.21.15.88:0/3638628311 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f38b80581a0 con 0x7f38c81004e0 2024-02-19T19:27:29.403 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:27:29.403 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:27:29.403 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:27:29.403 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:27:29.403 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:27:29.403 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:27:29.403 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:27:29.403 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:27:29.403 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:27:29.404 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:27:29.404 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:27:29.404 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:27:29.404 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:27:29.404 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:27:29.404 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:27:29.404 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:27:29.404 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:27:29.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.400+0000 7f38ce407700 1 -- 172.21.15.88:0/3638628311 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f38b006d8a0 msgr2=0x7f38b006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.400+0000 7f38ce407700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f38b006d8a0 0x7f38b006fd50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f38bc005fd0 tx=0x7f38bc005ee0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.400+0000 7f38ce407700 1 -- 172.21.15.88:0/3638628311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c81004e0 msgr2=0x7f38c8195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.400+0000 7f38ce407700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c81004e0 0x7f38c8195ef0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f38b8006010 tx=0x7f38b800be20 comp rx=0 tx=0).stop 2024-02-19T19:27:29.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.400+0000 7f38ce407700 1 -- 172.21.15.88:0/3638628311 shutdown_connections 2024-02-19T19:27:29.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.400+0000 7f38ce407700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f38b006d8a0 0x7f38b006fd50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.400+0000 7f38ce407700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c81004e0 0x7f38c8195ef0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.400+0000 7f38ce407700 1 --2- 172.21.15.88:0/3638628311 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f38c8100e80 0x7f38c8196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.400+0000 7f38ce407700 1 -- 172.21.15.88:0/3638628311 >> 172.21.15.88:0/3638628311 conn(0x7f38c80fbda0 msgr2=0x7f38c80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:29.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.400+0000 7f38ce407700 1 -- 172.21.15.88:0/3638628311 shutdown_connections 2024-02-19T19:27:29.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.400+0000 7f38ce407700 1 -- 172.21.15.88:0/3638628311 wait complete. 2024-02-19T19:27:29.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.505+0000 7efd92bd0700 1 -- 172.21.15.88:0/2109418615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd8c0ff3c0 msgr2=0x7efd8c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.505+0000 7efd92bd0700 1 --2- 172.21.15.88:0/2109418615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd8c0ff3c0 0x7efd8c0ff790 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efd80009910 tx=0x7efd80009c20 comp rx=0 tx=0).stop 2024-02-19T19:27:29.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.505+0000 7efd92bd0700 1 -- 172.21.15.88:0/2109418615 shutdown_connections 2024-02-19T19:27:29.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.505+0000 7efd92bd0700 1 --2- 172.21.15.88:0/2109418615 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8c0ffd60 0x7efd8c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.505+0000 7efd92bd0700 1 --2- 172.21.15.88:0/2109418615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd8c0ff3c0 0x7efd8c0ff790 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.505+0000 7efd92bd0700 1 -- 172.21.15.88:0/2109418615 >> 172.21.15.88:0/2109418615 conn(0x7efd8c0fac80 msgr2=0x7efd8c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:29.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.505+0000 7efd92bd0700 1 -- 172.21.15.88:0/2109418615 shutdown_connections 2024-02-19T19:27:29.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.505+0000 7efd92bd0700 1 -- 172.21.15.88:0/2109418615 wait complete. 2024-02-19T19:27:29.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd92bd0700 1 Processor -- start 2024-02-19T19:27:29.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd92bd0700 1 -- start start 2024-02-19T19:27:29.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd92bd0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8c0ff3c0 0x7efd8c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd92bd0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd8c0ffd60 0x7efd8c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd92bd0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c196b10 con 0x7efd8c0ffd60 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd92bd0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd8c19a8a0 con 0x7efd8c0ff3c0 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd8bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd8c0ffd60 0x7efd8c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd8bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd8c0ffd60 0x7efd8c196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58676/0 (socket says 172.21.15.88:58676) 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd8bfff700 1 -- 172.21.15.88:0/1546328045 learned_addr learned my addr 172.21.15.88:0/1546328045 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd9096c700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8c0ff3c0 0x7efd8c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd8bfff700 1 -- 172.21.15.88:0/1546328045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8c0ff3c0 msgr2=0x7efd8c195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd8bfff700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8c0ff3c0 0x7efd8c195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd8bfff700 1 -- 172.21.15.88:0/1546328045 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800095f0 con 0x7efd8c0ffd60 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd9096c700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8c0ff3c0 0x7efd8c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.506+0000 7efd8bfff700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd8c0ffd60 0x7efd8c196430 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efd7800c8f0 tx=0x7efd7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:29.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.507+0000 7efd89ffb700 1 -- 172.21.15.88:0/1546328045 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd78007a50 con 0x7efd8c0ffd60 2024-02-19T19:27:29.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.507+0000 7efd89ffb700 1 -- 172.21.15.88:0/1546328045 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd78007bb0 con 0x7efd8c0ffd60 2024-02-19T19:27:29.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.507+0000 7efd92bd0700 1 -- 172.21.15.88:0/1546328045 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd8c19ab80 con 0x7efd8c0ffd60 2024-02-19T19:27:29.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.507+0000 7efd89ffb700 1 -- 172.21.15.88:0/1546328045 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd78018630 con 0x7efd8c0ffd60 2024-02-19T19:27:29.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.507+0000 7efd92bd0700 1 -- 172.21.15.88:0/1546328045 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd8c19b0d0 con 0x7efd8c0ffd60 2024-02-19T19:27:29.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.508+0000 7efd92bd0700 1 -- 172.21.15.88:0/1546328045 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd8c04f2a0 con 0x7efd8c0ffd60 2024-02-19T19:27:29.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.508+0000 7efd89ffb700 1 -- 172.21.15.88:0/1546328045 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7efd78007d20 con 0x7efd8c0ffd60 2024-02-19T19:27:29.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.509+0000 7efd89ffb700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd7c06d8a0 0x7efd7c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:29.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.509+0000 7efd9096c700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd7c06d8a0 0x7efd7c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:29.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.509+0000 7efd89ffb700 1 -- 172.21.15.88:0/1546328045 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efd7808d510 con 0x7efd8c0ffd60 2024-02-19T19:27:29.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.509+0000 7efd9096c700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd7c06d8a0 0x7efd7c06fd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efd800059d0 tx=0x7efd8001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:29.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.513+0000 7efd89ffb700 1 -- 172.21.15.88:0/1546328045 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efd78056cc0 con 0x7efd8c0ffd60 2024-02-19T19:27:29.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.756+0000 7efd92bd0700 1 -- 172.21.15.88:0/1546328045 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efd8c197250 con 0x7efd8c0ffd60 2024-02-19T19:27:29.762 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:29 smithi088 ceph-mon[88643]: from='client.35314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:27:29.762 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:29 smithi088 ceph-mon[88643]: pgmap v718: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:29.762 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:29 smithi088 ceph-mon[88643]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:27:29.762 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:29 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3638628311' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:27:29.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.757+0000 7efd89ffb700 1 -- 172.21.15.88:0/1546328045 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7efd7805a380 con 0x7efd8c0ffd60 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:27:29.763 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:27:29.764 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:27:29.765 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:27:29.766 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:27:29.766 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:27:29.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.759+0000 7efd92bd0700 1 -- 172.21.15.88:0/1546328045 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd7c06d8a0 msgr2=0x7efd7c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.759+0000 7efd92bd0700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd7c06d8a0 0x7efd7c06fd50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7efd800059d0 tx=0x7efd8001a040 comp rx=0 tx=0).stop 2024-02-19T19:27:29.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.759+0000 7efd92bd0700 1 -- 172.21.15.88:0/1546328045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd8c0ffd60 msgr2=0x7efd8c196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.759+0000 7efd92bd0700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd8c0ffd60 0x7efd8c196430 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efd7800c8f0 tx=0x7efd7800ccb0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.759+0000 7efd92bd0700 1 -- 172.21.15.88:0/1546328045 shutdown_connections 2024-02-19T19:27:29.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.759+0000 7efd92bd0700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efd7c06d8a0 0x7efd7c06fd50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.759+0000 7efd92bd0700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efd8c0ff3c0 0x7efd8c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.759+0000 7efd92bd0700 1 --2- 172.21.15.88:0/1546328045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd8c0ffd60 0x7efd8c196430 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.759+0000 7efd92bd0700 1 -- 172.21.15.88:0/1546328045 >> 172.21.15.88:0/1546328045 conn(0x7efd8c0fac80 msgr2=0x7efd8c1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:29.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.759+0000 7efd92bd0700 1 -- 172.21.15.88:0/1546328045 shutdown_connections 2024-02-19T19:27:29.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.759+0000 7efd92bd0700 1 -- 172.21.15.88:0/1546328045 wait complete. 2024-02-19T19:27:29.767 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:27:29.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.864+0000 7f9a0ee67700 1 -- 172.21.15.88:0/2935732544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a080ffbd0 msgr2=0x7f9a08100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.864+0000 7f9a0ee67700 1 --2- 172.21.15.88:0/2935732544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a080ffbd0 0x7f9a08100040 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009910 tx=0x7f99f4009c20 comp rx=0 tx=0).stop 2024-02-19T19:27:29.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.864+0000 7f9a0ee67700 1 -- 172.21.15.88:0/2935732544 shutdown_connections 2024-02-19T19:27:29.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.864+0000 7f9a0ee67700 1 --2- 172.21.15.88:0/2935732544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a080ffbd0 0x7f9a08100040 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.864+0000 7f9a0ee67700 1 --2- 172.21.15.88:0/2935732544 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a08104130 0x7f9a08104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.864+0000 7f9a0ee67700 1 -- 172.21.15.88:0/2935732544 >> 172.21.15.88:0/2935732544 conn(0x7f9a08075130 msgr2=0x7f9a08075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:29.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.864+0000 7f9a0ee67700 1 -- 172.21.15.88:0/2935732544 shutdown_connections 2024-02-19T19:27:29.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.864+0000 7f9a0ee67700 1 -- 172.21.15.88:0/2935732544 wait complete. 2024-02-19T19:27:29.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.864+0000 7f9a0ee67700 1 Processor -- start 2024-02-19T19:27:29.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a0ee67700 1 -- start start 2024-02-19T19:27:29.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a0ee67700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a080ffbd0 0x7f9a08198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:29.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a0ee67700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a08104130 0x7f9a08198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:29.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a0ee67700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a08198c50 con 0x7f9a08104130 2024-02-19T19:27:29.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a0ee67700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0819c9e0 con 0x7f9a080ffbd0 2024-02-19T19:27:29.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a07fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a08104130 0x7f9a08198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:29.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a07fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a08104130 0x7f9a08198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58702/0 (socket says 172.21.15.88:58702) 2024-02-19T19:27:29.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a07fff700 1 -- 172.21.15.88:0/1070149748 learned_addr learned my addr 172.21.15.88:0/1070149748 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:27:29.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a07fff700 1 -- 172.21.15.88:0/1070149748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a080ffbd0 msgr2=0x7f9a08198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:29.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a0cc03700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a080ffbd0 0x7f9a08198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:29.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a07fff700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a080ffbd0 0x7f9a08198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:29.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a07fff700 1 -- 172.21.15.88:0/1070149748 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f40095f0 con 0x7f9a08104130 2024-02-19T19:27:29.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a0cc03700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a080ffbd0 0x7f9a08198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:27:29.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a07fff700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a08104130 0x7f9a08198570 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f99f400bd70 tx=0x7f99f400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:29.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a05ffb700 1 -- 172.21.15.88:0/1070149748 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99f401d070 con 0x7f9a08104130 2024-02-19T19:27:29.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a0ee67700 1 -- 172.21.15.88:0/1070149748 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a0819cc60 con 0x7f9a08104130 2024-02-19T19:27:29.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a05ffb700 1 -- 172.21.15.88:0/1070149748 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99f400f460 con 0x7f9a08104130 2024-02-19T19:27:29.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.865+0000 7f9a05ffb700 1 -- 172.21.15.88:0/1070149748 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99f4017630 con 0x7f9a08104130 2024-02-19T19:27:29.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.866+0000 7f9a0ee67700 1 -- 172.21.15.88:0/1070149748 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0819d150 con 0x7f9a08104130 2024-02-19T19:27:29.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.867+0000 7f9a0ee67700 1 -- 172.21.15.88:0/1070149748 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0804f2a0 con 0x7f9a08104130 2024-02-19T19:27:29.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.867+0000 7f9a05ffb700 1 -- 172.21.15.88:0/1070149748 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f99f400f5d0 con 0x7f9a08104130 2024-02-19T19:27:29.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.867+0000 7f9a05ffb700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f99f806d8a0 0x7f99f806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:29.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.868+0000 7f9a05ffb700 1 -- 172.21.15.88:0/1070149748 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f99f408eb60 con 0x7f9a08104130 2024-02-19T19:27:29.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.868+0000 7f9a0cc03700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f99f806d8a0 0x7f99f806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:29.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.868+0000 7f9a0cc03700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f99f806d8a0 0x7f99f806fd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f99fc005fd0 tx=0x7f99fc005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:29.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:29.872+0000 7f9a05ffb700 1 -- 172.21.15.88:0/1070149748 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f99f4058310 con 0x7f9a08104130 2024-02-19T19:27:29.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:29 smithi132 ceph-mon[70676]: from='client.35314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:27:29.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:29 smithi132 ceph-mon[70676]: pgmap v718: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:29.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:29 smithi132 ceph-mon[70676]: from='client.35318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:27:29.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:29 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3638628311' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:27:30.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.090+0000 7f9a0ee67700 1 -- 172.21.15.88:0/1070149748 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a0819d4a0 con 0x7f99f806d8a0 2024-02-19T19:27:30.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.092+0000 7f9a05ffb700 1 -- 172.21.15.88:0/1070149748 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9a0819d4a0 con 0x7f99f806d8a0 2024-02-19T19:27:30.097 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:27:30.097 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:27:30.097 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:27:30.097 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:27:30.097 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:27:30.097 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:27:30.097 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:27:30.097 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:27:30.098 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:27:30.098 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:27:30.098 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:27:30.098 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:27:30.098 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:27:30.098 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:27:30.098 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:27:30.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.094+0000 7f9a0ee67700 1 -- 172.21.15.88:0/1070149748 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f99f806d8a0 msgr2=0x7f99f806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:30.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.094+0000 7f9a0ee67700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f99f806d8a0 0x7f99f806fd50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f99fc005fd0 tx=0x7f99fc005dc0 comp rx=0 tx=0).stop 2024-02-19T19:27:30.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.094+0000 7f9a0ee67700 1 -- 172.21.15.88:0/1070149748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a08104130 msgr2=0x7f9a08198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:30.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.094+0000 7f9a0ee67700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a08104130 0x7f9a08198570 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f99f400bd70 tx=0x7f99f400be50 comp rx=0 tx=0).stop 2024-02-19T19:27:30.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.094+0000 7f9a0ee67700 1 -- 172.21.15.88:0/1070149748 shutdown_connections 2024-02-19T19:27:30.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.094+0000 7f9a0ee67700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f99f806d8a0 0x7f99f806fd50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:30.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.094+0000 7f9a0ee67700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9a080ffbd0 0x7f9a08198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:30.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.094+0000 7f9a0ee67700 1 --2- 172.21.15.88:0/1070149748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a08104130 0x7f9a08198570 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:30.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.094+0000 7f9a0ee67700 1 -- 172.21.15.88:0/1070149748 >> 172.21.15.88:0/1070149748 conn(0x7f9a08075130 msgr2=0x7f9a08068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:30.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.094+0000 7f9a0ee67700 1 -- 172.21.15.88:0/1070149748 shutdown_connections 2024-02-19T19:27:30.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.094+0000 7f9a0ee67700 1 -- 172.21.15.88:0/1070149748 wait complete. 2024-02-19T19:27:30.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.198+0000 7f1c23236700 1 -- 172.21.15.88:0/2269796249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0ff350 msgr2=0x7f1c1c0ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:30.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.198+0000 7f1c23236700 1 --2- 172.21.15.88:0/2269796249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0ff350 0x7f1c1c0ff720 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009910 tx=0x7f1c10009c20 comp rx=0 tx=0).stop 2024-02-19T19:27:30.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 -- 172.21.15.88:0/2269796249 shutdown_connections 2024-02-19T19:27:30.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 --2- 172.21.15.88:0/2269796249 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c0ffc60 0x7f1c1c10a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:30.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 --2- 172.21.15.88:0/2269796249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0ff350 0x7f1c1c0ff720 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 -- 172.21.15.88:0/2269796249 >> 172.21.15.88:0/2269796249 conn(0x7f1c1c0748b0 msgr2=0x7f1c1c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 -- 172.21.15.88:0/2269796249 shutdown_connections 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 -- 172.21.15.88:0/2269796249 wait complete. 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 Processor -- start 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 -- start start 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0ff350 0x7f1c1c195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c0ffc60 0x7f1c1c1964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c196bc0 con 0x7f1c1c0ff350 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.199+0000 7f1c23236700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1c19a900 con 0x7f1c1c0ffc60 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c20fd2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0ff350 0x7f1c1c195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c20fd2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0ff350 0x7f1c1c195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58716/0 (socket says 172.21.15.88:58716) 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c20fd2700 1 -- 172.21.15.88:0/1521799326 learned_addr learned my addr 172.21.15.88:0/1521799326 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:27:30.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c1bfff700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c0ffc60 0x7f1c1c1964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c20fd2700 1 -- 172.21.15.88:0/1521799326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c0ffc60 msgr2=0x7f1c1c1964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c20fd2700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c0ffc60 0x7f1c1c1964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c20fd2700 1 -- 172.21.15.88:0/1521799326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c100095f0 con 0x7f1c1c0ff350 2024-02-19T19:27:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c1bfff700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c0ffc60 0x7f1c1c1964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:27:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c20fd2700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0ff350 0x7f1c1c195fa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1c10006010 tx=0x7f1c1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c19ffb700 1 -- 172.21.15.88:0/1521799326 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c1001d070 con 0x7f1c1c0ff350 2024-02-19T19:27:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c19ffb700 1 -- 172.21.15.88:0/1521799326 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1c10004fb0 con 0x7f1c1c0ff350 2024-02-19T19:27:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c19ffb700 1 -- 172.21.15.88:0/1521799326 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c10022ea0 con 0x7f1c1c0ff350 2024-02-19T19:27:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c23236700 1 -- 172.21.15.88:0/1521799326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1c19ab80 con 0x7f1c1c0ff350 2024-02-19T19:27:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.200+0000 7f1c23236700 1 -- 172.21.15.88:0/1521799326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c1c19b070 con 0x7f1c1c0ff350 2024-02-19T19:27:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.201+0000 7f1c23236700 1 -- 172.21.15.88:0/1521799326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1c107dd0 con 0x7f1c1c0ff350 2024-02-19T19:27:30.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.202+0000 7f1c19ffb700 1 -- 172.21.15.88:0/1521799326 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1c10022470 con 0x7f1c1c0ff350 2024-02-19T19:27:30.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.202+0000 7f1c19ffb700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c0c06d8a0 0x7f1c0c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:27:30.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.203+0000 7f1c1bfff700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c0c06d8a0 0x7f1c0c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:27:30.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.203+0000 7f1c19ffb700 1 -- 172.21.15.88:0/1521799326 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1c1008e9c0 con 0x7f1c1c0ff350 2024-02-19T19:27:30.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.203+0000 7f1c1bfff700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c0c06d8a0 0x7f1c0c06fd50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c197570 tx=0x7f1c08008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:27:30.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.207+0000 7f1c19ffb700 1 -- 172.21.15.88:0/1521799326 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c100582a0 con 0x7f1c1c0ff350 2024-02-19T19:27:30.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.486+0000 7f1c23236700 1 -- 172.21.15.88:0/1521799326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1c1c04f2a0 con 0x7f1c1c0ff350 2024-02-19T19:27:30.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.487+0000 7f1c19ffb700 1 -- 172.21.15.88:0/1521799326 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f1c10022720 con 0x7f1c1c0ff350 2024-02-19T19:27:30.492 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:27:30.492 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:27:30.492 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:27:30.492 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:27:30.492 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:27:30.492 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:27:30.493 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:27:30.494 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:27:30.494 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:27:30.494 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:27:30.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.489+0000 7f1c23236700 1 -- 172.21.15.88:0/1521799326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c0c06d8a0 msgr2=0x7f1c0c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:30.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.489+0000 7f1c23236700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c0c06d8a0 0x7f1c0c06fd50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c197570 tx=0x7f1c08008040 comp rx=0 tx=0).stop 2024-02-19T19:27:30.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.489+0000 7f1c23236700 1 -- 172.21.15.88:0/1521799326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0ff350 msgr2=0x7f1c1c195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:27:30.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.489+0000 7f1c23236700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0ff350 0x7f1c1c195fa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1c10006010 tx=0x7f1c1000be20 comp rx=0 tx=0).stop 2024-02-19T19:27:30.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.489+0000 7f1c23236700 1 -- 172.21.15.88:0/1521799326 shutdown_connections 2024-02-19T19:27:30.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.489+0000 7f1c23236700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1c0c06d8a0 0x7f1c0c06fd50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:30.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.489+0000 7f1c23236700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c1c0ff350 0x7f1c1c195fa0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:30.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.489+0000 7f1c23236700 1 --2- 172.21.15.88:0/1521799326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1c1c0ffc60 0x7f1c1c1964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:27:30.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.489+0000 7f1c23236700 1 -- 172.21.15.88:0/1521799326 >> 172.21.15.88:0/1521799326 conn(0x7f1c1c0748b0 msgr2=0x7f1c1c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:27:30.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.489+0000 7f1c23236700 1 -- 172.21.15.88:0/1521799326 shutdown_connections 2024-02-19T19:27:30.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:27:30.489+0000 7f1c23236700 1 -- 172.21.15.88:0/1521799326 wait complete. 2024-02-19T19:27:30.835 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:30 smithi088 ceph-mon[88643]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:27:30.835 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:30 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1546328045' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:27:30.835 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:30 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1521799326' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:27:30.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:30 smithi132 ceph-mon[70676]: from='client.35322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:27:30.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:30 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1546328045' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:27:30.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:30 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1521799326' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:27:31.919 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:31 smithi132 ceph-mon[70676]: from='client.35334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:27:31.920 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:31 smithi132 ceph-mon[70676]: pgmap v719: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:32.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:31 smithi088 ceph-mon[88643]: from='client.35334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:27:32.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:31 smithi088 ceph-mon[88643]: pgmap v719: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:33.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:33 smithi132 ceph-mon[70676]: pgmap v720: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:34.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:33 smithi088 ceph-mon[88643]: pgmap v720: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:35.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:35 smithi132 ceph-mon[70676]: pgmap v721: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:36.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:35 smithi088 ceph-mon[88643]: pgmap v721: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:37.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:37 smithi132 ceph-mon[70676]: pgmap v722: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:38.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:37 smithi088 ceph-mon[88643]: pgmap v722: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:39.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:39 smithi132 ceph-mon[70676]: pgmap v723: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:40.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:39 smithi088 ceph-mon[88643]: pgmap v723: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:41.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:41 smithi132 ceph-mon[70676]: pgmap v724: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:42.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:41 smithi088 ceph-mon[88643]: pgmap v724: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:43.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:43 smithi132 ceph-mon[70676]: pgmap v725: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:43.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:27:44.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:43 smithi088 ceph-mon[88643]: pgmap v725: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:44.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:27:45.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:45 smithi132 ceph-mon[70676]: pgmap v726: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:46.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:45 smithi088 ceph-mon[88643]: pgmap v726: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:47.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:47 smithi132 ceph-mon[70676]: pgmap v727: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:48.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:47 smithi088 ceph-mon[88643]: pgmap v727: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:49.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:49 smithi132 ceph-mon[70676]: pgmap v728: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:50.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:49 smithi088 ceph-mon[88643]: pgmap v728: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:50.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:50 smithi132 ceph-mon[70676]: pgmap v729: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:51.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:50 smithi088 ceph-mon[88643]: pgmap v729: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:53.916 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:53 smithi088 ceph-mon[88643]: pgmap v730: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:53.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:53 smithi132 ceph-mon[70676]: pgmap v730: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:55.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:55 smithi132 ceph-mon[70676]: pgmap v731: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:56.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:55 smithi088 ceph-mon[88643]: pgmap v731: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:57.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:57 smithi132 ceph-mon[70676]: pgmap v732: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:58.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:57 smithi088 ceph-mon[88643]: pgmap v732: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:27:58.968 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:27:58.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:27:59.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:27:59 smithi132 ceph-mon[70676]: pgmap v733: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:00.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:27:59 smithi088 ceph-mon[88643]: pgmap v733: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:00.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.608+0000 7ff0444d0700 1 -- 172.21.15.88:0/1258620159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c100e80 msgr2=0x7ff03c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.608+0000 7ff0444d0700 1 --2- 172.21.15.88:0/1258620159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c100e80 0x7ff03c104e50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff02c009910 tx=0x7ff02c009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.608+0000 7ff0444d0700 1 -- 172.21.15.88:0/1258620159 shutdown_connections 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.608+0000 7ff0444d0700 1 --2- 172.21.15.88:0/1258620159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c100e80 0x7ff03c104e50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.608+0000 7ff0444d0700 1 --2- 172.21.15.88:0/1258620159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff03c1004e0 0x7ff03c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.608+0000 7ff0444d0700 1 -- 172.21.15.88:0/1258620159 >> 172.21.15.88:0/1258620159 conn(0x7ff03c0fbda0 msgr2=0x7ff03c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.608+0000 7ff0444d0700 1 -- 172.21.15.88:0/1258620159 shutdown_connections 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.608+0000 7ff0444d0700 1 -- 172.21.15.88:0/1258620159 wait complete. 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.608+0000 7ff0444d0700 1 Processor -- start 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.608+0000 7ff0444d0700 1 -- start start 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff0444d0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff03c1004e0 0x7ff03c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff0444d0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c100e80 0x7ff03c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:00.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff0444d0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c073ad0 con 0x7ff03c100e80 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff0444d0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff03c073c10 con 0x7ff03c1004e0 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff041a6b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c100e80 0x7ff03c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff041a6b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c100e80 0x7ff03c073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35118/0 (socket says 172.21.15.88:35118) 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff041a6b700 1 -- 172.21.15.88:0/2686820540 learned_addr learned my addr 172.21.15.88:0/2686820540 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff04226c700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff03c1004e0 0x7ff03c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff041a6b700 1 -- 172.21.15.88:0/2686820540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff03c1004e0 msgr2=0x7ff03c073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff041a6b700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff03c1004e0 0x7ff03c073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff041a6b700 1 -- 172.21.15.88:0/2686820540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff02c0095f0 con 0x7ff03c100e80 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff04226c700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff03c1004e0 0x7ff03c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff041a6b700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c100e80 0x7ff03c073590 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0092e0 tx=0x7ff02c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff0337fe700 1 -- 172.21.15.88:0/2686820540 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff02c01d070 con 0x7ff03c100e80 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff0337fe700 1 -- 172.21.15.88:0/2686820540 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff02c005dc0 con 0x7ff03c100e80 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff0444d0700 1 -- 172.21.15.88:0/2686820540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff03c073e90 con 0x7ff03c100e80 2024-02-19T19:28:00.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff0337fe700 1 -- 172.21.15.88:0/2686820540 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff02c017400 con 0x7ff03c100e80 2024-02-19T19:28:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.609+0000 7ff0444d0700 1 -- 172.21.15.88:0/2686820540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff03c1a2410 con 0x7ff03c100e80 2024-02-19T19:28:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.610+0000 7ff0444d0700 1 -- 172.21.15.88:0/2686820540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff03c108850 con 0x7ff03c100e80 2024-02-19T19:28:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.611+0000 7ff0337fe700 1 -- 172.21.15.88:0/2686820540 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff02c00f460 con 0x7ff03c100e80 2024-02-19T19:28:00.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.611+0000 7ff0337fe700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff02806d8a0 0x7ff02806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:00.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.611+0000 7ff0337fe700 1 -- 172.21.15.88:0/2686820540 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff02c08ea30 con 0x7ff03c100e80 2024-02-19T19:28:00.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.611+0000 7ff04226c700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff02806d8a0 0x7ff02806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:00.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.612+0000 7ff04226c700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff02806d8a0 0x7ff02806fd50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff038005950 tx=0x7ff0380058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:00.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.616+0000 7ff0337fe700 1 -- 172.21.15.88:0/2686820540 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff02c0581e0 con 0x7ff03c100e80 2024-02-19T19:28:00.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.835+0000 7ff0444d0700 1 -- 172.21.15.88:0/2686820540 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff03c074690 con 0x7ff02806d8a0 2024-02-19T19:28:00.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.836+0000 7ff0337fe700 1 -- 172.21.15.88:0/2686820540 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff03c074690 con 0x7ff02806d8a0 2024-02-19T19:28:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.838+0000 7ff0444d0700 1 -- 172.21.15.88:0/2686820540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff02806d8a0 msgr2=0x7ff02806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.838+0000 7ff0444d0700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff02806d8a0 0x7ff02806fd50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff038005950 tx=0x7ff0380058e0 comp rx=0 tx=0).stop 2024-02-19T19:28:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.838+0000 7ff0444d0700 1 -- 172.21.15.88:0/2686820540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c100e80 msgr2=0x7ff03c073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.838+0000 7ff0444d0700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c100e80 0x7ff03c073590 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff02c0092e0 tx=0x7ff02c00be20 comp rx=0 tx=0).stop 2024-02-19T19:28:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.838+0000 7ff0444d0700 1 -- 172.21.15.88:0/2686820540 shutdown_connections 2024-02-19T19:28:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.838+0000 7ff0444d0700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff02806d8a0 0x7ff02806fd50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.838+0000 7ff0444d0700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff03c1004e0 0x7ff03c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:00.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.838+0000 7ff0444d0700 1 --2- 172.21.15.88:0/2686820540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c100e80 0x7ff03c073590 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:00.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.838+0000 7ff0444d0700 1 -- 172.21.15.88:0/2686820540 >> 172.21.15.88:0/2686820540 conn(0x7ff03c0fbda0 msgr2=0x7ff03c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:00.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.838+0000 7ff0444d0700 1 -- 172.21.15.88:0/2686820540 shutdown_connections 2024-02-19T19:28:00.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.838+0000 7ff0444d0700 1 -- 172.21.15.88:0/2686820540 wait complete. 2024-02-19T19:28:00.853 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:28:00.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.942+0000 7f1a01c87700 1 -- 172.21.15.88:0/2642386929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19fc0ff3c0 msgr2=0x7f19fc0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:00.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.942+0000 7f1a01c87700 1 --2- 172.21.15.88:0/2642386929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19fc0ff3c0 0x7f19fc0ff790 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009910 tx=0x7f19f0009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:00.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.942+0000 7f1a01c87700 1 -- 172.21.15.88:0/2642386929 shutdown_connections 2024-02-19T19:28:00.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.942+0000 7f1a01c87700 1 --2- 172.21.15.88:0/2642386929 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19fc0ffd60 0x7f19fc103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:00.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.942+0000 7f1a01c87700 1 --2- 172.21.15.88:0/2642386929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19fc0ff3c0 0x7f19fc0ff790 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:00.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.942+0000 7f1a01c87700 1 -- 172.21.15.88:0/2642386929 >> 172.21.15.88:0/2642386929 conn(0x7f19fc0fac80 msgr2=0x7f19fc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:00.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.942+0000 7f1a01c87700 1 -- 172.21.15.88:0/2642386929 shutdown_connections 2024-02-19T19:28:00.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.942+0000 7f1a01c87700 1 -- 172.21.15.88:0/2642386929 wait complete. 2024-02-19T19:28:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.943+0000 7f1a01c87700 1 Processor -- start 2024-02-19T19:28:00.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.943+0000 7f1a01c87700 1 -- start start 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.943+0000 7f1a01c87700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19fc0ff3c0 0x7f19fc194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.943+0000 7f1a01c87700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19fc0ffd60 0x7f19fc1952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.943+0000 7f1a01c87700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc1959a0 con 0x7f19fc0ffd60 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.943+0000 7f1a01c87700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc199730 con 0x7f19fc0ff3c0 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f19faffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19fc0ffd60 0x7f19fc1952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f19faffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19fc0ffd60 0x7f19fc1952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35126/0 (socket says 172.21.15.88:35126) 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f19faffd700 1 -- 172.21.15.88:0/585221257 learned_addr learned my addr 172.21.15.88:0/585221257 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f19fb7fe700 1 --2- 172.21.15.88:0/585221257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19fc0ff3c0 0x7f19fc194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f19faffd700 1 -- 172.21.15.88:0/585221257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19fc0ff3c0 msgr2=0x7f19fc194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f19faffd700 1 --2- 172.21.15.88:0/585221257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19fc0ff3c0 0x7f19fc194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f19faffd700 1 -- 172.21.15.88:0/585221257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f00095f0 con 0x7f19fc0ffd60 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f19faffd700 1 --2- 172.21.15.88:0/585221257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19fc0ffd60 0x7f19fc1952c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00d900 tx=0x7f19ec00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f19f8ff9700 1 -- 172.21.15.88:0/585221257 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19ec009900 con 0x7f19fc0ffd60 2024-02-19T19:28:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f1a01c87700 1 -- 172.21.15.88:0/585221257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19fc1999b0 con 0x7f19fc0ffd60 2024-02-19T19:28:00.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f19f8ff9700 1 -- 172.21.15.88:0/585221257 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19ec010460 con 0x7f19fc0ffd60 2024-02-19T19:28:00.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f1a01c87700 1 -- 172.21.15.88:0/585221257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19fc199f00 con 0x7f19fc0ffd60 2024-02-19T19:28:00.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.944+0000 7f19f8ff9700 1 -- 172.21.15.88:0/585221257 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19ec00f5f0 con 0x7f19fc0ffd60 2024-02-19T19:28:00.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.945+0000 7f1a01c87700 1 -- 172.21.15.88:0/585221257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e8005320 con 0x7f19fc0ffd60 2024-02-19T19:28:00.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.946+0000 7f19f8ff9700 1 -- 172.21.15.88:0/585221257 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f19ec009a60 con 0x7f19fc0ffd60 2024-02-19T19:28:00.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.946+0000 7f19f8ff9700 1 --2- 172.21.15.88:0/585221257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19e406d8a0 0x7f19e406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:00.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.947+0000 7f19f8ff9700 1 -- 172.21.15.88:0/585221257 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f19ec08cff0 con 0x7f19fc0ffd60 2024-02-19T19:28:00.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.947+0000 7f19fb7fe700 1 --2- 172.21.15.88:0/585221257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19e406d8a0 0x7f19e406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:00.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.947+0000 7f19fb7fe700 1 --2- 172.21.15.88:0/585221257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19e406d8a0 0x7f19e406fd50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f19f0006010 tx=0x7f19f001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:00.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:00.951+0000 7f19f8ff9700 1 -- 172.21.15.88:0/585221257 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f19ec0567a0 con 0x7f19fc0ffd60 2024-02-19T19:28:01.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.163+0000 7f1a01c87700 1 -- 172.21.15.88:0/585221257 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19e8000bf0 con 0x7f19e406d8a0 2024-02-19T19:28:01.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.164+0000 7f19f8ff9700 1 -- 172.21.15.88:0/585221257 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f19e8000bf0 con 0x7f19e406d8a0 2024-02-19T19:28:01.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.166+0000 7f1a01c87700 1 -- 172.21.15.88:0/585221257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19e406d8a0 msgr2=0x7f19e406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:01.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.166+0000 7f1a01c87700 1 --2- 172.21.15.88:0/585221257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19e406d8a0 0x7f19e406fd50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f19f0006010 tx=0x7f19f001a040 comp rx=0 tx=0).stop 2024-02-19T19:28:01.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.166+0000 7f1a01c87700 1 -- 172.21.15.88:0/585221257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19fc0ffd60 msgr2=0x7f19fc1952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:01.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.166+0000 7f1a01c87700 1 --2- 172.21.15.88:0/585221257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19fc0ffd60 0x7f19fc1952c0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00d900 tx=0x7f19ec00dc10 comp rx=0 tx=0).stop 2024-02-19T19:28:01.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.166+0000 7f1a01c87700 1 -- 172.21.15.88:0/585221257 shutdown_connections 2024-02-19T19:28:01.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.166+0000 7f1a01c87700 1 --2- 172.21.15.88:0/585221257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19e406d8a0 0x7f19e406fd50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.166+0000 7f1a01c87700 1 --2- 172.21.15.88:0/585221257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19fc0ff3c0 0x7f19fc194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.166+0000 7f1a01c87700 1 --2- 172.21.15.88:0/585221257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19fc0ffd60 0x7f19fc1952c0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.166+0000 7f1a01c87700 1 -- 172.21.15.88:0/585221257 >> 172.21.15.88:0/585221257 conn(0x7f19fc0fac80 msgr2=0x7f19fc0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:01.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.166+0000 7f1a01c87700 1 -- 172.21.15.88:0/585221257 shutdown_connections 2024-02-19T19:28:01.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.166+0000 7f1a01c87700 1 -- 172.21.15.88:0/585221257 wait complete. 2024-02-19T19:28:01.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.271+0000 7f24196a4700 1 -- 172.21.15.88:0/2719100175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24140ffbd0 msgr2=0x7f2414100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:01.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.271+0000 7f24196a4700 1 --2- 172.21.15.88:0/2719100175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24140ffbd0 0x7f2414100040 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2408009910 tx=0x7f2408009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:01.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.271+0000 7f24196a4700 1 -- 172.21.15.88:0/2719100175 shutdown_connections 2024-02-19T19:28:01.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.271+0000 7f24196a4700 1 --2- 172.21.15.88:0/2719100175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24140ffbd0 0x7f2414100040 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.271+0000 7f24196a4700 1 --2- 172.21.15.88:0/2719100175 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2414104130 0x7f2414104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.271+0000 7f24196a4700 1 -- 172.21.15.88:0/2719100175 >> 172.21.15.88:0/2719100175 conn(0x7f2414075130 msgr2=0x7f2414075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:01.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.271+0000 7f24196a4700 1 -- 172.21.15.88:0/2719100175 shutdown_connections 2024-02-19T19:28:01.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.271+0000 7f24196a4700 1 -- 172.21.15.88:0/2719100175 wait complete. 2024-02-19T19:28:01.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.271+0000 7f24196a4700 1 Processor -- start 2024-02-19T19:28:01.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f24196a4700 1 -- start start 2024-02-19T19:28:01.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f24196a4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24140ffbd0 0x7f2414195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:01.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f24196a4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2414104130 0x7f2414196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:01.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f24196a4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2414196a70 con 0x7f2414104130 2024-02-19T19:28:01.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f24196a4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241419a800 con 0x7f24140ffbd0 2024-02-19T19:28:01.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f24127fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2414104130 0x7f2414196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:01.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f24127fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2414104130 0x7f2414196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35140/0 (socket says 172.21.15.88:35140) 2024-02-19T19:28:01.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f24127fc700 1 -- 172.21.15.88:0/2792234665 learned_addr learned my addr 172.21.15.88:0/2792234665 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:01.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f2412ffd700 1 --2- 172.21.15.88:0/2792234665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24140ffbd0 0x7f2414195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:01.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f24127fc700 1 -- 172.21.15.88:0/2792234665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24140ffbd0 msgr2=0x7f2414195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:01.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f24127fc700 1 --2- 172.21.15.88:0/2792234665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24140ffbd0 0x7f2414195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.272+0000 7f24127fc700 1 -- 172.21.15.88:0/2792234665 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24080095f0 con 0x7f2414104130 2024-02-19T19:28:01.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.273+0000 7f24127fc700 1 --2- 172.21.15.88:0/2792234665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2414104130 0x7f2414196390 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f240800b7d0 tx=0x7f240800f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:01.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.273+0000 7f23fbfff700 1 -- 172.21.15.88:0/2792234665 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f240801d070 con 0x7f2414104130 2024-02-19T19:28:01.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.273+0000 7f23fbfff700 1 -- 172.21.15.88:0/2792234665 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24080052a0 con 0x7f2414104130 2024-02-19T19:28:01.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.273+0000 7f24196a4700 1 -- 172.21.15.88:0/2792234665 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241419aa80 con 0x7f2414104130 2024-02-19T19:28:01.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.273+0000 7f23fbfff700 1 -- 172.21.15.88:0/2792234665 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2408022470 con 0x7f2414104130 2024-02-19T19:28:01.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.273+0000 7f24196a4700 1 -- 172.21.15.88:0/2792234665 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241419af70 con 0x7f2414104130 2024-02-19T19:28:01.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.274+0000 7f24196a4700 1 -- 172.21.15.88:0/2792234665 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24141086a0 con 0x7f2414104130 2024-02-19T19:28:01.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.274+0000 7f23fbfff700 1 -- 172.21.15.88:0/2792234665 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f240800f8c0 con 0x7f2414104130 2024-02-19T19:28:01.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.275+0000 7f23fbfff700 1 --2- 172.21.15.88:0/2792234665 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23fc06d8a0 0x7f23fc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:01.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.275+0000 7f23fbfff700 1 -- 172.21.15.88:0/2792234665 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f240808e9e0 con 0x7f2414104130 2024-02-19T19:28:01.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.275+0000 7f2412ffd700 1 --2- 172.21.15.88:0/2792234665 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23fc06d8a0 0x7f23fc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:01.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.275+0000 7f2412ffd700 1 --2- 172.21.15.88:0/2792234665 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23fc06d8a0 0x7f23fc06fd50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2404009bf0 tx=0x7f2404009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:01.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.280+0000 7f23fbfff700 1 -- 172.21.15.88:0/2792234665 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2408058190 con 0x7f2414104130 2024-02-19T19:28:01.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.487+0000 7f24196a4700 1 -- 172.21.15.88:0/2792234665 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f241402cca0 con 0x7f23fc06d8a0 2024-02-19T19:28:01.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.497+0000 7f23fbfff700 1 -- 172.21.15.88:0/2792234665 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f241402cca0 con 0x7f23fc06d8a0 2024-02-19T19:28:01.503 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:28:01.503 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (25m) 91s ago 35m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:28:01.503 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (36m) 91s ago 36m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:28:01.503 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (18m) 93s ago 34m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:28:01.503 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (21m) 91s ago 36m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:28:01.503 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (21m) 93s ago 34m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:28:01.503 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (24m) 91s ago 35m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:28:01.503 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (17m) 91s ago 29m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (17m) 91s ago 29m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (16m) 93s ago 29m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (97s) 93s ago 29m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (26m) 91s ago 37m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (26m) 93s ago 33m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (23m) 91s ago 37m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (22m) 93s ago 33m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (25m) 91s ago 35m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (25m) 93s ago 33m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (20m) 91s ago 33m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (20m) 91s ago 32m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (19m) 91s ago 32m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (18m) 93s ago 31m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:28:01.504 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (18m) 93s ago 31m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:28:01.505 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (17m) 93s ago 30m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:28:01.505 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (25m) 91s ago 34m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:28:01.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.499+0000 7f24196a4700 1 -- 172.21.15.88:0/2792234665 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23fc06d8a0 msgr2=0x7f23fc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:01.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.499+0000 7f24196a4700 1 --2- 172.21.15.88:0/2792234665 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23fc06d8a0 0x7f23fc06fd50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2404009bf0 tx=0x7f2404009260 comp rx=0 tx=0).stop 2024-02-19T19:28:01.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.499+0000 7f24196a4700 1 -- 172.21.15.88:0/2792234665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2414104130 msgr2=0x7f2414196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:01.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.499+0000 7f24196a4700 1 --2- 172.21.15.88:0/2792234665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2414104130 0x7f2414196390 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f240800b7d0 tx=0x7f240800f460 comp rx=0 tx=0).stop 2024-02-19T19:28:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.500+0000 7f24196a4700 1 -- 172.21.15.88:0/2792234665 shutdown_connections 2024-02-19T19:28:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.500+0000 7f24196a4700 1 --2- 172.21.15.88:0/2792234665 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f23fc06d8a0 0x7f23fc06fd50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.500+0000 7f24196a4700 1 --2- 172.21.15.88:0/2792234665 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24140ffbd0 0x7f2414195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.500+0000 7f24196a4700 1 --2- 172.21.15.88:0/2792234665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2414104130 0x7f2414196390 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.500+0000 7f24196a4700 1 -- 172.21.15.88:0/2792234665 >> 172.21.15.88:0/2792234665 conn(0x7f2414075130 msgr2=0x7f24140fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.500+0000 7f24196a4700 1 -- 172.21.15.88:0/2792234665 shutdown_connections 2024-02-19T19:28:01.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.500+0000 7f24196a4700 1 -- 172.21.15.88:0/2792234665 wait complete. 2024-02-19T19:28:01.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.602+0000 7fd72d6e1700 1 -- 172.21.15.88:0/3545622193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7280ff3c0 msgr2=0x7fd7280ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:01.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.602+0000 7fd72d6e1700 1 --2- 172.21.15.88:0/3545622193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7280ff3c0 0x7fd7280ff790 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009910 tx=0x7fd71c009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:01.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.602+0000 7fd72d6e1700 1 -- 172.21.15.88:0/3545622193 shutdown_connections 2024-02-19T19:28:01.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.602+0000 7fd72d6e1700 1 --2- 172.21.15.88:0/3545622193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7280ffd60 0x7fd728103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.602+0000 7fd72d6e1700 1 --2- 172.21.15.88:0/3545622193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7280ff3c0 0x7fd7280ff790 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.602+0000 7fd72d6e1700 1 -- 172.21.15.88:0/3545622193 >> 172.21.15.88:0/3545622193 conn(0x7fd7280fac80 msgr2=0x7fd7280fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:01.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.602+0000 7fd72d6e1700 1 -- 172.21.15.88:0/3545622193 shutdown_connections 2024-02-19T19:28:01.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.602+0000 7fd72d6e1700 1 -- 172.21.15.88:0/3545622193 wait complete. 2024-02-19T19:28:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd72d6e1700 1 Processor -- start 2024-02-19T19:28:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd72d6e1700 1 -- start start 2024-02-19T19:28:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd72d6e1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7280ff3c0 0x7fd728196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd72d6e1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7280ffd60 0x7fd728197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd72d6e1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd728197940 con 0x7fd7280ffd60 2024-02-19T19:28:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd72d6e1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd728197ab0 con 0x7fd7280ff3c0 2024-02-19T19:28:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd7267fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7280ffd60 0x7fd728197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd7267fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7280ffd60 0x7fd728197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35158/0 (socket says 172.21.15.88:35158) 2024-02-19T19:28:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd7267fc700 1 -- 172.21.15.88:0/739195451 learned_addr learned my addr 172.21.15.88:0/739195451 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd726ffd700 1 --2- 172.21.15.88:0/739195451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7280ff3c0 0x7fd728196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd7267fc700 1 -- 172.21.15.88:0/739195451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7280ff3c0 msgr2=0x7fd728196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.603+0000 7fd7267fc700 1 --2- 172.21.15.88:0/739195451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7280ff3c0 0x7fd728196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.604+0000 7fd7267fc700 1 -- 172.21.15.88:0/739195451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c0095f0 con 0x7fd7280ffd60 2024-02-19T19:28:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.604+0000 7fd7267fc700 1 --2- 172.21.15.88:0/739195451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7280ffd60 0x7fd728197320 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd71800b480 tx=0x7fd71800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.604+0000 7fd70ffff700 1 -- 172.21.15.88:0/739195451 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7180118c0 con 0x7fd7280ffd60 2024-02-19T19:28:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.604+0000 7fd70ffff700 1 -- 172.21.15.88:0/739195451 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd718010380 con 0x7fd7280ffd60 2024-02-19T19:28:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.604+0000 7fd72d6e1700 1 -- 172.21.15.88:0/739195451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72819fe50 con 0x7fd7280ffd60 2024-02-19T19:28:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.604+0000 7fd70ffff700 1 -- 172.21.15.88:0/739195451 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71800f4f0 con 0x7fd7280ffd60 2024-02-19T19:28:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.604+0000 7fd72d6e1700 1 -- 172.21.15.88:0/739195451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7281a0310 con 0x7fd7280ffd60 2024-02-19T19:28:01.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.605+0000 7fd72d6e1700 1 -- 172.21.15.88:0/739195451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7281076d0 con 0x7fd7280ffd60 2024-02-19T19:28:01.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.605+0000 7fd70ffff700 1 -- 172.21.15.88:0/739195451 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd7180104f0 con 0x7fd7280ffd60 2024-02-19T19:28:01.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.606+0000 7fd70ffff700 1 --2- 172.21.15.88:0/739195451 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd71006d8a0 0x7fd71006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:01.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.606+0000 7fd726ffd700 1 --2- 172.21.15.88:0/739195451 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd71006d8a0 0x7fd71006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:01.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.606+0000 7fd70ffff700 1 -- 172.21.15.88:0/739195451 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd71808d160 con 0x7fd7280ffd60 2024-02-19T19:28:01.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.606+0000 7fd726ffd700 1 --2- 172.21.15.88:0/739195451 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd71006d8a0 0x7fd71006fd50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00b5c0 tx=0x7fd71c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:01.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.611+0000 7fd70ffff700 1 -- 172.21.15.88:0/739195451 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd718056910 con 0x7fd7280ffd60 2024-02-19T19:28:01.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.891+0000 7fd72d6e1700 1 -- 172.21.15.88:0/739195451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd72804f2a0 con 0x7fd7280ffd60 2024-02-19T19:28:01.897 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:01 smithi088 ceph-mon[88643]: pgmap v734: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:01.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.891+0000 7fd70ffff700 1 -- 172.21.15.88:0/739195451 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd718059fd0 con 0x7fd7280ffd60 2024-02-19T19:28:01.897 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:28:01.897 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:28:01.897 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:28:01.897 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:28:01.898 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:28:01.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.893+0000 7fd72d6e1700 1 -- 172.21.15.88:0/739195451 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd71006d8a0 msgr2=0x7fd71006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:01.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.893+0000 7fd72d6e1700 1 --2- 172.21.15.88:0/739195451 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd71006d8a0 0x7fd71006fd50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00b5c0 tx=0x7fd71c01a040 comp rx=0 tx=0).stop 2024-02-19T19:28:01.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.893+0000 7fd72d6e1700 1 -- 172.21.15.88:0/739195451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7280ffd60 msgr2=0x7fd728197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:01.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.893+0000 7fd72d6e1700 1 --2- 172.21.15.88:0/739195451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7280ffd60 0x7fd728197320 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd71800b480 tx=0x7fd71800b840 comp rx=0 tx=0).stop 2024-02-19T19:28:01.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.893+0000 7fd72d6e1700 1 -- 172.21.15.88:0/739195451 shutdown_connections 2024-02-19T19:28:01.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.893+0000 7fd72d6e1700 1 --2- 172.21.15.88:0/739195451 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd71006d8a0 0x7fd71006fd50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.893+0000 7fd72d6e1700 1 --2- 172.21.15.88:0/739195451 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7280ff3c0 0x7fd728196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.893+0000 7fd72d6e1700 1 --2- 172.21.15.88:0/739195451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7280ffd60 0x7fd728197320 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:01.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.893+0000 7fd72d6e1700 1 -- 172.21.15.88:0/739195451 >> 172.21.15.88:0/739195451 conn(0x7fd7280fac80 msgr2=0x7fd7280fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:01.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.893+0000 7fd72d6e1700 1 -- 172.21.15.88:0/739195451 shutdown_connections 2024-02-19T19:28:01.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.894+0000 7fd72d6e1700 1 -- 172.21.15.88:0/739195451 wait complete. 2024-02-19T19:28:01.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:01 smithi132 ceph-mon[70676]: pgmap v734: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:02.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.997+0000 7f543df36700 1 -- 172.21.15.88:0/18378156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5438101420 msgr2=0x7f54381017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.997+0000 7f543df36700 1 --2- 172.21.15.88:0/18378156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5438101420 0x7f54381017f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5428009910 tx=0x7f5428009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:02.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.998+0000 7f543df36700 1 -- 172.21.15.88:0/18378156 shutdown_connections 2024-02-19T19:28:02.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.998+0000 7f543df36700 1 --2- 172.21.15.88:0/18378156 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5438101d30 0x7f543810a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.998+0000 7f543df36700 1 --2- 172.21.15.88:0/18378156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5438101420 0x7f54381017f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.998+0000 7f543df36700 1 -- 172.21.15.88:0/18378156 >> 172.21.15.88:0/18378156 conn(0x7f54380fac80 msgr2=0x7f54380fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:02.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.998+0000 7f543df36700 1 -- 172.21.15.88:0/18378156 shutdown_connections 2024-02-19T19:28:02.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.998+0000 7f543df36700 1 -- 172.21.15.88:0/18378156 wait complete. 2024-02-19T19:28:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.998+0000 7f543df36700 1 Processor -- start 2024-02-19T19:28:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.998+0000 7f543df36700 1 -- start start 2024-02-19T19:28:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f543df36700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5438101420 0x7f5438195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f543df36700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5438101d30 0x7f54381963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f543df36700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5438196a00 con 0x7f5438101420 2024-02-19T19:28:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f543df36700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5438196b40 con 0x7f5438101d30 2024-02-19T19:28:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f54377fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5438101420 0x7f5438195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f54377fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5438101420 0x7f5438195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35180/0 (socket says 172.21.15.88:35180) 2024-02-19T19:28:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f54377fe700 1 -- 172.21.15.88:0/28952931 learned_addr learned my addr 172.21.15.88:0/28952931 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:02.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f54377fe700 1 -- 172.21.15.88:0/28952931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5438101d30 msgr2=0x7f54381963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f5436ffd700 1 --2- 172.21.15.88:0/28952931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5438101d30 0x7f54381963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:02.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f54377fe700 1 --2- 172.21.15.88:0/28952931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5438101d30 0x7f54381963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f54377fe700 1 -- 172.21.15.88:0/28952931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54280095f0 con 0x7f5438101420 2024-02-19T19:28:02.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f54377fe700 1 --2- 172.21.15.88:0/28952931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5438101420 0x7f5438195e70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f542800b5c0 tx=0x7f542800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:02.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f5434ff9700 1 -- 172.21.15.88:0/28952931 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f542801d070 con 0x7f5438101420 2024-02-19T19:28:02.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f543df36700 1 -- 172.21.15.88:0/28952931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543819a930 con 0x7f5438101420 2024-02-19T19:28:02.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:01.999+0000 7f5434ff9700 1 -- 172.21.15.88:0/28952931 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f542800f460 con 0x7f5438101420 2024-02-19T19:28:02.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.000+0000 7f5434ff9700 1 -- 172.21.15.88:0/28952931 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5428017610 con 0x7f5438101420 2024-02-19T19:28:02.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.000+0000 7f543df36700 1 -- 172.21.15.88:0/28952931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543819ae20 con 0x7f5438101420 2024-02-19T19:28:02.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.001+0000 7f543df36700 1 -- 172.21.15.88:0/28952931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5438107900 con 0x7f5438101420 2024-02-19T19:28:02.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.001+0000 7f5434ff9700 1 -- 172.21.15.88:0/28952931 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f542800f5d0 con 0x7f5438101420 2024-02-19T19:28:02.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.001+0000 7f5434ff9700 1 --2- 172.21.15.88:0/28952931 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542006d8a0 0x7f542006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:02.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.001+0000 7f5434ff9700 1 -- 172.21.15.88:0/28952931 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f542808eb50 con 0x7f5438101420 2024-02-19T19:28:02.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.001+0000 7f5436ffd700 1 --2- 172.21.15.88:0/28952931 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542006d8a0 0x7f542006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:02.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.002+0000 7f5436ffd700 1 --2- 172.21.15.88:0/28952931 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542006d8a0 0x7f542006fd50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5438197490 tx=0x7f542c005d50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:02.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.006+0000 7f5434ff9700 1 -- 172.21.15.88:0/28952931 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5428058300 con 0x7f5438101420 2024-02-19T19:28:02.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.246+0000 7f543df36700 1 -- 172.21.15.88:0/28952931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f54381072e0 con 0x7f5438101420 2024-02-19T19:28:02.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.247+0000 7f5434ff9700 1 -- 172.21.15.88:0/28952931 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f542805b9c0 con 0x7f5438101420 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:28:02.253 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:28:02.254 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:28:02.255 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:28:02.255 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:28:02.255 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:28:02.255 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:28:02.255 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:28:02.255 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:28:02.255 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:28:02.255 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:28:02.255 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:28:02.255 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:28:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.250+0000 7f543df36700 1 -- 172.21.15.88:0/28952931 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542006d8a0 msgr2=0x7f542006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.250+0000 7f543df36700 1 --2- 172.21.15.88:0/28952931 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542006d8a0 0x7f542006fd50 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5438197490 tx=0x7f542c005d50 comp rx=0 tx=0).stop 2024-02-19T19:28:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.250+0000 7f543df36700 1 -- 172.21.15.88:0/28952931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5438101420 msgr2=0x7f5438195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.250+0000 7f543df36700 1 --2- 172.21.15.88:0/28952931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5438101420 0x7f5438195e70 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f542800b5c0 tx=0x7f542800be00 comp rx=0 tx=0).stop 2024-02-19T19:28:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.250+0000 7f543df36700 1 -- 172.21.15.88:0/28952931 shutdown_connections 2024-02-19T19:28:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.250+0000 7f543df36700 1 --2- 172.21.15.88:0/28952931 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f542006d8a0 0x7f542006fd50 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.250+0000 7f543df36700 1 --2- 172.21.15.88:0/28952931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5438101420 0x7f5438195e70 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.250+0000 7f543df36700 1 --2- 172.21.15.88:0/28952931 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5438101d30 0x7f54381963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.250+0000 7f543df36700 1 -- 172.21.15.88:0/28952931 >> 172.21.15.88:0/28952931 conn(0x7f54380fac80 msgr2=0x7f5438104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:02.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.250+0000 7f543df36700 1 -- 172.21.15.88:0/28952931 shutdown_connections 2024-02-19T19:28:02.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.250+0000 7f543df36700 1 -- 172.21.15.88:0/28952931 wait complete. 2024-02-19T19:28:02.257 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:28:02.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.354+0000 7f36acfe9700 1 -- 172.21.15.88:0/1652628810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36a80ffb20 msgr2=0x7f36a8104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.354+0000 7f36acfe9700 1 --2- 172.21.15.88:0/1652628810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36a80ffb20 0x7f36a8104000 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f369c009910 tx=0x7f369c009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:02.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.355+0000 7f36acfe9700 1 -- 172.21.15.88:0/1652628810 shutdown_connections 2024-02-19T19:28:02.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.355+0000 7f36acfe9700 1 --2- 172.21.15.88:0/1652628810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36a80ffb20 0x7f36a8104000 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.355+0000 7f36acfe9700 1 --2- 172.21.15.88:0/1652628810 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a80ff210 0x7f36a80ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.355+0000 7f36acfe9700 1 -- 172.21.15.88:0/1652628810 >> 172.21.15.88:0/1652628810 conn(0x7f36a80fac80 msgr2=0x7f36a80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:02.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.355+0000 7f36acfe9700 1 -- 172.21.15.88:0/1652628810 shutdown_connections 2024-02-19T19:28:02.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.355+0000 7f36acfe9700 1 -- 172.21.15.88:0/1652628810 wait complete. 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.355+0000 7f36acfe9700 1 Processor -- start 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.355+0000 7f36acfe9700 1 -- start start 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36acfe9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36a80ff210 0x7f36a819c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36acfe9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a80ffb20 0x7f36a819cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36acfe9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a819d1e0 con 0x7f36a80ff210 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36acfe9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a81a0f70 con 0x7f36a80ffb20 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36a659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36a80ff210 0x7f36a819c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36a659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36a80ff210 0x7f36a819c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35198/0 (socket says 172.21.15.88:35198) 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36a659c700 1 -- 172.21.15.88:0/1440570140 learned_addr learned my addr 172.21.15.88:0/1440570140 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36a5d9b700 1 --2- 172.21.15.88:0/1440570140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a80ffb20 0x7f36a819cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36a659c700 1 -- 172.21.15.88:0/1440570140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a80ffb20 msgr2=0x7f36a819cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36a659c700 1 --2- 172.21.15.88:0/1440570140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a80ffb20 0x7f36a819cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36a659c700 1 -- 172.21.15.88:0/1440570140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c0095f0 con 0x7f36a80ff210 2024-02-19T19:28:02.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36a659c700 1 --2- 172.21.15.88:0/1440570140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36a80ff210 0x7f36a819c5c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f369800b4b0 tx=0x7f369800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:02.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36977fe700 1 -- 172.21.15.88:0/1440570140 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3698011800 con 0x7f36a80ff210 2024-02-19T19:28:02.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36977fe700 1 -- 172.21.15.88:0/1440570140 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3698011e20 con 0x7f36a80ff210 2024-02-19T19:28:02.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36977fe700 1 -- 172.21.15.88:0/1440570140 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f369800f2e0 con 0x7f36a80ff210 2024-02-19T19:28:02.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36acfe9700 1 -- 172.21.15.88:0/1440570140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a81a1250 con 0x7f36a80ff210 2024-02-19T19:28:02.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.356+0000 7f36acfe9700 1 -- 172.21.15.88:0/1440570140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a81a17a0 con 0x7f36a80ff210 2024-02-19T19:28:02.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.357+0000 7f36acfe9700 1 -- 172.21.15.88:0/1440570140 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a8101690 con 0x7f36a80ff210 2024-02-19T19:28:02.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.358+0000 7f36977fe700 1 -- 172.21.15.88:0/1440570140 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3698010380 con 0x7f36a80ff210 2024-02-19T19:28:02.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.358+0000 7f36977fe700 1 --2- 172.21.15.88:0/1440570140 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f369006d8a0 0x7f369006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:02.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.358+0000 7f36977fe700 1 -- 172.21.15.88:0/1440570140 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f369808d070 con 0x7f36a80ff210 2024-02-19T19:28:02.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.358+0000 7f36a5d9b700 1 --2- 172.21.15.88:0/1440570140 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f369006d8a0 0x7f369006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:02.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.359+0000 7f36a5d9b700 1 --2- 172.21.15.88:0/1440570140 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f369006d8a0 0x7f369006fd50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f36a819dbe0 tx=0x7f369c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:02.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.363+0000 7f36977fe700 1 -- 172.21.15.88:0/1440570140 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3698056820 con 0x7f36a80ff210 2024-02-19T19:28:02.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.588+0000 7f36acfe9700 1 -- 172.21.15.88:0/1440570140 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36a819da50 con 0x7f369006d8a0 2024-02-19T19:28:02.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.589+0000 7f36977fe700 1 -- 172.21.15.88:0/1440570140 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f36a819da50 con 0x7f369006d8a0 2024-02-19T19:28:02.595 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:28:02.595 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:28:02.595 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:28:02.595 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:28:02.595 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:28:02.595 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:28:02.595 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:28:02.596 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:28:02.596 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:28:02.596 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:28:02.596 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:28:02.596 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:28:02.596 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:28:02.596 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:28:02.596 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:28:02.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.591+0000 7f36acfe9700 1 -- 172.21.15.88:0/1440570140 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f369006d8a0 msgr2=0x7f369006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.591+0000 7f36acfe9700 1 --2- 172.21.15.88:0/1440570140 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f369006d8a0 0x7f369006fd50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f36a819dbe0 tx=0x7f369c01a040 comp rx=0 tx=0).stop 2024-02-19T19:28:02.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.591+0000 7f36acfe9700 1 -- 172.21.15.88:0/1440570140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36a80ff210 msgr2=0x7f36a819c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.591+0000 7f36acfe9700 1 --2- 172.21.15.88:0/1440570140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36a80ff210 0x7f36a819c5c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f369800b4b0 tx=0x7f369800b7c0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.591+0000 7f36acfe9700 1 -- 172.21.15.88:0/1440570140 shutdown_connections 2024-02-19T19:28:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.591+0000 7f36acfe9700 1 --2- 172.21.15.88:0/1440570140 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f369006d8a0 0x7f369006fd50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.591+0000 7f36acfe9700 1 --2- 172.21.15.88:0/1440570140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36a80ff210 0x7f36a819c5c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.591+0000 7f36acfe9700 1 --2- 172.21.15.88:0/1440570140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36a80ffb20 0x7f36a819cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.591+0000 7f36acfe9700 1 -- 172.21.15.88:0/1440570140 >> 172.21.15.88:0/1440570140 conn(0x7f36a80fac80 msgr2=0x7f36a8110680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.591+0000 7f36acfe9700 1 -- 172.21.15.88:0/1440570140 shutdown_connections 2024-02-19T19:28:02.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.591+0000 7f36acfe9700 1 -- 172.21.15.88:0/1440570140 wait complete. 2024-02-19T19:28:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.696+0000 7f1cc1ada700 1 -- 172.21.15.88:0/324201487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc1004e0 msgr2=0x7f1cbc1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.696+0000 7f1cc1ada700 1 --2- 172.21.15.88:0/324201487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc1004e0 0x7f1cbc1008b0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009910 tx=0x7f1cac009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.696+0000 7f1cc1ada700 1 -- 172.21.15.88:0/324201487 shutdown_connections 2024-02-19T19:28:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.696+0000 7f1cc1ada700 1 --2- 172.21.15.88:0/324201487 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cbc100e80 0x7f1cbc104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.696+0000 7f1cc1ada700 1 --2- 172.21.15.88:0/324201487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc1004e0 0x7f1cbc1008b0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.696+0000 7f1cc1ada700 1 -- 172.21.15.88:0/324201487 >> 172.21.15.88:0/324201487 conn(0x7f1cbc0fbda0 msgr2=0x7f1cbc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.696+0000 7f1cc1ada700 1 -- 172.21.15.88:0/324201487 shutdown_connections 2024-02-19T19:28:02.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.696+0000 7f1cc1ada700 1 -- 172.21.15.88:0/324201487 wait complete. 2024-02-19T19:28:02.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.696+0000 7f1cc1ada700 1 Processor -- start 2024-02-19T19:28:02.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.696+0000 7f1cc1ada700 1 -- start start 2024-02-19T19:28:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cc1ada700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc1004e0 0x7f1cbc195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cc1ada700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cbc100e80 0x7f1cbc196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cc1ada700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc196b30 con 0x7f1cbc1004e0 2024-02-19T19:28:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cc1ada700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc19a8c0 con 0x7f1cbc100e80 2024-02-19T19:28:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cbb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc1004e0 0x7f1cbc195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cbb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc1004e0 0x7f1cbc195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35220/0 (socket says 172.21.15.88:35220) 2024-02-19T19:28:02.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cbb7fe700 1 -- 172.21.15.88:0/2615365990 learned_addr learned my addr 172.21.15.88:0/2615365990 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cbb7fe700 1 -- 172.21.15.88:0/2615365990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cbc100e80 msgr2=0x7f1cbc196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cb2ffd700 1 --2- 172.21.15.88:0/2615365990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cbc100e80 0x7f1cbc196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cbb7fe700 1 --2- 172.21.15.88:0/2615365990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cbc100e80 0x7f1cbc196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cbb7fe700 1 -- 172.21.15.88:0/2615365990 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0095f0 con 0x7f1cbc1004e0 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cbb7fe700 1 --2- 172.21.15.88:0/2615365990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc1004e0 0x7f1cbc195f10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1cac006010 tx=0x7f1cac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.697+0000 7f1cb97fa700 1 -- 172.21.15.88:0/2615365990 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cac01d070 con 0x7f1cbc1004e0 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.698+0000 7f1cb97fa700 1 -- 172.21.15.88:0/2615365990 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1cac004fb0 con 0x7f1cbc1004e0 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.698+0000 7f1cb97fa700 1 -- 172.21.15.88:0/2615365990 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cac022ea0 con 0x7f1cbc1004e0 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.698+0000 7f1cc1ada700 1 -- 172.21.15.88:0/2615365990 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc19ab40 con 0x7f1cbc1004e0 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.698+0000 7f1cc1ada700 1 -- 172.21.15.88:0/2615365990 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc19b030 con 0x7f1cbc1004e0 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.698+0000 7f1cc1ada700 1 -- 172.21.15.88:0/2615365990 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc1087f0 con 0x7f1cbc1004e0 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.699+0000 7f1cb97fa700 1 -- 172.21.15.88:0/2615365990 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1cac022470 con 0x7f1cbc1004e0 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.699+0000 7f1cb97fa700 1 --2- 172.21.15.88:0/2615365990 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ca806d8a0 0x7f1ca806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:02.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.699+0000 7f1cb97fa700 1 -- 172.21.15.88:0/2615365990 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1cac08ebb0 con 0x7f1cbc1004e0 2024-02-19T19:28:02.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.699+0000 7f1cb2ffd700 1 --2- 172.21.15.88:0/2615365990 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ca806d8a0 0x7f1ca806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:02.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.700+0000 7f1cb2ffd700 1 --2- 172.21.15.88:0/2615365990 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ca806d8a0 0x7f1ca806fd50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc197530 tx=0x7f1ca4005cf0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:02.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.704+0000 7f1cb97fa700 1 -- 172.21.15.88:0/2615365990 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1cac095080 con 0x7f1cbc1004e0 2024-02-19T19:28:02.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:02 smithi132 ceph-mon[70676]: from='client.35342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:02.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:02 smithi132 ceph-mon[70676]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:02.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:02 smithi132 ceph-mon[70676]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:02.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:02 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/739195451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:28:02.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:02 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/28952931' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:28:02.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.982+0000 7f1cc1ada700 1 -- 172.21.15.88:0/2615365990 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1cbc04f2a0 con 0x7f1cbc1004e0 2024-02-19T19:28:02.988 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:02 smithi088 ceph-mon[88643]: from='client.35342 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:02.988 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:02 smithi088 ceph-mon[88643]: from='client.35346 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:02.989 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:02 smithi088 ceph-mon[88643]: from='client.35350 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:02.989 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:02 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/739195451' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:28:02.989 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:02 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/28952931' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:28:02.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.983+0000 7f1cb97fa700 1 -- 172.21.15.88:0/2615365990 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f1cac027090 con 0x7f1cbc1004e0 2024-02-19T19:28:02.989 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:28:02.989 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:28:02.989 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:28:02.990 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:28:02.991 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:28:02.991 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:28:02.991 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:28:02.991 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:28:02.991 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:28:02.991 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:28:02.991 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:28:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.985+0000 7f1cc1ada700 1 -- 172.21.15.88:0/2615365990 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ca806d8a0 msgr2=0x7f1ca806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.985+0000 7f1cc1ada700 1 --2- 172.21.15.88:0/2615365990 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ca806d8a0 0x7f1ca806fd50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc197530 tx=0x7f1ca4005cf0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.985+0000 7f1cc1ada700 1 -- 172.21.15.88:0/2615365990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc1004e0 msgr2=0x7f1cbc195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.985+0000 7f1cc1ada700 1 --2- 172.21.15.88:0/2615365990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc1004e0 0x7f1cbc195f10 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f1cac006010 tx=0x7f1cac00be20 comp rx=0 tx=0).stop 2024-02-19T19:28:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.985+0000 7f1cc1ada700 1 -- 172.21.15.88:0/2615365990 shutdown_connections 2024-02-19T19:28:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.985+0000 7f1cc1ada700 1 --2- 172.21.15.88:0/2615365990 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1ca806d8a0 0x7f1ca806fd50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.985+0000 7f1cc1ada700 1 --2- 172.21.15.88:0/2615365990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1cbc1004e0 0x7f1cbc195f10 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.985+0000 7f1cc1ada700 1 --2- 172.21.15.88:0/2615365990 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1cbc100e80 0x7f1cbc196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.985+0000 7f1cc1ada700 1 -- 172.21.15.88:0/2615365990 >> 172.21.15.88:0/2615365990 conn(0x7f1cbc0fbda0 msgr2=0x7f1cbc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.986+0000 7f1cc1ada700 1 -- 172.21.15.88:0/2615365990 shutdown_connections 2024-02-19T19:28:02.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:02.986+0000 7f1cc1ada700 1 -- 172.21.15.88:0/2615365990 wait complete. 2024-02-19T19:28:03.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:03 smithi132 ceph-mon[70676]: pgmap v735: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:03.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:03 smithi132 ceph-mon[70676]: from='client.35362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:03.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:03 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2615365990' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:28:04.022 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:03 smithi088 ceph-mon[88643]: pgmap v735: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:04.022 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:03 smithi088 ceph-mon[88643]: from='client.35362 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:04.022 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:03 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2615365990' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:28:05.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:05 smithi132 ceph-mon[70676]: pgmap v736: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:06.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:05 smithi088 ceph-mon[88643]: pgmap v736: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:07.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:07 smithi132 ceph-mon[70676]: pgmap v737: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:08.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:07 smithi088 ceph-mon[88643]: pgmap v737: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:09.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:09 smithi132 ceph-mon[70676]: pgmap v738: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:10.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:09 smithi088 ceph-mon[88643]: pgmap v738: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:11.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:11 smithi132 ceph-mon[70676]: pgmap v739: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:12.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:11 smithi088 ceph-mon[88643]: pgmap v739: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:13.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:13 smithi132 ceph-mon[70676]: pgmap v740: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:13.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:28:14.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:13 smithi088 ceph-mon[88643]: pgmap v740: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:14.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:28:15.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:15 smithi132 ceph-mon[70676]: pgmap v741: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:16.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:15 smithi088 ceph-mon[88643]: pgmap v741: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:17.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:17 smithi132 ceph-mon[70676]: pgmap v742: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:18.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:17 smithi088 ceph-mon[88643]: pgmap v742: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:19.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:19 smithi132 ceph-mon[70676]: pgmap v743: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:20.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:19 smithi088 ceph-mon[88643]: pgmap v743: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:21.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:21 smithi132 ceph-mon[70676]: pgmap v744: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:22.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:21 smithi088 ceph-mon[88643]: pgmap v744: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:23.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:23 smithi132 ceph-mon[70676]: pgmap v745: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:24.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:23 smithi088 ceph-mon[88643]: pgmap v745: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:24.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:24 smithi132 ceph-mon[70676]: pgmap v746: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:25.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:24 smithi088 ceph-mon[88643]: pgmap v746: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:27.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:27 smithi132 ceph-mon[70676]: pgmap v747: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:28.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:27 smithi088 ceph-mon[88643]: pgmap v747: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:28.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:28:29.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:28:29.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:29 smithi132 ceph-mon[70676]: pgmap v748: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:30.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:29 smithi088 ceph-mon[88643]: pgmap v748: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:31.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:31 smithi132 ceph-mon[70676]: pgmap v749: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:32.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:31 smithi088 ceph-mon[88643]: pgmap v749: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:33.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.104+0000 7ff5b8812700 1 -- 172.21.15.88:0/120326577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5b41003b0 msgr2=0x7ff5b4100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:33.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.104+0000 7ff5b8812700 1 --2- 172.21.15.88:0/120326577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5b41003b0 0x7ff5b4100780 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4009910 tx=0x7ff5a4009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.104+0000 7ff5b8812700 1 -- 172.21.15.88:0/120326577 shutdown_connections 2024-02-19T19:28:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.104+0000 7ff5b8812700 1 --2- 172.21.15.88:0/120326577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b4100d50 0x7ff5b4104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.104+0000 7ff5b8812700 1 --2- 172.21.15.88:0/120326577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5b41003b0 0x7ff5b4100780 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.104+0000 7ff5b8812700 1 -- 172.21.15.88:0/120326577 >> 172.21.15.88:0/120326577 conn(0x7ff5b40fbc70 msgr2=0x7ff5b40fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.104+0000 7ff5b8812700 1 -- 172.21.15.88:0/120326577 shutdown_connections 2024-02-19T19:28:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.104+0000 7ff5b8812700 1 -- 172.21.15.88:0/120326577 wait complete. 2024-02-19T19:28:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b8812700 1 Processor -- start 2024-02-19T19:28:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b8812700 1 -- start start 2024-02-19T19:28:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b8812700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5b41003b0 0x7ff5b4198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b8812700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b4100d50 0x7ff5b4198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b8812700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b4198c30 con 0x7ff5b41003b0 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b8812700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b419c9c0 con 0x7ff5b4100d50 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5b41003b0 0x7ff5b4198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5b41003b0 0x7ff5b4198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49014/0 (socket says 172.21.15.88:49014) 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b259c700 1 -- 172.21.15.88:0/487704614 learned_addr learned my addr 172.21.15.88:0/487704614 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b259c700 1 -- 172.21.15.88:0/487704614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b4100d50 msgr2=0x7ff5b4198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b1d9b700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b4100d50 0x7ff5b4198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b259c700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b4100d50 0x7ff5b4198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b259c700 1 -- 172.21.15.88:0/487704614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a40095f0 con 0x7ff5b41003b0 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b1d9b700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b4100d50 0x7ff5b4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.105+0000 7ff5b259c700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5b41003b0 0x7ff5b4198010 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4006010 tx=0x7ff5a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.106+0000 7ff5a37fe700 1 -- 172.21.15.88:0/487704614 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5a401d070 con 0x7ff5b41003b0 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.106+0000 7ff5b8812700 1 -- 172.21.15.88:0/487704614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b419cc40 con 0x7ff5b41003b0 2024-02-19T19:28:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.106+0000 7ff5a37fe700 1 -- 172.21.15.88:0/487704614 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5a4004fb0 con 0x7ff5b41003b0 2024-02-19T19:28:33.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.106+0000 7ff5a37fe700 1 -- 172.21.15.88:0/487704614 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5a4022ea0 con 0x7ff5b41003b0 2024-02-19T19:28:33.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.106+0000 7ff5b8812700 1 -- 172.21.15.88:0/487704614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b419d130 con 0x7ff5b41003b0 2024-02-19T19:28:33.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.107+0000 7ff5b8812700 1 -- 172.21.15.88:0/487704614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b41044a0 con 0x7ff5b41003b0 2024-02-19T19:28:33.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.107+0000 7ff5a37fe700 1 -- 172.21.15.88:0/487704614 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff5a4022470 con 0x7ff5b41003b0 2024-02-19T19:28:33.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.108+0000 7ff5a37fe700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff59c06d8a0 0x7ff59c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:33.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.108+0000 7ff5a37fe700 1 -- 172.21.15.88:0/487704614 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff5a408ec10 con 0x7ff5b41003b0 2024-02-19T19:28:33.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.108+0000 7ff5b1d9b700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff59c06d8a0 0x7ff59c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:33.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.108+0000 7ff5b1d9b700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff59c06d8a0 0x7ff59c06fd50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4199630 tx=0x7ff5a8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:33.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.113+0000 7ff5a37fe700 1 -- 172.21.15.88:0/487704614 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff5a4058590 con 0x7ff5b41003b0 2024-02-19T19:28:33.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.328+0000 7ff5b8812700 1 -- 172.21.15.88:0/487704614 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5b402ccf0 con 0x7ff59c06d8a0 2024-02-19T19:28:33.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.329+0000 7ff5a37fe700 1 -- 172.21.15.88:0/487704614 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff5b402ccf0 con 0x7ff59c06d8a0 2024-02-19T19:28:33.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.332+0000 7ff5b8812700 1 -- 172.21.15.88:0/487704614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff59c06d8a0 msgr2=0x7ff59c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:33.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.332+0000 7ff5b8812700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff59c06d8a0 0x7ff59c06fd50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff5b4199630 tx=0x7ff5a8008040 comp rx=0 tx=0).stop 2024-02-19T19:28:33.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.332+0000 7ff5b8812700 1 -- 172.21.15.88:0/487704614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5b41003b0 msgr2=0x7ff5b4198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:33.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.332+0000 7ff5b8812700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5b41003b0 0x7ff5b4198010 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4006010 tx=0x7ff5a400be20 comp rx=0 tx=0).stop 2024-02-19T19:28:33.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.332+0000 7ff5b8812700 1 -- 172.21.15.88:0/487704614 shutdown_connections 2024-02-19T19:28:33.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.332+0000 7ff5b8812700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff59c06d8a0 0x7ff59c06fd50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.332+0000 7ff5b8812700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5b41003b0 0x7ff5b4198010 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.332+0000 7ff5b8812700 1 --2- 172.21.15.88:0/487704614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5b4100d50 0x7ff5b4198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.332+0000 7ff5b8812700 1 -- 172.21.15.88:0/487704614 >> 172.21.15.88:0/487704614 conn(0x7ff5b40fbc70 msgr2=0x7ff5b40fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:33.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.332+0000 7ff5b8812700 1 -- 172.21.15.88:0/487704614 shutdown_connections 2024-02-19T19:28:33.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.332+0000 7ff5b8812700 1 -- 172.21.15.88:0/487704614 wait complete. 2024-02-19T19:28:33.346 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:28:33.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.436+0000 7f1d68a80700 1 -- 172.21.15.88:0/3893530056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d64100e80 msgr2=0x7f1d64104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.436+0000 7f1d68a80700 1 --2- 172.21.15.88:0/3893530056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d64100e80 0x7f1d64104e50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009910 tx=0x7f1d58009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.436+0000 7f1d68a80700 1 -- 172.21.15.88:0/3893530056 shutdown_connections 2024-02-19T19:28:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.436+0000 7f1d68a80700 1 --2- 172.21.15.88:0/3893530056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d64100e80 0x7f1d64104e50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.436+0000 7f1d68a80700 1 --2- 172.21.15.88:0/3893530056 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d641004e0 0x7f1d641008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.436+0000 7f1d68a80700 1 -- 172.21.15.88:0/3893530056 >> 172.21.15.88:0/3893530056 conn(0x7f1d640fbda0 msgr2=0x7f1d640fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.436+0000 7f1d68a80700 1 -- 172.21.15.88:0/3893530056 shutdown_connections 2024-02-19T19:28:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.436+0000 7f1d68a80700 1 -- 172.21.15.88:0/3893530056 wait complete. 2024-02-19T19:28:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d68a80700 1 Processor -- start 2024-02-19T19:28:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d68a80700 1 -- start start 2024-02-19T19:28:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d68a80700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d641004e0 0x7f1d64195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d68a80700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64100e80 0x7f1d64196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d68a80700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d64196b50 con 0x7f1d641004e0 2024-02-19T19:28:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d68a80700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6419a8e0 con 0x7f1d64100e80 2024-02-19T19:28:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d6259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d641004e0 0x7f1d64195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d6259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d641004e0 0x7f1d64195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49036/0 (socket says 172.21.15.88:49036) 2024-02-19T19:28:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d6259c700 1 -- 172.21.15.88:0/2899619643 learned_addr learned my addr 172.21.15.88:0/2899619643 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d61d9b700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64100e80 0x7f1d64196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d6259c700 1 -- 172.21.15.88:0/2899619643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64100e80 msgr2=0x7f1d64196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d6259c700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64100e80 0x7f1d64196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d6259c700 1 -- 172.21.15.88:0/2899619643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d580095f0 con 0x7f1d641004e0 2024-02-19T19:28:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.437+0000 7f1d61d9b700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64100e80 0x7f1d64196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:28:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.438+0000 7f1d6259c700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d641004e0 0x7f1d64195f30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c007e90 tx=0x7f1d5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.438+0000 7f1d537fe700 1 -- 172.21.15.88:0/2899619643 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d5c00de60 con 0x7f1d641004e0 2024-02-19T19:28:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.438+0000 7f1d537fe700 1 -- 172.21.15.88:0/2899619643 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d5c00f040 con 0x7f1d641004e0 2024-02-19T19:28:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.438+0000 7f1d68a80700 1 -- 172.21.15.88:0/2899619643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d6419abc0 con 0x7f1d641004e0 2024-02-19T19:28:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.438+0000 7f1d537fe700 1 -- 172.21.15.88:0/2899619643 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d5c0045b0 con 0x7f1d641004e0 2024-02-19T19:28:33.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.438+0000 7f1d68a80700 1 -- 172.21.15.88:0/2899619643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d6419b110 con 0x7f1d641004e0 2024-02-19T19:28:33.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.439+0000 7f1d68a80700 1 -- 172.21.15.88:0/2899619643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d64108850 con 0x7f1d641004e0 2024-02-19T19:28:33.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.440+0000 7f1d537fe700 1 -- 172.21.15.88:0/2899619643 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1d5c004ad0 con 0x7f1d641004e0 2024-02-19T19:28:33.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.440+0000 7f1d537fe700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d4c06d8a0 0x7f1d4c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:33.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.440+0000 7f1d537fe700 1 -- 172.21.15.88:0/2899619643 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1d5c08d0d0 con 0x7f1d641004e0 2024-02-19T19:28:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.440+0000 7f1d61d9b700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d4c06d8a0 0x7f1d4c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.441+0000 7f1d61d9b700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d4c06d8a0 0x7f1d4c06fd50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1d64197550 tx=0x7f1d5801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:33.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.445+0000 7f1d537fe700 1 -- 172.21.15.88:0/2899619643 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1d5c056880 con 0x7f1d641004e0 2024-02-19T19:28:33.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.663+0000 7f1d68a80700 1 -- 172.21.15.88:0/2899619643 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d6419b330 con 0x7f1d4c06d8a0 2024-02-19T19:28:33.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.665+0000 7f1d537fe700 1 -- 172.21.15.88:0/2899619643 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1d6419b330 con 0x7f1d4c06d8a0 2024-02-19T19:28:33.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.667+0000 7f1d68a80700 1 -- 172.21.15.88:0/2899619643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d4c06d8a0 msgr2=0x7f1d4c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:33.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.667+0000 7f1d68a80700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d4c06d8a0 0x7f1d4c06fd50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1d64197550 tx=0x7f1d5801a040 comp rx=0 tx=0).stop 2024-02-19T19:28:33.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.667+0000 7f1d68a80700 1 -- 172.21.15.88:0/2899619643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d641004e0 msgr2=0x7f1d64195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:33.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.667+0000 7f1d68a80700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d641004e0 0x7f1d64195f30 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c007e90 tx=0x7f1d5c00d3b0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.667+0000 7f1d68a80700 1 -- 172.21.15.88:0/2899619643 shutdown_connections 2024-02-19T19:28:33.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.667+0000 7f1d68a80700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1d4c06d8a0 0x7f1d4c06fd50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.667+0000 7f1d68a80700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d641004e0 0x7f1d64195f30 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.667+0000 7f1d68a80700 1 --2- 172.21.15.88:0/2899619643 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1d64100e80 0x7f1d64196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.667+0000 7f1d68a80700 1 -- 172.21.15.88:0/2899619643 >> 172.21.15.88:0/2899619643 conn(0x7f1d640fbda0 msgr2=0x7f1d640fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:33.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.667+0000 7f1d68a80700 1 -- 172.21.15.88:0/2899619643 shutdown_connections 2024-02-19T19:28:33.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.667+0000 7f1d68a80700 1 -- 172.21.15.88:0/2899619643 wait complete. 2024-02-19T19:28:33.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.771+0000 7fca6774d700 1 -- 172.21.15.88:0/2958929234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca600ffbd0 msgr2=0x7fca60100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:33.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.771+0000 7fca6774d700 1 --2- 172.21.15.88:0/2958929234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca600ffbd0 0x7fca60100040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009910 tx=0x7fca5c009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:33.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.771+0000 7fca6774d700 1 -- 172.21.15.88:0/2958929234 shutdown_connections 2024-02-19T19:28:33.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.771+0000 7fca6774d700 1 --2- 172.21.15.88:0/2958929234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca600ffbd0 0x7fca60100040 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.771+0000 7fca6774d700 1 --2- 172.21.15.88:0/2958929234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca60104130 0x7fca60104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.771+0000 7fca6774d700 1 -- 172.21.15.88:0/2958929234 >> 172.21.15.88:0/2958929234 conn(0x7fca60075130 msgr2=0x7fca60075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:33.776 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:33 smithi088 ceph-mon[88643]: pgmap v750: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:33.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.771+0000 7fca6774d700 1 -- 172.21.15.88:0/2958929234 shutdown_connections 2024-02-19T19:28:33.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.771+0000 7fca6774d700 1 -- 172.21.15.88:0/2958929234 wait complete. 2024-02-19T19:28:33.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca6774d700 1 Processor -- start 2024-02-19T19:28:33.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca6774d700 1 -- start start 2024-02-19T19:28:33.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca6774d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca600ffbd0 0x7fca60100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca6774d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca60104130 0x7fca60101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca6774d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca601018b0 con 0x7fca60104130 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca6774d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca601019f0 con 0x7fca600ffbd0 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca64ce8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca60104130 0x7fca60101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca64ce8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca60104130 0x7fca60101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49048/0 (socket says 172.21.15.88:49048) 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca64ce8700 1 -- 172.21.15.88:0/4223918558 learned_addr learned my addr 172.21.15.88:0/4223918558 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca654e9700 1 --2- 172.21.15.88:0/4223918558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca600ffbd0 0x7fca60100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca64ce8700 1 -- 172.21.15.88:0/4223918558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca600ffbd0 msgr2=0x7fca60100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca64ce8700 1 --2- 172.21.15.88:0/4223918558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca600ffbd0 0x7fca60100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.772+0000 7fca64ce8700 1 -- 172.21.15.88:0/4223918558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c0095f0 con 0x7fca60104130 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.773+0000 7fca64ce8700 1 --2- 172.21.15.88:0/4223918558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca60104130 0x7fca60101370 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fca5c006010 tx=0x7fca5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:33.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.773+0000 7fca567fc700 1 -- 172.21.15.88:0/4223918558 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca5c01d070 con 0x7fca60104130 2024-02-19T19:28:33.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.773+0000 7fca6774d700 1 -- 172.21.15.88:0/4223918558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca60101c70 con 0x7fca60104130 2024-02-19T19:28:33.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.773+0000 7fca567fc700 1 -- 172.21.15.88:0/4223918558 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca5c00f460 con 0x7fca60104130 2024-02-19T19:28:33.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.773+0000 7fca567fc700 1 -- 172.21.15.88:0/4223918558 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca5c017630 con 0x7fca60104130 2024-02-19T19:28:33.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.773+0000 7fca6774d700 1 -- 172.21.15.88:0/4223918558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca601a2430 con 0x7fca60104130 2024-02-19T19:28:33.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.774+0000 7fca6774d700 1 -- 172.21.15.88:0/4223918558 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca44005320 con 0x7fca60104130 2024-02-19T19:28:33.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.774+0000 7fca567fc700 1 -- 172.21.15.88:0/4223918558 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fca5c003a60 con 0x7fca60104130 2024-02-19T19:28:33.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.775+0000 7fca567fc700 1 --2- 172.21.15.88:0/4223918558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fca4c06d8a0 0x7fca4c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:33.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.775+0000 7fca567fc700 1 -- 172.21.15.88:0/4223918558 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fca5c08e980 con 0x7fca60104130 2024-02-19T19:28:33.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.775+0000 7fca654e9700 1 --2- 172.21.15.88:0/4223918558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fca4c06d8a0 0x7fca4c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:33.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.775+0000 7fca654e9700 1 --2- 172.21.15.88:0/4223918558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fca4c06d8a0 0x7fca4c06fd50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fca50009a10 tx=0x7fca50009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:33.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.779+0000 7fca567fc700 1 -- 172.21.15.88:0/4223918558 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca5c0581f0 con 0x7fca60104130 2024-02-19T19:28:33.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:33 smithi132 ceph-mon[70676]: pgmap v750: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:33.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.986+0000 7fca6774d700 1 -- 172.21.15.88:0/4223918558 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fca44000bf0 con 0x7fca4c06d8a0 2024-02-19T19:28:34.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.997+0000 7fca567fc700 1 -- 172.21.15.88:0/4223918558 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fca44000bf0 con 0x7fca4c06d8a0 2024-02-19T19:28:34.001 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:28:34.001 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (25m) 2m ago 36m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:28:34.001 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (36m) 2m ago 36m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:28:34.001 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (18m) 2m ago 34m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:28:34.001 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (21m) 2m ago 36m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (21m) 2m ago 34m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (25m) 2m ago 35m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (17m) 2m ago 29m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (17m) 2m ago 29m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (17m) 2m ago 29m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (2m) 2m ago 29m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (27m) 2m ago 37m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (26m) 2m ago 34m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (23m) 2m ago 38m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (23m) 2m ago 34m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (26m) 2m ago 36m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (26m) 2m ago 34m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (21m) 2m ago 33m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:28:34.002 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (20m) 2m ago 33m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:28:34.003 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (19m) 2m ago 32m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:28:34.003 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (19m) 2m ago 32m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:28:34.003 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (18m) 2m ago 31m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:28:34.003 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (18m) 2m ago 31m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:28:34.003 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (26m) 2m ago 35m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:28:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.999+0000 7fca6774d700 1 -- 172.21.15.88:0/4223918558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fca4c06d8a0 msgr2=0x7fca4c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.999+0000 7fca6774d700 1 --2- 172.21.15.88:0/4223918558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fca4c06d8a0 0x7fca4c06fd50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fca50009a10 tx=0x7fca50009190 comp rx=0 tx=0).stop 2024-02-19T19:28:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.999+0000 7fca6774d700 1 -- 172.21.15.88:0/4223918558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca60104130 msgr2=0x7fca60101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.999+0000 7fca6774d700 1 --2- 172.21.15.88:0/4223918558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca60104130 0x7fca60101370 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fca5c006010 tx=0x7fca5c00bd70 comp rx=0 tx=0).stop 2024-02-19T19:28:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.999+0000 7fca6774d700 1 -- 172.21.15.88:0/4223918558 shutdown_connections 2024-02-19T19:28:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.999+0000 7fca6774d700 1 --2- 172.21.15.88:0/4223918558 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fca4c06d8a0 0x7fca4c06fd50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.999+0000 7fca6774d700 1 --2- 172.21.15.88:0/4223918558 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fca600ffbd0 0x7fca60100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.999+0000 7fca6774d700 1 --2- 172.21.15.88:0/4223918558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca60104130 0x7fca60101370 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.999+0000 7fca6774d700 1 -- 172.21.15.88:0/4223918558 >> 172.21.15.88:0/4223918558 conn(0x7fca60075130 msgr2=0x7fca60068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.999+0000 7fca6774d700 1 -- 172.21.15.88:0/4223918558 shutdown_connections 2024-02-19T19:28:34.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:33.999+0000 7fca6774d700 1 -- 172.21.15.88:0/4223918558 wait complete. 2024-02-19T19:28:34.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.102+0000 7efeeb003700 1 -- 172.21.15.88:0/1840200269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee40ff3c0 msgr2=0x7efee40ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.102+0000 7efeeb003700 1 --2- 172.21.15.88:0/1840200269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee40ff3c0 0x7efee40ff790 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7efed8009910 tx=0x7efed8009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:34.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.102+0000 7efeeb003700 1 -- 172.21.15.88:0/1840200269 shutdown_connections 2024-02-19T19:28:34.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.102+0000 7efeeb003700 1 --2- 172.21.15.88:0/1840200269 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee40ffd60 0x7efee4103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.102+0000 7efeeb003700 1 --2- 172.21.15.88:0/1840200269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee40ff3c0 0x7efee40ff790 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.102+0000 7efeeb003700 1 -- 172.21.15.88:0/1840200269 >> 172.21.15.88:0/1840200269 conn(0x7efee40fac80 msgr2=0x7efee40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.102+0000 7efeeb003700 1 -- 172.21.15.88:0/1840200269 shutdown_connections 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.102+0000 7efeeb003700 1 -- 172.21.15.88:0/1840200269 wait complete. 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efeeb003700 1 Processor -- start 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efeeb003700 1 -- start start 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efeeb003700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee40ff3c0 0x7efee4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efeeb003700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee40ffd60 0x7efee41985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efeeb003700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee4198c80 con 0x7efee40ff3c0 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efeeb003700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee419ca10 con 0x7efee40ffd60 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efee8d9f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee40ff3c0 0x7efee4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efee8d9f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee40ff3c0 0x7efee4198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49070/0 (socket says 172.21.15.88:49070) 2024-02-19T19:28:34.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efee8d9f700 1 -- 172.21.15.88:0/3495029388 learned_addr learned my addr 172.21.15.88:0/3495029388 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efee8d9f700 1 -- 172.21.15.88:0/3495029388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee40ffd60 msgr2=0x7efee41985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efee3fff700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee40ffd60 0x7efee41985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efee8d9f700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee40ffd60 0x7efee41985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efee8d9f700 1 -- 172.21.15.88:0/3495029388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80095f0 con 0x7efee40ff3c0 2024-02-19T19:28:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efee3fff700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee40ffd60 0x7efee41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:28:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efee8d9f700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee40ff3c0 0x7efee4198060 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7efed8006010 tx=0x7efed800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.103+0000 7efee1ffb700 1 -- 172.21.15.88:0/3495029388 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efed801d070 con 0x7efee40ff3c0 2024-02-19T19:28:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.104+0000 7efee1ffb700 1 -- 172.21.15.88:0/3495029388 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efed8004fb0 con 0x7efee40ff3c0 2024-02-19T19:28:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.104+0000 7efeeb003700 1 -- 172.21.15.88:0/3495029388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee419cc90 con 0x7efee40ff3c0 2024-02-19T19:28:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.104+0000 7efee1ffb700 1 -- 172.21.15.88:0/3495029388 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efed8022ea0 con 0x7efee40ff3c0 2024-02-19T19:28:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.104+0000 7efeeb003700 1 -- 172.21.15.88:0/3495029388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee419d180 con 0x7efee40ff3c0 2024-02-19T19:28:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.105+0000 7efeeb003700 1 -- 172.21.15.88:0/3495029388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee404f2a0 con 0x7efee40ff3c0 2024-02-19T19:28:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.105+0000 7efee1ffb700 1 -- 172.21.15.88:0/3495029388 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7efed8022470 con 0x7efee40ff3c0 2024-02-19T19:28:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.106+0000 7efee1ffb700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efed406d8a0 0x7efed406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.106+0000 7efee1ffb700 1 -- 172.21.15.88:0/3495029388 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efed808ebb0 con 0x7efee40ff3c0 2024-02-19T19:28:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.106+0000 7efee3fff700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efed406d8a0 0x7efed406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.106+0000 7efee3fff700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efed406d8a0 0x7efed406fd50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efed0005fd0 tx=0x7efed0005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:34.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.110+0000 7efee1ffb700 1 -- 172.21.15.88:0/3495029388 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efed8095080 con 0x7efee40ff3c0 2024-02-19T19:28:34.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.395+0000 7efeeb003700 1 -- 172.21.15.88:0/3495029388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efee419d460 con 0x7efee40ff3c0 2024-02-19T19:28:34.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.396+0000 7efee1ffb700 1 -- 172.21.15.88:0/3495029388 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7efed8027280 con 0x7efee40ff3c0 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:28:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:28:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:28:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:28:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:28:34.402 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:28:34.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.399+0000 7efeeb003700 1 -- 172.21.15.88:0/3495029388 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efed406d8a0 msgr2=0x7efed406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.399+0000 7efeeb003700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efed406d8a0 0x7efed406fd50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efed0005fd0 tx=0x7efed0005dc0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.399+0000 7efeeb003700 1 -- 172.21.15.88:0/3495029388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee40ff3c0 msgr2=0x7efee4198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.399+0000 7efeeb003700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee40ff3c0 0x7efee4198060 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7efed8006010 tx=0x7efed800be20 comp rx=0 tx=0).stop 2024-02-19T19:28:34.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.399+0000 7efeeb003700 1 -- 172.21.15.88:0/3495029388 shutdown_connections 2024-02-19T19:28:34.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.399+0000 7efeeb003700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7efed406d8a0 0x7efed406fd50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.399+0000 7efeeb003700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee40ff3c0 0x7efee4198060 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.399+0000 7efeeb003700 1 --2- 172.21.15.88:0/3495029388 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7efee40ffd60 0x7efee41985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.399+0000 7efeeb003700 1 -- 172.21.15.88:0/3495029388 >> 172.21.15.88:0/3495029388 conn(0x7efee40fac80 msgr2=0x7efee4102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:34.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.399+0000 7efeeb003700 1 -- 172.21.15.88:0/3495029388 shutdown_connections 2024-02-19T19:28:34.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.399+0000 7efeeb003700 1 -- 172.21.15.88:0/3495029388 wait complete. 2024-02-19T19:28:34.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.505+0000 7fdd7c250700 1 -- 172.21.15.88:0/388028220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd741004a0 msgr2=0x7fdd74100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.505+0000 7fdd7c250700 1 --2- 172.21.15.88:0/388028220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd741004a0 0x7fdd74100870 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009910 tx=0x7fdd6c009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:34.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.505+0000 7fdd7c250700 1 -- 172.21.15.88:0/388028220 shutdown_connections 2024-02-19T19:28:34.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.505+0000 7fdd7c250700 1 --2- 172.21.15.88:0/388028220 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd74100e40 0x7fdd74104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.505+0000 7fdd7c250700 1 --2- 172.21.15.88:0/388028220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd741004a0 0x7fdd74100870 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.505+0000 7fdd7c250700 1 -- 172.21.15.88:0/388028220 >> 172.21.15.88:0/388028220 conn(0x7fdd740fbd40 msgr2=0x7fdd740fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:34.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.505+0000 7fdd7c250700 1 -- 172.21.15.88:0/388028220 shutdown_connections 2024-02-19T19:28:34.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.505+0000 7fdd7c250700 1 -- 172.21.15.88:0/388028220 wait complete. 2024-02-19T19:28:34.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd7c250700 1 Processor -- start 2024-02-19T19:28:34.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd7c250700 1 -- start start 2024-02-19T19:28:34.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd7c250700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd741004a0 0x7fdd74195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:34.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd7c250700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd74100e40 0x7fdd74196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:34.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd7c250700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd74196a60 con 0x7fdd74100e40 2024-02-19T19:28:34.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd7c250700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd74196ba0 con 0x7fdd741004a0 2024-02-19T19:28:34.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd797eb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd74100e40 0x7fdd74196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:34.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd797eb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd74100e40 0x7fdd74196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49086/0 (socket says 172.21.15.88:49086) 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd797eb700 1 -- 172.21.15.88:0/3088639659 learned_addr learned my addr 172.21.15.88:0/3088639659 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd79fec700 1 --2- 172.21.15.88:0/3088639659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd741004a0 0x7fdd74195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd797eb700 1 -- 172.21.15.88:0/3088639659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd741004a0 msgr2=0x7fdd74195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd797eb700 1 --2- 172.21.15.88:0/3088639659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd741004a0 0x7fdd74195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.506+0000 7fdd797eb700 1 -- 172.21.15.88:0/3088639659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0095f0 con 0x7fdd74100e40 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.507+0000 7fdd797eb700 1 --2- 172.21.15.88:0/3088639659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd74100e40 0x7fdd74196410 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c960 tx=0x7fdd6400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.507+0000 7fdd6affd700 1 -- 172.21.15.88:0/3088639659 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd64007a70 con 0x7fdd74100e40 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.507+0000 7fdd6affd700 1 -- 172.21.15.88:0/3088639659 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd64007bd0 con 0x7fdd74100e40 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.507+0000 7fdd7c250700 1 -- 172.21.15.88:0/3088639659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7419a9f0 con 0x7fdd74100e40 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.507+0000 7fdd6affd700 1 -- 172.21.15.88:0/3088639659 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd640186c0 con 0x7fdd74100e40 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.507+0000 7fdd7c250700 1 -- 172.21.15.88:0/3088639659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7419af40 con 0x7fdd74100e40 2024-02-19T19:28:34.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.508+0000 7fdd7c250700 1 -- 172.21.15.88:0/3088639659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd741087b0 con 0x7fdd74100e40 2024-02-19T19:28:34.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.508+0000 7fdd6affd700 1 -- 172.21.15.88:0/3088639659 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdd6400faa0 con 0x7fdd74100e40 2024-02-19T19:28:34.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.509+0000 7fdd6affd700 1 --2- 172.21.15.88:0/3088639659 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd6006d8a0 0x7fdd6006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:34.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.509+0000 7fdd79fec700 1 --2- 172.21.15.88:0/3088639659 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd6006d8a0 0x7fdd6006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:34.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.509+0000 7fdd6affd700 1 -- 172.21.15.88:0/3088639659 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdd6408d560 con 0x7fdd74100e40 2024-02-19T19:28:34.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.509+0000 7fdd79fec700 1 --2- 172.21.15.88:0/3088639659 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd6006d8a0 0x7fdd6006fd50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006010 tx=0x7fdd6c00b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:34.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.513+0000 7fdd6affd700 1 -- 172.21.15.88:0/3088639659 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdd6405a840 con 0x7fdd74100e40 2024-02-19T19:28:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.757+0000 7fdd7c250700 1 -- 172.21.15.88:0/3088639659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdd74108190 con 0x7fdd74100e40 2024-02-19T19:28:34.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.757+0000 7fdd6affd700 1 -- 172.21.15.88:0/3088639659 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fdd6405a3d0 con 0x7fdd74100e40 2024-02-19T19:28:34.762 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:28:34.762 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:28:34.762 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:28:34.762 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:28:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:28:34.762 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:28:34.762 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:28:34.763 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:28:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:28:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:28:34.765 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:28:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:28:34.765 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:28:34.765 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:28:34.765 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:28:34.765 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:28:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.760+0000 7fdd7c250700 1 -- 172.21.15.88:0/3088639659 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd6006d8a0 msgr2=0x7fdd6006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.760+0000 7fdd7c250700 1 --2- 172.21.15.88:0/3088639659 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd6006d8a0 0x7fdd6006fd50 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c006010 tx=0x7fdd6c00b540 comp rx=0 tx=0).stop 2024-02-19T19:28:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.760+0000 7fdd7c250700 1 -- 172.21.15.88:0/3088639659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd74100e40 msgr2=0x7fdd74196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.760+0000 7fdd7c250700 1 --2- 172.21.15.88:0/3088639659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd74100e40 0x7fdd74196410 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400c960 tx=0x7fdd6400cd20 comp rx=0 tx=0).stop 2024-02-19T19:28:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.760+0000 7fdd7c250700 1 -- 172.21.15.88:0/3088639659 shutdown_connections 2024-02-19T19:28:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.760+0000 7fdd7c250700 1 --2- 172.21.15.88:0/3088639659 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdd6006d8a0 0x7fdd6006fd50 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.760+0000 7fdd7c250700 1 --2- 172.21.15.88:0/3088639659 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdd741004a0 0x7fdd74195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.760+0000 7fdd7c250700 1 --2- 172.21.15.88:0/3088639659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd74100e40 0x7fdd74196410 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.760+0000 7fdd7c250700 1 -- 172.21.15.88:0/3088639659 >> 172.21.15.88:0/3088639659 conn(0x7fdd740fbd40 msgr2=0x7fdd740fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.760+0000 7fdd7c250700 1 -- 172.21.15.88:0/3088639659 shutdown_connections 2024-02-19T19:28:34.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.760+0000 7fdd7c250700 1 -- 172.21.15.88:0/3088639659 wait complete. 2024-02-19T19:28:34.767 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:28:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.865+0000 7f16a9371700 1 -- 172.21.15.88:0/109338082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16a41004c0 msgr2=0x7f16a4100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.865+0000 7f16a9371700 1 --2- 172.21.15.88:0/109338082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16a41004c0 0x7f16a4100890 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f1694009910 tx=0x7f1694009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.865+0000 7f16a9371700 1 -- 172.21.15.88:0/109338082 shutdown_connections 2024-02-19T19:28:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.865+0000 7f16a9371700 1 --2- 172.21.15.88:0/109338082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16a4100e60 0x7f16a4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.865+0000 7f16a9371700 1 --2- 172.21.15.88:0/109338082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16a41004c0 0x7f16a4100890 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.865+0000 7f16a9371700 1 -- 172.21.15.88:0/109338082 >> 172.21.15.88:0/109338082 conn(0x7f16a40fbd80 msgr2=0x7f16a40fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.865+0000 7f16a9371700 1 -- 172.21.15.88:0/109338082 shutdown_connections 2024-02-19T19:28:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.865+0000 7f16a9371700 1 -- 172.21.15.88:0/109338082 wait complete. 2024-02-19T19:28:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.866+0000 7f16a9371700 1 Processor -- start 2024-02-19T19:28:34.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.866+0000 7f16a9371700 1 -- start start 2024-02-19T19:28:34.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.866+0000 7f16a9371700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16a41004c0 0x7f16a4195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.866+0000 7f16a9371700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16a4100e60 0x7f16a4196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.866+0000 7f16a9371700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a4196b50 con 0x7f16a41004c0 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.866+0000 7f16a9371700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a419a8e0 con 0x7f16a4100e60 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.866+0000 7f16a2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16a41004c0 0x7f16a4195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.866+0000 7f16a2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16a41004c0 0x7f16a4195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49108/0 (socket says 172.21.15.88:49108) 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.866+0000 7f16a2ffd700 1 -- 172.21.15.88:0/4127875122 learned_addr learned my addr 172.21.15.88:0/4127875122 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.867+0000 7f16a2ffd700 1 -- 172.21.15.88:0/4127875122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16a4100e60 msgr2=0x7f16a4196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.867+0000 7f16a2ffd700 1 --2- 172.21.15.88:0/4127875122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16a4100e60 0x7f16a4196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.867+0000 7f16a2ffd700 1 -- 172.21.15.88:0/4127875122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16940095f0 con 0x7f16a41004c0 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.867+0000 7f16a2ffd700 1 --2- 172.21.15.88:0/4127875122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16a41004c0 0x7f16a4195f30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1694006010 tx=0x7f169400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.867+0000 7f168bfff700 1 -- 172.21.15.88:0/4127875122 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f169401d070 con 0x7f16a41004c0 2024-02-19T19:28:34.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.867+0000 7f16a9371700 1 -- 172.21.15.88:0/4127875122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a419ab60 con 0x7f16a41004c0 2024-02-19T19:28:34.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.867+0000 7f168bfff700 1 -- 172.21.15.88:0/4127875122 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f169400f460 con 0x7f16a41004c0 2024-02-19T19:28:34.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.867+0000 7f168bfff700 1 -- 172.21.15.88:0/4127875122 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1694017630 con 0x7f16a41004c0 2024-02-19T19:28:34.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.867+0000 7f16a9371700 1 -- 172.21.15.88:0/4127875122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a419b050 con 0x7f16a41004c0 2024-02-19T19:28:34.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.868+0000 7f16a9371700 1 -- 172.21.15.88:0/4127875122 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a41087d0 con 0x7f16a41004c0 2024-02-19T19:28:34.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.869+0000 7f168bfff700 1 -- 172.21.15.88:0/4127875122 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f169400f5d0 con 0x7f16a41004c0 2024-02-19T19:28:34.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.869+0000 7f168bfff700 1 --2- 172.21.15.88:0/4127875122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f168c06d8a0 0x7f168c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:34.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.869+0000 7f168bfff700 1 -- 172.21.15.88:0/4127875122 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f169408ea00 con 0x7f16a41004c0 2024-02-19T19:28:34.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.869+0000 7f16a27fc700 1 --2- 172.21.15.88:0/4127875122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f168c06d8a0 0x7f168c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:34.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.870+0000 7f16a27fc700 1 --2- 172.21.15.88:0/4127875122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f168c06d8a0 0x7f168c06fd50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1698005950 tx=0x7f16980058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:34.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:34.874+0000 7f168bfff700 1 -- 172.21.15.88:0/4127875122 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16940581b0 con 0x7f16a41004c0 2024-02-19T19:28:34.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:34 smithi132 ceph-mon[70676]: from='client.35370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:34.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:34 smithi132 ceph-mon[70676]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:34.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:34 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3495029388' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:28:35.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:34 smithi088 ceph-mon[88643]: from='client.35370 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:35.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:34 smithi088 ceph-mon[88643]: from='client.35374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:35.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:34 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3495029388' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:28:35.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.088+0000 7f16a9371700 1 -- 172.21.15.88:0/4127875122 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f16a4197380 con 0x7f168c06d8a0 2024-02-19T19:28:35.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.089+0000 7f168bfff700 1 -- 172.21.15.88:0/4127875122 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f16a4197380 con 0x7f168c06d8a0 2024-02-19T19:28:35.093 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:28:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:28:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:28:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:28:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:28:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:28:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:28:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:28:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:28:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:28:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:28:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:28:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:28:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:28:35.094 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:28:35.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.091+0000 7f16a9371700 1 -- 172.21.15.88:0/4127875122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f168c06d8a0 msgr2=0x7f168c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:35.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.091+0000 7f16a9371700 1 --2- 172.21.15.88:0/4127875122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f168c06d8a0 0x7f168c06fd50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1698005950 tx=0x7f16980058e0 comp rx=0 tx=0).stop 2024-02-19T19:28:35.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.091+0000 7f16a9371700 1 -- 172.21.15.88:0/4127875122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16a41004c0 msgr2=0x7f16a4195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:35.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.091+0000 7f16a9371700 1 --2- 172.21.15.88:0/4127875122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16a41004c0 0x7f16a4195f30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f1694006010 tx=0x7f169400be20 comp rx=0 tx=0).stop 2024-02-19T19:28:35.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.091+0000 7f16a9371700 1 -- 172.21.15.88:0/4127875122 shutdown_connections 2024-02-19T19:28:35.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.091+0000 7f16a9371700 1 --2- 172.21.15.88:0/4127875122 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f168c06d8a0 0x7f168c06fd50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:35.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.091+0000 7f16a9371700 1 --2- 172.21.15.88:0/4127875122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16a41004c0 0x7f16a4195f30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:35.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.091+0000 7f16a9371700 1 --2- 172.21.15.88:0/4127875122 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16a4100e60 0x7f16a4196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:35.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.091+0000 7f16a9371700 1 -- 172.21.15.88:0/4127875122 >> 172.21.15.88:0/4127875122 conn(0x7f16a40fbd80 msgr2=0x7f16a40fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:35.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.091+0000 7f16a9371700 1 -- 172.21.15.88:0/4127875122 shutdown_connections 2024-02-19T19:28:35.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.091+0000 7f16a9371700 1 -- 172.21.15.88:0/4127875122 wait complete. 2024-02-19T19:28:35.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.195+0000 7fd1d2d17700 1 -- 172.21.15.88:0/309228273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1cc100e80 msgr2=0x7fd1cc104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.195+0000 7fd1d2d17700 1 --2- 172.21.15.88:0/309228273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1cc100e80 0x7fd1cc104e50 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8009910 tx=0x7fd1b8009c20 comp rx=0 tx=0).stop 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.195+0000 7fd1d2d17700 1 -- 172.21.15.88:0/309228273 shutdown_connections 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.195+0000 7fd1d2d17700 1 --2- 172.21.15.88:0/309228273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1cc100e80 0x7fd1cc104e50 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.195+0000 7fd1d2d17700 1 --2- 172.21.15.88:0/309228273 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1cc1004e0 0x7fd1cc1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.195+0000 7fd1d2d17700 1 -- 172.21.15.88:0/309228273 >> 172.21.15.88:0/309228273 conn(0x7fd1cc0fbda0 msgr2=0x7fd1cc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.195+0000 7fd1d2d17700 1 -- 172.21.15.88:0/309228273 shutdown_connections 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.195+0000 7fd1d2d17700 1 -- 172.21.15.88:0/309228273 wait complete. 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.196+0000 7fd1d2d17700 1 Processor -- start 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.196+0000 7fd1d2d17700 1 -- start start 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.196+0000 7fd1d2d17700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1cc1004e0 0x7fd1cc196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.196+0000 7fd1d2d17700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1cc100e80 0x7fd1cc196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.196+0000 7fd1d2d17700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc196c20 con 0x7fd1cc1004e0 2024-02-19T19:28:35.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.196+0000 7fd1d2d17700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc19a9b0 con 0x7fd1cc100e80 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.196+0000 7fd1d0ab3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1cc1004e0 0x7fd1cc196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.196+0000 7fd1d0ab3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1cc1004e0 0x7fd1cc196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49126/0 (socket says 172.21.15.88:49126) 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.196+0000 7fd1d0ab3700 1 -- 172.21.15.88:0/2209995844 learned_addr learned my addr 172.21.15.88:0/2209995844 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.197+0000 7fd1d0ab3700 1 -- 172.21.15.88:0/2209995844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1cc100e80 msgr2=0x7fd1cc196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.197+0000 7fd1d0ab3700 1 --2- 172.21.15.88:0/2209995844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1cc100e80 0x7fd1cc196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.197+0000 7fd1d0ab3700 1 -- 172.21.15.88:0/2209995844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b80095f0 con 0x7fd1cc1004e0 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.197+0000 7fd1d0ab3700 1 --2- 172.21.15.88:0/2209995844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1cc1004e0 0x7fd1cc196000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000b4b0 tx=0x7fd1c000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.197+0000 7fd1c9ffb700 1 -- 172.21.15.88:0/2209995844 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1c0011800 con 0x7fd1cc1004e0 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.197+0000 7fd1d2d17700 1 -- 172.21.15.88:0/2209995844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc19ac90 con 0x7fd1cc1004e0 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.197+0000 7fd1c9ffb700 1 -- 172.21.15.88:0/2209995844 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd1c0011e20 con 0x7fd1cc1004e0 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.197+0000 7fd1c9ffb700 1 -- 172.21.15.88:0/2209995844 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1c000f2e0 con 0x7fd1cc1004e0 2024-02-19T19:28:35.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.197+0000 7fd1d2d17700 1 -- 172.21.15.88:0/2209995844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc19b1e0 con 0x7fd1cc1004e0 2024-02-19T19:28:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.198+0000 7fd1d2d17700 1 -- 172.21.15.88:0/2209995844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc108780 con 0x7fd1cc1004e0 2024-02-19T19:28:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.198+0000 7fd1c9ffb700 1 -- 172.21.15.88:0/2209995844 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd1c0010380 con 0x7fd1cc1004e0 2024-02-19T19:28:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.199+0000 7fd1c9ffb700 1 --2- 172.21.15.88:0/2209995844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd1bc06d8a0 0x7fd1bc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:28:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.199+0000 7fd1c9ffb700 1 -- 172.21.15.88:0/2209995844 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd1c008d170 con 0x7fd1cc1004e0 2024-02-19T19:28:35.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.199+0000 7fd1cbfff700 1 --2- 172.21.15.88:0/2209995844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd1bc06d8a0 0x7fd1bc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:28:35.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.199+0000 7fd1cbfff700 1 --2- 172.21.15.88:0/2209995844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd1bc06d8a0 0x7fd1bc06fd50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8006010 tx=0x7fd1b800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:28:35.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.204+0000 7fd1c9ffb700 1 -- 172.21.15.88:0/2209995844 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd1c00568c0 con 0x7fd1cc1004e0 2024-02-19T19:28:35.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.480+0000 7fd1d2d17700 1 -- 172.21.15.88:0/2209995844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd1cc04f2a0 con 0x7fd1cc1004e0 2024-02-19T19:28:35.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.481+0000 7fd1c9ffb700 1 -- 172.21.15.88:0/2209995844 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fd1c0059f80 con 0x7fd1cc1004e0 2024-02-19T19:28:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:28:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:28:35.486 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:28:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:28:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:28:35.486 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:28:35.486 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:28:35.486 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:28:35.486 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:28:35.486 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:28:35.487 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:28:35.488 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:28:35.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.483+0000 7fd1d2d17700 1 -- 172.21.15.88:0/2209995844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd1bc06d8a0 msgr2=0x7fd1bc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:35.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.483+0000 7fd1d2d17700 1 --2- 172.21.15.88:0/2209995844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd1bc06d8a0 0x7fd1bc06fd50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd1b8006010 tx=0x7fd1b800b540 comp rx=0 tx=0).stop 2024-02-19T19:28:35.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.483+0000 7fd1d2d17700 1 -- 172.21.15.88:0/2209995844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1cc1004e0 msgr2=0x7fd1cc196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:28:35.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.483+0000 7fd1d2d17700 1 --2- 172.21.15.88:0/2209995844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1cc1004e0 0x7fd1cc196000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd1c000b4b0 tx=0x7fd1c000b870 comp rx=0 tx=0).stop 2024-02-19T19:28:35.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.483+0000 7fd1d2d17700 1 -- 172.21.15.88:0/2209995844 shutdown_connections 2024-02-19T19:28:35.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.483+0000 7fd1d2d17700 1 --2- 172.21.15.88:0/2209995844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd1bc06d8a0 0x7fd1bc06fd50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:35.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.483+0000 7fd1d2d17700 1 --2- 172.21.15.88:0/2209995844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1cc1004e0 0x7fd1cc196000 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:35.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.483+0000 7fd1d2d17700 1 --2- 172.21.15.88:0/2209995844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1cc100e80 0x7fd1cc196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:28:35.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.483+0000 7fd1d2d17700 1 -- 172.21.15.88:0/2209995844 >> 172.21.15.88:0/2209995844 conn(0x7fd1cc0fbda0 msgr2=0x7fd1cc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:28:35.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.483+0000 7fd1d2d17700 1 -- 172.21.15.88:0/2209995844 shutdown_connections 2024-02-19T19:28:35.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:28:35.483+0000 7fd1d2d17700 1 -- 172.21.15.88:0/2209995844 wait complete. 2024-02-19T19:28:35.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:35 smithi088 ceph-mon[88643]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:35.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:35 smithi088 ceph-mon[88643]: pgmap v751: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:35.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:35 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3088639659' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:28:35.835 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:35 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2209995844' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:28:35.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:35 smithi132 ceph-mon[70676]: from='client.35378 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:35.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:35 smithi132 ceph-mon[70676]: pgmap v751: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:35.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:35 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3088639659' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:28:35.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:35 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2209995844' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:28:36.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:36 smithi132 ceph-mon[70676]: from='client.35390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:37.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:36 smithi088 ceph-mon[88643]: from='client.35390 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:28:37.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:37 smithi132 ceph-mon[70676]: pgmap v752: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:38.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:37 smithi088 ceph-mon[88643]: pgmap v752: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:39.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:39 smithi132 ceph-mon[70676]: pgmap v753: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:40.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:39 smithi088 ceph-mon[88643]: pgmap v753: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:41.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:41 smithi132 ceph-mon[70676]: pgmap v754: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:42.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:41 smithi088 ceph-mon[88643]: pgmap v754: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:43.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:43 smithi132 ceph-mon[70676]: pgmap v755: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:43.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:28:44.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:43 smithi088 ceph-mon[88643]: pgmap v755: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:44.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:28:45.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:45 smithi132 ceph-mon[70676]: pgmap v756: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:46.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:45 smithi088 ceph-mon[88643]: pgmap v756: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:47.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:47 smithi132 ceph-mon[70676]: pgmap v757: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:48.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:47 smithi088 ceph-mon[88643]: pgmap v757: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:49.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:49 smithi088 ceph-mon[88643]: pgmap v758: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:49.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:49 smithi132 ceph-mon[70676]: pgmap v758: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:51.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:50 smithi088 ceph-mon[88643]: pgmap v759: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:51.237 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:50 smithi132 ceph-mon[70676]: pgmap v759: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:53.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:53 smithi132 ceph-mon[70676]: pgmap v760: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:54.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:53 smithi088 ceph-mon[88643]: pgmap v760: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:55.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:55 smithi132 ceph-mon[70676]: pgmap v761: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:56.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:55 smithi088 ceph-mon[88643]: pgmap v761: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:57.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:57 smithi132 ceph-mon[70676]: pgmap v762: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:58.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:57 smithi088 ceph-mon[88643]: pgmap v762: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:28:58.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:28:59.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:28:59.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:28:59 smithi132 ceph-mon[70676]: pgmap v763: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:00.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:28:59 smithi088 ceph-mon[88643]: pgmap v763: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:01.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:01 smithi132 ceph-mon[70676]: pgmap v764: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:02.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:01 smithi088 ceph-mon[88643]: pgmap v764: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:03.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:03 smithi132 ceph-mon[70676]: pgmap v765: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:04.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:03 smithi088 ceph-mon[88643]: pgmap v765: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:05.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.605+0000 7fb91d15d700 1 -- 172.21.15.88:0/3221991978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9181004a0 msgr2=0x7fb918100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:05.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.605+0000 7fb91d15d700 1 --2- 172.21.15.88:0/3221991978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9181004a0 0x7fb918100870 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009910 tx=0x7fb90c009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:05.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.605+0000 7fb91d15d700 1 -- 172.21.15.88:0/3221991978 shutdown_connections 2024-02-19T19:29:05.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.605+0000 7fb91d15d700 1 --2- 172.21.15.88:0/3221991978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb918100e40 0x7fb918104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:05.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.605+0000 7fb91d15d700 1 --2- 172.21.15.88:0/3221991978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9181004a0 0x7fb918100870 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:05.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.605+0000 7fb91d15d700 1 -- 172.21.15.88:0/3221991978 >> 172.21.15.88:0/3221991978 conn(0x7fb9180fbd40 msgr2=0x7fb9180fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:05.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.605+0000 7fb91d15d700 1 -- 172.21.15.88:0/3221991978 shutdown_connections 2024-02-19T19:29:05.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.605+0000 7fb91d15d700 1 -- 172.21.15.88:0/3221991978 wait complete. 2024-02-19T19:29:05.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.606+0000 7fb91d15d700 1 Processor -- start 2024-02-19T19:29:05.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.606+0000 7fb91d15d700 1 -- start start 2024-02-19T19:29:05.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.606+0000 7fb91d15d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9181004a0 0x7fb918195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:05.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb91d15d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb918100e40 0x7fb9181964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:05.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb91d15d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb918196b90 con 0x7fb918100e40 2024-02-19T19:29:05.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb91d15d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91819a920 con 0x7fb9181004a0 2024-02-19T19:29:05.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb91659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb918100e40 0x7fb9181964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb91659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb918100e40 0x7fb9181964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37014/0 (socket says 172.21.15.88:37014) 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb91659c700 1 -- 172.21.15.88:0/2818406028 learned_addr learned my addr 172.21.15.88:0/2818406028 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb916d9d700 1 --2- 172.21.15.88:0/2818406028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9181004a0 0x7fb918195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb91659c700 1 -- 172.21.15.88:0/2818406028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9181004a0 msgr2=0x7fb918195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb91659c700 1 --2- 172.21.15.88:0/2818406028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9181004a0 0x7fb918195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb91659c700 1 -- 172.21.15.88:0/2818406028 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c0095f0 con 0x7fb918100e40 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb91659c700 1 --2- 172.21.15.88:0/2818406028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb918100e40 0x7fb9181964b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b480 tx=0x7fb90800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb8fffff700 1 -- 172.21.15.88:0/2818406028 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb908011800 con 0x7fb918100e40 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb8fffff700 1 -- 172.21.15.88:0/2818406028 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb908011e20 con 0x7fb918100e40 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb91d15d700 1 -- 172.21.15.88:0/2818406028 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91819ac00 con 0x7fb918100e40 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.607+0000 7fb8fffff700 1 -- 172.21.15.88:0/2818406028 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb908010ea0 con 0x7fb918100e40 2024-02-19T19:29:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.608+0000 7fb91d15d700 1 -- 172.21.15.88:0/2818406028 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91819b150 con 0x7fb918100e40 2024-02-19T19:29:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.608+0000 7fb91d15d700 1 -- 172.21.15.88:0/2818406028 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9181087b0 con 0x7fb918100e40 2024-02-19T19:29:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.609+0000 7fb8fffff700 1 -- 172.21.15.88:0/2818406028 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb908010380 con 0x7fb918100e40 2024-02-19T19:29:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.609+0000 7fb8fffff700 1 --2- 172.21.15.88:0/2818406028 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90006d8a0 0x7fb90006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.609+0000 7fb916d9d700 1 --2- 172.21.15.88:0/2818406028 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90006d8a0 0x7fb90006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:05.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.610+0000 7fb8fffff700 1 -- 172.21.15.88:0/2818406028 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb90808d000 con 0x7fb918100e40 2024-02-19T19:29:05.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.610+0000 7fb916d9d700 1 --2- 172.21.15.88:0/2818406028 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90006d8a0 0x7fb90006fd50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006010 tx=0x7fb90c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:05.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.614+0000 7fb8fffff700 1 -- 172.21.15.88:0/2818406028 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb9080567b0 con 0x7fb918100e40 2024-02-19T19:29:05.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.829+0000 7fb91d15d700 1 -- 172.21.15.88:0/2818406028 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb91819b440 con 0x7fb90006d8a0 2024-02-19T19:29:05.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.831+0000 7fb8fffff700 1 -- 172.21.15.88:0/2818406028 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb91819b440 con 0x7fb90006d8a0 2024-02-19T19:29:05.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.833+0000 7fb91d15d700 1 -- 172.21.15.88:0/2818406028 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90006d8a0 msgr2=0x7fb90006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:05.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.833+0000 7fb91d15d700 1 --2- 172.21.15.88:0/2818406028 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90006d8a0 0x7fb90006fd50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006010 tx=0x7fb90c01a040 comp rx=0 tx=0).stop 2024-02-19T19:29:05.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.833+0000 7fb91d15d700 1 -- 172.21.15.88:0/2818406028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb918100e40 msgr2=0x7fb9181964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:05.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.833+0000 7fb91d15d700 1 --2- 172.21.15.88:0/2818406028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb918100e40 0x7fb9181964b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b480 tx=0x7fb90800b790 comp rx=0 tx=0).stop 2024-02-19T19:29:05.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.833+0000 7fb91d15d700 1 -- 172.21.15.88:0/2818406028 shutdown_connections 2024-02-19T19:29:05.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.833+0000 7fb91d15d700 1 --2- 172.21.15.88:0/2818406028 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb90006d8a0 0x7fb90006fd50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:05.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.833+0000 7fb91d15d700 1 --2- 172.21.15.88:0/2818406028 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb9181004a0 0x7fb918195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:05.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.833+0000 7fb91d15d700 1 --2- 172.21.15.88:0/2818406028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb918100e40 0x7fb9181964b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:05.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.833+0000 7fb91d15d700 1 -- 172.21.15.88:0/2818406028 >> 172.21.15.88:0/2818406028 conn(0x7fb9180fbd40 msgr2=0x7fb9180fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:05.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.833+0000 7fb91d15d700 1 -- 172.21.15.88:0/2818406028 shutdown_connections 2024-02-19T19:29:05.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.833+0000 7fb91d15d700 1 -- 172.21.15.88:0/2818406028 wait complete. 2024-02-19T19:29:05.847 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:29:05.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.938+0000 7fe84e27c700 1 -- 172.21.15.88:0/1524105433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8480ff720 msgr2=0x7fe8480ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:05.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.939+0000 7fe84e27c700 1 --2- 172.21.15.88:0/1524105433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8480ff720 0x7fe8480ffb90 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe83c009910 tx=0x7fe83c009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:05.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.939+0000 7fe84e27c700 1 -- 172.21.15.88:0/1524105433 shutdown_connections 2024-02-19T19:29:05.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.939+0000 7fe84e27c700 1 --2- 172.21.15.88:0/1524105433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8480ff720 0x7fe8480ffb90 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:05.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.939+0000 7fe84e27c700 1 --2- 172.21.15.88:0/1524105433 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8480ff350 0x7fe8481063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:05.943 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:05 smithi088 ceph-mon[88643]: pgmap v766: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:05.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.939+0000 7fe84e27c700 1 -- 172.21.15.88:0/1524105433 >> 172.21.15.88:0/1524105433 conn(0x7fe8480748b0 msgr2=0x7fe848074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:05.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.939+0000 7fe84e27c700 1 -- 172.21.15.88:0/1524105433 shutdown_connections 2024-02-19T19:29:05.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.939+0000 7fe84e27c700 1 -- 172.21.15.88:0/1524105433 wait complete. 2024-02-19T19:29:05.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.939+0000 7fe84e27c700 1 Processor -- start 2024-02-19T19:29:05.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.939+0000 7fe84e27c700 1 -- start start 2024-02-19T19:29:05.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe84e27c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8480ff350 0x7fe84819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:05.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe84e27c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8480ff720 0x7fe84819ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:05.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe84e27c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84819d170 con 0x7fe8480ff350 2024-02-19T19:29:05.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe84e27c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8481a0f00 con 0x7fe8480ff720 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe847fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8480ff350 0x7fe84819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe847fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8480ff350 0x7fe84819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37038/0 (socket says 172.21.15.88:37038) 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe847fff700 1 -- 172.21.15.88:0/2539809843 learned_addr learned my addr 172.21.15.88:0/2539809843 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe8477fe700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8480ff720 0x7fe84819ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe847fff700 1 -- 172.21.15.88:0/2539809843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8480ff720 msgr2=0x7fe84819ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe847fff700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8480ff720 0x7fe84819ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe847fff700 1 -- 172.21.15.88:0/2539809843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c0095f0 con 0x7fe8480ff350 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe8477fe700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8480ff720 0x7fe84819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe847fff700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8480ff350 0x7fe84819c550 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe83800b4b0 tx=0x7fe83800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.940+0000 7fe8457fa700 1 -- 172.21.15.88:0/2539809843 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe838011800 con 0x7fe8480ff350 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.941+0000 7fe84e27c700 1 -- 172.21.15.88:0/2539809843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8481a11e0 con 0x7fe8480ff350 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.941+0000 7fe8457fa700 1 -- 172.21.15.88:0/2539809843 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe838011e20 con 0x7fe8480ff350 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.941+0000 7fe8457fa700 1 -- 172.21.15.88:0/2539809843 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83800f2e0 con 0x7fe8480ff350 2024-02-19T19:29:05.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.941+0000 7fe84e27c700 1 -- 172.21.15.88:0/2539809843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8481a1730 con 0x7fe8480ff350 2024-02-19T19:29:05.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.941+0000 7fe84e27c700 1 -- 172.21.15.88:0/2539809843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe848103ae0 con 0x7fe8480ff350 2024-02-19T19:29:05.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.942+0000 7fe8457fa700 1 -- 172.21.15.88:0/2539809843 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe838010380 con 0x7fe8480ff350 2024-02-19T19:29:05.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.942+0000 7fe8457fa700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe83006d8a0 0x7fe83006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:05.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.942+0000 7fe8457fa700 1 -- 172.21.15.88:0/2539809843 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe83808d0f0 con 0x7fe8480ff350 2024-02-19T19:29:05.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.942+0000 7fe8477fe700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe83006d8a0 0x7fe83006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:05.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.943+0000 7fe8477fe700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe83006d8a0 0x7fe83006fd50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0059d0 tx=0x7fe83c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:05.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:05.947+0000 7fe8457fa700 1 -- 172.21.15.88:0/2539809843 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe8380568a0 con 0x7fe8480ff350 2024-02-19T19:29:05.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:05 smithi132 ceph-mon[70676]: pgmap v766: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:06.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.161+0000 7fe84e27c700 1 -- 172.21.15.88:0/2539809843 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8481a19b0 con 0x7fe83006d8a0 2024-02-19T19:29:06.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.162+0000 7fe8457fa700 1 -- 172.21.15.88:0/2539809843 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe8481a19b0 con 0x7fe83006d8a0 2024-02-19T19:29:06.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.164+0000 7fe84e27c700 1 -- 172.21.15.88:0/2539809843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe83006d8a0 msgr2=0x7fe83006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:06.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.164+0000 7fe84e27c700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe83006d8a0 0x7fe83006fd50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0059d0 tx=0x7fe83c01a040 comp rx=0 tx=0).stop 2024-02-19T19:29:06.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.164+0000 7fe84e27c700 1 -- 172.21.15.88:0/2539809843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8480ff350 msgr2=0x7fe84819c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:06.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.164+0000 7fe84e27c700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8480ff350 0x7fe84819c550 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe83800b4b0 tx=0x7fe83800b870 comp rx=0 tx=0).stop 2024-02-19T19:29:06.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.164+0000 7fe84e27c700 1 -- 172.21.15.88:0/2539809843 shutdown_connections 2024-02-19T19:29:06.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.164+0000 7fe84e27c700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe83006d8a0 0x7fe83006fd50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.164+0000 7fe84e27c700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8480ff350 0x7fe84819c550 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.164+0000 7fe84e27c700 1 --2- 172.21.15.88:0/2539809843 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8480ff720 0x7fe84819ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.164+0000 7fe84e27c700 1 -- 172.21.15.88:0/2539809843 >> 172.21.15.88:0/2539809843 conn(0x7fe8480748b0 msgr2=0x7fe8480fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:06.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.164+0000 7fe84e27c700 1 -- 172.21.15.88:0/2539809843 shutdown_connections 2024-02-19T19:29:06.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.164+0000 7fe84e27c700 1 -- 172.21.15.88:0/2539809843 wait complete. 2024-02-19T19:29:06.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.269+0000 7f4bf81c4700 1 -- 172.21.15.88:0/4044764510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf00ffbd0 msgr2=0x7f4bf0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:06.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.269+0000 7f4bf81c4700 1 --2- 172.21.15.88:0/4044764510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf00ffbd0 0x7f4bf0100040 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f4be0009910 tx=0x7f4be0009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:06.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.269+0000 7f4bf81c4700 1 -- 172.21.15.88:0/4044764510 shutdown_connections 2024-02-19T19:29:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.269+0000 7f4bf81c4700 1 --2- 172.21.15.88:0/4044764510 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf00ffbd0 0x7f4bf0100040 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.269+0000 7f4bf81c4700 1 --2- 172.21.15.88:0/4044764510 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf0104130 0x7f4bf0104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.269+0000 7f4bf81c4700 1 -- 172.21.15.88:0/4044764510 >> 172.21.15.88:0/4044764510 conn(0x7f4bf0075130 msgr2=0x7f4bf0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.269+0000 7f4bf81c4700 1 -- 172.21.15.88:0/4044764510 shutdown_connections 2024-02-19T19:29:06.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.269+0000 7f4bf81c4700 1 -- 172.21.15.88:0/4044764510 wait complete. 2024-02-19T19:29:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf81c4700 1 Processor -- start 2024-02-19T19:29:06.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf81c4700 1 -- start start 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf81c4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf00ffbd0 0x7f4bf0194a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf81c4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf0104130 0x7f4bf0194fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf81c4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0195510 con 0x7f4bf00ffbd0 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf81c4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf0195680 con 0x7f4bf0104130 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf5f60700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf00ffbd0 0x7f4bf0194a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf5f60700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf00ffbd0 0x7f4bf0194a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37050/0 (socket says 172.21.15.88:37050) 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf5f60700 1 -- 172.21.15.88:0/961011524 learned_addr learned my addr 172.21.15.88:0/961011524 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf5f60700 1 -- 172.21.15.88:0/961011524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf0104130 msgr2=0x7f4bf0194fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf5f60700 1 --2- 172.21.15.88:0/961011524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf0104130 0x7f4bf0194fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf5f60700 1 -- 172.21.15.88:0/961011524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be00095f0 con 0x7f4bf00ffbd0 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.270+0000 7f4bf5f60700 1 --2- 172.21.15.88:0/961011524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf00ffbd0 0x7f4bf0194a90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00d8d0 tx=0x7f4bec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:06.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.271+0000 7f4be6ffd700 1 -- 172.21.15.88:0/961011524 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bec009940 con 0x7f4bf00ffbd0 2024-02-19T19:29:06.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.271+0000 7f4be6ffd700 1 -- 172.21.15.88:0/961011524 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bec010460 con 0x7f4bf00ffbd0 2024-02-19T19:29:06.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.271+0000 7f4bf81c4700 1 -- 172.21.15.88:0/961011524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf01a2100 con 0x7f4bf00ffbd0 2024-02-19T19:29:06.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.271+0000 7f4be6ffd700 1 -- 172.21.15.88:0/961011524 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bec00f5f0 con 0x7f4bf00ffbd0 2024-02-19T19:29:06.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.271+0000 7f4bf81c4700 1 -- 172.21.15.88:0/961011524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf01a2460 con 0x7f4bf00ffbd0 2024-02-19T19:29:06.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.272+0000 7f4bf81c4700 1 -- 172.21.15.88:0/961011524 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf004f2a0 con 0x7f4bf00ffbd0 2024-02-19T19:29:06.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.272+0000 7f4be6ffd700 1 -- 172.21.15.88:0/961011524 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4bec0105d0 con 0x7f4bf00ffbd0 2024-02-19T19:29:06.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.273+0000 7f4be6ffd700 1 --2- 172.21.15.88:0/961011524 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4bdc06d8a0 0x7f4bdc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:06.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.273+0000 7f4be6ffd700 1 -- 172.21.15.88:0/961011524 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4bec08d4d0 con 0x7f4bf00ffbd0 2024-02-19T19:29:06.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.273+0000 7f4bf575f700 1 --2- 172.21.15.88:0/961011524 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4bdc06d8a0 0x7f4bdc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:06.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.273+0000 7f4bf575f700 1 --2- 172.21.15.88:0/961011524 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4bdc06d8a0 0x7f4bdc06fd50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4be000b5c0 tx=0x7f4be0005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:06.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.277+0000 7f4be6ffd700 1 -- 172.21.15.88:0/961011524 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4bec05a7b0 con 0x7f4bf00ffbd0 2024-02-19T19:29:06.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.485+0000 7f4bf81c4700 1 -- 172.21.15.88:0/961011524 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bf0195840 con 0x7f4bdc06d8a0 2024-02-19T19:29:06.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.495+0000 7f4be6ffd700 1 -- 172.21.15.88:0/961011524 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4bf0195840 con 0x7f4bdc06d8a0 2024-02-19T19:29:06.499 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (26m) 2m ago 36m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (37m) 2m ago 37m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (19m) 2m ago 35m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (22m) 2m ago 37m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (22m) 2m ago 35m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (25m) 2m ago 36m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (18m) 2m ago 30m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (18m) 2m ago 30m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (17m) 2m ago 30m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (2m) 2m ago 30m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (27m) 2m ago 38m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:29:06.500 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (27m) 2m ago 35m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:29:06.501 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (24m) 2m ago 38m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:29:06.501 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (23m) 2m ago 34m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:29:06.501 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (26m) 2m ago 37m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:29:06.501 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (26m) 2m ago 35m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:29:06.501 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (21m) 2m ago 34m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:29:06.501 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (21m) 2m ago 33m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:29:06.501 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (20m) 2m ago 33m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:29:06.501 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (19m) 2m ago 32m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:29:06.501 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (19m) 2m ago 32m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:29:06.501 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (18m) 2m ago 31m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:29:06.501 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (26m) 2m ago 35m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:29:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.497+0000 7f4bf81c4700 1 -- 172.21.15.88:0/961011524 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4bdc06d8a0 msgr2=0x7f4bdc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.497+0000 7f4bf81c4700 1 --2- 172.21.15.88:0/961011524 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4bdc06d8a0 0x7f4bdc06fd50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4be000b5c0 tx=0x7f4be0005e20 comp rx=0 tx=0).stop 2024-02-19T19:29:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.497+0000 7f4bf81c4700 1 -- 172.21.15.88:0/961011524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf00ffbd0 msgr2=0x7f4bf0194a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.497+0000 7f4bf81c4700 1 --2- 172.21.15.88:0/961011524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf00ffbd0 0x7f4bf0194a90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00d8d0 tx=0x7f4bec00dc90 comp rx=0 tx=0).stop 2024-02-19T19:29:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.498+0000 7f4bf81c4700 1 -- 172.21.15.88:0/961011524 shutdown_connections 2024-02-19T19:29:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.498+0000 7f4bf81c4700 1 --2- 172.21.15.88:0/961011524 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4bdc06d8a0 0x7f4bdc06fd50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.498+0000 7f4bf81c4700 1 --2- 172.21.15.88:0/961011524 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4bf00ffbd0 0x7f4bf0194a90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.498+0000 7f4bf81c4700 1 --2- 172.21.15.88:0/961011524 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4bf0104130 0x7f4bf0194fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.498+0000 7f4bf81c4700 1 -- 172.21.15.88:0/961011524 >> 172.21.15.88:0/961011524 conn(0x7f4bf0075130 msgr2=0x7f4bf00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.498+0000 7f4bf81c4700 1 -- 172.21.15.88:0/961011524 shutdown_connections 2024-02-19T19:29:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.498+0000 7f4bf81c4700 1 -- 172.21.15.88:0/961011524 wait complete. 2024-02-19T19:29:06.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.600+0000 7f2da7fa3700 1 -- 172.21.15.88:0/233422591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da00ffd60 msgr2=0x7f2da0103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:06.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.600+0000 7f2da7fa3700 1 --2- 172.21.15.88:0/233422591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da00ffd60 0x7f2da0103d30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2d90009910 tx=0x7f2d90009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:06.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.600+0000 7f2da7fa3700 1 -- 172.21.15.88:0/233422591 shutdown_connections 2024-02-19T19:29:06.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.600+0000 7f2da7fa3700 1 --2- 172.21.15.88:0/233422591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da00ffd60 0x7f2da0103d30 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.600+0000 7f2da7fa3700 1 --2- 172.21.15.88:0/233422591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da00ff3c0 0x7f2da00ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.600+0000 7f2da7fa3700 1 -- 172.21.15.88:0/233422591 >> 172.21.15.88:0/233422591 conn(0x7f2da00fac80 msgr2=0x7f2da00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:06.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.600+0000 7f2da7fa3700 1 -- 172.21.15.88:0/233422591 shutdown_connections 2024-02-19T19:29:06.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.600+0000 7f2da7fa3700 1 -- 172.21.15.88:0/233422591 wait complete. 2024-02-19T19:29:06.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.600+0000 7f2da7fa3700 1 Processor -- start 2024-02-19T19:29:06.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da7fa3700 1 -- start start 2024-02-19T19:29:06.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da7fa3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da00ff3c0 0x7f2da01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:06.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da7fa3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da00ffd60 0x7f2da01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:06.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da7fa3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0198cd0 con 0x7f2da00ffd60 2024-02-19T19:29:06.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da7fa3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da019ca60 con 0x7f2da00ff3c0 2024-02-19T19:29:06.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da553e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da00ffd60 0x7f2da01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:06.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da553e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da00ffd60 0x7f2da01985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37058/0 (socket says 172.21.15.88:37058) 2024-02-19T19:29:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da553e700 1 -- 172.21.15.88:0/247597850 learned_addr learned my addr 172.21.15.88:0/247597850 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da5d3f700 1 --2- 172.21.15.88:0/247597850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da00ff3c0 0x7f2da01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da553e700 1 -- 172.21.15.88:0/247597850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da00ff3c0 msgr2=0x7f2da01980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da553e700 1 --2- 172.21.15.88:0/247597850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da00ff3c0 0x7f2da01980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da553e700 1 -- 172.21.15.88:0/247597850 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d900095f0 con 0x7f2da00ffd60 2024-02-19T19:29:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da553e700 1 --2- 172.21.15.88:0/247597850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da00ffd60 0x7f2da01985f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2d900092e0 tx=0x7f2d9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2d96ffd700 1 -- 172.21.15.88:0/247597850 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d9001d070 con 0x7f2da00ffd60 2024-02-19T19:29:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2d96ffd700 1 -- 172.21.15.88:0/247597850 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d90005dc0 con 0x7f2da00ffd60 2024-02-19T19:29:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da7fa3700 1 -- 172.21.15.88:0/247597850 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da019cce0 con 0x7f2da00ffd60 2024-02-19T19:29:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.601+0000 7f2da7fa3700 1 -- 172.21.15.88:0/247597850 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da019d1d0 con 0x7f2da00ffd60 2024-02-19T19:29:06.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.602+0000 7f2d96ffd700 1 -- 172.21.15.88:0/247597850 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d90017400 con 0x7f2da00ffd60 2024-02-19T19:29:06.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.602+0000 7f2da7fa3700 1 -- 172.21.15.88:0/247597850 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2da004f2a0 con 0x7f2da00ffd60 2024-02-19T19:29:06.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.603+0000 7f2d96ffd700 1 -- 172.21.15.88:0/247597850 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2d9000f460 con 0x7f2da00ffd60 2024-02-19T19:29:06.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.603+0000 7f2d96ffd700 1 --2- 172.21.15.88:0/247597850 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d8c06d8a0 0x7f2d8c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:06.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.603+0000 7f2d96ffd700 1 -- 172.21.15.88:0/247597850 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2d9008e960 con 0x7f2da00ffd60 2024-02-19T19:29:06.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.603+0000 7f2da5d3f700 1 --2- 172.21.15.88:0/247597850 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d8c06d8a0 0x7f2d8c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:06.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.604+0000 7f2da5d3f700 1 --2- 172.21.15.88:0/247597850 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d8c06d8a0 0x7f2d8c06fd50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009d10 tx=0x7f2d98009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:06.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.608+0000 7f2d96ffd700 1 -- 172.21.15.88:0/247597850 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d90058190 con 0x7f2da00ffd60 2024-02-19T19:29:06.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.896+0000 7f2da7fa3700 1 -- 172.21.15.88:0/247597850 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2da019d500 con 0x7f2da00ffd60 2024-02-19T19:29:06.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.897+0000 7f2d96ffd700 1 -- 172.21.15.88:0/247597850 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2d9005b850 con 0x7f2da00ffd60 2024-02-19T19:29:06.901 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:29:06.901 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:29:06.902 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:29:06.903 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:29:06.903 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:29:06.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.899+0000 7f2da7fa3700 1 -- 172.21.15.88:0/247597850 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d8c06d8a0 msgr2=0x7f2d8c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:06.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.899+0000 7f2da7fa3700 1 --2- 172.21.15.88:0/247597850 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d8c06d8a0 0x7f2d8c06fd50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009d10 tx=0x7f2d98009040 comp rx=0 tx=0).stop 2024-02-19T19:29:06.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.899+0000 7f2da7fa3700 1 -- 172.21.15.88:0/247597850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da00ffd60 msgr2=0x7f2da01985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:06.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.899+0000 7f2da7fa3700 1 --2- 172.21.15.88:0/247597850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da00ffd60 0x7f2da01985f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2d900092e0 tx=0x7f2d9000be20 comp rx=0 tx=0).stop 2024-02-19T19:29:06.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.899+0000 7f2da7fa3700 1 -- 172.21.15.88:0/247597850 shutdown_connections 2024-02-19T19:29:06.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.899+0000 7f2da7fa3700 1 --2- 172.21.15.88:0/247597850 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d8c06d8a0 0x7f2d8c06fd50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.899+0000 7f2da7fa3700 1 --2- 172.21.15.88:0/247597850 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2da00ff3c0 0x7f2da01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.899+0000 7f2da7fa3700 1 --2- 172.21.15.88:0/247597850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2da00ffd60 0x7f2da01985f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:06.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.899+0000 7f2da7fa3700 1 -- 172.21.15.88:0/247597850 >> 172.21.15.88:0/247597850 conn(0x7f2da00fac80 msgr2=0x7f2da00696e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:06.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.899+0000 7f2da7fa3700 1 -- 172.21.15.88:0/247597850 shutdown_connections 2024-02-19T19:29:06.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:06.899+0000 7f2da7fa3700 1 -- 172.21.15.88:0/247597850 wait complete. 2024-02-19T19:29:07.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.002+0000 7f9b4bab7700 1 -- 172.21.15.88:0/2392713334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b440ffbd0 msgr2=0x7f9b44100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.002+0000 7f9b4bab7700 1 --2- 172.21.15.88:0/2392713334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b440ffbd0 0x7f9b44100040 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9b40009910 tx=0x7f9b40009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:07.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.002+0000 7f9b4bab7700 1 -- 172.21.15.88:0/2392713334 shutdown_connections 2024-02-19T19:29:07.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.002+0000 7f9b4bab7700 1 --2- 172.21.15.88:0/2392713334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b440ffbd0 0x7f9b44100040 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.002+0000 7f9b4bab7700 1 --2- 172.21.15.88:0/2392713334 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b44104130 0x7f9b44104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.002+0000 7f9b4bab7700 1 -- 172.21.15.88:0/2392713334 >> 172.21.15.88:0/2392713334 conn(0x7f9b44075130 msgr2=0x7f9b44075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:07.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.002+0000 7f9b4bab7700 1 -- 172.21.15.88:0/2392713334 shutdown_connections 2024-02-19T19:29:07.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.002+0000 7f9b4bab7700 1 -- 172.21.15.88:0/2392713334 wait complete. 2024-02-19T19:29:07.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b4bab7700 1 Processor -- start 2024-02-19T19:29:07.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b4bab7700 1 -- start start 2024-02-19T19:29:07.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b4bab7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b440ffbd0 0x7f9b44194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:07.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b4bab7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b44104130 0x7f9b44195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:07.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b4bab7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b44195950 con 0x7f9b440ffbd0 2024-02-19T19:29:07.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b4bab7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b441996e0 con 0x7f9b44104130 2024-02-19T19:29:07.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b49853700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b440ffbd0 0x7f9b44194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:07.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b49853700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b440ffbd0 0x7f9b44194d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37078/0 (socket says 172.21.15.88:37078) 2024-02-19T19:29:07.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b49853700 1 -- 172.21.15.88:0/217688127 learned_addr learned my addr 172.21.15.88:0/217688127 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:07.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b49052700 1 --2- 172.21.15.88:0/217688127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b44104130 0x7f9b44195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:07.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b49853700 1 -- 172.21.15.88:0/217688127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b44104130 msgr2=0x7f9b44195270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b49853700 1 --2- 172.21.15.88:0/217688127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b44104130 0x7f9b44195270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.003+0000 7f9b49853700 1 -- 172.21.15.88:0/217688127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b400095f0 con 0x7f9b440ffbd0 2024-02-19T19:29:07.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.004+0000 7f9b49853700 1 --2- 172.21.15.88:0/217688127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b440ffbd0 0x7f9b44194d30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400d8d0 tx=0x7f9b3400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:07.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.004+0000 7f9b3affd700 1 -- 172.21.15.88:0/217688127 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b340098a0 con 0x7f9b440ffbd0 2024-02-19T19:29:07.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.004+0000 7f9b3affd700 1 -- 172.21.15.88:0/217688127 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b34010460 con 0x7f9b440ffbd0 2024-02-19T19:29:07.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.004+0000 7f9b4bab7700 1 -- 172.21.15.88:0/217688127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b441999c0 con 0x7f9b440ffbd0 2024-02-19T19:29:07.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.004+0000 7f9b3affd700 1 -- 172.21.15.88:0/217688127 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3400f5f0 con 0x7f9b440ffbd0 2024-02-19T19:29:07.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.004+0000 7f9b4bab7700 1 -- 172.21.15.88:0/217688127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b44199ee0 con 0x7f9b440ffbd0 2024-02-19T19:29:07.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.005+0000 7f9b4bab7700 1 -- 172.21.15.88:0/217688127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b4404f2a0 con 0x7f9b440ffbd0 2024-02-19T19:29:07.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.005+0000 7f9b3affd700 1 -- 172.21.15.88:0/217688127 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9b34009a00 con 0x7f9b440ffbd0 2024-02-19T19:29:07.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.006+0000 7f9b3affd700 1 --2- 172.21.15.88:0/217688127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b3006d8a0 0x7f9b3006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:07.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.006+0000 7f9b3affd700 1 -- 172.21.15.88:0/217688127 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9b3408d430 con 0x7f9b440ffbd0 2024-02-19T19:29:07.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.006+0000 7f9b49052700 1 --2- 172.21.15.88:0/217688127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b3006d8a0 0x7f9b3006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:07.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.006+0000 7f9b49052700 1 --2- 172.21.15.88:0/217688127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b3006d8a0 0x7f9b3006fd50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9b40006010 tx=0x7f9b4001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:07.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.010+0000 7f9b3affd700 1 -- 172.21.15.88:0/217688127 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9b34056be0 con 0x7f9b440ffbd0 2024-02-19T19:29:07.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.249+0000 7f9b4bab7700 1 -- 172.21.15.88:0/217688127 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9b4419a1a0 con 0x7f9b440ffbd0 2024-02-19T19:29:07.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.250+0000 7f9b3affd700 1 -- 172.21.15.88:0/217688127 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f9b3405a2a0 con 0x7f9b440ffbd0 2024-02-19T19:29:07.254 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:29:07.254 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:29:07.254 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:29:07.255 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:29:07.256 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:29:07.257 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:29:07.257 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:29:07.257 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:29:07.257 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:29:07.257 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:29:07.257 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:29:07.257 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.252+0000 7f9b4bab7700 1 -- 172.21.15.88:0/217688127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b3006d8a0 msgr2=0x7f9b3006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.252+0000 7f9b4bab7700 1 --2- 172.21.15.88:0/217688127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b3006d8a0 0x7f9b3006fd50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f9b40006010 tx=0x7f9b4001a040 comp rx=0 tx=0).stop 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.252+0000 7f9b4bab7700 1 -- 172.21.15.88:0/217688127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b440ffbd0 msgr2=0x7f9b44194d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.252+0000 7f9b4bab7700 1 --2- 172.21.15.88:0/217688127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b440ffbd0 0x7f9b44194d30 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400d8d0 tx=0x7f9b3400dbe0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.252+0000 7f9b4bab7700 1 -- 172.21.15.88:0/217688127 shutdown_connections 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.252+0000 7f9b4bab7700 1 --2- 172.21.15.88:0/217688127 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9b3006d8a0 0x7f9b3006fd50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.252+0000 7f9b4bab7700 1 --2- 172.21.15.88:0/217688127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b440ffbd0 0x7f9b44194d30 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.252+0000 7f9b4bab7700 1 --2- 172.21.15.88:0/217688127 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9b44104130 0x7f9b44195270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.252+0000 7f9b4bab7700 1 -- 172.21.15.88:0/217688127 >> 172.21.15.88:0/217688127 conn(0x7f9b44075130 msgr2=0x7f9b440feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.253+0000 7f9b4bab7700 1 -- 172.21.15.88:0/217688127 shutdown_connections 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.253+0000 7f9b4bab7700 1 -- 172.21.15.88:0/217688127 wait complete. 2024-02-19T19:29:07.258 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:29:07.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.357+0000 7fab15767700 1 -- 172.21.15.88:0/2085231370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab10100e80 msgr2=0x7fab10104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.357+0000 7fab15767700 1 --2- 172.21.15.88:0/2085231370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab10100e80 0x7fab10104e50 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009910 tx=0x7faaf8009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:07.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.357+0000 7fab15767700 1 -- 172.21.15.88:0/2085231370 shutdown_connections 2024-02-19T19:29:07.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.357+0000 7fab15767700 1 --2- 172.21.15.88:0/2085231370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab10100e80 0x7fab10104e50 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.357+0000 7fab15767700 1 --2- 172.21.15.88:0/2085231370 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab101004e0 0x7fab101008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.357+0000 7fab15767700 1 -- 172.21.15.88:0/2085231370 >> 172.21.15.88:0/2085231370 conn(0x7fab100fbda0 msgr2=0x7fab100fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:07.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.357+0000 7fab15767700 1 -- 172.21.15.88:0/2085231370 shutdown_connections 2024-02-19T19:29:07.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.357+0000 7fab15767700 1 -- 172.21.15.88:0/2085231370 wait complete. 2024-02-19T19:29:07.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.357+0000 7fab15767700 1 Processor -- start 2024-02-19T19:29:07.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab15767700 1 -- start start 2024-02-19T19:29:07.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab15767700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab101004e0 0x7fab10073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:07.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab15767700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab10100e80 0x7fab10073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:07.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab15767700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab10073ad0 con 0x7fab101004e0 2024-02-19T19:29:07.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab15767700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab10073c10 con 0x7fab10100e80 2024-02-19T19:29:07.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab0effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab101004e0 0x7fab10073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:07.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab0effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab101004e0 0x7fab10073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37100/0 (socket says 172.21.15.88:37100) 2024-02-19T19:29:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab0effd700 1 -- 172.21.15.88:0/1864836146 learned_addr learned my addr 172.21.15.88:0/1864836146 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab067fc700 1 --2- 172.21.15.88:0/1864836146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab10100e80 0x7fab10073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab0effd700 1 -- 172.21.15.88:0/1864836146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab10100e80 msgr2=0x7fab10073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab0effd700 1 --2- 172.21.15.88:0/1864836146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab10100e80 0x7fab10073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab0effd700 1 -- 172.21.15.88:0/1864836146 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf80095f0 con 0x7fab101004e0 2024-02-19T19:29:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.358+0000 7fab0effd700 1 --2- 172.21.15.88:0/1864836146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab101004e0 0x7fab10073050 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fab0000ada0 tx=0x7fab0000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.359+0000 7fab0cff9700 1 -- 172.21.15.88:0/1864836146 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab0000f070 con 0x7fab101004e0 2024-02-19T19:29:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.359+0000 7fab0cff9700 1 -- 172.21.15.88:0/1864836146 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab00007bf0 con 0x7fab101004e0 2024-02-19T19:29:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.359+0000 7fab15767700 1 -- 172.21.15.88:0/1864836146 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab101a20e0 con 0x7fab101004e0 2024-02-19T19:29:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.359+0000 7fab0cff9700 1 -- 172.21.15.88:0/1864836146 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab00005690 con 0x7fab101004e0 2024-02-19T19:29:07.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.359+0000 7fab15767700 1 -- 172.21.15.88:0/1864836146 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab101a24f0 con 0x7fab101004e0 2024-02-19T19:29:07.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.360+0000 7fab15767700 1 -- 172.21.15.88:0/1864836146 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab10108850 con 0x7fab101004e0 2024-02-19T19:29:07.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.360+0000 7fab0cff9700 1 -- 172.21.15.88:0/1864836146 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fab0000c900 con 0x7fab101004e0 2024-02-19T19:29:07.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.360+0000 7fab0cff9700 1 --2- 172.21.15.88:0/1864836146 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faafc06d8a0 0x7faafc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:07.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.361+0000 7fab0cff9700 1 -- 172.21.15.88:0/1864836146 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fab0008cf60 con 0x7fab101004e0 2024-02-19T19:29:07.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.361+0000 7fab067fc700 1 --2- 172.21.15.88:0/1864836146 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faafc06d8a0 0x7faafc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:07.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.361+0000 7fab067fc700 1 --2- 172.21.15.88:0/1864836146 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faafc06d8a0 0x7faafc06fd50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fab10074840 tx=0x7faaf801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:07.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.365+0000 7fab0cff9700 1 -- 172.21.15.88:0/1864836146 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fab000c0d20 con 0x7fab101004e0 2024-02-19T19:29:07.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.577+0000 7fab15767700 1 -- 172.21.15.88:0/1864836146 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fab10061190 con 0x7faafc06d8a0 2024-02-19T19:29:07.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.579+0000 7fab0cff9700 1 -- 172.21.15.88:0/1864836146 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fab10061190 con 0x7faafc06d8a0 2024-02-19T19:29:07.583 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:29:07.583 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:29:07.583 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:29:07.583 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:29:07.583 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:29:07.583 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:29:07.584 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:29:07.584 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:29:07.584 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:29:07.584 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:29:07.584 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:29:07.584 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:29:07.584 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:29:07.584 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:29:07.584 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:29:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.581+0000 7fab15767700 1 -- 172.21.15.88:0/1864836146 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faafc06d8a0 msgr2=0x7faafc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.581+0000 7fab15767700 1 --2- 172.21.15.88:0/1864836146 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faafc06d8a0 0x7faafc06fd50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fab10074840 tx=0x7faaf801a040 comp rx=0 tx=0).stop 2024-02-19T19:29:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.581+0000 7fab15767700 1 -- 172.21.15.88:0/1864836146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab101004e0 msgr2=0x7fab10073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.581+0000 7fab15767700 1 --2- 172.21.15.88:0/1864836146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab101004e0 0x7fab10073050 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fab0000ada0 tx=0x7fab0000c330 comp rx=0 tx=0).stop 2024-02-19T19:29:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.581+0000 7fab15767700 1 -- 172.21.15.88:0/1864836146 shutdown_connections 2024-02-19T19:29:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.581+0000 7fab15767700 1 --2- 172.21.15.88:0/1864836146 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faafc06d8a0 0x7faafc06fd50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.581+0000 7fab15767700 1 --2- 172.21.15.88:0/1864836146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab101004e0 0x7fab10073050 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.581+0000 7fab15767700 1 --2- 172.21.15.88:0/1864836146 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fab10100e80 0x7fab10073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.581+0000 7fab15767700 1 -- 172.21.15.88:0/1864836146 >> 172.21.15.88:0/1864836146 conn(0x7fab100fbda0 msgr2=0x7fab100fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.581+0000 7fab15767700 1 -- 172.21.15.88:0/1864836146 shutdown_connections 2024-02-19T19:29:07.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.581+0000 7fab15767700 1 -- 172.21.15.88:0/1864836146 wait complete. 2024-02-19T19:29:07.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.684+0000 7f5d67252700 1 -- 172.21.15.88:0/472413640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d600ffd60 msgr2=0x7f5d60103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.684+0000 7f5d67252700 1 --2- 172.21.15.88:0/472413640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d600ffd60 0x7f5d60103d30 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5d54009910 tx=0x7f5d54009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:07.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.684+0000 7f5d67252700 1 -- 172.21.15.88:0/472413640 shutdown_connections 2024-02-19T19:29:07.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.684+0000 7f5d67252700 1 --2- 172.21.15.88:0/472413640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d600ffd60 0x7f5d60103d30 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.684+0000 7f5d67252700 1 --2- 172.21.15.88:0/472413640 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d600ff3c0 0x7f5d600ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.684+0000 7f5d67252700 1 -- 172.21.15.88:0/472413640 >> 172.21.15.88:0/472413640 conn(0x7f5d600fac80 msgr2=0x7f5d600fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:07.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.684+0000 7f5d67252700 1 -- 172.21.15.88:0/472413640 shutdown_connections 2024-02-19T19:29:07.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.684+0000 7f5d67252700 1 -- 172.21.15.88:0/472413640 wait complete. 2024-02-19T19:29:07.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d67252700 1 Processor -- start 2024-02-19T19:29:07.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d67252700 1 -- start start 2024-02-19T19:29:07.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d67252700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d600ff3c0 0x7f5d60100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:07.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d67252700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d600ffd60 0x7f5d60101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:07.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d67252700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60101850 con 0x7f5d600ffd60 2024-02-19T19:29:07.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d67252700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d60101990 con 0x7f5d600ff3c0 2024-02-19T19:29:07.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d5ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d600ffd60 0x7f5d60101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:07.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d5ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d600ffd60 0x7f5d60101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37116/0 (socket says 172.21.15.88:37116) 2024-02-19T19:29:07.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d5ffff700 1 -- 172.21.15.88:0/2088756331 learned_addr learned my addr 172.21.15.88:0/2088756331 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:07.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d5ffff700 1 -- 172.21.15.88:0/2088756331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d600ff3c0 msgr2=0x7f5d60100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d5ffff700 1 --2- 172.21.15.88:0/2088756331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d600ff3c0 0x7f5d60100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.685+0000 7f5d5ffff700 1 -- 172.21.15.88:0/2088756331 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540095f0 con 0x7f5d600ffd60 2024-02-19T19:29:07.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.686+0000 7f5d5ffff700 1 --2- 172.21.15.88:0/2088756331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d600ffd60 0x7f5d60101310 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400bda0 tx=0x7f5d5400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:07.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.686+0000 7f5d5dffb700 1 -- 172.21.15.88:0/2088756331 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d5401d070 con 0x7f5d600ffd60 2024-02-19T19:29:07.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.686+0000 7f5d5dffb700 1 -- 172.21.15.88:0/2088756331 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d54004fb0 con 0x7f5d600ffd60 2024-02-19T19:29:07.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.686+0000 7f5d67252700 1 -- 172.21.15.88:0/2088756331 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d60101c10 con 0x7f5d600ffd60 2024-02-19T19:29:07.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.686+0000 7f5d5dffb700 1 -- 172.21.15.88:0/2088756331 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d5400f480 con 0x7f5d600ffd60 2024-02-19T19:29:07.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.686+0000 7f5d67252700 1 -- 172.21.15.88:0/2088756331 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d601a2330 con 0x7f5d600ffd60 2024-02-19T19:29:07.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.687+0000 7f5d67252700 1 -- 172.21.15.88:0/2088756331 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d6004f2a0 con 0x7f5d600ffd60 2024-02-19T19:29:07.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.687+0000 7f5d5dffb700 1 -- 172.21.15.88:0/2088756331 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5d54022870 con 0x7f5d600ffd60 2024-02-19T19:29:07.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.688+0000 7f5d5dffb700 1 --2- 172.21.15.88:0/2088756331 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d4806d8a0 0x7f5d4806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:07.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.688+0000 7f5d5dffb700 1 -- 172.21.15.88:0/2088756331 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5d5408ecb0 con 0x7f5d600ffd60 2024-02-19T19:29:07.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.688+0000 7f5d64fee700 1 --2- 172.21.15.88:0/2088756331 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d4806d8a0 0x7f5d4806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:07.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.688+0000 7f5d64fee700 1 --2- 172.21.15.88:0/2088756331 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d4806d8a0 0x7f5d4806fd50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800d0e0 tx=0x7f5d580085a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:07.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.692+0000 7f5d5dffb700 1 -- 172.21.15.88:0/2088756331 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d54058630 con 0x7f5d600ffd60 2024-02-19T19:29:07.835 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:07 smithi088 ceph-mon[88643]: from='client.35398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:07.835 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:07 smithi088 ceph-mon[88643]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:07.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:07 smithi088 ceph-mon[88643]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:07.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:07 smithi088 ceph-mon[88643]: pgmap v767: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:07.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:07 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/247597850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:29:07.836 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:07 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/217688127' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:29:07.937 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:07 smithi132 ceph-mon[70676]: from='client.35398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:07.937 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:07 smithi132 ceph-mon[70676]: from='client.35402 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:07.937 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:07 smithi132 ceph-mon[70676]: from='client.35406 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:07.937 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:07 smithi132 ceph-mon[70676]: pgmap v767: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:07.937 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:07 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/247597850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:29:07.937 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:07 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/217688127' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:29:07.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.971+0000 7f5d67252700 1 -- 172.21.15.88:0/2088756331 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5d601a25c0 con 0x7f5d600ffd60 2024-02-19T19:29:07.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.972+0000 7f5d5dffb700 1 -- 172.21.15.88:0/2088756331 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f5d5405bcf0 con 0x7f5d600ffd60 2024-02-19T19:29:07.976 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:29:07.976 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:29:07.976 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:29:07.976 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:29:07.976 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:29:07.976 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:29:07.976 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:29:07.976 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:29:07.976 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:29:07.977 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:29:07.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.974+0000 7f5d67252700 1 -- 172.21.15.88:0/2088756331 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d4806d8a0 msgr2=0x7f5d4806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.974+0000 7f5d67252700 1 --2- 172.21.15.88:0/2088756331 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d4806d8a0 0x7f5d4806fd50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f5d5800d0e0 tx=0x7f5d580085a0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.974+0000 7f5d67252700 1 -- 172.21.15.88:0/2088756331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d600ffd60 msgr2=0x7f5d60101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.974+0000 7f5d67252700 1 --2- 172.21.15.88:0/2088756331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d600ffd60 0x7f5d60101310 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5d5400bda0 tx=0x7f5d5400be80 comp rx=0 tx=0).stop 2024-02-19T19:29:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.974+0000 7f5d67252700 1 -- 172.21.15.88:0/2088756331 shutdown_connections 2024-02-19T19:29:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.974+0000 7f5d67252700 1 --2- 172.21.15.88:0/2088756331 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d4806d8a0 0x7f5d4806fd50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.974+0000 7f5d67252700 1 --2- 172.21.15.88:0/2088756331 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d600ff3c0 0x7f5d60100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.974+0000 7f5d67252700 1 --2- 172.21.15.88:0/2088756331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d600ffd60 0x7f5d60101310 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.974+0000 7f5d67252700 1 -- 172.21.15.88:0/2088756331 >> 172.21.15.88:0/2088756331 conn(0x7f5d600fac80 msgr2=0x7f5d60068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.974+0000 7f5d67252700 1 -- 172.21.15.88:0/2088756331 shutdown_connections 2024-02-19T19:29:07.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:07.974+0000 7f5d67252700 1 -- 172.21.15.88:0/2088756331 wait complete. 2024-02-19T19:29:08.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:08 smithi132 ceph-mon[70676]: from='client.35418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:08.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:08 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2088756331' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:29:09.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:08 smithi088 ceph-mon[88643]: from='client.35418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:09.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:08 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2088756331' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:29:09.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:09 smithi132 ceph-mon[70676]: pgmap v768: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:10.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:09 smithi088 ceph-mon[88643]: pgmap v768: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:11.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:11 smithi132 ceph-mon[70676]: pgmap v769: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:12.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:11 smithi088 ceph-mon[88643]: pgmap v769: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:12.991 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:12 smithi132 ceph-mon[70676]: pgmap v770: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:13.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:12 smithi088 ceph-mon[88643]: pgmap v770: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:14.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:29:14.236 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:29:14.985 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:14 smithi088 ceph-mon[88643]: pgmap v771: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:15.236 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:14 smithi132 ceph-mon[70676]: pgmap v771: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:17.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:17 smithi132 ceph-mon[70676]: pgmap v772: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:18.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:17 smithi088 ceph-mon[88643]: pgmap v772: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:19.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:19 smithi132 ceph-mon[70676]: pgmap v773: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:20.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:19 smithi088 ceph-mon[88643]: pgmap v773: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:21.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:21 smithi132 ceph-mon[70676]: pgmap v774: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:22.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:21 smithi088 ceph-mon[88643]: pgmap v774: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:23.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:23 smithi132 ceph-mon[70676]: pgmap v775: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:24.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:23 smithi088 ceph-mon[88643]: pgmap v775: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:25.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:25 smithi132 ceph-mon[70676]: pgmap v776: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:26.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:25 smithi088 ceph-mon[88643]: pgmap v776: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:27.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:27 smithi132 ceph-mon[70676]: pgmap v777: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:28.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:27 smithi088 ceph-mon[88643]: pgmap v777: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:28.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:29:29.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:29:29.928 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:29 smithi088 ceph-mon[88643]: pgmap v778: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:29.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:29 smithi132 ceph-mon[70676]: pgmap v778: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:31.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:31 smithi132 ceph-mon[70676]: pgmap v779: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:32.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:31 smithi088 ceph-mon[88643]: pgmap v779: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:33.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:33 smithi132 ceph-mon[70676]: pgmap v780: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:34.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:33 smithi088 ceph-mon[88643]: pgmap v780: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:35.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:35 smithi132 ceph-mon[70676]: pgmap v781: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:36.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:35 smithi088 ceph-mon[88643]: pgmap v781: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:37.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:37 smithi132 ceph-mon[70676]: pgmap v782: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:38.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:37 smithi088 ceph-mon[88643]: pgmap v782: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:38.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.090+0000 7f33a7484700 1 -- 172.21.15.88:0/3298416108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33a0100e80 msgr2=0x7f33a0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.090+0000 7f33a7484700 1 --2- 172.21.15.88:0/3298416108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33a0100e80 0x7f33a0104e50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f3390009910 tx=0x7f3390009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:38.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.090+0000 7f33a7484700 1 -- 172.21.15.88:0/3298416108 shutdown_connections 2024-02-19T19:29:38.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.090+0000 7f33a7484700 1 --2- 172.21.15.88:0/3298416108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33a0100e80 0x7f33a0104e50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.090+0000 7f33a7484700 1 --2- 172.21.15.88:0/3298416108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33a01004e0 0x7f33a01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.090+0000 7f33a7484700 1 -- 172.21.15.88:0/3298416108 >> 172.21.15.88:0/3298416108 conn(0x7f33a00fbda0 msgr2=0x7f33a00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:38.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.090+0000 7f33a7484700 1 -- 172.21.15.88:0/3298416108 shutdown_connections 2024-02-19T19:29:38.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.090+0000 7f33a7484700 1 -- 172.21.15.88:0/3298416108 wait complete. 2024-02-19T19:29:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a7484700 1 Processor -- start 2024-02-19T19:29:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a7484700 1 -- start start 2024-02-19T19:29:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a7484700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33a01004e0 0x7f33a0195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:38.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a7484700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33a0100e80 0x7f33a0196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:38.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a7484700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0196b30 con 0x7f33a0100e80 2024-02-19T19:29:38.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a7484700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a019a8c0 con 0x7f33a01004e0 2024-02-19T19:29:38.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a4a1f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33a0100e80 0x7f33a0196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:38.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a4a1f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33a0100e80 0x7f33a0196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36756/0 (socket says 172.21.15.88:36756) 2024-02-19T19:29:38.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a4a1f700 1 -- 172.21.15.88:0/1518204316 learned_addr learned my addr 172.21.15.88:0/1518204316 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:38.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a5220700 1 --2- 172.21.15.88:0/1518204316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33a01004e0 0x7f33a0195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:38.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a4a1f700 1 -- 172.21.15.88:0/1518204316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33a01004e0 msgr2=0x7f33a0195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a4a1f700 1 --2- 172.21.15.88:0/1518204316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33a01004e0 0x7f33a0195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.091+0000 7f33a4a1f700 1 -- 172.21.15.88:0/1518204316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33900095f0 con 0x7f33a0100e80 2024-02-19T19:29:38.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.092+0000 7f33a4a1f700 1 --2- 172.21.15.88:0/1518204316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33a0100e80 0x7f33a0196450 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f33900092e0 tx=0x7f339000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:38.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.092+0000 7f33967fc700 1 -- 172.21.15.88:0/1518204316 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f339001d070 con 0x7f33a0100e80 2024-02-19T19:29:38.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.092+0000 7f33a7484700 1 -- 172.21.15.88:0/1518204316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a019ab40 con 0x7f33a0100e80 2024-02-19T19:29:38.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.092+0000 7f33967fc700 1 -- 172.21.15.88:0/1518204316 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f339000f460 con 0x7f33a0100e80 2024-02-19T19:29:38.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.092+0000 7f33967fc700 1 -- 172.21.15.88:0/1518204316 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33900175d0 con 0x7f33a0100e80 2024-02-19T19:29:38.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.092+0000 7f33a7484700 1 -- 172.21.15.88:0/1518204316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a019b030 con 0x7f33a0100e80 2024-02-19T19:29:38.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.093+0000 7f33a7484700 1 -- 172.21.15.88:0/1518204316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a0108850 con 0x7f33a0100e80 2024-02-19T19:29:38.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.093+0000 7f33967fc700 1 -- 172.21.15.88:0/1518204316 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f339000f5d0 con 0x7f33a0100e80 2024-02-19T19:29:38.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.094+0000 7f33967fc700 1 --2- 172.21.15.88:0/1518204316 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f338c06d8a0 0x7f338c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:38.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.094+0000 7f33967fc700 1 -- 172.21.15.88:0/1518204316 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f339008ea00 con 0x7f33a0100e80 2024-02-19T19:29:38.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.094+0000 7f33a5220700 1 --2- 172.21.15.88:0/1518204316 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f338c06d8a0 0x7f338c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:38.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.094+0000 7f33a5220700 1 --2- 172.21.15.88:0/1518204316 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f338c06d8a0 0x7f338c06fd50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f339c0060b0 tx=0x7f339c00c040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:38.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.099+0000 7f33967fc700 1 -- 172.21.15.88:0/1518204316 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f33900581b0 con 0x7f33a0100e80 2024-02-19T19:29:38.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.313+0000 7f33a7484700 1 -- 172.21.15.88:0/1518204316 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f33a019b250 con 0x7f338c06d8a0 2024-02-19T19:29:38.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.314+0000 7f33967fc700 1 -- 172.21.15.88:0/1518204316 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f33a019b250 con 0x7f338c06d8a0 2024-02-19T19:29:38.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.316+0000 7f33a7484700 1 -- 172.21.15.88:0/1518204316 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f338c06d8a0 msgr2=0x7f338c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.316+0000 7f33a7484700 1 --2- 172.21.15.88:0/1518204316 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f338c06d8a0 0x7f338c06fd50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f339c0060b0 tx=0x7f339c00c040 comp rx=0 tx=0).stop 2024-02-19T19:29:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.316+0000 7f33a7484700 1 -- 172.21.15.88:0/1518204316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33a0100e80 msgr2=0x7f33a0196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.316+0000 7f33a7484700 1 --2- 172.21.15.88:0/1518204316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33a0100e80 0x7f33a0196450 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f33900092e0 tx=0x7f339000bd70 comp rx=0 tx=0).stop 2024-02-19T19:29:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.317+0000 7f33a7484700 1 -- 172.21.15.88:0/1518204316 shutdown_connections 2024-02-19T19:29:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.317+0000 7f33a7484700 1 --2- 172.21.15.88:0/1518204316 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f338c06d8a0 0x7f338c06fd50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.317+0000 7f33a7484700 1 --2- 172.21.15.88:0/1518204316 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f33a01004e0 0x7f33a0195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.317+0000 7f33a7484700 1 --2- 172.21.15.88:0/1518204316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33a0100e80 0x7f33a0196450 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.317+0000 7f33a7484700 1 -- 172.21.15.88:0/1518204316 >> 172.21.15.88:0/1518204316 conn(0x7f33a00fbda0 msgr2=0x7f33a00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.317+0000 7f33a7484700 1 -- 172.21.15.88:0/1518204316 shutdown_connections 2024-02-19T19:29:38.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.317+0000 7f33a7484700 1 -- 172.21.15.88:0/1518204316 wait complete. 2024-02-19T19:29:38.332 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:29:38.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.420+0000 7ff546d48700 1 -- 172.21.15.88:0/1920468115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff540100e80 msgr2=0x7ff540104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.420+0000 7ff546d48700 1 --2- 172.21.15.88:0/1920468115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff540100e80 0x7ff540104e50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff52c009910 tx=0x7ff52c009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:38.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.420+0000 7ff546d48700 1 -- 172.21.15.88:0/1920468115 shutdown_connections 2024-02-19T19:29:38.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.420+0000 7ff546d48700 1 --2- 172.21.15.88:0/1920468115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff540100e80 0x7ff540104e50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.420+0000 7ff546d48700 1 --2- 172.21.15.88:0/1920468115 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5401004e0 0x7ff5401008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.420+0000 7ff546d48700 1 -- 172.21.15.88:0/1920468115 >> 172.21.15.88:0/1920468115 conn(0x7ff5400fbda0 msgr2=0x7ff5400fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:38.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.420+0000 7ff546d48700 1 -- 172.21.15.88:0/1920468115 shutdown_connections 2024-02-19T19:29:38.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.420+0000 7ff546d48700 1 -- 172.21.15.88:0/1920468115 wait complete. 2024-02-19T19:29:38.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.421+0000 7ff546d48700 1 Processor -- start 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.421+0000 7ff546d48700 1 -- start start 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.421+0000 7ff546d48700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5401004e0 0x7ff540195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.421+0000 7ff546d48700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff540100e80 0x7ff540196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.421+0000 7ff546d48700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff540196aa0 con 0x7ff540100e80 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.421+0000 7ff546d48700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff540196be0 con 0x7ff5401004e0 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.421+0000 7ff53ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff540100e80 0x7ff540196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.421+0000 7ff53ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff540100e80 0x7ff540196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36784/0 (socket says 172.21.15.88:36784) 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.421+0000 7ff53ffff700 1 -- 172.21.15.88:0/3368122784 learned_addr learned my addr 172.21.15.88:0/3368122784 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.422+0000 7ff544ae4700 1 --2- 172.21.15.88:0/3368122784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5401004e0 0x7ff540195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.422+0000 7ff53ffff700 1 -- 172.21.15.88:0/3368122784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5401004e0 msgr2=0x7ff540195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.422+0000 7ff53ffff700 1 --2- 172.21.15.88:0/3368122784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5401004e0 0x7ff540195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.422+0000 7ff53ffff700 1 -- 172.21.15.88:0/3368122784 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff52c0095f0 con 0x7ff540100e80 2024-02-19T19:29:38.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.422+0000 7ff53ffff700 1 --2- 172.21.15.88:0/3368122784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff540100e80 0x7ff540196450 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00bda0 tx=0x7ff52c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:38.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.422+0000 7ff53dffb700 1 -- 172.21.15.88:0/3368122784 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff52c01d070 con 0x7ff540100e80 2024-02-19T19:29:38.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.422+0000 7ff53dffb700 1 -- 172.21.15.88:0/3368122784 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff52c00f460 con 0x7ff540100e80 2024-02-19T19:29:38.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.422+0000 7ff546d48700 1 -- 172.21.15.88:0/3368122784 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54019a9d0 con 0x7ff540100e80 2024-02-19T19:29:38.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.422+0000 7ff53dffb700 1 -- 172.21.15.88:0/3368122784 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff52c017630 con 0x7ff540100e80 2024-02-19T19:29:38.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.423+0000 7ff546d48700 1 -- 172.21.15.88:0/3368122784 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54019aec0 con 0x7ff540100e80 2024-02-19T19:29:38.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.423+0000 7ff546d48700 1 -- 172.21.15.88:0/3368122784 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff540108850 con 0x7ff540100e80 2024-02-19T19:29:38.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.425+0000 7ff53dffb700 1 -- 172.21.15.88:0/3368122784 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff52c00fab0 con 0x7ff540100e80 2024-02-19T19:29:38.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.425+0000 7ff53dffb700 1 --2- 172.21.15.88:0/3368122784 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff53006d8a0 0x7ff53006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:38.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.425+0000 7ff53dffb700 1 -- 172.21.15.88:0/3368122784 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff52c08e910 con 0x7ff540100e80 2024-02-19T19:29:38.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.425+0000 7ff544ae4700 1 --2- 172.21.15.88:0/3368122784 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff53006d8a0 0x7ff53006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:38.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.426+0000 7ff544ae4700 1 --2- 172.21.15.88:0/3368122784 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff53006d8a0 0x7ff53006fd50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff53400ac50 tx=0x7ff53400a380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:38.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.430+0000 7ff53dffb700 1 -- 172.21.15.88:0/3368122784 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff52c058140 con 0x7ff540100e80 2024-02-19T19:29:38.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.653+0000 7ff546d48700 1 -- 172.21.15.88:0/3368122784 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff54019b1a0 con 0x7ff53006d8a0 2024-02-19T19:29:38.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.654+0000 7ff53dffb700 1 -- 172.21.15.88:0/3368122784 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff54019b1a0 con 0x7ff53006d8a0 2024-02-19T19:29:38.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.656+0000 7ff546d48700 1 -- 172.21.15.88:0/3368122784 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff53006d8a0 msgr2=0x7ff53006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.656+0000 7ff546d48700 1 --2- 172.21.15.88:0/3368122784 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff53006d8a0 0x7ff53006fd50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7ff53400ac50 tx=0x7ff53400a380 comp rx=0 tx=0).stop 2024-02-19T19:29:38.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.657+0000 7ff546d48700 1 -- 172.21.15.88:0/3368122784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff540100e80 msgr2=0x7ff540196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.657+0000 7ff546d48700 1 --2- 172.21.15.88:0/3368122784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff540100e80 0x7ff540196450 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff52c00bda0 tx=0x7ff52c00bdd0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.657+0000 7ff546d48700 1 -- 172.21.15.88:0/3368122784 shutdown_connections 2024-02-19T19:29:38.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.657+0000 7ff546d48700 1 --2- 172.21.15.88:0/3368122784 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff53006d8a0 0x7ff53006fd50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.657+0000 7ff546d48700 1 --2- 172.21.15.88:0/3368122784 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff5401004e0 0x7ff540195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.657+0000 7ff546d48700 1 --2- 172.21.15.88:0/3368122784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff540100e80 0x7ff540196450 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.657+0000 7ff546d48700 1 -- 172.21.15.88:0/3368122784 >> 172.21.15.88:0/3368122784 conn(0x7ff5400fbda0 msgr2=0x7ff5400fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:38.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.657+0000 7ff546d48700 1 -- 172.21.15.88:0/3368122784 shutdown_connections 2024-02-19T19:29:38.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.657+0000 7ff546d48700 1 -- 172.21.15.88:0/3368122784 wait complete. 2024-02-19T19:29:38.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.760+0000 7fd1229f1700 1 -- 172.21.15.88:0/3856481600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c1004e0 msgr2=0x7fd11c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.760+0000 7fd1229f1700 1 --2- 172.21.15.88:0/3856481600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c1004e0 0x7fd11c1008b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd10c009910 tx=0x7fd10c009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.760+0000 7fd1229f1700 1 -- 172.21.15.88:0/3856481600 shutdown_connections 2024-02-19T19:29:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.760+0000 7fd1229f1700 1 --2- 172.21.15.88:0/3856481600 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd11c100e80 0x7fd11c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.760+0000 7fd1229f1700 1 --2- 172.21.15.88:0/3856481600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c1004e0 0x7fd11c1008b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.760+0000 7fd1229f1700 1 -- 172.21.15.88:0/3856481600 >> 172.21.15.88:0/3856481600 conn(0x7fd11c0fbda0 msgr2=0x7fd11c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.760+0000 7fd1229f1700 1 -- 172.21.15.88:0/3856481600 shutdown_connections 2024-02-19T19:29:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.760+0000 7fd1229f1700 1 -- 172.21.15.88:0/3856481600 wait complete. 2024-02-19T19:29:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.761+0000 7fd1229f1700 1 Processor -- start 2024-02-19T19:29:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.761+0000 7fd1229f1700 1 -- start start 2024-02-19T19:29:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd1229f1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd11c1004e0 0x7fd11c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd1229f1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c100e80 0x7fd11c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd1229f1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c196b10 con 0x7fd11c100e80 2024-02-19T19:29:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd1229f1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c19a8a0 con 0x7fd11c1004e0 2024-02-19T19:29:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd11b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c100e80 0x7fd11c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd11b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c100e80 0x7fd11c196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36798/0 (socket says 172.21.15.88:36798) 2024-02-19T19:29:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd11b7fe700 1 -- 172.21.15.88:0/367979180 learned_addr learned my addr 172.21.15.88:0/367979180 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd11bfff700 1 --2- 172.21.15.88:0/367979180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd11c1004e0 0x7fd11c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd11b7fe700 1 -- 172.21.15.88:0/367979180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd11c1004e0 msgr2=0x7fd11c195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd11b7fe700 1 --2- 172.21.15.88:0/367979180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd11c1004e0 0x7fd11c195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd11b7fe700 1 -- 172.21.15.88:0/367979180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10c0095f0 con 0x7fd11c100e80 2024-02-19T19:29:38.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd11b7fe700 1 --2- 172.21.15.88:0/367979180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c100e80 0x7fd11c196430 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd11000c8f0 tx=0x7fd11000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:38.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.762+0000 7fd1197fa700 1 -- 172.21.15.88:0/367979180 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1100078a0 con 0x7fd11c100e80 2024-02-19T19:29:38.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.763+0000 7fd1229f1700 1 -- 172.21.15.88:0/367979180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11c19ab80 con 0x7fd11c100e80 2024-02-19T19:29:38.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.763+0000 7fd1197fa700 1 -- 172.21.15.88:0/367979180 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd11000f450 con 0x7fd11c100e80 2024-02-19T19:29:38.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.763+0000 7fd1197fa700 1 -- 172.21.15.88:0/367979180 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd110018610 con 0x7fd11c100e80 2024-02-19T19:29:38.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.763+0000 7fd1229f1700 1 -- 172.21.15.88:0/367979180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11c19b0d0 con 0x7fd11c100e80 2024-02-19T19:29:38.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.764+0000 7fd1229f1700 1 -- 172.21.15.88:0/367979180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11c1087f0 con 0x7fd11c100e80 2024-02-19T19:29:38.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.764+0000 7fd1197fa700 1 -- 172.21.15.88:0/367979180 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd11000f5c0 con 0x7fd11c100e80 2024-02-19T19:29:38.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.765+0000 7fd1197fa700 1 --2- 172.21.15.88:0/367979180 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd10406d8a0 0x7fd10406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:38.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.765+0000 7fd1197fa700 1 -- 172.21.15.88:0/367979180 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd11008d4a0 con 0x7fd11c100e80 2024-02-19T19:29:38.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.765+0000 7fd11bfff700 1 --2- 172.21.15.88:0/367979180 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd10406d8a0 0x7fd10406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:38.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.765+0000 7fd11bfff700 1 --2- 172.21.15.88:0/367979180 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd10406d8a0 0x7fd10406fd50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd10c006010 tx=0x7fd10c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:38.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.769+0000 7fd1197fa700 1 -- 172.21.15.88:0/367979180 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd110056c50 con 0x7fd11c100e80 2024-02-19T19:29:38.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.976+0000 7fd1229f1700 1 -- 172.21.15.88:0/367979180 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd11c19b350 con 0x7fd10406d8a0 2024-02-19T19:29:38.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.987+0000 7fd1197fa700 1 -- 172.21.15.88:0/367979180 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fd11c19b350 con 0x7fd10406d8a0 2024-02-19T19:29:38.992 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:29:38.992 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (26m) 3m ago 37m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:29:38.992 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (37m) 3m ago 37m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:29:38.992 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (19m) 3m ago 35m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:29:38.992 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (22m) 3m ago 37m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:29:38.992 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (22m) 3m ago 35m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:29:38.992 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (26m) 3m ago 36m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:29:38.992 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (19m) 3m ago 31m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:29:38.992 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (18m) 3m ago 30m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (18m) 3m ago 30m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (3m) 3m ago 30m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (28m) 3m ago 38m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (27m) 3m ago 35m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (24m) 3m ago 39m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (24m) 3m ago 35m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (27m) 3m ago 37m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (27m) 3m ago 35m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (22m) 3m ago 34m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (21m) 3m ago 34m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (21m) 3m ago 33m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (20m) 3m ago 33m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (19m) 3m ago 32m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (19m) 3m ago 32m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:29:38.993 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (27m) 3m ago 36m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:29:38.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.989+0000 7fd1229f1700 1 -- 172.21.15.88:0/367979180 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd10406d8a0 msgr2=0x7fd10406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.989+0000 7fd1229f1700 1 --2- 172.21.15.88:0/367979180 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd10406d8a0 0x7fd10406fd50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fd10c006010 tx=0x7fd10c01a040 comp rx=0 tx=0).stop 2024-02-19T19:29:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.989+0000 7fd1229f1700 1 -- 172.21.15.88:0/367979180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c100e80 msgr2=0x7fd11c196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.989+0000 7fd1229f1700 1 --2- 172.21.15.88:0/367979180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c100e80 0x7fd11c196430 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fd11000c8f0 tx=0x7fd11000cc00 comp rx=0 tx=0).stop 2024-02-19T19:29:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.989+0000 7fd1229f1700 1 -- 172.21.15.88:0/367979180 shutdown_connections 2024-02-19T19:29:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.989+0000 7fd1229f1700 1 --2- 172.21.15.88:0/367979180 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd10406d8a0 0x7fd10406fd50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.989+0000 7fd1229f1700 1 --2- 172.21.15.88:0/367979180 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd11c1004e0 0x7fd11c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.989+0000 7fd1229f1700 1 --2- 172.21.15.88:0/367979180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd11c100e80 0x7fd11c196430 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.989+0000 7fd1229f1700 1 -- 172.21.15.88:0/367979180 >> 172.21.15.88:0/367979180 conn(0x7fd11c0fbda0 msgr2=0x7fd11c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.989+0000 7fd1229f1700 1 -- 172.21.15.88:0/367979180 shutdown_connections 2024-02-19T19:29:38.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:38.989+0000 7fd1229f1700 1 -- 172.21.15.88:0/367979180 wait complete. 2024-02-19T19:29:39.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.092+0000 7fd0269a2700 1 -- 172.21.15.88:0/3012304342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0201004e0 msgr2=0x7fd0201008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:39.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.092+0000 7fd0269a2700 1 --2- 172.21.15.88:0/3012304342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0201004e0 0x7fd0201008b0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fd010009910 tx=0x7fd010009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:39.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.093+0000 7fd0269a2700 1 -- 172.21.15.88:0/3012304342 shutdown_connections 2024-02-19T19:29:39.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.093+0000 7fd0269a2700 1 --2- 172.21.15.88:0/3012304342 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd020100e80 0x7fd020104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.093+0000 7fd0269a2700 1 --2- 172.21.15.88:0/3012304342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0201004e0 0x7fd0201008b0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.093+0000 7fd0269a2700 1 -- 172.21.15.88:0/3012304342 >> 172.21.15.88:0/3012304342 conn(0x7fd0200fbda0 msgr2=0x7fd0200fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:39.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.093+0000 7fd0269a2700 1 -- 172.21.15.88:0/3012304342 shutdown_connections 2024-02-19T19:29:39.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.093+0000 7fd0269a2700 1 -- 172.21.15.88:0/3012304342 wait complete. 2024-02-19T19:29:39.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.093+0000 7fd0269a2700 1 Processor -- start 2024-02-19T19:29:39.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.093+0000 7fd0269a2700 1 -- start start 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd0269a2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0201004e0 0x7fd020195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd0269a2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd020100e80 0x7fd0201964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd0269a2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd020196bb0 con 0x7fd0201004e0 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd0269a2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd02019a940 con 0x7fd020100e80 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd01ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0201004e0 0x7fd020195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd01ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0201004e0 0x7fd020195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36810/0 (socket says 172.21.15.88:36810) 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd01ffff700 1 -- 172.21.15.88:0/2953212892 learned_addr learned my addr 172.21.15.88:0/2953212892 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd01ffff700 1 -- 172.21.15.88:0/2953212892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd020100e80 msgr2=0x7fd0201964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd01f7fe700 1 --2- 172.21.15.88:0/2953212892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd020100e80 0x7fd0201964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd01ffff700 1 --2- 172.21.15.88:0/2953212892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd020100e80 0x7fd0201964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd01ffff700 1 -- 172.21.15.88:0/2953212892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0100095f0 con 0x7fd0201004e0 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd01ffff700 1 --2- 172.21.15.88:0/2953212892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0201004e0 0x7fd020195f90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd010006010 tx=0x7fd01000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd01d7fa700 1 -- 172.21.15.88:0/2953212892 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd01001d070 con 0x7fd0201004e0 2024-02-19T19:29:39.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd0269a2700 1 -- 172.21.15.88:0/2953212892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd02019abc0 con 0x7fd0201004e0 2024-02-19T19:29:39.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd01d7fa700 1 -- 172.21.15.88:0/2953212892 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd010004fb0 con 0x7fd0201004e0 2024-02-19T19:29:39.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd01d7fa700 1 -- 172.21.15.88:0/2953212892 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd010022ea0 con 0x7fd0201004e0 2024-02-19T19:29:39.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.094+0000 7fd0269a2700 1 -- 172.21.15.88:0/2953212892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd02019b0b0 con 0x7fd0201004e0 2024-02-19T19:29:39.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.095+0000 7fd0269a2700 1 -- 172.21.15.88:0/2953212892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0201087f0 con 0x7fd0201004e0 2024-02-19T19:29:39.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.096+0000 7fd01d7fa700 1 -- 172.21.15.88:0/2953212892 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd010022470 con 0x7fd0201004e0 2024-02-19T19:29:39.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.096+0000 7fd01d7fa700 1 --2- 172.21.15.88:0/2953212892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00806d8a0 0x7fd00806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:39.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.096+0000 7fd01f7fe700 1 --2- 172.21.15.88:0/2953212892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00806d8a0 0x7fd00806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:39.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.096+0000 7fd01d7fa700 1 -- 172.21.15.88:0/2953212892 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd01008ec10 con 0x7fd0201004e0 2024-02-19T19:29:39.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.097+0000 7fd01f7fe700 1 --2- 172.21.15.88:0/2953212892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00806d8a0 0x7fd00806fd50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd014006fd0 tx=0x7fd014008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:39.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.101+0000 7fd01d7fa700 1 -- 172.21.15.88:0/2953212892 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd010058590 con 0x7fd0201004e0 2024-02-19T19:29:39.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.379+0000 7fd0269a2700 1 -- 172.21.15.88:0/2953212892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd02004f2a0 con 0x7fd0201004e0 2024-02-19T19:29:39.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.380+0000 7fd01d7fa700 1 -- 172.21.15.88:0/2953212892 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fd010027070 con 0x7fd0201004e0 2024-02-19T19:29:39.385 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:29:39.385 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:29:39.385 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:29:39.385 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:29:39.385 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:29:39.385 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:29:39.385 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:29:39.385 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:29:39.385 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:29:39.386 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:29:39.386 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:29:39.386 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:29:39.386 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:29:39.386 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:29:39.386 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:29:39.386 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:29:39.386 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:29:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.382+0000 7fd0269a2700 1 -- 172.21.15.88:0/2953212892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00806d8a0 msgr2=0x7fd00806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.382+0000 7fd0269a2700 1 --2- 172.21.15.88:0/2953212892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00806d8a0 0x7fd00806fd50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fd014006fd0 tx=0x7fd014008040 comp rx=0 tx=0).stop 2024-02-19T19:29:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.382+0000 7fd0269a2700 1 -- 172.21.15.88:0/2953212892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0201004e0 msgr2=0x7fd020195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.382+0000 7fd0269a2700 1 --2- 172.21.15.88:0/2953212892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0201004e0 0x7fd020195f90 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fd010006010 tx=0x7fd01000be20 comp rx=0 tx=0).stop 2024-02-19T19:29:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.382+0000 7fd0269a2700 1 -- 172.21.15.88:0/2953212892 shutdown_connections 2024-02-19T19:29:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.382+0000 7fd0269a2700 1 --2- 172.21.15.88:0/2953212892 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd00806d8a0 0x7fd00806fd50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.382+0000 7fd0269a2700 1 --2- 172.21.15.88:0/2953212892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd0201004e0 0x7fd020195f90 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.382+0000 7fd0269a2700 1 --2- 172.21.15.88:0/2953212892 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd020100e80 0x7fd0201964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.382+0000 7fd0269a2700 1 -- 172.21.15.88:0/2953212892 >> 172.21.15.88:0/2953212892 conn(0x7fd0200fbda0 msgr2=0x7fd0200fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.383+0000 7fd0269a2700 1 -- 172.21.15.88:0/2953212892 shutdown_connections 2024-02-19T19:29:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.383+0000 7fd0269a2700 1 -- 172.21.15.88:0/2953212892 wait complete. 2024-02-19T19:29:39.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.486+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3909270677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d94104130 msgr2=0x7f6d94104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:39.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.486+0000 7f6d99d4e700 1 --2- 172.21.15.88:0/3909270677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d94104130 0x7f6d94104500 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f6d88009910 tx=0x7f6d88009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:39.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.486+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3909270677 shutdown_connections 2024-02-19T19:29:39.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.486+0000 7f6d99d4e700 1 --2- 172.21.15.88:0/3909270677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d940ffbd0 0x7f6d94100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.486+0000 7f6d99d4e700 1 --2- 172.21.15.88:0/3909270677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d94104130 0x7f6d94104500 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.486+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3909270677 >> 172.21.15.88:0/3909270677 conn(0x7f6d94075130 msgr2=0x7f6d94075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:39.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.486+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3909270677 shutdown_connections 2024-02-19T19:29:39.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.486+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3909270677 wait complete. 2024-02-19T19:29:39.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d99d4e700 1 Processor -- start 2024-02-19T19:29:39.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d99d4e700 1 -- start start 2024-02-19T19:29:39.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d99d4e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d940ffbd0 0x7f6d94194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:39.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d99d4e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d94104130 0x7f6d94195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:39.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d99d4e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d94195960 con 0x7f6d94104130 2024-02-19T19:29:39.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d99d4e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d941996f0 con 0x7f6d940ffbd0 2024-02-19T19:29:39.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d92ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d94104130 0x7f6d94195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:39.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d92ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d94104130 0x7f6d94195280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36826/0 (socket says 172.21.15.88:36826) 2024-02-19T19:29:39.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d92ffd700 1 -- 172.21.15.88:0/3000760718 learned_addr learned my addr 172.21.15.88:0/3000760718 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:39.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d937fe700 1 --2- 172.21.15.88:0/3000760718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d940ffbd0 0x7f6d94194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:39.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d92ffd700 1 -- 172.21.15.88:0/3000760718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d940ffbd0 msgr2=0x7f6d94194d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:39.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d92ffd700 1 --2- 172.21.15.88:0/3000760718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d940ffbd0 0x7f6d94194d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.487+0000 7f6d92ffd700 1 -- 172.21.15.88:0/3000760718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d880095f0 con 0x7f6d94104130 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.488+0000 7f6d92ffd700 1 --2- 172.21.15.88:0/3000760718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d94104130 0x7f6d94195280 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400d8d0 tx=0x7f6d8400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.488+0000 7f6d90ff9700 1 -- 172.21.15.88:0/3000760718 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d840098a0 con 0x7f6d94104130 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.488+0000 7f6d90ff9700 1 -- 172.21.15.88:0/3000760718 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d84010460 con 0x7f6d94104130 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.488+0000 7f6d90ff9700 1 -- 172.21.15.88:0/3000760718 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d8400f5f0 con 0x7f6d94104130 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.488+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3000760718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d941999d0 con 0x7f6d94104130 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.488+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3000760718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d94199f20 con 0x7f6d94104130 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.489+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3000760718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d941076a0 con 0x7f6d94104130 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.489+0000 7f6d90ff9700 1 -- 172.21.15.88:0/3000760718 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6d84009a00 con 0x7f6d94104130 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.490+0000 7f6d90ff9700 1 --2- 172.21.15.88:0/3000760718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d7c06d8a0 0x7f6d7c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.490+0000 7f6d937fe700 1 --2- 172.21.15.88:0/3000760718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d7c06d8a0 0x7f6d7c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.490+0000 7f6d90ff9700 1 -- 172.21.15.88:0/3000760718 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6d8408d340 con 0x7f6d94104130 2024-02-19T19:29:39.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.490+0000 7f6d937fe700 1 --2- 172.21.15.88:0/3000760718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d7c06d8a0 0x7f6d7c06fd50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000c00 tx=0x7f6d8801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:39.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.494+0000 7f6d90ff9700 1 -- 172.21.15.88:0/3000760718 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6d84056b70 con 0x7f6d94104130 2024-02-19T19:29:39.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.733+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3000760718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6d9404f2a0 con 0x7f6d94104130 2024-02-19T19:29:39.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.733+0000 7f6d90ff9700 1 -- 172.21.15.88:0/3000760718 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f6d84016740 con 0x7f6d94104130 2024-02-19T19:29:39.738 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:29:39.738 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:29:39.738 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:29:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:29:39.740 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:29:39.741 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:29:39.741 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:29:39.741 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:29:39.741 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:29:39.741 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:29:39.741 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:29:39.741 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:29:39.741 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:29:39.741 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:29:39.741 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:29:39.741 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:29:39.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.735+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3000760718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d7c06d8a0 msgr2=0x7f6d7c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:39.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.735+0000 7f6d99d4e700 1 --2- 172.21.15.88:0/3000760718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d7c06d8a0 0x7f6d7c06fd50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6d88000c00 tx=0x7f6d8801a040 comp rx=0 tx=0).stop 2024-02-19T19:29:39.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.735+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3000760718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d94104130 msgr2=0x7f6d94195280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:39.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.735+0000 7f6d99d4e700 1 --2- 172.21.15.88:0/3000760718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d94104130 0x7f6d94195280 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400d8d0 tx=0x7f6d8400dbe0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.735+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3000760718 shutdown_connections 2024-02-19T19:29:39.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.735+0000 7f6d99d4e700 1 --2- 172.21.15.88:0/3000760718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d7c06d8a0 0x7f6d7c06fd50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.735+0000 7f6d99d4e700 1 --2- 172.21.15.88:0/3000760718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d940ffbd0 0x7f6d94194d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.735+0000 7f6d99d4e700 1 --2- 172.21.15.88:0/3000760718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d94104130 0x7f6d94195280 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.735+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3000760718 >> 172.21.15.88:0/3000760718 conn(0x7f6d94075130 msgr2=0x7f6d940fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:39.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.735+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3000760718 shutdown_connections 2024-02-19T19:29:39.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.735+0000 7f6d99d4e700 1 -- 172.21.15.88:0/3000760718 wait complete. 2024-02-19T19:29:39.743 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:29:39.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.841+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2899182691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c480ffbd0 msgr2=0x7f9c48100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:39.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.841+0000 7f9c4dedb700 1 --2- 172.21.15.88:0/2899182691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c480ffbd0 0x7f9c48100040 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c009910 tx=0x7f9c3c009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:39.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.841+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2899182691 shutdown_connections 2024-02-19T19:29:39.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.841+0000 7f9c4dedb700 1 --2- 172.21.15.88:0/2899182691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c480ffbd0 0x7f9c48100040 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.841+0000 7f9c4dedb700 1 --2- 172.21.15.88:0/2899182691 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c48104130 0x7f9c48104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.841+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2899182691 >> 172.21.15.88:0/2899182691 conn(0x7f9c48075130 msgr2=0x7f9c48075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:39.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.841+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2899182691 shutdown_connections 2024-02-19T19:29:39.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.841+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2899182691 wait complete. 2024-02-19T19:29:39.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.841+0000 7f9c4dedb700 1 Processor -- start 2024-02-19T19:29:39.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c4dedb700 1 -- start start 2024-02-19T19:29:39.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c4dedb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c480ffbd0 0x7f9c48196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:39.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c4dedb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c48104130 0x7f9c481972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c4dedb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c48197900 con 0x7f9c48104130 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c4dedb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c48197a70 con 0x7f9c480ffbd0 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c46ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c48104130 0x7f9c481972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c46ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c48104130 0x7f9c481972e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36830/0 (socket says 172.21.15.88:36830) 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c46ffd700 1 -- 172.21.15.88:0/2198292936 learned_addr learned my addr 172.21.15.88:0/2198292936 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c477fe700 1 --2- 172.21.15.88:0/2198292936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c480ffbd0 0x7f9c48196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c46ffd700 1 -- 172.21.15.88:0/2198292936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c480ffbd0 msgr2=0x7f9c48196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c46ffd700 1 --2- 172.21.15.88:0/2198292936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c480ffbd0 0x7f9c48196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.842+0000 7f9c46ffd700 1 -- 172.21.15.88:0/2198292936 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0095f0 con 0x7f9c48104130 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.843+0000 7f9c46ffd700 1 --2- 172.21.15.88:0/2198292936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c48104130 0x7f9c481972e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00bd70 tx=0x7f9c3c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.843+0000 7f9c44ff9700 1 -- 172.21.15.88:0/2198292936 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c3c01d070 con 0x7f9c48104130 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.843+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2198292936 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4819fe10 con 0x7f9c48104130 2024-02-19T19:29:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.843+0000 7f9c44ff9700 1 -- 172.21.15.88:0/2198292936 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c3c00f460 con 0x7f9c48104130 2024-02-19T19:29:39.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.843+0000 7f9c44ff9700 1 -- 172.21.15.88:0/2198292936 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c3c017630 con 0x7f9c48104130 2024-02-19T19:29:39.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.843+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2198292936 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c481a0280 con 0x7f9c48104130 2024-02-19T19:29:39.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.844+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2198292936 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4804f2a0 con 0x7f9c48104130 2024-02-19T19:29:39.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.844+0000 7f9c44ff9700 1 -- 172.21.15.88:0/2198292936 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9c3c00f5d0 con 0x7f9c48104130 2024-02-19T19:29:39.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.844+0000 7f9c44ff9700 1 --2- 172.21.15.88:0/2198292936 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c3006d8a0 0x7f9c3006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:39.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.845+0000 7f9c44ff9700 1 -- 172.21.15.88:0/2198292936 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9c3c08eaa0 con 0x7f9c48104130 2024-02-19T19:29:39.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.845+0000 7f9c477fe700 1 --2- 172.21.15.88:0/2198292936 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c3006d8a0 0x7f9c3006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:39.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.845+0000 7f9c477fe700 1 --2- 172.21.15.88:0/2198292936 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c3006d8a0 0x7f9c3006fd50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009a10 tx=0x7f9c38009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:39.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:39.849+0000 7f9c44ff9700 1 -- 172.21.15.88:0/2198292936 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c3c058250 con 0x7f9c48104130 2024-02-19T19:29:39.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:39 smithi132 ceph-mon[70676]: from='client.35426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:39.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:39 smithi132 ceph-mon[70676]: pgmap v783: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:39.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:39 smithi132 ceph-mon[70676]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:39.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:39 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2953212892' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:29:40.033 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:39 smithi088 ceph-mon[88643]: from='client.35426 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:40.033 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:39 smithi088 ceph-mon[88643]: pgmap v783: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:40.033 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:39 smithi088 ceph-mon[88643]: from='client.35430 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:40.033 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:39 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2953212892' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:29:40.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.071+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2198292936 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c480ff3a0 con 0x7f9c3006d8a0 2024-02-19T19:29:40.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.072+0000 7f9c44ff9700 1 -- 172.21.15.88:0/2198292936 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9c480ff3a0 con 0x7f9c3006d8a0 2024-02-19T19:29:40.077 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:29:40.077 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:29:40.077 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:29:40.077 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:29:40.077 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:29:40.077 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:29:40.077 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:29:40.077 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:29:40.077 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:29:40.077 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:29:40.077 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:29:40.078 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:29:40.078 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:29:40.078 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:29:40.078 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:29:40.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.074+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2198292936 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c3006d8a0 msgr2=0x7f9c3006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:40.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.074+0000 7f9c4dedb700 1 --2- 172.21.15.88:0/2198292936 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c3006d8a0 0x7f9c3006fd50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9c38009a10 tx=0x7f9c38009190 comp rx=0 tx=0).stop 2024-02-19T19:29:40.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.074+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2198292936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c48104130 msgr2=0x7f9c481972e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.074+0000 7f9c4dedb700 1 --2- 172.21.15.88:0/2198292936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c48104130 0x7f9c481972e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00bd70 tx=0x7f9c3c00bda0 comp rx=0 tx=0).stop 2024-02-19T19:29:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.074+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2198292936 shutdown_connections 2024-02-19T19:29:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.074+0000 7f9c4dedb700 1 --2- 172.21.15.88:0/2198292936 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c3006d8a0 0x7f9c3006fd50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.074+0000 7f9c4dedb700 1 --2- 172.21.15.88:0/2198292936 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c480ffbd0 0x7f9c48196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.074+0000 7f9c4dedb700 1 --2- 172.21.15.88:0/2198292936 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c48104130 0x7f9c481972e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.074+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2198292936 >> 172.21.15.88:0/2198292936 conn(0x7f9c48075130 msgr2=0x7f9c480fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.074+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2198292936 shutdown_connections 2024-02-19T19:29:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.074+0000 7f9c4dedb700 1 -- 172.21.15.88:0/2198292936 wait complete. 2024-02-19T19:29:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.178+0000 7f5206125700 1 -- 172.21.15.88:0/4008695675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5200069140 msgr2=0x7f5200104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.178+0000 7f5206125700 1 --2- 172.21.15.88:0/4008695675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5200069140 0x7f5200104eb0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009910 tx=0x7f51e8009c20 comp rx=0 tx=0).stop 2024-02-19T19:29:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.178+0000 7f5206125700 1 -- 172.21.15.88:0/4008695675 shutdown_connections 2024-02-19T19:29:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.178+0000 7f5206125700 1 --2- 172.21.15.88:0/4008695675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5200069140 0x7f5200104eb0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.178+0000 7f5206125700 1 --2- 172.21.15.88:0/4008695675 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f52000687a0 0x7f5200068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.178+0000 7f5206125700 1 -- 172.21.15.88:0/4008695675 >> 172.21.15.88:0/4008695675 conn(0x7f5200075130 msgr2=0x7f5200075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.178+0000 7f5206125700 1 -- 172.21.15.88:0/4008695675 shutdown_connections 2024-02-19T19:29:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.178+0000 7f5206125700 1 -- 172.21.15.88:0/4008695675 wait complete. 2024-02-19T19:29:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.178+0000 7f5206125700 1 Processor -- start 2024-02-19T19:29:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f5206125700 1 -- start start 2024-02-19T19:29:40.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f5206125700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52000687a0 0x7f5200195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:40.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f5206125700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5200069140 0x7f52001963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:40.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f5206125700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5200196a80 con 0x7f52000687a0 2024-02-19T19:29:40.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f5206125700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520019a810 con 0x7f5200069140 2024-02-19T19:29:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f51ff7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52000687a0 0x7f5200195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f51ff7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52000687a0 0x7f5200195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36834/0 (socket says 172.21.15.88:36834) 2024-02-19T19:29:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f51ff7fe700 1 -- 172.21.15.88:0/1714403943 learned_addr learned my addr 172.21.15.88:0/1714403943 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:29:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f51f6ffd700 1 --2- 172.21.15.88:0/1714403943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5200069140 0x7f52001963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f51ff7fe700 1 -- 172.21.15.88:0/1714403943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5200069140 msgr2=0x7f52001963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f51ff7fe700 1 --2- 172.21.15.88:0/1714403943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5200069140 0x7f52001963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f51ff7fe700 1 -- 172.21.15.88:0/1714403943 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e80095f0 con 0x7f52000687a0 2024-02-19T19:29:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.179+0000 7f51ff7fe700 1 --2- 172.21.15.88:0/1714403943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52000687a0 0x7f5200195e60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f51f000e920 tx=0x7f51f000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.180+0000 7f51fd7fa700 1 -- 172.21.15.88:0/1714403943 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f000c9d0 con 0x7f52000687a0 2024-02-19T19:29:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.180+0000 7f51fd7fa700 1 -- 172.21.15.88:0/1714403943 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51f0010430 con 0x7f52000687a0 2024-02-19T19:29:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.180+0000 7f5206125700 1 -- 172.21.15.88:0/1714403943 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520019aaf0 con 0x7f52000687a0 2024-02-19T19:29:40.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.180+0000 7f51fd7fa700 1 -- 172.21.15.88:0/1714403943 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51f0018630 con 0x7f52000687a0 2024-02-19T19:29:40.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.180+0000 7f5206125700 1 -- 172.21.15.88:0/1714403943 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520019b010 con 0x7f52000687a0 2024-02-19T19:29:40.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.181+0000 7f5206125700 1 -- 172.21.15.88:0/1714403943 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5200108750 con 0x7f52000687a0 2024-02-19T19:29:40.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.181+0000 7f51fd7fa700 1 -- 172.21.15.88:0/1714403943 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f51f00105a0 con 0x7f52000687a0 2024-02-19T19:29:40.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.182+0000 7f51fd7fa700 1 --2- 172.21.15.88:0/1714403943 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f51ec06d8a0 0x7f51ec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:29:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.182+0000 7f51f6ffd700 1 --2- 172.21.15.88:0/1714403943 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f51ec06d8a0 0x7f51ec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:29:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.182+0000 7f51fd7fa700 1 -- 172.21.15.88:0/1714403943 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f51f0014070 con 0x7f52000687a0 2024-02-19T19:29:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.182+0000 7f51f6ffd700 1 --2- 172.21.15.88:0/1714403943 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f51ec06d8a0 0x7f51ec06fd50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005b40 tx=0x7f51e8005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:29:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.186+0000 7f51fd7fa700 1 -- 172.21.15.88:0/1714403943 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f51f0052d20 con 0x7f52000687a0 2024-02-19T19:29:40.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.464+0000 7f5206125700 1 -- 172.21.15.88:0/1714403943 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f52001972c0 con 0x7f52000687a0 2024-02-19T19:29:40.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.464+0000 7f51fd7fa700 1 -- 172.21.15.88:0/1714403943 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f51f0056fe0 con 0x7f52000687a0 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:29:40.470 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:29:40.471 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:29:40.471 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:29:40.471 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:29:40.471 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:29:40.471 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:29:40.471 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:29:40.471 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:29:40.471 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:29:40.471 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:29:40.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.467+0000 7f5206125700 1 -- 172.21.15.88:0/1714403943 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f51ec06d8a0 msgr2=0x7f51ec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:40.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.467+0000 7f5206125700 1 --2- 172.21.15.88:0/1714403943 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f51ec06d8a0 0x7f51ec06fd50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f51e8005b40 tx=0x7f51e8005a90 comp rx=0 tx=0).stop 2024-02-19T19:29:40.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.467+0000 7f5206125700 1 -- 172.21.15.88:0/1714403943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52000687a0 msgr2=0x7f5200195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:29:40.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.467+0000 7f5206125700 1 --2- 172.21.15.88:0/1714403943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52000687a0 0x7f5200195e60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f51f000e920 tx=0x7f51f000ec30 comp rx=0 tx=0).stop 2024-02-19T19:29:40.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.467+0000 7f5206125700 1 -- 172.21.15.88:0/1714403943 shutdown_connections 2024-02-19T19:29:40.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.467+0000 7f5206125700 1 --2- 172.21.15.88:0/1714403943 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f51ec06d8a0 0x7f51ec06fd50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:40.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.467+0000 7f5206125700 1 --2- 172.21.15.88:0/1714403943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52000687a0 0x7f5200195e60 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:40.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.467+0000 7f5206125700 1 --2- 172.21.15.88:0/1714403943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5200069140 0x7f52001963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:29:40.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.467+0000 7f5206125700 1 -- 172.21.15.88:0/1714403943 >> 172.21.15.88:0/1714403943 conn(0x7f5200075130 msgr2=0x7f5200104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:29:40.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.467+0000 7f5206125700 1 -- 172.21.15.88:0/1714403943 shutdown_connections 2024-02-19T19:29:40.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:29:40.467+0000 7f5206125700 1 -- 172.21.15.88:0/1714403943 wait complete. 2024-02-19T19:29:40.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:40 smithi088 ceph-mon[88643]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:40.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:40 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3000760718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:29:40.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:40 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1714403943' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:29:40.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:40 smithi132 ceph-mon[70676]: from='client.35434 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:40.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:40 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3000760718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:29:40.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:40 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1714403943' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:29:41.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:41 smithi132 ceph-mon[70676]: from='client.35446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:41.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:41 smithi132 ceph-mon[70676]: pgmap v784: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:42.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:41 smithi088 ceph-mon[88643]: from='client.35446 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:29:42.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:41 smithi088 ceph-mon[88643]: pgmap v784: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:42.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:42 smithi132 ceph-mon[70676]: pgmap v785: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:43.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:42 smithi088 ceph-mon[88643]: pgmap v785: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:43.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:29:44.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:29:44.985 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:44 smithi088 ceph-mon[88643]: pgmap v786: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:44.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:44 smithi132 ceph-mon[70676]: pgmap v786: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:47.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:47 smithi132 ceph-mon[70676]: pgmap v787: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:48.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:47 smithi088 ceph-mon[88643]: pgmap v787: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:49.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:49 smithi132 ceph-mon[70676]: pgmap v788: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:50.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:49 smithi088 ceph-mon[88643]: pgmap v788: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:51.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:51 smithi132 ceph-mon[70676]: pgmap v789: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:52.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:51 smithi088 ceph-mon[88643]: pgmap v789: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:53.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:53 smithi132 ceph-mon[70676]: pgmap v790: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:54.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:53 smithi088 ceph-mon[88643]: pgmap v790: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:55.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:55 smithi132 ceph-mon[70676]: pgmap v791: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:56.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:55 smithi088 ceph-mon[88643]: pgmap v791: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:57.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:57 smithi132 ceph-mon[70676]: pgmap v792: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:58.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:57 smithi088 ceph-mon[88643]: pgmap v792: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:29:58.736 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:29:59.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:29:59.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:29:59 smithi132 ceph-mon[70676]: pgmap v793: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:00.003 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:29:59 smithi088 ceph-mon[88643]: pgmap v793: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:00.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:30:00.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:30:00.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:30:00.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-02-19T19:30:00.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:30:00.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-02-19T19:30:00.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:30:00.335 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : return _run_code(code, main_globals, None, 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:30:00.336 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:30:00.337 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-02-19T19:30:00.337 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:30:00.337 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:29:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-02-19T19:30:00.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:30:00.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:30:00.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:30:00.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: /usr/bin/podman: timeout after 895 seconds 2024-02-19T19:30:00.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:30:00.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: Traceback (most recent call last): 2024-02-19T19:30:00.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:30:00.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: return _run_code(code, main_globals, None, 2024-02-19T19:30:00.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:30:00.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: exec(code, run_globals) 2024-02-19T19:30:00.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:30:00.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:30:00.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:30:00.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:30:00.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:30:00.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:30:00.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:30:00.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:30:00.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:30:00.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:30:00.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: fs cephfs is degraded 2024-02-19T19:30:00.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:30:00.988 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:00 smithi132 ceph-mon[70676]: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:30:01.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: /usr/bin/podman: timeout after 895 seconds 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: Traceback (most recent call last): 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: return _run_code(code, main_globals, None, 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: exec(code, run_globals) 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:30:01.085 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:30:01.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:30:01.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:30:01.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:30:01.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:30:01.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:30:01.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:30:01.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: fs cephfs is degraded 2024-02-19T19:30:01.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:30:01.086 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:00 smithi088 ceph-mon[88643]: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:30:01.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:01 smithi132 ceph-mon[70676]: pgmap v794: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:02.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:01 smithi088 ceph-mon[88643]: pgmap v794: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:03.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:03 smithi132 ceph-mon[70676]: pgmap v795: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:04.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:03 smithi088 ceph-mon[88643]: pgmap v795: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:05.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:05 smithi132 ceph-mon[70676]: pgmap v796: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:06.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:05 smithi088 ceph-mon[88643]: pgmap v796: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:06.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:06 smithi132 ceph-mon[70676]: pgmap v797: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:07.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:06 smithi088 ceph-mon[88643]: pgmap v797: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:09.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:09 smithi132 ceph-mon[70676]: pgmap v798: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:10.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:09 smithi088 ceph-mon[88643]: pgmap v798: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.580+0000 7f20ae4b2700 1 -- 172.21.15.88:0/1641542661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20a8100460 msgr2=0x7f20a8100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.580+0000 7f20ae4b2700 1 --2- 172.21.15.88:0/1641542661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20a8100460 0x7f20a8100830 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f2098009940 tx=0x7f2098009c50 comp rx=0 tx=0).stop 2024-02-19T19:30:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.580+0000 7f20ae4b2700 1 -- 172.21.15.88:0/1641542661 shutdown_connections 2024-02-19T19:30:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.580+0000 7f20ae4b2700 1 --2- 172.21.15.88:0/1641542661 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20a8100d70 0x7f20a81094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.580+0000 7f20ae4b2700 1 --2- 172.21.15.88:0/1641542661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20a8100460 0x7f20a8100830 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.580+0000 7f20ae4b2700 1 -- 172.21.15.88:0/1641542661 >> 172.21.15.88:0/1641542661 conn(0x7f20a80748b0 msgr2=0x7f20a8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.580+0000 7f20ae4b2700 1 -- 172.21.15.88:0/1641542661 shutdown_connections 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.580+0000 7f20ae4b2700 1 -- 172.21.15.88:0/1641542661 wait complete. 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.581+0000 7f20ae4b2700 1 Processor -- start 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.581+0000 7f20ae4b2700 1 -- start start 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.581+0000 7f20ae4b2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20a8100460 0x7f20a8101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.581+0000 7f20ae4b2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20a8100d70 0x7f20a8102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.581+0000 7f20ae4b2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a8102850 con 0x7f20a8100d70 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.581+0000 7f20ae4b2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a8102990 con 0x7f20a8100460 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.581+0000 7f20a77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20a8100d70 0x7f20a8102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.581+0000 7f20a77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20a8100d70 0x7f20a8102310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51424/0 (socket says 172.21.15.88:51424) 2024-02-19T19:30:10.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.581+0000 7f20a77fe700 1 -- 172.21.15.88:0/4284189596 learned_addr learned my addr 172.21.15.88:0/4284189596 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:10.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.582+0000 7f20a7fff700 1 --2- 172.21.15.88:0/4284189596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20a8100460 0x7f20a8101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:10.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.582+0000 7f20a77fe700 1 -- 172.21.15.88:0/4284189596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20a8100460 msgr2=0x7f20a8101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:10.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.582+0000 7f20a77fe700 1 --2- 172.21.15.88:0/4284189596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20a8100460 0x7f20a8101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:10.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.582+0000 7f20a77fe700 1 -- 172.21.15.88:0/4284189596 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20980095f0 con 0x7f20a8100d70 2024-02-19T19:30:10.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.582+0000 7f20a77fe700 1 --2- 172.21.15.88:0/4284189596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20a8100d70 0x7f20a8102310 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f209c00d8d0 tx=0x7f209c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:10.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.582+0000 7f20a57fa700 1 -- 172.21.15.88:0/4284189596 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f209c009940 con 0x7f20a8100d70 2024-02-19T19:30:10.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.582+0000 7f20a57fa700 1 -- 172.21.15.88:0/4284189596 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f209c010460 con 0x7f20a8100d70 2024-02-19T19:30:10.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.582+0000 7f20ae4b2700 1 -- 172.21.15.88:0/4284189596 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a81a1f30 con 0x7f20a8100d70 2024-02-19T19:30:10.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.582+0000 7f20a57fa700 1 -- 172.21.15.88:0/4284189596 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f209c00f5f0 con 0x7f20a8100d70 2024-02-19T19:30:10.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.582+0000 7f20ae4b2700 1 -- 172.21.15.88:0/4284189596 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a81a2340 con 0x7f20a8100d70 2024-02-19T19:30:10.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.583+0000 7f20ae4b2700 1 -- 172.21.15.88:0/4284189596 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a804f2a0 con 0x7f20a8100d70 2024-02-19T19:30:10.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.584+0000 7f20a57fa700 1 -- 172.21.15.88:0/4284189596 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f209c0105d0 con 0x7f20a8100d70 2024-02-19T19:30:10.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.584+0000 7f20a57fa700 1 --2- 172.21.15.88:0/4284189596 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f209006d8a0 0x7f209006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:10.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.584+0000 7f20a57fa700 1 -- 172.21.15.88:0/4284189596 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f209c08d540 con 0x7f20a8100d70 2024-02-19T19:30:10.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.584+0000 7f20a7fff700 1 --2- 172.21.15.88:0/4284189596 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f209006d8a0 0x7f209006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.585+0000 7f20a7fff700 1 --2- 172.21.15.88:0/4284189596 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f209006d8a0 0x7f209006fd50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2098005b40 tx=0x7f2098005ab0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:10.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.589+0000 7f20a57fa700 1 -- 172.21.15.88:0/4284189596 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f209c056cf0 con 0x7f20a8100d70 2024-02-19T19:30:10.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.803+0000 7f20ae4b2700 1 -- 172.21.15.88:0/4284189596 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20a80ff350 con 0x7f209006d8a0 2024-02-19T19:30:10.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.804+0000 7f20a57fa700 1 -- 172.21.15.88:0/4284189596 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f20a80ff350 con 0x7f209006d8a0 2024-02-19T19:30:10.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.806+0000 7f20ae4b2700 1 -- 172.21.15.88:0/4284189596 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f209006d8a0 msgr2=0x7f209006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:10.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.806+0000 7f20ae4b2700 1 --2- 172.21.15.88:0/4284189596 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f209006d8a0 0x7f209006fd50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2098005b40 tx=0x7f2098005ab0 comp rx=0 tx=0).stop 2024-02-19T19:30:10.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.806+0000 7f20ae4b2700 1 -- 172.21.15.88:0/4284189596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20a8100d70 msgr2=0x7f20a8102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:10.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.806+0000 7f20ae4b2700 1 --2- 172.21.15.88:0/4284189596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20a8100d70 0x7f20a8102310 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f209c00d8d0 tx=0x7f209c00dc90 comp rx=0 tx=0).stop 2024-02-19T19:30:10.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.806+0000 7f20ae4b2700 1 -- 172.21.15.88:0/4284189596 shutdown_connections 2024-02-19T19:30:10.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.806+0000 7f20ae4b2700 1 --2- 172.21.15.88:0/4284189596 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f209006d8a0 0x7f209006fd50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:10.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.806+0000 7f20ae4b2700 1 --2- 172.21.15.88:0/4284189596 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20a8100460 0x7f20a8101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:10.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.806+0000 7f20ae4b2700 1 --2- 172.21.15.88:0/4284189596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20a8100d70 0x7f20a8102310 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:10.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.806+0000 7f20ae4b2700 1 -- 172.21.15.88:0/4284189596 >> 172.21.15.88:0/4284189596 conn(0x7f20a80748b0 msgr2=0x7f20a80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:10.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.806+0000 7f20ae4b2700 1 -- 172.21.15.88:0/4284189596 shutdown_connections 2024-02-19T19:30:10.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.806+0000 7f20ae4b2700 1 -- 172.21.15.88:0/4284189596 wait complete. 2024-02-19T19:30:10.820 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:30:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.911+0000 7f271f615700 1 -- 172.21.15.88:0/272513256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2718100460 msgr2=0x7f2718100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.911+0000 7f271f615700 1 --2- 172.21.15.88:0/272513256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2718100460 0x7f2718100830 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f2714009990 tx=0x7f2714009ca0 comp rx=0 tx=0).stop 2024-02-19T19:30:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.911+0000 7f271f615700 1 -- 172.21.15.88:0/272513256 shutdown_connections 2024-02-19T19:30:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.911+0000 7f271f615700 1 --2- 172.21.15.88:0/272513256 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2718100d70 0x7f27181094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.911+0000 7f271f615700 1 --2- 172.21.15.88:0/272513256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2718100460 0x7f2718100830 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.911+0000 7f271f615700 1 -- 172.21.15.88:0/272513256 >> 172.21.15.88:0/272513256 conn(0x7f27180748b0 msgr2=0x7f2718074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.911+0000 7f271f615700 1 -- 172.21.15.88:0/272513256 shutdown_connections 2024-02-19T19:30:10.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.911+0000 7f271f615700 1 -- 172.21.15.88:0/272513256 wait complete. 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271f615700 1 Processor -- start 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271f615700 1 -- start start 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271f615700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2718100460 0x7f2718199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271f615700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2718100d70 0x7f2718199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271f615700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f271819a5e0 con 0x7f2718100460 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271f615700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2718193a90 con 0x7f2718100d70 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271d3b1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2718100460 0x7f2718199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271d3b1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2718100460 0x7f2718199a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51438/0 (socket says 172.21.15.88:51438) 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271d3b1700 1 -- 172.21.15.88:0/1818257763 learned_addr learned my addr 172.21.15.88:0/1818257763 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271d3b1700 1 -- 172.21.15.88:0/1818257763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2718100d70 msgr2=0x7f2718199f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271d3b1700 1 --2- 172.21.15.88:0/1818257763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2718100d70 0x7f2718199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271d3b1700 1 -- 172.21.15.88:0/1818257763 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27140095f0 con 0x7f2718100460 2024-02-19T19:30:10.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.912+0000 7f271d3b1700 1 --2- 172.21.15.88:0/1818257763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2718100460 0x7f2718199a10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2714000c00 tx=0x7f271400e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:10.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.913+0000 7f270e7fc700 1 -- 172.21.15.88:0/1818257763 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27140052e0 con 0x7f2718100460 2024-02-19T19:30:10.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.913+0000 7f270e7fc700 1 -- 172.21.15.88:0/1818257763 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2714003680 con 0x7f2718100460 2024-02-19T19:30:10.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.913+0000 7f271f615700 1 -- 172.21.15.88:0/1818257763 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2718193d10 con 0x7f2718100460 2024-02-19T19:30:10.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.913+0000 7f270e7fc700 1 -- 172.21.15.88:0/1818257763 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2714020850 con 0x7f2718100460 2024-02-19T19:30:10.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.913+0000 7f271f615700 1 -- 172.21.15.88:0/1818257763 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2718194200 con 0x7f2718100460 2024-02-19T19:30:10.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.914+0000 7f271f615700 1 -- 172.21.15.88:0/1818257763 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2718106bd0 con 0x7f2718100460 2024-02-19T19:30:10.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.914+0000 7f270e7fc700 1 -- 172.21.15.88:0/1818257763 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f27140037f0 con 0x7f2718100460 2024-02-19T19:30:10.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.914+0000 7f270e7fc700 1 --2- 172.21.15.88:0/1818257763 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f270406d8a0 0x7f270406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:10.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.914+0000 7f270e7fc700 1 -- 172.21.15.88:0/1818257763 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2714090ee0 con 0x7f2718100460 2024-02-19T19:30:10.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.915+0000 7f271cbb0700 1 --2- 172.21.15.88:0/1818257763 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f270406d8a0 0x7f270406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:10.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.915+0000 7f271cbb0700 1 --2- 172.21.15.88:0/1818257763 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f270406d8a0 0x7f270406fd50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2708005950 tx=0x7f27080058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:10.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:10.919+0000 7f270e7fc700 1 -- 172.21.15.88:0/1818257763 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f271405e1c0 con 0x7f2718100460 2024-02-19T19:30:11.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.137+0000 7f271f615700 1 -- 172.21.15.88:0/1818257763 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f27181944e0 con 0x7f270406d8a0 2024-02-19T19:30:11.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.139+0000 7f270e7fc700 1 -- 172.21.15.88:0/1818257763 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f27181944e0 con 0x7f270406d8a0 2024-02-19T19:30:11.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.141+0000 7f271f615700 1 -- 172.21.15.88:0/1818257763 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f270406d8a0 msgr2=0x7f270406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.141+0000 7f271f615700 1 --2- 172.21.15.88:0/1818257763 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f270406d8a0 0x7f270406fd50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2708005950 tx=0x7f27080058e0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.141+0000 7f271f615700 1 -- 172.21.15.88:0/1818257763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2718100460 msgr2=0x7f2718199a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.141+0000 7f271f615700 1 --2- 172.21.15.88:0/1818257763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2718100460 0x7f2718199a10 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f2714000c00 tx=0x7f271400e7c0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.141+0000 7f271f615700 1 -- 172.21.15.88:0/1818257763 shutdown_connections 2024-02-19T19:30:11.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.141+0000 7f271f615700 1 --2- 172.21.15.88:0/1818257763 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f270406d8a0 0x7f270406fd50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.141+0000 7f271f615700 1 --2- 172.21.15.88:0/1818257763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2718100460 0x7f2718199a10 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.141+0000 7f271f615700 1 --2- 172.21.15.88:0/1818257763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2718100d70 0x7f2718199f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.141+0000 7f271f615700 1 -- 172.21.15.88:0/1818257763 >> 172.21.15.88:0/1818257763 conn(0x7f27180748b0 msgr2=0x7f2718103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:11.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.141+0000 7f271f615700 1 -- 172.21.15.88:0/1818257763 shutdown_connections 2024-02-19T19:30:11.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.141+0000 7f271f615700 1 -- 172.21.15.88:0/1818257763 wait complete. 2024-02-19T19:30:11.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.245+0000 7f26832be700 1 -- 172.21.15.88:0/3104690574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f267c100e80 msgr2=0x7f267c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.245+0000 7f26832be700 1 --2- 172.21.15.88:0/3104690574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f267c100e80 0x7f267c104e50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f266c009910 tx=0x7f266c009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:11.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.245+0000 7f26832be700 1 -- 172.21.15.88:0/3104690574 shutdown_connections 2024-02-19T19:30:11.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.245+0000 7f26832be700 1 --2- 172.21.15.88:0/3104690574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f267c100e80 0x7f267c104e50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.245+0000 7f26832be700 1 --2- 172.21.15.88:0/3104690574 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f267c1004e0 0x7f267c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.245+0000 7f26832be700 1 -- 172.21.15.88:0/3104690574 >> 172.21.15.88:0/3104690574 conn(0x7f267c0fbda0 msgr2=0x7f267c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:11.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.245+0000 7f26832be700 1 -- 172.21.15.88:0/3104690574 shutdown_connections 2024-02-19T19:30:11.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.245+0000 7f26832be700 1 -- 172.21.15.88:0/3104690574 wait complete. 2024-02-19T19:30:11.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f26832be700 1 Processor -- start 2024-02-19T19:30:11.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f26832be700 1 -- start start 2024-02-19T19:30:11.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f26832be700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f267c1004e0 0x7f267c195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:11.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f26832be700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f267c100e80 0x7f267c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:11.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f26832be700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c196b30 con 0x7f267c100e80 2024-02-19T19:30:11.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f26832be700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c19a8c0 con 0x7f267c1004e0 2024-02-19T19:30:11.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f2680859700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f267c100e80 0x7f267c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f2680859700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f267c100e80 0x7f267c196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51446/0 (socket says 172.21.15.88:51446) 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f2680859700 1 -- 172.21.15.88:0/416430150 learned_addr learned my addr 172.21.15.88:0/416430150 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f268105a700 1 --2- 172.21.15.88:0/416430150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f267c1004e0 0x7f267c195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f2680859700 1 -- 172.21.15.88:0/416430150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f267c1004e0 msgr2=0x7f267c195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f2680859700 1 --2- 172.21.15.88:0/416430150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f267c1004e0 0x7f267c195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f2680859700 1 -- 172.21.15.88:0/416430150 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c0095f0 con 0x7f267c100e80 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.246+0000 7f2680859700 1 --2- 172.21.15.88:0/416430150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f267c100e80 0x7f267c196450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f266c0092e0 tx=0x7f266c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.247+0000 7f26727fc700 1 -- 172.21.15.88:0/416430150 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f266c01d070 con 0x7f267c100e80 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.247+0000 7f26727fc700 1 -- 172.21.15.88:0/416430150 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f266c005dc0 con 0x7f267c100e80 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.247+0000 7f26832be700 1 -- 172.21.15.88:0/416430150 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c19ab40 con 0x7f267c100e80 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.247+0000 7f26727fc700 1 -- 172.21.15.88:0/416430150 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f266c017400 con 0x7f267c100e80 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.247+0000 7f26832be700 1 -- 172.21.15.88:0/416430150 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c19b030 con 0x7f267c100e80 2024-02-19T19:30:11.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.248+0000 7f26832be700 1 -- 172.21.15.88:0/416430150 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267c108850 con 0x7f267c100e80 2024-02-19T19:30:11.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.248+0000 7f26727fc700 1 -- 172.21.15.88:0/416430150 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f266c00f6f0 con 0x7f267c100e80 2024-02-19T19:30:11.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.248+0000 7f26727fc700 1 --2- 172.21.15.88:0/416430150 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f266806d8a0 0x7f266806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:11.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.249+0000 7f26727fc700 1 -- 172.21.15.88:0/416430150 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f266c08e910 con 0x7f267c100e80 2024-02-19T19:30:11.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.249+0000 7f268105a700 1 --2- 172.21.15.88:0/416430150 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f266806d8a0 0x7f266806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:11.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.249+0000 7f268105a700 1 --2- 172.21.15.88:0/416430150 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f266806d8a0 0x7f266806fd50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2678005950 tx=0x7f267800b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:11.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.253+0000 7f26727fc700 1 -- 172.21.15.88:0/416430150 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f266c0580c0 con 0x7f267c100e80 2024-02-19T19:30:11.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.461+0000 7f26832be700 1 -- 172.21.15.88:0/416430150 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f267c1972e0 con 0x7f266806d8a0 2024-02-19T19:30:11.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.472+0000 7f26727fc700 1 -- 172.21.15.88:0/416430150 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f267c1972e0 con 0x7f266806d8a0 2024-02-19T19:30:11.476 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:30:11.476 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (27m) 3m ago 37m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:30:11.476 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (38m) 3m ago 38m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:30:11.476 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (20m) 3m ago 36m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:30:11.476 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (23m) 3m ago 38m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:30:11.476 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (23m) 3m ago 36m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:30:11.476 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (26m) 3m ago 37m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:30:11.476 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (19m) 3m ago 31m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (19m) 3m ago 31m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (18m) 3m ago 31m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (3m) 3m ago 31m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (28m) 3m ago 39m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (28m) 3m ago 36m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (25m) 3m ago 39m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (24m) 3m ago 36m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (28m) 3m ago 38m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (27m) 3m ago 36m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (22m) 3m ago 35m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (22m) 3m ago 34m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (21m) 3m ago 34m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:30:11.477 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (21m) 3m ago 33m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:30:11.478 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (20m) 3m ago 33m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:30:11.478 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (19m) 3m ago 33m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:30:11.478 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (27m) 3m ago 36m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:30:11.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.475+0000 7f26832be700 1 -- 172.21.15.88:0/416430150 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f266806d8a0 msgr2=0x7f266806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.475+0000 7f26832be700 1 --2- 172.21.15.88:0/416430150 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f266806d8a0 0x7f266806fd50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2678005950 tx=0x7f267800b410 comp rx=0 tx=0).stop 2024-02-19T19:30:11.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.475+0000 7f26832be700 1 -- 172.21.15.88:0/416430150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f267c100e80 msgr2=0x7f267c196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.475+0000 7f26832be700 1 --2- 172.21.15.88:0/416430150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f267c100e80 0x7f267c196450 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f266c0092e0 tx=0x7f266c00bd70 comp rx=0 tx=0).stop 2024-02-19T19:30:11.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.475+0000 7f26832be700 1 -- 172.21.15.88:0/416430150 shutdown_connections 2024-02-19T19:30:11.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.475+0000 7f26832be700 1 --2- 172.21.15.88:0/416430150 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f266806d8a0 0x7f266806fd50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.475+0000 7f26832be700 1 --2- 172.21.15.88:0/416430150 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f267c1004e0 0x7f267c195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.475+0000 7f26832be700 1 --2- 172.21.15.88:0/416430150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f267c100e80 0x7f267c196450 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.475+0000 7f26832be700 1 -- 172.21.15.88:0/416430150 >> 172.21.15.88:0/416430150 conn(0x7f267c0fbda0 msgr2=0x7f267c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:11.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.475+0000 7f26832be700 1 -- 172.21.15.88:0/416430150 shutdown_connections 2024-02-19T19:30:11.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.475+0000 7f26832be700 1 -- 172.21.15.88:0/416430150 wait complete. 2024-02-19T19:30:11.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.582+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3780118842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac104130 msgr2=0x7f39ac104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.582+0000 7f39b1e1c700 1 --2- 172.21.15.88:0/3780118842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac104130 0x7f39ac104500 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f399c009910 tx=0x7f399c009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:11.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.582+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3780118842 shutdown_connections 2024-02-19T19:30:11.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.582+0000 7f39b1e1c700 1 --2- 172.21.15.88:0/3780118842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39ac0ffbd0 0x7f39ac100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.582+0000 7f39b1e1c700 1 --2- 172.21.15.88:0/3780118842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac104130 0x7f39ac104500 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.582+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3780118842 >> 172.21.15.88:0/3780118842 conn(0x7f39ac075130 msgr2=0x7f39ac075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:11.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.582+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3780118842 shutdown_connections 2024-02-19T19:30:11.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.582+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3780118842 wait complete. 2024-02-19T19:30:11.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.582+0000 7f39b1e1c700 1 Processor -- start 2024-02-19T19:30:11.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39b1e1c700 1 -- start start 2024-02-19T19:30:11.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39b1e1c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39ac0ffbd0 0x7f39ac195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:11.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39b1e1c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac104130 0x7f39ac196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39b1e1c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac196bf0 con 0x7f39ac104130 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39b1e1c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39ac19a980 con 0x7f39ac0ffbd0 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39a2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac104130 0x7f39ac196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39a2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac104130 0x7f39ac196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51478/0 (socket says 172.21.15.88:51478) 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39a2ffd700 1 -- 172.21.15.88:0/3329715556 learned_addr learned my addr 172.21.15.88:0/3329715556 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39a2ffd700 1 -- 172.21.15.88:0/3329715556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39ac0ffbd0 msgr2=0x7f39ac195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39a2ffd700 1 --2- 172.21.15.88:0/3329715556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39ac0ffbd0 0x7f39ac195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39a2ffd700 1 -- 172.21.15.88:0/3329715556 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c0095f0 con 0x7f39ac104130 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.583+0000 7f39a2ffd700 1 --2- 172.21.15.88:0/3329715556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac104130 0x7f39ac196510 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f399400b4b0 tx=0x7f399400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.584+0000 7f39a97fa700 1 -- 172.21.15.88:0/3329715556 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3994011800 con 0x7f39ac104130 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.584+0000 7f39a97fa700 1 -- 172.21.15.88:0/3329715556 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3994011e20 con 0x7f39ac104130 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.584+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3329715556 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39ac19ac60 con 0x7f39ac104130 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.584+0000 7f39a97fa700 1 -- 172.21.15.88:0/3329715556 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f399400f2e0 con 0x7f39ac104130 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.584+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3329715556 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39ac19b1b0 con 0x7f39ac104130 2024-02-19T19:30:11.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.585+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3329715556 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39ac04f2a0 con 0x7f39ac104130 2024-02-19T19:30:11.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.585+0000 7f39a97fa700 1 -- 172.21.15.88:0/3329715556 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3994011960 con 0x7f39ac104130 2024-02-19T19:30:11.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.585+0000 7f39a97fa700 1 --2- 172.21.15.88:0/3329715556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 0x7f399806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:11.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.586+0000 7f39a97fa700 1 -- 172.21.15.88:0/3329715556 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f399408d0e0 con 0x7f39ac104130 2024-02-19T19:30:11.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.586+0000 7f39ab7fe700 1 --2- 172.21.15.88:0/3329715556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 0x7f399806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:11.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.586+0000 7f39ab7fe700 1 --2- 172.21.15.88:0/3329715556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 0x7f399806fd50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f399c00b5c0 tx=0x7f399c005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:11.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.590+0000 7f39a97fa700 1 -- 172.21.15.88:0/3329715556 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3994056890 con 0x7f39ac104130 2024-02-19T19:30:11.833 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:11 smithi088 ceph-mon[88643]: pgmap v799: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:11.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.870+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3329715556 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f39ac19b430 con 0x7f39ac104130 2024-02-19T19:30:11.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.871+0000 7f39a97fa700 1 -- 172.21.15.88:0/3329715556 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3994059f50 con 0x7f39ac104130 2024-02-19T19:30:11.875 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:30:11.875 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:30:11.875 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:30:11.875 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:30:11.875 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:30:11.876 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:30:11.876 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:30:11.876 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:30:11.876 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:30:11.876 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:30:11.877 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:30:11.877 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:30:11.877 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:30:11.877 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:30:11.877 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:30:11.877 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:30:11.878 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:30:11.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.873+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3329715556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 msgr2=0x7f399806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.873+0000 7f39b1e1c700 1 --2- 172.21.15.88:0/3329715556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 0x7f399806fd50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f399c00b5c0 tx=0x7f399c005e20 comp rx=0 tx=0).stop 2024-02-19T19:30:11.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.873+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3329715556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac104130 msgr2=0x7f39ac196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.873+0000 7f39b1e1c700 1 --2- 172.21.15.88:0/3329715556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac104130 0x7f39ac196510 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f399400b4b0 tx=0x7f399400b870 comp rx=0 tx=0).stop 2024-02-19T19:30:11.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.873+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3329715556 shutdown_connections 2024-02-19T19:30:11.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.873+0000 7f39b1e1c700 1 --2- 172.21.15.88:0/3329715556 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f399806d8a0 0x7f399806fd50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.873+0000 7f39b1e1c700 1 --2- 172.21.15.88:0/3329715556 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f39ac0ffbd0 0x7f39ac195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.873+0000 7f39b1e1c700 1 --2- 172.21.15.88:0/3329715556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39ac104130 0x7f39ac196510 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.873+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3329715556 >> 172.21.15.88:0/3329715556 conn(0x7f39ac075130 msgr2=0x7f39ac109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:11.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.873+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3329715556 shutdown_connections 2024-02-19T19:30:11.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.873+0000 7f39b1e1c700 1 -- 172.21.15.88:0/3329715556 wait complete. 2024-02-19T19:30:11.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.976+0000 7f7cf2b07700 1 -- 172.21.15.88:0/1232938761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cec0ffbd0 msgr2=0x7f7cec100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.976+0000 7f7cf2b07700 1 --2- 172.21.15.88:0/1232938761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cec0ffbd0 0x7f7cec100040 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009910 tx=0x7f7cd8009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:11.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.976+0000 7f7cf2b07700 1 -- 172.21.15.88:0/1232938761 shutdown_connections 2024-02-19T19:30:11.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.976+0000 7f7cf2b07700 1 --2- 172.21.15.88:0/1232938761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cec0ffbd0 0x7f7cec100040 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.976+0000 7f7cf2b07700 1 --2- 172.21.15.88:0/1232938761 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cec104130 0x7f7cec104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.976+0000 7f7cf2b07700 1 -- 172.21.15.88:0/1232938761 >> 172.21.15.88:0/1232938761 conn(0x7f7cec075130 msgr2=0x7f7cec075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:11.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.976+0000 7f7cf2b07700 1 -- 172.21.15.88:0/1232938761 shutdown_connections 2024-02-19T19:30:11.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.977+0000 7f7cf2b07700 1 -- 172.21.15.88:0/1232938761 wait complete. 2024-02-19T19:30:11.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.977+0000 7f7cf2b07700 1 Processor -- start 2024-02-19T19:30:11.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.977+0000 7f7cf2b07700 1 -- start start 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.977+0000 7f7cf2b07700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cec0ffbd0 0x7f7cec194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.977+0000 7f7cf2b07700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cec104130 0x7f7cec195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.977+0000 7f7cf2b07700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec195950 con 0x7f7cec104130 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.977+0000 7f7cf2b07700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cec1996e0 con 0x7f7cec0ffbd0 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.977+0000 7f7cebfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cec104130 0x7f7cec195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.977+0000 7f7cebfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cec104130 0x7f7cec195270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51500/0 (socket says 172.21.15.88:51500) 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.977+0000 7f7cebfff700 1 -- 172.21.15.88:0/2284032542 learned_addr learned my addr 172.21.15.88:0/2284032542 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.977+0000 7f7cf08a3700 1 --2- 172.21.15.88:0/2284032542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cec0ffbd0 0x7f7cec194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.978+0000 7f7cebfff700 1 -- 172.21.15.88:0/2284032542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cec0ffbd0 msgr2=0x7f7cec194d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.978+0000 7f7cebfff700 1 --2- 172.21.15.88:0/2284032542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cec0ffbd0 0x7f7cec194d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.978+0000 7f7cebfff700 1 -- 172.21.15.88:0/2284032542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd80095f0 con 0x7f7cec104130 2024-02-19T19:30:11.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.978+0000 7f7cebfff700 1 --2- 172.21.15.88:0/2284032542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cec104130 0x7f7cec195270 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800bd70 tx=0x7f7cd800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:11.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.978+0000 7f7ce9ffb700 1 -- 172.21.15.88:0/2284032542 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd801d070 con 0x7f7cec104130 2024-02-19T19:30:11.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.978+0000 7f7ce9ffb700 1 -- 172.21.15.88:0/2284032542 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cd8005dc0 con 0x7f7cec104130 2024-02-19T19:30:11.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.978+0000 7f7cf2b07700 1 -- 172.21.15.88:0/2284032542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cec199960 con 0x7f7cec104130 2024-02-19T19:30:11.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.978+0000 7f7ce9ffb700 1 -- 172.21.15.88:0/2284032542 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd8017400 con 0x7f7cec104130 2024-02-19T19:30:11.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.978+0000 7f7cf2b07700 1 -- 172.21.15.88:0/2284032542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cec199e50 con 0x7f7cec104130 2024-02-19T19:30:11.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.979+0000 7f7cf2b07700 1 -- 172.21.15.88:0/2284032542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cec04f2a0 con 0x7f7cec104130 2024-02-19T19:30:11.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.980+0000 7f7ce9ffb700 1 -- 172.21.15.88:0/2284032542 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7cd800f810 con 0x7f7cec104130 2024-02-19T19:30:11.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.980+0000 7f7ce9ffb700 1 --2- 172.21.15.88:0/2284032542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cdc06d8a0 0x7f7cdc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:11.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.980+0000 7f7ce9ffb700 1 -- 172.21.15.88:0/2284032542 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7cd808ea20 con 0x7f7cec104130 2024-02-19T19:30:11.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.980+0000 7f7cf08a3700 1 --2- 172.21.15.88:0/2284032542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cdc06d8a0 0x7f7cdc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:11.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.981+0000 7f7cf08a3700 1 --2- 172.21.15.88:0/2284032542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cdc06d8a0 0x7f7cdc06fd50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0005950 tx=0x7f7ce000b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:11.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:11 smithi132 ceph-mon[70676]: pgmap v799: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:11.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:11.985+0000 7f7ce9ffb700 1 -- 172.21.15.88:0/2284032542 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7cd80581d0 con 0x7f7cec104130 2024-02-19T19:30:12.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.224+0000 7f7cf2b07700 1 -- 172.21.15.88:0/2284032542 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7cec19a110 con 0x7f7cec104130 2024-02-19T19:30:12.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.224+0000 7f7ce9ffb700 1 -- 172.21.15.88:0/2284032542 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f7cd805b890 con 0x7f7cec104130 2024-02-19T19:30:12.228 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:30:12.228 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:30:12.229 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:30:12.230 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:30:12.231 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:30:12.232 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:30:12.232 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:30:12.232 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:30:12.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.227+0000 7f7cf2b07700 1 -- 172.21.15.88:0/2284032542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cdc06d8a0 msgr2=0x7f7cdc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.227+0000 7f7cf2b07700 1 --2- 172.21.15.88:0/2284032542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cdc06d8a0 0x7f7cdc06fd50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0005950 tx=0x7f7ce000b410 comp rx=0 tx=0).stop 2024-02-19T19:30:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.227+0000 7f7cf2b07700 1 -- 172.21.15.88:0/2284032542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cec104130 msgr2=0x7f7cec195270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.227+0000 7f7cf2b07700 1 --2- 172.21.15.88:0/2284032542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cec104130 0x7f7cec195270 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800bd70 tx=0x7f7cd800bda0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.227+0000 7f7cf2b07700 1 -- 172.21.15.88:0/2284032542 shutdown_connections 2024-02-19T19:30:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.227+0000 7f7cf2b07700 1 --2- 172.21.15.88:0/2284032542 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cdc06d8a0 0x7f7cdc06fd50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.227+0000 7f7cf2b07700 1 --2- 172.21.15.88:0/2284032542 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7cec0ffbd0 0x7f7cec194d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.227+0000 7f7cf2b07700 1 --2- 172.21.15.88:0/2284032542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cec104130 0x7f7cec195270 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.227+0000 7f7cf2b07700 1 -- 172.21.15.88:0/2284032542 >> 172.21.15.88:0/2284032542 conn(0x7f7cec075130 msgr2=0x7f7cec0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.227+0000 7f7cf2b07700 1 -- 172.21.15.88:0/2284032542 shutdown_connections 2024-02-19T19:30:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.227+0000 7f7cf2b07700 1 -- 172.21.15.88:0/2284032542 wait complete. 2024-02-19T19:30:12.233 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:30:12.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.332+0000 7f79724cf700 1 -- 172.21.15.88:0/2251581521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f796c0ffbd0 msgr2=0x7f796c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:12.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.332+0000 7f79724cf700 1 --2- 172.21.15.88:0/2251581521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f796c0ffbd0 0x7f796c100040 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f7960009910 tx=0x7f7960009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:12.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.332+0000 7f79724cf700 1 -- 172.21.15.88:0/2251581521 shutdown_connections 2024-02-19T19:30:12.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.332+0000 7f79724cf700 1 --2- 172.21.15.88:0/2251581521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f796c0ffbd0 0x7f796c100040 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.332+0000 7f79724cf700 1 --2- 172.21.15.88:0/2251581521 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f796c104130 0x7f796c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.332+0000 7f79724cf700 1 -- 172.21.15.88:0/2251581521 >> 172.21.15.88:0/2251581521 conn(0x7f796c075130 msgr2=0x7f796c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:12.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.332+0000 7f79724cf700 1 -- 172.21.15.88:0/2251581521 shutdown_connections 2024-02-19T19:30:12.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.332+0000 7f79724cf700 1 -- 172.21.15.88:0/2251581521 wait complete. 2024-02-19T19:30:12.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.332+0000 7f79724cf700 1 Processor -- start 2024-02-19T19:30:12.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.332+0000 7f79724cf700 1 -- start start 2024-02-19T19:30:12.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f79724cf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f796c0ffbd0 0x7f796c195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:12.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f79724cf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f796c104130 0x7f796c196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:12.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f79724cf700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c196a70 con 0x7f796c104130 2024-02-19T19:30:12.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f79724cf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c19a800 con 0x7f796c0ffbd0 2024-02-19T19:30:12.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f796b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f796c104130 0x7f796c196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f796b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f796c104130 0x7f796c196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51524/0 (socket says 172.21.15.88:51524) 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f796b7fe700 1 -- 172.21.15.88:0/193179124 learned_addr learned my addr 172.21.15.88:0/193179124 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f796bfff700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f796c0ffbd0 0x7f796c195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f796b7fe700 1 -- 172.21.15.88:0/193179124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f796c0ffbd0 msgr2=0x7f796c195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f796b7fe700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f796c0ffbd0 0x7f796c195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f796b7fe700 1 -- 172.21.15.88:0/193179124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79600095f0 con 0x7f796c104130 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f796bfff700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f796c0ffbd0 0x7f796c195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.333+0000 7f796b7fe700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f796c104130 0x7f796c196390 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f796000bd70 tx=0x7f796000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.334+0000 7f79697fa700 1 -- 172.21.15.88:0/193179124 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f796001d070 con 0x7f796c104130 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.334+0000 7f79724cf700 1 -- 172.21.15.88:0/193179124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c19aa80 con 0x7f796c104130 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.334+0000 7f79697fa700 1 -- 172.21.15.88:0/193179124 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f796000f460 con 0x7f796c104130 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.334+0000 7f79697fa700 1 -- 172.21.15.88:0/193179124 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7960017630 con 0x7f796c104130 2024-02-19T19:30:12.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.334+0000 7f79724cf700 1 -- 172.21.15.88:0/193179124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c19af70 con 0x7f796c104130 2024-02-19T19:30:12.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.335+0000 7f79724cf700 1 -- 172.21.15.88:0/193179124 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f796c1086a0 con 0x7f796c104130 2024-02-19T19:30:12.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.335+0000 7f79697fa700 1 -- 172.21.15.88:0/193179124 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7960003a60 con 0x7f796c104130 2024-02-19T19:30:12.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.335+0000 7f79697fa700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795406d8a0 0x7f795406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:12.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.336+0000 7f79697fa700 1 -- 172.21.15.88:0/193179124 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f796008e9b0 con 0x7f796c104130 2024-02-19T19:30:12.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.336+0000 7f796bfff700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795406d8a0 0x7f795406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:12.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.336+0000 7f796bfff700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795406d8a0 0x7f795406fd50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f795c00bc10 tx=0x7f795c00b5f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:12.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.340+0000 7f79697fa700 1 -- 172.21.15.88:0/193179124 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f79600582a0 con 0x7f796c104130 2024-02-19T19:30:12.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.554+0000 7f79724cf700 1 -- 172.21.15.88:0/193179124 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f796c19b2a0 con 0x7f795406d8a0 2024-02-19T19:30:12.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.555+0000 7f79697fa700 1 -- 172.21.15.88:0/193179124 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f796c19b2a0 con 0x7f795406d8a0 2024-02-19T19:30:12.559 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:30:12.559 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:30:12.559 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:30:12.559 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:30:12.559 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:30:12.559 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:30:12.559 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:30:12.559 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:30:12.559 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:30:12.559 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:30:12.560 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:30:12.560 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:30:12.560 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:30:12.560 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:30:12.560 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:30:12.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.557+0000 7f79724cf700 1 -- 172.21.15.88:0/193179124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795406d8a0 msgr2=0x7f795406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:12.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.557+0000 7f79724cf700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795406d8a0 0x7f795406fd50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f795c00bc10 tx=0x7f795c00b5f0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.557+0000 7f79724cf700 1 -- 172.21.15.88:0/193179124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f796c104130 msgr2=0x7f796c196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:12.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.557+0000 7f79724cf700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f796c104130 0x7f796c196390 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f796000bd70 tx=0x7f796000be50 comp rx=0 tx=0).stop 2024-02-19T19:30:12.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.558+0000 7f79724cf700 1 -- 172.21.15.88:0/193179124 shutdown_connections 2024-02-19T19:30:12.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.558+0000 7f79724cf700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795406d8a0 0x7f795406fd50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.558+0000 7f79724cf700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f796c0ffbd0 0x7f796c195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.558+0000 7f79724cf700 1 --2- 172.21.15.88:0/193179124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f796c104130 0x7f796c196390 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.558+0000 7f79724cf700 1 -- 172.21.15.88:0/193179124 >> 172.21.15.88:0/193179124 conn(0x7f796c075130 msgr2=0x7f796c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:12.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.558+0000 7f79724cf700 1 -- 172.21.15.88:0/193179124 shutdown_connections 2024-02-19T19:30:12.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.558+0000 7f79724cf700 1 -- 172.21.15.88:0/193179124 wait complete. 2024-02-19T19:30:12.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.661+0000 7f8daea49700 1 -- 172.21.15.88:0/577674683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da80ffd50 msgr2=0x7f8da8103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:12.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.661+0000 7f8daea49700 1 --2- 172.21.15.88:0/577674683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da80ffd50 0x7f8da8103d20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009910 tx=0x7f8d9c009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:12.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.661+0000 7f8daea49700 1 -- 172.21.15.88:0/577674683 shutdown_connections 2024-02-19T19:30:12.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.661+0000 7f8daea49700 1 --2- 172.21.15.88:0/577674683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da80ffd50 0x7f8da8103d20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.661+0000 7f8daea49700 1 --2- 172.21.15.88:0/577674683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8da80ff3b0 0x7f8da80ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.661+0000 7f8daea49700 1 -- 172.21.15.88:0/577674683 >> 172.21.15.88:0/577674683 conn(0x7f8da80fac90 msgr2=0x7f8da80fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.661+0000 7f8daea49700 1 -- 172.21.15.88:0/577674683 shutdown_connections 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.661+0000 7f8daea49700 1 -- 172.21.15.88:0/577674683 wait complete. 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.661+0000 7f8daea49700 1 Processor -- start 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.661+0000 7f8daea49700 1 -- start start 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8daea49700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8da80ff3b0 0x7f8da819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8daea49700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da80ffd50 0x7f8da819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8daea49700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da819d110 con 0x7f8da80ffd50 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8daea49700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da81a0ea0 con 0x7f8da80ff3b0 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8da77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da80ffd50 0x7f8da819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8da77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da80ffd50 0x7f8da819ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51548/0 (socket says 172.21.15.88:51548) 2024-02-19T19:30:12.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8da77fe700 1 -- 172.21.15.88:0/1411300971 learned_addr learned my addr 172.21.15.88:0/1411300971 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:12.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8da7fff700 1 --2- 172.21.15.88:0/1411300971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8da80ff3b0 0x7f8da819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:12.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8da77fe700 1 -- 172.21.15.88:0/1411300971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8da80ff3b0 msgr2=0x7f8da819c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:12.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8da77fe700 1 --2- 172.21.15.88:0/1411300971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8da80ff3b0 0x7f8da819c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8da77fe700 1 -- 172.21.15.88:0/1411300971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c0095f0 con 0x7f8da80ffd50 2024-02-19T19:30:12.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8da77fe700 1 --2- 172.21.15.88:0/1411300971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da80ffd50 0x7f8da819ca30 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0092e0 tx=0x7f8d9c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:12.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8da57fa700 1 -- 172.21.15.88:0/1411300971 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9c01d070 con 0x7f8da80ffd50 2024-02-19T19:30:12.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8da57fa700 1 -- 172.21.15.88:0/1411300971 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d9c005dc0 con 0x7f8da80ffd50 2024-02-19T19:30:12.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8daea49700 1 -- 172.21.15.88:0/1411300971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da81a1120 con 0x7f8da80ffd50 2024-02-19T19:30:12.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.662+0000 7f8da57fa700 1 -- 172.21.15.88:0/1411300971 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d9c017400 con 0x7f8da80ffd50 2024-02-19T19:30:12.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.663+0000 7f8daea49700 1 -- 172.21.15.88:0/1411300971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da81a1610 con 0x7f8da80ffd50 2024-02-19T19:30:12.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.663+0000 7f8daea49700 1 -- 172.21.15.88:0/1411300971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da8107720 con 0x7f8da80ffd50 2024-02-19T19:30:12.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.664+0000 7f8da57fa700 1 -- 172.21.15.88:0/1411300971 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8d9c00f6f0 con 0x7f8da80ffd50 2024-02-19T19:30:12.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.664+0000 7f8da57fa700 1 --2- 172.21.15.88:0/1411300971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d9006d8a0 0x7f8d9006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:12.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.664+0000 7f8da57fa700 1 -- 172.21.15.88:0/1411300971 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8d9c08e910 con 0x7f8da80ffd50 2024-02-19T19:30:12.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.664+0000 7f8da7fff700 1 --2- 172.21.15.88:0/1411300971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d9006d8a0 0x7f8d9006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:12.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.665+0000 7f8da7fff700 1 --2- 172.21.15.88:0/1411300971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d9006d8a0 0x7f8d9006fd50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005950 tx=0x7f8d9800b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:12.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.669+0000 7f8da57fa700 1 -- 172.21.15.88:0/1411300971 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d9c0580c0 con 0x7f8da80ffd50 2024-02-19T19:30:12.833 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:12 smithi088 ceph-mon[88643]: from='client.35454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:12.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:12 smithi088 ceph-mon[88643]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:12.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:12 smithi088 ceph-mon[88643]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:12.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:12 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3329715556' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:30:12.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:12 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2284032542' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:30:12.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.949+0000 7f8daea49700 1 -- 172.21.15.88:0/1411300971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8da8107100 con 0x7f8da80ffd50 2024-02-19T19:30:12.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.949+0000 7f8da57fa700 1 -- 172.21.15.88:0/1411300971 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f8d9c05b780 con 0x7f8da80ffd50 2024-02-19T19:30:12.953 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:30:12.953 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:30:12.953 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:30:12.953 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:30:12.953 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:30:12.954 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:30:12.955 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:30:12.955 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:30:12.955 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:30:12.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.951+0000 7f8daea49700 1 -- 172.21.15.88:0/1411300971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d9006d8a0 msgr2=0x7f8d9006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:12.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.952+0000 7f8daea49700 1 --2- 172.21.15.88:0/1411300971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d9006d8a0 0x7f8d9006fd50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8d98005950 tx=0x7f8d9800b410 comp rx=0 tx=0).stop 2024-02-19T19:30:12.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.952+0000 7f8daea49700 1 -- 172.21.15.88:0/1411300971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da80ffd50 msgr2=0x7f8da819ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:12.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.952+0000 7f8daea49700 1 --2- 172.21.15.88:0/1411300971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da80ffd50 0x7f8da819ca30 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0092e0 tx=0x7f8d9c00bd70 comp rx=0 tx=0).stop 2024-02-19T19:30:12.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.952+0000 7f8daea49700 1 -- 172.21.15.88:0/1411300971 shutdown_connections 2024-02-19T19:30:12.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.952+0000 7f8daea49700 1 --2- 172.21.15.88:0/1411300971 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d9006d8a0 0x7f8d9006fd50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.952+0000 7f8daea49700 1 --2- 172.21.15.88:0/1411300971 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8da80ff3b0 0x7f8da819c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.952+0000 7f8daea49700 1 --2- 172.21.15.88:0/1411300971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8da80ffd50 0x7f8da819ca30 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:12.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.952+0000 7f8daea49700 1 -- 172.21.15.88:0/1411300971 >> 172.21.15.88:0/1411300971 conn(0x7f8da80fac90 msgr2=0x7f8da80fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:12.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.952+0000 7f8daea49700 1 -- 172.21.15.88:0/1411300971 shutdown_connections 2024-02-19T19:30:12.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:12.952+0000 7f8daea49700 1 -- 172.21.15.88:0/1411300971 wait complete. 2024-02-19T19:30:12.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:12 smithi132 ceph-mon[70676]: from='client.35454 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:12.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:12 smithi132 ceph-mon[70676]: from='client.35458 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:12.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:12 smithi132 ceph-mon[70676]: from='client.35462 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:12.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:12 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3329715556' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:30:12.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:12 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2284032542' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:30:13.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:13 smithi132 ceph-mon[70676]: from='client.35474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:13.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:13 smithi132 ceph-mon[70676]: pgmap v800: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:13.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:13 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1411300971' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:30:13.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:30:14.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:13 smithi088 ceph-mon[88643]: from='client.35474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:14.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:13 smithi088 ceph-mon[88643]: pgmap v800: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:14.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:13 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1411300971' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:30:14.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:30:15.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:15 smithi132 ceph-mon[70676]: pgmap v801: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:16.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:15 smithi088 ceph-mon[88643]: pgmap v801: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:17.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:17 smithi132 ceph-mon[70676]: pgmap v802: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:18.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:17 smithi088 ceph-mon[88643]: pgmap v802: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:19.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:19 smithi132 ceph-mon[70676]: pgmap v803: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:20.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:19 smithi088 ceph-mon[88643]: pgmap v803: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:21.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:21 smithi132 ceph-mon[70676]: pgmap v804: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:22.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:21 smithi088 ceph-mon[88643]: pgmap v804: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:23.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:23 smithi132 ceph-mon[70676]: pgmap v805: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:24.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:23 smithi088 ceph-mon[88643]: pgmap v805: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:25.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:25 smithi088 ceph-mon[88643]: pgmap v806: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:25.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:25 smithi132 ceph-mon[70676]: pgmap v806: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:26.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:26 smithi132 ceph-mon[70676]: pgmap v807: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:27.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:26 smithi088 ceph-mon[88643]: pgmap v807: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:28.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:30:28.736 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:30:29.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:29 smithi088 ceph-mon[88643]: pgmap v808: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:29.736 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:29 smithi132 ceph-mon[70676]: pgmap v808: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:31.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:31 smithi132 ceph-mon[70676]: pgmap v809: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:32.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:31 smithi088 ceph-mon[88643]: pgmap v809: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:33.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:33 smithi132 ceph-mon[70676]: pgmap v810: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:34.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:33 smithi088 ceph-mon[88643]: pgmap v810: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:35.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:35 smithi132 ceph-mon[70676]: pgmap v811: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:36.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:35 smithi088 ceph-mon[88643]: pgmap v811: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:37.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:37 smithi132 ceph-mon[70676]: pgmap v812: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:38.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:37 smithi088 ceph-mon[88643]: pgmap v812: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:39.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:39 smithi132 ceph-mon[70676]: pgmap v813: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:40.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:39 smithi088 ceph-mon[88643]: pgmap v813: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:41.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:41 smithi132 ceph-mon[70676]: pgmap v814: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:42.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:41 smithi088 ceph-mon[88643]: pgmap v814: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:43.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.070+0000 7f9d57921700 1 -- 172.21.15.88:0/3166507738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d500730f0 msgr2=0x7f9d500734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.070+0000 7f9d57921700 1 --2- 172.21.15.88:0/3166507738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d500730f0 0x7f9d500734c0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009910 tx=0x7f9d4c009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:43.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.070+0000 7f9d57921700 1 -- 172.21.15.88:0/3166507738 shutdown_connections 2024-02-19T19:30:43.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.070+0000 7f9d57921700 1 --2- 172.21.15.88:0/3166507738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d50073a00 0x7f9d50110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.070+0000 7f9d57921700 1 --2- 172.21.15.88:0/3166507738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d500730f0 0x7f9d500734c0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.070+0000 7f9d57921700 1 -- 172.21.15.88:0/3166507738 >> 172.21.15.88:0/3166507738 conn(0x7f9d500fbda0 msgr2=0x7f9d500fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:43.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.070+0000 7f9d57921700 1 -- 172.21.15.88:0/3166507738 shutdown_connections 2024-02-19T19:30:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.070+0000 7f9d57921700 1 -- 172.21.15.88:0/3166507738 wait complete. 2024-02-19T19:30:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.070+0000 7f9d57921700 1 Processor -- start 2024-02-19T19:30:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.070+0000 7f9d57921700 1 -- start start 2024-02-19T19:30:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d57921700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d500730f0 0x7f9d5019c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d57921700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d50073a00 0x7f9d5019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d57921700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5019d000 con 0x7f9d500730f0 2024-02-19T19:30:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d57921700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d501a0d90 con 0x7f9d50073a00 2024-02-19T19:30:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d556bd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d500730f0 0x7f9d5019c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d556bd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d500730f0 0x7f9d5019c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54612/0 (socket says 172.21.15.88:54612) 2024-02-19T19:30:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d556bd700 1 -- 172.21.15.88:0/725290901 learned_addr learned my addr 172.21.15.88:0/725290901 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d54ebc700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d50073a00 0x7f9d5019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d556bd700 1 -- 172.21.15.88:0/725290901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d50073a00 msgr2=0x7f9d5019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d556bd700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d50073a00 0x7f9d5019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d556bd700 1 -- 172.21.15.88:0/725290901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c0095f0 con 0x7f9d500730f0 2024-02-19T19:30:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d54ebc700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d50073a00 0x7f9d5019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:30:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d556bd700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d500730f0 0x7f9d5019c3e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c006010 tx=0x7f9d4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d467fc700 1 -- 172.21.15.88:0/725290901 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4c01d070 con 0x7f9d500730f0 2024-02-19T19:30:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d467fc700 1 -- 172.21.15.88:0/725290901 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d4c00f460 con 0x7f9d500730f0 2024-02-19T19:30:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d467fc700 1 -- 172.21.15.88:0/725290901 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d4c017630 con 0x7f9d500730f0 2024-02-19T19:30:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d57921700 1 -- 172.21.15.88:0/725290901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d501a1070 con 0x7f9d500730f0 2024-02-19T19:30:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.071+0000 7f9d57921700 1 -- 172.21.15.88:0/725290901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d501a15c0 con 0x7f9d500730f0 2024-02-19T19:30:43.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.072+0000 7f9d57921700 1 -- 172.21.15.88:0/725290901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5010e4d0 con 0x7f9d500730f0 2024-02-19T19:30:43.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.073+0000 7f9d467fc700 1 -- 172.21.15.88:0/725290901 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9d4c00f5d0 con 0x7f9d500730f0 2024-02-19T19:30:43.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.073+0000 7f9d467fc700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d3c06d8a0 0x7f9d3c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:43.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.073+0000 7f9d467fc700 1 -- 172.21.15.88:0/725290901 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9d4c08e980 con 0x7f9d500730f0 2024-02-19T19:30:43.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.073+0000 7f9d54ebc700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d3c06d8a0 0x7f9d3c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:43.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.074+0000 7f9d54ebc700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d3c06d8a0 0x7f9d3c06fd50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9d40005950 tx=0x7f9d400058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:43.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.078+0000 7f9d467fc700 1 -- 172.21.15.88:0/725290901 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d4c058130 con 0x7f9d500730f0 2024-02-19T19:30:43.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.297+0000 7f9d57921700 1 -- 172.21.15.88:0/725290901 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d501a18a0 con 0x7f9d3c06d8a0 2024-02-19T19:30:43.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.299+0000 7f9d467fc700 1 -- 172.21.15.88:0/725290901 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9d501a18a0 con 0x7f9d3c06d8a0 2024-02-19T19:30:43.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.301+0000 7f9d57921700 1 -- 172.21.15.88:0/725290901 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d3c06d8a0 msgr2=0x7f9d3c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.301+0000 7f9d57921700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d3c06d8a0 0x7f9d3c06fd50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f9d40005950 tx=0x7f9d400058e0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.301+0000 7f9d57921700 1 -- 172.21.15.88:0/725290901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d500730f0 msgr2=0x7f9d5019c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.301+0000 7f9d57921700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d500730f0 0x7f9d5019c3e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c006010 tx=0x7f9d4c00be20 comp rx=0 tx=0).stop 2024-02-19T19:30:43.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.301+0000 7f9d57921700 1 -- 172.21.15.88:0/725290901 shutdown_connections 2024-02-19T19:30:43.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.301+0000 7f9d57921700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d3c06d8a0 0x7f9d3c06fd50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.301+0000 7f9d57921700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d500730f0 0x7f9d5019c3e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.301+0000 7f9d57921700 1 --2- 172.21.15.88:0/725290901 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d50073a00 0x7f9d5019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.301+0000 7f9d57921700 1 -- 172.21.15.88:0/725290901 >> 172.21.15.88:0/725290901 conn(0x7f9d500fbda0 msgr2=0x7f9d50102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:43.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.301+0000 7f9d57921700 1 -- 172.21.15.88:0/725290901 shutdown_connections 2024-02-19T19:30:43.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.301+0000 7f9d57921700 1 -- 172.21.15.88:0/725290901 wait complete. 2024-02-19T19:30:43.314 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:30:43.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.405+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/2759127707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf0100e80 msgr2=0x7fbdf0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.405+0000 7fbdf6f6c700 1 --2- 172.21.15.88:0/2759127707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf0100e80 0x7fbdf0104e50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009910 tx=0x7fbddc009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:43.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/2759127707 shutdown_connections 2024-02-19T19:30:43.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 --2- 172.21.15.88:0/2759127707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf0100e80 0x7fbdf0104e50 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 --2- 172.21.15.88:0/2759127707 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf01004e0 0x7fbdf01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/2759127707 >> 172.21.15.88:0/2759127707 conn(0x7fbdf00fbda0 msgr2=0x7fbdf00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:43.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/2759127707 shutdown_connections 2024-02-19T19:30:43.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/2759127707 wait complete. 2024-02-19T19:30:43.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 Processor -- start 2024-02-19T19:30:43.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 -- start start 2024-02-19T19:30:43.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf01004e0 0x7fbdf0195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:43.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf0100e80 0x7fbdf0196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:43.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf0196b30 con 0x7fbdf01004e0 2024-02-19T19:30:43.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf6f6c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf019a850 con 0x7fbdf0100e80 2024-02-19T19:30:43.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.406+0000 7fbdf4d08700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf01004e0 0x7fbdf0195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:43.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdf4d08700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf01004e0 0x7fbdf0195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54644/0 (socket says 172.21.15.88:54644) 2024-02-19T19:30:43.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdf4d08700 1 -- 172.21.15.88:0/1579991736 learned_addr learned my addr 172.21.15.88:0/1579991736 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:43.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdeffff700 1 --2- 172.21.15.88:0/1579991736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf0100e80 0x7fbdf0196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:43.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdf4d08700 1 -- 172.21.15.88:0/1579991736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf0100e80 msgr2=0x7fbdf0196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdf4d08700 1 --2- 172.21.15.88:0/1579991736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf0100e80 0x7fbdf0196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdf4d08700 1 -- 172.21.15.88:0/1579991736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc0095f0 con 0x7fbdf01004e0 2024-02-19T19:30:43.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdf4d08700 1 --2- 172.21.15.88:0/1579991736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf01004e0 0x7fbdf0195f10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbde400d8d0 tx=0x7fbde400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:43.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdedffb700 1 -- 172.21.15.88:0/1579991736 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde4009940 con 0x7fbdf01004e0 2024-02-19T19:30:43.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdedffb700 1 -- 172.21.15.88:0/1579991736 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbde4010460 con 0x7fbdf01004e0 2024-02-19T19:30:43.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/1579991736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf019ab30 con 0x7fbdf01004e0 2024-02-19T19:30:43.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdedffb700 1 -- 172.21.15.88:0/1579991736 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde400f5f0 con 0x7fbdf01004e0 2024-02-19T19:30:43.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.407+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/1579991736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf019b080 con 0x7fbdf01004e0 2024-02-19T19:30:43.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.408+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/1579991736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf0108850 con 0x7fbdf01004e0 2024-02-19T19:30:43.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.409+0000 7fbdedffb700 1 -- 172.21.15.88:0/1579991736 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbde40105d0 con 0x7fbdf01004e0 2024-02-19T19:30:43.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.409+0000 7fbdedffb700 1 --2- 172.21.15.88:0/1579991736 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbde006d8a0 0x7fbde006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:43.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.409+0000 7fbdedffb700 1 -- 172.21.15.88:0/1579991736 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbde408d430 con 0x7fbdf01004e0 2024-02-19T19:30:43.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.409+0000 7fbdeffff700 1 --2- 172.21.15.88:0/1579991736 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbde006d8a0 0x7fbde006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:43.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.409+0000 7fbdeffff700 1 --2- 172.21.15.88:0/1579991736 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbde006d8a0 0x7fbde006fd50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbdf01974c0 tx=0x7fbddc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:43.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.414+0000 7fbdedffb700 1 -- 172.21.15.88:0/1579991736 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbde4056c60 con 0x7fbdf01004e0 2024-02-19T19:30:43.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.633+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/1579991736 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdf019b2f0 con 0x7fbde006d8a0 2024-02-19T19:30:43.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.634+0000 7fbdedffb700 1 -- 172.21.15.88:0/1579991736 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbdf019b2f0 con 0x7fbde006d8a0 2024-02-19T19:30:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.636+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/1579991736 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbde006d8a0 msgr2=0x7fbde006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.636+0000 7fbdf6f6c700 1 --2- 172.21.15.88:0/1579991736 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbde006d8a0 0x7fbde006fd50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbdf01974c0 tx=0x7fbddc01a040 comp rx=0 tx=0).stop 2024-02-19T19:30:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.636+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/1579991736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf01004e0 msgr2=0x7fbdf0195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.636+0000 7fbdf6f6c700 1 --2- 172.21.15.88:0/1579991736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf01004e0 0x7fbdf0195f10 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbde400d8d0 tx=0x7fbde400dc90 comp rx=0 tx=0).stop 2024-02-19T19:30:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.636+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/1579991736 shutdown_connections 2024-02-19T19:30:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.636+0000 7fbdf6f6c700 1 --2- 172.21.15.88:0/1579991736 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbde006d8a0 0x7fbde006fd50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.636+0000 7fbdf6f6c700 1 --2- 172.21.15.88:0/1579991736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbdf01004e0 0x7fbdf0195f10 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.636+0000 7fbdf6f6c700 1 --2- 172.21.15.88:0/1579991736 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbdf0100e80 0x7fbdf0196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.636+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/1579991736 >> 172.21.15.88:0/1579991736 conn(0x7fbdf00fbda0 msgr2=0x7fbdf00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.636+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/1579991736 shutdown_connections 2024-02-19T19:30:43.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.636+0000 7fbdf6f6c700 1 -- 172.21.15.88:0/1579991736 wait complete. 2024-02-19T19:30:43.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.741+0000 7f1b89afc700 1 -- 172.21.15.88:0/579470599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b84101420 msgr2=0x7f1b841017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.741+0000 7f1b89afc700 1 --2- 172.21.15.88:0/579470599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b84101420 0x7f1b841017f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1b74009910 tx=0x7f1b74009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:43.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.741+0000 7f1b89afc700 1 -- 172.21.15.88:0/579470599 shutdown_connections 2024-02-19T19:30:43.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.741+0000 7f1b89afc700 1 --2- 172.21.15.88:0/579470599 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b84101d30 0x7f1b8410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.741+0000 7f1b89afc700 1 --2- 172.21.15.88:0/579470599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b84101420 0x7f1b841017f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.741+0000 7f1b89afc700 1 -- 172.21.15.88:0/579470599 >> 172.21.15.88:0/579470599 conn(0x7f1b840fac80 msgr2=0x7f1b840fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:43.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.741+0000 7f1b89afc700 1 -- 172.21.15.88:0/579470599 shutdown_connections 2024-02-19T19:30:43.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.741+0000 7f1b89afc700 1 -- 172.21.15.88:0/579470599 wait complete. 2024-02-19T19:30:43.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.741+0000 7f1b89afc700 1 Processor -- start 2024-02-19T19:30:43.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.741+0000 7f1b89afc700 1 -- start start 2024-02-19T19:30:43.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b89afc700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b84101420 0x7f1b84195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:43.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b89afc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b84101d30 0x7f1b841963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:43.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b89afc700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b84196a90 con 0x7f1b84101d30 2024-02-19T19:30:43.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b89afc700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8419a820 con 0x7f1b84101420 2024-02-19T19:30:43.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b7affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b84101d30 0x7f1b841963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:43.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b7affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b84101d30 0x7f1b841963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54668/0 (socket says 172.21.15.88:54668) 2024-02-19T19:30:43.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b7affd700 1 -- 172.21.15.88:0/1845753606 learned_addr learned my addr 172.21.15.88:0/1845753606 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:43.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b837fe700 1 --2- 172.21.15.88:0/1845753606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b84101420 0x7f1b84195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:43.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b7affd700 1 -- 172.21.15.88:0/1845753606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b84101420 msgr2=0x7f1b84195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b7affd700 1 --2- 172.21.15.88:0/1845753606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b84101420 0x7f1b84195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b7affd700 1 -- 172.21.15.88:0/1845753606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b740095f0 con 0x7f1b84101d30 2024-02-19T19:30:43.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b7affd700 1 --2- 172.21.15.88:0/1845753606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b84101d30 0x7f1b841963b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00d8d0 tx=0x7f1b6c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:43.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b817fa700 1 -- 172.21.15.88:0/1845753606 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b6c0098a0 con 0x7f1b84101d30 2024-02-19T19:30:43.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b817fa700 1 -- 172.21.15.88:0/1845753606 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b6c010460 con 0x7f1b84101d30 2024-02-19T19:30:43.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b89afc700 1 -- 172.21.15.88:0/1845753606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8419ab00 con 0x7f1b84101d30 2024-02-19T19:30:43.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.742+0000 7f1b817fa700 1 -- 172.21.15.88:0/1845753606 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b6c00f5f0 con 0x7f1b84101d30 2024-02-19T19:30:43.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.743+0000 7f1b89afc700 1 -- 172.21.15.88:0/1845753606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8419b050 con 0x7f1b84101d30 2024-02-19T19:30:43.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.744+0000 7f1b89afc700 1 -- 172.21.15.88:0/1845753606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b84107900 con 0x7f1b84101d30 2024-02-19T19:30:43.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.744+0000 7f1b817fa700 1 -- 172.21.15.88:0/1845753606 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1b6c009a00 con 0x7f1b84101d30 2024-02-19T19:30:43.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.744+0000 7f1b817fa700 1 --2- 172.21.15.88:0/1845753606 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b7006d8a0 0x7f1b7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:43.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.744+0000 7f1b817fa700 1 -- 172.21.15.88:0/1845753606 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1b6c08d440 con 0x7f1b84101d30 2024-02-19T19:30:43.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.744+0000 7f1b837fe700 1 --2- 172.21.15.88:0/1845753606 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b7006d8a0 0x7f1b7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:43.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.745+0000 7f1b837fe700 1 --2- 172.21.15.88:0/1845753606 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b7006d8a0 0x7f1b7006fd50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b74006010 tx=0x7f1b7401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:43.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.749+0000 7f1b817fa700 1 -- 172.21.15.88:0/1845753606 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1b6c056bf0 con 0x7f1b84101d30 2024-02-19T19:30:43.955 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:43 smithi132 ceph-mon[70676]: pgmap v815: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:43.956 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:30:43.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.956+0000 7f1b89afc700 1 -- 172.21.15.88:0/1845753606 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b8419b2e0 con 0x7f1b7006d8a0 2024-02-19T19:30:43.960 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:43 smithi088 ceph-mon[88643]: pgmap v815: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:43.960 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:30:43.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.967+0000 7f1b817fa700 1 -- 172.21.15.88:0/1845753606 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1b8419b2e0 con 0x7f1b7006d8a0 2024-02-19T19:30:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:30:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (27m) 4m ago 38m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:30:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (38m) 4m ago 38m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:30:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (20m) 4m ago 36m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:30:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (24m) 4m ago 38m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:30:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (23m) 4m ago 36m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:30:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (27m) 4m ago 37m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:30:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (20m) 4m ago 32m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:30:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (19m) 4m ago 31m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:30:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (19m) 4m ago 32m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:30:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (4m) 4m ago 31m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (29m) 4m ago 39m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (28m) 4m ago 36m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (25m) 4m ago 40m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (25m) 4m ago 36m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (28m) 4m ago 38m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (28m) 4m ago 36m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (23m) 4m ago 35m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (22m) 4m ago 35m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (22m) 4m ago 34m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (21m) 4m ago 34m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (21m) 4m ago 34m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:30:43.972 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (20m) 4m ago 33m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:30:43.973 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (28m) 4m ago 37m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:30:43.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.969+0000 7f1b89afc700 1 -- 172.21.15.88:0/1845753606 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b7006d8a0 msgr2=0x7f1b7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.969+0000 7f1b89afc700 1 --2- 172.21.15.88:0/1845753606 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b7006d8a0 0x7f1b7006fd50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1b74006010 tx=0x7f1b7401a040 comp rx=0 tx=0).stop 2024-02-19T19:30:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.969+0000 7f1b89afc700 1 -- 172.21.15.88:0/1845753606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b84101d30 msgr2=0x7f1b841963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.969+0000 7f1b89afc700 1 --2- 172.21.15.88:0/1845753606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b84101d30 0x7f1b841963b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00d8d0 tx=0x7f1b6c00dbe0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.969+0000 7f1b89afc700 1 -- 172.21.15.88:0/1845753606 shutdown_connections 2024-02-19T19:30:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.969+0000 7f1b89afc700 1 --2- 172.21.15.88:0/1845753606 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1b7006d8a0 0x7f1b7006fd50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.969+0000 7f1b89afc700 1 --2- 172.21.15.88:0/1845753606 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1b84101420 0x7f1b84195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.969+0000 7f1b89afc700 1 --2- 172.21.15.88:0/1845753606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b84101d30 0x7f1b841963b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.969+0000 7f1b89afc700 1 -- 172.21.15.88:0/1845753606 >> 172.21.15.88:0/1845753606 conn(0x7f1b840fac80 msgr2=0x7f1b84104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.969+0000 7f1b89afc700 1 -- 172.21.15.88:0/1845753606 shutdown_connections 2024-02-19T19:30:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:43.969+0000 7f1b89afc700 1 -- 172.21.15.88:0/1845753606 wait complete. 2024-02-19T19:30:44.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.071+0000 7f9e1f71e700 1 -- 172.21.15.88:0/2827349350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e18100e80 msgr2=0x7f9e18104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:44.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.071+0000 7f9e1f71e700 1 --2- 172.21.15.88:0/2827349350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e18100e80 0x7f9e18104e50 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f9e08009910 tx=0x7f9e08009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:44.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.071+0000 7f9e1f71e700 1 -- 172.21.15.88:0/2827349350 shutdown_connections 2024-02-19T19:30:44.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.071+0000 7f9e1f71e700 1 --2- 172.21.15.88:0/2827349350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e18100e80 0x7f9e18104e50 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.071+0000 7f9e1f71e700 1 --2- 172.21.15.88:0/2827349350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e181004e0 0x7f9e181008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.071+0000 7f9e1f71e700 1 -- 172.21.15.88:0/2827349350 >> 172.21.15.88:0/2827349350 conn(0x7f9e180fbda0 msgr2=0x7f9e180fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:44.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.072+0000 7f9e1f71e700 1 -- 172.21.15.88:0/2827349350 shutdown_connections 2024-02-19T19:30:44.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.072+0000 7f9e1f71e700 1 -- 172.21.15.88:0/2827349350 wait complete. 2024-02-19T19:30:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.072+0000 7f9e1f71e700 1 Processor -- start 2024-02-19T19:30:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.072+0000 7f9e1f71e700 1 -- start start 2024-02-19T19:30:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.072+0000 7f9e1f71e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e181004e0 0x7f9e18196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.072+0000 7f9e1f71e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e18100e80 0x7f9e18196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.072+0000 7f9e1f71e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e18196c20 con 0x7f9e18100e80 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.072+0000 7f9e1f71e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1819a9b0 con 0x7f9e181004e0 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.072+0000 7f9e1ccb9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e18100e80 0x7f9e18196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.072+0000 7f9e1ccb9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e18100e80 0x7f9e18196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54686/0 (socket says 172.21.15.88:54686) 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.073+0000 7f9e1ccb9700 1 -- 172.21.15.88:0/591482532 learned_addr learned my addr 172.21.15.88:0/591482532 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.073+0000 7f9e1d4ba700 1 --2- 172.21.15.88:0/591482532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e181004e0 0x7f9e18196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.073+0000 7f9e1ccb9700 1 -- 172.21.15.88:0/591482532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e181004e0 msgr2=0x7f9e18196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.073+0000 7f9e1ccb9700 1 --2- 172.21.15.88:0/591482532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e181004e0 0x7f9e18196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.073+0000 7f9e1ccb9700 1 -- 172.21.15.88:0/591482532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e080095f0 con 0x7f9e18100e80 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.073+0000 7f9e1ccb9700 1 --2- 172.21.15.88:0/591482532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e18100e80 0x7f9e18196540 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9e08005f50 tx=0x7f9e0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.073+0000 7f9e0e7fc700 1 -- 172.21.15.88:0/591482532 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e0801d070 con 0x7f9e18100e80 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.073+0000 7f9e1f71e700 1 -- 172.21.15.88:0/591482532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1819ac30 con 0x7f9e18100e80 2024-02-19T19:30:44.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.073+0000 7f9e0e7fc700 1 -- 172.21.15.88:0/591482532 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e0800f460 con 0x7f9e18100e80 2024-02-19T19:30:44.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.073+0000 7f9e0e7fc700 1 -- 172.21.15.88:0/591482532 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e08017630 con 0x7f9e18100e80 2024-02-19T19:30:44.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.073+0000 7f9e1f71e700 1 -- 172.21.15.88:0/591482532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1819b120 con 0x7f9e18100e80 2024-02-19T19:30:44.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.074+0000 7f9e1f71e700 1 -- 172.21.15.88:0/591482532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e181087a0 con 0x7f9e18100e80 2024-02-19T19:30:44.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.075+0000 7f9e0e7fc700 1 -- 172.21.15.88:0/591482532 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9e0800f5d0 con 0x7f9e18100e80 2024-02-19T19:30:44.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.075+0000 7f9e0e7fc700 1 --2- 172.21.15.88:0/591482532 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e0406d8a0 0x7f9e0406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:44.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.075+0000 7f9e0e7fc700 1 -- 172.21.15.88:0/591482532 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9e0808eb50 con 0x7f9e18100e80 2024-02-19T19:30:44.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.075+0000 7f9e1d4ba700 1 --2- 172.21.15.88:0/591482532 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e0406d8a0 0x7f9e0406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:44.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.075+0000 7f9e1d4ba700 1 --2- 172.21.15.88:0/591482532 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e0406d8a0 0x7f9e0406fd50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9e14006fd0 tx=0x7f9e14008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:44.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.080+0000 7f9e0e7fc700 1 -- 172.21.15.88:0/591482532 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9e08058300 con 0x7f9e18100e80 2024-02-19T19:30:44.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.360+0000 7f9e1f71e700 1 -- 172.21.15.88:0/591482532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9e1804f2a0 con 0x7f9e18100e80 2024-02-19T19:30:44.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.360+0000 7f9e0e7fc700 1 -- 172.21.15.88:0/591482532 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f9e0805b9c0 con 0x7f9e18100e80 2024-02-19T19:30:44.364 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:30:44.364 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:30:44.364 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:30:44.364 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:30:44.364 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:30:44.364 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:30:44.364 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:30:44.364 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:30:44.365 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:30:44.365 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:30:44.365 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:30:44.365 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:30:44.365 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:30:44.365 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:30:44.365 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:30:44.365 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:30:44.365 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:30:44.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.363+0000 7f9e1f71e700 1 -- 172.21.15.88:0/591482532 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e0406d8a0 msgr2=0x7f9e0406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:44.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.363+0000 7f9e1f71e700 1 --2- 172.21.15.88:0/591482532 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e0406d8a0 0x7f9e0406fd50 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f9e14006fd0 tx=0x7f9e14008040 comp rx=0 tx=0).stop 2024-02-19T19:30:44.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.363+0000 7f9e1f71e700 1 -- 172.21.15.88:0/591482532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e18100e80 msgr2=0x7f9e18196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:44.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.363+0000 7f9e1f71e700 1 --2- 172.21.15.88:0/591482532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e18100e80 0x7f9e18196540 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9e08005f50 tx=0x7f9e0800be20 comp rx=0 tx=0).stop 2024-02-19T19:30:44.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.363+0000 7f9e1f71e700 1 -- 172.21.15.88:0/591482532 shutdown_connections 2024-02-19T19:30:44.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.363+0000 7f9e1f71e700 1 --2- 172.21.15.88:0/591482532 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9e0406d8a0 0x7f9e0406fd50 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.363+0000 7f9e1f71e700 1 --2- 172.21.15.88:0/591482532 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9e181004e0 0x7f9e18196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.363+0000 7f9e1f71e700 1 --2- 172.21.15.88:0/591482532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e18100e80 0x7f9e18196540 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.363+0000 7f9e1f71e700 1 -- 172.21.15.88:0/591482532 >> 172.21.15.88:0/591482532 conn(0x7f9e180fbda0 msgr2=0x7f9e180fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:44.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.363+0000 7f9e1f71e700 1 -- 172.21.15.88:0/591482532 shutdown_connections 2024-02-19T19:30:44.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.363+0000 7f9e1f71e700 1 -- 172.21.15.88:0/591482532 wait complete. 2024-02-19T19:30:44.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.466+0000 7f6918e13700 1 -- 172.21.15.88:0/80272371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69140ffd60 msgr2=0x7f6914103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:44.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.466+0000 7f6918e13700 1 --2- 172.21.15.88:0/80272371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69140ffd60 0x7f6914103d30 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6908009910 tx=0x7f6908009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:44.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.467+0000 7f6918e13700 1 -- 172.21.15.88:0/80272371 shutdown_connections 2024-02-19T19:30:44.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.467+0000 7f6918e13700 1 --2- 172.21.15.88:0/80272371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69140ffd60 0x7f6914103d30 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.467+0000 7f6918e13700 1 --2- 172.21.15.88:0/80272371 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69140ff3c0 0x7f69140ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.467+0000 7f6918e13700 1 -- 172.21.15.88:0/80272371 >> 172.21.15.88:0/80272371 conn(0x7f69140fac80 msgr2=0x7f69140fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:44.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.467+0000 7f6918e13700 1 -- 172.21.15.88:0/80272371 shutdown_connections 2024-02-19T19:30:44.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.467+0000 7f6918e13700 1 -- 172.21.15.88:0/80272371 wait complete. 2024-02-19T19:30:44.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.467+0000 7f6918e13700 1 Processor -- start 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.467+0000 7f6918e13700 1 -- start start 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f6918e13700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69140ff3c0 0x7f6914196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f6918e13700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69140ffd60 0x7f69141972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f6918e13700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69141978c0 con 0x7f69140ff3c0 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f6918e13700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6914197a30 con 0x7f69140ffd60 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f691259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69140ff3c0 0x7f6914196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f691259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69140ff3c0 0x7f6914196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54708/0 (socket says 172.21.15.88:54708) 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f691259c700 1 -- 172.21.15.88:0/2570024734 learned_addr learned my addr 172.21.15.88:0/2570024734 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f6911d9b700 1 --2- 172.21.15.88:0/2570024734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69140ffd60 0x7f69141972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f691259c700 1 -- 172.21.15.88:0/2570024734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69140ffd60 msgr2=0x7f69141972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f691259c700 1 --2- 172.21.15.88:0/2570024734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69140ffd60 0x7f69141972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f691259c700 1 -- 172.21.15.88:0/2570024734 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69080095f0 con 0x7f69140ff3c0 2024-02-19T19:30:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f691259c700 1 --2- 172.21.15.88:0/2570024734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69140ff3c0 0x7f6914196d60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f690400c8a0 tx=0x7f690400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f69037fe700 1 -- 172.21.15.88:0/2570024734 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f690400cea0 con 0x7f69140ff3c0 2024-02-19T19:30:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f69037fe700 1 -- 172.21.15.88:0/2570024734 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6904004830 con 0x7f69140ff3c0 2024-02-19T19:30:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f6918e13700 1 -- 172.21.15.88:0/2570024734 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f691419fdd0 con 0x7f69140ff3c0 2024-02-19T19:30:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f69037fe700 1 -- 172.21.15.88:0/2570024734 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69040055b0 con 0x7f69140ff3c0 2024-02-19T19:30:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.468+0000 7f6918e13700 1 -- 172.21.15.88:0/2570024734 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69141a0290 con 0x7f69140ff3c0 2024-02-19T19:30:44.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.469+0000 7f6918e13700 1 -- 172.21.15.88:0/2570024734 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6914107730 con 0x7f69140ff3c0 2024-02-19T19:30:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.470+0000 7f69037fe700 1 -- 172.21.15.88:0/2570024734 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f69040049a0 con 0x7f69140ff3c0 2024-02-19T19:30:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.470+0000 7f69037fe700 1 --2- 172.21.15.88:0/2570024734 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68fc06d8a0 0x7f68fc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.470+0000 7f6911d9b700 1 --2- 172.21.15.88:0/2570024734 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68fc06d8a0 0x7f68fc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.470+0000 7f69037fe700 1 -- 172.21.15.88:0/2570024734 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f690408d1d0 con 0x7f69140ff3c0 2024-02-19T19:30:44.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.471+0000 7f6911d9b700 1 --2- 172.21.15.88:0/2570024734 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68fc06d8a0 0x7f68fc06fd50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6908006010 tx=0x7f690801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:44.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.475+0000 7f69037fe700 1 -- 172.21.15.88:0/2570024734 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6904056980 con 0x7f69140ff3c0 2024-02-19T19:30:44.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.718+0000 7f6918e13700 1 -- 172.21.15.88:0/2570024734 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f691404f2a0 con 0x7f69140ff3c0 2024-02-19T19:30:44.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.719+0000 7f69037fe700 1 -- 172.21.15.88:0/2570024734 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f6904004c50 con 0x7f69140ff3c0 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:30:44.723 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:30:44.724 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:30:44.725 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:30:44.725 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:30:44.725 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:30:44.725 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:30:44.725 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:30:44.725 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:30:44.725 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:30:44.725 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:30:44.725 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:30:44.725 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:30:44.725 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.721+0000 7f6918e13700 1 -- 172.21.15.88:0/2570024734 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68fc06d8a0 msgr2=0x7f68fc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.721+0000 7f6918e13700 1 --2- 172.21.15.88:0/2570024734 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68fc06d8a0 0x7f68fc06fd50 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6908006010 tx=0x7f690801a040 comp rx=0 tx=0).stop 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.721+0000 7f6918e13700 1 -- 172.21.15.88:0/2570024734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69140ff3c0 msgr2=0x7f6914196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.721+0000 7f6918e13700 1 --2- 172.21.15.88:0/2570024734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69140ff3c0 0x7f6914196d60 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f690400c8a0 tx=0x7f690400cc60 comp rx=0 tx=0).stop 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.721+0000 7f6918e13700 1 -- 172.21.15.88:0/2570024734 shutdown_connections 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.721+0000 7f6918e13700 1 --2- 172.21.15.88:0/2570024734 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68fc06d8a0 0x7f68fc06fd50 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.721+0000 7f6918e13700 1 --2- 172.21.15.88:0/2570024734 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69140ff3c0 0x7f6914196d60 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.721+0000 7f6918e13700 1 --2- 172.21.15.88:0/2570024734 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69140ffd60 0x7f69141972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.721+0000 7f6918e13700 1 -- 172.21.15.88:0/2570024734 >> 172.21.15.88:0/2570024734 conn(0x7f69140fac80 msgr2=0x7f69140fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.721+0000 7f6918e13700 1 -- 172.21.15.88:0/2570024734 shutdown_connections 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.721+0000 7f6918e13700 1 -- 172.21.15.88:0/2570024734 wait complete. 2024-02-19T19:30:44.726 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:30:44.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.825+0000 7f2497491700 1 -- 172.21.15.88:0/1136763549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24900ff3c0 msgr2=0x7f24900ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:44.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.825+0000 7f2497491700 1 --2- 172.21.15.88:0/1136763549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24900ff3c0 0x7f24900ff790 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f2480009910 tx=0x7f2480009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:44.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.825+0000 7f2497491700 1 -- 172.21.15.88:0/1136763549 shutdown_connections 2024-02-19T19:30:44.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.825+0000 7f2497491700 1 --2- 172.21.15.88:0/1136763549 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24900ffd60 0x7f2490103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.825+0000 7f2497491700 1 --2- 172.21.15.88:0/1136763549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24900ff3c0 0x7f24900ff790 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.825+0000 7f2497491700 1 -- 172.21.15.88:0/1136763549 >> 172.21.15.88:0/1136763549 conn(0x7f24900fac60 msgr2=0x7f24900fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:44.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.825+0000 7f2497491700 1 -- 172.21.15.88:0/1136763549 shutdown_connections 2024-02-19T19:30:44.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.825+0000 7f2497491700 1 -- 172.21.15.88:0/1136763549 wait complete. 2024-02-19T19:30:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2497491700 1 Processor -- start 2024-02-19T19:30:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2497491700 1 -- start start 2024-02-19T19:30:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2497491700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24900ff3c0 0x7f2490198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2497491700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24900ffd60 0x7f24901985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2497491700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490198bf0 con 0x7f24900ffd60 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2497491700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490198d30 con 0x7f24900ff3c0 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2494a2c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24900ffd60 0x7f24901985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2494a2c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24900ffd60 0x7f24901985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54734/0 (socket says 172.21.15.88:54734) 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2494a2c700 1 -- 172.21.15.88:0/1935455162 learned_addr learned my addr 172.21.15.88:0/1935455162 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f249522d700 1 --2- 172.21.15.88:0/1935455162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24900ff3c0 0x7f2490198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2494a2c700 1 -- 172.21.15.88:0/1935455162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24900ff3c0 msgr2=0x7f2490198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2494a2c700 1 --2- 172.21.15.88:0/1935455162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24900ff3c0 0x7f2490198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2494a2c700 1 -- 172.21.15.88:0/1935455162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24800095f0 con 0x7f24900ffd60 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.826+0000 7f2494a2c700 1 --2- 172.21.15.88:0/1935455162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24900ffd60 0x7f24901985a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f248c00c960 tx=0x7f248c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.827+0000 7f24867fc700 1 -- 172.21.15.88:0/1935455162 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248c007a70 con 0x7f24900ffd60 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.827+0000 7f24867fc700 1 -- 172.21.15.88:0/1935455162 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f248c007bd0 con 0x7f24900ffd60 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.827+0000 7f2497491700 1 -- 172.21.15.88:0/1935455162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249019cb80 con 0x7f24900ffd60 2024-02-19T19:30:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.827+0000 7f24867fc700 1 -- 172.21.15.88:0/1935455162 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f248c0186c0 con 0x7f24900ffd60 2024-02-19T19:30:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.827+0000 7f2497491700 1 -- 172.21.15.88:0/1935455162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249019d0d0 con 0x7f24900ffd60 2024-02-19T19:30:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.828+0000 7f2497491700 1 -- 172.21.15.88:0/1935455162 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249004f2a0 con 0x7f24900ffd60 2024-02-19T19:30:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.828+0000 7f24867fc700 1 -- 172.21.15.88:0/1935455162 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f248c00faa0 con 0x7f24900ffd60 2024-02-19T19:30:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.829+0000 7f24867fc700 1 --2- 172.21.15.88:0/1935455162 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f247c06d8a0 0x7f247c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.829+0000 7f24867fc700 1 -- 172.21.15.88:0/1935455162 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f248c08d4b0 con 0x7f24900ffd60 2024-02-19T19:30:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.829+0000 7f249522d700 1 --2- 172.21.15.88:0/1935455162 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f247c06d8a0 0x7f247c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.829+0000 7f249522d700 1 --2- 172.21.15.88:0/1935455162 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f247c06d8a0 0x7f247c06fd50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f248000b5c0 tx=0x7f248001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:44.833+0000 7f24867fc700 1 -- 172.21.15.88:0/1935455162 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f248c05a8b0 con 0x7f24900ffd60 2024-02-19T19:30:44.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:44 smithi132 ceph-mon[70676]: from='client.35482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:44.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:44 smithi132 ceph-mon[70676]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:44.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:44 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/591482532' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:30:45.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.046+0000 7f2497491700 1 -- 172.21.15.88:0/1935455162 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f249019d4a0 con 0x7f247c06d8a0 2024-02-19T19:30:45.050 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:44 smithi088 ceph-mon[88643]: from='client.35482 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:45.050 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:44 smithi088 ceph-mon[88643]: from='client.35486 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:45.050 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:44 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/591482532' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:30:45.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.047+0000 7f24867fc700 1 -- 172.21.15.88:0/1935455162 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f249019d4a0 con 0x7f247c06d8a0 2024-02-19T19:30:45.051 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:30:45.051 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:30:45.051 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:30:45.051 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:30:45.051 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:30:45.051 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:30:45.052 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:30:45.052 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:30:45.052 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:30:45.052 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:30:45.052 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:30:45.052 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:30:45.052 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:30:45.052 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:30:45.052 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:30:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.050+0000 7f2497491700 1 -- 172.21.15.88:0/1935455162 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f247c06d8a0 msgr2=0x7f247c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.050+0000 7f2497491700 1 --2- 172.21.15.88:0/1935455162 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f247c06d8a0 0x7f247c06fd50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f248000b5c0 tx=0x7f248001a040 comp rx=0 tx=0).stop 2024-02-19T19:30:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.050+0000 7f2497491700 1 -- 172.21.15.88:0/1935455162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24900ffd60 msgr2=0x7f24901985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.050+0000 7f2497491700 1 --2- 172.21.15.88:0/1935455162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24900ffd60 0x7f24901985a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f248c00c960 tx=0x7f248c00cd20 comp rx=0 tx=0).stop 2024-02-19T19:30:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.050+0000 7f2497491700 1 -- 172.21.15.88:0/1935455162 shutdown_connections 2024-02-19T19:30:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.050+0000 7f2497491700 1 --2- 172.21.15.88:0/1935455162 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f247c06d8a0 0x7f247c06fd50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.050+0000 7f2497491700 1 --2- 172.21.15.88:0/1935455162 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24900ff3c0 0x7f2490198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.050+0000 7f2497491700 1 --2- 172.21.15.88:0/1935455162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f24900ffd60 0x7f24901985a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.050+0000 7f2497491700 1 -- 172.21.15.88:0/1935455162 >> 172.21.15.88:0/1935455162 conn(0x7f24900fac60 msgr2=0x7f2490102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.050+0000 7f2497491700 1 -- 172.21.15.88:0/1935455162 shutdown_connections 2024-02-19T19:30:45.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.050+0000 7f2497491700 1 -- 172.21.15.88:0/1935455162 wait complete. 2024-02-19T19:30:45.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.155+0000 7f42d9e03700 1 -- 172.21.15.88:0/3600243869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d4101420 msgr2=0x7f42d41017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:45.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.155+0000 7f42d9e03700 1 --2- 172.21.15.88:0/3600243869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d4101420 0x7f42d41017f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f42c4009910 tx=0x7f42c4009c20 comp rx=0 tx=0).stop 2024-02-19T19:30:45.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.155+0000 7f42d9e03700 1 -- 172.21.15.88:0/3600243869 shutdown_connections 2024-02-19T19:30:45.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.155+0000 7f42d9e03700 1 --2- 172.21.15.88:0/3600243869 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d4101d30 0x7f42d410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:45.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.155+0000 7f42d9e03700 1 --2- 172.21.15.88:0/3600243869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d4101420 0x7f42d41017f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:45.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.155+0000 7f42d9e03700 1 -- 172.21.15.88:0/3600243869 >> 172.21.15.88:0/3600243869 conn(0x7f42d40fac80 msgr2=0x7f42d40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:45.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.156+0000 7f42d9e03700 1 -- 172.21.15.88:0/3600243869 shutdown_connections 2024-02-19T19:30:45.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.156+0000 7f42d9e03700 1 -- 172.21.15.88:0/3600243869 wait complete. 2024-02-19T19:30:45.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.156+0000 7f42d9e03700 1 Processor -- start 2024-02-19T19:30:45.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.156+0000 7f42d9e03700 1 -- start start 2024-02-19T19:30:45.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.156+0000 7f42d9e03700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d4101420 0x7f42d40ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d9e03700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d4101d30 0x7f42d40ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d9e03700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40ffb00 con 0x7f42d4101d30 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d9e03700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d40ffc40 con 0x7f42d4101420 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d4101d30 0x7f42d40ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d4101d30 0x7f42d40ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54760/0 (socket says 172.21.15.88:54760) 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d2ffd700 1 -- 172.21.15.88:0/654508900 learned_addr learned my addr 172.21.15.88:0/654508900 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d37fe700 1 --2- 172.21.15.88:0/654508900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d4101420 0x7f42d40ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d2ffd700 1 -- 172.21.15.88:0/654508900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d4101420 msgr2=0x7f42d40ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d2ffd700 1 --2- 172.21.15.88:0/654508900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d4101420 0x7f42d40ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d2ffd700 1 -- 172.21.15.88:0/654508900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c40095f0 con 0x7f42d4101d30 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d2ffd700 1 --2- 172.21.15.88:0/654508900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d4101d30 0x7f42d40ff5c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f42c800d8d0 tx=0x7f42c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d0ff9700 1 -- 172.21.15.88:0/654508900 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c8009940 con 0x7f42d4101d30 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d9e03700 1 -- 172.21.15.88:0/654508900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d41a2040 con 0x7f42d4101d30 2024-02-19T19:30:45.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d0ff9700 1 -- 172.21.15.88:0/654508900 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42c8010460 con 0x7f42d4101d30 2024-02-19T19:30:45.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d0ff9700 1 -- 172.21.15.88:0/654508900 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c800f5f0 con 0x7f42d4101d30 2024-02-19T19:30:45.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.157+0000 7f42d9e03700 1 -- 172.21.15.88:0/654508900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d41a2450 con 0x7f42d4101d30 2024-02-19T19:30:45.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.158+0000 7f42d9e03700 1 -- 172.21.15.88:0/654508900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d4107900 con 0x7f42d4101d30 2024-02-19T19:30:45.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.159+0000 7f42d0ff9700 1 -- 172.21.15.88:0/654508900 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f42c80105d0 con 0x7f42d4101d30 2024-02-19T19:30:45.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.159+0000 7f42d0ff9700 1 --2- 172.21.15.88:0/654508900 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42bc06d8a0 0x7f42bc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:30:45.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.159+0000 7f42d37fe700 1 --2- 172.21.15.88:0/654508900 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42bc06d8a0 0x7f42bc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:30:45.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.159+0000 7f42d0ff9700 1 -- 172.21.15.88:0/654508900 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f42c808d540 con 0x7f42d4101d30 2024-02-19T19:30:45.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.160+0000 7f42d37fe700 1 --2- 172.21.15.88:0/654508900 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42bc06d8a0 0x7f42bc06fd50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f42c400b5c0 tx=0x7f42c401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:30:45.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.164+0000 7f42d0ff9700 1 -- 172.21.15.88:0/654508900 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f42c8056cf0 con 0x7f42d4101d30 2024-02-19T19:30:45.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.443+0000 7f42d9e03700 1 -- 172.21.15.88:0/654508900 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f42d40ffe00 con 0x7f42d4101d30 2024-02-19T19:30:45.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.443+0000 7f42d0ff9700 1 -- 172.21.15.88:0/654508900 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f42d40ffe00 con 0x7f42d4101d30 2024-02-19T19:30:45.447 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:30:45.447 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:30:45.447 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:30:45.447 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:30:45.447 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:30:45.447 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:30:45.448 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:30:45.449 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:30:45.449 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:30:45.449 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:30:45.449 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:30:45.449 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:30:45.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.445+0000 7f42d9e03700 1 -- 172.21.15.88:0/654508900 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42bc06d8a0 msgr2=0x7f42bc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:45.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.445+0000 7f42d9e03700 1 --2- 172.21.15.88:0/654508900 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42bc06d8a0 0x7f42bc06fd50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f42c400b5c0 tx=0x7f42c401a040 comp rx=0 tx=0).stop 2024-02-19T19:30:45.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.445+0000 7f42d9e03700 1 -- 172.21.15.88:0/654508900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d4101d30 msgr2=0x7f42d40ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:30:45.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.445+0000 7f42d9e03700 1 --2- 172.21.15.88:0/654508900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d4101d30 0x7f42d40ff5c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f42c800d8d0 tx=0x7f42c800dc90 comp rx=0 tx=0).stop 2024-02-19T19:30:45.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.445+0000 7f42d9e03700 1 -- 172.21.15.88:0/654508900 shutdown_connections 2024-02-19T19:30:45.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.445+0000 7f42d9e03700 1 --2- 172.21.15.88:0/654508900 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f42bc06d8a0 0x7f42bc06fd50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:45.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.445+0000 7f42d9e03700 1 --2- 172.21.15.88:0/654508900 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f42d4101420 0x7f42d40ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:45.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.445+0000 7f42d9e03700 1 --2- 172.21.15.88:0/654508900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d4101d30 0x7f42d40ff5c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:30:45.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.445+0000 7f42d9e03700 1 -- 172.21.15.88:0/654508900 >> 172.21.15.88:0/654508900 conn(0x7f42d40fac80 msgr2=0x7f42d4104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:30:45.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.445+0000 7f42d9e03700 1 -- 172.21.15.88:0/654508900 shutdown_connections 2024-02-19T19:30:45.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:30:45.445+0000 7f42d9e03700 1 -- 172.21.15.88:0/654508900 wait complete. 2024-02-19T19:30:45.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:45 smithi132 ceph-mon[70676]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:45.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:45 smithi132 ceph-mon[70676]: pgmap v816: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:45.995 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:45 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2570024734' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:30:45.996 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:45 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/654508900' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:30:46.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:45 smithi088 ceph-mon[88643]: from='client.35490 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:46.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:45 smithi088 ceph-mon[88643]: pgmap v816: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:46.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:45 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2570024734' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:30:46.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:45 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/654508900' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:30:46.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:46 smithi132 ceph-mon[70676]: from='client.35502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:47.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:46 smithi088 ceph-mon[88643]: from='client.35502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:30:47.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:47 smithi132 ceph-mon[70676]: pgmap v817: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:48.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:47 smithi088 ceph-mon[88643]: pgmap v817: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:48.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:48 smithi132 ceph-mon[70676]: pgmap v818: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:49.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:48 smithi088 ceph-mon[88643]: pgmap v818: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:51.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:51 smithi132 ceph-mon[70676]: pgmap v819: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:52.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:51 smithi088 ceph-mon[88643]: pgmap v819: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:53.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:53 smithi132 ceph-mon[70676]: pgmap v820: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:54.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:53 smithi088 ceph-mon[88643]: pgmap v820: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:55.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:55 smithi132 ceph-mon[70676]: pgmap v821: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:56.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:55 smithi088 ceph-mon[88643]: pgmap v821: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:57.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:57 smithi132 ceph-mon[70676]: pgmap v822: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:58.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:57 smithi088 ceph-mon[88643]: pgmap v822: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:30:58.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:30:59.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:30:59.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:30:59 smithi132 ceph-mon[70676]: pgmap v823: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:00.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:30:59 smithi088 ceph-mon[88643]: pgmap v823: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:01.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:01 smithi132 ceph-mon[70676]: pgmap v824: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:02.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:01 smithi088 ceph-mon[88643]: pgmap v824: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:03.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:03 smithi132 ceph-mon[70676]: pgmap v825: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:04.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:03 smithi088 ceph-mon[88643]: pgmap v825: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:05.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:04 smithi088 ceph-mon[88643]: pgmap v826: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:05.236 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:04 smithi132 ceph-mon[70676]: pgmap v826: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:07.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:07 smithi132 ceph-mon[70676]: pgmap v827: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:08.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:07 smithi088 ceph-mon[88643]: pgmap v827: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:09.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:09 smithi132 ceph-mon[70676]: pgmap v828: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:10.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:09 smithi088 ceph-mon[88643]: pgmap v828: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:11.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:11 smithi132 ceph-mon[70676]: pgmap v829: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:12.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:11 smithi088 ceph-mon[88643]: pgmap v829: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:13.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:13 smithi132 ceph-mon[70676]: pgmap v830: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:13.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:31:14.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:13 smithi088 ceph-mon[88643]: pgmap v830: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:14.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:31:15.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.561+0000 7f2581e10700 1 -- 172.21.15.88:0/3189619001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f257c101d30 msgr2=0x7f257c10a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:15.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.561+0000 7f2581e10700 1 --2- 172.21.15.88:0/3189619001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f257c101d30 0x7f257c10a230 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f256c009910 tx=0x7f256c009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:15.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.561+0000 7f2581e10700 1 -- 172.21.15.88:0/3189619001 shutdown_connections 2024-02-19T19:31:15.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.562+0000 7f2581e10700 1 --2- 172.21.15.88:0/3189619001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f257c101d30 0x7f257c10a230 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:15.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.562+0000 7f2581e10700 1 --2- 172.21.15.88:0/3189619001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257c101420 0x7f257c1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:15.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.562+0000 7f2581e10700 1 -- 172.21.15.88:0/3189619001 >> 172.21.15.88:0/3189619001 conn(0x7f257c0fac80 msgr2=0x7f257c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:15.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.562+0000 7f2581e10700 1 -- 172.21.15.88:0/3189619001 shutdown_connections 2024-02-19T19:31:15.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.562+0000 7f2581e10700 1 -- 172.21.15.88:0/3189619001 wait complete. 2024-02-19T19:31:15.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.562+0000 7f2581e10700 1 Processor -- start 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f2581e10700 1 -- start start 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f2581e10700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f257c101420 0x7f257c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f2581e10700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257c101d30 0x7f257c1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f2581e10700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c196a80 con 0x7f257c101420 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f2581e10700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f257c19a810 con 0x7f257c101d30 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f257b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f257c101420 0x7f257c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f257b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f257c101420 0x7f257c195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53578/0 (socket says 172.21.15.88:53578) 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f257b7fe700 1 -- 172.21.15.88:0/3523521926 learned_addr learned my addr 172.21.15.88:0/3523521926 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f257affd700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257c101d30 0x7f257c1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f257b7fe700 1 -- 172.21.15.88:0/3523521926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257c101d30 msgr2=0x7f257c1963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f257b7fe700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257c101d30 0x7f257c1963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:15.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f257b7fe700 1 -- 172.21.15.88:0/3523521926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f256c0095f0 con 0x7f257c101420 2024-02-19T19:31:15.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f257affd700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257c101d30 0x7f257c1963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:31:15.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.563+0000 7f257b7fe700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f257c101420 0x7f257c195e60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f257000d8d0 tx=0x7f257000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:15.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.564+0000 7f2578ff9700 1 -- 172.21.15.88:0/3523521926 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2570009940 con 0x7f257c101420 2024-02-19T19:31:15.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.564+0000 7f2581e10700 1 -- 172.21.15.88:0/3523521926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f257c19aaf0 con 0x7f257c101420 2024-02-19T19:31:15.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.564+0000 7f2578ff9700 1 -- 172.21.15.88:0/3523521926 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2570010460 con 0x7f257c101420 2024-02-19T19:31:15.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.564+0000 7f2578ff9700 1 -- 172.21.15.88:0/3523521926 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f257000f5f0 con 0x7f257c101420 2024-02-19T19:31:15.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.564+0000 7f2581e10700 1 -- 172.21.15.88:0/3523521926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f257c19b040 con 0x7f257c101420 2024-02-19T19:31:15.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.565+0000 7f2581e10700 1 -- 172.21.15.88:0/3523521926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f257c1078a0 con 0x7f257c101420 2024-02-19T19:31:15.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.565+0000 7f2578ff9700 1 -- 172.21.15.88:0/3523521926 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2570009aa0 con 0x7f257c101420 2024-02-19T19:31:15.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.566+0000 7f2578ff9700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f256406d8a0 0x7f256406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:15.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.566+0000 7f2578ff9700 1 -- 172.21.15.88:0/3523521926 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f257008d550 con 0x7f257c101420 2024-02-19T19:31:15.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.566+0000 7f257affd700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f256406d8a0 0x7f256406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:15.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.566+0000 7f257affd700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f256406d8a0 0x7f256406fd50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f256c00b5c0 tx=0x7f256c0053a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:15.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.571+0000 7f2578ff9700 1 -- 172.21.15.88:0/3523521926 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2570056d00 con 0x7f257c101420 2024-02-19T19:31:15.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.788+0000 7f2581e10700 1 -- 172.21.15.88:0/3523521926 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f257c19b260 con 0x7f256406d8a0 2024-02-19T19:31:15.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.790+0000 7f2578ff9700 1 -- 172.21.15.88:0/3523521926 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f257c19b260 con 0x7f256406d8a0 2024-02-19T19:31:15.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.793+0000 7f2581e10700 1 -- 172.21.15.88:0/3523521926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f256406d8a0 msgr2=0x7f256406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:15.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.793+0000 7f2581e10700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f256406d8a0 0x7f256406fd50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f256c00b5c0 tx=0x7f256c0053a0 comp rx=0 tx=0).stop 2024-02-19T19:31:15.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.793+0000 7f2581e10700 1 -- 172.21.15.88:0/3523521926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f257c101420 msgr2=0x7f257c195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:15.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.793+0000 7f2581e10700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f257c101420 0x7f257c195e60 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f257000d8d0 tx=0x7f257000dc90 comp rx=0 tx=0).stop 2024-02-19T19:31:15.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.793+0000 7f2581e10700 1 -- 172.21.15.88:0/3523521926 shutdown_connections 2024-02-19T19:31:15.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.793+0000 7f2581e10700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f256406d8a0 0x7f256406fd50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:15.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.793+0000 7f2581e10700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f257c101420 0x7f257c195e60 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:15.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.793+0000 7f2581e10700 1 --2- 172.21.15.88:0/3523521926 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f257c101d30 0x7f257c1963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:15.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.793+0000 7f2581e10700 1 -- 172.21.15.88:0/3523521926 >> 172.21.15.88:0/3523521926 conn(0x7f257c0fac80 msgr2=0x7f257c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:15.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.793+0000 7f2581e10700 1 -- 172.21.15.88:0/3523521926 shutdown_connections 2024-02-19T19:31:15.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.793+0000 7f2581e10700 1 -- 172.21.15.88:0/3523521926 wait complete. 2024-02-19T19:31:15.806 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:31:15.833 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:15 smithi088 ceph-mon[88643]: pgmap v831: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:15.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.897+0000 7fb3918ee700 1 -- 172.21.15.88:0/130445383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c0ff3c0 msgr2=0x7fb38c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:15.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.897+0000 7fb3918ee700 1 --2- 172.21.15.88:0/130445383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c0ff3c0 0x7fb38c0ff790 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009910 tx=0x7fb37c009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:15.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.897+0000 7fb3918ee700 1 -- 172.21.15.88:0/130445383 shutdown_connections 2024-02-19T19:31:15.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.897+0000 7fb3918ee700 1 --2- 172.21.15.88:0/130445383 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb38c0ffd60 0x7fb38c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:15.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.897+0000 7fb3918ee700 1 --2- 172.21.15.88:0/130445383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c0ff3c0 0x7fb38c0ff790 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:15.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.897+0000 7fb3918ee700 1 -- 172.21.15.88:0/130445383 >> 172.21.15.88:0/130445383 conn(0x7fb38c0fac80 msgr2=0x7fb38c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:15.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.897+0000 7fb3918ee700 1 -- 172.21.15.88:0/130445383 shutdown_connections 2024-02-19T19:31:15.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.897+0000 7fb3918ee700 1 -- 172.21.15.88:0/130445383 wait complete. 2024-02-19T19:31:15.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.898+0000 7fb3918ee700 1 Processor -- start 2024-02-19T19:31:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.898+0000 7fb3918ee700 1 -- start start 2024-02-19T19:31:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.898+0000 7fb3918ee700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb38c0ff3c0 0x7fb38c194e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.898+0000 7fb3918ee700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c0ffd60 0x7fb38c1953d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.898+0000 7fb3918ee700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c195ab0 con 0x7fb38c0ffd60 2024-02-19T19:31:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.898+0000 7fb3918ee700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c199840 con 0x7fb38c0ff3c0 2024-02-19T19:31:15.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.898+0000 7fb38a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c0ffd60 0x7fb38c1953d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.898+0000 7fb38a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c0ffd60 0x7fb38c1953d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53584/0 (socket says 172.21.15.88:53584) 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.898+0000 7fb38a7fc700 1 -- 172.21.15.88:0/2446754538 learned_addr learned my addr 172.21.15.88:0/2446754538 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.899+0000 7fb38affd700 1 --2- 172.21.15.88:0/2446754538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb38c0ff3c0 0x7fb38c194e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.899+0000 7fb38a7fc700 1 -- 172.21.15.88:0/2446754538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb38c0ff3c0 msgr2=0x7fb38c194e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.899+0000 7fb38a7fc700 1 --2- 172.21.15.88:0/2446754538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb38c0ff3c0 0x7fb38c194e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.899+0000 7fb38a7fc700 1 -- 172.21.15.88:0/2446754538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c0095f0 con 0x7fb38c0ffd60 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.899+0000 7fb38a7fc700 1 --2- 172.21.15.88:0/2446754538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c0ffd60 0x7fb38c1953d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb38000d930 tx=0x7fb38000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.899+0000 7fb3908ec700 1 -- 172.21.15.88:0/2446754538 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb38000ec20 con 0x7fb38c0ffd60 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.899+0000 7fb3908ec700 1 -- 172.21.15.88:0/2446754538 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb38000ed80 con 0x7fb38c0ffd60 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.899+0000 7fb3918ee700 1 -- 172.21.15.88:0/2446754538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c199ac0 con 0x7fb38c0ffd60 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.899+0000 7fb3908ec700 1 -- 172.21.15.88:0/2446754538 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3800147a0 con 0x7fb38c0ffd60 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.899+0000 7fb3918ee700 1 -- 172.21.15.88:0/2446754538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c19a010 con 0x7fb38c0ffd60 2024-02-19T19:31:15.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.900+0000 7fb3918ee700 1 -- 172.21.15.88:0/2446754538 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c04f2a0 con 0x7fb38c0ffd60 2024-02-19T19:31:15.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.901+0000 7fb3908ec700 1 -- 172.21.15.88:0/2446754538 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb3800149d0 con 0x7fb38c0ffd60 2024-02-19T19:31:15.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.901+0000 7fb3908ec700 1 --2- 172.21.15.88:0/2446754538 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb37406d8a0 0x7fb37406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:15.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.901+0000 7fb3908ec700 1 -- 172.21.15.88:0/2446754538 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb38008eea0 con 0x7fb38c0ffd60 2024-02-19T19:31:15.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.901+0000 7fb38affd700 1 --2- 172.21.15.88:0/2446754538 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb37406d8a0 0x7fb37406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:15.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.902+0000 7fb38affd700 1 --2- 172.21.15.88:0/2446754538 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb37406d8a0 0x7fb37406fd50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00b5c0 tx=0x7fb37c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:15.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:15.907+0000 7fb3908ec700 1 -- 172.21.15.88:0/2446754538 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb380058700 con 0x7fb38c0ffd60 2024-02-19T19:31:15.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:15 smithi132 ceph-mon[70676]: pgmap v831: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:16.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.132+0000 7fb3918ee700 1 -- 172.21.15.88:0/2446754538 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb38c19a400 con 0x7fb37406d8a0 2024-02-19T19:31:16.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.133+0000 7fb3908ec700 1 -- 172.21.15.88:0/2446754538 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb38c19a400 con 0x7fb37406d8a0 2024-02-19T19:31:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.135+0000 7fb3918ee700 1 -- 172.21.15.88:0/2446754538 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb37406d8a0 msgr2=0x7fb37406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.135+0000 7fb3918ee700 1 --2- 172.21.15.88:0/2446754538 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb37406d8a0 0x7fb37406fd50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00b5c0 tx=0x7fb37c01a040 comp rx=0 tx=0).stop 2024-02-19T19:31:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.135+0000 7fb3918ee700 1 -- 172.21.15.88:0/2446754538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c0ffd60 msgr2=0x7fb38c1953d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.135+0000 7fb3918ee700 1 --2- 172.21.15.88:0/2446754538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c0ffd60 0x7fb38c1953d0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb38000d930 tx=0x7fb38000dcf0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.135+0000 7fb3918ee700 1 -- 172.21.15.88:0/2446754538 shutdown_connections 2024-02-19T19:31:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.135+0000 7fb3918ee700 1 --2- 172.21.15.88:0/2446754538 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb37406d8a0 0x7fb37406fd50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.135+0000 7fb3918ee700 1 --2- 172.21.15.88:0/2446754538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb38c0ff3c0 0x7fb38c194e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.135+0000 7fb3918ee700 1 --2- 172.21.15.88:0/2446754538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb38c0ffd60 0x7fb38c1953d0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.135+0000 7fb3918ee700 1 -- 172.21.15.88:0/2446754538 >> 172.21.15.88:0/2446754538 conn(0x7fb38c0fac80 msgr2=0x7fb38c0fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.136+0000 7fb3918ee700 1 -- 172.21.15.88:0/2446754538 shutdown_connections 2024-02-19T19:31:16.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.136+0000 7fb3918ee700 1 -- 172.21.15.88:0/2446754538 wait complete. 2024-02-19T19:31:16.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.239+0000 7f5d37f29700 1 -- 172.21.15.88:0/2790915622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d30100e80 msgr2=0x7f5d30104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.239+0000 7f5d37f29700 1 --2- 172.21.15.88:0/2790915622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d30100e80 0x7f5d30104e50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009910 tx=0x7f5d2c009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:16.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.239+0000 7f5d37f29700 1 -- 172.21.15.88:0/2790915622 shutdown_connections 2024-02-19T19:31:16.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.239+0000 7f5d37f29700 1 --2- 172.21.15.88:0/2790915622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d30100e80 0x7f5d30104e50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.239+0000 7f5d37f29700 1 --2- 172.21.15.88:0/2790915622 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d301004e0 0x7f5d301008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.239+0000 7f5d37f29700 1 -- 172.21.15.88:0/2790915622 >> 172.21.15.88:0/2790915622 conn(0x7f5d300fbda0 msgr2=0x7f5d300fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.239+0000 7f5d37f29700 1 -- 172.21.15.88:0/2790915622 shutdown_connections 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.239+0000 7f5d37f29700 1 -- 172.21.15.88:0/2790915622 wait complete. 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d37f29700 1 Processor -- start 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d37f29700 1 -- start start 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d37f29700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d301004e0 0x7f5d30073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d37f29700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d30100e80 0x7f5d30073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d37f29700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d30073ad0 con 0x7f5d30100e80 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d37f29700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d30073c10 con 0x7f5d301004e0 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d354c4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d30100e80 0x7f5d30073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d354c4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d30100e80 0x7f5d30073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53604/0 (socket says 172.21.15.88:53604) 2024-02-19T19:31:16.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d354c4700 1 -- 172.21.15.88:0/1531580396 learned_addr learned my addr 172.21.15.88:0/1531580396 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:16.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d35cc5700 1 --2- 172.21.15.88:0/1531580396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d301004e0 0x7f5d30073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:16.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d354c4700 1 -- 172.21.15.88:0/1531580396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d301004e0 msgr2=0x7f5d30073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d354c4700 1 --2- 172.21.15.88:0/1531580396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d301004e0 0x7f5d30073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.240+0000 7f5d354c4700 1 -- 172.21.15.88:0/1531580396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c0095f0 con 0x7f5d30100e80 2024-02-19T19:31:16.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.241+0000 7f5d354c4700 1 --2- 172.21.15.88:0/1531580396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d30100e80 0x7f5d30073590 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00bda0 tx=0x7f5d2c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:16.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.241+0000 7f5d26ffd700 1 -- 172.21.15.88:0/1531580396 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d2c01d070 con 0x7f5d30100e80 2024-02-19T19:31:16.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.241+0000 7f5d37f29700 1 -- 172.21.15.88:0/1531580396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d30073e90 con 0x7f5d30100e80 2024-02-19T19:31:16.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.241+0000 7f5d26ffd700 1 -- 172.21.15.88:0/1531580396 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d2c004fb0 con 0x7f5d30100e80 2024-02-19T19:31:16.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.241+0000 7f5d26ffd700 1 -- 172.21.15.88:0/1531580396 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d2c00f480 con 0x7f5d30100e80 2024-02-19T19:31:16.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.241+0000 7f5d37f29700 1 -- 172.21.15.88:0/1531580396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d301a23c0 con 0x7f5d30100e80 2024-02-19T19:31:16.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.242+0000 7f5d37f29700 1 -- 172.21.15.88:0/1531580396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d30108850 con 0x7f5d30100e80 2024-02-19T19:31:16.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.242+0000 7f5d26ffd700 1 -- 172.21.15.88:0/1531580396 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5d2c022870 con 0x7f5d30100e80 2024-02-19T19:31:16.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.243+0000 7f5d26ffd700 1 --2- 172.21.15.88:0/1531580396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d1c06d8a0 0x7f5d1c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:16.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.243+0000 7f5d35cc5700 1 --2- 172.21.15.88:0/1531580396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d1c06d8a0 0x7f5d1c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:16.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.243+0000 7f5d26ffd700 1 -- 172.21.15.88:0/1531580396 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5d2c08ecb0 con 0x7f5d30100e80 2024-02-19T19:31:16.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.243+0000 7f5d35cc5700 1 --2- 172.21.15.88:0/1531580396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d1c06d8a0 0x7f5d1c06fd50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5d20005ea0 tx=0x7f5d20005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:16.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.248+0000 7f5d26ffd700 1 -- 172.21.15.88:0/1531580396 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5d2c058630 con 0x7f5d30100e80 2024-02-19T19:31:16.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.453+0000 7f5d37f29700 1 -- 172.21.15.88:0/1531580396 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d30074660 con 0x7f5d1c06d8a0 2024-02-19T19:31:16.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.464+0000 7f5d26ffd700 1 -- 172.21.15.88:0/1531580396 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5d30074660 con 0x7f5d1c06d8a0 2024-02-19T19:31:16.468 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:31:16.468 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (28m) 4m ago 38m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:31:16.468 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (39m) 4m ago 39m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:31:16.468 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (21m) 4m ago 37m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:31:16.468 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (24m) 4m ago 39m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:31:16.468 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (24m) 4m ago 37m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:31:16.468 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (27m) 4m ago 38m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:31:16.468 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (20m) 4m ago 32m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:31:16.468 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (20m) 4m ago 32m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (20m) 4m ago 32m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (4m) 4m ago 32m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (29m) 4m ago 40m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (29m) 4m ago 37m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (26m) 4m ago 40m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (25m) 4m ago 37m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (29m) 4m ago 39m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (29m) 4m ago 37m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (23m) 4m ago 36m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (23m) 4m ago 35m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (22m) 4m ago 35m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (22m) 4m ago 35m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (21m) 4m ago 34m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (20m) 4m ago 34m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:31:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (28m) 4m ago 37m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:31:16.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.467+0000 7f5d37f29700 1 -- 172.21.15.88:0/1531580396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d1c06d8a0 msgr2=0x7f5d1c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.467+0000 7f5d37f29700 1 --2- 172.21.15.88:0/1531580396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d1c06d8a0 0x7f5d1c06fd50 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5d20005ea0 tx=0x7f5d20005e30 comp rx=0 tx=0).stop 2024-02-19T19:31:16.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.467+0000 7f5d37f29700 1 -- 172.21.15.88:0/1531580396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d30100e80 msgr2=0x7f5d30073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.467+0000 7f5d37f29700 1 --2- 172.21.15.88:0/1531580396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d30100e80 0x7f5d30073590 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00bda0 tx=0x7f5d2c00be80 comp rx=0 tx=0).stop 2024-02-19T19:31:16.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.467+0000 7f5d37f29700 1 -- 172.21.15.88:0/1531580396 shutdown_connections 2024-02-19T19:31:16.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.467+0000 7f5d37f29700 1 --2- 172.21.15.88:0/1531580396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5d1c06d8a0 0x7f5d1c06fd50 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.467+0000 7f5d37f29700 1 --2- 172.21.15.88:0/1531580396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5d301004e0 0x7f5d30073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.467+0000 7f5d37f29700 1 --2- 172.21.15.88:0/1531580396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d30100e80 0x7f5d30073590 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.467+0000 7f5d37f29700 1 -- 172.21.15.88:0/1531580396 >> 172.21.15.88:0/1531580396 conn(0x7f5d300fbda0 msgr2=0x7f5d300fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:16.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.467+0000 7f5d37f29700 1 -- 172.21.15.88:0/1531580396 shutdown_connections 2024-02-19T19:31:16.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.467+0000 7f5d37f29700 1 -- 172.21.15.88:0/1531580396 wait complete. 2024-02-19T19:31:16.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.568+0000 7fdf516e2700 1 -- 172.21.15.88:0/2616013593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c0ffd60 msgr2=0x7fdf4c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.568+0000 7fdf516e2700 1 --2- 172.21.15.88:0/2616013593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c0ffd60 0x7fdf4c103d30 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdf40009910 tx=0x7fdf40009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:16.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.568+0000 7fdf516e2700 1 -- 172.21.15.88:0/2616013593 shutdown_connections 2024-02-19T19:31:16.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.568+0000 7fdf516e2700 1 --2- 172.21.15.88:0/2616013593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c0ffd60 0x7fdf4c103d30 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.568+0000 7fdf516e2700 1 --2- 172.21.15.88:0/2616013593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0ff3c0 0x7fdf4c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.568+0000 7fdf516e2700 1 -- 172.21.15.88:0/2616013593 >> 172.21.15.88:0/2616013593 conn(0x7fdf4c0fac80 msgr2=0x7fdf4c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:16.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.568+0000 7fdf516e2700 1 -- 172.21.15.88:0/2616013593 shutdown_connections 2024-02-19T19:31:16.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.568+0000 7fdf516e2700 1 -- 172.21.15.88:0/2616013593 wait complete. 2024-02-19T19:31:16.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.569+0000 7fdf516e2700 1 Processor -- start 2024-02-19T19:31:16.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.569+0000 7fdf516e2700 1 -- start start 2024-02-19T19:31:16.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.569+0000 7fdf516e2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0ff3c0 0x7fdf4c195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:16.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.569+0000 7fdf516e2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c0ffd60 0x7fdf4c1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:16.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.569+0000 7fdf516e2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c196aa0 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.569+0000 7fdf516e2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4c19a830 con 0x7fdf4c0ff3c0 2024-02-19T19:31:16.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf4a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c0ffd60 0x7fdf4c1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:16.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf4a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c0ffd60 0x7fdf4c1963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53624/0 (socket says 172.21.15.88:53624) 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf4a7fc700 1 -- 172.21.15.88:0/487214667 learned_addr learned my addr 172.21.15.88:0/487214667 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf4affd700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0ff3c0 0x7fdf4c195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf4a7fc700 1 -- 172.21.15.88:0/487214667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0ff3c0 msgr2=0x7fdf4c195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf4a7fc700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0ff3c0 0x7fdf4c195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf4a7fc700 1 -- 172.21.15.88:0/487214667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf400095f0 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf4affd700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0ff3c0 0x7fdf4c195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf4a7fc700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c0ffd60 0x7fdf4c1963c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fdf400092e0 tx=0x7fdf4000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf33fff700 1 -- 172.21.15.88:0/487214667 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf4001d070 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf516e2700 1 -- 172.21.15.88:0/487214667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf4c19aab0 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf33fff700 1 -- 172.21.15.88:0/487214667 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf40005dc0 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf33fff700 1 -- 172.21.15.88:0/487214667 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf40017400 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.570+0000 7fdf516e2700 1 -- 172.21.15.88:0/487214667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4c19afa0 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.571+0000 7fdf516e2700 1 -- 172.21.15.88:0/487214667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf4c04f2a0 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.572+0000 7fdf33fff700 1 -- 172.21.15.88:0/487214667 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdf4000f460 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.572+0000 7fdf33fff700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3406d8a0 0x7fdf3406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:16.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.572+0000 7fdf33fff700 1 -- 172.21.15.88:0/487214667 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdf4008ea10 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.572+0000 7fdf4affd700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3406d8a0 0x7fdf3406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:16.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.573+0000 7fdf4affd700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3406d8a0 0x7fdf3406fd50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c005950 tx=0x7fdf3c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:16.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.578+0000 7fdf33fff700 1 -- 172.21.15.88:0/487214667 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf400581c0 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.855+0000 7fdf516e2700 1 -- 172.21.15.88:0/487214667 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdf4c19b280 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.856+0000 7fdf33fff700 1 -- 172.21.15.88:0/487214667 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fdf4005b880 con 0x7fdf4c0ffd60 2024-02-19T19:31:16.860 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:31:16.860 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:31:16.860 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:31:16.860 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:31:16.860 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:31:16.860 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:31:16.860 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:31:16.860 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:31:16.860 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:31:16.860 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:31:16.860 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:31:16.861 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:31:16.861 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:31:16.861 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:31:16.861 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:31:16.861 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:31:16.861 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:31:16.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.858+0000 7fdf516e2700 1 -- 172.21.15.88:0/487214667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3406d8a0 msgr2=0x7fdf3406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.858+0000 7fdf516e2700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3406d8a0 0x7fdf3406fd50 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c005950 tx=0x7fdf3c0058e0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.858+0000 7fdf516e2700 1 -- 172.21.15.88:0/487214667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c0ffd60 msgr2=0x7fdf4c1963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.858+0000 7fdf516e2700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c0ffd60 0x7fdf4c1963c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fdf400092e0 tx=0x7fdf4000be20 comp rx=0 tx=0).stop 2024-02-19T19:31:16.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.858+0000 7fdf516e2700 1 -- 172.21.15.88:0/487214667 shutdown_connections 2024-02-19T19:31:16.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.858+0000 7fdf516e2700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3406d8a0 0x7fdf3406fd50 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.858+0000 7fdf516e2700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf4c0ff3c0 0x7fdf4c195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.858+0000 7fdf516e2700 1 --2- 172.21.15.88:0/487214667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c0ffd60 0x7fdf4c1963c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.858+0000 7fdf516e2700 1 -- 172.21.15.88:0/487214667 >> 172.21.15.88:0/487214667 conn(0x7fdf4c0fac80 msgr2=0x7fdf4c068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:16.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.858+0000 7fdf516e2700 1 -- 172.21.15.88:0/487214667 shutdown_connections 2024-02-19T19:31:16.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.858+0000 7fdf516e2700 1 -- 172.21.15.88:0/487214667 wait complete. 2024-02-19T19:31:16.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.961+0000 7f68029de700 1 -- 172.21.15.88:0/893867958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc1012f0 msgr2=0x7f67fc1016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.961+0000 7f68029de700 1 --2- 172.21.15.88:0/893867958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc1012f0 0x7f67fc1016c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009910 tx=0x7f67ec009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:16.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.961+0000 7f68029de700 1 -- 172.21.15.88:0/893867958 shutdown_connections 2024-02-19T19:31:16.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.961+0000 7f68029de700 1 --2- 172.21.15.88:0/893867958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67fc101c00 0x7f67fc10a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.961+0000 7f68029de700 1 --2- 172.21.15.88:0/893867958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc1012f0 0x7f67fc1016c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.961+0000 7f68029de700 1 -- 172.21.15.88:0/893867958 >> 172.21.15.88:0/893867958 conn(0x7f67fc0fac60 msgr2=0x7f67fc0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:16.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.961+0000 7f68029de700 1 -- 172.21.15.88:0/893867958 shutdown_connections 2024-02-19T19:31:16.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.961+0000 7f68029de700 1 -- 172.21.15.88:0/893867958 wait complete. 2024-02-19T19:31:16.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.962+0000 7f68029de700 1 Processor -- start 2024-02-19T19:31:16.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.962+0000 7f68029de700 1 -- start start 2024-02-19T19:31:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.962+0000 7f68029de700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67fc1012f0 0x7f67fc195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.962+0000 7f68029de700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc101c00 0x7f67fc196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.962+0000 7f68029de700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc196ae0 con 0x7f67fc101c00 2024-02-19T19:31:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.962+0000 7f68029de700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc19a820 con 0x7f67fc1012f0 2024-02-19T19:31:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.962+0000 7f67fb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc101c00 0x7f67fc196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.962+0000 7f67fb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc101c00 0x7f67fc196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53642/0 (socket says 172.21.15.88:53642) 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.962+0000 7f67fb7fe700 1 -- 172.21.15.88:0/3969308299 learned_addr learned my addr 172.21.15.88:0/3969308299 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.963+0000 7f67fbfff700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67fc1012f0 0x7f67fc195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.963+0000 7f67fb7fe700 1 -- 172.21.15.88:0/3969308299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67fc1012f0 msgr2=0x7f67fc195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.963+0000 7f67fb7fe700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67fc1012f0 0x7f67fc195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.963+0000 7f67fb7fe700 1 -- 172.21.15.88:0/3969308299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec0095f0 con 0x7f67fc101c00 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.963+0000 7f67fbfff700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67fc1012f0 0x7f67fc195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.963+0000 7f67fb7fe700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc101c00 0x7f67fc196400 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f67f000e920 tx=0x7f67f000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.963+0000 7f67f97fa700 1 -- 172.21.15.88:0/3969308299 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f0009790 con 0x7f67fc101c00 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.963+0000 7f68029de700 1 -- 172.21.15.88:0/3969308299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc19ab00 con 0x7f67fc101c00 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.963+0000 7f67f97fa700 1 -- 172.21.15.88:0/3969308299 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67f000cba0 con 0x7f67fc101c00 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.963+0000 7f67f97fa700 1 -- 172.21.15.88:0/3969308299 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67f0018650 con 0x7f67fc101c00 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.963+0000 7f68029de700 1 -- 172.21.15.88:0/3969308299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc19b050 con 0x7f67fc101c00 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.964+0000 7f68029de700 1 -- 172.21.15.88:0/3969308299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67fc107970 con 0x7f67fc101c00 2024-02-19T19:31:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.965+0000 7f67f97fa700 1 -- 172.21.15.88:0/3969308299 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f67f000cd10 con 0x7f67fc101c00 2024-02-19T19:31:16.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.965+0000 7f67f97fa700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67e406d8a0 0x7f67e406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:16.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.965+0000 7f67f97fa700 1 -- 172.21.15.88:0/3969308299 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f67f0014070 con 0x7f67fc101c00 2024-02-19T19:31:16.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.965+0000 7f67fbfff700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67e406d8a0 0x7f67e406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:16.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.966+0000 7f67fbfff700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67e406d8a0 0x7f67e406fd50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00b760 tx=0x7f67ec005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:16.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:16.970+0000 7f67f97fa700 1 -- 172.21.15.88:0/3969308299 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f67f005ad60 con 0x7f67fc101c00 2024-02-19T19:31:17.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.208+0000 7f68029de700 1 -- 172.21.15.88:0/3969308299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f67fc197270 con 0x7f67fc101c00 2024-02-19T19:31:17.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.208+0000 7f67f97fa700 1 -- 172.21.15.88:0/3969308299 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f67f005a8f0 con 0x7f67fc101c00 2024-02-19T19:31:17.212 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:31:17.212 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:31:17.212 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:31:17.212 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:31:17.212 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:31:17.212 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:31:17.213 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:31:17.213 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:31:17.213 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:31:17.213 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:31:17.213 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:31:17.213 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:31:17.213 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:31:17.213 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:31:17.213 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:31:17.214 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:31:17.214 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:31:17.214 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:31:17.214 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:31:17.214 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:31:17.214 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:31:17.214 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:31:17.214 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:31:17.214 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:31:17.215 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:31:17.215 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:31:17.215 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:31:17.215 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:31:17.215 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:31:17.215 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:31:17.215 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:31:17.215 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:31:17.215 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:31:17.215 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:31:17.216 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:31:17.216 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:31:17.216 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:31:17.216 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:31:17.216 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:31:17.216 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:31:17.216 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:31:17.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.211+0000 7f68029de700 1 -- 172.21.15.88:0/3969308299 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67e406d8a0 msgr2=0x7f67e406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:17.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.211+0000 7f68029de700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67e406d8a0 0x7f67e406fd50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00b760 tx=0x7f67ec005940 comp rx=0 tx=0).stop 2024-02-19T19:31:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.211+0000 7f68029de700 1 -- 172.21.15.88:0/3969308299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc101c00 msgr2=0x7f67fc196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.211+0000 7f68029de700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc101c00 0x7f67fc196400 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f67f000e920 tx=0x7f67f000ece0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.211+0000 7f68029de700 1 -- 172.21.15.88:0/3969308299 shutdown_connections 2024-02-19T19:31:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.211+0000 7f68029de700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f67e406d8a0 0x7f67e406fd50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.211+0000 7f68029de700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f67fc1012f0 0x7f67fc195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.211+0000 7f68029de700 1 --2- 172.21.15.88:0/3969308299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67fc101c00 0x7f67fc196400 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.211+0000 7f68029de700 1 -- 172.21.15.88:0/3969308299 >> 172.21.15.88:0/3969308299 conn(0x7f67fc0fac60 msgr2=0x7f67fc104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.211+0000 7f68029de700 1 -- 172.21.15.88:0/3969308299 shutdown_connections 2024-02-19T19:31:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.211+0000 7f68029de700 1 -- 172.21.15.88:0/3969308299 wait complete. 2024-02-19T19:31:17.219 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:31:17.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.316+0000 7f6ff773f700 1 -- 172.21.15.88:0/2852675732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0069140 msgr2=0x7f6ff0104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:17.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.316+0000 7f6ff773f700 1 --2- 172.21.15.88:0/2852675732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0069140 0x7f6ff0104eb0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009910 tx=0x7f6fe0009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:17.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.316+0000 7f6ff773f700 1 -- 172.21.15.88:0/2852675732 shutdown_connections 2024-02-19T19:31:17.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.316+0000 7f6ff773f700 1 --2- 172.21.15.88:0/2852675732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0069140 0x7f6ff0104eb0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.316+0000 7f6ff773f700 1 --2- 172.21.15.88:0/2852675732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff00687a0 0x7f6ff0068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.316+0000 7f6ff773f700 1 -- 172.21.15.88:0/2852675732 >> 172.21.15.88:0/2852675732 conn(0x7f6ff0075130 msgr2=0x7f6ff0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:17.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.316+0000 7f6ff773f700 1 -- 172.21.15.88:0/2852675732 shutdown_connections 2024-02-19T19:31:17.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.316+0000 7f6ff773f700 1 -- 172.21.15.88:0/2852675732 wait complete. 2024-02-19T19:31:17.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.316+0000 7f6ff773f700 1 Processor -- start 2024-02-19T19:31:17.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.316+0000 7f6ff773f700 1 -- start start 2024-02-19T19:31:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff773f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff00687a0 0x7f6ff0195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:17.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff773f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0069140 0x7f6ff01963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:17.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff773f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff0196a80 con 0x7f6ff0069140 2024-02-19T19:31:17.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff773f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ff019a810 con 0x7f6ff00687a0 2024-02-19T19:31:17.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff4cda700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0069140 0x7f6ff01963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:17.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff4cda700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0069140 0x7f6ff01963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53666/0 (socket says 172.21.15.88:53666) 2024-02-19T19:31:17.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff4cda700 1 -- 172.21.15.88:0/1086899686 learned_addr learned my addr 172.21.15.88:0/1086899686 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:17.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff54db700 1 --2- 172.21.15.88:0/1086899686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff00687a0 0x7f6ff0195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:17.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff4cda700 1 -- 172.21.15.88:0/1086899686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff00687a0 msgr2=0x7f6ff0195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:17.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff4cda700 1 --2- 172.21.15.88:0/1086899686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff00687a0 0x7f6ff0195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff4cda700 1 -- 172.21.15.88:0/1086899686 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe00095f0 con 0x7f6ff0069140 2024-02-19T19:31:17.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff4cda700 1 --2- 172.21.15.88:0/1086899686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0069140 0x7f6ff01963a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6fe00092e0 tx=0x7f6fe000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:17.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6fe67fc700 1 -- 172.21.15.88:0/1086899686 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fe001d070 con 0x7f6ff0069140 2024-02-19T19:31:17.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff773f700 1 -- 172.21.15.88:0/1086899686 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff019aa90 con 0x7f6ff0069140 2024-02-19T19:31:17.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6fe67fc700 1 -- 172.21.15.88:0/1086899686 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fe0005dc0 con 0x7f6ff0069140 2024-02-19T19:31:17.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6fe67fc700 1 -- 172.21.15.88:0/1086899686 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fe0017400 con 0x7f6ff0069140 2024-02-19T19:31:17.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.317+0000 7f6ff773f700 1 -- 172.21.15.88:0/1086899686 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ff019af80 con 0x7f6ff0069140 2024-02-19T19:31:17.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.318+0000 7f6ff773f700 1 -- 172.21.15.88:0/1086899686 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ff0108860 con 0x7f6ff0069140 2024-02-19T19:31:17.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.319+0000 7f6fe67fc700 1 -- 172.21.15.88:0/1086899686 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6fe000f460 con 0x7f6ff0069140 2024-02-19T19:31:17.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.319+0000 7f6fe67fc700 1 --2- 172.21.15.88:0/1086899686 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fdc06d8a0 0x7f6fdc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:17.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.319+0000 7f6fe67fc700 1 -- 172.21.15.88:0/1086899686 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6fe008ea10 con 0x7f6ff0069140 2024-02-19T19:31:17.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.319+0000 7f6ff54db700 1 --2- 172.21.15.88:0/1086899686 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fdc06d8a0 0x7f6fdc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:17.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.320+0000 7f6ff54db700 1 --2- 172.21.15.88:0/1086899686 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fdc06d8a0 0x7f6fdc06fd50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6fec005950 tx=0x7f6fec0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:17.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.324+0000 7f6fe67fc700 1 -- 172.21.15.88:0/1086899686 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6fe00581c0 con 0x7f6ff0069140 2024-02-19T19:31:17.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.537+0000 7f6ff773f700 1 -- 172.21.15.88:0/1086899686 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ff01972d0 con 0x7f6fdc06d8a0 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.539+0000 7f6fe67fc700 1 -- 172.21.15.88:0/1086899686 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6ff01972d0 con 0x7f6fdc06d8a0 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:31:17.542 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:31:17.543 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:31:17.543 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:31:17.543 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:31:17.543 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:31:17.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.541+0000 7f6ff773f700 1 -- 172.21.15.88:0/1086899686 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fdc06d8a0 msgr2=0x7f6fdc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:17.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.541+0000 7f6ff773f700 1 --2- 172.21.15.88:0/1086899686 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fdc06d8a0 0x7f6fdc06fd50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6fec005950 tx=0x7f6fec0058e0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.541+0000 7f6ff773f700 1 -- 172.21.15.88:0/1086899686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0069140 msgr2=0x7f6ff01963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:17.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.541+0000 7f6ff773f700 1 --2- 172.21.15.88:0/1086899686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0069140 0x7f6ff01963a0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f6fe00092e0 tx=0x7f6fe000be20 comp rx=0 tx=0).stop 2024-02-19T19:31:17.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.541+0000 7f6ff773f700 1 -- 172.21.15.88:0/1086899686 shutdown_connections 2024-02-19T19:31:17.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.541+0000 7f6ff773f700 1 --2- 172.21.15.88:0/1086899686 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6fdc06d8a0 0x7f6fdc06fd50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.541+0000 7f6ff773f700 1 --2- 172.21.15.88:0/1086899686 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ff00687a0 0x7f6ff0195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.541+0000 7f6ff773f700 1 --2- 172.21.15.88:0/1086899686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0069140 0x7f6ff01963a0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.541+0000 7f6ff773f700 1 -- 172.21.15.88:0/1086899686 >> 172.21.15.88:0/1086899686 conn(0x7f6ff0075130 msgr2=0x7f6ff00ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:17.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.541+0000 7f6ff773f700 1 -- 172.21.15.88:0/1086899686 shutdown_connections 2024-02-19T19:31:17.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.541+0000 7f6ff773f700 1 -- 172.21.15.88:0/1086899686 wait complete. 2024-02-19T19:31:17.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.644+0000 7f2f518ff700 1 -- 172.21.15.88:0/1375194397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f4c1004e0 msgr2=0x7f2f4c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:17.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.644+0000 7f2f518ff700 1 --2- 172.21.15.88:0/1375194397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f4c1004e0 0x7f2f4c1008b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009910 tx=0x7f2f3c009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:17.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.644+0000 7f2f518ff700 1 -- 172.21.15.88:0/1375194397 shutdown_connections 2024-02-19T19:31:17.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.644+0000 7f2f518ff700 1 --2- 172.21.15.88:0/1375194397 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c100e80 0x7f2f4c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.644+0000 7f2f518ff700 1 --2- 172.21.15.88:0/1375194397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f4c1004e0 0x7f2f4c1008b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.644+0000 7f2f518ff700 1 -- 172.21.15.88:0/1375194397 >> 172.21.15.88:0/1375194397 conn(0x7f2f4c0fbda0 msgr2=0x7f2f4c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:17.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.644+0000 7f2f518ff700 1 -- 172.21.15.88:0/1375194397 shutdown_connections 2024-02-19T19:31:17.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.644+0000 7f2f518ff700 1 -- 172.21.15.88:0/1375194397 wait complete. 2024-02-19T19:31:17.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.645+0000 7f2f518ff700 1 Processor -- start 2024-02-19T19:31:17.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.645+0000 7f2f518ff700 1 -- start start 2024-02-19T19:31:17.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.645+0000 7f2f518ff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f4c1004e0 0x7f2f4c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:17.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.645+0000 7f2f518ff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c100e80 0x7f2f4c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.645+0000 7f2f518ff700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c196b10 con 0x7f2f4c1004e0 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.645+0000 7f2f518ff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c19a8a0 con 0x7f2f4c100e80 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.645+0000 7f2f4affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f4c1004e0 0x7f2f4c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.645+0000 7f2f4affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f4c1004e0 0x7f2f4c195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53698/0 (socket says 172.21.15.88:53698) 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.645+0000 7f2f4affd700 1 -- 172.21.15.88:0/245523732 learned_addr learned my addr 172.21.15.88:0/245523732 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.646+0000 7f2f4a7fc700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c100e80 0x7f2f4c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.646+0000 7f2f4affd700 1 -- 172.21.15.88:0/245523732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c100e80 msgr2=0x7f2f4c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.646+0000 7f2f4affd700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c100e80 0x7f2f4c196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.646+0000 7f2f4affd700 1 -- 172.21.15.88:0/245523732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0095f0 con 0x7f2f4c1004e0 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.646+0000 7f2f4a7fc700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c100e80 0x7f2f4c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.646+0000 7f2f4affd700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f4c1004e0 0x7f2f4c195ef0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c006010 tx=0x7f2f3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.646+0000 7f2f508fd700 1 -- 172.21.15.88:0/245523732 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f3c01d070 con 0x7f2f4c1004e0 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.646+0000 7f2f508fd700 1 -- 172.21.15.88:0/245523732 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f3c00f460 con 0x7f2f4c1004e0 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.646+0000 7f2f518ff700 1 -- 172.21.15.88:0/245523732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4c19ab20 con 0x7f2f4c1004e0 2024-02-19T19:31:17.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.646+0000 7f2f508fd700 1 -- 172.21.15.88:0/245523732 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f3c017630 con 0x7f2f4c1004e0 2024-02-19T19:31:17.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.646+0000 7f2f518ff700 1 -- 172.21.15.88:0/245523732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c19b010 con 0x7f2f4c1004e0 2024-02-19T19:31:17.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.647+0000 7f2f518ff700 1 -- 172.21.15.88:0/245523732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4c1087f0 con 0x7f2f4c1004e0 2024-02-19T19:31:17.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.648+0000 7f2f508fd700 1 -- 172.21.15.88:0/245523732 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2f3c00f5d0 con 0x7f2f4c1004e0 2024-02-19T19:31:17.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.648+0000 7f2f508fd700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f3406d8a0 0x7f2f3406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:17.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.648+0000 7f2f508fd700 1 -- 172.21.15.88:0/245523732 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2f3c08e9f0 con 0x7f2f4c1004e0 2024-02-19T19:31:17.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.648+0000 7f2f4a7fc700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f3406d8a0 0x7f2f3406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:17.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.648+0000 7f2f4a7fc700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f3406d8a0 0x7f2f3406fd50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2f40005fd0 tx=0x7f2f40005ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:17.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.653+0000 7f2f508fd700 1 -- 172.21.15.88:0/245523732 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f3c090e60 con 0x7f2f4c1004e0 2024-02-19T19:31:17.833 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:17 smithi088 ceph-mon[88643]: from='client.35510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:17.833 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:17 smithi088 ceph-mon[88643]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:17.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:17 smithi088 ceph-mon[88643]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:17.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:17 smithi088 ceph-mon[88643]: pgmap v832: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:17.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:17 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/487214667' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:31:17.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:17 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3969308299' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:31:17.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.943+0000 7f2f518ff700 1 -- 172.21.15.88:0/245523732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2f4c1081d0 con 0x7f2f4c1004e0 2024-02-19T19:31:17.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.943+0000 7f2f508fd700 1 -- 172.21.15.88:0/245523732 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f2f3c0581a0 con 0x7f2f4c1004e0 2024-02-19T19:31:17.946 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:31:17.946 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:31:17.946 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:31:17.947 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:31:17.948 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:31:17.948 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:31:17.948 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:31:17.948 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:31:17.948 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:31:17.948 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:31:17.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.945+0000 7f2f518ff700 1 -- 172.21.15.88:0/245523732 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f3406d8a0 msgr2=0x7f2f3406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:17.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.945+0000 7f2f518ff700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f3406d8a0 0x7f2f3406fd50 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f2f40005fd0 tx=0x7f2f40005ee0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.945+0000 7f2f518ff700 1 -- 172.21.15.88:0/245523732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f4c1004e0 msgr2=0x7f2f4c195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:17.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.945+0000 7f2f518ff700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f4c1004e0 0x7f2f4c195ef0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c006010 tx=0x7f2f3c00be20 comp rx=0 tx=0).stop 2024-02-19T19:31:17.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.945+0000 7f2f518ff700 1 -- 172.21.15.88:0/245523732 shutdown_connections 2024-02-19T19:31:17.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.945+0000 7f2f518ff700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2f3406d8a0 0x7f2f3406fd50 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.945+0000 7f2f518ff700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f4c1004e0 0x7f2f4c195ef0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.945+0000 7f2f518ff700 1 --2- 172.21.15.88:0/245523732 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2f4c100e80 0x7f2f4c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:17.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.945+0000 7f2f518ff700 1 -- 172.21.15.88:0/245523732 >> 172.21.15.88:0/245523732 conn(0x7f2f4c0fbda0 msgr2=0x7f2f4c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:17.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.945+0000 7f2f518ff700 1 -- 172.21.15.88:0/245523732 shutdown_connections 2024-02-19T19:31:17.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:17.946+0000 7f2f518ff700 1 -- 172.21.15.88:0/245523732 wait complete. 2024-02-19T19:31:17.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:17 smithi132 ceph-mon[70676]: from='client.35510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:17.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:17 smithi132 ceph-mon[70676]: from='client.35514 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:17.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:17 smithi132 ceph-mon[70676]: from='client.35518 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:17.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:17 smithi132 ceph-mon[70676]: pgmap v832: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:17.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:17 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/487214667' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:31:17.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:17 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3969308299' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:31:18.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:18 smithi132 ceph-mon[70676]: from='client.35530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:18.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:18 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/245523732' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:31:19.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:18 smithi088 ceph-mon[88643]: from='client.35530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:19.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:18 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/245523732' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:31:19.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:19 smithi132 ceph-mon[70676]: pgmap v833: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:20.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:19 smithi088 ceph-mon[88643]: pgmap v833: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:21.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:21 smithi132 ceph-mon[70676]: pgmap v834: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:22.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:21 smithi088 ceph-mon[88643]: pgmap v834: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:23.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:22 smithi088 ceph-mon[88643]: pgmap v835: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:23.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:22 smithi132 ceph-mon[70676]: pgmap v835: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:25.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:25 smithi132 ceph-mon[70676]: pgmap v836: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:26.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:25 smithi088 ceph-mon[88643]: pgmap v836: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:27.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:27 smithi132 ceph-mon[70676]: pgmap v837: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:28.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:27 smithi088 ceph-mon[88643]: pgmap v837: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:28.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:31:29.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:31:29.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:29 smithi132 ceph-mon[70676]: pgmap v838: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:30.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:29 smithi088 ceph-mon[88643]: pgmap v838: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:31.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:31 smithi132 ceph-mon[70676]: pgmap v839: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:32.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:31 smithi088 ceph-mon[88643]: pgmap v839: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:33.958 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:33 smithi132 ceph-mon[70676]: pgmap v840: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:34.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:33 smithi088 ceph-mon[88643]: pgmap v840: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:35.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:34 smithi088 ceph-mon[88643]: pgmap v841: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:35.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:34 smithi132 ceph-mon[70676]: pgmap v841: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:37.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:37 smithi132 ceph-mon[70676]: pgmap v842: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:38.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:37 smithi088 ceph-mon[88643]: pgmap v842: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:39.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:39 smithi132 ceph-mon[70676]: pgmap v843: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:40.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:39 smithi088 ceph-mon[88643]: pgmap v843: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:41.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:41 smithi132 ceph-mon[70676]: pgmap v844: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:42.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:41 smithi088 ceph-mon[88643]: pgmap v844: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:43.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:43 smithi132 ceph-mon[70676]: pgmap v845: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:43.986 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:31:44.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:43 smithi088 ceph-mon[88643]: pgmap v845: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:44.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:31:45.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:45 smithi132 ceph-mon[70676]: pgmap v846: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:46.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:45 smithi088 ceph-mon[88643]: pgmap v846: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:47.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:47 smithi132 ceph-mon[70676]: pgmap v847: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:48.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.062+0000 7f22cb9f2700 1 -- 172.21.15.88:0/1208075294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22c4100d70 msgr2=0x7f22c41094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.062+0000 7f22cb9f2700 1 --2- 172.21.15.88:0/1208075294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22c4100d70 0x7f22c41094d0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009940 tx=0x7f22b4009c50 comp rx=0 tx=0).stop 2024-02-19T19:31:48.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.062+0000 7f22cb9f2700 1 -- 172.21.15.88:0/1208075294 shutdown_connections 2024-02-19T19:31:48.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.062+0000 7f22cb9f2700 1 --2- 172.21.15.88:0/1208075294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22c4100d70 0x7f22c41094d0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.062+0000 7f22cb9f2700 1 --2- 172.21.15.88:0/1208075294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22c4100460 0x7f22c4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.063+0000 7f22cb9f2700 1 -- 172.21.15.88:0/1208075294 >> 172.21.15.88:0/1208075294 conn(0x7f22c40748b0 msgr2=0x7f22c4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:48.067 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:47 smithi088 ceph-mon[88643]: pgmap v847: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:48.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.063+0000 7f22cb9f2700 1 -- 172.21.15.88:0/1208075294 shutdown_connections 2024-02-19T19:31:48.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.063+0000 7f22cb9f2700 1 -- 172.21.15.88:0/1208075294 wait complete. 2024-02-19T19:31:48.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.063+0000 7f22cb9f2700 1 Processor -- start 2024-02-19T19:31:48.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.063+0000 7f22cb9f2700 1 -- start start 2024-02-19T19:31:48.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.063+0000 7f22cb9f2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22c4100460 0x7f22c4196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:48.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.063+0000 7f22cb9f2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22c4100d70 0x7f22c4197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:48.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22cb9f2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c4197750 con 0x7f22c4100d70 2024-02-19T19:31:48.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22cb9f2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22c4197890 con 0x7f22c4100460 2024-02-19T19:31:48.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22c8f8d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22c4100d70 0x7f22c4197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:48.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22c8f8d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22c4100d70 0x7f22c4197070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44208/0 (socket says 172.21.15.88:44208) 2024-02-19T19:31:48.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22c8f8d700 1 -- 172.21.15.88:0/531542689 learned_addr learned my addr 172.21.15.88:0/531542689 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:48.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22c978e700 1 --2- 172.21.15.88:0/531542689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22c4100460 0x7f22c4196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:48.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22c8f8d700 1 -- 172.21.15.88:0/531542689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22c4100460 msgr2=0x7f22c4196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22c8f8d700 1 --2- 172.21.15.88:0/531542689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22c4100460 0x7f22c4196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22c8f8d700 1 -- 172.21.15.88:0/531542689 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40095f0 con 0x7f22c4100d70 2024-02-19T19:31:48.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22c8f8d700 1 --2- 172.21.15.88:0/531542689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22c4100d70 0x7f22c4197070 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f22b400b580 tx=0x7f22b40056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:48.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22ba7fc700 1 -- 172.21.15.88:0/531542689 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22b401d070 con 0x7f22c4100d70 2024-02-19T19:31:48.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22ba7fc700 1 -- 172.21.15.88:0/531542689 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22b4005dc0 con 0x7f22c4100d70 2024-02-19T19:31:48.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22cb9f2700 1 -- 172.21.15.88:0/531542689 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22c419fba0 con 0x7f22c4100d70 2024-02-19T19:31:48.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22ba7fc700 1 -- 172.21.15.88:0/531542689 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22b4021410 con 0x7f22c4100d70 2024-02-19T19:31:48.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.064+0000 7f22cb9f2700 1 -- 172.21.15.88:0/531542689 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22c41a00c0 con 0x7f22c4100d70 2024-02-19T19:31:48.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.065+0000 7f22cb9f2700 1 -- 172.21.15.88:0/531542689 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22c4106bd0 con 0x7f22c4100d70 2024-02-19T19:31:48.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.066+0000 7f22ba7fc700 1 -- 172.21.15.88:0/531542689 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f22b400f7b0 con 0x7f22c4100d70 2024-02-19T19:31:48.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.066+0000 7f22ba7fc700 1 --2- 172.21.15.88:0/531542689 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22b006d8a0 0x7f22b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:48.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.066+0000 7f22ba7fc700 1 -- 172.21.15.88:0/531542689 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f22b408e660 con 0x7f22c4100d70 2024-02-19T19:31:48.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.066+0000 7f22c978e700 1 --2- 172.21.15.88:0/531542689 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22b006d8a0 0x7f22b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:48.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.067+0000 7f22c978e700 1 --2- 172.21.15.88:0/531542689 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22b006d8a0 0x7f22b006fd50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f22c0005950 tx=0x7f22c00058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:48.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.071+0000 7f22ba7fc700 1 -- 172.21.15.88:0/531542689 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22b4057e10 con 0x7f22c4100d70 2024-02-19T19:31:48.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.286+0000 7f22cb9f2700 1 -- 172.21.15.88:0/531542689 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22c4061190 con 0x7f22b006d8a0 2024-02-19T19:31:48.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.287+0000 7f22ba7fc700 1 -- 172.21.15.88:0/531542689 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f22c4061190 con 0x7f22b006d8a0 2024-02-19T19:31:48.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.289+0000 7f22cb9f2700 1 -- 172.21.15.88:0/531542689 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22b006d8a0 msgr2=0x7f22b006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.289+0000 7f22cb9f2700 1 --2- 172.21.15.88:0/531542689 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22b006d8a0 0x7f22b006fd50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f22c0005950 tx=0x7f22c00058e0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.289+0000 7f22cb9f2700 1 -- 172.21.15.88:0/531542689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22c4100d70 msgr2=0x7f22c4197070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.289+0000 7f22cb9f2700 1 --2- 172.21.15.88:0/531542689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22c4100d70 0x7f22c4197070 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f22b400b580 tx=0x7f22b40056a0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.289+0000 7f22cb9f2700 1 -- 172.21.15.88:0/531542689 shutdown_connections 2024-02-19T19:31:48.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.289+0000 7f22cb9f2700 1 --2- 172.21.15.88:0/531542689 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22b006d8a0 0x7f22b006fd50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.289+0000 7f22cb9f2700 1 --2- 172.21.15.88:0/531542689 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22c4100460 0x7f22c4196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.289+0000 7f22cb9f2700 1 --2- 172.21.15.88:0/531542689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22c4100d70 0x7f22c4197070 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.289+0000 7f22cb9f2700 1 -- 172.21.15.88:0/531542689 >> 172.21.15.88:0/531542689 conn(0x7f22c40748b0 msgr2=0x7f22c4103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:48.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.290+0000 7f22cb9f2700 1 -- 172.21.15.88:0/531542689 shutdown_connections 2024-02-19T19:31:48.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.290+0000 7f22cb9f2700 1 -- 172.21.15.88:0/531542689 wait complete. 2024-02-19T19:31:48.303 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:31:48.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.394+0000 7f8fa3519700 1 -- 172.21.15.88:0/2248472041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f9c100e40 msgr2=0x7f8f9c104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.394+0000 7f8fa3519700 1 --2- 172.21.15.88:0/2248472041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f9c100e40 0x7f8f9c104e10 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009910 tx=0x7f8f90009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:48.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.394+0000 7f8fa3519700 1 -- 172.21.15.88:0/2248472041 shutdown_connections 2024-02-19T19:31:48.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.394+0000 7f8fa3519700 1 --2- 172.21.15.88:0/2248472041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f9c100e40 0x7f8f9c104e10 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.394+0000 7f8fa3519700 1 --2- 172.21.15.88:0/2248472041 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f9c1004a0 0x7f8f9c100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.394+0000 7f8fa3519700 1 -- 172.21.15.88:0/2248472041 >> 172.21.15.88:0/2248472041 conn(0x7f8f9c0fbd40 msgr2=0x7f8f9c0fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:48.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.394+0000 7f8fa3519700 1 -- 172.21.15.88:0/2248472041 shutdown_connections 2024-02-19T19:31:48.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.394+0000 7f8fa3519700 1 -- 172.21.15.88:0/2248472041 wait complete. 2024-02-19T19:31:48.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa3519700 1 Processor -- start 2024-02-19T19:31:48.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa3519700 1 -- start start 2024-02-19T19:31:48.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa3519700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f9c1004a0 0x7f8f9c195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa3519700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f9c100e40 0x7f8f9c1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa3519700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c196ad0 con 0x7f8f9c100e40 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa3519700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c19a860 con 0x7f8f9c1004a0 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa0ab4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f9c100e40 0x7f8f9c1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa0ab4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f9c100e40 0x7f8f9c1963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44226/0 (socket says 172.21.15.88:44226) 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa0ab4700 1 -- 172.21.15.88:0/3692626039 learned_addr learned my addr 172.21.15.88:0/3692626039 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa0ab4700 1 -- 172.21.15.88:0/3692626039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f9c1004a0 msgr2=0x7f8f9c195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa0ab4700 1 --2- 172.21.15.88:0/3692626039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f9c1004a0 0x7f8f9c195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.395+0000 7f8fa0ab4700 1 -- 172.21.15.88:0/3692626039 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f900095f0 con 0x7f8f9c100e40 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.396+0000 7f8fa0ab4700 1 --2- 172.21.15.88:0/3692626039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f9c100e40 0x7f8f9c1963f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8f900092e0 tx=0x7f8f9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.396+0000 7f8f9a7fc700 1 -- 172.21.15.88:0/3692626039 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f9001d070 con 0x7f8f9c100e40 2024-02-19T19:31:48.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.396+0000 7f8f9a7fc700 1 -- 172.21.15.88:0/3692626039 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f90005dc0 con 0x7f8f9c100e40 2024-02-19T19:31:48.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.396+0000 7f8fa3519700 1 -- 172.21.15.88:0/3692626039 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9c19aae0 con 0x7f8f9c100e40 2024-02-19T19:31:48.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.396+0000 7f8f9a7fc700 1 -- 172.21.15.88:0/3692626039 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f90017400 con 0x7f8f9c100e40 2024-02-19T19:31:48.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.396+0000 7f8fa3519700 1 -- 172.21.15.88:0/3692626039 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9c19afd0 con 0x7f8f9c100e40 2024-02-19T19:31:48.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.397+0000 7f8fa3519700 1 -- 172.21.15.88:0/3692626039 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9c108810 con 0x7f8f9c100e40 2024-02-19T19:31:48.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.397+0000 7f8f9a7fc700 1 -- 172.21.15.88:0/3692626039 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8f9000f460 con 0x7f8f9c100e40 2024-02-19T19:31:48.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.397+0000 7f8f9a7fc700 1 --2- 172.21.15.88:0/3692626039 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8f8406d8a0 0x7f8f8406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:48.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.398+0000 7f8f9a7fc700 1 -- 172.21.15.88:0/3692626039 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8f9008ea20 con 0x7f8f9c100e40 2024-02-19T19:31:48.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.398+0000 7f8fa12b5700 1 --2- 172.21.15.88:0/3692626039 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8f8406d8a0 0x7f8f8406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:48.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.398+0000 7f8fa12b5700 1 --2- 172.21.15.88:0/3692626039 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8f8406d8a0 0x7f8f8406fd50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8f940060b0 tx=0x7f8f94006040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:48.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.402+0000 7f8f9a7fc700 1 -- 172.21.15.88:0/3692626039 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8f900581d0 con 0x7f8f9c100e40 2024-02-19T19:31:48.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.616+0000 7f8fa3519700 1 -- 172.21.15.88:0/3692626039 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f9c19b1f0 con 0x7f8f8406d8a0 2024-02-19T19:31:48.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.618+0000 7f8f9a7fc700 1 -- 172.21.15.88:0/3692626039 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8f9c19b1f0 con 0x7f8f8406d8a0 2024-02-19T19:31:48.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.620+0000 7f8fa3519700 1 -- 172.21.15.88:0/3692626039 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8f8406d8a0 msgr2=0x7f8f8406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.620+0000 7f8fa3519700 1 --2- 172.21.15.88:0/3692626039 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8f8406d8a0 0x7f8f8406fd50 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8f940060b0 tx=0x7f8f94006040 comp rx=0 tx=0).stop 2024-02-19T19:31:48.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.620+0000 7f8fa3519700 1 -- 172.21.15.88:0/3692626039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f9c100e40 msgr2=0x7f8f9c1963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.620+0000 7f8fa3519700 1 --2- 172.21.15.88:0/3692626039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f9c100e40 0x7f8f9c1963f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8f900092e0 tx=0x7f8f9000be20 comp rx=0 tx=0).stop 2024-02-19T19:31:48.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.620+0000 7f8fa3519700 1 -- 172.21.15.88:0/3692626039 shutdown_connections 2024-02-19T19:31:48.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.620+0000 7f8fa3519700 1 --2- 172.21.15.88:0/3692626039 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8f8406d8a0 0x7f8f8406fd50 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.620+0000 7f8fa3519700 1 --2- 172.21.15.88:0/3692626039 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8f9c1004a0 0x7f8f9c195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.620+0000 7f8fa3519700 1 --2- 172.21.15.88:0/3692626039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f9c100e40 0x7f8f9c1963f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.620+0000 7f8fa3519700 1 -- 172.21.15.88:0/3692626039 >> 172.21.15.88:0/3692626039 conn(0x7f8f9c0fbd40 msgr2=0x7f8f9c0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:48.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.620+0000 7f8fa3519700 1 -- 172.21.15.88:0/3692626039 shutdown_connections 2024-02-19T19:31:48.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.620+0000 7f8fa3519700 1 -- 172.21.15.88:0/3692626039 wait complete. 2024-02-19T19:31:48.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.725+0000 7f65b970e700 1 -- 172.21.15.88:0/550817005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65b4104130 msgr2=0x7f65b4104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.725+0000 7f65b970e700 1 --2- 172.21.15.88:0/550817005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65b4104130 0x7f65b4104500 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f65a4009910 tx=0x7f65a4009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:48.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.725+0000 7f65b970e700 1 -- 172.21.15.88:0/550817005 shutdown_connections 2024-02-19T19:31:48.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.725+0000 7f65b970e700 1 --2- 172.21.15.88:0/550817005 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65b40ffbd0 0x7f65b4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.725+0000 7f65b970e700 1 --2- 172.21.15.88:0/550817005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65b4104130 0x7f65b4104500 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.725+0000 7f65b970e700 1 -- 172.21.15.88:0/550817005 >> 172.21.15.88:0/550817005 conn(0x7f65b4075130 msgr2=0x7f65b4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:48.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.725+0000 7f65b970e700 1 -- 172.21.15.88:0/550817005 shutdown_connections 2024-02-19T19:31:48.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.725+0000 7f65b970e700 1 -- 172.21.15.88:0/550817005 wait complete. 2024-02-19T19:31:48.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b970e700 1 Processor -- start 2024-02-19T19:31:48.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b970e700 1 -- start start 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b970e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65b40ffbd0 0x7f65b4196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b970e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65b4104130 0x7f65b41972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b970e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b41979d0 con 0x7f65b40ffbd0 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b970e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b4197b40 con 0x7f65b4104130 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65b40ffbd0 0x7f65b4196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65b40ffbd0 0x7f65b4196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44244/0 (socket says 172.21.15.88:44244) 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b2ffd700 1 -- 172.21.15.88:0/1670576721 learned_addr learned my addr 172.21.15.88:0/1670576721 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b2ffd700 1 -- 172.21.15.88:0/1670576721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65b4104130 msgr2=0x7f65b41972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b27fc700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65b4104130 0x7f65b41972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b2ffd700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65b4104130 0x7f65b41972f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b2ffd700 1 -- 172.21.15.88:0/1670576721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65a40095f0 con 0x7f65b40ffbd0 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.726+0000 7f65b27fc700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65b4104130 0x7f65b41972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:31:48.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.727+0000 7f65b2ffd700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65b40ffbd0 0x7f65b4196db0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f65a4006010 tx=0x7f65a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:48.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.727+0000 7f659bfff700 1 -- 172.21.15.88:0/1670576721 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65a401d070 con 0x7f65b40ffbd0 2024-02-19T19:31:48.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.727+0000 7f659bfff700 1 -- 172.21.15.88:0/1670576721 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65a4004fb0 con 0x7f65b40ffbd0 2024-02-19T19:31:48.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.727+0000 7f65b970e700 1 -- 172.21.15.88:0/1670576721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b419fe20 con 0x7f65b40ffbd0 2024-02-19T19:31:48.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.727+0000 7f659bfff700 1 -- 172.21.15.88:0/1670576721 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65a400f690 con 0x7f65b40ffbd0 2024-02-19T19:31:48.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.727+0000 7f65b970e700 1 -- 172.21.15.88:0/1670576721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b41a0290 con 0x7f65b40ffbd0 2024-02-19T19:31:48.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.728+0000 7f65b970e700 1 -- 172.21.15.88:0/1670576721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b41076a0 con 0x7f65b40ffbd0 2024-02-19T19:31:48.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.729+0000 7f659bfff700 1 -- 172.21.15.88:0/1670576721 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f65a4022470 con 0x7f65b40ffbd0 2024-02-19T19:31:48.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.729+0000 7f659bfff700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f659c06d8a0 0x7f659c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:48.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.729+0000 7f659bfff700 1 -- 172.21.15.88:0/1670576721 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f65a408ec10 con 0x7f65b40ffbd0 2024-02-19T19:31:48.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.729+0000 7f65b27fc700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f659c06d8a0 0x7f659c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:48.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.729+0000 7f65b27fc700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f659c06d8a0 0x7f659c06fd50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f65b4068df0 tx=0x7f65a8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:48.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.734+0000 7f659bfff700 1 -- 172.21.15.88:0/1670576721 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65a4058440 con 0x7f65b40ffbd0 2024-02-19T19:31:48.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.940+0000 7f65b970e700 1 -- 172.21.15.88:0/1670576721 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f65b4061190 con 0x7f659c06d8a0 2024-02-19T19:31:48.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.951+0000 7f659bfff700 1 -- 172.21.15.88:0/1670576721 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f65b4061190 con 0x7f659c06d8a0 2024-02-19T19:31:48.955 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:31:48.956 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (28m) 5m ago 39m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:31:48.956 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (39m) 5m ago 39m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:31:48.956 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (21m) 5m ago 37m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:31:48.956 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (25m) 5m ago 39m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:31:48.956 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (24m) 5m ago 37m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:31:48.956 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (28m) 5m ago 38m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:31:48.956 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (21m) 5m ago 33m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:31:48.956 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (20m) 5m ago 33m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:31:48.956 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (20m) 5m ago 33m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:31:48.956 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (5m) 5m ago 33m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (30m) 5m ago 41m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (29m) 5m ago 37m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (27m) 5m ago 41m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (26m) 5m ago 37m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (29m) 5m ago 39m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (29m) 5m ago 37m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (24m) 5m ago 36m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (23m) 5m ago 36m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (23m) 5m ago 35m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (22m) 5m ago 35m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (22m) 5m ago 35m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (21m) 5m ago 34m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:31:48.957 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (29m) 5m ago 38m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:31:48.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.953+0000 7f65b970e700 1 -- 172.21.15.88:0/1670576721 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f659c06d8a0 msgr2=0x7f659c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.953+0000 7f65b970e700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f659c06d8a0 0x7f659c06fd50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f65b4068df0 tx=0x7f65a8008040 comp rx=0 tx=0).stop 2024-02-19T19:31:48.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.953+0000 7f65b970e700 1 -- 172.21.15.88:0/1670576721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65b40ffbd0 msgr2=0x7f65b4196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:48.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.953+0000 7f65b970e700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65b40ffbd0 0x7f65b4196db0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f65a4006010 tx=0x7f65a400be20 comp rx=0 tx=0).stop 2024-02-19T19:31:48.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.953+0000 7f65b970e700 1 -- 172.21.15.88:0/1670576721 shutdown_connections 2024-02-19T19:31:48.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.953+0000 7f65b970e700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f659c06d8a0 0x7f659c06fd50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.953+0000 7f65b970e700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65b40ffbd0 0x7f65b4196db0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.953+0000 7f65b970e700 1 --2- 172.21.15.88:0/1670576721 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f65b4104130 0x7f65b41972f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:48.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.953+0000 7f65b970e700 1 -- 172.21.15.88:0/1670576721 >> 172.21.15.88:0/1670576721 conn(0x7f65b4075130 msgr2=0x7f65b40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:48.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.953+0000 7f65b970e700 1 -- 172.21.15.88:0/1670576721 shutdown_connections 2024-02-19T19:31:48.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:48.953+0000 7f65b970e700 1 -- 172.21.15.88:0/1670576721 wait complete. 2024-02-19T19:31:49.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.056+0000 7f396d712700 1 -- 172.21.15.88:0/3086673058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39681004c0 msgr2=0x7f3968100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:49.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.056+0000 7f396d712700 1 --2- 172.21.15.88:0/3086673058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39681004c0 0x7f3968100890 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f3958009910 tx=0x7f3958009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:49.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.056+0000 7f396d712700 1 -- 172.21.15.88:0/3086673058 shutdown_connections 2024-02-19T19:31:49.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.056+0000 7f396d712700 1 --2- 172.21.15.88:0/3086673058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3968100e60 0x7f3968104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.056+0000 7f396d712700 1 --2- 172.21.15.88:0/3086673058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39681004c0 0x7f3968100890 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.056+0000 7f396d712700 1 -- 172.21.15.88:0/3086673058 >> 172.21.15.88:0/3086673058 conn(0x7f39680fbd80 msgr2=0x7f39680fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:49.060 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:48 smithi088 ceph-mon[88643]: from='client.35538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:49.060 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:48 smithi088 ceph-mon[88643]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:49.060 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:48 smithi088 ceph-mon[88643]: pgmap v848: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.056+0000 7f396d712700 1 -- 172.21.15.88:0/3086673058 shutdown_connections 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.056+0000 7f396d712700 1 -- 172.21.15.88:0/3086673058 wait complete. 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.057+0000 7f396d712700 1 Processor -- start 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.057+0000 7f396d712700 1 -- start start 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.057+0000 7f396d712700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39681004c0 0x7f3968073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.057+0000 7f396d712700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3968100e60 0x7f3968073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.057+0000 7f396d712700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3968073ad0 con 0x7f39681004c0 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.057+0000 7f396d712700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3968073c10 con 0x7f3968100e60 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.057+0000 7f3966ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39681004c0 0x7f3968073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.057+0000 7f3966ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39681004c0 0x7f3968073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44268/0 (socket says 172.21.15.88:44268) 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.057+0000 7f3966ffd700 1 -- 172.21.15.88:0/454367229 learned_addr learned my addr 172.21.15.88:0/454367229 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:49.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.058+0000 7f39667fc700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3968100e60 0x7f3968073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:49.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.058+0000 7f3966ffd700 1 -- 172.21.15.88:0/454367229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3968100e60 msgr2=0x7f3968073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:49.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.058+0000 7f3966ffd700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3968100e60 0x7f3968073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.058+0000 7f3966ffd700 1 -- 172.21.15.88:0/454367229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39580095f0 con 0x7f39681004c0 2024-02-19T19:31:49.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.058+0000 7f39667fc700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3968100e60 0x7f3968073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-19T19:31:49.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.058+0000 7f3966ffd700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39681004c0 0x7f3968073050 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3958005f50 tx=0x7f395800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:49.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.058+0000 7f394ffff700 1 -- 172.21.15.88:0/454367229 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f395801d070 con 0x7f39681004c0 2024-02-19T19:31:49.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.058+0000 7f396d712700 1 -- 172.21.15.88:0/454367229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3968073e90 con 0x7f39681004c0 2024-02-19T19:31:49.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.058+0000 7f394ffff700 1 -- 172.21.15.88:0/454367229 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f395800b610 con 0x7f39681004c0 2024-02-19T19:31:49.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.058+0000 7f394ffff700 1 -- 172.21.15.88:0/454367229 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3958022470 con 0x7f39681004c0 2024-02-19T19:31:49.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.058+0000 7f396d712700 1 -- 172.21.15.88:0/454367229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39681a2430 con 0x7f39681004c0 2024-02-19T19:31:49.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.060+0000 7f396d712700 1 -- 172.21.15.88:0/454367229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39681087d0 con 0x7f39681004c0 2024-02-19T19:31:49.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.060+0000 7f394ffff700 1 -- 172.21.15.88:0/454367229 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3958003780 con 0x7f39681004c0 2024-02-19T19:31:49.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.060+0000 7f394ffff700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f395006d8a0 0x7f395006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:49.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.061+0000 7f394ffff700 1 -- 172.21.15.88:0/454367229 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f395808e920 con 0x7f39681004c0 2024-02-19T19:31:49.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.061+0000 7f39667fc700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f395006d8a0 0x7f395006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:49.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.061+0000 7f39667fc700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f395006d8a0 0x7f395006fd50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3968074840 tx=0x7f395c009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:49.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.065+0000 7f394ffff700 1 -- 172.21.15.88:0/454367229 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f39580580d0 con 0x7f39681004c0 2024-02-19T19:31:49.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:48 smithi132 ceph-mon[70676]: from='client.35538 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:49.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:48 smithi132 ceph-mon[70676]: from='client.35542 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:49.236 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:48 smithi132 ceph-mon[70676]: pgmap v848: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:49.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.343+0000 7f396d712700 1 -- 172.21.15.88:0/454367229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f396804f2a0 con 0x7f39681004c0 2024-02-19T19:31:49.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.344+0000 7f394ffff700 1 -- 172.21.15.88:0/454367229 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f395805b790 con 0x7f39681004c0 2024-02-19T19:31:49.347 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:31:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:31:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:31:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:31:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:31:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:31:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:31:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:31:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:31:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:31:49.347 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:31:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:31:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:31:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:31:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:31:49.348 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:31:49.348 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:31:49.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.346+0000 7f396d712700 1 -- 172.21.15.88:0/454367229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f395006d8a0 msgr2=0x7f395006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:49.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.346+0000 7f396d712700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f395006d8a0 0x7f395006fd50 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3968074840 tx=0x7f395c009260 comp rx=0 tx=0).stop 2024-02-19T19:31:49.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.346+0000 7f396d712700 1 -- 172.21.15.88:0/454367229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39681004c0 msgr2=0x7f3968073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:49.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.346+0000 7f396d712700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39681004c0 0x7f3968073050 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3958005f50 tx=0x7f395800f690 comp rx=0 tx=0).stop 2024-02-19T19:31:49.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.346+0000 7f396d712700 1 -- 172.21.15.88:0/454367229 shutdown_connections 2024-02-19T19:31:49.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.346+0000 7f396d712700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f395006d8a0 0x7f395006fd50 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.346+0000 7f396d712700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39681004c0 0x7f3968073050 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.346+0000 7f396d712700 1 --2- 172.21.15.88:0/454367229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3968100e60 0x7f3968073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.346+0000 7f396d712700 1 -- 172.21.15.88:0/454367229 >> 172.21.15.88:0/454367229 conn(0x7f39680fbd80 msgr2=0x7f39680fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:49.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.346+0000 7f396d712700 1 -- 172.21.15.88:0/454367229 shutdown_connections 2024-02-19T19:31:49.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.346+0000 7f396d712700 1 -- 172.21.15.88:0/454367229 wait complete. 2024-02-19T19:31:49.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.450+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1557081670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc70101d30 msgr2=0x7fdc7010a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:49.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.450+0000 7fdc74fb3700 1 --2- 172.21.15.88:0/1557081670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc70101d30 0x7fdc7010a230 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400b0a0 tx=0x7fdc6400b3b0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.450+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1557081670 shutdown_connections 2024-02-19T19:31:49.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.450+0000 7fdc74fb3700 1 --2- 172.21.15.88:0/1557081670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc70101d30 0x7fdc7010a230 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.450+0000 7fdc74fb3700 1 --2- 172.21.15.88:0/1557081670 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc70101420 0x7fdc701017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.450+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1557081670 >> 172.21.15.88:0/1557081670 conn(0x7fdc700fac60 msgr2=0x7fdc700fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:49.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.450+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1557081670 shutdown_connections 2024-02-19T19:31:49.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.450+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1557081670 wait complete. 2024-02-19T19:31:49.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc74fb3700 1 Processor -- start 2024-02-19T19:31:49.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc74fb3700 1 -- start start 2024-02-19T19:31:49.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc74fb3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc70101420 0x7fdc70195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:49.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc74fb3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc70101d30 0x7fdc70196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:49.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc74fb3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc70196a70 con 0x7fdc70101420 2024-02-19T19:31:49.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc74fb3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7019a800 con 0x7fdc70101d30 2024-02-19T19:31:49.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc6e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc70101420 0x7fdc70195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:49.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc6e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc70101420 0x7fdc70195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44286/0 (socket says 172.21.15.88:44286) 2024-02-19T19:31:49.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc6e59c700 1 -- 172.21.15.88:0/1358652476 learned_addr learned my addr 172.21.15.88:0/1358652476 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:49.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc6e59c700 1 -- 172.21.15.88:0/1358652476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc70101d30 msgr2=0x7fdc70196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:49.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc6e59c700 1 --2- 172.21.15.88:0/1358652476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc70101d30 0x7fdc70196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.451+0000 7fdc6e59c700 1 -- 172.21.15.88:0/1358652476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc64009d00 con 0x7fdc70101420 2024-02-19T19:31:49.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.452+0000 7fdc6e59c700 1 --2- 172.21.15.88:0/1358652476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc70101420 0x7fdc70195e50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800d8d0 tx=0x7fdc6800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:49.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.452+0000 7fdc5f7fe700 1 -- 172.21.15.88:0/1358652476 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc68009940 con 0x7fdc70101420 2024-02-19T19:31:49.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.452+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1358652476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc7019aae0 con 0x7fdc70101420 2024-02-19T19:31:49.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.452+0000 7fdc5f7fe700 1 -- 172.21.15.88:0/1358652476 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc68010460 con 0x7fdc70101420 2024-02-19T19:31:49.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.452+0000 7fdc5f7fe700 1 -- 172.21.15.88:0/1358652476 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc6800f5f0 con 0x7fdc70101420 2024-02-19T19:31:49.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.452+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1358652476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc7019b030 con 0x7fdc70101420 2024-02-19T19:31:49.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.453+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1358652476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc701078a0 con 0x7fdc70101420 2024-02-19T19:31:49.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.453+0000 7fdc5f7fe700 1 -- 172.21.15.88:0/1358652476 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdc680105d0 con 0x7fdc70101420 2024-02-19T19:31:49.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.454+0000 7fdc5f7fe700 1 --2- 172.21.15.88:0/1358652476 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdc5806d8a0 0x7fdc5806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:49.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.454+0000 7fdc5f7fe700 1 -- 172.21.15.88:0/1358652476 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdc6808d540 con 0x7fdc70101420 2024-02-19T19:31:49.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.454+0000 7fdc6dd9b700 1 --2- 172.21.15.88:0/1358652476 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdc5806d8a0 0x7fdc5806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:49.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.454+0000 7fdc6dd9b700 1 --2- 172.21.15.88:0/1358652476 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdc5806d8a0 0x7fdc5806fd50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009480 tx=0x7fdc64006000 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:49.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.459+0000 7fdc5f7fe700 1 -- 172.21.15.88:0/1358652476 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdc68056cf0 con 0x7fdc70101420 2024-02-19T19:31:49.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.700+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1358652476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdc70197280 con 0x7fdc70101420 2024-02-19T19:31:49.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.701+0000 7fdc5f7fe700 1 -- 172.21.15.88:0/1358652476 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fdc6805a3b0 con 0x7fdc70101420 2024-02-19T19:31:49.704 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:31:49.704 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:31:49.704 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:31:49.704 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:31:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:31:49.705 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:31:49.706 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:31:49.707 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:31:49.707 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:31:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:31:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:31:49.707 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:31:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:31:49.707 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:31:49.707 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:31:49.707 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:31:49.707 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:31:49.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.703+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1358652476 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdc5806d8a0 msgr2=0x7fdc5806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:49.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.703+0000 7fdc74fb3700 1 --2- 172.21.15.88:0/1358652476 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdc5806d8a0 0x7fdc5806fd50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fdc64009480 tx=0x7fdc64006000 comp rx=0 tx=0).stop 2024-02-19T19:31:49.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.703+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1358652476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc70101420 msgr2=0x7fdc70195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:49.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.703+0000 7fdc74fb3700 1 --2- 172.21.15.88:0/1358652476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc70101420 0x7fdc70195e50 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdc6800d8d0 tx=0x7fdc6800dc90 comp rx=0 tx=0).stop 2024-02-19T19:31:49.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.703+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1358652476 shutdown_connections 2024-02-19T19:31:49.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.703+0000 7fdc74fb3700 1 --2- 172.21.15.88:0/1358652476 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdc5806d8a0 0x7fdc5806fd50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.703+0000 7fdc74fb3700 1 --2- 172.21.15.88:0/1358652476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc70101420 0x7fdc70195e50 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.703+0000 7fdc74fb3700 1 --2- 172.21.15.88:0/1358652476 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdc70101d30 0x7fdc70196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.703+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1358652476 >> 172.21.15.88:0/1358652476 conn(0x7fdc700fac60 msgr2=0x7fdc701049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:49.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.703+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1358652476 shutdown_connections 2024-02-19T19:31:49.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.703+0000 7fdc74fb3700 1 -- 172.21.15.88:0/1358652476 wait complete. 2024-02-19T19:31:49.709 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:31:49.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.809+0000 7f858d539700 1 -- 172.21.15.88:0/3430061001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8588100e80 msgr2=0x7f8588104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:49.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.810+0000 7f858d539700 1 --2- 172.21.15.88:0/3430061001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8588100e80 0x7f8588104e50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f857c009910 tx=0x7f857c009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:49.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.810+0000 7f858d539700 1 -- 172.21.15.88:0/3430061001 shutdown_connections 2024-02-19T19:31:49.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.810+0000 7f858d539700 1 --2- 172.21.15.88:0/3430061001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8588100e80 0x7f8588104e50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.810+0000 7f858d539700 1 --2- 172.21.15.88:0/3430061001 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881004e0 0x7f85881008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.810+0000 7f858d539700 1 -- 172.21.15.88:0/3430061001 >> 172.21.15.88:0/3430061001 conn(0x7f85880fbda0 msgr2=0x7f85880fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:49.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.810+0000 7f858d539700 1 -- 172.21.15.88:0/3430061001 shutdown_connections 2024-02-19T19:31:49.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.810+0000 7f858d539700 1 -- 172.21.15.88:0/3430061001 wait complete. 2024-02-19T19:31:49.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.810+0000 7f858d539700 1 Processor -- start 2024-02-19T19:31:49.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.810+0000 7f858d539700 1 -- start start 2024-02-19T19:31:49.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f858d539700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881004e0 0x7f8588195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:49.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f858d539700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8588100e80 0x7f8588196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f858d539700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8588196b30 con 0x7f8588100e80 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f858d539700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858819a8c0 con 0x7f85881004e0 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f85867fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8588100e80 0x7f8588196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f85867fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8588100e80 0x7f8588196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44312/0 (socket says 172.21.15.88:44312) 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f85867fc700 1 -- 172.21.15.88:0/2787460727 learned_addr learned my addr 172.21.15.88:0/2787460727 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f8586ffd700 1 --2- 172.21.15.88:0/2787460727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881004e0 0x7f8588195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f85867fc700 1 -- 172.21.15.88:0/2787460727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881004e0 msgr2=0x7f8588195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f85867fc700 1 --2- 172.21.15.88:0/2787460727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881004e0 0x7f8588195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f85867fc700 1 -- 172.21.15.88:0/2787460727 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857c0095f0 con 0x7f8588100e80 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f85867fc700 1 --2- 172.21.15.88:0/2787460727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8588100e80 0x7f8588196450 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f857c0092e0 tx=0x7f857c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f856ffff700 1 -- 172.21.15.88:0/2787460727 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f857c01d070 con 0x7f8588100e80 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f856ffff700 1 -- 172.21.15.88:0/2787460727 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f857c00f460 con 0x7f8588100e80 2024-02-19T19:31:49.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f858d539700 1 -- 172.21.15.88:0/2787460727 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858819ab40 con 0x7f8588100e80 2024-02-19T19:31:49.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f856ffff700 1 -- 172.21.15.88:0/2787460727 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f857c0175d0 con 0x7f8588100e80 2024-02-19T19:31:49.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.811+0000 7f858d539700 1 -- 172.21.15.88:0/2787460727 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858819b030 con 0x7f8588100e80 2024-02-19T19:31:49.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.812+0000 7f858d539700 1 -- 172.21.15.88:0/2787460727 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8588108850 con 0x7f8588100e80 2024-02-19T19:31:49.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.813+0000 7f856ffff700 1 -- 172.21.15.88:0/2787460727 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f857c003a60 con 0x7f8588100e80 2024-02-19T19:31:49.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.813+0000 7f856ffff700 1 --2- 172.21.15.88:0/2787460727 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f857006d8a0 0x7f857006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:49.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.813+0000 7f856ffff700 1 -- 172.21.15.88:0/2787460727 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f857c08e9d0 con 0x7f8588100e80 2024-02-19T19:31:49.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.813+0000 7f8586ffd700 1 --2- 172.21.15.88:0/2787460727 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f857006d8a0 0x7f857006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:49.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.814+0000 7f8586ffd700 1 --2- 172.21.15.88:0/2787460727 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f857006d8a0 0x7f857006fd50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8578005950 tx=0x7f857800a300 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:49.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:49.818+0000 7f856ffff700 1 -- 172.21.15.88:0/2787460727 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f857c058180 con 0x7f8588100e80 2024-02-19T19:31:49.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:49 smithi132 ceph-mon[70676]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:49.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:49 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/454367229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:31:49.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:49 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1358652476' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:31:50.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.032+0000 7f858d539700 1 -- 172.21.15.88:0/2787460727 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85881972e0 con 0x7f857006d8a0 2024-02-19T19:31:50.035 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:49 smithi088 ceph-mon[88643]: from='client.35546 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:50.035 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:49 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/454367229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:31:50.036 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:49 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1358652476' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:31:50.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.034+0000 7f856ffff700 1 -- 172.21.15.88:0/2787460727 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f85881972e0 con 0x7f857006d8a0 2024-02-19T19:31:50.037 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:31:50.037 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:31:50.037 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:31:50.037 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:31:50.037 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:31:50.037 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:31:50.037 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:31:50.037 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:31:50.037 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:31:50.037 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:31:50.037 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:31:50.038 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:31:50.038 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:31:50.038 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:31:50.038 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:31:50.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.036+0000 7f858d539700 1 -- 172.21.15.88:0/2787460727 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f857006d8a0 msgr2=0x7f857006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:50.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.036+0000 7f858d539700 1 --2- 172.21.15.88:0/2787460727 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f857006d8a0 0x7f857006fd50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8578005950 tx=0x7f857800a300 comp rx=0 tx=0).stop 2024-02-19T19:31:50.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.036+0000 7f858d539700 1 -- 172.21.15.88:0/2787460727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8588100e80 msgr2=0x7f8588196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:50.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.036+0000 7f858d539700 1 --2- 172.21.15.88:0/2787460727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8588100e80 0x7f8588196450 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f857c0092e0 tx=0x7f857c00bd70 comp rx=0 tx=0).stop 2024-02-19T19:31:50.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.036+0000 7f858d539700 1 -- 172.21.15.88:0/2787460727 shutdown_connections 2024-02-19T19:31:50.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.036+0000 7f858d539700 1 --2- 172.21.15.88:0/2787460727 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f857006d8a0 0x7f857006fd50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:50.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.036+0000 7f858d539700 1 --2- 172.21.15.88:0/2787460727 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85881004e0 0x7f8588195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:50.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.036+0000 7f858d539700 1 --2- 172.21.15.88:0/2787460727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8588100e80 0x7f8588196450 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:50.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.036+0000 7f858d539700 1 -- 172.21.15.88:0/2787460727 >> 172.21.15.88:0/2787460727 conn(0x7f85880fbda0 msgr2=0x7f85880fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:50.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.036+0000 7f858d539700 1 -- 172.21.15.88:0/2787460727 shutdown_connections 2024-02-19T19:31:50.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.036+0000 7f858d539700 1 -- 172.21.15.88:0/2787460727 wait complete. 2024-02-19T19:31:50.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.140+0000 7f289527b700 1 -- 172.21.15.88:0/1255903861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900ffbd0 msgr2=0x7f2890100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:50.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.140+0000 7f289527b700 1 --2- 172.21.15.88:0/1255903861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900ffbd0 0x7f2890100040 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f2884009910 tx=0x7f2884009c20 comp rx=0 tx=0).stop 2024-02-19T19:31:50.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.140+0000 7f289527b700 1 -- 172.21.15.88:0/1255903861 shutdown_connections 2024-02-19T19:31:50.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.140+0000 7f289527b700 1 --2- 172.21.15.88:0/1255903861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900ffbd0 0x7f2890100040 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:50.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.140+0000 7f289527b700 1 --2- 172.21.15.88:0/1255903861 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890104130 0x7f2890104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:50.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.140+0000 7f289527b700 1 -- 172.21.15.88:0/1255903861 >> 172.21.15.88:0/1255903861 conn(0x7f2890075130 msgr2=0x7f2890075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:50.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.140+0000 7f289527b700 1 -- 172.21.15.88:0/1255903861 shutdown_connections 2024-02-19T19:31:50.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.140+0000 7f289527b700 1 -- 172.21.15.88:0/1255903861 wait complete. 2024-02-19T19:31:50.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.141+0000 7f289527b700 1 Processor -- start 2024-02-19T19:31:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.141+0000 7f289527b700 1 -- start start 2024-02-19T19:31:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.141+0000 7f289527b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900ffbd0 0x7f2890196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.141+0000 7f289527b700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890104130 0x7f2890197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.141+0000 7f289527b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2890197860 con 0x7f28900ffbd0 2024-02-19T19:31:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.141+0000 7f289527b700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28901979a0 con 0x7f2890104130 2024-02-19T19:31:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.141+0000 7f288effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900ffbd0 0x7f2890196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.141+0000 7f288effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900ffbd0 0x7f2890196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44330/0 (socket says 172.21.15.88:44330) 2024-02-19T19:31:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.141+0000 7f288effd700 1 -- 172.21.15.88:0/2631167161 learned_addr learned my addr 172.21.15.88:0/2631167161 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:31:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.142+0000 7f288e7fc700 1 --2- 172.21.15.88:0/2631167161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890104130 0x7f2890197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.142+0000 7f288effd700 1 -- 172.21.15.88:0/2631167161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890104130 msgr2=0x7f2890197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:50.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.142+0000 7f288effd700 1 --2- 172.21.15.88:0/2631167161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890104130 0x7f2890197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:50.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.142+0000 7f288effd700 1 -- 172.21.15.88:0/2631167161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840095f0 con 0x7f28900ffbd0 2024-02-19T19:31:50.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.142+0000 7f288effd700 1 --2- 172.21.15.88:0/2631167161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900ffbd0 0x7f2890196c40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f288000e920 tx=0x7f288000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:50.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.142+0000 7f2877fff700 1 -- 172.21.15.88:0/2631167161 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2880009790 con 0x7f28900ffbd0 2024-02-19T19:31:50.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.142+0000 7f289527b700 1 -- 172.21.15.88:0/2631167161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289019fcb0 con 0x7f28900ffbd0 2024-02-19T19:31:50.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.142+0000 7f2877fff700 1 -- 172.21.15.88:0/2631167161 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f288000cba0 con 0x7f28900ffbd0 2024-02-19T19:31:50.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.142+0000 7f2877fff700 1 -- 172.21.15.88:0/2631167161 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2880018650 con 0x7f28900ffbd0 2024-02-19T19:31:50.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.142+0000 7f289527b700 1 -- 172.21.15.88:0/2631167161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28901a01d0 con 0x7f28900ffbd0 2024-02-19T19:31:50.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.143+0000 7f289527b700 1 -- 172.21.15.88:0/2631167161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289004f2a0 con 0x7f28900ffbd0 2024-02-19T19:31:50.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.144+0000 7f2877fff700 1 -- 172.21.15.88:0/2631167161 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f288000cd10 con 0x7f28900ffbd0 2024-02-19T19:31:50.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.144+0000 7f2877fff700 1 --2- 172.21.15.88:0/2631167161 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f287806d8a0 0x7f287806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:31:50.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.144+0000 7f2877fff700 1 -- 172.21.15.88:0/2631167161 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2880014070 con 0x7f28900ffbd0 2024-02-19T19:31:50.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.144+0000 7f288e7fc700 1 --2- 172.21.15.88:0/2631167161 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f287806d8a0 0x7f287806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:31:50.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.144+0000 7f288e7fc700 1 --2- 172.21.15.88:0/2631167161 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f287806d8a0 0x7f287806fd50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2884005850 tx=0x7f2884005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:31:50.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.149+0000 7f2877fff700 1 -- 172.21.15.88:0/2631167161 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2880052d20 con 0x7f28900ffbd0 2024-02-19T19:31:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.430+0000 7f289527b700 1 -- 172.21.15.88:0/2631167161 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f28901a0490 con 0x7f28900ffbd0 2024-02-19T19:31:50.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.430+0000 7f2877fff700 1 -- 172.21.15.88:0/2631167161 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f288005ac50 con 0x7f28900ffbd0 2024-02-19T19:31:50.433 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:31:50.433 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:31:50.433 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:31:50.434 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:31:50.435 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:31:50.435 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:31:50.435 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:31:50.435 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:31:50.435 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:31:50.435 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:31:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.432+0000 7f289527b700 1 -- 172.21.15.88:0/2631167161 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f287806d8a0 msgr2=0x7f287806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.432+0000 7f289527b700 1 --2- 172.21.15.88:0/2631167161 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f287806d8a0 0x7f287806fd50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2884005850 tx=0x7f2884005e20 comp rx=0 tx=0).stop 2024-02-19T19:31:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.432+0000 7f289527b700 1 -- 172.21.15.88:0/2631167161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900ffbd0 msgr2=0x7f2890196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:31:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.432+0000 7f289527b700 1 --2- 172.21.15.88:0/2631167161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900ffbd0 0x7f2890196c40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f288000e920 tx=0x7f288000ece0 comp rx=0 tx=0).stop 2024-02-19T19:31:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.432+0000 7f289527b700 1 -- 172.21.15.88:0/2631167161 shutdown_connections 2024-02-19T19:31:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.432+0000 7f289527b700 1 --2- 172.21.15.88:0/2631167161 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f287806d8a0 0x7f287806fd50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.432+0000 7f289527b700 1 --2- 172.21.15.88:0/2631167161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900ffbd0 0x7f2890196c40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.432+0000 7f289527b700 1 --2- 172.21.15.88:0/2631167161 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2890104130 0x7f2890197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:31:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.432+0000 7f289527b700 1 -- 172.21.15.88:0/2631167161 >> 172.21.15.88:0/2631167161 conn(0x7f2890075130 msgr2=0x7f28900feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:31:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.432+0000 7f289527b700 1 -- 172.21.15.88:0/2631167161 shutdown_connections 2024-02-19T19:31:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:31:50.433+0000 7f289527b700 1 -- 172.21.15.88:0/2631167161 wait complete. 2024-02-19T19:31:51.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:50 smithi088 ceph-mon[88643]: from='client.35558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:51.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:50 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2631167161' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:31:51.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:50 smithi088 ceph-mon[88643]: pgmap v849: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:51.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:50 smithi132 ceph-mon[70676]: from='client.35558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:31:51.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:50 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2631167161' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:31:51.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:50 smithi132 ceph-mon[70676]: pgmap v849: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:53.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:53 smithi132 ceph-mon[70676]: pgmap v850: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:54.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:53 smithi088 ceph-mon[88643]: pgmap v850: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:55.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:55 smithi132 ceph-mon[70676]: pgmap v851: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:56.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:55 smithi088 ceph-mon[88643]: pgmap v851: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:57.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:57 smithi132 ceph-mon[70676]: pgmap v852: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:58.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:57 smithi088 ceph-mon[88643]: pgmap v852: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:31:58.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:31:59.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:31:59.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:31:59 smithi132 ceph-mon[70676]: pgmap v853: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:00.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:31:59 smithi088 ceph-mon[88643]: pgmap v853: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:01.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:00 smithi088 ceph-mon[88643]: pgmap v854: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:01.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:00 smithi132 ceph-mon[70676]: pgmap v854: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:03.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:03 smithi132 ceph-mon[70676]: pgmap v855: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:04.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:03 smithi088 ceph-mon[88643]: pgmap v855: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:05.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:05 smithi132 ceph-mon[70676]: pgmap v856: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:06.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:05 smithi088 ceph-mon[88643]: pgmap v856: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:07.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:07 smithi132 ceph-mon[70676]: pgmap v857: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:08.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:07 smithi088 ceph-mon[88643]: pgmap v857: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:09.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:09 smithi132 ceph-mon[70676]: pgmap v858: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:10.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:09 smithi088 ceph-mon[88643]: pgmap v858: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:11.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:11 smithi132 ceph-mon[70676]: pgmap v859: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:12.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:11 smithi088 ceph-mon[88643]: pgmap v859: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:12.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:12 smithi132 ceph-mon[70676]: pgmap v860: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:13.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:12 smithi088 ceph-mon[88643]: pgmap v860: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:13.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:32:14.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:32:14.983 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:14 smithi088 ceph-mon[88643]: pgmap v861: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:14.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:14 smithi132 ceph-mon[70676]: pgmap v861: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:17.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:17 smithi132 ceph-mon[70676]: pgmap v862: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:18.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:17 smithi088 ceph-mon[88643]: pgmap v862: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:19.973 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:19 smithi132 ceph-mon[70676]: pgmap v863: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:20.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:19 smithi088 ceph-mon[88643]: pgmap v863: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.547+0000 7f96691f2700 1 -- 172.21.15.88:0/737767605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96640ffbd0 msgr2=0x7f9664100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.547+0000 7f96691f2700 1 --2- 172.21.15.88:0/737767605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96640ffbd0 0x7f9664100040 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9658009910 tx=0x7f9658009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.547+0000 7f96691f2700 1 -- 172.21.15.88:0/737767605 shutdown_connections 2024-02-19T19:32:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.547+0000 7f96691f2700 1 --2- 172.21.15.88:0/737767605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96640ffbd0 0x7f9664100040 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.547+0000 7f96691f2700 1 --2- 172.21.15.88:0/737767605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9664104130 0x7f9664104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:20.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.547+0000 7f96691f2700 1 -- 172.21.15.88:0/737767605 >> 172.21.15.88:0/737767605 conn(0x7f9664075130 msgr2=0x7f9664075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:20.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.547+0000 7f96691f2700 1 -- 172.21.15.88:0/737767605 shutdown_connections 2024-02-19T19:32:20.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.547+0000 7f96691f2700 1 -- 172.21.15.88:0/737767605 wait complete. 2024-02-19T19:32:20.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.548+0000 7f96691f2700 1 Processor -- start 2024-02-19T19:32:20.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.548+0000 7f96691f2700 1 -- start start 2024-02-19T19:32:20.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.548+0000 7f96691f2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96640ffbd0 0x7f9664195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:20.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.548+0000 7f96691f2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9664104130 0x7f96641963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:20.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.548+0000 7f96691f2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9664196a90 con 0x7f96640ffbd0 2024-02-19T19:32:20.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.548+0000 7f96691f2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f966419a820 con 0x7f9664104130 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.548+0000 7f9662d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96640ffbd0 0x7f9664195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f9662d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96640ffbd0 0x7f9664195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40002/0 (socket says 172.21.15.88:40002) 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f9662d9d700 1 -- 172.21.15.88:0/1270979082 learned_addr learned my addr 172.21.15.88:0/1270979082 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f966259c700 1 --2- 172.21.15.88:0/1270979082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9664104130 0x7f96641963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f9662d9d700 1 -- 172.21.15.88:0/1270979082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9664104130 msgr2=0x7f96641963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f9662d9d700 1 --2- 172.21.15.88:0/1270979082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9664104130 0x7f96641963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f9662d9d700 1 -- 172.21.15.88:0/1270979082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96580095f0 con 0x7f96640ffbd0 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f9662d9d700 1 --2- 172.21.15.88:0/1270979082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96640ffbd0 0x7f9664195e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f965400c8a0 tx=0x7f965400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f964bfff700 1 -- 172.21.15.88:0/1270979082 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f965400cea0 con 0x7f96640ffbd0 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f964bfff700 1 -- 172.21.15.88:0/1270979082 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9654004830 con 0x7f96640ffbd0 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f96691f2700 1 -- 172.21.15.88:0/1270979082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f966419ab00 con 0x7f96640ffbd0 2024-02-19T19:32:20.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f964bfff700 1 -- 172.21.15.88:0/1270979082 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96540055b0 con 0x7f96640ffbd0 2024-02-19T19:32:20.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.549+0000 7f96691f2700 1 -- 172.21.15.88:0/1270979082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f966419b050 con 0x7f96640ffbd0 2024-02-19T19:32:20.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.550+0000 7f96691f2700 1 -- 172.21.15.88:0/1270979082 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96641086a0 con 0x7f96640ffbd0 2024-02-19T19:32:20.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.551+0000 7f964bfff700 1 -- 172.21.15.88:0/1270979082 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9654007ad0 con 0x7f96640ffbd0 2024-02-19T19:32:20.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.551+0000 7f964bfff700 1 --2- 172.21.15.88:0/1270979082 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f964c06d8a0 0x7f964c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:20.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.551+0000 7f964bfff700 1 -- 172.21.15.88:0/1270979082 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f965408d260 con 0x7f96640ffbd0 2024-02-19T19:32:20.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.551+0000 7f966259c700 1 --2- 172.21.15.88:0/1270979082 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f964c06d8a0 0x7f964c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:20.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.552+0000 7f966259c700 1 --2- 172.21.15.88:0/1270979082 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f964c06d8a0 0x7f964c06fd50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f96580059d0 tx=0x7f965801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:20.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.556+0000 7f964bfff700 1 -- 172.21.15.88:0/1270979082 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9654056a10 con 0x7f96640ffbd0 2024-02-19T19:32:20.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.772+0000 7f96691f2700 1 -- 172.21.15.88:0/1270979082 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f966419b2d0 con 0x7f964c06d8a0 2024-02-19T19:32:20.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.773+0000 7f964bfff700 1 -- 172.21.15.88:0/1270979082 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f966419b2d0 con 0x7f964c06d8a0 2024-02-19T19:32:20.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.775+0000 7f96691f2700 1 -- 172.21.15.88:0/1270979082 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f964c06d8a0 msgr2=0x7f964c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:20.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.775+0000 7f96691f2700 1 --2- 172.21.15.88:0/1270979082 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f964c06d8a0 0x7f964c06fd50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f96580059d0 tx=0x7f965801a040 comp rx=0 tx=0).stop 2024-02-19T19:32:20.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.775+0000 7f96691f2700 1 -- 172.21.15.88:0/1270979082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96640ffbd0 msgr2=0x7f9664195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:20.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.775+0000 7f96691f2700 1 --2- 172.21.15.88:0/1270979082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96640ffbd0 0x7f9664195e70 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f965400c8a0 tx=0x7f965400cc60 comp rx=0 tx=0).stop 2024-02-19T19:32:20.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.775+0000 7f96691f2700 1 -- 172.21.15.88:0/1270979082 shutdown_connections 2024-02-19T19:32:20.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.775+0000 7f96691f2700 1 --2- 172.21.15.88:0/1270979082 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f964c06d8a0 0x7f964c06fd50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:20.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.775+0000 7f96691f2700 1 --2- 172.21.15.88:0/1270979082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96640ffbd0 0x7f9664195e70 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:20.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.775+0000 7f96691f2700 1 --2- 172.21.15.88:0/1270979082 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9664104130 0x7f96641963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:20.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.775+0000 7f96691f2700 1 -- 172.21.15.88:0/1270979082 >> 172.21.15.88:0/1270979082 conn(0x7f9664075130 msgr2=0x7f96640fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:20.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.776+0000 7f96691f2700 1 -- 172.21.15.88:0/1270979082 shutdown_connections 2024-02-19T19:32:20.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.776+0000 7f96691f2700 1 -- 172.21.15.88:0/1270979082 wait complete. 2024-02-19T19:32:20.788 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:32:20.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.883+0000 7fd312039700 1 -- 172.21.15.88:0/1035912221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd30c0ffd60 msgr2=0x7fd30c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:20.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.883+0000 7fd312039700 1 --2- 172.21.15.88:0/1035912221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd30c0ffd60 0x7fd30c103d30 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4009910 tx=0x7fd2f4009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:20.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.883+0000 7fd312039700 1 -- 172.21.15.88:0/1035912221 shutdown_connections 2024-02-19T19:32:20.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.883+0000 7fd312039700 1 --2- 172.21.15.88:0/1035912221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd30c0ffd60 0x7fd30c103d30 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:20.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.883+0000 7fd312039700 1 --2- 172.21.15.88:0/1035912221 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c0ff3c0 0x7fd30c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:20.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.883+0000 7fd312039700 1 -- 172.21.15.88:0/1035912221 >> 172.21.15.88:0/1035912221 conn(0x7fd30c0fac80 msgr2=0x7fd30c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:20.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.883+0000 7fd312039700 1 -- 172.21.15.88:0/1035912221 shutdown_connections 2024-02-19T19:32:20.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.883+0000 7fd312039700 1 -- 172.21.15.88:0/1035912221 wait complete. 2024-02-19T19:32:20.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.884+0000 7fd312039700 1 Processor -- start 2024-02-19T19:32:20.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.884+0000 7fd312039700 1 -- start start 2024-02-19T19:32:20.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.884+0000 7fd312039700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd30c0ff3c0 0x7fd30c195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:20.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.884+0000 7fd312039700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c0ffd60 0x7fd30c1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:20.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.884+0000 7fd312039700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c196aa0 con 0x7fd30c0ff3c0 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.884+0000 7fd312039700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c19a830 con 0x7fd30c0ffd60 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.884+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd30c0ff3c0 0x7fd30c195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.884+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd30c0ff3c0 0x7fd30c195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40024/0 (socket says 172.21.15.88:40024) 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.884+0000 7fd30b7fe700 1 -- 172.21.15.88:0/3263827948 learned_addr learned my addr 172.21.15.88:0/3263827948 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.885+0000 7fd302ffd700 1 --2- 172.21.15.88:0/3263827948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c0ffd60 0x7fd30c1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.885+0000 7fd30b7fe700 1 -- 172.21.15.88:0/3263827948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c0ffd60 msgr2=0x7fd30c1963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.885+0000 7fd30b7fe700 1 --2- 172.21.15.88:0/3263827948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c0ffd60 0x7fd30c1963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.885+0000 7fd30b7fe700 1 -- 172.21.15.88:0/3263827948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f40095f0 con 0x7fd30c0ff3c0 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.885+0000 7fd30b7fe700 1 --2- 172.21.15.88:0/3263827948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd30c0ff3c0 0x7fd30c195e80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00d8d0 tx=0x7fd2fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.885+0000 7fd3097fa700 1 -- 172.21.15.88:0/3263827948 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2fc009940 con 0x7fd30c0ff3c0 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.885+0000 7fd312039700 1 -- 172.21.15.88:0/3263827948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c19ab10 con 0x7fd30c0ff3c0 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.885+0000 7fd3097fa700 1 -- 172.21.15.88:0/3263827948 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2fc010460 con 0x7fd30c0ff3c0 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.885+0000 7fd3097fa700 1 -- 172.21.15.88:0/3263827948 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2fc00f5f0 con 0x7fd30c0ff3c0 2024-02-19T19:32:20.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.885+0000 7fd312039700 1 -- 172.21.15.88:0/3263827948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c19b060 con 0x7fd30c0ff3c0 2024-02-19T19:32:20.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.886+0000 7fd312039700 1 -- 172.21.15.88:0/3263827948 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c04f2a0 con 0x7fd30c0ff3c0 2024-02-19T19:32:20.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.886+0000 7fd3097fa700 1 -- 172.21.15.88:0/3263827948 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd2fc0105d0 con 0x7fd30c0ff3c0 2024-02-19T19:32:20.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.887+0000 7fd3097fa700 1 --2- 172.21.15.88:0/3263827948 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd2f806d8a0 0x7fd2f806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:20.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.887+0000 7fd3097fa700 1 -- 172.21.15.88:0/3263827948 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd2fc08d540 con 0x7fd30c0ff3c0 2024-02-19T19:32:20.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.887+0000 7fd302ffd700 1 --2- 172.21.15.88:0/3263827948 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd2f806d8a0 0x7fd2f806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:20.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.887+0000 7fd302ffd700 1 --2- 172.21.15.88:0/3263827948 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd2f806d8a0 0x7fd2f806fd50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40059d0 tx=0x7fd2f4005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:20.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:20.892+0000 7fd3097fa700 1 -- 172.21.15.88:0/3263827948 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd2fc056cf0 con 0x7fd30c0ff3c0 2024-02-19T19:32:21.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.105+0000 7fd312039700 1 -- 172.21.15.88:0/3263827948 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd30c19b390 con 0x7fd2f806d8a0 2024-02-19T19:32:21.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.106+0000 7fd3097fa700 1 -- 172.21.15.88:0/3263827948 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd30c19b390 con 0x7fd2f806d8a0 2024-02-19T19:32:21.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.108+0000 7fd312039700 1 -- 172.21.15.88:0/3263827948 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd2f806d8a0 msgr2=0x7fd2f806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.108+0000 7fd312039700 1 --2- 172.21.15.88:0/3263827948 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd2f806d8a0 0x7fd2f806fd50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40059d0 tx=0x7fd2f4005960 comp rx=0 tx=0).stop 2024-02-19T19:32:21.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.108+0000 7fd312039700 1 -- 172.21.15.88:0/3263827948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd30c0ff3c0 msgr2=0x7fd30c195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.108+0000 7fd312039700 1 --2- 172.21.15.88:0/3263827948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd30c0ff3c0 0x7fd30c195e80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc00d8d0 tx=0x7fd2fc00dc90 comp rx=0 tx=0).stop 2024-02-19T19:32:21.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.108+0000 7fd312039700 1 -- 172.21.15.88:0/3263827948 shutdown_connections 2024-02-19T19:32:21.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.108+0000 7fd312039700 1 --2- 172.21.15.88:0/3263827948 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd2f806d8a0 0x7fd2f806fd50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.108+0000 7fd312039700 1 --2- 172.21.15.88:0/3263827948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd30c0ff3c0 0x7fd30c195e80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.108+0000 7fd312039700 1 --2- 172.21.15.88:0/3263827948 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd30c0ffd60 0x7fd30c1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.108+0000 7fd312039700 1 -- 172.21.15.88:0/3263827948 >> 172.21.15.88:0/3263827948 conn(0x7fd30c0fac80 msgr2=0x7fd30c103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:21.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.108+0000 7fd312039700 1 -- 172.21.15.88:0/3263827948 shutdown_connections 2024-02-19T19:32:21.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.108+0000 7fd312039700 1 -- 172.21.15.88:0/3263827948 wait complete. 2024-02-19T19:32:21.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.212+0000 7f68fceae700 1 -- 172.21.15.88:0/4067640477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f8100530 msgr2=0x7f68f8100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.212+0000 7f68fceae700 1 --2- 172.21.15.88:0/4067640477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f8100530 0x7f68f8100900 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f68ec009910 tx=0x7f68ec009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:21.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.212+0000 7f68fceae700 1 -- 172.21.15.88:0/4067640477 shutdown_connections 2024-02-19T19:32:21.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.212+0000 7f68fceae700 1 --2- 172.21.15.88:0/4067640477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f8100ed0 0x7f68f8104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.212+0000 7f68fceae700 1 --2- 172.21.15.88:0/4067640477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f8100530 0x7f68f8100900 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.212+0000 7f68fceae700 1 -- 172.21.15.88:0/4067640477 >> 172.21.15.88:0/4067640477 conn(0x7f68f80fbe10 msgr2=0x7f68f80fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:21.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.212+0000 7f68fceae700 1 -- 172.21.15.88:0/4067640477 shutdown_connections 2024-02-19T19:32:21.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.212+0000 7f68fceae700 1 -- 172.21.15.88:0/4067640477 wait complete. 2024-02-19T19:32:21.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68fceae700 1 Processor -- start 2024-02-19T19:32:21.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68fceae700 1 -- start start 2024-02-19T19:32:21.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68fceae700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f8100530 0x7f68f8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:21.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68fceae700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f8100ed0 0x7f68f8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:21.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68fceae700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f8073ad0 con 0x7f68f8100ed0 2024-02-19T19:32:21.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68fceae700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f8073c10 con 0x7f68f8100530 2024-02-19T19:32:21.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68f5d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f8100ed0 0x7f68f8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:21.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68f5d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f8100ed0 0x7f68f8073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40032/0 (socket says 172.21.15.88:40032) 2024-02-19T19:32:21.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68f5d9b700 1 -- 172.21.15.88:0/4049048432 learned_addr learned my addr 172.21.15.88:0/4049048432 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:21.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68f659c700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f8100530 0x7f68f8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:21.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68f5d9b700 1 -- 172.21.15.88:0/4049048432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f8100530 msgr2=0x7f68f8073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68f5d9b700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f8100530 0x7f68f8073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68f5d9b700 1 -- 172.21.15.88:0/4049048432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ec0095f0 con 0x7f68f8100ed0 2024-02-19T19:32:21.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.213+0000 7f68f659c700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f8100530 0x7f68f8073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:32:21.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.214+0000 7f68f5d9b700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f8100ed0 0x7f68f8073590 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f68e800c960 tx=0x7f68e800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:21.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.214+0000 7f68e77fe700 1 -- 172.21.15.88:0/4049048432 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68e8007a70 con 0x7f68f8100ed0 2024-02-19T19:32:21.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.214+0000 7f68e77fe700 1 -- 172.21.15.88:0/4049048432 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68e8007bd0 con 0x7f68f8100ed0 2024-02-19T19:32:21.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.214+0000 7f68fceae700 1 -- 172.21.15.88:0/4049048432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f81a2000 con 0x7f68f8100ed0 2024-02-19T19:32:21.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.214+0000 7f68e77fe700 1 -- 172.21.15.88:0/4049048432 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68e80186c0 con 0x7f68f8100ed0 2024-02-19T19:32:21.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.214+0000 7f68fceae700 1 -- 172.21.15.88:0/4049048432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f81a2360 con 0x7f68f8100ed0 2024-02-19T19:32:21.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.215+0000 7f68fceae700 1 -- 172.21.15.88:0/4049048432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f8108840 con 0x7f68f8100ed0 2024-02-19T19:32:21.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.215+0000 7f68e77fe700 1 -- 172.21.15.88:0/4049048432 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f68e800faa0 con 0x7f68f8100ed0 2024-02-19T19:32:21.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.216+0000 7f68e77fe700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68e006d8a0 0x7f68e006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:21.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.216+0000 7f68e77fe700 1 -- 172.21.15.88:0/4049048432 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f68e808d560 con 0x7f68f8100ed0 2024-02-19T19:32:21.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.216+0000 7f68f659c700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68e006d8a0 0x7f68e006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:21.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.216+0000 7f68f659c700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68e006d8a0 0x7f68e006fd50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00b5c0 tx=0x7f68ec005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:21.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.222+0000 7f68e77fe700 1 -- 172.21.15.88:0/4049048432 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f68e805a840 con 0x7f68f8100ed0 2024-02-19T19:32:21.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.429+0000 7f68fceae700 1 -- 172.21.15.88:0/4049048432 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f68f802cca0 con 0x7f68e006d8a0 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.440+0000 7f68e77fe700 1 -- 172.21.15.88:0/4049048432 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f68f802cca0 con 0x7f68e006d8a0 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (29m) 5m ago 39m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (40m) 5m ago 40m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (22m) 5m ago 38m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (25m) 5m ago 40m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (25m) 5m ago 38m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (28m) 5m ago 39m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (21m) 5m ago 33m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (21m) 5m ago 33m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (21m) 5m ago 33m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (5m) 5m ago 33m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:32:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (31m) 5m ago 41m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (30m) 5m ago 38m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (27m) 5m ago 41m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (26m) 5m ago 38m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (30m) 5m ago 40m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (30m) 5m ago 38m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (25m) 5m ago 37m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (24m) 5m ago 37m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (23m) 5m ago 36m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (23m) 5m ago 36m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (22m) 5m ago 35m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (22m) 5m ago 35m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:32:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (29m) 5m ago 38m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:32:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.442+0000 7f68fceae700 1 -- 172.21.15.88:0/4049048432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68e006d8a0 msgr2=0x7f68e006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.442+0000 7f68fceae700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68e006d8a0 0x7f68e006fd50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00b5c0 tx=0x7f68ec005960 comp rx=0 tx=0).stop 2024-02-19T19:32:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.442+0000 7f68fceae700 1 -- 172.21.15.88:0/4049048432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f8100ed0 msgr2=0x7f68f8073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.442+0000 7f68fceae700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f8100ed0 0x7f68f8073590 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f68e800c960 tx=0x7f68e800cd20 comp rx=0 tx=0).stop 2024-02-19T19:32:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.442+0000 7f68fceae700 1 -- 172.21.15.88:0/4049048432 shutdown_connections 2024-02-19T19:32:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.442+0000 7f68fceae700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f68e006d8a0 0x7f68e006fd50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.442+0000 7f68fceae700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f68f8100530 0x7f68f8073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.442+0000 7f68fceae700 1 --2- 172.21.15.88:0/4049048432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68f8100ed0 0x7f68f8073590 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.442+0000 7f68fceae700 1 -- 172.21.15.88:0/4049048432 >> 172.21.15.88:0/4049048432 conn(0x7f68f80fbe10 msgr2=0x7f68f80fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:21.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.442+0000 7f68fceae700 1 -- 172.21.15.88:0/4049048432 shutdown_connections 2024-02-19T19:32:21.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.442+0000 7f68fceae700 1 -- 172.21.15.88:0/4049048432 wait complete. 2024-02-19T19:32:21.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.545+0000 7ff248812700 1 -- 172.21.15.88:0/3724087898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2441038b0 msgr2=0x7ff244103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.545+0000 7ff248812700 1 --2- 172.21.15.88:0/3724087898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2441038b0 0x7ff244103c80 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ff234009910 tx=0x7ff234009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:21.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.545+0000 7ff248812700 1 -- 172.21.15.88:0/3724087898 shutdown_connections 2024-02-19T19:32:21.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.545+0000 7ff248812700 1 --2- 172.21.15.88:0/3724087898 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2440ff350 0x7ff2440ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.545+0000 7ff248812700 1 --2- 172.21.15.88:0/3724087898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2441038b0 0x7ff244103c80 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.545+0000 7ff248812700 1 -- 172.21.15.88:0/3724087898 >> 172.21.15.88:0/3724087898 conn(0x7ff2440748b0 msgr2=0x7ff244074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:21.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.545+0000 7ff248812700 1 -- 172.21.15.88:0/3724087898 shutdown_connections 2024-02-19T19:32:21.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.545+0000 7ff248812700 1 -- 172.21.15.88:0/3724087898 wait complete. 2024-02-19T19:32:21.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff248812700 1 Processor -- start 2024-02-19T19:32:21.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff248812700 1 -- start start 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff248812700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2440ff350 0x7ff244198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff248812700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2441038b0 0x7ff244198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff248812700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff244198c50 con 0x7ff2440ff350 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff248812700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24419c9e0 con 0x7ff2441038b0 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff24259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2440ff350 0x7ff244198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff24259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2440ff350 0x7ff244198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40056/0 (socket says 172.21.15.88:40056) 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff24259c700 1 -- 172.21.15.88:0/3831321443 learned_addr learned my addr 172.21.15.88:0/3831321443 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff241d9b700 1 --2- 172.21.15.88:0/3831321443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2441038b0 0x7ff244198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff24259c700 1 -- 172.21.15.88:0/3831321443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2441038b0 msgr2=0x7ff244198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff24259c700 1 --2- 172.21.15.88:0/3831321443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2441038b0 0x7ff244198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff24259c700 1 -- 172.21.15.88:0/3831321443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2340095f0 con 0x7ff2440ff350 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.546+0000 7ff24259c700 1 --2- 172.21.15.88:0/3831321443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2440ff350 0x7ff244198030 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff23400bd70 tx=0x7ff23400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.547+0000 7ff2337fe700 1 -- 172.21.15.88:0/3831321443 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff23401d070 con 0x7ff2440ff350 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.547+0000 7ff2337fe700 1 -- 172.21.15.88:0/3831321443 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff234005dc0 con 0x7ff2440ff350 2024-02-19T19:32:21.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.547+0000 7ff248812700 1 -- 172.21.15.88:0/3831321443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24419cc60 con 0x7ff2440ff350 2024-02-19T19:32:21.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.547+0000 7ff2337fe700 1 -- 172.21.15.88:0/3831321443 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff234017400 con 0x7ff2440ff350 2024-02-19T19:32:21.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.547+0000 7ff248812700 1 -- 172.21.15.88:0/3831321443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24419d150 con 0x7ff2440ff350 2024-02-19T19:32:21.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.548+0000 7ff248812700 1 -- 172.21.15.88:0/3831321443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff24404f2a0 con 0x7ff2440ff350 2024-02-19T19:32:21.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.548+0000 7ff2337fe700 1 -- 172.21.15.88:0/3831321443 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff23400f460 con 0x7ff2440ff350 2024-02-19T19:32:21.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.549+0000 7ff2337fe700 1 --2- 172.21.15.88:0/3831321443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff22c06d8a0 0x7ff22c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:21.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.549+0000 7ff2337fe700 1 -- 172.21.15.88:0/3831321443 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff23408e9e0 con 0x7ff2440ff350 2024-02-19T19:32:21.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.549+0000 7ff241d9b700 1 --2- 172.21.15.88:0/3831321443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff22c06d8a0 0x7ff22c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:21.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.549+0000 7ff241d9b700 1 --2- 172.21.15.88:0/3831321443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff22c06d8a0 0x7ff22c06fd50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff244199650 tx=0x7ff23800b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:21.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.554+0000 7ff2337fe700 1 -- 172.21.15.88:0/3831321443 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff234058250 con 0x7ff2440ff350 2024-02-19T19:32:21.833 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:21 smithi088 ceph-mon[88643]: pgmap v864: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:21.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.853+0000 7ff248812700 1 -- 172.21.15.88:0/3831321443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff24419d430 con 0x7ff2440ff350 2024-02-19T19:32:21.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.853+0000 7ff2337fe700 1 -- 172.21.15.88:0/3831321443 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff23405b910 con 0x7ff2440ff350 2024-02-19T19:32:21.856 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:32:21.856 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:32:21.856 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:32:21.856 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:32:21.856 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:32:21.857 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:32:21.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.855+0000 7ff248812700 1 -- 172.21.15.88:0/3831321443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff22c06d8a0 msgr2=0x7ff22c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.855+0000 7ff248812700 1 --2- 172.21.15.88:0/3831321443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff22c06d8a0 0x7ff22c06fd50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff244199650 tx=0x7ff23800b410 comp rx=0 tx=0).stop 2024-02-19T19:32:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.855+0000 7ff248812700 1 -- 172.21.15.88:0/3831321443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2440ff350 msgr2=0x7ff244198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.855+0000 7ff248812700 1 --2- 172.21.15.88:0/3831321443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2440ff350 0x7ff244198030 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ff23400bd70 tx=0x7ff23400bda0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.855+0000 7ff248812700 1 -- 172.21.15.88:0/3831321443 shutdown_connections 2024-02-19T19:32:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.855+0000 7ff248812700 1 --2- 172.21.15.88:0/3831321443 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff22c06d8a0 0x7ff22c06fd50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.855+0000 7ff248812700 1 --2- 172.21.15.88:0/3831321443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2440ff350 0x7ff244198030 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.855+0000 7ff248812700 1 --2- 172.21.15.88:0/3831321443 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff2441038b0 0x7ff244198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.855+0000 7ff248812700 1 -- 172.21.15.88:0/3831321443 >> 172.21.15.88:0/3831321443 conn(0x7ff2440748b0 msgr2=0x7ff244102d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.856+0000 7ff248812700 1 -- 172.21.15.88:0/3831321443 shutdown_connections 2024-02-19T19:32:21.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.856+0000 7ff248812700 1 -- 172.21.15.88:0/3831321443 wait complete. 2024-02-19T19:32:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.959+0000 7fd582e60700 1 -- 172.21.15.88:0/2561735934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c0730f0 msgr2=0x7fd57c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.959+0000 7fd582e60700 1 --2- 172.21.15.88:0/2561735934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c0730f0 0x7fd57c0734c0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fd578009910 tx=0x7fd578009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.959+0000 7fd582e60700 1 -- 172.21.15.88:0/2561735934 shutdown_connections 2024-02-19T19:32:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.959+0000 7fd582e60700 1 --2- 172.21.15.88:0/2561735934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c073a00 0x7fd57c110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.959+0000 7fd582e60700 1 --2- 172.21.15.88:0/2561735934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c0730f0 0x7fd57c0734c0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.959+0000 7fd582e60700 1 -- 172.21.15.88:0/2561735934 >> 172.21.15.88:0/2561735934 conn(0x7fd57c0fbda0 msgr2=0x7fd57c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:21.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.959+0000 7fd582e60700 1 -- 172.21.15.88:0/2561735934 shutdown_connections 2024-02-19T19:32:21.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.959+0000 7fd582e60700 1 -- 172.21.15.88:0/2561735934 wait complete. 2024-02-19T19:32:21.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.959+0000 7fd582e60700 1 Processor -- start 2024-02-19T19:32:21.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.959+0000 7fd582e60700 1 -- start start 2024-02-19T19:32:21.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd582e60700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c0730f0 0x7fd57c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:21.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd582e60700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c073a00 0x7fd57c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd582e60700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c19d000 con 0x7fd57c073a00 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd582e60700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57c1a0d90 con 0x7fd57c0730f0 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd573fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c073a00 0x7fd57c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd573fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c073a00 0x7fd57c19c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40080/0 (socket says 172.21.15.88:40080) 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd573fff700 1 -- 172.21.15.88:0/3268144579 learned_addr learned my addr 172.21.15.88:0/3268144579 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd573fff700 1 -- 172.21.15.88:0/3268144579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c0730f0 msgr2=0x7fd57c19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd580bfc700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c0730f0 0x7fd57c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd573fff700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c0730f0 0x7fd57c19c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd573fff700 1 -- 172.21.15.88:0/3268144579 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5780095f0 con 0x7fd57c073a00 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd580bfc700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c0730f0 0x7fd57c19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd573fff700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c073a00 0x7fd57c19c920 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00c8a0 tx=0x7fd56c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd571ffb700 1 -- 172.21.15.88:0/3268144579 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd56c00cea0 con 0x7fd57c073a00 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd571ffb700 1 -- 172.21.15.88:0/3268144579 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd56c004830 con 0x7fd57c073a00 2024-02-19T19:32:21.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd582e60700 1 -- 172.21.15.88:0/3268144579 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57c1a1070 con 0x7fd57c073a00 2024-02-19T19:32:21.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.960+0000 7fd571ffb700 1 -- 172.21.15.88:0/3268144579 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd56c0055b0 con 0x7fd57c073a00 2024-02-19T19:32:21.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.961+0000 7fd582e60700 1 -- 172.21.15.88:0/3268144579 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57c1a15c0 con 0x7fd57c073a00 2024-02-19T19:32:21.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.962+0000 7fd582e60700 1 -- 172.21.15.88:0/3268144579 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd57c10e4d0 con 0x7fd57c073a00 2024-02-19T19:32:21.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.962+0000 7fd571ffb700 1 -- 172.21.15.88:0/3268144579 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd56c0049a0 con 0x7fd57c073a00 2024-02-19T19:32:21.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.962+0000 7fd571ffb700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56406d8a0 0x7fd56406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:21.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.962+0000 7fd571ffb700 1 -- 172.21.15.88:0/3268144579 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd56c08d270 con 0x7fd57c073a00 2024-02-19T19:32:21.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.962+0000 7fd580bfc700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56406d8a0 0x7fd56406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:21.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.963+0000 7fd580bfc700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56406d8a0 0x7fd56406fd50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd578006010 tx=0x7fd57800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:21.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:21.967+0000 7fd571ffb700 1 -- 172.21.15.88:0/3268144579 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd56c056980 con 0x7fd57c073a00 2024-02-19T19:32:21.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:21 smithi132 ceph-mon[70676]: pgmap v864: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:22.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.204+0000 7fd582e60700 1 -- 172.21.15.88:0/3268144579 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd57c1a1840 con 0x7fd57c073a00 2024-02-19T19:32:22.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.204+0000 7fd571ffb700 1 -- 172.21.15.88:0/3268144579 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fd56c004c50 con 0x7fd57c073a00 2024-02-19T19:32:22.207 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:32:22.208 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:32:22.209 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:32:22.209 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:32:22.209 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:32:22.209 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:32:22.209 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:32:22.209 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:32:22.209 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:32:22.209 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:32:22.209 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:32:22.209 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:32:22.210 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:32:22.210 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:32:22.210 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:32:22.210 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:32:22.210 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:32:22.210 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:32:22.210 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:32:22.210 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:32:22.210 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:32:22.211 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:32:22.211 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:32:22.211 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:32:22.211 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:32:22.211 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:32:22.211 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:32:22.211 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:32:22.211 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:32:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.206+0000 7fd582e60700 1 -- 172.21.15.88:0/3268144579 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56406d8a0 msgr2=0x7fd56406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.206+0000 7fd582e60700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56406d8a0 0x7fd56406fd50 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd578006010 tx=0x7fd57800b540 comp rx=0 tx=0).stop 2024-02-19T19:32:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.206+0000 7fd582e60700 1 -- 172.21.15.88:0/3268144579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c073a00 msgr2=0x7fd57c19c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.206+0000 7fd582e60700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c073a00 0x7fd57c19c920 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00c8a0 tx=0x7fd56c00cc60 comp rx=0 tx=0).stop 2024-02-19T19:32:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.206+0000 7fd582e60700 1 -- 172.21.15.88:0/3268144579 shutdown_connections 2024-02-19T19:32:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.206+0000 7fd582e60700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd56406d8a0 0x7fd56406fd50 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.206+0000 7fd582e60700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd57c0730f0 0x7fd57c19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.206+0000 7fd582e60700 1 --2- 172.21.15.88:0/3268144579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c073a00 0x7fd57c19c920 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.206+0000 7fd582e60700 1 -- 172.21.15.88:0/3268144579 >> 172.21.15.88:0/3268144579 conn(0x7fd57c0fbda0 msgr2=0x7fd57c102860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:22.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.207+0000 7fd582e60700 1 -- 172.21.15.88:0/3268144579 shutdown_connections 2024-02-19T19:32:22.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.207+0000 7fd582e60700 1 -- 172.21.15.88:0/3268144579 wait complete. 2024-02-19T19:32:22.213 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:32:22.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.311+0000 7f9d29872700 1 -- 172.21.15.88:0/2482141013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d241004c0 msgr2=0x7f9d24100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:22.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.311+0000 7f9d29872700 1 --2- 172.21.15.88:0/2482141013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d241004c0 0x7f9d24100890 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9d14009910 tx=0x7f9d14009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:22.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.311+0000 7f9d29872700 1 -- 172.21.15.88:0/2482141013 shutdown_connections 2024-02-19T19:32:22.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.311+0000 7f9d29872700 1 --2- 172.21.15.88:0/2482141013 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d24100e60 0x7f9d24104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.311+0000 7f9d29872700 1 --2- 172.21.15.88:0/2482141013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d241004c0 0x7f9d24100890 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.311+0000 7f9d29872700 1 -- 172.21.15.88:0/2482141013 >> 172.21.15.88:0/2482141013 conn(0x7f9d240fbd80 msgr2=0x7f9d240fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:22.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.311+0000 7f9d29872700 1 -- 172.21.15.88:0/2482141013 shutdown_connections 2024-02-19T19:32:22.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.311+0000 7f9d29872700 1 -- 172.21.15.88:0/2482141013 wait complete. 2024-02-19T19:32:22.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.312+0000 7f9d29872700 1 Processor -- start 2024-02-19T19:32:22.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.312+0000 7f9d29872700 1 -- start start 2024-02-19T19:32:22.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.312+0000 7f9d29872700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d241004c0 0x7f9d24195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:22.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.312+0000 7f9d29872700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d24100e60 0x7f9d24196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:22.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.312+0000 7f9d29872700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24196b50 con 0x7f9d24100e60 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.312+0000 7f9d29872700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2419a8e0 con 0x7f9d241004c0 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.312+0000 7f9d227fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d24100e60 0x7f9d24196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.312+0000 7f9d227fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d24100e60 0x7f9d24196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40098/0 (socket says 172.21.15.88:40098) 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.312+0000 7f9d227fc700 1 -- 172.21.15.88:0/798745055 learned_addr learned my addr 172.21.15.88:0/798745055 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.312+0000 7f9d22ffd700 1 --2- 172.21.15.88:0/798745055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d241004c0 0x7f9d24195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.313+0000 7f9d227fc700 1 -- 172.21.15.88:0/798745055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d241004c0 msgr2=0x7f9d24195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.313+0000 7f9d227fc700 1 --2- 172.21.15.88:0/798745055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d241004c0 0x7f9d24195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.313+0000 7f9d227fc700 1 -- 172.21.15.88:0/798745055 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d140095f0 con 0x7f9d24100e60 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.313+0000 7f9d227fc700 1 --2- 172.21.15.88:0/798745055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d24100e60 0x7f9d24196470 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800e8f0 tx=0x7f9d1800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.313+0000 7f9d28870700 1 -- 172.21.15.88:0/798745055 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d18009790 con 0x7f9d24100e60 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.313+0000 7f9d28870700 1 -- 172.21.15.88:0/798745055 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d1800c810 con 0x7f9d24100e60 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.313+0000 7f9d29872700 1 -- 172.21.15.88:0/798745055 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2419abc0 con 0x7f9d24100e60 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.313+0000 7f9d28870700 1 -- 172.21.15.88:0/798745055 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d18005580 con 0x7f9d24100e60 2024-02-19T19:32:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.313+0000 7f9d29872700 1 -- 172.21.15.88:0/798745055 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2419b110 con 0x7f9d24100e60 2024-02-19T19:32:22.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.314+0000 7f9d29872700 1 -- 172.21.15.88:0/798745055 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d241087d0 con 0x7f9d24100e60 2024-02-19T19:32:22.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.314+0000 7f9d28870700 1 -- 172.21.15.88:0/798745055 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9d1800c980 con 0x7f9d24100e60 2024-02-19T19:32:22.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.315+0000 7f9d28870700 1 --2- 172.21.15.88:0/798745055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d0c06d8a0 0x7f9d0c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:22.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.315+0000 7f9d28870700 1 -- 172.21.15.88:0/798745055 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9d18014070 con 0x7f9d24100e60 2024-02-19T19:32:22.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.315+0000 7f9d22ffd700 1 --2- 172.21.15.88:0/798745055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d0c06d8a0 0x7f9d0c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:22.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.315+0000 7f9d22ffd700 1 --2- 172.21.15.88:0/798745055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d0c06d8a0 0x7f9d0c06fd50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9d140059d0 tx=0x7f9d14005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:22.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.319+0000 7f9d28870700 1 -- 172.21.15.88:0/798745055 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9d18056f20 con 0x7f9d24100e60 2024-02-19T19:32:22.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.532+0000 7f9d29872700 1 -- 172.21.15.88:0/798745055 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d2419b3e0 con 0x7f9d0c06d8a0 2024-02-19T19:32:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.533+0000 7f9d28870700 1 -- 172.21.15.88:0/798745055 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9d2419b3e0 con 0x7f9d0c06d8a0 2024-02-19T19:32:22.536 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:32:22.536 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:32:22.536 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:32:22.537 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:32:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.536+0000 7f9d29872700 1 -- 172.21.15.88:0/798745055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d0c06d8a0 msgr2=0x7f9d0c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.536+0000 7f9d29872700 1 --2- 172.21.15.88:0/798745055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d0c06d8a0 0x7f9d0c06fd50 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9d140059d0 tx=0x7f9d14005940 comp rx=0 tx=0).stop 2024-02-19T19:32:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.536+0000 7f9d29872700 1 -- 172.21.15.88:0/798745055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d24100e60 msgr2=0x7f9d24196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.536+0000 7f9d29872700 1 --2- 172.21.15.88:0/798745055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d24100e60 0x7f9d24196470 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f9d1800e8f0 tx=0x7f9d1800ec00 comp rx=0 tx=0).stop 2024-02-19T19:32:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.536+0000 7f9d29872700 1 -- 172.21.15.88:0/798745055 shutdown_connections 2024-02-19T19:32:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.536+0000 7f9d29872700 1 --2- 172.21.15.88:0/798745055 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9d0c06d8a0 0x7f9d0c06fd50 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.536+0000 7f9d29872700 1 --2- 172.21.15.88:0/798745055 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9d241004c0 0x7f9d24195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.536+0000 7f9d29872700 1 --2- 172.21.15.88:0/798745055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d24100e60 0x7f9d24196470 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.536+0000 7f9d29872700 1 -- 172.21.15.88:0/798745055 >> 172.21.15.88:0/798745055 conn(0x7f9d240fbd80 msgr2=0x7f9d240fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:22.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.536+0000 7f9d29872700 1 -- 172.21.15.88:0/798745055 shutdown_connections 2024-02-19T19:32:22.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.536+0000 7f9d29872700 1 -- 172.21.15.88:0/798745055 wait complete. 2024-02-19T19:32:22.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.640+0000 7f12191b5700 1 -- 172.21.15.88:0/299186565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12141004e0 msgr2=0x7f12141008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:22.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.640+0000 7f12191b5700 1 --2- 172.21.15.88:0/299186565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12141004e0 0x7f12141008b0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f1208009910 tx=0x7f1208009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:22.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.640+0000 7f12191b5700 1 -- 172.21.15.88:0/299186565 shutdown_connections 2024-02-19T19:32:22.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.640+0000 7f12191b5700 1 --2- 172.21.15.88:0/299186565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1214100e80 0x7f1214104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.640+0000 7f12191b5700 1 --2- 172.21.15.88:0/299186565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12141004e0 0x7f12141008b0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.640+0000 7f12191b5700 1 -- 172.21.15.88:0/299186565 >> 172.21.15.88:0/299186565 conn(0x7f12140fbda0 msgr2=0x7f12140fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:22.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.640+0000 7f12191b5700 1 -- 172.21.15.88:0/299186565 shutdown_connections 2024-02-19T19:32:22.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.640+0000 7f12191b5700 1 -- 172.21.15.88:0/299186565 wait complete. 2024-02-19T19:32:22.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.641+0000 7f12191b5700 1 Processor -- start 2024-02-19T19:32:22.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.641+0000 7f12191b5700 1 -- start start 2024-02-19T19:32:22.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.641+0000 7f12191b5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12141004e0 0x7f1214195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.641+0000 7f12191b5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1214100e80 0x7f1214196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.641+0000 7f12191b5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1214196b10 con 0x7f1214100e80 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.641+0000 7f12191b5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121419a8a0 con 0x7f12141004e0 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.641+0000 7f121259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1214100e80 0x7f1214196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.641+0000 7f121259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1214100e80 0x7f1214196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40120/0 (socket says 172.21.15.88:40120) 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.641+0000 7f121259c700 1 -- 172.21.15.88:0/3522628942 learned_addr learned my addr 172.21.15.88:0/3522628942 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.641+0000 7f1212d9d700 1 --2- 172.21.15.88:0/3522628942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12141004e0 0x7f1214195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.642+0000 7f121259c700 1 -- 172.21.15.88:0/3522628942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12141004e0 msgr2=0x7f1214195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.642+0000 7f121259c700 1 --2- 172.21.15.88:0/3522628942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12141004e0 0x7f1214195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.642+0000 7f121259c700 1 -- 172.21.15.88:0/3522628942 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12080095f0 con 0x7f1214100e80 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.642+0000 7f121259c700 1 --2- 172.21.15.88:0/3522628942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1214100e80 0x7f1214196430 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f120400d900 tx=0x7f120400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.642+0000 7f11fbfff700 1 -- 172.21.15.88:0/3522628942 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1204009900 con 0x7f1214100e80 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.642+0000 7f11fbfff700 1 -- 172.21.15.88:0/3522628942 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1204010460 con 0x7f1214100e80 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.642+0000 7f12191b5700 1 -- 172.21.15.88:0/3522628942 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121419ab80 con 0x7f1214100e80 2024-02-19T19:32:22.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.642+0000 7f11fbfff700 1 -- 172.21.15.88:0/3522628942 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f120400f5f0 con 0x7f1214100e80 2024-02-19T19:32:22.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.642+0000 7f12191b5700 1 -- 172.21.15.88:0/3522628942 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121419b0d0 con 0x7f1214100e80 2024-02-19T19:32:22.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.643+0000 7f12191b5700 1 -- 172.21.15.88:0/3522628942 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12141087f0 con 0x7f1214100e80 2024-02-19T19:32:22.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.644+0000 7f11fbfff700 1 -- 172.21.15.88:0/3522628942 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f12040105d0 con 0x7f1214100e80 2024-02-19T19:32:22.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.644+0000 7f11fbfff700 1 --2- 172.21.15.88:0/3522628942 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11fc06d8a0 0x7f11fc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:22.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.644+0000 7f11fbfff700 1 -- 172.21.15.88:0/3522628942 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f120408d0e0 con 0x7f1214100e80 2024-02-19T19:32:22.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.644+0000 7f1212d9d700 1 --2- 172.21.15.88:0/3522628942 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11fc06d8a0 0x7f11fc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:22.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.644+0000 7f1212d9d700 1 --2- 172.21.15.88:0/3522628942 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11fc06d8a0 0x7f11fc06fd50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1208006010 tx=0x7f120801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:22.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.649+0000 7f11fbfff700 1 -- 172.21.15.88:0/3522628942 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1204056890 con 0x7f1214100e80 2024-02-19T19:32:22.833 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:22 smithi088 ceph-mon[88643]: from='client.35566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:22.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:22 smithi088 ceph-mon[88643]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:22.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:22 smithi088 ceph-mon[88643]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:22.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:22 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3831321443' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:32:22.834 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:22 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3268144579' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:32:22.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.926+0000 7f12191b5700 1 -- 172.21.15.88:0/3522628942 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f121419b2f0 con 0x7f1214100e80 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.927+0000 7f11fbfff700 1 -- 172.21.15.88:0/3522628942 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f1204059f50 con 0x7f1214100e80 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:32:22.930 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:32:22.931 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:32:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.929+0000 7f12191b5700 1 -- 172.21.15.88:0/3522628942 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11fc06d8a0 msgr2=0x7f11fc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.929+0000 7f12191b5700 1 --2- 172.21.15.88:0/3522628942 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11fc06d8a0 0x7f11fc06fd50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1208006010 tx=0x7f120801a040 comp rx=0 tx=0).stop 2024-02-19T19:32:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.929+0000 7f12191b5700 1 -- 172.21.15.88:0/3522628942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1214100e80 msgr2=0x7f1214196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:22.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.929+0000 7f12191b5700 1 --2- 172.21.15.88:0/3522628942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1214100e80 0x7f1214196430 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f120400d900 tx=0x7f120400dc10 comp rx=0 tx=0).stop 2024-02-19T19:32:22.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.929+0000 7f12191b5700 1 -- 172.21.15.88:0/3522628942 shutdown_connections 2024-02-19T19:32:22.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.929+0000 7f12191b5700 1 --2- 172.21.15.88:0/3522628942 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f11fc06d8a0 0x7f11fc06fd50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.929+0000 7f12191b5700 1 --2- 172.21.15.88:0/3522628942 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f12141004e0 0x7f1214195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.929+0000 7f12191b5700 1 --2- 172.21.15.88:0/3522628942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1214100e80 0x7f1214196430 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:22.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.929+0000 7f12191b5700 1 -- 172.21.15.88:0/3522628942 >> 172.21.15.88:0/3522628942 conn(0x7f12140fbda0 msgr2=0x7f12140fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:22.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.930+0000 7f12191b5700 1 -- 172.21.15.88:0/3522628942 shutdown_connections 2024-02-19T19:32:22.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:22.930+0000 7f12191b5700 1 -- 172.21.15.88:0/3522628942 wait complete. 2024-02-19T19:32:22.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:22 smithi132 ceph-mon[70676]: from='client.35566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:22.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:22 smithi132 ceph-mon[70676]: from='client.35570 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:22.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:22 smithi132 ceph-mon[70676]: from='client.35574 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:22.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:22 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3831321443' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:32:22.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:22 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3268144579' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:32:23.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:23 smithi132 ceph-mon[70676]: from='client.35586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:23.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:23 smithi132 ceph-mon[70676]: pgmap v865: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:23.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:23 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3522628942' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:32:24.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:23 smithi088 ceph-mon[88643]: from='client.35586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:24.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:23 smithi088 ceph-mon[88643]: pgmap v865: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:24.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:23 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3522628942' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:32:24.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:24 smithi132 ceph-mon[70676]: pgmap v866: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:25.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:24 smithi088 ceph-mon[88643]: pgmap v866: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:27.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:27 smithi132 ceph-mon[70676]: pgmap v867: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:28.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:27 smithi088 ceph-mon[88643]: pgmap v867: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:28.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:32:29.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:32:29.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:29 smithi132 ceph-mon[70676]: pgmap v868: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:30.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:29 smithi088 ceph-mon[88643]: pgmap v868: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:30.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:30 smithi132 ceph-mon[70676]: pgmap v869: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:31.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:30 smithi088 ceph-mon[88643]: pgmap v869: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:33.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:33 smithi132 ceph-mon[70676]: pgmap v870: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:34.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:33 smithi088 ceph-mon[88643]: pgmap v870: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:35.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:35 smithi132 ceph-mon[70676]: pgmap v871: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:36.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:35 smithi088 ceph-mon[88643]: pgmap v871: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:37.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:37 smithi132 ceph-mon[70676]: pgmap v872: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:38.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:37 smithi088 ceph-mon[88643]: pgmap v872: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:39.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:38 smithi088 ceph-mon[88643]: pgmap v873: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:39.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:38 smithi132 ceph-mon[70676]: pgmap v873: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:41.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:41 smithi132 ceph-mon[70676]: pgmap v874: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:42.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:41 smithi088 ceph-mon[88643]: pgmap v874: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:43.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:43 smithi132 ceph-mon[70676]: pgmap v875: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:43.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:32:44.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:43 smithi088 ceph-mon[88643]: pgmap v875: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:44.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:32:44.984 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:44 smithi088 ceph-mon[88643]: pgmap v876: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:44.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:44 smithi132 ceph-mon[70676]: pgmap v876: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:47.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:47 smithi132 ceph-mon[70676]: pgmap v877: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:48.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:47 smithi088 ceph-mon[88643]: pgmap v877: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:49.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:49 smithi132 ceph-mon[70676]: pgmap v878: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:50.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:49 smithi088 ceph-mon[88643]: pgmap v878: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:51.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:50 smithi088 ceph-mon[88643]: pgmap v879: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:51.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:50 smithi132 ceph-mon[70676]: pgmap v879: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:53.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.047+0000 7f9101ec4700 1 -- 172.21.15.88:0/2830495895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90fc0ffd60 msgr2=0x7f90fc103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.047+0000 7f9101ec4700 1 --2- 172.21.15.88:0/2830495895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90fc0ffd60 0x7f90fc103d30 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f90f0009910 tx=0x7f90f0009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:53.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.047+0000 7f9101ec4700 1 -- 172.21.15.88:0/2830495895 shutdown_connections 2024-02-19T19:32:53.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.047+0000 7f9101ec4700 1 --2- 172.21.15.88:0/2830495895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90fc0ffd60 0x7f90fc103d30 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.047+0000 7f9101ec4700 1 --2- 172.21.15.88:0/2830495895 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90fc0ff3c0 0x7f90fc0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.047+0000 7f9101ec4700 1 -- 172.21.15.88:0/2830495895 >> 172.21.15.88:0/2830495895 conn(0x7f90fc0fac80 msgr2=0x7f90fc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:53.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.047+0000 7f9101ec4700 1 -- 172.21.15.88:0/2830495895 shutdown_connections 2024-02-19T19:32:53.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.047+0000 7f9101ec4700 1 -- 172.21.15.88:0/2830495895 wait complete. 2024-02-19T19:32:53.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f9101ec4700 1 Processor -- start 2024-02-19T19:32:53.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f9101ec4700 1 -- start start 2024-02-19T19:32:53.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f9101ec4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90fc0ff3c0 0x7f90fc195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:53.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f9101ec4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90fc0ffd60 0x7f90fc1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:53.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f9101ec4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc196aa0 con 0x7f90fc0ff3c0 2024-02-19T19:32:53.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f9101ec4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90fc19a830 con 0x7f90fc0ffd60 2024-02-19T19:32:53.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f90fb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90fc0ff3c0 0x7f90fc195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:53.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f90fb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90fc0ff3c0 0x7f90fc195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35926/0 (socket says 172.21.15.88:35926) 2024-02-19T19:32:53.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f90fb7fe700 1 -- 172.21.15.88:0/2484940774 learned_addr learned my addr 172.21.15.88:0/2484940774 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f90faffd700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90fc0ffd60 0x7f90fc1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f90fb7fe700 1 -- 172.21.15.88:0/2484940774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90fc0ffd60 msgr2=0x7f90fc1963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f90fb7fe700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90fc0ffd60 0x7f90fc1963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f90fb7fe700 1 -- 172.21.15.88:0/2484940774 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f00095f0 con 0x7f90fc0ff3c0 2024-02-19T19:32:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.048+0000 7f90faffd700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90fc0ffd60 0x7f90fc1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:32:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.049+0000 7f90fb7fe700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90fc0ff3c0 0x7f90fc195e80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00ada0 tx=0x7f90ec00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.049+0000 7f90f8ff9700 1 -- 172.21.15.88:0/2484940774 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ec00f070 con 0x7f90fc0ff3c0 2024-02-19T19:32:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.049+0000 7f90f8ff9700 1 -- 172.21.15.88:0/2484940774 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90ec007bf0 con 0x7f90fc0ff3c0 2024-02-19T19:32:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.049+0000 7f90f8ff9700 1 -- 172.21.15.88:0/2484940774 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90ec005690 con 0x7f90fc0ff3c0 2024-02-19T19:32:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.049+0000 7f9101ec4700 1 -- 172.21.15.88:0/2484940774 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90fc19ab10 con 0x7f90fc0ff3c0 2024-02-19T19:32:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.049+0000 7f9101ec4700 1 -- 172.21.15.88:0/2484940774 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90fc19b060 con 0x7f90fc0ff3c0 2024-02-19T19:32:53.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.050+0000 7f9101ec4700 1 -- 172.21.15.88:0/2484940774 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90fc04f2a0 con 0x7f90fc0ff3c0 2024-02-19T19:32:53.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.051+0000 7f90f8ff9700 1 -- 172.21.15.88:0/2484940774 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f90ec005850 con 0x7f90fc0ff3c0 2024-02-19T19:32:53.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.051+0000 7f90f8ff9700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90e406d8a0 0x7f90e406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:53.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.051+0000 7f90f8ff9700 1 -- 172.21.15.88:0/2484940774 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f90ec08ce90 con 0x7f90fc0ff3c0 2024-02-19T19:32:53.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.051+0000 7f90faffd700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90e406d8a0 0x7f90e406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:53.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.052+0000 7f90faffd700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90e406d8a0 0x7f90e406fd50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90fc1974a0 tx=0x7f90f001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:53.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.056+0000 7f90f8ff9700 1 -- 172.21.15.88:0/2484940774 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f90ec0566c0 con 0x7f90fc0ff3c0 2024-02-19T19:32:53.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.273+0000 7f9101ec4700 1 -- 172.21.15.88:0/2484940774 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f90fc19b3e0 con 0x7f90e406d8a0 2024-02-19T19:32:53.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.275+0000 7f90f8ff9700 1 -- 172.21.15.88:0/2484940774 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f90fc19b3e0 con 0x7f90e406d8a0 2024-02-19T19:32:53.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.277+0000 7f9101ec4700 1 -- 172.21.15.88:0/2484940774 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90e406d8a0 msgr2=0x7f90e406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.277+0000 7f9101ec4700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90e406d8a0 0x7f90e406fd50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90fc1974a0 tx=0x7f90f001a040 comp rx=0 tx=0).stop 2024-02-19T19:32:53.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.277+0000 7f9101ec4700 1 -- 172.21.15.88:0/2484940774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90fc0ff3c0 msgr2=0x7f90fc195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.277+0000 7f9101ec4700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90fc0ff3c0 0x7f90fc195e80 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f90ec00ada0 tx=0x7f90ec00c330 comp rx=0 tx=0).stop 2024-02-19T19:32:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.277+0000 7f9101ec4700 1 -- 172.21.15.88:0/2484940774 shutdown_connections 2024-02-19T19:32:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.277+0000 7f9101ec4700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f90e406d8a0 0x7f90e406fd50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.277+0000 7f9101ec4700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90fc0ff3c0 0x7f90fc195e80 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.277+0000 7f9101ec4700 1 --2- 172.21.15.88:0/2484940774 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f90fc0ffd60 0x7f90fc1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.277+0000 7f9101ec4700 1 -- 172.21.15.88:0/2484940774 >> 172.21.15.88:0/2484940774 conn(0x7f90fc0fac80 msgr2=0x7f90fc068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.277+0000 7f9101ec4700 1 -- 172.21.15.88:0/2484940774 shutdown_connections 2024-02-19T19:32:53.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.277+0000 7f9101ec4700 1 -- 172.21.15.88:0/2484940774 wait complete. 2024-02-19T19:32:53.290 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:32:53.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.382+0000 7f305b3f4700 1 -- 172.21.15.88:0/1853883996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30540ff3c0 msgr2=0x7f30540ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.383+0000 7f305b3f4700 1 --2- 172.21.15.88:0/1853883996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30540ff3c0 0x7f30540ff790 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f3050009910 tx=0x7f3050009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:53.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.383+0000 7f305b3f4700 1 -- 172.21.15.88:0/1853883996 shutdown_connections 2024-02-19T19:32:53.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.383+0000 7f305b3f4700 1 --2- 172.21.15.88:0/1853883996 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30540ffd60 0x7f3054103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.383+0000 7f305b3f4700 1 --2- 172.21.15.88:0/1853883996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30540ff3c0 0x7f30540ff790 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.383+0000 7f305b3f4700 1 -- 172.21.15.88:0/1853883996 >> 172.21.15.88:0/1853883996 conn(0x7f30540fac80 msgr2=0x7f30540fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:53.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.383+0000 7f305b3f4700 1 -- 172.21.15.88:0/1853883996 shutdown_connections 2024-02-19T19:32:53.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.383+0000 7f305b3f4700 1 -- 172.21.15.88:0/1853883996 wait complete. 2024-02-19T19:32:53.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.383+0000 7f305b3f4700 1 Processor -- start 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.383+0000 7f305b3f4700 1 -- start start 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.383+0000 7f305b3f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30540ff3c0 0x7f3054196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305b3f4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30540ffd60 0x7f3054197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305b3f4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3054197850 con 0x7f30540ffd60 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305b3f4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3054197990 con 0x7f30540ff3c0 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305898f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30540ffd60 0x7f3054197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305898f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30540ffd60 0x7f3054197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35946/0 (socket says 172.21.15.88:35946) 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305898f700 1 -- 172.21.15.88:0/1237302253 learned_addr learned my addr 172.21.15.88:0/1237302253 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f3059190700 1 --2- 172.21.15.88:0/1237302253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30540ff3c0 0x7f3054196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305898f700 1 -- 172.21.15.88:0/1237302253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30540ff3c0 msgr2=0x7f3054196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305898f700 1 --2- 172.21.15.88:0/1237302253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30540ff3c0 0x7f3054196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305898f700 1 -- 172.21.15.88:0/1237302253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30500095f0 con 0x7f30540ffd60 2024-02-19T19:32:53.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305898f700 1 --2- 172.21.15.88:0/1237302253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30540ffd60 0x7f3054197230 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f304400c8f0 tx=0x7f304400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:53.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f304a7fc700 1 -- 172.21.15.88:0/1237302253 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30440078a0 con 0x7f30540ffd60 2024-02-19T19:32:53.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f304a7fc700 1 -- 172.21.15.88:0/1237302253 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f304400f450 con 0x7f30540ffd60 2024-02-19T19:32:53.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305b3f4700 1 -- 172.21.15.88:0/1237302253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f305419fd60 con 0x7f30540ffd60 2024-02-19T19:32:53.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f304a7fc700 1 -- 172.21.15.88:0/1237302253 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3044018610 con 0x7f30540ffd60 2024-02-19T19:32:53.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.384+0000 7f305b3f4700 1 -- 172.21.15.88:0/1237302253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30541a0220 con 0x7f30540ffd60 2024-02-19T19:32:53.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.385+0000 7f305b3f4700 1 -- 172.21.15.88:0/1237302253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30541076d0 con 0x7f30540ffd60 2024-02-19T19:32:53.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.386+0000 7f304a7fc700 1 -- 172.21.15.88:0/1237302253 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f304400f5c0 con 0x7f30540ffd60 2024-02-19T19:32:53.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.386+0000 7f304a7fc700 1 --2- 172.21.15.88:0/1237302253 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f304006d8a0 0x7f304006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:53.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.386+0000 7f304a7fc700 1 -- 172.21.15.88:0/1237302253 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f304408d3f0 con 0x7f30540ffd60 2024-02-19T19:32:53.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.386+0000 7f3059190700 1 --2- 172.21.15.88:0/1237302253 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f304006d8a0 0x7f304006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:53.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.387+0000 7f3059190700 1 --2- 172.21.15.88:0/1237302253 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f304006d8a0 0x7f304006fd50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3050006010 tx=0x7f305001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:53.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.391+0000 7f304a7fc700 1 -- 172.21.15.88:0/1237302253 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3044056c20 con 0x7f30540ffd60 2024-02-19T19:32:53.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.611+0000 7f305b3f4700 1 -- 172.21.15.88:0/1237302253 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3054061190 con 0x7f304006d8a0 2024-02-19T19:32:53.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.612+0000 7f304a7fc700 1 -- 172.21.15.88:0/1237302253 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3054061190 con 0x7f304006d8a0 2024-02-19T19:32:53.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.614+0000 7f305b3f4700 1 -- 172.21.15.88:0/1237302253 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f304006d8a0 msgr2=0x7f304006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.614+0000 7f305b3f4700 1 --2- 172.21.15.88:0/1237302253 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f304006d8a0 0x7f304006fd50 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f3050006010 tx=0x7f305001a040 comp rx=0 tx=0).stop 2024-02-19T19:32:53.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.615+0000 7f305b3f4700 1 -- 172.21.15.88:0/1237302253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30540ffd60 msgr2=0x7f3054197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.615+0000 7f305b3f4700 1 --2- 172.21.15.88:0/1237302253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30540ffd60 0x7f3054197230 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f304400c8f0 tx=0x7f304400cc00 comp rx=0 tx=0).stop 2024-02-19T19:32:53.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.615+0000 7f305b3f4700 1 -- 172.21.15.88:0/1237302253 shutdown_connections 2024-02-19T19:32:53.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.615+0000 7f305b3f4700 1 --2- 172.21.15.88:0/1237302253 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f304006d8a0 0x7f304006fd50 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.615+0000 7f305b3f4700 1 --2- 172.21.15.88:0/1237302253 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f30540ff3c0 0x7f3054196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.615+0000 7f305b3f4700 1 --2- 172.21.15.88:0/1237302253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30540ffd60 0x7f3054197230 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.615+0000 7f305b3f4700 1 -- 172.21.15.88:0/1237302253 >> 172.21.15.88:0/1237302253 conn(0x7f30540fac80 msgr2=0x7f30540fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:53.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.615+0000 7f305b3f4700 1 -- 172.21.15.88:0/1237302253 shutdown_connections 2024-02-19T19:32:53.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.615+0000 7f305b3f4700 1 -- 172.21.15.88:0/1237302253 wait complete. 2024-02-19T19:32:53.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.720+0000 7f299d3cf700 1 -- 172.21.15.88:0/1322110537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998100460 msgr2=0x7f2998100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.720+0000 7f299d3cf700 1 --2- 172.21.15.88:0/1322110537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998100460 0x7f2998100830 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2988009940 tx=0x7f2988009c50 comp rx=0 tx=0).stop 2024-02-19T19:32:53.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.720+0000 7f299d3cf700 1 -- 172.21.15.88:0/1322110537 shutdown_connections 2024-02-19T19:32:53.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.720+0000 7f299d3cf700 1 --2- 172.21.15.88:0/1322110537 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2998100d70 0x7f29981094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.720+0000 7f299d3cf700 1 --2- 172.21.15.88:0/1322110537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998100460 0x7f2998100830 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.720+0000 7f299d3cf700 1 -- 172.21.15.88:0/1322110537 >> 172.21.15.88:0/1322110537 conn(0x7f29980748b0 msgr2=0x7f2998074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:53.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.720+0000 7f299d3cf700 1 -- 172.21.15.88:0/1322110537 shutdown_connections 2024-02-19T19:32:53.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.720+0000 7f299d3cf700 1 -- 172.21.15.88:0/1322110537 wait complete. 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f299d3cf700 1 Processor -- start 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f299d3cf700 1 -- start start 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f299d3cf700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998100460 0x7f2998195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f299d3cf700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2998100d70 0x7f2998196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f299d3cf700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29981968d0 con 0x7f2998100460 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f299d3cf700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2998196a10 con 0x7f2998100d70 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f2996ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998100460 0x7f2998195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f2996ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998100460 0x7f2998195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35962/0 (socket says 172.21.15.88:35962) 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f2996ffd700 1 -- 172.21.15.88:0/331584612 learned_addr learned my addr 172.21.15.88:0/331584612 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f29967fc700 1 --2- 172.21.15.88:0/331584612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2998100d70 0x7f2998196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f2996ffd700 1 -- 172.21.15.88:0/331584612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2998100d70 msgr2=0x7f2998196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f2996ffd700 1 --2- 172.21.15.88:0/331584612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2998100d70 0x7f2998196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.721+0000 7f2996ffd700 1 -- 172.21.15.88:0/331584612 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29880095f0 con 0x7f2998100460 2024-02-19T19:32:53.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.722+0000 7f2996ffd700 1 --2- 172.21.15.88:0/331584612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998100460 0x7f2998195d40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f29880055f0 tx=0x7f2988005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:53.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.722+0000 7f297ffff700 1 -- 172.21.15.88:0/331584612 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f298801d070 con 0x7f2998100460 2024-02-19T19:32:53.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.722+0000 7f297ffff700 1 -- 172.21.15.88:0/331584612 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f298800f460 con 0x7f2998100460 2024-02-19T19:32:53.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.722+0000 7f299d3cf700 1 -- 172.21.15.88:0/331584612 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299819a860 con 0x7f2998100460 2024-02-19T19:32:53.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.722+0000 7f297ffff700 1 -- 172.21.15.88:0/331584612 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2988021640 con 0x7f2998100460 2024-02-19T19:32:53.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.722+0000 7f299d3cf700 1 -- 172.21.15.88:0/331584612 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299819adb0 con 0x7f2998100460 2024-02-19T19:32:53.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.723+0000 7f299d3cf700 1 -- 172.21.15.88:0/331584612 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2998107e40 con 0x7f2998100460 2024-02-19T19:32:53.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.723+0000 7f297ffff700 1 -- 172.21.15.88:0/331584612 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f298800fab0 con 0x7f2998100460 2024-02-19T19:32:53.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.724+0000 7f297ffff700 1 --2- 172.21.15.88:0/331584612 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f298006d8a0 0x7f298006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:53.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.724+0000 7f29967fc700 1 --2- 172.21.15.88:0/331584612 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f298006d8a0 0x7f298006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:53.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.724+0000 7f297ffff700 1 -- 172.21.15.88:0/331584612 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f298808e7a0 con 0x7f2998100460 2024-02-19T19:32:53.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.724+0000 7f29967fc700 1 --2- 172.21.15.88:0/331584612 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f298006d8a0 0x7f298006fd50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2998197360 tx=0x7f298c00a380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:53.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.728+0000 7f297ffff700 1 -- 172.21.15.88:0/331584612 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2988057f50 con 0x7f2998100460 2024-02-19T19:32:53.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.935+0000 7f299d3cf700 1 -- 172.21.15.88:0/331584612 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f29981971e0 con 0x7f298006d8a0 2024-02-19T19:32:53.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.946+0000 7f297ffff700 1 -- 172.21.15.88:0/331584612 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f298c00f9c0 con 0x7f298006d8a0 2024-02-19T19:32:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:32:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (30m) 6m ago 40m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:32:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (41m) 6m ago 41m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:32:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (22m) 6m ago 39m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:32:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (26m) 6m ago 40m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:32:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (26m) 6m ago 38m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:32:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (29m) 6m ago 39m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:32:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (22m) 6m ago 34m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:32:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (22m) 6m ago 34m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:32:53.949 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (21m) 6m ago 34m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:32:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (6m) 6m ago 34m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:32:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (31m) 6m ago 42m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:32:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (30m) 6m ago 38m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:32:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (28m) 6m ago 42m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:32:53.950 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (27m) 6m ago 38m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:32:53.992 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (30m) 6m ago 40m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:32:53.992 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (30m) 6m ago 38m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:32:53.992 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (25m) 6m ago 38m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:32:53.992 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (24m) 6m ago 37m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:32:53.992 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (24m) 6m ago 37m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:32:53.992 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (23m) 6m ago 36m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:32:53.992 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (23m) 6m ago 36m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:32:53.993 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (22m) 6m ago 35m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:32:53.993 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (30m) 6m ago 39m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:32:53.993 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:53 smithi132 ceph-mon[70676]: pgmap v880: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:53.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.948+0000 7f299d3cf700 1 -- 172.21.15.88:0/331584612 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f298006d8a0 msgr2=0x7f298006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.948+0000 7f299d3cf700 1 --2- 172.21.15.88:0/331584612 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f298006d8a0 0x7f298006fd50 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2998197360 tx=0x7f298c00a380 comp rx=0 tx=0).stop 2024-02-19T19:32:53.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.948+0000 7f299d3cf700 1 -- 172.21.15.88:0/331584612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998100460 msgr2=0x7f2998195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:53.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.948+0000 7f299d3cf700 1 --2- 172.21.15.88:0/331584612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998100460 0x7f2998195d40 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f29880055f0 tx=0x7f2988005620 comp rx=0 tx=0).stop 2024-02-19T19:32:53.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.948+0000 7f299d3cf700 1 -- 172.21.15.88:0/331584612 shutdown_connections 2024-02-19T19:32:53.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.948+0000 7f299d3cf700 1 --2- 172.21.15.88:0/331584612 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f298006d8a0 0x7f298006fd50 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.948+0000 7f299d3cf700 1 --2- 172.21.15.88:0/331584612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2998100460 0x7f2998195d40 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.948+0000 7f299d3cf700 1 --2- 172.21.15.88:0/331584612 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2998100d70 0x7f2998196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:53.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.948+0000 7f299d3cf700 1 -- 172.21.15.88:0/331584612 >> 172.21.15.88:0/331584612 conn(0x7f29980748b0 msgr2=0x7f29980ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:53.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.948+0000 7f299d3cf700 1 -- 172.21.15.88:0/331584612 shutdown_connections 2024-02-19T19:32:53.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:53.948+0000 7f299d3cf700 1 -- 172.21.15.88:0/331584612 wait complete. 2024-02-19T19:32:54.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.052+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/1625261889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc80ffd60 msgr2=0x7f4cc8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.052+0000 7f4ccdc3c700 1 --2- 172.21.15.88:0/1625261889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc80ffd60 0x7f4cc8103d30 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009910 tx=0x7f4cbc009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.052+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/1625261889 shutdown_connections 2024-02-19T19:32:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.052+0000 7f4ccdc3c700 1 --2- 172.21.15.88:0/1625261889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc80ffd60 0x7f4cc8103d30 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.052+0000 7f4ccdc3c700 1 --2- 172.21.15.88:0/1625261889 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc80ff3c0 0x7f4cc80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.052+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/1625261889 >> 172.21.15.88:0/1625261889 conn(0x7f4cc80fac80 msgr2=0x7f4cc80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:54.055 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:53 smithi088 ceph-mon[88643]: pgmap v880: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.052+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/1625261889 shutdown_connections 2024-02-19T19:32:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.052+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/1625261889 wait complete. 2024-02-19T19:32:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.052+0000 7f4ccdc3c700 1 Processor -- start 2024-02-19T19:32:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4ccdc3c700 1 -- start start 2024-02-19T19:32:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4ccdc3c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc80ff3c0 0x7f4cc8196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4ccdc3c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc80ffd60 0x7f4cc81972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4ccdc3c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc81978c0 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4ccdc3c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc8197a30 con 0x7f4cc80ffd60 2024-02-19T19:32:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4cc77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc80ff3c0 0x7f4cc8196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:54.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4cc77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc80ff3c0 0x7f4cc8196d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35978/0 (socket says 172.21.15.88:35978) 2024-02-19T19:32:54.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4cc77fe700 1 -- 172.21.15.88:0/3148821229 learned_addr learned my addr 172.21.15.88:0/3148821229 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:54.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4cc6ffd700 1 --2- 172.21.15.88:0/3148821229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc80ffd60 0x7f4cc81972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:54.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4cc77fe700 1 -- 172.21.15.88:0/3148821229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc80ffd60 msgr2=0x7f4cc81972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:54.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4cc77fe700 1 --2- 172.21.15.88:0/3148821229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc80ffd60 0x7f4cc81972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.053+0000 7f4cc77fe700 1 -- 172.21.15.88:0/3148821229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cbc0095f0 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.054+0000 7f4cc77fe700 1 --2- 172.21.15.88:0/3148821229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc80ff3c0 0x7f4cc8196d60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800d900 tx=0x7f4cb800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:54.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.054+0000 7f4cc4ff9700 1 -- 172.21.15.88:0/3148821229 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cb80041f0 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.054+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/3148821229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc819fdd0 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.054+0000 7f4cc4ff9700 1 -- 172.21.15.88:0/3148821229 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cb8004cf0 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.054+0000 7f4cc4ff9700 1 -- 172.21.15.88:0/3148821229 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cb800b750 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.054+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/3148821229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc81a0290 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.056+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/3148821229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc8107730 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.056+0000 7f4cc4ff9700 1 -- 172.21.15.88:0/3148821229 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4cb8004a40 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.056+0000 7f4cc4ff9700 1 --2- 172.21.15.88:0/3148821229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4cb006d6a0 0x7f4cb006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:54.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.056+0000 7f4cc4ff9700 1 -- 172.21.15.88:0/3148821229 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4cb808d440 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.056+0000 7f4cc6ffd700 1 --2- 172.21.15.88:0/3148821229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4cb006d6a0 0x7f4cb006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:54.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.057+0000 7f4cc6ffd700 1 --2- 172.21.15.88:0/3148821229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4cb006d6a0 0x7f4cb006fb50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006010 tx=0x7f4cbc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:54.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.061+0000 7f4cc4ff9700 1 -- 172.21.15.88:0/3148821229 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4cb8056bf0 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.339+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/3148821229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4cc80693e0 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.339+0000 7f4cc4ff9700 1 -- 172.21.15.88:0/3148821229 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4cb805a2b0 con 0x7f4cc80ff3c0 2024-02-19T19:32:54.342 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:32:54.342 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:32:54.342 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:32:54.342 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:32:54.342 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:32:54.342 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:32:54.342 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:32:54.343 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:32:54.343 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:32:54.343 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:32:54.343 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:32:54.343 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:32:54.343 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:32:54.343 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:32:54.343 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:32:54.343 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:32:54.343 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:32:54.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.341+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/3148821229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4cb006d6a0 msgr2=0x7f4cb006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:54.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.341+0000 7f4ccdc3c700 1 --2- 172.21.15.88:0/3148821229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4cb006d6a0 0x7f4cb006fb50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc006010 tx=0x7f4cbc01a040 comp rx=0 tx=0).stop 2024-02-19T19:32:54.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.341+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/3148821229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc80ff3c0 msgr2=0x7f4cc8196d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:54.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.341+0000 7f4ccdc3c700 1 --2- 172.21.15.88:0/3148821229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc80ff3c0 0x7f4cc8196d60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4cb800d900 tx=0x7f4cb800dcc0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.341+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/3148821229 shutdown_connections 2024-02-19T19:32:54.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.341+0000 7f4ccdc3c700 1 --2- 172.21.15.88:0/3148821229 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4cb006d6a0 0x7f4cb006fb50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.341+0000 7f4ccdc3c700 1 --2- 172.21.15.88:0/3148821229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cc80ff3c0 0x7f4cc8196d60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.341+0000 7f4ccdc3c700 1 --2- 172.21.15.88:0/3148821229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4cc80ffd60 0x7f4cc81972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.341+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/3148821229 >> 172.21.15.88:0/3148821229 conn(0x7f4cc80fac80 msgr2=0x7f4cc80fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:54.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.342+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/3148821229 shutdown_connections 2024-02-19T19:32:54.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.342+0000 7f4ccdc3c700 1 -- 172.21.15.88:0/3148821229 wait complete. 2024-02-19T19:32:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.445+0000 7f9931517700 1 -- 172.21.15.88:0/1282516235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c101d30 msgr2=0x7f992c10a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.445+0000 7f9931517700 1 --2- 172.21.15.88:0/1282516235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c101d30 0x7f992c10a230 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9914009910 tx=0x7f9914009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.445+0000 7f9931517700 1 -- 172.21.15.88:0/1282516235 shutdown_connections 2024-02-19T19:32:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.445+0000 7f9931517700 1 --2- 172.21.15.88:0/1282516235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c101d30 0x7f992c10a230 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.445+0000 7f9931517700 1 --2- 172.21.15.88:0/1282516235 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c101420 0x7f992c1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.445+0000 7f9931517700 1 -- 172.21.15.88:0/1282516235 >> 172.21.15.88:0/1282516235 conn(0x7f992c0fac60 msgr2=0x7f992c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.445+0000 7f9931517700 1 -- 172.21.15.88:0/1282516235 shutdown_connections 2024-02-19T19:32:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.445+0000 7f9931517700 1 -- 172.21.15.88:0/1282516235 wait complete. 2024-02-19T19:32:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.445+0000 7f9931517700 1 Processor -- start 2024-02-19T19:32:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f9931517700 1 -- start start 2024-02-19T19:32:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f9931517700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c101420 0x7f992c0ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f9931517700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c101d30 0x7f992c0ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f9931517700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c0ffb00 con 0x7f992c101420 2024-02-19T19:32:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f9931517700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c0ffc40 con 0x7f992c101d30 2024-02-19T19:32:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f992affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c101420 0x7f992c0ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f992affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c101420 0x7f992c0ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35990/0 (socket says 172.21.15.88:35990) 2024-02-19T19:32:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f992affd700 1 -- 172.21.15.88:0/3710499034 learned_addr learned my addr 172.21.15.88:0/3710499034 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:54.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f99227fc700 1 --2- 172.21.15.88:0/3710499034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c101d30 0x7f992c0ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f992affd700 1 -- 172.21.15.88:0/3710499034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c101d30 msgr2=0x7f992c0ff5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f992affd700 1 --2- 172.21.15.88:0/3710499034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c101d30 0x7f992c0ff5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f992affd700 1 -- 172.21.15.88:0/3710499034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99140095f0 con 0x7f992c101420 2024-02-19T19:32:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.446+0000 7f992affd700 1 --2- 172.21.15.88:0/3710499034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c101420 0x7f992c0ff080 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f991c00b4b0 tx=0x7f991c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.447+0000 7f9928ff9700 1 -- 172.21.15.88:0/3710499034 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f991c011800 con 0x7f992c101420 2024-02-19T19:32:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.447+0000 7f9928ff9700 1 -- 172.21.15.88:0/3710499034 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f991c011e20 con 0x7f992c101420 2024-02-19T19:32:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.447+0000 7f9928ff9700 1 -- 172.21.15.88:0/3710499034 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f991c00f2e0 con 0x7f992c101420 2024-02-19T19:32:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.447+0000 7f9931517700 1 -- 172.21.15.88:0/3710499034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992c1a21d0 con 0x7f992c101420 2024-02-19T19:32:54.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.447+0000 7f9931517700 1 -- 172.21.15.88:0/3710499034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992c1a25e0 con 0x7f992c101420 2024-02-19T19:32:54.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.448+0000 7f9931517700 1 -- 172.21.15.88:0/3710499034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992c1078a0 con 0x7f992c101420 2024-02-19T19:32:54.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.448+0000 7f9928ff9700 1 -- 172.21.15.88:0/3710499034 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f991c010380 con 0x7f992c101420 2024-02-19T19:32:54.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.448+0000 7f9928ff9700 1 --2- 172.21.15.88:0/3710499034 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f991806d8a0 0x7f991806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:54.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.449+0000 7f9928ff9700 1 -- 172.21.15.88:0/3710499034 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f991c08d0f0 con 0x7f992c101420 2024-02-19T19:32:54.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.449+0000 7f99227fc700 1 --2- 172.21.15.88:0/3710499034 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f991806d8a0 0x7f991806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:54.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.449+0000 7f99227fc700 1 --2- 172.21.15.88:0/3710499034 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f991806d8a0 0x7f991806fd50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f99140059d0 tx=0x7f991401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:54.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.453+0000 7f9928ff9700 1 -- 172.21.15.88:0/3710499034 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f991c0568a0 con 0x7f992c101420 2024-02-19T19:32:54.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.700+0000 7f9931517700 1 -- 172.21.15.88:0/3710499034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f992c100650 con 0x7f992c101420 2024-02-19T19:32:54.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.700+0000 7f9928ff9700 1 -- 172.21.15.88:0/3710499034 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f991c056070 con 0x7f992c101420 2024-02-19T19:32:54.703 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:32:54.703 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:32:54.703 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:32:54.703 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:32:54.703 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:32:54.703 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:32:54.703 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:32:54.703 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:32:54.703 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:32:54.704 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:32:54.705 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:32:54.706 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:32:54.706 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:32:54.706 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:32:54.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.702+0000 7f9931517700 1 -- 172.21.15.88:0/3710499034 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f991806d8a0 msgr2=0x7f991806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:54.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.702+0000 7f9931517700 1 --2- 172.21.15.88:0/3710499034 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f991806d8a0 0x7f991806fd50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f99140059d0 tx=0x7f991401a040 comp rx=0 tx=0).stop 2024-02-19T19:32:54.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.702+0000 7f9931517700 1 -- 172.21.15.88:0/3710499034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c101420 msgr2=0x7f992c0ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:54.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.702+0000 7f9931517700 1 --2- 172.21.15.88:0/3710499034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c101420 0x7f992c0ff080 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f991c00b4b0 tx=0x7f991c00b870 comp rx=0 tx=0).stop 2024-02-19T19:32:54.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.702+0000 7f9931517700 1 -- 172.21.15.88:0/3710499034 shutdown_connections 2024-02-19T19:32:54.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.702+0000 7f9931517700 1 --2- 172.21.15.88:0/3710499034 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f991806d8a0 0x7f991806fd50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.702+0000 7f9931517700 1 --2- 172.21.15.88:0/3710499034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c101420 0x7f992c0ff080 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.702+0000 7f9931517700 1 --2- 172.21.15.88:0/3710499034 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f992c101d30 0x7f992c0ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.702+0000 7f9931517700 1 -- 172.21.15.88:0/3710499034 >> 172.21.15.88:0/3710499034 conn(0x7f992c0fac60 msgr2=0x7f992c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:54.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.703+0000 7f9931517700 1 -- 172.21.15.88:0/3710499034 shutdown_connections 2024-02-19T19:32:54.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.703+0000 7f9931517700 1 -- 172.21.15.88:0/3710499034 wait complete. 2024-02-19T19:32:54.707 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:32:54.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.808+0000 7fc76ed8f700 1 -- 172.21.15.88:0/3981712923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc768101d30 msgr2=0x7fc76810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:54.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.808+0000 7fc76ed8f700 1 --2- 172.21.15.88:0/3981712923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc768101d30 0x7fc76810a230 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fc754009910 tx=0x7fc754009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:54.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.808+0000 7fc76ed8f700 1 -- 172.21.15.88:0/3981712923 shutdown_connections 2024-02-19T19:32:54.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.808+0000 7fc76ed8f700 1 --2- 172.21.15.88:0/3981712923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc768101d30 0x7fc76810a230 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.808+0000 7fc76ed8f700 1 --2- 172.21.15.88:0/3981712923 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc768101420 0x7fc7681017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.808+0000 7fc76ed8f700 1 -- 172.21.15.88:0/3981712923 >> 172.21.15.88:0/3981712923 conn(0x7fc7680fac80 msgr2=0x7fc7680fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:54.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.808+0000 7fc76ed8f700 1 -- 172.21.15.88:0/3981712923 shutdown_connections 2024-02-19T19:32:54.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.809+0000 7fc76ed8f700 1 -- 172.21.15.88:0/3981712923 wait complete. 2024-02-19T19:32:54.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.809+0000 7fc76ed8f700 1 Processor -- start 2024-02-19T19:32:54.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.809+0000 7fc76ed8f700 1 -- start start 2024-02-19T19:32:54.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.809+0000 7fc76ed8f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc768101420 0x7fc7680ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:54.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.809+0000 7fc76ed8f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc768101d30 0x7fc7680ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:54.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.809+0000 7fc76ed8f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680ffb00 con 0x7fc768101d30 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.809+0000 7fc76ed8f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7680ffc40 con 0x7fc768101420 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.809+0000 7fc767fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc768101d30 0x7fc7680ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.809+0000 7fc767fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc768101d30 0x7fc7680ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36006/0 (socket says 172.21.15.88:36006) 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.809+0000 7fc767fff700 1 -- 172.21.15.88:0/2204821366 learned_addr learned my addr 172.21.15.88:0/2204821366 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.809+0000 7fc76cb2b700 1 --2- 172.21.15.88:0/2204821366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc768101420 0x7fc7680ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.810+0000 7fc767fff700 1 -- 172.21.15.88:0/2204821366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc768101420 msgr2=0x7fc7680ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.810+0000 7fc767fff700 1 --2- 172.21.15.88:0/2204821366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc768101420 0x7fc7680ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.810+0000 7fc767fff700 1 -- 172.21.15.88:0/2204821366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7540095f0 con 0x7fc768101d30 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.810+0000 7fc767fff700 1 --2- 172.21.15.88:0/2204821366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc768101d30 0x7fc7680ff5c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc7540092e0 tx=0x7fc75400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.810+0000 7fc765ffb700 1 -- 172.21.15.88:0/2204821366 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc75401d070 con 0x7fc768101d30 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.810+0000 7fc765ffb700 1 -- 172.21.15.88:0/2204821366 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc754005e00 con 0x7fc768101d30 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.810+0000 7fc765ffb700 1 -- 172.21.15.88:0/2204821366 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc754017400 con 0x7fc768101d30 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.810+0000 7fc76ed8f700 1 -- 172.21.15.88:0/2204821366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7680ffec0 con 0x7fc768101d30 2024-02-19T19:32:54.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.810+0000 7fc76ed8f700 1 -- 172.21.15.88:0/2204821366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7681a2310 con 0x7fc768101d30 2024-02-19T19:32:54.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.811+0000 7fc76ed8f700 1 -- 172.21.15.88:0/2204821366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7681078a0 con 0x7fc768101d30 2024-02-19T19:32:54.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.812+0000 7fc765ffb700 1 -- 172.21.15.88:0/2204821366 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc75400f6f0 con 0x7fc768101d30 2024-02-19T19:32:54.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.812+0000 7fc765ffb700 1 --2- 172.21.15.88:0/2204821366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc75806d8a0 0x7fc75806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:54.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.812+0000 7fc76cb2b700 1 --2- 172.21.15.88:0/2204821366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc75806d8a0 0x7fc75806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:54.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.812+0000 7fc765ffb700 1 -- 172.21.15.88:0/2204821366 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc75408e990 con 0x7fc768101d30 2024-02-19T19:32:54.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.813+0000 7fc76cb2b700 1 --2- 172.21.15.88:0/2204821366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc75806d8a0 0x7fc75806fd50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc75c005950 tx=0x7fc75c00b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:54.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:54.817+0000 7fc765ffb700 1 -- 172.21.15.88:0/2204821366 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc754058140 con 0x7fc768101d30 2024-02-19T19:32:54.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:54 smithi132 ceph-mon[70676]: from='client.35594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:54.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:54 smithi132 ceph-mon[70676]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:54.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:54 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3148821229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:32:54.987 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:54 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3710499034' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:32:55.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.030+0000 7fc76ed8f700 1 -- 172.21.15.88:0/2204821366 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc768061190 con 0x7fc75806d8a0 2024-02-19T19:32:55.032 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:54 smithi088 ceph-mon[88643]: from='client.35594 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:55.032 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:54 smithi088 ceph-mon[88643]: from='client.35598 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:55.033 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:54 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3148821229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:32:55.033 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:54 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3710499034' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:32:55.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.031+0000 7fc765ffb700 1 -- 172.21.15.88:0/2204821366 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc768061190 con 0x7fc75806d8a0 2024-02-19T19:32:55.033 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:32:55.034 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:32:55.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.033+0000 7fc76ed8f700 1 -- 172.21.15.88:0/2204821366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc75806d8a0 msgr2=0x7fc75806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:55.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.033+0000 7fc76ed8f700 1 --2- 172.21.15.88:0/2204821366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc75806d8a0 0x7fc75806fd50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc75c005950 tx=0x7fc75c00b410 comp rx=0 tx=0).stop 2024-02-19T19:32:55.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.033+0000 7fc76ed8f700 1 -- 172.21.15.88:0/2204821366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc768101d30 msgr2=0x7fc7680ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:55.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.033+0000 7fc76ed8f700 1 --2- 172.21.15.88:0/2204821366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc768101d30 0x7fc7680ff5c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc7540092e0 tx=0x7fc75400bd70 comp rx=0 tx=0).stop 2024-02-19T19:32:55.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.033+0000 7fc76ed8f700 1 -- 172.21.15.88:0/2204821366 shutdown_connections 2024-02-19T19:32:55.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.033+0000 7fc76ed8f700 1 --2- 172.21.15.88:0/2204821366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc75806d8a0 0x7fc75806fd50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:55.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.033+0000 7fc76ed8f700 1 --2- 172.21.15.88:0/2204821366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc768101420 0x7fc7680ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:55.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.033+0000 7fc76ed8f700 1 --2- 172.21.15.88:0/2204821366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc768101d30 0x7fc7680ff5c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:55.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.033+0000 7fc76ed8f700 1 -- 172.21.15.88:0/2204821366 >> 172.21.15.88:0/2204821366 conn(0x7fc7680fac80 msgr2=0x7fc7681049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:55.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.033+0000 7fc76ed8f700 1 -- 172.21.15.88:0/2204821366 shutdown_connections 2024-02-19T19:32:55.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.033+0000 7fc76ed8f700 1 -- 172.21.15.88:0/2204821366 wait complete. 2024-02-19T19:32:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.137+0000 7fe8367fb700 1 -- 172.21.15.88:0/2038027004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8300ffc60 msgr2=0x7fe83010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.137+0000 7fe8367fb700 1 --2- 172.21.15.88:0/2038027004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8300ffc60 0x7fe83010a6d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe820009910 tx=0x7fe820009c20 comp rx=0 tx=0).stop 2024-02-19T19:32:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.138+0000 7fe8367fb700 1 -- 172.21.15.88:0/2038027004 shutdown_connections 2024-02-19T19:32:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.138+0000 7fe8367fb700 1 --2- 172.21.15.88:0/2038027004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8300ffc60 0x7fe83010a6d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.138+0000 7fe8367fb700 1 --2- 172.21.15.88:0/2038027004 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8300ff350 0x7fe8300ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:55.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.138+0000 7fe8367fb700 1 -- 172.21.15.88:0/2038027004 >> 172.21.15.88:0/2038027004 conn(0x7fe8300748b0 msgr2=0x7fe830074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.138+0000 7fe8367fb700 1 -- 172.21.15.88:0/2038027004 shutdown_connections 2024-02-19T19:32:55.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.138+0000 7fe8367fb700 1 -- 172.21.15.88:0/2038027004 wait complete. 2024-02-19T19:32:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.138+0000 7fe8367fb700 1 Processor -- start 2024-02-19T19:32:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.138+0000 7fe8367fb700 1 -- start start 2024-02-19T19:32:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe8367fb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8300ff350 0x7fe830195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe8367fb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8300ffc60 0x7fe830196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe8367fb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe830196ae0 con 0x7fe8300ff350 2024-02-19T19:32:55.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe8367fb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83019a870 con 0x7fe8300ffc60 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8300ff350 0x7fe830195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8300ff350 0x7fe830195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36030/0 (socket says 172.21.15.88:36030) 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82ffff700 1 -- 172.21.15.88:0/3099989391 learned_addr learned my addr 172.21.15.88:0/3099989391 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82f7fe700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8300ffc60 0x7fe830196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82ffff700 1 -- 172.21.15.88:0/3099989391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8300ffc60 msgr2=0x7fe830196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82ffff700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8300ffc60 0x7fe830196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82ffff700 1 -- 172.21.15.88:0/3099989391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8200095f0 con 0x7fe8300ff350 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82f7fe700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8300ffc60 0x7fe830196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82ffff700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8300ff350 0x7fe830195ec0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe82400d8d0 tx=0x7fe82400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82d7fa700 1 -- 172.21.15.88:0/3099989391 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe824009940 con 0x7fe8300ff350 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe8367fb700 1 -- 172.21.15.88:0/3099989391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83019ab50 con 0x7fe8300ff350 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82d7fa700 1 -- 172.21.15.88:0/3099989391 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe824010460 con 0x7fe8300ff350 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe82d7fa700 1 -- 172.21.15.88:0/3099989391 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82400f5f0 con 0x7fe8300ff350 2024-02-19T19:32:55.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.139+0000 7fe8367fb700 1 -- 172.21.15.88:0/3099989391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83019b0a0 con 0x7fe8300ff350 2024-02-19T19:32:55.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.140+0000 7fe8367fb700 1 -- 172.21.15.88:0/3099989391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe830107dd0 con 0x7fe8300ff350 2024-02-19T19:32:55.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.141+0000 7fe82d7fa700 1 -- 172.21.15.88:0/3099989391 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe8240105d0 con 0x7fe8300ff350 2024-02-19T19:32:55.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.141+0000 7fe82d7fa700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe81806d8a0 0x7fe81806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:32:55.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.141+0000 7fe82f7fe700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe81806d8a0 0x7fe81806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:32:55.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.141+0000 7fe82d7fa700 1 -- 172.21.15.88:0/3099989391 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe82408d540 con 0x7fe8300ff350 2024-02-19T19:32:55.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.142+0000 7fe82f7fe700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe81806d8a0 0x7fe81806fd50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe820005b40 tx=0x7fe82001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:32:55.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.146+0000 7fe82d7fa700 1 -- 172.21.15.88:0/3099989391 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe824056cf0 con 0x7fe8300ff350 2024-02-19T19:32:55.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.422+0000 7fe8367fb700 1 -- 172.21.15.88:0/3099989391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe8301972f0 con 0x7fe8300ff350 2024-02-19T19:32:55.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.423+0000 7fe82d7fa700 1 -- 172.21.15.88:0/3099989391 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fe82405a3b0 con 0x7fe8300ff350 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:32:55.426 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:32:55.427 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:32:55.427 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:32:55.427 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:32:55.427 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:32:55.427 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:32:55.427 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:32:55.427 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:32:55.427 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:32:55.427 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:32:55.427 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:32:55.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.425+0000 7fe8367fb700 1 -- 172.21.15.88:0/3099989391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe81806d8a0 msgr2=0x7fe81806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:55.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.425+0000 7fe8367fb700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe81806d8a0 0x7fe81806fd50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe820005b40 tx=0x7fe82001a040 comp rx=0 tx=0).stop 2024-02-19T19:32:55.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.425+0000 7fe8367fb700 1 -- 172.21.15.88:0/3099989391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8300ff350 msgr2=0x7fe830195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:32:55.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.425+0000 7fe8367fb700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8300ff350 0x7fe830195ec0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fe82400d8d0 tx=0x7fe82400dc90 comp rx=0 tx=0).stop 2024-02-19T19:32:55.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.425+0000 7fe8367fb700 1 -- 172.21.15.88:0/3099989391 shutdown_connections 2024-02-19T19:32:55.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.425+0000 7fe8367fb700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe81806d8a0 0x7fe81806fd50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:55.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.425+0000 7fe8367fb700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8300ff350 0x7fe830195ec0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:55.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.425+0000 7fe8367fb700 1 --2- 172.21.15.88:0/3099989391 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe8300ffc60 0x7fe830196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:32:55.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.425+0000 7fe8367fb700 1 -- 172.21.15.88:0/3099989391 >> 172.21.15.88:0/3099989391 conn(0x7fe8300748b0 msgr2=0x7fe830104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:32:55.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.425+0000 7fe8367fb700 1 -- 172.21.15.88:0/3099989391 shutdown_connections 2024-02-19T19:32:55.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:32:55.425+0000 7fe8367fb700 1 -- 172.21.15.88:0/3099989391 wait complete. 2024-02-19T19:32:55.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:55 smithi132 ceph-mon[70676]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:55.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:55 smithi132 ceph-mon[70676]: pgmap v881: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:55.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:55 smithi132 ceph-mon[70676]: from='client.35614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:55.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:55 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3099989391' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:32:56.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:55 smithi088 ceph-mon[88643]: from='client.35602 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:56.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:55 smithi088 ceph-mon[88643]: pgmap v881: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:56.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:55 smithi088 ceph-mon[88643]: from='client.35614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:32:56.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:55 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3099989391' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:32:56.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:56 smithi132 ceph-mon[70676]: pgmap v882: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:57.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:56 smithi088 ceph-mon[88643]: pgmap v882: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:58.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:32:58.735 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:32:59.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:32:59 smithi088 ceph-mon[88643]: pgmap v883: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:32:59.735 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:32:59 smithi132 ceph-mon[70676]: pgmap v883: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:01.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:01 smithi132 ceph-mon[70676]: pgmap v884: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:02.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:01 smithi088 ceph-mon[88643]: pgmap v884: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:03.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:02 smithi088 ceph-mon[88643]: pgmap v885: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:03.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:02 smithi132 ceph-mon[70676]: pgmap v885: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:05.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:05 smithi132 ceph-mon[70676]: pgmap v886: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:06.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:05 smithi088 ceph-mon[88643]: pgmap v886: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:06.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:06 smithi132 ceph-mon[70676]: pgmap v887: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:07.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:06 smithi088 ceph-mon[88643]: pgmap v887: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:09.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:09 smithi132 ceph-mon[70676]: pgmap v888: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:10.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:09 smithi088 ceph-mon[88643]: pgmap v888: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:10.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:10 smithi132 ceph-mon[70676]: pgmap v889: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:11.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:10 smithi088 ceph-mon[88643]: pgmap v889: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:13.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:12 smithi088 ceph-mon[88643]: pgmap v890: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:13.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:12 smithi132 ceph-mon[70676]: pgmap v890: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:14.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:33:14.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:33:14.984 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:14 smithi088 ceph-mon[88643]: pgmap v891: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:15.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:14 smithi132 ceph-mon[70676]: pgmap v891: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:17.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:16 smithi088 ceph-mon[88643]: pgmap v892: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:17.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:16 smithi132 ceph-mon[70676]: pgmap v892: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:19.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:18 smithi088 ceph-mon[88643]: pgmap v893: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:19.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:18 smithi132 ceph-mon[70676]: pgmap v893: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:20.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:20 smithi132 ceph-mon[70676]: pgmap v894: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:21.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:20 smithi088 ceph-mon[88643]: pgmap v894: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:23.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:22 smithi088 ceph-mon[88643]: pgmap v895: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:22 smithi132 ceph-mon[70676]: pgmap v895: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:25.037 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:24 smithi132 ceph-mon[70676]: pgmap v896: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:25.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:24 smithi088 ceph-mon[88643]: pgmap v896: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:25.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.542+0000 7fc11d7d6700 1 -- 172.21.15.88:0/710856591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1181038b0 msgr2=0x7fc118103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:25.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.542+0000 7fc11d7d6700 1 --2- 172.21.15.88:0/710856591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1181038b0 0x7fc118103c80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fc10c009910 tx=0x7fc10c009c20 comp rx=0 tx=0).stop 2024-02-19T19:33:25.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.542+0000 7fc11d7d6700 1 -- 172.21.15.88:0/710856591 shutdown_connections 2024-02-19T19:33:25.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.542+0000 7fc11d7d6700 1 --2- 172.21.15.88:0/710856591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1180ff350 0x7fc1180ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:25.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.542+0000 7fc11d7d6700 1 --2- 172.21.15.88:0/710856591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1181038b0 0x7fc118103c80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:25.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.542+0000 7fc11d7d6700 1 -- 172.21.15.88:0/710856591 >> 172.21.15.88:0/710856591 conn(0x7fc1180748b0 msgr2=0x7fc118074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:25.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.542+0000 7fc11d7d6700 1 -- 172.21.15.88:0/710856591 shutdown_connections 2024-02-19T19:33:25.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.542+0000 7fc11d7d6700 1 -- 172.21.15.88:0/710856591 wait complete. 2024-02-19T19:33:25.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.543+0000 7fc11d7d6700 1 Processor -- start 2024-02-19T19:33:25.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.543+0000 7fc11d7d6700 1 -- start start 2024-02-19T19:33:25.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.543+0000 7fc11d7d6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1180ff350 0x7fc118195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:25.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.543+0000 7fc11d7d6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1181038b0 0x7fc118196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:25.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.543+0000 7fc11d7d6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1181969d0 con 0x7fc1180ff350 2024-02-19T19:33:25.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.543+0000 7fc11d7d6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc118196b10 con 0x7fc1181038b0 2024-02-19T19:33:25.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.543+0000 7fc116ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1180ff350 0x7fc118195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:25.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.543+0000 7fc116ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1180ff350 0x7fc118195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35710/0 (socket says 172.21.15.88:35710) 2024-02-19T19:33:25.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.543+0000 7fc116ffd700 1 -- 172.21.15.88:0/1549221842 learned_addr learned my addr 172.21.15.88:0/1549221842 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:25.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.544+0000 7fc1167fc700 1 --2- 172.21.15.88:0/1549221842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1181038b0 0x7fc118196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:25.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.544+0000 7fc116ffd700 1 -- 172.21.15.88:0/1549221842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1181038b0 msgr2=0x7fc118196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:25.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.544+0000 7fc116ffd700 1 --2- 172.21.15.88:0/1549221842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1181038b0 0x7fc118196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:25.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.544+0000 7fc116ffd700 1 -- 172.21.15.88:0/1549221842 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc10c0095f0 con 0x7fc1180ff350 2024-02-19T19:33:25.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.544+0000 7fc116ffd700 1 --2- 172.21.15.88:0/1549221842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1180ff350 0x7fc118195e40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc10c006010 tx=0x7fc10c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:25.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.544+0000 7fc0fffff700 1 -- 172.21.15.88:0/1549221842 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc10c01d070 con 0x7fc1180ff350 2024-02-19T19:33:25.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.544+0000 7fc0fffff700 1 -- 172.21.15.88:0/1549221842 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc10c00f460 con 0x7fc1180ff350 2024-02-19T19:33:25.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.544+0000 7fc11d7d6700 1 -- 172.21.15.88:0/1549221842 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc11819a900 con 0x7fc1180ff350 2024-02-19T19:33:25.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.544+0000 7fc0fffff700 1 -- 172.21.15.88:0/1549221842 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc10c0175f0 con 0x7fc1180ff350 2024-02-19T19:33:25.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.544+0000 7fc11d7d6700 1 -- 172.21.15.88:0/1549221842 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11819adf0 con 0x7fc1180ff350 2024-02-19T19:33:25.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.545+0000 7fc11d7d6700 1 -- 172.21.15.88:0/1549221842 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11804f2a0 con 0x7fc1180ff350 2024-02-19T19:33:25.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.546+0000 7fc0fffff700 1 -- 172.21.15.88:0/1549221842 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc10c00f5d0 con 0x7fc1180ff350 2024-02-19T19:33:25.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.546+0000 7fc0fffff700 1 --2- 172.21.15.88:0/1549221842 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10006d8a0 0x7fc10006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:25.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.546+0000 7fc0fffff700 1 -- 172.21.15.88:0/1549221842 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc10c08ea90 con 0x7fc1180ff350 2024-02-19T19:33:25.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.546+0000 7fc1167fc700 1 --2- 172.21.15.88:0/1549221842 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10006d8a0 0x7fc10006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:25.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.547+0000 7fc1167fc700 1 --2- 172.21.15.88:0/1549221842 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10006d8a0 0x7fc10006fd50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc118197460 tx=0x7fc10800a380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:25.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.551+0000 7fc0fffff700 1 -- 172.21.15.88:0/1549221842 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc10c058240 con 0x7fc1180ff350 2024-02-19T19:33:25.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.767+0000 7fc11d7d6700 1 -- 172.21.15.88:0/1549221842 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc118108920 con 0x7fc10006d8a0 2024-02-19T19:33:25.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.768+0000 7fc0fffff700 1 -- 172.21.15.88:0/1549221842 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc118108920 con 0x7fc10006d8a0 2024-02-19T19:33:25.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.770+0000 7fc11d7d6700 1 -- 172.21.15.88:0/1549221842 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10006d8a0 msgr2=0x7fc10006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:25.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.770+0000 7fc11d7d6700 1 --2- 172.21.15.88:0/1549221842 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10006d8a0 0x7fc10006fd50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc118197460 tx=0x7fc10800a380 comp rx=0 tx=0).stop 2024-02-19T19:33:25.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.770+0000 7fc11d7d6700 1 -- 172.21.15.88:0/1549221842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1180ff350 msgr2=0x7fc118195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:25.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.770+0000 7fc11d7d6700 1 --2- 172.21.15.88:0/1549221842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1180ff350 0x7fc118195e40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fc10c006010 tx=0x7fc10c00bd70 comp rx=0 tx=0).stop 2024-02-19T19:33:25.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.770+0000 7fc11d7d6700 1 -- 172.21.15.88:0/1549221842 shutdown_connections 2024-02-19T19:33:25.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.770+0000 7fc11d7d6700 1 --2- 172.21.15.88:0/1549221842 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc10006d8a0 0x7fc10006fd50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:25.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.770+0000 7fc11d7d6700 1 --2- 172.21.15.88:0/1549221842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1180ff350 0x7fc118195e40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:25.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.770+0000 7fc11d7d6700 1 --2- 172.21.15.88:0/1549221842 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc1181038b0 0x7fc118196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:25.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.770+0000 7fc11d7d6700 1 -- 172.21.15.88:0/1549221842 >> 172.21.15.88:0/1549221842 conn(0x7fc1180748b0 msgr2=0x7fc118102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:25.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.770+0000 7fc11d7d6700 1 -- 172.21.15.88:0/1549221842 shutdown_connections 2024-02-19T19:33:25.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.770+0000 7fc11d7d6700 1 -- 172.21.15.88:0/1549221842 wait complete. 2024-02-19T19:33:25.783 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:33:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.875+0000 7f109ef41700 1 -- 172.21.15.88:0/4175432852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10980ff030 msgr2=0x7f10980ff400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:25.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.875+0000 7f109ef41700 1 --2- 172.21.15.88:0/4175432852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10980ff030 0x7f10980ff400 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f108c009910 tx=0x7f108c009c20 comp rx=0 tx=0).stop 2024-02-19T19:33:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 -- 172.21.15.88:0/4175432852 shutdown_connections 2024-02-19T19:33:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 --2- 172.21.15.88:0/4175432852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10980ff940 0x7f109810c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 --2- 172.21.15.88:0/4175432852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10980ff030 0x7f10980ff400 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 -- 172.21.15.88:0/4175432852 >> 172.21.15.88:0/4175432852 conn(0x7f10980fac80 msgr2=0x7f10980fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 -- 172.21.15.88:0/4175432852 shutdown_connections 2024-02-19T19:33:25.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 -- 172.21.15.88:0/4175432852 wait complete. 2024-02-19T19:33:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 Processor -- start 2024-02-19T19:33:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 -- start start 2024-02-19T19:33:25.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10980ff030 0x7f1098198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10980ff940 0x7f10981985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1098198cb0 con 0x7f10980ff940 2024-02-19T19:33:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f109ef41700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109819ca40 con 0x7f10980ff030 2024-02-19T19:33:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.876+0000 7f1097fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10980ff940 0x7f10981985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f1097fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10980ff940 0x7f10981985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35716/0 (socket says 172.21.15.88:35716) 2024-02-19T19:33:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f1097fff700 1 -- 172.21.15.88:0/846847100 learned_addr learned my addr 172.21.15.88:0/846847100 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:25.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f109ccdd700 1 --2- 172.21.15.88:0/846847100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10980ff030 0x7f1098198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f1097fff700 1 -- 172.21.15.88:0/846847100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10980ff030 msgr2=0x7f1098198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f1097fff700 1 --2- 172.21.15.88:0/846847100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10980ff030 0x7f1098198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f1097fff700 1 -- 172.21.15.88:0/846847100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f108c0095f0 con 0x7f10980ff940 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f1097fff700 1 --2- 172.21.15.88:0/846847100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10980ff940 0x7f10981985d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f108400c8a0 tx=0x7f108400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f1095ffb700 1 -- 172.21.15.88:0/846847100 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10840078e0 con 0x7f10980ff940 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f1095ffb700 1 -- 172.21.15.88:0/846847100 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f108400f450 con 0x7f10980ff940 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f109ef41700 1 -- 172.21.15.88:0/846847100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109819cd20 con 0x7f10980ff940 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f1095ffb700 1 -- 172.21.15.88:0/846847100 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108400e5e0 con 0x7f10980ff940 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.877+0000 7f109ef41700 1 -- 172.21.15.88:0/846847100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109819d270 con 0x7f10980ff940 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.878+0000 7f109ef41700 1 -- 172.21.15.88:0/846847100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1098109e10 con 0x7f10980ff940 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.879+0000 7f1095ffb700 1 -- 172.21.15.88:0/846847100 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f108400f5c0 con 0x7f10980ff940 2024-02-19T19:33:25.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.879+0000 7f1095ffb700 1 --2- 172.21.15.88:0/846847100 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f108806d8a0 0x7f108806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:25.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.879+0000 7f109ccdd700 1 --2- 172.21.15.88:0/846847100 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f108806d8a0 0x7f108806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:25.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.879+0000 7f1095ffb700 1 -- 172.21.15.88:0/846847100 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f108408d3d0 con 0x7f10980ff940 2024-02-19T19:33:25.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.880+0000 7f109ccdd700 1 --2- 172.21.15.88:0/846847100 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f108806d8a0 0x7f108806fd50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f108c005f50 tx=0x7f108c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:25.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:25.884+0000 7f1095ffb700 1 -- 172.21.15.88:0/846847100 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1084056c00 con 0x7f10980ff940 2024-02-19T19:33:26.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.097+0000 7f109ef41700 1 -- 172.21.15.88:0/846847100 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f109819d5b0 con 0x7f108806d8a0 2024-02-19T19:33:26.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.099+0000 7f1095ffb700 1 -- 172.21.15.88:0/846847100 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f109819d5b0 con 0x7f108806d8a0 2024-02-19T19:33:26.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.101+0000 7f109ef41700 1 -- 172.21.15.88:0/846847100 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f108806d8a0 msgr2=0x7f108806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.101+0000 7f109ef41700 1 --2- 172.21.15.88:0/846847100 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f108806d8a0 0x7f108806fd50 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f108c005f50 tx=0x7f108c01a040 comp rx=0 tx=0).stop 2024-02-19T19:33:26.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.101+0000 7f109ef41700 1 -- 172.21.15.88:0/846847100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10980ff940 msgr2=0x7f10981985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.101+0000 7f109ef41700 1 --2- 172.21.15.88:0/846847100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10980ff940 0x7f10981985d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f108400c8a0 tx=0x7f108400cbb0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.101+0000 7f109ef41700 1 -- 172.21.15.88:0/846847100 shutdown_connections 2024-02-19T19:33:26.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.101+0000 7f109ef41700 1 --2- 172.21.15.88:0/846847100 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f108806d8a0 0x7f108806fd50 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.101+0000 7f109ef41700 1 --2- 172.21.15.88:0/846847100 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f10980ff030 0x7f1098198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.101+0000 7f109ef41700 1 --2- 172.21.15.88:0/846847100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10980ff940 0x7f10981985d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.101+0000 7f109ef41700 1 -- 172.21.15.88:0/846847100 >> 172.21.15.88:0/846847100 conn(0x7f10980fac80 msgr2=0x7f10980fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:26.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.101+0000 7f109ef41700 1 -- 172.21.15.88:0/846847100 shutdown_connections 2024-02-19T19:33:26.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.101+0000 7f109ef41700 1 -- 172.21.15.88:0/846847100 wait complete. 2024-02-19T19:33:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.205+0000 7fc8178b3700 1 -- 172.21.15.88:0/4033960833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8101004c0 msgr2=0x7fc810100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.205+0000 7fc8178b3700 1 --2- 172.21.15.88:0/4033960833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8101004c0 0x7fc810100890 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b0a0 tx=0x7fc80800b3b0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.205+0000 7fc8178b3700 1 -- 172.21.15.88:0/4033960833 shutdown_connections 2024-02-19T19:33:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.205+0000 7fc8178b3700 1 --2- 172.21.15.88:0/4033960833 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc810100e60 0x7fc810104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.205+0000 7fc8178b3700 1 --2- 172.21.15.88:0/4033960833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8101004c0 0x7fc810100890 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.205+0000 7fc8178b3700 1 -- 172.21.15.88:0/4033960833 >> 172.21.15.88:0/4033960833 conn(0x7fc8100fbd80 msgr2=0x7fc8100fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.205+0000 7fc8178b3700 1 -- 172.21.15.88:0/4033960833 shutdown_connections 2024-02-19T19:33:26.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.205+0000 7fc8178b3700 1 -- 172.21.15.88:0/4033960833 wait complete. 2024-02-19T19:33:26.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc8178b3700 1 Processor -- start 2024-02-19T19:33:26.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc8178b3700 1 -- start start 2024-02-19T19:33:26.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc8178b3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8101004c0 0x7fc810195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:26.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc8178b3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc810100e60 0x7fc8101964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:26.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc8178b3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810196bb0 con 0x7fc810100e60 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc8178b3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81019a940 con 0x7fc8101004c0 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc814e4e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc810100e60 0x7fc8101964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc814e4e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc810100e60 0x7fc8101964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35748/0 (socket says 172.21.15.88:35748) 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc814e4e700 1 -- 172.21.15.88:0/648598304 learned_addr learned my addr 172.21.15.88:0/648598304 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc81564f700 1 --2- 172.21.15.88:0/648598304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8101004c0 0x7fc810195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc814e4e700 1 -- 172.21.15.88:0/648598304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8101004c0 msgr2=0x7fc810195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc814e4e700 1 --2- 172.21.15.88:0/648598304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8101004c0 0x7fc810195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.206+0000 7fc814e4e700 1 -- 172.21.15.88:0/648598304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808009d00 con 0x7fc810100e60 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.207+0000 7fc814e4e700 1 --2- 172.21.15.88:0/648598304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc810100e60 0x7fc8101964d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc80000b480 tx=0x7fc80000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.207+0000 7fc8067fc700 1 -- 172.21.15.88:0/648598304 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8000118c0 con 0x7fc810100e60 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.207+0000 7fc8067fc700 1 -- 172.21.15.88:0/648598304 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc800010380 con 0x7fc810100e60 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.207+0000 7fc8178b3700 1 -- 172.21.15.88:0/648598304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81019ac20 con 0x7fc810100e60 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.207+0000 7fc8067fc700 1 -- 172.21.15.88:0/648598304 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80000f4f0 con 0x7fc810100e60 2024-02-19T19:33:26.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.207+0000 7fc8178b3700 1 -- 172.21.15.88:0/648598304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81019b170 con 0x7fc810100e60 2024-02-19T19:33:26.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.208+0000 7fc8178b3700 1 -- 172.21.15.88:0/648598304 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8101087d0 con 0x7fc810100e60 2024-02-19T19:33:26.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.208+0000 7fc8067fc700 1 -- 172.21.15.88:0/648598304 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc8000104f0 con 0x7fc810100e60 2024-02-19T19:33:26.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.209+0000 7fc8067fc700 1 --2- 172.21.15.88:0/648598304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc7fc06d8a0 0x7fc7fc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:26.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.209+0000 7fc8067fc700 1 -- 172.21.15.88:0/648598304 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc80008d0a0 con 0x7fc810100e60 2024-02-19T19:33:26.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.209+0000 7fc81564f700 1 --2- 172.21.15.88:0/648598304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc7fc06d8a0 0x7fc7fc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:26.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.209+0000 7fc81564f700 1 --2- 172.21.15.88:0/648598304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc7fc06d8a0 0x7fc7fc06fd50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b830 tx=0x7fc8080097c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:26.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.214+0000 7fc8067fc700 1 -- 172.21.15.88:0/648598304 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc8000568d0 con 0x7fc810100e60 2024-02-19T19:33:26.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.419+0000 7fc8178b3700 1 -- 172.21.15.88:0/648598304 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc81019b4a0 con 0x7fc7fc06d8a0 2024-02-19T19:33:26.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.430+0000 7fc8067fc700 1 -- 172.21.15.88:0/648598304 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc81019b4a0 con 0x7fc7fc06d8a0 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (30m) 6m ago 41m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (41m) 6m ago 41m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (23m) 6m ago 39m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (26m) 6m ago 41m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (26m) 6m ago 39m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (29m) 6m ago 40m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (22m) 6m ago 34m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (22m) 6m ago 34m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (22m) 6m ago 34m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (7m) 6m ago 34m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (32m) 6m ago 42m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (31m) 6m ago 39m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:33:26.433 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (28m) 6m ago 42m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:33:26.434 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (27m) 6m ago 39m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:33:26.434 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (31m) 6m ago 41m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:33:26.434 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (31m) 6m ago 39m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:33:26.434 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (26m) 6m ago 38m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:33:26.434 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (25m) 6m ago 38m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:33:26.434 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (24m) 6m ago 37m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:33:26.434 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (24m) 6m ago 37m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:33:26.434 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (23m) 6m ago 36m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:33:26.434 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (23m) 6m ago 36m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:33:26.434 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (30m) 6m ago 39m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:33:26.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.432+0000 7fc8178b3700 1 -- 172.21.15.88:0/648598304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc7fc06d8a0 msgr2=0x7fc7fc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.432+0000 7fc8178b3700 1 --2- 172.21.15.88:0/648598304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc7fc06d8a0 0x7fc7fc06fd50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b830 tx=0x7fc8080097c0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.432+0000 7fc8178b3700 1 -- 172.21.15.88:0/648598304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc810100e60 msgr2=0x7fc8101964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.432+0000 7fc8178b3700 1 --2- 172.21.15.88:0/648598304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc810100e60 0x7fc8101964d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc80000b480 tx=0x7fc80000b840 comp rx=0 tx=0).stop 2024-02-19T19:33:26.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.432+0000 7fc8178b3700 1 -- 172.21.15.88:0/648598304 shutdown_connections 2024-02-19T19:33:26.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.432+0000 7fc8178b3700 1 --2- 172.21.15.88:0/648598304 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc7fc06d8a0 0x7fc7fc06fd50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.432+0000 7fc8178b3700 1 --2- 172.21.15.88:0/648598304 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc8101004c0 0x7fc810195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.432+0000 7fc8178b3700 1 --2- 172.21.15.88:0/648598304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc810100e60 0x7fc8101964d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.432+0000 7fc8178b3700 1 -- 172.21.15.88:0/648598304 >> 172.21.15.88:0/648598304 conn(0x7fc8100fbd80 msgr2=0x7fc8100fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:26.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.432+0000 7fc8178b3700 1 -- 172.21.15.88:0/648598304 shutdown_connections 2024-02-19T19:33:26.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.432+0000 7fc8178b3700 1 -- 172.21.15.88:0/648598304 wait complete. 2024-02-19T19:33:26.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.534+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/3532207021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e080ff3b0 msgr2=0x7f2e080ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.534+0000 7f2e0fbc9700 1 --2- 172.21.15.88:0/3532207021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e080ff3b0 0x7f2e080ff780 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f2e0000b0a0 tx=0x7f2e0000b3b0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.534+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/3532207021 shutdown_connections 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.534+0000 7f2e0fbc9700 1 --2- 172.21.15.88:0/3532207021 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e080ffd50 0x7f2e08103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.534+0000 7f2e0fbc9700 1 --2- 172.21.15.88:0/3532207021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e080ff3b0 0x7f2e080ff780 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.534+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/3532207021 >> 172.21.15.88:0/3532207021 conn(0x7f2e080fac90 msgr2=0x7f2e080fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.534+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/3532207021 shutdown_connections 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.534+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/3532207021 wait complete. 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.535+0000 7f2e0fbc9700 1 Processor -- start 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.535+0000 7f2e0fbc9700 1 -- start start 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.535+0000 7f2e0fbc9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e080ff3b0 0x7f2e08100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.535+0000 7f2e0fbc9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e080ffd50 0x7f2e08101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.535+0000 7f2e0fbc9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e08101840 con 0x7f2e080ffd50 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.535+0000 7f2e0fbc9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e08101980 con 0x7f2e080ff3b0 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.535+0000 7f2e0d164700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e080ffd50 0x7f2e08101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.535+0000 7f2e0d164700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e080ffd50 0x7f2e08101300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35780/0 (socket says 172.21.15.88:35780) 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.535+0000 7f2e0d164700 1 -- 172.21.15.88:0/4070442585 learned_addr learned my addr 172.21.15.88:0/4070442585 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.535+0000 7f2e0d965700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e080ff3b0 0x7f2e08100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.536+0000 7f2e0d164700 1 -- 172.21.15.88:0/4070442585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e080ff3b0 msgr2=0x7f2e08100dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.536+0000 7f2e0d164700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e080ff3b0 0x7f2e08100dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.536+0000 7f2e0d164700 1 -- 172.21.15.88:0/4070442585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e00009d00 con 0x7f2e080ffd50 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.536+0000 7f2e0d965700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e080ff3b0 0x7f2e08100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.536+0000 7f2e0d164700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e080ffd50 0x7f2e08101300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2df800c8f0 tx=0x7f2df800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.536+0000 7f2dfeffd700 1 -- 172.21.15.88:0/4070442585 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2df8007a50 con 0x7f2e080ffd50 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.536+0000 7f2dfeffd700 1 -- 172.21.15.88:0/4070442585 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2df8007bb0 con 0x7f2e080ffd50 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.536+0000 7f2dfeffd700 1 -- 172.21.15.88:0/4070442585 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2df8018630 con 0x7f2e080ffd50 2024-02-19T19:33:26.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.536+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/4070442585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e081a2020 con 0x7f2e080ffd50 2024-02-19T19:33:26.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.536+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/4070442585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e081a23d0 con 0x7f2e080ffd50 2024-02-19T19:33:26.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.537+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/4070442585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e081086d0 con 0x7f2e080ffd50 2024-02-19T19:33:26.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.537+0000 7f2dfeffd700 1 -- 172.21.15.88:0/4070442585 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2df800f450 con 0x7f2e080ffd50 2024-02-19T19:33:26.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.538+0000 7f2dfeffd700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2df406d8a0 0x7f2df406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:26.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.538+0000 7f2e0d965700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2df406d8a0 0x7f2df406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:26.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.538+0000 7f2dfeffd700 1 -- 172.21.15.88:0/4070442585 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2df808d550 con 0x7f2e080ffd50 2024-02-19T19:33:26.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.538+0000 7f2e0d965700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2df406d8a0 0x7f2df406fd50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2e000062a0 tx=0x7f2e00006210 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:26.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.543+0000 7f2dfeffd700 1 -- 172.21.15.88:0/4070442585 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2df8056d00 con 0x7f2e080ffd50 2024-02-19T19:33:26.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.828+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/4070442585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2e0804f2a0 con 0x7f2e080ffd50 2024-02-19T19:33:26.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:26 smithi088 ceph-mon[88643]: from='client.35622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:26.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:26 smithi088 ceph-mon[88643]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:26.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:26 smithi088 ceph-mon[88643]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:26.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:26 smithi088 ceph-mon[88643]: pgmap v897: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:26.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.828+0000 7f2dfeffd700 1 -- 172.21.15.88:0/4070442585 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2df805a3c0 con 0x7f2e080ffd50 2024-02-19T19:33:26.831 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:33:26.832 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:33:26.833 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:33:26.833 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:33:26.833 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:33:26.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.830+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/4070442585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2df406d8a0 msgr2=0x7f2df406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.830+0000 7f2e0fbc9700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2df406d8a0 0x7f2df406fd50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f2e000062a0 tx=0x7f2e00006210 comp rx=0 tx=0).stop 2024-02-19T19:33:26.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.831+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/4070442585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e080ffd50 msgr2=0x7f2e08101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.831+0000 7f2e0fbc9700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e080ffd50 0x7f2e08101300 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f2df800c8f0 tx=0x7f2df800ccb0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.831+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/4070442585 shutdown_connections 2024-02-19T19:33:26.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.831+0000 7f2e0fbc9700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2df406d8a0 0x7f2df406fd50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.831+0000 7f2e0fbc9700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2e080ff3b0 0x7f2e08100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.831+0000 7f2e0fbc9700 1 --2- 172.21.15.88:0/4070442585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e080ffd50 0x7f2e08101300 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.831+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/4070442585 >> 172.21.15.88:0/4070442585 conn(0x7f2e080fac90 msgr2=0x7f2e08069130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:26.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.831+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/4070442585 shutdown_connections 2024-02-19T19:33:26.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.831+0000 7f2e0fbc9700 1 -- 172.21.15.88:0/4070442585 wait complete. 2024-02-19T19:33:26.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.934+0000 7f969b750700 1 -- 172.21.15.88:0/2476621481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96940ffde0 msgr2=0x7f9694103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.934+0000 7f969b750700 1 --2- 172.21.15.88:0/2476621481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96940ffde0 0x7f9694103db0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9684009910 tx=0x7f9684009c20 comp rx=0 tx=0).stop 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 -- 172.21.15.88:0/2476621481 shutdown_connections 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 --2- 172.21.15.88:0/2476621481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96940ffde0 0x7f9694103db0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 --2- 172.21.15.88:0/2476621481 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96940ff440 0x7f96940ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 -- 172.21.15.88:0/2476621481 >> 172.21.15.88:0/2476621481 conn(0x7f96940face0 msgr2=0x7f96940fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 -- 172.21.15.88:0/2476621481 shutdown_connections 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 -- 172.21.15.88:0/2476621481 wait complete. 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 Processor -- start 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 -- start start 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96940ff440 0x7f9694196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96940ffde0 0x7f9694197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9694197940 con 0x7f96940ff440 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.935+0000 7f969b750700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9694197ab0 con 0x7f96940ffde0 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f96994ec700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96940ff440 0x7f9694196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:26.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f96994ec700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96940ff440 0x7f9694196de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35796/0 (socket says 172.21.15.88:35796) 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f96994ec700 1 -- 172.21.15.88:0/3901780461 learned_addr learned my addr 172.21.15.88:0/3901780461 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f9698ceb700 1 --2- 172.21.15.88:0/3901780461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96940ffde0 0x7f9694197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f96994ec700 1 -- 172.21.15.88:0/3901780461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96940ffde0 msgr2=0x7f9694197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f96994ec700 1 --2- 172.21.15.88:0/3901780461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96940ffde0 0x7f9694197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f96994ec700 1 -- 172.21.15.88:0/3901780461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96840095f0 con 0x7f96940ff440 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f96994ec700 1 --2- 172.21.15.88:0/3901780461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96940ff440 0x7f9694196de0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f969000b4b0 tx=0x7f969000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f968a7fc700 1 -- 172.21.15.88:0/3901780461 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9690011800 con 0x7f96940ff440 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f968a7fc700 1 -- 172.21.15.88:0/3901780461 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9690011e20 con 0x7f96940ff440 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f969b750700 1 -- 172.21.15.88:0/3901780461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f969419fe50 con 0x7f96940ff440 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f968a7fc700 1 -- 172.21.15.88:0/3901780461 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f969000f2e0 con 0x7f96940ff440 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.936+0000 7f969b750700 1 -- 172.21.15.88:0/3901780461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96941a0310 con 0x7f96940ff440 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.937+0000 7f969b750700 1 -- 172.21.15.88:0/3901780461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96941076e0 con 0x7f96940ff440 2024-02-19T19:33:26.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.938+0000 7f968a7fc700 1 -- 172.21.15.88:0/3901780461 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9690010380 con 0x7f96940ff440 2024-02-19T19:33:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.938+0000 7f968a7fc700 1 --2- 172.21.15.88:0/3901780461 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f968006d8a0 0x7f968006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.938+0000 7f968a7fc700 1 -- 172.21.15.88:0/3901780461 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f969008d130 con 0x7f96940ff440 2024-02-19T19:33:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.938+0000 7f9698ceb700 1 --2- 172.21.15.88:0/3901780461 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f968006d8a0 0x7f968006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:26.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.938+0000 7f9698ceb700 1 --2- 172.21.15.88:0/3901780461 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f968006d8a0 0x7f968006fd50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9684006010 tx=0x7f9684005a90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:26.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:26.943+0000 7f968a7fc700 1 -- 172.21.15.88:0/3901780461 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f96900568e0 con 0x7f96940ff440 2024-02-19T19:33:27.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.182+0000 7f969b750700 1 -- 172.21.15.88:0/3901780461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f969404f2a0 con 0x7f96940ff440 2024-02-19T19:33:27.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.182+0000 7f968a7fc700 1 -- 172.21.15.88:0/3901780461 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f9690059fa0 con 0x7f96940ff440 2024-02-19T19:33:27.185 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:33:27.185 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:33:27.185 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:33:27.185 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:33:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:33:27.185 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:33:27.185 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:33:27.185 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:33:27.185 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:33:27.186 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:33:27.187 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:33:27.188 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:33:27.188 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:33:27.188 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:33:27.188 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:33:27.188 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:33:27.188 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:33:27.188 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:33:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.184+0000 7f969b750700 1 -- 172.21.15.88:0/3901780461 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f968006d8a0 msgr2=0x7f968006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.184+0000 7f969b750700 1 --2- 172.21.15.88:0/3901780461 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f968006d8a0 0x7f968006fd50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9684006010 tx=0x7f9684005a90 comp rx=0 tx=0).stop 2024-02-19T19:33:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.184+0000 7f969b750700 1 -- 172.21.15.88:0/3901780461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96940ff440 msgr2=0x7f9694196de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.184+0000 7f969b750700 1 --2- 172.21.15.88:0/3901780461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96940ff440 0x7f9694196de0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f969000b4b0 tx=0x7f969000b870 comp rx=0 tx=0).stop 2024-02-19T19:33:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.184+0000 7f969b750700 1 -- 172.21.15.88:0/3901780461 shutdown_connections 2024-02-19T19:33:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.184+0000 7f969b750700 1 --2- 172.21.15.88:0/3901780461 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f968006d8a0 0x7f968006fd50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.184+0000 7f969b750700 1 --2- 172.21.15.88:0/3901780461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96940ff440 0x7f9694196de0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.184+0000 7f969b750700 1 --2- 172.21.15.88:0/3901780461 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f96940ffde0 0x7f9694197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.184+0000 7f969b750700 1 -- 172.21.15.88:0/3901780461 >> 172.21.15.88:0/3901780461 conn(0x7f96940face0 msgr2=0x7f96940fc520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:27.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.184+0000 7f969b750700 1 -- 172.21.15.88:0/3901780461 shutdown_connections 2024-02-19T19:33:27.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.184+0000 7f969b750700 1 -- 172.21.15.88:0/3901780461 wait complete. 2024-02-19T19:33:27.190 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:33:27.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:26 smithi132 ceph-mon[70676]: from='client.35622 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:27.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:26 smithi132 ceph-mon[70676]: from='client.35626 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:27.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:26 smithi132 ceph-mon[70676]: from='client.35630 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:27.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:26 smithi132 ceph-mon[70676]: pgmap v897: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:27.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.288+0000 7fb555161700 1 -- 172.21.15.88:0/2604836173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5500ff350 msgr2=0x7fb5501063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:27.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.288+0000 7fb555161700 1 --2- 172.21.15.88:0/2604836173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5500ff350 0x7fb5501063b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb540009910 tx=0x7fb540009c20 comp rx=0 tx=0).stop 2024-02-19T19:33:27.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.288+0000 7fb555161700 1 -- 172.21.15.88:0/2604836173 shutdown_connections 2024-02-19T19:33:27.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.288+0000 7fb555161700 1 --2- 172.21.15.88:0/2604836173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5500ff720 0x7fb5500ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.288+0000 7fb555161700 1 --2- 172.21.15.88:0/2604836173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5500ff350 0x7fb5501063b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.288+0000 7fb555161700 1 -- 172.21.15.88:0/2604836173 >> 172.21.15.88:0/2604836173 conn(0x7fb5500748b0 msgr2=0x7fb550074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:27.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.288+0000 7fb555161700 1 -- 172.21.15.88:0/2604836173 shutdown_connections 2024-02-19T19:33:27.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.288+0000 7fb555161700 1 -- 172.21.15.88:0/2604836173 wait complete. 2024-02-19T19:33:27.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.289+0000 7fb555161700 1 Processor -- start 2024-02-19T19:33:27.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.289+0000 7fb555161700 1 -- start start 2024-02-19T19:33:27.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb555161700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5500ff350 0x7fb55019c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:27.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb555161700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5500ff720 0x7fb55019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:27.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb555161700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55019d040 con 0x7fb5500ff720 2024-02-19T19:33:27.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb555161700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5501a0dd0 con 0x7fb5500ff350 2024-02-19T19:33:27.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb54e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5500ff720 0x7fb55019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:27.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb54e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5500ff720 0x7fb55019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35810/0 (socket says 172.21.15.88:35810) 2024-02-19T19:33:27.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb54e59c700 1 -- 172.21.15.88:0/3162867437 learned_addr learned my addr 172.21.15.88:0/3162867437 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:27.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb54ed9d700 1 --2- 172.21.15.88:0/3162867437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5500ff350 0x7fb55019c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:27.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb54e59c700 1 -- 172.21.15.88:0/3162867437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5500ff350 msgr2=0x7fb55019c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb54e59c700 1 --2- 172.21.15.88:0/3162867437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5500ff350 0x7fb55019c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb54e59c700 1 -- 172.21.15.88:0/3162867437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5400095f0 con 0x7fb5500ff720 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb54e59c700 1 --2- 172.21.15.88:0/3162867437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5500ff720 0x7fb55019c960 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb54400c8a0 tx=0x7fb54400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb537fff700 1 -- 172.21.15.88:0/3162867437 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5440078e0 con 0x7fb5500ff720 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb555161700 1 -- 172.21.15.88:0/3162867437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5501a10b0 con 0x7fb5500ff720 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb537fff700 1 -- 172.21.15.88:0/3162867437 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb54400f450 con 0x7fb5500ff720 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb537fff700 1 -- 172.21.15.88:0/3162867437 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54400e5e0 con 0x7fb5500ff720 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.290+0000 7fb555161700 1 -- 172.21.15.88:0/3162867437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5501a1600 con 0x7fb5500ff720 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.292+0000 7fb555161700 1 -- 172.21.15.88:0/3162867437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb550103ae0 con 0x7fb5500ff720 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.292+0000 7fb537fff700 1 -- 172.21.15.88:0/3162867437 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb54400f5c0 con 0x7fb5500ff720 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.292+0000 7fb537fff700 1 --2- 172.21.15.88:0/3162867437 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb53806d8a0 0x7fb53806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.293+0000 7fb54ed9d700 1 --2- 172.21.15.88:0/3162867437 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb53806d8a0 0x7fb53806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:27.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.293+0000 7fb537fff700 1 -- 172.21.15.88:0/3162867437 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb54408d3d0 con 0x7fb5500ff720 2024-02-19T19:33:27.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.293+0000 7fb54ed9d700 1 --2- 172.21.15.88:0/3162867437 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb53806d8a0 0x7fb53806fd50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb540006010 tx=0x7fb54001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:27.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.297+0000 7fb537fff700 1 -- 172.21.15.88:0/3162867437 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb544056c00 con 0x7fb5500ff720 2024-02-19T19:33:27.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.520+0000 7fb555161700 1 -- 172.21.15.88:0/3162867437 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5501a18d0 con 0x7fb53806d8a0 2024-02-19T19:33:27.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.521+0000 7fb537fff700 1 -- 172.21.15.88:0/3162867437 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb5501a18d0 con 0x7fb53806d8a0 2024-02-19T19:33:27.523 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:33:27.523 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:33:27.523 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:33:27.523 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:33:27.524 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:33:27.524 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:33:27.524 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:33:27.524 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:33:27.524 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:33:27.524 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:33:27.524 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:33:27.524 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:33:27.524 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:33:27.524 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:33:27.524 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:33:27.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.524+0000 7fb555161700 1 -- 172.21.15.88:0/3162867437 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb53806d8a0 msgr2=0x7fb53806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:27.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.524+0000 7fb555161700 1 --2- 172.21.15.88:0/3162867437 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb53806d8a0 0x7fb53806fd50 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb540006010 tx=0x7fb54001a040 comp rx=0 tx=0).stop 2024-02-19T19:33:27.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.524+0000 7fb555161700 1 -- 172.21.15.88:0/3162867437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5500ff720 msgr2=0x7fb55019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:27.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.524+0000 7fb555161700 1 --2- 172.21.15.88:0/3162867437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5500ff720 0x7fb55019c960 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb54400c8a0 tx=0x7fb54400cbb0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.524+0000 7fb555161700 1 -- 172.21.15.88:0/3162867437 shutdown_connections 2024-02-19T19:33:27.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.524+0000 7fb555161700 1 --2- 172.21.15.88:0/3162867437 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb53806d8a0 0x7fb53806fd50 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.524+0000 7fb555161700 1 --2- 172.21.15.88:0/3162867437 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5500ff350 0x7fb55019c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.524+0000 7fb555161700 1 --2- 172.21.15.88:0/3162867437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5500ff720 0x7fb55019c960 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.524+0000 7fb555161700 1 -- 172.21.15.88:0/3162867437 >> 172.21.15.88:0/3162867437 conn(0x7fb5500748b0 msgr2=0x7fb5500fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:27.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.524+0000 7fb555161700 1 -- 172.21.15.88:0/3162867437 shutdown_connections 2024-02-19T19:33:27.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.524+0000 7fb555161700 1 -- 172.21.15.88:0/3162867437 wait complete. 2024-02-19T19:33:27.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.629+0000 7f66ea3e3700 1 -- 172.21.15.88:0/3800114067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e41004a0 msgr2=0x7f66e4100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:27.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.629+0000 7f66ea3e3700 1 --2- 172.21.15.88:0/3800114067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e41004a0 0x7f66e4100870 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009910 tx=0x7f66d4009c20 comp rx=0 tx=0).stop 2024-02-19T19:33:27.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.629+0000 7f66ea3e3700 1 -- 172.21.15.88:0/3800114067 shutdown_connections 2024-02-19T19:33:27.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.629+0000 7f66ea3e3700 1 --2- 172.21.15.88:0/3800114067 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e4100e40 0x7f66e4104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.629+0000 7f66ea3e3700 1 --2- 172.21.15.88:0/3800114067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e41004a0 0x7f66e4100870 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.629+0000 7f66ea3e3700 1 -- 172.21.15.88:0/3800114067 >> 172.21.15.88:0/3800114067 conn(0x7f66e40fbd40 msgr2=0x7f66e40fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:27.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.629+0000 7f66ea3e3700 1 -- 172.21.15.88:0/3800114067 shutdown_connections 2024-02-19T19:33:27.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.629+0000 7f66ea3e3700 1 -- 172.21.15.88:0/3800114067 wait complete. 2024-02-19T19:33:27.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.630+0000 7f66ea3e3700 1 Processor -- start 2024-02-19T19:33:27.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.630+0000 7f66ea3e3700 1 -- start start 2024-02-19T19:33:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.630+0000 7f66ea3e3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e41004a0 0x7f66e4195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.630+0000 7f66ea3e3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e4100e40 0x7f66e4196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.630+0000 7f66ea3e3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e4196af0 con 0x7f66e41004a0 2024-02-19T19:33:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.630+0000 7f66ea3e3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e419a880 con 0x7f66e4100e40 2024-02-19T19:33:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.630+0000 7f66e3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e41004a0 0x7f66e4195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66e3fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e41004a0 0x7f66e4195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35820/0 (socket says 172.21.15.88:35820) 2024-02-19T19:33:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66e3fff700 1 -- 172.21.15.88:0/2042239339 learned_addr learned my addr 172.21.15.88:0/2042239339 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66e37fe700 1 --2- 172.21.15.88:0/2042239339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e4100e40 0x7f66e4196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:27.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66e3fff700 1 -- 172.21.15.88:0/2042239339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e4100e40 msgr2=0x7f66e4196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:27.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66e3fff700 1 --2- 172.21.15.88:0/2042239339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e4100e40 0x7f66e4196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66e3fff700 1 -- 172.21.15.88:0/2042239339 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d40095f0 con 0x7f66e41004a0 2024-02-19T19:33:27.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66e3fff700 1 --2- 172.21.15.88:0/2042239339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e41004a0 0x7f66e4195ed0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f66d4006010 tx=0x7f66d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:27.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66e17fa700 1 -- 172.21.15.88:0/2042239339 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d401d070 con 0x7f66e41004a0 2024-02-19T19:33:27.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66e17fa700 1 -- 172.21.15.88:0/2042239339 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66d4005dc0 con 0x7f66e41004a0 2024-02-19T19:33:27.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66ea3e3700 1 -- 172.21.15.88:0/2042239339 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e419ab00 con 0x7f66e41004a0 2024-02-19T19:33:27.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66e17fa700 1 -- 172.21.15.88:0/2042239339 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d4017400 con 0x7f66e41004a0 2024-02-19T19:33:27.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.631+0000 7f66ea3e3700 1 -- 172.21.15.88:0/2042239339 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e419aff0 con 0x7f66e41004a0 2024-02-19T19:33:27.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.632+0000 7f66ea3e3700 1 -- 172.21.15.88:0/2042239339 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e41087b0 con 0x7f66e41004a0 2024-02-19T19:33:27.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.633+0000 7f66e17fa700 1 -- 172.21.15.88:0/2042239339 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f66d4003a60 con 0x7f66e41004a0 2024-02-19T19:33:27.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.633+0000 7f66e17fa700 1 --2- 172.21.15.88:0/2042239339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66cc06d8a0 0x7f66cc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:27.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.633+0000 7f66e17fa700 1 -- 172.21.15.88:0/2042239339 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f66d408e900 con 0x7f66e41004a0 2024-02-19T19:33:27.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.633+0000 7f66e37fe700 1 --2- 172.21.15.88:0/2042239339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66cc06d8a0 0x7f66cc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:27.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.633+0000 7f66e37fe700 1 --2- 172.21.15.88:0/2042239339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66cc06d8a0 0x7f66cc06fd50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f66e41974f0 tx=0x7f66d800b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:27.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.638+0000 7f66e17fa700 1 -- 172.21.15.88:0/2042239339 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f66d4058130 con 0x7f66e41004a0 2024-02-19T19:33:27.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.919+0000 7f66ea3e3700 1 -- 172.21.15.88:0/2042239339 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f66e404f2a0 con 0x7f66e41004a0 2024-02-19T19:33:27.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.919+0000 7f66e17fa700 1 -- 172.21.15.88:0/2042239339 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f66d405b7f0 con 0x7f66e41004a0 2024-02-19T19:33:27.922 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:33:27.922 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:33:27.922 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:33:27.922 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:33:27.922 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:33:27.922 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:33:27.923 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:33:27.924 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:33:27.924 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:33:27.924 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:33:27.924 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:33:27.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.922+0000 7f66ea3e3700 1 -- 172.21.15.88:0/2042239339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66cc06d8a0 msgr2=0x7f66cc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:27.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.922+0000 7f66ea3e3700 1 --2- 172.21.15.88:0/2042239339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66cc06d8a0 0x7f66cc06fd50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f66e41974f0 tx=0x7f66d800b410 comp rx=0 tx=0).stop 2024-02-19T19:33:27.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.922+0000 7f66ea3e3700 1 -- 172.21.15.88:0/2042239339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e41004a0 msgr2=0x7f66e4195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:27.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.922+0000 7f66ea3e3700 1 --2- 172.21.15.88:0/2042239339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e41004a0 0x7f66e4195ed0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f66d4006010 tx=0x7f66d400bd70 comp rx=0 tx=0).stop 2024-02-19T19:33:27.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.922+0000 7f66ea3e3700 1 -- 172.21.15.88:0/2042239339 shutdown_connections 2024-02-19T19:33:27.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.922+0000 7f66ea3e3700 1 --2- 172.21.15.88:0/2042239339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66cc06d8a0 0x7f66cc06fd50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.922+0000 7f66ea3e3700 1 --2- 172.21.15.88:0/2042239339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e41004a0 0x7f66e4195ed0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.922+0000 7f66ea3e3700 1 --2- 172.21.15.88:0/2042239339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66e4100e40 0x7f66e4196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:27.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.922+0000 7f66ea3e3700 1 -- 172.21.15.88:0/2042239339 >> 172.21.15.88:0/2042239339 conn(0x7f66e40fbd40 msgr2=0x7f66e40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:27.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.922+0000 7f66ea3e3700 1 -- 172.21.15.88:0/2042239339 shutdown_connections 2024-02-19T19:33:27.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:27.922+0000 7f66ea3e3700 1 -- 172.21.15.88:0/2042239339 wait complete. 2024-02-19T19:33:28.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:27 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4070442585' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:33:28.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:27 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3901780461' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:33:28.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:27 smithi088 ceph-mon[88643]: from='client.35642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:28.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:27 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4070442585' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:33:28.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:27 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3901780461' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:33:28.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:27 smithi132 ceph-mon[70676]: from='client.35642 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:29.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:28 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2042239339' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:33:29.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:33:29.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:28 smithi088 ceph-mon[88643]: pgmap v898: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:29.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:28 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2042239339' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:33:29.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:33:29.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:28 smithi132 ceph-mon[70676]: pgmap v898: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:30.985 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:30 smithi132 ceph-mon[70676]: pgmap v899: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:31.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:30 smithi088 ceph-mon[88643]: pgmap v899: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:33.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:32 smithi088 ceph-mon[88643]: pgmap v900: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:33.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:32 smithi132 ceph-mon[70676]: pgmap v900: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:35.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:34 smithi088 ceph-mon[88643]: pgmap v901: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:35.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:34 smithi132 ceph-mon[70676]: pgmap v901: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:37.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:36 smithi088 ceph-mon[88643]: pgmap v902: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:37.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:36 smithi132 ceph-mon[70676]: pgmap v902: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:39.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:38 smithi088 ceph-mon[88643]: pgmap v903: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:39.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:38 smithi132 ceph-mon[70676]: pgmap v903: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:41.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:40 smithi088 ceph-mon[88643]: pgmap v904: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:41.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:40 smithi132 ceph-mon[70676]: pgmap v904: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:43.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:42 smithi088 ceph-mon[88643]: pgmap v905: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:43.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:42 smithi132 ceph-mon[70676]: pgmap v905: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:44.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:33:44.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:33:44.984 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:44 smithi088 ceph-mon[88643]: pgmap v906: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:45.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:44 smithi132 ceph-mon[70676]: pgmap v906: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:47.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:46 smithi088 ceph-mon[88643]: pgmap v907: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:47.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:46 smithi132 ceph-mon[70676]: pgmap v907: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:49.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:48 smithi088 ceph-mon[88643]: pgmap v908: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:49.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:48 smithi132 ceph-mon[70676]: pgmap v908: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:51.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:50 smithi088 ceph-mon[88643]: pgmap v909: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:51.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:50 smithi132 ceph-mon[70676]: pgmap v909: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:53.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:52 smithi088 ceph-mon[88643]: pgmap v910: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:53.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:52 smithi132 ceph-mon[70676]: pgmap v910: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:55.037 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:54 smithi132 ceph-mon[70676]: pgmap v911: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:55.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:54 smithi088 ceph-mon[88643]: pgmap v911: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:57.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:56 smithi088 ceph-mon[88643]: pgmap v912: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:57.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:56 smithi132 ceph-mon[70676]: pgmap v912: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:58.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.035+0000 7fa404605700 1 -- 172.21.15.88:0/245868109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc0ffc60 msgr2=0x7fa3fc10a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.035+0000 7fa404605700 1 --2- 172.21.15.88:0/245868109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc0ffc60 0x7fa3fc10a6d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009910 tx=0x7fa3f8009c20 comp rx=0 tx=0).stop 2024-02-19T19:33:58.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.035+0000 7fa404605700 1 -- 172.21.15.88:0/245868109 shutdown_connections 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.035+0000 7fa404605700 1 --2- 172.21.15.88:0/245868109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc0ffc60 0x7fa3fc10a6d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.035+0000 7fa404605700 1 --2- 172.21.15.88:0/245868109 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3fc0ff350 0x7fa3fc0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.035+0000 7fa404605700 1 -- 172.21.15.88:0/245868109 >> 172.21.15.88:0/245868109 conn(0x7fa3fc0748b0 msgr2=0x7fa3fc074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.035+0000 7fa404605700 1 -- 172.21.15.88:0/245868109 shutdown_connections 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.035+0000 7fa404605700 1 -- 172.21.15.88:0/245868109 wait complete. 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa404605700 1 Processor -- start 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa404605700 1 -- start start 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa404605700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc0ff350 0x7fa3fc100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa404605700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3fc0ffc60 0x7fa3fc100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa404605700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc100fb0 con 0x7fa3fc0ff350 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa404605700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc1010f0 con 0x7fa3fc0ffc60 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa4023a1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc0ff350 0x7fa3fc100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:58.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa4023a1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc0ff350 0x7fa3fc100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42894/0 (socket says 172.21.15.88:42894) 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa4023a1700 1 -- 172.21.15.88:0/2067647148 learned_addr learned my addr 172.21.15.88:0/2067647148 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa401ba0700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3fc0ffc60 0x7fa3fc100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa4023a1700 1 -- 172.21.15.88:0/2067647148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3fc0ffc60 msgr2=0x7fa3fc100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa4023a1700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3fc0ffc60 0x7fa3fc100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa4023a1700 1 -- 172.21.15.88:0/2067647148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f80095f0 con 0x7fa3fc0ff350 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.036+0000 7fa401ba0700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3fc0ffc60 0x7fa3fc100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.037+0000 7fa4023a1700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc0ff350 0x7fa3fc100530 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00b480 tx=0x7fa3ec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.037+0000 7fa3f37fe700 1 -- 172.21.15.88:0/2067647148 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3ec0118c0 con 0x7fa3fc0ff350 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.037+0000 7fa3f37fe700 1 -- 172.21.15.88:0/2067647148 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3ec010380 con 0x7fa3fc0ff350 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.037+0000 7fa404605700 1 -- 172.21.15.88:0/2067647148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc1a2130 con 0x7fa3fc0ff350 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.037+0000 7fa3f37fe700 1 -- 172.21.15.88:0/2067647148 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3ec00f4f0 con 0x7fa3fc0ff350 2024-02-19T19:33:58.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.037+0000 7fa404605700 1 -- 172.21.15.88:0/2067647148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc1a2540 con 0x7fa3fc0ff350 2024-02-19T19:33:58.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.038+0000 7fa404605700 1 -- 172.21.15.88:0/2067647148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3fc107dd0 con 0x7fa3fc0ff350 2024-02-19T19:33:58.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.038+0000 7fa3f37fe700 1 -- 172.21.15.88:0/2067647148 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa3ec0104f0 con 0x7fa3fc0ff350 2024-02-19T19:33:58.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.039+0000 7fa3f37fe700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3e806d8a0 0x7fa3e806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:58.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.039+0000 7fa3f37fe700 1 -- 172.21.15.88:0/2067647148 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa3ec08d1a0 con 0x7fa3fc0ff350 2024-02-19T19:33:58.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.039+0000 7fa401ba0700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3e806d8a0 0x7fa3e806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:58.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.039+0000 7fa401ba0700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3e806d8a0 0x7fa3e806fd50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800b5c0 tx=0x7fa3f8005480 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:58.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.044+0000 7fa3f37fe700 1 -- 172.21.15.88:0/2067647148 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa3ec056950 con 0x7fa3fc0ff350 2024-02-19T19:33:58.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.260+0000 7fa404605700 1 -- 172.21.15.88:0/2067647148 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3fc061190 con 0x7fa3e806d8a0 2024-02-19T19:33:58.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.261+0000 7fa3f37fe700 1 -- 172.21.15.88:0/2067647148 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa3fc061190 con 0x7fa3e806d8a0 2024-02-19T19:33:58.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.263+0000 7fa404605700 1 -- 172.21.15.88:0/2067647148 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3e806d8a0 msgr2=0x7fa3e806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.263+0000 7fa404605700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3e806d8a0 0x7fa3e806fd50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa3f800b5c0 tx=0x7fa3f8005480 comp rx=0 tx=0).stop 2024-02-19T19:33:58.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.263+0000 7fa404605700 1 -- 172.21.15.88:0/2067647148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc0ff350 msgr2=0x7fa3fc100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.263+0000 7fa404605700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc0ff350 0x7fa3fc100530 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00b480 tx=0x7fa3ec00b840 comp rx=0 tx=0).stop 2024-02-19T19:33:58.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.263+0000 7fa404605700 1 -- 172.21.15.88:0/2067647148 shutdown_connections 2024-02-19T19:33:58.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.263+0000 7fa404605700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3e806d8a0 0x7fa3e806fd50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.263+0000 7fa404605700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3fc0ff350 0x7fa3fc100530 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.263+0000 7fa404605700 1 --2- 172.21.15.88:0/2067647148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3fc0ffc60 0x7fa3fc100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.263+0000 7fa404605700 1 -- 172.21.15.88:0/2067647148 >> 172.21.15.88:0/2067647148 conn(0x7fa3fc0748b0 msgr2=0x7fa3fc104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:58.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.264+0000 7fa404605700 1 -- 172.21.15.88:0/2067647148 shutdown_connections 2024-02-19T19:33:58.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.264+0000 7fa404605700 1 -- 172.21.15.88:0/2067647148 wait complete. 2024-02-19T19:33:58.276 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:33:58.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.373+0000 7f3d15b80700 1 -- 172.21.15.88:0/117449538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d100ff3c0 msgr2=0x7f3d100ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.373+0000 7f3d15b80700 1 --2- 172.21.15.88:0/117449538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d100ff3c0 0x7f3d100ff790 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009910 tx=0x7f3d00009c20 comp rx=0 tx=0).stop 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.373+0000 7f3d15b80700 1 -- 172.21.15.88:0/117449538 shutdown_connections 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.373+0000 7f3d15b80700 1 --2- 172.21.15.88:0/117449538 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d100ffd60 0x7f3d10103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.373+0000 7f3d15b80700 1 --2- 172.21.15.88:0/117449538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d100ff3c0 0x7f3d100ff790 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.373+0000 7f3d15b80700 1 -- 172.21.15.88:0/117449538 >> 172.21.15.88:0/117449538 conn(0x7f3d100fac80 msgr2=0x7f3d100fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.373+0000 7f3d15b80700 1 -- 172.21.15.88:0/117449538 shutdown_connections 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.373+0000 7f3d15b80700 1 -- 172.21.15.88:0/117449538 wait complete. 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d15b80700 1 Processor -- start 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d15b80700 1 -- start start 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d15b80700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d100ff3c0 0x7f3d10196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d15b80700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d100ffd60 0x7f3d10197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d15b80700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d10197830 con 0x7f3d100ffd60 2024-02-19T19:33:58.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d15b80700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d101979a0 con 0x7f3d100ff3c0 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d06ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d100ffd60 0x7f3d10197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d06ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d100ffd60 0x7f3d10197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42924/0 (socket says 172.21.15.88:42924) 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d06ffd700 1 -- 172.21.15.88:0/1219605823 learned_addr learned my addr 172.21.15.88:0/1219605823 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d06ffd700 1 -- 172.21.15.88:0/1219605823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d100ff3c0 msgr2=0x7f3d10196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d06ffd700 1 --2- 172.21.15.88:0/1219605823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d100ff3c0 0x7f3d10196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.374+0000 7f3d06ffd700 1 -- 172.21.15.88:0/1219605823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d000095f0 con 0x7f3d100ffd60 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.375+0000 7f3d06ffd700 1 --2- 172.21.15.88:0/1219605823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d100ffd60 0x7f3d10197210 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800d900 tx=0x7f3cf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.375+0000 7f3d0d7fa700 1 -- 172.21.15.88:0/1219605823 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cf8009900 con 0x7f3d100ffd60 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.375+0000 7f3d0d7fa700 1 -- 172.21.15.88:0/1219605823 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cf8010460 con 0x7f3d100ffd60 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.375+0000 7f3d0d7fa700 1 -- 172.21.15.88:0/1219605823 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cf800f5f0 con 0x7f3d100ffd60 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.375+0000 7f3d15b80700 1 -- 172.21.15.88:0/1219605823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d1019fd40 con 0x7f3d100ffd60 2024-02-19T19:33:58.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.375+0000 7f3d15b80700 1 -- 172.21.15.88:0/1219605823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d101a0230 con 0x7f3d100ffd60 2024-02-19T19:33:58.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.376+0000 7f3d15b80700 1 -- 172.21.15.88:0/1219605823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d101076d0 con 0x7f3d100ffd60 2024-02-19T19:33:58.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.376+0000 7f3d0d7fa700 1 -- 172.21.15.88:0/1219605823 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3cf80105d0 con 0x7f3d100ffd60 2024-02-19T19:33:58.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.377+0000 7f3d0d7fa700 1 --2- 172.21.15.88:0/1219605823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3cfc06d8a0 0x7f3cfc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:58.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.377+0000 7f3d0f7fe700 1 --2- 172.21.15.88:0/1219605823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3cfc06d8a0 0x7f3cfc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:58.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.377+0000 7f3d0d7fa700 1 -- 172.21.15.88:0/1219605823 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3cf808d3a0 con 0x7f3d100ffd60 2024-02-19T19:33:58.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.377+0000 7f3d0f7fe700 1 --2- 172.21.15.88:0/1219605823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3cfc06d8a0 0x7f3cfc06fd50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000b5c0 tx=0x7f3d00005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:58.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.381+0000 7f3d0d7fa700 1 -- 172.21.15.88:0/1219605823 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3cf8056b50 con 0x7f3d100ffd60 2024-02-19T19:33:58.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:33:58.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.600+0000 7f3d15b80700 1 -- 172.21.15.88:0/1219605823 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d1002cca0 con 0x7f3cfc06d8a0 2024-02-19T19:33:58.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.601+0000 7f3d0d7fa700 1 -- 172.21.15.88:0/1219605823 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3d1002cca0 con 0x7f3cfc06d8a0 2024-02-19T19:33:58.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.603+0000 7f3d15b80700 1 -- 172.21.15.88:0/1219605823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3cfc06d8a0 msgr2=0x7f3cfc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.603+0000 7f3d15b80700 1 --2- 172.21.15.88:0/1219605823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3cfc06d8a0 0x7f3cfc06fd50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3d0000b5c0 tx=0x7f3d00005960 comp rx=0 tx=0).stop 2024-02-19T19:33:58.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.603+0000 7f3d15b80700 1 -- 172.21.15.88:0/1219605823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d100ffd60 msgr2=0x7f3d10197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.603+0000 7f3d15b80700 1 --2- 172.21.15.88:0/1219605823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d100ffd60 0x7f3d10197210 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800d900 tx=0x7f3cf800dcc0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.603+0000 7f3d15b80700 1 -- 172.21.15.88:0/1219605823 shutdown_connections 2024-02-19T19:33:58.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.603+0000 7f3d15b80700 1 --2- 172.21.15.88:0/1219605823 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3cfc06d8a0 0x7f3cfc06fd50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.603+0000 7f3d15b80700 1 --2- 172.21.15.88:0/1219605823 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3d100ff3c0 0x7f3d10196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.603+0000 7f3d15b80700 1 --2- 172.21.15.88:0/1219605823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d100ffd60 0x7f3d10197210 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.603+0000 7f3d15b80700 1 -- 172.21.15.88:0/1219605823 >> 172.21.15.88:0/1219605823 conn(0x7f3d100fac80 msgr2=0x7f3d100fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:58.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.603+0000 7f3d15b80700 1 -- 172.21.15.88:0/1219605823 shutdown_connections 2024-02-19T19:33:58.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.603+0000 7f3d15b80700 1 -- 172.21.15.88:0/1219605823 wait complete. 2024-02-19T19:33:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.706+0000 7f31f8e58700 1 -- 172.21.15.88:0/4263777666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31f4100e80 msgr2=0x7f31f4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.706+0000 7f31f8e58700 1 --2- 172.21.15.88:0/4263777666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31f4100e80 0x7f31f4104e50 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f31e8009910 tx=0x7f31e8009c20 comp rx=0 tx=0).stop 2024-02-19T19:33:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.706+0000 7f31f8e58700 1 -- 172.21.15.88:0/4263777666 shutdown_connections 2024-02-19T19:33:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.706+0000 7f31f8e58700 1 --2- 172.21.15.88:0/4263777666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31f4100e80 0x7f31f4104e50 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.706+0000 7f31f8e58700 1 --2- 172.21.15.88:0/4263777666 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f41004e0 0x7f31f41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.706+0000 7f31f8e58700 1 -- 172.21.15.88:0/4263777666 >> 172.21.15.88:0/4263777666 conn(0x7f31f40fbda0 msgr2=0x7f31f40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:58.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.707+0000 7f31f8e58700 1 -- 172.21.15.88:0/4263777666 shutdown_connections 2024-02-19T19:33:58.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.707+0000 7f31f8e58700 1 -- 172.21.15.88:0/4263777666 wait complete. 2024-02-19T19:33:58.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.707+0000 7f31f8e58700 1 Processor -- start 2024-02-19T19:33:58.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.707+0000 7f31f8e58700 1 -- start start 2024-02-19T19:33:58.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.707+0000 7f31f8e58700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f41004e0 0x7f31f4196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:58.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.707+0000 7f31f8e58700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31f4100e80 0x7f31f4196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:58.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.707+0000 7f31f8e58700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f4196c20 con 0x7f31f4100e80 2024-02-19T19:33:58.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.707+0000 7f31f8e58700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f419a9b0 con 0x7f31f41004e0 2024-02-19T19:33:58.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31f1d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31f4100e80 0x7f31f4196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:58.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31f1d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31f4100e80 0x7f31f4196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42934/0 (socket says 172.21.15.88:42934) 2024-02-19T19:33:58.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31f1d9b700 1 -- 172.21.15.88:0/2187540455 learned_addr learned my addr 172.21.15.88:0/2187540455 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:58.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31f259c700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f41004e0 0x7f31f4196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31f1d9b700 1 -- 172.21.15.88:0/2187540455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f41004e0 msgr2=0x7f31f4196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31f1d9b700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f41004e0 0x7f31f4196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31f1d9b700 1 -- 172.21.15.88:0/2187540455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e80095f0 con 0x7f31f4100e80 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31f259c700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f41004e0 0x7f31f4196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31f1d9b700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31f4100e80 0x7f31f4196540 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005f50 tx=0x7f31e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31e37fe700 1 -- 172.21.15.88:0/2187540455 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31e801d070 con 0x7f31f4100e80 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31f8e58700 1 -- 172.21.15.88:0/2187540455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f419ac30 con 0x7f31f4100e80 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31e37fe700 1 -- 172.21.15.88:0/2187540455 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31e800f460 con 0x7f31f4100e80 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31e37fe700 1 -- 172.21.15.88:0/2187540455 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31e8017630 con 0x7f31f4100e80 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.708+0000 7f31f8e58700 1 -- 172.21.15.88:0/2187540455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f419b120 con 0x7f31f4100e80 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.709+0000 7f31f8e58700 1 -- 172.21.15.88:0/2187540455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f41087a0 con 0x7f31f4100e80 2024-02-19T19:33:58.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.710+0000 7f31e37fe700 1 -- 172.21.15.88:0/2187540455 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f31e800f5d0 con 0x7f31f4100e80 2024-02-19T19:33:58.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.710+0000 7f31e37fe700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31dc06d8a0 0x7f31dc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:58.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.710+0000 7f31e37fe700 1 -- 172.21.15.88:0/2187540455 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f31e808eb50 con 0x7f31f4100e80 2024-02-19T19:33:58.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.710+0000 7f31f259c700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31dc06d8a0 0x7f31dc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:58.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.711+0000 7f31f259c700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31dc06d8a0 0x7f31dc06fd50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f31e40099d0 tx=0x7f31e4008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:58.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.715+0000 7f31e37fe700 1 -- 172.21.15.88:0/2187540455 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31e8058300 con 0x7f31f4100e80 2024-02-19T19:33:58.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:33:58.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.921+0000 7f31f8e58700 1 -- 172.21.15.88:0/2187540455 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f31f402cca0 con 0x7f31dc06d8a0 2024-02-19T19:33:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.934+0000 7f31e37fe700 1 -- 172.21.15.88:0/2187540455 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f31f402cca0 con 0x7f31dc06d8a0 2024-02-19T19:33:58.936 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:33:58.936 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (31m) 7m ago 41m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:33:58.936 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (42m) 7m ago 42m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:33:58.936 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (23m) 7m ago 40m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:33:58.936 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (27m) 7m ago 42m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:33:58.936 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (27m) 7m ago 40m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (30m) 7m ago 40m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (23m) 7m ago 35m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (23m) 7m ago 35m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (22m) 7m ago 35m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (7m) 7m ago 35m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (32m) 7m ago 43m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (32m) 7m ago 39m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (29m) 7m ago 43m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (28m) 7m ago 39m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (31m) 7m ago 41m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (31m) 7m ago 39m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (26m) 7m ago 39m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (26m) 7m ago 38m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:33:58.937 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (25m) 7m ago 38m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:33:58.938 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (24m) 7m ago 37m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:33:58.938 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (24m) 7m ago 37m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:33:58.938 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (23m) 7m ago 36m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:33:58.938 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (31m) 7m ago 40m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:33:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.936+0000 7f31f8e58700 1 -- 172.21.15.88:0/2187540455 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31dc06d8a0 msgr2=0x7f31dc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.936+0000 7f31f8e58700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31dc06d8a0 0x7f31dc06fd50 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f31e40099d0 tx=0x7f31e4008040 comp rx=0 tx=0).stop 2024-02-19T19:33:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.936+0000 7f31f8e58700 1 -- 172.21.15.88:0/2187540455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31f4100e80 msgr2=0x7f31f4196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.936+0000 7f31f8e58700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31f4100e80 0x7f31f4196540 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f31e8005f50 tx=0x7f31e800be20 comp rx=0 tx=0).stop 2024-02-19T19:33:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.936+0000 7f31f8e58700 1 -- 172.21.15.88:0/2187540455 shutdown_connections 2024-02-19T19:33:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.936+0000 7f31f8e58700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f31dc06d8a0 0x7f31dc06fd50 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.936+0000 7f31f8e58700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f31f41004e0 0x7f31f4196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.936+0000 7f31f8e58700 1 --2- 172.21.15.88:0/2187540455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31f4100e80 0x7f31f4196540 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.937+0000 7f31f8e58700 1 -- 172.21.15.88:0/2187540455 >> 172.21.15.88:0/2187540455 conn(0x7f31f40fbda0 msgr2=0x7f31f40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.937+0000 7f31f8e58700 1 -- 172.21.15.88:0/2187540455 shutdown_connections 2024-02-19T19:33:58.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:58.937+0000 7f31f8e58700 1 -- 172.21.15.88:0/2187540455 wait complete. 2024-02-19T19:33:59.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.038+0000 7f66adb76700 1 -- 172.21.15.88:0/2813314649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66a80ff3b0 msgr2=0x7f66a80ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.038+0000 7f66adb76700 1 --2- 172.21.15.88:0/2813314649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66a80ff3b0 0x7f66a80ff780 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f6690009910 tx=0x7f6690009c20 comp rx=0 tx=0).stop 2024-02-19T19:33:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.039+0000 7f66adb76700 1 -- 172.21.15.88:0/2813314649 shutdown_connections 2024-02-19T19:33:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.039+0000 7f66adb76700 1 --2- 172.21.15.88:0/2813314649 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66a80ffd50 0x7f66a8103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.039+0000 7f66adb76700 1 --2- 172.21.15.88:0/2813314649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66a80ff3b0 0x7f66a80ff780 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.039+0000 7f66adb76700 1 -- 172.21.15.88:0/2813314649 >> 172.21.15.88:0/2813314649 conn(0x7f66a80fac90 msgr2=0x7f66a80fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.039+0000 7f66adb76700 1 -- 172.21.15.88:0/2813314649 shutdown_connections 2024-02-19T19:33:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.039+0000 7f66adb76700 1 -- 172.21.15.88:0/2813314649 wait complete. 2024-02-19T19:33:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.039+0000 7f66adb76700 1 Processor -- start 2024-02-19T19:33:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.039+0000 7f66adb76700 1 -- start start 2024-02-19T19:33:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66adb76700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66a80ff3b0 0x7f66a8196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:59.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66adb76700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66a80ffd50 0x7f66a81972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66adb76700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a81978c0 con 0x7f66a80ffd50 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66adb76700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a8197a00 con 0x7f66a80ff3b0 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66a6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66a80ffd50 0x7f66a81972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66a6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66a80ffd50 0x7f66a81972a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42958/0 (socket says 172.21.15.88:42958) 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66a6ffd700 1 -- 172.21.15.88:0/3755539243 learned_addr learned my addr 172.21.15.88:0/3755539243 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66a77fe700 1 --2- 172.21.15.88:0/3755539243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66a80ff3b0 0x7f66a8196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66a6ffd700 1 -- 172.21.15.88:0/3755539243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66a80ff3b0 msgr2=0x7f66a8196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66a6ffd700 1 --2- 172.21.15.88:0/3755539243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66a80ff3b0 0x7f66a8196d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66a6ffd700 1 -- 172.21.15.88:0/3755539243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66900095f0 con 0x7f66a80ffd50 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66a6ffd700 1 --2- 172.21.15.88:0/3755539243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66a80ffd50 0x7f66a81972a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f669800d900 tx=0x7f669800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66a4ff9700 1 -- 172.21.15.88:0/3755539243 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6698009900 con 0x7f66a80ffd50 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66a4ff9700 1 -- 172.21.15.88:0/3755539243 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6698010460 con 0x7f66a80ffd50 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66adb76700 1 -- 172.21.15.88:0/3755539243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a819fdd0 con 0x7f66a80ffd50 2024-02-19T19:33:59.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66a4ff9700 1 -- 172.21.15.88:0/3755539243 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f669800f5f0 con 0x7f66a80ffd50 2024-02-19T19:33:59.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.040+0000 7f66adb76700 1 -- 172.21.15.88:0/3755539243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a81a0290 con 0x7f66a80ffd50 2024-02-19T19:33:59.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.042+0000 7f66adb76700 1 -- 172.21.15.88:0/3755539243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a81076c0 con 0x7f66a80ffd50 2024-02-19T19:33:59.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.042+0000 7f66a4ff9700 1 -- 172.21.15.88:0/3755539243 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f669800f870 con 0x7f66a80ffd50 2024-02-19T19:33:59.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.042+0000 7f66a4ff9700 1 --2- 172.21.15.88:0/3755539243 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f669406d8a0 0x7f669406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:59.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.042+0000 7f66a77fe700 1 --2- 172.21.15.88:0/3755539243 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f669406d8a0 0x7f669406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:59.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.042+0000 7f66a4ff9700 1 -- 172.21.15.88:0/3755539243 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f669808d0d0 con 0x7f66a80ffd50 2024-02-19T19:33:59.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.043+0000 7f66a77fe700 1 --2- 172.21.15.88:0/3755539243 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f669406d8a0 0x7f669406fd50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6690006010 tx=0x7f669001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:59.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.047+0000 7f66a4ff9700 1 -- 172.21.15.88:0/3755539243 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f66980568f0 con 0x7f66a80ffd50 2024-02-19T19:33:59.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.328+0000 7f66adb76700 1 -- 172.21.15.88:0/3755539243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f66a804f2a0 con 0x7f66a80ffd50 2024-02-19T19:33:59.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.328+0000 7f66a4ff9700 1 -- 172.21.15.88:0/3755539243 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6698059fb0 con 0x7f66a80ffd50 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:33:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:33:59.332 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:33:59.332 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:33:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.330+0000 7f66adb76700 1 -- 172.21.15.88:0/3755539243 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f669406d8a0 msgr2=0x7f669406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.330+0000 7f66adb76700 1 --2- 172.21.15.88:0/3755539243 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f669406d8a0 0x7f669406fd50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6690006010 tx=0x7f669001a040 comp rx=0 tx=0).stop 2024-02-19T19:33:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.330+0000 7f66adb76700 1 -- 172.21.15.88:0/3755539243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66a80ffd50 msgr2=0x7f66a81972a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.330+0000 7f66adb76700 1 --2- 172.21.15.88:0/3755539243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66a80ffd50 0x7f66a81972a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f669800d900 tx=0x7f669800dc10 comp rx=0 tx=0).stop 2024-02-19T19:33:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.330+0000 7f66adb76700 1 -- 172.21.15.88:0/3755539243 shutdown_connections 2024-02-19T19:33:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.330+0000 7f66adb76700 1 --2- 172.21.15.88:0/3755539243 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f669406d8a0 0x7f669406fd50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.330+0000 7f66adb76700 1 --2- 172.21.15.88:0/3755539243 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66a80ff3b0 0x7f66a8196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.330+0000 7f66adb76700 1 --2- 172.21.15.88:0/3755539243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66a80ffd50 0x7f66a81972a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.330+0000 7f66adb76700 1 -- 172.21.15.88:0/3755539243 >> 172.21.15.88:0/3755539243 conn(0x7f66a80fac90 msgr2=0x7f66a80fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.330+0000 7f66adb76700 1 -- 172.21.15.88:0/3755539243 shutdown_connections 2024-02-19T19:33:59.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.330+0000 7f66adb76700 1 -- 172.21.15.88:0/3755539243 wait complete. 2024-02-19T19:33:59.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.433+0000 7fb3ed907700 1 -- 172.21.15.88:0/3213764251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3e8101420 msgr2=0x7fb3e81017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:59.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.433+0000 7fb3ed907700 1 --2- 172.21.15.88:0/3213764251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3e8101420 0x7fb3e81017f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009910 tx=0x7fb3d8009c20 comp rx=0 tx=0).stop 2024-02-19T19:33:59.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.433+0000 7fb3ed907700 1 -- 172.21.15.88:0/3213764251 shutdown_connections 2024-02-19T19:33:59.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.433+0000 7fb3ed907700 1 --2- 172.21.15.88:0/3213764251 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3e8101d30 0x7fb3e810a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.433+0000 7fb3ed907700 1 --2- 172.21.15.88:0/3213764251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3e8101420 0x7fb3e81017f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.433+0000 7fb3ed907700 1 -- 172.21.15.88:0/3213764251 >> 172.21.15.88:0/3213764251 conn(0x7fb3e80fac80 msgr2=0x7fb3e80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:59.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.433+0000 7fb3ed907700 1 -- 172.21.15.88:0/3213764251 shutdown_connections 2024-02-19T19:33:59.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.433+0000 7fb3ed907700 1 -- 172.21.15.88:0/3213764251 wait complete. 2024-02-19T19:33:59.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.434+0000 7fb3ed907700 1 Processor -- start 2024-02-19T19:33:59.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.434+0000 7fb3ed907700 1 -- start start 2024-02-19T19:33:59.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.434+0000 7fb3ed907700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3e8101420 0x7fb3e80ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:59.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.434+0000 7fb3ed907700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3e8101d30 0x7fb3e80ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:59.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.434+0000 7fb3ed907700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80ffb00 con 0x7fb3e8101420 2024-02-19T19:33:59.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.434+0000 7fb3ed907700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e80ffc40 con 0x7fb3e8101d30 2024-02-19T19:33:59.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.434+0000 7fb3e6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3e8101420 0x7fb3e80ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.434+0000 7fb3e6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3e8101420 0x7fb3e80ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42978/0 (socket says 172.21.15.88:42978) 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.434+0000 7fb3e6ffd700 1 -- 172.21.15.88:0/1722502446 learned_addr learned my addr 172.21.15.88:0/1722502446 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.435+0000 7fb3e67fc700 1 --2- 172.21.15.88:0/1722502446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3e8101d30 0x7fb3e80ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.435+0000 7fb3e6ffd700 1 -- 172.21.15.88:0/1722502446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3e8101d30 msgr2=0x7fb3e80ff5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.435+0000 7fb3e6ffd700 1 --2- 172.21.15.88:0/1722502446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3e8101d30 0x7fb3e80ff5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.435+0000 7fb3e6ffd700 1 -- 172.21.15.88:0/1722502446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d80095f0 con 0x7fb3e8101420 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.435+0000 7fb3e6ffd700 1 --2- 172.21.15.88:0/1722502446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3e8101420 0x7fb3e80ff080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8006010 tx=0x7fb3d800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.435+0000 7fb3ec905700 1 -- 172.21.15.88:0/1722502446 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3d801d070 con 0x7fb3e8101420 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.435+0000 7fb3ec905700 1 -- 172.21.15.88:0/1722502446 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3d800f460 con 0x7fb3e8101420 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.435+0000 7fb3ed907700 1 -- 172.21.15.88:0/1722502446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e80ffec0 con 0x7fb3e8101420 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.435+0000 7fb3ec905700 1 -- 172.21.15.88:0/1722502446 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3d8017610 con 0x7fb3e8101420 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.435+0000 7fb3ed907700 1 -- 172.21.15.88:0/1722502446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e81a2320 con 0x7fb3e8101420 2024-02-19T19:33:59.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.436+0000 7fb3ed907700 1 -- 172.21.15.88:0/1722502446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e8107900 con 0x7fb3e8101420 2024-02-19T19:33:59.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.437+0000 7fb3ec905700 1 -- 172.21.15.88:0/1722502446 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb3d800f5d0 con 0x7fb3e8101420 2024-02-19T19:33:59.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.437+0000 7fb3ec905700 1 --2- 172.21.15.88:0/1722502446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb3d006d8a0 0x7fb3d006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:59.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.437+0000 7fb3ec905700 1 -- 172.21.15.88:0/1722502446 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb3d808e9f0 con 0x7fb3e8101420 2024-02-19T19:33:59.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.437+0000 7fb3e67fc700 1 --2- 172.21.15.88:0/1722502446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb3d006d8a0 0x7fb3d006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:59.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.437+0000 7fb3e67fc700 1 --2- 172.21.15.88:0/1722502446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb3d006d8a0 0x7fb3d006fd50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8100820 tx=0x7fb3dc00a300 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:59.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.442+0000 7fb3ec905700 1 -- 172.21.15.88:0/1722502446 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb3d8090e60 con 0x7fb3e8101420 2024-02-19T19:33:59.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:59 smithi088 ceph-mon[88643]: from='client.35650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:59.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:59 smithi088 ceph-mon[88643]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:59.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:33:59 smithi088 ceph-mon[88643]: pgmap v913: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:59.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.681+0000 7fb3ed907700 1 -- 172.21.15.88:0/1722502446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb3e804f2a0 con 0x7fb3e8101420 2024-02-19T19:33:59.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.681+0000 7fb3ec905700 1 -- 172.21.15.88:0/1722502446 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fb3d8058220 con 0x7fb3e8101420 2024-02-19T19:33:59.684 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:33:59.684 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:33:59.684 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:33:59.684 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:33:59.684 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:33:59.684 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:33:59.684 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:33:59.684 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:33:59.685 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:33:59.686 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:33:59.687 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:33:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:33:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:33:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:33:59.687 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.684+0000 7fb3ed907700 1 -- 172.21.15.88:0/1722502446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb3d006d8a0 msgr2=0x7fb3d006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.684+0000 7fb3ed907700 1 --2- 172.21.15.88:0/1722502446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb3d006d8a0 0x7fb3d006fd50 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8100820 tx=0x7fb3dc00a300 comp rx=0 tx=0).stop 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.684+0000 7fb3ed907700 1 -- 172.21.15.88:0/1722502446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3e8101420 msgr2=0x7fb3e80ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.684+0000 7fb3ed907700 1 --2- 172.21.15.88:0/1722502446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3e8101420 0x7fb3e80ff080 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8006010 tx=0x7fb3d800bd70 comp rx=0 tx=0).stop 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.684+0000 7fb3ed907700 1 -- 172.21.15.88:0/1722502446 shutdown_connections 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.684+0000 7fb3ed907700 1 --2- 172.21.15.88:0/1722502446 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb3d006d8a0 0x7fb3d006fd50 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.684+0000 7fb3ed907700 1 --2- 172.21.15.88:0/1722502446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3e8101420 0x7fb3e80ff080 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.684+0000 7fb3ed907700 1 --2- 172.21.15.88:0/1722502446 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb3e8101d30 0x7fb3e80ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.684+0000 7fb3ed907700 1 -- 172.21.15.88:0/1722502446 >> 172.21.15.88:0/1722502446 conn(0x7fb3e80fac80 msgr2=0x7fb3e8104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.684+0000 7fb3ed907700 1 -- 172.21.15.88:0/1722502446 shutdown_connections 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.684+0000 7fb3ed907700 1 -- 172.21.15.88:0/1722502446 wait complete. 2024-02-19T19:33:59.688 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:33:59.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:59 smithi132 ceph-mon[70676]: from='client.35650 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:59.735 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:59 smithi132 ceph-mon[70676]: from='client.35654 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:33:59.735 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:33:59 smithi132 ceph-mon[70676]: pgmap v913: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:33:59.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.788+0000 7fded2107700 1 -- 172.21.15.88:0/550229760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdecc0687a0 msgr2=0x7fdecc068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:59.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.788+0000 7fded2107700 1 --2- 172.21.15.88:0/550229760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdecc0687a0 0x7fdecc068b70 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009910 tx=0x7fdebc009c20 comp rx=0 tx=0).stop 2024-02-19T19:33:59.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.788+0000 7fded2107700 1 -- 172.21.15.88:0/550229760 shutdown_connections 2024-02-19T19:33:59.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.788+0000 7fded2107700 1 --2- 172.21.15.88:0/550229760 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdecc069140 0x7fdecc104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.788+0000 7fded2107700 1 --2- 172.21.15.88:0/550229760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdecc0687a0 0x7fdecc068b70 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.788+0000 7fded2107700 1 -- 172.21.15.88:0/550229760 >> 172.21.15.88:0/550229760 conn(0x7fdecc075130 msgr2=0x7fdecc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.788+0000 7fded2107700 1 -- 172.21.15.88:0/550229760 shutdown_connections 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.788+0000 7fded2107700 1 -- 172.21.15.88:0/550229760 wait complete. 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fded2107700 1 Processor -- start 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fded2107700 1 -- start start 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fded2107700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdecc0687a0 0x7fdecc195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fded2107700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdecc069140 0x7fdecc196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fded2107700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc196b20 con 0x7fdecc0687a0 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fded2107700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc19a8b0 con 0x7fdecc069140 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fdecb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdecc0687a0 0x7fdecc195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fdecb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdecc0687a0 0x7fdecc195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43004/0 (socket says 172.21.15.88:43004) 2024-02-19T19:33:59.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fdecb7fe700 1 -- 172.21.15.88:0/3237592079 learned_addr learned my addr 172.21.15.88:0/3237592079 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:33:59.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fdecb7fe700 1 -- 172.21.15.88:0/3237592079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdecc069140 msgr2=0x7fdecc196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:33:59.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fdecb7fe700 1 --2- 172.21.15.88:0/3237592079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdecc069140 0x7fdecc196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:33:59.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.789+0000 7fdecb7fe700 1 -- 172.21.15.88:0/3237592079 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0095f0 con 0x7fdecc0687a0 2024-02-19T19:33:59.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.790+0000 7fdecb7fe700 1 --2- 172.21.15.88:0/3237592079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdecc0687a0 0x7fdecc195f00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0092e0 tx=0x7fdebc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:59.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.790+0000 7fdec8ff9700 1 -- 172.21.15.88:0/3237592079 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdebc01d070 con 0x7fdecc0687a0 2024-02-19T19:33:59.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.790+0000 7fdec8ff9700 1 -- 172.21.15.88:0/3237592079 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdebc005dc0 con 0x7fdecc0687a0 2024-02-19T19:33:59.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.790+0000 7fded2107700 1 -- 172.21.15.88:0/3237592079 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc19ab30 con 0x7fdecc0687a0 2024-02-19T19:33:59.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.790+0000 7fdec8ff9700 1 -- 172.21.15.88:0/3237592079 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdebc017400 con 0x7fdecc0687a0 2024-02-19T19:33:59.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.790+0000 7fded2107700 1 -- 172.21.15.88:0/3237592079 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc19b020 con 0x7fdecc0687a0 2024-02-19T19:33:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.791+0000 7fded2107700 1 -- 172.21.15.88:0/3237592079 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdecc108820 con 0x7fdecc0687a0 2024-02-19T19:33:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.791+0000 7fdec8ff9700 1 -- 172.21.15.88:0/3237592079 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdebc0175c0 con 0x7fdecc0687a0 2024-02-19T19:33:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.792+0000 7fdec8ff9700 1 --2- 172.21.15.88:0/3237592079 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdeb406d8a0 0x7fdeb406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:33:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.792+0000 7fdecaffd700 1 --2- 172.21.15.88:0/3237592079 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdeb406d8a0 0x7fdeb406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:33:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.792+0000 7fdec8ff9700 1 -- 172.21.15.88:0/3237592079 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdebc08ea80 con 0x7fdecc0687a0 2024-02-19T19:33:59.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.792+0000 7fdecaffd700 1 --2- 172.21.15.88:0/3237592079 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdeb406d8a0 0x7fdeb406fd50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdec0005950 tx=0x7fdec00058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:33:59.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:33:59.797+0000 7fdec8ff9700 1 -- 172.21.15.88:0/3237592079 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdebc059230 con 0x7fdecc0687a0 2024-02-19T19:34:00.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.012+0000 7fded2107700 1 -- 172.21.15.88:0/3237592079 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdecc19b240 con 0x7fdeb406d8a0 2024-02-19T19:34:00.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.013+0000 7fdec8ff9700 1 -- 172.21.15.88:0/3237592079 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdecc19b240 con 0x7fdeb406d8a0 2024-02-19T19:34:00.015 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:34:00.016 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:34:00.017 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:34:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.015+0000 7fded2107700 1 -- 172.21.15.88:0/3237592079 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdeb406d8a0 msgr2=0x7fdeb406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.015+0000 7fded2107700 1 --2- 172.21.15.88:0/3237592079 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdeb406d8a0 0x7fdeb406fd50 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdec0005950 tx=0x7fdec00058e0 comp rx=0 tx=0).stop 2024-02-19T19:34:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.015+0000 7fded2107700 1 -- 172.21.15.88:0/3237592079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdecc0687a0 msgr2=0x7fdecc195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.015+0000 7fded2107700 1 --2- 172.21.15.88:0/3237592079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdecc0687a0 0x7fdecc195f00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0092e0 tx=0x7fdebc00be20 comp rx=0 tx=0).stop 2024-02-19T19:34:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.015+0000 7fded2107700 1 -- 172.21.15.88:0/3237592079 shutdown_connections 2024-02-19T19:34:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.015+0000 7fded2107700 1 --2- 172.21.15.88:0/3237592079 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdeb406d8a0 0x7fdeb406fd50 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.015+0000 7fded2107700 1 --2- 172.21.15.88:0/3237592079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdecc0687a0 0x7fdecc195f00 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.015+0000 7fded2107700 1 --2- 172.21.15.88:0/3237592079 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdecc069140 0x7fdecc196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.015+0000 7fded2107700 1 -- 172.21.15.88:0/3237592079 >> 172.21.15.88:0/3237592079 conn(0x7fdecc075130 msgr2=0x7fdecc0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:00.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.016+0000 7fded2107700 1 -- 172.21.15.88:0/3237592079 shutdown_connections 2024-02-19T19:34:00.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.016+0000 7fded2107700 1 -- 172.21.15.88:0/3237592079 wait complete. 2024-02-19T19:34:00.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.119+0000 7f72fb5d8700 1 -- 172.21.15.88:0/2579589194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72f40ff3c0 msgr2=0x7f72f40ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:00.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.119+0000 7f72fb5d8700 1 --2- 172.21.15.88:0/2579589194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72f40ff3c0 0x7f72f40ff790 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f72f0009910 tx=0x7f72f0009c20 comp rx=0 tx=0).stop 2024-02-19T19:34:00.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 -- 172.21.15.88:0/2579589194 shutdown_connections 2024-02-19T19:34:00.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 --2- 172.21.15.88:0/2579589194 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72f40ffd60 0x7f72f4103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:00.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 --2- 172.21.15.88:0/2579589194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72f40ff3c0 0x7f72f40ff790 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:00.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 -- 172.21.15.88:0/2579589194 >> 172.21.15.88:0/2579589194 conn(0x7f72f40fac80 msgr2=0x7f72f40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:00.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 -- 172.21.15.88:0/2579589194 shutdown_connections 2024-02-19T19:34:00.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 -- 172.21.15.88:0/2579589194 wait complete. 2024-02-19T19:34:00.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 Processor -- start 2024-02-19T19:34:00.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 -- start start 2024-02-19T19:34:00.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72f40ff3c0 0x7f72f4195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:00.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72f40ffd60 0x7f72f4196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f4196b40 con 0x7f72f40ff3c0 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.120+0000 7f72fb5d8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f419a8d0 con 0x7f72f40ffd60 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72f9374700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72f40ff3c0 0x7f72f4195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72f9374700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72f40ff3c0 0x7f72f4195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43020/0 (socket says 172.21.15.88:43020) 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72f9374700 1 -- 172.21.15.88:0/1662063919 learned_addr learned my addr 172.21.15.88:0/1662063919 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72f8b73700 1 --2- 172.21.15.88:0/1662063919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72f40ffd60 0x7f72f4196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72f9374700 1 -- 172.21.15.88:0/1662063919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72f40ffd60 msgr2=0x7f72f4196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72f9374700 1 --2- 172.21.15.88:0/1662063919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72f40ffd60 0x7f72f4196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72f9374700 1 -- 172.21.15.88:0/1662063919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f00095f0 con 0x7f72f40ff3c0 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72f9374700 1 --2- 172.21.15.88:0/1662063919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72f40ff3c0 0x7f72f4195f20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f72f0006010 tx=0x7f72f000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72ea7fc700 1 -- 172.21.15.88:0/1662063919 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72f001d070 con 0x7f72f40ff3c0 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72ea7fc700 1 -- 172.21.15.88:0/1662063919 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72f000f460 con 0x7f72f40ff3c0 2024-02-19T19:34:00.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72fb5d8700 1 -- 172.21.15.88:0/1662063919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f419ab50 con 0x7f72f40ff3c0 2024-02-19T19:34:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72ea7fc700 1 -- 172.21.15.88:0/1662063919 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72f0017630 con 0x7f72f40ff3c0 2024-02-19T19:34:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.121+0000 7f72fb5d8700 1 -- 172.21.15.88:0/1662063919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f419b040 con 0x7f72f40ff3c0 2024-02-19T19:34:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.122+0000 7f72fb5d8700 1 -- 172.21.15.88:0/1662063919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72f41086e0 con 0x7f72f40ff3c0 2024-02-19T19:34:00.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.123+0000 7f72ea7fc700 1 -- 172.21.15.88:0/1662063919 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f72f000f5d0 con 0x7f72f40ff3c0 2024-02-19T19:34:00.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.123+0000 7f72ea7fc700 1 --2- 172.21.15.88:0/1662063919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72e006d8a0 0x7f72e006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:00.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.123+0000 7f72ea7fc700 1 -- 172.21.15.88:0/1662063919 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f72f008e980 con 0x7f72f40ff3c0 2024-02-19T19:34:00.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.123+0000 7f72f8b73700 1 --2- 172.21.15.88:0/1662063919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72e006d8a0 0x7f72e006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:00.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.124+0000 7f72f8b73700 1 --2- 172.21.15.88:0/1662063919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72e006d8a0 0x7f72e006fd50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f72e4005950 tx=0x7f72e40058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:00.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.128+0000 7f72ea7fc700 1 -- 172.21.15.88:0/1662063919 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f72f00581b0 con 0x7f72f40ff3c0 2024-02-19T19:34:00.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.404+0000 7f72fb5d8700 1 -- 172.21.15.88:0/1662063919 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f72f41080c0 con 0x7f72f40ff3c0 2024-02-19T19:34:00.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.405+0000 7f72ea7fc700 1 -- 172.21.15.88:0/1662063919 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f72f005b870 con 0x7f72f40ff3c0 2024-02-19T19:34:00.407 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:34:00.407 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:34:00.407 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:34:00.408 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:34:00.409 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:34:00.409 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:34:00.409 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:34:00.409 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:34:00.409 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:34:00.409 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:34:00.409 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:34:00.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.406+0000 7f72fb5d8700 1 -- 172.21.15.88:0/1662063919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72e006d8a0 msgr2=0x7f72e006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:00.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.406+0000 7f72fb5d8700 1 --2- 172.21.15.88:0/1662063919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72e006d8a0 0x7f72e006fd50 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f72e4005950 tx=0x7f72e40058e0 comp rx=0 tx=0).stop 2024-02-19T19:34:00.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.406+0000 7f72fb5d8700 1 -- 172.21.15.88:0/1662063919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72f40ff3c0 msgr2=0x7f72f4195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:00.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.406+0000 7f72fb5d8700 1 --2- 172.21.15.88:0/1662063919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72f40ff3c0 0x7f72f4195f20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f72f0006010 tx=0x7f72f000be20 comp rx=0 tx=0).stop 2024-02-19T19:34:00.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.407+0000 7f72fb5d8700 1 -- 172.21.15.88:0/1662063919 shutdown_connections 2024-02-19T19:34:00.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.407+0000 7f72fb5d8700 1 --2- 172.21.15.88:0/1662063919 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f72e006d8a0 0x7f72e006fd50 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:00.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.407+0000 7f72fb5d8700 1 --2- 172.21.15.88:0/1662063919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72f40ff3c0 0x7f72f4195f20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:00.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.407+0000 7f72fb5d8700 1 --2- 172.21.15.88:0/1662063919 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f72f40ffd60 0x7f72f4196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:00.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.407+0000 7f72fb5d8700 1 -- 172.21.15.88:0/1662063919 >> 172.21.15.88:0/1662063919 conn(0x7f72f40fac80 msgr2=0x7f72f4069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:00.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.407+0000 7f72fb5d8700 1 -- 172.21.15.88:0/1662063919 shutdown_connections 2024-02-19T19:34:00.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:00.407+0000 7f72fb5d8700 1 -- 172.21.15.88:0/1662063919 wait complete. 2024-02-19T19:34:00.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:00 smithi088 ceph-mon[88643]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:00.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:00 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3755539243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:34:00.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:00 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1722502446' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:34:00.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:00 smithi132 ceph-mon[70676]: from='client.35658 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:00.735 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:00 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3755539243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:34:00.735 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:00 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1722502446' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:34:01.562 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:01 smithi132 ceph-mon[70676]: from='client.35670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:01.562 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:01 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1662063919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:34:01.562 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:01 smithi132 ceph-mon[70676]: pgmap v914: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:01.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:01 smithi088 ceph-mon[88643]: from='client.35670 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:01.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:01 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1662063919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:34:01.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:01 smithi088 ceph-mon[88643]: pgmap v914: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:03.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:02 smithi088 ceph-mon[88643]: pgmap v915: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:03.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:02 smithi132 ceph-mon[70676]: pgmap v915: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:05.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:04 smithi088 ceph-mon[88643]: pgmap v916: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:05.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:04 smithi132 ceph-mon[70676]: pgmap v916: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:07.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:06 smithi088 ceph-mon[88643]: pgmap v917: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:07.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:06 smithi132 ceph-mon[70676]: pgmap v917: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:09.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:08 smithi088 ceph-mon[88643]: pgmap v918: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:09.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:08 smithi132 ceph-mon[70676]: pgmap v918: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:11.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:10 smithi088 ceph-mon[88643]: pgmap v919: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:11.146 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:10 smithi132 ceph-mon[70676]: pgmap v919: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:13.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:12 smithi088 ceph-mon[88643]: pgmap v920: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:13.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:12 smithi132 ceph-mon[70676]: pgmap v920: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:14.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:34:14.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:34:14.984 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:14 smithi088 ceph-mon[88643]: pgmap v921: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:15.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:14 smithi132 ceph-mon[70676]: pgmap v921: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:17.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:16 smithi088 ceph-mon[88643]: pgmap v922: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:17.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:16 smithi132 ceph-mon[70676]: pgmap v922: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:19.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:18 smithi088 ceph-mon[88643]: pgmap v923: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:18 smithi132 ceph-mon[70676]: pgmap v923: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:21.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:20 smithi088 ceph-mon[88643]: pgmap v924: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:21.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:20 smithi132 ceph-mon[70676]: pgmap v924: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:23.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:22 smithi088 ceph-mon[88643]: pgmap v925: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:22 smithi132 ceph-mon[70676]: pgmap v925: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:25.037 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:24 smithi132 ceph-mon[70676]: pgmap v926: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:25.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:24 smithi088 ceph-mon[88643]: pgmap v926: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:27.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:26 smithi088 ceph-mon[88643]: pgmap v927: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:27.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:26 smithi132 ceph-mon[70676]: pgmap v927: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:28.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:34:28.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:34:29.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:29 smithi088 ceph-mon[88643]: pgmap v928: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:29.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:29 smithi132 ceph-mon[70676]: pgmap v928: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:30.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.523+0000 7f462bd4d700 1 -- 172.21.15.88:0/694533130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4624100e80 msgr2=0x7f4624104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:30.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.523+0000 7f462bd4d700 1 --2- 172.21.15.88:0/694533130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4624100e80 0x7f4624104e50 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4614009910 tx=0x7f4614009c20 comp rx=0 tx=0).stop 2024-02-19T19:34:30.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.523+0000 7f462bd4d700 1 -- 172.21.15.88:0/694533130 shutdown_connections 2024-02-19T19:34:30.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.523+0000 7f462bd4d700 1 --2- 172.21.15.88:0/694533130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4624100e80 0x7f4624104e50 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:30.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.523+0000 7f462bd4d700 1 --2- 172.21.15.88:0/694533130 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f46241004e0 0x7f46241008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:30.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.523+0000 7f462bd4d700 1 -- 172.21.15.88:0/694533130 >> 172.21.15.88:0/694533130 conn(0x7f46240fbda0 msgr2=0x7f46240fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:30.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.523+0000 7f462bd4d700 1 -- 172.21.15.88:0/694533130 shutdown_connections 2024-02-19T19:34:30.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.523+0000 7f462bd4d700 1 -- 172.21.15.88:0/694533130 wait complete. 2024-02-19T19:34:30.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.523+0000 7f462bd4d700 1 Processor -- start 2024-02-19T19:34:30.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.523+0000 7f462bd4d700 1 -- start start 2024-02-19T19:34:30.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f462bd4d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46241004e0 0x7f4624195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:30.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f462bd4d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4624100e80 0x7f4624196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:30.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f462bd4d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4624196b30 con 0x7f46241004e0 2024-02-19T19:34:30.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f462bd4d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f462419a8c0 con 0x7f4624100e80 2024-02-19T19:34:30.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f4629ae9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46241004e0 0x7f4624195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f4629ae9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46241004e0 0x7f4624195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52582/0 (socket says 172.21.15.88:52582) 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f4629ae9700 1 -- 172.21.15.88:0/2533037804 learned_addr learned my addr 172.21.15.88:0/2533037804 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f46292e8700 1 --2- 172.21.15.88:0/2533037804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4624100e80 0x7f4624196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f4629ae9700 1 -- 172.21.15.88:0/2533037804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4624100e80 msgr2=0x7f4624196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f4629ae9700 1 --2- 172.21.15.88:0/2533037804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4624100e80 0x7f4624196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f4629ae9700 1 -- 172.21.15.88:0/2533037804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46140095f0 con 0x7f46241004e0 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f4629ae9700 1 --2- 172.21.15.88:0/2533037804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46241004e0 0x7f4624195f10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f462000d8d0 tx=0x7f462000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f461affd700 1 -- 172.21.15.88:0/2533037804 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46200098a0 con 0x7f46241004e0 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f461affd700 1 -- 172.21.15.88:0/2533037804 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4620010460 con 0x7f46241004e0 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f462bd4d700 1 -- 172.21.15.88:0/2533037804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f462419aba0 con 0x7f46241004e0 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f461affd700 1 -- 172.21.15.88:0/2533037804 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f462000f5f0 con 0x7f46241004e0 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.524+0000 7f462bd4d700 1 -- 172.21.15.88:0/2533037804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462419b0f0 con 0x7f46241004e0 2024-02-19T19:34:30.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.525+0000 7f462bd4d700 1 -- 172.21.15.88:0/2533037804 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4624108850 con 0x7f46241004e0 2024-02-19T19:34:30.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.526+0000 7f461affd700 1 -- 172.21.15.88:0/2533037804 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4620009a00 con 0x7f46241004e0 2024-02-19T19:34:30.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.526+0000 7f461affd700 1 --2- 172.21.15.88:0/2533037804 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f461006d8a0 0x7f461006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:30.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.526+0000 7f461affd700 1 -- 172.21.15.88:0/2533037804 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f462008d400 con 0x7f46241004e0 2024-02-19T19:34:30.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.526+0000 7f46292e8700 1 --2- 172.21.15.88:0/2533037804 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f461006d8a0 0x7f461006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:30.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.527+0000 7f46292e8700 1 --2- 172.21.15.88:0/2533037804 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f461006d8a0 0x7f461006fd50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4624197530 tx=0x7f461401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:30.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.531+0000 7f461affd700 1 -- 172.21.15.88:0/2533037804 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4620056bb0 con 0x7f46241004e0 2024-02-19T19:34:30.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.750+0000 7f462bd4d700 1 -- 172.21.15.88:0/2533037804 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f462419b310 con 0x7f461006d8a0 2024-02-19T19:34:30.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.751+0000 7f461affd700 1 -- 172.21.15.88:0/2533037804 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f462419b310 con 0x7f461006d8a0 2024-02-19T19:34:30.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.753+0000 7f462bd4d700 1 -- 172.21.15.88:0/2533037804 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f461006d8a0 msgr2=0x7f461006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:30.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.753+0000 7f462bd4d700 1 --2- 172.21.15.88:0/2533037804 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f461006d8a0 0x7f461006fd50 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4624197530 tx=0x7f461401a040 comp rx=0 tx=0).stop 2024-02-19T19:34:30.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.753+0000 7f462bd4d700 1 -- 172.21.15.88:0/2533037804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46241004e0 msgr2=0x7f4624195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.753+0000 7f462bd4d700 1 --2- 172.21.15.88:0/2533037804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46241004e0 0x7f4624195f10 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f462000d8d0 tx=0x7f462000dbe0 comp rx=0 tx=0).stop 2024-02-19T19:34:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.753+0000 7f462bd4d700 1 -- 172.21.15.88:0/2533037804 shutdown_connections 2024-02-19T19:34:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.753+0000 7f462bd4d700 1 --2- 172.21.15.88:0/2533037804 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f461006d8a0 0x7f461006fd50 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.753+0000 7f462bd4d700 1 --2- 172.21.15.88:0/2533037804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46241004e0 0x7f4624195f10 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.753+0000 7f462bd4d700 1 --2- 172.21.15.88:0/2533037804 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4624100e80 0x7f4624196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.753+0000 7f462bd4d700 1 -- 172.21.15.88:0/2533037804 >> 172.21.15.88:0/2533037804 conn(0x7f46240fbda0 msgr2=0x7f46240fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.753+0000 7f462bd4d700 1 -- 172.21.15.88:0/2533037804 shutdown_connections 2024-02-19T19:34:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.753+0000 7f462bd4d700 1 -- 172.21.15.88:0/2533037804 wait complete. 2024-02-19T19:34:30.766 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:34:30.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.859+0000 7f6c37fb9700 1 -- 172.21.15.88:0/4274028566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c300ff3c0 msgr2=0x7f6c300ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:30.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.859+0000 7f6c37fb9700 1 --2- 172.21.15.88:0/4274028566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c300ff3c0 0x7f6c300ff790 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800b0a0 tx=0x7f6c2800b3b0 comp rx=0 tx=0).stop 2024-02-19T19:34:30.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.859+0000 7f6c37fb9700 1 -- 172.21.15.88:0/4274028566 shutdown_connections 2024-02-19T19:34:30.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.859+0000 7f6c37fb9700 1 --2- 172.21.15.88:0/4274028566 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c300ffd60 0x7f6c30103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:30.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.859+0000 7f6c37fb9700 1 --2- 172.21.15.88:0/4274028566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c300ff3c0 0x7f6c300ff790 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:30.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.859+0000 7f6c37fb9700 1 -- 172.21.15.88:0/4274028566 >> 172.21.15.88:0/4274028566 conn(0x7f6c300fac60 msgr2=0x7f6c300fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:30.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.859+0000 7f6c37fb9700 1 -- 172.21.15.88:0/4274028566 shutdown_connections 2024-02-19T19:34:30.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.859+0000 7f6c37fb9700 1 -- 172.21.15.88:0/4274028566 wait complete. 2024-02-19T19:34:30.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.859+0000 7f6c37fb9700 1 Processor -- start 2024-02-19T19:34:30.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.859+0000 7f6c37fb9700 1 -- start start 2024-02-19T19:34:30.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c37fb9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c300ff3c0 0x7f6c30196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:30.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c37fb9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c300ffd60 0x7f6c301972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:30.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c37fb9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c301978d0 con 0x7f6c300ff3c0 2024-02-19T19:34:30.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c37fb9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c30197a10 con 0x7f6c300ffd60 2024-02-19T19:34:30.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c35d55700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c300ff3c0 0x7f6c30196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:30.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c35d55700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c300ff3c0 0x7f6c30196d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52588/0 (socket says 172.21.15.88:52588) 2024-02-19T19:34:30.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c35d55700 1 -- 172.21.15.88:0/2523835959 learned_addr learned my addr 172.21.15.88:0/2523835959 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:34:30.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c35554700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c300ffd60 0x7f6c301972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c35d55700 1 -- 172.21.15.88:0/2523835959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c300ffd60 msgr2=0x7f6c301972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c35d55700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c300ffd60 0x7f6c301972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c35d55700 1 -- 172.21.15.88:0/2523835959 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c28009d00 con 0x7f6c300ff3c0 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c35554700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c300ffd60 0x7f6c301972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c35d55700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c300ff3c0 0x7f6c30196d70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6c28000f80 tx=0x7f6c2800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c2effd700 1 -- 172.21.15.88:0/2523835959 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c2800e040 con 0x7f6c300ff3c0 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c37fb9700 1 -- 172.21.15.88:0/2523835959 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3019fde0 con 0x7f6c300ff3c0 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c2effd700 1 -- 172.21.15.88:0/2523835959 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c28003de0 con 0x7f6c300ff3c0 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c2effd700 1 -- 172.21.15.88:0/2523835959 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c2801d790 con 0x7f6c300ff3c0 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.860+0000 7f6c37fb9700 1 -- 172.21.15.88:0/2523835959 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c301a01f0 con 0x7f6c300ff3c0 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.861+0000 7f6c37fb9700 1 -- 172.21.15.88:0/2523835959 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c301076d0 con 0x7f6c300ff3c0 2024-02-19T19:34:30.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.862+0000 7f6c2effd700 1 -- 172.21.15.88:0/2523835959 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6c28004040 con 0x7f6c300ff3c0 2024-02-19T19:34:30.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.862+0000 7f6c2effd700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 0x7f6c1806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:30.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.862+0000 7f6c35554700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 0x7f6c1806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:30.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.862+0000 7f6c2effd700 1 -- 172.21.15.88:0/2523835959 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6c2808ed20 con 0x7f6c300ff3c0 2024-02-19T19:34:30.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.863+0000 7f6c35554700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 0x7f6c1806fd50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6c24005fd0 tx=0x7f6c24005f00 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:30.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:30.867+0000 7f6c2effd700 1 -- 172.21.15.88:0/2523835959 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6c280584d0 con 0x7f6c300ff3c0 2024-02-19T19:34:31.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.080+0000 7f6c37fb9700 1 -- 172.21.15.88:0/2523835959 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c30061190 con 0x7f6c1806d8a0 2024-02-19T19:34:31.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.082+0000 7f6c2effd700 1 -- 172.21.15.88:0/2523835959 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6c30061190 con 0x7f6c1806d8a0 2024-02-19T19:34:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.084+0000 7f6c37fb9700 1 -- 172.21.15.88:0/2523835959 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 msgr2=0x7f6c1806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.084+0000 7f6c37fb9700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 0x7f6c1806fd50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6c24005fd0 tx=0x7f6c24005f00 comp rx=0 tx=0).stop 2024-02-19T19:34:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.084+0000 7f6c37fb9700 1 -- 172.21.15.88:0/2523835959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c300ff3c0 msgr2=0x7f6c30196d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.084+0000 7f6c37fb9700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c300ff3c0 0x7f6c30196d70 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6c28000f80 tx=0x7f6c2800cdd0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.084+0000 7f6c37fb9700 1 -- 172.21.15.88:0/2523835959 shutdown_connections 2024-02-19T19:34:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.084+0000 7f6c37fb9700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6c1806d8a0 0x7f6c1806fd50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.084+0000 7f6c37fb9700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c300ff3c0 0x7f6c30196d70 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.084+0000 7f6c37fb9700 1 --2- 172.21.15.88:0/2523835959 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6c300ffd60 0x7f6c301972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.084+0000 7f6c37fb9700 1 -- 172.21.15.88:0/2523835959 >> 172.21.15.88:0/2523835959 conn(0x7f6c300fac60 msgr2=0x7f6c300fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.084+0000 7f6c37fb9700 1 -- 172.21.15.88:0/2523835959 shutdown_connections 2024-02-19T19:34:31.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.084+0000 7f6c37fb9700 1 -- 172.21.15.88:0/2523835959 wait complete. 2024-02-19T19:34:31.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.187+0000 7f17204ff700 1 -- 172.21.15.88:0/2861223460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17180ffd60 msgr2=0x7f1718103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.187+0000 7f17204ff700 1 --2- 172.21.15.88:0/2861223460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17180ffd60 0x7f1718103d30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1714009910 tx=0x7f1714009c20 comp rx=0 tx=0).stop 2024-02-19T19:34:31.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.188+0000 7f17204ff700 1 -- 172.21.15.88:0/2861223460 shutdown_connections 2024-02-19T19:34:31.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.188+0000 7f17204ff700 1 --2- 172.21.15.88:0/2861223460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17180ffd60 0x7f1718103d30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.188+0000 7f17204ff700 1 --2- 172.21.15.88:0/2861223460 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17180ff3c0 0x7f17180ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.188+0000 7f17204ff700 1 -- 172.21.15.88:0/2861223460 >> 172.21.15.88:0/2861223460 conn(0x7f17180fac80 msgr2=0x7f17180fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:31.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.188+0000 7f17204ff700 1 -- 172.21.15.88:0/2861223460 shutdown_connections 2024-02-19T19:34:31.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.188+0000 7f17204ff700 1 -- 172.21.15.88:0/2861223460 wait complete. 2024-02-19T19:34:31.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.188+0000 7f17204ff700 1 Processor -- start 2024-02-19T19:34:31.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.188+0000 7f17204ff700 1 -- start start 2024-02-19T19:34:31.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f17204ff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17180ff3c0 0x7f1718196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:31.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f17204ff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17180ffd60 0x7f17181972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:31.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f17204ff700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17181978e0 con 0x7f17180ff3c0 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f17204ff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1718197a20 con 0x7f17180ffd60 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f171e29b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17180ff3c0 0x7f1718196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f171e29b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17180ff3c0 0x7f1718196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52598/0 (socket says 172.21.15.88:52598) 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f171e29b700 1 -- 172.21.15.88:0/2310800879 learned_addr learned my addr 172.21.15.88:0/2310800879 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f171da9a700 1 --2- 172.21.15.88:0/2310800879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17180ffd60 0x7f17181972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f171e29b700 1 -- 172.21.15.88:0/2310800879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17180ffd60 msgr2=0x7f17181972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f171e29b700 1 --2- 172.21.15.88:0/2310800879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17180ffd60 0x7f17181972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f171e29b700 1 -- 172.21.15.88:0/2310800879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17140095f0 con 0x7f17180ff3c0 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f171e29b700 1 --2- 172.21.15.88:0/2310800879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17180ff3c0 0x7f1718196d80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f170800c8a0 tx=0x7f170800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f170f7fe700 1 -- 172.21.15.88:0/2310800879 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f170800cea0 con 0x7f17180ff3c0 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f17204ff700 1 -- 172.21.15.88:0/2310800879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f171819fdf0 con 0x7f17180ff3c0 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f170f7fe700 1 -- 172.21.15.88:0/2310800879 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1708004830 con 0x7f17180ff3c0 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.189+0000 7f170f7fe700 1 -- 172.21.15.88:0/2310800879 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17080055b0 con 0x7f17180ff3c0 2024-02-19T19:34:31.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.190+0000 7f17204ff700 1 -- 172.21.15.88:0/2310800879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17181a02b0 con 0x7f17180ff3c0 2024-02-19T19:34:31.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.191+0000 7f17204ff700 1 -- 172.21.15.88:0/2310800879 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1718107730 con 0x7f17180ff3c0 2024-02-19T19:34:31.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.191+0000 7f170f7fe700 1 -- 172.21.15.88:0/2310800879 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f17080049a0 con 0x7f17180ff3c0 2024-02-19T19:34:31.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.192+0000 7f170f7fe700 1 --2- 172.21.15.88:0/2310800879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f170406d8a0 0x7f170406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:31.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.192+0000 7f170f7fe700 1 -- 172.21.15.88:0/2310800879 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f170808d1d0 con 0x7f17180ff3c0 2024-02-19T19:34:31.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.192+0000 7f171da9a700 1 --2- 172.21.15.88:0/2310800879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f170406d8a0 0x7f170406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:31.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.192+0000 7f171da9a700 1 --2- 172.21.15.88:0/2310800879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f170406d8a0 0x7f170406fd50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f17140059d0 tx=0x7f1714005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:31.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.197+0000 7f170f7fe700 1 -- 172.21.15.88:0/2310800879 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1708056980 con 0x7f17180ff3c0 2024-02-19T19:34:31.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.405+0000 7f17204ff700 1 -- 172.21.15.88:0/2310800879 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1718061190 con 0x7f170406d8a0 2024-02-19T19:34:31.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.415+0000 7f170f7fe700 1 -- 172.21.15.88:0/2310800879 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1718061190 con 0x7f170406d8a0 2024-02-19T19:34:31.417 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:34:31.417 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (31m) 8m ago 42m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (42m) 8m ago 42m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (24m) 8m ago 40m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (27m) 8m ago 42m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (27m) 8m ago 40m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (31m) 8m ago 41m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (23m) 8m ago 35m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (23m) 8m ago 35m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (23m) 8m ago 35m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (8m) 8m ago 35m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (33m) 8m ago 43m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (32m) 8m ago 40m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (29m) 8m ago 44m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (29m) 8m ago 40m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:34:31.418 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (32m) 8m ago 42m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:34:31.419 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (32m) 8m ago 40m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:34:31.419 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (27m) 8m ago 39m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:34:31.419 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (26m) 8m ago 39m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:34:31.419 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (25m) 8m ago 38m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:34:31.419 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (25m) 8m ago 38m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:34:31.419 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (24m) 8m ago 37m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:34:31.419 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (24m) 8m ago 37m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:34:31.419 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (32m) 8m ago 40m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:34:31.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.417+0000 7f17204ff700 1 -- 172.21.15.88:0/2310800879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f170406d8a0 msgr2=0x7f170406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.417+0000 7f17204ff700 1 --2- 172.21.15.88:0/2310800879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f170406d8a0 0x7f170406fd50 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f17140059d0 tx=0x7f1714005960 comp rx=0 tx=0).stop 2024-02-19T19:34:31.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.417+0000 7f17204ff700 1 -- 172.21.15.88:0/2310800879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17180ff3c0 msgr2=0x7f1718196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.417+0000 7f17204ff700 1 --2- 172.21.15.88:0/2310800879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17180ff3c0 0x7f1718196d80 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f170800c8a0 tx=0x7f170800cc60 comp rx=0 tx=0).stop 2024-02-19T19:34:31.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.417+0000 7f17204ff700 1 -- 172.21.15.88:0/2310800879 shutdown_connections 2024-02-19T19:34:31.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.417+0000 7f17204ff700 1 --2- 172.21.15.88:0/2310800879 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f170406d8a0 0x7f170406fd50 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.417+0000 7f17204ff700 1 --2- 172.21.15.88:0/2310800879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17180ff3c0 0x7f1718196d80 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.417+0000 7f17204ff700 1 --2- 172.21.15.88:0/2310800879 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17180ffd60 0x7f17181972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.417+0000 7f17204ff700 1 -- 172.21.15.88:0/2310800879 >> 172.21.15.88:0/2310800879 conn(0x7f17180fac80 msgr2=0x7f17180fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:31.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.418+0000 7f17204ff700 1 -- 172.21.15.88:0/2310800879 shutdown_connections 2024-02-19T19:34:31.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.418+0000 7f17204ff700 1 -- 172.21.15.88:0/2310800879 wait complete. 2024-02-19T19:34:31.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:31 smithi132 ceph-mon[70676]: pgmap v929: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:31.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.519+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/3064391846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e340ffd60 msgr2=0x7f3e34103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.519+0000 7f3e3a2a4700 1 --2- 172.21.15.88:0/3064391846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e340ffd60 0x7f3e34103d30 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009910 tx=0x7f3e24009c20 comp rx=0 tx=0).stop 2024-02-19T19:34:31.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.519+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/3064391846 shutdown_connections 2024-02-19T19:34:31.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.519+0000 7f3e3a2a4700 1 --2- 172.21.15.88:0/3064391846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e340ffd60 0x7f3e34103d30 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.519+0000 7f3e3a2a4700 1 --2- 172.21.15.88:0/3064391846 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e340ff3c0 0x7f3e340ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.519+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/3064391846 >> 172.21.15.88:0/3064391846 conn(0x7f3e340fac80 msgr2=0x7f3e340fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:31.522 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:31 smithi088 ceph-mon[88643]: pgmap v929: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.519+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/3064391846 shutdown_connections 2024-02-19T19:34:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.519+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/3064391846 wait complete. 2024-02-19T19:34:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e3a2a4700 1 Processor -- start 2024-02-19T19:34:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e3a2a4700 1 -- start start 2024-02-19T19:34:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e3a2a4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e340ff3c0 0x7f3e34194e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e3a2a4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e340ffd60 0x7f3e34195350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e3a2a4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e34195a30 con 0x7f3e340ff3c0 2024-02-19T19:34:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e3a2a4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e341997c0 con 0x7f3e340ffd60 2024-02-19T19:34:31.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e33fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e340ff3c0 0x7f3e34194e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e33fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e340ff3c0 0x7f3e34194e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52606/0 (socket says 172.21.15.88:52606) 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e33fff700 1 -- 172.21.15.88:0/1748320402 learned_addr learned my addr 172.21.15.88:0/1748320402 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e337fe700 1 --2- 172.21.15.88:0/1748320402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e340ffd60 0x7f3e34195350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e33fff700 1 -- 172.21.15.88:0/1748320402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e340ffd60 msgr2=0x7f3e34195350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.520+0000 7f3e33fff700 1 --2- 172.21.15.88:0/1748320402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e340ffd60 0x7f3e34195350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.521+0000 7f3e33fff700 1 -- 172.21.15.88:0/1748320402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e240095f0 con 0x7f3e340ff3c0 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.521+0000 7f3e33fff700 1 --2- 172.21.15.88:0/1748320402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e340ff3c0 0x7f3e34194e10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3e28007b50 tx=0x7f3e28007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.521+0000 7f3e317fa700 1 -- 172.21.15.88:0/1748320402 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e28011070 con 0x7f3e340ff3c0 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.521+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/1748320402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e34199a40 con 0x7f3e340ff3c0 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.521+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/1748320402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e34199f90 con 0x7f3e340ff3c0 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.521+0000 7f3e317fa700 1 -- 172.21.15.88:0/1748320402 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e2800c2e0 con 0x7f3e340ff3c0 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.521+0000 7f3e317fa700 1 -- 172.21.15.88:0/1748320402 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e2800e800 con 0x7f3e340ff3c0 2024-02-19T19:34:31.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.522+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/1748320402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e20005320 con 0x7f3e340ff3c0 2024-02-19T19:34:31.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.523+0000 7f3e317fa700 1 -- 172.21.15.88:0/1748320402 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3e28005340 con 0x7f3e340ff3c0 2024-02-19T19:34:31.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.523+0000 7f3e317fa700 1 --2- 172.21.15.88:0/1748320402 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e1c06d8a0 0x7f3e1c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:31.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.523+0000 7f3e317fa700 1 -- 172.21.15.88:0/1748320402 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3e2808eaa0 con 0x7f3e340ff3c0 2024-02-19T19:34:31.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.523+0000 7f3e337fe700 1 --2- 172.21.15.88:0/1748320402 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e1c06d8a0 0x7f3e1c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:31.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.523+0000 7f3e337fe700 1 --2- 172.21.15.88:0/1748320402 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e1c06d8a0 0x7f3e1c06fd50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3e34196430 tx=0x7f3e2401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:31.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.528+0000 7f3e317fa700 1 -- 172.21.15.88:0/1748320402 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3e280582d0 con 0x7f3e340ff3c0 2024-02-19T19:34:31.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.802+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/1748320402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3e20006200 con 0x7f3e340ff3c0 2024-02-19T19:34:31.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.803+0000 7f3e317fa700 1 -- 172.21.15.88:0/1748320402 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f3e2805b990 con 0x7f3e340ff3c0 2024-02-19T19:34:31.805 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:34:31.805 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:34:31.806 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:34:31.807 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:34:31.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.805+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/1748320402 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e1c06d8a0 msgr2=0x7f3e1c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.805+0000 7f3e3a2a4700 1 --2- 172.21.15.88:0/1748320402 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e1c06d8a0 0x7f3e1c06fd50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3e34196430 tx=0x7f3e2401a040 comp rx=0 tx=0).stop 2024-02-19T19:34:31.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.805+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/1748320402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e340ff3c0 msgr2=0x7f3e34194e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.805+0000 7f3e3a2a4700 1 --2- 172.21.15.88:0/1748320402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e340ff3c0 0x7f3e34194e10 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3e28007b50 tx=0x7f3e28007f10 comp rx=0 tx=0).stop 2024-02-19T19:34:31.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.806+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/1748320402 shutdown_connections 2024-02-19T19:34:31.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.806+0000 7f3e3a2a4700 1 --2- 172.21.15.88:0/1748320402 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3e1c06d8a0 0x7f3e1c06fd50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.806+0000 7f3e3a2a4700 1 --2- 172.21.15.88:0/1748320402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e340ff3c0 0x7f3e34194e10 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.806+0000 7f3e3a2a4700 1 --2- 172.21.15.88:0/1748320402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3e340ffd60 0x7f3e34195350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.806+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/1748320402 >> 172.21.15.88:0/1748320402 conn(0x7f3e340fac80 msgr2=0x7f3e340fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:31.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.806+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/1748320402 shutdown_connections 2024-02-19T19:34:31.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.806+0000 7f3e3a2a4700 1 -- 172.21.15.88:0/1748320402 wait complete. 2024-02-19T19:34:31.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.909+0000 7f2bbb610700 1 -- 172.21.15.88:0/4178934688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bb41004e0 msgr2=0x7f2bb41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.909+0000 7f2bbb610700 1 --2- 172.21.15.88:0/4178934688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bb41004e0 0x7f2bb41008b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009910 tx=0x7f2bb0009c20 comp rx=0 tx=0).stop 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.909+0000 7f2bbb610700 1 -- 172.21.15.88:0/4178934688 shutdown_connections 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.909+0000 7f2bbb610700 1 --2- 172.21.15.88:0/4178934688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb4100e80 0x7f2bb4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.909+0000 7f2bbb610700 1 --2- 172.21.15.88:0/4178934688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bb41004e0 0x7f2bb41008b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.909+0000 7f2bbb610700 1 -- 172.21.15.88:0/4178934688 >> 172.21.15.88:0/4178934688 conn(0x7f2bb40fbda0 msgr2=0x7f2bb40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.909+0000 7f2bbb610700 1 -- 172.21.15.88:0/4178934688 shutdown_connections 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.909+0000 7f2bbb610700 1 -- 172.21.15.88:0/4178934688 wait complete. 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.910+0000 7f2bbb610700 1 Processor -- start 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.910+0000 7f2bbb610700 1 -- start start 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.910+0000 7f2bbb610700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb41004e0 0x7f2bb4195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.910+0000 7f2bbb610700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bb4100e80 0x7f2bb41964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.910+0000 7f2bbb610700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb4196bb0 con 0x7f2bb4100e80 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.910+0000 7f2bbb610700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb419a940 con 0x7f2bb41004e0 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.910+0000 7f2bb8bab700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bb4100e80 0x7f2bb41964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:31.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.910+0000 7f2bb8bab700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bb4100e80 0x7f2bb41964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52624/0 (socket says 172.21.15.88:52624) 2024-02-19T19:34:31.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.910+0000 7f2bb8bab700 1 -- 172.21.15.88:0/2247517977 learned_addr learned my addr 172.21.15.88:0/2247517977 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:34:31.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.911+0000 7f2bb8bab700 1 -- 172.21.15.88:0/2247517977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb41004e0 msgr2=0x7f2bb4195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:31.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.911+0000 7f2bb8bab700 1 --2- 172.21.15.88:0/2247517977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb41004e0 0x7f2bb4195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:31.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.911+0000 7f2bb8bab700 1 -- 172.21.15.88:0/2247517977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb00095f0 con 0x7f2bb4100e80 2024-02-19T19:34:31.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.911+0000 7f2bb8bab700 1 --2- 172.21.15.88:0/2247517977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bb4100e80 0x7f2bb41964d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400b480 tx=0x7f2ba400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:31.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.911+0000 7f2baa7fc700 1 -- 172.21.15.88:0/2247517977 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba40118c0 con 0x7f2bb4100e80 2024-02-19T19:34:31.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.911+0000 7f2baa7fc700 1 -- 172.21.15.88:0/2247517977 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ba4010380 con 0x7f2bb4100e80 2024-02-19T19:34:31.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.911+0000 7f2baa7fc700 1 -- 172.21.15.88:0/2247517977 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba400f4f0 con 0x7f2bb4100e80 2024-02-19T19:34:31.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.911+0000 7f2bbb610700 1 -- 172.21.15.88:0/2247517977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb419ac20 con 0x7f2bb4100e80 2024-02-19T19:34:31.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.911+0000 7f2bbb610700 1 -- 172.21.15.88:0/2247517977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb419b170 con 0x7f2bb4100e80 2024-02-19T19:34:31.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.912+0000 7f2bbb610700 1 -- 172.21.15.88:0/2247517977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb41087f0 con 0x7f2bb4100e80 2024-02-19T19:34:31.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.913+0000 7f2baa7fc700 1 -- 172.21.15.88:0/2247517977 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2ba40104f0 con 0x7f2bb4100e80 2024-02-19T19:34:31.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.913+0000 7f2baa7fc700 1 --2- 172.21.15.88:0/2247517977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2ba006d8a0 0x7f2ba006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:31.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.913+0000 7f2baa7fc700 1 -- 172.21.15.88:0/2247517977 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2ba408d180 con 0x7f2bb4100e80 2024-02-19T19:34:31.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.913+0000 7f2bb93ac700 1 --2- 172.21.15.88:0/2247517977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2ba006d8a0 0x7f2ba006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:31.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.913+0000 7f2bb93ac700 1 --2- 172.21.15.88:0/2247517977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2ba006d8a0 0x7f2ba006fd50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009fd0 tx=0x7f2bb000b560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:31.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:31.918+0000 7f2baa7fc700 1 -- 172.21.15.88:0/2247517977 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2ba4056930 con 0x7f2bb4100e80 2024-02-19T19:34:32.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.158+0000 7f2bbb610700 1 -- 172.21.15.88:0/2247517977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2bb404f2a0 con 0x7f2bb4100e80 2024-02-19T19:34:32.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.159+0000 7f2baa7fc700 1 -- 172.21.15.88:0/2247517977 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f2ba4059ff0 con 0x7f2bb4100e80 2024-02-19T19:34:32.161 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:34:32.161 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:34:32.161 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:34:32.161 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:34:32.161 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:34:32.162 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:34:32.163 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:34:32.164 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:34:32.164 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:34:32.164 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:34:32.164 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:34:32.164 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:34:32.164 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:34:32.164 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:34:32.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.161+0000 7f2bbb610700 1 -- 172.21.15.88:0/2247517977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2ba006d8a0 msgr2=0x7f2ba006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:32.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.161+0000 7f2bbb610700 1 --2- 172.21.15.88:0/2247517977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2ba006d8a0 0x7f2ba006fd50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009fd0 tx=0x7f2bb000b560 comp rx=0 tx=0).stop 2024-02-19T19:34:32.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.161+0000 7f2bbb610700 1 -- 172.21.15.88:0/2247517977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bb4100e80 msgr2=0x7f2bb41964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:32.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.161+0000 7f2bbb610700 1 --2- 172.21.15.88:0/2247517977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bb4100e80 0x7f2bb41964d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400b480 tx=0x7f2ba400b840 comp rx=0 tx=0).stop 2024-02-19T19:34:32.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.161+0000 7f2bbb610700 1 -- 172.21.15.88:0/2247517977 shutdown_connections 2024-02-19T19:34:32.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.161+0000 7f2bbb610700 1 --2- 172.21.15.88:0/2247517977 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2ba006d8a0 0x7f2ba006fd50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.161+0000 7f2bbb610700 1 --2- 172.21.15.88:0/2247517977 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2bb41004e0 0x7f2bb4195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.161+0000 7f2bbb610700 1 --2- 172.21.15.88:0/2247517977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bb4100e80 0x7f2bb41964d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.161+0000 7f2bbb610700 1 -- 172.21.15.88:0/2247517977 >> 172.21.15.88:0/2247517977 conn(0x7f2bb40fbda0 msgr2=0x7f2bb40fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:32.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.161+0000 7f2bbb610700 1 -- 172.21.15.88:0/2247517977 shutdown_connections 2024-02-19T19:34:32.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.161+0000 7f2bbb610700 1 -- 172.21.15.88:0/2247517977 wait complete. 2024-02-19T19:34:32.166 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:34:32.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.266+0000 7f73de552700 1 -- 172.21.15.88:0/2833542500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d80ffd60 msgr2=0x7f73d8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:32.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.266+0000 7f73de552700 1 --2- 172.21.15.88:0/2833542500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d80ffd60 0x7f73d8103d30 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f73c8009910 tx=0x7f73c8009c20 comp rx=0 tx=0).stop 2024-02-19T19:34:32.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.266+0000 7f73de552700 1 -- 172.21.15.88:0/2833542500 shutdown_connections 2024-02-19T19:34:32.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.266+0000 7f73de552700 1 --2- 172.21.15.88:0/2833542500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d80ffd60 0x7f73d8103d30 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.266+0000 7f73de552700 1 --2- 172.21.15.88:0/2833542500 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d80ff3c0 0x7f73d80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.266+0000 7f73de552700 1 -- 172.21.15.88:0/2833542500 >> 172.21.15.88:0/2833542500 conn(0x7f73d80fac80 msgr2=0x7f73d80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:32.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.267+0000 7f73de552700 1 -- 172.21.15.88:0/2833542500 shutdown_connections 2024-02-19T19:34:32.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.267+0000 7f73de552700 1 -- 172.21.15.88:0/2833542500 wait complete. 2024-02-19T19:34:32.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.267+0000 7f73de552700 1 Processor -- start 2024-02-19T19:34:32.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.267+0000 7f73de552700 1 -- start start 2024-02-19T19:34:32.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.267+0000 7f73de552700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d80ff3c0 0x7f73d8196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:32.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73de552700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d80ffd60 0x7f73d8197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:32.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73de552700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d8197940 con 0x7f73d80ffd60 2024-02-19T19:34:32.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73de552700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d8197a80 con 0x7f73d80ff3c0 2024-02-19T19:34:32.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73d77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d80ffd60 0x7f73d8197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:32.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73d77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d80ffd60 0x7f73d8197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52648/0 (socket says 172.21.15.88:52648) 2024-02-19T19:34:32.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73d77fe700 1 -- 172.21.15.88:0/808705167 learned_addr learned my addr 172.21.15.88:0/808705167 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:34:32.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73d7fff700 1 --2- 172.21.15.88:0/808705167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d80ff3c0 0x7f73d8196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:32.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73d77fe700 1 -- 172.21.15.88:0/808705167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d80ff3c0 msgr2=0x7f73d8196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:32.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73d77fe700 1 --2- 172.21.15.88:0/808705167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d80ff3c0 0x7f73d8196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73d77fe700 1 -- 172.21.15.88:0/808705167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73c80095f0 con 0x7f73d80ffd60 2024-02-19T19:34:32.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73d77fe700 1 --2- 172.21.15.88:0/808705167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d80ffd60 0x7f73d8197260 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f73c800bd70 tx=0x7f73c800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:32.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73d57fa700 1 -- 172.21.15.88:0/808705167 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c801d070 con 0x7f73d80ffd60 2024-02-19T19:34:32.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73d57fa700 1 -- 172.21.15.88:0/808705167 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73c8005dc0 con 0x7f73d80ffd60 2024-02-19T19:34:32.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73de552700 1 -- 172.21.15.88:0/808705167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d819fd90 con 0x7f73d80ffd60 2024-02-19T19:34:32.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73d57fa700 1 -- 172.21.15.88:0/808705167 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c8017400 con 0x7f73d80ffd60 2024-02-19T19:34:32.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.268+0000 7f73de552700 1 -- 172.21.15.88:0/808705167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d81a0200 con 0x7f73d80ffd60 2024-02-19T19:34:32.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.269+0000 7f73de552700 1 -- 172.21.15.88:0/808705167 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d8107730 con 0x7f73d80ffd60 2024-02-19T19:34:32.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.270+0000 7f73d57fa700 1 -- 172.21.15.88:0/808705167 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f73c800f460 con 0x7f73d80ffd60 2024-02-19T19:34:32.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.270+0000 7f73d57fa700 1 --2- 172.21.15.88:0/808705167 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f73c006d8a0 0x7f73c006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:32.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.270+0000 7f73d57fa700 1 -- 172.21.15.88:0/808705167 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f73c808e8d0 con 0x7f73d80ffd60 2024-02-19T19:34:32.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.270+0000 7f73d7fff700 1 --2- 172.21.15.88:0/808705167 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f73c006d8a0 0x7f73c006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:32.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.271+0000 7f73d7fff700 1 --2- 172.21.15.88:0/808705167 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f73c006d8a0 0x7f73c006fd50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f73cc005950 tx=0x7f73cc00b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:32.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.275+0000 7f73d57fa700 1 -- 172.21.15.88:0/808705167 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f73c8058100 con 0x7f73d80ffd60 2024-02-19T19:34:32.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:32 smithi132 ceph-mon[70676]: from='client.35678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:32.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:32 smithi132 ceph-mon[70676]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:32.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:32 smithi132 ceph-mon[70676]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:32.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:32 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1748320402' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:34:32.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:32 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2247517977' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:34:32.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.488+0000 7f73de552700 1 -- 172.21.15.88:0/808705167 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73d8061190 con 0x7f73c006d8a0 2024-02-19T19:34:32.490 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:32 smithi088 ceph-mon[88643]: from='client.35678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:32.490 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:32 smithi088 ceph-mon[88643]: from='client.35682 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:32.491 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:32 smithi088 ceph-mon[88643]: from='client.35686 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:32.491 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:32 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1748320402' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:34:32.491 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:32 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2247517977' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:34:32.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.489+0000 7f73d57fa700 1 -- 172.21.15.88:0/808705167 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f73d8061190 con 0x7f73c006d8a0 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:34:32.492 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:34:32.493 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:34:32.493 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:34:32.493 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:34:32.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.492+0000 7f73de552700 1 -- 172.21.15.88:0/808705167 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f73c006d8a0 msgr2=0x7f73c006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:32.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.492+0000 7f73de552700 1 --2- 172.21.15.88:0/808705167 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f73c006d8a0 0x7f73c006fd50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f73cc005950 tx=0x7f73cc00b410 comp rx=0 tx=0).stop 2024-02-19T19:34:32.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.492+0000 7f73de552700 1 -- 172.21.15.88:0/808705167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d80ffd60 msgr2=0x7f73d8197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:32.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.492+0000 7f73de552700 1 --2- 172.21.15.88:0/808705167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d80ffd60 0x7f73d8197260 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f73c800bd70 tx=0x7f73c800bda0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.492+0000 7f73de552700 1 -- 172.21.15.88:0/808705167 shutdown_connections 2024-02-19T19:34:32.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.492+0000 7f73de552700 1 --2- 172.21.15.88:0/808705167 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f73c006d8a0 0x7f73c006fd50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.492+0000 7f73de552700 1 --2- 172.21.15.88:0/808705167 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f73d80ff3c0 0x7f73d8196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.492+0000 7f73de552700 1 --2- 172.21.15.88:0/808705167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d80ffd60 0x7f73d8197260 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.492+0000 7f73de552700 1 -- 172.21.15.88:0/808705167 >> 172.21.15.88:0/808705167 conn(0x7f73d80fac80 msgr2=0x7f73d80fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:32.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.492+0000 7f73de552700 1 -- 172.21.15.88:0/808705167 shutdown_connections 2024-02-19T19:34:32.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.492+0000 7f73de552700 1 -- 172.21.15.88:0/808705167 wait complete. 2024-02-19T19:34:32.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.597+0000 7fd185f31700 1 -- 172.21.15.88:0/1073970318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1800ff3c0 msgr2=0x7fd1800ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:32.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.597+0000 7fd185f31700 1 --2- 172.21.15.88:0/1073970318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1800ff3c0 0x7fd1800ff790 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd170009910 tx=0x7fd170009c20 comp rx=0 tx=0).stop 2024-02-19T19:34:32.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.597+0000 7fd185f31700 1 -- 172.21.15.88:0/1073970318 shutdown_connections 2024-02-19T19:34:32.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.597+0000 7fd185f31700 1 --2- 172.21.15.88:0/1073970318 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1800ffd60 0x7fd180103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.597+0000 7fd185f31700 1 --2- 172.21.15.88:0/1073970318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1800ff3c0 0x7fd1800ff790 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.597+0000 7fd185f31700 1 -- 172.21.15.88:0/1073970318 >> 172.21.15.88:0/1073970318 conn(0x7fd1800fac80 msgr2=0x7fd1800fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:32.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.597+0000 7fd185f31700 1 -- 172.21.15.88:0/1073970318 shutdown_connections 2024-02-19T19:34:32.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.597+0000 7fd185f31700 1 -- 172.21.15.88:0/1073970318 wait complete. 2024-02-19T19:34:32.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.597+0000 7fd185f31700 1 Processor -- start 2024-02-19T19:34:32.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd185f31700 1 -- start start 2024-02-19T19:34:32.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd185f31700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1800ff3c0 0x7fd180196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:32.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd185f31700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1800ffd60 0x7fd180197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:32.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd185f31700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180197850 con 0x7fd1800ff3c0 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd185f31700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd180197990 con 0x7fd1800ffd60 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd17f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1800ff3c0 0x7fd180196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd17f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1800ff3c0 0x7fd180196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52674/0 (socket says 172.21.15.88:52674) 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd17f7fe700 1 -- 172.21.15.88:0/1774277754 learned_addr learned my addr 172.21.15.88:0/1774277754 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd176ffd700 1 --2- 172.21.15.88:0/1774277754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1800ffd60 0x7fd180197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd17f7fe700 1 -- 172.21.15.88:0/1774277754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1800ffd60 msgr2=0x7fd180197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd17f7fe700 1 --2- 172.21.15.88:0/1774277754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1800ffd60 0x7fd180197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.598+0000 7fd17f7fe700 1 -- 172.21.15.88:0/1774277754 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700095f0 con 0x7fd1800ff3c0 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.599+0000 7fd17f7fe700 1 --2- 172.21.15.88:0/1774277754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1800ff3c0 0x7fd180196cf0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd170000c00 tx=0x7fd17000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.599+0000 7fd17d7fa700 1 -- 172.21.15.88:0/1774277754 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd17001d070 con 0x7fd1800ff3c0 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.599+0000 7fd17d7fa700 1 -- 172.21.15.88:0/1774277754 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd170005dc0 con 0x7fd1800ff3c0 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.599+0000 7fd17d7fa700 1 -- 172.21.15.88:0/1774277754 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd170017400 con 0x7fd1800ff3c0 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.599+0000 7fd185f31700 1 -- 172.21.15.88:0/1774277754 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd18019fd60 con 0x7fd1800ff3c0 2024-02-19T19:34:32.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.599+0000 7fd185f31700 1 -- 172.21.15.88:0/1774277754 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1801a0170 con 0x7fd1800ff3c0 2024-02-19T19:34:32.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.600+0000 7fd185f31700 1 -- 172.21.15.88:0/1774277754 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1801076d0 con 0x7fd1800ff3c0 2024-02-19T19:34:32.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.600+0000 7fd17d7fa700 1 -- 172.21.15.88:0/1774277754 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd17000f7f0 con 0x7fd1800ff3c0 2024-02-19T19:34:32.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.600+0000 7fd17d7fa700 1 --2- 172.21.15.88:0/1774277754 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd16c06d8a0 0x7fd16c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:34:32.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.601+0000 7fd17d7fa700 1 -- 172.21.15.88:0/1774277754 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd17008ea50 con 0x7fd1800ff3c0 2024-02-19T19:34:32.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.601+0000 7fd176ffd700 1 --2- 172.21.15.88:0/1774277754 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd16c06d8a0 0x7fd16c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:34:32.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.601+0000 7fd176ffd700 1 --2- 172.21.15.88:0/1774277754 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd16c06d8a0 0x7fd16c06fd50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd168005950 tx=0x7fd1680058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:34:32.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.605+0000 7fd17d7fa700 1 -- 172.21.15.88:0/1774277754 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd170058200 con 0x7fd1800ff3c0 2024-02-19T19:34:32.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.884+0000 7fd185f31700 1 -- 172.21.15.88:0/1774277754 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd1801070b0 con 0x7fd1800ff3c0 2024-02-19T19:34:32.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.884+0000 7fd17d7fa700 1 -- 172.21.15.88:0/1774277754 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fd17005b8c0 con 0x7fd1800ff3c0 2024-02-19T19:34:32.886 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:34:32.886 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:34:32.886 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:34:32.887 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:34:32.888 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:34:32.888 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:34:32.888 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:34:32.888 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:34:32.888 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:34:32.888 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:34:32.888 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:34:32.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.886+0000 7fd185f31700 1 -- 172.21.15.88:0/1774277754 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd16c06d8a0 msgr2=0x7fd16c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:32.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.886+0000 7fd185f31700 1 --2- 172.21.15.88:0/1774277754 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd16c06d8a0 0x7fd16c06fd50 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd168005950 tx=0x7fd1680058e0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.886+0000 7fd185f31700 1 -- 172.21.15.88:0/1774277754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1800ff3c0 msgr2=0x7fd180196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:34:32.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.886+0000 7fd185f31700 1 --2- 172.21.15.88:0/1774277754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1800ff3c0 0x7fd180196cf0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd170000c00 tx=0x7fd17000be20 comp rx=0 tx=0).stop 2024-02-19T19:34:32.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.886+0000 7fd185f31700 1 -- 172.21.15.88:0/1774277754 shutdown_connections 2024-02-19T19:34:32.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.886+0000 7fd185f31700 1 --2- 172.21.15.88:0/1774277754 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd16c06d8a0 0x7fd16c06fd50 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.886+0000 7fd185f31700 1 --2- 172.21.15.88:0/1774277754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1800ff3c0 0x7fd180196cf0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.886+0000 7fd185f31700 1 --2- 172.21.15.88:0/1774277754 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd1800ffd60 0x7fd180197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:34:32.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.886+0000 7fd185f31700 1 -- 172.21.15.88:0/1774277754 >> 172.21.15.88:0/1774277754 conn(0x7fd1800fac80 msgr2=0x7fd1800fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:34:32.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.886+0000 7fd185f31700 1 -- 172.21.15.88:0/1774277754 shutdown_connections 2024-02-19T19:34:32.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:34:32.886+0000 7fd185f31700 1 -- 172.21.15.88:0/1774277754 wait complete. 2024-02-19T19:34:33.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:33 smithi132 ceph-mon[70676]: from='client.35698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:33.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:33 smithi132 ceph-mon[70676]: pgmap v930: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:33.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:33 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1774277754' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:34:33.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:33 smithi088 ceph-mon[88643]: from='client.35698 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:34:33.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:33 smithi088 ceph-mon[88643]: pgmap v930: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:33.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:33 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1774277754' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:34:35.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:34 smithi088 ceph-mon[88643]: pgmap v931: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:35.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:34 smithi132 ceph-mon[70676]: pgmap v931: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:37.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:36 smithi088 ceph-mon[88643]: pgmap v932: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:37.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:36 smithi132 ceph-mon[70676]: pgmap v932: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:39.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:38 smithi088 ceph-mon[88643]: pgmap v933: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:39.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:38 smithi132 ceph-mon[70676]: pgmap v933: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:41.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:40 smithi088 ceph-mon[88643]: pgmap v934: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:41.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:40 smithi132 ceph-mon[70676]: pgmap v934: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:43.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:42 smithi088 ceph-mon[88643]: pgmap v935: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:43.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:42 smithi132 ceph-mon[70676]: pgmap v935: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:44.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:34:44.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:34:45.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:44 smithi088 ceph-mon[88643]: pgmap v936: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:45.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:44 smithi132 ceph-mon[70676]: pgmap v936: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:47.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:46 smithi088 ceph-mon[88643]: pgmap v937: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:47.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:46 smithi132 ceph-mon[70676]: pgmap v937: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:49.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:48 smithi088 ceph-mon[88643]: pgmap v938: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:49.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:48 smithi132 ceph-mon[70676]: pgmap v938: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:51.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:50 smithi088 ceph-mon[88643]: pgmap v939: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:51.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:50 smithi132 ceph-mon[70676]: pgmap v939: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:53.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:52 smithi088 ceph-mon[88643]: pgmap v940: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:53.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:52 smithi132 ceph-mon[70676]: pgmap v940: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:55.037 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:54 smithi132 ceph-mon[70676]: pgmap v941: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:55.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:54 smithi088 ceph-mon[88643]: pgmap v941: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:56.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:56 smithi132 ceph-mon[70676]: pgmap v942: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:57.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:56 smithi088 ceph-mon[88643]: pgmap v942: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:58.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:34:58.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:34:59.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:34:59 smithi088 ceph-mon[88643]: pgmap v943: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:34:59.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:34:59 smithi132 ceph-mon[70676]: pgmap v943: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:01.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:00 smithi088 ceph-mon[88643]: pgmap v944: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:01.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:00 smithi132 ceph-mon[70676]: pgmap v944: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.002+0000 7fdf769c6700 1 -- 172.21.15.88:0/3410312205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf700ff3c0 msgr2=0x7fdf700ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.002+0000 7fdf769c6700 1 --2- 172.21.15.88:0/3410312205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf700ff3c0 0x7fdf700ff790 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdf60009910 tx=0x7fdf60009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.002+0000 7fdf769c6700 1 -- 172.21.15.88:0/3410312205 shutdown_connections 2024-02-19T19:35:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.002+0000 7fdf769c6700 1 --2- 172.21.15.88:0/3410312205 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf700ffd60 0x7fdf70103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.002+0000 7fdf769c6700 1 --2- 172.21.15.88:0/3410312205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf700ff3c0 0x7fdf700ff790 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.002+0000 7fdf769c6700 1 -- 172.21.15.88:0/3410312205 >> 172.21.15.88:0/3410312205 conn(0x7fdf700fac80 msgr2=0x7fdf700fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.003+0000 7fdf769c6700 1 -- 172.21.15.88:0/3410312205 shutdown_connections 2024-02-19T19:35:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.003+0000 7fdf769c6700 1 -- 172.21.15.88:0/3410312205 wait complete. 2024-02-19T19:35:03.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.003+0000 7fdf769c6700 1 Processor -- start 2024-02-19T19:35:03.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.003+0000 7fdf769c6700 1 -- start start 2024-02-19T19:35:03.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.003+0000 7fdf769c6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf700ff3c0 0x7fdf70195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:03.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf769c6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf700ffd60 0x7fdf701963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:03.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf769c6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf701969f0 con 0x7fdf700ff3c0 2024-02-19T19:35:03.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf769c6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf70196b30 con 0x7fdf700ffd60 2024-02-19T19:35:03.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf6ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf700ff3c0 0x7fdf70195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:03.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf6ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf700ff3c0 0x7fdf70195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43456/0 (socket says 172.21.15.88:43456) 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf6ffff700 1 -- 172.21.15.88:0/810493278 learned_addr learned my addr 172.21.15.88:0/810493278 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf6f7fe700 1 --2- 172.21.15.88:0/810493278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf700ffd60 0x7fdf701963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf6ffff700 1 -- 172.21.15.88:0/810493278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf700ffd60 msgr2=0x7fdf701963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf6ffff700 1 --2- 172.21.15.88:0/810493278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf700ffd60 0x7fdf701963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf6ffff700 1 -- 172.21.15.88:0/810493278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf600095f0 con 0x7fdf700ff3c0 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf6ffff700 1 --2- 172.21.15.88:0/810493278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf700ff3c0 0x7fdf70195e60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdf60006010 tx=0x7fdf6000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf6d7fa700 1 -- 172.21.15.88:0/810493278 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf6001d070 con 0x7fdf700ff3c0 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf769c6700 1 -- 172.21.15.88:0/810493278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7019a920 con 0x7fdf700ff3c0 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf6d7fa700 1 -- 172.21.15.88:0/810493278 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf6000f460 con 0x7fdf700ff3c0 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf6d7fa700 1 -- 172.21.15.88:0/810493278 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf60017630 con 0x7fdf700ff3c0 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.004+0000 7fdf769c6700 1 -- 172.21.15.88:0/810493278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7019ae10 con 0x7fdf700ff3c0 2024-02-19T19:35:03.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.005+0000 7fdf769c6700 1 -- 172.21.15.88:0/810493278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf7004f2a0 con 0x7fdf700ff3c0 2024-02-19T19:35:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.006+0000 7fdf6d7fa700 1 -- 172.21.15.88:0/810493278 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdf6000f5d0 con 0x7fdf700ff3c0 2024-02-19T19:35:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.006+0000 7fdf6d7fa700 1 --2- 172.21.15.88:0/810493278 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5806d8a0 0x7fdf5806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.006+0000 7fdf6d7fa700 1 -- 172.21.15.88:0/810493278 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdf6008ea70 con 0x7fdf700ff3c0 2024-02-19T19:35:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.006+0000 7fdf6f7fe700 1 --2- 172.21.15.88:0/810493278 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5806d8a0 0x7fdf5806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:03.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.007+0000 7fdf6f7fe700 1 --2- 172.21.15.88:0/810493278 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5806d8a0 0x7fdf5806fd50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdf70197480 tx=0x7fdf6400a380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:03.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.011+0000 7fdf6d7fa700 1 -- 172.21.15.88:0/810493278 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf60058220 con 0x7fdf700ff3c0 2024-02-19T19:35:03.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:02 smithi088 ceph-mon[88643]: pgmap v945: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:03.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.229+0000 7fdf769c6700 1 -- 172.21.15.88:0/810493278 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf7019b140 con 0x7fdf5806d8a0 2024-02-19T19:35:03.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.230+0000 7fdf6d7fa700 1 -- 172.21.15.88:0/810493278 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdf7019b140 con 0x7fdf5806d8a0 2024-02-19T19:35:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.232+0000 7fdf769c6700 1 -- 172.21.15.88:0/810493278 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5806d8a0 msgr2=0x7fdf5806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.232+0000 7fdf769c6700 1 --2- 172.21.15.88:0/810493278 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5806d8a0 0x7fdf5806fd50 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fdf70197480 tx=0x7fdf6400a380 comp rx=0 tx=0).stop 2024-02-19T19:35:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.232+0000 7fdf769c6700 1 -- 172.21.15.88:0/810493278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf700ff3c0 msgr2=0x7fdf70195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.232+0000 7fdf769c6700 1 --2- 172.21.15.88:0/810493278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf700ff3c0 0x7fdf70195e60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdf60006010 tx=0x7fdf6000bd70 comp rx=0 tx=0).stop 2024-02-19T19:35:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.232+0000 7fdf769c6700 1 -- 172.21.15.88:0/810493278 shutdown_connections 2024-02-19T19:35:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.232+0000 7fdf769c6700 1 --2- 172.21.15.88:0/810493278 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf5806d8a0 0x7fdf5806fd50 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.232+0000 7fdf769c6700 1 --2- 172.21.15.88:0/810493278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf700ff3c0 0x7fdf70195e60 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.232+0000 7fdf769c6700 1 --2- 172.21.15.88:0/810493278 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf700ffd60 0x7fdf701963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.232+0000 7fdf769c6700 1 -- 172.21.15.88:0/810493278 >> 172.21.15.88:0/810493278 conn(0x7fdf700fac80 msgr2=0x7fdf700fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.232+0000 7fdf769c6700 1 -- 172.21.15.88:0/810493278 shutdown_connections 2024-02-19T19:35:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.232+0000 7fdf769c6700 1 -- 172.21.15.88:0/810493278 wait complete. 2024-02-19T19:35:03.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:02 smithi132 ceph-mon[70676]: pgmap v945: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:03.245 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:35:03.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.337+0000 7f134edc4700 1 -- 172.21.15.88:0/3807172044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13480ff350 msgr2=0x7f13480ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.337+0000 7f134edc4700 1 --2- 172.21.15.88:0/3807172044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13480ff350 0x7f13480ff720 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f133c009910 tx=0x7f133c009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:03.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.337+0000 7f134edc4700 1 -- 172.21.15.88:0/3807172044 shutdown_connections 2024-02-19T19:35:03.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.337+0000 7f134edc4700 1 --2- 172.21.15.88:0/3807172044 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13480ffc60 0x7f134810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.337+0000 7f134edc4700 1 --2- 172.21.15.88:0/3807172044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13480ff350 0x7f13480ff720 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.337+0000 7f134edc4700 1 -- 172.21.15.88:0/3807172044 >> 172.21.15.88:0/3807172044 conn(0x7f13480748b0 msgr2=0x7f1348074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:03.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.337+0000 7f134edc4700 1 -- 172.21.15.88:0/3807172044 shutdown_connections 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.337+0000 7f134edc4700 1 -- 172.21.15.88:0/3807172044 wait complete. 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.338+0000 7f134edc4700 1 Processor -- start 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.338+0000 7f134edc4700 1 -- start start 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.338+0000 7f134edc4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13480ff350 0x7f1348100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.338+0000 7f134edc4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13480ffc60 0x7f1348100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.338+0000 7f134edc4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1348100fb0 con 0x7f13480ff350 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.338+0000 7f134edc4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13481010f0 con 0x7f13480ffc60 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.338+0000 7f134cb60700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13480ff350 0x7f1348100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.338+0000 7f134cb60700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13480ff350 0x7f1348100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43474/0 (socket says 172.21.15.88:43474) 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.338+0000 7f134cb60700 1 -- 172.21.15.88:0/816191340 learned_addr learned my addr 172.21.15.88:0/816191340 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.339+0000 7f134cb60700 1 -- 172.21.15.88:0/816191340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13480ffc60 msgr2=0x7f1348100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.339+0000 7f1347fff700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13480ffc60 0x7f1348100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.339+0000 7f134cb60700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13480ffc60 0x7f1348100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.339+0000 7f134cb60700 1 -- 172.21.15.88:0/816191340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0095f0 con 0x7f13480ff350 2024-02-19T19:35:03.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.339+0000 7f1347fff700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13480ffc60 0x7f1348100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:35:03.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.339+0000 7f134cb60700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13480ff350 0x7f1348100530 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f133c00bda0 tx=0x7f133c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:03.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.339+0000 7f1345ffb700 1 -- 172.21.15.88:0/816191340 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f133c01d070 con 0x7f13480ff350 2024-02-19T19:35:03.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.339+0000 7f1345ffb700 1 -- 172.21.15.88:0/816191340 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f133c004fb0 con 0x7f13480ff350 2024-02-19T19:35:03.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.339+0000 7f1345ffb700 1 -- 172.21.15.88:0/816191340 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f133c00f480 con 0x7f13480ff350 2024-02-19T19:35:03.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.339+0000 7f134edc4700 1 -- 172.21.15.88:0/816191340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1348101370 con 0x7f13480ff350 2024-02-19T19:35:03.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.339+0000 7f134edc4700 1 -- 172.21.15.88:0/816191340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13481a2200 con 0x7f13480ff350 2024-02-19T19:35:03.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.340+0000 7f134edc4700 1 -- 172.21.15.88:0/816191340 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1348107dd0 con 0x7f13480ff350 2024-02-19T19:35:03.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.341+0000 7f1345ffb700 1 -- 172.21.15.88:0/816191340 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f133c022870 con 0x7f13480ff350 2024-02-19T19:35:03.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.341+0000 7f1345ffb700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f133806d8a0 0x7f133806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:03.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.341+0000 7f1345ffb700 1 -- 172.21.15.88:0/816191340 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f133c08ea50 con 0x7f13480ff350 2024-02-19T19:35:03.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.341+0000 7f1347fff700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f133806d8a0 0x7f133806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:03.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.342+0000 7f1347fff700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f133806d8a0 0x7f133806fd50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1334005fd0 tx=0x7f1334005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:03.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.346+0000 7f1345ffb700 1 -- 172.21.15.88:0/816191340 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f133c0583d0 con 0x7f13480ff350 2024-02-19T19:35:03.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.571+0000 7f134edc4700 1 -- 172.21.15.88:0/816191340 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f134802cc80 con 0x7f133806d8a0 2024-02-19T19:35:03.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.573+0000 7f1345ffb700 1 -- 172.21.15.88:0/816191340 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f134802cc80 con 0x7f133806d8a0 2024-02-19T19:35:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.576+0000 7f134edc4700 1 -- 172.21.15.88:0/816191340 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f133806d8a0 msgr2=0x7f133806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.576+0000 7f134edc4700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f133806d8a0 0x7f133806fd50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1334005fd0 tx=0x7f1334005e20 comp rx=0 tx=0).stop 2024-02-19T19:35:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.576+0000 7f134edc4700 1 -- 172.21.15.88:0/816191340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13480ff350 msgr2=0x7f1348100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.576+0000 7f134edc4700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13480ff350 0x7f1348100530 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f133c00bda0 tx=0x7f133c00be80 comp rx=0 tx=0).stop 2024-02-19T19:35:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.576+0000 7f134edc4700 1 -- 172.21.15.88:0/816191340 shutdown_connections 2024-02-19T19:35:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.576+0000 7f134edc4700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f133806d8a0 0x7f133806fd50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.576+0000 7f134edc4700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13480ff350 0x7f1348100530 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.576+0000 7f134edc4700 1 --2- 172.21.15.88:0/816191340 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f13480ffc60 0x7f1348100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.576+0000 7f134edc4700 1 -- 172.21.15.88:0/816191340 >> 172.21.15.88:0/816191340 conn(0x7f13480748b0 msgr2=0x7f1348104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.576+0000 7f134edc4700 1 -- 172.21.15.88:0/816191340 shutdown_connections 2024-02-19T19:35:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.576+0000 7f134edc4700 1 -- 172.21.15.88:0/816191340 wait complete. 2024-02-19T19:35:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.680+0000 7ff947d22700 1 -- 172.21.15.88:0/2715671157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9400ff720 msgr2=0x7ff9400ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.680+0000 7ff947d22700 1 --2- 172.21.15.88:0/2715671157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9400ff720 0x7ff9400ffb90 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff930009910 tx=0x7ff930009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.680+0000 7ff947d22700 1 -- 172.21.15.88:0/2715671157 shutdown_connections 2024-02-19T19:35:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.680+0000 7ff947d22700 1 --2- 172.21.15.88:0/2715671157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9400ff720 0x7ff9400ffb90 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.680+0000 7ff947d22700 1 --2- 172.21.15.88:0/2715671157 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9400ff350 0x7ff9401063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.680+0000 7ff947d22700 1 -- 172.21.15.88:0/2715671157 >> 172.21.15.88:0/2715671157 conn(0x7ff9400748b0 msgr2=0x7ff940074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.680+0000 7ff947d22700 1 -- 172.21.15.88:0/2715671157 shutdown_connections 2024-02-19T19:35:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.680+0000 7ff947d22700 1 -- 172.21.15.88:0/2715671157 wait complete. 2024-02-19T19:35:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.680+0000 7ff947d22700 1 Processor -- start 2024-02-19T19:35:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff947d22700 1 -- start start 2024-02-19T19:35:03.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff947d22700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9400ff350 0x7ff94019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:03.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff947d22700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9400ff720 0x7ff94019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff947d22700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94019d030 con 0x7ff9400ff350 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff947d22700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9401a0dc0 con 0x7ff9400ff720 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff945abe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9400ff350 0x7ff94019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff945abe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9400ff350 0x7ff94019c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43480/0 (socket says 172.21.15.88:43480) 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff945abe700 1 -- 172.21.15.88:0/583055581 learned_addr learned my addr 172.21.15.88:0/583055581 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff9452bd700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9400ff720 0x7ff94019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff945abe700 1 -- 172.21.15.88:0/583055581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9400ff720 msgr2=0x7ff94019c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff945abe700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9400ff720 0x7ff94019c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff945abe700 1 -- 172.21.15.88:0/583055581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9300095f0 con 0x7ff9400ff350 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff9452bd700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9400ff720 0x7ff94019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff945abe700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9400ff350 0x7ff94019c410 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00d8d0 tx=0x7ff93c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff936ffd700 1 -- 172.21.15.88:0/583055581 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff93c009940 con 0x7ff9400ff350 2024-02-19T19:35:03.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff936ffd700 1 -- 172.21.15.88:0/583055581 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff93c010460 con 0x7ff9400ff350 2024-02-19T19:35:03.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff936ffd700 1 -- 172.21.15.88:0/583055581 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff93c00f5f0 con 0x7ff9400ff350 2024-02-19T19:35:03.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.681+0000 7ff947d22700 1 -- 172.21.15.88:0/583055581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9401a10a0 con 0x7ff9400ff350 2024-02-19T19:35:03.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.682+0000 7ff947d22700 1 -- 172.21.15.88:0/583055581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9401a15f0 con 0x7ff9400ff350 2024-02-19T19:35:03.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.683+0000 7ff947d22700 1 -- 172.21.15.88:0/583055581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff940103ae0 con 0x7ff9400ff350 2024-02-19T19:35:03.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.683+0000 7ff936ffd700 1 -- 172.21.15.88:0/583055581 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff93c0105d0 con 0x7ff9400ff350 2024-02-19T19:35:03.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.683+0000 7ff936ffd700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff92c06d8a0 0x7ff92c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.684+0000 7ff9452bd700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff92c06d8a0 0x7ff92c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.684+0000 7ff936ffd700 1 -- 172.21.15.88:0/583055581 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff93c08d540 con 0x7ff9400ff350 2024-02-19T19:35:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.684+0000 7ff9452bd700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff92c06d8a0 0x7ff92c06fd50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff930005b40 tx=0x7ff93001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:03.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.688+0000 7ff936ffd700 1 -- 172.21.15.88:0/583055581 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff93c056cf0 con 0x7ff9400ff350 2024-02-19T19:35:03.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:03 smithi088 ceph-mon[88643]: from='client.35706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:03.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:03 smithi088 ceph-mon[88643]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:03.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.897+0000 7ff947d22700 1 -- 172.21.15.88:0/583055581 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9401a1810 con 0x7ff92c06d8a0 2024-02-19T19:35:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.907+0000 7ff936ffd700 1 -- 172.21.15.88:0/583055581 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ff9401a1810 con 0x7ff92c06d8a0 2024-02-19T19:35:03.910 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (32m) 8m ago 42m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (43m) 8m ago 43m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (25m) 8m ago 41m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (28m) 8m ago 43m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (28m) 8m ago 41m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (31m) 8m ago 42m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (24m) 8m ago 36m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (24m) 8m ago 36m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (23m) 8m ago 36m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (8m) 8m ago 36m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (33m) 8m ago 44m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (33m) 8m ago 40m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:35:03.911 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (30m) 8m ago 44m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:35:03.912 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (29m) 8m ago 40m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:35:03.912 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (32m) 8m ago 42m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:35:03.912 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (32m) 8m ago 41m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:35:03.912 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (27m) 8m ago 40m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:35:03.912 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (27m) 8m ago 39m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:35:03.912 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (26m) 8m ago 39m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:35:03.912 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (25m) 8m ago 38m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:35:03.912 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (25m) 8m ago 38m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:35:03.912 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (24m) 8m ago 37m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:35:03.912 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (32m) 8m ago 41m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:35:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.909+0000 7ff947d22700 1 -- 172.21.15.88:0/583055581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff92c06d8a0 msgr2=0x7ff92c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.909+0000 7ff947d22700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff92c06d8a0 0x7ff92c06fd50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff930005b40 tx=0x7ff93001a040 comp rx=0 tx=0).stop 2024-02-19T19:35:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.909+0000 7ff947d22700 1 -- 172.21.15.88:0/583055581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9400ff350 msgr2=0x7ff94019c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.909+0000 7ff947d22700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9400ff350 0x7ff94019c410 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00d8d0 tx=0x7ff93c00dc90 comp rx=0 tx=0).stop 2024-02-19T19:35:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.910+0000 7ff947d22700 1 -- 172.21.15.88:0/583055581 shutdown_connections 2024-02-19T19:35:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.910+0000 7ff947d22700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff92c06d8a0 0x7ff92c06fd50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.910+0000 7ff947d22700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9400ff350 0x7ff94019c410 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.910+0000 7ff947d22700 1 --2- 172.21.15.88:0/583055581 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9400ff720 0x7ff94019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:03.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.910+0000 7ff947d22700 1 -- 172.21.15.88:0/583055581 >> 172.21.15.88:0/583055581 conn(0x7ff9400748b0 msgr2=0x7ff9400fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:03.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.910+0000 7ff947d22700 1 -- 172.21.15.88:0/583055581 shutdown_connections 2024-02-19T19:35:03.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:03.910+0000 7ff947d22700 1 -- 172.21.15.88:0/583055581 wait complete. 2024-02-19T19:35:04.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.013+0000 7f6d64aa0700 1 -- 172.21.15.88:0/2014576764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d600ffd60 msgr2=0x7f6d60103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.013+0000 7f6d64aa0700 1 --2- 172.21.15.88:0/2014576764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d600ffd60 0x7f6d60103d30 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009910 tx=0x7f6d50009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:04.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.013+0000 7f6d64aa0700 1 -- 172.21.15.88:0/2014576764 shutdown_connections 2024-02-19T19:35:04.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.013+0000 7f6d64aa0700 1 --2- 172.21.15.88:0/2014576764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d600ffd60 0x7f6d60103d30 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.013+0000 7f6d64aa0700 1 --2- 172.21.15.88:0/2014576764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d600ff3c0 0x7f6d600ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.013+0000 7f6d64aa0700 1 -- 172.21.15.88:0/2014576764 >> 172.21.15.88:0/2014576764 conn(0x7f6d600fac80 msgr2=0x7f6d600fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:04.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.013+0000 7f6d64aa0700 1 -- 172.21.15.88:0/2014576764 shutdown_connections 2024-02-19T19:35:04.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.013+0000 7f6d64aa0700 1 -- 172.21.15.88:0/2014576764 wait complete. 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.013+0000 7f6d64aa0700 1 Processor -- start 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d64aa0700 1 -- start start 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d64aa0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d600ff3c0 0x7f6d60195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d64aa0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d600ffd60 0x7f6d601963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d64aa0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d60196aa0 con 0x7f6d600ffd60 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d64aa0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6019a830 con 0x7f6d600ff3c0 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d5dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d600ffd60 0x7f6d601963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d5dd9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d600ffd60 0x7f6d601963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43488/0 (socket says 172.21.15.88:43488) 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d5dd9b700 1 -- 172.21.15.88:0/676920096 learned_addr learned my addr 172.21.15.88:0/676920096 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d5e59c700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d600ff3c0 0x7f6d60195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d5dd9b700 1 -- 172.21.15.88:0/676920096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d600ff3c0 msgr2=0x7f6d60195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d5dd9b700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d600ff3c0 0x7f6d60195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d5dd9b700 1 -- 172.21.15.88:0/676920096 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d500095f0 con 0x7f6d600ffd60 2024-02-19T19:35:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d5e59c700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d600ff3c0 0x7f6d60195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:35:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d5dd9b700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d600ffd60 0x7f6d601963c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6d500092e0 tx=0x7f6d5000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d577fe700 1 -- 172.21.15.88:0/676920096 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d5001d070 con 0x7f6d600ffd60 2024-02-19T19:35:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d577fe700 1 -- 172.21.15.88:0/676920096 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d5000f460 con 0x7f6d600ffd60 2024-02-19T19:35:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d64aa0700 1 -- 172.21.15.88:0/676920096 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6019aab0 con 0x7f6d600ffd60 2024-02-19T19:35:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.014+0000 7f6d577fe700 1 -- 172.21.15.88:0/676920096 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d500175f0 con 0x7f6d600ffd60 2024-02-19T19:35:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.015+0000 7f6d64aa0700 1 -- 172.21.15.88:0/676920096 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6019afa0 con 0x7f6d600ffd60 2024-02-19T19:35:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.016+0000 7f6d64aa0700 1 -- 172.21.15.88:0/676920096 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6004f2a0 con 0x7f6d600ffd60 2024-02-19T19:35:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.016+0000 7f6d577fe700 1 -- 172.21.15.88:0/676920096 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6d5000f5d0 con 0x7f6d600ffd60 2024-02-19T19:35:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.016+0000 7f6d577fe700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d4c06d8a0 0x7f6d4c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.016+0000 7f6d577fe700 1 -- 172.21.15.88:0/676920096 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6d5008eab0 con 0x7f6d600ffd60 2024-02-19T19:35:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.016+0000 7f6d5e59c700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d4c06d8a0 0x7f6d4c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.017+0000 7f6d5e59c700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d4c06d8a0 0x7f6d4c06fd50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6d48005fd0 tx=0x7f6d48005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:04.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.021+0000 7f6d577fe700 1 -- 172.21.15.88:0/676920096 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6d50058260 con 0x7f6d600ffd60 2024-02-19T19:35:04.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:03 smithi132 ceph-mon[70676]: from='client.35706 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:04.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:03 smithi132 ceph-mon[70676]: from='client.35710 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:04.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.298+0000 7f6d64aa0700 1 -- 172.21.15.88:0/676920096 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6d6019b280 con 0x7f6d600ffd60 2024-02-19T19:35:04.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.298+0000 7f6d577fe700 1 -- 172.21.15.88:0/676920096 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f6d5005b920 con 0x7f6d600ffd60 2024-02-19T19:35:04.301 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:35:04.301 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:35:04.301 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:35:04.302 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:35:04.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.301+0000 7f6d64aa0700 1 -- 172.21.15.88:0/676920096 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d4c06d8a0 msgr2=0x7f6d4c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.301+0000 7f6d64aa0700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d4c06d8a0 0x7f6d4c06fd50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6d48005fd0 tx=0x7f6d48005dc0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.301+0000 7f6d64aa0700 1 -- 172.21.15.88:0/676920096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d600ffd60 msgr2=0x7f6d601963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.301+0000 7f6d64aa0700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d600ffd60 0x7f6d601963c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6d500092e0 tx=0x7f6d5000be20 comp rx=0 tx=0).stop 2024-02-19T19:35:04.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.301+0000 7f6d64aa0700 1 -- 172.21.15.88:0/676920096 shutdown_connections 2024-02-19T19:35:04.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.301+0000 7f6d64aa0700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d4c06d8a0 0x7f6d4c06fd50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.301+0000 7f6d64aa0700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d600ff3c0 0x7f6d60195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.301+0000 7f6d64aa0700 1 --2- 172.21.15.88:0/676920096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d600ffd60 0x7f6d601963c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.301+0000 7f6d64aa0700 1 -- 172.21.15.88:0/676920096 >> 172.21.15.88:0/676920096 conn(0x7f6d600fac80 msgr2=0x7f6d60068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:04.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.301+0000 7f6d64aa0700 1 -- 172.21.15.88:0/676920096 shutdown_connections 2024-02-19T19:35:04.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.301+0000 7f6d64aa0700 1 -- 172.21.15.88:0/676920096 wait complete. 2024-02-19T19:35:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.403+0000 7fc870159700 1 -- 172.21.15.88:0/689842252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc868068540 msgr2=0x7fc8680689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.403+0000 7fc870159700 1 --2- 172.21.15.88:0/689842252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc868068540 0x7fc8680689b0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc858009910 tx=0x7fc858009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.403+0000 7fc870159700 1 -- 172.21.15.88:0/689842252 shutdown_connections 2024-02-19T19:35:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.403+0000 7fc870159700 1 --2- 172.21.15.88:0/689842252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc868068540 0x7fc8680689b0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.404+0000 7fc870159700 1 --2- 172.21.15.88:0/689842252 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc868106320 0x7fc8681066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.404+0000 7fc870159700 1 -- 172.21.15.88:0/689842252 >> 172.21.15.88:0/689842252 conn(0x7fc868075130 msgr2=0x7fc868075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.404+0000 7fc870159700 1 -- 172.21.15.88:0/689842252 shutdown_connections 2024-02-19T19:35:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.404+0000 7fc870159700 1 -- 172.21.15.88:0/689842252 wait complete. 2024-02-19T19:35:04.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.404+0000 7fc870159700 1 Processor -- start 2024-02-19T19:35:04.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.404+0000 7fc870159700 1 -- start start 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.404+0000 7fc870159700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc868068540 0x7fc868195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.404+0000 7fc870159700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc868106320 0x7fc8681963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.404+0000 7fc870159700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc868196a80 con 0x7fc868068540 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.404+0000 7fc870159700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86819a810 con 0x7fc868106320 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc86def5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc868068540 0x7fc868195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc86def5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc868068540 0x7fc868195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43514/0 (socket says 172.21.15.88:43514) 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc86def5700 1 -- 172.21.15.88:0/613663655 learned_addr learned my addr 172.21.15.88:0/613663655 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc86d6f4700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc868106320 0x7fc8681963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc86def5700 1 -- 172.21.15.88:0/613663655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc868106320 msgr2=0x7fc8681963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc86def5700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc868106320 0x7fc8681963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc86def5700 1 -- 172.21.15.88:0/613663655 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8580095f0 con 0x7fc868068540 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc86d6f4700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc868106320 0x7fc8681963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:35:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc86def5700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc868068540 0x7fc868195e60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc86400d8d0 tx=0x7fc86400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc85effd700 1 -- 172.21.15.88:0/613663655 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc864009940 con 0x7fc868068540 2024-02-19T19:35:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc85effd700 1 -- 172.21.15.88:0/613663655 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc864010460 con 0x7fc868068540 2024-02-19T19:35:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc870159700 1 -- 172.21.15.88:0/613663655 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc86819aaf0 con 0x7fc868068540 2024-02-19T19:35:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc85effd700 1 -- 172.21.15.88:0/613663655 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc86400f5f0 con 0x7fc868068540 2024-02-19T19:35:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.405+0000 7fc870159700 1 -- 172.21.15.88:0/613663655 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc86819b040 con 0x7fc868068540 2024-02-19T19:35:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.406+0000 7fc870159700 1 -- 172.21.15.88:0/613663655 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc868108770 con 0x7fc868068540 2024-02-19T19:35:04.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.407+0000 7fc85effd700 1 -- 172.21.15.88:0/613663655 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fc8640105d0 con 0x7fc868068540 2024-02-19T19:35:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.407+0000 7fc85effd700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc85406d8a0 0x7fc85406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.407+0000 7fc85effd700 1 -- 172.21.15.88:0/613663655 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc86408d540 con 0x7fc868068540 2024-02-19T19:35:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.407+0000 7fc86d6f4700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc85406d8a0 0x7fc85406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.408+0000 7fc86d6f4700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc85406d8a0 0x7fc85406fd50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc858005b40 tx=0x7fc85801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.412+0000 7fc85effd700 1 -- 172.21.15.88:0/613663655 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc864056cf0 con 0x7fc868068540 2024-02-19T19:35:04.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.654+0000 7fc870159700 1 -- 172.21.15.88:0/613663655 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc86804f2a0 con 0x7fc868068540 2024-02-19T19:35:04.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.655+0000 7fc85effd700 1 -- 172.21.15.88:0/613663655 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fc86405a3b0 con 0x7fc868068540 2024-02-19T19:35:04.658 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:35:04.658 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:35:04.658 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:35:04.658 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:35:04.658 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:35:04.658 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:35:04.658 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:35:04.658 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:35:04.658 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:35:04.659 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:35:04.660 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:35:04.661 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:35:04.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.657+0000 7fc870159700 1 -- 172.21.15.88:0/613663655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc85406d8a0 msgr2=0x7fc85406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.657+0000 7fc870159700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc85406d8a0 0x7fc85406fd50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fc858005b40 tx=0x7fc85801a040 comp rx=0 tx=0).stop 2024-02-19T19:35:04.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.657+0000 7fc870159700 1 -- 172.21.15.88:0/613663655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc868068540 msgr2=0x7fc868195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.657+0000 7fc870159700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc868068540 0x7fc868195e60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc86400d8d0 tx=0x7fc86400dc90 comp rx=0 tx=0).stop 2024-02-19T19:35:04.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.658+0000 7fc870159700 1 -- 172.21.15.88:0/613663655 shutdown_connections 2024-02-19T19:35:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.658+0000 7fc870159700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fc85406d8a0 0x7fc85406fd50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.658+0000 7fc870159700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc868068540 0x7fc868195e60 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.658+0000 7fc870159700 1 --2- 172.21.15.88:0/613663655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fc868106320 0x7fc8681963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.658+0000 7fc870159700 1 -- 172.21.15.88:0/613663655 >> 172.21.15.88:0/613663655 conn(0x7fc868075130 msgr2=0x7fc8680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.658+0000 7fc870159700 1 -- 172.21.15.88:0/613663655 shutdown_connections 2024-02-19T19:35:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.658+0000 7fc870159700 1 -- 172.21.15.88:0/613663655 wait complete. 2024-02-19T19:35:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:35:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.763+0000 7fd91bced700 1 -- 172.21.15.88:0/3855736389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9140ffd50 msgr2=0x7fd914103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.763+0000 7fd91bced700 1 --2- 172.21.15.88:0/3855736389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9140ffd50 0x7fd914103d20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fd904009910 tx=0x7fd904009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.763+0000 7fd91bced700 1 -- 172.21.15.88:0/3855736389 shutdown_connections 2024-02-19T19:35:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.763+0000 7fd91bced700 1 --2- 172.21.15.88:0/3855736389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9140ffd50 0x7fd914103d20 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.763+0000 7fd91bced700 1 --2- 172.21.15.88:0/3855736389 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff3b0 0x7fd9140ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.763+0000 7fd91bced700 1 -- 172.21.15.88:0/3855736389 >> 172.21.15.88:0/3855736389 conn(0x7fd9140fac90 msgr2=0x7fd9140fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.763+0000 7fd91bced700 1 -- 172.21.15.88:0/3855736389 shutdown_connections 2024-02-19T19:35:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.763+0000 7fd91bced700 1 -- 172.21.15.88:0/3855736389 wait complete. 2024-02-19T19:35:04.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.763+0000 7fd91bced700 1 Processor -- start 2024-02-19T19:35:04.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.763+0000 7fd91bced700 1 -- start start 2024-02-19T19:35:04.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.763+0000 7fd91bced700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff3b0 0x7fd914196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:04.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd91bced700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9140ffd50 0x7fd914197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd91bced700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914197930 con 0x7fd9140ffd50 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd91bced700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd914197a70 con 0x7fd9140ff3b0 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd919288700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9140ffd50 0x7fd914197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd919288700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9140ffd50 0x7fd914197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43528/0 (socket says 172.21.15.88:43528) 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd919288700 1 -- 172.21.15.88:0/4291205603 learned_addr learned my addr 172.21.15.88:0/4291205603 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd919a89700 1 --2- 172.21.15.88:0/4291205603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff3b0 0x7fd914196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd919288700 1 -- 172.21.15.88:0/4291205603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff3b0 msgr2=0x7fd914196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd919288700 1 --2- 172.21.15.88:0/4291205603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff3b0 0x7fd914196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd919288700 1 -- 172.21.15.88:0/4291205603 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9040095f0 con 0x7fd9140ffd50 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd919288700 1 --2- 172.21.15.88:0/4291205603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9140ffd50 0x7fd914197250 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd9040059d0 tx=0x7fd90400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd90affd700 1 -- 172.21.15.88:0/4291205603 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd90401d070 con 0x7fd9140ffd50 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd90affd700 1 -- 172.21.15.88:0/4291205603 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd90400b610 con 0x7fd9140ffd50 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd91bced700 1 -- 172.21.15.88:0/4291205603 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd91419fd80 con 0x7fd9140ffd50 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd90affd700 1 -- 172.21.15.88:0/4291205603 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd904022470 con 0x7fd9140ffd50 2024-02-19T19:35:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.764+0000 7fd91bced700 1 -- 172.21.15.88:0/4291205603 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9141a01f0 con 0x7fd9140ffd50 2024-02-19T19:35:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.765+0000 7fd91bced700 1 -- 172.21.15.88:0/4291205603 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd914107720 con 0x7fd9140ffd50 2024-02-19T19:35:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.766+0000 7fd90affd700 1 -- 172.21.15.88:0/4291205603 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd90400fc70 con 0x7fd9140ffd50 2024-02-19T19:35:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.766+0000 7fd90affd700 1 --2- 172.21.15.88:0/4291205603 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd90006d8a0 0x7fd90006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.766+0000 7fd90affd700 1 -- 172.21.15.88:0/4291205603 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd90408e8c0 con 0x7fd9140ffd50 2024-02-19T19:35:04.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.766+0000 7fd919a89700 1 --2- 172.21.15.88:0/4291205603 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd90006d8a0 0x7fd90006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:04.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.767+0000 7fd919a89700 1 --2- 172.21.15.88:0/4291205603 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd90006d8a0 0x7fd90006fd50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd910009bf0 tx=0x7fd910009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:04.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.771+0000 7fd90affd700 1 -- 172.21.15.88:0/4291205603 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd9040580f0 con 0x7fd9140ffd50 2024-02-19T19:35:04.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.986+0000 7fd91bced700 1 -- 172.21.15.88:0/4291205603 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd914061190 con 0x7fd90006d8a0 2024-02-19T19:35:04.989 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:04 smithi088 ceph-mon[88643]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:04.989 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:04 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/676920096' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:35:04.989 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:04 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/613663655' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:35:04.989 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:04 smithi088 ceph-mon[88643]: pgmap v946: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:04.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.987+0000 7fd90affd700 1 -- 172.21.15.88:0/4291205603 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd914061190 con 0x7fd90006d8a0 2024-02-19T19:35:04.990 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:35:04.990 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:35:04.990 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:35:04.990 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:35:04.990 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:35:04.990 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:35:04.991 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:35:04.991 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:35:04.991 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:35:04.991 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:35:04.991 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:35:04.991 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:35:04.991 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:35:04.991 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:35:04.991 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:35:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.990+0000 7fd91bced700 1 -- 172.21.15.88:0/4291205603 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd90006d8a0 msgr2=0x7fd90006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.990+0000 7fd91bced700 1 --2- 172.21.15.88:0/4291205603 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd90006d8a0 0x7fd90006fd50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd910009bf0 tx=0x7fd910009260 comp rx=0 tx=0).stop 2024-02-19T19:35:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.990+0000 7fd91bced700 1 -- 172.21.15.88:0/4291205603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9140ffd50 msgr2=0x7fd914197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.990+0000 7fd91bced700 1 --2- 172.21.15.88:0/4291205603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9140ffd50 0x7fd914197250 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd9040059d0 tx=0x7fd90400f740 comp rx=0 tx=0).stop 2024-02-19T19:35:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.990+0000 7fd91bced700 1 -- 172.21.15.88:0/4291205603 shutdown_connections 2024-02-19T19:35:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.990+0000 7fd91bced700 1 --2- 172.21.15.88:0/4291205603 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd90006d8a0 0x7fd90006fd50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.990+0000 7fd91bced700 1 --2- 172.21.15.88:0/4291205603 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd9140ff3b0 0x7fd914196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.990+0000 7fd91bced700 1 --2- 172.21.15.88:0/4291205603 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9140ffd50 0x7fd914197250 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.990+0000 7fd91bced700 1 -- 172.21.15.88:0/4291205603 >> 172.21.15.88:0/4291205603 conn(0x7fd9140fac90 msgr2=0x7fd9140fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.991+0000 7fd91bced700 1 -- 172.21.15.88:0/4291205603 shutdown_connections 2024-02-19T19:35:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:04.991+0000 7fd91bced700 1 -- 172.21.15.88:0/4291205603 wait complete. 2024-02-19T19:35:05.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.095+0000 7f83c7335700 1 -- 172.21.15.88:0/1130211193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c0104130 msgr2=0x7f83c0104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:05.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.095+0000 7f83c7335700 1 --2- 172.21.15.88:0/1130211193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c0104130 0x7f83c0104500 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009910 tx=0x7f83b0009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:05.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.095+0000 7f83c7335700 1 -- 172.21.15.88:0/1130211193 shutdown_connections 2024-02-19T19:35:05.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.095+0000 7f83c7335700 1 --2- 172.21.15.88:0/1130211193 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83c00ffbd0 0x7f83c0100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:05.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.095+0000 7f83c7335700 1 --2- 172.21.15.88:0/1130211193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c0104130 0x7f83c0104500 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:05.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.095+0000 7f83c7335700 1 -- 172.21.15.88:0/1130211193 >> 172.21.15.88:0/1130211193 conn(0x7f83c0075130 msgr2=0x7f83c0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:05.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.095+0000 7f83c7335700 1 -- 172.21.15.88:0/1130211193 shutdown_connections 2024-02-19T19:35:05.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.095+0000 7f83c7335700 1 -- 172.21.15.88:0/1130211193 wait complete. 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.096+0000 7f83c7335700 1 Processor -- start 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.096+0000 7f83c7335700 1 -- start start 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.096+0000 7f83c7335700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83c00ffbd0 0x7f83c0195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.096+0000 7f83c7335700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c0104130 0x7f83c0196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.096+0000 7f83c7335700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c0196a40 con 0x7f83c0104130 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.096+0000 7f83c7335700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c019a7d0 con 0x7f83c00ffbd0 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.096+0000 7f83c48d0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c0104130 0x7f83c0196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.096+0000 7f83c48d0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c0104130 0x7f83c0196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43536/0 (socket says 172.21.15.88:43536) 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.096+0000 7f83c48d0700 1 -- 172.21.15.88:0/646646089 learned_addr learned my addr 172.21.15.88:0/646646089 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.097+0000 7f83c50d1700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83c00ffbd0 0x7f83c0195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.097+0000 7f83c48d0700 1 -- 172.21.15.88:0/646646089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83c00ffbd0 msgr2=0x7f83c0195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:05.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.097+0000 7f83c48d0700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83c00ffbd0 0x7f83c0195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.097+0000 7f83c48d0700 1 -- 172.21.15.88:0/646646089 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b00095f0 con 0x7f83c0104130 2024-02-19T19:35:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.097+0000 7f83c50d1700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83c00ffbd0 0x7f83c0195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:35:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.097+0000 7f83c48d0700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c0104130 0x7f83c0196360 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00c8a0 tx=0x7f83bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.097+0000 7f83b67fc700 1 -- 172.21.15.88:0/646646089 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83bc00cea0 con 0x7f83c0104130 2024-02-19T19:35:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.097+0000 7f83b67fc700 1 -- 172.21.15.88:0/646646089 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83bc004830 con 0x7f83c0104130 2024-02-19T19:35:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.097+0000 7f83b67fc700 1 -- 172.21.15.88:0/646646089 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83bc0055b0 con 0x7f83c0104130 2024-02-19T19:35:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.097+0000 7f83c7335700 1 -- 172.21.15.88:0/646646089 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c019aab0 con 0x7f83c0104130 2024-02-19T19:35:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.097+0000 7f83c7335700 1 -- 172.21.15.88:0/646646089 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c019b000 con 0x7f83c0104130 2024-02-19T19:35:05.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.098+0000 7f83c7335700 1 -- 172.21.15.88:0/646646089 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c01076a0 con 0x7f83c0104130 2024-02-19T19:35:05.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.098+0000 7f83b67fc700 1 -- 172.21.15.88:0/646646089 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f83bc0049a0 con 0x7f83c0104130 2024-02-19T19:35:05.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.099+0000 7f83b67fc700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f83ac06d8a0 0x7f83ac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:05.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.099+0000 7f83b67fc700 1 -- 172.21.15.88:0/646646089 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f83bc08d1d0 con 0x7f83c0104130 2024-02-19T19:35:05.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.099+0000 7f83c50d1700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f83ac06d8a0 0x7f83ac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:05.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.099+0000 7f83c50d1700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f83ac06d8a0 0x7f83ac06fd50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f83b0006010 tx=0x7f83b000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:05.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.103+0000 7f83b67fc700 1 -- 172.21.15.88:0/646646089 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f83bc056980 con 0x7f83c0104130 2024-02-19T19:35:05.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:04 smithi132 ceph-mon[70676]: from='client.35714 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:05.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:04 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/676920096' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:35:05.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:04 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/613663655' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:35:05.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:04 smithi132 ceph-mon[70676]: pgmap v946: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:05.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.386+0000 7f83c7335700 1 -- 172.21.15.88:0/646646089 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f83c019b280 con 0x7f83c0104130 2024-02-19T19:35:05.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.387+0000 7f83b67fc700 1 -- 172.21.15.88:0/646646089 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f83bc004c50 con 0x7f83c0104130 2024-02-19T19:35:05.390 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:35:05.390 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:35:05.390 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:35:05.390 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:35:05.390 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:35:05.390 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:35:05.390 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:35:05.390 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:35:05.390 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:35:05.391 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:35:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.389+0000 7f83c7335700 1 -- 172.21.15.88:0/646646089 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f83ac06d8a0 msgr2=0x7f83ac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.389+0000 7f83c7335700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f83ac06d8a0 0x7f83ac06fd50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f83b0006010 tx=0x7f83b000b540 comp rx=0 tx=0).stop 2024-02-19T19:35:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.389+0000 7f83c7335700 1 -- 172.21.15.88:0/646646089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c0104130 msgr2=0x7f83c0196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.389+0000 7f83c7335700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c0104130 0x7f83c0196360 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f83bc00c8a0 tx=0x7f83bc00cc60 comp rx=0 tx=0).stop 2024-02-19T19:35:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.389+0000 7f83c7335700 1 -- 172.21.15.88:0/646646089 shutdown_connections 2024-02-19T19:35:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.389+0000 7f83c7335700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f83ac06d8a0 0x7f83ac06fd50 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:05.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.389+0000 7f83c7335700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f83c00ffbd0 0x7f83c0195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.389+0000 7f83c7335700 1 --2- 172.21.15.88:0/646646089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83c0104130 0x7f83c0196360 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.389+0000 7f83c7335700 1 -- 172.21.15.88:0/646646089 >> 172.21.15.88:0/646646089 conn(0x7f83c0075130 msgr2=0x7f83c00feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.389+0000 7f83c7335700 1 -- 172.21.15.88:0/646646089 shutdown_connections 2024-02-19T19:35:05.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:05.389+0000 7f83c7335700 1 -- 172.21.15.88:0/646646089 wait complete. 2024-02-19T19:35:06.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:05 smithi088 ceph-mon[88643]: from='client.35726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:06.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:05 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/646646089' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:35:06.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:05 smithi132 ceph-mon[70676]: from='client.35726 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:06.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:05 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/646646089' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:35:07.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:06 smithi088 ceph-mon[88643]: pgmap v947: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:07.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:06 smithi132 ceph-mon[70676]: pgmap v947: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:09.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:08 smithi088 ceph-mon[88643]: pgmap v948: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:08 smithi132 ceph-mon[70676]: pgmap v948: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:11.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:10 smithi088 ceph-mon[88643]: pgmap v949: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:10 smithi132 ceph-mon[70676]: pgmap v949: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:13.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:12 smithi088 ceph-mon[88643]: pgmap v950: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:13.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:12 smithi132 ceph-mon[70676]: pgmap v950: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:14.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:35:14.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:35:15.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:14 smithi088 ceph-mon[88643]: pgmap v951: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:15.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:14 smithi132 ceph-mon[70676]: pgmap v951: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:17.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:16 smithi088 ceph-mon[88643]: pgmap v952: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:17.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:16 smithi132 ceph-mon[70676]: pgmap v952: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:19.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:18 smithi088 ceph-mon[88643]: pgmap v953: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:18 smithi132 ceph-mon[70676]: pgmap v953: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:21.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:20 smithi088 ceph-mon[88643]: pgmap v954: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:21.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:20 smithi132 ceph-mon[70676]: pgmap v954: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:23.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:22 smithi088 ceph-mon[88643]: pgmap v955: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:22 smithi132 ceph-mon[70676]: pgmap v955: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:25.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:24 smithi088 ceph-mon[88643]: pgmap v956: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:25.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:24 smithi132 ceph-mon[70676]: pgmap v956: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:27.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:26 smithi088 ceph-mon[88643]: pgmap v957: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:27.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:26 smithi132 ceph-mon[70676]: pgmap v957: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:28.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:35:28.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:35:29.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:29 smithi088 ceph-mon[88643]: pgmap v958: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:29.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:29 smithi132 ceph-mon[70676]: pgmap v958: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:31.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:30 smithi088 ceph-mon[88643]: pgmap v959: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:31.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:30 smithi132 ceph-mon[70676]: pgmap v959: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:33.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:32 smithi088 ceph-mon[88643]: pgmap v960: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:33.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:32 smithi132 ceph-mon[70676]: pgmap v960: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:35.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:34 smithi088 ceph-mon[88643]: pgmap v961: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:35.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:34 smithi132 ceph-mon[70676]: pgmap v961: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:35.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.507+0000 7f6d1af32700 1 -- 172.21.15.88:0/3225653136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d140ff3c0 msgr2=0x7f6d140ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:35.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.507+0000 7f6d1af32700 1 --2- 172.21.15.88:0/3225653136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d140ff3c0 0x7f6d140ff790 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009910 tx=0x7f6d08009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:35.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.507+0000 7f6d1af32700 1 -- 172.21.15.88:0/3225653136 shutdown_connections 2024-02-19T19:35:35.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.507+0000 7f6d1af32700 1 --2- 172.21.15.88:0/3225653136 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d140ffd60 0x7f6d14103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:35.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.507+0000 7f6d1af32700 1 --2- 172.21.15.88:0/3225653136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d140ff3c0 0x7f6d140ff790 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:35.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.507+0000 7f6d1af32700 1 -- 172.21.15.88:0/3225653136 >> 172.21.15.88:0/3225653136 conn(0x7f6d140fac80 msgr2=0x7f6d140fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:35.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.507+0000 7f6d1af32700 1 -- 172.21.15.88:0/3225653136 shutdown_connections 2024-02-19T19:35:35.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.507+0000 7f6d1af32700 1 -- 172.21.15.88:0/3225653136 wait complete. 2024-02-19T19:35:35.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.508+0000 7f6d1af32700 1 Processor -- start 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.508+0000 7f6d1af32700 1 -- start start 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.508+0000 7f6d1af32700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d140ff3c0 0x7f6d14196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.508+0000 7f6d1af32700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d140ffd60 0x7f6d14197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.508+0000 7f6d1af32700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d14197850 con 0x7f6d140ff3c0 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.508+0000 7f6d1af32700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d14197990 con 0x7f6d140ffd60 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.508+0000 7f6d18cce700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d140ff3c0 0x7f6d14196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.508+0000 7f6d18cce700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d140ff3c0 0x7f6d14196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38876/0 (socket says 172.21.15.88:38876) 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.508+0000 7f6d18cce700 1 -- 172.21.15.88:0/1680662584 learned_addr learned my addr 172.21.15.88:0/1680662584 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.508+0000 7f6d13fff700 1 --2- 172.21.15.88:0/1680662584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d140ffd60 0x7f6d14197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.509+0000 7f6d18cce700 1 -- 172.21.15.88:0/1680662584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d140ffd60 msgr2=0x7f6d14197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.509+0000 7f6d18cce700 1 --2- 172.21.15.88:0/1680662584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d140ffd60 0x7f6d14197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.509+0000 7f6d18cce700 1 -- 172.21.15.88:0/1680662584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080095f0 con 0x7f6d140ff3c0 2024-02-19T19:35:35.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.509+0000 7f6d18cce700 1 --2- 172.21.15.88:0/1680662584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d140ff3c0 0x7f6d14196cf0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6d08006010 tx=0x7f6d0800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:35.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.509+0000 7f6d11ffb700 1 -- 172.21.15.88:0/1680662584 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d0801d070 con 0x7f6d140ff3c0 2024-02-19T19:35:35.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.509+0000 7f6d11ffb700 1 -- 172.21.15.88:0/1680662584 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d08005dc0 con 0x7f6d140ff3c0 2024-02-19T19:35:35.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.509+0000 7f6d11ffb700 1 -- 172.21.15.88:0/1680662584 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d08017400 con 0x7f6d140ff3c0 2024-02-19T19:35:35.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.509+0000 7f6d1af32700 1 -- 172.21.15.88:0/1680662584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1419fd60 con 0x7f6d140ff3c0 2024-02-19T19:35:35.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.509+0000 7f6d1af32700 1 -- 172.21.15.88:0/1680662584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d141a0170 con 0x7f6d140ff3c0 2024-02-19T19:35:35.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.510+0000 7f6d1af32700 1 -- 172.21.15.88:0/1680662584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d141076d0 con 0x7f6d140ff3c0 2024-02-19T19:35:35.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.511+0000 7f6d11ffb700 1 -- 172.21.15.88:0/1680662584 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6d08003a60 con 0x7f6d140ff3c0 2024-02-19T19:35:35.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.511+0000 7f6d11ffb700 1 --2- 172.21.15.88:0/1680662584 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d0406d8a0 0x7f6d0406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:35.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.511+0000 7f6d11ffb700 1 -- 172.21.15.88:0/1680662584 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6d0808e9d0 con 0x7f6d140ff3c0 2024-02-19T19:35:35.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.511+0000 7f6d13fff700 1 --2- 172.21.15.88:0/1680662584 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d0406d8a0 0x7f6d0406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:35.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.512+0000 7f6d13fff700 1 --2- 172.21.15.88:0/1680662584 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d0406d8a0 0x7f6d0406fd50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6d14069680 tx=0x7f6d0000b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:35.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.516+0000 7f6d11ffb700 1 -- 172.21.15.88:0/1680662584 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6d08058180 con 0x7f6d140ff3c0 2024-02-19T19:35:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.731+0000 7f6d1af32700 1 -- 172.21.15.88:0/1680662584 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d14061190 con 0x7f6d0406d8a0 2024-02-19T19:35:35.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.733+0000 7f6d11ffb700 1 -- 172.21.15.88:0/1680662584 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6d14061190 con 0x7f6d0406d8a0 2024-02-19T19:35:35.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.735+0000 7f6d1af32700 1 -- 172.21.15.88:0/1680662584 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d0406d8a0 msgr2=0x7f6d0406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:35.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.735+0000 7f6d1af32700 1 --2- 172.21.15.88:0/1680662584 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d0406d8a0 0x7f6d0406fd50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6d14069680 tx=0x7f6d0000b410 comp rx=0 tx=0).stop 2024-02-19T19:35:35.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.735+0000 7f6d1af32700 1 -- 172.21.15.88:0/1680662584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d140ff3c0 msgr2=0x7f6d14196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:35.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.735+0000 7f6d1af32700 1 --2- 172.21.15.88:0/1680662584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d140ff3c0 0x7f6d14196cf0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6d08006010 tx=0x7f6d0800bd70 comp rx=0 tx=0).stop 2024-02-19T19:35:35.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.735+0000 7f6d1af32700 1 -- 172.21.15.88:0/1680662584 shutdown_connections 2024-02-19T19:35:35.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.735+0000 7f6d1af32700 1 --2- 172.21.15.88:0/1680662584 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6d0406d8a0 0x7f6d0406fd50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:35.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.735+0000 7f6d1af32700 1 --2- 172.21.15.88:0/1680662584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d140ff3c0 0x7f6d14196cf0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:35.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.735+0000 7f6d1af32700 1 --2- 172.21.15.88:0/1680662584 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6d140ffd60 0x7f6d14197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:35.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.735+0000 7f6d1af32700 1 -- 172.21.15.88:0/1680662584 >> 172.21.15.88:0/1680662584 conn(0x7f6d140fac80 msgr2=0x7f6d140fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:35.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.735+0000 7f6d1af32700 1 -- 172.21.15.88:0/1680662584 shutdown_connections 2024-02-19T19:35:35.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.735+0000 7f6d1af32700 1 -- 172.21.15.88:0/1680662584 wait complete. 2024-02-19T19:35:35.747 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:35:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.841+0000 7f491bc29700 1 -- 172.21.15.88:0/2547203943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49140ffc50 msgr2=0x7f4914103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.841+0000 7f491bc29700 1 --2- 172.21.15.88:0/2547203943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49140ffc50 0x7f4914103c20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f4904009910 tx=0x7f4904009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.841+0000 7f491bc29700 1 -- 172.21.15.88:0/2547203943 shutdown_connections 2024-02-19T19:35:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.841+0000 7f491bc29700 1 --2- 172.21.15.88:0/2547203943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49140ffc50 0x7f4914103c20 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.841+0000 7f491bc29700 1 --2- 172.21.15.88:0/2547203943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49140ff2b0 0x7f49140ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.841+0000 7f491bc29700 1 -- 172.21.15.88:0/2547203943 >> 172.21.15.88:0/2547203943 conn(0x7f49140fab50 msgr2=0x7f49140fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.841+0000 7f491bc29700 1 -- 172.21.15.88:0/2547203943 shutdown_connections 2024-02-19T19:35:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.841+0000 7f491bc29700 1 -- 172.21.15.88:0/2547203943 wait complete. 2024-02-19T19:35:35.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.841+0000 7f491bc29700 1 Processor -- start 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.841+0000 7f491bc29700 1 -- start start 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f491bc29700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49140ff2b0 0x7f4914195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f491bc29700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49140ffc50 0x7f49141963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f491bc29700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4914196ad0 con 0x7f49140ffc50 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f491bc29700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491419a860 con 0x7f49140ff2b0 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f49191c4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49140ffc50 0x7f49141963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f49191c4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49140ffc50 0x7f49141963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38892/0 (socket says 172.21.15.88:38892) 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f49191c4700 1 -- 172.21.15.88:0/3523786885 learned_addr learned my addr 172.21.15.88:0/3523786885 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f49191c4700 1 -- 172.21.15.88:0/3523786885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49140ff2b0 msgr2=0x7f4914195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f49191c4700 1 --2- 172.21.15.88:0/3523786885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49140ff2b0 0x7f4914195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f49191c4700 1 -- 172.21.15.88:0/3523786885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49040095f0 con 0x7f49140ffc50 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f49191c4700 1 --2- 172.21.15.88:0/3523786885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49140ffc50 0x7f49141963f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4904005f50 tx=0x7f490400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f490affd700 1 -- 172.21.15.88:0/3523786885 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f490401d070 con 0x7f49140ffc50 2024-02-19T19:35:35.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f491bc29700 1 -- 172.21.15.88:0/3523786885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491419aae0 con 0x7f49140ffc50 2024-02-19T19:35:35.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f490affd700 1 -- 172.21.15.88:0/3523786885 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f490400f460 con 0x7f49140ffc50 2024-02-19T19:35:35.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f490affd700 1 -- 172.21.15.88:0/3523786885 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4904017630 con 0x7f49140ffc50 2024-02-19T19:35:35.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.842+0000 7f491bc29700 1 -- 172.21.15.88:0/3523786885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491419afd0 con 0x7f49140ffc50 2024-02-19T19:35:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.843+0000 7f491bc29700 1 -- 172.21.15.88:0/3523786885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f491404f2a0 con 0x7f49140ffc50 2024-02-19T19:35:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.844+0000 7f490affd700 1 -- 172.21.15.88:0/3523786885 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f490400f5d0 con 0x7f49140ffc50 2024-02-19T19:35:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.844+0000 7f490affd700 1 --2- 172.21.15.88:0/3523786885 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490006d8a0 0x7f490006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.844+0000 7f490affd700 1 -- 172.21.15.88:0/3523786885 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f490408eb20 con 0x7f49140ffc50 2024-02-19T19:35:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.844+0000 7f49199c5700 1 --2- 172.21.15.88:0/3523786885 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490006d8a0 0x7f490006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.845+0000 7f49199c5700 1 --2- 172.21.15.88:0/3523786885 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490006d8a0 0x7f490006fd50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f490c00ab90 tx=0x7f490c009250 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:35.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:35.849+0000 7f490affd700 1 -- 172.21.15.88:0/3523786885 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f49040582d0 con 0x7f49140ffc50 2024-02-19T19:35:36.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.061+0000 7f491bc29700 1 -- 172.21.15.88:0/3523786885 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f491419b3c0 con 0x7f490006d8a0 2024-02-19T19:35:36.064 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:35 smithi088 ceph-mon[88643]: from='client.35734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:36.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.063+0000 7f490affd700 1 -- 172.21.15.88:0/3523786885 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f491419b3c0 con 0x7f490006d8a0 2024-02-19T19:35:36.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.065+0000 7f491bc29700 1 -- 172.21.15.88:0/3523786885 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490006d8a0 msgr2=0x7f490006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.065+0000 7f491bc29700 1 --2- 172.21.15.88:0/3523786885 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490006d8a0 0x7f490006fd50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f490c00ab90 tx=0x7f490c009250 comp rx=0 tx=0).stop 2024-02-19T19:35:36.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.065+0000 7f491bc29700 1 -- 172.21.15.88:0/3523786885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49140ffc50 msgr2=0x7f49141963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.065+0000 7f491bc29700 1 --2- 172.21.15.88:0/3523786885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49140ffc50 0x7f49141963f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4904005f50 tx=0x7f490400be20 comp rx=0 tx=0).stop 2024-02-19T19:35:36.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.065+0000 7f491bc29700 1 -- 172.21.15.88:0/3523786885 shutdown_connections 2024-02-19T19:35:36.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.065+0000 7f491bc29700 1 --2- 172.21.15.88:0/3523786885 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f490006d8a0 0x7f490006fd50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.065+0000 7f491bc29700 1 --2- 172.21.15.88:0/3523786885 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49140ff2b0 0x7f4914195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.065+0000 7f491bc29700 1 --2- 172.21.15.88:0/3523786885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49140ffc50 0x7f49141963f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.065+0000 7f491bc29700 1 -- 172.21.15.88:0/3523786885 >> 172.21.15.88:0/3523786885 conn(0x7f49140fab50 msgr2=0x7f4914103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:36.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.065+0000 7f491bc29700 1 -- 172.21.15.88:0/3523786885 shutdown_connections 2024-02-19T19:35:36.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.065+0000 7f491bc29700 1 -- 172.21.15.88:0/3523786885 wait complete. 2024-02-19T19:35:36.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.168+0000 7f1ed04e6700 1 -- 172.21.15.88:0/151096178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec80ff3c0 msgr2=0x7f1ec80ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.168+0000 7f1ed04e6700 1 --2- 172.21.15.88:0/151096178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec80ff3c0 0x7f1ec80ff790 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009910 tx=0x7f1ec4009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:36.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.168+0000 7f1ed04e6700 1 -- 172.21.15.88:0/151096178 shutdown_connections 2024-02-19T19:35:36.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.168+0000 7f1ed04e6700 1 --2- 172.21.15.88:0/151096178 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec80ffd60 0x7f1ec8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ed04e6700 1 --2- 172.21.15.88:0/151096178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec80ff3c0 0x7f1ec80ff790 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ed04e6700 1 -- 172.21.15.88:0/151096178 >> 172.21.15.88:0/151096178 conn(0x7f1ec80fac80 msgr2=0x7f1ec80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:36.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ed04e6700 1 -- 172.21.15.88:0/151096178 shutdown_connections 2024-02-19T19:35:36.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ed04e6700 1 -- 172.21.15.88:0/151096178 wait complete. 2024-02-19T19:35:36.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ed04e6700 1 Processor -- start 2024-02-19T19:35:36.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ed04e6700 1 -- start start 2024-02-19T19:35:36.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ed04e6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec80ff3c0 0x7f1ec8195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:36.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ed04e6700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec80ffd60 0x7f1ec81964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:36.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ed04e6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec8196b90 con 0x7f1ec80ff3c0 2024-02-19T19:35:36.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ed04e6700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec819a920 con 0x7f1ec80ffd60 2024-02-19T19:35:36.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ece282700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec80ff3c0 0x7f1ec8195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:36.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ece282700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec80ff3c0 0x7f1ec8195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38898/0 (socket says 172.21.15.88:38898) 2024-02-19T19:35:36.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.169+0000 7f1ece282700 1 -- 172.21.15.88:0/3182788698 learned_addr learned my addr 172.21.15.88:0/3182788698 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:36.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.170+0000 7f1ecda81700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec80ffd60 0x7f1ec81964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:36.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.170+0000 7f1ece282700 1 -- 172.21.15.88:0/3182788698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec80ffd60 msgr2=0x7f1ec81964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.170+0000 7f1ece282700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec80ffd60 0x7f1ec81964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.170+0000 7f1ece282700 1 -- 172.21.15.88:0/3182788698 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec40095f0 con 0x7f1ec80ff3c0 2024-02-19T19:35:36.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.170+0000 7f1ecda81700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec80ffd60 0x7f1ec81964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:35:36.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.170+0000 7f1ece282700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec80ff3c0 0x7f1ec8195f70 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4006010 tx=0x7f1ec400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:36.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.170+0000 7f1ebf7fe700 1 -- 172.21.15.88:0/3182788698 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec401d070 con 0x7f1ec80ff3c0 2024-02-19T19:35:36.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.170+0000 7f1ed04e6700 1 -- 172.21.15.88:0/3182788698 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec819aba0 con 0x7f1ec80ff3c0 2024-02-19T19:35:36.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.170+0000 7f1ebf7fe700 1 -- 172.21.15.88:0/3182788698 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ec4004fb0 con 0x7f1ec80ff3c0 2024-02-19T19:35:36.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.170+0000 7f1ebf7fe700 1 -- 172.21.15.88:0/3182788698 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ec4022ea0 con 0x7f1ec80ff3c0 2024-02-19T19:35:36.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.170+0000 7f1ed04e6700 1 -- 172.21.15.88:0/3182788698 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec819b090 con 0x7f1ec80ff3c0 2024-02-19T19:35:36.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.171+0000 7f1ed04e6700 1 -- 172.21.15.88:0/3182788698 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec81086e0 con 0x7f1ec80ff3c0 2024-02-19T19:35:36.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.171+0000 7f1ebf7fe700 1 -- 172.21.15.88:0/3182788698 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1ec4022470 con 0x7f1ec80ff3c0 2024-02-19T19:35:36.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.172+0000 7f1ebf7fe700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb406d8a0 0x7f1eb406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:36.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.172+0000 7f1ebf7fe700 1 -- 172.21.15.88:0/3182788698 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1ec408ebb0 con 0x7f1ec80ff3c0 2024-02-19T19:35:36.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.172+0000 7f1ecda81700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb406d8a0 0x7f1eb406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:36.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.172+0000 7f1ecda81700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb406d8a0 0x7f1eb406fd50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8197590 tx=0x7f1eb8008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:36.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.177+0000 7f1ebf7fe700 1 -- 172.21.15.88:0/3182788698 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1ec4095080 con 0x7f1ec80ff3c0 2024-02-19T19:35:36.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:35 smithi132 ceph-mon[70676]: from='client.35734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:36.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.386+0000 7f1ed04e6700 1 -- 172.21.15.88:0/3182788698 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ec8197410 con 0x7f1eb406d8a0 2024-02-19T19:35:36.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.396+0000 7f1ebf7fe700 1 -- 172.21.15.88:0/3182788698 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1eb8007d40 con 0x7f1eb406d8a0 2024-02-19T19:35:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:35:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (32m) 9m ago 43m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:35:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (43m) 9m ago 43m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:35:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (25m) 9m ago 41m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:35:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (28m) 9m ago 43m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:35:36.398 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (28m) 9m ago 41m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (32m) 9m ago 42m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (25m) 9m ago 36m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (24m) 9m ago 36m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (24m) 9m ago 36m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (9m) 9m ago 36m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (34m) 9m ago 44m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (33m) 9m ago 41m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (30m) 9m ago 45m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (30m) 9m ago 41m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (33m) 9m ago 43m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (33m) 9m ago 41m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (28m) 9m ago 40m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (27m) 9m ago 40m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (27m) 9m ago 39m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:35:36.399 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (26m) 9m ago 39m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:35:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (25m) 9m ago 38m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:35:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (25m) 9m ago 38m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:35:36.400 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (33m) 9m ago 42m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:35:36.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.398+0000 7f1ed04e6700 1 -- 172.21.15.88:0/3182788698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb406d8a0 msgr2=0x7f1eb406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.398+0000 7f1ed04e6700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb406d8a0 0x7f1eb406fd50 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8197590 tx=0x7f1eb8008040 comp rx=0 tx=0).stop 2024-02-19T19:35:36.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.398+0000 7f1ed04e6700 1 -- 172.21.15.88:0/3182788698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec80ff3c0 msgr2=0x7f1ec8195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.398+0000 7f1ed04e6700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec80ff3c0 0x7f1ec8195f70 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4006010 tx=0x7f1ec400be20 comp rx=0 tx=0).stop 2024-02-19T19:35:36.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.398+0000 7f1ed04e6700 1 -- 172.21.15.88:0/3182788698 shutdown_connections 2024-02-19T19:35:36.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.398+0000 7f1ed04e6700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb406d8a0 0x7f1eb406fd50 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.398+0000 7f1ed04e6700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec80ff3c0 0x7f1ec8195f70 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.398+0000 7f1ed04e6700 1 --2- 172.21.15.88:0/3182788698 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec80ffd60 0x7f1ec81964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.398+0000 7f1ed04e6700 1 -- 172.21.15.88:0/3182788698 >> 172.21.15.88:0/3182788698 conn(0x7f1ec80fac80 msgr2=0x7f1ec8069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:36.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.398+0000 7f1ed04e6700 1 -- 172.21.15.88:0/3182788698 shutdown_connections 2024-02-19T19:35:36.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.398+0000 7f1ed04e6700 1 -- 172.21.15.88:0/3182788698 wait complete. 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.501+0000 7f53cec76700 1 -- 172.21.15.88:0/2089061184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c80ffd60 msgr2=0x7f53c8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.501+0000 7f53cec76700 1 --2- 172.21.15.88:0/2089061184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c80ffd60 0x7f53c8103d30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009910 tx=0x7f53b0009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.501+0000 7f53cec76700 1 -- 172.21.15.88:0/2089061184 shutdown_connections 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.501+0000 7f53cec76700 1 --2- 172.21.15.88:0/2089061184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c80ffd60 0x7f53c8103d30 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.501+0000 7f53cec76700 1 --2- 172.21.15.88:0/2089061184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53c80ff3c0 0x7f53c80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.501+0000 7f53cec76700 1 -- 172.21.15.88:0/2089061184 >> 172.21.15.88:0/2089061184 conn(0x7f53c80fac80 msgr2=0x7f53c80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.501+0000 7f53cec76700 1 -- 172.21.15.88:0/2089061184 shutdown_connections 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.501+0000 7f53cec76700 1 -- 172.21.15.88:0/2089061184 wait complete. 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53cec76700 1 Processor -- start 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53cec76700 1 -- start start 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53cec76700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53c80ff3c0 0x7f53c8100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:36.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53cec76700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c80ffd60 0x7f53c8101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53cec76700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8101850 con 0x7f53c80ffd60 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53cec76700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c8101990 con 0x7f53c80ff3c0 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c80ffd60 0x7f53c8101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53c7fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c80ffd60 0x7f53c8101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38918/0 (socket says 172.21.15.88:38918) 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53c7fff700 1 -- 172.21.15.88:0/1095746984 learned_addr learned my addr 172.21.15.88:0/1095746984 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53cca12700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53c80ff3c0 0x7f53c8100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53c7fff700 1 -- 172.21.15.88:0/1095746984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53c80ff3c0 msgr2=0x7f53c8100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53c7fff700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53c80ff3c0 0x7f53c8100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53c7fff700 1 -- 172.21.15.88:0/1095746984 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b00095f0 con 0x7f53c80ffd60 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.502+0000 7f53cca12700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53c80ff3c0 0x7f53c8100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.503+0000 7f53c7fff700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c80ffd60 0x7f53c8101310 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f53b0000c00 tx=0x7f53b000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.503+0000 7f53c5ffb700 1 -- 172.21.15.88:0/1095746984 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53b001d070 con 0x7f53c80ffd60 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.503+0000 7f53c5ffb700 1 -- 172.21.15.88:0/1095746984 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f53b0005dc0 con 0x7f53c80ffd60 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.503+0000 7f53c5ffb700 1 -- 172.21.15.88:0/1095746984 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53b0017400 con 0x7f53c80ffd60 2024-02-19T19:35:36.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.503+0000 7f53cec76700 1 -- 172.21.15.88:0/1095746984 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c8101c10 con 0x7f53c80ffd60 2024-02-19T19:35:36.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.503+0000 7f53cec76700 1 -- 172.21.15.88:0/1095746984 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c81a2390 con 0x7f53c80ffd60 2024-02-19T19:35:36.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.504+0000 7f53cec76700 1 -- 172.21.15.88:0/1095746984 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c8108740 con 0x7f53c80ffd60 2024-02-19T19:35:36.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.504+0000 7f53c5ffb700 1 -- 172.21.15.88:0/1095746984 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f53b000f7f0 con 0x7f53c80ffd60 2024-02-19T19:35:36.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.505+0000 7f53c5ffb700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f53b406d8a0 0x7f53b406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.505+0000 7f53c5ffb700 1 -- 172.21.15.88:0/1095746984 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f53b008e920 con 0x7f53c80ffd60 2024-02-19T19:35:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.505+0000 7f53cca12700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f53b406d8a0 0x7f53b406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:36.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.505+0000 7f53cca12700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f53b406d8a0 0x7f53b406fd50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f53c00094f0 tx=0x7f53c0009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:36.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.509+0000 7f53c5ffb700 1 -- 172.21.15.88:0/1095746984 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f53b0058150 con 0x7f53c80ffd60 2024-02-19T19:35:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.788+0000 7f53cec76700 1 -- 172.21.15.88:0/1095746984 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f53c804f2a0 con 0x7f53c80ffd60 2024-02-19T19:35:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.789+0000 7f53c5ffb700 1 -- 172.21.15.88:0/1095746984 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f53b005b810 con 0x7f53c80ffd60 2024-02-19T19:35:36.790 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:35:36.791 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:35:36.792 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:35:36.792 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:35:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.791+0000 7f53cec76700 1 -- 172.21.15.88:0/1095746984 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f53b406d8a0 msgr2=0x7f53b406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.791+0000 7f53cec76700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f53b406d8a0 0x7f53b406fd50 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f53c00094f0 tx=0x7f53c0009040 comp rx=0 tx=0).stop 2024-02-19T19:35:36.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.791+0000 7f53cec76700 1 -- 172.21.15.88:0/1095746984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c80ffd60 msgr2=0x7f53c8101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.791+0000 7f53cec76700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c80ffd60 0x7f53c8101310 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f53b0000c00 tx=0x7f53b000be20 comp rx=0 tx=0).stop 2024-02-19T19:35:36.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.791+0000 7f53cec76700 1 -- 172.21.15.88:0/1095746984 shutdown_connections 2024-02-19T19:35:36.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.791+0000 7f53cec76700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f53b406d8a0 0x7f53b406fd50 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.791+0000 7f53cec76700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f53c80ff3c0 0x7f53c8100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.791+0000 7f53cec76700 1 --2- 172.21.15.88:0/1095746984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53c80ffd60 0x7f53c8101310 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.791+0000 7f53cec76700 1 -- 172.21.15.88:0/1095746984 >> 172.21.15.88:0/1095746984 conn(0x7f53c80fac80 msgr2=0x7f53c80690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:36.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.792+0000 7f53cec76700 1 -- 172.21.15.88:0/1095746984 shutdown_connections 2024-02-19T19:35:36.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.792+0000 7f53cec76700 1 -- 172.21.15.88:0/1095746984 wait complete. 2024-02-19T19:35:36.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.895+0000 7f66de102700 1 -- 172.21.15.88:0/3066652047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66d8101420 msgr2=0x7f66d81017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.895+0000 7f66de102700 1 --2- 172.21.15.88:0/3066652047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66d8101420 0x7f66d81017f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f66d000b0a0 tx=0x7f66d000b3b0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.895+0000 7f66de102700 1 -- 172.21.15.88:0/3066652047 shutdown_connections 2024-02-19T19:35:36.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.895+0000 7f66de102700 1 --2- 172.21.15.88:0/3066652047 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d8101d30 0x7f66d810a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.895+0000 7f66de102700 1 --2- 172.21.15.88:0/3066652047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66d8101420 0x7f66d81017f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.895+0000 7f66de102700 1 -- 172.21.15.88:0/3066652047 >> 172.21.15.88:0/3066652047 conn(0x7f66d80fac80 msgr2=0x7f66d80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:36.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.895+0000 7f66de102700 1 -- 172.21.15.88:0/3066652047 shutdown_connections 2024-02-19T19:35:36.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.895+0000 7f66de102700 1 -- 172.21.15.88:0/3066652047 wait complete. 2024-02-19T19:35:36.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.895+0000 7f66de102700 1 Processor -- start 2024-02-19T19:35:36.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66de102700 1 -- start start 2024-02-19T19:35:36.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66de102700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66d8101420 0x7f66d8195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:36.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66de102700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d8101d30 0x7f66d81963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:36.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66de102700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d8196a90 con 0x7f66d8101420 2024-02-19T19:35:36.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66de102700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66d819a820 con 0x7f66d8101d30 2024-02-19T19:35:36.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66d77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66d8101420 0x7f66d8195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:36.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66d77fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66d8101420 0x7f66d8195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38942/0 (socket says 172.21.15.88:38942) 2024-02-19T19:35:36.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66d77fe700 1 -- 172.21.15.88:0/3163993795 learned_addr learned my addr 172.21.15.88:0/3163993795 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:36.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66d6ffd700 1 --2- 172.21.15.88:0/3163993795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d8101d30 0x7f66d81963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:36.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66d77fe700 1 -- 172.21.15.88:0/3163993795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d8101d30 msgr2=0x7f66d81963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:36.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66d77fe700 1 --2- 172.21.15.88:0/3163993795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d8101d30 0x7f66d81963b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:36.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66d77fe700 1 -- 172.21.15.88:0/3163993795 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66d0009d00 con 0x7f66d8101420 2024-02-19T19:35:36.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.896+0000 7f66d77fe700 1 --2- 172.21.15.88:0/3163993795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66d8101420 0x7f66d8195e70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f66d0000f80 tx=0x7f66d000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:36.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.897+0000 7f66d4ff9700 1 -- 172.21.15.88:0/3163993795 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d000e040 con 0x7f66d8101420 2024-02-19T19:35:36.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.897+0000 7f66de102700 1 -- 172.21.15.88:0/3163993795 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66d819aaa0 con 0x7f66d8101420 2024-02-19T19:35:36.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.897+0000 7f66d4ff9700 1 -- 172.21.15.88:0/3163993795 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66d0003de0 con 0x7f66d8101420 2024-02-19T19:35:36.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.897+0000 7f66d4ff9700 1 -- 172.21.15.88:0/3163993795 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66d001d790 con 0x7f66d8101420 2024-02-19T19:35:36.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.897+0000 7f66de102700 1 -- 172.21.15.88:0/3163993795 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66d819af90 con 0x7f66d8101420 2024-02-19T19:35:36.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.898+0000 7f66de102700 1 -- 172.21.15.88:0/3163993795 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66d8107900 con 0x7f66d8101420 2024-02-19T19:35:36.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.898+0000 7f66d4ff9700 1 -- 172.21.15.88:0/3163993795 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f66d0004040 con 0x7f66d8101420 2024-02-19T19:35:36.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.898+0000 7f66d4ff9700 1 --2- 172.21.15.88:0/3163993795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66c406d8a0 0x7f66c406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:36.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.899+0000 7f66d4ff9700 1 -- 172.21.15.88:0/3163993795 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f66d008ed60 con 0x7f66d8101420 2024-02-19T19:35:36.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.899+0000 7f66d6ffd700 1 --2- 172.21.15.88:0/3163993795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66c406d8a0 0x7f66c406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:36.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.899+0000 7f66d6ffd700 1 --2- 172.21.15.88:0/3163993795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66c406d8a0 0x7f66c406fd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f66c0005950 tx=0x7f66c00058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:36.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:36.903+0000 7f66d4ff9700 1 -- 172.21.15.88:0/3163993795 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f66d0058510 con 0x7f66d8101420 2024-02-19T19:35:37.059 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:36 smithi132 ceph-mon[70676]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:37.060 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:36 smithi132 ceph-mon[70676]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:37.060 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:36 smithi132 ceph-mon[70676]: pgmap v962: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:37.060 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:36 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1095746984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:35:37.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:36 smithi088 ceph-mon[88643]: from='client.35738 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:37.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:36 smithi088 ceph-mon[88643]: from='client.35742 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:37.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:36 smithi088 ceph-mon[88643]: pgmap v962: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:37.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:36 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1095746984' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:35:37.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.143+0000 7f66de102700 1 -- 172.21.15.88:0/3163993795 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f66d81072e0 con 0x7f66d8101420 2024-02-19T19:35:37.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.143+0000 7f66d4ff9700 1 -- 172.21.15.88:0/3163993795 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f66d005bbd0 con 0x7f66d8101420 2024-02-19T19:35:37.145 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:35:37.145 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:35:37.145 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:35:37.145 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:35:37.145 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:35:37.146 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:35:37.147 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:35:37.148 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:35:37.148 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:35:37.148 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:35:37.148 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:35:37.148 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:35:37.148 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:35:37.148 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:35:37.148 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:35:37.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.146+0000 7f66de102700 1 -- 172.21.15.88:0/3163993795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66c406d8a0 msgr2=0x7f66c406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:37.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.146+0000 7f66de102700 1 --2- 172.21.15.88:0/3163993795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66c406d8a0 0x7f66c406fd50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f66c0005950 tx=0x7f66c00058e0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.146+0000 7f66de102700 1 -- 172.21.15.88:0/3163993795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66d8101420 msgr2=0x7f66d8195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:37.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.146+0000 7f66de102700 1 --2- 172.21.15.88:0/3163993795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66d8101420 0x7f66d8195e70 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f66d0000f80 tx=0x7f66d000cdd0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.146+0000 7f66de102700 1 -- 172.21.15.88:0/3163993795 shutdown_connections 2024-02-19T19:35:37.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.146+0000 7f66de102700 1 --2- 172.21.15.88:0/3163993795 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f66c406d8a0 0x7f66c406fd50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.146+0000 7f66de102700 1 --2- 172.21.15.88:0/3163993795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66d8101420 0x7f66d8195e70 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.146+0000 7f66de102700 1 --2- 172.21.15.88:0/3163993795 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f66d8101d30 0x7f66d81963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.146+0000 7f66de102700 1 -- 172.21.15.88:0/3163993795 >> 172.21.15.88:0/3163993795 conn(0x7f66d80fac80 msgr2=0x7f66d8104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:37.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.146+0000 7f66de102700 1 -- 172.21.15.88:0/3163993795 shutdown_connections 2024-02-19T19:35:37.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.146+0000 7f66de102700 1 -- 172.21.15.88:0/3163993795 wait complete. 2024-02-19T19:35:37.153 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:35:37.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.251+0000 7fe64a1a4700 1 -- 172.21.15.88:0/2390999063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6440ffd60 msgr2=0x7fe644103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:37.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.251+0000 7fe64a1a4700 1 --2- 172.21.15.88:0/2390999063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6440ffd60 0x7fe644103d30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fe62c009910 tx=0x7fe62c009c20 comp rx=0 tx=0).stop 2024-02-19T19:35:37.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.251+0000 7fe64a1a4700 1 -- 172.21.15.88:0/2390999063 shutdown_connections 2024-02-19T19:35:37.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.251+0000 7fe64a1a4700 1 --2- 172.21.15.88:0/2390999063 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6440ffd60 0x7fe644103d30 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.251+0000 7fe64a1a4700 1 --2- 172.21.15.88:0/2390999063 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6440ff3c0 0x7fe6440ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.251+0000 7fe64a1a4700 1 -- 172.21.15.88:0/2390999063 >> 172.21.15.88:0/2390999063 conn(0x7fe6440fac80 msgr2=0x7fe6440fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:37.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.251+0000 7fe64a1a4700 1 -- 172.21.15.88:0/2390999063 shutdown_connections 2024-02-19T19:35:37.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.251+0000 7fe64a1a4700 1 -- 172.21.15.88:0/2390999063 wait complete. 2024-02-19T19:35:37.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.252+0000 7fe64a1a4700 1 Processor -- start 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.252+0000 7fe64a1a4700 1 -- start start 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.252+0000 7fe64a1a4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6440ff3c0 0x7fe644194b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.252+0000 7fe64a1a4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6440ffd60 0x7fe6441950b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.252+0000 7fe64a1a4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe644069d90 con 0x7fe6440ffd60 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.252+0000 7fe64a1a4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6441955f0 con 0x7fe6440ff3c0 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.252+0000 7fe63bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6440ffd60 0x7fe6441950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.252+0000 7fe63bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6440ffd60 0x7fe6441950b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38952/0 (socket says 172.21.15.88:38952) 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.252+0000 7fe63bfff700 1 -- 172.21.15.88:0/213291485 learned_addr learned my addr 172.21.15.88:0/213291485 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.252+0000 7fe6437fe700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6440ff3c0 0x7fe644194b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.253+0000 7fe63bfff700 1 -- 172.21.15.88:0/213291485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6440ff3c0 msgr2=0x7fe644194b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.253+0000 7fe63bfff700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6440ff3c0 0x7fe644194b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.253+0000 7fe63bfff700 1 -- 172.21.15.88:0/213291485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c0095f0 con 0x7fe6440ffd60 2024-02-19T19:35:37.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.253+0000 7fe6437fe700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6440ff3c0 0x7fe644194b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:35:37.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.253+0000 7fe63bfff700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6440ffd60 0x7fe6441950b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00bda0 tx=0x7fe62c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:37.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.253+0000 7fe6417fa700 1 -- 172.21.15.88:0/213291485 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe62c01d070 con 0x7fe6440ffd60 2024-02-19T19:35:37.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.253+0000 7fe64a1a4700 1 -- 172.21.15.88:0/213291485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe644195870 con 0x7fe6440ffd60 2024-02-19T19:35:37.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.253+0000 7fe6417fa700 1 -- 172.21.15.88:0/213291485 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe62c004fb0 con 0x7fe6440ffd60 2024-02-19T19:35:37.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.253+0000 7fe6417fa700 1 -- 172.21.15.88:0/213291485 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe62c00f480 con 0x7fe6440ffd60 2024-02-19T19:35:37.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.253+0000 7fe64a1a4700 1 -- 172.21.15.88:0/213291485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6441a2400 con 0x7fe6440ffd60 2024-02-19T19:35:37.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.254+0000 7fe64a1a4700 1 -- 172.21.15.88:0/213291485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe644107730 con 0x7fe6440ffd60 2024-02-19T19:35:37.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.255+0000 7fe6417fa700 1 -- 172.21.15.88:0/213291485 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe62c022870 con 0x7fe6440ffd60 2024-02-19T19:35:37.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.255+0000 7fe6417fa700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe63006d8a0 0x7fe63006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:37.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.255+0000 7fe6417fa700 1 -- 172.21.15.88:0/213291485 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe62c08ecd0 con 0x7fe6440ffd60 2024-02-19T19:35:37.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.255+0000 7fe6437fe700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe63006d8a0 0x7fe63006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:37.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.255+0000 7fe6437fe700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe63006d8a0 0x7fe63006fd50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe634005fd0 tx=0x7fe634005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:37.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.260+0000 7fe6417fa700 1 -- 172.21.15.88:0/213291485 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe62c05c180 con 0x7fe6440ffd60 2024-02-19T19:35:37.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.479+0000 7fe64a1a4700 1 -- 172.21.15.88:0/213291485 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe644061190 con 0x7fe63006d8a0 2024-02-19T19:35:37.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.480+0000 7fe6417fa700 1 -- 172.21.15.88:0/213291485 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe644061190 con 0x7fe63006d8a0 2024-02-19T19:35:37.482 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:35:37.482 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:35:37.482 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:35:37.482 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:35:37.483 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:35:37.483 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:35:37.483 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:35:37.483 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:35:37.483 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:35:37.483 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:35:37.483 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:35:37.483 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:35:37.483 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:35:37.483 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:35:37.483 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:35:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.482+0000 7fe64a1a4700 1 -- 172.21.15.88:0/213291485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe63006d8a0 msgr2=0x7fe63006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.482+0000 7fe64a1a4700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe63006d8a0 0x7fe63006fd50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fe634005fd0 tx=0x7fe634005e30 comp rx=0 tx=0).stop 2024-02-19T19:35:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.482+0000 7fe64a1a4700 1 -- 172.21.15.88:0/213291485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6440ffd60 msgr2=0x7fe6441950b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.482+0000 7fe64a1a4700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6440ffd60 0x7fe6441950b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00bda0 tx=0x7fe62c00be80 comp rx=0 tx=0).stop 2024-02-19T19:35:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.483+0000 7fe64a1a4700 1 -- 172.21.15.88:0/213291485 shutdown_connections 2024-02-19T19:35:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.483+0000 7fe64a1a4700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe63006d8a0 0x7fe63006fd50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.483+0000 7fe64a1a4700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe6440ff3c0 0x7fe644194b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.483+0000 7fe64a1a4700 1 --2- 172.21.15.88:0/213291485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6440ffd60 0x7fe6441950b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.483+0000 7fe64a1a4700 1 -- 172.21.15.88:0/213291485 >> 172.21.15.88:0/213291485 conn(0x7fe6440fac80 msgr2=0x7fe6440fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.483+0000 7fe64a1a4700 1 -- 172.21.15.88:0/213291485 shutdown_connections 2024-02-19T19:35:37.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.483+0000 7fe64a1a4700 1 -- 172.21.15.88:0/213291485 wait complete. 2024-02-19T19:35:37.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.587+0000 7f6933fc8700 1 -- 172.21.15.88:0/4057826211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff350 msgr2=0x7f692c0ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:37.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.587+0000 7f6933fc8700 1 --2- 172.21.15.88:0/4057826211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff350 0x7f692c0ff720 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f692400b0a0 tx=0x7f692400b3b0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.587+0000 7f6933fc8700 1 -- 172.21.15.88:0/4057826211 shutdown_connections 2024-02-19T19:35:37.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.587+0000 7f6933fc8700 1 --2- 172.21.15.88:0/4057826211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692c0ffc60 0x7f692c10a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.587+0000 7f6933fc8700 1 --2- 172.21.15.88:0/4057826211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff350 0x7f692c0ff720 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.587+0000 7f6933fc8700 1 -- 172.21.15.88:0/4057826211 >> 172.21.15.88:0/4057826211 conn(0x7f692c0748b0 msgr2=0x7f692c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:37.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.587+0000 7f6933fc8700 1 -- 172.21.15.88:0/4057826211 shutdown_connections 2024-02-19T19:35:37.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.587+0000 7f6933fc8700 1 -- 172.21.15.88:0/4057826211 wait complete. 2024-02-19T19:35:37.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.587+0000 7f6933fc8700 1 Processor -- start 2024-02-19T19:35:37.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6933fc8700 1 -- start start 2024-02-19T19:35:37.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6933fc8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff350 0x7f692c195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:37.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6933fc8700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692c0ffc60 0x7f692c1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:37.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6933fc8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c196ac0 con 0x7f692c0ff350 2024-02-19T19:35:37.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6933fc8700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c19a850 con 0x7f692c0ffc60 2024-02-19T19:35:37.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6931d64700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff350 0x7f692c195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:37.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6931d64700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff350 0x7f692c195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38978/0 (socket says 172.21.15.88:38978) 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6931d64700 1 -- 172.21.15.88:0/515452023 learned_addr learned my addr 172.21.15.88:0/515452023 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6931563700 1 --2- 172.21.15.88:0/515452023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692c0ffc60 0x7f692c1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6931d64700 1 -- 172.21.15.88:0/515452023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692c0ffc60 msgr2=0x7f692c1963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6931d64700 1 --2- 172.21.15.88:0/515452023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692c0ffc60 0x7f692c1963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6931d64700 1 -- 172.21.15.88:0/515452023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924009d00 con 0x7f692c0ff350 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6931d64700 1 --2- 172.21.15.88:0/515452023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff350 0x7f692c195ea0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6924006130 tx=0x7f692400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6922ffd700 1 -- 172.21.15.88:0/515452023 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f692400e040 con 0x7f692c0ff350 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6922ffd700 1 -- 172.21.15.88:0/515452023 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6924003de0 con 0x7f692c0ff350 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6933fc8700 1 -- 172.21.15.88:0/515452023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c19aad0 con 0x7f692c0ff350 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6922ffd700 1 -- 172.21.15.88:0/515452023 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f692401d740 con 0x7f692c0ff350 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.588+0000 7f6933fc8700 1 -- 172.21.15.88:0/515452023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c19afc0 con 0x7f692c0ff350 2024-02-19T19:35:37.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.589+0000 7f6933fc8700 1 -- 172.21.15.88:0/515452023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692c107dd0 con 0x7f692c0ff350 2024-02-19T19:35:37.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.590+0000 7f6922ffd700 1 -- 172.21.15.88:0/515452023 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6924004040 con 0x7f692c0ff350 2024-02-19T19:35:37.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.590+0000 7f6922ffd700 1 --2- 172.21.15.88:0/515452023 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f691806d8a0 0x7f691806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:35:37.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.591+0000 7f6922ffd700 1 -- 172.21.15.88:0/515452023 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f692408ed10 con 0x7f692c0ff350 2024-02-19T19:35:37.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.591+0000 7f6931563700 1 --2- 172.21.15.88:0/515452023 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f691806d8a0 0x7f691806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:35:37.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.591+0000 7f6931563700 1 --2- 172.21.15.88:0/515452023 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f691806d8a0 0x7f691806fd50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f691c005950 tx=0x7f691c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:35:37.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.595+0000 7f6922ffd700 1 -- 172.21.15.88:0/515452023 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6924058540 con 0x7f692c0ff350 2024-02-19T19:35:37.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.877+0000 7f6933fc8700 1 -- 172.21.15.88:0/515452023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f692c04f2a0 con 0x7f692c0ff350 2024-02-19T19:35:37.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.877+0000 7f6922ffd700 1 -- 172.21.15.88:0/515452023 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f692405bc00 con 0x7f692c0ff350 2024-02-19T19:35:37.879 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:35:37.880 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:35:37.881 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:35:37.881 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:35:37.881 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:35:37.881 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:35:37.881 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:35:37.881 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:35:37.881 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:35:37.881 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:35:37.881 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:35:37.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.879+0000 7f6933fc8700 1 -- 172.21.15.88:0/515452023 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f691806d8a0 msgr2=0x7f691806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:37.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.879+0000 7f6933fc8700 1 --2- 172.21.15.88:0/515452023 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f691806d8a0 0x7f691806fd50 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f691c005950 tx=0x7f691c0058e0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.879+0000 7f6933fc8700 1 -- 172.21.15.88:0/515452023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff350 msgr2=0x7f692c195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:35:37.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.879+0000 7f6933fc8700 1 --2- 172.21.15.88:0/515452023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff350 0x7f692c195ea0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f6924006130 tx=0x7f692400cdb0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.879+0000 7f6933fc8700 1 -- 172.21.15.88:0/515452023 shutdown_connections 2024-02-19T19:35:37.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.879+0000 7f6933fc8700 1 --2- 172.21.15.88:0/515452023 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f691806d8a0 0x7f691806fd50 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.879+0000 7f6933fc8700 1 --2- 172.21.15.88:0/515452023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f692c0ff350 0x7f692c195ea0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.879+0000 7f6933fc8700 1 --2- 172.21.15.88:0/515452023 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f692c0ffc60 0x7f692c1963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:35:37.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.880+0000 7f6933fc8700 1 -- 172.21.15.88:0/515452023 >> 172.21.15.88:0/515452023 conn(0x7f692c0748b0 msgr2=0x7f692c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:35:37.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.880+0000 7f6933fc8700 1 -- 172.21.15.88:0/515452023 shutdown_connections 2024-02-19T19:35:37.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:35:37.880+0000 7f6933fc8700 1 -- 172.21.15.88:0/515452023 wait complete. 2024-02-19T19:35:38.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:37 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3163993795' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:35:38.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:37 smithi088 ceph-mon[88643]: from='client.35754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:38.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:37 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3163993795' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:35:38.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:37 smithi132 ceph-mon[70676]: from='client.35754 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:35:39.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:38 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/515452023' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:35:39.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:38 smithi088 ceph-mon[88643]: pgmap v963: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:39.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:38 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/515452023' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:35:39.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:38 smithi132 ceph-mon[70676]: pgmap v963: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:41.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:40 smithi088 ceph-mon[88643]: pgmap v964: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:41.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:40 smithi132 ceph-mon[70676]: pgmap v964: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:43.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:42 smithi088 ceph-mon[88643]: pgmap v965: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:43.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:42 smithi132 ceph-mon[70676]: pgmap v965: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:44.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:35:44.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:35:44.983 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:44 smithi088 ceph-mon[88643]: pgmap v966: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:45.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:44 smithi132 ceph-mon[70676]: pgmap v966: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:47.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:46 smithi088 ceph-mon[88643]: pgmap v967: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:47.166 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:46 smithi132 ceph-mon[70676]: pgmap v967: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:49.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:48 smithi088 ceph-mon[88643]: pgmap v968: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:49.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:48 smithi132 ceph-mon[70676]: pgmap v968: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:51.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:50 smithi088 ceph-mon[88643]: pgmap v969: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:51.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:50 smithi132 ceph-mon[70676]: pgmap v969: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:53.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:52 smithi088 ceph-mon[88643]: pgmap v970: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:53.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:52 smithi132 ceph-mon[70676]: pgmap v970: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:55.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:54 smithi088 ceph-mon[88643]: pgmap v971: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:55.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:54 smithi132 ceph-mon[70676]: pgmap v971: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:57.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:56 smithi088 ceph-mon[88643]: pgmap v972: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:57.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:56 smithi132 ceph-mon[70676]: pgmap v972: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:58.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:35:58.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:35:59.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:35:59 smithi088 ceph-mon[88643]: pgmap v973: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:35:59.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:35:59 smithi132 ceph-mon[70676]: pgmap v973: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:01.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:00 smithi088 ceph-mon[88643]: pgmap v974: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:01.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:00 smithi132 ceph-mon[70676]: pgmap v974: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:03.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:02 smithi088 ceph-mon[88643]: pgmap v975: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:03.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:02 smithi132 ceph-mon[70676]: pgmap v975: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:05.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:04 smithi088 ceph-mon[88643]: pgmap v976: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:05.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:04 smithi132 ceph-mon[70676]: pgmap v976: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:07.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:06 smithi088 ceph-mon[88643]: pgmap v977: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:07.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:06 smithi132 ceph-mon[70676]: pgmap v977: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:08.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.997+0000 7f928037f700 1 -- 172.21.15.88:0/1314459159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92780ff3c0 msgr2=0x7f92780ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.997+0000 7f928037f700 1 --2- 172.21.15.88:0/1314459159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92780ff3c0 0x7f92780ff790 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9274009910 tx=0x7f9274009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:08.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.997+0000 7f928037f700 1 -- 172.21.15.88:0/1314459159 shutdown_connections 2024-02-19T19:36:08.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.997+0000 7f928037f700 1 --2- 172.21.15.88:0/1314459159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92780ffd60 0x7f9278103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.997+0000 7f928037f700 1 --2- 172.21.15.88:0/1314459159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92780ff3c0 0x7f92780ff790 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.997+0000 7f928037f700 1 -- 172.21.15.88:0/1314459159 >> 172.21.15.88:0/1314459159 conn(0x7f92780fac80 msgr2=0x7f92780fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:08.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.997+0000 7f928037f700 1 -- 172.21.15.88:0/1314459159 shutdown_connections 2024-02-19T19:36:08.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.997+0000 7f928037f700 1 -- 172.21.15.88:0/1314459159 wait complete. 2024-02-19T19:36:08.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f928037f700 1 Processor -- start 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f928037f700 1 -- start start 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f928037f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92780ff3c0 0x7f9278194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f928037f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92780ffd60 0x7f92781952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f928037f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92781959a0 con 0x7f92780ffd60 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f928037f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9278199730 con 0x7f92780ff3c0 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f927d91a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92780ffd60 0x7f92781952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f927d91a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92780ffd60 0x7f92781952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42924/0 (socket says 172.21.15.88:42924) 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f927d91a700 1 -- 172.21.15.88:0/3849051323 learned_addr learned my addr 172.21.15.88:0/3849051323 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f927e11b700 1 --2- 172.21.15.88:0/3849051323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92780ff3c0 0x7f9278194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f927d91a700 1 -- 172.21.15.88:0/3849051323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92780ff3c0 msgr2=0x7f9278194d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f927d91a700 1 --2- 172.21.15.88:0/3849051323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92780ff3c0 0x7f9278194d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.998+0000 7f927d91a700 1 -- 172.21.15.88:0/3849051323 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92740095f0 con 0x7f92780ffd60 2024-02-19T19:36:08.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.999+0000 7f927d91a700 1 --2- 172.21.15.88:0/3849051323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92780ffd60 0x7f92781952c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f926800d900 tx=0x7f926800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:08.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.999+0000 7f926f7fe700 1 -- 172.21.15.88:0/3849051323 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9268009900 con 0x7f92780ffd60 2024-02-19T19:36:08.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.999+0000 7f926f7fe700 1 -- 172.21.15.88:0/3849051323 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9268010460 con 0x7f92780ffd60 2024-02-19T19:36:08.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.999+0000 7f926f7fe700 1 -- 172.21.15.88:0/3849051323 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f926800f5f0 con 0x7f92780ffd60 2024-02-19T19:36:08.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.999+0000 7f928037f700 1 -- 172.21.15.88:0/3849051323 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9278199a10 con 0x7f92780ffd60 2024-02-19T19:36:08.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:07.999+0000 7f928037f700 1 -- 172.21.15.88:0/3849051323 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9278199f60 con 0x7f92780ffd60 2024-02-19T19:36:08.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.000+0000 7f928037f700 1 -- 172.21.15.88:0/3849051323 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92781076d0 con 0x7f92780ffd60 2024-02-19T19:36:08.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.000+0000 7f926f7fe700 1 -- 172.21.15.88:0/3849051323 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f92680105d0 con 0x7f92780ffd60 2024-02-19T19:36:08.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.001+0000 7f926f7fe700 1 --2- 172.21.15.88:0/3849051323 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f926406d8a0 0x7f926406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.001+0000 7f927e11b700 1 --2- 172.21.15.88:0/3849051323 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f926406d8a0 0x7f926406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.001+0000 7f926f7fe700 1 -- 172.21.15.88:0/3849051323 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f926808cfd0 con 0x7f92780ffd60 2024-02-19T19:36:08.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.001+0000 7f927e11b700 1 --2- 172.21.15.88:0/3849051323 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f926406d8a0 0x7f926406fd50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9274006010 tx=0x7f927401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:08.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.006+0000 7f926f7fe700 1 -- 172.21.15.88:0/3849051323 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f92680c0d20 con 0x7f92780ffd60 2024-02-19T19:36:08.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.219+0000 7f928037f700 1 -- 172.21.15.88:0/3849051323 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f927819a1d0 con 0x7f926406d8a0 2024-02-19T19:36:08.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.220+0000 7f926f7fe700 1 -- 172.21.15.88:0/3849051323 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f927819a1d0 con 0x7f926406d8a0 2024-02-19T19:36:08.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.222+0000 7f928037f700 1 -- 172.21.15.88:0/3849051323 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f926406d8a0 msgr2=0x7f926406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.222+0000 7f928037f700 1 --2- 172.21.15.88:0/3849051323 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f926406d8a0 0x7f926406fd50 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9274006010 tx=0x7f927401a040 comp rx=0 tx=0).stop 2024-02-19T19:36:08.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.222+0000 7f928037f700 1 -- 172.21.15.88:0/3849051323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92780ffd60 msgr2=0x7f92781952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.222+0000 7f928037f700 1 --2- 172.21.15.88:0/3849051323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92780ffd60 0x7f92781952c0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f926800d900 tx=0x7f926800dc10 comp rx=0 tx=0).stop 2024-02-19T19:36:08.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.222+0000 7f928037f700 1 -- 172.21.15.88:0/3849051323 shutdown_connections 2024-02-19T19:36:08.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.222+0000 7f928037f700 1 --2- 172.21.15.88:0/3849051323 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f926406d8a0 0x7f926406fd50 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.222+0000 7f928037f700 1 --2- 172.21.15.88:0/3849051323 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f92780ff3c0 0x7f9278194d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.222+0000 7f928037f700 1 --2- 172.21.15.88:0/3849051323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92780ffd60 0x7f92781952c0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.222+0000 7f928037f700 1 -- 172.21.15.88:0/3849051323 >> 172.21.15.88:0/3849051323 conn(0x7f92780fac80 msgr2=0x7f92780fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:08.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.223+0000 7f928037f700 1 -- 172.21.15.88:0/3849051323 shutdown_connections 2024-02-19T19:36:08.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.223+0000 7f928037f700 1 -- 172.21.15.88:0/3849051323 wait complete. 2024-02-19T19:36:08.235 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:36:08.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.326+0000 7f5908861700 1 -- 172.21.15.88:0/1257220299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59041004e0 msgr2=0x7f59041008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.326+0000 7f5908861700 1 --2- 172.21.15.88:0/1257220299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59041004e0 0x7f59041008b0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f58f4009910 tx=0x7f58f4009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:08.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.326+0000 7f5908861700 1 -- 172.21.15.88:0/1257220299 shutdown_connections 2024-02-19T19:36:08.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.326+0000 7f5908861700 1 --2- 172.21.15.88:0/1257220299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5904100e80 0x7f5904104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.326+0000 7f5908861700 1 --2- 172.21.15.88:0/1257220299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59041004e0 0x7f59041008b0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.326+0000 7f5908861700 1 -- 172.21.15.88:0/1257220299 >> 172.21.15.88:0/1257220299 conn(0x7f59040fbda0 msgr2=0x7f59040fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:08.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.327+0000 7f5908861700 1 -- 172.21.15.88:0/1257220299 shutdown_connections 2024-02-19T19:36:08.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.327+0000 7f5908861700 1 -- 172.21.15.88:0/1257220299 wait complete. 2024-02-19T19:36:08.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.327+0000 7f5908861700 1 Processor -- start 2024-02-19T19:36:08.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.327+0000 7f5908861700 1 -- start start 2024-02-19T19:36:08.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f5908861700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59041004e0 0x7f5904073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f5908861700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5904100e80 0x7f5904073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f5908861700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5904073ad0 con 0x7f5904100e80 2024-02-19T19:36:08.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f5908861700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5904073c10 con 0x7f59041004e0 2024-02-19T19:36:08.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f5901d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5904100e80 0x7f5904073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f5901d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5904100e80 0x7f5904073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42936/0 (socket says 172.21.15.88:42936) 2024-02-19T19:36:08.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f5901d9b700 1 -- 172.21.15.88:0/1636277700 learned_addr learned my addr 172.21.15.88:0/1636277700 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:08.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f590259c700 1 --2- 172.21.15.88:0/1636277700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59041004e0 0x7f5904073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f5901d9b700 1 -- 172.21.15.88:0/1636277700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59041004e0 msgr2=0x7f5904073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f5901d9b700 1 --2- 172.21.15.88:0/1636277700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59041004e0 0x7f5904073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f5901d9b700 1 -- 172.21.15.88:0/1636277700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f40095f0 con 0x7f5904100e80 2024-02-19T19:36:08.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f5901d9b700 1 --2- 172.21.15.88:0/1636277700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5904100e80 0x7f5904073590 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f58f800d900 tx=0x7f58f800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:08.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.328+0000 7f58f37fe700 1 -- 172.21.15.88:0/1636277700 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58f8009900 con 0x7f5904100e80 2024-02-19T19:36:08.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.329+0000 7f58f37fe700 1 -- 172.21.15.88:0/1636277700 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58f8010460 con 0x7f5904100e80 2024-02-19T19:36:08.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.329+0000 7f58f37fe700 1 -- 172.21.15.88:0/1636277700 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58f800f5f0 con 0x7f5904100e80 2024-02-19T19:36:08.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.329+0000 7f5908861700 1 -- 172.21.15.88:0/1636277700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59041a20c0 con 0x7f5904100e80 2024-02-19T19:36:08.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.329+0000 7f5908861700 1 -- 172.21.15.88:0/1636277700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59041a24d0 con 0x7f5904100e80 2024-02-19T19:36:08.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.330+0000 7f5908861700 1 -- 172.21.15.88:0/1636277700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59041087f0 con 0x7f5904100e80 2024-02-19T19:36:08.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.330+0000 7f58f37fe700 1 -- 172.21.15.88:0/1636277700 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f58f80105d0 con 0x7f5904100e80 2024-02-19T19:36:08.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.331+0000 7f58f37fe700 1 --2- 172.21.15.88:0/1636277700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f58ec06d8a0 0x7f58ec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.331+0000 7f58f37fe700 1 -- 172.21.15.88:0/1636277700 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f58f808cfd0 con 0x7f5904100e80 2024-02-19T19:36:08.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.331+0000 7f590259c700 1 --2- 172.21.15.88:0/1636277700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f58ec06d8a0 0x7f58ec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.331+0000 7f590259c700 1 --2- 172.21.15.88:0/1636277700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f58ec06d8a0 0x7f58ec06fd50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f58f4006010 tx=0x7f58f401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:08.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.336+0000 7f58f37fe700 1 -- 172.21.15.88:0/1636277700 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f58f8056800 con 0x7f5904100e80 2024-02-19T19:36:08.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.551+0000 7f5908861700 1 -- 172.21.15.88:0/1636277700 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5904074690 con 0x7f58ec06d8a0 2024-02-19T19:36:08.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.552+0000 7f58f37fe700 1 -- 172.21.15.88:0/1636277700 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5904074690 con 0x7f58ec06d8a0 2024-02-19T19:36:08.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.554+0000 7f5908861700 1 -- 172.21.15.88:0/1636277700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f58ec06d8a0 msgr2=0x7f58ec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.555+0000 7f5908861700 1 --2- 172.21.15.88:0/1636277700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f58ec06d8a0 0x7f58ec06fd50 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f58f4006010 tx=0x7f58f401a040 comp rx=0 tx=0).stop 2024-02-19T19:36:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.555+0000 7f5908861700 1 -- 172.21.15.88:0/1636277700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5904100e80 msgr2=0x7f5904073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.555+0000 7f5908861700 1 --2- 172.21.15.88:0/1636277700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5904100e80 0x7f5904073590 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f58f800d900 tx=0x7f58f800dc10 comp rx=0 tx=0).stop 2024-02-19T19:36:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.555+0000 7f5908861700 1 -- 172.21.15.88:0/1636277700 shutdown_connections 2024-02-19T19:36:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.555+0000 7f5908861700 1 --2- 172.21.15.88:0/1636277700 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f58ec06d8a0 0x7f58ec06fd50 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.555+0000 7f5908861700 1 --2- 172.21.15.88:0/1636277700 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f59041004e0 0x7f5904073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.555+0000 7f5908861700 1 --2- 172.21.15.88:0/1636277700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5904100e80 0x7f5904073590 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.555+0000 7f5908861700 1 -- 172.21.15.88:0/1636277700 >> 172.21.15.88:0/1636277700 conn(0x7f59040fbda0 msgr2=0x7f59040fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.555+0000 7f5908861700 1 -- 172.21.15.88:0/1636277700 shutdown_connections 2024-02-19T19:36:08.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.555+0000 7f5908861700 1 -- 172.21.15.88:0/1636277700 wait complete. 2024-02-19T19:36:08.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.660+0000 7f582d381700 1 -- 172.21.15.88:0/709587287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58280ffbd0 msgr2=0x7f5828100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.660+0000 7f582d381700 1 --2- 172.21.15.88:0/709587287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58280ffbd0 0x7f5828100040 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f581c009910 tx=0x7f581c009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:08.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.660+0000 7f582d381700 1 -- 172.21.15.88:0/709587287 shutdown_connections 2024-02-19T19:36:08.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.660+0000 7f582d381700 1 --2- 172.21.15.88:0/709587287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58280ffbd0 0x7f5828100040 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.660+0000 7f582d381700 1 --2- 172.21.15.88:0/709587287 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5828104130 0x7f5828104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.660+0000 7f582d381700 1 -- 172.21.15.88:0/709587287 >> 172.21.15.88:0/709587287 conn(0x7f5828075130 msgr2=0x7f5828075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:08.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.660+0000 7f582d381700 1 -- 172.21.15.88:0/709587287 shutdown_connections 2024-02-19T19:36:08.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.660+0000 7f582d381700 1 -- 172.21.15.88:0/709587287 wait complete. 2024-02-19T19:36:08.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.660+0000 7f582d381700 1 Processor -- start 2024-02-19T19:36:08.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f582d381700 1 -- start start 2024-02-19T19:36:08.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f582d381700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58280ffbd0 0x7f5828195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f582d381700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5828104130 0x7f5828196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f582d381700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58281969b0 con 0x7f58280ffbd0 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f582d381700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828196af0 con 0x7f5828104130 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f5826ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58280ffbd0 0x7f5828195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f5826ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58280ffbd0 0x7f5828195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42958/0 (socket says 172.21.15.88:42958) 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f5826ffd700 1 -- 172.21.15.88:0/2282949702 learned_addr learned my addr 172.21.15.88:0/2282949702 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f58267fc700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5828104130 0x7f5828196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f5826ffd700 1 -- 172.21.15.88:0/2282949702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5828104130 msgr2=0x7f5828196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f5826ffd700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5828104130 0x7f5828196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f5826ffd700 1 -- 172.21.15.88:0/2282949702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0095f0 con 0x7f58280ffbd0 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f58267fc700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5828104130 0x7f5828196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.661+0000 7f5826ffd700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58280ffbd0 0x7f5828195e20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f581800c930 tx=0x7f581800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:08.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.662+0000 7f580ffff700 1 -- 172.21.15.88:0/2282949702 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5818007ad0 con 0x7f58280ffbd0 2024-02-19T19:36:08.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.662+0000 7f580ffff700 1 -- 172.21.15.88:0/2282949702 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5818007c30 con 0x7f58280ffbd0 2024-02-19T19:36:08.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.662+0000 7f582d381700 1 -- 172.21.15.88:0/2282949702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582819a940 con 0x7f58280ffbd0 2024-02-19T19:36:08.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.662+0000 7f580ffff700 1 -- 172.21.15.88:0/2282949702 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5818018610 con 0x7f58280ffbd0 2024-02-19T19:36:08.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.662+0000 7f582d381700 1 -- 172.21.15.88:0/2282949702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582819ae90 con 0x7f58280ffbd0 2024-02-19T19:36:08.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.663+0000 7f582d381700 1 -- 172.21.15.88:0/2282949702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58281086a0 con 0x7f58280ffbd0 2024-02-19T19:36:08.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.663+0000 7f580ffff700 1 -- 172.21.15.88:0/2282949702 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f581800f450 con 0x7f58280ffbd0 2024-02-19T19:36:08.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.663+0000 7f580ffff700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 0x7f581006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.664+0000 7f580ffff700 1 -- 172.21.15.88:0/2282949702 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f581808d640 con 0x7f58280ffbd0 2024-02-19T19:36:08.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.664+0000 7f58267fc700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 0x7f581006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.664+0000 7f58267fc700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 0x7f581006fd50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5828197440 tx=0x7f581c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:08.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.668+0000 7f580ffff700 1 -- 172.21.15.88:0/2282949702 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5818056e70 con 0x7f58280ffbd0 2024-02-19T19:36:08.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.873+0000 7f582d381700 1 -- 172.21.15.88:0/2282949702 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f582819b110 con 0x7f581006d8a0 2024-02-19T19:36:08.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.884+0000 7f580ffff700 1 -- 172.21.15.88:0/2282949702 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f582819b110 con 0x7f581006d8a0 2024-02-19T19:36:08.886 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:36:08.886 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (33m) 9m ago 43m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:36:08.886 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (44m) 9m ago 44m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:36:08.886 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (26m) 9m ago 42m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:36:08.886 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (29m) 9m ago 44m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:36:08.886 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (29m) 9m ago 42m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:36:08.886 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (32m) 9m ago 43m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:36:08.886 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (25m) 9m ago 37m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:36:08.886 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (25m) 9m ago 37m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:36:08.886 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (24m) 9m ago 37m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:36:08.887 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (9m) 9m ago 37m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:36:08.887 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (34m) 9m ago 45m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:36:08.887 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (34m) 9m ago 42m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:36:08.887 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (31m) 9m ago 45m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:36:08.887 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (30m) 9m ago 41m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:36:08.887 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (34m) 9m ago 44m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:36:08.887 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (33m) 9m ago 42m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:36:08.887 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (28m) 9m ago 41m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:36:08.887 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (28m) 9m ago 40m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:36:08.887 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (27m) 9m ago 40m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:36:08.887 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (26m) 9m ago 39m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:36:08.888 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (26m) 9m ago 39m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:36:08.888 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (25m) 9m ago 39m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:36:08.888 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (33m) 9m ago 42m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:36:08.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.886+0000 7f582d381700 1 -- 172.21.15.88:0/2282949702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 msgr2=0x7f581006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.886+0000 7f582d381700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 0x7f581006fd50 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5828197440 tx=0x7f581c01a040 comp rx=0 tx=0).stop 2024-02-19T19:36:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.886+0000 7f582d381700 1 -- 172.21.15.88:0/2282949702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58280ffbd0 msgr2=0x7f5828195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.886+0000 7f582d381700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58280ffbd0 0x7f5828195e20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f581800c930 tx=0x7f581800ccf0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.886+0000 7f582d381700 1 -- 172.21.15.88:0/2282949702 shutdown_connections 2024-02-19T19:36:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.886+0000 7f582d381700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f581006d8a0 0x7f581006fd50 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.886+0000 7f582d381700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58280ffbd0 0x7f5828195e20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.886+0000 7f582d381700 1 --2- 172.21.15.88:0/2282949702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5828104130 0x7f5828196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.886+0000 7f582d381700 1 -- 172.21.15.88:0/2282949702 >> 172.21.15.88:0/2282949702 conn(0x7f5828075130 msgr2=0x7f58280fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.886+0000 7f582d381700 1 -- 172.21.15.88:0/2282949702 shutdown_connections 2024-02-19T19:36:08.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.886+0000 7f582d381700 1 -- 172.21.15.88:0/2282949702 wait complete. 2024-02-19T19:36:08.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.988+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3577056637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25dc0730f0 msgr2=0x7f25dc0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.988+0000 7f25e0f2e700 1 --2- 172.21.15.88:0/3577056637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25dc0730f0 0x7f25dc0734c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009910 tx=0x7f25cc009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:08.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.989+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3577056637 shutdown_connections 2024-02-19T19:36:08.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.989+0000 7f25e0f2e700 1 --2- 172.21.15.88:0/3577056637 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25dc073a00 0x7f25dc10c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.989+0000 7f25e0f2e700 1 --2- 172.21.15.88:0/3577056637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25dc0730f0 0x7f25dc0734c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.989+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3577056637 >> 172.21.15.88:0/3577056637 conn(0x7f25dc0fbda0 msgr2=0x7f25dc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:08.991 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:08 smithi088 ceph-mon[88643]: from='client.35762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:08.991 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:08 smithi088 ceph-mon[88643]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:08.991 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:08 smithi088 ceph-mon[88643]: pgmap v978: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:08.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.989+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3577056637 shutdown_connections 2024-02-19T19:36:08.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.989+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3577056637 wait complete. 2024-02-19T19:36:08.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.989+0000 7f25e0f2e700 1 Processor -- start 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.989+0000 7f25e0f2e700 1 -- start start 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25e0f2e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25dc0730f0 0x7f25dc197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25e0f2e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25dc073a00 0x7f25dc198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25e0f2e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc198be0 con 0x7f25dc0730f0 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25e0f2e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc19c970 con 0x7f25dc073a00 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25da59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25dc0730f0 0x7f25dc197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25da59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25dc0730f0 0x7f25dc197fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42968/0 (socket says 172.21.15.88:42968) 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25da59c700 1 -- 172.21.15.88:0/3099787525 learned_addr learned my addr 172.21.15.88:0/3099787525 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25d9d9b700 1 --2- 172.21.15.88:0/3099787525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25dc073a00 0x7f25dc198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25da59c700 1 -- 172.21.15.88:0/3099787525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25dc073a00 msgr2=0x7f25dc198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25da59c700 1 --2- 172.21.15.88:0/3099787525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25dc073a00 0x7f25dc198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25da59c700 1 -- 172.21.15.88:0/3099787525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0095f0 con 0x7f25dc0730f0 2024-02-19T19:36:08.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25da59c700 1 --2- 172.21.15.88:0/3099787525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25dc0730f0 0x7f25dc197fc0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f25cc006010 tx=0x7f25cc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:08.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.990+0000 7f25cb7fe700 1 -- 172.21.15.88:0/3099787525 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25cc01d070 con 0x7f25dc0730f0 2024-02-19T19:36:08.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.991+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3099787525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25dc19cc50 con 0x7f25dc0730f0 2024-02-19T19:36:08.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.991+0000 7f25cb7fe700 1 -- 172.21.15.88:0/3099787525 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25cc00f460 con 0x7f25dc0730f0 2024-02-19T19:36:08.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.991+0000 7f25cb7fe700 1 -- 172.21.15.88:0/3099787525 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25cc017630 con 0x7f25dc0730f0 2024-02-19T19:36:08.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.991+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3099787525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25dc19d1a0 con 0x7f25dc0730f0 2024-02-19T19:36:08.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.992+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3099787525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25dc109c80 con 0x7f25dc0730f0 2024-02-19T19:36:08.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.992+0000 7f25cb7fe700 1 -- 172.21.15.88:0/3099787525 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f25cc00f5d0 con 0x7f25dc0730f0 2024-02-19T19:36:08.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.993+0000 7f25cb7fe700 1 --2- 172.21.15.88:0/3099787525 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25c406d8a0 0x7f25c406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:08.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.993+0000 7f25d9d9b700 1 --2- 172.21.15.88:0/3099787525 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25c406d8a0 0x7f25c406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:08.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.993+0000 7f25cb7fe700 1 -- 172.21.15.88:0/3099787525 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f25cc08e930 con 0x7f25dc0730f0 2024-02-19T19:36:08.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.993+0000 7f25d9d9b700 1 --2- 172.21.15.88:0/3099787525 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25c406d8a0 0x7f25c406fd50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f25dc1995e0 tx=0x7f25d000a380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:08.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:08.998+0000 7f25cb7fe700 1 -- 172.21.15.88:0/3099787525 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f25cc058160 con 0x7f25dc0730f0 2024-02-19T19:36:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:08 smithi132 ceph-mon[70676]: from='client.35762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:08 smithi132 ceph-mon[70676]: from='client.35766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:08 smithi132 ceph-mon[70676]: pgmap v978: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:09.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.280+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3099787525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f25dc04f2a0 con 0x7f25dc0730f0 2024-02-19T19:36:09.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.281+0000 7f25cb7fe700 1 -- 172.21.15.88:0/3099787525 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f25cc05b820 con 0x7f25dc0730f0 2024-02-19T19:36:09.282 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:36:09.282 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:36:09.283 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:36:09.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.283+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3099787525 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25c406d8a0 msgr2=0x7f25c406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:09.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.283+0000 7f25e0f2e700 1 --2- 172.21.15.88:0/3099787525 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25c406d8a0 0x7f25c406fd50 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f25dc1995e0 tx=0x7f25d000a380 comp rx=0 tx=0).stop 2024-02-19T19:36:09.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.283+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3099787525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25dc0730f0 msgr2=0x7f25dc197fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:09.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.283+0000 7f25e0f2e700 1 --2- 172.21.15.88:0/3099787525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25dc0730f0 0x7f25dc197fc0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f25cc006010 tx=0x7f25cc00bd70 comp rx=0 tx=0).stop 2024-02-19T19:36:09.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.283+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3099787525 shutdown_connections 2024-02-19T19:36:09.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.283+0000 7f25e0f2e700 1 --2- 172.21.15.88:0/3099787525 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f25c406d8a0 0x7f25c406fd50 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.283+0000 7f25e0f2e700 1 --2- 172.21.15.88:0/3099787525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25dc0730f0 0x7f25dc197fc0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.283+0000 7f25e0f2e700 1 --2- 172.21.15.88:0/3099787525 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f25dc073a00 0x7f25dc198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.283+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3099787525 >> 172.21.15.88:0/3099787525 conn(0x7f25dc0fbda0 msgr2=0x7f25dc106dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:09.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.283+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3099787525 shutdown_connections 2024-02-19T19:36:09.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.283+0000 7f25e0f2e700 1 -- 172.21.15.88:0/3099787525 wait complete. 2024-02-19T19:36:09.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.388+0000 7f6dc2655700 1 -- 172.21.15.88:0/3633030357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dbc100d70 msgr2=0x7f6dbc1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:09.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.388+0000 7f6dc2655700 1 --2- 172.21.15.88:0/3633030357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dbc100d70 0x7f6dbc1094d0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6db0009910 tx=0x7f6db0009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:09.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.388+0000 7f6dc2655700 1 -- 172.21.15.88:0/3633030357 shutdown_connections 2024-02-19T19:36:09.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.388+0000 7f6dc2655700 1 --2- 172.21.15.88:0/3633030357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dbc100d70 0x7f6dbc1094d0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.388+0000 7f6dc2655700 1 --2- 172.21.15.88:0/3633030357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dbc100460 0x7f6dbc100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.388+0000 7f6dc2655700 1 -- 172.21.15.88:0/3633030357 >> 172.21.15.88:0/3633030357 conn(0x7f6dbc0748b0 msgr2=0x7f6dbc074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:09.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.388+0000 7f6dc2655700 1 -- 172.21.15.88:0/3633030357 shutdown_connections 2024-02-19T19:36:09.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.388+0000 7f6dc2655700 1 -- 172.21.15.88:0/3633030357 wait complete. 2024-02-19T19:36:09.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dc2655700 1 Processor -- start 2024-02-19T19:36:09.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dc2655700 1 -- start start 2024-02-19T19:36:09.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dc2655700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dbc100460 0x7f6dbc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dc2655700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dbc100d70 0x7f6dbc19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dc2655700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc19b600 con 0x7f6dbc100460 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dc2655700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc194b40 con 0x7f6dbc100d70 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dbbfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dbc100460 0x7f6dbc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dbbfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dbc100460 0x7f6dbc19aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42986/0 (socket says 172.21.15.88:42986) 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dbbfff700 1 -- 172.21.15.88:0/2530821562 learned_addr learned my addr 172.21.15.88:0/2530821562 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dbb7fe700 1 --2- 172.21.15.88:0/2530821562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dbc100d70 0x7f6dbc19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dbbfff700 1 -- 172.21.15.88:0/2530821562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dbc100d70 msgr2=0x7f6dbc19b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dbbfff700 1 --2- 172.21.15.88:0/2530821562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dbc100d70 0x7f6dbc19b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.389+0000 7f6dbbfff700 1 -- 172.21.15.88:0/2530821562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db00095f0 con 0x7f6dbc100460 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.390+0000 7f6dbbfff700 1 --2- 172.21.15.88:0/2530821562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dbc100460 0x7f6dbc19aac0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6dac007b50 tx=0x7f6dac007f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:09.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.390+0000 7f6db97fa700 1 -- 172.21.15.88:0/2530821562 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dac011070 con 0x7f6dbc100460 2024-02-19T19:36:09.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.390+0000 7f6dc2655700 1 -- 172.21.15.88:0/2530821562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc194dc0 con 0x7f6dbc100460 2024-02-19T19:36:09.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.390+0000 7f6db97fa700 1 -- 172.21.15.88:0/2530821562 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6dac00c2e0 con 0x7f6dbc100460 2024-02-19T19:36:09.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.390+0000 7f6db97fa700 1 -- 172.21.15.88:0/2530821562 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6dac00acc0 con 0x7f6dbc100460 2024-02-19T19:36:09.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.390+0000 7f6dc2655700 1 -- 172.21.15.88:0/2530821562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc195310 con 0x7f6dbc100460 2024-02-19T19:36:09.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.391+0000 7f6dc2655700 1 -- 172.21.15.88:0/2530821562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dbc04f2a0 con 0x7f6dbc100460 2024-02-19T19:36:09.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.391+0000 7f6db97fa700 1 -- 172.21.15.88:0/2530821562 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6dac00e710 con 0x7f6dbc100460 2024-02-19T19:36:09.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.391+0000 7f6db97fa700 1 --2- 172.21.15.88:0/2530821562 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6da406d8a0 0x7f6da406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:09.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.392+0000 7f6db97fa700 1 -- 172.21.15.88:0/2530821562 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6dac08e760 con 0x7f6dbc100460 2024-02-19T19:36:09.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.392+0000 7f6dbb7fe700 1 --2- 172.21.15.88:0/2530821562 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6da406d8a0 0x7f6da406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:09.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.392+0000 7f6dbb7fe700 1 --2- 172.21.15.88:0/2530821562 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6da406d8a0 0x7f6da406fd50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6db0005f50 tx=0x7f6db001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:09.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.396+0000 7f6db97fa700 1 -- 172.21.15.88:0/2530821562 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6dac05bac0 con 0x7f6dbc100460 2024-02-19T19:36:09.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.641+0000 7f6dc2655700 1 -- 172.21.15.88:0/2530821562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6dbc195610 con 0x7f6dbc100460 2024-02-19T19:36:09.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.642+0000 7f6db97fa700 1 -- 172.21.15.88:0/2530821562 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f6dac05b650 con 0x7f6dbc100460 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:36:09.644 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:36:09.645 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:36:09.646 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:36:09.647 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:36:09.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.644+0000 7f6dc2655700 1 -- 172.21.15.88:0/2530821562 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6da406d8a0 msgr2=0x7f6da406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:09.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.644+0000 7f6dc2655700 1 --2- 172.21.15.88:0/2530821562 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6da406d8a0 0x7f6da406fd50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6db0005f50 tx=0x7f6db001a040 comp rx=0 tx=0).stop 2024-02-19T19:36:09.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.644+0000 7f6dc2655700 1 -- 172.21.15.88:0/2530821562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dbc100460 msgr2=0x7f6dbc19aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:09.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.644+0000 7f6dc2655700 1 --2- 172.21.15.88:0/2530821562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dbc100460 0x7f6dbc19aac0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6dac007b50 tx=0x7f6dac007f10 comp rx=0 tx=0).stop 2024-02-19T19:36:09.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.644+0000 7f6dc2655700 1 -- 172.21.15.88:0/2530821562 shutdown_connections 2024-02-19T19:36:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.644+0000 7f6dc2655700 1 --2- 172.21.15.88:0/2530821562 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6da406d8a0 0x7f6da406fd50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.644+0000 7f6dc2655700 1 --2- 172.21.15.88:0/2530821562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dbc100460 0x7f6dbc19aac0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.644+0000 7f6dc2655700 1 --2- 172.21.15.88:0/2530821562 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6dbc100d70 0x7f6dbc19b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.644+0000 7f6dc2655700 1 -- 172.21.15.88:0/2530821562 >> 172.21.15.88:0/2530821562 conn(0x7f6dbc0748b0 msgr2=0x7f6dbc103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.644+0000 7f6dc2655700 1 -- 172.21.15.88:0/2530821562 shutdown_connections 2024-02-19T19:36:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.644+0000 7f6dc2655700 1 -- 172.21.15.88:0/2530821562 wait complete. 2024-02-19T19:36:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:36:09.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.747+0000 7fbf4053e700 1 -- 172.21.15.88:0/1564535135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf380ff3c0 msgr2=0x7fbf380ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:09.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.747+0000 7fbf4053e700 1 --2- 172.21.15.88:0/1564535135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf380ff3c0 0x7fbf380ff790 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009910 tx=0x7fbf34009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:09.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.747+0000 7fbf4053e700 1 -- 172.21.15.88:0/1564535135 shutdown_connections 2024-02-19T19:36:09.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.747+0000 7fbf4053e700 1 --2- 172.21.15.88:0/1564535135 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf380ffd60 0x7fbf38103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.747+0000 7fbf4053e700 1 --2- 172.21.15.88:0/1564535135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf380ff3c0 0x7fbf380ff790 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.747+0000 7fbf4053e700 1 -- 172.21.15.88:0/1564535135 >> 172.21.15.88:0/1564535135 conn(0x7fbf380fac80 msgr2=0x7fbf380fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:09.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.747+0000 7fbf4053e700 1 -- 172.21.15.88:0/1564535135 shutdown_connections 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.748+0000 7fbf4053e700 1 -- 172.21.15.88:0/1564535135 wait complete. 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.748+0000 7fbf4053e700 1 Processor -- start 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.748+0000 7fbf4053e700 1 -- start start 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.748+0000 7fbf4053e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf380ff3c0 0x7fbf38194ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.748+0000 7fbf4053e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf380ffd60 0x7fbf38195020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.748+0000 7fbf4053e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf38195560 con 0x7fbf380ffd60 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.748+0000 7fbf4053e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf381956d0 con 0x7fbf380ff3c0 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.748+0000 7fbf3dad9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf380ffd60 0x7fbf38195020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.748+0000 7fbf3dad9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf380ffd60 0x7fbf38195020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43008/0 (socket says 172.21.15.88:43008) 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.748+0000 7fbf3dad9700 1 -- 172.21.15.88:0/1074405066 learned_addr learned my addr 172.21.15.88:0/1074405066 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.749+0000 7fbf3e2da700 1 --2- 172.21.15.88:0/1074405066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf380ff3c0 0x7fbf38194ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.749+0000 7fbf3dad9700 1 -- 172.21.15.88:0/1074405066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf380ff3c0 msgr2=0x7fbf38194ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:09.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.749+0000 7fbf3dad9700 1 --2- 172.21.15.88:0/1074405066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf380ff3c0 0x7fbf38194ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.749+0000 7fbf3dad9700 1 -- 172.21.15.88:0/1074405066 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340095f0 con 0x7fbf380ffd60 2024-02-19T19:36:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.749+0000 7fbf3dad9700 1 --2- 172.21.15.88:0/1074405066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf380ffd60 0x7fbf38195020 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800d900 tx=0x7fbf2800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.749+0000 7fbf2f7fe700 1 -- 172.21.15.88:0/1074405066 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf28009900 con 0x7fbf380ffd60 2024-02-19T19:36:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.749+0000 7fbf2f7fe700 1 -- 172.21.15.88:0/1074405066 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf28010460 con 0x7fbf380ffd60 2024-02-19T19:36:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.749+0000 7fbf4053e700 1 -- 172.21.15.88:0/1074405066 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf381a2150 con 0x7fbf380ffd60 2024-02-19T19:36:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.749+0000 7fbf2f7fe700 1 -- 172.21.15.88:0/1074405066 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf2800f5f0 con 0x7fbf380ffd60 2024-02-19T19:36:09.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.749+0000 7fbf4053e700 1 -- 172.21.15.88:0/1074405066 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf381a24b0 con 0x7fbf380ffd60 2024-02-19T19:36:09.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.750+0000 7fbf4053e700 1 -- 172.21.15.88:0/1074405066 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf381076d0 con 0x7fbf380ffd60 2024-02-19T19:36:09.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.750+0000 7fbf2f7fe700 1 -- 172.21.15.88:0/1074405066 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbf280105d0 con 0x7fbf380ffd60 2024-02-19T19:36:09.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.751+0000 7fbf2f7fe700 1 --2- 172.21.15.88:0/1074405066 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf2406d8a0 0x7fbf2406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:09.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.751+0000 7fbf2f7fe700 1 -- 172.21.15.88:0/1074405066 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbf2808d300 con 0x7fbf380ffd60 2024-02-19T19:36:09.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.751+0000 7fbf3e2da700 1 --2- 172.21.15.88:0/1074405066 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf2406d8a0 0x7fbf2406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:09.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.751+0000 7fbf3e2da700 1 --2- 172.21.15.88:0/1074405066 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf2406d8a0 0x7fbf2406fd50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006010 tx=0x7fbf3401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:09.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.756+0000 7fbf2f7fe700 1 -- 172.21.15.88:0/1074405066 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbf2805a5e0 con 0x7fbf380ffd60 2024-02-19T19:36:09.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.971+0000 7fbf4053e700 1 -- 172.21.15.88:0/1074405066 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf38061190 con 0x7fbf2406d8a0 2024-02-19T19:36:09.973 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:09 smithi088 ceph-mon[88643]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:09.973 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:09 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3099787525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:36:09.973 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:09 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2530821562' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.972+0000 7fbf2f7fe700 1 -- 172.21.15.88:0/1074405066 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbf38061190 con 0x7fbf2406d8a0 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:36:09.974 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:36:09.975 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:36:09.975 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:36:09.975 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:36:09.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.975+0000 7fbf4053e700 1 -- 172.21.15.88:0/1074405066 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf2406d8a0 msgr2=0x7fbf2406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:09.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.975+0000 7fbf4053e700 1 --2- 172.21.15.88:0/1074405066 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf2406d8a0 0x7fbf2406fd50 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006010 tx=0x7fbf3401a040 comp rx=0 tx=0).stop 2024-02-19T19:36:09.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.975+0000 7fbf4053e700 1 -- 172.21.15.88:0/1074405066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf380ffd60 msgr2=0x7fbf38195020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:09.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.975+0000 7fbf4053e700 1 --2- 172.21.15.88:0/1074405066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf380ffd60 0x7fbf38195020 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800d900 tx=0x7fbf2800dcc0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.975+0000 7fbf4053e700 1 -- 172.21.15.88:0/1074405066 shutdown_connections 2024-02-19T19:36:09.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.975+0000 7fbf4053e700 1 --2- 172.21.15.88:0/1074405066 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf2406d8a0 0x7fbf2406fd50 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.975+0000 7fbf4053e700 1 --2- 172.21.15.88:0/1074405066 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf380ff3c0 0x7fbf38194ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.975+0000 7fbf4053e700 1 --2- 172.21.15.88:0/1074405066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf380ffd60 0x7fbf38195020 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:09.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.975+0000 7fbf4053e700 1 -- 172.21.15.88:0/1074405066 >> 172.21.15.88:0/1074405066 conn(0x7fbf380fac80 msgr2=0x7fbf380fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:09.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.975+0000 7fbf4053e700 1 -- 172.21.15.88:0/1074405066 shutdown_connections 2024-02-19T19:36:09.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:09.975+0000 7fbf4053e700 1 -- 172.21.15.88:0/1074405066 wait complete. 2024-02-19T19:36:10.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.078+0000 7f40cda0d700 1 -- 172.21.15.88:0/1311760229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40c80ff3c0 msgr2=0x7f40c80ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:10.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.078+0000 7f40cda0d700 1 --2- 172.21.15.88:0/1311760229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40c80ff3c0 0x7f40c80ff790 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f40b8009910 tx=0x7f40b8009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:10.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.078+0000 7f40cda0d700 1 -- 172.21.15.88:0/1311760229 shutdown_connections 2024-02-19T19:36:10.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.078+0000 7f40cda0d700 1 --2- 172.21.15.88:0/1311760229 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40c80ffd60 0x7f40c8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:10.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.078+0000 7f40cda0d700 1 --2- 172.21.15.88:0/1311760229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40c80ff3c0 0x7f40c80ff790 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:10.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.078+0000 7f40cda0d700 1 -- 172.21.15.88:0/1311760229 >> 172.21.15.88:0/1311760229 conn(0x7f40c80fac80 msgr2=0x7f40c80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:10.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.078+0000 7f40cda0d700 1 -- 172.21.15.88:0/1311760229 shutdown_connections 2024-02-19T19:36:10.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.078+0000 7f40cda0d700 1 -- 172.21.15.88:0/1311760229 wait complete. 2024-02-19T19:36:10.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.079+0000 7f40cda0d700 1 Processor -- start 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.079+0000 7f40cda0d700 1 -- start start 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.079+0000 7f40cda0d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40c80ff3c0 0x7f40c8196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.079+0000 7f40cda0d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40c80ffd60 0x7f40c8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.079+0000 7f40cda0d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8197960 con 0x7f40c80ff3c0 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.079+0000 7f40cda0d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c8197aa0 con 0x7f40c80ffd60 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40c6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40c80ff3c0 0x7f40c8196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40c6ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40c80ff3c0 0x7f40c8196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43022/0 (socket says 172.21.15.88:43022) 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40c6ffd700 1 -- 172.21.15.88:0/1617447654 learned_addr learned my addr 172.21.15.88:0/1617447654 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40c67fc700 1 --2- 172.21.15.88:0/1617447654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40c80ffd60 0x7f40c8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40c6ffd700 1 -- 172.21.15.88:0/1617447654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40c80ffd60 msgr2=0x7f40c8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40c6ffd700 1 --2- 172.21.15.88:0/1617447654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40c80ffd60 0x7f40c8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40c6ffd700 1 -- 172.21.15.88:0/1617447654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b80095f0 con 0x7f40c80ff3c0 2024-02-19T19:36:10.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40c6ffd700 1 --2- 172.21.15.88:0/1617447654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40c80ff3c0 0x7f40c8196e00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f40b8006010 tx=0x7f40b800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:10.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40cca0b700 1 -- 172.21.15.88:0/1617447654 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b801d070 con 0x7f40c80ff3c0 2024-02-19T19:36:10.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40cda0d700 1 -- 172.21.15.88:0/1617447654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c819fe70 con 0x7f40c80ff3c0 2024-02-19T19:36:10.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40cca0b700 1 -- 172.21.15.88:0/1617447654 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40b800f460 con 0x7f40c80ff3c0 2024-02-19T19:36:10.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40cca0b700 1 -- 172.21.15.88:0/1617447654 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b8017610 con 0x7f40c80ff3c0 2024-02-19T19:36:10.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.080+0000 7f40cda0d700 1 -- 172.21.15.88:0/1617447654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c81a0280 con 0x7f40c80ff3c0 2024-02-19T19:36:10.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.082+0000 7f40cda0d700 1 -- 172.21.15.88:0/1617447654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c81076d0 con 0x7f40c80ff3c0 2024-02-19T19:36:10.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.082+0000 7f40cca0b700 1 -- 172.21.15.88:0/1617447654 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f40b800f5d0 con 0x7f40c80ff3c0 2024-02-19T19:36:10.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.082+0000 7f40cca0b700 1 --2- 172.21.15.88:0/1617447654 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40b006d8a0 0x7f40b006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:10.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.082+0000 7f40cca0b700 1 -- 172.21.15.88:0/1617447654 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f40b808ea70 con 0x7f40c80ff3c0 2024-02-19T19:36:10.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.082+0000 7f40c67fc700 1 --2- 172.21.15.88:0/1617447654 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40b006d8a0 0x7f40b006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:10.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.083+0000 7f40c67fc700 1 --2- 172.21.15.88:0/1617447654 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40b006d8a0 0x7f40b006fd50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f40c8069680 tx=0x7f40bc009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:10.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.087+0000 7f40cca0b700 1 -- 172.21.15.88:0/1617447654 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f40b8058220 con 0x7f40c80ff3c0 2024-02-19T19:36:10.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:09 smithi132 ceph-mon[70676]: from='client.35770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:10.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:09 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3099787525' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:36:10.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:09 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2530821562' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:36:10.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.368+0000 7f40cda0d700 1 -- 172.21.15.88:0/1617447654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f40c81070b0 con 0x7f40c80ff3c0 2024-02-19T19:36:10.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.368+0000 7f40cca0b700 1 -- 172.21.15.88:0/1617447654 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f40b805b8e0 con 0x7f40c80ff3c0 2024-02-19T19:36:10.370 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:36:10.370 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:36:10.370 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:36:10.370 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:36:10.370 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:36:10.371 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:36:10.371 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:36:10.371 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:36:10.371 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:36:10.371 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:36:10.371 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:36:10.371 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:36:10.371 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:36:10.371 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:36:10.371 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:36:10.371 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:36:10.372 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:36:10.372 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:36:10.372 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:36:10.372 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:36:10.372 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:36:10.372 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:36:10.372 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:36:10.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.370+0000 7f40cda0d700 1 -- 172.21.15.88:0/1617447654 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40b006d8a0 msgr2=0x7f40b006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:10.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.370+0000 7f40cda0d700 1 --2- 172.21.15.88:0/1617447654 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40b006d8a0 0x7f40b006fd50 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f40c8069680 tx=0x7f40bc009190 comp rx=0 tx=0).stop 2024-02-19T19:36:10.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.370+0000 7f40cda0d700 1 -- 172.21.15.88:0/1617447654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40c80ff3c0 msgr2=0x7f40c8196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:10.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.370+0000 7f40cda0d700 1 --2- 172.21.15.88:0/1617447654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40c80ff3c0 0x7f40c8196e00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f40b8006010 tx=0x7f40b800bd70 comp rx=0 tx=0).stop 2024-02-19T19:36:10.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.370+0000 7f40cda0d700 1 -- 172.21.15.88:0/1617447654 shutdown_connections 2024-02-19T19:36:10.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.370+0000 7f40cda0d700 1 --2- 172.21.15.88:0/1617447654 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40b006d8a0 0x7f40b006fd50 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:10.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.370+0000 7f40cda0d700 1 --2- 172.21.15.88:0/1617447654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40c80ff3c0 0x7f40c8196e00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:10.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.370+0000 7f40cda0d700 1 --2- 172.21.15.88:0/1617447654 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f40c80ffd60 0x7f40c8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:10.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.370+0000 7f40cda0d700 1 -- 172.21.15.88:0/1617447654 >> 172.21.15.88:0/1617447654 conn(0x7f40c80fac80 msgr2=0x7f40c80fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:10.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.370+0000 7f40cda0d700 1 -- 172.21.15.88:0/1617447654 shutdown_connections 2024-02-19T19:36:10.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:10.370+0000 7f40cda0d700 1 -- 172.21.15.88:0/1617447654 wait complete. 2024-02-19T19:36:11.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:10 smithi088 ceph-mon[88643]: from='client.35782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:11.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:10 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1617447654' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:36:11.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:10 smithi088 ceph-mon[88643]: pgmap v979: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:10 smithi132 ceph-mon[70676]: from='client.35782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:10 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1617447654' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:36:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:10 smithi132 ceph-mon[70676]: pgmap v979: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:13.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:12 smithi088 ceph-mon[88643]: pgmap v980: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:13.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:12 smithi132 ceph-mon[70676]: pgmap v980: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:36:14.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:36:14.852 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:14 smithi088 ceph-mon[88643]: pgmap v981: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:15.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:14 smithi132 ceph-mon[70676]: pgmap v981: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:17.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:16 smithi088 ceph-mon[88643]: pgmap v982: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:17.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:16 smithi132 ceph-mon[70676]: pgmap v982: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:19.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:18 smithi088 ceph-mon[88643]: pgmap v983: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:18 smithi132 ceph-mon[70676]: pgmap v983: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:21.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:20 smithi088 ceph-mon[88643]: pgmap v984: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:21.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:20 smithi132 ceph-mon[70676]: pgmap v984: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:23.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:22 smithi088 ceph-mon[88643]: pgmap v985: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:22 smithi132 ceph-mon[70676]: pgmap v985: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:25.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:24 smithi088 ceph-mon[88643]: pgmap v986: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:25.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:24 smithi132 ceph-mon[70676]: pgmap v986: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:27.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:26 smithi088 ceph-mon[88643]: pgmap v987: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:27.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:26 smithi132 ceph-mon[70676]: pgmap v987: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:28.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:36:28.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:36:29.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:29 smithi088 ceph-mon[88643]: pgmap v988: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:29.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:29 smithi132 ceph-mon[70676]: pgmap v988: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:31.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:30 smithi088 ceph-mon[88643]: pgmap v989: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:31.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:30 smithi132 ceph-mon[70676]: pgmap v989: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:32.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:32 smithi132 ceph-mon[70676]: pgmap v990: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:32 smithi088 ceph-mon[88643]: pgmap v990: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:34 smithi088 ceph-mon[88643]: pgmap v991: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:35.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:34 smithi132 ceph-mon[70676]: pgmap v991: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:36 smithi088 ceph-mon[88643]: pgmap v992: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:37.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:36 smithi132 ceph-mon[70676]: pgmap v992: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:39.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:38 smithi088 ceph-mon[88643]: pgmap v993: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:39.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:38 smithi132 ceph-mon[70676]: pgmap v993: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:40.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.487+0000 7f039651c700 1 -- 172.21.15.88:0/80944279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03900687a0 msgr2=0x7f0390068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:40.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.487+0000 7f039651c700 1 --2- 172.21.15.88:0/80944279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03900687a0 0x7f0390068b70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0380009910 tx=0x7f0380009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:40.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.487+0000 7f039651c700 1 -- 172.21.15.88:0/80944279 shutdown_connections 2024-02-19T19:36:40.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.487+0000 7f039651c700 1 --2- 172.21.15.88:0/80944279 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0390069140 0x7f0390104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:40.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.487+0000 7f039651c700 1 --2- 172.21.15.88:0/80944279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03900687a0 0x7f0390068b70 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:40.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.487+0000 7f039651c700 1 -- 172.21.15.88:0/80944279 >> 172.21.15.88:0/80944279 conn(0x7f0390075130 msgr2=0x7f0390075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:40.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.487+0000 7f039651c700 1 -- 172.21.15.88:0/80944279 shutdown_connections 2024-02-19T19:36:40.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.487+0000 7f039651c700 1 -- 172.21.15.88:0/80944279 wait complete. 2024-02-19T19:36:40.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.488+0000 7f039651c700 1 Processor -- start 2024-02-19T19:36:40.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.488+0000 7f039651c700 1 -- start start 2024-02-19T19:36:40.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.488+0000 7f039651c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03900687a0 0x7f03900ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:40.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.488+0000 7f039651c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0390069140 0x7f03901001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:40.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.488+0000 7f039651c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0390100720 con 0x7f0390069140 2024-02-19T19:36:40.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.488+0000 7f039651c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0390100860 con 0x7f03900687a0 2024-02-19T19:36:40.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0390069140 0x7f03901001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:40.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0390069140 0x7f03901001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47352/0 (socket says 172.21.15.88:47352) 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038f7fe700 1 -- 172.21.15.88:0/3110433655 learned_addr learned my addr 172.21.15.88:0/3110433655 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038ffff700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03900687a0 0x7f03900ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038f7fe700 1 -- 172.21.15.88:0/3110433655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03900687a0 msgr2=0x7f03900ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038f7fe700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03900687a0 0x7f03900ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038f7fe700 1 -- 172.21.15.88:0/3110433655 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03800095f0 con 0x7f0390069140 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038ffff700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03900687a0 0x7f03900ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038f7fe700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0390069140 0x7f03901001e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f038400c960 tx=0x7f038400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038d7fa700 1 -- 172.21.15.88:0/3110433655 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0384007a70 con 0x7f0390069140 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f039651c700 1 -- 172.21.15.88:0/3110433655 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03901a20b0 con 0x7f0390069140 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038d7fa700 1 -- 172.21.15.88:0/3110433655 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0384007bd0 con 0x7f0390069140 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f039651c700 1 -- 172.21.15.88:0/3110433655 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03901a2410 con 0x7f0390069140 2024-02-19T19:36:40.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.489+0000 7f038d7fa700 1 -- 172.21.15.88:0/3110433655 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03840186c0 con 0x7f0390069140 2024-02-19T19:36:40.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.490+0000 7f039651c700 1 -- 172.21.15.88:0/3110433655 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0390108770 con 0x7f0390069140 2024-02-19T19:36:40.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.491+0000 7f038d7fa700 1 -- 172.21.15.88:0/3110433655 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f038400faa0 con 0x7f0390069140 2024-02-19T19:36:40.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.491+0000 7f038d7fa700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f037806d8a0 0x7f037806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:40.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.491+0000 7f038d7fa700 1 -- 172.21.15.88:0/3110433655 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f038408d570 con 0x7f0390069140 2024-02-19T19:36:40.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.491+0000 7f038ffff700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f037806d8a0 0x7f037806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:40.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.492+0000 7f038ffff700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f037806d8a0 0x7f037806fd50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f038000b5c0 tx=0x7f038001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:40.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.496+0000 7f038d7fa700 1 -- 172.21.15.88:0/3110433655 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f038405a850 con 0x7f0390069140 2024-02-19T19:36:40.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.714+0000 7f039651c700 1 -- 172.21.15.88:0/3110433655 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0390061190 con 0x7f037806d8a0 2024-02-19T19:36:40.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.715+0000 7f038d7fa700 1 -- 172.21.15.88:0/3110433655 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f0390061190 con 0x7f037806d8a0 2024-02-19T19:36:40.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.717+0000 7f039651c700 1 -- 172.21.15.88:0/3110433655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f037806d8a0 msgr2=0x7f037806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:40.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.717+0000 7f039651c700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f037806d8a0 0x7f037806fd50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f038000b5c0 tx=0x7f038001a040 comp rx=0 tx=0).stop 2024-02-19T19:36:40.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.717+0000 7f039651c700 1 -- 172.21.15.88:0/3110433655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0390069140 msgr2=0x7f03901001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:40.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.717+0000 7f039651c700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0390069140 0x7f03901001e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f038400c960 tx=0x7f038400cd20 comp rx=0 tx=0).stop 2024-02-19T19:36:40.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.717+0000 7f039651c700 1 -- 172.21.15.88:0/3110433655 shutdown_connections 2024-02-19T19:36:40.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.717+0000 7f039651c700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f037806d8a0 0x7f037806fd50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:40.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.717+0000 7f039651c700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f03900687a0 0x7f03900ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:40.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.717+0000 7f039651c700 1 --2- 172.21.15.88:0/3110433655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0390069140 0x7f03901001e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:40.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.717+0000 7f039651c700 1 -- 172.21.15.88:0/3110433655 >> 172.21.15.88:0/3110433655 conn(0x7f0390075130 msgr2=0x7f03900fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:40.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.717+0000 7f039651c700 1 -- 172.21.15.88:0/3110433655 shutdown_connections 2024-02-19T19:36:40.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.717+0000 7f039651c700 1 -- 172.21.15.88:0/3110433655 wait complete. 2024-02-19T19:36:40.730 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:36:40.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.825+0000 7f85f237d700 1 -- 172.21.15.88:0/2058627282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec1004e0 msgr2=0x7f85ec1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:40.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.825+0000 7f85f237d700 1 --2- 172.21.15.88:0/2058627282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec1004e0 0x7f85ec1008b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009910 tx=0x7f85dc009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:40.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.825+0000 7f85f237d700 1 -- 172.21.15.88:0/2058627282 shutdown_connections 2024-02-19T19:36:40.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.825+0000 7f85f237d700 1 --2- 172.21.15.88:0/2058627282 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85ec100e80 0x7f85ec104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:40.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.825+0000 7f85f237d700 1 --2- 172.21.15.88:0/2058627282 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec1004e0 0x7f85ec1008b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:40.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.825+0000 7f85f237d700 1 -- 172.21.15.88:0/2058627282 >> 172.21.15.88:0/2058627282 conn(0x7f85ec0fbda0 msgr2=0x7f85ec0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:40.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.825+0000 7f85f237d700 1 -- 172.21.15.88:0/2058627282 shutdown_connections 2024-02-19T19:36:40.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.825+0000 7f85f237d700 1 -- 172.21.15.88:0/2058627282 wait complete. 2024-02-19T19:36:40.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.826+0000 7f85f237d700 1 Processor -- start 2024-02-19T19:36:40.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.826+0000 7f85f237d700 1 -- start start 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.826+0000 7f85f237d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec1004e0 0x7f85ec073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.826+0000 7f85f237d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85ec100e80 0x7f85ec073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.826+0000 7f85f237d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec073ad0 con 0x7f85ec1004e0 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.826+0000 7f85f237d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec073c10 con 0x7f85ec100e80 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.826+0000 7f85ebfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec1004e0 0x7f85ec073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.826+0000 7f85ebfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec1004e0 0x7f85ec073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47376/0 (socket says 172.21.15.88:47376) 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.826+0000 7f85ebfff700 1 -- 172.21.15.88:0/2397925387 learned_addr learned my addr 172.21.15.88:0/2397925387 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.827+0000 7f85eb7fe700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85ec100e80 0x7f85ec073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.827+0000 7f85ebfff700 1 -- 172.21.15.88:0/2397925387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85ec100e80 msgr2=0x7f85ec073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.827+0000 7f85ebfff700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85ec100e80 0x7f85ec073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.827+0000 7f85ebfff700 1 -- 172.21.15.88:0/2397925387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc0095f0 con 0x7f85ec1004e0 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.827+0000 7f85eb7fe700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85ec100e80 0x7f85ec073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:36:40.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.827+0000 7f85ebfff700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec1004e0 0x7f85ec073050 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f85dc006010 tx=0x7f85dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:40.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.827+0000 7f85e97fa700 1 -- 172.21.15.88:0/2397925387 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85dc01d070 con 0x7f85ec1004e0 2024-02-19T19:36:40.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.827+0000 7f85f237d700 1 -- 172.21.15.88:0/2397925387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ec073e90 con 0x7f85ec1004e0 2024-02-19T19:36:40.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.827+0000 7f85e97fa700 1 -- 172.21.15.88:0/2397925387 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f85dc004fb0 con 0x7f85ec1004e0 2024-02-19T19:36:40.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.827+0000 7f85e97fa700 1 -- 172.21.15.88:0/2397925387 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85dc022ea0 con 0x7f85ec1004e0 2024-02-19T19:36:40.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.827+0000 7f85f237d700 1 -- 172.21.15.88:0/2397925387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec1a2490 con 0x7f85ec1004e0 2024-02-19T19:36:40.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.828+0000 7f85f237d700 1 -- 172.21.15.88:0/2397925387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ec1087f0 con 0x7f85ec1004e0 2024-02-19T19:36:40.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.828+0000 7f85e97fa700 1 -- 172.21.15.88:0/2397925387 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f85dc022470 con 0x7f85ec1004e0 2024-02-19T19:36:40.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.829+0000 7f85e97fa700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f85d406d8a0 0x7f85d406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:40.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.829+0000 7f85e97fa700 1 -- 172.21.15.88:0/2397925387 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f85dc08ecd0 con 0x7f85ec1004e0 2024-02-19T19:36:40.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.829+0000 7f85eb7fe700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f85d406d8a0 0x7f85d406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:40.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.829+0000 7f85eb7fe700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f85d406d8a0 0x7f85d406fd50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f85ec074840 tx=0x7f85e0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:40.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:40.834+0000 7f85e97fa700 1 -- 172.21.15.88:0/2397925387 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f85dc058650 con 0x7f85ec1004e0 2024-02-19T19:36:41.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.050+0000 7f85f237d700 1 -- 172.21.15.88:0/2397925387 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85ec061190 con 0x7f85d406d8a0 2024-02-19T19:36:41.053 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:40 smithi088 ceph-mon[88643]: from='client.35790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:41.053 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:40 smithi088 ceph-mon[88643]: pgmap v994: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:41.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.051+0000 7f85e97fa700 1 -- 172.21.15.88:0/2397925387 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f85ec061190 con 0x7f85d406d8a0 2024-02-19T19:36:41.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.053+0000 7f85f237d700 1 -- 172.21.15.88:0/2397925387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f85d406d8a0 msgr2=0x7f85d406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.053+0000 7f85f237d700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f85d406d8a0 0x7f85d406fd50 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f85ec074840 tx=0x7f85e0008040 comp rx=0 tx=0).stop 2024-02-19T19:36:41.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.053+0000 7f85f237d700 1 -- 172.21.15.88:0/2397925387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec1004e0 msgr2=0x7f85ec073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.053+0000 7f85f237d700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec1004e0 0x7f85ec073050 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f85dc006010 tx=0x7f85dc00be20 comp rx=0 tx=0).stop 2024-02-19T19:36:41.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.053+0000 7f85f237d700 1 -- 172.21.15.88:0/2397925387 shutdown_connections 2024-02-19T19:36:41.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.053+0000 7f85f237d700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f85d406d8a0 0x7f85d406fd50 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.053+0000 7f85f237d700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec1004e0 0x7f85ec073050 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.053+0000 7f85f237d700 1 --2- 172.21.15.88:0/2397925387 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f85ec100e80 0x7f85ec073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.053+0000 7f85f237d700 1 -- 172.21.15.88:0/2397925387 >> 172.21.15.88:0/2397925387 conn(0x7f85ec0fbda0 msgr2=0x7f85ec0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:41.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.053+0000 7f85f237d700 1 -- 172.21.15.88:0/2397925387 shutdown_connections 2024-02-19T19:36:41.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.053+0000 7f85f237d700 1 -- 172.21.15.88:0/2397925387 wait complete. 2024-02-19T19:36:41.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.157+0000 7f5a310bb700 1 -- 172.21.15.88:0/1765446475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2c0ffd60 msgr2=0x7f5a2c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.157+0000 7f5a310bb700 1 --2- 172.21.15.88:0/1765446475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2c0ffd60 0x7f5a2c103d30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009910 tx=0x7f5a20009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:41.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.157+0000 7f5a310bb700 1 -- 172.21.15.88:0/1765446475 shutdown_connections 2024-02-19T19:36:41.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.157+0000 7f5a310bb700 1 --2- 172.21.15.88:0/1765446475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2c0ffd60 0x7f5a2c103d30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.157+0000 7f5a310bb700 1 --2- 172.21.15.88:0/1765446475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a2c0ff3c0 0x7f5a2c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.157+0000 7f5a310bb700 1 -- 172.21.15.88:0/1765446475 >> 172.21.15.88:0/1765446475 conn(0x7f5a2c0fac80 msgr2=0x7f5a2c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:41.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.157+0000 7f5a310bb700 1 -- 172.21.15.88:0/1765446475 shutdown_connections 2024-02-19T19:36:41.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.157+0000 7f5a310bb700 1 -- 172.21.15.88:0/1765446475 wait complete. 2024-02-19T19:36:41.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.158+0000 7f5a310bb700 1 Processor -- start 2024-02-19T19:36:41.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.158+0000 7f5a310bb700 1 -- start start 2024-02-19T19:36:41.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a310bb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2c0ff3c0 0x7f5a2c194b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:41.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a310bb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a2c0ffd60 0x7f5a2c1950b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a310bb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c1955f0 con 0x7f5a2c0ff3c0 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a310bb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c195760 con 0x7f5a2c0ffd60 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a2ad9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2c0ff3c0 0x7f5a2c194b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a2ad9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2c0ff3c0 0x7f5a2c194b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47388/0 (socket says 172.21.15.88:47388) 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a2ad9d700 1 -- 172.21.15.88:0/486149159 learned_addr learned my addr 172.21.15.88:0/486149159 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a2a59c700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a2c0ffd60 0x7f5a2c1950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a2ad9d700 1 -- 172.21.15.88:0/486149159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a2c0ffd60 msgr2=0x7f5a2c1950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a2ad9d700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a2c0ffd60 0x7f5a2c1950b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a2ad9d700 1 -- 172.21.15.88:0/486149159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a200095f0 con 0x7f5a2c0ff3c0 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a2a59c700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a2c0ffd60 0x7f5a2c1950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a2ad9d700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2c0ff3c0 0x7f5a2c194b70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00d8d0 tx=0x7f5a1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a13fff700 1 -- 172.21.15.88:0/486149159 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a1c009940 con 0x7f5a2c0ff3c0 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a13fff700 1 -- 172.21.15.88:0/486149159 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a1c010460 con 0x7f5a2c0ff3c0 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a310bb700 1 -- 172.21.15.88:0/486149159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2c1a21e0 con 0x7f5a2c0ff3c0 2024-02-19T19:36:41.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.159+0000 7f5a13fff700 1 -- 172.21.15.88:0/486149159 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a1c00f5f0 con 0x7f5a2c0ff3c0 2024-02-19T19:36:41.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.160+0000 7f5a310bb700 1 -- 172.21.15.88:0/486149159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c1a2540 con 0x7f5a2c0ff3c0 2024-02-19T19:36:41.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.161+0000 7f5a310bb700 1 -- 172.21.15.88:0/486149159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2c107730 con 0x7f5a2c0ff3c0 2024-02-19T19:36:41.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.161+0000 7f5a13fff700 1 -- 172.21.15.88:0/486149159 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5a1c0105d0 con 0x7f5a2c0ff3c0 2024-02-19T19:36:41.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.161+0000 7f5a13fff700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a1406d8a0 0x7f5a1406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:41.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.162+0000 7f5a13fff700 1 -- 172.21.15.88:0/486149159 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5a1c08d4d0 con 0x7f5a2c0ff3c0 2024-02-19T19:36:41.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.162+0000 7f5a2a59c700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a1406d8a0 0x7f5a1406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:41.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.162+0000 7f5a2a59c700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a1406d8a0 0x7f5a1406fd50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000b5c0 tx=0x7f5a200053a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:41.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.166+0000 7f5a13fff700 1 -- 172.21.15.88:0/486149159 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a1c05a7b0 con 0x7f5a2c0ff3c0 2024-02-19T19:36:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:40 smithi132 ceph-mon[70676]: from='client.35790 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:41.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:40 smithi132 ceph-mon[70676]: pgmap v994: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:41.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.374+0000 7f5a310bb700 1 -- 172.21.15.88:0/486149159 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a2c061190 con 0x7f5a1406d8a0 2024-02-19T19:36:41.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.384+0000 7f5a13fff700 1 -- 172.21.15.88:0/486149159 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5a2c061190 con 0x7f5a1406d8a0 2024-02-19T19:36:41.387 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:36:41.387 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (33m) 10m ago 44m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:36:41.387 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (44m) 10m ago 44m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:36:41.387 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (26m) 10m ago 42m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:36:41.387 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (29m) 10m ago 44m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (29m) 10m ago 42m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (33m) 10m ago 43m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (26m) 10m ago 38m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (25m) 10m ago 37m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (25m) 10m ago 38m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (10m) 10m ago 37m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (35m) 10m ago 45m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (34m) 10m ago 42m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (31m) 10m ago 46m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (31m) 10m ago 42m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (34m) 10m ago 44m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (34m) 10m ago 42m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:36:41.388 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (29m) 10m ago 41m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:36:41.389 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (28m) 10m ago 41m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:36:41.389 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (28m) 10m ago 40m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:36:41.389 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (27m) 10m ago 40m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:36:41.389 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (26m) 10m ago 39m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:36:41.389 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (26m) 10m ago 39m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:36:41.389 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (34m) 10m ago 43m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:36:41.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.387+0000 7f5a310bb700 1 -- 172.21.15.88:0/486149159 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a1406d8a0 msgr2=0x7f5a1406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.387+0000 7f5a310bb700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a1406d8a0 0x7f5a1406fd50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000b5c0 tx=0x7f5a200053a0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.387+0000 7f5a310bb700 1 -- 172.21.15.88:0/486149159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2c0ff3c0 msgr2=0x7f5a2c194b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.387+0000 7f5a310bb700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2c0ff3c0 0x7f5a2c194b70 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00d8d0 tx=0x7f5a1c00dc90 comp rx=0 tx=0).stop 2024-02-19T19:36:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.387+0000 7f5a310bb700 1 -- 172.21.15.88:0/486149159 shutdown_connections 2024-02-19T19:36:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.387+0000 7f5a310bb700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a1406d8a0 0x7f5a1406fd50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.387+0000 7f5a310bb700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2c0ff3c0 0x7f5a2c194b70 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.387+0000 7f5a310bb700 1 --2- 172.21.15.88:0/486149159 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a2c0ffd60 0x7f5a2c1950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.387+0000 7f5a310bb700 1 -- 172.21.15.88:0/486149159 >> 172.21.15.88:0/486149159 conn(0x7f5a2c0fac80 msgr2=0x7f5a2c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.387+0000 7f5a310bb700 1 -- 172.21.15.88:0/486149159 shutdown_connections 2024-02-19T19:36:41.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.387+0000 7f5a310bb700 1 -- 172.21.15.88:0/486149159 wait complete. 2024-02-19T19:36:41.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.490+0000 7f7b538f4700 1 -- 172.21.15.88:0/3202340199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b4c068540 msgr2=0x7f7b4c068910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.490+0000 7f7b538f4700 1 --2- 172.21.15.88:0/3202340199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b4c068540 0x7f7b4c068910 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7b48009940 tx=0x7f7b48009c50 comp rx=0 tx=0).stop 2024-02-19T19:36:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.490+0000 7f7b538f4700 1 -- 172.21.15.88:0/3202340199 shutdown_connections 2024-02-19T19:36:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.490+0000 7f7b538f4700 1 --2- 172.21.15.88:0/3202340199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b4c068ee0 0x7f7b4c110a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.490+0000 7f7b538f4700 1 --2- 172.21.15.88:0/3202340199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b4c068540 0x7f7b4c068910 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.490+0000 7f7b538f4700 1 -- 172.21.15.88:0/3202340199 >> 172.21.15.88:0/3202340199 conn(0x7f7b4c075130 msgr2=0x7f7b4c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.490+0000 7f7b538f4700 1 -- 172.21.15.88:0/3202340199 shutdown_connections 2024-02-19T19:36:41.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.490+0000 7f7b538f4700 1 -- 172.21.15.88:0/3202340199 wait complete. 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b538f4700 1 Processor -- start 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b538f4700 1 -- start start 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b538f4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b4c068540 0x7f7b4c19c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b538f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b4c068ee0 0x7f7b4c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b538f4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c19d070 con 0x7f7b4c068540 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b538f4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4c1a0e00 con 0x7f7b4c068ee0 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b51690700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b4c068540 0x7f7b4c19c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b51690700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b4c068540 0x7f7b4c19c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47400/0 (socket says 172.21.15.88:47400) 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b51690700 1 -- 172.21.15.88:0/665440529 learned_addr learned my addr 172.21.15.88:0/665440529 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b50e8f700 1 --2- 172.21.15.88:0/665440529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b4c068ee0 0x7f7b4c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b51690700 1 -- 172.21.15.88:0/665440529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b4c068ee0 msgr2=0x7f7b4c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b51690700 1 --2- 172.21.15.88:0/665440529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b4c068ee0 0x7f7b4c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b51690700 1 -- 172.21.15.88:0/665440529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b480095f0 con 0x7f7b4c068540 2024-02-19T19:36:41.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.491+0000 7f7b51690700 1 --2- 172.21.15.88:0/665440529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b4c068540 0x7f7b4c19c450 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800bab0 tx=0x7f7b4800bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:41.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.492+0000 7f7b427fc700 1 -- 172.21.15.88:0/665440529 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b4801d070 con 0x7f7b4c068540 2024-02-19T19:36:41.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.492+0000 7f7b538f4700 1 -- 172.21.15.88:0/665440529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b4c1a1080 con 0x7f7b4c068540 2024-02-19T19:36:41.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.492+0000 7f7b427fc700 1 -- 172.21.15.88:0/665440529 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7b48003d10 con 0x7f7b4c068540 2024-02-19T19:36:41.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.492+0000 7f7b427fc700 1 -- 172.21.15.88:0/665440529 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b48021410 con 0x7f7b4c068540 2024-02-19T19:36:41.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.492+0000 7f7b538f4700 1 -- 172.21.15.88:0/665440529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b4c1a15d0 con 0x7f7b4c068540 2024-02-19T19:36:41.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.492+0000 7f7b538f4700 1 -- 172.21.15.88:0/665440529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b4c101410 con 0x7f7b4c068540 2024-02-19T19:36:41.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.493+0000 7f7b427fc700 1 -- 172.21.15.88:0/665440529 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7b4800f460 con 0x7f7b4c068540 2024-02-19T19:36:41.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.493+0000 7f7b427fc700 1 --2- 172.21.15.88:0/665440529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7b3806d8a0 0x7f7b3806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:41.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.494+0000 7f7b427fc700 1 -- 172.21.15.88:0/665440529 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7b4808e6b0 con 0x7f7b4c068540 2024-02-19T19:36:41.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.494+0000 7f7b50e8f700 1 --2- 172.21.15.88:0/665440529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7b3806d8a0 0x7f7b3806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:41.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.494+0000 7f7b50e8f700 1 --2- 172.21.15.88:0/665440529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7b3806d8a0 0x7f7b3806fd50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c19da70 tx=0x7f7b3c009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:41.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.498+0000 7f7b427fc700 1 -- 172.21.15.88:0/665440529 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b48057e60 con 0x7f7b4c068540 2024-02-19T19:36:41.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.777+0000 7f7b538f4700 1 -- 172.21.15.88:0/665440529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7b4c1a1210 con 0x7f7b4c068540 2024-02-19T19:36:41.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.778+0000 7f7b427fc700 1 -- 172.21.15.88:0/665440529 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7b4c1a1210 con 0x7f7b4c068540 2024-02-19T19:36:41.780 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:36:41.780 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:36:41.780 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:36:41.781 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:36:41.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.780+0000 7f7b538f4700 1 -- 172.21.15.88:0/665440529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7b3806d8a0 msgr2=0x7f7b3806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.780+0000 7f7b538f4700 1 --2- 172.21.15.88:0/665440529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7b3806d8a0 0x7f7b3806fd50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7b4c19da70 tx=0x7f7b3c009260 comp rx=0 tx=0).stop 2024-02-19T19:36:41.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.780+0000 7f7b538f4700 1 -- 172.21.15.88:0/665440529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b4c068540 msgr2=0x7f7b4c19c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.780+0000 7f7b538f4700 1 --2- 172.21.15.88:0/665440529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b4c068540 0x7f7b4c19c450 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7b4800bab0 tx=0x7f7b4800bb90 comp rx=0 tx=0).stop 2024-02-19T19:36:41.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.780+0000 7f7b538f4700 1 -- 172.21.15.88:0/665440529 shutdown_connections 2024-02-19T19:36:41.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.780+0000 7f7b538f4700 1 --2- 172.21.15.88:0/665440529 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7b3806d8a0 0x7f7b3806fd50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.780+0000 7f7b538f4700 1 --2- 172.21.15.88:0/665440529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7b4c068540 0x7f7b4c19c450 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.780+0000 7f7b538f4700 1 --2- 172.21.15.88:0/665440529 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7b4c068ee0 0x7f7b4c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.780+0000 7f7b538f4700 1 -- 172.21.15.88:0/665440529 >> 172.21.15.88:0/665440529 conn(0x7f7b4c075130 msgr2=0x7f7b4c10eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:41.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.780+0000 7f7b538f4700 1 -- 172.21.15.88:0/665440529 shutdown_connections 2024-02-19T19:36:41.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.780+0000 7f7b538f4700 1 -- 172.21.15.88:0/665440529 wait complete. 2024-02-19T19:36:41.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.884+0000 7f931ab9d700 1 -- 172.21.15.88:0/2772562683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9314100b30 msgr2=0x7f9314100fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.884+0000 7f931ab9d700 1 --2- 172.21.15.88:0/2772562683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9314100b30 0x7f9314100fa0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9310009910 tx=0x7f9310009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:41.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.884+0000 7f931ab9d700 1 -- 172.21.15.88:0/2772562683 shutdown_connections 2024-02-19T19:36:41.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.884+0000 7f931ab9d700 1 --2- 172.21.15.88:0/2772562683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9314100b30 0x7f9314100fa0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.884+0000 7f931ab9d700 1 --2- 172.21.15.88:0/2772562683 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93140ff2d0 0x7f93140ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.884+0000 7f931ab9d700 1 -- 172.21.15.88:0/2772562683 >> 172.21.15.88:0/2772562683 conn(0x7f93140faad0 msgr2=0x7f93140fcee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:41.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.884+0000 7f931ab9d700 1 -- 172.21.15.88:0/2772562683 shutdown_connections 2024-02-19T19:36:41.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.884+0000 7f931ab9d700 1 -- 172.21.15.88:0/2772562683 wait complete. 2024-02-19T19:36:41.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.885+0000 7f931ab9d700 1 Processor -- start 2024-02-19T19:36:41.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.885+0000 7f931ab9d700 1 -- start start 2024-02-19T19:36:41.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.885+0000 7f931ab9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93140ff2d0 0x7f931419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:41.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.885+0000 7f931ab9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9314100b30 0x7f931419aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:41.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.885+0000 7f931ab9d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931419b540 con 0x7f93140ff2d0 2024-02-19T19:36:41.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.885+0000 7f931ab9d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93141949f0 con 0x7f9314100b30 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.885+0000 7f9319b9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93140ff2d0 0x7f931419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.885+0000 7f9319b9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93140ff2d0 0x7f931419a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47412/0 (socket says 172.21.15.88:47412) 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.885+0000 7f9319b9b700 1 -- 172.21.15.88:0/701962815 learned_addr learned my addr 172.21.15.88:0/701962815 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.886+0000 7f931939a700 1 --2- 172.21.15.88:0/701962815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9314100b30 0x7f931419aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.886+0000 7f9319b9b700 1 -- 172.21.15.88:0/701962815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9314100b30 msgr2=0x7f931419aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.886+0000 7f9319b9b700 1 --2- 172.21.15.88:0/701962815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9314100b30 0x7f931419aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.886+0000 7f9319b9b700 1 -- 172.21.15.88:0/701962815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93100095f0 con 0x7f93140ff2d0 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.886+0000 7f9319b9b700 1 --2- 172.21.15.88:0/701962815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93140ff2d0 0x7f931419a970 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f930400d900 tx=0x7f930400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.886+0000 7f930affd700 1 -- 172.21.15.88:0/701962815 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93040041f0 con 0x7f93140ff2d0 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.886+0000 7f931ab9d700 1 -- 172.21.15.88:0/701962815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9314194cd0 con 0x7f93140ff2d0 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.886+0000 7f930affd700 1 -- 172.21.15.88:0/701962815 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9304004cf0 con 0x7f93140ff2d0 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.886+0000 7f930affd700 1 -- 172.21.15.88:0/701962815 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f930400b750 con 0x7f93140ff2d0 2024-02-19T19:36:41.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.886+0000 7f931ab9d700 1 -- 172.21.15.88:0/701962815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9314195220 con 0x7f93140ff2d0 2024-02-19T19:36:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.887+0000 7f931ab9d700 1 -- 172.21.15.88:0/701962815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9314062380 con 0x7f93140ff2d0 2024-02-19T19:36:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.887+0000 7f930affd700 1 -- 172.21.15.88:0/701962815 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9304004350 con 0x7f93140ff2d0 2024-02-19T19:36:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.888+0000 7f930affd700 1 --2- 172.21.15.88:0/701962815 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f930006d8a0 0x7f930006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.888+0000 7f930affd700 1 -- 172.21.15.88:0/701962815 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f930408d6f0 con 0x7f93140ff2d0 2024-02-19T19:36:41.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.888+0000 7f931939a700 1 --2- 172.21.15.88:0/701962815 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f930006d8a0 0x7f930006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:41.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.888+0000 7f931939a700 1 --2- 172.21.15.88:0/701962815 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f930006d8a0 0x7f930006fd50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9310005b40 tx=0x7f931001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:41.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:41.893+0000 7f930affd700 1 -- 172.21.15.88:0/701962815 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f930405a9d0 con 0x7f93140ff2d0 2024-02-19T19:36:42.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:41 smithi088 ceph-mon[88643]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:42.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:41 smithi088 ceph-mon[88643]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:42.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:41 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/665440529' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:36:42.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.132+0000 7f931ab9d700 1 -- 172.21.15.88:0/701962815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9314195490 con 0x7f93140ff2d0 2024-02-19T19:36:42.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.132+0000 7f930affd700 1 -- 172.21.15.88:0/701962815 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f930405a560 con 0x7f93140ff2d0 2024-02-19T19:36:42.135 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:36:42.135 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:36:42.135 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:36:42.135 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:36:42.135 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:36:42.136 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:36:42.137 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:36:42.138 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:36:42.138 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:36:42.138 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:36:42.138 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:36:42.138 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:36:42.138 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:36:42.138 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.136+0000 7f931ab9d700 1 -- 172.21.15.88:0/701962815 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f930006d8a0 msgr2=0x7f930006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.136+0000 7f931ab9d700 1 --2- 172.21.15.88:0/701962815 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f930006d8a0 0x7f930006fd50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f9310005b40 tx=0x7f931001a040 comp rx=0 tx=0).stop 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.136+0000 7f931ab9d700 1 -- 172.21.15.88:0/701962815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93140ff2d0 msgr2=0x7f931419a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.136+0000 7f931ab9d700 1 --2- 172.21.15.88:0/701962815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93140ff2d0 0x7f931419a970 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f930400d900 tx=0x7f930400dcc0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.136+0000 7f931ab9d700 1 -- 172.21.15.88:0/701962815 shutdown_connections 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.136+0000 7f931ab9d700 1 --2- 172.21.15.88:0/701962815 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f930006d8a0 0x7f930006fd50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.136+0000 7f931ab9d700 1 --2- 172.21.15.88:0/701962815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93140ff2d0 0x7f931419a970 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.136+0000 7f931ab9d700 1 --2- 172.21.15.88:0/701962815 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9314100b30 0x7f931419aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.136+0000 7f931ab9d700 1 -- 172.21.15.88:0/701962815 >> 172.21.15.88:0/701962815 conn(0x7f93140faad0 msgr2=0x7f93140fb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.136+0000 7f931ab9d700 1 -- 172.21.15.88:0/701962815 shutdown_connections 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.136+0000 7f931ab9d700 1 -- 172.21.15.88:0/701962815 wait complete. 2024-02-19T19:36:42.139 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:36:42.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:41 smithi132 ceph-mon[70676]: from='client.35794 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:42.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:41 smithi132 ceph-mon[70676]: from='client.35798 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:42.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:41 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/665440529' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:36:42.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.239+0000 7f7427034700 1 -- 172.21.15.88:0/2519583090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74201004e0 msgr2=0x7f74201008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:42.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.239+0000 7f7427034700 1 --2- 172.21.15.88:0/2519583090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74201004e0 0x7f74201008b0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7414009910 tx=0x7f7414009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.239+0000 7f7427034700 1 -- 172.21.15.88:0/2519583090 shutdown_connections 2024-02-19T19:36:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.239+0000 7f7427034700 1 --2- 172.21.15.88:0/2519583090 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7420100e80 0x7f7420104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.239+0000 7f7427034700 1 --2- 172.21.15.88:0/2519583090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74201004e0 0x7f74201008b0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.239+0000 7f7427034700 1 -- 172.21.15.88:0/2519583090 >> 172.21.15.88:0/2519583090 conn(0x7f74200fbda0 msgr2=0x7f74200fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.239+0000 7f7427034700 1 -- 172.21.15.88:0/2519583090 shutdown_connections 2024-02-19T19:36:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.239+0000 7f7427034700 1 -- 172.21.15.88:0/2519583090 wait complete. 2024-02-19T19:36:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7427034700 1 Processor -- start 2024-02-19T19:36:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7427034700 1 -- start start 2024-02-19T19:36:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7427034700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74201004e0 0x7f7420195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:42.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7427034700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7420100e80 0x7f7420196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7427034700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7420196a80 con 0x7f74201004e0 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7427034700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7420196bc0 con 0x7f7420100e80 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7424dd0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74201004e0 0x7f7420195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7424dd0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74201004e0 0x7f7420195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47426/0 (socket says 172.21.15.88:47426) 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7424dd0700 1 -- 172.21.15.88:0/11589449 learned_addr learned my addr 172.21.15.88:0/11589449 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f741ffff700 1 --2- 172.21.15.88:0/11589449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7420100e80 0x7f7420196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7424dd0700 1 -- 172.21.15.88:0/11589449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7420100e80 msgr2=0x7f7420196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7424dd0700 1 --2- 172.21.15.88:0/11589449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7420100e80 0x7f7420196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.240+0000 7f7424dd0700 1 -- 172.21.15.88:0/11589449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74140095f0 con 0x7f74201004e0 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.241+0000 7f7424dd0700 1 --2- 172.21.15.88:0/11589449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74201004e0 0x7f7420195ef0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7414006010 tx=0x7f741400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.241+0000 7f741dffb700 1 -- 172.21.15.88:0/11589449 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f741401d070 con 0x7f74201004e0 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.241+0000 7f741dffb700 1 -- 172.21.15.88:0/11589449 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7414004fb0 con 0x7f74201004e0 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.241+0000 7f7427034700 1 -- 172.21.15.88:0/11589449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742019a9b0 con 0x7f74201004e0 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.241+0000 7f741dffb700 1 -- 172.21.15.88:0/11589449 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7414022ea0 con 0x7f74201004e0 2024-02-19T19:36:42.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.241+0000 7f7427034700 1 -- 172.21.15.88:0/11589449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742019aea0 con 0x7f74201004e0 2024-02-19T19:36:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.242+0000 7f7427034700 1 -- 172.21.15.88:0/11589449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74201087f0 con 0x7f74201004e0 2024-02-19T19:36:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.242+0000 7f741dffb700 1 -- 172.21.15.88:0/11589449 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7414022470 con 0x7f74201004e0 2024-02-19T19:36:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.243+0000 7f741dffb700 1 --2- 172.21.15.88:0/11589449 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f741006d8a0 0x7f741006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.243+0000 7f741ffff700 1 --2- 172.21.15.88:0/11589449 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f741006d8a0 0x7f741006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.243+0000 7f741dffb700 1 -- 172.21.15.88:0/11589449 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f741408ea80 con 0x7f74201004e0 2024-02-19T19:36:42.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.243+0000 7f741ffff700 1 --2- 172.21.15.88:0/11589449 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f741006d8a0 0x7f741006fd50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f740c005ea0 tx=0x7f740c005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:42.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.248+0000 7f741dffb700 1 -- 172.21.15.88:0/11589449 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f74140583e0 con 0x7f74201004e0 2024-02-19T19:36:42.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.460+0000 7f7427034700 1 -- 172.21.15.88:0/11589449 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f742019b1d0 con 0x7f741006d8a0 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.461+0000 7f741dffb700 1 -- 172.21.15.88:0/11589449 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f742019b1d0 con 0x7f741006d8a0 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:36:42.464 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:36:42.465 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:36:42.465 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:36:42.465 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:36:42.465 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:36:42.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.463+0000 7f7427034700 1 -- 172.21.15.88:0/11589449 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f741006d8a0 msgr2=0x7f741006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:42.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.463+0000 7f7427034700 1 --2- 172.21.15.88:0/11589449 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f741006d8a0 0x7f741006fd50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f740c005ea0 tx=0x7f740c005e30 comp rx=0 tx=0).stop 2024-02-19T19:36:42.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.463+0000 7f7427034700 1 -- 172.21.15.88:0/11589449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74201004e0 msgr2=0x7f7420195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:42.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.463+0000 7f7427034700 1 --2- 172.21.15.88:0/11589449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74201004e0 0x7f7420195ef0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7414006010 tx=0x7f741400be20 comp rx=0 tx=0).stop 2024-02-19T19:36:42.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.463+0000 7f7427034700 1 -- 172.21.15.88:0/11589449 shutdown_connections 2024-02-19T19:36:42.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.463+0000 7f7427034700 1 --2- 172.21.15.88:0/11589449 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f741006d8a0 0x7f741006fd50 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.463+0000 7f7427034700 1 --2- 172.21.15.88:0/11589449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74201004e0 0x7f7420195ef0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.463+0000 7f7427034700 1 --2- 172.21.15.88:0/11589449 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7420100e80 0x7f7420196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.463+0000 7f7427034700 1 -- 172.21.15.88:0/11589449 >> 172.21.15.88:0/11589449 conn(0x7f74200fbda0 msgr2=0x7f74200fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:42.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.464+0000 7f7427034700 1 -- 172.21.15.88:0/11589449 shutdown_connections 2024-02-19T19:36:42.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.464+0000 7f7427034700 1 -- 172.21.15.88:0/11589449 wait complete. 2024-02-19T19:36:42.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.568+0000 7f8b915f4700 1 -- 172.21.15.88:0/1121819232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b8c100e80 msgr2=0x7f8b8c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:42.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.568+0000 7f8b915f4700 1 --2- 172.21.15.88:0/1121819232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b8c100e80 0x7f8b8c104e50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009910 tx=0x7f8b80009c20 comp rx=0 tx=0).stop 2024-02-19T19:36:42.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.568+0000 7f8b915f4700 1 -- 172.21.15.88:0/1121819232 shutdown_connections 2024-02-19T19:36:42.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.568+0000 7f8b915f4700 1 --2- 172.21.15.88:0/1121819232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b8c100e80 0x7f8b8c104e50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.568+0000 7f8b915f4700 1 --2- 172.21.15.88:0/1121819232 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b8c1004e0 0x7f8b8c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.568+0000 7f8b915f4700 1 -- 172.21.15.88:0/1121819232 >> 172.21.15.88:0/1121819232 conn(0x7f8b8c0fbda0 msgr2=0x7f8b8c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:42.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.568+0000 7f8b915f4700 1 -- 172.21.15.88:0/1121819232 shutdown_connections 2024-02-19T19:36:42.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.568+0000 7f8b915f4700 1 -- 172.21.15.88:0/1121819232 wait complete. 2024-02-19T19:36:42.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.568+0000 7f8b915f4700 1 Processor -- start 2024-02-19T19:36:42.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b915f4700 1 -- start start 2024-02-19T19:36:42.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b915f4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b8c1004e0 0x7f8b8c195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:42.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b915f4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b8c100e80 0x7f8b8c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:42.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b915f4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c196b30 con 0x7f8b8c100e80 2024-02-19T19:36:42.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b915f4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c19a8c0 con 0x7f8b8c1004e0 2024-02-19T19:36:42.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b8c100e80 0x7f8b8c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b8a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b8c100e80 0x7f8b8c196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:47446/0 (socket says 172.21.15.88:47446) 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b8a7fc700 1 -- 172.21.15.88:0/1945330061 learned_addr learned my addr 172.21.15.88:0/1945330061 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b8affd700 1 --2- 172.21.15.88:0/1945330061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b8c1004e0 0x7f8b8c195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b8a7fc700 1 -- 172.21.15.88:0/1945330061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b8c1004e0 msgr2=0x7f8b8c195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b8a7fc700 1 --2- 172.21.15.88:0/1945330061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b8c1004e0 0x7f8b8c195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b8a7fc700 1 -- 172.21.15.88:0/1945330061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b800095f0 con 0x7f8b8c100e80 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.569+0000 7f8b8a7fc700 1 --2- 172.21.15.88:0/1945330061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b8c100e80 0x7f8b8c196450 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8b800059d0 tx=0x7f8b8000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.570+0000 7f8b73fff700 1 -- 172.21.15.88:0/1945330061 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b8001d070 con 0x7f8b8c100e80 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.570+0000 7f8b73fff700 1 -- 172.21.15.88:0/1945330061 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b8000b610 con 0x7f8b8c100e80 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.570+0000 7f8b915f4700 1 -- 172.21.15.88:0/1945330061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c19ab40 con 0x7f8b8c100e80 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.570+0000 7f8b73fff700 1 -- 172.21.15.88:0/1945330061 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b80022470 con 0x7f8b8c100e80 2024-02-19T19:36:42.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.570+0000 7f8b915f4700 1 -- 172.21.15.88:0/1945330061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c19b030 con 0x7f8b8c100e80 2024-02-19T19:36:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.571+0000 7f8b915f4700 1 -- 172.21.15.88:0/1945330061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c108850 con 0x7f8b8c100e80 2024-02-19T19:36:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.571+0000 7f8b73fff700 1 -- 172.21.15.88:0/1945330061 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8b8000fc70 con 0x7f8b8c100e80 2024-02-19T19:36:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.571+0000 7f8b73fff700 1 --2- 172.21.15.88:0/1945330061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8b7406d8a0 0x7f8b7406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:36:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.572+0000 7f8b8affd700 1 --2- 172.21.15.88:0/1945330061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8b7406d8a0 0x7f8b7406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:36:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.572+0000 7f8b73fff700 1 -- 172.21.15.88:0/1945330061 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8b8008e990 con 0x7f8b8c100e80 2024-02-19T19:36:42.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.572+0000 7f8b8affd700 1 --2- 172.21.15.88:0/1945330061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8b7406d8a0 0x7f8b7406fd50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009bf0 tx=0x7f8b7c009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:36:42.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.576+0000 7f8b73fff700 1 -- 172.21.15.88:0/1945330061 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8b80058140 con 0x7f8b8c100e80 2024-02-19T19:36:42.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.855+0000 7f8b915f4700 1 -- 172.21.15.88:0/1945330061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8b8c108230 con 0x7f8b8c100e80 2024-02-19T19:36:42.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.855+0000 7f8b73fff700 1 -- 172.21.15.88:0/1945330061 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f8b8005b800 con 0x7f8b8c100e80 2024-02-19T19:36:42.858 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:36:42.858 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:36:42.858 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:36:42.858 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:36:42.858 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:36:42.859 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:36:42.860 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:36:42.860 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:36:42.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.858+0000 7f8b915f4700 1 -- 172.21.15.88:0/1945330061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8b7406d8a0 msgr2=0x7f8b7406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:42.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.858+0000 7f8b915f4700 1 --2- 172.21.15.88:0/1945330061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8b7406d8a0 0x7f8b7406fd50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c009bf0 tx=0x7f8b7c009260 comp rx=0 tx=0).stop 2024-02-19T19:36:42.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.858+0000 7f8b915f4700 1 -- 172.21.15.88:0/1945330061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b8c100e80 msgr2=0x7f8b8c196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:36:42.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.858+0000 7f8b915f4700 1 --2- 172.21.15.88:0/1945330061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b8c100e80 0x7f8b8c196450 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8b800059d0 tx=0x7f8b8000f740 comp rx=0 tx=0).stop 2024-02-19T19:36:42.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.858+0000 7f8b915f4700 1 -- 172.21.15.88:0/1945330061 shutdown_connections 2024-02-19T19:36:42.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.858+0000 7f8b915f4700 1 --2- 172.21.15.88:0/1945330061 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8b7406d8a0 0x7f8b7406fd50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.858+0000 7f8b915f4700 1 --2- 172.21.15.88:0/1945330061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8b8c1004e0 0x7f8b8c195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.858+0000 7f8b915f4700 1 --2- 172.21.15.88:0/1945330061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b8c100e80 0x7f8b8c196450 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:36:42.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.858+0000 7f8b915f4700 1 -- 172.21.15.88:0/1945330061 >> 172.21.15.88:0/1945330061 conn(0x7f8b8c0fbda0 msgr2=0x7f8b8c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:36:42.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.858+0000 7f8b915f4700 1 -- 172.21.15.88:0/1945330061 shutdown_connections 2024-02-19T19:36:42.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:36:42.858+0000 7f8b915f4700 1 -- 172.21.15.88:0/1945330061 wait complete. 2024-02-19T19:36:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:42 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/701962815' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:36:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:42 smithi088 ceph-mon[88643]: from='client.35810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:43.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:42 smithi088 ceph-mon[88643]: pgmap v995: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:43.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:42 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/701962815' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:36:43.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:42 smithi132 ceph-mon[70676]: from='client.35810 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:36:43.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:42 smithi132 ceph-mon[70676]: pgmap v995: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:43 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1945330061' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:36:44.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:36:44.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:43 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1945330061' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:36:44.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:36:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:44 smithi088 ceph-mon[88643]: pgmap v996: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:45.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:44 smithi132 ceph-mon[70676]: pgmap v996: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:47.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:46 smithi088 ceph-mon[88643]: pgmap v997: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:46 smithi132 ceph-mon[70676]: pgmap v997: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:49.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:48 smithi088 ceph-mon[88643]: pgmap v998: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:49.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:48 smithi132 ceph-mon[70676]: pgmap v998: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:51.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:50 smithi088 ceph-mon[88643]: pgmap v999: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:50 smithi132 ceph-mon[70676]: pgmap v999: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:53.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:52 smithi088 ceph-mon[88643]: pgmap v1000: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:53.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:52 smithi132 ceph-mon[70676]: pgmap v1000: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:55.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:54 smithi088 ceph-mon[88643]: pgmap v1001: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:55.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:54 smithi132 ceph-mon[70676]: pgmap v1001: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:56 smithi088 ceph-mon[88643]: pgmap v1002: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:57.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:56 smithi132 ceph-mon[70676]: pgmap v1002: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:58.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:36:58.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:36:59.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:36:59 smithi088 ceph-mon[88643]: pgmap v1003: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:36:59.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:36:59 smithi132 ceph-mon[70676]: pgmap v1003: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:00 smithi088 ceph-mon[88643]: pgmap v1004: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:00 smithi132 ceph-mon[70676]: pgmap v1004: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:02 smithi088 ceph-mon[88643]: pgmap v1005: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:03.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:02 smithi132 ceph-mon[70676]: pgmap v1005: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:05.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:04 smithi088 ceph-mon[88643]: pgmap v1006: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:05.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:04 smithi132 ceph-mon[70676]: pgmap v1006: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:06 smithi088 ceph-mon[88643]: pgmap v1007: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:07.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:06 smithi132 ceph-mon[70676]: pgmap v1007: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:09.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:08 smithi088 ceph-mon[88643]: pgmap v1008: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:08 smithi132 ceph-mon[70676]: pgmap v1008: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:11.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:10 smithi088 ceph-mon[88643]: pgmap v1009: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:10 smithi132 ceph-mon[70676]: pgmap v1009: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:12.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.976+0000 7f841ad6a700 1 -- 172.21.15.88:0/994291978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8414100e80 msgr2=0x7f8414104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.976+0000 7f841ad6a700 1 --2- 172.21.15.88:0/994291978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8414100e80 0x7f8414104e50 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f8400009910 tx=0x7f8400009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.976+0000 7f841ad6a700 1 -- 172.21.15.88:0/994291978 shutdown_connections 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.976+0000 7f841ad6a700 1 --2- 172.21.15.88:0/994291978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8414100e80 0x7f8414104e50 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.976+0000 7f841ad6a700 1 --2- 172.21.15.88:0/994291978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f84141004e0 0x7f84141008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.976+0000 7f841ad6a700 1 -- 172.21.15.88:0/994291978 >> 172.21.15.88:0/994291978 conn(0x7f84140fbda0 msgr2=0x7f84140fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.976+0000 7f841ad6a700 1 -- 172.21.15.88:0/994291978 shutdown_connections 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.976+0000 7f841ad6a700 1 -- 172.21.15.88:0/994291978 wait complete. 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f841ad6a700 1 Processor -- start 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f841ad6a700 1 -- start start 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f841ad6a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84141004e0 0x7f8414073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f841ad6a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8414100e80 0x7f8414073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:12.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f841ad6a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8414073ad0 con 0x7f84141004e0 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f841ad6a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8414073c10 con 0x7f8414100e80 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f8418b06700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84141004e0 0x7f8414073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f8418b06700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84141004e0 0x7f8414073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44602/0 (socket says 172.21.15.88:44602) 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f8418b06700 1 -- 172.21.15.88:0/1796528272 learned_addr learned my addr 172.21.15.88:0/1796528272 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f8418b06700 1 -- 172.21.15.88:0/1796528272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8414100e80 msgr2=0x7f8414073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f8418b06700 1 --2- 172.21.15.88:0/1796528272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8414100e80 0x7f8414073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.977+0000 7f8418b06700 1 -- 172.21.15.88:0/1796528272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84000095f0 con 0x7f84141004e0 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.978+0000 7f8418b06700 1 --2- 172.21.15.88:0/1796528272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84141004e0 0x7f8414073050 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f840800d900 tx=0x7f840800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.978+0000 7f8411ffb700 1 -- 172.21.15.88:0/1796528272 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8408004a00 con 0x7f84141004e0 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.978+0000 7f8411ffb700 1 -- 172.21.15.88:0/1796528272 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84080054e0 con 0x7f84141004e0 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.978+0000 7f841ad6a700 1 -- 172.21.15.88:0/1796528272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84141a2070 con 0x7f84141004e0 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.978+0000 7f8411ffb700 1 -- 172.21.15.88:0/1796528272 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8408009d60 con 0x7f84141004e0 2024-02-19T19:37:12.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.978+0000 7f841ad6a700 1 -- 172.21.15.88:0/1796528272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84141a2480 con 0x7f84141004e0 2024-02-19T19:37:12.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.979+0000 7f841ad6a700 1 -- 172.21.15.88:0/1796528272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8414108760 con 0x7f84141004e0 2024-02-19T19:37:12.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.979+0000 7f8411ffb700 1 -- 172.21.15.88:0/1796528272 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8408005020 con 0x7f84141004e0 2024-02-19T19:37:12.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.980+0000 7f8411ffb700 1 --2- 172.21.15.88:0/1796528272 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f840406d8a0 0x7f840406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:12.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.980+0000 7f8411ffb700 1 -- 172.21.15.88:0/1796528272 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f840808d690 con 0x7f84141004e0 2024-02-19T19:37:12.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.980+0000 7f8413fff700 1 --2- 172.21.15.88:0/1796528272 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f840406d8a0 0x7f840406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:12.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.980+0000 7f8413fff700 1 --2- 172.21.15.88:0/1796528272 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f840406d8a0 0x7f840406fd50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f840000b5c0 tx=0x7f8400005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:12.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:12.985+0000 7f8411ffb700 1 -- 172.21.15.88:0/1796528272 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8408056ec0 con 0x7f84141004e0 2024-02-19T19:37:13.075 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:12 smithi132 ceph-mon[70676]: pgmap v1010: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:12 smithi088 ceph-mon[88643]: pgmap v1010: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:13.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.200+0000 7f841ad6a700 1 -- 172.21.15.88:0/1796528272 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f841402cca0 con 0x7f840406d8a0 2024-02-19T19:37:13.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.201+0000 7f8411ffb700 1 -- 172.21.15.88:0/1796528272 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f841402cca0 con 0x7f840406d8a0 2024-02-19T19:37:13.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.203+0000 7f841ad6a700 1 -- 172.21.15.88:0/1796528272 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f840406d8a0 msgr2=0x7f840406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.203+0000 7f841ad6a700 1 --2- 172.21.15.88:0/1796528272 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f840406d8a0 0x7f840406fd50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f840000b5c0 tx=0x7f8400005960 comp rx=0 tx=0).stop 2024-02-19T19:37:13.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.203+0000 7f841ad6a700 1 -- 172.21.15.88:0/1796528272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84141004e0 msgr2=0x7f8414073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.203+0000 7f841ad6a700 1 --2- 172.21.15.88:0/1796528272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84141004e0 0x7f8414073050 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f840800d900 tx=0x7f840800dcc0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.204+0000 7f841ad6a700 1 -- 172.21.15.88:0/1796528272 shutdown_connections 2024-02-19T19:37:13.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.204+0000 7f841ad6a700 1 --2- 172.21.15.88:0/1796528272 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f840406d8a0 0x7f840406fd50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.204+0000 7f841ad6a700 1 --2- 172.21.15.88:0/1796528272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84141004e0 0x7f8414073050 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.204+0000 7f841ad6a700 1 --2- 172.21.15.88:0/1796528272 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8414100e80 0x7f8414073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.204+0000 7f841ad6a700 1 -- 172.21.15.88:0/1796528272 >> 172.21.15.88:0/1796528272 conn(0x7f84140fbda0 msgr2=0x7f84141046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:13.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.204+0000 7f841ad6a700 1 -- 172.21.15.88:0/1796528272 shutdown_connections 2024-02-19T19:37:13.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.204+0000 7f841ad6a700 1 -- 172.21.15.88:0/1796528272 wait complete. 2024-02-19T19:37:13.215 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:37:13.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.310+0000 7fe49d2ef700 1 -- 172.21.15.88:0/3051441592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4981004e0 msgr2=0x7fe4981008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.310+0000 7fe49d2ef700 1 --2- 172.21.15.88:0/3051441592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4981004e0 0x7fe4981008b0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe488009910 tx=0x7fe488009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:13.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.311+0000 7fe49d2ef700 1 -- 172.21.15.88:0/3051441592 shutdown_connections 2024-02-19T19:37:13.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.311+0000 7fe49d2ef700 1 --2- 172.21.15.88:0/3051441592 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe498100e80 0x7fe498104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.311+0000 7fe49d2ef700 1 --2- 172.21.15.88:0/3051441592 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4981004e0 0x7fe4981008b0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.311+0000 7fe49d2ef700 1 -- 172.21.15.88:0/3051441592 >> 172.21.15.88:0/3051441592 conn(0x7fe4980fbda0 msgr2=0x7fe4980fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:13.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.311+0000 7fe49d2ef700 1 -- 172.21.15.88:0/3051441592 shutdown_connections 2024-02-19T19:37:13.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.311+0000 7fe49d2ef700 1 -- 172.21.15.88:0/3051441592 wait complete. 2024-02-19T19:37:13.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.311+0000 7fe49d2ef700 1 Processor -- start 2024-02-19T19:37:13.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.311+0000 7fe49d2ef700 1 -- start start 2024-02-19T19:37:13.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe49d2ef700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4981004e0 0x7fe498073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:13.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe49d2ef700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe498100e80 0x7fe498073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:13.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe49d2ef700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe498073ad0 con 0x7fe498100e80 2024-02-19T19:37:13.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe49d2ef700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe498073c10 con 0x7fe4981004e0 2024-02-19T19:37:13.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe4967fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe498100e80 0x7fe498073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:13.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe4967fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe498100e80 0x7fe498073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44628/0 (socket says 172.21.15.88:44628) 2024-02-19T19:37:13.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe4967fc700 1 -- 172.21.15.88:0/2132983560 learned_addr learned my addr 172.21.15.88:0/2132983560 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:13.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe496ffd700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4981004e0 0x7fe498073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:13.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe4967fc700 1 -- 172.21.15.88:0/2132983560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4981004e0 msgr2=0x7fe498073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe4967fc700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4981004e0 0x7fe498073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe4967fc700 1 -- 172.21.15.88:0/2132983560 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4880095f0 con 0x7fe498100e80 2024-02-19T19:37:13.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe496ffd700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4981004e0 0x7fe498073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:37:13.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe4967fc700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe498100e80 0x7fe498073590 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00b480 tx=0x7fe48c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:13.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe47ffff700 1 -- 172.21.15.88:0/2132983560 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe48c0118c0 con 0x7fe498100e80 2024-02-19T19:37:13.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe49d2ef700 1 -- 172.21.15.88:0/2132983560 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4981a2160 con 0x7fe498100e80 2024-02-19T19:37:13.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe47ffff700 1 -- 172.21.15.88:0/2132983560 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe48c010380 con 0x7fe498100e80 2024-02-19T19:37:13.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe47ffff700 1 -- 172.21.15.88:0/2132983560 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe48c00f4f0 con 0x7fe498100e80 2024-02-19T19:37:13.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.312+0000 7fe49d2ef700 1 -- 172.21.15.88:0/2132983560 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4981a2570 con 0x7fe498100e80 2024-02-19T19:37:13.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.313+0000 7fe49d2ef700 1 -- 172.21.15.88:0/2132983560 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4981087f0 con 0x7fe498100e80 2024-02-19T19:37:13.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.314+0000 7fe47ffff700 1 -- 172.21.15.88:0/2132983560 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe48c0104f0 con 0x7fe498100e80 2024-02-19T19:37:13.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.314+0000 7fe47ffff700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe48006d8a0 0x7fe48006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:13.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.314+0000 7fe496ffd700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe48006d8a0 0x7fe48006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:13.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.314+0000 7fe47ffff700 1 -- 172.21.15.88:0/2132983560 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe48c08d180 con 0x7fe498100e80 2024-02-19T19:37:13.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.315+0000 7fe496ffd700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe48006d8a0 0x7fe48006fd50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe488006010 tx=0x7fe48800b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:13.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.319+0000 7fe47ffff700 1 -- 172.21.15.88:0/2132983560 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe48c056930 con 0x7fe498100e80 2024-02-19T19:37:13.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.535+0000 7fe49d2ef700 1 -- 172.21.15.88:0/2132983560 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe498061190 con 0x7fe48006d8a0 2024-02-19T19:37:13.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.537+0000 7fe47ffff700 1 -- 172.21.15.88:0/2132983560 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe498061190 con 0x7fe48006d8a0 2024-02-19T19:37:13.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.539+0000 7fe49d2ef700 1 -- 172.21.15.88:0/2132983560 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe48006d8a0 msgr2=0x7fe48006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.539+0000 7fe49d2ef700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe48006d8a0 0x7fe48006fd50 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe488006010 tx=0x7fe48800b540 comp rx=0 tx=0).stop 2024-02-19T19:37:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.539+0000 7fe49d2ef700 1 -- 172.21.15.88:0/2132983560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe498100e80 msgr2=0x7fe498073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.539+0000 7fe49d2ef700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe498100e80 0x7fe498073590 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00b480 tx=0x7fe48c00b840 comp rx=0 tx=0).stop 2024-02-19T19:37:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.539+0000 7fe49d2ef700 1 -- 172.21.15.88:0/2132983560 shutdown_connections 2024-02-19T19:37:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.539+0000 7fe49d2ef700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe48006d8a0 0x7fe48006fd50 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.539+0000 7fe49d2ef700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe4981004e0 0x7fe498073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.539+0000 7fe49d2ef700 1 --2- 172.21.15.88:0/2132983560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe498100e80 0x7fe498073590 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.539+0000 7fe49d2ef700 1 -- 172.21.15.88:0/2132983560 >> 172.21.15.88:0/2132983560 conn(0x7fe4980fbda0 msgr2=0x7fe4980fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.539+0000 7fe49d2ef700 1 -- 172.21.15.88:0/2132983560 shutdown_connections 2024-02-19T19:37:13.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.539+0000 7fe49d2ef700 1 -- 172.21.15.88:0/2132983560 wait complete. 2024-02-19T19:37:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.643+0000 7fbbefc63700 1 -- 172.21.15.88:0/520718553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbe8100e80 msgr2=0x7fbbe8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.643+0000 7fbbefc63700 1 --2- 172.21.15.88:0/520718553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbe8100e80 0x7fbbe8104e50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009910 tx=0x7fbbd8009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.643+0000 7fbbefc63700 1 -- 172.21.15.88:0/520718553 shutdown_connections 2024-02-19T19:37:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.643+0000 7fbbefc63700 1 --2- 172.21.15.88:0/520718553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbe8100e80 0x7fbbe8104e50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.643+0000 7fbbefc63700 1 --2- 172.21.15.88:0/520718553 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe81004e0 0x7fbbe81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.643+0000 7fbbefc63700 1 -- 172.21.15.88:0/520718553 >> 172.21.15.88:0/520718553 conn(0x7fbbe80fbda0 msgr2=0x7fbbe80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.643+0000 7fbbefc63700 1 -- 172.21.15.88:0/520718553 shutdown_connections 2024-02-19T19:37:13.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.643+0000 7fbbefc63700 1 -- 172.21.15.88:0/520718553 wait complete. 2024-02-19T19:37:13.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbefc63700 1 Processor -- start 2024-02-19T19:37:13.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbefc63700 1 -- start start 2024-02-19T19:37:13.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbefc63700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe81004e0 0x7fbbe8195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:13.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbefc63700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbe8100e80 0x7fbbe8196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:13.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbefc63700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe8196b30 con 0x7fbbe8100e80 2024-02-19T19:37:13.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbefc63700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe819a8c0 con 0x7fbbe81004e0 2024-02-19T19:37:13.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbed1fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbe8100e80 0x7fbbe8196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbed1fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbe8100e80 0x7fbbe8196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44642/0 (socket says 172.21.15.88:44642) 2024-02-19T19:37:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbed1fe700 1 -- 172.21.15.88:0/2699153231 learned_addr learned my addr 172.21.15.88:0/2699153231 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbed1fe700 1 -- 172.21.15.88:0/2699153231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe81004e0 msgr2=0x7fbbe8195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbed1fe700 1 --2- 172.21.15.88:0/2699153231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe81004e0 0x7fbbe8195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.644+0000 7fbbed1fe700 1 -- 172.21.15.88:0/2699153231 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd80095f0 con 0x7fbbe8100e80 2024-02-19T19:37:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.645+0000 7fbbed1fe700 1 --2- 172.21.15.88:0/2699153231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbe8100e80 0x7fbbe8196450 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8005f50 tx=0x7fbbd800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.645+0000 7fbbdeffd700 1 -- 172.21.15.88:0/2699153231 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbd801d070 con 0x7fbbe8100e80 2024-02-19T19:37:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.645+0000 7fbbefc63700 1 -- 172.21.15.88:0/2699153231 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe819ab40 con 0x7fbbe8100e80 2024-02-19T19:37:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.645+0000 7fbbdeffd700 1 -- 172.21.15.88:0/2699153231 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbbd800f460 con 0x7fbbe8100e80 2024-02-19T19:37:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.645+0000 7fbbdeffd700 1 -- 172.21.15.88:0/2699153231 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbd8017630 con 0x7fbbe8100e80 2024-02-19T19:37:13.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.645+0000 7fbbefc63700 1 -- 172.21.15.88:0/2699153231 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe819b030 con 0x7fbbe8100e80 2024-02-19T19:37:13.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.646+0000 7fbbefc63700 1 -- 172.21.15.88:0/2699153231 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe8108850 con 0x7fbbe8100e80 2024-02-19T19:37:13.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.646+0000 7fbbdeffd700 1 -- 172.21.15.88:0/2699153231 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbbd800f5d0 con 0x7fbbe8100e80 2024-02-19T19:37:13.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.647+0000 7fbbdeffd700 1 --2- 172.21.15.88:0/2699153231 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbbd406d8a0 0x7fbbd406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:13.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.647+0000 7fbbdeffd700 1 -- 172.21.15.88:0/2699153231 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbbd808eb50 con 0x7fbbe8100e80 2024-02-19T19:37:13.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.647+0000 7fbbed9ff700 1 --2- 172.21.15.88:0/2699153231 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbbd406d8a0 0x7fbbd406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:13.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.647+0000 7fbbed9ff700 1 --2- 172.21.15.88:0/2699153231 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbbd406d8a0 0x7fbbd406fd50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400a990 tx=0x7fbbe4005c80 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:13.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.652+0000 7fbbdeffd700 1 -- 172.21.15.88:0/2699153231 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbbd8058300 con 0x7fbbe8100e80 2024-02-19T19:37:13.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.862+0000 7fbbefc63700 1 -- 172.21.15.88:0/2699153231 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbe8197310 con 0x7fbbd406d8a0 2024-02-19T19:37:13.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.872+0000 7fbbdeffd700 1 -- 172.21.15.88:0/2699153231 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fbbe8197310 con 0x7fbbd406d8a0 2024-02-19T19:37:13.874 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:37:13.874 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (34m) 10m ago 44m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:37:13.874 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (45m) 10m ago 45m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:37:13.874 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (27m) 10m ago 43m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:37:13.874 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (30m) 10m ago 45m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:37:13.874 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (30m) 10m ago 43m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:37:13.874 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (33m) 10m ago 44m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (26m) 10m ago 38m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (26m) 10m ago 38m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (26m) 10m ago 38m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (10m) 10m ago 38m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (35m) 10m ago 46m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (35m) 10m ago 43m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (32m) 10m ago 46m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (31m) 10m ago 43m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (35m) 10m ago 45m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (35m) 10m ago 43m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (29m) 10m ago 42m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (29m) 10m ago 41m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:37:13.875 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (28m) 10m ago 41m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:37:13.876 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (28m) 10m ago 40m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:37:13.876 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (27m) 10m ago 40m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:37:13.876 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (26m) 10m ago 40m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:37:13.876 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (34m) 10m ago 43m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:37:13.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.875+0000 7fbbefc63700 1 -- 172.21.15.88:0/2699153231 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbbd406d8a0 msgr2=0x7fbbd406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.875+0000 7fbbefc63700 1 --2- 172.21.15.88:0/2699153231 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbbd406d8a0 0x7fbbd406fd50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbbe400a990 tx=0x7fbbe4005c80 comp rx=0 tx=0).stop 2024-02-19T19:37:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.875+0000 7fbbefc63700 1 -- 172.21.15.88:0/2699153231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbe8100e80 msgr2=0x7fbbe8196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.875+0000 7fbbefc63700 1 --2- 172.21.15.88:0/2699153231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbe8100e80 0x7fbbe8196450 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8005f50 tx=0x7fbbd800be20 comp rx=0 tx=0).stop 2024-02-19T19:37:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.875+0000 7fbbefc63700 1 -- 172.21.15.88:0/2699153231 shutdown_connections 2024-02-19T19:37:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.875+0000 7fbbefc63700 1 --2- 172.21.15.88:0/2699153231 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbbd406d8a0 0x7fbbd406fd50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.875+0000 7fbbefc63700 1 --2- 172.21.15.88:0/2699153231 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbbe81004e0 0x7fbbe8195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.875+0000 7fbbefc63700 1 --2- 172.21.15.88:0/2699153231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbe8100e80 0x7fbbe8196450 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.875+0000 7fbbefc63700 1 -- 172.21.15.88:0/2699153231 >> 172.21.15.88:0/2699153231 conn(0x7fbbe80fbda0 msgr2=0x7fbbe80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.875+0000 7fbbefc63700 1 -- 172.21.15.88:0/2699153231 shutdown_connections 2024-02-19T19:37:13.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.875+0000 7fbbefc63700 1 -- 172.21.15.88:0/2699153231 wait complete. 2024-02-19T19:37:13.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.977+0000 7f341a43c700 1 -- 172.21.15.88:0/2346399674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414100e80 msgr2=0x7f3414104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.977+0000 7f341a43c700 1 --2- 172.21.15.88:0/2346399674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414100e80 0x7f3414104e50 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f3400009910 tx=0x7f3400009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:13.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.977+0000 7f341a43c700 1 -- 172.21.15.88:0/2346399674 shutdown_connections 2024-02-19T19:37:13.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.977+0000 7f341a43c700 1 --2- 172.21.15.88:0/2346399674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414100e80 0x7f3414104e50 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.977+0000 7f341a43c700 1 --2- 172.21.15.88:0/2346399674 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34141004e0 0x7f34141008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.977+0000 7f341a43c700 1 -- 172.21.15.88:0/2346399674 >> 172.21.15.88:0/2346399674 conn(0x7f34140fbda0 msgr2=0x7f34140fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:13.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.977+0000 7f341a43c700 1 -- 172.21.15.88:0/2346399674 shutdown_connections 2024-02-19T19:37:13.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.977+0000 7f341a43c700 1 -- 172.21.15.88:0/2346399674 wait complete. 2024-02-19T19:37:13.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.978+0000 7f341a43c700 1 Processor -- start 2024-02-19T19:37:13.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.978+0000 7f341a43c700 1 -- start start 2024-02-19T19:37:13.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.978+0000 7f341a43c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34141004e0 0x7f3414198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.978+0000 7f341a43c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414100e80 0x7f34141986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.978+0000 7f341a43c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3414198db0 con 0x7f3414100e80 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.978+0000 7f341a43c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341419cb40 con 0x7f34141004e0 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.978+0000 7f34137fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414100e80 0x7f34141986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f34137fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414100e80 0x7f34141986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44666/0 (socket says 172.21.15.88:44666) 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f34137fe700 1 -- 172.21.15.88:0/198892250 learned_addr learned my addr 172.21.15.88:0/198892250 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f3413fff700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34141004e0 0x7f3414198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f34137fe700 1 -- 172.21.15.88:0/198892250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34141004e0 msgr2=0x7f3414198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f34137fe700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34141004e0 0x7f3414198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f34137fe700 1 -- 172.21.15.88:0/198892250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34000095f0 con 0x7f3414100e80 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f3413fff700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34141004e0 0x7f3414198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:37:13.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f34137fe700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414100e80 0x7f34141986d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3400005f50 tx=0x7f340000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:13.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f34117fa700 1 -- 172.21.15.88:0/198892250 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f340001d070 con 0x7f3414100e80 2024-02-19T19:37:13.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f341a43c700 1 -- 172.21.15.88:0/198892250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f341419cdc0 con 0x7f3414100e80 2024-02-19T19:37:13.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f34117fa700 1 -- 172.21.15.88:0/198892250 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f340000f460 con 0x7f3414100e80 2024-02-19T19:37:13.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f341a43c700 1 -- 172.21.15.88:0/198892250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f341419d2b0 con 0x7f3414100e80 2024-02-19T19:37:13.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.979+0000 7f34117fa700 1 -- 172.21.15.88:0/198892250 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3400017630 con 0x7f3414100e80 2024-02-19T19:37:13.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.980+0000 7f341a43c700 1 -- 172.21.15.88:0/198892250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34141045d0 con 0x7f3414100e80 2024-02-19T19:37:13.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.981+0000 7f34117fa700 1 -- 172.21.15.88:0/198892250 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f340000f5d0 con 0x7f3414100e80 2024-02-19T19:37:13.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.981+0000 7f34117fa700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340406d8a0 0x7f340406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:13.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.981+0000 7f34117fa700 1 -- 172.21.15.88:0/198892250 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f340008eb50 con 0x7f3414100e80 2024-02-19T19:37:13.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.981+0000 7f3413fff700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340406d8a0 0x7f340406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:13.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.982+0000 7f3413fff700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340406d8a0 0x7f340406fd50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3408007900 tx=0x7f3408008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:13.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:13.986+0000 7f34117fa700 1 -- 172.21.15.88:0/198892250 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3400058300 con 0x7f3414100e80 2024-02-19T19:37:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:13 smithi088 ceph-mon[88643]: from='client.35818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:37:14.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:13 smithi088 ceph-mon[88643]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:14.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:13 smithi132 ceph-mon[70676]: from='client.35818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:14.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:37:14.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:13 smithi132 ceph-mon[70676]: from='client.35822 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:14.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.265+0000 7f341a43c700 1 -- 172.21.15.88:0/198892250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3414199590 con 0x7f3414100e80 2024-02-19T19:37:14.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.265+0000 7f34117fa700 1 -- 172.21.15.88:0/198892250 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f340005b9c0 con 0x7f3414100e80 2024-02-19T19:37:14.267 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:37:14.267 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:37:14.267 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:37:14.267 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:37:14.267 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:37:14.267 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:37:14.267 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:37:14.268 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:37:14.268 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:37:14.268 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:37:14.268 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:37:14.268 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:37:14.268 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:37:14.268 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:37:14.268 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:37:14.268 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:37:14.268 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:37:14.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.267+0000 7f341a43c700 1 -- 172.21.15.88:0/198892250 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340406d8a0 msgr2=0x7f340406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:14.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.267+0000 7f341a43c700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340406d8a0 0x7f340406fd50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3408007900 tx=0x7f3408008040 comp rx=0 tx=0).stop 2024-02-19T19:37:14.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.267+0000 7f341a43c700 1 -- 172.21.15.88:0/198892250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414100e80 msgr2=0x7f34141986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:14.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.267+0000 7f341a43c700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414100e80 0x7f34141986d0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f3400005f50 tx=0x7f340000be20 comp rx=0 tx=0).stop 2024-02-19T19:37:14.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.268+0000 7f341a43c700 1 -- 172.21.15.88:0/198892250 shutdown_connections 2024-02-19T19:37:14.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.268+0000 7f341a43c700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f340406d8a0 0x7f340406fd50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.268+0000 7f341a43c700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f34141004e0 0x7f3414198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.268+0000 7f341a43c700 1 --2- 172.21.15.88:0/198892250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3414100e80 0x7f34141986d0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.268+0000 7f341a43c700 1 -- 172.21.15.88:0/198892250 >> 172.21.15.88:0/198892250 conn(0x7f34140fbda0 msgr2=0x7f34140fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:14.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.268+0000 7f341a43c700 1 -- 172.21.15.88:0/198892250 shutdown_connections 2024-02-19T19:37:14.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.268+0000 7f341a43c700 1 -- 172.21.15.88:0/198892250 wait complete. 2024-02-19T19:37:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.372+0000 7f9791378700 1 -- 172.21.15.88:0/1967870918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f978c1004c0 msgr2=0x7f978c100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.372+0000 7f9791378700 1 --2- 172.21.15.88:0/1967870918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f978c1004c0 0x7f978c100890 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f978400b0a0 tx=0x7f978400b3b0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.372+0000 7f9791378700 1 -- 172.21.15.88:0/1967870918 shutdown_connections 2024-02-19T19:37:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.372+0000 7f9791378700 1 --2- 172.21.15.88:0/1967870918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f978c100e60 0x7f978c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.372+0000 7f9791378700 1 --2- 172.21.15.88:0/1967870918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f978c1004c0 0x7f978c100890 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.372+0000 7f9791378700 1 -- 172.21.15.88:0/1967870918 >> 172.21.15.88:0/1967870918 conn(0x7f978c0fbd80 msgr2=0x7f978c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.372+0000 7f9791378700 1 -- 172.21.15.88:0/1967870918 shutdown_connections 2024-02-19T19:37:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.372+0000 7f9791378700 1 -- 172.21.15.88:0/1967870918 wait complete. 2024-02-19T19:37:14.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f9791378700 1 Processor -- start 2024-02-19T19:37:14.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f9791378700 1 -- start start 2024-02-19T19:37:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f9791378700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f978c1004c0 0x7f978c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f9791378700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f978c100e60 0x7f978c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f9791378700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c198d10 con 0x7f978c1004c0 2024-02-19T19:37:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f9791378700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f978c19caa0 con 0x7f978c100e60 2024-02-19T19:37:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f978affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f978c1004c0 0x7f978c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f978affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f978c1004c0 0x7f978c1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44698/0 (socket says 172.21.15.88:44698) 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f978affd700 1 -- 172.21.15.88:0/1646711614 learned_addr learned my addr 172.21.15.88:0/1646711614 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f978a7fc700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f978c100e60 0x7f978c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f978affd700 1 -- 172.21.15.88:0/1646711614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f978c100e60 msgr2=0x7f978c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f978affd700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f978c100e60 0x7f978c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f978affd700 1 -- 172.21.15.88:0/1646711614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9784009d00 con 0x7f978c1004c0 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.373+0000 7f978a7fc700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f978c100e60 0x7f978c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.374+0000 7f978affd700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f978c1004c0 0x7f978c1980f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9784000f80 tx=0x7f978400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.374+0000 7f9773fff700 1 -- 172.21.15.88:0/1646711614 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f978400e040 con 0x7f978c1004c0 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.374+0000 7f9773fff700 1 -- 172.21.15.88:0/1646711614 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9784003e60 con 0x7f978c1004c0 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.374+0000 7f9773fff700 1 -- 172.21.15.88:0/1646711614 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f978401d910 con 0x7f978c1004c0 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.374+0000 7f9791378700 1 -- 172.21.15.88:0/1646711614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f978c19cd20 con 0x7f978c1004c0 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.374+0000 7f9791378700 1 -- 172.21.15.88:0/1646711614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f978c19d1c0 con 0x7f978c1004c0 2024-02-19T19:37:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.375+0000 7f9791378700 1 -- 172.21.15.88:0/1646711614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f978c1045b0 con 0x7f978c1004c0 2024-02-19T19:37:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.376+0000 7f9773fff700 1 -- 172.21.15.88:0/1646711614 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9784004820 con 0x7f978c1004c0 2024-02-19T19:37:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.376+0000 7f9773fff700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f977406d8a0 0x7f977406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.376+0000 7f9773fff700 1 -- 172.21.15.88:0/1646711614 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f978408ee10 con 0x7f978c1004c0 2024-02-19T19:37:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.376+0000 7f978a7fc700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f977406d8a0 0x7f977406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.377+0000 7f978a7fc700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f977406d8a0 0x7f977406fd50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9780005fd0 tx=0x7f9780005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:14.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.381+0000 7f9773fff700 1 -- 172.21.15.88:0/1646711614 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f97840585c0 con 0x7f978c1004c0 2024-02-19T19:37:14.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.624+0000 7f9791378700 1 -- 172.21.15.88:0/1646711614 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f978c04f2a0 con 0x7f978c1004c0 2024-02-19T19:37:14.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.625+0000 7f9773fff700 1 -- 172.21.15.88:0/1646711614 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f978405bc80 con 0x7f978c1004c0 2024-02-19T19:37:14.627 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:37:14.627 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:37:14.627 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:37:14.627 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:37:14.627 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:37:14.627 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:37:14.627 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:37:14.627 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:37:14.627 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:37:14.628 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:37:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:37:14.630 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:37:14.630 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:37:14.630 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:37:14.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.627+0000 7f9791378700 1 -- 172.21.15.88:0/1646711614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f977406d8a0 msgr2=0x7f977406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:14.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.627+0000 7f9791378700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f977406d8a0 0x7f977406fd50 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9780005fd0 tx=0x7f9780005dc0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.627+0000 7f9791378700 1 -- 172.21.15.88:0/1646711614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f978c1004c0 msgr2=0x7f978c1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:14.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.627+0000 7f9791378700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f978c1004c0 0x7f978c1980f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9784000f80 tx=0x7f978400cdd0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.627+0000 7f9791378700 1 -- 172.21.15.88:0/1646711614 shutdown_connections 2024-02-19T19:37:14.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.627+0000 7f9791378700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f977406d8a0 0x7f977406fd50 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.627+0000 7f9791378700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f978c1004c0 0x7f978c1980f0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.627+0000 7f9791378700 1 --2- 172.21.15.88:0/1646711614 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f978c100e60 0x7f978c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.627+0000 7f9791378700 1 -- 172.21.15.88:0/1646711614 >> 172.21.15.88:0/1646711614 conn(0x7f978c0fbd80 msgr2=0x7f978c0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:14.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.627+0000 7f9791378700 1 -- 172.21.15.88:0/1646711614 shutdown_connections 2024-02-19T19:37:14.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.627+0000 7f9791378700 1 -- 172.21.15.88:0/1646711614 wait complete. 2024-02-19T19:37:14.631 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:37:14.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.733+0000 7fac312a7700 1 -- 172.21.15.88:0/4185808677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac2c0ff3c0 msgr2=0x7fac2c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:14.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.733+0000 7fac312a7700 1 --2- 172.21.15.88:0/4185808677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac2c0ff3c0 0x7fac2c0ff790 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009910 tx=0x7fac1c009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:14.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.733+0000 7fac312a7700 1 -- 172.21.15.88:0/4185808677 shutdown_connections 2024-02-19T19:37:14.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.733+0000 7fac312a7700 1 --2- 172.21.15.88:0/4185808677 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac2c0ffd60 0x7fac2c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.733+0000 7fac312a7700 1 --2- 172.21.15.88:0/4185808677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac2c0ff3c0 0x7fac2c0ff790 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.733+0000 7fac312a7700 1 -- 172.21.15.88:0/4185808677 >> 172.21.15.88:0/4185808677 conn(0x7fac2c0fac80 msgr2=0x7fac2c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:14.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.733+0000 7fac312a7700 1 -- 172.21.15.88:0/4185808677 shutdown_connections 2024-02-19T19:37:14.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.733+0000 7fac312a7700 1 -- 172.21.15.88:0/4185808677 wait complete. 2024-02-19T19:37:14.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.733+0000 7fac312a7700 1 Processor -- start 2024-02-19T19:37:14.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.733+0000 7fac312a7700 1 -- start start 2024-02-19T19:37:14.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac312a7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac2c0ff3c0 0x7fac2c195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:14.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac312a7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac2c0ffd60 0x7fac2c1963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:14.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac312a7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c196a80 con 0x7fac2c0ffd60 2024-02-19T19:37:14.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac312a7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c19a810 con 0x7fac2c0ff3c0 2024-02-19T19:37:14.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac227fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac2c0ffd60 0x7fac2c1963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:14.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac227fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac2c0ffd60 0x7fac2c1963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44702/0 (socket says 172.21.15.88:44702) 2024-02-19T19:37:14.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac227fc700 1 -- 172.21.15.88:0/4169695165 learned_addr learned my addr 172.21.15.88:0/4169695165 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:14.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac2affd700 1 --2- 172.21.15.88:0/4169695165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac2c0ff3c0 0x7fac2c195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:14.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac227fc700 1 -- 172.21.15.88:0/4169695165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac2c0ff3c0 msgr2=0x7fac2c195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:14.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac227fc700 1 --2- 172.21.15.88:0/4169695165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac2c0ff3c0 0x7fac2c195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac227fc700 1 -- 172.21.15.88:0/4169695165 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c0095f0 con 0x7fac2c0ffd60 2024-02-19T19:37:14.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac227fc700 1 --2- 172.21.15.88:0/4169695165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac2c0ffd60 0x7fac2c1963a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fac1400d900 tx=0x7fac1400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:14.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.734+0000 7fac28ff9700 1 -- 172.21.15.88:0/4169695165 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac14009900 con 0x7fac2c0ffd60 2024-02-19T19:37:14.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.735+0000 7fac28ff9700 1 -- 172.21.15.88:0/4169695165 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac14010460 con 0x7fac2c0ffd60 2024-02-19T19:37:14.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.735+0000 7fac312a7700 1 -- 172.21.15.88:0/4169695165 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2c19aaf0 con 0x7fac2c0ffd60 2024-02-19T19:37:14.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.735+0000 7fac28ff9700 1 -- 172.21.15.88:0/4169695165 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac1400f5f0 con 0x7fac2c0ffd60 2024-02-19T19:37:14.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.735+0000 7fac312a7700 1 -- 172.21.15.88:0/4169695165 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c19b040 con 0x7fac2c0ffd60 2024-02-19T19:37:14.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.735+0000 7fac312a7700 1 -- 172.21.15.88:0/4169695165 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2c04f2a0 con 0x7fac2c0ffd60 2024-02-19T19:37:14.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.736+0000 7fac28ff9700 1 -- 172.21.15.88:0/4169695165 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fac140105d0 con 0x7fac2c0ffd60 2024-02-19T19:37:14.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.736+0000 7fac28ff9700 1 --2- 172.21.15.88:0/4169695165 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fac1806d8a0 0x7fac1806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:14.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.737+0000 7fac2affd700 1 --2- 172.21.15.88:0/4169695165 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fac1806d8a0 0x7fac1806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:14.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.737+0000 7fac28ff9700 1 -- 172.21.15.88:0/4169695165 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fac1408d140 con 0x7fac2c0ffd60 2024-02-19T19:37:14.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.737+0000 7fac2affd700 1 --2- 172.21.15.88:0/4169695165 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fac1806d8a0 0x7fac1806fd50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006010 tx=0x7fac1c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:14.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.741+0000 7fac28ff9700 1 -- 172.21.15.88:0/4169695165 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fac14056890 con 0x7fac2c0ffd60 2024-02-19T19:37:14.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.959+0000 7fac312a7700 1 -- 172.21.15.88:0/4169695165 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac2c19b370 con 0x7fac1806d8a0 2024-02-19T19:37:14.961 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:14 smithi088 ceph-mon[88643]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:14.961 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:14 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/198892250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:37:14.961 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:14 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1646711614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:37:14.961 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:14 smithi088 ceph-mon[88643]: pgmap v1011: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:14.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.961+0000 7fac28ff9700 1 -- 172.21.15.88:0/4169695165 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fac2c19b370 con 0x7fac1806d8a0 2024-02-19T19:37:14.962 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:37:14.962 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:37:14.963 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:37:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.963+0000 7fac312a7700 1 -- 172.21.15.88:0/4169695165 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fac1806d8a0 msgr2=0x7fac1806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.963+0000 7fac312a7700 1 --2- 172.21.15.88:0/4169695165 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fac1806d8a0 0x7fac1806fd50 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006010 tx=0x7fac1c01a040 comp rx=0 tx=0).stop 2024-02-19T19:37:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.963+0000 7fac312a7700 1 -- 172.21.15.88:0/4169695165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac2c0ffd60 msgr2=0x7fac2c1963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.963+0000 7fac312a7700 1 --2- 172.21.15.88:0/4169695165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac2c0ffd60 0x7fac2c1963a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fac1400d900 tx=0x7fac1400dc10 comp rx=0 tx=0).stop 2024-02-19T19:37:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.963+0000 7fac312a7700 1 -- 172.21.15.88:0/4169695165 shutdown_connections 2024-02-19T19:37:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.963+0000 7fac312a7700 1 --2- 172.21.15.88:0/4169695165 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fac1806d8a0 0x7fac1806fd50 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.963+0000 7fac312a7700 1 --2- 172.21.15.88:0/4169695165 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fac2c0ff3c0 0x7fac2c195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.963+0000 7fac312a7700 1 --2- 172.21.15.88:0/4169695165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac2c0ffd60 0x7fac2c1963a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.963+0000 7fac312a7700 1 -- 172.21.15.88:0/4169695165 >> 172.21.15.88:0/4169695165 conn(0x7fac2c0fac80 msgr2=0x7fac2c0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.963+0000 7fac312a7700 1 -- 172.21.15.88:0/4169695165 shutdown_connections 2024-02-19T19:37:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:14.963+0000 7fac312a7700 1 -- 172.21.15.88:0/4169695165 wait complete. 2024-02-19T19:37:15.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.067+0000 7f75df9b5700 1 -- 172.21.15.88:0/2432602678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75d8101410 msgr2=0x7f75d81017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:15.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.067+0000 7f75df9b5700 1 --2- 172.21.15.88:0/2432602678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75d8101410 0x7f75d81017e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f75c8009910 tx=0x7f75c8009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:15.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.067+0000 7f75df9b5700 1 -- 172.21.15.88:0/2432602678 shutdown_connections 2024-02-19T19:37:15.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.067+0000 7f75df9b5700 1 --2- 172.21.15.88:0/2432602678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75d8101d20 0x7f75d810a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:15.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.067+0000 7f75df9b5700 1 --2- 172.21.15.88:0/2432602678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75d8101410 0x7f75d81017e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:15.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.067+0000 7f75df9b5700 1 -- 172.21.15.88:0/2432602678 >> 172.21.15.88:0/2432602678 conn(0x7f75d80fac90 msgr2=0x7f75d80fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:15.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.067+0000 7f75df9b5700 1 -- 172.21.15.88:0/2432602678 shutdown_connections 2024-02-19T19:37:15.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.067+0000 7f75df9b5700 1 -- 172.21.15.88:0/2432602678 wait complete. 2024-02-19T19:37:15.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75df9b5700 1 Processor -- start 2024-02-19T19:37:15.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75df9b5700 1 -- start start 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75df9b5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75d8101410 0x7f75d8195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75df9b5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75d8101d20 0x7f75d81964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75df9b5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8196ba0 con 0x7f75d8101d20 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75df9b5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d819a930 con 0x7f75d8101410 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75dcf50700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75d8101d20 0x7f75d81964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75dcf50700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75d8101d20 0x7f75d81964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44710/0 (socket says 172.21.15.88:44710) 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75dcf50700 1 -- 172.21.15.88:0/4232759254 learned_addr learned my addr 172.21.15.88:0/4232759254 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75dd751700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75d8101410 0x7f75d8195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75dcf50700 1 -- 172.21.15.88:0/4232759254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75d8101410 msgr2=0x7f75d8195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75dcf50700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75d8101410 0x7f75d8195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75dcf50700 1 -- 172.21.15.88:0/4232759254 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c80095f0 con 0x7f75d8101d20 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.068+0000 7f75dd751700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75d8101410 0x7f75d8195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.069+0000 7f75dcf50700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75d8101d20 0x7f75d81964c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f75d400b4b0 tx=0x7f75d400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:15.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.069+0000 7f75ce7fc700 1 -- 172.21.15.88:0/4232759254 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75d4011800 con 0x7f75d8101d20 2024-02-19T19:37:15.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.069+0000 7f75ce7fc700 1 -- 172.21.15.88:0/4232759254 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75d4011e20 con 0x7f75d8101d20 2024-02-19T19:37:15.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.069+0000 7f75df9b5700 1 -- 172.21.15.88:0/4232759254 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d819ac10 con 0x7f75d8101d20 2024-02-19T19:37:15.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.069+0000 7f75ce7fc700 1 -- 172.21.15.88:0/4232759254 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75d400f2e0 con 0x7f75d8101d20 2024-02-19T19:37:15.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.069+0000 7f75df9b5700 1 -- 172.21.15.88:0/4232759254 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d819b160 con 0x7f75d8101d20 2024-02-19T19:37:15.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.070+0000 7f75df9b5700 1 -- 172.21.15.88:0/4232759254 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d81078d0 con 0x7f75d8101d20 2024-02-19T19:37:15.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.070+0000 7f75ce7fc700 1 -- 172.21.15.88:0/4232759254 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f75d4010380 con 0x7f75d8101d20 2024-02-19T19:37:15.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.071+0000 7f75ce7fc700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75c406d8a0 0x7f75c406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:15.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.071+0000 7f75ce7fc700 1 -- 172.21.15.88:0/4232759254 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f75d408d110 con 0x7f75d8101d20 2024-02-19T19:37:15.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.071+0000 7f75dd751700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75c406d8a0 0x7f75c406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:15.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.071+0000 7f75dd751700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75c406d8a0 0x7f75c406fd50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f75c80059d0 tx=0x7f75c801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:15.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.075+0000 7f75ce7fc700 1 -- 172.21.15.88:0/4232759254 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f75d40568c0 con 0x7f75d8101d20 2024-02-19T19:37:15.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:14 smithi132 ceph-mon[70676]: from='client.35826 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:15.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:14 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/198892250' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:37:15.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:14 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1646711614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:37:15.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:14 smithi132 ceph-mon[70676]: pgmap v1011: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:15.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.354+0000 7f75df9b5700 1 -- 172.21.15.88:0/4232759254 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f75d819b3e0 con 0x7f75d8101d20 2024-02-19T19:37:15.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.354+0000 7f75ce7fc700 1 -- 172.21.15.88:0/4232759254 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f75d4059f80 con 0x7f75d8101d20 2024-02-19T19:37:15.356 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:37:15.356 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:37:15.356 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:37:15.356 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:37:15.356 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:37:15.356 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:37:15.356 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:37:15.356 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:37:15.356 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:37:15.356 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:37:15.356 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:37:15.357 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:37:15.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.356+0000 7f75df9b5700 1 -- 172.21.15.88:0/4232759254 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75c406d8a0 msgr2=0x7f75c406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:15.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.356+0000 7f75df9b5700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75c406d8a0 0x7f75c406fd50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f75c80059d0 tx=0x7f75c801a040 comp rx=0 tx=0).stop 2024-02-19T19:37:15.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.356+0000 7f75df9b5700 1 -- 172.21.15.88:0/4232759254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75d8101d20 msgr2=0x7f75d81964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:15.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.356+0000 7f75df9b5700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75d8101d20 0x7f75d81964c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f75d400b4b0 tx=0x7f75d400b870 comp rx=0 tx=0).stop 2024-02-19T19:37:15.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.356+0000 7f75df9b5700 1 -- 172.21.15.88:0/4232759254 shutdown_connections 2024-02-19T19:37:15.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.356+0000 7f75df9b5700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75c406d8a0 0x7f75c406fd50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:15.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.356+0000 7f75df9b5700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75d8101410 0x7f75d8195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:15.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.356+0000 7f75df9b5700 1 --2- 172.21.15.88:0/4232759254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75d8101d20 0x7f75d81964c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:15.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.356+0000 7f75df9b5700 1 -- 172.21.15.88:0/4232759254 >> 172.21.15.88:0/4232759254 conn(0x7f75d80fac90 msgr2=0x7f75d8104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:15.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.356+0000 7f75df9b5700 1 -- 172.21.15.88:0/4232759254 shutdown_connections 2024-02-19T19:37:15.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:15.356+0000 7f75df9b5700 1 -- 172.21.15.88:0/4232759254 wait complete. 2024-02-19T19:37:16.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:15 smithi088 ceph-mon[88643]: from='client.35838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:16.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:15 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4232759254' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:37:16.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:15 smithi132 ceph-mon[70676]: from='client.35838 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:16.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:15 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4232759254' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:37:17.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:16 smithi088 ceph-mon[88643]: pgmap v1012: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:17.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:16 smithi132 ceph-mon[70676]: pgmap v1012: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:18 smithi088 ceph-mon[88643]: pgmap v1013: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:18 smithi132 ceph-mon[70676]: pgmap v1013: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:20 smithi088 ceph-mon[88643]: pgmap v1014: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:21.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:20 smithi132 ceph-mon[70676]: pgmap v1014: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:22 smithi088 ceph-mon[88643]: pgmap v1015: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:23.181 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:22 smithi132 ceph-mon[70676]: pgmap v1015: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:24 smithi088 ceph-mon[88643]: pgmap v1016: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:25.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:24 smithi132 ceph-mon[70676]: pgmap v1016: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:26 smithi088 ceph-mon[88643]: pgmap v1017: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:27.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:26 smithi132 ceph-mon[70676]: pgmap v1017: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:28.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:37:28.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:37:29.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:29 smithi088 ceph-mon[88643]: pgmap v1018: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:29.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:29 smithi132 ceph-mon[70676]: pgmap v1018: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:31.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:30 smithi088 ceph-mon[88643]: pgmap v1019: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:31.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:30 smithi132 ceph-mon[70676]: pgmap v1019: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:33.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:32 smithi088 ceph-mon[88643]: pgmap v1020: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:33.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:32 smithi132 ceph-mon[70676]: pgmap v1020: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:35.018 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:34 smithi088 ceph-mon[88643]: pgmap v1021: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:34 smithi132 ceph-mon[70676]: pgmap v1021: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:36 smithi088 ceph-mon[88643]: pgmap v1022: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:36 smithi132 ceph-mon[70676]: pgmap v1022: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:38 smithi088 ceph-mon[88643]: pgmap v1023: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:39.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:38 smithi132 ceph-mon[70676]: pgmap v1023: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:40 smithi088 ceph-mon[88643]: pgmap v1024: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:40 smithi132 ceph-mon[70676]: pgmap v1024: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:42 smithi088 ceph-mon[88643]: pgmap v1025: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:43.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:42 smithi132 ceph-mon[70676]: pgmap v1025: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:37:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:37:44.982 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:44 smithi088 ceph-mon[88643]: pgmap v1026: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:44 smithi132 ceph-mon[70676]: pgmap v1026: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:45.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.471+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/28310512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880ffd60 msgr2=0x7f5a88103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:45.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.471+0000 7f5a8f0c5700 1 --2- 172.21.15.88:0/28310512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880ffd60 0x7f5a88103d30 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009910 tx=0x7f5a7c009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:45.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.471+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/28310512 shutdown_connections 2024-02-19T19:37:45.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.471+0000 7f5a8f0c5700 1 --2- 172.21.15.88:0/28310512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880ffd60 0x7f5a88103d30 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:45.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.471+0000 7f5a8f0c5700 1 --2- 172.21.15.88:0/28310512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a880ff3c0 0x7f5a880ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:45.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.471+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/28310512 >> 172.21.15.88:0/28310512 conn(0x7f5a880fac80 msgr2=0x7f5a880fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.471+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/28310512 shutdown_connections 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.472+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/28310512 wait complete. 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.472+0000 7f5a8f0c5700 1 Processor -- start 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.472+0000 7f5a8f0c5700 1 -- start start 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.472+0000 7f5a8f0c5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880ff3c0 0x7f5a88196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.472+0000 7f5a8f0c5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a880ffd60 0x7f5a88196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.472+0000 7f5a8f0c5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a88196c20 con 0x7f5a880ff3c0 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.472+0000 7f5a8f0c5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a8819a9b0 con 0x7f5a880ffd60 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.472+0000 7f5a8ce61700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880ff3c0 0x7f5a88196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.472+0000 7f5a8ce61700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880ff3c0 0x7f5a88196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42090/0 (socket says 172.21.15.88:42090) 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.472+0000 7f5a8ce61700 1 -- 172.21.15.88:0/1687194230 learned_addr learned my addr 172.21.15.88:0/1687194230 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.473+0000 7f5a87fff700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a880ffd60 0x7f5a88196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:45.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.473+0000 7f5a8ce61700 1 -- 172.21.15.88:0/1687194230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a880ffd60 msgr2=0x7f5a88196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.473+0000 7f5a8ce61700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a880ffd60 0x7f5a88196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.473+0000 7f5a8ce61700 1 -- 172.21.15.88:0/1687194230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c0095f0 con 0x7f5a880ff3c0 2024-02-19T19:37:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.473+0000 7f5a87fff700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a880ffd60 0x7f5a88196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:37:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.473+0000 7f5a8ce61700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880ff3c0 0x7f5a88196000 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000ba50 tx=0x7f5a8000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.473+0000 7f5a85ffb700 1 -- 172.21.15.88:0/1687194230 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a8000ca00 con 0x7f5a880ff3c0 2024-02-19T19:37:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.473+0000 7f5a85ffb700 1 -- 172.21.15.88:0/1687194230 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a8000cb60 con 0x7f5a880ff3c0 2024-02-19T19:37:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.473+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/1687194230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a8819ac90 con 0x7f5a880ff3c0 2024-02-19T19:37:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.473+0000 7f5a85ffb700 1 -- 172.21.15.88:0/1687194230 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a800075a0 con 0x7f5a880ff3c0 2024-02-19T19:37:45.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.473+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/1687194230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a8819b1e0 con 0x7f5a880ff3c0 2024-02-19T19:37:45.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.474+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/1687194230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a8804f2a0 con 0x7f5a880ff3c0 2024-02-19T19:37:45.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.475+0000 7f5a85ffb700 1 -- 172.21.15.88:0/1687194230 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5a8000ccd0 con 0x7f5a880ff3c0 2024-02-19T19:37:45.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.475+0000 7f5a85ffb700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a7006d8a0 0x7f5a7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:45.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.475+0000 7f5a85ffb700 1 -- 172.21.15.88:0/1687194230 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5a80006e40 con 0x7f5a880ff3c0 2024-02-19T19:37:45.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.475+0000 7f5a87fff700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a7006d8a0 0x7f5a7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:45.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.476+0000 7f5a87fff700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a7006d8a0 0x7f5a7006fd50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00b5c0 tx=0x7f5a7c005480 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:45.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.480+0000 7f5a85ffb700 1 -- 172.21.15.88:0/1687194230 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a80056570 con 0x7f5a880ff3c0 2024-02-19T19:37:45.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.695+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/1687194230 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a8819b4d0 con 0x7f5a7006d8a0 2024-02-19T19:37:45.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.696+0000 7f5a85ffb700 1 -- 172.21.15.88:0/1687194230 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5a8819b4d0 con 0x7f5a7006d8a0 2024-02-19T19:37:45.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.699+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/1687194230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a7006d8a0 msgr2=0x7f5a7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:45.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.699+0000 7f5a8f0c5700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a7006d8a0 0x7f5a7006fd50 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c00b5c0 tx=0x7f5a7c005480 comp rx=0 tx=0).stop 2024-02-19T19:37:45.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.699+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/1687194230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880ff3c0 msgr2=0x7f5a88196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:45.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.699+0000 7f5a8f0c5700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880ff3c0 0x7f5a88196000 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000ba50 tx=0x7f5a8000be10 comp rx=0 tx=0).stop 2024-02-19T19:37:45.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.699+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/1687194230 shutdown_connections 2024-02-19T19:37:45.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.699+0000 7f5a8f0c5700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5a7006d8a0 0x7f5a7006fd50 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:45.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.699+0000 7f5a8f0c5700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a880ff3c0 0x7f5a88196000 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:45.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.699+0000 7f5a8f0c5700 1 --2- 172.21.15.88:0/1687194230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5a880ffd60 0x7f5a88196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:45.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.699+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/1687194230 >> 172.21.15.88:0/1687194230 conn(0x7f5a880fac80 msgr2=0x7f5a881034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:45.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.699+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/1687194230 shutdown_connections 2024-02-19T19:37:45.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.699+0000 7f5a8f0c5700 1 -- 172.21.15.88:0/1687194230 wait complete. 2024-02-19T19:37:45.710 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:37:45.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.804+0000 7faa51752700 1 -- 172.21.15.88:0/1914595981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa4c1004e0 msgr2=0x7faa4c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:45.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.804+0000 7faa51752700 1 --2- 172.21.15.88:0/1914595981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa4c1004e0 0x7faa4c1008b0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009910 tx=0x7faa3c009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:45.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.804+0000 7faa51752700 1 -- 172.21.15.88:0/1914595981 shutdown_connections 2024-02-19T19:37:45.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.804+0000 7faa51752700 1 --2- 172.21.15.88:0/1914595981 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa4c100e80 0x7faa4c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.804+0000 7faa51752700 1 --2- 172.21.15.88:0/1914595981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa4c1004e0 0x7faa4c1008b0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.804+0000 7faa51752700 1 -- 172.21.15.88:0/1914595981 >> 172.21.15.88:0/1914595981 conn(0x7faa4c0fbda0 msgr2=0x7faa4c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.804+0000 7faa51752700 1 -- 172.21.15.88:0/1914595981 shutdown_connections 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.804+0000 7faa51752700 1 -- 172.21.15.88:0/1914595981 wait complete. 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.805+0000 7faa51752700 1 Processor -- start 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.805+0000 7faa51752700 1 -- start start 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.805+0000 7faa51752700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa4c1004e0 0x7faa4c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.805+0000 7faa51752700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa4c100e80 0x7faa4c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.805+0000 7faa51752700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c196b10 con 0x7faa4c1004e0 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.805+0000 7faa51752700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c19a8a0 con 0x7faa4c100e80 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.805+0000 7faa4affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa4c1004e0 0x7faa4c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa4affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa4c1004e0 0x7faa4c195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42106/0 (socket says 172.21.15.88:42106) 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa4affd700 1 -- 172.21.15.88:0/3356922350 learned_addr learned my addr 172.21.15.88:0/3356922350 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:45.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa4a7fc700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa4c100e80 0x7faa4c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:45.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa4affd700 1 -- 172.21.15.88:0/3356922350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa4c100e80 msgr2=0x7faa4c196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:45.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa4affd700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa4c100e80 0x7faa4c196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:45.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa4affd700 1 -- 172.21.15.88:0/3356922350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa3c0095f0 con 0x7faa4c1004e0 2024-02-19T19:37:45.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa4a7fc700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa4c100e80 0x7faa4c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:37:45.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa4affd700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa4c1004e0 0x7faa4c195ef0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faa3c006010 tx=0x7faa3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:45.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa33fff700 1 -- 172.21.15.88:0/3356922350 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa3c01d070 con 0x7faa4c1004e0 2024-02-19T19:37:45.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa33fff700 1 -- 172.21.15.88:0/3356922350 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa3c004fb0 con 0x7faa4c1004e0 2024-02-19T19:37:45.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa51752700 1 -- 172.21.15.88:0/3356922350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4c19ab20 con 0x7faa4c1004e0 2024-02-19T19:37:45.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa33fff700 1 -- 172.21.15.88:0/3356922350 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa3c022ea0 con 0x7faa4c1004e0 2024-02-19T19:37:45.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.806+0000 7faa51752700 1 -- 172.21.15.88:0/3356922350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4c19b010 con 0x7faa4c1004e0 2024-02-19T19:37:45.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.807+0000 7faa51752700 1 -- 172.21.15.88:0/3356922350 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4c1087f0 con 0x7faa4c1004e0 2024-02-19T19:37:45.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.808+0000 7faa33fff700 1 -- 172.21.15.88:0/3356922350 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7faa3c022470 con 0x7faa4c1004e0 2024-02-19T19:37:45.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.808+0000 7faa33fff700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa3406d8a0 0x7faa3406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:45.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.808+0000 7faa33fff700 1 -- 172.21.15.88:0/3356922350 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faa3c08e9c0 con 0x7faa4c1004e0 2024-02-19T19:37:45.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.808+0000 7faa4a7fc700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa3406d8a0 0x7faa3406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:45.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.809+0000 7faa4a7fc700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa3406d8a0 0x7faa3406fd50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7faa4c197510 tx=0x7faa40005f50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:45.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:45.813+0000 7faa33fff700 1 -- 172.21.15.88:0/3356922350 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7faa3c0582a0 con 0x7faa4c1004e0 2024-02-19T19:37:46.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.030+0000 7faa51752700 1 -- 172.21.15.88:0/3356922350 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa4c19b340 con 0x7faa3406d8a0 2024-02-19T19:37:46.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.031+0000 7faa33fff700 1 -- 172.21.15.88:0/3356922350 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7faa4c19b340 con 0x7faa3406d8a0 2024-02-19T19:37:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.033+0000 7faa51752700 1 -- 172.21.15.88:0/3356922350 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa3406d8a0 msgr2=0x7faa3406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.033+0000 7faa51752700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa3406d8a0 0x7faa3406fd50 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7faa4c197510 tx=0x7faa40005f50 comp rx=0 tx=0).stop 2024-02-19T19:37:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.033+0000 7faa51752700 1 -- 172.21.15.88:0/3356922350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa4c1004e0 msgr2=0x7faa4c195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.033+0000 7faa51752700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa4c1004e0 0x7faa4c195ef0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faa3c006010 tx=0x7faa3c00be20 comp rx=0 tx=0).stop 2024-02-19T19:37:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.033+0000 7faa51752700 1 -- 172.21.15.88:0/3356922350 shutdown_connections 2024-02-19T19:37:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.033+0000 7faa51752700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7faa3406d8a0 0x7faa3406fd50 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.033+0000 7faa51752700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa4c1004e0 0x7faa4c195ef0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.033+0000 7faa51752700 1 --2- 172.21.15.88:0/3356922350 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7faa4c100e80 0x7faa4c196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.033+0000 7faa51752700 1 -- 172.21.15.88:0/3356922350 >> 172.21.15.88:0/3356922350 conn(0x7faa4c0fbda0 msgr2=0x7faa4c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.033+0000 7faa51752700 1 -- 172.21.15.88:0/3356922350 shutdown_connections 2024-02-19T19:37:46.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.033+0000 7faa51752700 1 -- 172.21.15.88:0/3356922350 wait complete. 2024-02-19T19:37:46.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:45 smithi088 ceph-mon[88643]: from='client.35846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.138+0000 7f375de50700 1 -- 172.21.15.88:0/3273337653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3758069140 msgr2=0x7f3758104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.138+0000 7f375de50700 1 --2- 172.21.15.88:0/3273337653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3758069140 0x7f3758104eb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3740009910 tx=0x7f3740009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.138+0000 7f375de50700 1 -- 172.21.15.88:0/3273337653 shutdown_connections 2024-02-19T19:37:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.138+0000 7f375de50700 1 --2- 172.21.15.88:0/3273337653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3758069140 0x7f3758104eb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.138+0000 7f375de50700 1 --2- 172.21.15.88:0/3273337653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37580687a0 0x7f3758068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.138+0000 7f375de50700 1 -- 172.21.15.88:0/3273337653 >> 172.21.15.88:0/3273337653 conn(0x7f3758075130 msgr2=0x7f3758075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:46.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.138+0000 7f375de50700 1 -- 172.21.15.88:0/3273337653 shutdown_connections 2024-02-19T19:37:46.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.138+0000 7f375de50700 1 -- 172.21.15.88:0/3273337653 wait complete. 2024-02-19T19:37:46.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.139+0000 7f375de50700 1 Processor -- start 2024-02-19T19:37:46.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.139+0000 7f375de50700 1 -- start start 2024-02-19T19:37:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.139+0000 7f375de50700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37580687a0 0x7f3758195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.139+0000 7f375de50700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3758069140 0x7f37581963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.139+0000 7f375de50700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3758196a80 con 0x7f3758069140 2024-02-19T19:37:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.139+0000 7f375de50700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375819a810 con 0x7f37580687a0 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.139+0000 7f3756ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3758069140 0x7f37581963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.139+0000 7f3756ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3758069140 0x7f37581963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42120/0 (socket says 172.21.15.88:42120) 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.139+0000 7f3756ffd700 1 -- 172.21.15.88:0/1463258662 learned_addr learned my addr 172.21.15.88:0/1463258662 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.139+0000 7f37577fe700 1 --2- 172.21.15.88:0/1463258662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37580687a0 0x7f3758195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.140+0000 7f3756ffd700 1 -- 172.21.15.88:0/1463258662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37580687a0 msgr2=0x7f3758195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.140+0000 7f3756ffd700 1 --2- 172.21.15.88:0/1463258662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37580687a0 0x7f3758195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.140+0000 7f3756ffd700 1 -- 172.21.15.88:0/1463258662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37400095f0 con 0x7f3758069140 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.140+0000 7f3756ffd700 1 --2- 172.21.15.88:0/1463258662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3758069140 0x7f37581963a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f374000bd70 tx=0x7f374000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.140+0000 7f3754ff9700 1 -- 172.21.15.88:0/1463258662 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f374001d070 con 0x7f3758069140 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.140+0000 7f3754ff9700 1 -- 172.21.15.88:0/1463258662 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3740005dc0 con 0x7f3758069140 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.140+0000 7f375de50700 1 -- 172.21.15.88:0/1463258662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375819aa90 con 0x7f3758069140 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.140+0000 7f3754ff9700 1 -- 172.21.15.88:0/1463258662 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3740017400 con 0x7f3758069140 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.140+0000 7f375de50700 1 -- 172.21.15.88:0/1463258662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375819af80 con 0x7f3758069140 2024-02-19T19:37:46.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.141+0000 7f375de50700 1 -- 172.21.15.88:0/1463258662 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3758108860 con 0x7f3758069140 2024-02-19T19:37:46.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.141+0000 7f3754ff9700 1 -- 172.21.15.88:0/1463258662 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f374000f7f0 con 0x7f3758069140 2024-02-19T19:37:46.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.142+0000 7f3754ff9700 1 --2- 172.21.15.88:0/1463258662 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f374406d8a0 0x7f374406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:46.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.142+0000 7f3754ff9700 1 -- 172.21.15.88:0/1463258662 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f374008e960 con 0x7f3758069140 2024-02-19T19:37:46.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.142+0000 7f37577fe700 1 --2- 172.21.15.88:0/1463258662 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f374406d8a0 0x7f374406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:46.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.142+0000 7f37577fe700 1 --2- 172.21.15.88:0/1463258662 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f374406d8a0 0x7f374406fd50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3758073860 tx=0x7f3750009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:46.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.146+0000 7f3754ff9700 1 -- 172.21.15.88:0/1463258662 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3740058190 con 0x7f3758069140 2024-02-19T19:37:46.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:45 smithi132 ceph-mon[70676]: from='client.35846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:46.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.354+0000 7f375de50700 1 -- 172.21.15.88:0/1463258662 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f375802cca0 con 0x7f374406d8a0 2024-02-19T19:37:46.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.365+0000 7f3754ff9700 1 -- 172.21.15.88:0/1463258662 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f375802cca0 con 0x7f374406d8a0 2024-02-19T19:37:46.366 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:37:46.366 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (34m) 11m ago 45m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:37:46.366 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (45m) 11m ago 45m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:37:46.366 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (27m) 11m ago 43m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (31m) 11m ago 45m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (30m) 11m ago 43m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (34m) 11m ago 44m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (27m) 11m ago 39m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (26m) 11m ago 39m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (26m) 11m ago 39m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (11m) 11m ago 38m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (36m) 11m ago 47m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (35m) 11m ago 43m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (33m) 11m ago 47m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (32m) 11m ago 43m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (35m) 11m ago 45m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (35m) 11m ago 43m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:37:46.367 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (30m) 11m ago 42m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:37:46.368 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (29m) 11m ago 42m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:37:46.368 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (29m) 11m ago 41m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:37:46.368 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (28m) 11m ago 41m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:37:46.368 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (28m) 11m ago 41m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:37:46.368 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (27m) 11m ago 40m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:37:46.368 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (35m) 11m ago 44m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:37:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.367+0000 7f375de50700 1 -- 172.21.15.88:0/1463258662 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f374406d8a0 msgr2=0x7f374406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.367+0000 7f375de50700 1 --2- 172.21.15.88:0/1463258662 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f374406d8a0 0x7f374406fd50 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f3758073860 tx=0x7f3750009040 comp rx=0 tx=0).stop 2024-02-19T19:37:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.367+0000 7f375de50700 1 -- 172.21.15.88:0/1463258662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3758069140 msgr2=0x7f37581963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.367+0000 7f375de50700 1 --2- 172.21.15.88:0/1463258662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3758069140 0x7f37581963a0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f374000bd70 tx=0x7f374000be50 comp rx=0 tx=0).stop 2024-02-19T19:37:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.367+0000 7f375de50700 1 -- 172.21.15.88:0/1463258662 shutdown_connections 2024-02-19T19:37:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.367+0000 7f375de50700 1 --2- 172.21.15.88:0/1463258662 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f374406d8a0 0x7f374406fd50 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.367+0000 7f375de50700 1 --2- 172.21.15.88:0/1463258662 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f37580687a0 0x7f3758195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.367+0000 7f375de50700 1 --2- 172.21.15.88:0/1463258662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3758069140 0x7f37581963a0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.367+0000 7f375de50700 1 -- 172.21.15.88:0/1463258662 >> 172.21.15.88:0/1463258662 conn(0x7f3758075130 msgr2=0x7f37580ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.367+0000 7f375de50700 1 -- 172.21.15.88:0/1463258662 shutdown_connections 2024-02-19T19:37:46.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.367+0000 7f375de50700 1 -- 172.21.15.88:0/1463258662 wait complete. 2024-02-19T19:37:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.470+0000 7ff43baff700 1 -- 172.21.15.88:0/3683988139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4341004e0 msgr2=0x7ff4341008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.470+0000 7ff43baff700 1 --2- 172.21.15.88:0/3683988139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4341004e0 0x7ff4341008b0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009910 tx=0x7ff42c009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.470+0000 7ff43baff700 1 -- 172.21.15.88:0/3683988139 shutdown_connections 2024-02-19T19:37:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.470+0000 7ff43baff700 1 --2- 172.21.15.88:0/3683988139 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff434100e80 0x7ff434104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.470+0000 7ff43baff700 1 --2- 172.21.15.88:0/3683988139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4341004e0 0x7ff4341008b0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.470+0000 7ff43baff700 1 -- 172.21.15.88:0/3683988139 >> 172.21.15.88:0/3683988139 conn(0x7ff4340fbda0 msgr2=0x7ff4340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.470+0000 7ff43baff700 1 -- 172.21.15.88:0/3683988139 shutdown_connections 2024-02-19T19:37:46.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.470+0000 7ff43baff700 1 -- 172.21.15.88:0/3683988139 wait complete. 2024-02-19T19:37:46.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.471+0000 7ff43baff700 1 Processor -- start 2024-02-19T19:37:46.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.471+0000 7ff43baff700 1 -- start start 2024-02-19T19:37:46.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.471+0000 7ff43baff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4341004e0 0x7ff434195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:46.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.471+0000 7ff43baff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff434100e80 0x7ff434196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:46.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.471+0000 7ff43baff700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff434196b10 con 0x7ff4341004e0 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.471+0000 7ff43baff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43419a8a0 con 0x7ff434100e80 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.471+0000 7ff43989b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4341004e0 0x7ff434195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.471+0000 7ff43989b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4341004e0 0x7ff434195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42124/0 (socket says 172.21.15.88:42124) 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.471+0000 7ff43989b700 1 -- 172.21.15.88:0/3001529351 learned_addr learned my addr 172.21.15.88:0/3001529351 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.472+0000 7ff43909a700 1 --2- 172.21.15.88:0/3001529351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff434100e80 0x7ff434196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.472+0000 7ff43989b700 1 -- 172.21.15.88:0/3001529351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff434100e80 msgr2=0x7ff434196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.472+0000 7ff43989b700 1 --2- 172.21.15.88:0/3001529351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff434100e80 0x7ff434196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.472+0000 7ff43989b700 1 -- 172.21.15.88:0/3001529351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42c0095f0 con 0x7ff4341004e0 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.472+0000 7ff43989b700 1 --2- 172.21.15.88:0/3001529351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4341004e0 0x7ff434195ef0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff42c006010 tx=0x7ff42c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.472+0000 7ff42affd700 1 -- 172.21.15.88:0/3001529351 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff42c01d070 con 0x7ff4341004e0 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.472+0000 7ff42affd700 1 -- 172.21.15.88:0/3001529351 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff42c004fb0 con 0x7ff4341004e0 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.472+0000 7ff43baff700 1 -- 172.21.15.88:0/3001529351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff43419ab20 con 0x7ff4341004e0 2024-02-19T19:37:46.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.472+0000 7ff42affd700 1 -- 172.21.15.88:0/3001529351 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff42c022ea0 con 0x7ff4341004e0 2024-02-19T19:37:46.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.472+0000 7ff43baff700 1 -- 172.21.15.88:0/3001529351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43419b010 con 0x7ff4341004e0 2024-02-19T19:37:46.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.473+0000 7ff43baff700 1 -- 172.21.15.88:0/3001529351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4341087f0 con 0x7ff4341004e0 2024-02-19T19:37:46.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.474+0000 7ff42affd700 1 -- 172.21.15.88:0/3001529351 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff42c022470 con 0x7ff4341004e0 2024-02-19T19:37:46.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.474+0000 7ff42affd700 1 --2- 172.21.15.88:0/3001529351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff42006d8a0 0x7ff42006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:46.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.474+0000 7ff43909a700 1 --2- 172.21.15.88:0/3001529351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff42006d8a0 0x7ff42006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:46.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.474+0000 7ff42affd700 1 -- 172.21.15.88:0/3001529351 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff42c08ebb0 con 0x7ff4341004e0 2024-02-19T19:37:46.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.475+0000 7ff43909a700 1 --2- 172.21.15.88:0/3001529351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff42006d8a0 0x7ff42006fd50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff424005fd0 tx=0x7ff42400a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:46.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.479+0000 7ff42affd700 1 -- 172.21.15.88:0/3001529351 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff42c095080 con 0x7ff4341004e0 2024-02-19T19:37:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.758+0000 7ff43baff700 1 -- 172.21.15.88:0/3001529351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff434197360 con 0x7ff4341004e0 2024-02-19T19:37:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.758+0000 7ff42affd700 1 -- 172.21.15.88:0/3001529351 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7ff42c027280 con 0x7ff4341004e0 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:37:46.761 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:37:46.762 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:37:46.762 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:37:46.762 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:37:46.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.761+0000 7ff43baff700 1 -- 172.21.15.88:0/3001529351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff42006d8a0 msgr2=0x7ff42006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.761+0000 7ff43baff700 1 --2- 172.21.15.88:0/3001529351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff42006d8a0 0x7ff42006fd50 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff424005fd0 tx=0x7ff42400a560 comp rx=0 tx=0).stop 2024-02-19T19:37:46.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.761+0000 7ff43baff700 1 -- 172.21.15.88:0/3001529351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4341004e0 msgr2=0x7ff434195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.761+0000 7ff43baff700 1 --2- 172.21.15.88:0/3001529351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4341004e0 0x7ff434195ef0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7ff42c006010 tx=0x7ff42c00be20 comp rx=0 tx=0).stop 2024-02-19T19:37:46.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.761+0000 7ff43baff700 1 -- 172.21.15.88:0/3001529351 shutdown_connections 2024-02-19T19:37:46.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.761+0000 7ff43baff700 1 --2- 172.21.15.88:0/3001529351 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff42006d8a0 0x7ff42006fd50 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.761+0000 7ff43baff700 1 --2- 172.21.15.88:0/3001529351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4341004e0 0x7ff434195ef0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.761+0000 7ff43baff700 1 --2- 172.21.15.88:0/3001529351 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff434100e80 0x7ff434196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.761+0000 7ff43baff700 1 -- 172.21.15.88:0/3001529351 >> 172.21.15.88:0/3001529351 conn(0x7ff4340fbda0 msgr2=0x7ff4340fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:46.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.761+0000 7ff43baff700 1 -- 172.21.15.88:0/3001529351 shutdown_connections 2024-02-19T19:37:46.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.761+0000 7ff43baff700 1 -- 172.21.15.88:0/3001529351 wait complete. 2024-02-19T19:37:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.865+0000 7ff9c272c700 1 -- 172.21.15.88:0/3726845344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9bc0ffc60 msgr2=0x7ff9bc10a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.865+0000 7ff9c272c700 1 --2- 172.21.15.88:0/3726845344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9bc0ffc60 0x7ff9bc10a6d0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0009910 tx=0x7ff9b0009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:46.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.865+0000 7ff9c272c700 1 -- 172.21.15.88:0/3726845344 shutdown_connections 2024-02-19T19:37:46.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.865+0000 7ff9c272c700 1 --2- 172.21.15.88:0/3726845344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9bc0ffc60 0x7ff9bc10a6d0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.865+0000 7ff9c272c700 1 --2- 172.21.15.88:0/3726845344 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9bc0ff350 0x7ff9bc0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.865+0000 7ff9c272c700 1 -- 172.21.15.88:0/3726845344 >> 172.21.15.88:0/3726845344 conn(0x7ff9bc0748b0 msgr2=0x7ff9bc074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:46.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.865+0000 7ff9c272c700 1 -- 172.21.15.88:0/3726845344 shutdown_connections 2024-02-19T19:37:46.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.865+0000 7ff9c272c700 1 -- 172.21.15.88:0/3726845344 wait complete. 2024-02-19T19:37:46.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.865+0000 7ff9c272c700 1 Processor -- start 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.865+0000 7ff9c272c700 1 -- start start 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9c272c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9bc0ff350 0x7ff9bc195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9c272c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9bc0ffc60 0x7ff9bc196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9c272c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc196b00 con 0x7ff9bc0ffc60 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9c272c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc19a890 con 0x7ff9bc0ff350 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9bb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9bc0ffc60 0x7ff9bc196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9bb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9bc0ffc60 0x7ff9bc196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42136/0 (socket says 172.21.15.88:42136) 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9bb7fe700 1 -- 172.21.15.88:0/3371360806 learned_addr learned my addr 172.21.15.88:0/3371360806 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9bbfff700 1 --2- 172.21.15.88:0/3371360806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9bc0ff350 0x7ff9bc195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9bb7fe700 1 -- 172.21.15.88:0/3371360806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9bc0ff350 msgr2=0x7ff9bc195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9bb7fe700 1 --2- 172.21.15.88:0/3371360806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9bc0ff350 0x7ff9bc195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9bb7fe700 1 -- 172.21.15.88:0/3371360806 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00095f0 con 0x7ff9bc0ffc60 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.866+0000 7ff9bb7fe700 1 --2- 172.21.15.88:0/3371360806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9bc0ffc60 0x7ff9bc196420 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005f50 tx=0x7ff9b000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:46.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.867+0000 7ff9b97fa700 1 -- 172.21.15.88:0/3371360806 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9b001d070 con 0x7ff9bc0ffc60 2024-02-19T19:37:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.867+0000 7ff9b97fa700 1 -- 172.21.15.88:0/3371360806 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff9b000b610 con 0x7ff9bc0ffc60 2024-02-19T19:37:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.867+0000 7ff9c272c700 1 -- 172.21.15.88:0/3371360806 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc19ab10 con 0x7ff9bc0ffc60 2024-02-19T19:37:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.867+0000 7ff9b97fa700 1 -- 172.21.15.88:0/3371360806 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9b0022470 con 0x7ff9bc0ffc60 2024-02-19T19:37:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.867+0000 7ff9c272c700 1 -- 172.21.15.88:0/3371360806 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc19b000 con 0x7ff9bc0ffc60 2024-02-19T19:37:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.868+0000 7ff9c272c700 1 -- 172.21.15.88:0/3371360806 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc107dd0 con 0x7ff9bc0ffc60 2024-02-19T19:37:46.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.868+0000 7ff9b97fa700 1 -- 172.21.15.88:0/3371360806 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ff9b0003780 con 0x7ff9bc0ffc60 2024-02-19T19:37:46.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.869+0000 7ff9b97fa700 1 --2- 172.21.15.88:0/3371360806 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff9a406d8a0 0x7ff9a406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:46.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.869+0000 7ff9b97fa700 1 -- 172.21.15.88:0/3371360806 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff9b008e900 con 0x7ff9bc0ffc60 2024-02-19T19:37:46.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.869+0000 7ff9bbfff700 1 --2- 172.21.15.88:0/3371360806 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff9a406d8a0 0x7ff9a406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:46.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.869+0000 7ff9bbfff700 1 --2- 172.21.15.88:0/3371360806 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff9a406d8a0 0x7ff9a406fd50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc06a640 tx=0x7ff9ac009260 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:46.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:46.874+0000 7ff9b97fa700 1 -- 172.21.15.88:0/3371360806 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff9b00580b0 con 0x7ff9bc0ffc60 2024-02-19T19:37:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:46 smithi088 ceph-mon[88643]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:46 smithi088 ceph-mon[88643]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:47.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:46 smithi088 ceph-mon[88643]: pgmap v1027: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:47.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:46 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3001529351' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:37:47.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.115+0000 7ff9c272c700 1 -- 172.21.15.88:0/3371360806 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff9bc1077b0 con 0x7ff9bc0ffc60 2024-02-19T19:37:47.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.115+0000 7ff9b97fa700 1 -- 172.21.15.88:0/3371360806 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7ff9b005b770 con 0x7ff9bc0ffc60 2024-02-19T19:37:47.117 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:37:47.117 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:37:47.117 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:37:47.117 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:37:47.117 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:37:47.117 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:37:47.117 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:37:47.117 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:37:47.117 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:37:47.117 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:37:47.117 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:37:47.118 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:37:47.119 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:37:47.120 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:37:47.120 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.117+0000 7ff9c272c700 1 -- 172.21.15.88:0/3371360806 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff9a406d8a0 msgr2=0x7ff9a406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.117+0000 7ff9c272c700 1 --2- 172.21.15.88:0/3371360806 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff9a406d8a0 0x7ff9a406fd50 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc06a640 tx=0x7ff9ac009260 comp rx=0 tx=0).stop 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.117+0000 7ff9c272c700 1 -- 172.21.15.88:0/3371360806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9bc0ffc60 msgr2=0x7ff9bc196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.117+0000 7ff9c272c700 1 --2- 172.21.15.88:0/3371360806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9bc0ffc60 0x7ff9bc196420 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0005f50 tx=0x7ff9b000f690 comp rx=0 tx=0).stop 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.117+0000 7ff9c272c700 1 -- 172.21.15.88:0/3371360806 shutdown_connections 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.117+0000 7ff9c272c700 1 --2- 172.21.15.88:0/3371360806 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ff9a406d8a0 0x7ff9a406fd50 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.118+0000 7ff9c272c700 1 --2- 172.21.15.88:0/3371360806 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ff9bc0ff350 0x7ff9bc195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.118+0000 7ff9c272c700 1 --2- 172.21.15.88:0/3371360806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9bc0ffc60 0x7ff9bc196420 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.118+0000 7ff9c272c700 1 -- 172.21.15.88:0/3371360806 >> 172.21.15.88:0/3371360806 conn(0x7ff9bc0748b0 msgr2=0x7ff9bc104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.118+0000 7ff9c272c700 1 -- 172.21.15.88:0/3371360806 shutdown_connections 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.118+0000 7ff9c272c700 1 -- 172.21.15.88:0/3371360806 wait complete. 2024-02-19T19:37:47.121 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:37:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.222+0000 7f4955a1f700 1 -- 172.21.15.88:0/1664206113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500ff3c0 msgr2=0x7f49500ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.222+0000 7f4955a1f700 1 --2- 172.21.15.88:0/1664206113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500ff3c0 0x7f49500ff790 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f4940009910 tx=0x7f4940009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:47.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.222+0000 7f4955a1f700 1 -- 172.21.15.88:0/1664206113 shutdown_connections 2024-02-19T19:37:47.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.222+0000 7f4955a1f700 1 --2- 172.21.15.88:0/1664206113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49500ffd60 0x7f4950103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.222+0000 7f4955a1f700 1 --2- 172.21.15.88:0/1664206113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500ff3c0 0x7f49500ff790 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.222+0000 7f4955a1f700 1 -- 172.21.15.88:0/1664206113 >> 172.21.15.88:0/1664206113 conn(0x7f49500fac80 msgr2=0x7f49500fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:47.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.222+0000 7f4955a1f700 1 -- 172.21.15.88:0/1664206113 shutdown_connections 2024-02-19T19:37:47.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.222+0000 7f4955a1f700 1 -- 172.21.15.88:0/1664206113 wait complete. 2024-02-19T19:37:47.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.223+0000 7f4955a1f700 1 Processor -- start 2024-02-19T19:37:47.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.223+0000 7f4955a1f700 1 -- start start 2024-02-19T19:37:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.223+0000 7f4955a1f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49500ff3c0 0x7f4950196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.223+0000 7f4955a1f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500ffd60 0x7f4950197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.223+0000 7f4955a1f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4950197850 con 0x7f49500ffd60 2024-02-19T19:37:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.223+0000 7f4955a1f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4950197990 con 0x7f49500ff3c0 2024-02-19T19:37:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.223+0000 7f494e7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500ffd60 0x7f4950197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.223+0000 7f494e7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500ffd60 0x7f4950197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42148/0 (socket says 172.21.15.88:42148) 2024-02-19T19:37:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.223+0000 7f494e7fc700 1 -- 172.21.15.88:0/1409298297 learned_addr learned my addr 172.21.15.88:0/1409298297 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.224+0000 7f494effd700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49500ff3c0 0x7f4950196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.224+0000 7f494e7fc700 1 -- 172.21.15.88:0/1409298297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49500ff3c0 msgr2=0x7f4950196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.224+0000 7f494e7fc700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49500ff3c0 0x7f4950196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.224+0000 7f494e7fc700 1 -- 172.21.15.88:0/1409298297 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49400095f0 con 0x7f49500ffd60 2024-02-19T19:37:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.224+0000 7f494effd700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49500ff3c0 0x7f4950196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:37:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.224+0000 7f494e7fc700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500ffd60 0x7f4950197230 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f494400d900 tx=0x7f494400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.224+0000 7f4954a1d700 1 -- 172.21.15.88:0/1409298297 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4944009900 con 0x7f49500ffd60 2024-02-19T19:37:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.224+0000 7f4955a1f700 1 -- 172.21.15.88:0/1409298297 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f495019fd60 con 0x7f49500ffd60 2024-02-19T19:37:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.224+0000 7f4954a1d700 1 -- 172.21.15.88:0/1409298297 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4944010460 con 0x7f49500ffd60 2024-02-19T19:37:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.224+0000 7f4954a1d700 1 -- 172.21.15.88:0/1409298297 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f494400f5f0 con 0x7f49500ffd60 2024-02-19T19:37:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.224+0000 7f4955a1f700 1 -- 172.21.15.88:0/1409298297 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49501a0220 con 0x7f49500ffd60 2024-02-19T19:37:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.225+0000 7f4955a1f700 1 -- 172.21.15.88:0/1409298297 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49501076d0 con 0x7f49500ffd60 2024-02-19T19:37:47.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.226+0000 7f4954a1d700 1 -- 172.21.15.88:0/1409298297 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f49440105d0 con 0x7f49500ffd60 2024-02-19T19:37:47.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.226+0000 7f4954a1d700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f493806d8a0 0x7f493806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:47.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.226+0000 7f4954a1d700 1 -- 172.21.15.88:0/1409298297 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f494408d380 con 0x7f49500ffd60 2024-02-19T19:37:47.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.226+0000 7f494effd700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f493806d8a0 0x7f493806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:47.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.227+0000 7f494effd700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f493806d8a0 0x7f493806fd50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4940005b40 tx=0x7f494001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:47.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.231+0000 7f4954a1d700 1 -- 172.21.15.88:0/1409298297 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4944056b30 con 0x7f49500ffd60 2024-02-19T19:37:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:46 smithi132 ceph-mon[70676]: from='client.35850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:46 smithi132 ceph-mon[70676]: from='client.35854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:47.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:46 smithi132 ceph-mon[70676]: pgmap v1027: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:47.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:46 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3001529351' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:37:47.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.446+0000 7f4955a1f700 1 -- 172.21.15.88:0/1409298297 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4950061190 con 0x7f493806d8a0 2024-02-19T19:37:47.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.447+0000 7f4954a1d700 1 -- 172.21.15.88:0/1409298297 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4950061190 con 0x7f493806d8a0 2024-02-19T19:37:47.448 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:37:47.448 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:37:47.449 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:37:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.449+0000 7f4955a1f700 1 -- 172.21.15.88:0/1409298297 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f493806d8a0 msgr2=0x7f493806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.449+0000 7f4955a1f700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f493806d8a0 0x7f493806fd50 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f4940005b40 tx=0x7f494001a040 comp rx=0 tx=0).stop 2024-02-19T19:37:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.449+0000 7f4955a1f700 1 -- 172.21.15.88:0/1409298297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500ffd60 msgr2=0x7f4950197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.449+0000 7f4955a1f700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500ffd60 0x7f4950197230 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f494400d900 tx=0x7f494400dcc0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.449+0000 7f4955a1f700 1 -- 172.21.15.88:0/1409298297 shutdown_connections 2024-02-19T19:37:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.449+0000 7f4955a1f700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f493806d8a0 0x7f493806fd50 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.449+0000 7f4955a1f700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f49500ff3c0 0x7f4950196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.449+0000 7f4955a1f700 1 --2- 172.21.15.88:0/1409298297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49500ffd60 0x7f4950197230 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.449+0000 7f4955a1f700 1 -- 172.21.15.88:0/1409298297 >> 172.21.15.88:0/1409298297 conn(0x7f49500fac80 msgr2=0x7f49500fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.449+0000 7f4955a1f700 1 -- 172.21.15.88:0/1409298297 shutdown_connections 2024-02-19T19:37:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.449+0000 7f4955a1f700 1 -- 172.21.15.88:0/1409298297 wait complete. 2024-02-19T19:37:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.552+0000 7fa607546700 1 -- 172.21.15.88:0/1043756353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6000ff3c0 msgr2=0x7fa6000ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.552+0000 7fa607546700 1 --2- 172.21.15.88:0/1043756353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6000ff3c0 0x7fa6000ff790 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009910 tx=0x7fa5f0009c20 comp rx=0 tx=0).stop 2024-02-19T19:37:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.552+0000 7fa607546700 1 -- 172.21.15.88:0/1043756353 shutdown_connections 2024-02-19T19:37:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.552+0000 7fa607546700 1 --2- 172.21.15.88:0/1043756353 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6000ffd60 0x7fa600103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.552+0000 7fa607546700 1 --2- 172.21.15.88:0/1043756353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6000ff3c0 0x7fa6000ff790 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.552+0000 7fa607546700 1 -- 172.21.15.88:0/1043756353 >> 172.21.15.88:0/1043756353 conn(0x7fa6000fac80 msgr2=0x7fa6000fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:47.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.553+0000 7fa607546700 1 -- 172.21.15.88:0/1043756353 shutdown_connections 2024-02-19T19:37:47.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.553+0000 7fa607546700 1 -- 172.21.15.88:0/1043756353 wait complete. 2024-02-19T19:37:47.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.553+0000 7fa607546700 1 Processor -- start 2024-02-19T19:37:47.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.553+0000 7fa607546700 1 -- start start 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa607546700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6000ff3c0 0x7fa600194d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa607546700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6000ffd60 0x7fa6001952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa607546700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001959a0 con 0x7fa6000ff3c0 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa607546700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa600199730 con 0x7fa6000ffd60 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa6052e2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6000ff3c0 0x7fa600194d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa6052e2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6000ff3c0 0x7fa600194d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42176/0 (socket says 172.21.15.88:42176) 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa6052e2700 1 -- 172.21.15.88:0/2437097312 learned_addr learned my addr 172.21.15.88:0/2437097312 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa604ae1700 1 --2- 172.21.15.88:0/2437097312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6000ffd60 0x7fa6001952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa6052e2700 1 -- 172.21.15.88:0/2437097312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6000ffd60 msgr2=0x7fa6001952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa6052e2700 1 --2- 172.21.15.88:0/2437097312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6000ffd60 0x7fa6001952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa6052e2700 1 -- 172.21.15.88:0/2437097312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f00095f0 con 0x7fa6000ff3c0 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa6052e2700 1 --2- 172.21.15.88:0/2437097312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6000ff3c0 0x7fa600194d80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0006010 tx=0x7fa5f000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:47.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.554+0000 7fa5f67fc700 1 -- 172.21.15.88:0/2437097312 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5f001d070 con 0x7fa6000ff3c0 2024-02-19T19:37:47.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.555+0000 7fa5f67fc700 1 -- 172.21.15.88:0/2437097312 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5f0004fb0 con 0x7fa6000ff3c0 2024-02-19T19:37:47.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.555+0000 7fa5f67fc700 1 -- 172.21.15.88:0/2437097312 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5f0022ea0 con 0x7fa6000ff3c0 2024-02-19T19:37:47.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.555+0000 7fa607546700 1 -- 172.21.15.88:0/2437097312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6001999b0 con 0x7fa6000ff3c0 2024-02-19T19:37:47.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.555+0000 7fa607546700 1 -- 172.21.15.88:0/2437097312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa600199ea0 con 0x7fa6000ff3c0 2024-02-19T19:37:47.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.556+0000 7fa607546700 1 -- 172.21.15.88:0/2437097312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6001076d0 con 0x7fa6000ff3c0 2024-02-19T19:37:47.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.557+0000 7fa5f67fc700 1 -- 172.21.15.88:0/2437097312 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa5f0022470 con 0x7fa6000ff3c0 2024-02-19T19:37:47.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.557+0000 7fa5f67fc700 1 --2- 172.21.15.88:0/2437097312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa5ec06d8a0 0x7fa5ec06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:37:47.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.557+0000 7fa5f67fc700 1 -- 172.21.15.88:0/2437097312 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa5f008e9c0 con 0x7fa6000ff3c0 2024-02-19T19:37:47.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.557+0000 7fa604ae1700 1 --2- 172.21.15.88:0/2437097312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa5ec06d8a0 0x7fa5ec06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:37:47.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.558+0000 7fa604ae1700 1 --2- 172.21.15.88:0/2437097312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa5ec06d8a0 0x7fa5ec06fd50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc005fd0 tx=0x7fa5fc00a560 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:37:47.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.562+0000 7fa5f67fc700 1 -- 172.21.15.88:0/2437097312 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa5f00582a0 con 0x7fa6000ff3c0 2024-02-19T19:37:47.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.841+0000 7fa607546700 1 -- 172.21.15.88:0/2437097312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa60019a1d0 con 0x7fa6000ff3c0 2024-02-19T19:37:47.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.841+0000 7fa5f67fc700 1 -- 172.21.15.88:0/2437097312 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7fa5f005b960 con 0x7fa6000ff3c0 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:37:47.843 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:37:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:37:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:37:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:37:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:37:47.844 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:37:47.844 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:37:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:37:47.844 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:37:47.844 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:37:47.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.843+0000 7fa607546700 1 -- 172.21.15.88:0/2437097312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa5ec06d8a0 msgr2=0x7fa5ec06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:47.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.843+0000 7fa607546700 1 --2- 172.21.15.88:0/2437097312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa5ec06d8a0 0x7fa5ec06fd50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc005fd0 tx=0x7fa5fc00a560 comp rx=0 tx=0).stop 2024-02-19T19:37:47.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.843+0000 7fa607546700 1 -- 172.21.15.88:0/2437097312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6000ff3c0 msgr2=0x7fa600194d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:37:47.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.843+0000 7fa607546700 1 --2- 172.21.15.88:0/2437097312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6000ff3c0 0x7fa600194d80 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0006010 tx=0x7fa5f000be20 comp rx=0 tx=0).stop 2024-02-19T19:37:47.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.843+0000 7fa607546700 1 -- 172.21.15.88:0/2437097312 shutdown_connections 2024-02-19T19:37:47.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.843+0000 7fa607546700 1 --2- 172.21.15.88:0/2437097312 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa5ec06d8a0 0x7fa5ec06fd50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.843+0000 7fa607546700 1 --2- 172.21.15.88:0/2437097312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6000ff3c0 0x7fa600194d80 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.843+0000 7fa607546700 1 --2- 172.21.15.88:0/2437097312 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa6000ffd60 0x7fa6001952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:37:47.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.843+0000 7fa607546700 1 -- 172.21.15.88:0/2437097312 >> 172.21.15.88:0/2437097312 conn(0x7fa6000fac80 msgr2=0x7fa6000fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:37:47.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.843+0000 7fa607546700 1 -- 172.21.15.88:0/2437097312 shutdown_connections 2024-02-19T19:37:47.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:37:47.843+0000 7fa607546700 1 -- 172.21.15.88:0/2437097312 wait complete. 2024-02-19T19:37:48.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:47 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3371360806' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:37:48.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:47 smithi088 ceph-mon[88643]: from='client.35866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:48.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:47 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3371360806' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:37:48.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:47 smithi132 ceph-mon[70676]: from='client.35866 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:37:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:48 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2437097312' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:37:49.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:48 smithi088 ceph-mon[88643]: pgmap v1028: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:49.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:48 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2437097312' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:37:49.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:48 smithi132 ceph-mon[70676]: pgmap v1028: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:50 smithi088 ceph-mon[88643]: pgmap v1029: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:50 smithi132 ceph-mon[70676]: pgmap v1029: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:52 smithi088 ceph-mon[88643]: pgmap v1030: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:52 smithi132 ceph-mon[70676]: pgmap v1030: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:54 smithi088 ceph-mon[88643]: pgmap v1031: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:54 smithi132 ceph-mon[70676]: pgmap v1031: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:56 smithi088 ceph-mon[88643]: pgmap v1032: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:56 smithi132 ceph-mon[70676]: pgmap v1032: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:58.550 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:37:58.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:37:59.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:37:59 smithi088 ceph-mon[88643]: pgmap v1033: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:37:59.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:37:59 smithi132 ceph-mon[70676]: pgmap v1033: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:00 smithi088 ceph-mon[88643]: pgmap v1034: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:00 smithi132 ceph-mon[70676]: pgmap v1034: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:02 smithi088 ceph-mon[88643]: pgmap v1035: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:03.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:02 smithi132 ceph-mon[70676]: pgmap v1035: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:04 smithi088 ceph-mon[88643]: pgmap v1036: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:04 smithi132 ceph-mon[70676]: pgmap v1036: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:06 smithi088 ceph-mon[88643]: pgmap v1037: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:07.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:06 smithi132 ceph-mon[70676]: pgmap v1037: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:08.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:08 smithi132 ceph-mon[70676]: pgmap v1038: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:08 smithi088 ceph-mon[88643]: pgmap v1038: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:10 smithi088 ceph-mon[88643]: pgmap v1039: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:10 smithi132 ceph-mon[70676]: pgmap v1039: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:12 smithi088 ceph-mon[88643]: pgmap v1040: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:13.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:12 smithi132 ceph-mon[70676]: pgmap v1040: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:13.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:38:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:38:14.983 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:14 smithi088 ceph-mon[88643]: pgmap v1041: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:14 smithi132 ceph-mon[70676]: pgmap v1041: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:16 smithi088 ceph-mon[88643]: pgmap v1042: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:17.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:16 smithi132 ceph-mon[70676]: pgmap v1042: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:17.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.961+0000 7f5edb001700 1 -- 172.21.15.88:0/3208852741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ed40ff350 msgr2=0x7f5ed40ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:17.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.961+0000 7f5edb001700 1 --2- 172.21.15.88:0/3208852741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ed40ff350 0x7f5ed40ff7c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000aa20 tx=0x7f5ec000ad30 comp rx=0 tx=0).stop 2024-02-19T19:38:17.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.961+0000 7f5edb001700 1 -- 172.21.15.88:0/3208852741 shutdown_connections 2024-02-19T19:38:17.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.961+0000 7f5edb001700 1 --2- 172.21.15.88:0/3208852741 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ed40ff350 0x7f5ed40ff7c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:17.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.961+0000 7f5edb001700 1 --2- 172.21.15.88:0/3208852741 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ed41038b0 0x7f5ed4103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:17.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.961+0000 7f5edb001700 1 -- 172.21.15.88:0/3208852741 >> 172.21.15.88:0/3208852741 conn(0x7f5ed40748b0 msgr2=0x7f5ed4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:17.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.961+0000 7f5edb001700 1 -- 172.21.15.88:0/3208852741 shutdown_connections 2024-02-19T19:38:17.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.961+0000 7f5edb001700 1 -- 172.21.15.88:0/3208852741 wait complete. 2024-02-19T19:38:17.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.961+0000 7f5edb001700 1 Processor -- start 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5edb001700 1 -- start start 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5edb001700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ed40ff350 0x7f5ed4195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5edb001700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ed41038b0 0x7f5ed4196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5edb001700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed4196a60 con 0x7f5ed40ff350 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5edb001700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed419a7f0 con 0x7f5ed41038b0 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5ed8d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ed40ff350 0x7f5ed4195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5ed8d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ed40ff350 0x7f5ed4195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49412/0 (socket says 172.21.15.88:49412) 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5ed8d9d700 1 -- 172.21.15.88:0/466646528 learned_addr learned my addr 172.21.15.88:0/466646528 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5ed8d9d700 1 -- 172.21.15.88:0/466646528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ed41038b0 msgr2=0x7f5ed4196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5ed8d9d700 1 --2- 172.21.15.88:0/466646528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ed41038b0 0x7f5ed4196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5ed8d9d700 1 -- 172.21.15.88:0/466646528 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec000a700 con 0x7f5ed40ff350 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.962+0000 7f5ed8d9d700 1 --2- 172.21.15.88:0/466646528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ed40ff350 0x7f5ed4195e40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800d8d0 tx=0x7f5ec800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:17.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.963+0000 7f5ed1ffb700 1 -- 172.21.15.88:0/466646528 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ec8009940 con 0x7f5ed40ff350 2024-02-19T19:38:17.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.963+0000 7f5ed1ffb700 1 -- 172.21.15.88:0/466646528 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ec8010460 con 0x7f5ed40ff350 2024-02-19T19:38:17.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.963+0000 7f5edb001700 1 -- 172.21.15.88:0/466646528 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed419aad0 con 0x7f5ed40ff350 2024-02-19T19:38:17.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.963+0000 7f5ed1ffb700 1 -- 172.21.15.88:0/466646528 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ec800f5f0 con 0x7f5ed40ff350 2024-02-19T19:38:17.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.963+0000 7f5edb001700 1 -- 172.21.15.88:0/466646528 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed419b020 con 0x7f5ed40ff350 2024-02-19T19:38:17.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.964+0000 7f5edb001700 1 -- 172.21.15.88:0/466646528 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed4108e30 con 0x7f5ed40ff350 2024-02-19T19:38:17.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.964+0000 7f5ed1ffb700 1 -- 172.21.15.88:0/466646528 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5ec80105d0 con 0x7f5ed40ff350 2024-02-19T19:38:17.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.965+0000 7f5ed1ffb700 1 --2- 172.21.15.88:0/466646528 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ec406d8a0 0x7f5ec406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:17.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.965+0000 7f5ed1ffb700 1 -- 172.21.15.88:0/466646528 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5ec808d540 con 0x7f5ed40ff350 2024-02-19T19:38:17.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.965+0000 7f5ed3fff700 1 --2- 172.21.15.88:0/466646528 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ec406d8a0 0x7f5ec406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:17.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.965+0000 7f5ed3fff700 1 --2- 172.21.15.88:0/466646528 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ec406d8a0 0x7f5ec406fd50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c6d0 tx=0x7f5ec000af90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:17.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:17.970+0000 7f5ed1ffb700 1 -- 172.21.15.88:0/466646528 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ec8056cf0 con 0x7f5ed40ff350 2024-02-19T19:38:18.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.185+0000 7f5edb001700 1 -- 172.21.15.88:0/466646528 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ed419b240 con 0x7f5ec406d8a0 2024-02-19T19:38:18.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.186+0000 7f5ed1ffb700 1 -- 172.21.15.88:0/466646528 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5ed419b240 con 0x7f5ec406d8a0 2024-02-19T19:38:18.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.188+0000 7f5edb001700 1 -- 172.21.15.88:0/466646528 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ec406d8a0 msgr2=0x7f5ec406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.188+0000 7f5edb001700 1 --2- 172.21.15.88:0/466646528 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ec406d8a0 0x7f5ec406fd50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c6d0 tx=0x7f5ec000af90 comp rx=0 tx=0).stop 2024-02-19T19:38:18.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.188+0000 7f5edb001700 1 -- 172.21.15.88:0/466646528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ed40ff350 msgr2=0x7f5ed4195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.188+0000 7f5edb001700 1 --2- 172.21.15.88:0/466646528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ed40ff350 0x7f5ed4195e40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800d8d0 tx=0x7f5ec800dc90 comp rx=0 tx=0).stop 2024-02-19T19:38:18.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.188+0000 7f5edb001700 1 -- 172.21.15.88:0/466646528 shutdown_connections 2024-02-19T19:38:18.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.188+0000 7f5edb001700 1 --2- 172.21.15.88:0/466646528 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ec406d8a0 0x7f5ec406fd50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.188+0000 7f5edb001700 1 --2- 172.21.15.88:0/466646528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ed40ff350 0x7f5ed4195e40 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.188+0000 7f5edb001700 1 --2- 172.21.15.88:0/466646528 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5ed41038b0 0x7f5ed4196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.188+0000 7f5edb001700 1 -- 172.21.15.88:0/466646528 >> 172.21.15.88:0/466646528 conn(0x7f5ed40748b0 msgr2=0x7f5ed4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:18.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.189+0000 7f5edb001700 1 -- 172.21.15.88:0/466646528 shutdown_connections 2024-02-19T19:38:18.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.189+0000 7f5edb001700 1 -- 172.21.15.88:0/466646528 wait complete. 2024-02-19T19:38:18.201 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:38:18.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.293+0000 7f410d67c700 1 -- 172.21.15.88:0/478391089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41080ff3b0 msgr2=0x7f41080ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.293+0000 7f410d67c700 1 --2- 172.21.15.88:0/478391089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41080ff3b0 0x7f41080ff780 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f40f8009910 tx=0x7f40f8009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:18.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.293+0000 7f410d67c700 1 -- 172.21.15.88:0/478391089 shutdown_connections 2024-02-19T19:38:18.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.293+0000 7f410d67c700 1 --2- 172.21.15.88:0/478391089 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41080ffd50 0x7f4108103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.293+0000 7f410d67c700 1 --2- 172.21.15.88:0/478391089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41080ff3b0 0x7f41080ff780 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.293+0000 7f410d67c700 1 -- 172.21.15.88:0/478391089 >> 172.21.15.88:0/478391089 conn(0x7f41080fac90 msgr2=0x7f41080fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:18.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.293+0000 7f410d67c700 1 -- 172.21.15.88:0/478391089 shutdown_connections 2024-02-19T19:38:18.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.293+0000 7f410d67c700 1 -- 172.21.15.88:0/478391089 wait complete. 2024-02-19T19:38:18.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.294+0000 7f410d67c700 1 Processor -- start 2024-02-19T19:38:18.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.294+0000 7f410d67c700 1 -- start start 2024-02-19T19:38:18.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.294+0000 7f410d67c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41080ff3b0 0x7f4108195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:18.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.294+0000 7f410d67c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41080ffd50 0x7f4108196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:18.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.294+0000 7f410d67c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4108196b70 con 0x7f41080ffd50 2024-02-19T19:38:18.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.294+0000 7f410d67c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410819a900 con 0x7f41080ff3b0 2024-02-19T19:38:18.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.294+0000 7f41067fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41080ffd50 0x7f4108196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.294+0000 7f41067fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41080ffd50 0x7f4108196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49434/0 (socket says 172.21.15.88:49434) 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.294+0000 7f41067fc700 1 -- 172.21.15.88:0/2970022702 learned_addr learned my addr 172.21.15.88:0/2970022702 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.294+0000 7f4106ffd700 1 --2- 172.21.15.88:0/2970022702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41080ff3b0 0x7f4108195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.295+0000 7f41067fc700 1 -- 172.21.15.88:0/2970022702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41080ff3b0 msgr2=0x7f4108195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.295+0000 7f41067fc700 1 --2- 172.21.15.88:0/2970022702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41080ff3b0 0x7f4108195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.295+0000 7f41067fc700 1 -- 172.21.15.88:0/2970022702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f80095f0 con 0x7f41080ffd50 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.295+0000 7f41067fc700 1 --2- 172.21.15.88:0/2970022702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41080ffd50 0x7f4108196490 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00b480 tx=0x7f40fc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.295+0000 7f40effff700 1 -- 172.21.15.88:0/2970022702 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40fc011800 con 0x7f41080ffd50 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.295+0000 7f40effff700 1 -- 172.21.15.88:0/2970022702 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40fc011e20 con 0x7f41080ffd50 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.295+0000 7f410d67c700 1 -- 172.21.15.88:0/2970022702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410819abe0 con 0x7f41080ffd50 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.295+0000 7f40effff700 1 -- 172.21.15.88:0/2970022702 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40fc010ea0 con 0x7f41080ffd50 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.295+0000 7f410d67c700 1 -- 172.21.15.88:0/2970022702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410819b0e0 con 0x7f41080ffd50 2024-02-19T19:38:18.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.296+0000 7f410d67c700 1 -- 172.21.15.88:0/2970022702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410804f2a0 con 0x7f41080ffd50 2024-02-19T19:38:18.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.297+0000 7f40effff700 1 -- 172.21.15.88:0/2970022702 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f40fc010380 con 0x7f41080ffd50 2024-02-19T19:38:18.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.297+0000 7f40effff700 1 --2- 172.21.15.88:0/2970022702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40f006d8a0 0x7f40f006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:18.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.297+0000 7f40effff700 1 -- 172.21.15.88:0/2970022702 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f40fc08cfa0 con 0x7f41080ffd50 2024-02-19T19:38:18.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.297+0000 7f4106ffd700 1 --2- 172.21.15.88:0/2970022702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40f006d8a0 0x7f40f006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:18.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.298+0000 7f4106ffd700 1 --2- 172.21.15.88:0/2970022702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40f006d8a0 0x7f40f006fd50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f40f8006010 tx=0x7f40f801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:18.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.302+0000 7f40effff700 1 -- 172.21.15.88:0/2970022702 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f40fc0567d0 con 0x7f41080ffd50 2024-02-19T19:38:18.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.518+0000 7f410d67c700 1 -- 172.21.15.88:0/2970022702 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f410819b490 con 0x7f40f006d8a0 2024-02-19T19:38:18.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.519+0000 7f40effff700 1 -- 172.21.15.88:0/2970022702 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f410819b490 con 0x7f40f006d8a0 2024-02-19T19:38:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.521+0000 7f410d67c700 1 -- 172.21.15.88:0/2970022702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40f006d8a0 msgr2=0x7f40f006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.521+0000 7f410d67c700 1 --2- 172.21.15.88:0/2970022702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40f006d8a0 0x7f40f006fd50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f40f8006010 tx=0x7f40f801a040 comp rx=0 tx=0).stop 2024-02-19T19:38:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.521+0000 7f410d67c700 1 -- 172.21.15.88:0/2970022702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41080ffd50 msgr2=0x7f4108196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.521+0000 7f410d67c700 1 --2- 172.21.15.88:0/2970022702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41080ffd50 0x7f4108196490 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f40fc00b480 tx=0x7f40fc00b790 comp rx=0 tx=0).stop 2024-02-19T19:38:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.521+0000 7f410d67c700 1 -- 172.21.15.88:0/2970022702 shutdown_connections 2024-02-19T19:38:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.521+0000 7f410d67c700 1 --2- 172.21.15.88:0/2970022702 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f40f006d8a0 0x7f40f006fd50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.521+0000 7f410d67c700 1 --2- 172.21.15.88:0/2970022702 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f41080ff3b0 0x7f4108195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.521+0000 7f410d67c700 1 --2- 172.21.15.88:0/2970022702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41080ffd50 0x7f4108196490 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.521+0000 7f410d67c700 1 -- 172.21.15.88:0/2970022702 >> 172.21.15.88:0/2970022702 conn(0x7f41080fac90 msgr2=0x7f4108103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.522+0000 7f410d67c700 1 -- 172.21.15.88:0/2970022702 shutdown_connections 2024-02-19T19:38:18.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.522+0000 7f410d67c700 1 -- 172.21.15.88:0/2970022702 wait complete. 2024-02-19T19:38:18.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.627+0000 7ffb69b24700 1 -- 172.21.15.88:0/2305768678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb640ff350 msgr2=0x7ffb640ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.627+0000 7ffb69b24700 1 --2- 172.21.15.88:0/2305768678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb640ff350 0x7ffb640ff720 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7ffb54009910 tx=0x7ffb54009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:18.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.627+0000 7ffb69b24700 1 -- 172.21.15.88:0/2305768678 shutdown_connections 2024-02-19T19:38:18.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.627+0000 7ffb69b24700 1 --2- 172.21.15.88:0/2305768678 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb640ffc60 0x7ffb6410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.627+0000 7ffb69b24700 1 --2- 172.21.15.88:0/2305768678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb640ff350 0x7ffb640ff720 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.627+0000 7ffb69b24700 1 -- 172.21.15.88:0/2305768678 >> 172.21.15.88:0/2305768678 conn(0x7ffb640748b0 msgr2=0x7ffb64074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:18.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.627+0000 7ffb69b24700 1 -- 172.21.15.88:0/2305768678 shutdown_connections 2024-02-19T19:38:18.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.627+0000 7ffb69b24700 1 -- 172.21.15.88:0/2305768678 wait complete. 2024-02-19T19:38:18.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.627+0000 7ffb69b24700 1 Processor -- start 2024-02-19T19:38:18.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.627+0000 7ffb69b24700 1 -- start start 2024-02-19T19:38:18.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb69b24700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb640ff350 0x7ffb64100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:18.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb69b24700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb640ffc60 0x7ffb64100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:18.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb69b24700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb64100fb0 con 0x7ffb640ffc60 2024-02-19T19:38:18.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb69b24700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb641010f0 con 0x7ffb640ff350 2024-02-19T19:38:18.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb5affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb640ffc60 0x7ffb64100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:18.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb5affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb640ffc60 0x7ffb64100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49446/0 (socket says 172.21.15.88:49446) 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb5affd700 1 -- 172.21.15.88:0/2502449317 learned_addr learned my addr 172.21.15.88:0/2502449317 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb637fe700 1 --2- 172.21.15.88:0/2502449317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb640ff350 0x7ffb64100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb5affd700 1 -- 172.21.15.88:0/2502449317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb640ff350 msgr2=0x7ffb64100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb5affd700 1 --2- 172.21.15.88:0/2502449317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb640ff350 0x7ffb64100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb5affd700 1 -- 172.21.15.88:0/2502449317 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb540095f0 con 0x7ffb640ffc60 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb5affd700 1 --2- 172.21.15.88:0/2502449317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb640ffc60 0x7ffb64100a70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00c8a0 tx=0x7ffb4c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.628+0000 7ffb617fa700 1 -- 172.21.15.88:0/2502449317 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb4c0078e0 con 0x7ffb640ffc60 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.629+0000 7ffb69b24700 1 -- 172.21.15.88:0/2502449317 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb641a2030 con 0x7ffb640ffc60 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.629+0000 7ffb617fa700 1 -- 172.21.15.88:0/2502449317 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffb4c00f450 con 0x7ffb640ffc60 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.629+0000 7ffb617fa700 1 -- 172.21.15.88:0/2502449317 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb4c00e5e0 con 0x7ffb640ffc60 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.629+0000 7ffb69b24700 1 -- 172.21.15.88:0/2502449317 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb641a2440 con 0x7ffb640ffc60 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.630+0000 7ffb69b24700 1 -- 172.21.15.88:0/2502449317 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb64107dd0 con 0x7ffb640ffc60 2024-02-19T19:38:18.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.630+0000 7ffb617fa700 1 -- 172.21.15.88:0/2502449317 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7ffb4c00f5c0 con 0x7ffb640ffc60 2024-02-19T19:38:18.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.630+0000 7ffb617fa700 1 --2- 172.21.15.88:0/2502449317 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffb5006d8a0 0x7ffb5006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:18.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.630+0000 7ffb617fa700 1 -- 172.21.15.88:0/2502449317 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ffb4c08d510 con 0x7ffb640ffc60 2024-02-19T19:38:18.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.630+0000 7ffb637fe700 1 --2- 172.21.15.88:0/2502449317 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffb5006d8a0 0x7ffb5006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:18.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.631+0000 7ffb637fe700 1 --2- 172.21.15.88:0/2502449317 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffb5006d8a0 0x7ffb5006fd50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ffb54006010 tx=0x7ffb5401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:18.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.635+0000 7ffb617fa700 1 -- 172.21.15.88:0/2502449317 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb4c056cc0 con 0x7ffb640ffc60 2024-02-19T19:38:18.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.848+0000 7ffb69b24700 1 -- 172.21.15.88:0/2502449317 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb64061190 con 0x7ffb5006d8a0 2024-02-19T19:38:18.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.859+0000 7ffb617fa700 1 -- 172.21.15.88:0/2502449317 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7ffb64061190 con 0x7ffb5006d8a0 2024-02-19T19:38:18.862 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:38:18.862 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (35m) 11m ago 45m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:38:18.862 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (46m) 11m ago 46m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:38:18.862 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (28m) 11m ago 44m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:38:18.862 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (31m) 11m ago 46m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:38:18.863 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (31m) 11m ago 44m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:38:18.863 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (34m) 11m ago 45m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:38:18.863 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (27m) 11m ago 39m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:38:18.863 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (27m) 11m ago 39m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:38:18.863 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (27m) 11m ago 39m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:38:18.864 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (11m) 11m ago 39m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:38:18.864 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (36m) 11m ago 47m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:38:18.865 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (36m) 11m ago 44m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:38:18.866 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (33m) 11m ago 47m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:38:18.866 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (32m) 11m ago 44m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:38:18.866 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (36m) 11m ago 46m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:38:18.867 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (36m) 11m ago 44m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:38:18.867 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (30m) 11m ago 43m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:38:18.868 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (30m) 11m ago 42m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:38:18.868 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (29m) 11m ago 42m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:38:18.868 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (29m) 11m ago 42m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:38:18.868 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (28m) 11m ago 41m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:38:18.868 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (27m) 11m ago 41m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:38:18.868 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (35m) 11m ago 44m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:38:18.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.861+0000 7ffb69b24700 1 -- 172.21.15.88:0/2502449317 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffb5006d8a0 msgr2=0x7ffb5006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.861+0000 7ffb69b24700 1 --2- 172.21.15.88:0/2502449317 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffb5006d8a0 0x7ffb5006fd50 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ffb54006010 tx=0x7ffb5401a040 comp rx=0 tx=0).stop 2024-02-19T19:38:18.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.861+0000 7ffb69b24700 1 -- 172.21.15.88:0/2502449317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb640ffc60 msgr2=0x7ffb64100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.861+0000 7ffb69b24700 1 --2- 172.21.15.88:0/2502449317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb640ffc60 0x7ffb64100a70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00c8a0 tx=0x7ffb4c00cbb0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.861+0000 7ffb69b24700 1 -- 172.21.15.88:0/2502449317 shutdown_connections 2024-02-19T19:38:18.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.861+0000 7ffb69b24700 1 --2- 172.21.15.88:0/2502449317 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7ffb5006d8a0 0x7ffb5006fd50 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.861+0000 7ffb69b24700 1 --2- 172.21.15.88:0/2502449317 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7ffb640ff350 0x7ffb64100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.861+0000 7ffb69b24700 1 --2- 172.21.15.88:0/2502449317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb640ffc60 0x7ffb64100a70 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.861+0000 7ffb69b24700 1 -- 172.21.15.88:0/2502449317 >> 172.21.15.88:0/2502449317 conn(0x7ffb640748b0 msgr2=0x7ffb64104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:18.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.861+0000 7ffb69b24700 1 -- 172.21.15.88:0/2502449317 shutdown_connections 2024-02-19T19:38:18.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.861+0000 7ffb69b24700 1 -- 172.21.15.88:0/2502449317 wait complete. 2024-02-19T19:38:18.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.963+0000 7f54ca48a700 1 -- 172.21.15.88:0/275839413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c40fefb0 msgr2=0x7f54c40ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.963+0000 7f54ca48a700 1 --2- 172.21.15.88:0/275839413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c40fefb0 0x7f54c40ff380 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f54b4009910 tx=0x7f54b4009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:18.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.963+0000 7f54ca48a700 1 -- 172.21.15.88:0/275839413 shutdown_connections 2024-02-19T19:38:18.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.963+0000 7f54ca48a700 1 --2- 172.21.15.88:0/275839413 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c40ff8c0 0x7f54c410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.963+0000 7f54ca48a700 1 --2- 172.21.15.88:0/275839413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c40fefb0 0x7f54c40ff380 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.963+0000 7f54ca48a700 1 -- 172.21.15.88:0/275839413 >> 172.21.15.88:0/275839413 conn(0x7f54c40fac80 msgr2=0x7f54c40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:18.967 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:18 smithi088 ceph-mon[88643]: from='client.35874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:18.967 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:18 smithi088 ceph-mon[88643]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:18.967 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:18 smithi088 ceph-mon[88643]: pgmap v1043: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:18.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.964+0000 7f54ca48a700 1 -- 172.21.15.88:0/275839413 shutdown_connections 2024-02-19T19:38:18.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.964+0000 7f54ca48a700 1 -- 172.21.15.88:0/275839413 wait complete. 2024-02-19T19:38:18.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.964+0000 7f54ca48a700 1 Processor -- start 2024-02-19T19:38:18.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.964+0000 7f54ca48a700 1 -- start start 2024-02-19T19:38:18.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54ca48a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c40fefb0 0x7f54c4198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:18.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54ca48a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c40ff8c0 0x7f54c4198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:18.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54ca48a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c4198c60 con 0x7f54c40ff8c0 2024-02-19T19:38:18.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54ca48a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c419c9f0 con 0x7f54c40fefb0 2024-02-19T19:38:18.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c40ff8c0 0x7f54c4198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:18.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54c37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c40ff8c0 0x7f54c4198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49456/0 (socket says 172.21.15.88:49456) 2024-02-19T19:38:18.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54c37fe700 1 -- 172.21.15.88:0/1623341366 learned_addr learned my addr 172.21.15.88:0/1623341366 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:18.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54c3fff700 1 --2- 172.21.15.88:0/1623341366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c40fefb0 0x7f54c4198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:18.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54c37fe700 1 -- 172.21.15.88:0/1623341366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c40fefb0 msgr2=0x7f54c4198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:18.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54c37fe700 1 --2- 172.21.15.88:0/1623341366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c40fefb0 0x7f54c4198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:18.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54c37fe700 1 -- 172.21.15.88:0/1623341366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b40095f0 con 0x7f54c40ff8c0 2024-02-19T19:38:18.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54c37fe700 1 --2- 172.21.15.88:0/1623341366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c40ff8c0 0x7f54c4198580 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f54b800d900 tx=0x7f54b800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:18.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.965+0000 7f54c17fa700 1 -- 172.21.15.88:0/1623341366 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54b8004a00 con 0x7f54c40ff8c0 2024-02-19T19:38:18.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.966+0000 7f54ca48a700 1 -- 172.21.15.88:0/1623341366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c419ccd0 con 0x7f54c40ff8c0 2024-02-19T19:38:18.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.966+0000 7f54c17fa700 1 -- 172.21.15.88:0/1623341366 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54b80054e0 con 0x7f54c40ff8c0 2024-02-19T19:38:18.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.966+0000 7f54c17fa700 1 -- 172.21.15.88:0/1623341366 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54b8009d00 con 0x7f54c40ff8c0 2024-02-19T19:38:18.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.966+0000 7f54ca48a700 1 -- 172.21.15.88:0/1623341366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c419d220 con 0x7f54c40ff8c0 2024-02-19T19:38:18.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.967+0000 7f54ca48a700 1 -- 172.21.15.88:0/1623341366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c4109d90 con 0x7f54c40ff8c0 2024-02-19T19:38:18.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.967+0000 7f54c17fa700 1 -- 172.21.15.88:0/1623341366 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f54b8005020 con 0x7f54c40ff8c0 2024-02-19T19:38:18.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.967+0000 7f54c17fa700 1 --2- 172.21.15.88:0/1623341366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f54ac06d8a0 0x7f54ac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:18.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.968+0000 7f54c17fa700 1 -- 172.21.15.88:0/1623341366 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f54b808d3f0 con 0x7f54c40ff8c0 2024-02-19T19:38:18.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.968+0000 7f54c3fff700 1 --2- 172.21.15.88:0/1623341366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f54ac06d8a0 0x7f54ac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:18.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.968+0000 7f54c3fff700 1 --2- 172.21.15.88:0/1623341366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f54ac06d8a0 0x7f54ac06fd50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f54b4006010 tx=0x7f54b401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:18.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:18.972+0000 7f54c17fa700 1 -- 172.21.15.88:0/1623341366 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f54b8056c20 con 0x7f54c40ff8c0 2024-02-19T19:38:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:18 smithi132 ceph-mon[70676]: from='client.35874 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:18 smithi132 ceph-mon[70676]: from='client.35878 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:18 smithi132 ceph-mon[70676]: pgmap v1043: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:19.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.251+0000 7f54ca48a700 1 -- 172.21.15.88:0/1623341366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f54c419d490 con 0x7f54c40ff8c0 2024-02-19T19:38:19.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.252+0000 7f54c17fa700 1 -- 172.21.15.88:0/1623341366 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f54b8009e60 con 0x7f54c40ff8c0 2024-02-19T19:38:19.254 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:38:19.254 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:38:19.254 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:38:19.254 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:38:19.254 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:38:19.254 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:38:19.254 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:38:19.254 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:38:19.254 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:38:19.254 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:38:19.255 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:38:19.255 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:38:19.255 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:38:19.255 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:38:19.255 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:38:19.255 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:38:19.255 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:38:19.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.253+0000 7f54ca48a700 1 -- 172.21.15.88:0/1623341366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f54ac06d8a0 msgr2=0x7f54ac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:19.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.253+0000 7f54ca48a700 1 --2- 172.21.15.88:0/1623341366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f54ac06d8a0 0x7f54ac06fd50 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f54b4006010 tx=0x7f54b401a040 comp rx=0 tx=0).stop 2024-02-19T19:38:19.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.253+0000 7f54ca48a700 1 -- 172.21.15.88:0/1623341366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c40ff8c0 msgr2=0x7f54c4198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:19.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.253+0000 7f54ca48a700 1 --2- 172.21.15.88:0/1623341366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c40ff8c0 0x7f54c4198580 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f54b800d900 tx=0x7f54b800dc10 comp rx=0 tx=0).stop 2024-02-19T19:38:19.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.253+0000 7f54ca48a700 1 -- 172.21.15.88:0/1623341366 shutdown_connections 2024-02-19T19:38:19.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.253+0000 7f54ca48a700 1 --2- 172.21.15.88:0/1623341366 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f54ac06d8a0 0x7f54ac06fd50 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.253+0000 7f54ca48a700 1 --2- 172.21.15.88:0/1623341366 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f54c40fefb0 0x7f54c4198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.253+0000 7f54ca48a700 1 --2- 172.21.15.88:0/1623341366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c40ff8c0 0x7f54c4198580 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.253+0000 7f54ca48a700 1 -- 172.21.15.88:0/1623341366 >> 172.21.15.88:0/1623341366 conn(0x7f54c40fac80 msgr2=0x7f54c40fc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:19.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.253+0000 7f54ca48a700 1 -- 172.21.15.88:0/1623341366 shutdown_connections 2024-02-19T19:38:19.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.253+0000 7f54ca48a700 1 -- 172.21.15.88:0/1623341366 wait complete. 2024-02-19T19:38:19.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.361+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/1239953775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f840ffc60 msgr2=0x7f5f8410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:19.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.361+0000 7f5f8c5ac700 1 --2- 172.21.15.88:0/1239953775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f840ffc60 0x7f5f8410a6d0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5f74009910 tx=0x7f5f74009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:19.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.361+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/1239953775 shutdown_connections 2024-02-19T19:38:19.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.361+0000 7f5f8c5ac700 1 --2- 172.21.15.88:0/1239953775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f840ffc60 0x7f5f8410a6d0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f8c5ac700 1 --2- 172.21.15.88:0/1239953775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f840ff350 0x7f5f840ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/1239953775 >> 172.21.15.88:0/1239953775 conn(0x7f5f840748b0 msgr2=0x7f5f84074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:19.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/1239953775 shutdown_connections 2024-02-19T19:38:19.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/1239953775 wait complete. 2024-02-19T19:38:19.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f8c5ac700 1 Processor -- start 2024-02-19T19:38:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f8c5ac700 1 -- start start 2024-02-19T19:38:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f8c5ac700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f840ff350 0x7f5f84100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f8c5ac700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f840ffc60 0x7f5f84100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f8c5ac700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f84100fb0 con 0x7f5f840ffc60 2024-02-19T19:38:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f8c5ac700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f841010f0 con 0x7f5f840ff350 2024-02-19T19:38:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f89b47700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f840ffc60 0x7f5f84100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:19.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f89b47700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f840ffc60 0x7f5f84100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49478/0 (socket says 172.21.15.88:49478) 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f89b47700 1 -- 172.21.15.88:0/2908045474 learned_addr learned my addr 172.21.15.88:0/2908045474 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.362+0000 7f5f8a348700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f840ff350 0x7f5f84100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.363+0000 7f5f89b47700 1 -- 172.21.15.88:0/2908045474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f840ff350 msgr2=0x7f5f84100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.363+0000 7f5f89b47700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f840ff350 0x7f5f84100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.363+0000 7f5f89b47700 1 -- 172.21.15.88:0/2908045474 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740095f0 con 0x7f5f840ffc60 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.363+0000 7f5f8a348700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f840ff350 0x7f5f84100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.363+0000 7f5f89b47700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f840ffc60 0x7f5f84100a70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5f74006010 tx=0x7f5f7400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.363+0000 7f5f7b7fe700 1 -- 172.21.15.88:0/2908045474 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f7401d070 con 0x7f5f840ffc60 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.363+0000 7f5f7b7fe700 1 -- 172.21.15.88:0/2908045474 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f7400f460 con 0x7f5f840ffc60 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.363+0000 7f5f7b7fe700 1 -- 172.21.15.88:0/2908045474 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f74017630 con 0x7f5f840ffc60 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.363+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/2908045474 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f84101370 con 0x7f5f840ffc60 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.363+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/2908045474 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f841a2360 con 0x7f5f840ffc60 2024-02-19T19:38:19.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.364+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/2908045474 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f84107dd0 con 0x7f5f840ffc60 2024-02-19T19:38:19.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.364+0000 7f5f7b7fe700 1 -- 172.21.15.88:0/2908045474 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5f7400f5d0 con 0x7f5f840ffc60 2024-02-19T19:38:19.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.365+0000 7f5f7b7fe700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7006d8a0 0x7f5f7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:19.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.365+0000 7f5f7b7fe700 1 -- 172.21.15.88:0/2908045474 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5f7408ea00 con 0x7f5f840ffc60 2024-02-19T19:38:19.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.365+0000 7f5f8a348700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7006d8a0 0x7f5f7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:19.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.365+0000 7f5f8a348700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7006d8a0 0x7f5f7006fd50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005950 tx=0x7f5f800058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:19.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.370+0000 7f5f7b7fe700 1 -- 172.21.15.88:0/2908045474 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f740581b0 con 0x7f5f840ffc60 2024-02-19T19:38:19.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.611+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/2908045474 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5f8404f2a0 con 0x7f5f840ffc60 2024-02-19T19:38:19.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.611+0000 7f5f7b7fe700 1 -- 172.21.15.88:0/2908045474 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f5f7405b870 con 0x7f5f840ffc60 2024-02-19T19:38:19.614 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:38:19.614 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:38:19.614 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:38:19.614 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:38:19.614 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:38:19.614 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:38:19.614 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:38:19.614 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:38:19.614 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:38:19.614 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:38:19.614 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:38:19.615 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:38:19.616 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:38:19.616 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:38:19.616 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:38:19.616 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:38:19.616 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:38:19.616 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:38:19.616 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:38:19.616 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:38:19.616 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:38:19.616 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:38:19.617 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:38:19.617 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:38:19.617 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:38:19.617 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:38:19.617 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:38:19.617 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:38:19.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.613+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/2908045474 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7006d8a0 msgr2=0x7f5f7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:19.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.613+0000 7f5f8c5ac700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7006d8a0 0x7f5f7006fd50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5f80005950 tx=0x7f5f800058e0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.613+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/2908045474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f840ffc60 msgr2=0x7f5f84100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:19.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.613+0000 7f5f8c5ac700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f840ffc60 0x7f5f84100a70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5f74006010 tx=0x7f5f7400be20 comp rx=0 tx=0).stop 2024-02-19T19:38:19.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.613+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/2908045474 shutdown_connections 2024-02-19T19:38:19.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.613+0000 7f5f8c5ac700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7006d8a0 0x7f5f7006fd50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.613+0000 7f5f8c5ac700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f840ff350 0x7f5f84100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.613+0000 7f5f8c5ac700 1 --2- 172.21.15.88:0/2908045474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f840ffc60 0x7f5f84100a70 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.613+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/2908045474 >> 172.21.15.88:0/2908045474 conn(0x7f5f840748b0 msgr2=0x7f5f84104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:19.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.614+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/2908045474 shutdown_connections 2024-02-19T19:38:19.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.614+0000 7f5f8c5ac700 1 -- 172.21.15.88:0/2908045474 wait complete. 2024-02-19T19:38:19.621 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:38:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.719+0000 7f0582f8c700 1 -- 172.21.15.88:0/2855106108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f057c104130 msgr2=0x7f057c104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.719+0000 7f0582f8c700 1 --2- 172.21.15.88:0/2855106108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f057c104130 0x7f057c104500 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0570009910 tx=0x7f0570009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.719+0000 7f0582f8c700 1 -- 172.21.15.88:0/2855106108 shutdown_connections 2024-02-19T19:38:19.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.719+0000 7f0582f8c700 1 --2- 172.21.15.88:0/2855106108 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f057c0ffbd0 0x7f057c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.719+0000 7f0582f8c700 1 --2- 172.21.15.88:0/2855106108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f057c104130 0x7f057c104500 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.719+0000 7f0582f8c700 1 -- 172.21.15.88:0/2855106108 >> 172.21.15.88:0/2855106108 conn(0x7f057c075130 msgr2=0x7f057c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:19.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.719+0000 7f0582f8c700 1 -- 172.21.15.88:0/2855106108 shutdown_connections 2024-02-19T19:38:19.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.719+0000 7f0582f8c700 1 -- 172.21.15.88:0/2855106108 wait complete. 2024-02-19T19:38:19.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.719+0000 7f0582f8c700 1 Processor -- start 2024-02-19T19:38:19.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f0582f8c700 1 -- start start 2024-02-19T19:38:19.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f0582f8c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f057c0ffbd0 0x7f057c195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:19.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f0582f8c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f057c104130 0x7f057c196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:19.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f0582f8c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c196b70 con 0x7f057c104130 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f0582f8c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057c19a900 con 0x7f057c0ffbd0 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f057bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f057c104130 0x7f057c196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f057bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f057c104130 0x7f057c196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49498/0 (socket says 172.21.15.88:49498) 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f057bfff700 1 -- 172.21.15.88:0/220579657 learned_addr learned my addr 172.21.15.88:0/220579657 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f057bfff700 1 -- 172.21.15.88:0/220579657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f057c0ffbd0 msgr2=0x7f057c195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f057bfff700 1 --2- 172.21.15.88:0/220579657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f057c0ffbd0 0x7f057c195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f057bfff700 1 -- 172.21.15.88:0/220579657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05700095f0 con 0x7f057c104130 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f057bfff700 1 --2- 172.21.15.88:0/220579657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f057c104130 0x7f057c196490 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f056800b4b0 tx=0x7f056800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f0579ffb700 1 -- 172.21.15.88:0/220579657 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0568011800 con 0x7f057c104130 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f0582f8c700 1 -- 172.21.15.88:0/220579657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c19abe0 con 0x7f057c104130 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f0579ffb700 1 -- 172.21.15.88:0/220579657 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0568011e20 con 0x7f057c104130 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.720+0000 7f0579ffb700 1 -- 172.21.15.88:0/220579657 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f056800f2e0 con 0x7f057c104130 2024-02-19T19:38:19.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.721+0000 7f0582f8c700 1 -- 172.21.15.88:0/220579657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057c19b130 con 0x7f057c104130 2024-02-19T19:38:19.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.722+0000 7f0582f8c700 1 -- 172.21.15.88:0/220579657 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057c1088a0 con 0x7f057c104130 2024-02-19T19:38:19.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.722+0000 7f0579ffb700 1 -- 172.21.15.88:0/220579657 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0568010380 con 0x7f057c104130 2024-02-19T19:38:19.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.722+0000 7f0579ffb700 1 --2- 172.21.15.88:0/220579657 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f056c06d8a0 0x7f056c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:19.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.722+0000 7f0579ffb700 1 -- 172.21.15.88:0/220579657 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f056808d0f0 con 0x7f057c104130 2024-02-19T19:38:19.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.723+0000 7f0580d28700 1 --2- 172.21.15.88:0/220579657 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f056c06d8a0 0x7f056c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:19.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.723+0000 7f0580d28700 1 --2- 172.21.15.88:0/220579657 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f056c06d8a0 0x7f056c06fd50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0570006010 tx=0x7f057000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:19.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.727+0000 7f0579ffb700 1 -- 172.21.15.88:0/220579657 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f05680568a0 con 0x7f057c104130 2024-02-19T19:38:19.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.945+0000 7f0582f8c700 1 -- 172.21.15.88:0/220579657 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f057c19b3b0 con 0x7f056c06d8a0 2024-02-19T19:38:19.948 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:19 smithi088 ceph-mon[88643]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:19.948 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:19 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1623341366' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:38:19.948 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:19 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2908045474' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:38:19.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.947+0000 7f0579ffb700 1 -- 172.21.15.88:0/220579657 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f057c19b3b0 con 0x7f056c06d8a0 2024-02-19T19:38:19.949 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:38:19.949 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:38:19.949 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:38:19.949 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:38:19.949 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:38:19.949 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:38:19.949 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:38:19.949 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:38:19.949 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:38:19.950 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:38:19.950 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:38:19.950 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:38:19.950 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:38:19.950 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:38:19.950 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:38:19.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.949+0000 7f0582f8c700 1 -- 172.21.15.88:0/220579657 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f056c06d8a0 msgr2=0x7f056c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:19.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.949+0000 7f0582f8c700 1 --2- 172.21.15.88:0/220579657 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f056c06d8a0 0x7f056c06fd50 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0570006010 tx=0x7f057000b540 comp rx=0 tx=0).stop 2024-02-19T19:38:19.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.949+0000 7f0582f8c700 1 -- 172.21.15.88:0/220579657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f057c104130 msgr2=0x7f057c196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:19.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.949+0000 7f0582f8c700 1 --2- 172.21.15.88:0/220579657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f057c104130 0x7f057c196490 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f056800b4b0 tx=0x7f056800b870 comp rx=0 tx=0).stop 2024-02-19T19:38:19.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.949+0000 7f0582f8c700 1 -- 172.21.15.88:0/220579657 shutdown_connections 2024-02-19T19:38:19.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.949+0000 7f0582f8c700 1 --2- 172.21.15.88:0/220579657 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f056c06d8a0 0x7f056c06fd50 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.949+0000 7f0582f8c700 1 --2- 172.21.15.88:0/220579657 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f057c0ffbd0 0x7f057c195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.949+0000 7f0582f8c700 1 --2- 172.21.15.88:0/220579657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f057c104130 0x7f057c196490 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:19.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.949+0000 7f0582f8c700 1 -- 172.21.15.88:0/220579657 >> 172.21.15.88:0/220579657 conn(0x7f057c075130 msgr2=0x7f057c068680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:19.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.949+0000 7f0582f8c700 1 -- 172.21.15.88:0/220579657 shutdown_connections 2024-02-19T19:38:19.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:19.949+0000 7f0582f8c700 1 -- 172.21.15.88:0/220579657 wait complete. 2024-02-19T19:38:20.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.053+0000 7f7100791700 1 -- 172.21.15.88:0/3968668653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70f80ffd60 msgr2=0x7f70f8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:20.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.053+0000 7f7100791700 1 --2- 172.21.15.88:0/3968668653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70f80ffd60 0x7f70f8103d30 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f70e8009910 tx=0x7f70e8009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:20.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.053+0000 7f7100791700 1 -- 172.21.15.88:0/3968668653 shutdown_connections 2024-02-19T19:38:20.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.053+0000 7f7100791700 1 --2- 172.21.15.88:0/3968668653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70f80ffd60 0x7f70f8103d30 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:20.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.053+0000 7f7100791700 1 --2- 172.21.15.88:0/3968668653 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70f80ff3c0 0x7f70f80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:20.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.053+0000 7f7100791700 1 -- 172.21.15.88:0/3968668653 >> 172.21.15.88:0/3968668653 conn(0x7f70f80fac80 msgr2=0x7f70f80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:20.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.053+0000 7f7100791700 1 -- 172.21.15.88:0/3968668653 shutdown_connections 2024-02-19T19:38:20.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.053+0000 7f7100791700 1 -- 172.21.15.88:0/3968668653 wait complete. 2024-02-19T19:38:20.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.053+0000 7f7100791700 1 Processor -- start 2024-02-19T19:38:20.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f7100791700 1 -- start start 2024-02-19T19:38:20.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f7100791700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70f80ff3c0 0x7f70f8194c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:20.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f7100791700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70f80ffd60 0x7f70f8195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f7100791700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f81956c0 con 0x7f70f80ff3c0 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f7100791700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f8195830 con 0x7f70f80ffd60 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70fe52d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70f80ff3c0 0x7f70f8194c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70fe52d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70f80ff3c0 0x7f70f8194c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49526/0 (socket says 172.21.15.88:49526) 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70fe52d700 1 -- 172.21.15.88:0/4083040064 learned_addr learned my addr 172.21.15.88:0/4083040064 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70fdd2c700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70f80ffd60 0x7f70f8195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70fe52d700 1 -- 172.21.15.88:0/4083040064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70f80ffd60 msgr2=0x7f70f8195180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70fe52d700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70f80ffd60 0x7f70f8195180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70fe52d700 1 -- 172.21.15.88:0/4083040064 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e80095f0 con 0x7f70f80ff3c0 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70fdd2c700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70f80ffd60 0x7f70f8195180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70fe52d700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70f80ff3c0 0x7f70f8194c40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f70f000b4b0 tx=0x7f70f000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70ef7fe700 1 -- 172.21.15.88:0/4083040064 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f0011800 con 0x7f70f80ff3c0 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f7100791700 1 -- 172.21.15.88:0/4083040064 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f81a22b0 con 0x7f70f80ff3c0 2024-02-19T19:38:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70ef7fe700 1 -- 172.21.15.88:0/4083040064 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f70f0011e20 con 0x7f70f80ff3c0 2024-02-19T19:38:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.054+0000 7f70ef7fe700 1 -- 172.21.15.88:0/4083040064 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70f000f2e0 con 0x7f70f80ff3c0 2024-02-19T19:38:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.055+0000 7f7100791700 1 -- 172.21.15.88:0/4083040064 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f81a2610 con 0x7f70f80ff3c0 2024-02-19T19:38:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.056+0000 7f7100791700 1 -- 172.21.15.88:0/4083040064 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f8107660 con 0x7f70f80ff3c0 2024-02-19T19:38:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.056+0000 7f70ef7fe700 1 -- 172.21.15.88:0/4083040064 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f70f0010380 con 0x7f70f80ff3c0 2024-02-19T19:38:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.056+0000 7f70ef7fe700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70e406d8a0 0x7f70e406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:20.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.056+0000 7f70ef7fe700 1 -- 172.21.15.88:0/4083040064 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f70f008cff0 con 0x7f70f80ff3c0 2024-02-19T19:38:20.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.056+0000 7f70fdd2c700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70e406d8a0 0x7f70e406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:20.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.057+0000 7f70fdd2c700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70e406d8a0 0x7f70e406fd50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f70e800b5c0 tx=0x7f70e8005480 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:20.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.061+0000 7f70ef7fe700 1 -- 172.21.15.88:0/4083040064 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f70f005a400 con 0x7f70f80ff3c0 2024-02-19T19:38:20.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:19 smithi132 ceph-mon[70676]: from='client.35882 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:20.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:19 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1623341366' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:38:20.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:19 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2908045474' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:38:20.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.341+0000 7f7100791700 1 -- 172.21.15.88:0/4083040064 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f70f804f2a0 con 0x7f70f80ff3c0 2024-02-19T19:38:20.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.342+0000 7f70ef7fe700 1 -- 172.21.15.88:0/4083040064 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f70f0059f90 con 0x7f70f80ff3c0 2024-02-19T19:38:20.344 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:38:20.344 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:38:20.344 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:38:20.345 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:38:20.345 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:38:20.345 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:38:20.345 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:38:20.345 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:38:20.345 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:38:20.345 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:38:20.345 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:38:20.345 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:38:20.345 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:38:20.346 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:38:20.346 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:38:20.346 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:38:20.346 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:38:20.346 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:38:20.346 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:38:20.346 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:38:20.346 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:38:20.346 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:38:20.347 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:38:20.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.344+0000 7f7100791700 1 -- 172.21.15.88:0/4083040064 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70e406d8a0 msgr2=0x7f70e406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:20.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.344+0000 7f7100791700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70e406d8a0 0x7f70e406fd50 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f70e800b5c0 tx=0x7f70e8005480 comp rx=0 tx=0).stop 2024-02-19T19:38:20.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.344+0000 7f7100791700 1 -- 172.21.15.88:0/4083040064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70f80ff3c0 msgr2=0x7f70f8194c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:20.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.344+0000 7f7100791700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70f80ff3c0 0x7f70f8194c40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f70f000b4b0 tx=0x7f70f000b870 comp rx=0 tx=0).stop 2024-02-19T19:38:20.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.344+0000 7f7100791700 1 -- 172.21.15.88:0/4083040064 shutdown_connections 2024-02-19T19:38:20.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.344+0000 7f7100791700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f70e406d8a0 0x7f70e406fd50 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:20.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.344+0000 7f7100791700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70f80ff3c0 0x7f70f8194c40 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:20.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.344+0000 7f7100791700 1 --2- 172.21.15.88:0/4083040064 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f70f80ffd60 0x7f70f8195180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:20.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.344+0000 7f7100791700 1 -- 172.21.15.88:0/4083040064 >> 172.21.15.88:0/4083040064 conn(0x7f70f80fac80 msgr2=0x7f70f80fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:20.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.344+0000 7f7100791700 1 -- 172.21.15.88:0/4083040064 shutdown_connections 2024-02-19T19:38:20.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:20.345+0000 7f7100791700 1 -- 172.21.15.88:0/4083040064 wait complete. 2024-02-19T19:38:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:20 smithi088 ceph-mon[88643]: from='client.35894 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:21.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:20 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4083040064' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:38:21.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:20 smithi088 ceph-mon[88643]: pgmap v1044: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:20 smithi132 ceph-mon[70676]: from='client.35894 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:20 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4083040064' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:38:21.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:20 smithi132 ceph-mon[70676]: pgmap v1044: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:22 smithi088 ceph-mon[88643]: pgmap v1045: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:23.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:22 smithi132 ceph-mon[70676]: pgmap v1045: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:24 smithi088 ceph-mon[88643]: pgmap v1046: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:24 smithi132 ceph-mon[70676]: pgmap v1046: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:26 smithi088 ceph-mon[88643]: pgmap v1047: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:27.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:26 smithi132 ceph-mon[70676]: pgmap v1047: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:28.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:38:28.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:38:29.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:29 smithi088 ceph-mon[88643]: pgmap v1048: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:29.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:29 smithi132 ceph-mon[70676]: pgmap v1048: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:30 smithi088 ceph-mon[88643]: pgmap v1049: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:30 smithi132 ceph-mon[70676]: pgmap v1049: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:32 smithi088 ceph-mon[88643]: pgmap v1050: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:33.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:32 smithi132 ceph-mon[70676]: pgmap v1050: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:34 smithi088 ceph-mon[88643]: pgmap v1051: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:34 smithi132 ceph-mon[70676]: pgmap v1051: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:36 smithi088 ceph-mon[88643]: pgmap v1052: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:36 smithi132 ceph-mon[70676]: pgmap v1052: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:38 smithi088 ceph-mon[88643]: pgmap v1053: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:38 smithi132 ceph-mon[70676]: pgmap v1053: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:40 smithi088 ceph-mon[88643]: pgmap v1054: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:40 smithi132 ceph-mon[70676]: pgmap v1054: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:42 smithi088 ceph-mon[88643]: pgmap v1055: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:43.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:42 smithi132 ceph-mon[70676]: pgmap v1055: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:38:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:38:44.982 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:44 smithi088 ceph-mon[88643]: pgmap v1056: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:44 smithi132 ceph-mon[70676]: pgmap v1056: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:46 smithi088 ceph-mon[88643]: pgmap v1057: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:46 smithi132 ceph-mon[70676]: pgmap v1057: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:48 smithi088 ceph-mon[88643]: pgmap v1058: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:49.081 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:48 smithi132 ceph-mon[70676]: pgmap v1058: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.462+0000 7f19ca022700 1 -- 172.21.15.88:0/1922769825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c41004e0 msgr2=0x7f19c41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.462+0000 7f19ca022700 1 --2- 172.21.15.88:0/1922769825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c41004e0 0x7f19c41008b0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f19b4009910 tx=0x7f19b4009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.462+0000 7f19ca022700 1 -- 172.21.15.88:0/1922769825 shutdown_connections 2024-02-19T19:38:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.462+0000 7f19ca022700 1 --2- 172.21.15.88:0/1922769825 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19c4100e80 0x7f19c4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.462+0000 7f19ca022700 1 --2- 172.21.15.88:0/1922769825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c41004e0 0x7f19c41008b0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.462+0000 7f19ca022700 1 -- 172.21.15.88:0/1922769825 >> 172.21.15.88:0/1922769825 conn(0x7f19c40fbda0 msgr2=0x7f19c40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.462+0000 7f19ca022700 1 -- 172.21.15.88:0/1922769825 shutdown_connections 2024-02-19T19:38:50.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.462+0000 7f19ca022700 1 -- 172.21.15.88:0/1922769825 wait complete. 2024-02-19T19:38:50.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.463+0000 7f19ca022700 1 Processor -- start 2024-02-19T19:38:50.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.463+0000 7f19ca022700 1 -- start start 2024-02-19T19:38:50.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.463+0000 7f19ca022700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19c41004e0 0x7f19c4195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:50.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.463+0000 7f19ca022700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c4100e80 0x7f19c4196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19ca022700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c4196b10 con 0x7f19c4100e80 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19ca022700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c419a8a0 con 0x7f19c41004e0 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19c2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c4100e80 0x7f19c4196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19c2ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c4100e80 0x7f19c4196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53048/0 (socket says 172.21.15.88:53048) 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19c2ffd700 1 -- 172.21.15.88:0/3539325019 learned_addr learned my addr 172.21.15.88:0/3539325019 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19c37fe700 1 --2- 172.21.15.88:0/3539325019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19c41004e0 0x7f19c4195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19c2ffd700 1 -- 172.21.15.88:0/3539325019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19c41004e0 msgr2=0x7f19c4195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19c2ffd700 1 --2- 172.21.15.88:0/3539325019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19c41004e0 0x7f19c4195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19c2ffd700 1 -- 172.21.15.88:0/3539325019 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b40095f0 con 0x7f19c4100e80 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19c2ffd700 1 --2- 172.21.15.88:0/3539325019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c4100e80 0x7f19c4196430 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f19b800d900 tx=0x7f19b800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19c0ff9700 1 -- 172.21.15.88:0/3539325019 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19b8009900 con 0x7f19c4100e80 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19c0ff9700 1 -- 172.21.15.88:0/3539325019 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19b8010460 con 0x7f19c4100e80 2024-02-19T19:38:50.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19ca022700 1 -- 172.21.15.88:0/3539325019 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c419ab80 con 0x7f19c4100e80 2024-02-19T19:38:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19c0ff9700 1 -- 172.21.15.88:0/3539325019 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19b800f5f0 con 0x7f19c4100e80 2024-02-19T19:38:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.464+0000 7f19ca022700 1 -- 172.21.15.88:0/3539325019 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c419b0d0 con 0x7f19c4100e80 2024-02-19T19:38:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.465+0000 7f19ca022700 1 -- 172.21.15.88:0/3539325019 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c41087f0 con 0x7f19c4100e80 2024-02-19T19:38:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.466+0000 7f19c0ff9700 1 -- 172.21.15.88:0/3539325019 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f19b80105d0 con 0x7f19c4100e80 2024-02-19T19:38:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.466+0000 7f19c0ff9700 1 --2- 172.21.15.88:0/3539325019 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19ac06d8a0 0x7f19ac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:50.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.467+0000 7f19c37fe700 1 --2- 172.21.15.88:0/3539325019 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19ac06d8a0 0x7f19ac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:50.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.467+0000 7f19c0ff9700 1 -- 172.21.15.88:0/3539325019 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f19b808cfd0 con 0x7f19c4100e80 2024-02-19T19:38:50.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.467+0000 7f19c37fe700 1 --2- 172.21.15.88:0/3539325019 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19ac06d8a0 0x7f19ac06fd50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f19b4006010 tx=0x7f19b401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:50.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.471+0000 7f19c0ff9700 1 -- 172.21.15.88:0/3539325019 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f19b8056800 con 0x7f19c4100e80 2024-02-19T19:38:50.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.688+0000 7f19ca022700 1 -- 172.21.15.88:0/3539325019 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19c419b340 con 0x7f19ac06d8a0 2024-02-19T19:38:50.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.689+0000 7f19c0ff9700 1 -- 172.21.15.88:0/3539325019 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f19c419b340 con 0x7f19ac06d8a0 2024-02-19T19:38:50.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.692+0000 7f19ca022700 1 -- 172.21.15.88:0/3539325019 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19ac06d8a0 msgr2=0x7f19ac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:50.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.692+0000 7f19ca022700 1 --2- 172.21.15.88:0/3539325019 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19ac06d8a0 0x7f19ac06fd50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f19b4006010 tx=0x7f19b401a040 comp rx=0 tx=0).stop 2024-02-19T19:38:50.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.692+0000 7f19ca022700 1 -- 172.21.15.88:0/3539325019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c4100e80 msgr2=0x7f19c4196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:50.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.692+0000 7f19ca022700 1 --2- 172.21.15.88:0/3539325019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c4100e80 0x7f19c4196430 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f19b800d900 tx=0x7f19b800dc10 comp rx=0 tx=0).stop 2024-02-19T19:38:50.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.692+0000 7f19ca022700 1 -- 172.21.15.88:0/3539325019 shutdown_connections 2024-02-19T19:38:50.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.692+0000 7f19ca022700 1 --2- 172.21.15.88:0/3539325019 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f19ac06d8a0 0x7f19ac06fd50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:50.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.692+0000 7f19ca022700 1 --2- 172.21.15.88:0/3539325019 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f19c41004e0 0x7f19c4195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:50.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.692+0000 7f19ca022700 1 --2- 172.21.15.88:0/3539325019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19c4100e80 0x7f19c4196430 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:50.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.692+0000 7f19ca022700 1 -- 172.21.15.88:0/3539325019 >> 172.21.15.88:0/3539325019 conn(0x7f19c40fbda0 msgr2=0x7f19c40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:50.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.692+0000 7f19ca022700 1 -- 172.21.15.88:0/3539325019 shutdown_connections 2024-02-19T19:38:50.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.692+0000 7f19ca022700 1 -- 172.21.15.88:0/3539325019 wait complete. 2024-02-19T19:38:50.703 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:38:50.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.798+0000 7fba9adce700 1 -- 172.21.15.88:0/48935394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba940ff350 msgr2=0x7fba940ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:50.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.798+0000 7fba9adce700 1 --2- 172.21.15.88:0/48935394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba940ff350 0x7fba940ff720 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fba88009910 tx=0x7fba88009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.798+0000 7fba9adce700 1 -- 172.21.15.88:0/48935394 shutdown_connections 2024-02-19T19:38:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.798+0000 7fba9adce700 1 --2- 172.21.15.88:0/48935394 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba940ffc60 0x7fba9410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.798+0000 7fba9adce700 1 --2- 172.21.15.88:0/48935394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba940ff350 0x7fba940ff720 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.798+0000 7fba9adce700 1 -- 172.21.15.88:0/48935394 >> 172.21.15.88:0/48935394 conn(0x7fba940748b0 msgr2=0x7fba94074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.798+0000 7fba9adce700 1 -- 172.21.15.88:0/48935394 shutdown_connections 2024-02-19T19:38:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.798+0000 7fba9adce700 1 -- 172.21.15.88:0/48935394 wait complete. 2024-02-19T19:38:50.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.799+0000 7fba9adce700 1 Processor -- start 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.799+0000 7fba9adce700 1 -- start start 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.799+0000 7fba9adce700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba940ff350 0x7fba94195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.799+0000 7fba9adce700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba940ffc60 0x7fba941963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.799+0000 7fba9adce700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba94196a80 con 0x7fba940ffc60 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.799+0000 7fba9adce700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9419a810 con 0x7fba940ff350 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.799+0000 7fba93fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba940ffc60 0x7fba941963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.799+0000 7fba93fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba940ffc60 0x7fba941963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53058/0 (socket says 172.21.15.88:53058) 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.799+0000 7fba93fff700 1 -- 172.21.15.88:0/1323878505 learned_addr learned my addr 172.21.15.88:0/1323878505 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.800+0000 7fba98b6a700 1 --2- 172.21.15.88:0/1323878505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba940ff350 0x7fba94195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.800+0000 7fba93fff700 1 -- 172.21.15.88:0/1323878505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba940ff350 msgr2=0x7fba94195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.800+0000 7fba93fff700 1 --2- 172.21.15.88:0/1323878505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba940ff350 0x7fba94195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.800+0000 7fba93fff700 1 -- 172.21.15.88:0/1323878505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba880095f0 con 0x7fba940ffc60 2024-02-19T19:38:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.800+0000 7fba93fff700 1 --2- 172.21.15.88:0/1323878505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba940ffc60 0x7fba941963a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fba8000d8d0 tx=0x7fba8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:50.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.800+0000 7fba91ffb700 1 -- 172.21.15.88:0/1323878505 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba80009940 con 0x7fba940ffc60 2024-02-19T19:38:50.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.800+0000 7fba91ffb700 1 -- 172.21.15.88:0/1323878505 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba80010460 con 0x7fba940ffc60 2024-02-19T19:38:50.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.800+0000 7fba9adce700 1 -- 172.21.15.88:0/1323878505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9419aaf0 con 0x7fba940ffc60 2024-02-19T19:38:50.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.800+0000 7fba91ffb700 1 -- 172.21.15.88:0/1323878505 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba8000f5f0 con 0x7fba940ffc60 2024-02-19T19:38:50.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.800+0000 7fba9adce700 1 -- 172.21.15.88:0/1323878505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9419b040 con 0x7fba940ffc60 2024-02-19T19:38:50.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.802+0000 7fba9adce700 1 -- 172.21.15.88:0/1323878505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba94107dd0 con 0x7fba940ffc60 2024-02-19T19:38:50.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.802+0000 7fba91ffb700 1 -- 172.21.15.88:0/1323878505 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fba800105d0 con 0x7fba940ffc60 2024-02-19T19:38:50.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.802+0000 7fba91ffb700 1 --2- 172.21.15.88:0/1323878505 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba8406d8a0 0x7fba8406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:50.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.803+0000 7fba91ffb700 1 -- 172.21.15.88:0/1323878505 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fba8008d540 con 0x7fba940ffc60 2024-02-19T19:38:50.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.803+0000 7fba98b6a700 1 --2- 172.21.15.88:0/1323878505 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba8406d8a0 0x7fba8406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:50.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.803+0000 7fba98b6a700 1 --2- 172.21.15.88:0/1323878505 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba8406d8a0 0x7fba8406fd50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fba8800b5c0 tx=0x7fba8801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:50.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:50.807+0000 7fba91ffb700 1 -- 172.21.15.88:0/1323878505 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fba80056cf0 con 0x7fba940ffc60 2024-02-19T19:38:51.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.027+0000 7fba9adce700 1 -- 172.21.15.88:0/1323878505 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fba9419b2b0 con 0x7fba8406d8a0 2024-02-19T19:38:51.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.028+0000 7fba91ffb700 1 -- 172.21.15.88:0/1323878505 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fba9419b2b0 con 0x7fba8406d8a0 2024-02-19T19:38:51.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.030+0000 7fba9adce700 1 -- 172.21.15.88:0/1323878505 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba8406d8a0 msgr2=0x7fba8406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.030+0000 7fba9adce700 1 --2- 172.21.15.88:0/1323878505 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba8406d8a0 0x7fba8406fd50 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fba8800b5c0 tx=0x7fba8801a040 comp rx=0 tx=0).stop 2024-02-19T19:38:51.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.030+0000 7fba9adce700 1 -- 172.21.15.88:0/1323878505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba940ffc60 msgr2=0x7fba941963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.030+0000 7fba9adce700 1 --2- 172.21.15.88:0/1323878505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba940ffc60 0x7fba941963a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fba8000d8d0 tx=0x7fba8000dc90 comp rx=0 tx=0).stop 2024-02-19T19:38:51.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.030+0000 7fba9adce700 1 -- 172.21.15.88:0/1323878505 shutdown_connections 2024-02-19T19:38:51.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.030+0000 7fba9adce700 1 --2- 172.21.15.88:0/1323878505 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fba8406d8a0 0x7fba8406fd50 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.030+0000 7fba9adce700 1 --2- 172.21.15.88:0/1323878505 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fba940ff350 0x7fba94195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.030+0000 7fba9adce700 1 --2- 172.21.15.88:0/1323878505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba940ffc60 0x7fba941963a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.030+0000 7fba9adce700 1 -- 172.21.15.88:0/1323878505 >> 172.21.15.88:0/1323878505 conn(0x7fba940748b0 msgr2=0x7fba94104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:51.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.030+0000 7fba9adce700 1 -- 172.21.15.88:0/1323878505 shutdown_connections 2024-02-19T19:38:51.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.031+0000 7fba9adce700 1 -- 172.21.15.88:0/1323878505 wait complete. 2024-02-19T19:38:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:50 smithi088 ceph-mon[88643]: from='client.35902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:50 smithi088 ceph-mon[88643]: pgmap v1059: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.135+0000 7f98244d2700 1 -- 172.21.15.88:0/1938220186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f981c0ffd60 msgr2=0x7f981c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.135+0000 7f98244d2700 1 --2- 172.21.15.88:0/1938220186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f981c0ffd60 0x7f981c103d30 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9818009910 tx=0x7f9818009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.135+0000 7f98244d2700 1 -- 172.21.15.88:0/1938220186 shutdown_connections 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.135+0000 7f98244d2700 1 --2- 172.21.15.88:0/1938220186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f981c0ffd60 0x7f981c103d30 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.135+0000 7f98244d2700 1 --2- 172.21.15.88:0/1938220186 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981c0ff3c0 0x7f981c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.135+0000 7f98244d2700 1 -- 172.21.15.88:0/1938220186 >> 172.21.15.88:0/1938220186 conn(0x7f981c0fac80 msgr2=0x7f981c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.135+0000 7f98244d2700 1 -- 172.21.15.88:0/1938220186 shutdown_connections 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.135+0000 7f98244d2700 1 -- 172.21.15.88:0/1938220186 wait complete. 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f98244d2700 1 Processor -- start 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f98244d2700 1 -- start start 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f98244d2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f981c0ff3c0 0x7f981c196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f98244d2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981c0ffd60 0x7f981c196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:51.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f98244d2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c196c20 con 0x7f981c0ff3c0 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f98244d2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f981c19a9b0 con 0x7f981c0ffd60 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f982226e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f981c0ff3c0 0x7f981c196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f982226e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f981c0ff3c0 0x7f981c196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53074/0 (socket says 172.21.15.88:53074) 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f982226e700 1 -- 172.21.15.88:0/3374542939 learned_addr learned my addr 172.21.15.88:0/3374542939 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f9821a6d700 1 --2- 172.21.15.88:0/3374542939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981c0ffd60 0x7f981c196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f982226e700 1 -- 172.21.15.88:0/3374542939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981c0ffd60 msgr2=0x7f981c196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f982226e700 1 --2- 172.21.15.88:0/3374542939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981c0ffd60 0x7f981c196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f982226e700 1 -- 172.21.15.88:0/3374542939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98180095f0 con 0x7f981c0ff3c0 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.136+0000 7f982226e700 1 --2- 172.21.15.88:0/3374542939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f981c0ff3c0 0x7f981c196000 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f980c00b4b0 tx=0x7f980c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.137+0000 7f98137fe700 1 -- 172.21.15.88:0/3374542939 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f980c011800 con 0x7f981c0ff3c0 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.137+0000 7f98137fe700 1 -- 172.21.15.88:0/3374542939 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f980c011e20 con 0x7f981c0ff3c0 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.137+0000 7f98244d2700 1 -- 172.21.15.88:0/3374542939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f981c19ac90 con 0x7f981c0ff3c0 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.137+0000 7f98137fe700 1 -- 172.21.15.88:0/3374542939 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f980c00f2e0 con 0x7f981c0ff3c0 2024-02-19T19:38:51.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.137+0000 7f98244d2700 1 -- 172.21.15.88:0/3374542939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f981c19b1e0 con 0x7f981c0ff3c0 2024-02-19T19:38:51.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.138+0000 7f98244d2700 1 -- 172.21.15.88:0/3374542939 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f981c04f2a0 con 0x7f981c0ff3c0 2024-02-19T19:38:51.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.138+0000 7f98137fe700 1 -- 172.21.15.88:0/3374542939 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f980c010380 con 0x7f981c0ff3c0 2024-02-19T19:38:51.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.138+0000 7f98137fe700 1 --2- 172.21.15.88:0/3374542939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f980806d8a0 0x7f980806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:51.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.139+0000 7f98137fe700 1 -- 172.21.15.88:0/3374542939 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f980c08cfa0 con 0x7f981c0ff3c0 2024-02-19T19:38:51.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.139+0000 7f9821a6d700 1 --2- 172.21.15.88:0/3374542939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f980806d8a0 0x7f980806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:51.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.139+0000 7f9821a6d700 1 --2- 172.21.15.88:0/3374542939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f980806d8a0 0x7f980806fd50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f981c197620 tx=0x7f981801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:51.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.143+0000 7f98137fe700 1 -- 172.21.15.88:0/3374542939 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f980c0567d0 con 0x7f981c0ff3c0 2024-02-19T19:38:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:50 smithi132 ceph-mon[70676]: from='client.35902 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:50 smithi132 ceph-mon[70676]: pgmap v1059: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:51.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.350+0000 7f98244d2700 1 -- 172.21.15.88:0/3374542939 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f981c19b520 con 0x7f980806d8a0 2024-02-19T19:38:51.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.360+0000 7f98137fe700 1 -- 172.21.15.88:0/3374542939 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f981c19b520 con 0x7f980806d8a0 2024-02-19T19:38:51.361 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (36m) 12m ago 46m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (46m) 12m ago 46m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (28m) 12m ago 45m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (32m) 12m ago 46m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (32m) 12m ago 44m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (35m) 12m ago 45m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (28m) 12m ago 40m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (27m) 12m ago 40m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (27m) 12m ago 40m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (12m) 12m ago 40m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (37m) 12m ago 48m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (36m) 12m ago 44m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (34m) 12m ago 48m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (33m) 12m ago 44m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (36m) 12m ago 46m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:38:51.362 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (36m) 12m ago 44m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:38:51.363 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (31m) 12m ago 43m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:38:51.363 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (30m) 12m ago 43m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:38:51.363 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (30m) 12m ago 43m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:38:51.363 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (29m) 12m ago 42m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:38:51.363 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (29m) 12m ago 42m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:38:51.363 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (28m) 12m ago 41m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:38:51.363 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (36m) 12m ago 45m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:38:51.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.362+0000 7f98244d2700 1 -- 172.21.15.88:0/3374542939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f980806d8a0 msgr2=0x7f980806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.362+0000 7f98244d2700 1 --2- 172.21.15.88:0/3374542939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f980806d8a0 0x7f980806fd50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f981c197620 tx=0x7f981801a040 comp rx=0 tx=0).stop 2024-02-19T19:38:51.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.362+0000 7f98244d2700 1 -- 172.21.15.88:0/3374542939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f981c0ff3c0 msgr2=0x7f981c196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.362+0000 7f98244d2700 1 --2- 172.21.15.88:0/3374542939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f981c0ff3c0 0x7f981c196000 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f980c00b4b0 tx=0x7f980c00b7c0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.362+0000 7f98244d2700 1 -- 172.21.15.88:0/3374542939 shutdown_connections 2024-02-19T19:38:51.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.362+0000 7f98244d2700 1 --2- 172.21.15.88:0/3374542939 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f980806d8a0 0x7f980806fd50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.362+0000 7f98244d2700 1 --2- 172.21.15.88:0/3374542939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f981c0ff3c0 0x7f981c196000 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.362+0000 7f98244d2700 1 --2- 172.21.15.88:0/3374542939 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f981c0ffd60 0x7f981c196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.362+0000 7f98244d2700 1 -- 172.21.15.88:0/3374542939 >> 172.21.15.88:0/3374542939 conn(0x7f981c0fac80 msgr2=0x7f981c1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:51.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.363+0000 7f98244d2700 1 -- 172.21.15.88:0/3374542939 shutdown_connections 2024-02-19T19:38:51.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.363+0000 7f98244d2700 1 -- 172.21.15.88:0/3374542939 wait complete. 2024-02-19T19:38:51.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.465+0000 7fdbfb138700 1 -- 172.21.15.88:0/1894622593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf40ffc60 msgr2=0x7fdbf410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.465+0000 7fdbfb138700 1 --2- 172.21.15.88:0/1894622593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf40ffc60 0x7fdbf410a6d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fdbe0009910 tx=0x7fdbe0009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:51.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.465+0000 7fdbfb138700 1 -- 172.21.15.88:0/1894622593 shutdown_connections 2024-02-19T19:38:51.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.465+0000 7fdbfb138700 1 --2- 172.21.15.88:0/1894622593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf40ffc60 0x7fdbf410a6d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.466+0000 7fdbfb138700 1 --2- 172.21.15.88:0/1894622593 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf40ff350 0x7fdbf40ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.466+0000 7fdbfb138700 1 -- 172.21.15.88:0/1894622593 >> 172.21.15.88:0/1894622593 conn(0x7fdbf40748b0 msgr2=0x7fdbf4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:51.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.466+0000 7fdbfb138700 1 -- 172.21.15.88:0/1894622593 shutdown_connections 2024-02-19T19:38:51.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.466+0000 7fdbfb138700 1 -- 172.21.15.88:0/1894622593 wait complete. 2024-02-19T19:38:51.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.466+0000 7fdbfb138700 1 Processor -- start 2024-02-19T19:38:51.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.466+0000 7fdbfb138700 1 -- start start 2024-02-19T19:38:51.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbfb138700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf40ff350 0x7fdbf4195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:51.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbfb138700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf40ffc60 0x7fdbf4196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:51.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbfb138700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf4196a20 con 0x7fdbf40ff350 2024-02-19T19:38:51.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbfb138700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf419a7b0 con 0x7fdbf40ffc60 2024-02-19T19:38:51.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbf8ed4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf40ff350 0x7fdbf4195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:51.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbf8ed4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf40ff350 0x7fdbf4195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53090/0 (socket says 172.21.15.88:53090) 2024-02-19T19:38:51.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbf8ed4700 1 -- 172.21.15.88:0/522439384 learned_addr learned my addr 172.21.15.88:0/522439384 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:51.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbf3fff700 1 --2- 172.21.15.88:0/522439384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf40ffc60 0x7fdbf4196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:51.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbf8ed4700 1 -- 172.21.15.88:0/522439384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf40ffc60 msgr2=0x7fdbf4196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbf8ed4700 1 --2- 172.21.15.88:0/522439384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf40ffc60 0x7fdbf4196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbf8ed4700 1 -- 172.21.15.88:0/522439384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe00095f0 con 0x7fdbf40ff350 2024-02-19T19:38:51.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbf8ed4700 1 --2- 172.21.15.88:0/522439384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf40ff350 0x7fdbf4195e00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800e8f0 tx=0x7fdbe800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:51.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.467+0000 7fdbf1ffb700 1 -- 172.21.15.88:0/522439384 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbe8009790 con 0x7fdbf40ff350 2024-02-19T19:38:51.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.468+0000 7fdbf1ffb700 1 -- 172.21.15.88:0/522439384 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdbe800c810 con 0x7fdbf40ff350 2024-02-19T19:38:51.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.468+0000 7fdbfb138700 1 -- 172.21.15.88:0/522439384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf419aa90 con 0x7fdbf40ff350 2024-02-19T19:38:51.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.468+0000 7fdbf1ffb700 1 -- 172.21.15.88:0/522439384 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbe8003d20 con 0x7fdbf40ff350 2024-02-19T19:38:51.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.468+0000 7fdbfb138700 1 -- 172.21.15.88:0/522439384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf419afe0 con 0x7fdbf40ff350 2024-02-19T19:38:51.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.469+0000 7fdbfb138700 1 -- 172.21.15.88:0/522439384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf4107dd0 con 0x7fdbf40ff350 2024-02-19T19:38:51.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.469+0000 7fdbf1ffb700 1 -- 172.21.15.88:0/522439384 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdbe80041f0 con 0x7fdbf40ff350 2024-02-19T19:38:51.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.470+0000 7fdbf1ffb700 1 --2- 172.21.15.88:0/522439384 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdbe406d8a0 0x7fdbe406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:51.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.470+0000 7fdbf1ffb700 1 -- 172.21.15.88:0/522439384 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdbe8014070 con 0x7fdbf40ff350 2024-02-19T19:38:51.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.470+0000 7fdbf3fff700 1 --2- 172.21.15.88:0/522439384 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdbe406d8a0 0x7fdbe406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:51.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.470+0000 7fdbf3fff700 1 --2- 172.21.15.88:0/522439384 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdbe406d8a0 0x7fdbe406fd50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4197420 tx=0x7fdbe0005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:51.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.475+0000 7fdbf1ffb700 1 -- 172.21.15.88:0/522439384 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdbe8056fc0 con 0x7fdbf40ff350 2024-02-19T19:38:51.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.752+0000 7fdbfb138700 1 -- 172.21.15.88:0/522439384 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdbf404f2a0 con 0x7fdbf40ff350 2024-02-19T19:38:51.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.753+0000 7fdbf1ffb700 1 -- 172.21.15.88:0/522439384 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fdbe805a680 con 0x7fdbf40ff350 2024-02-19T19:38:51.754 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:38:51.754 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:38:51.754 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:38:51.754 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:38:51.754 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:38:51.754 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:38:51.754 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:38:51.755 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:38:51.755 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:38:51.755 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:38:51.755 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:38:51.755 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:38:51.755 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:38:51.755 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:38:51.755 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:38:51.755 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:38:51.755 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:38:51.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.755+0000 7fdbfb138700 1 -- 172.21.15.88:0/522439384 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdbe406d8a0 msgr2=0x7fdbe406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.755+0000 7fdbfb138700 1 --2- 172.21.15.88:0/522439384 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdbe406d8a0 0x7fdbe406fd50 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4197420 tx=0x7fdbe0005940 comp rx=0 tx=0).stop 2024-02-19T19:38:51.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.755+0000 7fdbfb138700 1 -- 172.21.15.88:0/522439384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf40ff350 msgr2=0x7fdbf4195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.755+0000 7fdbfb138700 1 --2- 172.21.15.88:0/522439384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf40ff350 0x7fdbf4195e00 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fdbe800e8f0 tx=0x7fdbe800ec00 comp rx=0 tx=0).stop 2024-02-19T19:38:51.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.755+0000 7fdbfb138700 1 -- 172.21.15.88:0/522439384 shutdown_connections 2024-02-19T19:38:51.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.755+0000 7fdbfb138700 1 --2- 172.21.15.88:0/522439384 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdbe406d8a0 0x7fdbe406fd50 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.755+0000 7fdbfb138700 1 --2- 172.21.15.88:0/522439384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbf40ff350 0x7fdbf4195e00 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.755+0000 7fdbfb138700 1 --2- 172.21.15.88:0/522439384 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbf40ffc60 0x7fdbf4196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.755+0000 7fdbfb138700 1 -- 172.21.15.88:0/522439384 >> 172.21.15.88:0/522439384 conn(0x7fdbf40748b0 msgr2=0x7fdbf4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:51.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.755+0000 7fdbfb138700 1 -- 172.21.15.88:0/522439384 shutdown_connections 2024-02-19T19:38:51.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.755+0000 7fdbfb138700 1 -- 172.21.15.88:0/522439384 wait complete. 2024-02-19T19:38:51.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.859+0000 7f20bb0e5700 1 -- 172.21.15.88:0/4186934738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b4104130 msgr2=0x7f20b4104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.859+0000 7f20bb0e5700 1 --2- 172.21.15.88:0/4186934738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b4104130 0x7f20b4104500 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009910 tx=0x7f20a8009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:51.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.859+0000 7f20bb0e5700 1 -- 172.21.15.88:0/4186934738 shutdown_connections 2024-02-19T19:38:51.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.859+0000 7f20bb0e5700 1 --2- 172.21.15.88:0/4186934738 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20b40ffbd0 0x7f20b4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.859+0000 7f20bb0e5700 1 --2- 172.21.15.88:0/4186934738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b4104130 0x7f20b4104500 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.859+0000 7f20bb0e5700 1 -- 172.21.15.88:0/4186934738 >> 172.21.15.88:0/4186934738 conn(0x7f20b4075130 msgr2=0x7f20b4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:51.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.859+0000 7f20bb0e5700 1 -- 172.21.15.88:0/4186934738 shutdown_connections 2024-02-19T19:38:51.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.859+0000 7f20bb0e5700 1 -- 172.21.15.88:0/4186934738 wait complete. 2024-02-19T19:38:51.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.859+0000 7f20bb0e5700 1 Processor -- start 2024-02-19T19:38:51.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.859+0000 7f20bb0e5700 1 -- start start 2024-02-19T19:38:51.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20bb0e5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b40ffbd0 0x7f20b4195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20bb0e5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20b4104130 0x7f20b4196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20bb0e5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b41969e0 con 0x7f20b40ffbd0 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20bb0e5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b4196b20 con 0x7f20b4104130 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b8e81700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b40ffbd0 0x7f20b4195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b8e81700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b40ffbd0 0x7f20b4195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53104/0 (socket says 172.21.15.88:53104) 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b8e81700 1 -- 172.21.15.88:0/3540777835 learned_addr learned my addr 172.21.15.88:0/3540777835 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b3fff700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20b4104130 0x7f20b4196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b8e81700 1 -- 172.21.15.88:0/3540777835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20b4104130 msgr2=0x7f20b4196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b8e81700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20b4104130 0x7f20b4196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b8e81700 1 -- 172.21.15.88:0/3540777835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a80095f0 con 0x7f20b40ffbd0 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b3fff700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20b4104130 0x7f20b4196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b8e81700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b40ffbd0 0x7f20b4195e50 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f20a8006010 tx=0x7f20a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b1ffb700 1 -- 172.21.15.88:0/3540777835 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a801d070 con 0x7f20b40ffbd0 2024-02-19T19:38:51.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20bb0e5700 1 -- 172.21.15.88:0/3540777835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b419a910 con 0x7f20b40ffbd0 2024-02-19T19:38:51.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b1ffb700 1 -- 172.21.15.88:0/3540777835 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20a8004fb0 con 0x7f20b40ffbd0 2024-02-19T19:38:51.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.860+0000 7f20b1ffb700 1 -- 172.21.15.88:0/3540777835 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20a800f690 con 0x7f20b40ffbd0 2024-02-19T19:38:51.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.861+0000 7f20bb0e5700 1 -- 172.21.15.88:0/3540777835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b419ae00 con 0x7f20b40ffbd0 2024-02-19T19:38:51.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.861+0000 7f20bb0e5700 1 -- 172.21.15.88:0/3540777835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20b404f2a0 con 0x7f20b40ffbd0 2024-02-19T19:38:51.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.862+0000 7f20b1ffb700 1 -- 172.21.15.88:0/3540777835 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f20a8022470 con 0x7f20b40ffbd0 2024-02-19T19:38:51.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.862+0000 7f20b1ffb700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20a406d8a0 0x7f20a406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:51.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.862+0000 7f20b1ffb700 1 -- 172.21.15.88:0/3540777835 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f20a808eca0 con 0x7f20b40ffbd0 2024-02-19T19:38:51.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.862+0000 7f20b3fff700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20a406d8a0 0x7f20a406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:51.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.863+0000 7f20b3fff700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20a406d8a0 0x7f20a406fd50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f20a0005fd0 tx=0x7f20a0005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:51.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:51.867+0000 7f20b1ffb700 1 -- 172.21.15.88:0/3540777835 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20a8093070 con 0x7f20b40ffbd0 2024-02-19T19:38:52.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:51 smithi088 ceph-mon[88643]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:52.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:51 smithi088 ceph-mon[88643]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:52.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:51 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/522439384' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:38:52.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.106+0000 7f20bb0e5700 1 -- 172.21.15.88:0/3540777835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f20b41971b0 con 0x7f20b40ffbd0 2024-02-19T19:38:52.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.106+0000 7f20b1ffb700 1 -- 172.21.15.88:0/3540777835 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f20a8027740 con 0x7f20b40ffbd0 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:38:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:38:52.109 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:38:52.110 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:38:52.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.108+0000 7f20bb0e5700 1 -- 172.21.15.88:0/3540777835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20a406d8a0 msgr2=0x7f20a406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:52.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.108+0000 7f20bb0e5700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20a406d8a0 0x7f20a406fd50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f20a0005fd0 tx=0x7f20a0005e20 comp rx=0 tx=0).stop 2024-02-19T19:38:52.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.108+0000 7f20bb0e5700 1 -- 172.21.15.88:0/3540777835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b40ffbd0 msgr2=0x7f20b4195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:52.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.109+0000 7f20bb0e5700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b40ffbd0 0x7f20b4195e50 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f20a8006010 tx=0x7f20a800be20 comp rx=0 tx=0).stop 2024-02-19T19:38:52.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.109+0000 7f20bb0e5700 1 -- 172.21.15.88:0/3540777835 shutdown_connections 2024-02-19T19:38:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.109+0000 7f20bb0e5700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20a406d8a0 0x7f20a406fd50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.109+0000 7f20bb0e5700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20b40ffbd0 0x7f20b4195e50 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.109+0000 7f20bb0e5700 1 --2- 172.21.15.88:0/3540777835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20b4104130 0x7f20b4196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.109+0000 7f20bb0e5700 1 -- 172.21.15.88:0/3540777835 >> 172.21.15.88:0/3540777835 conn(0x7f20b4075130 msgr2=0x7f20b40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.109+0000 7f20bb0e5700 1 -- 172.21.15.88:0/3540777835 shutdown_connections 2024-02-19T19:38:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.109+0000 7f20bb0e5700 1 -- 172.21.15.88:0/3540777835 wait complete. 2024-02-19T19:38:52.112 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:38:52.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.212+0000 7fdbb2b80700 1 -- 172.21.15.88:0/1308132852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbac0ff720 msgr2=0x7fdbac0ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:52.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.212+0000 7fdbb2b80700 1 --2- 172.21.15.88:0/1308132852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbac0ff720 0x7fdbac0ffb90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009910 tx=0x7fdb98009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:52.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.212+0000 7fdbb2b80700 1 -- 172.21.15.88:0/1308132852 shutdown_connections 2024-02-19T19:38:52.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.212+0000 7fdbb2b80700 1 --2- 172.21.15.88:0/1308132852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbac0ff720 0x7fdbac0ffb90 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.212+0000 7fdbb2b80700 1 --2- 172.21.15.88:0/1308132852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbac0ff350 0x7fdbac1063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.212+0000 7fdbb2b80700 1 -- 172.21.15.88:0/1308132852 >> 172.21.15.88:0/1308132852 conn(0x7fdbac0748b0 msgr2=0x7fdbac074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:52.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.213+0000 7fdbb2b80700 1 -- 172.21.15.88:0/1308132852 shutdown_connections 2024-02-19T19:38:52.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.213+0000 7fdbb2b80700 1 -- 172.21.15.88:0/1308132852 wait complete. 2024-02-19T19:38:52.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.213+0000 7fdbb2b80700 1 Processor -- start 2024-02-19T19:38:52.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.213+0000 7fdbb2b80700 1 -- start start 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.213+0000 7fdbb2b80700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbac0ff350 0x7fdbac19c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.213+0000 7fdbb2b80700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbac0ff720 0x7fdbac19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.213+0000 7fdbb2b80700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac19d030 con 0x7fdbac0ff350 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.213+0000 7fdbb2b80700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac1a0dc0 con 0x7fdbac0ff720 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.213+0000 7fdbb091c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbac0ff350 0x7fdbac19c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.213+0000 7fdbb091c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbac0ff350 0x7fdbac19c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53118/0 (socket says 172.21.15.88:53118) 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.214+0000 7fdbb091c700 1 -- 172.21.15.88:0/3383866832 learned_addr learned my addr 172.21.15.88:0/3383866832 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.214+0000 7fdbabfff700 1 --2- 172.21.15.88:0/3383866832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbac0ff720 0x7fdbac19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.214+0000 7fdbb091c700 1 -- 172.21.15.88:0/3383866832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbac0ff720 msgr2=0x7fdbac19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.214+0000 7fdbb091c700 1 --2- 172.21.15.88:0/3383866832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbac0ff720 0x7fdbac19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.214+0000 7fdbb091c700 1 -- 172.21.15.88:0/3383866832 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb980095f0 con 0x7fdbac0ff350 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.214+0000 7fdbb091c700 1 --2- 172.21.15.88:0/3383866832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbac0ff350 0x7fdbac19c410 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdba000d8d0 tx=0x7fdba000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.214+0000 7fdba9ffb700 1 -- 172.21.15.88:0/3383866832 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba0009940 con 0x7fdbac0ff350 2024-02-19T19:38:52.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.214+0000 7fdba9ffb700 1 -- 172.21.15.88:0/3383866832 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdba0010460 con 0x7fdbac0ff350 2024-02-19T19:38:52.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.214+0000 7fdbb2b80700 1 -- 172.21.15.88:0/3383866832 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac1a10a0 con 0x7fdbac0ff350 2024-02-19T19:38:52.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.214+0000 7fdba9ffb700 1 -- 172.21.15.88:0/3383866832 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdba000f5f0 con 0x7fdbac0ff350 2024-02-19T19:38:52.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.214+0000 7fdbb2b80700 1 -- 172.21.15.88:0/3383866832 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac1a15f0 con 0x7fdbac0ff350 2024-02-19T19:38:52.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.215+0000 7fdbb2b80700 1 -- 172.21.15.88:0/3383866832 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbac103ae0 con 0x7fdbac0ff350 2024-02-19T19:38:52.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.216+0000 7fdba9ffb700 1 -- 172.21.15.88:0/3383866832 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdba00105d0 con 0x7fdbac0ff350 2024-02-19T19:38:52.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.216+0000 7fdba9ffb700 1 --2- 172.21.15.88:0/3383866832 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb9c06d8a0 0x7fdb9c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:52.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.216+0000 7fdbabfff700 1 --2- 172.21.15.88:0/3383866832 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb9c06d8a0 0x7fdb9c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:52.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.216+0000 7fdba9ffb700 1 -- 172.21.15.88:0/3383866832 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdba008d540 con 0x7fdbac0ff350 2024-02-19T19:38:52.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.217+0000 7fdbabfff700 1 --2- 172.21.15.88:0/3383866832 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb9c06d8a0 0x7fdb9c06fd50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdbac19da30 tx=0x7fdb9801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:52.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.221+0000 7fdba9ffb700 1 -- 172.21.15.88:0/3383866832 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdba0056cf0 con 0x7fdbac0ff350 2024-02-19T19:38:52.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:51 smithi132 ceph-mon[70676]: from='client.35906 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:52.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:51 smithi132 ceph-mon[70676]: from='client.35910 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:52.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:51 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/522439384' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:38:52.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.437+0000 7fdbb2b80700 1 -- 172.21.15.88:0/3383866832 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbac1a1810 con 0x7fdb9c06d8a0 2024-02-19T19:38:52.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.438+0000 7fdba9ffb700 1 -- 172.21.15.88:0/3383866832 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdbac1a1810 con 0x7fdb9c06d8a0 2024-02-19T19:38:52.439 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:38:52.439 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:38:52.439 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:38:52.439 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:38:52.439 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:38:52.439 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:38:52.439 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:38:52.439 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:38:52.440 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:38:52.440 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:38:52.440 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:38:52.440 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:38:52.440 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:38:52.440 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:38:52.440 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:38:52.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.440+0000 7fdbb2b80700 1 -- 172.21.15.88:0/3383866832 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb9c06d8a0 msgr2=0x7fdb9c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:52.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.440+0000 7fdbb2b80700 1 --2- 172.21.15.88:0/3383866832 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb9c06d8a0 0x7fdb9c06fd50 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fdbac19da30 tx=0x7fdb9801a040 comp rx=0 tx=0).stop 2024-02-19T19:38:52.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.440+0000 7fdbb2b80700 1 -- 172.21.15.88:0/3383866832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbac0ff350 msgr2=0x7fdbac19c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:52.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.440+0000 7fdbb2b80700 1 --2- 172.21.15.88:0/3383866832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbac0ff350 0x7fdbac19c410 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdba000d8d0 tx=0x7fdba000dc90 comp rx=0 tx=0).stop 2024-02-19T19:38:52.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.440+0000 7fdbb2b80700 1 -- 172.21.15.88:0/3383866832 shutdown_connections 2024-02-19T19:38:52.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.440+0000 7fdbb2b80700 1 --2- 172.21.15.88:0/3383866832 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb9c06d8a0 0x7fdb9c06fd50 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.440+0000 7fdbb2b80700 1 --2- 172.21.15.88:0/3383866832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbac0ff350 0x7fdbac19c410 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.440+0000 7fdbb2b80700 1 --2- 172.21.15.88:0/3383866832 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdbac0ff720 0x7fdbac19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.440+0000 7fdbb2b80700 1 -- 172.21.15.88:0/3383866832 >> 172.21.15.88:0/3383866832 conn(0x7fdbac0748b0 msgr2=0x7fdbac0fe500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:52.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.440+0000 7fdbb2b80700 1 -- 172.21.15.88:0/3383866832 shutdown_connections 2024-02-19T19:38:52.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.440+0000 7fdbb2b80700 1 -- 172.21.15.88:0/3383866832 wait complete. 2024-02-19T19:38:52.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.543+0000 7f27ad987700 1 -- 172.21.15.88:0/780224668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27a80ff3c0 msgr2=0x7f27a80ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:52.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.543+0000 7f27ad987700 1 --2- 172.21.15.88:0/780224668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27a80ff3c0 0x7f27a80ff790 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f279c009910 tx=0x7f279c009c20 comp rx=0 tx=0).stop 2024-02-19T19:38:52.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.543+0000 7f27ad987700 1 -- 172.21.15.88:0/780224668 shutdown_connections 2024-02-19T19:38:52.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.543+0000 7f27ad987700 1 --2- 172.21.15.88:0/780224668 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27a80ffd60 0x7f27a8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.543+0000 7f27ad987700 1 --2- 172.21.15.88:0/780224668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27a80ff3c0 0x7f27a80ff790 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.543+0000 7f27ad987700 1 -- 172.21.15.88:0/780224668 >> 172.21.15.88:0/780224668 conn(0x7f27a80fac80 msgr2=0x7f27a80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:52.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.543+0000 7f27ad987700 1 -- 172.21.15.88:0/780224668 shutdown_connections 2024-02-19T19:38:52.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.543+0000 7f27ad987700 1 -- 172.21.15.88:0/780224668 wait complete. 2024-02-19T19:38:52.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27ad987700 1 Processor -- start 2024-02-19T19:38:52.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27ad987700 1 -- start start 2024-02-19T19:38:52.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27ad987700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27a80ff3c0 0x7f27a8196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:52.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27ad987700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27a80ffd60 0x7f27a8197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27ad987700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a8197850 con 0x7f27a80ffd60 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27ad987700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a8197990 con 0x7f27a80ff3c0 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27a67fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27a80ffd60 0x7f27a8197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27a67fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27a80ffd60 0x7f27a8197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53136/0 (socket says 172.21.15.88:53136) 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27a67fc700 1 -- 172.21.15.88:0/2442638506 learned_addr learned my addr 172.21.15.88:0/2442638506 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27a6ffd700 1 --2- 172.21.15.88:0/2442638506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27a80ff3c0 0x7f27a8196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27a67fc700 1 -- 172.21.15.88:0/2442638506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27a80ff3c0 msgr2=0x7f27a8196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27a67fc700 1 --2- 172.21.15.88:0/2442638506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27a80ff3c0 0x7f27a8196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.544+0000 7f27a67fc700 1 -- 172.21.15.88:0/2442638506 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c0095f0 con 0x7f27a80ffd60 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.545+0000 7f27a67fc700 1 --2- 172.21.15.88:0/2442638506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27a80ffd60 0x7f27a8197230 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f279800d900 tx=0x7f279800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.545+0000 7f27ac985700 1 -- 172.21.15.88:0/2442638506 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2798009900 con 0x7f27a80ffd60 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.545+0000 7f27ac985700 1 -- 172.21.15.88:0/2442638506 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2798010460 con 0x7f27a80ffd60 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.545+0000 7f27ad987700 1 -- 172.21.15.88:0/2442638506 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a819fd60 con 0x7f27a80ffd60 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.545+0000 7f27ac985700 1 -- 172.21.15.88:0/2442638506 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f279800f5f0 con 0x7f27a80ffd60 2024-02-19T19:38:52.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.545+0000 7f27ad987700 1 -- 172.21.15.88:0/2442638506 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a81a0220 con 0x7f27a80ffd60 2024-02-19T19:38:52.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.546+0000 7f27ad987700 1 -- 172.21.15.88:0/2442638506 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a81076d0 con 0x7f27a80ffd60 2024-02-19T19:38:52.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.546+0000 7f27ac985700 1 -- 172.21.15.88:0/2442638506 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f27980105d0 con 0x7f27a80ffd60 2024-02-19T19:38:52.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.547+0000 7f27ac985700 1 --2- 172.21.15.88:0/2442638506 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f279006d8a0 0x7f279006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:38:52.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.547+0000 7f27ac985700 1 -- 172.21.15.88:0/2442638506 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f279808d380 con 0x7f27a80ffd60 2024-02-19T19:38:52.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.547+0000 7f27a6ffd700 1 --2- 172.21.15.88:0/2442638506 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f279006d8a0 0x7f279006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:38:52.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.547+0000 7f27a6ffd700 1 --2- 172.21.15.88:0/2442638506 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f279006d8a0 0x7f279006fd50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f279c00b5c0 tx=0x7f279c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:38:52.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.551+0000 7f27ac985700 1 -- 172.21.15.88:0/2442638506 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2798056b30 con 0x7f27a80ffd60 2024-02-19T19:38:52.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.828+0000 7f27ad987700 1 -- 172.21.15.88:0/2442638506 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f27a804f2a0 con 0x7f27a80ffd60 2024-02-19T19:38:52.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.829+0000 7f27ac985700 1 -- 172.21.15.88:0/2442638506 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f279805a1f0 con 0x7f27a80ffd60 2024-02-19T19:38:52.830 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:38:52.830 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:38:52.830 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:38:52.830 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:38:52.830 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:38:52.830 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:38:52.830 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:38:52.830 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:38:52.831 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:38:52.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.831+0000 7f27ad987700 1 -- 172.21.15.88:0/2442638506 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f279006d8a0 msgr2=0x7f279006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:52.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.831+0000 7f27ad987700 1 --2- 172.21.15.88:0/2442638506 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f279006d8a0 0x7f279006fd50 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f279c00b5c0 tx=0x7f279c01a040 comp rx=0 tx=0).stop 2024-02-19T19:38:52.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.831+0000 7f27ad987700 1 -- 172.21.15.88:0/2442638506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27a80ffd60 msgr2=0x7f27a8197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:38:52.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.831+0000 7f27ad987700 1 --2- 172.21.15.88:0/2442638506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27a80ffd60 0x7f27a8197230 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f279800d900 tx=0x7f279800dcc0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.831+0000 7f27ad987700 1 -- 172.21.15.88:0/2442638506 shutdown_connections 2024-02-19T19:38:52.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.831+0000 7f27ad987700 1 --2- 172.21.15.88:0/2442638506 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f279006d8a0 0x7f279006fd50 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.831+0000 7f27ad987700 1 --2- 172.21.15.88:0/2442638506 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f27a80ff3c0 0x7f27a8196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.831+0000 7f27ad987700 1 --2- 172.21.15.88:0/2442638506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27a80ffd60 0x7f27a8197230 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:38:52.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.831+0000 7f27ad987700 1 -- 172.21.15.88:0/2442638506 >> 172.21.15.88:0/2442638506 conn(0x7f27a80fac80 msgr2=0x7f27a80fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:38:52.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.831+0000 7f27ad987700 1 -- 172.21.15.88:0/2442638506 shutdown_connections 2024-02-19T19:38:52.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:38:52.831+0000 7f27ad987700 1 -- 172.21.15.88:0/2442638506 wait complete. 2024-02-19T19:38:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:52 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3540777835' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:38:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:52 smithi088 ceph-mon[88643]: from='client.35922 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:52 smithi088 ceph-mon[88643]: pgmap v1060: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:52 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3540777835' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:38:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:52 smithi132 ceph-mon[70676]: from='client.35922 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:38:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:52 smithi132 ceph-mon[70676]: pgmap v1060: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:54.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:53 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2442638506' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:38:54.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:53 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2442638506' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:38:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:54 smithi088 ceph-mon[88643]: pgmap v1061: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:54 smithi132 ceph-mon[70676]: pgmap v1061: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:56 smithi088 ceph-mon[88643]: pgmap v1062: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:56 smithi132 ceph-mon[70676]: pgmap v1062: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:58.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:38:58.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:38:59.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:38:59 smithi132 ceph-mon[70676]: pgmap v1063: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:38:59.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:38:59 smithi088 ceph-mon[88643]: pgmap v1063: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:00 smithi088 ceph-mon[88643]: pgmap v1064: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:00 smithi132 ceph-mon[70676]: pgmap v1064: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:02 smithi088 ceph-mon[88643]: pgmap v1065: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:03.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:02 smithi132 ceph-mon[70676]: pgmap v1065: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:04 smithi088 ceph-mon[88643]: pgmap v1066: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:04 smithi132 ceph-mon[70676]: pgmap v1066: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:06 smithi088 ceph-mon[88643]: pgmap v1067: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:07.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:06 smithi132 ceph-mon[70676]: pgmap v1067: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:08 smithi088 ceph-mon[88643]: pgmap v1068: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:09.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:08 smithi132 ceph-mon[70676]: pgmap v1068: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:10 smithi088 ceph-mon[88643]: pgmap v1069: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:11.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:10 smithi132 ceph-mon[70676]: pgmap v1069: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:12 smithi088 ceph-mon[88643]: pgmap v1070: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:13.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:12 smithi132 ceph-mon[70676]: pgmap v1070: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:39:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:39:14.982 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:14 smithi088 ceph-mon[88643]: pgmap v1071: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:14 smithi132 ceph-mon[70676]: pgmap v1071: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:16 smithi088 ceph-mon[88643]: pgmap v1072: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:17.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:16 smithi132 ceph-mon[70676]: pgmap v1072: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:18 smithi088 ceph-mon[88643]: pgmap v1073: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:19.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:18 smithi132 ceph-mon[70676]: pgmap v1073: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:20 smithi088 ceph-mon[88643]: pgmap v1074: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:20 smithi132 ceph-mon[70676]: pgmap v1074: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:22.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.947+0000 7fcd5d769700 1 -- 172.21.15.88:0/3926848696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd58101db0 msgr2=0x7fcd5810a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:22.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.947+0000 7fcd5d769700 1 --2- 172.21.15.88:0/3926848696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd58101db0 0x7fcd5810a2b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fcd48009910 tx=0x7fcd48009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:22.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.947+0000 7fcd5d769700 1 -- 172.21.15.88:0/3926848696 shutdown_connections 2024-02-19T19:39:22.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.947+0000 7fcd5d769700 1 --2- 172.21.15.88:0/3926848696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd58101db0 0x7fcd5810a2b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:22.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.947+0000 7fcd5d769700 1 --2- 172.21.15.88:0/3926848696 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd581014a0 0x7fcd58101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:22.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.947+0000 7fcd5d769700 1 -- 172.21.15.88:0/3926848696 >> 172.21.15.88:0/3926848696 conn(0x7fcd580fac80 msgr2=0x7fcd580fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:22.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.948+0000 7fcd5d769700 1 -- 172.21.15.88:0/3926848696 shutdown_connections 2024-02-19T19:39:22.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.948+0000 7fcd5d769700 1 -- 172.21.15.88:0/3926848696 wait complete. 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.948+0000 7fcd5d769700 1 Processor -- start 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd5d769700 1 -- start start 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd5d769700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd581014a0 0x7fcd58195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd5d769700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd58101db0 0x7fcd58196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd5d769700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd58196a70 con 0x7fcd58101db0 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd5d769700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd5819a800 con 0x7fcd581014a0 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd567fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd58101db0 0x7fcd58196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd567fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd58101db0 0x7fcd58196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52728/0 (socket says 172.21.15.88:52728) 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd567fc700 1 -- 172.21.15.88:0/4091380277 learned_addr learned my addr 172.21.15.88:0/4091380277 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd56ffd700 1 --2- 172.21.15.88:0/4091380277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd581014a0 0x7fcd58195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd567fc700 1 -- 172.21.15.88:0/4091380277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd581014a0 msgr2=0x7fcd58195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd567fc700 1 --2- 172.21.15.88:0/4091380277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd581014a0 0x7fcd58195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:22.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd567fc700 1 -- 172.21.15.88:0/4091380277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd480095f0 con 0x7fcd58101db0 2024-02-19T19:39:22.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.949+0000 7fcd567fc700 1 --2- 172.21.15.88:0/4091380277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd58101db0 0x7fcd58196390 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fcd48006010 tx=0x7fcd4800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:22.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.950+0000 7fcd3ffff700 1 -- 172.21.15.88:0/4091380277 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd4801d070 con 0x7fcd58101db0 2024-02-19T19:39:22.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.950+0000 7fcd3ffff700 1 -- 172.21.15.88:0/4091380277 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd48005dc0 con 0x7fcd58101db0 2024-02-19T19:39:22.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.950+0000 7fcd5d769700 1 -- 172.21.15.88:0/4091380277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd5819aa80 con 0x7fcd58101db0 2024-02-19T19:39:22.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.950+0000 7fcd3ffff700 1 -- 172.21.15.88:0/4091380277 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd48017400 con 0x7fcd58101db0 2024-02-19T19:39:22.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.950+0000 7fcd5d769700 1 -- 172.21.15.88:0/4091380277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd5819af70 con 0x7fcd58101db0 2024-02-19T19:39:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.951+0000 7fcd5d769700 1 -- 172.21.15.88:0/4091380277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd58107920 con 0x7fcd58101db0 2024-02-19T19:39:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.951+0000 7fcd3ffff700 1 -- 172.21.15.88:0/4091380277 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcd48003a60 con 0x7fcd58101db0 2024-02-19T19:39:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.951+0000 7fcd3ffff700 1 --2- 172.21.15.88:0/4091380277 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd4006d8a0 0x7fcd4006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.952+0000 7fcd3ffff700 1 -- 172.21.15.88:0/4091380277 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcd4808e9d0 con 0x7fcd58101db0 2024-02-19T19:39:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.952+0000 7fcd56ffd700 1 --2- 172.21.15.88:0/4091380277 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd4006d8a0 0x7fcd4006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:22.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.952+0000 7fcd56ffd700 1 --2- 172.21.15.88:0/4091380277 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd4006d8a0 0x7fcd4006fd50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c005950 tx=0x7fcd4c00b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:22.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:22.956+0000 7fcd3ffff700 1 -- 172.21.15.88:0/4091380277 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcd48058180 con 0x7fcd58101db0 2024-02-19T19:39:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:22 smithi088 ceph-mon[88643]: pgmap v1075: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:23.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.171+0000 7fcd5d769700 1 -- 172.21.15.88:0/4091380277 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd581972c0 con 0x7fcd4006d8a0 2024-02-19T19:39:23.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.172+0000 7fcd3ffff700 1 -- 172.21.15.88:0/4091380277 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcd581972c0 con 0x7fcd4006d8a0 2024-02-19T19:39:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.174+0000 7fcd5d769700 1 -- 172.21.15.88:0/4091380277 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd4006d8a0 msgr2=0x7fcd4006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.174+0000 7fcd5d769700 1 --2- 172.21.15.88:0/4091380277 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd4006d8a0 0x7fcd4006fd50 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c005950 tx=0x7fcd4c00b410 comp rx=0 tx=0).stop 2024-02-19T19:39:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.174+0000 7fcd5d769700 1 -- 172.21.15.88:0/4091380277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd58101db0 msgr2=0x7fcd58196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.174+0000 7fcd5d769700 1 --2- 172.21.15.88:0/4091380277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd58101db0 0x7fcd58196390 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fcd48006010 tx=0x7fcd4800bd70 comp rx=0 tx=0).stop 2024-02-19T19:39:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.175+0000 7fcd5d769700 1 -- 172.21.15.88:0/4091380277 shutdown_connections 2024-02-19T19:39:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.175+0000 7fcd5d769700 1 --2- 172.21.15.88:0/4091380277 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd4006d8a0 0x7fcd4006fd50 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.175+0000 7fcd5d769700 1 --2- 172.21.15.88:0/4091380277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcd581014a0 0x7fcd58195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.175+0000 7fcd5d769700 1 --2- 172.21.15.88:0/4091380277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd58101db0 0x7fcd58196390 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.175+0000 7fcd5d769700 1 -- 172.21.15.88:0/4091380277 >> 172.21.15.88:0/4091380277 conn(0x7fcd580fac80 msgr2=0x7fcd58104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.175+0000 7fcd5d769700 1 -- 172.21.15.88:0/4091380277 shutdown_connections 2024-02-19T19:39:23.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.175+0000 7fcd5d769700 1 -- 172.21.15.88:0/4091380277 wait complete. 2024-02-19T19:39:23.186 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:39:23.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:22 smithi132 ceph-mon[70676]: pgmap v1075: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:23.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.280+0000 7f5ad1fea700 1 -- 172.21.15.88:0/1805562246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5acc068540 msgr2=0x7f5acc0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.280+0000 7f5ad1fea700 1 --2- 172.21.15.88:0/1805562246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5acc068540 0x7f5acc0689b0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009910 tx=0x7f5ab4009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:23.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.280+0000 7f5ad1fea700 1 -- 172.21.15.88:0/1805562246 shutdown_connections 2024-02-19T19:39:23.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.280+0000 7f5ad1fea700 1 --2- 172.21.15.88:0/1805562246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5acc068540 0x7f5acc0689b0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.280+0000 7f5ad1fea700 1 --2- 172.21.15.88:0/1805562246 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc106320 0x7f5acc1066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.280+0000 7f5ad1fea700 1 -- 172.21.15.88:0/1805562246 >> 172.21.15.88:0/1805562246 conn(0x7f5acc075130 msgr2=0x7f5acc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:23.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.280+0000 7f5ad1fea700 1 -- 172.21.15.88:0/1805562246 shutdown_connections 2024-02-19T19:39:23.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.280+0000 7f5ad1fea700 1 -- 172.21.15.88:0/1805562246 wait complete. 2024-02-19T19:39:23.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.280+0000 7f5ad1fea700 1 Processor -- start 2024-02-19T19:39:23.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5ad1fea700 1 -- start start 2024-02-19T19:39:23.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5ad1fea700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5acc068540 0x7f5acc198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:23.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5ad1fea700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc106320 0x7f5acc198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:23.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5ad1fea700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc198c60 con 0x7f5acc068540 2024-02-19T19:39:23.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5ad1fea700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5acc19c9f0 con 0x7f5acc106320 2024-02-19T19:39:23.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5acb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5acc068540 0x7f5acc198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:23.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5acb7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5acc068540 0x7f5acc198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52746/0 (socket says 172.21.15.88:52746) 2024-02-19T19:39:23.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5acb7fe700 1 -- 172.21.15.88:0/736115717 learned_addr learned my addr 172.21.15.88:0/736115717 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:23.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5acb7fe700 1 -- 172.21.15.88:0/736115717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc106320 msgr2=0x7f5acc198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5acb7fe700 1 --2- 172.21.15.88:0/736115717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc106320 0x7f5acc198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5acb7fe700 1 -- 172.21.15.88:0/736115717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab40095f0 con 0x7f5acc068540 2024-02-19T19:39:23.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.281+0000 7f5acb7fe700 1 --2- 172.21.15.88:0/736115717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5acc068540 0x7f5acc198040 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00d900 tx=0x7f5abc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:23.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.282+0000 7f5ac97fa700 1 -- 172.21.15.88:0/736115717 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5abc004a00 con 0x7f5acc068540 2024-02-19T19:39:23.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.282+0000 7f5ac97fa700 1 -- 172.21.15.88:0/736115717 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5abc0054e0 con 0x7f5acc068540 2024-02-19T19:39:23.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.282+0000 7f5ad1fea700 1 -- 172.21.15.88:0/736115717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5acc19ccd0 con 0x7f5acc068540 2024-02-19T19:39:23.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.282+0000 7f5ac97fa700 1 -- 172.21.15.88:0/736115717 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5abc009d60 con 0x7f5acc068540 2024-02-19T19:39:23.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.282+0000 7f5ad1fea700 1 -- 172.21.15.88:0/736115717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5acc19d220 con 0x7f5acc068540 2024-02-19T19:39:23.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.283+0000 7f5ad1fea700 1 -- 172.21.15.88:0/736115717 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5acc04f2a0 con 0x7f5acc068540 2024-02-19T19:39:23.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.283+0000 7f5ac97fa700 1 -- 172.21.15.88:0/736115717 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5abc005020 con 0x7f5acc068540 2024-02-19T19:39:23.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.283+0000 7f5ac97fa700 1 --2- 172.21.15.88:0/736115717 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ab806d8a0 0x7f5ab806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:23.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.284+0000 7f5ac97fa700 1 -- 172.21.15.88:0/736115717 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5abc08d690 con 0x7f5acc068540 2024-02-19T19:39:23.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.284+0000 7f5ac2ffd700 1 --2- 172.21.15.88:0/736115717 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ab806d8a0 0x7f5ab806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:23.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.284+0000 7f5ac2ffd700 1 --2- 172.21.15.88:0/736115717 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ab806d8a0 0x7f5ab806fd50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400b5c0 tx=0x7f5ab4005960 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:23.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.288+0000 7f5ac97fa700 1 -- 172.21.15.88:0/736115717 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5abc056ec0 con 0x7f5acc068540 2024-02-19T19:39:23.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.500+0000 7f5ad1fea700 1 -- 172.21.15.88:0/736115717 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5acc19d5a0 con 0x7f5ab806d8a0 2024-02-19T19:39:23.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.501+0000 7f5ac97fa700 1 -- 172.21.15.88:0/736115717 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5acc19d5a0 con 0x7f5ab806d8a0 2024-02-19T19:39:23.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.503+0000 7f5ad1fea700 1 -- 172.21.15.88:0/736115717 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ab806d8a0 msgr2=0x7f5ab806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.503+0000 7f5ad1fea700 1 --2- 172.21.15.88:0/736115717 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ab806d8a0 0x7f5ab806fd50 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f5ab400b5c0 tx=0x7f5ab4005960 comp rx=0 tx=0).stop 2024-02-19T19:39:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.503+0000 7f5ad1fea700 1 -- 172.21.15.88:0/736115717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5acc068540 msgr2=0x7f5acc198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.503+0000 7f5ad1fea700 1 --2- 172.21.15.88:0/736115717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5acc068540 0x7f5acc198040 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f5abc00d900 tx=0x7f5abc00dcc0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.504+0000 7f5ad1fea700 1 -- 172.21.15.88:0/736115717 shutdown_connections 2024-02-19T19:39:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.504+0000 7f5ad1fea700 1 --2- 172.21.15.88:0/736115717 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5ab806d8a0 0x7f5ab806fd50 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.504+0000 7f5ad1fea700 1 --2- 172.21.15.88:0/736115717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5acc068540 0x7f5acc198040 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.504+0000 7f5ad1fea700 1 --2- 172.21.15.88:0/736115717 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5acc106320 0x7f5acc198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.504+0000 7f5ad1fea700 1 -- 172.21.15.88:0/736115717 >> 172.21.15.88:0/736115717 conn(0x7f5acc075130 msgr2=0x7f5acc0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.504+0000 7f5ad1fea700 1 -- 172.21.15.88:0/736115717 shutdown_connections 2024-02-19T19:39:23.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.504+0000 7f5ad1fea700 1 -- 172.21.15.88:0/736115717 wait complete. 2024-02-19T19:39:23.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.614+0000 7f0113db9700 1 -- 172.21.15.88:0/2595961335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010c100460 msgr2=0x7f010c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.614+0000 7f0113db9700 1 --2- 172.21.15.88:0/2595961335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010c100460 0x7f010c100830 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009910 tx=0x7f00fc009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:23.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.614+0000 7f0113db9700 1 -- 172.21.15.88:0/2595961335 shutdown_connections 2024-02-19T19:39:23.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.614+0000 7f0113db9700 1 --2- 172.21.15.88:0/2595961335 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010c100d70 0x7f010c1094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.614+0000 7f0113db9700 1 --2- 172.21.15.88:0/2595961335 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010c100460 0x7f010c100830 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.614+0000 7f0113db9700 1 -- 172.21.15.88:0/2595961335 >> 172.21.15.88:0/2595961335 conn(0x7f010c0748b0 msgr2=0x7f010c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:23.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.614+0000 7f0113db9700 1 -- 172.21.15.88:0/2595961335 shutdown_connections 2024-02-19T19:39:23.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.614+0000 7f0113db9700 1 -- 172.21.15.88:0/2595961335 wait complete. 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0113db9700 1 Processor -- start 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0113db9700 1 -- start start 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0113db9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010c100460 0x7f010c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0113db9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010c100d70 0x7f010c19b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0113db9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c19b6a0 con 0x7f010c100d70 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0113db9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010c194b50 con 0x7f010c100460 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0111354700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010c100d70 0x7f010c19b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0111354700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010c100d70 0x7f010c19b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52766/0 (socket says 172.21.15.88:52766) 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0111354700 1 -- 172.21.15.88:0/4114798358 learned_addr learned my addr 172.21.15.88:0/4114798358 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0111b55700 1 --2- 172.21.15.88:0/4114798358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010c100460 0x7f010c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0111354700 1 -- 172.21.15.88:0/4114798358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010c100460 msgr2=0x7f010c19aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0111354700 1 --2- 172.21.15.88:0/4114798358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010c100460 0x7f010c19aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.615+0000 7f0111354700 1 -- 172.21.15.88:0/4114798358 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc0095f0 con 0x7f010c100d70 2024-02-19T19:39:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.616+0000 7f0111354700 1 --2- 172.21.15.88:0/4114798358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010c100d70 0x7f010c19b010 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f010800c8a0 tx=0x7f010800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.616+0000 7f0102ffd700 1 -- 172.21.15.88:0/4114798358 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f010800cea0 con 0x7f010c100d70 2024-02-19T19:39:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.616+0000 7f0102ffd700 1 -- 172.21.15.88:0/4114798358 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0108004830 con 0x7f010c100d70 2024-02-19T19:39:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.616+0000 7f0113db9700 1 -- 172.21.15.88:0/4114798358 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010c194e30 con 0x7f010c100d70 2024-02-19T19:39:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.616+0000 7f0102ffd700 1 -- 172.21.15.88:0/4114798358 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01080055b0 con 0x7f010c100d70 2024-02-19T19:39:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.616+0000 7f0113db9700 1 -- 172.21.15.88:0/4114798358 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010c195380 con 0x7f010c100d70 2024-02-19T19:39:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.617+0000 7f0113db9700 1 -- 172.21.15.88:0/4114798358 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010c106bd0 con 0x7f010c100d70 2024-02-19T19:39:23.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.617+0000 7f0102ffd700 1 -- 172.21.15.88:0/4114798358 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f01080049a0 con 0x7f010c100d70 2024-02-19T19:39:23.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.617+0000 7f0102ffd700 1 --2- 172.21.15.88:0/4114798358 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f00f806d8a0 0x7f00f806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:23.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.618+0000 7f0102ffd700 1 -- 172.21.15.88:0/4114798358 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f010808d0f0 con 0x7f010c100d70 2024-02-19T19:39:23.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.618+0000 7f0111b55700 1 --2- 172.21.15.88:0/4114798358 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f00f806d8a0 0x7f00f806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:23.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.618+0000 7f0111b55700 1 --2- 172.21.15.88:0/4114798358 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f00f806d8a0 0x7f00f806fd50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00b5c0 tx=0x7f00fc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:23.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.622+0000 7f0102ffd700 1 -- 172.21.15.88:0/4114798358 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f010805a450 con 0x7f010c100d70 2024-02-19T19:39:23.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.830+0000 7f0113db9700 1 -- 172.21.15.88:0/4114798358 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f010c195650 con 0x7f00f806d8a0 2024-02-19T19:39:23.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.841+0000 7f0102ffd700 1 -- 172.21.15.88:0/4114798358 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f010c195650 con 0x7f00f806d8a0 2024-02-19T19:39:23.842 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:39:23.842 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (36m) 12m ago 46m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (47m) 12m ago 47m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (29m) 12m ago 45m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (32m) 12m ago 47m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (32m) 12m ago 45m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (35m) 12m ago 46m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (28m) 12m ago 40m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (28m) 12m ago 40m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (28m) 12m ago 40m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (13m) 12m ago 40m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (38m) 12m ago 48m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (37m) 12m ago 45m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (34m) 12m ago 48m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (33m) 12m ago 45m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (37m) 12m ago 47m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:39:23.843 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (37m) 12m ago 45m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:39:23.844 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (32m) 12m ago 44m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:39:23.844 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (31m) 12m ago 44m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:39:23.844 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (30m) 12m ago 43m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:39:23.844 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (30m) 12m ago 43m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:39:23.844 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (29m) 12m ago 42m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:39:23.844 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (29m) 12m ago 42m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:39:23.844 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (36m) 12m ago 45m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:39:23.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.843+0000 7f0113db9700 1 -- 172.21.15.88:0/4114798358 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f00f806d8a0 msgr2=0x7f00f806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.843+0000 7f0113db9700 1 --2- 172.21.15.88:0/4114798358 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f00f806d8a0 0x7f00f806fd50 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f00fc00b5c0 tx=0x7f00fc01a040 comp rx=0 tx=0).stop 2024-02-19T19:39:23.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.843+0000 7f0113db9700 1 -- 172.21.15.88:0/4114798358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010c100d70 msgr2=0x7f010c19b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.843+0000 7f0113db9700 1 --2- 172.21.15.88:0/4114798358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010c100d70 0x7f010c19b010 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f010800c8a0 tx=0x7f010800cc60 comp rx=0 tx=0).stop 2024-02-19T19:39:23.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.844+0000 7f0113db9700 1 -- 172.21.15.88:0/4114798358 shutdown_connections 2024-02-19T19:39:23.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.844+0000 7f0113db9700 1 --2- 172.21.15.88:0/4114798358 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f00f806d8a0 0x7f00f806fd50 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.844+0000 7f0113db9700 1 --2- 172.21.15.88:0/4114798358 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f010c100460 0x7f010c19aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.844+0000 7f0113db9700 1 --2- 172.21.15.88:0/4114798358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010c100d70 0x7f010c19b010 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.844+0000 7f0113db9700 1 -- 172.21.15.88:0/4114798358 >> 172.21.15.88:0/4114798358 conn(0x7f010c0748b0 msgr2=0x7f010c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:23.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.844+0000 7f0113db9700 1 -- 172.21.15.88:0/4114798358 shutdown_connections 2024-02-19T19:39:23.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.844+0000 7f0113db9700 1 -- 172.21.15.88:0/4114798358 wait complete. 2024-02-19T19:39:23.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.945+0000 7f88273d3700 1 -- 172.21.15.88:0/1051640974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8820100e40 msgr2=0x7f8820104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.945+0000 7f88273d3700 1 --2- 172.21.15.88:0/1051640974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8820100e40 0x7f8820104e10 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8810009910 tx=0x7f8810009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:23.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 -- 172.21.15.88:0/1051640974 shutdown_connections 2024-02-19T19:39:23.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 --2- 172.21.15.88:0/1051640974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8820100e40 0x7f8820104e10 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 --2- 172.21.15.88:0/1051640974 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88201004a0 0x7f8820100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 -- 172.21.15.88:0/1051640974 >> 172.21.15.88:0/1051640974 conn(0x7f88200fbd40 msgr2=0x7f88200fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:23.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 -- 172.21.15.88:0/1051640974 shutdown_connections 2024-02-19T19:39:23.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 -- 172.21.15.88:0/1051640974 wait complete. 2024-02-19T19:39:23.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 Processor -- start 2024-02-19T19:39:23.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 -- start start 2024-02-19T19:39:23.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88201004a0 0x7f8820073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8820100e40 0x7f8820073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8820073ad0 con 0x7f8820100e40 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.946+0000 7f88273d3700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8820073c10 con 0x7f88201004a0 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f882496e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8820100e40 0x7f8820073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f882516f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88201004a0 0x7f8820073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f882496e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8820100e40 0x7f8820073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52788/0 (socket says 172.21.15.88:52788) 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f882496e700 1 -- 172.21.15.88:0/937839058 learned_addr learned my addr 172.21.15.88:0/937839058 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f882496e700 1 -- 172.21.15.88:0/937839058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88201004a0 msgr2=0x7f8820073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f882496e700 1 --2- 172.21.15.88:0/937839058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88201004a0 0x7f8820073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f882496e700 1 -- 172.21.15.88:0/937839058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88100095f0 con 0x7f8820100e40 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f882496e700 1 --2- 172.21.15.88:0/937839058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8820100e40 0x7f8820073590 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f88100092e0 tx=0x7f881000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f88167fc700 1 -- 172.21.15.88:0/937839058 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f881001d070 con 0x7f8820100e40 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f88167fc700 1 -- 172.21.15.88:0/937839058 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8810005dc0 con 0x7f8820100e40 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f88273d3700 1 -- 172.21.15.88:0/937839058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8820073e90 con 0x7f8820100e40 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f88167fc700 1 -- 172.21.15.88:0/937839058 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8810017400 con 0x7f8820100e40 2024-02-19T19:39:23.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.947+0000 7f88273d3700 1 -- 172.21.15.88:0/937839058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88201a23b0 con 0x7f8820100e40 2024-02-19T19:39:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.948+0000 7f88273d3700 1 -- 172.21.15.88:0/937839058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8820108810 con 0x7f8820100e40 2024-02-19T19:39:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.949+0000 7f88167fc700 1 -- 172.21.15.88:0/937839058 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f881000f6f0 con 0x7f8820100e40 2024-02-19T19:39:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.949+0000 7f88167fc700 1 --2- 172.21.15.88:0/937839058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f880c06d8a0 0x7f880c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:23.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.949+0000 7f88167fc700 1 -- 172.21.15.88:0/937839058 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f881008e910 con 0x7f8820100e40 2024-02-19T19:39:23.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.949+0000 7f882516f700 1 --2- 172.21.15.88:0/937839058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f880c06d8a0 0x7f880c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:23.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.949+0000 7f882516f700 1 --2- 172.21.15.88:0/937839058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f880c06d8a0 0x7f880c06fd50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f881c005950 tx=0x7f881c00b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:23.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:23.954+0000 7f88167fc700 1 -- 172.21.15.88:0/937839058 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f88100580c0 con 0x7f8820100e40 2024-02-19T19:39:24.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:23 smithi088 ceph-mon[88643]: from='client.35930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:24.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:23 smithi088 ceph-mon[88643]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:24.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:23 smithi132 ceph-mon[70676]: from='client.35930 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:24.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:23 smithi132 ceph-mon[70676]: from='client.35934 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:24.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.232+0000 7f88273d3700 1 -- 172.21.15.88:0/937839058 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8820074650 con 0x7f8820100e40 2024-02-19T19:39:24.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.233+0000 7f88167fc700 1 -- 172.21.15.88:0/937839058 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f881005b780 con 0x7f8820100e40 2024-02-19T19:39:24.234 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:39:24.234 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:39:24.234 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:39:24.234 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:39:24.234 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:39:24.234 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:39:24.234 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:39:24.234 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:39:24.234 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:39:24.235 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:39:24.235 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:39:24.235 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:39:24.235 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:39:24.235 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:39:24.235 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:39:24.235 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:39:24.235 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:39:24.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.235+0000 7f88273d3700 1 -- 172.21.15.88:0/937839058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f880c06d8a0 msgr2=0x7f880c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:24.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.235+0000 7f88273d3700 1 --2- 172.21.15.88:0/937839058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f880c06d8a0 0x7f880c06fd50 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f881c005950 tx=0x7f881c00b410 comp rx=0 tx=0).stop 2024-02-19T19:39:24.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.235+0000 7f88273d3700 1 -- 172.21.15.88:0/937839058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8820100e40 msgr2=0x7f8820073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:24.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.235+0000 7f88273d3700 1 --2- 172.21.15.88:0/937839058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8820100e40 0x7f8820073590 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f88100092e0 tx=0x7f881000bd70 comp rx=0 tx=0).stop 2024-02-19T19:39:24.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.235+0000 7f88273d3700 1 -- 172.21.15.88:0/937839058 shutdown_connections 2024-02-19T19:39:24.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.235+0000 7f88273d3700 1 --2- 172.21.15.88:0/937839058 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f880c06d8a0 0x7f880c06fd50 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.235+0000 7f88273d3700 1 --2- 172.21.15.88:0/937839058 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f88201004a0 0x7f8820073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.235+0000 7f88273d3700 1 --2- 172.21.15.88:0/937839058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8820100e40 0x7f8820073590 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.235+0000 7f88273d3700 1 -- 172.21.15.88:0/937839058 >> 172.21.15.88:0/937839058 conn(0x7f88200fbd40 msgr2=0x7f88200fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:24.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.235+0000 7f88273d3700 1 -- 172.21.15.88:0/937839058 shutdown_connections 2024-02-19T19:39:24.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.235+0000 7f88273d3700 1 -- 172.21.15.88:0/937839058 wait complete. 2024-02-19T19:39:24.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.339+0000 7f7e62d9d700 1 -- 172.21.15.88:0/2544080330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c1002b0 msgr2=0x7f7e5c100680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:24.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.339+0000 7f7e62d9d700 1 --2- 172.21.15.88:0/2544080330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c1002b0 0x7f7e5c100680 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c009910 tx=0x7f7e4c009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:24.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.339+0000 7f7e62d9d700 1 -- 172.21.15.88:0/2544080330 shutdown_connections 2024-02-19T19:39:24.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.339+0000 7f7e62d9d700 1 --2- 172.21.15.88:0/2544080330 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e5c100c50 0x7f7e5c104c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.339+0000 7f7e62d9d700 1 --2- 172.21.15.88:0/2544080330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c1002b0 0x7f7e5c100680 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.339+0000 7f7e62d9d700 1 -- 172.21.15.88:0/2544080330 >> 172.21.15.88:0/2544080330 conn(0x7f7e5c0fbb50 msgr2=0x7f7e5c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:24.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.339+0000 7f7e62d9d700 1 -- 172.21.15.88:0/2544080330 shutdown_connections 2024-02-19T19:39:24.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.339+0000 7f7e62d9d700 1 -- 172.21.15.88:0/2544080330 wait complete. 2024-02-19T19:39:24.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e62d9d700 1 Processor -- start 2024-02-19T19:39:24.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e62d9d700 1 -- start start 2024-02-19T19:39:24.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e62d9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e5c1002b0 0x7f7e5c19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:24.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e62d9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c100c50 0x7f7e5c19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:24.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e62d9d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c19c5b0 con 0x7f7e5c100c50 2024-02-19T19:39:24.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e62d9d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c195a60 con 0x7f7e5c1002b0 2024-02-19T19:39:24.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e6159a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c100c50 0x7f7e5c19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e6159a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c100c50 0x7f7e5c19bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52810/0 (socket says 172.21.15.88:52810) 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e6159a700 1 -- 172.21.15.88:0/1993908113 learned_addr learned my addr 172.21.15.88:0/1993908113 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e61d9b700 1 --2- 172.21.15.88:0/1993908113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e5c1002b0 0x7f7e5c19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e6159a700 1 -- 172.21.15.88:0/1993908113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e5c1002b0 msgr2=0x7f7e5c19b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e6159a700 1 --2- 172.21.15.88:0/1993908113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e5c1002b0 0x7f7e5c19b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.340+0000 7f7e6159a700 1 -- 172.21.15.88:0/1993908113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4c0095f0 con 0x7f7e5c100c50 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.341+0000 7f7e6159a700 1 --2- 172.21.15.88:0/1993908113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c100c50 0x7f7e5c19bf20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800d900 tx=0x7f7e5800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.341+0000 7f7e52ffd700 1 -- 172.21.15.88:0/1993908113 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e58009900 con 0x7f7e5c100c50 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.341+0000 7f7e52ffd700 1 -- 172.21.15.88:0/1993908113 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e58010460 con 0x7f7e5c100c50 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.341+0000 7f7e62d9d700 1 -- 172.21.15.88:0/1993908113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c195d40 con 0x7f7e5c100c50 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.341+0000 7f7e52ffd700 1 -- 172.21.15.88:0/1993908113 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e5800f5f0 con 0x7f7e5c100c50 2024-02-19T19:39:24.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.341+0000 7f7e62d9d700 1 -- 172.21.15.88:0/1993908113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c196290 con 0x7f7e5c100c50 2024-02-19T19:39:24.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.342+0000 7f7e62d9d700 1 -- 172.21.15.88:0/1993908113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c1085c0 con 0x7f7e5c100c50 2024-02-19T19:39:24.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.342+0000 7f7e52ffd700 1 -- 172.21.15.88:0/1993908113 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7e580105d0 con 0x7f7e5c100c50 2024-02-19T19:39:24.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.343+0000 7f7e52ffd700 1 --2- 172.21.15.88:0/1993908113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e4806d8a0 0x7f7e4806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:24.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.343+0000 7f7e52ffd700 1 -- 172.21.15.88:0/1993908113 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7e5808d0e0 con 0x7f7e5c100c50 2024-02-19T19:39:24.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.343+0000 7f7e61d9b700 1 --2- 172.21.15.88:0/1993908113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e4806d8a0 0x7f7e4806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:24.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.343+0000 7f7e61d9b700 1 --2- 172.21.15.88:0/1993908113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e4806d8a0 0x7f7e4806fd50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c006010 tx=0x7f7e4c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:24.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.348+0000 7f7e52ffd700 1 -- 172.21.15.88:0/1993908113 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7e5805a3c0 con 0x7f7e5c100c50 2024-02-19T19:39:24.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.595+0000 7f7e62d9d700 1 -- 172.21.15.88:0/1993908113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7e5c062380 con 0x7f7e5c100c50 2024-02-19T19:39:24.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.595+0000 7f7e52ffd700 1 -- 172.21.15.88:0/1993908113 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f7e58059f50 con 0x7f7e5c100c50 2024-02-19T19:39:24.596 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:39:24.596 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:39:24.596 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:39:24.597 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:39:24.598 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:39:24.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:39:24.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:39:24.599 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:39:24.599 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:39:24.599 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:39:24.599 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:39:24.599 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:39:24.599 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:39:24.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.597+0000 7f7e62d9d700 1 -- 172.21.15.88:0/1993908113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e4806d8a0 msgr2=0x7f7e4806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:24.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.597+0000 7f7e62d9d700 1 --2- 172.21.15.88:0/1993908113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e4806d8a0 0x7f7e4806fd50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c006010 tx=0x7f7e4c01a040 comp rx=0 tx=0).stop 2024-02-19T19:39:24.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.597+0000 7f7e62d9d700 1 -- 172.21.15.88:0/1993908113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c100c50 msgr2=0x7f7e5c19bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:24.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.597+0000 7f7e62d9d700 1 --2- 172.21.15.88:0/1993908113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c100c50 0x7f7e5c19bf20 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7e5800d900 tx=0x7f7e5800dc10 comp rx=0 tx=0).stop 2024-02-19T19:39:24.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.597+0000 7f7e62d9d700 1 -- 172.21.15.88:0/1993908113 shutdown_connections 2024-02-19T19:39:24.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.597+0000 7f7e62d9d700 1 --2- 172.21.15.88:0/1993908113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7e4806d8a0 0x7f7e4806fd50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.597+0000 7f7e62d9d700 1 --2- 172.21.15.88:0/1993908113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7e5c1002b0 0x7f7e5c19b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.597+0000 7f7e62d9d700 1 --2- 172.21.15.88:0/1993908113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c100c50 0x7f7e5c19bf20 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.597+0000 7f7e62d9d700 1 -- 172.21.15.88:0/1993908113 >> 172.21.15.88:0/1993908113 conn(0x7f7e5c0fbb50 msgr2=0x7f7e5c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:24.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.597+0000 7f7e62d9d700 1 -- 172.21.15.88:0/1993908113 shutdown_connections 2024-02-19T19:39:24.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.597+0000 7f7e62d9d700 1 -- 172.21.15.88:0/1993908113 wait complete. 2024-02-19T19:39:24.601 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:39:24.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.702+0000 7f62d0de5700 1 -- 172.21.15.88:0/3174794870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62cc0ffbd0 msgr2=0x7f62cc100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:24.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.702+0000 7f62d0de5700 1 --2- 172.21.15.88:0/3174794870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62cc0ffbd0 0x7f62cc100040 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f62bc009910 tx=0x7f62bc009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:24.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.702+0000 7f62d0de5700 1 -- 172.21.15.88:0/3174794870 shutdown_connections 2024-02-19T19:39:24.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.702+0000 7f62d0de5700 1 --2- 172.21.15.88:0/3174794870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62cc0ffbd0 0x7f62cc100040 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.702+0000 7f62d0de5700 1 --2- 172.21.15.88:0/3174794870 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62cc104130 0x7f62cc104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.702+0000 7f62d0de5700 1 -- 172.21.15.88:0/3174794870 >> 172.21.15.88:0/3174794870 conn(0x7f62cc075130 msgr2=0x7f62cc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:24.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.702+0000 7f62d0de5700 1 -- 172.21.15.88:0/3174794870 shutdown_connections 2024-02-19T19:39:24.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.702+0000 7f62d0de5700 1 -- 172.21.15.88:0/3174794870 wait complete. 2024-02-19T19:39:24.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62d0de5700 1 Processor -- start 2024-02-19T19:39:24.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62d0de5700 1 -- start start 2024-02-19T19:39:24.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62d0de5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62cc0ffbd0 0x7f62cc194bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:24.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62d0de5700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62cc104130 0x7f62cc1950f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:24.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62d0de5700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc195630 con 0x7f62cc0ffbd0 2024-02-19T19:39:24.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62d0de5700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc1957a0 con 0x7f62cc104130 2024-02-19T19:39:24.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62ca59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62cc0ffbd0 0x7f62cc194bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62ca59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62cc0ffbd0 0x7f62cc194bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52814/0 (socket says 172.21.15.88:52814) 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62ca59c700 1 -- 172.21.15.88:0/773633935 learned_addr learned my addr 172.21.15.88:0/773633935 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62c9d9b700 1 --2- 172.21.15.88:0/773633935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62cc104130 0x7f62cc1950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62ca59c700 1 -- 172.21.15.88:0/773633935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62cc104130 msgr2=0x7f62cc1950f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62ca59c700 1 --2- 172.21.15.88:0/773633935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62cc104130 0x7f62cc1950f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.703+0000 7f62ca59c700 1 -- 172.21.15.88:0/773633935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62bc0095f0 con 0x7f62cc0ffbd0 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.704+0000 7f62ca59c700 1 --2- 172.21.15.88:0/773633935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62cc0ffbd0 0x7f62cc194bb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f62c000b4b0 tx=0x7f62c000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.704+0000 7f62bb7fe700 1 -- 172.21.15.88:0/773633935 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62c0011800 con 0x7f62cc0ffbd0 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.704+0000 7f62d0de5700 1 -- 172.21.15.88:0/773633935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc1a2220 con 0x7f62cc0ffbd0 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.704+0000 7f62bb7fe700 1 -- 172.21.15.88:0/773633935 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62c0011e20 con 0x7f62cc0ffbd0 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.704+0000 7f62bb7fe700 1 -- 172.21.15.88:0/773633935 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62c000f2e0 con 0x7f62cc0ffbd0 2024-02-19T19:39:24.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.704+0000 7f62d0de5700 1 -- 172.21.15.88:0/773633935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc1a2580 con 0x7f62cc0ffbd0 2024-02-19T19:39:24.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.705+0000 7f62d0de5700 1 -- 172.21.15.88:0/773633935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62cc04f2a0 con 0x7f62cc0ffbd0 2024-02-19T19:39:24.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.705+0000 7f62bb7fe700 1 -- 172.21.15.88:0/773633935 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f62c0010380 con 0x7f62cc0ffbd0 2024-02-19T19:39:24.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.706+0000 7f62bb7fe700 1 --2- 172.21.15.88:0/773633935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62b406d8a0 0x7f62b406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:24.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.706+0000 7f62bb7fe700 1 -- 172.21.15.88:0/773633935 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f62c008d0a0 con 0x7f62cc0ffbd0 2024-02-19T19:39:24.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.706+0000 7f62c9d9b700 1 --2- 172.21.15.88:0/773633935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62b406d8a0 0x7f62b406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:24.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.706+0000 7f62c9d9b700 1 --2- 172.21.15.88:0/773633935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62b406d8a0 0x7f62b406fd50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f62cc068ea0 tx=0x7f62bc01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:24.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.710+0000 7f62bb7fe700 1 -- 172.21.15.88:0/773633935 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f62c005a380 con 0x7f62cc0ffbd0 2024-02-19T19:39:24.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.926+0000 7f62d0de5700 1 -- 172.21.15.88:0/773633935 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62cc0ff3a0 con 0x7f62b406d8a0 2024-02-19T19:39:24.927 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:24 smithi088 ceph-mon[88643]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:24.927 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:24 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/937839058' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:39:24.927 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:24 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1993908113' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:39:24.927 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:24 smithi088 ceph-mon[88643]: pgmap v1076: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.927+0000 7f62bb7fe700 1 -- 172.21.15.88:0/773633935 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f62cc0ff3a0 con 0x7f62b406d8a0 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:39:24.928 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:39:24.929 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:39:24.929 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:39:24.929 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:39:24.929 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:39:24.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.929+0000 7f62d0de5700 1 -- 172.21.15.88:0/773633935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62b406d8a0 msgr2=0x7f62b406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:24.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.929+0000 7f62d0de5700 1 --2- 172.21.15.88:0/773633935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62b406d8a0 0x7f62b406fd50 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f62cc068ea0 tx=0x7f62bc01a040 comp rx=0 tx=0).stop 2024-02-19T19:39:24.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.929+0000 7f62d0de5700 1 -- 172.21.15.88:0/773633935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62cc0ffbd0 msgr2=0x7f62cc194bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:24.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.929+0000 7f62d0de5700 1 --2- 172.21.15.88:0/773633935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62cc0ffbd0 0x7f62cc194bb0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f62c000b4b0 tx=0x7f62c000b870 comp rx=0 tx=0).stop 2024-02-19T19:39:24.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.929+0000 7f62d0de5700 1 -- 172.21.15.88:0/773633935 shutdown_connections 2024-02-19T19:39:24.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.929+0000 7f62d0de5700 1 --2- 172.21.15.88:0/773633935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f62b406d8a0 0x7f62b406fd50 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.929+0000 7f62d0de5700 1 --2- 172.21.15.88:0/773633935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62cc0ffbd0 0x7f62cc194bb0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.929+0000 7f62d0de5700 1 --2- 172.21.15.88:0/773633935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62cc104130 0x7f62cc1950f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:24.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.929+0000 7f62d0de5700 1 -- 172.21.15.88:0/773633935 >> 172.21.15.88:0/773633935 conn(0x7f62cc075130 msgr2=0x7f62cc0fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:24.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.929+0000 7f62d0de5700 1 -- 172.21.15.88:0/773633935 shutdown_connections 2024-02-19T19:39:24.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:24.929+0000 7f62d0de5700 1 -- 172.21.15.88:0/773633935 wait complete. 2024-02-19T19:39:25.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.033+0000 7f17c0d32700 1 -- 172.21.15.88:0/1107690621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc1004e0 msgr2=0x7f17bc1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:25.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.033+0000 7f17c0d32700 1 --2- 172.21.15.88:0/1107690621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc1004e0 0x7f17bc1008b0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009910 tx=0x7f17ac009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:25.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.033+0000 7f17c0d32700 1 -- 172.21.15.88:0/1107690621 shutdown_connections 2024-02-19T19:39:25.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.033+0000 7f17c0d32700 1 --2- 172.21.15.88:0/1107690621 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17bc100e80 0x7f17bc104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:25.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.034+0000 7f17c0d32700 1 --2- 172.21.15.88:0/1107690621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc1004e0 0x7f17bc1008b0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:25.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.034+0000 7f17c0d32700 1 -- 172.21.15.88:0/1107690621 >> 172.21.15.88:0/1107690621 conn(0x7f17bc0fbda0 msgr2=0x7f17bc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:25.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.034+0000 7f17c0d32700 1 -- 172.21.15.88:0/1107690621 shutdown_connections 2024-02-19T19:39:25.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.034+0000 7f17c0d32700 1 -- 172.21.15.88:0/1107690621 wait complete. 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.034+0000 7f17c0d32700 1 Processor -- start 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.034+0000 7f17c0d32700 1 -- start start 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.034+0000 7f17c0d32700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17bc1004e0 0x7f17bc195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.034+0000 7f17c0d32700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc100e80 0x7f17bc196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.034+0000 7f17c0d32700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc196b10 con 0x7f17bc100e80 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.034+0000 7f17c0d32700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc19a8a0 con 0x7f17bc1004e0 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17b9d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc100e80 0x7f17bc196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17b9d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc100e80 0x7f17bc196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52826/0 (socket says 172.21.15.88:52826) 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17b9d9b700 1 -- 172.21.15.88:0/36653725 learned_addr learned my addr 172.21.15.88:0/36653725 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17ba59c700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17bc1004e0 0x7f17bc195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17b9d9b700 1 -- 172.21.15.88:0/36653725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17bc1004e0 msgr2=0x7f17bc195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:25.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17b9d9b700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17bc1004e0 0x7f17bc195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:25.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17b9d9b700 1 -- 172.21.15.88:0/36653725 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ac0095f0 con 0x7f17bc100e80 2024-02-19T19:39:25.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17ba59c700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17bc1004e0 0x7f17bc195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:39:25.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17b9d9b700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc100e80 0x7f17bc196430 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f17b000d900 tx=0x7f17b000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:25.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17ab7fe700 1 -- 172.21.15.88:0/36653725 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b0009900 con 0x7f17bc100e80 2024-02-19T19:39:25.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17ab7fe700 1 -- 172.21.15.88:0/36653725 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17b0010460 con 0x7f17bc100e80 2024-02-19T19:39:25.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17c0d32700 1 -- 172.21.15.88:0/36653725 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17bc19ab80 con 0x7f17bc100e80 2024-02-19T19:39:25.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17ab7fe700 1 -- 172.21.15.88:0/36653725 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17b000f5f0 con 0x7f17bc100e80 2024-02-19T19:39:25.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.035+0000 7f17c0d32700 1 -- 172.21.15.88:0/36653725 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17bc19b0d0 con 0x7f17bc100e80 2024-02-19T19:39:25.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.036+0000 7f17c0d32700 1 -- 172.21.15.88:0/36653725 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17bc1087f0 con 0x7f17bc100e80 2024-02-19T19:39:25.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.037+0000 7f17ab7fe700 1 -- 172.21.15.88:0/36653725 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f17b000fa30 con 0x7f17bc100e80 2024-02-19T19:39:25.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.037+0000 7f17ab7fe700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f17a406d8a0 0x7f17a406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:25.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.037+0000 7f17ab7fe700 1 -- 172.21.15.88:0/36653725 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f17b008d300 con 0x7f17bc100e80 2024-02-19T19:39:25.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.037+0000 7f17ba59c700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f17a406d8a0 0x7f17a406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:25.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.038+0000 7f17ba59c700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f17a406d8a0 0x7f17a406fd50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00b5c0 tx=0x7f17ac0053a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:25.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.042+0000 7f17ab7fe700 1 -- 172.21.15.88:0/36653725 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f17b0056c80 con 0x7f17bc100e80 2024-02-19T19:39:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:24 smithi132 ceph-mon[70676]: from='client.35938 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:24 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/937839058' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:39:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:24 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1993908113' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:39:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:24 smithi132 ceph-mon[70676]: pgmap v1076: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:25.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.317+0000 7f17c0d32700 1 -- 172.21.15.88:0/36653725 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f17bc19b340 con 0x7f17bc100e80 2024-02-19T19:39:25.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.317+0000 7f17ab7fe700 1 -- 172.21.15.88:0/36653725 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f17b005a340 con 0x7f17bc100e80 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:39:25.319 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:39:25.320 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:39:25.320 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:39:25.320 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:39:25.320 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:39:25.320 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:39:25.320 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:39:25.320 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:39:25.320 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:39:25.320 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:39:25.320 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:39:25.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.320+0000 7f17c0d32700 1 -- 172.21.15.88:0/36653725 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f17a406d8a0 msgr2=0x7f17a406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:25.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.320+0000 7f17c0d32700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f17a406d8a0 0x7f17a406fd50 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00b5c0 tx=0x7f17ac0053a0 comp rx=0 tx=0).stop 2024-02-19T19:39:25.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.320+0000 7f17c0d32700 1 -- 172.21.15.88:0/36653725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc100e80 msgr2=0x7f17bc196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:25.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.320+0000 7f17c0d32700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc100e80 0x7f17bc196430 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f17b000d900 tx=0x7f17b000dcc0 comp rx=0 tx=0).stop 2024-02-19T19:39:25.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.320+0000 7f17c0d32700 1 -- 172.21.15.88:0/36653725 shutdown_connections 2024-02-19T19:39:25.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.320+0000 7f17c0d32700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f17a406d8a0 0x7f17a406fd50 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:25.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.320+0000 7f17c0d32700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f17bc1004e0 0x7f17bc195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:25.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.320+0000 7f17c0d32700 1 --2- 172.21.15.88:0/36653725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17bc100e80 0x7f17bc196430 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:25.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.320+0000 7f17c0d32700 1 -- 172.21.15.88:0/36653725 >> 172.21.15.88:0/36653725 conn(0x7f17bc0fbda0 msgr2=0x7f17bc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:25.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.320+0000 7f17c0d32700 1 -- 172.21.15.88:0/36653725 shutdown_connections 2024-02-19T19:39:25.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:25.320+0000 7f17c0d32700 1 -- 172.21.15.88:0/36653725 wait complete. 2024-02-19T19:39:26.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:25 smithi088 ceph-mon[88643]: from='client.35950 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:26.181 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:25 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/36653725' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:39:26.236 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:25 smithi132 ceph-mon[70676]: from='client.35950 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:26.236 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:25 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/36653725' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:39:27.135 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:26 smithi132 ceph-mon[70676]: pgmap v1077: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:26 smithi088 ceph-mon[88643]: pgmap v1077: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:28.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:39:28.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:39:29.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:29 smithi088 ceph-mon[88643]: pgmap v1078: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:29.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:29 smithi132 ceph-mon[70676]: pgmap v1078: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:30 smithi132 ceph-mon[70676]: pgmap v1079: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:31.234 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:30 smithi088 ceph-mon[88643]: pgmap v1079: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:33.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:32 smithi132 ceph-mon[70676]: pgmap v1080: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:32 smithi088 ceph-mon[88643]: pgmap v1080: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:34 smithi088 ceph-mon[88643]: pgmap v1081: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:34 smithi132 ceph-mon[70676]: pgmap v1081: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:36 smithi132 ceph-mon[70676]: pgmap v1082: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:37.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:36 smithi088 ceph-mon[88643]: pgmap v1082: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:39.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:39 smithi132 ceph-mon[70676]: pgmap v1083: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:40.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:39 smithi088 ceph-mon[88643]: pgmap v1083: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:42.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:41 smithi132 ceph-mon[70676]: pgmap v1084: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:42.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:41 smithi088 ceph-mon[88643]: pgmap v1084: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:44.133 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:43 smithi088 ceph-mon[88643]: pgmap v1085: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:44.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:39:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:43 smithi132 ceph-mon[70676]: pgmap v1085: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:39:46.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:45 smithi132 ceph-mon[70676]: pgmap v1086: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:46.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:45 smithi088 ceph-mon[88643]: pgmap v1086: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:48.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:47 smithi132 ceph-mon[70676]: pgmap v1087: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:48.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:47 smithi088 ceph-mon[88643]: pgmap v1087: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:49.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:49 smithi132 ceph-mon[70676]: pgmap v1088: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:50.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:49 smithi088 ceph-mon[88643]: pgmap v1088: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:52.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:51 smithi132 ceph-mon[70676]: pgmap v1089: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:52.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:51 smithi088 ceph-mon[88643]: pgmap v1089: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:54.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:53 smithi132 ceph-mon[70676]: pgmap v1090: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:54.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:53 smithi088 ceph-mon[88643]: pgmap v1090: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:55.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.437+0000 7f6e6eceb700 1 -- 172.21.15.88:0/3122066834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e68101c90 msgr2=0x7f6e6810a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:55.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.437+0000 7f6e6eceb700 1 --2- 172.21.15.88:0/3122066834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e68101c90 0x7f6e6810a2d0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f6e50009910 tx=0x7f6e50009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:55.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.438+0000 7f6e6eceb700 1 -- 172.21.15.88:0/3122066834 shutdown_connections 2024-02-19T19:39:55.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.438+0000 7f6e6eceb700 1 --2- 172.21.15.88:0/3122066834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e68101c90 0x7f6e6810a2d0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:55.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.438+0000 7f6e6eceb700 1 --2- 172.21.15.88:0/3122066834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e681012f0 0x7f6e681016c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:55.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.438+0000 7f6e6eceb700 1 -- 172.21.15.88:0/3122066834 >> 172.21.15.88:0/3122066834 conn(0x7f6e680fac80 msgr2=0x7f6e680fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:55.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.438+0000 7f6e6eceb700 1 -- 172.21.15.88:0/3122066834 shutdown_connections 2024-02-19T19:39:55.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.438+0000 7f6e6eceb700 1 -- 172.21.15.88:0/3122066834 wait complete. 2024-02-19T19:39:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.438+0000 7f6e6eceb700 1 Processor -- start 2024-02-19T19:39:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.438+0000 7f6e6eceb700 1 -- start start 2024-02-19T19:39:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.438+0000 7f6e6eceb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e681012f0 0x7f6e68195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.438+0000 7f6e6eceb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e68101c90 0x7f6e681963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.438+0000 7f6e6eceb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e68196aa0 con 0x7f6e681012f0 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e6eceb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6819a830 con 0x7f6e68101c90 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e6ca87700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e681012f0 0x7f6e68195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e6ca87700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e681012f0 0x7f6e68195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55900/0 (socket says 172.21.15.88:55900) 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e6ca87700 1 -- 172.21.15.88:0/1470849113 learned_addr learned my addr 172.21.15.88:0/1470849113 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e67fff700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e68101c90 0x7f6e681963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e6ca87700 1 -- 172.21.15.88:0/1470849113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e68101c90 msgr2=0x7f6e681963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e6ca87700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e68101c90 0x7f6e681963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e6ca87700 1 -- 172.21.15.88:0/1470849113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e500095f0 con 0x7f6e681012f0 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e67fff700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e68101c90 0x7f6e681963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e6ca87700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e681012f0 0x7f6e68195e80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000ada0 tx=0x7f6e6000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e65ffb700 1 -- 172.21.15.88:0/1470849113 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e6000f070 con 0x7f6e681012f0 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e65ffb700 1 -- 172.21.15.88:0/1470849113 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e60007bf0 con 0x7f6e681012f0 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e65ffb700 1 -- 172.21.15.88:0/1470849113 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e60005690 con 0x7f6e681012f0 2024-02-19T19:39:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.439+0000 7f6e6eceb700 1 -- 172.21.15.88:0/1470849113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6819ab10 con 0x7f6e681012f0 2024-02-19T19:39:55.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.440+0000 7f6e6eceb700 1 -- 172.21.15.88:0/1470849113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6819b060 con 0x7f6e681012f0 2024-02-19T19:39:55.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.440+0000 7f6e6eceb700 1 -- 172.21.15.88:0/1470849113 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e681079d0 con 0x7f6e681012f0 2024-02-19T19:39:55.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.441+0000 7f6e65ffb700 1 -- 172.21.15.88:0/1470849113 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6e60005850 con 0x7f6e681012f0 2024-02-19T19:39:55.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.441+0000 7f6e65ffb700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e5406d8a0 0x7f6e5406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:55.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.441+0000 7f6e65ffb700 1 -- 172.21.15.88:0/1470849113 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6e6008ce90 con 0x7f6e681012f0 2024-02-19T19:39:55.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.441+0000 7f6e67fff700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e5406d8a0 0x7f6e5406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:55.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.442+0000 7f6e67fff700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e5406d8a0 0x7f6e5406fd50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000b850 tx=0x7f6e5000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:55.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.446+0000 7f6e65ffb700 1 -- 172.21.15.88:0/1470849113 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e600566c0 con 0x7f6e681012f0 2024-02-19T19:39:55.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.660+0000 7f6e6eceb700 1 -- 172.21.15.88:0/1470849113 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e6819b2d0 con 0x7f6e5406d8a0 2024-02-19T19:39:55.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.661+0000 7f6e65ffb700 1 -- 172.21.15.88:0/1470849113 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6e6819b2d0 con 0x7f6e5406d8a0 2024-02-19T19:39:55.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.664+0000 7f6e6eceb700 1 -- 172.21.15.88:0/1470849113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e5406d8a0 msgr2=0x7f6e5406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.664+0000 7f6e6eceb700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e5406d8a0 0x7f6e5406fd50 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6e5000b850 tx=0x7f6e5000b540 comp rx=0 tx=0).stop 2024-02-19T19:39:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.664+0000 7f6e6eceb700 1 -- 172.21.15.88:0/1470849113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e681012f0 msgr2=0x7f6e68195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.664+0000 7f6e6eceb700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e681012f0 0x7f6e68195e80 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6e6000ada0 tx=0x7f6e6000c330 comp rx=0 tx=0).stop 2024-02-19T19:39:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.664+0000 7f6e6eceb700 1 -- 172.21.15.88:0/1470849113 shutdown_connections 2024-02-19T19:39:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.664+0000 7f6e6eceb700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e5406d8a0 0x7f6e5406fd50 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.664+0000 7f6e6eceb700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e681012f0 0x7f6e68195e80 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.664+0000 7f6e6eceb700 1 --2- 172.21.15.88:0/1470849113 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e68101c90 0x7f6e681963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.664+0000 7f6e6eceb700 1 -- 172.21.15.88:0/1470849113 >> 172.21.15.88:0/1470849113 conn(0x7f6e680fac80 msgr2=0x7f6e68104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.664+0000 7f6e6eceb700 1 -- 172.21.15.88:0/1470849113 shutdown_connections 2024-02-19T19:39:55.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.664+0000 7f6e6eceb700 1 -- 172.21.15.88:0/1470849113 wait complete. 2024-02-19T19:39:55.676 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:39:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.768+0000 7f8d211f0700 1 -- 172.21.15.88:0/3025808030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1c0ffbd0 msgr2=0x7f8d1c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.768+0000 7f8d211f0700 1 --2- 172.21.15.88:0/3025808030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1c0ffbd0 0x7f8d1c100040 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8d04009910 tx=0x7f8d04009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.768+0000 7f8d211f0700 1 -- 172.21.15.88:0/3025808030 shutdown_connections 2024-02-19T19:39:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.768+0000 7f8d211f0700 1 --2- 172.21.15.88:0/3025808030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1c0ffbd0 0x7f8d1c100040 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.768+0000 7f8d211f0700 1 --2- 172.21.15.88:0/3025808030 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d1c104130 0x7f8d1c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.768+0000 7f8d211f0700 1 -- 172.21.15.88:0/3025808030 >> 172.21.15.88:0/3025808030 conn(0x7f8d1c075130 msgr2=0x7f8d1c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:55.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.768+0000 7f8d211f0700 1 -- 172.21.15.88:0/3025808030 shutdown_connections 2024-02-19T19:39:55.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.768+0000 7f8d211f0700 1 -- 172.21.15.88:0/3025808030 wait complete. 2024-02-19T19:39:55.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.769+0000 7f8d211f0700 1 Processor -- start 2024-02-19T19:39:55.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.769+0000 7f8d211f0700 1 -- start start 2024-02-19T19:39:55.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.769+0000 7f8d211f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d1c0ffbd0 0x7f8d1c194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:55.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.769+0000 7f8d211f0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1c104130 0x7f8d1c195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:55.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.769+0000 7f8d211f0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c195950 con 0x7f8d1c104130 2024-02-19T19:39:55.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.769+0000 7f8d211f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c1996e0 con 0x7f8d1c0ffbd0 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.769+0000 7f8d1259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1c104130 0x7f8d1c195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.769+0000 7f8d1259c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1c104130 0x7f8d1c195270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55914/0 (socket says 172.21.15.88:55914) 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.769+0000 7f8d1259c700 1 -- 172.21.15.88:0/2343225688 learned_addr learned my addr 172.21.15.88:0/2343225688 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.770+0000 7f8d1ad9d700 1 --2- 172.21.15.88:0/2343225688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d1c0ffbd0 0x7f8d1c194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.770+0000 7f8d1259c700 1 -- 172.21.15.88:0/2343225688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d1c0ffbd0 msgr2=0x7f8d1c194d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.770+0000 7f8d1259c700 1 --2- 172.21.15.88:0/2343225688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d1c0ffbd0 0x7f8d1c194d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.770+0000 7f8d1259c700 1 -- 172.21.15.88:0/2343225688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d040095f0 con 0x7f8d1c104130 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.770+0000 7f8d1259c700 1 --2- 172.21.15.88:0/2343225688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1c104130 0x7f8d1c195270 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400bd70 tx=0x7f8d0400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.770+0000 7f8d18d99700 1 -- 172.21.15.88:0/2343225688 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d0401d070 con 0x7f8d1c104130 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.770+0000 7f8d18d99700 1 -- 172.21.15.88:0/2343225688 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d04005dc0 con 0x7f8d1c104130 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.770+0000 7f8d211f0700 1 -- 172.21.15.88:0/2343225688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c199960 con 0x7f8d1c104130 2024-02-19T19:39:55.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.770+0000 7f8d18d99700 1 -- 172.21.15.88:0/2343225688 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d04017400 con 0x7f8d1c104130 2024-02-19T19:39:55.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.770+0000 7f8d211f0700 1 -- 172.21.15.88:0/2343225688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c199e50 con 0x7f8d1c104130 2024-02-19T19:39:55.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.771+0000 7f8d211f0700 1 -- 172.21.15.88:0/2343225688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c04f2a0 con 0x7f8d1c104130 2024-02-19T19:39:55.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.772+0000 7f8d18d99700 1 -- 172.21.15.88:0/2343225688 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8d0400f460 con 0x7f8d1c104130 2024-02-19T19:39:55.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.772+0000 7f8d18d99700 1 --2- 172.21.15.88:0/2343225688 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d0806d8a0 0x7f8d0806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:55.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.773+0000 7f8d18d99700 1 -- 172.21.15.88:0/2343225688 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8d0408e970 con 0x7f8d1c104130 2024-02-19T19:39:55.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.773+0000 7f8d1ad9d700 1 --2- 172.21.15.88:0/2343225688 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d0806d8a0 0x7f8d0806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:55.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.773+0000 7f8d1ad9d700 1 --2- 172.21.15.88:0/2343225688 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d0806d8a0 0x7f8d0806fd50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005950 tx=0x7f8d0c00b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:55.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.777+0000 7f8d18d99700 1 -- 172.21.15.88:0/2343225688 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8d04058200 con 0x7f8d1c104130 2024-02-19T19:39:56.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:55.999+0000 7f8d211f0700 1 -- 172.21.15.88:0/2343225688 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d1c19a1d0 con 0x7f8d0806d8a0 2024-02-19T19:39:56.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.000+0000 7f8d18d99700 1 -- 172.21.15.88:0/2343225688 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8d1c19a1d0 con 0x7f8d0806d8a0 2024-02-19T19:39:56.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.002+0000 7f8d211f0700 1 -- 172.21.15.88:0/2343225688 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d0806d8a0 msgr2=0x7f8d0806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.002+0000 7f8d211f0700 1 --2- 172.21.15.88:0/2343225688 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d0806d8a0 0x7f8d0806fd50 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c005950 tx=0x7f8d0c00b410 comp rx=0 tx=0).stop 2024-02-19T19:39:56.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.002+0000 7f8d211f0700 1 -- 172.21.15.88:0/2343225688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1c104130 msgr2=0x7f8d1c195270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.002+0000 7f8d211f0700 1 --2- 172.21.15.88:0/2343225688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1c104130 0x7f8d1c195270 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400bd70 tx=0x7f8d0400bda0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.002+0000 7f8d211f0700 1 -- 172.21.15.88:0/2343225688 shutdown_connections 2024-02-19T19:39:56.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.002+0000 7f8d211f0700 1 --2- 172.21.15.88:0/2343225688 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8d0806d8a0 0x7f8d0806fd50 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.002+0000 7f8d211f0700 1 --2- 172.21.15.88:0/2343225688 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8d1c0ffbd0 0x7f8d1c194d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.002+0000 7f8d211f0700 1 --2- 172.21.15.88:0/2343225688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d1c104130 0x7f8d1c195270 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.002+0000 7f8d211f0700 1 -- 172.21.15.88:0/2343225688 >> 172.21.15.88:0/2343225688 conn(0x7f8d1c075130 msgr2=0x7f8d1c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:56.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.002+0000 7f8d211f0700 1 -- 172.21.15.88:0/2343225688 shutdown_connections 2024-02-19T19:39:56.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.002+0000 7f8d211f0700 1 -- 172.21.15.88:0/2343225688 wait complete. 2024-02-19T19:39:56.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:55 smithi088 ceph-mon[88643]: pgmap v1091: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:56.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:55 smithi088 ceph-mon[88643]: from='client.35958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:56.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.106+0000 7f00408c7700 1 -- 172.21.15.88:0/1263290367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f003c100d70 msgr2=0x7f003c1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.106+0000 7f00408c7700 1 --2- 172.21.15.88:0/1263290367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f003c100d70 0x7f003c1094d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f0030009940 tx=0x7f0030009c50 comp rx=0 tx=0).stop 2024-02-19T19:39:56.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.106+0000 7f00408c7700 1 -- 172.21.15.88:0/1263290367 shutdown_connections 2024-02-19T19:39:56.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.106+0000 7f00408c7700 1 --2- 172.21.15.88:0/1263290367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f003c100d70 0x7f003c1094d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.106+0000 7f00408c7700 1 --2- 172.21.15.88:0/1263290367 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f003c100460 0x7f003c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.106+0000 7f00408c7700 1 -- 172.21.15.88:0/1263290367 >> 172.21.15.88:0/1263290367 conn(0x7f003c0748b0 msgr2=0x7f003c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:56.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.107+0000 7f00408c7700 1 -- 172.21.15.88:0/1263290367 shutdown_connections 2024-02-19T19:39:56.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.107+0000 7f00408c7700 1 -- 172.21.15.88:0/1263290367 wait complete. 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.107+0000 7f00408c7700 1 Processor -- start 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.107+0000 7f00408c7700 1 -- start start 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.107+0000 7f00408c7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f003c100460 0x7f003c196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.107+0000 7f00408c7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f003c100d70 0x7f003c1970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.107+0000 7f00408c7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c1976f0 con 0x7f003c100d70 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.107+0000 7f00408c7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c197830 con 0x7f003c100460 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f0039d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f003c100d70 0x7f003c1970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f0039d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f003c100d70 0x7f003c1970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55920/0 (socket says 172.21.15.88:55920) 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f0039d9b700 1 -- 172.21.15.88:0/487185771 learned_addr learned my addr 172.21.15.88:0/487185771 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f003a59c700 1 --2- 172.21.15.88:0/487185771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f003c100460 0x7f003c196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f0039d9b700 1 -- 172.21.15.88:0/487185771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f003c100460 msgr2=0x7f003c196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f0039d9b700 1 --2- 172.21.15.88:0/487185771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f003c100460 0x7f003c196b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f0039d9b700 1 -- 172.21.15.88:0/487185771 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00300095f0 con 0x7f003c100d70 2024-02-19T19:39:56.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f0039d9b700 1 --2- 172.21.15.88:0/487185771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f003c100d70 0x7f003c1970d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f003000b580 tx=0x7f00300055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:56.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f002b7fe700 1 -- 172.21.15.88:0/487185771 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f003001d070 con 0x7f003c100d70 2024-02-19T19:39:56.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f00408c7700 1 -- 172.21.15.88:0/487185771 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c19fc00 con 0x7f003c100d70 2024-02-19T19:39:56.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f002b7fe700 1 -- 172.21.15.88:0/487185771 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0030005dc0 con 0x7f003c100d70 2024-02-19T19:39:56.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f002b7fe700 1 -- 172.21.15.88:0/487185771 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0030021410 con 0x7f003c100d70 2024-02-19T19:39:56.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.108+0000 7f00408c7700 1 -- 172.21.15.88:0/487185771 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003c1a00c0 con 0x7f003c100d70 2024-02-19T19:39:56.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.109+0000 7f00408c7700 1 -- 172.21.15.88:0/487185771 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003c106bd0 con 0x7f003c100d70 2024-02-19T19:39:56.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.110+0000 7f002b7fe700 1 -- 172.21.15.88:0/487185771 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f003000f7f0 con 0x7f003c100d70 2024-02-19T19:39:56.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.110+0000 7f002b7fe700 1 --2- 172.21.15.88:0/487185771 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f002406d8a0 0x7f002406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:56.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.110+0000 7f003a59c700 1 --2- 172.21.15.88:0/487185771 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f002406d8a0 0x7f002406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:56.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.110+0000 7f002b7fe700 1 -- 172.21.15.88:0/487185771 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f003008e880 con 0x7f003c100d70 2024-02-19T19:39:56.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.111+0000 7f003a59c700 1 --2- 172.21.15.88:0/487185771 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f002406d8a0 0x7f002406fd50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f002c005950 tx=0x7f002c00b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:56.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.115+0000 7f002b7fe700 1 -- 172.21.15.88:0/487185771 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f00300580f0 con 0x7f003c100d70 2024-02-19T19:39:56.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:55 smithi132 ceph-mon[70676]: pgmap v1091: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:56.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:55 smithi132 ceph-mon[70676]: from='client.35958 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:56.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.322+0000 7f00408c7700 1 -- 172.21.15.88:0/487185771 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f003c061190 con 0x7f002406d8a0 2024-02-19T19:39:56.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.333+0000 7f002b7fe700 1 -- 172.21.15.88:0/487185771 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f003c061190 con 0x7f002406d8a0 2024-02-19T19:39:56.335 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:39:56.335 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (37m) 13m ago 47m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:39:56.335 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (48m) 13m ago 48m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:39:56.335 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (29m) 13m ago 46m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:39:56.335 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (33m) 13m ago 47m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:39:56.335 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (33m) 13m ago 46m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:39:56.335 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (36m) 13m ago 46m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:39:56.335 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (29m) 13m ago 41m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:39:56.335 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (29m) 13m ago 41m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (28m) 13m ago 41m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (13m) 13m ago 41m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (38m) 13m ago 49m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (38m) 13m ago 45m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (35m) 13m ago 49m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (34m) 13m ago 45m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (37m) 13m ago 47m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (37m) 13m ago 45m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (32m) 13m ago 45m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (31m) 13m ago 44m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (31m) 13m ago 44m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:39:56.336 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (30m) 13m ago 43m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:39:56.337 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (30m) 13m ago 43m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:39:56.337 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (29m) 13m ago 42m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:39:56.337 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (37m) 13m ago 46m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:39:56.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.335+0000 7f00408c7700 1 -- 172.21.15.88:0/487185771 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f002406d8a0 msgr2=0x7f002406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.335+0000 7f00408c7700 1 --2- 172.21.15.88:0/487185771 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f002406d8a0 0x7f002406fd50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f002c005950 tx=0x7f002c00b410 comp rx=0 tx=0).stop 2024-02-19T19:39:56.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.335+0000 7f00408c7700 1 -- 172.21.15.88:0/487185771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f003c100d70 msgr2=0x7f003c1970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.335+0000 7f00408c7700 1 --2- 172.21.15.88:0/487185771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f003c100d70 0x7f003c1970d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f003000b580 tx=0x7f00300055f0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.335+0000 7f00408c7700 1 -- 172.21.15.88:0/487185771 shutdown_connections 2024-02-19T19:39:56.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.335+0000 7f00408c7700 1 --2- 172.21.15.88:0/487185771 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f002406d8a0 0x7f002406fd50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.335+0000 7f00408c7700 1 --2- 172.21.15.88:0/487185771 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f003c100460 0x7f003c196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.335+0000 7f00408c7700 1 --2- 172.21.15.88:0/487185771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f003c100d70 0x7f003c1970d0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.335+0000 7f00408c7700 1 -- 172.21.15.88:0/487185771 >> 172.21.15.88:0/487185771 conn(0x7f003c0748b0 msgr2=0x7f003c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:56.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.336+0000 7f00408c7700 1 -- 172.21.15.88:0/487185771 shutdown_connections 2024-02-19T19:39:56.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.336+0000 7f00408c7700 1 -- 172.21.15.88:0/487185771 wait complete. 2024-02-19T19:39:56.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.438+0000 7f7f1c24e700 1 -- 172.21.15.88:0/900830568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f140ff350 msgr2=0x7f7f140ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.438+0000 7f7f1c24e700 1 --2- 172.21.15.88:0/900830568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f140ff350 0x7f7f140ff720 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f7f10009910 tx=0x7f7f10009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:56.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.438+0000 7f7f1c24e700 1 -- 172.21.15.88:0/900830568 shutdown_connections 2024-02-19T19:39:56.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.438+0000 7f7f1c24e700 1 --2- 172.21.15.88:0/900830568 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140ffc60 0x7f7f1410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.438+0000 7f7f1c24e700 1 --2- 172.21.15.88:0/900830568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f140ff350 0x7f7f140ff720 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.438+0000 7f7f1c24e700 1 -- 172.21.15.88:0/900830568 >> 172.21.15.88:0/900830568 conn(0x7f7f140748b0 msgr2=0x7f7f14074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:56.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.438+0000 7f7f1c24e700 1 -- 172.21.15.88:0/900830568 shutdown_connections 2024-02-19T19:39:56.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.438+0000 7f7f1c24e700 1 -- 172.21.15.88:0/900830568 wait complete. 2024-02-19T19:39:56.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f1c24e700 1 Processor -- start 2024-02-19T19:39:56.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f1c24e700 1 -- start start 2024-02-19T19:39:56.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f1c24e700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f140ff350 0x7f7f14195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:56.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f1c24e700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140ffc60 0x7f7f141964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:56.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f1c24e700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14196b80 con 0x7f7f140ff350 2024-02-19T19:39:56.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f1c24e700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1419a910 con 0x7f7f140ffc60 2024-02-19T19:39:56.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f19fea700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f140ff350 0x7f7f14195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:56.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f19fea700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f140ff350 0x7f7f14195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55930/0 (socket says 172.21.15.88:55930) 2024-02-19T19:39:56.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f19fea700 1 -- 172.21.15.88:0/2240132230 learned_addr learned my addr 172.21.15.88:0/2240132230 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f197e9700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140ffc60 0x7f7f141964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f19fea700 1 -- 172.21.15.88:0/2240132230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140ffc60 msgr2=0x7f7f141964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f19fea700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140ffc60 0x7f7f141964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f19fea700 1 -- 172.21.15.88:0/2240132230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f100095f0 con 0x7f7f140ff350 2024-02-19T19:39:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.439+0000 7f7f197e9700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140ffc60 0x7f7f141964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:39:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.440+0000 7f7f19fea700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f140ff350 0x7f7f14195f60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7f100092e0 tx=0x7f7f1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.440+0000 7f7f0affd700 1 -- 172.21.15.88:0/2240132230 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f1001d070 con 0x7f7f140ff350 2024-02-19T19:39:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.440+0000 7f7f0affd700 1 -- 172.21.15.88:0/2240132230 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f1000f460 con 0x7f7f140ff350 2024-02-19T19:39:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.440+0000 7f7f1c24e700 1 -- 172.21.15.88:0/2240132230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1419ab90 con 0x7f7f140ff350 2024-02-19T19:39:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.440+0000 7f7f0affd700 1 -- 172.21.15.88:0/2240132230 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f100175f0 con 0x7f7f140ff350 2024-02-19T19:39:56.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.440+0000 7f7f1c24e700 1 -- 172.21.15.88:0/2240132230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1419b080 con 0x7f7f140ff350 2024-02-19T19:39:56.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.441+0000 7f7f1c24e700 1 -- 172.21.15.88:0/2240132230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f14107dd0 con 0x7f7f140ff350 2024-02-19T19:39:56.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.441+0000 7f7f0affd700 1 -- 172.21.15.88:0/2240132230 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7f1000f5d0 con 0x7f7f140ff350 2024-02-19T19:39:56.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.442+0000 7f7f0affd700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f0006d8a0 0x7f7f0006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:56.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.442+0000 7f7f0affd700 1 -- 172.21.15.88:0/2240132230 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7f1008eaa0 con 0x7f7f140ff350 2024-02-19T19:39:56.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.442+0000 7f7f197e9700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f0006d8a0 0x7f7f0006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:56.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.442+0000 7f7f197e9700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f0006d8a0 0x7f7f0006fd50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7f04005fd0 tx=0x7f7f04005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:56.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.447+0000 7f7f0affd700 1 -- 172.21.15.88:0/2240132230 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f10058250 con 0x7f7f140ff350 2024-02-19T19:39:56.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.733+0000 7f7f1c24e700 1 -- 172.21.15.88:0/2240132230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7f141077b0 con 0x7f7f140ff350 2024-02-19T19:39:56.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.734+0000 7f7f0affd700 1 -- 172.21.15.88:0/2240132230 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7f1005b910 con 0x7f7f140ff350 2024-02-19T19:39:56.736 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:39:56.736 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:39:56.736 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:39:56.736 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:39:56.737 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:39:56.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.736+0000 7f7f1c24e700 1 -- 172.21.15.88:0/2240132230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f0006d8a0 msgr2=0x7f7f0006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.736+0000 7f7f1c24e700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f0006d8a0 0x7f7f0006fd50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7f04005fd0 tx=0x7f7f04005dc0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.736+0000 7f7f1c24e700 1 -- 172.21.15.88:0/2240132230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f140ff350 msgr2=0x7f7f14195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.736+0000 7f7f1c24e700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f140ff350 0x7f7f14195f60 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f7f100092e0 tx=0x7f7f1000be20 comp rx=0 tx=0).stop 2024-02-19T19:39:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.736+0000 7f7f1c24e700 1 -- 172.21.15.88:0/2240132230 shutdown_connections 2024-02-19T19:39:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.736+0000 7f7f1c24e700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f0006d8a0 0x7f7f0006fd50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.736+0000 7f7f1c24e700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f140ff350 0x7f7f14195f60 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.736+0000 7f7f1c24e700 1 --2- 172.21.15.88:0/2240132230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f140ffc60 0x7f7f141964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.736+0000 7f7f1c24e700 1 -- 172.21.15.88:0/2240132230 >> 172.21.15.88:0/2240132230 conn(0x7f7f140748b0 msgr2=0x7f7f14104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.736+0000 7f7f1c24e700 1 -- 172.21.15.88:0/2240132230 shutdown_connections 2024-02-19T19:39:56.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.736+0000 7f7f1c24e700 1 -- 172.21.15.88:0/2240132230 wait complete. 2024-02-19T19:39:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.840+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/679846958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ff40ffc60 msgr2=0x7f8ff410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.840+0000 7f8ff8e9d700 1 --2- 172.21.15.88:0/679846958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ff40ffc60 0x7f8ff410a6d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009910 tx=0x7f8fe8009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.840+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/679846958 shutdown_connections 2024-02-19T19:39:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.840+0000 7f8ff8e9d700 1 --2- 172.21.15.88:0/679846958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ff40ffc60 0x7f8ff410a6d0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.840+0000 7f8ff8e9d700 1 --2- 172.21.15.88:0/679846958 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff40ff350 0x7f8ff40ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.840+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/679846958 >> 172.21.15.88:0/679846958 conn(0x7f8ff40748b0 msgr2=0x7f8ff4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.840+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/679846958 shutdown_connections 2024-02-19T19:39:56.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.840+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/679846958 wait complete. 2024-02-19T19:39:56.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.841+0000 7f8ff8e9d700 1 Processor -- start 2024-02-19T19:39:56.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.841+0000 7f8ff8e9d700 1 -- start start 2024-02-19T19:39:56.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.841+0000 7f8ff8e9d700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff40ff350 0x7f8ff4195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:56.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.841+0000 7f8ff8e9d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ff40ffc60 0x7f8ff41963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:56.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.841+0000 7f8ff8e9d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4196a30 con 0x7f8ff40ffc60 2024-02-19T19:39:56.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.841+0000 7f8ff8e9d700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4196b70 con 0x7f8ff40ff350 2024-02-19T19:39:56.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.841+0000 7f8ff1d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ff40ffc60 0x7f8ff41963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.841+0000 7f8ff1d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ff40ffc60 0x7f8ff41963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55956/0 (socket says 172.21.15.88:55956) 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.841+0000 7f8ff1d9b700 1 -- 172.21.15.88:0/926090899 learned_addr learned my addr 172.21.15.88:0/926090899 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.841+0000 7f8ff259c700 1 --2- 172.21.15.88:0/926090899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff40ff350 0x7f8ff4195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.842+0000 7f8ff1d9b700 1 -- 172.21.15.88:0/926090899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff40ff350 msgr2=0x7f8ff4195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.842+0000 7f8ff1d9b700 1 --2- 172.21.15.88:0/926090899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff40ff350 0x7f8ff4195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.842+0000 7f8ff1d9b700 1 -- 172.21.15.88:0/926090899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe80095f0 con 0x7f8ff40ffc60 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.842+0000 7f8ff1d9b700 1 --2- 172.21.15.88:0/926090899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ff40ffc60 0x7f8ff41963e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8005b40 tx=0x7f8fe80056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.842+0000 7f8fe37fe700 1 -- 172.21.15.88:0/926090899 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fe801d070 con 0x7f8ff40ffc60 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.842+0000 7f8fe37fe700 1 -- 172.21.15.88:0/926090899 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fe800f460 con 0x7f8ff40ffc60 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.842+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/926090899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff419a9c0 con 0x7f8ff40ffc60 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.842+0000 7f8fe37fe700 1 -- 172.21.15.88:0/926090899 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fe8021640 con 0x7f8ff40ffc60 2024-02-19T19:39:56.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.842+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/926090899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff419af10 con 0x7f8ff40ffc60 2024-02-19T19:39:56.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.843+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/926090899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff4107dd0 con 0x7f8ff40ffc60 2024-02-19T19:39:56.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.843+0000 7f8fe37fe700 1 -- 172.21.15.88:0/926090899 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8fe800f5d0 con 0x7f8ff40ffc60 2024-02-19T19:39:56.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.844+0000 7f8fe37fe700 1 --2- 172.21.15.88:0/926090899 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8fdc06d8a0 0x7f8fdc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:56.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.844+0000 7f8ff259c700 1 --2- 172.21.15.88:0/926090899 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8fdc06d8a0 0x7f8fdc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:56.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.844+0000 7f8fe37fe700 1 -- 172.21.15.88:0/926090899 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8fe808e890 con 0x7f8ff40ffc60 2024-02-19T19:39:56.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.844+0000 7f8ff259c700 1 --2- 172.21.15.88:0/926090899 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8fdc06d8a0 0x7f8fdc06fd50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005ea0 tx=0x7f8fe4005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:56.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:56.849+0000 7f8fe37fe700 1 -- 172.21.15.88:0/926090899 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8fe8058040 con 0x7f8ff40ffc60 2024-02-19T19:39:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:56 smithi088 ceph-mon[88643]: from='client.35962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:56 smithi088 ceph-mon[88643]: from='client.35966 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:56 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2240132230' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:39:57.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.089+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/926090899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8ff404f2a0 con 0x7f8ff40ffc60 2024-02-19T19:39:57.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.089+0000 7f8fe37fe700 1 -- 172.21.15.88:0/926090899 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f8fe805b700 con 0x7f8ff40ffc60 2024-02-19T19:39:57.091 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:39:57.091 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:39:57.091 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:39:57.091 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:39:57.091 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:39:57.092 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:39:57.092 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:39:57.092 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:39:57.092 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:39:57.092 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:39:57.092 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:39:57.092 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:39:57.092 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:39:57.092 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:39:57.092 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:39:57.093 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:39:57.093 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:39:57.093 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:39:57.093 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:39:57.093 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:39:57.093 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:39:57.093 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:39:57.093 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:39:57.093 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:39:57.093 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:39:57.094 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:39:57.095 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:39:57.095 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:39:57.095 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:39:57.095 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:39:57.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.092+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/926090899 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8fdc06d8a0 msgr2=0x7f8fdc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:57.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.092+0000 7f8ff8e9d700 1 --2- 172.21.15.88:0/926090899 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8fdc06d8a0 0x7f8fdc06fd50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4005ea0 tx=0x7f8fe4005e30 comp rx=0 tx=0).stop 2024-02-19T19:39:57.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.092+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/926090899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ff40ffc60 msgr2=0x7f8ff41963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:57.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.092+0000 7f8ff8e9d700 1 --2- 172.21.15.88:0/926090899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ff40ffc60 0x7f8ff41963e0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8005b40 tx=0x7f8fe80056a0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.092+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/926090899 shutdown_connections 2024-02-19T19:39:57.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.092+0000 7f8ff8e9d700 1 --2- 172.21.15.88:0/926090899 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8fdc06d8a0 0x7f8fdc06fd50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.092+0000 7f8ff8e9d700 1 --2- 172.21.15.88:0/926090899 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8ff40ff350 0x7f8ff4195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.092+0000 7f8ff8e9d700 1 --2- 172.21.15.88:0/926090899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ff40ffc60 0x7f8ff41963e0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.092+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/926090899 >> 172.21.15.88:0/926090899 conn(0x7f8ff40748b0 msgr2=0x7f8ff4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:57.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.092+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/926090899 shutdown_connections 2024-02-19T19:39:57.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.092+0000 7f8ff8e9d700 1 -- 172.21.15.88:0/926090899 wait complete. 2024-02-19T19:39:57.097 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:39:57.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:56 smithi132 ceph-mon[70676]: from='client.35962 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:57.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:56 smithi132 ceph-mon[70676]: from='client.35966 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:57.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:56 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2240132230' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:39:57.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.196+0000 7f1e789c2700 1 -- 172.21.15.88:0/2156626173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e740ff350 msgr2=0x7f1e740ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:57.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.196+0000 7f1e789c2700 1 --2- 172.21.15.88:0/2156626173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e740ff350 0x7f1e740ff7c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f1e68009910 tx=0x7f1e68009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:57.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.197+0000 7f1e789c2700 1 -- 172.21.15.88:0/2156626173 shutdown_connections 2024-02-19T19:39:57.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.197+0000 7f1e789c2700 1 --2- 172.21.15.88:0/2156626173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e740ff350 0x7f1e740ff7c0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.197+0000 7f1e789c2700 1 --2- 172.21.15.88:0/2156626173 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e741038b0 0x7f1e74103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.197+0000 7f1e789c2700 1 -- 172.21.15.88:0/2156626173 >> 172.21.15.88:0/2156626173 conn(0x7f1e740748b0 msgr2=0x7f1e74074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:57.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.197+0000 7f1e789c2700 1 -- 172.21.15.88:0/2156626173 shutdown_connections 2024-02-19T19:39:57.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.197+0000 7f1e789c2700 1 -- 172.21.15.88:0/2156626173 wait complete. 2024-02-19T19:39:57.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.197+0000 7f1e789c2700 1 Processor -- start 2024-02-19T19:39:57.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.197+0000 7f1e789c2700 1 -- start start 2024-02-19T19:39:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e789c2700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e740ff350 0x7f1e74195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e789c2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e741038b0 0x7f1e741963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e789c2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e74196ab0 con 0x7f1e741038b0 2024-02-19T19:39:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e789c2700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7419a840 con 0x7f1e740ff350 2024-02-19T19:39:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e71d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e741038b0 0x7f1e741963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e71d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e741038b0 0x7f1e741963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55964/0 (socket says 172.21.15.88:55964) 2024-02-19T19:39:57.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e71d9b700 1 -- 172.21.15.88:0/4278319854 learned_addr learned my addr 172.21.15.88:0/4278319854 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e7259c700 1 --2- 172.21.15.88:0/4278319854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e740ff350 0x7f1e74195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e71d9b700 1 -- 172.21.15.88:0/4278319854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e740ff350 msgr2=0x7f1e74195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e71d9b700 1 --2- 172.21.15.88:0/4278319854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e740ff350 0x7f1e74195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e71d9b700 1 -- 172.21.15.88:0/4278319854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e680095f0 con 0x7f1e741038b0 2024-02-19T19:39:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e71d9b700 1 --2- 172.21.15.88:0/4278319854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e741038b0 0x7f1e741963d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1e680092e0 tx=0x7f1e6800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:57.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e637fe700 1 -- 172.21.15.88:0/4278319854 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e6801d070 con 0x7f1e741038b0 2024-02-19T19:39:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e789c2700 1 -- 172.21.15.88:0/4278319854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7419aac0 con 0x7f1e741038b0 2024-02-19T19:39:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e637fe700 1 -- 172.21.15.88:0/4278319854 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e6800f460 con 0x7f1e741038b0 2024-02-19T19:39:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e637fe700 1 -- 172.21.15.88:0/4278319854 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e68005e20 con 0x7f1e741038b0 2024-02-19T19:39:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.198+0000 7f1e789c2700 1 -- 172.21.15.88:0/4278319854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7419afb0 con 0x7f1e741038b0 2024-02-19T19:39:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.199+0000 7f1e789c2700 1 -- 172.21.15.88:0/4278319854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e74106e20 con 0x7f1e741038b0 2024-02-19T19:39:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.200+0000 7f1e637fe700 1 -- 172.21.15.88:0/4278319854 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1e6800f5d0 con 0x7f1e741038b0 2024-02-19T19:39:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.200+0000 7f1e637fe700 1 --2- 172.21.15.88:0/4278319854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e5c06d8a0 0x7f1e5c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.201+0000 7f1e7259c700 1 --2- 172.21.15.88:0/4278319854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e5c06d8a0 0x7f1e5c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.201+0000 7f1e637fe700 1 -- 172.21.15.88:0/4278319854 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1e6808e9b0 con 0x7f1e741038b0 2024-02-19T19:39:57.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.201+0000 7f1e7259c700 1 --2- 172.21.15.88:0/4278319854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e5c06d8a0 0x7f1e5c06fd50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1e64007be0 tx=0x7f1e640073d0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:57.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.206+0000 7f1e637fe700 1 -- 172.21.15.88:0/4278319854 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1e680582d0 con 0x7f1e741038b0 2024-02-19T19:39:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.418+0000 7f1e789c2700 1 -- 172.21.15.88:0/4278319854 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e7402cca0 con 0x7f1e5c06d8a0 2024-02-19T19:39:57.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.420+0000 7f1e637fe700 1 -- 172.21.15.88:0/4278319854 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1e7402cca0 con 0x7f1e5c06d8a0 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:39:57.422 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:39:57.423 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:39:57.423 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:39:57.423 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:39:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.422+0000 7f1e789c2700 1 -- 172.21.15.88:0/4278319854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e5c06d8a0 msgr2=0x7f1e5c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.422+0000 7f1e789c2700 1 --2- 172.21.15.88:0/4278319854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e5c06d8a0 0x7f1e5c06fd50 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1e64007be0 tx=0x7f1e640073d0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.422+0000 7f1e789c2700 1 -- 172.21.15.88:0/4278319854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e741038b0 msgr2=0x7f1e741963d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.422+0000 7f1e789c2700 1 --2- 172.21.15.88:0/4278319854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e741038b0 0x7f1e741963d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f1e680092e0 tx=0x7f1e6800bd70 comp rx=0 tx=0).stop 2024-02-19T19:39:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.422+0000 7f1e789c2700 1 -- 172.21.15.88:0/4278319854 shutdown_connections 2024-02-19T19:39:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.422+0000 7f1e789c2700 1 --2- 172.21.15.88:0/4278319854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1e5c06d8a0 0x7f1e5c06fd50 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.422+0000 7f1e789c2700 1 --2- 172.21.15.88:0/4278319854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1e740ff350 0x7f1e74195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.422+0000 7f1e789c2700 1 --2- 172.21.15.88:0/4278319854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e741038b0 0x7f1e741963d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.422+0000 7f1e789c2700 1 -- 172.21.15.88:0/4278319854 >> 172.21.15.88:0/4278319854 conn(0x7f1e740748b0 msgr2=0x7f1e740fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:57.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.422+0000 7f1e789c2700 1 -- 172.21.15.88:0/4278319854 shutdown_connections 2024-02-19T19:39:57.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.422+0000 7f1e789c2700 1 -- 172.21.15.88:0/4278319854 wait complete. 2024-02-19T19:39:57.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.525+0000 7f0da1933700 1 -- 172.21.15.88:0/488505363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9c0ffd60 msgr2=0x7f0d9c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:57.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.525+0000 7f0da1933700 1 --2- 172.21.15.88:0/488505363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9c0ffd60 0x7f0d9c103d30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f0d90009910 tx=0x7f0d90009c20 comp rx=0 tx=0).stop 2024-02-19T19:39:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.526+0000 7f0da1933700 1 -- 172.21.15.88:0/488505363 shutdown_connections 2024-02-19T19:39:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.526+0000 7f0da1933700 1 --2- 172.21.15.88:0/488505363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9c0ffd60 0x7f0d9c103d30 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.526+0000 7f0da1933700 1 --2- 172.21.15.88:0/488505363 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d9c0ff3c0 0x7f0d9c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.526+0000 7f0da1933700 1 -- 172.21.15.88:0/488505363 >> 172.21.15.88:0/488505363 conn(0x7f0d9c0fac80 msgr2=0x7f0d9c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.526+0000 7f0da1933700 1 -- 172.21.15.88:0/488505363 shutdown_connections 2024-02-19T19:39:57.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.526+0000 7f0da1933700 1 -- 172.21.15.88:0/488505363 wait complete. 2024-02-19T19:39:57.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.526+0000 7f0da1933700 1 Processor -- start 2024-02-19T19:39:57.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.526+0000 7f0da1933700 1 -- start start 2024-02-19T19:39:57.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0da1933700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d9c0ff3c0 0x7f0d9c196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:57.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0da1933700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9c0ffd60 0x7f0d9c1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:57.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0da1933700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c1978e0 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0da1933700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d9c197a20 con 0x7f0d9c0ff3c0 2024-02-19T19:39:57.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0d9a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9c0ffd60 0x7f0d9c1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:57.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0d9a7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9c0ffd60 0x7f0d9c1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55984/0 (socket says 172.21.15.88:55984) 2024-02-19T19:39:57.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0d9a7fc700 1 -- 172.21.15.88:0/2520655477 learned_addr learned my addr 172.21.15.88:0/2520655477 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:39:57.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0d9affd700 1 --2- 172.21.15.88:0/2520655477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d9c0ff3c0 0x7f0d9c196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:57.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0d9a7fc700 1 -- 172.21.15.88:0/2520655477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d9c0ff3c0 msgr2=0x7f0d9c196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:57.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0d9a7fc700 1 --2- 172.21.15.88:0/2520655477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d9c0ff3c0 0x7f0d9c196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0d9a7fc700 1 -- 172.21.15.88:0/2520655477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d900095f0 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0d9a7fc700 1 --2- 172.21.15.88:0/2520655477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9c0ffd60 0x7f0d9c1972c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0d900092e0 tx=0x7f0d9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:57.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0da0931700 1 -- 172.21.15.88:0/2520655477 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d9001d070 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0da1933700 1 -- 172.21.15.88:0/2520655477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d9c19fdf0 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0da0931700 1 -- 172.21.15.88:0/2520655477 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0d90005dc0 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0da0931700 1 -- 172.21.15.88:0/2520655477 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d90017400 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.527+0000 7f0da1933700 1 -- 172.21.15.88:0/2520655477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d9c1a0200 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.528+0000 7f0da1933700 1 -- 172.21.15.88:0/2520655477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d9c107730 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.529+0000 7f0da0931700 1 -- 172.21.15.88:0/2520655477 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0d9000f460 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.529+0000 7f0da0931700 1 --2- 172.21.15.88:0/2520655477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0d8406d8a0 0x7f0d8406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:39:57.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.529+0000 7f0da0931700 1 -- 172.21.15.88:0/2520655477 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0d9008ea00 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.529+0000 7f0d9affd700 1 --2- 172.21.15.88:0/2520655477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0d8406d8a0 0x7f0d8406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:39:57.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.530+0000 7f0d9affd700 1 --2- 172.21.15.88:0/2520655477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0d8406d8a0 0x7f0d8406fd50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c005950 tx=0x7f0d8c0058e0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:39:57.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.534+0000 7f0da0931700 1 -- 172.21.15.88:0/2520655477 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0d900581b0 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.817+0000 7f0da1933700 1 -- 172.21.15.88:0/2520655477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0d9c107110 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.818+0000 7f0da0931700 1 -- 172.21.15.88:0/2520655477 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f0d9005b870 con 0x7f0d9c0ffd60 2024-02-19T19:39:57.820 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:39:57.821 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:39:57.821 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:39:57.821 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:39:57.821 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:39:57.821 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:39:57.821 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:39:57.821 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:39:57.821 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:39:57.821 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:39:57.821 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:39:57.822 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:39:57.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.820+0000 7f0da1933700 1 -- 172.21.15.88:0/2520655477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0d8406d8a0 msgr2=0x7f0d8406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:57.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.820+0000 7f0da1933700 1 --2- 172.21.15.88:0/2520655477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0d8406d8a0 0x7f0d8406fd50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c005950 tx=0x7f0d8c0058e0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.820+0000 7f0da1933700 1 -- 172.21.15.88:0/2520655477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9c0ffd60 msgr2=0x7f0d9c1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:39:57.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.820+0000 7f0da1933700 1 --2- 172.21.15.88:0/2520655477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9c0ffd60 0x7f0d9c1972c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f0d900092e0 tx=0x7f0d9000be20 comp rx=0 tx=0).stop 2024-02-19T19:39:57.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.820+0000 7f0da1933700 1 -- 172.21.15.88:0/2520655477 shutdown_connections 2024-02-19T19:39:57.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.820+0000 7f0da1933700 1 --2- 172.21.15.88:0/2520655477 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f0d8406d8a0 0x7f0d8406fd50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.820+0000 7f0da1933700 1 --2- 172.21.15.88:0/2520655477 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0d9c0ff3c0 0x7f0d9c196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.820+0000 7f0da1933700 1 --2- 172.21.15.88:0/2520655477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9c0ffd60 0x7f0d9c1972c0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:39:57.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.820+0000 7f0da1933700 1 -- 172.21.15.88:0/2520655477 >> 172.21.15.88:0/2520655477 conn(0x7f0d9c0fac80 msgr2=0x7f0d9c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:39:57.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.820+0000 7f0da1933700 1 -- 172.21.15.88:0/2520655477 shutdown_connections 2024-02-19T19:39:57.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:39:57.820+0000 7f0da1933700 1 -- 172.21.15.88:0/2520655477 wait complete. 2024-02-19T19:39:58.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:57 smithi088 ceph-mon[88643]: pgmap v1092: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:58.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:57 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/926090899' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:39:58.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:57 smithi088 ceph-mon[88643]: from='client.35978 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:58.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:57 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2520655477' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:39:58.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:57 smithi132 ceph-mon[70676]: pgmap v1092: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:39:58.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:57 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/926090899' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:39:58.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:57 smithi132 ceph-mon[70676]: from='client.35978 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:39:58.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:57 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2520655477' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:39:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:39:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:40:00.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:39:59 smithi132 ceph-mon[70676]: pgmap v1093: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:00.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:39:59 smithi088 ceph-mon[88643]: pgmap v1093: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:00.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:00 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T19:39:59.999+0000 7fb604fbe700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:40:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:00 smithi132 ceph-mon[70676]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:40:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:00 smithi088 ceph-mon[88643]: overall HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:40:02.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:01 smithi132 ceph-mon[70676]: pgmap v1094: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:02.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:01 smithi088 ceph-mon[88643]: pgmap v1094: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:04.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:03 smithi132 ceph-mon[70676]: pgmap v1095: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:04.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:03 smithi088 ceph-mon[88643]: pgmap v1095: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:06.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:05 smithi132 ceph-mon[70676]: pgmap v1096: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:06.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:05 smithi088 ceph-mon[88643]: pgmap v1096: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:08.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:07 smithi132 ceph-mon[70676]: pgmap v1097: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:08.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:07 smithi088 ceph-mon[88643]: pgmap v1097: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:10.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:09 smithi132 ceph-mon[70676]: pgmap v1098: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:10.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:09 smithi088 ceph-mon[88643]: pgmap v1098: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:12.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:11 smithi088 ceph-mon[88643]: pgmap v1099: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:12.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:11 smithi132 ceph-mon[70676]: pgmap v1099: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:13 smithi132 ceph-mon[70676]: pgmap v1100: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:40:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:13 smithi088 ceph-mon[88643]: pgmap v1100: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:40:16.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:15 smithi132 ceph-mon[70676]: pgmap v1101: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:16.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:15 smithi088 ceph-mon[88643]: pgmap v1101: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:18.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:17 smithi132 ceph-mon[70676]: pgmap v1102: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:18.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:17 smithi088 ceph-mon[88643]: pgmap v1102: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:20.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:19 smithi132 ceph-mon[70676]: pgmap v1103: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:20.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:19 smithi088 ceph-mon[88643]: pgmap v1103: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:22.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:21 smithi088 ceph-mon[88643]: pgmap v1104: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:22.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:21 smithi132 ceph-mon[70676]: pgmap v1104: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:24.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:23 smithi132 ceph-mon[70676]: pgmap v1105: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:24.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:23 smithi088 ceph-mon[88643]: pgmap v1105: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:26.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:25 smithi132 ceph-mon[70676]: pgmap v1106: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:26.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:25 smithi088 ceph-mon[88643]: pgmap v1106: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:27.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.939+0000 7fbf1b797700 1 -- 172.21.15.88:0/1354763299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf140ff3c0 msgr2=0x7fbf140ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.939+0000 7fbf1b797700 1 --2- 172.21.15.88:0/1354763299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf140ff3c0 0x7fbf140ff790 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009910 tx=0x7fbf10009c20 comp rx=0 tx=0).stop 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.939+0000 7fbf1b797700 1 -- 172.21.15.88:0/1354763299 shutdown_connections 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.939+0000 7fbf1b797700 1 --2- 172.21.15.88:0/1354763299 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf140ffd60 0x7fbf14103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.939+0000 7fbf1b797700 1 --2- 172.21.15.88:0/1354763299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf140ff3c0 0x7fbf140ff790 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.939+0000 7fbf1b797700 1 -- 172.21.15.88:0/1354763299 >> 172.21.15.88:0/1354763299 conn(0x7fbf140fac80 msgr2=0x7fbf140fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.939+0000 7fbf1b797700 1 -- 172.21.15.88:0/1354763299 shutdown_connections 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.939+0000 7fbf1b797700 1 -- 172.21.15.88:0/1354763299 wait complete. 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.939+0000 7fbf1b797700 1 Processor -- start 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf1b797700 1 -- start start 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf1b797700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf140ff3c0 0x7fbf14195f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf1b797700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf140ffd60 0x7fbf14196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf1b797700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf14196a90 con 0x7fbf140ff3c0 2024-02-19T19:40:27.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf1b797700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf14196bd0 con 0x7fbf140ffd60 2024-02-19T19:40:27.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf19533700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf140ff3c0 0x7fbf14195f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:27.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf19533700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf140ff3c0 0x7fbf14195f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42450/0 (socket says 172.21.15.88:42450) 2024-02-19T19:40:27.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf19533700 1 -- 172.21.15.88:0/1085057357 learned_addr learned my addr 172.21.15.88:0/1085057357 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:40:27.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf18d32700 1 --2- 172.21.15.88:0/1085057357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf140ffd60 0x7fbf14196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:27.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf19533700 1 -- 172.21.15.88:0/1085057357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf140ffd60 msgr2=0x7fbf14196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:27.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf19533700 1 --2- 172.21.15.88:0/1085057357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf140ffd60 0x7fbf14196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:27.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf19533700 1 -- 172.21.15.88:0/1085057357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100095f0 con 0x7fbf140ff3c0 2024-02-19T19:40:27.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.940+0000 7fbf19533700 1 --2- 172.21.15.88:0/1085057357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf140ff3c0 0x7fbf14195f00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fbf10006010 tx=0x7fbf1000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:27.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.941+0000 7fbf0a7fc700 1 -- 172.21.15.88:0/1085057357 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf1001d070 con 0x7fbf140ff3c0 2024-02-19T19:40:27.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.941+0000 7fbf0a7fc700 1 -- 172.21.15.88:0/1085057357 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf10004fb0 con 0x7fbf140ff3c0 2024-02-19T19:40:27.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.941+0000 7fbf0a7fc700 1 -- 172.21.15.88:0/1085057357 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf10022ea0 con 0x7fbf140ff3c0 2024-02-19T19:40:27.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.941+0000 7fbf1b797700 1 -- 172.21.15.88:0/1085057357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1419a9c0 con 0x7fbf140ff3c0 2024-02-19T19:40:27.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.941+0000 7fbf1b797700 1 -- 172.21.15.88:0/1085057357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1419aeb0 con 0x7fbf140ff3c0 2024-02-19T19:40:27.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.942+0000 7fbf1b797700 1 -- 172.21.15.88:0/1085057357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1404f2a0 con 0x7fbf140ff3c0 2024-02-19T19:40:27.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.942+0000 7fbf0a7fc700 1 -- 172.21.15.88:0/1085057357 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbf10022470 con 0x7fbf140ff3c0 2024-02-19T19:40:27.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.942+0000 7fbf0a7fc700 1 --2- 172.21.15.88:0/1085057357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf0006d8a0 0x7fbf0006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:27.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.943+0000 7fbf0a7fc700 1 -- 172.21.15.88:0/1085057357 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbf1008ea20 con 0x7fbf140ff3c0 2024-02-19T19:40:27.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.943+0000 7fbf18d32700 1 --2- 172.21.15.88:0/1085057357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf0006d8a0 0x7fbf0006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:27.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.943+0000 7fbf18d32700 1 --2- 172.21.15.88:0/1085057357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf0006d8a0 0x7fbf0006fd50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbf04005ea0 tx=0x7fbf04005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:27.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:27.947+0000 7fbf0a7fc700 1 -- 172.21.15.88:0/1085057357 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbf10058420 con 0x7fbf140ff3c0 2024-02-19T19:40:28.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.163+0000 7fbf1b797700 1 -- 172.21.15.88:0/1085057357 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf1419b190 con 0x7fbf0006d8a0 2024-02-19T19:40:28.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.164+0000 7fbf0a7fc700 1 -- 172.21.15.88:0/1085057357 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fbf1419b190 con 0x7fbf0006d8a0 2024-02-19T19:40:28.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.167+0000 7fbf1b797700 1 -- 172.21.15.88:0/1085057357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf0006d8a0 msgr2=0x7fbf0006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.167+0000 7fbf1b797700 1 --2- 172.21.15.88:0/1085057357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf0006d8a0 0x7fbf0006fd50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fbf04005ea0 tx=0x7fbf04005e30 comp rx=0 tx=0).stop 2024-02-19T19:40:28.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.167+0000 7fbf1b797700 1 -- 172.21.15.88:0/1085057357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf140ff3c0 msgr2=0x7fbf14195f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.167+0000 7fbf1b797700 1 --2- 172.21.15.88:0/1085057357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf140ff3c0 0x7fbf14195f00 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fbf10006010 tx=0x7fbf1000be20 comp rx=0 tx=0).stop 2024-02-19T19:40:28.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.167+0000 7fbf1b797700 1 -- 172.21.15.88:0/1085057357 shutdown_connections 2024-02-19T19:40:28.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.167+0000 7fbf1b797700 1 --2- 172.21.15.88:0/1085057357 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf0006d8a0 0x7fbf0006fd50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.167+0000 7fbf1b797700 1 --2- 172.21.15.88:0/1085057357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf140ff3c0 0x7fbf14195f00 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.167+0000 7fbf1b797700 1 --2- 172.21.15.88:0/1085057357 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbf140ffd60 0x7fbf14196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.167+0000 7fbf1b797700 1 -- 172.21.15.88:0/1085057357 >> 172.21.15.88:0/1085057357 conn(0x7fbf140fac80 msgr2=0x7fbf141034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:28.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.167+0000 7fbf1b797700 1 -- 172.21.15.88:0/1085057357 shutdown_connections 2024-02-19T19:40:28.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.167+0000 7fbf1b797700 1 -- 172.21.15.88:0/1085057357 wait complete. 2024-02-19T19:40:28.178 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:40:28.178 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:27 smithi088 ceph-mon[88643]: pgmap v1107: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:28.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:27 smithi132 ceph-mon[70676]: pgmap v1107: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:28.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.272+0000 7f664e69a700 1 -- 172.21.15.88:0/2938595434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6648100d70 msgr2=0x7f66481094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.272+0000 7f664e69a700 1 --2- 172.21.15.88:0/2938595434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6648100d70 0x7f66481094d0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f663c009940 tx=0x7f663c009c50 comp rx=0 tx=0).stop 2024-02-19T19:40:28.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.272+0000 7f664e69a700 1 -- 172.21.15.88:0/2938595434 shutdown_connections 2024-02-19T19:40:28.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.272+0000 7f664e69a700 1 --2- 172.21.15.88:0/2938595434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6648100d70 0x7f66481094d0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.272+0000 7f664e69a700 1 --2- 172.21.15.88:0/2938595434 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6648100460 0x7f6648100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.272+0000 7f664e69a700 1 -- 172.21.15.88:0/2938595434 >> 172.21.15.88:0/2938595434 conn(0x7f66480748b0 msgr2=0x7f6648074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:28.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.272+0000 7f664e69a700 1 -- 172.21.15.88:0/2938595434 shutdown_connections 2024-02-19T19:40:28.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.272+0000 7f664e69a700 1 -- 172.21.15.88:0/2938595434 wait complete. 2024-02-19T19:40:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f664e69a700 1 Processor -- start 2024-02-19T19:40:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f664e69a700 1 -- start start 2024-02-19T19:40:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f664e69a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6648100460 0x7f6648196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f664e69a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6648100d70 0x7f6648197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f664e69a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6648197880 con 0x7f6648100d70 2024-02-19T19:40:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f664e69a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66481979c0 con 0x7f6648100460 2024-02-19T19:40:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f66477fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6648100d70 0x7f6648197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f66477fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6648100d70 0x7f6648197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42472/0 (socket says 172.21.15.88:42472) 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f66477fe700 1 -- 172.21.15.88:0/2249684605 learned_addr learned my addr 172.21.15.88:0/2249684605 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f6647fff700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6648100460 0x7f6648196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f66477fe700 1 -- 172.21.15.88:0/2249684605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6648100460 msgr2=0x7f6648196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f66477fe700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6648100460 0x7f6648196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.273+0000 7f66477fe700 1 -- 172.21.15.88:0/2249684605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663c0095f0 con 0x7f6648100d70 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.274+0000 7f6647fff700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6648100460 0x7f6648196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.274+0000 7f66477fe700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6648100d70 0x7f6648197260 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f663c005620 tx=0x7f663c005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.274+0000 7f66457fa700 1 -- 172.21.15.88:0/2249684605 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f663c01d070 con 0x7f6648100d70 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.274+0000 7f66457fa700 1 -- 172.21.15.88:0/2249684605 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f663c00f460 con 0x7f6648100d70 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.274+0000 7f66457fa700 1 -- 172.21.15.88:0/2249684605 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f663c021640 con 0x7f6648100d70 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.274+0000 7f664e69a700 1 -- 172.21.15.88:0/2249684605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f664819fd90 con 0x7f6648100d70 2024-02-19T19:40:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.274+0000 7f664e69a700 1 -- 172.21.15.88:0/2249684605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66481a0250 con 0x7f6648100d70 2024-02-19T19:40:28.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.275+0000 7f664e69a700 1 -- 172.21.15.88:0/2249684605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6648106bd0 con 0x7f6648100d70 2024-02-19T19:40:28.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.275+0000 7f66457fa700 1 -- 172.21.15.88:0/2249684605 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f663c00fa40 con 0x7f6648100d70 2024-02-19T19:40:28.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.276+0000 7f66457fa700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f663006d8a0 0x7f663006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:28.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.276+0000 7f66457fa700 1 -- 172.21.15.88:0/2249684605 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f663c08e780 con 0x7f6648100d70 2024-02-19T19:40:28.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.276+0000 7f6647fff700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f663006d8a0 0x7f663006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:28.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.276+0000 7f6647fff700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f663006d8a0 0x7f663006fd50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6638007900 tx=0x7f6638008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:28.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.280+0000 7f66457fa700 1 -- 172.21.15.88:0/2249684605 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f663c057fb0 con 0x7f6648100d70 2024-02-19T19:40:28.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.497+0000 7f664e69a700 1 -- 172.21.15.88:0/2249684605 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6648061190 con 0x7f663006d8a0 2024-02-19T19:40:28.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.498+0000 7f66457fa700 1 -- 172.21.15.88:0/2249684605 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6648061190 con 0x7f663006d8a0 2024-02-19T19:40:28.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.500+0000 7f664e69a700 1 -- 172.21.15.88:0/2249684605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f663006d8a0 msgr2=0x7f663006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.500+0000 7f664e69a700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f663006d8a0 0x7f663006fd50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6638007900 tx=0x7f6638008040 comp rx=0 tx=0).stop 2024-02-19T19:40:28.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.500+0000 7f664e69a700 1 -- 172.21.15.88:0/2249684605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6648100d70 msgr2=0x7f6648197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.500+0000 7f664e69a700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6648100d70 0x7f6648197260 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f663c005620 tx=0x7f663c005700 comp rx=0 tx=0).stop 2024-02-19T19:40:28.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.501+0000 7f664e69a700 1 -- 172.21.15.88:0/2249684605 shutdown_connections 2024-02-19T19:40:28.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.501+0000 7f664e69a700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f663006d8a0 0x7f663006fd50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.501+0000 7f664e69a700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6648100460 0x7f6648196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.501+0000 7f664e69a700 1 --2- 172.21.15.88:0/2249684605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6648100d70 0x7f6648197260 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.501+0000 7f664e69a700 1 -- 172.21.15.88:0/2249684605 >> 172.21.15.88:0/2249684605 conn(0x7f66480748b0 msgr2=0x7f6648103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:28.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.501+0000 7f664e69a700 1 -- 172.21.15.88:0/2249684605 shutdown_connections 2024-02-19T19:40:28.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.501+0000 7f664e69a700 1 -- 172.21.15.88:0/2249684605 wait complete. 2024-02-19T19:40:28.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.605+0000 7f5f92285700 1 -- 172.21.15.88:0/4199843799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f8c0ffbd0 msgr2=0x7f5f8c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.605+0000 7f5f92285700 1 --2- 172.21.15.88:0/4199843799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f8c0ffbd0 0x7f5f8c100040 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009910 tx=0x7f5f7c009c20 comp rx=0 tx=0).stop 2024-02-19T19:40:28.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.605+0000 7f5f92285700 1 -- 172.21.15.88:0/4199843799 shutdown_connections 2024-02-19T19:40:28.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.605+0000 7f5f92285700 1 --2- 172.21.15.88:0/4199843799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f8c0ffbd0 0x7f5f8c100040 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.605+0000 7f5f92285700 1 --2- 172.21.15.88:0/4199843799 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f8c104130 0x7f5f8c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.605+0000 7f5f92285700 1 -- 172.21.15.88:0/4199843799 >> 172.21.15.88:0/4199843799 conn(0x7f5f8c075130 msgr2=0x7f5f8c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:28.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.605+0000 7f5f92285700 1 -- 172.21.15.88:0/4199843799 shutdown_connections 2024-02-19T19:40:28.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.605+0000 7f5f92285700 1 -- 172.21.15.88:0/4199843799 wait complete. 2024-02-19T19:40:28.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f92285700 1 Processor -- start 2024-02-19T19:40:28.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f92285700 1 -- start start 2024-02-19T19:40:28.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f92285700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f8c0ffbd0 0x7f5f8c196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:28.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f92285700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f8c104130 0x7f5f8c1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:28.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f92285700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c197800 con 0x7f5f8c0ffbd0 2024-02-19T19:40:28.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f92285700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8c197940 con 0x7f5f8c104130 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f8bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f8c0ffbd0 0x7f5f8c196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f8bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f8c0ffbd0 0x7f5f8c196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42488/0 (socket says 172.21.15.88:42488) 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f8bfff700 1 -- 172.21.15.88:0/1785812854 learned_addr learned my addr 172.21.15.88:0/1785812854 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f8b7fe700 1 --2- 172.21.15.88:0/1785812854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f8c104130 0x7f5f8c1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f8bfff700 1 -- 172.21.15.88:0/1785812854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f8c104130 msgr2=0x7f5f8c1971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f8bfff700 1 --2- 172.21.15.88:0/1785812854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f8c104130 0x7f5f8c1971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.606+0000 7f5f8bfff700 1 -- 172.21.15.88:0/1785812854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0095f0 con 0x7f5f8c0ffbd0 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.607+0000 7f5f8bfff700 1 --2- 172.21.15.88:0/1785812854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f8c0ffbd0 0x7f5f8c196ca0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000d900 tx=0x7f5f8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.607+0000 7f5f897fa700 1 -- 172.21.15.88:0/1785812854 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f800041f0 con 0x7f5f8c0ffbd0 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.607+0000 7f5f897fa700 1 -- 172.21.15.88:0/1785812854 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f80004cf0 con 0x7f5f8c0ffbd0 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.607+0000 7f5f92285700 1 -- 172.21.15.88:0/1785812854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8c19fd10 con 0x7f5f8c0ffbd0 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.607+0000 7f5f897fa700 1 -- 172.21.15.88:0/1785812854 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f8000b750 con 0x7f5f8c0ffbd0 2024-02-19T19:40:28.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.607+0000 7f5f92285700 1 -- 172.21.15.88:0/1785812854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8c1a01d0 con 0x7f5f8c0ffbd0 2024-02-19T19:40:28.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.608+0000 7f5f92285700 1 -- 172.21.15.88:0/1785812854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8c04f2a0 con 0x7f5f8c0ffbd0 2024-02-19T19:40:28.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.608+0000 7f5f897fa700 1 -- 172.21.15.88:0/1785812854 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5f80004830 con 0x7f5f8c0ffbd0 2024-02-19T19:40:28.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.609+0000 7f5f897fa700 1 --2- 172.21.15.88:0/1785812854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7406d8a0 0x7f5f7406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:28.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.609+0000 7f5f897fa700 1 -- 172.21.15.88:0/1785812854 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5f8008d430 con 0x7f5f8c0ffbd0 2024-02-19T19:40:28.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.609+0000 7f5f8b7fe700 1 --2- 172.21.15.88:0/1785812854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7406d8a0 0x7f5f7406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:28.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.609+0000 7f5f8b7fe700 1 --2- 172.21.15.88:0/1785812854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7406d8a0 0x7f5f7406fd50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c068df0 tx=0x7f5f7c01a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:28.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.613+0000 7f5f897fa700 1 -- 172.21.15.88:0/1785812854 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5f80056be0 con 0x7f5f8c0ffbd0 2024-02-19T19:40:28.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.821+0000 7f5f92285700 1 -- 172.21.15.88:0/1785812854 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f8c0ff240 con 0x7f5f7406d8a0 2024-02-19T19:40:28.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.832+0000 7f5f897fa700 1 -- 172.21.15.88:0/1785812854 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5f8c0ff240 con 0x7f5f7406d8a0 2024-02-19T19:40:28.832 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (37m) 13m ago 48m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (48m) 13m ago 48m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (30m) 14m ago 46m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (33m) 13m ago 48m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (33m) 14m ago 46m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (37m) 13m ago 47m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (29m) 13m ago 41m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (29m) 13m ago 41m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (29m) 14m ago 41m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (14m) 14m ago 41m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (39m) 13m ago 49m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (38m) 14m ago 46m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (35m) 13m ago 50m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (34m) 14m ago 46m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:40:28.833 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (38m) 13m ago 48m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:40:28.834 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (38m) 14m ago 46m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:40:28.834 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (33m) 13m ago 45m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:40:28.834 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (32m) 13m ago 45m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:40:28.834 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (31m) 13m ago 44m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:40:28.834 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (31m) 14m ago 44m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:40:28.834 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (30m) 14m ago 43m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:40:28.834 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (30m) 14m ago 43m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:40:28.834 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (37m) 13m ago 46m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:40:28.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.834+0000 7f5f92285700 1 -- 172.21.15.88:0/1785812854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7406d8a0 msgr2=0x7f5f7406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.834+0000 7f5f92285700 1 --2- 172.21.15.88:0/1785812854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7406d8a0 0x7f5f7406fd50 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c068df0 tx=0x7f5f7c01a040 comp rx=0 tx=0).stop 2024-02-19T19:40:28.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.834+0000 7f5f92285700 1 -- 172.21.15.88:0/1785812854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f8c0ffbd0 msgr2=0x7f5f8c196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.834+0000 7f5f92285700 1 --2- 172.21.15.88:0/1785812854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f8c0ffbd0 0x7f5f8c196ca0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000d900 tx=0x7f5f8000dcc0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.834+0000 7f5f92285700 1 -- 172.21.15.88:0/1785812854 shutdown_connections 2024-02-19T19:40:28.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.834+0000 7f5f92285700 1 --2- 172.21.15.88:0/1785812854 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5f7406d8a0 0x7f5f7406fd50 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.834+0000 7f5f92285700 1 --2- 172.21.15.88:0/1785812854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f8c0ffbd0 0x7f5f8c196ca0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.834+0000 7f5f92285700 1 --2- 172.21.15.88:0/1785812854 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5f8c104130 0x7f5f8c1971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.834+0000 7f5f92285700 1 -- 172.21.15.88:0/1785812854 >> 172.21.15.88:0/1785812854 conn(0x7f5f8c075130 msgr2=0x7f5f8c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:28.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.834+0000 7f5f92285700 1 -- 172.21.15.88:0/1785812854 shutdown_connections 2024-02-19T19:40:28.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.834+0000 7f5f92285700 1 -- 172.21.15.88:0/1785812854 wait complete. 2024-02-19T19:40:28.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.938+0000 7f5cf663a700 1 -- 172.21.15.88:0/3994142131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf01004e0 msgr2=0x7f5cf01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.938+0000 7f5cf663a700 1 --2- 172.21.15.88:0/3994142131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf01004e0 0x7f5cf01008b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009910 tx=0x7f5cd8009c20 comp rx=0 tx=0).stop 2024-02-19T19:40:28.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.938+0000 7f5cf663a700 1 -- 172.21.15.88:0/3994142131 shutdown_connections 2024-02-19T19:40:28.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.938+0000 7f5cf663a700 1 --2- 172.21.15.88:0/3994142131 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf0100e80 0x7f5cf0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.938+0000 7f5cf663a700 1 --2- 172.21.15.88:0/3994142131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf01004e0 0x7f5cf01008b0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.938+0000 7f5cf663a700 1 -- 172.21.15.88:0/3994142131 >> 172.21.15.88:0/3994142131 conn(0x7f5cf00fbda0 msgr2=0x7f5cf00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:28.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.938+0000 7f5cf663a700 1 -- 172.21.15.88:0/3994142131 shutdown_connections 2024-02-19T19:40:28.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.938+0000 7f5cf663a700 1 -- 172.21.15.88:0/3994142131 wait complete. 2024-02-19T19:40:28.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.938+0000 7f5cf663a700 1 Processor -- start 2024-02-19T19:40:28.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cf663a700 1 -- start start 2024-02-19T19:40:28.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cf663a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf01004e0 0x7f5cf0195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:28.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cf663a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf0100e80 0x7f5cf01964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:28.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cf663a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf0196bd0 con 0x7f5cf0100e80 2024-02-19T19:40:28.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cf663a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf019a960 con 0x7f5cf01004e0 2024-02-19T19:40:28.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cef7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf0100e80 0x7f5cf01964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:28.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cef7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf0100e80 0x7f5cf01964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42504/0 (socket says 172.21.15.88:42504) 2024-02-19T19:40:28.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cef7fe700 1 -- 172.21.15.88:0/1226680559 learned_addr learned my addr 172.21.15.88:0/1226680559 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:40:28.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5ceffff700 1 --2- 172.21.15.88:0/1226680559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf01004e0 0x7f5cf0195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:28.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cef7fe700 1 -- 172.21.15.88:0/1226680559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf01004e0 msgr2=0x7f5cf0195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:28.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cef7fe700 1 --2- 172.21.15.88:0/1226680559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf01004e0 0x7f5cf0195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:28.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cef7fe700 1 -- 172.21.15.88:0/1226680559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cd80095f0 con 0x7f5cf0100e80 2024-02-19T19:40:28.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.939+0000 7f5cef7fe700 1 --2- 172.21.15.88:0/1226680559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf0100e80 0x7f5cf01964f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000b480 tx=0x7f5ce000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:28.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.940+0000 7f5ced7fa700 1 -- 172.21.15.88:0/1226680559 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ce00118c0 con 0x7f5cf0100e80 2024-02-19T19:40:28.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.940+0000 7f5cf663a700 1 -- 172.21.15.88:0/1226680559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cf019ac40 con 0x7f5cf0100e80 2024-02-19T19:40:28.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.940+0000 7f5ced7fa700 1 -- 172.21.15.88:0/1226680559 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ce0010380 con 0x7f5cf0100e80 2024-02-19T19:40:28.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.940+0000 7f5ced7fa700 1 -- 172.21.15.88:0/1226680559 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ce000f4f0 con 0x7f5cf0100e80 2024-02-19T19:40:28.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.940+0000 7f5cf663a700 1 -- 172.21.15.88:0/1226680559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf019b190 con 0x7f5cf0100e80 2024-02-19T19:40:28.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.941+0000 7f5cf663a700 1 -- 172.21.15.88:0/1226680559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cf01087f0 con 0x7f5cf0100e80 2024-02-19T19:40:28.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.941+0000 7f5ced7fa700 1 -- 172.21.15.88:0/1226680559 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5ce00104f0 con 0x7f5cf0100e80 2024-02-19T19:40:28.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.941+0000 7f5ced7fa700 1 --2- 172.21.15.88:0/1226680559 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5cdc06d8a0 0x7f5cdc06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:28.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.942+0000 7f5ced7fa700 1 -- 172.21.15.88:0/1226680559 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5ce008d180 con 0x7f5cf0100e80 2024-02-19T19:40:28.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.942+0000 7f5ceffff700 1 --2- 172.21.15.88:0/1226680559 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5cdc06d8a0 0x7f5cdc06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:28.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.942+0000 7f5ceffff700 1 --2- 172.21.15.88:0/1226680559 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5cdc06d8a0 0x7f5cdc06fd50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800b5c0 tx=0x7f5cd801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:28.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:28.946+0000 7f5ced7fa700 1 -- 172.21.15.88:0/1226680559 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ce0056930 con 0x7f5cf0100e80 2024-02-19T19:40:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:28 smithi088 ceph-mon[88643]: from='client.35986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:40:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:40:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:28 smithi088 ceph-mon[88643]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:40:29.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.229+0000 7f5cf663a700 1 -- 172.21.15.88:0/1226680559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5cf019b410 con 0x7f5cf0100e80 2024-02-19T19:40:29.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.230+0000 7f5ced7fa700 1 -- 172.21.15.88:0/1226680559 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f5ce0059ff0 con 0x7f5cf0100e80 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:40:29.231 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:40:29.232 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:40:29.232 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:40:29.232 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:40:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.232+0000 7f5cf663a700 1 -- 172.21.15.88:0/1226680559 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5cdc06d8a0 msgr2=0x7f5cdc06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.232+0000 7f5cf663a700 1 --2- 172.21.15.88:0/1226680559 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5cdc06d8a0 0x7f5cdc06fd50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5cd800b5c0 tx=0x7f5cd801a040 comp rx=0 tx=0).stop 2024-02-19T19:40:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.232+0000 7f5cf663a700 1 -- 172.21.15.88:0/1226680559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf0100e80 msgr2=0x7f5cf01964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.232+0000 7f5cf663a700 1 --2- 172.21.15.88:0/1226680559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf0100e80 0x7f5cf01964f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000b480 tx=0x7f5ce000b840 comp rx=0 tx=0).stop 2024-02-19T19:40:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.232+0000 7f5cf663a700 1 -- 172.21.15.88:0/1226680559 shutdown_connections 2024-02-19T19:40:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.232+0000 7f5cf663a700 1 --2- 172.21.15.88:0/1226680559 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5cdc06d8a0 0x7f5cdc06fd50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.232+0000 7f5cf663a700 1 --2- 172.21.15.88:0/1226680559 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5cf01004e0 0x7f5cf0195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.232+0000 7f5cf663a700 1 --2- 172.21.15.88:0/1226680559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cf0100e80 0x7f5cf01964f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.232+0000 7f5cf663a700 1 -- 172.21.15.88:0/1226680559 >> 172.21.15.88:0/1226680559 conn(0x7f5cf00fbda0 msgr2=0x7f5cf00fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.232+0000 7f5cf663a700 1 -- 172.21.15.88:0/1226680559 shutdown_connections 2024-02-19T19:40:29.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.232+0000 7f5cf663a700 1 -- 172.21.15.88:0/1226680559 wait complete. 2024-02-19T19:40:29.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:28 smithi132 ceph-mon[70676]: from='client.35986 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:40:29.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:40:29.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:28 smithi132 ceph-mon[70676]: from='client.35990 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.335+0000 7fa5204ee700 1 -- 172.21.15.88:0/4116475973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5181012f0 msgr2=0x7fa5181016c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.335+0000 7fa5204ee700 1 --2- 172.21.15.88:0/4116475973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5181012f0 0x7fa5181016c0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fa508009910 tx=0x7fa508009c20 comp rx=0 tx=0).stop 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.335+0000 7fa5204ee700 1 -- 172.21.15.88:0/4116475973 shutdown_connections 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.335+0000 7fa5204ee700 1 --2- 172.21.15.88:0/4116475973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa518101c90 0x7fa51810a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.335+0000 7fa5204ee700 1 --2- 172.21.15.88:0/4116475973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5181012f0 0x7fa5181016c0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.335+0000 7fa5204ee700 1 -- 172.21.15.88:0/4116475973 >> 172.21.15.88:0/4116475973 conn(0x7fa5180fac80 msgr2=0x7fa5180fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.335+0000 7fa5204ee700 1 -- 172.21.15.88:0/4116475973 shutdown_connections 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.335+0000 7fa5204ee700 1 -- 172.21.15.88:0/4116475973 wait complete. 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.336+0000 7fa5204ee700 1 Processor -- start 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.336+0000 7fa5204ee700 1 -- start start 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.336+0000 7fa5204ee700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5181012f0 0x7fa518195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.336+0000 7fa5204ee700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa518101c90 0x7fa5181963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:29.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.336+0000 7fa5204ee700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa518196a30 con 0x7fa518101c90 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.336+0000 7fa5204ee700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa518196b70 con 0x7fa5181012f0 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.336+0000 7fa51da89700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa518101c90 0x7fa5181963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.336+0000 7fa51da89700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa518101c90 0x7fa5181963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42526/0 (socket says 172.21.15.88:42526) 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.336+0000 7fa51da89700 1 -- 172.21.15.88:0/4003675337 learned_addr learned my addr 172.21.15.88:0/4003675337 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.336+0000 7fa51e28a700 1 --2- 172.21.15.88:0/4003675337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5181012f0 0x7fa518195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.337+0000 7fa51da89700 1 -- 172.21.15.88:0/4003675337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5181012f0 msgr2=0x7fa518195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.337+0000 7fa51da89700 1 --2- 172.21.15.88:0/4003675337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5181012f0 0x7fa518195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.337+0000 7fa51da89700 1 -- 172.21.15.88:0/4003675337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5080095f0 con 0x7fa518101c90 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.337+0000 7fa51da89700 1 --2- 172.21.15.88:0/4003675337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa518101c90 0x7fa5181963e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c930 tx=0x7fa51400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.337+0000 7fa50f7fe700 1 -- 172.21.15.88:0/4003675337 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa514007ad0 con 0x7fa518101c90 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.337+0000 7fa50f7fe700 1 -- 172.21.15.88:0/4003675337 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa51400ce80 con 0x7fa518101c90 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.337+0000 7fa50f7fe700 1 -- 172.21.15.88:0/4003675337 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa514018630 con 0x7fa518101c90 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.337+0000 7fa5204ee700 1 -- 172.21.15.88:0/4003675337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa51819a970 con 0x7fa518101c90 2024-02-19T19:40:29.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.337+0000 7fa5204ee700 1 -- 172.21.15.88:0/4003675337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa51819aec0 con 0x7fa518101c90 2024-02-19T19:40:29.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.338+0000 7fa5204ee700 1 -- 172.21.15.88:0/4003675337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5181079d0 con 0x7fa518101c90 2024-02-19T19:40:29.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.339+0000 7fa50f7fe700 1 -- 172.21.15.88:0/4003675337 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa514007c30 con 0x7fa518101c90 2024-02-19T19:40:29.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.339+0000 7fa50f7fe700 1 --2- 172.21.15.88:0/4003675337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa50406d8a0 0x7fa50406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:29.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.339+0000 7fa50f7fe700 1 -- 172.21.15.88:0/4003675337 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa51408d560 con 0x7fa518101c90 2024-02-19T19:40:29.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.339+0000 7fa51e28a700 1 --2- 172.21.15.88:0/4003675337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa50406d8a0 0x7fa50406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:29.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.339+0000 7fa51e28a700 1 --2- 172.21.15.88:0/4003675337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa50406d8a0 0x7fa50406fd50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa508006010 tx=0x7fa50800b580 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:29.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.344+0000 7fa50f7fe700 1 -- 172.21.15.88:0/4003675337 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa514056d10 con 0x7fa518101c90 2024-02-19T19:40:29.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.585+0000 7fa5204ee700 1 -- 172.21.15.88:0/4003675337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa51802cc30 con 0x7fa518101c90 2024-02-19T19:40:29.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.585+0000 7fa50f7fe700 1 -- 172.21.15.88:0/4003675337 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7fa51405a3d0 con 0x7fa518101c90 2024-02-19T19:40:29.586 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:40:29.586 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:40:29.586 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:40:29.586 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:40:29.587 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:40:29.588 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:40:29.589 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:40:29.589 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:40:29.589 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:40:29.589 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:40:29.589 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:40:29.589 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:40:29.589 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:40:29.589 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:40:29.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.587+0000 7fa5204ee700 1 -- 172.21.15.88:0/4003675337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa50406d8a0 msgr2=0x7fa50406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:29.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.587+0000 7fa5204ee700 1 --2- 172.21.15.88:0/4003675337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa50406d8a0 0x7fa50406fd50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fa508006010 tx=0x7fa50800b580 comp rx=0 tx=0).stop 2024-02-19T19:40:29.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.587+0000 7fa5204ee700 1 -- 172.21.15.88:0/4003675337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa518101c90 msgr2=0x7fa5181963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:29.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.587+0000 7fa5204ee700 1 --2- 172.21.15.88:0/4003675337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa518101c90 0x7fa5181963e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7fa51400c930 tx=0x7fa51400cc40 comp rx=0 tx=0).stop 2024-02-19T19:40:29.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.587+0000 7fa5204ee700 1 -- 172.21.15.88:0/4003675337 shutdown_connections 2024-02-19T19:40:29.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.587+0000 7fa5204ee700 1 --2- 172.21.15.88:0/4003675337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa50406d8a0 0x7fa50406fd50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.587+0000 7fa5204ee700 1 --2- 172.21.15.88:0/4003675337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa5181012f0 0x7fa518195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.587+0000 7fa5204ee700 1 --2- 172.21.15.88:0/4003675337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa518101c90 0x7fa5181963e0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.587+0000 7fa5204ee700 1 -- 172.21.15.88:0/4003675337 >> 172.21.15.88:0/4003675337 conn(0x7fa5180fac80 msgr2=0x7fa518104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:29.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.588+0000 7fa5204ee700 1 -- 172.21.15.88:0/4003675337 shutdown_connections 2024-02-19T19:40:29.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.588+0000 7fa5204ee700 1 -- 172.21.15.88:0/4003675337 wait complete. 2024-02-19T19:40:29.591 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:40:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.692+0000 7f932efa4700 1 -- 172.21.15.88:0/986141512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9328104130 msgr2=0x7f9328104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.692+0000 7f932efa4700 1 --2- 172.21.15.88:0/986141512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9328104130 0x7f9328104500 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f931c009910 tx=0x7f931c009c20 comp rx=0 tx=0).stop 2024-02-19T19:40:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.692+0000 7f932efa4700 1 -- 172.21.15.88:0/986141512 shutdown_connections 2024-02-19T19:40:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.692+0000 7f932efa4700 1 --2- 172.21.15.88:0/986141512 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f93280ffbd0 0x7f9328100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.692+0000 7f932efa4700 1 --2- 172.21.15.88:0/986141512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9328104130 0x7f9328104500 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.692+0000 7f932efa4700 1 -- 172.21.15.88:0/986141512 >> 172.21.15.88:0/986141512 conn(0x7f9328075130 msgr2=0x7f9328075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.692+0000 7f932efa4700 1 -- 172.21.15.88:0/986141512 shutdown_connections 2024-02-19T19:40:29.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.692+0000 7f932efa4700 1 -- 172.21.15.88:0/986141512 wait complete. 2024-02-19T19:40:29.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932efa4700 1 Processor -- start 2024-02-19T19:40:29.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932efa4700 1 -- start start 2024-02-19T19:40:29.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932efa4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93280ffbd0 0x7f9328195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:29.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932efa4700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9328104130 0x7f9328196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:29.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932efa4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281969e0 con 0x7f93280ffbd0 2024-02-19T19:40:29.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932efa4700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9328196b20 con 0x7f9328104130 2024-02-19T19:40:29.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932cd40700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93280ffbd0 0x7f9328195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:29.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932cd40700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93280ffbd0 0x7f9328195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42544/0 (socket says 172.21.15.88:42544) 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932cd40700 1 -- 172.21.15.88:0/271937184 learned_addr learned my addr 172.21.15.88:0/271937184 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f9327fff700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9328104130 0x7f9328196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932cd40700 1 -- 172.21.15.88:0/271937184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9328104130 msgr2=0x7f9328196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932cd40700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9328104130 0x7f9328196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932cd40700 1 -- 172.21.15.88:0/271937184 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c0095f0 con 0x7f93280ffbd0 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f9327fff700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9328104130 0x7f9328196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.693+0000 7f932cd40700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93280ffbd0 0x7f9328195e50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f931c006010 tx=0x7f931c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.694+0000 7f9325ffb700 1 -- 172.21.15.88:0/271937184 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f931c01d070 con 0x7f93280ffbd0 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.694+0000 7f9325ffb700 1 -- 172.21.15.88:0/271937184 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f931c004fb0 con 0x7f93280ffbd0 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.694+0000 7f932efa4700 1 -- 172.21.15.88:0/271937184 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932819a910 con 0x7f93280ffbd0 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.694+0000 7f9325ffb700 1 -- 172.21.15.88:0/271937184 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f931c00f690 con 0x7f93280ffbd0 2024-02-19T19:40:29.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.694+0000 7f932efa4700 1 -- 172.21.15.88:0/271937184 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932819ae00 con 0x7f93280ffbd0 2024-02-19T19:40:29.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.695+0000 7f932efa4700 1 -- 172.21.15.88:0/271937184 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93281086a0 con 0x7f93280ffbd0 2024-02-19T19:40:29.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.695+0000 7f9325ffb700 1 -- 172.21.15.88:0/271937184 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f931c022470 con 0x7f93280ffbd0 2024-02-19T19:40:29.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.695+0000 7f9325ffb700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f931806d8a0 0x7f931806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:29.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.696+0000 7f9325ffb700 1 -- 172.21.15.88:0/271937184 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f931c08ec10 con 0x7f93280ffbd0 2024-02-19T19:40:29.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.696+0000 7f9327fff700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f931806d8a0 0x7f931806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:29.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.696+0000 7f9327fff700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f931806d8a0 0x7f931806fd50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9314005fd0 tx=0x7f9314005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:29.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.700+0000 7f9325ffb700 1 -- 172.21.15.88:0/271937184 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f931c058440 con 0x7f93280ffbd0 2024-02-19T19:40:29.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.917+0000 7f932efa4700 1 -- 172.21.15.88:0/271937184 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f932819b070 con 0x7f931806d8a0 2024-02-19T19:40:29.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.918+0000 7f9325ffb700 1 -- 172.21.15.88:0/271937184 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f932819b070 con 0x7f931806d8a0 2024-02-19T19:40:29.919 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:40:29.919 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:40:29.919 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:40:29.919 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:40:29.919 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:40:29.920 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:40:29.920 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:40:29.920 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:40:29.920 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:40:29.920 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:40:29.920 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:40:29.920 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:40:29.920 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:40:29.920 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:40:29.920 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:40:29.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.921+0000 7f932efa4700 1 -- 172.21.15.88:0/271937184 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f931806d8a0 msgr2=0x7f931806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:29.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.921+0000 7f932efa4700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f931806d8a0 0x7f931806fd50 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9314005fd0 tx=0x7f9314005e20 comp rx=0 tx=0).stop 2024-02-19T19:40:29.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.921+0000 7f932efa4700 1 -- 172.21.15.88:0/271937184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93280ffbd0 msgr2=0x7f9328195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:29.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.921+0000 7f932efa4700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93280ffbd0 0x7f9328195e50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f931c006010 tx=0x7f931c00be20 comp rx=0 tx=0).stop 2024-02-19T19:40:29.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.922+0000 7f932efa4700 1 -- 172.21.15.88:0/271937184 shutdown_connections 2024-02-19T19:40:29.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.922+0000 7f932efa4700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f931806d8a0 0x7f931806fd50 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.922+0000 7f932efa4700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93280ffbd0 0x7f9328195e50 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.922+0000 7f932efa4700 1 --2- 172.21.15.88:0/271937184 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9328104130 0x7f9328196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:29.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.922+0000 7f932efa4700 1 -- 172.21.15.88:0/271937184 >> 172.21.15.88:0/271937184 conn(0x7f9328075130 msgr2=0x7f93280fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:29.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.922+0000 7f932efa4700 1 -- 172.21.15.88:0/271937184 shutdown_connections 2024-02-19T19:40:29.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:29.922+0000 7f932efa4700 1 -- 172.21.15.88:0/271937184 wait complete. 2024-02-19T19:40:30.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.028+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2929864663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00687a0 msgr2=0x7f94a0068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:30.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.028+0000 7f94a5ba7700 1 --2- 172.21.15.88:0/2929864663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00687a0 0x7f94a0068b70 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9490009910 tx=0x7f9490009c20 comp rx=0 tx=0).stop 2024-02-19T19:40:30.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.028+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2929864663 shutdown_connections 2024-02-19T19:40:30.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.028+0000 7f94a5ba7700 1 --2- 172.21.15.88:0/2929864663 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a0069140 0x7f94a0104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:30.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.028+0000 7f94a5ba7700 1 --2- 172.21.15.88:0/2929864663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00687a0 0x7f94a0068b70 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:30.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.028+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2929864663 >> 172.21.15.88:0/2929864663 conn(0x7f94a0075130 msgr2=0x7f94a0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:30.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.028+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2929864663 shutdown_connections 2024-02-19T19:40:30.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.028+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2929864663 wait complete. 2024-02-19T19:40:30.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.028+0000 7f94a5ba7700 1 Processor -- start 2024-02-19T19:40:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f94a5ba7700 1 -- start start 2024-02-19T19:40:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f94a5ba7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a00687a0 0x7f94a0195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f94a5ba7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a0069140 0x7f94a0196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f94a5ba7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0196b00 con 0x7f94a0069140 2024-02-19T19:40:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f94a5ba7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a019a890 con 0x7f94a00687a0 2024-02-19T19:40:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f949effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a0069140 0x7f94a0196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f949effd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a0069140 0x7f94a0196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:42558/0 (socket says 172.21.15.88:42558) 2024-02-19T19:40:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f949effd700 1 -- 172.21.15.88:0/2496979758 learned_addr learned my addr 172.21.15.88:0/2496979758 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:40:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f949f7fe700 1 --2- 172.21.15.88:0/2496979758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a00687a0 0x7f94a0195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f949effd700 1 -- 172.21.15.88:0/2496979758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a00687a0 msgr2=0x7f94a0195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:30.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f949effd700 1 --2- 172.21.15.88:0/2496979758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a00687a0 0x7f94a0195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:30.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f949effd700 1 -- 172.21.15.88:0/2496979758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94900095f0 con 0x7f94a0069140 2024-02-19T19:40:30.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f949effd700 1 --2- 172.21.15.88:0/2496979758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a0069140 0x7f94a0196420 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f949400d900 tx=0x7f949400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:30.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.029+0000 7f949cff9700 1 -- 172.21.15.88:0/2496979758 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9494009900 con 0x7f94a0069140 2024-02-19T19:40:30.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.030+0000 7f949cff9700 1 -- 172.21.15.88:0/2496979758 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9494010460 con 0x7f94a0069140 2024-02-19T19:40:30.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.030+0000 7f949cff9700 1 -- 172.21.15.88:0/2496979758 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f949400f5f0 con 0x7f94a0069140 2024-02-19T19:40:30.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.030+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2496979758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a019ab70 con 0x7f94a0069140 2024-02-19T19:40:30.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.030+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2496979758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a019b0c0 con 0x7f94a0069140 2024-02-19T19:40:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.031+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2496979758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a0108770 con 0x7f94a0069140 2024-02-19T19:40:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.031+0000 7f949cff9700 1 -- 172.21.15.88:0/2496979758 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f94940105d0 con 0x7f94a0069140 2024-02-19T19:40:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.031+0000 7f949cff9700 1 --2- 172.21.15.88:0/2496979758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 0x7f948806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:40:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.032+0000 7f949cff9700 1 -- 172.21.15.88:0/2496979758 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f949408d180 con 0x7f94a0069140 2024-02-19T19:40:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.032+0000 7f949f7fe700 1 --2- 172.21.15.88:0/2496979758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 0x7f948806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:40:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.032+0000 7f949f7fe700 1 --2- 172.21.15.88:0/2496979758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 0x7f948806fd50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f949000b5c0 tx=0x7f949001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:40:30.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.036+0000 7f949cff9700 1 -- 172.21.15.88:0/2496979758 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9494056930 con 0x7f94a0069140 2024-02-19T19:40:30.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:29 smithi132 ceph-mon[70676]: pgmap v1108: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:30.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:29 smithi132 ceph-mon[70676]: from='client.35994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:40:30.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:29 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1226680559' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:40:30.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:29 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4003675337' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:40:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.317+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2496979758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f94a004f2a0 con 0x7f94a0069140 2024-02-19T19:40:30.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:29 smithi088 ceph-mon[88643]: pgmap v1108: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:30.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:29 smithi088 ceph-mon[88643]: from='client.35994 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:40:30.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:29 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1226680559' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:40:30.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:29 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4003675337' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:40:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.317+0000 7f949cff9700 1 -- 172.21.15.88:0/2496979758 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f9494059ff0 con 0x7f94a0069140 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:40:30.319 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:40:30.320 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:40:30.320 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:40:30.320 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:40:30.320 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:40:30.320 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:40:30.320 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:40:30.320 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:40:30.320 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:40:30.320 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:40:30.320 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:40:30.320 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:40:30.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.319+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2496979758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 msgr2=0x7f948806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:30.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.319+0000 7f94a5ba7700 1 --2- 172.21.15.88:0/2496979758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 0x7f948806fd50 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f949000b5c0 tx=0x7f949001a040 comp rx=0 tx=0).stop 2024-02-19T19:40:30.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.319+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2496979758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a0069140 msgr2=0x7f94a0196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:40:30.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.319+0000 7f94a5ba7700 1 --2- 172.21.15.88:0/2496979758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a0069140 0x7f94a0196420 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f949400d900 tx=0x7f949400dcc0 comp rx=0 tx=0).stop 2024-02-19T19:40:30.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.320+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2496979758 shutdown_connections 2024-02-19T19:40:30.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.320+0000 7f94a5ba7700 1 --2- 172.21.15.88:0/2496979758 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f948806d8a0 0x7f948806fd50 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:30.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.320+0000 7f94a5ba7700 1 --2- 172.21.15.88:0/2496979758 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f94a00687a0 0x7f94a0195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:30.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.320+0000 7f94a5ba7700 1 --2- 172.21.15.88:0/2496979758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a0069140 0x7f94a0196420 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:40:30.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.320+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2496979758 >> 172.21.15.88:0/2496979758 conn(0x7f94a0075130 msgr2=0x7f94a00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:40:30.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.320+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2496979758 shutdown_connections 2024-02-19T19:40:30.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:40:30.320+0000 7f94a5ba7700 1 -- 172.21.15.88:0/2496979758 wait complete. 2024-02-19T19:40:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:30 smithi132 ceph-mon[70676]: from='client.36006 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:40:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:30 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2496979758' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:40:31.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:30 smithi088 ceph-mon[88643]: from='client.36006 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:40:31.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:30 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2496979758' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:40:32.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:31 smithi132 ceph-mon[70676]: pgmap v1109: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:32.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:31 smithi088 ceph-mon[88643]: pgmap v1109: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:34.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:33 smithi132 ceph-mon[70676]: pgmap v1110: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:34.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:33 smithi088 ceph-mon[88643]: pgmap v1110: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:36.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:35 smithi132 ceph-mon[70676]: pgmap v1111: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:36.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:35 smithi088 ceph-mon[88643]: pgmap v1111: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:38.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:37 smithi132 ceph-mon[70676]: pgmap v1112: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:38.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:37 smithi088 ceph-mon[88643]: pgmap v1112: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:40.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:39 smithi132 ceph-mon[70676]: pgmap v1113: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:40.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:39 smithi088 ceph-mon[88643]: pgmap v1113: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:42.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:41 smithi132 ceph-mon[70676]: pgmap v1114: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:42.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:41 smithi088 ceph-mon[88643]: pgmap v1114: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:43 smithi132 ceph-mon[70676]: pgmap v1115: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:40:44.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:43 smithi088 ceph-mon[88643]: pgmap v1115: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:40:46.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:45 smithi132 ceph-mon[70676]: pgmap v1116: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:46.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:45 smithi088 ceph-mon[88643]: pgmap v1116: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:48.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:47 smithi132 ceph-mon[70676]: pgmap v1117: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:48.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:47 smithi088 ceph-mon[88643]: pgmap v1117: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:50.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:49 smithi132 ceph-mon[70676]: pgmap v1118: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:50.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:49 smithi088 ceph-mon[88643]: pgmap v1118: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:52.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:51 smithi132 ceph-mon[70676]: pgmap v1119: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:52.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:51 smithi088 ceph-mon[88643]: pgmap v1119: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:54.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:53 smithi132 ceph-mon[70676]: pgmap v1120: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:54.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:53 smithi088 ceph-mon[88643]: pgmap v1120: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:56.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:55 smithi132 ceph-mon[70676]: pgmap v1121: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:56.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:55 smithi088 ceph-mon[88643]: pgmap v1121: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:58.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:57 smithi132 ceph-mon[70676]: pgmap v1122: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:58.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:57 smithi088 ceph-mon[88643]: pgmap v1122: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:40:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:40:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:41:00.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:40:59 smithi088 ceph-mon[88643]: pgmap v1123: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:00.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.438+0000 7f3cabddb700 1 -- 172.21.15.88:0/1790466985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ca4100e80 msgr2=0x7f3ca4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.438+0000 7f3cabddb700 1 --2- 172.21.15.88:0/1790466985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ca4100e80 0x7f3ca4104e50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f3c94009910 tx=0x7f3c94009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.438+0000 7f3cabddb700 1 -- 172.21.15.88:0/1790466985 shutdown_connections 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.438+0000 7f3cabddb700 1 --2- 172.21.15.88:0/1790466985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ca4100e80 0x7f3ca4104e50 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.438+0000 7f3cabddb700 1 --2- 172.21.15.88:0/1790466985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ca41004e0 0x7f3ca41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.438+0000 7f3cabddb700 1 -- 172.21.15.88:0/1790466985 >> 172.21.15.88:0/1790466985 conn(0x7f3ca40fbda0 msgr2=0x7f3ca40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.438+0000 7f3cabddb700 1 -- 172.21.15.88:0/1790466985 shutdown_connections 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.438+0000 7f3cabddb700 1 -- 172.21.15.88:0/1790466985 wait complete. 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.438+0000 7f3cabddb700 1 Processor -- start 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.438+0000 7f3cabddb700 1 -- start start 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3cabddb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ca41004e0 0x7f3ca4195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3cabddb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ca4100e80 0x7f3ca4196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3cabddb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca4196b30 con 0x7f3ca41004e0 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3cabddb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca419a8c0 con 0x7f3ca4100e80 2024-02-19T19:41:00.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3ca9b77700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ca41004e0 0x7f3ca4195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3ca9b77700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ca41004e0 0x7f3ca4195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49302/0 (socket says 172.21.15.88:49302) 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3ca9b77700 1 -- 172.21.15.88:0/3370011915 learned_addr learned my addr 172.21.15.88:0/3370011915 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3ca9376700 1 --2- 172.21.15.88:0/3370011915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ca4100e80 0x7f3ca4196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3ca9b77700 1 -- 172.21.15.88:0/3370011915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ca4100e80 msgr2=0x7f3ca4196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3ca9b77700 1 --2- 172.21.15.88:0/3370011915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ca4100e80 0x7f3ca4196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3ca9b77700 1 -- 172.21.15.88:0/3370011915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c940095f0 con 0x7f3ca41004e0 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3ca9b77700 1 --2- 172.21.15.88:0/3370011915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ca41004e0 0x7f3ca4195f10 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000d8d0 tx=0x7f3ca000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.439+0000 7f3c9affd700 1 -- 172.21.15.88:0/3370011915 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ca0009940 con 0x7f3ca41004e0 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.440+0000 7f3c9affd700 1 -- 172.21.15.88:0/3370011915 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ca0010460 con 0x7f3ca41004e0 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.440+0000 7f3cabddb700 1 -- 172.21.15.88:0/3370011915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca419aba0 con 0x7f3ca41004e0 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.440+0000 7f3c9affd700 1 -- 172.21.15.88:0/3370011915 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ca000f5f0 con 0x7f3ca41004e0 2024-02-19T19:41:00.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.440+0000 7f3cabddb700 1 -- 172.21.15.88:0/3370011915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca419b0f0 con 0x7f3ca41004e0 2024-02-19T19:41:00.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.441+0000 7f3cabddb700 1 -- 172.21.15.88:0/3370011915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca4108850 con 0x7f3ca41004e0 2024-02-19T19:41:00.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.441+0000 7f3c9affd700 1 -- 172.21.15.88:0/3370011915 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3ca00105d0 con 0x7f3ca41004e0 2024-02-19T19:41:00.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.442+0000 7f3c9affd700 1 --2- 172.21.15.88:0/3370011915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c9006d8a0 0x7f3c9006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:00.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.442+0000 7f3ca9376700 1 --2- 172.21.15.88:0/3370011915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c9006d8a0 0x7f3c9006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:00.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.442+0000 7f3c9affd700 1 -- 172.21.15.88:0/3370011915 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3ca008d540 con 0x7f3ca41004e0 2024-02-19T19:41:00.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.442+0000 7f3ca9376700 1 --2- 172.21.15.88:0/3370011915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c9006d8a0 0x7f3c9006fd50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4197530 tx=0x7f3c9401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:00.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.447+0000 7f3c9affd700 1 -- 172.21.15.88:0/3370011915 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3ca0056cf0 con 0x7f3ca41004e0 2024-02-19T19:41:00.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:40:59 smithi132 ceph-mon[70676]: pgmap v1123: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:00.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.663+0000 7f3cabddb700 1 -- 172.21.15.88:0/3370011915 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ca4197380 con 0x7f3c9006d8a0 2024-02-19T19:41:00.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.665+0000 7f3c9affd700 1 -- 172.21.15.88:0/3370011915 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f3ca4197380 con 0x7f3c9006d8a0 2024-02-19T19:41:00.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.667+0000 7f3cabddb700 1 -- 172.21.15.88:0/3370011915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c9006d8a0 msgr2=0x7f3c9006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:00.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.667+0000 7f3cabddb700 1 --2- 172.21.15.88:0/3370011915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c9006d8a0 0x7f3c9006fd50 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4197530 tx=0x7f3c9401a040 comp rx=0 tx=0).stop 2024-02-19T19:41:00.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.667+0000 7f3cabddb700 1 -- 172.21.15.88:0/3370011915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ca41004e0 msgr2=0x7f3ca4195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:00.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.667+0000 7f3cabddb700 1 --2- 172.21.15.88:0/3370011915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ca41004e0 0x7f3ca4195f10 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000d8d0 tx=0x7f3ca000dc90 comp rx=0 tx=0).stop 2024-02-19T19:41:00.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.667+0000 7f3cabddb700 1 -- 172.21.15.88:0/3370011915 shutdown_connections 2024-02-19T19:41:00.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.667+0000 7f3cabddb700 1 --2- 172.21.15.88:0/3370011915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f3c9006d8a0 0x7f3c9006fd50 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:00.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.667+0000 7f3cabddb700 1 --2- 172.21.15.88:0/3370011915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ca41004e0 0x7f3ca4195f10 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:00.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.667+0000 7f3cabddb700 1 --2- 172.21.15.88:0/3370011915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3ca4100e80 0x7f3ca4196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:00.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.667+0000 7f3cabddb700 1 -- 172.21.15.88:0/3370011915 >> 172.21.15.88:0/3370011915 conn(0x7f3ca40fbda0 msgr2=0x7f3ca40fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:00.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.667+0000 7f3cabddb700 1 -- 172.21.15.88:0/3370011915 shutdown_connections 2024-02-19T19:41:00.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.667+0000 7f3cabddb700 1 -- 172.21.15.88:0/3370011915 wait complete. 2024-02-19T19:41:00.678 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:41:00.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.771+0000 7fd6b71f0700 1 -- 172.21.15.88:0/379388275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b0100e80 msgr2=0x7fd6b0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:00.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.771+0000 7fd6b71f0700 1 --2- 172.21.15.88:0/379388275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b0100e80 0x7fd6b0104e50 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd69c009910 tx=0x7fd69c009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:00.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.771+0000 7fd6b71f0700 1 -- 172.21.15.88:0/379388275 shutdown_connections 2024-02-19T19:41:00.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.771+0000 7fd6b71f0700 1 --2- 172.21.15.88:0/379388275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b0100e80 0x7fd6b0104e50 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:00.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.771+0000 7fd6b71f0700 1 --2- 172.21.15.88:0/379388275 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6b01004e0 0x7fd6b01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:00.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.771+0000 7fd6b71f0700 1 -- 172.21.15.88:0/379388275 >> 172.21.15.88:0/379388275 conn(0x7fd6b00fbda0 msgr2=0x7fd6b00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:00.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.771+0000 7fd6b71f0700 1 -- 172.21.15.88:0/379388275 shutdown_connections 2024-02-19T19:41:00.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.772+0000 7fd6b71f0700 1 -- 172.21.15.88:0/379388275 wait complete. 2024-02-19T19:41:00.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.772+0000 7fd6b71f0700 1 Processor -- start 2024-02-19T19:41:00.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.772+0000 7fd6b71f0700 1 -- start start 2024-02-19T19:41:00.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.772+0000 7fd6b71f0700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6b01004e0 0x7fd6b0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:00.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.772+0000 7fd6b71f0700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b0100e80 0x7fd6b0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:00.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.772+0000 7fd6b71f0700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b0073ad0 con 0x7fd6b0100e80 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.772+0000 7fd6b71f0700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b0073c10 con 0x7fd6b01004e0 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.772+0000 7fd6affff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b0100e80 0x7fd6b0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.772+0000 7fd6affff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b0100e80 0x7fd6b0073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49310/0 (socket says 172.21.15.88:49310) 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.772+0000 7fd6affff700 1 -- 172.21.15.88:0/4274279007 learned_addr learned my addr 172.21.15.88:0/4274279007 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.773+0000 7fd6b4f8c700 1 --2- 172.21.15.88:0/4274279007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6b01004e0 0x7fd6b0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.773+0000 7fd6affff700 1 -- 172.21.15.88:0/4274279007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6b01004e0 msgr2=0x7fd6b0073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.773+0000 7fd6affff700 1 --2- 172.21.15.88:0/4274279007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6b01004e0 0x7fd6b0073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.773+0000 7fd6affff700 1 -- 172.21.15.88:0/4274279007 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c0095f0 con 0x7fd6b0100e80 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.773+0000 7fd6affff700 1 --2- 172.21.15.88:0/4274279007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b0100e80 0x7fd6b0073590 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00bd70 tx=0x7fd69c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.773+0000 7fd6adffb700 1 -- 172.21.15.88:0/4274279007 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd69c01d070 con 0x7fd6b0100e80 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.773+0000 7fd6adffb700 1 -- 172.21.15.88:0/4274279007 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd69c005dc0 con 0x7fd6b0100e80 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.773+0000 7fd6b71f0700 1 -- 172.21.15.88:0/4274279007 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6b0073e90 con 0x7fd6b0100e80 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.773+0000 7fd6adffb700 1 -- 172.21.15.88:0/4274279007 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd69c017400 con 0x7fd6b0100e80 2024-02-19T19:41:00.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.773+0000 7fd6b71f0700 1 -- 172.21.15.88:0/4274279007 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6b01a23c0 con 0x7fd6b0100e80 2024-02-19T19:41:00.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.774+0000 7fd6b71f0700 1 -- 172.21.15.88:0/4274279007 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b0108850 con 0x7fd6b0100e80 2024-02-19T19:41:00.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.775+0000 7fd6adffb700 1 -- 172.21.15.88:0/4274279007 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd69c00f460 con 0x7fd6b0100e80 2024-02-19T19:41:00.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.776+0000 7fd6adffb700 1 --2- 172.21.15.88:0/4274279007 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd6a006d8a0 0x7fd6a006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:00.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.776+0000 7fd6adffb700 1 -- 172.21.15.88:0/4274279007 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd69c08e980 con 0x7fd6b0100e80 2024-02-19T19:41:00.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.776+0000 7fd6b4f8c700 1 --2- 172.21.15.88:0/4274279007 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd6a006d8a0 0x7fd6a006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:00.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.776+0000 7fd6b4f8c700 1 --2- 172.21.15.88:0/4274279007 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd6a006d8a0 0x7fd6a006fd50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4005950 tx=0x7fd6a400b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:00.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.781+0000 7fd6adffb700 1 -- 172.21.15.88:0/4274279007 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd69c058130 con 0x7fd6b0100e80 2024-02-19T19:41:00.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.996+0000 7fd6b71f0700 1 -- 172.21.15.88:0/4274279007 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6b0074580 con 0x7fd6a006d8a0 2024-02-19T19:41:00.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:00.998+0000 7fd6adffb700 1 -- 172.21.15.88:0/4274279007 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd6b0074580 con 0x7fd6a006d8a0 2024-02-19T19:41:01.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.000+0000 7fd6b71f0700 1 -- 172.21.15.88:0/4274279007 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd6a006d8a0 msgr2=0x7fd6a006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.000+0000 7fd6b71f0700 1 --2- 172.21.15.88:0/4274279007 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd6a006d8a0 0x7fd6a006fd50 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4005950 tx=0x7fd6a400b410 comp rx=0 tx=0).stop 2024-02-19T19:41:01.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.000+0000 7fd6b71f0700 1 -- 172.21.15.88:0/4274279007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b0100e80 msgr2=0x7fd6b0073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.000+0000 7fd6b71f0700 1 --2- 172.21.15.88:0/4274279007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b0100e80 0x7fd6b0073590 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00bd70 tx=0x7fd69c00bda0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.000+0000 7fd6b71f0700 1 -- 172.21.15.88:0/4274279007 shutdown_connections 2024-02-19T19:41:01.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.000+0000 7fd6b71f0700 1 --2- 172.21.15.88:0/4274279007 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd6a006d8a0 0x7fd6a006fd50 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.000+0000 7fd6b71f0700 1 --2- 172.21.15.88:0/4274279007 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd6b01004e0 0x7fd6b0073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.000+0000 7fd6b71f0700 1 --2- 172.21.15.88:0/4274279007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6b0100e80 0x7fd6b0073590 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.000+0000 7fd6b71f0700 1 -- 172.21.15.88:0/4274279007 >> 172.21.15.88:0/4274279007 conn(0x7fd6b00fbda0 msgr2=0x7fd6b00fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:01.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.000+0000 7fd6b71f0700 1 -- 172.21.15.88:0/4274279007 shutdown_connections 2024-02-19T19:41:01.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.000+0000 7fd6b71f0700 1 -- 172.21.15.88:0/4274279007 wait complete. 2024-02-19T19:41:01.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:00 smithi088 ceph-mon[88643]: from='client.36014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:01.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.104+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/1250306200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c781004e0 msgr2=0x7f8c781008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.104+0000 7f8c7d7d1700 1 --2- 172.21.15.88:0/1250306200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c781004e0 0x7f8c781008b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009910 tx=0x7f8c68009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:01.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.104+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/1250306200 shutdown_connections 2024-02-19T19:41:01.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.104+0000 7f8c7d7d1700 1 --2- 172.21.15.88:0/1250306200 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c78100e80 0x7f8c78104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.104+0000 7f8c7d7d1700 1 --2- 172.21.15.88:0/1250306200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c781004e0 0x7f8c781008b0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.104+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/1250306200 >> 172.21.15.88:0/1250306200 conn(0x7f8c780fbda0 msgr2=0x7f8c780fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:01.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.104+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/1250306200 shutdown_connections 2024-02-19T19:41:01.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.104+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/1250306200 wait complete. 2024-02-19T19:41:01.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.105+0000 7f8c7d7d1700 1 Processor -- start 2024-02-19T19:41:01.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.105+0000 7f8c7d7d1700 1 -- start start 2024-02-19T19:41:01.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.105+0000 7f8c7d7d1700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c781004e0 0x7f8c78195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:01.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.105+0000 7f8c7d7d1700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c78100e80 0x7f8c78196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.105+0000 7f8c7d7d1700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78196b10 con 0x7f8c781004e0 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.105+0000 7f8c7d7d1700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7819a8a0 con 0x7f8c78100e80 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.105+0000 7f8c76ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c781004e0 0x7f8c78195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.105+0000 7f8c76ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c781004e0 0x7f8c78195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49330/0 (socket says 172.21.15.88:49330) 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.105+0000 7f8c76ffd700 1 -- 172.21.15.88:0/3397828117 learned_addr learned my addr 172.21.15.88:0/3397828117 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.105+0000 7f8c767fc700 1 --2- 172.21.15.88:0/3397828117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c78100e80 0x7f8c78196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.106+0000 7f8c76ffd700 1 -- 172.21.15.88:0/3397828117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c78100e80 msgr2=0x7f8c78196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.106+0000 7f8c76ffd700 1 --2- 172.21.15.88:0/3397828117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c78100e80 0x7f8c78196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.106+0000 7f8c76ffd700 1 -- 172.21.15.88:0/3397828117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c680095f0 con 0x7f8c781004e0 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.106+0000 7f8c76ffd700 1 --2- 172.21.15.88:0/3397828117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c781004e0 0x7f8c78195ef0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8c68006010 tx=0x7f8c6800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.106+0000 7f8c5ffff700 1 -- 172.21.15.88:0/3397828117 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c6801d070 con 0x7f8c781004e0 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.106+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/3397828117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7819ab20 con 0x7f8c781004e0 2024-02-19T19:41:01.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.106+0000 7f8c5ffff700 1 -- 172.21.15.88:0/3397828117 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c68005dc0 con 0x7f8c781004e0 2024-02-19T19:41:01.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.106+0000 7f8c5ffff700 1 -- 172.21.15.88:0/3397828117 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c68017400 con 0x7f8c781004e0 2024-02-19T19:41:01.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.106+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/3397828117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7819b010 con 0x7f8c781004e0 2024-02-19T19:41:01.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.107+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/3397828117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c781087f0 con 0x7f8c781004e0 2024-02-19T19:41:01.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.107+0000 7f8c5ffff700 1 -- 172.21.15.88:0/3397828117 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8c68017560 con 0x7f8c781004e0 2024-02-19T19:41:01.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.108+0000 7f8c5ffff700 1 --2- 172.21.15.88:0/3397828117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8c6006d8a0 0x7f8c6006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:01.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.108+0000 7f8c5ffff700 1 -- 172.21.15.88:0/3397828117 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8c6808e9d0 con 0x7f8c781004e0 2024-02-19T19:41:01.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.108+0000 7f8c767fc700 1 --2- 172.21.15.88:0/3397828117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8c6006d8a0 0x7f8c6006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:01.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.108+0000 7f8c767fc700 1 --2- 172.21.15.88:0/3397828117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8c6006d8a0 0x7f8c6006fd50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8c78197510 tx=0x7f8c6c00b410 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:01.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.113+0000 7f8c5ffff700 1 -- 172.21.15.88:0/3397828117 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8c68058180 con 0x7f8c781004e0 2024-02-19T19:41:01.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.320+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/3397828117 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c7819b2f0 con 0x7f8c6006d8a0 2024-02-19T19:41:01.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.330+0000 7f8c5ffff700 1 -- 172.21.15.88:0/3397828117 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f8c7819b2f0 con 0x7f8c6006d8a0 2024-02-19T19:41:01.331 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:41:01.331 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (38m) 14m ago 48m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:41:01.332 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (49m) 14m ago 49m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:41:01.332 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (31m) 14m ago 47m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:41:01.332 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (34m) 14m ago 49m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:41:01.332 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (34m) 14m ago 47m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:41:01.332 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (37m) 14m ago 48m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:41:01.332 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (30m) 14m ago 42m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:41:01.332 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (30m) 14m ago 42m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:41:01.332 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (29m) 14m ago 42m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:41:01.332 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (14m) 14m ago 42m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:41:01.332 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (39m) 14m ago 50m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (39m) 14m ago 46m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (36m) 14m ago 50m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (35m) 14m ago 46m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (38m) 14m ago 48m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (38m) 14m ago 46m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (33m) 14m ago 46m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (33m) 14m ago 45m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (32m) 14m ago 45m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (31m) 14m ago 44m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (31m) 14m ago 44m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (30m) 14m ago 43m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:41:01.333 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (38m) 14m ago 47m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:41:01.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.333+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/3397828117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8c6006d8a0 msgr2=0x7f8c6006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.333+0000 7f8c7d7d1700 1 --2- 172.21.15.88:0/3397828117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8c6006d8a0 0x7f8c6006fd50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f8c78197510 tx=0x7f8c6c00b410 comp rx=0 tx=0).stop 2024-02-19T19:41:01.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.333+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/3397828117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c781004e0 msgr2=0x7f8c78195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.333+0000 7f8c7d7d1700 1 --2- 172.21.15.88:0/3397828117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c781004e0 0x7f8c78195ef0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8c68006010 tx=0x7f8c6800bd70 comp rx=0 tx=0).stop 2024-02-19T19:41:01.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.333+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/3397828117 shutdown_connections 2024-02-19T19:41:01.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.333+0000 7f8c7d7d1700 1 --2- 172.21.15.88:0/3397828117 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f8c6006d8a0 0x7f8c6006fd50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.333+0000 7f8c7d7d1700 1 --2- 172.21.15.88:0/3397828117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c781004e0 0x7f8c78195ef0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.333+0000 7f8c7d7d1700 1 --2- 172.21.15.88:0/3397828117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f8c78100e80 0x7f8c78196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.333+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/3397828117 >> 172.21.15.88:0/3397828117 conn(0x7f8c780fbda0 msgr2=0x7f8c780fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:01.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.333+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/3397828117 shutdown_connections 2024-02-19T19:41:01.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.333+0000 7f8c7d7d1700 1 -- 172.21.15.88:0/3397828117 wait complete. 2024-02-19T19:41:01.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.436+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/337160140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e00ffbd0 msgr2=0x7fa1e0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.436+0000 7fa1e7c5a700 1 --2- 172.21.15.88:0/337160140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e00ffbd0 0x7fa1e0100040 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4009910 tx=0x7fa1d4009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:01.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.436+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/337160140 shutdown_connections 2024-02-19T19:41:01.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.436+0000 7fa1e7c5a700 1 --2- 172.21.15.88:0/337160140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e00ffbd0 0x7fa1e0100040 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.436+0000 7fa1e7c5a700 1 --2- 172.21.15.88:0/337160140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e0104130 0x7fa1e0104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.436+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/337160140 >> 172.21.15.88:0/337160140 conn(0x7fa1e0075130 msgr2=0x7fa1e0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:01.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.436+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/337160140 shutdown_connections 2024-02-19T19:41:01.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.436+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/337160140 wait complete. 2024-02-19T19:41:01.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e7c5a700 1 Processor -- start 2024-02-19T19:41:01.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e7c5a700 1 -- start start 2024-02-19T19:41:01.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e7c5a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e00ffbd0 0x7fa1e0195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:01.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e7c5a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e0104130 0x7fa1e0196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:01.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e7c5a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e0196b50 con 0x7fa1e0104130 2024-02-19T19:41:01.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e7c5a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e019a8e0 con 0x7fa1e00ffbd0 2024-02-19T19:41:01.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e51f5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e0104130 0x7fa1e0196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e51f5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e0104130 0x7fa1e0196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49344/0 (socket says 172.21.15.88:49344) 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e51f5700 1 -- 172.21.15.88:0/1417273257 learned_addr learned my addr 172.21.15.88:0/1417273257 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e59f6700 1 --2- 172.21.15.88:0/1417273257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e00ffbd0 0x7fa1e0195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e51f5700 1 -- 172.21.15.88:0/1417273257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e00ffbd0 msgr2=0x7fa1e0195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e51f5700 1 --2- 172.21.15.88:0/1417273257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e00ffbd0 0x7fa1e0195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.437+0000 7fa1e51f5700 1 -- 172.21.15.88:0/1417273257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d40095f0 con 0x7fa1e0104130 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.438+0000 7fa1e51f5700 1 --2- 172.21.15.88:0/1417273257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e0104130 0x7fa1e0196470 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4006010 tx=0x7fa1d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.438+0000 7fa1deffd700 1 -- 172.21.15.88:0/1417273257 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1d401d070 con 0x7fa1e0104130 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.438+0000 7fa1deffd700 1 -- 172.21.15.88:0/1417273257 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1d400f460 con 0x7fa1e0104130 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.438+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/1417273257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e019ab60 con 0x7fa1e0104130 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.438+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/1417273257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e019b050 con 0x7fa1e0104130 2024-02-19T19:41:01.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.438+0000 7fa1deffd700 1 -- 172.21.15.88:0/1417273257 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1d4017630 con 0x7fa1e0104130 2024-02-19T19:41:01.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.439+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/1417273257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e01086a0 con 0x7fa1e0104130 2024-02-19T19:41:01.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.439+0000 7fa1deffd700 1 -- 172.21.15.88:0/1417273257 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa1d400f5d0 con 0x7fa1e0104130 2024-02-19T19:41:01.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.440+0000 7fa1deffd700 1 --2- 172.21.15.88:0/1417273257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1c806d8a0 0x7fa1c806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:01.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.440+0000 7fa1deffd700 1 -- 172.21.15.88:0/1417273257 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa1d408e990 con 0x7fa1e0104130 2024-02-19T19:41:01.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.440+0000 7fa1e59f6700 1 --2- 172.21.15.88:0/1417273257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1c806d8a0 0x7fa1c806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:01.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.440+0000 7fa1e59f6700 1 --2- 172.21.15.88:0/1417273257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1c806d8a0 0x7fa1c806fd50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800a400 tx=0x7fa1d800d040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:01.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.445+0000 7fa1deffd700 1 -- 172.21.15.88:0/1417273257 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa1d4058280 con 0x7fa1e0104130 2024-02-19T19:41:01.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:00 smithi132 ceph-mon[70676]: from='client.36014 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:01.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.725+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/1417273257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa1e0197330 con 0x7fa1e0104130 2024-02-19T19:41:01.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.725+0000 7fa1deffd700 1 -- 172.21.15.88:0/1417273257 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa1d4020370 con 0x7fa1e0104130 2024-02-19T19:41:01.726 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:41:01.726 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:41:01.726 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:41:01.726 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:41:01.726 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:41:01.726 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:41:01.727 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:41:01.727 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:41:01.727 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:41:01.727 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:41:01.727 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:41:01.727 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:41:01.727 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:41:01.727 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:41:01.727 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:41:01.727 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:41:01.727 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:41:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.728+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/1417273257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1c806d8a0 msgr2=0x7fa1c806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.728+0000 7fa1e7c5a700 1 --2- 172.21.15.88:0/1417273257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1c806d8a0 0x7fa1c806fd50 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800a400 tx=0x7fa1d800d040 comp rx=0 tx=0).stop 2024-02-19T19:41:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.728+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/1417273257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e0104130 msgr2=0x7fa1e0196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.728+0000 7fa1e7c5a700 1 --2- 172.21.15.88:0/1417273257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e0104130 0x7fa1e0196470 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4006010 tx=0x7fa1d400bd70 comp rx=0 tx=0).stop 2024-02-19T19:41:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.728+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/1417273257 shutdown_connections 2024-02-19T19:41:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.728+0000 7fa1e7c5a700 1 --2- 172.21.15.88:0/1417273257 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa1c806d8a0 0x7fa1c806fd50 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.728+0000 7fa1e7c5a700 1 --2- 172.21.15.88:0/1417273257 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa1e00ffbd0 0x7fa1e0195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.728+0000 7fa1e7c5a700 1 --2- 172.21.15.88:0/1417273257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e0104130 0x7fa1e0196470 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.728+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/1417273257 >> 172.21.15.88:0/1417273257 conn(0x7fa1e0075130 msgr2=0x7fa1e0068820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.728+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/1417273257 shutdown_connections 2024-02-19T19:41:01.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.728+0000 7fa1e7c5a700 1 -- 172.21.15.88:0/1417273257 wait complete. 2024-02-19T19:41:01.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.830+0000 7f6edd158700 1 -- 172.21.15.88:0/3195643117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ed81004e0 msgr2=0x7f6ed81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.830+0000 7f6edd158700 1 --2- 172.21.15.88:0/3195643117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ed81004e0 0x7f6ed81008b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8009910 tx=0x7f6ec8009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:01.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.830+0000 7f6edd158700 1 -- 172.21.15.88:0/3195643117 shutdown_connections 2024-02-19T19:41:01.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.830+0000 7f6edd158700 1 --2- 172.21.15.88:0/3195643117 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ed8100e80 0x7f6ed8104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.830+0000 7f6edd158700 1 --2- 172.21.15.88:0/3195643117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ed81004e0 0x7f6ed81008b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.830+0000 7f6edd158700 1 -- 172.21.15.88:0/3195643117 >> 172.21.15.88:0/3195643117 conn(0x7f6ed80fbda0 msgr2=0x7f6ed80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.830+0000 7f6edd158700 1 -- 172.21.15.88:0/3195643117 shutdown_connections 2024-02-19T19:41:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.830+0000 7f6edd158700 1 -- 172.21.15.88:0/3195643117 wait complete. 2024-02-19T19:41:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.831+0000 7f6edd158700 1 Processor -- start 2024-02-19T19:41:01.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.831+0000 7f6edd158700 1 -- start start 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6edd158700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ed81004e0 0x7f6ed8195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6edd158700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ed8100e80 0x7f6ed8196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6edd158700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed8196b10 con 0x7f6ed8100e80 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6edd158700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed819a8a0 con 0x7f6ed81004e0 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6ed659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ed8100e80 0x7f6ed8196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6ed659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ed8100e80 0x7f6ed8196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49364/0 (socket says 172.21.15.88:49364) 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6ed659c700 1 -- 172.21.15.88:0/3189167935 learned_addr learned my addr 172.21.15.88:0/3189167935 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6ed6d9d700 1 --2- 172.21.15.88:0/3189167935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ed81004e0 0x7f6ed8195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6ed659c700 1 -- 172.21.15.88:0/3189167935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ed81004e0 msgr2=0x7f6ed8195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6ed659c700 1 --2- 172.21.15.88:0/3189167935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ed81004e0 0x7f6ed8195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6ed659c700 1 -- 172.21.15.88:0/3189167935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec80095f0 con 0x7f6ed8100e80 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6ed659c700 1 --2- 172.21.15.88:0/3189167935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ed8100e80 0x7f6ed8196430 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00c8f0 tx=0x7f6ecc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6ebffff700 1 -- 172.21.15.88:0/3189167935 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ecc0078a0 con 0x7f6ed8100e80 2024-02-19T19:41:01.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6edd158700 1 -- 172.21.15.88:0/3189167935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed819ab80 con 0x7f6ed8100e80 2024-02-19T19:41:01.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6ebffff700 1 -- 172.21.15.88:0/3189167935 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ecc00f450 con 0x7f6ed8100e80 2024-02-19T19:41:01.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6ebffff700 1 -- 172.21.15.88:0/3189167935 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ecc018610 con 0x7f6ed8100e80 2024-02-19T19:41:01.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.832+0000 7f6edd158700 1 -- 172.21.15.88:0/3189167935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed819b0d0 con 0x7f6ed8100e80 2024-02-19T19:41:01.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.833+0000 7f6edd158700 1 -- 172.21.15.88:0/3189167935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed81087f0 con 0x7f6ed8100e80 2024-02-19T19:41:01.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.834+0000 7f6ebffff700 1 -- 172.21.15.88:0/3189167935 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6ecc00f5c0 con 0x7f6ed8100e80 2024-02-19T19:41:01.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.834+0000 7f6ebffff700 1 --2- 172.21.15.88:0/3189167935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ec006d8a0 0x7f6ec006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:01.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.834+0000 7f6ebffff700 1 -- 172.21.15.88:0/3189167935 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6ecc08d400 con 0x7f6ed8100e80 2024-02-19T19:41:01.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.834+0000 7f6ed6d9d700 1 --2- 172.21.15.88:0/3189167935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ec006d8a0 0x7f6ec006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:01.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.835+0000 7f6ed6d9d700 1 --2- 172.21.15.88:0/3189167935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ec006d8a0 0x7f6ec006fd50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006010 tx=0x7f6ec801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:01.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:01.839+0000 7f6ebffff700 1 -- 172.21.15.88:0/3189167935 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6ecc056c30 con 0x7f6ed8100e80 2024-02-19T19:41:02.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.084+0000 7f6edd158700 1 -- 172.21.15.88:0/3189167935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6ed81972f0 con 0x7f6ed8100e80 2024-02-19T19:41:02.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.084+0000 7f6ebffff700 1 -- 172.21.15.88:0/3189167935 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f6ecc05a2f0 con 0x7f6ed8100e80 2024-02-19T19:41:02.085 INFO:teuthology.orchestra.run.smithi088.stdout:e32 2024-02-19T19:41:02.085 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:41:02.085 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:41:02.085 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:41:02.085 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:41:02.085 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:41:02.085 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:41:02.085 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 23 2024-02-19T19:41:02.085 INFO:teuthology.orchestra.run.smithi088.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-02-19T19:41:02.085 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:10:28.265493+0000 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:up {} 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:failed 0,1 2024-02-19T19:41:02.086 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{-1:34430} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:41:02.087 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{-1:34462} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:41:02.088 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:41:02.088 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.086+0000 7f6edd158700 1 -- 172.21.15.88:0/3189167935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ec006d8a0 msgr2=0x7f6ec006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.086+0000 7f6edd158700 1 --2- 172.21.15.88:0/3189167935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ec006d8a0 0x7f6ec006fd50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8006010 tx=0x7f6ec801a040 comp rx=0 tx=0).stop 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.086+0000 7f6edd158700 1 -- 172.21.15.88:0/3189167935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ed8100e80 msgr2=0x7f6ed8196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.086+0000 7f6edd158700 1 --2- 172.21.15.88:0/3189167935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ed8100e80 0x7f6ed8196430 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00c8f0 tx=0x7f6ecc00cc00 comp rx=0 tx=0).stop 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.086+0000 7f6edd158700 1 -- 172.21.15.88:0/3189167935 shutdown_connections 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.086+0000 7f6edd158700 1 --2- 172.21.15.88:0/3189167935 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6ec006d8a0 0x7f6ec006fd50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.086+0000 7f6edd158700 1 --2- 172.21.15.88:0/3189167935 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6ed81004e0 0x7f6ed8195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.086+0000 7f6edd158700 1 --2- 172.21.15.88:0/3189167935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ed8100e80 0x7f6ed8196430 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.086+0000 7f6edd158700 1 -- 172.21.15.88:0/3189167935 >> 172.21.15.88:0/3189167935 conn(0x7f6ed80fbda0 msgr2=0x7f6ed80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.086+0000 7f6edd158700 1 -- 172.21.15.88:0/3189167935 shutdown_connections 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.087+0000 7f6edd158700 1 -- 172.21.15.88:0/3189167935 wait complete. 2024-02-19T19:41:02.089 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:41:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.191+0000 7fce654ff700 1 -- 172.21.15.88:0/2826028107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce600ff350 msgr2=0x7fce600ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.191+0000 7fce654ff700 1 --2- 172.21.15.88:0/2826028107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce600ff350 0x7fce600ff720 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fce50009910 tx=0x7fce50009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.191+0000 7fce654ff700 1 -- 172.21.15.88:0/2826028107 shutdown_connections 2024-02-19T19:41:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.191+0000 7fce654ff700 1 --2- 172.21.15.88:0/2826028107 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce600ffc60 0x7fce6010a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.191+0000 7fce654ff700 1 --2- 172.21.15.88:0/2826028107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce600ff350 0x7fce600ff720 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.191+0000 7fce654ff700 1 -- 172.21.15.88:0/2826028107 >> 172.21.15.88:0/2826028107 conn(0x7fce600748b0 msgr2=0x7fce60074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.191+0000 7fce654ff700 1 -- 172.21.15.88:0/2826028107 shutdown_connections 2024-02-19T19:41:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.191+0000 7fce654ff700 1 -- 172.21.15.88:0/2826028107 wait complete. 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.192+0000 7fce654ff700 1 Processor -- start 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.192+0000 7fce654ff700 1 -- start start 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.192+0000 7fce654ff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce600ff350 0x7fce60100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.192+0000 7fce654ff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce600ffc60 0x7fce60100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.192+0000 7fce654ff700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce60100fb0 con 0x7fce600ffc60 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.192+0000 7fce654ff700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601010f0 con 0x7fce600ff350 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce5e7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce600ffc60 0x7fce60100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce5e7fc700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce600ffc60 0x7fce60100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49374/0 (socket says 172.21.15.88:49374) 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce5e7fc700 1 -- 172.21.15.88:0/3840394196 learned_addr learned my addr 172.21.15.88:0/3840394196 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce5effd700 1 --2- 172.21.15.88:0/3840394196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce600ff350 0x7fce60100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce5e7fc700 1 -- 172.21.15.88:0/3840394196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce600ff350 msgr2=0x7fce60100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce5e7fc700 1 --2- 172.21.15.88:0/3840394196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce600ff350 0x7fce60100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce5e7fc700 1 -- 172.21.15.88:0/3840394196 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce500095f0 con 0x7fce600ffc60 2024-02-19T19:41:02.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce5e7fc700 1 --2- 172.21.15.88:0/3840394196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce600ffc60 0x7fce60100a70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c930 tx=0x7fce5400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:02.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce47fff700 1 -- 172.21.15.88:0/3840394196 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce54007ad0 con 0x7fce600ffc60 2024-02-19T19:41:02.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce654ff700 1 -- 172.21.15.88:0/3840394196 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce601a2070 con 0x7fce600ffc60 2024-02-19T19:41:02.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce47fff700 1 -- 172.21.15.88:0/3840394196 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce5400ce80 con 0x7fce600ffc60 2024-02-19T19:41:02.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce47fff700 1 -- 172.21.15.88:0/3840394196 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce54018630 con 0x7fce600ffc60 2024-02-19T19:41:02.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.193+0000 7fce654ff700 1 -- 172.21.15.88:0/3840394196 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce601a23d0 con 0x7fce600ffc60 2024-02-19T19:41:02.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.194+0000 7fce654ff700 1 -- 172.21.15.88:0/3840394196 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce60107dd0 con 0x7fce600ffc60 2024-02-19T19:41:02.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.195+0000 7fce47fff700 1 -- 172.21.15.88:0/3840394196 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fce5400f450 con 0x7fce600ffc60 2024-02-19T19:41:02.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.195+0000 7fce47fff700 1 --2- 172.21.15.88:0/3840394196 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fce4806d8a0 0x7fce4806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:02.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.195+0000 7fce47fff700 1 -- 172.21.15.88:0/3840394196 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fce5408d520 con 0x7fce600ffc60 2024-02-19T19:41:02.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.195+0000 7fce5effd700 1 --2- 172.21.15.88:0/3840394196 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fce4806d8a0 0x7fce4806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:02.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.196+0000 7fce5effd700 1 --2- 172.21.15.88:0/3840394196 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fce4806d8a0 0x7fce4806fd50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fce50006010 tx=0x7fce5001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:02.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.200+0000 7fce47fff700 1 -- 172.21.15.88:0/3840394196 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fce54056cd0 con 0x7fce600ffc60 2024-02-19T19:41:02.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:01 smithi088 ceph-mon[88643]: pgmap v1124: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:02.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:01 smithi088 ceph-mon[88643]: from='client.36018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:02.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:01 smithi088 ceph-mon[88643]: from='client.36022 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:02.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:01 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1417273257' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.423+0000 7fce654ff700 1 -- 172.21.15.88:0/3840394196 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce6002cca0 con 0x7fce4806d8a0 2024-02-19T19:41:02.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.424+0000 7fce47fff700 1 -- 172.21.15.88:0/3840394196 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fce6002cca0 con 0x7fce4806d8a0 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:41:02.425 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:41:02.426 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading mds daemons", 2024-02-19T19:41:02.426 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:41:02.426 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:41:02.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.426+0000 7fce654ff700 1 -- 172.21.15.88:0/3840394196 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fce4806d8a0 msgr2=0x7fce4806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:02.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.426+0000 7fce654ff700 1 --2- 172.21.15.88:0/3840394196 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fce4806d8a0 0x7fce4806fd50 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fce50006010 tx=0x7fce5001a040 comp rx=0 tx=0).stop 2024-02-19T19:41:02.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.426+0000 7fce654ff700 1 -- 172.21.15.88:0/3840394196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce600ffc60 msgr2=0x7fce60100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:02.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.426+0000 7fce654ff700 1 --2- 172.21.15.88:0/3840394196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce600ffc60 0x7fce60100a70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fce5400c930 tx=0x7fce5400cc40 comp rx=0 tx=0).stop 2024-02-19T19:41:02.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.426+0000 7fce654ff700 1 -- 172.21.15.88:0/3840394196 shutdown_connections 2024-02-19T19:41:02.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.426+0000 7fce654ff700 1 --2- 172.21.15.88:0/3840394196 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fce4806d8a0 0x7fce4806fd50 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.426+0000 7fce654ff700 1 --2- 172.21.15.88:0/3840394196 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fce600ff350 0x7fce60100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.426+0000 7fce654ff700 1 --2- 172.21.15.88:0/3840394196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce600ffc60 0x7fce60100a70 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.426+0000 7fce654ff700 1 -- 172.21.15.88:0/3840394196 >> 172.21.15.88:0/3840394196 conn(0x7fce600748b0 msgr2=0x7fce60104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:02.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.426+0000 7fce654ff700 1 -- 172.21.15.88:0/3840394196 shutdown_connections 2024-02-19T19:41:02.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.426+0000 7fce654ff700 1 -- 172.21.15.88:0/3840394196 wait complete. 2024-02-19T19:41:02.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:01 smithi132 ceph-mon[70676]: pgmap v1124: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:02.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:01 smithi132 ceph-mon[70676]: from='client.36018 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:02.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:01 smithi132 ceph-mon[70676]: from='client.36022 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:02.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:01 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1417273257' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.530+0000 7f163a09c700 1 -- 172.21.15.88:0/909364140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16340ff3c0 msgr2=0x7f16340ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:02.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.530+0000 7f163a09c700 1 --2- 172.21.15.88:0/909364140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16340ff3c0 0x7f16340ff790 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f1624009910 tx=0x7f1624009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.530+0000 7f163a09c700 1 -- 172.21.15.88:0/909364140 shutdown_connections 2024-02-19T19:41:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.530+0000 7f163a09c700 1 --2- 172.21.15.88:0/909364140 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16340ffd60 0x7f1634103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.530+0000 7f163a09c700 1 --2- 172.21.15.88:0/909364140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16340ff3c0 0x7f16340ff790 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.530+0000 7f163a09c700 1 -- 172.21.15.88:0/909364140 >> 172.21.15.88:0/909364140 conn(0x7f16340fac80 msgr2=0x7f16340fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.530+0000 7f163a09c700 1 -- 172.21.15.88:0/909364140 shutdown_connections 2024-02-19T19:41:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.530+0000 7f163a09c700 1 -- 172.21.15.88:0/909364140 wait complete. 2024-02-19T19:41:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.531+0000 7f163a09c700 1 Processor -- start 2024-02-19T19:41:02.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.531+0000 7f163a09c700 1 -- start start 2024-02-19T19:41:02.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.531+0000 7f163a09c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16340ff3c0 0x7f1634196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.531+0000 7f163a09c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16340ffd60 0x7f1634197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.531+0000 7f163a09c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1634197850 con 0x7f16340ff3c0 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.531+0000 7f163a09c700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1634197990 con 0x7f16340ffd60 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.531+0000 7f16337fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16340ff3c0 0x7f1634196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f16337fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16340ff3c0 0x7f1634196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49390/0 (socket says 172.21.15.88:49390) 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f16337fe700 1 -- 172.21.15.88:0/374607294 learned_addr learned my addr 172.21.15.88:0/374607294 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f1632ffd700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16340ffd60 0x7f1634197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f16337fe700 1 -- 172.21.15.88:0/374607294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16340ffd60 msgr2=0x7f1634197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f16337fe700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16340ffd60 0x7f1634197230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f16337fe700 1 -- 172.21.15.88:0/374607294 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16240095f0 con 0x7f16340ff3c0 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f1632ffd700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16340ffd60 0x7f1634197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f16337fe700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16340ff3c0 0x7f1634196cf0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1624006010 tx=0x7f162400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:02.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f1630ff9700 1 -- 172.21.15.88:0/374607294 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f162401d070 con 0x7f16340ff3c0 2024-02-19T19:41:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f1630ff9700 1 -- 172.21.15.88:0/374607294 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1624004fb0 con 0x7f16340ff3c0 2024-02-19T19:41:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f1630ff9700 1 -- 172.21.15.88:0/374607294 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1624022ea0 con 0x7f16340ff3c0 2024-02-19T19:41:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f163a09c700 1 -- 172.21.15.88:0/374607294 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163419fd60 con 0x7f16340ff3c0 2024-02-19T19:41:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.532+0000 7f163a09c700 1 -- 172.21.15.88:0/374607294 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16341a0170 con 0x7f16340ff3c0 2024-02-19T19:41:02.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.533+0000 7f163a09c700 1 -- 172.21.15.88:0/374607294 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16341076d0 con 0x7f16340ff3c0 2024-02-19T19:41:02.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.534+0000 7f1630ff9700 1 -- 172.21.15.88:0/374607294 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1624022470 con 0x7f16340ff3c0 2024-02-19T19:41:02.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.534+0000 7f1630ff9700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f161c06d8a0 0x7f161c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:02.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.534+0000 7f1630ff9700 1 -- 172.21.15.88:0/374607294 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f162408ea10 con 0x7f16340ff3c0 2024-02-19T19:41:02.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.534+0000 7f1632ffd700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f161c06d8a0 0x7f161c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:02.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.534+0000 7f1632ffd700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f161c06d8a0 0x7f161c06fd50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1634069680 tx=0x7f1628005f50 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:02.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.539+0000 7f1630ff9700 1 -- 172.21.15.88:0/374607294 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16240582f0 con 0x7f16340ff3c0 2024-02-19T19:41:02.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.826+0000 7f163a09c700 1 -- 172.21.15.88:0/374607294 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f16341070b0 con 0x7f16340ff3c0 2024-02-19T19:41:02.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.826+0000 7f1630ff9700 1 -- 172.21.15.88:0/374607294 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4303 (secure 0 0 0) 0x7f1624027020 con 0x7f16340ff3c0 2024-02-19T19:41:02.827 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem is offline 2024-02-19T19:41:02.827 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:41:02.827 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:41:02.827 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:41:02.827 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:41:02.827 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:41:02.827 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:41:02.827 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:41:02.827 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:41:02.827 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp7onlu_vh:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-02-19T19:41:02.828 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is offline because no MDS is active for it. 2024-02-19T19:41:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.829+0000 7f163a09c700 1 -- 172.21.15.88:0/374607294 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f161c06d8a0 msgr2=0x7f161c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.829+0000 7f163a09c700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f161c06d8a0 0x7f161c06fd50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1634069680 tx=0x7f1628005f50 comp rx=0 tx=0).stop 2024-02-19T19:41:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.829+0000 7f163a09c700 1 -- 172.21.15.88:0/374607294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16340ff3c0 msgr2=0x7f1634196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.829+0000 7f163a09c700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16340ff3c0 0x7f1634196cf0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1624006010 tx=0x7f162400be20 comp rx=0 tx=0).stop 2024-02-19T19:41:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.829+0000 7f163a09c700 1 -- 172.21.15.88:0/374607294 shutdown_connections 2024-02-19T19:41:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.829+0000 7f163a09c700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f161c06d8a0 0x7f161c06fd50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.829+0000 7f163a09c700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f16340ff3c0 0x7f1634196cf0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.829+0000 7f163a09c700 1 --2- 172.21.15.88:0/374607294 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f16340ffd60 0x7f1634197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.829+0000 7f163a09c700 1 -- 172.21.15.88:0/374607294 >> 172.21.15.88:0/374607294 conn(0x7f16340fac80 msgr2=0x7f16340fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:02.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.829+0000 7f163a09c700 1 -- 172.21.15.88:0/374607294 shutdown_connections 2024-02-19T19:41:02.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:02.829+0000 7f163a09c700 1 -- 172.21.15.88:0/374607294 wait complete. 2024-02-19T19:41:03.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:02 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3189167935' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:41:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:02 smithi088 ceph-mon[88643]: from='client.36034 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:02 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/374607294' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:41:03.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:02 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3189167935' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:41:03.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:02 smithi132 ceph-mon[70676]: from='client.36034 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:03.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:02 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/374607294' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:41:04.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:03 smithi088 ceph-mon[88643]: pgmap v1125: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:04.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:03 smithi132 ceph-mon[70676]: pgmap v1125: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:06.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:06 smithi088 ceph-mon[88643]: pgmap v1126: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:06.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:06 smithi132 ceph-mon[70676]: pgmap v1126: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:08.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:08 smithi088 ceph-mon[88643]: pgmap v1127: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:08.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:08 smithi132 ceph-mon[70676]: pgmap v1127: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:10.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:10 smithi088 ceph-mon[88643]: pgmap v1128: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:10.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:10 smithi132 ceph-mon[70676]: pgmap v1128: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:12.303 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:12 smithi088 ceph-mon[88643]: pgmap v1129: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:12.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:12 smithi132 ceph-mon[70676]: pgmap v1129: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:14 smithi088 ceph-mon[88643]: pgmap v1130: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:41:14.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:14 smithi132 ceph-mon[70676]: pgmap v1130: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:14.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:41:16.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:16 smithi088 ceph-mon[88643]: pgmap v1131: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:16.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:16 smithi132 ceph-mon[70676]: pgmap v1131: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:18.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:18 smithi088 ceph-mon[88643]: pgmap v1132: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:18.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:18 smithi132 ceph-mon[70676]: pgmap v1132: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:20.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:20 smithi088 ceph-mon[88643]: pgmap v1133: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:20.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:20 smithi132 ceph-mon[70676]: pgmap v1133: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:22.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:22 smithi088 ceph-mon[88643]: pgmap v1134: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:22.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:22 smithi132 ceph-mon[70676]: pgmap v1134: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:24.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:24 smithi088 ceph-mon[88643]: pgmap v1135: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:24.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:24 smithi132 ceph-mon[70676]: pgmap v1135: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:26.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: pgmap v1136: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:26.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:26.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:41:26.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: pgmap v1137: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:26.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: pgmap v1138: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:26.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:26.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.ajgvpz"}]': finished 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.psjiah"}]': finished 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi132.afhgdi"}]': finished 2024-02-19T19:41:26.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:41:26.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi132.uyjxaz"}]': finished 2024-02-19T19:41:26.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:26 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-19T19:41:26.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: pgmap v1136: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:26.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:26.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:41:26.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: pgmap v1137: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:26.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: pgmap v1138: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.ajgvpz"}]: dispatch 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.ajgvpz"}]': finished 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.psjiah"}]: dispatch 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi088.psjiah"}]': finished 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi132.afhgdi"}]: dispatch 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi132.afhgdi"}]': finished 2024-02-19T19:41:26.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi132.uyjxaz"}]: dispatch 2024-02-19T19:41:26.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi132.uyjxaz"}]': finished 2024-02-19T19:41:26.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:26 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-19T19:41:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: Upgrade: Setting container_image for all mds 2024-02-19T19:41:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: Upgrade: Setting filesystem cephfs Joinable 2024-02-19T19:41:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-19T19:41:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: Standby daemon mds.cephfs.smithi088.ajgvpz assigned to filesystem cephfs as rank 0 2024-02-19T19:41:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: Standby daemon mds.cephfs.smithi088.psjiah assigned to filesystem cephfs as rank 1 2024-02-19T19:41:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-19T19:41:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:41:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: fsmap cephfs:2/2 {0=cephfs.smithi088.ajgvpz=up:replay,1=cephfs.smithi088.psjiah=up:replay} 2 up:standby 2024-02-19T19:41:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:27.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:27.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:27.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:27 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:27.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: Upgrade: Setting container_image for all mds 2024-02-19T19:41:27.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: Upgrade: Setting filesystem cephfs Joinable 2024-02-19T19:41:27.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-19T19:41:27.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: Standby daemon mds.cephfs.smithi088.ajgvpz assigned to filesystem cephfs as rank 0 2024-02-19T19:41:27.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: Standby daemon mds.cephfs.smithi088.psjiah assigned to filesystem cephfs as rank 1 2024-02-19T19:41:27.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-19T19:41:27.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-02-19T19:41:27.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:27.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: fsmap cephfs:2/2 {0=cephfs.smithi088.ajgvpz=up:replay,1=cephfs.smithi088.psjiah=up:replay} 2 up:standby 2024-02-19T19:41:27.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:27.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:27.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:27.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:27 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:28.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:28 smithi088 ceph-mon[88643]: Upgrade: Setting container_image for all rgw 2024-02-19T19:41:28.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:28 smithi088 ceph-mon[88643]: Upgrade: Setting container_image for all rbd-mirror 2024-02-19T19:41:28.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:28 smithi088 ceph-mon[88643]: pgmap v1139: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:28.384 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:28 smithi132 ceph-mon[70676]: Upgrade: Setting container_image for all rgw 2024-02-19T19:41:28.384 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:28 smithi132 ceph-mon[70676]: Upgrade: Setting container_image for all rbd-mirror 2024-02-19T19:41:28.384 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:28 smithi132 ceph-mon[70676]: pgmap v1139: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:29.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:29.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:41:29.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:29.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:41:29.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:29.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:29.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:41:29.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:29.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi088", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:41:29.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:30.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:30 smithi088 ceph-mon[88643]: Upgrade: Updating ceph-exporter.smithi088 (1/2) 2024-02-19T19:41:30.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:30 smithi088 ceph-mon[88643]: Deploying daemon ceph-exporter.smithi088 on smithi088 2024-02-19T19:41:30.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:30 smithi088 ceph-mon[88643]: pgmap v1140: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s rd, 1 op/s 2024-02-19T19:41:30.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:30 smithi132 ceph-mon[70676]: Upgrade: Updating ceph-exporter.smithi088 (1/2) 2024-02-19T19:41:30.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:30 smithi132 ceph-mon[70676]: Deploying daemon ceph-exporter.smithi088 on smithi088 2024-02-19T19:41:30.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:30 smithi132 ceph-mon[70676]: pgmap v1140: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.6 MiB/s rd, 1 op/s 2024-02-19T19:41:32.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:32 smithi132 ceph-mon[70676]: pgmap v1141: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-02-19T19:41:32.996 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:32 smithi088 ceph-mon[88643]: pgmap v1141: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-02-19T19:41:32.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.994+0000 7fdf5918f700 1 -- 172.21.15.88:0/4091320024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf54069140 msgr2=0x7fdf54104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:32.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.994+0000 7fdf5918f700 1 --2- 172.21.15.88:0/4091320024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf54069140 0x7fdf54104eb0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fdf48009910 tx=0x7fdf48009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:32.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.994+0000 7fdf5918f700 1 -- 172.21.15.88:0/4091320024 shutdown_connections 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.994+0000 7fdf5918f700 1 --2- 172.21.15.88:0/4091320024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf54069140 0x7fdf54104eb0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.994+0000 7fdf5918f700 1 --2- 172.21.15.88:0/4091320024 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf540687a0 0x7fdf54068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.994+0000 7fdf5918f700 1 -- 172.21.15.88:0/4091320024 >> 172.21.15.88:0/4091320024 conn(0x7fdf54075130 msgr2=0x7fdf54075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.994+0000 7fdf5918f700 1 -- 172.21.15.88:0/4091320024 shutdown_connections 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.994+0000 7fdf5918f700 1 -- 172.21.15.88:0/4091320024 wait complete. 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.995+0000 7fdf5918f700 1 Processor -- start 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.995+0000 7fdf5918f700 1 -- start start 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.995+0000 7fdf5918f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf540687a0 0x7fdf54195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.995+0000 7fdf5918f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf54069140 0x7fdf541964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.995+0000 7fdf5918f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54196ba0 con 0x7fdf540687a0 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.995+0000 7fdf5918f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5419a930 con 0x7fdf54069140 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.996+0000 7fdf5259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf54069140 0x7fdf541964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:32.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.996+0000 7fdf5259c700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf54069140 0x7fdf541964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:37114/0 (socket says 172.21.15.88:37114) 2024-02-19T19:41:32.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.996+0000 7fdf5259c700 1 -- 172.21.15.88:0/1392157439 learned_addr learned my addr 172.21.15.88:0/1392157439 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:32.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.996+0000 7fdf5259c700 1 -- 172.21.15.88:0/1392157439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf540687a0 msgr2=0x7fdf54195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:32.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.996+0000 7fdf5259c700 1 --2- 172.21.15.88:0/1392157439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf540687a0 0x7fdf54195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:32.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.996+0000 7fdf5259c700 1 -- 172.21.15.88:0/1392157439 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480095f0 con 0x7fdf54069140 2024-02-19T19:41:32.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.997+0000 7fdf5259c700 1 --2- 172.21.15.88:0/1392157439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf54069140 0x7fdf541964c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800bda0 tx=0x7fdf4800be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:33.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.997+0000 7fdf3bfff700 1 -- 172.21.15.88:0/1392157439 <== mon.1 v2:172.21.15.132:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf4801d070 con 0x7fdf54069140 2024-02-19T19:41:33.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.997+0000 7fdf5918f700 1 -- 172.21.15.88:0/1392157439 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5419abb0 con 0x7fdf54069140 2024-02-19T19:41:33.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.997+0000 7fdf5918f700 1 -- 172.21.15.88:0/1392157439 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5419b0a0 con 0x7fdf54069140 2024-02-19T19:41:33.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.997+0000 7fdf3bfff700 1 -- 172.21.15.88:0/1392157439 <== mon.1 v2:172.21.15.132:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf48004fb0 con 0x7fdf54069140 2024-02-19T19:41:33.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.998+0000 7fdf3bfff700 1 -- 172.21.15.88:0/1392157439 <== mon.1 v2:172.21.15.132:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf4800f480 con 0x7fdf54069140 2024-02-19T19:41:33.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.998+0000 7fdf5918f700 1 -- 172.21.15.88:0/1392157439 --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf40005320 con 0x7fdf54069140 2024-02-19T19:41:33.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:32.999+0000 7fdf3bfff700 1 -- 172.21.15.88:0/1392157439 <== mon.1 v2:172.21.15.132:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdf4800f700 con 0x7fdf54069140 2024-02-19T19:41:33.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.000+0000 7fdf3bfff700 1 --2- 172.21.15.88:0/1392157439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3c06d690 0x7fdf3c06fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:33.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.000+0000 7fdf3bfff700 1 -- 172.21.15.88:0/1392157439 <== mon.1 v2:172.21.15.132:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdf4808e960 con 0x7fdf54069140 2024-02-19T19:41:33.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.000+0000 7fdf52d9d700 1 --2- 172.21.15.88:0/1392157439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3c06d690 0x7fdf3c06fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:33.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.000+0000 7fdf52d9d700 1 --2- 172.21.15.88:0/1392157439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3c06d690 0x7fdf3c06fb40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdf440097b0 tx=0x7fdf44006d20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:33.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.005+0000 7fdf3bfff700 1 -- 172.21.15.88:0/1392157439 <== mon.1 v2:172.21.15.132:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf48058440 con 0x7fdf54069140 2024-02-19T19:41:33.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.227+0000 7fdf5918f700 1 -- 172.21.15.88:0/1392157439 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf40000bf0 con 0x7fdf3c06d690 2024-02-19T19:41:33.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.228+0000 7fdf3bfff700 1 -- 172.21.15.88:0/1392157439 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fdf40000bf0 con 0x7fdf3c06d690 2024-02-19T19:41:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.232+0000 7fdf5918f700 1 -- 172.21.15.88:0/1392157439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3c06d690 msgr2=0x7fdf3c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.232+0000 7fdf5918f700 1 --2- 172.21.15.88:0/1392157439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3c06d690 0x7fdf3c06fb40 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fdf440097b0 tx=0x7fdf44006d20 comp rx=0 tx=0).stop 2024-02-19T19:41:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.232+0000 7fdf5918f700 1 -- 172.21.15.88:0/1392157439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf54069140 msgr2=0x7fdf541964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.232+0000 7fdf5918f700 1 --2- 172.21.15.88:0/1392157439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf54069140 0x7fdf541964c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdf4800bda0 tx=0x7fdf4800be80 comp rx=0 tx=0).stop 2024-02-19T19:41:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.232+0000 7fdf5918f700 1 -- 172.21.15.88:0/1392157439 shutdown_connections 2024-02-19T19:41:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.232+0000 7fdf5918f700 1 --2- 172.21.15.88:0/1392157439 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdf3c06d690 0x7fdf3c06fb40 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.232+0000 7fdf5918f700 1 --2- 172.21.15.88:0/1392157439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf540687a0 0x7fdf54195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.232+0000 7fdf5918f700 1 --2- 172.21.15.88:0/1392157439 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdf54069140 0x7fdf541964c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.232+0000 7fdf5918f700 1 -- 172.21.15.88:0/1392157439 >> 172.21.15.88:0/1392157439 conn(0x7fdf54075130 msgr2=0x7fdf540fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:33.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.232+0000 7fdf5918f700 1 -- 172.21.15.88:0/1392157439 shutdown_connections 2024-02-19T19:41:33.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.232+0000 7fdf5918f700 1 -- 172.21.15.88:0/1392157439 wait complete. 2024-02-19T19:41:33.248 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:41:33.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.345+0000 7f5b82894700 1 -- 172.21.15.88:0/2568660061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b7c0ffbd0 msgr2=0x7f5b7c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:33.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.345+0000 7f5b82894700 1 --2- 172.21.15.88:0/2568660061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b7c0ffbd0 0x7f5b7c100040 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009910 tx=0x7f5b70009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:33.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.345+0000 7f5b82894700 1 -- 172.21.15.88:0/2568660061 shutdown_connections 2024-02-19T19:41:33.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.345+0000 7f5b82894700 1 --2- 172.21.15.88:0/2568660061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b7c0ffbd0 0x7f5b7c100040 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.345+0000 7f5b82894700 1 --2- 172.21.15.88:0/2568660061 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b7c104130 0x7f5b7c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.345+0000 7f5b82894700 1 -- 172.21.15.88:0/2568660061 >> 172.21.15.88:0/2568660061 conn(0x7f5b7c075130 msgr2=0x7f5b7c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:33.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.346+0000 7f5b82894700 1 -- 172.21.15.88:0/2568660061 shutdown_connections 2024-02-19T19:41:33.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.346+0000 7f5b82894700 1 -- 172.21.15.88:0/2568660061 wait complete. 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.346+0000 7f5b82894700 1 Processor -- start 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.346+0000 7f5b82894700 1 -- start start 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b82894700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b7c0ffbd0 0x7f5b7c195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b82894700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b7c104130 0x7f5b7c196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b82894700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c1969b0 con 0x7f5b7c104130 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b82894700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c196af0 con 0x7f5b7c0ffbd0 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b7b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b7c104130 0x7f5b7c196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b7b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b7c104130 0x7f5b7c196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48282/0 (socket says 172.21.15.88:48282) 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b7b7fe700 1 -- 172.21.15.88:0/1052713803 learned_addr learned my addr 172.21.15.88:0/1052713803 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b7bfff700 1 --2- 172.21.15.88:0/1052713803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b7c0ffbd0 0x7f5b7c195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b7b7fe700 1 -- 172.21.15.88:0/1052713803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b7c0ffbd0 msgr2=0x7f5b7c195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:33.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b7b7fe700 1 --2- 172.21.15.88:0/1052713803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b7c0ffbd0 0x7f5b7c195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b7b7fe700 1 -- 172.21.15.88:0/1052713803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b700095f0 con 0x7f5b7c104130 2024-02-19T19:41:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b7b7fe700 1 --2- 172.21.15.88:0/1052713803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b7c104130 0x7f5b7c196360 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5b700092e0 tx=0x7f5b7001ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.347+0000 7f5b797fa700 1 -- 172.21.15.88:0/1052713803 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b70021070 con 0x7f5b7c104130 2024-02-19T19:41:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.348+0000 7f5b797fa700 1 -- 172.21.15.88:0/1052713803 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b7000bc30 con 0x7f5b7c104130 2024-02-19T19:41:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.348+0000 7f5b797fa700 1 -- 172.21.15.88:0/1052713803 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b7000f870 con 0x7f5b7c104130 2024-02-19T19:41:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.348+0000 7f5b82894700 1 -- 172.21.15.88:0/1052713803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c19a8e0 con 0x7f5b7c104130 2024-02-19T19:41:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.348+0000 7f5b82894700 1 -- 172.21.15.88:0/1052713803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c19ad10 con 0x7f5b7c104130 2024-02-19T19:41:33.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.349+0000 7f5b82894700 1 -- 172.21.15.88:0/1052713803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b7c04f2a0 con 0x7f5b7c104130 2024-02-19T19:41:33.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.350+0000 7f5b797fa700 1 -- 172.21.15.88:0/1052713803 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5b70003c40 con 0x7f5b7c104130 2024-02-19T19:41:33.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.351+0000 7f5b797fa700 1 --2- 172.21.15.88:0/1052713803 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b6406d8a0 0x7f5b6406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:33.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.351+0000 7f5b797fa700 1 -- 172.21.15.88:0/1052713803 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5b7008eb00 con 0x7f5b7c104130 2024-02-19T19:41:33.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.351+0000 7f5b7bfff700 1 --2- 172.21.15.88:0/1052713803 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b6406d8a0 0x7f5b6406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:33.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.352+0000 7f5b7bfff700 1 --2- 172.21.15.88:0/1052713803 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b6406d8a0 0x7f5b6406fd50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c005ea0 tx=0x7f5b6c005e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:33.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.355+0000 7f5b797fa700 1 -- 172.21.15.88:0/1052713803 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5b700584f0 con 0x7f5b7c104130 2024-02-19T19:41:33.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.704+0000 7f5b82894700 1 -- 172.21.15.88:0/1052713803 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b7c108230 con 0x7f5b6406d8a0 2024-02-19T19:41:33.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.705+0000 7f5b797fa700 1 -- 172.21.15.88:0/1052713803 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f5b7c108230 con 0x7f5b6406d8a0 2024-02-19T19:41:33.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.709+0000 7f5b82894700 1 -- 172.21.15.88:0/1052713803 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b6406d8a0 msgr2=0x7f5b6406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:33.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.709+0000 7f5b82894700 1 --2- 172.21.15.88:0/1052713803 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b6406d8a0 0x7f5b6406fd50 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c005ea0 tx=0x7f5b6c005e30 comp rx=0 tx=0).stop 2024-02-19T19:41:33.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.709+0000 7f5b82894700 1 -- 172.21.15.88:0/1052713803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b7c104130 msgr2=0x7f5b7c196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.709+0000 7f5b82894700 1 --2- 172.21.15.88:0/1052713803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b7c104130 0x7f5b7c196360 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5b700092e0 tx=0x7f5b7001ce50 comp rx=0 tx=0).stop 2024-02-19T19:41:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.709+0000 7f5b82894700 1 -- 172.21.15.88:0/1052713803 shutdown_connections 2024-02-19T19:41:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.709+0000 7f5b82894700 1 --2- 172.21.15.88:0/1052713803 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5b6406d8a0 0x7f5b6406fd50 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.709+0000 7f5b82894700 1 --2- 172.21.15.88:0/1052713803 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5b7c0ffbd0 0x7f5b7c195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.709+0000 7f5b82894700 1 --2- 172.21.15.88:0/1052713803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b7c104130 0x7f5b7c196360 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.709+0000 7f5b82894700 1 -- 172.21.15.88:0/1052713803 >> 172.21.15.88:0/1052713803 conn(0x7f5b7c075130 msgr2=0x7f5b7c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.709+0000 7f5b82894700 1 -- 172.21.15.88:0/1052713803 shutdown_connections 2024-02-19T19:41:33.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.709+0000 7f5b82894700 1 -- 172.21.15.88:0/1052713803 wait complete. 2024-02-19T19:41:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.823+0000 7f7ce6343700 1 -- 172.21.15.88:0/513925591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00ffbd0 msgr2=0x7f7ce0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.823+0000 7f7ce6343700 1 --2- 172.21.15.88:0/513925591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00ffbd0 0x7f7ce0100040 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4009910 tx=0x7f7cd4009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.823+0000 7f7ce6343700 1 -- 172.21.15.88:0/513925591 shutdown_connections 2024-02-19T19:41:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.823+0000 7f7ce6343700 1 --2- 172.21.15.88:0/513925591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce00ffbd0 0x7f7ce0100040 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.823+0000 7f7ce6343700 1 --2- 172.21.15.88:0/513925591 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce0104130 0x7f7ce0104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.823+0000 7f7ce6343700 1 -- 172.21.15.88:0/513925591 >> 172.21.15.88:0/513925591 conn(0x7f7ce0075130 msgr2=0x7f7ce0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.823+0000 7f7ce6343700 1 -- 172.21.15.88:0/513925591 shutdown_connections 2024-02-19T19:41:33.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.823+0000 7f7ce6343700 1 -- 172.21.15.88:0/513925591 wait complete. 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.824+0000 7f7ce6343700 1 Processor -- start 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.824+0000 7f7ce6343700 1 -- start start 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.824+0000 7f7ce6343700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce00ffbd0 0x7f7ce0194d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.824+0000 7f7ce6343700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce0104130 0x7f7ce0195270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.824+0000 7f7ce6343700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce0195950 con 0x7f7ce0104130 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.824+0000 7f7ce6343700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ce01996e0 con 0x7f7ce00ffbd0 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.824+0000 7f7cdf7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce0104130 0x7f7ce0195270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.824+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce00ffbd0 0x7f7ce0194d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.824+0000 7f7cdffff700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce00ffbd0 0x7f7ce0194d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.132:3300/0 says I am v2:172.21.15.88:37152/0 (socket says 172.21.15.88:37152) 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.824+0000 7f7cdf7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce0104130 0x7f7ce0195270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48296/0 (socket says 172.21.15.88:48296) 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.824+0000 7f7cdffff700 1 -- 172.21.15.88:0/2562706835 learned_addr learned my addr 172.21.15.88:0/2562706835 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.825+0000 7f7cdf7fe700 1 -- 172.21.15.88:0/2562706835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce00ffbd0 msgr2=0x7f7ce0194d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:33.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.825+0000 7f7cdf7fe700 1 --2- 172.21.15.88:0/2562706835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce00ffbd0 0x7f7ce0194d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:33.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.825+0000 7f7cdf7fe700 1 -- 172.21.15.88:0/2562706835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cd40095f0 con 0x7f7ce0104130 2024-02-19T19:41:33.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.825+0000 7f7cdf7fe700 1 --2- 172.21.15.88:0/2562706835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce0104130 0x7f7ce0195270 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4005b40 tx=0x7f7cd400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:33.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.826+0000 7f7cdd7fa700 1 -- 172.21.15.88:0/2562706835 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd401d070 con 0x7f7ce0104130 2024-02-19T19:41:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.826+0000 7f7cdd7fa700 1 -- 172.21.15.88:0/2562706835 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cd4005dc0 con 0x7f7ce0104130 2024-02-19T19:41:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.826+0000 7f7cdd7fa700 1 -- 172.21.15.88:0/2562706835 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cd400fe40 con 0x7f7ce0104130 2024-02-19T19:41:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.826+0000 7f7ce6343700 1 -- 172.21.15.88:0/2562706835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ce0199960 con 0x7f7ce0104130 2024-02-19T19:41:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.826+0000 7f7ce6343700 1 -- 172.21.15.88:0/2562706835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ce0199d40 con 0x7f7ce0104130 2024-02-19T19:41:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.827+0000 7f7ce6343700 1 -- 172.21.15.88:0/2562706835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ce004f2a0 con 0x7f7ce0104130 2024-02-19T19:41:33.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.827+0000 7f7cdd7fa700 1 -- 172.21.15.88:0/2562706835 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7cd4003a60 con 0x7f7ce0104130 2024-02-19T19:41:33.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.828+0000 7f7cdd7fa700 1 --2- 172.21.15.88:0/2562706835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cc806d8f0 0x7f7cc806fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:33.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.828+0000 7f7cdd7fa700 1 -- 172.21.15.88:0/2562706835 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7cd408e710 con 0x7f7ce0104130 2024-02-19T19:41:33.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.828+0000 7f7cdffff700 1 --2- 172.21.15.88:0/2562706835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cc806d8f0 0x7f7cc806fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:33.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.829+0000 7f7cdffff700 1 --2- 172.21.15.88:0/2562706835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cc806d8f0 0x7f7cc806fda0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0005950 tx=0x7f7cd000d710 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:33.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:33.833+0000 7f7cdd7fa700 1 -- 172.21.15.88:0/2562706835 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7cd4057fe0 con 0x7f7ce0104130 2024-02-19T19:41:34.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.056+0000 7f7ce6343700 1 -- 172.21.15.88:0/2562706835 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ce019a180 con 0x7f7cc806d8f0 2024-02-19T19:41:34.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.067+0000 7f7cdd7fa700 1 -- 172.21.15.88:0/2562706835 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7ce019a180 con 0x7f7cc806d8f0 2024-02-19T19:41:34.069 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:41:34.069 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (38m) 15m ago 49m 23.9M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:41:34.069 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (49m) 15m ago 49m 10.1M - 18.2.0 dc2bc1663786 147e8b52a9e6 2024-02-19T19:41:34.069 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (31m) 15m ago 47m 9131k - 18.2.0 dc2bc1663786 fa6056e1ed37 2024-02-19T19:41:34.069 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (34m) 15m ago 49m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:41:34.069 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (34m) 15m ago 47m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:41:34.069 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (38m) 15m ago 48m 84.6M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:41:34.069 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (30m) 15m ago 42m 20.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (30m) 15m ago 42m 20.3M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (30m) 15m ago 42m 21.5M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (15m) 15m ago 42m 12.5M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (40m) 15m ago 50m 512M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (39m) 15m ago 47m 458M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (36m) 15m ago 51m 74.1M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (36m) 15m ago 47m 52.9M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (39m) 15m ago 49m 15.6M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (39m) 15m ago 47m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (34m) 15m ago 46m 132M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (33m) 15m ago 46m 107M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (32m) 15m ago 45m 89.0M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (32m) 15m ago 45m 108M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (31m) 15m ago 44m 107M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (31m) 15m ago 44m 88.3M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:41:34.070 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (39m) 15m ago 48m 86.4M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:41:34.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.070+0000 7f7ce6343700 1 -- 172.21.15.88:0/2562706835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cc806d8f0 msgr2=0x7f7cc806fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.070+0000 7f7ce6343700 1 --2- 172.21.15.88:0/2562706835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cc806d8f0 0x7f7cc806fda0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7cd0005950 tx=0x7f7cd000d710 comp rx=0 tx=0).stop 2024-02-19T19:41:34.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.070+0000 7f7ce6343700 1 -- 172.21.15.88:0/2562706835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce0104130 msgr2=0x7f7ce0195270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.070+0000 7f7ce6343700 1 --2- 172.21.15.88:0/2562706835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce0104130 0x7f7ce0195270 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7cd4005b40 tx=0x7f7cd400bd70 comp rx=0 tx=0).stop 2024-02-19T19:41:34.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.071+0000 7f7ce6343700 1 -- 172.21.15.88:0/2562706835 shutdown_connections 2024-02-19T19:41:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.071+0000 7f7ce6343700 1 --2- 172.21.15.88:0/2562706835 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7cc806d8f0 0x7f7cc806fda0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.071+0000 7f7ce6343700 1 --2- 172.21.15.88:0/2562706835 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7ce00ffbd0 0x7f7ce0194d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.071+0000 7f7ce6343700 1 --2- 172.21.15.88:0/2562706835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ce0104130 0x7f7ce0195270 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.071+0000 7f7ce6343700 1 -- 172.21.15.88:0/2562706835 >> 172.21.15.88:0/2562706835 conn(0x7f7ce0075130 msgr2=0x7f7ce00feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.071+0000 7f7ce6343700 1 -- 172.21.15.88:0/2562706835 shutdown_connections 2024-02-19T19:41:34.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.071+0000 7f7ce6343700 1 -- 172.21.15.88:0/2562706835 wait complete. 2024-02-19T19:41:34.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.176+0000 7f36ca4c9700 1 -- 172.21.15.88:0/3274323834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36c40ff3c0 msgr2=0x7f36c40ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.176+0000 7f36ca4c9700 1 --2- 172.21.15.88:0/3274323834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36c40ff3c0 0x7f36c40ff790 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009910 tx=0x7f36b4009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:34.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.176+0000 7f36ca4c9700 1 -- 172.21.15.88:0/3274323834 shutdown_connections 2024-02-19T19:41:34.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.176+0000 7f36ca4c9700 1 --2- 172.21.15.88:0/3274323834 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36c40ffd60 0x7f36c4103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.176+0000 7f36ca4c9700 1 --2- 172.21.15.88:0/3274323834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36c40ff3c0 0x7f36c40ff790 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.176+0000 7f36ca4c9700 1 -- 172.21.15.88:0/3274323834 >> 172.21.15.88:0/3274323834 conn(0x7f36c40fac80 msgr2=0x7f36c40fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:34.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.176+0000 7f36ca4c9700 1 -- 172.21.15.88:0/3274323834 shutdown_connections 2024-02-19T19:41:34.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.176+0000 7f36ca4c9700 1 -- 172.21.15.88:0/3274323834 wait complete. 2024-02-19T19:41:34.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.176+0000 7f36ca4c9700 1 Processor -- start 2024-02-19T19:41:34.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.176+0000 7f36ca4c9700 1 -- start start 2024-02-19T19:41:34.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.177+0000 7f36ca4c9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36c40ff3c0 0x7f36c41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:34.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.177+0000 7f36ca4c9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36c40ffd60 0x7f36c4198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:34.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.177+0000 7f36ca4c9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c4193aa0 con 0x7f36c40ffd60 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.177+0000 7f36ca4c9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c4193c10 con 0x7f36c40ff3c0 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.177+0000 7f36c37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36c40ffd60 0x7f36c4198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.177+0000 7f36c37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36c40ffd60 0x7f36c4198f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48330/0 (socket says 172.21.15.88:48330) 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.177+0000 7f36c37fe700 1 -- 172.21.15.88:0/1107250985 learned_addr learned my addr 172.21.15.88:0/1107250985 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.177+0000 7f36c3fff700 1 --2- 172.21.15.88:0/1107250985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36c40ff3c0 0x7f36c41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.177+0000 7f36c37fe700 1 -- 172.21.15.88:0/1107250985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36c40ff3c0 msgr2=0x7f36c41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.177+0000 7f36c37fe700 1 --2- 172.21.15.88:0/1107250985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36c40ff3c0 0x7f36c41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.177+0000 7f36c37fe700 1 -- 172.21.15.88:0/1107250985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b40095f0 con 0x7f36c40ffd60 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.178+0000 7f36c37fe700 1 --2- 172.21.15.88:0/1107250985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36c40ffd60 0x7f36c4198f50 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f36b800d900 tx=0x7f36b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.178+0000 7f36c17fa700 1 -- 172.21.15.88:0/1107250985 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36b8009900 con 0x7f36c40ffd60 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.178+0000 7f36ca4c9700 1 -- 172.21.15.88:0/1107250985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c4193ef0 con 0x7f36c40ffd60 2024-02-19T19:41:34.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.178+0000 7f36c17fa700 1 -- 172.21.15.88:0/1107250985 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36b8010460 con 0x7f36c40ffd60 2024-02-19T19:41:34.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.178+0000 7f36c17fa700 1 -- 172.21.15.88:0/1107250985 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36b800f5f0 con 0x7f36c40ffd60 2024-02-19T19:41:34.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.178+0000 7f36ca4c9700 1 -- 172.21.15.88:0/1107250985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c4194440 con 0x7f36c40ffd60 2024-02-19T19:41:34.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.179+0000 7f36ca4c9700 1 -- 172.21.15.88:0/1107250985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c404f2a0 con 0x7f36c40ffd60 2024-02-19T19:41:34.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.180+0000 7f36c17fa700 1 -- 172.21.15.88:0/1107250985 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f36b80105d0 con 0x7f36c40ffd60 2024-02-19T19:41:34.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.181+0000 7f36c17fa700 1 --2- 172.21.15.88:0/1107250985 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f36ac06d8a0 0x7f36ac06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:34.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.181+0000 7f36c17fa700 1 -- 172.21.15.88:0/1107250985 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f36b808d360 con 0x7f36c40ffd60 2024-02-19T19:41:34.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.181+0000 7f36c3fff700 1 --2- 172.21.15.88:0/1107250985 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f36ac06d8a0 0x7f36ac06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:34.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.181+0000 7f36c3fff700 1 --2- 172.21.15.88:0/1107250985 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f36ac06d8a0 0x7f36ac06fd50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f36b400b5c0 tx=0x7f36b401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:34.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.186+0000 7f36c17fa700 1 -- 172.21.15.88:0/1107250985 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f36b8056b10 con 0x7f36c40ffd60 2024-02-19T19:41:34.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.478+0000 7f36ca4c9700 1 -- 172.21.15.88:0/1107250985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f36c4194fd0 con 0x7f36c40ffd60 2024-02-19T19:41:34.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.479+0000 7f36c17fa700 1 -- 172.21.15.88:0/1107250985 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f36b805a1d0 con 0x7f36c40ffd60 2024-02-19T19:41:34.480 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:41:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:41:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:41:34.481 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:41:34.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.481+0000 7f36ca4c9700 1 -- 172.21.15.88:0/1107250985 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f36ac06d8a0 msgr2=0x7f36ac06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.481+0000 7f36ca4c9700 1 --2- 172.21.15.88:0/1107250985 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f36ac06d8a0 0x7f36ac06fd50 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f36b400b5c0 tx=0x7f36b401a040 comp rx=0 tx=0).stop 2024-02-19T19:41:34.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.481+0000 7f36ca4c9700 1 -- 172.21.15.88:0/1107250985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36c40ffd60 msgr2=0x7f36c4198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.481+0000 7f36ca4c9700 1 --2- 172.21.15.88:0/1107250985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36c40ffd60 0x7f36c4198f50 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f36b800d900 tx=0x7f36b800dcc0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.481+0000 7f36ca4c9700 1 -- 172.21.15.88:0/1107250985 shutdown_connections 2024-02-19T19:41:34.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.481+0000 7f36ca4c9700 1 --2- 172.21.15.88:0/1107250985 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f36ac06d8a0 0x7f36ac06fd50 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.481+0000 7f36ca4c9700 1 --2- 172.21.15.88:0/1107250985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f36c40ff3c0 0x7f36c41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.481+0000 7f36ca4c9700 1 --2- 172.21.15.88:0/1107250985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36c40ffd60 0x7f36c4198f50 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.481+0000 7f36ca4c9700 1 -- 172.21.15.88:0/1107250985 >> 172.21.15.88:0/1107250985 conn(0x7f36c40fac80 msgr2=0x7f36c4069720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:34.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.481+0000 7f36ca4c9700 1 -- 172.21.15.88:0/1107250985 shutdown_connections 2024-02-19T19:41:34.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.481+0000 7f36ca4c9700 1 -- 172.21.15.88:0/1107250985 wait complete. 2024-02-19T19:41:34.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.586+0000 7f581dfe7700 1 -- 172.21.15.88:0/3475784375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5818104130 msgr2=0x7f5818104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.586+0000 7f581dfe7700 1 --2- 172.21.15.88:0/3475784375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5818104130 0x7f5818104500 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f5808009910 tx=0x7f5808009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:34.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.586+0000 7f581dfe7700 1 -- 172.21.15.88:0/3475784375 shutdown_connections 2024-02-19T19:41:34.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.586+0000 7f581dfe7700 1 --2- 172.21.15.88:0/3475784375 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58180ffbd0 0x7f5818100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.586+0000 7f581dfe7700 1 --2- 172.21.15.88:0/3475784375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5818104130 0x7f5818104500 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.586+0000 7f581dfe7700 1 -- 172.21.15.88:0/3475784375 >> 172.21.15.88:0/3475784375 conn(0x7f5818075130 msgr2=0x7f5818075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:34.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.586+0000 7f581dfe7700 1 -- 172.21.15.88:0/3475784375 shutdown_connections 2024-02-19T19:41:34.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.586+0000 7f581dfe7700 1 -- 172.21.15.88:0/3475784375 wait complete. 2024-02-19T19:41:34.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.586+0000 7f581dfe7700 1 Processor -- start 2024-02-19T19:41:34.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f581dfe7700 1 -- start start 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f581dfe7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58180ffbd0 0x7f5818195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f581dfe7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5818104130 0x7f58181963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f581dfe7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5818196aa0 con 0x7f5818104130 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f581dfe7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581819a830 con 0x7f58180ffbd0 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5818104130 0x7f58181963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5818104130 0x7f58181963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48340/0 (socket says 172.21.15.88:48340) 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f5816ffd700 1 -- 172.21.15.88:0/4238837844 learned_addr learned my addr 172.21.15.88:0/4238837844 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f58177fe700 1 --2- 172.21.15.88:0/4238837844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58180ffbd0 0x7f5818195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f5816ffd700 1 -- 172.21.15.88:0/4238837844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58180ffbd0 msgr2=0x7f5818195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f5816ffd700 1 --2- 172.21.15.88:0/4238837844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58180ffbd0 0x7f5818195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f5816ffd700 1 -- 172.21.15.88:0/4238837844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58080095f0 con 0x7f5818104130 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f5816ffd700 1 --2- 172.21.15.88:0/4238837844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5818104130 0x7f58181963c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f580000d8d0 tx=0x7f580000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.587+0000 7f5814ff9700 1 -- 172.21.15.88:0/4238837844 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5800009940 con 0x7f5818104130 2024-02-19T19:41:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.588+0000 7f581dfe7700 1 -- 172.21.15.88:0/4238837844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581819ab10 con 0x7f5818104130 2024-02-19T19:41:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.588+0000 7f5814ff9700 1 -- 172.21.15.88:0/4238837844 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5800010460 con 0x7f5818104130 2024-02-19T19:41:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.588+0000 7f5814ff9700 1 -- 172.21.15.88:0/4238837844 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f580000f5f0 con 0x7f5818104130 2024-02-19T19:41:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.588+0000 7f581dfe7700 1 -- 172.21.15.88:0/4238837844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581819b060 con 0x7f5818104130 2024-02-19T19:41:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.588+0000 7f581dfe7700 1 -- 172.21.15.88:0/4238837844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58181086a0 con 0x7f5818104130 2024-02-19T19:41:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.589+0000 7f5814ff9700 1 -- 172.21.15.88:0/4238837844 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f58000105d0 con 0x7f5818104130 2024-02-19T19:41:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.589+0000 7f5814ff9700 1 --2- 172.21.15.88:0/4238837844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f580406d8a0 0x7f580406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.590+0000 7f5814ff9700 1 -- 172.21.15.88:0/4238837844 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f580008d430 con 0x7f5818104130 2024-02-19T19:41:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.590+0000 7f58177fe700 1 --2- 172.21.15.88:0/4238837844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f580406d8a0 0x7f580406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:34.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.590+0000 7f58177fe700 1 --2- 172.21.15.88:0/4238837844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f580406d8a0 0x7f580406fd50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f580800b5c0 tx=0x7f580801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:34.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.594+0000 7f5814ff9700 1 -- 172.21.15.88:0/4238837844 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5800056c60 con 0x7f5818104130 2024-02-19T19:41:34.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:34 smithi088 ceph-mon[88643]: from='client.45223 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:34.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:34 smithi088 ceph-mon[88643]: pgmap v1142: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-02-19T19:41:34.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:34 smithi088 ceph-mon[88643]: from='client.36044 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:34.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:34 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] up:resolve 2024-02-19T19:41:34.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:34 smithi088 ceph-mon[88643]: fsmap cephfs:2/2 {0=cephfs.smithi088.ajgvpz=up:replay,1=cephfs.smithi088.psjiah=up:resolve} 2 up:standby 2024-02-19T19:41:34.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:34 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1107250985' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:34.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.848+0000 7f581dfe7700 1 -- 172.21.15.88:0/4238837844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f581819b2d0 con 0x7f5818104130 2024-02-19T19:41:34.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.849+0000 7f5814ff9700 1 -- 172.21.15.88:0/4238837844 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 36 v36) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f5800016770 con 0x7f5818104130 2024-02-19T19:41:34.850 INFO:teuthology.orchestra.run.smithi088.stdout:e36 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 36 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:41:34.450025+0000 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:41:34.851 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=34430,1=34462} 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:41:34.852 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{0:34430} state up:resolve seq 466 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{1:34462} state up:resolve seq 461 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:41:34.853 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:41:34.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.852+0000 7f581dfe7700 1 -- 172.21.15.88:0/4238837844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f580406d8a0 msgr2=0x7f580406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.852+0000 7f581dfe7700 1 --2- 172.21.15.88:0/4238837844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f580406d8a0 0x7f580406fd50 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f580800b5c0 tx=0x7f580801a040 comp rx=0 tx=0).stop 2024-02-19T19:41:34.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.852+0000 7f581dfe7700 1 -- 172.21.15.88:0/4238837844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5818104130 msgr2=0x7f58181963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.852+0000 7f581dfe7700 1 --2- 172.21.15.88:0/4238837844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5818104130 0x7f58181963c0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f580000d8d0 tx=0x7f580000dc90 comp rx=0 tx=0).stop 2024-02-19T19:41:34.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.852+0000 7f581dfe7700 1 -- 172.21.15.88:0/4238837844 shutdown_connections 2024-02-19T19:41:34.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.852+0000 7f581dfe7700 1 --2- 172.21.15.88:0/4238837844 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f580406d8a0 0x7f580406fd50 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.852+0000 7f581dfe7700 1 --2- 172.21.15.88:0/4238837844 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f58180ffbd0 0x7f5818195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.852+0000 7f581dfe7700 1 --2- 172.21.15.88:0/4238837844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5818104130 0x7f58181963c0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.852+0000 7f581dfe7700 1 -- 172.21.15.88:0/4238837844 >> 172.21.15.88:0/4238837844 conn(0x7f5818075130 msgr2=0x7f58180fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:34.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.852+0000 7f581dfe7700 1 -- 172.21.15.88:0/4238837844 shutdown_connections 2024-02-19T19:41:34.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.852+0000 7f581dfe7700 1 -- 172.21.15.88:0/4238837844 wait complete. 2024-02-19T19:41:34.855 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 36 2024-02-19T19:41:34.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.965+0000 7f621bcc7700 1 -- 172.21.15.88:0/2967537934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6214069140 msgr2=0x7f6214104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.965+0000 7f621bcc7700 1 --2- 172.21.15.88:0/2967537934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6214069140 0x7f6214104eb0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6210009910 tx=0x7f6210009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:34.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.965+0000 7f621bcc7700 1 -- 172.21.15.88:0/2967537934 shutdown_connections 2024-02-19T19:41:34.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.965+0000 7f621bcc7700 1 --2- 172.21.15.88:0/2967537934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6214069140 0x7f6214104eb0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.965+0000 7f621bcc7700 1 --2- 172.21.15.88:0/2967537934 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62140687a0 0x7f6214068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.965+0000 7f621bcc7700 1 -- 172.21.15.88:0/2967537934 >> 172.21.15.88:0/2967537934 conn(0x7f6214075130 msgr2=0x7f6214075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:34.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.966+0000 7f621bcc7700 1 -- 172.21.15.88:0/2967537934 shutdown_connections 2024-02-19T19:41:34.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.966+0000 7f621bcc7700 1 -- 172.21.15.88:0/2967537934 wait complete. 2024-02-19T19:41:34.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.966+0000 7f621bcc7700 1 Processor -- start 2024-02-19T19:41:34.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.966+0000 7f621bcc7700 1 -- start start 2024-02-19T19:41:34.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.966+0000 7f621bcc7700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62140687a0 0x7f6214195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:34.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.966+0000 7f621bcc7700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214069140 0x7f62141963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:34.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.966+0000 7f621bcc7700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6214196a80 con 0x7f62140687a0 2024-02-19T19:41:34.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.966+0000 7f621bcc7700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621419a810 con 0x7f6214069140 2024-02-19T19:41:34.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f6219a63700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62140687a0 0x7f6214195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:34.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f6219a63700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62140687a0 0x7f6214195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48350/0 (socket says 172.21.15.88:48350) 2024-02-19T19:41:34.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f6219a63700 1 -- 172.21.15.88:0/3032189829 learned_addr learned my addr 172.21.15.88:0/3032189829 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:34.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f6219262700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214069140 0x7f62141963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:34.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f6219a63700 1 -- 172.21.15.88:0/3032189829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214069140 msgr2=0x7f62141963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:34.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f6219a63700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214069140 0x7f62141963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f6219a63700 1 -- 172.21.15.88:0/3032189829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62100095f0 con 0x7f62140687a0 2024-02-19T19:41:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f6219262700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214069140 0x7f62141963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:41:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f6219a63700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62140687a0 0x7f6214195e60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f620400d8d0 tx=0x7f620400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f620affd700 1 -- 172.21.15.88:0/3032189829 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6204009940 con 0x7f62140687a0 2024-02-19T19:41:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f621bcc7700 1 -- 172.21.15.88:0/3032189829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621419aaf0 con 0x7f62140687a0 2024-02-19T19:41:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f620affd700 1 -- 172.21.15.88:0/3032189829 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6204010460 con 0x7f62140687a0 2024-02-19T19:41:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.967+0000 7f621bcc7700 1 -- 172.21.15.88:0/3032189829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621419b040 con 0x7f62140687a0 2024-02-19T19:41:34.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.969+0000 7f620affd700 1 -- 172.21.15.88:0/3032189829 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f620400f5f0 con 0x7f62140687a0 2024-02-19T19:41:34.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.969+0000 7f621bcc7700 1 -- 172.21.15.88:0/3032189829 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6214108750 con 0x7f62140687a0 2024-02-19T19:41:34.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.969+0000 7f620affd700 1 -- 172.21.15.88:0/3032189829 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f62040105d0 con 0x7f62140687a0 2024-02-19T19:41:34.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.969+0000 7f620affd700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f620006d8a0 0x7f620006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:34.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.970+0000 7f620affd700 1 -- 172.21.15.88:0/3032189829 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f620408d430 con 0x7f62140687a0 2024-02-19T19:41:34.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.970+0000 7f6219262700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f620006d8a0 0x7f620006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:34.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.970+0000 7f6219262700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f620006d8a0 0x7f620006fd50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f621000b5c0 tx=0x7f62100053a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:34.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:34.974+0000 7f620affd700 1 -- 172.21.15.88:0/3032189829 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6204056c60 con 0x7f62140687a0 2024-02-19T19:41:34.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:34 smithi132 ceph-mon[70676]: from='client.45223 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:34 smithi132 ceph-mon[70676]: pgmap v1142: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-02-19T19:41:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:34 smithi132 ceph-mon[70676]: from='client.36044 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:34 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] up:resolve 2024-02-19T19:41:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:34 smithi132 ceph-mon[70676]: fsmap cephfs:2/2 {0=cephfs.smithi088.ajgvpz=up:replay,1=cephfs.smithi088.psjiah=up:resolve} 2 up:standby 2024-02-19T19:41:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:34 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1107250985' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.200+0000 7f621bcc7700 1 -- 172.21.15.88:0/3032189829 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f621419b2b0 con 0x7f620006d8a0 2024-02-19T19:41:35.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.201+0000 7f620affd700 1 -- 172.21.15.88:0/3032189829 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f621419b2b0 con 0x7f620006d8a0 2024-02-19T19:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:41:35.201 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526", 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": true, 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "Upgrading all daemon types on all hosts", 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [ 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: "mds", 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: "osd", 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: "mon", 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr" 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": "16/23 daemons upgraded", 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:41:35.202 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:41:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.203+0000 7f621bcc7700 1 -- 172.21.15.88:0/3032189829 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f620006d8a0 msgr2=0x7f620006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.203+0000 7f621bcc7700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f620006d8a0 0x7f620006fd50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f621000b5c0 tx=0x7f62100053a0 comp rx=0 tx=0).stop 2024-02-19T19:41:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.203+0000 7f621bcc7700 1 -- 172.21.15.88:0/3032189829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62140687a0 msgr2=0x7f6214195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.203+0000 7f621bcc7700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62140687a0 0x7f6214195e60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f620400d8d0 tx=0x7f620400dc90 comp rx=0 tx=0).stop 2024-02-19T19:41:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.203+0000 7f621bcc7700 1 -- 172.21.15.88:0/3032189829 shutdown_connections 2024-02-19T19:41:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.203+0000 7f621bcc7700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f620006d8a0 0x7f620006fd50 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.203+0000 7f621bcc7700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62140687a0 0x7f6214195e60 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:35.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.203+0000 7f621bcc7700 1 --2- 172.21.15.88:0/3032189829 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6214069140 0x7f62141963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:35.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.203+0000 7f621bcc7700 1 -- 172.21.15.88:0/3032189829 >> 172.21.15.88:0/3032189829 conn(0x7f6214075130 msgr2=0x7f6214104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:35.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.203+0000 7f621bcc7700 1 -- 172.21.15.88:0/3032189829 shutdown_connections 2024-02-19T19:41:35.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.203+0000 7f621bcc7700 1 -- 172.21.15.88:0/3032189829 wait complete. 2024-02-19T19:41:35.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.315+0000 7f4d870fb700 1 -- 172.21.15.88:0/1088447871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d800ff3c0 msgr2=0x7f4d800ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:35.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.315+0000 7f4d870fb700 1 --2- 172.21.15.88:0/1088447871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d800ff3c0 0x7f4d800ff790 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f4d74009910 tx=0x7f4d74009c20 comp rx=0 tx=0).stop 2024-02-19T19:41:35.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.315+0000 7f4d870fb700 1 -- 172.21.15.88:0/1088447871 shutdown_connections 2024-02-19T19:41:35.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.315+0000 7f4d870fb700 1 --2- 172.21.15.88:0/1088447871 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d800ffd60 0x7f4d80103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:35.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.315+0000 7f4d870fb700 1 --2- 172.21.15.88:0/1088447871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d800ff3c0 0x7f4d800ff790 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:35.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.315+0000 7f4d870fb700 1 -- 172.21.15.88:0/1088447871 >> 172.21.15.88:0/1088447871 conn(0x7f4d800fac80 msgr2=0x7f4d800fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:35.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.315+0000 7f4d870fb700 1 -- 172.21.15.88:0/1088447871 shutdown_connections 2024-02-19T19:41:35.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.315+0000 7f4d870fb700 1 -- 172.21.15.88:0/1088447871 wait complete. 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d870fb700 1 Processor -- start 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d870fb700 1 -- start start 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d870fb700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d800ff3c0 0x7f4d80196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d870fb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d800ffd60 0x7f4d80197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d870fb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80197850 con 0x7f4d800ffd60 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d870fb700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d80197990 con 0x7f4d800ff3c0 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d7ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d800ffd60 0x7f4d80197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d7ffff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d800ffd60 0x7f4d80197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48368/0 (socket says 172.21.15.88:48368) 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d7ffff700 1 -- 172.21.15.88:0/4134707326 learned_addr learned my addr 172.21.15.88:0/4134707326 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d84e97700 1 --2- 172.21.15.88:0/4134707326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d800ff3c0 0x7f4d80196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d7ffff700 1 -- 172.21.15.88:0/4134707326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d800ff3c0 msgr2=0x7f4d80196cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:35.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d7ffff700 1 --2- 172.21.15.88:0/4134707326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d800ff3c0 0x7f4d80196cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.316+0000 7f4d7ffff700 1 -- 172.21.15.88:0/4134707326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d740095f0 con 0x7f4d800ffd60 2024-02-19T19:41:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.317+0000 7f4d7ffff700 1 --2- 172.21.15.88:0/4134707326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d800ffd60 0x7f4d80197230 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00d8d0 tx=0x7f4d6c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.317+0000 7f4d7dffb700 1 -- 172.21.15.88:0/4134707326 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d6c0041f0 con 0x7f4d800ffd60 2024-02-19T19:41:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.317+0000 7f4d7dffb700 1 -- 172.21.15.88:0/4134707326 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d6c007620 con 0x7f4d800ffd60 2024-02-19T19:41:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.317+0000 7f4d870fb700 1 -- 172.21.15.88:0/4134707326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d8019fd60 con 0x7f4d800ffd60 2024-02-19T19:41:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.317+0000 7f4d7dffb700 1 -- 172.21.15.88:0/4134707326 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d6c0056b0 con 0x7f4d800ffd60 2024-02-19T19:41:35.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.317+0000 7f4d870fb700 1 -- 172.21.15.88:0/4134707326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d801a0220 con 0x7f4d800ffd60 2024-02-19T19:41:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.318+0000 7f4d870fb700 1 -- 172.21.15.88:0/4134707326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d801076d0 con 0x7f4d800ffd60 2024-02-19T19:41:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.318+0000 7f4d7dffb700 1 -- 172.21.15.88:0/4134707326 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4d6c004750 con 0x7f4d800ffd60 2024-02-19T19:41:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.319+0000 7f4d7dffb700 1 --2- 172.21.15.88:0/4134707326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4d7006d8a0 0x7f4d7006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:41:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.319+0000 7f4d7dffb700 1 -- 172.21.15.88:0/4134707326 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4d6c08d2d0 con 0x7f4d800ffd60 2024-02-19T19:41:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.319+0000 7f4d84e97700 1 --2- 172.21.15.88:0/4134707326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4d7006d8a0 0x7f4d7006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:41:35.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.319+0000 7f4d84e97700 1 --2- 172.21.15.88:0/4134707326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4d7006d8a0 0x7f4d7006fd50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400b5c0 tx=0x7f4d7401a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:41:35.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.324+0000 7f4d7dffb700 1 -- 172.21.15.88:0/4134707326 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4d6c056b00 con 0x7f4d800ffd60 2024-02-19T19:41:35.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.619+0000 7f4d870fb700 1 -- 172.21.15.88:0/4134707326 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4d8002cc30 con 0x7f4d800ffd60 2024-02-19T19:41:35.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.619+0000 7f4d7dffb700 1 -- 172.21.15.88:0/4134707326 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+4177 (secure 0 0 0) 0x7f4d6c05a1c0 con 0x7f4d800ffd60 2024-02-19T19:41:35.620 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_WARN failed to probe daemons or devices; 1 filesystem is degraded 2024-02-19T19:41:35.620 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T19:41:35.620 INFO:teuthology.orchestra.run.smithi088.stdout: host smithi132 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi132/config 2024-02-19T19:41:35.620 INFO:teuthology.orchestra.run.smithi088.stdout:/usr/bin/podman: timeout after 895 seconds 2024-02-19T19:41:35.620 INFO:teuthology.orchestra.run.smithi088.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpyjbn_228:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:41:35.620 INFO:teuthology.orchestra.run.smithi088.stdout:Traceback (most recent call last): 2024-02-19T19:41:35.620 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-02-19T19:41:35.620 INFO:teuthology.orchestra.run.smithi088.stdout: return _run_code(code, main_globals, None, 2024-02-19T19:41:35.620 INFO:teuthology.orchestra.run.smithi088.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout: exec(code, run_globals) 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5520, in 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 5508, in main 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 406, in _infer_config 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 321, in _infer_fsid 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 434, in _infer_image 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 308, in _validate_fsid 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/__main__.py", line 3301, in command_ceph_volume 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout: File "/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/cephadm.b0083ef89477701c918bf0c54d03f197657c333493138352c65fd624e1029b5f/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 -e NODE_NAME=smithi132 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/run/ceph:z -v /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715:/var/log/ceph:z -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/selinux:/sys/fs/selinux:ro -v /:/rootfs -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmpyjbn_228:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 inventory --format=json-pretty --filter-for-batch 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-02-19T19:41:35.621 INFO:teuthology.orchestra.run.smithi088.stdout: fs cephfs is degraded 2024-02-19T19:41:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.621+0000 7f4d870fb700 1 -- 172.21.15.88:0/4134707326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4d7006d8a0 msgr2=0x7f4d7006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.621+0000 7f4d870fb700 1 --2- 172.21.15.88:0/4134707326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4d7006d8a0 0x7f4d7006fd50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400b5c0 tx=0x7f4d7401a040 comp rx=0 tx=0).stop 2024-02-19T19:41:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.621+0000 7f4d870fb700 1 -- 172.21.15.88:0/4134707326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d800ffd60 msgr2=0x7f4d80197230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:41:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.621+0000 7f4d870fb700 1 --2- 172.21.15.88:0/4134707326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d800ffd60 0x7f4d80197230 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c00d8d0 tx=0x7f4d6c00dbe0 comp rx=0 tx=0).stop 2024-02-19T19:41:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.621+0000 7f4d870fb700 1 -- 172.21.15.88:0/4134707326 shutdown_connections 2024-02-19T19:41:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.621+0000 7f4d870fb700 1 --2- 172.21.15.88:0/4134707326 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4d7006d8a0 0x7f4d7006fd50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.621+0000 7f4d870fb700 1 --2- 172.21.15.88:0/4134707326 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4d800ff3c0 0x7f4d80196cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.621+0000 7f4d870fb700 1 --2- 172.21.15.88:0/4134707326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d800ffd60 0x7f4d80197230 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:41:35.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.621+0000 7f4d870fb700 1 -- 172.21.15.88:0/4134707326 >> 172.21.15.88:0/4134707326 conn(0x7f4d800fac80 msgr2=0x7f4d800fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:41:35.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.621+0000 7f4d870fb700 1 -- 172.21.15.88:0/4134707326 shutdown_connections 2024-02-19T19:41:35.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:41:35.621+0000 7f4d870fb700 1 -- 172.21.15.88:0/4134707326 wait complete. 2024-02-19T19:41:35.763 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:35 smithi132 ceph-mon[70676]: from='client.36048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:35.764 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:35 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] up:resolve 2024-02-19T19:41:35.764 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:35 smithi132 ceph-mon[70676]: fsmap cephfs:2/2 {0=cephfs.smithi088.ajgvpz=up:resolve,1=cephfs.smithi088.psjiah=up:resolve} 2 up:standby 2024-02-19T19:41:35.764 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:35 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4238837844' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:41:35.764 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:35.764 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:35.764 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:35 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4134707326' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:41:36.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:35 smithi088 ceph-mon[88643]: from='client.36048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:36.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:35 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] up:resolve 2024-02-19T19:41:36.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:35 smithi088 ceph-mon[88643]: fsmap cephfs:2/2 {0=cephfs.smithi088.ajgvpz=up:resolve,1=cephfs.smithi088.psjiah=up:resolve} 2 up:standby 2024-02-19T19:41:36.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:35 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4238837844' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:41:36.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:36.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:36.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:35 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4134707326' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:41:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: from='client.36062 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: pgmap v1143: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-02-19T19:41:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: reconnect by client.14562 192.168.0.1:0/3373415141 after 0.00199996 2024-02-19T19:41:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: reconnect by client.14562 192.168.0.1:0/3373415141 after 0.00199996 2024-02-19T19:41:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: reconnect by client.24305 192.168.0.1:0/1879662666 after 0.00199996 2024-02-19T19:41:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: reconnect by client.24305 192.168.0.1:0/1879662666 after 0.00199996 2024-02-19T19:41:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] up:reconnect 2024-02-19T19:41:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] up:reconnect 2024-02-19T19:41:36.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: fsmap cephfs:2/2 {0=cephfs.smithi088.ajgvpz=up:reconnect,1=cephfs.smithi088.psjiah=up:reconnect} 2 up:standby 2024-02-19T19:41:36.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:36.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:41:36.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:36 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: from='client.36062 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:41:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: pgmap v1143: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 8 op/s 2024-02-19T19:41:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: reconnect by client.14562 192.168.0.1:0/3373415141 after 0.00199996 2024-02-19T19:41:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: reconnect by client.14562 192.168.0.1:0/3373415141 after 0.00199996 2024-02-19T19:41:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: reconnect by client.24305 192.168.0.1:0/1879662666 after 0.00199996 2024-02-19T19:41:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: reconnect by client.24305 192.168.0.1:0/1879662666 after 0.00199996 2024-02-19T19:41:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] up:reconnect 2024-02-19T19:41:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] up:reconnect 2024-02-19T19:41:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: fsmap cephfs:2/2 {0=cephfs.smithi088.ajgvpz=up:reconnect,1=cephfs.smithi088.psjiah=up:reconnect} 2 up:standby 2024-02-19T19:41:37.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:37.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi132", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-02-19T19:41:37.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:36 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:37.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:37 smithi132 ceph-mon[70676]: Upgrade: Updating ceph-exporter.smithi132 (2/2) 2024-02-19T19:41:37.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:37 smithi132 ceph-mon[70676]: Deploying daemon ceph-exporter.smithi132 on smithi132 2024-02-19T19:41:37.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:37 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] up:rejoin 2024-02-19T19:41:37.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:37 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] up:rejoin 2024-02-19T19:41:37.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:37 smithi132 ceph-mon[70676]: fsmap cephfs:2/2 {0=cephfs.smithi088.ajgvpz=up:rejoin,1=cephfs.smithi088.psjiah=up:rejoin} 2 up:standby 2024-02-19T19:41:37.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:37 smithi132 ceph-mon[70676]: daemon mds.cephfs.smithi088.ajgvpz is now active in filesystem cephfs as rank 0 2024-02-19T19:41:37.738 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:37 smithi132 ceph-mon[70676]: daemon mds.cephfs.smithi088.psjiah is now active in filesystem cephfs as rank 1 2024-02-19T19:41:38.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:37 smithi088 ceph-mon[88643]: Upgrade: Updating ceph-exporter.smithi132 (2/2) 2024-02-19T19:41:38.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:37 smithi088 ceph-mon[88643]: Deploying daemon ceph-exporter.smithi132 on smithi132 2024-02-19T19:41:38.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:37 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] up:rejoin 2024-02-19T19:41:38.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:37 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] up:rejoin 2024-02-19T19:41:38.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:37 smithi088 ceph-mon[88643]: fsmap cephfs:2/2 {0=cephfs.smithi088.ajgvpz=up:rejoin,1=cephfs.smithi088.psjiah=up:rejoin} 2 up:standby 2024-02-19T19:41:38.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:37 smithi088 ceph-mon[88643]: daemon mds.cephfs.smithi088.ajgvpz is now active in filesystem cephfs as rank 0 2024-02-19T19:41:38.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:37 smithi088 ceph-mon[88643]: daemon mds.cephfs.smithi088.psjiah is now active in filesystem cephfs as rank 1 2024-02-19T19:41:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:38 smithi132 ceph-mon[70676]: pgmap v1144: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 7 op/s 2024-02-19T19:41:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:38 smithi132 ceph-mon[70676]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-19T19:41:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:38 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] up:active 2024-02-19T19:41:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:38 smithi132 ceph-mon[70676]: mds.? [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] up:active 2024-02-19T19:41:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:38 smithi132 ceph-mon[70676]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi088.psjiah=up:active} 2 up:standby 2024-02-19T19:41:39.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:38 smithi088 ceph-mon[88643]: pgmap v1144: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 7 op/s 2024-02-19T19:41:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:38 smithi088 ceph-mon[88643]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-19T19:41:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:38 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] up:active 2024-02-19T19:41:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:38 smithi088 ceph-mon[88643]: mds.? [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] up:active 2024-02-19T19:41:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:38 smithi088 ceph-mon[88643]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi088.psjiah=up:active} 2 up:standby 2024-02-19T19:41:40.908 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:40 smithi132 ceph-mon[70676]: pgmap v1145: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 85 B/s wr, 8 op/s 2024-02-19T19:41:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:40 smithi088 ceph-mon[88643]: pgmap v1145: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 85 B/s wr, 8 op/s 2024-02-19T19:41:42.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:42.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:42.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:41:42.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:42 smithi132 ceph-mon[70676]: pgmap v1146: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 9.2 KiB/s wr, 17 op/s 2024-02-19T19:41:42.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:42.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:42.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:42.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:42.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:42.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:42.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:41:42.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:42 smithi088 ceph-mon[88643]: pgmap v1146: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 9.2 KiB/s wr, 17 op/s 2024-02-19T19:41:42.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:42.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:42.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:42.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:44.522 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:44.522 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:41:44.522 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:44 smithi088 ceph-mon[88643]: pgmap v1147: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-02-19T19:41:44.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:44.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:41:44.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:44 smithi132 ceph-mon[70676]: pgmap v1147: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-02-19T19:41:45.872 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:45.872 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:45 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:45.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:45.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:45 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:46.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:46 smithi132 ceph-mon[70676]: pgmap v1148: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-02-19T19:41:46.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:46.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:46.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:46.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:46 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:47.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:46 smithi088 ceph-mon[88643]: pgmap v1148: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 MiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-02-19T19:41:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:46 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:48 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:48.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:48 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:48.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:48 smithi088 ceph-mon[88643]: pgmap v1149: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-02-19T19:41:48.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:48 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:48.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:48 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:48.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:48 smithi132 ceph-mon[70676]: pgmap v1149: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-02-19T19:41:50.673 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:50.673 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:50.673 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:50 smithi088 ceph-mon[88643]: pgmap v1150: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6.6 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-02-19T19:41:50.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:50.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:50.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:50 smithi132 ceph-mon[70676]: pgmap v1150: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6.6 KiB/s rd, 9.2 KiB/s wr, 10 op/s 2024-02-19T19:41:52.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:52.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:52.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:52.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:41:52.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: pgmap v1151: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.8 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2024-02-19T19:41:52.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:52.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:41:52.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:52.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:52.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:52.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:52.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:52.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: Upgrade: Setting filesystem cephfs Joinable 2024-02-19T19:41:52.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-19T19:41:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:41:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: pgmap v1151: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.8 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2024-02-19T19:41:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:41:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:52.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:52.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: Upgrade: Setting filesystem cephfs Joinable 2024-02-19T19:41:52.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-02-19T19:41:53.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-02-19T19:41:53.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: Cluster is now healthy 2024-02-19T19:41:53.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-19T19:41:53.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi088.psjiah=up:active} 2 up:standby 2024-02-19T19:41:53.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: Upgrade: Setting container_image for all ceph-exporter 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi088"}]: dispatch 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi088"}]': finished 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi132"}]: dispatch 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi132"}]': finished 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: Upgrade: Setting container_image for all iscsi 2024-02-19T19:41:53.485 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: Upgrade: Setting container_image for all nfs 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: Upgrade: Setting container_image for all nvmeof 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: Upgrade: Finalizing container_image settings 2024-02-19T19:41:53.486 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-19T19:41:53.487 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: Upgrade: Complete! 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:53.489 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:41:53.490 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:53.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-02-19T19:41:53.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: Cluster is now healthy 2024-02-19T19:41:53.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-02-19T19:41:53.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: fsmap cephfs:2 {0=cephfs.smithi088.ajgvpz=up:active,1=cephfs.smithi088.psjiah=up:active} 2 up:standby 2024-02-19T19:41:53.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: Upgrade: Setting container_image for all ceph-exporter 2024-02-19T19:41:53.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:53.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi088"}]: dispatch 2024-02-19T19:41:53.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi088"}]': finished 2024-02-19T19:41:53.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi132"}]: dispatch 2024-02-19T19:41:53.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi132"}]': finished 2024-02-19T19:41:53.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: Upgrade: Setting container_image for all iscsi 2024-02-19T19:41:53.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:53.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: Upgrade: Setting container_image for all nfs 2024-02-19T19:41:53.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:53.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: Upgrade: Setting container_image for all nvmeof 2024-02-19T19:41:53.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:53.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: Upgrade: Finalizing container_image settings 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-02-19T19:41:53.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-02-19T19:41:53.585 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: Upgrade: Complete! 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:41:53.586 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:53.587 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:41:53.587 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:53.587 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:41:53.587 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:41:53.587 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:41:53.587 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:54.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:54 smithi132 ceph-mon[70676]: pgmap v1152: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:54.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:54 smithi088 ceph-mon[88643]: pgmap v1152: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:56.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:56 smithi132 ceph-mon[70676]: pgmap v1153: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:57.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:56 smithi088 ceph-mon[88643]: pgmap v1153: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:58.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:58 smithi132 ceph-mon[70676]: pgmap v1154: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:58.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:58.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:41:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:41:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:58 smithi088 ceph-mon[88643]: pgmap v1154: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:41:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:41:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:41:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:42:00.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:00 smithi132 ceph-mon[70676]: pgmap v1155: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:01.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:00 smithi088 ceph-mon[88643]: pgmap v1155: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:02.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:02 smithi132 ceph-mon[70676]: pgmap v1156: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:03.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:02 smithi088 ceph-mon[88643]: pgmap v1156: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:04 smithi132 ceph-mon[70676]: pgmap v1157: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:04 smithi088 ceph-mon[88643]: pgmap v1157: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:05.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.736+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1324787911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e980ffde0 msgr2=0x7f6e98103db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:05.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.736+0000 7f6ea04a9700 1 --2- 172.21.15.88:0/1324787911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e980ffde0 0x7f6e98103db0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009910 tx=0x7f6e94009c20 comp rx=0 tx=0).stop 2024-02-19T19:42:05.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.736+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1324787911 shutdown_connections 2024-02-19T19:42:05.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.736+0000 7f6ea04a9700 1 --2- 172.21.15.88:0/1324787911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e980ffde0 0x7f6e98103db0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:05.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.736+0000 7f6ea04a9700 1 --2- 172.21.15.88:0/1324787911 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e980ff440 0x7f6e980ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:05.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.736+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1324787911 >> 172.21.15.88:0/1324787911 conn(0x7f6e980face0 msgr2=0x7f6e980fd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:42:05.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.736+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1324787911 shutdown_connections 2024-02-19T19:42:05.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.736+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1324787911 wait complete. 2024-02-19T19:42:05.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.736+0000 7f6ea04a9700 1 Processor -- start 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.736+0000 7f6ea04a9700 1 -- start start 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6ea04a9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e980ff440 0x7f6e98196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6ea04a9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e980ffde0 0x7f6e981971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6ea04a9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e981978b0 con 0x7f6e980ff440 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6ea04a9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e981979f0 con 0x7f6e980ffde0 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6e9e245700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e980ff440 0x7f6e98196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6e9e245700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e980ff440 0x7f6e98196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53474/0 (socket says 172.21.15.88:53474) 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6e9e245700 1 -- 172.21.15.88:0/1664700947 learned_addr learned my addr 172.21.15.88:0/1664700947 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6e9da44700 1 --2- 172.21.15.88:0/1664700947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e980ffde0 0x7f6e981971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6e9e245700 1 -- 172.21.15.88:0/1664700947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e980ffde0 msgr2=0x7f6e981971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6e9e245700 1 --2- 172.21.15.88:0/1664700947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e980ffde0 0x7f6e981971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6e9e245700 1 -- 172.21.15.88:0/1664700947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e940095f0 con 0x7f6e980ff440 2024-02-19T19:42:05.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6e9e245700 1 --2- 172.21.15.88:0/1664700947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e980ff440 0x7f6e98196c90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800e920 tx=0x7f6e8800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:42:05.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6e8f7fe700 1 -- 172.21.15.88:0/1664700947 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e88009790 con 0x7f6e980ff440 2024-02-19T19:42:05.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1664700947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9819fd00 con 0x7f6e980ff440 2024-02-19T19:42:05.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6e8f7fe700 1 -- 172.21.15.88:0/1664700947 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e8800cba0 con 0x7f6e980ff440 2024-02-19T19:42:05.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6e8f7fe700 1 -- 172.21.15.88:0/1664700947 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e88018650 con 0x7f6e980ff440 2024-02-19T19:42:05.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.737+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1664700947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e981a0220 con 0x7f6e980ff440 2024-02-19T19:42:05.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.738+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1664700947 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e981077b0 con 0x7f6e980ff440 2024-02-19T19:42:05.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.739+0000 7f6e8f7fe700 1 -- 172.21.15.88:0/1664700947 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6e8800cd10 con 0x7f6e980ff440 2024-02-19T19:42:05.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.739+0000 7f6e8f7fe700 1 --2- 172.21.15.88:0/1664700947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8406d8a0 0x7f6e8406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:05.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.739+0000 7f6e8f7fe700 1 -- 172.21.15.88:0/1664700947 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6e88014070 con 0x7f6e980ff440 2024-02-19T19:42:05.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.739+0000 7f6e9da44700 1 --2- 172.21.15.88:0/1664700947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8406d8a0 0x7f6e8406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:05.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.740+0000 7f6e9da44700 1 --2- 172.21.15.88:0/1664700947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8406d8a0 0x7f6e8406fd50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009fd0 tx=0x7f6e94005940 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:42:05.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.744+0000 7f6e8f7fe700 1 -- 172.21.15.88:0/1664700947 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6e88052d20 con 0x7f6e980ff440 2024-02-19T19:42:05.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.959+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1664700947 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e98061190 con 0x7f6e8406d8a0 2024-02-19T19:42:05.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.960+0000 7f6e8f7fe700 1 -- 172.21.15.88:0/1664700947 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f6e98061190 con 0x7f6e8406d8a0 2024-02-19T19:42:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.962+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1664700947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8406d8a0 msgr2=0x7f6e8406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.962+0000 7f6ea04a9700 1 --2- 172.21.15.88:0/1664700947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8406d8a0 0x7f6e8406fd50 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009fd0 tx=0x7f6e94005940 comp rx=0 tx=0).stop 2024-02-19T19:42:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.962+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1664700947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e980ff440 msgr2=0x7f6e98196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.962+0000 7f6ea04a9700 1 --2- 172.21.15.88:0/1664700947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e980ff440 0x7f6e98196c90 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800e920 tx=0x7f6e8800ece0 comp rx=0 tx=0).stop 2024-02-19T19:42:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.962+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1664700947 shutdown_connections 2024-02-19T19:42:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.962+0000 7f6ea04a9700 1 --2- 172.21.15.88:0/1664700947 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f6e8406d8a0 0x7f6e8406fd50 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.962+0000 7f6ea04a9700 1 --2- 172.21.15.88:0/1664700947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e980ff440 0x7f6e98196c90 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.962+0000 7f6ea04a9700 1 --2- 172.21.15.88:0/1664700947 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f6e980ffde0 0x7f6e981971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.962+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1664700947 >> 172.21.15.88:0/1664700947 conn(0x7f6e980face0 msgr2=0x7f6e980fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:42:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.962+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1664700947 shutdown_connections 2024-02-19T19:42:05.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:05.962+0000 7f6ea04a9700 1 -- 172.21.15.88:0/1664700947 wait complete. 2024-02-19T19:42:06.598 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-19T19:42:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:06 smithi088 ceph-mon[88643]: pgmap v1158: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:06.845 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:42:06.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:06 smithi132 ceph-mon[70676]: pgmap v1158: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:07.909 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:07 smithi088 ceph-mon[88643]: from='client.36072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:42:07.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:07 smithi132 ceph-mon[70676]: from='client.36072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:42:08.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.387+0000 7fa3cefe9700 1 -- 172.21.15.88:0/3776418016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c81004e0 msgr2=0x7fa3c81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:08.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.387+0000 7fa3cefe9700 1 --2- 172.21.15.88:0/3776418016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c81004e0 0x7fa3c81008b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009910 tx=0x7fa3bc009c20 comp rx=0 tx=0).stop 2024-02-19T19:42:08.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.387+0000 7fa3cefe9700 1 -- 172.21.15.88:0/3776418016 shutdown_connections 2024-02-19T19:42:08.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.387+0000 7fa3cefe9700 1 --2- 172.21.15.88:0/3776418016 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8100e80 0x7fa3c8104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:08.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.387+0000 7fa3cefe9700 1 --2- 172.21.15.88:0/3776418016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c81004e0 0x7fa3c81008b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:08.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.387+0000 7fa3cefe9700 1 -- 172.21.15.88:0/3776418016 >> 172.21.15.88:0/3776418016 conn(0x7fa3c80fbda0 msgr2=0x7fa3c80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:42:08.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.387+0000 7fa3cefe9700 1 -- 172.21.15.88:0/3776418016 shutdown_connections 2024-02-19T19:42:08.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.387+0000 7fa3cefe9700 1 -- 172.21.15.88:0/3776418016 wait complete. 2024-02-19T19:42:08.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.388+0000 7fa3cefe9700 1 Processor -- start 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.388+0000 7fa3cefe9700 1 -- start start 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.388+0000 7fa3cefe9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c81004e0 0x7fa3c8195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.388+0000 7fa3cefe9700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8100e80 0x7fa3c8196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.388+0000 7fa3cefe9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8196a80 con 0x7fa3c81004e0 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.388+0000 7fa3cefe9700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c8196bc0 con 0x7fa3c8100e80 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.388+0000 7fa3ccd85700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c81004e0 0x7fa3c8195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.388+0000 7fa3ccd85700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c81004e0 0x7fa3c8195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53496/0 (socket says 172.21.15.88:53496) 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.388+0000 7fa3ccd85700 1 -- 172.21.15.88:0/2553736480 learned_addr learned my addr 172.21.15.88:0/2553736480 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.388+0000 7fa3c7fff700 1 --2- 172.21.15.88:0/2553736480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8100e80 0x7fa3c8196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.389+0000 7fa3ccd85700 1 -- 172.21.15.88:0/2553736480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8100e80 msgr2=0x7fa3c8196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.389+0000 7fa3ccd85700 1 --2- 172.21.15.88:0/2553736480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8100e80 0x7fa3c8196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:08.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.389+0000 7fa3ccd85700 1 -- 172.21.15.88:0/2553736480 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc0095f0 con 0x7fa3c81004e0 2024-02-19T19:42:08.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.389+0000 7fa3ccd85700 1 --2- 172.21.15.88:0/2553736480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c81004e0 0x7fa3c8195ef0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc006010 tx=0x7fa3bc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:42:08.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.389+0000 7fa3c5ffb700 1 -- 172.21.15.88:0/2553736480 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3bc01d070 con 0x7fa3c81004e0 2024-02-19T19:42:08.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.389+0000 7fa3c5ffb700 1 -- 172.21.15.88:0/2553736480 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3bc00f460 con 0x7fa3c81004e0 2024-02-19T19:42:08.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.389+0000 7fa3c5ffb700 1 -- 172.21.15.88:0/2553736480 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3bc017630 con 0x7fa3c81004e0 2024-02-19T19:42:08.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.389+0000 7fa3cefe9700 1 -- 172.21.15.88:0/2553736480 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c819a9b0 con 0x7fa3c81004e0 2024-02-19T19:42:08.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.389+0000 7fa3cefe9700 1 -- 172.21.15.88:0/2553736480 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c819aea0 con 0x7fa3c81004e0 2024-02-19T19:42:08.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.390+0000 7fa3cefe9700 1 -- 172.21.15.88:0/2553736480 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c81087f0 con 0x7fa3c81004e0 2024-02-19T19:42:08.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.391+0000 7fa3c5ffb700 1 -- 172.21.15.88:0/2553736480 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa3bc00f5d0 con 0x7fa3c81004e0 2024-02-19T19:42:08.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.391+0000 7fa3c5ffb700 1 --2- 172.21.15.88:0/2553736480 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3b806d8a0 0x7fa3b806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:08.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.391+0000 7fa3c5ffb700 1 -- 172.21.15.88:0/2553736480 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa3bc08e9a0 con 0x7fa3c81004e0 2024-02-19T19:42:08.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.391+0000 7fa3c7fff700 1 --2- 172.21.15.88:0/2553736480 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3b806d8a0 0x7fa3b806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:08.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.392+0000 7fa3c7fff700 1 --2- 172.21.15.88:0/2553736480 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3b806d8a0 0x7fa3b806fd50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8197510 tx=0x7fa3b400a380 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:42:08.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.396+0000 7fa3c5ffb700 1 -- 172.21.15.88:0/2553736480 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa3bc0581d0 con 0x7fa3c81004e0 2024-02-19T19:42:08.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.606+0000 7fa3cefe9700 1 -- 172.21.15.88:0/2553736480 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3c819b110 con 0x7fa3b806d8a0 2024-02-19T19:42:08.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.617+0000 7fa3c5ffb700 1 -- 172.21.15.88:0/2553736480 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fa3c819b110 con 0x7fa3b806d8a0 2024-02-19T19:42:08.617 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (39m) 22s ago 49m 23.6M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (36s) 22s ago 50m 7386k - 19.0.0-1450-gf78a58c0 f081f847070a 9e949ec33fbf 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (27s) 24s ago 48m 6958k - 19.0.0-1450-gf78a58c0 f081f847070a a4682d5b635b 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (35m) 22s ago 50m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (35m) 24s ago 48m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (38m) 22s ago 49m 86.2M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (31m) 22s ago 43m 70.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (31m) 22s ago 43m 63.9M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (30m) 24s ago 43m 23.9M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (15m) 24s ago 43m 20.2M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (40m) 22s ago 51m 516M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:42:08.618 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (40m) 24s ago 48m 459M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:42:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (37m) 22s ago 51m 88.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:42:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (36m) 24s ago 47m 61.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:42:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (40m) 22s ago 50m 16.1M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:42:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (39m) 24s ago 48m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:42:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (34m) 22s ago 47m 137M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:42:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (34m) 22s ago 46m 115M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:42:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (33m) 22s ago 46m 97.3M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:42:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (32m) 24s ago 45m 120M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:42:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (32m) 24s ago 45m 112M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:42:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (31m) 24s ago 45m 96.0M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:42:08.619 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (39m) 22s ago 48m 90.8M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:42:08.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.619+0000 7fa3cefe9700 1 -- 172.21.15.88:0/2553736480 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3b806d8a0 msgr2=0x7fa3b806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:08.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.619+0000 7fa3cefe9700 1 --2- 172.21.15.88:0/2553736480 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3b806d8a0 0x7fa3b806fd50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8197510 tx=0x7fa3b400a380 comp rx=0 tx=0).stop 2024-02-19T19:42:08.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.619+0000 7fa3cefe9700 1 -- 172.21.15.88:0/2553736480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c81004e0 msgr2=0x7fa3c8195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:08.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.619+0000 7fa3cefe9700 1 --2- 172.21.15.88:0/2553736480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c81004e0 0x7fa3c8195ef0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc006010 tx=0x7fa3bc00bd70 comp rx=0 tx=0).stop 2024-02-19T19:42:08.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.619+0000 7fa3cefe9700 1 -- 172.21.15.88:0/2553736480 shutdown_connections 2024-02-19T19:42:08.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.619+0000 7fa3cefe9700 1 --2- 172.21.15.88:0/2553736480 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3b806d8a0 0x7fa3b806fd50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:08.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.619+0000 7fa3cefe9700 1 --2- 172.21.15.88:0/2553736480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c81004e0 0x7fa3c8195ef0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:08.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.619+0000 7fa3cefe9700 1 --2- 172.21.15.88:0/2553736480 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3c8100e80 0x7fa3c8196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:08.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.619+0000 7fa3cefe9700 1 -- 172.21.15.88:0/2553736480 >> 172.21.15.88:0/2553736480 conn(0x7fa3c80fbda0 msgr2=0x7fa3c80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:42:08.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.620+0000 7fa3cefe9700 1 -- 172.21.15.88:0/2553736480 shutdown_connections 2024-02-19T19:42:08.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:08.620+0000 7fa3cefe9700 1 -- 172.21.15.88:0/2553736480 wait complete. 2024-02-19T19:42:08.975 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:08 smithi088 ceph-mon[88643]: pgmap v1159: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:08.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:08 smithi132 ceph-mon[70676]: pgmap v1159: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:09.296 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch upgrade status' 2024-02-19T19:42:09.513 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:42:09.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:09 smithi088 ceph-mon[88643]: from='client.36076 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:42:09.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:09 smithi132 ceph-mon[70676]: from='client.36076 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:42:10.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:10 smithi088 ceph-mon[88643]: pgmap v1160: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:10.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:10 smithi132 ceph-mon[70676]: pgmap v1160: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:11.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.087+0000 7f8091a74700 1 -- 172.21.15.88:0/599505955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f808c101d30 msgr2=0x7f808c10a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:11.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.087+0000 7f8091a74700 1 --2- 172.21.15.88:0/599505955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f808c101d30 0x7f808c10a230 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f8080009910 tx=0x7f8080009c20 comp rx=0 tx=0).stop 2024-02-19T19:42:11.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.087+0000 7f8091a74700 1 -- 172.21.15.88:0/599505955 shutdown_connections 2024-02-19T19:42:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.087+0000 7f8091a74700 1 --2- 172.21.15.88:0/599505955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f808c101d30 0x7f808c10a230 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.087+0000 7f8091a74700 1 --2- 172.21.15.88:0/599505955 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c101420 0x7f808c1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.087+0000 7f8091a74700 1 -- 172.21.15.88:0/599505955 >> 172.21.15.88:0/599505955 conn(0x7f808c0fac80 msgr2=0x7f808c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:42:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.087+0000 7f8091a74700 1 -- 172.21.15.88:0/599505955 shutdown_connections 2024-02-19T19:42:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.088+0000 7f8091a74700 1 -- 172.21.15.88:0/599505955 wait complete. 2024-02-19T19:42:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.088+0000 7f8091a74700 1 Processor -- start 2024-02-19T19:42:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.088+0000 7f8091a74700 1 -- start start 2024-02-19T19:42:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.088+0000 7f8091a74700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c101420 0x7f808c0ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.088+0000 7f8091a74700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f808c101d30 0x7f808c0ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.088+0000 7f8091a74700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0ffb00 con 0x7f808c101d30 2024-02-19T19:42:11.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.088+0000 7f8091a74700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0ffc40 con 0x7f808c101420 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f808affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f808c101d30 0x7f808c0ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f808affd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f808c101d30 0x7f808c0ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53512/0 (socket says 172.21.15.88:53512) 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f808affd700 1 -- 172.21.15.88:0/4287190907 learned_addr learned my addr 172.21.15.88:0/4287190907 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f808b7fe700 1 --2- 172.21.15.88:0/4287190907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c101420 0x7f808c0ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f808affd700 1 -- 172.21.15.88:0/4287190907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c101420 msgr2=0x7f808c0ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f808affd700 1 --2- 172.21.15.88:0/4287190907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c101420 0x7f808c0ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f808affd700 1 -- 172.21.15.88:0/4287190907 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80800095f0 con 0x7f808c101d30 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f808affd700 1 --2- 172.21.15.88:0/4287190907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f808c101d30 0x7f808c0ff5c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8080006010 tx=0x7f808000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f8088ff9700 1 -- 172.21.15.88:0/4287190907 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f808001d070 con 0x7f808c101d30 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f8088ff9700 1 -- 172.21.15.88:0/4287190907 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f808000f460 con 0x7f808c101d30 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f8088ff9700 1 -- 172.21.15.88:0/4287190907 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8080017610 con 0x7f808c101d30 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f8091a74700 1 -- 172.21.15.88:0/4287190907 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c0ffec0 con 0x7f808c101d30 2024-02-19T19:42:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.089+0000 7f8091a74700 1 -- 172.21.15.88:0/4287190907 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c1a24f0 con 0x7f808c101d30 2024-02-19T19:42:11.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.090+0000 7f8091a74700 1 -- 172.21.15.88:0/4287190907 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808c1078a0 con 0x7f808c101d30 2024-02-19T19:42:11.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.091+0000 7f8088ff9700 1 -- 172.21.15.88:0/4287190907 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f808000f5d0 con 0x7f808c101d30 2024-02-19T19:42:11.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.091+0000 7f8088ff9700 1 --2- 172.21.15.88:0/4287190907 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f807406d8a0 0x7f807406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:11.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.091+0000 7f8088ff9700 1 -- 172.21.15.88:0/4287190907 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f808008eaa0 con 0x7f808c101d30 2024-02-19T19:42:11.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.091+0000 7f808b7fe700 1 --2- 172.21.15.88:0/4287190907 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f807406d8a0 0x7f807406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:11.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.092+0000 7f808b7fe700 1 --2- 172.21.15.88:0/4287190907 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f807406d8a0 0x7f807406fd50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f807c009a10 tx=0x7f807c009190 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:42:11.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.096+0000 7f8088ff9700 1 -- 172.21.15.88:0/4287190907 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8080058250 con 0x7f808c101d30 2024-02-19T19:42:11.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.311+0000 7f8091a74700 1 -- 172.21.15.88:0/4287190907 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f808c1006c0 con 0x7f807406d8a0 2024-02-19T19:42:11.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.312+0000 7f8088ff9700 1 -- 172.21.15.88:0/4287190907 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f808c1006c0 con 0x7f807406d8a0 2024-02-19T19:42:11.313 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:42:11.313 INFO:teuthology.orchestra.run.smithi088.stdout: "target_image": null, 2024-02-19T19:42:11.313 INFO:teuthology.orchestra.run.smithi088.stdout: "in_progress": false, 2024-02-19T19:42:11.313 INFO:teuthology.orchestra.run.smithi088.stdout: "which": "", 2024-02-19T19:42:11.313 INFO:teuthology.orchestra.run.smithi088.stdout: "services_complete": [], 2024-02-19T19:42:11.313 INFO:teuthology.orchestra.run.smithi088.stdout: "progress": null, 2024-02-19T19:42:11.313 INFO:teuthology.orchestra.run.smithi088.stdout: "message": "", 2024-02-19T19:42:11.313 INFO:teuthology.orchestra.run.smithi088.stdout: "is_paused": false 2024-02-19T19:42:11.313 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:42:11.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.314+0000 7f8091a74700 1 -- 172.21.15.88:0/4287190907 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f807406d8a0 msgr2=0x7f807406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:11.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.314+0000 7f8091a74700 1 --2- 172.21.15.88:0/4287190907 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f807406d8a0 0x7f807406fd50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f807c009a10 tx=0x7f807c009190 comp rx=0 tx=0).stop 2024-02-19T19:42:11.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.314+0000 7f8091a74700 1 -- 172.21.15.88:0/4287190907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f808c101d30 msgr2=0x7f808c0ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:11.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.314+0000 7f8091a74700 1 --2- 172.21.15.88:0/4287190907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f808c101d30 0x7f808c0ff5c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f8080006010 tx=0x7f808000bd70 comp rx=0 tx=0).stop 2024-02-19T19:42:11.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.314+0000 7f8091a74700 1 -- 172.21.15.88:0/4287190907 shutdown_connections 2024-02-19T19:42:11.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.314+0000 7f8091a74700 1 --2- 172.21.15.88:0/4287190907 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f807406d8a0 0x7f807406fd50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:11.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.314+0000 7f8091a74700 1 --2- 172.21.15.88:0/4287190907 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f808c101420 0x7f808c0ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:11.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.314+0000 7f8091a74700 1 --2- 172.21.15.88:0/4287190907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f808c101d30 0x7f808c0ff5c0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:11.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.314+0000 7f8091a74700 1 -- 172.21.15.88:0/4287190907 >> 172.21.15.88:0/4287190907 conn(0x7f808c0fac80 msgr2=0x7f808c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:42:11.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.314+0000 7f8091a74700 1 -- 172.21.15.88:0/4287190907 shutdown_connections 2024-02-19T19:42:11.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:11.315+0000 7f8091a74700 1 -- 172.21.15.88:0/4287190907 wait complete. 2024-02-19T19:42:11.969 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph health detail' 2024-02-19T19:42:12.178 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:42:12.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:12 smithi088 ceph-mon[88643]: pgmap v1161: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:12.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:12 smithi088 ceph-mon[88643]: from='client.36080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:42:12.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:12 smithi132 ceph-mon[70676]: pgmap v1161: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:12.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:12 smithi132 ceph-mon[70676]: from='client.36080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:42:13.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.869+0000 7fa2a250a700 1 -- 172.21.15.88:0/2386113043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c100e60 msgr2=0x7fa29c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:13.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.869+0000 7fa2a250a700 1 --2- 172.21.15.88:0/2386113043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c100e60 0x7fa29c104e30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fa290009910 tx=0x7fa290009c20 comp rx=0 tx=0).stop 2024-02-19T19:42:13.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.869+0000 7fa2a250a700 1 -- 172.21.15.88:0/2386113043 shutdown_connections 2024-02-19T19:42:13.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.869+0000 7fa2a250a700 1 --2- 172.21.15.88:0/2386113043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c100e60 0x7fa29c104e30 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:13.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.869+0000 7fa2a250a700 1 --2- 172.21.15.88:0/2386113043 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa29c1004c0 0x7fa29c100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:13.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.869+0000 7fa2a250a700 1 -- 172.21.15.88:0/2386113043 >> 172.21.15.88:0/2386113043 conn(0x7fa29c0fbd80 msgr2=0x7fa29c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.869+0000 7fa2a250a700 1 -- 172.21.15.88:0/2386113043 shutdown_connections 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.869+0000 7fa2a250a700 1 -- 172.21.15.88:0/2386113043 wait complete. 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.870+0000 7fa2a250a700 1 Processor -- start 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.870+0000 7fa2a250a700 1 -- start start 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.870+0000 7fa2a250a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c1004c0 0x7fa29c196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.870+0000 7fa2a250a700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa29c100e60 0x7fa29c196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.870+0000 7fa2a250a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c196c50 con 0x7fa29c1004c0 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.870+0000 7fa2a250a700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c19a9e0 con 0x7fa29c100e60 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.870+0000 7fa29bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c1004c0 0x7fa29c196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.870+0000 7fa29bfff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c1004c0 0x7fa29c196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53540/0 (socket says 172.21.15.88:53540) 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.870+0000 7fa29bfff700 1 -- 172.21.15.88:0/2825457852 learned_addr learned my addr 172.21.15.88:0/2825457852 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.870+0000 7fa29b7fe700 1 --2- 172.21.15.88:0/2825457852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa29c100e60 0x7fa29c196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:13.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.871+0000 7fa29bfff700 1 -- 172.21.15.88:0/2825457852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa29c100e60 msgr2=0x7fa29c196570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:13.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.871+0000 7fa29bfff700 1 --2- 172.21.15.88:0/2825457852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa29c100e60 0x7fa29c196570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:13.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.871+0000 7fa29bfff700 1 -- 172.21.15.88:0/2825457852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2900095f0 con 0x7fa29c1004c0 2024-02-19T19:42:13.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.871+0000 7fa29bfff700 1 --2- 172.21.15.88:0/2825457852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c1004c0 0x7fa29c196030 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00b4b0 tx=0x7fa28c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:42:13.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.871+0000 7fa2997fa700 1 -- 172.21.15.88:0/2825457852 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa28c011800 con 0x7fa29c1004c0 2024-02-19T19:42:13.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.871+0000 7fa2997fa700 1 -- 172.21.15.88:0/2825457852 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa28c011e20 con 0x7fa29c1004c0 2024-02-19T19:42:13.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.871+0000 7fa2a250a700 1 -- 172.21.15.88:0/2825457852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29c19acc0 con 0x7fa29c1004c0 2024-02-19T19:42:13.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.871+0000 7fa2997fa700 1 -- 172.21.15.88:0/2825457852 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa28c00f2e0 con 0x7fa29c1004c0 2024-02-19T19:42:13.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.871+0000 7fa2a250a700 1 -- 172.21.15.88:0/2825457852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29c19b210 con 0x7fa29c1004c0 2024-02-19T19:42:13.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.872+0000 7fa2a250a700 1 -- 172.21.15.88:0/2825457852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29c108760 con 0x7fa29c1004c0 2024-02-19T19:42:13.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.872+0000 7fa2997fa700 1 -- 172.21.15.88:0/2825457852 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa28c010380 con 0x7fa29c1004c0 2024-02-19T19:42:13.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.873+0000 7fa2997fa700 1 --2- 172.21.15.88:0/2825457852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa28406d8a0 0x7fa28406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:13.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.873+0000 7fa2997fa700 1 -- 172.21.15.88:0/2825457852 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa28c08cfa0 con 0x7fa29c1004c0 2024-02-19T19:42:13.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.873+0000 7fa29b7fe700 1 --2- 172.21.15.88:0/2825457852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa28406d8a0 0x7fa28406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:13.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.873+0000 7fa29b7fe700 1 --2- 172.21.15.88:0/2825457852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa28406d8a0 0x7fa28406fd50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa29c197650 tx=0x7fa29001a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:42:13.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:13.877+0000 7fa2997fa700 1 -- 172.21.15.88:0/2825457852 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa28c0567d0 con 0x7fa29c1004c0 2024-02-19T19:42:13.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:42:14.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:42:14.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.164+0000 7fa2a250a700 1 -- 172.21.15.88:0/2825457852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa29c04f2a0 con 0x7fa29c1004c0 2024-02-19T19:42:14.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.165+0000 7fa2997fa700 1 -- 172.21.15.88:0/2825457852 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa28c059e90 con 0x7fa29c1004c0 2024-02-19T19:42:14.165 INFO:teuthology.orchestra.run.smithi088.stdout:HEALTH_OK 2024-02-19T19:42:14.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.166+0000 7fa2a250a700 1 -- 172.21.15.88:0/2825457852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa28406d8a0 msgr2=0x7fa28406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:14.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.166+0000 7fa2a250a700 1 --2- 172.21.15.88:0/2825457852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa28406d8a0 0x7fa28406fd50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa29c197650 tx=0x7fa29001a040 comp rx=0 tx=0).stop 2024-02-19T19:42:14.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.167+0000 7fa2a250a700 1 -- 172.21.15.88:0/2825457852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c1004c0 msgr2=0x7fa29c196030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:14.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.167+0000 7fa2a250a700 1 --2- 172.21.15.88:0/2825457852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c1004c0 0x7fa29c196030 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fa28c00b4b0 tx=0x7fa28c00b7c0 comp rx=0 tx=0).stop 2024-02-19T19:42:14.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.167+0000 7fa2a250a700 1 -- 172.21.15.88:0/2825457852 shutdown_connections 2024-02-19T19:42:14.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.167+0000 7fa2a250a700 1 --2- 172.21.15.88:0/2825457852 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa28406d8a0 0x7fa28406fd50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:14.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.167+0000 7fa2a250a700 1 --2- 172.21.15.88:0/2825457852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa29c1004c0 0x7fa29c196030 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:14.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.167+0000 7fa2a250a700 1 --2- 172.21.15.88:0/2825457852 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa29c100e60 0x7fa29c196570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:14.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.167+0000 7fa2a250a700 1 -- 172.21.15.88:0/2825457852 >> 172.21.15.88:0/2825457852 conn(0x7fa29c0fbd80 msgr2=0x7fa29c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:42:14.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.167+0000 7fa2a250a700 1 -- 172.21.15.88:0/2825457852 shutdown_connections 2024-02-19T19:42:14.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:14.167+0000 7fa2a250a700 1 -- 172.21.15.88:0/2825457852 wait complete. 2024-02-19T19:42:14.780 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:14 smithi088 ceph-mon[88643]: pgmap v1162: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:14.780 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:14 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2825457852' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:42:14.800 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions' 2024-02-19T19:42:14.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:14 smithi132 ceph-mon[70676]: pgmap v1162: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:14.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:14 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2825457852' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-02-19T19:42:15.007 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:42:16.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.655+0000 7fcdba128700 1 -- 172.21.15.88:0/1939784065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb4104130 msgr2=0x7fcdb4104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.655+0000 7fcdba128700 1 --2- 172.21.15.88:0/1939784065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb4104130 0x7fcdb4104500 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fcdac00b0a0 tx=0x7fcdac00b3b0 comp rx=0 tx=0).stop 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.655+0000 7fcdba128700 1 -- 172.21.15.88:0/1939784065 shutdown_connections 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.655+0000 7fcdba128700 1 --2- 172.21.15.88:0/1939784065 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb40ffbd0 0x7fcdb4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.655+0000 7fcdba128700 1 --2- 172.21.15.88:0/1939784065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb4104130 0x7fcdb4104500 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.655+0000 7fcdba128700 1 -- 172.21.15.88:0/1939784065 >> 172.21.15.88:0/1939784065 conn(0x7fcdb4075130 msgr2=0x7fcdb4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.655+0000 7fcdba128700 1 -- 172.21.15.88:0/1939784065 shutdown_connections 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.655+0000 7fcdba128700 1 -- 172.21.15.88:0/1939784065 wait complete. 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.655+0000 7fcdba128700 1 Processor -- start 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdba128700 1 -- start start 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdba128700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb40ffbd0 0x7fcdb4194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdba128700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb4104130 0x7fcdb4195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdba128700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb4195960 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdba128700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb41996f0 con 0x7fcdb4104130 2024-02-19T19:42:16.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdb37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb40ffbd0 0x7fcdb4194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdb37fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb40ffbd0 0x7fcdb4194d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35878/0 (socket says 172.21.15.88:35878) 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdb37fe700 1 -- 172.21.15.88:0/1813854390 learned_addr learned my addr 172.21.15.88:0/1813854390 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdb37fe700 1 -- 172.21.15.88:0/1813854390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb4104130 msgr2=0x7fcdb4195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdb2ffd700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb4104130 0x7fcdb4195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdb37fe700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb4104130 0x7fcdb4195280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdb37fe700 1 -- 172.21.15.88:0/1813854390 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac009d00 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.656+0000 7fcdb2ffd700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb4104130 0x7fcdb4195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.657+0000 7fcdb37fe700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb40ffbd0 0x7fcdb4194d40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000f80 tx=0x7fcdac00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.657+0000 7fcdb0ff9700 1 -- 172.21.15.88:0/1813854390 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdac00e040 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.657+0000 7fcdb0ff9700 1 -- 172.21.15.88:0/1813854390 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcdac003e60 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.657+0000 7fcdba128700 1 -- 172.21.15.88:0/1813854390 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb4199970 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.657+0000 7fcdb0ff9700 1 -- 172.21.15.88:0/1813854390 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdac01d910 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.657+0000 7fcdba128700 1 -- 172.21.15.88:0/1813854390 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb4199e60 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.658+0000 7fcdba128700 1 -- 172.21.15.88:0/1813854390 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb41076a0 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.658+0000 7fcdb0ff9700 1 -- 172.21.15.88:0/1813854390 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcdac004820 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.659+0000 7fcdb0ff9700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd9c06d8a0 0x7fcd9c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:42:16.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.659+0000 7fcdb0ff9700 1 -- 172.21.15.88:0/1813854390 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcdac08ee10 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.659+0000 7fcdb2ffd700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd9c06d8a0 0x7fcd9c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:42:16.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.659+0000 7fcdb2ffd700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd9c06d8a0 0x7fcd9c06fd50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcda8005fd0 tx=0x7fcda8005dc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:42:16.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.664+0000 7fcdb0ff9700 1 -- 172.21.15.88:0/1813854390 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcdac0585c0 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:16 smithi088 ceph-mon[88643]: pgmap v1163: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:16.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.947+0000 7fcdba128700 1 -- 172.21.15.88:0/1813854390 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcdb4107080 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.948+0000 7fcdb0ff9700 1 -- 172.21.15.88:0/1813854390 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fcdac05bc80 con 0x7fcdb40ffbd0 2024-02-19T19:42:16.949 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:42:16.949 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:42:16.949 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:42:16.949 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:42:16.950 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:42:16.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.950+0000 7fcdba128700 1 -- 172.21.15.88:0/1813854390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd9c06d8a0 msgr2=0x7fcd9c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:16.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.950+0000 7fcdba128700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd9c06d8a0 0x7fcd9c06fd50 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcda8005fd0 tx=0x7fcda8005dc0 comp rx=0 tx=0).stop 2024-02-19T19:42:16.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.950+0000 7fcdba128700 1 -- 172.21.15.88:0/1813854390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb40ffbd0 msgr2=0x7fcdb4194d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:42:16.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.950+0000 7fcdba128700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb40ffbd0 0x7fcdb4194d40 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fcdac000f80 tx=0x7fcdac00cdd0 comp rx=0 tx=0).stop 2024-02-19T19:42:16.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.950+0000 7fcdba128700 1 -- 172.21.15.88:0/1813854390 shutdown_connections 2024-02-19T19:42:16.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.950+0000 7fcdba128700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcd9c06d8a0 0x7fcd9c06fd50 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:16.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.950+0000 7fcdba128700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdb40ffbd0 0x7fcdb4194d40 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:16.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.950+0000 7fcdba128700 1 --2- 172.21.15.88:0/1813854390 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcdb4104130 0x7fcdb4195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:42:16.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.950+0000 7fcdba128700 1 -- 172.21.15.88:0/1813854390 >> 172.21.15.88:0/1813854390 conn(0x7fcdb4075130 msgr2=0x7fcdb40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:42:16.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.950+0000 7fcdba128700 1 -- 172.21.15.88:0/1813854390 shutdown_connections 2024-02-19T19:42:16.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:42:16.950+0000 7fcdba128700 1 -- 172.21.15.88:0/1813854390 wait complete. 2024-02-19T19:42:16.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:16 smithi132 ceph-mon[70676]: pgmap v1163: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:17.591 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-02-19T19:42:17.799 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:42:17.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:17 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1813854390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:42:17.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:17 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1813854390' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:42:18.936 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:18 smithi088 ceph-mon[88643]: pgmap v1164: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:18.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:18 smithi132 ceph-mon[70676]: pgmap v1164: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:19.440 INFO:teuthology.orchestra.run.smithi088.stdout:wait for servicemap items w/ changing names to refresh 2024-02-19T19:42:20.110 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'sleep 60' 2024-02-19T19:42:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:42:20.773 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:20 smithi088 ceph-mon[88643]: pgmap v1165: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:20.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:20 smithi132 ceph-mon[70676]: pgmap v1165: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:22.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:22 smithi132 ceph-mon[70676]: pgmap v1166: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:23.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:22 smithi088 ceph-mon[88643]: pgmap v1166: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:24.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:24 smithi132 ceph-mon[70676]: pgmap v1167: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:25.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:24 smithi088 ceph-mon[88643]: pgmap v1167: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:26.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:26 smithi132 ceph-mon[70676]: pgmap v1168: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:27.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:26 smithi088 ceph-mon[88643]: pgmap v1168: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:28 smithi132 ceph-mon[70676]: pgmap v1169: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:42:29.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:28 smithi088 ceph-mon[88643]: pgmap v1169: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:29.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:42:30.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:30 smithi132 ceph-mon[70676]: pgmap v1170: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:31.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:30 smithi088 ceph-mon[88643]: pgmap v1170: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:32.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:32 smithi132 ceph-mon[70676]: pgmap v1171: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:33.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:32 smithi088 ceph-mon[88643]: pgmap v1171: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:34.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:34 smithi088 ceph-mon[88643]: pgmap v1172: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:34.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:34 smithi132 ceph-mon[70676]: pgmap v1172: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:36 smithi132 ceph-mon[70676]: pgmap v1173: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:37.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:36 smithi088 ceph-mon[88643]: pgmap v1173: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:38 smithi132 ceph-mon[70676]: pgmap v1174: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:39.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:38 smithi088 ceph-mon[88643]: pgmap v1174: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:40.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:40 smithi132 ceph-mon[70676]: pgmap v1175: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:41.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:40 smithi088 ceph-mon[88643]: pgmap v1175: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:42.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:42 smithi132 ceph-mon[70676]: pgmap v1176: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:43.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:42 smithi088 ceph-mon[88643]: pgmap v1176: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:43.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:42:44.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:42:44.982 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:44 smithi088 ceph-mon[88643]: pgmap v1177: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:44.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:44 smithi132 ceph-mon[70676]: pgmap v1177: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:46.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:46 smithi132 ceph-mon[70676]: pgmap v1178: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:47.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:46 smithi088 ceph-mon[88643]: pgmap v1178: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:48.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:48 smithi132 ceph-mon[70676]: pgmap v1179: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:49.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:48 smithi088 ceph-mon[88643]: pgmap v1179: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:50.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:50 smithi132 ceph-mon[70676]: pgmap v1180: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:51.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:50 smithi088 ceph-mon[88643]: pgmap v1180: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:52.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:52 smithi132 ceph-mon[70676]: pgmap v1181: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:52.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:42:53.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:52 smithi088 ceph-mon[88643]: pgmap v1181: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:53.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:42:53.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:42:53.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:42:53.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:42:54.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:42:54.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:42:54.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:42:55.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:54 smithi088 ceph-mon[88643]: pgmap v1182: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:55.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:54 smithi132 ceph-mon[70676]: pgmap v1182: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:57.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:56 smithi088 ceph-mon[88643]: pgmap v1183: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:57.130 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:56 smithi132 ceph-mon[70676]: pgmap v1183: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:58 smithi088 ceph-mon[88643]: pgmap v1184: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:42:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:42:59.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:58 smithi132 ceph-mon[70676]: pgmap v1184: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:42:59.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:42:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:43:01.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:00 smithi088 ceph-mon[88643]: pgmap v1185: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:01.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:00 smithi132 ceph-mon[70676]: pgmap v1185: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:03.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:02 smithi088 ceph-mon[88643]: pgmap v1186: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:03.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:02 smithi132 ceph-mon[70676]: pgmap v1186: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:05.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:04 smithi088 ceph-mon[88643]: pgmap v1187: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:05.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:04 smithi132 ceph-mon[70676]: pgmap v1187: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:07.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:06 smithi088 ceph-mon[88643]: pgmap v1188: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:07.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:06 smithi132 ceph-mon[70676]: pgmap v1188: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:09.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:08 smithi088 ceph-mon[88643]: pgmap v1189: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:09.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:08 smithi132 ceph-mon[70676]: pgmap v1189: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:11.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:10 smithi088 ceph-mon[88643]: pgmap v1190: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:11.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:10 smithi132 ceph-mon[70676]: pgmap v1190: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:13.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:12 smithi088 ceph-mon[88643]: pgmap v1191: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:13.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:12 smithi132 ceph-mon[70676]: pgmap v1191: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:14.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:43:14.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:43:14.955 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:14 smithi088 ceph-mon[88643]: pgmap v1192: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:14 smithi132 ceph-mon[70676]: pgmap v1192: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:17.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:16 smithi088 ceph-mon[88643]: pgmap v1193: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:17.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:16 smithi132 ceph-mon[70676]: pgmap v1193: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:19.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:18 smithi088 ceph-mon[88643]: pgmap v1194: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:19.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:18 smithi132 ceph-mon[70676]: pgmap v1194: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:21.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:20 smithi088 ceph-mon[88643]: pgmap v1195: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:21.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:20 smithi132 ceph-mon[70676]: pgmap v1195: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:22.404 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph orch ps' 2024-02-19T19:43:22.620 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:43:23.048 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:22 smithi088 ceph-mon[88643]: pgmap v1196: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:23.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:22 smithi132 ceph-mon[70676]: pgmap v1196: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:24.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.189+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1369446046 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c700ffd60 msgr2=0x7f9c70103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:24.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.189+0000 7f9c75f8f700 1 --2- 172.21.15.88:0/1369446046 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c700ffd60 0x7f9c70103d30 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f9c58009910 tx=0x7f9c58009c20 comp rx=0 tx=0).stop 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.189+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1369446046 shutdown_connections 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.189+0000 7f9c75f8f700 1 --2- 172.21.15.88:0/1369446046 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c700ffd60 0x7f9c70103d30 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.189+0000 7f9c75f8f700 1 --2- 172.21.15.88:0/1369446046 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c700ff3c0 0x7f9c700ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.189+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1369446046 >> 172.21.15.88:0/1369446046 conn(0x7f9c700fac80 msgr2=0x7f9c700fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.189+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1369446046 shutdown_connections 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.189+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1369446046 wait complete. 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.190+0000 7f9c75f8f700 1 Processor -- start 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.190+0000 7f9c75f8f700 1 -- start start 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.190+0000 7f9c75f8f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c700ff3c0 0x7f9c70196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.190+0000 7f9c75f8f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c700ffd60 0x7f9c701972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.190+0000 7f9c75f8f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c70197890 con 0x7f9c700ff3c0 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.190+0000 7f9c75f8f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c701979d0 con 0x7f9c700ffd60 2024-02-19T19:43:24.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.190+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c700ff3c0 0x7f9c70196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.190+0000 7f9c6f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c700ff3c0 0x7f9c70196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55010/0 (socket says 172.21.15.88:55010) 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.190+0000 7f9c6f7fe700 1 -- 172.21.15.88:0/1829653504 learned_addr learned my addr 172.21.15.88:0/1829653504 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.191+0000 7f9c66ffd700 1 --2- 172.21.15.88:0/1829653504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c700ffd60 0x7f9c701972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.191+0000 7f9c6f7fe700 1 -- 172.21.15.88:0/1829653504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c700ffd60 msgr2=0x7f9c701972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.191+0000 7f9c6f7fe700 1 --2- 172.21.15.88:0/1829653504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c700ffd60 0x7f9c701972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.191+0000 7f9c6f7fe700 1 -- 172.21.15.88:0/1829653504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c580095f0 con 0x7f9c700ff3c0 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.191+0000 7f9c6f7fe700 1 --2- 172.21.15.88:0/1829653504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c700ff3c0 0x7f9c70196d80 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c8a0 tx=0x7f9c6000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.191+0000 7f9c6d7fa700 1 -- 172.21.15.88:0/1829653504 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c600078e0 con 0x7f9c700ff3c0 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.191+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1829653504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7019fdf0 con 0x7f9c700ff3c0 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.191+0000 7f9c6d7fa700 1 -- 172.21.15.88:0/1829653504 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c6000f450 con 0x7f9c700ff3c0 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.191+0000 7f9c6d7fa700 1 -- 172.21.15.88:0/1829653504 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c6000e5e0 con 0x7f9c700ff3c0 2024-02-19T19:43:24.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.191+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1829653504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c701a0200 con 0x7f9c700ff3c0 2024-02-19T19:43:24.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.192+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1829653504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c70107730 con 0x7f9c700ff3c0 2024-02-19T19:43:24.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.193+0000 7f9c6d7fa700 1 -- 172.21.15.88:0/1829653504 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9c60007a40 con 0x7f9c700ff3c0 2024-02-19T19:43:24.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.193+0000 7f9c6d7fa700 1 --2- 172.21.15.88:0/1829653504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c5c06d8a0 0x7f9c5c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:24.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.193+0000 7f9c6d7fa700 1 -- 172.21.15.88:0/1829653504 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9c6008d510 con 0x7f9c700ff3c0 2024-02-19T19:43:24.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.193+0000 7f9c66ffd700 1 --2- 172.21.15.88:0/1829653504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c5c06d8a0 0x7f9c5c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:24.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.194+0000 7f9c66ffd700 1 --2- 172.21.15.88:0/1829653504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c5c06d8a0 0x7f9c5c06fd50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9c58006010 tx=0x7f9c5801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:24.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.198+0000 7f9c6d7fa700 1 -- 172.21.15.88:0/1829653504 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9c60056cc0 con 0x7f9c700ff3c0 2024-02-19T19:43:24.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.404+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1829653504 --> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c70061190 con 0x7f9c5c06d8a0 2024-02-19T19:43:24.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.414+0000 7f9c6d7fa700 1 -- 172.21.15.88:0/1829653504 <== mgr.34100 v2:172.21.15.88:6800/3859407641 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f9c70061190 con 0x7f9c5c06d8a0 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:alertmanager.smithi088 smithi088 *:9093,9094 running (40m) 97s ago 50m 23.6M - 0.25.0 c8568f914cd2 0495fe51424c 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi088 smithi088 running (111s) 97s ago 51m 7386k - 19.0.0-1450-gf78a58c0 f081f847070a 9e949ec33fbf 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-exporter.smithi132 smithi132 running (103s) 99s ago 49m 6958k - 19.0.0-1450-gf78a58c0 f081f847070a a4682d5b635b 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi088 smithi088 running (36m) 97s ago 51m 6656k - 19.0.0-1450-gf78a58c0 f081f847070a 84d033673683 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:crash.smithi132 smithi132 running (36m) 99s ago 49m 6660k - 19.0.0-1450-gf78a58c0 f081f847070a ffaa8f4f2e55 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:grafana.smithi088 smithi088 *:3000 running (39m) 97s ago 50m 86.2M - 9.4.12 a0ea5e3c46d9 5693a6938302 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.ajgvpz smithi088 running (32m) 97s ago 44m 70.4M - 19.0.0-1450-gf78a58c0 f081f847070a a1f82bf1da15 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi088.psjiah smithi088 running (32m) 97s ago 44m 63.9M - 19.0.0-1450-gf78a58c0 f081f847070a 2353594f992a 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.afhgdi smithi132 running (32m) 99s ago 44m 23.9M - 19.0.0-1450-gf78a58c0 f081f847070a b634d55fbc0c 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:mds.cephfs.smithi132.uyjxaz smithi132 running (17m) 99s ago 44m 20.2M - 19.0.0-1450-gf78a58c0 f081f847070a bd5b8134c721 2024-02-19T19:43:24.415 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi088.gyviqq smithi088 *:8443,9283,8765 running (42m) 97s ago 52m 516M - 19.0.0-1450-gf78a58c0 f081f847070a b8b451799b1c 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:mgr.smithi132.psllsa smithi132 *:8443,9283,8765 running (41m) 99s ago 49m 459M - 19.0.0-1450-gf78a58c0 f081f847070a 5e2bc5234821 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi088 smithi088 running (38m) 97s ago 52m 88.4M 2048M 19.0.0-1450-gf78a58c0 f081f847070a 8ff9c135b061 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:mon.smithi132 smithi132 running (37m) 99s ago 49m 61.0M 2048M 19.0.0-1450-gf78a58c0 f081f847070a dabaa14880db 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi088 smithi088 *:9100 running (41m) 97s ago 51m 16.1M - 1.5.0 0da6a335fe13 56262533ba49 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:node-exporter.smithi132 smithi132 *:9100 running (41m) 99s ago 49m 15.6M - 1.5.0 0da6a335fe13 ae44f5cd53df 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 smithi088 running (36m) 97s ago 48m 137M 1216M 19.0.0-1450-gf78a58c0 f081f847070a b544778e0f5d 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 smithi088 running (35m) 97s ago 48m 115M 1216M 19.0.0-1450-gf78a58c0 f081f847070a 5b940966d883 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 smithi088 running (34m) 97s ago 47m 97.3M 1216M 19.0.0-1450-gf78a58c0 f081f847070a f72e97fd38a5 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 smithi132 running (34m) 99s ago 47m 120M 2240M 19.0.0-1450-gf78a58c0 f081f847070a 6dc5d99df1e5 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 smithi132 running (33m) 99s ago 46m 112M 2240M 19.0.0-1450-gf78a58c0 f081f847070a c7a967e24a55 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 smithi132 running (33m) 99s ago 46m 96.0M 2240M 19.0.0-1450-gf78a58c0 f081f847070a be60ff50bffe 2024-02-19T19:43:24.416 INFO:teuthology.orchestra.run.smithi088.stdout:prometheus.smithi088 smithi088 *:9095 running (40m) 97s ago 49m 90.8M - 2.43.0 a07b618ecd1d 6e66ddd40051 2024-02-19T19:43:24.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.416+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1829653504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c5c06d8a0 msgr2=0x7f9c5c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:24.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.416+0000 7f9c75f8f700 1 --2- 172.21.15.88:0/1829653504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c5c06d8a0 0x7f9c5c06fd50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9c58006010 tx=0x7f9c5801a040 comp rx=0 tx=0).stop 2024-02-19T19:43:24.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.416+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1829653504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c700ff3c0 msgr2=0x7f9c70196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:24.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.416+0000 7f9c75f8f700 1 --2- 172.21.15.88:0/1829653504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c700ff3c0 0x7f9c70196d80 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c8a0 tx=0x7f9c6000cbb0 comp rx=0 tx=0).stop 2024-02-19T19:43:24.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.417+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1829653504 shutdown_connections 2024-02-19T19:43:24.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.417+0000 7f9c75f8f700 1 --2- 172.21.15.88:0/1829653504 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f9c5c06d8a0 0x7f9c5c06fd50 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:24.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.417+0000 7f9c75f8f700 1 --2- 172.21.15.88:0/1829653504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9c700ff3c0 0x7f9c70196d80 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:24.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.417+0000 7f9c75f8f700 1 --2- 172.21.15.88:0/1829653504 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9c700ffd60 0x7f9c701972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:24.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.417+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1829653504 >> 172.21.15.88:0/1829653504 conn(0x7f9c700fac80 msgr2=0x7f9c700fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:24.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.417+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1829653504 shutdown_connections 2024-02-19T19:43:24.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:24.417+0000 7f9c75f8f700 1 -- 172.21.15.88:0/1829653504 wait complete. 2024-02-19T19:43:25.009 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:24 smithi088 ceph-mon[88643]: pgmap v1197: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:25.031 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions' 2024-02-19T19:43:25.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:24 smithi132 ceph-mon[70676]: pgmap v1197: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:25.241 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:43:26.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:25 smithi088 ceph-mon[88643]: from='client.36092 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:43:26.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:25 smithi132 ceph-mon[70676]: from='client.36092 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-19T19:43:26.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.897+0000 7fe01d792700 1 -- 172.21.15.88:0/1004096297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe018101d30 msgr2=0x7fe01810a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:26.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.897+0000 7fe01d792700 1 --2- 172.21.15.88:0/1004096297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe018101d30 0x7fe01810a230 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe000009910 tx=0x7fe000009c20 comp rx=0 tx=0).stop 2024-02-19T19:43:26.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.897+0000 7fe01d792700 1 -- 172.21.15.88:0/1004096297 shutdown_connections 2024-02-19T19:43:26.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.897+0000 7fe01d792700 1 --2- 172.21.15.88:0/1004096297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe018101d30 0x7fe01810a230 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:26.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.897+0000 7fe01d792700 1 --2- 172.21.15.88:0/1004096297 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe018101420 0x7fe0181017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.897+0000 7fe01d792700 1 -- 172.21.15.88:0/1004096297 >> 172.21.15.88:0/1004096297 conn(0x7fe0180fac80 msgr2=0x7fe0180fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.897+0000 7fe01d792700 1 -- 172.21.15.88:0/1004096297 shutdown_connections 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.897+0000 7fe01d792700 1 -- 172.21.15.88:0/1004096297 wait complete. 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.897+0000 7fe01d792700 1 Processor -- start 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe01d792700 1 -- start start 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe01d792700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe018101420 0x7fe018195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe01d792700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe018101d30 0x7fe018196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe01d792700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe018196c10 con 0x7fe018101420 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe01d792700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01819a9a0 con 0x7fe018101d30 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe016ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe018101420 0x7fe018195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe016ffd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe018101420 0x7fe018195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50576/0 (socket says 172.21.15.88:50576) 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe016ffd700 1 -- 172.21.15.88:0/1255510845 learned_addr learned my addr 172.21.15.88:0/1255510845 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:26.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe0167fc700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe018101d30 0x7fe018196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe016ffd700 1 -- 172.21.15.88:0/1255510845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe018101d30 msgr2=0x7fe018196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe016ffd700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe018101d30 0x7fe018196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe016ffd700 1 -- 172.21.15.88:0/1255510845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0000095f0 con 0x7fe018101420 2024-02-19T19:43:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe0167fc700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe018101d30 0x7fe018196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:43:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.898+0000 7fe016ffd700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe018101420 0x7fe018195ff0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe00800b4b0 tx=0x7fe00800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.899+0000 7fe00ffff700 1 -- 172.21.15.88:0/1255510845 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe008011800 con 0x7fe018101420 2024-02-19T19:43:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.899+0000 7fe00ffff700 1 -- 172.21.15.88:0/1255510845 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe008011e20 con 0x7fe018101420 2024-02-19T19:43:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.899+0000 7fe01d792700 1 -- 172.21.15.88:0/1255510845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01819ac80 con 0x7fe018101420 2024-02-19T19:43:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.899+0000 7fe00ffff700 1 -- 172.21.15.88:0/1255510845 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe00800f2e0 con 0x7fe018101420 2024-02-19T19:43:26.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.899+0000 7fe01d792700 1 -- 172.21.15.88:0/1255510845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01819b1d0 con 0x7fe018101420 2024-02-19T19:43:26.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.900+0000 7fe01d792700 1 -- 172.21.15.88:0/1255510845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0181078a0 con 0x7fe018101420 2024-02-19T19:43:26.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.900+0000 7fe00ffff700 1 -- 172.21.15.88:0/1255510845 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe008010380 con 0x7fe018101420 2024-02-19T19:43:26.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.901+0000 7fe00ffff700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe00406d8a0 0x7fe00406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:26.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.901+0000 7fe00ffff700 1 -- 172.21.15.88:0/1255510845 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe00808d110 con 0x7fe018101420 2024-02-19T19:43:26.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.901+0000 7fe0167fc700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe00406d8a0 0x7fe00406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:26.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.901+0000 7fe0167fc700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe00406d8a0 0x7fe00406fd50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe000006010 tx=0x7fe00000b540 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:26.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:26.905+0000 7fe00ffff700 1 -- 172.21.15.88:0/1255510845 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe0080568c0 con 0x7fe018101420 2024-02-19T19:43:27.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:26 smithi088 ceph-mon[88643]: pgmap v1198: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:27.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.183+0000 7fe01d792700 1 -- 172.21.15.88:0/1255510845 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe01819b450 con 0x7fe018101420 2024-02-19T19:43:27.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.184+0000 7fe00ffff700 1 -- 172.21.15.88:0/1255510845 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fe008059f80 con 0x7fe018101420 2024-02-19T19:43:27.184 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-02-19T19:43:27.184 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": { 2024-02-19T19:43:27.184 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": { 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 2 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": { 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 6 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 4 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: "overall": { 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)": 14 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-02-19T19:43:27.185 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-02-19T19:43:27.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.186+0000 7fe01d792700 1 -- 172.21.15.88:0/1255510845 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe00406d8a0 msgr2=0x7fe00406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:27.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.186+0000 7fe01d792700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe00406d8a0 0x7fe00406fd50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe000006010 tx=0x7fe00000b540 comp rx=0 tx=0).stop 2024-02-19T19:43:27.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.186+0000 7fe01d792700 1 -- 172.21.15.88:0/1255510845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe018101420 msgr2=0x7fe018195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:27.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.186+0000 7fe01d792700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe018101420 0x7fe018195ff0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe00800b4b0 tx=0x7fe00800b870 comp rx=0 tx=0).stop 2024-02-19T19:43:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.186+0000 7fe01d792700 1 -- 172.21.15.88:0/1255510845 shutdown_connections 2024-02-19T19:43:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.186+0000 7fe01d792700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe00406d8a0 0x7fe00406fd50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.186+0000 7fe01d792700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe018101420 0x7fe018195ff0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.186+0000 7fe01d792700 1 --2- 172.21.15.88:0/1255510845 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe018101d30 0x7fe018196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.186+0000 7fe01d792700 1 -- 172.21.15.88:0/1255510845 >> 172.21.15.88:0/1255510845 conn(0x7fe0180fac80 msgr2=0x7fe0181049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.186+0000 7fe01d792700 1 -- 172.21.15.88:0/1255510845 shutdown_connections 2024-02-19T19:43:27.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:27.186+0000 7fe01d792700 1 -- 172.21.15.88:0/1255510845 wait complete. 2024-02-19T19:43:27.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:26 smithi132 ceph-mon[70676]: pgmap v1198: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:27.854 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-02-19T19:43:28.062 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:43:28.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:27 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1255510845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:43:28.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:27 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1255510845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:43:29.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:28 smithi088 ceph-mon[88643]: pgmap v1199: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:29.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:43:29.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:28 smithi132 ceph-mon[70676]: pgmap v1199: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:29.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:43:29.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.678+0000 7f797aa8f700 1 -- 172.21.15.88:0/2229348883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7974105230 msgr2=0x7f7974105600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:29.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.678+0000 7f797aa8f700 1 --2- 172.21.15.88:0/2229348883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7974105230 0x7f7974105600 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f796c00b0a0 tx=0x7f796c00b3b0 comp rx=0 tx=0).stop 2024-02-19T19:43:29.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.678+0000 7f797aa8f700 1 -- 172.21.15.88:0/2229348883 shutdown_connections 2024-02-19T19:43:29.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.678+0000 7f797aa8f700 1 --2- 172.21.15.88:0/2229348883 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7974068540 0x7f79740689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:29.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.678+0000 7f797aa8f700 1 --2- 172.21.15.88:0/2229348883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7974105230 0x7f7974105600 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:29.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.678+0000 7f797aa8f700 1 -- 172.21.15.88:0/2229348883 >> 172.21.15.88:0/2229348883 conn(0x7f7974075130 msgr2=0x7f7974075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:29.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.678+0000 7f797aa8f700 1 -- 172.21.15.88:0/2229348883 shutdown_connections 2024-02-19T19:43:29.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.678+0000 7f797aa8f700 1 -- 172.21.15.88:0/2229348883 wait complete. 2024-02-19T19:43:29.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f797aa8f700 1 Processor -- start 2024-02-19T19:43:29.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f797aa8f700 1 -- start start 2024-02-19T19:43:29.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f797aa8f700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7974068540 0x7f7974195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:29.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f797aa8f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7974105230 0x7f7974196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f797aa8f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7974196980 con 0x7f7974105230 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f797aa8f700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7974196ac0 con 0x7f7974068540 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f7973fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7974105230 0x7f7974196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f7973fff700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7974105230 0x7f7974196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50596/0 (socket says 172.21.15.88:50596) 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f7973fff700 1 -- 172.21.15.88:0/354493598 learned_addr learned my addr 172.21.15.88:0/354493598 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f797882b700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7974068540 0x7f7974195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f7973fff700 1 -- 172.21.15.88:0/354493598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7974068540 msgr2=0x7f7974195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f7973fff700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7974068540 0x7f7974195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f7973fff700 1 -- 172.21.15.88:0/354493598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c009d00 con 0x7f7974105230 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f797882b700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7974068540 0x7f7974195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.679+0000 7f7973fff700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7974105230 0x7f7974196330 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f796800c930 tx=0x7f796800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.680+0000 7f7971ffb700 1 -- 172.21.15.88:0/354493598 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7968007ad0 con 0x7f7974105230 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.680+0000 7f797aa8f700 1 -- 172.21.15.88:0/354493598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f797419a910 con 0x7f7974105230 2024-02-19T19:43:29.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.680+0000 7f7971ffb700 1 -- 172.21.15.88:0/354493598 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7968007c30 con 0x7f7974105230 2024-02-19T19:43:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.680+0000 7f7971ffb700 1 -- 172.21.15.88:0/354493598 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7968018610 con 0x7f7974105230 2024-02-19T19:43:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.680+0000 7f797aa8f700 1 -- 172.21.15.88:0/354493598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f797419ae60 con 0x7f7974105230 2024-02-19T19:43:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.681+0000 7f797aa8f700 1 -- 172.21.15.88:0/354493598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f797404f2a0 con 0x7f7974105230 2024-02-19T19:43:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.681+0000 7f7971ffb700 1 -- 172.21.15.88:0/354493598 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f796800f450 con 0x7f7974105230 2024-02-19T19:43:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.682+0000 7f7971ffb700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795c06d8a0 0x7f795c06fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.682+0000 7f7971ffb700 1 -- 172.21.15.88:0/354493598 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f796808d780 con 0x7f7974105230 2024-02-19T19:43:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.682+0000 7f797882b700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795c06d8a0 0x7f795c06fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.682+0000 7f797882b700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795c06d8a0 0x7f795c06fd50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f796c009480 tx=0x7f796c000f40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.686+0000 7f7971ffb700 1 -- 172.21.15.88:0/354493598 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7968056f30 con 0x7f7974105230 2024-02-19T19:43:29.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.969+0000 7f797aa8f700 1 -- 172.21.15.88:0/354493598 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f797419b180 con 0x7f7974105230 2024-02-19T19:43:29.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.970+0000 7f7971ffb700 1 -- 172.21.15.88:0/354493598 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f796805a5f0 con 0x7f7974105230 2024-02-19T19:43:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.972+0000 7f797aa8f700 1 -- 172.21.15.88:0/354493598 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795c06d8a0 msgr2=0x7f795c06fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.972+0000 7f797aa8f700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795c06d8a0 0x7f795c06fd50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f796c009480 tx=0x7f796c000f40 comp rx=0 tx=0).stop 2024-02-19T19:43:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.972+0000 7f797aa8f700 1 -- 172.21.15.88:0/354493598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7974105230 msgr2=0x7f7974196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.972+0000 7f797aa8f700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7974105230 0x7f7974196330 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f796800c930 tx=0x7f796800ccf0 comp rx=0 tx=0).stop 2024-02-19T19:43:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.972+0000 7f797aa8f700 1 -- 172.21.15.88:0/354493598 shutdown_connections 2024-02-19T19:43:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.972+0000 7f797aa8f700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f795c06d8a0 0x7f795c06fd50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.972+0000 7f797aa8f700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7974068540 0x7f7974195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.972+0000 7f797aa8f700 1 --2- 172.21.15.88:0/354493598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7974105230 0x7f7974196330 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.972+0000 7f797aa8f700 1 -- 172.21.15.88:0/354493598 >> 172.21.15.88:0/354493598 conn(0x7f7974075130 msgr2=0x7f79740fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.972+0000 7f797aa8f700 1 -- 172.21.15.88:0/354493598 shutdown_connections 2024-02-19T19:43:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:29.972+0000 7f797aa8f700 1 -- 172.21.15.88:0/354493598 wait complete. 2024-02-19T19:43:29.983 INFO:teuthology.orchestra.run.smithi088.stdout:true 2024-02-19T19:43:30.577 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -e sha1=f78a58c0ffd401d1493058a1022c35f011d65275 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-02-19T19:43:30.786 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:43:31.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:30 smithi088 ceph-mon[88643]: pgmap v1200: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:31.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:30 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/354493598' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:43:31.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:30 smithi132 ceph-mon[70676]: pgmap v1200: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:30 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/354493598' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:43:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.374+0000 7f5000491700 1 -- 172.21.15.88:0/3924391241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ff8104130 msgr2=0x7f4ff8104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.374+0000 7f5000491700 1 --2- 172.21.15.88:0/3924391241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ff8104130 0x7f4ff8104500 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f4fe8009910 tx=0x7f4fe8009c20 comp rx=0 tx=0).stop 2024-02-19T19:43:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.374+0000 7f5000491700 1 -- 172.21.15.88:0/3924391241 shutdown_connections 2024-02-19T19:43:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.374+0000 7f5000491700 1 --2- 172.21.15.88:0/3924391241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff80ffbd0 0x7f4ff8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.374+0000 7f5000491700 1 --2- 172.21.15.88:0/3924391241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ff8104130 0x7f4ff8104500 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.374+0000 7f5000491700 1 -- 172.21.15.88:0/3924391241 >> 172.21.15.88:0/3924391241 conn(0x7f4ff8075130 msgr2=0x7f4ff8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.374+0000 7f5000491700 1 -- 172.21.15.88:0/3924391241 shutdown_connections 2024-02-19T19:43:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.374+0000 7f5000491700 1 -- 172.21.15.88:0/3924391241 wait complete. 2024-02-19T19:43:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.375+0000 7f5000491700 1 Processor -- start 2024-02-19T19:43:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.375+0000 7f5000491700 1 -- start start 2024-02-19T19:43:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.375+0000 7f5000491700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff80ffbd0 0x7f4ff8195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.375+0000 7f5000491700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ff8104130 0x7f4ff8196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.375+0000 7f5000491700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff8196bf0 con 0x7f4ff8104130 2024-02-19T19:43:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.375+0000 7f5000491700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff819a980 con 0x7f4ff80ffbd0 2024-02-19T19:43:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.375+0000 7f4ffda2c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ff8104130 0x7f4ff8196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.375+0000 7f4ffda2c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ff8104130 0x7f4ff8196510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50616/0 (socket says 172.21.15.88:50616) 2024-02-19T19:43:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.375+0000 7f4ffda2c700 1 -- 172.21.15.88:0/2164464718 learned_addr learned my addr 172.21.15.88:0/2164464718 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.375+0000 7f4ffe22d700 1 --2- 172.21.15.88:0/2164464718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff80ffbd0 0x7f4ff8195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.376+0000 7f4ffda2c700 1 -- 172.21.15.88:0/2164464718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff80ffbd0 msgr2=0x7f4ff8195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.376+0000 7f4ffda2c700 1 --2- 172.21.15.88:0/2164464718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff80ffbd0 0x7f4ff8195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.376+0000 7f4ffda2c700 1 -- 172.21.15.88:0/2164464718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe80095f0 con 0x7f4ff8104130 2024-02-19T19:43:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.376+0000 7f4ffda2c700 1 --2- 172.21.15.88:0/2164464718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ff8104130 0x7f4ff8196510 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400b4b0 tx=0x7f4ff400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.376+0000 7f4fef7fe700 1 -- 172.21.15.88:0/2164464718 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ff4011800 con 0x7f4ff8104130 2024-02-19T19:43:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.376+0000 7f4fef7fe700 1 -- 172.21.15.88:0/2164464718 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ff4011e20 con 0x7f4ff8104130 2024-02-19T19:43:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.376+0000 7f5000491700 1 -- 172.21.15.88:0/2164464718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff819ac60 con 0x7f4ff8104130 2024-02-19T19:43:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.376+0000 7f4fef7fe700 1 -- 172.21.15.88:0/2164464718 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ff400f2e0 con 0x7f4ff8104130 2024-02-19T19:43:32.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.376+0000 7f5000491700 1 -- 172.21.15.88:0/2164464718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff819b1b0 con 0x7f4ff8104130 2024-02-19T19:43:32.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.377+0000 7f5000491700 1 -- 172.21.15.88:0/2164464718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff804f2a0 con 0x7f4ff8104130 2024-02-19T19:43:32.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.377+0000 7f4fef7fe700 1 -- 172.21.15.88:0/2164464718 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f4ff4010380 con 0x7f4ff8104130 2024-02-19T19:43:32.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.378+0000 7f4fef7fe700 1 --2- 172.21.15.88:0/2164464718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fe406d8a0 0x7f4fe406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:32.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.378+0000 7f4fef7fe700 1 -- 172.21.15.88:0/2164464718 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4ff408d110 con 0x7f4ff8104130 2024-02-19T19:43:32.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.378+0000 7f4ffe22d700 1 --2- 172.21.15.88:0/2164464718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fe406d8a0 0x7f4fe406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:32.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.378+0000 7f4ffe22d700 1 --2- 172.21.15.88:0/2164464718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fe406d8a0 0x7f4fe406fd50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800b5c0 tx=0x7f4fe801a040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:32.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.383+0000 7f4fef7fe700 1 -- 172.21.15.88:0/2164464718 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4ff40568c0 con 0x7f4ff8104130 2024-02-19T19:43:32.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.667+0000 7f5000491700 1 -- 172.21.15.88:0/2164464718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4ff819b430 con 0x7f4ff8104130 2024-02-19T19:43:32.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.667+0000 7f4fef7fe700 1 -- 172.21.15.88:0/2164464718 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f4ff4059f80 con 0x7f4ff8104130 2024-02-19T19:43:32.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.669+0000 7f5000491700 1 -- 172.21.15.88:0/2164464718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fe406d8a0 msgr2=0x7f4fe406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:32.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.669+0000 7f5000491700 1 --2- 172.21.15.88:0/2164464718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fe406d8a0 0x7f4fe406fd50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f4fe800b5c0 tx=0x7f4fe801a040 comp rx=0 tx=0).stop 2024-02-19T19:43:32.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.669+0000 7f5000491700 1 -- 172.21.15.88:0/2164464718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ff8104130 msgr2=0x7f4ff8196510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:32.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.669+0000 7f5000491700 1 --2- 172.21.15.88:0/2164464718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ff8104130 0x7f4ff8196510 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f4ff400b4b0 tx=0x7f4ff400b870 comp rx=0 tx=0).stop 2024-02-19T19:43:32.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.669+0000 7f5000491700 1 -- 172.21.15.88:0/2164464718 shutdown_connections 2024-02-19T19:43:32.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.669+0000 7f5000491700 1 --2- 172.21.15.88:0/2164464718 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f4fe406d8a0 0x7f4fe406fd50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:32.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.669+0000 7f5000491700 1 --2- 172.21.15.88:0/2164464718 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f4ff80ffbd0 0x7f4ff8195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:32.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.669+0000 7f5000491700 1 --2- 172.21.15.88:0/2164464718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ff8104130 0x7f4ff8196510 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:32.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.669+0000 7f5000491700 1 -- 172.21.15.88:0/2164464718 >> 172.21.15.88:0/2164464718 conn(0x7f4ff8075130 msgr2=0x7f4ff8109220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:32.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.669+0000 7f5000491700 1 -- 172.21.15.88:0/2164464718 shutdown_connections 2024-02-19T19:43:32.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:32.669+0000 7f5000491700 1 -- 172.21.15.88:0/2164464718 wait complete. 2024-02-19T19:43:32.680 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph version 19.0.0-1450-gf78a58c0 (f78a58c0ffd401d1493058a1022c35f011d65275) squid (dev)" 2024-02-19T19:43:32.979 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:32 smithi088 ceph-mon[88643]: pgmap v1201: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:32.979 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:32 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2164464718' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:43:33.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:32 smithi132 ceph-mon[70676]: pgmap v1201: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:33.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:32 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2164464718' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-02-19T19:43:33.267 DEBUG:teuthology.parallel:result is None 2024-02-19T19:43:33.267 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-19T19:43:33.279 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T19:43:33.279 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-19T19:43:33.486 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/mon.smithi088/config 2024-02-19T19:43:35.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:34 smithi088 ceph-mon[88643]: pgmap v1202: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:35.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.133+0000 7f3374b71700 1 -- 172.21.15.88:0/3795827475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3370100d70 msgr2=0x7f33701094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:35.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.133+0000 7f3374b71700 1 --2- 172.21.15.88:0/3795827475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3370100d70 0x7f33701094d0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3364009940 tx=0x7f3364009c50 comp rx=0 tx=0).stop 2024-02-19T19:43:35.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.134+0000 7f3374b71700 1 -- 172.21.15.88:0/3795827475 shutdown_connections 2024-02-19T19:43:35.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.134+0000 7f3374b71700 1 --2- 172.21.15.88:0/3795827475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3370100d70 0x7f33701094d0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:35.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.134+0000 7f3374b71700 1 --2- 172.21.15.88:0/3795827475 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3370100460 0x7f3370100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:35.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.134+0000 7f3374b71700 1 -- 172.21.15.88:0/3795827475 >> 172.21.15.88:0/3795827475 conn(0x7f33700748b0 msgr2=0x7f3370074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:35.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.134+0000 7f3374b71700 1 -- 172.21.15.88:0/3795827475 shutdown_connections 2024-02-19T19:43:35.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.134+0000 7f3374b71700 1 -- 172.21.15.88:0/3795827475 wait complete. 2024-02-19T19:43:35.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.134+0000 7f3374b71700 1 Processor -- start 2024-02-19T19:43:35.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.134+0000 7f3374b71700 1 -- start start 2024-02-19T19:43:35.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.134+0000 7f3374b71700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3370100460 0x7f3370195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:35.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f3374b71700 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3370100d70 0x7f3370196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f3374b71700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33701968b0 con 0x7f3370100460 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f3374b71700 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33701969f0 con 0x7f3370100d70 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f336e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3370100460 0x7f3370195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f336e59c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3370100460 0x7f3370195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50638/0 (socket says 172.21.15.88:50638) 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f336e59c700 1 -- 172.21.15.88:0/4198109722 learned_addr learned my addr 172.21.15.88:0/4198109722 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f336dd9b700 1 --2- 172.21.15.88:0/4198109722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3370100d70 0x7f3370196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f336e59c700 1 -- 172.21.15.88:0/4198109722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3370100d70 msgr2=0x7f3370196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f336e59c700 1 --2- 172.21.15.88:0/4198109722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3370100d70 0x7f3370196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f336e59c700 1 -- 172.21.15.88:0/4198109722 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33640095f0 con 0x7f3370100460 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f336e59c700 1 --2- 172.21.15.88:0/4198109722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3370100460 0x7f3370195d20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f336000c930 tx=0x7f336000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f335f7fe700 1 -- 172.21.15.88:0/4198109722 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3360007ad0 con 0x7f3370100460 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f335f7fe700 1 -- 172.21.15.88:0/4198109722 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f336000ce80 con 0x7f3370100460 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f3374b71700 1 -- 172.21.15.88:0/4198109722 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337019a840 con 0x7f3370100460 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f335f7fe700 1 -- 172.21.15.88:0/4198109722 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3360018630 con 0x7f3370100460 2024-02-19T19:43:35.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.135+0000 7f3374b71700 1 -- 172.21.15.88:0/4198109722 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337019ad60 con 0x7f3370100460 2024-02-19T19:43:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.136+0000 7f3374b71700 1 -- 172.21.15.88:0/4198109722 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3370107e40 con 0x7f3370100460 2024-02-19T19:43:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.137+0000 7f335f7fe700 1 -- 172.21.15.88:0/4198109722 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f3360007c30 con 0x7f3370100460 2024-02-19T19:43:35.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.137+0000 7f335f7fe700 1 --2- 172.21.15.88:0/4198109722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f335806d8a0 0x7f335806fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:35.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.137+0000 7f336dd9b700 1 --2- 172.21.15.88:0/4198109722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f335806d8a0 0x7f335806fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:35.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.137+0000 7f335f7fe700 1 -- 172.21.15.88:0/4198109722 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f336008d560 con 0x7f3370100460 2024-02-19T19:43:35.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.138+0000 7f336dd9b700 1 --2- 172.21.15.88:0/4198109722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f335806d8a0 0x7f335806fd50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3364006010 tx=0x7f336400b9d0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:35.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.142+0000 7f335f7fe700 1 -- 172.21.15.88:0/4198109722 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3360056d10 con 0x7f3370100460 2024-02-19T19:43:35.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:34 smithi132 ceph-mon[70676]: pgmap v1202: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:35.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.385+0000 7f3374b71700 1 -- 172.21.15.88:0/4198109722 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3370107820 con 0x7f3370100460 2024-02-19T19:43:35.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.386+0000 7f335f7fe700 1 -- 172.21.15.88:0/4198109722 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 40 v40) v1 ==== 76+0+1876 (secure 0 0 0) 0x7f336005a3d0 con 0x7f3370100460 2024-02-19T19:43:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:e40 2024-02-19T19:43:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-19T19:43:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-19T19:43:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:legacy client fscid: 1 2024-02-19T19:43:35.386 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:Filesystem 'cephfs' (1) 2024-02-19T19:43:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:fs_name cephfs 2024-02-19T19:43:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 40 2024-02-19T19:43:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-02-19T19:43:35.386 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-02-19T18:58:32.192123+0000 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:modified 2024-02-19T19:41:51.228566+0000 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:tableserver 0 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:root 0 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:session_timeout 60 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:session_autoclose 300 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:max_file_size 1099511627776 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:max_xattr_size 65536 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:required_client_features {} 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure 0 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:last_failure_osd_epoch 84 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:max_mds 2 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:in 0,1 2024-02-19T19:43:35.387 INFO:teuthology.orchestra.run.smithi088.stdout:up {0=34430,1=34462} 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:failed 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:damaged 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:stopped 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:data_pools [3] 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:metadata_pool 2 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:inline_data disabled 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:balancer 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:bal_rank_mask -1 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:standby_count_wanted 1 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.ajgvpz{0:34430} state up:active seq 469 join_fscid=1 addr [v2:172.21.15.88:6826/183364349,v1:172.21.15.88:6827/183364349] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi088.psjiah{1:34462} state up:active seq 464 join_fscid=1 addr [v2:172.21.15.88:6828/830186541,v1:172.21.15.88:6829/830186541] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:35.388 INFO:teuthology.orchestra.run.smithi088.stdout:Standby daemons: 2024-02-19T19:43:35.389 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.afhgdi{-1:44365} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6824/594928725,v1:172.21.15.132:6825/594928725] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:43:35.389 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.cephfs.smithi132.uyjxaz{-1:44753} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.132:6826/858155091,v1:172.21.15.132:6827/858155091] compat {c=[1],r=[1],i=[fff]}] 2024-02-19T19:43:35.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.388+0000 7f3374b71700 1 -- 172.21.15.88:0/4198109722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f335806d8a0 msgr2=0x7f335806fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:35.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.388+0000 7f3374b71700 1 --2- 172.21.15.88:0/4198109722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f335806d8a0 0x7f335806fd50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f3364006010 tx=0x7f336400b9d0 comp rx=0 tx=0).stop 2024-02-19T19:43:35.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.388+0000 7f3374b71700 1 -- 172.21.15.88:0/4198109722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3370100460 msgr2=0x7f3370195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:35.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.388+0000 7f3374b71700 1 --2- 172.21.15.88:0/4198109722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3370100460 0x7f3370195d20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f336000c930 tx=0x7f336000cc40 comp rx=0 tx=0).stop 2024-02-19T19:43:35.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.388+0000 7f3374b71700 1 -- 172.21.15.88:0/4198109722 shutdown_connections 2024-02-19T19:43:35.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.388+0000 7f3374b71700 1 --2- 172.21.15.88:0/4198109722 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f335806d8a0 0x7f335806fd50 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:35.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.388+0000 7f3374b71700 1 --2- 172.21.15.88:0/4198109722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3370100460 0x7f3370195d20 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:35.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.388+0000 7f3374b71700 1 --2- 172.21.15.88:0/4198109722 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f3370100d70 0x7f3370196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:35.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.388+0000 7f3374b71700 1 -- 172.21.15.88:0/4198109722 >> 172.21.15.88:0/4198109722 conn(0x7f33700748b0 msgr2=0x7f33700ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:35.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.389+0000 7f3374b71700 1 -- 172.21.15.88:0/4198109722 shutdown_connections 2024-02-19T19:43:35.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:35.389+0000 7f3374b71700 1 -- 172.21.15.88:0/4198109722 wait complete. 2024-02-19T19:43:35.390 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 40 2024-02-19T19:43:35.977 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:35 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4198109722' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:43:35.998 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-02-19T19:43:36.009 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T19:43:36.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.112+0000 7f1eca0ce640 1 -- 172.21.15.88:0/2893589644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4100730 msgr2=0x7f1ec4100b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.112+0000 7f1eca0ce640 1 --2- 172.21.15.88:0/2893589644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4100730 0x7f1ec4100b00 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1eb40098b0 tx=0x7f1eb402bd60 comp rx=0 tx=0).stop 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.112+0000 7f1eca0ce640 1 -- 172.21.15.88:0/2893589644 shutdown_connections 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.112+0000 7f1eca0ce640 1 --2- 172.21.15.88:0/2893589644 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec4101040 0x7f1ec4109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.112+0000 7f1eca0ce640 1 --2- 172.21.15.88:0/2893589644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4100730 0x7f1ec4100b00 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.112+0000 7f1eca0ce640 1 -- 172.21.15.88:0/2893589644 >> 172.21.15.88:0/2893589644 conn(0x7f1ec4075270 msgr2=0x7f1ec4075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.113+0000 7f1eca0ce640 1 -- 172.21.15.88:0/2893589644 shutdown_connections 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.113+0000 7f1eca0ce640 1 -- 172.21.15.88:0/2893589644 wait complete. 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.113+0000 7f1eca0ce640 1 Processor -- start 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.113+0000 7f1eca0ce640 1 -- start start 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.113+0000 7f1eca0ce640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec4100730 0x7f1ec41952a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.113+0000 7f1eca0ce640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4101040 0x7f1ec41957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:36.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.113+0000 7f1eca0ce640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec4195e70 con 0x7f1ec4101040 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.113+0000 7f1eca0ce640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec4199c40 con 0x7f1ec4100730 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1ebaffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4101040 0x7f1ec41957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1ebaffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4101040 0x7f1ec41957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57606/0 (socket says 172.21.15.88:57606) 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1ebaffd640 1 -- 172.21.15.88:0/1500614915 learned_addr learned my addr 172.21.15.88:0/1500614915 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1ec37fe640 1 --2- 172.21.15.88:0/1500614915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec4100730 0x7f1ec41952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1ebaffd640 1 -- 172.21.15.88:0/1500614915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec4100730 msgr2=0x7f1ec41952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1ebaffd640 1 --2- 172.21.15.88:0/1500614915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec4100730 0x7f1ec41952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1ebaffd640 1 -- 172.21.15.88:0/1500614915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb4009560 con 0x7f1ec4101040 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1ebaffd640 1 --2- 172.21.15.88:0/1500614915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4101040 0x7f1ec41957e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00c8f0 tx=0x7f1eac00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1ec17fa640 1 -- 172.21.15.88:0/1500614915 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1eac007b80 con 0x7f1ec4101040 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1ec17fa640 1 -- 172.21.15.88:0/1500614915 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1eac0054e0 con 0x7f1ec4101040 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1ec17fa640 1 -- 172.21.15.88:0/1500614915 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1eac00f450 con 0x7f1ec4101040 2024-02-19T19:43:36.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1eca0ce640 1 -- 172.21.15.88:0/1500614915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec4199f20 con 0x7f1ec4101040 2024-02-19T19:43:36.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.114+0000 7f1eca0ce640 1 -- 172.21.15.88:0/1500614915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec419a480 con 0x7f1ec4101040 2024-02-19T19:43:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.115+0000 7f1eca0ce640 1 -- 172.21.15.88:0/1500614915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec404e000 con 0x7f1ec4101040 2024-02-19T19:43:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.115+0000 7f1ec17fa640 1 -- 172.21.15.88:0/1500614915 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f1eac004f60 con 0x7f1ec4101040 2024-02-19T19:43:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.116+0000 7f1ec17fa640 1 --2- 172.21.15.88:0/1500614915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb006d8d0 0x7f1eb006fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.116+0000 7f1ec17fa640 1 -- 172.21.15.88:0/1500614915 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1eac08d2a0 con 0x7f1ec4101040 2024-02-19T19:43:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.116+0000 7f1ec37fe640 1 --2- 172.21.15.88:0/1500614915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb006d8d0 0x7f1eb006fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:36.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.116+0000 7f1ec37fe640 1 --2- 172.21.15.88:0/1500614915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb006d8d0 0x7f1eb006fd80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4000c00 tx=0x7f1eb4005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:36.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.121+0000 7f1ec17fa640 1 -- 172.21.15.88:0/1500614915 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1eac056b00 con 0x7f1ec4101040 2024-02-19T19:43:36.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:35 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4198109722' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-19T19:43:36.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.295+0000 7f1eca0ce640 1 -- 172.21.15.88:0/1500614915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ec410c880 con 0x7f1ec4101040 2024-02-19T19:43:36.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.296+0000 7f1ec17fa640 1 -- 172.21.15.88:0/1500614915 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16777 (secure 0 0 0) 0x7f1eac05a1c0 con 0x7f1ec4101040 2024-02-19T19:43:36.297 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:36.297 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":88,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","created":"2024-02-19T18:50:27.026977+0000","modified":"2024-02-19T19:26:18.541223+0000","last_up_change":"2024-02-19T19:10:25.399688+0000","last_in_change":"2024-02-19T18:56:55.518417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T18:55:57.936009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-19T18:58:31.129412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-19T18:58:31.420377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"787c5fec-c403-4975-92c3-8b68fcf9e399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":81,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6803","nonce":4275403566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6805","nonce":4275403566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6809","nonce":4275403566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6807","nonce":4275403566}]},"public_addr":"172.21.15.88:6803/4275403566","cluster_addr":"172.21.15.88:6805/4275403566","heartbeat_back_addr":"172.21.15.88:6809/4275403566","heartbeat_front_addr":"172.21.15.88:6807/4275403566","state":["exists","up"]},{"osd":1,"uuid":"a459843c-43d7-480f-a76b-d52d6e8ca061","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":57,"up_from":61,"up_thru":81,"down_at":58,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6811","nonce":1931660446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6813","nonce":1931660446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6817","nonce":1931660446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6815","nonce":1931660446}]},"public_addr":"172.21.15.88:6811/1931660446","cluster_addr":"172.21.15.88:6813/1931660446","heartbeat_back_addr":"172.21.15.88:6817/1931660446","heartbeat_front_addr":"172.21.15.88:6815/1931660446","state":["exists","up"]},{"osd":2,"uuid":"66334f86-7411-4587-b96b-22b61bf5201a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":62,"up_from":66,"up_thru":81,"down_at":63,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6819","nonce":2845929374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6821","nonce":2845929374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6825","nonce":2845929374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6823","nonce":2845929374}]},"public_addr":"172.21.15.88:6819/2845929374","cluster_addr":"172.21.15.88:6821/2845929374","heartbeat_back_addr":"172.21.15.88:6825/2845929374","heartbeat_front_addr":"172.21.15.88:6823/2845929374","state":["exists","up"]},{"osd":3,"uuid":"733318f2-be90-44d7-8c38-fb773f7da169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":67,"up_from":71,"up_thru":81,"down_at":68,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6801","nonce":2930324400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6803","nonce":2930324400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6807","nonce":2930324400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6805","nonce":2930324400}]},"public_addr":"172.21.15.132:6801/2930324400","cluster_addr":"172.21.15.132:6803/2930324400","heartbeat_back_addr":"172.21.15.132:6807/2930324400","heartbeat_front_addr":"172.21.15.132:6805/2930324400","state":["exists","up"]},{"osd":4,"uuid":"3b41c4ff-b4b1-4279-b973-0c3c1c62373e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":72,"up_from":76,"up_thru":81,"down_at":73,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6809","nonce":682905746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6811","nonce":682905746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6815","nonce":682905746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6813","nonce":682905746}]},"public_addr":"172.21.15.132:6809/682905746","cluster_addr":"172.21.15.132:6811/682905746","heartbeat_back_addr":"172.21.15.132:6815/682905746","heartbeat_front_addr":"172.21.15.132:6813/682905746","state":["exists","up"]},{"osd":5,"uuid":"367bd67a-e45d-40ff-9aac-1f88755641ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":77,"up_from":81,"up_thru":81,"down_at":78,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6817","nonce":1555619083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6819","nonce":1555619083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6823","nonce":1555619083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6821","nonce":1555619083}]},"public_addr":"172.21.15.132:6817/1555619083","cluster_addr":"172.21.15.132:6819/1555619083","heartbeat_back_addr":"172.21.15.132:6823/1555619083","heartbeat_front_addr":"172.21.15.132:6821/1555619083","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:54:58.653733+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:26.141001+0000","dead_epoch":57},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:55.352595+0000","dead_epoch":62},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:19.994273+0000","dead_epoch":67},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:46.399893+0000","dead_epoch":72},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:57:13.114456+0000","dead_epoch":77}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6829/2325623973":"2024-02-20T19:10:28.265472+0000","172.21.15.88:6828/2325623973":"2024-02-20T19:10:28.265472+0000","172.21.15.132:6825/2841430379":"2024-02-20T19:10:28.265488+0000","172.21.15.88:0/1344352507":"2024-02-20T19:04:58.075334+0000","172.21.15.88:0/2177498492":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6801/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6801/221244694":"2024-02-20T19:04:58.075334+0000","172.21.15.88:6801/2764007082":"2024-02-20T19:00:47.495177+0000","172.21.15.132:6829/3141443905":"2024-02-20T19:01:32.237246+0000","172.21.15.88:0/3237698093":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6800/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6826/694475098":"2024-02-20T18:58:49.196048+0000","172.21.15.88:0/2389966380":"2024-02-20T19:00:47.495177+0000","172.21.15.132:0/2573781289":"2024-02-20T19:01:32.237246+0000","172.21.15.132:6828/3141443905":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6800/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/3497052224":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/3565328320":"2024-02-20T18:51:37.844957+0000","172.21.15.132:6824/2841430379":"2024-02-20T19:10:28.265488+0000","172.21.15.88:6801/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.132:0/2202935815":"2024-02-20T19:01:32.237246+0000","172.21.15.88:0/387402116":"2024-02-20T18:51:05.484896+0000","172.21.15.88:0/2372050564":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/2300938361":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/2164615684":"2024-02-20T19:04:58.075334+0000","172.21.15.132:0/2425517471":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6800/2764007082":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/3838380819":"2024-02-20T18:51:05.484896+0000","172.21.15.132:0/1356863923":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6801/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6827/694475098":"2024-02-20T18:58:49.196048+0000","172.21.15.132:0/3601890998":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6800/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.132:0/2131958733":"2024-02-20T19:01:32.237246+0000","172.21.15.88:0/2418191043":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/4069571809":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/3491684321":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/2338689461":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6800/221244694":"2024-02-20T19:04:58.075334+0000","172.21.15.88:0/4002678054":"2024-02-20T19:04:58.075334+0000","172.21.15.88:0/3163483263":"2024-02-20T19:04:58.075334+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T19:43:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.298+0000 7f1eca0ce640 1 -- 172.21.15.88:0/1500614915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb006d8d0 msgr2=0x7f1eb006fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.298+0000 7f1eca0ce640 1 --2- 172.21.15.88:0/1500614915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb006d8d0 0x7f1eb006fd80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4000c00 tx=0x7f1eb4005fd0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.298+0000 7f1eca0ce640 1 -- 172.21.15.88:0/1500614915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4101040 msgr2=0x7f1ec41957e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.298+0000 7f1eca0ce640 1 --2- 172.21.15.88:0/1500614915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4101040 0x7f1ec41957e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f1eac00c8f0 tx=0x7f1eac00cdb0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.298+0000 7f1eca0ce640 1 -- 172.21.15.88:0/1500614915 shutdown_connections 2024-02-19T19:43:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.299+0000 7f1eca0ce640 1 --2- 172.21.15.88:0/1500614915 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f1eb006d8d0 0x7f1eb006fd80 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.299+0000 7f1eca0ce640 1 --2- 172.21.15.88:0/1500614915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4101040 0x7f1ec41957e0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.299+0000 7f1eca0ce640 1 --2- 172.21.15.88:0/1500614915 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f1ec4100730 0x7f1ec41952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.299+0000 7f1eca0ce640 1 -- 172.21.15.88:0/1500614915 >> 172.21.15.88:0/1500614915 conn(0x7f1ec4075270 msgr2=0x7f1ec4103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:36.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.299+0000 7f1eca0ce640 1 -- 172.21.15.88:0/1500614915 shutdown_connections 2024-02-19T19:43:36.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.299+0000 7f1eca0ce640 1 -- 172.21.15.88:0/1500614915 wait complete. 2024-02-19T19:43:36.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T18:55:57.936009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-19T18:58:31.129412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-19T18:58:31.420377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-19T19:43:36.310 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T19:43:36.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.414+0000 7f5e7f21c640 1 -- 172.21.15.88:0/179396218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e781045c0 msgr2=0x7f5e78104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.414+0000 7f5e7f21c640 1 --2- 172.21.15.88:0/179396218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e781045c0 0x7f5e78104990 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c009880 tx=0x7f5e6c02bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.414+0000 7f5e7f21c640 1 -- 172.21.15.88:0/179396218 shutdown_connections 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.414+0000 7f5e7f21c640 1 --2- 172.21.15.88:0/179396218 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780fe590 0x7f5e780fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.414+0000 7f5e7f21c640 1 --2- 172.21.15.88:0/179396218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e781045c0 0x7f5e78104990 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.414+0000 7f5e7f21c640 1 -- 172.21.15.88:0/179396218 >> 172.21.15.88:0/179396218 conn(0x7f5e780fa0b0 msgr2=0x7f5e780fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.414+0000 7f5e7f21c640 1 -- 172.21.15.88:0/179396218 shutdown_connections 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.414+0000 7f5e7f21c640 1 -- 172.21.15.88:0/179396218 wait complete. 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e7f21c640 1 Processor -- start 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e7f21c640 1 -- start start 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e7f21c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780fe590 0x7f5e78196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e7f21c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e781045c0 0x7f5e78196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e7f21c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e78196c30 con 0x7f5e781045c0 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e7f21c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e78196da0 con 0x7f5e780fe590 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e77fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e781045c0 0x7f5e78196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:36.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e77fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e781045c0 0x7f5e78196660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57626/0 (socket says 172.21.15.88:57626) 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e77fff640 1 -- 172.21.15.88:0/3007586917 learned_addr learned my addr 172.21.15.88:0/3007586917 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e7cfb8640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780fe590 0x7f5e78196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e77fff640 1 -- 172.21.15.88:0/3007586917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780fe590 msgr2=0x7f5e78196120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e77fff640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780fe590 0x7f5e78196120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e77fff640 1 -- 172.21.15.88:0/3007586917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e6c009560 con 0x7f5e781045c0 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e7cfb8640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780fe590 0x7f5e78196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e77fff640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e781045c0 0x7f5e78196660 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400c8f0 tx=0x7f5e6400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.415+0000 7f5e75ffb640 1 -- 172.21.15.88:0/3007586917 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e64007b80 con 0x7f5e781045c0 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.416+0000 7f5e75ffb640 1 -- 172.21.15.88:0/3007586917 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e640054e0 con 0x7f5e781045c0 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.416+0000 7f5e7f21c640 1 -- 172.21.15.88:0/3007586917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e7819f190 con 0x7f5e781045c0 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.416+0000 7f5e75ffb640 1 -- 172.21.15.88:0/3007586917 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e6400f450 con 0x7f5e781045c0 2024-02-19T19:43:36.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.416+0000 7f5e7f21c640 1 -- 172.21.15.88:0/3007586917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7819f6c0 con 0x7f5e781045c0 2024-02-19T19:43:36.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.417+0000 7f5e7f21c640 1 -- 172.21.15.88:0/3007586917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7804e000 con 0x7f5e781045c0 2024-02-19T19:43:36.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.417+0000 7f5e75ffb640 1 -- 172.21.15.88:0/3007586917 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f5e64004f60 con 0x7f5e781045c0 2024-02-19T19:43:36.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.417+0000 7f5e75ffb640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6806d810 0x7f5e6806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:36.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.417+0000 7f5e75ffb640 1 -- 172.21.15.88:0/3007586917 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f5e6408d350 con 0x7f5e781045c0 2024-02-19T19:43:36.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.417+0000 7f5e7cfb8640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6806d810 0x7f5e6806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:36.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.418+0000 7f5e7cfb8640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6806d810 0x7f5e6806fcc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c000c00 tx=0x7f5e6c02cc20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:36.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.422+0000 7f5e75ffb640 1 -- 172.21.15.88:0/3007586917 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5e64056b00 con 0x7f5e781045c0 2024-02-19T19:43:36.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.594+0000 7f5e7f21c640 1 -- 172.21.15.88:0/3007586917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5e7810b980 con 0x7f5e781045c0 2024-02-19T19:43:36.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.595+0000 7f5e75ffb640 1 -- 172.21.15.88:0/3007586917 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5e6405a1c0 con 0x7f5e781045c0 2024-02-19T19:43:36.595 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-02-19T19:43:36.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.597+0000 7f5e7f21c640 1 -- 172.21.15.88:0/3007586917 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6806d810 msgr2=0x7f5e6806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.597+0000 7f5e7f21c640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6806d810 0x7f5e6806fcc0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c000c00 tx=0x7f5e6c02cc20 comp rx=0 tx=0).stop 2024-02-19T19:43:36.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.597+0000 7f5e7f21c640 1 -- 172.21.15.88:0/3007586917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e781045c0 msgr2=0x7f5e78196660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.597+0000 7f5e7f21c640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e781045c0 0x7f5e78196660 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f5e6400c8f0 tx=0x7f5e6400cdb0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.597+0000 7f5e7f21c640 1 -- 172.21.15.88:0/3007586917 shutdown_connections 2024-02-19T19:43:36.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.597+0000 7f5e7f21c640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f5e6806d810 0x7f5e6806fcc0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.597+0000 7f5e7f21c640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e781045c0 0x7f5e78196660 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.597+0000 7f5e7f21c640 1 --2- 172.21.15.88:0/3007586917 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f5e780fe590 0x7f5e78196120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.597+0000 7f5e7f21c640 1 -- 172.21.15.88:0/3007586917 >> 172.21.15.88:0/3007586917 conn(0x7f5e780fa0b0 msgr2=0x7f5e780fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:36.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.597+0000 7f5e7f21c640 1 -- 172.21.15.88:0/3007586917 shutdown_connections 2024-02-19T19:43:36.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.597+0000 7f5e7f21c640 1 -- 172.21.15.88:0/3007586917 wait complete. 2024-02-19T19:43:36.608 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-19T19:43:36.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.711+0000 7f063ee6a640 1 -- 172.21.15.88:0/4287797952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06380ff000 msgr2=0x7f0638109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.711+0000 7f063ee6a640 1 --2- 172.21.15.88:0/4287797952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06380ff000 0x7f0638109a70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f062c0098b0 tx=0x7f062c02bd20 comp rx=0 tx=0).stop 2024-02-19T19:43:36.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.711+0000 7f063ee6a640 1 -- 172.21.15.88:0/4287797952 shutdown_connections 2024-02-19T19:43:36.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.711+0000 7f063ee6a640 1 --2- 172.21.15.88:0/4287797952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06380ff000 0x7f0638109a70 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.711+0000 7f063ee6a640 1 --2- 172.21.15.88:0/4287797952 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06380fe6f0 0x7f06380feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.711+0000 7f063ee6a640 1 -- 172.21.15.88:0/4287797952 >> 172.21.15.88:0/4287797952 conn(0x7f0638073d20 msgr2=0x7f0638074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.711+0000 7f063ee6a640 1 -- 172.21.15.88:0/4287797952 shutdown_connections 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.711+0000 7f063ee6a640 1 -- 172.21.15.88:0/4287797952 wait complete. 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.711+0000 7f063ee6a640 1 Processor -- start 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f063ee6a640 1 -- start start 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f063ee6a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06380fe6f0 0x7f0638195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f063ee6a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06380ff000 0x7f06381958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f063ee6a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638195f30 con 0x7f06380ff000 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f063ee6a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0638199cb0 con 0x7f06380fe6f0 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f0637fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06380ff000 0x7f06381958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f0637fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06380ff000 0x7f06381958a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57646/0 (socket says 172.21.15.88:57646) 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f0637fff640 1 -- 172.21.15.88:0/3734315927 learned_addr learned my addr 172.21.15.88:0/3734315927 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f063cc06640 1 --2- 172.21.15.88:0/3734315927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06380fe6f0 0x7f0638195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f0637fff640 1 -- 172.21.15.88:0/3734315927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06380fe6f0 msgr2=0x7f0638195360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f0637fff640 1 --2- 172.21.15.88:0/3734315927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06380fe6f0 0x7f0638195360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f0637fff640 1 -- 172.21.15.88:0/3734315927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c009560 con 0x7f06380ff000 2024-02-19T19:43:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f0637fff640 1 --2- 172.21.15.88:0/3734315927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06380ff000 0x7f06381958a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f062c009880 tx=0x7f062c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f0635ffb640 1 -- 172.21.15.88:0/3734315927 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f062c0035f0 con 0x7f06380ff000 2024-02-19T19:43:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f0635ffb640 1 -- 172.21.15.88:0/3734315927 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f062c003c10 con 0x7f06380ff000 2024-02-19T19:43:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f063ee6a640 1 -- 172.21.15.88:0/3734315927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0638199f30 con 0x7f06380ff000 2024-02-19T19:43:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.712+0000 7f0635ffb640 1 -- 172.21.15.88:0/3734315927 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f062c034440 con 0x7f06380ff000 2024-02-19T19:43:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.713+0000 7f063ee6a640 1 -- 172.21.15.88:0/3734315927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063819a510 con 0x7f06380ff000 2024-02-19T19:43:36.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.714+0000 7f063ee6a640 1 -- 172.21.15.88:0/3734315927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063804e000 con 0x7f06380ff000 2024-02-19T19:43:36.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.714+0000 7f0635ffb640 1 -- 172.21.15.88:0/3734315927 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f062c004ff0 con 0x7f06380ff000 2024-02-19T19:43:36.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.714+0000 7f0635ffb640 1 --2- 172.21.15.88:0/3734315927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f062006d810 0x7f062006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:36.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.715+0000 7f0635ffb640 1 -- 172.21.15.88:0/3734315927 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f062c0afa00 con 0x7f06380ff000 2024-02-19T19:43:36.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.715+0000 7f063cc06640 1 --2- 172.21.15.88:0/3734315927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f062006d810 0x7f062006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:36.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.715+0000 7f063cc06640 1 --2- 172.21.15.88:0/3734315927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f062006d810 0x7f062006fcc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f062800a810 tx=0x7f0628008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.719+0000 7f0635ffb640 1 -- 172.21.15.88:0/3734315927 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f062c0791b0 con 0x7f06380ff000 2024-02-19T19:43:36.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.893+0000 7f063ee6a640 1 -- 172.21.15.88:0/3734315927 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f063810c810 con 0x7f06380ff000 2024-02-19T19:43:36.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.893+0000 7f0635ffb640 1 -- 172.21.15.88:0/3734315927 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v88) v1 ==== 107+0+11 (secure 0 0 0) 0x7f062c07c870 con 0x7f06380ff000 2024-02-19T19:43:36.894 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-02-19T19:43:36.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.895+0000 7f063ee6a640 1 -- 172.21.15.88:0/3734315927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f062006d810 msgr2=0x7f062006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.895+0000 7f063ee6a640 1 --2- 172.21.15.88:0/3734315927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f062006d810 0x7f062006fcc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f062800a810 tx=0x7f0628008040 comp rx=0 tx=0).stop 2024-02-19T19:43:36.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.895+0000 7f063ee6a640 1 -- 172.21.15.88:0/3734315927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06380ff000 msgr2=0x7f06381958a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:36.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.895+0000 7f063ee6a640 1 --2- 172.21.15.88:0/3734315927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06380ff000 0x7f06381958a0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f062c009880 tx=0x7f062c004040 comp rx=0 tx=0).stop 2024-02-19T19:43:36.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.895+0000 7f063ee6a640 1 -- 172.21.15.88:0/3734315927 shutdown_connections 2024-02-19T19:43:36.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.895+0000 7f063ee6a640 1 --2- 172.21.15.88:0/3734315927 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f062006d810 0x7f062006fcc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.895+0000 7f063ee6a640 1 --2- 172.21.15.88:0/3734315927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06380ff000 0x7f06381958a0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.895+0000 7f063ee6a640 1 --2- 172.21.15.88:0/3734315927 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f06380fe6f0 0x7f0638195360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:36.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.895+0000 7f063ee6a640 1 -- 172.21.15.88:0/3734315927 >> 172.21.15.88:0/3734315927 conn(0x7f0638073d20 msgr2=0x7f06381042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:36.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.895+0000 7f063ee6a640 1 -- 172.21.15.88:0/3734315927 shutdown_connections 2024-02-19T19:43:36.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:36.895+0000 7f063ee6a640 1 -- 172.21.15.88:0/3734315927 wait complete. 2024-02-19T19:43:36.905 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:36 smithi088 ceph-mon[88643]: pgmap v1203: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:36.905 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:36 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1500614915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T19:43:36.905 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:36 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3007586917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T19:43:36.906 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-19T19:43:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.010+0000 7f69250b9640 1 -- 172.21.15.88:0/2905958918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6920104770 msgr2=0x7f6920104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.010+0000 7f69250b9640 1 --2- 172.21.15.88:0/2905958918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6920104770 0x7f6920104b40 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f6914009880 tx=0x7f691402bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.010+0000 7f69250b9640 1 -- 172.21.15.88:0/2905958918 shutdown_connections 2024-02-19T19:43:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.010+0000 7f69250b9640 1 --2- 172.21.15.88:0/2905958918 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200fe760 0x7f69200febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.010+0000 7f69250b9640 1 --2- 172.21.15.88:0/2905958918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6920104770 0x7f6920104b40 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.010+0000 7f69250b9640 1 -- 172.21.15.88:0/2905958918 >> 172.21.15.88:0/2905958918 conn(0x7f69200fa260 msgr2=0x7f69200fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.010+0000 7f69250b9640 1 -- 172.21.15.88:0/2905958918 shutdown_connections 2024-02-19T19:43:37.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.010+0000 7f69250b9640 1 -- 172.21.15.88:0/2905958918 wait complete. 2024-02-19T19:43:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.011+0000 7f69250b9640 1 Processor -- start 2024-02-19T19:43:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.011+0000 7f69250b9640 1 -- start start 2024-02-19T19:43:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.011+0000 7f69250b9640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200fe760 0x7f6920196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.011+0000 7f69250b9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6920104770 0x7f6920196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.011+0000 7f69250b9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6920196e50 con 0x7f6920104770 2024-02-19T19:43:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.011+0000 7f69250b9640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6920196f90 con 0x7f69200fe760 2024-02-19T19:43:37.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f691e59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6920104770 0x7f6920196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f691e59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6920104770 0x7f6920196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57658/0 (socket says 172.21.15.88:57658) 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f691e59c640 1 -- 172.21.15.88:0/2693168211 learned_addr learned my addr 172.21.15.88:0/2693168211 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f691ed9d640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200fe760 0x7f6920196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f691e59c640 1 -- 172.21.15.88:0/2693168211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200fe760 msgr2=0x7f6920196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f691e59c640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200fe760 0x7f6920196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f691e59c640 1 -- 172.21.15.88:0/2693168211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6914009560 con 0x7f6920104770 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f691ed9d640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200fe760 0x7f6920196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f691e59c640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6920104770 0x7f6920196880 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f691000d8d0 tx=0x7f691000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f6907fff640 1 -- 172.21.15.88:0/2693168211 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691000bea0 con 0x7f6920104770 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f6907fff640 1 -- 172.21.15.88:0/2693168211 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6910009c00 con 0x7f6920104770 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f69250b9640 1 -- 172.21.15.88:0/2693168211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692019f3b0 con 0x7f6920104770 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f6907fff640 1 -- 172.21.15.88:0/2693168211 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691000f710 con 0x7f6920104770 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.012+0000 7f69250b9640 1 -- 172.21.15.88:0/2693168211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692019f8b0 con 0x7f6920104770 2024-02-19T19:43:37.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.013+0000 7f69250b9640 1 -- 172.21.15.88:0/2693168211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692004e000 con 0x7f6920104770 2024-02-19T19:43:37.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.014+0000 7f6907fff640 1 -- 172.21.15.88:0/2693168211 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f691000f990 con 0x7f6920104770 2024-02-19T19:43:37.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.014+0000 7f6907fff640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f690806d810 0x7f690806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.014+0000 7f6907fff640 1 -- 172.21.15.88:0/2693168211 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f691008d290 con 0x7f6920104770 2024-02-19T19:43:37.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.014+0000 7f691ed9d640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f690806d810 0x7f690806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.015+0000 7f691ed9d640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f690806d810 0x7f690806fcc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6914005ec0 tx=0x7f6914005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:37.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.019+0000 7f6907fff640 1 -- 172.21.15.88:0/2693168211 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6910056a40 con 0x7f6920104770 2024-02-19T19:43:37.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.190+0000 7f69250b9640 1 -- 172.21.15.88:0/2693168211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f6920068940 con 0x7f6920104770 2024-02-19T19:43:37.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.190+0000 7f6907fff640 1 -- 172.21.15.88:0/2693168211 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v88) v1 ==== 107+0+11 (secure 0 0 0) 0x7f691005a100 con 0x7f6920104770 2024-02-19T19:43:37.191 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-02-19T19:43:37.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.192+0000 7f69250b9640 1 -- 172.21.15.88:0/2693168211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f690806d810 msgr2=0x7f690806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.192+0000 7f69250b9640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f690806d810 0x7f690806fcc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6914005ec0 tx=0x7f6914005fd0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.192+0000 7f69250b9640 1 -- 172.21.15.88:0/2693168211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6920104770 msgr2=0x7f6920196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.192+0000 7f69250b9640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6920104770 0x7f6920196880 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f691000d8d0 tx=0x7f691000dd90 comp rx=0 tx=0).stop 2024-02-19T19:43:37.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.193+0000 7f69250b9640 1 -- 172.21.15.88:0/2693168211 shutdown_connections 2024-02-19T19:43:37.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.193+0000 7f69250b9640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f690806d810 0x7f690806fcc0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.193+0000 7f69250b9640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6920104770 0x7f6920196880 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.193+0000 7f69250b9640 1 --2- 172.21.15.88:0/2693168211 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f69200fe760 0x7f6920196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.193+0000 7f69250b9640 1 -- 172.21.15.88:0/2693168211 >> 172.21.15.88:0/2693168211 conn(0x7f69200fa260 msgr2=0x7f69200fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:37.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.193+0000 7f69250b9640 1 -- 172.21.15.88:0/2693168211 shutdown_connections 2024-02-19T19:43:37.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.193+0000 7f69250b9640 1 -- 172.21.15.88:0/2693168211 wait complete. 2024-02-19T19:43:37.204 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-19T19:43:37.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:36 smithi132 ceph-mon[70676]: pgmap v1203: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:36 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1500614915' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T19:43:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:36 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3007586917' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T19:43:37.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.304+0000 7f2cf4841640 1 -- 172.21.15.88:0/1543741009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf01047f0 msgr2=0x7f2cf0104bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.304+0000 7f2cf4841640 1 --2- 172.21.15.88:0/1543741009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf01047f0 0x7f2cf0104bc0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009880 tx=0x7f2cd802bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.305+0000 7f2cf4841640 1 -- 172.21.15.88:0/1543741009 shutdown_connections 2024-02-19T19:43:37.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.305+0000 7f2cf4841640 1 --2- 172.21.15.88:0/1543741009 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf00fe7e0 0x7f2cf00fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.305+0000 7f2cf4841640 1 --2- 172.21.15.88:0/1543741009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf01047f0 0x7f2cf0104bc0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.305+0000 7f2cf4841640 1 -- 172.21.15.88:0/1543741009 >> 172.21.15.88:0/1543741009 conn(0x7f2cf00fa300 msgr2=0x7f2cf00fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:37.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.305+0000 7f2cf4841640 1 -- 172.21.15.88:0/1543741009 shutdown_connections 2024-02-19T19:43:37.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.305+0000 7f2cf4841640 1 -- 172.21.15.88:0/1543741009 wait complete. 2024-02-19T19:43:37.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.306+0000 7f2cf4841640 1 Processor -- start 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.306+0000 7f2cf4841640 1 -- start start 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.306+0000 7f2cf4841640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf00fe7e0 0x7f2cf0194430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.306+0000 7f2cf4841640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf01047f0 0x7f2cf0194970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.306+0000 7f2cf4841640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf0195000 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.306+0000 7f2cf4841640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf0198dd0 con 0x7f2cf01047f0 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.306+0000 7f2cee59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf00fe7e0 0x7f2cf0194430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.306+0000 7f2cee59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf00fe7e0 0x7f2cf0194430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57670/0 (socket says 172.21.15.88:57670) 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.306+0000 7f2cee59c640 1 -- 172.21.15.88:0/481712212 learned_addr learned my addr 172.21.15.88:0/481712212 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.306+0000 7f2cedd9b640 1 --2- 172.21.15.88:0/481712212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf01047f0 0x7f2cf0194970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.307+0000 7f2cee59c640 1 -- 172.21.15.88:0/481712212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf01047f0 msgr2=0x7f2cf0194970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.307+0000 7f2cee59c640 1 --2- 172.21.15.88:0/481712212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf01047f0 0x7f2cf0194970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.307+0000 7f2cee59c640 1 -- 172.21.15.88:0/481712212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd8009560 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.307+0000 7f2cee59c640 1 --2- 172.21.15.88:0/481712212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf00fe7e0 0x7f2cf0194430 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009230 tx=0x7f2cd8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:37.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.307+0000 7f2ce77fe640 1 -- 172.21.15.88:0/481712212 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd80037c0 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.307+0000 7f2ce77fe640 1 -- 172.21.15.88:0/481712212 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2cd8004aa0 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.307+0000 7f2cf4841640 1 -- 172.21.15.88:0/481712212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf0199050 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.307+0000 7f2ce77fe640 1 -- 172.21.15.88:0/481712212 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd8034440 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.307+0000 7f2cf4841640 1 -- 172.21.15.88:0/481712212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf01995d0 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.308+0000 7f2cf4841640 1 -- 172.21.15.88:0/481712212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf004e000 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.308+0000 7f2ce77fe640 1 -- 172.21.15.88:0/481712212 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2cd8004f60 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.309+0000 7f2ce77fe640 1 --2- 172.21.15.88:0/481712212 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2cdc06d8d0 0x7f2cdc06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.309+0000 7f2ce77fe640 1 -- 172.21.15.88:0/481712212 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2cd80af960 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.309+0000 7f2cedd9b640 1 --2- 172.21.15.88:0/481712212 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2cdc06d8d0 0x7f2cdc06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.309+0000 7f2cedd9b640 1 --2- 172.21.15.88:0/481712212 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2cdc06d8d0 0x7f2cdc06fd80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0195a40 tx=0x7f2ce0009110 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:37.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.313+0000 7f2ce77fe640 1 -- 172.21.15.88:0/481712212 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2cd8079110 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.521+0000 7f2cf4841640 1 -- 172.21.15.88:0/481712212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cf010ba30 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.522+0000 7f2ce77fe640 1 -- 172.21.15.88:0/481712212 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+5009 (secure 0 0 0) 0x7f2cd807c7d0 con 0x7f2cf00fe7e0 2024-02-19T19:43:37.522 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:37.523 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44365,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/594928725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":594928725},{"type":"v1","addr":"172.21.15.132:6825","nonce":594928725}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44753,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6827/858155091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":858155091},{"type":"v1","addr":"172.21.15.132:6827","nonce":858155091}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:41:51.228566+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34430,"mds_1":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":0,"incarnation":34,"state":"up:active","state_seq":469,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34462":{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":1,"incarnation":34,"state":"up:active","state_seq":464,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:37.523 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 40 2024-02-19T19:43:37.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.524+0000 7f2cf4841640 1 -- 172.21.15.88:0/481712212 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2cdc06d8d0 msgr2=0x7f2cdc06fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.524+0000 7f2cf4841640 1 --2- 172.21.15.88:0/481712212 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2cdc06d8d0 0x7f2cdc06fd80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0195a40 tx=0x7f2ce0009110 comp rx=0 tx=0).stop 2024-02-19T19:43:37.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.524+0000 7f2cf4841640 1 -- 172.21.15.88:0/481712212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf00fe7e0 msgr2=0x7f2cf0194430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.524+0000 7f2cf4841640 1 --2- 172.21.15.88:0/481712212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf00fe7e0 0x7f2cf0194430 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009230 tx=0x7f2cd8004270 comp rx=0 tx=0).stop 2024-02-19T19:43:37.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.524+0000 7f2cf4841640 1 -- 172.21.15.88:0/481712212 shutdown_connections 2024-02-19T19:43:37.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.524+0000 7f2cf4841640 1 --2- 172.21.15.88:0/481712212 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2cdc06d8d0 0x7f2cdc06fd80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.524+0000 7f2cf4841640 1 --2- 172.21.15.88:0/481712212 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2cf01047f0 0x7f2cf0194970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.524+0000 7f2cf4841640 1 --2- 172.21.15.88:0/481712212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2cf00fe7e0 0x7f2cf0194430 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.524+0000 7f2cf4841640 1 -- 172.21.15.88:0/481712212 >> 172.21.15.88:0/481712212 conn(0x7f2cf00fa300 msgr2=0x7f2cf00fbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:37.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.524+0000 7f2cf4841640 1 -- 172.21.15.88:0/481712212 shutdown_connections 2024-02-19T19:43:37.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.524+0000 7f2cf4841640 1 -- 172.21.15.88:0/481712212 wait complete. 2024-02-19T19:43:37.535 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 18, 'max_mds': 2, 'flags': 18} 2024-02-19T19:43:37.535 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-02-19T19:43:37.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.637+0000 7f8048803640 1 -- 172.21.15.88:0/175383302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80440fedc0 msgr2=0x7f80440ff230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.637+0000 7f8048803640 1 --2- 172.21.15.88:0/175383302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80440fedc0 0x7f80440ff230 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8038009880 tx=0x7f803802bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.637+0000 7f8048803640 1 -- 172.21.15.88:0/175383302 shutdown_connections 2024-02-19T19:43:37.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.637+0000 7f8048803640 1 --2- 172.21.15.88:0/175383302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80440fedc0 0x7f80440ff230 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.637+0000 7f8048803640 1 --2- 172.21.15.88:0/175383302 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80441034f0 0x7f80441038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.637+0000 7f8048803640 1 -- 172.21.15.88:0/175383302 >> 172.21.15.88:0/175383302 conn(0x7f80440743d0 msgr2=0x7f80440747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:37.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.637+0000 7f8048803640 1 -- 172.21.15.88:0/175383302 shutdown_connections 2024-02-19T19:43:37.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.637+0000 7f8048803640 1 -- 172.21.15.88:0/175383302 wait complete. 2024-02-19T19:43:37.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.638+0000 7f8048803640 1 Processor -- start 2024-02-19T19:43:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.638+0000 7f8048803640 1 -- start start 2024-02-19T19:43:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.638+0000 7f8048803640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440fedc0 0x7f80441952a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.638+0000 7f8048803640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80441034f0 0x7f80441957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.638+0000 7f8048803640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044195e70 con 0x7f80441034f0 2024-02-19T19:43:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.638+0000 7f8048803640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044199c40 con 0x7f80440fedc0 2024-02-19T19:43:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f804259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80441034f0 0x7f80441957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f804259c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80441034f0 0x7f80441957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57674/0 (socket says 172.21.15.88:57674) 2024-02-19T19:43:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f804259c640 1 -- 172.21.15.88:0/1797152412 learned_addr learned my addr 172.21.15.88:0/1797152412 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f8042d9d640 1 --2- 172.21.15.88:0/1797152412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440fedc0 0x7f80441952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f804259c640 1 -- 172.21.15.88:0/1797152412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440fedc0 msgr2=0x7f80441952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f804259c640 1 --2- 172.21.15.88:0/1797152412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440fedc0 0x7f80441952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f804259c640 1 -- 172.21.15.88:0/1797152412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8038009560 con 0x7f80441034f0 2024-02-19T19:43:37.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f804259c640 1 --2- 172.21.15.88:0/1797152412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80441034f0 0x7f80441957e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8038009230 tx=0x7f8038004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:37.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f802bfff640 1 -- 172.21.15.88:0/1797152412 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80380035f0 con 0x7f80441034f0 2024-02-19T19:43:37.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f802bfff640 1 -- 172.21.15.88:0/1797152412 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8038003c10 con 0x7f80441034f0 2024-02-19T19:43:37.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f8048803640 1 -- 172.21.15.88:0/1797152412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8044199ec0 con 0x7f80441034f0 2024-02-19T19:43:37.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f802bfff640 1 -- 172.21.15.88:0/1797152412 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8038034440 con 0x7f80441034f0 2024-02-19T19:43:37.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.639+0000 7f8048803640 1 -- 172.21.15.88:0/1797152412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804419a3c0 con 0x7f80441034f0 2024-02-19T19:43:37.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.640+0000 7f8048803640 1 -- 172.21.15.88:0/1797152412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804404e000 con 0x7f80441034f0 2024-02-19T19:43:37.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.641+0000 7f802bfff640 1 -- 172.21.15.88:0/1797152412 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f8038004fd0 con 0x7f80441034f0 2024-02-19T19:43:37.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.641+0000 7f802bfff640 1 --2- 172.21.15.88:0/1797152412 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802c06d810 0x7f802c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.641+0000 7f8042d9d640 1 --2- 172.21.15.88:0/1797152412 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802c06d810 0x7f802c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.641+0000 7f802bfff640 1 -- 172.21.15.88:0/1797152412 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f80380af9c0 con 0x7f80441034f0 2024-02-19T19:43:37.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.642+0000 7f8042d9d640 1 --2- 172.21.15.88:0/1797152412 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802c06d810 0x7f802c06fcc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8034009f20 tx=0x7f80340094f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:37.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.646+0000 7f802bfff640 1 -- 172.21.15.88:0/1797152412 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8038079170 con 0x7f80441034f0 2024-02-19T19:43:37.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.865+0000 7f8048803640 1 -- 172.21.15.88:0/1797152412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7f8044107720 con 0x7f80441034f0 2024-02-19T19:43:37.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.866+0000 7f802bfff640 1 -- 172.21.15.88:0/1797152412 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v40) v1 ==== 107+0+4847 (secure 0 0 0) 0x7f803807c830 con 0x7f80441034f0 2024-02-19T19:43:37.866 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:37.866 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14522,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/2124144962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2124144962},{"type":"v1","addr":"172.21.15.88:6827","nonce":2124144962}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:04:10.337219+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14510,"mds_1":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14510":{"gid":14510,"name":"cephfs.smithi088.psjiah","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6829/2325623973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2325623973},{"type":"v1","addr":"172.21.15.88:6829","nonce":2325623973}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24263":{"gid":24263,"name":"cephfs.smithi132.afhgdi","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.132:6825/2841430379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2841430379},{"type":"v1","addr":"172.21.15.132:6825","nonce":2841430379}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:37.866 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 19 2024-02-19T19:43:37.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.868+0000 7f8048803640 1 -- 172.21.15.88:0/1797152412 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802c06d810 msgr2=0x7f802c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.868+0000 7f8048803640 1 --2- 172.21.15.88:0/1797152412 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802c06d810 0x7f802c06fcc0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f8034009f20 tx=0x7f80340094f0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.868+0000 7f8048803640 1 -- 172.21.15.88:0/1797152412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80441034f0 msgr2=0x7f80441957e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.868+0000 7f8048803640 1 --2- 172.21.15.88:0/1797152412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80441034f0 0x7f80441957e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8038009230 tx=0x7f8038004040 comp rx=0 tx=0).stop 2024-02-19T19:43:37.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.868+0000 7f8048803640 1 -- 172.21.15.88:0/1797152412 shutdown_connections 2024-02-19T19:43:37.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.868+0000 7f8048803640 1 --2- 172.21.15.88:0/1797152412 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f802c06d810 0x7f802c06fcc0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.868+0000 7f8048803640 1 --2- 172.21.15.88:0/1797152412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80441034f0 0x7f80441957e0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.868+0000 7f8048803640 1 --2- 172.21.15.88:0/1797152412 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f80440fedc0 0x7f80441952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.868+0000 7f8048803640 1 -- 172.21.15.88:0/1797152412 >> 172.21.15.88:0/1797152412 conn(0x7f80440743d0 msgr2=0x7f8044102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:37.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.868+0000 7f8048803640 1 -- 172.21.15.88:0/1797152412 shutdown_connections 2024-02-19T19:43:37.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.868+0000 7f8048803640 1 -- 172.21.15.88:0/1797152412 wait complete. 2024-02-19T19:43:37.879 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-02-19T19:43:37.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.981+0000 7f616c389640 1 -- 172.21.15.88:0/2765832405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61640fe6e0 msgr2=0x7f61640feab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.981+0000 7f616c389640 1 --2- 172.21.15.88:0/2765832405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61640fe6e0 0x7f61640feab0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f6160009880 tx=0x7f616002bd90 comp rx=0 tx=0).stop 2024-02-19T19:43:37.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.981+0000 7f616c389640 1 -- 172.21.15.88:0/2765832405 shutdown_connections 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.981+0000 7f616c389640 1 --2- 172.21.15.88:0/2765832405 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61640feff0 0x7f6164109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.981+0000 7f616c389640 1 --2- 172.21.15.88:0/2765832405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61640fe6e0 0x7f61640feab0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.981+0000 7f616c389640 1 -- 172.21.15.88:0/2765832405 >> 172.21.15.88:0/2765832405 conn(0x7f6164074cf0 msgr2=0x7f61640750f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.981+0000 7f616c389640 1 -- 172.21.15.88:0/2765832405 shutdown_connections 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.981+0000 7f616c389640 1 -- 172.21.15.88:0/2765832405 wait complete. 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616c389640 1 Processor -- start 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616c389640 1 -- start start 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616c389640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61640fe6e0 0x7f61641952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616c389640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61640feff0 0x7f61641957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616c389640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164195df0 con 0x7f61640fe6e0 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616c389640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164195f60 con 0x7f61640feff0 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616a125640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61640fe6e0 0x7f61641952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616a125640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61640fe6e0 0x7f61641952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57694/0 (socket says 172.21.15.88:57694) 2024-02-19T19:43:37.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616a125640 1 -- 172.21.15.88:0/4164001120 learned_addr learned my addr 172.21.15.88:0/4164001120 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:37.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f6169924640 1 --2- 172.21.15.88:0/4164001120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61640feff0 0x7f61641957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616a125640 1 -- 172.21.15.88:0/4164001120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61640feff0 msgr2=0x7f61641957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:37.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616a125640 1 --2- 172.21.15.88:0/4164001120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61640feff0 0x7f61641957f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:37.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.982+0000 7f616a125640 1 -- 172.21.15.88:0/4164001120 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6160009560 con 0x7f61640fe6e0 2024-02-19T19:43:37.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.983+0000 7f616a125640 1 --2- 172.21.15.88:0/4164001120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61640fe6e0 0x7f61641952b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6160036040 tx=0x7f6160004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:37.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.983+0000 7f615b7fe640 1 -- 172.21.15.88:0/4164001120 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61600035f0 con 0x7f61640fe6e0 2024-02-19T19:43:37.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.983+0000 7f615b7fe640 1 -- 172.21.15.88:0/4164001120 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6160003c10 con 0x7f61640fe6e0 2024-02-19T19:43:37.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.983+0000 7f615b7fe640 1 -- 172.21.15.88:0/4164001120 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f616003e450 con 0x7f61640fe6e0 2024-02-19T19:43:37.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.983+0000 7f616c389640 1 -- 172.21.15.88:0/4164001120 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6164199d40 con 0x7f61640fe6e0 2024-02-19T19:43:37.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.983+0000 7f616c389640 1 -- 172.21.15.88:0/4164001120 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f616419a320 con 0x7f61640fe6e0 2024-02-19T19:43:37.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.984+0000 7f616c389640 1 -- 172.21.15.88:0/4164001120 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f616404e000 con 0x7f61640fe6e0 2024-02-19T19:43:37.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.984+0000 7f615b7fe640 1 -- 172.21.15.88:0/4164001120 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f6160004f60 con 0x7f61640fe6e0 2024-02-19T19:43:37.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.985+0000 7f615b7fe640 1 --2- 172.21.15.88:0/4164001120 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f615006d810 0x7f615006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:37.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.985+0000 7f6169924640 1 --2- 172.21.15.88:0/4164001120 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f615006d810 0x7f615006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:37.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.985+0000 7f615b7fe640 1 -- 172.21.15.88:0/4164001120 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f61600b0290 con 0x7f61640fe6e0 2024-02-19T19:43:37.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.985+0000 7f6169924640 1 --2- 172.21.15.88:0/4164001120 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f615006d810 0x7f615006fcc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6154005e10 tx=0x7f6154005da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:37.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:37.989+0000 7f615b7fe640 1 -- 172.21.15.88:0/4164001120 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6160079be0 con 0x7f61640fe6e0 2024-02-19T19:43:38.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:37 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3734315927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-19T19:43:38.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:37 smithi088 ceph-mon[88643]: pgmap v1204: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:38.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:37 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2693168211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-19T19:43:38.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:37 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/481712212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T19:43:38.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.198+0000 7f616c389640 1 -- 172.21.15.88:0/4164001120 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f61641070d0 con 0x7f61640fe6e0 2024-02-19T19:43:38.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.199+0000 7f615b7fe640 1 -- 172.21.15.88:0/4164001120 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v40) v1 ==== 107+0+4848 (secure 0 0 0) 0x7f6160003d80 con 0x7f61640fe6e0 2024-02-19T19:43:38.199 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:38.199 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14522,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/2124144962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2124144962},{"type":"v1","addr":"172.21.15.88:6827","nonce":2124144962}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:05:14.913778+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14510,"mds_1":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14510":{"gid":14510,"name":"cephfs.smithi088.psjiah","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6829/2325623973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2325623973},{"type":"v1","addr":"172.21.15.88:6829","nonce":2325623973}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24263":{"gid":24263,"name":"cephfs.smithi132.afhgdi","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.132:6825/2841430379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2841430379},{"type":"v1","addr":"172.21.15.132:6825","nonce":2841430379}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:38.200 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-02-19T19:43:38.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.201+0000 7f616c389640 1 -- 172.21.15.88:0/4164001120 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f615006d810 msgr2=0x7f615006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.201+0000 7f616c389640 1 --2- 172.21.15.88:0/4164001120 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f615006d810 0x7f615006fcc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6154005e10 tx=0x7f6154005da0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.201+0000 7f616c389640 1 -- 172.21.15.88:0/4164001120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61640fe6e0 msgr2=0x7f61641952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.201+0000 7f616c389640 1 --2- 172.21.15.88:0/4164001120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61640fe6e0 0x7f61641952b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6160036040 tx=0x7f6160004040 comp rx=0 tx=0).stop 2024-02-19T19:43:38.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.201+0000 7f616c389640 1 -- 172.21.15.88:0/4164001120 shutdown_connections 2024-02-19T19:43:38.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.201+0000 7f616c389640 1 --2- 172.21.15.88:0/4164001120 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f615006d810 0x7f615006fcc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.201+0000 7f616c389640 1 --2- 172.21.15.88:0/4164001120 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f61640feff0 0x7f61641957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.201+0000 7f616c389640 1 --2- 172.21.15.88:0/4164001120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61640fe6e0 0x7f61641952b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.201+0000 7f616c389640 1 -- 172.21.15.88:0/4164001120 >> 172.21.15.88:0/4164001120 conn(0x7f6164074cf0 msgr2=0x7f61641042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:38.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.201+0000 7f616c389640 1 -- 172.21.15.88:0/4164001120 shutdown_connections 2024-02-19T19:43:38.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.201+0000 7f616c389640 1 -- 172.21.15.88:0/4164001120 wait complete. 2024-02-19T19:43:38.212 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-02-19T19:43:38.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:37 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3734315927' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-19T19:43:38.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:37 smithi132 ceph-mon[70676]: pgmap v1204: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:38.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:37 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2693168211' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-19T19:43:38.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:37 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/481712212' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-19T19:43:38.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.313+0000 7f20eb069640 1 -- 172.21.15.88:0/296292765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e4105880 msgr2=0x7f20e4105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.313+0000 7f20eb069640 1 --2- 172.21.15.88:0/296292765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e4105880 0x7f20e4105c50 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009880 tx=0x7f20d802bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.313+0000 7f20eb069640 1 -- 172.21.15.88:0/296292765 shutdown_connections 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.313+0000 7f20eb069640 1 --2- 172.21.15.88:0/296292765 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e40ff870 0x7f20e40ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.313+0000 7f20eb069640 1 --2- 172.21.15.88:0/296292765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e4105880 0x7f20e4105c50 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.313+0000 7f20eb069640 1 -- 172.21.15.88:0/296292765 >> 172.21.15.88:0/296292765 conn(0x7f20e40fb370 msgr2=0x7f20e40fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.313+0000 7f20eb069640 1 -- 172.21.15.88:0/296292765 shutdown_connections 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.313+0000 7f20eb069640 1 -- 172.21.15.88:0/296292765 wait complete. 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20eb069640 1 Processor -- start 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20eb069640 1 -- start start 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20eb069640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e40ff870 0x7f20e41954e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20eb069640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e4105880 0x7f20e4195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20eb069640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e41960b0 con 0x7f20e40ff870 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20eb069640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e4199e80 con 0x7f20e4105880 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20e8e05640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e40ff870 0x7f20e41954e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:38.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20e8e05640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e40ff870 0x7f20e41954e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57712/0 (socket says 172.21.15.88:57712) 2024-02-19T19:43:38.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20e8e05640 1 -- 172.21.15.88:0/3602746642 learned_addr learned my addr 172.21.15.88:0/3602746642 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:38.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20e8e05640 1 -- 172.21.15.88:0/3602746642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e4105880 msgr2=0x7f20e4195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20e8e05640 1 --2- 172.21.15.88:0/3602746642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e4105880 0x7f20e4195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20e8e05640 1 -- 172.21.15.88:0/3602746642 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d8009560 con 0x7f20e40ff870 2024-02-19T19:43:38.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20e8e05640 1 --2- 172.21.15.88:0/3602746642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e40ff870 0x7f20e41954e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009230 tx=0x7f20d802c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:38.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.314+0000 7f20e1ffb640 1 -- 172.21.15.88:0/3602746642 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20d80035f0 con 0x7f20e40ff870 2024-02-19T19:43:38.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.315+0000 7f20e1ffb640 1 -- 172.21.15.88:0/3602746642 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20d8003c10 con 0x7f20e40ff870 2024-02-19T19:43:38.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.315+0000 7f20e1ffb640 1 -- 172.21.15.88:0/3602746642 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20d8005690 con 0x7f20e40ff870 2024-02-19T19:43:38.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.315+0000 7f20eb069640 1 -- 172.21.15.88:0/3602746642 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20e419a100 con 0x7f20e40ff870 2024-02-19T19:43:38.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.315+0000 7f20eb069640 1 -- 172.21.15.88:0/3602746642 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e419a630 con 0x7f20e40ff870 2024-02-19T19:43:38.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.316+0000 7f20eb069640 1 -- 172.21.15.88:0/3602746642 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e404e000 con 0x7f20e40ff870 2024-02-19T19:43:38.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.316+0000 7f20e1ffb640 1 -- 172.21.15.88:0/3602746642 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f20d8004ed0 con 0x7f20e40ff870 2024-02-19T19:43:38.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.317+0000 7f20e1ffb640 1 --2- 172.21.15.88:0/3602746642 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20d406d810 0x7f20d406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:38.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.317+0000 7f20e1ffb640 1 -- 172.21.15.88:0/3602746642 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f20d80af960 con 0x7f20e40ff870 2024-02-19T19:43:38.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.317+0000 7f20e3fff640 1 --2- 172.21.15.88:0/3602746642 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20d406d810 0x7f20d406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:38.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.317+0000 7f20e3fff640 1 --2- 172.21.15.88:0/3602746642 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20d406d810 0x7f20d406fcc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005fd0 tx=0x7f20d0005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:38.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.321+0000 7f20e1ffb640 1 -- 172.21.15.88:0/3602746642 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f20d8079110 con 0x7f20e40ff870 2024-02-19T19:43:38.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.530+0000 7f20eb069640 1 -- 172.21.15.88:0/3602746642 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f20e4066860 con 0x7f20e40ff870 2024-02-19T19:43:38.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.530+0000 7f20e1ffb640 1 -- 172.21.15.88:0/3602746642 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v40) v1 ==== 107+0+4847 (secure 0 0 0) 0x7f20d807c7d0 con 0x7f20e40ff870 2024-02-19T19:43:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:38.531 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14522,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/2124144962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2124144962},{"type":"v1","addr":"172.21.15.88:6827","nonce":2124144962}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:04.398801+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14510,"mds_1":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14510":{"gid":14510,"name":"cephfs.smithi088.psjiah","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6829/2325623973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2325623973},{"type":"v1","addr":"172.21.15.88:6829","nonce":2325623973}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24263":{"gid":24263,"name":"cephfs.smithi132.afhgdi","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.132:6825/2841430379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2841430379},{"type":"v1","addr":"172.21.15.132:6825","nonce":2841430379}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:38.531 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 21 2024-02-19T19:43:38.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.532+0000 7f20eb069640 1 -- 172.21.15.88:0/3602746642 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20d406d810 msgr2=0x7f20d406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.532+0000 7f20eb069640 1 --2- 172.21.15.88:0/3602746642 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20d406d810 0x7f20d406fcc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005fd0 tx=0x7f20d0005850 comp rx=0 tx=0).stop 2024-02-19T19:43:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.532+0000 7f20eb069640 1 -- 172.21.15.88:0/3602746642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e40ff870 msgr2=0x7f20e41954e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.532+0000 7f20eb069640 1 --2- 172.21.15.88:0/3602746642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e40ff870 0x7f20e41954e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f20d8009230 tx=0x7f20d802c850 comp rx=0 tx=0).stop 2024-02-19T19:43:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.532+0000 7f20eb069640 1 -- 172.21.15.88:0/3602746642 shutdown_connections 2024-02-19T19:43:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.532+0000 7f20eb069640 1 --2- 172.21.15.88:0/3602746642 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f20d406d810 0x7f20d406fcc0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.532+0000 7f20eb069640 1 --2- 172.21.15.88:0/3602746642 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f20e4105880 0x7f20e4195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.532+0000 7f20eb069640 1 --2- 172.21.15.88:0/3602746642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e40ff870 0x7f20e41954e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.532+0000 7f20eb069640 1 -- 172.21.15.88:0/3602746642 >> 172.21.15.88:0/3602746642 conn(0x7f20e40fb370 msgr2=0x7f20e40fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.532+0000 7f20eb069640 1 -- 172.21.15.88:0/3602746642 shutdown_connections 2024-02-19T19:43:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.532+0000 7f20eb069640 1 -- 172.21.15.88:0/3602746642 wait complete. 2024-02-19T19:43:38.544 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.646+0000 7f22db681640 1 -- 172.21.15.88:0/2325687210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4105680 msgr2=0x7f22d4105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.646+0000 7f22db681640 1 --2- 172.21.15.88:0/2325687210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4105680 0x7f22d4105a50 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009c20 tx=0x7f22d002c1b0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.646+0000 7f22db681640 1 -- 172.21.15.88:0/2325687210 shutdown_connections 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.646+0000 7f22db681640 1 --2- 172.21.15.88:0/2325687210 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d40ff670 0x7f22d40ffae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.646+0000 7f22db681640 1 --2- 172.21.15.88:0/2325687210 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d4105680 0x7f22d4105a50 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.646+0000 7f22db681640 1 -- 172.21.15.88:0/2325687210 >> 172.21.15.88:0/2325687210 conn(0x7f22d40fb190 msgr2=0x7f22d40fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.646+0000 7f22db681640 1 -- 172.21.15.88:0/2325687210 shutdown_connections 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.646+0000 7f22db681640 1 -- 172.21.15.88:0/2325687210 wait complete. 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22db681640 1 Processor -- start 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22db681640 1 -- start start 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22db681640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d40ff670 0x7f22d41952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22db681640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d4105680 0x7f22d4195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:38.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22db681640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4195e90 con 0x7f22d40ff670 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22db681640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d4199c60 con 0x7f22d4105680 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22d941d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d40ff670 0x7f22d41952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22d941d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d40ff670 0x7f22d41952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57724/0 (socket says 172.21.15.88:57724) 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22d941d640 1 -- 172.21.15.88:0/794787860 learned_addr learned my addr 172.21.15.88:0/794787860 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22d8c1c640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d4105680 0x7f22d4195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22d941d640 1 -- 172.21.15.88:0/794787860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d4105680 msgr2=0x7f22d4195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.647+0000 7f22d941d640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d4105680 0x7f22d4195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.648+0000 7f22d941d640 1 -- 172.21.15.88:0/794787860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d0009900 con 0x7f22d40ff670 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.648+0000 7f22d8c1c640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d4105680 0x7f22d4195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.648+0000 7f22d941d640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d40ff670 0x7f22d41952c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009430 tx=0x7f22d0032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.648+0000 7f22ca7fc640 1 -- 172.21.15.88:0/794787860 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22d0032ed0 con 0x7f22d40ff670 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.648+0000 7f22db681640 1 -- 172.21.15.88:0/794787860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d4199ee0 con 0x7f22d40ff670 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.648+0000 7f22ca7fc640 1 -- 172.21.15.88:0/794787860 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22d0003e30 con 0x7f22d40ff670 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.648+0000 7f22ca7fc640 1 -- 172.21.15.88:0/794787860 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22d003f440 con 0x7f22d40ff670 2024-02-19T19:43:38.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.648+0000 7f22db681640 1 -- 172.21.15.88:0/794787860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d419a3e0 con 0x7f22d40ff670 2024-02-19T19:43:38.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.649+0000 7f22db681640 1 -- 172.21.15.88:0/794787860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d404e000 con 0x7f22d40ff670 2024-02-19T19:43:38.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.649+0000 7f22ca7fc640 1 -- 172.21.15.88:0/794787860 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f22d003b040 con 0x7f22d40ff670 2024-02-19T19:43:38.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.650+0000 7f22ca7fc640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22c006d810 0x7f22c006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:38.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.650+0000 7f22ca7fc640 1 -- 172.21.15.88:0/794787860 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f22d00b0230 con 0x7f22d40ff670 2024-02-19T19:43:38.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.650+0000 7f22d8c1c640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22c006d810 0x7f22c006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:38.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.650+0000 7f22d8c1c640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22c006d810 0x7f22c006fcc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f22c4005d00 tx=0x7f22c4005ed0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:38.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.654+0000 7f22ca7fc640 1 -- 172.21.15.88:0/794787860 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f22d0079aa0 con 0x7f22d40ff670 2024-02-19T19:43:38.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.865+0000 7f22db681640 1 -- 172.21.15.88:0/794787860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f22d410cab0 con 0x7f22d40ff670 2024-02-19T19:43:38.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.866+0000 7f22ca7fc640 1 -- 172.21.15.88:0/794787860 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v40) v1 ==== 107+0+4846 (secure 0 0 0) 0x7f22d007d160 con 0x7f22d40ff670 2024-02-19T19:43:38.866 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:38.866 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14522,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/2124144962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2124144962},{"type":"v1","addr":"172.21.15.88:6827","nonce":2124144962}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:25.061879+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14510,"mds_1":24263},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14510":{"gid":14510,"name":"cephfs.smithi088.psjiah","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.88:6829/2325623973","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":2325623973},{"type":"v1","addr":"172.21.15.88:6829","nonce":2325623973}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24263":{"gid":24263,"name":"cephfs.smithi132.afhgdi","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.132:6825/2841430379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":2841430379},{"type":"v1","addr":"172.21.15.132:6825","nonce":2841430379}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:38.867 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 22 2024-02-19T19:43:38.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.868+0000 7f22db681640 1 -- 172.21.15.88:0/794787860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22c006d810 msgr2=0x7f22c006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.868+0000 7f22db681640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22c006d810 0x7f22c006fcc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f22c4005d00 tx=0x7f22c4005ed0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.868+0000 7f22db681640 1 -- 172.21.15.88:0/794787860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d40ff670 msgr2=0x7f22d41952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.868+0000 7f22db681640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d40ff670 0x7f22d41952c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f22d0009430 tx=0x7f22d0032b60 comp rx=0 tx=0).stop 2024-02-19T19:43:38.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.868+0000 7f22db681640 1 -- 172.21.15.88:0/794787860 shutdown_connections 2024-02-19T19:43:38.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.868+0000 7f22db681640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f22c006d810 0x7f22c006fcc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.868+0000 7f22db681640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f22d4105680 0x7f22d4195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.868+0000 7f22db681640 1 --2- 172.21.15.88:0/794787860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22d40ff670 0x7f22d41952c0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.868+0000 7f22db681640 1 -- 172.21.15.88:0/794787860 >> 172.21.15.88:0/794787860 conn(0x7f22d40fb190 msgr2=0x7f22d40fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:38.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.868+0000 7f22db681640 1 -- 172.21.15.88:0/794787860 shutdown_connections 2024-02-19T19:43:38.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.868+0000 7f22db681640 1 -- 172.21.15.88:0/794787860 wait complete. 2024-02-19T19:43:38.879 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-02-19T19:43:38.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.981+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/332612148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcc0104590 msgr2=0x7fbcc0104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.981+0000 7fbcc6a0b640 1 --2- 172.21.15.88:0/332612148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcc0104590 0x7fbcc0104960 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009880 tx=0x7fbcb002bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.981+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/332612148 shutdown_connections 2024-02-19T19:43:38.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.981+0000 7fbcc6a0b640 1 --2- 172.21.15.88:0/332612148 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc00fe5b0 0x7fbcc00fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.981+0000 7fbcc6a0b640 1 --2- 172.21.15.88:0/332612148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcc0104590 0x7fbcc0104960 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.981+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/332612148 >> 172.21.15.88:0/332612148 conn(0x7fbcc00743d0 msgr2=0x7fbcc00747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:38.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.981+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/332612148 shutdown_connections 2024-02-19T19:43:38.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.981+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/332612148 wait complete. 2024-02-19T19:43:38.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.982+0000 7fbcc6a0b640 1 Processor -- start 2024-02-19T19:43:38.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.982+0000 7fbcc6a0b640 1 -- start start 2024-02-19T19:43:38.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.982+0000 7fbcc6a0b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcc00fe5b0 0x7fbcc01960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:38.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.982+0000 7fbcc6a0b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc0104590 0x7fbcc0196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:38.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.982+0000 7fbcc6a0b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc0196d00 con 0x7fbcc00fe5b0 2024-02-19T19:43:38.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.982+0000 7fbcc6a0b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc0196e70 con 0x7fbcc0104590 2024-02-19T19:43:38.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.982+0000 7fbcbffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcc00fe5b0 0x7fbcc01960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:38.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.982+0000 7fbcbffff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcc00fe5b0 0x7fbcc01960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57750/0 (socket says 172.21.15.88:57750) 2024-02-19T19:43:38.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.982+0000 7fbcbffff640 1 -- 172.21.15.88:0/3601432234 learned_addr learned my addr 172.21.15.88:0/3601432234 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.983+0000 7fbcbf7fe640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc0104590 0x7fbcc0196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.983+0000 7fbcbffff640 1 -- 172.21.15.88:0/3601432234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc0104590 msgr2=0x7fbcc0196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.983+0000 7fbcbffff640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc0104590 0x7fbcc0196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.983+0000 7fbcbffff640 1 -- 172.21.15.88:0/3601432234 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb0009560 con 0x7fbcc00fe5b0 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.983+0000 7fbcbf7fe640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc0104590 0x7fbcc0196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.983+0000 7fbcbffff640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcc00fe5b0 0x7fbcc01960e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009230 tx=0x7fbcb0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.983+0000 7fbcbd7fa640 1 -- 172.21.15.88:0/3601432234 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcb00035f0 con 0x7fbcc00fe5b0 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.983+0000 7fbcbd7fa640 1 -- 172.21.15.88:0/3601432234 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbcb0003c10 con 0x7fbcc00fe5b0 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.983+0000 7fbcbd7fa640 1 -- 172.21.15.88:0/3601432234 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcb0034440 con 0x7fbcc00fe5b0 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.983+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/3601432234 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc019f150 con 0x7fbcc00fe5b0 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.983+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/3601432234 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc019f5d0 con 0x7fbcc00fe5b0 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.984+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/3601432234 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc004e000 con 0x7fbcc00fe5b0 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.984+0000 7fbcbd7fa640 1 -- 172.21.15.88:0/3601432234 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbcb0004fd0 con 0x7fbcc00fe5b0 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.985+0000 7fbcbd7fa640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbca806d810 0x7fbca806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:38.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.985+0000 7fbcbf7fe640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbca806d810 0x7fbca806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:38.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.985+0000 7fbcbd7fa640 1 -- 172.21.15.88:0/3601432234 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbcb00af9c0 con 0x7fbcc00fe5b0 2024-02-19T19:43:38.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.985+0000 7fbcbf7fe640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbca806d810 0x7fbca806fcc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00683f0 tx=0x7fbcb4005ec0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:38.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:38.990+0000 7fbcbd7fa640 1 -- 172.21.15.88:0/3601432234 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbcb0079170 con 0x7fbcc00fe5b0 2024-02-19T19:43:39.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:38 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1797152412' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-19T19:43:39.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:38 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4164001120' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-19T19:43:39.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:38 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3602746642' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-19T19:43:39.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.202+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/3601432234 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fbcc0067920 con 0x7fbcc00fe5b0 2024-02-19T19:43:39.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.202+0000 7fbcbd7fa640 1 -- 172.21.15.88:0/3601432234 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v40) v1 ==== 107+0+3264 (secure 0 0 0) 0x7fbcb0041cd0 con 0x7fbcc00fe5b0 2024-02-19T19:43:39.202 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:39.203 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14522,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/2124144962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2124144962},{"type":"v1","addr":"172.21.15.88:6827","nonce":2124144962}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:28.265493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:39.203 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 23 2024-02-19T19:43:39.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.204+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/3601432234 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbca806d810 msgr2=0x7fbca806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.204+0000 7fbcc6a0b640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbca806d810 0x7fbca806fcc0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00683f0 tx=0x7fbcb4005ec0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.204+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/3601432234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcc00fe5b0 msgr2=0x7fbcc01960e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.204+0000 7fbcc6a0b640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcc00fe5b0 0x7fbcc01960e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0009230 tx=0x7fbcb0004040 comp rx=0 tx=0).stop 2024-02-19T19:43:39.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.204+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/3601432234 shutdown_connections 2024-02-19T19:43:39.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.204+0000 7fbcc6a0b640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbca806d810 0x7fbca806fcc0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.204+0000 7fbcc6a0b640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbcc0104590 0x7fbcc0196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.204+0000 7fbcc6a0b640 1 --2- 172.21.15.88:0/3601432234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcc00fe5b0 0x7fbcc01960e0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.204+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/3601432234 >> 172.21.15.88:0/3601432234 conn(0x7fbcc00743d0 msgr2=0x7fbcc00fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:39.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.204+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/3601432234 shutdown_connections 2024-02-19T19:43:39.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.204+0000 7fbcc6a0b640 1 -- 172.21.15.88:0/3601432234 wait complete. 2024-02-19T19:43:39.215 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-19T19:43:39.215 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-02-19T19:43:39.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:38 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1797152412' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-02-19T19:43:39.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:38 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4164001120' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-02-19T19:43:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:38 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3602746642' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-02-19T19:43:39.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.317+0000 7f620b8b6640 1 -- 172.21.15.88:0/2261563277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62041045d0 msgr2=0x7f62041049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.317+0000 7f620b8b6640 1 --2- 172.21.15.88:0/2261563277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62041045d0 0x7f62041049a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009c20 tx=0x7f61fc02c1b0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.317+0000 7f620b8b6640 1 -- 172.21.15.88:0/2261563277 shutdown_connections 2024-02-19T19:43:39.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.317+0000 7f620b8b6640 1 --2- 172.21.15.88:0/2261563277 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62040fe5a0 0x7f62040fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.317+0000 7f620b8b6640 1 --2- 172.21.15.88:0/2261563277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62041045d0 0x7f62041049a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.317+0000 7f620b8b6640 1 -- 172.21.15.88:0/2261563277 >> 172.21.15.88:0/2261563277 conn(0x7f62040fa060 msgr2=0x7f62040fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:39.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.317+0000 7f620b8b6640 1 -- 172.21.15.88:0/2261563277 shutdown_connections 2024-02-19T19:43:39.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.317+0000 7f620b8b6640 1 -- 172.21.15.88:0/2261563277 wait complete. 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f620b8b6640 1 Processor -- start 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f620b8b6640 1 -- start start 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f620b8b6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62040fe5a0 0x7f62041941a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f620b8b6640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62041045d0 0x7f62041946e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f620b8b6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6204194d70 con 0x7f62040fe5a0 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f620b8b6640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6204198b40 con 0x7f62041045d0 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f6209652640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62040fe5a0 0x7f62041941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f6209652640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62040fe5a0 0x7f62041941a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57776/0 (socket says 172.21.15.88:57776) 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f6209652640 1 -- 172.21.15.88:0/2251439565 learned_addr learned my addr 172.21.15.88:0/2251439565 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f6208e51640 1 --2- 172.21.15.88:0/2251439565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62041045d0 0x7f62041946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f6209652640 1 -- 172.21.15.88:0/2251439565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62041045d0 msgr2=0x7f62041946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f6209652640 1 --2- 172.21.15.88:0/2251439565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62041045d0 0x7f62041946e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f6209652640 1 -- 172.21.15.88:0/2251439565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc009900 con 0x7f62040fe5a0 2024-02-19T19:43:39.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f6209652640 1 --2- 172.21.15.88:0/2251439565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62040fe5a0 0x7f62041941a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f61fc037040 tx=0x7f61fc032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:39.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.318+0000 7f61fa7fc640 1 -- 172.21.15.88:0/2251439565 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61fc034820 con 0x7f62040fe5a0 2024-02-19T19:43:39.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.319+0000 7f620b8b6640 1 -- 172.21.15.88:0/2251439565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6204198dc0 con 0x7f62040fe5a0 2024-02-19T19:43:39.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.319+0000 7f61fa7fc640 1 -- 172.21.15.88:0/2251439565 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61fc003e30 con 0x7f62040fe5a0 2024-02-19T19:43:39.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.319+0000 7f61fa7fc640 1 -- 172.21.15.88:0/2251439565 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61fc03f440 con 0x7f62040fe5a0 2024-02-19T19:43:39.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.319+0000 7f620b8b6640 1 -- 172.21.15.88:0/2251439565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6204199340 con 0x7f62040fe5a0 2024-02-19T19:43:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.320+0000 7f620b8b6640 1 -- 172.21.15.88:0/2251439565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f620404e000 con 0x7f62040fe5a0 2024-02-19T19:43:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.320+0000 7f61fa7fc640 1 -- 172.21.15.88:0/2251439565 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f61fc03b040 con 0x7f62040fe5a0 2024-02-19T19:43:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.320+0000 7f61fa7fc640 1 --2- 172.21.15.88:0/2251439565 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61f006d810 0x7f61f006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.320+0000 7f61fa7fc640 1 -- 172.21.15.88:0/2251439565 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f61fc0b04e0 con 0x7f62040fe5a0 2024-02-19T19:43:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.320+0000 7f6208e51640 1 --2- 172.21.15.88:0/2251439565 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61f006d810 0x7f61f006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:39.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.321+0000 7f6208e51640 1 --2- 172.21.15.88:0/2251439565 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61f006d810 0x7f61f006fcc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f61f4005fd0 tx=0x7f61f4005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:39.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.325+0000 7f61fa7fc640 1 -- 172.21.15.88:0/2251439565 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f61fc079d50 con 0x7f62040fe5a0 2024-02-19T19:43:39.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.533+0000 7f620b8b6640 1 -- 172.21.15.88:0/2251439565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f62041954f0 con 0x7f62040fe5a0 2024-02-19T19:43:39.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.533+0000 7f61fa7fc640 1 -- 172.21.15.88:0/2251439565 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v40) v1 ==== 107+0+4825 (secure 0 0 0) 0x7f61fc07d410 con 0x7f62040fe5a0 2024-02-19T19:43:39.534 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:39.534 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14522,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/2124144962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":2124144962},{"type":"v1","addr":"172.21.15.88:6827","nonce":2124144962}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34426,"name":"cephfs.smithi088.psjiah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6829/3440020652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3440020652},{"type":"v1","addr":"172.21.15.88:6829","nonce":3440020652}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44339,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/1971026183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1971026183},{"type":"v1","addr":"172.21.15.132:6825","nonce":1971026183}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:28.265493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:39.534 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 24 2024-02-19T19:43:39.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.535+0000 7f620b8b6640 1 -- 172.21.15.88:0/2251439565 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61f006d810 msgr2=0x7f61f006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.535+0000 7f620b8b6640 1 --2- 172.21.15.88:0/2251439565 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61f006d810 0x7f61f006fcc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f61f4005fd0 tx=0x7f61f4005e20 comp rx=0 tx=0).stop 2024-02-19T19:43:39.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.535+0000 7f620b8b6640 1 -- 172.21.15.88:0/2251439565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62040fe5a0 msgr2=0x7f62041941a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.535+0000 7f620b8b6640 1 --2- 172.21.15.88:0/2251439565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62040fe5a0 0x7f62041941a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f61fc037040 tx=0x7f61fc032b60 comp rx=0 tx=0).stop 2024-02-19T19:43:39.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.535+0000 7f620b8b6640 1 -- 172.21.15.88:0/2251439565 shutdown_connections 2024-02-19T19:43:39.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.535+0000 7f620b8b6640 1 --2- 172.21.15.88:0/2251439565 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f61f006d810 0x7f61f006fcc0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.535+0000 7f620b8b6640 1 --2- 172.21.15.88:0/2251439565 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f62041045d0 0x7f62041946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.535+0000 7f620b8b6640 1 --2- 172.21.15.88:0/2251439565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62040fe5a0 0x7f62041941a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.535+0000 7f620b8b6640 1 -- 172.21.15.88:0/2251439565 >> 172.21.15.88:0/2251439565 conn(0x7f62040fa060 msgr2=0x7f62040fb980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:39.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.536+0000 7f620b8b6640 1 -- 172.21.15.88:0/2251439565 shutdown_connections 2024-02-19T19:43:39.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.536+0000 7f620b8b6640 1 -- 172.21.15.88:0/2251439565 wait complete. 2024-02-19T19:43:39.546 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-19T19:43:39.547 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.647+0000 7fdb5502d640 1 -- 172.21.15.88:0/7180112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb500ff840 msgr2=0x7fdb500ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.647+0000 7fdb5502d640 1 --2- 172.21.15.88:0/7180112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb500ff840 0x7fdb500ffc10 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009880 tx=0x7fdb4402bde0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.648+0000 7fdb5502d640 1 -- 172.21.15.88:0/7180112 shutdown_connections 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.648+0000 7fdb5502d640 1 --2- 172.21.15.88:0/7180112 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb50100150 0x7fdb501088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.648+0000 7fdb5502d640 1 --2- 172.21.15.88:0/7180112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb500ff840 0x7fdb500ffc10 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.648+0000 7fdb5502d640 1 -- 172.21.15.88:0/7180112 >> 172.21.15.88:0/7180112 conn(0x7fdb50073d20 msgr2=0x7fdb50074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.648+0000 7fdb5502d640 1 -- 172.21.15.88:0/7180112 shutdown_connections 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.648+0000 7fdb5502d640 1 -- 172.21.15.88:0/7180112 wait complete. 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.648+0000 7fdb5502d640 1 Processor -- start 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.648+0000 7fdb5502d640 1 -- start start 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb5502d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb500ff840 0x7fdb50195240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:39.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb5502d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb50100150 0x7fdb50195780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb5502d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb50195d80 con 0x7fdb500ff840 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb5502d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb50195ef0 con 0x7fdb50100150 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb4ed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb500ff840 0x7fdb50195240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb4ed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb500ff840 0x7fdb50195240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57792/0 (socket says 172.21.15.88:57792) 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb4ed9d640 1 -- 172.21.15.88:0/87420897 learned_addr learned my addr 172.21.15.88:0/87420897 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb4e59c640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb50100150 0x7fdb50195780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb4ed9d640 1 -- 172.21.15.88:0/87420897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb50100150 msgr2=0x7fdb50195780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb4ed9d640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb50100150 0x7fdb50195780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb4ed9d640 1 -- 172.21.15.88:0/87420897 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb44009560 con 0x7fdb500ff840 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb4e59c640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb50100150 0x7fdb50195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb4ed9d640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb500ff840 0x7fdb50195240 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009230 tx=0x7fdb44004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb37fff640 1 -- 172.21.15.88:0/87420897 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb440035f0 con 0x7fdb500ff840 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb5502d640 1 -- 172.21.15.88:0/87420897 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb50199d20 con 0x7fdb500ff840 2024-02-19T19:43:39.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb37fff640 1 -- 172.21.15.88:0/87420897 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb44003c10 con 0x7fdb500ff840 2024-02-19T19:43:39.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb37fff640 1 -- 172.21.15.88:0/87420897 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb4403e450 con 0x7fdb500ff840 2024-02-19T19:43:39.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.649+0000 7fdb5502d640 1 -- 172.21.15.88:0/87420897 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5019a300 con 0x7fdb500ff840 2024-02-19T19:43:39.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.650+0000 7fdb5502d640 1 -- 172.21.15.88:0/87420897 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb5004e000 con 0x7fdb500ff840 2024-02-19T19:43:39.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.651+0000 7fdb37fff640 1 -- 172.21.15.88:0/87420897 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fdb44004ff0 con 0x7fdb500ff840 2024-02-19T19:43:39.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.651+0000 7fdb37fff640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb3806d810 0x7fdb3806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:39.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.651+0000 7fdb37fff640 1 -- 172.21.15.88:0/87420897 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fdb440b01c0 con 0x7fdb500ff840 2024-02-19T19:43:39.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.651+0000 7fdb4e59c640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb3806d810 0x7fdb3806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:39.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.652+0000 7fdb4e59c640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb3806d810 0x7fdb3806fcc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdb50196850 tx=0x7fdb4000a5c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:39.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.656+0000 7fdb37fff640 1 -- 172.21.15.88:0/87420897 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdb44079970 con 0x7fdb500ff840 2024-02-19T19:43:39.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.866+0000 7fdb5502d640 1 -- 172.21.15.88:0/87420897 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fdb5010b970 con 0x7fdb500ff840 2024-02-19T19:43:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.867+0000 7fdb37fff640 1 -- 172.21.15.88:0/87420897 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v40) v1 ==== 107+0+4046 (secure 0 0 0) 0x7fdb4407d030 con 0x7fdb500ff840 2024-02-19T19:43:39.867 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:39.867 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34426,"name":"cephfs.smithi088.psjiah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6829/3440020652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3440020652},{"type":"v1","addr":"172.21.15.88:6829","nonce":3440020652}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":44339,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/1971026183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1971026183},{"type":"v1","addr":"172.21.15.132:6825","nonce":1971026183}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:28.265493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 25 2024-02-19T19:43:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.869+0000 7fdb5502d640 1 -- 172.21.15.88:0/87420897 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb3806d810 msgr2=0x7fdb3806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.869+0000 7fdb5502d640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb3806d810 0x7fdb3806fcc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdb50196850 tx=0x7fdb4000a5c0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.869+0000 7fdb5502d640 1 -- 172.21.15.88:0/87420897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb500ff840 msgr2=0x7fdb50195240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.869+0000 7fdb5502d640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb500ff840 0x7fdb50195240 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009230 tx=0x7fdb44004040 comp rx=0 tx=0).stop 2024-02-19T19:43:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.869+0000 7fdb5502d640 1 -- 172.21.15.88:0/87420897 shutdown_connections 2024-02-19T19:43:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.869+0000 7fdb5502d640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fdb3806d810 0x7fdb3806fcc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.869+0000 7fdb5502d640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fdb50100150 0x7fdb50195780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.869+0000 7fdb5502d640 1 --2- 172.21.15.88:0/87420897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb500ff840 0x7fdb50195240 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.869+0000 7fdb5502d640 1 -- 172.21.15.88:0/87420897 >> 172.21.15.88:0/87420897 conn(0x7fdb50073d20 msgr2=0x7fdb500fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:39.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.869+0000 7fdb5502d640 1 -- 172.21.15.88:0/87420897 shutdown_connections 2024-02-19T19:43:39.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.869+0000 7fdb5502d640 1 -- 172.21.15.88:0/87420897 wait complete. 2024-02-19T19:43:39.880 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-19T19:43:39.880 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-02-19T19:43:39.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.981+0000 7f9494c4a640 1 -- 172.21.15.88:0/3643321984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94900fd610 msgr2=0x7f94900fda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.981+0000 7f9494c4a640 1 --2- 172.21.15.88:0/3643321984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94900fd610 0x7f94900fda60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9480009880 tx=0x7f948002bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.981+0000 7f9494c4a640 1 -- 172.21.15.88:0/3643321984 shutdown_connections 2024-02-19T19:43:39.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.981+0000 7f9494c4a640 1 --2- 172.21.15.88:0/3643321984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94900fd610 0x7f94900fda60 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.981+0000 7f9494c4a640 1 --2- 172.21.15.88:0/3643321984 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9490105750 0x7f9490105b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.981+0000 7f9494c4a640 1 -- 172.21.15.88:0/3643321984 >> 172.21.15.88:0/3643321984 conn(0x7f94900f9340 msgr2=0x7f94900fb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:39.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.981+0000 7f9494c4a640 1 -- 172.21.15.88:0/3643321984 shutdown_connections 2024-02-19T19:43:39.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.981+0000 7f9494c4a640 1 -- 172.21.15.88:0/3643321984 wait complete. 2024-02-19T19:43:39.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f9494c4a640 1 Processor -- start 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f9494c4a640 1 -- start start 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f9494c4a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94900fd610 0x7f94901953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f9494c4a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9490105750 0x7f9490195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f9494c4a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490195f90 con 0x7f94900fd610 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f9494c4a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9490199d60 con 0x7f9490105750 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f948e59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94900fd610 0x7f94901953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f948e59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94900fd610 0x7f94901953c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57796/0 (socket says 172.21.15.88:57796) 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f948e59c640 1 -- 172.21.15.88:0/1198610288 learned_addr learned my addr 172.21.15.88:0/1198610288 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f948e59c640 1 -- 172.21.15.88:0/1198610288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9490105750 msgr2=0x7f9490195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f948e59c640 1 --2- 172.21.15.88:0/1198610288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9490105750 0x7f9490195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f948e59c640 1 -- 172.21.15.88:0/1198610288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9480009560 con 0x7f94900fd610 2024-02-19T19:43:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f948e59c640 1 --2- 172.21.15.88:0/1198610288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94900fd610 0x7f94901953c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f948400b420 tx=0x7f948400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f947f7fe640 1 -- 172.21.15.88:0/1198610288 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9484011800 con 0x7f94900fd610 2024-02-19T19:43:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f947f7fe640 1 -- 172.21.15.88:0/1198610288 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9484011e20 con 0x7f94900fd610 2024-02-19T19:43:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.982+0000 7f9494c4a640 1 -- 172.21.15.88:0/1198610288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949019a040 con 0x7f94900fd610 2024-02-19T19:43:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.983+0000 7f947f7fe640 1 -- 172.21.15.88:0/1198610288 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f948400f2e0 con 0x7f94900fd610 2024-02-19T19:43:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.983+0000 7f9494c4a640 1 -- 172.21.15.88:0/1198610288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949019a5a0 con 0x7f94900fd610 2024-02-19T19:43:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.983+0000 7f9494c4a640 1 -- 172.21.15.88:0/1198610288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949004e000 con 0x7f94900fd610 2024-02-19T19:43:39.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.984+0000 7f947f7fe640 1 -- 172.21.15.88:0/1198610288 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f9484010380 con 0x7f94900fd610 2024-02-19T19:43:39.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.984+0000 7f947f7fe640 1 --2- 172.21.15.88:0/1198610288 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f947806d810 0x7f947806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:39.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.984+0000 7f947f7fe640 1 -- 172.21.15.88:0/1198610288 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f948408cec0 con 0x7f94900fd610 2024-02-19T19:43:39.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.984+0000 7f948dd9b640 1 --2- 172.21.15.88:0/1198610288 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f947806d810 0x7f947806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:39.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.985+0000 7f948dd9b640 1 --2- 172.21.15.88:0/1198610288 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f947806d810 0x7f947806fcc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9480036040 tx=0x7f9480000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:39.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:39.989+0000 7f947f7fe640 1 -- 172.21.15.88:0/1198610288 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f9484056670 con 0x7f94900fd610 2024-02-19T19:43:40.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:39 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/794787860' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-19T19:43:40.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:39 smithi088 ceph-mon[88643]: pgmap v1205: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:40.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:39 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3601432234' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-19T19:43:40.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:39 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2251439565' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-19T19:43:40.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.198+0000 7f9494c4a640 1 -- 172.21.15.88:0/1198610288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f9490107a90 con 0x7f94900fd610 2024-02-19T19:43:40.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.198+0000 7f947f7fe640 1 -- 172.21.15.88:0/1198610288 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v40) v1 ==== 107+0+4856 (secure 0 0 0) 0x7f9484059d30 con 0x7f94900fd610 2024-02-19T19:43:40.199 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:40.199 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34426,"name":"cephfs.smithi088.psjiah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6829/3440020652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":3440020652},{"type":"v1","addr":"172.21.15.88:6829","nonce":3440020652}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24},{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44339,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/1971026183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1971026183},{"type":"v1","addr":"172.21.15.132:6825","nonce":1971026183}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:28.265493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:40.199 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 26 2024-02-19T19:43:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.200+0000 7f9494c4a640 1 -- 172.21.15.88:0/1198610288 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f947806d810 msgr2=0x7f947806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.200+0000 7f9494c4a640 1 --2- 172.21.15.88:0/1198610288 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f947806d810 0x7f947806fcc0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f9480036040 tx=0x7f9480000bc0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.200+0000 7f9494c4a640 1 -- 172.21.15.88:0/1198610288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94900fd610 msgr2=0x7f94901953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.200+0000 7f9494c4a640 1 --2- 172.21.15.88:0/1198610288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94900fd610 0x7f94901953c0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f948400b420 tx=0x7f948400b8e0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.200+0000 7f9494c4a640 1 -- 172.21.15.88:0/1198610288 shutdown_connections 2024-02-19T19:43:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.200+0000 7f9494c4a640 1 --2- 172.21.15.88:0/1198610288 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f947806d810 0x7f947806fcc0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.200+0000 7f9494c4a640 1 --2- 172.21.15.88:0/1198610288 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9490105750 0x7f9490195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.200+0000 7f9494c4a640 1 --2- 172.21.15.88:0/1198610288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94900fd610 0x7f94901953c0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.201+0000 7f9494c4a640 1 -- 172.21.15.88:0/1198610288 >> 172.21.15.88:0/1198610288 conn(0x7f94900f9340 msgr2=0x7f94900fafe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:40.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.201+0000 7f9494c4a640 1 -- 172.21.15.88:0/1198610288 shutdown_connections 2024-02-19T19:43:40.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.201+0000 7f9494c4a640 1 -- 172.21.15.88:0/1198610288 wait complete. 2024-02-19T19:43:40.212 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-19T19:43:40.212 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-02-19T19:43:40.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:39 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/794787860' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-02-19T19:43:40.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:39 smithi132 ceph-mon[70676]: pgmap v1205: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:40.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:39 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3601432234' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-02-19T19:43:40.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:39 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2251439565' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-02-19T19:43:40.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.316+0000 7f0828389640 1 -- 172.21.15.88:0/3518491985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08200fe2e0 msgr2=0x7f08201042b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.316+0000 7f0828389640 1 --2- 172.21.15.88:0/3518491985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08200fe2e0 0x7f08201042b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0810009880 tx=0x7f081002bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.316+0000 7f0828389640 1 -- 172.21.15.88:0/3518491985 shutdown_connections 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.316+0000 7f0828389640 1 --2- 172.21.15.88:0/3518491985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08200fe2e0 0x7f08201042b0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.316+0000 7f0828389640 1 --2- 172.21.15.88:0/3518491985 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08200fd940 0x7f08200fdd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.316+0000 7f0828389640 1 -- 172.21.15.88:0/3518491985 >> 172.21.15.88:0/3518491985 conn(0x7f08200f9340 msgr2=0x7f08200fb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.316+0000 7f0828389640 1 -- 172.21.15.88:0/3518491985 shutdown_connections 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.316+0000 7f0828389640 1 -- 172.21.15.88:0/3518491985 wait complete. 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0828389640 1 Processor -- start 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0828389640 1 -- start start 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0828389640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08200fd940 0x7f0820195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0828389640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08200fe2e0 0x7f0820195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0828389640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0820195f10 con 0x7f08200fd940 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0828389640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0820199ce0 con 0x7f08200fe2e0 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0826125640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08200fd940 0x7f0820195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0826125640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08200fd940 0x7f0820195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57826/0 (socket says 172.21.15.88:57826) 2024-02-19T19:43:40.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0826125640 1 -- 172.21.15.88:0/1406405241 learned_addr learned my addr 172.21.15.88:0/1406405241 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:40.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0825924640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08200fe2e0 0x7f0820195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:40.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0826125640 1 -- 172.21.15.88:0/1406405241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08200fe2e0 msgr2=0x7f0820195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0826125640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08200fe2e0 0x7f0820195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0826125640 1 -- 172.21.15.88:0/1406405241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0810009560 con 0x7f08200fd940 2024-02-19T19:43:40.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.317+0000 7f0825924640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08200fe2e0 0x7f0820195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:40.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.318+0000 7f0826125640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08200fd940 0x7f0820195340 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f081c00d6f0 tx=0x7f081c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:40.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.318+0000 7f08177fe640 1 -- 172.21.15.88:0/1406405241 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f081c009a30 con 0x7f08200fd940 2024-02-19T19:43:40.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.318+0000 7f0828389640 1 -- 172.21.15.88:0/1406405241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0820199fc0 con 0x7f08200fd940 2024-02-19T19:43:40.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.318+0000 7f08177fe640 1 -- 172.21.15.88:0/1406405241 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f081c0054e0 con 0x7f08200fd940 2024-02-19T19:43:40.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.318+0000 7f08177fe640 1 -- 172.21.15.88:0/1406405241 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f081c010460 con 0x7f08200fd940 2024-02-19T19:43:40.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.318+0000 7f0828389640 1 -- 172.21.15.88:0/1406405241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082019a4f0 con 0x7f08200fd940 2024-02-19T19:43:40.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.319+0000 7f0828389640 1 -- 172.21.15.88:0/1406405241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082004e000 con 0x7f08200fd940 2024-02-19T19:43:40.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.319+0000 7f08177fe640 1 -- 172.21.15.88:0/1406405241 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f081c009b90 con 0x7f08200fd940 2024-02-19T19:43:40.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.320+0000 7f08177fe640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f080c06d810 0x7f080c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:40.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.320+0000 7f08177fe640 1 -- 172.21.15.88:0/1406405241 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f081c08d0f0 con 0x7f08200fd940 2024-02-19T19:43:40.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.320+0000 7f0825924640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f080c06d810 0x7f080c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:40.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.320+0000 7f0825924640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f080c06d810 0x7f080c06fcc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0810009230 tx=0x7f0810005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:40.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.325+0000 7f08177fe640 1 -- 172.21.15.88:0/1406405241 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f081c0568a0 con 0x7f08200fd940 2024-02-19T19:43:40.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.535+0000 7f0828389640 1 -- 172.21.15.88:0/1406405241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f0820196690 con 0x7f08200fd940 2024-02-19T19:43:40.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.536+0000 7f08177fe640 1 -- 172.21.15.88:0/1406405241 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v40) v1 ==== 107+0+4077 (secure 0 0 0) 0x7f081c056020 con 0x7f08200fd940 2024-02-19T19:43:40.536 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:40.536 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":44339,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/1971026183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1971026183},{"type":"v1","addr":"172.21.15.132:6825","nonce":1971026183}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:28.265493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:40.536 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-02-19T19:43:40.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.538+0000 7f0828389640 1 -- 172.21.15.88:0/1406405241 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f080c06d810 msgr2=0x7f080c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.538+0000 7f0828389640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f080c06d810 0x7f080c06fcc0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f0810009230 tx=0x7f0810005fd0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.538+0000 7f0828389640 1 -- 172.21.15.88:0/1406405241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08200fd940 msgr2=0x7f0820195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.538+0000 7f0828389640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08200fd940 0x7f0820195340 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f081c00d6f0 tx=0x7f081c00dbb0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.538+0000 7f0828389640 1 -- 172.21.15.88:0/1406405241 shutdown_connections 2024-02-19T19:43:40.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.538+0000 7f0828389640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f080c06d810 0x7f080c06fcc0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.538+0000 7f0828389640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f08200fe2e0 0x7f0820195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.538+0000 7f0828389640 1 --2- 172.21.15.88:0/1406405241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08200fd940 0x7f0820195340 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.538+0000 7f0828389640 1 -- 172.21.15.88:0/1406405241 >> 172.21.15.88:0/1406405241 conn(0x7f08200f9340 msgr2=0x7f08200fb750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:40.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.538+0000 7f0828389640 1 -- 172.21.15.88:0/1406405241 shutdown_connections 2024-02-19T19:43:40.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.538+0000 7f0828389640 1 -- 172.21.15.88:0/1406405241 wait complete. 2024-02-19T19:43:40.549 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-19T19:43:40.549 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-02-19T19:43:40.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.651+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3891875820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a00ff870 msgr2=0x7f05a00ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.651+0000 7f05a5fd0640 1 --2- 172.21.15.88:0/3891875820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a00ff870 0x7f05a00ffce0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f0594009880 tx=0x7f059402bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.651+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3891875820 shutdown_connections 2024-02-19T19:43:40.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.651+0000 7f05a5fd0640 1 --2- 172.21.15.88:0/3891875820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a00ff870 0x7f05a00ffce0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.651+0000 7f05a5fd0640 1 --2- 172.21.15.88:0/3891875820 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05a0105880 0x7f05a0105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.651+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3891875820 >> 172.21.15.88:0/3891875820 conn(0x7f05a00fb370 msgr2=0x7f05a00fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.651+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3891875820 shutdown_connections 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.651+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3891875820 wait complete. 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.651+0000 7f05a5fd0640 1 Processor -- start 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.651+0000 7f05a5fd0640 1 -- start start 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f05a5fd0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a00ff870 0x7f05a0195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f05a5fd0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05a0105880 0x7f05a0195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f05a5fd0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0196150 con 0x7f05a00ff870 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f05a5fd0640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0199f20 con 0x7f05a0105880 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a00ff870 0x7f05a0195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059f7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a00ff870 0x7f05a0195580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57848/0 (socket says 172.21.15.88:57848) 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059f7fe640 1 -- 172.21.15.88:0/3537642540 learned_addr learned my addr 172.21.15.88:0/3537642540 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059effd640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05a0105880 0x7f05a0195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059f7fe640 1 -- 172.21.15.88:0/3537642540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05a0105880 msgr2=0x7f05a0195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059f7fe640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05a0105880 0x7f05a0195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059f7fe640 1 -- 172.21.15.88:0/3537642540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0594009560 con 0x7f05a00ff870 2024-02-19T19:43:40.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059effd640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05a0105880 0x7f05a0195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:40.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059f7fe640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a00ff870 0x7f05a0195580 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f059000c8f0 tx=0x7f059000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:40.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059cff9640 1 -- 172.21.15.88:0/3537642540 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05900041f0 con 0x7f05a00ff870 2024-02-19T19:43:40.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059cff9640 1 -- 172.21.15.88:0/3537642540 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0590007e70 con 0x7f05a00ff870 2024-02-19T19:43:40.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3537642540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a019a200 con 0x7f05a00ff870 2024-02-19T19:43:40.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f059cff9640 1 -- 172.21.15.88:0/3537642540 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05900049d0 con 0x7f05a00ff870 2024-02-19T19:43:40.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.652+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3537642540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a019a760 con 0x7f05a00ff870 2024-02-19T19:43:40.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.653+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3537642540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a004e000 con 0x7f05a00ff870 2024-02-19T19:43:40.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.654+0000 7f059cff9640 1 -- 172.21.15.88:0/3537642540 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f0590004350 con 0x7f05a00ff870 2024-02-19T19:43:40.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.654+0000 7f059cff9640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f058806d810 0x7f058806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:40.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.654+0000 7f059cff9640 1 -- 172.21.15.88:0/3537642540 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f059008d380 con 0x7f05a00ff870 2024-02-19T19:43:40.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.654+0000 7f059effd640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f058806d810 0x7f058806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:40.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.655+0000 7f059effd640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f058806d810 0x7f058806fcc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f05a0196b90 tx=0x7f059402cc00 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:40.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.659+0000 7f059cff9640 1 -- 172.21.15.88:0/3537642540 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f0590056b30 con 0x7f05a00ff870 2024-02-19T19:43:40.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.870+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3537642540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f05a0066860 con 0x7f05a00ff870 2024-02-19T19:43:40.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.870+0000 7f059cff9640 1 -- 172.21.15.88:0/3537642540 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v40) v1 ==== 107+0+4887 (secure 0 0 0) 0x7f059005a1f0 con 0x7f05a00ff870 2024-02-19T19:43:40.871 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:40.871 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44339,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/1971026183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":1971026183},{"type":"v1","addr":"172.21.15.132:6825","nonce":1971026183}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:28.265493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:40.871 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 28 2024-02-19T19:43:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.872+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3537642540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f058806d810 msgr2=0x7f058806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.872+0000 7f05a5fd0640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f058806d810 0x7f058806fcc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f05a0196b90 tx=0x7f059402cc00 comp rx=0 tx=0).stop 2024-02-19T19:43:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.872+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3537642540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a00ff870 msgr2=0x7f05a0195580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.872+0000 7f05a5fd0640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a00ff870 0x7f05a0195580 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f059000c8f0 tx=0x7f059000cdb0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.872+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3537642540 shutdown_connections 2024-02-19T19:43:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.872+0000 7f05a5fd0640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f058806d810 0x7f058806fcc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.872+0000 7f05a5fd0640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f05a0105880 0x7f05a0195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.872+0000 7f05a5fd0640 1 --2- 172.21.15.88:0/3537642540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a00ff870 0x7f05a0195580 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.872+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3537642540 >> 172.21.15.88:0/3537642540 conn(0x7f05a00fb370 msgr2=0x7f05a00fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.872+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3537642540 shutdown_connections 2024-02-19T19:43:40.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.872+0000 7f05a5fd0640 1 -- 172.21.15.88:0/3537642540 wait complete. 2024-02-19T19:43:40.883 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-19T19:43:40.884 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-02-19T19:43:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.985+0000 7f9237c5c640 1 -- 172.21.15.88:0/2110714767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92300ff870 msgr2=0x7f92300ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.985+0000 7f9237c5c640 1 --2- 172.21.15.88:0/2110714767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92300ff870 0x7f92300ffce0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f9220009880 tx=0x7f922002bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.985+0000 7f9237c5c640 1 -- 172.21.15.88:0/2110714767 shutdown_connections 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.985+0000 7f9237c5c640 1 --2- 172.21.15.88:0/2110714767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92300ff870 0x7f92300ffce0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.985+0000 7f9237c5c640 1 --2- 172.21.15.88:0/2110714767 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9230105880 0x7f9230105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.985+0000 7f9237c5c640 1 -- 172.21.15.88:0/2110714767 >> 172.21.15.88:0/2110714767 conn(0x7f92300fb370 msgr2=0x7f92300fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.985+0000 7f9237c5c640 1 -- 172.21.15.88:0/2110714767 shutdown_connections 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.985+0000 7f9237c5c640 1 -- 172.21.15.88:0/2110714767 wait complete. 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f9237c5c640 1 Processor -- start 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f9237c5c640 1 -- start start 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f9237c5c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92300ff870 0x7f9230072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f9237c5c640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9230105880 0x7f9230072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f9237c5c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9230072fe0 con 0x7f92300ff870 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f9237c5c640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9230073150 con 0x7f9230105880 2024-02-19T19:43:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f92359f8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92300ff870 0x7f9230072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f92359f8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92300ff870 0x7f9230072560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57872/0 (socket says 172.21.15.88:57872) 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f92359f8640 1 -- 172.21.15.88:0/2526296017 learned_addr learned my addr 172.21.15.88:0/2526296017 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f92351f7640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9230105880 0x7f9230072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f92359f8640 1 -- 172.21.15.88:0/2526296017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9230105880 msgr2=0x7f9230072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f92359f8640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9230105880 0x7f9230072aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f92359f8640 1 -- 172.21.15.88:0/2526296017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9220009560 con 0x7f92300ff870 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.986+0000 7f92351f7640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9230105880 0x7f9230072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.987+0000 7f92359f8640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92300ff870 0x7f9230072560 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f922c00d8d0 tx=0x7f922c00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.987+0000 7f9226ffd640 1 -- 172.21.15.88:0/2526296017 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f922c009a30 con 0x7f92300ff870 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.987+0000 7f9237c5c640 1 -- 172.21.15.88:0/2526296017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92300733d0 con 0x7f92300ff870 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.987+0000 7f9226ffd640 1 -- 172.21.15.88:0/2526296017 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f922c009b90 con 0x7f92300ff870 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.987+0000 7f9226ffd640 1 -- 172.21.15.88:0/2526296017 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f922c00f680 con 0x7f92300ff870 2024-02-19T19:43:40.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.987+0000 7f9237c5c640 1 -- 172.21.15.88:0/2526296017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92301a1b40 con 0x7f92300ff870 2024-02-19T19:43:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.988+0000 7f9237c5c640 1 -- 172.21.15.88:0/2526296017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923004e000 con 0x7f92300ff870 2024-02-19T19:43:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.988+0000 7f9226ffd640 1 -- 172.21.15.88:0/2526296017 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f922c009d00 con 0x7f92300ff870 2024-02-19T19:43:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.989+0000 7f9226ffd640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921c06d810 0x7f921c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:40.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.989+0000 7f9226ffd640 1 -- 172.21.15.88:0/2526296017 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f922c08d1e0 con 0x7f92300ff870 2024-02-19T19:43:40.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.989+0000 7f92351f7640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921c06d810 0x7f921c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:40.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.989+0000 7f92351f7640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921c06d810 0x7f921c06fcc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9230073d50 tx=0x7f922002cb40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:40.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:40.993+0000 7f9226ffd640 1 -- 172.21.15.88:0/2526296017 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f922c056990 con 0x7f92300ff870 2024-02-19T19:43:41.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.201+0000 7f9237c5c640 1 -- 172.21.15.88:0/2526296017 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f9230066860 con 0x7f92300ff870 2024-02-19T19:43:41.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.201+0000 7f9226ffd640 1 -- 172.21.15.88:0/2526296017 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v40) v1 ==== 107+0+4105 (secure 0 0 0) 0x7f922c05a050 con 0x7f92300ff870 2024-02-19T19:43:41.202 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:41.202 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:28.265493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:41.202 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 29 2024-02-19T19:43:41.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.203+0000 7f9237c5c640 1 -- 172.21.15.88:0/2526296017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921c06d810 msgr2=0x7f921c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.203+0000 7f9237c5c640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921c06d810 0x7f921c06fcc0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f9230073d50 tx=0x7f922002cb40 comp rx=0 tx=0).stop 2024-02-19T19:43:41.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.203+0000 7f9237c5c640 1 -- 172.21.15.88:0/2526296017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92300ff870 msgr2=0x7f9230072560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.203+0000 7f9237c5c640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92300ff870 0x7f9230072560 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f922c00d8d0 tx=0x7f922c00dd90 comp rx=0 tx=0).stop 2024-02-19T19:43:41.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.203+0000 7f9237c5c640 1 -- 172.21.15.88:0/2526296017 shutdown_connections 2024-02-19T19:43:41.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.203+0000 7f9237c5c640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f921c06d810 0x7f921c06fcc0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.203+0000 7f9237c5c640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f9230105880 0x7f9230072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.203+0000 7f9237c5c640 1 --2- 172.21.15.88:0/2526296017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92300ff870 0x7f9230072560 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.203+0000 7f9237c5c640 1 -- 172.21.15.88:0/2526296017 >> 172.21.15.88:0/2526296017 conn(0x7f92300fb370 msgr2=0x7f92300fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:41.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.203+0000 7f9237c5c640 1 -- 172.21.15.88:0/2526296017 shutdown_connections 2024-02-19T19:43:41.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.203+0000 7f9237c5c640 1 -- 172.21.15.88:0/2526296017 wait complete. 2024-02-19T19:43:41.213 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:40 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/87420897' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-19T19:43:41.213 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:40 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1198610288' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-19T19:43:41.213 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:40 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1406405241' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-19T19:43:41.214 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-19T19:43:41.214 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-02-19T19:43:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:40 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/87420897' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-02-19T19:43:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:40 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1198610288' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-02-19T19:43:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:40 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1406405241' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-02-19T19:43:41.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.316+0000 7fb943f18640 1 -- 172.21.15.88:0/1442469402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb93c0fe7f0 msgr2=0x7fb93c0fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.316+0000 7fb943f18640 1 --2- 172.21.15.88:0/1442469402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb93c0fe7f0 0x7fb93c0fec60 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb930009880 tx=0x7fb93002bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.316+0000 7fb943f18640 1 -- 172.21.15.88:0/1442469402 shutdown_connections 2024-02-19T19:43:41.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.316+0000 7fb943f18640 1 --2- 172.21.15.88:0/1442469402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb93c0fe7f0 0x7fb93c0fec60 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.316+0000 7fb943f18640 1 --2- 172.21.15.88:0/1442469402 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb93c104800 0x7fb93c104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.316+0000 7fb943f18640 1 -- 172.21.15.88:0/1442469402 >> 172.21.15.88:0/1442469402 conn(0x7fb93c0fa2f0 msgr2=0x7fb93c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:41.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.316+0000 7fb943f18640 1 -- 172.21.15.88:0/1442469402 shutdown_connections 2024-02-19T19:43:41.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.316+0000 7fb943f18640 1 -- 172.21.15.88:0/1442469402 wait complete. 2024-02-19T19:43:41.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.316+0000 7fb943f18640 1 Processor -- start 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.316+0000 7fb943f18640 1 -- start start 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb943f18640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb93c0fe7f0 0x7fb93c194420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb943f18640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb93c104800 0x7fb93c194960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb943f18640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c194f60 con 0x7fb93c104800 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb943f18640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb93c1950d0 con 0x7fb93c0fe7f0 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb9414b3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb93c104800 0x7fb93c194960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb9414b3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb93c104800 0x7fb93c194960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57896/0 (socket says 172.21.15.88:57896) 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb9414b3640 1 -- 172.21.15.88:0/2487903493 learned_addr learned my addr 172.21.15.88:0/2487903493 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb941cb4640 1 --2- 172.21.15.88:0/2487903493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb93c0fe7f0 0x7fb93c194420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb9414b3640 1 -- 172.21.15.88:0/2487903493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb93c0fe7f0 msgr2=0x7fb93c194420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb9414b3640 1 --2- 172.21.15.88:0/2487903493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb93c0fe7f0 0x7fb93c194420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb9414b3640 1 -- 172.21.15.88:0/2487903493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930009560 con 0x7fb93c104800 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb9414b3640 1 --2- 172.21.15.88:0/2487903493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb93c104800 0x7fb93c194960 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fb930009230 tx=0x7fb930004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb93affd640 1 -- 172.21.15.88:0/2487903493 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9300035f0 con 0x7fb93c104800 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb943f18640 1 -- 172.21.15.88:0/2487903493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb93c198f00 con 0x7fb93c104800 2024-02-19T19:43:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb93affd640 1 -- 172.21.15.88:0/2487903493 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb930003c10 con 0x7fb93c104800 2024-02-19T19:43:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb93affd640 1 -- 172.21.15.88:0/2487903493 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb930034440 con 0x7fb93c104800 2024-02-19T19:43:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.317+0000 7fb943f18640 1 -- 172.21.15.88:0/2487903493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb93c199480 con 0x7fb93c104800 2024-02-19T19:43:41.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.318+0000 7fb943f18640 1 -- 172.21.15.88:0/2487903493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb93c04e000 con 0x7fb93c104800 2024-02-19T19:43:41.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.319+0000 7fb93affd640 1 -- 172.21.15.88:0/2487903493 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb930004fd0 con 0x7fb93c104800 2024-02-19T19:43:41.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.319+0000 7fb93affd640 1 --2- 172.21.15.88:0/2487903493 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb92406d810 0x7fb92406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:41.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.319+0000 7fb93affd640 1 -- 172.21.15.88:0/2487903493 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb9300af9c0 con 0x7fb93c104800 2024-02-19T19:43:41.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.319+0000 7fb941cb4640 1 --2- 172.21.15.88:0/2487903493 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb92406d810 0x7fb92406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:41.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.320+0000 7fb941cb4640 1 --2- 172.21.15.88:0/2487903493 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb92406d810 0x7fb92406fcc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb934005c40 tx=0x7fb934005bd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:41.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.324+0000 7fb93affd640 1 -- 172.21.15.88:0/2487903493 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb930079170 con 0x7fb93c104800 2024-02-19T19:43:41.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.533+0000 7fb943f18640 1 -- 172.21.15.88:0/2487903493 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7fb93c10ba60 con 0x7fb93c104800 2024-02-19T19:43:41.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.533+0000 7fb93affd640 1 -- 172.21.15.88:0/2487903493 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v40) v1 ==== 107+0+4918 (secure 0 0 0) 0x7fb93007c830 con 0x7fb93c104800 2024-02-19T19:43:41.534 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:41.534 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24275,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.132:6827/3724797063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":3724797063},{"type":"v1","addr":"172.21.15.132:6827","nonce":3724797063}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":16},{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44365,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/594928725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":594928725},{"type":"v1","addr":"172.21.15.132:6825","nonce":594928725}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:28.265493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:41.534 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 30 2024-02-19T19:43:41.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.536+0000 7fb943f18640 1 -- 172.21.15.88:0/2487903493 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb92406d810 msgr2=0x7fb92406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.536+0000 7fb943f18640 1 --2- 172.21.15.88:0/2487903493 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb92406d810 0x7fb92406fcc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb934005c40 tx=0x7fb934005bd0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.536+0000 7fb943f18640 1 -- 172.21.15.88:0/2487903493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb93c104800 msgr2=0x7fb93c194960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.536+0000 7fb943f18640 1 --2- 172.21.15.88:0/2487903493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb93c104800 0x7fb93c194960 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fb930009230 tx=0x7fb930004040 comp rx=0 tx=0).stop 2024-02-19T19:43:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.536+0000 7fb943f18640 1 -- 172.21.15.88:0/2487903493 shutdown_connections 2024-02-19T19:43:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.536+0000 7fb943f18640 1 --2- 172.21.15.88:0/2487903493 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb92406d810 0x7fb92406fcc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.536+0000 7fb943f18640 1 --2- 172.21.15.88:0/2487903493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb93c104800 0x7fb93c194960 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.536+0000 7fb943f18640 1 --2- 172.21.15.88:0/2487903493 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb93c0fe7f0 0x7fb93c194420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.536+0000 7fb943f18640 1 -- 172.21.15.88:0/2487903493 >> 172.21.15.88:0/2487903493 conn(0x7fb93c0fa2f0 msgr2=0x7fb93c0fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.536+0000 7fb943f18640 1 -- 172.21.15.88:0/2487903493 shutdown_connections 2024-02-19T19:43:41.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.536+0000 7fb943f18640 1 -- 172.21.15.88:0/2487903493 wait complete. 2024-02-19T19:43:41.549 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-19T19:43:41.550 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-02-19T19:43:41.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.653+0000 7f2d50389640 1 -- 172.21.15.88:0/4017867503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d480fe6f0 msgr2=0x7f2d480feac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.653+0000 7f2d50389640 1 --2- 172.21.15.88:0/4017867503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d480fe6f0 0x7f2d480feac0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f2d380098b0 tx=0x7f2d3802bcd0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.653+0000 7f2d50389640 1 -- 172.21.15.88:0/4017867503 shutdown_connections 2024-02-19T19:43:41.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.653+0000 7f2d50389640 1 --2- 172.21.15.88:0/4017867503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d480ff000 0x7f2d48109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.653+0000 7f2d50389640 1 --2- 172.21.15.88:0/4017867503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d480fe6f0 0x7f2d480feac0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.653+0000 7f2d50389640 1 -- 172.21.15.88:0/4017867503 >> 172.21.15.88:0/4017867503 conn(0x7f2d48073d20 msgr2=0x7f2d48074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.653+0000 7f2d50389640 1 -- 172.21.15.88:0/4017867503 shutdown_connections 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.653+0000 7f2d50389640 1 -- 172.21.15.88:0/4017867503 wait complete. 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.653+0000 7f2d50389640 1 Processor -- start 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d50389640 1 -- start start 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d50389640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d480fe6f0 0x7f2d481952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d50389640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d480ff000 0x7f2d48195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d50389640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d48195e90 con 0x7f2d480ff000 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d50389640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d48199c60 con 0x7f2d480fe6f0 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d4d924640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d480ff000 0x7f2d48195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d4d924640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d480ff000 0x7f2d48195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57900/0 (socket says 172.21.15.88:57900) 2024-02-19T19:43:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d4d924640 1 -- 172.21.15.88:0/3977207097 learned_addr learned my addr 172.21.15.88:0/3977207097 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d4e125640 1 --2- 172.21.15.88:0/3977207097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d480fe6f0 0x7f2d481952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d4d924640 1 -- 172.21.15.88:0/3977207097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d480fe6f0 msgr2=0x7f2d481952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d4d924640 1 --2- 172.21.15.88:0/3977207097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d480fe6f0 0x7f2d481952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d4d924640 1 -- 172.21.15.88:0/3977207097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d38009560 con 0x7f2d480ff000 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d4d924640 1 --2- 172.21.15.88:0/3977207097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d480ff000 0x7f2d48195800 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400d8d0 tx=0x7f2d4400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d3f7fe640 1 -- 172.21.15.88:0/3977207097 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d4400bea0 con 0x7f2d480ff000 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d3f7fe640 1 -- 172.21.15.88:0/3977207097 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d44009c00 con 0x7f2d480ff000 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d50389640 1 -- 172.21.15.88:0/3977207097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d48199f40 con 0x7f2d480ff000 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.654+0000 7f2d3f7fe640 1 -- 172.21.15.88:0/3977207097 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d4400f710 con 0x7f2d480ff000 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.655+0000 7f2d50389640 1 -- 172.21.15.88:0/3977207097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4819a4a0 con 0x7f2d480ff000 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.655+0000 7f2d50389640 1 -- 172.21.15.88:0/3977207097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4804e000 con 0x7f2d480ff000 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.656+0000 7f2d3f7fe640 1 -- 172.21.15.88:0/3977207097 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2d4400f990 con 0x7f2d480ff000 2024-02-19T19:43:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.656+0000 7f2d3f7fe640 1 --2- 172.21.15.88:0/3977207097 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d3406d810 0x7f2d3406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:41.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.656+0000 7f2d3f7fe640 1 -- 172.21.15.88:0/3977207097 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2d4408d1e0 con 0x7f2d480ff000 2024-02-19T19:43:41.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.656+0000 7f2d4e125640 1 --2- 172.21.15.88:0/3977207097 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d3406d810 0x7f2d3406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:41.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.657+0000 7f2d4e125640 1 --2- 172.21.15.88:0/3977207097 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d3406d810 0x7f2d3406fcc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2d38005940 tx=0x7f2d380058d0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:41.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.661+0000 7f2d3f7fe640 1 -- 172.21.15.88:0/3977207097 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2d44056a40 con 0x7f2d480ff000 2024-02-19T19:43:41.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.874+0000 7f2d50389640 1 -- 172.21.15.88:0/3977207097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f2d481070e0 con 0x7f2d480ff000 2024-02-19T19:43:41.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.874+0000 7f2d3f7fe640 1 -- 172.21.15.88:0/3977207097 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v40) v1 ==== 107+0+4136 (secure 0 0 0) 0x7f2d4405a100 con 0x7f2d480ff000 2024-02-19T19:43:41.875 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:41.875 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44365,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/594928725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":594928725},{"type":"v1","addr":"172.21.15.132:6825","nonce":594928725}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:28.265493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:41.875 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 31 2024-02-19T19:43:41.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.877+0000 7f2d50389640 1 -- 172.21.15.88:0/3977207097 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d3406d810 msgr2=0x7f2d3406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.877+0000 7f2d50389640 1 --2- 172.21.15.88:0/3977207097 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d3406d810 0x7f2d3406fcc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2d38005940 tx=0x7f2d380058d0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.877+0000 7f2d50389640 1 -- 172.21.15.88:0/3977207097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d480ff000 msgr2=0x7f2d48195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.877+0000 7f2d50389640 1 --2- 172.21.15.88:0/3977207097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d480ff000 0x7f2d48195800 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f2d4400d8d0 tx=0x7f2d4400dd90 comp rx=0 tx=0).stop 2024-02-19T19:43:41.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.877+0000 7f2d50389640 1 -- 172.21.15.88:0/3977207097 shutdown_connections 2024-02-19T19:43:41.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.877+0000 7f2d50389640 1 --2- 172.21.15.88:0/3977207097 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f2d3406d810 0x7f2d3406fcc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.877+0000 7f2d50389640 1 --2- 172.21.15.88:0/3977207097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d480ff000 0x7f2d48195800 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.877+0000 7f2d50389640 1 --2- 172.21.15.88:0/3977207097 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f2d480fe6f0 0x7f2d481952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.877+0000 7f2d50389640 1 -- 172.21.15.88:0/3977207097 >> 172.21.15.88:0/3977207097 conn(0x7f2d48073d20 msgr2=0x7f2d481042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:41.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.877+0000 7f2d50389640 1 -- 172.21.15.88:0/3977207097 shutdown_connections 2024-02-19T19:43:41.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.877+0000 7f2d50389640 1 -- 172.21.15.88:0/3977207097 wait complete. 2024-02-19T19:43:41.888 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-19T19:43:41.889 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-02-19T19:43:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.991+0000 7fb5d2769640 1 -- 172.21.15.88:0/503036973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc0fe6f0 msgr2=0x7fb5cc0feac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.991+0000 7fb5d2769640 1 --2- 172.21.15.88:0/503036973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc0fe6f0 0x7fb5cc0feac0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc0098b0 tx=0x7fb5bc02bcd0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.991+0000 7fb5d2769640 1 -- 172.21.15.88:0/503036973 shutdown_connections 2024-02-19T19:43:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.991+0000 7fb5d2769640 1 --2- 172.21.15.88:0/503036973 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5cc0ff000 0x7fb5cc109a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.991+0000 7fb5d2769640 1 --2- 172.21.15.88:0/503036973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc0fe6f0 0x7fb5cc0feac0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.991+0000 7fb5d2769640 1 -- 172.21.15.88:0/503036973 >> 172.21.15.88:0/503036973 conn(0x7fb5cc073d20 msgr2=0x7fb5cc074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.991+0000 7fb5d2769640 1 -- 172.21.15.88:0/503036973 shutdown_connections 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.991+0000 7fb5d2769640 1 -- 172.21.15.88:0/503036973 wait complete. 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.991+0000 7fb5d2769640 1 Processor -- start 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.991+0000 7fb5d2769640 1 -- start start 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5d2769640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5cc0fe6f0 0x7fb5cc1952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5d2769640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc0ff000 0x7fb5cc195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5d2769640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc195e90 con 0x7fb5cc0ff000 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5d2769640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5cc199c60 con 0x7fb5cc0fe6f0 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5cb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc0ff000 0x7fb5cc195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5cb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc0ff000 0x7fb5cc195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57910/0 (socket says 172.21.15.88:57910) 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5cb7fe640 1 -- 172.21.15.88:0/1908302764 learned_addr learned my addr 172.21.15.88:0/1908302764 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5cbfff640 1 --2- 172.21.15.88:0/1908302764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5cc0fe6f0 0x7fb5cc1952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5cb7fe640 1 -- 172.21.15.88:0/1908302764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5cc0fe6f0 msgr2=0x7fb5cc1952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5cb7fe640 1 --2- 172.21.15.88:0/1908302764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5cc0fe6f0 0x7fb5cc1952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:41.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5cb7fe640 1 -- 172.21.15.88:0/1908302764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5bc009560 con 0x7fb5cc0ff000 2024-02-19T19:43:41.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5cb7fe640 1 --2- 172.21.15.88:0/1908302764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc0ff000 0x7fb5cc195800 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000c810 tx=0x7fb5c000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:41.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5c97fa640 1 -- 172.21.15.88:0/1908302764 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5c00041f0 con 0x7fb5cc0ff000 2024-02-19T19:43:41.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5c97fa640 1 -- 172.21.15.88:0/1908302764 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5c0007e70 con 0x7fb5cc0ff000 2024-02-19T19:43:41.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5d2769640 1 -- 172.21.15.88:0/1908302764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5cc199f40 con 0x7fb5cc0ff000 2024-02-19T19:43:41.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5c97fa640 1 -- 172.21.15.88:0/1908302764 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5c00049d0 con 0x7fb5cc0ff000 2024-02-19T19:43:41.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.992+0000 7fb5d2769640 1 -- 172.21.15.88:0/1908302764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc19a4a0 con 0x7fb5cc0ff000 2024-02-19T19:43:41.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.994+0000 7fb5d2769640 1 -- 172.21.15.88:0/1908302764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5cc04e000 con 0x7fb5cc0ff000 2024-02-19T19:43:41.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.994+0000 7fb5c97fa640 1 -- 172.21.15.88:0/1908302764 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fb5c00079b0 con 0x7fb5cc0ff000 2024-02-19T19:43:41.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.994+0000 7fb5c97fa640 1 --2- 172.21.15.88:0/1908302764 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb5b406d810 0x7fb5b406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:41.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.994+0000 7fb5c97fa640 1 -- 172.21.15.88:0/1908302764 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb5c008d3b0 con 0x7fb5cc0ff000 2024-02-19T19:43:41.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.994+0000 7fb5cbfff640 1 --2- 172.21.15.88:0/1908302764 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb5b406d810 0x7fb5b406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:41.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.995+0000 7fb5cbfff640 1 --2- 172.21.15.88:0/1908302764 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb5b406d810 0x7fb5b406fcc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc036040 tx=0x7fb5bc000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:41.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:41.999+0000 7fb5c97fa640 1 -- 172.21.15.88:0/1908302764 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb5c0056b60 con 0x7fb5cc0ff000 2024-02-19T19:43:42.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.208+0000 7fb5d2769640 1 -- 172.21.15.88:0/1908302764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7fb5cc1070e0 con 0x7fb5cc0ff000 2024-02-19T19:43:42.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.209+0000 7fb5c97fa640 1 -- 172.21.15.88:0/1908302764 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v40) v1 ==== 107+0+4949 (secure 0 0 0) 0x7fb5c005a220 con 0x7fb5cc0ff000 2024-02-19T19:43:42.209 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:42.209 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44365,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/594928725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":594928725},{"type":"v1","addr":"172.21.15.132:6825","nonce":594928725}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44753,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6827/858155091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":858155091},{"type":"v1","addr":"172.21.15.132:6827","nonce":858155091}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":23,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:10:28.265493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:42.209 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 32 2024-02-19T19:43:42.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.211+0000 7fb5d2769640 1 -- 172.21.15.88:0/1908302764 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb5b406d810 msgr2=0x7fb5b406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.211+0000 7fb5d2769640 1 --2- 172.21.15.88:0/1908302764 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb5b406d810 0x7fb5b406fcc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb5bc036040 tx=0x7fb5bc000bc0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.211+0000 7fb5d2769640 1 -- 172.21.15.88:0/1908302764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc0ff000 msgr2=0x7fb5cc195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.211+0000 7fb5d2769640 1 --2- 172.21.15.88:0/1908302764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc0ff000 0x7fb5cc195800 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000c810 tx=0x7fb5c000ccd0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.211+0000 7fb5d2769640 1 -- 172.21.15.88:0/1908302764 shutdown_connections 2024-02-19T19:43:42.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.211+0000 7fb5d2769640 1 --2- 172.21.15.88:0/1908302764 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fb5b406d810 0x7fb5b406fcc0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.211+0000 7fb5d2769640 1 --2- 172.21.15.88:0/1908302764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc0ff000 0x7fb5cc195800 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.211+0000 7fb5d2769640 1 --2- 172.21.15.88:0/1908302764 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fb5cc0fe6f0 0x7fb5cc1952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.211+0000 7fb5d2769640 1 -- 172.21.15.88:0/1908302764 >> 172.21.15.88:0/1908302764 conn(0x7fb5cc073d20 msgr2=0x7fb5cc1042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:42.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.211+0000 7fb5d2769640 1 -- 172.21.15.88:0/1908302764 shutdown_connections 2024-02-19T19:43:42.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.211+0000 7fb5d2769640 1 -- 172.21.15.88:0/1908302764 wait complete. 2024-02-19T19:43:42.221 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:41 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3537642540' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-19T19:43:42.221 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:41 smithi088 ceph-mon[88643]: pgmap v1206: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:42.221 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:41 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2526296017' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-19T19:43:42.221 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:41 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2487903493' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-19T19:43:42.222 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-02-19T19:43:42.222 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-02-19T19:43:42.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:41 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3537642540' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-02-19T19:43:42.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:41 smithi132 ceph-mon[70676]: pgmap v1206: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:42.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:41 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2526296017' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-02-19T19:43:42.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:41 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2487903493' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-02-19T19:43:42.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.324+0000 7f026e3a2640 1 -- 172.21.15.88:0/2316519763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02680fd9f0 msgr2=0x7f02680fde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.324+0000 7f026e3a2640 1 --2- 172.21.15.88:0/2316519763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02680fd9f0 0x7f02680fde60 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f025c009880 tx=0x7f025c02bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.324+0000 7f026e3a2640 1 -- 172.21.15.88:0/2316519763 shutdown_connections 2024-02-19T19:43:42.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.324+0000 7f026e3a2640 1 --2- 172.21.15.88:0/2316519763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02680fd9f0 0x7f02680fde60 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.324+0000 7f026e3a2640 1 --2- 172.21.15.88:0/2316519763 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f0268105510 0x7f02681058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.324+0000 7f026e3a2640 1 -- 172.21.15.88:0/2316519763 >> 172.21.15.88:0/2316519763 conn(0x7f0268075a60 msgr2=0x7f0268075e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:42.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.324+0000 7f026e3a2640 1 -- 172.21.15.88:0/2316519763 shutdown_connections 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.324+0000 7f026e3a2640 1 -- 172.21.15.88:0/2316519763 wait complete. 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026e3a2640 1 Processor -- start 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026e3a2640 1 -- start start 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026e3a2640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02680fd9f0 0x7f02681951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026e3a2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0268105510 0x7f0268195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026e3a2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0268195d90 con 0x7f0268105510 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026e3a2640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0268199b60 con 0x7f02680fd9f0 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026cb9f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0268105510 0x7f0268195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026cb9f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0268105510 0x7f0268195700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57928/0 (socket says 172.21.15.88:57928) 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026cb9f640 1 -- 172.21.15.88:0/3491724770 learned_addr learned my addr 172.21.15.88:0/3491724770 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026d3a0640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02680fd9f0 0x7f02681951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026cb9f640 1 -- 172.21.15.88:0/3491724770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02680fd9f0 msgr2=0x7f02681951c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026cb9f640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02680fd9f0 0x7f02681951c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026cb9f640 1 -- 172.21.15.88:0/3491724770 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c009560 con 0x7f0268105510 2024-02-19T19:43:42.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026d3a0640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02680fd9f0 0x7f02681951c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:42.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.325+0000 7f026cb9f640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0268105510 0x7f0268195700 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f025c009230 tx=0x7f025c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:42.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.326+0000 7f025a7fc640 1 -- 172.21.15.88:0/3491724770 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f025c0035f0 con 0x7f0268105510 2024-02-19T19:43:42.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.326+0000 7f026e3a2640 1 -- 172.21.15.88:0/3491724770 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0268199de0 con 0x7f0268105510 2024-02-19T19:43:42.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.326+0000 7f025a7fc640 1 -- 172.21.15.88:0/3491724770 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f025c003c10 con 0x7f0268105510 2024-02-19T19:43:42.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.326+0000 7f025a7fc640 1 -- 172.21.15.88:0/3491724770 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f025c034440 con 0x7f0268105510 2024-02-19T19:43:42.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.326+0000 7f026e3a2640 1 -- 172.21.15.88:0/3491724770 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f026819a2e0 con 0x7f0268105510 2024-02-19T19:43:42.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.327+0000 7f026e3a2640 1 -- 172.21.15.88:0/3491724770 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f026804e000 con 0x7f0268105510 2024-02-19T19:43:42.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.327+0000 7f025a7fc640 1 -- 172.21.15.88:0/3491724770 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f025c004fd0 con 0x7f0268105510 2024-02-19T19:43:42.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.327+0000 7f025a7fc640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f025406d810 0x7f025406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:42.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.328+0000 7f026d3a0640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f025406d810 0x7f025406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:42.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.328+0000 7f025a7fc640 1 -- 172.21.15.88:0/3491724770 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f025c0af9c0 con 0x7f0268105510 2024-02-19T19:43:42.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.328+0000 7f026d3a0640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f025406d810 0x7f025406fcc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0264005fd0 tx=0x7f0264005ea0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:42.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.332+0000 7f025a7fc640 1 -- 172.21.15.88:0/3491724770 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f025c079170 con 0x7f0268105510 2024-02-19T19:43:42.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.540+0000 7f026e3a2640 1 -- 172.21.15.88:0/3491724770 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f02681079e0 con 0x7f0268105510 2024-02-19T19:43:42.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.540+0000 7f025a7fc640 1 -- 172.21.15.88:0/3491724770 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v40) v1 ==== 107+0+4948 (secure 0 0 0) 0x7f025c07c830 con 0x7f0268105510 2024-02-19T19:43:42.541 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:42.541 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":26},{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":44365,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/594928725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":594928725},{"type":"v1","addr":"172.21.15.132:6825","nonce":594928725}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44753,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6827/858155091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":858155091},{"type":"v1","addr":"172.21.15.132:6827","nonce":858155091}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:41:25.832554+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:42.541 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 33 2024-02-19T19:43:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.542+0000 7f026e3a2640 1 -- 172.21.15.88:0/3491724770 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f025406d810 msgr2=0x7f025406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.542+0000 7f026e3a2640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f025406d810 0x7f025406fcc0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f0264005fd0 tx=0x7f0264005ea0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.542+0000 7f026e3a2640 1 -- 172.21.15.88:0/3491724770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0268105510 msgr2=0x7f0268195700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.542+0000 7f026e3a2640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0268105510 0x7f0268195700 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f025c009230 tx=0x7f025c004040 comp rx=0 tx=0).stop 2024-02-19T19:43:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.542+0000 7f026e3a2640 1 -- 172.21.15.88:0/3491724770 shutdown_connections 2024-02-19T19:43:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.542+0000 7f026e3a2640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f025406d810 0x7f025406fcc0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.542+0000 7f026e3a2640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0268105510 0x7f0268195700 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.542+0000 7f026e3a2640 1 --2- 172.21.15.88:0/3491724770 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f02680fd9f0 0x7f02681951c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.542+0000 7f026e3a2640 1 -- 172.21.15.88:0/3491724770 >> 172.21.15.88:0/3491724770 conn(0x7f0268075a60 msgr2=0x7f02680fb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.542+0000 7f026e3a2640 1 -- 172.21.15.88:0/3491724770 shutdown_connections 2024-02-19T19:43:42.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.542+0000 7f026e3a2640 1 -- 172.21.15.88:0/3491724770 wait complete. 2024-02-19T19:43:42.554 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-02-19T19:43:42.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 -- 172.21.15.88:0/3952514503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc00104770 msgr2=0x7fcc00104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 --2- 172.21.15.88:0/3952514503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc00104770 0x7fcc00104b40 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0009880 tx=0x7fcbf002bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 -- 172.21.15.88:0/3952514503 shutdown_connections 2024-02-19T19:43:42.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 --2- 172.21.15.88:0/3952514503 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc000fe760 0x7fcc000febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 --2- 172.21.15.88:0/3952514503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc00104770 0x7fcc00104b40 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 -- 172.21.15.88:0/3952514503 >> 172.21.15.88:0/3952514503 conn(0x7fcc000fa260 msgr2=0x7fcc000fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:42.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 -- 172.21.15.88:0/3952514503 shutdown_connections 2024-02-19T19:43:42.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 -- 172.21.15.88:0/3952514503 wait complete. 2024-02-19T19:43:42.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 Processor -- start 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 -- start start 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc000fe760 0x7fcc00196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc00104770 0x7fcc00196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc00196e90 con 0x7fcc00104770 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.657+0000 7fcc05e95640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc00196fd0 con 0x7fcc000fe760 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcbf7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc00104770 0x7fcc00196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcbf7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc00104770 0x7fcc00196870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57946/0 (socket says 172.21.15.88:57946) 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcbf7fff640 1 -- 172.21.15.88:0/2690884337 learned_addr learned my addr 172.21.15.88:0/2690884337 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcbff7fe640 1 --2- 172.21.15.88:0/2690884337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc000fe760 0x7fcc00196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcbf7fff640 1 -- 172.21.15.88:0/2690884337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc000fe760 msgr2=0x7fcc00196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcbf7fff640 1 --2- 172.21.15.88:0/2690884337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc000fe760 0x7fcc00196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcbf7fff640 1 -- 172.21.15.88:0/2690884337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf0009560 con 0x7fcc00104770 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcbf7fff640 1 --2- 172.21.15.88:0/2690884337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc00104770 0x7fcc00196870 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800d6f0 tx=0x7fcbe800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:42.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcbfd7fa640 1 -- 172.21.15.88:0/2690884337 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbe8009a30 con 0x7fcc00104770 2024-02-19T19:43:42.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcbfd7fa640 1 -- 172.21.15.88:0/2690884337 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcbe80054e0 con 0x7fcc00104770 2024-02-19T19:43:42.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcc05e95640 1 -- 172.21.15.88:0/2690884337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0019f3a0 con 0x7fcc00104770 2024-02-19T19:43:42.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcbfd7fa640 1 -- 172.21.15.88:0/2690884337 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbe8010460 con 0x7fcc00104770 2024-02-19T19:43:42.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.658+0000 7fcc05e95640 1 -- 172.21.15.88:0/2690884337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0019f8f0 con 0x7fcc00104770 2024-02-19T19:43:42.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.659+0000 7fcc05e95640 1 -- 172.21.15.88:0/2690884337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0004e000 con 0x7fcc00104770 2024-02-19T19:43:42.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.660+0000 7fcbfd7fa640 1 -- 172.21.15.88:0/2690884337 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcbe8009b90 con 0x7fcc00104770 2024-02-19T19:43:42.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.660+0000 7fcbfd7fa640 1 --2- 172.21.15.88:0/2690884337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcbec06d810 0x7fcbec06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:42.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.660+0000 7fcbfd7fa640 1 -- 172.21.15.88:0/2690884337 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcbe808cf20 con 0x7fcc00104770 2024-02-19T19:43:42.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.660+0000 7fcbff7fe640 1 --2- 172.21.15.88:0/2690884337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcbec06d810 0x7fcbec06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:42.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.661+0000 7fcbff7fe640 1 --2- 172.21.15.88:0/2690884337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcbec06d810 0x7fcbec06fcc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0000c00 tx=0x7fcbf0005fd0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:42.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.665+0000 7fcbfd7fa640 1 -- 172.21.15.88:0/2690884337 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcbe80c0d20 con 0x7fcc00104770 2024-02-19T19:43:42.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.876+0000 7fcc05e95640 1 -- 172.21.15.88:0/2690884337 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7fcc00066860 con 0x7fcc00104770 2024-02-19T19:43:42.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.876+0000 7fcbfd7fa640 1 -- 172.21.15.88:0/2690884337 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v40) v1 ==== 107+0+5005 (secure 0 0 0) 0x7fcbe8017090 con 0x7fcc00104770 2024-02-19T19:43:42.876 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:42.877 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44365,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/594928725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":594928725},{"type":"v1","addr":"172.21.15.132:6825","nonce":594928725}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44753,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6827/858155091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":858155091},{"type":"v1","addr":"172.21.15.132:6827","nonce":858155091}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:41:26.834051+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34430,"mds_1":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34462":{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":1,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:42.877 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 34 2024-02-19T19:43:42.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.878+0000 7fcc05e95640 1 -- 172.21.15.88:0/2690884337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcbec06d810 msgr2=0x7fcbec06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.878+0000 7fcc05e95640 1 --2- 172.21.15.88:0/2690884337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcbec06d810 0x7fcbec06fcc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0000c00 tx=0x7fcbf0005fd0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.878+0000 7fcc05e95640 1 -- 172.21.15.88:0/2690884337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc00104770 msgr2=0x7fcc00196870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.878+0000 7fcc05e95640 1 --2- 172.21.15.88:0/2690884337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc00104770 0x7fcc00196870 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fcbe800d6f0 tx=0x7fcbe800dbb0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.878+0000 7fcc05e95640 1 -- 172.21.15.88:0/2690884337 shutdown_connections 2024-02-19T19:43:42.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.878+0000 7fcc05e95640 1 --2- 172.21.15.88:0/2690884337 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcbec06d810 0x7fcbec06fcc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.878+0000 7fcc05e95640 1 --2- 172.21.15.88:0/2690884337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc00104770 0x7fcc00196870 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.878+0000 7fcc05e95640 1 --2- 172.21.15.88:0/2690884337 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcc000fe760 0x7fcc00196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.878+0000 7fcc05e95640 1 -- 172.21.15.88:0/2690884337 >> 172.21.15.88:0/2690884337 conn(0x7fcc000fa260 msgr2=0x7fcc000fbcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:42.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.878+0000 7fcc05e95640 1 -- 172.21.15.88:0/2690884337 shutdown_connections 2024-02-19T19:43:42.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.878+0000 7fcc05e95640 1 -- 172.21.15.88:0/2690884337 wait complete. 2024-02-19T19:43:42.890 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-02-19T19:43:42.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.992+0000 7fed5a8e0640 1 -- 172.21.15.88:0/3315774658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed54067350 msgr2=0x7fed540677c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.992+0000 7fed5a8e0640 1 --2- 172.21.15.88:0/3315774658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed54067350 0x7fed540677c0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fed48009880 tx=0x7fed4802bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.992+0000 7fed5a8e0640 1 -- 172.21.15.88:0/3315774658 shutdown_connections 2024-02-19T19:43:42.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.992+0000 7fed5a8e0640 1 --2- 172.21.15.88:0/3315774658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed54067350 0x7fed540677c0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.992+0000 7fed5a8e0640 1 --2- 172.21.15.88:0/3315774658 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed54105690 0x7fed54105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.992+0000 7fed5a8e0640 1 -- 172.21.15.88:0/3315774658 >> 172.21.15.88:0/3315774658 conn(0x7fed540743d0 msgr2=0x7fed540747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.992+0000 7fed5a8e0640 1 -- 172.21.15.88:0/3315774658 shutdown_connections 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.992+0000 7fed5a8e0640 1 -- 172.21.15.88:0/3315774658 wait complete. 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.993+0000 7fed5a8e0640 1 Processor -- start 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.993+0000 7fed5a8e0640 1 -- start start 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.993+0000 7fed5a8e0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed54067350 0x7fed54195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.993+0000 7fed5a8e0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed54105690 0x7fed54195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.993+0000 7fed5a8e0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed54195e80 con 0x7fed54067350 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.993+0000 7fed5a8e0640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed54195ff0 con 0x7fed54105690 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.993+0000 7fed53fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed54067350 0x7fed54195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.993+0000 7fed53fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed54067350 0x7fed54195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57960/0 (socket says 172.21.15.88:57960) 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.993+0000 7fed53fff640 1 -- 172.21.15.88:0/586274775 learned_addr learned my addr 172.21.15.88:0/586274775 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.994+0000 7fed53fff640 1 -- 172.21.15.88:0/586274775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed54105690 msgr2=0x7fed54195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.994+0000 7fed53fff640 1 --2- 172.21.15.88:0/586274775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed54105690 0x7fed54195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.994+0000 7fed53fff640 1 -- 172.21.15.88:0/586274775 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48009560 con 0x7fed54067350 2024-02-19T19:43:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.994+0000 7fed53fff640 1 --2- 172.21.15.88:0/586274775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed54067350 0x7fed54195340 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fed4400c8a0 tx=0x7fed4400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.994+0000 7fed517fa640 1 -- 172.21.15.88:0/586274775 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed44007b80 con 0x7fed54067350 2024-02-19T19:43:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.994+0000 7fed517fa640 1 -- 172.21.15.88:0/586274775 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed44007ce0 con 0x7fed54067350 2024-02-19T19:43:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.994+0000 7fed5a8e0640 1 -- 172.21.15.88:0/586274775 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed54199e80 con 0x7fed54067350 2024-02-19T19:43:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.994+0000 7fed517fa640 1 -- 172.21.15.88:0/586274775 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed44018810 con 0x7fed54067350 2024-02-19T19:43:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.994+0000 7fed5a8e0640 1 -- 172.21.15.88:0/586274775 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed5419a3e0 con 0x7fed54067350 2024-02-19T19:43:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.995+0000 7fed5a8e0640 1 -- 172.21.15.88:0/586274775 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed5404e000 con 0x7fed54067350 2024-02-19T19:43:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.995+0000 7fed517fa640 1 -- 172.21.15.88:0/586274775 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fed44018a80 con 0x7fed54067350 2024-02-19T19:43:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.996+0000 7fed517fa640 1 --2- 172.21.15.88:0/586274775 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed3c06d810 0x7fed3c06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.996+0000 7fed517fa640 1 -- 172.21.15.88:0/586274775 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fed4408d700 con 0x7fed54067350 2024-02-19T19:43:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.996+0000 7fed537fe640 1 --2- 172.21.15.88:0/586274775 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed3c06d810 0x7fed3c06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:42.996+0000 7fed537fe640 1 --2- 172.21.15.88:0/586274775 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed3c06d810 0x7fed3c06fcc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fed48036040 tx=0x7fed48000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:43.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.000+0000 7fed517fa640 1 -- 172.21.15.88:0/586274775 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed44056eb0 con 0x7fed54067350 2024-02-19T19:43:43.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.209+0000 7fed5a8e0640 1 -- 172.21.15.88:0/586274775 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7fed54107aa0 con 0x7fed54067350 2024-02-19T19:43:43.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.210+0000 7fed517fa640 1 -- 172.21.15.88:0/586274775 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v40) v1 ==== 107+0+5008 (secure 0 0 0) 0x7fed4405a570 con 0x7fed54067350 2024-02-19T19:43:43.211 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:43.211 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44365,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/594928725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":594928725},{"type":"v1","addr":"172.21.15.132:6825","nonce":594928725}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44753,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6827/858155091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":858155091},{"type":"v1","addr":"172.21.15.132:6827","nonce":858155091}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:41:33.599036+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34430,"mds_1":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":0,"incarnation":34,"state":"up:replay","state_seq":1,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34462":{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":1,"incarnation":34,"state":"up:resolve","state_seq":461,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:43.211 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 35 2024-02-19T19:43:43.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.212+0000 7fed5a8e0640 1 -- 172.21.15.88:0/586274775 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed3c06d810 msgr2=0x7fed3c06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:43.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.212+0000 7fed5a8e0640 1 --2- 172.21.15.88:0/586274775 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed3c06d810 0x7fed3c06fcc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fed48036040 tx=0x7fed48000bc0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.212+0000 7fed5a8e0640 1 -- 172.21.15.88:0/586274775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed54067350 msgr2=0x7fed54195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:43.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.212+0000 7fed5a8e0640 1 --2- 172.21.15.88:0/586274775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed54067350 0x7fed54195340 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fed4400c8a0 tx=0x7fed4400cd60 comp rx=0 tx=0).stop 2024-02-19T19:43:43.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.212+0000 7fed5a8e0640 1 -- 172.21.15.88:0/586274775 shutdown_connections 2024-02-19T19:43:43.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.212+0000 7fed5a8e0640 1 --2- 172.21.15.88:0/586274775 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fed3c06d810 0x7fed3c06fcc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.212+0000 7fed5a8e0640 1 --2- 172.21.15.88:0/586274775 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fed54105690 0x7fed54195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.212+0000 7fed5a8e0640 1 --2- 172.21.15.88:0/586274775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed54067350 0x7fed54195340 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.212+0000 7fed5a8e0640 1 -- 172.21.15.88:0/586274775 >> 172.21.15.88:0/586274775 conn(0x7fed540743d0 msgr2=0x7fed540fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:43.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.212+0000 7fed5a8e0640 1 -- 172.21.15.88:0/586274775 shutdown_connections 2024-02-19T19:43:43.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.212+0000 7fed5a8e0640 1 -- 172.21.15.88:0/586274775 wait complete. 2024-02-19T19:43:43.222 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:42 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3977207097' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-19T19:43:43.222 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:42 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1908302764' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-19T19:43:43.223 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:42 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3491724770' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-19T19:43:43.224 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-02-19T19:43:43.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:42 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3977207097' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-02-19T19:43:43.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:42 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1908302764' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-02-19T19:43:43.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:42 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3491724770' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-02-19T19:43:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.325+0000 7f75f786a640 1 -- 172.21.15.88:0/1259546230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75f0101410 msgr2=0x7f75f0109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.325+0000 7f75f786a640 1 --2- 172.21.15.88:0/1259546230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75f0101410 0x7f75f0109900 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f75e0009880 tx=0x7f75e002bd70 comp rx=0 tx=0).stop 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.325+0000 7f75f786a640 1 -- 172.21.15.88:0/1259546230 shutdown_connections 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.325+0000 7f75f786a640 1 --2- 172.21.15.88:0/1259546230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75f0101410 0x7f75f0109900 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.325+0000 7f75f786a640 1 --2- 172.21.15.88:0/1259546230 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0100b00 0x7f75f0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.325+0000 7f75f786a640 1 -- 172.21.15.88:0/1259546230 >> 172.21.15.88:0/1259546230 conn(0x7f75f00fa2f0 msgr2=0x7f75f00fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.325+0000 7f75f786a640 1 -- 172.21.15.88:0/1259546230 shutdown_connections 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.325+0000 7f75f786a640 1 -- 172.21.15.88:0/1259546230 wait complete. 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f786a640 1 Processor -- start 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f786a640 1 -- start start 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f786a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75f0100b00 0x7f75f0195590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f786a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0101410 0x7f75f0195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f786a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f0196160 con 0x7f75f0100b00 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f786a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f0199f30 con 0x7f75f0101410 2024-02-19T19:43:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f5606640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75f0100b00 0x7f75f0195590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f5606640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75f0100b00 0x7f75f0195590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57972/0 (socket says 172.21.15.88:57972) 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f5606640 1 -- 172.21.15.88:0/2492504418 learned_addr learned my addr 172.21.15.88:0/2492504418 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f4e05640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0101410 0x7f75f0195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f5606640 1 -- 172.21.15.88:0/2492504418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0101410 msgr2=0x7f75f0195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f5606640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0101410 0x7f75f0195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f5606640 1 -- 172.21.15.88:0/2492504418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0009560 con 0x7f75f0100b00 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f4e05640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0101410 0x7f75f0195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75f5606640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75f0100b00 0x7f75f0195590 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00c8f0 tx=0x7f75ec00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.326+0000 7f75e67fc640 1 -- 172.21.15.88:0/2492504418 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75ec007b80 con 0x7f75f0100b00 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.327+0000 7f75e67fc640 1 -- 172.21.15.88:0/2492504418 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75ec0054e0 con 0x7f75f0100b00 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.327+0000 7f75f786a640 1 -- 172.21.15.88:0/2492504418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f019a210 con 0x7f75f0100b00 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.327+0000 7f75e67fc640 1 -- 172.21.15.88:0/2492504418 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75ec00f450 con 0x7f75f0100b00 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.327+0000 7f75f786a640 1 -- 172.21.15.88:0/2492504418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f019a770 con 0x7f75f0100b00 2024-02-19T19:43:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.327+0000 7f75f786a640 1 -- 172.21.15.88:0/2492504418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f004e000 con 0x7f75f0100b00 2024-02-19T19:43:43.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.328+0000 7f75e67fc640 1 -- 172.21.15.88:0/2492504418 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f75ec004f60 con 0x7f75f0100b00 2024-02-19T19:43:43.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.328+0000 7f75e67fc640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75dc06d810 0x7f75dc06fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:43.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.329+0000 7f75f4e05640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75dc06d810 0x7f75dc06fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:43.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.329+0000 7f75e67fc640 1 -- 172.21.15.88:0/2492504418 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f75ec08d3f0 con 0x7f75f0100b00 2024-02-19T19:43:43.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.329+0000 7f75f4e05640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75dc06d810 0x7f75dc06fcc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f75f0196ba0 tx=0x7f75e002cca0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:43.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.333+0000 7f75e67fc640 1 -- 172.21.15.88:0/2492504418 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f75ec056b00 con 0x7f75f0100b00 2024-02-19T19:43:43.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.543+0000 7f75f786a640 1 -- 172.21.15.88:0/2492504418 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f75f0066860 con 0x7f75f0100b00 2024-02-19T19:43:43.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.544+0000 7f75e67fc640 1 -- 172.21.15.88:0/2492504418 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v40) v1 ==== 107+0+5011 (secure 0 0 0) 0x7f75ec05a1c0 con 0x7f75f0100b00 2024-02-19T19:43:43.544 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:43.544 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44365,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/594928725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":594928725},{"type":"v1","addr":"172.21.15.132:6825","nonce":594928725}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44753,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6827/858155091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":858155091},{"type":"v1","addr":"172.21.15.132:6827","nonce":858155091}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:41:34.450025+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34430,"mds_1":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":0,"incarnation":34,"state":"up:resolve","state_seq":466,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34462":{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":1,"incarnation":34,"state":"up:resolve","state_seq":461,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:43.544 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 36 2024-02-19T19:43:43.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.546+0000 7f75f786a640 1 -- 172.21.15.88:0/2492504418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75dc06d810 msgr2=0x7f75dc06fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:43.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.546+0000 7f75f786a640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75dc06d810 0x7f75dc06fcc0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f75f0196ba0 tx=0x7f75e002cca0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.546+0000 7f75f786a640 1 -- 172.21.15.88:0/2492504418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75f0100b00 msgr2=0x7f75f0195590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:43.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.546+0000 7f75f786a640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75f0100b00 0x7f75f0195590 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f75ec00c8f0 tx=0x7f75ec00cdb0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.546+0000 7f75f786a640 1 -- 172.21.15.88:0/2492504418 shutdown_connections 2024-02-19T19:43:43.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.546+0000 7f75f786a640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f75dc06d810 0x7f75dc06fcc0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.546+0000 7f75f786a640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f75f0101410 0x7f75f0195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.546+0000 7f75f786a640 1 --2- 172.21.15.88:0/2492504418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75f0100b00 0x7f75f0195590 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.546+0000 7f75f786a640 1 -- 172.21.15.88:0/2492504418 >> 172.21.15.88:0/2492504418 conn(0x7f75f00fa2f0 msgr2=0x7f75f01040a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:43.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.546+0000 7f75f786a640 1 -- 172.21.15.88:0/2492504418 shutdown_connections 2024-02-19T19:43:43.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.546+0000 7f75f786a640 1 -- 172.21.15.88:0/2492504418 wait complete. 2024-02-19T19:43:43.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-02-19T19:43:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 -- 172.21.15.88:0/1703822841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09340fe590 msgr2=0x7f09340fea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 --2- 172.21.15.88:0/1703822841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09340fe590 0x7f09340fea00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f091c009880 tx=0x7f091c02bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 -- 172.21.15.88:0/1703822841 shutdown_connections 2024-02-19T19:43:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 --2- 172.21.15.88:0/1703822841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09340fe590 0x7f09340fea00 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 --2- 172.21.15.88:0/1703822841 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09341045c0 0x7f0934104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 -- 172.21.15.88:0/1703822841 >> 172.21.15.88:0/1703822841 conn(0x7f09340fa070 msgr2=0x7f09340fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 -- 172.21.15.88:0/1703822841 shutdown_connections 2024-02-19T19:43:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 -- 172.21.15.88:0/1703822841 wait complete. 2024-02-19T19:43:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 Processor -- start 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 -- start start 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09340fe590 0x7f0934196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.659+0000 7f09392e0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09341045c0 0x7f09341969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f09392e0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934196f90 con 0x7f09341045c0 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f09392e0640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934197100 con 0x7f09340fe590 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f09327fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09341045c0 0x7f09341969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f09327fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09341045c0 0x7f09341969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57986/0 (socket says 172.21.15.88:57986) 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f09327fc640 1 -- 172.21.15.88:0/3061030339 learned_addr learned my addr 172.21.15.88:0/3061030339 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f0932ffd640 1 --2- 172.21.15.88:0/3061030339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09340fe590 0x7f0934196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f09327fc640 1 -- 172.21.15.88:0/3061030339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09340fe590 msgr2=0x7f0934196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f09327fc640 1 --2- 172.21.15.88:0/3061030339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09340fe590 0x7f0934196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f09327fc640 1 -- 172.21.15.88:0/3061030339 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c009560 con 0x7f09341045c0 2024-02-19T19:43:43.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f09327fc640 1 --2- 172.21.15.88:0/3061030339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09341045c0 0x7f09341969c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f091c009230 tx=0x7f091c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:43.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f092bfff640 1 -- 172.21.15.88:0/3061030339 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f091c0035f0 con 0x7f09341045c0 2024-02-19T19:43:43.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f09392e0640 1 -- 172.21.15.88:0/3061030339 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093419f4f0 con 0x7f09341045c0 2024-02-19T19:43:43.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f092bfff640 1 -- 172.21.15.88:0/3061030339 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f091c003c10 con 0x7f09341045c0 2024-02-19T19:43:43.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f092bfff640 1 -- 172.21.15.88:0/3061030339 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f091c034440 con 0x7f09341045c0 2024-02-19T19:43:43.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.660+0000 7f09392e0640 1 -- 172.21.15.88:0/3061030339 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093419f910 con 0x7f09341045c0 2024-02-19T19:43:43.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.661+0000 7f09392e0640 1 -- 172.21.15.88:0/3061030339 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093404e000 con 0x7f09341045c0 2024-02-19T19:43:43.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.662+0000 7f092bfff640 1 -- 172.21.15.88:0/3061030339 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f091c004fd0 con 0x7f09341045c0 2024-02-19T19:43:43.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.662+0000 7f092bfff640 1 --2- 172.21.15.88:0/3061030339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f092006d810 0x7f092006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:43.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.662+0000 7f092bfff640 1 -- 172.21.15.88:0/3061030339 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f091c0af9c0 con 0x7f09341045c0 2024-02-19T19:43:43.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.662+0000 7f0932ffd640 1 --2- 172.21.15.88:0/3061030339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f092006d810 0x7f092006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:43.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.663+0000 7f0932ffd640 1 --2- 172.21.15.88:0/3061030339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f092006d810 0x7f092006fcc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f092c005fd0 tx=0x7f092c005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:43.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.667+0000 7f092bfff640 1 -- 172.21.15.88:0/3061030339 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f091c079170 con 0x7f09341045c0 2024-02-19T19:43:43.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.880+0000 7f09392e0640 1 -- 172.21.15.88:0/3061030339 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f09341069d0 con 0x7f09341045c0 2024-02-19T19:43:43.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.880+0000 7f092bfff640 1 -- 172.21.15.88:0/3061030339 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v40) v1 ==== 107+0+5015 (secure 0 0 0) 0x7f091c07c830 con 0x7f09341045c0 2024-02-19T19:43:43.881 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:43.881 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44365,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/594928725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":594928725},{"type":"v1","addr":"172.21.15.132:6825","nonce":594928725}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44753,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6827/858155091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":858155091},{"type":"v1","addr":"172.21.15.132:6827","nonce":858155091}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:41:34.760870+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34430,"mds_1":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":0,"incarnation":34,"state":"up:reconnect","state_seq":467,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34462":{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":1,"incarnation":34,"state":"up:reconnect","state_seq":462,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:43.881 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 37 2024-02-19T19:43:43.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.882+0000 7f09392e0640 1 -- 172.21.15.88:0/3061030339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f092006d810 msgr2=0x7f092006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:43.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.882+0000 7f09392e0640 1 --2- 172.21.15.88:0/3061030339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f092006d810 0x7f092006fcc0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f092c005fd0 tx=0x7f092c005e20 comp rx=0 tx=0).stop 2024-02-19T19:43:43.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.882+0000 7f09392e0640 1 -- 172.21.15.88:0/3061030339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09341045c0 msgr2=0x7f09341969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:43.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.882+0000 7f09392e0640 1 --2- 172.21.15.88:0/3061030339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09341045c0 0x7f09341969c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f091c009230 tx=0x7f091c004040 comp rx=0 tx=0).stop 2024-02-19T19:43:43.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.882+0000 7f09392e0640 1 -- 172.21.15.88:0/3061030339 shutdown_connections 2024-02-19T19:43:43.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.882+0000 7f09392e0640 1 --2- 172.21.15.88:0/3061030339 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f092006d810 0x7f092006fcc0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.882+0000 7f09392e0640 1 --2- 172.21.15.88:0/3061030339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09341045c0 0x7f09341969c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.882+0000 7f09392e0640 1 --2- 172.21.15.88:0/3061030339 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f09340fe590 0x7f0934196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:43.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.882+0000 7f09392e0640 1 -- 172.21.15.88:0/3061030339 >> 172.21.15.88:0/3061030339 conn(0x7f09340fa070 msgr2=0x7f09340fbf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:43.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.882+0000 7f09392e0640 1 -- 172.21.15.88:0/3061030339 shutdown_connections 2024-02-19T19:43:43.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.883+0000 7f09392e0640 1 -- 172.21.15.88:0/3061030339 wait complete. 2024-02-19T19:43:43.894 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-02-19T19:43:43.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 -- 172.21.15.88:0/2622298943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f447c104800 msgr2=0x7f447c104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:43.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 --2- 172.21.15.88:0/2622298943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f447c104800 0x7f447c104bd0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f446c009880 tx=0x7f446c02bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 -- 172.21.15.88:0/2622298943 shutdown_connections 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 --2- 172.21.15.88:0/2622298943 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c0fe7f0 0x7f447c0fec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 --2- 172.21.15.88:0/2622298943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f447c104800 0x7f447c104bd0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 -- 172.21.15.88:0/2622298943 >> 172.21.15.88:0/2622298943 conn(0x7f447c0fa2f0 msgr2=0x7f447c0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 -- 172.21.15.88:0/2622298943 shutdown_connections 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 -- 172.21.15.88:0/2622298943 wait complete. 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 Processor -- start 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 -- start start 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f447c0fe7f0 0x7f447c195510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c104800 0x7f447c195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1960e0 con 0x7f447c0fe7f0 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.996+0000 7f44841b1640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c199eb0 con 0x7f447c104800 2024-02-19T19:43:44.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f4481f4d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f447c0fe7f0 0x7f447c195510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f4481f4d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f447c0fe7f0 0x7f447c195510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58006/0 (socket says 172.21.15.88:58006) 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f4481f4d640 1 -- 172.21.15.88:0/967213980 learned_addr learned my addr 172.21.15.88:0/967213980 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f448174c640 1 --2- 172.21.15.88:0/967213980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c104800 0x7f447c195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f4481f4d640 1 -- 172.21.15.88:0/967213980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c104800 msgr2=0x7f447c195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f4481f4d640 1 --2- 172.21.15.88:0/967213980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c104800 0x7f447c195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f4481f4d640 1 -- 172.21.15.88:0/967213980 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446c009560 con 0x7f447c0fe7f0 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f4481f4d640 1 --2- 172.21.15.88:0/967213980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f447c0fe7f0 0x7f447c195510 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f446c009230 tx=0x7f446c02c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f4472ffd640 1 -- 172.21.15.88:0/967213980 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f446c0035f0 con 0x7f447c0fe7f0 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f4472ffd640 1 -- 172.21.15.88:0/967213980 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f446c003c10 con 0x7f447c0fe7f0 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f44841b1640 1 -- 172.21.15.88:0/967213980 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c19a130 con 0x7f447c0fe7f0 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f4472ffd640 1 -- 172.21.15.88:0/967213980 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f446c005690 con 0x7f447c0fe7f0 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.997+0000 7f44841b1640 1 -- 172.21.15.88:0/967213980 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c19a6b0 con 0x7f447c0fe7f0 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.998+0000 7f44841b1640 1 -- 172.21.15.88:0/967213980 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c04e000 con 0x7f447c0fe7f0 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.999+0000 7f4472ffd640 1 -- 172.21.15.88:0/967213980 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f446c004ed0 con 0x7f447c0fe7f0 2024-02-19T19:43:44.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.999+0000 7f4472ffd640 1 --2- 172.21.15.88:0/967213980 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f446806d810 0x7f446806fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:44.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.999+0000 7f4472ffd640 1 -- 172.21.15.88:0/967213980 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f446c0af990 con 0x7f447c0fe7f0 2024-02-19T19:43:44.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.999+0000 7f448174c640 1 --2- 172.21.15.88:0/967213980 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f446806d810 0x7f446806fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:44.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:43.999+0000 7f448174c640 1 --2- 172.21.15.88:0/967213980 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f446806d810 0x7f446806fcc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4478005fd0 tx=0x7f4478005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:44.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.004+0000 7f4472ffd640 1 -- 172.21.15.88:0/967213980 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f446c079140 con 0x7f447c0fe7f0 2024-02-19T19:43:44.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.213+0000 7f44841b1640 1 -- 172.21.15.88:0/967213980 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f447c066860 con 0x7f447c0fe7f0 2024-02-19T19:43:44.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.213+0000 7f4472ffd640 1 -- 172.21.15.88:0/967213980 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v40) v1 ==== 107+0+5009 (secure 0 0 0) 0x7f446c07c800 con 0x7f447c0fe7f0 2024-02-19T19:43:44.214 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:44.214 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44365,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/594928725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":594928725},{"type":"v1","addr":"172.21.15.132:6825","nonce":594928725}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44753,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6827/858155091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":858155091},{"type":"v1","addr":"172.21.15.132:6827","nonce":858155091}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:41:35.723460+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34430,"mds_1":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":0,"incarnation":34,"state":"up:rejoin","state_seq":468,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34462":{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":1,"incarnation":34,"state":"up:rejoin","state_seq":463,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:44.214 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 38 2024-02-19T19:43:44.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.215+0000 7f44841b1640 1 -- 172.21.15.88:0/967213980 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f446806d810 msgr2=0x7f446806fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:44.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.215+0000 7f44841b1640 1 --2- 172.21.15.88:0/967213980 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f446806d810 0x7f446806fcc0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4478005fd0 tx=0x7f4478005850 comp rx=0 tx=0).stop 2024-02-19T19:43:44.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.215+0000 7f44841b1640 1 -- 172.21.15.88:0/967213980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f447c0fe7f0 msgr2=0x7f447c195510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:44.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.216+0000 7f44841b1640 1 --2- 172.21.15.88:0/967213980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f447c0fe7f0 0x7f447c195510 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f446c009230 tx=0x7f446c02c850 comp rx=0 tx=0).stop 2024-02-19T19:43:44.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.216+0000 7f44841b1640 1 -- 172.21.15.88:0/967213980 shutdown_connections 2024-02-19T19:43:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.216+0000 7f44841b1640 1 --2- 172.21.15.88:0/967213980 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f446806d810 0x7f446806fcc0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.216+0000 7f44841b1640 1 --2- 172.21.15.88:0/967213980 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f447c104800 0x7f447c195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.216+0000 7f44841b1640 1 --2- 172.21.15.88:0/967213980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f447c0fe7f0 0x7f447c195510 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.216+0000 7f44841b1640 1 -- 172.21.15.88:0/967213980 >> 172.21.15.88:0/967213980 conn(0x7f447c0fa2f0 msgr2=0x7f447c102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.216+0000 7f44841b1640 1 -- 172.21.15.88:0/967213980 shutdown_connections 2024-02-19T19:43:44.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.216+0000 7f44841b1640 1 -- 172.21.15.88:0/967213980 wait complete. 2024-02-19T19:43:44.225 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:43 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2690884337' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-19T19:43:44.226 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:43 smithi088 ceph-mon[88643]: pgmap v1207: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:44.226 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:43 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/586274775' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-19T19:43:44.226 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:43:44.226 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:43 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2492504418' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-19T19:43:44.227 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 39 2024-02-19T19:43:44.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:43 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2690884337' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-02-19T19:43:44.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:43 smithi132 ceph-mon[70676]: pgmap v1207: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:44.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:43 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/586274775' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-02-19T19:43:44.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:43:44.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:43 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2492504418' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-02-19T19:43:44.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.332+0000 7f772e00b640 1 -- 172.21.15.88:0/1314188577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7728104590 msgr2=0x7f7728104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:44.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.332+0000 7f772e00b640 1 --2- 172.21.15.88:0/1314188577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7728104590 0x7f7728104960 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f771c009880 tx=0x7f771c02bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.332+0000 7f772e00b640 1 -- 172.21.15.88:0/1314188577 shutdown_connections 2024-02-19T19:43:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.332+0000 7f772e00b640 1 --2- 172.21.15.88:0/1314188577 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77280fe5b0 0x7f77280fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.332+0000 7f772e00b640 1 --2- 172.21.15.88:0/1314188577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7728104590 0x7f7728104960 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.332+0000 7f772e00b640 1 -- 172.21.15.88:0/1314188577 >> 172.21.15.88:0/1314188577 conn(0x7f77280743d0 msgr2=0x7f77280747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.332+0000 7f772e00b640 1 -- 172.21.15.88:0/1314188577 shutdown_connections 2024-02-19T19:43:44.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.332+0000 7f772e00b640 1 -- 172.21.15.88:0/1314188577 wait complete. 2024-02-19T19:43:44.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f772e00b640 1 Processor -- start 2024-02-19T19:43:44.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f772e00b640 1 -- start start 2024-02-19T19:43:44.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f772e00b640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77280fe5b0 0x7f77281941c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f772e00b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7728104590 0x7f7728194700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f772e00b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7728194d90 con 0x7f7728104590 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f772e00b640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7728198b60 con 0x7f77280fe5b0 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f7726ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7728104590 0x7f7728194700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f7726ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7728104590 0x7f7728194700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58028/0 (socket says 172.21.15.88:58028) 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f7726ffd640 1 -- 172.21.15.88:0/1369893978 learned_addr learned my addr 172.21.15.88:0/1369893978 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f77277fe640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77280fe5b0 0x7f77281941c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f7726ffd640 1 -- 172.21.15.88:0/1369893978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77280fe5b0 msgr2=0x7f77281941c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f7726ffd640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77280fe5b0 0x7f77281941c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.333+0000 7f7726ffd640 1 -- 172.21.15.88:0/1369893978 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c009560 con 0x7f7728104590 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.334+0000 7f77277fe640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77280fe5b0 0x7f77281941c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:44.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.334+0000 7f7726ffd640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7728104590 0x7f7728194700 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f771800e970 tx=0x7f771800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:44.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.334+0000 7f7724ff9640 1 -- 172.21.15.88:0/1369893978 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f771800cc90 con 0x7f7728104590 2024-02-19T19:43:44.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.334+0000 7f772e00b640 1 -- 172.21.15.88:0/1369893978 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7728198e40 con 0x7f7728104590 2024-02-19T19:43:44.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.334+0000 7f7724ff9640 1 -- 172.21.15.88:0/1369893978 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f771800cdf0 con 0x7f7728104590 2024-02-19T19:43:44.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.334+0000 7f7724ff9640 1 -- 172.21.15.88:0/1369893978 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7718010430 con 0x7f7728104590 2024-02-19T19:43:44.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.334+0000 7f772e00b640 1 -- 172.21.15.88:0/1369893978 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7728199420 con 0x7f7728104590 2024-02-19T19:43:44.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.335+0000 7f772e00b640 1 -- 172.21.15.88:0/1369893978 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f772804e000 con 0x7f7728104590 2024-02-19T19:43:44.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.335+0000 7f7724ff9640 1 -- 172.21.15.88:0/1369893978 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f77180046c0 con 0x7f7728104590 2024-02-19T19:43:44.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.336+0000 7f7724ff9640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f771006d810 0x7f771006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:44.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.336+0000 7f77277fe640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f771006d810 0x7f771006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:44.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.336+0000 7f7724ff9640 1 -- 172.21.15.88:0/1369893978 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7718014070 con 0x7f7728104590 2024-02-19T19:43:44.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.336+0000 7f77277fe640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f771006d810 0x7f771006fcc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f771c000c00 tx=0x7f771c02cb40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:44.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.341+0000 7f7724ff9640 1 -- 172.21.15.88:0/1369893978 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7718057300 con 0x7f7728104590 2024-02-19T19:43:44.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.560+0000 7f772e00b640 1 -- 172.21.15.88:0/1369893978 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7f7728195510 con 0x7f7728104590 2024-02-19T19:43:44.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.561+0000 7f7724ff9640 1 -- 172.21.15.88:0/1369893978 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v40) v1 ==== 107+0+5009 (secure 0 0 0) 0x7f771805a9c0 con 0x7f7728104590 2024-02-19T19:43:44.561 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:44.561 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44365,"name":"cephfs.smithi132.afhgdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6825/594928725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6824","nonce":594928725},{"type":"v1","addr":"172.21.15.132:6825","nonce":594928725}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":44753,"name":"cephfs.smithi132.uyjxaz","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.132:6827/858155091","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6826","nonce":858155091},{"type":"v1","addr":"172.21.15.132:6827","nonce":858155091}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-19T18:58:32.192123+0000","modified":"2024-02-19T19:41:36.773119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":84,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34430,"mds_1":34462},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34430":{"gid":34430,"name":"cephfs.smithi088.ajgvpz","rank":0,"incarnation":34,"state":"up:active","state_seq":469,"addr":"172.21.15.88:6827/183364349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":183364349},{"type":"v1","addr":"172.21.15.88:6827","nonce":183364349}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_34462":{"gid":34462,"name":"cephfs.smithi088.psjiah","rank":1,"incarnation":34,"state":"up:active","state_seq":464,"addr":"172.21.15.88:6829/830186541","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":830186541},{"type":"v1","addr":"172.21.15.88:6829","nonce":830186541}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-19T19:43:44.561 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 39 2024-02-19T19:43:44.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.563+0000 7f772e00b640 1 -- 172.21.15.88:0/1369893978 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f771006d810 msgr2=0x7f771006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:44.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.563+0000 7f772e00b640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f771006d810 0x7f771006fcc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f771c000c00 tx=0x7f771c02cb40 comp rx=0 tx=0).stop 2024-02-19T19:43:44.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.563+0000 7f772e00b640 1 -- 172.21.15.88:0/1369893978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7728104590 msgr2=0x7f7728194700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:44.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.563+0000 7f772e00b640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7728104590 0x7f7728194700 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f771800e970 tx=0x7f771800ed70 comp rx=0 tx=0).stop 2024-02-19T19:43:44.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.563+0000 7f772e00b640 1 -- 172.21.15.88:0/1369893978 shutdown_connections 2024-02-19T19:43:44.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.563+0000 7f772e00b640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f771006d810 0x7f771006fcc0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.563+0000 7f772e00b640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7728104590 0x7f7728194700 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.563+0000 7f772e00b640 1 --2- 172.21.15.88:0/1369893978 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f77280fe5b0 0x7f77281941c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.563+0000 7f772e00b640 1 -- 172.21.15.88:0/1369893978 >> 172.21.15.88:0/1369893978 conn(0x7f77280743d0 msgr2=0x7f77280fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:44.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.563+0000 7f772e00b640 1 -- 172.21.15.88:0/1369893978 shutdown_connections 2024-02-19T19:43:44.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.563+0000 7f772e00b640 1 -- 172.21.15.88:0/1369893978 wait complete. 2024-02-19T19:43:44.575 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-02-19T19:43:44.586 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-02-19T19:43:44.586 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T19:43:44.586 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-19T19:43:44.603 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T19:43:44.603 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-19T19:43:44.660 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T19:43:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.803+0000 7fd460378640 1 -- 172.21.15.88:0/1017889866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4580ff690 msgr2=0x7fd4580ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.803+0000 7fd460378640 1 --2- 172.21.15.88:0/1017889866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4580ff690 0x7fd4580ffb00 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fd448009880 tx=0x7fd44802bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.803+0000 7fd460378640 1 -- 172.21.15.88:0/1017889866 shutdown_connections 2024-02-19T19:43:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.803+0000 7fd460378640 1 --2- 172.21.15.88:0/1017889866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4580ff690 0x7fd4580ffb00 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.803+0000 7fd460378640 1 --2- 172.21.15.88:0/1017889866 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4581056a0 0x7fd458105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.803+0000 7fd460378640 1 -- 172.21.15.88:0/1017889866 >> 172.21.15.88:0/1017889866 conn(0x7fd4580fb190 msgr2=0x7fd4580fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.803+0000 7fd460378640 1 -- 172.21.15.88:0/1017889866 shutdown_connections 2024-02-19T19:43:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.803+0000 7fd460378640 1 -- 172.21.15.88:0/1017889866 wait complete. 2024-02-19T19:43:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd460378640 1 Processor -- start 2024-02-19T19:43:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd460378640 1 -- start start 2024-02-19T19:43:44.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd460378640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4580ff690 0x7fd4581951d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd460378640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4581056a0 0x7fd458195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd460378640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd458195da0 con 0x7fd4581056a0 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd460378640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd458199b70 con 0x7fd4580ff690 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd45d913640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4581056a0 0x7fd458195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd45d913640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4581056a0 0x7fd458195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58044/0 (socket says 172.21.15.88:58044) 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd45d913640 1 -- 172.21.15.88:0/864112801 learned_addr learned my addr 172.21.15.88:0/864112801 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd45d913640 1 -- 172.21.15.88:0/864112801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4580ff690 msgr2=0x7fd4581951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd45e114640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4580ff690 0x7fd4581951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd45d913640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4580ff690 0x7fd4581951d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd45d913640 1 -- 172.21.15.88:0/864112801 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd448009560 con 0x7fd4581056a0 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.804+0000 7fd45e114640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4580ff690 0x7fd4581951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.805+0000 7fd45d913640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4581056a0 0x7fd458195710 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd448009230 tx=0x7fd448004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.805+0000 7fd44f7fe640 1 -- 172.21.15.88:0/864112801 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4480035f0 con 0x7fd4581056a0 2024-02-19T19:43:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.805+0000 7fd460378640 1 -- 172.21.15.88:0/864112801 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd458199df0 con 0x7fd4581056a0 2024-02-19T19:43:44.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.805+0000 7fd44f7fe640 1 -- 172.21.15.88:0/864112801 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd448003c10 con 0x7fd4581056a0 2024-02-19T19:43:44.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.805+0000 7fd44f7fe640 1 -- 172.21.15.88:0/864112801 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd448005670 con 0x7fd4581056a0 2024-02-19T19:43:44.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.805+0000 7fd460378640 1 -- 172.21.15.88:0/864112801 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45819a370 con 0x7fd4581056a0 2024-02-19T19:43:44.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.806+0000 7fd460378640 1 -- 172.21.15.88:0/864112801 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45804e000 con 0x7fd4581056a0 2024-02-19T19:43:44.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.806+0000 7fd44f7fe640 1 -- 172.21.15.88:0/864112801 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd448004dd0 con 0x7fd4581056a0 2024-02-19T19:43:44.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.807+0000 7fd44f7fe640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd44406d870 0x7fd44406fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:44.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.807+0000 7fd44f7fe640 1 -- 172.21.15.88:0/864112801 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd4480af910 con 0x7fd4581056a0 2024-02-19T19:43:44.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.807+0000 7fd45e114640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd44406d870 0x7fd44406fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:44.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.807+0000 7fd45e114640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd44406d870 0x7fd44406fd20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd4581007d0 tx=0x7fd454005e70 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.811+0000 7fd44f7fe640 1 -- 172.21.15.88:0/864112801 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd4480790c0 con 0x7fd4581056a0 2024-02-19T19:43:44.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.994+0000 7fd460378640 1 -- 172.21.15.88:0/864112801 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4580008d0 con 0x7fd4581056a0 2024-02-19T19:43:44.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.995+0000 7fd44f7fe640 1 -- 172.21.15.88:0/864112801 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16777 (secure 0 0 0) 0x7fd44807c780 con 0x7fd4581056a0 2024-02-19T19:43:44.995 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:44.995 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":88,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","created":"2024-02-19T18:50:27.026977+0000","modified":"2024-02-19T19:26:18.541223+0000","last_up_change":"2024-02-19T19:10:25.399688+0000","last_in_change":"2024-02-19T18:56:55.518417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T18:55:57.936009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-19T18:58:31.129412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-19T18:58:31.420377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"787c5fec-c403-4975-92c3-8b68fcf9e399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":81,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6803","nonce":4275403566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6805","nonce":4275403566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6809","nonce":4275403566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6807","nonce":4275403566}]},"public_addr":"172.21.15.88:6803/4275403566","cluster_addr":"172.21.15.88:6805/4275403566","heartbeat_back_addr":"172.21.15.88:6809/4275403566","heartbeat_front_addr":"172.21.15.88:6807/4275403566","state":["exists","up"]},{"osd":1,"uuid":"a459843c-43d7-480f-a76b-d52d6e8ca061","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":57,"up_from":61,"up_thru":81,"down_at":58,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6811","nonce":1931660446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6813","nonce":1931660446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6817","nonce":1931660446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6815","nonce":1931660446}]},"public_addr":"172.21.15.88:6811/1931660446","cluster_addr":"172.21.15.88:6813/1931660446","heartbeat_back_addr":"172.21.15.88:6817/1931660446","heartbeat_front_addr":"172.21.15.88:6815/1931660446","state":["exists","up"]},{"osd":2,"uuid":"66334f86-7411-4587-b96b-22b61bf5201a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":62,"up_from":66,"up_thru":81,"down_at":63,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6819","nonce":2845929374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6821","nonce":2845929374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6825","nonce":2845929374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6823","nonce":2845929374}]},"public_addr":"172.21.15.88:6819/2845929374","cluster_addr":"172.21.15.88:6821/2845929374","heartbeat_back_addr":"172.21.15.88:6825/2845929374","heartbeat_front_addr":"172.21.15.88:6823/2845929374","state":["exists","up"]},{"osd":3,"uuid":"733318f2-be90-44d7-8c38-fb773f7da169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":67,"up_from":71,"up_thru":81,"down_at":68,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6801","nonce":2930324400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6803","nonce":2930324400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6807","nonce":2930324400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6805","nonce":2930324400}]},"public_addr":"172.21.15.132:6801/2930324400","cluster_addr":"172.21.15.132:6803/2930324400","heartbeat_back_addr":"172.21.15.132:6807/2930324400","heartbeat_front_addr":"172.21.15.132:6805/2930324400","state":["exists","up"]},{"osd":4,"uuid":"3b41c4ff-b4b1-4279-b973-0c3c1c62373e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":72,"up_from":76,"up_thru":81,"down_at":73,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6809","nonce":682905746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6811","nonce":682905746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6815","nonce":682905746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6813","nonce":682905746}]},"public_addr":"172.21.15.132:6809/682905746","cluster_addr":"172.21.15.132:6811/682905746","heartbeat_back_addr":"172.21.15.132:6815/682905746","heartbeat_front_addr":"172.21.15.132:6813/682905746","state":["exists","up"]},{"osd":5,"uuid":"367bd67a-e45d-40ff-9aac-1f88755641ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":77,"up_from":81,"up_thru":81,"down_at":78,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6817","nonce":1555619083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6819","nonce":1555619083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6823","nonce":1555619083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6821","nonce":1555619083}]},"public_addr":"172.21.15.132:6817/1555619083","cluster_addr":"172.21.15.132:6819/1555619083","heartbeat_back_addr":"172.21.15.132:6823/1555619083","heartbeat_front_addr":"172.21.15.132:6821/1555619083","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:54:58.653733+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:26.141001+0000","dead_epoch":57},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:55.352595+0000","dead_epoch":62},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:19.994273+0000","dead_epoch":67},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:46.399893+0000","dead_epoch":72},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:57:13.114456+0000","dead_epoch":77}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6829/2325623973":"2024-02-20T19:10:28.265472+0000","172.21.15.88:6828/2325623973":"2024-02-20T19:10:28.265472+0000","172.21.15.132:6825/2841430379":"2024-02-20T19:10:28.265488+0000","172.21.15.88:0/1344352507":"2024-02-20T19:04:58.075334+0000","172.21.15.88:0/2177498492":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6801/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6801/221244694":"2024-02-20T19:04:58.075334+0000","172.21.15.88:6801/2764007082":"2024-02-20T19:00:47.495177+0000","172.21.15.132:6829/3141443905":"2024-02-20T19:01:32.237246+0000","172.21.15.88:0/3237698093":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6800/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6826/694475098":"2024-02-20T18:58:49.196048+0000","172.21.15.88:0/2389966380":"2024-02-20T19:00:47.495177+0000","172.21.15.132:0/2573781289":"2024-02-20T19:01:32.237246+0000","172.21.15.132:6828/3141443905":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6800/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/3497052224":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/3565328320":"2024-02-20T18:51:37.844957+0000","172.21.15.132:6824/2841430379":"2024-02-20T19:10:28.265488+0000","172.21.15.88:6801/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.132:0/2202935815":"2024-02-20T19:01:32.237246+0000","172.21.15.88:0/387402116":"2024-02-20T18:51:05.484896+0000","172.21.15.88:0/2372050564":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/2300938361":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/2164615684":"2024-02-20T19:04:58.075334+0000","172.21.15.132:0/2425517471":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6800/2764007082":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/3838380819":"2024-02-20T18:51:05.484896+0000","172.21.15.132:0/1356863923":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6801/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6827/694475098":"2024-02-20T18:58:49.196048+0000","172.21.15.132:0/3601890998":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6800/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.132:0/2131958733":"2024-02-20T19:01:32.237246+0000","172.21.15.88:0/2418191043":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/4069571809":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/3491684321":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/2338689461":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6800/221244694":"2024-02-20T19:04:58.075334+0000","172.21.15.88:0/4002678054":"2024-02-20T19:04:58.075334+0000","172.21.15.88:0/3163483263":"2024-02-20T19:04:58.075334+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T19:43:44.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.997+0000 7fd460378640 1 -- 172.21.15.88:0/864112801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd44406d870 msgr2=0x7fd44406fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:44.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.997+0000 7fd460378640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd44406d870 0x7fd44406fd20 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fd4581007d0 tx=0x7fd454005e70 comp rx=0 tx=0).stop 2024-02-19T19:43:44.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.997+0000 7fd460378640 1 -- 172.21.15.88:0/864112801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4581056a0 msgr2=0x7fd458195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:44.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.997+0000 7fd460378640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4581056a0 0x7fd458195710 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd448009230 tx=0x7fd448004040 comp rx=0 tx=0).stop 2024-02-19T19:43:44.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.997+0000 7fd460378640 1 -- 172.21.15.88:0/864112801 shutdown_connections 2024-02-19T19:43:44.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.997+0000 7fd460378640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd44406d870 0x7fd44406fd20 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.997+0000 7fd460378640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4581056a0 0x7fd458195710 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.997+0000 7fd460378640 1 --2- 172.21.15.88:0/864112801 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd4580ff690 0x7fd4581951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:44.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.997+0000 7fd460378640 1 -- 172.21.15.88:0/864112801 >> 172.21.15.88:0/864112801 conn(0x7fd4580fb190 msgr2=0x7fd4580fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:44.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.997+0000 7fd460378640 1 -- 172.21.15.88:0/864112801 shutdown_connections 2024-02-19T19:43:44.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:44.997+0000 7fd460378640 1 -- 172.21.15.88:0/864112801 wait complete. 2024-02-19T19:43:45.008 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T18:55:57.936009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-19T18:58:31.129412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-19T18:58:31.420377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-19T19:43:45.008 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T19:43:45.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.112+0000 7feef50d5640 1 -- 172.21.15.88:0/2061216292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef0104590 msgr2=0x7feef0104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.112+0000 7feef50d5640 1 --2- 172.21.15.88:0/2061216292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef0104590 0x7feef0104960 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7feee0009880 tx=0x7feee002bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.112+0000 7feef50d5640 1 -- 172.21.15.88:0/2061216292 shutdown_connections 2024-02-19T19:43:45.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.112+0000 7feef50d5640 1 --2- 172.21.15.88:0/2061216292 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef00fe5b0 0x7feef00fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.112+0000 7feef50d5640 1 --2- 172.21.15.88:0/2061216292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef0104590 0x7feef0104960 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.112+0000 7feef50d5640 1 -- 172.21.15.88:0/2061216292 >> 172.21.15.88:0/2061216292 conn(0x7feef00743d0 msgr2=0x7feef00747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:45.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.112+0000 7feef50d5640 1 -- 172.21.15.88:0/2061216292 shutdown_connections 2024-02-19T19:43:45.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.112+0000 7feef50d5640 1 -- 172.21.15.88:0/2061216292 wait complete. 2024-02-19T19:43:45.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feef50d5640 1 Processor -- start 2024-02-19T19:43:45.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feef50d5640 1 -- start start 2024-02-19T19:43:45.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feef50d5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef00fe5b0 0x7feef01960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:45.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feef50d5640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef0104590 0x7feef01965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:45.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feef50d5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef0196cc0 con 0x7feef00fe5b0 2024-02-19T19:43:45.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feef50d5640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef0196e00 con 0x7feef0104590 2024-02-19T19:43:45.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feeeed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef00fe5b0 0x7feef01960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:45.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feeeed9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef00fe5b0 0x7feef01960a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58062/0 (socket says 172.21.15.88:58062) 2024-02-19T19:43:45.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feeeed9d640 1 -- 172.21.15.88:0/351104396 learned_addr learned my addr 172.21.15.88:0/351104396 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feeee59c640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef0104590 0x7feef01965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feeeed9d640 1 -- 172.21.15.88:0/351104396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef0104590 msgr2=0x7feef01965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feeeed9d640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef0104590 0x7feef01965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feeeed9d640 1 -- 172.21.15.88:0/351104396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee0009560 con 0x7feef00fe5b0 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.113+0000 7feeee59c640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef0104590 0x7feef01965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.114+0000 7feeeed9d640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef00fe5b0 0x7feef01960a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7feee0009230 tx=0x7feee0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.114+0000 7feed7fff640 1 -- 172.21.15.88:0/351104396 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee00035f0 con 0x7feef00fe5b0 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.114+0000 7feed7fff640 1 -- 172.21.15.88:0/351104396 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feee0003c10 con 0x7feef00fe5b0 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.114+0000 7feed7fff640 1 -- 172.21.15.88:0/351104396 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feee0005670 con 0x7feef00fe5b0 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.114+0000 7feef50d5640 1 -- 172.21.15.88:0/351104396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef019f110 con 0x7feef00fe5b0 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.114+0000 7feef50d5640 1 -- 172.21.15.88:0/351104396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef019f5e0 con 0x7feef00fe5b0 2024-02-19T19:43:45.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.115+0000 7feef50d5640 1 -- 172.21.15.88:0/351104396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef004e000 con 0x7feef00fe5b0 2024-02-19T19:43:45.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.116+0000 7feed7fff640 1 -- 172.21.15.88:0/351104396 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7feee0004dd0 con 0x7feef00fe5b0 2024-02-19T19:43:45.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.116+0000 7feed7fff640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feed806d7c0 0x7feed806fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:45.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.116+0000 7feed7fff640 1 -- 172.21.15.88:0/351104396 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7feee00af910 con 0x7feef00fe5b0 2024-02-19T19:43:45.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.116+0000 7feeee59c640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feed806d7c0 0x7feed806fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:45.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.117+0000 7feeee59c640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feed806d7c0 0x7feed806fc70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feef0067fa0 tx=0x7feee400b5f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:45.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.121+0000 7feed7fff640 1 -- 172.21.15.88:0/351104396 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feee00790c0 con 0x7feef00fe5b0 2024-02-19T19:43:45.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:44 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3061030339' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-02-19T19:43:45.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:44 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/967213980' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-02-19T19:43:45.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:44 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1369893978' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-02-19T19:43:45.295 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:44 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3061030339' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-02-19T19:43:45.295 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:44 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/967213980' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-02-19T19:43:45.295 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:44 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1369893978' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-02-19T19:43:45.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.296+0000 7feef50d5640 1 -- 172.21.15.88:0/351104396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feef010b970 con 0x7feef00fe5b0 2024-02-19T19:43:45.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.297+0000 7feed7fff640 1 -- 172.21.15.88:0/351104396 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7feee007c780 con 0x7feef00fe5b0 2024-02-19T19:43:45.297 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-02-19T19:43:45.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.299+0000 7feef50d5640 1 -- 172.21.15.88:0/351104396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feed806d7c0 msgr2=0x7feed806fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.299+0000 7feef50d5640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feed806d7c0 0x7feed806fc70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7feef0067fa0 tx=0x7feee400b5f0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.299+0000 7feef50d5640 1 -- 172.21.15.88:0/351104396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef00fe5b0 msgr2=0x7feef01960a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.299+0000 7feef50d5640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef00fe5b0 0x7feef01960a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7feee0009230 tx=0x7feee0004040 comp rx=0 tx=0).stop 2024-02-19T19:43:45.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.299+0000 7feef50d5640 1 -- 172.21.15.88:0/351104396 shutdown_connections 2024-02-19T19:43:45.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.299+0000 7feef50d5640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7feed806d7c0 0x7feed806fc70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.299+0000 7feef50d5640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7feef0104590 0x7feef01965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.299+0000 7feef50d5640 1 --2- 172.21.15.88:0/351104396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feef00fe5b0 0x7feef01960a0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.299+0000 7feef50d5640 1 -- 172.21.15.88:0/351104396 >> 172.21.15.88:0/351104396 conn(0x7feef00743d0 msgr2=0x7feef00fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:45.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.300+0000 7feef50d5640 1 -- 172.21.15.88:0/351104396 shutdown_connections 2024-02-19T19:43:45.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.300+0000 7feef50d5640 1 -- 172.21.15.88:0/351104396 wait complete. 2024-02-19T19:43:45.312 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-19T19:43:45.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.418+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1263267731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f48105690 msgr2=0x7f7f48105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.418+0000 7f7f4dc6a640 1 --2- 172.21.15.88:0/1263267731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f48105690 0x7f7f48105a60 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009880 tx=0x7f7f3802bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.418+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1263267731 shutdown_connections 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.418+0000 7f7f4dc6a640 1 --2- 172.21.15.88:0/1263267731 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f48067350 0x7f7f480677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.418+0000 7f7f4dc6a640 1 --2- 172.21.15.88:0/1263267731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f48105690 0x7f7f48105a60 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.418+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1263267731 >> 172.21.15.88:0/1263267731 conn(0x7f7f480743d0 msgr2=0x7f7f480747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.418+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1263267731 shutdown_connections 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.418+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1263267731 wait complete. 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.419+0000 7f7f4dc6a640 1 Processor -- start 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.419+0000 7f7f4dc6a640 1 -- start start 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.419+0000 7f7f4dc6a640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f48067350 0x7f7f481952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.419+0000 7f7f4dc6a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f48105690 0x7f7f48195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.419+0000 7f7f4dc6a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f48195e90 con 0x7f7f48105690 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.419+0000 7f7f4dc6a640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f48199c60 con 0x7f7f48067350 2024-02-19T19:43:45.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.419+0000 7f7f46ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f48105690 0x7f7f48195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.419+0000 7f7f46ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f48105690 0x7f7f48195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51524/0 (socket says 172.21.15.88:51524) 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.420+0000 7f7f46ffd640 1 -- 172.21.15.88:0/1265265199 learned_addr learned my addr 172.21.15.88:0/1265265199 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.420+0000 7f7f477fe640 1 --2- 172.21.15.88:0/1265265199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f48067350 0x7f7f481952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.420+0000 7f7f46ffd640 1 -- 172.21.15.88:0/1265265199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f48067350 msgr2=0x7f7f481952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.420+0000 7f7f46ffd640 1 --2- 172.21.15.88:0/1265265199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f48067350 0x7f7f481952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.420+0000 7f7f46ffd640 1 -- 172.21.15.88:0/1265265199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f38009560 con 0x7f7f48105690 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.420+0000 7f7f46ffd640 1 --2- 172.21.15.88:0/1265265199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f48105690 0x7f7f48195800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00d6f0 tx=0x7f7f3c00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.420+0000 7f7f44ff9640 1 -- 172.21.15.88:0/1265265199 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f3c009a30 con 0x7f7f48105690 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.420+0000 7f7f44ff9640 1 -- 172.21.15.88:0/1265265199 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f3c0054e0 con 0x7f7f48105690 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.420+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1265265199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f48199f40 con 0x7f7f48105690 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.420+0000 7f7f44ff9640 1 -- 172.21.15.88:0/1265265199 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f3c010460 con 0x7f7f48105690 2024-02-19T19:43:45.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.420+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1265265199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4819a520 con 0x7f7f48105690 2024-02-19T19:43:45.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.421+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1265265199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f4804e000 con 0x7f7f48105690 2024-02-19T19:43:45.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.421+0000 7f7f44ff9640 1 -- 172.21.15.88:0/1265265199 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f7f3c009b90 con 0x7f7f48105690 2024-02-19T19:43:45.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.422+0000 7f7f44ff9640 1 --2- 172.21.15.88:0/1265265199 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f3006d810 0x7f7f3006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:45.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.422+0000 7f7f44ff9640 1 -- 172.21.15.88:0/1265265199 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7f3c08d0f0 con 0x7f7f48105690 2024-02-19T19:43:45.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.422+0000 7f7f477fe640 1 --2- 172.21.15.88:0/1265265199 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f3006d810 0x7f7f3006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:45.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.422+0000 7f7f477fe640 1 --2- 172.21.15.88:0/1265265199 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f3006d810 0x7f7f3006fcc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7f38006010 tx=0x7f7f38000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:45.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.427+0000 7f7f44ff9640 1 -- 172.21.15.88:0/1265265199 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7f3c0568a0 con 0x7f7f48105690 2024-02-19T19:43:45.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.601+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1265265199 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f7f48196610 con 0x7f7f48105690 2024-02-19T19:43:45.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.602+0000 7f7f44ff9640 1 -- 172.21.15.88:0/1265265199 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v88) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7f3c056020 con 0x7f7f48105690 2024-02-19T19:43:45.602 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-02-19T19:43:45.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.604+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1265265199 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f3006d810 msgr2=0x7f7f3006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.604+0000 7f7f4dc6a640 1 --2- 172.21.15.88:0/1265265199 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f3006d810 0x7f7f3006fcc0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7f38006010 tx=0x7f7f38000bc0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.604+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1265265199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f48105690 msgr2=0x7f7f48195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.604+0000 7f7f4dc6a640 1 --2- 172.21.15.88:0/1265265199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f48105690 0x7f7f48195800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c00d6f0 tx=0x7f7f3c00dbb0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.604+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1265265199 shutdown_connections 2024-02-19T19:43:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.604+0000 7f7f4dc6a640 1 --2- 172.21.15.88:0/1265265199 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f7f3006d810 0x7f7f3006fcc0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.604+0000 7f7f4dc6a640 1 --2- 172.21.15.88:0/1265265199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f48105690 0x7f7f48195800 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.604+0000 7f7f4dc6a640 1 --2- 172.21.15.88:0/1265265199 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f7f48067350 0x7f7f481952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.604+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1265265199 >> 172.21.15.88:0/1265265199 conn(0x7f7f480743d0 msgr2=0x7f7f480fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.604+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1265265199 shutdown_connections 2024-02-19T19:43:45.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.604+0000 7f7f4dc6a640 1 -- 172.21.15.88:0/1265265199 wait complete. 2024-02-19T19:43:45.615 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-19T19:43:45.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.719+0000 7fcf99718640 1 -- 172.21.15.88:0/1266688118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf94104590 msgr2=0x7fcf94104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.719+0000 7fcf99718640 1 --2- 172.21.15.88:0/1266688118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf94104590 0x7fcf94104960 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009880 tx=0x7fcf8402bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.719+0000 7fcf99718640 1 -- 172.21.15.88:0/1266688118 shutdown_connections 2024-02-19T19:43:45.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.719+0000 7fcf99718640 1 --2- 172.21.15.88:0/1266688118 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf940fe5b0 0x7fcf940fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.719+0000 7fcf99718640 1 --2- 172.21.15.88:0/1266688118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf94104590 0x7fcf94104960 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.719+0000 7fcf99718640 1 -- 172.21.15.88:0/1266688118 >> 172.21.15.88:0/1266688118 conn(0x7fcf940743d0 msgr2=0x7fcf940747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.719+0000 7fcf99718640 1 -- 172.21.15.88:0/1266688118 shutdown_connections 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.719+0000 7fcf99718640 1 -- 172.21.15.88:0/1266688118 wait complete. 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf99718640 1 Processor -- start 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf99718640 1 -- start start 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf99718640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf940fe5b0 0x7fcf941960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf99718640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf94104590 0x7fcf94196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf99718640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94196d00 con 0x7fcf94104590 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf99718640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf94196e70 con 0x7fcf940fe5b0 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf8bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf94104590 0x7fcf94196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf8bfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf94104590 0x7fcf94196620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51546/0 (socket says 172.21.15.88:51546) 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf8bfff640 1 -- 172.21.15.88:0/3860239423 learned_addr learned my addr 172.21.15.88:0/3860239423 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf92ffd640 1 --2- 172.21.15.88:0/3860239423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf940fe5b0 0x7fcf941960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf8bfff640 1 -- 172.21.15.88:0/3860239423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf940fe5b0 msgr2=0x7fcf941960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf8bfff640 1 --2- 172.21.15.88:0/3860239423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf940fe5b0 0x7fcf941960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf8bfff640 1 -- 172.21.15.88:0/3860239423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf84009560 con 0x7fcf94104590 2024-02-19T19:43:45.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf8bfff640 1 --2- 172.21.15.88:0/3860239423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf94104590 0x7fcf94196620 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00c810 tx=0x7fcf7c00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:45.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf90ff9640 1 -- 172.21.15.88:0/3860239423 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf7c0041f0 con 0x7fcf94104590 2024-02-19T19:43:45.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf90ff9640 1 -- 172.21.15.88:0/3860239423 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf7c007e70 con 0x7fcf94104590 2024-02-19T19:43:45.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf99718640 1 -- 172.21.15.88:0/3860239423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9419f150 con 0x7fcf94104590 2024-02-19T19:43:45.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.720+0000 7fcf90ff9640 1 -- 172.21.15.88:0/3860239423 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf7c0049d0 con 0x7fcf94104590 2024-02-19T19:43:45.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.721+0000 7fcf99718640 1 -- 172.21.15.88:0/3860239423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9419f6b0 con 0x7fcf94104590 2024-02-19T19:43:45.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.721+0000 7fcf99718640 1 -- 172.21.15.88:0/3860239423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9404e000 con 0x7fcf94104590 2024-02-19T19:43:45.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.722+0000 7fcf90ff9640 1 -- 172.21.15.88:0/3860239423 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fcf7c0079b0 con 0x7fcf94104590 2024-02-19T19:43:45.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.722+0000 7fcf90ff9640 1 --2- 172.21.15.88:0/3860239423 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcf8006d810 0x7fcf8006fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:45.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.722+0000 7fcf90ff9640 1 -- 172.21.15.88:0/3860239423 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcf7c08d3b0 con 0x7fcf94104590 2024-02-19T19:43:45.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.722+0000 7fcf92ffd640 1 --2- 172.21.15.88:0/3860239423 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcf8006d810 0x7fcf8006fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:45.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.723+0000 7fcf92ffd640 1 --2- 172.21.15.88:0/3860239423 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcf8006d810 0x7fcf8006fcc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcf84006010 tx=0x7fcf84000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:45.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.727+0000 7fcf90ff9640 1 -- 172.21.15.88:0/3860239423 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fcf7c056b60 con 0x7fcf94104590 2024-02-19T19:43:45.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.899+0000 7fcf99718640 1 -- 172.21.15.88:0/3860239423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fcf94067920 con 0x7fcf94104590 2024-02-19T19:43:45.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.899+0000 7fcf90ff9640 1 -- 172.21.15.88:0/3860239423 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v88) v1 ==== 107+0+11 (secure 0 0 0) 0x7fcf7c05a220 con 0x7fcf94104590 2024-02-19T19:43:45.900 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-02-19T19:43:45.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.901+0000 7fcf99718640 1 -- 172.21.15.88:0/3860239423 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcf8006d810 msgr2=0x7fcf8006fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.901+0000 7fcf99718640 1 --2- 172.21.15.88:0/3860239423 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcf8006d810 0x7fcf8006fcc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fcf84006010 tx=0x7fcf84000bc0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.901+0000 7fcf99718640 1 -- 172.21.15.88:0/3860239423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf94104590 msgr2=0x7fcf94196620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:45.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.901+0000 7fcf99718640 1 --2- 172.21.15.88:0/3860239423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf94104590 0x7fcf94196620 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c00c810 tx=0x7fcf7c00ccd0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.901+0000 7fcf99718640 1 -- 172.21.15.88:0/3860239423 shutdown_connections 2024-02-19T19:43:45.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.901+0000 7fcf99718640 1 --2- 172.21.15.88:0/3860239423 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fcf8006d810 0x7fcf8006fcc0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.901+0000 7fcf99718640 1 --2- 172.21.15.88:0/3860239423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf94104590 0x7fcf94196620 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.901+0000 7fcf99718640 1 --2- 172.21.15.88:0/3860239423 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fcf940fe5b0 0x7fcf941960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:45.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.901+0000 7fcf99718640 1 -- 172.21.15.88:0/3860239423 >> 172.21.15.88:0/3860239423 conn(0x7fcf940743d0 msgr2=0x7fcf940fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:45.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.902+0000 7fcf99718640 1 -- 172.21.15.88:0/3860239423 shutdown_connections 2024-02-19T19:43:45.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:45.902+0000 7fcf99718640 1 -- 172.21.15.88:0/3860239423 wait complete. 2024-02-19T19:43:45.911 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:45 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/864112801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T19:43:45.911 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:45 smithi088 ceph-mon[88643]: pgmap v1208: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:45.911 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:45 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/351104396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T19:43:45.911 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:45 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1265265199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-19T19:43:45.913 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-19T19:43:46.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.014+0000 7fbd4b877640 1 -- 172.21.15.88:0/3080596244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd441047d0 msgr2=0x7fbd44104ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.014+0000 7fbd4b877640 1 --2- 172.21.15.88:0/3080596244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd441047d0 0x7fbd44104ba0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009880 tx=0x7fbd3c02bcf0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.014+0000 7fbd4b877640 1 -- 172.21.15.88:0/3080596244 shutdown_connections 2024-02-19T19:43:46.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.014+0000 7fbd4b877640 1 --2- 172.21.15.88:0/3080596244 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440fe7c0 0x7fbd440fec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.014+0000 7fbd4b877640 1 --2- 172.21.15.88:0/3080596244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd441047d0 0x7fbd44104ba0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.014+0000 7fbd4b877640 1 -- 172.21.15.88:0/3080596244 >> 172.21.15.88:0/3080596244 conn(0x7fbd440fa2a0 msgr2=0x7fbd440fc6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.014+0000 7fbd4b877640 1 -- 172.21.15.88:0/3080596244 shutdown_connections 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.014+0000 7fbd4b877640 1 -- 172.21.15.88:0/3080596244 wait complete. 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd4b877640 1 Processor -- start 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd4b877640 1 -- start start 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd4b877640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440fe7c0 0x7fbd44196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd4b877640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd441047d0 0x7fbd44196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd4b877640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd44196f20 con 0x7fbd441047d0 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd4b877640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd44197090 con 0x7fbd440fe7c0 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd48e12640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd441047d0 0x7fbd44196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd48e12640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd441047d0 0x7fbd44196950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51560/0 (socket says 172.21.15.88:51560) 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd48e12640 1 -- 172.21.15.88:0/2552865536 learned_addr learned my addr 172.21.15.88:0/2552865536 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd49613640 1 --2- 172.21.15.88:0/2552865536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440fe7c0 0x7fbd44196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:46.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd48e12640 1 -- 172.21.15.88:0/2552865536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440fe7c0 msgr2=0x7fbd44196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.015+0000 7fbd48e12640 1 --2- 172.21.15.88:0/2552865536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440fe7c0 0x7fbd44196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.016+0000 7fbd48e12640 1 -- 172.21.15.88:0/2552865536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3c009560 con 0x7fbd441047d0 2024-02-19T19:43:46.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.016+0000 7fbd48e12640 1 --2- 172.21.15.88:0/2552865536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd441047d0 0x7fbd44196950 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800b420 tx=0x7fbd3800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:46.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.016+0000 7fbd427fc640 1 -- 172.21.15.88:0/2552865536 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd38011800 con 0x7fbd441047d0 2024-02-19T19:43:46.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.016+0000 7fbd4b877640 1 -- 172.21.15.88:0/2552865536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4419f480 con 0x7fbd441047d0 2024-02-19T19:43:46.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.016+0000 7fbd427fc640 1 -- 172.21.15.88:0/2552865536 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd38011e20 con 0x7fbd441047d0 2024-02-19T19:43:46.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.016+0000 7fbd427fc640 1 -- 172.21.15.88:0/2552865536 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd3800f2e0 con 0x7fbd441047d0 2024-02-19T19:43:46.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.016+0000 7fbd4b877640 1 -- 172.21.15.88:0/2552865536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4419f980 con 0x7fbd441047d0 2024-02-19T19:43:46.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.017+0000 7fbd4b877640 1 -- 172.21.15.88:0/2552865536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4404e000 con 0x7fbd441047d0 2024-02-19T19:43:46.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.017+0000 7fbd427fc640 1 -- 172.21.15.88:0/2552865536 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbd38010380 con 0x7fbd441047d0 2024-02-19T19:43:46.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.018+0000 7fbd427fc640 1 --2- 172.21.15.88:0/2552865536 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd2c06d8d0 0x7fbd2c06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.018+0000 7fbd427fc640 1 -- 172.21.15.88:0/2552865536 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbd3808cec0 con 0x7fbd441047d0 2024-02-19T19:43:46.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.018+0000 7fbd49613640 1 --2- 172.21.15.88:0/2552865536 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd2c06d8d0 0x7fbd2c06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:46.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.018+0000 7fbd49613640 1 --2- 172.21.15.88:0/2552865536 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd2c06d8d0 0x7fbd2c06fd80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006010 tx=0x7fbd3c000bc0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:46.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.022+0000 7fbd427fc640 1 -- 172.21.15.88:0/2552865536 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbd38056670 con 0x7fbd441047d0 2024-02-19T19:43:46.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.207+0000 7fbd4b877640 1 -- 172.21.15.88:0/2552865536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbd44066860 con 0x7fbd441047d0 2024-02-19T19:43:46.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.208+0000 7fbd427fc640 1 -- 172.21.15.88:0/2552865536 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v88) v1 ==== 81+0+2435 (secure 0 0 0) 0x7fbd38059d30 con 0x7fbd441047d0 2024-02-19T19:43:46.208 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6829/2325623973 2024-02-20T19:10:28.265472+0000 2024-02-19T19:43:46.208 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6828/2325623973 2024-02-20T19:10:28.265472+0000 2024-02-19T19:43:46.208 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:6825/2841430379 2024-02-20T19:10:28.265488+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/1344352507 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2177498492 2024-02-20T18:53:39.595841+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6801/681311088 2024-02-20T18:51:05.484896+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6801/221244694 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6801/2764007082 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:6829/3141443905 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3237698093 2024-02-20T18:51:05.484896+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6800/681311088 2024-02-20T18:51:05.484896+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6826/694475098 2024-02-20T18:58:49.196048+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2389966380 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/2573781289 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:6828/3141443905 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6800/3043610921 2024-02-20T18:53:39.595841+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3497052224 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3565328320 2024-02-20T18:51:37.844957+0000 2024-02-19T19:43:46.209 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:6824/2841430379 2024-02-20T19:10:28.265488+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6801/3043610921 2024-02-20T18:53:39.595841+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/2202935815 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/387402116 2024-02-20T18:51:05.484896+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2372050564 2024-02-20T18:53:39.595841+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2300938361 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2164615684 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/2425517471 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6800/2764007082 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3838380819 2024-02-20T18:51:05.484896+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/1356863923 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6801/3414266099 2024-02-20T18:51:37.844957+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6827/694475098 2024-02-20T18:58:49.196048+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/3601890998 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:46.210 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6800/3414266099 2024-02-20T18:51:37.844957+0000 2024-02-19T19:43:46.211 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/2131958733 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:46.211 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2418191043 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:46.211 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/4069571809 2024-02-20T18:53:39.595841+0000 2024-02-19T19:43:46.211 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3491684321 2024-02-20T18:51:37.844957+0000 2024-02-19T19:43:46.211 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2338689461 2024-02-20T18:51:37.844957+0000 2024-02-19T19:43:46.211 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6800/221244694 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:46.211 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/4002678054 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:46.211 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3163483263 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:listed 41 entries 2024-02-19T19:43:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.210+0000 7fbd4b877640 1 -- 172.21.15.88:0/2552865536 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd2c06d8d0 msgr2=0x7fbd2c06fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.210+0000 7fbd4b877640 1 --2- 172.21.15.88:0/2552865536 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd2c06d8d0 0x7fbd2c06fd80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006010 tx=0x7fbd3c000bc0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.210+0000 7fbd4b877640 1 -- 172.21.15.88:0/2552865536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd441047d0 msgr2=0x7fbd44196950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.210+0000 7fbd4b877640 1 --2- 172.21.15.88:0/2552865536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd441047d0 0x7fbd44196950 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800b420 tx=0x7fbd3800b8e0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.210+0000 7fbd4b877640 1 -- 172.21.15.88:0/2552865536 shutdown_connections 2024-02-19T19:43:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.210+0000 7fbd4b877640 1 --2- 172.21.15.88:0/2552865536 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbd2c06d8d0 0x7fbd2c06fd80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.210+0000 7fbd4b877640 1 --2- 172.21.15.88:0/2552865536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd441047d0 0x7fbd44196950 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.210+0000 7fbd4b877640 1 --2- 172.21.15.88:0/2552865536 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbd440fe7c0 0x7fbd44196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.210+0000 7fbd4b877640 1 -- 172.21.15.88:0/2552865536 >> 172.21.15.88:0/2552865536 conn(0x7fbd440fa2a0 msgr2=0x7fbd440fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:46.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.210+0000 7fbd4b877640 1 -- 172.21.15.88:0/2552865536 shutdown_connections 2024-02-19T19:43:46.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.210+0000 7fbd4b877640 1 -- 172.21.15.88:0/2552865536 wait complete. 2024-02-19T19:43:46.221 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-02-19T19:43:46.221 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-19T19:43:46.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:45 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/864112801' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T19:43:46.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:45 smithi132 ceph-mon[70676]: pgmap v1208: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:46.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:45 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/351104396' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T19:43:46.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:45 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1265265199' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-19T19:43:46.237 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-19T19:43:46.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.378+0000 7fbfaafc0640 1 -- 172.21.15.88:0/1146271877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa41045e0 msgr2=0x7fbfa41049b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.378+0000 7fbfaafc0640 1 --2- 172.21.15.88:0/1146271877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa41045e0 0x7fbfa41049b0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009880 tx=0x7fbf9402bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.378+0000 7fbfaafc0640 1 -- 172.21.15.88:0/1146271877 shutdown_connections 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.378+0000 7fbfaafc0640 1 --2- 172.21.15.88:0/1146271877 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa40fe5b0 0x7fbfa40fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.378+0000 7fbfaafc0640 1 --2- 172.21.15.88:0/1146271877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa41045e0 0x7fbfa41049b0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.378+0000 7fbfaafc0640 1 -- 172.21.15.88:0/1146271877 >> 172.21.15.88:0/1146271877 conn(0x7fbfa40fa070 msgr2=0x7fbfa40fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.378+0000 7fbfaafc0640 1 -- 172.21.15.88:0/1146271877 shutdown_connections 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.378+0000 7fbfaafc0640 1 -- 172.21.15.88:0/1146271877 wait complete. 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfaafc0640 1 Processor -- start 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfaafc0640 1 -- start start 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfaafc0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa40fe5b0 0x7fbfa41951e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfaafc0640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa41045e0 0x7fbfa4195720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfaafc0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4195db0 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfaafc0640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa4199b80 con 0x7fbfa41045e0 2024-02-19T19:43:46.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfa8d5c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa40fe5b0 0x7fbfa41951e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfa8d5c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa40fe5b0 0x7fbfa41951e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51578/0 (socket says 172.21.15.88:51578) 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfa8d5c640 1 -- 172.21.15.88:0/3028077711 learned_addr learned my addr 172.21.15.88:0/3028077711 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfa3fff640 1 --2- 172.21.15.88:0/3028077711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa41045e0 0x7fbfa4195720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfa8d5c640 1 -- 172.21.15.88:0/3028077711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa41045e0 msgr2=0x7fbfa4195720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfa8d5c640 1 --2- 172.21.15.88:0/3028077711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa41045e0 0x7fbfa4195720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.379+0000 7fbfa8d5c640 1 -- 172.21.15.88:0/3028077711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf94009560 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.380+0000 7fbfa8d5c640 1 --2- 172.21.15.88:0/3028077711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa40fe5b0 0x7fbfa41951e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009230 tx=0x7fbf94004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.380+0000 7fbfa1ffb640 1 -- 172.21.15.88:0/3028077711 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf940035f0 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.380+0000 7fbfa1ffb640 1 -- 172.21.15.88:0/3028077711 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf94003c10 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.380+0000 7fbfaafc0640 1 -- 172.21.15.88:0/3028077711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa4199e00 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.380+0000 7fbfa1ffb640 1 -- 172.21.15.88:0/3028077711 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf94034440 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.380+0000 7fbfaafc0640 1 -- 172.21.15.88:0/3028077711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa419a380 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.381+0000 7fbfaafc0640 1 -- 172.21.15.88:0/3028077711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa404e000 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.381+0000 7fbfa1ffb640 1 -- 172.21.15.88:0/3028077711 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fbf94004fd0 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.382+0000 7fbfa1ffb640 1 --2- 172.21.15.88:0/3028077711 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf8c06d8d0 0x7fbf8c06fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.382+0000 7fbfa1ffb640 1 -- 172.21.15.88:0/3028077711 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbf940af8f0 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.382+0000 7fbfa3fff640 1 --2- 172.21.15.88:0/3028077711 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf8c06d8d0 0x7fbf8c06fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.382+0000 7fbfa3fff640 1 --2- 172.21.15.88:0/3028077711 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf8c06d8d0 0x7fbf8c06fd80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbf98005fd0 tx=0x7fbf98005e20 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:46.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.387+0000 7fbfa1ffb640 1 -- 172.21.15.88:0/3028077711 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbf94079120 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.562+0000 7fbfaafc0640 1 -- 172.21.15.88:0/3028077711 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbfa410b8b0 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.563+0000 7fbfa1ffb640 1 -- 172.21.15.88:0/3028077711 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16777 (secure 0 0 0) 0x7fbf9407c7e0 con 0x7fbfa40fe5b0 2024-02-19T19:43:46.564 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-02-19T19:43:46.564 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":88,"fsid":"a60b4d3a-cf57-11ee-95bb-87774f69a715","created":"2024-02-19T18:50:27.026977+0000","modified":"2024-02-19T19:26:18.541223+0000","last_up_change":"2024-02-19T19:10:25.399688+0000","last_in_change":"2024-02-19T18:56:55.518417+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-19T18:55:57.936009+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-02-19T18:58:31.129412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-02-19T18:58:31.420377+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"787c5fec-c403-4975-92c3-8b68fcf9e399","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":81,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6803","nonce":4275403566}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6805","nonce":4275403566}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6809","nonce":4275403566}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":4275403566},{"type":"v1","addr":"172.21.15.88:6807","nonce":4275403566}]},"public_addr":"172.21.15.88:6803/4275403566","cluster_addr":"172.21.15.88:6805/4275403566","heartbeat_back_addr":"172.21.15.88:6809/4275403566","heartbeat_front_addr":"172.21.15.88:6807/4275403566","state":["exists","up"]},{"osd":1,"uuid":"a459843c-43d7-480f-a76b-d52d6e8ca061","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":57,"up_from":61,"up_thru":81,"down_at":58,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6811","nonce":1931660446}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6813","nonce":1931660446}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6817","nonce":1931660446}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1931660446},{"type":"v1","addr":"172.21.15.88:6815","nonce":1931660446}]},"public_addr":"172.21.15.88:6811/1931660446","cluster_addr":"172.21.15.88:6813/1931660446","heartbeat_back_addr":"172.21.15.88:6817/1931660446","heartbeat_front_addr":"172.21.15.88:6815/1931660446","state":["exists","up"]},{"osd":2,"uuid":"66334f86-7411-4587-b96b-22b61bf5201a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":62,"up_from":66,"up_thru":81,"down_at":63,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6819","nonce":2845929374}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6821","nonce":2845929374}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6825","nonce":2845929374}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":2845929374},{"type":"v1","addr":"172.21.15.88:6823","nonce":2845929374}]},"public_addr":"172.21.15.88:6819/2845929374","cluster_addr":"172.21.15.88:6821/2845929374","heartbeat_back_addr":"172.21.15.88:6825/2845929374","heartbeat_front_addr":"172.21.15.88:6823/2845929374","state":["exists","up"]},{"osd":3,"uuid":"733318f2-be90-44d7-8c38-fb773f7da169","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":67,"up_from":71,"up_thru":81,"down_at":68,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6800","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6801","nonce":2930324400}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6802","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6803","nonce":2930324400}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6806","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6807","nonce":2930324400}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6804","nonce":2930324400},{"type":"v1","addr":"172.21.15.132:6805","nonce":2930324400}]},"public_addr":"172.21.15.132:6801/2930324400","cluster_addr":"172.21.15.132:6803/2930324400","heartbeat_back_addr":"172.21.15.132:6807/2930324400","heartbeat_front_addr":"172.21.15.132:6805/2930324400","state":["exists","up"]},{"osd":4,"uuid":"3b41c4ff-b4b1-4279-b973-0c3c1c62373e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":72,"up_from":76,"up_thru":81,"down_at":73,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6808","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6809","nonce":682905746}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6810","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6811","nonce":682905746}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6814","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6815","nonce":682905746}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6812","nonce":682905746},{"type":"v1","addr":"172.21.15.132:6813","nonce":682905746}]},"public_addr":"172.21.15.132:6809/682905746","cluster_addr":"172.21.15.132:6811/682905746","heartbeat_back_addr":"172.21.15.132:6815/682905746","heartbeat_front_addr":"172.21.15.132:6813/682905746","state":["exists","up"]},{"osd":5,"uuid":"367bd67a-e45d-40ff-9aac-1f88755641ad","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":77,"up_from":81,"up_thru":81,"down_at":78,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6816","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6817","nonce":1555619083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6818","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6819","nonce":1555619083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6822","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6823","nonce":1555619083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.132:6820","nonce":1555619083},{"type":"v1","addr":"172.21.15.132:6821","nonce":1555619083}]},"public_addr":"172.21.15.132:6817/1555619083","cluster_addr":"172.21.15.132:6819/1555619083","heartbeat_back_addr":"172.21.15.132:6823/1555619083","heartbeat_front_addr":"172.21.15.132:6821/1555619083","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:54:58.653733+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:26.141001+0000","dead_epoch":57},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:55:55.352595+0000","dead_epoch":62},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:19.994273+0000","dead_epoch":67},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:56:46.399893+0000","dead_epoch":72},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-19T18:57:13.114456+0000","dead_epoch":77}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6829/2325623973":"2024-02-20T19:10:28.265472+0000","172.21.15.88:6828/2325623973":"2024-02-20T19:10:28.265472+0000","172.21.15.132:6825/2841430379":"2024-02-20T19:10:28.265488+0000","172.21.15.88:0/1344352507":"2024-02-20T19:04:58.075334+0000","172.21.15.88:0/2177498492":"2024-02-20T18:53:39.595841+0000","172.21.15.88:6801/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6801/221244694":"2024-02-20T19:04:58.075334+0000","172.21.15.88:6801/2764007082":"2024-02-20T19:00:47.495177+0000","172.21.15.132:6829/3141443905":"2024-02-20T19:01:32.237246+0000","172.21.15.88:0/3237698093":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6800/681311088":"2024-02-20T18:51:05.484896+0000","172.21.15.88:6826/694475098":"2024-02-20T18:58:49.196048+0000","172.21.15.88:0/2389966380":"2024-02-20T19:00:47.495177+0000","172.21.15.132:0/2573781289":"2024-02-20T19:01:32.237246+0000","172.21.15.132:6828/3141443905":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6800/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/3497052224":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/3565328320":"2024-02-20T18:51:37.844957+0000","172.21.15.132:6824/2841430379":"2024-02-20T19:10:28.265488+0000","172.21.15.88:6801/3043610921":"2024-02-20T18:53:39.595841+0000","172.21.15.132:0/2202935815":"2024-02-20T19:01:32.237246+0000","172.21.15.88:0/387402116":"2024-02-20T18:51:05.484896+0000","172.21.15.88:0/2372050564":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/2300938361":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/2164615684":"2024-02-20T19:04:58.075334+0000","172.21.15.132:0/2425517471":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6800/2764007082":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/3838380819":"2024-02-20T18:51:05.484896+0000","172.21.15.132:0/1356863923":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6801/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6827/694475098":"2024-02-20T18:58:49.196048+0000","172.21.15.132:0/3601890998":"2024-02-20T19:01:32.237246+0000","172.21.15.88:6800/3414266099":"2024-02-20T18:51:37.844957+0000","172.21.15.132:0/2131958733":"2024-02-20T19:01:32.237246+0000","172.21.15.88:0/2418191043":"2024-02-20T19:00:47.495177+0000","172.21.15.88:0/4069571809":"2024-02-20T18:53:39.595841+0000","172.21.15.88:0/3491684321":"2024-02-20T18:51:37.844957+0000","172.21.15.88:0/2338689461":"2024-02-20T18:51:37.844957+0000","172.21.15.88:6800/221244694":"2024-02-20T19:04:58.075334+0000","172.21.15.88:0/4002678054":"2024-02-20T19:04:58.075334+0000","172.21.15.88:0/3163483263":"2024-02-20T19:04:58.075334+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-19T19:43:46.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.565+0000 7fbfaafc0640 1 -- 172.21.15.88:0/3028077711 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf8c06d8d0 msgr2=0x7fbf8c06fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.565+0000 7fbfaafc0640 1 --2- 172.21.15.88:0/3028077711 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf8c06d8d0 0x7fbf8c06fd80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbf98005fd0 tx=0x7fbf98005e20 comp rx=0 tx=0).stop 2024-02-19T19:43:46.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.565+0000 7fbfaafc0640 1 -- 172.21.15.88:0/3028077711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa40fe5b0 msgr2=0x7fbfa41951e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.565+0000 7fbfaafc0640 1 --2- 172.21.15.88:0/3028077711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa40fe5b0 0x7fbfa41951e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fbf94009230 tx=0x7fbf94004040 comp rx=0 tx=0).stop 2024-02-19T19:43:46.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.565+0000 7fbfaafc0640 1 -- 172.21.15.88:0/3028077711 shutdown_connections 2024-02-19T19:43:46.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.565+0000 7fbfaafc0640 1 --2- 172.21.15.88:0/3028077711 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fbf8c06d8d0 0x7fbf8c06fd80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.565+0000 7fbfaafc0640 1 --2- 172.21.15.88:0/3028077711 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fbfa41045e0 0x7fbfa4195720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.565+0000 7fbfaafc0640 1 --2- 172.21.15.88:0/3028077711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfa40fe5b0 0x7fbfa41951e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.565+0000 7fbfaafc0640 1 -- 172.21.15.88:0/3028077711 >> 172.21.15.88:0/3028077711 conn(0x7fbfa40fa070 msgr2=0x7fbfa40fbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:46.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.565+0000 7fbfaafc0640 1 -- 172.21.15.88:0/3028077711 shutdown_connections 2024-02-19T19:43:46.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.566+0000 7fbfaafc0640 1 -- 172.21.15.88:0/3028077711 wait complete. 2024-02-19T19:43:46.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-19T18:55:57.936009+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-02-19T18:58:31.129412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-02-19T18:58:31.420377+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-19T19:43:46.577 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-19T19:43:46.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.681+0000 7fa3e0b18640 1 -- 172.21.15.88:0/216868570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3dc0fe5b0 msgr2=0x7fa3dc0fea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.681+0000 7fa3e0b18640 1 --2- 172.21.15.88:0/216868570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3dc0fe5b0 0x7fa3dc0fea20 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009c20 tx=0x7fa3d002c1b0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.681+0000 7fa3e0b18640 1 -- 172.21.15.88:0/216868570 shutdown_connections 2024-02-19T19:43:46.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.681+0000 7fa3e0b18640 1 --2- 172.21.15.88:0/216868570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3dc0fe5b0 0x7fa3dc0fea20 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.681+0000 7fa3e0b18640 1 --2- 172.21.15.88:0/216868570 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3dc104590 0x7fa3dc104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.681+0000 7fa3e0b18640 1 -- 172.21.15.88:0/216868570 >> 172.21.15.88:0/216868570 conn(0x7fa3dc0743d0 msgr2=0x7fa3dc0747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:46.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.681+0000 7fa3e0b18640 1 -- 172.21.15.88:0/216868570 shutdown_connections 2024-02-19T19:43:46.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.681+0000 7fa3e0b18640 1 -- 172.21.15.88:0/216868570 wait complete. 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.681+0000 7fa3e0b18640 1 Processor -- start 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.681+0000 7fa3e0b18640 1 -- start start 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3e0b18640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3dc0fe5b0 0x7fa3dc1941b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3e0b18640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3dc104590 0x7fa3dc1946f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3e0b18640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc194d80 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3e0b18640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc198b50 con 0x7fa3dc104590 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3da59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3dc0fe5b0 0x7fa3dc1941b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3da59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3dc0fe5b0 0x7fa3dc1941b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51602/0 (socket says 172.21.15.88:51602) 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3da59c640 1 -- 172.21.15.88:0/328077296 learned_addr learned my addr 172.21.15.88:0/328077296 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3da59c640 1 -- 172.21.15.88:0/328077296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3dc104590 msgr2=0x7fa3dc1946f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3da59c640 1 --2- 172.21.15.88:0/328077296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3dc104590 0x7fa3dc1946f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3da59c640 1 -- 172.21.15.88:0/328077296 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0009900 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3da59c640 1 --2- 172.21.15.88:0/328077296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3dc0fe5b0 0x7fa3dc1941b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400c8f0 tx=0x7fa3d400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:46.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3cb7fe640 1 -- 172.21.15.88:0/328077296 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3d4007b80 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3e0b18640 1 -- 172.21.15.88:0/328077296 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc198e30 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3cb7fe640 1 -- 172.21.15.88:0/328077296 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3d40054e0 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3cb7fe640 1 -- 172.21.15.88:0/328077296 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3d400f450 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.682+0000 7fa3e0b18640 1 -- 172.21.15.88:0/328077296 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc199410 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.683+0000 7fa3e0b18640 1 -- 172.21.15.88:0/328077296 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc04e000 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.684+0000 7fa3cb7fe640 1 -- 172.21.15.88:0/328077296 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fa3d4004f60 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.684+0000 7fa3cb7fe640 1 --2- 172.21.15.88:0/328077296 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3c406d810 0x7fa3c406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.684+0000 7fa3cb7fe640 1 -- 172.21.15.88:0/328077296 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa3d408d350 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.684+0000 7fa3d9d9b640 1 --2- 172.21.15.88:0/328077296 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3c406d810 0x7fa3c406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:46.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.685+0000 7fa3d9d9b640 1 --2- 172.21.15.88:0/328077296 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3c406d810 0x7fa3c406fcc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0006010 tx=0x7fa3d0004a40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:46.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.689+0000 7fa3cb7fe640 1 -- 172.21.15.88:0/328077296 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa3d4056b00 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.861+0000 7fa3e0b18640 1 -- 172.21.15.88:0/328077296 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3dc10b970 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.861+0000 7fa3cb7fe640 1 -- 172.21.15.88:0/328077296 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa3d405a1c0 con 0x7fa3dc0fe5b0 2024-02-19T19:43:46.862 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-02-19T19:43:46.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.863+0000 7fa3e0b18640 1 -- 172.21.15.88:0/328077296 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3c406d810 msgr2=0x7fa3c406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.863+0000 7fa3e0b18640 1 --2- 172.21.15.88:0/328077296 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3c406d810 0x7fa3c406fcc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0006010 tx=0x7fa3d0004a40 comp rx=0 tx=0).stop 2024-02-19T19:43:46.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.863+0000 7fa3e0b18640 1 -- 172.21.15.88:0/328077296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3dc0fe5b0 msgr2=0x7fa3dc1941b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.863+0000 7fa3e0b18640 1 --2- 172.21.15.88:0/328077296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3dc0fe5b0 0x7fa3dc1941b0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400c8f0 tx=0x7fa3d400cdb0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.864+0000 7fa3e0b18640 1 -- 172.21.15.88:0/328077296 shutdown_connections 2024-02-19T19:43:46.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.864+0000 7fa3e0b18640 1 --2- 172.21.15.88:0/328077296 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fa3c406d810 0x7fa3c406fcc0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.864+0000 7fa3e0b18640 1 --2- 172.21.15.88:0/328077296 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fa3dc104590 0x7fa3dc1946f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.864+0000 7fa3e0b18640 1 --2- 172.21.15.88:0/328077296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3dc0fe5b0 0x7fa3dc1941b0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.864+0000 7fa3e0b18640 1 -- 172.21.15.88:0/328077296 >> 172.21.15.88:0/328077296 conn(0x7fa3dc0743d0 msgr2=0x7fa3dc0fcdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.864+0000 7fa3e0b18640 1 -- 172.21.15.88:0/328077296 shutdown_connections 2024-02-19T19:43:46.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.864+0000 7fa3e0b18640 1 -- 172.21.15.88:0/328077296 wait complete. 2024-02-19T19:43:46.875 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.978+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/225744301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9cc104590 msgr2=0x7fe9cc104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.978+0000 7fe9d1f7d640 1 --2- 172.21.15.88:0/225744301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9cc104590 0x7fe9cc104960 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009880 tx=0x7fe9bc02bce0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.978+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/225744301 shutdown_connections 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.978+0000 7fe9d1f7d640 1 --2- 172.21.15.88:0/225744301 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9cc0fe5b0 0x7fe9cc0fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.978+0000 7fe9d1f7d640 1 --2- 172.21.15.88:0/225744301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9cc104590 0x7fe9cc104960 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.978+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/225744301 >> 172.21.15.88:0/225744301 conn(0x7fe9cc0743d0 msgr2=0x7fe9cc0747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.978+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/225744301 shutdown_connections 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.978+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/225744301 wait complete. 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.979+0000 7fe9d1f7d640 1 Processor -- start 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.979+0000 7fe9d1f7d640 1 -- start start 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.979+0000 7fe9d1f7d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9cc0fe5b0 0x7fe9cc196120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.979+0000 7fe9d1f7d640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9cc104590 0x7fe9cc196660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.979+0000 7fe9d1f7d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc196d40 con 0x7fe9cc0fe5b0 2024-02-19T19:43:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.979+0000 7fe9d1f7d640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc196e80 con 0x7fe9cc104590 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.979+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9cc0fe5b0 0x7fe9cc196120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.979+0000 7fe9cb7fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9cc0fe5b0 0x7fe9cc196120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51626/0 (socket says 172.21.15.88:51626) 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.979+0000 7fe9cb7fe640 1 -- 172.21.15.88:0/2303270602 learned_addr learned my addr 172.21.15.88:0/2303270602 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.980+0000 7fe9c2ffd640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9cc104590 0x7fe9cc196660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.980+0000 7fe9cb7fe640 1 -- 172.21.15.88:0/2303270602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9cc104590 msgr2=0x7fe9cc196660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.980+0000 7fe9cb7fe640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9cc104590 0x7fe9cc196660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.980+0000 7fe9cb7fe640 1 -- 172.21.15.88:0/2303270602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc009560 con 0x7fe9cc0fe5b0 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.980+0000 7fe9c2ffd640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9cc104590 0x7fe9cc196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.980+0000 7fe9cb7fe640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9cc0fe5b0 0x7fe9cc196120 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009230 tx=0x7fe9bc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.980+0000 7fe9c97fa640 1 -- 172.21.15.88:0/2303270602 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9bc0035f0 con 0x7fe9cc0fe5b0 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.980+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/2303270602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc19f190 con 0x7fe9cc0fe5b0 2024-02-19T19:43:46.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.980+0000 7fe9c97fa640 1 -- 172.21.15.88:0/2303270602 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe9bc003c10 con 0x7fe9cc0fe5b0 2024-02-19T19:43:46.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.980+0000 7fe9c97fa640 1 -- 172.21.15.88:0/2303270602 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9bc005670 con 0x7fe9cc0fe5b0 2024-02-19T19:43:46.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.980+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/2303270602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc19f610 con 0x7fe9cc0fe5b0 2024-02-19T19:43:46.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.981+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/2303270602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc04e000 con 0x7fe9cc0fe5b0 2024-02-19T19:43:46.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.981+0000 7fe9c97fa640 1 -- 172.21.15.88:0/2303270602 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fe9bc004dd0 con 0x7fe9cc0fe5b0 2024-02-19T19:43:46.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.982+0000 7fe9c97fa640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe9b806d7c0 0x7fe9b806fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:46.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.982+0000 7fe9c97fa640 1 -- 172.21.15.88:0/2303270602 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe9bc0af910 con 0x7fe9cc0fe5b0 2024-02-19T19:43:46.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.982+0000 7fe9c2ffd640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe9b806d7c0 0x7fe9b806fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:46.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.982+0000 7fe9c2ffd640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe9b806d7c0 0x7fe9b806fc70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40058c0 tx=0x7fe9b4005850 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:46.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:46.986+0000 7fe9c97fa640 1 -- 172.21.15.88:0/2303270602 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe9bc0790c0 con 0x7fe9cc0fe5b0 2024-02-19T19:43:47.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.157+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/2303270602 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fe9cc10b970 con 0x7fe9cc0fe5b0 2024-02-19T19:43:47.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.157+0000 7fe9c97fa640 1 -- 172.21.15.88:0/2303270602 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v88) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe9bc07c780 con 0x7fe9cc0fe5b0 2024-02-19T19:43:47.157 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-02-19T19:43:47.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.159+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/2303270602 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe9b806d7c0 msgr2=0x7fe9b806fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:47.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.159+0000 7fe9d1f7d640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe9b806d7c0 0x7fe9b806fc70 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fe9b40058c0 tx=0x7fe9b4005850 comp rx=0 tx=0).stop 2024-02-19T19:43:47.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.159+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/2303270602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9cc0fe5b0 msgr2=0x7fe9cc196120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:47.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.159+0000 7fe9d1f7d640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9cc0fe5b0 0x7fe9cc196120 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009230 tx=0x7fe9bc004040 comp rx=0 tx=0).stop 2024-02-19T19:43:47.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.159+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/2303270602 shutdown_connections 2024-02-19T19:43:47.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.159+0000 7fe9d1f7d640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fe9b806d7c0 0x7fe9b806fc70 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.159+0000 7fe9d1f7d640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fe9cc104590 0x7fe9cc196660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.159+0000 7fe9d1f7d640 1 --2- 172.21.15.88:0/2303270602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9cc0fe5b0 0x7fe9cc196120 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.159+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/2303270602 >> 172.21.15.88:0/2303270602 conn(0x7fe9cc0743d0 msgr2=0x7fe9cc0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:47.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.159+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/2303270602 shutdown_connections 2024-02-19T19:43:47.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.159+0000 7fe9d1f7d640 1 -- 172.21.15.88:0/2303270602 wait complete. 2024-02-19T19:43:47.169 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:46 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3860239423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-19T19:43:47.169 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:46 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2552865536' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T19:43:47.169 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:46 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/3028077711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T19:43:47.170 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-02-19T19:43:47.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:46 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3860239423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-19T19:43:47.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:46 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2552865536' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T19:43:47.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:46 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/3028077711' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-19T19:43:47.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.272+0000 7f243ed00640 1 -- 172.21.15.88:0/3679974401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2438104590 msgr2=0x7f2438104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:47.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.272+0000 7f243ed00640 1 --2- 172.21.15.88:0/3679974401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2438104590 0x7f2438104960 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2428009c20 tx=0x7f242802c1b0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.272+0000 7f243ed00640 1 -- 172.21.15.88:0/3679974401 shutdown_connections 2024-02-19T19:43:47.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.272+0000 7f243ed00640 1 --2- 172.21.15.88:0/3679974401 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24380fe5b0 0x7f24380fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.272+0000 7f243ed00640 1 --2- 172.21.15.88:0/3679974401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2438104590 0x7f2438104960 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.272+0000 7f243ed00640 1 -- 172.21.15.88:0/3679974401 >> 172.21.15.88:0/3679974401 conn(0x7f24380743d0 msgr2=0x7f24380747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.272+0000 7f243ed00640 1 -- 172.21.15.88:0/3679974401 shutdown_connections 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.272+0000 7f243ed00640 1 -- 172.21.15.88:0/3679974401 wait complete. 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.273+0000 7f243ed00640 1 Processor -- start 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.273+0000 7f243ed00640 1 -- start start 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.273+0000 7f243ed00640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24380fe5b0 0x7f24381942d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.273+0000 7f243ed00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2438104590 0x7f2438194810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.273+0000 7f243ed00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2438194ea0 con 0x7f2438104590 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.273+0000 7f243ed00640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2438198c70 con 0x7f24380fe5b0 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.273+0000 7f2437fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2438104590 0x7f2438194810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.273+0000 7f2437fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2438104590 0x7f2438194810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51644/0 (socket says 172.21.15.88:51644) 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.273+0000 7f2437fff640 1 -- 172.21.15.88:0/4015398300 learned_addr learned my addr 172.21.15.88:0/4015398300 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.273+0000 7f243ca9c640 1 --2- 172.21.15.88:0/4015398300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24380fe5b0 0x7f24381942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:47.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.273+0000 7f2437fff640 1 -- 172.21.15.88:0/4015398300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24380fe5b0 msgr2=0x7f24381942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:47.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.274+0000 7f2437fff640 1 --2- 172.21.15.88:0/4015398300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24380fe5b0 0x7f24381942d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.274+0000 7f2437fff640 1 -- 172.21.15.88:0/4015398300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2428009900 con 0x7f2438104590 2024-02-19T19:43:47.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.274+0000 7f2437fff640 1 --2- 172.21.15.88:0/4015398300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2438104590 0x7f2438194810 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f242000d910 tx=0x7f242000ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:47.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.274+0000 7f2435ffb640 1 -- 172.21.15.88:0/4015398300 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f242000ed40 con 0x7f2438104590 2024-02-19T19:43:47.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.274+0000 7f243ed00640 1 -- 172.21.15.88:0/4015398300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2438198ef0 con 0x7f2438104590 2024-02-19T19:43:47.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.274+0000 7f2435ffb640 1 -- 172.21.15.88:0/4015398300 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2420016900 con 0x7f2438104590 2024-02-19T19:43:47.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.274+0000 7f2435ffb640 1 -- 172.21.15.88:0/4015398300 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2420014990 con 0x7f2438104590 2024-02-19T19:43:47.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.274+0000 7f243ed00640 1 -- 172.21.15.88:0/4015398300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24381994d0 con 0x7f2438104590 2024-02-19T19:43:47.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.275+0000 7f243ed00640 1 -- 172.21.15.88:0/4015398300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243804e000 con 0x7f2438104590 2024-02-19T19:43:47.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.275+0000 7f2435ffb640 1 -- 172.21.15.88:0/4015398300 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7f2420016440 con 0x7f2438104590 2024-02-19T19:43:47.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.276+0000 7f2435ffb640 1 --2- 172.21.15.88:0/4015398300 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f242406d810 0x7f242406fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:47.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.276+0000 7f2435ffb640 1 -- 172.21.15.88:0/4015398300 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f242008efc0 con 0x7f2438104590 2024-02-19T19:43:47.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.276+0000 7f243ca9c640 1 --2- 172.21.15.88:0/4015398300 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f242406d810 0x7f242406fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:47.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.276+0000 7f243ca9c640 1 --2- 172.21.15.88:0/4015398300 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f242406d810 0x7f242406fcc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2428033000 tx=0x7f2428005f10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:47.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.280+0000 7f2435ffb640 1 -- 172.21.15.88:0/4015398300 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2420058820 con 0x7f2438104590 2024-02-19T19:43:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.451+0000 7f243ed00640 1 -- 172.21.15.88:0/4015398300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f243810b9d0 con 0x7f2438104590 2024-02-19T19:43:47.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.451+0000 7f2435ffb640 1 -- 172.21.15.88:0/4015398300 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v88) v1 ==== 107+0+11 (secure 0 0 0) 0x7f242005bee0 con 0x7f2438104590 2024-02-19T19:43:47.452 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-02-19T19:43:47.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.453+0000 7f243ed00640 1 -- 172.21.15.88:0/4015398300 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f242406d810 msgr2=0x7f242406fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:47.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.453+0000 7f243ed00640 1 --2- 172.21.15.88:0/4015398300 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f242406d810 0x7f242406fcc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2428033000 tx=0x7f2428005f10 comp rx=0 tx=0).stop 2024-02-19T19:43:47.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.453+0000 7f243ed00640 1 -- 172.21.15.88:0/4015398300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2438104590 msgr2=0x7f2438194810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:47.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.453+0000 7f243ed00640 1 --2- 172.21.15.88:0/4015398300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2438104590 0x7f2438194810 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f242000d910 tx=0x7f242000ddd0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.453+0000 7f243ed00640 1 -- 172.21.15.88:0/4015398300 shutdown_connections 2024-02-19T19:43:47.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.453+0000 7f243ed00640 1 --2- 172.21.15.88:0/4015398300 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7f242406d810 0x7f242406fcc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.453+0000 7f243ed00640 1 --2- 172.21.15.88:0/4015398300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2438104590 0x7f2438194810 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.453+0000 7f243ed00640 1 --2- 172.21.15.88:0/4015398300 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7f24380fe5b0 0x7f24381942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.453+0000 7f243ed00640 1 -- 172.21.15.88:0/4015398300 >> 172.21.15.88:0/4015398300 conn(0x7f24380743d0 msgr2=0x7f24380fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.454+0000 7f243ed00640 1 -- 172.21.15.88:0/4015398300 shutdown_connections 2024-02-19T19:43:47.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.454+0000 7f243ed00640 1 -- 172.21.15.88:0/4015398300 wait complete. 2024-02-19T19:43:47.464 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-19T19:43:47.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.566+0000 7fd7dccaa640 1 -- 172.21.15.88:0/2017571837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7d80ff880 msgr2=0x7fd7d80ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:47.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.566+0000 7fd7dccaa640 1 --2- 172.21.15.88:0/2017571837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7d80ff880 0x7fd7d80ffcf0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009c20 tx=0x7fd7c002c1b0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.566+0000 7fd7dccaa640 1 -- 172.21.15.88:0/2017571837 shutdown_connections 2024-02-19T19:43:47.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.566+0000 7fd7dccaa640 1 --2- 172.21.15.88:0/2017571837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7d80ff880 0x7fd7d80ffcf0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.566+0000 7fd7dccaa640 1 --2- 172.21.15.88:0/2017571837 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d8105890 0x7fd7d8105c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.566+0000 7fd7dccaa640 1 -- 172.21.15.88:0/2017571837 >> 172.21.15.88:0/2017571837 conn(0x7fd7d80fb3a0 msgr2=0x7fd7d80fd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.566+0000 7fd7dccaa640 1 -- 172.21.15.88:0/2017571837 shutdown_connections 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.566+0000 7fd7dccaa640 1 -- 172.21.15.88:0/2017571837 wait complete. 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7dccaa640 1 Processor -- start 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7dccaa640 1 -- start start 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7dccaa640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7d80ff880 0x7fd7d81953c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7dccaa640 1 --2- >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d8105890 0x7fd7d8195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7dccaa640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d8195f90 con 0x7fd7d80ff880 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7dccaa640 1 -- --> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d8199d60 con 0x7fd7d8105890 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7d659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7d80ff880 0x7fd7d81953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7d659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7d80ff880 0x7fd7d81953c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51676/0 (socket says 172.21.15.88:51676) 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7d659c640 1 -- 172.21.15.88:0/1462853192 learned_addr learned my addr 172.21.15.88:0/1462853192 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7d5d9b640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d8105890 0x7fd7d8195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7d659c640 1 -- 172.21.15.88:0/1462853192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d8105890 msgr2=0x7fd7d8195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:47.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7d659c640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d8105890 0x7fd7d8195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7d659c640 1 -- 172.21.15.88:0/1462853192 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c0009900 con 0x7fd7d80ff880 2024-02-19T19:43:47.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7d5d9b640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d8105890 0x7fd7d8195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-19T19:43:47.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.567+0000 7fd7d659c640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7d80ff880 0x7fd7d81953c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000d8d0 tx=0x7fd7d000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:47.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.568+0000 7fd7cf7fe640 1 -- 172.21.15.88:0/1462853192 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7d0009a30 con 0x7fd7d80ff880 2024-02-19T19:43:47.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.568+0000 7fd7cf7fe640 1 -- 172.21.15.88:0/1462853192 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7d0009b90 con 0x7fd7d80ff880 2024-02-19T19:43:47.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.568+0000 7fd7dccaa640 1 -- 172.21.15.88:0/1462853192 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d819a040 con 0x7fd7d80ff880 2024-02-19T19:43:47.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.568+0000 7fd7cf7fe640 1 -- 172.21.15.88:0/1462853192 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7d000f680 con 0x7fd7d80ff880 2024-02-19T19:43:47.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.568+0000 7fd7dccaa640 1 -- 172.21.15.88:0/1462853192 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d819a620 con 0x7fd7d80ff880 2024-02-19T19:43:47.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.569+0000 7fd7dccaa640 1 -- 172.21.15.88:0/1462853192 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d804e000 con 0x7fd7d80ff880 2024-02-19T19:43:47.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.569+0000 7fd7cf7fe640 1 -- 172.21.15.88:0/1462853192 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 39) v1 ==== 91493+0+0 (secure 0 0 0) 0x7fd7d0009d00 con 0x7fd7d80ff880 2024-02-19T19:43:47.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.569+0000 7fd7cf7fe640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd7c406d8d0 0x7fd7c406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-19T19:43:47.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.570+0000 7fd7cf7fe640 1 -- 172.21.15.88:0/1462853192 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd7d008d390 con 0x7fd7d80ff880 2024-02-19T19:43:47.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.570+0000 7fd7d5d9b640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd7c406d8d0 0x7fd7c406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-19T19:43:47.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.570+0000 7fd7d5d9b640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd7c406d8d0 0x7fd7c406fd80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd7d81969d0 tx=0x7fd7c0005da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-19T19:43:47.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.574+0000 7fd7cf7fe640 1 -- 172.21.15.88:0/1462853192 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd7d0056b40 con 0x7fd7d80ff880 2024-02-19T19:43:47.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.758+0000 7fd7dccaa640 1 -- 172.21.15.88:0/1462853192 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd7d8066860 con 0x7fd7d80ff880 2024-02-19T19:43:47.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.759+0000 7fd7cf7fe640 1 -- 172.21.15.88:0/1462853192 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v88) v1 ==== 81+0+2435 (secure 0 0 0) 0x7fd7d005a200 con 0x7fd7d80ff880 2024-02-19T19:43:47.759 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6829/2325623973 2024-02-20T19:10:28.265472+0000 2024-02-19T19:43:47.759 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6828/2325623973 2024-02-20T19:10:28.265472+0000 2024-02-19T19:43:47.759 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:6825/2841430379 2024-02-20T19:10:28.265488+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/1344352507 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2177498492 2024-02-20T18:53:39.595841+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6801/681311088 2024-02-20T18:51:05.484896+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6801/221244694 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6801/2764007082 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:6829/3141443905 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3237698093 2024-02-20T18:51:05.484896+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6800/681311088 2024-02-20T18:51:05.484896+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6826/694475098 2024-02-20T18:58:49.196048+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2389966380 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/2573781289 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:47.760 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:6828/3141443905 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6800/3043610921 2024-02-20T18:53:39.595841+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3497052224 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3565328320 2024-02-20T18:51:37.844957+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:6824/2841430379 2024-02-20T19:10:28.265488+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6801/3043610921 2024-02-20T18:53:39.595841+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/2202935815 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/387402116 2024-02-20T18:51:05.484896+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2372050564 2024-02-20T18:53:39.595841+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2300938361 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2164615684 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/2425517471 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6800/2764007082 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3838380819 2024-02-20T18:51:05.484896+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/1356863923 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6801/3414266099 2024-02-20T18:51:37.844957+0000 2024-02-19T19:43:47.761 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6827/694475098 2024-02-20T18:58:49.196048+0000 2024-02-19T19:43:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/3601890998 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6800/3414266099 2024-02-20T18:51:37.844957+0000 2024-02-19T19:43:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.132:0/2131958733 2024-02-20T19:01:32.237246+0000 2024-02-19T19:43:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2418191043 2024-02-20T19:00:47.495177+0000 2024-02-19T19:43:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/4069571809 2024-02-20T18:53:39.595841+0000 2024-02-19T19:43:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3491684321 2024-02-20T18:51:37.844957+0000 2024-02-19T19:43:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/2338689461 2024-02-20T18:51:37.844957+0000 2024-02-19T19:43:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:6800/221244694 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/4002678054 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:47.762 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.88:0/3163483263 2024-02-20T19:04:58.075334+0000 2024-02-19T19:43:47.763 INFO:teuthology.orchestra.run.smithi088.stderr:listed 41 entries 2024-02-19T19:43:47.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.761+0000 7fd7dccaa640 1 -- 172.21.15.88:0/1462853192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd7c406d8d0 msgr2=0x7fd7c406fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:47.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.761+0000 7fd7dccaa640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd7c406d8d0 0x7fd7c406fd80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fd7d81969d0 tx=0x7fd7c0005da0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.761+0000 7fd7dccaa640 1 -- 172.21.15.88:0/1462853192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7d80ff880 msgr2=0x7fd7d81953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-19T19:43:47.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.761+0000 7fd7dccaa640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7d80ff880 0x7fd7d81953c0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fd7d000d8d0 tx=0x7fd7d000dd90 comp rx=0 tx=0).stop 2024-02-19T19:43:47.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.761+0000 7fd7dccaa640 1 -- 172.21.15.88:0/1462853192 shutdown_connections 2024-02-19T19:43:47.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.761+0000 7fd7dccaa640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.88:6800/3859407641,v1:172.21.15.88:6801/3859407641] conn(0x7fd7c406d8d0 0x7fd7c406fd80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.761+0000 7fd7dccaa640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.132:3300/0,v1:172.21.15.132:6789/0] conn(0x7fd7d8105890 0x7fd7d8195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.761+0000 7fd7dccaa640 1 --2- 172.21.15.88:0/1462853192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd7d80ff880 0x7fd7d81953c0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-19T19:43:47.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.761+0000 7fd7dccaa640 1 -- 172.21.15.88:0/1462853192 >> 172.21.15.88:0/1462853192 conn(0x7fd7d80fb3a0 msgr2=0x7fd7d80fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-19T19:43:47.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.761+0000 7fd7dccaa640 1 -- 172.21.15.88:0/1462853192 shutdown_connections 2024-02-19T19:43:47.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-02-19T19:43:47.761+0000 7fd7dccaa640 1 -- 172.21.15.88:0/1462853192 wait complete. 2024-02-19T19:43:47.772 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi088.front.sepia.ceph.com... 2024-02-19T19:43:47.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-19T19:43:47.772 DEBUG:teuthology.orchestra.run.smithi088:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-02-19T19:43:47.799 INFO:teuthology.orchestra.run:waiting for 300 2024-02-19T19:43:47.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:47 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/328077296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T19:43:47.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:47 smithi132 ceph-mon[70676]: pgmap v1209: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:47.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:47 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/2303270602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-19T19:43:47.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:47 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/4015398300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-19T19:43:47.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:47 smithi132 ceph-mon[70676]: from='client.? 172.21.15.88:0/1462853192' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T19:43:48.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:47 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/328077296' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-19T19:43:48.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:47 smithi088 ceph-mon[88643]: pgmap v1209: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:48.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:47 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/2303270602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-02-19T19:43:48.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:47 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/4015398300' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-02-19T19:43:48.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:47 smithi088 ceph-mon[88643]: from='client.? 172.21.15.88:0/1462853192' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-19T19:43:50.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:50 smithi132 ceph-mon[70676]: pgmap v1210: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:50.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:50 smithi088 ceph-mon[88643]: pgmap v1210: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:52.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:52 smithi132 ceph-mon[70676]: pgmap v1211: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:52.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:52 smithi088 ceph-mon[88643]: pgmap v1211: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:53.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:43:53.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:43:54.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:54 smithi132 ceph-mon[70676]: pgmap v1212: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:54.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:43:54.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:43:54.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:43:54.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:54 smithi088 ceph-mon[88643]: pgmap v1212: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:54.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:43:54.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:43:54.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:43:56.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:56 smithi132 ceph-mon[70676]: pgmap v1213: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:56.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:56 smithi088 ceph-mon[88643]: pgmap v1213: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:58.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:58 smithi132 ceph-mon[70676]: pgmap v1214: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:58.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:58 smithi088 ceph-mon[88643]: pgmap v1214: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:43:59.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:43:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:43:59.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:43:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:44:00.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:00 smithi132 ceph-mon[70676]: pgmap v1215: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:00.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:00 smithi088 ceph-mon[88643]: pgmap v1215: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:02.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:02 smithi132 ceph-mon[70676]: pgmap v1216: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:02.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:02 smithi088 ceph-mon[88643]: pgmap v1216: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:04.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:04 smithi132 ceph-mon[70676]: pgmap v1217: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:04.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:04 smithi088 ceph-mon[88643]: pgmap v1217: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:06.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:06 smithi132 ceph-mon[70676]: pgmap v1218: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:06.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:06 smithi088 ceph-mon[88643]: pgmap v1218: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:08.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:08 smithi132 ceph-mon[70676]: pgmap v1219: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:08.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:08 smithi088 ceph-mon[88643]: pgmap v1219: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:10.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:10 smithi132 ceph-mon[70676]: pgmap v1220: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:10.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:10 smithi088 ceph-mon[88643]: pgmap v1220: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:12.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:12 smithi088 ceph-mon[88643]: pgmap v1221: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:12.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:12 smithi132 ceph-mon[70676]: pgmap v1221: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:14.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:14 smithi088 ceph-mon[88643]: pgmap v1222: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:14.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:44:14.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:14 smithi132 ceph-mon[70676]: pgmap v1222: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:14.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:44:16.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:16 smithi088 ceph-mon[88643]: pgmap v1223: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:16.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:16 smithi132 ceph-mon[70676]: pgmap v1223: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:18.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:18 smithi088 ceph-mon[88643]: pgmap v1224: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:18.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:18 smithi132 ceph-mon[70676]: pgmap v1224: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:20.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:20 smithi088 ceph-mon[88643]: pgmap v1225: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:20.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:20 smithi132 ceph-mon[70676]: pgmap v1225: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:22.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:22 smithi088 ceph-mon[88643]: pgmap v1226: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:22.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:22 smithi132 ceph-mon[70676]: pgmap v1226: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:24.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:24 smithi088 ceph-mon[88643]: pgmap v1227: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:24.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:24 smithi132 ceph-mon[70676]: pgmap v1227: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:26.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:26 smithi088 ceph-mon[88643]: pgmap v1228: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:26.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:26 smithi132 ceph-mon[70676]: pgmap v1228: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:28.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:28 smithi088 ceph-mon[88643]: pgmap v1229: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:28.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:44:28.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:28 smithi132 ceph-mon[70676]: pgmap v1229: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:28.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:44:30.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:30 smithi088 ceph-mon[88643]: pgmap v1230: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:30.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:30 smithi132 ceph-mon[70676]: pgmap v1230: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:32.583 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:32 smithi088 ceph-mon[88643]: pgmap v1231: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:32.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:32 smithi132 ceph-mon[70676]: pgmap v1231: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:34.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:34 smithi088 ceph-mon[88643]: pgmap v1232: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:34.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:34 smithi132 ceph-mon[70676]: pgmap v1232: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:36.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:36 smithi088 ceph-mon[88643]: pgmap v1233: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:36.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:36 smithi132 ceph-mon[70676]: pgmap v1233: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:38.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:38 smithi132 ceph-mon[70676]: pgmap v1234: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:38.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:38 smithi088 ceph-mon[88643]: pgmap v1234: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:40.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:40 smithi088 ceph-mon[88643]: pgmap v1235: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:40.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:40 smithi132 ceph-mon[70676]: pgmap v1235: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:42.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:42 smithi132 ceph-mon[70676]: pgmap v1236: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:42.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:42 smithi088 ceph-mon[88643]: pgmap v1236: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:43.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:44:43.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:44:44.694 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:44 smithi088 ceph-mon[88643]: pgmap v1237: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:44.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:44 smithi132 ceph-mon[70676]: pgmap v1237: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:46.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:46 smithi132 ceph-mon[70676]: pgmap v1238: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:46.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:46 smithi088 ceph-mon[88643]: pgmap v1238: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:48.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:48 smithi132 ceph-mon[70676]: pgmap v1239: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:48 smithi088 ceph-mon[88643]: pgmap v1239: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:50.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:50 smithi132 ceph-mon[70676]: pgmap v1240: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:50.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:50 smithi088 ceph-mon[88643]: pgmap v1240: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:52.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:52 smithi132 ceph-mon[70676]: pgmap v1241: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:52.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:52 smithi088 ceph-mon[88643]: pgmap v1241: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:54.692 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:54 smithi088 ceph-mon[88643]: pgmap v1242: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:54.693 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:44:54.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:54 smithi132 ceph-mon[70676]: pgmap v1242: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:54.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:44:55.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:44:55.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:44:55.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:44:55.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:44:55.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:44:55.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:44:56.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:56 smithi132 ceph-mon[70676]: pgmap v1243: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:57.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:56 smithi088 ceph-mon[88643]: pgmap v1243: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:58.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:58 smithi132 ceph-mon[70676]: pgmap v1244: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:58.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:44:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:44:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:58 smithi088 ceph-mon[88643]: pgmap v1244: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:44:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:44:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:45:00.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:00 smithi132 ceph-mon[70676]: pgmap v1245: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:01.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:00 smithi088 ceph-mon[88643]: pgmap v1245: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:02.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:02 smithi132 ceph-mon[70676]: pgmap v1246: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:03.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:02 smithi088 ceph-mon[88643]: pgmap v1246: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:04.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:04 smithi132 ceph-mon[70676]: pgmap v1247: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:05.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:04 smithi088 ceph-mon[88643]: pgmap v1247: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:06.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:06 smithi132 ceph-mon[70676]: pgmap v1248: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:07.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:06 smithi088 ceph-mon[88643]: pgmap v1248: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:08.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:08 smithi132 ceph-mon[70676]: pgmap v1249: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:09.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:08 smithi088 ceph-mon[88643]: pgmap v1249: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:10.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:10 smithi132 ceph-mon[70676]: pgmap v1250: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:11.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:10 smithi088 ceph-mon[88643]: pgmap v1250: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:12.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:12 smithi132 ceph-mon[70676]: pgmap v1251: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:13.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:12 smithi088 ceph-mon[88643]: pgmap v1251: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:13.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:45:14.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:45:14.981 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:14 smithi088 ceph-mon[88643]: pgmap v1252: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:14.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:14 smithi132 ceph-mon[70676]: pgmap v1252: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:16.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:16 smithi132 ceph-mon[70676]: pgmap v1253: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:17.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:16 smithi088 ceph-mon[88643]: pgmap v1253: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:18.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:18 smithi132 ceph-mon[70676]: pgmap v1254: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:19.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:18 smithi088 ceph-mon[88643]: pgmap v1254: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:20.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:20 smithi132 ceph-mon[70676]: pgmap v1255: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:21.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:20 smithi088 ceph-mon[88643]: pgmap v1255: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:22.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:22 smithi132 ceph-mon[70676]: pgmap v1256: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:23.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:22 smithi088 ceph-mon[88643]: pgmap v1256: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:24.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:24 smithi132 ceph-mon[70676]: pgmap v1257: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:25.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:24 smithi088 ceph-mon[88643]: pgmap v1257: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:26.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:26 smithi088 ceph-mon[88643]: pgmap v1258: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:26.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:26 smithi132 ceph-mon[70676]: pgmap v1258: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:28.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:28 smithi132 ceph-mon[70676]: pgmap v1259: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:28.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:45:29.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:28 smithi088 ceph-mon[88643]: pgmap v1259: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:29.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:45:30.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:30 smithi132 ceph-mon[70676]: pgmap v1260: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:31.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:30 smithi088 ceph-mon[88643]: pgmap v1260: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:32.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:32 smithi132 ceph-mon[70676]: pgmap v1261: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:33.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:32 smithi088 ceph-mon[88643]: pgmap v1261: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:34.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:34 smithi132 ceph-mon[70676]: pgmap v1262: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:35.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:34 smithi088 ceph-mon[88643]: pgmap v1262: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:36.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:36 smithi088 ceph-mon[88643]: pgmap v1263: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:36.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:36 smithi132 ceph-mon[70676]: pgmap v1263: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:38.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:38 smithi132 ceph-mon[70676]: pgmap v1264: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:39.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:38 smithi088 ceph-mon[88643]: pgmap v1264: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:40.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:40 smithi132 ceph-mon[70676]: pgmap v1265: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:41.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:40 smithi088 ceph-mon[88643]: pgmap v1265: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:42.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:42 smithi132 ceph-mon[70676]: pgmap v1266: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:43.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:42 smithi088 ceph-mon[88643]: pgmap v1266: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:43.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:45:44.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:45:44.981 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:44 smithi088 ceph-mon[88643]: pgmap v1267: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:44.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:44 smithi132 ceph-mon[70676]: pgmap v1267: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:46.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:46 smithi132 ceph-mon[70676]: pgmap v1268: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:47.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:46 smithi088 ceph-mon[88643]: pgmap v1268: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:48.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:48 smithi132 ceph-mon[70676]: pgmap v1269: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:49.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:48 smithi088 ceph-mon[88643]: pgmap v1269: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:50.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:50 smithi132 ceph-mon[70676]: pgmap v1270: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:51.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:50 smithi088 ceph-mon[88643]: pgmap v1270: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:52.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:52 smithi132 ceph-mon[70676]: pgmap v1271: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:53.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:52 smithi088 ceph-mon[88643]: pgmap v1271: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:54.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:54 smithi132 ceph-mon[70676]: pgmap v1272: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:54.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:45:55.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:54 smithi088 ceph-mon[88643]: pgmap v1272: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:55.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:45:55.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:45:55.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:45:55.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:45:56.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:45:56.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:45:56.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:45:56.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:56 smithi132 ceph-mon[70676]: pgmap v1273: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:57.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:56 smithi088 ceph-mon[88643]: pgmap v1273: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:58.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:58 smithi132 ceph-mon[70676]: pgmap v1274: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:58.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:45:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:45:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:58 smithi088 ceph-mon[88643]: pgmap v1274: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:45:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:45:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:46:00.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:00 smithi132 ceph-mon[70676]: pgmap v1275: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:01.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:00 smithi088 ceph-mon[88643]: pgmap v1275: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:02.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:02 smithi132 ceph-mon[70676]: pgmap v1276: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:03.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:02 smithi088 ceph-mon[88643]: pgmap v1276: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:04.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:04 smithi132 ceph-mon[70676]: pgmap v1277: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:05.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:04 smithi088 ceph-mon[88643]: pgmap v1277: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:06.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:06 smithi132 ceph-mon[70676]: pgmap v1278: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:07.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:06 smithi088 ceph-mon[88643]: pgmap v1278: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:08.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:08 smithi132 ceph-mon[70676]: pgmap v1279: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:09.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:08 smithi088 ceph-mon[88643]: pgmap v1279: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:10.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:10 smithi132 ceph-mon[70676]: pgmap v1280: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:11.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:10 smithi088 ceph-mon[88643]: pgmap v1280: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:13.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:12 smithi088 ceph-mon[88643]: pgmap v1281: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:13.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:12 smithi132 ceph-mon[70676]: pgmap v1281: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:14.048 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:46:14.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:46:14.981 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:14 smithi088 ceph-mon[88643]: pgmap v1282: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:15.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:14 smithi132 ceph-mon[70676]: pgmap v1282: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:16.998 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:16 smithi088 ceph-mon[88643]: pgmap v1283: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:17.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:16 smithi132 ceph-mon[70676]: pgmap v1283: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:19.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:18 smithi088 ceph-mon[88643]: pgmap v1284: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:19.097 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:18 smithi132 ceph-mon[70676]: pgmap v1284: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:21.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:20 smithi088 ceph-mon[88643]: pgmap v1285: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:21.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:20 smithi132 ceph-mon[70676]: pgmap v1285: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:23.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:22 smithi088 ceph-mon[88643]: pgmap v1286: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:23.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:22 smithi132 ceph-mon[70676]: pgmap v1286: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:25.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:24 smithi088 ceph-mon[88643]: pgmap v1287: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:25.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:24 smithi132 ceph-mon[70676]: pgmap v1287: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:27.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:26 smithi088 ceph-mon[88643]: pgmap v1288: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:27.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:26 smithi132 ceph-mon[70676]: pgmap v1288: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:29.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:28 smithi088 ceph-mon[88643]: pgmap v1289: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:29.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:46:29.196 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:28 smithi132 ceph-mon[70676]: pgmap v1289: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:29.196 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:46:31.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:30 smithi088 ceph-mon[88643]: pgmap v1290: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:31.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:30 smithi132 ceph-mon[70676]: pgmap v1290: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:33.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:32 smithi088 ceph-mon[88643]: pgmap v1291: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:33.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:32 smithi132 ceph-mon[70676]: pgmap v1291: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:35.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:34 smithi088 ceph-mon[88643]: pgmap v1292: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:35.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:34 smithi132 ceph-mon[70676]: pgmap v1292: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:37.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:36 smithi088 ceph-mon[88643]: pgmap v1293: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:37.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:36 smithi132 ceph-mon[70676]: pgmap v1293: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:39.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:38 smithi088 ceph-mon[88643]: pgmap v1294: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:39.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:38 smithi132 ceph-mon[70676]: pgmap v1294: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:41.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:40 smithi088 ceph-mon[88643]: pgmap v1295: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:41.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:40 smithi132 ceph-mon[70676]: pgmap v1295: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:43.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:42 smithi088 ceph-mon[88643]: pgmap v1296: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:43.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:42 smithi132 ceph-mon[70676]: pgmap v1296: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:44.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:46:44.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:46:44.981 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:44 smithi088 ceph-mon[88643]: pgmap v1297: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:45.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:44 smithi132 ceph-mon[70676]: pgmap v1297: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:47.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:46 smithi088 ceph-mon[88643]: pgmap v1298: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:47.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:46 smithi132 ceph-mon[70676]: pgmap v1298: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:49.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:48 smithi088 ceph-mon[88643]: pgmap v1299: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:49.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:48 smithi132 ceph-mon[70676]: pgmap v1299: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:51.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:50 smithi088 ceph-mon[88643]: pgmap v1300: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:51.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:50 smithi132 ceph-mon[70676]: pgmap v1300: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:53.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:52 smithi088 ceph-mon[88643]: pgmap v1301: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:53.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:52 smithi132 ceph-mon[70676]: pgmap v1301: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:55.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:54 smithi088 ceph-mon[88643]: pgmap v1302: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:55.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:54 smithi132 ceph-mon[70676]: pgmap v1302: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:56.079 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:55 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:46:56.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:55 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:46:57.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:56 smithi088 ceph-mon[88643]: pgmap v1303: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:57.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:56 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:46:57.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:56 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:46:57.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:56 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:46:57.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:56 smithi132 ceph-mon[70676]: pgmap v1303: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:57.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:56 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:46:57.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:56 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:46:57.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:56 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:46:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:58 smithi088 ceph-mon[88643]: pgmap v1304: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:46:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:46:59.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:58 smithi132 ceph-mon[70676]: pgmap v1304: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:46:59.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:46:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:47:01.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:00 smithi088 ceph-mon[88643]: pgmap v1305: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:01.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:00 smithi132 ceph-mon[70676]: pgmap v1305: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:03.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:02 smithi132 ceph-mon[70676]: pgmap v1306: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:03.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:02 smithi088 ceph-mon[88643]: pgmap v1306: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:05.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:04 smithi088 ceph-mon[88643]: pgmap v1307: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:05.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:04 smithi132 ceph-mon[70676]: pgmap v1307: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:07.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:06 smithi132 ceph-mon[70676]: pgmap v1308: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:07.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:06 smithi088 ceph-mon[88643]: pgmap v1308: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:09.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:08 smithi132 ceph-mon[70676]: pgmap v1309: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:09.329 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:08 smithi088 ceph-mon[88643]: pgmap v1309: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:11.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:10 smithi132 ceph-mon[70676]: pgmap v1310: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:11.329 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:10 smithi088 ceph-mon[88643]: pgmap v1310: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:13.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:12 smithi088 ceph-mon[88643]: pgmap v1311: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:13.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:12 smithi132 ceph-mon[70676]: pgmap v1311: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:14.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:47:14.329 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:47:14.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:14 smithi132 ceph-mon[70676]: pgmap v1312: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:15.079 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:14 smithi088 ceph-mon[88643]: pgmap v1312: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:16.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:15 smithi132 ceph-mon[70676]: pgmap v1313: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:16.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:15 smithi088 ceph-mon[88643]: pgmap v1313: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:18.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:18 smithi088 ceph-mon[88643]: pgmap v1314: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:18.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:18 smithi132 ceph-mon[70676]: pgmap v1314: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:20.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:20 smithi088 ceph-mon[88643]: pgmap v1315: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:20.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:20 smithi132 ceph-mon[70676]: pgmap v1315: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:22.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:22 smithi088 ceph-mon[88643]: pgmap v1316: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:22.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:22 smithi132 ceph-mon[70676]: pgmap v1316: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:24.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:24 smithi088 ceph-mon[88643]: pgmap v1317: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:24.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:24 smithi132 ceph-mon[70676]: pgmap v1317: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:26.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:26 smithi088 ceph-mon[88643]: pgmap v1318: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:26.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:26 smithi132 ceph-mon[70676]: pgmap v1318: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:28.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:28 smithi088 ceph-mon[88643]: pgmap v1319: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:28.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:47:28.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:28 smithi132 ceph-mon[70676]: pgmap v1319: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:28.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:47:30.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:30 smithi088 ceph-mon[88643]: pgmap v1320: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:30.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:30 smithi132 ceph-mon[70676]: pgmap v1320: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:32.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:32 smithi088 ceph-mon[88643]: pgmap v1321: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:32.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:32 smithi132 ceph-mon[70676]: pgmap v1321: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:34.534 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:34 smithi132 ceph-mon[70676]: pgmap v1322: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:34.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:34 smithi088 ceph-mon[88643]: pgmap v1322: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:36.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:36 smithi088 ceph-mon[88643]: pgmap v1323: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:36.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:36 smithi132 ceph-mon[70676]: pgmap v1323: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:38.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:38 smithi088 ceph-mon[88643]: pgmap v1324: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:38.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:38 smithi132 ceph-mon[70676]: pgmap v1324: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:40.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:40 smithi088 ceph-mon[88643]: pgmap v1325: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:40.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:40 smithi132 ceph-mon[70676]: pgmap v1325: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:42.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:42 smithi088 ceph-mon[88643]: pgmap v1326: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:42.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:42 smithi132 ceph-mon[70676]: pgmap v1326: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:43.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:47:43.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:47:44.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:44 smithi088 ceph-mon[88643]: pgmap v1327: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:44.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:44 smithi132 ceph-mon[70676]: pgmap v1327: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:46.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:46 smithi088 ceph-mon[88643]: pgmap v1328: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:46.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:46 smithi132 ceph-mon[70676]: pgmap v1328: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:48.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:48 smithi088 ceph-mon[88643]: pgmap v1329: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:48.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:48 smithi132 ceph-mon[70676]: pgmap v1329: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:50.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:50 smithi088 ceph-mon[88643]: pgmap v1330: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:50.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:50 smithi132 ceph-mon[70676]: pgmap v1330: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:52.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:52 smithi088 ceph-mon[88643]: pgmap v1331: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:52.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:52 smithi132 ceph-mon[70676]: pgmap v1331: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:54.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:54 smithi088 ceph-mon[88643]: pgmap v1332: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:54.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:54 smithi132 ceph-mon[70676]: pgmap v1332: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:56.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:56 smithi088 ceph-mon[88643]: pgmap v1333: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:56.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:56 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:47:56.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:56 smithi132 ceph-mon[70676]: pgmap v1333: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:56.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:56 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:47:57.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:57 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:47:57.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:57 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:47:57.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:57 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:47:57.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:57 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:47:57.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:57 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:47:57.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:57 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:47:59.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:58 smithi132 ceph-mon[70676]: pgmap v1334: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:59.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:47:59.329 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:58 smithi088 ceph-mon[88643]: pgmap v1334: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:47:59.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:48:00.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:47:59 smithi132 ceph-mon[70676]: pgmap v1335: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:00.329 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:47:59 smithi088 ceph-mon[88643]: pgmap v1335: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:02.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:02 smithi088 ceph-mon[88643]: pgmap v1336: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:02.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:02 smithi132 ceph-mon[70676]: pgmap v1336: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:04.535 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:04 smithi132 ceph-mon[70676]: pgmap v1337: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:04.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:04 smithi088 ceph-mon[88643]: pgmap v1337: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:06.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:06 smithi088 ceph-mon[88643]: pgmap v1338: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:06.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:06 smithi132 ceph-mon[70676]: pgmap v1338: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:08.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:08 smithi088 ceph-mon[88643]: pgmap v1339: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:08.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:08 smithi132 ceph-mon[70676]: pgmap v1339: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:10.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:10 smithi132 ceph-mon[70676]: pgmap v1340: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:10.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:10 smithi088 ceph-mon[88643]: pgmap v1340: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:12.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:12 smithi088 ceph-mon[88643]: pgmap v1341: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:12.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:12 smithi132 ceph-mon[70676]: pgmap v1341: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:13.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:48:13.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:48:14.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:14 smithi088 ceph-mon[88643]: pgmap v1342: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:14.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:14 smithi132 ceph-mon[70676]: pgmap v1342: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:16.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:16 smithi088 ceph-mon[88643]: pgmap v1343: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:16.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:16 smithi132 ceph-mon[70676]: pgmap v1343: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:18.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:18 smithi088 ceph-mon[88643]: pgmap v1344: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:18.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:18 smithi132 ceph-mon[70676]: pgmap v1344: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:20.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:20 smithi088 ceph-mon[88643]: pgmap v1345: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:20.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:20 smithi132 ceph-mon[70676]: pgmap v1345: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:22.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:22 smithi088 ceph-mon[88643]: pgmap v1346: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:22.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:22 smithi132 ceph-mon[70676]: pgmap v1346: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:24.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:24 smithi088 ceph-mon[88643]: pgmap v1347: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:24.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:24 smithi132 ceph-mon[70676]: pgmap v1347: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:26.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:26 smithi088 ceph-mon[88643]: pgmap v1348: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:26.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:26 smithi132 ceph-mon[70676]: pgmap v1348: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:28.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:28 smithi088 ceph-mon[88643]: pgmap v1349: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:28.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:48:28.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:28 smithi132 ceph-mon[70676]: pgmap v1349: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:28.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:48:30.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:30 smithi088 ceph-mon[88643]: pgmap v1350: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:30.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:30 smithi132 ceph-mon[70676]: pgmap v1350: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:32.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:32 smithi088 ceph-mon[88643]: pgmap v1351: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:32.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:32 smithi132 ceph-mon[70676]: pgmap v1351: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:34.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:34 smithi088 ceph-mon[88643]: pgmap v1352: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:34.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:34 smithi132 ceph-mon[70676]: pgmap v1352: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:36.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:36 smithi088 ceph-mon[88643]: pgmap v1353: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:36.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:36 smithi132 ceph-mon[70676]: pgmap v1353: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:38.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:38 smithi088 ceph-mon[88643]: pgmap v1354: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:38.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:38 smithi132 ceph-mon[70676]: pgmap v1354: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:40.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:40 smithi088 ceph-mon[88643]: pgmap v1355: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:40.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:40 smithi132 ceph-mon[70676]: pgmap v1355: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:42.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:42 smithi088 ceph-mon[88643]: pgmap v1356: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:42.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:42 smithi132 ceph-mon[70676]: pgmap v1356: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:43.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:48:43.731 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:48:44.693 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:44 smithi088 ceph-mon[88643]: pgmap v1357: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:44.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:44 smithi132 ceph-mon[70676]: pgmap v1357: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:46.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:46 smithi132 ceph-mon[70676]: pgmap v1358: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:46.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:46 smithi088 ceph-mon[88643]: pgmap v1358: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:46.861 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-02-19T19:48:46.861 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-19T19:48:46.921 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-19T19:48:46.937 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task sleep... 2024-02-19T19:48:46.938 INFO:teuthology.task.sleep:Sleeping for 3600 seconds 2024-02-19T19:48:48.697 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:48 smithi088 ceph-mon[88643]: pgmap v1359: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:48.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:48 smithi132 ceph-mon[70676]: pgmap v1359: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:50.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:50 smithi132 ceph-mon[70676]: pgmap v1360: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:50.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:50 smithi088 ceph-mon[88643]: pgmap v1360: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:52.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:52 smithi132 ceph-mon[70676]: pgmap v1361: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:52.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:52 smithi088 ceph-mon[88643]: pgmap v1361: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:54.693 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:54 smithi088 ceph-mon[88643]: pgmap v1362: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:54.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:54 smithi132 ceph-mon[70676]: pgmap v1362: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:56.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:56 smithi132 ceph-mon[70676]: pgmap v1363: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:56.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:56 smithi088 ceph-mon[88643]: pgmap v1363: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:57.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:57 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:48:57.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:57 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:48:58.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:58 smithi132 ceph-mon[70676]: pgmap v1364: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:58.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:48:58.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:48:58.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:48:58.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:48:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:48:58.810 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:58 smithi088 ceph-mon[88643]: pgmap v1364: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:48:58.810 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:48:58.810 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:48:58.810 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:48:58.810 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:48:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:49:00.683 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:00 smithi132 ceph-mon[70676]: pgmap v1365: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:00.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:00 smithi088 ceph-mon[88643]: pgmap v1365: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:02.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:02 smithi132 ceph-mon[70676]: pgmap v1366: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:02.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:02 smithi088 ceph-mon[88643]: pgmap v1366: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:04.692 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:04 smithi088 ceph-mon[88643]: pgmap v1367: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:04.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:04 smithi132 ceph-mon[70676]: pgmap v1367: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:06.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:06 smithi132 ceph-mon[70676]: pgmap v1368: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:06 smithi088 ceph-mon[88643]: pgmap v1368: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:08.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:08 smithi132 ceph-mon[70676]: pgmap v1369: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:08.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:08 smithi088 ceph-mon[88643]: pgmap v1369: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:10.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:10 smithi132 ceph-mon[70676]: pgmap v1370: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:10.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:10 smithi088 ceph-mon[88643]: pgmap v1370: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:12.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:12 smithi132 ceph-mon[70676]: pgmap v1371: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:12.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:12 smithi088 ceph-mon[88643]: pgmap v1371: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:13.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:49:13.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:49:14.693 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:14 smithi088 ceph-mon[88643]: pgmap v1372: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:14.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:14 smithi132 ceph-mon[70676]: pgmap v1372: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:16.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:16 smithi132 ceph-mon[70676]: pgmap v1373: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:16.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:16 smithi088 ceph-mon[88643]: pgmap v1373: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:18.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:18 smithi132 ceph-mon[70676]: pgmap v1374: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:18.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:18 smithi088 ceph-mon[88643]: pgmap v1374: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:20.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:20 smithi132 ceph-mon[70676]: pgmap v1375: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:20.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:20 smithi088 ceph-mon[88643]: pgmap v1375: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:22.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:22 smithi132 ceph-mon[70676]: pgmap v1376: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:22.829 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:22 smithi088 ceph-mon[88643]: pgmap v1376: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:24.693 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:24 smithi088 ceph-mon[88643]: pgmap v1377: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:24.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:24 smithi132 ceph-mon[70676]: pgmap v1377: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:26.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:26 smithi132 ceph-mon[70676]: pgmap v1378: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:26.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:26 smithi088 ceph-mon[88643]: pgmap v1378: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:28.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:28 smithi132 ceph-mon[70676]: pgmap v1379: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:28.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:49:28.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:28 smithi088 ceph-mon[88643]: pgmap v1379: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:28.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:49:30.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:30 smithi132 ceph-mon[70676]: pgmap v1380: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:30.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:30 smithi088 ceph-mon[88643]: pgmap v1380: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:32.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:32 smithi132 ceph-mon[70676]: pgmap v1381: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:32.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:32 smithi088 ceph-mon[88643]: pgmap v1381: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:34.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:34 smithi088 ceph-mon[88643]: pgmap v1382: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:34.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:34 smithi132 ceph-mon[70676]: pgmap v1382: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:36.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:36 smithi132 ceph-mon[70676]: pgmap v1383: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:36.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:36 smithi088 ceph-mon[88643]: pgmap v1383: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:38.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:38 smithi132 ceph-mon[70676]: pgmap v1384: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:38.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:38 smithi088 ceph-mon[88643]: pgmap v1384: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:40.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:40 smithi132 ceph-mon[70676]: pgmap v1385: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:40.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:40 smithi088 ceph-mon[88643]: pgmap v1385: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:42.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:42 smithi132 ceph-mon[70676]: pgmap v1386: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:42.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:42 smithi088 ceph-mon[88643]: pgmap v1386: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:43.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:49:43.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:49:44.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:44 smithi088 ceph-mon[88643]: pgmap v1387: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:44.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:44 smithi132 ceph-mon[70676]: pgmap v1387: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:46.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:46 smithi132 ceph-mon[70676]: pgmap v1388: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:46.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:46 smithi088 ceph-mon[88643]: pgmap v1388: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:48.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:48 smithi132 ceph-mon[70676]: pgmap v1389: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:48 smithi088 ceph-mon[88643]: pgmap v1389: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:50.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:50 smithi132 ceph-mon[70676]: pgmap v1390: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:50.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:50 smithi088 ceph-mon[88643]: pgmap v1390: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:52.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:52 smithi132 ceph-mon[70676]: pgmap v1391: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:52.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:52 smithi088 ceph-mon[88643]: pgmap v1391: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:54.693 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:54 smithi088 ceph-mon[88643]: pgmap v1392: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:54.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:54 smithi132 ceph-mon[70676]: pgmap v1392: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:56.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:56 smithi132 ceph-mon[70676]: pgmap v1393: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:56.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:56 smithi088 ceph-mon[88643]: pgmap v1393: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:58.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:58 smithi132 ceph-mon[70676]: pgmap v1394: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:58.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:49:58.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:49:58.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:49:58.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:49:58.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:49:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:49:58.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:58 smithi088 ceph-mon[88643]: pgmap v1394: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:49:58.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:49:58.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:49:58.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:49:58.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:49:58.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:49:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:50:00.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:00 smithi132 ceph-mon[70676]: pgmap v1395: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:00.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:00 smithi132 ceph-mon[70676]: overall HEALTH_OK 2024-02-19T19:50:00.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:00 smithi088 ceph-mon[88643]: pgmap v1395: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:00.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:00 smithi088 ceph-mon[88643]: overall HEALTH_OK 2024-02-19T19:50:02.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:02 smithi132 ceph-mon[70676]: pgmap v1396: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:02.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:02 smithi088 ceph-mon[88643]: pgmap v1396: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:04.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:04 smithi132 ceph-mon[70676]: pgmap v1397: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:04.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:04 smithi088 ceph-mon[88643]: pgmap v1397: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:06.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:06 smithi132 ceph-mon[70676]: pgmap v1398: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:06.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:06 smithi088 ceph-mon[88643]: pgmap v1398: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:08.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:08 smithi132 ceph-mon[70676]: pgmap v1399: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:08.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:08 smithi088 ceph-mon[88643]: pgmap v1399: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:10.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:10 smithi088 ceph-mon[88643]: pgmap v1400: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:10.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:10 smithi132 ceph-mon[70676]: pgmap v1400: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:12.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:12 smithi088 ceph-mon[88643]: pgmap v1401: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:12.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:12 smithi132 ceph-mon[70676]: pgmap v1401: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:13.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:50:13.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:50:14.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:14 smithi088 ceph-mon[88643]: pgmap v1402: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:14.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:14 smithi132 ceph-mon[70676]: pgmap v1402: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:16.732 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:16 smithi132 ceph-mon[70676]: pgmap v1403: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:16.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:16 smithi088 ceph-mon[88643]: pgmap v1403: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:18.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:18 smithi088 ceph-mon[88643]: pgmap v1404: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:18.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:18 smithi132 ceph-mon[70676]: pgmap v1404: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:20.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:20 smithi088 ceph-mon[88643]: pgmap v1405: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:20.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:20 smithi132 ceph-mon[70676]: pgmap v1405: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:22.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:22 smithi088 ceph-mon[88643]: pgmap v1406: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:22.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:22 smithi132 ceph-mon[70676]: pgmap v1406: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:24.777 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:24 smithi088 ceph-mon[88643]: pgmap v1407: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:24.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:24 smithi132 ceph-mon[70676]: pgmap v1407: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:26.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:26 smithi088 ceph-mon[88643]: pgmap v1408: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:26.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:26 smithi132 ceph-mon[70676]: pgmap v1408: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:28.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:28 smithi088 ceph-mon[88643]: pgmap v1409: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:28.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:50:28.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:28 smithi132 ceph-mon[70676]: pgmap v1409: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:28.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:50:30.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:30 smithi088 ceph-mon[88643]: pgmap v1410: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:30.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:30 smithi132 ceph-mon[70676]: pgmap v1410: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:32.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:32 smithi088 ceph-mon[88643]: pgmap v1411: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:32.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:32 smithi132 ceph-mon[70676]: pgmap v1411: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:34.785 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:34 smithi132 ceph-mon[70676]: pgmap v1412: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:34.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:34 smithi088 ceph-mon[88643]: pgmap v1412: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:36.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:36 smithi088 ceph-mon[88643]: pgmap v1413: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:36.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:36 smithi132 ceph-mon[70676]: pgmap v1413: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:38.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:38 smithi088 ceph-mon[88643]: pgmap v1414: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:38.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:38 smithi132 ceph-mon[70676]: pgmap v1414: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:40.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:40 smithi088 ceph-mon[88643]: pgmap v1415: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:40.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:40 smithi132 ceph-mon[70676]: pgmap v1415: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:42.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:42 smithi088 ceph-mon[88643]: pgmap v1416: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:42.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:42 smithi132 ceph-mon[70676]: pgmap v1416: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:43.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:50:43.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:50:44.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:44 smithi088 ceph-mon[88643]: pgmap v1417: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:44.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:44 smithi132 ceph-mon[70676]: pgmap v1417: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:46.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:46 smithi088 ceph-mon[88643]: pgmap v1418: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:46.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:46 smithi132 ceph-mon[70676]: pgmap v1418: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:48.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:48 smithi088 ceph-mon[88643]: pgmap v1419: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:48.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:48 smithi132 ceph-mon[70676]: pgmap v1419: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:50.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:50 smithi088 ceph-mon[88643]: pgmap v1420: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:50.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:50 smithi132 ceph-mon[70676]: pgmap v1420: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:52.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:52 smithi088 ceph-mon[88643]: pgmap v1421: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:52.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:52 smithi132 ceph-mon[70676]: pgmap v1421: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:54.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:54 smithi088 ceph-mon[88643]: pgmap v1422: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:54.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:54 smithi132 ceph-mon[70676]: pgmap v1422: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:56.819 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:56 smithi132 ceph-mon[70676]: pgmap v1423: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:56.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:56 smithi088 ceph-mon[88643]: pgmap v1423: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:58.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:58 smithi088 ceph-mon[88643]: pgmap v1424: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:58.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:50:58.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:50:58.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:58 smithi132 ceph-mon[70676]: pgmap v1424: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:50:58.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:50:58.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:50:59.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:50:59.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:50:59.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:50:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:50:59.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:50:59.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:50:59.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:50:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:51:00.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:00 smithi088 ceph-mon[88643]: pgmap v1425: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:00.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:00 smithi132 ceph-mon[70676]: pgmap v1425: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:02.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:02 smithi088 ceph-mon[88643]: pgmap v1426: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:02.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:02 smithi132 ceph-mon[70676]: pgmap v1426: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:04 smithi132 ceph-mon[70676]: pgmap v1427: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:05.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:04 smithi088 ceph-mon[88643]: pgmap v1427: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:06.917 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:06 smithi132 ceph-mon[70676]: pgmap v1428: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:07.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:06 smithi088 ceph-mon[88643]: pgmap v1428: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:08.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:08 smithi132 ceph-mon[70676]: pgmap v1429: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:09.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:08 smithi088 ceph-mon[88643]: pgmap v1429: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:10.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:10 smithi132 ceph-mon[70676]: pgmap v1430: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:11.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:10 smithi088 ceph-mon[88643]: pgmap v1430: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:12.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:12 smithi132 ceph-mon[70676]: pgmap v1431: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:13.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:12 smithi088 ceph-mon[88643]: pgmap v1431: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:13.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:51:14.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:51:14.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:14 smithi132 ceph-mon[70676]: pgmap v1432: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:15.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:14 smithi088 ceph-mon[88643]: pgmap v1432: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:16.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:16 smithi132 ceph-mon[70676]: pgmap v1433: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:17.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:16 smithi088 ceph-mon[88643]: pgmap v1433: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:18.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:18 smithi132 ceph-mon[70676]: pgmap v1434: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:19.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:18 smithi088 ceph-mon[88643]: pgmap v1434: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:20.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:20 smithi088 ceph-mon[88643]: pgmap v1435: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:20.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:20 smithi132 ceph-mon[70676]: pgmap v1435: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:22.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:22 smithi132 ceph-mon[70676]: pgmap v1436: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:23.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:22 smithi088 ceph-mon[88643]: pgmap v1436: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:24.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:24 smithi132 ceph-mon[70676]: pgmap v1437: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:25.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:24 smithi088 ceph-mon[88643]: pgmap v1437: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:26.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:26 smithi132 ceph-mon[70676]: pgmap v1438: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:27.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:26 smithi088 ceph-mon[88643]: pgmap v1438: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:28 smithi132 ceph-mon[70676]: pgmap v1439: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:51:29.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:28 smithi088 ceph-mon[88643]: pgmap v1439: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:29.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:51:30.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:30 smithi088 ceph-mon[88643]: pgmap v1440: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:30.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:30 smithi132 ceph-mon[70676]: pgmap v1440: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:32.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:32 smithi132 ceph-mon[70676]: pgmap v1441: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:33.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:32 smithi088 ceph-mon[88643]: pgmap v1441: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:34 smithi132 ceph-mon[70676]: pgmap v1442: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:35.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:34 smithi088 ceph-mon[88643]: pgmap v1442: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:36 smithi132 ceph-mon[70676]: pgmap v1443: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:37.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:36 smithi088 ceph-mon[88643]: pgmap v1443: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:38 smithi132 ceph-mon[70676]: pgmap v1444: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:39.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:38 smithi088 ceph-mon[88643]: pgmap v1444: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:40.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:40 smithi132 ceph-mon[70676]: pgmap v1445: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:41.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:40 smithi088 ceph-mon[88643]: pgmap v1445: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:42.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:42 smithi132 ceph-mon[70676]: pgmap v1446: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:43.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:42 smithi088 ceph-mon[88643]: pgmap v1446: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:43.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:51:44.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:51:44.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:44 smithi132 ceph-mon[70676]: pgmap v1447: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:45.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:44 smithi088 ceph-mon[88643]: pgmap v1447: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:46.982 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:46 smithi088 ceph-mon[88643]: pgmap v1448: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:46.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:46 smithi132 ceph-mon[70676]: pgmap v1448: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:48.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:48 smithi132 ceph-mon[70676]: pgmap v1449: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:49.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:48 smithi088 ceph-mon[88643]: pgmap v1449: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:50.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:50 smithi132 ceph-mon[70676]: pgmap v1450: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:50 smithi088 ceph-mon[88643]: pgmap v1450: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:52.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:52 smithi132 ceph-mon[70676]: pgmap v1451: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:53.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:52 smithi088 ceph-mon[88643]: pgmap v1451: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:54.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:54 smithi132 ceph-mon[70676]: pgmap v1452: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:55.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:54 smithi088 ceph-mon[88643]: pgmap v1452: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:56.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:56 smithi132 ceph-mon[70676]: pgmap v1453: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:57.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:56 smithi088 ceph-mon[88643]: pgmap v1453: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:58.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:58 smithi132 ceph-mon[70676]: pgmap v1454: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:58.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:51:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:58 smithi088 ceph-mon[88643]: pgmap v1454: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:51:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:51:59.982 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:51:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:52:00.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:51:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:52:00.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:00 smithi132 ceph-mon[70676]: pgmap v1455: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:00.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:00.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:00.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:00.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:00.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:00 smithi088 ceph-mon[88643]: pgmap v1455: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:00.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:00.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:00.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:00.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:02.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:02 smithi132 ceph-mon[70676]: pgmap v1456: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:02.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:02.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:03.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:02 smithi088 ceph-mon[88643]: pgmap v1456: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:03.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:03.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:04 smithi132 ceph-mon[70676]: pgmap v1457: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:52:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:52:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:05.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:04 smithi088 ceph-mon[88643]: pgmap v1457: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:05.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:52:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:52:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:52:06.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:06 smithi132 ceph-mon[70676]: pgmap v1458: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:07.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:06 smithi088 ceph-mon[88643]: pgmap v1458: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:08.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:08 smithi132 ceph-mon[70676]: pgmap v1459: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:09.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:08 smithi088 ceph-mon[88643]: pgmap v1459: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:11.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:10 smithi088 ceph-mon[88643]: pgmap v1460: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:11.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:10 smithi132 ceph-mon[70676]: pgmap v1460: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:12.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:12 smithi132 ceph-mon[70676]: pgmap v1461: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:13.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:12 smithi088 ceph-mon[88643]: pgmap v1461: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:52:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:52:15.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:14 smithi088 ceph-mon[88643]: pgmap v1462: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:14 smithi132 ceph-mon[70676]: pgmap v1462: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:17.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:16 smithi088 ceph-mon[88643]: pgmap v1463: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:17.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:16 smithi132 ceph-mon[70676]: pgmap v1463: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:18 smithi088 ceph-mon[88643]: pgmap v1464: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:19.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:18 smithi132 ceph-mon[70676]: pgmap v1464: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:21.070 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:20 smithi088 ceph-mon[88643]: pgmap v1465: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:20 smithi132 ceph-mon[70676]: pgmap v1465: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:22.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:22 smithi132 ceph-mon[70676]: pgmap v1466: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:23.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:22 smithi088 ceph-mon[88643]: pgmap v1466: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:25.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:24 smithi088 ceph-mon[88643]: pgmap v1467: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:24 smithi132 ceph-mon[70676]: pgmap v1467: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:27.028 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:26 smithi088 ceph-mon[88643]: pgmap v1468: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:27.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:26 smithi132 ceph-mon[70676]: pgmap v1468: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:29.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:28 smithi088 ceph-mon[88643]: pgmap v1469: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:52:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:28 smithi132 ceph-mon[70676]: pgmap v1469: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:52:31.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:30 smithi088 ceph-mon[88643]: pgmap v1470: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:30 smithi132 ceph-mon[70676]: pgmap v1470: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:33.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:32 smithi088 ceph-mon[88643]: pgmap v1471: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:33.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:32 smithi132 ceph-mon[70676]: pgmap v1471: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:35.035 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:34 smithi132 ceph-mon[70676]: pgmap v1472: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:35.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:34 smithi088 ceph-mon[88643]: pgmap v1472: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:37.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:36 smithi088 ceph-mon[88643]: pgmap v1473: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:36 smithi132 ceph-mon[70676]: pgmap v1473: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:38 smithi088 ceph-mon[88643]: pgmap v1474: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:38 smithi132 ceph-mon[70676]: pgmap v1474: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:41.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:40 smithi088 ceph-mon[88643]: pgmap v1475: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:40 smithi132 ceph-mon[70676]: pgmap v1475: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:43.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:42 smithi088 ceph-mon[88643]: pgmap v1476: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:43.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:42 smithi132 ceph-mon[70676]: pgmap v1476: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:44.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:52:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:52:45.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:44 smithi088 ceph-mon[88643]: pgmap v1477: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:44 smithi132 ceph-mon[70676]: pgmap v1477: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:46 smithi088 ceph-mon[88643]: pgmap v1478: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:46 smithi132 ceph-mon[70676]: pgmap v1478: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:49.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:48 smithi088 ceph-mon[88643]: pgmap v1479: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:49.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:48 smithi132 ceph-mon[70676]: pgmap v1479: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:51.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:50 smithi088 ceph-mon[88643]: pgmap v1480: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:50 smithi132 ceph-mon[70676]: pgmap v1480: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:53.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:52 smithi088 ceph-mon[88643]: pgmap v1481: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:52 smithi132 ceph-mon[70676]: pgmap v1481: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:55.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:54 smithi088 ceph-mon[88643]: pgmap v1482: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:54 smithi132 ceph-mon[70676]: pgmap v1482: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:56 smithi088 ceph-mon[88643]: pgmap v1483: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:56 smithi132 ceph-mon[70676]: pgmap v1483: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:59.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:58 smithi088 ceph-mon[88643]: pgmap v1484: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:52:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:52:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:58 smithi132 ceph-mon[70676]: pgmap v1484: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:52:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:52:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:53:01.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:00 smithi088 ceph-mon[88643]: pgmap v1485: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:00 smithi132 ceph-mon[70676]: pgmap v1485: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:02 smithi088 ceph-mon[88643]: pgmap v1486: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:03.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:02 smithi132 ceph-mon[70676]: pgmap v1486: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:05.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:04 smithi088 ceph-mon[88643]: pgmap v1487: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:53:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:04 smithi132 ceph-mon[70676]: pgmap v1487: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:53:06.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:53:06.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:53:06.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:53:06.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:53:06.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:53:06.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:53:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:06 smithi088 ceph-mon[88643]: pgmap v1488: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:07.236 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:06 smithi132 ceph-mon[70676]: pgmap v1488: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:09.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:08 smithi132 ceph-mon[70676]: pgmap v1489: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:09.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:08 smithi088 ceph-mon[88643]: pgmap v1489: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:11.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:10 smithi132 ceph-mon[70676]: pgmap v1490: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:11.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:10 smithi088 ceph-mon[88643]: pgmap v1490: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:13.157 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:12 smithi132 ceph-mon[70676]: pgmap v1491: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:13.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:12 smithi088 ceph-mon[88643]: pgmap v1491: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:53:14.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:53:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:14 smithi088 ceph-mon[88643]: pgmap v1492: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:14 smithi132 ceph-mon[70676]: pgmap v1492: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:17.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:16 smithi088 ceph-mon[88643]: pgmap v1493: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:17.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:16 smithi132 ceph-mon[70676]: pgmap v1493: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:19.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:18 smithi132 ceph-mon[70676]: pgmap v1494: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:19.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:18 smithi088 ceph-mon[88643]: pgmap v1494: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:20 smithi132 ceph-mon[70676]: pgmap v1495: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:21.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:20 smithi088 ceph-mon[88643]: pgmap v1495: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:23.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:22 smithi132 ceph-mon[70676]: pgmap v1496: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:23.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:22 smithi088 ceph-mon[88643]: pgmap v1496: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:25.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:24 smithi088 ceph-mon[88643]: pgmap v1497: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:24 smithi132 ceph-mon[70676]: pgmap v1497: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:27.080 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:26 smithi088 ceph-mon[88643]: pgmap v1498: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:27.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:26 smithi132 ceph-mon[70676]: pgmap v1498: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:28 smithi132 ceph-mon[70676]: pgmap v1499: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:53:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:28 smithi088 ceph-mon[88643]: pgmap v1499: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:53:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:30 smithi132 ceph-mon[70676]: pgmap v1500: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:31.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:30 smithi088 ceph-mon[88643]: pgmap v1500: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:33.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:32 smithi132 ceph-mon[70676]: pgmap v1501: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:32 smithi088 ceph-mon[88643]: pgmap v1501: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:34 smithi088 ceph-mon[88643]: pgmap v1502: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:34 smithi132 ceph-mon[70676]: pgmap v1502: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:36 smithi132 ceph-mon[70676]: pgmap v1503: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:37.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:36 smithi088 ceph-mon[88643]: pgmap v1503: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:38 smithi132 ceph-mon[70676]: pgmap v1504: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:39.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:38 smithi088 ceph-mon[88643]: pgmap v1504: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:40 smithi132 ceph-mon[70676]: pgmap v1505: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:41.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:40 smithi088 ceph-mon[88643]: pgmap v1505: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:43.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:42 smithi132 ceph-mon[70676]: pgmap v1506: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:43.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:42 smithi088 ceph-mon[88643]: pgmap v1506: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:53:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:53:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:44 smithi088 ceph-mon[88643]: pgmap v1507: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:44 smithi132 ceph-mon[70676]: pgmap v1507: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:46 smithi132 ceph-mon[70676]: pgmap v1508: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:47.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:46 smithi088 ceph-mon[88643]: pgmap v1508: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:49.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:48 smithi132 ceph-mon[70676]: pgmap v1509: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:49.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:48 smithi088 ceph-mon[88643]: pgmap v1509: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:50 smithi132 ceph-mon[70676]: pgmap v1510: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:51.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:50 smithi088 ceph-mon[88643]: pgmap v1510: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:52 smithi132 ceph-mon[70676]: pgmap v1511: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:53.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:52 smithi088 ceph-mon[88643]: pgmap v1511: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:54 smithi088 ceph-mon[88643]: pgmap v1512: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:54 smithi132 ceph-mon[70676]: pgmap v1512: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:56 smithi132 ceph-mon[70676]: pgmap v1513: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:57.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:56 smithi088 ceph-mon[88643]: pgmap v1513: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:58 smithi132 ceph-mon[70676]: pgmap v1514: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:53:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:53:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:58 smithi088 ceph-mon[88643]: pgmap v1514: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:53:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:53:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:54:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:00 smithi132 ceph-mon[70676]: pgmap v1515: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:01.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:00 smithi088 ceph-mon[88643]: pgmap v1515: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:02.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:01 smithi132 ceph-mon[70676]: pgmap v1516: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:02.330 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:01 smithi088 ceph-mon[88643]: pgmap v1516: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:04.694 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:04 smithi088 ceph-mon[88643]: pgmap v1517: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:04.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:04 smithi132 ceph-mon[70676]: pgmap v1517: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:06.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:06 smithi132 ceph-mon[70676]: pgmap v1518: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:06.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:54:06.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:54:06.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:54:06.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:54:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:06 smithi088 ceph-mon[88643]: pgmap v1518: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:54:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:54:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:54:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:54:08.701 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:08 smithi132 ceph-mon[70676]: pgmap v1519: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:08.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:08 smithi088 ceph-mon[88643]: pgmap v1519: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:10.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:10 smithi132 ceph-mon[70676]: pgmap v1520: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:10.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:10 smithi088 ceph-mon[88643]: pgmap v1520: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:12.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:12 smithi088 ceph-mon[88643]: pgmap v1521: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:12.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:12 smithi132 ceph-mon[70676]: pgmap v1521: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:13.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:54:13.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:54:14.694 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:14 smithi088 ceph-mon[88643]: pgmap v1522: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:14.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:14 smithi132 ceph-mon[70676]: pgmap v1522: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:16.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:16 smithi132 ceph-mon[70676]: pgmap v1523: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:16.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:16 smithi088 ceph-mon[88643]: pgmap v1523: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:18.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:18 smithi132 ceph-mon[70676]: pgmap v1524: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:18.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:18 smithi088 ceph-mon[88643]: pgmap v1524: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:20.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:20 smithi132 ceph-mon[70676]: pgmap v1525: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:20.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:20 smithi088 ceph-mon[88643]: pgmap v1525: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:22.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:22 smithi132 ceph-mon[70676]: pgmap v1526: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:22.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:22 smithi088 ceph-mon[88643]: pgmap v1526: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:24.694 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:24 smithi088 ceph-mon[88643]: pgmap v1527: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:24.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:24 smithi132 ceph-mon[70676]: pgmap v1527: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:26.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:26 smithi132 ceph-mon[70676]: pgmap v1528: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:26.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:26 smithi088 ceph-mon[88643]: pgmap v1528: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:28.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:28 smithi132 ceph-mon[70676]: pgmap v1529: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:28.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:54:28.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:28 smithi088 ceph-mon[88643]: pgmap v1529: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:28.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:54:30.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:30 smithi132 ceph-mon[70676]: pgmap v1530: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:30.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:30 smithi088 ceph-mon[88643]: pgmap v1530: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:32.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:32 smithi132 ceph-mon[70676]: pgmap v1531: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:32.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:32 smithi088 ceph-mon[88643]: pgmap v1531: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:34.694 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:34 smithi088 ceph-mon[88643]: pgmap v1532: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:34.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:34 smithi132 ceph-mon[70676]: pgmap v1532: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:36.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:36 smithi132 ceph-mon[70676]: pgmap v1533: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:36.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:36 smithi088 ceph-mon[88643]: pgmap v1533: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:38.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:38 smithi132 ceph-mon[70676]: pgmap v1534: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:38.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:38 smithi088 ceph-mon[88643]: pgmap v1534: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:40.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:40 smithi132 ceph-mon[70676]: pgmap v1535: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:40.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:40 smithi088 ceph-mon[88643]: pgmap v1535: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:42.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:42 smithi132 ceph-mon[70676]: pgmap v1536: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:42.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:42 smithi088 ceph-mon[88643]: pgmap v1536: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:43.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:54:43.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:54:44.694 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:44 smithi088 ceph-mon[88643]: pgmap v1537: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:44.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:44 smithi132 ceph-mon[70676]: pgmap v1537: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:46.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:46 smithi132 ceph-mon[70676]: pgmap v1538: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:46.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:46 smithi088 ceph-mon[88643]: pgmap v1538: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:48.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:48 smithi132 ceph-mon[70676]: pgmap v1539: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:48.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:48 smithi088 ceph-mon[88643]: pgmap v1539: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:50.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:50 smithi132 ceph-mon[70676]: pgmap v1540: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:50.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:50 smithi088 ceph-mon[88643]: pgmap v1540: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:52.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:52 smithi132 ceph-mon[70676]: pgmap v1541: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:52.770 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:52 smithi088 ceph-mon[88643]: pgmap v1541: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:54.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:54 smithi132 ceph-mon[70676]: pgmap v1542: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:54.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:54 smithi088 ceph-mon[88643]: pgmap v1542: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:56.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:56 smithi132 ceph-mon[70676]: pgmap v1543: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:56.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:56 smithi088 ceph-mon[88643]: pgmap v1543: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:58.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:58 smithi132 ceph-mon[70676]: pgmap v1544: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:58.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:54:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:54:58.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:58 smithi088 ceph-mon[88643]: pgmap v1544: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:54:58.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:54:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:55:00.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:00 smithi132 ceph-mon[70676]: pgmap v1545: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:00.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:00 smithi088 ceph-mon[88643]: pgmap v1545: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:02.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:02 smithi132 ceph-mon[70676]: pgmap v1546: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:02.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:02 smithi088 ceph-mon[88643]: pgmap v1546: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:04.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:04 smithi132 ceph-mon[70676]: pgmap v1547: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:04.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:04 smithi088 ceph-mon[88643]: pgmap v1547: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:06.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:06 smithi132 ceph-mon[70676]: pgmap v1548: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:06.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:55:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:06 smithi088 ceph-mon[88643]: pgmap v1548: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:55:07.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:55:07.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:55:07.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:55:07.830 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:55:07.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:55:07.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:55:08.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:08 smithi132 ceph-mon[70676]: pgmap v1549: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:08.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:08 smithi088 ceph-mon[88643]: pgmap v1549: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:10.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:10 smithi132 ceph-mon[70676]: pgmap v1550: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:10.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:10 smithi088 ceph-mon[88643]: pgmap v1550: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:12.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:12 smithi132 ceph-mon[70676]: pgmap v1551: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:12.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:12 smithi088 ceph-mon[88643]: pgmap v1551: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:13.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:55:13.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:55:14.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:14 smithi132 ceph-mon[70676]: pgmap v1552: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:14.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:14 smithi088 ceph-mon[88643]: pgmap v1552: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:16.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:16 smithi088 ceph-mon[88643]: pgmap v1553: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:16.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:16 smithi132 ceph-mon[70676]: pgmap v1553: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:18.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:18 smithi088 ceph-mon[88643]: pgmap v1554: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:18.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:18 smithi132 ceph-mon[70676]: pgmap v1554: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:20.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:20 smithi088 ceph-mon[88643]: pgmap v1555: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:20.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:20 smithi132 ceph-mon[70676]: pgmap v1555: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:22.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:22 smithi088 ceph-mon[88643]: pgmap v1556: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:22.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:22 smithi132 ceph-mon[70676]: pgmap v1556: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:24.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:24 smithi132 ceph-mon[70676]: pgmap v1557: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:24.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:24 smithi088 ceph-mon[88643]: pgmap v1557: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:26.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:26 smithi088 ceph-mon[88643]: pgmap v1558: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:26.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:26 smithi132 ceph-mon[70676]: pgmap v1558: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:28.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:28 smithi088 ceph-mon[88643]: pgmap v1559: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:28.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:55:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:28 smithi132 ceph-mon[70676]: pgmap v1559: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:55:30.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:30 smithi088 ceph-mon[88643]: pgmap v1560: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:30.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:30 smithi132 ceph-mon[70676]: pgmap v1560: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:32.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:32 smithi088 ceph-mon[88643]: pgmap v1561: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:32.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:32 smithi132 ceph-mon[70676]: pgmap v1561: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:34.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:34 smithi088 ceph-mon[88643]: pgmap v1562: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:34 smithi132 ceph-mon[70676]: pgmap v1562: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:36.785 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:36 smithi132 ceph-mon[70676]: pgmap v1563: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:36.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:36 smithi088 ceph-mon[88643]: pgmap v1563: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:38.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:38 smithi088 ceph-mon[88643]: pgmap v1564: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:38 smithi132 ceph-mon[70676]: pgmap v1564: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:40.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:40 smithi088 ceph-mon[88643]: pgmap v1565: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:40.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:40 smithi132 ceph-mon[70676]: pgmap v1565: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:42.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:42 smithi088 ceph-mon[88643]: pgmap v1566: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:42.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:42 smithi132 ceph-mon[70676]: pgmap v1566: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:43.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:55:43.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:55:44.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:44 smithi088 ceph-mon[88643]: pgmap v1567: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:44.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:44 smithi132 ceph-mon[70676]: pgmap v1567: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:46.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:46 smithi088 ceph-mon[88643]: pgmap v1568: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:46.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:46 smithi132 ceph-mon[70676]: pgmap v1568: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:48.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:48 smithi088 ceph-mon[88643]: pgmap v1569: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:48.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:48 smithi132 ceph-mon[70676]: pgmap v1569: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:50.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:50 smithi088 ceph-mon[88643]: pgmap v1570: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:50.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:50 smithi132 ceph-mon[70676]: pgmap v1570: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:52.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:52 smithi088 ceph-mon[88643]: pgmap v1571: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:52.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:52 smithi132 ceph-mon[70676]: pgmap v1571: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:54.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:54 smithi088 ceph-mon[88643]: pgmap v1572: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:54.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:54 smithi132 ceph-mon[70676]: pgmap v1572: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:56.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:56 smithi088 ceph-mon[88643]: pgmap v1573: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:56.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:56 smithi132 ceph-mon[70676]: pgmap v1573: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:58.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:58 smithi088 ceph-mon[88643]: pgmap v1574: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:58.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:55:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:55:58.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:58 smithi132 ceph-mon[70676]: pgmap v1574: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:55:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:55:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:56:00.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:00 smithi088 ceph-mon[88643]: pgmap v1575: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:00.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:00 smithi132 ceph-mon[70676]: pgmap v1575: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:02.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:02 smithi088 ceph-mon[88643]: pgmap v1576: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:02.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:02 smithi132 ceph-mon[70676]: pgmap v1576: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:04.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:04 smithi088 ceph-mon[88643]: pgmap v1577: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:04.836 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:04 smithi132 ceph-mon[70676]: pgmap v1577: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:06 smithi088 ceph-mon[88643]: pgmap v1578: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:06.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:06 smithi132 ceph-mon[70676]: pgmap v1578: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:07.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:56:08.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:56:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:08 smithi132 ceph-mon[70676]: pgmap v1579: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:56:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:56:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:56:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:08 smithi088 ceph-mon[88643]: pgmap v1579: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:56:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:56:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:56:10.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:10 smithi132 ceph-mon[70676]: pgmap v1580: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:10 smithi088 ceph-mon[88643]: pgmap v1580: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:12.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:12 smithi132 ceph-mon[70676]: pgmap v1581: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:12 smithi088 ceph-mon[88643]: pgmap v1581: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:13.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:56:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:56:14.933 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:14 smithi132 ceph-mon[70676]: pgmap v1582: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:14 smithi088 ceph-mon[88643]: pgmap v1582: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:16.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:16 smithi132 ceph-mon[70676]: pgmap v1583: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:16 smithi088 ceph-mon[88643]: pgmap v1583: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:18.884 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:18 smithi088 ceph-mon[88643]: pgmap v1584: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:18.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:18 smithi132 ceph-mon[70676]: pgmap v1584: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:20.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:20 smithi132 ceph-mon[70676]: pgmap v1585: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:20 smithi088 ceph-mon[88643]: pgmap v1585: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:22.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:22 smithi132 ceph-mon[70676]: pgmap v1586: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:22 smithi088 ceph-mon[88643]: pgmap v1586: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:24.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:24 smithi132 ceph-mon[70676]: pgmap v1587: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:24 smithi088 ceph-mon[88643]: pgmap v1587: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:26.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:26 smithi132 ceph-mon[70676]: pgmap v1588: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:26 smithi088 ceph-mon[88643]: pgmap v1588: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:28 smithi132 ceph-mon[70676]: pgmap v1589: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:28.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:56:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:28 smithi088 ceph-mon[88643]: pgmap v1589: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:56:30.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:30 smithi132 ceph-mon[70676]: pgmap v1590: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:30 smithi088 ceph-mon[88643]: pgmap v1590: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:32.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:32 smithi132 ceph-mon[70676]: pgmap v1591: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:32 smithi088 ceph-mon[88643]: pgmap v1591: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:34 smithi132 ceph-mon[70676]: pgmap v1592: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:34 smithi088 ceph-mon[88643]: pgmap v1592: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:36 smithi132 ceph-mon[70676]: pgmap v1593: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:36 smithi088 ceph-mon[88643]: pgmap v1593: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:38.955 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:38 smithi088 ceph-mon[88643]: pgmap v1594: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:38 smithi132 ceph-mon[70676]: pgmap v1594: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:40.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:40 smithi132 ceph-mon[70676]: pgmap v1595: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:40 smithi088 ceph-mon[88643]: pgmap v1595: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:42.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:42 smithi132 ceph-mon[70676]: pgmap v1596: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:42 smithi088 ceph-mon[88643]: pgmap v1596: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:43.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:56:44.011 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:56:44.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:44 smithi132 ceph-mon[70676]: pgmap v1597: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:44 smithi088 ceph-mon[88643]: pgmap v1597: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:46.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:46 smithi132 ceph-mon[70676]: pgmap v1598: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:46 smithi088 ceph-mon[88643]: pgmap v1598: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:48.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:48 smithi132 ceph-mon[70676]: pgmap v1599: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:49.068 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:48 smithi088 ceph-mon[88643]: pgmap v1599: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:50.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:50 smithi132 ceph-mon[70676]: pgmap v1600: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:50 smithi088 ceph-mon[88643]: pgmap v1600: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:52.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:52 smithi132 ceph-mon[70676]: pgmap v1601: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:52 smithi088 ceph-mon[88643]: pgmap v1601: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:54.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:54 smithi132 ceph-mon[70676]: pgmap v1602: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:54 smithi088 ceph-mon[88643]: pgmap v1602: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:56.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:56 smithi132 ceph-mon[70676]: pgmap v1603: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:56 smithi088 ceph-mon[88643]: pgmap v1603: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:58.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:58 smithi132 ceph-mon[70676]: pgmap v1604: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:58.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:56:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:56:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:58 smithi088 ceph-mon[88643]: pgmap v1604: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:56:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:56:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:57:00.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:00 smithi132 ceph-mon[70676]: pgmap v1605: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:00 smithi088 ceph-mon[88643]: pgmap v1605: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:02.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:02 smithi132 ceph-mon[70676]: pgmap v1606: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:02 smithi088 ceph-mon[88643]: pgmap v1606: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:04 smithi132 ceph-mon[70676]: pgmap v1607: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:04 smithi088 ceph-mon[88643]: pgmap v1607: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:06.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:06 smithi132 ceph-mon[70676]: pgmap v1608: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:06 smithi088 ceph-mon[88643]: pgmap v1608: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:08.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:08 smithi132 ceph-mon[70676]: pgmap v1609: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:57:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:57:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:57:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:57:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:08 smithi088 ceph-mon[88643]: pgmap v1609: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:57:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:57:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:57:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:57:10.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:10 smithi132 ceph-mon[70676]: pgmap v1610: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:10 smithi088 ceph-mon[88643]: pgmap v1610: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:12.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:12 smithi132 ceph-mon[70676]: pgmap v1611: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:12 smithi088 ceph-mon[88643]: pgmap v1611: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:13.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:57:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:57:14.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:14 smithi132 ceph-mon[70676]: pgmap v1612: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:14 smithi088 ceph-mon[88643]: pgmap v1612: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:16.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:16 smithi132 ceph-mon[70676]: pgmap v1613: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:16 smithi088 ceph-mon[88643]: pgmap v1613: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:18.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:18 smithi132 ceph-mon[70676]: pgmap v1614: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:18 smithi088 ceph-mon[88643]: pgmap v1614: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:20.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:20 smithi132 ceph-mon[70676]: pgmap v1615: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:20 smithi088 ceph-mon[88643]: pgmap v1615: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:22.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:22 smithi132 ceph-mon[70676]: pgmap v1616: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:22 smithi088 ceph-mon[88643]: pgmap v1616: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:24.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:24 smithi132 ceph-mon[70676]: pgmap v1617: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:24 smithi088 ceph-mon[88643]: pgmap v1617: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:26.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:26 smithi132 ceph-mon[70676]: pgmap v1618: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:26 smithi088 ceph-mon[88643]: pgmap v1618: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:28 smithi132 ceph-mon[70676]: pgmap v1619: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:57:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:28 smithi088 ceph-mon[88643]: pgmap v1619: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:57:30.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:30 smithi132 ceph-mon[70676]: pgmap v1620: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:30 smithi088 ceph-mon[88643]: pgmap v1620: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:32.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:32 smithi132 ceph-mon[70676]: pgmap v1621: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:32 smithi088 ceph-mon[88643]: pgmap v1621: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:34.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:34 smithi088 ceph-mon[88643]: pgmap v1622: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:34 smithi132 ceph-mon[70676]: pgmap v1622: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:36 smithi132 ceph-mon[70676]: pgmap v1623: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:36 smithi088 ceph-mon[88643]: pgmap v1623: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:38 smithi088 ceph-mon[88643]: pgmap v1624: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:38 smithi132 ceph-mon[70676]: pgmap v1624: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:40 smithi088 ceph-mon[88643]: pgmap v1625: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:40 smithi132 ceph-mon[70676]: pgmap v1625: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:42 smithi088 ceph-mon[88643]: pgmap v1626: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:43.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:42 smithi132 ceph-mon[70676]: pgmap v1626: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:57:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:57:45.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:44 smithi088 ceph-mon[88643]: pgmap v1627: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:44 smithi132 ceph-mon[70676]: pgmap v1627: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:46 smithi088 ceph-mon[88643]: pgmap v1628: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:46 smithi132 ceph-mon[70676]: pgmap v1628: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:48 smithi088 ceph-mon[88643]: pgmap v1629: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:49.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:48 smithi132 ceph-mon[70676]: pgmap v1629: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:50 smithi088 ceph-mon[88643]: pgmap v1630: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:50 smithi132 ceph-mon[70676]: pgmap v1630: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:52 smithi088 ceph-mon[88643]: pgmap v1631: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:52 smithi132 ceph-mon[70676]: pgmap v1631: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:54 smithi088 ceph-mon[88643]: pgmap v1632: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:54 smithi132 ceph-mon[70676]: pgmap v1632: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:56 smithi088 ceph-mon[88643]: pgmap v1633: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:56 smithi132 ceph-mon[70676]: pgmap v1633: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:58 smithi088 ceph-mon[88643]: pgmap v1634: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:57:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:57:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:58 smithi132 ceph-mon[70676]: pgmap v1634: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:57:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:57:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:58:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:00 smithi088 ceph-mon[88643]: pgmap v1635: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:00 smithi132 ceph-mon[70676]: pgmap v1635: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:02 smithi088 ceph-mon[88643]: pgmap v1636: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:03.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:02 smithi132 ceph-mon[70676]: pgmap v1636: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:04 smithi088 ceph-mon[88643]: pgmap v1637: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:04 smithi132 ceph-mon[70676]: pgmap v1637: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:06 smithi088 ceph-mon[88643]: pgmap v1638: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:07.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:06 smithi132 ceph-mon[70676]: pgmap v1638: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:08 smithi088 ceph-mon[88643]: pgmap v1639: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:08 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:58:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:08 smithi132 ceph-mon[70676]: pgmap v1639: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:08 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:58:10.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:58:10.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:58:10.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:58:10.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:58:10.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:58:10.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:58:11.058 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:10 smithi132 ceph-mon[70676]: pgmap v1640: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:10 smithi088 ceph-mon[88643]: pgmap v1640: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:12 smithi088 ceph-mon[88643]: pgmap v1641: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:13.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:12 smithi132 ceph-mon[70676]: pgmap v1641: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:58:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:58:15.027 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:14 smithi088 ceph-mon[88643]: pgmap v1642: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:14 smithi132 ceph-mon[70676]: pgmap v1642: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:16 smithi088 ceph-mon[88643]: pgmap v1643: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:17.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:16 smithi132 ceph-mon[70676]: pgmap v1643: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:18 smithi088 ceph-mon[88643]: pgmap v1644: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:19.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:18 smithi132 ceph-mon[70676]: pgmap v1644: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:20 smithi088 ceph-mon[88643]: pgmap v1645: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:21.155 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:20 smithi132 ceph-mon[70676]: pgmap v1645: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:22 smithi088 ceph-mon[88643]: pgmap v1646: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:22 smithi132 ceph-mon[70676]: pgmap v1646: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:24 smithi088 ceph-mon[88643]: pgmap v1647: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:24 smithi132 ceph-mon[70676]: pgmap v1647: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:26 smithi088 ceph-mon[88643]: pgmap v1648: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:27.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:26 smithi132 ceph-mon[70676]: pgmap v1648: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:28 smithi088 ceph-mon[88643]: pgmap v1649: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:58:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:28 smithi132 ceph-mon[70676]: pgmap v1649: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:29.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:58:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:30 smithi088 ceph-mon[88643]: pgmap v1650: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:30 smithi132 ceph-mon[70676]: pgmap v1650: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:32 smithi088 ceph-mon[88643]: pgmap v1651: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:33.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:32 smithi132 ceph-mon[70676]: pgmap v1651: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:34 smithi088 ceph-mon[88643]: pgmap v1652: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:34 smithi132 ceph-mon[70676]: pgmap v1652: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:36 smithi132 ceph-mon[70676]: pgmap v1653: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:37.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:36 smithi088 ceph-mon[88643]: pgmap v1653: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:38 smithi132 ceph-mon[70676]: pgmap v1654: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:39.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:38 smithi088 ceph-mon[88643]: pgmap v1654: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:40 smithi132 ceph-mon[70676]: pgmap v1655: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:40 smithi088 ceph-mon[88643]: pgmap v1655: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:43.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:42 smithi132 ceph-mon[70676]: pgmap v1656: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:43.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:42 smithi088 ceph-mon[88643]: pgmap v1656: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:58:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:58:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:44 smithi088 ceph-mon[88643]: pgmap v1657: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:44 smithi132 ceph-mon[70676]: pgmap v1657: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:46 smithi132 ceph-mon[70676]: pgmap v1658: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:47.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:46 smithi088 ceph-mon[88643]: pgmap v1658: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:49.133 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:48 smithi088 ceph-mon[88643]: pgmap v1659: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:49.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:48 smithi132 ceph-mon[70676]: pgmap v1659: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:50 smithi132 ceph-mon[70676]: pgmap v1660: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:51.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:50 smithi088 ceph-mon[88643]: pgmap v1660: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:53.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:52 smithi132 ceph-mon[70676]: pgmap v1661: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:53.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:52 smithi088 ceph-mon[88643]: pgmap v1661: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:54 smithi088 ceph-mon[88643]: pgmap v1662: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:54 smithi132 ceph-mon[70676]: pgmap v1662: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:57.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:56 smithi132 ceph-mon[70676]: pgmap v1663: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:57.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:56 smithi088 ceph-mon[88643]: pgmap v1663: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:58 smithi132 ceph-mon[70676]: pgmap v1664: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:58:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:58:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:58 smithi088 ceph-mon[88643]: pgmap v1664: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:58:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:58:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:59:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:00 smithi132 ceph-mon[70676]: pgmap v1665: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:00 smithi088 ceph-mon[88643]: pgmap v1665: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:03.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:02 smithi132 ceph-mon[70676]: pgmap v1666: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:03.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:02 smithi088 ceph-mon[88643]: pgmap v1666: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:05.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:04 smithi088 ceph-mon[88643]: pgmap v1667: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:04 smithi132 ceph-mon[70676]: pgmap v1667: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:07.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:06 smithi132 ceph-mon[70676]: pgmap v1668: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:07.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:06 smithi088 ceph-mon[88643]: pgmap v1668: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:09.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:08 smithi132 ceph-mon[70676]: pgmap v1669: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:09.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:08 smithi088 ceph-mon[88643]: pgmap v1669: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:10.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:59:10.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T19:59:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:10 smithi088 ceph-mon[88643]: pgmap v1670: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:10 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:59:11.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:10 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:59:11.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:10 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:59:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:10 smithi132 ceph-mon[70676]: pgmap v1670: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:10 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T19:59:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:10 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T19:59:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:10 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T19:59:13.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:12 smithi132 ceph-mon[70676]: pgmap v1671: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:13.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:12 smithi088 ceph-mon[88643]: pgmap v1671: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:59:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:59:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:14 smithi088 ceph-mon[88643]: pgmap v1672: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:14 smithi132 ceph-mon[70676]: pgmap v1672: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:16.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:16 smithi132 ceph-mon[70676]: pgmap v1673: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:17.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:16 smithi088 ceph-mon[88643]: pgmap v1673: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:19.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:18 smithi132 ceph-mon[70676]: pgmap v1674: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:19.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:18 smithi088 ceph-mon[88643]: pgmap v1674: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:20 smithi088 ceph-mon[88643]: pgmap v1675: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:20 smithi132 ceph-mon[70676]: pgmap v1675: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:22 smithi132 ceph-mon[70676]: pgmap v1676: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:23.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:22 smithi088 ceph-mon[88643]: pgmap v1676: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:24 smithi088 ceph-mon[88643]: pgmap v1677: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:24 smithi132 ceph-mon[70676]: pgmap v1677: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:27.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:26 smithi132 ceph-mon[70676]: pgmap v1678: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:26 smithi088 ceph-mon[88643]: pgmap v1678: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:29.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:28 smithi132 ceph-mon[70676]: pgmap v1679: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:29.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:59:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:28 smithi088 ceph-mon[88643]: pgmap v1679: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:59:31.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:30 smithi132 ceph-mon[70676]: pgmap v1680: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:31.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:30 smithi088 ceph-mon[88643]: pgmap v1680: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:33.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:32 smithi132 ceph-mon[70676]: pgmap v1681: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:32 smithi088 ceph-mon[88643]: pgmap v1681: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:34 smithi088 ceph-mon[88643]: pgmap v1682: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:34 smithi132 ceph-mon[70676]: pgmap v1682: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:36 smithi132 ceph-mon[70676]: pgmap v1683: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:37.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:36 smithi088 ceph-mon[88643]: pgmap v1683: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:39.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:38 smithi132 ceph-mon[70676]: pgmap v1684: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:39.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:38 smithi088 ceph-mon[88643]: pgmap v1684: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:40 smithi132 ceph-mon[70676]: pgmap v1685: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:40 smithi088 ceph-mon[88643]: pgmap v1685: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:43.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:42 smithi132 ceph-mon[70676]: pgmap v1686: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:43.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:42 smithi088 ceph-mon[88643]: pgmap v1686: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:59:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:59:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:44 smithi132 ceph-mon[70676]: pgmap v1687: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:45.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:44 smithi088 ceph-mon[88643]: pgmap v1687: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:47.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:46 smithi132 ceph-mon[70676]: pgmap v1688: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:47.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:46 smithi088 ceph-mon[88643]: pgmap v1688: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:49.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:48 smithi132 ceph-mon[70676]: pgmap v1689: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:49.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:48 smithi088 ceph-mon[88643]: pgmap v1689: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:51.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:50 smithi132 ceph-mon[70676]: pgmap v1690: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:51.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:50 smithi088 ceph-mon[88643]: pgmap v1690: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:53.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:52 smithi132 ceph-mon[70676]: pgmap v1691: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:53.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:52 smithi088 ceph-mon[88643]: pgmap v1691: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:54 smithi132 ceph-mon[70676]: pgmap v1692: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:55.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:54 smithi088 ceph-mon[88643]: pgmap v1692: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:57.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:56 smithi132 ceph-mon[70676]: pgmap v1693: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:57.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:56 smithi088 ceph-mon[88643]: pgmap v1693: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:58 smithi132 ceph-mon[70676]: pgmap v1694: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 19:59:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T19:59:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:58 smithi088 ceph-mon[88643]: pgmap v1694: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T19:59:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 19:59:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:00:01.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:00 smithi132 ceph-mon[70676]: pgmap v1695: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:01.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:00 smithi132 ceph-mon[70676]: overall HEALTH_OK 2024-02-19T20:00:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:00 smithi088 ceph-mon[88643]: pgmap v1695: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:00 smithi088 ceph-mon[88643]: overall HEALTH_OK 2024-02-19T20:00:03.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:02 smithi132 ceph-mon[70676]: pgmap v1696: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:03.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:02 smithi088 ceph-mon[88643]: pgmap v1696: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:05.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:04 smithi132 ceph-mon[70676]: pgmap v1697: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:05.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:04 smithi088 ceph-mon[88643]: pgmap v1697: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:06.265 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:05 smithi088 ceph-mon[88643]: pgmap v1698: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:06.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:05 smithi132 ceph-mon[70676]: pgmap v1698: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:08.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:08 smithi132 ceph-mon[70676]: pgmap v1699: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:08.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:08 smithi088 ceph-mon[88643]: pgmap v1699: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:10.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:10 smithi132 ceph-mon[70676]: pgmap v1700: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:10.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:10 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:00:10.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:10 smithi088 ceph-mon[88643]: pgmap v1700: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:10.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:10 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:00:11.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:00:11.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:00:11.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:00:11.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:00:11.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:00:11.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:00:12.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:12 smithi132 ceph-mon[70676]: pgmap v1701: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:12 smithi088 ceph-mon[88643]: pgmap v1701: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:13.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:00:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:00:14.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:14 smithi132 ceph-mon[70676]: pgmap v1702: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:14 smithi088 ceph-mon[88643]: pgmap v1702: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:16.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:16 smithi132 ceph-mon[70676]: pgmap v1703: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:16 smithi088 ceph-mon[88643]: pgmap v1703: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:18.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:18 smithi132 ceph-mon[70676]: pgmap v1704: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:18 smithi088 ceph-mon[88643]: pgmap v1704: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:20 smithi132 ceph-mon[70676]: pgmap v1705: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:20 smithi088 ceph-mon[88643]: pgmap v1705: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:22.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:22 smithi132 ceph-mon[70676]: pgmap v1706: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:22 smithi088 ceph-mon[88643]: pgmap v1706: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:24.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:24 smithi132 ceph-mon[70676]: pgmap v1707: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:24 smithi088 ceph-mon[88643]: pgmap v1707: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:26.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:26 smithi088 ceph-mon[88643]: pgmap v1708: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:26.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:26 smithi132 ceph-mon[70676]: pgmap v1708: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:28 smithi132 ceph-mon[70676]: pgmap v1709: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:28.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:00:29.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:28 smithi088 ceph-mon[88643]: pgmap v1709: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:29.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:00:30.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:30 smithi132 ceph-mon[70676]: pgmap v1710: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:30 smithi088 ceph-mon[88643]: pgmap v1710: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:32.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:32 smithi132 ceph-mon[70676]: pgmap v1711: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:32 smithi088 ceph-mon[88643]: pgmap v1711: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:34.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:34 smithi132 ceph-mon[70676]: pgmap v1712: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:34 smithi088 ceph-mon[88643]: pgmap v1712: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:36 smithi132 ceph-mon[70676]: pgmap v1713: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:36 smithi088 ceph-mon[88643]: pgmap v1713: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:38.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:38 smithi132 ceph-mon[70676]: pgmap v1714: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:38 smithi088 ceph-mon[88643]: pgmap v1714: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:40.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:40 smithi132 ceph-mon[70676]: pgmap v1715: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:40 smithi088 ceph-mon[88643]: pgmap v1715: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:42.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:42 smithi132 ceph-mon[70676]: pgmap v1716: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:42 smithi088 ceph-mon[88643]: pgmap v1716: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:43.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:00:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:00:44.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:44 smithi132 ceph-mon[70676]: pgmap v1717: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:44 smithi088 ceph-mon[88643]: pgmap v1717: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:46.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:46 smithi132 ceph-mon[70676]: pgmap v1718: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:46 smithi088 ceph-mon[88643]: pgmap v1718: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:48.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:48 smithi132 ceph-mon[70676]: pgmap v1719: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:48 smithi088 ceph-mon[88643]: pgmap v1719: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:50.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:50 smithi132 ceph-mon[70676]: pgmap v1720: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:50 smithi088 ceph-mon[88643]: pgmap v1720: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:52.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:52 smithi132 ceph-mon[70676]: pgmap v1721: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:52 smithi088 ceph-mon[88643]: pgmap v1721: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:54.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:54 smithi132 ceph-mon[70676]: pgmap v1722: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:54 smithi088 ceph-mon[88643]: pgmap v1722: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:56.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:56 smithi132 ceph-mon[70676]: pgmap v1723: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:56 smithi088 ceph-mon[88643]: pgmap v1723: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:58 smithi132 ceph-mon[70676]: pgmap v1724: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:00:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:00:59.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:58 smithi088 ceph-mon[88643]: pgmap v1724: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:00:59.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:00:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:01:00.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:00 smithi132 ceph-mon[70676]: pgmap v1725: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:00 smithi088 ceph-mon[88643]: pgmap v1725: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:02.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:02 smithi132 ceph-mon[70676]: pgmap v1726: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:02 smithi088 ceph-mon[88643]: pgmap v1726: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:04 smithi132 ceph-mon[70676]: pgmap v1727: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:04 smithi088 ceph-mon[88643]: pgmap v1727: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:06.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:06 smithi132 ceph-mon[70676]: pgmap v1728: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:06 smithi088 ceph-mon[88643]: pgmap v1728: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:08 smithi132 ceph-mon[70676]: pgmap v1729: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:08 smithi088 ceph-mon[88643]: pgmap v1729: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:10 smithi088 ceph-mon[88643]: pgmap v1730: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:10 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:01:11.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:10 smithi132 ceph-mon[70676]: pgmap v1730: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:11.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:10 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:01:12.000 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:01:12.000 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:01:12.000 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:01:12.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:01:12.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:01:12.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:01:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:12 smithi088 ceph-mon[88643]: pgmap v1731: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:13.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:12 smithi132 ceph-mon[70676]: pgmap v1731: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:01:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:01:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:14 smithi088 ceph-mon[88643]: pgmap v1732: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:14 smithi132 ceph-mon[70676]: pgmap v1732: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:17.052 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:16 smithi088 ceph-mon[88643]: pgmap v1733: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:17.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:16 smithi132 ceph-mon[70676]: pgmap v1733: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:18 smithi088 ceph-mon[88643]: pgmap v1734: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:18 smithi132 ceph-mon[70676]: pgmap v1734: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:20 smithi088 ceph-mon[88643]: pgmap v1735: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:21.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:20 smithi132 ceph-mon[70676]: pgmap v1735: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:22 smithi088 ceph-mon[88643]: pgmap v1736: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:22 smithi132 ceph-mon[70676]: pgmap v1736: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:24 smithi088 ceph-mon[88643]: pgmap v1737: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:24 smithi132 ceph-mon[70676]: pgmap v1737: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:26 smithi088 ceph-mon[88643]: pgmap v1738: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:27.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:26 smithi132 ceph-mon[70676]: pgmap v1738: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:28 smithi088 ceph-mon[88643]: pgmap v1739: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:01:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:28 smithi132 ceph-mon[70676]: pgmap v1739: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:01:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:30 smithi088 ceph-mon[88643]: pgmap v1740: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:31.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:30 smithi132 ceph-mon[70676]: pgmap v1740: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:33.028 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:32 smithi132 ceph-mon[70676]: pgmap v1741: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:32 smithi088 ceph-mon[88643]: pgmap v1741: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:34 smithi088 ceph-mon[88643]: pgmap v1742: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:34 smithi132 ceph-mon[70676]: pgmap v1742: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:37.036 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:36 smithi132 ceph-mon[70676]: pgmap v1743: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:36 smithi088 ceph-mon[88643]: pgmap v1743: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:38 smithi088 ceph-mon[88643]: pgmap v1744: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:38 smithi132 ceph-mon[70676]: pgmap v1744: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:40 smithi088 ceph-mon[88643]: pgmap v1745: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:41.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:40 smithi132 ceph-mon[70676]: pgmap v1745: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:42 smithi088 ceph-mon[88643]: pgmap v1746: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:43.127 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:42 smithi132 ceph-mon[70676]: pgmap v1746: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:01:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:01:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:44 smithi088 ceph-mon[88643]: pgmap v1747: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:44 smithi132 ceph-mon[70676]: pgmap v1747: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:46 smithi088 ceph-mon[88643]: pgmap v1748: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:46 smithi132 ceph-mon[70676]: pgmap v1748: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:48 smithi088 ceph-mon[88643]: pgmap v1749: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:49.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:48 smithi132 ceph-mon[70676]: pgmap v1749: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:50 smithi088 ceph-mon[88643]: pgmap v1750: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:51.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:50 smithi132 ceph-mon[70676]: pgmap v1750: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:52 smithi088 ceph-mon[88643]: pgmap v1751: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:53.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:52 smithi132 ceph-mon[70676]: pgmap v1751: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:54 smithi088 ceph-mon[88643]: pgmap v1752: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:54 smithi132 ceph-mon[70676]: pgmap v1752: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:56 smithi088 ceph-mon[88643]: pgmap v1753: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:57.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:56 smithi132 ceph-mon[70676]: pgmap v1753: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:58 smithi088 ceph-mon[88643]: pgmap v1754: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:01:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:01:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:58 smithi132 ceph-mon[70676]: pgmap v1754: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:01:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:01:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:02:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:00 smithi088 ceph-mon[88643]: pgmap v1755: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:01.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:00 smithi132 ceph-mon[70676]: pgmap v1755: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:02 smithi088 ceph-mon[88643]: pgmap v1756: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:03.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:02 smithi132 ceph-mon[70676]: pgmap v1756: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:04 smithi088 ceph-mon[88643]: pgmap v1757: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:04 smithi132 ceph-mon[70676]: pgmap v1757: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:06 smithi088 ceph-mon[88643]: pgmap v1758: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:07.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:06 smithi132 ceph-mon[70676]: pgmap v1758: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:08 smithi088 ceph-mon[88643]: pgmap v1759: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:08 smithi132 ceph-mon[70676]: pgmap v1759: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:10 smithi132 ceph-mon[70676]: pgmap v1760: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:11.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:10 smithi088 ceph-mon[88643]: pgmap v1760: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:12.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:02:12.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:02:13.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:12 smithi132 ceph-mon[70676]: pgmap v1761: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:13.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:12 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:13.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:12 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:13.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:12 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:13.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:12 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:13.263 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:12 smithi088 ceph-mon[88643]: pgmap v1761: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:13.263 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:12 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:13.263 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:12 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:13.263 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:12 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:13.263 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:12 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:02:14.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:02:15.307 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:14 smithi088 ceph-mon[88643]: pgmap v1762: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:15.307 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:15.307 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:15.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:14 smithi132 ceph-mon[70676]: pgmap v1762: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:15.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:15.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:17.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:16 smithi088 ceph-mon[88643]: pgmap v1763: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:17.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:17.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:17.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:16 smithi132 ceph-mon[70676]: pgmap v1763: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:17.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:17.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:18.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:02:18.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:02:18.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:18.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:02:18.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:02:18.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:02:19.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:18 smithi088 ceph-mon[88643]: pgmap v1764: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:19.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:18 smithi132 ceph-mon[70676]: pgmap v1764: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:21.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:20 smithi088 ceph-mon[88643]: pgmap v1765: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:21.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:20 smithi132 ceph-mon[70676]: pgmap v1765: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:22 smithi088 ceph-mon[88643]: pgmap v1766: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:23.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:22 smithi132 ceph-mon[70676]: pgmap v1766: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:25.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:24 smithi088 ceph-mon[88643]: pgmap v1767: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:25.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:25 smithi132 ceph-mon[70676]: pgmap v1767: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:27 smithi088 ceph-mon[88643]: pgmap v1768: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:27.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:27 smithi132 ceph-mon[70676]: pgmap v1768: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:28.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:28 smithi088 ceph-mon[88643]: pgmap v1769: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:28.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:28 smithi132 ceph-mon[70676]: pgmap v1769: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:02:29.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:02:30.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:30 smithi088 ceph-mon[88643]: pgmap v1770: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:30.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:30 smithi132 ceph-mon[70676]: pgmap v1770: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:32.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:32 smithi088 ceph-mon[88643]: pgmap v1771: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:32.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:32 smithi132 ceph-mon[70676]: pgmap v1771: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:34.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:34 smithi088 ceph-mon[88643]: pgmap v1772: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:34.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:34 smithi132 ceph-mon[70676]: pgmap v1772: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:36.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:36 smithi088 ceph-mon[88643]: pgmap v1773: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:36.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:36 smithi132 ceph-mon[70676]: pgmap v1773: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:38.786 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:38 smithi132 ceph-mon[70676]: pgmap v1774: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:38.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:38 smithi088 ceph-mon[88643]: pgmap v1774: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:40.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:40 smithi088 ceph-mon[88643]: pgmap v1775: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:40.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:40 smithi132 ceph-mon[70676]: pgmap v1775: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:42.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:42 smithi088 ceph-mon[88643]: pgmap v1776: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:42.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:42 smithi132 ceph-mon[70676]: pgmap v1776: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:43.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:02:43.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:02:44.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:44 smithi088 ceph-mon[88643]: pgmap v1777: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:44.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:44 smithi132 ceph-mon[70676]: pgmap v1777: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:46.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:46 smithi088 ceph-mon[88643]: pgmap v1778: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:46.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:46 smithi132 ceph-mon[70676]: pgmap v1778: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:48.772 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:48 smithi132 ceph-mon[70676]: pgmap v1779: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:48.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:48 smithi088 ceph-mon[88643]: pgmap v1779: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:50.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:50 smithi088 ceph-mon[88643]: pgmap v1780: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:50.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:50 smithi132 ceph-mon[70676]: pgmap v1780: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:52.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:52 smithi088 ceph-mon[88643]: pgmap v1781: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:52.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:52 smithi132 ceph-mon[70676]: pgmap v1781: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:54.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:54 smithi088 ceph-mon[88643]: pgmap v1782: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:54.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:54 smithi132 ceph-mon[70676]: pgmap v1782: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:56.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:56 smithi088 ceph-mon[88643]: pgmap v1783: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:56.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:56 smithi132 ceph-mon[70676]: pgmap v1783: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:58.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:58 smithi088 ceph-mon[88643]: pgmap v1784: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:58.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:02:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:02:58.870 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:58 smithi132 ceph-mon[70676]: pgmap v1784: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:02:58.870 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:02:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:03:00.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:00 smithi088 ceph-mon[88643]: pgmap v1785: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:00.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:00 smithi132 ceph-mon[70676]: pgmap v1785: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:02.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:02 smithi088 ceph-mon[88643]: pgmap v1786: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:02.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:02 smithi132 ceph-mon[70676]: pgmap v1786: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:04.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:04 smithi088 ceph-mon[88643]: pgmap v1787: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:04.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:04 smithi132 ceph-mon[70676]: pgmap v1787: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:06 smithi088 ceph-mon[88643]: pgmap v1788: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:06.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:06 smithi132 ceph-mon[70676]: pgmap v1788: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:08.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:08 smithi088 ceph-mon[88643]: pgmap v1789: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:08.969 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:08 smithi132 ceph-mon[70676]: pgmap v1789: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:10.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:10 smithi088 ceph-mon[88643]: pgmap v1790: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:10.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:10 smithi132 ceph-mon[70676]: pgmap v1790: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:12.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:12 smithi088 ceph-mon[88643]: pgmap v1791: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:12.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:12 smithi132 ceph-mon[70676]: pgmap v1791: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:13.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:03:13.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:03:14.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:14 smithi088 ceph-mon[88643]: pgmap v1792: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:14.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:14 smithi132 ceph-mon[70676]: pgmap v1792: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:16.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:16 smithi088 ceph-mon[88643]: pgmap v1793: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:16.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:16 smithi132 ceph-mon[70676]: pgmap v1793: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:17.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:03:17.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:03:18.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:18 smithi088 ceph-mon[88643]: pgmap v1794: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:18.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:03:18.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:03:18.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:03:18.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:18 smithi132 ceph-mon[70676]: pgmap v1794: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:18.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:03:18.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:03:18.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:03:20.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:20 smithi088 ceph-mon[88643]: pgmap v1795: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:20 smithi132 ceph-mon[70676]: pgmap v1795: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:22.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:22 smithi088 ceph-mon[88643]: pgmap v1796: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:22.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:22 smithi132 ceph-mon[70676]: pgmap v1796: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:24.839 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:24 smithi088 ceph-mon[88643]: pgmap v1797: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:24.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:24 smithi132 ceph-mon[70676]: pgmap v1797: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:26.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:26 smithi132 ceph-mon[70676]: pgmap v1798: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:26 smithi088 ceph-mon[88643]: pgmap v1798: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:28.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:28 smithi088 ceph-mon[88643]: pgmap v1799: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:28.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:03:28.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:28 smithi132 ceph-mon[70676]: pgmap v1799: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:28.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:03:30.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:30 smithi132 ceph-mon[70676]: pgmap v1800: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:30 smithi088 ceph-mon[88643]: pgmap v1800: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:32.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:32 smithi132 ceph-mon[70676]: pgmap v1801: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:33.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:32 smithi088 ceph-mon[88643]: pgmap v1801: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:34.892 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:34 smithi088 ceph-mon[88643]: pgmap v1802: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:34 smithi132 ceph-mon[70676]: pgmap v1802: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:36.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:36 smithi132 ceph-mon[70676]: pgmap v1803: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:36 smithi088 ceph-mon[88643]: pgmap v1803: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:38.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:38 smithi132 ceph-mon[70676]: pgmap v1804: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:38 smithi088 ceph-mon[88643]: pgmap v1804: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:40.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:40 smithi132 ceph-mon[70676]: pgmap v1805: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:40 smithi088 ceph-mon[88643]: pgmap v1805: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:42.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:42 smithi132 ceph-mon[70676]: pgmap v1806: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:42 smithi088 ceph-mon[88643]: pgmap v1806: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:43.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:03:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:03:44.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:44 smithi132 ceph-mon[70676]: pgmap v1807: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:44 smithi088 ceph-mon[88643]: pgmap v1807: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:46.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:46 smithi132 ceph-mon[70676]: pgmap v1808: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:46 smithi088 ceph-mon[88643]: pgmap v1808: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:48.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:48 smithi132 ceph-mon[70676]: pgmap v1809: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:48 smithi088 ceph-mon[88643]: pgmap v1809: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:50.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:50 smithi132 ceph-mon[70676]: pgmap v1810: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:50 smithi088 ceph-mon[88643]: pgmap v1810: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:52.884 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:52 smithi088 ceph-mon[88643]: pgmap v1811: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:52.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:52 smithi132 ceph-mon[70676]: pgmap v1811: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:54.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:54 smithi132 ceph-mon[70676]: pgmap v1812: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:54 smithi088 ceph-mon[88643]: pgmap v1812: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:56.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:56 smithi132 ceph-mon[70676]: pgmap v1813: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:56 smithi088 ceph-mon[88643]: pgmap v1813: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:58 smithi132 ceph-mon[70676]: pgmap v1814: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:03:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:03:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:58 smithi088 ceph-mon[88643]: pgmap v1814: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:03:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:03:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:04:00.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:00 smithi132 ceph-mon[70676]: pgmap v1815: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:00 smithi088 ceph-mon[88643]: pgmap v1815: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:02.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:02 smithi132 ceph-mon[70676]: pgmap v1816: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:02 smithi088 ceph-mon[88643]: pgmap v1816: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:04 smithi132 ceph-mon[70676]: pgmap v1817: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:04 smithi088 ceph-mon[88643]: pgmap v1817: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:06.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:06 smithi132 ceph-mon[70676]: pgmap v1818: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:06 smithi088 ceph-mon[88643]: pgmap v1818: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:08.953 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:08 smithi088 ceph-mon[88643]: pgmap v1819: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:08.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:08 smithi132 ceph-mon[70676]: pgmap v1819: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:10.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:10 smithi132 ceph-mon[70676]: pgmap v1820: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:10 smithi088 ceph-mon[88643]: pgmap v1820: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:12.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:12 smithi132 ceph-mon[70676]: pgmap v1821: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:12 smithi088 ceph-mon[88643]: pgmap v1821: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:13.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:04:14.009 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:04:14.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:14 smithi132 ceph-mon[70676]: pgmap v1822: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:14 smithi088 ceph-mon[88643]: pgmap v1822: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:16.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:16 smithi132 ceph-mon[70676]: pgmap v1823: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:16 smithi088 ceph-mon[88643]: pgmap v1823: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:18.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:18 smithi132 ceph-mon[70676]: pgmap v1824: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:18.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:04:19.066 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:18 smithi088 ceph-mon[88643]: pgmap v1824: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:19.066 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:04:19.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:04:19.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:04:19.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:04:20.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:04:20.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:04:20.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:04:21.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:20 smithi132 ceph-mon[70676]: pgmap v1825: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:21.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:20 smithi088 ceph-mon[88643]: pgmap v1825: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:22 smithi132 ceph-mon[70676]: pgmap v1826: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:23.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:22 smithi088 ceph-mon[88643]: pgmap v1826: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:25.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:24 smithi132 ceph-mon[70676]: pgmap v1827: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:25.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:24 smithi088 ceph-mon[88643]: pgmap v1827: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:27.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:26 smithi132 ceph-mon[70676]: pgmap v1828: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:26 smithi088 ceph-mon[88643]: pgmap v1828: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:29.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:28 smithi088 ceph-mon[88643]: pgmap v1829: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:29.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:04:29.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:28 smithi132 ceph-mon[70676]: pgmap v1829: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:29.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:04:31.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:30 smithi088 ceph-mon[88643]: pgmap v1830: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:31.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:30 smithi132 ceph-mon[70676]: pgmap v1830: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:32 smithi088 ceph-mon[88643]: pgmap v1831: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:33.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:32 smithi132 ceph-mon[70676]: pgmap v1831: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:35.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:34 smithi132 ceph-mon[70676]: pgmap v1832: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:35.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:34 smithi088 ceph-mon[88643]: pgmap v1832: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:37.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:36 smithi088 ceph-mon[88643]: pgmap v1833: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:37.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:36 smithi132 ceph-mon[70676]: pgmap v1833: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:39.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:38 smithi132 ceph-mon[70676]: pgmap v1834: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:39.286 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:38 smithi088 ceph-mon[88643]: pgmap v1834: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:41 smithi088 ceph-mon[88643]: pgmap v1835: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:41.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:41 smithi132 ceph-mon[70676]: pgmap v1835: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:43.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:43 smithi088 ceph-mon[88643]: pgmap v1836: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:43.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:43 smithi132 ceph-mon[70676]: pgmap v1836: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:04:44.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:04:45.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:45 smithi132 ceph-mon[70676]: pgmap v1837: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:45.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:45 smithi088 ceph-mon[88643]: pgmap v1837: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:47.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:47 smithi088 ceph-mon[88643]: pgmap v1838: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:47.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:47 smithi132 ceph-mon[70676]: pgmap v1838: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:49.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:49 smithi088 ceph-mon[88643]: pgmap v1839: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:49.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:49 smithi132 ceph-mon[70676]: pgmap v1839: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:51.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:51 smithi088 ceph-mon[88643]: pgmap v1840: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:51.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:51 smithi132 ceph-mon[70676]: pgmap v1840: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:53.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:53 smithi088 ceph-mon[88643]: pgmap v1841: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:53.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:53 smithi132 ceph-mon[70676]: pgmap v1841: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:54.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:54 smithi088 ceph-mon[88643]: pgmap v1842: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:54.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:54 smithi132 ceph-mon[70676]: pgmap v1842: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:56.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:56 smithi088 ceph-mon[88643]: pgmap v1843: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:56.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:56 smithi132 ceph-mon[70676]: pgmap v1843: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:58.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:58 smithi088 ceph-mon[88643]: pgmap v1844: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:58.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:04:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:04:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:58 smithi132 ceph-mon[70676]: pgmap v1844: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:04:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:04:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:05:00.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:00 smithi088 ceph-mon[88643]: pgmap v1845: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:00.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:00 smithi132 ceph-mon[70676]: pgmap v1845: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:02.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:02 smithi088 ceph-mon[88643]: pgmap v1846: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:02.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:02 smithi132 ceph-mon[70676]: pgmap v1846: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:04.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:04 smithi088 ceph-mon[88643]: pgmap v1847: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:04.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:04 smithi132 ceph-mon[70676]: pgmap v1847: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:06.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:06 smithi088 ceph-mon[88643]: pgmap v1848: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:06.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:06 smithi132 ceph-mon[70676]: pgmap v1848: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:08.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:08 smithi088 ceph-mon[88643]: pgmap v1849: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:08 smithi132 ceph-mon[70676]: pgmap v1849: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:10.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:10 smithi088 ceph-mon[88643]: pgmap v1850: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:10.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:10 smithi132 ceph-mon[70676]: pgmap v1850: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:12.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:12 smithi088 ceph-mon[88643]: pgmap v1851: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:12.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:12 smithi132 ceph-mon[70676]: pgmap v1851: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:13.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:05:13.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:05:14.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:14 smithi088 ceph-mon[88643]: pgmap v1852: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:14.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:14 smithi132 ceph-mon[70676]: pgmap v1852: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:16.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:16 smithi088 ceph-mon[88643]: pgmap v1853: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:16.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:16 smithi132 ceph-mon[70676]: pgmap v1853: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:18.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:18 smithi088 ceph-mon[88643]: pgmap v1854: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:18.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:18 smithi132 ceph-mon[70676]: pgmap v1854: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:19.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:05:19.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:05:20.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:20 smithi088 ceph-mon[88643]: pgmap v1855: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:20.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:05:20.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:05:20.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:05:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:20 smithi132 ceph-mon[70676]: pgmap v1855: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:05:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:05:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:05:22.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:22 smithi088 ceph-mon[88643]: pgmap v1856: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:22.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:22 smithi132 ceph-mon[70676]: pgmap v1856: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:24.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:24 smithi088 ceph-mon[88643]: pgmap v1857: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:24.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:24 smithi132 ceph-mon[70676]: pgmap v1857: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:26.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:26 smithi088 ceph-mon[88643]: pgmap v1858: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:26.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:26 smithi132 ceph-mon[70676]: pgmap v1858: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:28.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:28 smithi088 ceph-mon[88643]: pgmap v1859: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:28.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:05:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:28 smithi132 ceph-mon[70676]: pgmap v1859: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:28.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:05:30.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:30 smithi132 ceph-mon[70676]: pgmap v1860: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:30 smithi088 ceph-mon[88643]: pgmap v1860: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:32.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:32 smithi132 ceph-mon[70676]: pgmap v1861: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:32 smithi088 ceph-mon[88643]: pgmap v1861: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:34.909 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:34 smithi088 ceph-mon[88643]: pgmap v1862: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:34 smithi132 ceph-mon[70676]: pgmap v1862: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:36.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:36 smithi132 ceph-mon[70676]: pgmap v1863: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:36 smithi088 ceph-mon[88643]: pgmap v1863: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:38.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:38 smithi132 ceph-mon[70676]: pgmap v1864: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:38 smithi088 ceph-mon[88643]: pgmap v1864: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:40.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:40 smithi132 ceph-mon[70676]: pgmap v1865: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:40 smithi088 ceph-mon[88643]: pgmap v1865: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:42.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:42 smithi132 ceph-mon[70676]: pgmap v1866: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:42 smithi088 ceph-mon[88643]: pgmap v1866: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:43.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:05:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:05:44.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:44 smithi132 ceph-mon[70676]: pgmap v1867: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:45.022 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:44 smithi088 ceph-mon[88643]: pgmap v1867: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:46.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:46 smithi132 ceph-mon[70676]: pgmap v1868: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:46 smithi088 ceph-mon[88643]: pgmap v1868: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:48.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:48 smithi132 ceph-mon[70676]: pgmap v1869: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:48 smithi088 ceph-mon[88643]: pgmap v1869: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:50.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:50 smithi132 ceph-mon[70676]: pgmap v1870: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:50 smithi088 ceph-mon[88643]: pgmap v1870: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:52.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:52 smithi132 ceph-mon[70676]: pgmap v1871: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:52 smithi088 ceph-mon[88643]: pgmap v1871: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:54.883 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:54 smithi088 ceph-mon[88643]: pgmap v1872: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:54.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:54 smithi132 ceph-mon[70676]: pgmap v1872: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:56.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:56 smithi132 ceph-mon[70676]: pgmap v1873: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:56 smithi088 ceph-mon[88643]: pgmap v1873: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:58 smithi132 ceph-mon[70676]: pgmap v1874: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:05:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:05:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:58 smithi088 ceph-mon[88643]: pgmap v1874: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:05:59.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:05:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:06:00.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:00 smithi132 ceph-mon[70676]: pgmap v1875: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:00 smithi088 ceph-mon[88643]: pgmap v1875: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:02.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:02 smithi132 ceph-mon[70676]: pgmap v1876: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:02 smithi088 ceph-mon[88643]: pgmap v1876: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:04 smithi132 ceph-mon[70676]: pgmap v1877: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:04 smithi088 ceph-mon[88643]: pgmap v1877: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:06.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:06 smithi132 ceph-mon[70676]: pgmap v1878: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:06 smithi088 ceph-mon[88643]: pgmap v1878: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:08 smithi132 ceph-mon[70676]: pgmap v1879: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:08 smithi088 ceph-mon[88643]: pgmap v1879: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:10.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:10 smithi132 ceph-mon[70676]: pgmap v1880: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:10 smithi088 ceph-mon[88643]: pgmap v1880: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:12.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:12 smithi132 ceph-mon[70676]: pgmap v1881: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:12 smithi088 ceph-mon[88643]: pgmap v1881: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:13.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:06:14.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:06:14.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:14 smithi132 ceph-mon[70676]: pgmap v1882: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:14 smithi088 ceph-mon[88643]: pgmap v1882: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:16.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:16 smithi132 ceph-mon[70676]: pgmap v1883: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:16 smithi088 ceph-mon[88643]: pgmap v1883: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:18.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:18 smithi132 ceph-mon[70676]: pgmap v1884: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:18 smithi088 ceph-mon[88643]: pgmap v1884: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:20 smithi132 ceph-mon[70676]: pgmap v1885: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:06:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:06:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:06:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:06:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:20 smithi088 ceph-mon[88643]: pgmap v1885: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:06:21.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:06:21.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:06:21.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:06:22.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:22 smithi132 ceph-mon[70676]: pgmap v1886: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:22 smithi088 ceph-mon[88643]: pgmap v1886: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:24.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:24 smithi132 ceph-mon[70676]: pgmap v1887: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:24 smithi088 ceph-mon[88643]: pgmap v1887: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:26.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:26 smithi132 ceph-mon[70676]: pgmap v1888: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:26 smithi088 ceph-mon[88643]: pgmap v1888: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:28.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:28 smithi132 ceph-mon[70676]: pgmap v1889: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:28.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:06:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:28 smithi088 ceph-mon[88643]: pgmap v1889: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:06:30.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:30 smithi088 ceph-mon[88643]: pgmap v1890: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:30.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:30 smithi132 ceph-mon[70676]: pgmap v1890: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:32.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:32 smithi132 ceph-mon[70676]: pgmap v1891: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:32 smithi088 ceph-mon[88643]: pgmap v1891: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:34.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:34 smithi132 ceph-mon[70676]: pgmap v1892: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:34 smithi088 ceph-mon[88643]: pgmap v1892: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:36.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:36 smithi132 ceph-mon[70676]: pgmap v1893: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:36 smithi088 ceph-mon[88643]: pgmap v1893: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:38 smithi132 ceph-mon[70676]: pgmap v1894: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:38 smithi088 ceph-mon[88643]: pgmap v1894: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:40.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:40 smithi132 ceph-mon[70676]: pgmap v1895: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:40 smithi088 ceph-mon[88643]: pgmap v1895: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:42.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:42 smithi132 ceph-mon[70676]: pgmap v1896: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:42 smithi088 ceph-mon[88643]: pgmap v1896: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:43.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:06:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:06:44.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:44 smithi132 ceph-mon[70676]: pgmap v1897: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:44 smithi088 ceph-mon[88643]: pgmap v1897: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:46.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:46 smithi132 ceph-mon[70676]: pgmap v1898: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:47.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:46 smithi088 ceph-mon[88643]: pgmap v1898: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:48.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:48 smithi132 ceph-mon[70676]: pgmap v1899: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:48 smithi088 ceph-mon[88643]: pgmap v1899: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:50.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:50 smithi132 ceph-mon[70676]: pgmap v1900: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:51.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:50 smithi088 ceph-mon[88643]: pgmap v1900: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:52.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:52 smithi132 ceph-mon[70676]: pgmap v1901: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:52 smithi088 ceph-mon[88643]: pgmap v1901: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:54.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:54 smithi132 ceph-mon[70676]: pgmap v1902: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:54 smithi088 ceph-mon[88643]: pgmap v1902: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:56 smithi088 ceph-mon[88643]: pgmap v1903: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:56 smithi132 ceph-mon[70676]: pgmap v1903: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:58 smithi088 ceph-mon[88643]: pgmap v1904: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:59.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:06:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:06:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:58 smithi132 ceph-mon[70676]: pgmap v1904: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:06:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:06:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:07:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:00 smithi088 ceph-mon[88643]: pgmap v1905: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:01.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:00 smithi132 ceph-mon[70676]: pgmap v1905: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:02 smithi088 ceph-mon[88643]: pgmap v1906: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:03.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:02 smithi132 ceph-mon[70676]: pgmap v1906: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:05.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:04 smithi088 ceph-mon[88643]: pgmap v1907: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:05.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:04 smithi132 ceph-mon[70676]: pgmap v1907: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:06 smithi088 ceph-mon[88643]: pgmap v1908: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:07.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:06 smithi132 ceph-mon[70676]: pgmap v1908: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:08 smithi088 ceph-mon[88643]: pgmap v1909: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:08 smithi132 ceph-mon[70676]: pgmap v1909: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:11.035 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:10 smithi132 ceph-mon[70676]: pgmap v1910: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:10 smithi088 ceph-mon[88643]: pgmap v1910: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:12 smithi088 ceph-mon[88643]: pgmap v1911: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:13.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:12 smithi132 ceph-mon[70676]: pgmap v1911: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:07:14.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:07:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:14 smithi088 ceph-mon[88643]: pgmap v1912: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:15.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:14 smithi132 ceph-mon[70676]: pgmap v1912: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:17.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:16 smithi088 ceph-mon[88643]: pgmap v1913: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:17.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:16 smithi132 ceph-mon[70676]: pgmap v1913: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:18 smithi088 ceph-mon[88643]: pgmap v1914: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:18 smithi132 ceph-mon[70676]: pgmap v1914: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:20 smithi088 ceph-mon[88643]: pgmap v1915: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:20 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:07:21.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:20 smithi132 ceph-mon[70676]: pgmap v1915: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:21.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:20 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:07:22.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:21 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:07:22.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:21 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:07:22.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:21 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:07:22.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:21 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:07:22.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:21 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:07:22.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:21 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:07:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:22 smithi088 ceph-mon[88643]: pgmap v1916: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:23.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:22 smithi132 ceph-mon[70676]: pgmap v1916: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:24 smithi088 ceph-mon[88643]: pgmap v1917: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:24 smithi132 ceph-mon[70676]: pgmap v1917: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:26 smithi088 ceph-mon[88643]: pgmap v1918: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:27.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:26 smithi132 ceph-mon[70676]: pgmap v1918: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:28 smithi088 ceph-mon[88643]: pgmap v1919: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:29.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:07:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:28 smithi132 ceph-mon[70676]: pgmap v1919: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:29.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:07:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:30 smithi088 ceph-mon[88643]: pgmap v1920: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:30 smithi132 ceph-mon[70676]: pgmap v1920: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:32 smithi088 ceph-mon[88643]: pgmap v1921: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:33.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:32 smithi132 ceph-mon[70676]: pgmap v1921: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:34 smithi088 ceph-mon[88643]: pgmap v1922: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:35.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:34 smithi132 ceph-mon[70676]: pgmap v1922: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:36 smithi132 ceph-mon[70676]: pgmap v1923: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:36 smithi088 ceph-mon[88643]: pgmap v1923: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:38 smithi088 ceph-mon[88643]: pgmap v1924: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:39.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:38 smithi132 ceph-mon[70676]: pgmap v1924: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:40 smithi088 ceph-mon[88643]: pgmap v1925: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:40 smithi132 ceph-mon[70676]: pgmap v1925: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:42 smithi088 ceph-mon[88643]: pgmap v1926: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:43.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:42 smithi132 ceph-mon[70676]: pgmap v1926: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:07:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:07:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:44 smithi088 ceph-mon[88643]: pgmap v1927: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:44 smithi132 ceph-mon[70676]: pgmap v1927: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:46.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:46 smithi132 ceph-mon[70676]: pgmap v1928: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:46 smithi088 ceph-mon[88643]: pgmap v1928: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:48 smithi088 ceph-mon[88643]: pgmap v1929: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:49.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:48 smithi132 ceph-mon[70676]: pgmap v1929: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:50 smithi088 ceph-mon[88643]: pgmap v1930: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:50 smithi132 ceph-mon[70676]: pgmap v1930: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:52 smithi088 ceph-mon[88643]: pgmap v1931: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:52 smithi132 ceph-mon[70676]: pgmap v1931: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:54 smithi088 ceph-mon[88643]: pgmap v1932: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:54 smithi132 ceph-mon[70676]: pgmap v1932: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:56 smithi132 ceph-mon[70676]: pgmap v1933: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:57.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:56 smithi088 ceph-mon[88643]: pgmap v1933: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:58 smithi132 ceph-mon[70676]: pgmap v1934: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:07:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:07:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:58 smithi088 ceph-mon[88643]: pgmap v1934: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:07:59.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:07:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:08:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:00 smithi132 ceph-mon[70676]: pgmap v1935: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:00 smithi088 ceph-mon[88643]: pgmap v1935: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:03.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:02 smithi132 ceph-mon[70676]: pgmap v1936: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:03.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:02 smithi088 ceph-mon[88643]: pgmap v1936: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:04 smithi088 ceph-mon[88643]: pgmap v1937: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:04 smithi132 ceph-mon[70676]: pgmap v1937: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:06 smithi088 ceph-mon[88643]: pgmap v1938: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:07.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:06 smithi132 ceph-mon[70676]: pgmap v1938: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:08 smithi132 ceph-mon[70676]: pgmap v1939: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:09.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:08 smithi088 ceph-mon[88643]: pgmap v1939: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:10 smithi132 ceph-mon[70676]: pgmap v1940: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:11.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:10 smithi088 ceph-mon[88643]: pgmap v1940: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:13.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:12 smithi132 ceph-mon[70676]: pgmap v1941: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:13.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:12 smithi088 ceph-mon[88643]: pgmap v1941: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:14.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:08:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:08:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:14 smithi088 ceph-mon[88643]: pgmap v1942: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:14 smithi132 ceph-mon[70676]: pgmap v1942: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:16 smithi088 ceph-mon[88643]: pgmap v1943: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:17.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:16 smithi132 ceph-mon[70676]: pgmap v1943: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:18 smithi132 ceph-mon[70676]: pgmap v1944: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:19.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:18 smithi088 ceph-mon[88643]: pgmap v1944: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:21.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:20 smithi132 ceph-mon[70676]: pgmap v1945: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:21.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:20 smithi088 ceph-mon[88643]: pgmap v1945: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:22.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:21 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:08:22.235 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:21 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:08:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:22 smithi132 ceph-mon[70676]: pgmap v1946: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:22 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:08:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:22 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:08:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:22 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:08:23.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:22 smithi088 ceph-mon[88643]: pgmap v1946: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:23.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:22 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:08:23.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:22 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:08:23.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:22 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:08:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:24 smithi088 ceph-mon[88643]: pgmap v1947: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:24 smithi132 ceph-mon[70676]: pgmap v1947: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:26 smithi088 ceph-mon[88643]: pgmap v1948: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:27.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:26 smithi132 ceph-mon[70676]: pgmap v1948: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:29.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:28 smithi132 ceph-mon[70676]: pgmap v1949: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:29.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:08:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:28 smithi088 ceph-mon[88643]: pgmap v1949: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:08:31.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:30 smithi132 ceph-mon[70676]: pgmap v1950: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:31.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:30 smithi088 ceph-mon[88643]: pgmap v1950: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:33.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:32 smithi132 ceph-mon[70676]: pgmap v1951: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:32 smithi088 ceph-mon[88643]: pgmap v1951: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:34 smithi088 ceph-mon[88643]: pgmap v1952: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:35.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:34 smithi132 ceph-mon[70676]: pgmap v1952: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:37.164 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:36 smithi132 ceph-mon[70676]: pgmap v1953: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:37.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:36 smithi088 ceph-mon[88643]: pgmap v1953: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:39.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:38 smithi132 ceph-mon[70676]: pgmap v1954: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:39.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:38 smithi088 ceph-mon[88643]: pgmap v1954: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:41.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:40 smithi132 ceph-mon[70676]: pgmap v1955: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:40 smithi088 ceph-mon[88643]: pgmap v1955: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:43.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:42 smithi132 ceph-mon[70676]: pgmap v1956: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:43.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:42 smithi088 ceph-mon[88643]: pgmap v1956: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:08:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:08:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:44 smithi088 ceph-mon[88643]: pgmap v1957: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:44 smithi132 ceph-mon[70676]: pgmap v1957: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:46 smithi132 ceph-mon[70676]: pgmap v1958: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:47.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:46 smithi088 ceph-mon[88643]: pgmap v1958: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:49.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:48 smithi132 ceph-mon[70676]: pgmap v1959: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:49.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:48 smithi088 ceph-mon[88643]: pgmap v1959: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:50 smithi132 ceph-mon[70676]: pgmap v1960: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:51.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:50 smithi088 ceph-mon[88643]: pgmap v1960: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:52 smithi132 ceph-mon[70676]: pgmap v1961: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:53.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:52 smithi088 ceph-mon[88643]: pgmap v1961: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:54 smithi088 ceph-mon[88643]: pgmap v1962: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:54 smithi132 ceph-mon[70676]: pgmap v1962: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:57.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:56 smithi132 ceph-mon[70676]: pgmap v1963: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:57.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:56 smithi088 ceph-mon[88643]: pgmap v1963: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:58 smithi132 ceph-mon[70676]: pgmap v1964: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:08:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:08:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:58 smithi088 ceph-mon[88643]: pgmap v1964: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:08:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:08:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:09:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:00 smithi132 ceph-mon[70676]: pgmap v1965: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:00 smithi088 ceph-mon[88643]: pgmap v1965: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:03.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:02 smithi132 ceph-mon[70676]: pgmap v1966: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:03.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:02 smithi088 ceph-mon[88643]: pgmap v1966: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:04 smithi132 ceph-mon[70676]: pgmap v1967: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:05.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:04 smithi088 ceph-mon[88643]: pgmap v1967: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:07.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:06 smithi132 ceph-mon[70676]: pgmap v1968: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:07.283 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:06 smithi088 ceph-mon[88643]: pgmap v1968: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:08 smithi132 ceph-mon[70676]: pgmap v1969: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:09.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:08 smithi088 ceph-mon[88643]: pgmap v1969: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:11.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:10 smithi132 ceph-mon[70676]: pgmap v1970: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:11.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:10 smithi088 ceph-mon[88643]: pgmap v1970: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:13.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:12 smithi132 ceph-mon[70676]: pgmap v1971: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:13.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:12 smithi088 ceph-mon[88643]: pgmap v1971: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:09:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:09:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:14 smithi132 ceph-mon[70676]: pgmap v1972: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:15.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:14 smithi088 ceph-mon[88643]: pgmap v1972: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:17.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:16 smithi132 ceph-mon[70676]: pgmap v1973: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:17.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:16 smithi088 ceph-mon[88643]: pgmap v1973: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:18 smithi132 ceph-mon[70676]: pgmap v1974: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:19.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:18 smithi088 ceph-mon[88643]: pgmap v1974: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:20 smithi132 ceph-mon[70676]: pgmap v1975: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:21.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:20 smithi088 ceph-mon[88643]: pgmap v1975: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:22 smithi132 ceph-mon[70676]: pgmap v1976: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:22 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:09:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:22 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:09:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:22 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:09:23.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:22 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:09:23.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:22 smithi088 ceph-mon[88643]: pgmap v1976: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:23.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:22 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:09:23.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:22 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:09:23.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:22 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:09:23.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:22 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:09:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:24 smithi132 ceph-mon[70676]: pgmap v1977: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:25.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:24 smithi088 ceph-mon[88643]: pgmap v1977: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:26 smithi088 ceph-mon[88643]: pgmap v1978: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:27.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:26 smithi132 ceph-mon[70676]: pgmap v1978: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:28 smithi088 ceph-mon[88643]: pgmap v1979: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:09:29.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:28 smithi132 ceph-mon[70676]: pgmap v1979: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:29.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:09:31.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:30 smithi088 ceph-mon[88643]: pgmap v1980: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:31.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:30 smithi132 ceph-mon[70676]: pgmap v1980: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:32 smithi088 ceph-mon[88643]: pgmap v1981: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:33.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:32 smithi132 ceph-mon[70676]: pgmap v1981: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:35.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:34 smithi088 ceph-mon[88643]: pgmap v1982: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:35.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:34 smithi132 ceph-mon[70676]: pgmap v1982: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:37.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:37 smithi088 ceph-mon[88643]: pgmap v1983: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:37.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:37 smithi132 ceph-mon[70676]: pgmap v1983: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:39.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:39 smithi088 ceph-mon[88643]: pgmap v1984: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:39.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:39 smithi132 ceph-mon[70676]: pgmap v1984: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:41.286 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:41 smithi132 ceph-mon[70676]: pgmap v1985: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:41 smithi088 ceph-mon[88643]: pgmap v1985: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:43.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:43 smithi132 ceph-mon[70676]: pgmap v1986: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:43.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:43 smithi088 ceph-mon[88643]: pgmap v1986: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:09:44.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:09:45.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:45 smithi088 ceph-mon[88643]: pgmap v1987: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:45.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:45 smithi132 ceph-mon[70676]: pgmap v1987: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:47.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:47 smithi088 ceph-mon[88643]: pgmap v1988: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:47.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:47 smithi132 ceph-mon[70676]: pgmap v1988: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:49.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:49 smithi088 ceph-mon[88643]: pgmap v1989: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:49.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:49 smithi132 ceph-mon[70676]: pgmap v1989: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:51.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:51 smithi088 ceph-mon[88643]: pgmap v1990: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:51.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:51 smithi132 ceph-mon[70676]: pgmap v1990: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:53 smithi088 ceph-mon[88643]: pgmap v1991: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:53.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:53 smithi132 ceph-mon[70676]: pgmap v1991: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:55.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:55 smithi088 ceph-mon[88643]: pgmap v1992: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:55.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:55 smithi132 ceph-mon[70676]: pgmap v1992: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:57.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:57 smithi088 ceph-mon[88643]: pgmap v1993: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:57.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:57 smithi132 ceph-mon[70676]: pgmap v1993: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:59 smithi088 ceph-mon[88643]: pgmap v1994: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:09:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:09:59.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:59 smithi132 ceph-mon[70676]: pgmap v1994: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:09:59.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:09:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:10:00.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:00 smithi088 ceph-mon[88643]: overall HEALTH_OK 2024-02-19T20:10:00.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:00 smithi132 ceph-mon[70676]: overall HEALTH_OK 2024-02-19T20:10:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:01 smithi088 ceph-mon[88643]: pgmap v1995: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:01.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:01 smithi132 ceph-mon[70676]: pgmap v1995: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:03.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:03 smithi088 ceph-mon[88643]: pgmap v1996: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:03.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:03 smithi132 ceph-mon[70676]: pgmap v1996: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:05.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:05 smithi088 ceph-mon[88643]: pgmap v1997: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:05.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:05 smithi132 ceph-mon[70676]: pgmap v1997: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:07.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:07 smithi088 ceph-mon[88643]: pgmap v1998: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:07.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:07 smithi132 ceph-mon[70676]: pgmap v1998: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:09.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:09 smithi088 ceph-mon[88643]: pgmap v1999: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:09.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:09 smithi132 ceph-mon[70676]: pgmap v1999: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:11.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:11 smithi088 ceph-mon[88643]: pgmap v2000: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:11.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:11 smithi132 ceph-mon[70676]: pgmap v2000: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:13.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:13 smithi088 ceph-mon[88643]: pgmap v2001: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:13.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:13 smithi132 ceph-mon[70676]: pgmap v2001: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:10:14.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:10:15.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:15 smithi088 ceph-mon[88643]: pgmap v2002: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:15.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:15 smithi132 ceph-mon[70676]: pgmap v2002: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:17.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:17 smithi088 ceph-mon[88643]: pgmap v2003: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:17.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:17 smithi132 ceph-mon[70676]: pgmap v2003: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:19.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:19 smithi132 ceph-mon[70676]: pgmap v2004: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:19.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:19 smithi088 ceph-mon[88643]: pgmap v2004: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:20.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:20 smithi132 ceph-mon[70676]: pgmap v2005: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:20.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:20 smithi088 ceph-mon[88643]: pgmap v2005: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:22.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:22 smithi132 ceph-mon[70676]: pgmap v2006: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:22 smithi088 ceph-mon[88643]: pgmap v2006: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:23.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:23 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:10:23.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:23 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:10:23.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:23 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:10:23.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:23 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:10:24.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:10:24.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:10:24.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:10:24.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:10:24.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:24 smithi132 ceph-mon[70676]: pgmap v2007: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:24 smithi088 ceph-mon[88643]: pgmap v2007: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:26.884 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:26 smithi088 ceph-mon[88643]: pgmap v2008: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:26.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:26 smithi132 ceph-mon[70676]: pgmap v2008: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:28 smithi132 ceph-mon[70676]: pgmap v2009: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:28.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:10:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:28 smithi088 ceph-mon[88643]: pgmap v2009: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:10:30.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:30 smithi132 ceph-mon[70676]: pgmap v2010: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:30 smithi088 ceph-mon[88643]: pgmap v2010: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:32.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:32 smithi132 ceph-mon[70676]: pgmap v2011: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:32 smithi088 ceph-mon[88643]: pgmap v2011: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:34 smithi132 ceph-mon[70676]: pgmap v2012: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:34 smithi088 ceph-mon[88643]: pgmap v2012: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:36.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:36 smithi132 ceph-mon[70676]: pgmap v2013: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:36 smithi088 ceph-mon[88643]: pgmap v2013: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:38 smithi132 ceph-mon[70676]: pgmap v2014: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:38 smithi088 ceph-mon[88643]: pgmap v2014: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:40.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:40 smithi132 ceph-mon[70676]: pgmap v2015: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:40 smithi088 ceph-mon[88643]: pgmap v2015: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:42.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:42 smithi132 ceph-mon[70676]: pgmap v2016: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:42 smithi088 ceph-mon[88643]: pgmap v2016: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:43.734 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:10:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:10:44.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:44 smithi132 ceph-mon[70676]: pgmap v2017: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:45.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:44 smithi088 ceph-mon[88643]: pgmap v2017: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:46.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:46 smithi132 ceph-mon[70676]: pgmap v2018: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:46 smithi088 ceph-mon[88643]: pgmap v2018: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:48.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:48 smithi132 ceph-mon[70676]: pgmap v2019: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:48.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:48 smithi088 ceph-mon[88643]: pgmap v2019: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:50.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:50 smithi132 ceph-mon[70676]: pgmap v2020: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:50 smithi088 ceph-mon[88643]: pgmap v2020: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:52.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:52 smithi132 ceph-mon[70676]: pgmap v2021: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:52 smithi088 ceph-mon[88643]: pgmap v2021: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:54.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:54 smithi132 ceph-mon[70676]: pgmap v2022: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:54 smithi088 ceph-mon[88643]: pgmap v2022: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:56.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:56 smithi132 ceph-mon[70676]: pgmap v2023: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:56 smithi088 ceph-mon[88643]: pgmap v2023: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:58.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:58 smithi088 ceph-mon[88643]: pgmap v2024: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:58.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:10:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:10:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:58 smithi132 ceph-mon[70676]: pgmap v2024: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:10:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:10:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:11:00.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:00 smithi132 ceph-mon[70676]: pgmap v2025: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:00 smithi088 ceph-mon[88643]: pgmap v2025: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:02.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:02 smithi132 ceph-mon[70676]: pgmap v2026: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:02 smithi088 ceph-mon[88643]: pgmap v2026: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:04 smithi132 ceph-mon[70676]: pgmap v2027: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:04 smithi088 ceph-mon[88643]: pgmap v2027: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:06.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:06 smithi132 ceph-mon[70676]: pgmap v2028: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:06 smithi088 ceph-mon[88643]: pgmap v2028: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:08.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:08 smithi132 ceph-mon[70676]: pgmap v2029: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:08 smithi088 ceph-mon[88643]: pgmap v2029: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:10.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:10 smithi132 ceph-mon[70676]: pgmap v2030: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:10 smithi088 ceph-mon[88643]: pgmap v2030: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:12.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:12 smithi132 ceph-mon[70676]: pgmap v2031: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:12 smithi088 ceph-mon[88643]: pgmap v2031: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:13.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:11:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:11:14.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:14 smithi132 ceph-mon[70676]: pgmap v2032: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:14 smithi088 ceph-mon[88643]: pgmap v2032: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:16.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:16 smithi132 ceph-mon[70676]: pgmap v2033: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:16 smithi088 ceph-mon[88643]: pgmap v2033: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:18.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:18 smithi132 ceph-mon[70676]: pgmap v2034: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:18 smithi088 ceph-mon[88643]: pgmap v2034: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:20 smithi132 ceph-mon[70676]: pgmap v2035: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:20 smithi088 ceph-mon[88643]: pgmap v2035: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:22.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:22 smithi132 ceph-mon[70676]: pgmap v2036: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:22 smithi088 ceph-mon[88643]: pgmap v2036: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:23.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:23 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:11:24.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:23 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:11:24.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:24 smithi132 ceph-mon[70676]: pgmap v2037: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:24.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:11:24.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:11:24.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:11:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:24 smithi088 ceph-mon[88643]: pgmap v2037: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:11:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:11:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:11:26.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:26 smithi132 ceph-mon[70676]: pgmap v2038: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:26 smithi088 ceph-mon[88643]: pgmap v2038: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:28 smithi132 ceph-mon[70676]: pgmap v2039: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:28.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:11:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:28 smithi088 ceph-mon[88643]: pgmap v2039: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:11:30.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:30 smithi132 ceph-mon[70676]: pgmap v2040: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:30 smithi088 ceph-mon[88643]: pgmap v2040: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:32.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:32 smithi132 ceph-mon[70676]: pgmap v2041: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:32 smithi088 ceph-mon[88643]: pgmap v2041: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:34 smithi132 ceph-mon[70676]: pgmap v2042: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:34 smithi088 ceph-mon[88643]: pgmap v2042: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:36 smithi088 ceph-mon[88643]: pgmap v2043: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:36 smithi132 ceph-mon[70676]: pgmap v2043: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:38 smithi088 ceph-mon[88643]: pgmap v2044: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:38 smithi132 ceph-mon[70676]: pgmap v2044: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:40 smithi088 ceph-mon[88643]: pgmap v2045: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:40 smithi132 ceph-mon[70676]: pgmap v2045: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:43.035 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:42 smithi132 ceph-mon[70676]: pgmap v2046: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:42 smithi088 ceph-mon[88643]: pgmap v2046: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:44.010 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:11:44.026 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:11:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:44 smithi088 ceph-mon[88643]: pgmap v2047: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:44 smithi132 ceph-mon[70676]: pgmap v2047: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:46 smithi088 ceph-mon[88643]: pgmap v2048: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:46 smithi132 ceph-mon[70676]: pgmap v2048: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:49.060 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:48 smithi132 ceph-mon[70676]: pgmap v2049: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:48 smithi088 ceph-mon[88643]: pgmap v2049: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:50 smithi088 ceph-mon[88643]: pgmap v2050: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:50 smithi132 ceph-mon[70676]: pgmap v2050: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:52 smithi088 ceph-mon[88643]: pgmap v2051: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:52 smithi132 ceph-mon[70676]: pgmap v2051: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:54 smithi088 ceph-mon[88643]: pgmap v2052: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:54 smithi132 ceph-mon[70676]: pgmap v2052: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:56 smithi088 ceph-mon[88643]: pgmap v2053: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:56 smithi132 ceph-mon[70676]: pgmap v2053: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:58 smithi088 ceph-mon[88643]: pgmap v2054: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:11:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:11:59.159 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:58 smithi132 ceph-mon[70676]: pgmap v2054: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:11:59.159 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:11:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:12:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:00 smithi088 ceph-mon[88643]: pgmap v2055: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:00 smithi132 ceph-mon[70676]: pgmap v2055: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:02 smithi088 ceph-mon[88643]: pgmap v2056: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:03.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:02 smithi132 ceph-mon[70676]: pgmap v2056: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:04 smithi088 ceph-mon[88643]: pgmap v2057: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:04 smithi132 ceph-mon[70676]: pgmap v2057: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:06 smithi088 ceph-mon[88643]: pgmap v2058: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:07.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:06 smithi132 ceph-mon[70676]: pgmap v2058: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:08 smithi088 ceph-mon[88643]: pgmap v2059: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:09.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:08 smithi132 ceph-mon[70676]: pgmap v2059: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:10 smithi088 ceph-mon[88643]: pgmap v2060: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:11.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:10 smithi132 ceph-mon[70676]: pgmap v2060: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:12 smithi088 ceph-mon[88643]: pgmap v2061: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:13.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:12 smithi132 ceph-mon[70676]: pgmap v2061: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:12:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:12:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:14 smithi088 ceph-mon[88643]: pgmap v2062: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:14 smithi132 ceph-mon[70676]: pgmap v2062: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:16 smithi088 ceph-mon[88643]: pgmap v2063: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:17.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:16 smithi132 ceph-mon[70676]: pgmap v2063: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:18 smithi088 ceph-mon[88643]: pgmap v2064: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:19.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:18 smithi132 ceph-mon[70676]: pgmap v2064: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:20 smithi088 ceph-mon[88643]: pgmap v2065: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:20 smithi132 ceph-mon[70676]: pgmap v2065: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:22 smithi088 ceph-mon[88643]: pgmap v2066: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:23.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:22 smithi132 ceph-mon[70676]: pgmap v2066: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:24 smithi088 ceph-mon[88643]: pgmap v2067: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:12:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:24 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:24 smithi132 ceph-mon[70676]: pgmap v2067: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:25.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:12:25.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:24 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:25.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:25.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:25.832 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:25 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:26.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:26.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:26.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:25 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:26 smithi088 ceph-mon[88643]: pgmap v2068: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:27.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:26 smithi132 ceph-mon[70676]: pgmap v2068: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:28.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:28.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:28.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:28.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:29 smithi088 ceph-mon[88643]: pgmap v2069: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:29.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:12:29.455 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:29 smithi132 ceph-mon[70676]: pgmap v2069: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:29.455 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:29.456 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:29.456 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:12:30.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:30 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:30.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:30 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:30.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:30 smithi132 ceph-mon[70676]: pgmap v2070: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:30.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:30 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:30.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:30 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:30.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:30 smithi088 ceph-mon[88643]: pgmap v2070: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:31.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:31.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:31 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:31.598 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:31.598 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:31 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:32.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:32 smithi132 ceph-mon[70676]: pgmap v2071: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:32.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:32 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:32.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:32 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:32 smithi088 ceph-mon[88643]: pgmap v2071: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:33.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:32 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:33.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:32 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:34 smithi132 ceph-mon[70676]: pgmap v2072: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:35.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:34 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:35.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:34 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:35.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:34 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:12:35.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:34 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:12:35.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:34 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:35.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:34 smithi088 ceph-mon[88643]: pgmap v2072: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:35.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:34 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:35.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:34 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:35.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:34 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:12:35.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:34 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:12:35.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:34 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:12:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:36 smithi132 ceph-mon[70676]: pgmap v2073: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:37.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:36 smithi088 ceph-mon[88643]: pgmap v2073: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:39.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:38 smithi088 ceph-mon[88643]: pgmap v2074: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:39.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:38 smithi132 ceph-mon[70676]: pgmap v2074: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:40 smithi088 ceph-mon[88643]: pgmap v2075: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:41.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:40 smithi132 ceph-mon[70676]: pgmap v2075: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:43.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:42 smithi088 ceph-mon[88643]: pgmap v2076: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:43.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:42 smithi132 ceph-mon[70676]: pgmap v2076: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:12:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:12:45.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:44 smithi088 ceph-mon[88643]: pgmap v2077: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:45.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:44 smithi132 ceph-mon[70676]: pgmap v2077: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:47.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:47 smithi088 ceph-mon[88643]: pgmap v2078: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:47.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:47 smithi132 ceph-mon[70676]: pgmap v2078: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:49.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:49 smithi088 ceph-mon[88643]: pgmap v2079: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:49.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:49 smithi132 ceph-mon[70676]: pgmap v2079: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:51.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:51 smithi088 ceph-mon[88643]: pgmap v2080: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:51.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:51 smithi132 ceph-mon[70676]: pgmap v2080: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:53.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:53 smithi088 ceph-mon[88643]: pgmap v2081: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:53.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:53 smithi132 ceph-mon[70676]: pgmap v2081: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:55.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:55 smithi088 ceph-mon[88643]: pgmap v2082: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:55.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:55 smithi132 ceph-mon[70676]: pgmap v2082: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:57.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:57 smithi088 ceph-mon[88643]: pgmap v2083: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:57.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:57 smithi132 ceph-mon[70676]: pgmap v2083: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:59 smithi088 ceph-mon[88643]: pgmap v2084: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:12:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:12:59.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:59 smithi132 ceph-mon[70676]: pgmap v2084: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:12:59.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:12:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:13:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:01 smithi088 ceph-mon[88643]: pgmap v2085: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:01.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:01 smithi132 ceph-mon[70676]: pgmap v2085: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:03.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:03 smithi088 ceph-mon[88643]: pgmap v2086: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:03.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:03 smithi132 ceph-mon[70676]: pgmap v2086: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:05 smithi132 ceph-mon[70676]: pgmap v2087: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:05.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:05 smithi088 ceph-mon[88643]: pgmap v2087: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:07.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:07 smithi088 ceph-mon[88643]: pgmap v2088: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:07.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:07 smithi132 ceph-mon[70676]: pgmap v2088: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:09.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:09 smithi088 ceph-mon[88643]: pgmap v2089: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:09.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:09 smithi132 ceph-mon[70676]: pgmap v2089: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:11.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:11 smithi088 ceph-mon[88643]: pgmap v2090: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:11.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:11 smithi132 ceph-mon[70676]: pgmap v2090: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:13.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:13 smithi088 ceph-mon[88643]: pgmap v2091: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:13.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:13 smithi132 ceph-mon[70676]: pgmap v2091: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:13:14.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:13:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:15 smithi132 ceph-mon[70676]: pgmap v2092: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:15.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:15 smithi088 ceph-mon[88643]: pgmap v2092: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:17.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:17 smithi088 ceph-mon[88643]: pgmap v2093: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:17.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:17 smithi132 ceph-mon[70676]: pgmap v2093: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:19.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:19 smithi088 ceph-mon[88643]: pgmap v2094: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:19.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:19 smithi132 ceph-mon[70676]: pgmap v2094: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:21.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:21 smithi088 ceph-mon[88643]: pgmap v2095: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:21.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:21 smithi132 ceph-mon[70676]: pgmap v2095: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:23.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:23 smithi088 ceph-mon[88643]: pgmap v2096: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:23.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:23 smithi132 ceph-mon[70676]: pgmap v2096: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:25.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:25 smithi088 ceph-mon[88643]: pgmap v2097: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:25.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:25 smithi132 ceph-mon[70676]: pgmap v2097: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:27 smithi088 ceph-mon[88643]: pgmap v2098: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:27.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:27 smithi132 ceph-mon[70676]: pgmap v2098: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:29 smithi088 ceph-mon[88643]: pgmap v2099: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:29.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:13:29.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:29 smithi132 ceph-mon[70676]: pgmap v2099: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:29.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:13:31.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:31 smithi088 ceph-mon[88643]: pgmap v2100: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:31.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:31 smithi132 ceph-mon[70676]: pgmap v2100: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:33 smithi088 ceph-mon[88643]: pgmap v2101: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:33.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:33 smithi132 ceph-mon[70676]: pgmap v2101: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:35.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:35 smithi132 ceph-mon[70676]: pgmap v2102: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:35.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:13:35.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:13:35.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:13:35.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:13:35.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:35 smithi088 ceph-mon[88643]: pgmap v2102: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:35.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:13:35.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:13:35.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:13:35.582 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:13:37.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:37 smithi132 ceph-mon[70676]: pgmap v2103: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:37.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:37 smithi088 ceph-mon[88643]: pgmap v2103: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:39.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:39 smithi132 ceph-mon[70676]: pgmap v2104: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:39.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:39 smithi088 ceph-mon[88643]: pgmap v2104: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:41.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:41 smithi132 ceph-mon[70676]: pgmap v2105: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:41.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:41 smithi088 ceph-mon[88643]: pgmap v2105: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:43.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:43 smithi132 ceph-mon[70676]: pgmap v2106: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:43.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:43 smithi088 ceph-mon[88643]: pgmap v2106: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:44.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:13:44.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:13:45.383 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:45 smithi088 ceph-mon[88643]: pgmap v2107: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:45.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:45 smithi132 ceph-mon[70676]: pgmap v2107: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:47.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:47 smithi132 ceph-mon[70676]: pgmap v2108: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:47.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:47 smithi088 ceph-mon[88643]: pgmap v2108: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:49.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:49 smithi132 ceph-mon[70676]: pgmap v2109: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:49.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:49 smithi088 ceph-mon[88643]: pgmap v2109: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:51.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:51 smithi132 ceph-mon[70676]: pgmap v2110: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:51.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:51 smithi088 ceph-mon[88643]: pgmap v2110: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:53.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:53 smithi132 ceph-mon[70676]: pgmap v2111: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:53.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:53 smithi088 ceph-mon[88643]: pgmap v2111: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:54.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:54 smithi132 ceph-mon[70676]: pgmap v2112: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:54.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:54 smithi088 ceph-mon[88643]: pgmap v2112: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:56.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:56 smithi132 ceph-mon[70676]: pgmap v2113: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:56 smithi088 ceph-mon[88643]: pgmap v2113: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:58.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:58 smithi132 ceph-mon[70676]: pgmap v2114: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:58.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:13:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:13:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:58 smithi088 ceph-mon[88643]: pgmap v2114: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:13:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:13:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:14:00.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:00 smithi088 ceph-mon[88643]: pgmap v2115: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:00.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:00 smithi132 ceph-mon[70676]: pgmap v2115: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:02.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:02 smithi132 ceph-mon[70676]: pgmap v2116: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:02 smithi088 ceph-mon[88643]: pgmap v2116: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:04 smithi132 ceph-mon[70676]: pgmap v2117: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:04 smithi088 ceph-mon[88643]: pgmap v2117: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:06.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:06 smithi132 ceph-mon[70676]: pgmap v2118: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:06 smithi088 ceph-mon[88643]: pgmap v2118: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:08.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:08 smithi132 ceph-mon[70676]: pgmap v2119: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:08 smithi088 ceph-mon[88643]: pgmap v2119: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:10.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:10 smithi132 ceph-mon[70676]: pgmap v2120: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:10 smithi088 ceph-mon[88643]: pgmap v2120: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:12.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:12 smithi132 ceph-mon[70676]: pgmap v2121: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:12 smithi088 ceph-mon[88643]: pgmap v2121: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:13.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:14:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:14:14.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:14 smithi132 ceph-mon[70676]: pgmap v2122: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:14 smithi088 ceph-mon[88643]: pgmap v2122: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:16.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:16 smithi132 ceph-mon[70676]: pgmap v2123: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:16 smithi088 ceph-mon[88643]: pgmap v2123: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:18.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:18 smithi132 ceph-mon[70676]: pgmap v2124: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:18 smithi088 ceph-mon[88643]: pgmap v2124: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:20.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:20 smithi132 ceph-mon[70676]: pgmap v2125: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:20 smithi088 ceph-mon[88643]: pgmap v2125: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:22.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:22 smithi132 ceph-mon[70676]: pgmap v2126: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:22 smithi088 ceph-mon[88643]: pgmap v2126: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:24.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:24 smithi132 ceph-mon[70676]: pgmap v2127: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:24 smithi088 ceph-mon[88643]: pgmap v2127: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:26.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:26 smithi132 ceph-mon[70676]: pgmap v2128: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:26 smithi088 ceph-mon[88643]: pgmap v2128: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:28 smithi132 ceph-mon[70676]: pgmap v2129: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:28.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:14:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:28 smithi088 ceph-mon[88643]: pgmap v2129: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:14:30.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:30 smithi132 ceph-mon[70676]: pgmap v2130: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:30 smithi088 ceph-mon[88643]: pgmap v2130: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:32.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:32 smithi132 ceph-mon[70676]: pgmap v2131: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:32 smithi088 ceph-mon[88643]: pgmap v2131: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:34.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:34 smithi132 ceph-mon[70676]: pgmap v2132: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:34 smithi088 ceph-mon[88643]: pgmap v2132: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:35.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:14:35.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:14:35.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:14:35.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:14:36.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:14:36.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:14:36.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:14:36.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:14:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:36 smithi088 ceph-mon[88643]: pgmap v2133: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:36 smithi132 ceph-mon[70676]: pgmap v2133: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:38 smithi088 ceph-mon[88643]: pgmap v2134: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:38 smithi132 ceph-mon[70676]: pgmap v2134: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:41.000 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:40 smithi088 ceph-mon[88643]: pgmap v2135: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:40 smithi132 ceph-mon[70676]: pgmap v2135: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:43.035 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:42 smithi132 ceph-mon[70676]: pgmap v2136: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:42 smithi088 ceph-mon[88643]: pgmap v2136: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:14:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:14:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:44 smithi088 ceph-mon[88643]: pgmap v2137: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:44 smithi132 ceph-mon[70676]: pgmap v2137: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:46 smithi088 ceph-mon[88643]: pgmap v2138: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:46 smithi132 ceph-mon[70676]: pgmap v2138: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:48 smithi088 ceph-mon[88643]: pgmap v2139: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:49.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:48 smithi132 ceph-mon[70676]: pgmap v2139: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:50 smithi088 ceph-mon[88643]: pgmap v2140: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:50 smithi132 ceph-mon[70676]: pgmap v2140: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:52 smithi088 ceph-mon[88643]: pgmap v2141: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:52 smithi132 ceph-mon[70676]: pgmap v2141: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:54 smithi088 ceph-mon[88643]: pgmap v2142: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:54 smithi132 ceph-mon[70676]: pgmap v2142: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:56.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:56 smithi132 ceph-mon[70676]: pgmap v2143: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:56 smithi088 ceph-mon[88643]: pgmap v2143: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:58.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:58 smithi132 ceph-mon[70676]: pgmap v2144: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:58.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:14:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:14:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:58 smithi088 ceph-mon[88643]: pgmap v2144: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:14:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:14:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:15:00.940 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:00 smithi132 ceph-mon[70676]: pgmap v2145: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:00 smithi088 ceph-mon[88643]: pgmap v2145: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:02.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:02 smithi132 ceph-mon[70676]: pgmap v2146: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:02 smithi088 ceph-mon[88643]: pgmap v2146: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:04.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:04 smithi132 ceph-mon[70676]: pgmap v2147: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:04 smithi088 ceph-mon[88643]: pgmap v2147: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:06.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:06 smithi132 ceph-mon[70676]: pgmap v2148: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:06 smithi088 ceph-mon[88643]: pgmap v2148: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:08.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:08 smithi132 ceph-mon[70676]: pgmap v2149: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:08 smithi088 ceph-mon[88643]: pgmap v2149: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:10.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:10 smithi132 ceph-mon[70676]: pgmap v2150: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:10 smithi088 ceph-mon[88643]: pgmap v2150: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:12.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:12 smithi132 ceph-mon[70676]: pgmap v2151: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:12 smithi088 ceph-mon[88643]: pgmap v2151: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:13.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:15:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:15:14.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:14 smithi132 ceph-mon[70676]: pgmap v2152: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:15.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:14 smithi088 ceph-mon[88643]: pgmap v2152: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:16.984 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:16 smithi132 ceph-mon[70676]: pgmap v2153: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:16 smithi088 ceph-mon[88643]: pgmap v2153: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:18.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:18 smithi132 ceph-mon[70676]: pgmap v2154: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:18 smithi088 ceph-mon[88643]: pgmap v2154: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:20.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:20 smithi132 ceph-mon[70676]: pgmap v2155: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:20 smithi088 ceph-mon[88643]: pgmap v2155: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:22.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:22 smithi132 ceph-mon[70676]: pgmap v2156: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:22 smithi088 ceph-mon[88643]: pgmap v2156: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:24.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:24 smithi132 ceph-mon[70676]: pgmap v2157: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:24 smithi088 ceph-mon[88643]: pgmap v2157: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:26.831 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:26 smithi088 ceph-mon[88643]: pgmap v2158: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:26.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:26 smithi132 ceph-mon[70676]: pgmap v2158: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:28 smithi132 ceph-mon[70676]: pgmap v2159: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:15:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:28 smithi088 ceph-mon[88643]: pgmap v2159: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:15:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:30 smithi088 ceph-mon[88643]: pgmap v2160: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:30 smithi132 ceph-mon[70676]: pgmap v2160: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:32 smithi088 ceph-mon[88643]: pgmap v2161: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:33.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:32 smithi132 ceph-mon[70676]: pgmap v2161: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:34 smithi088 ceph-mon[88643]: pgmap v2162: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:34 smithi132 ceph-mon[70676]: pgmap v2162: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:36.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:35 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:15:36.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:35 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:15:37.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:36 smithi088 ceph-mon[88643]: pgmap v2163: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:37.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:36 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:15:37.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:36 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:15:37.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:36 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:15:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:36 smithi132 ceph-mon[70676]: pgmap v2163: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:37.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:36 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:15:37.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:36 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:15:37.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:36 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:15:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:38 smithi088 ceph-mon[88643]: pgmap v2164: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:38 smithi132 ceph-mon[70676]: pgmap v2164: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:40 smithi088 ceph-mon[88643]: pgmap v2165: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:40 smithi132 ceph-mon[70676]: pgmap v2165: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:42 smithi088 ceph-mon[88643]: pgmap v2166: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:43.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:42 smithi132 ceph-mon[70676]: pgmap v2166: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:15:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:15:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:44 smithi088 ceph-mon[88643]: pgmap v2167: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:44 smithi132 ceph-mon[70676]: pgmap v2167: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:46 smithi088 ceph-mon[88643]: pgmap v2168: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:46 smithi132 ceph-mon[70676]: pgmap v2168: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:48 smithi088 ceph-mon[88643]: pgmap v2169: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:49.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:48 smithi132 ceph-mon[70676]: pgmap v2169: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:50 smithi088 ceph-mon[88643]: pgmap v2170: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:50 smithi132 ceph-mon[70676]: pgmap v2170: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:52 smithi088 ceph-mon[88643]: pgmap v2171: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:52 smithi132 ceph-mon[70676]: pgmap v2171: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:54 smithi088 ceph-mon[88643]: pgmap v2172: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:54 smithi132 ceph-mon[70676]: pgmap v2172: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:56 smithi088 ceph-mon[88643]: pgmap v2173: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:56 smithi132 ceph-mon[70676]: pgmap v2173: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:58 smithi088 ceph-mon[88643]: pgmap v2174: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:15:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:15:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:58 smithi132 ceph-mon[70676]: pgmap v2174: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:15:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:15:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:16:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:00 smithi088 ceph-mon[88643]: pgmap v2175: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:00 smithi132 ceph-mon[70676]: pgmap v2175: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:02 smithi088 ceph-mon[88643]: pgmap v2176: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:03.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:02 smithi132 ceph-mon[70676]: pgmap v2176: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:04 smithi088 ceph-mon[88643]: pgmap v2177: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:04 smithi132 ceph-mon[70676]: pgmap v2177: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:06.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:06 smithi132 ceph-mon[70676]: pgmap v2178: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:06 smithi088 ceph-mon[88643]: pgmap v2178: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:08 smithi088 ceph-mon[88643]: pgmap v2179: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:09.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:08 smithi132 ceph-mon[70676]: pgmap v2179: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:10 smithi088 ceph-mon[88643]: pgmap v2180: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:11.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:10 smithi132 ceph-mon[70676]: pgmap v2180: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:12 smithi088 ceph-mon[88643]: pgmap v2181: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:13.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:12 smithi132 ceph-mon[70676]: pgmap v2181: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:16:14.134 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:16:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:14 smithi088 ceph-mon[88643]: pgmap v2182: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:14 smithi132 ceph-mon[70676]: pgmap v2182: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:16.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:16 smithi132 ceph-mon[70676]: pgmap v2183: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:17.074 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:16 smithi088 ceph-mon[88643]: pgmap v2183: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:18 smithi088 ceph-mon[88643]: pgmap v2184: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:19.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:18 smithi132 ceph-mon[70676]: pgmap v2184: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:20 smithi088 ceph-mon[88643]: pgmap v2185: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:20 smithi132 ceph-mon[70676]: pgmap v2185: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:22 smithi088 ceph-mon[88643]: pgmap v2186: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:23.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:22 smithi132 ceph-mon[70676]: pgmap v2186: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:24 smithi088 ceph-mon[88643]: pgmap v2187: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:24 smithi132 ceph-mon[70676]: pgmap v2187: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:27.187 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:26 smithi088 ceph-mon[88643]: pgmap v2188: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:27.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:26 smithi132 ceph-mon[70676]: pgmap v2188: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:28 smithi132 ceph-mon[70676]: pgmap v2189: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:16:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:28 smithi088 ceph-mon[88643]: pgmap v2189: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:16:31.134 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:30 smithi088 ceph-mon[88643]: pgmap v2190: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:30 smithi132 ceph-mon[70676]: pgmap v2190: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:33.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:32 smithi132 ceph-mon[70676]: pgmap v2191: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:32 smithi088 ceph-mon[88643]: pgmap v2191: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:34 smithi088 ceph-mon[88643]: pgmap v2192: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:34 smithi132 ceph-mon[70676]: pgmap v2192: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:36 smithi132 ceph-mon[70676]: pgmap v2193: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:37.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:36 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:16:37.300 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:36 smithi088 ceph-mon[88643]: pgmap v2193: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:37.300 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:36 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:16:38.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:37 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:16:38.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:37 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:16:38.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:37 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:16:38.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:37 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:16:38.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:37 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:16:38.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:37 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:16:39.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:39 smithi088 ceph-mon[88643]: pgmap v2194: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:39.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:39 smithi132 ceph-mon[70676]: pgmap v2194: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:41 smithi088 ceph-mon[88643]: pgmap v2195: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:41.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:41 smithi132 ceph-mon[70676]: pgmap v2195: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:43.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:43 smithi088 ceph-mon[88643]: pgmap v2196: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:43.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:43 smithi132 ceph-mon[70676]: pgmap v2196: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:16:44.385 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:16:45.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:45 smithi088 ceph-mon[88643]: pgmap v2197: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:45.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:45 smithi132 ceph-mon[70676]: pgmap v2197: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:47.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:47 smithi132 ceph-mon[70676]: pgmap v2198: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:47.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:47 smithi088 ceph-mon[88643]: pgmap v2198: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:49.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:49 smithi088 ceph-mon[88643]: pgmap v2199: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:49.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:49 smithi132 ceph-mon[70676]: pgmap v2199: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:51.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:51 smithi088 ceph-mon[88643]: pgmap v2200: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:51.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:51 smithi132 ceph-mon[70676]: pgmap v2200: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:53.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:53 smithi088 ceph-mon[88643]: pgmap v2201: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:53.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:53 smithi132 ceph-mon[70676]: pgmap v2201: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:55.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:55 smithi088 ceph-mon[88643]: pgmap v2202: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:55.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:55 smithi132 ceph-mon[70676]: pgmap v2202: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:57.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:57 smithi088 ceph-mon[88643]: pgmap v2203: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:57.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:57 smithi132 ceph-mon[70676]: pgmap v2203: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:59 smithi088 ceph-mon[88643]: pgmap v2204: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:16:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:16:59.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:59 smithi132 ceph-mon[70676]: pgmap v2204: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:16:59.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:16:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:17:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:01 smithi088 ceph-mon[88643]: pgmap v2205: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:01.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:01 smithi132 ceph-mon[70676]: pgmap v2205: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:03.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:03 smithi088 ceph-mon[88643]: pgmap v2206: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:03.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:03 smithi132 ceph-mon[70676]: pgmap v2206: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:05.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:05 smithi088 ceph-mon[88643]: pgmap v2207: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:05.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:05 smithi132 ceph-mon[70676]: pgmap v2207: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:07.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:07 smithi088 ceph-mon[88643]: pgmap v2208: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:07.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:07 smithi132 ceph-mon[70676]: pgmap v2208: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:09.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:09 smithi088 ceph-mon[88643]: pgmap v2209: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:09.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:09 smithi132 ceph-mon[70676]: pgmap v2209: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:11.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:11 smithi132 ceph-mon[70676]: pgmap v2210: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:11.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:11 smithi088 ceph-mon[88643]: pgmap v2210: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:13.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:13 smithi132 ceph-mon[70676]: pgmap v2211: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:13.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:13 smithi088 ceph-mon[88643]: pgmap v2211: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:14.384 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:17:14.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:17:15.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:15 smithi132 ceph-mon[70676]: pgmap v2212: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:15.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:15 smithi088 ceph-mon[88643]: pgmap v2212: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:17.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:17 smithi132 ceph-mon[70676]: pgmap v2213: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:17.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:17 smithi088 ceph-mon[88643]: pgmap v2213: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:19.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:19 smithi132 ceph-mon[70676]: pgmap v2214: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:19.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:19 smithi088 ceph-mon[88643]: pgmap v2214: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:21.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:21 smithi132 ceph-mon[70676]: pgmap v2215: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:21.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:21 smithi088 ceph-mon[88643]: pgmap v2215: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:23.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:23 smithi132 ceph-mon[70676]: pgmap v2216: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:23.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:23 smithi088 ceph-mon[88643]: pgmap v2216: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:25.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:25 smithi132 ceph-mon[70676]: pgmap v2217: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:25.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:25 smithi088 ceph-mon[88643]: pgmap v2217: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:27.380 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:27 smithi132 ceph-mon[70676]: pgmap v2218: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:27.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:27 smithi088 ceph-mon[88643]: pgmap v2218: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:29.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:29 smithi132 ceph-mon[70676]: pgmap v2219: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:29.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:17:29.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:29 smithi088 ceph-mon[88643]: pgmap v2219: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:29.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:17:31.383 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:31 smithi088 ceph-mon[88643]: pgmap v2220: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:31.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:31 smithi132 ceph-mon[70676]: pgmap v2220: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:33 smithi088 ceph-mon[88643]: pgmap v2221: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:33.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:33 smithi132 ceph-mon[70676]: pgmap v2221: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:35.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:35 smithi132 ceph-mon[70676]: pgmap v2222: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:35.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:35 smithi088 ceph-mon[88643]: pgmap v2222: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:37.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:37 smithi132 ceph-mon[70676]: pgmap v2223: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:37.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:37 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:17:37.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:37 smithi088 ceph-mon[88643]: pgmap v2223: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:37.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:37 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:17:38.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:38 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:17:38.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:38 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:17:38.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:38 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:17:38.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:38 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:17:38.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:38 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:17:38.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:38 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:17:39.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:39 smithi132 ceph-mon[70676]: pgmap v2224: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:39.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:39 smithi088 ceph-mon[88643]: pgmap v2224: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:41.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:41 smithi132 ceph-mon[70676]: pgmap v2225: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:41.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:41 smithi088 ceph-mon[88643]: pgmap v2225: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:43.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:43 smithi088 ceph-mon[88643]: pgmap v2226: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:43.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:43 smithi132 ceph-mon[70676]: pgmap v2226: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:44.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:17:44.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:17:45.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:45 smithi132 ceph-mon[70676]: pgmap v2227: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:45.584 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:45 smithi088 ceph-mon[88643]: pgmap v2227: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:47.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:47 smithi132 ceph-mon[70676]: pgmap v2228: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:47.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:47 smithi088 ceph-mon[88643]: pgmap v2228: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:49.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:49 smithi132 ceph-mon[70676]: pgmap v2229: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:49.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:49 smithi088 ceph-mon[88643]: pgmap v2229: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:51.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:51 smithi132 ceph-mon[70676]: pgmap v2230: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:51.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:51 smithi088 ceph-mon[88643]: pgmap v2230: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:52.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:52 smithi132 ceph-mon[70676]: pgmap v2231: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:52.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:52 smithi088 ceph-mon[88643]: pgmap v2231: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:54.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:54 smithi132 ceph-mon[70676]: pgmap v2232: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:54 smithi088 ceph-mon[88643]: pgmap v2232: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:56.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:56 smithi132 ceph-mon[70676]: pgmap v2233: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:56 smithi088 ceph-mon[88643]: pgmap v2233: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:58 smithi088 ceph-mon[88643]: pgmap v2234: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:17:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:17:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:58 smithi132 ceph-mon[70676]: pgmap v2234: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:17:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:17:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:18:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:00 smithi088 ceph-mon[88643]: pgmap v2235: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:00 smithi132 ceph-mon[70676]: pgmap v2235: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:02.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:02 smithi132 ceph-mon[70676]: pgmap v2236: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:02 smithi088 ceph-mon[88643]: pgmap v2236: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:04 smithi088 ceph-mon[88643]: pgmap v2237: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:04 smithi132 ceph-mon[70676]: pgmap v2237: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:06 smithi088 ceph-mon[88643]: pgmap v2238: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:07.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:06 smithi132 ceph-mon[70676]: pgmap v2238: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:08 smithi088 ceph-mon[88643]: pgmap v2239: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:09.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:08 smithi132 ceph-mon[70676]: pgmap v2239: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:11.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:10 smithi088 ceph-mon[88643]: pgmap v2240: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:11.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:10 smithi132 ceph-mon[70676]: pgmap v2240: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:12 smithi088 ceph-mon[88643]: pgmap v2241: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:13.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:12 smithi132 ceph-mon[70676]: pgmap v2241: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:18:14.135 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:18:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:14 smithi088 ceph-mon[88643]: pgmap v2242: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:14 smithi132 ceph-mon[70676]: pgmap v2242: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:16 smithi088 ceph-mon[88643]: pgmap v2243: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:17.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:16 smithi132 ceph-mon[70676]: pgmap v2243: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:18 smithi088 ceph-mon[88643]: pgmap v2244: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:19.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:18 smithi132 ceph-mon[70676]: pgmap v2244: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:20 smithi088 ceph-mon[88643]: pgmap v2245: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:20 smithi132 ceph-mon[70676]: pgmap v2245: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:22 smithi088 ceph-mon[88643]: pgmap v2246: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:23.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:22 smithi132 ceph-mon[70676]: pgmap v2246: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:25.083 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:24 smithi088 ceph-mon[88643]: pgmap v2247: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:24 smithi132 ceph-mon[70676]: pgmap v2247: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:26 smithi088 ceph-mon[88643]: pgmap v2248: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:27.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:26 smithi132 ceph-mon[70676]: pgmap v2248: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:28 smithi088 ceph-mon[88643]: pgmap v2249: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:18:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:28 smithi132 ceph-mon[70676]: pgmap v2249: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:29.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:18:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:30 smithi088 ceph-mon[88643]: pgmap v2250: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:30 smithi132 ceph-mon[70676]: pgmap v2250: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:33.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:32 smithi088 ceph-mon[88643]: pgmap v2251: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:33.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:32 smithi132 ceph-mon[70676]: pgmap v2251: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:34 smithi088 ceph-mon[88643]: pgmap v2252: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:34 smithi132 ceph-mon[70676]: pgmap v2252: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:36 smithi132 ceph-mon[70676]: pgmap v2253: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:36 smithi088 ceph-mon[88643]: pgmap v2253: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:38 smithi088 ceph-mon[88643]: pgmap v2254: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:38 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:18:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:38 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:18:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:38 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:18:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:38 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:18:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:38 smithi132 ceph-mon[70676]: pgmap v2254: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:38 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:18:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:38 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:18:39.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:38 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:18:39.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:38 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:18:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:40 smithi088 ceph-mon[88643]: pgmap v2255: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:41.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:40 smithi132 ceph-mon[70676]: pgmap v2255: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:43.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:42 smithi088 ceph-mon[88643]: pgmap v2256: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:43.133 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:42 smithi132 ceph-mon[70676]: pgmap v2256: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:44.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:18:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:18:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:44 smithi088 ceph-mon[88643]: pgmap v2257: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:44 smithi132 ceph-mon[70676]: pgmap v2257: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:47.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:46 smithi088 ceph-mon[88643]: pgmap v2258: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:46 smithi132 ceph-mon[70676]: pgmap v2258: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:49.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:48 smithi088 ceph-mon[88643]: pgmap v2259: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:49.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:48 smithi132 ceph-mon[70676]: pgmap v2259: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:51.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:50 smithi088 ceph-mon[88643]: pgmap v2260: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:50 smithi132 ceph-mon[70676]: pgmap v2260: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:53.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:52 smithi088 ceph-mon[88643]: pgmap v2261: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:52 smithi132 ceph-mon[70676]: pgmap v2261: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:55.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:54 smithi088 ceph-mon[88643]: pgmap v2262: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:54 smithi132 ceph-mon[70676]: pgmap v2262: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:57.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:56 smithi088 ceph-mon[88643]: pgmap v2263: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:56 smithi132 ceph-mon[70676]: pgmap v2263: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:59.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:58 smithi088 ceph-mon[88643]: pgmap v2264: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:59.082 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:18:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:18:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:58 smithi132 ceph-mon[70676]: pgmap v2264: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:18:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:18:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:19:01.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:00 smithi088 ceph-mon[88643]: pgmap v2265: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:00 smithi132 ceph-mon[70676]: pgmap v2265: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:03.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:02 smithi088 ceph-mon[88643]: pgmap v2266: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:03.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:02 smithi132 ceph-mon[70676]: pgmap v2266: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:05.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:04 smithi088 ceph-mon[88643]: pgmap v2267: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:04 smithi132 ceph-mon[70676]: pgmap v2267: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:07.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:06 smithi088 ceph-mon[88643]: pgmap v2268: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:07.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:06 smithi132 ceph-mon[70676]: pgmap v2268: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:09.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:08 smithi088 ceph-mon[88643]: pgmap v2269: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:09.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:08 smithi132 ceph-mon[70676]: pgmap v2269: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:11.084 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:10 smithi088 ceph-mon[88643]: pgmap v2270: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:11.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:10 smithi132 ceph-mon[70676]: pgmap v2270: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:13.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:12 smithi088 ceph-mon[88643]: pgmap v2271: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:13.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:12 smithi132 ceph-mon[70676]: pgmap v2271: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:14.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:19:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:19:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:14 smithi088 ceph-mon[88643]: pgmap v2272: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:14 smithi132 ceph-mon[70676]: pgmap v2272: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:17.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:16 smithi088 ceph-mon[88643]: pgmap v2273: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:17.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:16 smithi132 ceph-mon[70676]: pgmap v2273: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:19.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:18 smithi088 ceph-mon[88643]: pgmap v2274: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:19.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:18 smithi132 ceph-mon[70676]: pgmap v2274: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:21.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:20 smithi088 ceph-mon[88643]: pgmap v2275: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:20 smithi132 ceph-mon[70676]: pgmap v2275: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:23.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:22 smithi088 ceph-mon[88643]: pgmap v2276: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:23.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:22 smithi132 ceph-mon[70676]: pgmap v2276: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:24 smithi088 ceph-mon[88643]: pgmap v2277: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:24 smithi132 ceph-mon[70676]: pgmap v2277: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:27.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:26 smithi088 ceph-mon[88643]: pgmap v2278: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:27.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:26 smithi132 ceph-mon[70676]: pgmap v2278: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:28 smithi132 ceph-mon[70676]: pgmap v2279: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:28.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:19:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:28 smithi088 ceph-mon[88643]: pgmap v2279: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:29.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:19:31.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:30 smithi088 ceph-mon[88643]: pgmap v2280: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:30 smithi132 ceph-mon[70676]: pgmap v2280: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:33.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:32 smithi132 ceph-mon[70676]: pgmap v2281: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:32 smithi088 ceph-mon[88643]: pgmap v2281: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:34 smithi088 ceph-mon[88643]: pgmap v2282: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:34 smithi132 ceph-mon[70676]: pgmap v2282: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:36 smithi132 ceph-mon[70676]: pgmap v2283: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:37.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:36 smithi088 ceph-mon[88643]: pgmap v2283: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:38 smithi132 ceph-mon[70676]: pgmap v2284: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:38.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:38 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:19:39.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:38 smithi088 ceph-mon[88643]: pgmap v2284: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:39.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:38 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:19:40.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:39 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:19:40.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:39 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:19:40.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:39 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:19:40.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:39 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:19:40.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:39 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:19:40.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:39 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:19:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:40 smithi132 ceph-mon[70676]: pgmap v2285: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:40 smithi088 ceph-mon[88643]: pgmap v2285: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:43.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:42 smithi132 ceph-mon[70676]: pgmap v2286: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:43.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:42 smithi088 ceph-mon[88643]: pgmap v2286: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:44.135 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:19:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:19:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:44 smithi088 ceph-mon[88643]: pgmap v2287: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:44 smithi132 ceph-mon[70676]: pgmap v2287: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:46 smithi132 ceph-mon[70676]: pgmap v2288: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:47.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:46 smithi088 ceph-mon[88643]: pgmap v2288: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:49.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:48 smithi132 ceph-mon[70676]: pgmap v2289: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:49.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:48 smithi088 ceph-mon[88643]: pgmap v2289: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:50 smithi132 ceph-mon[70676]: pgmap v2290: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:51.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:50 smithi088 ceph-mon[88643]: pgmap v2290: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:52 smithi132 ceph-mon[70676]: pgmap v2291: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:53.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:52 smithi088 ceph-mon[88643]: pgmap v2291: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:54 smithi132 ceph-mon[70676]: pgmap v2292: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:55.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:54 smithi088 ceph-mon[88643]: pgmap v2292: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:56 smithi132 ceph-mon[70676]: pgmap v2293: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:57.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:56 smithi088 ceph-mon[88643]: pgmap v2293: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:58 smithi132 ceph-mon[70676]: pgmap v2294: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:19:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:19:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:58 smithi088 ceph-mon[88643]: pgmap v2294: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:19:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:19:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:20:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:00 smithi132 ceph-mon[70676]: pgmap v2295: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:01.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:00 smithi132 ceph-mon[70676]: overall HEALTH_OK 2024-02-19T20:20:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:00 smithi088 ceph-mon[88643]: pgmap v2295: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:00 smithi088 ceph-mon[88643]: overall HEALTH_OK 2024-02-19T20:20:03.132 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:02 smithi088 ceph-mon[88643]: pgmap v2296: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:03.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:02 smithi132 ceph-mon[70676]: pgmap v2296: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:05.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:04 smithi132 ceph-mon[70676]: pgmap v2297: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:05.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:04 smithi088 ceph-mon[88643]: pgmap v2297: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:07.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:06 smithi132 ceph-mon[70676]: pgmap v2298: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:07.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:06 smithi088 ceph-mon[88643]: pgmap v2298: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:09.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:08 smithi132 ceph-mon[70676]: pgmap v2299: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:09.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:08 smithi088 ceph-mon[88643]: pgmap v2299: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:11.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:10 smithi132 ceph-mon[70676]: pgmap v2300: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:11.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:10 smithi088 ceph-mon[88643]: pgmap v2300: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:13.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:12 smithi132 ceph-mon[70676]: pgmap v2301: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:13.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:12 smithi088 ceph-mon[88643]: pgmap v2301: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:14.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:20:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:20:15.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:14 smithi088 ceph-mon[88643]: pgmap v2302: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:15.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:14 smithi132 ceph-mon[70676]: pgmap v2302: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:17.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:16 smithi132 ceph-mon[70676]: pgmap v2303: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:17.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:16 smithi088 ceph-mon[88643]: pgmap v2303: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:19.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:18 smithi132 ceph-mon[70676]: pgmap v2304: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:19.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:18 smithi088 ceph-mon[88643]: pgmap v2304: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:21.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:20 smithi132 ceph-mon[70676]: pgmap v2305: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:21.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:20 smithi088 ceph-mon[88643]: pgmap v2305: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:23.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:22 smithi132 ceph-mon[70676]: pgmap v2306: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:23.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:22 smithi088 ceph-mon[88643]: pgmap v2306: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:25.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:24 smithi088 ceph-mon[88643]: pgmap v2307: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:25.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:24 smithi132 ceph-mon[70676]: pgmap v2307: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:27.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:26 smithi132 ceph-mon[70676]: pgmap v2308: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:26 smithi088 ceph-mon[88643]: pgmap v2308: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:29.188 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:28 smithi132 ceph-mon[70676]: pgmap v2309: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:29.188 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:20:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:28 smithi088 ceph-mon[88643]: pgmap v2309: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:20:31.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:30 smithi132 ceph-mon[70676]: pgmap v2310: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:31.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:30 smithi088 ceph-mon[88643]: pgmap v2310: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:33.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:32 smithi132 ceph-mon[70676]: pgmap v2311: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:32 smithi088 ceph-mon[88643]: pgmap v2311: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:35.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:34 smithi088 ceph-mon[88643]: pgmap v2312: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:34 smithi132 ceph-mon[70676]: pgmap v2312: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:37.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:36 smithi132 ceph-mon[70676]: pgmap v2313: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:37.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:36 smithi088 ceph-mon[88643]: pgmap v2313: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:38.945 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:38 smithi088 ceph-mon[88643]: pgmap v2314: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:38 smithi132 ceph-mon[70676]: pgmap v2314: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:40.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:39 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:20:40.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:39 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:20:40.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:39 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:20:40.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:39 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:20:40.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:39 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:20:40.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:39 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:20:40.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:39 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:20:40.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:39 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:20:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:40 smithi132 ceph-mon[70676]: pgmap v2315: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:40 smithi088 ceph-mon[88643]: pgmap v2315: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:43.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:42 smithi132 ceph-mon[70676]: pgmap v2316: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:43.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:42 smithi088 ceph-mon[88643]: pgmap v2316: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:44.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:20:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:20:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:44 smithi132 ceph-mon[70676]: pgmap v2317: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:45.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:44 smithi088 ceph-mon[88643]: pgmap v2317: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:47.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:46 smithi132 ceph-mon[70676]: pgmap v2318: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:47.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:46 smithi088 ceph-mon[88643]: pgmap v2318: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:49.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:48 smithi132 ceph-mon[70676]: pgmap v2319: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:49.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:48 smithi088 ceph-mon[88643]: pgmap v2319: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:51.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:50 smithi132 ceph-mon[70676]: pgmap v2320: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:51.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:50 smithi088 ceph-mon[88643]: pgmap v2320: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:53.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:52 smithi132 ceph-mon[70676]: pgmap v2321: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:53.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:52 smithi088 ceph-mon[88643]: pgmap v2321: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:55.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:54 smithi132 ceph-mon[70676]: pgmap v2322: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:55.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:54 smithi088 ceph-mon[88643]: pgmap v2322: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:57.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:56 smithi132 ceph-mon[70676]: pgmap v2323: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:57.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:56 smithi088 ceph-mon[88643]: pgmap v2323: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:59.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:58 smithi132 ceph-mon[70676]: pgmap v2324: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:59.234 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:20:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:20:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:58 smithi088 ceph-mon[88643]: pgmap v2324: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:20:59.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:20:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:21:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:00 smithi088 ceph-mon[88643]: pgmap v2325: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:01.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:00 smithi132 ceph-mon[70676]: pgmap v2325: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:03.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:02 smithi088 ceph-mon[88643]: pgmap v2326: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:03.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:02 smithi132 ceph-mon[70676]: pgmap v2326: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:05.277 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:04 smithi088 ceph-mon[88643]: pgmap v2327: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:05.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:04 smithi132 ceph-mon[70676]: pgmap v2327: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:07.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:06 smithi088 ceph-mon[88643]: pgmap v2328: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:07.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:06 smithi132 ceph-mon[70676]: pgmap v2328: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:09.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:08 smithi088 ceph-mon[88643]: pgmap v2329: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:09.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:08 smithi132 ceph-mon[70676]: pgmap v2329: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:11.332 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:11 smithi088 ceph-mon[88643]: pgmap v2330: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:11.488 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:11 smithi132 ceph-mon[70676]: pgmap v2330: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:13.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:13 smithi088 ceph-mon[88643]: pgmap v2331: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:13.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:13 smithi132 ceph-mon[70676]: pgmap v2331: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:14.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:21:14.384 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:21:15.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:15 smithi088 ceph-mon[88643]: pgmap v2332: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:15.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:15 smithi132 ceph-mon[70676]: pgmap v2332: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:17.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:17 smithi088 ceph-mon[88643]: pgmap v2333: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:17.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:17 smithi132 ceph-mon[70676]: pgmap v2333: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:19.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:19 smithi088 ceph-mon[88643]: pgmap v2334: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:19.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:19 smithi132 ceph-mon[70676]: pgmap v2334: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:21.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:21 smithi088 ceph-mon[88643]: pgmap v2335: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:21.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:21 smithi132 ceph-mon[70676]: pgmap v2335: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:23.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:23 smithi088 ceph-mon[88643]: pgmap v2336: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:23.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:23 smithi132 ceph-mon[70676]: pgmap v2336: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:25.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:25 smithi088 ceph-mon[88643]: pgmap v2337: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:25.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:25 smithi132 ceph-mon[70676]: pgmap v2337: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:27.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:27 smithi088 ceph-mon[88643]: pgmap v2338: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:27.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:27 smithi132 ceph-mon[70676]: pgmap v2338: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:29 smithi088 ceph-mon[88643]: pgmap v2339: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:29.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:21:29.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:29 smithi132 ceph-mon[70676]: pgmap v2339: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:29.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:21:31.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:31 smithi088 ceph-mon[88643]: pgmap v2340: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:31.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:31 smithi132 ceph-mon[70676]: pgmap v2340: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:33.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:33 smithi088 ceph-mon[88643]: pgmap v2341: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:33.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:33 smithi132 ceph-mon[70676]: pgmap v2341: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:35.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:35 smithi132 ceph-mon[70676]: pgmap v2342: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:35.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:35 smithi088 ceph-mon[88643]: pgmap v2342: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:37.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:37 smithi088 ceph-mon[88643]: pgmap v2343: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:37.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:37 smithi132 ceph-mon[70676]: pgmap v2343: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:39.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:39 smithi088 ceph-mon[88643]: pgmap v2344: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:39.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:39 smithi132 ceph-mon[70676]: pgmap v2344: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:40.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:40 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:21:40.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:40 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:21:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:41 smithi088 ceph-mon[88643]: pgmap v2345: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:41 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:21:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:41 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:21:41.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:41 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:21:41.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:41 smithi132 ceph-mon[70676]: pgmap v2345: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:41.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:41 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:21:41.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:41 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:21:41.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:41 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:21:43.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:43 smithi088 ceph-mon[88643]: pgmap v2346: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:43.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:43 smithi132 ceph-mon[70676]: pgmap v2346: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:44.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:21:44.384 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:21:45.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:45 smithi132 ceph-mon[70676]: pgmap v2347: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:45.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:45 smithi088 ceph-mon[88643]: pgmap v2347: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:47.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:47 smithi088 ceph-mon[88643]: pgmap v2348: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:47.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:47 smithi132 ceph-mon[70676]: pgmap v2348: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:49.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:49 smithi088 ceph-mon[88643]: pgmap v2349: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:49.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:49 smithi132 ceph-mon[70676]: pgmap v2349: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:51.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:51 smithi088 ceph-mon[88643]: pgmap v2350: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:51.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:51 smithi132 ceph-mon[70676]: pgmap v2350: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:53.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:53 smithi132 ceph-mon[70676]: pgmap v2351: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:53.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:53 smithi088 ceph-mon[88643]: pgmap v2351: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:55.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:55 smithi132 ceph-mon[70676]: pgmap v2352: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:55.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:55 smithi088 ceph-mon[88643]: pgmap v2352: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:57.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:57 smithi132 ceph-mon[70676]: pgmap v2353: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:57.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:57 smithi088 ceph-mon[88643]: pgmap v2353: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:59.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:59 smithi132 ceph-mon[70676]: pgmap v2354: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:59.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:21:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:21:59.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:59 smithi088 ceph-mon[88643]: pgmap v2354: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:21:59.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:21:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:22:01.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:01 smithi088 ceph-mon[88643]: pgmap v2355: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:01.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:01 smithi132 ceph-mon[70676]: pgmap v2355: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:03.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:03 smithi132 ceph-mon[70676]: pgmap v2356: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:03.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:03 smithi088 ceph-mon[88643]: pgmap v2356: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:05.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:05 smithi132 ceph-mon[70676]: pgmap v2357: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:05.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:05 smithi088 ceph-mon[88643]: pgmap v2357: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:07.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:07 smithi132 ceph-mon[70676]: pgmap v2358: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:07.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:07 smithi088 ceph-mon[88643]: pgmap v2358: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:09.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:09 smithi132 ceph-mon[70676]: pgmap v2359: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:09.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:09 smithi088 ceph-mon[88643]: pgmap v2359: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:11.331 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:11 smithi088 ceph-mon[88643]: pgmap v2360: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:11.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:11 smithi132 ceph-mon[70676]: pgmap v2360: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:13.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:13 smithi132 ceph-mon[70676]: pgmap v2361: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:13.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:13 smithi088 ceph-mon[88643]: pgmap v2361: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:14.384 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:22:14.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:22:15.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:15 smithi132 ceph-mon[70676]: pgmap v2362: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:15.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:15 smithi088 ceph-mon[88643]: pgmap v2362: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:17.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:17 smithi132 ceph-mon[70676]: pgmap v2363: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:17.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:17 smithi088 ceph-mon[88643]: pgmap v2363: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:19.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:19 smithi132 ceph-mon[70676]: pgmap v2364: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:19.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:19 smithi088 ceph-mon[88643]: pgmap v2364: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:21.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:21 smithi132 ceph-mon[70676]: pgmap v2365: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:21.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:21 smithi088 ceph-mon[88643]: pgmap v2365: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:23.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:23 smithi132 ceph-mon[70676]: pgmap v2366: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:23.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:23 smithi088 ceph-mon[88643]: pgmap v2366: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:25.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:25 smithi132 ceph-mon[70676]: pgmap v2367: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:25.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:25 smithi088 ceph-mon[88643]: pgmap v2367: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:27.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:27 smithi132 ceph-mon[70676]: pgmap v2368: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:27.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:27 smithi088 ceph-mon[88643]: pgmap v2368: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:29.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:29 smithi132 ceph-mon[70676]: pgmap v2369: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:29.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:22:29.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:29 smithi088 ceph-mon[88643]: pgmap v2369: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:29.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:22:31.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:31 smithi132 ceph-mon[70676]: pgmap v2370: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:31.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:31 smithi088 ceph-mon[88643]: pgmap v2370: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:33.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:33 smithi132 ceph-mon[70676]: pgmap v2371: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:33.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:33 smithi088 ceph-mon[88643]: pgmap v2371: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:35.418 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:35 smithi132 ceph-mon[70676]: pgmap v2372: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:35.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:35 smithi088 ceph-mon[88643]: pgmap v2372: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:36.983 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:36 smithi132 ceph-mon[70676]: pgmap v2373: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:37.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:36 smithi088 ceph-mon[88643]: pgmap v2373: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:39.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:38 smithi088 ceph-mon[88643]: pgmap v2374: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:39.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:38 smithi132 ceph-mon[70676]: pgmap v2374: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:40 smithi088 ceph-mon[88643]: pgmap v2375: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:41.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:40 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:22:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:40 smithi132 ceph-mon[70676]: pgmap v2375: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:41.233 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:40 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:22:42.282 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:42.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:42.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:42.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:42 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:42.283 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:42 smithi132 ceph-mon[70676]: pgmap v2376: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:42.540 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:42.540 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:42.540 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:42.540 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:42 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:42.540 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:42 smithi088 ceph-mon[88643]: pgmap v2376: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:43.425 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:22:43.733 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:22:44.888 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:44.888 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:44.888 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:44 smithi132 ceph-mon[70676]: pgmap v2377: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:45.081 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:44 smithi088 ceph-mon[88643]: pgmap v2377: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:47.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:47 smithi132 ceph-mon[70676]: pgmap v2378: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:47.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:47.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:47.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:22:47.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:22:47.484 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:47.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:47 smithi088 ceph-mon[88643]: pgmap v2378: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:47.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:47.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:47.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:22:47.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:22:47.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:22:49.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:49 smithi132 ceph-mon[70676]: pgmap v2379: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:49.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:49 smithi088 ceph-mon[88643]: pgmap v2379: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:51.481 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:51 smithi088 ceph-mon[88643]: pgmap v2380: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:51.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:51 smithi132 ceph-mon[70676]: pgmap v2380: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:53.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:53 smithi132 ceph-mon[70676]: pgmap v2381: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:53.581 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:53 smithi088 ceph-mon[88643]: pgmap v2381: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:55.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:55 smithi132 ceph-mon[70676]: pgmap v2382: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:55.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:55 smithi088 ceph-mon[88643]: pgmap v2382: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:57.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:57 smithi132 ceph-mon[70676]: pgmap v2383: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:57.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:57 smithi088 ceph-mon[88643]: pgmap v2383: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:59.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:59 smithi132 ceph-mon[70676]: pgmap v2384: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:59.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:22:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:22:59.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:59 smithi088 ceph-mon[88643]: pgmap v2384: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:22:59.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:22:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:23:01.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:01 smithi132 ceph-mon[70676]: pgmap v2385: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:01.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:01 smithi088 ceph-mon[88643]: pgmap v2385: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:03.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:03 smithi132 ceph-mon[70676]: pgmap v2386: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:03.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:03 smithi088 ceph-mon[88643]: pgmap v2386: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:05.483 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:05 smithi132 ceph-mon[70676]: pgmap v2387: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:05.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:05 smithi088 ceph-mon[88643]: pgmap v2387: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:07.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:07 smithi132 ceph-mon[70676]: pgmap v2388: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:07.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:07 smithi088 ceph-mon[88643]: pgmap v2388: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:09.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:09 smithi132 ceph-mon[70676]: pgmap v2389: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:09.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:09 smithi088 ceph-mon[88643]: pgmap v2389: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:11.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:11 smithi132 ceph-mon[70676]: pgmap v2390: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:11.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:11 smithi088 ceph-mon[88643]: pgmap v2390: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:13.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:13 smithi132 ceph-mon[70676]: pgmap v2391: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:13.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:13 smithi088 ceph-mon[88643]: pgmap v2391: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:14.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:23:14.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:23:15.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:15 smithi132 ceph-mon[70676]: pgmap v2392: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:15.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:15 smithi088 ceph-mon[88643]: pgmap v2392: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:17.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:17 smithi132 ceph-mon[70676]: pgmap v2393: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:17.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:17 smithi088 ceph-mon[88643]: pgmap v2393: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:19.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:19 smithi132 ceph-mon[70676]: pgmap v2394: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:19.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:19 smithi088 ceph-mon[88643]: pgmap v2394: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:21.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:21 smithi132 ceph-mon[70676]: pgmap v2395: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:21.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:21 smithi088 ceph-mon[88643]: pgmap v2395: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:23.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:23 smithi132 ceph-mon[70676]: pgmap v2396: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:23.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:23 smithi088 ceph-mon[88643]: pgmap v2396: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:25.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:25 smithi132 ceph-mon[70676]: pgmap v2397: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:25.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:25 smithi088 ceph-mon[88643]: pgmap v2397: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:27.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:27 smithi132 ceph-mon[70676]: pgmap v2398: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:27.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:27 smithi088 ceph-mon[88643]: pgmap v2398: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:29.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:29 smithi132 ceph-mon[70676]: pgmap v2399: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:29.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:23:29.542 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:29 smithi088 ceph-mon[88643]: pgmap v2399: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:29.542 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:23:31.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:31 smithi132 ceph-mon[70676]: pgmap v2400: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:31.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:31 smithi088 ceph-mon[88643]: pgmap v2400: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:33.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:33 smithi132 ceph-mon[70676]: pgmap v2401: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:33.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:33 smithi088 ceph-mon[88643]: pgmap v2401: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:35.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:35 smithi132 ceph-mon[70676]: pgmap v2402: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:35.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:35 smithi088 ceph-mon[88643]: pgmap v2402: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:37.329 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:37 smithi088 ceph-mon[88643]: pgmap v2403: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:37.481 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:37 smithi132 ceph-mon[70676]: pgmap v2403: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:39.481 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:39 smithi132 ceph-mon[70676]: pgmap v2404: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:39.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:39 smithi088 ceph-mon[88643]: pgmap v2404: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:41.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:41 smithi132 ceph-mon[70676]: pgmap v2405: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:41.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:41 smithi088 ceph-mon[88643]: pgmap v2405: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:43.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:43 smithi132 ceph-mon[70676]: pgmap v2406: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:43.788 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:43 smithi088 ceph-mon[88643]: pgmap v2406: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:44.481 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:23:44.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:23:45.481 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:45 smithi132 ceph-mon[70676]: pgmap v2407: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:45.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:45 smithi088 ceph-mon[88643]: pgmap v2407: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:47.481 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:47 smithi132 ceph-mon[70676]: pgmap v2408: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:47.481 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:23:47.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:47 smithi088 ceph-mon[88643]: pgmap v2408: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:47.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:23:48.481 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:48 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:23:48.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:48 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:23:48.482 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:48 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:23:48.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:48 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:23:48.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:48 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:23:48.580 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:48 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:23:49.481 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:49 smithi132 ceph-mon[70676]: pgmap v2409: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:49.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:49 smithi088 ceph-mon[88643]: pgmap v2409: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:50.481 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:50 smithi132 ceph-mon[70676]: pgmap v2410: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:50.579 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:50 smithi088 ceph-mon[88643]: pgmap v2410: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:53.079 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:52 smithi088 ceph-mon[88643]: pgmap v2411: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:53.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:52 smithi132 ceph-mon[70676]: pgmap v2411: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:55.079 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:54 smithi088 ceph-mon[88643]: pgmap v2412: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:55.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:54 smithi132 ceph-mon[70676]: pgmap v2412: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:57.079 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:56 smithi088 ceph-mon[88643]: pgmap v2413: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:57.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:56 smithi132 ceph-mon[70676]: pgmap v2413: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:59.079 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:58 smithi088 ceph-mon[88643]: pgmap v2414: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:59.079 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:23:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:23:59.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:58 smithi132 ceph-mon[70676]: pgmap v2414: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:23:59.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:23:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:24:01.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:00 smithi132 ceph-mon[70676]: pgmap v2415: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:01.329 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:00 smithi088 ceph-mon[88643]: pgmap v2415: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:03.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:02 smithi132 ceph-mon[70676]: pgmap v2416: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:03.329 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:02 smithi088 ceph-mon[88643]: pgmap v2416: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:05.078 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:04 smithi088 ceph-mon[88643]: pgmap v2417: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:05.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:04 smithi132 ceph-mon[70676]: pgmap v2417: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:07.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:06 smithi132 ceph-mon[70676]: pgmap v2418: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:07.329 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:06 smithi088 ceph-mon[88643]: pgmap v2418: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:09.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:08 smithi132 ceph-mon[70676]: pgmap v2419: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:09.329 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:08 smithi088 ceph-mon[88643]: pgmap v2419: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:11.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:10 smithi132 ceph-mon[70676]: pgmap v2420: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:11.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:10 smithi088 ceph-mon[88643]: pgmap v2420: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:13.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:12 smithi132 ceph-mon[70676]: pgmap v2421: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:13.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:12 smithi088 ceph-mon[88643]: pgmap v2421: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:14.132 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:24:14.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:24:15.040 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:14 smithi088 ceph-mon[88643]: pgmap v2422: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:15.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:14 smithi132 ceph-mon[70676]: pgmap v2422: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:17.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:16 smithi132 ceph-mon[70676]: pgmap v2423: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:17.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:16 smithi088 ceph-mon[88643]: pgmap v2423: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:19.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:18 smithi132 ceph-mon[70676]: pgmap v2424: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:19.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:18 smithi088 ceph-mon[88643]: pgmap v2424: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:21.231 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:20 smithi132 ceph-mon[70676]: pgmap v2425: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:21.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:20 smithi088 ceph-mon[88643]: pgmap v2425: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:23.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:22 smithi132 ceph-mon[70676]: pgmap v2426: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:23.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:22 smithi088 ceph-mon[88643]: pgmap v2426: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:25.078 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:24 smithi088 ceph-mon[88643]: pgmap v2427: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:25.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:24 smithi132 ceph-mon[70676]: pgmap v2427: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:27.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:26 smithi132 ceph-mon[70676]: pgmap v2428: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:27.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:26 smithi088 ceph-mon[88643]: pgmap v2428: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:29.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:28 smithi132 ceph-mon[70676]: pgmap v2429: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:29.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:24:29.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:28 smithi088 ceph-mon[88643]: pgmap v2429: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:29.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:24:31.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:30 smithi132 ceph-mon[70676]: pgmap v2430: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:31.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:30 smithi088 ceph-mon[88643]: pgmap v2430: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:33.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:32 smithi132 ceph-mon[70676]: pgmap v2431: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:33.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:32 smithi088 ceph-mon[88643]: pgmap v2431: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:35.078 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:34 smithi088 ceph-mon[88643]: pgmap v2432: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:35.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:34 smithi132 ceph-mon[70676]: pgmap v2432: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:37.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:36 smithi132 ceph-mon[70676]: pgmap v2433: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:37.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:36 smithi088 ceph-mon[88643]: pgmap v2433: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:39.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:38 smithi132 ceph-mon[70676]: pgmap v2434: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:39.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:38 smithi088 ceph-mon[88643]: pgmap v2434: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:41.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:40 smithi132 ceph-mon[70676]: pgmap v2435: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:41.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:40 smithi088 ceph-mon[88643]: pgmap v2435: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:43.078 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:42 smithi088 ceph-mon[88643]: pgmap v2436: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:43.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:42 smithi132 ceph-mon[70676]: pgmap v2436: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:44.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:24:44.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:24:45.078 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:44 smithi088 ceph-mon[88643]: pgmap v2437: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:45.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:44 smithi132 ceph-mon[70676]: pgmap v2437: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:46.906 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:46 smithi132 ceph-mon[70676]: pgmap v2438: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:47.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:46 smithi088 ceph-mon[88643]: pgmap v2438: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:48.077 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:47 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:24:48.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:47 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:24:49.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:48 smithi132 ceph-mon[70676]: pgmap v2439: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:49.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:48 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:24:49.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:48 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:24:49.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:48 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:24:49.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:48 smithi088 ceph-mon[88643]: pgmap v2439: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:49.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:48 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:24:49.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:48 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:24:49.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:48 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:24:51.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:50 smithi132 ceph-mon[70676]: pgmap v2440: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:51.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:50 smithi088 ceph-mon[88643]: pgmap v2440: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:53.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:52 smithi132 ceph-mon[70676]: pgmap v2441: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:53.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:52 smithi088 ceph-mon[88643]: pgmap v2441: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:55.078 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:54 smithi088 ceph-mon[88643]: pgmap v2442: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:55.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:54 smithi132 ceph-mon[70676]: pgmap v2442: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:57.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:56 smithi132 ceph-mon[70676]: pgmap v2443: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:57.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:56 smithi088 ceph-mon[88643]: pgmap v2443: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:59.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:58 smithi132 ceph-mon[70676]: pgmap v2444: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:59.230 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:24:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:24:59.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:58 smithi088 ceph-mon[88643]: pgmap v2444: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:24:59.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:24:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:25:01.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:00 smithi132 ceph-mon[70676]: pgmap v2445: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:01.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:00 smithi088 ceph-mon[88643]: pgmap v2445: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:03.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:02 smithi132 ceph-mon[70676]: pgmap v2446: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:03.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:02 smithi088 ceph-mon[88643]: pgmap v2446: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:05.077 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:04 smithi088 ceph-mon[88643]: pgmap v2447: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:05.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:04 smithi132 ceph-mon[70676]: pgmap v2447: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:07.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:06 smithi132 ceph-mon[70676]: pgmap v2448: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:07.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:06 smithi088 ceph-mon[88643]: pgmap v2448: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:09.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:08 smithi132 ceph-mon[70676]: pgmap v2449: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:09.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:08 smithi088 ceph-mon[88643]: pgmap v2449: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:11.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:10 smithi132 ceph-mon[70676]: pgmap v2450: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:11.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:10 smithi088 ceph-mon[88643]: pgmap v2450: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:13.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:12 smithi132 ceph-mon[70676]: pgmap v2451: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:13.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:12 smithi088 ceph-mon[88643]: pgmap v2451: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:14.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:25:14.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:25:15.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:14 smithi132 ceph-mon[70676]: pgmap v2452: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:15.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:14 smithi088 ceph-mon[88643]: pgmap v2452: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:17.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:16 smithi132 ceph-mon[70676]: pgmap v2453: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:17.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:16 smithi088 ceph-mon[88643]: pgmap v2453: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:19.232 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:18 smithi132 ceph-mon[70676]: pgmap v2454: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:19.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:18 smithi088 ceph-mon[88643]: pgmap v2454: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:21.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:20 smithi132 ceph-mon[70676]: pgmap v2455: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:21.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:20 smithi088 ceph-mon[88643]: pgmap v2455: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:23.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:22 smithi132 ceph-mon[70676]: pgmap v2456: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:23.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:22 smithi088 ceph-mon[88643]: pgmap v2456: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:25.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:24 smithi132 ceph-mon[70676]: pgmap v2457: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:25.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:24 smithi088 ceph-mon[88643]: pgmap v2457: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:27.229 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:26 smithi132 ceph-mon[70676]: pgmap v2458: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:27.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:26 smithi088 ceph-mon[88643]: pgmap v2458: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:29.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:28 smithi088 ceph-mon[88643]: pgmap v2459: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:29.327 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:25:29.479 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:28 smithi132 ceph-mon[70676]: pgmap v2459: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:29.479 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:25:31.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:30 smithi088 ceph-mon[88643]: pgmap v2460: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:31.479 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:30 smithi132 ceph-mon[70676]: pgmap v2460: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:33.291 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:32 smithi088 ceph-mon[88643]: pgmap v2461: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:33.479 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:32 smithi132 ceph-mon[70676]: pgmap v2461: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:35.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:34 smithi088 ceph-mon[88643]: pgmap v2462: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:35.479 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:34 smithi132 ceph-mon[70676]: pgmap v2462: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:37.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:36 smithi088 ceph-mon[88643]: pgmap v2463: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:37.479 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:36 smithi132 ceph-mon[70676]: pgmap v2463: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:39.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:38 smithi088 ceph-mon[88643]: pgmap v2464: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:39.480 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:38 smithi132 ceph-mon[70676]: pgmap v2464: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:41.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:41 smithi088 ceph-mon[88643]: pgmap v2465: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:41.479 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:41 smithi132 ceph-mon[70676]: pgmap v2465: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:43.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:43 smithi088 ceph-mon[88643]: pgmap v2466: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:43.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:43 smithi132 ceph-mon[70676]: pgmap v2466: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:44.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:25:44.380 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:25:45.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:45 smithi088 ceph-mon[88643]: pgmap v2467: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:45.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:45 smithi132 ceph-mon[70676]: pgmap v2467: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:47.294 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:47 smithi132 ceph-mon[70676]: pgmap v2468: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:47.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:47 smithi088 ceph-mon[88643]: pgmap v2468: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:49.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:49 smithi088 ceph-mon[88643]: pgmap v2469: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:49.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:49 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:25:49.479 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:49 smithi132 ceph-mon[70676]: pgmap v2469: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:49.479 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:49 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:25:50.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:25:50.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:25:50.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:25:50.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:25:50.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:25:50.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:25:51.328 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:51 smithi088 ceph-mon[88643]: pgmap v2470: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:51.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:51 smithi132 ceph-mon[70676]: pgmap v2470: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:53.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:53 smithi088 ceph-mon[88643]: pgmap v2471: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:53.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:53 smithi132 ceph-mon[70676]: pgmap v2471: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:55.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:55 smithi088 ceph-mon[88643]: pgmap v2472: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:55.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:55 smithi132 ceph-mon[70676]: pgmap v2472: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:57.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:57 smithi088 ceph-mon[88643]: pgmap v2473: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:57.393 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:57 smithi132 ceph-mon[70676]: pgmap v2473: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:59.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:59 smithi088 ceph-mon[88643]: pgmap v2474: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:59.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:25:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:25:59.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:59 smithi132 ceph-mon[70676]: pgmap v2474: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:25:59.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:25:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:26:01.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:01 smithi088 ceph-mon[88643]: pgmap v2475: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:01.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:01 smithi132 ceph-mon[70676]: pgmap v2475: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:03.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:03 smithi088 ceph-mon[88643]: pgmap v2476: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:03.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:03 smithi132 ceph-mon[70676]: pgmap v2476: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:05.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:05 smithi088 ceph-mon[88643]: pgmap v2477: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:05.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:05 smithi132 ceph-mon[70676]: pgmap v2477: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:07.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:07 smithi088 ceph-mon[88643]: pgmap v2478: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:07.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:07 smithi132 ceph-mon[70676]: pgmap v2478: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:09.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:09 smithi088 ceph-mon[88643]: pgmap v2479: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:09.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:09 smithi132 ceph-mon[70676]: pgmap v2479: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:11.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:11 smithi088 ceph-mon[88643]: pgmap v2480: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:11.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:11 smithi132 ceph-mon[70676]: pgmap v2480: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:13.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:13 smithi088 ceph-mon[88643]: pgmap v2481: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:13.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:13 smithi132 ceph-mon[70676]: pgmap v2481: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:14.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:26:14.379 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:26:15.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:15 smithi132 ceph-mon[70676]: pgmap v2482: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:15.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:15 smithi088 ceph-mon[88643]: pgmap v2482: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:17.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:17 smithi132 ceph-mon[70676]: pgmap v2483: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:17.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:17 smithi088 ceph-mon[88643]: pgmap v2483: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:19.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:19 smithi132 ceph-mon[70676]: pgmap v2484: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:19.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:19 smithi088 ceph-mon[88643]: pgmap v2484: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:21.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:21 smithi132 ceph-mon[70676]: pgmap v2485: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:21.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:21 smithi088 ceph-mon[88643]: pgmap v2485: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:23.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:23 smithi132 ceph-mon[70676]: pgmap v2486: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:23.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:23 smithi088 ceph-mon[88643]: pgmap v2486: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:25.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:25 smithi132 ceph-mon[70676]: pgmap v2487: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:25.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:25 smithi088 ceph-mon[88643]: pgmap v2487: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:27.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:27 smithi132 ceph-mon[70676]: pgmap v2488: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:27.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:27 smithi088 ceph-mon[88643]: pgmap v2488: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:29.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:29 smithi088 ceph-mon[88643]: pgmap v2489: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:29.326 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:26:29.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:29 smithi132 ceph-mon[70676]: pgmap v2489: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:29.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:26:31.477 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:31 smithi132 ceph-mon[70676]: pgmap v2490: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:31.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:31 smithi088 ceph-mon[88643]: pgmap v2490: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:33.378 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:33 smithi088 ceph-mon[88643]: pgmap v2491: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:33.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:33 smithi132 ceph-mon[70676]: pgmap v2491: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:35.478 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:35 smithi132 ceph-mon[70676]: pgmap v2492: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:35.575 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:35 smithi088 ceph-mon[88643]: pgmap v2492: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:37.075 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:36 smithi088 ceph-mon[88643]: pgmap v2493: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:37.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:36 smithi132 ceph-mon[70676]: pgmap v2493: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:39.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:38 smithi132 ceph-mon[70676]: pgmap v2494: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:39.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:38 smithi088 ceph-mon[88643]: pgmap v2494: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:41.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:40 smithi132 ceph-mon[70676]: pgmap v2495: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:41.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:40 smithi088 ceph-mon[88643]: pgmap v2495: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:43.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:42 smithi132 ceph-mon[70676]: pgmap v2496: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:43.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:42 smithi088 ceph-mon[88643]: pgmap v2496: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:44.128 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:26:44.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:26:45.075 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:44 smithi088 ceph-mon[88643]: pgmap v2497: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:45.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:44 smithi132 ceph-mon[70676]: pgmap v2497: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:47.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:46 smithi132 ceph-mon[70676]: pgmap v2498: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:47.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:46 smithi088 ceph-mon[88643]: pgmap v2498: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:49.143 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:48 smithi088 ceph-mon[88643]: pgmap v2499: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:49.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:48 smithi132 ceph-mon[70676]: pgmap v2499: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:50.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:49 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:26:50.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:49 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:26:50.228 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:49 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:26:50.228 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:49 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:26:50.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:49 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:26:50.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:49 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:26:50.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:49 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:26:50.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:49 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:26:51.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:50 smithi132 ceph-mon[70676]: pgmap v2500: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:51.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:50 smithi088 ceph-mon[88643]: pgmap v2500: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:53.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:52 smithi132 ceph-mon[70676]: pgmap v2501: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:53.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:52 smithi088 ceph-mon[88643]: pgmap v2501: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:55.075 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:54 smithi088 ceph-mon[88643]: pgmap v2502: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:55.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:54 smithi132 ceph-mon[70676]: pgmap v2502: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:57.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:56 smithi132 ceph-mon[70676]: pgmap v2503: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:57.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:56 smithi088 ceph-mon[88643]: pgmap v2503: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:59.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:58 smithi132 ceph-mon[70676]: pgmap v2504: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:59.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:26:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:26:59.257 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:58 smithi088 ceph-mon[88643]: pgmap v2504: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:26:59.258 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:26:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:27:01.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:00 smithi132 ceph-mon[70676]: pgmap v2505: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:01.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:00 smithi088 ceph-mon[88643]: pgmap v2505: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:03.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:02 smithi132 ceph-mon[70676]: pgmap v2506: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:03.325 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:02 smithi088 ceph-mon[88643]: pgmap v2506: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:05.075 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:04 smithi088 ceph-mon[88643]: pgmap v2507: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:05.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:04 smithi132 ceph-mon[70676]: pgmap v2507: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:07.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:06 smithi132 ceph-mon[70676]: pgmap v2508: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:07.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:06 smithi088 ceph-mon[88643]: pgmap v2508: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:09.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:08 smithi132 ceph-mon[70676]: pgmap v2509: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:09.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:08 smithi088 ceph-mon[88643]: pgmap v2509: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:11.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:10 smithi132 ceph-mon[70676]: pgmap v2510: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:11.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:10 smithi088 ceph-mon[88643]: pgmap v2510: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:13.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:12 smithi132 ceph-mon[70676]: pgmap v2511: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:13.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:12 smithi088 ceph-mon[88643]: pgmap v2511: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:14.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:27:14.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:27:15.074 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:14 smithi088 ceph-mon[88643]: pgmap v2512: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:15.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:14 smithi132 ceph-mon[70676]: pgmap v2512: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:17.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:16 smithi132 ceph-mon[70676]: pgmap v2513: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:17.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:16 smithi088 ceph-mon[88643]: pgmap v2513: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:19.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:18 smithi132 ceph-mon[70676]: pgmap v2514: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:19.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:18 smithi088 ceph-mon[88643]: pgmap v2514: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:21.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:20 smithi132 ceph-mon[70676]: pgmap v2515: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:21.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:20 smithi088 ceph-mon[88643]: pgmap v2515: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:23.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:22 smithi132 ceph-mon[70676]: pgmap v2516: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:23.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:22 smithi088 ceph-mon[88643]: pgmap v2516: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:25.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:24 smithi132 ceph-mon[70676]: pgmap v2517: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:25.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:24 smithi088 ceph-mon[88643]: pgmap v2517: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:27.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:26 smithi132 ceph-mon[70676]: pgmap v2518: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:27.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:26 smithi088 ceph-mon[88643]: pgmap v2518: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:29.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:28 smithi132 ceph-mon[70676]: pgmap v2519: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:29.227 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:27:29.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:28 smithi088 ceph-mon[88643]: pgmap v2519: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:29.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:27:31.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:30 smithi132 ceph-mon[70676]: pgmap v2520: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:31.329 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:30 smithi088 ceph-mon[88643]: pgmap v2520: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:33.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:32 smithi132 ceph-mon[70676]: pgmap v2521: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:33.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:32 smithi088 ceph-mon[88643]: pgmap v2521: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:35.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:34 smithi132 ceph-mon[70676]: pgmap v2522: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:35.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:34 smithi088 ceph-mon[88643]: pgmap v2522: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:37.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:36 smithi132 ceph-mon[70676]: pgmap v2523: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:37.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:36 smithi088 ceph-mon[88643]: pgmap v2523: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:39.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:38 smithi132 ceph-mon[70676]: pgmap v2524: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:39.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:38 smithi088 ceph-mon[88643]: pgmap v2524: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:41.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:40 smithi132 ceph-mon[70676]: pgmap v2525: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:41.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:40 smithi088 ceph-mon[88643]: pgmap v2525: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:43.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:42 smithi132 ceph-mon[70676]: pgmap v2526: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:43.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:42 smithi088 ceph-mon[88643]: pgmap v2526: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:44.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:27:44.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:27:45.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:44 smithi132 ceph-mon[70676]: pgmap v2527: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:45.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:44 smithi088 ceph-mon[88643]: pgmap v2527: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:47.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:46 smithi132 ceph-mon[70676]: pgmap v2528: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:47.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:46 smithi088 ceph-mon[88643]: pgmap v2528: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:49.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:48 smithi132 ceph-mon[70676]: pgmap v2529: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:49.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:48 smithi088 ceph-mon[88643]: pgmap v2529: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:50.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:49 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:27:50.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:49 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:27:51.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:50 smithi088 ceph-mon[88643]: pgmap v2530: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:51.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:27:51.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:27:51.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:50 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:27:51.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:50 smithi132 ceph-mon[70676]: pgmap v2530: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:51.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:27:51.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:27:51.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:50 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:27:53.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:52 smithi088 ceph-mon[88643]: pgmap v2531: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:53.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:52 smithi132 ceph-mon[70676]: pgmap v2531: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:55.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:54 smithi088 ceph-mon[88643]: pgmap v2532: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:55.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:54 smithi132 ceph-mon[70676]: pgmap v2532: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:57.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:56 smithi088 ceph-mon[88643]: pgmap v2533: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:57.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:56 smithi132 ceph-mon[70676]: pgmap v2533: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:59.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:58 smithi088 ceph-mon[88643]: pgmap v2534: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:59.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:27:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:27:59.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:58 smithi132 ceph-mon[70676]: pgmap v2534: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:27:59.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:27:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:28:01.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:01 smithi088 ceph-mon[88643]: pgmap v2535: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:01.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:01 smithi132 ceph-mon[70676]: pgmap v2535: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:03.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:03 smithi088 ceph-mon[88643]: pgmap v2536: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:03.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:03 smithi132 ceph-mon[70676]: pgmap v2536: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:05.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:05 smithi088 ceph-mon[88643]: pgmap v2537: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:05.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:05 smithi132 ceph-mon[70676]: pgmap v2537: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:07.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:07 smithi088 ceph-mon[88643]: pgmap v2538: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:07.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:07 smithi132 ceph-mon[70676]: pgmap v2538: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:09.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:09 smithi088 ceph-mon[88643]: pgmap v2539: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:09.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:09 smithi132 ceph-mon[70676]: pgmap v2539: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:11.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:11 smithi088 ceph-mon[88643]: pgmap v2540: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:11.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:11 smithi132 ceph-mon[70676]: pgmap v2540: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:13.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:13 smithi088 ceph-mon[88643]: pgmap v2541: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:13.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:13 smithi132 ceph-mon[70676]: pgmap v2541: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:14.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:28:14.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:28:15.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:15 smithi088 ceph-mon[88643]: pgmap v2542: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:15.476 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:15 smithi132 ceph-mon[70676]: pgmap v2542: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:17.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:17 smithi088 ceph-mon[88643]: pgmap v2543: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:17.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:17 smithi132 ceph-mon[70676]: pgmap v2543: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:19.226 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:19 smithi132 ceph-mon[70676]: pgmap v2544: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:19.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:19 smithi088 ceph-mon[88643]: pgmap v2544: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:21.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:21 smithi088 ceph-mon[88643]: pgmap v2545: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:21.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:21 smithi132 ceph-mon[70676]: pgmap v2545: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:23.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:23 smithi088 ceph-mon[88643]: pgmap v2546: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:23.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:23 smithi132 ceph-mon[70676]: pgmap v2546: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:25.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:25 smithi088 ceph-mon[88643]: pgmap v2547: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:25.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:25 smithi132 ceph-mon[70676]: pgmap v2547: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:27.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:27 smithi088 ceph-mon[88643]: pgmap v2548: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:27.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:27 smithi132 ceph-mon[70676]: pgmap v2548: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:29.225 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:29 smithi132 ceph-mon[70676]: pgmap v2549: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:29.225 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:28:29.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:29 smithi088 ceph-mon[88643]: pgmap v2549: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:29.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:28:31.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:31 smithi088 ceph-mon[88643]: pgmap v2550: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:31.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:31 smithi132 ceph-mon[70676]: pgmap v2550: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:33.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:33 smithi088 ceph-mon[88643]: pgmap v2551: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:33.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:33 smithi132 ceph-mon[70676]: pgmap v2551: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:35.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:35 smithi088 ceph-mon[88643]: pgmap v2552: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:35.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:35 smithi132 ceph-mon[70676]: pgmap v2552: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:37.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:37 smithi088 ceph-mon[88643]: pgmap v2553: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:37.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:37 smithi132 ceph-mon[70676]: pgmap v2553: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:39.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:39 smithi088 ceph-mon[88643]: pgmap v2554: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:39.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:39 smithi132 ceph-mon[70676]: pgmap v2554: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:41.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:41 smithi088 ceph-mon[88643]: pgmap v2555: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:41.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:41 smithi132 ceph-mon[70676]: pgmap v2555: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:43.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:43 smithi088 ceph-mon[88643]: pgmap v2556: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:43.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:43 smithi132 ceph-mon[70676]: pgmap v2556: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:44.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:28:44.376 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:28:45.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:45 smithi088 ceph-mon[88643]: pgmap v2557: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:45.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:45 smithi132 ceph-mon[70676]: pgmap v2557: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:47.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:47 smithi132 ceph-mon[70676]: pgmap v2558: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:47.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:47 smithi088 ceph-mon[88643]: pgmap v2558: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:49.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:49 smithi132 ceph-mon[70676]: pgmap v2559: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:49.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:49 smithi088 ceph-mon[88643]: pgmap v2559: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:51.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:51 smithi132 ceph-mon[70676]: pgmap v2560: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:51.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:51 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:28:51.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:51 smithi088 ceph-mon[88643]: pgmap v2560: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:51.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:51 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:28:52.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:28:52.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:28:52.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:28:52.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:28:52.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:28:52.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:28:53.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:53 smithi132 ceph-mon[70676]: pgmap v2561: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:53.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:53 smithi088 ceph-mon[88643]: pgmap v2561: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:55.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:55 smithi132 ceph-mon[70676]: pgmap v2562: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:55.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:55 smithi088 ceph-mon[88643]: pgmap v2562: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:57.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:57 smithi132 ceph-mon[70676]: pgmap v2563: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:57.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:57 smithi088 ceph-mon[88643]: pgmap v2563: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:59.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:59 smithi132 ceph-mon[70676]: pgmap v2564: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:59.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:28:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:28:59.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:59 smithi088 ceph-mon[88643]: pgmap v2564: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:28:59.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:28:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:29:01.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:01 smithi132 ceph-mon[70676]: pgmap v2565: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:01.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:01 smithi088 ceph-mon[88643]: pgmap v2565: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:03.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:03 smithi132 ceph-mon[70676]: pgmap v2566: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:03.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:03 smithi088 ceph-mon[88643]: pgmap v2566: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:05.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:05 smithi132 ceph-mon[70676]: pgmap v2567: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:05.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:05 smithi088 ceph-mon[88643]: pgmap v2567: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:07.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:07 smithi132 ceph-mon[70676]: pgmap v2568: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:07.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:07 smithi088 ceph-mon[88643]: pgmap v2568: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:09.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:09 smithi132 ceph-mon[70676]: pgmap v2569: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:09.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:09 smithi088 ceph-mon[88643]: pgmap v2569: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:11.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:11 smithi132 ceph-mon[70676]: pgmap v2570: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:11.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:11 smithi088 ceph-mon[88643]: pgmap v2570: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:13.474 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:13 smithi132 ceph-mon[70676]: pgmap v2571: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:13.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:13 smithi088 ceph-mon[88643]: pgmap v2571: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:14.474 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:29:14.573 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:29:15.475 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:15 smithi132 ceph-mon[70676]: pgmap v2572: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:15.577 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:15 smithi088 ceph-mon[88643]: pgmap v2572: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:17.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:17 smithi088 ceph-mon[88643]: pgmap v2573: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:17.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:17 smithi132 ceph-mon[70676]: pgmap v2573: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:19.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:19 smithi088 ceph-mon[88643]: pgmap v2574: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:19.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:19 smithi132 ceph-mon[70676]: pgmap v2574: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:21.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:21 smithi088 ceph-mon[88643]: pgmap v2575: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:21.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:21 smithi132 ceph-mon[70676]: pgmap v2575: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:23.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:23 smithi088 ceph-mon[88643]: pgmap v2576: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:23.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:23 smithi132 ceph-mon[70676]: pgmap v2576: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:25.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:25 smithi088 ceph-mon[88643]: pgmap v2577: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:25.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:25 smithi132 ceph-mon[70676]: pgmap v2577: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:27.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:27 smithi088 ceph-mon[88643]: pgmap v2578: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:27.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:27 smithi132 ceph-mon[70676]: pgmap v2578: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:29.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:29 smithi088 ceph-mon[88643]: pgmap v2579: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:29.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:29:29.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:29 smithi132 ceph-mon[70676]: pgmap v2579: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:29.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:29:31.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:31 smithi088 ceph-mon[88643]: pgmap v2580: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:31.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:31 smithi132 ceph-mon[70676]: pgmap v2580: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:33.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:33 smithi088 ceph-mon[88643]: pgmap v2581: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:33.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:33 smithi132 ceph-mon[70676]: pgmap v2581: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:35.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:35 smithi088 ceph-mon[88643]: pgmap v2582: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:35.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:35 smithi132 ceph-mon[70676]: pgmap v2582: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:37.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:37 smithi088 ceph-mon[88643]: pgmap v2583: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:37.725 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:37 smithi132 ceph-mon[70676]: pgmap v2583: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:39.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:39 smithi088 ceph-mon[88643]: pgmap v2584: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:39.574 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:39 smithi132 ceph-mon[70676]: pgmap v2584: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:41.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:41 smithi088 ceph-mon[88643]: pgmap v2585: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:41.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:41 smithi132 ceph-mon[70676]: pgmap v2585: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:42.572 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:42 smithi088 ceph-mon[88643]: pgmap v2586: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:42.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:42 smithi132 ceph-mon[70676]: pgmap v2586: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:43.724 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:29:43.822 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:29:44.624 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:44 smithi132 ceph-mon[70676]: pgmap v2587: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:44.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:44 smithi088 ceph-mon[88643]: pgmap v2587: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:47.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:46 smithi132 ceph-mon[70676]: pgmap v2588: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:47.322 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:46 smithi088 ceph-mon[88643]: pgmap v2588: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:49.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:48 smithi132 ceph-mon[70676]: pgmap v2589: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:49.322 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:48 smithi088 ceph-mon[88643]: pgmap v2589: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:51.197 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:50 smithi088 ceph-mon[88643]: pgmap v2590: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:51.225 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:50 smithi132 ceph-mon[70676]: pgmap v2590: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:52.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:51 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:29:52.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:51 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:29:52.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:51 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:29:52.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:51 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:29:52.322 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:51 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:29:52.322 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:51 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:29:52.322 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:51 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:29:52.322 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:51 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:29:53.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:52 smithi132 ceph-mon[70676]: pgmap v2591: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:53.322 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:52 smithi088 ceph-mon[88643]: pgmap v2591: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:54.974 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:54 smithi132 ceph-mon[70676]: pgmap v2592: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:55.071 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:54 smithi088 ceph-mon[88643]: pgmap v2592: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:57.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:56 smithi132 ceph-mon[70676]: pgmap v2593: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:57.322 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:56 smithi088 ceph-mon[88643]: pgmap v2593: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:59.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:58 smithi132 ceph-mon[70676]: pgmap v2594: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:59.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:29:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:29:59.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:58 smithi088 ceph-mon[88643]: pgmap v2594: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:29:59.322 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:29:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:30:01.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:00 smithi132 ceph-mon[70676]: pgmap v2595: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:01.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:00 smithi132 ceph-mon[70676]: overall HEALTH_OK 2024-02-19T20:30:01.310 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:00 smithi088 ceph-mon[88643]: pgmap v2595: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:01.311 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:00 smithi088 ceph-mon[88643]: overall HEALTH_OK 2024-02-19T20:30:03.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:02 smithi132 ceph-mon[70676]: pgmap v2596: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:03.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:02 smithi088 ceph-mon[88643]: pgmap v2596: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:05.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:04 smithi132 ceph-mon[70676]: pgmap v2597: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:05.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:04 smithi088 ceph-mon[88643]: pgmap v2597: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:07.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:06 smithi132 ceph-mon[70676]: pgmap v2598: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:07.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:06 smithi088 ceph-mon[88643]: pgmap v2598: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:09.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:08 smithi132 ceph-mon[70676]: pgmap v2599: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:09.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:08 smithi088 ceph-mon[88643]: pgmap v2599: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:11.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:10 smithi132 ceph-mon[70676]: pgmap v2600: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:11.322 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:10 smithi088 ceph-mon[88643]: pgmap v2600: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:13.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:12 smithi132 ceph-mon[70676]: pgmap v2601: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:13.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:12 smithi088 ceph-mon[88643]: pgmap v2601: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:14.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:30:14.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:30:15.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:14 smithi132 ceph-mon[70676]: pgmap v2602: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:15.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:14 smithi088 ceph-mon[88643]: pgmap v2602: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:17.071 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:16 smithi088 ceph-mon[88643]: pgmap v2603: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:17.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:16 smithi132 ceph-mon[70676]: pgmap v2603: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:19.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:18 smithi132 ceph-mon[70676]: pgmap v2604: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:19.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:18 smithi088 ceph-mon[88643]: pgmap v2604: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:21.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:20 smithi132 ceph-mon[70676]: pgmap v2605: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:21.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:20 smithi088 ceph-mon[88643]: pgmap v2605: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:23.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:22 smithi132 ceph-mon[70676]: pgmap v2606: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:23.322 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:22 smithi088 ceph-mon[88643]: pgmap v2606: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:25.071 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:24 smithi088 ceph-mon[88643]: pgmap v2607: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:25.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:24 smithi132 ceph-mon[70676]: pgmap v2607: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:27.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:26 smithi132 ceph-mon[70676]: pgmap v2608: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:27.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:26 smithi088 ceph-mon[88643]: pgmap v2608: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:29.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:28 smithi132 ceph-mon[70676]: pgmap v2609: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:29.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:30:29.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:28 smithi088 ceph-mon[88643]: pgmap v2609: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:29.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:30:31.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:30 smithi132 ceph-mon[70676]: pgmap v2610: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:31.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:30 smithi088 ceph-mon[88643]: pgmap v2610: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:33.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:32 smithi132 ceph-mon[70676]: pgmap v2611: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:33.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:32 smithi088 ceph-mon[88643]: pgmap v2611: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:35.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:34 smithi132 ceph-mon[70676]: pgmap v2612: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:35.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:34 smithi088 ceph-mon[88643]: pgmap v2612: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:37.071 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:36 smithi088 ceph-mon[88643]: pgmap v2613: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:37.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:36 smithi132 ceph-mon[70676]: pgmap v2613: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:39.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:38 smithi132 ceph-mon[70676]: pgmap v2614: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:39.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:38 smithi088 ceph-mon[88643]: pgmap v2614: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:41.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:40 smithi132 ceph-mon[70676]: pgmap v2615: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:41.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:40 smithi088 ceph-mon[88643]: pgmap v2615: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:43.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:42 smithi132 ceph-mon[70676]: pgmap v2616: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:43.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:42 smithi088 ceph-mon[88643]: pgmap v2616: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:44.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:30:44.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:30:45.071 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:44 smithi088 ceph-mon[88643]: pgmap v2617: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:45.217 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:44 smithi132 ceph-mon[70676]: pgmap v2617: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:47.071 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:46 smithi088 ceph-mon[88643]: pgmap v2618: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:47.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:46 smithi132 ceph-mon[70676]: pgmap v2618: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:49.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:48 smithi132 ceph-mon[70676]: pgmap v2619: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:49.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:48 smithi088 ceph-mon[88643]: pgmap v2619: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:51.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:50 smithi132 ceph-mon[70676]: pgmap v2620: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:51.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:50 smithi088 ceph-mon[88643]: pgmap v2620: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:53.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:52 smithi132 ceph-mon[70676]: pgmap v2621: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:53.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:30:53.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:30:53.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:30:53.224 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:52 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:30:53.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:52 smithi088 ceph-mon[88643]: pgmap v2621: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:53.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:30:53.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:30:53.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:30:53.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:52 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:30:55.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:54 smithi132 ceph-mon[70676]: pgmap v2622: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:55.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:54 smithi088 ceph-mon[88643]: pgmap v2622: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:57.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:56 smithi132 ceph-mon[70676]: pgmap v2623: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:57.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:56 smithi088 ceph-mon[88643]: pgmap v2623: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:59.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:58 smithi132 ceph-mon[70676]: pgmap v2624: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:59.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:30:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:30:59.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:58 smithi088 ceph-mon[88643]: pgmap v2624: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:30:59.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:30:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:31:01.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:00 smithi132 ceph-mon[70676]: pgmap v2625: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:01.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:00 smithi088 ceph-mon[88643]: pgmap v2625: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:03.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:02 smithi132 ceph-mon[70676]: pgmap v2626: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:03.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:02 smithi088 ceph-mon[88643]: pgmap v2626: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:05.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:04 smithi132 ceph-mon[70676]: pgmap v2627: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:05.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:04 smithi088 ceph-mon[88643]: pgmap v2627: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:07.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:06 smithi132 ceph-mon[70676]: pgmap v2628: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:07.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:06 smithi088 ceph-mon[88643]: pgmap v2628: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:09.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:08 smithi132 ceph-mon[70676]: pgmap v2629: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:09.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:08 smithi088 ceph-mon[88643]: pgmap v2629: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:11.223 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:10 smithi132 ceph-mon[70676]: pgmap v2630: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:11.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:10 smithi088 ceph-mon[88643]: pgmap v2630: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:13.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:12 smithi088 ceph-mon[88643]: pgmap v2631: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:13.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:12 smithi132 ceph-mon[70676]: pgmap v2631: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:14.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:31:14.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:31:15.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:14 smithi088 ceph-mon[88643]: pgmap v2632: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:15.374 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:14 smithi132 ceph-mon[70676]: pgmap v2632: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:17.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:16 smithi088 ceph-mon[88643]: pgmap v2633: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:17.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:16 smithi132 ceph-mon[70676]: pgmap v2633: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:19.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:18 smithi088 ceph-mon[88643]: pgmap v2634: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:19.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:18 smithi132 ceph-mon[70676]: pgmap v2634: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:21.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:21 smithi088 ceph-mon[88643]: pgmap v2635: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:21.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:21 smithi132 ceph-mon[70676]: pgmap v2635: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:23.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:23 smithi088 ceph-mon[88643]: pgmap v2636: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:23.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:23 smithi132 ceph-mon[70676]: pgmap v2636: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:25.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:25 smithi088 ceph-mon[88643]: pgmap v2637: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:25.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:25 smithi132 ceph-mon[70676]: pgmap v2637: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:27.267 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:27 smithi088 ceph-mon[88643]: pgmap v2638: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:27.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:27 smithi132 ceph-mon[70676]: pgmap v2638: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:29.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:29 smithi088 ceph-mon[88643]: pgmap v2639: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:29.321 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:31:29.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:29 smithi132 ceph-mon[70676]: pgmap v2639: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:29.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:31:31.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:31 smithi088 ceph-mon[88643]: pgmap v2640: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:31.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:31 smithi132 ceph-mon[70676]: pgmap v2640: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:33.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:33 smithi088 ceph-mon[88643]: pgmap v2641: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:33.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:33 smithi132 ceph-mon[70676]: pgmap v2641: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:35.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:35 smithi088 ceph-mon[88643]: pgmap v2642: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:35.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:35 smithi132 ceph-mon[70676]: pgmap v2642: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:37.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:37 smithi088 ceph-mon[88643]: pgmap v2643: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:37.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:37 smithi132 ceph-mon[70676]: pgmap v2643: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:39.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:39 smithi088 ceph-mon[88643]: pgmap v2644: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:39.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:39 smithi132 ceph-mon[70676]: pgmap v2644: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:41.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:41 smithi088 ceph-mon[88643]: pgmap v2645: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:41.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:41 smithi132 ceph-mon[70676]: pgmap v2645: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:43.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:43 smithi088 ceph-mon[88643]: pgmap v2646: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:43.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:43 smithi132 ceph-mon[70676]: pgmap v2646: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:44.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:31:44.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:31:45.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:45 smithi088 ceph-mon[88643]: pgmap v2647: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:45.373 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:45 smithi132 ceph-mon[70676]: pgmap v2647: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:47.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:47 smithi088 ceph-mon[88643]: pgmap v2648: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:47.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:47 smithi132 ceph-mon[70676]: pgmap v2648: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:49.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:49 smithi088 ceph-mon[88643]: pgmap v2649: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:49.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:49 smithi132 ceph-mon[70676]: pgmap v2649: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:51.222 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:51 smithi132 ceph-mon[70676]: pgmap v2650: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:51.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:51 smithi088 ceph-mon[88643]: pgmap v2650: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:53.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:53 smithi088 ceph-mon[88643]: pgmap v2651: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:53.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:53 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:31:53.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:53 smithi132 ceph-mon[70676]: pgmap v2651: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:53.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:53 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:31:54.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:31:54.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:31:54.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:31:54.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:31:54.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:31:54.473 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:31:55.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:55 smithi088 ceph-mon[88643]: pgmap v2652: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:55.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:55 smithi132 ceph-mon[70676]: pgmap v2652: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:57.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:57 smithi088 ceph-mon[88643]: pgmap v2653: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:57.474 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:57 smithi132 ceph-mon[70676]: pgmap v2653: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:59.323 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:59 smithi088 ceph-mon[88643]: pgmap v2654: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:59.324 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:31:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:31:59.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:59 smithi132 ceph-mon[70676]: pgmap v2654: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:31:59.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:31:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:32:01.222 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:01 smithi132 ceph-mon[70676]: pgmap v2655: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:01.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:01 smithi088 ceph-mon[88643]: pgmap v2655: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:03.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:03 smithi088 ceph-mon[88643]: pgmap v2656: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:03.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:03 smithi132 ceph-mon[70676]: pgmap v2656: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:05.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:05 smithi088 ceph-mon[88643]: pgmap v2657: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:05.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:05 smithi132 ceph-mon[70676]: pgmap v2657: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:07.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:07 smithi088 ceph-mon[88643]: pgmap v2658: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:07.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:07 smithi132 ceph-mon[70676]: pgmap v2658: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:09.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:09 smithi088 ceph-mon[88643]: pgmap v2659: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:09.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:09 smithi132 ceph-mon[70676]: pgmap v2659: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:11.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:11 smithi132 ceph-mon[70676]: pgmap v2660: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:11.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:11 smithi088 ceph-mon[88643]: pgmap v2660: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:13.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:13 smithi132 ceph-mon[70676]: pgmap v2661: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:13.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:13 smithi088 ceph-mon[88643]: pgmap v2661: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:14.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:32:14.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:32:15.373 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:15 smithi132 ceph-mon[70676]: pgmap v2662: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:15.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:15 smithi088 ceph-mon[88643]: pgmap v2662: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:17.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:17 smithi132 ceph-mon[70676]: pgmap v2663: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:17.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:17 smithi088 ceph-mon[88643]: pgmap v2663: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:19.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:19 smithi132 ceph-mon[70676]: pgmap v2664: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:19.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:19 smithi088 ceph-mon[88643]: pgmap v2664: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:21.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:21 smithi132 ceph-mon[70676]: pgmap v2665: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:21.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:21 smithi088 ceph-mon[88643]: pgmap v2665: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:23.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:23 smithi088 ceph-mon[88643]: pgmap v2666: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:23.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:23 smithi132 ceph-mon[70676]: pgmap v2666: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:25.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:25 smithi132 ceph-mon[70676]: pgmap v2667: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:25.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:25 smithi088 ceph-mon[88643]: pgmap v2667: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:27.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:27 smithi132 ceph-mon[70676]: pgmap v2668: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:27.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:27 smithi088 ceph-mon[88643]: pgmap v2668: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:29.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:29 smithi132 ceph-mon[70676]: pgmap v2669: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:29.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:32:29.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:29 smithi088 ceph-mon[88643]: pgmap v2669: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:29.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:32:31.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:31 smithi132 ceph-mon[70676]: pgmap v2670: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:31.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:31 smithi088 ceph-mon[88643]: pgmap v2670: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:33.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:33 smithi088 ceph-mon[88643]: pgmap v2671: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:33.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:33 smithi132 ceph-mon[70676]: pgmap v2671: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:35.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:35 smithi132 ceph-mon[70676]: pgmap v2672: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:35.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:35 smithi088 ceph-mon[88643]: pgmap v2672: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:37.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:37 smithi132 ceph-mon[70676]: pgmap v2673: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:37.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:37 smithi088 ceph-mon[88643]: pgmap v2673: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:39.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:39 smithi132 ceph-mon[70676]: pgmap v2674: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:39.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:39 smithi088 ceph-mon[88643]: pgmap v2674: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:41.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:41 smithi132 ceph-mon[70676]: pgmap v2675: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:41.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:41 smithi088 ceph-mon[88643]: pgmap v2675: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:43.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:43 smithi132 ceph-mon[70676]: pgmap v2676: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:43.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:43 smithi088 ceph-mon[88643]: pgmap v2676: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:44.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:32:44.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:32:45.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:45 smithi132 ceph-mon[70676]: pgmap v2677: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:45.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:45 smithi088 ceph-mon[88643]: pgmap v2677: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:47.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:47 smithi132 ceph-mon[70676]: pgmap v2678: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:47.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:47 smithi088 ceph-mon[88643]: pgmap v2678: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:49.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:49 smithi132 ceph-mon[70676]: pgmap v2679: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:49.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:49 smithi088 ceph-mon[88643]: pgmap v2679: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:51.452 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:51 smithi132 ceph-mon[70676]: pgmap v2680: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:51.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:51 smithi088 ceph-mon[88643]: pgmap v2680: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:53.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:53 smithi132 ceph-mon[70676]: pgmap v2681: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:53.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:53 smithi088 ceph-mon[88643]: pgmap v2681: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:54.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:32:54.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:54.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:54.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:54.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:54 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:54.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:32:54.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:54.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:54.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:54.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:54 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:55.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:55 smithi088 ceph-mon[88643]: pgmap v2682: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:55.409 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:55 smithi132 ceph-mon[70676]: pgmap v2682: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:57.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:57 smithi088 ceph-mon[88643]: pgmap v2683: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:57.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:57 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:57.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:57 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:57.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:57 smithi132 ceph-mon[70676]: pgmap v2683: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:57.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:57 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:57.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:57 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:59.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:59 smithi132 ceph-mon[70676]: pgmap v2684: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:59.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:32:59.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:59.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:59.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:32:59.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:32:59.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:32:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:59.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:59 smithi088 ceph-mon[88643]: pgmap v2684: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:32:59.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:32:59.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:59.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:32:59.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:32:59.570 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:32:59.571 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:32:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:33:01.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:01 smithi132 ceph-mon[70676]: pgmap v2685: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:01.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:01 smithi088 ceph-mon[88643]: pgmap v2685: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:03.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:03 smithi132 ceph-mon[70676]: pgmap v2686: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:03.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:03 smithi088 ceph-mon[88643]: pgmap v2686: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:05.341 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:05 smithi088 ceph-mon[88643]: pgmap v2687: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:05.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:05 smithi132 ceph-mon[70676]: pgmap v2687: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:07.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:07 smithi088 ceph-mon[88643]: pgmap v2688: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:07.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:07 smithi132 ceph-mon[70676]: pgmap v2688: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:09.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:09 smithi088 ceph-mon[88643]: pgmap v2689: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:09.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:09 smithi132 ceph-mon[70676]: pgmap v2689: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:11.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:11 smithi088 ceph-mon[88643]: pgmap v2690: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:11.651 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:11 smithi132 ceph-mon[70676]: pgmap v2690: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:13.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:13 smithi088 ceph-mon[88643]: pgmap v2691: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:13.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:13 smithi132 ceph-mon[70676]: pgmap v2691: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:14.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:33:14.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:33:15.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:15 smithi088 ceph-mon[88643]: pgmap v2692: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:15.623 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:15 smithi132 ceph-mon[70676]: pgmap v2692: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:17.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:17 smithi088 ceph-mon[88643]: pgmap v2693: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:17.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:17 smithi132 ceph-mon[70676]: pgmap v2693: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:19.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:19 smithi088 ceph-mon[88643]: pgmap v2694: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:19.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:19 smithi132 ceph-mon[70676]: pgmap v2694: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:21.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:21 smithi088 ceph-mon[88643]: pgmap v2695: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:21.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:21 smithi132 ceph-mon[70676]: pgmap v2695: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:23.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:23 smithi088 ceph-mon[88643]: pgmap v2696: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:23.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:23 smithi132 ceph-mon[70676]: pgmap v2696: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:25.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:25 smithi088 ceph-mon[88643]: pgmap v2697: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:25.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:25 smithi132 ceph-mon[70676]: pgmap v2697: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:27.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:27 smithi088 ceph-mon[88643]: pgmap v2698: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:27.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:27 smithi132 ceph-mon[70676]: pgmap v2698: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:29.531 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:29 smithi088 ceph-mon[88643]: pgmap v2699: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:29.531 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:33:29.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:29 smithi132 ceph-mon[70676]: pgmap v2699: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:29.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:33:31.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:31 smithi088 ceph-mon[88643]: pgmap v2700: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:31.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:31 smithi132 ceph-mon[70676]: pgmap v2700: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:33.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:33 smithi088 ceph-mon[88643]: pgmap v2701: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:33.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:33 smithi132 ceph-mon[70676]: pgmap v2701: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:35.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:35 smithi088 ceph-mon[88643]: pgmap v2702: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:35.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:35 smithi132 ceph-mon[70676]: pgmap v2702: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:37.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:37 smithi088 ceph-mon[88643]: pgmap v2703: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:37.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:37 smithi132 ceph-mon[70676]: pgmap v2703: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:39.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:39 smithi088 ceph-mon[88643]: pgmap v2704: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:39.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:39 smithi132 ceph-mon[70676]: pgmap v2704: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:41.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:41 smithi088 ceph-mon[88643]: pgmap v2705: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:41.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:41 smithi132 ceph-mon[70676]: pgmap v2705: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:43.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:43 smithi088 ceph-mon[88643]: pgmap v2706: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:43.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:43 smithi132 ceph-mon[70676]: pgmap v2706: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:44.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:33:44.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:33:45.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:45 smithi088 ceph-mon[88643]: pgmap v2707: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:45.622 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:45 smithi132 ceph-mon[70676]: pgmap v2707: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:47.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:47 smithi088 ceph-mon[88643]: pgmap v2708: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:47.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:47 smithi132 ceph-mon[70676]: pgmap v2708: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:49.569 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:49 smithi088 ceph-mon[88643]: pgmap v2709: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:49.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:49 smithi132 ceph-mon[70676]: pgmap v2709: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:50.721 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:50 smithi132 ceph-mon[70676]: pgmap v2710: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:50.818 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:50 smithi088 ceph-mon[88643]: pgmap v2710: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:53.221 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:52 smithi132 ceph-mon[70676]: pgmap v2711: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:53.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:52 smithi088 ceph-mon[88643]: pgmap v2711: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:55.221 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:54 smithi132 ceph-mon[70676]: pgmap v2712: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:55.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:54 smithi088 ceph-mon[88643]: pgmap v2712: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:57.221 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:56 smithi132 ceph-mon[70676]: pgmap v2713: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:57.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:56 smithi088 ceph-mon[88643]: pgmap v2713: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:59.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:58 smithi088 ceph-mon[88643]: pgmap v2714: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:59.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:33:59.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:58 smithi132 ceph-mon[70676]: pgmap v2714: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:33:59.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:34:00.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:34:00.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:34:00.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:34:00.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:33:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:34:00.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:34:00.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:34:00.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:34:00.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:33:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:34:01.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:00 smithi088 ceph-mon[88643]: pgmap v2715: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:01.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:00 smithi132 ceph-mon[70676]: pgmap v2715: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:03.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:02 smithi088 ceph-mon[88643]: pgmap v2716: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:03.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:02 smithi132 ceph-mon[70676]: pgmap v2716: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:05.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:04 smithi088 ceph-mon[88643]: pgmap v2717: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:05.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:04 smithi132 ceph-mon[70676]: pgmap v2717: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:07.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:07 smithi088 ceph-mon[88643]: pgmap v2718: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:07.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:07 smithi132 ceph-mon[70676]: pgmap v2718: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:09.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:09 smithi088 ceph-mon[88643]: pgmap v2719: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:09.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:09 smithi132 ceph-mon[70676]: pgmap v2719: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:11.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:11 smithi088 ceph-mon[88643]: pgmap v2720: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:11.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:11 smithi132 ceph-mon[70676]: pgmap v2720: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:13.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:13 smithi088 ceph-mon[88643]: pgmap v2721: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:13.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:13 smithi132 ceph-mon[70676]: pgmap v2721: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:14.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:34:14.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:34:15.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:15 smithi088 ceph-mon[88643]: pgmap v2722: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:15.372 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:15 smithi132 ceph-mon[70676]: pgmap v2722: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:17.289 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:17 smithi132 ceph-mon[70676]: pgmap v2723: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:17.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:17 smithi088 ceph-mon[88643]: pgmap v2723: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:19.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:19 smithi088 ceph-mon[88643]: pgmap v2724: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:19.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:19 smithi132 ceph-mon[70676]: pgmap v2724: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:21.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:21 smithi088 ceph-mon[88643]: pgmap v2725: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:21.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:21 smithi132 ceph-mon[70676]: pgmap v2725: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:23.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:23 smithi088 ceph-mon[88643]: pgmap v2726: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:23.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:23 smithi132 ceph-mon[70676]: pgmap v2726: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:25.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:25 smithi088 ceph-mon[88643]: pgmap v2727: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:25.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:25 smithi132 ceph-mon[70676]: pgmap v2727: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:27.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:27 smithi088 ceph-mon[88643]: pgmap v2728: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:27.383 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:27 smithi132 ceph-mon[70676]: pgmap v2728: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:29.312 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:29 smithi088 ceph-mon[88643]: pgmap v2729: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:29.312 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:34:29.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:29 smithi132 ceph-mon[70676]: pgmap v2729: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:29.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:34:31.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:31 smithi088 ceph-mon[88643]: pgmap v2730: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:31.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:31 smithi132 ceph-mon[70676]: pgmap v2730: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:33.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:33 smithi088 ceph-mon[88643]: pgmap v2731: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:33.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:33 smithi132 ceph-mon[70676]: pgmap v2731: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:35.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:35 smithi088 ceph-mon[88643]: pgmap v2732: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:35.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:35 smithi132 ceph-mon[70676]: pgmap v2732: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:37.220 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:36 smithi132 ceph-mon[70676]: pgmap v2733: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:37.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:36 smithi088 ceph-mon[88643]: pgmap v2733: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:39.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:38 smithi088 ceph-mon[88643]: pgmap v2734: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:39.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:38 smithi132 ceph-mon[70676]: pgmap v2734: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:41.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:40 smithi088 ceph-mon[88643]: pgmap v2735: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:41.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:40 smithi132 ceph-mon[70676]: pgmap v2735: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:43.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:42 smithi088 ceph-mon[88643]: pgmap v2736: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:43.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:42 smithi132 ceph-mon[70676]: pgmap v2736: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:44.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:34:44.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:34:45.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:45 smithi088 ceph-mon[88643]: pgmap v2737: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:45.371 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:45 smithi132 ceph-mon[70676]: pgmap v2737: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:47.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:47 smithi088 ceph-mon[88643]: pgmap v2738: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:47.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:47 smithi132 ceph-mon[70676]: pgmap v2738: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:49.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:49 smithi088 ceph-mon[88643]: pgmap v2739: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:49.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:49 smithi132 ceph-mon[70676]: pgmap v2739: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:51.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:51 smithi088 ceph-mon[88643]: pgmap v2740: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:51.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:51 smithi132 ceph-mon[70676]: pgmap v2740: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:53.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:53 smithi088 ceph-mon[88643]: pgmap v2741: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:53.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:53 smithi132 ceph-mon[70676]: pgmap v2741: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:55.319 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:55 smithi088 ceph-mon[88643]: pgmap v2742: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:55.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:55 smithi132 ceph-mon[70676]: pgmap v2742: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:57.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:57 smithi088 ceph-mon[88643]: pgmap v2743: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:57.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:57 smithi132 ceph-mon[70676]: pgmap v2743: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:59.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:59 smithi088 ceph-mon[88643]: pgmap v2744: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:59.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:34:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:34:59.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:59 smithi132 ceph-mon[70676]: pgmap v2744: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:34:59.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:34:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:35:00.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:00 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:35:00.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:00 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:35:01.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:01 smithi088 ceph-mon[88643]: pgmap v2745: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:01.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:01 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:35:01.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:01 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:35:01.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:01 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:35:01.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:01 smithi132 ceph-mon[70676]: pgmap v2745: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:01.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:01 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:35:01.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:01 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:35:01.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:01 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:35:03.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:03 smithi088 ceph-mon[88643]: pgmap v2746: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:03.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:03 smithi132 ceph-mon[70676]: pgmap v2746: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:05.320 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:05 smithi088 ceph-mon[88643]: pgmap v2747: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:05.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:05 smithi132 ceph-mon[70676]: pgmap v2747: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:07.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:07 smithi088 ceph-mon[88643]: pgmap v2748: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:07.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:07 smithi132 ceph-mon[70676]: pgmap v2748: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:09.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:09 smithi088 ceph-mon[88643]: pgmap v2749: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:09.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:09 smithi132 ceph-mon[70676]: pgmap v2749: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:11.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:11 smithi088 ceph-mon[88643]: pgmap v2750: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:11.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:11 smithi132 ceph-mon[70676]: pgmap v2750: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:13.220 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:13 smithi132 ceph-mon[70676]: pgmap v2751: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:13.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:13 smithi088 ceph-mon[88643]: pgmap v2751: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:14.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:35:14.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:35:15.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:15 smithi088 ceph-mon[88643]: pgmap v2752: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:15.371 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:15 smithi132 ceph-mon[70676]: pgmap v2752: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:17.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:17 smithi088 ceph-mon[88643]: pgmap v2753: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:17.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:17 smithi132 ceph-mon[70676]: pgmap v2753: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:19.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:19 smithi088 ceph-mon[88643]: pgmap v2754: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:19.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:19 smithi132 ceph-mon[70676]: pgmap v2754: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:21.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:21 smithi088 ceph-mon[88643]: pgmap v2755: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:21.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:21 smithi132 ceph-mon[70676]: pgmap v2755: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:23.221 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:23 smithi132 ceph-mon[70676]: pgmap v2756: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:23.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:23 smithi088 ceph-mon[88643]: pgmap v2756: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:25.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:25 smithi088 ceph-mon[88643]: pgmap v2757: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:25.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:25 smithi132 ceph-mon[70676]: pgmap v2757: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:27.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:27 smithi088 ceph-mon[88643]: pgmap v2758: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:27.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:27 smithi132 ceph-mon[70676]: pgmap v2758: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:29.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:29 smithi132 ceph-mon[70676]: pgmap v2759: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:29.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:35:29.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:29 smithi088 ceph-mon[88643]: pgmap v2759: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:29.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:35:31.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:31 smithi132 ceph-mon[70676]: pgmap v2760: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:31.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:31 smithi088 ceph-mon[88643]: pgmap v2760: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:33.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:33 smithi132 ceph-mon[70676]: pgmap v2761: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:33.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:33 smithi088 ceph-mon[88643]: pgmap v2761: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:35.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:35 smithi088 ceph-mon[88643]: pgmap v2762: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:35.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:35 smithi132 ceph-mon[70676]: pgmap v2762: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:37.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:37 smithi132 ceph-mon[70676]: pgmap v2763: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:37.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:37 smithi088 ceph-mon[88643]: pgmap v2763: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:39.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:39 smithi132 ceph-mon[70676]: pgmap v2764: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:39.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:39 smithi088 ceph-mon[88643]: pgmap v2764: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:41.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:41 smithi132 ceph-mon[70676]: pgmap v2765: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:41.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:41 smithi088 ceph-mon[88643]: pgmap v2765: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:43.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:43 smithi132 ceph-mon[70676]: pgmap v2766: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:43.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:43 smithi088 ceph-mon[88643]: pgmap v2766: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:44.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:35:44.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:35:45.371 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:45 smithi132 ceph-mon[70676]: pgmap v2767: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:45.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:45 smithi088 ceph-mon[88643]: pgmap v2767: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:47.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:47 smithi132 ceph-mon[70676]: pgmap v2768: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:47.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:47 smithi088 ceph-mon[88643]: pgmap v2768: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:49.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:49 smithi132 ceph-mon[70676]: pgmap v2769: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:49.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:49 smithi088 ceph-mon[88643]: pgmap v2769: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:51.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:51 smithi132 ceph-mon[70676]: pgmap v2770: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:51.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:51 smithi088 ceph-mon[88643]: pgmap v2770: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:53.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:53 smithi132 ceph-mon[70676]: pgmap v2771: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:53.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:53 smithi088 ceph-mon[88643]: pgmap v2771: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:55.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:55 smithi132 ceph-mon[70676]: pgmap v2772: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:55.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:55 smithi088 ceph-mon[88643]: pgmap v2772: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:57.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:57 smithi132 ceph-mon[70676]: pgmap v2773: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:57.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:57 smithi088 ceph-mon[88643]: pgmap v2773: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:59.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:59 smithi132 ceph-mon[70676]: pgmap v2774: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:59.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:35:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:35:59.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:59 smithi088 ceph-mon[88643]: pgmap v2774: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:35:59.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:35:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:36:01.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:01 smithi132 ceph-mon[70676]: pgmap v2775: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:01.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:01 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:36:01.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:01 smithi088 ceph-mon[88643]: pgmap v2775: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:01.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:01 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:36:02.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:36:02.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:36:02.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:36:02.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:36:02.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:36:02.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:36:03.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:03 smithi088 ceph-mon[88643]: pgmap v2776: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:03.720 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:03 smithi132 ceph-mon[70676]: pgmap v2776: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:05.383 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:05 smithi088 ceph-mon[88643]: pgmap v2777: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:05.720 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:05 smithi132 ceph-mon[70676]: pgmap v2777: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:07.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:07 smithi088 ceph-mon[88643]: pgmap v2778: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:07.720 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:07 smithi132 ceph-mon[70676]: pgmap v2778: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:09.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:09 smithi088 ceph-mon[88643]: pgmap v2779: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:09.720 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:09 smithi132 ceph-mon[70676]: pgmap v2779: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:11.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:11 smithi088 ceph-mon[88643]: pgmap v2780: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:11.719 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:11 smithi132 ceph-mon[70676]: pgmap v2780: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:13.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:13 smithi088 ceph-mon[88643]: pgmap v2781: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:13.720 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:13 smithi132 ceph-mon[70676]: pgmap v2781: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:14.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:36:14.719 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:36:15.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:15 smithi088 ceph-mon[88643]: pgmap v2782: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:15.621 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:15 smithi132 ceph-mon[70676]: pgmap v2782: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:17.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:17 smithi088 ceph-mon[88643]: pgmap v2783: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:17.719 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:17 smithi132 ceph-mon[70676]: pgmap v2783: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:19.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:19 smithi088 ceph-mon[88643]: pgmap v2784: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:19.720 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:19 smithi132 ceph-mon[70676]: pgmap v2784: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:21.719 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:21 smithi132 ceph-mon[70676]: pgmap v2785: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:21.817 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:21 smithi088 ceph-mon[88643]: pgmap v2785: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:23.719 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:23 smithi132 ceph-mon[70676]: pgmap v2786: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:23.817 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:23 smithi088 ceph-mon[88643]: pgmap v2786: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:24.681 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:24 smithi088 ceph-mon[88643]: pgmap v2787: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:24.719 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:24 smithi132 ceph-mon[70676]: pgmap v2787: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:27.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:27 smithi088 ceph-mon[88643]: pgmap v2788: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:27.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:27 smithi132 ceph-mon[70676]: pgmap v2788: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:29.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:29 smithi088 ceph-mon[88643]: pgmap v2789: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:29.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:36:29.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:29 smithi132 ceph-mon[70676]: pgmap v2789: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:29.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:36:31.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:31 smithi088 ceph-mon[88643]: pgmap v2790: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:31.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:31 smithi132 ceph-mon[70676]: pgmap v2790: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:33.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:33 smithi088 ceph-mon[88643]: pgmap v2791: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:33.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:33 smithi132 ceph-mon[70676]: pgmap v2791: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:35.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:35 smithi088 ceph-mon[88643]: pgmap v2792: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:35.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:35 smithi132 ceph-mon[70676]: pgmap v2792: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:37.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:37 smithi088 ceph-mon[88643]: pgmap v2793: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:37.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:37 smithi132 ceph-mon[70676]: pgmap v2793: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:39.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:39 smithi088 ceph-mon[88643]: pgmap v2794: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:39.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:39 smithi132 ceph-mon[70676]: pgmap v2794: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:41.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:41 smithi088 ceph-mon[88643]: pgmap v2795: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:41.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:41 smithi132 ceph-mon[70676]: pgmap v2795: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:43.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:43 smithi088 ceph-mon[88643]: pgmap v2796: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:43.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:43 smithi132 ceph-mon[70676]: pgmap v2796: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:44.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:36:44.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:36:45.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:45 smithi088 ceph-mon[88643]: pgmap v2797: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:45.371 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:45 smithi132 ceph-mon[70676]: pgmap v2797: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:47.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:47 smithi088 ceph-mon[88643]: pgmap v2798: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:47.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:47 smithi132 ceph-mon[70676]: pgmap v2798: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:49.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:49 smithi088 ceph-mon[88643]: pgmap v2799: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:49.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:49 smithi132 ceph-mon[70676]: pgmap v2799: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:51.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:51 smithi088 ceph-mon[88643]: pgmap v2800: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:51.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:51 smithi132 ceph-mon[70676]: pgmap v2800: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:53.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:53 smithi088 ceph-mon[88643]: pgmap v2801: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:53.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:53 smithi132 ceph-mon[70676]: pgmap v2801: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:55.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:55 smithi088 ceph-mon[88643]: pgmap v2802: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:55.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:55 smithi132 ceph-mon[70676]: pgmap v2802: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:57.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:57 smithi088 ceph-mon[88643]: pgmap v2803: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:57.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:57 smithi132 ceph-mon[70676]: pgmap v2803: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:59.219 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:59 smithi132 ceph-mon[70676]: pgmap v2804: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:59.219 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:36:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:36:59.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:59 smithi088 ceph-mon[88643]: pgmap v2804: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:36:59.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:36:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:37:01.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:01 smithi088 ceph-mon[88643]: pgmap v2805: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:01.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:01 smithi132 ceph-mon[70676]: pgmap v2805: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:02.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:37:02.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:37:02.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:37:02.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:02 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:37:02.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:37:02.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:37:02.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:37:02.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:02 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:37:03.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:03 smithi132 ceph-mon[70676]: pgmap v2806: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:03.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:03 smithi088 ceph-mon[88643]: pgmap v2806: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:05.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:05 smithi132 ceph-mon[70676]: pgmap v2807: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:05.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:05 smithi088 ceph-mon[88643]: pgmap v2807: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:07.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:07 smithi132 ceph-mon[70676]: pgmap v2808: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:07.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:07 smithi088 ceph-mon[88643]: pgmap v2808: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:09.219 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:09 smithi132 ceph-mon[70676]: pgmap v2809: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:09.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:09 smithi088 ceph-mon[88643]: pgmap v2809: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:11.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:11 smithi132 ceph-mon[70676]: pgmap v2810: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:11.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:11 smithi088 ceph-mon[88643]: pgmap v2810: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:13.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:13 smithi132 ceph-mon[70676]: pgmap v2811: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:13.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:13 smithi088 ceph-mon[88643]: pgmap v2811: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:14.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:37:14.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:37:15.370 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:15 smithi132 ceph-mon[70676]: pgmap v2812: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:15.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:15 smithi088 ceph-mon[88643]: pgmap v2812: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:17.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:17 smithi132 ceph-mon[70676]: pgmap v2813: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:17.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:17 smithi088 ceph-mon[88643]: pgmap v2813: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:19.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:19 smithi132 ceph-mon[70676]: pgmap v2814: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:19.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:19 smithi088 ceph-mon[88643]: pgmap v2814: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:21.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:21 smithi132 ceph-mon[70676]: pgmap v2815: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:21.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:21 smithi088 ceph-mon[88643]: pgmap v2815: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:23.472 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:23 smithi132 ceph-mon[70676]: pgmap v2816: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:23.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:23 smithi088 ceph-mon[88643]: pgmap v2816: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:25.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:25 smithi132 ceph-mon[70676]: pgmap v2817: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:25.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:25 smithi088 ceph-mon[88643]: pgmap v2817: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:27.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:27 smithi132 ceph-mon[70676]: pgmap v2818: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:27.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:27 smithi088 ceph-mon[88643]: pgmap v2818: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:29.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:29 smithi132 ceph-mon[70676]: pgmap v2819: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:29.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:37:29.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:29 smithi088 ceph-mon[88643]: pgmap v2819: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:29.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:37:31.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:31 smithi132 ceph-mon[70676]: pgmap v2820: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:31.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:31 smithi088 ceph-mon[88643]: pgmap v2820: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:33.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:33 smithi132 ceph-mon[70676]: pgmap v2821: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:33.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:33 smithi088 ceph-mon[88643]: pgmap v2821: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:35.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:35 smithi132 ceph-mon[70676]: pgmap v2822: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:35.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:35 smithi088 ceph-mon[88643]: pgmap v2822: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:37.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:37 smithi132 ceph-mon[70676]: pgmap v2823: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:37.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:37 smithi088 ceph-mon[88643]: pgmap v2823: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:39.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:39 smithi132 ceph-mon[70676]: pgmap v2824: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:39.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:39 smithi088 ceph-mon[88643]: pgmap v2824: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:41.458 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:41 smithi088 ceph-mon[88643]: pgmap v2825: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:41.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:41 smithi132 ceph-mon[70676]: pgmap v2825: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:43.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:43 smithi132 ceph-mon[70676]: pgmap v2826: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:43.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:43 smithi088 ceph-mon[88643]: pgmap v2826: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:44.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:37:44.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:37:45.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:45 smithi132 ceph-mon[70676]: pgmap v2827: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:45.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:45 smithi088 ceph-mon[88643]: pgmap v2827: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:47.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:47 smithi132 ceph-mon[70676]: pgmap v2828: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:47.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:47 smithi088 ceph-mon[88643]: pgmap v2828: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:49.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:49 smithi132 ceph-mon[70676]: pgmap v2829: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:49.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:49 smithi088 ceph-mon[88643]: pgmap v2829: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:51.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:51 smithi132 ceph-mon[70676]: pgmap v2830: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:51.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:51 smithi088 ceph-mon[88643]: pgmap v2830: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:53.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:53 smithi132 ceph-mon[70676]: pgmap v2831: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:53.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:53 smithi088 ceph-mon[88643]: pgmap v2831: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:55.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:55 smithi132 ceph-mon[70676]: pgmap v2832: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:55.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:55 smithi088 ceph-mon[88643]: pgmap v2832: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:57.067 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:56 smithi088 ceph-mon[88643]: pgmap v2833: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:57.219 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:56 smithi132 ceph-mon[70676]: pgmap v2833: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:59.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:59 smithi088 ceph-mon[88643]: pgmap v2834: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:59.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:37:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:37:59.448 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:59 smithi132 ceph-mon[70676]: pgmap v2834: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:37:59.448 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:37:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:38:01.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:01 smithi088 ceph-mon[88643]: pgmap v2835: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:01.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:01 smithi132 ceph-mon[70676]: pgmap v2835: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:03.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:03 smithi088 ceph-mon[88643]: pgmap v2836: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:03.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:03 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:38:03.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:03 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:38:03.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:03 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:38:03.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:03 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:38:03.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:03 smithi132 ceph-mon[70676]: pgmap v2836: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:03.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:03 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:38:03.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:03 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:38:03.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:03 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:38:03.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:03 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:38:05.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:05 smithi088 ceph-mon[88643]: pgmap v2837: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:05.471 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:05 smithi132 ceph-mon[70676]: pgmap v2837: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:07.317 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:07 smithi088 ceph-mon[88643]: pgmap v2838: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:07.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:07 smithi132 ceph-mon[70676]: pgmap v2838: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:09.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:09 smithi088 ceph-mon[88643]: pgmap v2839: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:09.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:09 smithi132 ceph-mon[70676]: pgmap v2839: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:11.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:11 smithi088 ceph-mon[88643]: pgmap v2840: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:11.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:11 smithi132 ceph-mon[70676]: pgmap v2840: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:13.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:13 smithi088 ceph-mon[88643]: pgmap v2841: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:13.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:13 smithi132 ceph-mon[70676]: pgmap v2841: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:14.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:38:14.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:38:15.318 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:15 smithi088 ceph-mon[88643]: pgmap v2842: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:15.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:15 smithi132 ceph-mon[70676]: pgmap v2842: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:17.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:17 smithi088 ceph-mon[88643]: pgmap v2843: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:17.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:17 smithi132 ceph-mon[70676]: pgmap v2843: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:19.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:19 smithi088 ceph-mon[88643]: pgmap v2844: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:19.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:19 smithi132 ceph-mon[70676]: pgmap v2844: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:21.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:21 smithi132 ceph-mon[70676]: pgmap v2845: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:21.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:21 smithi088 ceph-mon[88643]: pgmap v2845: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:23.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:23 smithi132 ceph-mon[70676]: pgmap v2846: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:23.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:23 smithi088 ceph-mon[88643]: pgmap v2846: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:25.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:25 smithi132 ceph-mon[70676]: pgmap v2847: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:25.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:25 smithi088 ceph-mon[88643]: pgmap v2847: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:27.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:27 smithi088 ceph-mon[88643]: pgmap v2848: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:27.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:27 smithi132 ceph-mon[70676]: pgmap v2848: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:29.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:29 smithi132 ceph-mon[70676]: pgmap v2849: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:29.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:38:29.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:29 smithi088 ceph-mon[88643]: pgmap v2849: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:29.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:38:31.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:31 smithi132 ceph-mon[70676]: pgmap v2850: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:31.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:31 smithi088 ceph-mon[88643]: pgmap v2850: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:33.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:33 smithi132 ceph-mon[70676]: pgmap v2851: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:33.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:33 smithi088 ceph-mon[88643]: pgmap v2851: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:35.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:35 smithi132 ceph-mon[70676]: pgmap v2852: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:35.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:35 smithi088 ceph-mon[88643]: pgmap v2852: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:37.218 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:36 smithi132 ceph-mon[70676]: pgmap v2853: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:37.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:36 smithi088 ceph-mon[88643]: pgmap v2853: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:39.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:39 smithi088 ceph-mon[88643]: pgmap v2854: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:39.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:39 smithi132 ceph-mon[70676]: pgmap v2854: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:41.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:41 smithi088 ceph-mon[88643]: pgmap v2855: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:41.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:41 smithi132 ceph-mon[70676]: pgmap v2855: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:43.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:43 smithi088 ceph-mon[88643]: pgmap v2856: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:43.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:43 smithi132 ceph-mon[70676]: pgmap v2856: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:44.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:38:44.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:38:45.218 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:45 smithi132 ceph-mon[70676]: pgmap v2857: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:45.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:45 smithi088 ceph-mon[88643]: pgmap v2857: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:47.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:47 smithi088 ceph-mon[88643]: pgmap v2858: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:47.469 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:47 smithi132 ceph-mon[70676]: pgmap v2858: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:49.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:49 smithi088 ceph-mon[88643]: pgmap v2859: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:49.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:49 smithi132 ceph-mon[70676]: pgmap v2859: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:51.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:51 smithi088 ceph-mon[88643]: pgmap v2860: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:51.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:51 smithi132 ceph-mon[70676]: pgmap v2860: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:53.316 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:53 smithi088 ceph-mon[88643]: pgmap v2861: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:53.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:53 smithi132 ceph-mon[70676]: pgmap v2861: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:55.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:55 smithi132 ceph-mon[70676]: pgmap v2862: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:55.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:55 smithi088 ceph-mon[88643]: pgmap v2862: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:57.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:57 smithi132 ceph-mon[70676]: pgmap v2863: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:57.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:57 smithi088 ceph-mon[88643]: pgmap v2863: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:59.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:59 smithi132 ceph-mon[70676]: pgmap v2864: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:59.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:38:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:38:59.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:59 smithi088 ceph-mon[88643]: pgmap v2864: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:38:59.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:38:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:39:01.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:01 smithi132 ceph-mon[70676]: pgmap v2865: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:01.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:01 smithi088 ceph-mon[88643]: pgmap v2865: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:03.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:03 smithi132 ceph-mon[70676]: pgmap v2866: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:03.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:03 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:39:03.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:03 smithi088 ceph-mon[88643]: pgmap v2866: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:03.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:03 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:39:04.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:39:04.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:39:04.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:39:04.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:39:04.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:39:04.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:39:05.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:05 smithi132 ceph-mon[70676]: pgmap v2867: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:05.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:05 smithi088 ceph-mon[88643]: pgmap v2867: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:07.417 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:07 smithi088 ceph-mon[88643]: pgmap v2868: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:07.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:07 smithi132 ceph-mon[70676]: pgmap v2868: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:09.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:09 smithi132 ceph-mon[70676]: pgmap v2869: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:09.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:09 smithi088 ceph-mon[88643]: pgmap v2869: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:11.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:11 smithi132 ceph-mon[70676]: pgmap v2870: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:11.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:11 smithi088 ceph-mon[88643]: pgmap v2870: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:13.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:13 smithi132 ceph-mon[70676]: pgmap v2871: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:13.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:13 smithi088 ceph-mon[88643]: pgmap v2871: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:14.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:39:14.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:39:15.369 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:15 smithi132 ceph-mon[70676]: pgmap v2872: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:15.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:15 smithi088 ceph-mon[88643]: pgmap v2872: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:17.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:17 smithi132 ceph-mon[70676]: pgmap v2873: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:17.531 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:17 smithi088 ceph-mon[88643]: pgmap v2873: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:19.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:19 smithi132 ceph-mon[70676]: pgmap v2874: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:19.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:19 smithi088 ceph-mon[88643]: pgmap v2874: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:21.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:21 smithi132 ceph-mon[70676]: pgmap v2875: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:21.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:21 smithi088 ceph-mon[88643]: pgmap v2875: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:23.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:23 smithi132 ceph-mon[70676]: pgmap v2876: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:23.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:23 smithi088 ceph-mon[88643]: pgmap v2876: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:25.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:25 smithi132 ceph-mon[70676]: pgmap v2877: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:25.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:25 smithi088 ceph-mon[88643]: pgmap v2877: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:27.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:27 smithi132 ceph-mon[70676]: pgmap v2878: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:27.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:27 smithi088 ceph-mon[88643]: pgmap v2878: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:29.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:29 smithi132 ceph-mon[70676]: pgmap v2879: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:29.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:39:29.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:29 smithi088 ceph-mon[88643]: pgmap v2879: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:29.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:39:31.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:31 smithi132 ceph-mon[70676]: pgmap v2880: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:31.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:31 smithi088 ceph-mon[88643]: pgmap v2880: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:33.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:33 smithi132 ceph-mon[70676]: pgmap v2881: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:33.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:33 smithi088 ceph-mon[88643]: pgmap v2881: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:35.386 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:35 smithi132 ceph-mon[70676]: pgmap v2882: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:35.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:35 smithi088 ceph-mon[88643]: pgmap v2882: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:37.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:37 smithi132 ceph-mon[70676]: pgmap v2883: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:37.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:37 smithi088 ceph-mon[88643]: pgmap v2883: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:39.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:39 smithi132 ceph-mon[70676]: pgmap v2884: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:39.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:39 smithi088 ceph-mon[88643]: pgmap v2884: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:41.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:41 smithi132 ceph-mon[70676]: pgmap v2885: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:41.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:41 smithi088 ceph-mon[88643]: pgmap v2885: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:43.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:43 smithi132 ceph-mon[70676]: pgmap v2886: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:43.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:43 smithi088 ceph-mon[88643]: pgmap v2886: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:44.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:39:44.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:39:45.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:45 smithi132 ceph-mon[70676]: pgmap v2887: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:45.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:45 smithi088 ceph-mon[88643]: pgmap v2887: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:47.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:47 smithi132 ceph-mon[70676]: pgmap v2888: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:47.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:47 smithi088 ceph-mon[88643]: pgmap v2888: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:49.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:49 smithi132 ceph-mon[70676]: pgmap v2889: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:49.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:49 smithi088 ceph-mon[88643]: pgmap v2889: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:51.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:51 smithi132 ceph-mon[70676]: pgmap v2890: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:51.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:51 smithi088 ceph-mon[88643]: pgmap v2890: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:53.315 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:53 smithi088 ceph-mon[88643]: pgmap v2891: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:53.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:53 smithi132 ceph-mon[70676]: pgmap v2891: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:55.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:55 smithi132 ceph-mon[70676]: pgmap v2892: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:55.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:55 smithi088 ceph-mon[88643]: pgmap v2892: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:57.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:57 smithi132 ceph-mon[70676]: pgmap v2893: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:57.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:57 smithi088 ceph-mon[88643]: pgmap v2893: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:59.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:59 smithi132 ceph-mon[70676]: pgmap v2894: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:59.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:39:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:39:59.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:59 smithi088 ceph-mon[88643]: pgmap v2894: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:39:59.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:39:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:40:00.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:00 smithi132 ceph-mon[70676]: overall HEALTH_OK 2024-02-19T20:40:00.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:00 smithi088 ceph-mon[88643]: overall HEALTH_OK 2024-02-19T20:40:01.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:01 smithi132 ceph-mon[70676]: pgmap v2895: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:01.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:01 smithi088 ceph-mon[88643]: pgmap v2895: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:03.315 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:03 smithi088 ceph-mon[88643]: pgmap v2896: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:03.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:03 smithi132 ceph-mon[70676]: pgmap v2896: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:04.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:04 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:40:04.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:04 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:40:05.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:05 smithi132 ceph-mon[70676]: pgmap v2897: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:05.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:40:05.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:40:05.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:40:05.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:05 smithi088 ceph-mon[88643]: pgmap v2897: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:05.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:40:05.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:40:05.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:40:07.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:07 smithi132 ceph-mon[70676]: pgmap v2898: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:07.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:07 smithi088 ceph-mon[88643]: pgmap v2898: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:09.470 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:09 smithi132 ceph-mon[70676]: pgmap v2899: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:09.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:09 smithi088 ceph-mon[88643]: pgmap v2899: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:11.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:11 smithi132 ceph-mon[70676]: pgmap v2900: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:11.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:11 smithi088 ceph-mon[88643]: pgmap v2900: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:13.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:13 smithi132 ceph-mon[70676]: pgmap v2901: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:13.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:13 smithi088 ceph-mon[88643]: pgmap v2901: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:14.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:40:14.567 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:40:15.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:15 smithi132 ceph-mon[70676]: pgmap v2902: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:15.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:15 smithi088 ceph-mon[88643]: pgmap v2902: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:17.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:17 smithi132 ceph-mon[70676]: pgmap v2903: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:17.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:17 smithi088 ceph-mon[88643]: pgmap v2903: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:19.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:19 smithi132 ceph-mon[70676]: pgmap v2904: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:19.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:19 smithi088 ceph-mon[88643]: pgmap v2904: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:21.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:21 smithi132 ceph-mon[70676]: pgmap v2905: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:21.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:21 smithi088 ceph-mon[88643]: pgmap v2905: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:23.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:23 smithi088 ceph-mon[88643]: pgmap v2906: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:23.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:23 smithi132 ceph-mon[70676]: pgmap v2906: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:25.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:25 smithi088 ceph-mon[88643]: pgmap v2907: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:25.718 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:25 smithi132 ceph-mon[70676]: pgmap v2907: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:27.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:27 smithi088 ceph-mon[88643]: pgmap v2908: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:27.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:27 smithi132 ceph-mon[70676]: pgmap v2908: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:29.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:29 smithi088 ceph-mon[88643]: pgmap v2909: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:29.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:40:29.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:29 smithi132 ceph-mon[70676]: pgmap v2909: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:29.718 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:40:31.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:31 smithi088 ceph-mon[88643]: pgmap v2910: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:31.718 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:31 smithi132 ceph-mon[70676]: pgmap v2910: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:33.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:33 smithi088 ceph-mon[88643]: pgmap v2911: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:33.718 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:33 smithi132 ceph-mon[70676]: pgmap v2911: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:35.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:35 smithi088 ceph-mon[88643]: pgmap v2912: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:35.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:35 smithi132 ceph-mon[70676]: pgmap v2912: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:37.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:37 smithi088 ceph-mon[88643]: pgmap v2913: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:37.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:37 smithi132 ceph-mon[70676]: pgmap v2913: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:39.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:39 smithi088 ceph-mon[88643]: pgmap v2914: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:39.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:39 smithi132 ceph-mon[70676]: pgmap v2914: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:41.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:41 smithi132 ceph-mon[70676]: pgmap v2915: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:41.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:41 smithi088 ceph-mon[88643]: pgmap v2915: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:43.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:43 smithi088 ceph-mon[88643]: pgmap v2916: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:43.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:43 smithi132 ceph-mon[70676]: pgmap v2916: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:44.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:40:44.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:40:45.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:45 smithi088 ceph-mon[88643]: pgmap v2917: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:45.619 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:45 smithi132 ceph-mon[70676]: pgmap v2917: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:47.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:47 smithi088 ceph-mon[88643]: pgmap v2918: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:47.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:47 smithi132 ceph-mon[70676]: pgmap v2918: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:49.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:49 smithi088 ceph-mon[88643]: pgmap v2919: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:49.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:49 smithi132 ceph-mon[70676]: pgmap v2919: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:51.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:51 smithi132 ceph-mon[70676]: pgmap v2920: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:51.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:51 smithi088 ceph-mon[88643]: pgmap v2920: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:53.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:53 smithi088 ceph-mon[88643]: pgmap v2921: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:53.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:53 smithi132 ceph-mon[70676]: pgmap v2921: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:55.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:55 smithi088 ceph-mon[88643]: pgmap v2922: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:55.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:55 smithi132 ceph-mon[70676]: pgmap v2922: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:57.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:57 smithi088 ceph-mon[88643]: pgmap v2923: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:57.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:57 smithi132 ceph-mon[70676]: pgmap v2923: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:59.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:59 smithi088 ceph-mon[88643]: pgmap v2924: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:59.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:40:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:40:59.718 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:59 smithi132 ceph-mon[70676]: pgmap v2924: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:40:59.718 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:40:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:41:01.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:01 smithi088 ceph-mon[88643]: pgmap v2925: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:01.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:01 smithi132 ceph-mon[70676]: pgmap v2925: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:03.568 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:03 smithi088 ceph-mon[88643]: pgmap v2926: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:03.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:03 smithi132 ceph-mon[70676]: pgmap v2926: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:05.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:05 smithi088 ceph-mon[88643]: pgmap v2927: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:05.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:41:05.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:41:05.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:41:05.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:05 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:41:05.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:05 smithi132 ceph-mon[70676]: pgmap v2927: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:05.718 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:41:05.718 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:41:05.718 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:41:05.718 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:05 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:41:07.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:07 smithi088 ceph-mon[88643]: pgmap v2928: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:07.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:07 smithi132 ceph-mon[70676]: pgmap v2928: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:09.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:09 smithi088 ceph-mon[88643]: pgmap v2929: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:09.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:09 smithi132 ceph-mon[70676]: pgmap v2929: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:11.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:11 smithi088 ceph-mon[88643]: pgmap v2930: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:11.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:11 smithi132 ceph-mon[70676]: pgmap v2930: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:13.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:13 smithi088 ceph-mon[88643]: pgmap v2931: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:13.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:13 smithi132 ceph-mon[70676]: pgmap v2931: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:14.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:41:14.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:41:15.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:15 smithi088 ceph-mon[88643]: pgmap v2932: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:15.618 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:15 smithi132 ceph-mon[70676]: pgmap v2932: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:17.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:17 smithi132 ceph-mon[70676]: pgmap v2933: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:17.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:17 smithi088 ceph-mon[88643]: pgmap v2933: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:19.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:19 smithi132 ceph-mon[70676]: pgmap v2934: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:19.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:19 smithi088 ceph-mon[88643]: pgmap v2934: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:21.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:21 smithi132 ceph-mon[70676]: pgmap v2935: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:21.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:21 smithi088 ceph-mon[88643]: pgmap v2935: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:23.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:23 smithi132 ceph-mon[70676]: pgmap v2936: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:23.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:23 smithi088 ceph-mon[88643]: pgmap v2936: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:25.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:25 smithi132 ceph-mon[70676]: pgmap v2937: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:25.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:25 smithi088 ceph-mon[88643]: pgmap v2937: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:27.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:27 smithi132 ceph-mon[70676]: pgmap v2938: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:27.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:27 smithi088 ceph-mon[88643]: pgmap v2938: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:28.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:41:28.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:41:29.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:29 smithi132 ceph-mon[70676]: pgmap v2939: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:29.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:29 smithi088 ceph-mon[88643]: pgmap v2939: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:31.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:31 smithi132 ceph-mon[70676]: pgmap v2940: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:31.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:31 smithi088 ceph-mon[88643]: pgmap v2940: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:33.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:33 smithi132 ceph-mon[70676]: pgmap v2941: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:33.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:33 smithi088 ceph-mon[88643]: pgmap v2941: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:35.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:35 smithi132 ceph-mon[70676]: pgmap v2942: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:35.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:35 smithi088 ceph-mon[88643]: pgmap v2942: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:37.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:37 smithi132 ceph-mon[70676]: pgmap v2943: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:37.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:37 smithi088 ceph-mon[88643]: pgmap v2943: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:39.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:39 smithi088 ceph-mon[88643]: pgmap v2944: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:39.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:39 smithi132 ceph-mon[70676]: pgmap v2944: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:41.625 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:41 smithi132 ceph-mon[70676]: pgmap v2945: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:41.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:41 smithi088 ceph-mon[88643]: pgmap v2945: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:43.720 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:43 smithi132 ceph-mon[70676]: pgmap v2946: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:43.720 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:41:43.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:43 smithi088 ceph-mon[88643]: pgmap v2946: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:43.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:41:45.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:45 smithi132 ceph-mon[70676]: pgmap v2947: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:45.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:45 smithi088 ceph-mon[88643]: pgmap v2947: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:47.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:47 smithi132 ceph-mon[70676]: pgmap v2948: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:47.814 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:47 smithi088 ceph-mon[88643]: pgmap v2948: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:49.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:49 smithi088 ceph-mon[88643]: pgmap v2949: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:49.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:49 smithi132 ceph-mon[70676]: pgmap v2949: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:51.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:51 smithi132 ceph-mon[70676]: pgmap v2950: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:51.814 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:51 smithi088 ceph-mon[88643]: pgmap v2950: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:52.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:52 smithi132 ceph-mon[70676]: pgmap v2951: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:52.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:52 smithi088 ceph-mon[88643]: pgmap v2951: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:55.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:55 smithi132 ceph-mon[70676]: pgmap v2952: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:55.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:55 smithi088 ceph-mon[88643]: pgmap v2952: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:57.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:57 smithi132 ceph-mon[70676]: pgmap v2953: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:57.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:57 smithi088 ceph-mon[88643]: pgmap v2953: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:59.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:59 smithi132 ceph-mon[70676]: pgmap v2954: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:59.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:41:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:41:59.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:59 smithi088 ceph-mon[88643]: pgmap v2954: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:41:59.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:41:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:42:01.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:01 smithi132 ceph-mon[70676]: pgmap v2955: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:01.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:01 smithi088 ceph-mon[88643]: pgmap v2955: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:03.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:03 smithi132 ceph-mon[70676]: pgmap v2956: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:03.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:03 smithi088 ceph-mon[88643]: pgmap v2956: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:05.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:05 smithi132 ceph-mon[70676]: pgmap v2957: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:05.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:05 smithi088 ceph-mon[88643]: pgmap v2957: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:06.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:42:06.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:42:06.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:42:06.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:42:06.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:42:06.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:42:06.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:42:06.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:42:07.217 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:07 smithi132 ceph-mon[70676]: pgmap v2958: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:07.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:07 smithi088 ceph-mon[88643]: pgmap v2958: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:09.451 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:09 smithi088 ceph-mon[88643]: pgmap v2959: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:09.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:09 smithi132 ceph-mon[70676]: pgmap v2959: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:11.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:11 smithi132 ceph-mon[70676]: pgmap v2960: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:11.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:11 smithi088 ceph-mon[88643]: pgmap v2960: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:13.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:13 smithi132 ceph-mon[70676]: pgmap v2961: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:13.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:13 smithi088 ceph-mon[88643]: pgmap v2961: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:14.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:42:14.508 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:42:15.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:15 smithi132 ceph-mon[70676]: pgmap v2962: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:15.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:15 smithi088 ceph-mon[88643]: pgmap v2962: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:17.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:17 smithi132 ceph-mon[70676]: pgmap v2963: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:17.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:17 smithi088 ceph-mon[88643]: pgmap v2963: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:19.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:19 smithi132 ceph-mon[70676]: pgmap v2964: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:19.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:19 smithi088 ceph-mon[88643]: pgmap v2964: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:21.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:21 smithi132 ceph-mon[70676]: pgmap v2965: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:21.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:21 smithi088 ceph-mon[88643]: pgmap v2965: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:23.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:23 smithi132 ceph-mon[70676]: pgmap v2966: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:23.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:23 smithi088 ceph-mon[88643]: pgmap v2966: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:25.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:25 smithi132 ceph-mon[70676]: pgmap v2967: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:25.566 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:25 smithi088 ceph-mon[88643]: pgmap v2967: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:27.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:27 smithi132 ceph-mon[70676]: pgmap v2968: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:27.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:27 smithi088 ceph-mon[88643]: pgmap v2968: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:29.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:29 smithi132 ceph-mon[70676]: pgmap v2969: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:29.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:42:29.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:29 smithi088 ceph-mon[88643]: pgmap v2969: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:29.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:42:31.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:31 smithi132 ceph-mon[70676]: pgmap v2970: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:31.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:31 smithi088 ceph-mon[88643]: pgmap v2970: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:33.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:33 smithi132 ceph-mon[70676]: pgmap v2971: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:33.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:33 smithi088 ceph-mon[88643]: pgmap v2971: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:35.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:35 smithi132 ceph-mon[70676]: pgmap v2972: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:35.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:35 smithi088 ceph-mon[88643]: pgmap v2972: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:37.164 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:36 smithi132 ceph-mon[70676]: pgmap v2973: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:37.314 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:36 smithi088 ceph-mon[88643]: pgmap v2973: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:39.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:39 smithi132 ceph-mon[70676]: pgmap v2974: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:39.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:39 smithi088 ceph-mon[88643]: pgmap v2974: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:41.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:41 smithi132 ceph-mon[70676]: pgmap v2975: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:41.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:41 smithi088 ceph-mon[88643]: pgmap v2975: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:43.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:43 smithi132 ceph-mon[70676]: pgmap v2976: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:43.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:43 smithi088 ceph-mon[88643]: pgmap v2976: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:44.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:42:44.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:42:45.368 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:45 smithi132 ceph-mon[70676]: pgmap v2977: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:45.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:45 smithi088 ceph-mon[88643]: pgmap v2977: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:47.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:47 smithi132 ceph-mon[70676]: pgmap v2978: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:47.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:47 smithi088 ceph-mon[88643]: pgmap v2978: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:49.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:49 smithi132 ceph-mon[70676]: pgmap v2979: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:49.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:49 smithi088 ceph-mon[88643]: pgmap v2979: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:51.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:51 smithi132 ceph-mon[70676]: pgmap v2980: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:51.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:51 smithi088 ceph-mon[88643]: pgmap v2980: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:53.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:53 smithi132 ceph-mon[70676]: pgmap v2981: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:53.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:53 smithi088 ceph-mon[88643]: pgmap v2981: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:55.314 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:55 smithi088 ceph-mon[88643]: pgmap v2982: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:55.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:55 smithi132 ceph-mon[70676]: pgmap v2982: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:57.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:57 smithi132 ceph-mon[70676]: pgmap v2983: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:57.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:57 smithi088 ceph-mon[88643]: pgmap v2983: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:59.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:59 smithi132 ceph-mon[70676]: pgmap v2984: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:59.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:42:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:42:59.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:59 smithi088 ceph-mon[88643]: pgmap v2984: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:42:59.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:42:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:43:01.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:01 smithi132 ceph-mon[70676]: pgmap v2985: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:01.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:01 smithi088 ceph-mon[88643]: pgmap v2985: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:03.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:03 smithi132 ceph-mon[70676]: pgmap v2986: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:03.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:03 smithi088 ceph-mon[88643]: pgmap v2986: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:05.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:05 smithi132 ceph-mon[70676]: pgmap v2987: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:05.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:05 smithi088 ceph-mon[88643]: pgmap v2987: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:06.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:06 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:43:06.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:06 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:43:07.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:07 smithi088 ceph-mon[88643]: pgmap v2988: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:07.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:07.416 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:07.417 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:07.417 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:07 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:07.458 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:07 smithi132 ceph-mon[70676]: pgmap v2988: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:07.458 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:07.458 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:07.458 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:07.459 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:07 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:09.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:09 smithi132 ceph-mon[70676]: pgmap v2989: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:09.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:09.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:09 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:09.501 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:09 smithi088 ceph-mon[88643]: pgmap v2989: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:09.501 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:09.501 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:09 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:11.064 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:10 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:11.064 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:10 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:11.064 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:10 smithi088 ceph-mon[88643]: pgmap v2990: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:11.216 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:10 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:11.217 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:10 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:11.217 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:10 smithi132 ceph-mon[70676]: pgmap v2990: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:12.217 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:12.217 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:11 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:12.314 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:12.314 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:11 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:13.215 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:12 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:13.215 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:12 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:13.215 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:12 smithi132 ceph-mon[70676]: pgmap v2991: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:13.314 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:12 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:13.314 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:12 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:13.314 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:12 smithi088 ceph-mon[88643]: pgmap v2991: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:14.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:14.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:14.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:43:14.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:14.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:14.717 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:43:15.472 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:15 smithi088 ceph-mon[88643]: pgmap v2992: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:15.618 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:15 smithi132 ceph-mon[70676]: pgmap v2992: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:16.814 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:16.814 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:16.814 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:43:16.814 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:43:16.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:16.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:16 smithi088 ceph-mon[88643]: pgmap v2993: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:16.966 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:16.967 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:16.967 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:43:16.967 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:43:16.967 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:43:16.967 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:16 smithi132 ceph-mon[70676]: pgmap v2993: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:19.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:19 smithi132 ceph-mon[70676]: pgmap v2994: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:19.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:19 smithi088 ceph-mon[88643]: pgmap v2994: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:21.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:21 smithi132 ceph-mon[70676]: pgmap v2995: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:21.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:21 smithi088 ceph-mon[88643]: pgmap v2995: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:23.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:23 smithi132 ceph-mon[70676]: pgmap v2996: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:23.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:23 smithi088 ceph-mon[88643]: pgmap v2996: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:25.296 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:25 smithi088 ceph-mon[88643]: pgmap v2997: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:25.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:25 smithi132 ceph-mon[70676]: pgmap v2997: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:27.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:27 smithi132 ceph-mon[70676]: pgmap v2998: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:27.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:27 smithi088 ceph-mon[88643]: pgmap v2998: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:29.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:29 smithi132 ceph-mon[70676]: pgmap v2999: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:29.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:43:29.526 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:29 smithi088 ceph-mon[88643]: pgmap v2999: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:29.526 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:43:31.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:31 smithi132 ceph-mon[70676]: pgmap v3000: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:31.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:31 smithi088 ceph-mon[88643]: pgmap v3000: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:33.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:33 smithi132 ceph-mon[70676]: pgmap v3001: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:33.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:33 smithi088 ceph-mon[88643]: pgmap v3001: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:35.334 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:35 smithi088 ceph-mon[88643]: pgmap v3002: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:35.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:35 smithi132 ceph-mon[70676]: pgmap v3002: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:37.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:37 smithi132 ceph-mon[70676]: pgmap v3003: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:37.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:37 smithi088 ceph-mon[88643]: pgmap v3003: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:39.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:39 smithi132 ceph-mon[70676]: pgmap v3004: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:39.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:39 smithi088 ceph-mon[88643]: pgmap v3004: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:41.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:41 smithi132 ceph-mon[70676]: pgmap v3005: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:41.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:41 smithi088 ceph-mon[88643]: pgmap v3005: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:43.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:43 smithi132 ceph-mon[70676]: pgmap v3006: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:43.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:43 smithi088 ceph-mon[88643]: pgmap v3006: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:44.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:43:44.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:43:45.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:45 smithi132 ceph-mon[70676]: pgmap v3007: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:45.523 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:45 smithi088 ceph-mon[88643]: pgmap v3007: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:47.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:47 smithi132 ceph-mon[70676]: pgmap v3008: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:47.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:47 smithi088 ceph-mon[88643]: pgmap v3008: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:49.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:49 smithi132 ceph-mon[70676]: pgmap v3009: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:49.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:49 smithi088 ceph-mon[88643]: pgmap v3009: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:51.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:51 smithi132 ceph-mon[70676]: pgmap v3010: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:51.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:51 smithi088 ceph-mon[88643]: pgmap v3010: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:53.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:53 smithi132 ceph-mon[70676]: pgmap v3011: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:53.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:53 smithi088 ceph-mon[88643]: pgmap v3011: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:55.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:55 smithi132 ceph-mon[70676]: pgmap v3012: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:55.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:55 smithi088 ceph-mon[88643]: pgmap v3012: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:57.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:57 smithi132 ceph-mon[70676]: pgmap v3013: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:57.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:57 smithi088 ceph-mon[88643]: pgmap v3013: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:59.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:59 smithi132 ceph-mon[70676]: pgmap v3014: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:59.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:43:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:43:59.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:59 smithi088 ceph-mon[88643]: pgmap v3014: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:43:59.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:43:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:44:01.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:01 smithi132 ceph-mon[70676]: pgmap v3015: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:01.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:01 smithi088 ceph-mon[88643]: pgmap v3015: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:03.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:03 smithi132 ceph-mon[70676]: pgmap v3016: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:03.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:03 smithi088 ceph-mon[88643]: pgmap v3016: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:05.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:05 smithi132 ceph-mon[70676]: pgmap v3017: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:05.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:05 smithi088 ceph-mon[88643]: pgmap v3017: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:07.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:07 smithi132 ceph-mon[70676]: pgmap v3018: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:07.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:07 smithi088 ceph-mon[88643]: pgmap v3018: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:09.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:09 smithi132 ceph-mon[70676]: pgmap v3019: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:09.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:09 smithi088 ceph-mon[88643]: pgmap v3019: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:11.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:11 smithi132 ceph-mon[70676]: pgmap v3020: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:11.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:11 smithi088 ceph-mon[88643]: pgmap v3020: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:13.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:13 smithi132 ceph-mon[70676]: pgmap v3021: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:13.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:13 smithi088 ceph-mon[88643]: pgmap v3021: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:14.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:44:14.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:44:15.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:15 smithi132 ceph-mon[70676]: pgmap v3022: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:15.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:15 smithi088 ceph-mon[88643]: pgmap v3022: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:16.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:16 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:44:16.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:16 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:44:17.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:17 smithi132 ceph-mon[70676]: pgmap v3023: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:17.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:44:17.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:44:17.467 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:44:17.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:17 smithi088 ceph-mon[88643]: pgmap v3023: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:17.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:44:17.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:44:17.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:44:19.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:19 smithi088 ceph-mon[88643]: pgmap v3024: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:19.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:19 smithi132 ceph-mon[70676]: pgmap v3024: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:21.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:21 smithi088 ceph-mon[88643]: pgmap v3025: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:21.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:21 smithi132 ceph-mon[70676]: pgmap v3025: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:23.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:23 smithi132 ceph-mon[70676]: pgmap v3026: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:23.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:23 smithi088 ceph-mon[88643]: pgmap v3026: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:25.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:25 smithi088 ceph-mon[88643]: pgmap v3027: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:25.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:25 smithi132 ceph-mon[70676]: pgmap v3027: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:27.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:27 smithi088 ceph-mon[88643]: pgmap v3028: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:27.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:27 smithi132 ceph-mon[70676]: pgmap v3028: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:29.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:29 smithi088 ceph-mon[88643]: pgmap v3029: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:29.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:44:29.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:29 smithi132 ceph-mon[70676]: pgmap v3029: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:29.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:44:31.313 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:31 smithi088 ceph-mon[88643]: pgmap v3030: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:31.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:31 smithi132 ceph-mon[70676]: pgmap v3030: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:33.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:33 smithi088 ceph-mon[88643]: pgmap v3031: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:33.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:33 smithi132 ceph-mon[70676]: pgmap v3031: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:35.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:35 smithi088 ceph-mon[88643]: pgmap v3032: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:35.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:35 smithi132 ceph-mon[70676]: pgmap v3032: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:37.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:37 smithi088 ceph-mon[88643]: pgmap v3033: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:37.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:37 smithi132 ceph-mon[70676]: pgmap v3033: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:39.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:39 smithi088 ceph-mon[88643]: pgmap v3034: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:39.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:39 smithi132 ceph-mon[70676]: pgmap v3034: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:41.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:41 smithi088 ceph-mon[88643]: pgmap v3035: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:41.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:41 smithi132 ceph-mon[70676]: pgmap v3035: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:43.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:43 smithi088 ceph-mon[88643]: pgmap v3036: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:43.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:43 smithi132 ceph-mon[70676]: pgmap v3036: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:44.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:44:44.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:44:45.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:45 smithi088 ceph-mon[88643]: pgmap v3037: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:45.617 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:45 smithi132 ceph-mon[70676]: pgmap v3037: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:47.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:47 smithi088 ceph-mon[88643]: pgmap v3038: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:47.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:47 smithi132 ceph-mon[70676]: pgmap v3038: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:49.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:49 smithi088 ceph-mon[88643]: pgmap v3039: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:49.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:49 smithi132 ceph-mon[70676]: pgmap v3039: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:51.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:51 smithi088 ceph-mon[88643]: pgmap v3040: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:51.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:51 smithi132 ceph-mon[70676]: pgmap v3040: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:53.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:53 smithi088 ceph-mon[88643]: pgmap v3041: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:53.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:53 smithi132 ceph-mon[70676]: pgmap v3041: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:55.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:55 smithi088 ceph-mon[88643]: pgmap v3042: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:55.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:55 smithi132 ceph-mon[70676]: pgmap v3042: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:57.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:57 smithi088 ceph-mon[88643]: pgmap v3043: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:57.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:57 smithi132 ceph-mon[70676]: pgmap v3043: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:59.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:59 smithi088 ceph-mon[88643]: pgmap v3044: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:59.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:44:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:44:59.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:59 smithi132 ceph-mon[70676]: pgmap v3044: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:44:59.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:44:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:45:01.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:01 smithi088 ceph-mon[88643]: pgmap v3045: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:01.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:01 smithi132 ceph-mon[70676]: pgmap v3045: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:03.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:03 smithi088 ceph-mon[88643]: pgmap v3046: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:03.589 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:03 smithi132 ceph-mon[70676]: pgmap v3046: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:05.333 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:05 smithi088 ceph-mon[88643]: pgmap v3047: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:05.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:05 smithi132 ceph-mon[70676]: pgmap v3047: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:07.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:07 smithi088 ceph-mon[88643]: pgmap v3048: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:07.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:07 smithi132 ceph-mon[70676]: pgmap v3048: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:09.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:09 smithi088 ceph-mon[88643]: pgmap v3049: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:09.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:09 smithi132 ceph-mon[70676]: pgmap v3049: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:11.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:11 smithi088 ceph-mon[88643]: pgmap v3050: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:11.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:11 smithi132 ceph-mon[70676]: pgmap v3050: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:13.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:13 smithi088 ceph-mon[88643]: pgmap v3051: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:13.688 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:13 smithi132 ceph-mon[70676]: pgmap v3051: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:14.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:45:14.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:45:15.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:15 smithi088 ceph-mon[88643]: pgmap v3052: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:15.617 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:15 smithi132 ceph-mon[70676]: pgmap v3052: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:17.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:17 smithi088 ceph-mon[88643]: pgmap v3053: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:17.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:45:17.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:45:17.564 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:45:17.565 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:45:17.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:17 smithi132 ceph-mon[70676]: pgmap v3053: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:17.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:45:17.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:45:17.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:45:17.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:45:19.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:19 smithi088 ceph-mon[88643]: pgmap v3054: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:19.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:19 smithi132 ceph-mon[70676]: pgmap v3054: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:21.587 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:21 smithi088 ceph-mon[88643]: pgmap v3055: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:21.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:21 smithi132 ceph-mon[70676]: pgmap v3055: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:23.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:23 smithi132 ceph-mon[70676]: pgmap v3056: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:23.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:23 smithi088 ceph-mon[88643]: pgmap v3056: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:25.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:25 smithi132 ceph-mon[70676]: pgmap v3057: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:25.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:25 smithi088 ceph-mon[88643]: pgmap v3057: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:27.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:27 smithi132 ceph-mon[70676]: pgmap v3058: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:27.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:27 smithi088 ceph-mon[88643]: pgmap v3058: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:28.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:45:28.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:45:29.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:29 smithi132 ceph-mon[70676]: pgmap v3059: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:29.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:29 smithi088 ceph-mon[88643]: pgmap v3059: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:31.700 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:31 smithi088 ceph-mon[88643]: pgmap v3060: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:31.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:31 smithi132 ceph-mon[70676]: pgmap v3060: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:33.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:33 smithi132 ceph-mon[70676]: pgmap v3061: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:33.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:33 smithi088 ceph-mon[88643]: pgmap v3061: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:35.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:35 smithi132 ceph-mon[70676]: pgmap v3062: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:35.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:35 smithi088 ceph-mon[88643]: pgmap v3062: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:37.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:37 smithi132 ceph-mon[70676]: pgmap v3063: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:37.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:37 smithi088 ceph-mon[88643]: pgmap v3063: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:39.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:39 smithi132 ceph-mon[70676]: pgmap v3064: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:39.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:39 smithi088 ceph-mon[88643]: pgmap v3064: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:41.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:41 smithi132 ceph-mon[70676]: pgmap v3065: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:41.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:41 smithi088 ceph-mon[88643]: pgmap v3065: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:43.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:43 smithi132 ceph-mon[70676]: pgmap v3066: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:43.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:45:43.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:43 smithi088 ceph-mon[88643]: pgmap v3066: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:43.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:45:45.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:45 smithi132 ceph-mon[70676]: pgmap v3067: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:45.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:45 smithi088 ceph-mon[88643]: pgmap v3067: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:47.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:47 smithi132 ceph-mon[70676]: pgmap v3068: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:47.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:47 smithi088 ceph-mon[88643]: pgmap v3068: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:49.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:49 smithi132 ceph-mon[70676]: pgmap v3069: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:49.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:49 smithi088 ceph-mon[88643]: pgmap v3069: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:51.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:51 smithi132 ceph-mon[70676]: pgmap v3070: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:51.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:51 smithi088 ceph-mon[88643]: pgmap v3070: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:53.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:53 smithi132 ceph-mon[70676]: pgmap v3071: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:53.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:53 smithi088 ceph-mon[88643]: pgmap v3071: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:55.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:55 smithi132 ceph-mon[70676]: pgmap v3072: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:55.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:55 smithi088 ceph-mon[88643]: pgmap v3072: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:57.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:57 smithi132 ceph-mon[70676]: pgmap v3073: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:57.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:57 smithi088 ceph-mon[88643]: pgmap v3073: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:58.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:58 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:45:58.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:58 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:45:59.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:45:59 smithi132 ceph-mon[70676]: pgmap v3074: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:45:59.814 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:45:59 smithi088 ceph-mon[88643]: pgmap v3074: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:01.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:01 smithi132 ceph-mon[70676]: pgmap v3075: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:01.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:01 smithi088 ceph-mon[88643]: pgmap v3075: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:03.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:03 smithi132 ceph-mon[70676]: pgmap v3076: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:03.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:03 smithi088 ceph-mon[88643]: pgmap v3076: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:05.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:05 smithi132 ceph-mon[70676]: pgmap v3077: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:05.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:05 smithi088 ceph-mon[88643]: pgmap v3077: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:07.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:07 smithi132 ceph-mon[70676]: pgmap v3078: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:07.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:07 smithi088 ceph-mon[88643]: pgmap v3078: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:09.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:09 smithi132 ceph-mon[70676]: pgmap v3079: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:09.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:09 smithi088 ceph-mon[88643]: pgmap v3079: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:11.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:11 smithi132 ceph-mon[70676]: pgmap v3080: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:11.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:11 smithi088 ceph-mon[88643]: pgmap v3080: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:13.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:13 smithi132 ceph-mon[70676]: pgmap v3081: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:13.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:46:13.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:13 smithi088 ceph-mon[88643]: pgmap v3081: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:13.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:46:15.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:15 smithi132 ceph-mon[70676]: pgmap v3082: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:15.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:15 smithi088 ceph-mon[88643]: pgmap v3082: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:17.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:17 smithi088 ceph-mon[88643]: pgmap v3083: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:17.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:17 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:46:17.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:17 smithi132 ceph-mon[70676]: pgmap v3083: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:17.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:17 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:46:18.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:46:18.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:46:18.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:46:18.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:46:18.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:46:18.815 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:46:19.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:19 smithi132 ceph-mon[70676]: pgmap v3084: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:19.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:19 smithi088 ceph-mon[88643]: pgmap v3084: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:21.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:21 smithi132 ceph-mon[70676]: pgmap v3085: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:21.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:21 smithi088 ceph-mon[88643]: pgmap v3085: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:23.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:23 smithi132 ceph-mon[70676]: pgmap v3086: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:23.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:23 smithi088 ceph-mon[88643]: pgmap v3086: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:24.676 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:24 smithi088 ceph-mon[88643]: pgmap v3087: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:24.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:24 smithi132 ceph-mon[70676]: pgmap v3087: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:27.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:27 smithi132 ceph-mon[70676]: pgmap v3088: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:27.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:27 smithi088 ceph-mon[88643]: pgmap v3088: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:29.424 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:29 smithi132 ceph-mon[70676]: pgmap v3089: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:29.424 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:46:29.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:29 smithi088 ceph-mon[88643]: pgmap v3089: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:29.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:46:31.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:31 smithi132 ceph-mon[70676]: pgmap v3090: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:31.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:31 smithi088 ceph-mon[88643]: pgmap v3090: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:33.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:33 smithi132 ceph-mon[70676]: pgmap v3091: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:33.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:33 smithi088 ceph-mon[88643]: pgmap v3091: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:35.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:35 smithi132 ceph-mon[70676]: pgmap v3092: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:35.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:35 smithi088 ceph-mon[88643]: pgmap v3092: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:37.215 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:36 smithi132 ceph-mon[70676]: pgmap v3093: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:37.313 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:36 smithi088 ceph-mon[88643]: pgmap v3093: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:39.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:39 smithi132 ceph-mon[70676]: pgmap v3094: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:39.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:39 smithi088 ceph-mon[88643]: pgmap v3094: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:41.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:41 smithi132 ceph-mon[70676]: pgmap v3095: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:41.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:41 smithi088 ceph-mon[88643]: pgmap v3095: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:43.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:43 smithi132 ceph-mon[70676]: pgmap v3096: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:43.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:43 smithi088 ceph-mon[88643]: pgmap v3096: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:44.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:46:44.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:46:45.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:45 smithi132 ceph-mon[70676]: pgmap v3097: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:45.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:45 smithi088 ceph-mon[88643]: pgmap v3097: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:47.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:47 smithi132 ceph-mon[70676]: pgmap v3098: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:47.546 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:47 smithi088 ceph-mon[88643]: pgmap v3098: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:49.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:49 smithi132 ceph-mon[70676]: pgmap v3099: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:49.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:49 smithi088 ceph-mon[88643]: pgmap v3099: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:51.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:51 smithi132 ceph-mon[70676]: pgmap v3100: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:51.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:51 smithi088 ceph-mon[88643]: pgmap v3100: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:53.468 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:53 smithi132 ceph-mon[70676]: pgmap v3101: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:53.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:53 smithi088 ceph-mon[88643]: pgmap v3101: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:55.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:55 smithi132 ceph-mon[70676]: pgmap v3102: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:55.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:55 smithi088 ceph-mon[88643]: pgmap v3102: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:57.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:57 smithi132 ceph-mon[70676]: pgmap v3103: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:57.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:57 smithi088 ceph-mon[88643]: pgmap v3103: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:59.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:59 smithi132 ceph-mon[70676]: pgmap v3104: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:59.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:46:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:46:59.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:59 smithi088 ceph-mon[88643]: pgmap v3104: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:46:59.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:46:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:47:01.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:01 smithi132 ceph-mon[70676]: pgmap v3105: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:01.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:01 smithi088 ceph-mon[88643]: pgmap v3105: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:03.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:03 smithi132 ceph-mon[70676]: pgmap v3106: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:03.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:03 smithi088 ceph-mon[88643]: pgmap v3106: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:05.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:05 smithi132 ceph-mon[70676]: pgmap v3107: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:05.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:05 smithi088 ceph-mon[88643]: pgmap v3107: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:07.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:07 smithi132 ceph-mon[70676]: pgmap v3108: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:07.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:07 smithi088 ceph-mon[88643]: pgmap v3108: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:09.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:09 smithi132 ceph-mon[70676]: pgmap v3109: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:09.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:09 smithi088 ceph-mon[88643]: pgmap v3109: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:11.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:11 smithi132 ceph-mon[70676]: pgmap v3110: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:11.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:11 smithi088 ceph-mon[88643]: pgmap v3110: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:13.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:13 smithi132 ceph-mon[70676]: pgmap v3111: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:13.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:13 smithi088 ceph-mon[88643]: pgmap v3111: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:14.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:14 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:47:14.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:14 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:47:15.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:15 smithi088 ceph-mon[88643]: pgmap v3112: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:15.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:15 smithi132 ceph-mon[70676]: pgmap v3112: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:17.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:17 smithi088 ceph-mon[88643]: pgmap v3113: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:17.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:17 smithi132 ceph-mon[70676]: pgmap v3113: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:18.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:18 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:47:18.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:18 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:47:19.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:19 smithi088 ceph-mon[88643]: pgmap v3114: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:19.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:47:19.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:47:19.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:47:19.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:19 smithi132 ceph-mon[70676]: pgmap v3114: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:19.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:47:19.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:47:19.716 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:47:21.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:21 smithi088 ceph-mon[88643]: pgmap v3115: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:21.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:21 smithi132 ceph-mon[70676]: pgmap v3115: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:23.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:23 smithi088 ceph-mon[88643]: pgmap v3116: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:23.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:23 smithi132 ceph-mon[70676]: pgmap v3116: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:25.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:25 smithi088 ceph-mon[88643]: pgmap v3117: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:25.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:25 smithi132 ceph-mon[70676]: pgmap v3117: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:27.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:27 smithi088 ceph-mon[88643]: pgmap v3118: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:27.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:27 smithi132 ceph-mon[70676]: pgmap v3118: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:29.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:29 smithi088 ceph-mon[88643]: pgmap v3119: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:29.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:29 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:47:29.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:29 smithi132 ceph-mon[70676]: pgmap v3119: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:29.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:29 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:47:31.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:31 smithi088 ceph-mon[88643]: pgmap v3120: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:31.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:31 smithi132 ceph-mon[70676]: pgmap v3120: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:33.312 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:33 smithi088 ceph-mon[88643]: pgmap v3121: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:33.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:33 smithi132 ceph-mon[70676]: pgmap v3121: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:35.465 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:35 smithi132 ceph-mon[70676]: pgmap v3122: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:35.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:35 smithi088 ceph-mon[88643]: pgmap v3122: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:37.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:37 smithi088 ceph-mon[88643]: pgmap v3123: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:37.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:37 smithi132 ceph-mon[70676]: pgmap v3123: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:39.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:39 smithi088 ceph-mon[88643]: pgmap v3124: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:39.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:39 smithi132 ceph-mon[70676]: pgmap v3124: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:41.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:41 smithi088 ceph-mon[88643]: pgmap v3125: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:41.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:41 smithi132 ceph-mon[70676]: pgmap v3125: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:43.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:43 smithi088 ceph-mon[88643]: pgmap v3126: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:43.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:43 smithi132 ceph-mon[70676]: pgmap v3126: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:44.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:44 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:47:44.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:44 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:47:45.466 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:45 smithi132 ceph-mon[70676]: pgmap v3127: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:45.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:45 smithi088 ceph-mon[88643]: pgmap v3127: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:47.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:47 smithi088 ceph-mon[88643]: pgmap v3128: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:47.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:47 smithi132 ceph-mon[70676]: pgmap v3128: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:49.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:49 smithi088 ceph-mon[88643]: pgmap v3129: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:49.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:49 smithi132 ceph-mon[70676]: pgmap v3129: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:51.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:51 smithi088 ceph-mon[88643]: pgmap v3130: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:51.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:51 smithi132 ceph-mon[70676]: pgmap v3130: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:53.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:53 smithi088 ceph-mon[88643]: pgmap v3131: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:53.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:53 smithi132 ceph-mon[70676]: pgmap v3131: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:55.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:55 smithi088 ceph-mon[88643]: pgmap v3132: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:55.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:55 smithi132 ceph-mon[70676]: pgmap v3132: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:57.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:57 smithi088 ceph-mon[88643]: pgmap v3133: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:57.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:57 smithi132 ceph-mon[70676]: pgmap v3133: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:59.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:59 smithi088 ceph-mon[88643]: pgmap v3134: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:59.563 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:47:59 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:47:59.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:59 smithi132 ceph-mon[70676]: pgmap v3134: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:47:59.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:47:59 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:48:01.562 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:01 smithi088 ceph-mon[88643]: pgmap v3135: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:01.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:01 smithi132 ceph-mon[70676]: pgmap v3135: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:03.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:03 smithi132 ceph-mon[70676]: pgmap v3136: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:03.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:03 smithi088 ceph-mon[88643]: pgmap v3136: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:05.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:05 smithi132 ceph-mon[70676]: pgmap v3137: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:05.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:05 smithi088 ceph-mon[88643]: pgmap v3137: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:07.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:07 smithi132 ceph-mon[70676]: pgmap v3138: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:07.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:07 smithi088 ceph-mon[88643]: pgmap v3138: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:09.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:09 smithi132 ceph-mon[70676]: pgmap v3139: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:09.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:09 smithi088 ceph-mon[88643]: pgmap v3139: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:11.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:11 smithi132 ceph-mon[70676]: pgmap v3140: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:11.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:11 smithi088 ceph-mon[88643]: pgmap v3140: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:13.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:13 smithi132 ceph-mon[70676]: pgmap v3141: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:13.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:13 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:48:13.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:13 smithi088 ceph-mon[88643]: pgmap v3141: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:13.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:13 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:48:15.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:15 smithi132 ceph-mon[70676]: pgmap v3142: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:15.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:15 smithi088 ceph-mon[88643]: pgmap v3142: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:17.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:17 smithi132 ceph-mon[70676]: pgmap v3143: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:17.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:17 smithi088 ceph-mon[88643]: pgmap v3143: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:19.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:19 smithi132 ceph-mon[70676]: pgmap v3144: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:19.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:48:19.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:48:19.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:48:19.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:19 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:48:19.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:19 smithi088 ceph-mon[88643]: pgmap v3144: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:19.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-19T20:48:19.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-19T20:48:19.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-19T20:48:19.813 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:19 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' 2024-02-19T20:48:21.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:21 smithi132 ceph-mon[70676]: pgmap v3145: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:21.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:21 smithi088 ceph-mon[88643]: pgmap v3145: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:23.627 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:23 smithi088 ceph-mon[88643]: pgmap v3146: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:23.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:23 smithi132 ceph-mon[70676]: pgmap v3146: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:25.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:25 smithi132 ceph-mon[70676]: pgmap v3147: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:25.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:25 smithi088 ceph-mon[88643]: pgmap v3147: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:27.714 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:27 smithi132 ceph-mon[70676]: pgmap v3148: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:27.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:27 smithi088 ceph-mon[88643]: pgmap v3148: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:28.685 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:28 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:48:28.714 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:28 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:48:29.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:29 smithi132 ceph-mon[70676]: pgmap v3149: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:29.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:29 smithi088 ceph-mon[88643]: pgmap v3149: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:31.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:31 smithi132 ceph-mon[70676]: pgmap v3150: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:31.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:31 smithi088 ceph-mon[88643]: pgmap v3150: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:33.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:33 smithi132 ceph-mon[70676]: pgmap v3151: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:33.741 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:33 smithi088 ceph-mon[88643]: pgmap v3151: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:35.651 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:35 smithi132 ceph-mon[70676]: pgmap v3152: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:35.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:35 smithi088 ceph-mon[88643]: pgmap v3152: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:37.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:37 smithi132 ceph-mon[70676]: pgmap v3153: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:37.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:37 smithi088 ceph-mon[88643]: pgmap v3153: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:39.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:39 smithi132 ceph-mon[70676]: pgmap v3154: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:39.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:39 smithi088 ceph-mon[88643]: pgmap v3154: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:41.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:41 smithi132 ceph-mon[70676]: pgmap v3155: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:41.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:41 smithi088 ceph-mon[88643]: pgmap v3155: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:43.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:43 smithi132 ceph-mon[70676]: pgmap v3156: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:43.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:43 smithi132 ceph-mon[70676]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:48:43.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:43 smithi088 ceph-mon[88643]: pgmap v3156: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:43.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:43 smithi088 ceph-mon[88643]: from='mgr.34100 172.21.15.88:0/2831300284' entity='mgr.smithi088.gyviqq' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-19T20:48:45.714 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:45 smithi132 ceph-mon[70676]: pgmap v3157: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:45.812 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:45 smithi088 ceph-mon[88643]: pgmap v3157: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:46.939 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-19T20:48:46.956 INFO:tasks.cephadm:Teardown begin 2024-02-19T20:48:46.957 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa/tasks/cephadm.py", line 1807, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-19T20:48:46.958 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-19T20:48:46.994 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-19T20:48:47.029 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-19T20:48:47.029 DEBUG:teuthology.orchestra.run.smithi088:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-19T20:48:47.050 DEBUG:teuthology.orchestra.run.smithi132:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-19T20:48:47.087 INFO:tasks.cephadm:Stopping all daemons... 2024-02-19T20:48:47.087 INFO:tasks.cephadm.mon.smithi088:Stopping mon.smithi088... 2024-02-19T20:48:47.087 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi088 2024-02-19T20:48:47.312 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:47 smithi088 systemd[1]: Stopping Ceph mon.smithi088 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T20:48:47.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:47 smithi132 ceph-mon[70676]: pgmap v3158: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:47.804 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:47 smithi088 ceph-mon[88643]: pgmap v3158: 65 pgs: 65 active+clean; 272 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-02-19T20:48:47.805 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:47 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T20:48:47.471+0000 7fb607fc4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi088 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T20:48:47.805 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:47 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088[88639]: 2024-02-19T20:48:47.471+0000 7fb607fc4700 -1 mon.smithi088@0(leader) e3 *** Got Signal Terminated *** 2024-02-19T20:48:47.805 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:47 smithi088 podman[163940]: 2024-02-19 20:48:47.492563654 +0000 UTC m=+0.194082075 container died 8ff9c135b06187d3315d135fec91816e7d828be1c3326afb0f0deeae07081108 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-19T20:48:48.062 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:47 smithi088 podman[163940]: 2024-02-19 20:48:47.997074207 +0000 UTC m=+0.698592624 container cleanup 8ff9c135b06187d3315d135fec91816e7d828be1c3326afb0f0deeae07081108 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-19T20:48:48.062 INFO:journalctl@ceph.mon.smithi088.smithi088.stdout:Feb 19 20:48:47 smithi088 bash[163940]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi088 2024-02-19T20:48:48.386 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi088.service' 2024-02-19T20:48:48.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T20:48:48.433 INFO:tasks.cephadm.mon.smithi088:Stopped mon.smithi088 2024-02-19T20:48:48.433 INFO:tasks.cephadm.mon.smithi132:Stopping mon.smithi132... 2024-02-19T20:48:48.433 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi132 2024-02-19T20:48:48.714 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:48 smithi132 systemd[1]: Stopping Ceph mon.smithi132 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T20:48:49.215 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:48 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132[70672]: 2024-02-19T20:48:48.906+0000 7fe597225700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi132 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T20:48:49.215 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:48 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132[70672]: 2024-02-19T20:48:48.906+0000 7fe597225700 -1 mon.smithi132@1(peon) e3 *** Got Signal Terminated *** 2024-02-19T20:48:49.215 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:48 smithi132 podman[125091]: 2024-02-19 20:48:48.957665501 +0000 UTC m=+0.311089420 container died dabaa14880db943321b7811ddf25b67634b11e3f2ba4729f86377603cceb603b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-02-19T20:48:49.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:49 smithi132 podman[125091]: 2024-02-19 20:48:49.482061314 +0000 UTC m=+0.835485235 container cleanup dabaa14880db943321b7811ddf25b67634b11e3f2ba4729f86377603cceb603b (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, org.label-schema.schema-version=1.0) 2024-02-19T20:48:49.715 INFO:journalctl@ceph.mon.smithi132.smithi132.stdout:Feb 19 20:48:49 smithi132 bash[125091]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-mon-smithi132 2024-02-19T20:48:49.941 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@mon.smithi132.service' 2024-02-19T20:48:49.986 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T20:48:49.986 INFO:tasks.cephadm.mon.smithi132:Stopped mon.smithi132 2024-02-19T20:48:49.986 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-19T20:48:49.986 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.0 2024-02-19T20:48:50.308 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:50 smithi088 systemd[1]: Stopping Ceph osd.0 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T20:48:50.562 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:50 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[100756]: 2024-02-19T20:48:50.326+0000 7efd904d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T20:48:50.562 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:50 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[100756]: 2024-02-19T20:48:50.326+0000 7efd904d4700 -1 osd.0 88 *** Got signal Terminated *** 2024-02-19T20:48:50.562 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:50 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0[100756]: 2024-02-19T20:48:50.326+0000 7efd904d4700 -1 osd.0 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T20:48:56.051 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:55 smithi088 podman[164075]: 2024-02-19 20:48:55.784292135 +0000 UTC m=+5.589130016 container died b544778e0f5de9443e1c981b32b5ce3ac406ebfcca41a925a363928442b18ee7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T20:48:56.312 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:56 smithi088 podman[164075]: 2024-02-19 20:48:56.142971953 +0000 UTC m=+5.947809831 container cleanup b544778e0f5de9443e1c981b32b5ce3ac406ebfcca41a925a363928442b18ee7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-19T20:48:56.312 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:56 smithi088 bash[164075]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0 2024-02-19T20:48:56.799 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:56 smithi088 podman[164092]: 2024-02-19 20:48:56.461191584 +0000 UTC m=+0.678213247 container remove b544778e0f5de9443e1c981b32b5ce3ac406ebfcca41a925a363928442b18ee7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-02-19T20:48:57.062 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:56 smithi088 podman[164187]: 2024-02-19 20:48:56.721342275 +0000 UTC m=+0.018299657 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T20:48:57.062 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:56 smithi088 podman[164187]: 2024-02-19 20:48:56.859783861 +0000 UTC m=+0.156741274 container create 611789f7ec831ac6616d71654922d84a286c7a489fb6e9c81afada3d814b0518 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-02-19T20:48:57.333 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:57 smithi088 podman[164187]: 2024-02-19 20:48:57.090099432 +0000 UTC m=+0.387056807 container init 611789f7ec831ac6616d71654922d84a286c7a489fb6e9c81afada3d814b0518 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T20:48:57.333 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:57 smithi088 podman[164187]: 2024-02-19 20:48:57.093762873 +0000 UTC m=+0.390720246 container start 611789f7ec831ac6616d71654922d84a286c7a489fb6e9c81afada3d814b0518 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-19T20:48:57.333 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:57 smithi088 podman[164187]: 2024-02-19 20:48:57.189998544 +0000 UTC m=+0.486955935 container attach 611789f7ec831ac6616d71654922d84a286c7a489fb6e9c81afada3d814b0518 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-19T20:48:57.742 INFO:journalctl@ceph.osd.0.smithi088.stdout:Feb 19 20:48:57 smithi088 podman[164187]: 2024-02-19 20:48:57.409059564 +0000 UTC m=+0.706016936 container died 611789f7ec831ac6616d71654922d84a286c7a489fb6e9c81afada3d814b0518 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-02-19T20:48:58.015 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.0.service' 2024-02-19T20:48:58.056 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T20:48:58.057 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-19T20:48:58.057 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-19T20:48:58.057 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.1 2024-02-19T20:48:58.296 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:48:58 smithi088 systemd[1]: Stopping Ceph osd.1 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T20:48:58.562 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:48:58 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[106700]: 2024-02-19T20:48:58.430+0000 7f2409843700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T20:48:58.562 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:48:58 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[106700]: 2024-02-19T20:48:58.430+0000 7f2409843700 -1 osd.1 88 *** Got signal Terminated *** 2024-02-19T20:48:58.563 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:48:58 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1[106700]: 2024-02-19T20:48:58.430+0000 7f2409843700 -1 osd.1 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T20:49:04.038 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:49:03 smithi088 podman[164304]: 2024-02-19 20:49:03.793184815 +0000 UTC m=+5.492983598 container died 5b940966d8833fc65603ca204d3e18f06791796a684c0645ab9b2435e94042c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-19T20:49:04.312 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:49:04 smithi088 podman[164304]: 2024-02-19 20:49:04.155570731 +0000 UTC m=+5.855369530 container cleanup 5b940966d8833fc65603ca204d3e18f06791796a684c0645ab9b2435e94042c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-19T20:49:04.312 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:49:04 smithi088 bash[164304]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1 2024-02-19T20:49:04.808 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:49:04 smithi088 podman[164321]: 2024-02-19 20:49:04.490127879 +0000 UTC m=+0.700380112 container remove 5b940966d8833fc65603ca204d3e18f06791796a684c0645ab9b2435e94042c7 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-19T20:49:05.062 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:49:04 smithi088 podman[164412]: 2024-02-19 20:49:04.730263566 +0000 UTC m=+0.018504494 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T20:49:05.062 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:49:04 smithi088 podman[164412]: 2024-02-19 20:49:04.991110796 +0000 UTC m=+0.279351787 container create d42b5c1df8a1bd0244a275d317c96f8daf8af8cdb276f9bcbda292919185d56c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-02-19T20:49:05.506 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:49:05 smithi088 podman[164412]: 2024-02-19 20:49:05.236002956 +0000 UTC m=+0.524243894 container init d42b5c1df8a1bd0244a275d317c96f8daf8af8cdb276f9bcbda292919185d56c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-02-19T20:49:05.506 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:49:05 smithi088 podman[164412]: 2024-02-19 20:49:05.239387919 +0000 UTC m=+0.527628849 container start d42b5c1df8a1bd0244a275d317c96f8daf8af8cdb276f9bcbda292919185d56c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-19T20:49:05.506 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:49:05 smithi088 podman[164412]: 2024-02-19 20:49:05.310357955 +0000 UTC m=+0.598598892 container attach d42b5c1df8a1bd0244a275d317c96f8daf8af8cdb276f9bcbda292919185d56c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-19T20:49:05.506 INFO:journalctl@ceph.osd.1.smithi088.stdout:Feb 19 20:49:05 smithi088 podman[164412]: 2024-02-19 20:49:05.52443372 +0000 UTC m=+0.812674654 container died d42b5c1df8a1bd0244a275d317c96f8daf8af8cdb276f9bcbda292919185d56c (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T20:49:06.069 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.1.service' 2024-02-19T20:49:06.113 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T20:49:06.113 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-19T20:49:06.113 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-19T20:49:06.114 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.2 2024-02-19T20:49:06.312 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:06 smithi088 systemd[1]: Stopping Ceph osd.2 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T20:49:06.812 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:06 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[112688]: 2024-02-19T20:49:06.474+0000 7f8d7434d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T20:49:06.812 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:06 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[112688]: 2024-02-19T20:49:06.474+0000 7f8d7434d700 -1 osd.2 88 *** Got signal Terminated *** 2024-02-19T20:49:06.813 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:06 smithi088 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2[112688]: 2024-02-19T20:49:06.474+0000 7f8d7434d700 -1 osd.2 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T20:49:12.312 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:12 smithi088 podman[164519]: 2024-02-19 20:49:12.025237629 +0000 UTC m=+5.667519555 container died f72e97fd38a53f34b5798540949b4aa015de64fab8ebb4585ae26ae17a9e5cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c) 2024-02-19T20:49:12.658 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:12 smithi088 podman[164519]: 2024-02-19 20:49:12.433413097 +0000 UTC m=+6.075695031 container cleanup f72e97fd38a53f34b5798540949b4aa015de64fab8ebb4585ae26ae17a9e5cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T20:49:12.658 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:12 smithi088 bash[164519]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2 2024-02-19T20:49:12.987 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:12 smithi088 podman[164536]: 2024-02-19 20:49:12.676603624 +0000 UTC m=+0.646132828 container remove f72e97fd38a53f34b5798540949b4aa015de64fab8ebb4585ae26ae17a9e5cc6 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T20:49:13.286 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:13 smithi088 podman[164629]: 2024-02-19 20:49:12.908929331 +0000 UTC m=+0.018826507 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T20:49:13.286 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:13 smithi088 podman[164629]: 2024-02-19 20:49:13.019284692 +0000 UTC m=+0.129181855 container create 4689dffe0c12b5e4592f81e5823c070555d33030812e28696efdda81c5eb48f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux ) 2024-02-19T20:49:13.286 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:13 smithi088 podman[164629]: 2024-02-19 20:49:13.221998521 +0000 UTC m=+0.331895705 container init 4689dffe0c12b5e4592f81e5823c070555d33030812e28696efdda81c5eb48f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T20:49:13.287 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:13 smithi088 podman[164629]: 2024-02-19 20:49:13.225571324 +0000 UTC m=+0.335468489 container start 4689dffe0c12b5e4592f81e5823c070555d33030812e28696efdda81c5eb48f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-02-19T20:49:13.287 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:13 smithi088 podman[164629]: 2024-02-19 20:49:13.305038817 +0000 UTC m=+0.414936002 container attach 4689dffe0c12b5e4592f81e5823c070555d33030812e28696efdda81c5eb48f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2) 2024-02-19T20:49:13.562 INFO:journalctl@ceph.osd.2.smithi088.stdout:Feb 19 20:49:13 smithi088 podman[164629]: 2024-02-19 20:49:13.516580622 +0000 UTC m=+0.626477792 container died 4689dffe0c12b5e4592f81e5823c070555d33030812e28696efdda81c5eb48f9 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-2-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-02-19T20:49:14.068 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.2.service' 2024-02-19T20:49:14.109 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T20:49:14.109 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-19T20:49:14.110 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-19T20:49:14.110 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.3 2024-02-19T20:49:14.399 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:14 smithi132 systemd[1]: Stopping Ceph osd.3 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T20:49:14.714 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:14 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[80695]: 2024-02-19T20:49:14.416+0000 7fa2e8622700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T20:49:14.714 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:14 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[80695]: 2024-02-19T20:49:14.416+0000 7fa2e8622700 -1 osd.3 88 *** Got signal Terminated *** 2024-02-19T20:49:14.715 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:14 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[80695]: 2024-02-19T20:49:14.416+0000 7fa2e8622700 -1 osd.3 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T20:49:18.965 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:18 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:18.633+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:19.630 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:19 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3[80695]: 2024-02-19T20:49:19.303+0000 7fa2e5417700 -1 osd.3 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:53.720381+0000 front 2024-02-19T20:48:53.720216+0000 (oldest deadline 2024-02-19T20:49:19.020156+0000) 2024-02-19T20:49:19.630 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:19 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:19.446+0000 7fb5533c8700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:50.203659+0000 front 2024-02-19T20:48:50.203739+0000 (oldest deadline 2024-02-19T20:49:19.004079+0000) 2024-02-19T20:49:19.965 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:19 smithi132 podman[125215]: 2024-02-19 20:49:19.74949674 +0000 UTC m=+5.430335709 container died 6dc5d99df1e5edec7d0005ac665903d35217f827693fb35041e421cf8d42c50f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, ceph=True) 2024-02-19T20:49:19.965 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:19 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:19.647+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:20.292 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:20 smithi132 podman[125215]: 2024-02-19 20:49:20.093361984 +0000 UTC m=+5.774200949 container cleanup 6dc5d99df1e5edec7d0005ac665903d35217f827693fb35041e421cf8d42c50f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-02-19T20:49:20.293 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:20 smithi132 bash[125215]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3 2024-02-19T20:49:20.617 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:20 smithi132 podman[125244]: 2024-02-19 20:49:20.311300239 +0000 UTC m=+0.557963918 container remove 6dc5d99df1e5edec7d0005ac665903d35217f827693fb35041e421cf8d42c50f (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3, GIT_CLEAN=True, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T20:49:20.618 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:20 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:20.469+0000 7fb5533c8700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:50.203659+0000 front 2024-02-19T20:48:50.203739+0000 (oldest deadline 2024-02-19T20:49:19.004079+0000) 2024-02-19T20:49:20.946 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:20 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:20.635+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:20.946 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:20 smithi132 podman[125337]: 2024-02-19 20:49:20.610870726 +0000 UTC m=+0.018539444 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T20:49:20.946 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:20 smithi132 podman[125337]: 2024-02-19 20:49:20.737244799 +0000 UTC m=+0.144913517 container create 1556c9a21522f97aced0202e8950472370b18e099f56caf3c800274371aaf4da (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T20:49:20.946 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:20 smithi132 podman[125337]: 2024-02-19 20:49:20.964681155 +0000 UTC m=+0.372349873 container init 1556c9a21522f97aced0202e8950472370b18e099f56caf3c800274371aaf4da (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240214) 2024-02-19T20:49:21.215 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:20 smithi132 podman[125337]: 2024-02-19 20:49:20.967978544 +0000 UTC m=+0.375647270 container start 1556c9a21522f97aced0202e8950472370b18e099f56caf3c800274371aaf4da (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True) 2024-02-19T20:49:21.215 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:21 smithi132 podman[125337]: 2024-02-19 20:49:21.056340916 +0000 UTC m=+0.464009662 container attach 1556c9a21522f97aced0202e8950472370b18e099f56caf3c800274371aaf4da (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-02-19T20:49:21.549 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:21 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:21.478+0000 7fb5533c8700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:50.203659+0000 front 2024-02-19T20:48:50.203739+0000 (oldest deadline 2024-02-19T20:49:19.004079+0000) 2024-02-19T20:49:21.550 INFO:journalctl@ceph.osd.3.smithi132.stdout:Feb 19 20:49:21 smithi132 podman[125337]: 2024-02-19 20:49:21.274541919 +0000 UTC m=+0.682210641 container died 1556c9a21522f97aced0202e8950472370b18e099f56caf3c800274371aaf4da (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-3-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T20:49:21.842 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:21 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:21.621+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:21.860 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.3.service' 2024-02-19T20:49:21.902 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T20:49:21.902 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-19T20:49:21.902 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-19T20:49:21.902 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.4 2024-02-19T20:49:22.214 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:22 smithi132 systemd[1]: Stopping Ceph osd.4 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T20:49:22.578 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:22 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:22.280+0000 7fb55a5db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T20:49:22.579 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:22 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:22.280+0000 7fb55a5db700 -1 osd.4 88 *** Got signal Terminated *** 2024-02-19T20:49:22.579 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:22 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:22.280+0000 7fb55a5db700 -1 osd.4 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T20:49:22.579 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:22 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:22.528+0000 7fb5533c8700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:50.203659+0000 front 2024-02-19T20:48:50.203739+0000 (oldest deadline 2024-02-19T20:49:19.004079+0000) 2024-02-19T20:49:22.964 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:22 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:22.596+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:23.964 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:23 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:23.562+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:23.965 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:23 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:23.499+0000 7fb5533c8700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:50.203659+0000 front 2024-02-19T20:48:50.203739+0000 (oldest deadline 2024-02-19T20:49:19.004079+0000) 2024-02-19T20:49:24.714 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:24 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:24.477+0000 7fb5533c8700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:50.203659+0000 front 2024-02-19T20:48:50.203739+0000 (oldest deadline 2024-02-19T20:49:19.004079+0000) 2024-02-19T20:49:24.715 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:24 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:24.590+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:25.714 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:25 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:25.635+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:25.715 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:25 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:25.464+0000 7fb5533c8700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:50.203659+0000 front 2024-02-19T20:48:50.203739+0000 (oldest deadline 2024-02-19T20:49:19.004079+0000) 2024-02-19T20:49:25.715 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:25 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:25.464+0000 7fb5533c8700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.88:6814 osd.1 since back 2024-02-19T20:49:00.704981+0000 front 2024-02-19T20:49:00.704905+0000 (oldest deadline 2024-02-19T20:49:24.804800+0000) 2024-02-19T20:49:26.715 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:26 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:26.444+0000 7fb5533c8700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:50.203659+0000 front 2024-02-19T20:48:50.203739+0000 (oldest deadline 2024-02-19T20:49:19.004079+0000) 2024-02-19T20:49:26.715 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:26 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4[86070]: 2024-02-19T20:49:26.444+0000 7fb5533c8700 -1 osd.4 88 heartbeat_check: no reply from 172.21.15.88:6814 osd.1 since back 2024-02-19T20:49:00.704981+0000 front 2024-02-19T20:49:00.704905+0000 (oldest deadline 2024-02-19T20:49:24.804800+0000) 2024-02-19T20:49:26.715 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:26 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:26.625+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:27.946 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:27 smithi132 podman[125442]: 2024-02-19 20:49:27.695198263 +0000 UTC m=+5.554197318 container died c7a967e24a5519df7b41a855412d80c4cb0f47d21fca7d98ee3ca05bc43f1915 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-19T20:49:27.946 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:27 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:27.655+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:28.214 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:28 smithi132 podman[125442]: 2024-02-19 20:49:28.03871624 +0000 UTC m=+5.897715295 container cleanup c7a967e24a5519df7b41a855412d80c4cb0f47d21fca7d98ee3ca05bc43f1915 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd) 2024-02-19T20:49:28.215 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:28 smithi132 bash[125442]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4 2024-02-19T20:49:28.605 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:28 smithi132 podman[125459]: 2024-02-19 20:49:28.239973814 +0000 UTC m=+0.538660693 container remove c7a967e24a5519df7b41a855412d80c4cb0f47d21fca7d98ee3ca05bc43f1915 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240214) 2024-02-19T20:49:28.950 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:28 smithi132 podman[125555]: 2024-02-19 20:49:28.55135414 +0000 UTC m=+0.018205674 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T20:49:28.950 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:28 smithi132 podman[125555]: 2024-02-19 20:49:28.657509745 +0000 UTC m=+0.124361265 container create 63a9761aff49b6d0eea1e8ead20161d256b217783153442c3f99a01beb6c3f9e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-deactivate, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-19T20:49:28.950 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:28 smithi132 podman[125555]: 2024-02-19 20:49:28.8683242 +0000 UTC m=+0.335175721 container init 63a9761aff49b6d0eea1e8ead20161d256b217783153442c3f99a01beb6c3f9e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5) 2024-02-19T20:49:28.950 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:28 smithi132 podman[125555]: 2024-02-19 20:49:28.871421202 +0000 UTC m=+0.338272731 container start 63a9761aff49b6d0eea1e8ead20161d256b217783153442c3f99a01beb6c3f9e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, org.label-schema.build-date=20240214, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-19T20:49:28.950 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:28 smithi132 podman[125555]: 2024-02-19 20:49:28.968349933 +0000 UTC m=+0.435201471 container attach 63a9761aff49b6d0eea1e8ead20161d256b217783153442c3f99a01beb6c3f9e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-deactivate, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240214, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-02-19T20:49:28.950 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:28 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:28.623+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:28.950 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:28 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:28.623+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6814 osd.1 since back 2024-02-19T20:49:03.149542+0000 front 2024-02-19T20:49:03.149484+0000 (oldest deadline 2024-02-19T20:49:27.849482+0000) 2024-02-19T20:49:29.214 INFO:journalctl@ceph.osd.4.smithi132.stdout:Feb 19 20:49:29 smithi132 podman[125555]: 2024-02-19 20:49:29.162976174 +0000 UTC m=+0.629827696 container died 63a9761aff49b6d0eea1e8ead20161d256b217783153442c3f99a01beb6c3f9e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240214, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T20:49:29.736 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:29 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:29.662+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:29.737 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:29 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:29.662+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6814 osd.1 since back 2024-02-19T20:49:03.149542+0000 front 2024-02-19T20:49:03.149484+0000 (oldest deadline 2024-02-19T20:49:27.849482+0000) 2024-02-19T20:49:29.764 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.4.service' 2024-02-19T20:49:29.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T20:49:29.806 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-19T20:49:29.806 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-19T20:49:29.806 DEBUG:teuthology.orchestra.run.smithi132:> sudo systemctl stop ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.5 2024-02-19T20:49:30.140 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:29 smithi132 systemd[1]: Stopping Ceph osd.5 for a60b4d3a-cf57-11ee-95bb-87774f69a715... 2024-02-19T20:49:30.140 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:30 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:30.158+0000 7f9d04828700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-19T20:49:30.140 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:30 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:30.158+0000 7f9d04828700 -1 osd.5 88 *** Got signal Terminated *** 2024-02-19T20:49:30.140 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:30 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:30.158+0000 7f9d04828700 -1 osd.5 88 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-19T20:49:30.964 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:30 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:30.636+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:30.965 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:30 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:30.636+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6814 osd.1 since back 2024-02-19T20:49:03.149542+0000 front 2024-02-19T20:49:03.149484+0000 (oldest deadline 2024-02-19T20:49:27.849482+0000) 2024-02-19T20:49:31.964 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:31 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:31.671+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:31.965 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:31 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:31.671+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6814 osd.1 since back 2024-02-19T20:49:03.149542+0000 front 2024-02-19T20:49:03.149484+0000 (oldest deadline 2024-02-19T20:49:27.849482+0000) 2024-02-19T20:49:32.964 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:32 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:32.700+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:32.964 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:32 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:32.700+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6814 osd.1 since back 2024-02-19T20:49:03.149542+0000 front 2024-02-19T20:49:03.149484+0000 (oldest deadline 2024-02-19T20:49:27.849482+0000) 2024-02-19T20:49:33.964 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:33 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:33.675+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:33.964 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:33 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:33.675+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6814 osd.1 since back 2024-02-19T20:49:03.149542+0000 front 2024-02-19T20:49:03.149484+0000 (oldest deadline 2024-02-19T20:49:27.849482+0000) 2024-02-19T20:49:34.964 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:34 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:34.722+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6806 osd.0 since back 2024-02-19T20:48:52.548972+0000 front 2024-02-19T20:48:52.548999+0000 (oldest deadline 2024-02-19T20:49:18.448797+0000) 2024-02-19T20:49:34.964 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:34 smithi132 ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5[91576]: 2024-02-19T20:49:34.722+0000 7f9d0161d700 -1 osd.5 88 heartbeat_check: no reply from 172.21.15.88:6814 osd.1 since back 2024-02-19T20:49:03.149542+0000 front 2024-02-19T20:49:03.149484+0000 (oldest deadline 2024-02-19T20:49:27.849482+0000) 2024-02-19T20:49:35.898 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:35 smithi132 podman[125663]: 2024-02-19 20:49:35.633201418 +0000 UTC m=+5.584152521 container died be60ff50bffe95fb74c096cf466cd3c87397142c04c4f14fddb540728794666e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240214, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-02-19T20:49:36.214 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:35 smithi132 podman[125663]: 2024-02-19 20:49:35.999717279 +0000 UTC m=+5.950668373 container cleanup be60ff50bffe95fb74c096cf466cd3c87397142c04c4f14fddb540728794666e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240214) 2024-02-19T20:49:36.215 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:35 smithi132 bash[125663]: ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5 2024-02-19T20:49:36.589 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:36 smithi132 podman[125681]: 2024-02-19 20:49:36.284311363 +0000 UTC m=+0.652246461 container remove be60ff50bffe95fb74c096cf466cd3c87397142c04c4f14fddb540728794666e (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-19T20:49:36.935 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:36 smithi132 podman[125773]: 2024-02-19 20:49:36.511773952 +0000 UTC m=+0.017888127 image pull f081f847070acba44285717fbebd438c29e1171bb26f61956e59e7adb866ff0a quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526 2024-02-19T20:49:36.935 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:36 smithi132 podman[125773]: 2024-02-19 20:49:36.629017021 +0000 UTC m=+0.135131190 container create d1f751884746136664dcc64f4000c65245f1066d0bc647f46ccd9b2c81005326 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-deactivate, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240214, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-19T20:49:36.936 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:36 smithi132 podman[125773]: 2024-02-19 20:49:36.837503752 +0000 UTC m=+0.343617920 container init d1f751884746136664dcc64f4000c65245f1066d0bc647f46ccd9b2c81005326 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-deactivate, org.label-schema.build-date=20240214, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-02-19T20:49:36.936 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:36 smithi132 podman[125773]: 2024-02-19 20:49:36.840667459 +0000 UTC m=+0.346781635 container start d1f751884746136664dcc64f4000c65245f1066d0bc647f46ccd9b2c81005326 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-19T20:49:36.936 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:36 smithi132 podman[125773]: 2024-02-19 20:49:36.954114904 +0000 UTC m=+0.460229081 container attach d1f751884746136664dcc64f4000c65245f1066d0bc647f46ccd9b2c81005326 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-batrick-testing-20240215.160715-f78a58c, GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.schema-version=1.0) 2024-02-19T20:49:37.215 INFO:journalctl@ceph.osd.5.smithi132.stdout:Feb 19 20:49:37 smithi132 podman[125773]: 2024-02-19 20:49:37.124461275 +0000 UTC m=+0.630575437 container died d1f751884746136664dcc64f4000c65245f1066d0bc647f46ccd9b2c81005326 (image=quay.ceph.io/ceph-ci/ceph@sha256:20c75febd3cc6fee70b3b846b358905b187f9293f7c7c6707c7c369b69fd0526, name=ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=1617517b9622744995c4661989e3c30d036a7cfd, org.label-schema.build-date=20240214, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-batrick-testing-20240215.160715-f78a58c, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-02-19T20:49:37.762 DEBUG:teuthology.orchestra.run.smithi132:> sudo pkill -f 'journalctl -f -n 0 -u ceph-a60b4d3a-cf57-11ee-95bb-87774f69a715@osd.5.service' 2024-02-19T20:49:37.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-19T20:49:37.803 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-19T20:49:37.803 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 --force --keep-logs 2024-02-19T20:49:37.964 INFO:teuthology.orchestra.run.smithi088.stdout:Deleting cluster with fsid: a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T20:49:42.171 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi088.stderr:ceph-fuse[67419]: fuse finished with error 0 and tester_r 0 2024-02-19T20:49:59.455 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 --force --keep-logs 2024-02-19T20:49:59.616 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T20:50:14.013 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-19T20:50:14.043 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-19T20:50:14.073 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-19T20:50:14.075 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646/remote/smithi088/crash 2024-02-19T20:50:14.076 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash -- . 2024-02-19T20:50:14.109 INFO:teuthology.orchestra.run.smithi088.stderr:tar: /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash: Cannot open: No such file or directory 2024-02-19T20:50:14.109 INFO:teuthology.orchestra.run.smithi088.stderr:tar: Error is not recoverable: exiting now 2024-02-19T20:50:14.111 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash to /home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646/remote/smithi132/crash 2024-02-19T20:50:14.112 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash -- . 2024-02-19T20:50:14.139 INFO:teuthology.orchestra.run.smithi132.stderr:tar: /var/lib/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/crash: Cannot open: No such file or directory 2024-02-19T20:50:14.139 INFO:teuthology.orchestra.run.smithi132.stderr:tar: Error is not recoverable: exiting now 2024-02-19T20:50:14.140 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-19T20:50:14.141 DEBUG:teuthology.orchestra.run.smithi088:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2024-02-19T20:50:14.184 INFO:teuthology.orchestra.run.smithi088.stdout:2024-02-19T19:30:00.000181+0000 mon.smithi088 (mon.0) 853 : cluster 4 [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-02-19T20:50:14.184 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-19T20:50:14.184 INFO:tasks.cephadm:Compressing logs... 2024-02-19T20:50:14.184 DEBUG:teuthology.orchestra.run.smithi088:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-19T20:50:14.227 DEBUG:teuthology.orchestra.run.smithi132:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-19T20:50:14.251 INFO:teuthology.orchestra.run.smithi088.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-19T20:50:14.251 INFO:teuthology.orchestra.run.smithi088.stderr:: No such file or directory 2024-02-19T20:50:14.251 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-02-19T20:50:14.251 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mgr.smithi088.gyviqq.log 2024-02-19T20:50:14.251 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mon.smithi088.log 2024-02-19T20:50:14.253 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mgr.smithi088.gyviqq.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-client.ceph-exporter.smithi088.log 2024-02-19T20:50:14.254 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mon.smithi088.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.audit.log 2024-02-19T20:50:14.254 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-client.ceph-exporter.smithi088.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.1.log 2024-02-19T20:50:14.254 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi088.psjiah.log 2024-02-19T20:50:14.254 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.0.log 2024-02-19T20:50:14.255 INFO:teuthology.orchestra.run.smithi132.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-19T20:50:14.255 INFO:teuthology.orchestra.run.smithi132.stderr:: No such file or directory 2024-02-19T20:50:14.256 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-02-19T20:50:14.256 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mon.smithi132.log 2024-02-19T20:50:14.256 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.audit.log 2024-02-19T20:50:14.257 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mon.smithi132.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.4.log 2024-02-19T20:50:14.257 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-client.ceph-exporter.smithi132.log 2024-02-19T20:50:14.257 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.audit.log: /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.5.log 2024-02-19T20:50:14.257 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-client.ceph-exporter.smithi132.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi132.uyjxaz.log 2024-02-19T20:50:14.258 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.3.log 2024-02-19T20:50:14.258 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi132.uyjxaz.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-19T20:50:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi088.psjiah.log: 91.7% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.audit.log.gz 2024-02-19T20:50:14.261 INFO:teuthology.orchestra.run.smithi132.stderr: 91.8% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.audit.log.gz 2024-02-19T20:50:14.262 INFO:teuthology.orchestra.run.smithi088.stderr: 94.7% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-client.ceph-exporter.smithi088.log.gz 2024-02-19T20:50:14.263 INFO:teuthology.orchestra.run.smithi132.stderr: 94.1% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-client.ceph-exporter.smithi132.log.gz 2024-02-19T20:50:14.265 INFO:teuthology.orchestra.run.smithi088.stderr: 90.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-19T20:50:14.267 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.cephadm.log 2024-02-19T20:50:14.267 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.log 2024-02-19T20:50:14.267 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.2.log 2024-02-19T20:50:14.267 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.log: 86.0% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.cephadm.log.gz 2024-02-19T20:50:14.268 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mgr.smithi132.psllsa.log 2024-02-19T20:50:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.cephadm.log 2024-02-19T20:50:14.269 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mgr.smithi132.psllsa.log: gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.log 2024-02-19T20:50:14.270 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.cephadm.log: 85.9% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.cephadm.log.gz 2024-02-19T20:50:14.271 INFO:teuthology.orchestra.run.smithi132.stderr:gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-volume.log 2024-02-19T20:50:14.275 INFO:teuthology.orchestra.run.smithi088.stderr: 90.6% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.log.gz 2024-02-19T20:50:14.277 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-volume.log 2024-02-19T20:50:14.277 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.2.log: --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi088.ajgvpz.log 2024-02-19T20:50:14.280 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.log: 90.6%gzip -5 --verbose -- /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi132.afhgdi.log 2024-02-19T20:50:14.282 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-volume.log: 94.7% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi132.uyjxaz.log.gz 2024-02-19T20:50:14.282 INFO:teuthology.orchestra.run.smithi132.stderr: -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph.log.gz 2024-02-19T20:50:14.295 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-volume.log: /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi088.ajgvpz.log: 91.7% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-volume.log.gz 2024-02-19T20:50:14.297 INFO:teuthology.orchestra.run.smithi132.stderr:/var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi132.afhgdi.log: 91.5% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-volume.log.gz 2024-02-19T20:50:14.330 INFO:teuthology.orchestra.run.smithi132.stderr: 91.2% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mgr.smithi132.psllsa.log.gz 2024-02-19T20:50:15.334 INFO:teuthology.orchestra.run.smithi088.stderr: 90.1% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mgr.smithi088.gyviqq.log.gz 2024-02-19T20:50:15.817 INFO:teuthology.orchestra.run.smithi132.stderr: 92.7% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mon.smithi132.log.gz 2024-02-19T20:50:17.480 INFO:teuthology.orchestra.run.smithi088.stderr: 89.7% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mon.smithi088.log.gz 2024-02-19T20:50:20.966 INFO:teuthology.orchestra.run.smithi088.stderr: 93.3% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.2.log.gz 2024-02-19T20:50:21.246 INFO:teuthology.orchestra.run.smithi088.stderr: 94.9% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi088.ajgvpz.log.gz 2024-02-19T20:50:21.764 INFO:teuthology.orchestra.run.smithi088.stderr: 93.4% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.0.log.gz 2024-02-19T20:50:22.211 INFO:teuthology.orchestra.run.smithi132.stderr: 93.5% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.3.log.gz 2024-02-19T20:50:22.523 INFO:teuthology.orchestra.run.smithi132.stderr: 93.5% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.4.log.gz 2024-02-19T20:50:23.482 INFO:teuthology.orchestra.run.smithi088.stderr: 93.4% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.1.log.gz 2024-02-19T20:50:23.779 INFO:teuthology.orchestra.run.smithi132.stderr: 93.7% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-osd.5.log.gz 2024-02-19T20:50:28.694 INFO:teuthology.orchestra.run.smithi132.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-02-19T20:50:28.694 INFO:teuthology.orchestra.run.smithi132.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-02-19T20:50:31.639 INFO:teuthology.orchestra.run.smithi088.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-02-19T20:50:54.029 INFO:teuthology.orchestra.run.smithi132.stderr: 93.4% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi132.afhgdi.log.gz 2024-02-19T20:50:54.031 INFO:teuthology.orchestra.run.smithi132.stderr: 2024-02-19T20:50:54.031 INFO:teuthology.orchestra.run.smithi132.stderr:real 0m39.790s 2024-02-19T20:50:54.031 INFO:teuthology.orchestra.run.smithi132.stderr:user 1m19.035s 2024-02-19T20:50:54.031 INFO:teuthology.orchestra.run.smithi132.stderr:sys 0m2.457s 2024-02-19T20:51:09.125 INFO:teuthology.orchestra.run.smithi088.stderr: 93.5% -- replaced with /var/log/ceph/a60b4d3a-cf57-11ee-95bb-87774f69a715/ceph-mds.cephfs.smithi088.psjiah.log.gz 2024-02-19T20:51:09.126 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-02-19T20:51:09.126 INFO:teuthology.orchestra.run.smithi088.stderr:real 0m54.888s 2024-02-19T20:51:09.126 INFO:teuthology.orchestra.run.smithi088.stderr:user 1m43.324s 2024-02-19T20:51:09.126 INFO:teuthology.orchestra.run.smithi088.stderr:sys 0m3.298s 2024-02-19T20:51:09.127 INFO:tasks.cephadm:Archiving logs... 2024-02-19T20:51:09.127 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646/remote/smithi088/log 2024-02-19T20:51:09.128 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-19T20:51:23.420 DEBUG:teuthology.misc:Transferring archived files from smithi132:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646/remote/smithi132/log 2024-02-19T20:51:23.421 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-19T20:51:34.474 INFO:tasks.cephadm:Removing cluster... 2024-02-19T20:51:34.474 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 --force 2024-02-19T20:51:34.629 INFO:teuthology.orchestra.run.smithi088.stdout:Deleting cluster with fsid: a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T20:51:34.990 DEBUG:teuthology.orchestra.run.smithi132:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid a60b4d3a-cf57-11ee-95bb-87774f69a715 --force 2024-02-19T20:51:35.136 INFO:teuthology.orchestra.run.smithi132.stdout:Deleting cluster with fsid: a60b4d3a-cf57-11ee-95bb-87774f69a715 2024-02-19T20:51:35.480 INFO:tasks.cephadm:Removing cephadm ... 2024-02-19T20:51:35.481 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-19T20:51:35.496 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-19T20:51:35.511 INFO:tasks.cephadm:Teardown complete 2024-02-19T20:51:35.511 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-19T20:51:35.523 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-19T20:51:35.524 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-19T20:51:35.524 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-19T20:51:35.540 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-19T20:51:35.631 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-19T20:51:35.642 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-19T20:51:35.643 DEBUG:teuthology.orchestra.run.smithi088:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-19T20:51:35.646 DEBUG:teuthology.orchestra.run.smithi132:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-19T20:51:35.657 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-02-19T20:51:35.659 INFO:teuthology.orchestra.run.smithi132.stderr:bash: line 1: ntpq: command not found 2024-02-19T20:51:35.661 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-19T20:51:35.661 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-02-19T20:51:35.661 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 +246us[ +246us] +/- 96ms 2024-02-19T20:51:35.661 INFO:teuthology.orchestra.run.smithi088.stdout:^* hv02.front.sepia.ceph.com 3 7 377 40 -712us[ -730us] +/- 78ms 2024-02-19T20:51:35.662 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 +551us[ +551us] +/- 71ms 2024-02-19T20:51:35.662 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T20:51:35.663 INFO:teuthology.orchestra.run.smithi132.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-19T20:51:35.664 INFO:teuthology.orchestra.run.smithi132.stdout:=============================================================================== 2024-02-19T20:51:35.664 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 +292us[ +292us] +/- 96ms 2024-02-19T20:51:35.664 INFO:teuthology.orchestra.run.smithi132.stdout:^* hv02.front.sepia.ceph.com 3 9 377 86 -665us[ -697us] +/- 77ms 2024-02-19T20:51:35.664 INFO:teuthology.orchestra.run.smithi132.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 29 +580us[ +580us] +/- 71ms 2024-02-19T20:51:35.664 INFO:teuthology.orchestra.run.smithi132.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-02-19T20:51:35.664 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-19T20:51:35.676 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-19T20:51:35.677 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-19T20:51:35.688 DEBUG:teuthology.orchestra.run.smithi088:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-19T20:51:35.793 DEBUG:teuthology.orchestra.run.smithi132:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-19T20:51:35.877 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-19T20:51:35.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T20:51:35.909 DEBUG:teuthology.orchestra.run.smithi132:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-19T20:51:35.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T20:51:35.941 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-19T20:51:35.953 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-19T20:51:35.965 INFO:teuthology.task.internal:Duration was 7817.979625 seconds 2024-02-19T20:51:35.966 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-19T20:51:35.977 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-19T20:51:35.977 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-19T20:51:35.980 DEBUG:teuthology.orchestra.run.smithi132:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-19T20:51:36.020 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-19T20:51:36.024 INFO:teuthology.orchestra.run.smithi132.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-19T20:51:36.501 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-19T20:51:36.502 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi088.front.sepia.ceph.com 2024-02-19T20:51:36.502 DEBUG:teuthology.orchestra.run.smithi088:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-19T20:51:36.522 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi132.front.sepia.ceph.com 2024-02-19T20:51:36.522 DEBUG:teuthology.orchestra.run.smithi132:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-19T20:51:36.562 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-19T20:51:36.563 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-19T20:51:36.565 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-19T20:51:36.632 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-19T20:51:36.632 DEBUG:teuthology.orchestra.run.smithi088:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-19T20:51:36.634 DEBUG:teuthology.orchestra.run.smithi132:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-19T20:51:37.850 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-19T20:51:37.867 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-19T20:51:37.867 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-19T20:51:37.894 DEBUG:teuthology.orchestra.run.smithi132:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-19T20:51:37.926 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-19T20:51:37.938 DEBUG:teuthology.orchestra.run.smithi088:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-19T20:51:37.940 DEBUG:teuthology.orchestra.run.smithi132:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-19T20:51:37.963 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = core 2024-02-19T20:51:37.995 INFO:teuthology.orchestra.run.smithi132.stdout:kernel.core_pattern = core 2024-02-19T20:51:38.015 DEBUG:teuthology.orchestra.run.smithi088:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-19T20:51:38.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T20:51:38.065 DEBUG:teuthology.orchestra.run.smithi132:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-19T20:51:38.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T20:51:38.080 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-19T20:51:38.093 INFO:teuthology.task.internal:Transferring archived files... 2024-02-19T20:51:38.095 DEBUG:teuthology.misc:Transferring archived files from smithi088:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646/remote/smithi088 2024-02-19T20:51:38.095 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-19T20:51:38.171 DEBUG:teuthology.misc:Transferring archived files from smithi132:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646/remote/smithi132 2024-02-19T20:51:38.171 DEBUG:teuthology.orchestra.run.smithi132:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-19T20:51:38.216 INFO:teuthology.task.internal:Removing archive directory... 2024-02-19T20:51:38.217 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-19T20:51:38.219 DEBUG:teuthology.orchestra.run.smithi132:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-19T20:51:38.265 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-19T20:51:38.278 INFO:teuthology.task.internal:Not uploading archives. 2024-02-19T20:51:38.278 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-19T20:51:38.290 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-19T20:51:38.291 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-19T20:51:38.293 DEBUG:teuthology.orchestra.run.smithi132:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-19T20:51:38.305 INFO:teuthology.orchestra.run.smithi088.stdout: 265295 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 19 20:51 /home/ubuntu/cephtest 2024-02-19T20:51:38.305 INFO:teuthology.orchestra.run.smithi088.stdout: 550318 4 d--------- 2 ubuntu ubuntu 4096 Feb 19 18:59 /home/ubuntu/cephtest/mnt.0 2024-02-19T20:51:38.306 INFO:teuthology.orchestra.run.smithi088.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-02-19T20:51:38.306 INFO:teuthology.orchestra.run.smithi088.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-02-19T20:51:38.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-19T20:51:38.308 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/github.com_batrick_ceph_1ce7ea8877c709fa862eac608550374bd96e166e/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_b1dac5519c57c269ea98a22fb7729016a1d0e4d2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi088 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-02-19T20:51:38.309 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-19T20:51:38.322 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-19T20:51:38.336 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-02-19T20:51:38.336 INFO:teuthology.nuke:Checking targets against current locks 2024-02-19T20:51:38.364 DEBUG:teuthology.nuke:shortname: smithi088 2024-02-19T20:51:38.364 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-19T20:51:38.388 DEBUG:teuthology.nuke:shortname: smithi132 2024-02-19T20:51:38.388 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-19T20:51:38.391 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-19 18:29:59.661105', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPtDDHr2o50eWVnV9CPqYYq7STB65lDrHSrVT5wdXvjchvcjwFFmGArhdR3fCPD8QlRNAzqhFiiVenj7DG938Fw='} 2024-02-19T20:51:38.415 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi132.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-02-19_18:28:45-fs:upgrade:mds_upgrade_sequence-wip-batrick-testing-20240215.160715-distro-default-smithi/7566646', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-19 18:29:59.662526', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC/NtErFbvag6kQx7tNCcOaAIuzoCX3oIoOKi9mTgnQwZvPQiqBWoHr+67OR+URfwwUfJ8bM4hamLqcff/iNC9c='} 2024-02-19T20:51:38.442 INFO:teuthology.orchestra.console.smithi088:Power off 2024-02-19T20:51:38.442 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-19T20:51:38.467 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off 2024-02-19T20:51:38.467 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T20:51:38.488 INFO:teuthology.orchestra.console.smithi132:Power off 2024-02-19T20:51:38.489 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-19T20:51:38.499 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-02-19T20:51:38.513 DEBUG:teuthology.orchestra.console.smithi132:power off output: Chassis Power Control: Down/Off 2024-02-19T20:51:38.513 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T20:51:38.537 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-02-19T20:51:42.501 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T20:51:42.538 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T20:51:42.611 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-02-19T20:51:42.648 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is on 2024-02-19T20:51:46.616 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T20:51:46.649 DEBUG:teuthology.orchestra.console.smithi132:pexpect command: ipmitool -H smithi132.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-19T20:51:46.728 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-02-19T20:51:46.759 DEBUG:teuthology.orchestra.console.smithi132:check power output: Chassis Power is off 2024-02-19T20:51:46.829 INFO:teuthology.orchestra.console.smithi088:Power off completed 2024-02-19T20:51:46.860 INFO:teuthology.orchestra.console.smithi132:Power off completed 2024-02-19T20:51:46.981 INFO:teuthology.run:Summary data: description: fs:upgrade:mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 0a 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 7817.97962474823 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-02-19T20:51:46.981 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-19T20:51:47.076 INFO:teuthology.run:FAIL